Skip to content

Claude Code. Any Model. The most powerful AI coding agent now speaks every language.

feat(recovery): give the retry ladder a face — the magmux recovery pane

Jack Rudenkofce758eab2ca
diff --git a/ai-docs/reports/network-recovery-phase3-pane-20260911.md b/ai-docs/reports/network-recovery-phase3-pane-20260911.md
new file mode 100644
index 00000000..6e6afa96
--- /dev/null
+++ b/ai-docs/reports/network-recovery-phase3-pane-20260911.md
@@ -0,0 +1,328 @@
+# Network recovery, design-Phase 3 — the recovery pane, measured
+
+> Promoted out of the gitignored session directory deliberately, exactly as
+> `network-recovery-phase0-measurements.md` and `network-recovery-phase2-verification-20260911.md`
+> were. CLAUDE.md records three write-ups already lost because they lived in `ai-docs/sessions/`,
+> which does not survive a fresh clone or `git worktree remove`.
+>
+> Session: `dev-feature-network-recovery-20260910-110514-0ad0a5a0` · measured 2026-09-11 on
+> macOS 25.6.0, bun 1.4.0, `magmux 0.11.0 (8ce5ef8)`. Baseline commit `343e35e`.
+
+Phase 2 shipped the tier-1 retry ladder. Phase 3 gives it a face: a magmux pane claudish owns, a
+versioned NDJSON link between the two processes, and the **lease** that will decide, one phase from
+now, whether an exhausted episode may answer a retryable status. **Nothing here changes a status.**
+The lease is computed and logged only — shipping the 503 flip before the banner exists would
+recreate the buried-reason bug this feature exists to kill.
+
+Suite: **3271 → 3331 pass**, 18 → 18 skip, 2 → 2 fail (+60 tests, zero new failures). The two are the
+pre-existing `displayWidth` Unicode-oracle cases in `tui/viz/color.test.ts`, red on clean `main`
+under bun 1.4.0 while CI pins 1.3.10. `typecheck` clean. **`bun run lint` now exits 0, which it did
+not at `343e35e`.**
+
+One run in the middle of this work showed a third failure — `fallback-handler.test.ts`'s
+`kimi-k2.5 without prefix` at exactly 30 001 ms, a live-API call timing out under load. It passes
+alone and passed on the next full run; it is the documented contention-flake class, not a
+regression.
+
+---
+
+## 1. Two uncertainties, resolved before any production code
+
+### U-4 — `open_pane`'s `split` is a 50/50 split, not a strip
+
+One live `open_pane` against a headless magmux at `COLUMNS=120 LINES=40`, `list` before and after:
+
+```
+before: {"pane":0,"cols":120,"rows":40,"state":"running"} ← the whole terminal
+after: {"pane":0,"cols":120,"rows":20,"state":"running"} ← HALVED
+ {"pane":1,"control":true,"hidden":true,"state":"panel"}
+ {"pane":2,"cols":120,"rows":19,"state":"running","cmd":"/bin/zsh -l -c …"}
+```
+
+There is no strip option in the protocol. **The architecture's stated fallback is taken: accept
+50/50 for the outage's duration.** Claude Code handles SIGWINCH, and the pane is closed again when
+the last episode ends.
+
+Two further facts from the same capture, both load-bearing:
+
+1. **The opened pane is index 2, not 1.** magmux's control panel occupies index 1 — its own help
+ says "magmux's own panes are appended after them, whether or not `-c` was passed". The index is
+ read from the `open_pane` reply (`{"id":1,"ok":true,"result":{"pane":2}}`) and never assumed.
+2. **`cmd` really does run through the login shell** — `"/bin/zsh -l -c …"`, verbatim. RISK-5 is not
+ hypothetical, so the launcher script is required rather than defensive.
+
+### U-6 — magmux answers OSC 11, so theme detection works inside the pane
+
+A probe run AS a pane command, in a headless magmux with no terminal behind it at all:
+
+```
+stdin.isTTY=true stdout.isTTY=true
+TERM=screen-256color TERM_THEME=light MAGMUX_THEME=light COLORFGBG=15;0
+MAGMUX_SOCK=/tmp/magmux-osc11probe.sock
+RESULT: reply="\x1b]11;rgb:efef/f1f1/f5f5\x07" len=24
+```
+
+magmux synthesises the reply itself, so the bounded OSC 11 query in `theme-mode.ts` resolves inside
+a pane — and outranks the `COLORFGBG=15;0` it also inherits, which would have said *dark*: the exact
+lie `theming.md` demoted `COLORFGBG` for. Safe either way, because the banner's fill is deliberately
+theme-independent (self-contained mid-dark red + bright-white ink, the `team-grid.ts` precedent), but
+now known rather than assumed. `MAGMUX_SOCK` is confirmed exported to children.
+
+---
+
+## 2. The pane, live — C-3, C-4, C-5
+
+Everything below `installRecoveryUi` in the harness is production code: the real coordinator, the
+real UI manager, the real socket server, the real `claudish recovery-pane` process launched by
+`open_pane`, inside a real magmux. The harness supplies only the two ends — a genuinely refused
+endpoint and a stand-in for the Claude Code pane.
+
+**On capture.** magmux's control socket has no screenshot command — `snapshot`, `screenshot` and
+`capture` were each tried live and none returns pane text — and this session had no access to the
+terminal plugin's `mux` MCP tools. So magmux was given a real PTY and its byte stream replayed
+through a small VT grid (`phase3/vt.ts`), which is how a terminal produces a screen in the first
+place. What follows is that grid.
+
+### C-3 — the red banner, with provider, host and reason
+
+```
+CLAUDE CODE (stand-in) — this pane is the session
+────────────────────────────────────────────────────────────────────────────────────────────────────
+ ██ NETWORK · Live-probe refused
+ Cannot connect to Live-probe at http://127.0.0.1:53999/v1/messages. Make sure the server is
+ running.
+ host 127.0.0.1:53999 · ConnectionRefused · local
+ next attempt in 2s · attempt 1 · 3s in recovery
+ last: ConnectionRefused · 1 request held
+ click here or Ctrl-G Tab, then [r] try now · [q] give up
+```
+
+The sentence is `buildConnectionErrorMessage`'s own, verbatim — the same words the inline 400 would
+have carried. If those two ever diverge, the user is being told two different stories about one
+fault.
+
+### C-4 — the countdown is live
+
+Two captures four seconds apart, during the 30-second rung:
+
+```
+ next attempt in 28s · attempt 3 · 17s in recovery ← A
+ next attempt in 24s · attempt 3 · 21s in recovery ← B, +4s
+```
+
+### C-5 — `[r]` collapses the wait
+
+`focus` the recovery pane over the control socket, then one `r` into magmux's PTY:
+
+```
+[Recovery] attempt 4 (manual) for episode b6475bcd-…
+[Recovery] Live-probe attempt 4 failed: ConnectionRefused (episode b6475bcd-…, ladder 3)
+```
+
+Keypress → attempt: **1 ms**, against a scheduled wait that still had **23 s** to run. The banner
+then shows the next rung, `next attempt in 57s`, and a transient `retrying now…`.
+
+The full episode log for that run, which also shows the pane attaching and the lease being granted:
+
+```
+[Recovery] episode b6475bcd-… opened for Live-probe at http://127.0.0.1:53999/v1/messages (refused/ConnectionRefused)
+[Recovery] Live-probe attempt 1 failed: ConnectionRefused (…, ladder 0)
+[Recovery] Live-probe waiting 5s before attempt 2 (…, 1 waiting)
+[Recovery] socket listening at /tmp/claudish-recovery-27799446dda0a9a875cee1f0/r.sock
+[Recovery] recovery pane opened (magmux pane 2) at /tmp/claudish-recovery-…/r.sock
+[Recovery] pane connected (pid 68812, protocol 1)
+[Recovery] pane is painting episode b6475bcd-… — lease granted (paneOpen=true)
+[Recovery] Live-probe attempt 2 failed: ConnectionRefused (…, ladder 1)
+[Recovery] Live-probe waiting 10s before attempt 3 (…, 1 waiting)
+[Recovery] Live-probe attempt 3 failed: ConnectionRefused (…, ladder 2)
+[Recovery] Live-probe waiting 30s before attempt 4 (…, 1 waiting)
+[Recovery] attempt 4 (manual) for episode b6475bcd-…
+[Recovery] Live-probe attempt 4 failed: ConnectionRefused (…, ladder 3)
+[Recovery] Live-probe waiting 60s before attempt 5 (…, 1 waiting)
+[Recovery] client gone during wait (episode b6475bcd-…)
+[Recovery] episode b6475bcd-… closed: client_gone after 25412ms and 4 attempts
+```
+
+**The pane opens 4 ms after the episode does, and is painting 115 ms after that.** The socket path
+carries 96 bits of randomness rather than the proxy port, which is printed at startup.
+
+---
+
+## 3. C-7 — the healthy request path, against `343e35e`
+
+Measured where the request path lives: `ComposedHandler.handle()` against a REACHABLE upstream — a
+real `Bun.serve` speaking real SSE over a real socket — timed to the first byte of the response
+body. 40 samples each, after 10 warm-up calls. The `343e35e` column is a second worktree checked out
+at that commit, running the same benchmark file.
+
+| | median | p90 | min |
+|---|---|---|---|
+| `343e35e` (the phase-2 baseline) | **0.618 ms** | 5.047 ms | 0.367 ms |
+| phase 3, recovery UI not installed | **0.627 ms** | 1.835 ms | 0.347 ms |
+| phase 3, recovery UI installed | **0.448 ms** | 3.423 ms | 0.307 ms |
+
+Δ median vs the baseline: **+0.009 ms** and **−0.170 ms** — both inside the run-to-run spread. That
+is what the structure predicts: the only thing phase 3 adds to a request is one `uiLeaseValid()` call
+**inside the exhaustion arm of a catch block**, which a healthy request never enters.
+
+**What this does NOT measure: a real provider.** No local model server was running and the sandbox
+had no credentials, so the upstream is a stub. The stub exercises the whole handler — adapters,
+stream parser, response construction — and differs from a real provider only in network latency,
+which is common to both columns.
+
+## 4. C-16 / C-7's separate half — the magmux launch overhead
+
+Spawn to the child's first frame, wrapped versus direct, 7 samples each, with a stand-in for Claude
+Code so the number is the wrapper's cost and nothing else:
+
+| | median | p90 | min | max |
+|---|---|---|---|---|
+| direct (login shell only) | 39.4 ms | 55.4 ms | 33.8 ms | 55.4 ms |
+| wrapped in magmux | 136.2 ms | 179.4 ms | 131.4 ms | 179.4 ms |
+
+**Δ median: 96.8 ms.** The bar is 500 ms; above 1 s revisits default-on. Phase 0 measured +107 ms
+median on the same machine for the same thing, so this reproduces to within 10 ms.
+
+**magmux's own exit status does not carry the pane's** — measured: pane 0 exited 42 while magmux
+stayed alive waiting for a second pane. It does announce the death on the control socket
+(`{"type":"exit","pane":0,"exitCode":42}`), so that event is the authority for Claude Code's exit
+code, and it is also what lets the recovery pane be closed the instant Claude Code is gone rather
+than after its linger — which matters because `-w` waits for EVERY pane.
+
+---
+
+## 5. C-8 — the precedence matrix, one real process per row
+
+`flag > env > project > global > true`, nine rows, a fresh process each (the flag layer is module
+state and the config readers hit real files, so two rows in one process would test the leftovers of
+the first). `HOME` is redirected per row, so nothing reads or writes the user's own configuration.
+Each row reports both what the switch resolves to and what it causes — whether a magmux wrap plan is
+built — because a switch is only meaningful through its effect.
+
+| flag | env | project | global | `resolveRecoveryUi()` | magmux wrap built | expected |
+|---|---|---|---|---|---|---|
+| — | — | — | — | **true** | true | true |
+| — | — | — | `{"enabled":false}` | **false** | false | false |
+| — | — | — | `{"enabled":true}` | **true** | true | true |
+| — | — | `{}` | `{"enabled":false}` | **false** | false | false |
+| — | — | `{"enabled":false}` | `{"enabled":true}` | **false** | false | false |
+| — | `0` | `{"enabled":true}` | — | **false** | false | false |
+| — | `1` | — | `{"enabled":false}` | **true** | true | true |
+| `--recovery-ui` | `0` | `{"enabled":false}` | `{"enabled":false}` | **true** | true | true |
+| `--no-recovery-ui` | `1` | — | — | **false** | false | false |
+
+All nine pass. Row 4 is the one worth having: a project file carrying `"recoveryUi": {}` means **no
+opinion** and falls through to the global scope rather than reading as `true`. That is why the config
+field is an object rather than a bare boolean, and it is invisible to any test that only writes
+`true`/`false`.
+
+---
+
+## 6. The lease — the round-2 CRITICAL, and the test that can fail on it
+
+The rule: *a retryable status is permissible exactly while claudish still has a surface on which the
+reason is legible.* The superseded design renewed that lease from **frame receipts**, and frames
+ticked only while the ladder was `waiting`. A connect against an unreachable host takes 20–75 s
+(measured on this machine: `192.0.2.1` = 75 005 ms), during which the proxy has nothing new to say —
+so the lease died ten seconds into every slow attempt, and the exhaustion arm, which reads it
+immediately after an attempt returns, saw it false for the **entire failure class the feature was
+built for**, with a live painted banner on screen. Every test passed, because every fault in the
+suite was a ~1 ms loopback refusal.
+
+What ships instead:
+
+```
+uiLeaseValid(id) ⇔ paneOpen (WE opened it, and hold the open_pane reply)
+ ∧ some client heartbeated `ack` naming THIS episode within UI_LEASE_MS
+```
+
+- **granted by an `ack` sent AFTER PAINTING**, not on frame receipt;
+- **renewed by the renderer's own 1 Hz timer**, independent of frame arrival;
+- **revoked** by `bye`, by socket EOF, by a killed or frozen pane — all through one expiry path;
+- **`paneOpen` is required**, so a forged same-uid client cannot manufacture the forbidden state.
+
+Two tests exist specifically to fail on the old behaviour:
+
+1. *unit, fake clock* — 45 seconds of attempt, 4.5 lease windows, **not one frame emitted**; the
+ renderer heartbeats on its own timer and the lease holds throughout.
+2. *end to end, real socket, real renderer* — the server sends exactly ONE frame (the replay) and
+ then nothing, which is what a hung connect looks like on the wire. The real
+ `claudish recovery-pane` still emits ≥3 acks and the lease is valid with the lease clock 20 s past
+ that only frame. **Mutating the renderer back to ack-per-frame fails this test.**
+
+Also pinned: the lease expires exactly `UI_LEASE_MS` after the last heartbeat; it is per-episode, so
+heartbeating one grants nothing to another; switching the painted episode lets the old lease lapse
+with no extra protocol; and `[q]` closes the pane so that no later heartbeat can resurrect anything.
+
+---
+
+## 7. Mutation proof — 21 properties, 21 killed
+
+Each mutation reverts ONE property, runs the tests that claim to pin it, and restores the file **by
+copy — never with git**, because the stash stack and the index are shared with the main checkout and
+every sibling worktree.
+
+Killed: replay-on-connect · the tick in every live state · the frame version gate · `hello` parsing
+at the wrong version · the 0600 socket / 0700 directory · unlinking both on close · the countdown
+reading the live clock · **the palette read at RENDER time** · `attempting` showing the connect
+rather than a frozen countdown · the dead-proxy notice threshold · `q` sending `bye` · `r` sending
+`retry_now` · clipping to the pane width · **only the deltas reaching disk** · the launcher script's
+0600 mode · **exporting AFTER the profile** · stripping `CLAUDECODE` / `CLAUDE_CODE_CHILD_SESSION` ·
+the no-nesting gate · the interactive + TTY gate · `--id` / `--no-idle-done` / `-w` / `--no-status` ·
+`MAGMUX_SCROLLBACK` never overriding the user. Plus the four lease mutations above, run separately.
+
+One near-miss worth recording: `writeFileSync(..., { mode: 0o600 })` and the `chmodSync` that follows
+it are **both** present, and only the chmod is observable — `mode:` is subject to the umask, so
+mutating it alone leaves the final mode correct. `mode:` still earns its place (it closes the window
+between creation and chmod during which the file would be world-readable), but no assertion on a
+finished file can see a race, so it is documented rather than pinned.
+
+---
+
+## 8. Three things found while building, none of them in the plan
+
+### 8.1 A `team --grid` slot would have wrapped a second magmux inside its own pane
+
+`team --grid --mode interactive` launches one `claudish --model X -i` per pane. Each of those IS
+interactive with a real TTY, so the wrapper's stated gate (`interactive && stdout is a TTY && the
+binary resolves`) is satisfied in every slot — and each would have started its own magmux **inside
+the pane it was already running in**. N nested multiplexers, each with its own control socket and its
+own idea of the layout.
+
+magmux exports `MAGMUX_SOCK` to its children precisely so a child can tell. The wrapper now returns
+null when it is set, and `claude-runner` installs the recovery UI against that **ambient** socket
+instead. The cross-process `O_EXCL` lock then keeps the grid to ONE banner: the winner serves it, the
+losers still retry and still recover, hold no lease, and answer inline at exhaustion — which is
+exactly what the biconditional requires.
+
+### 8.2 The pane command must be built from `process.argv`, and the first attempt proved why
+
+`recoveryPaneCommandLine()` builds `<runtime> <script> recovery-pane --socket <path>` from
+`process.argv`, which is correct in production because `argv[1]` IS claudish's entry point. The first
+live run had `argv[1]` pointing at the harness, so the pane **re-launched the harness** — it printed
+"waiting for magmux…" into the banner's pane and left a directory of captures named `recovery-pane/`
+in the repo root. The failure mode is precisely the one the builder exists to avoid (a pane that is
+not this build), and it is loud rather than silent, which is the good news.
+
+### 8.3 A split pane is half the terminal, so every line is clipped to its width
+
+Following from U-4: the banner is rendered into a pane that may be 40 columns wide. Every line now
+goes through one clipping step that steps OVER the escapes — `String.length` counts an SGR sequence
+as eight or nine characters and would cut a line mid-escape, leaving the rest of the pane painted in
+whatever colour the fragment started. Below 64 columns the key hint sheds the part a user can work
+out (where to click) and keeps the part they cannot (which keys exist).
+
+---
+
+## 9. What is NOT verified, and why
+
+- **C-7 against a real provider.** No local model server was running and the sandbox has no
+ credentials; the measurement above uses a stub upstream through the real handler. The stub differs
+ from a provider only in network latency, which is common to both columns.
+- **C-16's Ctrl-C and exit-code propagation in a real terminal.** The exit-code path is implemented
+ from the measured `{"type":"exit","pane":0,"exitCode":42}` event and unit-tested at the plan level,
+ but driving a real Ctrl-C into a real attached terminal needs a TTY this session does not have.
+ The `[Claude Code] Exited…` structural log line is unchanged code on an unchanged path.
+- **C-6, C-12, C-14, C-15, C-17, C-18, C-19.** Not in this phase's scope. C-18's lease half is
+ covered by §6 — the lease is logged at exhaustion (`ui_lease=…`) rather than acted on, one phase
+ before it can cost anything.
diff --git a/packages/cli/src/claude-runner.ts b/packages/cli/src/claude-runner.ts
index 1bba166b..e6b9a36e 100644
--- a/packages/cli/src/claude-runner.ts
+++ b/packages/cli/src/claude-runner.ts
@@ -17,6 +17,7 @@ import { isatty } from "node:tty";
import { lookupModelForProvider } from "./adapters/model-catalog.js";
import { classifierPassthroughEnabled } from "./classifier-passthrough.js";
import { ENV } from "./config.js";
+import { planMagmuxWrap } from "./launcher/magmux-wrapper.js";
// Aliased: runClaudeWithProxy declares its own local `log` (a quiet-aware
// console printer), and an unaliased import would be shadowed inside it.
import { log as debugLog, logStderr } from "./logger.js";
@@ -25,6 +26,8 @@ import { discoverContextWindow } from "./providers/model-discovery.js";
import { parseModelSpec } from "./providers/model-parser.js";
import { getProviderByName } from "./providers/provider-definitions.js";
import { route } from "./providers/routing-rules.js";
+import { installRecoveryUi, shutdownRecoveryUi } from "./recovery/magmux-ui.js";
+import { resolveRecoveryUi } from "./recovery/settings.js";
import { setClaudeCodeRunning } from "./telemetry.js";
import { beginTerminalIsolation } from "./terminal-isolation.js";
import { getThemeMode } from "./theme/theme-mode.js";
@@ -1932,11 +1935,66 @@ export async function runClaudeWithProxy(
? ["inherit", "inherit", "pipe"]
: "inherit";
- const proc = spawn(spawnCommand, claudeArgs, {
- env,
- stdio,
- shell: needsShell,
- });
+ // Wrap the spawn in a magmux pane when — and ONLY when — the session is
+ // interactive, stdout is a real TTY, and the binary resolves. That is the
+ // whole gate: `-p`, `--stdin`, `serve`, the MCP server and the `team --grid`
+ // panes take the untouched path below, and a machine without magmux launches
+ // exactly as it did before this feature existed.
+ //
+ // The pane is what earns claudish the right to WAIT on a network fault
+ // instead of killing the turn: it is the only surface claudish owns during
+ // an interactive session, and a retryable status is only honest while the
+ // reason is legible somewhere.
+ //
+ // `resolveRecoveryUi()` is the user's half of the gate — flag > env > project
+ // > global > true. It decides whether claudish may own a SURFACE; it never
+ // decides whether a retry happens. Off means no pane, therefore no lease,
+ // therefore an inline error at exhaustion — which is the 503-vs-400 decision
+ // made structurally rather than by a second predicate somewhere else.
+ const wrap = resolveRecoveryUi()
+ ? planMagmuxWrap({
+ claudeBinary,
+ claudeArgs,
+ childEnv: env,
+ cwd: process.cwd(),
+ interactive: Boolean(config.interactive),
+ stdoutIsTty: Boolean(process.stdout.isTTY),
+ })
+ : null;
+
+ const proc = wrap
+ ? spawn(wrap.command, wrap.args, { env: wrap.env, stdio, shell: false })
+ : spawn(spawnCommand, claudeArgs, {
+ env,
+ stdio,
+ shell: needsShell,
+ });
+
+ if (wrap) {
+ wrap.watch(proc);
+ // Hand the control socket to the proxy as an EXPLICIT input. It cannot be
+ // discovered: claudish is magmux's parent and `MAGMUX_SOCK` is exported
+ // downward only, so the `--id` path — fixed before magmux starts — is the
+ // only way the parent can ever know where to send `open_pane`.
+ installRecoveryUi(wrap.controlSocket);
+ wrap.onClaudeExit(() => {
+ // Claude Code is gone, so nothing can consume a recovery any more. Take
+ // the pane down at once rather than after its linger: `-w` waits for
+ // EVERY pane, so a lingering banner would hold the whole session open
+ // after the user had already quit.
+ void shutdownRecoveryUi();
+ });
+ } else if (resolveRecoveryUi() && config.interactive && process.env.MAGMUX_SOCK) {
+ // Already inside someone else's magmux — `team --grid --mode interactive`,
+ // or a user who launched claudish in a pane by hand. There is nothing to
+ // wrap, but there IS a multiplexer to ask for a pane, so the recovery UI
+ // installs against the ambient control socket. In a grid this runs in N
+ // processes at once; the cross-process `O_EXCL` lock decides which one
+ // serves the banner, and the losers still retry and still recover — they
+ // simply hold no lease and answer inline at exhaustion, which is exactly
+ // what "a retryable status only while the reason is legible" requires.
+ installRecoveryUi(null);
+ }
if (filterChildStderr && proc.stderr) {
relayChildStderr(proc.stderr);
@@ -1988,12 +2046,25 @@ export async function runClaudeWithProxy(
}>((resolve) => {
proc.on("exit", (code, signal) => {
setClaudeCodeRunning(false);
+ // Under the wrapper the process that just exited is MAGMUX, and magmux's
+ // status does not carry the pane's — measured: pane 0 exited 42 while
+ // magmux stayed alive waiting for a second pane. The pane's own exit is
+ // announced on the control socket, and that announcement is the
+ // authority for Claude Code's exit code. Falling back to magmux's is
+ // right for the case where magmux itself died first.
+ const paneExit = wrap?.paneExitCode() ?? null;
resolve({
- exitCode: signal ? 128 + (SIGNAL_EXIT_NUMBERS[signal] ?? 0) : (code ?? 1),
+ exitCode:
+ paneExit !== null
+ ? paneExit
+ : signal
+ ? 128 + (SIGNAL_EXIT_NUMBERS[signal] ?? 0)
+ : (code ?? 1),
exitSignal: signal,
});
});
});
+ wrap?.cleanup();
// The only durable record of WHY the run ended. The always-on session log
// carries proxy traffic, so a session that died before its first request left
diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts
index 1b73a2bd..028cbe91 100644
--- a/packages/cli/src/cli.ts
+++ b/packages/cli/src/cli.ts
@@ -69,6 +69,7 @@ import {
loadRoutingRules,
matchRoutingRule,
} from "./providers/routing-rules.js";
+import { setRecoveryFlagOverrides } from "./recovery/settings.js";
import { cliAnsi } from "./theme/ansi.js";
import type { ClaudishConfig } from "./types.js";
import { VERSION } from "./version.js";
@@ -338,6 +339,19 @@ export async function parseArgs(args: string[]): Promise<ClaudishConfig> {
if (config.logLevel === "info") {
config.logLevel = "debug";
}
+ } else if (arg === "--recovery-ui" || arg === "--no-recovery-ui") {
+ // Whether claudish may own a surface — a magmux pane — on which a network
+ // outage's reason is legible. The flag layer of the precedence chain
+ // `flag > env > project > global > true` that `recovery/settings.ts`
+ // implements; the same shape `--debug-claudish` uses two branches up.
+ setRecoveryFlagOverrides({ recoveryUi: arg === "--recovery-ui" });
+ } else if (arg === "--recovery" || arg === "--no-recovery") {
+ // The master switch for the retry ladder ITSELF, independent of the UI.
+ // Off restores the pre-recovery behaviour everywhere, byte for byte, which
+ // is what makes it the documented CI switch: a scripted `-p` against a
+ // dead endpoint goes back to failing in milliseconds instead of holding
+ // the request for the whole tier-1 deadline.
+ setRecoveryFlagOverrides({ recovery: arg === "--recovery" });
} else if (arg === "--no-debug-claudish") {
// Escape hatch when debug is globally enabled (config.json / CLAUDISH_DEBUG):
// turn the debug file log off for this single run.
diff --git a/packages/cli/src/handlers/composed-handler.ts b/packages/cli/src/handlers/composed-handler.ts
index 4b3fb181..fc6d8bfa 100644
--- a/packages/cli/src/handlers/composed-handler.ts
+++ b/packages/cli/src/handlers/composed-handler.ts
@@ -39,6 +39,7 @@ import { deepMergeParams } from "../model-params.js";
import { describeSiblingKeys, getProviderByName } from "../providers/provider-definitions.js";
import { isTerminal429 } from "../providers/transport/openai.js";
import { recoveryClock } from "../recovery/clock.js";
+import { uiLeaseValid } from "../recovery/coordinator.js";
import { resolveRecoveryEnabled } from "../recovery/settings.js";
import {
type OpenAIImageBlock,
@@ -548,10 +549,21 @@ export class ComposedHandler implements ModelHandler {
// socket that is already gone.
return { kind: "respond", response: new Response(null, { status: 499 }) };
default: {
+ // THE LEASE IS READ HERE AND NOWHERE ELSE, at the instant the status is
+ // chosen — never cached off the result object, which would be stale by
+ // the time it was destructured.
+ //
+ // In THIS phase it only goes into the log. That is deliberate: shipping
+ // the 503 flip before the banner exists recreates the buried-reason bug
+ // this whole feature exists to kill. What the log buys now is the
+ // ability to prove, one phase early, that the lease is still valid at
+ // exhaustion after a 45-second attempt — the defect that made every
+ // loopback-only test pass while the feature did not work.
+ const leased = uiLeaseValid(result.episodeId);
log(
`[Recovery] ${this.provider.displayName} exhausted after ${result.attempts} attempts ` +
- `in ${result.recoveryMs}ms (episode ${result.episodeId}, outcome ${result.kind}) — ` +
- "answering connection_error"
+ `in ${result.recoveryMs}ms (episode ${result.episodeId}, outcome ${result.kind}, ` +
+ `ui_lease=${leased}) — answering connection_error`
);
return {
kind: "respond",
diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts
index 37c6317f..d702af4b 100644
--- a/packages/cli/src/index.ts
+++ b/packages/cli/src/index.ts
@@ -52,15 +52,25 @@ function classifyStartupKind(): string {
"quota",
"usage",
]);
- if ((first && management.has(first)) || argv.includes("--mcp") || first === "serve") {
+ if (
+ (first && management.has(first)) ||
+ argv.includes("--mcp") ||
+ first === "serve" ||
+ first === "recovery-pane"
+ ) {
return "other";
}
return "run";
}
process.on("exit", () => {
const argv = process.argv.slice(2);
+ const first = argv.find((a) => !a.startsWith("-"));
+ // `recovery-pane` joins the long-running exclusions for the same reason
+ // `serve` and `--mcp` are there: it lives for as long as the outage does, so
+ // an at-exit total is its lifetime rather than anyone's startup, and it would
+ // pollute the metrics with a number that measures a network failure.
const longRunningServer =
- argv.includes("--mcp") || argv.find((a) => !a.startsWith("-")) === "serve";
+ argv.includes("--mcp") || first === "serve" || first === "recovery-pane";
if (longRunningServer) return;
finalizeStartupTrace(classifyStartupKind(), { quiet: true });
});
@@ -318,6 +328,11 @@ const isStatsCommand = firstPositional === "stats";
const isConfigCommand = firstPositional === "config";
// Serve subcommand: claudish serve --port <n> --models <path> (Claude Desktop redirect gateway)
const isServeCommand = firstPositional === "serve";
+// Recovery pane: claudish recovery-pane --socket <path>. Launched BY claudish
+// into a magmux pane, never by a user — but it must be routed here all the
+// same, because an unrouted subcommand does not error: it falls through to the
+// default path and becomes a catalog search for the literal string.
+const isRecoveryPaneCommand = firstPositional === "recovery-pane";
// Providers subcommand: claudish providers --json (credential presence, no key material)
const isProvidersCommand = firstPositional === "providers";
// Keychain subcommand: claudish keychain status|list|import|set|rm|enable|disable
@@ -373,6 +388,17 @@ if (isMcpMode) {
process.exit(1);
})
);
+} else if (isRecoveryPaneCommand) {
+ // The network-recovery banner. Its own process, its own PTY, its own stdin —
+ // which is the entire reason the recovery UI is a pane: keys typed into it
+ // reach THIS process rather than the Claude Code session beside it.
+ const paneArgIndex = args.indexOf("recovery-pane");
+ import("./recovery/pane-app.js").then((m) =>
+ m.recoveryPaneCommand(args.slice(paneArgIndex + 1)).catch((e) => {
+ console.error(`[claudish recovery-pane] ${e instanceof Error ? e.message : String(e)}`);
+ process.exit(1);
+ })
+ );
} else if (isBehaviorCommand) {
// Layer 4 introspection: claudish behavior rules|corpus. Read-only unless
// --write is passed to `corpus`.
diff --git a/packages/cli/src/launcher/magmux-binary.ts b/packages/cli/src/launcher/magmux-binary.ts
new file mode 100644
index 00000000..5757784a
--- /dev/null
+++ b/packages/cli/src/launcher/magmux-binary.ts
@@ -0,0 +1,74 @@
+/**
+ * Where magmux lives, in the three shapes claudish ships in.
+ *
+ * Extracted from `team-grid.ts` rather than copied, because a second roster is
+ * a second thing to forget: `team --grid` and the recovery wrapper must agree
+ * about which binary is running, or a machine where one of them finds magmux
+ * and the other does not produces a feature that works in one surface and is
+ * silently absent from the other.
+ */
+
+import { execSync } from "node:child_process";
+import { existsSync } from "node:fs";
+import { dirname, join } from "node:path";
+import { fileURLToPath } from "node:url";
+
+/**
+ * Find the magmux binary, or null.
+ *
+ * Priority:
+ * 1. Bundled (`native/magmux-<platform>-<arch>`)
+ * 2. The platform-specific npm package (npm installs only the matching one)
+ * 3. `magmux` in PATH (e.g. Homebrew)
+ */
+export function findMagmuxBinaryOrNull(): string | null {
+ const thisFile = fileURLToPath(import.meta.url);
+ const thisDir = dirname(thisFile);
+ // `launcher/` is one level below `src/`, which is itself below the package
+ // root — so two levels up from here is where `native/` sits in a build.
+ const pkgRoots = [join(thisDir, ".."), join(thisDir, "..", "..")];
+ const platform = process.platform;
+ const arch = process.arch;
+
+ for (const pkgRoot of pkgRoots) {
+ const bundled = join(pkgRoot, "native", `magmux-${platform}-${arch}`);
+ if (existsSync(bundled)) return bundled;
+ }
+
+ try {
+ const pkgName = `@claudish/magmux-${platform}-${arch}`;
+ let searchDir = pkgRoots[0] as string;
+ for (let i = 0; i < 5; i++) {
+ const candidate = join(searchDir, "node_modules", pkgName, "bin", "magmux");
+ if (existsSync(candidate)) return candidate;
+ const parent = dirname(searchDir);
+ if (parent === searchDir) break;
+ searchDir = parent;
+ }
+ } catch {
+ /* not installed */
+ }
+
+ try {
+ const result = execSync("which magmux", { encoding: "utf-8" }).trim();
+ if (result) return result;
+ } catch {
+ /* not in PATH */
+ }
+
+ return null;
+}
+
+/**
+ * The same lookup, but fatal.
+ *
+ * `team --grid` cannot run at all without magmux, so it wants the throw. The
+ * recovery wrapper wants the null: a missing multiplexer there means the
+ * session launches exactly as it did before this feature existed, which is the
+ * correct degradation and not an error.
+ */
+export function findMagmuxBinary(): string {
+ const found = findMagmuxBinaryOrNull();
+ if (found) return found;
+ throw new Error("magmux not found. Install it:\n brew install MadAppGang/tap/magmux");
+}
diff --git a/packages/cli/src/launcher/magmux-wrapper.test.ts b/packages/cli/src/launcher/magmux-wrapper.test.ts
new file mode 100644
index 00000000..a7e05d3b
--- /dev/null
+++ b/packages/cli/src/launcher/magmux-wrapper.test.ts
@@ -0,0 +1,240 @@
+/**
+ * The launch wrapper.
+ *
+ * Two of these assertions are about SECURITY (only deltas reach the disk, and
+ * the file is 0600) and two are about a contract that fails silently if it
+ * breaks (the exports come after the login shell, and the strip list is
+ * applied). The rest pin the gate: an implementation that wrapped `-p`, a
+ * non-TTY, or a machine without magmux would break surfaces that have nothing
+ * to do with this feature.
+ */
+
+import { afterEach, describe, expect, test } from "bun:test";
+import { mkdtempSync, readFileSync, rmSync, statSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import {
+ STRIPPED_CHILD_VARS,
+ buildLauncherScript,
+ envDeltas,
+ planMagmuxWrap,
+} from "./magmux-wrapper.js";
+
+const roots: string[] = [];
+function scratch(): string {
+ const d = mkdtempSync(join(tmpdir(), "claudish-wrap-test-"));
+ roots.push(d);
+ return d;
+}
+
+afterEach(() => {
+ for (const d of roots.splice(0)) rmSync(d, { recursive: true, force: true });
+});
+
+const BASE = {
+ claudeBinary: "/usr/local/bin/claude",
+ claudeArgs: ["--dangerously-skip-permissions"],
+ interactive: true,
+ stdoutIsTty: true,
+ magmuxBinary: "/opt/homebrew/bin/magmux",
+ pid: 4242,
+};
+
+describe("envDeltas", () => {
+ test("carries ONLY what claudish changed — never the inherited environment", () => {
+ // The difference between a delta and a dump is whether the user's exported
+ // secrets end up in a file on disk.
+ const parent = { PATH: "/usr/bin", HOME: "/Users/jack", OPENAI_API_KEY: "sk-secret" };
+ const child = { ...parent, ANTHROPIC_BASE_URL: "http://127.0.0.1:8787" };
+ const { set, unset } = envDeltas(parent, child);
+ expect(set).toEqual([["ANTHROPIC_BASE_URL", "http://127.0.0.1:8787"]]);
+ expect(set.map(([k]) => k)).not.toContain("OPENAI_API_KEY");
+ expect(set.map(([k]) => k)).not.toContain("PATH");
+ expect(unset).toEqual([...STRIPPED_CHILD_VARS]);
+ });
+
+ test("a CHANGED value is a delta; an identical one is not", () => {
+ const parent = { A: "1", B: "2" };
+ const child = { A: "1", B: "changed" };
+ expect(envDeltas(parent, child).set).toEqual([["B", "changed"]]);
+ });
+
+ test("a variable claudish deleted is unset, not silently inherited", () => {
+ // Monitor mode deletes ANTHROPIC_API_KEY so Claude Code uses its own
+ // credentials. If the login shell re-inherited it, the session would bill
+ // the API instead of the subscription.
+ const parent = { ANTHROPIC_API_KEY: "sk-real" };
+ const child = {};
+ expect(envDeltas(parent, child).unset).toContain("ANTHROPIC_API_KEY");
+ });
+
+ test("CLAUDECODE and CLAUDE_CODE_CHILD_SESSION are always unset, never exported", () => {
+ // Inherited, the pair turns transcript saving off and degrades magmux's own
+ // controller to terminal-idle heuristics. `claude-runner.ts` spreads
+ // `...process.env` and strips only the first.
+ const parent = { CLAUDECODE: "1", CLAUDE_CODE_CHILD_SESSION: "1" };
+ const child = { CLAUDECODE: "1", CLAUDE_CODE_CHILD_SESSION: "1", X: "y" };
+ const { set, unset } = envDeltas(parent, child);
+ expect(set.map(([k]) => k)).not.toContain("CLAUDECODE");
+ expect(set.map(([k]) => k)).not.toContain("CLAUDE_CODE_CHILD_SESSION");
+ expect(unset).toContain("CLAUDECODE");
+ expect(unset).toContain("CLAUDE_CODE_CHILD_SESSION");
+ });
+});
+
+describe("buildLauncherScript", () => {
+ test("exports come AFTER the profile and immediately before exec — that is the point", () => {
+ // RISK-5, measured rather than assumed: a live `list` reply shows magmux
+ // runs a pane command as `/bin/zsh -l -c …`. A user's `.zprofile` therefore
+ // runs between claudish and Claude Code, and claudish's whole contract with
+ // Claude Code — starting with ANTHROPIC_BASE_URL — travels in the
+ // environment. Re-asserting after the profile is what survives it.
+ const script = buildLauncherScript("/usr/local/bin/claude", ["-i"], {
+ set: [["ANTHROPIC_BASE_URL", "http://127.0.0.1:8787"]],
+ unset: ["CLAUDECODE"],
+ });
+ const lines = script.trim().split("\n");
+ const exportAt = lines.findIndex((l) => l.startsWith("export ANTHROPIC_BASE_URL="));
+ const execAt = lines.findIndex((l) => l.startsWith("exec "));
+ expect(exportAt).toBeGreaterThan(-1);
+ expect(execAt).toBe(lines.length - 1);
+ expect(exportAt).toBeLessThan(execAt);
+ });
+
+ test("values and arguments are single-quoted, so a space or a quote cannot escape", () => {
+ const script = buildLauncherScript("/path with space/claude", ["it's", "--flag"], {
+ set: [["WEIRD", "a'b c"]],
+ unset: [],
+ });
+ expect(script).toContain(`export WEIRD='a'\\''b c'`);
+ expect(script).toContain(`exec '/path with space/claude' 'it'\\''s' '--flag'`);
+ });
+});
+
+describe("planMagmuxWrap", () => {
+ test("returns null for a non-interactive run — `-p` and `--stdin` are untouched", () => {
+ expect(
+ planMagmuxWrap({ ...BASE, interactive: false, childEnv: {}, tmpRoot: scratch() })
+ ).toBeNull();
+ });
+
+ test("returns null when stdout is not a TTY", () => {
+ expect(
+ planMagmuxWrap({ ...BASE, stdoutIsTty: false, childEnv: {}, tmpRoot: scratch() })
+ ).toBeNull();
+ });
+
+ test("returns null INSIDE an existing magmux — a grid pane must not nest one", () => {
+ // `team --grid --mode interactive` runs one `claudish -i` per pane, each
+ // interactive with a real TTY. Without this, every slot would wrap itself
+ // in a second magmux inside the pane it is already running in. magmux
+ // exports MAGMUX_SOCK to its children precisely so a child can tell.
+ expect(
+ planMagmuxWrap({
+ ...BASE,
+ parentEnv: { MAGMUX_SOCK: "/tmp/magmux-someone-else.sock" },
+ childEnv: {},
+ tmpRoot: scratch(),
+ })
+ ).toBeNull();
+ });
+
+ test("returns null when magmux is missing — the session launches exactly as before", () => {
+ // The correct degradation, and it needs no version check: without a
+ // multiplexer nothing can open a pane, so no lease is ever granted and the
+ // feature is simply absent.
+ expect(
+ planMagmuxWrap({ ...BASE, magmuxBinary: null, childEnv: {}, tmpRoot: scratch() })
+ ).toBeNull();
+ });
+
+ test("the control socket path is DERIVED from --id, so it is known before magmux starts", () => {
+ // claudish is magmux's parent and `MAGMUX_SOCK` is exported downward only,
+ // so a pid-derived socket could never be discovered by the parent. `--id`
+ // is what makes the recovery UI reachable at all.
+ const plan = planMagmuxWrap({ ...BASE, childEnv: {}, tmpRoot: scratch() });
+ expect(plan).not.toBeNull();
+ expect(plan?.controlSocket).toBe("/tmp/magmux-claudish-4242.sock");
+ expect(plan?.args).toContain("--id");
+ expect(plan?.args[plan.args.indexOf("--id") + 1]).toBe("claudish-4242");
+ });
+
+ test("passes --no-idle-done, -w and --no-status", () => {
+ const plan = planMagmuxWrap({ ...BASE, childEnv: {}, tmpRoot: scratch() });
+ // `--no-idle-done` is the fix for the `-w` trap: an idle pane is a resting
+ // session, not a finished one, and with the flag `-w` waits for the process
+ // to exit rather than for a turn to end.
+ expect(plan?.args).toContain("--no-idle-done");
+ expect(plan?.args).toContain("-w");
+ expect(plan?.args).toContain("--no-status");
+ });
+
+ test("the pane command SOURCES the generated script", () => {
+ const plan = planMagmuxWrap({ ...BASE, childEnv: {}, tmpRoot: scratch() });
+ const cmd = plan?.args[plan.args.length - 1] as string;
+ expect(cmd.startsWith(". '")).toBe(true);
+ expect(cmd).toContain(plan?.scriptPath as string);
+ });
+
+ test("the script is 0600 inside a 0700 directory and carries only the deltas", () => {
+ const root = scratch();
+ const plan = planMagmuxWrap({
+ ...BASE,
+ parentEnv: { PATH: "/usr/bin", OPENAI_API_KEY: "sk-inherited-secret" },
+ childEnv: {
+ PATH: "/usr/bin",
+ OPENAI_API_KEY: "sk-inherited-secret",
+ ANTHROPIC_BASE_URL: "http://127.0.0.1:8787",
+ },
+ tmpRoot: root,
+ });
+ const script = readFileSync(plan?.scriptPath as string, "utf-8");
+ expect(script).toContain("export ANTHROPIC_BASE_URL='http://127.0.0.1:8787'");
+ // The inherited secret is NOT written to disk. This is the assertion that
+ // makes "only the deltas" a property rather than a comment.
+ expect(script).not.toContain("sk-inherited-secret");
+ expect(script).not.toContain("PATH=");
+ expect(statSync(plan?.scriptPath as string).mode & 0o777).toBe(0o600);
+ expect(statSync(join(root, "claudish-launch-4242")).mode & 0o777).toBe(0o700);
+ });
+
+ test("MAGMUX_SCROLLBACK defaults to 10000 but never overrides the user's own", () => {
+ // RISK-4: the pane replaces the emulator's native scrollback with magmux's
+ // ring. A bigger ring does not give selection back, but it stops the
+ // session's history from being the thing that is lost.
+ const a = planMagmuxWrap({ ...BASE, childEnv: {}, tmpRoot: scratch() });
+ expect(a?.env.MAGMUX_SCROLLBACK).toBe("10000");
+ const b = planMagmuxWrap({
+ ...BASE,
+ childEnv: { MAGMUX_SCROLLBACK: "500" },
+ tmpRoot: scratch(),
+ });
+ expect(b?.env.MAGMUX_SCROLLBACK).toBe("500");
+ });
+
+ test("the magmux process itself never inherits the stripped variables", () => {
+ const plan = planMagmuxWrap({
+ ...BASE,
+ childEnv: { CLAUDECODE: "1", CLAUDE_CODE_CHILD_SESSION: "1", KEEP: "yes" },
+ tmpRoot: scratch(),
+ });
+ expect(plan?.env.CLAUDECODE).toBeUndefined();
+ expect(plan?.env.CLAUDE_CODE_CHILD_SESSION).toBeUndefined();
+ expect(plan?.env.KEEP).toBe("yes");
+ });
+
+ test("cleanup removes the script and its directory", () => {
+ const root = scratch();
+ const plan = planMagmuxWrap({ ...BASE, childEnv: {}, tmpRoot: root });
+ expect(statSync(plan?.scriptPath as string).isFile()).toBe(true);
+ plan?.cleanup();
+ expect(() => statSync(plan?.scriptPath as string)).toThrow();
+ });
+
+ test("the pane exit code starts unknown — magmux's own status is not the child's", () => {
+ // Measured: pane 0 exited 42 while magmux stayed alive waiting for a second
+ // pane, so the exit code has to come off the control socket's `exit` event.
+ const plan = planMagmuxWrap({ ...BASE, childEnv: {}, tmpRoot: scratch() });
+ expect(plan?.paneExitCode()).toBeNull();
+ });
+});
diff --git a/packages/cli/src/launcher/magmux-wrapper.ts b/packages/cli/src/launcher/magmux-wrapper.ts
new file mode 100644
index 00000000..a1c6da55
--- /dev/null
+++ b/packages/cli/src/launcher/magmux-wrapper.ts
@@ -0,0 +1,302 @@
+/**
+ * Launch Claude Code inside a magmux pane, so claudish owns a surface it can
+ * draw a recovery banner on.
+ *
+ * There is no other surface. The status line repaints on Claude Code's
+ * schedule and has no input channel; stderr is firewalled precisely because a
+ * stray write corrupted the TUI; an OSC-9 notification cannot render a
+ * countdown; a second terminal window is platform-specific and hostile; and
+ * allocating our own PTY is writing a multiplexer. A pane is the cheapest thing
+ * claudish can own.
+ *
+ * APPLIES ONLY WHEN `interactive === true` AND stdout is a TTY AND the binary
+ * resolves. `-p`, `--stdin`, `serve`, the MCP server and the `team --grid`
+ * panes are untouched, by construction rather than by a flag.
+ *
+ * ── RISK-5, and why the launcher script exists ──────────────────────────────
+ * magmux runs a pane command through the LOGIN SHELL. Measured, verbatim from
+ * a live `list` reply: `"cmd": "/bin/zsh -l -c bash -c '…'"`. So a user's
+ * `.zprofile` runs between claudish and Claude Code, and claudish's entire
+ * contract with Claude Code travels in the environment — `ANTHROPIC_BASE_URL`
+ * is the whole proxy. A profile that sets any of those variables would silently
+ * take the session off the proxy.
+ *
+ * The mitigation is a generated script that re-exports ONLY claudish's own
+ * DELTAS immediately before `exec`. Only the deltas: writing the full
+ * environment to disk would put every secret the user has exported into a file.
+ *
+ * ── C-16, and why the exit code comes off the socket ────────────────────────
+ * magmux's own exit status does not carry the pane's — measured: pane 0 exited
+ * 42 while magmux stayed alive waiting for a second pane. It does announce the
+ * death on the control socket (`{"type":"exit","pane":0,"exitCode":42}`), so
+ * that event is the authority for the child's exit code, and it is also what
+ * lets the recovery pane be closed the instant Claude Code is gone rather than
+ * after its linger.
+ */
+
+import type { ChildProcess } from "node:child_process";
+import { chmodSync, existsSync, mkdirSync, rmSync, writeFileSync } from "node:fs";
+import { type Socket, connect as netConnect } from "node:net";
+import { join } from "node:path";
+import { findMagmuxBinaryOrNull } from "./magmux-binary.js";
+
+/**
+ * Variables that must NOT reach the child.
+ *
+ * `claude-runner.ts` spreads `...process.env` and deletes `CLAUDECODE` but not
+ * `CLAUDE_CODE_CHILD_SESSION`; inherited, the pair turns transcript saving off
+ * and degrades magmux's own controller to terminal-idle heuristics. Stripping
+ * them here covers both the magmux process and the login shell, which is a
+ * second place they could have re-entered from.
+ */
+export const STRIPPED_CHILD_VARS = ["CLAUDECODE", "CLAUDE_CODE_CHILD_SESSION"] as const;
+
+export interface MagmuxWrapInput {
+ claudeBinary: string;
+ claudeArgs: string[];
+ /** The environment `claude-runner` built for Claude Code. */
+ childEnv: Record<string, string>;
+ /** What the login shell will start from. Defaults to `process.env`. */
+ parentEnv?: NodeJS.ProcessEnv;
+ cwd?: string;
+ interactive: boolean;
+ stdoutIsTty: boolean;
+ /** Overridable so a test does not need magmux installed. */
+ magmuxBinary?: string | null;
+ /** Overridable so a test does not write into the real temp directory. */
+ tmpRoot?: string;
+ pid?: number;
+}
+
+export interface MagmuxWrapPlan {
+ command: string;
+ args: string[];
+ env: Record<string, string>;
+ /** `/tmp/magmux-claudish-<pid>.sock` — known BEFORE magmux starts. */
+ controlSocket: string;
+ scriptPath: string;
+ /** Start watching the control socket. Call once, after spawning. */
+ watch(proc: ChildProcess): void;
+ /** Claude Code's own exit code, once magmux has announced it. */
+ paneExitCode(): number | null;
+ /** Called when pane 0 dies, before magmux itself exits. */
+ onClaudeExit(fn: (exitCode: number | null) => void): void;
+ cleanup(): void;
+}
+
+/**
+ * What claudish added to, changed in, or removed from the environment.
+ *
+ * The delta — not the whole environment — is what the launcher script carries,
+ * and the difference is whether a user's API keys end up in a file on disk.
+ */
+export function envDeltas(
+ parentEnv: NodeJS.ProcessEnv,
+ childEnv: Record<string, string>
+): { set: Array<[string, string]>; unset: string[] } {
+ const set: Array<[string, string]> = [];
+ const unset: string[] = [];
+ for (const [k, v] of Object.entries(childEnv)) {
+ if (STRIPPED_CHILD_VARS.includes(k as (typeof STRIPPED_CHILD_VARS)[number])) continue;
+ if (parentEnv[k] !== v) set.push([k, v]);
+ }
+ for (const k of Object.keys(parentEnv)) {
+ if (!(k in childEnv)) unset.push(k);
+ }
+ for (const k of STRIPPED_CHILD_VARS) {
+ if (!unset.includes(k)) unset.push(k);
+ }
+ return { set, unset };
+}
+
+/** POSIX single-quoting. The only escaping a `'…'` string ever needs. */
+function shq(s: string): string {
+ return `'${s.replace(/'/g, "'\\''")}'`;
+}
+
+/**
+ * The script magmux's login shell sources, then `exec`s.
+ *
+ * Pure, so the exact bytes that will land on disk can be asserted in a test —
+ * which matters because the thing being asserted is a security property (only
+ * deltas) and a correctness property (the exports come AFTER the profile).
+ */
+export function buildLauncherScript(
+ claudeBinary: string,
+ claudeArgs: string[],
+ deltas: { set: Array<[string, string]>; unset: string[] }
+): string {
+ const lines = [
+ "#!/bin/sh",
+ "# Generated by claudish. Re-asserts claudish's OWN environment deltas after",
+ "# the login shell has run the user's profile, then hands the terminal to",
+ "# Claude Code. Only deltas are written here: the full environment would put",
+ "# every exported secret into a file.",
+ ];
+ for (const k of deltas.unset) lines.push(`unset ${k}`);
+ for (const [k, v] of deltas.set) lines.push(`export ${k}=${shq(v)}`);
+ lines.push(`exec ${shq(claudeBinary)} ${claudeArgs.map(shq).join(" ")}`);
+ return `${lines.join("\n")}\n`;
+}
+
+/**
+ * Build the plan, or null when this launch must not be wrapped.
+ *
+ * Returns null — never throws — for every "not applicable" case, because a
+ * missing multiplexer means the session should launch exactly as it did before
+ * this feature existed.
+ */
+export function planMagmuxWrap(input: MagmuxWrapInput): MagmuxWrapPlan | null {
+ if (!input.interactive || !input.stdoutIsTty) return null;
+ // ALREADY INSIDE A MULTIPLEXER — do not nest one.
+ //
+ // `team --grid --mode interactive` launches one `claudish --model X -i` per
+ // pane, and each of those is interactive with a real TTY, so without this it
+ // would wrap ITSELF in a second magmux inside the grid pane it is already
+ // running in — N nested multiplexers, each with its own control socket and
+ // its own idea of the layout. magmux exports `MAGMUX_SOCK` to its children
+ // precisely so a child can tell, and a process that finds it does not need a
+ // new magmux: it already has one it can ask for a pane. The recovery UI is
+ // installed against that AMBIENT socket instead, and the cross-process
+ // `O_EXCL` lock in `magmux-ui.ts` is what keeps the grid to ONE banner rather
+ // than N.
+ if ((input.parentEnv ?? process.env).MAGMUX_SOCK) return null;
+ const magmux = input.magmuxBinary === undefined ? findMagmuxBinaryOrNull() : input.magmuxBinary;
+ if (!magmux) return null;
+
+ const pid = input.pid ?? process.pid;
+ const id = `claudish-${pid}`;
+ const controlSocket = `/tmp/magmux-${id}.sock`;
+
+ // 0700 directory, 0600 file. The script carries claudish's deltas, which on
+ // the proxy path include the key Claude Code authenticates to us with.
+ const tmpRoot = input.tmpRoot ?? "/tmp";
+ const scriptDir = join(tmpRoot, `claudish-launch-${pid}`);
+ mkdirSync(scriptDir, { mode: 0o700, recursive: true });
+ // Named for Claude Code on purpose: magmux attaches its controller by
+ // spotting `claude`/`claudish` in the pane's command.
+ const scriptPath = join(scriptDir, "claude-launch.sh");
+
+ const parentEnv = input.parentEnv ?? process.env;
+ const deltas = envDeltas(parentEnv, input.childEnv);
+ writeFileSync(scriptPath, buildLauncherScript(input.claudeBinary, input.claudeArgs, deltas), {
+ mode: 0o600,
+ });
+ try {
+ chmodSync(scriptPath, 0o600);
+ } catch {
+ /* the 0700 directory is the real gate */
+ }
+
+ const env: Record<string, string> = { ...input.childEnv };
+ for (const k of STRIPPED_CHILD_VARS) delete env[k];
+ // RISK-4: the pane replaces the emulator's native scrollback with magmux's
+ // ring, which is the single most user-visible cost of wrapping by default.
+ // A bigger ring does not give the emulator's selection back, but it does stop
+ // the session's history from being the thing that is lost.
+ if (!env.MAGMUX_SCROLLBACK) env.MAGMUX_SCROLLBACK = "10000";
+
+ const args = [
+ "--id",
+ id,
+ // An idle pane is a resting session, not a finished one — and with this
+ // flag `-w` waits for the process to exit rather than for a turn to end,
+ // which is the exact trap `headless-vs-interactive.md` records.
+ "--no-idle-done",
+ "-w",
+ "--no-status",
+ "-e",
+ // Sourced, not executed: `exec` then replaces the login shell itself, so
+ // the pane's process IS Claude Code rather than a shell holding it.
+ `. ${shq(scriptPath)}`,
+ ];
+
+ let paneExit: number | null = null;
+ const exitListeners: Array<(code: number | null) => void> = [];
+ let control: Socket | null = null;
+
+ const cleanup = () => {
+ try {
+ control?.destroy();
+ } catch {
+ /* already gone */
+ }
+ control = null;
+ try {
+ rmSync(scriptDir, { recursive: true, force: true });
+ } catch {
+ /* already gone */
+ }
+ try {
+ // magmux's startup reaper only removes PID-named sockets whose owner it
+ // can prove is dead, so a `--id` socket is never reaped for us.
+ if (existsSync(controlSocket)) rmSync(controlSocket, { force: true });
+ } catch {
+ /* already gone */
+ }
+ };
+
+ return {
+ command: magmux,
+ args,
+ env,
+ controlSocket,
+ scriptPath,
+ paneExitCode: () => paneExit,
+ onClaudeExit(fn) {
+ exitListeners.push(fn);
+ },
+ watch(proc: ChildProcess) {
+ let stopped = false;
+ const attempt = async () => {
+ for (let i = 0; i < 60 && !stopped; i++) {
+ if (existsSync(controlSocket)) {
+ try {
+ const s = await new Promise<Socket>((resolve, reject) => {
+ const sock = netConnect(controlSocket);
+ sock.once("connect", () => resolve(sock));
+ sock.once("error", reject);
+ });
+ control = s;
+ let buf = "";
+ s.on("data", (chunk: Buffer) => {
+ buf += chunk.toString("utf-8");
+ let nl = buf.indexOf("\n");
+ while (nl >= 0) {
+ const line = buf.slice(0, nl).trim();
+ buf = buf.slice(nl + 1);
+ nl = buf.indexOf("\n");
+ if (!line) continue;
+ try {
+ const evt = JSON.parse(line) as Record<string, unknown>;
+ // Pane 0 is Claude Code: `-e` panes get ids 0..N-1 in
+ // argument order and magmux's own panes are appended after.
+ if (evt.type === "exit" && evt.pane === 0) {
+ paneExit = typeof evt.exitCode === "number" ? evt.exitCode : null;
+ for (const fn of exitListeners) fn(paneExit);
+ }
+ } catch {
+ /* not ours */
+ }
+ }
+ });
+ s.on("error", () => {
+ control = null;
+ });
+ return;
+ } catch {
+ /* magmux binds the socket a moment after the file appears */
+ }
+ }
+ await new Promise((r) => setTimeout(r, 50));
+ }
+ };
+ proc.once("exit", () => {
+ stopped = true;
+ cleanup();
+ });
+ void attempt();
+ },
+ cleanup,
+ };
+}
diff --git a/packages/cli/src/recovery/coordinator.ts b/packages/cli/src/recovery/coordinator.ts
index 5f384bab..1bec8921 100644
--- a/packages/cli/src/recovery/coordinator.ts
+++ b/packages/cli/src/recovery/coordinator.ts
@@ -19,17 +19,23 @@
* A waiter past its own deadline answers for itself and leaves.
* - the LADDER belongs to the episode. It ends when the last waiter leaves
* or an attempt succeeds — never because one waiter ran out of time.
- * - the PANE LEASE belongs to a renderer painting this episode. Nothing in
- * this file grants one yet; see `uiLeaseValid`.
+ * - the PANE LEASE belongs to a renderer painting this episode. It is granted
+ * and revoked in `magmux-ui.ts`, which registers itself here; this file
+ * only ever ASKS, via `uiLeaseValid`, and never caches the answer.
*
* Letting one field mean all three is what made a four-waiter episode show
* `waiters: 4` while exactly one of them was still being retried.
*/
import { randomUUID } from "node:crypto";
-import type { ConnectionErrorKind } from "../handlers/shared/connection-error.js";
+import { type ConnectionErrorKind, isLoopback } from "../handlers/shared/connection-error.js";
import { log } from "../logger.js";
import { recoveryClock } from "./clock.js";
+import {
+ RECOVERY_PROTOCOL_VERSION,
+ type RecoveryEpisodeFrame,
+ type RecoveryFrameState,
+} from "./types.js";
export type RecoveryState = "attempting" | "waiting" | "recovered" | "handoff" | "abandoned";
@@ -95,7 +101,11 @@ interface Episode {
state: RecoveryState;
startedAtPerf: number;
startedAtMs: number;
- /** Set when a pane is first ASKED for. Null for the whole of this phase. */
+ /**
+ * Set when a pane is first ASKED for — including when the answer is an
+ * immediate no, because asking and being told no is still asking. Null on
+ * every surface that cannot open one at all.
+ */
paneRequestedAtPerf: number | null;
ladderIndex: number;
attempts: number;
@@ -151,18 +161,134 @@ export function describeEpisode(episodeId: string): Readonly<Record<string, unkn
};
}
+/**
+ * What the UI manager (`magmux-ui.ts`) plugs into the coordinator.
+ *
+ * A REGISTRATION SEAM RATHER THAN AN IMPORT, and not for testability. The UI
+ * manager reads episode state (to build a frame) and drives the ladder (`[r]`,
+ * `[q]`), so it must import this module; a direct import back the other way is
+ * a cycle. More importantly it keeps the dependency HONEST in the other
+ * direction: nothing is installed in `-p`, in `serve`, in the MCP server or in
+ * the test suite, so those paths cannot open a pane, cannot hold a lease, and
+ * behave exactly as they did before this phase — by construction rather than by
+ * a flag someone has to remember to check.
+ */
+export interface RecoveryUiHooks {
+ /** A new episode exists. The manager decides whether to ask for a pane. */
+ onEpisodeOpened(episodeId: string): void;
+ /** An episode reached a terminal state. */
+ onEpisodeClosed(episodeId: string, outcome: RecoveryOutcome): void;
+ /** Is a renderer painting this episode RIGHT NOW? */
+ leaseValid(episodeId: string): boolean;
+}
+
+let uiHooks: RecoveryUiHooks | null = null;
+
+/** Install (or, with null, remove) the recovery UI. Process-scoped. */
+export function registerRecoveryUi(hooks: RecoveryUiHooks | null): void {
+ uiHooks = hooks;
+}
+
/**
* Is a renderer currently painting this episode?
*
- * SEAM, deliberately inert in this phase. There is no socket, no pane and no
- * heartbeat yet, so the honest answer is false and the exhaustion arm keeps
- * answering today's 400. It is a function rather than a field because the
- * answer is LIVE — a lease read once and cached into a boolean is stale by the
- * time the status is chosen, which is the whole reason the previous design's
- * `uiAttached` latch was wrong.
+ * A FUNCTION AND NOT A FIELD, because the answer is LIVE. A lease read once and
+ * cached into a boolean is already stale by the time a status is chosen from
+ * it — which is precisely how the superseded `uiAttached` latch could report a
+ * banner that had been killed, closed or frozen minutes earlier.
+ *
+ * FALSE whenever no UI is installed, which is every non-interactive surface.
*/
-export function uiLeaseValid(_episodeId: string): boolean {
- return false;
+export function uiLeaseValid(episodeId: string): boolean {
+ if (!uiHooks) return false;
+ try {
+ return uiHooks.leaseValid(episodeId);
+ } catch {
+ // A throwing lease probe must read as "no surface", never as "yes". The
+ // asymmetry is the point: false costs a legible inline error, true costs
+ // the turn.
+ return false;
+ }
+}
+
+/** Record that a pane was ASKED for. Asking and being told no is still asking. */
+export function markPaneRequested(episodeId: string): void {
+ const ep = byId.get(episodeId);
+ if (!ep || ep.paneRequestedAtPerf !== null) return;
+ ep.paneRequestedAtPerf = recoveryClock().now();
+}
+
+/** Live episodes, most waiters first, tie-broken by earliest start. */
+function liveEpisodes(): Episode[] {
+ const live = [...episodes.values()].filter(
+ (e) => e.state === "attempting" || e.state === "waiting" || e.state === "handoff"
+ );
+ live.sort((a, b) => b.waiters.size - a.waiters.size || a.startedAtMs - b.startedAtMs);
+ return live;
+}
+
+/**
+ * The episode a renderer should paint, as a wire frame — or null.
+ *
+ * Concurrent episodes are ORDINARY: the main loop's provider and the small
+ * title model's provider can be unreachable at the same moment. Which one wins
+ * is not load-bearing, because the lease is per-episode — a non-rendered
+ * episode simply holds no lease, and at exhaustion answers the inline error,
+ * which is exactly what "a retryable status is permissible only while the
+ * reason is legible" demands.
+ *
+ * THE UNIT CONVERSION HAPPENS HERE AND NOWHERE ELSE. Intervals inside this
+ * module are process ms; everything on the wire is epoch ms, because the pane
+ * is a different process with a different `performance.now()` origin and would
+ * render a process-ms instant as an arbitrary number with no error anywhere.
+ */
+export function renderableEpisodeFrame(): RecoveryEpisodeFrame | null {
+ const live = liveEpisodes();
+ const ep = live[0];
+ if (!ep) return null;
+ const clock = recoveryClock();
+ const perfNow = clock.now();
+ const epochNow = Date.now();
+ const nextAttemptAtMs =
+ ep.nextAttemptAtPerf === null ? null : epochNow + (ep.nextAttemptAtPerf - perfNow);
+ return {
+ v: RECOVERY_PROTOCOL_VERSION,
+ type: "episode",
+ episodeId: ep.episodeId,
+ state: ep.state as RecoveryFrameState,
+ tier: ep.maxTier,
+ providerDisplayName: ep.providerDisplayName,
+ host: ep.host,
+ endpoint: ep.endpoint,
+ kind: ep.kind,
+ code: ep.code,
+ loopback: ep.loopback,
+ reason: ep.reason,
+ attempts: ep.attempts,
+ clientRetries: ep.clientRetries,
+ startedAtMs: ep.startedAtMs,
+ nextAttemptAtMs,
+ lastOutcome: ep.lastOutcome,
+ waiters: ep.waiters.size,
+ otherEpisodes: live.length - 1,
+ };
+}
+
+/** How many attempts this episode has made. For the manual-retry log line. */
+export function attemptsSoFar(episodeId: string): number {
+ return byId.get(episodeId)?.attempts ?? 0;
+}
+
+/** Is this episode still live? Used by the UI manager's lease check. */
+export function episodeIsLive(episodeId: string): boolean {
+ const ep = byId.get(episodeId);
+ if (!ep) return false;
+ return ep.state === "attempting" || ep.state === "waiting" || ep.state === "handoff";
+}
+
+/** `[q] give up` — the user said stop, so every live episode stops. */
+export function giveUpAll(): void {
+ for (const ep of liveEpisodes()) giveUp(ep.episodeId);
}
export interface EpisodeHandle {
@@ -247,7 +373,11 @@ export function joinEpisode(seed: EpisodeSeed): EpisodeHandle {
endpoint: seed.endpoint,
host,
kind: seed.kind,
- loopback: false,
+ // Decided ONCE, at creation, from the endpoint — never from the error
+ // code. On macOS an unrouted REMOTE address reports `ECONNREFUSED` after
+ // 75 s, the same code a loopback port refuses in 5 ms, so the code cannot
+ // tell them apart and only the address can.
+ loopback: isLoopback(seed.endpoint),
code: seed.code,
reason: seed.reason,
state: "attempting",
@@ -279,6 +409,18 @@ export function joinEpisode(seed: EpisodeSeed): EpisodeHandle {
`[Recovery] episode ${ep.episodeId} opened for ${ep.providerDisplayName} at ${ep.endpoint} ` +
`(${ep.kind}/${ep.code ?? "?"})`
);
+ // Ask for a surface. Nothing is installed unless this process was launched
+ // through the magmux wrapper, so on every other surface this is a no-op —
+ // which is why `-p`, `serve` and the suite are untouched.
+ if (uiHooks) {
+ try {
+ uiHooks.onEpisodeOpened(ep.episodeId);
+ } catch (err) {
+ // A UI failure must never break recovery. The banner is what EARNS the
+ // wait; it is not what performs it.
+ log(`[Recovery] UI could not open for episode ${ep.episodeId}: ${String(err)}`);
+ }
+ }
}
const episode = ep;
@@ -466,6 +608,13 @@ function closeEpisode(episode: Episode, outcome: RecoveryOutcome): void {
`[Recovery] episode ${episode.episodeId} closed: ${outcome} after ` +
`${Math.round(clock.now() - episode.startedAtPerf)}ms and ${episode.attempts} attempts`
);
+ if (uiHooks) {
+ try {
+ uiHooks.onEpisodeClosed(episode.episodeId, outcome);
+ } catch {
+ /* see onEpisodeOpened — the UI never breaks recovery */
+ }
+ }
}
/**
@@ -482,12 +631,22 @@ export function tryNow(episodeId: string): void {
for (const w of [...ep.waiters]) w.wake?.({ kind: "attempt" });
}
-/** Wake every waiter of every episode with `gave_up`. The `[q]` key, later. */
+/**
+ * `[q] give up` for ONE episode: every waiter answers today's inline error and
+ * the episode is over.
+ *
+ * `gaveUp` latches before the close so a waiter that has not parked yet — one
+ * between attempts at the instant the key was pressed — still reads the answer
+ * rather than parking on a timer that no longer exists. The user said stop, so
+ * stop, and say why inline: this is the missing half of an affordance the
+ * superseded design labelled "give up" and gave no effect at all.
+ */
export function giveUp(episodeId: string): void {
const ep = byId.get(episodeId);
if (!ep) return;
ep.gaveUp = true;
clearAttemptTimer(ep);
+ closeEpisode(ep, "gave_up");
for (const w of [...ep.waiters]) w.wake?.({ kind: "gave_up" });
}
diff --git a/packages/cli/src/recovery/lease.test.ts b/packages/cli/src/recovery/lease.test.ts
new file mode 100644
index 00000000..59b203b0
--- /dev/null
+++ b/packages/cli/src/recovery/lease.test.ts
@@ -0,0 +1,252 @@
+/**
+ * The UI lease — the round-2 CRITICAL, and the reason this file exists at all.
+ *
+ * The lease is what will decide, one phase from now, whether an exhausted
+ * episode may answer a RETRYABLE status instead of an inline error. The rule it
+ * encodes is: *a retryable status is permissible exactly while claudish still
+ * has a surface on which the reason is legible.*
+ *
+ * The superseded design renewed that lease from FRAME RECEIPTS, and frames only
+ * ticked while the ladder was `waiting`. A connect against an unreachable host
+ * takes 20–75 s (measured on this machine: `192.0.2.1` = 75 005 ms), during
+ * which the proxy has nothing new to say — so the lease died ten seconds into
+ * every slow attempt, and the exhaustion arm, which reads it immediately after
+ * an attempt returns, saw it false for the ENTIRE failure class the feature was
+ * built for, with a live painted banner on screen.
+ *
+ * And every test passed, because every fault in the suite was a ~1 ms loopback
+ * refusal. These are the tests that can fail on it.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { EventEmitter } from "node:events";
+import { resetRecoveryClock, setRecoveryClock } from "./clock.js";
+import {
+ __handleCommandForTests,
+ __resetUiStateForTests,
+ __setPaneOpenForTests,
+ describeLease,
+ uiLeaseValid,
+} from "./magmux-ui.js";
+import { runRecoveryPane } from "./pane-app.js";
+import {
+ type RecoverySocketServer,
+ createRecoverySocketDir,
+ recoverySocketPathIn,
+ startRecoverySocketServer,
+} from "./socket-server.js";
+import { FakeClock } from "./test-helpers/fake-clock.js";
+import {
+ RECOVERY_PROTOCOL_VERSION,
+ type RecoveryEpisodeFrame,
+ UI_HEARTBEAT_MS,
+ UI_LEASE_MS,
+} from "./types.js";
+
+const EPISODE: RecoveryEpisodeFrame = {
+ v: RECOVERY_PROTOCOL_VERSION,
+ type: "episode",
+ episodeId: "ep-slow",
+ // `attempting`, not `waiting`: this is the state the proxy sits in for the
+ // whole of a 45-second connect, and the state in which the old design went
+ // silent.
+ state: "attempting",
+ tier: 1,
+ providerDisplayName: "Openai-codex",
+ host: "chatgpt.com",
+ endpoint: "https://chatgpt.com/backend-api/codex/responses",
+ kind: "unreachable",
+ code: "ConnectionClosed",
+ loopback: false,
+ reason: "Cannot reach Openai-codex at https://chatgpt.com/backend-api/codex/responses.",
+ attempts: 4,
+ clientRetries: 0,
+ startedAtMs: 1_700_000_000_000,
+ nextAttemptAtMs: null,
+ lastOutcome: "ConnectionClosed after 45001ms",
+ waiters: 1,
+ otherEpisodes: 0,
+};
+
+let clock: FakeClock;
+
+beforeEach(() => {
+ clock = new FakeClock(0);
+ setRecoveryClock(clock);
+ __resetUiStateForTests();
+});
+
+afterEach(() => {
+ __resetUiStateForTests();
+ resetRecoveryClock();
+});
+
+const ack = (episodeId: string) =>
+ __handleCommandForTests({
+ v: RECOVERY_PROTOCOL_VERSION,
+ type: "ack",
+ episodeId,
+ paintedAt: Date.now(),
+ });
+
+describe("the UI lease (unit, fake clock)", () => {
+ test("SURVIVES an attempt far longer than UI_LEASE_MS while the renderer heartbeats", async () => {
+ // THE round-2 CRITICAL, expressed as an assertion. 45 seconds of attempt —
+ // 4.5 leases — with NOT ONE FRAME emitted, because the proxy has nothing to
+ // say while a connect hangs. The renderer's own heartbeat is the only thing
+ // renewing anything.
+ __setPaneOpenForTests(true);
+ ack("ep-slow");
+
+ for (let elapsed = 0; elapsed < 45_000; elapsed += UI_HEARTBEAT_MS) {
+ await clock.advance(UI_HEARTBEAT_MS);
+ ack("ep-slow"); // the renderer, on ITS timer, not on a frame
+ expect(uiLeaseValid("ep-slow")).toBe(true);
+ }
+
+ expect(clock.now()).toBeGreaterThanOrEqual(45_000);
+ expect(uiLeaseValid("ep-slow")).toBe(true);
+ });
+
+ test("expires UI_LEASE_MS after the last heartbeat — a frozen renderer holds nothing", async () => {
+ __setPaneOpenForTests(true);
+ ack("ep-slow");
+ await clock.advance(UI_LEASE_MS);
+ expect(uiLeaseValid("ep-slow")).toBe(true); // exactly at the boundary
+ await clock.advance(1);
+ expect(uiLeaseValid("ep-slow")).toBe(false);
+ });
+
+ test("a forged client cannot grant itself a lease: paneOpen is required", () => {
+ // The socket has no same-uid boundary and cannot have one. What is defended
+ // is the CONSEQUENCE: the one forbidden state (a retryable status with no
+ // banner) additionally requires that WE opened a pane and hold the reply,
+ // which is a fact no client can cause.
+ __setPaneOpenForTests(false);
+ ack("ep-slow");
+ expect(uiLeaseValid("ep-slow")).toBe(false);
+ expect(describeLease("ep-slow").lastAckAgoMs).toBe(0); // it DID heartbeat…
+ expect(describeLease("ep-slow").paneOpen).toBe(false); // …and it still gets nothing
+ });
+
+ test("the lease is per-EPISODE: heartbeating one does not lease another", () => {
+ __setPaneOpenForTests(true);
+ ack("ep-slow");
+ expect(uiLeaseValid("ep-slow")).toBe(true);
+ expect(uiLeaseValid("ep-other")).toBe(false);
+ });
+
+ test("switching the painted episode lets the old lease lapse with no extra protocol", async () => {
+ __setPaneOpenForTests(true);
+ ack("ep-slow");
+ for (let i = 0; i < 12; i++) {
+ await clock.advance(UI_HEARTBEAT_MS);
+ ack("ep-other"); // the renderer moved on
+ }
+ expect(uiLeaseValid("ep-other")).toBe(true);
+ expect(uiLeaseValid("ep-slow")).toBe(false);
+ });
+
+ test("`bye` revokes every lease, and no later heartbeat can resurrect one", () => {
+ // `[q] give up` is the affordance the superseded design labelled
+ // "give up" and gave no server-side effect at all — so a user could close
+ // the banner while the machine kept retrying and eventually answered a
+ // retryable status with nothing on screen. That is the design's OWN stated
+ // worst case, reached through a key that says "give up".
+ __setPaneOpenForTests(true);
+ ack("ep-slow");
+ ack("ep-other");
+ expect(uiLeaseValid("ep-slow")).toBe(true);
+
+ __handleCommandForTests({ v: RECOVERY_PROTOCOL_VERSION, type: "bye", reason: "user_quit" });
+
+ expect(uiLeaseValid("ep-slow")).toBe(false);
+ expect(uiLeaseValid("ep-other")).toBe(false);
+ // The pane is genuinely gone, not merely un-leased…
+ expect(describeLease("ep-slow").paneOpen).toBe(false);
+ // …so a client that keeps heartbeating into the closed socket gets nothing.
+ // Without this, `bye` would be undone by the very next tick of a renderer
+ // that had not noticed it was closed.
+ ack("ep-slow");
+ expect(uiLeaseValid("ep-slow")).toBe(false);
+ });
+});
+
+// ─── The same property, end to end, over a real socket ───────────────────────
+
+class FakeStdin extends EventEmitter {
+ rawMode = false;
+ setRawMode(on: boolean) {
+ this.rawMode = on;
+ return this;
+ }
+ resume() {
+ return this;
+ }
+ pause() {
+ return this;
+ }
+}
+
+class FakeStdout extends EventEmitter {
+ columns = 100;
+ chunks: string[] = [];
+ write(s: string) {
+ this.chunks.push(s);
+ return true;
+ }
+}
+
+describe("the UI lease (end to end, real socket, real renderer)", () => {
+ test("the REAL pane heartbeats through total frame silence, and the lease holds", async () => {
+ // One frame in the whole run — the replay — and then nothing, which is what
+ // a hung connect looks like on the wire. If the renderer ever goes back to
+ // acking per received frame, the ack count collapses to 1 and the lease is
+ // dead by fake-t=10s. That is the defect this test exists to fail on.
+ const path = recoverySocketPathIn(createRecoverySocketDir());
+ let framesSent = 0;
+ let acks = 0;
+
+ let server: RecoverySocketServer | null = null;
+ const stdin = new FakeStdin();
+ const stdout = new FakeStdout();
+
+ server = await startRecoverySocketServer({
+ socketPath: path,
+ snapshot: () => {
+ framesSent++;
+ return EPISODE;
+ },
+ onCommand: (cmd) => {
+ if (cmd.type === "ack") acks++;
+ __handleCommandForTests(cmd);
+ },
+ tickMs: 0, // the proxy says NOTHING after the replay
+ });
+
+ __setPaneOpenForTests(true);
+ const paneDone = runRecoveryPane({
+ socketPath: path,
+ stdin: stdin as unknown as NodeJS.ReadStream,
+ stdout: stdout as unknown as NodeJS.WriteStream,
+ });
+
+ // ~4 real heartbeat intervals, with the LEASE clock advancing 5 s each time
+ // — so the episode is 20 s old in lease terms while the socket has carried
+ // exactly one frame.
+ for (let i = 0; i < 4; i++) {
+ await new Promise((r) => setTimeout(r, UI_HEARTBEAT_MS + 120));
+ await clock.advance(5_000);
+ expect(uiLeaseValid("ep-slow")).toBe(true);
+ }
+
+ expect(framesSent).toBe(1);
+ expect(acks).toBeGreaterThanOrEqual(3);
+ expect(clock.now()).toBeGreaterThan(UI_LEASE_MS);
+ expect(uiLeaseValid("ep-slow")).toBe(true);
+
+ stdin.emit("data", Buffer.from("q"));
+ await paneDone;
+ await server.close();
+ }, 15_000);
+});
diff --git a/packages/cli/src/recovery/magmux-ui.ts b/packages/cli/src/recovery/magmux-ui.ts
new file mode 100644
index 00000000..f09bd8bd
--- /dev/null
+++ b/packages/cli/src/recovery/magmux-ui.ts
@@ -0,0 +1,568 @@
+/**
+ * The recovery UI manager — the single owner of the socket, the pane, the
+ * `paneOpen` fact, the post-`bye` suppression window, the cross-process lock
+ * and the lease.
+ *
+ * OWNERSHIP IS PROCESS-SCOPED; ONLY EPISODES ARE EPISODE-SCOPED. There is ONE
+ * pane and it multiplexes every episode, so "a pane exists" is a property of
+ * this module and never of an episode. An earlier design hung `paneOpened` on
+ * each episode, which meant the second concurrent episode could never be leased
+ * — only the first one ever received an `open_pane` reply — and that closing
+ * one episode tore down a socket a different episode's renderer was reading.
+ * `releaseEpisodeUi` therefore emits one episode's `closed` frame and NOTHING
+ * else; the socket and the pane come down only when no episode is live at all.
+ *
+ * THE LEASE IS THE WHOLE POINT OF THIS FILE. It is not a latch, not a config
+ * read and not "a client connected":
+ *
+ * uiLeaseValid(id) ⇔ paneOpen (WE opened it, and hold the reply)
+ * ∧ some client heartbeated `ack` naming THIS episode
+ * within UI_LEASE_MS
+ *
+ * Both clauses are load-bearing. `paneOpen` is what a forged same-uid client
+ * cannot manufacture, so it cannot talk the proxy into the one forbidden state
+ * (a retryable status with no banner on screen). The heartbeat window is what
+ * makes the lease self-clearing on EOF, on a killed pane, on a frozen renderer
+ * and on `[q]`, with no cleanup path anyone has to remember.
+ *
+ * AND THE HEARTBEAT IS THE RENDERER'S, NOT OURS. The pane sends `ack` on its
+ * own 1 Hz timer for as long as it is painting an episode, whether or not a
+ * frame arrived. Renewing from frame receipts would tie the renderer's apparent
+ * liveness to OUR emission cadence — and we have nothing to say for the 20–75 s
+ * an unreachable connect takes (measured: `192.0.2.1` = 75 005 ms). The lease
+ * would then read false at exhaustion for precisely the failure class this
+ * feature exists for, with a live painted banner on screen, and every
+ * loopback-only test would still pass.
+ *
+ * NOTHING HERE CHANGES A STATUS IN THIS PHASE. The lease is computed and
+ * logged; the 503-vs-400 flip is the next one. Shipping the flip before the
+ * banner exists recreates the buried-reason bug the feature is meant to kill.
+ */
+
+import { closeSync, existsSync, openSync, readFileSync, unlinkSync, writeSync } from "node:fs";
+import { type Socket, connect as netConnect } from "node:net";
+import { basename, join } from "node:path";
+import { log } from "../logger.js";
+import { recoveryClock } from "./clock.js";
+import {
+ type RecoveryOutcome,
+ type RecoveryUiHooks,
+ attemptsSoFar,
+ episodeIsLive,
+ giveUpAll,
+ markPaneRequested,
+ registerRecoveryUi,
+ renderableEpisodeFrame,
+ tryNow,
+} from "./coordinator.js";
+import {
+ type RecoverySocketServer,
+ cleanupRecoverySocket,
+ createRecoverySocketDir,
+ recoverySocketPathIn,
+ startRecoverySocketServer,
+} from "./socket-server.js";
+import {
+ PANE_LINGER_MS,
+ RECOVERY_PROTOCOL_VERSION,
+ type RecoveryCommand,
+ UI_LEASE_MS,
+} from "./types.js";
+
+/** After `[q]`, do not re-open a pane for this long. */
+export const UI_SUPPRESS_AFTER_BYE_MS = 60_000;
+
+// ─── The control endpoint ────────────────────────────────────────────────────
+
+let explicitControlSock: string | null = null;
+
+/**
+ * The magmux control socket, handed in by the launch wrapper.
+ *
+ * AN EXPLICIT INPUT, NOT AN ENV LOOKUP, and that is the only thing that makes
+ * this work at all: claudish is magmux's PARENT, and `MAGMUX_SOCK` is exported
+ * DOWNWARD, so the parent can never learn a pid-derived path. The wrapper
+ * generates `--id claudish-<pid>`, which fixes the path before magmux starts,
+ * and hands it here.
+ */
+export function setMagmuxControlSocket(sock: string | null): void {
+ explicitControlSock = sock;
+}
+
+/**
+ * Where can this process send `open_pane` right now?
+ *
+ * Order: the path the wrapper handed us, then an ambient `MAGMUX_SOCK` (the
+ * already-inside-a-pane case, e.g. `team --grid --mode interactive`), then null.
+ *
+ * IT ANSWERS A LAUNCH-ORDER QUESTION, NOT A USER PREFERENCE, and only this
+ * module may read it. Null means "this process cannot open a pane" — true in
+ * `-p`, in `serve`, in the MCP server, in every unwrapped launch and in the
+ * whole test suite. It must never gate whether a RETRY happens: keying the
+ * retry ladder on it made recovery skip universally in every phase before the
+ * wrapper existed, twice, and both times the same rule decided the tests.
+ */
+export function resolveMagmuxControl(): { sock: string } | null {
+ if (explicitControlSock && existsSync(explicitControlSock)) return { sock: explicitControlSock };
+ const ambient = process.env.MAGMUX_SOCK;
+ if (ambient && existsSync(ambient)) return { sock: ambient };
+ return null;
+}
+
+// ─── Manager state ───────────────────────────────────────────────────────────
+
+interface ManagerState {
+ server: RecoverySocketServer | null;
+ socketPath: string | null;
+ control: Socket | null;
+ /** True only once magmux has REPLIED to our `open_pane`. */
+ paneOpen: boolean;
+ /** magmux's own index for our pane. Read from the reply — never assumed. */
+ paneIndex: number | null;
+ /** episodeId → PROCESS ms of the last heartbeat naming it. */
+ leases: Map<string, number>;
+ /** PROCESS ms before which no pane may be opened (post-`bye`). */
+ suppressUntilPerf: number;
+ /** Guards the async open against re-entry. */
+ opening: boolean;
+ lockPath: string | null;
+ lingerTimer: unknown | null;
+ nextControlId: number;
+}
+
+const state: ManagerState = {
+ server: null,
+ socketPath: null,
+ control: null,
+ paneOpen: false,
+ paneIndex: null,
+ leases: new Map(),
+ suppressUntilPerf: 0,
+ opening: false,
+ lockPath: null,
+ lingerTimer: null,
+ nextControlId: 1,
+};
+
+let installed = false;
+
+// ─── The lease ───────────────────────────────────────────────────────────────
+
+/**
+ * Is a renderer painting this episode right now?
+ *
+ * Read LIVE at the moment a status is chosen. Never cached: by the time a
+ * caller destructures a boolean off a result object, the pane it describes may
+ * have been killed.
+ */
+export function uiLeaseValid(episodeId: string): boolean {
+ if (!state.paneOpen) return false;
+ const last = state.leases.get(episodeId);
+ if (last === undefined) return false;
+ return recoveryClock().now() - last <= UI_LEASE_MS;
+}
+
+/** Diagnostics for the log and for tests. Never used to decide anything. */
+export function describeLease(episodeId: string): {
+ paneOpen: boolean;
+ lastAckAgoMs: number | null;
+ valid: boolean;
+} {
+ const last = state.leases.get(episodeId);
+ return {
+ paneOpen: state.paneOpen,
+ lastAckAgoMs: last === undefined ? null : Math.round(recoveryClock().now() - last),
+ valid: uiLeaseValid(episodeId),
+ };
+}
+
+// ─── The magmux control connection ───────────────────────────────────────────
+
+async function connectControl(sock: string): Promise<Socket | null> {
+ try {
+ return await new Promise<Socket>((resolve, reject) => {
+ const s = netConnect(sock);
+ s.once("connect", () => resolve(s));
+ s.once("error", reject);
+ });
+ } catch {
+ return null;
+ }
+}
+
+/** Send one control message and wait for the `reply` carrying the same id. */
+function request(
+ socket: Socket,
+ msg: Record<string, unknown>,
+ timeoutMs = 3_000
+): Promise<Record<string, unknown> | null> {
+ const id = state.nextControlId++;
+ return new Promise((resolve) => {
+ let buf = "";
+ let settled = false;
+ const finish = (v: Record<string, unknown> | null) => {
+ if (settled) return;
+ settled = true;
+ socket.removeListener("data", onData);
+ clearTimeout(timer);
+ resolve(v);
+ };
+ const onData = (chunk: Buffer) => {
+ buf += chunk.toString("utf-8");
+ let nl = buf.indexOf("\n");
+ while (nl >= 0) {
+ const line = buf.slice(0, nl).trim();
+ buf = buf.slice(nl + 1);
+ nl = buf.indexOf("\n");
+ if (!line) continue;
+ try {
+ const evt = JSON.parse(line) as Record<string, unknown>;
+ if (evt.type === "reply" && evt.id === id) finish(evt);
+ } catch {
+ /* magmux emits live events on the same stream; ignore what is not ours */
+ }
+ }
+ };
+ const timer = setTimeout(() => finish(null), timeoutMs);
+ (timer as { unref?: () => void }).unref?.();
+ socket.on("data", onData);
+ try {
+ socket.write(`${JSON.stringify({ ...msg, id })}\n`);
+ } catch {
+ finish(null);
+ }
+ });
+}
+
+// ─── The cross-process single-pane lock ──────────────────────────────────────
+
+/**
+ * ONE recovery pane per magmux, not N.
+ *
+ * `team --grid --mode interactive` launches one claudish per pane and magmux
+ * exports the same `MAGMUX_SOCK` to all of them, so one outage hits N processes
+ * at once and a merely process-level singleton would open N panes into one
+ * grid. The losers still run the ladder — they retry and they recover — they
+ * just never open a pane, hold no lease, and answer the inline error at
+ * exhaustion. That is honest: one banner names one slot's provider, and every
+ * slot still recovers.
+ */
+function acquirePaneLock(controlSock: string): string | null {
+ const lockPath = join("/tmp", `claudish-recovery-${basename(controlSock)}.lock`);
+ for (let attempt = 0; attempt < 2; attempt++) {
+ try {
+ const fd = openSync(lockPath, "wx", 0o600);
+ writeSync(fd, String(process.pid));
+ closeSync(fd);
+ return lockPath;
+ } catch {
+ // Held — by a live peer, or by debris from one that was SIGKILLed.
+ try {
+ const owner = Number(readFileSync(lockPath, "utf-8").trim());
+ if (Number.isFinite(owner) && owner > 0) {
+ try {
+ process.kill(owner, 0);
+ return null; // a live peer owns the banner
+ } catch {
+ unlinkSync(lockPath); // the owner is gone; the lock is debris
+ continue;
+ }
+ }
+ unlinkSync(lockPath);
+ } catch {
+ return null;
+ }
+ }
+ }
+ return null;
+}
+
+function releasePaneLock(): void {
+ if (!state.lockPath) return;
+ try {
+ if (existsSync(state.lockPath)) {
+ const owner = Number(readFileSync(state.lockPath, "utf-8").trim());
+ if (owner === process.pid) unlinkSync(state.lockPath);
+ }
+ } catch {
+ /* best effort */
+ }
+ state.lockPath = null;
+}
+
+// ─── The pane command line ───────────────────────────────────────────────────
+
+/**
+ * How to run `claudish recovery-pane` from wherever this build lives.
+ *
+ * Built from `process.argv` rather than from the name `claudish`, because the
+ * process that must start is THIS build: a global install mid-update, a `bun
+ * run src/index.ts` in a worktree and an npm-installed bundle are three
+ * different files, and a pane from a different build is the version-skew case
+ * the protocol has to print a notice for. `claudish` on PATH stays as the
+ * fallback for a packaging shape neither argv entry describes.
+ */
+export function recoveryPaneCommandLine(socketPath: string): string {
+ const script = process.argv[1];
+ const q = (s: string) => `'${s.replace(/'/g, "'\\''")}'`;
+ if (script && (script.endsWith(".ts") || script.endsWith(".js") || script.endsWith(".cjs"))) {
+ return `${q(process.execPath)} ${q(script)} recovery-pane --socket ${q(socketPath)}`;
+ }
+ return `claudish recovery-pane --socket ${q(socketPath)}`;
+}
+
+// ─── Opening and closing the pane ────────────────────────────────────────────
+
+function onCommand(cmd: RecoveryCommand): void {
+ switch (cmd.type) {
+ case "hello":
+ log(`[Recovery] pane connected (pid ${cmd.pid}, protocol ${cmd.protocol})`);
+ break;
+ case "ack": {
+ // THE LEASE RENEWAL. Only here, and only from a renderer that painted.
+ const first = !state.leases.has(cmd.episodeId);
+ state.leases.set(cmd.episodeId, recoveryClock().now());
+ if (first) {
+ log(
+ `[Recovery] pane is painting episode ${cmd.episodeId} — lease granted ` +
+ `(paneOpen=${state.paneOpen})`
+ );
+ }
+ break;
+ }
+ case "retry_now":
+ if (episodeIsLive(cmd.episodeId)) {
+ const n = attemptsSoFar(cmd.episodeId) + 1;
+ log(`[Recovery] attempt ${n} (manual) for episode ${cmd.episodeId}`);
+ tryNow(cmd.episodeId);
+ }
+ break;
+ case "bye":
+ log(`[Recovery] pane said bye (${cmd.reason}) — giving up on every live episode`);
+ state.suppressUntilPerf = recoveryClock().now() + UI_SUPPRESS_AFTER_BYE_MS;
+ giveUpAll();
+ // The pane the user just closed must not reappear two seconds later, and
+ // closing it is ALSO what revokes every lease: `teardown()` drops
+ // `paneOpen`, and without that clause no heartbeat can grant anything.
+ // One revocation path, not two — a second one is a second thing to
+ // forget, and the superseded design's bug was exactly a revocation path
+ // that existed on paper and cleared nothing.
+ void closePane();
+ break;
+ }
+}
+
+async function openPane(): Promise<void> {
+ const control = resolveMagmuxControl();
+ if (!control) {
+ log("[Recovery] no magmux control endpoint — no pane, recovery continues without a banner");
+ return;
+ }
+ const lock = acquirePaneLock(control.sock);
+ if (!lock) {
+ log("[Recovery] another claudish already owns the recovery pane for this magmux");
+ return;
+ }
+ state.lockPath = lock;
+
+ const dir = createRecoverySocketDir();
+ const socketPath = recoverySocketPathIn(dir);
+ state.socketPath = socketPath;
+ state.server = await startRecoverySocketServer({
+ socketPath,
+ snapshot: () => renderableEpisodeFrame(),
+ onCommand,
+ onDisconnect: () => {
+ // EOF revokes nothing by itself — the lease expires on its own within
+ // UI_LEASE_MS, and having exactly one expiry path means there is no
+ // second one to forget. The retry loop is not the pane's business.
+ log("[Recovery] pane disconnected");
+ },
+ });
+
+ const socket = await connectControl(control.sock);
+ if (!socket) {
+ log(`[Recovery] could not reach magmux at ${control.sock}`);
+ await teardown();
+ return;
+ }
+ state.control = socket;
+ socket.on("error", () => {
+ state.paneOpen = false;
+ });
+ socket.on("close", () => {
+ state.paneOpen = false;
+ });
+
+ const reply = await request(socket, {
+ type: "open_pane",
+ cmd: recoveryPaneCommandLine(socketPath),
+ cwd: process.cwd(),
+ split: "vertical",
+ });
+ if (!reply || reply.ok !== true) {
+ log(`[Recovery] magmux refused open_pane: ${JSON.stringify(reply)}`);
+ await teardown();
+ return;
+ }
+ const result = reply.result as { pane?: number } | undefined;
+ // Read the index back. magmux's own control panel occupies an index, so the
+ // first pane an agent opens is NOT necessarily 1 — measured: 2.
+ state.paneIndex = typeof result?.pane === "number" ? result.pane : null;
+ state.paneOpen = true;
+ log(`[Recovery] recovery pane opened (magmux pane ${state.paneIndex}) at ${socketPath}`);
+}
+
+async function closePane(): Promise<void> {
+ if (state.control && state.paneIndex !== null) {
+ await request(state.control, { type: "close_pane", pane: state.paneIndex, force: true }, 1_000);
+ }
+ await teardown();
+}
+
+async function teardown(): Promise<void> {
+ state.paneOpen = false;
+ state.paneIndex = null;
+ state.leases.clear();
+ if (state.control) {
+ try {
+ state.control.end();
+ } catch {
+ /* already gone */
+ }
+ state.control = null;
+ }
+ if (state.server) {
+ await state.server.close();
+ state.server = null;
+ } else if (state.socketPath) {
+ cleanupRecoverySocket(state.socketPath);
+ }
+ state.socketPath = null;
+ releasePaneLock();
+}
+
+// ─── The hooks ───────────────────────────────────────────────────────────────
+
+/**
+ * Ask for a surface for this episode. Idempotent, and never awaited by the
+ * retry loop — a pane that takes 200 ms to appear must not delay the first
+ * retry by 200 ms.
+ */
+export function ensureRecoveryUi(episodeId: string): void {
+ // Asking and being told no is still asking: the timestamp is what a later
+ // phase's grace rule measures from, and it must be set even when the answer
+ // is an immediate no.
+ markPaneRequested(episodeId);
+ if (state.lingerTimer !== null) {
+ recoveryClock().clearTimeout(state.lingerTimer);
+ state.lingerTimer = null;
+ }
+ if (state.paneOpen || state.opening) return;
+ if (recoveryClock().now() < state.suppressUntilPerf) {
+ log("[Recovery] pane suppressed — the user pressed [q] less than a minute ago");
+ return;
+ }
+ state.opening = true;
+ void openPane()
+ .catch((err) => {
+ log(`[Recovery] pane could not be opened: ${String(err)}`);
+ })
+ .finally(() => {
+ state.opening = false;
+ });
+}
+
+/**
+ * One episode ended. Tell the renderer, and NOTHING else — a concurrent
+ * episode may still be painting through the same socket and the same pane.
+ */
+export function releaseEpisodeUi(episodeId: string, outcome: RecoveryOutcome): void {
+ state.leases.delete(episodeId);
+ state.server?.broadcast({
+ v: RECOVERY_PROTOCOL_VERSION,
+ type: "closed",
+ episodeId,
+ outcome,
+ });
+ if (renderableEpisodeFrame() !== null) return; // something else is still live
+ // Nothing left to paint. Linger, so the pane does not flap open and shut on
+ // every client-retry cycle, and so the last thing that happened stays
+ // readable for someone who looked away.
+ if (state.lingerTimer !== null) recoveryClock().clearTimeout(state.lingerTimer);
+ const clock = recoveryClock();
+ const timer = clock.setTimeout(() => {
+ state.lingerTimer = null;
+ if (renderableEpisodeFrame() !== null) return;
+ void closePane();
+ }, PANE_LINGER_MS);
+ clock.unref?.(timer);
+ state.lingerTimer = timer;
+}
+
+const hooks: RecoveryUiHooks = {
+ onEpisodeOpened: (episodeId) => ensureRecoveryUi(episodeId),
+ onEpisodeClosed: (episodeId, outcome) => releaseEpisodeUi(episodeId, outcome),
+ leaseValid: (episodeId) => uiLeaseValid(episodeId),
+};
+
+/**
+ * Install the recovery UI into the coordinator.
+ *
+ * Called from exactly one place — the magmux launch wrapper, once it knows the
+ * control socket. Every other entry point leaves it uninstalled, so `-p`,
+ * `--stdin`, `serve`, the MCP server and the test suite cannot open a pane and
+ * cannot hold a lease, without any of them having to opt out.
+ */
+export function installRecoveryUi(controlSock: string | null): void {
+ setMagmuxControlSocket(controlSock);
+ if (installed) return;
+ installed = true;
+ registerRecoveryUi(hooks);
+ process.on("exit", () => {
+ releasePaneLock();
+ if (state.socketPath) cleanupRecoverySocket(state.socketPath);
+ });
+}
+
+/** Tear everything down. Proxy shutdown, and tests. */
+export async function shutdownRecoveryUi(): Promise<void> {
+ if (state.lingerTimer !== null) {
+ recoveryClock().clearTimeout(state.lingerTimer);
+ state.lingerTimer = null;
+ }
+ await closePane();
+ registerRecoveryUi(null);
+ installed = false;
+ state.suppressUntilPerf = 0;
+ explicitControlSock = null;
+}
+
+/** Tests only: the manager is process state shared with every sibling test. */
+export function __setPaneOpenForTests(open: boolean, socketPath?: string): void {
+ state.paneOpen = open;
+ if (socketPath !== undefined) state.socketPath = socketPath;
+}
+
+/** Tests only: feed a command as though a pane had sent it. */
+export function __handleCommandForTests(cmd: RecoveryCommand): void {
+ onCommand(cmd);
+}
+
+/** Tests only: forget every lease and every suppression. */
+export function __resetUiStateForTests(): void {
+ state.leases.clear();
+ state.paneOpen = false;
+ state.paneIndex = null;
+ state.suppressUntilPerf = 0;
+ state.opening = false;
+ state.socketPath = null;
+ state.server = null;
+ state.control = null;
+ if (state.lingerTimer !== null) {
+ recoveryClock().clearTimeout(state.lingerTimer);
+ state.lingerTimer = null;
+ }
+ releasePaneLock();
+ explicitControlSock = null;
+}
diff --git a/packages/cli/src/recovery/pane-app.test.ts b/packages/cli/src/recovery/pane-app.test.ts
new file mode 100644
index 00000000..7b92cc52
--- /dev/null
+++ b/packages/cli/src/recovery/pane-app.test.ts
@@ -0,0 +1,342 @@
+/**
+ * The recovery banner.
+ *
+ * `renderPane` is pure, which is why it is the thing asserted on: the live
+ * evidence (a capture of the real pane in a real magmux) and these tests then
+ * look at the SAME function rather than at two implementations that drift.
+ */
+
+import { afterEach, describe, expect, test } from "bun:test";
+import { EventEmitter } from "node:events";
+import { resetThemeModeForTests, setThemeMode } from "../theme/theme-mode.js";
+import {
+ type PaneViewState,
+ formatDuration,
+ parsePaneArgs,
+ renderPane,
+ runRecoveryPane,
+ stripAnsi,
+ wrapText,
+} from "./pane-app.js";
+import {
+ type RecoverySocketServer,
+ createRecoverySocketDir,
+ recoverySocketPathIn,
+ startRecoverySocketServer,
+} from "./socket-server.js";
+import {
+ PROXY_SILENT_MS,
+ RECOVERY_PROTOCOL_VERSION,
+ type RecoveryCommand,
+ type RecoveryEpisodeFrame,
+} from "./types.js";
+
+const T0 = 1_700_000_000_000;
+
+const EPISODE: RecoveryEpisodeFrame = {
+ v: RECOVERY_PROTOCOL_VERSION,
+ type: "episode",
+ episodeId: "ep-1",
+ state: "waiting",
+ tier: 1,
+ providerDisplayName: "Openai-codex",
+ host: "chatgpt.com",
+ endpoint: "https://chatgpt.com/backend-api/codex/responses",
+ kind: "unreachable",
+ code: "ConnectionClosed",
+ loopback: false,
+ reason:
+ "Cannot reach Openai-codex at https://chatgpt.com/backend-api/codex/responses. Check your network connection.",
+ attempts: 5,
+ clientRetries: 0,
+ startedAtMs: T0,
+ nextAttemptAtMs: T0 + 132_000,
+ lastOutcome: "ConnectionClosed after 11ms",
+ waiters: 2,
+ otherEpisodes: 0,
+};
+
+const view = (over: Partial<PaneViewState> = {}): PaneViewState => ({
+ frame: EPISODE,
+ closed: null,
+ nowMs: T0 + 120_000,
+ lastFrameAtMs: T0 + 120_000,
+ connected: true,
+ protocolMismatch: false,
+ notice: null,
+ ...over,
+});
+
+/** Escapes are the medium, not the message — strip them to read the banner. */
+/** The banner's text, without the paint. Shared with the renderer on purpose:
+ * a second escape-stripper in the test would be a second thing to get wrong. */
+const plain = stripAnsi;
+
+afterEach(() => {
+ // `setThemeMode` flips a PROCESS-GLOBAL palette and Bun runs sibling test
+ // files in one process. Never leave it flipped.
+ resetThemeModeForTests();
+});
+
+describe("renderPane", () => {
+ test("names the provider, the host and the reason — C-3's three facts", () => {
+ const out = plain(renderPane(view(), 100));
+ expect(out).toContain("Openai-codex");
+ expect(out).toContain("chatgpt.com");
+ // The reason is `buildConnectionErrorMessage`'s OWN sentence, not a
+ // paraphrase: if the banner and the inline error ever disagree, the user is
+ // being told two different stories about one fault.
+ expect(out).toContain("Cannot reach Openai-codex at");
+ // The sentence is wrapped to the pane, so read it as one flow.
+ expect(out.replace(/\s+/g, " ")).toContain("Check your network connection.");
+ expect(out).toContain("ConnectionClosed");
+ });
+
+ test("offers both keys and does not promise focus it will not steal", () => {
+ const out = plain(renderPane(view(), 100));
+ expect(out).toContain("[r]");
+ expect(out).toContain("[q]");
+ // magmux forwards clicks to focus a pane. Stealing focus would swallow
+ // keystrokes the user was queueing into Claude Code, which accepts messages
+ // while a turn is in flight — a worse failure than one extra click.
+ expect(out).toContain("click here");
+ });
+
+ test("the countdown DECREASES between two renders — C-4, as a unit", () => {
+ const early = plain(renderPane(view({ nowMs: T0 + 70_000 }), 100));
+ const later = plain(renderPane(view({ nowMs: T0 + 73_000 }), 100));
+ expect(early).toContain("next attempt in 1m 02s");
+ expect(later).toContain("next attempt in 59s");
+ expect(early).not.toBe(later);
+ });
+
+ test("a countdown that has gone past reads 0s, never a negative", () => {
+ expect(plain(renderPane(view({ nowMs: T0 + 200_000 }), 100))).toContain("next attempt in 0s");
+ });
+
+ test("`attempting` shows what we are doing, not a frozen countdown", () => {
+ // A 45-second connect with a countdown stuck at the same number reads as a
+ // hang. This line is why the proxy ticks in every live state.
+ const out = plain(
+ renderPane(view({ frame: { ...EPISODE, state: "attempting", nextAttemptAtMs: null } }), 100)
+ );
+ expect(out).toContain("connecting to chatgpt.com");
+ expect(out).not.toContain("next attempt in");
+ });
+
+ test("`handoff` says the client is expected back, which is the truth", () => {
+ const out = plain(
+ renderPane(
+ view({
+ frame: { ...EPISODE, state: "handoff", nextAttemptAtMs: null, tier: 2, clientRetries: 2 },
+ }),
+ 100
+ )
+ );
+ expect(out).toContain("waiting for Claude Code to retry");
+ expect(out).toContain("client retry 2");
+ });
+
+ test("counters: attempts, elapsed, last outcome, held requests, other episodes", () => {
+ const out = plain(renderPane(view({ frame: { ...EPISODE, otherEpisodes: 1 } }), 100));
+ expect(out).toContain("attempt 5");
+ expect(out).toContain("2m 00s in recovery");
+ expect(out).toContain("last: ConnectionClosed after 11ms");
+ expect(out).toContain("2 requests held");
+ expect(out).toContain("+1 more");
+ });
+
+ test("a loopback endpoint is marked local", () => {
+ const out = plain(renderPane(view({ frame: { ...EPISODE, loopback: true } }), 100));
+ expect(out).toContain("local");
+ });
+
+ test("the dead-proxy notice appears only after real silence", () => {
+ // C-18 asserts this notice is ABSENT for a whole slow-connect run, so it
+ // must key on silence longer than the proxy's own tick and nothing else.
+ const quiet = view({ lastFrameAtMs: T0 + 120_000 - (PROXY_SILENT_MS - 500) });
+ expect(plain(renderPane(quiet, 100))).not.toContain("has not reported");
+ const dead = view({ lastFrameAtMs: T0 + 120_000 - (PROXY_SILENT_MS + 500) });
+ expect(plain(renderPane(dead, 100))).toContain("has not reported");
+ });
+
+ test("a closed episode says how it ended and that the pane will go by itself", () => {
+ const ok = plain(
+ renderPane(
+ view({ closed: { v: 1, type: "closed", episodeId: "ep-1", outcome: "recovered" } }),
+ 100
+ )
+ );
+ expect(ok).toContain("recovered");
+ expect(ok).toContain("this pane closes on its own");
+ const bad = plain(
+ renderPane(
+ view({ closed: { v: 1, type: "closed", episodeId: "ep-1", outcome: "gave_up" } }),
+ 100
+ )
+ );
+ expect(bad).toContain("gave up");
+ });
+
+ test("version skew prints a notice instead of rendering fields it cannot read", () => {
+ const out = plain(renderPane(view({ protocolMismatch: true }), 100));
+ expect(out).toContain("version skew");
+ expect(out).not.toContain("Openai-codex");
+ });
+
+ test("before the first frame it says what it is waiting for", () => {
+ expect(plain(renderPane(view({ frame: null }), 100))).toContain("waiting for the proxy");
+ expect(plain(renderPane(view({ frame: null, connected: false }), 100))).toContain(
+ "connecting to the claudish proxy"
+ );
+ });
+
+ test("every line fits a 40-column pane — a split pane is HALF the terminal", () => {
+ // Measured on magmux 0.11.0: `split:"vertical"` took pane 0 from 40 rows to
+ // 20 and gave the new pane 19. A user with a narrow window gets a narrow
+ // pane, and a banner that runs off it is a banner that cannot be read.
+ const narrow = renderPane(view(), 40)
+ .split("\n")
+ .map((l) => plain(l));
+ for (const line of narrow) expect(line.length).toBeLessThanOrEqual(40);
+ // …and the keys survive the squeeze, because they are the part a user
+ // cannot work out for themselves.
+ const joined = narrow.join(" ");
+ expect(joined).toContain("[r]");
+ expect(joined).toContain("[q]");
+ });
+
+ // ── THE THEMING INVARIANT ────────────────────────────────────────────────
+ test("the palette is read at RENDER time, not at module load", () => {
+ // This module was imported at the top of this file, long before the line
+ // below runs — which is exactly the production ordering: command modules
+ // are imported before theme detection completes. A module-level
+ // `const C = cliAnsi()` would snapshot the dark palette here and never
+ // change again. That bug class has been found six times in this codebase.
+ setThemeMode("dark");
+ const dark = renderPane(view(), 100);
+ setThemeMode("light");
+ const light = renderPane(view(), 100);
+ expect(light).not.toBe(dark);
+ expect(light).toContain("\x1b[38;2;"); // truecolor: the LIGHT palette
+ expect(dark).not.toContain("\x1b[38;2;"); // classic 16-colour: dark/unknown
+ });
+
+ test("the red banner fill is the SAME in both themes — it is theme-independent", () => {
+ // The one element that must be legible whatever happens does not depend on
+ // detection succeeding: a self-contained mid-dark fill with bright-white
+ // ink, the pairing `team-grid.ts` established.
+ const FILL = "\x1b[1;97;48;2;160;50;70m";
+ setThemeMode("dark");
+ expect(renderPane(view(), 100)).toContain(FILL);
+ setThemeMode("light");
+ expect(renderPane(view(), 100)).toContain(FILL);
+ });
+});
+
+describe("formatDuration / wrapText / parsePaneArgs", () => {
+ test("durations", () => {
+ expect(formatDuration(0)).toBe("0s");
+ expect(formatDuration(-5_000)).toBe("0s");
+ expect(formatDuration(47_000)).toBe("47s");
+ expect(formatDuration(132_000)).toBe("2m 12s");
+ });
+
+ test("wrapping never drops or splits a word", () => {
+ const words = "alpha beta gamma delta epsilon".split(" ");
+ const lines = wrapText(words.join(" "), 12);
+ expect(lines.join(" ").split(/\s+/)).toEqual(words);
+ for (const l of lines) expect(l.length).toBeLessThanOrEqual(12);
+ });
+
+ test("--socket parses in both spellings, and is required", () => {
+ expect(parsePaneArgs(["--socket", "/tmp/x/r.sock"]).socketPath).toBe("/tmp/x/r.sock");
+ expect(parsePaneArgs(["--socket=/tmp/y/r.sock"]).socketPath).toBe("/tmp/y/r.sock");
+ expect(parsePaneArgs([]).socketPath).toBeNull();
+ });
+});
+
+// ─── Keys, over a real socket ────────────────────────────────────────────────
+
+class FakeStdin extends EventEmitter {
+ setRawMode() {
+ return this;
+ }
+ resume() {
+ return this;
+ }
+ pause() {
+ return this;
+ }
+}
+class FakeStdout extends EventEmitter {
+ columns = 100;
+ chunks: string[] = [];
+ write(s: string) {
+ this.chunks.push(s);
+ return true;
+ }
+}
+
+describe("the pane's two keys", () => {
+ test("`r`, `R` and Enter each ask for an attempt now; `q` gives up and exits", async () => {
+ const path = recoverySocketPathIn(createRecoverySocketDir());
+ const received: RecoveryCommand[] = [];
+ let server: RecoverySocketServer | null = null;
+ server = await startRecoverySocketServer({
+ socketPath: path,
+ snapshot: () => EPISODE,
+ onCommand: (cmd) => received.push(cmd),
+ tickMs: 0,
+ });
+
+ const stdin = new FakeStdin();
+ const stdout = new FakeStdout();
+ const done = runRecoveryPane({
+ socketPath: path,
+ stdin: stdin as unknown as NodeJS.ReadStream,
+ stdout: stdout as unknown as NodeJS.WriteStream,
+ });
+ await new Promise((r) => setTimeout(r, 150));
+
+ for (const key of ["r", "R", "\r"]) stdin.emit("data", Buffer.from(key));
+ await new Promise((r) => setTimeout(r, 120));
+ expect(received.filter((c) => c.type === "retry_now")).toHaveLength(3);
+ expect(
+ received
+ .filter((c) => c.type === "retry_now")
+ .every((c) => "episodeId" in c && c.episodeId === "ep-1")
+ ).toBe(true);
+
+ stdin.emit("data", Buffer.from("q"));
+ expect(await done).toBe(0);
+ await new Promise((r) => setTimeout(r, 80));
+ expect(received.filter((c) => c.type === "bye")).toHaveLength(1);
+ await server.close();
+ }, 15_000);
+
+ test("the banner is painted from the REPLAYED frame, with no tick at all", async () => {
+ const path = recoverySocketPathIn(createRecoverySocketDir());
+ let server: RecoverySocketServer | null = null;
+ server = await startRecoverySocketServer({
+ socketPath: path,
+ snapshot: () => EPISODE,
+ onCommand: () => {},
+ tickMs: 0,
+ });
+ const stdin = new FakeStdin();
+ const stdout = new FakeStdout();
+ const done = runRecoveryPane({
+ socketPath: path,
+ stdin: stdin as unknown as NodeJS.ReadStream,
+ stdout: stdout as unknown as NodeJS.WriteStream,
+ });
+ await new Promise((r) => setTimeout(r, 150));
+ const painted = plain(stdout.chunks.join(""));
+ expect(painted).toContain("Openai-codex");
+ expect(painted).toContain("chatgpt.com");
+ stdin.emit("data", Buffer.from("q"));
+ await done;
+ await server.close();
+ }, 15_000);
+});
diff --git a/packages/cli/src/recovery/pane-app.ts b/packages/cli/src/recovery/pane-app.ts
new file mode 100644
index 00000000..ff95f971
--- /dev/null
+++ b/packages/cli/src/recovery/pane-app.ts
@@ -0,0 +1,489 @@
+/**
+ * `claudish recovery-pane` — the five-line banner that earns claudish the right
+ * to wait.
+ *
+ * PLAIN ANSI WITH RAW STDIN, NOT OpenTUI. This process exists to paint five
+ * lines and read two keys. claudish already boots OpenTUI three times (config
+ * TUI, probe TUI, resume picker) and a fourth boot for a banner is
+ * disproportionate — it would pay a React reconciler and a renderer handshake
+ * on the critical path of a user who is, by construction, already having a bad
+ * minute. `team-grid.ts`'s self-contained banner is the established precedent
+ * in this repo for exactly this shape.
+ *
+ * THEMING. `cliAnsi()` is called INSIDE `renderPane`, never at module load:
+ * command modules are imported before theme detection runs, so a module-level
+ * `const C = cliAnsi()` snapshots the pre-detection (dark) palette forever.
+ * That bug class has been found six times in this codebase. The banner FILL is
+ * additionally theme-independent by construction — a self-contained mid-dark
+ * red with bright-white ink, the same pairing `team-grid.ts` uses — so the one
+ * element that must be legible no matter what does not depend on detection
+ * succeeding at all. (Measured: magmux DOES answer OSC 11 inside a pane, so
+ * detection works; the banner does not rely on it.)
+ *
+ * WHY A PANE AND NOT AN OVERLAY. Keys typed here reach THIS process's own PTY,
+ * not Claude Code's. That is the whole reason the recovery UI is a pane: it
+ * dissolves the "how does a keystroke reach the proxy" question instead of
+ * working around it. Focus is deliberately NOT stolen — Claude Code accepts
+ * messages while a turn is in flight, and capturing those keystrokes into our
+ * renderer would be a worse failure than one extra click.
+ */
+
+import { cliAnsi } from "../theme/ansi.js";
+import { type RecoverySocketClient, connectRecoverySocket } from "./socket-client.js";
+import {
+ PANE_LINGER_MS,
+ PROXY_SILENT_MS,
+ RECOVERY_PROTOCOL_VERSION,
+ type RecoveryClosedFrame,
+ type RecoveryEpisodeFrame,
+ UI_HEARTBEAT_MS,
+} from "./types.js";
+
+export interface PaneViewState {
+ /** The episode being painted, or null before the first frame. */
+ frame: RecoveryEpisodeFrame | null;
+ /** Set once the episode ends; the pane lingers on this rather than vanishing. */
+ closed: RecoveryClosedFrame | null;
+ /** EPOCH ms — the same clock `nextAttemptAtMs` and `startedAtMs` are in. */
+ nowMs: number;
+ /** EPOCH ms of the last frame of any kind, or null. */
+ lastFrameAtMs: number | null;
+ connected: boolean;
+ protocolMismatch: boolean;
+ /** A transient line, e.g. after `[r]`. */
+ notice: string | null;
+}
+
+function pad(text: string, width: number): string {
+ return text.length >= width ? text.slice(0, width) : text + " ".repeat(width - text.length);
+}
+
+/**
+ * Truncate to a VISIBLE width, stepping over the escapes.
+ *
+ * `String.length` counts an SGR escape as eight or nine characters and would
+ * cut a line in the middle of one, leaving the rest of the pane painted in
+ * whatever colour the fragment happened to start. A split pane is HALF the
+ * terminal — measured: `split:"vertical"` took pane 0 from 40 rows to 20 — so
+ * "the pane is always wide" is not an assumption this can make.
+ */
+/** The SGR sequence starting at `i` (`ESC [ … m`), or null. */
+function matchSgr(text: string, i: number): string | null {
+ if (text[i] !== "\x1b" || text[i + 1] !== "[") return null;
+ let j = i + 2;
+ while (j < text.length) {
+ const c = text.charCodeAt(j);
+ const isParam = (c >= 0x30 && c <= 0x39) || c === 0x3b; // 0-9 ;
+ if (!isParam) break;
+ j++;
+ }
+ return text[j] === "m" ? text.slice(i, j + 1) : null;
+}
+
+/**
+ * Drop every SGR sequence, leaving the text.
+ *
+ * Exported for the tests and for the evidence harness: assertions are about
+ * what a person reads, and a capture full of escapes cannot be read by either.
+ */
+export function stripAnsi(text: string): string {
+ let out = "";
+ for (let i = 0; i < text.length; i++) {
+ const m = matchSgr(text, i);
+ if (m) {
+ i += m.length - 1;
+ continue;
+ }
+ out += text[i];
+ }
+ return out;
+}
+
+export function clipAnsi(text: string, width: number): string {
+ let out = "";
+ let visible = 0;
+ for (let i = 0; i < text.length; i++) {
+ const ch = text[i] as string;
+ if (ch === "\x1b") {
+ // ESC is the subject matter here, so the scan is written by hand rather
+ // than as a regex: a regex literal containing the control character trips
+ // `noControlCharactersInRegex`, and suppressing a lint rule to parse the
+ // one thing this function exists to parse is worse than four lines of
+ // explicit scanning.
+ const m = matchSgr(text, i);
+ if (m) {
+ out += m;
+ i += m.length - 1;
+ continue;
+ }
+ }
+ if (visible >= width) return `${out}\x1b[0m`;
+ out += ch;
+ visible++;
+ }
+ return out;
+}
+
+/** `2m 12s`, `47s`, `0s`. Never negative — a countdown that went past reads `0s`. */
+export function formatDuration(ms: number): string {
+ const total = Math.max(0, Math.round(ms / 1000));
+ if (total < 60) return `${total}s`;
+ const m = Math.floor(total / 60);
+ const s = total % 60;
+ return `${m}m ${String(s).padStart(2, "0")}s`;
+}
+
+/** Greedy wrap. The reason sentence is the one thing that must stay readable. */
+export function wrapText(text: string, width: number): string[] {
+ if (width <= 0) return [text];
+ const words = text.split(/\s+/).filter(Boolean);
+ const lines: string[] = [];
+ let line = "";
+ for (const w of words) {
+ if (!line) {
+ line = w;
+ } else if (line.length + 1 + w.length <= width) {
+ line += ` ${w}`;
+ } else {
+ lines.push(line);
+ line = w;
+ }
+ }
+ if (line) lines.push(line);
+ return lines.length > 0 ? lines : [""];
+}
+
+/**
+ * Build the banner. PURE — no clock, no terminal, no escape-code state.
+ *
+ * Pure because it is the only part of this process worth asserting on in a
+ * unit test, and because the live evidence (a capture of the real pane) and the
+ * test then look at the same function rather than at two implementations that
+ * could drift.
+ */
+export function renderPane(state: PaneViewState, width: number): string {
+ // RENDER TIME. Never at module load.
+ const C = cliAnsi();
+ const w = Math.max(28, Math.min(width || 80, 200));
+ const RED_FILL = "\x1b[1;97;48;2;160;50;70m";
+ const GREEN_FILL = "\x1b[1;97;48;2;30;120;70m";
+ const lines: string[] = [];
+
+ // Every return goes through `finish`, so there is ONE place the pane's width
+ // is enforced and no later branch can forget it.
+ const finish = (ls: string[]) => ls.map((l) => clipAnsi(l, w)).join("\n");
+
+ if (state.protocolMismatch) {
+ return finish([
+ `${RED_FILL}${pad(" ██ CLAUDISH RECOVERY · version skew", w)}\x1b[0m`,
+ `${C.GRAY} This pane speaks protocol v${RECOVERY_PROTOCOL_VERSION}; the proxy does not.${C.RESET}`,
+ `${C.GRAY} Restart the session so both halves are the same build.${C.RESET}`,
+ "",
+ ]);
+ }
+
+ const ep = state.frame;
+ const closed = state.closed;
+
+ if (closed) {
+ const ok = closed.outcome === "recovered";
+ const fill = ok ? GREEN_FILL : RED_FILL;
+ const title = ok
+ ? ` ██ NETWORK · ${ep?.providerDisplayName ?? "provider"} recovered`
+ : ` ██ NETWORK · recovery ended (${closed.outcome.replace(/_/g, " ")})`;
+ lines.push(`${fill}${pad(title, w)}\x1b[0m`);
+ if (ep) {
+ lines.push(
+ `${C.GRAY} ${pad(`${ep.attempts} attempts over ${formatDuration(state.nowMs - ep.startedAtMs)}`, w - 2)}${C.RESET}`
+ );
+ }
+ lines.push(`${C.GRAY} this pane closes on its own${C.RESET}`);
+ return finish(lines);
+ }
+
+ if (!ep) {
+ lines.push(`${RED_FILL}${pad(" ██ NETWORK · claudish is retrying", w)}\x1b[0m`);
+ lines.push(
+ `${C.GRAY} ${state.connected ? "waiting for the proxy to describe the failure…" : "connecting to the claudish proxy…"}${C.RESET}`
+ );
+ return finish(lines);
+ }
+
+ // 1 — the red line. Provider and the failure kind, in the fill that reads on
+ // any page.
+ const kindWord =
+ ep.kind === "dns" ? "DNS failure" : ep.kind === "refused" ? "refused" : "unreachable";
+ lines.push(`${RED_FILL}${pad(` ██ NETWORK · ${ep.providerDisplayName} ${kindWord}`, w)}\x1b[0m`);
+
+ // 2 — the reason, in `buildConnectionErrorMessage`'s OWN words. The same
+ // sentence the inline 400 would have carried; if the two ever diverge the
+ // user is being told two different stories about one fault.
+ for (const l of wrapText(ep.reason, w - 4)) lines.push(`${C.STRONG} ${l}${C.RESET}`);
+
+ // 3 — host and code, as discrete tokens. The sentence above contains the host
+ // too, but embedded in prose; this line is what a capture can be grepped for.
+ lines.push(
+ `${C.GRAY} host ${C.RESET}${C.CYAN}${ep.host}${C.RESET}${C.GRAY} · ${ep.code ?? "no code"}${
+ ep.loopback ? " · local" : ""
+ }${C.RESET}`
+ );
+
+ // 4 — the live line. A countdown while waiting; what we are doing instead
+ // while an attempt is actually in flight, because a frozen countdown during a
+ // 45-second connect reads as a hang.
+ const elapsed = formatDuration(state.nowMs - ep.startedAtMs);
+ const when =
+ ep.state === "waiting" && ep.nextAttemptAtMs !== null
+ ? `${C.YELLOW}next attempt in ${formatDuration(ep.nextAttemptAtMs - state.nowMs)}${C.RESET}`
+ : ep.state === "handoff"
+ ? `${C.YELLOW}waiting for Claude Code to retry${C.RESET}`
+ : `${C.YELLOW}connecting to ${ep.host}…${C.RESET}`;
+ const tierNote = ep.tier === 2 ? ` · client retry ${ep.clientRetries}` : "";
+ lines.push(
+ ` ${when}${C.GRAY} · attempt ${ep.attempts} · ${elapsed} in recovery${tierNote}${C.RESET}`
+ );
+
+ // 5 — last outcome and the other-episode count.
+ const others = ep.otherEpisodes > 0 ? ` · +${ep.otherEpisodes} more` : "";
+ const waiters = ep.waiters === 1 ? "1 request held" : `${ep.waiters} requests held`;
+ lines.push(`${C.GRAY} last: ${ep.lastOutcome} · ${waiters}${others}${C.RESET}`);
+
+ // 6 — the keys. The click instruction is not decoration: magmux forwards
+ // mouse clicks to focus a pane, and focus is not stolen — so "click, then
+ // press" is the actual interaction and the banner has to say so.
+ //
+ // A `split` pane is HALF the terminal (measured: 40 rows became 20 and 19),
+ // and a user with a narrow window gets a narrow pane. Below 64 columns the
+ // full sentence wraps or is cut, so the hint sheds the part a user can work
+ // out — where to click — and keeps the part they cannot: which keys exist.
+ lines.push(
+ w < 64
+ ? `${C.GRAY} ${C.RESET}${C.BOLD}[r]${C.RESET} try now${C.GRAY} · ${C.RESET}${C.BOLD}[q]${C.RESET} give up`
+ : `${C.GRAY} click here or Ctrl-G Tab, then ${C.RESET}${C.BOLD}[r]${C.RESET} try now${C.GRAY} · ${C.RESET}${C.BOLD}[q]${C.RESET} give up`
+ );
+
+ if (state.notice) lines.push(`${C.GREEN} ${state.notice}${C.RESET}`);
+
+ // The dead-proxy notice. Frame silence is the heuristic, which is exactly why
+ // the proxy ticks in EVERY live state — a 45-second connect used to look
+ // identical to a dead proxy.
+ if (
+ state.lastFrameAtMs !== null &&
+ state.nowMs - state.lastFrameAtMs > PROXY_SILENT_MS &&
+ state.connected
+ ) {
+ lines.push(
+ `${C.RED} the claudish proxy has not reported in ${formatDuration(state.nowMs - state.lastFrameAtMs)}${C.RESET}`
+ );
+ }
+ if (!state.connected) {
+ lines.push(`${C.RED} the claudish proxy closed this connection${C.RESET}`);
+ }
+
+ return finish(lines);
+}
+
+export interface RecoveryPaneOptions {
+ socketPath: string;
+ stdin?: NodeJS.ReadStream;
+ stdout?: NodeJS.WriteStream;
+ /** EPOCH ms. Injectable so a test can drive the countdown deterministically. */
+ now?: () => number;
+ /** Stop after this long with no live episode. Tests pass something small. */
+ lingerMs?: number;
+}
+
+/** Parse `--socket <path>` out of a recovery-pane argv slice. */
+export function parsePaneArgs(argv: string[]): { socketPath: string | null } {
+ for (let i = 0; i < argv.length; i++) {
+ const a = argv[i];
+ if (a === "--socket" && argv[i + 1]) return { socketPath: argv[i + 1] as string };
+ if (a?.startsWith("--socket=")) return { socketPath: a.slice("--socket=".length) };
+ }
+ return { socketPath: null };
+}
+
+/**
+ * Run the pane until the user quits, the episode ends and the linger expires,
+ * or the proxy goes away. Resolves with the process exit code.
+ */
+export async function runRecoveryPane(opts: RecoveryPaneOptions): Promise<number> {
+ const stdin = opts.stdin ?? process.stdin;
+ const stdout = opts.stdout ?? process.stdout;
+ const now = opts.now ?? (() => Date.now());
+ const lingerMs = opts.lingerMs ?? PANE_LINGER_MS;
+
+ const state: PaneViewState = {
+ frame: null,
+ closed: null,
+ nowMs: now(),
+ lastFrameAtMs: null,
+ connected: true,
+ protocolMismatch: false,
+ notice: null,
+ };
+
+ let noticeUntilMs = 0;
+ let closedAtMs: number | null = null;
+ /** The episode this renderer is heartbeating. Exactly one at a time. */
+ let paintedEpisodeId: string | null = null;
+ let client: RecoverySocketClient | null = null;
+
+ const paint = () => {
+ state.nowMs = now();
+ if (state.notice && state.nowMs > noticeUntilMs) state.notice = null;
+ const width = stdout.columns ?? 80;
+ const body = renderPane(state, width);
+ // Home, repaint each line clearing to EOL, then clear the rest. No
+ // alternate screen: the pane's content stays in magmux's scrollback, which
+ // is where a user looks after the fact to find out what happened.
+ const painted = body.split("\n").join("\x1b[K\n");
+ stdout.write(`\x1b[H${painted}\x1b[K\x1b[J`);
+
+ // THE LEASE. Sent after painting, and on this timer rather than on frame
+ // arrival — the proxy has nothing to say for the 20–75 s an unreachable
+ // connect takes, and a lease renewed by frames would expire in exactly the
+ // failure class this feature exists for.
+ if (paintedEpisodeId && client?.connected) {
+ client.send({
+ v: RECOVERY_PROTOCOL_VERSION,
+ type: "ack",
+ episodeId: paintedEpisodeId,
+ paintedAt: state.nowMs,
+ });
+ }
+ };
+
+ stdout.write("\x1b[?25l"); // hide the cursor; there is nothing to type into
+ const restore = () => {
+ try {
+ stdout.write("\x1b[?25h\n");
+ stdin.setRawMode?.(false);
+ stdin.pause();
+ } catch {
+ /* the terminal went away first */
+ }
+ };
+
+ client = await connectRecoverySocket({
+ socketPath: opts.socketPath,
+ onFrame: (frame) => {
+ state.lastFrameAtMs = now();
+ if (frame.type === "closed") {
+ if (frame.outcome === "protocol_mismatch") {
+ state.protocolMismatch = true;
+ paint();
+ return;
+ }
+ if (!state.frame || frame.episodeId === state.frame.episodeId) {
+ state.closed = frame;
+ closedAtMs = now();
+ // Stop heartbeating: this episode is over, and a lease that outlived
+ // its episode is the latch this design replaced.
+ paintedEpisodeId = null;
+ }
+ paint();
+ return;
+ }
+ // A new episode supersedes the old one. The renderer heartbeats exactly
+ // one episode; the previous one's lease lapses on its own within
+ // UI_LEASE_MS with no extra protocol.
+ state.frame = frame;
+ state.closed = null;
+ closedAtMs = null;
+ paintedEpisodeId = frame.episodeId;
+ paint();
+ },
+ onClose: () => {
+ state.connected = false;
+ paint();
+ },
+ });
+
+ if (!client) {
+ state.connected = false;
+ paint();
+ restore();
+ return 1;
+ }
+
+ let done: (code: number) => void = () => {};
+ const finished = new Promise<number>((resolve) => {
+ done = resolve;
+ });
+
+ // Raw stdin: the pane's two keys must not need a newline, and the terminal
+ // must not echo them into the banner.
+ try {
+ stdin.setRawMode?.(true);
+ } catch {
+ /* not a TTY — the banner still renders, the keys simply do nothing */
+ }
+ // Outside the try: a stdin that cannot go raw can still deliver bytes, and
+ // resume() is also what keeps this process's event loop alive.
+ stdin.resume();
+ stdin.on("data", (buf: Buffer) => {
+ const s = buf.toString("utf-8");
+ for (const ch of s) {
+ if (ch === "\x03") {
+ // Ctrl-C in this pane means the same as [q]: the user asked to stop.
+ client?.send({ v: RECOVERY_PROTOCOL_VERSION, type: "bye", reason: "user_quit" });
+ restore();
+ done(0);
+ return;
+ }
+ if (ch === "q" || ch === "Q") {
+ client?.send({ v: RECOVERY_PROTOCOL_VERSION, type: "bye", reason: "user_quit" });
+ restore();
+ done(0);
+ return;
+ }
+ if (ch === "r" || ch === "R" || ch === "\r" || ch === "\n") {
+ if (state.frame && client?.connected) {
+ client.send({
+ v: RECOVERY_PROTOCOL_VERSION,
+ type: "retry_now",
+ episodeId: state.frame.episodeId,
+ });
+ state.notice = "retrying now…";
+ noticeUntilMs = now() + 3_000;
+ paint();
+ }
+ }
+ }
+ });
+
+ // ONE timer drives both the countdown and the heartbeat, at UI_HEARTBEAT_MS.
+ // They are the same act — "I am still painting this episode" — and splitting
+ // them into two timers would let a frozen renderer keep its lease.
+ const ticker = setInterval(() => {
+ paint();
+ if (closedAtMs !== null && now() - closedAtMs > lingerMs) {
+ restore();
+ done(0);
+ }
+ if (!state.connected && client && !client.connected) {
+ // The proxy is gone. Linger briefly so the last state is readable, then
+ // leave rather than sitting on a dead socket forever.
+ if (closedAtMs === null) closedAtMs = now();
+ }
+ }, UI_HEARTBEAT_MS);
+ ticker.unref?.();
+
+ paint();
+ const code = await finished;
+ clearInterval(ticker);
+ client.close();
+ return code;
+}
+
+/** The `claudish recovery-pane` entry point. */
+export async function recoveryPaneCommand(argv: string[]): Promise<void> {
+ const { socketPath } = parsePaneArgs(argv);
+ if (!socketPath) {
+ console.error("Usage: claudish recovery-pane --socket <path>");
+ process.exit(2);
+ }
+ const code = await runRecoveryPane({ socketPath });
+ process.exit(code);
+}
diff --git a/packages/cli/src/recovery/socket-client.ts b/packages/cli/src/recovery/socket-client.ts
new file mode 100644
index 00000000..14e8683d
--- /dev/null
+++ b/packages/cli/src/recovery/socket-client.ts
@@ -0,0 +1,130 @@
+/**
+ * The pane half of the recovery link.
+ *
+ * Shaped after `team-grid.ts`'s `subscribeToMagmux` — the same NDJSON framing,
+ * the same tolerance of malformed lines, and above all the same CONNECT-RETRY
+ * LOOP, because the socket appears asynchronously relative to the process that
+ * dials it. There the race is magmux creating its socket after spawn; here it
+ * is the pane starting ~100 ms after `open_pane` while the proxy is still
+ * binding, or — in the normal case — long after. Either way a single
+ * `connect()` that fails is indistinguishable from a proxy that is not coming,
+ * and the difference is the whole user experience.
+ */
+
+import { existsSync } from "node:fs";
+import { type Socket, connect as netConnect } from "node:net";
+import {
+ RECOVERY_PROTOCOL_VERSION,
+ type RecoveryCommand,
+ type RecoveryFrame,
+ encodeLine,
+ parseFrame,
+} from "./types.js";
+
+export interface RecoverySocketClientOptions {
+ socketPath: string;
+ onFrame: (frame: RecoveryFrame) => void;
+ /** The proxy hung up, or was never there. */
+ onClose?: () => void;
+ /** Total connect attempts before giving up. 40 × 50 ms ≈ 2 s, as in team-grid. */
+ attempts?: number;
+ retryDelayMs?: number;
+}
+
+export interface RecoverySocketClient {
+ send(cmd: RecoveryCommand): void;
+ close(): void;
+ readonly connected: boolean;
+}
+
+/**
+ * Dial the recovery socket, retrying while it does not yet exist.
+ *
+ * Resolves null when every attempt failed — the caller decides what to print.
+ * It never throws: a pane that crashes on a missing socket replaces a banner
+ * with a stack trace in the middle of the user's screen.
+ */
+export async function connectRecoverySocket(
+ opts: RecoverySocketClientOptions
+): Promise<RecoverySocketClient | null> {
+ const attempts = opts.attempts ?? 40;
+ const delay = opts.retryDelayMs ?? 50;
+ let socket: Socket | null = null;
+
+ for (let attempt = 0; attempt < attempts; attempt++) {
+ if (existsSync(opts.socketPath)) {
+ try {
+ socket = await new Promise<Socket>((resolve, reject) => {
+ const s = netConnect(opts.socketPath);
+ s.once("connect", () => resolve(s));
+ s.once("error", reject);
+ });
+ break;
+ } catch {
+ /* not ready yet — the server binds after mkdir, so the file can exist
+ for a few microseconds before anything is listening on it */
+ }
+ }
+ await new Promise((r) => setTimeout(r, delay));
+ }
+
+ if (!socket) return null;
+
+ const live = socket;
+ let open = true;
+ let buf = "";
+
+ live.on("data", (chunk: Buffer) => {
+ buf += chunk.toString("utf-8");
+ if (buf.length > 256 * 1024) buf = buf.slice(-4096);
+ let nl = buf.indexOf("\n");
+ while (nl >= 0) {
+ const line = buf.slice(0, nl).trim();
+ buf = buf.slice(nl + 1);
+ nl = buf.indexOf("\n");
+ if (!line) continue;
+ const frame = parseFrame(line);
+ if (frame) opts.onFrame(frame);
+ }
+ });
+
+ const gone = () => {
+ if (!open) return;
+ open = false;
+ opts.onClose?.();
+ };
+ live.on("error", gone);
+ live.on("close", gone);
+ live.on("end", gone);
+
+ const client: RecoverySocketClient = {
+ get connected() {
+ return open;
+ },
+ send(cmd: RecoveryCommand) {
+ if (!open || live.destroyed) return;
+ try {
+ live.write(encodeLine(cmd));
+ } catch {
+ /* the proxy went away; `gone` will fire */
+ }
+ },
+ close() {
+ open = false;
+ try {
+ live.end();
+ } catch {
+ /* already gone */
+ }
+ },
+ };
+
+ client.send({
+ v: RECOVERY_PROTOCOL_VERSION,
+ type: "hello",
+ protocol: RECOVERY_PROTOCOL_VERSION,
+ pid: process.pid,
+ });
+
+ return client;
+}
diff --git a/packages/cli/src/recovery/socket-server.test.ts b/packages/cli/src/recovery/socket-server.test.ts
new file mode 100644
index 00000000..fa7d5aaf
--- /dev/null
+++ b/packages/cli/src/recovery/socket-server.test.ts
@@ -0,0 +1,234 @@
+/**
+ * The recovery socket, over a REAL unix socket with REAL clients.
+ *
+ * No `mock.module()` anywhere: this repo's standing rule is that mocking shared
+ * infrastructure bleeds across sibling test files in Bun's module registry, and
+ * `node:net` is about as shared as infrastructure gets. The socket is cheap
+ * enough to use for real.
+ */
+
+import { afterEach, describe, expect, test } from "bun:test";
+import { existsSync, statSync } from "node:fs";
+import { connect } from "node:net";
+import { dirname } from "node:path";
+import {
+ type RecoverySocketServer,
+ createRecoverySocketDir,
+ recoverySocketPathIn,
+ startRecoverySocketServer,
+} from "./socket-server.js";
+import {
+ RECOVERY_PROTOCOL_VERSION,
+ type RecoveryCommand,
+ type RecoveryEpisodeFrame,
+ type RecoveryFrame,
+ encodeLine,
+ parseFrame,
+} from "./types.js";
+
+const EPISODE: RecoveryEpisodeFrame = {
+ v: RECOVERY_PROTOCOL_VERSION,
+ type: "episode",
+ episodeId: "ep-replay",
+ state: "attempting",
+ tier: 1,
+ providerDisplayName: "Openai-codex",
+ host: "chatgpt.com",
+ endpoint: "https://chatgpt.com/backend-api/codex/responses",
+ kind: "unreachable",
+ code: "ConnectionClosed",
+ loopback: false,
+ reason: "Cannot reach Openai-codex at https://chatgpt.com/backend-api/codex/responses.",
+ attempts: 3,
+ clientRetries: 0,
+ startedAtMs: 1_700_000_000_000,
+ nextAttemptAtMs: null,
+ lastOutcome: "ConnectionClosed after 11ms",
+ waiters: 1,
+ otherEpisodes: 0,
+};
+
+/** A raw NDJSON client, so the test exercises the same bytes the pane does. */
+function rawClient(path: string): Promise<{
+ frames: RecoveryFrame[];
+ send: (line: string) => void;
+ end: () => void;
+}> {
+ return new Promise((resolve, reject) => {
+ const s = connect(path);
+ const frames: RecoveryFrame[] = [];
+ let buf = "";
+ s.on("data", (chunk: Buffer) => {
+ buf += chunk.toString();
+ let nl = buf.indexOf("\n");
+ while (nl >= 0) {
+ const line = buf.slice(0, nl);
+ buf = buf.slice(nl + 1);
+ nl = buf.indexOf("\n");
+ const f = parseFrame(line);
+ if (f) frames.push(f);
+ }
+ });
+ s.once("connect", () =>
+ resolve({
+ frames,
+ send: (line: string) => s.write(line),
+ end: () => s.destroy(),
+ })
+ );
+ s.once("error", reject);
+ });
+}
+
+const settle = (ms = 60) => new Promise((r) => setTimeout(r, ms));
+
+let server: RecoverySocketServer | null = null;
+
+afterEach(async () => {
+ await server?.close();
+ server = null;
+});
+
+describe("recovery socket server", () => {
+ test("replays the current episode to a client that connects LATE", async () => {
+ // The pane starts ~100ms after `open_pane`, which is always after the
+ // episode began. Without a replay its first paint is blank, and that blank
+ // is the first thing a user ever sees of this feature.
+ const path = recoverySocketPathIn(createRecoverySocketDir());
+ server = await startRecoverySocketServer({
+ socketPath: path,
+ snapshot: () => EPISODE,
+ onCommand: () => {},
+ tickMs: 0,
+ });
+
+ const client = await rawClient(path);
+ await settle();
+ expect(client.frames).toHaveLength(1);
+ expect(client.frames[0]).toEqual(EPISODE);
+ client.end();
+ });
+
+ test("ticks in EVERY live state, including `attempting` with no state change", async () => {
+ // The state below never changes and no broadcast is ever called. Frames
+ // must arrive anyway: a 45-second connect is 45 seconds of silence
+ // otherwise, which looks identical to a dead proxy to the pane's own
+ // liveness heuristic.
+ const path = recoverySocketPathIn(createRecoverySocketDir());
+ server = await startRecoverySocketServer({
+ socketPath: path,
+ snapshot: () => ({ ...EPISODE, state: "attempting", nextAttemptAtMs: null }),
+ onCommand: () => {},
+ tickMs: 25,
+ });
+ const client = await rawClient(path);
+ await settle(180);
+ // 1 replay + several ticks.
+ expect(client.frames.length).toBeGreaterThanOrEqual(4);
+ expect(client.frames.every((f) => f.type === "episode" && f.state === "attempting")).toBe(true);
+ client.end();
+ });
+
+ test("a null snapshot suppresses the replay and the tick rather than sending junk", async () => {
+ const path = recoverySocketPathIn(createRecoverySocketDir());
+ server = await startRecoverySocketServer({
+ socketPath: path,
+ snapshot: () => null,
+ onCommand: () => {},
+ tickMs: 25,
+ });
+ const client = await rawClient(path);
+ await settle(120);
+ expect(client.frames).toHaveLength(0);
+ client.end();
+ });
+
+ test("routes ack, retry_now and bye; ignores malformed and skewed lines", async () => {
+ const path = recoverySocketPathIn(createRecoverySocketDir());
+ const seen: RecoveryCommand[] = [];
+ server = await startRecoverySocketServer({
+ socketPath: path,
+ snapshot: () => null,
+ onCommand: (cmd) => seen.push(cmd),
+ tickMs: 0,
+ });
+ const client = await rawClient(path);
+ client.send(
+ encodeLine({ v: 1, type: "ack", episodeId: "ep-replay", paintedAt: 1 }) +
+ encodeLine({ v: 1, type: "retry_now", episodeId: "ep-replay" }) +
+ "{bad json}\n" +
+ `${JSON.stringify({ v: 9, type: "ack", episodeId: "ep-replay" })}\n` +
+ encodeLine({ v: 1, type: "bye", reason: "user_quit" })
+ );
+ await settle();
+ expect(seen.map((c) => c.type)).toEqual(["ack", "retry_now", "bye"]);
+ client.end();
+ });
+
+ test("a client on the WRONG protocol is told so and dropped", async () => {
+ const path = recoverySocketPathIn(createRecoverySocketDir());
+ const seen: RecoveryCommand[] = [];
+ server = await startRecoverySocketServer({
+ socketPath: path,
+ snapshot: () => null,
+ onCommand: (cmd) => seen.push(cmd),
+ tickMs: 0,
+ });
+ const client = await rawClient(path);
+ client.send(`${JSON.stringify({ v: 9, type: "hello", protocol: 9, pid: 1 })}\n`);
+ await settle();
+ expect(client.frames).toHaveLength(1);
+ expect(client.frames[0]).toMatchObject({ type: "closed", outcome: "protocol_mismatch" });
+ // And the command is NOT delivered — a build we cannot talk to must not be
+ // able to drive the ladder.
+ expect(seen).toHaveLength(0);
+ });
+
+ test("the directory is 0700, the socket 0600, and both are gone after close", async () => {
+ const dir = createRecoverySocketDir();
+ const path = recoverySocketPathIn(dir);
+ server = await startRecoverySocketServer({
+ socketPath: path,
+ snapshot: () => null,
+ onCommand: () => {},
+ tickMs: 0,
+ });
+ expect(statSync(dirname(path)).mode & 0o777).toBe(0o700);
+ expect(statSync(path).mode & 0o777).toBe(0o600);
+ await server.close();
+ server = null;
+ expect(existsSync(path)).toBe(false);
+ expect(existsSync(dir)).toBe(false);
+ });
+
+ test("two directories from two calls never collide", async () => {
+ const a = createRecoverySocketDir();
+ const b = createRecoverySocketDir();
+ expect(a).not.toBe(b);
+ // 24 hex characters of randomness — not the proxy port, which is printed at
+ // startup and would make the path guessable by something not looking for us.
+ expect(a).toMatch(/^\/tmp\/claudish-recovery-[0-9a-f]{24}$/);
+ });
+
+ test("disconnects are reported, and broadcasting afterwards does not throw", async () => {
+ const path = recoverySocketPathIn(createRecoverySocketDir());
+ let gone = 0;
+ server = await startRecoverySocketServer({
+ socketPath: path,
+ snapshot: () => null,
+ onCommand: () => {},
+ onDisconnect: () => {
+ gone++;
+ },
+ tickMs: 0,
+ });
+ const client = await rawClient(path);
+ await settle();
+ expect(server.clientCount()).toBe(1);
+ client.end();
+ await settle();
+ expect(gone).toBe(1);
+ expect(server.clientCount()).toBe(0);
+ server.broadcast(EPISODE); // must be a no-op, not a throw
+ });
+});
diff --git a/packages/cli/src/recovery/socket-server.ts b/packages/cli/src/recovery/socket-server.ts
new file mode 100644
index 00000000..91374b1b
--- /dev/null
+++ b/packages/cli/src/recovery/socket-server.ts
@@ -0,0 +1,266 @@
+/**
+ * The proxy half of the recovery link: an NDJSON server on a unix socket.
+ *
+ * WHY A SOCKET AND NOT THE MAGMUX CHANNEL. magmux's `send` writes KEYSTROKES
+ * into a pane, and the receiving program reads them as user input. Using it as
+ * a data bus would make the renderer parse one stream for both proxy state and
+ * the user's `[r]`/`[q]` — two protocols on one fd, with this repo's own
+ * paste-detection trap waiting at the bottom of it.
+ *
+ * WHY NOT A FILE. No poller exists to read one (`docs/usage/magmux.md`'s 500 ms
+ * status-bar poller was removed in `168c814` and the docs were never
+ * corrected), building one buys 500 ms granularity and whole-file rewrite
+ * races, and decisively there is no upstream path for `[r] try now`.
+ *
+ * WHY `/tmp` AND NOT `~/.claudish`. macOS `sun_path` is 104 bytes, `$TMPDIR` on
+ * macOS is `/var/folders/<28 chars>/T/`, and `~` is longer still for many
+ * users. `/tmp` is also magmux's own convention for its control socket.
+ *
+ * WHAT THIS DEFENDS AND WHAT IT DOES NOT. The directory is 0700 with an
+ * unguessable name and the socket is 0600, which excludes other uids. It is NOT
+ * a boundary against a SAME-UID process and no unix socket can be: that process
+ * can read our argv, read the pane's argv, or attach a debugger. Saying
+ * otherwise would be theatre. What is defended is the consequence — a forged
+ * client cannot manufacture the one forbidden state (a retryable status with no
+ * banner), because the lease additionally requires that the proxy ITSELF opened
+ * a pane, which is a fact no client can cause.
+ */
+
+import { randomBytes } from "node:crypto";
+import { chmodSync, existsSync, mkdirSync, rmSync, unlinkSync } from "node:fs";
+import { type Server, type Socket, createServer } from "node:net";
+import { dirname, join } from "node:path";
+import { log } from "../logger.js";
+import {
+ FRAME_TICK_MS,
+ RECOVERY_PROTOCOL_VERSION,
+ type RecoveryCommand,
+ type RecoveryEpisodeFrame,
+ type RecoveryFrame,
+ encodeLine,
+ parseCommand,
+} from "./types.js";
+
+export interface RecoverySocketClientHandle {
+ readonly id: number;
+ /** The pid the client claimed in its `hello`, once it has sent one. */
+ pid: number | null;
+ send(frame: RecoveryFrame): void;
+ close(): void;
+}
+
+export interface RecoverySocketServerOptions {
+ socketPath: string;
+ /**
+ * The frame a NEWLY CONNECTED client must see immediately, and the frame the
+ * tick broadcasts. Null means there is nothing live to paint.
+ *
+ * REPLAY-ON-CONNECT is what makes a late pane correct rather than blank. The
+ * pane starts roughly 100 ms after `open_pane` and the episode began before
+ * that; without a replay its first paint would be empty until the next tick,
+ * which is the first thing a user sees of this feature.
+ */
+ snapshot: () => RecoveryEpisodeFrame | null;
+ onCommand: (cmd: RecoveryCommand, client: RecoverySocketClientHandle) => void;
+ onDisconnect?: (client: RecoverySocketClientHandle) => void;
+ /**
+ * Broadcast cadence. A tick fires in EVERY live state, not only while
+ * `waiting`: the banner has to say "connecting…" during a 45 s attempt, and
+ * the pane's dead-proxy heuristic is frame silence, so a quiet `attempting`
+ * phase would make a healthy proxy look dead to its own renderer.
+ */
+ tickMs?: number;
+}
+
+export interface RecoverySocketServer {
+ readonly path: string;
+ broadcast(frame: RecoveryFrame): void;
+ clientCount(): number;
+ close(): Promise<void>;
+}
+
+/**
+ * A private directory for one process's recovery socket.
+ *
+ * The name carries 96 bits of randomness rather than the proxy port, which is
+ * printed at startup and would make the path guessable — and while a same-uid
+ * process can find it anyway, an unguessable name costs one line and removes
+ * the only version of this that is trivially findable by something that is NOT
+ * looking for us.
+ */
+export function createRecoverySocketDir(): string {
+ const dir = join("/tmp", `claudish-recovery-${randomBytes(12).toString("hex")}`);
+ mkdirSync(dir, { mode: 0o700, recursive: true });
+ return dir;
+}
+
+/** The socket path inside a directory from `createRecoverySocketDir()`. */
+export function recoverySocketPathIn(dir: string): string {
+ return join(dir, "r.sock");
+}
+
+/** Remove the socket and, if we made it, its private directory. Never throws. */
+export function cleanupRecoverySocket(socketPath: string): void {
+ try {
+ if (existsSync(socketPath)) unlinkSync(socketPath);
+ } catch {
+ /* already gone */
+ }
+ try {
+ const dir = dirname(socketPath);
+ if (dir.startsWith("/tmp/claudish-recovery-")) rmSync(dir, { recursive: true, force: true });
+ } catch {
+ /* already gone */
+ }
+}
+
+export async function startRecoverySocketServer(
+ opts: RecoverySocketServerOptions
+): Promise<RecoverySocketServer> {
+ const { socketPath } = opts;
+ const tickMs = opts.tickMs ?? FRAME_TICK_MS;
+ const clients = new Map<number, { socket: Socket; handle: RecoverySocketClientHandle }>();
+ let nextId = 1;
+ let closed = false;
+
+ // A stale socket file from a SIGKILLed predecessor makes `listen` fail with
+ // EADDRINUSE even though nothing is listening. The directory name is unique
+ // per process, so anything already at this path is debris by construction.
+ try {
+ if (existsSync(socketPath)) unlinkSync(socketPath);
+ } catch {
+ /* best effort */
+ }
+
+ const server: Server = createServer((socket: Socket) => {
+ if (closed) {
+ socket.destroy();
+ return;
+ }
+ const id = nextId++;
+ const handle: RecoverySocketClientHandle = {
+ id,
+ pid: null,
+ send(frame: RecoveryFrame) {
+ if (socket.destroyed) return;
+ try {
+ socket.write(encodeLine(frame));
+ } catch {
+ /* the pane went away mid-write; the close handler cleans up */
+ }
+ },
+ close() {
+ try {
+ socket.end();
+ } catch {
+ /* already gone */
+ }
+ },
+ };
+ clients.set(id, { socket, handle });
+
+ let buf = "";
+ socket.on("data", (chunk: Buffer) => {
+ buf += chunk.toString("utf-8");
+ // Bound the buffer. A client that never sends a newline must not be able
+ // to grow the proxy's heap without limit.
+ if (buf.length > 64 * 1024) buf = buf.slice(-1024);
+ let nl = buf.indexOf("\n");
+ while (nl >= 0) {
+ const line = buf.slice(0, nl).trim();
+ buf = buf.slice(nl + 1);
+ nl = buf.indexOf("\n");
+ if (!line) continue;
+ const cmd = parseCommand(line);
+ if (!cmd) continue;
+ if (cmd.type === "hello") {
+ handle.pid = typeof cmd.pid === "number" ? cmd.pid : null;
+ if (cmd.protocol !== RECOVERY_PROTOCOL_VERSION) {
+ // Tell it, then drop it. A renderer that cannot read our frames must
+ // print one line about version skew rather than paint garbage.
+ handle.send({
+ v: RECOVERY_PROTOCOL_VERSION,
+ type: "closed",
+ episodeId: "",
+ outcome: "protocol_mismatch",
+ });
+ handle.close();
+ return;
+ }
+ }
+ opts.onCommand(cmd, handle);
+ }
+ });
+
+ const gone = () => {
+ if (!clients.delete(id)) return;
+ opts.onDisconnect?.(handle);
+ };
+ socket.on("error", gone);
+ socket.on("close", gone);
+ socket.on("end", gone);
+
+ // REPLAY. Before anything else this client could possibly ask for.
+ const current = opts.snapshot();
+ if (current) handle.send(current);
+ });
+
+ await new Promise<void>((resolve, reject) => {
+ server.once("error", reject);
+ server.listen(socketPath, () => {
+ server.removeListener("error", reject);
+ resolve();
+ });
+ });
+
+ try {
+ chmodSync(socketPath, 0o600);
+ } catch {
+ /* the 0700 directory is the real gate; a chmod failure does not open it */
+ }
+
+ let ticker: ReturnType<typeof setInterval> | null = null;
+ if (tickMs > 0) {
+ ticker = setInterval(() => {
+ if (closed || clients.size === 0) return;
+ const frame = opts.snapshot();
+ if (!frame) return;
+ for (const { handle } of clients.values()) handle.send(frame);
+ }, tickMs);
+ // The proxy must still be able to exit. A recovery ticker that holds the
+ // event loop open would turn every recovered outage into a hung process.
+ ticker.unref?.();
+ }
+
+ const onExit = () => {
+ cleanupRecoverySocket(socketPath);
+ };
+ process.on("exit", onExit);
+
+ log(`[Recovery] socket listening at ${socketPath}`);
+
+ return {
+ path: socketPath,
+ broadcast(frame: RecoveryFrame) {
+ if (closed) return;
+ for (const { handle } of clients.values()) handle.send(frame);
+ },
+ clientCount: () => clients.size,
+ async close() {
+ if (closed) return;
+ closed = true;
+ if (ticker) clearInterval(ticker);
+ for (const { socket } of clients.values()) {
+ try {
+ socket.destroy();
+ } catch {
+ /* already gone */
+ }
+ }
+ clients.clear();
+ await new Promise<void>((resolve) => server.close(() => resolve()));
+ process.removeListener("exit", onExit);
+ cleanupRecoverySocket(socketPath);
+ },
+ };
+}
diff --git a/packages/cli/src/recovery/types.test.ts b/packages/cli/src/recovery/types.test.ts
new file mode 100644
index 00000000..6a2f8378
--- /dev/null
+++ b/packages/cli/src/recovery/types.test.ts
@@ -0,0 +1,101 @@
+/**
+ * The wire contract.
+ *
+ * Worth its own file because both processes depend on it and NEITHER can
+ * detect a breakage in the other: a proxy that emits a frame the pane silently
+ * drops shows a blank banner, and a pane that sends a command the proxy silently
+ * drops holds no lease. Both failures are silent by construction, so the
+ * parsers' refusals are what has to be pinned.
+ */
+
+import { describe, expect, test } from "bun:test";
+import {
+ RECOVERY_PROTOCOL_VERSION,
+ type RecoveryEpisodeFrame,
+ UI_HEARTBEAT_MS,
+ UI_LEASE_MS,
+ encodeLine,
+ parseCommand,
+ parseFrame,
+} from "./types.js";
+
+const frame = (over: Partial<RecoveryEpisodeFrame> = {}): RecoveryEpisodeFrame => ({
+ v: RECOVERY_PROTOCOL_VERSION,
+ type: "episode",
+ episodeId: "ep-1",
+ state: "waiting",
+ tier: 1,
+ providerDisplayName: "Openai-codex",
+ host: "chatgpt.com",
+ endpoint: "https://chatgpt.com/backend-api/codex/responses",
+ kind: "unreachable",
+ code: "ConnectionClosed",
+ loopback: false,
+ reason: "Cannot reach Openai-codex at https://chatgpt.com/backend-api/codex/responses.",
+ attempts: 5,
+ clientRetries: 0,
+ startedAtMs: 1_789_234_567_890,
+ nextAttemptAtMs: 1_789_234_627_890,
+ lastOutcome: "ConnectionClosed after 11ms",
+ waiters: 2,
+ otherEpisodes: 0,
+ ...over,
+});
+
+describe("recovery wire contract", () => {
+ test("an episode frame round-trips through one NDJSON line", () => {
+ const line = encodeLine(frame());
+ expect(line.endsWith("\n")).toBe(true);
+ expect(line.indexOf("\n")).toBe(line.length - 1); // exactly one, at the end
+ expect(parseFrame(line)).toEqual(frame());
+ });
+
+ test("a frame from a DIFFERENT protocol version is refused, not guessed at", () => {
+ const skewed = encodeLine({ ...frame(), v: 2 });
+ expect(parseFrame(skewed)).toBeNull();
+ });
+
+ test("malformed JSON and unknown types parse to null rather than throwing", () => {
+ expect(parseFrame("{not json")).toBeNull();
+ expect(parseFrame("null")).toBeNull();
+ expect(parseFrame(JSON.stringify({ v: 1, type: "wat", episodeId: "x" }))).toBeNull();
+ expect(parseFrame(JSON.stringify({ v: 1, type: "episode" }))).toBeNull(); // no episodeId
+ });
+
+ test("`hello` parses even at the WRONG version — the mismatch must be answerable", () => {
+ // The server cannot tell a skewed client it is skewed if the one message it
+ // needs to answer is the one it drops.
+ const cmd = parseCommand(JSON.stringify({ v: 99, type: "hello", protocol: 99, pid: 7 }));
+ expect(cmd).not.toBeNull();
+ expect(cmd?.type).toBe("hello");
+ });
+
+ test("every OTHER command is version-gated", () => {
+ expect(parseCommand(JSON.stringify({ v: 2, type: "ack", episodeId: "ep-1" }))).toBeNull();
+ expect(parseCommand(JSON.stringify({ v: 2, type: "retry_now", episodeId: "ep-1" }))).toBeNull();
+ expect(parseCommand(JSON.stringify({ v: 2, type: "bye", reason: "user_quit" }))).toBeNull();
+ });
+
+ test("ack, retry_now and bye parse at the right version", () => {
+ expect(
+ parseCommand(JSON.stringify({ v: 1, type: "ack", episodeId: "ep-1", paintedAt: 5 }))
+ ).toEqual({ v: 1, type: "ack", episodeId: "ep-1", paintedAt: 5 });
+ expect(parseCommand(JSON.stringify({ v: 1, type: "retry_now", episodeId: "ep-1" }))).toEqual({
+ v: 1,
+ type: "retry_now",
+ episodeId: "ep-1",
+ });
+ expect(parseCommand(JSON.stringify({ v: 1, type: "bye", reason: "user_quit" }))).toEqual({
+ v: 1,
+ type: "bye",
+ reason: "user_quit",
+ });
+ });
+
+ test("the heartbeat leaves an order of magnitude of headroom under the lease", () => {
+ // Nine consecutive dropped heartbeats must be survivable. If these two ever
+ // come within 2x of each other, one slow paint revokes a live banner's
+ // lease and the exhaustion arm answers as though nothing were on screen.
+ expect(UI_LEASE_MS / UI_HEARTBEAT_MS).toBeGreaterThanOrEqual(10);
+ });
+});
diff --git a/packages/cli/src/recovery/types.ts b/packages/cli/src/recovery/types.ts
new file mode 100644
index 00000000..790f2999
--- /dev/null
+++ b/packages/cli/src/recovery/types.ts
@@ -0,0 +1,228 @@
+/**
+ * The recovery wire contract — the ONE module both processes import.
+ *
+ * It has no imports of its own, deliberately. The proxy links it into a Hono
+ * request path that has already loaded the whole adapter stack; the pane links
+ * it into a process whose entire job is to paint five lines and whose startup
+ * cost is on the critical path of a user already having a bad minute. Anything
+ * this file imported would be paid for twice, and the second time for nothing.
+ *
+ * FRAMES ARE FAT ON PURPOSE. Every `episode` frame carries the whole banner —
+ * provider, host, endpoint, reason sentence, counters, the next-attempt instant
+ * — rather than a delta or an id to look up. The pane has nothing to call back
+ * to: there is no request/response direction from renderer to proxy beyond the
+ * three commands below, and adding one would mean the banner could be blank
+ * while a round trip was in flight. A frame is ~400 bytes at 1 Hz.
+ *
+ * EVERY TIMESTAMP ON THE WIRE IS EPOCH MS (`Date.now()`). The two processes
+ * have different `performance.now()` origins, so a process-ms value would
+ * render as an arbitrary number in the pane with no error anywhere. The rule
+ * that makes this checkable: a field crossing this boundary is named `…AtMs`
+ * and is epoch; a field used in an interval comparison is named `…AtPerf` and
+ * never leaves its process.
+ */
+
+/**
+ * Bumped when a frame or command changes shape incompatibly.
+ *
+ * Both processes are the same binary in practice, but not always: a
+ * `team --grid` launches N claudish processes that may be different builds
+ * against one magmux, and a global install can be mid-update. A mismatched
+ * client is told so and prints one line, instead of rendering a banner with
+ * fields it does not understand.
+ */
+export const RECOVERY_PROTOCOL_VERSION = 1;
+
+/** The live states. `recovered`/`abandoned` arrive as a `closed` frame instead. */
+export type RecoveryFrameState = "attempting" | "waiting" | "handoff";
+
+export interface RecoveryEpisodeFrame {
+ v: number;
+ type: "episode";
+ episodeId: string;
+ state: RecoveryFrameState;
+ /** DISPLAY ONLY — the highest tier any waiter in this episode has reached. */
+ tier: 1 | 2;
+ providerDisplayName: string;
+ /** `chatgpt.com` — the host alone, for the banner's first line. */
+ host: string;
+ /** The endpoint that actually failed; the AUTH host on an auth-path failure. */
+ endpoint: string;
+ kind: "dns" | "refused" | "unreachable";
+ code: string | null;
+ loopback: boolean;
+ /** `buildConnectionErrorMessage(...)` verbatim — the SAME sentence the inline 400 carries. */
+ reason: string;
+ attempts: number;
+ clientRetries: number;
+ /** EPOCH ms. */
+ startedAtMs: number;
+ /** EPOCH ms, or null while an attempt is actually in flight. */
+ nextAttemptAtMs: number | null;
+ lastOutcome: string;
+ waiters: number;
+ /** How many OTHER live episodes exist, so the banner can say `+N more`. */
+ otherEpisodes: number;
+}
+
+export type RecoveryClosedOutcome =
+ | "recovered"
+ | "handoff"
+ | "client_gone"
+ | "gave_up"
+ | "grace_expired"
+ | "shutdown"
+ | "protocol_mismatch";
+
+export interface RecoveryClosedFrame {
+ v: number;
+ type: "closed";
+ episodeId: string;
+ outcome: RecoveryClosedOutcome;
+}
+
+export type RecoveryFrame = RecoveryEpisodeFrame | RecoveryClosedFrame;
+
+/** Version negotiation only. It grants NOTHING — see `ack`. */
+export interface RecoveryHelloCommand {
+ v: number;
+ type: "hello";
+ protocol: number;
+ pid: number;
+}
+
+/**
+ * The lease heartbeat, and the single most consequential message in the
+ * protocol.
+ *
+ * It is sent AFTER PAINTING, and then once per heartbeat interval for as long
+ * as the renderer is still painting THIS `episodeId` — whether or not a frame
+ * arrived in the meantime. It is not a frame receipt. Renewing the lease from
+ * frame arrivals ties the renderer's apparent liveness to the PROXY's emission
+ * cadence, and a connect against an unreachable host takes 20–75 s (measured:
+ * `192.0.2.1` = 75 005 ms) during which the proxy has nothing new to say. The
+ * lease would then be invalid at exhaustion for exactly the failure class this
+ * feature exists for, with a live, painted banner on screen.
+ */
+export interface RecoveryAckCommand {
+ v: number;
+ type: "ack";
+ episodeId: string;
+ /** EPOCH ms of the paint this ack attests to. */
+ paintedAt: number;
+}
+
+/** `[r]` — collapse the current wait. Ignored unless it names the live episode. */
+export interface RecoveryRetryNowCommand {
+ v: number;
+ type: "retry_now";
+ episodeId: string;
+}
+
+/** `[q]` — the user said stop. Every live episode gives up and answers inline. */
+export interface RecoveryByeCommand {
+ v: number;
+ type: "bye";
+ reason: string;
+}
+
+export type RecoveryCommand =
+ | RecoveryHelloCommand
+ | RecoveryAckCommand
+ | RecoveryRetryNowCommand
+ | RecoveryByeCommand;
+
+/** One NDJSON line, newline included. */
+export function encodeLine(value: RecoveryFrame | RecoveryCommand): string {
+ return `${JSON.stringify(value)}\n`;
+}
+
+function isRecord(value: unknown): value is Record<string, unknown> {
+ return typeof value === "object" && value !== null;
+}
+
+/**
+ * Parse one NDJSON line into a frame, or null.
+ *
+ * Null for malformed JSON, for an unknown `type`, and for a `v` this build does
+ * not speak — a reader that guesses at an unknown shape is how a version skew
+ * becomes a rendering bug instead of a one-line notice.
+ */
+export function parseFrame(line: string): RecoveryFrame | null {
+ let parsed: unknown;
+ try {
+ parsed = JSON.parse(line);
+ } catch {
+ return null;
+ }
+ if (!isRecord(parsed)) return null;
+ if (parsed.v !== RECOVERY_PROTOCOL_VERSION) return null;
+ if (parsed.type === "episode" && typeof parsed.episodeId === "string") {
+ return parsed as unknown as RecoveryEpisodeFrame;
+ }
+ if (parsed.type === "closed" && typeof parsed.episodeId === "string") {
+ return parsed as unknown as RecoveryClosedFrame;
+ }
+ return null;
+}
+
+/**
+ * Parse one NDJSON line into a command.
+ *
+ * A `hello` with the WRONG protocol still parses — the server has to be able to
+ * tell a mismatched client so, and it cannot do that if the message it needs to
+ * answer is the one it drops. Every other command is version-gated.
+ */
+export function parseCommand(line: string): RecoveryCommand | null {
+ let parsed: unknown;
+ try {
+ parsed = JSON.parse(line);
+ } catch {
+ return null;
+ }
+ if (!isRecord(parsed)) return null;
+ if (parsed.type === "hello" && typeof parsed.protocol === "number") {
+ return parsed as unknown as RecoveryHelloCommand;
+ }
+ if (parsed.v !== RECOVERY_PROTOCOL_VERSION) return null;
+ if (parsed.type === "ack" && typeof parsed.episodeId === "string") {
+ return parsed as unknown as RecoveryAckCommand;
+ }
+ if (parsed.type === "retry_now" && typeof parsed.episodeId === "string") {
+ return parsed as unknown as RecoveryRetryNowCommand;
+ }
+ if (parsed.type === "bye") {
+ return { v: RECOVERY_PROTOCOL_VERSION, type: "bye", reason: String(parsed.reason ?? "") };
+ }
+ return null;
+}
+
+// ─── Timings shared by both processes ────────────────────────────────────────
+
+/**
+ * A lease lives this long after the last heartbeat naming its episode.
+ *
+ * NEVER compare this against a per-attempt connect cap. They measure unrelated
+ * things — one the renderer's liveness, the other a network ceiling — and the
+ * whole fix for the round-2 CRITICAL is that they are independent.
+ */
+export const UI_LEASE_MS = 10_000;
+
+/** The pane heartbeats this often while painting. 10× headroom under the lease. */
+export const UI_HEARTBEAT_MS = 1_000;
+
+/** The proxy emits an `episode` frame this often in EVERY live state. */
+export const FRAME_TICK_MS = 1_000;
+
+/**
+ * No frame for this long and the pane says so.
+ *
+ * Strictly greater than `FRAME_TICK_MS` by enough to survive a scheduler hiccup
+ * and a slow paint: the notice means "the proxy is gone", and saying that while
+ * a healthy proxy is mid-connect is the symptom the every-live-state tick
+ * exists to prevent.
+ */
+export const PROXY_SILENT_MS = 6_000;
+
+/** Keep the pane up this long after the last episode ends, so it cannot flap. */
+export const PANE_LINGER_MS = 30_000;
diff --git a/packages/cli/src/team-grid.ts b/packages/cli/src/team-grid.ts
index f8b472d0..23d81306 100644
--- a/packages/cli/src/team-grid.ts
+++ b/packages/cli/src/team-grid.ts
@@ -1,10 +1,9 @@
import { spawn } from "node:child_process";
-import { execSync } from "node:child_process";
import { existsSync, readFileSync, writeFileSync } from "node:fs";
import { type Socket, connect as netConnect } from "node:net";
-import { dirname, join } from "node:path";
+import { join } from "node:path";
import { setTimeout as wait } from "node:timers/promises";
-import { fileURLToPath } from "node:url";
+import { findMagmuxBinary } from "./launcher/magmux-binary.js";
import { loadConfig, loadLocalConfig } from "./profile-config.js";
import { parseModelSpec } from "./providers/model-parser.js";
import {
@@ -175,53 +174,6 @@ function buildPaneHeader(model: string, prompt: string, bg: string): string {
return lines.join(" ");
}
-// ─── Multiplexer Binary Detection ────────────────────────────────────────────
-
-/**
- * Find the magmux binary. Priority:
- * 1. Bundled magmux (native/magmux-<platform>-<arch>)
- * 2. Platform-specific npm package (@claudish/magmux-<platform>-<arch>)
- * 3. magmux in PATH (e.g. via Homebrew)
- */
-function findMagmuxBinary(): string {
- const thisFile = fileURLToPath(import.meta.url);
- const thisDir = dirname(thisFile);
- const pkgRoot = join(thisDir, "..");
- const platform = process.platform;
- const arch = process.arch;
-
- // 1. Bundled magmux (native/magmux-<platform>-<arch>)
- const bundledMagmux = join(pkgRoot, "native", `magmux-${platform}-${arch}`);
- if (existsSync(bundledMagmux)) return bundledMagmux;
-
- // 2. Platform-specific npm package (@claudish/magmux-<platform>-<arch>)
- // npm installs only the matching platform's optional dep
- try {
- const pkgName = `@claudish/magmux-${platform}-${arch}`;
- // Walk up from this file to find node_modules
- let searchDir = pkgRoot;
- for (let i = 0; i < 5; i++) {
- const candidate = join(searchDir, "node_modules", pkgName, "bin", "magmux");
- if (existsSync(candidate)) return candidate;
- const parent = dirname(searchDir);
- if (parent === searchDir) break;
- searchDir = parent;
- }
- } catch {
- /* not installed */
- }
-
- // 3. magmux in PATH
- try {
- const result = execSync("which magmux", { encoding: "utf-8" }).trim();
- if (result) return result;
- } catch {
- /* not in PATH */
- }
-
- throw new Error("magmux not found. Install it:\n brew install MadAppGang/tap/magmux");
-}
-
// ─── Magmux Event Protocol ───────────────────────────────────────────────────
//
// magmux pushes events over its Unix socket. We care about: