Skip to content

Node-based terminal manager for AI coding agents — tmux-backed terminals and parallel agent sessions as draggable nodes on an infinite pan/zoom canvas. macOS, Linux, and a browser Server Edition.

fix(ssh): re-verify the reverse hook tunnel on a reused master

enes03564e497f63
diff --git a/CLAUDE.md b/CLAUDE.md
index 3b2337c04..9375bdb71 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -1584,6 +1584,30 @@ else, and its context links must keep classifying across restarts).
remote installers at once; a second repair mechanism would be exactly the duplicated rule this
file warns about. It strips only OUR handler out of a definition, so a hook a user hand-merged
beside ours survives.
+- **A reused ControlMaster is not evidence of a live hook tunnel** (issue #735 — remote sessions
+ stuck on **Unknown**, no completion notifications, no unread dots). `connect()`'s reuse branch
+ returned the cached `hookEndpointPath` whenever `ssh -O check` answered, on the written-down
+ assumption that *"a master that answered `-O check` never lost its tunnel"*. That is false, and
+ the mechanism is **our own self-heal**: `childArgs` uses `ControlMaster=auto` + `ControlPersist`
+ precisely so a dead master is rebuilt by the next child command (a status poll, a mirror push, a
+ remote git call) on the same ControlPath. The rebuilt master answers `-O check` — and carries no
+ `-R`, because `RemoteHooks.setup()` is the only caller of `hookForwardArgs` and it runs only on
+ the branch where a master has just come up. The 45 s watchdog then parks on the reuse branch
+ forever. Nothing reports it: the project says `connected`, terminals work, the mirror pushes, and
+ only the hook POSTs die — into a socket file that still EXISTS with nobody listening.
+ **MEASURED on the host that prompted the fix**: 10 per-project hook sockets on disk, exactly ONE
+ with a listener (`ss -lxp`); the dead project's socket answered `curl` exit 7 while that same
+ project's status mirror was being written the same second; **107 of 128** live `nodeterm-rmt`
+ tmux sessions were pinned to that dead endpoint. Sessions are pinned for life
+ (`new-session -A -e …` — tmux ignores `-e` on an existing session), so every one of them stayed
+ dark until the app restarted. The reuse branch now probes the tunnel (`RemoteHooks.tunnelAlive`,
+ one `curl` over the already-multiplexed master) and re-runs the idempotent `setup()` when it does
+ not answer, firing `onTunnelVerified` so the working agents resync. **The retry is backed off**
+ (`tunnel-repair.ts`, pure + tested): the FIRST failure repairs immediately — that is the common
+ case — while a host that can never forward (`AllowStreamLocalForwarding no`, no `curl`, a `$HOME`
+ the validator refuses) settles at one attempt per 15 minutes instead of rewriting every agent's
+ hook config every 45 s. A missing spec answers "not alive" rather than "unknown": nothing of ours
+ is bound, which is a tunnel that cannot deliver.
- **Per-node hook identity** (`src/core/agents/node-auth-*.ts`, `node-token-*.ts`,
`node-identity-policy.ts` — full write-up in **`docs/node-identity.md`**) — the shared bearer proves
"a session on this machine", never *which* session, so every node also gets a capability derived
diff --git a/src/main/remote-ssh/remote-hooks.ts b/src/main/remote-ssh/remote-hooks.ts
index 57fd4d1ae..1163794ff 100644
--- a/src/main/remote-ssh/remote-hooks.ts
+++ b/src/main/remote-ssh/remote-hooks.ts
@@ -110,6 +110,26 @@ export class RemoteHooks {
constructor(private r: RemoteRunner) {}
+ /**
+ * Is THIS project's reverse hook tunnel still answering? (issue #735)
+ *
+ * The connect() reuse branch used to assume a master that answered `-O check` still carried its
+ * `-R` forward. It does not, and the mechanism is our own self-heal: `childArgs` uses
+ * `ControlMaster=auto` + `ControlPersist` precisely so that when a master dies the next child
+ * command rebuilds one on the same ControlPath. That rebuilt master answers `-O check` — but
+ * nothing ever handed it `ssh -O forward -R`, because `setup()` is the only caller of
+ * `hookForwardArgs` and it runs only on the "master just came up" branch.
+ *
+ * A missing spec answers FALSE: it means nothing of ours is bound for this project in this app
+ * run, which is a tunnel that cannot deliver, not an unknown. The caller repairs by re-running
+ * `setup()`, which is idempotent and re-verifies end-to-end.
+ */
+ async tunnelAlive(projectId: string, conn: SshConnection, controlPath: string, token: string): Promise<boolean> {
+ const spec = this.specs.get(projectId)
+ if (!spec || !token) return false
+ return this.verifyTunnel(conn, controlPath, spec.sock, token)
+ }
+
async setup(
projectId: string,
conn: SshConnection,
diff --git a/src/main/remote-ssh/ssh-project.test.ts b/src/main/remote-ssh/ssh-project.test.ts
index 04318bb52..c0107ed6b 100644
--- a/src/main/remote-ssh/ssh-project.test.ts
+++ b/src/main/remote-ssh/ssh-project.test.ts
@@ -1345,14 +1345,16 @@ describe('SshProjectManager', () => {
* connect() takes the ordinary fresh-master path — a genuine establish. */
function makeVerifiedMgr(
onTunnelVerified: (projectId: string, controlPath: string, conn: SshConnection) => void,
- httpCode = '204'
+ /** A THUNK is what lets a test kill the tunnel between two connects — see the repair tests. */
+ httpCode: string | (() => string) = '204'
) {
vi.spyOn(fs, 'mkdir').mockResolvedValue(undefined as never)
vi.spyOn(fs, 'stat').mockRejectedValue(Object.assign(new Error('ENOENT'), { code: 'ENOENT' }))
const run = vi.fn(async (args: string[]) => {
const j = args.join(' ')
if (j.includes('$HOME')) return { code: 0, stdout: '/home/u' }
- if (j.includes('%{http_code}')) return { code: 0, stdout: httpCode }
+ if (j.includes('%{http_code}'))
+ return { code: 0, stdout: typeof httpCode === 'function' ? httpCode() : httpCode }
return { code: 0, stdout: '' }
})
return new SshProjectManager({
@@ -1390,12 +1392,73 @@ describe('SshProjectManager', () => {
expect(homeAtHookTime).toBe('/home/u')
})
- it('does NOT fire on the reuse branch — a live master never lost its tunnel', async () => {
+ it('does NOT fire on the reuse branch while the tunnel still ANSWERS', async () => {
+ // A healthy reuse must stay exactly as cheap and as quiet as it was: one `-O check`, one
+ // tunnel probe, no re-install and no resync.
const onTunnelVerified = vi.fn()
const mgr = makeVerifiedMgr(onTunnelVerified)
await mgr.connect('p1', conn, '/remote/cwd')
onTunnelVerified.mockClear()
- await mgr.connect('p1', conn, '/remote/cwd') // `-O check` answers → early return
+ await mgr.connect('p1', conn, '/remote/cwd')
+ expect(onTunnelVerified).not.toHaveBeenCalled()
+ })
+
+ it('REPAIRS a reused master whose tunnel stopped answering, and resyncs (issue #735)', async () => {
+ // The pin this replaces asserted "a live master never lost its tunnel", which is false and was
+ // the bug: `ControlMaster=auto` + `ControlPersist` mean the next child command rebuilds a dead
+ // master on the same ControlPath, and the rebuilt one carries no `-R`. It answers `-O check`
+ // all the same, so the watchdog parked on this branch forever while every remote hook POST
+ // vanished into a socket file with no listener — nodes stuck on "Unknown", no notifications.
+ //
+ // `deadProbes` models the real sequence exactly: the establish is clean, then the tunnel dies
+ // under a master that still answers `-O check`, the liveness probe finds it, and the rebuild
+ // binds a fresh `-R` which DOES verify. Arming it after the establish matters — the establish
+ // verifies too, and a queue primed up front would be eaten by that instead.
+ const onTunnelVerified = vi.fn()
+ let deadProbes = 0
+ const mgr = makeVerifiedMgr(onTunnelVerified, () => (deadProbes-- > 0 ? '000' : '204'))
+ await mgr.connect('p1', conn, '/remote/cwd')
+ onTunnelVerified.mockClear()
+ deadProbes = 1
+ await mgr.connect('p1', conn, '/remote/cwd')
+ expect(onTunnelVerified).toHaveBeenCalledWith('p1', controlPathFor('p1'), conn)
+ })
+
+ it('rebinds the forward on repair — the endpoint is re-advertised, not merely re-probed', async () => {
+ // The whole failure is a master with no `-R`, so a repair that did not call `-O forward`
+ // would leave every hook POST dying exactly as before while reporting success.
+ let deadProbes = 0
+ const mgr = makeVerifiedMgr(vi.fn(), () => (deadProbes-- > 0 ? '000' : '204'))
+ await mgr.connect('p1', conn, '/remote/cwd')
+ deadProbes = 1
+ const forwards = () =>
+ (mgr as unknown as { r: { run: ReturnType<typeof vi.fn> } }).r.run.mock.calls.filter(
+ (c: unknown[]) => (c[0] as string[]).includes('forward')
+ ).length
+ const afterEstablish = forwards()
+ await mgr.connect('p1', conn, '/remote/cwd')
+ expect(forwards()).toBeGreaterThan(afterEstablish)
+ })
+
+ it('backs off a host that can never forward instead of re-installing every tick', async () => {
+ // `setup()` rewrites the managed hook into every agent's config on the host, and the watchdog
+ // reuses every 45 s. On a host where the tunnel can never bind (sshd
+ // `AllowStreamLocalForwarding no`, no curl) an unthrottled repair would rewrite those files
+ // forever. The FIRST failure still repairs immediately — that is the case this exists for.
+ const onTunnelVerified = vi.fn()
+ const mgr = makeVerifiedMgr(onTunnelVerified, () => '000')
+ const forwards = () =>
+ (mgr as unknown as { r: { run: ReturnType<typeof vi.fn> } }).r.run.mock.calls.filter(
+ (c: unknown[]) => (c[0] as string[]).includes('forward')
+ ).length
+ await mgr.connect('p1', conn, '/remote/cwd')
+ const afterEstablish = forwards()
+ await mgr.connect('p1', conn, '/remote/cwd') // failure #1 → repair attempted
+ const afterFirstRepair = forwards()
+ expect(afterFirstRepair).toBeGreaterThan(afterEstablish)
+ await mgr.connect('p1', conn, '/remote/cwd') // inside the backoff window → no second rebuild
+ await mgr.connect('p1', conn, '/remote/cwd')
+ expect(forwards()).toBe(afterFirstRepair)
expect(onTunnelVerified).not.toHaveBeenCalled()
})
diff --git a/src/main/remote-ssh/ssh-project.ts b/src/main/remote-ssh/ssh-project.ts
index e2421a678..35dbf75be 100644
--- a/src/main/remote-ssh/ssh-project.ts
+++ b/src/main/remote-ssh/ssh-project.ts
@@ -42,6 +42,12 @@ import {
} from '../../core/remote-ssh/control-master'
import { claudeVersionProbeCommand, parseClaudeVersionProbe } from '../../core/remote-ssh/claude-version-probe'
import { RemoteHooks } from './remote-hooks'
+import {
+ recordTunnelRepair,
+ shouldAttemptTunnelRepair,
+ type TunnelRepairState
+} from './tunnel-repair'
+
import { hookServer } from '../../core/agents/hook-server'
import {
nodeIdsForCanvas,
@@ -122,10 +128,13 @@ interface Runners {
/** The last SSH connection just went away through a user-facing disconnect. Production schedules
* the app-private agent's shutdown, which is what "forget the key" actually means. */
onIdle?: () => void
- /** A project's reverse hook tunnel was just VERIFIED on a freshly established master. Production
- * resyncs that project's working agents: hook events lost while the tunnel was down are gone for
- * good, so a node can be stranded at `working` until the 20-minute stale sweep. Deliberately not
- * called on the reuse branch — a master that answered `-O check` never lost its tunnel. The
+ /** A project's reverse hook tunnel was just VERIFIED — on a freshly established master, or by the
+ * reuse branch's repair. Production resyncs that project's working agents: hook events lost while
+ * the tunnel was down are gone for good, so a node can be stranded at `working` until the
+ * 20-minute stale sweep. This used to say it was "deliberately not called on the reuse branch —
+ * a master that answered `-O check` never lost its tunnel", which was FALSE and was issue #735:
+ * `ControlMaster=auto` means a dead master is rebuilt by the next child command, and the rebuilt
+ * one carries no `-R`. It answers `-O check` all the same. The
* `conn` rides along because the resync builds its own remote commands (the host's tmux session
* list, a pane probe) and the alternative — looking the connection back up by control path —
* would add a public accessor for a fact this call site already holds. */
@@ -429,6 +438,64 @@ export class SshProjectManager {
* respawned, 'reconnecting' status so the renderer flow engages). Interval is unref'd so it
* never holds the process open; an empty conns map makes a tick a no-op.
*/
+ /** Consecutive failed tunnel repairs per project, so a host that can never forward is not
+ * re-installed on every watchdog tick. See `tunnel-repair.ts` for why the first one is free. */
+ private tunnelRepair = new Map<string, TunnelRepairState>()
+
+ /**
+ * Re-verify a REUSED master's reverse hook tunnel, and rebuild it if it stopped answering
+ * (issue #735 — remote sessions stuck on "Unknown", no notifications, no unread dots).
+ *
+ * `-O check` answering is not evidence the tunnel is alive, and the reason is our own self-heal:
+ * `childArgs` uses `ControlMaster=auto` + `ControlPersist` on purpose, so when a master dies the
+ * next child command (a status poll, a mirror push, a remote git call) rebuilds a background
+ * master on the same ControlPath. That rebuilt master answers `-O check` — and carries no `-R`,
+ * because `setup()` is the only caller of `hookForwardArgs` and it runs only on the branch where
+ * a master has just come up. The watchdog then lands on the reuse branch forever.
+ *
+ * Nothing reports this. The project says `connected`, terminals work, the mirror pushes; only
+ * the hook POSTs die, into a socket file that still exists with nobody listening. MEASURED on
+ * the host that prompted the fix: 107 of 128 live remote tmux sessions pinned to an endpoint
+ * whose socket answered `curl` exit 7, while that project's status mirror was being written the
+ * same second.
+ *
+ * Best-effort throughout, exactly like the establish path: a failed probe or a failed rebuild
+ * leaves the connection alone and the project keeps working without hooks. It must never cost
+ * the user a connection that is already up.
+ */
+ private async repairHookTunnelIfDead(projectId: string, existing: Conn): Promise<void> {
+ try {
+ const hook = this.r.getHook()
+ // No hook server yet ⇒ nothing to point at, and `setup()` would refuse anyway.
+ if (!hook?.port || !hook.token) return
+ if (await this.remoteHooks.tunnelAlive(projectId, existing.conn, existing.controlPath, hook.token)) {
+ this.tunnelRepair.delete(projectId)
+ return
+ }
+ const now = Date.now()
+ if (!shouldAttemptTunnelRepair(this.tunnelRepair.get(projectId), now)) return
+ const res = await this.remoteHooks.setup(projectId, existing.conn, existing.controlPath, hook)
+ this.tunnelRepair.set(projectId, recordTunnelRepair(this.tunnelRepair.get(projectId), !!res, now))
+ if (!res) return
+ // Ownership re-check: `setup()` is several round-trips, and a disconnect + reconnect inside
+ // that window means this entry is no longer the live one — the same rule the establish path
+ // applies before rebuilding a master. Writing the endpoint onto a superseded entry would
+ // point sessions at a socket belonging to a connection nobody holds.
+ if (this.conns.get(projectId) !== existing) return
+ existing.hookEndpointPath = res.endpointPath
+ // Same contract as the establish path: hook events lost while the tunnel was down are gone
+ // for good, so the working agents need a resync. Fire-and-forget behind a catch — a repair
+ // job must never surface to the user as a dead SSH project.
+ try {
+ this.r.onTunnelVerified?.(projectId, existing.controlPath, existing.conn)
+ } catch {
+ // undecided changes nothing: the stale sweep remains the backstop
+ }
+ } catch {
+ // fail-open: the reuse returns whatever it already had, exactly as before this repair existed
+ }
+ }
+
startWatchdog(intervalMs = MASTER_WATCHDOG_MS): void {
if (this.watchdog) return
this.watchdog = setInterval(() => {
@@ -577,6 +644,8 @@ export class SshProjectManager {
// Keep the remote git cwd current even on an idempotent reuse (the folder may have changed).
// Guard against a later connect without remoteCwd clearing a known cwd.
existing.remoteCwd = remoteCwd ?? existing.remoteCwd
+ // ...and CHECK THE TUNNEL, because `-O check` says nothing about it (issue #735).
+ await this.repairHookTunnelIfDead(projectId, existing)
return {
controlPath: existing.controlPath,
hookEndpointPath: existing.hookEndpointPath,
diff --git a/src/main/remote-ssh/tunnel-repair.test.ts b/src/main/remote-ssh/tunnel-repair.test.ts
new file mode 100644
index 000000000..c47db1bb6
--- /dev/null
+++ b/src/main/remote-ssh/tunnel-repair.test.ts
@@ -0,0 +1,78 @@
+import { describe, expect, it } from 'vitest'
+import {
+ TUNNEL_REPAIR_DELAYS_MS,
+ recordTunnelRepair,
+ shouldAttemptTunnelRepair,
+ tunnelRepairDelayMs
+} from './tunnel-repair'
+
+describe('tunnelRepairDelayMs', () => {
+ it('owes nothing before the first failure', () => {
+ expect(tunnelRepairDelayMs(0)).toBe(0)
+ expect(tunnelRepairDelayMs(-1)).toBe(0)
+ })
+
+ it('grows with consecutive failures and then CLAMPS', () => {
+ expect(tunnelRepairDelayMs(1)).toBe(TUNNEL_REPAIR_DELAYS_MS[0])
+ expect(tunnelRepairDelayMs(2)).toBe(TUNNEL_REPAIR_DELAYS_MS[1])
+ expect(tunnelRepairDelayMs(3)).toBe(TUNNEL_REPAIR_DELAYS_MS[2])
+ // A host that will never forward settles at the last delay rather than growing without bound —
+ // it must keep trying (sshd config changes, curl gets installed), just not every 45 s.
+ expect(tunnelRepairDelayMs(99)).toBe(TUNNEL_REPAIR_DELAYS_MS[TUNNEL_REPAIR_DELAYS_MS.length - 1])
+ })
+
+ it('is monotonic — a later failure never waits LESS than an earlier one', () => {
+ for (let n = 1; n < 10; n++) {
+ expect(tunnelRepairDelayMs(n + 1)).toBeGreaterThanOrEqual(tunnelRepairDelayMs(n))
+ }
+ })
+})
+
+describe('shouldAttemptTunnelRepair', () => {
+ it('the FIRST repair is free', () => {
+ // The asymmetry is the point. A needless repair costs one idempotent re-install; NOT repairing
+ // costs a project whose agents report nothing for as long as it stays connected — on the host
+ // that prompted this, 107 of 128 live sessions.
+ expect(shouldAttemptTunnelRepair(undefined, 1_000)).toBe(true)
+ expect(shouldAttemptTunnelRepair({ failures: 0, lastAttemptAt: 1_000 }, 1_000)).toBe(true)
+ })
+
+ it('refuses inside the backoff window and allows once it has elapsed', () => {
+ const state = { failures: 1, lastAttemptAt: 1_000 }
+ expect(shouldAttemptTunnelRepair(state, 1_000)).toBe(false)
+ expect(shouldAttemptTunnelRepair(state, 1_000 + TUNNEL_REPAIR_DELAYS_MS[0] - 1)).toBe(false)
+ expect(shouldAttemptTunnelRepair(state, 1_000 + TUNNEL_REPAIR_DELAYS_MS[0])).toBe(true)
+ })
+
+ it('a longer streak waits longer', () => {
+ const justPastFirst = 1_000 + TUNNEL_REPAIR_DELAYS_MS[0]
+ expect(shouldAttemptTunnelRepair({ failures: 1, lastAttemptAt: 1_000 }, justPastFirst)).toBe(true)
+ expect(shouldAttemptTunnelRepair({ failures: 2, lastAttemptAt: 1_000 }, justPastFirst)).toBe(false)
+ })
+
+ it('a clock that went BACKWARDS refuses rather than repairing in a loop', () => {
+ // NTP steps and sleep/wake both do this. Refusing is the safe direction: the window simply
+ // takes longer to expire, where the other direction is an unthrottled re-install.
+ expect(shouldAttemptTunnelRepair({ failures: 1, lastAttemptAt: 10_000 }, 1_000)).toBe(false)
+ })
+})
+
+describe('recordTunnelRepair', () => {
+ it('counts consecutive failures', () => {
+ let s = recordTunnelRepair(undefined, false, 100)
+ expect(s).toEqual({ failures: 1, lastAttemptAt: 100 })
+ s = recordTunnelRepair(s, false, 200)
+ expect(s).toEqual({ failures: 2, lastAttemptAt: 200 })
+ })
+
+ it('a SUCCESS resets the streak, so the next outage repairs immediately', () => {
+ const failed = recordTunnelRepair(recordTunnelRepair(undefined, false, 100), false, 200)
+ const ok = recordTunnelRepair(failed, true, 300)
+ expect(ok.failures).toBe(0)
+ expect(shouldAttemptTunnelRepair(ok, 300)).toBe(true)
+ })
+
+ it('stamps the ATTEMPT time, not the outcome time — the window starts when we spent the work', () => {
+ expect(recordTunnelRepair(undefined, false, 4_242).lastAttemptAt).toBe(4_242)
+ })
+})
diff --git a/src/main/remote-ssh/tunnel-repair.ts b/src/main/remote-ssh/tunnel-repair.ts
new file mode 100644
index 000000000..c26b207fb
--- /dev/null
+++ b/src/main/remote-ssh/tunnel-repair.ts
@@ -0,0 +1,61 @@
+/**
+ * When may the connect() reuse branch spend a full `RemoteHooks.setup()` on repairing a reverse
+ * hook tunnel that stopped answering? (issue #735)
+ *
+ * Pure, because the decision is the whole risk. A dead tunnel is silent — the project reports
+ * `connected`, terminals work, the mirror pushes — so the repair has to be automatic; but
+ * `setup()` re-installs the managed hook into every agent's config on the host, and the watchdog
+ * calls connect() every 45 s per project. On a host that can never bind one (sshd with
+ * `AllowStreamLocalForwarding no`, no `curl`, a `$HOME` the validator refuses) an unthrottled
+ * retry would rewrite those config files forever, for a tunnel that is never coming.
+ *
+ * So: the FIRST failure repairs immediately — that is the case this exists for, and it is the
+ * common one (a master died, a child rebuilt it, and the rebuilt one carries no `-R`) — and each
+ * consecutive failure backs off, capped. Success resets. Nothing here decides whether the tunnel
+ * is dead; it only decides whether we are allowed to try again yet.
+ */
+
+/** Backoff after consecutive FAILED repairs. The first entry is the delay after failure #1. */
+export const TUNNEL_REPAIR_DELAYS_MS = [60_000, 300_000, 900_000] as const
+
+export interface TunnelRepairState {
+ /** Consecutive failed repairs. Reset to 0 by a success. */
+ failures: number
+ /** When the last repair was ATTEMPTED (not when it finished). */
+ lastAttemptAt: number
+}
+
+/**
+ * The delay owed after `failures` consecutive failures. Clamped to the last entry, so a host that
+ * will never forward settles at one attempt per 15 minutes rather than one per watchdog tick.
+ */
+export function tunnelRepairDelayMs(failures: number): number {
+ if (failures <= 0) return 0
+ const i = Math.min(failures, TUNNEL_REPAIR_DELAYS_MS.length) - 1
+ return TUNNEL_REPAIR_DELAYS_MS[i]
+}
+
+/**
+ * May we attempt a repair now?
+ *
+ * `undefined` state = nothing has failed yet for this project ⇒ yes, immediately. That is the
+ * deliberate asymmetry: the cost of a needless repair is one idempotent re-install, while the cost
+ * of NOT repairing is a project whose agents report nothing at all for as long as it stays
+ * connected — which on the host that prompted this was 107 of 128 live sessions.
+ */
+export function shouldAttemptTunnelRepair(
+ state: TunnelRepairState | undefined,
+ now: number
+): boolean {
+ if (!state || state.failures <= 0) return true
+ return now - state.lastAttemptAt >= tunnelRepairDelayMs(state.failures)
+}
+
+/** Fold an attempt's outcome into the state a later `shouldAttemptTunnelRepair` will read. */
+export function recordTunnelRepair(
+ state: TunnelRepairState | undefined,
+ ok: boolean,
+ now: number
+): TunnelRepairState {
+ return { failures: ok ? 0 : (state?.failures ?? 0) + 1, lastAttemptAt: now }
+}