Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

fix: retry grid attach on reconnect, and stop telling users to restart

Kelvinb9c6f059ca8c
diff --git a/cli/src/cli.ts b/cli/src/cli.ts
index d498c04d8..ca108e197 100644
--- a/cli/src/cli.ts
+++ b/cli/src/cli.ts
@@ -281,10 +281,14 @@ const CONNECT_WAIT_MS = 10_000
/** Bounds the `POST /api/grid/name` inside the daemon-start grid reconcile, so a stalled control-plane
* connection cannot hold it open. */
const GRID_MINT_TIMEOUT_MS = 10_000
-/** How long the grid RPCs will gate on the daemon-start reconcile before answering without it,
- * whether or not it has settled — the ceiling that keeps a stuck reconcile from degrading every
- * grid RPC for the daemon's whole life (`gridReadyProbe`). */
+/** How long the grid RPCs will gate on a grid-attach attempt before answering without it, whether or
+ * not it has settled — the ceiling that keeps a stuck attempt from degrading every grid RPC for the
+ * daemon's whole life (`gridReadyProbe`). */
const GRID_ATTACH_CEILING_MS = 30_000
+/** How many grid-attach attempts one daemon makes before giving up until its next start. Retries are
+ * driven by backend reconnects, and a link that flaps could otherwise sign in to grid on every one;
+ * a machine still unattached after this many reachable moments has a problem retrying won't fix. */
+const GRID_ATTACH_MAX_ATTEMPTS = 5
/** Between session-binding attempts for a process whose engine store is not resolvable yet. */
const REPAIR_RETRY_MS = 60_000
@@ -1557,6 +1561,9 @@ async function runForeground(session: AuthSession): Promise<void> {
let appVoiceFocus: { machineId: string; agentId: string; connId: string } | undefined
let backendRef: BackendSocket | undefined
let fullReconcile: (announceDevice?: boolean) => Promise<void> = async () => {}
+ /** Assigned below, once the grid reconcile exists. A backend (re)connect is the signal that the
+ * control plane is reachable again, which is precisely what an earlier attempt may have lacked. */
+ let onBackendConnected: () => void = () => {}
const auth = new AuthSessionManager(backendHttpBase())
const backend = new BackendSocket(session.machineId ?? session.computerId, auth, (connected) => {
@@ -1566,10 +1573,11 @@ async function runForeground(session: AuthSession): Promise<void> {
void fullReconcile(true).catch((err) => {
console.error('[runtime-profile] connect reconcile failed:', err instanceof Error ? err.message : err)
})
+ onBackendConnected()
}, computerId())
backendRef = backend
- // Bring this machine's grid sign-in into line with its harness sign-in, once, in the background.
+ // Bring this machine's grid sign-in into line with its harness sign-in, in the background.
// This is what makes a machine that signed in to the harness BEFORE grid existed usable after an
// update: it has the `grid` binary now (above), but no grid credentials and no grid to point at
// until something signs it in — and the login *event* that used to do that never fires again for
@@ -1579,46 +1587,67 @@ async function runForeground(session: AuthSession): Promise<void> {
// Best-effort and non-blocking: it awaits the managed grid, mints/reads the account's name, and
// signs in + creates the grid only when the machine is not already there. The RPCs that need the
// name wait briefly for it through `gridReadyProbe`.
- let gridAttachSettled = false
- // A hard ceiling on how long the RPCs will gate on the reconcile, INDEPENDENT of whether it
- // settled. `mintName` is bounded below, but the hand-off child (`grid login --harness`) has no
- // watchdog of its own — a control plane that black-holes its connection could keep the reconcile
- // pending for the daemon's life, and without this ceiling every grid RPC would then pay the full
- // per-call wait forever. Past the ceiling the probe returns null and the RPCs stop waiting; a
- // reconcile that lands later still publishes the name through `onName`.
- const gridReadyDeadline = Date.now() + GRID_ATTACH_CEILING_MS
- const gridAttachInFlight = reconcileGridAttach({
- managedGridReady,
- gridAvailable: () => gridAvailable(),
- // The backend mints and remembers the name; this CLI holds neither the account's email nor its
- // id. An older backend (no route) answers nothing, which the reconcile treats as "no grid yet".
- // Bounded so a stalled control-plane connection cannot hold the reconcile open indefinitely.
- mintName: async () => {
- const { headers } = await controlPlaneAuth()
- return (await postJson<{ gridName?: string }>('/api/grid/name', {}, headers, AbortSignal.timeout(GRID_MINT_TIMEOUT_MS))).gridName ?? null
- },
- accessToken: () => new AuthSessionManager(backendHttpBase()).accessToken(),
- signedInEmail: () => signedInGridEmail(),
- gridNames: () => gridNamesLocal(),
- handoff: (token) => handOffToGrid(token, { json: true }),
- ensure: (name) => ensureHarnessGrid(name),
- onName: (name) => {
- // Answer the picker with this account's grid at once, and drop the memos a stale or absent
- // sign-in may have filled — the model list, the derived name, and the web-tools URL.
- backend.setHarnessGridName(name)
- forgetGridModels()
- resetGridDeriveMemo()
- clearGridMcpUrlCache()
- },
- log: (line) => console.log(`[grid-attach] ${line}`),
- }).catch((err) => {
- console.error('[grid-attach] reconcile failed:', err instanceof Error ? (err.stack ?? err.message) : err)
- return null
- }).finally(() => { gridAttachSettled = true })
- void gridAttachInFlight
- // While the reconcile is still running AND within the ceiling, the RPCs that need the grid name
- // wait briefly on it; once it settles, or the ceiling passes, they read the name directly.
- backend.gridReadyProbe = () => (gridAttachSettled || Date.now() >= gridReadyDeadline ? null : gridAttachInFlight)
+ //
+ // ⚠️ Retried on backend RECONNECT, not just at start, and this is not belt-and-braces: the daemon
+ // OUTLIVES the desktop app (it must keep running after the window closes), and `harness start`
+ // against a live daemon returns without starting a new one. So "start" can be days ago, and a
+ // single attempt that lost to a control plane which was not reachable yet — the ordinary shape of
+ // a daemon coming up with the network — would leave the account with no grid until the next real
+ // restart. A connect is the evidence the control plane is reachable, so it is when to try again.
+ let gridAttachInFlight: Promise<unknown> | null = null
+ let gridAttachDone = false
+ let gridAttachAttempts = 0
+ // A hard ceiling on how long the RPCs will gate on an attempt, INDEPENDENT of whether it settled.
+ // `mintName` is bounded below, but the hand-off child (`grid login --harness`) has no watchdog of
+ // its own — a control plane that black-holes its connection could keep an attempt pending for the
+ // daemon's life, and without this ceiling every grid RPC would then pay the full per-call wait
+ // forever. Past the ceiling the probe returns null and the RPCs stop waiting; an attempt that
+ // lands later still publishes the name through `onName`.
+ let gridAttachDeadline = 0
+
+ const runGridAttach = (): void => {
+ if (gridAttachDone || gridAttachInFlight || gridAttachAttempts >= GRID_ATTACH_MAX_ATTEMPTS) return
+ gridAttachAttempts += 1
+ gridAttachDeadline = Date.now() + GRID_ATTACH_CEILING_MS
+ gridAttachInFlight = reconcileGridAttach({
+ managedGridReady,
+ gridAvailable: () => gridAvailable(),
+ // The backend mints and remembers the name; this CLI holds neither the account's email nor its
+ // id. An older backend (no route) answers nothing, which the reconcile treats as "no grid yet".
+ // Bounded so a stalled control-plane connection cannot hold the attempt open indefinitely.
+ mintName: async () => {
+ const { headers } = await controlPlaneAuth()
+ return (await postJson<{ gridName?: string }>('/api/grid/name', {}, headers, AbortSignal.timeout(GRID_MINT_TIMEOUT_MS))).gridName ?? null
+ },
+ accessToken: () => new AuthSessionManager(backendHttpBase()).accessToken(),
+ signedInEmail: () => signedInGridEmail(),
+ gridNames: () => gridNamesLocal(),
+ handoff: (token) => handOffToGrid(token, { json: true }),
+ ensure: (name) => ensureHarnessGrid(name),
+ onName: (name) => {
+ // Answer the picker with this account's grid at once, and drop the memos a stale or absent
+ // sign-in may have filled — the model list, the derived name, and the web-tools URL.
+ backend.setHarnessGridName(name)
+ forgetGridModels()
+ resetGridDeriveMemo()
+ clearGridMcpUrlCache()
+ },
+ log: (line) => console.log(`[grid-attach] ${line}`),
+ }).then((result) => {
+ // Only an outcome that actually attached stops the retries. `no-cli`, `no-name` and
+ // `handoff-failed` are all "the control plane or this machine was not ready", which is exactly
+ // what a later connect may have fixed.
+ if (result.status === 'converged' || result.status === 'signed-in') gridAttachDone = true
+ }).catch((err) => {
+ console.error('[grid-attach] reconcile failed:', err instanceof Error ? (err.stack ?? err.message) : err)
+ }).finally(() => { gridAttachInFlight = null })
+ }
+
+ // While an attempt is running AND within its ceiling, the RPCs that need the grid name wait
+ // briefly on it; otherwise they read the name directly.
+ backend.gridReadyProbe = () => (gridAttachInFlight && Date.now() < gridAttachDeadline ? gridAttachInFlight : null)
+ onBackendConnected = runGridAttach
+ runGridAttach()
/**
* Is ANY device surface watching this machine?
diff --git a/cli/src/lib/gridAttach.spec.ts b/cli/src/lib/gridAttach.spec.ts
index 596497e4b..576e12f4a 100644
--- a/cli/src/lib/gridAttach.spec.ts
+++ b/cli/src/lib/gridAttach.spec.ts
@@ -90,6 +90,17 @@ describe('reconcileGridAttach — the daemon-start convergence', () => {
expect(d.ensure).toHaveBeenCalledWith(NAME)
})
+ it('signs in again when the local grid list cannot be read — an unreadable registry is not "no grids"', async () => {
+ const d = deps({ gridNames: async () => { throw new Error('grid ls exited 1') } })
+ const r = await reconcileGridAttach(d)
+
+ // It must NOT treat the failed read as proof of anything: the safe direction is to sign in
+ // again, which rewrites the registry the next start reads.
+ expect(r.status).toBe('signed-in')
+ expect(d.handoff).toHaveBeenCalledOnce()
+ expect(d.ensure).toHaveBeenCalledWith(NAME)
+ })
+
it('does nothing and reports no-cli when there is no grid binary', async () => {
const d = deps({ gridAvailable: () => false, mintName: vi.fn() as never })
const r = await reconcileGridAttach(d)
diff --git a/cli/src/lib/gridAttach.ts b/cli/src/lib/gridAttach.ts
index 2b3bd9e58..2f6831f13 100644
--- a/cli/src/lib/gridAttach.ts
+++ b/cli/src/lib/gridAttach.ts
@@ -23,6 +23,12 @@
* `POST /api/grid/name` (idempotent, mostly a read) and nothing else. A grid sign-in is created or
* refreshed only when the gate says it is missing or belongs to another account.
*
+ * ⚠️ It converges only where the grid CAN exist. An account that cannot hold this grid at all — the
+ * free-plan network limit is the real case — never satisfies the gate, so every daemon start signs in
+ * again and tries again. That costs a token rotation per start, not a loop within one, and it is the
+ * deliberate trade: the alternative is remembering a failure across starts on disk, which would also
+ * remember it after the account was fixed. `ensure`'s own message says which limit was hit.
+ *
* **Overwrite is deliberate.** When this machine is signed in to grid as a *different* account, the
* hand-off overwrites it, matching `grid login`'s own rule (a swap is a line, never a refusal) and
* the harness's "grid sign-in is part of harness sign-in" contract. `grid login --harness` never
@@ -127,8 +133,10 @@ export async function reconcileGridAttach(deps: GridAttachDeps): Promise<GridAtt
if (email) {
try {
names = await deps.gridNames()
- } catch {
- // A local read that failed reads as "not there", which the hand-off + ensure below handle.
+ } catch (err) {
+ // Could not read the local registry. That cannot PROVE the machine is set up, so it falls
+ // through to the hand-off below — which rewrites that registry, so the next start reads it.
+ deps.log(`could not read this machine's grid list (${msg(err)}) — signing in again to rebuild it`)
}
}
if (email && names.includes(name)) {
@@ -167,7 +175,14 @@ export async function reconcileGridAttach(deps: GridAttachDeps): Promise<GridAtt
* cheap. An unparsable answer is no names, which the caller treats as "ensure it".
*/
export async function gridNamesLocal(): Promise<string[]> {
- const { value } = await gridJson<Array<{ grid?: unknown }>>(['--remote', 'ls'])
- if (!Array.isArray(value)) return []
+ const { value, result } = await gridJson<Array<{ grid?: unknown }>>(['--remote', 'ls'])
+ // ⚠️ A FAILED read is NOT an empty list, and conflating the two is how "this machine has no grids"
+ // stops being a fact and becomes a guess — one that would make the gate below answer "not set up"
+ // forever on a machine whose registry is unreadable. Thrown rather than returned as `[]` so the
+ // caller says which of the two happened in its log.
+ if (result.code !== 'OK') {
+ throw new Error(result.stderr.trim() || result.message || `\`grid ls\` exited ${result.exitCode}`)
+ }
+ if (!Array.isArray(value)) throw new Error('`grid ls --json` did not answer a list')
return value.map((row) => (typeof row.grid === 'string' ? row.grid : '')).filter(Boolean)
}
diff --git a/docs/skills/harness-compute.md b/docs/skills/harness-compute.md
index eda4527a3..bffce63d9 100644
--- a/docs/skills/harness-compute.md
+++ b/docs/skills/harness-compute.md
@@ -97,10 +97,10 @@ they don't know, and there is nothing for them to choose. The name stays out of
it's an argument you pass, not a thing you mention.
No name (or more than one) means it isn't provisioned: stop and say, in plain words, "Harness
-Compute isn't set up on this machine yet — restart Harness and it will set itself up." Harness signs
-this machine in to its grid and creates the account's private grid on start, so a restart is what
-completes the setup; signing in again does the same. Don't invent a name or create one; a guessed
-name can collide with someone else's on the same shared service.
+Compute isn't set up on this machine yet — sign in to Harness again and it will be." Signing in is
+what forces the setup. Do NOT tell them to restart or reopen the app: the part of Harness that sets
+this up keeps running after the window closes, so reopening it re-runs nothing. Don't invent a name
+or create one; a guessed name can collide with someone else's on the same shared service.
**If a command fails because the user isn't signed in**, stop and tell them to run
`harness login`. That's the only command in this whole feature that's ever theirs to run by hand.