Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

perf(backend): a viewer's terminal frames go straight to its connection (#2413)

0xken157b07e79d0a42
diff --git a/backend/src/config/env.ts b/backend/src/config/env.ts
index 874bf30db..1fdc90686 100644
--- a/backend/src/config/env.ts
+++ b/backend/src/config/env.ts
@@ -39,6 +39,10 @@ const envSchema = z.object({
// What each process keeps of the Swarms it read, by their stamps in Redis (superTabs/cache.ts). 'off' is the kill switch:
// every read goes to Mongo again, while writes still bump the stamps, so processes that keep theirs on stay right.
SWARM_READ_CACHE: z.enum(['on', 'off']).default('on'),
+ // A frame an adapter sends for one viewer goes to the process holding that viewer (lib/terminalRoute.ts), found by its
+ // record in Redis, instead of to every process with a viewer of the machine. 'off' is the kill switch: such frames go
+ // through the machine's channels again; a viewer on the adapter's own process is still served directly.
+ TERMINAL_POD_ROUTES: z.enum(['on', 'off']).default('on'),
// Its size per process, counted as the JSON size of what it holds.
SWARM_CACHE_MB: z.string().default('64').transform(Number),
// The worker's hourly move of finished deliveries older than 14 days into their messages' receipts
diff --git a/backend/src/lib/adapterRoutes.test.ts b/backend/src/lib/adapterRoutes.test.ts
new file mode 100644
index 000000000..8c0ddd87f
--- /dev/null
+++ b/backend/src/lib/adapterRoutes.test.ts
@@ -0,0 +1,120 @@
+import { EventEmitter } from 'node:events'
+import { beforeEach, describe, expect, it, vi } from 'vitest'
+
+// The machine socket sends each frame for one viewer through that viewer's route (lib/terminalRoute.ts), on the real
+// handler; the harness is adapterShortTurns.test.ts's, with the routes faked.
+const m = vi.hoisted(() => ({
+ auth: vi.fn(), upgrade: vi.fn(), resolve: vi.fn(), publishUp: vi.fn(), recordTurn: vi.fn(), nodeRole: vi.fn(), warn: vi.fn(), info: vi.fn(),
+}))
+vi.mock('./ssoAuth.js', async original => ({ ...await original<typeof import('./ssoAuth.js')>(), authenticateAccessToken: m.auth }))
+vi.mock('./wsServer.js', async original => ({ ...await original<typeof import('./wsServer.js')>(), createWss: () => ({ handleUpgrade: m.upgrade }) }))
+vi.mock('../services/MachineService.js', () => ({ machineService: { resolveOrCreateForComputer: m.resolve } }))
+vi.mock('./prisma.js', () => ({
+ prisma: { machine: { update: vi.fn(async () => ({})), findUnique: vi.fn(async () => null) }, user: { findUnique: vi.fn(async () => null) } },
+ machineAlive: { OR: [{ deletedAt: null }, { deletedAt: { isSet: false } }] },
+}))
+vi.mock('./bus.js', () => ({
+ publishUp: m.publishUp, claimMachineOwner: vi.fn(async () => true), releaseMachineOwner: vi.fn(), publishDeviceE2eePair: vi.fn(),
+ publishTerminalUp: vi.fn(), subscribeTerminalDown: vi.fn(async () => () => {}), setMachineAppState: vi.fn(), clearMachineAppState: vi.fn(),
+}))
+vi.mock('./adapterAccountPushes.js', () => ({ relayAccountPushes: vi.fn(async () => () => {}) }))
+vi.mock('./deviceKeyLog.js', () => ({ appendDeviceKey: vi.fn(), touchDeviceKey: vi.fn() }))
+vi.mock('./nodeRole.js', () => ({ attachNodeRole: m.nodeRole, PRESENCE_TTL_SEC: 30 }))
+vi.mock('./hub.js', () => ({
+ CLIENTS_FRAME: '__clients', CLIENTS_DIRTY: '__clients_dirty', LIVENESS_TICK_MS: 5_000, CLIENT_IDLE_DEADLINE_MS: 75_000,
+ DEVICE_IDLE_DEADLINE_MS: 35_000, CLIENT_HEARTBEAT_MS: 25_000, deliverTerminalUpLocal: vi.fn(), deliverUpLocal: vi.fn(),
+ pushDeviceRevoked: vi.fn(), sendClientsControl: vi.fn(), recomputeAndSendClients: vi.fn(), attachHubClient: vi.fn(),
+ trackSocketLiveness: () => () => {},
+}))
+vi.mock('./dailyTracking.js', async original => ({ ...await original<typeof import('./dailyTracking.js')>(),
+ touchMachineOnlineDay: vi.fn(async () => {}), touchUserOnlineDay: vi.fn(async () => {}), touchClientOnlineDay: vi.fn(async () => {}),
+ recordTurnStarted: m.recordTurn }))
+const routes = vi.hoisted(() => ({ hop: vi.fn(), targeted: vi.fn(), close: vi.fn() }))
+vi.mock('./terminalRoute.js', () => ({ createConnectionRoutes: vi.fn(() => routes) }))
+vi.mock('./p2pSignaling.js', async original => ({ ...await original<typeof import('./p2pSignaling.js')>(),
+ terminalP2pPolicy: () => ({ enabled: true, protocolVersion: 1, stunUrls: ['stun:example'], openWaitMs: 1_000 }) }))
+vi.mock('../utils/logger.js', () => ({ logger: { warn: m.warn, info: m.info, error: vi.fn(), debug: vi.fn() } }))
+
+import { handleAdapterUpgrade } from './adapterWs.js'
+import { encodeTerminalHop, TerminalBinaryKind, TerminalHopDirection } from './terminalBinary.js'
+
+const flush = async () => { for (let i = 0; i < 400; i++) await Promise.resolve() }
+
+class Socket extends EventEmitter {
+ readyState = 1; bufferedAmount = 0
+ send = vi.fn()
+ close = vi.fn((code?: number, reason?: string) => { this.readyState = 3; this.emit('close', code, Buffer.from(reason ?? '')) })
+ terminate = vi.fn(() => this.close())
+ frame(body: unknown) { this.emit('message', Buffer.from(JSON.stringify(body)), false) }
+}
+
+async function machineSocket(query = ''): Promise<Socket> {
+ const ws = new Socket()
+ m.upgrade.mockImplementation((_r, _s, _h, done) => done(ws))
+ handleAdapterUpgrade({ url: `/api/adapter-ws?computer=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa&label=Mac&autonomousEnv=prod${query}`,
+ headers: { 'sec-websocket-protocol': 'access-token' } } as never,
+ { write: vi.fn(), destroy: vi.fn(), destroyed: false, on: vi.fn(), once: vi.fn() } as never, Buffer.alloc(0))
+ await flush()
+ return ws
+}
+const sealed = (k: 'p' | 'g' = 'p') => ({ __e2e: { v: 1, n: 1, k, ct: 'x' } })
+const viewer = '0f0e0d0c-0b0a-4908-8706-050403020100'
+
+beforeEach(() => {
+ vi.clearAllMocks()
+ m.auth.mockResolvedValue({ sub: 'owner', email: '[email protected]', role: 'user', autonomousEnv: 'prod' })
+ m.resolve.mockResolvedValue({ machine: { machineId: 'routed-mac', userId: 'owner', name: 'Mac', billingStatus: 'not_required', sessionKind: null }, created: false })
+ m.publishUp.mockResolvedValue(1)
+ m.recordTurn.mockResolvedValue(undefined)
+ m.nodeRole.mockResolvedValue({ touch: vi.fn(), detach: vi.fn(async () => {}) })
+})
+
+describe('a machine socket and its viewers\' frames', () => {
+ it('delivers an untargeted frame to this process\'s viewers before a targeted frame sent after it, and publishes it for the others', async () => {
+ const hub = await import('./hub.js')
+ const ws = await machineSocket()
+ ws.frame({ t: 'up', frame: { type: 'agent_synced', payload: { __e2e: 'y' } } })
+ ws.frame({ t: 'up', targetConnId: 'c1', frame: { type: 'rpc_result', payload: { __e2e: 'z' } } })
+ await flush()
+ const local = vi.mocked(hub.deliverUpLocal)
+ expect(local).toHaveBeenCalledWith('routed-mac', { webEligible: true, commanderEligible: false, frame: { type: 'agent_synced', payload: { __e2e: 'y' } } })
+ expect(local.mock.invocationCallOrder[0]).toBeLessThan(routes.targeted.mock.invocationCallOrder[0]!)
+ // The Redis copy carries this process's id, which this process's own subscription skips.
+ expect(m.publishUp.mock.calls[0]![1]).toMatchObject({ frame: { type: 'agent_synced' }, originPid: expect.any(String) })
+ })
+
+ it('publishes a frame whose target is not a plausible connection id as before, without a route', async () => {
+ const ws = await machineSocket()
+ ws.frame({ t: 'up', targetConnId: 'x'.repeat(200), frame: { type: 'rpc_result', payload: { __e2e: 'z' } } })
+ await flush()
+ expect(routes.targeted).not.toHaveBeenCalled()
+ expect(m.publishUp.mock.calls[0]![1]).toMatchObject({ targetConnId: 'x'.repeat(200) })
+ })
+
+ it('routes a hop, a terminal frame, a p2p frame and a targeted reply by their connection, and publishes the rest as before', async () => {
+ const ws = await machineSocket()
+ const frame = Buffer.alloc(36)
+ frame.write('HTRM'); frame[4] = 3; frame[5] = TerminalBinaryKind.output; frame.writeUInt32BE(16, 16)
+ const packet = encodeTerminalHop(TerminalHopDirection.up, viewer, frame)!
+ ws.emit('message', packet, true)
+ ws.frame({ t: 'up', targetConnId: viewer, frame: { type: 'terminal_output', payload: sealed() } })
+ ws.frame({ t: 'up', targetConnId: viewer, frame: { type: 'p2p_answer', payload: sealed() } })
+ ws.frame({ t: 'up', targetConnId: 'c2', frame: { type: 'e2e_welcome', payload: { x: 1 } } })
+ ws.frame({ t: 'up', frame: { type: 'text_delta', payload: { __e2e: 'y' } } })
+ // A terminal frame that fails today's checks (not sealed) is still dropped before any route.
+ ws.frame({ t: 'up', targetConnId: viewer, frame: { type: 'terminal_output', payload: { plain: true } } })
+ await flush()
+ expect(routes.hop).toHaveBeenCalledTimes(1)
+ expect(Buffer.from(routes.hop.mock.calls[0]![0] as Uint8Array)).toEqual(packet)
+ expect(routes.hop.mock.calls[0]![1]).toBe(viewer)
+ expect(routes.targeted.mock.calls.map(([msg]) => msg)).toEqual([
+ { webEligible: true, commanderEligible: false, targetConnId: viewer, targetKind: 'web', frame: { type: 'terminal_output', payload: sealed() } },
+ { webEligible: true, commanderEligible: false, targetConnId: viewer, targetKind: 'web', frame: { type: 'p2p_answer', payload: sealed() } },
+ { webEligible: true, commanderEligible: false, targetConnId: 'c2', frame: { type: 'e2e_welcome', payload: { x: 1 } } },
+ ])
+ expect(m.publishUp.mock.calls.map(([, msg]) => msg)).toEqual([
+ { webEligible: true, commanderEligible: false, frame: { type: 'text_delta', payload: { __e2e: 'y' } }, originPid: expect.any(String) }])
+ ws.emit('close', 1000, Buffer.alloc(0))
+ expect(routes.close).toHaveBeenCalled()
+ })
+})
diff --git a/backend/src/lib/adapterWs.ts b/backend/src/lib/adapterWs.ts
index 7e1da75ab..dd0a87c48 100644
--- a/backend/src/lib/adapterWs.ts
+++ b/backend/src/lib/adapterWs.ts
@@ -24,17 +24,18 @@ import { createWss, upgradeStatusText, WS_LIMITS } from './wsServer.js'
import { prisma } from './prisma.js'
import {
publishUp, claimMachineOwner, releaseMachineOwner, publishDeviceE2eePair,
- publishTerminalUp, subscribeTerminalDown,
+ subscribeTerminalDown,
setMachineAppState, clearMachineAppState,
} from './bus.js'
-import { trackSocketLiveness } from './hub.js'
+import { deliverUpLocal, trackSocketLiveness } from './hub.js'
+import { PROCESS_ID } from './registry.js'
import { guardedSend, guardedSendJson } from './wsSend.js'
import { trackSocket } from './wsAttribution.js'
import { attachNodeRole, PRESENCE_TTL_SEC } from './nodeRole.js'
import { presenceTracker, presenceWriteDue, recordTurnStarted, touchClientOnlineDay, touchMachineOnlineDay, touchUserOnlineDay, TUI_SURFACES, type ClientSurface, type PresenceWriteState } from './dailyTracking.js'
import { countryCodeFromHeaders } from './clientGeo.js'
import { recordCreatedAgent, recordDeletedAgent } from './agentTracker.js'
-import type { Frame } from './tunnel.js'
+import type { Frame, UpBusMsg } from './tunnel.js'
import { logger } from '../utils/logger.js'
import { utcDayKey } from '../types/analytics.js'
import { machineBillingAllowsDataPlane } from './billingState.js'
@@ -67,6 +68,8 @@ import {
} from './p2pSignaling.js'
import { ShortTurnGuard } from './shortTurnGuard.js'
import { markCpuKind } from './selfProfile.js'
+import { createConnectionRoutes } from './terminalRoute.js'
+import { recordRelay } from './relayStats.js'
const wss = createWss(WS_LIMITS.adapter, { echoFirstProtocol: true })
@@ -412,6 +415,17 @@ async function attachAdapter(ws: WebSocket, machineId: string, userId: string, c
// deadline instead, which is the same fix already applied to the hub's own sockets.
const releaseLiveness = trackSocketLiveness(ws, 'adapter')
+ // Frames for one viewer go straight to it when it is on this process, else to the process holding it
+ // (lib/terminalRoute.ts); a connection keeps the route its first frame took for this socket's life.
+ const routes = createConnectionRoutes(machineId)
+ // A frame for one connection takes its route; a target that is not a plausible connection id (the daemon sets it) is
+ // published as before, never used as a route or Redis key.
+ const sendTargeted = (msg: UpBusMsg & { targetConnId: unknown }): void => {
+ const target = msg.targetConnId
+ if (typeof target === 'string' && target.length > 0 && target.length <= 128) routes.targeted({ ...msg, targetConnId: target })
+ else void publishUp(machineId, msg as UpBusMsg)
+ }
+
onMessage = (raw: RawData, isBinary: boolean): void => {
if (isBinary) {
const hop = decodeTerminalHop(new Uint8Array(raw as Buffer))
@@ -424,7 +438,7 @@ async function attachAdapter(ws: WebSocket, machineId: string, userId: string, c
: hop.kind === TerminalBinaryKind.keyframe ? 'terminal_keyframe' : 'terminal_sync',
hop.clientFrame.length,
)) return
- void publishTerminalUp(machineId, Buffer.from(raw as Buffer))
+ routes.hop(Buffer.from(raw as Buffer), hop.connId)
return
}
let env: AdapterFrame
@@ -440,7 +454,7 @@ async function attachAdapter(ws: WebSocket, machineId: string, userId: string, c
const bytes = terminalFrameBytes(env.frame)
if (!env.targetConnId || bytes > TERMINAL_ENVELOPE_MAX_BYTES
|| !isEncryptedTerminalFrame(env.frame) || !terminalRate.allow(terminalType, bytes)) return
- void publishUp(machineId, {
+ sendTargeted({
webEligible: true,
commanderEligible: false,
targetConnId: env.targetConnId,
@@ -455,7 +469,7 @@ async function attachAdapter(ws: WebSocket, machineId: string, userId: string, c
|| !terminalP2pPolicy(userId, machineId).enabled
|| bytes > P2P_SIGNAL_MAX_BYTES || !isEncryptedP2pFrame(env.frame)
|| !p2pSignalRate.allow(bytes)) return
- void publishUp(machineId, {
+ sendTargeted({
webEligible: true,
commanderEligible: false,
targetConnId: env.targetConnId,
@@ -562,14 +576,19 @@ async function attachAdapter(ws: WebSocket, machineId: string, userId: string, c
}
if (env.webEligible === false && env.commanderEligible !== true && !env.targetConnId) return
}
- // No originPid: the hub's Redis loopback must deliver to this instance's own web clients too
- // (there's no manager-ws local fast-path on this route).
- void publishUp(machineId, {
+ const up = {
webEligible: env.webEligible !== false, // default true (plain event/RPC frames)
commanderEligible: env.commanderEligible === true, // devices get only opted-in frames
- ...(env.targetConnId ? { targetConnId: env.targetConnId } : {}), // E2EE per-client push
frame: env.frame,
- })
+ }
+ // An E2EE per-client push (pairing, welcome, an RPC reply) takes its connection's route, so it keeps its order with
+ // that connection's terminal frames.
+ if (env.targetConnId) { sendTargeted({ ...up, targetConnId: env.targetConnId }); return }
+ // Every client of the machine: this process's own first, then the rest through Redis, whose copy this process
+ // skips (originPid), as manager-ws does. A local viewer's targeted frames are delivered here at once, so its
+ // untargeted ones must be too, or an RPC reply would overtake the event the daemon sent before it (review 2026-10-31).
+ deliverUpLocal(machineId, up)
+ void publishUp(machineId, { ...up, originPid: PROCESS_ID }).then(n => recordRelay('up broadcast', 'published', n))
}
}
// Drain whatever arrived while the role was being attached, in the order it was sent.
@@ -578,6 +597,7 @@ async function attachAdapter(ws: WebSocket, machineId: string, userId: string, c
const cleanup = (): void => {
releaseLiveness()
+ routes.close()
terminalDownUnsub()
accountPushesUnsub()
swarmPushesUnsub()
diff --git a/backend/src/lib/bus.ts b/backend/src/lib/bus.ts
index 014c7bbe6..9d1b668b6 100644
--- a/backend/src/lib/bus.ts
+++ b/backend/src/lib/bus.ts
@@ -274,6 +274,69 @@ export function publishTerminalDown(machineId: string, payload: Uint8Array): Pro
return safePublish(terminalDownChannel(machineId), Buffer.from(payload))
}
+// ── per-process channels for frames addressed to one viewer (lib/terminalRoute.ts) ─────────────────
+
+/** How long a viewer's record lives without its process renewing it (hub heartbeat every 25 s). */
+export const CONN_OWNER_TTL_MS = 60_000
+const connOwnerKey = (connId: string): string => `conn:${connId}`
+const podHopChannel = (processId: string): string => `termpod:${processId}`
+const podJsonChannel = (processId: string): string => `uppod:${processId}`
+
+/** Records that these viewer connections live on `processId`, in one pipeline. A failure leaves them unrecorded: frames
+ * for them then go through the machine's channels, as before records existed. */
+export async function recordConnOwners(connIds: readonly string[], processId: string): Promise<void> {
+ if (connIds.length === 0) return
+ const p = pub.pipeline()
+ for (const connId of connIds) p.set(connOwnerKey(connId), processId, 'PX', CONN_OWNER_TTL_MS)
+ try { await p.exec() } catch (err) { logger.debug('[bus] conn owner record failed', { error: String(err) }) }
+}
+export async function forgetConnOwner(connId: string): Promise<void> {
+ try { await pub.del(connOwnerKey(connId)) } catch { /* expires on its own */ }
+}
+/** The process holding this viewer connection, or null when none is recorded (or Redis cannot say). */
+export async function connOwner(connId: string): Promise<string | null> {
+ try { return await pub.get(connOwnerKey(connId)) } catch { return null }
+}
+
+/** A terminal hop for one process: the machine id (2-byte length + UTF-8) before the hop, so the receiver delivers only
+ * to that machine's viewer with the hop's connection id. */
+export function encodePodHop(machineId: string, packet: Uint8Array): Buffer {
+ const id = Buffer.from(machineId, 'utf8')
+ const out = Buffer.allocUnsafe(2 + id.length + packet.length)
+ out.writeUInt16BE(id.length, 0)
+ id.copy(out, 2)
+ out.set(packet, 2 + id.length)
+ return out
+}
+export function decodePodHop(payload: Buffer): { machineId: string; packet: Uint8Array } | null {
+ if (payload.length < 2) return null
+ const length = payload.readUInt16BE(0)
+ if (payload.length < 2 + length) return null
+ return { machineId: payload.toString('utf8', 2, 2 + length), packet: payload.subarray(2 + length) }
+}
+export function publishPodHop(processId: string, machineId: string, packet: Uint8Array): Promise<number> {
+ return safePublish(podHopChannel(processId), encodePodHop(machineId, packet))
+}
+export function publishPodJson(processId: string, machineId: string, msg: UpBusMsg): Promise<number> {
+ return safePublish(podJsonChannel(processId), JSON.stringify({ machineId, msg }))
+}
+/** This process's own channels: hops on the terminal subscriber connection, JSON frames on the main one, the same
+ * connections the machine's channels use, so a viewer's frames keep their order on each. */
+export async function subscribePod(processId: string, on: { hop(machineId: string, packet: Uint8Array): void; json(machineId: string, msg: UpBusMsg): void }): Promise<() => void> {
+ const subscribed = await Promise.allSettled([
+ addBinarySub(podHopChannel(processId), payload => { const hop = decodePodHop(payload); if (hop) on.hop(hop.machineId, hop.packet) }),
+ // Only a frame for one connection comes here; anything else would reach every viewer of the machine on this process.
+ addSub(podJsonChannel(processId), message => {
+ const { machineId, msg } = (message ?? {}) as { machineId?: unknown; msg?: UpBusMsg }
+ if (typeof machineId === 'string' && msg && typeof msg.targetConnId === 'string') on.json(machineId, msg)
+ }),
+ ])
+ const stops = subscribed.flatMap(each => each.status === 'fulfilled' ? [each.value] : [])
+ const failed = subscribed.find((each): each is PromiseRejectedResult => each.status === 'rejected')
+ if (failed) { for (const stop of stops) stop(); throw failed.reason }
+ return () => { for (const stop of stops) stop() }
+}
+
// ── up / down data channels ──────────────────────────────────────────────────────────────────────
export function subscribeUp(machineId: string, cb: (msg: UpBusMsg) => void): Promise<() => void> {
diff --git a/backend/src/lib/hub.ts b/backend/src/lib/hub.ts
index b9ae7ef45..0573573ee 100644
--- a/backend/src/lib/hub.ts
+++ b/backend/src/lib/hub.ts
@@ -10,12 +10,15 @@ import { WebSocket } from 'ws'
import { randomUUID } from 'crypto'
import {
bumpCommanderJoinGeneration,
+ forgetConnOwner,
getAgentClientState,
publishDown,
publishTerminalDown,
publishUp,
setAgentClientCount,
setAgentClientCountsBatch,
+ recordConnOwners,
+ subscribePod,
subscribeUp,
subscribeTerminalUp,
} from './bus.js'
@@ -309,6 +312,8 @@ function clientHeartbeatTick(): void {
// for every local agent, not 2 round trips each.
const rows = registry.localAgentIds().map((machineId) => ({ machineId, ...registry.clientCounts(machineId) }))
void setAgentClientCountsBatch(INSTANCE_ID, rows)
+ // Each viewer's record (CONN_OWNER_TTL_MS, 60 s) outlives two missed beats; a crashed process's records expire.
+ if (podRoutes) void recordConnOwners(registry.allClients().map(c => c.connId), registry.PROCESS_ID)
// B_m periodic resync: re-emit the summed `__clients` for every agent whose control socket lives
// here. Field EXPIRY (a client-holding instance crashed — its fields age out of the hash after 45s)
// has no attach/detach event to poke a recompute, so without this the node keeps the ghost count
@@ -326,6 +331,20 @@ setTimeout(() => {
setInterval(clientHeartbeatTick, CLIENT_HEARTBEAT_MS).unref?.()
}, Math.floor(Math.random() * 5_000)).unref?.()
+/** Whether this process takes frames addressed to its viewers on its own channels (lib/terminalRoute.ts): set once by
+ * startPodRoutes, with TERMINAL_POD_ROUTES on. Off, no viewer is recorded and adapters publish as before. */
+let podRoutes = false
+export const podRoutesOn = (): boolean => podRoutes
+
+/** Records every viewer this process holds, subscribes to its own channels, and keeps both up to date (attach, detach,
+ * the client heartbeat). Called once by server.ts. Returns its stop. */
+export async function startPodRoutes(): Promise<() => void> {
+ const unsubscribe = await subscribePod(registry.PROCESS_ID, { hop: deliverTerminalUpLocal, json: deliverUpLocal })
+ podRoutes = true
+ void recordConnOwners(registry.allClients().map(c => c.connId), registry.PROCESS_ID)
+ return () => { podRoutes = false; unsubscribe() }
+}
+
/** Register a client socket into the hub. The caller still owns the socket's 'message' handler. */
export function attachHubClient(socket: WebSocket, machineId: string, kind: ClientKind, opts: { guest?: boolean } = {}): HubClient {
const connId = randomUUID()
@@ -334,6 +353,7 @@ export function attachHubClient(socket: WebSocket, machineId: string, kind: Clie
// touch __hubDeadlineMs: a device sets its own (shorter) deadline before any machine attaches.
wireLiveness(socket)
const first = registry.addClient({ connId, machineId, kind, socket, ...(opts.guest ? { guest: true } : {}) })
+ if (podRoutes) void recordConnOwners([connId], registry.PROCESS_ID)
if (first) {
void subscribeUp(machineId, (msg) => {
@@ -412,6 +432,7 @@ export function attachHubClient(socket: WebSocket, machineId: string, kind: Clie
* than inside the HubClient closure. */
function detachConn(machineId: string, connId: string): void {
const last = registry.removeClient(machineId, connId)
+ if (podRoutes) void forgetConnOwner(connId)
if (last) {
const unsub = upSubs.get(machineId)
if (unsub) { unsub(); upSubs.delete(machineId) }
diff --git a/backend/src/lib/relayStats.test.ts b/backend/src/lib/relayStats.test.ts
new file mode 100644
index 000000000..53d191465
--- /dev/null
+++ b/backend/src/lib/relayStats.test.ts
@@ -0,0 +1,21 @@
+import { afterEach, expect, it } from 'vitest'
+import { recordRelay, startRelayStats } from './relayStats.js'
+
+let stop = () => {}
+afterEach(() => stop())
+
+it('counts local and published frames and the receivers each publish reached, per kind, and logs nothing in a quiet window', async () => {
+ const logs: Array<Record<string, unknown>> = []
+ stop = startRelayStats({ everyMs: 50, log: (_m, c) => logs.push(c) })
+ recordRelay('termup', 'local')
+ recordRelay('termup', 'local')
+ recordRelay('termup', 'published', 2)
+ recordRelay('up targeted', 'published', 1)
+ while (!logs.length) await new Promise(resolve => setTimeout(resolve, 10))
+ expect(logs[0]!.kinds).toEqual({
+ termup: { local: 2, published: 1, receivers: 2 },
+ 'up targeted': { local: 0, published: 1, receivers: 1 },
+ })
+ await new Promise(resolve => setTimeout(resolve, 120))
+ expect(logs).toHaveLength(1)
+})
diff --git a/backend/src/lib/relayStats.ts b/backend/src/lib/relayStats.ts
new file mode 100644
index 000000000..45b72373e
--- /dev/null
+++ b/backend/src/lib/relayStats.ts
@@ -0,0 +1,31 @@
+/**
+ * How each frame an adapter sends for one viewer reaches it, logged every minute (`terminal relay`): delivered on this
+ * process, or published, and how many subscriptions each publish reached (Redis answers that). A frame for one viewer
+ * that wakes several processes is work for nothing (spec 2026-10-31-terminal-relay-direct). Counts only, never an id.
+ */
+import { logger } from '../utils/logger.js'
+
+export type RelayKind = 'termup' | 'up targeted' | 'up broadcast' | 'pod hop' | 'pod json'
+interface Counts { local: number; published: number; receivers: number }
+
+let kinds = new Map<RelayKind, Counts>()
+
+export function recordRelay(kind: RelayKind, path: 'local' | 'published', receivers = 0): void {
+ let counts = kinds.get(kind)
+ if (!counts) kinds.set(kind, counts = { local: 0, published: 0, receivers: 0 })
+ if (path === 'local') counts.local++
+ else { counts.published++; counts.receivers += receivers }
+}
+
+export function startRelayStats({ everyMs = 60_000, log = (message: string, context: Record<string, unknown>) => logger.info(message, context) }:
+ { everyMs?: number; log?: (message: string, context: Record<string, unknown>) => void } = {}): () => void {
+ let since = Date.now()
+ const timer = setInterval(() => {
+ const now = Date.now(), window = kinds
+ kinds = new Map()
+ if (window.size) log('terminal relay', { windowMs: now - since, kinds: Object.fromEntries(window) })
+ since = now
+ }, everyMs)
+ timer.unref()
+ return () => clearInterval(timer)
+}
diff --git a/backend/src/lib/terminalRoute.test.ts b/backend/src/lib/terminalRoute.test.ts
new file mode 100644
index 000000000..e655efeb4
--- /dev/null
+++ b/backend/src/lib/terminalRoute.test.ts
@@ -0,0 +1,227 @@
+import { beforeEach, describe, expect, it, vi } from 'vitest'
+import type { WebSocket } from 'ws'
+import { encodeTerminalHop, TerminalBinaryKind, TerminalHopDirection } from './terminalBinary.js'
+
+const bus = vi.hoisted(() => ({
+ publishUp: vi.fn(async (..._args: unknown[]) => 1),
+ publishTerminalUp: vi.fn(async (..._args: unknown[]) => 1),
+ publishPodHop: vi.fn(async (..._args: unknown[]) => 1),
+ publishPodJson: vi.fn(async (..._args: unknown[]) => 1),
+ connOwner: vi.fn(async (_connId: string): Promise<string | null> => null),
+ recordConnOwners: vi.fn(async () => undefined),
+ forgetConnOwner: vi.fn(async () => undefined),
+ subscribePod: vi.fn(async () => () => undefined),
+ subscribeUp: async () => () => undefined,
+ subscribeTerminalUp: async () => () => undefined,
+ publishDown: async () => 1,
+ publishTerminalDown: async () => 1,
+ setAgentClientCount: async () => undefined,
+ setAgentClientCountsBatch: async () => undefined,
+ getAgentClientState: async () => ({ totals: { ui: 0, commander: 0, commanderActive: 0 }, commanderJoinGeneration: 1 }),
+ bumpCommanderJoinGeneration: async () => 1,
+}))
+vi.mock('./bus.js', () => bus)
+vi.mock('./providerLink.js', () => ({ routeDown: async (_m: string, _d: unknown, fallback: () => Promise<number>) => fallback() }))
+vi.mock('../utils/logger.js', () => ({ logger: { warn: vi.fn(), info: vi.fn(), error: vi.fn(), debug: vi.fn() } }))
+
+const { attachHubClient, startPodRoutes } = await import('./hub.js')
+const { createConnectionRoutes, MAX_ROUTES } = await import('./terminalRoute.js')
+const registry = await import('./registry.js')
+
+const socket = (out: unknown[]): WebSocket => ({ readyState: 1, bufferedAmount: 0, send: (data: unknown) => out.push(data),
+ close: vi.fn(), terminate: vi.fn(), on: () => undefined }) as unknown as WebSocket
+const clientFrame = (byte: number) => {
+ const frame = Buffer.alloc(36)
+ frame.write('HTRM'); frame[4] = 3; frame[5] = TerminalBinaryKind.output; frame.writeUInt32BE(16, 16); frame[35] = byte
+ return frame
+}
+const hop = (connId: string, byte: number) => encodeTerminalHop(TerminalHopDirection.up, connId, clientFrame(byte))!
+const json = (connId: string, n: number) => ({ webEligible: true, commanderEligible: false, targetConnId: connId, targetKind: 'web' as const,
+ frame: { type: 'terminal_output', payload: { n } } })
+const flush = () => new Promise(resolve => setImmediate(resolve))
+const remoteConn = () => '0f0e0d0c-0b0a-4908-8706-050403020100'
+let machine = 0
+const machineId = () => `route-${Date.now()}-${machine++}`
+
+beforeEach(() => { for (const fn of Object.values(bus)) if (vi.isMockFunction(fn)) fn.mockClear(); bus.connOwner.mockImplementation(async () => null) })
+
+describe('pod routes off (Phase 1 alone)', () => {
+ it('delivers a hop and a targeted JSON frame to a local web connection without publishing, in order', () => {
+ const id = machineId(), out: unknown[] = []
+ const web = attachHubClient(socket(out), id, 'web')
+ const routes = createConnectionRoutes(id)
+ routes.hop(hop(web.connId, 1), web.connId)
+ routes.targeted(json(web.connId, 2))
+ routes.hop(hop(web.connId, 3), web.connId)
+ expect(out).toHaveLength(3)
+ expect(Buffer.from(out[0] as Uint8Array)).toEqual(clientFrame(1))
+ expect(out[1]).toBe(JSON.stringify(json(web.connId, 2).frame))
+ expect(Buffer.from(out[2] as Uint8Array)).toEqual(clientFrame(3))
+ expect(bus.publishTerminalUp).not.toHaveBeenCalled()
+ expect(bus.publishUp).not.toHaveBeenCalled()
+ web.detach()
+ })
+
+ it('publishes for a connection that is not on this process, as before', async () => {
+ const id = machineId(), routes = createConnectionRoutes(id), conn = remoteConn()
+ const packet = hop(conn, 1)
+ routes.hop(packet, conn)
+ routes.targeted(json(conn, 2))
+ expect(bus.publishTerminalUp).toHaveBeenCalledWith(id, packet)
+ expect(bus.publishUp).toHaveBeenCalledWith(id, json(conn, 2))
+ expect(bus.connOwner).not.toHaveBeenCalled()
+ })
+
+ it('keeps a connection on its first route: frames after a local viewer detaches are dropped, not published', () => {
+ const id = machineId(), out: unknown[] = []
+ const web = attachHubClient(socket(out), id, 'web')
+ const routes = createConnectionRoutes(id)
+ routes.hop(hop(web.connId, 1), web.connId)
+ web.detach()
+ routes.hop(hop(web.connId, 2), web.connId)
+ routes.targeted(json(web.connId, 3))
+ expect(out).toHaveLength(1)
+ expect(bus.publishTerminalUp).not.toHaveBeenCalled()
+ expect(bus.publishUp).not.toHaveBeenCalled()
+ })
+
+ it('delivers a frame targeted at a local device connection', () => {
+ const id = machineId(), out: string[] = []
+ const device = attachHubClient(socket(out), id, 'commander')
+ const routes = createConnectionRoutes(id)
+ routes.targeted({ webEligible: false, commanderEligible: true, targetConnId: device.connId, frame: { type: 'e2e_welcome', payload: { sealed: 1 } } })
+ expect(out).toHaveLength(1)
+ expect(JSON.parse(out[0]!)).toMatchObject({ type: 'e2e_welcome', machineId: id })
+ expect(bus.publishUp).not.toHaveBeenCalled()
+ device.detach()
+ })
+
+ it('sends a guest only the frames addressed to it', () => {
+ const id = machineId(), ownerOut: unknown[] = [], guestOut: unknown[] = []
+ const owner = attachHubClient(socket(ownerOut), id, 'web')
+ const guest = attachHubClient(socket(guestOut), id, 'web', { guest: true })
+ const routes = createConnectionRoutes(id)
+ routes.targeted(json(guest.connId, 1))
+ routes.hop(hop(owner.connId, 2), owner.connId)
+ expect(guestOut).toHaveLength(1)
+ expect(ownerOut).toHaveLength(1)
+ owner.detach(); guest.detach()
+ })
+
+ it('still delivers locally when Redis takes no publish', async () => {
+ bus.publishTerminalUp.mockImplementationOnce(async () => 0)
+ const id = machineId(), out: unknown[] = []
+ const web = attachHubClient(socket(out), id, 'web')
+ createConnectionRoutes(id).hop(hop(web.connId, 1), web.connId)
+ expect(out).toHaveLength(1)
+ web.detach()
+ })
+
+ it('keeps one adapter socket\'s routes when another closes', () => {
+ const id = machineId(), out: unknown[] = []
+ const web = attachHubClient(socket(out), id, 'web')
+ const first = createConnectionRoutes(id), second = createConnectionRoutes(id)
+ first.hop(hop(web.connId, 1), web.connId)
+ second.hop(hop(web.connId, 2), web.connId)
+ first.close()
+ first.hop(hop(web.connId, 3), web.connId)
+ second.hop(hop(web.connId, 4), web.connId)
+ expect(out.map(f => Buffer.from(f as Uint8Array)[35])).toEqual([1, 2, 4])
+ web.detach()
+ })
+})
+
+it('keeps at most MAX_ROUTES routes per adapter socket, forgetting the least recently used', () => {
+ const id = machineId(), out: unknown[] = []
+ const web = attachHubClient(socket(out), id, 'web')
+ const routes = createConnectionRoutes(id)
+ routes.hop(hop(web.connId, 1), web.connId)
+ // Many connections that come and go (Desktop attaches a new one per agent switch), the live one used now and then.
+ for (let i = 0; i < MAX_ROUTES * 2; i++) {
+ const gone = `0f0e0d0c-0b0a-4908-8706-${String(i).padStart(12, '0')}`
+ routes.hop(hop(gone, 2), gone)
+ if (i % 500 === 0) routes.hop(hop(web.connId, 3), web.connId)
+ }
+ routes.hop(hop(web.connId, 4), web.connId)
+ // The live connection kept its local route throughout: every one of its frames was delivered here.
+ expect(out.map(f => Buffer.from(f as Uint8Array)[35])).toEqual([1, ...Array.from({ length: (MAX_ROUTES * 2) / 500 }, () => 3), 4])
+ web.detach()
+})
+
+describe('pod routes on (Phase 2)', () => {
+ let stop: () => void = () => {}
+ beforeEach(async () => { stop(); stop = await startPodRoutes() })
+
+ it('records a viewer on attach and forgets it on detach', async () => {
+ const id = machineId()
+ const web = attachHubClient(socket([]), id, 'web')
+ expect(bus.recordConnOwners).toHaveBeenCalledWith([web.connId], registry.PROCESS_ID)
+ web.detach()
+ expect(bus.forgetConnOwner).toHaveBeenCalledWith(web.connId)
+ })
+
+ it('publishes on the machine\'s channels while the owner is looked up, then on the owner\'s alone, in send order', async () => {
+ let answer: (owner: string | null) => void = () => {}
+ bus.connOwner.mockImplementation(() => new Promise(resolve => { answer = resolve }))
+ const id = machineId(), conn = remoteConn(), routes = createConnectionRoutes(id)
+ const packets = Array.from({ length: 5 }, (_, i) => hop(conn, i))
+ routes.hop(packets[0]!, conn); routes.targeted(json(conn, 1)); routes.hop(packets[1]!, conn)
+ answer('other-process')
+ await flush()
+ routes.hop(packets[2]!, conn); routes.targeted(json(conn, 2)); routes.hop(packets[3]!, conn)
+ expect(bus.connOwner).toHaveBeenCalledTimes(1)
+ expect(bus.publishTerminalUp.mock.calls.map(c => c[1])).toEqual([packets[0], packets[1]])
+ expect(bus.publishUp).toHaveBeenCalledWith(id, json(conn, 1))
+ expect(bus.publishPodHop.mock.calls.map(c => [c[0], c[1], c[2]])).toEqual([['other-process', id, packets[2]], ['other-process', id, packets[3]]])
+ expect(bus.publishPodJson).toHaveBeenCalledWith('other-process', id, json(conn, 2))
+ // Every machine-channel publish was issued before any owner-channel publish: one Redis connection keeps that order.
+ const lastMachine = Math.max(...bus.publishTerminalUp.mock.invocationCallOrder, ...bus.publishUp.mock.invocationCallOrder)
+ const firstOwner = Math.min(...bus.publishPodHop.mock.invocationCallOrder, ...bus.publishPodJson.mock.invocationCallOrder)
+ expect(lastMachine).toBeLessThan(firstOwner)
+ })
+
+ it('stays on the machine\'s channels for a connection with no owner, even once an owner appears', async () => {
+ const id = machineId(), conn = remoteConn(), routes = createConnectionRoutes(id)
+ routes.hop(hop(conn, 1), conn)
+ await flush()
+ bus.connOwner.mockImplementation(async () => 'other-process')
+ routes.hop(hop(conn, 2), conn)
+ await flush()
+ expect(bus.publishTerminalUp).toHaveBeenCalledTimes(2)
+ expect(bus.publishPodHop).not.toHaveBeenCalled()
+ })
+
+ it('keeps the machine\'s channels when the lookup fails, and when this process is the recorded owner', async () => {
+ const id = machineId(), failed = remoteConn(), mine = '1f0e0d0c-0b0a-4908-8706-050403020100', routes = createConnectionRoutes(id)
+ bus.connOwner.mockImplementation(async (connId: string) => { if (connId === failed) throw new Error('down'); return registry.PROCESS_ID })
+ routes.hop(hop(failed, 1), failed); routes.hop(hop(mine, 1), mine)
+ await flush()
+ routes.hop(hop(failed, 2), failed); routes.hop(hop(mine, 2), mine)
+ expect(bus.publishTerminalUp).toHaveBeenCalledTimes(4)
+ expect(bus.publishPodHop).not.toHaveBeenCalled()
+ })
+
+ it('delivers nothing from the process channels to a connection of another machine', async () => {
+ const handlers = (bus.subscribePod.mock.calls.at(-1) as unknown as [string, { hop(m: string, p: Uint8Array): void; json(m: string, msg: unknown): void }])[1]
+ const machineA = machineId(), machineB = machineId(), out: unknown[] = []
+ const viewer = attachHubClient(socket(out), machineB, 'web')
+ handlers.hop(machineA, hop(viewer.connId, 1))
+ handlers.json(machineA, json(viewer.connId, 2))
+ expect(out).toHaveLength(0)
+ handlers.hop(machineB, hop(viewer.connId, 3))
+ handlers.json(machineB, json(viewer.connId, 4))
+ expect(out).toHaveLength(2)
+ viewer.detach()
+ })
+
+ it('does not look up, nor record, with the switch off', async () => {
+ stop(); stop = () => {}
+ bus.recordConnOwners.mockClear()
+ const id = machineId(), conn = remoteConn()
+ const web = attachHubClient(socket([]), id, 'web')
+ createConnectionRoutes(id).hop(hop(conn, 1), conn)
+ expect(bus.connOwner).not.toHaveBeenCalled()
+ expect(bus.recordConnOwners).not.toHaveBeenCalled()
+ web.detach()
+ })
+})
diff --git a/backend/src/lib/terminalRoute.ts b/backend/src/lib/terminalRoute.ts
new file mode 100644
index 000000000..0d3006f86
--- /dev/null
+++ b/backend/src/lib/terminalRoute.ts
@@ -0,0 +1,93 @@
+/**
+ * Where a frame an adapter sends for one viewer goes (spec docs/superpowers/specs/2026-10-31-terminal-relay-direct-design.md).
+ * Every terminal frame, binary hop or JSON, is for exactly one viewer connection, yet it was published to the machine's
+ * channel: every process holding any viewer of that machine decoded it and all but one dropped it, and the adapter's own
+ * process got it back from Redis even when it held the viewer itself.
+ *
+ * Per connection, on this adapter socket:
+ * - the viewer is on this process: delivered here, no Redis;
+ * - else, with TERMINAL_POD_ROUTES on and the viewer's process recorded (hub.ts): published to that process alone, once
+ * the record is read; until then, and for good without a record (a process on the previous release, a failed lookup),
+ * the machine's channels, as before (see createConnectionRoutes for why the one switch keeps order).
+ */
+import * as registry from './registry.js'
+import { deliverTerminalUpLocal, deliverUpLocal, podRoutesOn } from './hub.js'
+import { connOwner, publishPodHop, publishPodJson, publishTerminalUp, publishUp } from './bus.js'
+import { recordRelay } from './relayStats.js'
+import type { UpBusMsg } from './tunnel.js'
+
+/** Routes one adapter socket keeps, most recently used last. An adapter socket lives for days while Desktop attaches a new
+ * connection on every agent switch; an active connection is only forgotten after this many others were used since. */
+export const MAX_ROUTES = 2_000
+
+type Route = { to: 'local' } | { to: 'machine' } | { to: 'process'; owner: string }
+
+export interface ConnectionRoutes {
+ /** A binary terminal hop (output, keyframe, sync) for `connId`. */
+ hop(packet: Uint8Array, connId: string): void
+ /** A JSON up frame addressed to one connection (`targetConnId`). */
+ targeted(msg: UpBusMsg & { targetConnId: string }): void
+ /** The adapter socket closed: forget every route. */
+ close(): void
+}
+
+/**
+ * A connection's route: local if its viewer is here (for good: a `connId` never moves). Otherwise the machine's channels
+ * while its owner is looked up, then, if another process owns it, that process's channels. That one switch keeps order:
+ * both channels are published from the same Redis connection, the owner reads both on the same subscriber connection
+ * (`up:`/`uppod:`, `termup:`/`termpod:`), and it holds a viewer of the machine, so it listens to the machine's channels.
+ * Nothing waits for the lookup, so the machine's untargeted frames sent after a targeted one cannot overtake it.
+ */
+export function createConnectionRoutes(machineId: string): ConnectionRoutes {
+ const routes = new Map<string, Route>()
+ const looking = new Set<string>()
+ let closed = false
+
+ const keep = (connId: string, route: Route) => {
+ routes.delete(connId)
+ routes.set(connId, route)
+ if (routes.size > MAX_ROUTES) routes.delete(routes.keys().next().value!)
+ }
+ const route = (connId: string): Route => {
+ const known = routes.get(connId)
+ if (known) { keep(connId, known); return known }
+ if (registry.clientsFor(machineId).some(c => c.connId === connId)) {
+ const local: Route = { to: 'local' }
+ keep(connId, local)
+ return local
+ }
+ const machine: Route = { to: 'machine' }
+ if (!podRoutesOn()) { keep(connId, machine); return machine }
+ if (!looking.has(connId)) {
+ looking.add(connId)
+ void connOwner(connId).then(owner => {
+ looking.delete(connId)
+ if (closed) return
+ keep(connId, owner && owner !== registry.PROCESS_ID ? { to: 'process', owner } : machine)
+ }, () => { looking.delete(connId); if (!closed) keep(connId, machine) })
+ }
+ return machine
+ }
+
+ return {
+ hop(packet, connId) {
+ if (closed) return
+ const r = route(connId)
+ if (r.to === 'local') { deliverTerminalUpLocal(machineId, packet); recordRelay('termup', 'local'); return }
+ if (r.to === 'process') { void publishPodHop(r.owner, machineId, packet).then(n => recordRelay('pod hop', 'published', n)); return }
+ void publishTerminalUp(machineId, packet).then(n => recordRelay('termup', 'published', n))
+ },
+ targeted(msg) {
+ if (closed) return
+ const r = route(msg.targetConnId)
+ if (r.to === 'local') { deliverUpLocal(machineId, msg); recordRelay('up targeted', 'local'); return }
+ if (r.to === 'process') { void publishPodJson(r.owner, machineId, msg).then(n => recordRelay('pod json', 'published', n)); return }
+ void publishUp(machineId, msg).then(n => recordRelay('up targeted', 'published', n))
+ },
+ close() {
+ closed = true
+ routes.clear()
+ looking.clear()
+ },
+ }
+}
diff --git a/backend/src/lib/terminalRouteRedis.test.ts b/backend/src/lib/terminalRouteRedis.test.ts
new file mode 100644
index 000000000..750b22778
--- /dev/null
+++ b/backend/src/lib/terminalRouteRedis.test.ts
@@ -0,0 +1,103 @@
+import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest'
+import type { UpBusMsg } from './tunnel.js'
+
+// A real Redis: REDIS_TEST_URL, e.g. redis://127.0.0.1:6379/15 (CI has none, so this file is skipped there). The bus
+// connects to it as REDIS_URL. One process stands for several: a process's channel is just `termpod:`/`uppod:` + its id,
+// so subscribing to another id here is what that other process would do.
+const url = process.env.REDIS_TEST_URL
+vi.hoisted(() => { if (process.env.REDIS_TEST_URL) process.env.REDIS_URL = process.env.REDIS_TEST_URL })
+vi.mock('../utils/logger.js', () => ({ logger: { warn: vi.fn(), info: vi.fn(), error: vi.fn(), debug: vi.fn() } }))
+
+describe.skipIf(!url)('terminal routes over a real Redis', () => {
+ let bus: typeof import('./bus.js')
+ beforeAll(async () => {
+ bus = await import('./bus.js')
+ // The command connection refuses commands until it is up (no offline queue), as in production at boot.
+ if (bus.pub.status !== 'ready') await new Promise(resolve => bus.pub.once('ready', resolve))
+ })
+ afterAll(async () => { await bus?.closeBus() })
+
+ const waitFor = async (done: () => boolean, ms = 5_000) => {
+ const until = Date.now() + ms
+ while (!done() && Date.now() < until) await new Promise(resolve => setTimeout(resolve, 10))
+ }
+ const unique = (name: string) => `${name}-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}`
+
+ it('records a viewer\'s process for 60 s, reads it back, and forgets it', async () => {
+ const a = unique('conn'), b = unique('conn')
+ await bus.recordConnOwners([a, b], 'process-x')
+ expect(await bus.connOwner(a)).toBe('process-x')
+ const ttl = await bus.pub.pttl(`conn:${a}`)
+ expect(ttl).toBeGreaterThan(55_000)
+ expect(ttl).toBeLessThanOrEqual(60_000)
+ await bus.forgetConnOwner(a)
+ expect(await bus.connOwner(a)).toBeNull()
+ expect(await bus.connOwner(b)).toBe('process-x')
+ })
+
+ it('hands 1,000 hops and 1,000 JSON frames to their process once and in order, and nothing to another process', async () => {
+ const mine = unique('pid'), other = unique('pid')
+ const hops: Array<[string, number]> = [], json: Array<[string, number]> = [], elsewhere: unknown[] = []
+ const stopMine = await bus.subscribePod(mine, { hop: (m, p) => hops.push([m, p[0]!]), json: (m, msg) => json.push([m, (msg.frame.payload as { n: number }).n]) })
+ const stopOther = await bus.subscribePod(other, { hop: m => elsewhere.push(m), json: m => elsewhere.push(m) })
+ for (let i = 0; i < 1_000; i++) {
+ void bus.publishPodHop(mine, 'machine-a', Uint8Array.of(i % 256, 1, 2))
+ void bus.publishPodJson(mine, 'machine-a', { targetConnId: 'c', frame: { type: 'terminal_output', payload: { n: i } } } as UpBusMsg)
+ }
+ await waitFor(() => hops.length === 1_000 && json.length === 1_000)
+ expect(hops.map(([, b]) => b)).toEqual(Array.from({ length: 1_000 }, (_, i) => i % 256))
+ expect(json.map(([, n]) => n)).toEqual(Array.from({ length: 1_000 }, (_, i) => i))
+ expect(new Set([...hops, ...json].map(([m]) => m))).toEqual(new Set(['machine-a']))
+ expect(elsewhere).toEqual([])
+ stopMine(); stopOther()
+ })
+
+ it('ignores a process-channel JSON message that is not for one connection', async () => {
+ const mine = unique('pid'), got: unknown[] = []
+ const stop = await bus.subscribePod(mine, { hop: () => {}, json: (_m, msg) => got.push(msg) })
+ void bus.publishPodJson(mine, 'machine-a', { frame: { type: 'agent_synced', payload: {} } } as UpBusMsg)
+ void bus.publishPodJson(mine, 'machine-a', { targetConnId: 'c', frame: { type: 'agent_synced', payload: {} } } as UpBusMsg)
+ await waitFor(() => got.length === 1)
+ await new Promise(resolve => setTimeout(resolve, 50))
+ expect(got).toEqual([{ targetConnId: 'c', frame: { type: 'agent_synced', payload: {} } }])
+ stop()
+ })
+
+ it('keeps a viewer\'s order across the switch from the machine\'s channel to its process\'s, as the owner reads both', async () => {
+ const { createConnectionRoutes } = await import('./terminalRoute.js')
+ const { startPodRoutes } = await import('./hub.js')
+ const stopRoutes = await startPodRoutes()
+ const owner = unique('pid'), conn = unique('conn'), machine = unique('machine'), got: number[] = []
+ await bus.recordConnOwners([conn], owner)
+ // The owner holds a viewer of the machine, so it reads the machine's channel too, on the same subscriber connection.
+ const onFrame = (msg: UpBusMsg) => { if (msg.targetConnId === conn) got.push((msg.frame.payload as { n: number }).n) }
+ const stopMachine = await bus.subscribeUp(machine, onFrame)
+ const stopOwner = await bus.subscribePod(owner, { hop: () => {}, json: (_m, msg) => onFrame(msg) })
+ const routes = createConnectionRoutes(machine)
+ // The first frames go out while the owner is being looked up (the machine's channel), the rest after (the owner's).
+ for (let i = 0; i < 1_000; i++) {
+ routes.targeted({ targetConnId: conn, webEligible: true, frame: { type: 'terminal_output', payload: { n: i } } } as UpBusMsg & { targetConnId: string })
+ if (i % 100 === 0) await new Promise(resolve => setImmediate(resolve))
+ }
+ await waitFor(() => got.length === 1_000)
+ expect(got).toEqual(Array.from({ length: 1_000 }, (_, i) => i))
+ routes.close(); stopMachine(); stopOwner(); stopRoutes()
+ })
+
+ it('routes a viewer\'s frames to the process its record names, in order, from an adapter on another process', async () => {
+ const { createConnectionRoutes } = await import('./terminalRoute.js')
+ const { startPodRoutes } = await import('./hub.js')
+ const stopRoutes = await startPodRoutes()
+ const owner = unique('pid'), conn = unique('conn'), got: number[] = []
+ await bus.recordConnOwners([conn], owner)
+ const stop = await bus.subscribePod(owner, { hop: () => {}, json: (_m, msg) => got.push((msg.frame.payload as { n: number }).n) })
+ const routes = createConnectionRoutes('machine-b')
+ // The first frame goes out on the machine's channel while the owner is looked up; once known, every frame goes to it.
+ routes.targeted({ targetConnId: conn, webEligible: true, frame: { type: 'terminal_output', payload: { n: -1 } } } as UpBusMsg & { targetConnId: string })
+ await new Promise(resolve => setTimeout(resolve, 50))
+ for (let i = 0; i < 1_000; i++) routes.targeted({ targetConnId: conn, webEligible: true, frame: { type: 'terminal_output', payload: { n: i } } } as UpBusMsg & { targetConnId: string })
+ await waitFor(() => got.length === 1_000)
+ expect(got).toEqual(Array.from({ length: 1_000 }, (_, i) => i))
+ routes.close(); stop(); stopRoutes()
+ })
+})
diff --git a/backend/src/server.ts b/backend/src/server.ts
index e0c6b2bc3..17ab932a0 100644
--- a/backend/src/server.ts
+++ b/backend/src/server.ts
@@ -34,6 +34,8 @@ import { selfProfileFromEnv } from './lib/selfProfile.js'
import { startRelayUsageFlush } from './lib/productUsage.js'
import { registerWorkContext, startQueryStats } from './lib/queryStats.js'
import { startRetryStats } from './superTabs/retryStats.js'
+import { startRelayStats } from './lib/relayStats.js'
+import { startPodRoutes } from './lib/hub.js'
import { registerRouteLatency, startRouteLatency } from './lib/routeLatency.js'
import { drainAllSockets, openSocketCount, RELEASE_UPGRADE_SLOT, type SlotSocket } from './lib/wsServer.js'
import { endpointOf, refusedUpgrade, startWsCounterLog, watchUpgrade } from './lib/wsAttribution.js'
@@ -150,6 +152,13 @@ async function start(): Promise<void> {
const stopRouteLatency = startRouteLatency(registerRouteLatency(app))
// Every minute: who reads messages how, with the rows each page decoded and showed (superTabs/messageReadStats.ts).
const stopMessageReadStats = startMessageReadStats()
+ // Every minute: how each frame an adapter sends for one viewer reached it, and how many processes each publish woke.
+ const stopRelayStats = startRelayStats()
+ // Frames for one viewer go to the process holding it (lib/terminalRoute.ts), unless TERMINAL_POD_ROUTES=off.
+ let stopPodRoutes = (): void => {}
+ if (env.TERMINAL_POD_ROUTES !== 'off') {
+ startPodRoutes().then(stop => { stopPodRoutes = stop }, err => logger.warn('pod routes failed to start; frames use the machine channels', { error: String(err) }))
+ }
app.setErrorHandler(errorHandler)
@@ -231,6 +240,8 @@ async function start(): Promise<void> {
stopRouteLatency()
stopMachineReportLog()
stopMessageReadStats()
+ stopRelayStats()
+ stopPodRoutes()
// The database stays connected through the drain, for the requests still being served (lib/prisma.ts).
await Promise.allSettled([prisma.$disconnect(), closeBus()])
process.exit(exitCode)
diff --git a/docs/superpowers/plans/2026-10-31-terminal-relay-direct.md b/docs/superpowers/plans/2026-10-31-terminal-relay-direct.md
new file mode 100644
index 000000000..c1b38c16f
--- /dev/null
+++ b/docs/superpowers/plans/2026-10-31-terminal-relay-direct.md
@@ -0,0 +1,144 @@
+# Terminal relay: per-connection frames go straight to their connection (Phases 1 and 2) — Implementation Plan
+
+> **Implemented 2026-10-31** with the review changes noted in the spec (one switch instead of a queue; whole `UpBusMsg` on `uppod:`; untargeted frames local first). Task text below is the original plan.
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** An `adapter-ws` up frame for one connection is delivered directly when that connection is on this worker, and otherwise published only to the worker that holds it; a per-minute log says how many workers each published frame reaches.
+
+**Architecture:** `terminalRoute.ts` decides, per connection and per adapter socket, local, owner's channel or today's channels (sticky from the first frame). Phase 1 (Tasks 1–3) is local delivery and the counters; Phase 2 (Tasks 4–6) adds owner records in Redis, a channel per worker, and routing to it behind `TERMINAL_POD_ROUTES`.
+
+**Tech Stack:** TypeScript, `ws`, ioredis pub/sub (`lib/bus.ts`), vitest.
+
+**Spec:** `docs/superpowers/specs/2026-10-31-terminal-relay-direct-design.md` (both phases, one delivery)
+
+## Global Constraints
+
+- Both phases ship together. Phase 2 runs unless `TERMINAL_POD_ROUTES=off`; with it off, Phase 1 alone runs.
+- Owner record `PX 60 000`, renewed every `CLIENT_HEARTBEAT_MS` (25 s). A non-local connection uses the machine's channels until its owner is known, then the owner's; no owner or a failed lookup keeps the machine's channels (spec, Phase 2 "One switch, in order").
+- A `pod:` message carries `machineId`; a JSON one carries the whole `UpBusMsg` and is delivered with `deliverUpLocal` (implementation call, see spec); both reach only that machine's client with that `connId`.
+- Untargeted adapter frames are delivered locally first, then published with `originPid` (independent review).
+- Each `pod:` channel rides the subscriber connection of the channel it replaces: binary on `terminalSub`, JSON on `sub`.
+- A connection's frames arrive once, in order, byte-identical to today; send rules unchanged (`guardedSend` "must", kill-water).
+- "Local" means this Node worker (`registry.PROCESS_ID`), not this pod.
+- The route is sticky: decided by the connection's first frame on an adapter socket, kept until the adapter socket closes.
+- Frames for every client of a machine (no `targetConnId`) are unchanged.
+- No stored data, no client change, backend only. No ids in logs.
+- Do not commit or push until Ken asks; each task ends with a diff checkpoint. Remove dead code.
+- Commands from `backend/`: `npx vitest run <file>`; `npm run typecheck`. Real-Redis tests run with `REDIS_TEST_URL=redis://127.0.0.1:6379/15` (local docker `harness-redis`; CI skips them, as `src/superTabs/cacheRedis.test.ts` does).
+
+## Review Focus
+
+1. A local viewer that detaches mid-stream: later frames for its `connId` are dropped, never published to other pods — Task 2 test.
+2. A JSON frame targeted at a device (`commander`) connection on this worker (E2EE pairing): delivered locally like a web one — Task 2 test.
+3. A guest connection (shared machine) gets only frames addressed to it, locally as through Redis (`hub.test.ts` "a guest on a shared machine") — Task 2 test.
+4. Redis down (`safePublish` returns 0): a local viewer still gets frames; stats count 0 receivers — Task 2 test.
+5. Two adapter sockets for one machine (a reconnect overlap): each keeps its own sticky map; closing one does not clear the other's — Task 2 test.
+
+---
+
+### Task 1: Relay counters and the `terminal relay` log
+
+**Files:**
+- Create: `backend/src/lib/relayStats.ts`
+- Modify: `backend/src/server.ts` (start and stop beside `startRetryStats`)
+- Test: `backend/src/lib/relayStats.test.ts`
+
+**Interfaces:**
+- Produces: `type RelayKind = 'termup' | 'up targeted' | 'up broadcast'`; `recordRelay(kind: RelayKind, path: 'local' | 'published', receivers?: number): void`; `startRelayStats(options?: { everyMs?: number; log?: (message: string, context: Record<string, unknown>) => void }): () => void` logging `'terminal relay'` each `everyMs` (default 60 000) with `{ windowMs, kinds: { [kind]: { local, published, receivers } } }`, nothing in a window with no frames.
+
+- [ ] **Step 1: Write the failing test** `'counts local and published frames and the receivers each publish reached, per kind'`: record `('termup','local')` ×2, `('termup','published',2)`, `('up targeted','published',1)`; with `everyMs: 50` expect `{ termup: { local: 2, published: 1, receivers: 2 }, 'up targeted': { local: 0, published: 1, receivers: 1 } }`, and a later empty window logs nothing.
+- [ ] **Step 2: Run** `npx vitest run src/lib/relayStats.test.ts` — FAIL (module missing).
+- [ ] **Step 3: Implement** the module and wire it in `server.ts`.
+- [ ] **Step 4: Run** the test — PASS; `npm run typecheck` — passes.
+- [ ] **Step 5: Checkpoint** `git diff --stat`.
+
+### Task 2: Sticky per-connection routes
+
+**Files:**
+- Create: `backend/src/lib/terminalRoute.ts`
+- Test: `backend/src/lib/terminalRoute.test.ts` (mock `./bus.js` as `hub.test.ts` does; attach clients with `attachHubClient`)
+
+**Interfaces:**
+- Consumes: `deliverTerminalUpLocal(machineId, packet)` and `deliverUpLocal(machineId, msg)` (`hub.ts`); `publishTerminalUp`, `publishUp` (`bus.ts`, they resolve to the receiver count); `registry.clientsFor(machineId)`; Task 1's `recordRelay`.
+- Produces: `createConnectionRoutes(machineId: string): ConnectionRoutes` with
+ - `hop(packet: Uint8Array, connId: string): void` — a binary terminal hop for `connId` (a `web` client);
+ - `targeted(msg: UpBusMsg & { targetConnId: string }): void` — a JSON up frame for one connection (any client kind; `msg.targetKind` as today);
+ - `close(): void` — forgets the sticky map (adapter socket closed).
+ First frame per `connId`: local when `registry.clientsFor(machineId)` holds a client with that `connId` (and, for a hop, kind `web`), else published; the decision is kept in a `Map<connId, 'local' | 'published'>` for this object's life. Local → deliver with the hub function and `recordRelay(kind, 'local')`; published → today's publish, then `recordRelay(kind, 'published', receivers)`.
+- [ ] **Step 1: Write the failing tests:**
+ - `'delivers a hop and a targeted JSON frame to a local web connection without publishing'`: attach a web client, send a hop and a `terminal_output` JSON frame; the socket gets both, in order, the JSON text equal to `JSON.stringify(frame)`; `publishTerminalUp` and `publishUp` not called.
+ - `'publishes for a connection that is not on this worker, as today'`: no client attached; `publishTerminalUp`/`publishUp` called with today's arguments.
+ - `'keeps a connection on its first route: frames after a local viewer detaches are dropped, not published'` (Review Focus 1).
+ - `'delivers a frame targeted at a local device connection'` (Review Focus 2).
+ - `'sends a guest only frames addressed to it'` (Review Focus 3).
+ - `'still delivers locally when Redis takes no publish'` (Review Focus 4; `publishUp` resolves 0).
+ - `'keeps one adapter socket\'s routes when another closes'` (Review Focus 5).
+- [ ] **Step 2: Run** `npx vitest run src/lib/terminalRoute.test.ts` — FAIL (module missing).
+- [ ] **Step 3: Implement** `terminalRoute.ts`.
+- [ ] **Step 4: Run** it with `src/lib/hub.test.ts` — PASS.
+- [ ] **Step 5: Checkpoint** `git diff --stat`.
+
+### Task 3: `adapter-ws` sends per-connection frames through the routes
+
+**Files:**
+- Modify: `backend/src/lib/adapterWs.ts` (create one `ConnectionRoutes` once `machineId` is known; binary hop at ~427; JSON `terminal_*` at ~443; `p2p_*` at ~458; the general publish at ~567 when `env.targetConnId` is set; `close()` in `cleanup`)
+- Test: `backend/src/lib/adapterRoutes.test.ts` (copy the harness of `adapterShortTurns.test.ts`, lines 1–60, and mock `./terminalRoute.js`)
+
+**Interfaces:**
+- Consumes: Task 2's `createConnectionRoutes`; Task 1's `recordRelay` for the untargeted general publish (`'up broadcast'`, `'published'`, its receivers).
+- [ ] **Step 1: Write the failing test** `'routes a hop, a terminal frame, a p2p frame and a targeted reply by their connection, and publishes the rest as before'`: through the adapter socket send a binary output hop (connId `c1`), a JSON `terminal_output` with `targetConnId: 'c1'`, a `p2p_offer` with `targetConnId: 'c1'`, an `e2e_welcome` with `targetConnId: 'c2'`, and an untargeted event; expect `hop(packet, 'c1')`, `targeted` ×3 with today's message fields (`webEligible`, `commanderEligible`, `targetKind`, `targetConnId`, `frame`), and `publishUp` once for the untargeted event; the route's `close` is called when the socket closes. Today's validation still drops a bad frame before routing (one invalid terminal frame is not routed).
+- [ ] **Step 2: Run** `npx vitest run src/lib/adapterRoutes.test.ts` — FAIL.
+- [ ] **Step 3: Implement** the wiring; keep every existing check (rate limits, sizes, encryption) before routing.
+- [ ] **Step 4: Run** it with `src/lib/adapterShortTurns.test.ts`, `src/lib/hub.test.ts`, `src/lib/hubLiveness.test.ts`, `src/lib/webWs*.test.ts` — PASS.
+- [ ] **Step 5: Checkpoint** `git diff --stat`.
+
+### Task 4: Owner records (Phase 2)
+
+**Files:**
+- Modify: `backend/src/lib/bus.ts` (owner record functions), `backend/src/lib/hub.ts` (`attachHubClient`, the detach path, `clientHeartbeatTick`), `backend/src/config/env.ts` (`TERMINAL_POD_ROUTES`, default `'on'`)
+- Test: `backend/src/lib/connOwnersRedis.test.ts` (real Redis, `describe.skipIf(!process.env.REDIS_TEST_URL)`), `backend/src/lib/hub.test.ts` (mocked bus)
+
+**Interfaces:**
+- Produces in `bus.ts`: `recordConnOwners(connIds: string[], processId: string): Promise<void>` (one pipeline, `SET conn:{connId} processId PX 60000` each); `forgetConnOwner(connId: string): Promise<void>`; `connOwner(connId: string): Promise<string | null>`. Hub calls `recordConnOwners([connId])` on attach, `forgetConnOwner` on detach, and `recordConnOwners(all local connIds)` in `clientHeartbeatTick`, all only when the switch is on.
+- [ ] **Step 1: Write the failing tests:** real Redis: a recorded owner reads back, expires after 60 s (use `PTTL`), is gone after forget, and a batch of 500 is one round trip (`MONITOR`-free: count commands with a wrapped client). Mocked bus: attach records, detach forgets, the heartbeat renews every local `connId`; with `TERMINAL_POD_ROUTES=off` none of these are called.
+- [ ] **Step 2: Run** `REDIS_TEST_URL=redis://127.0.0.1:6379/15 npx vitest run src/lib/connOwnersRedis.test.ts src/lib/hub.test.ts` — FAIL.
+- [ ] **Step 3: Implement.**
+- [ ] **Step 4: Run** them — PASS.
+- [ ] **Step 5: Checkpoint** `git diff --stat`.
+
+### Task 5: A channel per worker (Phase 2)
+
+**Files:**
+- Modify: `backend/src/lib/bus.ts`, `backend/src/lib/hub.ts` (subscribe once at startup when the switch is on)
+- Test: `backend/src/lib/podChannelRedis.test.ts` (real Redis), `backend/src/lib/hub.test.ts`
+
+**Interfaces:**
+- Produces in `bus.ts`: `publishPodHop(processId: string, machineId: string, packet: Uint8Array): Promise<number>` on channel `termpod:{processId}` (payload = 2-byte length + `machineId` UTF-8 + packet); `publishPodJson(processId: string, msg: { machineId: string; connId: string; targetKind?: 'web' | 'commander'; text: string }): Promise<number>` on `uppod:{processId}`; `subscribePod(processId: string, on: { hop(machineId: string, packet: Uint8Array): void; json(msg: …): void }): Promise<() => void>` — `termpod:` on `terminalSub`, `uppod:` on the JSON `sub` connection.
+- Produces in `hub.ts`: `deliverTargetedTextLocal(machineId: string, connId: string, text: string, targetKind?: 'web' | 'commander'): boolean` — sends `text` to that one client with the same checks and send rules `deliverUpLocal` applies to a `targetConnId` frame; `false` when no such client of that machine. The hop side calls `deliverTerminalUpLocal(machineId, packet)`, which already checks the client is that machine's.
+- [ ] **Step 1: Write the failing tests:** real Redis: 1,000 hops and 1,000 JSON texts published to one worker arrive once and in order on their own connection; a message for `machineId` A with a `connId` that belongs to machine B is not delivered (both hop and JSON). Mocked: `deliverTargetedTextLocal` sends byte-identical text, honours `targetKind`, returns `false` for a closed connection; a guest gets only its own frames.
+- [ ] **Step 2: Run** them — FAIL.
+- [ ] **Step 3: Implement.**
+- [ ] **Step 4: Run** them with `src/lib/hub.test.ts` and `src/lib/bus.test.ts` — PASS.
+- [ ] **Step 5: Checkpoint** `git diff --stat`.
+
+### Task 6: Routes publish to the owner (Phase 2)
+
+**Files:**
+- Modify: `backend/src/lib/terminalRoute.ts`
+- Test: `backend/src/lib/terminalRoute.test.ts`, `backend/src/lib/terminalRouteRedis.test.ts` (real Redis)
+
+**Interfaces:**
+- Consumes: Task 4's `connOwner`; Task 5's `publishPodHop`, `publishPodJson`; Task 1's `recordRelay` with new kinds `'pod hop'` and `'pod json'`.
+- Produces: the same `ConnectionRoutes` API. The first frame of a non-local connection starts `connOwner(connId)`; frames for that connection queue in arrival order until it answers (or 250 ms pass); then the route is fixed: an owner that is not this worker → `pod:` publishes (the JSON frame stringified once, as `text`); no owner, a failure, a timeout, or the switch off → today's channels. The queue drains in order before any newer frame.
+- [ ] **Step 1: Write the failing tests:** an owner on another worker → `publishPodHop`/`publishPodJson` only, never `publishTerminalUp`/`publishUp`; no owner → today's channels for the connection's life, even when an owner appears later; a lookup that takes 300 ms → today's channels, frames sent in order; 50 frames sent while the lookup is pending go out in order after it; switch off → no lookup. Real Redis: two subscriber connections standing for two workers, an owner record for worker B, 1,000 frames from worker A for B's connection arrive at B in order and none at a third worker.
+- [ ] **Step 2: Run** `npx vitest run src/lib/terminalRoute.test.ts` and the Redis test — FAIL.
+- [ ] **Step 3: Implement.**
+- [ ] **Step 4: Run** them with Tasks 2–5's tests — PASS.
+- [ ] **Step 5: Checkpoint** `git diff --stat`.
+
+### Task 7: Checks
+
+- [ ] **Step 1:** `npm run typecheck`; `npx vitest run src/lib src/routes` — only failures already on clean `main`, listed in the PR.
+- [ ] **Step 2:** Run the real-Redis tests locally: `REDIS_TEST_URL=redis://127.0.0.1:6379/15 npx vitest run src/lib/*Redis.test.ts`; record the result in the PR (CI skips them).
+- [ ] **Step 3:** After release, from `cpu by kind` (`adapter-ws *`, `bus terminal *`, `bus main up`, `bus * termpod|uppod`) and `terminal relay`: total relay busy time before and after, the local share, and receivers per published frame. If receivers were already about 1 before, set `TERMINAL_POD_ROUTES=off` and file the removal of Phase 2.
diff --git a/docs/superpowers/specs/2026-10-31-terminal-relay-direct-design.md b/docs/superpowers/specs/2026-10-31-terminal-relay-direct-design.md
new file mode 100644
index 000000000..205c9caa2
--- /dev/null
+++ b/docs/superpowers/specs/2026-10-31-terminal-relay-direct-design.md
@@ -0,0 +1,145 @@
+# Design: per-connection frames go straight to the one connection they are for (terminal relay)
+
+Date: 2026-10-31. Code read at `origin/main` (664dd0f63). Spec 6 in the backend CPU series.
+
+## Why
+
+A remote terminal's output reaches its viewer like this: the machine's daemon sends it up its `adapter-ws` socket to
+whichever backend pod holds that socket; that pod publishes it to Redis; every pod with a viewer of that machine receives
+it and hands it to its viewer.
+
+**Every terminal frame is for exactly one viewer connection.** A binary terminal hop carries its `connId`
+(`deliverTerminalUpLocal`, `hub.ts:44`), and a JSON `terminal_*` / `p2p_*` up frame carries `targetConnId`
+(`adapterWs.ts:437-470`). Yet:
+
+- `adapterWs.ts:427` publishes every binary hop to `termup:{machineId}`, and `adapterWs.ts:443,458` every JSON one to
+ `up:{machineId}`, **even when the viewer is on the same pod**: the pod receives its own frame back from Redis.
+ (`managerWs.ts:255-256` already delivers locally first and skips its own echo with `originPid`; the adapter path does
+ not.)
+- Every pod holding any client of that machine receives the frame, decodes it, and drops it unless it owns the
+ `connId`.
+- A JSON up frame is encoded twice: `publishUp` stringifies the whole envelope, the receiver parses it, and
+ `deliverUpLocal` (`hub.ts:59`) stringifies `msg.frame` again before sending.
+
+**Prod (2026-10-30/31, 16 CPU profiles, 3 pods):** relay and socket work is ~13.5% of non-idle CPU (Redis bus 2.5%,
+ioredis 2.5%, socket writes 6.6%, `ws` receiver 0.9%), consistent with the 17.5% of event-loop time logged as
+`untagged`. #2393 names these kinds (`adapter-ws binary|text`, `bus terminal termup`, `bus main up`) from the next
+release, which will say exactly how much is terminal relay.
+
+## Goals
+
+- A terminal frame whose viewer is on the pod that received it never goes through Redis.
+- A frame whose viewer is on another pod reaches only that pod (Phase 2), behind a kill switch.
+- A JSON terminal frame is not parsed and re-stringified on its way to the viewer.
+- Ordering, delivery guarantees ("must" sends, kill-water) and E2EE are unchanged: the backend still forwards opaque
+ sealed bytes.
+
+## Non-goals
+
+- Up frames that are not for one connection (status, events, RPC replies to all clients): they keep `up:{machineId}`.
+- Down frames (viewer → machine): `sendDown` already goes direct when the control socket is local (`hub.ts`).
+- Moving terminals to p2p, the multiplexed account socket (WebSocket roadmap Phases 4–5).
+
+## Anti-goals
+
+- A frame delivered twice, lost, or out of order for its connection.
+- A viewer on a pod running the previous release missing frames during a rolling deploy.
+
+## Design decided
+
+*(Author's calls after the builder check: the phases, the sticky route, and every per-connection adapter frame.)*
+
+### Phase 1 — local first, encode once, count the receivers
+
+- **Which frames:** every `adapter-ws` up frame for one connection: a binary terminal hop (its `connId`) and any JSON up
+ frame with a `targetConnId` (`terminal_*`, `p2p_*`, and the general per-connection publish at `adapterWs.ts:565`, such
+ as `e2e_welcome` and per-client RPC replies). Keeping all of a connection's frames on one path keeps their order.
+ Frames for every client of a machine (no target) are unchanged.
+- **Local:** when that connection is on this process (`registry.clientsFor(machineId)` holds its `connId`; `PROCESS_ID`
+ is per Node worker, so "local" means the same worker), the frame is delivered there directly with today's send rules
+ (`deliverTerminalUpLocal` for a hop; for JSON the same per-connection path `deliverUpLocal` takes, from the adapter's
+ already-parsed frame, stringified once) and is not published.
+- **Local is for good:** a connection is local or not for its whole life (`connId` is a new `randomUUID()` per attach
+ and never moves), so a local route never changes.
+- **Untargeted frames go local first too** *(after the independent review)*: an adapter frame for every client of the
+ machine is delivered to this process's clients first, then published with `originPid` (this process's own Redis
+ subscription skips it), as `managerWs` does. Otherwise a local viewer's targeted reply would overtake an event the
+ daemon sent before it. Observers and `nodeRpc` ignore `originPid` and still get the Redis copy.
+- A `targetConnId` that is not a string of 1–128 characters is published as before, never used as a route or a key.
+- **Receiver count:** `PUBLISH` answers how many subscriptions received a message. A per-minute log `terminal relay`
+ counts, per channel kind (`termup`, `up` targeted, `up` broadcast): frames delivered locally, frames published, and the
+ receivers they reached in total. That says how many pods each published frame wakes for nothing.
+
+### Phase 2 — publish only to the owning process (built with Phase 1; Ken, 2026-10-31)
+
+- On attach a pod records `conn:{connId} → PROCESS_ID` in Redis (`SET … PX 60 000`, renewed with the client heartbeat,
+ deleted on detach) and each process subscribes to its own `pod:{PROCESS_ID}` channel(s).
+- **One switch, in order** *(after the independent review; replaces a queue and a 250 ms lookup timeout)*: a
+ non-local connection's frames go to the machine's channels while its owner is looked up, then, if another process
+ owns it, to that process's channels; with no owner (old process, expired record, failed lookup) they stay on the
+ machine's channels. The switch keeps order: both channels are published from the one `pub` connection; the owner reads
+ `up:`/`uppod:` on `sub` and `termup:`/`termpod:` on `terminalSub`, one subscriber connection per kind; and the owner holds
+ a viewer of the machine, so it reads the machine's channels. Nothing waits for the lookup, so untargeted frames sent
+ after a targeted one never overtake it.
+- The receiver delivers with the existing functions: a hop with `deliverTerminalUpLocal`, a JSON message (the whole
+ `UpBusMsg`, *author's call during implementation*: a device connection's frames need the machine tag and the voice
+ queue `deliverUpLocal` applies) with `deliverUpLocal`. Both only reach that process's clients **of the message's
+ `machineId`** with that `connId`, so a daemon's frame never reaches another machine's connection. A `uppod:` message
+ without a string `targetConnId` is ignored.
+- **Kill switch:** `TERMINAL_POD_ROUTES=off` stops writing records and routing to `pod:` (on by default). With it off,
+ Phase 1 still runs; records already written expire within 60 s.
+
+### Risks (and what covers each)
+
+| Risk | Cover |
+|---|---|
+| A frame lost or reordered when the path changes | A route changes at most once, machine → owner, which keeps order (one publisher connection, one subscriber connection per kind); real-Redis tests of 1,000 frames across that switch |
+| A viewer on an old pod during the rolling deploy | No record from old pods → today's channels; new pods keep today's subscriptions |
+| A pod that crashed leaves records for 60 s | Frames to its `pod:` channel reach nobody, as frames to its dead sockets do today; its viewers reconnect elsewhere with a new `connId`, which gets a new record |
+| A daemon targeting another machine's connection | The receiver checks `connId` against its clients of the frame's `machineId` |
+| More Redis writes: a record per attach, renewed per heartbeat (Desktop re-attaches on every agent switch) | One `SET` per attach and per 25 s per connection, one pipeline per heartbeat; one `GET` per connection, once; at most 2,000 routes kept per adapter socket |
+| A bug that only shows across real pods | A real-Redis test (two subscriber connections standing for two processes, `REDIS_TEST_URL`), the kill switch, and the `terminal relay` log after release |
+| Little gain if most machines have viewers on one pod | `terminal relay` counts receivers per publish before and after; the kill switch turns Phase 2 off if it does not pay |
+
+
+## Left to the builder
+
+- Module layout (a small `terminalRoute.ts` beside `hub.ts` is one option), log and key names.
+- Phase 2's exact channel and key names (two channels, one per subscriber connection, as decided).
+
+## Tests
+
+- **Phase 1:** a local viewer gets a binary hop, a JSON terminal frame and an `e2e_welcome` with no Redis publish, in the
+ order sent, byte-identical to today; a remote viewer gets them through today's channels; a local connection that
+ detaches mid-stream drops the rest; the `terminal relay` log counts local, published and receivers per kind.
+- **Phase 2:** a remote viewer's frames go to `pod:{owner}` only and a third pod with another client of the
+ machine receives nothing; an unknown owner stays on today's channels for the connection's life; 1,000 frames for one
+ connection arrive once and in order, with the record appearing midway; re-attach on another pod routes the new
+ `connId` to its pod.
+- Existing hub, adapter, webWs and terminal E2E suites stay green.
+
+## Measuring
+
+After each release, from #2393's kinds and Phase 1's log: the **total** relay busy time per pod-minute across
+`adapter-ws *`, `bus terminal *`, `bus main up` and (Phase 2) `bus pod *`; Redis publishes per minute; receivers per
+published frame. **Done when** Phase 1 shows the local share delivered without Redis and total relay time falls, with no
+rise in terminal stall reports (`TERMINAL_RENDER_STALLED`); with Phase 2, receivers per published frame fall to about 1.
+If they were already about 1, Phase 2 is switched off (`TERMINAL_POD_ROUTES=off`) and removed in a later change.
+
+## Design (AGENTS.md rules)
+
+- **Growth:** Phase 1 stores nothing. Phase 2: one Redis key per open viewer connection, expiring 60 s after its last
+ heartbeat; one channel per process.
+- **Concurrency:** no locks; a connection's frames keep one sticky path, in send order.
+- **Read cost:** Phase 1 none. Phase 2: one Redis `GET` per connection, once, on the sending process.
+- **Scale:** at 10× viewers or pods, a frame still costs one local send or one publish to one pod; the per-machine fan-out
+ no longer grows with pods. Redis publish rate is the first thing to watch.
+- **Compatibility:** old pods write no record, so frames for their viewers fall back to today's channels; new pods keep
+ subscribing to those channels.
+- **Migration + rollback:** nothing stored beyond expiring keys; `TERMINAL_POD_ROUTES=off` turns Phase 2 off without a
+ deploy; roll back by deploying the previous release.
+- **Audience:** unchanged; a frame reaches only the connection it names, as today.
+- **Identity:** N/A because nothing here acts as or reads the machine owner's login.
+- **Failure:** Redis unreachable → local viewers still get frames; remote ones fail as today. A missing record falls
+ back to broadcast, never drops.
+- **Platforms:** backend only.