The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.
TypeScript 3d-modelingagent-frameworkagent-harnessai-agentscadclaude-codecodexcoding-agentsdeveloper-toolsesp32game-developmentmcp-servermulti-agentopen-hardwareopencodeopenharnesspcbroboticsterminaltmux
Isolate Claude Code and Codex screen interpretation (#1027)
Dee668048acdf5f
diff --git a/cli/AGENTS.md b/cli/AGENTS.mdindex 5d0969a46..2058f59b0 100644--- a/cli/AGENTS.md+++ b/cli/AGENTS.md@@ -25,7 +25,7 @@ they show as `harnessd`, `harnessd-core` and `harnessd-<process>` (`src/harnessd|---|---|---|| master | supervision only (`src/harnessd/`) | by `harness start`, `harness start -f`, or launchd or systemd after `harness service install` || core | sessions (`src/core/`) | always |-| engine-claude, engine-codex | history, last-turn readers, live transcript parsers and runtime profiles/catalogs | on demand: an attach or read for that engine |+| engine-claude, engine-codex | history, last-turn readers, live transcript parsers, runtime profiles/catalogs and screen interpretation | on demand: an attach or read for that engine || search | session search | always || viewers | the harness viewers, their remote streams and rendered surfaces, and the Store | always || edge | the shell service, workspaces, usage, the monitor, the project readers, the change-agent handoff, the recaps | always |diff --git a/cli/e2e/engineReaders.e2e.ts b/cli/e2e/engineReaders.e2e.tsindex 47b1192ed..463aaebed 100644--- a/cli/e2e/engineReaders.e2e.ts+++ b/cli/e2e/engineReaders.e2e.ts@@ -59,7 +59,7 @@ describe('Claude Code and Codex readers in their own processes', () => {expect(readerPid(d, 'claude')).toBeUndefined()expect(readerPid(d, 'codex')).toBeUndefined()for (const type of ['engine_history_page', 'engine_last_turn', 'engine_live_capabilities', 'engine_live_prepare',- 'engine_live_read', 'engine_live_part', 'engine_live_close', 'engine_live_forget', 'engine_runtime_capabilities', 'engine_runtime_read']) {+ 'engine_live_read', 'engine_live_part', 'engine_live_close', 'engine_live_forget', 'engine_runtime_capabilities', 'engine_runtime_read', 'engine_screen_capabilities', 'engine_screen_read']) {const answer = await c.request(type, { version: 1, session: { transcriptPath: '/etc/passwd' } })expect(answer.error).toBe('UNSUPPORTED')}@@ -112,6 +112,43 @@ describe('Claude Code and Codex readers in their own processes', () => {expect(alive(bound.processIdentity.pid)).toBe(true)})+ it.each(['claude', 'codex'] as const)('%s refuses terminal writes and retains its question while its screen worker is frozen', async engine => {+ const { d, c } = await fresh({ HARNESSD_SERVICE_HEARTBEAT_TIMEOUT_MS: '60000', HARNESSD_SERVICE_STOP_GRACE_MS: '100' })+ const agent = await create(d, c, engine)+ const asked = c.next(frame => frame.type === 'commander_question' && frame.agentId === agent.id, 30_000, 'permission question')+ c.send('message', { agentId: agent.id, content: '!permit printf screen-worker' })+ const question = (await asked).payload!+ const shaped = question.questions[0]+ const bound = JSON.parse(readFileSync(join(d.dataDir, 'registry.json'), 'utf8')).find((row: any) => row.agentId === agent.id)+ const pid = readerPid(d, engine)!, core = d.corePid(), from = c.frames.length+ const answer = async () => {+ const result = c.next(frame => frame.type === 'question_response_result' && frame.payload?.requestId === question.requestId, 10_000, 'answer result')+ c.send('question_response', { agentId: agent.id, requestId: question.requestId, answers: { [shaped.q]: shaped.options[2] } })+ return (await result).payload!+ }+ process.kill(pid, 'SIGSTOP')+ try {+ const started = Date.now()+ expect((await answer()).error).toBe('ANSWER_FAILED')+ expect(Date.now() - started).toBeLessThan(5_000)+ const refused = c.next(frame => frame.type === 'error' && frame.agentId === agent.id, 25_000, 'unreadable-screen refusal')+ c.send('message', { agentId: agent.id, content: 'must-not-enter-the-permission' })+ expect((await refused).payload?.message).toContain('screen could not be read')+ const pane = await d.capture(bound.tmuxPane)+ expect(pane).not.toContain('must-not-enter-the-permission')+ expect(pane).not.toContain('did not run printf screen-worker')+ expect(c.frames.slice(from).filter(frame => frame.type === 'commander_question_close' && frame.agentId === agent.id)).toHaveLength(0)+ expect((await rows(c)).some(row => row.id === agent.id)).toBe(true)+ expect(d.corePid()).toBe(core)+ } finally { process.kill(pid, 'SIGCONT') }+ const ended = c.next(frame => frame.type === 'turn_ended' && frame.agentId === agent.id, 30_000, 'declined permission')+ expect((await answer()).error).toBeUndefined()+ await ended+ await turn(c, agent, `screen-recovered-${engine}`)+ expect(alive(bound.processIdentity.pid)).toBe(true)+ expect(d.coresStarted()).toBe(1)+ })+it('contains a killed or frozen reader, bounds failed reads, and recovers without restarting core or either CLI', async () => {const { d, c } = await fresh({ HARNESSD_SERVICE_HEARTBEAT_TIMEOUT_MS: '4000', HARNESSD_SERVICE_STOP_GRACE_MS: '100' })const agents = await Promise.all((['claude', 'codex'] as const).map(engine => create(d, c, engine)))@@ -217,7 +254,11 @@ describe('Claude Code and Codex readers in their own processes', () => {expect(await history(c, agent)).toMatchObject({ error: 'ENGINE_UNAVAILABLE', retryable: true })const first = c.frames.lengthconst ended = c.next(frame => frame.type === 'turn_ended' && frame.agentId === agent.id, 60_000, 'queued turn after worker recovery')- c.send('message', { agentId: agent.id, content: 'worker-is-parked' })+ // A person can still use the independent CLI while its worker is parked.+ // Harness messages now require screen evidence and are withheld in an+ // outage (covered above); type through this fixture's private terminal.+ await d.tmux.run('send-keys', '-t', registered().tmuxPane, '-l', 'worker-is-parked')+ await d.tmux.run('send-keys', '-t', registered().tmuxPane, 'Enter')await until('the independent CLI to finish writing its turn', () => {const file = registered().transcriptPathreturn file && readFileSync(file, 'utf8').split('\n').some(line => {diff --git a/cli/src/architecture.spec.ts b/cli/src/architecture.spec.tsindex 6ca28dd62..dc6a1d103 100644--- a/cli/src/architecture.spec.ts+++ b/cli/src/architecture.spec.ts@@ -143,6 +143,9 @@ const SERVICE_MAY_IMPORT: Record<string, string> = {/** What is not the core's, by path: each goes to a service or its own process, in the plan's order. */const EDGE: RegExp[] = [+ /^engines\/(claude|codex)\/(screen|composer|activity|stoppedGoal)\.ts$/,+ /^lib\/(askQuestion|runtimeProfileController|composerScreen|teamWriteHold|messageHold|terminalActivity|codexTurnRecovery)\.ts$/,+ /^engines\/screens\.ts$/,// The pilot reader implementations and their host are never loaded by supervised core./^engines\/(worker\/process|transcripts|(claude|codex)\/(transcript|\w+ReaderProcess))\.ts$/,/^engines\/(runtime|(claude|codex)\/runtimeProfile)\.ts$/, /^lib\/runtimeProfile\.ts$/,@@ -216,6 +219,12 @@ describe('the daemon\'s shape', () => {}})+ it('screen transport and input authority do not load native screen implementations', () => {+ for (const entry of ['core/input.ts', 'core/questions.ts', 'core/engines/screens.ts', 'core/engines/screenTransport.ts']) {+ expect([...closureOf(entry).keys()].filter(file => /^engines\/(claude|codex)\//.test(file)), entry).toEqual([])+ }+ })+it('the gateway reaches the core only through core/api.ts: never a core module, the registry, cli.ts or the socket', () => {// It speaks to the core through GatewayPort and GatewayEvents alone, so that it can run in a process of// its own (step 10, R2) without taking any of the core with it.diff --git a/cli/src/core/AGENTS.md b/cli/src/core/AGENTS.mdindex fa4688c54..c576f414c 100644--- a/cli/src/core/AGENTS.md+++ b/cli/src/core/AGENTS.md@@ -33,10 +33,11 @@ questions. Everything else is a service ([../services/AGENTS.md](../services/AGE- `terminals/`: who controls a pane (the control lease), opening a terminal with a literal argv (`open.ts`),and the requests about a terminal itself (`requests.ts`: `terminal_info`, `theme_set`).- `engines/`: hook coordination, reader ports and live worker coordination. Claude Code/Codex- history, last-turn reads, live parsing and runtime profile interpretation/catalogs run in supervised workers; core passes snapshots,+ history, last-turn reads, live parsing, runtime profiles/catalogs and screen interpretation run in supervised workers; core passes snapshots,validates bounded replies, retains acknowledged cursors and owns binding/turn state. The explicitinline/older-master compatibility path is the only live parser or runtime profile implementation- composition in core. Core keeps accepted profiles and control authority; native picker drivers+ composition in core. Screen reads are bounded and fenced to the worker connection and session binding+ ([screen boundary](../../../docs/design/2026-10-08-engine-screen.md)). Core keeps accepted profiles and control authority; native picker driversremain to migrate. See [runtime profiles](../../../docs/design/2026-10-08-engine-runtime.md).- `input.ts`, `questions.ts`: messages into a pane; an agent's question and its answer.`deviceInput.ts`: the pane writer lock every write takes, and a device's queued turns behind it.diff --git a/cli/src/core/agents/close.spec.ts b/cli/src/core/agents/close.spec.tsindex 0b9424e1c..6a8573934 100644--- a/cli/src/core/agents/close.spec.ts+++ b/cli/src/core/agents/close.spec.ts@@ -1,3 +1,5 @@+import { readInlineScreen } from '../../testing/inlineScreen.js'+import { screenFor } from '../../engines/screens.js'import { afterEach, describe, expect, it, vi } from 'vitest'import { inspectCloseActivity, type CloseAgentServiceDeps } from '../../lib/closeAgentService.js'import { projectDisplayName, type RegisteredSession } from '../../lib/registry.js'@@ -17,6 +19,7 @@ const row = (over: Partial<RegisteredSession> = {}) =>function setup(over: Partial<ClosingDeps> = {}, advertised: RegisteredSession[] = []) {const deps: ClosingDeps = {+ readScreen: readInlineScreen,registry: { advertised: vi.fn(() => advertised) } as unknown as ClosingDeps['registry'],cleanupTabs: { refresh: vi.fn(async () => {}), isHidden: vi.fn(() => true), assertHidden: vi.fn(async () => {}) },watcher: { pollSession: vi.fn(async () => {}) } as unknown as ClosingDeps['watcher'],@@ -53,7 +56,7 @@ describe('closing agents no window shows', () => {expect(deps.captureTerminal).toHaveBeenCalledWith('a1', 80)expect(deps.sessionTurnState).toHaveBeenCalledWith('s1')expect(deps.openQuestions.has).toHaveBeenCalledWith('s1')- expect(inspectCloseActivity).toHaveBeenCalledWith(row(), 'screen', false, false)+ expect(inspectCloseActivity).toHaveBeenCalledWith(row(), screenFor('claude').inspect('screen'), false, false)})it('with no session yet, has no lines to read', async () => {diff --git a/cli/src/core/agents/close.ts b/cli/src/core/agents/close.tsindex bd7d6b202..60c7fd369 100644--- a/cli/src/core/agents/close.ts+++ b/cli/src/core/agents/close.ts@@ -1,3 +1,4 @@+import type { ScreenReader } from '../../lib/screenReader.js'/*** Closing agents that no window shows: the close service (now, once idle, after the task) and the* cleanup preview that lists what a close would take, each with what it is doing right now.@@ -12,6 +13,7 @@ import type { TerminalBackendCoordinator } from '../../lib/terminalBackendCoordiimport type { Watcher } from '../../watcher/watcher.js'export interface ClosingDeps {+ readScreen: ScreenReaderregistry: typeof registrycleanupTabs: Pick<OpenTabProtection, 'refresh' | 'isHidden' | 'assertHidden'>watcher: Pick<Watcher, 'pollSession'>@@ -26,7 +28,7 @@ export interface ClosingDeps {}export function createAgentClosing({- registry, cleanupTabs, watcher, captureTerminal, sessionTurnState, openQuestions, terminals, sessionCheckpoints,+ readScreen, registry, cleanupTabs, watcher, captureTerminal, sessionTurnState, openQuestions, terminals, sessionCheckpoints,stopAgent, announceSession,}: ClosingDeps) {const closeAgentService = new CloseAgentService({@@ -35,7 +37,7 @@ export function createAgentClosing({activity: async s => {if (s.sessionId) await watcher.pollSession(s.sessionId)const screen = await captureTerminal(s.agentId, 80)- return inspectCloseActivity(s, screen, sessionTurnState(s.sessionId), openQuestions.has(s.sessionId))+ return inspectCloseActivity(s, await readScreen(s, screen), sessionTurnState(s.sessionId), openQuestions.has(s.sessionId))},checkpoint: async (s, phase) => {const captured = phase === 'before' ? await terminals.captureRetained(s, { historyLines: 2000 }) : nulldiff --git a/cli/src/core/agents/discovery.ts b/cli/src/core/agents/discovery.tsindex fd42ababd..3e957f663 100644--- a/cli/src/core/agents/discovery.ts+++ b/cli/src/core/agents/discovery.ts@@ -9,7 +9,7 @@*/import { isTerminalEngine } from '../../engines/types.js'import type { AutonomousDeviceInput } from '../deviceInput.js'-import type { QuestionWatcher } from '../../lib/askQuestion.js'+import type { QuestionWatcher } from '../../lib/questionController.js'import { sameGridAssignment } from '../../lib/gridAssignment.js'import { sid } from '../../lib/log.js'import type { registry, RegisteredSession } from '../../lib/registry.js'diff --git a/cli/src/core/agents/retarget.spec.ts b/cli/src/core/agents/retarget.spec.tsindex b59cfd8be..ef00cffa6 100644--- a/cli/src/core/agents/retarget.spec.ts+++ b/cli/src/core/agents/retarget.spec.ts@@ -45,6 +45,7 @@ const agent = (over: Partial<RegisteredSession> = {}): RegisteredSession => ({function setup(row: RegisteredSession | null = agent(), over: Partial<RetargetDeps> = {}) {const release = vi.fn()const deps: RetargetDeps = {+ readScreen: async (session, capture) => ({ pane: inspectRuntimePane(session.engine, capture ?? ''), question: null, messageHold: null, teamHold: null, activity: null, busy: false, stoppedGoal: false }),purgeBusy: vi.fn(() => false),tmuxBackend: { clearEnv: vi.fn(async () => ({ state: 'succeeded' })) } as unknown as RetargetDeps['tmuxBackend'],registry: {diff --git a/cli/src/core/agents/retarget.ts b/cli/src/core/agents/retarget.tsindex 872b4aba6..8c2270a25 100644--- a/cli/src/core/agents/retarget.ts+++ b/cli/src/core/agents/retarget.ts@@ -24,7 +24,7 @@ import type { registry, RegisteredSession } from '../../lib/registry.js'import { bypassPermissionFor, restartAgent } from '../../lib/restartAgent.js'import type { RuntimeProfileManager } from '../../lib/runtimeProfile.js'import { parseRuntimeProfile } from '../../lib/runtimeProfileWire.js'-import { inspectRuntimePane } from '../../lib/runtimeProfileController.js'+import type { ScreenReader } from '../../lib/screenReader.js'import type { TerminalAgentReconciler } from '../../lib/terminalAgentReconciler.js'import { terminalRouteKey } from '../../lib/terminalRuntime.js'import type { TmuxRuntimeRef } from '../../lib/terminalTypes.js'@@ -39,6 +39,7 @@ type LaunchHelpers = ReturnType<typeof createLaunchHelpers>type RetargetAgent = NonNullable<BackendSocket['onRetargetAgent']>export interface RetargetDeps {+ readScreen: ScreenReader/** Whether a purge holds this agent (PurgeAgentService.busy). */purgeBusy: (agentId: string) => boolean | undefinedtmuxBackend: TmuxBackend | null@@ -59,7 +60,7 @@ export interface RetargetDeps {}export function createAgentRetargeter({- purgeBusy, tmuxBackend, registry, runtimeProfiles, launchOverridesDeps, captureTerminal, acquireTerminalControl,+ readScreen, purgeBusy, tmuxBackend, registry, runtimeProfiles, launchOverridesDeps, captureTerminal, acquireTerminalControl,relaunchOverrides, downgradedPermission, agentReconciler, restartJobs, paneSwapDeps, liveBypassPermission,announceSession, opencodeDb,}: RetargetDeps) {@@ -144,7 +145,7 @@ export function createAgentRetargeter({// the grid.const capture = await captureTerminal(session.agentId, 100)if (!capture) return { ok: false, error: 'TMUX_FAILED' }- if (!inspectRuntimePane(session.engine, capture).idle) return { ok: false, error: 'AGENT_BUSY' }+ if (!(await readScreen(session, capture))?.pane.idle) return { ok: false, error: 'AGENT_BUSY' }// Nothing may type into the pane while it is being replaced.if (restartJobs.busy(session.agentId)) return { ok: false, error: 'AGENT_BUSY' }const release = acquireTerminalControl(session.agentId)diff --git a/cli/src/core/agents/update.ts b/cli/src/core/agents/update.tsindex 4013703d2..a5b16b449 100644--- a/cli/src/core/agents/update.ts+++ b/cli/src/core/agents/update.ts@@ -10,7 +10,7 @@ import type { AgentFrame } from '../../lib/agentFrame.js'import type { CloseAgentService } from '../../lib/closeAgentService.js'import { preview, sid } from '../../lib/log.js'import type { registry, RegisteredSession } from '../../lib/registry.js'-import { RuntimeProfileControlError, type RuntimeProfileErrorCode } from '../../lib/runtimeProfileController.js'+import { RuntimeProfileControlError, type RuntimeProfileErrorCode } from '../../lib/runtimeControl.js'/** An `agent_update {opened: true}` for an agent opened less than this long ago is answered but not* stamped or broadcast: a person flicking between two tabs, or two apps opening the same agent atdiff --git a/cli/src/core/deviceInput.spec.ts b/cli/src/core/deviceInput.spec.tsindex fdce70610..e568187a9 100644--- a/cli/src/core/deviceInput.spec.ts+++ b/cli/src/core/deviceInput.spec.ts@@ -1,3 +1,4 @@+import { readInlineScreen } from '../testing/inlineScreen.js'import { CodexNormalizer } from '../engines/codex/normalizer.js'import { lineToEvents, newTurnState, type LiveEvent } from '../lib/normalize.js'import { randomUUID } from 'node:crypto'@@ -29,7 +30,7 @@ function fixture(engine: 'claude' | 'codex' | 'commandcode' = 'codex') {submit: (id, text, delivery) => controller.submit(id, text, delivery), cancelDelivery: id => controller.cancelDelivery(id),stop: async () => true, answer: async () => true, recent: () => [], emit: frame => events.push(frame),})- const legacy: SessionInputController = new SessionInputController({+ const legacy: SessionInputController = new SessionInputController({ readScreen: readInlineScreen,getSession: () => available ? session : undefined, validateRuntime: async () => available,inject: (id, text) => controller.legacyWrite(id, () => inject(id, text)),sendKey: (id, key) => controller.legacyWrite(id, () => sendKey(id, key)), capture, onError: vi.fn(),diff --git a/cli/src/core/deviceInput.ts b/cli/src/core/deviceInput.tsindex c6f5b8fa0..dcbf0be17 100644--- a/cli/src/core/deviceInput.ts+++ b/cli/src/core/deviceInput.ts@@ -12,7 +12,7 @@ import { createHash } from 'node:crypto'import type { RegisteredSession } from '../lib/registry.js'import type { SessionInputDelivery } from '../lib/sessionInput.js'import type { LiveEvent } from '../lib/normalize.js'-import { isMessageHold } from '../lib/messageHold.js'+import { isMessageHold } from '../lib/messageHolds.js'import { enterWithheldReason, type TerminalActionResult } from '../lib/terminalTypes.js'const NATIVE = new Set(['claude', 'codex'])diff --git a/cli/src/core/engines/screenTransport.spec.ts b/cli/src/core/engines/screenTransport.spec.tsnew file mode 100644index 000000000..1fd305c5c--- /dev/null+++ b/cli/src/core/engines/screenTransport.spec.ts@@ -0,0 +1,100 @@+import { afterEach, describe, expect, it, vi } from 'vitest'+import { screenFor } from '../../engines/screens.js'+import { engineScreenRequests } from '../../engines/worker/screenRequests.js'+import { SCREEN_CAPABILITIES, SCREEN_IN_FLIGHT, SCREEN_QUEUED, SCREEN_READ, SCREEN_REPLY_BYTES, SCREEN_WAIT_MS } from '../../engines/worker/screenProtocol.js'+import { createScreenTransport, type ScreenTransport } from './screenTransport.js'++const capture = '────────────\n❯\n────────────\n? for shortcuts'+const capability = { version: 1, screen: 1, engine: 'claude' }+const answer = { version: 1, answer: screenFor('claude').inspect(capture) }+function setup() {+ const call = vi.fn(async (_service: string, method: string, _payload: Record<string, unknown>, _wait: number): Promise<Record<string, unknown>> => method === SCREEN_CAPABILITIES ? capability : answer)+ const transport = createScreenTransport({ call }); transport.connected('engine-claude')+ return { transport, call, read: () => transport.read('claude', capture) }+}+afterEach(() => { vi.useRealTimers(); vi.restoreAllMocks() })+describe('screen transport', () => {+ it('negotiates each connection and decodes real worker evidence', async () => {+ const requests = engineScreenRequests('claude')+ let transport: ScreenTransport+ const call = vi.fn(async (service, method, payload) => {+ if (call.mock.calls.length === 1) transport.connected(service)+ return await requests[method]({ ...payload, requestId: `core-route-${call.mock.calls.length}` }, { owner: true, local: true })+ })+ transport = createScreenTransport({ call })+ expect(await transport.read('claude', capture)).toEqual(answer.answer)+ await transport.read('claude', capture)+ expect(call.mock.calls.map(c => c[1])).toEqual([SCREEN_CAPABILITIES, SCREEN_READ, SCREEN_READ])+ transport.disconnected('engine-claude'); transport.connected('engine-claude')+ await transport.read('claude', capture)+ expect(call.mock.calls.filter(c => c[1] === SCREEN_CAPABILITIES)).toHaveLength(2)+ transport.connected('other'); transport.disconnected('other')+ await expect(transport.read('other', capture)).rejects.toThrow('ENGINE_INVALID_REQUEST')+ await expect(transport.read('claude', 'x'.repeat(300_000))).rejects.toThrow('ENGINE_INVALID_REQUEST')+ })+ it('bounds requests, fences replaced connections, and releases slots', async () => {+ const t = setup(); await t.read()+ let finish!: (value: typeof answer) => void+ const pending = new Promise<typeof answer>(resolve => { finish = resolve })+ t.call.mockImplementation(() => pending)+ const tasks = Array.from({ length: SCREEN_IN_FLIGHT + SCREEN_QUEUED }, () => t.read())+ await expect(t.read()).rejects.toThrow('ENGINE_BUSY')+ t.transport.disconnected('engine-claude'); t.transport.connected('engine-claude')+ finish(answer)+ for (const task of tasks) await expect(task).rejects.toThrow('ENGINE_STALE_REPLY')+ t.call.mockImplementation(async (_s, method) => method === SCREEN_CAPABILITIES ? capability : answer)+ expect(await t.read()).toEqual(answer.answer)+ })++ it('serves a burst of pane polls fairly and expires queued work without sending it late', async () => {+ const t = setup(); await t.read()+ const burst = await Promise.all(Array.from({ length: 24 }, () => t.read()))+ expect(burst).toHaveLength(24)+ expect(burst.every(value => value.messageHold === null)).toBe(true)+ vi.useFakeTimers()+ t.call.mockClear().mockImplementation(() => new Promise(() => {}))+ const waiting = Array.from({ length: 24 }, () => t.read().catch(error => error.code))+ await vi.advanceTimersByTimeAsync(SCREEN_WAIT_MS + 1)+ expect(await Promise.all(waiting)).toEqual(Array(24).fill('ENGINE_UNAVAILABLE'))+ expect(t.call).toHaveBeenCalledTimes(SCREEN_IN_FLIGHT)+ })+ it('enforces one deadline across cold negotiation and a read, including a hung link', async () => {+ const t = setup(); let time = 10+ vi.spyOn(performance, 'now').mockImplementation(() => time)+ t.call.mockImplementation(async (_s, method, _p, wait) => {+ if (method === SCREEN_CAPABILITIES) { expect(wait).toBe(SCREEN_WAIT_MS); time += 800; return capability }+ expect(wait).toBe(200); return answer+ })+ await t.read()+ t.call.mockImplementation(async () => { time += SCREEN_WAIT_MS + 1; return answer })+ await expect(t.read()).rejects.toThrow('ENGINE_UNAVAILABLE')+ const noTime = setup(); let calls = 0+ vi.spyOn(performance, 'now').mockImplementation(() => ++calls === 1 ? 0 : SCREEN_WAIT_MS)+ await expect(noTime.read()).rejects.toThrow('ENGINE_UNAVAILABLE')+ expect(noTime.call).not.toHaveBeenCalled()+ vi.restoreAllMocks(); vi.useFakeTimers()+ const hung = setup(); hung.call.mockImplementation(() => new Promise(() => {}))+ const result = expect(hung.read()).rejects.toThrow('ENGINE_UNAVAILABLE')+ await vi.advanceTimersByTimeAsync(SCREEN_WAIT_MS)+ await result+ })+ it('rejects wrong capabilities, errors and malformed or oversized answers', async () => {+ for (const reply of [{ ...capability, screen: 2 }, { ...capability, engine: 'codex' }, { ...capability, version: 2 },+ { version: 1, error: 'unexpected' }, { version: 1, answer: {} }]) {+ const t = setup(); t.call.mockResolvedValue(reply)+ await expect(t.read()).rejects.toThrow('ENGINE_INVALID_REPLY')+ }+ for (const error of ['SERVICE_FAILED', 'SERVICE_UNAVAILABLE', 'ENGINE_BUSY']) {+ const t = setup(); t.call.mockResolvedValue({ version: 1, error })+ await expect(t.read()).rejects.toThrow(error === 'ENGINE_BUSY' ? error : 'ENGINE_UNAVAILABLE')+ }+ const t = setup(); await t.read()+ t.call.mockRejectedValueOnce(new Error('lost'))+ await expect(t.read()).rejects.toThrow('ENGINE_UNAVAILABLE')+ t.call.mockResolvedValueOnce({ version: 1, answer: {} })+ await expect(t.read()).rejects.toThrow('ENGINE_INVALID_REPLY')+ t.call.mockResolvedValueOnce({ ...answer, extra: 'x'.repeat(SCREEN_REPLY_BYTES) })+ await expect(t.read()).rejects.toThrow('ENGINE_INVALID_REPLY')+ expect(await t.read()).toEqual(answer.answer)+ })+})diff --git a/cli/src/core/engines/screenTransport.ts b/cli/src/core/engines/screenTransport.tsnew file mode 100644index 000000000..d47357e7a--- /dev/null+++ b/cli/src/core/engines/screenTransport.ts@@ -0,0 +1,75 @@+/** Core's screen port: plain snapshots, a single deadline, and connection-generation fencing. */+import type { ScreenReading } from '../../engines/facets/screen.js'+import { EngineReadError, readerEngine, READER_ERRORS, READER_SERVICES, type ReaderEngine } from '../../engines/worker/protocol.js'+import { screenReading, screenCapture, SCREEN_CAPABILITIES, SCREEN_IN_FLIGHT, SCREEN_QUEUED, SCREEN_READ, SCREEN_REPLY_BYTES,+ SCREEN_VERSION, SCREEN_WAIT_MS } from '../../engines/worker/screenProtocol.js'++export interface ScreenTransportDeps {+ call(service: string, method: string, payload: Record<string, unknown>, waitMs: number): Promise<Record<string, unknown>>+}++export function createScreenTransport(deps: ScreenTransportDeps) {+ const states = new Map(Object.values(READER_SERVICES).map(service => [service as string,+ { generation: 0, connected: false, capable: false, pending: 0, waiting: [] as Array<() => void> }]))+ const fail = (code: EngineReadError['code']): never => { throw new EngineReadError(code) }+ return {+ connected(service: string): void {+ const state = states.get(service)+ if (state) { state.generation++; state.connected = true; state.capable = false }+ },+ disconnected(service: string): void {+ const state = states.get(service)+ if (state) { state.generation++; state.connected = false; state.capable = false }+ },+ async read(engine: string, capture: string): Promise<ScreenReading> {+ if (!readerEngine(engine) || !screenCapture(capture)) fail('ENGINE_INVALID_REQUEST')+ const service = READER_SERVICES[engine as ReaderEngine], state = states.get(service)!+ const generation = state.generation + (state.connected ? 0 : 1)+ const deadline = performance.now() + SCREEN_WAIT_MS+ // A poll tick can capture dozens of panes together. Refusing every read+ // after the first eight starves the same later sessions on every tick.+ // FIFO admission is bounded, and spends the caller's original deadline.+ // Earlier active reads have the same hard budget and release their slots+ // even on a hung link; expired queued reads send nothing and release next.+ if (state.pending >= SCREEN_IN_FLIGHT) {+ if (state.waiting.length >= SCREEN_QUEUED) fail('ENGINE_BUSY')+ await new Promise<void>(resolve => { state.waiting.push(resolve) })+ } else state.pending+++ const call = async (method: string, payload: Record<string, unknown>): Promise<Record<string, unknown>> => {+ const remaining = Math.ceil(deadline - performance.now())+ if (remaining <= 0) fail('ENGINE_UNAVAILABLE')+ // Service startup has its own wait. Enforce one wall-clock budget here,+ // including a worker that never answers or a link that loses its timer.+ let timer: ReturnType<typeof setTimeout> | undefined+ let reply: Record<string, unknown>+ try {+ reply = await Promise.race([deps.call(service, method, { version: SCREEN_VERSION, ...payload }, remaining),+ new Promise<never>((_, reject) => { timer = setTimeout(() => reject(new EngineReadError('ENGINE_UNAVAILABLE')), remaining) })])+ } finally { clearTimeout(timer) }+ if (performance.now() > deadline) fail('ENGINE_UNAVAILABLE')+ if (reply.error === 'SERVICE_FAILED' || reply.error === 'SERVICE_UNAVAILABLE') fail('ENGINE_UNAVAILABLE')+ if (state.generation !== generation) fail('ENGINE_STALE_REPLY')+ if (reply.version !== SCREEN_VERSION) fail('ENGINE_INVALID_REPLY')+ if (reply.error !== undefined) fail(READER_ERRORS.includes(reply.error as EngineReadError['code']) ? reply.error as EngineReadError['code'] : 'ENGINE_INVALID_REPLY')+ return reply+ }+ try {+ if (!state.capable) {+ const capability = await call(SCREEN_CAPABILITIES, {})+ if (capability.screen !== SCREEN_VERSION || capability.engine !== engine) fail('ENGINE_INVALID_REPLY')+ state.capable = true+ }+ const reply = await call(SCREEN_READ, { capture })+ if (Buffer.byteLength(JSON.stringify(reply)) > SCREEN_REPLY_BYTES || !screenReading(reply.answer)) fail('ENGINE_INVALID_REPLY')+ return reply.answer as ScreenReading+ } catch (error) { throw error instanceof EngineReadError ? error : new EngineReadError('ENGINE_UNAVAILABLE') }+ finally {+ const next = state.waiting.shift()+ if (next) next()+ else state.pending--+ }+ },+ }+}++export type ScreenTransport = ReturnType<typeof createScreenTransport>diff --git a/cli/src/core/engines/screens.spec.ts b/cli/src/core/engines/screens.spec.tsnew file mode 100644index 000000000..df7694368--- /dev/null+++ b/cli/src/core/engines/screens.spec.ts@@ -0,0 +1,54 @@+import { describe, expect, it, vi } from 'vitest'+import { screenFor } from '../../engines/screens.js'+import type { RegisteredSession } from '../../lib/registry.js'+import { createScreens } from './screens.js'++const capture = '────────────\n❯\n────────────\n? for shortcuts'+const row = () => ({ agentId: 'agent', sessionId: 'session', engine: 'claude', active: true,+ runtimes: [], processIdentity: { pid: 10, startMarker: 'one' } }) as unknown as RegisteredSession+function setup() {+ let current: RegisteredSession | undefined = row()+ const answer = screenFor('claude').inspect(capture)+ const remote = vi.fn(async () => answer), inline = vi.fn(() => answer), handles = vi.fn(() => true)+ return { screen: createScreens({ resolve: () => current, transport: { read: remote }, inline, handles }),+ remote, inline, handles, answer, row: current, replace: (next?: RegisteredSession) => { current = next } }+}+describe('core screen evidence', () => {+ it('reads a bound worker without retaining or reusing its screen', async () => {+ const t = setup()+ expect(await t.screen.read(t.row, capture)).toEqual(t.answer)+ expect(await t.screen.question(t.row, capture)).toBeNull()+ expect(t.remote).toHaveBeenCalledTimes(2)+ expect(t.inline).not.toHaveBeenCalled()+ t.handles.mockReturnValue(false)+ expect(await t.screen.read(t.row, capture)).toEqual(t.answer)+ expect(t.inline).toHaveBeenCalledWith('claude', capture)+ t.inline.mockReturnValueOnce(undefined as never)+ expect(await t.screen.read(t.row, capture)).toBeNull()+ })+ it('refuses missing, oversized or foreign captures before consulting an adapter', async () => {+ const t = setup()+ expect(await t.screen.read(t.row, null)).toBeNull()+ expect(await t.screen.read(t.row, 'é'.repeat(256 * 1024))).toBeNull()+ expect(await t.screen.read({ ...t.row, agentId: '' }, capture)).toBeNull()+ expect(await t.screen.read({ ...t.row, sessionId: 'old' }, capture)).toBeNull()+ t.replace()+ expect(await t.screen.read(t.row, capture)).toBeNull()+ expect(t.remote).not.toHaveBeenCalled()+ })+ it('does not substitute inline parsing for a failed worker or close a question on failure', async () => {+ const t = setup(); t.remote.mockRejectedValue(new Error('worker stopped'))+ expect(await t.screen.read(t.row, capture)).toBeNull()+ await expect(t.screen.question(t.row, capture)).rejects.toThrow('ENGINE_UNAVAILABLE')+ expect(t.inline).not.toHaveBeenCalled()+ })+ it.each(['sessionId', 'engine', 'active', 'processIdentity', 'runtimes'] as const)('discards evidence after an in-place %s change', async field => {+ const t = setup()+ let finish!: (value: typeof t.answer) => void+ t.remote.mockImplementation(() => new Promise(resolve => { finish = resolve }))+ const read = t.screen.read(t.row, capture)+ Object.assign(t.row, { [field]: field === 'active' ? false : field === 'processIdentity' ? { pid: 20 } : field === 'runtimes' ? [{ paneId: 'new' }] : 'changed' })+ finish(t.answer)+ expect(await read).toBeNull()+ })+})diff --git a/cli/src/core/engines/screens.ts b/cli/src/core/engines/screens.tsnew file mode 100644index 000000000..d183cfe90--- /dev/null+++ b/cli/src/core/engines/screens.ts@@ -0,0 +1,41 @@+import type { ScreenReading } from '../../engines/facets/screen.js'+import type { AgentEngine } from '../../engines/types.js'+import { screenCapture } from '../../engines/worker/screenProtocol.js'+import type { RegisteredSession } from '../../lib/registry.js'+import type { ScreenReader } from '../../lib/screenReader.js'+import type { ScreenTransport } from './screenTransport.js'+import { EngineReadError } from '../../engines/worker/protocol.js'++export interface ScreenDeps {+ handles(engine: AgentEngine): boolean+ transport: Pick<ScreenTransport, 'read'>+ resolve(id: string): RegisteredSession | undefined+ inline(engine: AgentEngine, capture: string | null): ScreenReading | undefined+}++/** Snapshot scalars: registry records may be edited in place while a worker answers. */+function identity(session: RegisteredSession | undefined): string {+ return session ? JSON.stringify([session.agentId, session.sessionId, session.engine, session.active,+ session.registeredAt, session.boundAt, session.transcriptPath, session.tmuxPane,+ session.primaryRuntimeKey, session.runtimes, session.processIdentity]) : ''+}++export function createScreens(deps: ScreenDeps) {+ const read: ScreenReader = async (session, capture) => {+ if (capture !== null && !screenCapture(capture)) return null+ const key = identity(session), engine = session.engine, agentId = session.agentId+ if (!agentId || identity(deps.resolve(agentId)) !== key) return null+ try {+ const answer = deps.handles(engine) ? capture === null ? null : await deps.transport.read(engine, capture) : deps.inline(engine, capture)+ return identity(deps.resolve(agentId)) === key && answer ? answer : null+ } catch { return null }+ }+ return {+ read,+ async question(session: RegisteredSession, capture: string) {+ const screen = await read(session, capture)+ if (!screen) throw new EngineReadError('ENGINE_UNAVAILABLE')+ return screen.question+ },+ }+}diff --git a/cli/src/core/input.spec.ts b/cli/src/core/input.spec.tsindex fd1a8d31c..8d585ebb9 100644--- a/cli/src/core/input.spec.ts+++ b/cli/src/core/input.spec.ts@@ -1,3 +1,4 @@+import { readInlineScreen } from '../testing/inlineScreen.js'import { readdirSync, readFileSync } from 'node:fs'import { join } from 'node:path'import { afterEach, describe, expect, it, vi } from 'vitest'@@ -38,6 +39,7 @@ function deps(over: Partial<InputDeps> = {}) {const normalizer = { openTurn: vi.fn((text: string) => [{ type: 'turn_started', text }]) }let pane: string | null = READY_PANEconst base: InputDeps = {+ readScreen: readInlineScreen,resolve: (id) => agents.get(id),byAgent: (agentId) => agents.get(agentId),terminal: {@@ -150,7 +152,7 @@ describe('the session input controller\'s dependencies', () => {it('reads the pane again right before the Enter, and gives the reason not to press it, a popup of the message\'s own aside', async () => {const screens = [READY_PANE, CLAUDE_PERMISSION, READY_PANE, null, READY_PANE, READY_PANE, `────────────\n❯ /mo\n────────────\n /model Set the AI model`]const answers: Array<string | null> = []- const write = messageWriter({+ const write = messageWriter({ readScreen: deps().deps.readScreen,resolve: (id) => agents.get(id),terminal: {validateTerminal: vi.fn(async () => true),@@ -178,7 +180,7 @@ describe('the session input controller\'s dependencies', () => {const order: string[] = []const promptTyped = vi.fn((session: { agentId: string }, capture: string | null) => { order.push(`prompt ${session.agentId} ${capture === READY_PANE}`) })const screens = [READY_PANE, CLAUDE_PERMISSION]- const write = messageWriter({+ const write = messageWriter({ readScreen: deps().deps.readScreen,resolve: (id) => agents.get(id),terminal: {validateTerminal: vi.fn(async () => true),@@ -199,7 +201,7 @@ describe('the session input controller\'s dependencies', () => {for (const screen of [null, '✻ Welcome to Claude Code']) {let reads = 0const answers: Array<string | null> = []- const write = messageWriter({+ const write = messageWriter({ readScreen: deps().deps.readScreen,resolve: (id) => agents.get(id),terminal: {validateTerminal: vi.fn(async () => true),@@ -234,6 +236,18 @@ describe('the session input controller\'s dependencies', () => {expect(run.calls.filter((call) => call.startsWith('submit'))).toEqual(['submit a1 hi'])})+ it('holds team and device writes when an otherwise readable screen has no worker evidence', async () => {+ const run = deps({ readScreen: async () => null })+ const wired = sessionInputDeps(run.deps, () => lock(run.calls))+ expect(await wired.beforeTeamWrite!(agents.get('a1')!)).toBe('team_waiting_unavailable')+ expect(await wired.inject('a1', 'hello')).toMatchObject({ dispatch: 'not_started', reason: 'screen_unreadable' })+ expect(await wired.injectTeam!('a1', 'hello', 'delivery')).toMatchObject({ dispatch: 'not_started', reason: 'team_waiting_unavailable' })+ const device = deviceInputDeps(run.deps, () => ({ acquireControl: vi.fn(), submit: vi.fn(), cancelDelivery: vi.fn() }))+ expect(await device.isAwaitingUser!(agents.get('a1')!)).toBe(true)+ expect(run.deps.terminal.submitTerminalAction).not.toHaveBeenCalled()+ expect(run.deps.terminal.keyTerminalAction).not.toHaveBeenCalled()+ })+it('reports an error to the app and, in the device\'s words, to the dial', () => {const { deps: given } = deps()sessionInputDeps(given, () => lock([])).onError('s1', 'the pane is gone')diff --git a/cli/src/core/input.ts b/cli/src/core/input.tsindex 66bddacdf..97beebc79 100644--- a/cli/src/core/input.ts+++ b/cli/src/core/input.ts@@ -18,14 +18,16 @@ import { sid } from '../lib/log.js'import type { LiveEvent } from '../lib/normalize.js'import type { RegisteredSession } from '../lib/registry.js'import { SessionInputController, type SessionInputDelivery, type SessionInputDeps } from '../lib/sessionInput.js'-import { messageHold, passingHold } from '../lib/messageHold.js'+import { passingHold } from '../lib/messageHolds.js'+import type { ScreenReader } from '../lib/screenReader.js'+import type { ScreenReading } from '../engines/facets/screen.js'import { TERMINAL_LEASE_REFUSED, terminalActionNotStarted, type TerminalActionResult } from '../lib/terminalTypes.js'-import { teamWriteHold } from '../lib/teamWriteHold.js'import type { TerminalControl } from './terminals/control.js'type Frame = { type: string; agentId?: string; dbSessionId?: string; payload: Record<string, unknown> }export interface InputDeps {+ readScreen: ScreenReaderresolve: (id: string) => RegisteredSession | undefinedbyAgent: (agentId: string) => RegisteredSession | undefinedterminal: Pick<TerminalControl, 'captureTerminal' | 'validateTerminal' | 'submitTerminalAction' | 'keyTerminalAction' | 'pinTerminalControl'>@@ -48,7 +50,7 @@ export interface InputDeps {emit: (sessionId: string, events: LiveEvent[]) => void/** A prompt is about to be typed, and `capture` is the pane as read right before: what is on it belongs* to the turns before the one it starts (the question watcher's, lib/askQuestion.ts `notePrompt`). */- promptTyped?: (session: RegisteredSession, capture: string | null) => void+ promptTyped?: (session: RegisteredSession, capture: string | null, screen: ScreenReading) => void}/**@@ -64,8 +66,8 @@ export interface InputDeps {* reason. `submitTerminalAction` is called here and nowhere else in this file, and input.spec.ts keeps* it so.*/-export function messageWriter({ resolve, terminal: { captureTerminal, validateTerminal, submitTerminalAction }, promptTyped }: Pick<InputDeps, 'resolve' | 'terminal' | 'promptTyped'>) {- return async (id: string, text: string, hold?: (session: RegisteredSession, capture: string | null) => string | null): Promise<TerminalActionResult> => {+export function messageWriter({ readScreen, resolve, terminal: { captureTerminal, validateTerminal, submitTerminalAction }, promptTyped }: Pick<InputDeps, 'resolve' | 'terminal' | 'promptTyped' | 'readScreen'>) {+ return async (id: string, text: string, hold?: (session: RegisteredSession, screen: ScreenReading | null) => string | null): Promise<TerminalActionResult> => {const session = resolve(id)if (!session) return terminalActionNotStarted('terminal agent is unavailable')// The engine first, then its screen. A check of the engine can wait, seconds, for an answer: a probe@@ -75,16 +77,19 @@ export function messageWriter({ resolve, terminal: { captureTerminal, validateTe// not there is refused as a lease is, which the caller asks again.if (!await validateTerminal(session)) return terminalActionNotStarted(TERMINAL_LEASE_REFUSED)const capture = await captureTerminal(id)- const reason = hold?.(session, capture) ?? messageHold(session.engine, capture)+ const screen = await readScreen(session, capture)+ if (!screen) return terminalActionNotStarted(hold?.(session, null) ?? 'screen_unreadable')+ const reason = hold?.(session, screen) ?? screen.messageHoldif (reason) return terminalActionNotStarted(reason)- promptTyped?.(session, capture)+ promptTyped?.(session, capture, screen)return submitTerminalAction(id, text, {beforeEnter: async () => {// A read that came back empty, or a composer caught between frames, is asked again for a moment// (a re-attach can blank one capture): only what is on screen holds the Enter back, never a// read that failed once. Still not read after that, it is held, as nothing says it is safe.for (let tries = 1; ; tries++) {- const before = messageHold(session.engine, await captureTerminal(id))+ const next = await readScreen(session, await captureTerminal(id))+ const before = next ? next.messageHold : 'screen_unreadable'if (before === 'popup_open') return nullif (!before || !passingHold(before) || tries >= ENTER_CHECK_TRIES) return beforeawait new Promise((settle) => setTimeout(settle, ENTER_CHECK_RETRY_MS))@@ -108,6 +113,7 @@ export function sessionInputDeps(const writeMessage = messageWriter(deps)return {beforeSubmit: (id, text, tabId, deliveryId) => teams.prepare(id, text, tabId, deliveryId),+ readScreen: deps.readScreen,getSession: (id) => resolve(id),onDelivery: (event) => {device()?.delivery(event)@@ -115,7 +121,8 @@ export function sessionInputDeps(},beforeTeamWrite: async session => {const capture = await captureTerminal(session.agentId)- return teamWriteHold(session.engine, capture)+ const screen = await deps.readScreen(session, capture)+ return screen ? screen.teamHold : 'team_waiting_unavailable'},validateRuntime: validateTerminal,// Typed into the pane as it stands, a draft or a running turn included, under the pane's write lock.@@ -123,7 +130,7 @@ export function sessionInputDeps(// A team's turn waits for a ready composer, and for its delivery to still hold the pane.injectTeam: (id, text, deliveryId) => deviceInput().legacyWrite(id, async () => {if (!resolve(id)) return terminalActionNotStarted('team_waiting_unavailable')- return writeMessage(id, text, (session, capture) => teamWriteHold(session.engine, capture)+ return writeMessage(id, text, (session, screen) => (screen ? screen.teamHold : 'team_waiting_unavailable')?? (teams.canWrite(deliveryId) ? null : 'team_waiting_control'))}),sendKey: (id, key) => deviceInput().legacyWrite(id, () => keyTerminalAction(id, key)),@@ -166,7 +173,8 @@ export function deviceInputDeps(// Whatever a message is not typed into, the Device's waits for it to close, rather than be refused.isAwaitingUser: async session => {const pane = await captureTerminal(session.agentId)- return pane === null || messageHold(session.engine, pane) !== null+ const screen = await deps.readScreen(session, pane)+ return pane === null || !screen || screen.messageHold !== null},acquireControl: id => input().acquireControl(id, { forAnswer: true }),legacySubmit: (id, text, deliveryId) => input().submit(id, text, deliveryId),diff --git a/cli/src/core/main.ts b/cli/src/core/main.tsindex ca503e9f7..f11d78793 100644--- a/cli/src/core/main.ts+++ b/cli/src/core/main.ts@@ -1,3 +1,7 @@+import { createScreens } from './engines/screens.js'+import { createScreenTransport } from './engines/screenTransport.js'+import { legacyScreen } from '../lib/legacyScreen.js'+import { masterRunsEngineScreen } from '../harnessd/services.js'/*** The core's entry: `harness __run`. `runForeground` is the composition root, which builds the core's* modules, starts the services through `serviceHost` and wires them to the socket@@ -20,7 +24,6 @@ import { sqlitePreflightMessage } from '../lib/sqliteAvailability.js'import { warmLoginShellEnvironment } from '../lib/loginShellEnv.js'import type { AppSwarms } from '../cable/cableSession.js'import type { UnreadNotification } from '../lib/notificationRead.js'-import { terminalActivity } from '../lib/terminalActivity.js'import { registry, projectDisplayName, validTranscriptPath, type RegisteredSession } from '../lib/registry.js'import { engineSessionTitle } from '../lib/sessionTitle.js'import { machineNames } from '../lib/machineNames.js'@@ -180,7 +183,7 @@ import { agentTokenUsage } from '../lib/agentTokenUsage.js'import { LegacyRuntimeProfileManager, type RuntimeModelOption } from '../lib/runtimeProfileManager.js'import { createRuntimeProfiles } from './engines/runtimeProfiles.js'import { createRuntimeTransport } from './engines/runtimeTransport.js'-import { RuntimeProfileController } from '../lib/runtimeProfileController.js'+import { RuntimeProfileController } from '../lib/runtimeControl.js'import { installTimestampedConsole, sid, prepareLogFile, trimLogFile, LOG_CHECK_INTERVAL_MS } from '../lib/log.js'import { backendHttpBase } from '../lib/controlPlane.js'@@ -542,13 +545,20 @@ async function runForeground(session: AuthSession | null): Promise<void> {// An older master's explicit capability report selects compatibility. A failed worker never does.const isolatedLive = (engine: string): boolean => liveHosted&& readerEngine(engine) && outOfProcess.has(READER_SERVICES[engine])+ const screenHosted = masterRunsEngineScreen(process.env, process.ppid)+ const isolatedScreen = (engine: string): boolean => screenHosted && isolatedLive(engine)const isolatedRuntime = (engine: string): boolean => runtimeHosted && isolatedLive(engine)- const inline = KNOWN_SERVICES.some((name) => !outOfProcess.has(name)) || !runtimeHosted+ const inline = KNOWN_SERVICES.some((name) => !outOfProcess.has(name)) || !runtimeHosted || !screenHosted? await import('../services/inline.js') : nulllet serviceLinksRef: ServiceLinks | null = nullconst runtimeTransport = createRuntimeTransport({call: (service, type, payload, waitMs) => serviceLinksRef?.call(service, type, payload, waitMs) ?? Promise.resolve({ error: 'SERVICE_UNAVAILABLE' }),})+ const screenTransport = createScreenTransport({+ call: (service, type, payload, waitMs) => serviceLinksRef?.call(service, type, payload, waitMs) ?? Promise.resolve({ error: 'SERVICE_UNAVAILABLE' }),+ })+ const screens = createScreens({ handles: isolatedScreen, transport: screenTransport, resolve: id => registry.resolve(id),+ inline: (engine, capture) => readerEngine(engine) ? capture === null ? undefined : inline?.screenFor(engine).inspect(capture) : legacyScreen(engine, capture) })const runtimeProfiles = createRuntimeProfiles({legacy: new LegacyRuntimeProfileManager(engine => isolatedRuntime(engine) ? undefined : inline?.runtimeFor(engine)),handles: isolatedRuntime,@@ -676,7 +686,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {const session = registry.resolve(agentId)if (!session || (session.engine !== 'claude' && session.engine !== 'codex')) return nullconst screen = await terminals.capture(session, { mode: 'visible', ansi: false })- return terminalActivity(session.engine, screen.state === 'succeeded' ? screen.value : null)+ return (await screens.read(session, screen.state === 'succeeded' ? screen.value : null))?.activity?.label ?? null},// The devices' frames for the windows on this computer, never the cloud (createCoreApi checks their types).sendLocal: (frame) => {@@ -929,6 +939,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {})// Whether a turn is really working, beyond its transcript (core/turns/activity.ts).const activity = createTurnActivity({+ readScreen: screens.read,terminals,bySession: (sessionId) => registry.bySession(sessionId),sessionTurnOpen,@@ -1016,6 +1027,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {backend.sessionsProvider = history.sessionsList// Everything the core writes into a pane, and the device's pane lock (core/input.ts).const inputs = createInput({+ readScreen: screens.read,resolve: (id) => registry.resolve(id),byAgent: (agentId) => registry.byAgent(agentId),terminal: terminalControl,@@ -1034,7 +1046,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {// Reassigned further down (the funnel): always the current one.emit: (sessionId, events) => emitSessionEvents(sessionId, events),// Declared further down: read as a prompt is typed, never now.- promptTyped: (session, capture) => { if (session.sessionId) questionWatcher.notePrompt(session.sessionId, capture) },+ promptTyped: (session, capture, screen) => { if (session.sessionId) questionWatcher.notePrompt(session.sessionId, capture, screen.question) },})const input = inputs.inputconst deviceInput = inputs.deviceInput@@ -1054,6 +1066,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {// Questions an agent asks the person: shown on the dial and the window, answered from anywhere// (core/questions.ts).const asking = createQuestions({+ readQuestion: screens.question,resolve: (id) => registry.resolve(id),terminal: terminalControl,acquireTerminalControl,@@ -1199,6 +1212,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {engineReaders.connected(service)liveTransport.connected(service)runtimeTransport.connected(service)+ screenTransport.connected(service)if (service === 'gateway') gatewayLink?.connected()if (service === 'teams') teamsLink.on()if (service === 'devices') devicesLink.connected()@@ -1209,6 +1223,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {engineReaders.disconnected(service)liveTransport.disconnected(service)runtimeTransport.disconnected(service)+ screenTransport.disconnected(service)if (service === 'gateway') gatewayLink?.disconnected()if (service === 'devices') devicesLink.disconnected()if (service === 'wifi') wifiCore.stopped()@@ -1285,6 +1300,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {backend.models = () => ports.models ?? MODELS_OFFconst runtimeController = new RuntimeProfileController({+ readScreen: screens.read,manager: runtimeProfiles,getSession: (id) => registry.resolve(id),validateRuntime: validateTerminal,@@ -2180,6 +2196,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {// Moving a running agent onto a grid, or back to its own login (core/agents/retarget.ts).backend.onRetargetAgent = createAgentRetargeter({+ readScreen: screens.read,purgeBusy: (agentId) => backend.purgeAgentService?.busy(agentId),tmuxBackend,registry,@@ -2229,6 +2246,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {backend.purgeProvider = createPurgeRequest({ purgeAgentService: () => lifecycle.purgeAgentService, invalidateStorage: () => { void (ports.monitor ?? MONITOR_OFF).storage([], true) } })// Closing agents no window shows, and the cleanup preview (core/agents/close.ts).const closing = createAgentClosing({+ readScreen: screens.read,registry,cleanupTabs,watcher,diff --git a/cli/src/core/questions.spec.ts b/cli/src/core/questions.spec.tsindex fea9256dc..82c146d5e 100644--- a/cli/src/core/questions.spec.ts+++ b/cli/src/core/questions.spec.ts@@ -1,3 +1,4 @@+import { parseEngineQuestionPane } from '../engines/screens.js'import { afterEach, describe, expect, it, vi } from 'vitest'import type { RegisteredSession } from '../lib/registry.js'import { createQuestionResponse, createQuestions, type QuestionDeps } from './questions.js'@@ -9,6 +10,7 @@ const agents = new Map<string, RegisteredSession>([function setup(over: Partial<QuestionDeps> = {}) {const deps: QuestionDeps = {+ readQuestion: (session, capture) => parseEngineQuestionPane(session.engine, capture),resolve: (id) => agents.get(id),terminal: {captureTerminal: vi.fn(async () => 'pane'),diff --git a/cli/src/core/questions.ts b/cli/src/core/questions.tsindex dad6fbbec..fed6cc7eb 100644--- a/cli/src/core/questions.ts+++ b/cli/src/core/questions.ts@@ -7,7 +7,7 @@* Moved verbatim out of `runForeground` (the core boundary, step 6: docs/design/2026-10-03-harnessd.md).* The socket's two handlers stay bound there: `answer` (question_response) and `monitorActivity`.*/-import { AskQuestionController, QuestionWatcher, type QuestionAnswerPayload, type QuestionAnswerResult } from '../lib/askQuestion.js'+import { AskQuestionController, QuestionWatcher, type QuestionAnswerPayload, type QuestionAnswerResult } from '../lib/questionController.js'import type { AutonomousDeviceInput } from './deviceInput.js'import { preview, sid } from '../lib/log.js'import type { RegisteredSession } from '../lib/registry.js'@@ -16,6 +16,7 @@ import type { TerminalControl } from './terminals/control.js'type Frame = { type: string; agentId?: string; dbSessionId?: string; payload: Record<string, unknown> }export interface QuestionDeps {+ readQuestion: import('../lib/questionController.js').QuestionWatcherDeps['readQuestion']resolve: (id: string) => RegisteredSession | undefinedterminal: Pick<TerminalControl, 'captureTerminal' | 'submitTerminal' | 'keyTerminal'>/** Holds a pane, queue and terminal both, for the whole multi-step answer (Input.acquireTerminalControl). */@@ -30,7 +31,7 @@ export interface QuestionDeps {}export function createQuestions({- resolve, terminal, acquireTerminalControl, clients, agentIdFor, sessionTurnOpen, someoneCanAnswer, deviceInput,+ readQuestion, resolve, terminal, acquireTerminalControl, clients, agentIdFor, sessionTurnOpen, someoneCanAnswer, deviceInput,}: QuestionDeps) {const { captureTerminal, submitTerminal, keyTerminal } = terminal// AskUserQuestion bridge: mirrors the question to the device's question screen, and keys the device's@@ -38,6 +39,7 @@ export function createQuestions({const questions = new AskQuestionController({getSession: (id) => resolve(id),capture: captureTerminal,+ readQuestion,sendText: submitTerminal,sendKey: keyTerminal,acquireControl: acquireTerminalControl,@@ -75,6 +77,7 @@ export function createQuestions({const questionWatcher = new QuestionWatcher({getSession: (id) => resolve(id),capture: captureTerminal,+ readQuestion,hasDevice: () => someoneCanAnswer(),isDriving: (sessionId) => questions.isDriving(sessionId),onQuestion: (sessionId, requestId, shaped, detail) => {diff --git a/cli/src/core/transcripts/attach.ts b/cli/src/core/transcripts/attach.tsindex 7039c8bec..55eb0c638 100644--- a/cli/src/core/transcripts/attach.ts+++ b/cli/src/core/transcripts/attach.ts@@ -23,7 +23,7 @@ import { MuseNormalizer } from '../../engines/muse/normalizer.js'import { OpencodeReader } from '../../engines/opencode/reader.js'import { PiNormalizer } from '../../engines/pi/normalizer.js'import type { AgentEngine } from '../../engines/types.js'-import { pollsQuestions, type QuestionWatcher } from '../../lib/askQuestion.js'+import { pollsQuestions, type QuestionWatcher } from '../../lib/questionController.js'import { attachTranscript, type AttachRead } from '../../lib/attachTranscript.js'import { AttachTracker } from '../../lib/attachTracker.js'import type { WifiFeed } from '../wifi.js'diff --git a/cli/src/core/turns/activity.spec.ts b/cli/src/core/turns/activity.spec.tsindex 4e80755f4..448a6c307 100644--- a/cli/src/core/turns/activity.spec.ts+++ b/cli/src/core/turns/activity.spec.ts@@ -1,3 +1,4 @@+import { readInlineScreen } from '../../testing/inlineScreen.js'import { afterEach, describe, expect, it, vi } from 'vitest'import type { RegisteredSession } from '../../lib/registry.js'import { activityRuntimeKey } from '../../lib/runtimeActivity.js'@@ -11,6 +12,7 @@ const sessions = new Map([[live.sessionId, live], [idle.sessionId, idle], [shellfunction setup(over: Partial<TurnActivityDeps> = {}) {const deps: TurnActivityDeps = {+ readScreen: readInlineScreen,terminals: { capture: vi.fn(async () => ({ state: 'succeeded' as const, value: 'screen' })) },bySession: (sessionId) => sessions.get(sessionId),sessionTurnOpen: (sessionId) => sessionId === 's1',diff --git a/cli/src/core/turns/activity.ts b/cli/src/core/turns/activity.tsindex 94c984125..979463662 100644--- a/cli/src/core/turns/activity.ts+++ b/cli/src/core/turns/activity.ts@@ -1,3 +1,4 @@+import type { ScreenReader } from '../../lib/screenReader.js'/*** Turn activity: whether an agent is really working, beyond what its transcript says. A turn's* transcript can go quiet while the agent thinks or runs a long tool, so an open turn is checked against@@ -12,6 +13,7 @@ import type { TerminalBackendCoordinator } from '../../lib/terminalBackendCoordiimport { TurnActivity, type ActivityFrame } from '../../lib/turnActivity.js'export interface TurnActivityDeps {+ readScreen: ScreenReaderterminals: Pick<TerminalBackendCoordinator, 'capture'>bySession: (sessionId: string) => RegisteredSession | undefinedsessionTurnOpen: (sessionId: string) => boolean@@ -19,9 +21,10 @@ export interface TurnActivityDeps {drain: (sessionId: string) => Promise<void>}-export function createTurnActivity({ terminals, bySession, sessionTurnOpen, drain }: TurnActivityDeps) {+export function createTurnActivity({ readScreen, terminals, bySession, sessionTurnOpen, drain }: TurnActivityDeps) {const codexActivity = new CodexActivityReader()const runtimeActivity = new RuntimeActivityReader({+ readScreen,codex: session => codexActivity.read(session),capture: async session => {const screen = await terminals.capture(session, { mode: 'visible', ansi: true })diff --git a/cli/src/core/turns/cancel.ts b/cli/src/core/turns/cancel.tsindex 0cb3d9010..06a41c777 100644--- a/cli/src/core/turns/cancel.ts+++ b/cli/src/core/turns/cancel.ts@@ -7,7 +7,7 @@*/import type { CursorSubagentManager } from '../../engines/cursor/subagent.js'import { correlateAgentEvent } from '../../lib/agentEvent.js'-import type { QuestionWatcher } from '../../lib/askQuestion.js'+import type { QuestionWatcher } from '../../lib/questionController.js'import type { WifiFeed } from '../wifi.js'import type { TurnRecaps } from './recaps.js'import type { RegisteredSession } from '../../lib/registry.js'diff --git a/cli/src/core/turns/funnel.ts b/cli/src/core/turns/funnel.tsindex 30f4d972d..adb834793 100644--- a/cli/src/core/turns/funnel.ts+++ b/cli/src/core/turns/funnel.ts@@ -18,7 +18,7 @@ import type { TurnRecaps } from './recaps.js'import { deviceErrorText } from '../cardText.js'import { preview, sid } from '../../lib/log.js'import type { CursorNormalizer } from '../../engines/cursor/normalizer.js'-import type { QuestionWatcher } from '../../lib/askQuestion.js'+import type { QuestionWatcher } from '../../lib/questionController.js'import type { RegisteredSession } from '../../lib/registry.js'import type { SessionInputController } from '../../lib/sessionInput.js'import type { TurnActivity } from '../../lib/turnActivity.js'diff --git a/cli/src/engines/README.md b/cli/src/engines/README.mdindex 4b0e916b6..7049fce0f 100644--- a/cli/src/engines/README.md+++ b/cli/src/engines/README.md@@ -18,8 +18,9 @@ that stream.launch contracts, history/last-turn readers and hook behavior behind `Engine` (`engine.ts`). `registry.ts`selects those engines; `launches.ts` and `hooks.ts` expose their respective facets without loading historyreaders. Hook transport, process binding and event delivery stay with core. Claude Code and Codex history/last-turn reads,-live transcript parsing and runtime profile interpretation/catalogs run on demand in supervised workers;+live transcript parsing, runtime profiles/catalogs and screen interpretation run on demand in supervised workers;the other facets still run in core. Explicit inline mode and older masters retain inline implementations. See+[screen interpretation](../../../docs/design/2026-10-08-engine-screen.md),[runtime profiles](../../../docs/design/2026-10-08-engine-runtime.md) and the[reader pilot](../../../docs/design/2026-10-07-engine-readers.md) for limits and failure behavior. Theremaining facets and engines still use the shared paths below. This is an internal interface, not andiff --git a/cli/src/engines/claude/activity.ts b/cli/src/engines/claude/activity.tsnew file mode 100644index 000000000..8089403e0--- /dev/null+++ b/cli/src/engines/claude/activity.ts@@ -0,0 +1,22 @@+/** Native live indicator, read only from the current footer. */+export function activity(screen: string): { label: string; indicator: string } | null {+ const lines = screen.replace(/\x1b\][^\x07]*(?:\x07|\x1b\\)/g, '')+ .replace(/\x1b\[[0-?]*[ -/]*[@-~]/g, '').split(/\r?\n/)+ let end = lines.length+ // The expanded agent tree sits BELOW the prompt and can push the live+ // spinner out of the last 16 rows. Anchor to the actual input/footer pair,+ // retaining a short search above it instead of scanning old output.+ for (let i = lines.length - 1; i >= Math.max(0, lines.length - 64); i--) {+ if (/^\s*[❯›>](?:\s|$)/u.test(lines[i]) &&+ lines.slice(i + 1, i + 9).some(line => /shift\+tab to cycle|esc to interrupt/i.test(line))) {+ end = i+ break+ }+ }+ for (let i = end - 1; i >= Math.max(0, end - 16); i--) {+ const line = lines[i].trim()+ const match = /^[✢✳✶✻✽·*]\s+([\p{L}][\p{L}\p{N} '\u2019-]{0,55}(?:…|\.{3}))(?:\s+\([^\n]*\))?\s*$/u.exec(line)+ if (match) return { label: match[1].trim().replace(/…/g, '...'), indicator: line }+ }+ return null+}diff --git a/cli/src/engines/claude/composer.ts b/cli/src/engines/claude/composer.tsnew file mode 100644index 000000000..783e6f8d9--- /dev/null+++ b/cli/src/engines/claude/composer.ts@@ -0,0 +1,36 @@+import type { ComposerState } from '../facets/screen.js'+import { screenLines, typedToken } from '../kit/composer.js'++/** A row of Claude Code's rule: all `─`, or `─` at both ends with text set in it. */+const CLAUDE_RULE = /^─{3,}$|^─{3,}.*─{3,}$/++export function composerState(capture: string): ComposerState {+ const { lines } = screenLines(capture)+ let prompt = lines.findLastIndex((line) => /^❯(?:\s|$)/.test(line))+ let bottom = lines.findIndex((line, index) => index > prompt && CLAUDE_RULE.test(line.trim()))+ if (prompt < 1 || !CLAUDE_RULE.test(lines[prompt - 1].trim()) || bottom < 0) {+ // A draft taller than the pane: Claude Code does not cap the prompt's height outside its fullscreen+ // renderer (2.1.289 passes `maxVisibleLines` only there), so the box's top rule and its `❯` row+ // scroll off, and the pane shows the draft's further rows, indented under the `❯`, down to the+ // bottom rule. Every row above that rule is one of them; a dialog's are indented by one column, the+ // conversation's start at the edge.+ bottom = lines.findIndex((line) => CLAUDE_RULE.test(line.trim()))+ if (bottom < 1 || !lines.slice(0, bottom).every((line) => !line || line.startsWith(' '))) return 'absent'+ prompt = -1+ }+ const below = lines.slice(bottom + 1)+ // Nothing ruled under the box: a dialog of Claude Code's own is drawn ruled, in its place.+ if (below.some((line) => CLAUDE_RULE.test(line.trim()))) return 'absent'+ const draft = [prompt < 0 ? '' : lines[prompt].replace(/^❯\s?/, ''), ...lines.slice(prompt + 1, bottom).map((line) => line.trim())].join(' ').trim()+ const token = typedToken(draft)+ const rows = below.map((line) => line.trim()).filter(Boolean)+ if (token.startsWith('/') && rows.some((row) => /^\/[\w:.-]/.test(row))) return 'popup'+ if (token.startsWith('@')) {+ const typed = token.slice(1).toLowerCase()+ // A file or agent suggestion, by what is typed after the `@`; with nothing typed yet, any row but the+ // footer's own (its mode and its hints).+ if (rows.some((row) => typed ? row.replace(/^@/, '').toLowerCase().startsWith(typed) : !/^(?:\?|⏵|esc\b)/.test(row))) return 'popup'+ }+ return 'ready'+}+diff --git a/cli/src/engines/claude/screen.ts b/cli/src/engines/claude/screen.tsnew file mode 100644index 000000000..07180001b--- /dev/null+++ b/cli/src/engines/claude/screen.ts@@ -0,0 +1,53 @@+import { activity } from './activity.js'+import type { EngineScreen, PaneTakeover, PaneView } from '../facets/screen.js'+import { inspectPane, paneModal as readModal, stripAnsi } from '../kit/pane.js'+import { locateQuestionPane } from '../kit/questionPane.js'+import { assembleScreen } from '../kit/screen.js'+import { composerState } from './composer.js'++export function questionPane(capture: string): PaneView {+ const found = locateQuestionPane(capture)+ if (!found || composerState(capture.split('\n').slice(found.at + 1).join('\n')) !== 'absent') return null+ return found.view+}+export const inspectRuntimePane = (capture: string) => inspectPane('claude', capture, takeover)+export const paneModal = (capture: string) => readModal('claude', capture, takeover)+export const screen: EngineScreen = {+ inspect: capture => assembleScreen('claude', capture, inspectRuntimePane(capture), questionPane(capture),+ paneModal(capture), composerState(capture), activity(capture)),+}++function takeover(capture: string): PaneTakeover | null {+ const lines = stripAnsi(capture).replace(/\u00a0/g, ' ').split('\n')+ const bottom = lines.filter(line => line.trim()).slice(-3)+ const composer = lines.findLastIndex((line, index) => /^\s*❯/.test(line) && index > 0 && /^\s*[─━]{8,}\s*$/.test(lines[index - 1]))+ const shown = (...patterns: RegExp[]) => patterns.every(pattern => lines.findLastIndex(line => pattern.test(line)) > composer)+ if (claudeRewindMenuOpen(capture)) return 'rewind'+ if (bottom.some((line) => CLAUDE_TRANSCRIPT_FOOTER.test(line))) return 'transcript'+ if (bottom.some((line) => CLAUDE_HISTORY_SEARCH.test(line))) return 'search'+ if (shown(/Quick safety check: Is this a project you created or one you trust\?/, /^\s*(?:❯\s*)?Yes, I trust this folder\s*$/)) return 'trust'+ if (shown(/^\s*Select login method:\s*$/, /^\s*(?:❯\s*)?1\. Claude account with subscription\b/)) return 'sign_in'+ return null+}+/**+ * Claude Code's transcript view (ctrl+o): the prompt is hidden, and the footer row starts+ * `Showing detailed transcript · ctrl+o to toggle`, after `dialog waiting · ` when a dialog sits behind+ * it (2.1.289). It has no Enter, so a message typed there is lost. Esc, q or ctrl+c close it.+ */+const CLAUDE_TRANSCRIPT_FOOTER = /^\s*(?:dialog waiting · )?Showing detailed transcript\b/++/** Claude Code's prompt-history search (ctrl+r), under the prompt: `search prompts: <query>`, or+ * `no matching prompt: <query>` (2.1.289). */+const CLAUDE_HISTORY_SEARCH = /^\s*(?:search prompts|no matching prompt): /++const CLAUDE_REWIND_BODY = /^(?:Restore the code\b|Restore and fork\b|Confirm you want to restore\b|Nothing to rewind to\b)/++function claudeRewindMenuOpen(capture: string): boolean {+ const lines = stripAnsi(capture).replace(/\u00a0/g, ' ').split('\n').map((line) => line.trim())+ // The title, with its body a blank line under it, and no rule after it: the prompt the menu hides+ // comes back between two rules once it closes.+ return lines.some((line, index) => line === 'Rewind'+ && lines.slice(index + 1, index + 4).some((next) => CLAUDE_REWIND_BODY.test(next))+ && !lines.slice(index + 1).some((next) => /[─━]{8,}/u.test(next)))+}+diff --git a/cli/src/engines/codex/activity.ts b/cli/src/engines/codex/activity.tsnew file mode 100644index 000000000..b99af4809--- /dev/null+++ b/cli/src/engines/codex/activity.ts@@ -0,0 +1,12 @@+/** Native live indicator, read only from the current footer. */+export function activity(screen: string): { label: string; indicator: string } | null {+ const lines = screen.replace(/\x1b\][^\x07]*(?:\x07|\x1b\\)/g, '')+ .replace(/\x1b\[[0-?]*[ -/]*[@-~]/g, '').split(/\r?\n/)+ const end = lines.length+ for (let i = end - 1; i >= Math.max(0, end - 16); i--) {+ const line = lines[i].trim()+ const match = /^[•◦⠁-⣿]\s+(.{1,60}?)\s+\([^\n]*\besc to interrupt\b[^\n]*\)\s*$/i.exec(line)+ if (match) return { label: match[1].trim().replace(/…/g, '...'), indicator: line }+ }+ return null+}diff --git a/cli/src/engines/codex/composer.ts b/cli/src/engines/codex/composer.tsnew file mode 100644index 000000000..676ba7536--- /dev/null+++ b/cli/src/engines/codex/composer.ts@@ -0,0 +1,32 @@+import type { ComposerState } from '../facets/screen.js'+import { screenLines, startsBold, typedToken } from '../kit/composer.js'++/** Codex's composer row: `›` (or `»`) at the left edge, never a numbered row of a picker. */+const CODEX_COMPOSER = /^[›»](?:\s|$)/+const CODEX_PICKER_ROW = /^[›»]\s*\d+\.\s/++export function composerState(capture: string): ComposerState {+ const { lines, raw } = screenLines(capture)+ const composer = lines.findLastIndex((line) => CODEX_COMPOSER.test(line) && !CODEX_PICKER_ROW.test(line))+ // Bold, on a row of its own under a blank one: not a line of the conversation, nor a disabled composer.+ if (composer < 0 || !startsBold(raw[composer]) || (composer > 0 && lines[composer - 1].trim())) return 'absent'+ // The draft's further lines, then a blank row and the footer: nothing more is drawn under Codex's+ // composer, so more is some other screen under a line of the conversation.+ let end = composer + 1+ while (end < lines.length && lines[end].trim()) end+++ const footer = lines.slice(end).filter((line) => line.trim())+ if (footer.length > 3) return 'absent'+ const draft = [lines[composer].replace(/^[›»]\s?/, ''), ...lines.slice(composer + 1, end).map((line) => line.trim())].join(' ').trim()+ const token = typedToken(draft)+ // A voice send was refused as popup_open even though the recording had transcribed. Looking through+ // twenty conversation rows also matched command examples and old menu hints. A popup is the block+ // immediately above the composer; a slash menu has a selected row and edits the initial /command.+ let popupEnd = composer - 1+ while (popupEnd >= 0 && !lines[popupEnd].trim()) popupEnd--+ let popupStart = popupEnd+ while (popupStart >= Math.max(0, composer - 20) && lines[popupStart].trim()) popupStart--+ const above = lines.slice(popupStart + 1, popupEnd + 1)+ if (draft.startsWith('/') && token.startsWith('/') && above.some((line) => /^›\s+\/[\w-]+(?:\s{2,}\S|$)/.test(line))) return 'popup'+ if (token.startsWith('@') && above.some((line) => /\benter\/tab insert · esc close\b/.test(line))) return 'popup'+ return 'ready'+}diff --git a/cli/src/engines/codex/screen.ts b/cli/src/engines/codex/screen.tsnew file mode 100644index 000000000..5276eefff--- /dev/null+++ b/cli/src/engines/codex/screen.ts@@ -0,0 +1,62 @@+import { stoppedGoal } from './stoppedGoal.js'+import { activity } from './activity.js'+import type { EngineScreen, PaneTakeover, PaneView } from '../facets/screen.js'+import { inspectPane, paneModal as readModal, stripAnsi } from '../kit/pane.js'+import { locateQuestionPane } from '../kit/questionPane.js'+import { assembleScreen } from '../kit/screen.js'+import { composerState } from './composer.js'++export function questionPane(capture: string): PaneView {+ const found = locateQuestionPane(capture)+ if (!found || composerState(capture.split('\n').slice(found.at + 1).join('\n')) !== 'absent') return null+ return withCodexLabels(found.view)+}+export const inspectRuntimePane = (capture: string) => inspectPane('codex', capture, takeover, /\bplan mode\b/i)+export const paneModal = (capture: string) => readModal('codex', capture, takeover)+export const screen: EngineScreen = {+ inspect(capture) {+ const pane = inspectRuntimePane(capture)+ return assembleScreen('codex', capture, pane, questionPane(capture), paneModal(capture),+ composerState(capture), activity(capture), stoppedGoal(capture, pane))+ },+}++function withCodexLabels(view: PaneView): PaneView {+ if (!view || view.kind !== 'question') return view+ return {+ ...view,+ rows: view.rows.map((r) => ({ ...r, label: r.label.split(/\s{2,}/)[0].trim() || r.label })),+ }+}++function takeover(capture: string, currentUi: string): PaneTakeover | null {+ const lines = stripAnsi(capture).replace(/\u00a0/g, ' ').split('\n')+ const bottom = lines.filter(line => line.trim()).slice(-3)+ const composer = lines.findLastIndex(line => /^\s*›/.test(line) && !/^\s*›\s*\d+\.\s/.test(line))+ const shown = (...patterns: RegExp[]) => patterns.every(pattern => lines.findLastIndex(line => pattern.test(line)) > composer)+ if (CODEX_TRANSCRIPT_BROWSING.test(currentUi)) return 'rewind'+ if (CODEX_PAGER_HEADER.test(lines.find((line) => line.trim()) ?? '')) return 'transcript'+ if (bottom.some((line) => CODEX_SEARCH_FOOTER.test(line))) return 'search'+ if (shown(CODEX_TRUST_QUESTION, CODEX_TRUST_ROW)) return 'trust'+ if (shown(/^\s*Update available\b/, CODEX_UPDATE_ROW)) return 'update'+ if (shown(/^\s*[›>]?\s*1\. Try new model\s*$/, /^\s*[›>]?\s*2\. Use existing model\s*$/)) return 'model'+ if (shown(/^\s*[›>]?\s*1\. Sign in with ChatGPT\s*$/)) return 'sign_in'+ return null+}+/** The header of Codex's pager over the whole pane, its top row: its transcript (ctrl+t) when it is not+ * on the composer's screen (pager_overlay/transcript.rs). Closed with q or ctrl+t; Esc browses prompts. */+const CODEX_PAGER_HEADER = /^\/ T R A N S C R I P T(?: \/)*\s*$/++/** Codex's search footers: through its prompt history (`reverse-i-search: … enter accept · esc cancel`,+ * chat_composer/history_search.rs) and through its transcript (`Find: …`, transcript_view/search.rs). */+const CODEX_SEARCH_FOOTER = /^\s*(?:reverse-i-search:|Find: )/++/** Codex's trust question, as 0.160 words it (onboarding/trust_directory.rs) and as 0.147 did. */+const CODEX_TRUST_QUESTION = /Trust this folder\? Codex can read, edit, and run files here|Do you trust the contents of this directory\?/+const CODEX_TRUST_ROW = /^\s*[›>]?\s*1\. (?:Trust and continue|Yes, continue|Open restricted|Open existing task)\s*$/++/** Codex's update prompt (update_prompt.rs): its first row runs the update. */+const CODEX_UPDATE_ROW = /^\s*[›>]?\s*1\. Update now \(runs /++const CODEX_TRANSCRIPT_BROWSING = /^\s*Browsing(?: transcript)?(?:\s+·|\s*$)/m+diff --git a/cli/src/engines/codex/stoppedGoal.ts b/cli/src/engines/codex/stoppedGoal.tsnew file mode 100644index 000000000..e07c42dd5--- /dev/null+++ b/cli/src/engines/codex/stoppedGoal.ts@@ -0,0 +1,20 @@+/** Some Codex goal interruptions never write task_complete/turn_aborted to the+ * rollout. An open transcript is then only historical evidence of work. The+ * live footer is one positive idle reading, never a synthetic turn end. */+import { stripVTControlCharacters } from 'node:util'+import type { PaneInspection } from '../facets/screen.js'++/** Read only the UI below the current empty composer, never output/history.+ * An empty composer alone says nothing: Codex accepts drafts while working. */+export function stoppedGoal(screen: string, pane: PaneInspection): boolean {+ if (!screen || !pane.idle) return false+ const lines = stripVTControlCharacters(screen).split(/\r?\n/)+ const prompt = lines.findLastIndex(line => /^\s*›(?:\s|$)/u.test(line))+ if (prompt < 0) return false+ // A live turn can still display the previous goal state during a redraw.+ if (lines.slice(-16).some(line => /\besc to interrupt\b/i.test(line))) return false+ const footer = lines.slice(prompt + 1).filter(line => line.trim())+ return footer.length <= 4+ && footer.some(line => /\bGoal (?:stalled|paused) \(\/goal resume\)\s*$/.test(line))+ && footer.some(line => /\? for shortcuts\b/.test(line))+}diff --git a/cli/src/engines/engine.ts b/cli/src/engines/engine.tsindex f8cef261c..707058fc0 100644--- a/cli/src/engines/engine.ts+++ b/cli/src/engines/engine.ts@@ -1,3 +1,4 @@+import type { EngineScreen } from './facets/screen.js'import type { ProcessEngine } from './types.js'import type { EngineTranscript } from './facets/transcript.js'import type { EngineLaunch } from './facets/launch.js'@@ -13,4 +14,5 @@ export interface Engine {readonly hooks: EngineHooksreadonly live: EngineLivereadonly runtime: EngineRuntime+ readonly screen: EngineScreen}diff --git a/cli/src/engines/facets/screen.ts b/cli/src/engines/facets/screen.tsnew file mode 100644index 000000000..c939d4132--- /dev/null+++ b/cli/src/engines/facets/screen.ts@@ -0,0 +1,89 @@+/** A read-only interpretation of one captured terminal screen. */+/** Device-facing question shape — byte-for-byte the hosted runtime’s `commanderQuestions()` output. */+export interface ShapedQuestion {+ key: string+ q: string+ options: string[]+ multi: boolean+ /** Observed free-text editor; never inferred for permission prompts. */+ canText?: boolean+}++export interface QuestionRow {+ /** The digit to press. */+ number: string+ /** Option text with any `[ ]` checkbox prefix removed. */+ label: string+ checked: boolean+ /**+ * Set only when the dialog does NOT number its rows: the row is then reached by walking to it and+ * pressing Enter, and `number` carries its INDEX rather than a key.+ *+ * The direction is a property of the DIALOG, not of the engine. OpenCode draws BOTH a numbered ask+ * dialog and — for permissions — the horizontal prompt kilo inherited from it, so keying by engine+ * would send digits into a dialog that numbers nothing and select nothing at all.+ */+ walk?: 'right' | 'down'+}++export interface QuestionView {+ kind: 'question'+ permission?: boolean+ /**+ * The dialog is on screen but its top — the question and its first rows — is scrolled out of the pane+ * (Codex's request_user_input keeps the footer anchored and lets a short pane cut the top off; measured+ * in a four-pane window: rows 3–5 and the footer visible, "1." and the question gone). Enough to know a+ * dialog is STILL OPEN — the watcher must not close it — and not enough to announce as a question.+ */+ partial?: boolean+ /** The footer says a digit only highlights and Enter commits (`enter to submit answer` — Codex's+ * request_user_input). Absent where one digit selects and submits, which is every other dialog. */+ enterSubmits?: boolean+ question: string+ rows: QuestionRow[]+ multi: boolean+ /** The "Type something." row, when the dialog offers free text. */+ typeRow: QuestionRow | null+ /**+ * A permission prompt's WHOLE dialog, every line as painted (ANSI stripped), from its frame to its last+ * row. `question` is one line clipped for a device's screen; a command that wraps — `npm test &&` on one+ * line, `git push` on the next — is only whole here. What the pair brain's floor reads (pair/classify.ts),+ * and part of the dialog's fingerprint, so two prompts that differ below their first line are two ids.+ */+ dialog?: string+}++export interface ReviewView {+ kind: 'review'+ submitRow: string+}++export type PaneView = QuestionView | ReviewView | null++export interface PaneInspection {+ idle: boolean+ plan: boolean+ dialog: boolean+ draft: boolean+}+++export type ComposerState = 'ready' | 'popup' | 'absent'+export type PaneTakeover = 'rewind' | 'transcript' | 'search' | 'trust' | 'update' | 'model' | 'sign_in'+export type PaneModal = PaneTakeover | 'permission' | 'menu' | null+export type MessageHold = 'permission_open' | 'question_open' | 'menu_open' | 'rewind_picker_open' | 'transcript_open'+ | 'search_open' | 'trust_open' | 'update_prompt_open' | 'model_prompt_open' | 'sign_in_open' | 'popup_open'+ | 'prompt_hidden' | 'screen_unreadable'+export interface FoundDialog { view: QuestionView | ReviewView; at: number }+export interface ScreenReading {+ activity: { label: string; indicator: string } | null+ busy: boolean+ stoppedGoal: boolean+ pane: PaneInspection+ question: PaneView+ messageHold: MessageHold | null+ teamHold: string | null+}+export interface EngineScreen {+ inspect(capture: string): ScreenReading+}diff --git a/cli/src/engines/kit/composer.ts b/cli/src/engines/kit/composer.tsnew file mode 100644index 000000000..dc70c12f1--- /dev/null+++ b/cli/src/engines/kit/composer.ts@@ -0,0 +1,29 @@+import { stripVTControlCharacters } from 'node:util'++export function screenLines(capture: string): { lines: string[]; raw: string[] } {+ const raw = capture.split('\n')+ const lines = raw.map((line) => stripVTControlCharacters(line).replace(/ /g, ' ').trimEnd())+ while (lines.length && !lines.at(-1)) { lines.pop(); raw.pop() }+ return { lines, raw }+}++/** Whether the first character of a row is drawn bold: the SGR in force when it is written. */+export function startsBold(row: string): boolean {+ let bold = false+ const sgr = /^\u001b\[([0-9;:]*)m/+ let rest = row+ for (let match = sgr.exec(rest); match; match = sgr.exec(rest)) {+ for (const code of match[1].split(/[;:]/)) {+ if (code === '1') bold = true+ else if (code === '' || code === '0' || code === '22') bold = false+ }+ rest = rest.slice(match[0].length)+ }+ return bold+}++/** The word being typed at the end of a draft, where a `/command` or an `@mention` opens its popup. */+export function typedToken(draft: string): string {+ return draft.split(/\s/).at(-1) ?? ''+}+diff --git a/cli/src/engines/kit/pane.ts b/cli/src/engines/kit/pane.tsnew file mode 100644index 000000000..360f447fb--- /dev/null+++ b/cli/src/engines/kit/pane.ts@@ -0,0 +1,400 @@+import type { AgentEngine } from '../types.js'+import type { PaneInspection, PaneTakeover } from '../facets/screen.js'++type Takeover = (capture: string, currentUi: string) => PaneTakeover | null+export function stripAnsi(value: string): string {+ return value.replace(/\u001b\[[0-9;:]*[A-Za-z]/g, '')+}++/**+ * The SGR attributes an escape sequence actually sets, with extended-colour arguments removed.+ *+ * `38`/`48`/`58` introduce a colour whose ARGUMENTS are numbers, not attributes: `5;<n>` for 256 and+ * `2;<r>;<g>;<b>` for truecolor. Without this skip a truecolor foreground (`38;2;…`) reads as SGR 2+ * — dim — and a line the user had typed would be mistaken for an empty composer, which is the one+ * direction this must never be wrong in: the pane would read idle and be respawned under a draft.+ */+function sgrAttributes(params: readonly string[]): string[] {+ const attributes: string[] = []+ for (let index = 0; index < params.length; index += 1) {+ const code = params[index]+ if (code !== '38' && code !== '48' && code !== '58') {+ attributes.push(code)+ continue+ }+ const kind = params[index + 1]+ index += kind === '5' ? 2 : kind === '2' ? 4 : 1+ }+ return attributes+}++/**+ * The background an SGR sequence sets, as a comparable string — or null if it sets none.+ *+ * Needed because a composer box is identified by the colour it PAINTS, and the parameters that+ * choose one are positional: `48;5;<n>` and `48;2;<r>;<g>;<b>` carry their arguments inline, so the+ * arguments have to be consumed rather than read as further attributes. `38`/`58` are skipped the+ * same way for the same reason — a truecolor FOREGROUND ends in numbers that would otherwise parse+ * as a background code of their own.+ *+ * SGR 0 counts: a reset puts the default background back, which is a change like any other.+ */+function sgrBackground(params: readonly string[]): string | null {+ for (let index = 0; index < params.length; index += 1) {+ const code = params[index]+ if (code === '48') {+ const kind = params[index + 1]+ return params.slice(index, index + (kind === '5' ? 3 : kind === '2' ? 5 : 2)).join(';')+ }+ if (code === '49' || /^(?:4[0-7]|10[0-7])$/.test(code)) return code+ if (code === '0' || code === '') return '49'+ if (code === '38' || code === '58') {+ const kind = params[index + 1]+ index += kind === '5' ? 2 : kind === '2' ? 4 : 1+ }+ }+ return null+}++/**+ * The span of a gutter line the BOX ITSELF paints — everything after the marker was wrong.+ *+ * A TUI draws one row of the terminal at a time, so anything floating OVER the composer lands on the+ * composer's own lines. OpenCode's `Getting started` card is exactly that: a panel on the right-hand+ * side, and its rows are written after the composer's on the same lines. Reading to end-of-line took+ * the card's text as something the user had typed —+ *+ * ┃ <121 spaces of composer> <sidebar> Connect provider /connect+ *+ * so `draft` was true over an EMPTY composer, the pane never read idle, and every model switch came+ * back AGENT_BUSY. Measured on a live pane (`harness-opencode-*`, 2026-09-15) with nothing typed in.+ * Fourth variant of the family above: the marker is found, the strip is dropped, and then the wrong+ * COLUMNS are read.+ *+ * The boundary is the box's own background. Everything a box paints is painted in it, so the+ * interior runs from the background set right after the marker to the next background change — in+ * the capture above, back to the pane's `48;2;10;10;10` where the box ends and the sidebar begins.+ *+ * Two deliberate conservatisms, both pointing the same way — a real draft must never be read as an+ * empty prompt, because that respawns the engine under text the user typed:+ * * A background that appears only AFTER visible text is not the interior's. It is a highlight+ * drawn mid-draft, and treating it as the boundary would cut the draft off at its own styling.+ * * A line that sets no background at all is not clipped. Plain-text panes (and every engine that+ * paints no box) keep the whole line, exactly as before.+ */+function gutterBoxInterior(line: string, markerIndex: number): string {+ const after = line.slice(markerIndex + 1)+ const sgr = /\u001b\[([0-9;:]*)m/g+ let interior: string | null = null+ let end = after.length+ let match: RegExpExecArray | null+ while ((match = sgr.exec(after)) !== null) {+ const background = sgrBackground(match[1].split(/[;:]/))+ if (background === null) continue+ if (interior === null) {+ if (stripAnsi(after.slice(0, match.index)).trim().length > 0) break+ interior = background+ continue+ }+ if (background !== interior) { end = match.index; break }+ }+ return after.slice(0, end)+}++/**+ * Is the text after the composer marker the PLACEHOLDER, rather than something the user typed?+ *+ * Read from the STYLING, never the words: the placeholder sentence differs per engine and changes+ * between versions, but every one of these TUIs greys its placeholder out somehow and none of them+ * styles a draft the same way.+ *+ * Two attributes, because two are in use. SGR 2 (dim) is what claude, codex and devin draw; SGR 3+ * (italic) is what hermes draws — its `Ask anything, or type / for commands…` is italic over a+ * 256-colour gold (`ESC[3m ESC[38;5;136m`) and carries no dim at all. Looking only for dim made+ * EVERY hermes pane look like a pane with a draft in it, so `idle` was false for the agent's whole+ * life and every retarget and model switch came back AGENT_BUSY over an engine sitting at an empty+ * prompt. Same failure the comment on `promptMarker` warns about, one step further along: the+ * marker was found, and then the placeholder was not recognised as one.+ */+function hasPlaceholderSgr(value: string): boolean {+ return [...value.matchAll(/\u001b\[([0-9;:]*)m/g)]+ .some((match) => {+ const attributes = sgrAttributes(match[1].split(/[;:]/))+ return attributes.includes('2') || attributes.includes('3')+ })+}++/**+ * Is this SGR foreground a MUTED one — the grey a TUI writes hints and placeholders in?+ *+ * Returns null when the sequence sets no foreground, so a caller can keep the colour already in+ * effect rather than treating "no change" as a change.+ *+ * Grey is judged by the colour itself: equal channels, and dark enough to read as secondary next to+ * the near-white a composer draws real text in. OpenCode's placeholder is `38;2;128;128;128` and its+ * own text `38;2;255;255;255`; the 256-colour ramp (232–255) and SGR 90 are the same intent spelled+ * differently.+ */+function sgrMutedForeground(params: readonly string[]): boolean | null {+ for (let index = 0; index < params.length; index += 1) {+ const code = params[index]+ if (code === '38') {+ const kind = params[index + 1]+ if (kind === '2') {+ const [r, g, b] = [params[index + 2], params[index + 3], params[index + 4]].map(Number)+ if (![r, g, b].every(Number.isFinite)) return null+ return r === g && g === b && r >= 0x40 && r <= 0xb0+ }+ if (kind === '5') {+ const n = Number(params[index + 2])+ if (!Number.isFinite(n)) return null+ // 8 is bright black; 232–255 is the grey ramp, whose middle is the secondary tone.+ return n === 8 || (n >= 236 && n <= 250)+ }+ return null+ }+ if (code === '90') return true+ if (code === '39' || code === '0' || code === '') return false+ if (/^(?:3[0-7]|9[1-7])$/.test(code)) return false+ if (code === '48' || code === '58') {+ const kind = params[index + 1]+ index += kind === '5' ? 2 : kind === '2' ? 4 : 1+ }+ }+ return null+}++/**+ * Is EVERY visible character here written in a muted colour — i.e. is this a hint rather than a draft?+ *+ * The third placeholder styling this module has had to learn, after dim (claude, codex, devin) and+ * italic (hermes). OpenCode draws `Ask anything… "What is the tech stack of this project?"` as a plain+ * TRUECOLOR GREY with no dim and no italic attribute at all, so [hasPlaceholderSgr] saw nothing and a+ * brand-new pane read as a pane with a draft in it: never idle, and every model switch on an agent+ * with no messages yet refused as AGENT_BUSY over an empty composer. Measured on 1.18.31.+ *+ * ⚠️ EVERY character, not any — which is what keeps this from swallowing a real draft. A composer+ * writes what the user typed in its normal near-white; one such character anywhere means this is+ * text, whatever grey hint may be sitting beside it. Spaces are not evidence of either and carry no+ * colour worth reading, so they are skipped.+ */+function allVisibleTextIsMuted(value: string): boolean {+ const sgr = /\u001b\[([0-9;:]*)m/g+ let muted: boolean | null = null+ let cursor = 0+ let sawText = false+ let match: RegExpExecArray | null+ const segment = (text: string): boolean => {+ if (!text.trim()) return true+ sawText = true+ return muted === true+ }+ while ((match = sgr.exec(value)) !== null) {+ if (!segment(value.slice(cursor, match.index))) return false+ const next = sgrMutedForeground(match[1].split(/[;:]/))+ if (next !== null) muted = next+ cursor = match.index + match[0].length+ }+ if (!segment(value.slice(cursor))) return false+ return sawText+}++/**+ * The glyph each CLI puts in front of its composer. They do not share one: cursor uses `→`, devin `❭`+ * (U+276D — close to, but not, claude's `❯` U+276F), and pi draws no marker at all. Getting this wrong is+ * not cosmetic — with no prompt found the pane never reads idle, and every switch is refused as BUSY.+ */+function promptMarker(engine: AgentEngine): RegExp {+ if (engine === 'cursor') return /→/u+ if (engine === 'devin') return /[❭›❯]/u+ if (engine === 'opencode') return /┃/u+ return /[›❯]/u+}++export function inspectPane(engine: AgentEngine, capture: string, takeover: Takeover = () => null, planPattern = /\bplan mode on\b/i): PaneInspection {+ if (engine === 'opencode') return inspectGutterBoxPane(capture, true)+ if (engine === 'copilot') return inspectGutterBoxPane(capture, false)+ if (engine === 'grok') return inspectGrokPane(capture)+ const rawLines = capture.split('\n')+ const marks = promptMarker(engine)+ const promptIndex = latestPromptLine(rawLines, marks)+ const prompt = promptIndex >= 0 ? rawLines[promptIndex] : ''+ const currentUi = currentPaneUi(rawLines, promptIndex)+ const dialog = DIALOG_UI.test(currentUi) || takeover(capture, currentUi) !== null+ const plan = planPattern.test(currentUi)+ const marker = stripAnsi(prompt).search(marks)+ let visible = marker >= 0 ? stripAnsi(prompt).slice(marker + 1).replace(/\u00a0/g, ' ').trim() : ''+ const rawMarker = prompt.search(marks)+ const rawTail = rawMarker >= 0 ? prompt.slice(rawMarker + 1) : ''+ const placeholder = hasPlaceholderSgr(rawTail)+ if (placeholder) visible = ''+ const draft = visible.length > 0+ return { idle: !!prompt && !dialog && !draft, plan, dialog, draft }+}++/** The latest composer prompt line; picker rows use the same glyphs, but numbered rows are not prompts. */+function latestPromptLine(rawLines: string[], marks: RegExp): number {+ return rawLines.findLastIndex((line) => {+ const visible = stripAnsi(line)+ const marker = visible.search(marks)+ return marker >= 0 && !/^\s*\d+\.\s/.test(visible.slice(marker + 1))+ })+}++/**+ * Old picker/plan text can remain in tmux history. Only UI below the latest prompt belongs to the+ * current interaction; if no prompt is visible, inspect the whole capture as a conservative fallback.+ */+function currentPaneUi(rawLines: string[], promptIndex: number): string {+ return stripAnsi((promptIndex >= 0 ? rawLines.slice(promptIndex) : rawLines).join('\n')).replace(/\u00a0/g, ' ')+}++export function paneModal(engine: AgentEngine, capture: string, readTakeover: Takeover = () => null): PaneTakeover | 'permission' | 'menu' | null {+ const rawLines = capture.split('\n')+ const currentUi = currentPaneUi(rawLines, latestPromptLine(rawLines, promptMarker(engine)))+ const takeover = readTakeover(capture, currentUi)+ if (takeover) return takeover+ // Below the prompt line: that line is the composer, or the echo of the last message, and either can+ // hold anything a person typed.+ if (PERMISSION_UI.test(currentUi.slice(currentUi.indexOf('\n') + 1))) return 'permission'+ return DIALOG_UI.test(currentUi.replace(/Starting MCP servers?/gi, '')) ? 'menu' : null+}++const DIALOG_UI = /Select Model(?: and Effort)?|Select Reasoning Level|Advanced Reasoning|Available models|Models matching|Edit Parameters|Type to filter.*Tab to edit|Esc to go back|Press enter to confirm|Do you want to proceed|Allow this action|permission required|Starting MCP servers?/i++/** The approval prompts among them. */+/**+ * An approval prompt's question. Claude Code asks "Do you want to proceed?", "Do you want to make this+ * edit to <file>?", "Do you want to allow Claude to fetch this content?", "Do you want to <verb> <target>?"+ * for the other tools, and "Would you like to proceed?" over a plan (2.1.289); Codex asks "Would you like+ * to run the following command?" and the like. The rows under it are read too (askQuestion.ts), but a+ * row wrapped in a narrow pane breaks that reading, and the question alone must still hold a message:+ * Enter approves the first row. Asked only below the prompt line, so never of what a person typed.+ */+const PERMISSION_UI = /\bDo you want to\b|\bWould you like to (?:proceed|run|make|apply|allow)\b|Allow this action|permission required/i++/** The rule a gutter-box composer is closed with: `╹▀▀▀▀…`. */+const GUTTER_BOX_RULE = /[─▀▁▔]{8,}/u++/**+ * A composer drawn as a BOX with a `┃` gutter — OpenCode's and Copilot's, which are the same shape.+ *+ * The generic reader cannot read either one. OpenCode's `┃` does not stop at the text: the last+ * gutter line is the box's own STATUS strip — `Build · <model> · <account>`, sitting directly on+ * the rule that closes the box — so taking the last line carrying the marker reads what the BOX says+ * about itself as something the user typed. Copilot fails one step earlier: its composer carries no+ * `›`/`❯` at all, while every message the user has ALREADY SENT is echoed into the transcript as+ * `❯ <text>` in full brightness, so the generic reader walks back to the last of those and reads a+ * sent message as a draft. Either way the pane was permanently "holding a draft": never idle, and+ * every retarget and model switch refused as AGENT_BUSY over an empty composer. Third variant of the+ * failure hermes had — there the placeholder was not recognised, here the wrong line is read.+ *+ * [hasStatusStrip] is the one thing that differs between the two, and it is OpenCode's alone. The+ * strip is identified by WHERE it is — the gutter line the closing rule sits under — rather than by+ * what it says, which is three variable fields that change with the model, the mode and the account.+ * When no rule follows, no strip is assumed and every gutter line counts as composer: a layout we do+ * not recognise must not silently swallow a line the user typed in. Copilot's box has no strip, so+ * its single gutter line IS the composer and dropping it would read a real draft as an empty prompt+ * — the one direction this must never be wrong in.+ *+ * Every gutter line is checked, not just the last: a long message wraps down the box, and reading+ * one line of it would call a full composer empty.+ *+ * A dialog needs no special case in either engine: both REPLACE the composer with the picker or the+ * permission prompt, so no gutter is on screen at all and the pane reads not-idle from the empty run+ * below rather than from [DIALOG_UI].+ */+function inspectGutterBoxPane(capture: string, hasStatusStrip: boolean): PaneInspection {+ const rawLines = capture.split('\n')+ const marks = /┃/u+ // The LAST contiguous run of gutter lines. Earlier runs are previous composers still in scrollback+ // — the message the user already sent is not a draft.+ const gutter: number[] = []+ for (let index = rawLines.length - 1; index >= 0; index -= 1) {+ if (marks.test(stripAnsi(rawLines[index]))) { gutter.unshift(index); continue }+ if (gutter.length > 0) break+ }+ if (gutter.length === 0) return { idle: false, plan: false, dialog: false, draft: false }+ const last = gutter[gutter.length - 1]+ const closed = last + 1 < rawLines.length && GUTTER_BOX_RULE.test(stripAnsi(rawLines[last + 1]))+ const composer = hasStatusStrip && closed ? gutter.slice(0, -1) : gutter+ const currentUi = stripAnsi(rawLines.slice(gutter[0]).join('\n')).replace(/\u00a0/g, ' ')+ const dialog = DIALOG_UI.test(currentUi)+ const draft = composer.some((index) => {+ const line = rawLines[index]+ const visibleMarker = stripAnsi(line).search(marks)+ if (visibleMarker < 0) return false+ const rawMarker = line.search(marks)+ // The box's interior, not the rest of the row: a panel floating over the composer writes into+ // these same lines, and its text is not a draft. See [gutterBoxInterior]. With no marker in the+ // RAW line there is no interior to bound — read the whole row, the direction that keeps a draft.+ const interior = rawMarker < 0+ ? stripAnsi(line).slice(visibleMarker + 1)+ : gutterBoxInterior(line, rawMarker)+ if (hasPlaceholderSgr(interior) || allVisibleTextIsMuted(interior)) return false+ return stripAnsi(interior).replace(/\u00a0/g, ' ').trim().length > 0+ })+ return { idle: !dialog && !draft, plan: /\bplan mode on\b/i.test(currentUi), dialog, draft }+}++/**+ * The context readout Pi's footer always carries — `0.0%/500k`, `1.5%/200k`. Present whether or not the+ * model advertises a thinking ladder, and drawn by the normal view alone, which is what makes it the+ * honest "the footer is on screen" signal. No `g` flag, so `test` carries no `lastIndex` between calls.+ */+++/**+ * Grok draws its composer as a ROUNDED BOX, and the generic reader counts the box's own right edge+ * as something the user typed.+ *+ * Its empty composer is one line — `│ ❯` … padding … `│` — inside `╭─…─╮` above and+ * `╰─… Grok 4.6 (xhigh) ─╯` below. The shared reader takes everything after `❯`, strips ANSI, trims,+ * and finds `│`: one character, non-empty, so `draft` was true. Measured on a real pane+ * (`harness-grok-*`, 2026-09-09) with nothing typed into it: `visible after marker = "│"`, `draft =+ * true`, `idle = false`. Not a transient — the edge is drawn on every frame, so idle was false for+ * the agent's whole life and EVERY model switch came back AGENT_BUSY over an empty prompt. Same+ * family as the hermes and gutter-box bugs above; here the culprit is the box, not the placeholder.+ *+ * So the composer is the box's INTERIOR: the run of `│` body lines, read between their first and+ * last `│` rather than from the marker to end-of-line. Every body line is checked, not just the one+ * carrying `❯` — a long message wraps down the box and reading one line of it would call a full+ * composer empty.+ *+ * ⚠️ `│` is also Grok's footer separator (`Shift+Tab:mode │ Ctrl+x:shortcuts`), which sits BELOW the+ * box. It is excluded by requiring a body line to both open and close with `│`, which the footer —+ * one separator, no edges — never does.+ */+function inspectGrokPane(capture: string): PaneInspection {+ const rawLines = capture.split('\n')+ const visibleLines = rawLines.map((line) => stripAnsi(line).replace(/ /g, ' '))+ // The LAST box in the capture. Earlier ones are composers already sent, still in scrollback.+ const bottom = visibleLines.findLastIndex((line) => /^\s*╰[─╌]*.*╯\s*$/u.test(line))+ const top = bottom < 0+ ? -1+ : visibleLines.slice(0, bottom).findLastIndex((line) => /^\s*╭[─╌]+╮\s*$/u.test(line))+ // No recognisable box is "we cannot tell", which the caller reads as busy — the safe direction.+ // Respawning a pane whose layout we do not understand could discard a draft.+ if (top < 0 || bottom - top < 2) return { idle: false, plan: false, dialog: false, draft: false }+ const currentUi = visibleLines.slice(top).join('\n')+ const dialog = DIALOG_UI.test(currentUi)+ const draft = rawLines.slice(top + 1, bottom).some((line, offset) => {+ const visible = visibleLines[top + 1 + offset]+ const first = visible.indexOf('│')+ const last = visible.lastIndexOf('│')+ // Both edges, and not the same character: anything else is not a body line of this box.+ if (first < 0 || last <= first) return false+ const rawMarker = line.search(/[›❯]/u)+ if (rawMarker >= 0 && hasPlaceholderSgr(line.slice(rawMarker + 1))) return false+ // The marker is chrome, not content — it is printed whether or not anything was typed.+ return visible.slice(first + 1, last).replace(/[›❯]/u, '').trim().length > 0+ })+ return { idle: !dialog && !draft, plan: /\bplan mode on\b/i.test(currentUi), dialog, draft }+}++/**+ * Pi draws no composer marker, so idleness is read from its layout instead: the composer is the band+ * between the last two `───` rules, and the footer (`minimax/minimax-m3 • high`) is only rendered by the+ * normal view. A non-empty band is a draft the user is still typing — injecting there would splice a+ * slash command into their sentence.+ */diff --git a/cli/src/engines/kit/questionPane.ts b/cli/src/engines/kit/questionPane.tsnew file mode 100644index 000000000..4feeebd43--- /dev/null+++ b/cli/src/engines/kit/questionPane.ts@@ -0,0 +1,395 @@+import type { FoundDialog, PaneView, QuestionRow, QuestionView } from '../facets/screen.js'+import { earlierDialogEnd, PERMISSION_FOOTER_RE, QUESTION_FOOTER_RE } from '../../lib/dialogEnd.js'++// Rows that exist in every dialog but can never BE an answer.+const CHAT_ROW = /^chat about (this|these)$/i+// Claude writes "Type something.", Command Code "Type something..." — one trailing dot or three, and+// missing it costs the free-text row: a voice answer would have nowhere to go and the row would be+// offered on the device as if it were a real option.+// Claude "Type something.", Command Code "Type something...", OpenCode "Type your own answer", Hermes+// "Other (type your answer)". Same row in every dialog: it opens an editor instead of choosing, so it must+// never be offered to the device as a selectable label.+const TYPE_ROW = /^(type something\.{0,3}|type your own( answer)?|other \(type your (own|answer)\))$/i++export function stripAnsi(value: string): string {+ return value.replace(/\u001b\[[0-9;:]*[A-Za-z]/g, '')+}++function norm(value: string): string {+ return value.toLowerCase().replace(/\s+/g, ' ').replace(/[.…]+$/, '').trim()+}++export function parseRow(line: string): QuestionRow | null {+ // Consume indentation once: two adjacent whitespace runs backtrack across+ // every possible split on the padded non-option lines in terminal captures.+ const m = /^[❯›>]?\s*(\d+)\.\s+(.+?)\s*$/.exec(line.trimStart())+ if (!m) return null+ const raw = m[2]+ const box = /^\[([^\]])\]\s*(.*)$/.exec(raw)+ return {+ number: m[1],+ label: (box ? box[2] : raw).trim(),+ checked: !!box && box[1].trim() !== '',+ }+}++/**+ * Read the CURRENT dialog off a pane capture. The capture includes scrollback (old dialogs, old plan+ * text), so everything is anchored to the LAST footer line — the only marker the live dialog always+ * paints at the bottom of the screen.+ */+/** "❯ 1. Submit" immediately followed by "2. Cancel" — Command Code's review screen, which carries no+ * other marker. Returns the Submit row's number and where it sits, or null. */+function findSubmitPair(lines: string[]): { row: string; index: number } | null {+ for (let i = lines.length - 1; i >= 1; i--) {+ const row = parseRow(lines[i])+ if (!row || !/^submit$/i.test(row.label)) continue+ const next = parseRow(lines[i + 1] ?? '')+ if (next && /^cancel$/i.test(next.label)) return { row: row.number, index: i }+ }+ return null+}++/** The tab bar of a footer-less dialog: "● Loại game | ◯ Review". Returns its LAST occurrence, or -1. */+function findTabBarDialog(lines: string[]): number {+ for (let i = lines.length - 1; i >= 0; i--) {+ const line = lines[i].trim()+ if (!/^[●◯✔✓]/.test(line) || !line.includes('|')) continue // answered tabs turn into ✔+ // Only a tab bar with a numbered list under it is a question; the same glyphs appear in prose.+ for (let j = i + 1; j < lines.length && j - i <= 8; j++) {+ const row = parseRow(lines[j])+ if (row?.number === '1') return i+ }+ }+ return -1+}++/** Read a dialog that has no footer: question first, then the numbered rows, top-down from the tab bar. */+function parseDownward(lines: string[], tabBar: number): PaneView {+ let question = ''+ const rows: QuestionRow[] = []+ let checkbox = false+ for (let i = tabBar + 1; i < lines.length; i++) {+ const row = parseRow(lines[i])+ if (row) {+ if (/^\s*[❯›>]?\s*\d+\.\s+\[/.test(lines[i])) checkbox = true+ rows.push(row)+ continue+ }+ // Between the tab bar and row 1 sits the question; after the rows start, plain lines are the option+ // descriptions and must not overwrite it.+ const line = lines[i].trim()+ if (!question && rows.length === 0 && line && !/^[─━-]{6,}$/.test(line)) question = line+ }+ if (!rows.length) return null+ const answerable = rows.filter((r) => !CHAT_ROW.test(r.label) && !TYPE_ROW.test(r.label))+ return {+ kind: 'question',+ question,+ rows: answerable,+ multi: checkbox,+ typeRow: rows.find((r) => TYPE_ROW.test(r.label)) ?? null,+ }+}++// \u2500\u2500 permission prompts \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500++/**+ * A permission prompt is a QUESTION whose options are the approval choices \u2014 the shape amp and kilo+ * already ship (`engines/amp/askQuestion.ts`, `engines/kilo/askQuestion.ts`). It matters far more on a+ * remote machine than a question does: the CLI attaches to an agent the USER started, under the user's+ * own config and with no permission flag of ours (`engines/README.md`), so a blocking approval is the+ * normal state of a pane, not an edge case. Unparsed, the turn simply sits at `Processing` until someone+ * walks to the computer.+ *+ * Claude and Command Code draw it as a framed block ending in numbered rows, and it carries NONE of the+ * three anchors `parseQuestionPane` knows. Captured live (`__fixtures__/permission-claude.txt`):+ *+ * \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500+ * Bash command \u2190 the header: what kind of approval this is+ *+ * curl -s https://api.coingecko.com/\u2026 \u2190 the argument, then a one-line description+ * Fetch Bitcoin price from CoinGecko API+ *+ * This command requires approval+ *+ * Do you want to proceed?+ * \u276f 1. Yes+ * 2. Yes, and don\u2019t ask again for: curl *+ * 3. No+ *+ * Esc to cancel \u00b7 Tab to amend \u00b7 ctrl+e to explain+ *+ * Command Code differs only in wording (`permission-commandcode.txt`): header `Execute Shell Command`,+ * a sentence for the body, footer `\u2191/\u2193 navigate \u00b7 enter select \u00b7 ctrl+e explain`. Same frame, same rows \u2014+ * which is why one parser covers both instead of two near-copies that would drift apart.+ *+ * **The prose is not the anchor.** Every engine words it differently and Claude alone writes at least+ * three ("Do you want to proceed?", "Do you want to make this edit to README.md?", "Do you want to create+ * \u2026?"). The ROWS are: an approval always offers a way to say yes and a way to say no. That pair is the+ * signal, guarded by the dialog's own key hints so an ordinary numbered list in assistant output can never+ * be read as a prompt.+ */++/** The first row of an approval, measured across claude, commandcode, codex, devin, grok, amp and kilo. */+const APPROVE_RE = /^(yes|allow|approve|accept|proceed|run|continue)\b/i+/** \u2026and the row that declines it. Never dropped \u2014 see the rule amp's parser states: a device user given+ * three ways to say yes and none to say no cannot answer the prompt at all. */+// `skip` is here because cursor's decline row is "Skip & tell the agent what to do instead" — an engine+// whose only way to refuse says neither "no" nor "reject" would otherwise fail the yes/no guard and its+// whole prompt would go unread.+// Claude's plan review does not use a negative verb at all: its only refusal+// row is "Tell Claude what to change". Keep `tell ... what to change` narrow so+// an ordinary numbered list beginning with "Tell" cannot become an approval.+const REJECT_RE = /^(no|reject|deny|decline|cancel|skip|don'?t|stop)\b|^tell\b.*\bwhat to change\b/i+/** The solid rule that opens the frame. Deliberately NOT the dashed one (`\u254c`) that brackets an edit diff,+ * which sits BELOW the header and would cost the title. */+const FRAME_RULE_RE = /^\s*[\u2500\u2501\u2550]{6,}\s*$/+/** Any rule, solid or dashed \u2014 used to skip them while reading the frame's contents. */+const ANY_RULE_RE = /^\s*[\u2500\u2501\u2550\u254c\u2504\u2508-]{6,}\s*$/+/** The dialog's own question line, and Command Code's `Press [ctrl+e] \u2026` hint: both sit between the+ * header and the rows, and neither says what is being approved. */+const PERMISSION_PROSE_RE = /^((do|would) you\b|press \[)/i++/** The opening rule of the frame the rows at `start` sit in, or -1 when they have none of their own. */+function frameTop(lines: string[], start: number, floor: number): number {+ for (let i = start - 1; i > floor && start - i <= 25; i--) {+ if (FRAME_RULE_RE.test(lines[i])) return i+ }+ return -1+}++/** Keep a synthesised title inside the device's `text[200]` buffer, with the tail marked as cut. */+function clipTitle(value: string): string {+ return value.length <= 160 ? value : `${value.slice(0, 159)}\u2026`+}++/**+ * What is being approved, as one line: `Approve <header>: <argument>`, the form amp's parser already+ * produces so the two read alike on the device.+ *+ * The frame's opening rule is the only reliable top \u2014 the prose under it varies per engine AND per tool.+ * Below the rule sit the header (`Bash command`, `Edit file`, `Execute Shell Command`) and then the+ * argument: the command, the file, or the sentence naming it.+ */+function permissionTitle(lines: string[], start: number): string {+ // The header must be THIS dialog's: never read past the end of an earlier one still in scrollback.+ const floor = earlierDialogEnd(lines, start, 25)+ const top = frameTop(lines, start, floor)+ // No frame above the rows (codex draws none): the nearest text is the dialog's own question, which+ // names the command outright. Better than inventing a header that is not on screen.+ if (top < 0) {+ for (let i = start - 1; i > floor && start - i <= 4; i--) {+ const line = lines[i].trim()+ if (line) return clipTitle(line)+ }+ return 'Approval required'+ }+ let header = ''+ let arg = ''+ for (let i = top + 1; i < start; i++) {+ const line = lines[i].trim()+ if (!line || ANY_RULE_RE.test(line)) continue+ if (!header) { header = line; continue }+ if (PERMISSION_PROSE_RE.test(line)) continue+ arg = line+ break+ }+ if (!header) return 'Approval required'+ return clipTitle(arg ? `Approve ${header}: ${arg}` : `Approve ${header}`)+}++/**+ * The permission prompt on screen, with the index of its first row so the caller can rank it against the+ * other anchors.+ *+ * Only the LOWEST numbered block on the pane is considered, and it is rejected outright if it is not an+ * approval. Digging further up would find an answered prompt in the scrollback and re-announce it \u2014 the+ * live dialog is always the bottom-most thing on screen, so there is nothing below it to miss.+ */+export function parsePermissionPane(lines: string[]): { view: QuestionView; index: number } | null {+ let end = -1+ for (let i = lines.length - 1; i >= 0; i--) {+ if (parseRow(lines[i])) { end = i; break }+ }+ if (end < 0) return null++ // Walk up through CONTIGUOUS rows to the one numbered 1. Contiguity is the point: a permission dialog+ // never interleaves prose with its options, while assistant output that happens to be numbered does.+ const rows: QuestionRow[] = []+ let start = -1+ for (let i = end; i >= 0 && end - i < 12; i--) {+ const row = parseRow(lines[i])+ if (!row) break+ rows.unshift(row)+ if (row.number === '1') { start = i; break }+ }+ if (start < 0 || rows.length < 2) return null+ if (!APPROVE_RE.test(rows[0].label) || !rows.some((row) => REJECT_RE.test(row.label))) return null++ let hinted = false+ for (let i = end + 1; i < lines.length && i - end <= 4; i++) {+ if (PERMISSION_FOOTER_RE.test(lines[i])) { hinted = true; break }+ }+ if (!hinted) return null++ // Single-select, and no free-text row: every option here is a choice to be TAPPED. Verified on live+ // panes for claude, codex, devin and grok \u2014 one digit selects and submits, exactly as `rowKeys` assumes.+ return {+ view: { kind: 'question', permission: true, question: permissionTitle(lines, start), rows, multi: false, typeRow: null, dialog: permissionDialog(lines, start, end) },+ index: start,+ }+}++/** Every line of the dialog, untruncated: from its opening rule (or, with no frame, up to 12 lines above+ * the rows) to its last row — never reaching back past the end of an earlier dialog. */+function permissionDialog(lines: string[], start: number, end: number): string {+ const floor = earlierDialogEnd(lines, start, 25)+ const rule = frameTop(lines, start, floor)+ const top = rule >= 0 ? rule + 1 : Math.max(0, start - 12, floor + 1)+ return lines.slice(top, end + 1).map((line) => line.trimEnd()).filter((line) => !ANY_RULE_RE.test(line)).join('\n').trim()+}++export function parseQuestionPane(capture: string): PaneView {+ return locateQuestionPane(capture)?.view ?? null+}++/** `parseQuestionPane`, with the line its dialog was anchored on. */+export function locateQuestionPane(capture: string): FoundDialog | null {+ const lines = stripAnsi(capture).replace(/\u00a0/g, ' ').split('\n')+ // Each CLI words its own footer, and OpenCode rewords it PER SCREEN — `enter submit` on a single+ // question, `enter toggle` on a multi-select, `enter confirm` on a step of a multi-question. They all+ // mark the same thing: the bottom of a live dialog.+ // `enter to submit answer` is Codex's request_user_input dialog (plan mode; captured live in+ // __fixtures__/question-codex.txt). Without it that dialog was never a question at all here: the dial+ // showed nothing while the pane waited, and a question it DID show could never be closed, because+ // the watcher had no fingerprint to notice leaving.+ const footer = lines.findLastIndex((l) => QUESTION_FOOTER_RE.test(l))+ // The review screen paints no footer and puts its rows BELOW the prompt, so it needs its own anchor.+ // Whichever anchor is LOWER on screen is the live one (the other is scrollback from an earlier step).+ const review = lines.findLastIndex((l) => /Ready to submit your answers/i.test(l))+ // Command Code's review screen has neither Claude's "Ready to submit your answers" line nor a footer:+ // it is a "Submit"/"Cancel" pair under a summary that is itself numbered. Anchor on that PAIR — the two+ // rows adjacent, in that order — because the summary lines above are numbered too and reading them as+ // options is how the device answered everything and then sat there, never submitting.+ const submit = findSubmitPair(lines)+ // Command Code paints the SAME dialog with no footer at all — the pane simply ends at the last option.+ // Its tab bar is the only thing above the rows that is unmistakably part of the dialog, so it anchors+ // that dialog, read DOWNWARD. Nothing else on either CLI's screen looks like "● X | ◯ Y" followed by a+ // numbered list, which is what keeps ordinary numbered output from being read as a question.+ const tabBar = findTabBarDialog(lines)+ // A permission prompt is one more anchor, and every anchor is ranked the same way: whichever sits+ // lowest on screen is the live dialog, the others are scrollback. That ordering is what keeps the two+ // apart in both directions — codex and hermes draw an approval whose footer the question anchor also+ // matches, and there the footer is BELOW the rows, so the question path (which reads a better title off+ // the same block) still wins. And a footer-less dialog under an answered one is the live one: anchored+ // on the answered one's footer instead, Command Code's question was announced as the OLD question.+ const permission = parsePermissionPane(lines)+ if (permission && permission.index > Math.max(footer, review, submit?.index ?? -1, tabBar)) {+ return { view: permission.view, at: permission.index }+ }+ if (tabBar > Math.max(footer, review, submit?.index ?? -1)) {+ const view = parseDownward(lines, tabBar)+ return view && { view, at: tabBar }+ }+ if (review > footer && review > (submit?.index ?? -1)) {+ for (let i = review + 1; i < lines.length && i - review <= 10; i++) {+ const row = parseRow(lines[i])+ if (row && /^submit answers$/i.test(row.label)) return { view: { kind: 'review', submitRow: row.number }, at: review }+ }+ return null+ }+ if (submit && submit.index > footer) return { view: { kind: 'review', submitRow: submit.row }, at: submit.index }+ if (footer < 0) return null++ // Rows belonging to this dialog: the numbered rows just above the footer, back to the row numbered 1 —+ // and never from above an earlier dialog's end, which is where a dialog whose top is scrolled out of+ // the pane would otherwise borrow its first rows.+ const rows: QuestionRow[] = []+ let checkbox = false // `[ ]` / `[✔]` on a row ⇒ this question is multi-select+ let start = -1+ const floor = earlierDialogEnd(lines, footer, 40)+ for (let i = footer - 1; i > floor && footer - i <= 40; i--) {+ const row = parseRow(lines[i])+ if (!row) continue+ rows.unshift(row)+ if (/^\s*[❯›>]?\s*\d+\.\s+\[/.test(lines[i])) checkbox = true+ if (row.number === '1') { start = i; break }+ }+ const enterSubmits = /enter to submit answer/i.test(lines[footer])+ if (rows.length && start < 0 && enterSubmits) {+ return { view: { kind: 'question', partial: true, enterSubmits, question: '', rows, multi: checkbox, typeRow: null }, at: footer }+ }+ if (start < 0 || rows.length === 0) return null++ // The question is the nearest real text line above the rows. The dialog paints it between its header+ // and its rows — `[tab bar | header chip] · blank · question · blank · rows` — so those, and a rule,+ // are the TOP of this frame. Stop there, never skip past: mid-repaint the question line can be blank+ // for one capture, and walking on would pick up the PREVIOUS question still sitting in scrollback and+ // pair a stale title with the live options. An empty result just means "look again next tick". The+ // end of an earlier dialog is a top too: its footer is not this dialog's question.+ let question = ''+ const top = earlierDialogEnd(lines, start, 12)+ for (let i = start - 1; i > top && start - i <= 12; i--) {+ const line = lines[i].trim()+ if (!line) continue+ if (/[←→]/.test(line) || /^[☐☒✔✓]/.test(line) || /^[─━-]{6,}$/.test(line)) break+ // Hermes's batch panel marks the active question with `▸`; that marker is chrome, not the question.+ question = line.replace(/^▸\s*/, '')+ break+ }++ const answerable = rows.filter((r) => !CHAT_ROW.test(r.label) && !TYPE_ROW.test(r.label))+ return {+ view: {+ kind: 'question',+ ...(enterSubmits ? { enterSubmits } : {}),+ question,+ rows: answerable,+ multi: checkbox,+ typeRow: rows.find((r) => TYPE_ROW.test(r.label)) ?? null,+ dialog: dialogAbove(lines, start, footer),+ },+ at: footer,+ }+}++/**+ * A footer-anchored dialog, whole: up to 12 lines above its rows — stopping at a rule, a tab bar, the+ * agent's own output (a `•`/`⏺` bullet) or the end of an earlier dialog — down to its last row. Codex puts+ * the command it asks about here (`$ …`, wrapped over as many lines as it takes), with its reason and+ * environment.+ */+function dialogAbove(lines: string[], start: number, footer: number): string {+ let top = start+ const floor = earlierDialogEnd(lines, start, 12)+ for (let i = start - 1; i > floor && start - i <= 12; i--) {+ const line = lines[i].trim()+ if (/^[•⏺●]/.test(line) || /[←→]/.test(line) || /^[─━═-]{6,}$/.test(line)) break+ top = i+ }+ return lines.slice(top, footer).map((line) => line.trimEnd()).join('\n').trim()+}++/**+ * A dialog that asks to RUN or CHANGE something (an approval), whichever parser read it: a framed+ * permission prompt, or a footer dialog whose first row approves, whose rows include a rejection, and+ * which names a command (`$ …`) or asks "would you like to run / make …". What the pair's floor treats as+ * a permission prompt (pair/classify.ts); nothing else reads it.+ */+export function isApprovalDialog(view: QuestionView): boolean {+ if (view.permission) return true+ if (!view.rows.length || !APPROVE_RE.test(view.rows[0].label) || !view.rows.some((row) => REJECT_RE.test(row.label))) return false+ const text = view.dialog ?? view.question+ return /(^|\n)\s*\$ /.test(text) || /would you like to (run|make|apply|execute|edit|write)/i.test(text)+}++/** Match an answer to a row. The device stores option labels in an 80-byte buffer, so a long label comes+ * back truncated — prefix matches count, in both directions. */+/**+ * The keystrokes that commit one row.+ *+ * Every engine but Amp numbers its options, so the digit both selects and submits in one press. Amp+ * draws an unnumbered list navigated with the arrow keys, so its rows carry an index and are reached by+ * walking down to them.+ */diff --git a/cli/src/engines/kit/screen.ts b/cli/src/engines/kit/screen.tsnew file mode 100644index 000000000..f4c42b5dc--- /dev/null+++ b/cli/src/engines/kit/screen.ts@@ -0,0 +1,42 @@+import { stripVTControlCharacters } from 'node:util'+import type { AgentEngine } from '../types.js'+import type { ComposerState, MessageHold, PaneInspection, PaneModal, PaneView, ScreenReading } from '../facets/screen.js'+import { isApprovalDialog } from './questionPane.js'+const HOLDS: Record<NonNullable<PaneModal>, MessageHold> = {+ rewind: 'rewind_picker_open', transcript: 'transcript_open', search: 'search_open', trust: 'trust_open',+ update: 'update_prompt_open', model: 'model_prompt_open', sign_in: 'sign_in_open', permission: 'permission_open', menu: 'menu_open',+}+export function assembleScreen(engine: AgentEngine, capture: string, pane: PaneInspection, question: PaneView,+ modal: PaneModal = null, composer?: ComposerState, activity: ScreenReading['activity'] = null, stoppedGoal = false): ScreenReading {+ const messageHold = question ? question.kind === 'question' && isApprovalDialog(question) ? 'permission_open' : 'question_open'+ : modal ? HOLDS[modal] : composer === undefined || composer === 'ready' ? null : composer === 'popup' ? 'popup_open' : 'prompt_hidden'+ const footer = stripVTControlCharacters(capture).split('\n').slice(-16).join('\n')+ const busy = !!activity || /\bgoal\s+active\b|\bpursuing goal\b|\b[1-9]\d*\s+background\s+(?:tasks?|agents?)\b/i.test(footer)+ return { pane, question, messageHold, teamHold: teamHold(engine, capture, pane, question), activity, busy, stoppedGoal }+}+function teamHold(engine: AgentEngine, capture: string, pane: PaneInspection, question: PaneView): string | null {+ if (!capture) return 'team_waiting_unavailable'+ if (pane.dialog || question) return 'team_waiting_user'+ if (pane.draft) return 'team_waiting_draft'+ if (!pane.idle) return 'team_waiting_idle'+ const lines = capture.split('\n').map(stripVTControlCharacters)+ if (/esc(?:ape)? to (?:interrupt|cancel|stop)/i.test(lines.slice(-15).join('\n'))) return 'team_waiting_idle'+ // These composers permit an empty first line followed by a multiline human draft.+ // The general profile inspector only needs the first line; team delivery needs every line.+ if (['claude', 'codex', 'cursor', 'hermes'].includes(engine)) {+ const index = lines.findLastIndex(line => /^\s*[›❯→]/u.test(line))+ if (index < 0) return 'team_waiting_idle'+ // Codex shows its styled placeholder only when the whole composer is empty.+ // Its configurable model/cwd/task footer below that placeholder is not draft+ // text. The profile inspector has already proved this visible prompt text is+ // a placeholder; a bare first line still needs the multiline-draft check.+ if (engine === 'codex' && lines[index].replace(/^\s*[›❯→]\s*/u, '').trim()) return null+ for (const line of lines.slice(index + 1)) {+ if (/[─━―]{8,}/u.test(line)) break+ const text = line.trim()+ if (!text || /^(?:\? for shortcuts|\d+% context|context:|ctrl\+|shift\+|tab to|plan mode|accept edits|bypass permissions)/i.test(text)) continue+ return 'team_waiting_draft'+ }+ }+ return null+}diff --git a/cli/src/engines/registry.ts b/cli/src/engines/registry.tsindex 6ec9b238e..25bc2ff18 100644--- a/cli/src/engines/registry.ts+++ b/cli/src/engines/registry.ts@@ -1,3 +1,5 @@+import { screen as claudeScreen } from './claude/screen.js'+import { screen as codexScreen } from './codex/screen.js'import type { Engine } from './engine.js'import { engineLaunches } from './launches.js'import { engineHooks } from './hooks.js'@@ -10,8 +12,8 @@ import { runtime as codexRuntime } from './codex/runtimeProfile.js'/** Only the migrated engines. Others keep their existing handlers until their own small batch. */const engines = {- claude: { name: 'claude', launch: engineLaunches.claude, transcript: claudeTranscript, hooks: engineHooks.claude, live: claudeLive, runtime: claudeRuntime },- codex: { name: 'codex', launch: engineLaunches.codex, transcript: codexTranscript, hooks: engineHooks.codex, live: codexLive, runtime: codexRuntime },+ claude: { name: 'claude', launch: engineLaunches.claude, transcript: claudeTranscript, hooks: engineHooks.claude, live: claudeLive, runtime: claudeRuntime, screen: claudeScreen },+ codex: { name: 'codex', launch: engineLaunches.codex, transcript: codexTranscript, hooks: engineHooks.codex, live: codexLive, runtime: codexRuntime, screen: codexScreen },} satisfies Record<string, Engine>type MigratedEngine = keyof typeof enginesexport function engineFor(name: string | null | undefined): Engine | undefined {diff --git a/cli/src/engines/screens.ts b/cli/src/engines/screens.tsnew file mode 100644index 000000000..d4d21f2e8--- /dev/null+++ b/cli/src/engines/screens.ts@@ -0,0 +1,12 @@+import type { AgentEngine } from './types.js'+import { screen as claude, inspectRuntimePane as claudePane, paneModal as claudeModal } from './claude/screen.js'+import { screen as codex, inspectRuntimePane as codexPane, paneModal as codexModal } from './codex/screen.js'+import { legacyScreen } from '../lib/legacyScreen.js'+import { paneModal as legacyModal } from './kit/pane.js'+import { inspectRuntimePane as inspectPane } from '../lib/legacyPane.js'+export function screenFor(engine: AgentEngine) { return engine === 'claude' ? claude : engine === 'codex' ? codex : { inspect: (capture: string) => legacyScreen(engine, capture) } }+export function inspectRuntimePane(engine: AgentEngine, capture: string) { return engine === 'claude' ? claudePane(capture) : engine === 'codex' ? codexPane(capture) : inspectPane(engine, capture) }+export function paneModal(engine: AgentEngine, capture: string) { return engine === 'claude' ? claudeModal(capture) : engine === 'codex' ? codexModal(capture) : legacyModal(engine, capture) }+export function parseEngineQuestionPane(engine: AgentEngine, capture: string) { return screenFor(engine).inspect(capture).question }+export function messageHold(engine: AgentEngine, capture: string | null) { return capture === null ? (engine === 'claude' || engine === 'codex' ? 'screen_unreadable' : null) : screenFor(engine).inspect(capture).messageHold }+export function teamWriteHold(engine: AgentEngine, capture: string | null) { return capture ? screenFor(engine).inspect(capture).teamHold : 'team_waiting_unavailable' }diff --git a/cli/src/engines/worker/process.ts b/cli/src/engines/worker/process.tsindex c0ba46326..6d734b984 100644--- a/cli/src/engines/worker/process.ts+++ b/cli/src/engines/worker/process.ts@@ -6,6 +6,7 @@ import { runServiceProcess, type ServiceProcess } from '../../services/process.jimport type { ServiceProcessOptions } from '../../serviceProcess.js'import type { EngineTranscript, HistoryAsk, TranscriptSession } from '../facets/transcript.js'import { engineLiveRequests } from './liveRequests.js'+import { engineScreenRequests } from './screenRequests.js'import { engineRuntimeRequests } from './runtimeRequests.js'import {EngineReadError, record, READER_HISTORY, READER_IN_FLIGHT, READER_LAST_TURN, READER_REPLY_BYTES,@@ -86,6 +87,6 @@ export interface EngineProcessOptions extends ServiceProcessOptions {export function runEngineReader(engine: ReaderEngine, options: EngineProcessOptions): ServiceProcess {return (options.run ?? runServiceProcess)({name: READER_SERVICES[engine], socketPath: options.socketPath, machineId: options.machineId,- token: options.token, requests: options.requests ?? { ...engineReaderRequests(engine), ...engineLiveRequests(engine), ...engineRuntimeRequests(engine) },+ token: options.token, requests: options.requests ?? { ...engineReaderRequests(engine), ...engineLiveRequests(engine), ...engineRuntimeRequests(engine), ...engineScreenRequests(engine) },})}diff --git a/cli/src/engines/worker/screenProtocol.ts b/cli/src/engines/worker/screenProtocol.tsnew file mode 100644index 000000000..2a94776e0--- /dev/null+++ b/cli/src/engines/worker/screenProtocol.ts@@ -0,0 +1,55 @@+/** Private, stateless screen reads. A worker receives text, never a terminal handle. */+import type { PaneView, QuestionRow, ScreenReading } from '../facets/screen.js'+import { record } from './protocol.js'++export const SCREEN_VERSION = 1+export const SCREEN_READ = 'engine_screen_read'+export const SCREEN_CAPABILITIES = 'engine_screen_capabilities'+export const SCREEN_WAIT_MS = 1_000+export const SCREEN_IN_FLIGHT = 8+export const SCREEN_QUEUED = 64+export const SCREEN_CAPTURE_BYTES = 256 * 1024+export const SCREEN_REPLY_BYTES = 1024 * 1024+const HOLDS = ['permission_open', 'question_open', 'menu_open', 'rewind_picker_open', 'transcript_open',+ 'search_open', 'trust_open', 'update_prompt_open', 'model_prompt_open', 'sign_in_open', 'popup_open',+ 'prompt_hidden', 'screen_unreadable']+const TEAM_HOLDS = ['team_waiting_unavailable', 'team_waiting_user', 'team_waiting_draft', 'team_waiting_idle']+const fields = (value: Record<string, unknown>, names: string[]) => Object.keys(value).every(key => names.includes(key))+const text = (value: unknown, max = SCREEN_CAPTURE_BYTES): value is string => typeof value === 'string' && value.length <= max+const flag = (value: unknown) => value === undefined || typeof value === 'boolean'++export function screenCapture(value: unknown): value is string {+ return text(value) && Buffer.byteLength(value) <= SCREEN_CAPTURE_BYTES+}++function questionRow(value: unknown): value is QuestionRow {+ return record(value) && fields(value, ['number', 'label', 'checked', 'walk'])+ && typeof value.number === 'string' && /^[1-9]\d?$/.test(value.number)+ && text(value.label) && typeof value.checked === 'boolean'+ && (value.walk === undefined || value.walk === 'right' || value.walk === 'down')+}++function questionView(value: unknown): value is PaneView {+ if (value === null) return true+ if (!record(value)) return false+ if (value.kind === 'review') return fields(value, ['kind', 'submitRow'])+ && typeof value.submitRow === 'string' && /^(?:[1-9]\d?|Enter)$/.test(value.submitRow)+ return value.kind === 'question' && fields(value, ['kind', 'permission', 'partial', 'enterSubmits', 'question', 'rows', 'multi', 'typeRow', 'dialog'])+ && flag(value.permission) && flag(value.partial) && flag(value.enterSubmits) && text(value.question)+ && Array.isArray(value.rows) && value.rows.length <= 99 && value.rows.every(questionRow)+ && typeof value.multi === 'boolean' && (value.typeRow === null || questionRow(value.typeRow))+ && (value.dialog === undefined || text(value.dialog))+}++/** Malformed or oversized evidence cannot make a screen writable. */+export function screenReading(value: unknown): value is ScreenReading {+ return record(value) && fields(value, ['pane', 'question', 'messageHold', 'teamHold', 'activity', 'busy', 'stoppedGoal']) && record(value.pane)+ && fields(value.pane, ['idle', 'plan', 'dialog', 'draft'])+ && ['idle', 'plan', 'dialog', 'draft'].every(key => typeof (value.pane as Record<string, unknown>)[key] === 'boolean')+ && typeof value.busy === 'boolean' && typeof value.stoppedGoal === 'boolean'+ && (value.activity === null || (record(value.activity) && fields(value.activity, ['label', 'indicator']) && text(value.activity.label, 100) && text(value.activity.indicator)))+ && questionView(value.question) && (value.messageHold === null || HOLDS.includes(value.messageHold as string))+ && (value.teamHold === null || TEAM_HOLDS.includes(value.teamHold as string))+ && !(value.pane.idle && (value.pane.dialog || value.pane.draft))+ && !(value.question && value.messageHold === null)+}diff --git a/cli/src/engines/worker/screenRequests.spec.ts b/cli/src/engines/worker/screenRequests.spec.tsnew file mode 100644index 000000000..3e95cac45--- /dev/null+++ b/cli/src/engines/worker/screenRequests.spec.ts@@ -0,0 +1,80 @@+import { readFileSync } from 'node:fs'+import { afterEach, describe, expect, it, vi } from 'vitest'+import { screenFor } from '../screens.js'+import { engineScreenRequests } from './screenRequests.js'+import { screenReading, SCREEN_CAPABILITIES, SCREEN_CAPTURE_BYTES, SCREEN_IN_FLIGHT, SCREEN_READ, SCREEN_REPLY_BYTES, SCREEN_WAIT_MS } from './screenProtocol.js'++const who = { owner: true, local: true }+const capture = '────────────\n❯\n────────────\n? for shortcuts'+const payload = { version: 1, capture }+const answer = screenFor('claude').inspect(capture)+const fail = (error: string) => ({ version: 1, error })+afterEach(() => { vi.useRealTimers(); vi.restoreAllMocks() })+describe('read-only engine screen workers', () => {+ it.each(['claude', 'codex'] as const)('%s preserves recorded permission evidence', async engine => {+ const text = readFileSync(new URL(`../../lib/__fixtures__/permission-${engine}.txt`, import.meta.url), 'utf8')+ const requests = engineScreenRequests(engine)+ expect(await requests[SCREEN_CAPABILITIES]({ version: 1 }, who)).toEqual({ version: 1, screen: 1, engine })+ const reply = await requests[SCREEN_READ]({ version: 1, capture: text }, who)+ expect(reply).toEqual({ version: 1, answer: screenFor(engine).inspect(text) })+ expect((reply as any).answer.messageHold).toBe('permission_open')+ expect(screenReading((reply as any).answer)).toBe(true)+ })+ it('accepts only private local core requests and bounded captures', async () => {+ const load = vi.fn(async () => screenFor('claude'))+ const requests = engineScreenRequests('claude', { load })+ const send = (p = payload, asker = who, closed?: AbortSignal) => requests[SCREEN_READ](p, asker, closed)+ for (const asker of [{ owner: false, local: true }, { owner: true, local: false }, { ...who, connection: 'client' }]) {+ expect(await send(payload, asker)).toEqual(fail('ENGINE_INVALID_REQUEST'))+ }+ for (const p of [{ version: 2, capture }, { version: 1, capture: 42 }, { ...payload, terminal: 'agent' },+ { ...payload, capture: 'é'.repeat(SCREEN_CAPTURE_BYTES) }]) {+ expect(await send(p as typeof payload)).toEqual(fail('ENGINE_INVALID_REQUEST'))+ }+ expect(await send(payload, who, AbortSignal.abort())).toEqual(fail('ENGINE_UNAVAILABLE'))+ expect(load).not.toHaveBeenCalled()+ expect(await requests[SCREEN_CAPABILITIES]({ version: 1 }, who)).toMatchObject({ screen: 1 })+ expect(load).not.toHaveBeenCalled()+ expect(await send({ ...payload, requestId: 'core-route-1' } as typeof payload)).toEqual({ version: 1, answer })+ expect(await send({ ...payload, requestId: 1 } as unknown as typeof payload)).toEqual(fail('ENGINE_INVALID_REQUEST'))+ expect(await send({ ...payload, requestId: 'x'.repeat(201) } as typeof payload)).toEqual(fail('ENGINE_INVALID_REQUEST'))+ expect(await send()).toEqual({ version: 1, answer })+ expect(load).toHaveBeenCalledOnce()+ })+ it('bounds concurrent loads and contains a stopped or failed parser', async () => {+ vi.useFakeTimers()+ const recycle = vi.fn()+ let finish!: (value: ReturnType<typeof screenFor>) => void+ const requests = engineScreenRequests('claude', { recycle, load: () => new Promise(resolve => { finish = resolve }) })+ const send = () => requests[SCREEN_READ](payload, who)+ const pending = Array.from({ length: SCREEN_IN_FLIGHT }, send)+ expect(await send()).toEqual(fail('ENGINE_BUSY'))+ await vi.advanceTimersByTimeAsync(SCREEN_WAIT_MS)+ for (const read of pending) expect(await read).toEqual(fail('ENGINE_UNAVAILABLE'))+ expect(recycle).toHaveBeenCalled()+ finish(screenFor('claude'))+ const load = vi.fn().mockRejectedValueOnce(new Error('import failed')).mockResolvedValue(screenFor('claude'))+ const retry = engineScreenRequests('claude', { load })+ expect(await retry[SCREEN_READ](payload, who)).toEqual(fail('ENGINE_UNAVAILABLE'))+ expect(await retry[SCREEN_READ](payload, who)).toEqual({ version: 1, answer })+ })+ it('drops disconnected work and rejects faulty adapter output', async () => {+ const closed = new AbortController()+ const requests = engineScreenRequests('claude', { load: async () => { closed.abort(); return screenFor('claude') } })+ expect(await requests[SCREEN_READ](payload, who, closed.signal)).toEqual(fail('ENGINE_UNAVAILABLE'))+ for (const [result, error] of [[{}, 'ENGINE_INVALID_REPLY'], [{ ...answer, teamHold: 'invented' }, 'ENGINE_INVALID_REPLY'],+ [{ ...answer, extra: 'x'.repeat(SCREEN_REPLY_BYTES) }, 'ENGINE_REPLY_TOO_LARGE']] as const) {+ const worker = engineScreenRequests('claude', { load: async () => ({ inspect: () => result as typeof answer }) })+ expect(await worker[SCREEN_READ](payload, who)).toEqual(fail(error))+ }+ const throwing = engineScreenRequests('claude', { load: async () => ({ inspect: () => { throw new Error('bad decoder') } }) })+ expect(await throwing[SCREEN_READ](payload, who)).toEqual(fail('ENGINE_UNAVAILABLE'))+ })+ it('rejects malformed UI facts before any caller can treat a pane as writable', () => {+ for (const value of [null, [], {}, { ...answer, pane: { ...answer.pane, draft: true } },+ { ...answer, question: { kind: 'review', submitRow: 'C-c' } }, { ...answer, activity: { label: 2, indicator: '' } },+ { ...answer, question: { kind: 'question', rows: [], question: 'approve?', multi: false, typeRow: null } },+ { ...answer, messageHold: 'unknown' }, { ...answer, execute: 'Enter' }]) expect(screenReading(value)).toBe(false)+ expect(screenReading({ ...answer, question: { kind: 'review', submitRow: '1' }, messageHold: 'question_open' })).toBe(true)+ })+})diff --git a/cli/src/engines/worker/screenRequests.ts b/cli/src/engines/worker/screenRequests.tsnew file mode 100644index 000000000..b468b8a83--- /dev/null+++ b/cli/src/engines/worker/screenRequests.ts@@ -0,0 +1,45 @@+import type { ServiceRequests } from '../../core/api.js'+import type { EngineScreen } from '../facets/screen.js'+import type { ReaderEngine } from './protocol.js'+import { screenCapture, screenReading, SCREEN_CAPABILITIES, SCREEN_IN_FLIGHT, SCREEN_READ,+ SCREEN_REPLY_BYTES, SCREEN_VERSION, SCREEN_WAIT_MS } from './screenProtocol.js'++const loadScreen = {+ claude: async () => (await import('../claude/screen.js')).screen,+ codex: async () => (await import('../codex/screen.js')).screen,+}+export interface ScreenRequestDeps { load?: () => Promise<EngineScreen>; recycle?: () => void }++export function engineScreenRequests(engine: ReaderEngine, deps: ScreenRequestDeps = {}): ServiceRequests {+ let loaded: Promise<EngineScreen> | null = null+ let pending = 0+ const failure = (error: string) => ({ version: SCREEN_VERSION, error })+ const handle = (method: string): ServiceRequests[string] => async (payload, asker, closed) => {+ if (!asker.owner || !asker.local || asker.connection !== undefined || payload.version !== SCREEN_VERSION) return failure('ENGINE_INVALID_REQUEST')+ if (closed?.aborted) return failure('ENGINE_UNAVAILABLE')+ if (method === SCREEN_CAPABILITIES) return { version: SCREEN_VERSION, screen: SCREEN_VERSION, engine }+ // ServiceLinks adds its routing id inside the payload. It is transport+ // metadata, not a terminal capability or engine input.+ if (!screenCapture(payload.capture) || Object.keys(payload).some(key => !['version', 'capture', 'requestId'].includes(key))+ || (payload.requestId !== undefined && (typeof payload.requestId !== 'string' || payload.requestId.length > 200))) return failure('ENGINE_INVALID_REQUEST')+ if (pending >= SCREEN_IN_FLIGHT) return failure('ENGINE_BUSY')+ pending+++ let timer: ReturnType<typeof setTimeout> | undefined+ const deadline = new Promise<Record<string, unknown>>(resolve => {+ timer = setTimeout(() => { resolve(failure('ENGINE_UNAVAILABLE')); (deps.recycle ?? (() => process.exit(1)))() }, SCREEN_WAIT_MS)+ })+ try {+ const capture = payload.capture+ return await Promise.race([deadline, (async () => {+ const adapter = await (loaded ??= (deps.load ?? loadScreen[engine])().catch(error => { loaded = null; throw error }))+ if (closed?.aborted) return failure('ENGINE_UNAVAILABLE')+ const answer = adapter.inspect(capture)+ const reply = { version: SCREEN_VERSION, answer }+ if (Buffer.byteLength(JSON.stringify(reply)) > SCREEN_REPLY_BYTES) return failure('ENGINE_REPLY_TOO_LARGE')+ return screenReading(answer) ? reply : failure('ENGINE_INVALID_REPLY')+ })()])+ } catch { return failure('ENGINE_UNAVAILABLE') }+ finally { clearTimeout(timer); pending-- }+ }+ return { [SCREEN_CAPABILITIES]: handle(SCREEN_CAPABILITIES), [SCREEN_READ]: handle(SCREEN_READ) }+}diff --git a/cli/src/harnessd/services.spec.ts b/cli/src/harnessd/services.spec.tsindex fa3ecc721..c91e7612e 100644--- a/cli/src/harnessd/services.spec.ts+++ b/cli/src/harnessd/services.spec.ts@@ -1,6 +1,6 @@import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'import {- DEFAULT_SERVICE_OPTIONS, ENGINE_LIVE_ENV, ENGINE_RUNTIME_ENV, KNOWN_SERVICES, SERVICE_HOSTS, SERVICE_PROCESSES_ENV, ServiceSupervisor, UPDATER_HOST, masterRunsLiveEngines, masterRunsEngineRuntime, serviceOptions, serviceProcessesEnv, serviceSpecs, servicesTheMasterRuns,+ DEFAULT_SERVICE_OPTIONS, ENGINE_LIVE_ENV, ENGINE_RUNTIME_ENV, ENGINE_SCREEN_ENV, KNOWN_SERVICES, SERVICE_HOSTS, SERVICE_PROCESSES_ENV, ServiceSupervisor, UPDATER_HOST, masterRunsLiveEngines, masterRunsEngineRuntime, masterRunsEngineScreen, serviceOptions, serviceProcessesEnv, serviceSpecs, servicesTheMasterRuns,type ServiceSpec, type ServiceSupervisorOptions,} from './services.js'import type { CoreHandle } from './supervisor.js'@@ -472,15 +472,15 @@ describe('which services the core leaves to its master', () => {it('is what the master says it runs, told in a form a core from before the list reads too', () => {const specs = serviceSpecs({ HARNESSD_SERVICES: 'search,workspaces' }, SERVICE_HOSTS)- expect(serviceProcessesEnv(specs, 42)).toEqual({ [SERVICE_PROCESSES_ENV]: 'search,workspaces', HARNESSD_SERVICES: 'search,workspaces', [ENGINE_LIVE_ENV]: '42:1', [ENGINE_RUNTIME_ENV]: '42:1' })+ expect(serviceProcessesEnv(specs, 42)).toEqual({ [SERVICE_PROCESSES_ENV]: 'search,workspaces', HARNESSD_SERVICES: 'search,workspaces', [ENGINE_LIVE_ENV]: '42:1', [ENGINE_RUNTIME_ENV]: '42:1', [ENGINE_SCREEN_ENV]: '42:1' })// A process named as one of its services is named whole: the viewers' process runs the Store beside them.expect(serviceProcessesEnv(serviceSpecs({ HARNESSD_SERVICES: 'viewers' }, SERVICE_HOSTS), 42)).toMatchObject({ [SERVICE_PROCESSES_ENV]: 'viewers,store' })expect(serviceProcessesEnv(serviceSpecs({ HARNESSD_SERVICES: 'store' }, SERVICE_HOSTS), 42)).toMatchObject({ [SERVICE_PROCESSES_ENV]: 'store' })// By service, not by process: a core from before the edge host routes the services it knows of it.const hosted = serviceSpecs({ HARNESSD_SERVICES: 'edge' }, SERVICE_HOSTS)- expect(serviceProcessesEnv(hosted, 42)).toEqual({ [SERVICE_PROCESSES_ENV]: 'workspaces,usage,monitor,projects,handoff,recaps,windowNames,shell', HARNESSD_SERVICES: 'workspaces,usage,monitor,projects,handoff,recaps,windowNames,shell', [ENGINE_LIVE_ENV]: '42:1', [ENGINE_RUNTIME_ENV]: '42:1' })+ expect(serviceProcessesEnv(hosted, 42)).toEqual({ [SERVICE_PROCESSES_ENV]: 'workspaces,usage,monitor,projects,handoff,recaps,windowNames,shell', HARNESSD_SERVICES: 'workspaces,usage,monitor,projects,handoff,recaps,windowNames,shell', [ENGINE_LIVE_ENV]: '42:1', [ENGINE_RUNTIME_ENV]: '42:1', [ENGINE_SCREEN_ENV]: '42:1' })expect([...servicesTheMasterRuns({ ...supervised, ...serviceProcessesEnv(hosted, 42) }, known)]).toEqual(['workspaces'])- expect(serviceProcessesEnv([], 42)).toEqual({ [SERVICE_PROCESSES_ENV]: '', HARNESSD_SERVICES: 'none', [ENGINE_LIVE_ENV]: '42:1', [ENGINE_RUNTIME_ENV]: '42:1' })+ expect(serviceProcessesEnv([], 42)).toEqual({ [SERVICE_PROCESSES_ENV]: '', HARNESSD_SERVICES: 'none', [ENGINE_LIVE_ENV]: '42:1', [ENGINE_RUNTIME_ENV]: '42:1', [ENGINE_SCREEN_ENV]: '42:1' })expect([...servicesTheMasterRuns({ ...supervised, ...serviceProcessesEnv(specs, 42) }, known)]).toEqual(['search', 'workspaces'])expect([...servicesTheMasterRuns({ ...supervised, ...serviceProcessesEnv([], 42) }, known)]).toEqual([])// What the master says wins over whatever HARNESSD_SERVICES the core inherited; names it does not know are its own.@@ -517,6 +517,15 @@ describe('which services the core leaves to its master', () => {expect(masterRunsLiveEngines({ ...env, [ENGINE_RUNTIME_ENV]: undefined }, 42)).toBe(true)})+ it('negotiates screen readers only from a capable supervising parent', () => {+ const env = { ...supervised, ...serviceProcessesEnv([], 42) }+ expect(masterRunsEngineScreen(env, 42)).toBe(true)+ expect(masterRunsEngineScreen(env, 43)).toBe(false)+ expect(masterRunsEngineScreen({ ...env, [ENGINE_SCREEN_ENV]: undefined }, 42)).toBe(false)+ expect(masterRunsEngineScreen({ ...env, [ENGINE_LIVE_ENV]: undefined }, 42)).toBe(false)+ expect(masterRunsEngineScreen({ ...env, HARNESSD_SERVICE_TOKEN: undefined }, 42)).toBe(false)+ })+it('is none without a master, or without the token its services would connect with', () => {expect([...servicesTheMasterRuns({ [SERVICE_PROCESSES_ENV]: 'search' }, known)]).toEqual([])expect([...servicesTheMasterRuns({ HARNESSD_SUPERVISED: '1', [SERVICE_PROCESSES_ENV]: 'search' }, known)]).toEqual([])diff --git a/cli/src/harnessd/services.ts b/cli/src/harnessd/services.tsindex aeca007d9..f243e8606 100644--- a/cli/src/harnessd/services.ts+++ b/cli/src/harnessd/services.ts@@ -459,6 +459,7 @@ export const SERVICE_PROCESSES_ENV = 'HARNESSD_SERVICE_PROCESSES'export const ENGINE_LIVE_ENV = 'HARNESSD_ENGINE_LIVE'/** Runtime profile methods, negotiated independently from live transcript parsing. */export const ENGINE_RUNTIME_ENV = 'HARNESSD_ENGINE_RUNTIME'+export const ENGINE_SCREEN_ENV = 'HARNESSD_ENGINE_SCREEN'/*** What a master puts in its core's environment about the services it runs in their own processes: the@@ -470,7 +471,7 @@ export function serviceProcessesEnv(specs: readonly ServiceSpec[], masterPid: nu// The services, not the processes: a core knows what it routes by service, and one from before the// edge host still finds the services it knows here (workspaces) and runs the rest itself.const names = specs.flatMap((spec) => spec.services).join(',')- return { [SERVICE_PROCESSES_ENV]: names, HARNESSD_SERVICES: names || 'none', [ENGINE_LIVE_ENV]: `${masterPid}:1`, [ENGINE_RUNTIME_ENV]: `${masterPid}:1` }+ return { [SERVICE_PROCESSES_ENV]: names, HARNESSD_SERVICES: names || 'none', [ENGINE_LIVE_ENV]: `${masterPid}:1`, [ENGINE_RUNTIME_ENV]: `${masterPid}:1`, [ENGINE_SCREEN_ENV]: `${masterPid}:1` }}/** An older master may inherit a newer master's environment after rollback. Trust only this parent. */@@ -521,3 +522,7 @@ export function serviceSpecs(env: NodeJS.ProcessEnv, hosts: Readonly<Record<strireturn services.length ? [{ name, ...rest, services, ...onDemand, ...(Number.isInteger(heap) && heap > 0 ? { heapLimitMiB: heap } : {}) }] : []})}++export function masterRunsEngineScreen(env: NodeJS.ProcessEnv, parentPid: number): boolean {+ return masterRunsLiveEngines(env, parentPid) && env[ENGINE_SCREEN_ENV] === `${parentPid}:1`+}diff --git a/cli/src/lib/askQuestion.ts b/cli/src/lib/askQuestion.tsindex 9dcc77e41..877ed9625 100644--- a/cli/src/lib/askQuestion.ts+++ b/cli/src/lib/askQuestion.ts@@ -1,1433 +1,18 @@-/**- * AskUserQuestion on a REMOTE machine — mirror the question to the device, then answer it by driving the- * interactive CLI's own terminal dialog.- *- * A runtime that speaks stream-json gets this for free: a `question_request` event carries- * the questions out and a `question_response` control-frame carries the answer back. A remote machine has no- * such channel — the adapter watches a JSONL transcript and talks to a real TUI. So both halves read and- * write the pane itself:- *- * OUT QuestionWatcher polls the pane while a turn is open and pushes the open dialog as- * `commander_question`, in the SAME shape the hosted runtime sends, so the firmware's existing question- * screen renders it unchanged (commanderQuestions in websocket.ts). NOT from the transcript —- * see the QuestionWatcher docblock for why that source cannot work.- * IN the device's `question_response` → keystrokes into the pane's dialog — only once the dialog on- * screen is shown to be the one that answer was for (its requestId); a late answer is refused- * with STALE_QUESTION and types nothing.- *- * Dialog mechanics (verified against Claude Code 2.1.220, `tmux capture-pane`):- * - single-select : the option's digit selects AND submits, advancing to the next question / review.- * - free text : the digit of the "Type something." row opens it for editing (does NOT submit) →- * type the text → Enter submits.- * - multi-select : rows render as `[ ]` / `[✔]`; a digit TOGGLES; Tab advances to the next question.- * - review : after the last question, "Ready to submit your answers?" → "1. Submit answers".- */--import type { RegisteredSession } from './registry.js'-import type { AgentEngine } from '../engines/types.js'-import { locateMuseQuestion } from '../engines/muse/askQuestion.js'-import { ampSelectionKeys, parseAmpQuestionPane } from '../engines/amp/askQuestion.js'-import { kiloSelectionKeys, locateKiloQuestion, parseKiloQuestionPane } from '../engines/kilo/askQuestion.js'-import { parseCursorPermissionPane } from '../engines/cursor/askQuestion.js'-import { locateDevinPermission, locateDevinQuestion } from '../engines/devin/askQuestion.js'-import { parseGrokQuestionPane } from '../engines/grok/askQuestion.js'-import { locateAgyQuestion } from '../engines/agy/askQuestion.js'-import { withCopilotSubject } from '../engines/copilot/askQuestion.js'-import { earlierDialogEnd, PERMISSION_FOOTER_RE, QUESTION_FOOTER_RE } from './dialogEnd.js'-import { composerState } from './composerScreen.js'--/** Device-facing question shape — byte-for-byte the hosted runtime’s `commanderQuestions()` output. */-export interface ShapedQuestion {- key: string- q: string- options: string[]- multi: boolean- /** Observed free-text editor; never inferred for permission prompts. */- canText?: boolean-}--export interface QuestionRow {- /** The digit to press. */- number: string- /** Option text with any `[ ]` checkbox prefix removed. */- label: string- checked: boolean- /**- * Set only when the dialog does NOT number its rows: the row is then reached by walking to it and- * pressing Enter, and `number` carries its INDEX rather than a key.- *- * The direction is a property of the DIALOG, not of the engine. OpenCode draws BOTH a numbered ask- * dialog and — for permissions — the horizontal prompt kilo inherited from it, so keying by engine- * would send digits into a dialog that numbers nothing and select nothing at all.- */- walk?: 'right' | 'down'-}--export interface QuestionView {- kind: 'question'- permission?: boolean- /**- * The dialog is on screen but its top — the question and its first rows — is scrolled out of the pane- * (Codex's request_user_input keeps the footer anchored and lets a short pane cut the top off; measured- * in a four-pane window: rows 3–5 and the footer visible, "1." and the question gone). Enough to know a- * dialog is STILL OPEN — the watcher must not close it — and not enough to announce as a question.- */- partial?: boolean- /** The footer says a digit only highlights and Enter commits (`enter to submit answer` — Codex's- * request_user_input). Absent where one digit selects and submits, which is every other dialog. */- enterSubmits?: boolean- question: string- rows: QuestionRow[]- multi: boolean- /** The "Type something." row, when the dialog offers free text. */- typeRow: QuestionRow | null- /**- * A permission prompt's WHOLE dialog, every line as painted (ANSI stripped), from its frame to its last- * row. `question` is one line clipped for a device's screen; a command that wraps — `npm test &&` on one- * line, `git push` on the next — is only whole here. What the pair brain's floor reads (pair/classify.ts),- * and part of the dialog's fingerprint, so two prompts that differ below their first line are two ids.- */- dialog?: string-}--export interface ReviewView {- kind: 'review'- submitRow: string-}--export type PaneView = QuestionView | ReviewView | null--const STEP_MS = 350 // let the TUI repaint between keystrokes-const TEXT_MS = 250-const MAX_STEPS = 14 // hard bound on the drive loop (questions × keys), never spin on a stuck pane-const CAPTURE_LINES = 60--// Rows that exist in every dialog but can never BE an answer.-const CHAT_ROW = /^chat about (this|these)$/i-// Claude writes "Type something.", Command Code "Type something..." — one trailing dot or three, and-// missing it costs the free-text row: a voice answer would have nowhere to go and the row would be-// offered on the device as if it were a real option.-// Claude "Type something.", Command Code "Type something...", OpenCode "Type your own answer", Hermes-// "Other (type your answer)". Same row in every dialog: it opens an editor instead of choosing, so it must-// never be offered to the device as a selectable label.-const TYPE_ROW = /^(type something\.{0,3}|type your own( answer)?|other \(type your (own|answer)\))$/i--function stripAnsi(value: string): string {- return value.replace(/\u001b\[[0-9;:]*[A-Za-z]/g, '')-}--function norm(value: string): string {- return value.toLowerCase().replace(/\s+/g, ' ').replace(/[.…]+$/, '').trim()-}--function sleep(ms: number): Promise<void> {- return new Promise((resolve) => setTimeout(resolve, ms))-}--/**- * Shape a raw AskUserQuestion input into the device form. Port of the hosted runtime’s `commanderQuestions`- * (websocket.ts): `key` MUST stay the value the CLI matches answers by — prompt/question first — because- * the device echoes it back as the answers-map key.- */-export function shapeQuestions(questions: unknown): ShapedQuestion[] {- return (Array.isArray(questions) ? questions : []).map((raw, i) => {- const q = (raw ?? {}) as Record<string, unknown>- const opts = Array.isArray(q.options) ? q.options : []- return {- key: (q.prompt as string) || (q.question as string) || (q.id as string) || (q.header as string) || `question_${i}`,- q: (q.prompt as string) || (q.question as string) || '',- options: opts- .map((o) => String((typeof o === 'string' ? o : (o as Record<string, unknown>)?.label) ?? ''))- .filter(Boolean),- // Three spellings because three CLIs: `allow_multiple` (the hosted runtime), `multiSelect` (claude),- // `multi_select` (devin — read from a real `ask_user_question` call in its SQLite store).- multi: (q.allow_multiple as boolean) ?? (q.multiSelect as boolean) ?? (q.multi_select as boolean) ?? false,- }- })-}--/** A dialog one reader found, and the line it anchored on: what ranks two readers of the same pane. */-export interface FoundDialog {- view: QuestionView | ReviewView- at: number-}--function permissionView(view: PaneView): PaneView {- return view?.kind === 'question' ? { ...view, permission: true } : view-}--function asPermission(found: FoundDialog | null): FoundDialog | null {- return found && { ...found, view: permissionView(found.view) as QuestionView }-}--/**- * The LOWEST dialog any reader found: the live one. A pane keeps an answered dialog in its scrollback, and- * an engine with two readers (a question and an approval, say) must never let the first reader's hit on- * that one shadow the second reader's hit on the dialog under it: devin's approval under its answered- * question, muse's approval under a question, opencode's question under an approval. A tie is one dialog- * read twice, and the earlier reader, the engine's own, keeps it.- */-function lowest(...found: Array<FoundDialog | null>): PaneView {- let best: FoundDialog | null = null- for (const f of found) if (f && (!best || f.at > best.at)) best = f- return best?.view ?? null-}--/**- * The dialog on screen, read the way `engine` paints it: the LAST one on the pane, and nothing of it from- * above an earlier dialog's end (`dialogEnd.ts`).- *- * Claude and Command Code share one shape (see parseQuestionPane); devin draws a different one and gets- * its own parser rather than more branches in here.- */-export function parseEngineQuestionPane(engine: AgentEngine, capture: string): PaneView {- if (engine === 'claude' || engine === 'codex') {- const found = locateQuestionPane(capture)- if (!found) return null- // Captures include scrollback. The long chaos run returned to the composer after answering, but- // the old dialog still held every prompt as question_open. A full composer BELOW that dialog makes- // it history. One above it proves nothing: a newer dialog can have only its last rows visible.- const below = capture.split('\n').slice(found.at + 1).join('\n')- if (composerState(engine, below) !== 'absent') return null- return engine === 'codex' ? withCodexLabels(found.view) : found.view- }- // Devin's two dialogs are told apart by one word in the footer (`↵ select` vs `↵ confirm`), so they can- // never both match the same dialog; but an answered one of either kind can sit above the live one.- if (engine === 'devin') return lowest(locateDevinQuestion(capture), asPermission(locateDevinPermission(capture)))- // Cursor has no ask-the-user tool, so its permission prompt is the ONLY dialog it ever draws — and it- // numbers nothing, stating each row's key in the row instead.- if (engine === 'cursor') return permissionView(parseCursorPermissionPane(capture))- // Muse pairs each option with a description on the same line and floats a live Preview box above the- // rows — both confuse the shared parser, so it reads its own. Its PERMISSION prompt is a different- // dialog entirely (`Would you like to allow this network access?` over `1. Yes, proceed (y)` rows under- // a `Press enter to confirm` footer, `__fixtures__/permission-muse.txt`) and that one the shared parser- // reads exactly, so it is read by both and the lower wins.- if (engine === 'muse') return lowest(locateMuseQuestion(capture), locateQuestionPane(capture))- // Amp's is a permission prompt with unnumbered rows — nothing the shared parser can anchor on.- if (engine === 'amp') return permissionView(parseAmpQuestionPane(capture))- // Kilo's is the same kind of prompt but laid out HORIZONTALLY, sharing its line with the key hints —- // it is a fork of opencode that did not keep opencode's dialog.- if (engine === 'kilo') return permissionView(parseKiloQuestionPane(capture))- // Grok tells its approval from its questionnaire itself, by the live dialog's own footer.- if (engine === 'grok') return parseGrokQuestionPane(capture)- // agy's ask-the-user dialog anchors on `Question N/M:` under an `↑/↓ Navigate` footer, which the- // shared parser cannot see. Its PERMISSION prompt is numbered rows under `Do you want to proceed?`- // and the shared parser reads that one exactly, so it is read by both and the lower wins.- if (engine === 'agy') return lowest(locateAgyQuestion(capture), locateQuestionPane(capture))- // Hermes, OpenCode and Copilot paint the dialog inside a box; peel the border and the shared parser- // fits. Measured on Copilot: framed it returns null, unframed it reads the question, the three- // options AND spots `4. Other (type your answer)` as the free-text row rather than an option.- if (engine === 'hermes') return parseQuestionPane(unframe(capture))- // Copilot boxes its dialog the same way, but names the SUBJECT of a permission prompt above the- // question — "attempting to access the following URL:" over a boxed value. Without it the device- // shows "Do you want to allow this access?" and a bare "Yes", with nothing to judge.- if (engine === 'copilot') {- const found = locateQuestionPane(unframe(capture))- return found ? withCopilotSubject(found.view, capture, found.at) : null- }- if (engine === 'opencode') {- // OpenCode's PERMISSION prompt is the horizontal one kilo inherited from it — same `△ Permission- // required` title, same `⇆ select · enter confirm` footer, same unnumbered rows. Measured: the live- // capture in `permission-opencode.txt` parses through kilo's parser unchanged, so it is shared rather- // than copied. Listed FIRST because that dialog numbers nothing: on its own dialog the shared parser- // still matches the `enter confirm` footer and walks up into whatever numbered rows are above, and- // the tie goes to kilo's reader.- const plain = unframe(capture)- return lowest(asPermission(locateKiloQuestion(capture)), locateOpencodeReview(plain), locateQuestionPane(plain))- }- return parseQuestionPane(capture)-}--/**- * Codex's request_user_input rows carry the option's description on the same line as its label, in- * an aligned column — `1. Red Creates a bold, high-contrast` — and wrap the rest of the description- * onto plain lines below. The label is the part before the column gap; the dial has an 80-byte option- * buffer and a 466px face, and "Red" is what the person is choosing. The approval prompt's rows- * (`Yes, proceed (y)`) have no such gap and pass through unchanged.- */-function withCodexLabels(view: PaneView): PaneView {- if (!view || view.kind !== 'question') return view- return {- ...view,- rows: view.rows.map((r) => ({ ...r, label: r.label.split(/\s{2,}/)[0].trim() || r.label })),- }-}--/**- * Strip the box-drawing frame Hermes draws around its `clarify` dialog.- *- * Measured on a live pane: Hermes paints the SAME dialog Claude does — `❯ 1. Xanh` rows, a footer reading- * `↑/↓ to select, Enter to confirm` — only wrapped in `│ … │`. Peeling the border makes the existing- * parser fit exactly, which is far better than a second parser that would drift from it over time.- */-/**- * OpenCode's final step of a multi-question dialog.- *- * It is a REVIEW, not a question: a "Review" heading over `label: answer` lines, with no numbered rows at- * all, submitted by pressing Enter. The shared parser needs rows to recognise a dialog, so without this- * the driver saw "nothing on screen", stopped, and left the agent sitting on an unsubmitted form after- * every answer had been given (measured on a live pane).- *- * `submitRow` carries the KEY to press, which for every other CLI happens to be a digit — 'Enter' rides- * the same field rather than widening the type for one engine.- */-function locateOpencodeReview(plain: string): FoundDialog | null {- const lines = stripAnsi(plain).split('\n')- const footer = lines.findLastIndex((l) => /enter\s+submit/i.test(l))- if (footer < 0) return null- const floor = earlierDialogEnd(lines, footer, 14)- let sawReview = false- for (let i = footer - 1; i > floor && footer - i <= 14; i--) {- if (parseRow(lines[i])) return null // rows above ⇒ still a question, not the review- if (/^\s*review\s*$/i.test(lines[i])) { sawReview = true; break }- }- return sawReview ? { view: { kind: 'review', submitRow: 'Enter' }, at: footer } : null-}--function unframe(capture: string): string {- // stripAnsi FIRST: on a real capture the border is preceded by SGR codes, so matching `^│` against the- // raw text silently never fired — hermes read as "no dialog open" with the dialog plainly on screen.- return stripAnsi(capture)- .split('\n')- .map((line) => {- const left = line.replace(/^(\s*)[│┃|]\s?/, '$1')- // An unanchored whitespace regex retries at every column on blank rows.- // Inspect the last non-space character once; keep unframed lines intact.- const right = left.trimEnd()- return /[│┃|]$/.test(right) ? right.slice(0, -1).trimEnd() : left- })- .join('\n')-}--/** How this engine's multi-select dialog is submitted once the boxes are ticked. */-function multiSubmitKey(engine: AgentEngine): string {- // Claude advances with Tab; devin submits with Enter (measured on a live pane — Tab does nothing there).- return engine === 'devin' ? 'Enter' : 'Tab'-}--function parseRow(line: string): QuestionRow | null {- // Consume indentation once: two adjacent whitespace runs backtrack across- // every possible split on the padded non-option lines in terminal captures.- const m = /^[❯›>]?\s*(\d+)\.\s+(.+?)\s*$/.exec(line.trimStart())- if (!m) return null- const raw = m[2]- const box = /^\[([^\]])\]\s*(.*)$/.exec(raw)- return {- number: m[1],- label: (box ? box[2] : raw).trim(),- checked: !!box && box[1].trim() !== '',- }-}--/**- * Read the CURRENT dialog off a pane capture. The capture includes scrollback (old dialogs, old plan- * text), so everything is anchored to the LAST footer line — the only marker the live dialog always- * paints at the bottom of the screen.- */-/** "❯ 1. Submit" immediately followed by "2. Cancel" — Command Code's review screen, which carries no- * other marker. Returns the Submit row's number and where it sits, or null. */-function findSubmitPair(lines: string[]): { row: string; index: number } | null {- for (let i = lines.length - 1; i >= 1; i--) {- const row = parseRow(lines[i])- if (!row || !/^submit$/i.test(row.label)) continue- const next = parseRow(lines[i + 1] ?? '')- if (next && /^cancel$/i.test(next.label)) return { row: row.number, index: i }- }- return null-}--/** The tab bar of a footer-less dialog: "● Loại game | ◯ Review". Returns its LAST occurrence, or -1. */-function findTabBarDialog(lines: string[]): number {- for (let i = lines.length - 1; i >= 0; i--) {- const line = lines[i].trim()- if (!/^[●◯✔✓]/.test(line) || !line.includes('|')) continue // answered tabs turn into ✔- // Only a tab bar with a numbered list under it is a question; the same glyphs appear in prose.- for (let j = i + 1; j < lines.length && j - i <= 8; j++) {- const row = parseRow(lines[j])- if (row?.number === '1') return i- }- }- return -1-}--/** Read a dialog that has no footer: question first, then the numbered rows, top-down from the tab bar. */-function parseDownward(lines: string[], tabBar: number): PaneView {- let question = ''- const rows: QuestionRow[] = []- let checkbox = false- for (let i = tabBar + 1; i < lines.length; i++) {- const row = parseRow(lines[i])- if (row) {- if (/^\s*[❯›>]?\s*\d+\.\s+\[/.test(lines[i])) checkbox = true- rows.push(row)- continue- }- // Between the tab bar and row 1 sits the question; after the rows start, plain lines are the option- // descriptions and must not overwrite it.- const line = lines[i].trim()- if (!question && rows.length === 0 && line && !/^[─━-]{6,}$/.test(line)) question = line- }- if (!rows.length) return null- const answerable = rows.filter((r) => !CHAT_ROW.test(r.label) && !TYPE_ROW.test(r.label))- return {- kind: 'question',- question,- rows: answerable,- multi: checkbox,- typeRow: rows.find((r) => TYPE_ROW.test(r.label)) ?? null,- }-}--// \u2500\u2500 permission prompts \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500--/**- * A permission prompt is a QUESTION whose options are the approval choices \u2014 the shape amp and kilo- * already ship (`engines/amp/askQuestion.ts`, `engines/kilo/askQuestion.ts`). It matters far more on a- * remote machine than a question does: the CLI attaches to an agent the USER started, under the user's- * own config and with no permission flag of ours (`engines/README.md`), so a blocking approval is the- * normal state of a pane, not an edge case. Unparsed, the turn simply sits at `Processing` until someone- * walks to the computer.- *- * Claude and Command Code draw it as a framed block ending in numbered rows, and it carries NONE of the- * three anchors `parseQuestionPane` knows. Captured live (`__fixtures__/permission-claude.txt`):- *- * \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500- * Bash command \u2190 the header: what kind of approval this is- *- * curl -s https://api.coingecko.com/\u2026 \u2190 the argument, then a one-line description- * Fetch Bitcoin price from CoinGecko API- *- * This command requires approval- *- * Do you want to proceed?- * \u276f 1. Yes- * 2. Yes, and don\u2019t ask again for: curl *- * 3. No- *- * Esc to cancel \u00b7 Tab to amend \u00b7 ctrl+e to explain- *- * Command Code differs only in wording (`permission-commandcode.txt`): header `Execute Shell Command`,- * a sentence for the body, footer `\u2191/\u2193 navigate \u00b7 enter select \u00b7 ctrl+e explain`. Same frame, same rows \u2014- * which is why one parser covers both instead of two near-copies that would drift apart.- *- * **The prose is not the anchor.** Every engine words it differently and Claude alone writes at least- * three ("Do you want to proceed?", "Do you want to make this edit to README.md?", "Do you want to create- * \u2026?"). The ROWS are: an approval always offers a way to say yes and a way to say no. That pair is the- * signal, guarded by the dialog's own key hints so an ordinary numbered list in assistant output can never- * be read as a prompt.- */--/** The first row of an approval, measured across claude, commandcode, codex, devin, grok, amp and kilo. */-const APPROVE_RE = /^(yes|allow|approve|accept|proceed|run|continue)\b/i-/** \u2026and the row that declines it. Never dropped \u2014 see the rule amp's parser states: a device user given- * three ways to say yes and none to say no cannot answer the prompt at all. */-// `skip` is here because cursor's decline row is "Skip & tell the agent what to do instead" — an engine-// whose only way to refuse says neither "no" nor "reject" would otherwise fail the yes/no guard and its-// whole prompt would go unread.-// Claude's plan review does not use a negative verb at all: its only refusal-// row is "Tell Claude what to change". Keep `tell ... what to change` narrow so-// an ordinary numbered list beginning with "Tell" cannot become an approval.-const REJECT_RE = /^(no|reject|deny|decline|cancel|skip|don'?t|stop)\b|^tell\b.*\bwhat to change\b/i-/** The solid rule that opens the frame. Deliberately NOT the dashed one (`\u254c`) that brackets an edit diff,- * which sits BELOW the header and would cost the title. */-const FRAME_RULE_RE = /^\s*[\u2500\u2501\u2550]{6,}\s*$/-/** Any rule, solid or dashed \u2014 used to skip them while reading the frame's contents. */-const ANY_RULE_RE = /^\s*[\u2500\u2501\u2550\u254c\u2504\u2508-]{6,}\s*$/-/** The dialog's own question line, and Command Code's `Press [ctrl+e] \u2026` hint: both sit between the- * header and the rows, and neither says what is being approved. */-const PERMISSION_PROSE_RE = /^((do|would) you\b|press \[)/i--/** The opening rule of the frame the rows at `start` sit in, or -1 when they have none of their own. */-function frameTop(lines: string[], start: number, floor: number): number {- for (let i = start - 1; i > floor && start - i <= 25; i--) {- if (FRAME_RULE_RE.test(lines[i])) return i- }- return -1-}--/** Keep a synthesised title inside the device's `text[200]` buffer, with the tail marked as cut. */-function clipTitle(value: string): string {- return value.length <= 160 ? value : `${value.slice(0, 159)}\u2026`-}--/**- * What is being approved, as one line: `Approve <header>: <argument>`, the form amp's parser already- * produces so the two read alike on the device.- *- * The frame's opening rule is the only reliable top \u2014 the prose under it varies per engine AND per tool.- * Below the rule sit the header (`Bash command`, `Edit file`, `Execute Shell Command`) and then the- * argument: the command, the file, or the sentence naming it.- */-function permissionTitle(lines: string[], start: number): string {- // The header must be THIS dialog's: never read past the end of an earlier one still in scrollback.- const floor = earlierDialogEnd(lines, start, 25)- const top = frameTop(lines, start, floor)- // No frame above the rows (codex draws none): the nearest text is the dialog's own question, which- // names the command outright. Better than inventing a header that is not on screen.- if (top < 0) {- for (let i = start - 1; i > floor && start - i <= 4; i--) {- const line = lines[i].trim()- if (line) return clipTitle(line)- }- return 'Approval required'- }- let header = ''- let arg = ''- for (let i = top + 1; i < start; i++) {- const line = lines[i].trim()- if (!line || ANY_RULE_RE.test(line)) continue- if (!header) { header = line; continue }- if (PERMISSION_PROSE_RE.test(line)) continue- arg = line- break- }- if (!header) return 'Approval required'- return clipTitle(arg ? `Approve ${header}: ${arg}` : `Approve ${header}`)-}--/**- * The permission prompt on screen, with the index of its first row so the caller can rank it against the- * other anchors.- *- * Only the LOWEST numbered block on the pane is considered, and it is rejected outright if it is not an- * approval. Digging further up would find an answered prompt in the scrollback and re-announce it \u2014 the- * live dialog is always the bottom-most thing on screen, so there is nothing below it to miss.- */-export function parsePermissionPane(lines: string[]): { view: QuestionView; index: number } | null {- let end = -1- for (let i = lines.length - 1; i >= 0; i--) {- if (parseRow(lines[i])) { end = i; break }- }- if (end < 0) return null-- // Walk up through CONTIGUOUS rows to the one numbered 1. Contiguity is the point: a permission dialog- // never interleaves prose with its options, while assistant output that happens to be numbered does.- const rows: QuestionRow[] = []- let start = -1- for (let i = end; i >= 0 && end - i < 12; i--) {- const row = parseRow(lines[i])- if (!row) break- rows.unshift(row)- if (row.number === '1') { start = i; break }- }- if (start < 0 || rows.length < 2) return null- if (!APPROVE_RE.test(rows[0].label) || !rows.some((row) => REJECT_RE.test(row.label))) return null-- let hinted = false- for (let i = end + 1; i < lines.length && i - end <= 4; i++) {- if (PERMISSION_FOOTER_RE.test(lines[i])) { hinted = true; break }- }- if (!hinted) return null-- // Single-select, and no free-text row: every option here is a choice to be TAPPED. Verified on live- // panes for claude, codex, devin and grok \u2014 one digit selects and submits, exactly as `rowKeys` assumes.- return {- view: { kind: 'question', permission: true, question: permissionTitle(lines, start), rows, multi: false, typeRow: null, dialog: permissionDialog(lines, start, end) },- index: start,- }-}--/** Every line of the dialog, untruncated: from its opening rule (or, with no frame, up to 12 lines above- * the rows) to its last row — never reaching back past the end of an earlier dialog. */-function permissionDialog(lines: string[], start: number, end: number): string {- const floor = earlierDialogEnd(lines, start, 25)- const rule = frameTop(lines, start, floor)- const top = rule >= 0 ? rule + 1 : Math.max(0, start - 12, floor + 1)- return lines.slice(top, end + 1).map((line) => line.trimEnd()).filter((line) => !ANY_RULE_RE.test(line)).join('\n').trim()-}--export function parseQuestionPane(capture: string): PaneView {- return locateQuestionPane(capture)?.view ?? null-}--/** `parseQuestionPane`, with the line its dialog was anchored on. */-function locateQuestionPane(capture: string): FoundDialog | null {- const lines = stripAnsi(capture).replace(/\u00a0/g, ' ').split('\n')- // Each CLI words its own footer, and OpenCode rewords it PER SCREEN — `enter submit` on a single- // question, `enter toggle` on a multi-select, `enter confirm` on a step of a multi-question. They all- // mark the same thing: the bottom of a live dialog.- // `enter to submit answer` is Codex's request_user_input dialog (plan mode; captured live in- // __fixtures__/question-codex.txt). Without it that dialog was never a question at all here: the dial- // showed nothing while the pane waited, and a question it DID show could never be closed, because- // the watcher had no fingerprint to notice leaving.- const footer = lines.findLastIndex((l) => QUESTION_FOOTER_RE.test(l))- // The review screen paints no footer and puts its rows BELOW the prompt, so it needs its own anchor.- // Whichever anchor is LOWER on screen is the live one (the other is scrollback from an earlier step).- const review = lines.findLastIndex((l) => /Ready to submit your answers/i.test(l))- // Command Code's review screen has neither Claude's "Ready to submit your answers" line nor a footer:- // it is a "Submit"/"Cancel" pair under a summary that is itself numbered. Anchor on that PAIR — the two- // rows adjacent, in that order — because the summary lines above are numbered too and reading them as- // options is how the device answered everything and then sat there, never submitting.- const submit = findSubmitPair(lines)- // Command Code paints the SAME dialog with no footer at all — the pane simply ends at the last option.- // Its tab bar is the only thing above the rows that is unmistakably part of the dialog, so it anchors- // that dialog, read DOWNWARD. Nothing else on either CLI's screen looks like "● X | ◯ Y" followed by a- // numbered list, which is what keeps ordinary numbered output from being read as a question.- const tabBar = findTabBarDialog(lines)- // A permission prompt is one more anchor, and every anchor is ranked the same way: whichever sits- // lowest on screen is the live dialog, the others are scrollback. That ordering is what keeps the two- // apart in both directions — codex and hermes draw an approval whose footer the question anchor also- // matches, and there the footer is BELOW the rows, so the question path (which reads a better title off- // the same block) still wins. And a footer-less dialog under an answered one is the live one: anchored- // on the answered one's footer instead, Command Code's question was announced as the OLD question.- const permission = parsePermissionPane(lines)- if (permission && permission.index > Math.max(footer, review, submit?.index ?? -1, tabBar)) {- return { view: permission.view, at: permission.index }- }- if (tabBar > Math.max(footer, review, submit?.index ?? -1)) {- const view = parseDownward(lines, tabBar)- return view && { view, at: tabBar }- }- if (review > footer && review > (submit?.index ?? -1)) {- for (let i = review + 1; i < lines.length && i - review <= 10; i++) {- const row = parseRow(lines[i])- if (row && /^submit answers$/i.test(row.label)) return { view: { kind: 'review', submitRow: row.number }, at: review }- }- return null- }- if (submit && submit.index > footer) return { view: { kind: 'review', submitRow: submit.row }, at: submit.index }- if (footer < 0) return null-- // Rows belonging to this dialog: the numbered rows just above the footer, back to the row numbered 1 —- // and never from above an earlier dialog's end, which is where a dialog whose top is scrolled out of- // the pane would otherwise borrow its first rows.- const rows: QuestionRow[] = []- let checkbox = false // `[ ]` / `[✔]` on a row ⇒ this question is multi-select- let start = -1- const floor = earlierDialogEnd(lines, footer, 40)- for (let i = footer - 1; i > floor && footer - i <= 40; i--) {- const row = parseRow(lines[i])- if (!row) continue- rows.unshift(row)- if (/^\s*[❯›>]?\s*\d+\.\s+\[/.test(lines[i])) checkbox = true- if (row.number === '1') { start = i; break }- }- const enterSubmits = /enter to submit answer/i.test(lines[footer])- if (rows.length && start < 0 && enterSubmits) {- return { view: { kind: 'question', partial: true, enterSubmits, question: '', rows, multi: checkbox, typeRow: null }, at: footer }- }- if (start < 0 || rows.length === 0) return null-- // The question is the nearest real text line above the rows. The dialog paints it between its header- // and its rows — `[tab bar | header chip] · blank · question · blank · rows` — so those, and a rule,- // are the TOP of this frame. Stop there, never skip past: mid-repaint the question line can be blank- // for one capture, and walking on would pick up the PREVIOUS question still sitting in scrollback and- // pair a stale title with the live options. An empty result just means "look again next tick". The- // end of an earlier dialog is a top too: its footer is not this dialog's question.- let question = ''- const top = earlierDialogEnd(lines, start, 12)- for (let i = start - 1; i > top && start - i <= 12; i--) {- const line = lines[i].trim()- if (!line) continue- if (/[←→]/.test(line) || /^[☐☒✔✓]/.test(line) || /^[─━-]{6,}$/.test(line)) break- // Hermes's batch panel marks the active question with `▸`; that marker is chrome, not the question.- question = line.replace(/^▸\s*/, '')- break- }-- const answerable = rows.filter((r) => !CHAT_ROW.test(r.label) && !TYPE_ROW.test(r.label))- return {- view: {- kind: 'question',- ...(enterSubmits ? { enterSubmits } : {}),- question,- rows: answerable,- multi: checkbox,- typeRow: rows.find((r) => TYPE_ROW.test(r.label)) ?? null,- dialog: dialogAbove(lines, start, footer),- },- at: footer,- }-}--/**- * A footer-anchored dialog, whole: up to 12 lines above its rows — stopping at a rule, a tab bar, the- * agent's own output (a `•`/`⏺` bullet) or the end of an earlier dialog — down to its last row. Codex puts- * the command it asks about here (`$ …`, wrapped over as many lines as it takes), with its reason and- * environment.- */-function dialogAbove(lines: string[], start: number, footer: number): string {- let top = start- const floor = earlierDialogEnd(lines, start, 12)- for (let i = start - 1; i > floor && start - i <= 12; i--) {- const line = lines[i].trim()- if (/^[•⏺●]/.test(line) || /[←→]/.test(line) || /^[─━═-]{6,}$/.test(line)) break- top = i- }- return lines.slice(top, footer).map((line) => line.trimEnd()).join('\n').trim()-}--/**- * A dialog that asks to RUN or CHANGE something (an approval), whichever parser read it: a framed- * permission prompt, or a footer dialog whose first row approves, whose rows include a rejection, and- * which names a command (`$ …`) or asks "would you like to run / make …". What the pair's floor treats as- * a permission prompt (pair/classify.ts); nothing else reads it.- */-export function isApprovalDialog(view: QuestionView): boolean {- if (view.permission) return true- if (!view.rows.length || !APPROVE_RE.test(view.rows[0].label) || !view.rows.some((row) => REJECT_RE.test(row.label))) return false- const text = view.dialog ?? view.question- return /(^|\n)\s*\$ /.test(text) || /would you like to (run|make|apply|execute|edit|write)/i.test(text)-}--/** Match an answer to a row. The device stores option labels in an 80-byte buffer, so a long label comes- * back truncated — prefix matches count, in both directions. */-/**- * The keystrokes that commit one row.- *- * Every engine but Amp numbers its options, so the digit both selects and submits in one press. Amp- * draws an unnumbered list navigated with the arrow keys, so its rows carry an index and are reached by- * walking down to them.- */-function rowKeys(engine: AgentEngine, row: QuestionRow, view?: QuestionView): string[] {- if (engine === 'amp') return ampSelectionKeys(row)- if (engine === 'codex') return codexRowKeys(row, view)- // Kilo's rows sit side by side, so its walk is horizontal — see engines/kilo/askQuestion.ts.- if (engine === 'kilo') return kiloSelectionKeys(row)- // Same dialog, different engine: opencode numbers its ask dialog but not its permission prompt, so the- // ROW says how it is reached and a per-engine rule would break one of the two.- if (row.walk === 'right') return kiloSelectionKeys(row)- if (row.walk === 'down') return ampSelectionKeys(row)- return [row.number]-}--/**- * Codex, measured 2026-09-15 on 0.149: in its request_user_input dialog a digit only MOVES the highlight- * and Enter submits (`enter to submit answer`) — a digit alone left the dialog up and the device's answer- * reported as stuck. Its approval prompt is the other way round and was verified earlier: one digit- * selects and commits, so that one keeps the single key.- */-export function codexRowKeys(row: QuestionRow, view?: QuestionView): string[] {- return view?.enterSubmits ? [row.number, 'Enter'] : [row.number]-}--export function matchRow(rows: QuestionRow[], answer: string): QuestionRow | null {- const a = norm(answer)- if (!a) return null- return rows.find((r) => norm(r.label) === a)- ?? (a.length >= 3 ? rows.find((r) => norm(r.label).startsWith(a)) ?? rows.find((r) => a.startsWith(norm(r.label)) && norm(r.label).length >= 3) : undefined)- ?? null-}--/**- * Pick the answer for the question the dialog is currently showing: the entry keyed by its own text.- *- * `positional` also takes the next unused entry when none names it. Only for a dialog the answer's- * requestId proves it was written for: without that proof, an answer that names no question on screen- * belongs to one that is gone, and typing it here would answer — or approve — something nobody saw.- *- * The text must be the question's OWN (case, spacing and a trailing `…` aside), never a prefix either way.- * An approval is titled `Approve <header>: <argument>` and the header is shared by every prompt of its- * kind: a key left from an earlier prompt — `Approve Bash command` (its argument unread) — was a prefix of- * `Approve Bash command: rm -rf ~/projects` and pressed Yes on it, and an old full title named a- * header-only prompt the other way round. A client echoes back the key it was announced, so the whole- * text is always there to match; one it has cut is answered through its requestId (`positional`).- */-export function pickAnswer(- answers: Record<string, string>,- question: string,- used: Set<string>,- opts: { positional?: boolean } = {},-): { key: string; value: string } | null {- const entries = Object.entries(answers)- const q = norm(question)- const byText = q ? entries.find(([k]) => norm(k) === q) : undefined- if (byText && !used.has(byText[0])) return { key: byText[0], value: byText[1] }- if (!opts.positional) return null- const next = entries.find(([k]) => !used.has(k))- return next ? { key: next[0], value: next[1] } : null-}--/**- * The id a dialog is announced under — the SAME function the watcher names it with, so the answer's- * requestId can be checked against the dialog on screen at the moment of typing rather than against- * whatever the watcher last saw (it polls every 1.5s, and forgets on a reset).- */-export function questionRequestId(sessionId: string, view: QuestionView): string {- return `q_${hash(sessionId + fingerprintOf(view))}`-}--/** Why an answer was not keyed. Sent back to the client as `question_response_result.error`. */-export type QuestionAnswerError = 'STALE_QUESTION' | 'AGENT_NOT_FOUND' | 'ANSWER_BUSY' | 'ANSWER_FAILED'--export type QuestionAnswerResult = { ok: true } | { ok: false; error: QuestionAnswerError; detail: string }--const STALE_CHANGED: QuestionAnswerResult = { ok: false, error: 'STALE_QUESTION', detail: 'That question changed before your answer arrived.' }-const STALE_GONE: QuestionAnswerResult = { ok: false, error: 'STALE_QUESTION', detail: 'That question is no longer open.' }-const failed = (detail: string): QuestionAnswerResult => ({ ok: false, error: 'ANSWER_FAILED', detail })-const KEYS_FAILED = failed('The answer could not be typed into the agent\'s terminal.')-const STUCK = failed('The question did not take the answer.')--export interface AskQuestionDeps {- getSession: (sessionId: string) => RegisteredSession | undefined- capture: (terminalTarget: string, historyLines?: number) => Promise<string | null>- sendText: (terminalTarget: string, text: string) => Promise<boolean>- sendKey: (terminalTarget: string, key: string) => Promise<boolean>- /** Pins one backend locator for the whole multi-step dialog drive. */- acquireControl?: (sessionId: string, opts?: { forAnswer?: boolean }) => (() => void) | null- /** Injected for tests. */- wait?: (ms: number) => Promise<void>-}--export interface QuestionAnswerPayload {- /** Exact contents reviewed on the device; guarded submissions never use positional fallback. */- expectedQuestions?: ShapedQuestion[]- selectedLabels?: Record<string, string[]>- freeTextKeys?: string[]- allowPermissions?: boolean- requestId?: string- sessionId?: string- agentId?: string- answers?: Record<string, string>-}--/**- * Owns the OUT side's pending map (requestId → session) and the IN side's pane driving. One answer at a- * time per agent: a second `question_response` for a dialog already being driven is dropped, not queued.- */-export class AskQuestionController {- private pending = new Map<string, string>() // requestId → sessionId- private driving = new Set<string>() // sessionIds currently keying a dialog-- constructor(private readonly deps: AskQuestionDeps) {}-- /** Remember which session a mirrored question belongs to (the device may answer minutes later). */- remember(requestId: string, sessionId: string): void {- if (!requestId) return- if (this.pending.size > 64) this.pending.delete(this.pending.keys().next().value as string)- this.pending.set(requestId, sessionId)- }-- async answer(payload: QuestionAnswerPayload): Promise<QuestionAnswerResult> {- const requestId = payload.requestId ?? ''- if (payload.expectedQuestions !== undefined && (!Array.isArray(payload.expectedQuestions) ||- payload.expectedQuestions.length < 1 || payload.expectedQuestions.length > 4 ||- payload.expectedQuestions.some(q => !q || typeof q.key !== 'string' || !q.key ||- typeof q.q !== 'string' || !q.q || !Array.isArray(q.options) || !q.options.length ||- q.options.some(option => typeof option !== 'string' || !option) ||- typeof q.multi !== 'boolean' || (q.multi && !Array.isArray(payload.selectedLabels?.[q.key]))))) {- return failed('The reviewed question metadata is invalid.')- }- if (payload.freeTextKeys !== undefined && (!payload.expectedQuestions || !Array.isArray(payload.freeTextKeys) ||- payload.freeTextKeys.some(key => typeof key !== 'string' || !payload.expectedQuestions!.some(q =>- q.key === key && q.canText === true && !q.multi)))) {- return failed('The reviewed text answer metadata is invalid.')- }- const remembered = this.pending.get(requestId)- const sessionId = payload.sessionId || payload.agentId || remembered || ''- const answers = payload.answers && typeof payload.answers === 'object' ? payload.answers : null- if (!sessionId || !answers || Object.keys(answers).length === 0) {- console.warn(`[question] ignoring answer with no session/answers (req=${requestId})`)- return failed('The answer named no agent or carried no choice.')- }- const session = this.deps.getSession(sessionId)- const terminalTarget = session?.agentId || session?.sessionId- if (!terminalTarget) {- console.warn(`[question] no terminal target for ${sessionId.slice(0, 8)} — answer dropped`)- return { ok: false, error: 'AGENT_NOT_FOUND', detail: 'That harness is no longer running.' }- }- if (remembered) {- const owner = this.deps.getSession(remembered)- if ((owner?.agentId || owner?.sessionId) !== terminalTarget) return STALE_CHANGED- }- if (this.driving.has(terminalTarget)) {- console.warn(`[question] ${sessionId.slice(0, 8)} answer dropped · already driving this dialog`)- return { ok: false, error: 'ANSWER_BUSY', detail: 'Another answer is already being entered for this harness.' }- }- // `forAnswer`: a dialog is the engine waiting for input mid-turn, so the open turn must not block it.- const release = this.deps.acquireControl?.(terminalTarget, { forAnswer: true })- // Silence here is the failure mode this whole file exists to prevent: the device sends an answer,- // nothing keys it in, and the pane sits on the dialog looking like a hung agent.- if (this.deps.acquireControl && !release) {- console.warn(`[question] ${sessionId.slice(0, 8)} answer dropped · terminal control unavailable`)- return { ok: false, error: 'ANSWER_BUSY', detail: 'The agent\'s terminal is busy. Try again.' }- }- // The ids the watcher could have announced this dialog under: the session it was remembered for, and- // the session as the registry knows it now.- const owners = [...new Set([remembered, session?.sessionId].filter((id): id is string => !!id))]- this.driving.add(terminalTarget)- try {- const result = await this.drive(terminalTarget, answers, this.deps.getSession(sessionId)?.engine ?? 'claude', payload.allowPermissions !== false, { requestId, owners }, payload.expectedQuestions ? payload : undefined)- this.pending.delete(requestId)- const outcome = result.ok ? 'submitted' : result.error === 'STALE_QUESTION' ? 'refused · STALE_QUESTION, nothing typed' : 'FAILED'- console.log(`[question] ${sessionId.slice(0, 8)} answered from device · ${outcome} (req=${requestId || 'none'})`)- return result- } finally {- this.driving.delete(terminalTarget)- release?.()- }+export { isApprovalDialog, parsePermissionPane, parseQuestionPane } from '../engines/kit/questionPane.js'+/** Explicit inline compatibility for direct users and recorded-screen fixtures. */+export * from './questionController.js'+export { parseEngineQuestionPane } from '../engines/screens.js'+import { parseEngineQuestionPane } from '../engines/screens.js'+import { AskQuestionController as Controller, QuestionWatcher as Watcher } from './questionController.js'+import type { AskQuestionDeps, QuestionWatcherDeps, PaneView } from './questionController.js'+const readQuestion: AskQuestionDeps['readQuestion'] = (session, capture) => parseEngineQuestionPane(session.engine ?? 'claude', capture)+export class AskQuestionController extends Controller {+ constructor(deps: Omit<AskQuestionDeps, 'readQuestion'> & Partial<Pick<AskQuestionDeps, 'readQuestion'>>) { super({ readQuestion, ...deps }) }+}+export class QuestionWatcher extends Watcher {+ constructor(private readonly inlineDeps: Omit<QuestionWatcherDeps, 'readQuestion'> & Partial<Pick<QuestionWatcherDeps, 'readQuestion'>>) { super({ readQuestion, ...inlineDeps }) }+ override notePrompt(sessionId: string, capture: string | null, view?: PaneView): void {+ const session = this.inlineDeps.getSession(sessionId)+ super.notePrompt(sessionId, capture, view === undefined && capture !== null && session ? parseEngineQuestionPane(session.engine ?? 'claude', capture) : view ?? null)}-- /**- * Key the answers into the pane's dialog, question by question, ending on the review screen.- *- * Nothing is typed until the dialog on screen is shown to be the question the answer was written for:- * its requestId when the answer carries one, else its own text. An answer can arrive late — the agent- * moved on, another client answered, the next question of the form is up — and positionally matching- * it to whatever is showing now is how a person's "Yes" lands on a permission prompt they never saw.- */- private async drive(- terminalTarget: string,- answers: Record<string, string>,- engine: AgentEngine,- allowPermissions: boolean,- asked: { requestId: string; owners: string[] },- reviewed?: QuestionAnswerPayload,- ): Promise<QuestionAnswerResult> {- const wait = this.deps.wait ?? sleep- const used = new Set<string>()- const reviewedComplete = () => !reviewed || used.size === reviewed.expectedQuestions!.length- let lastQuestion = ''- let repeats = 0- let blanks = 0- let answered = 0-- for (let step = 0; step < MAX_STEPS; step++) {- const capture = await this.deps.capture(terminalTarget, CAPTURE_LINES)- if (reviewed && capture === null) return failed('The question could not be read.')- const view = parseEngineQuestionPane(engine, capture ?? '')- if (!view) {- // Nothing on screen: either the dialog was already gone, or the last keystroke submitted it.- return answered > 0 && reviewedComplete() ? { ok: true } : STALE_GONE- }- if (!allowPermissions && view.kind === 'question' && view.permission) return failed('Permission prompts cannot be answered from here.')- if (view.kind === 'question' && view.partial) {- // The dialog's top is out of the pane. If we have keyed an answer it has not been taken yet;- // give the TUI a beat. If we have not, there is nothing to match an answer against.- if (answered === 0) { console.warn('[question] dialog scrolled out of view — cannot key an answer'); return failed('The question is scrolled out of view.') }- if (++repeats >= 2) { console.warn('[question] dialog stuck (scrolled)'); return STUCK }- await wait(STEP_MS)- continue- }- if (view.kind === 'review') {- // Reached by our own keys, this submits the form. Reached first, it means every question was- // answered somewhere else — submitting would send answers this person never gave.- if (answered === 0) return STALE_GONE- if (!reviewedComplete()) return STALE_CHANGED- return await this.deps.sendKey(terminalTarget, view.submitRow) ? { ok: true } : failed('The answers could not be submitted.')- }- // Mid-repaint the question line can read blank for a capture (see parseQuestionPane). Neither its id- // nor its text can be checked against a blank, so look again rather than judge the dialog by it.- if (!view.question) {- if (++blanks > 2) return answered > 0 && reviewedComplete() ? { ok: true } : failed('The question could not be read.')- await wait(STEP_MS)- continue- }- blanks = 0- // The same question still showing after we acted on it: give the TUI one more beat to repaint,- // then treat it as stuck rather than hammering the pane with more keystrokes. Never consume a- // second answer for it.- if (view.question === lastQuestion) {- if (++repeats >= 2) { console.warn(`[question] dialog stuck on "${view.question.slice(0, 60)}"`); return STUCK }- await wait(STEP_MS)- continue- }- repeats = 0- lastQuestion = view.question-- // Is this the question the answer was for? Only the FIRST one needs the id: every later screen is- // one our own keys advanced to, and must be named by its text (below).- let positional = false- if (answered === 0 && asked.requestId) {- if (!asked.owners.some((owner) => questionRequestId(owner, view) === asked.requestId)) {- console.warn(`[question] answer for req=${asked.requestId} arrived after the dialog changed to "${view.question.slice(0, 60)}" — nothing typed`)- return STALE_CHANGED- }- positional = true- }-- // Out of answers with the dialog still up = a multi-QUESTION dialog whose next question the device- // hasn't been shown yet. Leave it open: the watcher pushes that one and the device answers it next.- const expected = reviewed?.expectedQuestions?.find(q => q.q === view.question)- if (reviewed && (!expected || expected.multi !== view.multi ||- JSON.stringify(expected.options) !== JSON.stringify(view.rows.map(row => row.label)))) {- // A request id proves the initial dialog; every reviewed screen must also match its full choices.- return answered > 0 && reviewedComplete() ? { ok: true } : STALE_CHANGED- }- const picked = expected- ? (!used.has(expected.key) && typeof answers[expected.key] === 'string'- ? { key: expected.key, value: answers[expected.key] } : null)- : pickAnswer(answers, view.question, used, { positional })- if (!picked) {- if (answered > 0 && reviewedComplete()) return { ok: true }- console.warn(`[question] no answer names "${view.question.slice(0, 60)}" — nothing typed`)- return STALE_CHANGED- }- used.add(picked.key)- answered++-- if (reviewed?.freeTextKeys?.includes(picked.key)) {- // Spoken words are explicitly text, even when they happen to equal an option label.- // A permission prompt can never acquire consent through this path.- if (!expected?.canText || view.permission || view.multi || !view.typeRow ||- !picked.value.trim() || Buffer.byteLength(picked.value, 'utf8') > 1200 ||- /[\x00-\x09\x0b-\x1f\x7f]/.test(picked.value)) return failed('The text answer cannot be entered into this question.')- if (!await this.typeFreeText(terminalTarget, view.typeRow, picked.value, wait)) return KEYS_FAILED- await wait(STEP_MS)- continue- }-- if (view.multi) {- // Device joins the selected labels with ", " (q_done_tap).- const labels = reviewed ? reviewed.selectedLabels?.[picked.key] ?? []- : picked.value.split(',').map((s) => s.trim()).filter(Boolean)- if (reviewed && (!labels.length || labels.some(label => !view.rows.some(row => row.label === label)))) return failed('That answer matches no option.')- if (reviewed) {- // Set the exact reviewed set, including clearing choices selected in another client.- for (const row of view.rows) if (row.checked !== labels.includes(row.label)) {- if (!await this.deps.sendKey(terminalTarget, row.number)) return KEYS_FAILED- await wait(TEXT_MS)- }- if (!await this.deps.sendKey(terminalTarget, multiSubmitKey(engine))) return KEYS_FAILED- await wait(STEP_MS)- continue- }- let toggled = 0- for (const label of labels) {- const row = matchRow(view.rows, label)- if (!row || row.checked) continue- if (!await this.deps.sendKey(terminalTarget, row.number)) return KEYS_FAILED- await wait(TEXT_MS)- toggled++- }- if (!toggled && view.typeRow- && !await this.typeFreeText(terminalTarget, view.typeRow, picked.value, wait)) return KEYS_FAILED- if (!await this.deps.sendKey(terminalTarget, multiSubmitKey(engine))) return KEYS_FAILED // advance to the next question / review- await wait(STEP_MS)- continue- }-- const row = reviewed ? view.rows.find(row => row.label === picked.value) ?? null : matchRow(view.rows, picked.value)- if (row) {- // One digit selects AND submits — except on Amp, whose rows are unnumbered and reached by- // walking the list, so this is a short sequence rather than a single key.- for (const key of rowKeys(engine, row, view)) {- if (!await this.deps.sendKey(terminalTarget, key)) return KEYS_FAILED- await wait(TEXT_MS)- }- await wait(STEP_MS)- continue- }- if (reviewed) return failed('That answer matches no option.')- if (!view.typeRow) { console.warn(`[question] no option matched "${picked.value.slice(0, 40)}" and no free-text row`); return failed('That answer matches no option.') }- if (!await this.typeFreeText(terminalTarget, view.typeRow, picked.value, wait)) return KEYS_FAILED- await wait(STEP_MS)- }- return STUCK- }-- /** True while a dialog is being keyed — the watcher pauses so a half-driven dialog isn't re-announced. */- isDriving(sessionId: string): boolean {- const session = this.deps.getSession(sessionId)- return this.driving.has(session?.agentId || session?.sessionId || sessionId)- }-- /** Free-text answer (a voice answer is always free text): open the "Type something." row, type, Enter. */- private async typeFreeText(terminalTarget: string, typeRow: QuestionRow, text: string, wait: (ms: number) => Promise<void>): Promise<boolean> {- if (!await this.deps.sendKey(terminalTarget, typeRow.number)) return false- await wait(TEXT_MS)- if (!await this.deps.sendText(terminalTarget, text)) return false- await wait(TEXT_MS)- return this.deps.sendKey(terminalTarget, 'Enter')- }-}--// ── watching a terminal for an open question ─────────────────────────────────────────────────────--export interface QuestionWatcherDeps {- getSession: (sessionId: string) => RegisteredSession | undefined- capture: (terminalTarget: string, historyLines?: number) => Promise<string | null>- /** Skip the capture entirely when no device is listening — nothing would consume the question. */- hasDevice: () => boolean- /** A dialog is open on screen. Fires ONCE per distinct question (until it changes or closes). */- onQuestion: (sessionId: string, requestId: string, questions: ShapedQuestion[], detail?: { permission: boolean; dialog: string }) => void- /**- * An announced dialog LEFT the screen — answered somewhere else, or abandoned.- *- * This is the other half of `onQuestion`, and it was missing: a question answered in the app (or in the- * pane by hand) simply stopped being on screen, the watcher forgot it, and every OTHER client went on- * showing it. On the dial that is a screen you cannot leave without dismissing a question that no longer- * exists. The pane is the only source of truth here — there is no server-side question object — so- * "gone from the pane" is the only signal there is.- */- onQuestionGone?: (sessionId: string, requestId: string) => void-- /** True while that session's dialog is being keyed by an answer already in flight. */- isDriving?: (sessionId: string) => boolean-}--const POLL_MS = 1500-/** How long the pane read before a prompt stands for the turn it starts: past this, that turn's own start- * is read instead (a prompt that never became a turn). Turns were seen to start 8 s after their prompt- * under load. */-const PROMPT_BASELINE_MS = 120_000-// Consecutive empty polls before a question is declared gone.-//-// NOT 1. A capture taken mid-repaint parses as no-dialog, and announcing a close on that would yank a-// live question off the dial's screen — the exact failure this feature exists to prevent, inverted.-// Two ticks costs 1.5s of delay on a real close and makes a flicker unable to cause one.-const GONE_TICKS = 2--// Amp and codex are here for their PERMISSION prompt, not a question tool — neither has one. That prompt-// is drawn only in the pane and recorded nowhere, so polling the pane is the only way it is ever seen.-// Codex needs no parser of its own: it draws numbered rows under a `Press enter to confirm or esc to-// cancel` footer, which is the shared parser's anchor exactly (`__fixtures__/permission-codex.txt`), and-// the question it lands on is the command itself. Membership in this set is what starts the poll, so an-// engine belongs here only once something can actually read its pane.-const QUESTION_ENGINES = new Set<AgentEngine>(['claude', 'commandcode', 'codex', 'cursor', 'devin', 'hermes', 'opencode', 'muse', 'amp', 'kilo', 'grok', 'agy', 'copilot'])--/** Does this engine ever paint a question dialog? Callers use it to decide whether to watch its pane. */-export function pollsQuestions(engine: AgentEngine): boolean {- return QUESTION_ENGINES.has(engine)-}--/**- * The OUT half. The obvious source — the transcript's AskUserQuestion `tool_use` line — is useless here:- * the CLI does not flush that line until the question has been ANSWERED (its JSONL `timestamp` is the- * message's creation time, not its write time), so a device would only ever learn about a question after- * it no longer exists. The dialog itself, on the other hand, is on screen the whole time it is waiting.- *- * So while a turn is open we read the pane. This also means the question survives an adapter restart and- * re-announces to a device that attaches mid-question — neither of which a one-shot event could do.- */-export class QuestionWatcher {- private readonly watching = new Set<string>()- private timer: NodeJS.Timeout | undefined- private readonly pendingPolls = new Map<string, { cancelled: boolean }>()- private readonly pendingBaselines = new Map<string, { cancelled: boolean }>()- private last = new Map<string, string>() // sessionId → fingerprint of the announced question- private lastId = new Map<string, string>() // sessionId → requestId of the announced question- private misses = new Map<string, number>() // sessionId → consecutive polls with no dialog- private readonly blocked = new Map<string, string>()- /** sessionId → the dialog that was ALREADY on the pane when this turn began. */- private readonly preTurn = new Map<string, string>()- /** sessionId → the pane as the daemon read it right before typing a prompt: its dialog, if any, and when. */- private readonly beforePrompt = new Map<string, { fingerprint: string | null; at: number }>()--- constructor(private readonly deps: QuestionWatcherDeps) {}-- /**- * A turn just began — remember any dialog already on the pane, so it is not- * announced as this turn's question.- *- * A dialog belonging to a turn cannot have been on screen before that turn's- * first byte. That invariant is the only way to tell "waiting for an answer"- * from "answered a moment ago and still drawn", because a pane looks- * identical either way.- *- * MEASURED: the daemon attached to a Codex that had just been asked about an- * update and answered in the app. Its first capture, 0.8s into the NEXT turn,- * still held that prompt — so the dial was shown a question nobody was being- * asked, and it stood there for 25s until the engine's output scrolled it out- * of the captured window.- *- * Deliberately NOT applied on attach: re-announcing a genuinely open dialog to- * a client that arrives mid-question is a feature, and there is no turn- * boundary there to reason from.- */- noteTurnStart(sessionId: string): void {- this.preTurn.delete(sessionId)- this.cancelPending(sessionId)- // The daemon typed this turn's prompt, and read the pane as it did: what was on it then is the turns- // before's, and nothing drawn since is (notePrompt says why).- const typed = this.beforePrompt.get(sessionId)- this.beforePrompt.delete(sessionId)- if (typed && Date.now() - typed.at <= PROMPT_BASELINE_MS) {- if (typed.fingerprint) this.preTurn.set(sessionId, typed.fingerprint)- return- }- const pending = { cancelled: false }- this.pendingBaselines.set(sessionId, pending)- void (async () => {- try {- const session = this.deps.getSession(sessionId)- const target = session?.agentId || session?.sessionId- if (!target || session?.active === false || (session?.engine && !pollsQuestions(session.engine))) return- const generation = questionCaptureGeneration(session)- const capture = await this.deps.capture(target, CAPTURE_LINES)- if (pending.cancelled || generation !== questionCaptureGeneration(this.deps.getSession(sessionId))) return- const view = parseEngineQuestionPane(session?.engine ?? 'claude', capture ?? '')- if (!view || view.kind !== 'question' || !view.question || view.rows.length === 0) return- // Only if nothing has been announced for this turn yet: the capture takes- // a moment, and a dialog that opened inside that window is this turn's.- if (this.lastId.has(sessionId)) return- this.preTurn.set(sessionId, fingerprintOf(view))- } catch {- // A failed read supplies no baseline. Keep watching for a fresh dialog.- } finally {- if (this.pendingBaselines.get(sessionId) === pending) this.pendingBaselines.delete(sessionId)- }- })()- }-- /**- * The daemon is typing a prompt, and `capture` is the pane as it read it right before (core/input.ts- * messageWriter): the honest "before this turn" for the turn that prompt starts.- *- * A capture taken when the turn is SEEN to start can be too late for that. The turn is seen to start- * from the transcript, after the prompt is confirmed typed, and an engine can draw its question before- * that: measured in the soak run (e2e/endurance.e2e.ts) under load, Codex took the prompt at 20:42:54.9- * (its prompt hook), drew its question a moment later, and the turn was seen to start at 20:43:02.2, 7 s- * on. The capture at that start held the turn's own question, recorded it as the turns before's, and it- * was never announced: the agent waited on a question no window or device was shown.- *- * A read that failed supplies nothing: the turn's start reads the pane itself, as before.- */- notePrompt(sessionId: string, capture: string | null): void {- const session = this.deps.getSession(sessionId)- if (capture === null || !session || (session.engine && !pollsQuestions(session.engine))) return- const view = parseEngineQuestionPane(session.engine ?? 'claude', capture)- const dialog = view && view.kind === 'question' && view.question && view.rows.length > 0 ? fingerprintOf(view) : null- this.beforePrompt.set(sessionId, { fingerprint: dialog, at: Date.now() })- }-- /** Poll this session's pane while its turn is open (called on turn_started). */- start(sessionId: string): void {- if (this.watching.has(sessionId)) return- const session = this.deps.getSession(sessionId)- // Only the engines that actually paint a question dialog: Claude and Command Code share one shape,- // devin has its own (parseDevinQuestionPane). Polling any other pane would be pure waste.- if (!session || session.active === false || !(session.agentId || session.sessionId) || !QUESTION_ENGINES.has(session.engine)) return- this.watching.add(sessionId)- // One clock also starts the reads in one event-loop turn, allowing the tmux- // backend to batch them without delaying polls or caching terminal content.- this.timer ??= setInterval(() => {- for (const id of this.watching) void this.tick(id)- }, POLL_MS)- }-- /**- * Stop polling — and CLOSE any question still outstanding on this session.- *- * ⚠️ THIS IS THE COMMON CASE, NOT THE EDGE ONE, and leaving it out made the whole close mechanism look- * like it did not work. `stop()` is called on turn_ended, and answering the question is precisely what- * lets the turn end — so the dialog leaving the pane and the watcher being torn down happen within a- * second or two of each other, far inside the two-tick confirmation. Measured on hardware: the answer- * landed at ~16:50:19 and the turn ended at 16:50:29 with no close ever announced.- *- * No confirmation is needed here and none is wanted: the turn is over, so whatever we announced is- * definitively not waiting for anybody any more. The same holds for the other callers — an agent that- * was removed cannot answer either.- */- stop(sessionId: string): void {- this.cancelPending(sessionId)- this.preTurn.delete(sessionId)- this.blocked.delete(sessionId)- this.watching.delete(sessionId)- if (!this.watching.size && this.timer) { clearInterval(this.timer); this.timer = undefined }- const requestId = this.lastId.get(sessionId)- this.forget(sessionId)- if (requestId) this.deps.onQuestionGone?.(sessionId, requestId)- }--- stopAll(): void {- if (this.timer) clearInterval(this.timer)- this.timer = undefined- this.watching.clear()- for (const pending of this.pendingPolls.values()) pending.cancelled = true- for (const pending of this.pendingBaselines.values()) pending.cancelled = true- this.preTurn.clear()- this.beforePrompt.clear()- this.blocked.clear()- this.last.clear()- this.lastId.clear()- this.misses.clear()- }-- /** A device (re)joined: forget what we announced so an open question is pushed again. */- reset(): void {- this.last.clear()- this.lastId.clear()- this.misses.clear()- }-- private forget(sessionId: string): void {- this.last.delete(sessionId)- this.lastId.delete(sessionId)- this.misses.delete(sessionId)- }-- private cancelPending(sessionId: string): void {- const poll = this.pendingPolls.get(sessionId)- if (poll) poll.cancelled = true- const baseline = this.pendingBaselines.get(sessionId)- if (baseline) baseline.cancelled = true- }-- /**- * A poll found no dialog. Announce the close once the miss is CONFIRMED, and only if we announced the- * question in the first place — a session nobody was told about has nothing to take back.- */- private noteGone(sessionId: string): void {- this.last.delete(sessionId) // unchanged: a dialog that comes back announces fresh- const requestId = this.lastId.get(sessionId)- if (!requestId) return- const misses = (this.misses.get(sessionId) ?? 0) + 1- this.misses.set(sessionId, misses)- if (misses < GONE_TICKS) return- this.lastId.delete(sessionId)- this.misses.delete(sessionId)- this.deps.onQuestionGone?.(sessionId, requestId)- }--- private async tick(sessionId: string): Promise<void> {- // A capture can take longer than POLL_MS. Keep one outstanding poll per- // session, including a cancelled capture that has not returned yet.- if (this.pendingPolls.has(sessionId)) return- const session = this.deps.getSession(sessionId)- const terminalTarget = session?.agentId || session?.sessionId- if (!terminalTarget || session?.active === false || (session?.engine && !pollsQuestions(session.engine))) { this.stop(sessionId); return }- // Both of these silently do nothing, which is how a live dialog can sit on the terminal with no trace- // in the log. Say it once per transition rather than every 1.5s tick.- const blocked = !this.deps.hasDevice() ? 'no device' : this.deps.isDriving?.(sessionId) ? 'driving an answer' : ''- if (blocked !== (this.blocked.get(sessionId) ?? '')) {- this.blocked.set(sessionId, blocked)- console.log(`[question] ${sessionId.slice(0, 8)} watcher ${blocked ? `paused · ${blocked}` : 'polling'}`)- }- if (blocked) return- const pending = { cancelled: false }- this.pendingPolls.set(sessionId, pending)- const generation = questionCaptureGeneration(session)- let capture: string | null- try {- capture = await this.deps.capture(terminalTarget, CAPTURE_LINES)- } catch {- capture = null- } finally {- if (this.pendingPolls.get(sessionId) === pending) this.pendingPolls.delete(sessionId)- }- if (pending.cancelled || generation !== questionCaptureGeneration(this.deps.getSession(sessionId))- || !this.deps.hasDevice() || this.deps.isDriving?.(sessionId)) return- if (capture === null) {- // Unavailable is not an empty pane, and breaks a run of confirmed misses.- this.misses.delete(sessionId)- return- }- const view = parseEngineQuestionPane(session?.engine ?? 'claude', capture)- if (view?.kind === 'question' && view.partial) {- // Scrolled so its top is out of the pane: still open, so the client showing it keeps showing it —- // but there is no question text to announce, and the rows in view are whichever the scroll left.- this.misses.delete(sessionId)- return- }- if (!view || view.kind !== 'question' || !view.question || view.rows.length === 0) {- // Dialog closed, or moved to review. Either way it is no longer waiting on anybody, so the clients- // showing it are told to stop — see noteGone for why this is not announced on the first miss.- this.noteGone(sessionId)- return- }- this.misses.delete(sessionId) // a dialog on screen ends any run of misses- const fingerprint = fingerprintOf(view)- // Already on the pane before this turn started → it belongs to whatever came- // before, and has been answered. Say nothing until it changes or leaves.- if (this.preTurn.get(sessionId) === fingerprint) return- this.preTurn.delete(sessionId)- if (this.last.get(sessionId) === fingerprint) return- this.last.set(sessionId, fingerprint)-- // A pane-derived question has no tool_use id. The key only has to round-trip through the device and- // back (the answer is keyed into the pane, not matched to a tool call), so the question's own text- // serves as both — and the device dedups a repeated push by this id. The answer brings the id back,- // and AskQuestionController recomputes it off the live pane before typing: a different id there- // means a different dialog, and the answer is refused (STALE_QUESTION) instead of keyed into it.- const requestId = questionRequestId(sessionId, view)- this.lastId.set(sessionId, requestId)- const baseline = this.pendingBaselines.get(sessionId)- if (baseline) baseline.cancelled = true- this.deps.onQuestion(sessionId, requestId, [{- key: view.question,- q: view.question,- options: view.rows.map((r) => r.label),- multi: view.multi,- ...(view.typeRow && !view.multi && !view.permission ? { canText: true } : {}),- }], { permission: isApprovalDialog(view), dialog: view.dialog ?? view.question })-- }-}--/** Snapshot scalar values before awaiting I/O: registry records can change in- * place when an engine exits, moves, or is replaced in the same pane. */-function questionCaptureGeneration(session: RegisteredSession | undefined): string {- return session ? JSON.stringify([session.agentId, session.sessionId, session.engine, session.active,- session.tmuxPane, session.primaryRuntimeKey, session.runtimes, session.processIdentity]) : ''-}--/**- * What makes two captures the SAME dialog: its words, its options, its arity — and, when the parser kept- * the whole dialog, what it says below its first line (a command that differs only on its second line is- * another prompt).- *- * ⚠️ NOT the raw `dialog`. The id is recomputed every 1.5s poll and again at the moment an answer is typed,- * so it may only change when the QUESTION does. The raw dialog changes on its own: Hermes and Muse paint a- * live timer inside it (`(01m30s · ↓ 82 tok)`, `(21s · esc to interrupt)`), and every engine moves its- * `❯`/`›`/`>` cursor and ticks its `[✔]` boxes in place. Hashed raw, a question was re-announced as new on- * every poll — the needs-you alert, the sound, the dial push, again and again — and every answer from a- * dial, a device or the cable was refused as STALE_QUESTION. `dialogSignature` is the dialog with all of- * that taken out; the raw text still goes, unchanged, to the pair's floor (isApprovalDialog, pair/sensor).- */-function fingerprintOf(view: QuestionView): string {- const base = `${view.question}|${view.rows.map((r) => r.label).join('|')}|${view.multi}`- return view.dialog === undefined ? base : `${base}|${dialogSignature(view.dialog)}`-}--// A live status group: an elapsed time (`21s`, `30.5s`, `01m30s`, `1h02m`), a token counter (`↓ 82 tok`,-// `1.2k tokens`) or `esc to interrupt` inside one pair of parentheses. Units hug their digits, as every-// engine paints them, so `(see 2 files)` or `(tokens.json)` is never mistaken for one.-const TIMER_GROUP = String.raw`\([^()\n]*?(?:\b\d+(?:\.\d+)?(?:ms|s|m|h)\b|\b\d+m\d+s\b|\b\d+h\d+m\b|\d+(?:\.\d+)?k?\s*tok(?:en)?s?\b|esc to interrupt)[^()\n]*\)`-const TIMER_GROUP_RE = new RegExp(TIMER_GROUP, 'i')-const TIMER_GROUPS_RE = new RegExp(TIMER_GROUP, 'gi')-// The same, outside parentheses: `↓ 82 tokens · esc to interrupt`.-const STATUS_BITS_RE = /[↑↓]\s*\d+(?:\.\d+)?k?\s*tok(?:en)?s?\b|\besc to interrupt\b/i-// A bare elapsed time, with no parentheses and no ` · ` to mark it: a status line that is only a word or-// three and a duration — a verb in -ing/-ed (`waiting 3s`, `thinking 4s`, `Churned for 4s`) or anything-// trailing off in an ellipsis (`Waiting… 12s`, `Fetch Bitcoin price… 1m33s`). A command's own number stays:-// `sleep 30s` and `retry after 30s` are neither.-const DURATION = String.raw`(?:\d+h\d+m(?:\d+s)?|\d+m\d+s|\d+(?:\.\d+)?m?s)`-const WORDS = String.raw`(?:\p{L}[\p{L}'’-]*\s+){0,2}\p{L}[\p{L}'’-]*`-const BARE_TIMER_LINE_RE = new RegExp(String.raw`^${WORDS}(?:(?<=ing|ed)(?:\s+for)?\s+|\s*(?:…|\.{3})\s*)${DURATION}$`, 'iu')-// …and one hung off the end of a longer line, after an ellipsis or a column gap: grok's right-aligned-// `Waiting on answers for Which color should I report? 4.2s`. Only the time goes.-const TRAILING_TIMER_RE = new RegExp(String.raw`(?:(?<=…|\.{3})\s*|\s{2,})${DURATION}$`, 'u')-// Codex's cursor readout under its request_user_input rows: `option 2/4 | tab to add notes`.-const CURSOR_READOUT_RE = /^option\s+\d+\s*\/\s*\d+\b/i-// Whatever leads a line and moves on its own: a cursor (`❯ › > ▶`), a spinner frame (braille, `✻`, `◐`,-// Muse's `◇`/`◆`) or a box/tab state (`☐ ☒ ✔ ○ ● ◉`).-const LEAD_MARKS_RE = /^(?:[❯›>▶►▸➤\u2800-\u28ff✻✽✶✳✢✺✹✸✷◐◓◑◒◴◵◶◷◇◆☐☑☒✔✓✗✘○◯●◉◎]\s*)+/u-// A row's own state right after its number, or at the start of an unnumbered row: `[ ]`, `[✔]`, `(•)`, `◉`.-const ROW_STATE_RE = /^(\d+[.)]\s+|)(?:\[[^\]\n]?\]|\([^)\n]?\)|[☐☑☒✔✓○◯●◉◎])\s*/u-// Box drawing: frames and rules redraw to the pane's width.-const BOX_RE = /[\u2500-\u257f]+/g--/**- * The dialog as a person reads it, with nothing that changes while it waits: status lines (a live timer,- * bare or in parentheses, a token counter, `esc to interrupt`) and Codex's cursor readout dropped; cursor marks, spinner frames- * and checkbox/radio state stripped; frames and whitespace collapsed. Every word of the prompt stays —- * two commands that differ anywhere are still two signatures.- */-function dialogSignature(dialog: string): string {- const out: string[] = []- for (const raw of dialog.replace(/\u00a0/g, ' ').split('\n')) {- let line = raw.replace(BOX_RE, ' ').trim()- if (CURSOR_READOUT_RE.test(line)) continue- line = line.replace(LEAD_MARKS_RE, '')- const row = /^\d+[.)]\s/.test(line)- // A line that carries a live timer is the engine's status line (Hermes' `💻 curl … (01m30s · ↓ 82 tok)`- // under its frame, Muse's `◇ Calling tools (21s · esc to interrupt)` above its rule, a bare `waiting 3s`),- // not the prompt: the prompt is always painted on lines of its own. A ROW keeps its words; only the- // group goes.- if (!row && (TIMER_GROUP_RE.test(line) || STATUS_BITS_RE.test(line) || BARE_TIMER_LINE_RE.test(line))) continue- if (!row) line = line.replace(TRAILING_TIMER_RE, '')- line = line.replace(TIMER_GROUPS_RE, ' ').replace(ROW_STATE_RE, '$1').replace(/\s+/g, ' ').trim()- if (line) out.push(line)- }- return out.join('\n')-}--function hash(value: string): string {- let h = 0x811c9dc5- for (let i = 0; i < value.length; i++) h = Math.imul(h ^ value.charCodeAt(i), 0x01000193) >>> 0- return h.toString(16).padStart(8, '0')}diff --git a/cli/src/lib/closeAgentService.spec.ts b/cli/src/lib/closeAgentService.spec.tsindex 73323db94..5439a5527 100644--- a/cli/src/lib/closeAgentService.spec.ts+++ b/cli/src/lib/closeAgentService.spec.ts@@ -1,5 +1,6 @@+import { screenFor } from '../engines/screens.js'import { afterEach, beforeEach, expect, it, vi } from 'vitest'-import { CloseAgentService, inspectCloseActivity, type CloseActivity, type CloseAgentServiceDeps, type AgentCloseRequest } from './closeAgentService.js'+import { CloseAgentService, inspectCloseActivity as inspectCloseReading, type CloseActivity, type CloseAgentServiceDeps, type AgentCloseRequest } from './closeAgentService.js'import { registry, type RegisteredSession } from './registry.js'import { mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs'import { join } from 'node:path'@@ -373,3 +374,7 @@ it('a concurrent deferred tick never starts a second stop', async () => {expect(deps.stop).toHaveBeenCalledOnce()finish(); await closing})++function inspectCloseActivity(session: Parameters<typeof inspectCloseReading>[0], capture: string | null, turnOpen: boolean | undefined, needsInput: boolean) {+ return inspectCloseReading(session, capture === null ? null : screenFor(session.engine).inspect(capture), turnOpen, needsInput)+}diff --git a/cli/src/lib/closeAgentService.ts b/cli/src/lib/closeAgentService.tsindex d899f6043..3a7e70282 100644--- a/cli/src/lib/closeAgentService.ts+++ b/cli/src/lib/closeAgentService.ts@@ -1,8 +1,6 @@/** User Close is a disk-backed lifecycle operation. Hiding/switching a tab never calls this. */-import { stripVTControlCharacters } from 'node:util'import { randomUUID } from 'node:crypto'-import { teamWriteHold } from './teamWriteHold.js'-import { terminalActivity } from './terminalActivity.js'+import type { ScreenReading } from '../engines/facets/screen.js'import type { RegisteredSession, registry as liveRegistry } from './registry.js'import { terminalRouteKey } from './terminalRuntime.js'import type { StopAgentOptions } from './stopAgentService.js'@@ -30,21 +28,14 @@ function unusedChat(session: CloseSession): boolean {/** An empty new chat has no turn state yet. Existing chats still require that state,* and neither kind is idle without a recognized empty composer. */-export function inspectCloseActivity(session: CloseSession, screen: string | null,+export function inspectCloseActivity(session: CloseSession, screen: ScreenReading | null,turnOpen: boolean | undefined, needsInput: boolean): CloseActivity {const { engine } = sessionif (needsInput) return 'needs_input'if (turnOpen === true) return 'working'if (!screen) return 'unknown'- const footer = stripVTControlCharacters(screen).split('\n').slice(-16).join('\n')- // Codex can be between turns of an active goal, and Claude can have background tasks. Codex has- // said an active goal two ways: `◎ /goal active (41m)` in its older footers, and `Pursuing goal (41m)`- // on its status line since (0.160, tui/src/bottom_pane/footer.rs), where only the older wording was- // known and an agent between the turns of its goal read as idle, for a close to take. Its other goal- // states (paused, stalled, unmet, abandoned, achieved) are not work in progress.- if (/\bgoal\s+active\b|\bpursuing goal\b|\b[1-9]\d*\s+background\s+(?:tasks?|agents?)\b/i.test(footer)- || terminalActivity(engine, screen)) return 'working'- const hold = teamWriteHold(engine, screen)+ if (screen.busy) return 'working'+ const hold = screen.teamHoldif (hold === 'team_waiting_draft') return 'draft'if (hold === 'team_waiting_user') return 'needs_input'if (hold || (turnOpen === undefined && !unusedChat(session))) return 'unknown'diff --git a/cli/src/lib/codexTurnRecovery.ts b/cli/src/lib/codexTurnRecovery.tsindex 89d208fae..01aeb94fa 100644--- a/cli/src/lib/codexTurnRecovery.ts+++ b/cli/src/lib/codexTurnRecovery.ts@@ -1,20 +1,3 @@-/** Some Codex goal interruptions never write task_complete/turn_aborted to the- * rollout. An open transcript is then only historical evidence of work. The- * live footer is one positive idle reading, never a synthetic turn end. */-import { stripVTControlCharacters } from 'node:util'-import { inspectRuntimePane } from './runtimeProfileController.js'--/** Read only the UI below the current empty composer, never output/history.- * An empty composer alone says nothing: Codex accepts drafts while working. */-export function codexStoppedGoal(screen: string | null): boolean {- if (!screen || !inspectRuntimePane('codex', screen).idle) return false- const lines = stripVTControlCharacters(screen).split(/\r?\n/)- const prompt = lines.findLastIndex(line => /^\s*›(?:\s|$)/u.test(line))- if (prompt < 0) return false- // A live turn can still display the previous goal state during a redraw.- if (lines.slice(-16).some(line => /\besc to interrupt\b/i.test(line))) return false- const footer = lines.slice(prompt + 1).filter(line => line.trim())- return footer.length <= 4- && footer.some(line => /\bGoal (?:stalled|paused) \(\/goal resume\)\s*$/.test(line))- && footer.some(line => /\? for shortcuts\b/.test(line))-}+import { inspectRuntimePane } from '../engines/codex/screen.js'+import { stoppedGoal } from '../engines/codex/stoppedGoal.js'+export function codexStoppedGoal(screen: string | null): boolean { return !!screen && stoppedGoal(screen, inspectRuntimePane(screen)) }diff --git a/cli/src/lib/composerScreen.ts b/cli/src/lib/composerScreen.tsindex d86903728..59c02f550 100644--- a/cli/src/lib/composerScreen.ts+++ b/cli/src/lib/composerScreen.ts@@ -1,120 +1,7 @@-/**- * Whether Claude Code's or Codex's own composer is on screen, ready for a message: what a message is- * typed into, and nothing else (messageHold.ts). An allowlist, so a screen never seen before is not- * typed into: it is refused, where a list of screens to refuse would have let it through.- *- * Read from the engines' own code (Claude Code 2.1.289's bundle, Codex 0.160's tui/src), not run:- * - Claude Code draws its prompt in a box ruled above and below (`borderStyle: "round"` with no- * sides, the rule full width, text in the top rule at times), `❯` and the prompt in it, its further- * lines under the first; under the box, its footer, or the suggestions for a `/command` or an- * `@mention` being typed, which take the footer's place (`Kge`, padded to six rows or more).- * Every dialog, menu and view hides the box.- * - Codex draws a blank row, then `›` in bold (`»` at its top reasoning effort; dim while input is- * disabled, `!` in shell mode) and the draft or its dim placeholder at the left edge, its further- * lines under it, then a blank row and its footer, one or two rows, at the bottom of the pane- * (chat_composer.rs, snapshots `empty`, `draft_composer`, `status_and_queued_messages`). Its popups- * are drawn above the composer: the `/command` list (snapshot `slash_popup_footer_wide`) and the- * mention menu with its `enter/tab insert · esc close` row (`default_unified_mention_popup`). Its- * approvals, questions, pickers and startup screens replace it; a picker's rows are numbered.- * A popup open over the composer is no composer to type into: its Enter picks the highlighted row.- */-import { stripVTControlCharacters } from 'node:util'-import type { RegisteredSession } from './registry.js'--export type ComposerState = 'ready' | 'popup' | 'absent'--/** The engines read this way. Every other engine is read only for the screens known to refuse. */+/** Inline compatibility facade. Supervised core reads the engine screen worker. */+import type { AgentEngine } from '../engines/types.js'+import { composerState as claude } from '../engines/claude/composer.js'+import { composerState as codex } from '../engines/codex/composer.js'+export type { ComposerState } from '../engines/facets/screen.js'export const COMPOSER_ENGINES: ReadonlySet<string> = new Set(['claude', 'codex'])--/** A row of Claude Code's rule: all `─`, or `─` at both ends with text set in it. */-const CLAUDE_RULE = /^─{3,}$|^─{3,}.*─{3,}$/--/** Codex's composer row: `›` (or `»`) at the left edge, never a numbered row of a picker. */-const CODEX_COMPOSER = /^[›»](?:\s|$)/-const CODEX_PICKER_ROW = /^[›»]\s*\d+\.\s/--function screenLines(capture: string): { lines: string[]; raw: string[] } {- const raw = capture.split('\n')- const lines = raw.map((line) => stripVTControlCharacters(line).replace(/ /g, ' ').trimEnd())- while (lines.length && !lines.at(-1)) { lines.pop(); raw.pop() }- return { lines, raw }-}--/** Whether the first character of a row is drawn bold: the SGR in force when it is written. */-function startsBold(row: string): boolean {- let bold = false- const sgr = /^\u001b\[([0-9;:]*)m/- let rest = row- for (let match = sgr.exec(rest); match; match = sgr.exec(rest)) {- for (const code of match[1].split(/[;:]/)) {- if (code === '1') bold = true- else if (code === '' || code === '0' || code === '22') bold = false- }- rest = rest.slice(match[0].length)- }- return bold-}--/** The word being typed at the end of a draft, where a `/command` or an `@mention` opens its popup. */-function typedToken(draft: string): string {- return draft.split(/\s/).at(-1) ?? ''-}--export function composerState(engine: RegisteredSession['engine'], capture: string): ComposerState {- const { lines, raw } = screenLines(capture)- return engine === 'codex' ? codexComposer(lines, raw) : claudeComposer(lines)-}--function claudeComposer(lines: string[]): ComposerState {- let prompt = lines.findLastIndex((line) => /^❯(?:\s|$)/.test(line))- let bottom = lines.findIndex((line, index) => index > prompt && CLAUDE_RULE.test(line.trim()))- if (prompt < 1 || !CLAUDE_RULE.test(lines[prompt - 1].trim()) || bottom < 0) {- // A draft taller than the pane: Claude Code does not cap the prompt's height outside its fullscreen- // renderer (2.1.289 passes `maxVisibleLines` only there), so the box's top rule and its `❯` row- // scroll off, and the pane shows the draft's further rows, indented under the `❯`, down to the- // bottom rule. Every row above that rule is one of them; a dialog's are indented by one column, the- // conversation's start at the edge.- bottom = lines.findIndex((line) => CLAUDE_RULE.test(line.trim()))- if (bottom < 1 || !lines.slice(0, bottom).every((line) => !line || line.startsWith(' '))) return 'absent'- prompt = -1- }- const below = lines.slice(bottom + 1)- // Nothing ruled under the box: a dialog of Claude Code's own is drawn ruled, in its place.- if (below.some((line) => CLAUDE_RULE.test(line.trim()))) return 'absent'- const draft = [prompt < 0 ? '' : lines[prompt].replace(/^❯\s?/, ''), ...lines.slice(prompt + 1, bottom).map((line) => line.trim())].join(' ').trim()- const token = typedToken(draft)- const rows = below.map((line) => line.trim()).filter(Boolean)- if (token.startsWith('/') && rows.some((row) => /^\/[\w:.-]/.test(row))) return 'popup'- if (token.startsWith('@')) {- const typed = token.slice(1).toLowerCase()- // A file or agent suggestion, by what is typed after the `@`; with nothing typed yet, any row but the- // footer's own (its mode and its hints).- if (rows.some((row) => typed ? row.replace(/^@/, '').toLowerCase().startsWith(typed) : !/^(?:\?|⏵|esc\b)/.test(row))) return 'popup'- }- return 'ready'-}--function codexComposer(lines: string[], raw: string[]): ComposerState {- const composer = lines.findLastIndex((line) => CODEX_COMPOSER.test(line) && !CODEX_PICKER_ROW.test(line))- // Bold, on a row of its own under a blank one: not a line of the conversation, nor a disabled composer.- if (composer < 0 || !startsBold(raw[composer]) || (composer > 0 && lines[composer - 1].trim())) return 'absent'- // The draft's further lines, then a blank row and the footer: nothing more is drawn under Codex's- // composer, so more is some other screen under a line of the conversation.- let end = composer + 1- while (end < lines.length && lines[end].trim()) end++- const footer = lines.slice(end).filter((line) => line.trim())- if (footer.length > 3) return 'absent'- const draft = [lines[composer].replace(/^[›»]\s?/, ''), ...lines.slice(composer + 1, end).map((line) => line.trim())].join(' ').trim()- const token = typedToken(draft)- // A voice send was refused as popup_open even though the recording had transcribed. Looking through- // twenty conversation rows also matched command examples and old menu hints. A popup is the block- // immediately above the composer; a slash menu has a selected row and edits the initial /command.- let popupEnd = composer - 1- while (popupEnd >= 0 && !lines[popupEnd].trim()) popupEnd--- let popupStart = popupEnd- while (popupStart >= Math.max(0, composer - 20) && lines[popupStart].trim()) popupStart--- const above = lines.slice(popupStart + 1, popupEnd + 1)- if (draft.startsWith('/') && token.startsWith('/') && above.some((line) => /^›\s+\/[\w-]+(?:\s{2,}\S|$)/.test(line))) return 'popup'- if (token.startsWith('@') && above.some((line) => /\benter\/tab insert · esc close\b/.test(line))) return 'popup'- return 'ready'-}+export function composerState(engine: AgentEngine, capture: string) { return engine === 'codex' ? codex(capture) : claude(capture) }diff --git a/cli/src/lib/legacyPane.ts b/cli/src/lib/legacyPane.tsnew file mode 100644index 000000000..cf76154aa--- /dev/null+++ b/cli/src/lib/legacyPane.ts@@ -0,0 +1,36 @@+import type { AgentEngine } from '../engines/types.js'+import type { PaneInspection } from '../engines/facets/screen.js'+import { inspectPane, stripAnsi } from '../engines/kit/pane.js'+import { parsePiFooterProfile } from '../engines/pi/runtimeProfile.js'+const PI_FOOTER_BUDGET = /\d+(?:\.\d+)?%\s*\/\s*\S+/+export function inspectRuntimePane(engine: AgentEngine, capture: string): PaneInspection { return engine === 'pi' ? inspectPiPane(capture) : inspectPane(engine, capture) }+function inspectPiPane(capture: string): PaneInspection {+ const lines = stripAnsi(capture).split('\n').map((line) => line.replace(/\s+$/, ''))+ const text = lines.join('\n')+ // Whether the footer is DRAWN, which is not the same question as whether its profile can be READ —+ // and conflating the two is what has broken this check twice.+ //+ // `parsePiFooterProfile` is strict on purpose: it must return a model AND an effort or nothing, so a+ // half-drawn redraw is never mistaken for a profile change. Idleness needs far less — only that the+ // normal view is on screen rather than a picker — and borrowing the strict reader for it meant every+ // footer shape it could not fully parse read as "still busy".+ //+ // The shape it could not parse: a model with no thinking ladder draws no `• <level>` at all, so the+ // line ends at the model name. EVERY grid model is that shape, because the provider block written in+ // `gridLaunch.ts` declares `reasoning: false` — which left every Pi agent on a grid permanently+ // AGENT_BUSY, refusing model switches over an idle pane (measured, pi 0.82, footer+ // `↑5.4k ↓292 1.5%/200k (auto) Auto`).+ //+ // So: the profile still counts when it parses, and the token-budget readout — `0.0%/500k`,+ // `1.5%/200k`, present in both shapes and drawn by no picker — carries the rest.+ const footer = parsePiFooterProfile(text) !== null || PI_FOOTER_BUDGET.test(text)+ const rules: number[] = []+ lines.forEach((line, index) => { if (/^\s*─{8,}\s*$/.test(line)) rules.push(index) })+ const dialog = /Thinking Level|Select reasoning depth|Type to search|Enter to select|Only showing models from/i+ .test(lines.join('\n'))+ if (rules.length < 2) return { idle: false, plan: false, dialog, draft: false }+ const band = lines.slice(rules[rules.length - 2] + 1, rules[rules.length - 1]).join('').trim()+ const draft = band.length > 0+ return { idle: footer && !dialog && !draft, plan: false, dialog, draft }+}+diff --git a/cli/src/lib/legacyScreen.ts b/cli/src/lib/legacyScreen.tsnew file mode 100644index 000000000..6b8472eb3--- /dev/null+++ b/cli/src/lib/legacyScreen.ts@@ -0,0 +1,9 @@+import type { AgentEngine } from '../engines/types.js'+import { inspectRuntimePane as inspectPane } from './legacyPane.js'+import { assembleScreen } from '../engines/kit/screen.js'+import { parseEngineQuestionPane } from './questionPane.js'+export function legacyScreen(engine: AgentEngine, capture: string | null) {+ if (capture === null) return { pane: { idle: false, plan: false, dialog: false, draft: false }, question: null, messageHold: null,+ teamHold: 'team_waiting_unavailable', activity: null, busy: false, stoppedGoal: false }+ return assembleScreen(engine, capture, inspectPane(engine, capture), parseEngineQuestionPane(engine, capture))+}diff --git a/cli/src/lib/messageHold.ts b/cli/src/lib/messageHold.tsindex 13806bb1b..783e66705 100644--- a/cli/src/lib/messageHold.ts+++ b/cli/src/lib/messageHold.ts@@ -1,134 +1,3 @@-/**- * Whether a message can be typed into an agent's pane as it stands, and if not, why not and what the- * person can do about it.- *- * A message is a bracketed paste followed by Enter. Typed into the composer, that sends it. Typed into- * a dialog, it does something else: the dialogs below drop the paste, and the Enter confirms what is- * highlighted. Read from the engines' own code (Claude Code 2.1.289's bundle, Codex 0.160's tui/src):- * - a permission prompt approves its highlighted row, the first: Yes, run it (Claude Code's Select has- * no paste handler; Codex's ApprovalOverlay leaves `handle_paste` to its default, and its own test- * `enter_sets_last_selected_index_without_dismissing` expects Accept);- * - a question picks its highlighted option (Claude Code), or takes the paste as notes on it and- * submits that (Codex's request_user_input);- * - a menu (model, effort and the other pickers) picks its highlighted row;- * - a picker for a point to rewind to rewinds, or loses the message (Codex's transcript browser,- * Claude Code's Rewind menu);- * - a transcript view loses it (Claude Code's, ctrl+o; Codex's overlay, ctrl+t, in its scrollback- * mode);- * - a search takes it as what to search for, and Claude Code's prompt-history search then SENDS the- * earlier prompt it found (ctrl+r);- * - the questions asked at startup take the Enter as their highlighted answer: trust the folder- * (Claude Code's highlights `No, exit`, and quits), run Codex's update, switch Codex's model, start- * a sign-in.- * So none of them is typed into. A question is answered through its own path (`question_response`),- * never by a message.- *- * For Claude Code and Codex the screens above are named, to say what is open; anything else is typed- * into only when the engine's own composer is on screen (composerScreen.ts), so a screen never seen- * before is refused, never typed into. Other engines are read for their questions and permission- * prompts only.- *- * One function for every route a message takes to a pane, so that none can skip it (core/input.ts).- */-import { engineLabel } from './agentNames.js'-import { isApprovalDialog, parseEngineQuestionPane } from './askQuestion.js'-import type { RegisteredSession } from './registry.js'-import { COMPOSER_ENGINES, composerState } from './composerScreen.js'-import { paneModal } from './runtimeProfileController.js'--/** Why a message was not typed, as the reason its delivery is refused with. */-export type MessageHold = 'permission_open' | 'question_open' | 'menu_open' | 'rewind_picker_open' | 'transcript_open'- | 'search_open' | 'trust_open' | 'update_prompt_open' | 'model_prompt_open' | 'sign_in_open' | 'popup_open'- | 'prompt_hidden' | 'screen_unreadable'--const HOLDS: ReadonlySet<string> = new Set<MessageHold>(['permission_open', 'question_open', 'menu_open', 'rewind_picker_open',- 'transcript_open', 'search_open', 'trust_open', 'update_prompt_open', 'model_prompt_open', 'sign_in_open', 'popup_open',- 'prompt_hidden', 'screen_unreadable'])--/**- * The holds that can be passing: an engine still starting draws nothing yet, a redraw can be caught- * between frames, a capture can fail once. A message waits a little for these to clear before it is- * refused (sessionInput.ts), as it does for the pane's control lease.- */-const PASSING: ReadonlySet<string> = new Set<MessageHold>(['prompt_hidden', 'screen_unreadable'])-export function passingHold(reason: string): boolean {- return PASSING.has(reason)-}--/** The hold for each of the engines' own screens and modals (runtimeProfileController.ts `paneModal`). */-const MODAL_HOLDS: Record<NonNullable<ReturnType<typeof paneModal>>, MessageHold> = {- rewind: 'rewind_picker_open', transcript: 'transcript_open', search: 'search_open', trust: 'trust_open',- update: 'update_prompt_open', model: 'model_prompt_open', sign_in: 'sign_in_open', permission: 'permission_open', menu: 'menu_open',-}--/** What the pane shows that a message must not be typed into, or null when it can be typed. */-export function messageHold(engine: RegisteredSession['engine'], capture: string | null): MessageHold | null {- const allowlisted = COMPOSER_ENGINES.has(engine)- if (capture === null) return allowlisted ? 'screen_unreadable' : null- const view = parseEngineQuestionPane(engine, capture)- if (view) return view.kind === 'question' && isApprovalDialog(view) ? 'permission_open' : 'question_open'- if (!allowlisted) return null- const modal = paneModal(engine, capture)- if (modal) return MODAL_HOLDS[modal]- const composer = composerState(engine, capture)- return composer === 'ready' ? null : composer === 'popup' ? 'popup_open' : 'prompt_hidden'-}--export function isMessageHold(reason: string): reason is MessageHold {- return HOLDS.has(reason)-}--function engineName(engine: string): string {- return engine === 'claude' ? 'Claude Code' : engineLabel(engine)-}--/**- * What the person is told when their message was typed but its Enter not pressed: the engine opened- * something between the two, which the Enter would have answered. The text waits in the composer.- */-export function messageWithheldText(engine: string, hold: string): string {- const name = engineName(engine)- const [what, then] = hold === 'permission_open' ? ['asked for permission', 'Answer the request in the app or in its terminal']- : hold === 'question_open' ? ['asked you a question', 'Answer the question in the app or in its terminal']- : ['opened another screen', 'Finish what\'s on its screen in its terminal']- return `${name} ${what} just as your message was typed, so it was not sent; it waits in ${name}'s prompt. ${then}, then press Enter in its terminal to send the message, or clear it there.`-}--/** What the person is told: why their message was not typed, and what lets it through. */-export function messageHoldText(engine: string, hold: MessageHold): string {- const name = engineName(engine)- switch (hold) {- case 'permission_open':- return `${name} is asking for ${engine === 'codex' ? 'approval' : 'permission'}. Answer it first, in the app or in its terminal, then send the message again.`- case 'question_open':- return `${name} is asking you a question. Answer it first, in the app or in its terminal, then send the message again.`- case 'menu_open':- return `${name} has a menu open, where Enter would pick from it. Close it with Esc in its terminal, then send the message again.`- case 'rewind_picker_open':- return engine === 'codex'- ? 'Codex is browsing its transcript, where Enter would rewind the conversation. Close it with Esc in its terminal, then send the message again.'- : `${name} has its Rewind menu open, where Enter would pick a point to rewind to. Close it with Esc in its terminal, then send the message again.`- case 'transcript_open':- return engine === 'codex'- ? 'Codex is showing its transcript (ctrl+t), where a message is not typed. Close it with q in its terminal, then send the message again.'- : `${name} is showing its transcript (ctrl+o), where a message is not typed. Close it with Esc in its terminal, then send the message again.`- case 'search_open':- return engine === 'codex'- ? 'Codex has a search open, where a message would become what it searches for. Close it with Esc in its terminal, then send the message again.'- : `${name} is searching its prompt history (ctrl+r), where Enter would send an earlier prompt. Close it with ctrl+c in its terminal, then send the message again.`- case 'trust_open':- return `${name} is asking whether to trust this folder. Answer it in its terminal, then send the message again.`- case 'update_prompt_open':- return `${name} is asking whether to update. Answer it in its terminal, then send the message again.`- case 'model_prompt_open':- return `${name} is asking whether to switch to a new model. Answer it in its terminal, then send the message again.`- case 'sign_in_open':- return `${name} is asking how to sign in. Sign in in its terminal, then send the message again.`- case 'popup_open':- return `${name} message not sent. Close suggestions with Esc in its terminal, then retry.`- case 'prompt_hidden':- return `${name} isn't showing its prompt; finish what's on its screen in its terminal, then send the message again.`- case 'screen_unreadable':- return `${name}'s screen could not be read, so the message was not typed. Send it again in a moment.`- }-}+/** Inline compatibility API. Core imports the value-only messageHolds module. */+export * from './messageHolds.js'+export { messageHold } from '../engines/screens.js'diff --git a/cli/src/lib/messageHolds.ts b/cli/src/lib/messageHolds.tsnew file mode 100644index 000000000..c40169aa9--- /dev/null+++ b/cli/src/lib/messageHolds.ts@@ -0,0 +1,76 @@+import { engineLabel } from './agentNames.js'+import type { MessageHold } from '../engines/facets/screen.js'+export type { MessageHold } from '../engines/facets/screen.js'++const HOLDS: ReadonlySet<string> = new Set<MessageHold>(['permission_open', 'question_open', 'menu_open', 'rewind_picker_open',+ 'transcript_open', 'search_open', 'trust_open', 'update_prompt_open', 'model_prompt_open', 'sign_in_open', 'popup_open',+ 'prompt_hidden', 'screen_unreadable'])++/**+ * The holds that can be passing: an engine still starting draws nothing yet, a redraw can be caught+ * between frames, a capture can fail once. A message waits a little for these to clear before it is+ * refused (sessionInput.ts), as it does for the pane's control lease.+ */+const PASSING: ReadonlySet<string> = new Set<MessageHold>(['prompt_hidden', 'screen_unreadable'])+export function passingHold(reason: string): boolean {+ return PASSING.has(reason)+}++export function isMessageHold(reason: string): reason is MessageHold {+ return HOLDS.has(reason)+}++function engineName(engine: string): string {+ return engine === 'claude' ? 'Claude Code' : engineLabel(engine)+}++/**+ * What the person is told when their message was typed but its Enter not pressed: the engine opened+ * something between the two, which the Enter would have answered. The text waits in the composer.+ */+export function messageWithheldText(engine: string, hold: string): string {+ const name = engineName(engine)+ const [what, then] = hold === 'permission_open' ? ['asked for permission', 'Answer the request in the app or in its terminal']+ : hold === 'question_open' ? ['asked you a question', 'Answer the question in the app or in its terminal']+ : ['opened another screen', 'Finish what\'s on its screen in its terminal']+ return `${name} ${what} just as your message was typed, so it was not sent; it waits in ${name}'s prompt. ${then}, then press Enter in its terminal to send the message, or clear it there.`+}++/** What the person is told: why their message was not typed, and what lets it through. */+export function messageHoldText(engine: string, hold: MessageHold): string {+ const name = engineName(engine)+ switch (hold) {+ case 'permission_open':+ return `${name} is asking for ${engine === 'codex' ? 'approval' : 'permission'}. Answer it first, in the app or in its terminal, then send the message again.`+ case 'question_open':+ return `${name} is asking you a question. Answer it first, in the app or in its terminal, then send the message again.`+ case 'menu_open':+ return `${name} has a menu open, where Enter would pick from it. Close it with Esc in its terminal, then send the message again.`+ case 'rewind_picker_open':+ return engine === 'codex'+ ? 'Codex is browsing its transcript, where Enter would rewind the conversation. Close it with Esc in its terminal, then send the message again.'+ : `${name} has its Rewind menu open, where Enter would pick a point to rewind to. Close it with Esc in its terminal, then send the message again.`+ case 'transcript_open':+ return engine === 'codex'+ ? 'Codex is showing its transcript (ctrl+t), where a message is not typed. Close it with q in its terminal, then send the message again.'+ : `${name} is showing its transcript (ctrl+o), where a message is not typed. Close it with Esc in its terminal, then send the message again.`+ case 'search_open':+ return engine === 'codex'+ ? 'Codex has a search open, where a message would become what it searches for. Close it with Esc in its terminal, then send the message again.'+ : `${name} is searching its prompt history (ctrl+r), where Enter would send an earlier prompt. Close it with ctrl+c in its terminal, then send the message again.`+ case 'trust_open':+ return `${name} is asking whether to trust this folder. Answer it in its terminal, then send the message again.`+ case 'update_prompt_open':+ return `${name} is asking whether to update. Answer it in its terminal, then send the message again.`+ case 'model_prompt_open':+ return `${name} is asking whether to switch to a new model. Answer it in its terminal, then send the message again.`+ case 'sign_in_open':+ return `${name} is asking how to sign in. Sign in in its terminal, then send the message again.`+ case 'popup_open':+ return `${name} message not sent. Close suggestions with Esc in its terminal, then retry.`+ case 'prompt_hidden':+ return `${name} isn't showing its prompt; finish what's on its screen in its terminal, then send the message again.`+ case 'screen_unreadable':+ return `${name}'s screen could not be read, so the message was not typed. Send it again in a moment.`+ }+}diff --git a/cli/src/lib/questionController.spec.ts b/cli/src/lib/questionController.spec.tsnew file mode 100644index 000000000..fbdf365de--- /dev/null+++ b/cli/src/lib/questionController.spec.ts@@ -0,0 +1,63 @@+import { readFileSync } from 'node:fs'+import { afterEach, describe, expect, it, vi } from 'vitest'+import { screenFor } from '../engines/screens.js'+import type { PaneView } from '../engines/facets/screen.js'+import type { RegisteredSession } from './registry.js'+import { AskQuestionController, QuestionWatcher } from './questionController.js'++const capture = readFileSync(new URL('./__fixtures__/permission-claude.txt', import.meta.url), 'utf8')+const view = screenFor('claude').inspect(capture).question+const row = () => ({ agentId: 'agent', sessionId: 'session', engine: 'claude', active: true }) as RegisteredSession+const deferred = <T>() => { let resolve!: (value: T) => void; const promise = new Promise<T>(r => { resolve = r }); return { promise, resolve } }+afterEach(() => { vi.useRealTimers(); vi.restoreAllMocks() })++describe('asynchronous question evidence', () => {+ it('keeps a previously announced question while the screen reader is unavailable', async () => {+ vi.useFakeTimers()+ const current = row(), onQuestion = vi.fn(), onQuestionGone = vi.fn()+ const readQuestion = vi.fn(async (): Promise<PaneView> => view)+ const watcher = new QuestionWatcher({ getSession: () => current, capture: async () => capture,+ hasDevice: () => true, readQuestion, onQuestion, onQuestionGone })+ watcher.start(current.sessionId)+ await vi.advanceTimersByTimeAsync(1_500)+ expect(onQuestion).toHaveBeenCalledOnce()+ readQuestion.mockRejectedValue(new Error('worker unavailable'))+ await vi.advanceTimersByTimeAsync(6_000)+ expect(onQuestionGone).not.toHaveBeenCalled()+ readQuestion.mockResolvedValue(null)+ await vi.advanceTimersByTimeAsync(3_000)+ expect(onQuestionGone).toHaveBeenCalledOnce()+ watcher.stopAll()+ })+ it.each(['replace', 'stop'] as const)('holds one pending interpretation and discards it after %s', async change => {+ vi.useFakeTimers()+ const current = row(), pending = deferred<PaneView>(), onQuestion = vi.fn()+ const readQuestion = vi.fn(() => pending.promise)+ const watcher = new QuestionWatcher({ getSession: () => current, capture: async () => capture,+ hasDevice: () => true, readQuestion, onQuestion })+ watcher.start(current.sessionId)+ await vi.advanceTimersByTimeAsync(6_000)+ expect(readQuestion).toHaveBeenCalledOnce()+ if (change === 'stop') watcher.stop(current.sessionId)+ else current.sessionId = 'replacement'+ pending.resolve(view)+ await vi.advanceTimersByTimeAsync(0)+ expect(onQuestion).not.toHaveBeenCalled()+ watcher.stopAll()+ })+ it.each(['unavailable', 'rebound', 'capture-rebound'] as const)('types no answer after %s and releases its input lease', async change => {+ const current = row(), release = vi.fn(), sendKey = vi.fn(async () => true), sendText = vi.fn(async () => true)+ const controller = new AskQuestionController({ getSession: () => current,+ capture: async () => { if (change === 'capture-rebound') current.sessionId = 'replacement'; return capture },+ sendKey, sendText, acquireControl: () => release,+ readQuestion: async () => {+ if (change === 'unavailable') throw new Error('worker failed')+ current.sessionId = 'replacement'+ return view+ } })+ const result = await controller.answer({ agentId: 'agent', answers: { 'question': 'No' } })+ expect(result).toMatchObject({ ok: false, error: change === 'unavailable' ? 'ANSWER_FAILED' : 'STALE_QUESTION' })+ expect(sendKey).not.toHaveBeenCalled(); expect(sendText).not.toHaveBeenCalled()+ expect(release).toHaveBeenCalledOnce()+ })+})diff --git a/cli/src/lib/questionController.ts b/cli/src/lib/questionController.tsnew file mode 100644index 000000000..a2cc09772--- /dev/null+++ b/cli/src/lib/questionController.ts@@ -0,0 +1,805 @@+import type { RegisteredSession } from './registry.js'+import type { AgentEngine } from '../engines/types.js'+import type { ShapedQuestion, QuestionRow, QuestionView, ReviewView, PaneView } from '../engines/facets/screen.js'+export type { ShapedQuestion, QuestionRow, QuestionView, ReviewView, PaneView, FoundDialog } from '../engines/facets/screen.js'+import { ampSelectionKeys } from '../engines/amp/askQuestion.js'+import { kiloSelectionKeys } from '../engines/kilo/askQuestion.js'+import { isApprovalDialog } from '../engines/kit/questionPane.js'+function multiSubmitKey(engine: AgentEngine): string { return engine === 'devin' ? 'Enter' : 'Tab' }+const STEP_MS = 350 // let the TUI repaint between keystrokes+const TEXT_MS = 250+const MAX_STEPS = 14 // hard bound on the drive loop (questions × keys), never spin on a stuck pane+const CAPTURE_LINES = 60++function norm(value: string): string {+ return value.toLowerCase().replace(/\s+/g, ' ').replace(/[.…]+$/, '').trim()+}++function sleep(ms: number): Promise<void> {+ return new Promise((resolve) => setTimeout(resolve, ms))+}++/**+ * Shape a raw AskUserQuestion input into the device form. Port of the hosted runtime’s `commanderQuestions`+ * (websocket.ts): `key` MUST stay the value the CLI matches answers by — prompt/question first — because+ * the device echoes it back as the answers-map key.+ */+export function shapeQuestions(questions: unknown): ShapedQuestion[] {+ return (Array.isArray(questions) ? questions : []).map((raw, i) => {+ const q = (raw ?? {}) as Record<string, unknown>+ const opts = Array.isArray(q.options) ? q.options : []+ return {+ key: (q.prompt as string) || (q.question as string) || (q.id as string) || (q.header as string) || `question_${i}`,+ q: (q.prompt as string) || (q.question as string) || '',+ options: opts+ .map((o) => String((typeof o === 'string' ? o : (o as Record<string, unknown>)?.label) ?? ''))+ .filter(Boolean),+ // Three spellings because three CLIs: `allow_multiple` (the hosted runtime), `multiSelect` (claude),+ // `multi_select` (devin — read from a real `ask_user_question` call in its SQLite store).+ multi: (q.allow_multiple as boolean) ?? (q.multiSelect as boolean) ?? (q.multi_select as boolean) ?? false,+ }+ })+}++function rowKeys(engine: AgentEngine, row: QuestionRow, view?: QuestionView): string[] {+ if (engine === 'amp') return ampSelectionKeys(row)+ if (engine === 'codex') return codexRowKeys(row, view)+ // Kilo's rows sit side by side, so its walk is horizontal — see engines/kilo/askQuestion.ts.+ if (engine === 'kilo') return kiloSelectionKeys(row)+ // Same dialog, different engine: opencode numbers its ask dialog but not its permission prompt, so the+ // ROW says how it is reached and a per-engine rule would break one of the two.+ if (row.walk === 'right') return kiloSelectionKeys(row)+ if (row.walk === 'down') return ampSelectionKeys(row)+ return [row.number]+}++/**+ * Codex, measured 2026-09-15 on 0.149: in its request_user_input dialog a digit only MOVES the highlight+ * and Enter submits (`enter to submit answer`) — a digit alone left the dialog up and the device's answer+ * reported as stuck. Its approval prompt is the other way round and was verified earlier: one digit+ * selects and commits, so that one keeps the single key.+ */+export function codexRowKeys(row: QuestionRow, view?: QuestionView): string[] {+ return view?.enterSubmits ? [row.number, 'Enter'] : [row.number]+}++export function matchRow(rows: QuestionRow[], answer: string): QuestionRow | null {+ const a = norm(answer)+ if (!a) return null+ return rows.find((r) => norm(r.label) === a)+ ?? (a.length >= 3 ? rows.find((r) => norm(r.label).startsWith(a)) ?? rows.find((r) => a.startsWith(norm(r.label)) && norm(r.label).length >= 3) : undefined)+ ?? null+}++/**+ * Pick the answer for the question the dialog is currently showing: the entry keyed by its own text.+ *+ * `positional` also takes the next unused entry when none names it. Only for a dialog the answer's+ * requestId proves it was written for: without that proof, an answer that names no question on screen+ * belongs to one that is gone, and typing it here would answer — or approve — something nobody saw.+ *+ * The text must be the question's OWN (case, spacing and a trailing `…` aside), never a prefix either way.+ * An approval is titled `Approve <header>: <argument>` and the header is shared by every prompt of its+ * kind: a key left from an earlier prompt — `Approve Bash command` (its argument unread) — was a prefix of+ * `Approve Bash command: rm -rf ~/projects` and pressed Yes on it, and an old full title named a+ * header-only prompt the other way round. A client echoes back the key it was announced, so the whole+ * text is always there to match; one it has cut is answered through its requestId (`positional`).+ */+export function pickAnswer(+ answers: Record<string, string>,+ question: string,+ used: Set<string>,+ opts: { positional?: boolean } = {},+): { key: string; value: string } | null {+ const entries = Object.entries(answers)+ const q = norm(question)+ const byText = q ? entries.find(([k]) => norm(k) === q) : undefined+ if (byText && !used.has(byText[0])) return { key: byText[0], value: byText[1] }+ if (!opts.positional) return null+ const next = entries.find(([k]) => !used.has(k))+ return next ? { key: next[0], value: next[1] } : null+}++/**+ * The id a dialog is announced under — the SAME function the watcher names it with, so the answer's+ * requestId can be checked against the dialog on screen at the moment of typing rather than against+ * whatever the watcher last saw (it polls every 1.5s, and forgets on a reset).+ */+export function questionRequestId(sessionId: string, view: QuestionView): string {+ return `q_${hash(sessionId + fingerprintOf(view))}`+}++/** Why an answer was not keyed. Sent back to the client as `question_response_result.error`. */+export type QuestionAnswerError = 'STALE_QUESTION' | 'AGENT_NOT_FOUND' | 'ANSWER_BUSY' | 'ANSWER_FAILED'++export type QuestionAnswerResult = { ok: true } | { ok: false; error: QuestionAnswerError; detail: string }++const STALE_CHANGED: QuestionAnswerResult = { ok: false, error: 'STALE_QUESTION', detail: 'That question changed before your answer arrived.' }+const STALE_GONE: QuestionAnswerResult = { ok: false, error: 'STALE_QUESTION', detail: 'That question is no longer open.' }+const failed = (detail: string): QuestionAnswerResult => ({ ok: false, error: 'ANSWER_FAILED', detail })+const KEYS_FAILED = failed('The answer could not be typed into the agent\'s terminal.')+const STUCK = failed('The question did not take the answer.')++export interface AskQuestionDeps {+ readQuestion(session: RegisteredSession, capture: string): PaneView | Promise<PaneView>+ getSession: (sessionId: string) => RegisteredSession | undefined+ capture: (terminalTarget: string, historyLines?: number) => Promise<string | null>+ sendText: (terminalTarget: string, text: string) => Promise<boolean>+ sendKey: (terminalTarget: string, key: string) => Promise<boolean>+ /** Pins one backend locator for the whole multi-step dialog drive. */+ acquireControl?: (sessionId: string, opts?: { forAnswer?: boolean }) => (() => void) | null+ /** Injected for tests. */+ wait?: (ms: number) => Promise<void>+}++export interface QuestionAnswerPayload {+ /** Exact contents reviewed on the device; guarded submissions never use positional fallback. */+ expectedQuestions?: ShapedQuestion[]+ selectedLabels?: Record<string, string[]>+ freeTextKeys?: string[]+ allowPermissions?: boolean+ requestId?: string+ sessionId?: string+ agentId?: string+ answers?: Record<string, string>+}++/**+ * Owns the OUT side's pending map (requestId → session) and the IN side's pane driving. One answer at a+ * time per agent: a second `question_response` for a dialog already being driven is dropped, not queued.+ */+export class AskQuestionController {+ private pending = new Map<string, string>() // requestId → sessionId+ private driving = new Set<string>() // sessionIds currently keying a dialog++ constructor(private readonly deps: AskQuestionDeps) {}++ /** Remember which session a mirrored question belongs to (the device may answer minutes later). */+ remember(requestId: string, sessionId: string): void {+ if (!requestId) return+ if (this.pending.size > 64) this.pending.delete(this.pending.keys().next().value as string)+ this.pending.set(requestId, sessionId)+ }++ async answer(payload: QuestionAnswerPayload): Promise<QuestionAnswerResult> {+ const requestId = payload.requestId ?? ''+ if (payload.expectedQuestions !== undefined && (!Array.isArray(payload.expectedQuestions) ||+ payload.expectedQuestions.length < 1 || payload.expectedQuestions.length > 4 ||+ payload.expectedQuestions.some(q => !q || typeof q.key !== 'string' || !q.key ||+ typeof q.q !== 'string' || !q.q || !Array.isArray(q.options) || !q.options.length ||+ q.options.some(option => typeof option !== 'string' || !option) ||+ typeof q.multi !== 'boolean' || (q.multi && !Array.isArray(payload.selectedLabels?.[q.key]))))) {+ return failed('The reviewed question metadata is invalid.')+ }+ if (payload.freeTextKeys !== undefined && (!payload.expectedQuestions || !Array.isArray(payload.freeTextKeys) ||+ payload.freeTextKeys.some(key => typeof key !== 'string' || !payload.expectedQuestions!.some(q =>+ q.key === key && q.canText === true && !q.multi)))) {+ return failed('The reviewed text answer metadata is invalid.')+ }+ const remembered = this.pending.get(requestId)+ const sessionId = payload.sessionId || payload.agentId || remembered || ''+ const answers = payload.answers && typeof payload.answers === 'object' ? payload.answers : null+ if (!sessionId || !answers || Object.keys(answers).length === 0) {+ console.warn(`[question] ignoring answer with no session/answers (req=${requestId})`)+ return failed('The answer named no agent or carried no choice.')+ }+ const session = this.deps.getSession(sessionId)+ const terminalTarget = session?.agentId || session?.sessionId+ if (!terminalTarget) {+ console.warn(`[question] no terminal target for ${sessionId.slice(0, 8)} — answer dropped`)+ return { ok: false, error: 'AGENT_NOT_FOUND', detail: 'That harness is no longer running.' }+ }+ if (remembered) {+ const owner = this.deps.getSession(remembered)+ if ((owner?.agentId || owner?.sessionId) !== terminalTarget) return STALE_CHANGED+ }+ if (this.driving.has(terminalTarget)) {+ console.warn(`[question] ${sessionId.slice(0, 8)} answer dropped · already driving this dialog`)+ return { ok: false, error: 'ANSWER_BUSY', detail: 'Another answer is already being entered for this harness.' }+ }+ // `forAnswer`: a dialog is the engine waiting for input mid-turn, so the open turn must not block it.+ const release = this.deps.acquireControl?.(terminalTarget, { forAnswer: true })+ // Silence here is the failure mode this whole file exists to prevent: the device sends an answer,+ // nothing keys it in, and the pane sits on the dialog looking like a hung agent.+ if (this.deps.acquireControl && !release) {+ console.warn(`[question] ${sessionId.slice(0, 8)} answer dropped · terminal control unavailable`)+ return { ok: false, error: 'ANSWER_BUSY', detail: 'The agent\'s terminal is busy. Try again.' }+ }+ // The ids the watcher could have announced this dialog under: the session it was remembered for, and+ // the session as the registry knows it now.+ const owners = [...new Set([remembered, session?.sessionId].filter((id): id is string => !!id))]+ this.driving.add(terminalTarget)+ try {+ const result = await this.drive(terminalTarget, answers, this.deps.getSession(sessionId)?.engine ?? 'claude', payload.allowPermissions !== false, { requestId, owners }, payload.expectedQuestions ? payload : undefined)+ this.pending.delete(requestId)+ const outcome = result.ok ? 'submitted' : result.error === 'STALE_QUESTION' ? 'refused · STALE_QUESTION, nothing typed' : 'FAILED'+ console.log(`[question] ${sessionId.slice(0, 8)} answered from device · ${outcome} (req=${requestId || 'none'})`)+ return result+ } finally {+ this.driving.delete(terminalTarget)+ release?.()+ }+ }++ /**+ * Key the answers into the pane's dialog, question by question, ending on the review screen.+ *+ * Nothing is typed until the dialog on screen is shown to be the question the answer was written for:+ * its requestId when the answer carries one, else its own text. An answer can arrive late — the agent+ * moved on, another client answered, the next question of the form is up — and positionally matching+ * it to whatever is showing now is how a person's "Yes" lands on a permission prompt they never saw.+ */+ private async drive(+ terminalTarget: string,+ answers: Record<string, string>,+ engine: AgentEngine,+ allowPermissions: boolean,+ asked: { requestId: string; owners: string[] },+ reviewed?: QuestionAnswerPayload,+ ): Promise<QuestionAnswerResult> {+ const wait = this.deps.wait ?? sleep+ const used = new Set<string>()+ const reviewedComplete = () => !reviewed || used.size === reviewed.expectedQuestions!.length+ let lastQuestion = ''+ let repeats = 0+ let blanks = 0+ let answered = 0++ const generation = questionCaptureGeneration(this.deps.getSession(terminalTarget))+ for (let step = 0; step < MAX_STEPS; step++) {+ const current = this.deps.getSession(terminalTarget)+ if (!current || generation !== questionCaptureGeneration(current)) return STALE_CHANGED+ const capture = await this.deps.capture(terminalTarget, CAPTURE_LINES)+ if (reviewed && capture === null) return failed('The question could not be read.')+ if (generation !== questionCaptureGeneration(this.deps.getSession(terminalTarget))) return STALE_CHANGED+ let view: PaneView+ try { view = await this.deps.readQuestion(current, capture ?? '') }+ catch { return failed('The question could not be read.') }+ if (generation !== questionCaptureGeneration(this.deps.getSession(terminalTarget))) return STALE_CHANGED+ if (!view) {+ // Nothing on screen: either the dialog was already gone, or the last keystroke submitted it.+ return answered > 0 && reviewedComplete() ? { ok: true } : STALE_GONE+ }+ if (!allowPermissions && view.kind === 'question' && view.permission) return failed('Permission prompts cannot be answered from here.')+ if (view.kind === 'question' && view.partial) {+ // The dialog's top is out of the pane. If we have keyed an answer it has not been taken yet;+ // give the TUI a beat. If we have not, there is nothing to match an answer against.+ if (answered === 0) { console.warn('[question] dialog scrolled out of view — cannot key an answer'); return failed('The question is scrolled out of view.') }+ if (++repeats >= 2) { console.warn('[question] dialog stuck (scrolled)'); return STUCK }+ await wait(STEP_MS)+ continue+ }+ if (view.kind === 'review') {+ // Reached by our own keys, this submits the form. Reached first, it means every question was+ // answered somewhere else — submitting would send answers this person never gave.+ if (answered === 0) return STALE_GONE+ if (!reviewedComplete()) return STALE_CHANGED+ return await this.deps.sendKey(terminalTarget, view.submitRow) ? { ok: true } : failed('The answers could not be submitted.')+ }+ // Mid-repaint the question line can read blank for a capture (see parseQuestionPane). Neither its id+ // nor its text can be checked against a blank, so look again rather than judge the dialog by it.+ if (!view.question) {+ if (++blanks > 2) return answered > 0 && reviewedComplete() ? { ok: true } : failed('The question could not be read.')+ await wait(STEP_MS)+ continue+ }+ blanks = 0+ // The same question still showing after we acted on it: give the TUI one more beat to repaint,+ // then treat it as stuck rather than hammering the pane with more keystrokes. Never consume a+ // second answer for it.+ if (view.question === lastQuestion) {+ if (++repeats >= 2) { console.warn(`[question] dialog stuck on "${view.question.slice(0, 60)}"`); return STUCK }+ await wait(STEP_MS)+ continue+ }+ repeats = 0+ lastQuestion = view.question++ // Is this the question the answer was for? Only the FIRST one needs the id: every later screen is+ // one our own keys advanced to, and must be named by its text (below).+ let positional = false+ if (answered === 0 && asked.requestId) {+ if (!asked.owners.some((owner) => questionRequestId(owner, view) === asked.requestId)) {+ console.warn(`[question] answer for req=${asked.requestId} arrived after the dialog changed to "${view.question.slice(0, 60)}" — nothing typed`)+ return STALE_CHANGED+ }+ positional = true+ }++ // Out of answers with the dialog still up = a multi-QUESTION dialog whose next question the device+ // hasn't been shown yet. Leave it open: the watcher pushes that one and the device answers it next.+ const expected = reviewed?.expectedQuestions?.find(q => q.q === view.question)+ if (reviewed && (!expected || expected.multi !== view.multi ||+ JSON.stringify(expected.options) !== JSON.stringify(view.rows.map(row => row.label)))) {+ // A request id proves the initial dialog; every reviewed screen must also match its full choices.+ return answered > 0 && reviewedComplete() ? { ok: true } : STALE_CHANGED+ }+ const picked = expected+ ? (!used.has(expected.key) && typeof answers[expected.key] === 'string'+ ? { key: expected.key, value: answers[expected.key] } : null)+ : pickAnswer(answers, view.question, used, { positional })+ if (!picked) {+ if (answered > 0 && reviewedComplete()) return { ok: true }+ console.warn(`[question] no answer names "${view.question.slice(0, 60)}" — nothing typed`)+ return STALE_CHANGED+ }+ used.add(picked.key)+ answered++++ if (reviewed?.freeTextKeys?.includes(picked.key)) {+ // Spoken words are explicitly text, even when they happen to equal an option label.+ // A permission prompt can never acquire consent through this path.+ if (!expected?.canText || view.permission || view.multi || !view.typeRow ||+ !picked.value.trim() || Buffer.byteLength(picked.value, 'utf8') > 1200 ||+ /[\x00-\x09\x0b-\x1f\x7f]/.test(picked.value)) return failed('The text answer cannot be entered into this question.')+ if (!await this.typeFreeText(terminalTarget, view.typeRow, picked.value, wait)) return KEYS_FAILED+ await wait(STEP_MS)+ continue+ }++ if (view.multi) {+ // Device joins the selected labels with ", " (q_done_tap).+ const labels = reviewed ? reviewed.selectedLabels?.[picked.key] ?? []+ : picked.value.split(',').map((s) => s.trim()).filter(Boolean)+ if (reviewed && (!labels.length || labels.some(label => !view.rows.some(row => row.label === label)))) return failed('That answer matches no option.')+ if (reviewed) {+ // Set the exact reviewed set, including clearing choices selected in another client.+ for (const row of view.rows) if (row.checked !== labels.includes(row.label)) {+ if (!await this.deps.sendKey(terminalTarget, row.number)) return KEYS_FAILED+ await wait(TEXT_MS)+ }+ if (!await this.deps.sendKey(terminalTarget, multiSubmitKey(engine))) return KEYS_FAILED+ await wait(STEP_MS)+ continue+ }+ let toggled = 0+ for (const label of labels) {+ const row = matchRow(view.rows, label)+ if (!row || row.checked) continue+ if (!await this.deps.sendKey(terminalTarget, row.number)) return KEYS_FAILED+ await wait(TEXT_MS)+ toggled+++ }+ if (!toggled && view.typeRow+ && !await this.typeFreeText(terminalTarget, view.typeRow, picked.value, wait)) return KEYS_FAILED+ if (!await this.deps.sendKey(terminalTarget, multiSubmitKey(engine))) return KEYS_FAILED // advance to the next question / review+ await wait(STEP_MS)+ continue+ }++ const row = reviewed ? view.rows.find(row => row.label === picked.value) ?? null : matchRow(view.rows, picked.value)+ if (row) {+ // One digit selects AND submits — except on Amp, whose rows are unnumbered and reached by+ // walking the list, so this is a short sequence rather than a single key.+ for (const key of rowKeys(engine, row, view)) {+ if (!await this.deps.sendKey(terminalTarget, key)) return KEYS_FAILED+ await wait(TEXT_MS)+ }+ await wait(STEP_MS)+ continue+ }+ if (reviewed) return failed('That answer matches no option.')+ if (!view.typeRow) { console.warn(`[question] no option matched "${picked.value.slice(0, 40)}" and no free-text row`); return failed('That answer matches no option.') }+ if (!await this.typeFreeText(terminalTarget, view.typeRow, picked.value, wait)) return KEYS_FAILED+ await wait(STEP_MS)+ }+ return STUCK+ }++ /** True while a dialog is being keyed — the watcher pauses so a half-driven dialog isn't re-announced. */+ isDriving(sessionId: string): boolean {+ const session = this.deps.getSession(sessionId)+ return this.driving.has(session?.agentId || session?.sessionId || sessionId)+ }++ /** Free-text answer (a voice answer is always free text): open the "Type something." row, type, Enter. */+ private async typeFreeText(terminalTarget: string, typeRow: QuestionRow, text: string, wait: (ms: number) => Promise<void>): Promise<boolean> {+ if (!await this.deps.sendKey(terminalTarget, typeRow.number)) return false+ await wait(TEXT_MS)+ if (!await this.deps.sendText(terminalTarget, text)) return false+ await wait(TEXT_MS)+ return this.deps.sendKey(terminalTarget, 'Enter')+ }+}++// ── watching a terminal for an open question ─────────────────────────────────────────────────────++export interface QuestionWatcherDeps {+ readQuestion(session: RegisteredSession, capture: string): PaneView | Promise<PaneView>+ getSession: (sessionId: string) => RegisteredSession | undefined+ capture: (terminalTarget: string, historyLines?: number) => Promise<string | null>+ /** Skip the capture entirely when no device is listening — nothing would consume the question. */+ hasDevice: () => boolean+ /** A dialog is open on screen. Fires ONCE per distinct question (until it changes or closes). */+ onQuestion: (sessionId: string, requestId: string, questions: ShapedQuestion[], detail?: { permission: boolean; dialog: string }) => void+ /**+ * An announced dialog LEFT the screen — answered somewhere else, or abandoned.+ *+ * This is the other half of `onQuestion`, and it was missing: a question answered in the app (or in the+ * pane by hand) simply stopped being on screen, the watcher forgot it, and every OTHER client went on+ * showing it. On the dial that is a screen you cannot leave without dismissing a question that no longer+ * exists. The pane is the only source of truth here — there is no server-side question object — so+ * "gone from the pane" is the only signal there is.+ */+ onQuestionGone?: (sessionId: string, requestId: string) => void++ /** True while that session's dialog is being keyed by an answer already in flight. */+ isDriving?: (sessionId: string) => boolean+}++const POLL_MS = 1500+/** How long the pane read before a prompt stands for the turn it starts: past this, that turn's own start+ * is read instead (a prompt that never became a turn). Turns were seen to start 8 s after their prompt+ * under load. */+const PROMPT_BASELINE_MS = 120_000+// Consecutive empty polls before a question is declared gone.+//+// NOT 1. A capture taken mid-repaint parses as no-dialog, and announcing a close on that would yank a+// live question off the dial's screen — the exact failure this feature exists to prevent, inverted.+// Two ticks costs 1.5s of delay on a real close and makes a flicker unable to cause one.+const GONE_TICKS = 2++// Amp and codex are here for their PERMISSION prompt, not a question tool — neither has one. That prompt+// is drawn only in the pane and recorded nowhere, so polling the pane is the only way it is ever seen.+// Codex needs no parser of its own: it draws numbered rows under a `Press enter to confirm or esc to+// cancel` footer, which is the shared parser's anchor exactly (`__fixtures__/permission-codex.txt`), and+// the question it lands on is the command itself. Membership in this set is what starts the poll, so an+// engine belongs here only once something can actually read its pane.+const QUESTION_ENGINES = new Set<AgentEngine>(['claude', 'commandcode', 'codex', 'cursor', 'devin', 'hermes', 'opencode', 'muse', 'amp', 'kilo', 'grok', 'agy', 'copilot'])++/** Does this engine ever paint a question dialog? Callers use it to decide whether to watch its pane. */+export function pollsQuestions(engine: AgentEngine): boolean {+ return QUESTION_ENGINES.has(engine)+}++/**+ * The OUT half. The obvious source — the transcript's AskUserQuestion `tool_use` line — is useless here:+ * the CLI does not flush that line until the question has been ANSWERED (its JSONL `timestamp` is the+ * message's creation time, not its write time), so a device would only ever learn about a question after+ * it no longer exists. The dialog itself, on the other hand, is on screen the whole time it is waiting.+ *+ * So while a turn is open we read the pane. This also means the question survives an adapter restart and+ * re-announces to a device that attaches mid-question — neither of which a one-shot event could do.+ */+export class QuestionWatcher {+ private readonly watching = new Set<string>()+ private timer: NodeJS.Timeout | undefined+ private readonly pendingPolls = new Map<string, { cancelled: boolean }>()+ private readonly pendingBaselines = new Map<string, { cancelled: boolean }>()+ private last = new Map<string, string>() // sessionId → fingerprint of the announced question+ private lastId = new Map<string, string>() // sessionId → requestId of the announced question+ private misses = new Map<string, number>() // sessionId → consecutive polls with no dialog+ private readonly blocked = new Map<string, string>()+ /** sessionId → the dialog that was ALREADY on the pane when this turn began. */+ private readonly preTurn = new Map<string, string>()+ /** sessionId → the pane as the daemon read it right before typing a prompt: its dialog, if any, and when. */+ private readonly beforePrompt = new Map<string, { fingerprint: string | null; at: number }>()+++ constructor(private readonly deps: QuestionWatcherDeps) {}++ /**+ * A turn just began — remember any dialog already on the pane, so it is not+ * announced as this turn's question.+ *+ * A dialog belonging to a turn cannot have been on screen before that turn's+ * first byte. That invariant is the only way to tell "waiting for an answer"+ * from "answered a moment ago and still drawn", because a pane looks+ * identical either way.+ *+ * MEASURED: the daemon attached to a Codex that had just been asked about an+ * update and answered in the app. Its first capture, 0.8s into the NEXT turn,+ * still held that prompt — so the dial was shown a question nobody was being+ * asked, and it stood there for 25s until the engine's output scrolled it out+ * of the captured window.+ *+ * Deliberately NOT applied on attach: re-announcing a genuinely open dialog to+ * a client that arrives mid-question is a feature, and there is no turn+ * boundary there to reason from.+ */+ noteTurnStart(sessionId: string): void {+ this.preTurn.delete(sessionId)+ this.cancelPending(sessionId)+ // The daemon typed this turn's prompt, and read the pane as it did: what was on it then is the turns+ // before's, and nothing drawn since is (notePrompt says why).+ const typed = this.beforePrompt.get(sessionId)+ this.beforePrompt.delete(sessionId)+ if (typed && Date.now() - typed.at <= PROMPT_BASELINE_MS) {+ if (typed.fingerprint) this.preTurn.set(sessionId, typed.fingerprint)+ return+ }+ const pending = { cancelled: false }+ this.pendingBaselines.set(sessionId, pending)+ void (async () => {+ try {+ const session = this.deps.getSession(sessionId)+ const target = session?.agentId || session?.sessionId+ if (!target || session?.active === false || (session?.engine && !pollsQuestions(session.engine))) return+ const generation = questionCaptureGeneration(session)+ const capture = await this.deps.capture(target, CAPTURE_LINES)+ if (pending.cancelled || generation !== questionCaptureGeneration(this.deps.getSession(sessionId))) return+ const view = await this.deps.readQuestion(session!, capture ?? '')+ if (pending.cancelled || generation !== questionCaptureGeneration(this.deps.getSession(sessionId))) return+ if (!view || view.kind !== 'question' || !view.question || view.rows.length === 0) return+ // Only if nothing has been announced for this turn yet: the capture takes+ // a moment, and a dialog that opened inside that window is this turn's.+ if (this.lastId.has(sessionId)) return+ this.preTurn.set(sessionId, fingerprintOf(view))+ } catch {+ // A failed read supplies no baseline. Keep watching for a fresh dialog.+ } finally {+ if (this.pendingBaselines.get(sessionId) === pending) this.pendingBaselines.delete(sessionId)+ }+ })()+ }++ /**+ * The daemon is typing a prompt, and `capture` is the pane as it read it right before (core/input.ts+ * messageWriter): the honest "before this turn" for the turn that prompt starts.+ *+ * A capture taken when the turn is SEEN to start can be too late for that. The turn is seen to start+ * from the transcript, after the prompt is confirmed typed, and an engine can draw its question before+ * that: measured in the soak run (e2e/endurance.e2e.ts) under load, Codex took the prompt at 20:42:54.9+ * (its prompt hook), drew its question a moment later, and the turn was seen to start at 20:43:02.2, 7 s+ * on. The capture at that start held the turn's own question, recorded it as the turns before's, and it+ * was never announced: the agent waited on a question no window or device was shown.+ *+ * A read that failed supplies nothing: the turn's start reads the pane itself, as before.+ */+ notePrompt(sessionId: string, capture: string | null, view: PaneView): void {+ const session = this.deps.getSession(sessionId)+ if (capture === null || !session || (session.engine && !pollsQuestions(session.engine))) return+ const dialog = view && view.kind === 'question' && view.question && view.rows.length > 0 ? fingerprintOf(view) : null+ this.beforePrompt.set(sessionId, { fingerprint: dialog, at: Date.now() })+ }++ /** Poll this session's pane while its turn is open (called on turn_started). */+ start(sessionId: string): void {+ if (this.watching.has(sessionId)) return+ const session = this.deps.getSession(sessionId)+ // Only the engines that actually paint a question dialog: Claude and Command Code share one shape,+ // devin has its own (parseDevinQuestionPane). Polling any other pane would be pure waste.+ if (!session || session.active === false || !(session.agentId || session.sessionId) || !QUESTION_ENGINES.has(session.engine)) return+ this.watching.add(sessionId)+ // One clock also starts the reads in one event-loop turn, allowing the tmux+ // backend to batch them without delaying polls or caching terminal content.+ this.timer ??= setInterval(() => {+ for (const id of this.watching) void this.tick(id)+ }, POLL_MS)+ }++ /**+ * Stop polling — and CLOSE any question still outstanding on this session.+ *+ * ⚠️ THIS IS THE COMMON CASE, NOT THE EDGE ONE, and leaving it out made the whole close mechanism look+ * like it did not work. `stop()` is called on turn_ended, and answering the question is precisely what+ * lets the turn end — so the dialog leaving the pane and the watcher being torn down happen within a+ * second or two of each other, far inside the two-tick confirmation. Measured on hardware: the answer+ * landed at ~16:50:19 and the turn ended at 16:50:29 with no close ever announced.+ *+ * No confirmation is needed here and none is wanted: the turn is over, so whatever we announced is+ * definitively not waiting for anybody any more. The same holds for the other callers — an agent that+ * was removed cannot answer either.+ */+ stop(sessionId: string): void {+ this.cancelPending(sessionId)+ this.preTurn.delete(sessionId)+ this.blocked.delete(sessionId)+ this.watching.delete(sessionId)+ if (!this.watching.size && this.timer) { clearInterval(this.timer); this.timer = undefined }+ const requestId = this.lastId.get(sessionId)+ this.forget(sessionId)+ if (requestId) this.deps.onQuestionGone?.(sessionId, requestId)+ }+++ stopAll(): void {+ if (this.timer) clearInterval(this.timer)+ this.timer = undefined+ this.watching.clear()+ for (const pending of this.pendingPolls.values()) pending.cancelled = true+ for (const pending of this.pendingBaselines.values()) pending.cancelled = true+ this.preTurn.clear()+ this.beforePrompt.clear()+ this.blocked.clear()+ this.last.clear()+ this.lastId.clear()+ this.misses.clear()+ }++ /** A device (re)joined: forget what we announced so an open question is pushed again. */+ reset(): void {+ this.last.clear()+ this.lastId.clear()+ this.misses.clear()+ }++ private forget(sessionId: string): void {+ this.last.delete(sessionId)+ this.lastId.delete(sessionId)+ this.misses.delete(sessionId)+ }++ private cancelPending(sessionId: string): void {+ const poll = this.pendingPolls.get(sessionId)+ if (poll) poll.cancelled = true+ const baseline = this.pendingBaselines.get(sessionId)+ if (baseline) baseline.cancelled = true+ }++ /**+ * A poll found no dialog. Announce the close once the miss is CONFIRMED, and only if we announced the+ * question in the first place — a session nobody was told about has nothing to take back.+ */+ private noteGone(sessionId: string): void {+ this.last.delete(sessionId) // unchanged: a dialog that comes back announces fresh+ const requestId = this.lastId.get(sessionId)+ if (!requestId) return+ const misses = (this.misses.get(sessionId) ?? 0) + 1+ this.misses.set(sessionId, misses)+ if (misses < GONE_TICKS) return+ this.lastId.delete(sessionId)+ this.misses.delete(sessionId)+ this.deps.onQuestionGone?.(sessionId, requestId)+ }+++ private async tick(sessionId: string): Promise<void> {+ // A capture can take longer than POLL_MS. Keep one outstanding poll per+ // session, including a cancelled capture that has not returned yet.+ if (this.pendingPolls.has(sessionId)) return+ const session = this.deps.getSession(sessionId)+ const terminalTarget = session?.agentId || session?.sessionId+ if (!terminalTarget || session?.active === false || (session?.engine && !pollsQuestions(session.engine))) { this.stop(sessionId); return }+ // Both of these silently do nothing, which is how a live dialog can sit on the terminal with no trace+ // in the log. Say it once per transition rather than every 1.5s tick.+ const blocked = !this.deps.hasDevice() ? 'no device' : this.deps.isDriving?.(sessionId) ? 'driving an answer' : ''+ if (blocked !== (this.blocked.get(sessionId) ?? '')) {+ this.blocked.set(sessionId, blocked)+ console.log(`[question] ${sessionId.slice(0, 8)} watcher ${blocked ? `paused · ${blocked}` : 'polling'}`)+ }+ if (blocked) return+ const pending = { cancelled: false }+ this.pendingPolls.set(sessionId, pending)+ const generation = questionCaptureGeneration(session)+ let capture: string | null+ let view: PaneView = null+ try {+ capture = await this.deps.capture(terminalTarget, CAPTURE_LINES)+ if (capture !== null) view = await this.deps.readQuestion(session!, capture)+ } catch {+ capture = null+ } finally {+ if (this.pendingPolls.get(sessionId) === pending) this.pendingPolls.delete(sessionId)+ }+ if (pending.cancelled || generation !== questionCaptureGeneration(this.deps.getSession(sessionId))+ || !this.deps.hasDevice() || this.deps.isDriving?.(sessionId)) return+ if (capture === null) {+ // Unavailable is not an empty pane, and breaks a run of confirmed misses.+ this.misses.delete(sessionId)+ return+ }+ if (view?.kind === 'question' && view.partial) {+ // Scrolled so its top is out of the pane: still open, so the client showing it keeps showing it —+ // but there is no question text to announce, and the rows in view are whichever the scroll left.+ this.misses.delete(sessionId)+ return+ }+ if (!view || view.kind !== 'question' || !view.question || view.rows.length === 0) {+ // Dialog closed, or moved to review. Either way it is no longer waiting on anybody, so the clients+ // showing it are told to stop — see noteGone for why this is not announced on the first miss.+ this.noteGone(sessionId)+ return+ }+ this.misses.delete(sessionId) // a dialog on screen ends any run of misses+ const fingerprint = fingerprintOf(view)+ // Already on the pane before this turn started → it belongs to whatever came+ // before, and has been answered. Say nothing until it changes or leaves.+ if (this.preTurn.get(sessionId) === fingerprint) return+ this.preTurn.delete(sessionId)+ if (this.last.get(sessionId) === fingerprint) return+ this.last.set(sessionId, fingerprint)++ // A pane-derived question has no tool_use id. The key only has to round-trip through the device and+ // back (the answer is keyed into the pane, not matched to a tool call), so the question's own text+ // serves as both — and the device dedups a repeated push by this id. The answer brings the id back,+ // and AskQuestionController recomputes it off the live pane before typing: a different id there+ // means a different dialog, and the answer is refused (STALE_QUESTION) instead of keyed into it.+ const requestId = questionRequestId(sessionId, view)+ this.lastId.set(sessionId, requestId)+ const baseline = this.pendingBaselines.get(sessionId)+ if (baseline) baseline.cancelled = true+ this.deps.onQuestion(sessionId, requestId, [{+ key: view.question,+ q: view.question,+ options: view.rows.map((r) => r.label),+ multi: view.multi,+ ...(view.typeRow && !view.multi && !view.permission ? { canText: true } : {}),+ }], { permission: isApprovalDialog(view), dialog: view.dialog ?? view.question })++ }+}++/** Snapshot scalar values before awaiting I/O: registry records can change in+ * place when an engine exits, moves, or is replaced in the same pane. */+function questionCaptureGeneration(session: RegisteredSession | undefined): string {+ return session ? JSON.stringify([session.agentId, session.sessionId, session.engine, session.active,+ session.tmuxPane, session.primaryRuntimeKey, session.runtimes, session.processIdentity]) : ''+}++/**+ * What makes two captures the SAME dialog: its words, its options, its arity — and, when the parser kept+ * the whole dialog, what it says below its first line (a command that differs only on its second line is+ * another prompt).+ *+ * ⚠️ NOT the raw `dialog`. The id is recomputed every 1.5s poll and again at the moment an answer is typed,+ * so it may only change when the QUESTION does. The raw dialog changes on its own: Hermes and Muse paint a+ * live timer inside it (`(01m30s · ↓ 82 tok)`, `(21s · esc to interrupt)`), and every engine moves its+ * `❯`/`›`/`>` cursor and ticks its `[✔]` boxes in place. Hashed raw, a question was re-announced as new on+ * every poll — the needs-you alert, the sound, the dial push, again and again — and every answer from a+ * dial, a device or the cable was refused as STALE_QUESTION. `dialogSignature` is the dialog with all of+ * that taken out; the raw text still goes, unchanged, to the pair's floor (isApprovalDialog, pair/sensor).+ */+function fingerprintOf(view: QuestionView): string {+ const base = `${view.question}|${view.rows.map((r) => r.label).join('|')}|${view.multi}`+ return view.dialog === undefined ? base : `${base}|${dialogSignature(view.dialog)}`+}++// A live status group: an elapsed time (`21s`, `30.5s`, `01m30s`, `1h02m`), a token counter (`↓ 82 tok`,+// `1.2k tokens`) or `esc to interrupt` inside one pair of parentheses. Units hug their digits, as every+// engine paints them, so `(see 2 files)` or `(tokens.json)` is never mistaken for one.+const TIMER_GROUP = String.raw`\([^()\n]*?(?:\b\d+(?:\.\d+)?(?:ms|s|m|h)\b|\b\d+m\d+s\b|\b\d+h\d+m\b|\d+(?:\.\d+)?k?\s*tok(?:en)?s?\b|esc to interrupt)[^()\n]*\)`+const TIMER_GROUP_RE = new RegExp(TIMER_GROUP, 'i')+const TIMER_GROUPS_RE = new RegExp(TIMER_GROUP, 'gi')+// The same, outside parentheses: `↓ 82 tokens · esc to interrupt`.+const STATUS_BITS_RE = /[↑↓]\s*\d+(?:\.\d+)?k?\s*tok(?:en)?s?\b|\besc to interrupt\b/i+// A bare elapsed time, with no parentheses and no ` · ` to mark it: a status line that is only a word or+// three and a duration — a verb in -ing/-ed (`waiting 3s`, `thinking 4s`, `Churned for 4s`) or anything+// trailing off in an ellipsis (`Waiting… 12s`, `Fetch Bitcoin price… 1m33s`). A command's own number stays:+// `sleep 30s` and `retry after 30s` are neither.+const DURATION = String.raw`(?:\d+h\d+m(?:\d+s)?|\d+m\d+s|\d+(?:\.\d+)?m?s)`+const WORDS = String.raw`(?:\p{L}[\p{L}'’-]*\s+){0,2}\p{L}[\p{L}'’-]*`+const BARE_TIMER_LINE_RE = new RegExp(String.raw`^${WORDS}(?:(?<=ing|ed)(?:\s+for)?\s+|\s*(?:…|\.{3})\s*)${DURATION}$`, 'iu')+// …and one hung off the end of a longer line, after an ellipsis or a column gap: grok's right-aligned+// `Waiting on answers for Which color should I report? 4.2s`. Only the time goes.+const TRAILING_TIMER_RE = new RegExp(String.raw`(?:(?<=…|\.{3})\s*|\s{2,})${DURATION}$`, 'u')+// Codex's cursor readout under its request_user_input rows: `option 2/4 | tab to add notes`.+const CURSOR_READOUT_RE = /^option\s+\d+\s*\/\s*\d+\b/i+// Whatever leads a line and moves on its own: a cursor (`❯ › > ▶`), a spinner frame (braille, `✻`, `◐`,+// Muse's `◇`/`◆`) or a box/tab state (`☐ ☒ ✔ ○ ● ◉`).+const LEAD_MARKS_RE = /^(?:[❯›>▶►▸➤\u2800-\u28ff✻✽✶✳✢✺✹✸✷◐◓◑◒◴◵◶◷◇◆☐☑☒✔✓✗✘○◯●◉◎]\s*)+/u+// A row's own state right after its number, or at the start of an unnumbered row: `[ ]`, `[✔]`, `(•)`, `◉`.+const ROW_STATE_RE = /^(\d+[.)]\s+|)(?:\[[^\]\n]?\]|\([^)\n]?\)|[☐☑☒✔✓○◯●◉◎])\s*/u+// Box drawing: frames and rules redraw to the pane's width.+const BOX_RE = /[\u2500-\u257f]+/g++/**+ * The dialog as a person reads it, with nothing that changes while it waits: status lines (a live timer,+ * bare or in parentheses, a token counter, `esc to interrupt`) and Codex's cursor readout dropped; cursor marks, spinner frames+ * and checkbox/radio state stripped; frames and whitespace collapsed. Every word of the prompt stays —+ * two commands that differ anywhere are still two signatures.+ */+function dialogSignature(dialog: string): string {+ const out: string[] = []+ for (const raw of dialog.replace(/\u00a0/g, ' ').split('\n')) {+ let line = raw.replace(BOX_RE, ' ').trim()+ if (CURSOR_READOUT_RE.test(line)) continue+ line = line.replace(LEAD_MARKS_RE, '')+ const row = /^\d+[.)]\s/.test(line)+ // A line that carries a live timer is the engine's status line (Hermes' `💻 curl … (01m30s · ↓ 82 tok)`+ // under its frame, Muse's `◇ Calling tools (21s · esc to interrupt)` above its rule, a bare `waiting 3s`),+ // not the prompt: the prompt is always painted on lines of its own. A ROW keeps its words; only the+ // group goes.+ if (!row && (TIMER_GROUP_RE.test(line) || STATUS_BITS_RE.test(line) || BARE_TIMER_LINE_RE.test(line))) continue+ if (!row) line = line.replace(TRAILING_TIMER_RE, '')+ line = line.replace(TIMER_GROUPS_RE, ' ').replace(ROW_STATE_RE, '$1').replace(/\s+/g, ' ').trim()+ if (line) out.push(line)+ }+ return out.join('\n')+}++function hash(value: string): string {+ let h = 0x811c9dc5+ for (let i = 0; i < value.length; i++) h = Math.imul(h ^ value.charCodeAt(i), 0x01000193) >>> 0+ return h.toString(16).padStart(8, '0')+}diff --git a/cli/src/lib/questionPane.ts b/cli/src/lib/questionPane.tsnew file mode 100644index 000000000..7f4de65a0--- /dev/null+++ b/cli/src/lib/questionPane.ts@@ -0,0 +1,118 @@+import type { AgentEngine } from '../engines/types.js'+import type { FoundDialog, PaneView, QuestionView } from '../engines/facets/screen.js'+import { locateQuestionPane, parseQuestionPane, parseRow, stripAnsi } from '../engines/kit/questionPane.js'+import { earlierDialogEnd } from './dialogEnd.js'+import { locateMuseQuestion } from '../engines/muse/askQuestion.js'+import { parseAmpQuestionPane } from '../engines/amp/askQuestion.js'+import { locateKiloQuestion, parseKiloQuestionPane } from '../engines/kilo/askQuestion.js'+import { parseCursorPermissionPane } from '../engines/cursor/askQuestion.js'+import { locateDevinPermission, locateDevinQuestion } from '../engines/devin/askQuestion.js'+import { parseGrokQuestionPane } from '../engines/grok/askQuestion.js'+import { locateAgyQuestion } from '../engines/agy/askQuestion.js'+import { withCopilotSubject } from '../engines/copilot/askQuestion.js'+function permissionView(view: PaneView): PaneView {+ return view?.kind === 'question' ? { ...view, permission: true } : view+}++function asPermission(found: FoundDialog | null): FoundDialog | null {+ return found && { ...found, view: permissionView(found.view) as QuestionView }+}++/**+ * The LOWEST dialog any reader found: the live one. A pane keeps an answered dialog in its scrollback, and+ * an engine with two readers (a question and an approval, say) must never let the first reader's hit on+ * that one shadow the second reader's hit on the dialog under it: devin's approval under its answered+ * question, muse's approval under a question, opencode's question under an approval. A tie is one dialog+ * read twice, and the earlier reader, the engine's own, keeps it.+ */+function lowest(...found: Array<FoundDialog | null>): PaneView {+ let best: FoundDialog | null = null+ for (const f of found) if (f && (!best || f.at > best.at)) best = f+ return best?.view ?? null+}++/**+ * The dialog on screen, read the way `engine` paints it: the LAST one on the pane, and nothing of it from+ * above an earlier dialog's end (`dialogEnd.ts`).+ *+ * Claude and Command Code share one shape (see parseQuestionPane); devin draws a different one and gets+ * its own parser rather than more branches in here.+ */+export function parseEngineQuestionPane(engine: AgentEngine, capture: string): PaneView {+ if (engine === 'claude' || engine === 'codex') throw new Error('Native screen reader must be injected')+ // Devin's two dialogs are told apart by one word in the footer (`↵ select` vs `↵ confirm`), so they can+ // never both match the same dialog; but an answered one of either kind can sit above the live one.+ if (engine === 'devin') return lowest(locateDevinQuestion(capture), asPermission(locateDevinPermission(capture)))+ // Cursor has no ask-the-user tool, so its permission prompt is the ONLY dialog it ever draws — and it+ // numbers nothing, stating each row's key in the row instead.+ if (engine === 'cursor') return permissionView(parseCursorPermissionPane(capture))+ // Muse pairs each option with a description on the same line and floats a live Preview box above the+ // rows — both confuse the shared parser, so it reads its own. Its PERMISSION prompt is a different+ // dialog entirely (`Would you like to allow this network access?` over `1. Yes, proceed (y)` rows under+ // a `Press enter to confirm` footer, `__fixtures__/permission-muse.txt`) and that one the shared parser+ // reads exactly, so it is read by both and the lower wins.+ if (engine === 'muse') return lowest(locateMuseQuestion(capture), locateQuestionPane(capture))+ // Amp's is a permission prompt with unnumbered rows — nothing the shared parser can anchor on.+ if (engine === 'amp') return permissionView(parseAmpQuestionPane(capture))+ // Kilo's is the same kind of prompt but laid out HORIZONTALLY, sharing its line with the key hints —+ // it is a fork of opencode that did not keep opencode's dialog.+ if (engine === 'kilo') return permissionView(parseKiloQuestionPane(capture))+ // Grok tells its approval from its questionnaire itself, by the live dialog's own footer.+ if (engine === 'grok') return parseGrokQuestionPane(capture)+ // agy's ask-the-user dialog anchors on `Question N/M:` under an `↑/↓ Navigate` footer, which the+ // shared parser cannot see. Its PERMISSION prompt is numbered rows under `Do you want to proceed?`+ // and the shared parser reads that one exactly, so it is read by both and the lower wins.+ if (engine === 'agy') return lowest(locateAgyQuestion(capture), locateQuestionPane(capture))+ // Hermes, OpenCode and Copilot paint the dialog inside a box; peel the border and the shared parser+ // fits. Measured on Copilot: framed it returns null, unframed it reads the question, the three+ // options AND spots `4. Other (type your answer)` as the free-text row rather than an option.+ if (engine === 'hermes') return parseQuestionPane(unframe(capture))+ // Copilot boxes its dialog the same way, but names the SUBJECT of a permission prompt above the+ // question — "attempting to access the following URL:" over a boxed value. Without it the device+ // shows "Do you want to allow this access?" and a bare "Yes", with nothing to judge.+ if (engine === 'copilot') {+ const found = locateQuestionPane(unframe(capture))+ return found ? withCopilotSubject(found.view, capture, found.at) : null+ }+ if (engine === 'opencode') {+ // OpenCode's PERMISSION prompt is the horizontal one kilo inherited from it — same `△ Permission+ // required` title, same `⇆ select · enter confirm` footer, same unnumbered rows. Measured: the live+ // capture in `permission-opencode.txt` parses through kilo's parser unchanged, so it is shared rather+ // than copied. Listed FIRST because that dialog numbers nothing: on its own dialog the shared parser+ // still matches the `enter confirm` footer and walks up into whatever numbered rows are above, and+ // the tie goes to kilo's reader.+ const plain = unframe(capture)+ return lowest(asPermission(locateKiloQuestion(capture)), locateOpencodeReview(plain), locateQuestionPane(plain))+ }+ return parseQuestionPane(capture)+}++function locateOpencodeReview(plain: string): FoundDialog | null {+ const lines = stripAnsi(plain).split('\n')+ const footer = lines.findLastIndex((l) => /enter\s+submit/i.test(l))+ if (footer < 0) return null+ const floor = earlierDialogEnd(lines, footer, 14)+ let sawReview = false+ for (let i = footer - 1; i > floor && footer - i <= 14; i--) {+ if (parseRow(lines[i])) return null // rows above ⇒ still a question, not the review+ if (/^\s*review\s*$/i.test(lines[i])) { sawReview = true; break }+ }+ return sawReview ? { view: { kind: 'review', submitRow: 'Enter' }, at: footer } : null+}++function unframe(capture: string): string {+ // stripAnsi FIRST: on a real capture the border is preceded by SGR codes, so matching `^│` against the+ // raw text silently never fired — hermes read as "no dialog open" with the dialog plainly on screen.+ return stripAnsi(capture)+ .split('\n')+ .map((line) => {+ const left = line.replace(/^(\s*)[│┃|]\s?/, '$1')+ // An unanchored whitespace regex retries at every column on blank rows.+ // Inspect the last non-space character once; keep unframed lines intact.+ const right = left.trimEnd()+ return /[│┃|]$/.test(right) ? right.slice(0, -1).trimEnd() : left+ })+ .join('\n')+}++/** How this engine's multi-select dialog is submitted once the boxes are ticked. */diff --git a/cli/src/lib/runtimeActivity.spec.ts b/cli/src/lib/runtimeActivity.spec.tsindex a684dc912..e90d2dbb3 100644--- a/cli/src/lib/runtimeActivity.spec.ts+++ b/cli/src/lib/runtimeActivity.spec.ts@@ -1,3 +1,4 @@+import { readInlineScreen } from '../testing/inlineScreen.js'import { afterEach, describe, expect, it, vi } from 'vitest'import { CodexActivityReader, RuntimeActivityReader } from './runtimeActivity.js'import { registry, type RegisteredSession } from './registry.js'@@ -51,7 +52,7 @@ describe('live terminal fallback', () => {it('requires changing busy indicators and refuses a frozen footer or prose', async () => {const row = session()const capture = vi.fn(async () => '• Working (30s · esc to interrupt)\n› \x1b[2mAsk Codex to do anything\x1b[0m\n? for shortcuts')- const reader = new RuntimeActivityReader({ codex: async () => 'unknown', capture })+ const reader = new RuntimeActivityReader({ readScreen: readInlineScreen, codex: async () => 'unknown', capture })expect(await reader.read(row)).toBe('unknown')expect(await reader.read(row)).toBe('unknown')capture.mockResolvedValue('• Working (35s · esc to interrupt)\n› \x1b[2mAsk Codex to do anything\x1b[0m\n? for shortcuts')@@ -62,7 +63,7 @@ describe('live terminal fallback', () => {it('recognizes the actual cmd p stopped goal without requiring or generating a transcript end', async () => {const row = session()const capture = vi.fn(async () => '■ Conversation interrupted\n› \x1b[2mAsk Codex to do anything\x1b[0m\n GPT-6-Astra max · ~/work Goal stalled (/goal resume)\n? for shortcuts')- const reader = new RuntimeActivityReader({ codex: async () => 'unknown', capture })+ const reader = new RuntimeActivityReader({ readScreen: readInlineScreen, codex: async () => 'unknown', capture })expect(await reader.read(row)).toBe('unknown')expect(await reader.read(row)).toBe('idle')row.processIdentity = { ...row.processIdentity!, startMarker: 'replacement' }@@ -73,9 +74,9 @@ describe('live terminal fallback', () => {it('prefers structured status and leaves unsupported engines unknown', async () => {const row = session(); const capture = vi.fn(async () => null)const codex = vi.fn(async () => 'working' as const)- const reader = new RuntimeActivityReader({ codex, capture })+ const reader = new RuntimeActivityReader({ readScreen: readInlineScreen, codex, capture })expect(await reader.read(row)).toBe('working'); expect(capture).not.toHaveBeenCalled()- const unsupported = new RuntimeActivityReader({ codex: async () => 'unknown', capture })+ const unsupported = new RuntimeActivityReader({ readScreen: readInlineScreen, codex: async () => 'unknown', capture })row.engine = 'pi'; expect(await unsupported.read(row)).toBe('unknown'); expect(capture).not.toHaveBeenCalled()})})diff --git a/cli/src/lib/runtimeActivity.ts b/cli/src/lib/runtimeActivity.tsindex e9e6ac8d4..b28406afc 100644--- a/cli/src/lib/runtimeActivity.ts+++ b/cli/src/lib/runtimeActivity.ts@@ -1,8 +1,7 @@import { basename } from 'node:path'import { sessionCodexHome } from './engineHomes.js'-import { terminalActivityReading } from './terminalActivity.js'+import type { ScreenReader } from './screenReader.js'import { connectCodexControl, type CodexControl } from './codexSessionLifecycle.js'-import { codexStoppedGoal } from './codexTurnRecovery.js'import { argvTokens, processRows, type ProcessRow } from './tmux.js'import { sameProcessIdentity } from './terminalRuntime.js'import type { RegisteredSession } from './registry.js'@@ -63,6 +62,7 @@ export class CodexActivityReader {export class RuntimeActivityReader {private readonly footers = new Map<string, { runtime: string; indicator: string | null; stopped: boolean }>()constructor(private readonly deps: {+ readScreen: ScreenReadercodex(session: RegisteredSession): Promise<ActivityState>capture(session: RegisteredSession): Promise<string | null>}) {}@@ -74,8 +74,10 @@ export class RuntimeActivityReader {const screen = await this.deps.capture(session)const runtime = activityRuntimeKey(session)const previous = this.footers.get(session.sessionId)- const indicator = terminalActivityReading(session.engine, screen)?.indicator ?? null- const stopped = session.engine === 'codex' && codexStoppedGoal(screen)+ const reading = await this.deps.readScreen(session, screen)+ if (!reading) { this.footers.delete(session.sessionId); return 'unknown' }+ const indicator = reading.activity?.indicator ?? null+ const stopped = reading.stoppedGoalthis.footers.set(session.sessionId, { runtime, indicator, stopped })if (previous?.runtime !== runtime) return 'unknown'if (stopped && previous.stopped) return 'idle'diff --git a/cli/src/lib/runtimeControl.ts b/cli/src/lib/runtimeControl.tsnew file mode 100644index 000000000..8446d5a00--- /dev/null+++ b/cli/src/lib/runtimeControl.ts@@ -0,0 +1,892 @@+import type { ScreenReader } from './screenReader.js'+import { stripAnsi } from '../engines/kit/pane.js'+export type { PaneInspection, PaneTakeover } from '../engines/facets/screen.js'+import { parseRuntimeProfile } from './runtimeProfileWire.js'+import type { RegisteredSession } from './registry.js'+import { devinModelCommandResult } from '../engines/devin/runtimeProfile.js'+import { countCommandcodeRefusals } from '../engines/commandcode/runtimeProfile.js'+import { parseHermesPickerPage } from '../engines/hermes/runtimeProfile.js'+import { parsePiFooterProfile, parsePiThinkingSelection, piThinkingSteps } from '../engines/pi/runtimeProfile.js'+import {+ countOpencodePickers,+ opencodeRowMatches,+ opencodeRowNamesModel,+ parseOpencodePickerRows,+ type OpencodeModelTarget,+ type OpencodePickerRow,+} from '../engines/opencode/runtimeProfile.js'+import {+ chooseCodexRow,+ codexDigitPressable,+ codexEffortRows,+ parseCodexPicker,+ sameCodexPicker,+ type CodexCatalogModel,+ type CodexPicker,+} from '../engines/codex/modelPicker.js'+import {+ type CursorModelTarget,+ type RuntimeProfile,+ type RuntimeProfileManager,+} from './runtimeProfile.js'++const COMMAND_CONFIRM_MS = 8_000+const PICKER_OPEN_MS = 3_000+const PICKER_STEP_MS = 2_000+/**+ * How far apart the two reads of a Codex model list are, which must agree before a digit is pressed on+ * it (see reachCodexEfforts): long enough for a redraw in flight to land, short beside the seconds a+ * switch takes to confirm.+ */+const CODEX_SETTLE_MS = 250+/** Hard bound on ladder keystrokes — twice pi's seven levels, so a desynchronised walk still terminates. */+const PI_LADDER_MAX_STEPS = 14+/** Hermes' longest picker page is a provider's model list; twice its size still terminates. */+const HERMES_PAGE_MAX_STEPS = 60+/** "Anthropic (13 models)" names provider key `anthropic`; "GitHub Copilot (17 models)" names `copilot`. */+function hermesProviderMatches(row: string, provider: string): boolean {+ const name = row.replace(/\s*\([^)]*\)\s*$/, '').trim().toLowerCase()+ const key = provider.toLowerCase()+ return name === key || name.replace(/[^a-z0-9]/g, '').includes(key.replace(/[^a-z0-9]/g, ''))+}++export type RuntimeProfileErrorCode =+ | 'AGENT_NOT_FOUND'+ | 'INVALID_RUNTIME_PROFILE'+ | 'BUSY'+ | 'UNSUPPORTED_CLI_VERSION'+ | 'MODEL_UNAVAILABLE'+ | 'EFFORT_UNSUPPORTED'+ | 'PLAN_SCOPE_AMBIGUOUS'+ | 'CONFIRM_TIMEOUT'+ | 'TMUX_FAILED'++export class RuntimeProfileControlError extends Error {+ constructor(readonly code: RuntimeProfileErrorCode) {+ super(code)+ }+}++export interface RuntimeProfileControllerDeps {+ readScreen: ScreenReader+ manager: Pick<RuntimeProfileManager, 'getState' | 'selectedModel' | 'codexCatalog' | 'modelsForSession'+ | 'beginControl' | 'finishControl' | 'cancelControl' | 'confirmEffort' | 'waitForProfile' | 'waitForModel'+ | 'devinTarget' | 'opencodeCatalog' | 'commandcodeTarget' | 'ingestConfig' | 'hermesTarget' | 'cursorTarget' | 'confirmControlProfile'> & {+ supportsControl(session: RegisteredSession): boolean | Promise<boolean>+ effortAllowed(session: RegisteredSession, model: string, effort: string, listed: readonly string[] | null): boolean | Promise<boolean>+ ingestPane(session: RegisteredSession, text: string, silent?: boolean): boolean | Promise<boolean>+ }+ getSession: (sessionId: string) => RegisteredSession | undefined+ validateRuntime: (session: RegisteredSession) => Promise<boolean>+ capture: (terminalTarget: string, historyLines?: number) => Promise<string | null>+ sendText: (terminalTarget: string, text: string) => Promise<boolean>+ /** Type without submitting. */+ sendLiteral: (terminalTarget: string, text: string) => Promise<boolean>+ sendKey: (terminalTarget: string, key: string) => Promise<boolean>+ acquireInput: (sessionId: string) => (() => void) | null+}++export interface CursorParameterRow {+ kind: 'context' | 'reasoning' | 'fast' | 'thinking'+ value: string+ selected: boolean+ cursor: boolean+ index: number+}++export function parseCursorModelPicker(capture: string): { selectedFamily: string | null } | null {+ const lines = stripAnsi(capture).split('\n')+ if (!lines.some((line) => /Available models|Models matching/i.test(line)) || !lines.some((line) => /Tab to edit/i.test(line))) {+ return null+ }+ const selected = lines.findLast((line) => /^\s*→\s+/.test(line))+ if (!selected) return { selectedFamily: null }+ const value = selected.replace(/^\s*→\s+/, '').replace(/\s+\(Tab to modify\)\s*$/i, '')+ return { selectedFamily: value.split(/\s{2,}/)[0]?.trim() || null }+}++export function parseCursorParameterRows(capture: string): CursorParameterRow[] | null {+ const lines = stripAnsi(capture).split('\n')+ if (!lines.some((line) => /— Edit Parameters/i.test(line))) return null+ const rows: CursorParameterRow[] = []+ let section: 'context' | 'reasoning' | null = null+ for (const raw of lines) {+ const line = raw.replace(/\u00a0/g, ' ')+ if (/^\s*Context\s*$/.test(line)) { section = 'context'; continue }+ if (/^\s*Reasoning\s*$/.test(line)) { section = 'reasoning'; continue }+ const choice = /^\s*(→)?\s*([●○◉◯])\s+(.+?)(?:\s+✓)?\s*$/.exec(line)+ if (!choice) continue+ const label = choice[3].trim()+ let kind: CursorParameterRow['kind']+ let value: string+ if (/^Fast$/i.test(label)) {+ kind = 'fast'+ value = 'true'+ } else if (/^Thinking$/i.test(label)) {+ kind = 'thinking'+ value = 'true'+ } else if (section === 'context') {+ kind = 'context'+ value = label.toLowerCase()+ } else if (section === 'reasoning') {+ kind = 'reasoning'+ value = label.toLowerCase().replace(/\s+/g, '') === 'extrahigh'+ ? 'xhigh'+ : label.toLowerCase()+ } else {+ continue+ }+ rows.push({+ kind,+ value,+ selected: choice[2] === '●' || choice[2] === '◉' || /✓\s*$/.test(line),+ cursor: !!choice[1],+ index: rows.length,+ })+ }+ return rows.length ? rows : null+}++/** The lists a model is chosen from, as against the reasoning screens that follow them. */+function isCodexList(picker: CodexPicker): boolean {+ return picker.kind === 'quick' || picker.kind === 'models' || picker.kind === 'reserve'+}++function sleep(ms: number): Promise<void> {+ return new Promise((resolve) => setTimeout(resolve, ms))+}++export class RuntimeProfileController {+ constructor(private readonly deps: RuntimeProfileControllerDeps) {}++ async setProfile(sessionId: string, encoded: unknown): Promise<void> {+ const registeredSession = this.deps.getSession(sessionId)+ if (!registeredSession) throw new RuntimeProfileControlError('AGENT_NOT_FOUND')+ const observed = this.deps.manager.getState(registeredSession.sessionId)+ const session: RegisteredSession = registeredSession.cliVersion || !observed.cliVersion+ ? registeredSession+ : { ...registeredSession, cliVersion: observed.cliVersion }+ const target = parseRuntimeProfile(encoded)+ // Either id identifies the agent: the encoded profile now carries the agent id, but one minted+ // before the id cutover (or by a client that still knows the session) must keep working.+ if (!target || (target.sessionId !== session.agentId && target.sessionId !== session.sessionId) || target.engine !== session.engine) {+ throw new RuntimeProfileControlError('INVALID_RUNTIME_PROFILE')+ }+ const current = parseRuntimeProfile(this.deps.manager.selectedModel(session))+ if (current?.id === target.id) return+ if (!await this.deps.manager.supportsControl(session)) {+ console.warn(`[runtime-profile] unsupported ${session.engine} CLI version ${session.cliVersion ?? 'unknown'} for ${sessionId.slice(0, 8)}`)+ throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')+ }+ // Codex's own catalog: which efforts each model takes, and the names its picker shows them by.+ const codexCatalog = session.engine === 'codex' ? await this.deps.manager.codexCatalog(session) : []+ if (session.engine === 'codex') {+ const listed = codexCatalog.find((entry) => entry.slug === target.model)?.efforts ?? null+ if (!await this.deps.manager.effortAllowed(session, target.model, target.effort, listed)) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')+ }+ const options = await this.deps.manager.modelsForSession(session)+ if (!options.some((option) => option.id === target.id)) {+ const sameModel = options.some((option) => parseRuntimeProfile(option.id)?.model === target.model)+ throw new RuntimeProfileControlError(sameModel ? 'EFFORT_UNSUPPORTED' : 'MODEL_UNAVAILABLE')+ }+ if (session.engine === 'codex' && observed.mode === 'plan') throw new RuntimeProfileControlError('PLAN_SCOPE_AMBIGUOUS')+ const release = this.deps.acquireInput(session.agentId)+ if (!release) throw new RuntimeProfileControlError('BUSY')+ let controlStarted = false+ let pickerOpen = false+ try {+ if (!await this.deps.validateRuntime(session)) throw new RuntimeProfileControlError('TMUX_FAILED')+ const capture = await this.deps.capture(session.agentId, 100)+ if (!capture) throw new RuntimeProfileControlError('TMUX_FAILED')+ const inspection = (await this.deps.readScreen(session, capture))?.pane+ if (!inspection) throw new RuntimeProfileControlError('BUSY')+ if (session.engine === 'codex' && inspection.plan) throw new RuntimeProfileControlError('PLAN_SCOPE_AMBIGUOUS')+ if (!inspection.idle) throw new RuntimeProfileControlError('BUSY')+ if (!this.deps.manager.beginControl(session, target)) throw new RuntimeProfileControlError('BUSY')+ controlStarted = true+ if (session.engine === 'claude') {+ await this.setClaude(session, target, current, options)+ } else if (session.engine === 'codex') {+ pickerOpen = true+ await this.setCodex(session, target, codexCatalog)+ pickerOpen = false+ } else if (session.engine === 'devin') {+ // No picker is ever opened — `/model <id>` is a single command — so there is nothing to Escape out+ // of if this throws.+ await this.setDevin(session, target)+ } else if (session.engine === 'pi') {+ pickerOpen = true+ await this.setPi(session, target, current)+ pickerOpen = false+ } else if (session.engine === 'opencode') {+ pickerOpen = true+ await this.setOpencode(session, target)+ pickerOpen = false+ } else if (session.engine === 'hermes') {+ pickerOpen = true+ await this.setHermes(session, target)+ pickerOpen = false+ } else if (session.engine === 'commandcode') {+ // Two plain commands, no dialog — nothing to Escape out of if either throws.+ await this.setCommandcode(session, target, current)+ } else {+ pickerOpen = true+ await this.setCursor(session, target)+ pickerOpen = false+ }+ this.deps.manager.finishControl(session)+ controlStarted = false+ } catch (error) {+ if (pickerOpen) {+ for (let attempt = 0; attempt < 3; attempt++) {+ await this.deps.sendKey(session.agentId, 'Escape').catch(() => false)+ await sleep(100)+ const next = await this.deps.capture(session.agentId, 100).catch(() => null)+ if (!next || !(await this.deps.readScreen(session, next))?.pane.dialog) break+ }+ }+ // By the engine session, as beginControl keyed it. `sessionId` is whichever id the request named,+ // and the apps name the agent: cancelling under that left the control in place, and every later+ // switch of the agent answered BUSY until the daemon restarted (found by e2e/models.e2e.ts, a+ // refused switch followed by one that should have gone through).+ if (controlStarted) this.deps.manager.cancelControl(session.sessionId)+ // A failed switch used to leave NOTHING in the log — the picker just flashed open and shut on the+ // user's terminal and the device said nothing useful. Name the session, engine and reason.+ const code = error instanceof RuntimeProfileControlError ? error.code : 'TMUX_FAILED'+ console.warn(`[runtime-profile] ${sessionId.slice(0, 8)} ${session.engine} set ${target.model}@${target.effort} failed: ${code}`)+ if (error instanceof RuntimeProfileControlError) throw error+ throw new RuntimeProfileControlError('TMUX_FAILED')+ } finally {+ release()+ }+ }++ private async setClaude(+ session: RegisteredSession,+ target: RuntimeProfile,+ current: RuntimeProfile | null,+ options: Array<{ id: string }>,+ ): Promise<void> {+ if (current?.model !== target.model) {+ if (!await this.deps.sendText(session.agentId, `/model ${target.model}`)) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ if (!await this.waitClaudeModel(session)) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')+ }++ if (current?.effort === target.effort) {+ this.deps.manager.confirmEffort(session.sessionId, target.effort)+ return+ }++ const hasExplicitEffort = options.some((option) => {+ const profile = parseRuntimeProfile(option.id)+ return profile?.model === target.model && profile.effort !== 'auto'+ })+ if (!hasExplicitEffort && target.effort === 'auto') {+ this.deps.manager.confirmEffort(session.sessionId, 'auto')+ return+ }+ if (!await this.deps.sendText(session.agentId, `/effort ${target.effort}`)) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ if (!await this.deps.manager.waitForProfile(session.sessionId, COMMAND_CONFIRM_MS)) {+ await this.deps.sendKey(session.agentId, 'Enter')+ if (!await this.deps.manager.waitForProfile(session.sessionId, 2_000)) {+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')+ }+ }+ }++ private async waitClaudeModel(session: RegisteredSession): Promise<boolean> {+ if (await this.deps.manager.waitForModel(session.sessionId, 1_200)) return true+ const capture = await this.deps.capture(session.agentId, 80)+ if (capture && /switch model|change model|continue.*model|re-read.*history/i.test(stripAnsi(capture))) {+ await this.deps.sendKey(session.agentId, 'Enter')+ } else {+ // Claude and Codex can swallow the first Enter immediately after a terminal literal write.+ await this.deps.sendKey(session.agentId, 'Enter')+ }+ return this.deps.manager.waitForModel(session.sessionId, COMMAND_CONFIRM_MS)+ }++ /**+ * Devin takes the whole profile in one command: `/model <id>` accepts the exact id from+ * `devin models list` (effort included), and answers in the pane — `✓ Model set to <name>` or+ * `✗ Model not available`. That explicit answer is why devin needs no picker driving at all, and why an+ * unavailable model (most of them, on a free plan) fails fast instead of timing out.+ */+ private async setDevin(session: RegisteredSession, target: RuntimeProfile): Promise<void> {+ const devin = this.deps.manager.devinTarget(session.sessionId, target.id)+ if (!devin) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')+ if (!await this.deps.sendText(session.agentId, `/model ${devin.id}`)) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ const answered = await this.waitPane(+ session.agentId,+ (value) => devinModelCommandResult(stripAnsi(value)) !== null,+ COMMAND_CONFIRM_MS,+ )+ if (!answered) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')+ if (devinModelCommandResult(stripAnsi(answered)) === 'unavailable') {+ throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')+ }+ // The acknowledgement line is not the state — the footer is, and only `ingestPane` reads it. Nothing+ // else feeds the manager during a control, so the wait has to do the feeding itself (same shape as+ // setCursor); waiting on `waitForProfile` alone timed out on a switch that had already landed.+ if (!await this.waitObservedProfile(session, target)) {+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')+ }+ }++ /**+ * Pi splits the axes: `/model <provider/model>` takes a target directly, while the thinking level lives+ * only in `/settings` → "Thinking Level", a fixed ladder walked with the arrow keys. Both were driven by+ * hand first; the ladder's order is what makes the arrow count deterministic (see PI_THINKING_LEVELS).+ */+ private async setPi(+ session: RegisteredSession,+ target: RuntimeProfile,+ current: RuntimeProfile | null,+ ): Promise<void> {+ if (current?.model !== target.model) {+ if (!await this.deps.sendText(session.agentId, `/model ${target.model}`)) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ if (!await this.deps.manager.waitForModel(session.sessionId, COMMAND_CONFIRM_MS)) {+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')+ }+ }+ if (target.effort === 'auto' || current?.effort === target.effort) {+ this.deps.manager.confirmEffort(session.sessionId, target.effort)+ return+ }+ await this.setPiThinking(session, target)+ }++ private async setPiThinking(session: RegisteredSession, target: RuntimeProfile): Promise<void> {+ const effort = target.effort+ if (!await this.deps.sendText(session.agentId, '/settings')) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ if (!await this.waitPane(session.agentId, (v) => /Type to search/i.test(stripAnsi(v)), PICKER_OPEN_MS)) {+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')+ }+ // Typing alone is enough: the settings filter opens a row the moment it narrows to one. Submitting+ // here would land an Enter on the ladder that just opened and pick the level already highlighted —+ // which is the CURRENT one, so the switch silently did nothing.+ if (!await this.deps.sendLiteral(session.agentId, 'thinking level')) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ const ladder = await this.waitPane(+ session.agentId,+ (v) => /Select reasoning depth/i.test(stripAnsi(v)),+ PICKER_OPEN_MS,+ )+ if (!ladder) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')++ // Walk the ladder ONE key at a time, re-reading the cursor after each. Counting the steps up front and+ // firing them blind is what an open-loop drive looks like, and it is exactly how this went wrong: a+ // stale first read sent the cursor to `off` and committed it. Re-reading also means a ladder that+ // scrolls, clamps at an end, or gains a level cannot desynchronise the walk.+ let selection = parsePiThinkingSelection(stripAnsi(ladder))+ if (!selection || piThinkingSteps(selection, effort) === null) {+ throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')+ }+ for (let guard = 0; selection !== effort && guard < PI_LADDER_MAX_STEPS; guard++) {+ const steps = piThinkingSteps(selection as string, effort)+ if (steps === null || steps === 0) break+ if (!await this.deps.sendKey(session.agentId, steps > 0 ? 'Down' : 'Up')) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ const moved = await this.waitPane(+ session.agentId,+ (value) => {+ const next = parsePiThinkingSelection(stripAnsi(value))+ return !!next && next !== selection+ },+ PICKER_STEP_MS,+ )+ // No movement means the cursor is pinned at an end of the ladder — pressing on cannot reach it.+ if (!moved) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')+ selection = parsePiThinkingSelection(stripAnsi(moved))+ }+ if (selection !== effort) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')+ await this.deps.sendKey(session.agentId, 'Enter')+ // The ladder returns to the settings list; Escape closes it so the pane is idle again.+ await this.deps.sendKey(session.agentId, 'Escape')+ if (!await this.waitObservedProfile(session, target)) {+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')+ }+ }++ /** Poll the pane INTO the manager until it reports the profile we asked for. */+ private async waitObservedProfile(session: RegisteredSession, target: RuntimeProfile): Promise<boolean> {+ const confirmed = await this.waitPane(session.agentId, async (value) => {+ await this.deps.manager.ingestPane(session, value, true)+ return this.deps.manager.selectedModel(session) === target.id+ }, COMMAND_CONFIRM_MS)+ return !!confirmed+ }++ /**+ * OpenCode switches through `/models`, a picker with a typed filter. It is driven by NARROWING, never by+ * arrowing: type the model's words plus its provider's, and press Enter only once exactly one row is+ * left and that row is the requested model. If the filter leaves two rows (two providers can carry the+ * same model name) the switch is refused rather than guessed — picking the wrong model silently is worse+ * than not switching, and blind arrow-driving is what sank the Command Code attempt.+ */+ private async setOpencode(session: RegisteredSession, target: RuntimeProfile): Promise<void> {+ const entry = (await this.deps.manager.opencodeCatalog()).find((item) => item.id === target.model)+ if (!entry) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')+ await this.driveOpencodePicker(session, entry)+ if (!await this.waitObservedProfile(session, target)) {+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')+ }+ }++ /**+ * `/models`, filter, act only once exactly one row is left.+ *+ * Two attempts, because one filter does not fit both renders. With several providers connected a+ * row carries its provider (`Big Pickle OpenCode Zen`) and the provider-qualified filter is what+ * tells two same-named models apart; with one provider the row is bare (`Big Pickle`) and that same+ * filter matches NOTHING. Measured on a live picker: `big pickle opencode` → empty, `big pickle` →+ * the row.+ *+ * The fallback drops only the PROVIDER half, never the "exactly one row" rule — which is what makes+ * it safe: if one model answers to the name, there is no second provider to confuse it with, and if+ * two do, the list does not narrow and this refuses rather than guessing.+ */+ private async driveOpencodePicker(session: RegisteredSession, entry: OpencodeModelTarget): Promise<void> {+ if (await this.pickOpencodeRow(session, entry, entry.filter, opencodeRowMatches)) return+ if (entry.modelFilter === entry.filter) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')+ if (!await this.pickOpencodeRow(session, entry, entry.modelFilter, opencodeRowNamesModel)) {+ throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')+ }+ }++ /**+ * One attempt at the picker: open it, type `filter`, and press Enter only if exactly one row is left+ * AND `matches` accepts it. Returns false — with the picker CLOSED — when the list did not narrow to+ * a row this may act on, so the caller can try a different filter on a pane in its resting state.+ * Throws only for the states no retry can help: tmux refusing input, or the picker never opening.+ */+ private async pickOpencodeRow(+ session: RegisteredSession,+ entry: OpencodeModelTarget,+ filter: string,+ matches: (target: OpencodeModelTarget, row: OpencodePickerRow) => boolean,+ ): Promise<boolean> {+ // "Is a picker open?" cannot be asked of a capture that carries scrollback — an EARLIER picker is+ // still up there, so the check passes before this one opens and the filter would be typed into the+ // composer and submitted as a message. Count the openings instead and wait for one more.+ const before = countOpencodePickers(stripAnsi(await this.deps.capture(session.agentId, 100) ?? ''))+ if (!await this.deps.sendText(session.agentId, '/models')) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ if (!await this.waitPane(+ session.agentId,+ (v) => countOpencodePickers(stripAnsi(v)) > before,+ PICKER_OPEN_MS,+ )) {+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')+ }+ if (!await this.deps.sendLiteral(session.agentId, filter)) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ const narrowed = await this.waitPane(+ session.agentId,+ (v) => (parseOpencodePickerRows(stripAnsi(v)) ?? []).length === 1,+ PICKER_STEP_MS,+ )+ const rows = narrowed ? parseOpencodePickerRows(stripAnsi(narrowed)) ?? [] : []+ if (rows.length !== 1 || !matches(entry, rows[0])) {+ // Leave the pane as it was found. A picker left open with a dead filter in it swallows whatever+ // the user types next, and the retry below would type its filter on top of this one.+ await this.deps.sendKey(session.agentId, 'Escape')+ return false+ }+ if (!await this.deps.sendKey(session.agentId, 'Enter')) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ return true+ }++ /**+ * Command Code 1.6.0 takes both axes as arguments: `/model <vendor/id>` and `/effort <level>`. Neither+ * opens a dialog, which is the whole reason switching is back after the 2026-07-30 revert — the version+ * gate in supportsNativeRuntimeControl keeps older builds, where only an arrow-driven picker existed,+ * out of this path.+ *+ * Effort is per model and the catalog does not say which models have one, so a refusal is expected+ * traffic: the CLI answers "Reasoning effort not supported for X." and that becomes EFFORT_UNSUPPORTED.+ */+ private async setCommandcode(+ session: RegisteredSession,+ target: RuntimeProfile,+ current: RuntimeProfile | null,+ ): Promise<void> {+ const entry = this.deps.manager.commandcodeTarget(session.sessionId, target.id)+ if (!entry) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')++ if (current?.model !== target.model) {+ if (!await this.deps.sendText(session.agentId, `/model ${entry.id}`)) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ if (!await this.waitObservedModel(session, target)) {+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')+ }+ }++ if (target.effort === 'auto' || current?.effort === target.effort) {+ this.deps.manager.confirmEffort(session.sessionId, target.effort)+ return+ }+ // Success prints NOTHING and a refusal prints one line, so both have to be watched at once — and the+ // refusal has to be a NEW one. Matching the text anywhere in the capture reported failure in 25ms off+ // a refusal still sitting in the scrollback from an earlier model, while the level had in fact applied.+ const refusalsBefore = countCommandcodeRefusals(stripAnsi(await this.deps.capture(session.agentId, 100) ?? ''))+ if (!await this.deps.sendText(session.agentId, `/effort ${target.effort}`)) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ const deadline = Date.now() + COMMAND_CONFIRM_MS+ while (Date.now() < deadline) {+ // The level is recorded only in the CLI's global config, so confirming it means re-reading that.+ await this.deps.manager.ingestConfig(session, true).catch(() => false)+ if (this.deps.manager.selectedModel(session) === target.id) return+ const capture = stripAnsi(await this.deps.capture(session.agentId, 100) ?? '')+ if (countCommandcodeRefusals(capture) > refusalsBefore) {+ throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')+ }+ await sleep(150)+ }+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')+ }++ /**+ * Hermes' `/model` is a two-level picker — provider page, then that provider's models — with no way to+ * type or jump. It is walked one arrow at a time, re-reading the `❯` cursor after every key, so a page+ * that scrolls or reorders cannot desynchronise the walk. If the cursor stops moving before it reaches+ * the target, the target is not on the page and the switch is refused rather than committed blind.+ *+ * Only MODELS are switchable; hermes keeps its reasoning effort in config.yaml with no in-session+ * command, so hermesModels never offers an effort row to get here (see hermesModels).+ */+ private async setHermes(session: RegisteredSession, target: RuntimeProfile): Promise<void> {+ const entry = this.deps.manager.hermesTarget(session.sessionId, target.id)+ if (!entry) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')+ if (!await this.deps.sendText(session.agentId, '/model')) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ const opened = await this.waitPane(+ session.agentId,+ (v) => !!parseHermesPickerPage(stripAnsi(v)),+ PICKER_OPEN_MS,+ )+ if (!opened) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')++ // Page 1: the provider. Its row reads "Anthropic (13 models)", so match on the leading name.+ if (entry.provider) {+ await this.walkHermesPage(session, (row) => hermesProviderMatches(row, entry.provider))+ if (!await this.waitPane(+ session.agentId,+ (v) => (parseHermesPickerPage(stripAnsi(v))?.rows ?? []).some((row) => row === entry.id),+ PICKER_OPEN_MS,+ )) {+ throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')+ }+ }+ // Page 2: the model itself, listed by exact id.+ await this.walkHermesPage(session, (row) => row === entry.id)+ if (!await this.waitObservedModel(session, target)) {+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')+ }+ }++ /** Move the `❯` cursor onto the row `wanted` selects, then commit it with Enter. */+ private async walkHermesPage(+ session: RegisteredSession,+ wanted: (row: string) => boolean,+ ): Promise<void> {+ for (let guard = 0; guard < HERMES_PAGE_MAX_STEPS; guard++) {+ const capture = await this.deps.capture(session.agentId, 100)+ const page = capture ? parseHermesPickerPage(stripAnsi(capture)) : null+ if (!page || !page.selected) throw new RuntimeProfileControlError('TMUX_FAILED')+ if (wanted(page.selected)) {+ if (!await this.deps.sendKey(session.agentId, 'Enter')) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ return+ }+ const at = page.rows.indexOf(page.selected)+ const to = page.rows.findIndex(wanted)+ if (to < 0) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')+ if (!await this.deps.sendKey(session.agentId, to > at ? 'Down' : 'Up')) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ const moved = await this.waitPane(session.agentId, (v) => {+ const next = parseHermesPickerPage(stripAnsi(v))+ return !!next?.selected && next.selected !== page.selected+ }, PICKER_STEP_MS)+ if (!moved) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')+ }+ throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')+ }++ /** Like waitObservedProfile, but only the model half — used where effort is applied separately. */+ private async waitObservedModel(session: RegisteredSession, target: RuntimeProfile): Promise<boolean> {+ const confirmed = await this.waitPane(session.agentId, async (value) => {+ await this.deps.manager.ingestPane(session, value, true)+ return parseRuntimeProfile(this.deps.manager.selectedModel(session))?.model === target.model+ }, COMMAND_CONFIRM_MS)+ return !!confirmed+ }++ /**+ * Codex's `/model`: a quick menu of auto presets with an `All models` row (0.145 and later), the full+ * list (`Select Model and Effort`), then the chosen model's reasoning picker, each row pressed by its+ * digit, which selects the row and accepts it. How a row is read and chosen, and why, is in+ * engines/codex/modelPicker.ts. Confirmed, as it always was, by the `thread_settings_applied` record+ * Codex writes once the choice is applied (RuntimeProfileManager `ingestCodex`).+ */+ private async setCodex(session: RegisteredSession, target: RuntimeProfile, catalog: CodexCatalogModel[]): Promise<void> {+ if (!await this.deps.sendText(session.agentId, '/model')) throw new RuntimeProfileControlError('TMUX_FAILED')+ let picker = await this.waitCodexPicker(session, () => true, 900)+ if (!picker) {+ await this.deps.sendKey(session.agentId, 'Enter')+ picker = await this.waitCodexPicker(session, () => true, PICKER_OPEN_MS)+ }+ if (!picker) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')+ if (!isCodexList(picker)) {+ // Opened on a reasoning screen: back to the list it was opened from, so the model is chosen too.+ if (!await this.deps.sendKey(session.agentId, 'Escape')) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')+ if (!await this.waitCodexPicker(session, isCodexList, PICKER_STEP_MS)) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')+ }+ const reached = await this.reachCodexEfforts(session, target, catalog)+ if (reached !== 'applied') await this.pickCodexEffort(session, target, reached)+ if (!await this.deps.manager.waitForProfile(session.sessionId, COMMAND_CONFIRM_MS)) {+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')+ }+ }++ /**+ * From the list on screen to the target model's reasoning picker, through `All models` when the quick+ * menu is first; or 'applied' when the model's row applied it at once (a model with one effort).+ *+ * Every digit is pressed from a list read twice, CODEX_SETTLE_MS apart, the same both times. Codex+ * 0.160 draws the list from its cache and redraws it in place when the server answers, and a model+ * added or reordered then renumbers the rows: a digit read off the first drawing lands on another+ * model. The reasoning picker that opens names its model in its title, and a title naming any other+ * model than the row pressed means the list moved anyway, so nothing more is pressed.+ */+ private async reachCodexEfforts(+ session: RegisteredSession,+ target: RuntimeProfile,+ catalog: CodexCatalogModel[],+ ): Promise<CodexPicker | 'applied'> {+ let reread = false+ for (let lists = 0; lists < 2; lists++) {+ const list = await this.settledCodexList(session)+ if (!list) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')+ let choice = chooseCodexRow(list, target, catalog)+ if ('error' in choice && choice.error === 'MODEL_UNAVAILABLE' && !reread) {+ // Codex saves the catalog its picker refreshed from to models_cache.json, so a model or a name+ // that came with the refresh is there to read now.+ reread = true+ catalog = await this.deps.manager.codexCatalog(session)+ choice = chooseCodexRow(list, target, catalog)+ }+ if ('error' in choice) throw new RuntimeProfileControlError(choice.error)+ if (!await this.deps.sendKey(session.agentId, String(choice.row.number))) throw new RuntimeProfileControlError('TMUX_FAILED')+ if (choice.opens === 'list') {+ if (!await this.waitCodexPicker(session, (next) => next.kind === 'models', PICKER_STEP_MS)) {+ throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')+ }+ continue+ }+ if (choice.opens === 'applied') return 'applied'+ const next = await this.waitCodexAfterModelRow(session)+ if (next === 'closed') return 'applied'+ if (!next) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')+ if (next.model !== choice.row.name) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')+ return next+ }+ throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')+ }++ /** The target's effort in its reasoning picker, through Advanced Reasoning for Max and Ultra. */+ private async pickCodexEffort(session: RegisteredSession, target: RuntimeProfile, picker: CodexPicker): Promise<void> {+ const rows = codexEffortRows(picker)+ let row = target.effort === 'auto' ? rows.defaultRow : rows.efforts.get(target.effort) ?? null+ if (row === null && (target.effort === 'max' || target.effort === 'ultra') && rows.advancedRow !== null+ && codexDigitPressable(rows.advancedRow)) {+ if (!await this.deps.sendKey(session.agentId, String(rows.advancedRow))) throw new RuntimeProfileControlError('TMUX_FAILED')+ const advanced = await this.waitCodexPicker(session, (next) => next.kind === 'advanced', PICKER_STEP_MS)+ row = advanced ? codexEffortRows(advanced).efforts.get(target.effort) ?? null : null+ }+ if (row === null || !codexDigitPressable(row)) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')+ if (!await this.deps.sendKey(session.agentId, String(row))) throw new RuntimeProfileControlError('TMUX_FAILED')+ }++ /** The first Codex picker on the pane that `wanted` takes, polled until `timeoutMs`. */+ private async waitCodexPicker(+ session: RegisteredSession,+ wanted: (picker: CodexPicker) => boolean,+ timeoutMs: number,+ ): Promise<CodexPicker | null> {+ const capture = await this.waitPane(session.agentId, (value) => {+ const picker = parseCodexPicker(value)+ return !!picker && wanted(picker)+ }, timeoutMs)+ return capture ? parseCodexPicker(capture) : null+ }++ /** A model list that reads the same twice, CODEX_SETTLE_MS apart (see reachCodexEfforts). */+ private async settledCodexList(session: RegisteredSession): Promise<CodexPicker | null> {+ let previous: CodexPicker | null = null+ const deadline = Date.now() + PICKER_STEP_MS+ while (Date.now() < deadline) {+ const capture = await this.deps.capture(session.agentId, 100)+ const picker = capture ? parseCodexPicker(capture) : null+ const list = picker && isCodexList(picker) ? picker : null+ if (list && previous && sameCodexPicker(previous, list)) return list+ previous = list+ await sleep(CODEX_SETTLE_MS)+ }+ return null+ }++ /**+ * After a model's row: its reasoning picker, or 'closed' when the picker went away and the composer+ * is back, which is a model the catalog did not describe applying its one effort at once.+ */+ private async waitCodexAfterModelRow(session: RegisteredSession): Promise<CodexPicker | 'closed' | null> {+ const deadline = Date.now() + PICKER_STEP_MS+ while (Date.now() < deadline) {+ const capture = await this.deps.capture(session.agentId, 100)+ if (capture) {+ const picker = parseCodexPicker(capture)+ if (picker?.kind === 'efforts') return picker+ if (!picker && (await this.deps.readScreen(session, capture))?.pane.idle) return 'closed'+ }+ await sleep(100)+ }+ return null+ }++ private async setCursor(session: RegisteredSession, target: RuntimeProfile): Promise<void> {+ const cursorTarget = this.deps.manager.cursorTarget(session.sessionId, target.id)+ if (!cursorTarget) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')+ if (!await this.deps.sendText(session.agentId, `/model ${cursorTarget.familyLabel}`)) {+ throw new RuntimeProfileControlError('TMUX_FAILED')+ }+ let capture = await this.waitPane(session.agentId, (value) => !!parseCursorModelPicker(value), 900)+ if (!capture) {+ if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')+ capture = await this.waitPane(session.agentId, (value) => !!parseCursorModelPicker(value), PICKER_OPEN_MS)+ }+ const picker = capture ? parseCursorModelPicker(capture) : null+ if (!picker || picker.selectedFamily?.toLowerCase() !== cursorTarget.familyLabel.toLowerCase()) {+ throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')+ }++ if (cursorTarget.rawId !== 'auto') {+ if (!await this.deps.sendKey(session.agentId, 'Tab')) throw new RuntimeProfileControlError('TMUX_FAILED')+ capture = await this.waitPane(session.agentId, (value) => !!parseCursorParameterRows(value), PICKER_STEP_MS)+ if (!capture) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')++ if (cursorTarget.context) {+ await this.setCursorParameter(session, 'context', cursorTarget.context, true)+ }+ if (cursorTarget.reasoning) {+ await this.setCursorParameter(session, 'reasoning', cursorTarget.reasoning, true)+ }+ await this.setCursorParameter(session, 'thinking', 'true', cursorTarget.thinking ?? false)+ await this.setCursorParameter(session, 'fast', 'true', cursorTarget.fast ?? false)++ if (!await this.deps.sendKey(session.agentId, 'Escape')) throw new RuntimeProfileControlError('TMUX_FAILED')+ if (!await this.waitPane(session.agentId, (value) => !!parseCursorModelPicker(value), PICKER_STEP_MS)) {+ throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')+ }+ if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')+ } else {+ if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')+ }++ const confirmed = await this.waitPane(session.agentId, async (value) => {+ await this.deps.manager.ingestPane(session, value, true)+ return this.deps.manager.selectedModel(session) === target.id+ || this.cursorFooterMatches(value, cursorTarget, target.effort)+ }, COMMAND_CONFIRM_MS)+ if (!confirmed) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')+ if (this.deps.manager.selectedModel(session) !== target.id) {+ this.deps.manager.confirmControlProfile(target)+ }+ }++ private async setCursorParameter(+ session: RegisteredSession,+ kind: CursorParameterRow['kind'],+ value: string,+ selected: boolean,+ ): Promise<void> {+ const capture = await this.deps.capture(session.agentId, 100)+ const rows = capture ? parseCursorParameterRows(capture) : null+ if (!rows) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')+ const target = rows.find((row) => row.kind === kind && row.value === value)+ if (!target) {+ if (!selected && (kind === 'fast' || kind === 'thinking')) return+ throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')+ }+ if (target.selected === selected) return+ const current = rows.find((row) => row.cursor)+ if (!current) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')+ const direction = target.index > current.index ? 'Down' : 'Up'+ for (let i = 0; i < Math.abs(target.index - current.index); i++) {+ if (!await this.deps.sendKey(session.agentId, direction)) throw new RuntimeProfileControlError('TMUX_FAILED')+ await sleep(120)+ }+ if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')+ const updated = await this.waitPane(session.agentId, (next) => {+ const nextRows = parseCursorParameterRows(next)+ return nextRows?.some((row) => row.kind === kind && row.value === value && row.selected === selected) ?? false+ }, PICKER_STEP_MS)+ if (!updated) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')+ }++ private cursorFooterMatches(+ capture: string,+ target: CursorModelTarget,+ effort: string | null,+ ): boolean {+ const lines = stripAnsi(capture).split('\n').filter((line) => line.trim()).slice(-15)+ if (target.rawId === 'auto') return lines.some((line) => /^\s*Auto(?:\s*$|\s*[·│])/i.test(line))+ return lines.some((line) => {+ const normalized = line.trim().toLowerCase()+ if (!normalized.startsWith(target.familyLabel.toLowerCase())) return false+ const actualFast = /\bfast\b/i.test(line)+ const actualThinking = !/\bno\s+thinking\b/i.test(line) && /\bthinking\b/i.test(line)+ if (target.fast != null && actualFast !== target.fast) return false+ if (target.thinking != null && actualThinking !== target.thinking) return false+ if (target.context && !normalized.includes(target.context)) return false++ const effortMatch = /\b(extra high|none|low|medium|high|max)\b/i.exec(line)+ const actualEffort = effortMatch?.[1].toLowerCase().replace(/\s+/g, '') === 'extrahigh'+ ? 'xhigh'+ : effortMatch?.[1].toLowerCase() ?? null+ const footerEffort = target.footerEffort === undefined ? effort : target.footerEffort+ return footerEffort == null ? actualEffort == null : actualEffort === footerEffort+ })+ }++ private async waitPane(terminalTarget: string, predicate: (capture: string) => boolean | Promise<boolean>, timeoutMs: number): Promise<string | null> {+ const deadline = Date.now() + timeoutMs+ while (Date.now() < deadline) {+ const capture = await this.deps.capture(terminalTarget, 100)+ if (capture && await predicate(capture)) return capture+ await sleep(100)+ }+ return null+ }+}diff --git a/cli/src/lib/runtimeProfileController.ts b/cli/src/lib/runtimeProfileController.tsindex cd56a49db..73b0e72c8 100644--- a/cli/src/lib/runtimeProfileController.ts+++ b/cli/src/lib/runtimeProfileController.ts@@ -1,1442 +1,10 @@-import { parseRuntimeProfile } from './runtimeProfileWire.js'-import type { RegisteredSession } from './registry.js'-import { devinModelCommandResult } from '../engines/devin/runtimeProfile.js'-import { countCommandcodeRefusals } from '../engines/commandcode/runtimeProfile.js'-import { parseHermesPickerPage } from '../engines/hermes/runtimeProfile.js'-import { parsePiFooterProfile, parsePiThinkingSelection, piThinkingSteps } from '../engines/pi/runtimeProfile.js'-import {- countOpencodePickers,- opencodeRowMatches,- opencodeRowNamesModel,- parseOpencodePickerRows,- type OpencodeModelTarget,- type OpencodePickerRow,-} from '../engines/opencode/runtimeProfile.js'-import {- chooseCodexRow,- codexDigitPressable,- codexEffortRows,- parseCodexPicker,- sameCodexPicker,- type CodexCatalogModel,- type CodexPicker,-} from '../engines/codex/modelPicker.js'-import {- type CursorModelTarget,- type RuntimeProfile,- type RuntimeProfileManager,-} from './runtimeProfile.js'--const COMMAND_CONFIRM_MS = 8_000-const PICKER_OPEN_MS = 3_000-const PICKER_STEP_MS = 2_000-/**- * How far apart the two reads of a Codex model list are, which must agree before a digit is pressed on- * it (see reachCodexEfforts): long enough for a redraw in flight to land, short beside the seconds a- * switch takes to confirm.- */-const CODEX_SETTLE_MS = 250-/** Hard bound on ladder keystrokes — twice pi's seven levels, so a desynchronised walk still terminates. */-const PI_LADDER_MAX_STEPS = 14-/** Hermes' longest picker page is a provider's model list; twice its size still terminates. */-const HERMES_PAGE_MAX_STEPS = 60-/** "Anthropic (13 models)" names provider key `anthropic`; "GitHub Copilot (17 models)" names `copilot`. */-function hermesProviderMatches(row: string, provider: string): boolean {- const name = row.replace(/\s*\([^)]*\)\s*$/, '').trim().toLowerCase()- const key = provider.toLowerCase()- return name === key || name.replace(/[^a-z0-9]/g, '').includes(key.replace(/[^a-z0-9]/g, ''))-}--export type RuntimeProfileErrorCode =- | 'AGENT_NOT_FOUND'- | 'INVALID_RUNTIME_PROFILE'- | 'BUSY'- | 'UNSUPPORTED_CLI_VERSION'- | 'MODEL_UNAVAILABLE'- | 'EFFORT_UNSUPPORTED'- | 'PLAN_SCOPE_AMBIGUOUS'- | 'CONFIRM_TIMEOUT'- | 'TMUX_FAILED'--export class RuntimeProfileControlError extends Error {- constructor(readonly code: RuntimeProfileErrorCode) {- super(code)- }-}--export interface PaneInspection {- idle: boolean- plan: boolean- dialog: boolean- draft: boolean-}--export interface RuntimeProfileControllerDeps {- manager: Pick<RuntimeProfileManager, 'getState' | 'selectedModel' | 'codexCatalog' | 'modelsForSession'- | 'beginControl' | 'finishControl' | 'cancelControl' | 'confirmEffort' | 'waitForProfile' | 'waitForModel'- | 'devinTarget' | 'opencodeCatalog' | 'commandcodeTarget' | 'ingestConfig' | 'hermesTarget' | 'cursorTarget' | 'confirmControlProfile'> & {- supportsControl(session: RegisteredSession): boolean | Promise<boolean>- effortAllowed(session: RegisteredSession, model: string, effort: string, listed: readonly string[] | null): boolean | Promise<boolean>- ingestPane(session: RegisteredSession, text: string, silent?: boolean): boolean | Promise<boolean>- }- getSession: (sessionId: string) => RegisteredSession | undefined- validateRuntime: (session: RegisteredSession) => Promise<boolean>- capture: (terminalTarget: string, historyLines?: number) => Promise<string | null>- sendText: (terminalTarget: string, text: string) => Promise<boolean>- /** Type without submitting. */- sendLiteral: (terminalTarget: string, text: string) => Promise<boolean>- sendKey: (terminalTarget: string, key: string) => Promise<boolean>- acquireInput: (sessionId: string) => (() => void) | null-}--function stripAnsi(value: string): string {- return value.replace(/\u001b\[[0-9;:]*[A-Za-z]/g, '')-}--/**- * The SGR attributes an escape sequence actually sets, with extended-colour arguments removed.- *- * `38`/`48`/`58` introduce a colour whose ARGUMENTS are numbers, not attributes: `5;<n>` for 256 and- * `2;<r>;<g>;<b>` for truecolor. Without this skip a truecolor foreground (`38;2;…`) reads as SGR 2- * — dim — and a line the user had typed would be mistaken for an empty composer, which is the one- * direction this must never be wrong in: the pane would read idle and be respawned under a draft.- */-function sgrAttributes(params: readonly string[]): string[] {- const attributes: string[] = []- for (let index = 0; index < params.length; index += 1) {- const code = params[index]- if (code !== '38' && code !== '48' && code !== '58') {- attributes.push(code)- continue- }- const kind = params[index + 1]- index += kind === '5' ? 2 : kind === '2' ? 4 : 1- }- return attributes-}--/**- * The background an SGR sequence sets, as a comparable string — or null if it sets none.- *- * Needed because a composer box is identified by the colour it PAINTS, and the parameters that- * choose one are positional: `48;5;<n>` and `48;2;<r>;<g>;<b>` carry their arguments inline, so the- * arguments have to be consumed rather than read as further attributes. `38`/`58` are skipped the- * same way for the same reason — a truecolor FOREGROUND ends in numbers that would otherwise parse- * as a background code of their own.- *- * SGR 0 counts: a reset puts the default background back, which is a change like any other.- */-function sgrBackground(params: readonly string[]): string | null {- for (let index = 0; index < params.length; index += 1) {- const code = params[index]- if (code === '48') {- const kind = params[index + 1]- return params.slice(index, index + (kind === '5' ? 3 : kind === '2' ? 5 : 2)).join(';')- }- if (code === '49' || /^(?:4[0-7]|10[0-7])$/.test(code)) return code- if (code === '0' || code === '') return '49'- if (code === '38' || code === '58') {- const kind = params[index + 1]- index += kind === '5' ? 2 : kind === '2' ? 4 : 1- }- }- return null-}--/**- * The span of a gutter line the BOX ITSELF paints — everything after the marker was wrong.- *- * A TUI draws one row of the terminal at a time, so anything floating OVER the composer lands on the- * composer's own lines. OpenCode's `Getting started` card is exactly that: a panel on the right-hand- * side, and its rows are written after the composer's on the same lines. Reading to end-of-line took- * the card's text as something the user had typed —- *- * ┃ <121 spaces of composer> <sidebar> Connect provider /connect- *- * so `draft` was true over an EMPTY composer, the pane never read idle, and every model switch came- * back AGENT_BUSY. Measured on a live pane (`harness-opencode-*`, 2026-09-15) with nothing typed in.- * Fourth variant of the family above: the marker is found, the strip is dropped, and then the wrong- * COLUMNS are read.- *- * The boundary is the box's own background. Everything a box paints is painted in it, so the- * interior runs from the background set right after the marker to the next background change — in- * the capture above, back to the pane's `48;2;10;10;10` where the box ends and the sidebar begins.- *- * Two deliberate conservatisms, both pointing the same way — a real draft must never be read as an- * empty prompt, because that respawns the engine under text the user typed:- * * A background that appears only AFTER visible text is not the interior's. It is a highlight- * drawn mid-draft, and treating it as the boundary would cut the draft off at its own styling.- * * A line that sets no background at all is not clipped. Plain-text panes (and every engine that- * paints no box) keep the whole line, exactly as before.- */-function gutterBoxInterior(line: string, markerIndex: number): string {- const after = line.slice(markerIndex + 1)- const sgr = /\u001b\[([0-9;:]*)m/g- let interior: string | null = null- let end = after.length- let match: RegExpExecArray | null- while ((match = sgr.exec(after)) !== null) {- const background = sgrBackground(match[1].split(/[;:]/))- if (background === null) continue- if (interior === null) {- if (stripAnsi(after.slice(0, match.index)).trim().length > 0) break- interior = background- continue- }- if (background !== interior) { end = match.index; break }- }- return after.slice(0, end)-}--/**- * Is the text after the composer marker the PLACEHOLDER, rather than something the user typed?- *- * Read from the STYLING, never the words: the placeholder sentence differs per engine and changes- * between versions, but every one of these TUIs greys its placeholder out somehow and none of them- * styles a draft the same way.- *- * Two attributes, because two are in use. SGR 2 (dim) is what claude, codex and devin draw; SGR 3- * (italic) is what hermes draws — its `Ask anything, or type / for commands…` is italic over a- * 256-colour gold (`ESC[3m ESC[38;5;136m`) and carries no dim at all. Looking only for dim made- * EVERY hermes pane look like a pane with a draft in it, so `idle` was false for the agent's whole- * life and every retarget and model switch came back AGENT_BUSY over an engine sitting at an empty- * prompt. Same failure the comment on `promptMarker` warns about, one step further along: the- * marker was found, and then the placeholder was not recognised as one.- */-function hasPlaceholderSgr(value: string): boolean {- return [...value.matchAll(/\u001b\[([0-9;:]*)m/g)]- .some((match) => {- const attributes = sgrAttributes(match[1].split(/[;:]/))- return attributes.includes('2') || attributes.includes('3')- })-}--/**- * Is this SGR foreground a MUTED one — the grey a TUI writes hints and placeholders in?- *- * Returns null when the sequence sets no foreground, so a caller can keep the colour already in- * effect rather than treating "no change" as a change.- *- * Grey is judged by the colour itself: equal channels, and dark enough to read as secondary next to- * the near-white a composer draws real text in. OpenCode's placeholder is `38;2;128;128;128` and its- * own text `38;2;255;255;255`; the 256-colour ramp (232–255) and SGR 90 are the same intent spelled- * differently.- */-function sgrMutedForeground(params: readonly string[]): boolean | null {- for (let index = 0; index < params.length; index += 1) {- const code = params[index]- if (code === '38') {- const kind = params[index + 1]- if (kind === '2') {- const [r, g, b] = [params[index + 2], params[index + 3], params[index + 4]].map(Number)- if (![r, g, b].every(Number.isFinite)) return null- return r === g && g === b && r >= 0x40 && r <= 0xb0- }- if (kind === '5') {- const n = Number(params[index + 2])- if (!Number.isFinite(n)) return null- // 8 is bright black; 232–255 is the grey ramp, whose middle is the secondary tone.- return n === 8 || (n >= 236 && n <= 250)- }- return null- }- if (code === '90') return true- if (code === '39' || code === '0' || code === '') return false- if (/^(?:3[0-7]|9[1-7])$/.test(code)) return false- if (code === '48' || code === '58') {- const kind = params[index + 1]- index += kind === '5' ? 2 : kind === '2' ? 4 : 1- }- }- return null-}--/**- * Is EVERY visible character here written in a muted colour — i.e. is this a hint rather than a draft?- *- * The third placeholder styling this module has had to learn, after dim (claude, codex, devin) and- * italic (hermes). OpenCode draws `Ask anything… "What is the tech stack of this project?"` as a plain- * TRUECOLOR GREY with no dim and no italic attribute at all, so [hasPlaceholderSgr] saw nothing and a- * brand-new pane read as a pane with a draft in it: never idle, and every model switch on an agent- * with no messages yet refused as AGENT_BUSY over an empty composer. Measured on 1.18.31.- *- * ⚠️ EVERY character, not any — which is what keeps this from swallowing a real draft. A composer- * writes what the user typed in its normal near-white; one such character anywhere means this is- * text, whatever grey hint may be sitting beside it. Spaces are not evidence of either and carry no- * colour worth reading, so they are skipped.- */-function allVisibleTextIsMuted(value: string): boolean {- const sgr = /\u001b\[([0-9;:]*)m/g- let muted: boolean | null = null- let cursor = 0- let sawText = false- let match: RegExpExecArray | null- const segment = (text: string): boolean => {- if (!text.trim()) return true- sawText = true- return muted === true- }- while ((match = sgr.exec(value)) !== null) {- if (!segment(value.slice(cursor, match.index))) return false- const next = sgrMutedForeground(match[1].split(/[;:]/))- if (next !== null) muted = next- cursor = match.index + match[0].length- }- if (!segment(value.slice(cursor))) return false- return sawText-}--/**- * The glyph each CLI puts in front of its composer. They do not share one: cursor uses `→`, devin `❭`- * (U+276D — close to, but not, claude's `❯` U+276F), and pi draws no marker at all. Getting this wrong is- * not cosmetic — with no prompt found the pane never reads idle, and every switch is refused as BUSY.- */-function promptMarker(engine: RegisteredSession['engine']): RegExp {- if (engine === 'cursor') return /→/u- if (engine === 'devin') return /[❭›❯]/u- if (engine === 'opencode') return /┃/u- return /[›❯]/u-}--export function inspectRuntimePane(engine: RegisteredSession['engine'], capture: string): PaneInspection {- if (engine === 'pi') return inspectPiPane(capture)- if (engine === 'opencode') return inspectGutterBoxPane(capture, true)- if (engine === 'copilot') return inspectGutterBoxPane(capture, false)- if (engine === 'grok') return inspectGrokPane(capture)- const rawLines = capture.split('\n')- const marks = promptMarker(engine)- const promptIndex = latestPromptLine(rawLines, marks)- const prompt = promptIndex >= 0 ? rawLines[promptIndex] : ''- const currentUi = currentPaneUi(rawLines, promptIndex)- const dialog = DIALOG_UI.test(currentUi) || takeoverIn(engine, capture, currentUi) !== null- const plan = engine === 'codex' ? /\bplan mode\b/i.test(currentUi) : /\bplan mode on\b/i.test(currentUi)- const marker = stripAnsi(prompt).search(marks)- let visible = marker >= 0 ? stripAnsi(prompt).slice(marker + 1).replace(/\u00a0/g, ' ').trim() : ''- const rawMarker = prompt.search(marks)- const rawTail = rawMarker >= 0 ? prompt.slice(rawMarker + 1) : ''- const placeholder = hasPlaceholderSgr(rawTail)- if (placeholder) visible = ''- const draft = visible.length > 0- return { idle: !!prompt && !dialog && !draft, plan, dialog, draft }-}--/** The latest composer prompt line; picker rows use the same glyphs, but numbered rows are not prompts. */-function latestPromptLine(rawLines: string[], marks: RegExp): number {- return rawLines.findLastIndex((line) => {- const visible = stripAnsi(line)- const marker = visible.search(marks)- return marker >= 0 && !/^\s*\d+\.\s/.test(visible.slice(marker + 1))- })-}--/**- * Old picker/plan text can remain in tmux history. Only UI below the latest prompt belongs to the- * current interaction; if no prompt is visible, inspect the whole capture as a conservative fallback.- */-function currentPaneUi(rawLines: string[], promptIndex: number): string {- return stripAnsi((promptIndex >= 0 ? rawLines.slice(promptIndex) : rawLines).join('\n')).replace(/\u00a0/g, ' ')-}--/** A screen of Claude Code's or Codex's own that takes its composer's place (see [takeoverIn]). */-export type PaneTakeover = 'rewind' | 'transcript' | 'search' | 'trust' | 'update' | 'model' | 'sign_in'--/**- * The modal over a Claude Code or Codex pane, read for a message about to be typed into it- * (messageHold.ts): one of their own screens that takes the composer's place ([takeoverIn]), an- * approval prompt, or another menu. Every one of them is a dialog to [inspectRuntimePane], except the- * MCP boot notice: Claude Code takes typing while it shows, so a message sent the moment an agent- * starts still goes in.- */-export function paneModal(engine: RegisteredSession['engine'], capture: string): PaneTakeover | 'permission' | 'menu' | null {- const rawLines = capture.split('\n')- const currentUi = currentPaneUi(rawLines, latestPromptLine(rawLines, promptMarker(engine)))- const takeover = takeoverIn(engine, capture, currentUi)- if (takeover) return takeover- // Below the prompt line: that line is the composer, or the echo of the last message, and either can- // hold anything a person typed.- if (PERMISSION_UI.test(currentUi.slice(currentUi.indexOf('\n') + 1))) return 'permission'- return DIALOG_UI.test(currentUi.replace(/Starting MCP servers?/gi, '')) ? 'menu' : null-}--/**- * The screens of Claude Code 2.1.289 and Codex 0.160 that take the composer's place, where a pasted- * message and its Enter do something other than send it (read from their code, not run; the screens- * are in __fixtures__/rewindPickers.ts and takeoverScreens.ts):- * - rewind: a picker for a point in the conversation to go back to, whose Enter picks one;- * - transcript: Claude Code's transcript view (ctrl+o) and Codex's transcript overlay (ctrl+t, in its- * scrollback mode), which drop the paste and have no Enter: the message is lost;- * - search: a search through the prompt history (ctrl+r), or Codex's find in its transcript (F3),- * which takes the paste as what to search for; Claude Code's Enter then SENDS the earlier prompt it- * found, Codex's puts it in the composer or goes to the next match;- * - trust, update, model, sign_in: the questions Codex asks at startup (trust this folder, update- * now, switch to a new model, how to sign in) and Claude Code's trust and sign-in screens, which- * drop the paste and take the Enter as the highlighted answer: trust the folder (Claude Code's- * highlights `No, exit`, so it quits), run the update, switch the model, start a sign-in.- * Each is matched on its own wording and rows, not on a guess at what is missing from the screen.- */-function takeoverIn(engine: RegisteredSession['engine'], capture: string, currentUi: string): PaneTakeover | null {- if (engine !== 'claude' && engine !== 'codex') return null- const lines = stripAnsi(capture).replace(/\u00a0/g, ' ').split('\n')- const bottom = lines.filter((line) => line.trim()).slice(-3)- // A startup screen is the last thing on the pane: one with a composer below it was answered.- const composer = engine === 'codex'- ? lines.findLastIndex((line) => /^\s*›/.test(line) && !/^\s*›\s*\d+\.\s/.test(line))- : lines.findLastIndex((line, index) => /^\s*❯/.test(line) && index > 0 && /^\s*[─━]{8,}\s*$/.test(lines[index - 1]))- const shown = (...patterns: RegExp[]) => patterns.every((pattern) => lines.findLastIndex((line) => pattern.test(line)) > composer)- if (engine === 'codex') {- if (CODEX_TRANSCRIPT_BROWSING.test(currentUi)) return 'rewind'- if (CODEX_PAGER_HEADER.test(lines.find((line) => line.trim()) ?? '')) return 'transcript'- if (bottom.some((line) => CODEX_SEARCH_FOOTER.test(line))) return 'search'- if (shown(CODEX_TRUST_QUESTION, CODEX_TRUST_ROW)) return 'trust'- if (shown(/^\s*Update available\b/, CODEX_UPDATE_ROW)) return 'update'- if (shown(/^\s*[›>]?\s*1\. Try new model\s*$/, /^\s*[›>]?\s*2\. Use existing model\s*$/)) return 'model'- if (shown(/^\s*[›>]?\s*1\. Sign in with ChatGPT\s*$/)) return 'sign_in'- return null- }- if (claudeRewindMenuOpen(capture)) return 'rewind'- if (bottom.some((line) => CLAUDE_TRANSCRIPT_FOOTER.test(line))) return 'transcript'- if (bottom.some((line) => CLAUDE_HISTORY_SEARCH.test(line))) return 'search'- if (shown(/Quick safety check: Is this a project you created or one you trust\?/, /^\s*(?:❯\s*)?Yes, I trust this folder\s*$/)) return 'trust'- if (shown(/^\s*Select login method:\s*$/, /^\s*(?:❯\s*)?1\. Claude account with subscription\b/)) return 'sign_in'- return null-}--/** The header of Codex's pager over the whole pane, its top row: its transcript (ctrl+t) when it is not- * on the composer's screen (pager_overlay/transcript.rs). Closed with q or ctrl+t; Esc browses prompts. */-const CODEX_PAGER_HEADER = /^\/ T R A N S C R I P T(?: \/)*\s*$/--/** Codex's search footers: through its prompt history (`reverse-i-search: … enter accept · esc cancel`,- * chat_composer/history_search.rs) and through its transcript (`Find: …`, transcript_view/search.rs). */-const CODEX_SEARCH_FOOTER = /^\s*(?:reverse-i-search:|Find: )/--/** Codex's trust question, as 0.160 words it (onboarding/trust_directory.rs) and as 0.147 did. */-const CODEX_TRUST_QUESTION = /Trust this folder\? Codex can read, edit, and run files here|Do you trust the contents of this directory\?/-const CODEX_TRUST_ROW = /^\s*[›>]?\s*1\. (?:Trust and continue|Yes, continue|Open restricted|Open existing task)\s*$/--/** Codex's update prompt (update_prompt.rs): its first row runs the update. */-const CODEX_UPDATE_ROW = /^\s*[›>]?\s*1\. Update now \(runs /--/**- * Claude Code's transcript view (ctrl+o): the prompt is hidden, and the footer row starts- * `Showing detailed transcript · ctrl+o to toggle`, after `dialog waiting · ` when a dialog sits behind- * it (2.1.289). It has no Enter, so a message typed there is lost. Esc, q or ctrl+c close it.- */-const CLAUDE_TRANSCRIPT_FOOTER = /^\s*(?:dialog waiting · )?Showing detailed transcript\b/--/** Claude Code's prompt-history search (ctrl+r), under the prompt: `search prompts: <query>`, or- * `no matching prompt: <query>` (2.1.289). */-const CLAUDE_HISTORY_SEARCH = /^\s*(?:search prompts|no matching prompt): /--/**- * A modal drawn over the pane — a picker, a permission prompt, an MCP boot notice.- *- * One copy, shared by every reader here: an engine whose composer needs its own function still meets- * the same dialogs, and two lists of these would drift the first time either gained an entry.- * No `g` flag, so `test` carries no `lastIndex` between callers.- */-const DIALOG_UI = /Select Model(?: and Effort)?|Select Reasoning Level|Advanced Reasoning|Available models|Models matching|Edit Parameters|Type to filter.*Tab to edit|Esc to go back|Press enter to confirm|Do you want to proceed|Allow this action|permission required|Starting MCP servers?/i--/** The approval prompts among them. */-/**- * An approval prompt's question. Claude Code asks "Do you want to proceed?", "Do you want to make this- * edit to <file>?", "Do you want to allow Claude to fetch this content?", "Do you want to <verb> <target>?"- * for the other tools, and "Would you like to proceed?" over a plan (2.1.289); Codex asks "Would you like- * to run the following command?" and the like. The rows under it are read too (askQuestion.ts), but a- * row wrapped in a narrow pane breaks that reading, and the question alone must still hold a message:- * Enter approves the first row. Asked only below the prompt line, so never of what a person typed.- */-const PERMISSION_UI = /\bDo you want to\b|\bWould you like to (?:proceed|run|make|apply|allow)\b|Allow this action|permission required/i--/**- * Codex browsing its own transcript: Esc twice on an empty composer, and its footer row reads- * `Browsing transcript · ↑↓/jk scroll · ←→/hl prompts · … · ↵ rewind · esc back`, down to a bare- * `Browsing` as the pane narrows (0.160, tui/src/app_backtrack/prompt_navigation.rs). In its default- * fullscreen mode the composer stays, dimmed whole, placeholder and any draft in it alike, so it read- * as empty and idle; in its scrollback mode (`tui.fullscreen_transcript = false`, no alternate screen,- * or over SSH) the footer closes a transcript pager drawn over the whole pane. Either way Enter- * reverts the conversation to the prompt in view. A paste leaves the fullscreen browser and lands in- * the composer, but the scrollback one drops it, and its Enter then rewinds (app.rs and- * pager_overlay/transcript.rs).- */-const CODEX_TRANSCRIPT_BROWSING = /^\s*Browsing(?: transcript)?(?:\s+·|\s*$)/m--/**- * Claude Code's Rewind menu (Esc twice on an empty prompt, or `/rewind`), which hides the prompt while- * it is open. Under its title, `Rewind`, it lists the messages sent so far, `(current)` focused, behind- * one of these lines, or asks to confirm one picked (2.1.289, MessageSelector). Enter on a message asks- * to confirm it, and Enter again restores the conversation, the code or both to before it (with file- * checkpoints off, the first Enter restores the conversation); Enter on `(current)` closes the menu.- * A message pasted into it is dropped either way. Its focused row is drawn `❯ (current)` in italics,- * which read as an empty, idle prompt.- */-// Their opening words only: a pane narrower than a line wraps the rest onto the next, and the menu went-// unseen, its Enter picking a message (with checkpoints off, rewinding to it).-const CLAUDE_REWIND_BODY = /^(?:Restore the code\b|Restore and fork\b|Confirm you want to restore\b|Nothing to rewind to\b)/--function claudeRewindMenuOpen(capture: string): boolean {- const lines = stripAnsi(capture).replace(/\u00a0/g, ' ').split('\n').map((line) => line.trim())- // The title, with its body a blank line under it, and no rule after it: the prompt the menu hides- // comes back between two rules once it closes.- return lines.some((line, index) => line === 'Rewind'- && lines.slice(index + 1, index + 4).some((next) => CLAUDE_REWIND_BODY.test(next))- && !lines.slice(index + 1).some((next) => /[─━]{8,}/u.test(next)))-}--/** The rule a gutter-box composer is closed with: `╹▀▀▀▀…`. */-const GUTTER_BOX_RULE = /[─▀▁▔]{8,}/u--/**- * A composer drawn as a BOX with a `┃` gutter — OpenCode's and Copilot's, which are the same shape.- *- * The generic reader cannot read either one. OpenCode's `┃` does not stop at the text: the last- * gutter line is the box's own STATUS strip — `Build · <model> · <account>`, sitting directly on- * the rule that closes the box — so taking the last line carrying the marker reads what the BOX says- * about itself as something the user typed. Copilot fails one step earlier: its composer carries no- * `›`/`❯` at all, while every message the user has ALREADY SENT is echoed into the transcript as- * `❯ <text>` in full brightness, so the generic reader walks back to the last of those and reads a- * sent message as a draft. Either way the pane was permanently "holding a draft": never idle, and- * every retarget and model switch refused as AGENT_BUSY over an empty composer. Third variant of the- * failure hermes had — there the placeholder was not recognised, here the wrong line is read.- *- * [hasStatusStrip] is the one thing that differs between the two, and it is OpenCode's alone. The- * strip is identified by WHERE it is — the gutter line the closing rule sits under — rather than by- * what it says, which is three variable fields that change with the model, the mode and the account.- * When no rule follows, no strip is assumed and every gutter line counts as composer: a layout we do- * not recognise must not silently swallow a line the user typed in. Copilot's box has no strip, so- * its single gutter line IS the composer and dropping it would read a real draft as an empty prompt- * — the one direction this must never be wrong in.- *- * Every gutter line is checked, not just the last: a long message wraps down the box, and reading- * one line of it would call a full composer empty.- *- * A dialog needs no special case in either engine: both REPLACE the composer with the picker or the- * permission prompt, so no gutter is on screen at all and the pane reads not-idle from the empty run- * below rather than from [DIALOG_UI].- */-function inspectGutterBoxPane(capture: string, hasStatusStrip: boolean): PaneInspection {- const rawLines = capture.split('\n')- const marks = /┃/u- // The LAST contiguous run of gutter lines. Earlier runs are previous composers still in scrollback- // — the message the user already sent is not a draft.- const gutter: number[] = []- for (let index = rawLines.length - 1; index >= 0; index -= 1) {- if (marks.test(stripAnsi(rawLines[index]))) { gutter.unshift(index); continue }- if (gutter.length > 0) break- }- if (gutter.length === 0) return { idle: false, plan: false, dialog: false, draft: false }- const last = gutter[gutter.length - 1]- const closed = last + 1 < rawLines.length && GUTTER_BOX_RULE.test(stripAnsi(rawLines[last + 1]))- const composer = hasStatusStrip && closed ? gutter.slice(0, -1) : gutter- const currentUi = stripAnsi(rawLines.slice(gutter[0]).join('\n')).replace(/\u00a0/g, ' ')- const dialog = DIALOG_UI.test(currentUi)- const draft = composer.some((index) => {- const line = rawLines[index]- const visibleMarker = stripAnsi(line).search(marks)- if (visibleMarker < 0) return false- const rawMarker = line.search(marks)- // The box's interior, not the rest of the row: a panel floating over the composer writes into- // these same lines, and its text is not a draft. See [gutterBoxInterior]. With no marker in the- // RAW line there is no interior to bound — read the whole row, the direction that keeps a draft.- const interior = rawMarker < 0- ? stripAnsi(line).slice(visibleMarker + 1)- : gutterBoxInterior(line, rawMarker)- if (hasPlaceholderSgr(interior) || allVisibleTextIsMuted(interior)) return false- return stripAnsi(interior).replace(/\u00a0/g, ' ').trim().length > 0- })- return { idle: !dialog && !draft, plan: /\bplan mode on\b/i.test(currentUi), dialog, draft }-}--/**- * The context readout Pi's footer always carries — `0.0%/500k`, `1.5%/200k`. Present whether or not the- * model advertises a thinking ladder, and drawn by the normal view alone, which is what makes it the- * honest "the footer is on screen" signal. No `g` flag, so `test` carries no `lastIndex` between calls.- */-const PI_FOOTER_BUDGET = /\d+(?:\.\d+)?%\s*\/\s*\S+/--/**- * Grok draws its composer as a ROUNDED BOX, and the generic reader counts the box's own right edge- * as something the user typed.- *- * Its empty composer is one line — `│ ❯` … padding … `│` — inside `╭─…─╮` above and- * `╰─… Grok 4.6 (xhigh) ─╯` below. The shared reader takes everything after `❯`, strips ANSI, trims,- * and finds `│`: one character, non-empty, so `draft` was true. Measured on a real pane- * (`harness-grok-*`, 2026-09-09) with nothing typed into it: `visible after marker = "│"`, `draft =- * true`, `idle = false`. Not a transient — the edge is drawn on every frame, so idle was false for- * the agent's whole life and EVERY model switch came back AGENT_BUSY over an empty prompt. Same- * family as the hermes and gutter-box bugs above; here the culprit is the box, not the placeholder.- *- * So the composer is the box's INTERIOR: the run of `│` body lines, read between their first and- * last `│` rather than from the marker to end-of-line. Every body line is checked, not just the one- * carrying `❯` — a long message wraps down the box and reading one line of it would call a full- * composer empty.- *- * ⚠️ `│` is also Grok's footer separator (`Shift+Tab:mode │ Ctrl+x:shortcuts`), which sits BELOW the- * box. It is excluded by requiring a body line to both open and close with `│`, which the footer —- * one separator, no edges — never does.- */-function inspectGrokPane(capture: string): PaneInspection {- const rawLines = capture.split('\n')- const visibleLines = rawLines.map((line) => stripAnsi(line).replace(/ /g, ' '))- // The LAST box in the capture. Earlier ones are composers already sent, still in scrollback.- const bottom = visibleLines.findLastIndex((line) => /^\s*╰[─╌]*.*╯\s*$/u.test(line))- const top = bottom < 0- ? -1- : visibleLines.slice(0, bottom).findLastIndex((line) => /^\s*╭[─╌]+╮\s*$/u.test(line))- // No recognisable box is "we cannot tell", which the caller reads as busy — the safe direction.- // Respawning a pane whose layout we do not understand could discard a draft.- if (top < 0 || bottom - top < 2) return { idle: false, plan: false, dialog: false, draft: false }- const currentUi = visibleLines.slice(top).join('\n')- const dialog = DIALOG_UI.test(currentUi)- const draft = rawLines.slice(top + 1, bottom).some((line, offset) => {- const visible = visibleLines[top + 1 + offset]- const first = visible.indexOf('│')- const last = visible.lastIndexOf('│')- // Both edges, and not the same character: anything else is not a body line of this box.- if (first < 0 || last <= first) return false- const rawMarker = line.search(/[›❯]/u)- if (rawMarker >= 0 && hasPlaceholderSgr(line.slice(rawMarker + 1))) return false- // The marker is chrome, not content — it is printed whether or not anything was typed.- return visible.slice(first + 1, last).replace(/[›❯]/u, '').trim().length > 0- })- return { idle: !dialog && !draft, plan: /\bplan mode on\b/i.test(currentUi), dialog, draft }-}--/**- * Pi draws no composer marker, so idleness is read from its layout instead: the composer is the band- * between the last two `───` rules, and the footer (`minimax/minimax-m3 • high`) is only rendered by the- * normal view. A non-empty band is a draft the user is still typing — injecting there would splice a- * slash command into their sentence.- */-function inspectPiPane(capture: string): PaneInspection {- const lines = stripAnsi(capture).split('\n').map((line) => line.replace(/\s+$/, ''))- const text = lines.join('\n')- // Whether the footer is DRAWN, which is not the same question as whether its profile can be READ —- // and conflating the two is what has broken this check twice.- //- // `parsePiFooterProfile` is strict on purpose: it must return a model AND an effort or nothing, so a- // half-drawn redraw is never mistaken for a profile change. Idleness needs far less — only that the- // normal view is on screen rather than a picker — and borrowing the strict reader for it meant every- // footer shape it could not fully parse read as "still busy".- //- // The shape it could not parse: a model with no thinking ladder draws no `• <level>` at all, so the- // line ends at the model name. EVERY grid model is that shape, because the provider block written in- // `gridLaunch.ts` declares `reasoning: false` — which left every Pi agent on a grid permanently- // AGENT_BUSY, refusing model switches over an idle pane (measured, pi 0.82, footer- // `↑5.4k ↓292 1.5%/200k (auto) Auto`).- //- // So: the profile still counts when it parses, and the token-budget readout — `0.0%/500k`,- // `1.5%/200k`, present in both shapes and drawn by no picker — carries the rest.- const footer = parsePiFooterProfile(text) !== null || PI_FOOTER_BUDGET.test(text)- const rules: number[] = []- lines.forEach((line, index) => { if (/^\s*─{8,}\s*$/.test(line)) rules.push(index) })- const dialog = /Thinking Level|Select reasoning depth|Type to search|Enter to select|Only showing models from/i- .test(lines.join('\n'))- if (rules.length < 2) return { idle: false, plan: false, dialog, draft: false }- const band = lines.slice(rules[rules.length - 2] + 1, rules[rules.length - 1]).join('').trim()- const draft = band.length > 0- return { idle: footer && !dialog && !draft, plan: false, dialog, draft }-}--export interface CursorParameterRow {- kind: 'context' | 'reasoning' | 'fast' | 'thinking'- value: string- selected: boolean- cursor: boolean- index: number-}--export function parseCursorModelPicker(capture: string): { selectedFamily: string | null } | null {- const lines = stripAnsi(capture).split('\n')- if (!lines.some((line) => /Available models|Models matching/i.test(line)) || !lines.some((line) => /Tab to edit/i.test(line))) {- return null- }- const selected = lines.findLast((line) => /^\s*→\s+/.test(line))- if (!selected) return { selectedFamily: null }- const value = selected.replace(/^\s*→\s+/, '').replace(/\s+\(Tab to modify\)\s*$/i, '')- return { selectedFamily: value.split(/\s{2,}/)[0]?.trim() || null }-}--export function parseCursorParameterRows(capture: string): CursorParameterRow[] | null {- const lines = stripAnsi(capture).split('\n')- if (!lines.some((line) => /— Edit Parameters/i.test(line))) return null- const rows: CursorParameterRow[] = []- let section: 'context' | 'reasoning' | null = null- for (const raw of lines) {- const line = raw.replace(/\u00a0/g, ' ')- if (/^\s*Context\s*$/.test(line)) { section = 'context'; continue }- if (/^\s*Reasoning\s*$/.test(line)) { section = 'reasoning'; continue }- const choice = /^\s*(→)?\s*([●○◉◯])\s+(.+?)(?:\s+✓)?\s*$/.exec(line)- if (!choice) continue- const label = choice[3].trim()- let kind: CursorParameterRow['kind']- let value: string- if (/^Fast$/i.test(label)) {- kind = 'fast'- value = 'true'- } else if (/^Thinking$/i.test(label)) {- kind = 'thinking'- value = 'true'- } else if (section === 'context') {- kind = 'context'- value = label.toLowerCase()- } else if (section === 'reasoning') {- kind = 'reasoning'- value = label.toLowerCase().replace(/\s+/g, '') === 'extrahigh'- ? 'xhigh'- : label.toLowerCase()- } else {- continue- }- rows.push({- kind,- value,- selected: choice[2] === '●' || choice[2] === '◉' || /✓\s*$/.test(line),- cursor: !!choice[1],- index: rows.length,- })- }- return rows.length ? rows : null-}--/** The lists a model is chosen from, as against the reasoning screens that follow them. */-function isCodexList(picker: CodexPicker): boolean {- return picker.kind === 'quick' || picker.kind === 'models' || picker.kind === 'reserve'-}--function sleep(ms: number): Promise<void> {- return new Promise((resolve) => setTimeout(resolve, ms))-}--export class RuntimeProfileController {- constructor(private readonly deps: RuntimeProfileControllerDeps) {}-- async setProfile(sessionId: string, encoded: unknown): Promise<void> {- const registeredSession = this.deps.getSession(sessionId)- if (!registeredSession) throw new RuntimeProfileControlError('AGENT_NOT_FOUND')- const observed = this.deps.manager.getState(registeredSession.sessionId)- const session: RegisteredSession = registeredSession.cliVersion || !observed.cliVersion- ? registeredSession- : { ...registeredSession, cliVersion: observed.cliVersion }- const target = parseRuntimeProfile(encoded)- // Either id identifies the agent: the encoded profile now carries the agent id, but one minted- // before the id cutover (or by a client that still knows the session) must keep working.- if (!target || (target.sessionId !== session.agentId && target.sessionId !== session.sessionId) || target.engine !== session.engine) {- throw new RuntimeProfileControlError('INVALID_RUNTIME_PROFILE')- }- const current = parseRuntimeProfile(this.deps.manager.selectedModel(session))- if (current?.id === target.id) return- if (!await this.deps.manager.supportsControl(session)) {- console.warn(`[runtime-profile] unsupported ${session.engine} CLI version ${session.cliVersion ?? 'unknown'} for ${sessionId.slice(0, 8)}`)- throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')- }- // Codex's own catalog: which efforts each model takes, and the names its picker shows them by.- const codexCatalog = session.engine === 'codex' ? await this.deps.manager.codexCatalog(session) : []- if (session.engine === 'codex') {- const listed = codexCatalog.find((entry) => entry.slug === target.model)?.efforts ?? null- if (!await this.deps.manager.effortAllowed(session, target.model, target.effort, listed)) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')- }- const options = await this.deps.manager.modelsForSession(session)- if (!options.some((option) => option.id === target.id)) {- const sameModel = options.some((option) => parseRuntimeProfile(option.id)?.model === target.model)- throw new RuntimeProfileControlError(sameModel ? 'EFFORT_UNSUPPORTED' : 'MODEL_UNAVAILABLE')- }- if (session.engine === 'codex' && observed.mode === 'plan') throw new RuntimeProfileControlError('PLAN_SCOPE_AMBIGUOUS')- const release = this.deps.acquireInput(session.agentId)- if (!release) throw new RuntimeProfileControlError('BUSY')- let controlStarted = false- let pickerOpen = false- try {- if (!await this.deps.validateRuntime(session)) throw new RuntimeProfileControlError('TMUX_FAILED')- const capture = await this.deps.capture(session.agentId, 100)- if (!capture) throw new RuntimeProfileControlError('TMUX_FAILED')- const inspection = inspectRuntimePane(session.engine, capture)- if (session.engine === 'codex' && inspection.plan) throw new RuntimeProfileControlError('PLAN_SCOPE_AMBIGUOUS')- if (!inspection.idle) throw new RuntimeProfileControlError('BUSY')- if (!this.deps.manager.beginControl(session, target)) throw new RuntimeProfileControlError('BUSY')- controlStarted = true- if (session.engine === 'claude') {- await this.setClaude(session, target, current, options)- } else if (session.engine === 'codex') {- pickerOpen = true- await this.setCodex(session, target, codexCatalog)- pickerOpen = false- } else if (session.engine === 'devin') {- // No picker is ever opened — `/model <id>` is a single command — so there is nothing to Escape out- // of if this throws.- await this.setDevin(session, target)- } else if (session.engine === 'pi') {- pickerOpen = true- await this.setPi(session, target, current)- pickerOpen = false- } else if (session.engine === 'opencode') {- pickerOpen = true- await this.setOpencode(session, target)- pickerOpen = false- } else if (session.engine === 'hermes') {- pickerOpen = true- await this.setHermes(session, target)- pickerOpen = false- } else if (session.engine === 'commandcode') {- // Two plain commands, no dialog — nothing to Escape out of if either throws.- await this.setCommandcode(session, target, current)- } else {- pickerOpen = true- await this.setCursor(session, target)- pickerOpen = false- }- this.deps.manager.finishControl(session)- controlStarted = false- } catch (error) {- if (pickerOpen) {- for (let attempt = 0; attempt < 3; attempt++) {- await this.deps.sendKey(session.agentId, 'Escape').catch(() => false)- await sleep(100)- const next = await this.deps.capture(session.agentId, 100).catch(() => null)- if (!next || !inspectRuntimePane(session.engine, next).dialog) break- }- }- // By the engine session, as beginControl keyed it. `sessionId` is whichever id the request named,- // and the apps name the agent: cancelling under that left the control in place, and every later- // switch of the agent answered BUSY until the daemon restarted (found by e2e/models.e2e.ts, a- // refused switch followed by one that should have gone through).- if (controlStarted) this.deps.manager.cancelControl(session.sessionId)- // A failed switch used to leave NOTHING in the log — the picker just flashed open and shut on the- // user's terminal and the device said nothing useful. Name the session, engine and reason.- const code = error instanceof RuntimeProfileControlError ? error.code : 'TMUX_FAILED'- console.warn(`[runtime-profile] ${sessionId.slice(0, 8)} ${session.engine} set ${target.model}@${target.effort} failed: ${code}`)- if (error instanceof RuntimeProfileControlError) throw error- throw new RuntimeProfileControlError('TMUX_FAILED')- } finally {- release()- }- }-- private async setClaude(- session: RegisteredSession,- target: RuntimeProfile,- current: RuntimeProfile | null,- options: Array<{ id: string }>,- ): Promise<void> {- if (current?.model !== target.model) {- if (!await this.deps.sendText(session.agentId, `/model ${target.model}`)) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- if (!await this.waitClaudeModel(session)) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')- }-- if (current?.effort === target.effort) {- this.deps.manager.confirmEffort(session.sessionId, target.effort)- return- }-- const hasExplicitEffort = options.some((option) => {- const profile = parseRuntimeProfile(option.id)- return profile?.model === target.model && profile.effort !== 'auto'- })- if (!hasExplicitEffort && target.effort === 'auto') {- this.deps.manager.confirmEffort(session.sessionId, 'auto')- return- }- if (!await this.deps.sendText(session.agentId, `/effort ${target.effort}`)) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- if (!await this.deps.manager.waitForProfile(session.sessionId, COMMAND_CONFIRM_MS)) {- await this.deps.sendKey(session.agentId, 'Enter')- if (!await this.deps.manager.waitForProfile(session.sessionId, 2_000)) {- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')- }- }- }-- private async waitClaudeModel(session: RegisteredSession): Promise<boolean> {- if (await this.deps.manager.waitForModel(session.sessionId, 1_200)) return true- const capture = await this.deps.capture(session.agentId, 80)- if (capture && /switch model|change model|continue.*model|re-read.*history/i.test(stripAnsi(capture))) {- await this.deps.sendKey(session.agentId, 'Enter')- } else {- // Claude and Codex can swallow the first Enter immediately after a terminal literal write.- await this.deps.sendKey(session.agentId, 'Enter')- }- return this.deps.manager.waitForModel(session.sessionId, COMMAND_CONFIRM_MS)- }-- /**- * Devin takes the whole profile in one command: `/model <id>` accepts the exact id from- * `devin models list` (effort included), and answers in the pane — `✓ Model set to <name>` or- * `✗ Model not available`. That explicit answer is why devin needs no picker driving at all, and why an- * unavailable model (most of them, on a free plan) fails fast instead of timing out.- */- private async setDevin(session: RegisteredSession, target: RuntimeProfile): Promise<void> {- const devin = this.deps.manager.devinTarget(session.sessionId, target.id)- if (!devin) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')- if (!await this.deps.sendText(session.agentId, `/model ${devin.id}`)) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- const answered = await this.waitPane(- session.agentId,- (value) => devinModelCommandResult(stripAnsi(value)) !== null,- COMMAND_CONFIRM_MS,- )- if (!answered) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')- if (devinModelCommandResult(stripAnsi(answered)) === 'unavailable') {- throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')- }- // The acknowledgement line is not the state — the footer is, and only `ingestPane` reads it. Nothing- // else feeds the manager during a control, so the wait has to do the feeding itself (same shape as- // setCursor); waiting on `waitForProfile` alone timed out on a switch that had already landed.- if (!await this.waitObservedProfile(session, target)) {- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')- }- }-- /**- * Pi splits the axes: `/model <provider/model>` takes a target directly, while the thinking level lives- * only in `/settings` → "Thinking Level", a fixed ladder walked with the arrow keys. Both were driven by- * hand first; the ladder's order is what makes the arrow count deterministic (see PI_THINKING_LEVELS).- */- private async setPi(- session: RegisteredSession,- target: RuntimeProfile,- current: RuntimeProfile | null,- ): Promise<void> {- if (current?.model !== target.model) {- if (!await this.deps.sendText(session.agentId, `/model ${target.model}`)) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- if (!await this.deps.manager.waitForModel(session.sessionId, COMMAND_CONFIRM_MS)) {- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')- }- }- if (target.effort === 'auto' || current?.effort === target.effort) {- this.deps.manager.confirmEffort(session.sessionId, target.effort)- return- }- await this.setPiThinking(session, target)- }-- private async setPiThinking(session: RegisteredSession, target: RuntimeProfile): Promise<void> {- const effort = target.effort- if (!await this.deps.sendText(session.agentId, '/settings')) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- if (!await this.waitPane(session.agentId, (v) => /Type to search/i.test(stripAnsi(v)), PICKER_OPEN_MS)) {- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')- }- // Typing alone is enough: the settings filter opens a row the moment it narrows to one. Submitting- // here would land an Enter on the ladder that just opened and pick the level already highlighted —- // which is the CURRENT one, so the switch silently did nothing.- if (!await this.deps.sendLiteral(session.agentId, 'thinking level')) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- const ladder = await this.waitPane(- session.agentId,- (v) => /Select reasoning depth/i.test(stripAnsi(v)),- PICKER_OPEN_MS,- )- if (!ladder) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')-- // Walk the ladder ONE key at a time, re-reading the cursor after each. Counting the steps up front and- // firing them blind is what an open-loop drive looks like, and it is exactly how this went wrong: a- // stale first read sent the cursor to `off` and committed it. Re-reading also means a ladder that- // scrolls, clamps at an end, or gains a level cannot desynchronise the walk.- let selection = parsePiThinkingSelection(stripAnsi(ladder))- if (!selection || piThinkingSteps(selection, effort) === null) {- throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')- }- for (let guard = 0; selection !== effort && guard < PI_LADDER_MAX_STEPS; guard++) {- const steps = piThinkingSteps(selection as string, effort)- if (steps === null || steps === 0) break- if (!await this.deps.sendKey(session.agentId, steps > 0 ? 'Down' : 'Up')) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- const moved = await this.waitPane(- session.agentId,- (value) => {- const next = parsePiThinkingSelection(stripAnsi(value))- return !!next && next !== selection- },- PICKER_STEP_MS,- )- // No movement means the cursor is pinned at an end of the ladder — pressing on cannot reach it.- if (!moved) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')- selection = parsePiThinkingSelection(stripAnsi(moved))- }- if (selection !== effort) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')- await this.deps.sendKey(session.agentId, 'Enter')- // The ladder returns to the settings list; Escape closes it so the pane is idle again.- await this.deps.sendKey(session.agentId, 'Escape')- if (!await this.waitObservedProfile(session, target)) {- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')- }- }-- /** Poll the pane INTO the manager until it reports the profile we asked for. */- private async waitObservedProfile(session: RegisteredSession, target: RuntimeProfile): Promise<boolean> {- const confirmed = await this.waitPane(session.agentId, async (value) => {- await this.deps.manager.ingestPane(session, value, true)- return this.deps.manager.selectedModel(session) === target.id- }, COMMAND_CONFIRM_MS)- return !!confirmed- }-- /**- * OpenCode switches through `/models`, a picker with a typed filter. It is driven by NARROWING, never by- * arrowing: type the model's words plus its provider's, and press Enter only once exactly one row is- * left and that row is the requested model. If the filter leaves two rows (two providers can carry the- * same model name) the switch is refused rather than guessed — picking the wrong model silently is worse- * than not switching, and blind arrow-driving is what sank the Command Code attempt.- */- private async setOpencode(session: RegisteredSession, target: RuntimeProfile): Promise<void> {- const entry = (await this.deps.manager.opencodeCatalog()).find((item) => item.id === target.model)- if (!entry) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')- await this.driveOpencodePicker(session, entry)- if (!await this.waitObservedProfile(session, target)) {- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')- }- }-- /**- * `/models`, filter, act only once exactly one row is left.- *- * Two attempts, because one filter does not fit both renders. With several providers connected a- * row carries its provider (`Big Pickle OpenCode Zen`) and the provider-qualified filter is what- * tells two same-named models apart; with one provider the row is bare (`Big Pickle`) and that same- * filter matches NOTHING. Measured on a live picker: `big pickle opencode` → empty, `big pickle` →- * the row.- *- * The fallback drops only the PROVIDER half, never the "exactly one row" rule — which is what makes- * it safe: if one model answers to the name, there is no second provider to confuse it with, and if- * two do, the list does not narrow and this refuses rather than guessing.- */- private async driveOpencodePicker(session: RegisteredSession, entry: OpencodeModelTarget): Promise<void> {- if (await this.pickOpencodeRow(session, entry, entry.filter, opencodeRowMatches)) return- if (entry.modelFilter === entry.filter) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')- if (!await this.pickOpencodeRow(session, entry, entry.modelFilter, opencodeRowNamesModel)) {- throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')- }- }-- /**- * One attempt at the picker: open it, type `filter`, and press Enter only if exactly one row is left- * AND `matches` accepts it. Returns false — with the picker CLOSED — when the list did not narrow to- * a row this may act on, so the caller can try a different filter on a pane in its resting state.- * Throws only for the states no retry can help: tmux refusing input, or the picker never opening.- */- private async pickOpencodeRow(- session: RegisteredSession,- entry: OpencodeModelTarget,- filter: string,- matches: (target: OpencodeModelTarget, row: OpencodePickerRow) => boolean,- ): Promise<boolean> {- // "Is a picker open?" cannot be asked of a capture that carries scrollback — an EARLIER picker is- // still up there, so the check passes before this one opens and the filter would be typed into the- // composer and submitted as a message. Count the openings instead and wait for one more.- const before = countOpencodePickers(stripAnsi(await this.deps.capture(session.agentId, 100) ?? ''))- if (!await this.deps.sendText(session.agentId, '/models')) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- if (!await this.waitPane(- session.agentId,- (v) => countOpencodePickers(stripAnsi(v)) > before,- PICKER_OPEN_MS,- )) {- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')- }- if (!await this.deps.sendLiteral(session.agentId, filter)) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- const narrowed = await this.waitPane(- session.agentId,- (v) => (parseOpencodePickerRows(stripAnsi(v)) ?? []).length === 1,- PICKER_STEP_MS,- )- const rows = narrowed ? parseOpencodePickerRows(stripAnsi(narrowed)) ?? [] : []- if (rows.length !== 1 || !matches(entry, rows[0])) {- // Leave the pane as it was found. A picker left open with a dead filter in it swallows whatever- // the user types next, and the retry below would type its filter on top of this one.- await this.deps.sendKey(session.agentId, 'Escape')- return false- }- if (!await this.deps.sendKey(session.agentId, 'Enter')) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- return true- }-- /**- * Command Code 1.6.0 takes both axes as arguments: `/model <vendor/id>` and `/effort <level>`. Neither- * opens a dialog, which is the whole reason switching is back after the 2026-07-30 revert — the version- * gate in supportsNativeRuntimeControl keeps older builds, where only an arrow-driven picker existed,- * out of this path.- *- * Effort is per model and the catalog does not say which models have one, so a refusal is expected- * traffic: the CLI answers "Reasoning effort not supported for X." and that becomes EFFORT_UNSUPPORTED.- */- private async setCommandcode(- session: RegisteredSession,- target: RuntimeProfile,- current: RuntimeProfile | null,- ): Promise<void> {- const entry = this.deps.manager.commandcodeTarget(session.sessionId, target.id)- if (!entry) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')-- if (current?.model !== target.model) {- if (!await this.deps.sendText(session.agentId, `/model ${entry.id}`)) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- if (!await this.waitObservedModel(session, target)) {- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')- }- }-- if (target.effort === 'auto' || current?.effort === target.effort) {- this.deps.manager.confirmEffort(session.sessionId, target.effort)- return- }- // Success prints NOTHING and a refusal prints one line, so both have to be watched at once — and the- // refusal has to be a NEW one. Matching the text anywhere in the capture reported failure in 25ms off- // a refusal still sitting in the scrollback from an earlier model, while the level had in fact applied.- const refusalsBefore = countCommandcodeRefusals(stripAnsi(await this.deps.capture(session.agentId, 100) ?? ''))- if (!await this.deps.sendText(session.agentId, `/effort ${target.effort}`)) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- const deadline = Date.now() + COMMAND_CONFIRM_MS- while (Date.now() < deadline) {- // The level is recorded only in the CLI's global config, so confirming it means re-reading that.- await this.deps.manager.ingestConfig(session, true).catch(() => false)- if (this.deps.manager.selectedModel(session) === target.id) return- const capture = stripAnsi(await this.deps.capture(session.agentId, 100) ?? '')- if (countCommandcodeRefusals(capture) > refusalsBefore) {- throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')- }- await sleep(150)- }- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')- }-- /**- * Hermes' `/model` is a two-level picker — provider page, then that provider's models — with no way to- * type or jump. It is walked one arrow at a time, re-reading the `❯` cursor after every key, so a page- * that scrolls or reorders cannot desynchronise the walk. If the cursor stops moving before it reaches- * the target, the target is not on the page and the switch is refused rather than committed blind.- *- * Only MODELS are switchable; hermes keeps its reasoning effort in config.yaml with no in-session- * command, so hermesModels never offers an effort row to get here (see hermesModels).- */- private async setHermes(session: RegisteredSession, target: RuntimeProfile): Promise<void> {- const entry = this.deps.manager.hermesTarget(session.sessionId, target.id)- if (!entry) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')- if (!await this.deps.sendText(session.agentId, '/model')) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- const opened = await this.waitPane(- session.agentId,- (v) => !!parseHermesPickerPage(stripAnsi(v)),- PICKER_OPEN_MS,- )- if (!opened) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')-- // Page 1: the provider. Its row reads "Anthropic (13 models)", so match on the leading name.- if (entry.provider) {- await this.walkHermesPage(session, (row) => hermesProviderMatches(row, entry.provider))- if (!await this.waitPane(- session.agentId,- (v) => (parseHermesPickerPage(stripAnsi(v))?.rows ?? []).some((row) => row === entry.id),- PICKER_OPEN_MS,- )) {- throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')- }- }- // Page 2: the model itself, listed by exact id.- await this.walkHermesPage(session, (row) => row === entry.id)- if (!await this.waitObservedModel(session, target)) {- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')- }- }-- /** Move the `❯` cursor onto the row `wanted` selects, then commit it with Enter. */- private async walkHermesPage(- session: RegisteredSession,- wanted: (row: string) => boolean,- ): Promise<void> {- for (let guard = 0; guard < HERMES_PAGE_MAX_STEPS; guard++) {- const capture = await this.deps.capture(session.agentId, 100)- const page = capture ? parseHermesPickerPage(stripAnsi(capture)) : null- if (!page || !page.selected) throw new RuntimeProfileControlError('TMUX_FAILED')- if (wanted(page.selected)) {- if (!await this.deps.sendKey(session.agentId, 'Enter')) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- return- }- const at = page.rows.indexOf(page.selected)- const to = page.rows.findIndex(wanted)- if (to < 0) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')- if (!await this.deps.sendKey(session.agentId, to > at ? 'Down' : 'Up')) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- const moved = await this.waitPane(session.agentId, (v) => {- const next = parseHermesPickerPage(stripAnsi(v))- return !!next?.selected && next.selected !== page.selected- }, PICKER_STEP_MS)- if (!moved) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')- }- throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')- }-- /** Like waitObservedProfile, but only the model half — used where effort is applied separately. */- private async waitObservedModel(session: RegisteredSession, target: RuntimeProfile): Promise<boolean> {- const confirmed = await this.waitPane(session.agentId, async (value) => {- await this.deps.manager.ingestPane(session, value, true)- return parseRuntimeProfile(this.deps.manager.selectedModel(session))?.model === target.model- }, COMMAND_CONFIRM_MS)- return !!confirmed- }-- /**- * Codex's `/model`: a quick menu of auto presets with an `All models` row (0.145 and later), the full- * list (`Select Model and Effort`), then the chosen model's reasoning picker, each row pressed by its- * digit, which selects the row and accepts it. How a row is read and chosen, and why, is in- * engines/codex/modelPicker.ts. Confirmed, as it always was, by the `thread_settings_applied` record- * Codex writes once the choice is applied (RuntimeProfileManager `ingestCodex`).- */- private async setCodex(session: RegisteredSession, target: RuntimeProfile, catalog: CodexCatalogModel[]): Promise<void> {- if (!await this.deps.sendText(session.agentId, '/model')) throw new RuntimeProfileControlError('TMUX_FAILED')- let picker = await this.waitCodexPicker(session, () => true, 900)- if (!picker) {- await this.deps.sendKey(session.agentId, 'Enter')- picker = await this.waitCodexPicker(session, () => true, PICKER_OPEN_MS)- }- if (!picker) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')- if (!isCodexList(picker)) {- // Opened on a reasoning screen: back to the list it was opened from, so the model is chosen too.- if (!await this.deps.sendKey(session.agentId, 'Escape')) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')- if (!await this.waitCodexPicker(session, isCodexList, PICKER_STEP_MS)) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')- }- const reached = await this.reachCodexEfforts(session, target, catalog)- if (reached !== 'applied') await this.pickCodexEffort(session, target, reached)- if (!await this.deps.manager.waitForProfile(session.sessionId, COMMAND_CONFIRM_MS)) {- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')- }- }-- /**- * From the list on screen to the target model's reasoning picker, through `All models` when the quick- * menu is first; or 'applied' when the model's row applied it at once (a model with one effort).- *- * Every digit is pressed from a list read twice, CODEX_SETTLE_MS apart, the same both times. Codex- * 0.160 draws the list from its cache and redraws it in place when the server answers, and a model- * added or reordered then renumbers the rows: a digit read off the first drawing lands on another- * model. The reasoning picker that opens names its model in its title, and a title naming any other- * model than the row pressed means the list moved anyway, so nothing more is pressed.- */- private async reachCodexEfforts(- session: RegisteredSession,- target: RuntimeProfile,- catalog: CodexCatalogModel[],- ): Promise<CodexPicker | 'applied'> {- let reread = false- for (let lists = 0; lists < 2; lists++) {- const list = await this.settledCodexList(session)- if (!list) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')- let choice = chooseCodexRow(list, target, catalog)- if ('error' in choice && choice.error === 'MODEL_UNAVAILABLE' && !reread) {- // Codex saves the catalog its picker refreshed from to models_cache.json, so a model or a name- // that came with the refresh is there to read now.- reread = true- catalog = await this.deps.manager.codexCatalog(session)- choice = chooseCodexRow(list, target, catalog)- }- if ('error' in choice) throw new RuntimeProfileControlError(choice.error)- if (!await this.deps.sendKey(session.agentId, String(choice.row.number))) throw new RuntimeProfileControlError('TMUX_FAILED')- if (choice.opens === 'list') {- if (!await this.waitCodexPicker(session, (next) => next.kind === 'models', PICKER_STEP_MS)) {- throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')- }- continue- }- if (choice.opens === 'applied') return 'applied'- const next = await this.waitCodexAfterModelRow(session)- if (next === 'closed') return 'applied'- if (!next) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')- if (next.model !== choice.row.name) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')- return next- }- throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')- }-- /** The target's effort in its reasoning picker, through Advanced Reasoning for Max and Ultra. */- private async pickCodexEffort(session: RegisteredSession, target: RuntimeProfile, picker: CodexPicker): Promise<void> {- const rows = codexEffortRows(picker)- let row = target.effort === 'auto' ? rows.defaultRow : rows.efforts.get(target.effort) ?? null- if (row === null && (target.effort === 'max' || target.effort === 'ultra') && rows.advancedRow !== null- && codexDigitPressable(rows.advancedRow)) {- if (!await this.deps.sendKey(session.agentId, String(rows.advancedRow))) throw new RuntimeProfileControlError('TMUX_FAILED')- const advanced = await this.waitCodexPicker(session, (next) => next.kind === 'advanced', PICKER_STEP_MS)- row = advanced ? codexEffortRows(advanced).efforts.get(target.effort) ?? null : null- }- if (row === null || !codexDigitPressable(row)) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')- if (!await this.deps.sendKey(session.agentId, String(row))) throw new RuntimeProfileControlError('TMUX_FAILED')- }-- /** The first Codex picker on the pane that `wanted` takes, polled until `timeoutMs`. */- private async waitCodexPicker(- session: RegisteredSession,- wanted: (picker: CodexPicker) => boolean,- timeoutMs: number,- ): Promise<CodexPicker | null> {- const capture = await this.waitPane(session.agentId, (value) => {- const picker = parseCodexPicker(value)- return !!picker && wanted(picker)- }, timeoutMs)- return capture ? parseCodexPicker(capture) : null- }-- /** A model list that reads the same twice, CODEX_SETTLE_MS apart (see reachCodexEfforts). */- private async settledCodexList(session: RegisteredSession): Promise<CodexPicker | null> {- let previous: CodexPicker | null = null- const deadline = Date.now() + PICKER_STEP_MS- while (Date.now() < deadline) {- const capture = await this.deps.capture(session.agentId, 100)- const picker = capture ? parseCodexPicker(capture) : null- const list = picker && isCodexList(picker) ? picker : null- if (list && previous && sameCodexPicker(previous, list)) return list- previous = list- await sleep(CODEX_SETTLE_MS)- }- return null- }-- /**- * After a model's row: its reasoning picker, or 'closed' when the picker went away and the composer- * is back, which is a model the catalog did not describe applying its one effort at once.- */- private async waitCodexAfterModelRow(session: RegisteredSession): Promise<CodexPicker | 'closed' | null> {- const deadline = Date.now() + PICKER_STEP_MS- while (Date.now() < deadline) {- const capture = await this.deps.capture(session.agentId, 100)- if (capture) {- const picker = parseCodexPicker(capture)- if (picker?.kind === 'efforts') return picker- if (!picker && inspectRuntimePane('codex', capture).idle) return 'closed'- }- await sleep(100)- }- return null- }-- private async setCursor(session: RegisteredSession, target: RuntimeProfile): Promise<void> {- const cursorTarget = this.deps.manager.cursorTarget(session.sessionId, target.id)- if (!cursorTarget) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')- if (!await this.deps.sendText(session.agentId, `/model ${cursorTarget.familyLabel}`)) {- throw new RuntimeProfileControlError('TMUX_FAILED')- }- let capture = await this.waitPane(session.agentId, (value) => !!parseCursorModelPicker(value), 900)- if (!capture) {- if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')- capture = await this.waitPane(session.agentId, (value) => !!parseCursorModelPicker(value), PICKER_OPEN_MS)- }- const picker = capture ? parseCursorModelPicker(capture) : null- if (!picker || picker.selectedFamily?.toLowerCase() !== cursorTarget.familyLabel.toLowerCase()) {- throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')- }-- if (cursorTarget.rawId !== 'auto') {- if (!await this.deps.sendKey(session.agentId, 'Tab')) throw new RuntimeProfileControlError('TMUX_FAILED')- capture = await this.waitPane(session.agentId, (value) => !!parseCursorParameterRows(value), PICKER_STEP_MS)- if (!capture) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')-- if (cursorTarget.context) {- await this.setCursorParameter(session, 'context', cursorTarget.context, true)- }- if (cursorTarget.reasoning) {- await this.setCursorParameter(session, 'reasoning', cursorTarget.reasoning, true)- }- await this.setCursorParameter(session, 'thinking', 'true', cursorTarget.thinking ?? false)- await this.setCursorParameter(session, 'fast', 'true', cursorTarget.fast ?? false)-- if (!await this.deps.sendKey(session.agentId, 'Escape')) throw new RuntimeProfileControlError('TMUX_FAILED')- if (!await this.waitPane(session.agentId, (value) => !!parseCursorModelPicker(value), PICKER_STEP_MS)) {- throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')- }- if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')- } else {- if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')- }-- const confirmed = await this.waitPane(session.agentId, async (value) => {- await this.deps.manager.ingestPane(session, value, true)- return this.deps.manager.selectedModel(session) === target.id- || this.cursorFooterMatches(value, cursorTarget, target.effort)- }, COMMAND_CONFIRM_MS)- if (!confirmed) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')- if (this.deps.manager.selectedModel(session) !== target.id) {- this.deps.manager.confirmControlProfile(target)- }- }-- private async setCursorParameter(- session: RegisteredSession,- kind: CursorParameterRow['kind'],- value: string,- selected: boolean,- ): Promise<void> {- const capture = await this.deps.capture(session.agentId, 100)- const rows = capture ? parseCursorParameterRows(capture) : null- if (!rows) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')- const target = rows.find((row) => row.kind === kind && row.value === value)- if (!target) {- if (!selected && (kind === 'fast' || kind === 'thinking')) return- throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')- }- if (target.selected === selected) return- const current = rows.find((row) => row.cursor)- if (!current) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')- const direction = target.index > current.index ? 'Down' : 'Up'- for (let i = 0; i < Math.abs(target.index - current.index); i++) {- if (!await this.deps.sendKey(session.agentId, direction)) throw new RuntimeProfileControlError('TMUX_FAILED')- await sleep(120)- }- if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')- const updated = await this.waitPane(session.agentId, (next) => {- const nextRows = parseCursorParameterRows(next)- return nextRows?.some((row) => row.kind === kind && row.value === value && row.selected === selected) ?? false- }, PICKER_STEP_MS)- if (!updated) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')- }-- private cursorFooterMatches(- capture: string,- target: CursorModelTarget,- effort: string | null,- ): boolean {- const lines = stripAnsi(capture).split('\n').filter((line) => line.trim()).slice(-15)- if (target.rawId === 'auto') return lines.some((line) => /^\s*Auto(?:\s*$|\s*[·│])/i.test(line))- return lines.some((line) => {- const normalized = line.trim().toLowerCase()- if (!normalized.startsWith(target.familyLabel.toLowerCase())) return false- const actualFast = /\bfast\b/i.test(line)- const actualThinking = !/\bno\s+thinking\b/i.test(line) && /\bthinking\b/i.test(line)- if (target.fast != null && actualFast !== target.fast) return false- if (target.thinking != null && actualThinking !== target.thinking) return false- if (target.context && !normalized.includes(target.context)) return false-- const effortMatch = /\b(extra high|none|low|medium|high|max)\b/i.exec(line)- const actualEffort = effortMatch?.[1].toLowerCase().replace(/\s+/g, '') === 'extrahigh'- ? 'xhigh'- : effortMatch?.[1].toLowerCase() ?? null- const footerEffort = target.footerEffort === undefined ? effort : target.footerEffort- return footerEffort == null ? actualEffort == null : actualEffort === footerEffort- })- }-- private async waitPane(terminalTarget: string, predicate: (capture: string) => boolean | Promise<boolean>, timeoutMs: number): Promise<string | null> {- const deadline = Date.now() + timeoutMs- while (Date.now() < deadline) {- const capture = await this.deps.capture(terminalTarget, 100)- if (capture && await predicate(capture)) return capture- await sleep(100)- }- return null+/** Explicit inline compatibility for direct controller users. */+export * from './runtimeControl.js'+export { inspectRuntimePane, paneModal } from '../engines/screens.js'+import { screenFor } from '../engines/screens.js'+import { RuntimeProfileController as Controller, type RuntimeProfileControllerDeps } from './runtimeControl.js'+export class RuntimeProfileController extends Controller {+ constructor(deps: Omit<RuntimeProfileControllerDeps, 'readScreen'> & Partial<Pick<RuntimeProfileControllerDeps, 'readScreen'>>) {+ super({ readScreen: async (session, capture) => capture === null ? null : screenFor(session.engine).inspect(capture), ...deps })}}diff --git a/cli/src/lib/screenReader.ts b/cli/src/lib/screenReader.tsnew file mode 100644index 000000000..e897339d0--- /dev/null+++ b/cli/src/lib/screenReader.ts@@ -0,0 +1,5 @@+import type { ScreenReading } from '../engines/facets/screen.js'+import type { RegisteredSession } from './registry.js'++/** Null means unavailable, not an empty screen or a closed question. */+export type ScreenReader = (session: RegisteredSession, capture: string | null) => Promise<ScreenReading | null>diff --git a/cli/src/lib/sessionInput.spec.ts b/cli/src/lib/sessionInput.spec.tsindex 0c01cc21f..03623bc36 100644--- a/cli/src/lib/sessionInput.spec.ts+++ b/cli/src/lib/sessionInput.spec.ts@@ -1,3 +1,4 @@+import { readInlineScreen } from '../testing/inlineScreen.js'import { readFileSync } from 'node:fs'import { dirname, join } from 'node:path'import { fileURLToPath } from 'node:url'@@ -38,7 +39,7 @@ describe('SessionInputController', () => {] as const)('%s, %s: tells the person why and what to do, asks no lease again, and awaits nothing', async (engine, reason, text) => {const onError = vi.fn(), onSubmitted = vi.fn(), sleep = vi.fn(async () => {})const inject = vi.fn(async () => held(reason))- const controller = new SessionInputController({ getSession: () => session(engine), validateRuntime: async () => true,+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session(engine), validateRuntime: async () => true,inject, sendKey: async () => true, onError, onSubmitted, sleep })controller.submit('s1', 'a message from the app')await vi.waitFor(() => expect(onError).toHaveBeenCalledWith('s1', text))@@ -54,7 +55,7 @@ describe('SessionInputController', () => {const onError = vi.fn(), onSubmitted = vi.fn()const inject = vi.fn<() => Promise<TerminalActionResult>>().mockResolvedValueOnce(held('prompt_hidden')).mockResolvedValueOnce(held('screen_unreadable')).mockResolvedValue({ state: 'succeeded', dispatch: 'executed' })- const controller = new SessionInputController({ getSession: () => session('codex'), validateRuntime: async () => true,+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('codex'), validateRuntime: async () => true,inject, sendKey: async () => true, onError, onSubmitted, now: time.now, sleep: time.sleep })controller.submit('s1', 'sent while it starts')await vi.waitFor(() => expect(onSubmitted).toHaveBeenCalledWith('s1', 'sent while it starts'))@@ -67,7 +68,7 @@ describe('SessionInputController', () => {const time = { at: 0, now: () => time.at, sleep: vi.fn(async (ms: number) => { time.at += ms }) }const onError = vi.fn()const inject = vi.fn(async () => held('prompt_hidden'))- const controller = new SessionInputController({ getSession: () => session('claude'), validateRuntime: async () => true,+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('claude'), validateRuntime: async () => true,inject, sendKey: async () => true, onError, now: time.now, sleep: time.sleep })controller.submit('s1', 'into a screen never seen')await vi.waitFor(() => expect(onError).toHaveBeenCalledWith('s1', 'Claude Code isn\'t showing its prompt; finish what\'s on its screen in its terminal, then send the message again.'))@@ -79,7 +80,7 @@ describe('SessionInputController', () => {const withheld: TerminalActionResult = { state: 'unknown', dispatch: 'possibly_executed', reason: 'enter_withheld:permission_open' }const onError = vi.fn(), onSubmitted = vi.fn(), sendKey = vi.fn(async () => true), forget = vi.fn(), onDelivery = vi.fn()vi.useFakeTimers()- const controller = new SessionInputController({ getSession: () => session('claude'), validateRuntime: async () => true,+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('claude'), validateRuntime: async () => true,inject: async () => withheld, sendKey, onError, onSubmitted, onDelivery, beforeSubmit: () => forget, beforeTeamWrite: async () => null })controller.submit('s1', 'a message from the app')await vi.advanceTimersByTimeAsync(30_000)@@ -103,7 +104,7 @@ describe('SessionInputController', () => {it('rejects a message that wants a receipt with the reason, and says why', async () => {const onError = vi.fn(), onDelivery = vi.fn()- const controller = new SessionInputController({ getSession: () => session('codex'), validateRuntime: async () => true,+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('codex'), validateRuntime: async () => true,inject: async () => held('permission_open'), sendKey: async () => true, onError, onDelivery })controller.submit('s1', 'a message from the device', 'd1')await vi.waitFor(() => expect(onDelivery).toHaveBeenCalledWith({ sessionId: 's1', deliveryId: 'd1', state: 'rejected', reason: 'permission_open' }))@@ -113,7 +114,7 @@ describe('SessionInputController', () => {it('rejects a team\'s turn with the reason, and tells no person: its team hears of it', async () => {const onError = vi.fn(), onDelivery = vi.fn()- const controller = new SessionInputController({ getSession: () => session('claude'), validateRuntime: async () => true,+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('claude'), validateRuntime: async () => true,beforeTeamWrite: async () => null, inject: vi.fn(), injectTeam: async () => held('menu_open'), sendKey: async () => true, onError, onDelivery })controller.submit('s1', 'a team turn', 'team:1')await vi.waitFor(() => expect(onDelivery).toHaveBeenCalledWith({ sessionId: 's1', deliveryId: 'team:1', state: 'rejected', reason: 'menu_open' }))@@ -134,7 +135,7 @@ describe('SessionInputController', () => {it('waits for the lease — a resume whose new process is not confirmed yet — and then delivers, once', async () => {const time = clock(), onError = vi.fn(), onSubmitted = vi.fn()const inject = vi.fn<() => Promise<TerminalActionResult>>().mockResolvedValueOnce(refused).mockResolvedValueOnce(refused).mockResolvedValue(done)- const controller = new SessionInputController({ getSession: () => session('claude'), validateRuntime: async () => true,+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('claude'), validateRuntime: async () => true,inject, sendKey: async () => true, onError, onSubmitted, ...time })controller.submit('s1', 'right after the resume')await vi.waitFor(() => expect(onSubmitted).toHaveBeenCalledWith('s1', 'right after the resume'))@@ -147,7 +148,7 @@ describe('SessionInputController', () => {it('gives up at fifteen seconds, says the message was not delivered, and wrote nothing', async () => {const time = clock(), onError = vi.fn()const inject = vi.fn(async () => refused)- const controller = new SessionInputController({ getSession: () => session('claude'), validateRuntime: async () => true,+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('claude'), validateRuntime: async () => true,inject, sendKey: async () => true, onError, ...time })controller.submit('s1', 'never leased')await vi.waitFor(() => expect(onError).toHaveBeenCalledWith('s1', 'The message could not be delivered to the agent.'))@@ -160,14 +161,14 @@ describe('SessionInputController', () => {const time = clock(), onError = vi.fn()let present = trueconst inject = vi.fn(async () => { present = false; return refused })- const gone = new SessionInputController({ getSession: () => (present ? session('claude') : undefined), validateRuntime: async () => true,+ const gone = new SessionInputController({ readScreen: readInlineScreen, getSession: () => (present ? session('claude') : undefined), validateRuntime: async () => true,inject, sendKey: async () => true, onError, ...time })gone.submit('s1', 'to an agent that left')await vi.waitFor(() => expect(onError).toHaveBeenCalled())expect(inject).toHaveBeenCalledTimes(1)const other = vi.fn(async (): Promise<TerminalActionResult> => ({ state: 'failed', dispatch: 'not_started', reason: 'terminal agent is unavailable' }))- const elsewhere = new SessionInputController({ getSession: () => session('claude'), validateRuntime: async () => true,+ const elsewhere = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('claude'), validateRuntime: async () => true,inject: other, sendKey: async () => true, onError: vi.fn(), ...time })elsewhere.submit('s1', 'to an agent with no terminal')await vi.waitFor(() => expect(other).toHaveBeenCalledTimes(1))@@ -178,7 +179,7 @@ describe('SessionInputController', () => {it('waits the same way for a message that wants a receipt, until it is cancelled', async () => {const time = clock(), onDelivery = vi.fn()const inject = vi.fn<() => Promise<TerminalActionResult>>().mockResolvedValueOnce(refused).mockResolvedValue(done)- const controller = new SessionInputController({ getSession: () => session('codex'), validateRuntime: async () => true,+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('codex'), validateRuntime: async () => true,inject, sendKey: async () => true, onError: vi.fn(), onDelivery, ...time })controller.submit('s1', 'with a receipt', 'delivery-1')await vi.waitFor(() => expect(inject).toHaveBeenCalledTimes(2))@@ -186,7 +187,7 @@ describe('SessionInputController', () => {const cancelled = vi.fn(async () => refused)let cancel: () => void = () => {}- const waiting = new SessionInputController({ getSession: () => session('codex'), validateRuntime: async () => true,+ const waiting = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('codex'), validateRuntime: async () => true,inject: cancelled, sendKey: async () => true, onError: vi.fn(), onDelivery,now: time.now, sleep: async (ms) => { time.sleep(ms); cancel() } })cancel = () => waiting.forget('s1')@@ -200,7 +201,7 @@ describe('SessionInputController', () => {it('retains the submitted swarm through the input queue and records it only at dispatch', async () => {const beforeSubmit = vi.fn(() => vi.fn())- const controller = new SessionInputController({ getSession: () => session('cursor'), validateRuntime: async () => true,+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('cursor'), validateRuntime: async () => true,inject: async () => true, sendKey: async () => true, beforeSubmit, onError: vi.fn() })controller.setTurnOpen('s1', true)controller.submit('s1', 'task from A', undefined, 'swarm-a')@@ -212,7 +213,7 @@ describe('SessionInputController', () => {it.each(['team_waiting_draft', 'team_waiting_user', 'team_waiting_idle'])('team input preserves the composer on %s', async reason => {const inject = vi.fn(async () => true), sendKey = vi.fn(async () => true), onDelivery = vi.fn()- const controller = new SessionInputController({ getSession: () => session('cursor'), validateRuntime: async () => true,+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('cursor'), validateRuntime: async () => true,inject, sendKey, onError: vi.fn(), onDelivery, beforeTeamWrite: async () => reason })controller.submit('s1', 'peer question', 'team:fixture')await vi.waitFor(() => expect(onDelivery).toHaveBeenCalledWith(expect.objectContaining({ state: 'rejected', reason })))@@ -223,7 +224,7 @@ describe('SessionInputController', () => {it('rechecks a queued team message at the actual write boundary', async () => {const beforeTeamWrite = vi.fn(async () => 'team_waiting_draft'), inject = vi.fn(async () => true), onDelivery = vi.fn()- const controller = new SessionInputController({ getSession: () => session(), validateRuntime: async () => true,+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session(), validateRuntime: async () => true,inject, sendKey: async () => true, onError: vi.fn(), onDelivery, beforeTeamWrite })controller.setTurnOpen('s1', true)controller.submit('s1', 'peer question', 'team:fixture')@@ -237,7 +238,7 @@ describe('SessionInputController', () => {it('never retries Enter for an automatic team message with uncertain acceptance', async () => {vi.useFakeTimers()const sendKey = vi.fn(async () => true), onDelivery = vi.fn()- const controller = new SessionInputController({ getSession: () => session(), validateRuntime: async () => true,+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session(), validateRuntime: async () => true,inject: async () => true, sendKey, onError: vi.fn(), onDelivery, beforeTeamWrite: async () => null })controller.submit('s1', 'peer question', 'team:fixture')await vi.advanceTimersByTimeAsync(12_000)@@ -248,7 +249,7 @@ describe('SessionInputController', () => {it('requeues a team notice when a draft appears while waiting for the terminal writer', async () => {const inject = vi.fn(async () => true), onError = vi.fn(), onDelivery = vi.fn()- const controller = new SessionInputController({ getSession: () => session(), validateRuntime: async () => true,+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session(), validateRuntime: async () => true,beforeTeamWrite: async () => null, inject,injectTeam: async () => ({ state: 'failed', dispatch: 'not_started', reason: 'team_waiting_draft' }),sendKey: async () => true, onError, onDelivery })@@ -267,7 +268,7 @@ describe('SessionInputController', () => {{ engine: 'codex', observed: '<pasted_content id="04e3">\npeer question\n</pasted_content id="04e3">', state: 'unknown' },] as const)('attributes native paste evidence only to the exact $engine message: $state', async ({ engine, observed, state }) => {const inject = vi.fn(async () => true), sendKey = vi.fn(async () => true), onDelivery = vi.fn()- const controller = new SessionInputController({ getSession: () => session(engine), validateRuntime: async () => true,+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session(engine), validateRuntime: async () => true,inject, sendKey, onError: vi.fn(), onDelivery, beforeTeamWrite: async () => null })controller.submit('s1', 'peer question', 'team:fixture')await vi.waitFor(() => expect(onDelivery).toHaveBeenCalledWith(expect.objectContaining({ state: 'delivered' })))@@ -282,7 +283,7 @@ describe('SessionInputController', () => {// A voice command spoken while a Codex task ran used to sit in this controller's queue, invisible,// until the task ended. Codex queues composer input itself, so the daemon types straight away.const injected: string[] = []- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('codex'), validateRuntime: async () => true,inject: async (_pane, content) => { injected.push(content); return true },sendKey: async () => true, onError: vi.fn(),@@ -303,7 +304,7 @@ describe('SessionInputController', () => {it('queues Command Code prompts while busy and drains exactly one after turn end', async () => {const injected: string[] = []- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('commandcode'), validateRuntime: async () => true,inject: async (_pane, content) => { injected.push(content); return true },sendKey: async () => true, onError: vi.fn(),@@ -322,7 +323,7 @@ describe('SessionInputController', () => {it('clears the echoed prompt from the Cursor composer once the turn starts', async () => {const sendKey = vi.fn(async (_pane: string, _key: string) => true)- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('cursor'), validateRuntime: async () => true,inject: async () => true, sendKey,capture: async () => '⠰ Working\n\n→ hello\nAuto', // the submitted prompt is still on screen@@ -336,7 +337,7 @@ describe('SessionInputController', () => {it('leaves a fresh terminal draft alone when the Cursor composer no longer echoes our prompt', async () => {const sendKey = vi.fn(async (_pane: string, _key: string) => true)- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('cursor'), validateRuntime: async () => true,inject: async () => true, sendKey,capture: async () => '⠰ Working\n\n→ something the user just typed\nAuto',@@ -354,7 +355,7 @@ describe('SessionInputController', () => {// message is typed onto the end of it and the two are submitted as one run-on prompt — observed as a// turn starting with the PREVIOUS message's text.const order: string[] = []- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('cursor'), validateRuntime: async () => true,inject: async (_pane, content) => { order.push(`inject:${content}`); return true },sendKey: async (_pane, key) => { order.push(`key:${key}`); return true },@@ -371,7 +372,7 @@ describe('SessionInputController', () => {vi.useFakeTimers()const inject = vi.fn(async () => true)const sendKey = vi.fn(async () => true)- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session(), validateRuntime: async () => true, inject, sendKey, onError: vi.fn(),})controller.submit('s1', 'hello')@@ -385,7 +386,7 @@ describe('SessionInputController', () => {vi.useFakeTimers()const inject = vi.fn(async () => true)const sendKey = vi.fn(async () => true)- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('claude'), validateRuntime: async () => true, inject, sendKey, onError: vi.fn(),})controller.submit('s1', 'hello')@@ -401,7 +402,7 @@ describe('SessionInputController', () => {const inject = vi.fn(async () => true)const sendKey = vi.fn(async (_pane: string, _key: string) => true)const onError = vi.fn()- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('cursor'),validateRuntime: async () => true,inject,@@ -424,7 +425,7 @@ describe('SessionInputController', () => {it('retries Enter for Cursor only while the exact draft remains in an idle composer', async () => {vi.useFakeTimers()const sendKey = vi.fn(async (_pane: string, _key: string) => true)- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('cursor'),validateRuntime: async () => true,inject: async () => true,@@ -444,7 +445,7 @@ describe('SessionInputController', () => {it('waits for the Cursor composer to settle before draining the next prompt', async () => {vi.useFakeTimers()const injected: string[] = []- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('cursor'),validateRuntime: async () => true,inject: async (_pane, content) => { injected.push(content); return true },@@ -466,7 +467,7 @@ describe('SessionInputController', () => {vi.useFakeTimers()const sendKey = vi.fn(async () => true)const onError = vi.fn()- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('claude'),validateRuntime: async () => true,inject: async () => true,@@ -491,7 +492,7 @@ describe('SessionInputController', () => {vi.useFakeTimers()const sendKey = vi.fn(async () => true)const onError = vi.fn()- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('commandcode'),validateRuntime: async () => true,inject: async () => true,@@ -514,7 +515,7 @@ describe('SessionInputController', () => {const sendKey = vi.fn(async () => true)const onError = vi.fn()const capture = vi.fn(async () => '❯ What are you working on?\n✻ Waiting for 4 background agents to finish\n────\n❯ a different draft I am still writing\n────')- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('claude'), validateRuntime: async () => true,inject: async () => true, sendKey, capture, onError,})@@ -541,7 +542,7 @@ describe('SessionInputController', () => {vi.useFakeTimers()const sendKey = vi.fn(async () => true), onError = vi.fn(), onDelivery = vi.fn()let pasted = false- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('claude'), validateRuntime: async () => true, onDelivery,inject: async () => { pasted = true; return true }, sendKey, capture: async () => pasted ? screen : '❯ ', onError,})@@ -556,7 +557,7 @@ describe('SessionInputController', () => {vi.useFakeTimers()const sendKey = vi.fn(async () => true), onError = vi.fn(), onDelivery = vi.fn()let pasted = false- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('claude'), validateRuntime: async () => true, onDelivery,inject: async () => { pasted = true; return true }, sendKey, capture: async () => pasted ? screen : '❯ ', onError,})@@ -572,7 +573,7 @@ describe('SessionInputController', () => {const sendKey = vi.fn(async () => true), onError = vi.fn()// Codex browsing its transcript (0.160): Enter would rewind the conversation to the prompt in view.const browsing = `› ${message}\n\n Browsing transcript · ↑↓/jk scroll · ←→/hl prompts · ↵ rewind · esc back\n`- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('codex'), validateRuntime: async () => true,inject: async () => true, sendKey, capture: async () => browsing, onError,})@@ -588,7 +589,7 @@ describe('SessionInputController', () => {vi.useFakeTimers()const sendKey = vi.fn(async () => true)const onError = vi.fn()- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('claude'), validateRuntime: async () => true,inject: async () => true, sendKey, capture: async () => 'Sign in required', onError,})@@ -603,7 +604,7 @@ describe('SessionInputController', () => {vi.useFakeTimers()const sendKey = vi.fn(async () => true)const onError = vi.fn()- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('claude'),validateRuntime: async () => true,inject: async () => true,@@ -624,7 +625,7 @@ describe('SessionInputController', () => {vi.useFakeTimers()const sendKey = vi.fn(async () => true)const onError = vi.fn()- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('codex'),validateRuntime: async () => true,inject: async () => true,@@ -645,7 +646,7 @@ describe('SessionInputController', () => {vi.useFakeTimers()const sendKey = vi.fn(async () => true)const onError = vi.fn()- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('codex'),validateRuntime: async () => true,inject: async () => true,@@ -666,7 +667,7 @@ describe('SessionInputController', () => {vi.useFakeTimers()const sendKey = vi.fn(async () => true)const onError = vi.fn()- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('codex'),validateRuntime: async () => true,inject: async () => true,@@ -689,7 +690,7 @@ describe('SessionInputController', () => {const ambiguous: TerminalActionResult = {state: 'unknown', dispatch: 'possibly_executed', reason: 'response was lost',}- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('codex'),validateRuntime: async () => true,inject: async () => ambiguous,@@ -711,7 +712,7 @@ describe('SessionInputController', () => {const ambiguous: TerminalActionResult = {state: 'unknown', dispatch: 'possibly_executed', reason: 'response was lost',}- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('codex'),validateRuntime: async () => true,inject: async () => ambiguous,@@ -734,7 +735,7 @@ describe('SessionInputController', () => {const ambiguous: TerminalActionResult = {state: 'unknown', dispatch: 'possibly_executed', reason: 'response was lost',}- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('codex'),validateRuntime: async () => true,inject: async () => ambiguous,@@ -758,7 +759,7 @@ describe('SessionInputController', () => {state: 'unknown', dispatch: 'possibly_executed', reason: 'Enter response was lost',}))const onError = vi.fn()- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('codex'),validateRuntime: async () => true,inject: async () => true,@@ -782,7 +783,7 @@ describe('SessionInputController', () => {const ambiguous: TerminalActionResult = {state: 'unknown', dispatch: 'possibly_executed', reason: 'response was lost',}- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('cursor'),validateRuntime: async () => true,inject: async () => ambiguous,@@ -806,7 +807,7 @@ describe('SessionInputController', () => {state: 'unknown', dispatch: 'possibly_executed', reason: 'key response was lost',}))const onError = vi.fn()- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('cursor'),validateRuntime: async () => true,inject: async () => true,@@ -825,7 +826,7 @@ describe('SessionInputController', () => {it('serializes chat input behind a native runtime control lock', async () => {const injected: string[] = []- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('claude'), validateRuntime: async () => true,inject: async (_pane, content) => { injected.push(content); return true },sendKey: async () => true, onError: vi.fn(),@@ -849,7 +850,7 @@ describe('delivery correlation', () => {function setup(overrides: Partial<ConstructorParameters<typeof SessionInputController>[0]> = {}) {const onDelivery = vi.fn()const inject = vi.fn(async () => true)- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session(), validateRuntime: async () => true,inject, sendKey: async () => true, onError: vi.fn(), onDelivery, ...overrides,})@@ -982,7 +983,7 @@ describe('optional lamp delivery preserves legacy behavior', () => {// once: the second waits for the first's paste, not for the first's turn.const validations: Array<(valid: boolean) => void> = []const inject = vi.fn(async (_target: string, _text: string) => true)- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('claude'),validateRuntime: () => new Promise<boolean>(resolve => validations.push(resolve)),inject, sendKey: async () => true, onError: vi.fn(),@@ -1003,7 +1004,7 @@ describe('optional lamp delivery preserves legacy behavior', () => {it('writes messages that arrive together in the order they arrived, whichever check finishes first', async () => {const checks: Array<(valid: boolean) => void> = []const inject = vi.fn(async (_target: string, _text: string) => true)- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('codex'),validateRuntime: () => new Promise<boolean>(resolve => checks.push(resolve)),inject, sendKey: async () => true, onError: vi.fn(),@@ -1017,7 +1018,7 @@ describe('optional lamp delivery preserves legacy behavior', () => {expect(inject.mock.calls.map(([, text]) => text)).toEqual(['one', 'two', 'three'])// A write that fails does not hold up the ones behind it.const failing = vi.fn(async (_target: string, text: string) => { if (text === 'bad') throw new Error('pane gone'); return true })- const next = new SessionInputController({ getSession: () => session('codex'), validateRuntime: async () => true,+ const next = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('codex'), validateRuntime: async () => true,inject: failing, sendKey: async () => true, onError: vi.fn() })next.submit('s1', 'bad')next.submit('s1', 'good')@@ -1028,7 +1029,7 @@ describe('optional lamp delivery preserves legacy behavior', () => {it('keeps native control available during untracked runtime validation', async () => {let resolve!: (valid: boolean) => void- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('claude'),validateRuntime: () => new Promise<boolean>(done => { resolve = done }),inject: async () => true, sendKey: async () => true, onError: vi.fn(),@@ -1045,7 +1046,7 @@ describe('optional lamp delivery preserves legacy behavior', () => {it('reports the original legacy cancellation error for a vanished process', async () => {const onError = vi.fn()const sendKey = vi.fn(async () => true)- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session(), validateRuntime: async () => false,inject: async () => true, sendKey, onError,})@@ -1059,7 +1060,7 @@ describe('optional lamp delivery preserves legacy behavior', () => {vi.useFakeTimers()const onDelivery = vi.fn()const sendKey = vi.fn(async () => true)- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('commandcode'), validateRuntime: async () => true,inject: async () => true, sendKey, capture: async () => 'Thinking… esc to interrupt',onError: vi.fn(), onDelivery,@@ -1076,7 +1077,7 @@ describe('optional lamp delivery preserves legacy behavior', () => {it('does not invent rejection when a turn begins during pre-paste validation', async () => {let resolve!: (valid: boolean) => voidconst onDelivery = vi.fn()- const controller = new SessionInputController({+ const controller = new SessionInputController({ readScreen: readInlineScreen,getSession: () => session('claude'),validateRuntime: () => new Promise<boolean>(done => { resolve = done }),inject: async () => true, sendKey: async () => true, onError: vi.fn(), onDelivery,diff --git a/cli/src/lib/sessionInput.ts b/cli/src/lib/sessionInput.tsindex ed54b8b02..d180e117b 100644--- a/cli/src/lib/sessionInput.ts+++ b/cli/src/lib/sessionInput.ts@@ -1,7 +1,8 @@+import type { ScreenReader } from './screenReader.js'import { createHash } from 'crypto'import type { RegisteredSession } from './registry.js'import { sid } from './log.js'-import { isMessageHold, messageHold, messageHoldText, messageWithheldText, passingHold, type MessageHold } from './messageHold.js'+import { isMessageHold, messageHoldText, messageWithheldText, passingHold, type MessageHold } from './messageHolds.js'import { enterWithheldReason, TERMINAL_LEASE_REFUSED, type TerminalActionResult } from './terminalTypes.js'const MAX_QUEUE_ITEMS = 8@@ -105,6 +106,7 @@ interface InputState {}export interface SessionInputDeps {+ readScreen: ScreenReader/** A team-only preflight at the actual write boundary. A reason proves no paste occurred. */beforeTeamWrite?: (session: RegisteredSession) => Promise<string | null>onDelivery?: (event: SessionInputDelivery) => void@@ -645,7 +647,7 @@ export class SessionInputController {if (session.engine === 'cursor') {const capture = await this.deps.capture?.(session.agentId)if (!state.awaitingFingerprint || state.turnOpen) return- if (this.dialogOverComposer(sessionId, session, state, capture)) return+ if (await this.dialogOverComposer(sessionId, session, state, capture)) returnconst draftPending = !!capture && cursorComposerContains(capture, state.awaitingContent ?? '')if (state.ambiguousDispatch && !draftPending) {this.failAmbiguousSubmission(sessionId, state)@@ -684,7 +686,7 @@ export class SessionInputController {// claude/codex/commandcode: verify against the terminal before pressing Enter again or declaring failure.const capture = await this.deps.capture?.(session.agentId)if (!state.awaitingFingerprint || state.turnOpen) return- if (this.dialogOverComposer(sessionId, session, state, capture)) return+ if (await this.dialogOverComposer(sessionId, session, state, capture)) return// Command Code writes the user line to its transcript only once the model has finished THINKING, so// the turn_started this used to wait for can be half a minute late on a real task — and the user// watched the terminal accept the message and start working while the device claimed it had been@@ -775,8 +777,11 @@ export class SessionInputController {* agent asks means the message was taken and its turn has reached a tool; anything else leaves it* unknown, as does any delivery with a receipt, which the pane alone cannot settle.*/- private dialogOverComposer(sessionId: string, session: RegisteredSession, state: InputState, capture: string | null | undefined): boolean {- const hold = capture ? messageHold(session.engine, capture) : null+ private async dialogOverComposer(sessionId: string, session: RegisteredSession, state: InputState, capture: string | null | undefined): Promise<boolean> {+ if (capture === undefined) return false+ const screen = await this.deps.readScreen(session, capture)+ if (this.states.get(sessionId) !== state || state.turnOpen || !state.awaitingFingerprint) return true+ const hold = screen ? screen.messageHold : 'screen_unreadable'if (!hold) return falseif (state.deliveryId || (hold !== 'permission_open' && hold !== 'question_open')) {this.failAmbiguousSubmission(sessionId, state)diff --git a/cli/src/lib/teamWriteHold.ts b/cli/src/lib/teamWriteHold.tsindex d649d4719..74b174d6d 100644--- a/cli/src/lib/teamWriteHold.ts+++ b/cli/src/lib/teamWriteHold.ts@@ -1,38 +1,2 @@-import { stripVTControlCharacters } from 'node:util'-import type { RegisteredSession } from './registry.js'-import { inspectRuntimePane } from './runtimeProfileController.js'-import { parseEngineQuestionPane } from './askQuestion.js'--/**- * Conservative, read-only inspection at the serialized input write boundary. The core's: it writes a team's- * turn only into a ready composer (core/input.ts), and closes an agent only from one (lib/closeAgentService.ts).- * Here rather than in teams/ since Tab collaboration left the core's process (step 8); teams/preflight.ts says- * it as it always did.- */-export function teamWriteHold(engine: RegisteredSession['engine'], capture: string | null): string | null {- if (!capture) return 'team_waiting_unavailable'- const pane = inspectRuntimePane(engine, capture)- if (pane.dialog || parseEngineQuestionPane(engine, capture)) return 'team_waiting_user'- if (pane.draft) return 'team_waiting_draft'- if (!pane.idle) return 'team_waiting_idle'- const lines = capture.split('\n').map(stripVTControlCharacters)- if (/esc(?:ape)? to (?:interrupt|cancel|stop)/i.test(lines.slice(-15).join('\n'))) return 'team_waiting_idle'- // These composers permit an empty first line followed by a multiline human draft.- // The general profile inspector only needs the first line; team delivery needs every line.- if (['claude', 'codex', 'cursor', 'hermes'].includes(engine)) {- const index = lines.findLastIndex(line => /^\s*[›❯→]/u.test(line))- if (index < 0) return 'team_waiting_idle'- // Codex shows its styled placeholder only when the whole composer is empty.- // Its configurable model/cwd/task footer below that placeholder is not draft- // text. The profile inspector has already proved this visible prompt text is- // a placeholder; a bare first line still needs the multiline-draft check.- if (engine === 'codex' && lines[index].replace(/^\s*[›❯→]\s*/u, '').trim()) return null- for (const line of lines.slice(index + 1)) {- if (/[─━―]{8,}/u.test(line)) break- const text = line.trim()- if (!text || /^(?:\? for shortcuts|\d+% context|context:|ctrl\+|shift\+|tab to|plan mode|accept edits|bypass permissions)/i.test(text)) continue- return 'team_waiting_draft'- }- }- return null-}+/** Inline compatibility API. Supervised core reads the engine worker. */+export { teamWriteHold } from '../engines/screens.js'diff --git a/cli/src/lib/terminalActivity.ts b/cli/src/lib/terminalActivity.tsindex 5745860ea..dd5d507ef 100644--- a/cli/src/lib/terminalActivity.ts+++ b/cli/src/lib/terminalActivity.ts@@ -1,33 +1,7 @@-/** Read only an engine's live footer. No generated status words or transcript inference. */-export function terminalActivity(engine: string, screen: string | null): string | null {- return terminalActivityReading(engine, screen)?.label ?? null-}--/** Keep the live indicator as well as its label: successive changes to its- * timer/spinner prove the UI is alive; an abandoned static screen does not. */-export function terminalActivityReading(engine: string, screen: string | null): { label: string; indicator: string } | null {- if (!screen || (engine !== 'claude' && engine !== 'codex')) return null- const lines = screen.replace(/\x1b\][^\x07]*(?:\x07|\x1b\\)/g, '')- .replace(/\x1b\[[0-?]*[ -/]*[@-~]/g, '').split(/\r?\n/)- let end = lines.length- if (engine === 'claude') {- // The expanded agent tree sits BELOW the prompt and can push the live- // spinner out of the last 16 rows. Anchor to the actual input/footer pair,- // retaining a short search above it instead of scanning old output.- for (let i = lines.length - 1; i >= Math.max(0, lines.length - 64); i--) {- if (/^\s*[❯›>](?:\s|$)/u.test(lines[i]) &&- lines.slice(i + 1, i + 9).some(line => /shift\+tab to cycle|esc to interrupt/i.test(line))) {- end = i- break- }- }- }- for (let i = end - 1; i >= Math.max(0, end - 16); i--) {- const line = lines[i].trim()- const match = engine === 'codex'- ? /^[•◦⠁-⣿]\s+(.{1,60}?)\s+\([^\n]*\besc to interrupt\b[^\n]*\)\s*$/i.exec(line)- : /^[✢✳✶✻✽·*]\s+([\p{L}][\p{L}\p{N} '\u2019-]{0,55}(?:…|\.{3}))(?:\s+\([^\n]*\))?\s*$/u.exec(line)- if (match) return { label: match[1].trim().replace(/…/g, '...'), indicator: line }- }- return null+/** Inline compatibility; supervised core reads engine screen evidence. */+import { activity as claude } from '../engines/claude/activity.js'+import { activity as codex } from '../engines/codex/activity.js'+export function terminalActivityReading(engine: string, screen: string | null) {+ return !screen ? null : engine === 'claude' ? claude(screen) : engine === 'codex' ? codex(screen) : null}+export function terminalActivity(engine: string, screen: string | null): string | null { return terminalActivityReading(engine, screen)?.label ?? null }diff --git a/cli/src/lib/terminalClose.real.spec.ts b/cli/src/lib/terminalClose.real.spec.tsindex c7bfec041..1715aa3c1 100644--- a/cli/src/lib/terminalClose.real.spec.ts+++ b/cli/src/lib/terminalClose.real.spec.ts@@ -1,10 +1,11 @@+import { screenFor } from '../engines/screens.js'import { readFile, readdir, stat } from 'node:fs/promises'import { join } from 'node:path'import { expect, it, vi } from 'vitest'import { env } from '../config/env.js'import { createTerminalControl } from '../core/terminals/control.js'import { isolatedTmux } from '../testing/isolatedTmux.js'-import { CloseAgentService, inspectCloseActivity } from './closeAgentService.js'+import { CloseAgentService, inspectCloseActivity as inspectCloseReading } from './closeAgentService.js'import { registry } from './registry.js'import { AgentRestartCoordinator } from './restartAgent.js'import { SessionCheckpointStore } from './sessionCheckpoint.js'@@ -86,3 +87,7 @@ it.runIf(process.env.RUN_REAL_TMUX_DISCOVERY === '1')('saves and closes an inactawait server.close()}}, 15_000)++function inspectCloseActivity(session: Parameters<typeof inspectCloseReading>[0], capture: string | null, turnOpen: boolean | undefined, needsInput: boolean) {+ return inspectCloseReading(session, capture === null ? null : screenFor(session.engine).inspect(capture), turnOpen, needsInput)+}diff --git a/cli/src/lib/unusedCodexClose.spec.ts b/cli/src/lib/unusedCodexClose.spec.tsindex 65751180b..574bf60cb 100644--- a/cli/src/lib/unusedCodexClose.spec.ts+++ b/cli/src/lib/unusedCodexClose.spec.ts@@ -1,3 +1,4 @@+import { screenFor } from '../engines/screens.js'import { afterEach, beforeEach, expect, it, vi } from 'vitest'import { mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs'import { join } from 'node:path'@@ -5,7 +6,7 @@ import { env } from '../config/env.js'import { registry, type RegisteredSession } from './registry.js'import { stoppedAgents } from './stoppedAgents.js'import { AgentRestartCoordinator } from './restartAgent.js'-import { CloseAgentService, inspectCloseActivity } from './closeAgentService.js'+import { CloseAgentService, inspectCloseActivity as inspectCloseReading } from './closeAgentService.js'import { createStopAgentService } from './stopAgentService.js'import { SessionCheckpointStore } from './sessionCheckpoint.js'import { processRows } from './tmux.js'@@ -151,3 +152,7 @@ it.runIf(process.env.RUN_REAL_TMUX_DISCOVERY === '1')('closes only the exited clvi.unstubAllEnvs()}}, 15_000)++function inspectCloseActivity(session: Parameters<typeof inspectCloseReading>[0], capture: string | null, turnOpen: boolean | undefined, needsInput: boolean) {+ return inspectCloseReading(session, capture === null ? null : screenFor(session.engine).inspect(capture), turnOpen, needsInput)+}diff --git a/cli/src/services/inline.spec.ts b/cli/src/services/inline.spec.tsindex 11136ad8e..614c95f34 100644--- a/cli/src/services/inline.spec.ts+++ b/cli/src/services/inline.spec.ts@@ -1,3 +1,4 @@+import { screenFor } from '../engines/screens.js'import { describe, expect, it } from 'vitest'import { engineTranscriptFor } from '../engines/transcripts.js'import { liveFor } from '../engines/live.js'@@ -25,6 +26,6 @@ import { startWorkspaces } from './workspaces.js'describe('the services the core runs in its own process only when they do not run in theirs', () => {it('are their own starts, unchanged: the same services either way', () => {- expect({ ...inline }).toEqual({ engineTranscriptFor, liveFor, runtimeFor, startCommandBar, startDevices, startHandoff, startGateway, startModels, startMonitor, startOrchestrator, startProjects, startRecaps, startSearch, startSharing, startShell, startStore, startTeamsInCore, startUsage, startViewers, startWifi, startWindowNames, startWorkspaces })+ expect({ ...inline }).toEqual({ engineTranscriptFor, liveFor, runtimeFor, screenFor, startCommandBar, startDevices, startHandoff, startGateway, startModels, startMonitor, startOrchestrator, startProjects, startRecaps, startSearch, startSharing, startShell, startStore, startTeamsInCore, startUsage, startViewers, startWifi, startWindowNames, startWorkspaces })})})diff --git a/cli/src/services/inline.ts b/cli/src/services/inline.tsindex 9c7086042..2977f7afe 100644--- a/cli/src/services/inline.ts+++ b/cli/src/services/inline.ts@@ -14,6 +14,7 @@*/export { engineTranscriptFor } from '../engines/transcripts.js'export { liveFor } from '../engines/live.js'+export { screenFor } from '../engines/screens.js'export { runtimeFor } from '../engines/runtime.js'export { startGateway } from '../gateway/start.js'export { startCommandBar } from './commandBar.js'diff --git a/cli/src/testing/inlineScreen.ts b/cli/src/testing/inlineScreen.tsnew file mode 100644index 000000000..ffa1a56f4--- /dev/null+++ b/cli/src/testing/inlineScreen.ts@@ -0,0 +1,8 @@+/** Real native readers injected into unit hosts; production supervision never imports this. */+import { readerEngine } from '../engines/worker/protocol.js'+import { screenFor } from '../engines/screens.js'+import { legacyScreen } from '../lib/legacyScreen.js'+import type { ScreenReader } from '../lib/screenReader.js'+export const readInlineScreen: ScreenReader = async (session, capture) => readerEngine(session.engine)+ ? capture === null ? null : screenFor(session.engine).inspect(capture)+ : legacyScreen(session.engine, capture)diff --git a/docs/design/2026-10-08-engine-screen.md b/docs/design/2026-10-08-engine-screen.mdnew file mode 100644index 000000000..08a6e0b86--- /dev/null+++ b/docs/design/2026-10-08-engine-screen.md@@ -0,0 +1,66 @@+# Engine screen interpretation++Claude Code and Codex interpret terminal captures through `Engine.screen` in their+existing engine worker. Core captures the terminal and retains the input lease,+session binding, turn state and question lifecycle. A worker receives text and+returns facts; it has no terminal handle or authority to type, close or restart.++The facet reports the current prompt, draft, modal, question, message/team hold,+activity indicator and stopped-goal evidence. The engine folders own native+composers, takeover screens and activity footers. Shared ANSI and numbered-dialog+mechanics remain in `engines/kit`; other engines retain their existing readers.+Recorded screen fixtures are unchanged.++## Consumers++- Message admission checks the screen before a paste and again before Enter.+ A permission prompt appearing between those operations still withholds Enter.+- Question polling and answers await engine evidence. An unavailable reader is+ not an empty pane: existing questions remain open, and no answer is typed.+- Model control and retargeting use the reported pane for their preflight.+- Close checks and activity sampling consume reported facts. Core decides whether+ evidence establishes inactivity and whether a close is still authorized.++The question watcher keeps one pending capture/interpretation per session. Stop,+rebind and replacement invalidate it. Before a Harness prompt is typed, its+already-read question view becomes the next turn's baseline; a second late read+cannot mistake that turn's own question for a preceding one.++## Private transport and failure++`engine_screen_capabilities` and `engine_screen_read` are private methods on the+existing engine links. They require core's local owner identity with no client+connection. They are absent from the public client router. Version 1 allows:++| Bound | Value |+| --- | --- |+| Capture | 256 KiB of UTF-8 |+| Reply | 1 MiB including the envelope |+| Concurrent reads | 8 per engine, with at most 64 waiting in FIFO order |+| Core deadline | 1 second including queueing, capability negotiation and worker startup |++Core validates reply fields, pane flags, question rows and hold values. Each+result is fenced to the connection generation and the session/process/terminal+identity observed before sending it. No screen is cached across calls. A timeout,+malformed reply or changed identity supplies no evidence. Message and question+writes fail closed; close/activity checks remain unknown. The existing bounded+input queue may retry an unreadable screen, without typing anything first.++Worker startup and async loading have a deadline and recycle the worker on a+timeout. A synchronous parser stall is contained by core's deadline and the+master's existing heartbeat/restart policy. It can cost that engine's other+facets, never core's event loop or the other engine worker.++The parent-bound `HARNESSD_ENGINE_SCREEN=<master pid>:1` selects supervision.+Explicit inline mode and older masters select compatibility during composition.+A failed worker never selects an inline parser. No worker starts for an empty+core merely to install this facet.++## Remaining engine isolation++This moves the screen reads described above; the full engine boundary remains in progress. Native model+picker drivers in `lib/runtimeControl.ts`, answer key selection and fingerprint+normalization in `lib/questionController.ts`, submission verification, hooks and+installers, launch/discovery/resume and native control connections still need+their own engine facets/process boundaries. Core must retain effect authority+and identity fencing when those drivers move.