Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

Isolate Claude Code and Codex screen interpretation (#1027)

Dee668048acdf5f
diff --git a/cli/AGENTS.md b/cli/AGENTS.md
index 5d0969a46..2058f59b0 100644
--- a/cli/AGENTS.md
+++ b/cli/AGENTS.md
@@ -25,7 +25,7 @@ they show as `harnessd`, `harnessd-core` and `harnessd-<process>` (`src/harnessd
|---|---|---|
| master | supervision only (`src/harnessd/`) | by `harness start`, `harness start -f`, or launchd or systemd after `harness service install` |
| core | sessions (`src/core/`) | always |
-| engine-claude, engine-codex | history, last-turn readers, live transcript parsers and runtime profiles/catalogs | on demand: an attach or read for that engine |
+| engine-claude, engine-codex | history, last-turn readers, live transcript parsers, runtime profiles/catalogs and screen interpretation | on demand: an attach or read for that engine |
| search | session search | always |
| viewers | the harness viewers, their remote streams and rendered surfaces, and the Store | always |
| edge | the shell service, workspaces, usage, the monitor, the project readers, the change-agent handoff, the recaps | always |
diff --git a/cli/e2e/engineReaders.e2e.ts b/cli/e2e/engineReaders.e2e.ts
index 47b1192ed..463aaebed 100644
--- a/cli/e2e/engineReaders.e2e.ts
+++ b/cli/e2e/engineReaders.e2e.ts
@@ -59,7 +59,7 @@ describe('Claude Code and Codex readers in their own processes', () => {
expect(readerPid(d, 'claude')).toBeUndefined()
expect(readerPid(d, 'codex')).toBeUndefined()
for (const type of ['engine_history_page', 'engine_last_turn', 'engine_live_capabilities', 'engine_live_prepare',
- 'engine_live_read', 'engine_live_part', 'engine_live_close', 'engine_live_forget', 'engine_runtime_capabilities', 'engine_runtime_read']) {
+ 'engine_live_read', 'engine_live_part', 'engine_live_close', 'engine_live_forget', 'engine_runtime_capabilities', 'engine_runtime_read', 'engine_screen_capabilities', 'engine_screen_read']) {
const answer = await c.request(type, { version: 1, session: { transcriptPath: '/etc/passwd' } })
expect(answer.error).toBe('UNSUPPORTED')
}
@@ -112,6 +112,43 @@ describe('Claude Code and Codex readers in their own processes', () => {
expect(alive(bound.processIdentity.pid)).toBe(true)
})
+ it.each(['claude', 'codex'] as const)('%s refuses terminal writes and retains its question while its screen worker is frozen', async engine => {
+ const { d, c } = await fresh({ HARNESSD_SERVICE_HEARTBEAT_TIMEOUT_MS: '60000', HARNESSD_SERVICE_STOP_GRACE_MS: '100' })
+ const agent = await create(d, c, engine)
+ const asked = c.next(frame => frame.type === 'commander_question' && frame.agentId === agent.id, 30_000, 'permission question')
+ c.send('message', { agentId: agent.id, content: '!permit printf screen-worker' })
+ const question = (await asked).payload!
+ const shaped = question.questions[0]
+ const bound = JSON.parse(readFileSync(join(d.dataDir, 'registry.json'), 'utf8')).find((row: any) => row.agentId === agent.id)
+ const pid = readerPid(d, engine)!, core = d.corePid(), from = c.frames.length
+ const answer = async () => {
+ const result = c.next(frame => frame.type === 'question_response_result' && frame.payload?.requestId === question.requestId, 10_000, 'answer result')
+ c.send('question_response', { agentId: agent.id, requestId: question.requestId, answers: { [shaped.q]: shaped.options[2] } })
+ return (await result).payload!
+ }
+ process.kill(pid, 'SIGSTOP')
+ try {
+ const started = Date.now()
+ expect((await answer()).error).toBe('ANSWER_FAILED')
+ expect(Date.now() - started).toBeLessThan(5_000)
+ const refused = c.next(frame => frame.type === 'error' && frame.agentId === agent.id, 25_000, 'unreadable-screen refusal')
+ c.send('message', { agentId: agent.id, content: 'must-not-enter-the-permission' })
+ expect((await refused).payload?.message).toContain('screen could not be read')
+ const pane = await d.capture(bound.tmuxPane)
+ expect(pane).not.toContain('must-not-enter-the-permission')
+ expect(pane).not.toContain('did not run printf screen-worker')
+ expect(c.frames.slice(from).filter(frame => frame.type === 'commander_question_close' && frame.agentId === agent.id)).toHaveLength(0)
+ expect((await rows(c)).some(row => row.id === agent.id)).toBe(true)
+ expect(d.corePid()).toBe(core)
+ } finally { process.kill(pid, 'SIGCONT') }
+ const ended = c.next(frame => frame.type === 'turn_ended' && frame.agentId === agent.id, 30_000, 'declined permission')
+ expect((await answer()).error).toBeUndefined()
+ await ended
+ await turn(c, agent, `screen-recovered-${engine}`)
+ expect(alive(bound.processIdentity.pid)).toBe(true)
+ expect(d.coresStarted()).toBe(1)
+ })
+
it('contains a killed or frozen reader, bounds failed reads, and recovers without restarting core or either CLI', async () => {
const { d, c } = await fresh({ HARNESSD_SERVICE_HEARTBEAT_TIMEOUT_MS: '4000', HARNESSD_SERVICE_STOP_GRACE_MS: '100' })
const agents = await Promise.all((['claude', 'codex'] as const).map(engine => create(d, c, engine)))
@@ -217,7 +254,11 @@ describe('Claude Code and Codex readers in their own processes', () => {
expect(await history(c, agent)).toMatchObject({ error: 'ENGINE_UNAVAILABLE', retryable: true })
const first = c.frames.length
const ended = c.next(frame => frame.type === 'turn_ended' && frame.agentId === agent.id, 60_000, 'queued turn after worker recovery')
- c.send('message', { agentId: agent.id, content: 'worker-is-parked' })
+ // A person can still use the independent CLI while its worker is parked.
+ // Harness messages now require screen evidence and are withheld in an
+ // outage (covered above); type through this fixture's private terminal.
+ await d.tmux.run('send-keys', '-t', registered().tmuxPane, '-l', 'worker-is-parked')
+ await d.tmux.run('send-keys', '-t', registered().tmuxPane, 'Enter')
await until('the independent CLI to finish writing its turn', () => {
const file = registered().transcriptPath
return file && readFileSync(file, 'utf8').split('\n').some(line => {
diff --git a/cli/src/architecture.spec.ts b/cli/src/architecture.spec.ts
index 6ca28dd62..dc6a1d103 100644
--- a/cli/src/architecture.spec.ts
+++ b/cli/src/architecture.spec.ts
@@ -143,6 +143,9 @@ const SERVICE_MAY_IMPORT: Record<string, string> = {
/** What is not the core's, by path: each goes to a service or its own process, in the plan's order. */
const EDGE: RegExp[] = [
+ /^engines\/(claude|codex)\/(screen|composer|activity|stoppedGoal)\.ts$/,
+ /^lib\/(askQuestion|runtimeProfileController|composerScreen|teamWriteHold|messageHold|terminalActivity|codexTurnRecovery)\.ts$/,
+ /^engines\/screens\.ts$/,
// The pilot reader implementations and their host are never loaded by supervised core.
/^engines\/(worker\/process|transcripts|(claude|codex)\/(transcript|\w+ReaderProcess))\.ts$/,
/^engines\/(runtime|(claude|codex)\/runtimeProfile)\.ts$/, /^lib\/runtimeProfile\.ts$/,
@@ -216,6 +219,12 @@ describe('the daemon\'s shape', () => {
}
})
+ it('screen transport and input authority do not load native screen implementations', () => {
+ for (const entry of ['core/input.ts', 'core/questions.ts', 'core/engines/screens.ts', 'core/engines/screenTransport.ts']) {
+ expect([...closureOf(entry).keys()].filter(file => /^engines\/(claude|codex)\//.test(file)), entry).toEqual([])
+ }
+ })
+
it('the gateway reaches the core only through core/api.ts: never a core module, the registry, cli.ts or the socket', () => {
// It speaks to the core through GatewayPort and GatewayEvents alone, so that it can run in a process of
// its own (step 10, R2) without taking any of the core with it.
diff --git a/cli/src/core/AGENTS.md b/cli/src/core/AGENTS.md
index fa4688c54..c576f414c 100644
--- a/cli/src/core/AGENTS.md
+++ b/cli/src/core/AGENTS.md
@@ -33,10 +33,11 @@ questions. Everything else is a service ([../services/AGENTS.md](../services/AGE
- `terminals/`: who controls a pane (the control lease), opening a terminal with a literal argv (`open.ts`),
and the requests about a terminal itself (`requests.ts`: `terminal_info`, `theme_set`).
- `engines/`: hook coordination, reader ports and live worker coordination. Claude Code/Codex
- history, last-turn reads, live parsing and runtime profile interpretation/catalogs run in supervised workers; core passes snapshots,
+ history, last-turn reads, live parsing, runtime profiles/catalogs and screen interpretation run in supervised workers; core passes snapshots,
validates bounded replies, retains acknowledged cursors and owns binding/turn state. The explicit
inline/older-master compatibility path is the only live parser or runtime profile implementation
- composition in core. Core keeps accepted profiles and control authority; native picker drivers
+ composition in core. Screen reads are bounded and fenced to the worker connection and session binding
+ ([screen boundary](../../../docs/design/2026-10-08-engine-screen.md)). Core keeps accepted profiles and control authority; native picker drivers
remain to migrate. See [runtime profiles](../../../docs/design/2026-10-08-engine-runtime.md).
- `input.ts`, `questions.ts`: messages into a pane; an agent's question and its answer.
`deviceInput.ts`: the pane writer lock every write takes, and a device's queued turns behind it.
diff --git a/cli/src/core/agents/close.spec.ts b/cli/src/core/agents/close.spec.ts
index 0b9424e1c..6a8573934 100644
--- a/cli/src/core/agents/close.spec.ts
+++ b/cli/src/core/agents/close.spec.ts
@@ -1,3 +1,5 @@
+import { readInlineScreen } from '../../testing/inlineScreen.js'
+import { screenFor } from '../../engines/screens.js'
import { afterEach, describe, expect, it, vi } from 'vitest'
import { inspectCloseActivity, type CloseAgentServiceDeps } from '../../lib/closeAgentService.js'
import { projectDisplayName, type RegisteredSession } from '../../lib/registry.js'
@@ -17,6 +19,7 @@ const row = (over: Partial<RegisteredSession> = {}) =>
function setup(over: Partial<ClosingDeps> = {}, advertised: RegisteredSession[] = []) {
const deps: ClosingDeps = {
+ readScreen: readInlineScreen,
registry: { advertised: vi.fn(() => advertised) } as unknown as ClosingDeps['registry'],
cleanupTabs: { refresh: vi.fn(async () => {}), isHidden: vi.fn(() => true), assertHidden: vi.fn(async () => {}) },
watcher: { pollSession: vi.fn(async () => {}) } as unknown as ClosingDeps['watcher'],
@@ -53,7 +56,7 @@ describe('closing agents no window shows', () => {
expect(deps.captureTerminal).toHaveBeenCalledWith('a1', 80)
expect(deps.sessionTurnState).toHaveBeenCalledWith('s1')
expect(deps.openQuestions.has).toHaveBeenCalledWith('s1')
- expect(inspectCloseActivity).toHaveBeenCalledWith(row(), 'screen', false, false)
+ expect(inspectCloseActivity).toHaveBeenCalledWith(row(), screenFor('claude').inspect('screen'), false, false)
})
it('with no session yet, has no lines to read', async () => {
diff --git a/cli/src/core/agents/close.ts b/cli/src/core/agents/close.ts
index bd7d6b202..60c7fd369 100644
--- a/cli/src/core/agents/close.ts
+++ b/cli/src/core/agents/close.ts
@@ -1,3 +1,4 @@
+import type { ScreenReader } from '../../lib/screenReader.js'
/**
* Closing agents that no window shows: the close service (now, once idle, after the task) and the
* cleanup preview that lists what a close would take, each with what it is doing right now.
@@ -12,6 +13,7 @@ import type { TerminalBackendCoordinator } from '../../lib/terminalBackendCoordi
import type { Watcher } from '../../watcher/watcher.js'
export interface ClosingDeps {
+ readScreen: ScreenReader
registry: typeof registry
cleanupTabs: Pick<OpenTabProtection, 'refresh' | 'isHidden' | 'assertHidden'>
watcher: Pick<Watcher, 'pollSession'>
@@ -26,7 +28,7 @@ export interface ClosingDeps {
}
export function createAgentClosing({
- registry, cleanupTabs, watcher, captureTerminal, sessionTurnState, openQuestions, terminals, sessionCheckpoints,
+ readScreen, registry, cleanupTabs, watcher, captureTerminal, sessionTurnState, openQuestions, terminals, sessionCheckpoints,
stopAgent, announceSession,
}: ClosingDeps) {
const closeAgentService = new CloseAgentService({
@@ -35,7 +37,7 @@ export function createAgentClosing({
activity: async s => {
if (s.sessionId) await watcher.pollSession(s.sessionId)
const screen = await captureTerminal(s.agentId, 80)
- return inspectCloseActivity(s, screen, sessionTurnState(s.sessionId), openQuestions.has(s.sessionId))
+ return inspectCloseActivity(s, await readScreen(s, screen), sessionTurnState(s.sessionId), openQuestions.has(s.sessionId))
},
checkpoint: async (s, phase) => {
const captured = phase === 'before' ? await terminals.captureRetained(s, { historyLines: 2000 }) : null
diff --git a/cli/src/core/agents/discovery.ts b/cli/src/core/agents/discovery.ts
index fd42ababd..3e957f663 100644
--- a/cli/src/core/agents/discovery.ts
+++ b/cli/src/core/agents/discovery.ts
@@ -9,7 +9,7 @@
*/
import { isTerminalEngine } from '../../engines/types.js'
import type { AutonomousDeviceInput } from '../deviceInput.js'
-import type { QuestionWatcher } from '../../lib/askQuestion.js'
+import type { QuestionWatcher } from '../../lib/questionController.js'
import { sameGridAssignment } from '../../lib/gridAssignment.js'
import { sid } from '../../lib/log.js'
import type { registry, RegisteredSession } from '../../lib/registry.js'
diff --git a/cli/src/core/agents/retarget.spec.ts b/cli/src/core/agents/retarget.spec.ts
index b59cfd8be..ef00cffa6 100644
--- a/cli/src/core/agents/retarget.spec.ts
+++ b/cli/src/core/agents/retarget.spec.ts
@@ -45,6 +45,7 @@ const agent = (over: Partial<RegisteredSession> = {}): RegisteredSession => ({
function setup(row: RegisteredSession | null = agent(), over: Partial<RetargetDeps> = {}) {
const release = vi.fn()
const deps: RetargetDeps = {
+ readScreen: async (session, capture) => ({ pane: inspectRuntimePane(session.engine, capture ?? ''), question: null, messageHold: null, teamHold: null, activity: null, busy: false, stoppedGoal: false }),
purgeBusy: vi.fn(() => false),
tmuxBackend: { clearEnv: vi.fn(async () => ({ state: 'succeeded' })) } as unknown as RetargetDeps['tmuxBackend'],
registry: {
diff --git a/cli/src/core/agents/retarget.ts b/cli/src/core/agents/retarget.ts
index 872b4aba6..8c2270a25 100644
--- a/cli/src/core/agents/retarget.ts
+++ b/cli/src/core/agents/retarget.ts
@@ -24,7 +24,7 @@ import type { registry, RegisteredSession } from '../../lib/registry.js'
import { bypassPermissionFor, restartAgent } from '../../lib/restartAgent.js'
import type { RuntimeProfileManager } from '../../lib/runtimeProfile.js'
import { parseRuntimeProfile } from '../../lib/runtimeProfileWire.js'
-import { inspectRuntimePane } from '../../lib/runtimeProfileController.js'
+import type { ScreenReader } from '../../lib/screenReader.js'
import type { TerminalAgentReconciler } from '../../lib/terminalAgentReconciler.js'
import { terminalRouteKey } from '../../lib/terminalRuntime.js'
import type { TmuxRuntimeRef } from '../../lib/terminalTypes.js'
@@ -39,6 +39,7 @@ type LaunchHelpers = ReturnType<typeof createLaunchHelpers>
type RetargetAgent = NonNullable<BackendSocket['onRetargetAgent']>
export interface RetargetDeps {
+ readScreen: ScreenReader
/** Whether a purge holds this agent (PurgeAgentService.busy). */
purgeBusy: (agentId: string) => boolean | undefined
tmuxBackend: TmuxBackend | null
@@ -59,7 +60,7 @@ export interface RetargetDeps {
}
export function createAgentRetargeter({
- purgeBusy, tmuxBackend, registry, runtimeProfiles, launchOverridesDeps, captureTerminal, acquireTerminalControl,
+ readScreen, purgeBusy, tmuxBackend, registry, runtimeProfiles, launchOverridesDeps, captureTerminal, acquireTerminalControl,
relaunchOverrides, downgradedPermission, agentReconciler, restartJobs, paneSwapDeps, liveBypassPermission,
announceSession, opencodeDb,
}: RetargetDeps) {
@@ -144,7 +145,7 @@ export function createAgentRetargeter({
// the grid.
const capture = await captureTerminal(session.agentId, 100)
if (!capture) return { ok: false, error: 'TMUX_FAILED' }
- if (!inspectRuntimePane(session.engine, capture).idle) return { ok: false, error: 'AGENT_BUSY' }
+ if (!(await readScreen(session, capture))?.pane.idle) return { ok: false, error: 'AGENT_BUSY' }
// Nothing may type into the pane while it is being replaced.
if (restartJobs.busy(session.agentId)) return { ok: false, error: 'AGENT_BUSY' }
const release = acquireTerminalControl(session.agentId)
diff --git a/cli/src/core/agents/update.ts b/cli/src/core/agents/update.ts
index 4013703d2..a5b16b449 100644
--- a/cli/src/core/agents/update.ts
+++ b/cli/src/core/agents/update.ts
@@ -10,7 +10,7 @@ import type { AgentFrame } from '../../lib/agentFrame.js'
import type { CloseAgentService } from '../../lib/closeAgentService.js'
import { preview, sid } from '../../lib/log.js'
import type { registry, RegisteredSession } from '../../lib/registry.js'
-import { RuntimeProfileControlError, type RuntimeProfileErrorCode } from '../../lib/runtimeProfileController.js'
+import { RuntimeProfileControlError, type RuntimeProfileErrorCode } from '../../lib/runtimeControl.js'
/** An `agent_update {opened: true}` for an agent opened less than this long ago is answered but not
* stamped or broadcast: a person flicking between two tabs, or two apps opening the same agent at
diff --git a/cli/src/core/deviceInput.spec.ts b/cli/src/core/deviceInput.spec.ts
index fdce70610..e568187a9 100644
--- a/cli/src/core/deviceInput.spec.ts
+++ b/cli/src/core/deviceInput.spec.ts
@@ -1,3 +1,4 @@
+import { readInlineScreen } from '../testing/inlineScreen.js'
import { CodexNormalizer } from '../engines/codex/normalizer.js'
import { lineToEvents, newTurnState, type LiveEvent } from '../lib/normalize.js'
import { randomUUID } from 'node:crypto'
@@ -29,7 +30,7 @@ function fixture(engine: 'claude' | 'codex' | 'commandcode' = 'codex') {
submit: (id, text, delivery) => controller.submit(id, text, delivery), cancelDelivery: id => controller.cancelDelivery(id),
stop: async () => true, answer: async () => true, recent: () => [], emit: frame => events.push(frame),
})
- const legacy: SessionInputController = new SessionInputController({
+ const legacy: SessionInputController = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => available ? session : undefined, validateRuntime: async () => available,
inject: (id, text) => controller.legacyWrite(id, () => inject(id, text)),
sendKey: (id, key) => controller.legacyWrite(id, () => sendKey(id, key)), capture, onError: vi.fn(),
diff --git a/cli/src/core/deviceInput.ts b/cli/src/core/deviceInput.ts
index c6f5b8fa0..dcbf0be17 100644
--- a/cli/src/core/deviceInput.ts
+++ b/cli/src/core/deviceInput.ts
@@ -12,7 +12,7 @@ import { createHash } from 'node:crypto'
import type { RegisteredSession } from '../lib/registry.js'
import type { SessionInputDelivery } from '../lib/sessionInput.js'
import type { LiveEvent } from '../lib/normalize.js'
-import { isMessageHold } from '../lib/messageHold.js'
+import { isMessageHold } from '../lib/messageHolds.js'
import { enterWithheldReason, type TerminalActionResult } from '../lib/terminalTypes.js'
const NATIVE = new Set(['claude', 'codex'])
diff --git a/cli/src/core/engines/screenTransport.spec.ts b/cli/src/core/engines/screenTransport.spec.ts
new file mode 100644
index 000000000..1fd305c5c
--- /dev/null
+++ b/cli/src/core/engines/screenTransport.spec.ts
@@ -0,0 +1,100 @@
+import { afterEach, describe, expect, it, vi } from 'vitest'
+import { screenFor } from '../../engines/screens.js'
+import { engineScreenRequests } from '../../engines/worker/screenRequests.js'
+import { SCREEN_CAPABILITIES, SCREEN_IN_FLIGHT, SCREEN_QUEUED, SCREEN_READ, SCREEN_REPLY_BYTES, SCREEN_WAIT_MS } from '../../engines/worker/screenProtocol.js'
+import { createScreenTransport, type ScreenTransport } from './screenTransport.js'
+
+const capture = '────────────\n❯\n────────────\n? for shortcuts'
+const capability = { version: 1, screen: 1, engine: 'claude' }
+const answer = { version: 1, answer: screenFor('claude').inspect(capture) }
+function setup() {
+ const call = vi.fn(async (_service: string, method: string, _payload: Record<string, unknown>, _wait: number): Promise<Record<string, unknown>> => method === SCREEN_CAPABILITIES ? capability : answer)
+ const transport = createScreenTransport({ call }); transport.connected('engine-claude')
+ return { transport, call, read: () => transport.read('claude', capture) }
+}
+afterEach(() => { vi.useRealTimers(); vi.restoreAllMocks() })
+describe('screen transport', () => {
+ it('negotiates each connection and decodes real worker evidence', async () => {
+ const requests = engineScreenRequests('claude')
+ let transport: ScreenTransport
+ const call = vi.fn(async (service, method, payload) => {
+ if (call.mock.calls.length === 1) transport.connected(service)
+ return await requests[method]({ ...payload, requestId: `core-route-${call.mock.calls.length}` }, { owner: true, local: true })
+ })
+ transport = createScreenTransport({ call })
+ expect(await transport.read('claude', capture)).toEqual(answer.answer)
+ await transport.read('claude', capture)
+ expect(call.mock.calls.map(c => c[1])).toEqual([SCREEN_CAPABILITIES, SCREEN_READ, SCREEN_READ])
+ transport.disconnected('engine-claude'); transport.connected('engine-claude')
+ await transport.read('claude', capture)
+ expect(call.mock.calls.filter(c => c[1] === SCREEN_CAPABILITIES)).toHaveLength(2)
+ transport.connected('other'); transport.disconnected('other')
+ await expect(transport.read('other', capture)).rejects.toThrow('ENGINE_INVALID_REQUEST')
+ await expect(transport.read('claude', 'x'.repeat(300_000))).rejects.toThrow('ENGINE_INVALID_REQUEST')
+ })
+ it('bounds requests, fences replaced connections, and releases slots', async () => {
+ const t = setup(); await t.read()
+ let finish!: (value: typeof answer) => void
+ const pending = new Promise<typeof answer>(resolve => { finish = resolve })
+ t.call.mockImplementation(() => pending)
+ const tasks = Array.from({ length: SCREEN_IN_FLIGHT + SCREEN_QUEUED }, () => t.read())
+ await expect(t.read()).rejects.toThrow('ENGINE_BUSY')
+ t.transport.disconnected('engine-claude'); t.transport.connected('engine-claude')
+ finish(answer)
+ for (const task of tasks) await expect(task).rejects.toThrow('ENGINE_STALE_REPLY')
+ t.call.mockImplementation(async (_s, method) => method === SCREEN_CAPABILITIES ? capability : answer)
+ expect(await t.read()).toEqual(answer.answer)
+ })
+
+ it('serves a burst of pane polls fairly and expires queued work without sending it late', async () => {
+ const t = setup(); await t.read()
+ const burst = await Promise.all(Array.from({ length: 24 }, () => t.read()))
+ expect(burst).toHaveLength(24)
+ expect(burst.every(value => value.messageHold === null)).toBe(true)
+ vi.useFakeTimers()
+ t.call.mockClear().mockImplementation(() => new Promise(() => {}))
+ const waiting = Array.from({ length: 24 }, () => t.read().catch(error => error.code))
+ await vi.advanceTimersByTimeAsync(SCREEN_WAIT_MS + 1)
+ expect(await Promise.all(waiting)).toEqual(Array(24).fill('ENGINE_UNAVAILABLE'))
+ expect(t.call).toHaveBeenCalledTimes(SCREEN_IN_FLIGHT)
+ })
+ it('enforces one deadline across cold negotiation and a read, including a hung link', async () => {
+ const t = setup(); let time = 10
+ vi.spyOn(performance, 'now').mockImplementation(() => time)
+ t.call.mockImplementation(async (_s, method, _p, wait) => {
+ if (method === SCREEN_CAPABILITIES) { expect(wait).toBe(SCREEN_WAIT_MS); time += 800; return capability }
+ expect(wait).toBe(200); return answer
+ })
+ await t.read()
+ t.call.mockImplementation(async () => { time += SCREEN_WAIT_MS + 1; return answer })
+ await expect(t.read()).rejects.toThrow('ENGINE_UNAVAILABLE')
+ const noTime = setup(); let calls = 0
+ vi.spyOn(performance, 'now').mockImplementation(() => ++calls === 1 ? 0 : SCREEN_WAIT_MS)
+ await expect(noTime.read()).rejects.toThrow('ENGINE_UNAVAILABLE')
+ expect(noTime.call).not.toHaveBeenCalled()
+ vi.restoreAllMocks(); vi.useFakeTimers()
+ const hung = setup(); hung.call.mockImplementation(() => new Promise(() => {}))
+ const result = expect(hung.read()).rejects.toThrow('ENGINE_UNAVAILABLE')
+ await vi.advanceTimersByTimeAsync(SCREEN_WAIT_MS)
+ await result
+ })
+ it('rejects wrong capabilities, errors and malformed or oversized answers', async () => {
+ for (const reply of [{ ...capability, screen: 2 }, { ...capability, engine: 'codex' }, { ...capability, version: 2 },
+ { version: 1, error: 'unexpected' }, { version: 1, answer: {} }]) {
+ const t = setup(); t.call.mockResolvedValue(reply)
+ await expect(t.read()).rejects.toThrow('ENGINE_INVALID_REPLY')
+ }
+ for (const error of ['SERVICE_FAILED', 'SERVICE_UNAVAILABLE', 'ENGINE_BUSY']) {
+ const t = setup(); t.call.mockResolvedValue({ version: 1, error })
+ await expect(t.read()).rejects.toThrow(error === 'ENGINE_BUSY' ? error : 'ENGINE_UNAVAILABLE')
+ }
+ const t = setup(); await t.read()
+ t.call.mockRejectedValueOnce(new Error('lost'))
+ await expect(t.read()).rejects.toThrow('ENGINE_UNAVAILABLE')
+ t.call.mockResolvedValueOnce({ version: 1, answer: {} })
+ await expect(t.read()).rejects.toThrow('ENGINE_INVALID_REPLY')
+ t.call.mockResolvedValueOnce({ ...answer, extra: 'x'.repeat(SCREEN_REPLY_BYTES) })
+ await expect(t.read()).rejects.toThrow('ENGINE_INVALID_REPLY')
+ expect(await t.read()).toEqual(answer.answer)
+ })
+})
diff --git a/cli/src/core/engines/screenTransport.ts b/cli/src/core/engines/screenTransport.ts
new file mode 100644
index 000000000..d47357e7a
--- /dev/null
+++ b/cli/src/core/engines/screenTransport.ts
@@ -0,0 +1,75 @@
+/** Core's screen port: plain snapshots, a single deadline, and connection-generation fencing. */
+import type { ScreenReading } from '../../engines/facets/screen.js'
+import { EngineReadError, readerEngine, READER_ERRORS, READER_SERVICES, type ReaderEngine } from '../../engines/worker/protocol.js'
+import { screenReading, screenCapture, SCREEN_CAPABILITIES, SCREEN_IN_FLIGHT, SCREEN_QUEUED, SCREEN_READ, SCREEN_REPLY_BYTES,
+ SCREEN_VERSION, SCREEN_WAIT_MS } from '../../engines/worker/screenProtocol.js'
+
+export interface ScreenTransportDeps {
+ call(service: string, method: string, payload: Record<string, unknown>, waitMs: number): Promise<Record<string, unknown>>
+}
+
+export function createScreenTransport(deps: ScreenTransportDeps) {
+ const states = new Map(Object.values(READER_SERVICES).map(service => [service as string,
+ { generation: 0, connected: false, capable: false, pending: 0, waiting: [] as Array<() => void> }]))
+ const fail = (code: EngineReadError['code']): never => { throw new EngineReadError(code) }
+ return {
+ connected(service: string): void {
+ const state = states.get(service)
+ if (state) { state.generation++; state.connected = true; state.capable = false }
+ },
+ disconnected(service: string): void {
+ const state = states.get(service)
+ if (state) { state.generation++; state.connected = false; state.capable = false }
+ },
+ async read(engine: string, capture: string): Promise<ScreenReading> {
+ if (!readerEngine(engine) || !screenCapture(capture)) fail('ENGINE_INVALID_REQUEST')
+ const service = READER_SERVICES[engine as ReaderEngine], state = states.get(service)!
+ const generation = state.generation + (state.connected ? 0 : 1)
+ const deadline = performance.now() + SCREEN_WAIT_MS
+ // A poll tick can capture dozens of panes together. Refusing every read
+ // after the first eight starves the same later sessions on every tick.
+ // FIFO admission is bounded, and spends the caller's original deadline.
+ // Earlier active reads have the same hard budget and release their slots
+ // even on a hung link; expired queued reads send nothing and release next.
+ if (state.pending >= SCREEN_IN_FLIGHT) {
+ if (state.waiting.length >= SCREEN_QUEUED) fail('ENGINE_BUSY')
+ await new Promise<void>(resolve => { state.waiting.push(resolve) })
+ } else state.pending++
+ const call = async (method: string, payload: Record<string, unknown>): Promise<Record<string, unknown>> => {
+ const remaining = Math.ceil(deadline - performance.now())
+ if (remaining <= 0) fail('ENGINE_UNAVAILABLE')
+ // Service startup has its own wait. Enforce one wall-clock budget here,
+ // including a worker that never answers or a link that loses its timer.
+ let timer: ReturnType<typeof setTimeout> | undefined
+ let reply: Record<string, unknown>
+ try {
+ reply = await Promise.race([deps.call(service, method, { version: SCREEN_VERSION, ...payload }, remaining),
+ new Promise<never>((_, reject) => { timer = setTimeout(() => reject(new EngineReadError('ENGINE_UNAVAILABLE')), remaining) })])
+ } finally { clearTimeout(timer) }
+ if (performance.now() > deadline) fail('ENGINE_UNAVAILABLE')
+ if (reply.error === 'SERVICE_FAILED' || reply.error === 'SERVICE_UNAVAILABLE') fail('ENGINE_UNAVAILABLE')
+ if (state.generation !== generation) fail('ENGINE_STALE_REPLY')
+ if (reply.version !== SCREEN_VERSION) fail('ENGINE_INVALID_REPLY')
+ if (reply.error !== undefined) fail(READER_ERRORS.includes(reply.error as EngineReadError['code']) ? reply.error as EngineReadError['code'] : 'ENGINE_INVALID_REPLY')
+ return reply
+ }
+ try {
+ if (!state.capable) {
+ const capability = await call(SCREEN_CAPABILITIES, {})
+ if (capability.screen !== SCREEN_VERSION || capability.engine !== engine) fail('ENGINE_INVALID_REPLY')
+ state.capable = true
+ }
+ const reply = await call(SCREEN_READ, { capture })
+ if (Buffer.byteLength(JSON.stringify(reply)) > SCREEN_REPLY_BYTES || !screenReading(reply.answer)) fail('ENGINE_INVALID_REPLY')
+ return reply.answer as ScreenReading
+ } catch (error) { throw error instanceof EngineReadError ? error : new EngineReadError('ENGINE_UNAVAILABLE') }
+ finally {
+ const next = state.waiting.shift()
+ if (next) next()
+ else state.pending--
+ }
+ },
+ }
+}
+
+export type ScreenTransport = ReturnType<typeof createScreenTransport>
diff --git a/cli/src/core/engines/screens.spec.ts b/cli/src/core/engines/screens.spec.ts
new file mode 100644
index 000000000..df7694368
--- /dev/null
+++ b/cli/src/core/engines/screens.spec.ts
@@ -0,0 +1,54 @@
+import { describe, expect, it, vi } from 'vitest'
+import { screenFor } from '../../engines/screens.js'
+import type { RegisteredSession } from '../../lib/registry.js'
+import { createScreens } from './screens.js'
+
+const capture = '────────────\n❯\n────────────\n? for shortcuts'
+const row = () => ({ agentId: 'agent', sessionId: 'session', engine: 'claude', active: true,
+ runtimes: [], processIdentity: { pid: 10, startMarker: 'one' } }) as unknown as RegisteredSession
+function setup() {
+ let current: RegisteredSession | undefined = row()
+ const answer = screenFor('claude').inspect(capture)
+ const remote = vi.fn(async () => answer), inline = vi.fn(() => answer), handles = vi.fn(() => true)
+ return { screen: createScreens({ resolve: () => current, transport: { read: remote }, inline, handles }),
+ remote, inline, handles, answer, row: current, replace: (next?: RegisteredSession) => { current = next } }
+}
+describe('core screen evidence', () => {
+ it('reads a bound worker without retaining or reusing its screen', async () => {
+ const t = setup()
+ expect(await t.screen.read(t.row, capture)).toEqual(t.answer)
+ expect(await t.screen.question(t.row, capture)).toBeNull()
+ expect(t.remote).toHaveBeenCalledTimes(2)
+ expect(t.inline).not.toHaveBeenCalled()
+ t.handles.mockReturnValue(false)
+ expect(await t.screen.read(t.row, capture)).toEqual(t.answer)
+ expect(t.inline).toHaveBeenCalledWith('claude', capture)
+ t.inline.mockReturnValueOnce(undefined as never)
+ expect(await t.screen.read(t.row, capture)).toBeNull()
+ })
+ it('refuses missing, oversized or foreign captures before consulting an adapter', async () => {
+ const t = setup()
+ expect(await t.screen.read(t.row, null)).toBeNull()
+ expect(await t.screen.read(t.row, 'é'.repeat(256 * 1024))).toBeNull()
+ expect(await t.screen.read({ ...t.row, agentId: '' }, capture)).toBeNull()
+ expect(await t.screen.read({ ...t.row, sessionId: 'old' }, capture)).toBeNull()
+ t.replace()
+ expect(await t.screen.read(t.row, capture)).toBeNull()
+ expect(t.remote).not.toHaveBeenCalled()
+ })
+ it('does not substitute inline parsing for a failed worker or close a question on failure', async () => {
+ const t = setup(); t.remote.mockRejectedValue(new Error('worker stopped'))
+ expect(await t.screen.read(t.row, capture)).toBeNull()
+ await expect(t.screen.question(t.row, capture)).rejects.toThrow('ENGINE_UNAVAILABLE')
+ expect(t.inline).not.toHaveBeenCalled()
+ })
+ it.each(['sessionId', 'engine', 'active', 'processIdentity', 'runtimes'] as const)('discards evidence after an in-place %s change', async field => {
+ const t = setup()
+ let finish!: (value: typeof t.answer) => void
+ t.remote.mockImplementation(() => new Promise(resolve => { finish = resolve }))
+ const read = t.screen.read(t.row, capture)
+ Object.assign(t.row, { [field]: field === 'active' ? false : field === 'processIdentity' ? { pid: 20 } : field === 'runtimes' ? [{ paneId: 'new' }] : 'changed' })
+ finish(t.answer)
+ expect(await read).toBeNull()
+ })
+})
diff --git a/cli/src/core/engines/screens.ts b/cli/src/core/engines/screens.ts
new file mode 100644
index 000000000..d183cfe90
--- /dev/null
+++ b/cli/src/core/engines/screens.ts
@@ -0,0 +1,41 @@
+import type { ScreenReading } from '../../engines/facets/screen.js'
+import type { AgentEngine } from '../../engines/types.js'
+import { screenCapture } from '../../engines/worker/screenProtocol.js'
+import type { RegisteredSession } from '../../lib/registry.js'
+import type { ScreenReader } from '../../lib/screenReader.js'
+import type { ScreenTransport } from './screenTransport.js'
+import { EngineReadError } from '../../engines/worker/protocol.js'
+
+export interface ScreenDeps {
+ handles(engine: AgentEngine): boolean
+ transport: Pick<ScreenTransport, 'read'>
+ resolve(id: string): RegisteredSession | undefined
+ inline(engine: AgentEngine, capture: string | null): ScreenReading | undefined
+}
+
+/** Snapshot scalars: registry records may be edited in place while a worker answers. */
+function identity(session: RegisteredSession | undefined): string {
+ return session ? JSON.stringify([session.agentId, session.sessionId, session.engine, session.active,
+ session.registeredAt, session.boundAt, session.transcriptPath, session.tmuxPane,
+ session.primaryRuntimeKey, session.runtimes, session.processIdentity]) : ''
+}
+
+export function createScreens(deps: ScreenDeps) {
+ const read: ScreenReader = async (session, capture) => {
+ if (capture !== null && !screenCapture(capture)) return null
+ const key = identity(session), engine = session.engine, agentId = session.agentId
+ if (!agentId || identity(deps.resolve(agentId)) !== key) return null
+ try {
+ const answer = deps.handles(engine) ? capture === null ? null : await deps.transport.read(engine, capture) : deps.inline(engine, capture)
+ return identity(deps.resolve(agentId)) === key && answer ? answer : null
+ } catch { return null }
+ }
+ return {
+ read,
+ async question(session: RegisteredSession, capture: string) {
+ const screen = await read(session, capture)
+ if (!screen) throw new EngineReadError('ENGINE_UNAVAILABLE')
+ return screen.question
+ },
+ }
+}
diff --git a/cli/src/core/input.spec.ts b/cli/src/core/input.spec.ts
index fd1a8d31c..8d585ebb9 100644
--- a/cli/src/core/input.spec.ts
+++ b/cli/src/core/input.spec.ts
@@ -1,3 +1,4 @@
+import { readInlineScreen } from '../testing/inlineScreen.js'
import { readdirSync, readFileSync } from 'node:fs'
import { join } from 'node:path'
import { afterEach, describe, expect, it, vi } from 'vitest'
@@ -38,6 +39,7 @@ function deps(over: Partial<InputDeps> = {}) {
const normalizer = { openTurn: vi.fn((text: string) => [{ type: 'turn_started', text }]) }
let pane: string | null = READY_PANE
const base: InputDeps = {
+ readScreen: readInlineScreen,
resolve: (id) => agents.get(id),
byAgent: (agentId) => agents.get(agentId),
terminal: {
@@ -150,7 +152,7 @@ describe('the session input controller\'s dependencies', () => {
it('reads the pane again right before the Enter, and gives the reason not to press it, a popup of the message\'s own aside', async () => {
const screens = [READY_PANE, CLAUDE_PERMISSION, READY_PANE, null, READY_PANE, READY_PANE, `────────────\n❯ /mo\n────────────\n /model Set the AI model`]
const answers: Array<string | null> = []
- const write = messageWriter({
+ const write = messageWriter({ readScreen: deps().deps.readScreen,
resolve: (id) => agents.get(id),
terminal: {
validateTerminal: vi.fn(async () => true),
@@ -178,7 +180,7 @@ describe('the session input controller\'s dependencies', () => {
const order: string[] = []
const promptTyped = vi.fn((session: { agentId: string }, capture: string | null) => { order.push(`prompt ${session.agentId} ${capture === READY_PANE}`) })
const screens = [READY_PANE, CLAUDE_PERMISSION]
- const write = messageWriter({
+ const write = messageWriter({ readScreen: deps().deps.readScreen,
resolve: (id) => agents.get(id),
terminal: {
validateTerminal: vi.fn(async () => true),
@@ -199,7 +201,7 @@ describe('the session input controller\'s dependencies', () => {
for (const screen of [null, '✻ Welcome to Claude Code']) {
let reads = 0
const answers: Array<string | null> = []
- const write = messageWriter({
+ const write = messageWriter({ readScreen: deps().deps.readScreen,
resolve: (id) => agents.get(id),
terminal: {
validateTerminal: vi.fn(async () => true),
@@ -234,6 +236,18 @@ describe('the session input controller\'s dependencies', () => {
expect(run.calls.filter((call) => call.startsWith('submit'))).toEqual(['submit a1 hi'])
})
+ it('holds team and device writes when an otherwise readable screen has no worker evidence', async () => {
+ const run = deps({ readScreen: async () => null })
+ const wired = sessionInputDeps(run.deps, () => lock(run.calls))
+ expect(await wired.beforeTeamWrite!(agents.get('a1')!)).toBe('team_waiting_unavailable')
+ expect(await wired.inject('a1', 'hello')).toMatchObject({ dispatch: 'not_started', reason: 'screen_unreadable' })
+ expect(await wired.injectTeam!('a1', 'hello', 'delivery')).toMatchObject({ dispatch: 'not_started', reason: 'team_waiting_unavailable' })
+ const device = deviceInputDeps(run.deps, () => ({ acquireControl: vi.fn(), submit: vi.fn(), cancelDelivery: vi.fn() }))
+ expect(await device.isAwaitingUser!(agents.get('a1')!)).toBe(true)
+ expect(run.deps.terminal.submitTerminalAction).not.toHaveBeenCalled()
+ expect(run.deps.terminal.keyTerminalAction).not.toHaveBeenCalled()
+ })
+
it('reports an error to the app and, in the device\'s words, to the dial', () => {
const { deps: given } = deps()
sessionInputDeps(given, () => lock([])).onError('s1', 'the pane is gone')
diff --git a/cli/src/core/input.ts b/cli/src/core/input.ts
index 66bddacdf..97beebc79 100644
--- a/cli/src/core/input.ts
+++ b/cli/src/core/input.ts
@@ -18,14 +18,16 @@ import { sid } from '../lib/log.js'
import type { LiveEvent } from '../lib/normalize.js'
import type { RegisteredSession } from '../lib/registry.js'
import { SessionInputController, type SessionInputDelivery, type SessionInputDeps } from '../lib/sessionInput.js'
-import { messageHold, passingHold } from '../lib/messageHold.js'
+import { passingHold } from '../lib/messageHolds.js'
+import type { ScreenReader } from '../lib/screenReader.js'
+import type { ScreenReading } from '../engines/facets/screen.js'
import { TERMINAL_LEASE_REFUSED, terminalActionNotStarted, type TerminalActionResult } from '../lib/terminalTypes.js'
-import { teamWriteHold } from '../lib/teamWriteHold.js'
import type { TerminalControl } from './terminals/control.js'
type Frame = { type: string; agentId?: string; dbSessionId?: string; payload: Record<string, unknown> }
export interface InputDeps {
+ readScreen: ScreenReader
resolve: (id: string) => RegisteredSession | undefined
byAgent: (agentId: string) => RegisteredSession | undefined
terminal: Pick<TerminalControl, 'captureTerminal' | 'validateTerminal' | 'submitTerminalAction' | 'keyTerminalAction' | 'pinTerminalControl'>
@@ -48,7 +50,7 @@ export interface InputDeps {
emit: (sessionId: string, events: LiveEvent[]) => void
/** A prompt is about to be typed, and `capture` is the pane as read right before: what is on it belongs
* to the turns before the one it starts (the question watcher's, lib/askQuestion.ts `notePrompt`). */
- promptTyped?: (session: RegisteredSession, capture: string | null) => void
+ promptTyped?: (session: RegisteredSession, capture: string | null, screen: ScreenReading) => void
}
/**
@@ -64,8 +66,8 @@ export interface InputDeps {
* reason. `submitTerminalAction` is called here and nowhere else in this file, and input.spec.ts keeps
* it so.
*/
-export function messageWriter({ resolve, terminal: { captureTerminal, validateTerminal, submitTerminalAction }, promptTyped }: Pick<InputDeps, 'resolve' | 'terminal' | 'promptTyped'>) {
- return async (id: string, text: string, hold?: (session: RegisteredSession, capture: string | null) => string | null): Promise<TerminalActionResult> => {
+export function messageWriter({ readScreen, resolve, terminal: { captureTerminal, validateTerminal, submitTerminalAction }, promptTyped }: Pick<InputDeps, 'resolve' | 'terminal' | 'promptTyped' | 'readScreen'>) {
+ return async (id: string, text: string, hold?: (session: RegisteredSession, screen: ScreenReading | null) => string | null): Promise<TerminalActionResult> => {
const session = resolve(id)
if (!session) return terminalActionNotStarted('terminal agent is unavailable')
// The engine first, then its screen. A check of the engine can wait, seconds, for an answer: a probe
@@ -75,16 +77,19 @@ export function messageWriter({ resolve, terminal: { captureTerminal, validateTe
// not there is refused as a lease is, which the caller asks again.
if (!await validateTerminal(session)) return terminalActionNotStarted(TERMINAL_LEASE_REFUSED)
const capture = await captureTerminal(id)
- const reason = hold?.(session, capture) ?? messageHold(session.engine, capture)
+ const screen = await readScreen(session, capture)
+ if (!screen) return terminalActionNotStarted(hold?.(session, null) ?? 'screen_unreadable')
+ const reason = hold?.(session, screen) ?? screen.messageHold
if (reason) return terminalActionNotStarted(reason)
- promptTyped?.(session, capture)
+ promptTyped?.(session, capture, screen)
return submitTerminalAction(id, text, {
beforeEnter: async () => {
// A read that came back empty, or a composer caught between frames, is asked again for a moment
// (a re-attach can blank one capture): only what is on screen holds the Enter back, never a
// read that failed once. Still not read after that, it is held, as nothing says it is safe.
for (let tries = 1; ; tries++) {
- const before = messageHold(session.engine, await captureTerminal(id))
+ const next = await readScreen(session, await captureTerminal(id))
+ const before = next ? next.messageHold : 'screen_unreadable'
if (before === 'popup_open') return null
if (!before || !passingHold(before) || tries >= ENTER_CHECK_TRIES) return before
await new Promise((settle) => setTimeout(settle, ENTER_CHECK_RETRY_MS))
@@ -108,6 +113,7 @@ export function sessionInputDeps(
const writeMessage = messageWriter(deps)
return {
beforeSubmit: (id, text, tabId, deliveryId) => teams.prepare(id, text, tabId, deliveryId),
+ readScreen: deps.readScreen,
getSession: (id) => resolve(id),
onDelivery: (event) => {
device()?.delivery(event)
@@ -115,7 +121,8 @@ export function sessionInputDeps(
},
beforeTeamWrite: async session => {
const capture = await captureTerminal(session.agentId)
- return teamWriteHold(session.engine, capture)
+ const screen = await deps.readScreen(session, capture)
+ return screen ? screen.teamHold : 'team_waiting_unavailable'
},
validateRuntime: validateTerminal,
// Typed into the pane as it stands, a draft or a running turn included, under the pane's write lock.
@@ -123,7 +130,7 @@ export function sessionInputDeps(
// A team's turn waits for a ready composer, and for its delivery to still hold the pane.
injectTeam: (id, text, deliveryId) => deviceInput().legacyWrite(id, async () => {
if (!resolve(id)) return terminalActionNotStarted('team_waiting_unavailable')
- return writeMessage(id, text, (session, capture) => teamWriteHold(session.engine, capture)
+ return writeMessage(id, text, (session, screen) => (screen ? screen.teamHold : 'team_waiting_unavailable')
?? (teams.canWrite(deliveryId) ? null : 'team_waiting_control'))
}),
sendKey: (id, key) => deviceInput().legacyWrite(id, () => keyTerminalAction(id, key)),
@@ -166,7 +173,8 @@ export function deviceInputDeps(
// Whatever a message is not typed into, the Device's waits for it to close, rather than be refused.
isAwaitingUser: async session => {
const pane = await captureTerminal(session.agentId)
- return pane === null || messageHold(session.engine, pane) !== null
+ const screen = await deps.readScreen(session, pane)
+ return pane === null || !screen || screen.messageHold !== null
},
acquireControl: id => input().acquireControl(id, { forAnswer: true }),
legacySubmit: (id, text, deliveryId) => input().submit(id, text, deliveryId),
diff --git a/cli/src/core/main.ts b/cli/src/core/main.ts
index ca503e9f7..f11d78793 100644
--- a/cli/src/core/main.ts
+++ b/cli/src/core/main.ts
@@ -1,3 +1,7 @@
+import { createScreens } from './engines/screens.js'
+import { createScreenTransport } from './engines/screenTransport.js'
+import { legacyScreen } from '../lib/legacyScreen.js'
+import { masterRunsEngineScreen } from '../harnessd/services.js'
/**
* The core's entry: `harness __run`. `runForeground` is the composition root, which builds the core's
* modules, starts the services through `serviceHost` and wires them to the socket
@@ -20,7 +24,6 @@ import { sqlitePreflightMessage } from '../lib/sqliteAvailability.js'
import { warmLoginShellEnvironment } from '../lib/loginShellEnv.js'
import type { AppSwarms } from '../cable/cableSession.js'
import type { UnreadNotification } from '../lib/notificationRead.js'
-import { terminalActivity } from '../lib/terminalActivity.js'
import { registry, projectDisplayName, validTranscriptPath, type RegisteredSession } from '../lib/registry.js'
import { engineSessionTitle } from '../lib/sessionTitle.js'
import { machineNames } from '../lib/machineNames.js'
@@ -180,7 +183,7 @@ import { agentTokenUsage } from '../lib/agentTokenUsage.js'
import { LegacyRuntimeProfileManager, type RuntimeModelOption } from '../lib/runtimeProfileManager.js'
import { createRuntimeProfiles } from './engines/runtimeProfiles.js'
import { createRuntimeTransport } from './engines/runtimeTransport.js'
-import { RuntimeProfileController } from '../lib/runtimeProfileController.js'
+import { RuntimeProfileController } from '../lib/runtimeControl.js'
import { installTimestampedConsole, sid, prepareLogFile, trimLogFile, LOG_CHECK_INTERVAL_MS } from '../lib/log.js'
import { backendHttpBase } from '../lib/controlPlane.js'
@@ -542,13 +545,20 @@ async function runForeground(session: AuthSession | null): Promise<void> {
// An older master's explicit capability report selects compatibility. A failed worker never does.
const isolatedLive = (engine: string): boolean => liveHosted
&& readerEngine(engine) && outOfProcess.has(READER_SERVICES[engine])
+ const screenHosted = masterRunsEngineScreen(process.env, process.ppid)
+ const isolatedScreen = (engine: string): boolean => screenHosted && isolatedLive(engine)
const isolatedRuntime = (engine: string): boolean => runtimeHosted && isolatedLive(engine)
- const inline = KNOWN_SERVICES.some((name) => !outOfProcess.has(name)) || !runtimeHosted
+ const inline = KNOWN_SERVICES.some((name) => !outOfProcess.has(name)) || !runtimeHosted || !screenHosted
? await import('../services/inline.js') : null
let serviceLinksRef: ServiceLinks | null = null
const runtimeTransport = createRuntimeTransport({
call: (service, type, payload, waitMs) => serviceLinksRef?.call(service, type, payload, waitMs) ?? Promise.resolve({ error: 'SERVICE_UNAVAILABLE' }),
})
+ const screenTransport = createScreenTransport({
+ call: (service, type, payload, waitMs) => serviceLinksRef?.call(service, type, payload, waitMs) ?? Promise.resolve({ error: 'SERVICE_UNAVAILABLE' }),
+ })
+ const screens = createScreens({ handles: isolatedScreen, transport: screenTransport, resolve: id => registry.resolve(id),
+ inline: (engine, capture) => readerEngine(engine) ? capture === null ? undefined : inline?.screenFor(engine).inspect(capture) : legacyScreen(engine, capture) })
const runtimeProfiles = createRuntimeProfiles({
legacy: new LegacyRuntimeProfileManager(engine => isolatedRuntime(engine) ? undefined : inline?.runtimeFor(engine)),
handles: isolatedRuntime,
@@ -676,7 +686,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {
const session = registry.resolve(agentId)
if (!session || (session.engine !== 'claude' && session.engine !== 'codex')) return null
const screen = await terminals.capture(session, { mode: 'visible', ansi: false })
- return terminalActivity(session.engine, screen.state === 'succeeded' ? screen.value : null)
+ return (await screens.read(session, screen.state === 'succeeded' ? screen.value : null))?.activity?.label ?? null
},
// The devices' frames for the windows on this computer, never the cloud (createCoreApi checks their types).
sendLocal: (frame) => {
@@ -929,6 +939,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {
})
// Whether a turn is really working, beyond its transcript (core/turns/activity.ts).
const activity = createTurnActivity({
+ readScreen: screens.read,
terminals,
bySession: (sessionId) => registry.bySession(sessionId),
sessionTurnOpen,
@@ -1016,6 +1027,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {
backend.sessionsProvider = history.sessionsList
// Everything the core writes into a pane, and the device's pane lock (core/input.ts).
const inputs = createInput({
+ readScreen: screens.read,
resolve: (id) => registry.resolve(id),
byAgent: (agentId) => registry.byAgent(agentId),
terminal: terminalControl,
@@ -1034,7 +1046,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {
// Reassigned further down (the funnel): always the current one.
emit: (sessionId, events) => emitSessionEvents(sessionId, events),
// Declared further down: read as a prompt is typed, never now.
- promptTyped: (session, capture) => { if (session.sessionId) questionWatcher.notePrompt(session.sessionId, capture) },
+ promptTyped: (session, capture, screen) => { if (session.sessionId) questionWatcher.notePrompt(session.sessionId, capture, screen.question) },
})
const input = inputs.input
const deviceInput = inputs.deviceInput
@@ -1054,6 +1066,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {
// Questions an agent asks the person: shown on the dial and the window, answered from anywhere
// (core/questions.ts).
const asking = createQuestions({
+ readQuestion: screens.question,
resolve: (id) => registry.resolve(id),
terminal: terminalControl,
acquireTerminalControl,
@@ -1199,6 +1212,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {
engineReaders.connected(service)
liveTransport.connected(service)
runtimeTransport.connected(service)
+ screenTransport.connected(service)
if (service === 'gateway') gatewayLink?.connected()
if (service === 'teams') teamsLink.on()
if (service === 'devices') devicesLink.connected()
@@ -1209,6 +1223,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {
engineReaders.disconnected(service)
liveTransport.disconnected(service)
runtimeTransport.disconnected(service)
+ screenTransport.disconnected(service)
if (service === 'gateway') gatewayLink?.disconnected()
if (service === 'devices') devicesLink.disconnected()
if (service === 'wifi') wifiCore.stopped()
@@ -1285,6 +1300,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {
backend.models = () => ports.models ?? MODELS_OFF
const runtimeController = new RuntimeProfileController({
+ readScreen: screens.read,
manager: runtimeProfiles,
getSession: (id) => registry.resolve(id),
validateRuntime: validateTerminal,
@@ -2180,6 +2196,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {
// Moving a running agent onto a grid, or back to its own login (core/agents/retarget.ts).
backend.onRetargetAgent = createAgentRetargeter({
+ readScreen: screens.read,
purgeBusy: (agentId) => backend.purgeAgentService?.busy(agentId),
tmuxBackend,
registry,
@@ -2229,6 +2246,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {
backend.purgeProvider = createPurgeRequest({ purgeAgentService: () => lifecycle.purgeAgentService, invalidateStorage: () => { void (ports.monitor ?? MONITOR_OFF).storage([], true) } })
// Closing agents no window shows, and the cleanup preview (core/agents/close.ts).
const closing = createAgentClosing({
+ readScreen: screens.read,
registry,
cleanupTabs,
watcher,
diff --git a/cli/src/core/questions.spec.ts b/cli/src/core/questions.spec.ts
index fea9256dc..82c146d5e 100644
--- a/cli/src/core/questions.spec.ts
+++ b/cli/src/core/questions.spec.ts
@@ -1,3 +1,4 @@
+import { parseEngineQuestionPane } from '../engines/screens.js'
import { afterEach, describe, expect, it, vi } from 'vitest'
import type { RegisteredSession } from '../lib/registry.js'
import { createQuestionResponse, createQuestions, type QuestionDeps } from './questions.js'
@@ -9,6 +10,7 @@ const agents = new Map<string, RegisteredSession>([
function setup(over: Partial<QuestionDeps> = {}) {
const deps: QuestionDeps = {
+ readQuestion: (session, capture) => parseEngineQuestionPane(session.engine, capture),
resolve: (id) => agents.get(id),
terminal: {
captureTerminal: vi.fn(async () => 'pane'),
diff --git a/cli/src/core/questions.ts b/cli/src/core/questions.ts
index dad6fbbec..fed6cc7eb 100644
--- a/cli/src/core/questions.ts
+++ b/cli/src/core/questions.ts
@@ -7,7 +7,7 @@
* Moved verbatim out of `runForeground` (the core boundary, step 6: docs/design/2026-10-03-harnessd.md).
* The socket's two handlers stay bound there: `answer` (question_response) and `monitorActivity`.
*/
-import { AskQuestionController, QuestionWatcher, type QuestionAnswerPayload, type QuestionAnswerResult } from '../lib/askQuestion.js'
+import { AskQuestionController, QuestionWatcher, type QuestionAnswerPayload, type QuestionAnswerResult } from '../lib/questionController.js'
import type { AutonomousDeviceInput } from './deviceInput.js'
import { preview, sid } from '../lib/log.js'
import type { RegisteredSession } from '../lib/registry.js'
@@ -16,6 +16,7 @@ import type { TerminalControl } from './terminals/control.js'
type Frame = { type: string; agentId?: string; dbSessionId?: string; payload: Record<string, unknown> }
export interface QuestionDeps {
+ readQuestion: import('../lib/questionController.js').QuestionWatcherDeps['readQuestion']
resolve: (id: string) => RegisteredSession | undefined
terminal: Pick<TerminalControl, 'captureTerminal' | 'submitTerminal' | 'keyTerminal'>
/** Holds a pane, queue and terminal both, for the whole multi-step answer (Input.acquireTerminalControl). */
@@ -30,7 +31,7 @@ export interface QuestionDeps {
}
export function createQuestions({
- resolve, terminal, acquireTerminalControl, clients, agentIdFor, sessionTurnOpen, someoneCanAnswer, deviceInput,
+ readQuestion, resolve, terminal, acquireTerminalControl, clients, agentIdFor, sessionTurnOpen, someoneCanAnswer, deviceInput,
}: QuestionDeps) {
const { captureTerminal, submitTerminal, keyTerminal } = terminal
// AskUserQuestion bridge: mirrors the question to the device's question screen, and keys the device's
@@ -38,6 +39,7 @@ export function createQuestions({
const questions = new AskQuestionController({
getSession: (id) => resolve(id),
capture: captureTerminal,
+ readQuestion,
sendText: submitTerminal,
sendKey: keyTerminal,
acquireControl: acquireTerminalControl,
@@ -75,6 +77,7 @@ export function createQuestions({
const questionWatcher = new QuestionWatcher({
getSession: (id) => resolve(id),
capture: captureTerminal,
+ readQuestion,
hasDevice: () => someoneCanAnswer(),
isDriving: (sessionId) => questions.isDriving(sessionId),
onQuestion: (sessionId, requestId, shaped, detail) => {
diff --git a/cli/src/core/transcripts/attach.ts b/cli/src/core/transcripts/attach.ts
index 7039c8bec..55eb0c638 100644
--- a/cli/src/core/transcripts/attach.ts
+++ b/cli/src/core/transcripts/attach.ts
@@ -23,7 +23,7 @@ import { MuseNormalizer } from '../../engines/muse/normalizer.js'
import { OpencodeReader } from '../../engines/opencode/reader.js'
import { PiNormalizer } from '../../engines/pi/normalizer.js'
import type { AgentEngine } from '../../engines/types.js'
-import { pollsQuestions, type QuestionWatcher } from '../../lib/askQuestion.js'
+import { pollsQuestions, type QuestionWatcher } from '../../lib/questionController.js'
import { attachTranscript, type AttachRead } from '../../lib/attachTranscript.js'
import { AttachTracker } from '../../lib/attachTracker.js'
import type { WifiFeed } from '../wifi.js'
diff --git a/cli/src/core/turns/activity.spec.ts b/cli/src/core/turns/activity.spec.ts
index 4e80755f4..448a6c307 100644
--- a/cli/src/core/turns/activity.spec.ts
+++ b/cli/src/core/turns/activity.spec.ts
@@ -1,3 +1,4 @@
+import { readInlineScreen } from '../../testing/inlineScreen.js'
import { afterEach, describe, expect, it, vi } from 'vitest'
import type { RegisteredSession } from '../../lib/registry.js'
import { activityRuntimeKey } from '../../lib/runtimeActivity.js'
@@ -11,6 +12,7 @@ const sessions = new Map([[live.sessionId, live], [idle.sessionId, idle], [shell
function setup(over: Partial<TurnActivityDeps> = {}) {
const deps: TurnActivityDeps = {
+ readScreen: readInlineScreen,
terminals: { capture: vi.fn(async () => ({ state: 'succeeded' as const, value: 'screen' })) },
bySession: (sessionId) => sessions.get(sessionId),
sessionTurnOpen: (sessionId) => sessionId === 's1',
diff --git a/cli/src/core/turns/activity.ts b/cli/src/core/turns/activity.ts
index 94c984125..979463662 100644
--- a/cli/src/core/turns/activity.ts
+++ b/cli/src/core/turns/activity.ts
@@ -1,3 +1,4 @@
+import type { ScreenReader } from '../../lib/screenReader.js'
/**
* Turn activity: whether an agent is really working, beyond what its transcript says. A turn's
* transcript can go quiet while the agent thinks or runs a long tool, so an open turn is checked against
@@ -12,6 +13,7 @@ import type { TerminalBackendCoordinator } from '../../lib/terminalBackendCoordi
import { TurnActivity, type ActivityFrame } from '../../lib/turnActivity.js'
export interface TurnActivityDeps {
+ readScreen: ScreenReader
terminals: Pick<TerminalBackendCoordinator, 'capture'>
bySession: (sessionId: string) => RegisteredSession | undefined
sessionTurnOpen: (sessionId: string) => boolean
@@ -19,9 +21,10 @@ export interface TurnActivityDeps {
drain: (sessionId: string) => Promise<void>
}
-export function createTurnActivity({ terminals, bySession, sessionTurnOpen, drain }: TurnActivityDeps) {
+export function createTurnActivity({ readScreen, terminals, bySession, sessionTurnOpen, drain }: TurnActivityDeps) {
const codexActivity = new CodexActivityReader()
const runtimeActivity = new RuntimeActivityReader({
+ readScreen,
codex: session => codexActivity.read(session),
capture: async session => {
const screen = await terminals.capture(session, { mode: 'visible', ansi: true })
diff --git a/cli/src/core/turns/cancel.ts b/cli/src/core/turns/cancel.ts
index 0cb3d9010..06a41c777 100644
--- a/cli/src/core/turns/cancel.ts
+++ b/cli/src/core/turns/cancel.ts
@@ -7,7 +7,7 @@
*/
import type { CursorSubagentManager } from '../../engines/cursor/subagent.js'
import { correlateAgentEvent } from '../../lib/agentEvent.js'
-import type { QuestionWatcher } from '../../lib/askQuestion.js'
+import type { QuestionWatcher } from '../../lib/questionController.js'
import type { WifiFeed } from '../wifi.js'
import type { TurnRecaps } from './recaps.js'
import type { RegisteredSession } from '../../lib/registry.js'
diff --git a/cli/src/core/turns/funnel.ts b/cli/src/core/turns/funnel.ts
index 30f4d972d..adb834793 100644
--- a/cli/src/core/turns/funnel.ts
+++ b/cli/src/core/turns/funnel.ts
@@ -18,7 +18,7 @@ import type { TurnRecaps } from './recaps.js'
import { deviceErrorText } from '../cardText.js'
import { preview, sid } from '../../lib/log.js'
import type { CursorNormalizer } from '../../engines/cursor/normalizer.js'
-import type { QuestionWatcher } from '../../lib/askQuestion.js'
+import type { QuestionWatcher } from '../../lib/questionController.js'
import type { RegisteredSession } from '../../lib/registry.js'
import type { SessionInputController } from '../../lib/sessionInput.js'
import type { TurnActivity } from '../../lib/turnActivity.js'
diff --git a/cli/src/engines/README.md b/cli/src/engines/README.md
index 4b0e916b6..7049fce0f 100644
--- a/cli/src/engines/README.md
+++ b/cli/src/engines/README.md
@@ -18,8 +18,9 @@ that stream.
launch contracts, history/last-turn readers and hook behavior behind `Engine` (`engine.ts`). `registry.ts`
selects those engines; `launches.ts` and `hooks.ts` expose their respective facets without loading history
readers. Hook transport, process binding and event delivery stay with core. Claude Code and Codex history/last-turn reads,
-live transcript parsing and runtime profile interpretation/catalogs run on demand in supervised workers;
+live transcript parsing, runtime profiles/catalogs and screen interpretation run on demand in supervised workers;
the other facets still run in core. Explicit inline mode and older masters retain inline implementations. See
+[screen interpretation](../../../docs/design/2026-10-08-engine-screen.md),
[runtime profiles](../../../docs/design/2026-10-08-engine-runtime.md) and the
[reader pilot](../../../docs/design/2026-10-07-engine-readers.md) for limits and failure behavior. The
remaining facets and engines still use the shared paths below. This is an internal interface, not an
diff --git a/cli/src/engines/claude/activity.ts b/cli/src/engines/claude/activity.ts
new file mode 100644
index 000000000..8089403e0
--- /dev/null
+++ b/cli/src/engines/claude/activity.ts
@@ -0,0 +1,22 @@
+/** Native live indicator, read only from the current footer. */
+export function activity(screen: string): { label: string; indicator: string } | null {
+ const lines = screen.replace(/\x1b\][^\x07]*(?:\x07|\x1b\\)/g, '')
+ .replace(/\x1b\[[0-?]*[ -/]*[@-~]/g, '').split(/\r?\n/)
+ let end = lines.length
+ // The expanded agent tree sits BELOW the prompt and can push the live
+ // spinner out of the last 16 rows. Anchor to the actual input/footer pair,
+ // retaining a short search above it instead of scanning old output.
+ for (let i = lines.length - 1; i >= Math.max(0, lines.length - 64); i--) {
+ if (/^\s*[❯›>](?:\s|$)/u.test(lines[i]) &&
+ lines.slice(i + 1, i + 9).some(line => /shift\+tab to cycle|esc to interrupt/i.test(line))) {
+ end = i
+ break
+ }
+ }
+ for (let i = end - 1; i >= Math.max(0, end - 16); i--) {
+ const line = lines[i].trim()
+ const match = /^[✢✳✶✻✽·*]\s+([\p{L}][\p{L}\p{N} '\u2019-]{0,55}(?:…|\.{3}))(?:\s+\([^\n]*\))?\s*$/u.exec(line)
+ if (match) return { label: match[1].trim().replace(/…/g, '...'), indicator: line }
+ }
+ return null
+}
diff --git a/cli/src/engines/claude/composer.ts b/cli/src/engines/claude/composer.ts
new file mode 100644
index 000000000..783e6f8d9
--- /dev/null
+++ b/cli/src/engines/claude/composer.ts
@@ -0,0 +1,36 @@
+import type { ComposerState } from '../facets/screen.js'
+import { screenLines, typedToken } from '../kit/composer.js'
+
+/** A row of Claude Code's rule: all `─`, or `─` at both ends with text set in it. */
+const CLAUDE_RULE = /^─{3,}$|^─{3,}.*─{3,}$/
+
+export function composerState(capture: string): ComposerState {
+ const { lines } = screenLines(capture)
+ let prompt = lines.findLastIndex((line) => /^❯(?:\s|$)/.test(line))
+ let bottom = lines.findIndex((line, index) => index > prompt && CLAUDE_RULE.test(line.trim()))
+ if (prompt < 1 || !CLAUDE_RULE.test(lines[prompt - 1].trim()) || bottom < 0) {
+ // A draft taller than the pane: Claude Code does not cap the prompt's height outside its fullscreen
+ // renderer (2.1.289 passes `maxVisibleLines` only there), so the box's top rule and its `❯` row
+ // scroll off, and the pane shows the draft's further rows, indented under the `❯`, down to the
+ // bottom rule. Every row above that rule is one of them; a dialog's are indented by one column, the
+ // conversation's start at the edge.
+ bottom = lines.findIndex((line) => CLAUDE_RULE.test(line.trim()))
+ if (bottom < 1 || !lines.slice(0, bottom).every((line) => !line || line.startsWith(' '))) return 'absent'
+ prompt = -1
+ }
+ const below = lines.slice(bottom + 1)
+ // Nothing ruled under the box: a dialog of Claude Code's own is drawn ruled, in its place.
+ if (below.some((line) => CLAUDE_RULE.test(line.trim()))) return 'absent'
+ const draft = [prompt < 0 ? '' : lines[prompt].replace(/^❯\s?/, ''), ...lines.slice(prompt + 1, bottom).map((line) => line.trim())].join(' ').trim()
+ const token = typedToken(draft)
+ const rows = below.map((line) => line.trim()).filter(Boolean)
+ if (token.startsWith('/') && rows.some((row) => /^\/[\w:.-]/.test(row))) return 'popup'
+ if (token.startsWith('@')) {
+ const typed = token.slice(1).toLowerCase()
+ // A file or agent suggestion, by what is typed after the `@`; with nothing typed yet, any row but the
+ // footer's own (its mode and its hints).
+ if (rows.some((row) => typed ? row.replace(/^@/, '').toLowerCase().startsWith(typed) : !/^(?:\?|⏵|esc\b)/.test(row))) return 'popup'
+ }
+ return 'ready'
+}
+
diff --git a/cli/src/engines/claude/screen.ts b/cli/src/engines/claude/screen.ts
new file mode 100644
index 000000000..07180001b
--- /dev/null
+++ b/cli/src/engines/claude/screen.ts
@@ -0,0 +1,53 @@
+import { activity } from './activity.js'
+import type { EngineScreen, PaneTakeover, PaneView } from '../facets/screen.js'
+import { inspectPane, paneModal as readModal, stripAnsi } from '../kit/pane.js'
+import { locateQuestionPane } from '../kit/questionPane.js'
+import { assembleScreen } from '../kit/screen.js'
+import { composerState } from './composer.js'
+
+export function questionPane(capture: string): PaneView {
+ const found = locateQuestionPane(capture)
+ if (!found || composerState(capture.split('\n').slice(found.at + 1).join('\n')) !== 'absent') return null
+ return found.view
+}
+export const inspectRuntimePane = (capture: string) => inspectPane('claude', capture, takeover)
+export const paneModal = (capture: string) => readModal('claude', capture, takeover)
+export const screen: EngineScreen = {
+ inspect: capture => assembleScreen('claude', capture, inspectRuntimePane(capture), questionPane(capture),
+ paneModal(capture), composerState(capture), activity(capture)),
+}
+
+function takeover(capture: string): PaneTakeover | null {
+ const lines = stripAnsi(capture).replace(/\u00a0/g, ' ').split('\n')
+ const bottom = lines.filter(line => line.trim()).slice(-3)
+ const composer = lines.findLastIndex((line, index) => /^\s*❯/.test(line) && index > 0 && /^\s*[─━]{8,}\s*$/.test(lines[index - 1]))
+ const shown = (...patterns: RegExp[]) => patterns.every(pattern => lines.findLastIndex(line => pattern.test(line)) > composer)
+ if (claudeRewindMenuOpen(capture)) return 'rewind'
+ if (bottom.some((line) => CLAUDE_TRANSCRIPT_FOOTER.test(line))) return 'transcript'
+ if (bottom.some((line) => CLAUDE_HISTORY_SEARCH.test(line))) return 'search'
+ if (shown(/Quick safety check: Is this a project you created or one you trust\?/, /^\s*(?:❯\s*)?Yes, I trust this folder\s*$/)) return 'trust'
+ if (shown(/^\s*Select login method:\s*$/, /^\s*(?:❯\s*)?1\. Claude account with subscription\b/)) return 'sign_in'
+ return null
+}
+/**
+ * Claude Code's transcript view (ctrl+o): the prompt is hidden, and the footer row starts
+ * `Showing detailed transcript · ctrl+o to toggle`, after `dialog waiting · ` when a dialog sits behind
+ * it (2.1.289). It has no Enter, so a message typed there is lost. Esc, q or ctrl+c close it.
+ */
+const CLAUDE_TRANSCRIPT_FOOTER = /^\s*(?:dialog waiting · )?Showing detailed transcript\b/
+
+/** Claude Code's prompt-history search (ctrl+r), under the prompt: `search prompts: <query>`, or
+ * `no matching prompt: <query>` (2.1.289). */
+const CLAUDE_HISTORY_SEARCH = /^\s*(?:search prompts|no matching prompt): /
+
+const CLAUDE_REWIND_BODY = /^(?:Restore the code\b|Restore and fork\b|Confirm you want to restore\b|Nothing to rewind to\b)/
+
+function claudeRewindMenuOpen(capture: string): boolean {
+ const lines = stripAnsi(capture).replace(/\u00a0/g, ' ').split('\n').map((line) => line.trim())
+ // The title, with its body a blank line under it, and no rule after it: the prompt the menu hides
+ // comes back between two rules once it closes.
+ return lines.some((line, index) => line === 'Rewind'
+ && lines.slice(index + 1, index + 4).some((next) => CLAUDE_REWIND_BODY.test(next))
+ && !lines.slice(index + 1).some((next) => /[─━]{8,}/u.test(next)))
+}
+
diff --git a/cli/src/engines/codex/activity.ts b/cli/src/engines/codex/activity.ts
new file mode 100644
index 000000000..b99af4809
--- /dev/null
+++ b/cli/src/engines/codex/activity.ts
@@ -0,0 +1,12 @@
+/** Native live indicator, read only from the current footer. */
+export function activity(screen: string): { label: string; indicator: string } | null {
+ const lines = screen.replace(/\x1b\][^\x07]*(?:\x07|\x1b\\)/g, '')
+ .replace(/\x1b\[[0-?]*[ -/]*[@-~]/g, '').split(/\r?\n/)
+ const end = lines.length
+ for (let i = end - 1; i >= Math.max(0, end - 16); i--) {
+ const line = lines[i].trim()
+ const match = /^[•◦⠁-⣿]\s+(.{1,60}?)\s+\([^\n]*\besc to interrupt\b[^\n]*\)\s*$/i.exec(line)
+ if (match) return { label: match[1].trim().replace(/…/g, '...'), indicator: line }
+ }
+ return null
+}
diff --git a/cli/src/engines/codex/composer.ts b/cli/src/engines/codex/composer.ts
new file mode 100644
index 000000000..676ba7536
--- /dev/null
+++ b/cli/src/engines/codex/composer.ts
@@ -0,0 +1,32 @@
+import type { ComposerState } from '../facets/screen.js'
+import { screenLines, startsBold, typedToken } from '../kit/composer.js'
+
+/** Codex's composer row: `›` (or `»`) at the left edge, never a numbered row of a picker. */
+const CODEX_COMPOSER = /^[›»](?:\s|$)/
+const CODEX_PICKER_ROW = /^[›»]\s*\d+\.\s/
+
+export function composerState(capture: string): ComposerState {
+ const { lines, raw } = screenLines(capture)
+ const composer = lines.findLastIndex((line) => CODEX_COMPOSER.test(line) && !CODEX_PICKER_ROW.test(line))
+ // Bold, on a row of its own under a blank one: not a line of the conversation, nor a disabled composer.
+ if (composer < 0 || !startsBold(raw[composer]) || (composer > 0 && lines[composer - 1].trim())) return 'absent'
+ // The draft's further lines, then a blank row and the footer: nothing more is drawn under Codex's
+ // composer, so more is some other screen under a line of the conversation.
+ let end = composer + 1
+ while (end < lines.length && lines[end].trim()) end++
+ const footer = lines.slice(end).filter((line) => line.trim())
+ if (footer.length > 3) return 'absent'
+ const draft = [lines[composer].replace(/^[›»]\s?/, ''), ...lines.slice(composer + 1, end).map((line) => line.trim())].join(' ').trim()
+ const token = typedToken(draft)
+ // A voice send was refused as popup_open even though the recording had transcribed. Looking through
+ // twenty conversation rows also matched command examples and old menu hints. A popup is the block
+ // immediately above the composer; a slash menu has a selected row and edits the initial /command.
+ let popupEnd = composer - 1
+ while (popupEnd >= 0 && !lines[popupEnd].trim()) popupEnd--
+ let popupStart = popupEnd
+ while (popupStart >= Math.max(0, composer - 20) && lines[popupStart].trim()) popupStart--
+ const above = lines.slice(popupStart + 1, popupEnd + 1)
+ if (draft.startsWith('/') && token.startsWith('/') && above.some((line) => /^›\s+\/[\w-]+(?:\s{2,}\S|$)/.test(line))) return 'popup'
+ if (token.startsWith('@') && above.some((line) => /\benter\/tab insert · esc close\b/.test(line))) return 'popup'
+ return 'ready'
+}
diff --git a/cli/src/engines/codex/screen.ts b/cli/src/engines/codex/screen.ts
new file mode 100644
index 000000000..5276eefff
--- /dev/null
+++ b/cli/src/engines/codex/screen.ts
@@ -0,0 +1,62 @@
+import { stoppedGoal } from './stoppedGoal.js'
+import { activity } from './activity.js'
+import type { EngineScreen, PaneTakeover, PaneView } from '../facets/screen.js'
+import { inspectPane, paneModal as readModal, stripAnsi } from '../kit/pane.js'
+import { locateQuestionPane } from '../kit/questionPane.js'
+import { assembleScreen } from '../kit/screen.js'
+import { composerState } from './composer.js'
+
+export function questionPane(capture: string): PaneView {
+ const found = locateQuestionPane(capture)
+ if (!found || composerState(capture.split('\n').slice(found.at + 1).join('\n')) !== 'absent') return null
+ return withCodexLabels(found.view)
+}
+export const inspectRuntimePane = (capture: string) => inspectPane('codex', capture, takeover, /\bplan mode\b/i)
+export const paneModal = (capture: string) => readModal('codex', capture, takeover)
+export const screen: EngineScreen = {
+ inspect(capture) {
+ const pane = inspectRuntimePane(capture)
+ return assembleScreen('codex', capture, pane, questionPane(capture), paneModal(capture),
+ composerState(capture), activity(capture), stoppedGoal(capture, pane))
+ },
+}
+
+function withCodexLabels(view: PaneView): PaneView {
+ if (!view || view.kind !== 'question') return view
+ return {
+ ...view,
+ rows: view.rows.map((r) => ({ ...r, label: r.label.split(/\s{2,}/)[0].trim() || r.label })),
+ }
+}
+
+function takeover(capture: string, currentUi: string): PaneTakeover | null {
+ const lines = stripAnsi(capture).replace(/\u00a0/g, ' ').split('\n')
+ const bottom = lines.filter(line => line.trim()).slice(-3)
+ const composer = lines.findLastIndex(line => /^\s*›/.test(line) && !/^\s*›\s*\d+\.\s/.test(line))
+ const shown = (...patterns: RegExp[]) => patterns.every(pattern => lines.findLastIndex(line => pattern.test(line)) > composer)
+ if (CODEX_TRANSCRIPT_BROWSING.test(currentUi)) return 'rewind'
+ if (CODEX_PAGER_HEADER.test(lines.find((line) => line.trim()) ?? '')) return 'transcript'
+ if (bottom.some((line) => CODEX_SEARCH_FOOTER.test(line))) return 'search'
+ if (shown(CODEX_TRUST_QUESTION, CODEX_TRUST_ROW)) return 'trust'
+ if (shown(/^\s*Update available\b/, CODEX_UPDATE_ROW)) return 'update'
+ if (shown(/^\s*[›>]?\s*1\. Try new model\s*$/, /^\s*[›>]?\s*2\. Use existing model\s*$/)) return 'model'
+ if (shown(/^\s*[›>]?\s*1\. Sign in with ChatGPT\s*$/)) return 'sign_in'
+ return null
+}
+/** The header of Codex's pager over the whole pane, its top row: its transcript (ctrl+t) when it is not
+ * on the composer's screen (pager_overlay/transcript.rs). Closed with q or ctrl+t; Esc browses prompts. */
+const CODEX_PAGER_HEADER = /^\/ T R A N S C R I P T(?: \/)*\s*$/
+
+/** Codex's search footers: through its prompt history (`reverse-i-search: … enter accept · esc cancel`,
+ * chat_composer/history_search.rs) and through its transcript (`Find: …`, transcript_view/search.rs). */
+const CODEX_SEARCH_FOOTER = /^\s*(?:reverse-i-search:|Find: )/
+
+/** Codex's trust question, as 0.160 words it (onboarding/trust_directory.rs) and as 0.147 did. */
+const CODEX_TRUST_QUESTION = /Trust this folder\? Codex can read, edit, and run files here|Do you trust the contents of this directory\?/
+const CODEX_TRUST_ROW = /^\s*[›>]?\s*1\. (?:Trust and continue|Yes, continue|Open restricted|Open existing task)\s*$/
+
+/** Codex's update prompt (update_prompt.rs): its first row runs the update. */
+const CODEX_UPDATE_ROW = /^\s*[›>]?\s*1\. Update now \(runs /
+
+const CODEX_TRANSCRIPT_BROWSING = /^\s*Browsing(?: transcript)?(?:\s+·|\s*$)/m
+
diff --git a/cli/src/engines/codex/stoppedGoal.ts b/cli/src/engines/codex/stoppedGoal.ts
new file mode 100644
index 000000000..e07c42dd5
--- /dev/null
+++ b/cli/src/engines/codex/stoppedGoal.ts
@@ -0,0 +1,20 @@
+/** Some Codex goal interruptions never write task_complete/turn_aborted to the
+ * rollout. An open transcript is then only historical evidence of work. The
+ * live footer is one positive idle reading, never a synthetic turn end. */
+import { stripVTControlCharacters } from 'node:util'
+import type { PaneInspection } from '../facets/screen.js'
+
+/** Read only the UI below the current empty composer, never output/history.
+ * An empty composer alone says nothing: Codex accepts drafts while working. */
+export function stoppedGoal(screen: string, pane: PaneInspection): boolean {
+ if (!screen || !pane.idle) return false
+ const lines = stripVTControlCharacters(screen).split(/\r?\n/)
+ const prompt = lines.findLastIndex(line => /^\s*›(?:\s|$)/u.test(line))
+ if (prompt < 0) return false
+ // A live turn can still display the previous goal state during a redraw.
+ if (lines.slice(-16).some(line => /\besc to interrupt\b/i.test(line))) return false
+ const footer = lines.slice(prompt + 1).filter(line => line.trim())
+ return footer.length <= 4
+ && footer.some(line => /\bGoal (?:stalled|paused) \(\/goal resume\)\s*$/.test(line))
+ && footer.some(line => /\? for shortcuts\b/.test(line))
+}
diff --git a/cli/src/engines/engine.ts b/cli/src/engines/engine.ts
index f8cef261c..707058fc0 100644
--- a/cli/src/engines/engine.ts
+++ b/cli/src/engines/engine.ts
@@ -1,3 +1,4 @@
+import type { EngineScreen } from './facets/screen.js'
import type { ProcessEngine } from './types.js'
import type { EngineTranscript } from './facets/transcript.js'
import type { EngineLaunch } from './facets/launch.js'
@@ -13,4 +14,5 @@ export interface Engine {
readonly hooks: EngineHooks
readonly live: EngineLive
readonly runtime: EngineRuntime
+ readonly screen: EngineScreen
}
diff --git a/cli/src/engines/facets/screen.ts b/cli/src/engines/facets/screen.ts
new file mode 100644
index 000000000..c939d4132
--- /dev/null
+++ b/cli/src/engines/facets/screen.ts
@@ -0,0 +1,89 @@
+/** A read-only interpretation of one captured terminal screen. */
+/** Device-facing question shape — byte-for-byte the hosted runtime’s `commanderQuestions()` output. */
+export interface ShapedQuestion {
+ key: string
+ q: string
+ options: string[]
+ multi: boolean
+ /** Observed free-text editor; never inferred for permission prompts. */
+ canText?: boolean
+}
+
+export interface QuestionRow {
+ /** The digit to press. */
+ number: string
+ /** Option text with any `[ ]` checkbox prefix removed. */
+ label: string
+ checked: boolean
+ /**
+ * Set only when the dialog does NOT number its rows: the row is then reached by walking to it and
+ * pressing Enter, and `number` carries its INDEX rather than a key.
+ *
+ * The direction is a property of the DIALOG, not of the engine. OpenCode draws BOTH a numbered ask
+ * dialog and — for permissions — the horizontal prompt kilo inherited from it, so keying by engine
+ * would send digits into a dialog that numbers nothing and select nothing at all.
+ */
+ walk?: 'right' | 'down'
+}
+
+export interface QuestionView {
+ kind: 'question'
+ permission?: boolean
+ /**
+ * The dialog is on screen but its top — the question and its first rows — is scrolled out of the pane
+ * (Codex's request_user_input keeps the footer anchored and lets a short pane cut the top off; measured
+ * in a four-pane window: rows 3–5 and the footer visible, "1." and the question gone). Enough to know a
+ * dialog is STILL OPEN — the watcher must not close it — and not enough to announce as a question.
+ */
+ partial?: boolean
+ /** The footer says a digit only highlights and Enter commits (`enter to submit answer` — Codex's
+ * request_user_input). Absent where one digit selects and submits, which is every other dialog. */
+ enterSubmits?: boolean
+ question: string
+ rows: QuestionRow[]
+ multi: boolean
+ /** The "Type something." row, when the dialog offers free text. */
+ typeRow: QuestionRow | null
+ /**
+ * A permission prompt's WHOLE dialog, every line as painted (ANSI stripped), from its frame to its last
+ * row. `question` is one line clipped for a device's screen; a command that wraps — `npm test &&` on one
+ * line, `git push` on the next — is only whole here. What the pair brain's floor reads (pair/classify.ts),
+ * and part of the dialog's fingerprint, so two prompts that differ below their first line are two ids.
+ */
+ dialog?: string
+}
+
+export interface ReviewView {
+ kind: 'review'
+ submitRow: string
+}
+
+export type PaneView = QuestionView | ReviewView | null
+
+export interface PaneInspection {
+ idle: boolean
+ plan: boolean
+ dialog: boolean
+ draft: boolean
+}
+
+
+export type ComposerState = 'ready' | 'popup' | 'absent'
+export type PaneTakeover = 'rewind' | 'transcript' | 'search' | 'trust' | 'update' | 'model' | 'sign_in'
+export type PaneModal = PaneTakeover | 'permission' | 'menu' | null
+export type MessageHold = 'permission_open' | 'question_open' | 'menu_open' | 'rewind_picker_open' | 'transcript_open'
+ | 'search_open' | 'trust_open' | 'update_prompt_open' | 'model_prompt_open' | 'sign_in_open' | 'popup_open'
+ | 'prompt_hidden' | 'screen_unreadable'
+export interface FoundDialog { view: QuestionView | ReviewView; at: number }
+export interface ScreenReading {
+ activity: { label: string; indicator: string } | null
+ busy: boolean
+ stoppedGoal: boolean
+ pane: PaneInspection
+ question: PaneView
+ messageHold: MessageHold | null
+ teamHold: string | null
+}
+export interface EngineScreen {
+ inspect(capture: string): ScreenReading
+}
diff --git a/cli/src/engines/kit/composer.ts b/cli/src/engines/kit/composer.ts
new file mode 100644
index 000000000..dc70c12f1
--- /dev/null
+++ b/cli/src/engines/kit/composer.ts
@@ -0,0 +1,29 @@
+import { stripVTControlCharacters } from 'node:util'
+
+export function screenLines(capture: string): { lines: string[]; raw: string[] } {
+ const raw = capture.split('\n')
+ const lines = raw.map((line) => stripVTControlCharacters(line).replace(/ /g, ' ').trimEnd())
+ while (lines.length && !lines.at(-1)) { lines.pop(); raw.pop() }
+ return { lines, raw }
+}
+
+/** Whether the first character of a row is drawn bold: the SGR in force when it is written. */
+export function startsBold(row: string): boolean {
+ let bold = false
+ const sgr = /^\u001b\[([0-9;:]*)m/
+ let rest = row
+ for (let match = sgr.exec(rest); match; match = sgr.exec(rest)) {
+ for (const code of match[1].split(/[;:]/)) {
+ if (code === '1') bold = true
+ else if (code === '' || code === '0' || code === '22') bold = false
+ }
+ rest = rest.slice(match[0].length)
+ }
+ return bold
+}
+
+/** The word being typed at the end of a draft, where a `/command` or an `@mention` opens its popup. */
+export function typedToken(draft: string): string {
+ return draft.split(/\s/).at(-1) ?? ''
+}
+
diff --git a/cli/src/engines/kit/pane.ts b/cli/src/engines/kit/pane.ts
new file mode 100644
index 000000000..360f447fb
--- /dev/null
+++ b/cli/src/engines/kit/pane.ts
@@ -0,0 +1,400 @@
+import type { AgentEngine } from '../types.js'
+import type { PaneInspection, PaneTakeover } from '../facets/screen.js'
+
+type Takeover = (capture: string, currentUi: string) => PaneTakeover | null
+export function stripAnsi(value: string): string {
+ return value.replace(/\u001b\[[0-9;:]*[A-Za-z]/g, '')
+}
+
+/**
+ * The SGR attributes an escape sequence actually sets, with extended-colour arguments removed.
+ *
+ * `38`/`48`/`58` introduce a colour whose ARGUMENTS are numbers, not attributes: `5;<n>` for 256 and
+ * `2;<r>;<g>;<b>` for truecolor. Without this skip a truecolor foreground (`38;2;…`) reads as SGR 2
+ * — dim — and a line the user had typed would be mistaken for an empty composer, which is the one
+ * direction this must never be wrong in: the pane would read idle and be respawned under a draft.
+ */
+function sgrAttributes(params: readonly string[]): string[] {
+ const attributes: string[] = []
+ for (let index = 0; index < params.length; index += 1) {
+ const code = params[index]
+ if (code !== '38' && code !== '48' && code !== '58') {
+ attributes.push(code)
+ continue
+ }
+ const kind = params[index + 1]
+ index += kind === '5' ? 2 : kind === '2' ? 4 : 1
+ }
+ return attributes
+}
+
+/**
+ * The background an SGR sequence sets, as a comparable string — or null if it sets none.
+ *
+ * Needed because a composer box is identified by the colour it PAINTS, and the parameters that
+ * choose one are positional: `48;5;<n>` and `48;2;<r>;<g>;<b>` carry their arguments inline, so the
+ * arguments have to be consumed rather than read as further attributes. `38`/`58` are skipped the
+ * same way for the same reason — a truecolor FOREGROUND ends in numbers that would otherwise parse
+ * as a background code of their own.
+ *
+ * SGR 0 counts: a reset puts the default background back, which is a change like any other.
+ */
+function sgrBackground(params: readonly string[]): string | null {
+ for (let index = 0; index < params.length; index += 1) {
+ const code = params[index]
+ if (code === '48') {
+ const kind = params[index + 1]
+ return params.slice(index, index + (kind === '5' ? 3 : kind === '2' ? 5 : 2)).join(';')
+ }
+ if (code === '49' || /^(?:4[0-7]|10[0-7])$/.test(code)) return code
+ if (code === '0' || code === '') return '49'
+ if (code === '38' || code === '58') {
+ const kind = params[index + 1]
+ index += kind === '5' ? 2 : kind === '2' ? 4 : 1
+ }
+ }
+ return null
+}
+
+/**
+ * The span of a gutter line the BOX ITSELF paints — everything after the marker was wrong.
+ *
+ * A TUI draws one row of the terminal at a time, so anything floating OVER the composer lands on the
+ * composer's own lines. OpenCode's `Getting started` card is exactly that: a panel on the right-hand
+ * side, and its rows are written after the composer's on the same lines. Reading to end-of-line took
+ * the card's text as something the user had typed —
+ *
+ * ┃ <121 spaces of composer> <sidebar> Connect provider /connect
+ *
+ * so `draft` was true over an EMPTY composer, the pane never read idle, and every model switch came
+ * back AGENT_BUSY. Measured on a live pane (`harness-opencode-*`, 2026-09-15) with nothing typed in.
+ * Fourth variant of the family above: the marker is found, the strip is dropped, and then the wrong
+ * COLUMNS are read.
+ *
+ * The boundary is the box's own background. Everything a box paints is painted in it, so the
+ * interior runs from the background set right after the marker to the next background change — in
+ * the capture above, back to the pane's `48;2;10;10;10` where the box ends and the sidebar begins.
+ *
+ * Two deliberate conservatisms, both pointing the same way — a real draft must never be read as an
+ * empty prompt, because that respawns the engine under text the user typed:
+ * * A background that appears only AFTER visible text is not the interior's. It is a highlight
+ * drawn mid-draft, and treating it as the boundary would cut the draft off at its own styling.
+ * * A line that sets no background at all is not clipped. Plain-text panes (and every engine that
+ * paints no box) keep the whole line, exactly as before.
+ */
+function gutterBoxInterior(line: string, markerIndex: number): string {
+ const after = line.slice(markerIndex + 1)
+ const sgr = /\u001b\[([0-9;:]*)m/g
+ let interior: string | null = null
+ let end = after.length
+ let match: RegExpExecArray | null
+ while ((match = sgr.exec(after)) !== null) {
+ const background = sgrBackground(match[1].split(/[;:]/))
+ if (background === null) continue
+ if (interior === null) {
+ if (stripAnsi(after.slice(0, match.index)).trim().length > 0) break
+ interior = background
+ continue
+ }
+ if (background !== interior) { end = match.index; break }
+ }
+ return after.slice(0, end)
+}
+
+/**
+ * Is the text after the composer marker the PLACEHOLDER, rather than something the user typed?
+ *
+ * Read from the STYLING, never the words: the placeholder sentence differs per engine and changes
+ * between versions, but every one of these TUIs greys its placeholder out somehow and none of them
+ * styles a draft the same way.
+ *
+ * Two attributes, because two are in use. SGR 2 (dim) is what claude, codex and devin draw; SGR 3
+ * (italic) is what hermes draws — its `Ask anything, or type / for commands…` is italic over a
+ * 256-colour gold (`ESC[3m ESC[38;5;136m`) and carries no dim at all. Looking only for dim made
+ * EVERY hermes pane look like a pane with a draft in it, so `idle` was false for the agent's whole
+ * life and every retarget and model switch came back AGENT_BUSY over an engine sitting at an empty
+ * prompt. Same failure the comment on `promptMarker` warns about, one step further along: the
+ * marker was found, and then the placeholder was not recognised as one.
+ */
+function hasPlaceholderSgr(value: string): boolean {
+ return [...value.matchAll(/\u001b\[([0-9;:]*)m/g)]
+ .some((match) => {
+ const attributes = sgrAttributes(match[1].split(/[;:]/))
+ return attributes.includes('2') || attributes.includes('3')
+ })
+}
+
+/**
+ * Is this SGR foreground a MUTED one — the grey a TUI writes hints and placeholders in?
+ *
+ * Returns null when the sequence sets no foreground, so a caller can keep the colour already in
+ * effect rather than treating "no change" as a change.
+ *
+ * Grey is judged by the colour itself: equal channels, and dark enough to read as secondary next to
+ * the near-white a composer draws real text in. OpenCode's placeholder is `38;2;128;128;128` and its
+ * own text `38;2;255;255;255`; the 256-colour ramp (232–255) and SGR 90 are the same intent spelled
+ * differently.
+ */
+function sgrMutedForeground(params: readonly string[]): boolean | null {
+ for (let index = 0; index < params.length; index += 1) {
+ const code = params[index]
+ if (code === '38') {
+ const kind = params[index + 1]
+ if (kind === '2') {
+ const [r, g, b] = [params[index + 2], params[index + 3], params[index + 4]].map(Number)
+ if (![r, g, b].every(Number.isFinite)) return null
+ return r === g && g === b && r >= 0x40 && r <= 0xb0
+ }
+ if (kind === '5') {
+ const n = Number(params[index + 2])
+ if (!Number.isFinite(n)) return null
+ // 8 is bright black; 232–255 is the grey ramp, whose middle is the secondary tone.
+ return n === 8 || (n >= 236 && n <= 250)
+ }
+ return null
+ }
+ if (code === '90') return true
+ if (code === '39' || code === '0' || code === '') return false
+ if (/^(?:3[0-7]|9[1-7])$/.test(code)) return false
+ if (code === '48' || code === '58') {
+ const kind = params[index + 1]
+ index += kind === '5' ? 2 : kind === '2' ? 4 : 1
+ }
+ }
+ return null
+}
+
+/**
+ * Is EVERY visible character here written in a muted colour — i.e. is this a hint rather than a draft?
+ *
+ * The third placeholder styling this module has had to learn, after dim (claude, codex, devin) and
+ * italic (hermes). OpenCode draws `Ask anything… "What is the tech stack of this project?"` as a plain
+ * TRUECOLOR GREY with no dim and no italic attribute at all, so [hasPlaceholderSgr] saw nothing and a
+ * brand-new pane read as a pane with a draft in it: never idle, and every model switch on an agent
+ * with no messages yet refused as AGENT_BUSY over an empty composer. Measured on 1.18.31.
+ *
+ * ⚠️ EVERY character, not any — which is what keeps this from swallowing a real draft. A composer
+ * writes what the user typed in its normal near-white; one such character anywhere means this is
+ * text, whatever grey hint may be sitting beside it. Spaces are not evidence of either and carry no
+ * colour worth reading, so they are skipped.
+ */
+function allVisibleTextIsMuted(value: string): boolean {
+ const sgr = /\u001b\[([0-9;:]*)m/g
+ let muted: boolean | null = null
+ let cursor = 0
+ let sawText = false
+ let match: RegExpExecArray | null
+ const segment = (text: string): boolean => {
+ if (!text.trim()) return true
+ sawText = true
+ return muted === true
+ }
+ while ((match = sgr.exec(value)) !== null) {
+ if (!segment(value.slice(cursor, match.index))) return false
+ const next = sgrMutedForeground(match[1].split(/[;:]/))
+ if (next !== null) muted = next
+ cursor = match.index + match[0].length
+ }
+ if (!segment(value.slice(cursor))) return false
+ return sawText
+}
+
+/**
+ * The glyph each CLI puts in front of its composer. They do not share one: cursor uses `→`, devin `❭`
+ * (U+276D — close to, but not, claude's `❯` U+276F), and pi draws no marker at all. Getting this wrong is
+ * not cosmetic — with no prompt found the pane never reads idle, and every switch is refused as BUSY.
+ */
+function promptMarker(engine: AgentEngine): RegExp {
+ if (engine === 'cursor') return /→/u
+ if (engine === 'devin') return /[❭›❯]/u
+ if (engine === 'opencode') return /┃/u
+ return /[›❯]/u
+}
+
+export function inspectPane(engine: AgentEngine, capture: string, takeover: Takeover = () => null, planPattern = /\bplan mode on\b/i): PaneInspection {
+ if (engine === 'opencode') return inspectGutterBoxPane(capture, true)
+ if (engine === 'copilot') return inspectGutterBoxPane(capture, false)
+ if (engine === 'grok') return inspectGrokPane(capture)
+ const rawLines = capture.split('\n')
+ const marks = promptMarker(engine)
+ const promptIndex = latestPromptLine(rawLines, marks)
+ const prompt = promptIndex >= 0 ? rawLines[promptIndex] : ''
+ const currentUi = currentPaneUi(rawLines, promptIndex)
+ const dialog = DIALOG_UI.test(currentUi) || takeover(capture, currentUi) !== null
+ const plan = planPattern.test(currentUi)
+ const marker = stripAnsi(prompt).search(marks)
+ let visible = marker >= 0 ? stripAnsi(prompt).slice(marker + 1).replace(/\u00a0/g, ' ').trim() : ''
+ const rawMarker = prompt.search(marks)
+ const rawTail = rawMarker >= 0 ? prompt.slice(rawMarker + 1) : ''
+ const placeholder = hasPlaceholderSgr(rawTail)
+ if (placeholder) visible = ''
+ const draft = visible.length > 0
+ return { idle: !!prompt && !dialog && !draft, plan, dialog, draft }
+}
+
+/** The latest composer prompt line; picker rows use the same glyphs, but numbered rows are not prompts. */
+function latestPromptLine(rawLines: string[], marks: RegExp): number {
+ return rawLines.findLastIndex((line) => {
+ const visible = stripAnsi(line)
+ const marker = visible.search(marks)
+ return marker >= 0 && !/^\s*\d+\.\s/.test(visible.slice(marker + 1))
+ })
+}
+
+/**
+ * Old picker/plan text can remain in tmux history. Only UI below the latest prompt belongs to the
+ * current interaction; if no prompt is visible, inspect the whole capture as a conservative fallback.
+ */
+function currentPaneUi(rawLines: string[], promptIndex: number): string {
+ return stripAnsi((promptIndex >= 0 ? rawLines.slice(promptIndex) : rawLines).join('\n')).replace(/\u00a0/g, ' ')
+}
+
+export function paneModal(engine: AgentEngine, capture: string, readTakeover: Takeover = () => null): PaneTakeover | 'permission' | 'menu' | null {
+ const rawLines = capture.split('\n')
+ const currentUi = currentPaneUi(rawLines, latestPromptLine(rawLines, promptMarker(engine)))
+ const takeover = readTakeover(capture, currentUi)
+ if (takeover) return takeover
+ // Below the prompt line: that line is the composer, or the echo of the last message, and either can
+ // hold anything a person typed.
+ if (PERMISSION_UI.test(currentUi.slice(currentUi.indexOf('\n') + 1))) return 'permission'
+ return DIALOG_UI.test(currentUi.replace(/Starting MCP servers?/gi, '')) ? 'menu' : null
+}
+
+const DIALOG_UI = /Select Model(?: and Effort)?|Select Reasoning Level|Advanced Reasoning|Available models|Models matching|Edit Parameters|Type to filter.*Tab to edit|Esc to go back|Press enter to confirm|Do you want to proceed|Allow this action|permission required|Starting MCP servers?/i
+
+/** The approval prompts among them. */
+/**
+ * An approval prompt's question. Claude Code asks "Do you want to proceed?", "Do you want to make this
+ * edit to <file>?", "Do you want to allow Claude to fetch this content?", "Do you want to <verb> <target>?"
+ * for the other tools, and "Would you like to proceed?" over a plan (2.1.289); Codex asks "Would you like
+ * to run the following command?" and the like. The rows under it are read too (askQuestion.ts), but a
+ * row wrapped in a narrow pane breaks that reading, and the question alone must still hold a message:
+ * Enter approves the first row. Asked only below the prompt line, so never of what a person typed.
+ */
+const PERMISSION_UI = /\bDo you want to\b|\bWould you like to (?:proceed|run|make|apply|allow)\b|Allow this action|permission required/i
+
+/** The rule a gutter-box composer is closed with: `╹▀▀▀▀…`. */
+const GUTTER_BOX_RULE = /[─▀▁▔]{8,}/u
+
+/**
+ * A composer drawn as a BOX with a `┃` gutter — OpenCode's and Copilot's, which are the same shape.
+ *
+ * The generic reader cannot read either one. OpenCode's `┃` does not stop at the text: the last
+ * gutter line is the box's own STATUS strip — `Build · <model> · <account>`, sitting directly on
+ * the rule that closes the box — so taking the last line carrying the marker reads what the BOX says
+ * about itself as something the user typed. Copilot fails one step earlier: its composer carries no
+ * `›`/`❯` at all, while every message the user has ALREADY SENT is echoed into the transcript as
+ * `❯ <text>` in full brightness, so the generic reader walks back to the last of those and reads a
+ * sent message as a draft. Either way the pane was permanently "holding a draft": never idle, and
+ * every retarget and model switch refused as AGENT_BUSY over an empty composer. Third variant of the
+ * failure hermes had — there the placeholder was not recognised, here the wrong line is read.
+ *
+ * [hasStatusStrip] is the one thing that differs between the two, and it is OpenCode's alone. The
+ * strip is identified by WHERE it is — the gutter line the closing rule sits under — rather than by
+ * what it says, which is three variable fields that change with the model, the mode and the account.
+ * When no rule follows, no strip is assumed and every gutter line counts as composer: a layout we do
+ * not recognise must not silently swallow a line the user typed in. Copilot's box has no strip, so
+ * its single gutter line IS the composer and dropping it would read a real draft as an empty prompt
+ * — the one direction this must never be wrong in.
+ *
+ * Every gutter line is checked, not just the last: a long message wraps down the box, and reading
+ * one line of it would call a full composer empty.
+ *
+ * A dialog needs no special case in either engine: both REPLACE the composer with the picker or the
+ * permission prompt, so no gutter is on screen at all and the pane reads not-idle from the empty run
+ * below rather than from [DIALOG_UI].
+ */
+function inspectGutterBoxPane(capture: string, hasStatusStrip: boolean): PaneInspection {
+ const rawLines = capture.split('\n')
+ const marks = /┃/u
+ // The LAST contiguous run of gutter lines. Earlier runs are previous composers still in scrollback
+ // — the message the user already sent is not a draft.
+ const gutter: number[] = []
+ for (let index = rawLines.length - 1; index >= 0; index -= 1) {
+ if (marks.test(stripAnsi(rawLines[index]))) { gutter.unshift(index); continue }
+ if (gutter.length > 0) break
+ }
+ if (gutter.length === 0) return { idle: false, plan: false, dialog: false, draft: false }
+ const last = gutter[gutter.length - 1]
+ const closed = last + 1 < rawLines.length && GUTTER_BOX_RULE.test(stripAnsi(rawLines[last + 1]))
+ const composer = hasStatusStrip && closed ? gutter.slice(0, -1) : gutter
+ const currentUi = stripAnsi(rawLines.slice(gutter[0]).join('\n')).replace(/\u00a0/g, ' ')
+ const dialog = DIALOG_UI.test(currentUi)
+ const draft = composer.some((index) => {
+ const line = rawLines[index]
+ const visibleMarker = stripAnsi(line).search(marks)
+ if (visibleMarker < 0) return false
+ const rawMarker = line.search(marks)
+ // The box's interior, not the rest of the row: a panel floating over the composer writes into
+ // these same lines, and its text is not a draft. See [gutterBoxInterior]. With no marker in the
+ // RAW line there is no interior to bound — read the whole row, the direction that keeps a draft.
+ const interior = rawMarker < 0
+ ? stripAnsi(line).slice(visibleMarker + 1)
+ : gutterBoxInterior(line, rawMarker)
+ if (hasPlaceholderSgr(interior) || allVisibleTextIsMuted(interior)) return false
+ return stripAnsi(interior).replace(/\u00a0/g, ' ').trim().length > 0
+ })
+ return { idle: !dialog && !draft, plan: /\bplan mode on\b/i.test(currentUi), dialog, draft }
+}
+
+/**
+ * The context readout Pi's footer always carries — `0.0%/500k`, `1.5%/200k`. Present whether or not the
+ * model advertises a thinking ladder, and drawn by the normal view alone, which is what makes it the
+ * honest "the footer is on screen" signal. No `g` flag, so `test` carries no `lastIndex` between calls.
+ */
+
+
+/**
+ * Grok draws its composer as a ROUNDED BOX, and the generic reader counts the box's own right edge
+ * as something the user typed.
+ *
+ * Its empty composer is one line — `│ ❯` … padding … `│` — inside `╭─…─╮` above and
+ * `╰─… Grok 4.6 (xhigh) ─╯` below. The shared reader takes everything after `❯`, strips ANSI, trims,
+ * and finds `│`: one character, non-empty, so `draft` was true. Measured on a real pane
+ * (`harness-grok-*`, 2026-09-09) with nothing typed into it: `visible after marker = "│"`, `draft =
+ * true`, `idle = false`. Not a transient — the edge is drawn on every frame, so idle was false for
+ * the agent's whole life and EVERY model switch came back AGENT_BUSY over an empty prompt. Same
+ * family as the hermes and gutter-box bugs above; here the culprit is the box, not the placeholder.
+ *
+ * So the composer is the box's INTERIOR: the run of `│` body lines, read between their first and
+ * last `│` rather than from the marker to end-of-line. Every body line is checked, not just the one
+ * carrying `❯` — a long message wraps down the box and reading one line of it would call a full
+ * composer empty.
+ *
+ * ⚠️ `│` is also Grok's footer separator (`Shift+Tab:mode │ Ctrl+x:shortcuts`), which sits BELOW the
+ * box. It is excluded by requiring a body line to both open and close with `│`, which the footer —
+ * one separator, no edges — never does.
+ */
+function inspectGrokPane(capture: string): PaneInspection {
+ const rawLines = capture.split('\n')
+ const visibleLines = rawLines.map((line) => stripAnsi(line).replace(/ /g, ' '))
+ // The LAST box in the capture. Earlier ones are composers already sent, still in scrollback.
+ const bottom = visibleLines.findLastIndex((line) => /^\s*╰[─╌]*.*╯\s*$/u.test(line))
+ const top = bottom < 0
+ ? -1
+ : visibleLines.slice(0, bottom).findLastIndex((line) => /^\s*╭[─╌]+╮\s*$/u.test(line))
+ // No recognisable box is "we cannot tell", which the caller reads as busy — the safe direction.
+ // Respawning a pane whose layout we do not understand could discard a draft.
+ if (top < 0 || bottom - top < 2) return { idle: false, plan: false, dialog: false, draft: false }
+ const currentUi = visibleLines.slice(top).join('\n')
+ const dialog = DIALOG_UI.test(currentUi)
+ const draft = rawLines.slice(top + 1, bottom).some((line, offset) => {
+ const visible = visibleLines[top + 1 + offset]
+ const first = visible.indexOf('│')
+ const last = visible.lastIndexOf('│')
+ // Both edges, and not the same character: anything else is not a body line of this box.
+ if (first < 0 || last <= first) return false
+ const rawMarker = line.search(/[›❯]/u)
+ if (rawMarker >= 0 && hasPlaceholderSgr(line.slice(rawMarker + 1))) return false
+ // The marker is chrome, not content — it is printed whether or not anything was typed.
+ return visible.slice(first + 1, last).replace(/[›❯]/u, '').trim().length > 0
+ })
+ return { idle: !dialog && !draft, plan: /\bplan mode on\b/i.test(currentUi), dialog, draft }
+}
+
+/**
+ * Pi draws no composer marker, so idleness is read from its layout instead: the composer is the band
+ * between the last two `───` rules, and the footer (`minimax/minimax-m3 • high`) is only rendered by the
+ * normal view. A non-empty band is a draft the user is still typing — injecting there would splice a
+ * slash command into their sentence.
+ */
diff --git a/cli/src/engines/kit/questionPane.ts b/cli/src/engines/kit/questionPane.ts
new file mode 100644
index 000000000..4feeebd43
--- /dev/null
+++ b/cli/src/engines/kit/questionPane.ts
@@ -0,0 +1,395 @@
+import type { FoundDialog, PaneView, QuestionRow, QuestionView } from '../facets/screen.js'
+import { earlierDialogEnd, PERMISSION_FOOTER_RE, QUESTION_FOOTER_RE } from '../../lib/dialogEnd.js'
+
+// Rows that exist in every dialog but can never BE an answer.
+const CHAT_ROW = /^chat about (this|these)$/i
+// Claude writes "Type something.", Command Code "Type something..." — one trailing dot or three, and
+// missing it costs the free-text row: a voice answer would have nowhere to go and the row would be
+// offered on the device as if it were a real option.
+// Claude "Type something.", Command Code "Type something...", OpenCode "Type your own answer", Hermes
+// "Other (type your answer)". Same row in every dialog: it opens an editor instead of choosing, so it must
+// never be offered to the device as a selectable label.
+const TYPE_ROW = /^(type something\.{0,3}|type your own( answer)?|other \(type your (own|answer)\))$/i
+
+export function stripAnsi(value: string): string {
+ return value.replace(/\u001b\[[0-9;:]*[A-Za-z]/g, '')
+}
+
+function norm(value: string): string {
+ return value.toLowerCase().replace(/\s+/g, ' ').replace(/[.…]+$/, '').trim()
+}
+
+export function parseRow(line: string): QuestionRow | null {
+ // Consume indentation once: two adjacent whitespace runs backtrack across
+ // every possible split on the padded non-option lines in terminal captures.
+ const m = /^[❯›>]?\s*(\d+)\.\s+(.+?)\s*$/.exec(line.trimStart())
+ if (!m) return null
+ const raw = m[2]
+ const box = /^\[([^\]])\]\s*(.*)$/.exec(raw)
+ return {
+ number: m[1],
+ label: (box ? box[2] : raw).trim(),
+ checked: !!box && box[1].trim() !== '',
+ }
+}
+
+/**
+ * Read the CURRENT dialog off a pane capture. The capture includes scrollback (old dialogs, old plan
+ * text), so everything is anchored to the LAST footer line — the only marker the live dialog always
+ * paints at the bottom of the screen.
+ */
+/** "❯ 1. Submit" immediately followed by "2. Cancel" — Command Code's review screen, which carries no
+ * other marker. Returns the Submit row's number and where it sits, or null. */
+function findSubmitPair(lines: string[]): { row: string; index: number } | null {
+ for (let i = lines.length - 1; i >= 1; i--) {
+ const row = parseRow(lines[i])
+ if (!row || !/^submit$/i.test(row.label)) continue
+ const next = parseRow(lines[i + 1] ?? '')
+ if (next && /^cancel$/i.test(next.label)) return { row: row.number, index: i }
+ }
+ return null
+}
+
+/** The tab bar of a footer-less dialog: "● Loại game | ◯ Review". Returns its LAST occurrence, or -1. */
+function findTabBarDialog(lines: string[]): number {
+ for (let i = lines.length - 1; i >= 0; i--) {
+ const line = lines[i].trim()
+ if (!/^[●◯✔✓]/.test(line) || !line.includes('|')) continue // answered tabs turn into ✔
+ // Only a tab bar with a numbered list under it is a question; the same glyphs appear in prose.
+ for (let j = i + 1; j < lines.length && j - i <= 8; j++) {
+ const row = parseRow(lines[j])
+ if (row?.number === '1') return i
+ }
+ }
+ return -1
+}
+
+/** Read a dialog that has no footer: question first, then the numbered rows, top-down from the tab bar. */
+function parseDownward(lines: string[], tabBar: number): PaneView {
+ let question = ''
+ const rows: QuestionRow[] = []
+ let checkbox = false
+ for (let i = tabBar + 1; i < lines.length; i++) {
+ const row = parseRow(lines[i])
+ if (row) {
+ if (/^\s*[❯›>]?\s*\d+\.\s+\[/.test(lines[i])) checkbox = true
+ rows.push(row)
+ continue
+ }
+ // Between the tab bar and row 1 sits the question; after the rows start, plain lines are the option
+ // descriptions and must not overwrite it.
+ const line = lines[i].trim()
+ if (!question && rows.length === 0 && line && !/^[─━-]{6,}$/.test(line)) question = line
+ }
+ if (!rows.length) return null
+ const answerable = rows.filter((r) => !CHAT_ROW.test(r.label) && !TYPE_ROW.test(r.label))
+ return {
+ kind: 'question',
+ question,
+ rows: answerable,
+ multi: checkbox,
+ typeRow: rows.find((r) => TYPE_ROW.test(r.label)) ?? null,
+ }
+}
+
+// \u2500\u2500 permission prompts \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500
+
+/**
+ * A permission prompt is a QUESTION whose options are the approval choices \u2014 the shape amp and kilo
+ * already ship (`engines/amp/askQuestion.ts`, `engines/kilo/askQuestion.ts`). It matters far more on a
+ * remote machine than a question does: the CLI attaches to an agent the USER started, under the user's
+ * own config and with no permission flag of ours (`engines/README.md`), so a blocking approval is the
+ * normal state of a pane, not an edge case. Unparsed, the turn simply sits at `Processing` until someone
+ * walks to the computer.
+ *
+ * Claude and Command Code draw it as a framed block ending in numbered rows, and it carries NONE of the
+ * three anchors `parseQuestionPane` knows. Captured live (`__fixtures__/permission-claude.txt`):
+ *
+ * \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500
+ * Bash command \u2190 the header: what kind of approval this is
+ *
+ * curl -s https://api.coingecko.com/\u2026 \u2190 the argument, then a one-line description
+ * Fetch Bitcoin price from CoinGecko API
+ *
+ * This command requires approval
+ *
+ * Do you want to proceed?
+ * \u276f 1. Yes
+ * 2. Yes, and don\u2019t ask again for: curl *
+ * 3. No
+ *
+ * Esc to cancel \u00b7 Tab to amend \u00b7 ctrl+e to explain
+ *
+ * Command Code differs only in wording (`permission-commandcode.txt`): header `Execute Shell Command`,
+ * a sentence for the body, footer `\u2191/\u2193 navigate \u00b7 enter select \u00b7 ctrl+e explain`. Same frame, same rows \u2014
+ * which is why one parser covers both instead of two near-copies that would drift apart.
+ *
+ * **The prose is not the anchor.** Every engine words it differently and Claude alone writes at least
+ * three ("Do you want to proceed?", "Do you want to make this edit to README.md?", "Do you want to create
+ * \u2026?"). The ROWS are: an approval always offers a way to say yes and a way to say no. That pair is the
+ * signal, guarded by the dialog's own key hints so an ordinary numbered list in assistant output can never
+ * be read as a prompt.
+ */
+
+/** The first row of an approval, measured across claude, commandcode, codex, devin, grok, amp and kilo. */
+const APPROVE_RE = /^(yes|allow|approve|accept|proceed|run|continue)\b/i
+/** \u2026and the row that declines it. Never dropped \u2014 see the rule amp's parser states: a device user given
+ * three ways to say yes and none to say no cannot answer the prompt at all. */
+// `skip` is here because cursor's decline row is "Skip & tell the agent what to do instead" — an engine
+// whose only way to refuse says neither "no" nor "reject" would otherwise fail the yes/no guard and its
+// whole prompt would go unread.
+// Claude's plan review does not use a negative verb at all: its only refusal
+// row is "Tell Claude what to change". Keep `tell ... what to change` narrow so
+// an ordinary numbered list beginning with "Tell" cannot become an approval.
+const REJECT_RE = /^(no|reject|deny|decline|cancel|skip|don'?t|stop)\b|^tell\b.*\bwhat to change\b/i
+/** The solid rule that opens the frame. Deliberately NOT the dashed one (`\u254c`) that brackets an edit diff,
+ * which sits BELOW the header and would cost the title. */
+const FRAME_RULE_RE = /^\s*[\u2500\u2501\u2550]{6,}\s*$/
+/** Any rule, solid or dashed \u2014 used to skip them while reading the frame's contents. */
+const ANY_RULE_RE = /^\s*[\u2500\u2501\u2550\u254c\u2504\u2508-]{6,}\s*$/
+/** The dialog's own question line, and Command Code's `Press [ctrl+e] \u2026` hint: both sit between the
+ * header and the rows, and neither says what is being approved. */
+const PERMISSION_PROSE_RE = /^((do|would) you\b|press \[)/i
+
+/** The opening rule of the frame the rows at `start` sit in, or -1 when they have none of their own. */
+function frameTop(lines: string[], start: number, floor: number): number {
+ for (let i = start - 1; i > floor && start - i <= 25; i--) {
+ if (FRAME_RULE_RE.test(lines[i])) return i
+ }
+ return -1
+}
+
+/** Keep a synthesised title inside the device's `text[200]` buffer, with the tail marked as cut. */
+function clipTitle(value: string): string {
+ return value.length <= 160 ? value : `${value.slice(0, 159)}\u2026`
+}
+
+/**
+ * What is being approved, as one line: `Approve <header>: <argument>`, the form amp's parser already
+ * produces so the two read alike on the device.
+ *
+ * The frame's opening rule is the only reliable top \u2014 the prose under it varies per engine AND per tool.
+ * Below the rule sit the header (`Bash command`, `Edit file`, `Execute Shell Command`) and then the
+ * argument: the command, the file, or the sentence naming it.
+ */
+function permissionTitle(lines: string[], start: number): string {
+ // The header must be THIS dialog's: never read past the end of an earlier one still in scrollback.
+ const floor = earlierDialogEnd(lines, start, 25)
+ const top = frameTop(lines, start, floor)
+ // No frame above the rows (codex draws none): the nearest text is the dialog's own question, which
+ // names the command outright. Better than inventing a header that is not on screen.
+ if (top < 0) {
+ for (let i = start - 1; i > floor && start - i <= 4; i--) {
+ const line = lines[i].trim()
+ if (line) return clipTitle(line)
+ }
+ return 'Approval required'
+ }
+ let header = ''
+ let arg = ''
+ for (let i = top + 1; i < start; i++) {
+ const line = lines[i].trim()
+ if (!line || ANY_RULE_RE.test(line)) continue
+ if (!header) { header = line; continue }
+ if (PERMISSION_PROSE_RE.test(line)) continue
+ arg = line
+ break
+ }
+ if (!header) return 'Approval required'
+ return clipTitle(arg ? `Approve ${header}: ${arg}` : `Approve ${header}`)
+}
+
+/**
+ * The permission prompt on screen, with the index of its first row so the caller can rank it against the
+ * other anchors.
+ *
+ * Only the LOWEST numbered block on the pane is considered, and it is rejected outright if it is not an
+ * approval. Digging further up would find an answered prompt in the scrollback and re-announce it \u2014 the
+ * live dialog is always the bottom-most thing on screen, so there is nothing below it to miss.
+ */
+export function parsePermissionPane(lines: string[]): { view: QuestionView; index: number } | null {
+ let end = -1
+ for (let i = lines.length - 1; i >= 0; i--) {
+ if (parseRow(lines[i])) { end = i; break }
+ }
+ if (end < 0) return null
+
+ // Walk up through CONTIGUOUS rows to the one numbered 1. Contiguity is the point: a permission dialog
+ // never interleaves prose with its options, while assistant output that happens to be numbered does.
+ const rows: QuestionRow[] = []
+ let start = -1
+ for (let i = end; i >= 0 && end - i < 12; i--) {
+ const row = parseRow(lines[i])
+ if (!row) break
+ rows.unshift(row)
+ if (row.number === '1') { start = i; break }
+ }
+ if (start < 0 || rows.length < 2) return null
+ if (!APPROVE_RE.test(rows[0].label) || !rows.some((row) => REJECT_RE.test(row.label))) return null
+
+ let hinted = false
+ for (let i = end + 1; i < lines.length && i - end <= 4; i++) {
+ if (PERMISSION_FOOTER_RE.test(lines[i])) { hinted = true; break }
+ }
+ if (!hinted) return null
+
+ // Single-select, and no free-text row: every option here is a choice to be TAPPED. Verified on live
+ // panes for claude, codex, devin and grok \u2014 one digit selects and submits, exactly as `rowKeys` assumes.
+ return {
+ view: { kind: 'question', permission: true, question: permissionTitle(lines, start), rows, multi: false, typeRow: null, dialog: permissionDialog(lines, start, end) },
+ index: start,
+ }
+}
+
+/** Every line of the dialog, untruncated: from its opening rule (or, with no frame, up to 12 lines above
+ * the rows) to its last row — never reaching back past the end of an earlier dialog. */
+function permissionDialog(lines: string[], start: number, end: number): string {
+ const floor = earlierDialogEnd(lines, start, 25)
+ const rule = frameTop(lines, start, floor)
+ const top = rule >= 0 ? rule + 1 : Math.max(0, start - 12, floor + 1)
+ return lines.slice(top, end + 1).map((line) => line.trimEnd()).filter((line) => !ANY_RULE_RE.test(line)).join('\n').trim()
+}
+
+export function parseQuestionPane(capture: string): PaneView {
+ return locateQuestionPane(capture)?.view ?? null
+}
+
+/** `parseQuestionPane`, with the line its dialog was anchored on. */
+export function locateQuestionPane(capture: string): FoundDialog | null {
+ const lines = stripAnsi(capture).replace(/\u00a0/g, ' ').split('\n')
+ // Each CLI words its own footer, and OpenCode rewords it PER SCREEN — `enter submit` on a single
+ // question, `enter toggle` on a multi-select, `enter confirm` on a step of a multi-question. They all
+ // mark the same thing: the bottom of a live dialog.
+ // `enter to submit answer` is Codex's request_user_input dialog (plan mode; captured live in
+ // __fixtures__/question-codex.txt). Without it that dialog was never a question at all here: the dial
+ // showed nothing while the pane waited, and a question it DID show could never be closed, because
+ // the watcher had no fingerprint to notice leaving.
+ const footer = lines.findLastIndex((l) => QUESTION_FOOTER_RE.test(l))
+ // The review screen paints no footer and puts its rows BELOW the prompt, so it needs its own anchor.
+ // Whichever anchor is LOWER on screen is the live one (the other is scrollback from an earlier step).
+ const review = lines.findLastIndex((l) => /Ready to submit your answers/i.test(l))
+ // Command Code's review screen has neither Claude's "Ready to submit your answers" line nor a footer:
+ // it is a "Submit"/"Cancel" pair under a summary that is itself numbered. Anchor on that PAIR — the two
+ // rows adjacent, in that order — because the summary lines above are numbered too and reading them as
+ // options is how the device answered everything and then sat there, never submitting.
+ const submit = findSubmitPair(lines)
+ // Command Code paints the SAME dialog with no footer at all — the pane simply ends at the last option.
+ // Its tab bar is the only thing above the rows that is unmistakably part of the dialog, so it anchors
+ // that dialog, read DOWNWARD. Nothing else on either CLI's screen looks like "● X | ◯ Y" followed by a
+ // numbered list, which is what keeps ordinary numbered output from being read as a question.
+ const tabBar = findTabBarDialog(lines)
+ // A permission prompt is one more anchor, and every anchor is ranked the same way: whichever sits
+ // lowest on screen is the live dialog, the others are scrollback. That ordering is what keeps the two
+ // apart in both directions — codex and hermes draw an approval whose footer the question anchor also
+ // matches, and there the footer is BELOW the rows, so the question path (which reads a better title off
+ // the same block) still wins. And a footer-less dialog under an answered one is the live one: anchored
+ // on the answered one's footer instead, Command Code's question was announced as the OLD question.
+ const permission = parsePermissionPane(lines)
+ if (permission && permission.index > Math.max(footer, review, submit?.index ?? -1, tabBar)) {
+ return { view: permission.view, at: permission.index }
+ }
+ if (tabBar > Math.max(footer, review, submit?.index ?? -1)) {
+ const view = parseDownward(lines, tabBar)
+ return view && { view, at: tabBar }
+ }
+ if (review > footer && review > (submit?.index ?? -1)) {
+ for (let i = review + 1; i < lines.length && i - review <= 10; i++) {
+ const row = parseRow(lines[i])
+ if (row && /^submit answers$/i.test(row.label)) return { view: { kind: 'review', submitRow: row.number }, at: review }
+ }
+ return null
+ }
+ if (submit && submit.index > footer) return { view: { kind: 'review', submitRow: submit.row }, at: submit.index }
+ if (footer < 0) return null
+
+ // Rows belonging to this dialog: the numbered rows just above the footer, back to the row numbered 1 —
+ // and never from above an earlier dialog's end, which is where a dialog whose top is scrolled out of
+ // the pane would otherwise borrow its first rows.
+ const rows: QuestionRow[] = []
+ let checkbox = false // `[ ]` / `[✔]` on a row ⇒ this question is multi-select
+ let start = -1
+ const floor = earlierDialogEnd(lines, footer, 40)
+ for (let i = footer - 1; i > floor && footer - i <= 40; i--) {
+ const row = parseRow(lines[i])
+ if (!row) continue
+ rows.unshift(row)
+ if (/^\s*[❯›>]?\s*\d+\.\s+\[/.test(lines[i])) checkbox = true
+ if (row.number === '1') { start = i; break }
+ }
+ const enterSubmits = /enter to submit answer/i.test(lines[footer])
+ if (rows.length && start < 0 && enterSubmits) {
+ return { view: { kind: 'question', partial: true, enterSubmits, question: '', rows, multi: checkbox, typeRow: null }, at: footer }
+ }
+ if (start < 0 || rows.length === 0) return null
+
+ // The question is the nearest real text line above the rows. The dialog paints it between its header
+ // and its rows — `[tab bar | header chip] · blank · question · blank · rows` — so those, and a rule,
+ // are the TOP of this frame. Stop there, never skip past: mid-repaint the question line can be blank
+ // for one capture, and walking on would pick up the PREVIOUS question still sitting in scrollback and
+ // pair a stale title with the live options. An empty result just means "look again next tick". The
+ // end of an earlier dialog is a top too: its footer is not this dialog's question.
+ let question = ''
+ const top = earlierDialogEnd(lines, start, 12)
+ for (let i = start - 1; i > top && start - i <= 12; i--) {
+ const line = lines[i].trim()
+ if (!line) continue
+ if (/[←→]/.test(line) || /^[☐☒✔✓]/.test(line) || /^[─━-]{6,}$/.test(line)) break
+ // Hermes's batch panel marks the active question with `▸`; that marker is chrome, not the question.
+ question = line.replace(/^▸\s*/, '')
+ break
+ }
+
+ const answerable = rows.filter((r) => !CHAT_ROW.test(r.label) && !TYPE_ROW.test(r.label))
+ return {
+ view: {
+ kind: 'question',
+ ...(enterSubmits ? { enterSubmits } : {}),
+ question,
+ rows: answerable,
+ multi: checkbox,
+ typeRow: rows.find((r) => TYPE_ROW.test(r.label)) ?? null,
+ dialog: dialogAbove(lines, start, footer),
+ },
+ at: footer,
+ }
+}
+
+/**
+ * A footer-anchored dialog, whole: up to 12 lines above its rows — stopping at a rule, a tab bar, the
+ * agent's own output (a `•`/`⏺` bullet) or the end of an earlier dialog — down to its last row. Codex puts
+ * the command it asks about here (`$ …`, wrapped over as many lines as it takes), with its reason and
+ * environment.
+ */
+function dialogAbove(lines: string[], start: number, footer: number): string {
+ let top = start
+ const floor = earlierDialogEnd(lines, start, 12)
+ for (let i = start - 1; i > floor && start - i <= 12; i--) {
+ const line = lines[i].trim()
+ if (/^[•⏺●]/.test(line) || /[←→]/.test(line) || /^[─━═-]{6,}$/.test(line)) break
+ top = i
+ }
+ return lines.slice(top, footer).map((line) => line.trimEnd()).join('\n').trim()
+}
+
+/**
+ * A dialog that asks to RUN or CHANGE something (an approval), whichever parser read it: a framed
+ * permission prompt, or a footer dialog whose first row approves, whose rows include a rejection, and
+ * which names a command (`$ …`) or asks "would you like to run / make …". What the pair's floor treats as
+ * a permission prompt (pair/classify.ts); nothing else reads it.
+ */
+export function isApprovalDialog(view: QuestionView): boolean {
+ if (view.permission) return true
+ if (!view.rows.length || !APPROVE_RE.test(view.rows[0].label) || !view.rows.some((row) => REJECT_RE.test(row.label))) return false
+ const text = view.dialog ?? view.question
+ return /(^|\n)\s*\$ /.test(text) || /would you like to (run|make|apply|execute|edit|write)/i.test(text)
+}
+
+/** Match an answer to a row. The device stores option labels in an 80-byte buffer, so a long label comes
+ * back truncated — prefix matches count, in both directions. */
+/**
+ * The keystrokes that commit one row.
+ *
+ * Every engine but Amp numbers its options, so the digit both selects and submits in one press. Amp
+ * draws an unnumbered list navigated with the arrow keys, so its rows carry an index and are reached by
+ * walking down to them.
+ */
diff --git a/cli/src/engines/kit/screen.ts b/cli/src/engines/kit/screen.ts
new file mode 100644
index 000000000..f4c42b5dc
--- /dev/null
+++ b/cli/src/engines/kit/screen.ts
@@ -0,0 +1,42 @@
+import { stripVTControlCharacters } from 'node:util'
+import type { AgentEngine } from '../types.js'
+import type { ComposerState, MessageHold, PaneInspection, PaneModal, PaneView, ScreenReading } from '../facets/screen.js'
+import { isApprovalDialog } from './questionPane.js'
+const HOLDS: Record<NonNullable<PaneModal>, MessageHold> = {
+ rewind: 'rewind_picker_open', transcript: 'transcript_open', search: 'search_open', trust: 'trust_open',
+ update: 'update_prompt_open', model: 'model_prompt_open', sign_in: 'sign_in_open', permission: 'permission_open', menu: 'menu_open',
+}
+export function assembleScreen(engine: AgentEngine, capture: string, pane: PaneInspection, question: PaneView,
+ modal: PaneModal = null, composer?: ComposerState, activity: ScreenReading['activity'] = null, stoppedGoal = false): ScreenReading {
+ const messageHold = question ? question.kind === 'question' && isApprovalDialog(question) ? 'permission_open' : 'question_open'
+ : modal ? HOLDS[modal] : composer === undefined || composer === 'ready' ? null : composer === 'popup' ? 'popup_open' : 'prompt_hidden'
+ const footer = stripVTControlCharacters(capture).split('\n').slice(-16).join('\n')
+ const busy = !!activity || /\bgoal\s+active\b|\bpursuing goal\b|\b[1-9]\d*\s+background\s+(?:tasks?|agents?)\b/i.test(footer)
+ return { pane, question, messageHold, teamHold: teamHold(engine, capture, pane, question), activity, busy, stoppedGoal }
+}
+function teamHold(engine: AgentEngine, capture: string, pane: PaneInspection, question: PaneView): string | null {
+ if (!capture) return 'team_waiting_unavailable'
+ if (pane.dialog || question) return 'team_waiting_user'
+ if (pane.draft) return 'team_waiting_draft'
+ if (!pane.idle) return 'team_waiting_idle'
+ const lines = capture.split('\n').map(stripVTControlCharacters)
+ if (/esc(?:ape)? to (?:interrupt|cancel|stop)/i.test(lines.slice(-15).join('\n'))) return 'team_waiting_idle'
+ // These composers permit an empty first line followed by a multiline human draft.
+ // The general profile inspector only needs the first line; team delivery needs every line.
+ if (['claude', 'codex', 'cursor', 'hermes'].includes(engine)) {
+ const index = lines.findLastIndex(line => /^\s*[›❯→]/u.test(line))
+ if (index < 0) return 'team_waiting_idle'
+ // Codex shows its styled placeholder only when the whole composer is empty.
+ // Its configurable model/cwd/task footer below that placeholder is not draft
+ // text. The profile inspector has already proved this visible prompt text is
+ // a placeholder; a bare first line still needs the multiline-draft check.
+ if (engine === 'codex' && lines[index].replace(/^\s*[›❯→]\s*/u, '').trim()) return null
+ for (const line of lines.slice(index + 1)) {
+ if (/[─━―]{8,}/u.test(line)) break
+ const text = line.trim()
+ if (!text || /^(?:\? for shortcuts|\d+% context|context:|ctrl\+|shift\+|tab to|plan mode|accept edits|bypass permissions)/i.test(text)) continue
+ return 'team_waiting_draft'
+ }
+ }
+ return null
+}
diff --git a/cli/src/engines/registry.ts b/cli/src/engines/registry.ts
index 6ec9b238e..25bc2ff18 100644
--- a/cli/src/engines/registry.ts
+++ b/cli/src/engines/registry.ts
@@ -1,3 +1,5 @@
+import { screen as claudeScreen } from './claude/screen.js'
+import { screen as codexScreen } from './codex/screen.js'
import type { Engine } from './engine.js'
import { engineLaunches } from './launches.js'
import { engineHooks } from './hooks.js'
@@ -10,8 +12,8 @@ import { runtime as codexRuntime } from './codex/runtimeProfile.js'
/** Only the migrated engines. Others keep their existing handlers until their own small batch. */
const engines = {
- claude: { name: 'claude', launch: engineLaunches.claude, transcript: claudeTranscript, hooks: engineHooks.claude, live: claudeLive, runtime: claudeRuntime },
- codex: { name: 'codex', launch: engineLaunches.codex, transcript: codexTranscript, hooks: engineHooks.codex, live: codexLive, runtime: codexRuntime },
+ claude: { name: 'claude', launch: engineLaunches.claude, transcript: claudeTranscript, hooks: engineHooks.claude, live: claudeLive, runtime: claudeRuntime, screen: claudeScreen },
+ codex: { name: 'codex', launch: engineLaunches.codex, transcript: codexTranscript, hooks: engineHooks.codex, live: codexLive, runtime: codexRuntime, screen: codexScreen },
} satisfies Record<string, Engine>
type MigratedEngine = keyof typeof engines
export function engineFor(name: string | null | undefined): Engine | undefined {
diff --git a/cli/src/engines/screens.ts b/cli/src/engines/screens.ts
new file mode 100644
index 000000000..d4d21f2e8
--- /dev/null
+++ b/cli/src/engines/screens.ts
@@ -0,0 +1,12 @@
+import type { AgentEngine } from './types.js'
+import { screen as claude, inspectRuntimePane as claudePane, paneModal as claudeModal } from './claude/screen.js'
+import { screen as codex, inspectRuntimePane as codexPane, paneModal as codexModal } from './codex/screen.js'
+import { legacyScreen } from '../lib/legacyScreen.js'
+import { paneModal as legacyModal } from './kit/pane.js'
+import { inspectRuntimePane as inspectPane } from '../lib/legacyPane.js'
+export function screenFor(engine: AgentEngine) { return engine === 'claude' ? claude : engine === 'codex' ? codex : { inspect: (capture: string) => legacyScreen(engine, capture) } }
+export function inspectRuntimePane(engine: AgentEngine, capture: string) { return engine === 'claude' ? claudePane(capture) : engine === 'codex' ? codexPane(capture) : inspectPane(engine, capture) }
+export function paneModal(engine: AgentEngine, capture: string) { return engine === 'claude' ? claudeModal(capture) : engine === 'codex' ? codexModal(capture) : legacyModal(engine, capture) }
+export function parseEngineQuestionPane(engine: AgentEngine, capture: string) { return screenFor(engine).inspect(capture).question }
+export function messageHold(engine: AgentEngine, capture: string | null) { return capture === null ? (engine === 'claude' || engine === 'codex' ? 'screen_unreadable' : null) : screenFor(engine).inspect(capture).messageHold }
+export function teamWriteHold(engine: AgentEngine, capture: string | null) { return capture ? screenFor(engine).inspect(capture).teamHold : 'team_waiting_unavailable' }
diff --git a/cli/src/engines/worker/process.ts b/cli/src/engines/worker/process.ts
index c0ba46326..6d734b984 100644
--- a/cli/src/engines/worker/process.ts
+++ b/cli/src/engines/worker/process.ts
@@ -6,6 +6,7 @@ import { runServiceProcess, type ServiceProcess } from '../../services/process.j
import type { ServiceProcessOptions } from '../../serviceProcess.js'
import type { EngineTranscript, HistoryAsk, TranscriptSession } from '../facets/transcript.js'
import { engineLiveRequests } from './liveRequests.js'
+import { engineScreenRequests } from './screenRequests.js'
import { engineRuntimeRequests } from './runtimeRequests.js'
import {
EngineReadError, record, READER_HISTORY, READER_IN_FLIGHT, READER_LAST_TURN, READER_REPLY_BYTES,
@@ -86,6 +87,6 @@ export interface EngineProcessOptions extends ServiceProcessOptions {
export function runEngineReader(engine: ReaderEngine, options: EngineProcessOptions): ServiceProcess {
return (options.run ?? runServiceProcess)({
name: READER_SERVICES[engine], socketPath: options.socketPath, machineId: options.machineId,
- token: options.token, requests: options.requests ?? { ...engineReaderRequests(engine), ...engineLiveRequests(engine), ...engineRuntimeRequests(engine) },
+ token: options.token, requests: options.requests ?? { ...engineReaderRequests(engine), ...engineLiveRequests(engine), ...engineRuntimeRequests(engine), ...engineScreenRequests(engine) },
})
}
diff --git a/cli/src/engines/worker/screenProtocol.ts b/cli/src/engines/worker/screenProtocol.ts
new file mode 100644
index 000000000..2a94776e0
--- /dev/null
+++ b/cli/src/engines/worker/screenProtocol.ts
@@ -0,0 +1,55 @@
+/** Private, stateless screen reads. A worker receives text, never a terminal handle. */
+import type { PaneView, QuestionRow, ScreenReading } from '../facets/screen.js'
+import { record } from './protocol.js'
+
+export const SCREEN_VERSION = 1
+export const SCREEN_READ = 'engine_screen_read'
+export const SCREEN_CAPABILITIES = 'engine_screen_capabilities'
+export const SCREEN_WAIT_MS = 1_000
+export const SCREEN_IN_FLIGHT = 8
+export const SCREEN_QUEUED = 64
+export const SCREEN_CAPTURE_BYTES = 256 * 1024
+export const SCREEN_REPLY_BYTES = 1024 * 1024
+const HOLDS = ['permission_open', 'question_open', 'menu_open', 'rewind_picker_open', 'transcript_open',
+ 'search_open', 'trust_open', 'update_prompt_open', 'model_prompt_open', 'sign_in_open', 'popup_open',
+ 'prompt_hidden', 'screen_unreadable']
+const TEAM_HOLDS = ['team_waiting_unavailable', 'team_waiting_user', 'team_waiting_draft', 'team_waiting_idle']
+const fields = (value: Record<string, unknown>, names: string[]) => Object.keys(value).every(key => names.includes(key))
+const text = (value: unknown, max = SCREEN_CAPTURE_BYTES): value is string => typeof value === 'string' && value.length <= max
+const flag = (value: unknown) => value === undefined || typeof value === 'boolean'
+
+export function screenCapture(value: unknown): value is string {
+ return text(value) && Buffer.byteLength(value) <= SCREEN_CAPTURE_BYTES
+}
+
+function questionRow(value: unknown): value is QuestionRow {
+ return record(value) && fields(value, ['number', 'label', 'checked', 'walk'])
+ && typeof value.number === 'string' && /^[1-9]\d?$/.test(value.number)
+ && text(value.label) && typeof value.checked === 'boolean'
+ && (value.walk === undefined || value.walk === 'right' || value.walk === 'down')
+}
+
+function questionView(value: unknown): value is PaneView {
+ if (value === null) return true
+ if (!record(value)) return false
+ if (value.kind === 'review') return fields(value, ['kind', 'submitRow'])
+ && typeof value.submitRow === 'string' && /^(?:[1-9]\d?|Enter)$/.test(value.submitRow)
+ return value.kind === 'question' && fields(value, ['kind', 'permission', 'partial', 'enterSubmits', 'question', 'rows', 'multi', 'typeRow', 'dialog'])
+ && flag(value.permission) && flag(value.partial) && flag(value.enterSubmits) && text(value.question)
+ && Array.isArray(value.rows) && value.rows.length <= 99 && value.rows.every(questionRow)
+ && typeof value.multi === 'boolean' && (value.typeRow === null || questionRow(value.typeRow))
+ && (value.dialog === undefined || text(value.dialog))
+}
+
+/** Malformed or oversized evidence cannot make a screen writable. */
+export function screenReading(value: unknown): value is ScreenReading {
+ return record(value) && fields(value, ['pane', 'question', 'messageHold', 'teamHold', 'activity', 'busy', 'stoppedGoal']) && record(value.pane)
+ && fields(value.pane, ['idle', 'plan', 'dialog', 'draft'])
+ && ['idle', 'plan', 'dialog', 'draft'].every(key => typeof (value.pane as Record<string, unknown>)[key] === 'boolean')
+ && typeof value.busy === 'boolean' && typeof value.stoppedGoal === 'boolean'
+ && (value.activity === null || (record(value.activity) && fields(value.activity, ['label', 'indicator']) && text(value.activity.label, 100) && text(value.activity.indicator)))
+ && questionView(value.question) && (value.messageHold === null || HOLDS.includes(value.messageHold as string))
+ && (value.teamHold === null || TEAM_HOLDS.includes(value.teamHold as string))
+ && !(value.pane.idle && (value.pane.dialog || value.pane.draft))
+ && !(value.question && value.messageHold === null)
+}
diff --git a/cli/src/engines/worker/screenRequests.spec.ts b/cli/src/engines/worker/screenRequests.spec.ts
new file mode 100644
index 000000000..3e95cac45
--- /dev/null
+++ b/cli/src/engines/worker/screenRequests.spec.ts
@@ -0,0 +1,80 @@
+import { readFileSync } from 'node:fs'
+import { afterEach, describe, expect, it, vi } from 'vitest'
+import { screenFor } from '../screens.js'
+import { engineScreenRequests } from './screenRequests.js'
+import { screenReading, SCREEN_CAPABILITIES, SCREEN_CAPTURE_BYTES, SCREEN_IN_FLIGHT, SCREEN_READ, SCREEN_REPLY_BYTES, SCREEN_WAIT_MS } from './screenProtocol.js'
+
+const who = { owner: true, local: true }
+const capture = '────────────\n❯\n────────────\n? for shortcuts'
+const payload = { version: 1, capture }
+const answer = screenFor('claude').inspect(capture)
+const fail = (error: string) => ({ version: 1, error })
+afterEach(() => { vi.useRealTimers(); vi.restoreAllMocks() })
+describe('read-only engine screen workers', () => {
+ it.each(['claude', 'codex'] as const)('%s preserves recorded permission evidence', async engine => {
+ const text = readFileSync(new URL(`../../lib/__fixtures__/permission-${engine}.txt`, import.meta.url), 'utf8')
+ const requests = engineScreenRequests(engine)
+ expect(await requests[SCREEN_CAPABILITIES]({ version: 1 }, who)).toEqual({ version: 1, screen: 1, engine })
+ const reply = await requests[SCREEN_READ]({ version: 1, capture: text }, who)
+ expect(reply).toEqual({ version: 1, answer: screenFor(engine).inspect(text) })
+ expect((reply as any).answer.messageHold).toBe('permission_open')
+ expect(screenReading((reply as any).answer)).toBe(true)
+ })
+ it('accepts only private local core requests and bounded captures', async () => {
+ const load = vi.fn(async () => screenFor('claude'))
+ const requests = engineScreenRequests('claude', { load })
+ const send = (p = payload, asker = who, closed?: AbortSignal) => requests[SCREEN_READ](p, asker, closed)
+ for (const asker of [{ owner: false, local: true }, { owner: true, local: false }, { ...who, connection: 'client' }]) {
+ expect(await send(payload, asker)).toEqual(fail('ENGINE_INVALID_REQUEST'))
+ }
+ for (const p of [{ version: 2, capture }, { version: 1, capture: 42 }, { ...payload, terminal: 'agent' },
+ { ...payload, capture: 'é'.repeat(SCREEN_CAPTURE_BYTES) }]) {
+ expect(await send(p as typeof payload)).toEqual(fail('ENGINE_INVALID_REQUEST'))
+ }
+ expect(await send(payload, who, AbortSignal.abort())).toEqual(fail('ENGINE_UNAVAILABLE'))
+ expect(load).not.toHaveBeenCalled()
+ expect(await requests[SCREEN_CAPABILITIES]({ version: 1 }, who)).toMatchObject({ screen: 1 })
+ expect(load).not.toHaveBeenCalled()
+ expect(await send({ ...payload, requestId: 'core-route-1' } as typeof payload)).toEqual({ version: 1, answer })
+ expect(await send({ ...payload, requestId: 1 } as unknown as typeof payload)).toEqual(fail('ENGINE_INVALID_REQUEST'))
+ expect(await send({ ...payload, requestId: 'x'.repeat(201) } as typeof payload)).toEqual(fail('ENGINE_INVALID_REQUEST'))
+ expect(await send()).toEqual({ version: 1, answer })
+ expect(load).toHaveBeenCalledOnce()
+ })
+ it('bounds concurrent loads and contains a stopped or failed parser', async () => {
+ vi.useFakeTimers()
+ const recycle = vi.fn()
+ let finish!: (value: ReturnType<typeof screenFor>) => void
+ const requests = engineScreenRequests('claude', { recycle, load: () => new Promise(resolve => { finish = resolve }) })
+ const send = () => requests[SCREEN_READ](payload, who)
+ const pending = Array.from({ length: SCREEN_IN_FLIGHT }, send)
+ expect(await send()).toEqual(fail('ENGINE_BUSY'))
+ await vi.advanceTimersByTimeAsync(SCREEN_WAIT_MS)
+ for (const read of pending) expect(await read).toEqual(fail('ENGINE_UNAVAILABLE'))
+ expect(recycle).toHaveBeenCalled()
+ finish(screenFor('claude'))
+ const load = vi.fn().mockRejectedValueOnce(new Error('import failed')).mockResolvedValue(screenFor('claude'))
+ const retry = engineScreenRequests('claude', { load })
+ expect(await retry[SCREEN_READ](payload, who)).toEqual(fail('ENGINE_UNAVAILABLE'))
+ expect(await retry[SCREEN_READ](payload, who)).toEqual({ version: 1, answer })
+ })
+ it('drops disconnected work and rejects faulty adapter output', async () => {
+ const closed = new AbortController()
+ const requests = engineScreenRequests('claude', { load: async () => { closed.abort(); return screenFor('claude') } })
+ expect(await requests[SCREEN_READ](payload, who, closed.signal)).toEqual(fail('ENGINE_UNAVAILABLE'))
+ for (const [result, error] of [[{}, 'ENGINE_INVALID_REPLY'], [{ ...answer, teamHold: 'invented' }, 'ENGINE_INVALID_REPLY'],
+ [{ ...answer, extra: 'x'.repeat(SCREEN_REPLY_BYTES) }, 'ENGINE_REPLY_TOO_LARGE']] as const) {
+ const worker = engineScreenRequests('claude', { load: async () => ({ inspect: () => result as typeof answer }) })
+ expect(await worker[SCREEN_READ](payload, who)).toEqual(fail(error))
+ }
+ const throwing = engineScreenRequests('claude', { load: async () => ({ inspect: () => { throw new Error('bad decoder') } }) })
+ expect(await throwing[SCREEN_READ](payload, who)).toEqual(fail('ENGINE_UNAVAILABLE'))
+ })
+ it('rejects malformed UI facts before any caller can treat a pane as writable', () => {
+ for (const value of [null, [], {}, { ...answer, pane: { ...answer.pane, draft: true } },
+ { ...answer, question: { kind: 'review', submitRow: 'C-c' } }, { ...answer, activity: { label: 2, indicator: '' } },
+ { ...answer, question: { kind: 'question', rows: [], question: 'approve?', multi: false, typeRow: null } },
+ { ...answer, messageHold: 'unknown' }, { ...answer, execute: 'Enter' }]) expect(screenReading(value)).toBe(false)
+ expect(screenReading({ ...answer, question: { kind: 'review', submitRow: '1' }, messageHold: 'question_open' })).toBe(true)
+ })
+})
diff --git a/cli/src/engines/worker/screenRequests.ts b/cli/src/engines/worker/screenRequests.ts
new file mode 100644
index 000000000..b468b8a83
--- /dev/null
+++ b/cli/src/engines/worker/screenRequests.ts
@@ -0,0 +1,45 @@
+import type { ServiceRequests } from '../../core/api.js'
+import type { EngineScreen } from '../facets/screen.js'
+import type { ReaderEngine } from './protocol.js'
+import { screenCapture, screenReading, SCREEN_CAPABILITIES, SCREEN_IN_FLIGHT, SCREEN_READ,
+ SCREEN_REPLY_BYTES, SCREEN_VERSION, SCREEN_WAIT_MS } from './screenProtocol.js'
+
+const loadScreen = {
+ claude: async () => (await import('../claude/screen.js')).screen,
+ codex: async () => (await import('../codex/screen.js')).screen,
+}
+export interface ScreenRequestDeps { load?: () => Promise<EngineScreen>; recycle?: () => void }
+
+export function engineScreenRequests(engine: ReaderEngine, deps: ScreenRequestDeps = {}): ServiceRequests {
+ let loaded: Promise<EngineScreen> | null = null
+ let pending = 0
+ const failure = (error: string) => ({ version: SCREEN_VERSION, error })
+ const handle = (method: string): ServiceRequests[string] => async (payload, asker, closed) => {
+ if (!asker.owner || !asker.local || asker.connection !== undefined || payload.version !== SCREEN_VERSION) return failure('ENGINE_INVALID_REQUEST')
+ if (closed?.aborted) return failure('ENGINE_UNAVAILABLE')
+ if (method === SCREEN_CAPABILITIES) return { version: SCREEN_VERSION, screen: SCREEN_VERSION, engine }
+ // ServiceLinks adds its routing id inside the payload. It is transport
+ // metadata, not a terminal capability or engine input.
+ if (!screenCapture(payload.capture) || Object.keys(payload).some(key => !['version', 'capture', 'requestId'].includes(key))
+ || (payload.requestId !== undefined && (typeof payload.requestId !== 'string' || payload.requestId.length > 200))) return failure('ENGINE_INVALID_REQUEST')
+ if (pending >= SCREEN_IN_FLIGHT) return failure('ENGINE_BUSY')
+ pending++
+ let timer: ReturnType<typeof setTimeout> | undefined
+ const deadline = new Promise<Record<string, unknown>>(resolve => {
+ timer = setTimeout(() => { resolve(failure('ENGINE_UNAVAILABLE')); (deps.recycle ?? (() => process.exit(1)))() }, SCREEN_WAIT_MS)
+ })
+ try {
+ const capture = payload.capture
+ return await Promise.race([deadline, (async () => {
+ const adapter = await (loaded ??= (deps.load ?? loadScreen[engine])().catch(error => { loaded = null; throw error }))
+ if (closed?.aborted) return failure('ENGINE_UNAVAILABLE')
+ const answer = adapter.inspect(capture)
+ const reply = { version: SCREEN_VERSION, answer }
+ if (Buffer.byteLength(JSON.stringify(reply)) > SCREEN_REPLY_BYTES) return failure('ENGINE_REPLY_TOO_LARGE')
+ return screenReading(answer) ? reply : failure('ENGINE_INVALID_REPLY')
+ })()])
+ } catch { return failure('ENGINE_UNAVAILABLE') }
+ finally { clearTimeout(timer); pending-- }
+ }
+ return { [SCREEN_CAPABILITIES]: handle(SCREEN_CAPABILITIES), [SCREEN_READ]: handle(SCREEN_READ) }
+}
diff --git a/cli/src/harnessd/services.spec.ts b/cli/src/harnessd/services.spec.ts
index fa3ecc721..c91e7612e 100644
--- a/cli/src/harnessd/services.spec.ts
+++ b/cli/src/harnessd/services.spec.ts
@@ -1,6 +1,6 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import {
- DEFAULT_SERVICE_OPTIONS, ENGINE_LIVE_ENV, ENGINE_RUNTIME_ENV, KNOWN_SERVICES, SERVICE_HOSTS, SERVICE_PROCESSES_ENV, ServiceSupervisor, UPDATER_HOST, masterRunsLiveEngines, masterRunsEngineRuntime, serviceOptions, serviceProcessesEnv, serviceSpecs, servicesTheMasterRuns,
+ DEFAULT_SERVICE_OPTIONS, ENGINE_LIVE_ENV, ENGINE_RUNTIME_ENV, ENGINE_SCREEN_ENV, KNOWN_SERVICES, SERVICE_HOSTS, SERVICE_PROCESSES_ENV, ServiceSupervisor, UPDATER_HOST, masterRunsLiveEngines, masterRunsEngineRuntime, masterRunsEngineScreen, serviceOptions, serviceProcessesEnv, serviceSpecs, servicesTheMasterRuns,
type ServiceSpec, type ServiceSupervisorOptions,
} from './services.js'
import type { CoreHandle } from './supervisor.js'
@@ -472,15 +472,15 @@ describe('which services the core leaves to its master', () => {
it('is what the master says it runs, told in a form a core from before the list reads too', () => {
const specs = serviceSpecs({ HARNESSD_SERVICES: 'search,workspaces' }, SERVICE_HOSTS)
- expect(serviceProcessesEnv(specs, 42)).toEqual({ [SERVICE_PROCESSES_ENV]: 'search,workspaces', HARNESSD_SERVICES: 'search,workspaces', [ENGINE_LIVE_ENV]: '42:1', [ENGINE_RUNTIME_ENV]: '42:1' })
+ expect(serviceProcessesEnv(specs, 42)).toEqual({ [SERVICE_PROCESSES_ENV]: 'search,workspaces', HARNESSD_SERVICES: 'search,workspaces', [ENGINE_LIVE_ENV]: '42:1', [ENGINE_RUNTIME_ENV]: '42:1', [ENGINE_SCREEN_ENV]: '42:1' })
// A process named as one of its services is named whole: the viewers' process runs the Store beside them.
expect(serviceProcessesEnv(serviceSpecs({ HARNESSD_SERVICES: 'viewers' }, SERVICE_HOSTS), 42)).toMatchObject({ [SERVICE_PROCESSES_ENV]: 'viewers,store' })
expect(serviceProcessesEnv(serviceSpecs({ HARNESSD_SERVICES: 'store' }, SERVICE_HOSTS), 42)).toMatchObject({ [SERVICE_PROCESSES_ENV]: 'store' })
// By service, not by process: a core from before the edge host routes the services it knows of it.
const hosted = serviceSpecs({ HARNESSD_SERVICES: 'edge' }, SERVICE_HOSTS)
- expect(serviceProcessesEnv(hosted, 42)).toEqual({ [SERVICE_PROCESSES_ENV]: 'workspaces,usage,monitor,projects,handoff,recaps,windowNames,shell', HARNESSD_SERVICES: 'workspaces,usage,monitor,projects,handoff,recaps,windowNames,shell', [ENGINE_LIVE_ENV]: '42:1', [ENGINE_RUNTIME_ENV]: '42:1' })
+ expect(serviceProcessesEnv(hosted, 42)).toEqual({ [SERVICE_PROCESSES_ENV]: 'workspaces,usage,monitor,projects,handoff,recaps,windowNames,shell', HARNESSD_SERVICES: 'workspaces,usage,monitor,projects,handoff,recaps,windowNames,shell', [ENGINE_LIVE_ENV]: '42:1', [ENGINE_RUNTIME_ENV]: '42:1', [ENGINE_SCREEN_ENV]: '42:1' })
expect([...servicesTheMasterRuns({ ...supervised, ...serviceProcessesEnv(hosted, 42) }, known)]).toEqual(['workspaces'])
- expect(serviceProcessesEnv([], 42)).toEqual({ [SERVICE_PROCESSES_ENV]: '', HARNESSD_SERVICES: 'none', [ENGINE_LIVE_ENV]: '42:1', [ENGINE_RUNTIME_ENV]: '42:1' })
+ expect(serviceProcessesEnv([], 42)).toEqual({ [SERVICE_PROCESSES_ENV]: '', HARNESSD_SERVICES: 'none', [ENGINE_LIVE_ENV]: '42:1', [ENGINE_RUNTIME_ENV]: '42:1', [ENGINE_SCREEN_ENV]: '42:1' })
expect([...servicesTheMasterRuns({ ...supervised, ...serviceProcessesEnv(specs, 42) }, known)]).toEqual(['search', 'workspaces'])
expect([...servicesTheMasterRuns({ ...supervised, ...serviceProcessesEnv([], 42) }, known)]).toEqual([])
// What the master says wins over whatever HARNESSD_SERVICES the core inherited; names it does not know are its own.
@@ -517,6 +517,15 @@ describe('which services the core leaves to its master', () => {
expect(masterRunsLiveEngines({ ...env, [ENGINE_RUNTIME_ENV]: undefined }, 42)).toBe(true)
})
+ it('negotiates screen readers only from a capable supervising parent', () => {
+ const env = { ...supervised, ...serviceProcessesEnv([], 42) }
+ expect(masterRunsEngineScreen(env, 42)).toBe(true)
+ expect(masterRunsEngineScreen(env, 43)).toBe(false)
+ expect(masterRunsEngineScreen({ ...env, [ENGINE_SCREEN_ENV]: undefined }, 42)).toBe(false)
+ expect(masterRunsEngineScreen({ ...env, [ENGINE_LIVE_ENV]: undefined }, 42)).toBe(false)
+ expect(masterRunsEngineScreen({ ...env, HARNESSD_SERVICE_TOKEN: undefined }, 42)).toBe(false)
+ })
+
it('is none without a master, or without the token its services would connect with', () => {
expect([...servicesTheMasterRuns({ [SERVICE_PROCESSES_ENV]: 'search' }, known)]).toEqual([])
expect([...servicesTheMasterRuns({ HARNESSD_SUPERVISED: '1', [SERVICE_PROCESSES_ENV]: 'search' }, known)]).toEqual([])
diff --git a/cli/src/harnessd/services.ts b/cli/src/harnessd/services.ts
index aeca007d9..f243e8606 100644
--- a/cli/src/harnessd/services.ts
+++ b/cli/src/harnessd/services.ts
@@ -459,6 +459,7 @@ export const SERVICE_PROCESSES_ENV = 'HARNESSD_SERVICE_PROCESSES'
export const ENGINE_LIVE_ENV = 'HARNESSD_ENGINE_LIVE'
/** Runtime profile methods, negotiated independently from live transcript parsing. */
export const ENGINE_RUNTIME_ENV = 'HARNESSD_ENGINE_RUNTIME'
+export const ENGINE_SCREEN_ENV = 'HARNESSD_ENGINE_SCREEN'
/**
* What a master puts in its core's environment about the services it runs in their own processes: the
@@ -470,7 +471,7 @@ export function serviceProcessesEnv(specs: readonly ServiceSpec[], masterPid: nu
// The services, not the processes: a core knows what it routes by service, and one from before the
// edge host still finds the services it knows here (workspaces) and runs the rest itself.
const names = specs.flatMap((spec) => spec.services).join(',')
- return { [SERVICE_PROCESSES_ENV]: names, HARNESSD_SERVICES: names || 'none', [ENGINE_LIVE_ENV]: `${masterPid}:1`, [ENGINE_RUNTIME_ENV]: `${masterPid}:1` }
+ return { [SERVICE_PROCESSES_ENV]: names, HARNESSD_SERVICES: names || 'none', [ENGINE_LIVE_ENV]: `${masterPid}:1`, [ENGINE_RUNTIME_ENV]: `${masterPid}:1`, [ENGINE_SCREEN_ENV]: `${masterPid}:1` }
}
/** An older master may inherit a newer master's environment after rollback. Trust only this parent. */
@@ -521,3 +522,7 @@ export function serviceSpecs(env: NodeJS.ProcessEnv, hosts: Readonly<Record<stri
return services.length ? [{ name, ...rest, services, ...onDemand, ...(Number.isInteger(heap) && heap > 0 ? { heapLimitMiB: heap } : {}) }] : []
})
}
+
+export function masterRunsEngineScreen(env: NodeJS.ProcessEnv, parentPid: number): boolean {
+ return masterRunsLiveEngines(env, parentPid) && env[ENGINE_SCREEN_ENV] === `${parentPid}:1`
+}
diff --git a/cli/src/lib/askQuestion.ts b/cli/src/lib/askQuestion.ts
index 9dcc77e41..877ed9625 100644
--- a/cli/src/lib/askQuestion.ts
+++ b/cli/src/lib/askQuestion.ts
@@ -1,1433 +1,18 @@
-/**
- * AskUserQuestion on a REMOTE machine — mirror the question to the device, then answer it by driving the
- * interactive CLI's own terminal dialog.
- *
- * A runtime that speaks stream-json gets this for free: a `question_request` event carries
- * the questions out and a `question_response` control-frame carries the answer back. A remote machine has no
- * such channel — the adapter watches a JSONL transcript and talks to a real TUI. So both halves read and
- * write the pane itself:
- *
- * OUT QuestionWatcher polls the pane while a turn is open and pushes the open dialog as
- * `commander_question`, in the SAME shape the hosted runtime sends, so the firmware's existing question
- * screen renders it unchanged (commanderQuestions in websocket.ts). NOT from the transcript —
- * see the QuestionWatcher docblock for why that source cannot work.
- * IN the device's `question_response` → keystrokes into the pane's dialog — only once the dialog on
- * screen is shown to be the one that answer was for (its requestId); a late answer is refused
- * with STALE_QUESTION and types nothing.
- *
- * Dialog mechanics (verified against Claude Code 2.1.220, `tmux capture-pane`):
- * - single-select : the option's digit selects AND submits, advancing to the next question / review.
- * - free text : the digit of the "Type something." row opens it for editing (does NOT submit) →
- * type the text → Enter submits.
- * - multi-select : rows render as `[ ]` / `[✔]`; a digit TOGGLES; Tab advances to the next question.
- * - review : after the last question, "Ready to submit your answers?" → "1. Submit answers".
- */
-
-import type { RegisteredSession } from './registry.js'
-import type { AgentEngine } from '../engines/types.js'
-import { locateMuseQuestion } from '../engines/muse/askQuestion.js'
-import { ampSelectionKeys, parseAmpQuestionPane } from '../engines/amp/askQuestion.js'
-import { kiloSelectionKeys, locateKiloQuestion, parseKiloQuestionPane } from '../engines/kilo/askQuestion.js'
-import { parseCursorPermissionPane } from '../engines/cursor/askQuestion.js'
-import { locateDevinPermission, locateDevinQuestion } from '../engines/devin/askQuestion.js'
-import { parseGrokQuestionPane } from '../engines/grok/askQuestion.js'
-import { locateAgyQuestion } from '../engines/agy/askQuestion.js'
-import { withCopilotSubject } from '../engines/copilot/askQuestion.js'
-import { earlierDialogEnd, PERMISSION_FOOTER_RE, QUESTION_FOOTER_RE } from './dialogEnd.js'
-import { composerState } from './composerScreen.js'
-
-/** Device-facing question shape — byte-for-byte the hosted runtime’s `commanderQuestions()` output. */
-export interface ShapedQuestion {
- key: string
- q: string
- options: string[]
- multi: boolean
- /** Observed free-text editor; never inferred for permission prompts. */
- canText?: boolean
-}
-
-export interface QuestionRow {
- /** The digit to press. */
- number: string
- /** Option text with any `[ ]` checkbox prefix removed. */
- label: string
- checked: boolean
- /**
- * Set only when the dialog does NOT number its rows: the row is then reached by walking to it and
- * pressing Enter, and `number` carries its INDEX rather than a key.
- *
- * The direction is a property of the DIALOG, not of the engine. OpenCode draws BOTH a numbered ask
- * dialog and — for permissions — the horizontal prompt kilo inherited from it, so keying by engine
- * would send digits into a dialog that numbers nothing and select nothing at all.
- */
- walk?: 'right' | 'down'
-}
-
-export interface QuestionView {
- kind: 'question'
- permission?: boolean
- /**
- * The dialog is on screen but its top — the question and its first rows — is scrolled out of the pane
- * (Codex's request_user_input keeps the footer anchored and lets a short pane cut the top off; measured
- * in a four-pane window: rows 3–5 and the footer visible, "1." and the question gone). Enough to know a
- * dialog is STILL OPEN — the watcher must not close it — and not enough to announce as a question.
- */
- partial?: boolean
- /** The footer says a digit only highlights and Enter commits (`enter to submit answer` — Codex's
- * request_user_input). Absent where one digit selects and submits, which is every other dialog. */
- enterSubmits?: boolean
- question: string
- rows: QuestionRow[]
- multi: boolean
- /** The "Type something." row, when the dialog offers free text. */
- typeRow: QuestionRow | null
- /**
- * A permission prompt's WHOLE dialog, every line as painted (ANSI stripped), from its frame to its last
- * row. `question` is one line clipped for a device's screen; a command that wraps — `npm test &&` on one
- * line, `git push` on the next — is only whole here. What the pair brain's floor reads (pair/classify.ts),
- * and part of the dialog's fingerprint, so two prompts that differ below their first line are two ids.
- */
- dialog?: string
-}
-
-export interface ReviewView {
- kind: 'review'
- submitRow: string
-}
-
-export type PaneView = QuestionView | ReviewView | null
-
-const STEP_MS = 350 // let the TUI repaint between keystrokes
-const TEXT_MS = 250
-const MAX_STEPS = 14 // hard bound on the drive loop (questions × keys), never spin on a stuck pane
-const CAPTURE_LINES = 60
-
-// Rows that exist in every dialog but can never BE an answer.
-const CHAT_ROW = /^chat about (this|these)$/i
-// Claude writes "Type something.", Command Code "Type something..." — one trailing dot or three, and
-// missing it costs the free-text row: a voice answer would have nowhere to go and the row would be
-// offered on the device as if it were a real option.
-// Claude "Type something.", Command Code "Type something...", OpenCode "Type your own answer", Hermes
-// "Other (type your answer)". Same row in every dialog: it opens an editor instead of choosing, so it must
-// never be offered to the device as a selectable label.
-const TYPE_ROW = /^(type something\.{0,3}|type your own( answer)?|other \(type your (own|answer)\))$/i
-
-function stripAnsi(value: string): string {
- return value.replace(/\u001b\[[0-9;:]*[A-Za-z]/g, '')
-}
-
-function norm(value: string): string {
- return value.toLowerCase().replace(/\s+/g, ' ').replace(/[.…]+$/, '').trim()
-}
-
-function sleep(ms: number): Promise<void> {
- return new Promise((resolve) => setTimeout(resolve, ms))
-}
-
-/**
- * Shape a raw AskUserQuestion input into the device form. Port of the hosted runtime’s `commanderQuestions`
- * (websocket.ts): `key` MUST stay the value the CLI matches answers by — prompt/question first — because
- * the device echoes it back as the answers-map key.
- */
-export function shapeQuestions(questions: unknown): ShapedQuestion[] {
- return (Array.isArray(questions) ? questions : []).map((raw, i) => {
- const q = (raw ?? {}) as Record<string, unknown>
- const opts = Array.isArray(q.options) ? q.options : []
- return {
- key: (q.prompt as string) || (q.question as string) || (q.id as string) || (q.header as string) || `question_${i}`,
- q: (q.prompt as string) || (q.question as string) || '',
- options: opts
- .map((o) => String((typeof o === 'string' ? o : (o as Record<string, unknown>)?.label) ?? ''))
- .filter(Boolean),
- // Three spellings because three CLIs: `allow_multiple` (the hosted runtime), `multiSelect` (claude),
- // `multi_select` (devin — read from a real `ask_user_question` call in its SQLite store).
- multi: (q.allow_multiple as boolean) ?? (q.multiSelect as boolean) ?? (q.multi_select as boolean) ?? false,
- }
- })
-}
-
-/** A dialog one reader found, and the line it anchored on: what ranks two readers of the same pane. */
-export interface FoundDialog {
- view: QuestionView | ReviewView
- at: number
-}
-
-function permissionView(view: PaneView): PaneView {
- return view?.kind === 'question' ? { ...view, permission: true } : view
-}
-
-function asPermission(found: FoundDialog | null): FoundDialog | null {
- return found && { ...found, view: permissionView(found.view) as QuestionView }
-}
-
-/**
- * The LOWEST dialog any reader found: the live one. A pane keeps an answered dialog in its scrollback, and
- * an engine with two readers (a question and an approval, say) must never let the first reader's hit on
- * that one shadow the second reader's hit on the dialog under it: devin's approval under its answered
- * question, muse's approval under a question, opencode's question under an approval. A tie is one dialog
- * read twice, and the earlier reader, the engine's own, keeps it.
- */
-function lowest(...found: Array<FoundDialog | null>): PaneView {
- let best: FoundDialog | null = null
- for (const f of found) if (f && (!best || f.at > best.at)) best = f
- return best?.view ?? null
-}
-
-/**
- * The dialog on screen, read the way `engine` paints it: the LAST one on the pane, and nothing of it from
- * above an earlier dialog's end (`dialogEnd.ts`).
- *
- * Claude and Command Code share one shape (see parseQuestionPane); devin draws a different one and gets
- * its own parser rather than more branches in here.
- */
-export function parseEngineQuestionPane(engine: AgentEngine, capture: string): PaneView {
- if (engine === 'claude' || engine === 'codex') {
- const found = locateQuestionPane(capture)
- if (!found) return null
- // Captures include scrollback. The long chaos run returned to the composer after answering, but
- // the old dialog still held every prompt as question_open. A full composer BELOW that dialog makes
- // it history. One above it proves nothing: a newer dialog can have only its last rows visible.
- const below = capture.split('\n').slice(found.at + 1).join('\n')
- if (composerState(engine, below) !== 'absent') return null
- return engine === 'codex' ? withCodexLabels(found.view) : found.view
- }
- // Devin's two dialogs are told apart by one word in the footer (`↵ select` vs `↵ confirm`), so they can
- // never both match the same dialog; but an answered one of either kind can sit above the live one.
- if (engine === 'devin') return lowest(locateDevinQuestion(capture), asPermission(locateDevinPermission(capture)))
- // Cursor has no ask-the-user tool, so its permission prompt is the ONLY dialog it ever draws — and it
- // numbers nothing, stating each row's key in the row instead.
- if (engine === 'cursor') return permissionView(parseCursorPermissionPane(capture))
- // Muse pairs each option with a description on the same line and floats a live Preview box above the
- // rows — both confuse the shared parser, so it reads its own. Its PERMISSION prompt is a different
- // dialog entirely (`Would you like to allow this network access?` over `1. Yes, proceed (y)` rows under
- // a `Press enter to confirm` footer, `__fixtures__/permission-muse.txt`) and that one the shared parser
- // reads exactly, so it is read by both and the lower wins.
- if (engine === 'muse') return lowest(locateMuseQuestion(capture), locateQuestionPane(capture))
- // Amp's is a permission prompt with unnumbered rows — nothing the shared parser can anchor on.
- if (engine === 'amp') return permissionView(parseAmpQuestionPane(capture))
- // Kilo's is the same kind of prompt but laid out HORIZONTALLY, sharing its line with the key hints —
- // it is a fork of opencode that did not keep opencode's dialog.
- if (engine === 'kilo') return permissionView(parseKiloQuestionPane(capture))
- // Grok tells its approval from its questionnaire itself, by the live dialog's own footer.
- if (engine === 'grok') return parseGrokQuestionPane(capture)
- // agy's ask-the-user dialog anchors on `Question N/M:` under an `↑/↓ Navigate` footer, which the
- // shared parser cannot see. Its PERMISSION prompt is numbered rows under `Do you want to proceed?`
- // and the shared parser reads that one exactly, so it is read by both and the lower wins.
- if (engine === 'agy') return lowest(locateAgyQuestion(capture), locateQuestionPane(capture))
- // Hermes, OpenCode and Copilot paint the dialog inside a box; peel the border and the shared parser
- // fits. Measured on Copilot: framed it returns null, unframed it reads the question, the three
- // options AND spots `4. Other (type your answer)` as the free-text row rather than an option.
- if (engine === 'hermes') return parseQuestionPane(unframe(capture))
- // Copilot boxes its dialog the same way, but names the SUBJECT of a permission prompt above the
- // question — "attempting to access the following URL:" over a boxed value. Without it the device
- // shows "Do you want to allow this access?" and a bare "Yes", with nothing to judge.
- if (engine === 'copilot') {
- const found = locateQuestionPane(unframe(capture))
- return found ? withCopilotSubject(found.view, capture, found.at) : null
- }
- if (engine === 'opencode') {
- // OpenCode's PERMISSION prompt is the horizontal one kilo inherited from it — same `△ Permission
- // required` title, same `⇆ select · enter confirm` footer, same unnumbered rows. Measured: the live
- // capture in `permission-opencode.txt` parses through kilo's parser unchanged, so it is shared rather
- // than copied. Listed FIRST because that dialog numbers nothing: on its own dialog the shared parser
- // still matches the `enter confirm` footer and walks up into whatever numbered rows are above, and
- // the tie goes to kilo's reader.
- const plain = unframe(capture)
- return lowest(asPermission(locateKiloQuestion(capture)), locateOpencodeReview(plain), locateQuestionPane(plain))
- }
- return parseQuestionPane(capture)
-}
-
-/**
- * Codex's request_user_input rows carry the option's description on the same line as its label, in
- * an aligned column — `1. Red Creates a bold, high-contrast` — and wrap the rest of the description
- * onto plain lines below. The label is the part before the column gap; the dial has an 80-byte option
- * buffer and a 466px face, and "Red" is what the person is choosing. The approval prompt's rows
- * (`Yes, proceed (y)`) have no such gap and pass through unchanged.
- */
-function withCodexLabels(view: PaneView): PaneView {
- if (!view || view.kind !== 'question') return view
- return {
- ...view,
- rows: view.rows.map((r) => ({ ...r, label: r.label.split(/\s{2,}/)[0].trim() || r.label })),
- }
-}
-
-/**
- * Strip the box-drawing frame Hermes draws around its `clarify` dialog.
- *
- * Measured on a live pane: Hermes paints the SAME dialog Claude does — `❯ 1. Xanh` rows, a footer reading
- * `↑/↓ to select, Enter to confirm` — only wrapped in `│ … │`. Peeling the border makes the existing
- * parser fit exactly, which is far better than a second parser that would drift from it over time.
- */
-/**
- * OpenCode's final step of a multi-question dialog.
- *
- * It is a REVIEW, not a question: a "Review" heading over `label: answer` lines, with no numbered rows at
- * all, submitted by pressing Enter. The shared parser needs rows to recognise a dialog, so without this
- * the driver saw "nothing on screen", stopped, and left the agent sitting on an unsubmitted form after
- * every answer had been given (measured on a live pane).
- *
- * `submitRow` carries the KEY to press, which for every other CLI happens to be a digit — 'Enter' rides
- * the same field rather than widening the type for one engine.
- */
-function locateOpencodeReview(plain: string): FoundDialog | null {
- const lines = stripAnsi(plain).split('\n')
- const footer = lines.findLastIndex((l) => /enter\s+submit/i.test(l))
- if (footer < 0) return null
- const floor = earlierDialogEnd(lines, footer, 14)
- let sawReview = false
- for (let i = footer - 1; i > floor && footer - i <= 14; i--) {
- if (parseRow(lines[i])) return null // rows above ⇒ still a question, not the review
- if (/^\s*review\s*$/i.test(lines[i])) { sawReview = true; break }
- }
- return sawReview ? { view: { kind: 'review', submitRow: 'Enter' }, at: footer } : null
-}
-
-function unframe(capture: string): string {
- // stripAnsi FIRST: on a real capture the border is preceded by SGR codes, so matching `^│` against the
- // raw text silently never fired — hermes read as "no dialog open" with the dialog plainly on screen.
- return stripAnsi(capture)
- .split('\n')
- .map((line) => {
- const left = line.replace(/^(\s*)[│┃|]\s?/, '$1')
- // An unanchored whitespace regex retries at every column on blank rows.
- // Inspect the last non-space character once; keep unframed lines intact.
- const right = left.trimEnd()
- return /[│┃|]$/.test(right) ? right.slice(0, -1).trimEnd() : left
- })
- .join('\n')
-}
-
-/** How this engine's multi-select dialog is submitted once the boxes are ticked. */
-function multiSubmitKey(engine: AgentEngine): string {
- // Claude advances with Tab; devin submits with Enter (measured on a live pane — Tab does nothing there).
- return engine === 'devin' ? 'Enter' : 'Tab'
-}
-
-function parseRow(line: string): QuestionRow | null {
- // Consume indentation once: two adjacent whitespace runs backtrack across
- // every possible split on the padded non-option lines in terminal captures.
- const m = /^[❯›>]?\s*(\d+)\.\s+(.+?)\s*$/.exec(line.trimStart())
- if (!m) return null
- const raw = m[2]
- const box = /^\[([^\]])\]\s*(.*)$/.exec(raw)
- return {
- number: m[1],
- label: (box ? box[2] : raw).trim(),
- checked: !!box && box[1].trim() !== '',
- }
-}
-
-/**
- * Read the CURRENT dialog off a pane capture. The capture includes scrollback (old dialogs, old plan
- * text), so everything is anchored to the LAST footer line — the only marker the live dialog always
- * paints at the bottom of the screen.
- */
-/** "❯ 1. Submit" immediately followed by "2. Cancel" — Command Code's review screen, which carries no
- * other marker. Returns the Submit row's number and where it sits, or null. */
-function findSubmitPair(lines: string[]): { row: string; index: number } | null {
- for (let i = lines.length - 1; i >= 1; i--) {
- const row = parseRow(lines[i])
- if (!row || !/^submit$/i.test(row.label)) continue
- const next = parseRow(lines[i + 1] ?? '')
- if (next && /^cancel$/i.test(next.label)) return { row: row.number, index: i }
- }
- return null
-}
-
-/** The tab bar of a footer-less dialog: "● Loại game | ◯ Review". Returns its LAST occurrence, or -1. */
-function findTabBarDialog(lines: string[]): number {
- for (let i = lines.length - 1; i >= 0; i--) {
- const line = lines[i].trim()
- if (!/^[●◯✔✓]/.test(line) || !line.includes('|')) continue // answered tabs turn into ✔
- // Only a tab bar with a numbered list under it is a question; the same glyphs appear in prose.
- for (let j = i + 1; j < lines.length && j - i <= 8; j++) {
- const row = parseRow(lines[j])
- if (row?.number === '1') return i
- }
- }
- return -1
-}
-
-/** Read a dialog that has no footer: question first, then the numbered rows, top-down from the tab bar. */
-function parseDownward(lines: string[], tabBar: number): PaneView {
- let question = ''
- const rows: QuestionRow[] = []
- let checkbox = false
- for (let i = tabBar + 1; i < lines.length; i++) {
- const row = parseRow(lines[i])
- if (row) {
- if (/^\s*[❯›>]?\s*\d+\.\s+\[/.test(lines[i])) checkbox = true
- rows.push(row)
- continue
- }
- // Between the tab bar and row 1 sits the question; after the rows start, plain lines are the option
- // descriptions and must not overwrite it.
- const line = lines[i].trim()
- if (!question && rows.length === 0 && line && !/^[─━-]{6,}$/.test(line)) question = line
- }
- if (!rows.length) return null
- const answerable = rows.filter((r) => !CHAT_ROW.test(r.label) && !TYPE_ROW.test(r.label))
- return {
- kind: 'question',
- question,
- rows: answerable,
- multi: checkbox,
- typeRow: rows.find((r) => TYPE_ROW.test(r.label)) ?? null,
- }
-}
-
-// \u2500\u2500 permission prompts \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500
-
-/**
- * A permission prompt is a QUESTION whose options are the approval choices \u2014 the shape amp and kilo
- * already ship (`engines/amp/askQuestion.ts`, `engines/kilo/askQuestion.ts`). It matters far more on a
- * remote machine than a question does: the CLI attaches to an agent the USER started, under the user's
- * own config and with no permission flag of ours (`engines/README.md`), so a blocking approval is the
- * normal state of a pane, not an edge case. Unparsed, the turn simply sits at `Processing` until someone
- * walks to the computer.
- *
- * Claude and Command Code draw it as a framed block ending in numbered rows, and it carries NONE of the
- * three anchors `parseQuestionPane` knows. Captured live (`__fixtures__/permission-claude.txt`):
- *
- * \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500
- * Bash command \u2190 the header: what kind of approval this is
- *
- * curl -s https://api.coingecko.com/\u2026 \u2190 the argument, then a one-line description
- * Fetch Bitcoin price from CoinGecko API
- *
- * This command requires approval
- *
- * Do you want to proceed?
- * \u276f 1. Yes
- * 2. Yes, and don\u2019t ask again for: curl *
- * 3. No
- *
- * Esc to cancel \u00b7 Tab to amend \u00b7 ctrl+e to explain
- *
- * Command Code differs only in wording (`permission-commandcode.txt`): header `Execute Shell Command`,
- * a sentence for the body, footer `\u2191/\u2193 navigate \u00b7 enter select \u00b7 ctrl+e explain`. Same frame, same rows \u2014
- * which is why one parser covers both instead of two near-copies that would drift apart.
- *
- * **The prose is not the anchor.** Every engine words it differently and Claude alone writes at least
- * three ("Do you want to proceed?", "Do you want to make this edit to README.md?", "Do you want to create
- * \u2026?"). The ROWS are: an approval always offers a way to say yes and a way to say no. That pair is the
- * signal, guarded by the dialog's own key hints so an ordinary numbered list in assistant output can never
- * be read as a prompt.
- */
-
-/** The first row of an approval, measured across claude, commandcode, codex, devin, grok, amp and kilo. */
-const APPROVE_RE = /^(yes|allow|approve|accept|proceed|run|continue)\b/i
-/** \u2026and the row that declines it. Never dropped \u2014 see the rule amp's parser states: a device user given
- * three ways to say yes and none to say no cannot answer the prompt at all. */
-// `skip` is here because cursor's decline row is "Skip & tell the agent what to do instead" — an engine
-// whose only way to refuse says neither "no" nor "reject" would otherwise fail the yes/no guard and its
-// whole prompt would go unread.
-// Claude's plan review does not use a negative verb at all: its only refusal
-// row is "Tell Claude what to change". Keep `tell ... what to change` narrow so
-// an ordinary numbered list beginning with "Tell" cannot become an approval.
-const REJECT_RE = /^(no|reject|deny|decline|cancel|skip|don'?t|stop)\b|^tell\b.*\bwhat to change\b/i
-/** The solid rule that opens the frame. Deliberately NOT the dashed one (`\u254c`) that brackets an edit diff,
- * which sits BELOW the header and would cost the title. */
-const FRAME_RULE_RE = /^\s*[\u2500\u2501\u2550]{6,}\s*$/
-/** Any rule, solid or dashed \u2014 used to skip them while reading the frame's contents. */
-const ANY_RULE_RE = /^\s*[\u2500\u2501\u2550\u254c\u2504\u2508-]{6,}\s*$/
-/** The dialog's own question line, and Command Code's `Press [ctrl+e] \u2026` hint: both sit between the
- * header and the rows, and neither says what is being approved. */
-const PERMISSION_PROSE_RE = /^((do|would) you\b|press \[)/i
-
-/** The opening rule of the frame the rows at `start` sit in, or -1 when they have none of their own. */
-function frameTop(lines: string[], start: number, floor: number): number {
- for (let i = start - 1; i > floor && start - i <= 25; i--) {
- if (FRAME_RULE_RE.test(lines[i])) return i
- }
- return -1
-}
-
-/** Keep a synthesised title inside the device's `text[200]` buffer, with the tail marked as cut. */
-function clipTitle(value: string): string {
- return value.length <= 160 ? value : `${value.slice(0, 159)}\u2026`
-}
-
-/**
- * What is being approved, as one line: `Approve <header>: <argument>`, the form amp's parser already
- * produces so the two read alike on the device.
- *
- * The frame's opening rule is the only reliable top \u2014 the prose under it varies per engine AND per tool.
- * Below the rule sit the header (`Bash command`, `Edit file`, `Execute Shell Command`) and then the
- * argument: the command, the file, or the sentence naming it.
- */
-function permissionTitle(lines: string[], start: number): string {
- // The header must be THIS dialog's: never read past the end of an earlier one still in scrollback.
- const floor = earlierDialogEnd(lines, start, 25)
- const top = frameTop(lines, start, floor)
- // No frame above the rows (codex draws none): the nearest text is the dialog's own question, which
- // names the command outright. Better than inventing a header that is not on screen.
- if (top < 0) {
- for (let i = start - 1; i > floor && start - i <= 4; i--) {
- const line = lines[i].trim()
- if (line) return clipTitle(line)
- }
- return 'Approval required'
- }
- let header = ''
- let arg = ''
- for (let i = top + 1; i < start; i++) {
- const line = lines[i].trim()
- if (!line || ANY_RULE_RE.test(line)) continue
- if (!header) { header = line; continue }
- if (PERMISSION_PROSE_RE.test(line)) continue
- arg = line
- break
- }
- if (!header) return 'Approval required'
- return clipTitle(arg ? `Approve ${header}: ${arg}` : `Approve ${header}`)
-}
-
-/**
- * The permission prompt on screen, with the index of its first row so the caller can rank it against the
- * other anchors.
- *
- * Only the LOWEST numbered block on the pane is considered, and it is rejected outright if it is not an
- * approval. Digging further up would find an answered prompt in the scrollback and re-announce it \u2014 the
- * live dialog is always the bottom-most thing on screen, so there is nothing below it to miss.
- */
-export function parsePermissionPane(lines: string[]): { view: QuestionView; index: number } | null {
- let end = -1
- for (let i = lines.length - 1; i >= 0; i--) {
- if (parseRow(lines[i])) { end = i; break }
- }
- if (end < 0) return null
-
- // Walk up through CONTIGUOUS rows to the one numbered 1. Contiguity is the point: a permission dialog
- // never interleaves prose with its options, while assistant output that happens to be numbered does.
- const rows: QuestionRow[] = []
- let start = -1
- for (let i = end; i >= 0 && end - i < 12; i--) {
- const row = parseRow(lines[i])
- if (!row) break
- rows.unshift(row)
- if (row.number === '1') { start = i; break }
- }
- if (start < 0 || rows.length < 2) return null
- if (!APPROVE_RE.test(rows[0].label) || !rows.some((row) => REJECT_RE.test(row.label))) return null
-
- let hinted = false
- for (let i = end + 1; i < lines.length && i - end <= 4; i++) {
- if (PERMISSION_FOOTER_RE.test(lines[i])) { hinted = true; break }
- }
- if (!hinted) return null
-
- // Single-select, and no free-text row: every option here is a choice to be TAPPED. Verified on live
- // panes for claude, codex, devin and grok \u2014 one digit selects and submits, exactly as `rowKeys` assumes.
- return {
- view: { kind: 'question', permission: true, question: permissionTitle(lines, start), rows, multi: false, typeRow: null, dialog: permissionDialog(lines, start, end) },
- index: start,
- }
-}
-
-/** Every line of the dialog, untruncated: from its opening rule (or, with no frame, up to 12 lines above
- * the rows) to its last row — never reaching back past the end of an earlier dialog. */
-function permissionDialog(lines: string[], start: number, end: number): string {
- const floor = earlierDialogEnd(lines, start, 25)
- const rule = frameTop(lines, start, floor)
- const top = rule >= 0 ? rule + 1 : Math.max(0, start - 12, floor + 1)
- return lines.slice(top, end + 1).map((line) => line.trimEnd()).filter((line) => !ANY_RULE_RE.test(line)).join('\n').trim()
-}
-
-export function parseQuestionPane(capture: string): PaneView {
- return locateQuestionPane(capture)?.view ?? null
-}
-
-/** `parseQuestionPane`, with the line its dialog was anchored on. */
-function locateQuestionPane(capture: string): FoundDialog | null {
- const lines = stripAnsi(capture).replace(/\u00a0/g, ' ').split('\n')
- // Each CLI words its own footer, and OpenCode rewords it PER SCREEN — `enter submit` on a single
- // question, `enter toggle` on a multi-select, `enter confirm` on a step of a multi-question. They all
- // mark the same thing: the bottom of a live dialog.
- // `enter to submit answer` is Codex's request_user_input dialog (plan mode; captured live in
- // __fixtures__/question-codex.txt). Without it that dialog was never a question at all here: the dial
- // showed nothing while the pane waited, and a question it DID show could never be closed, because
- // the watcher had no fingerprint to notice leaving.
- const footer = lines.findLastIndex((l) => QUESTION_FOOTER_RE.test(l))
- // The review screen paints no footer and puts its rows BELOW the prompt, so it needs its own anchor.
- // Whichever anchor is LOWER on screen is the live one (the other is scrollback from an earlier step).
- const review = lines.findLastIndex((l) => /Ready to submit your answers/i.test(l))
- // Command Code's review screen has neither Claude's "Ready to submit your answers" line nor a footer:
- // it is a "Submit"/"Cancel" pair under a summary that is itself numbered. Anchor on that PAIR — the two
- // rows adjacent, in that order — because the summary lines above are numbered too and reading them as
- // options is how the device answered everything and then sat there, never submitting.
- const submit = findSubmitPair(lines)
- // Command Code paints the SAME dialog with no footer at all — the pane simply ends at the last option.
- // Its tab bar is the only thing above the rows that is unmistakably part of the dialog, so it anchors
- // that dialog, read DOWNWARD. Nothing else on either CLI's screen looks like "● X | ◯ Y" followed by a
- // numbered list, which is what keeps ordinary numbered output from being read as a question.
- const tabBar = findTabBarDialog(lines)
- // A permission prompt is one more anchor, and every anchor is ranked the same way: whichever sits
- // lowest on screen is the live dialog, the others are scrollback. That ordering is what keeps the two
- // apart in both directions — codex and hermes draw an approval whose footer the question anchor also
- // matches, and there the footer is BELOW the rows, so the question path (which reads a better title off
- // the same block) still wins. And a footer-less dialog under an answered one is the live one: anchored
- // on the answered one's footer instead, Command Code's question was announced as the OLD question.
- const permission = parsePermissionPane(lines)
- if (permission && permission.index > Math.max(footer, review, submit?.index ?? -1, tabBar)) {
- return { view: permission.view, at: permission.index }
- }
- if (tabBar > Math.max(footer, review, submit?.index ?? -1)) {
- const view = parseDownward(lines, tabBar)
- return view && { view, at: tabBar }
- }
- if (review > footer && review > (submit?.index ?? -1)) {
- for (let i = review + 1; i < lines.length && i - review <= 10; i++) {
- const row = parseRow(lines[i])
- if (row && /^submit answers$/i.test(row.label)) return { view: { kind: 'review', submitRow: row.number }, at: review }
- }
- return null
- }
- if (submit && submit.index > footer) return { view: { kind: 'review', submitRow: submit.row }, at: submit.index }
- if (footer < 0) return null
-
- // Rows belonging to this dialog: the numbered rows just above the footer, back to the row numbered 1 —
- // and never from above an earlier dialog's end, which is where a dialog whose top is scrolled out of
- // the pane would otherwise borrow its first rows.
- const rows: QuestionRow[] = []
- let checkbox = false // `[ ]` / `[✔]` on a row ⇒ this question is multi-select
- let start = -1
- const floor = earlierDialogEnd(lines, footer, 40)
- for (let i = footer - 1; i > floor && footer - i <= 40; i--) {
- const row = parseRow(lines[i])
- if (!row) continue
- rows.unshift(row)
- if (/^\s*[❯›>]?\s*\d+\.\s+\[/.test(lines[i])) checkbox = true
- if (row.number === '1') { start = i; break }
- }
- const enterSubmits = /enter to submit answer/i.test(lines[footer])
- if (rows.length && start < 0 && enterSubmits) {
- return { view: { kind: 'question', partial: true, enterSubmits, question: '', rows, multi: checkbox, typeRow: null }, at: footer }
- }
- if (start < 0 || rows.length === 0) return null
-
- // The question is the nearest real text line above the rows. The dialog paints it between its header
- // and its rows — `[tab bar | header chip] · blank · question · blank · rows` — so those, and a rule,
- // are the TOP of this frame. Stop there, never skip past: mid-repaint the question line can be blank
- // for one capture, and walking on would pick up the PREVIOUS question still sitting in scrollback and
- // pair a stale title with the live options. An empty result just means "look again next tick". The
- // end of an earlier dialog is a top too: its footer is not this dialog's question.
- let question = ''
- const top = earlierDialogEnd(lines, start, 12)
- for (let i = start - 1; i > top && start - i <= 12; i--) {
- const line = lines[i].trim()
- if (!line) continue
- if (/[←→]/.test(line) || /^[☐☒✔✓]/.test(line) || /^[─━-]{6,}$/.test(line)) break
- // Hermes's batch panel marks the active question with `▸`; that marker is chrome, not the question.
- question = line.replace(/^▸\s*/, '')
- break
- }
-
- const answerable = rows.filter((r) => !CHAT_ROW.test(r.label) && !TYPE_ROW.test(r.label))
- return {
- view: {
- kind: 'question',
- ...(enterSubmits ? { enterSubmits } : {}),
- question,
- rows: answerable,
- multi: checkbox,
- typeRow: rows.find((r) => TYPE_ROW.test(r.label)) ?? null,
- dialog: dialogAbove(lines, start, footer),
- },
- at: footer,
- }
-}
-
-/**
- * A footer-anchored dialog, whole: up to 12 lines above its rows — stopping at a rule, a tab bar, the
- * agent's own output (a `•`/`⏺` bullet) or the end of an earlier dialog — down to its last row. Codex puts
- * the command it asks about here (`$ …`, wrapped over as many lines as it takes), with its reason and
- * environment.
- */
-function dialogAbove(lines: string[], start: number, footer: number): string {
- let top = start
- const floor = earlierDialogEnd(lines, start, 12)
- for (let i = start - 1; i > floor && start - i <= 12; i--) {
- const line = lines[i].trim()
- if (/^[•⏺●]/.test(line) || /[←→]/.test(line) || /^[─━═-]{6,}$/.test(line)) break
- top = i
- }
- return lines.slice(top, footer).map((line) => line.trimEnd()).join('\n').trim()
-}
-
-/**
- * A dialog that asks to RUN or CHANGE something (an approval), whichever parser read it: a framed
- * permission prompt, or a footer dialog whose first row approves, whose rows include a rejection, and
- * which names a command (`$ …`) or asks "would you like to run / make …". What the pair's floor treats as
- * a permission prompt (pair/classify.ts); nothing else reads it.
- */
-export function isApprovalDialog(view: QuestionView): boolean {
- if (view.permission) return true
- if (!view.rows.length || !APPROVE_RE.test(view.rows[0].label) || !view.rows.some((row) => REJECT_RE.test(row.label))) return false
- const text = view.dialog ?? view.question
- return /(^|\n)\s*\$ /.test(text) || /would you like to (run|make|apply|execute|edit|write)/i.test(text)
-}
-
-/** Match an answer to a row. The device stores option labels in an 80-byte buffer, so a long label comes
- * back truncated — prefix matches count, in both directions. */
-/**
- * The keystrokes that commit one row.
- *
- * Every engine but Amp numbers its options, so the digit both selects and submits in one press. Amp
- * draws an unnumbered list navigated with the arrow keys, so its rows carry an index and are reached by
- * walking down to them.
- */
-function rowKeys(engine: AgentEngine, row: QuestionRow, view?: QuestionView): string[] {
- if (engine === 'amp') return ampSelectionKeys(row)
- if (engine === 'codex') return codexRowKeys(row, view)
- // Kilo's rows sit side by side, so its walk is horizontal — see engines/kilo/askQuestion.ts.
- if (engine === 'kilo') return kiloSelectionKeys(row)
- // Same dialog, different engine: opencode numbers its ask dialog but not its permission prompt, so the
- // ROW says how it is reached and a per-engine rule would break one of the two.
- if (row.walk === 'right') return kiloSelectionKeys(row)
- if (row.walk === 'down') return ampSelectionKeys(row)
- return [row.number]
-}
-
-/**
- * Codex, measured 2026-09-15 on 0.149: in its request_user_input dialog a digit only MOVES the highlight
- * and Enter submits (`enter to submit answer`) — a digit alone left the dialog up and the device's answer
- * reported as stuck. Its approval prompt is the other way round and was verified earlier: one digit
- * selects and commits, so that one keeps the single key.
- */
-export function codexRowKeys(row: QuestionRow, view?: QuestionView): string[] {
- return view?.enterSubmits ? [row.number, 'Enter'] : [row.number]
-}
-
-export function matchRow(rows: QuestionRow[], answer: string): QuestionRow | null {
- const a = norm(answer)
- if (!a) return null
- return rows.find((r) => norm(r.label) === a)
- ?? (a.length >= 3 ? rows.find((r) => norm(r.label).startsWith(a)) ?? rows.find((r) => a.startsWith(norm(r.label)) && norm(r.label).length >= 3) : undefined)
- ?? null
-}
-
-/**
- * Pick the answer for the question the dialog is currently showing: the entry keyed by its own text.
- *
- * `positional` also takes the next unused entry when none names it. Only for a dialog the answer's
- * requestId proves it was written for: without that proof, an answer that names no question on screen
- * belongs to one that is gone, and typing it here would answer — or approve — something nobody saw.
- *
- * The text must be the question's OWN (case, spacing and a trailing `…` aside), never a prefix either way.
- * An approval is titled `Approve <header>: <argument>` and the header is shared by every prompt of its
- * kind: a key left from an earlier prompt — `Approve Bash command` (its argument unread) — was a prefix of
- * `Approve Bash command: rm -rf ~/projects` and pressed Yes on it, and an old full title named a
- * header-only prompt the other way round. A client echoes back the key it was announced, so the whole
- * text is always there to match; one it has cut is answered through its requestId (`positional`).
- */
-export function pickAnswer(
- answers: Record<string, string>,
- question: string,
- used: Set<string>,
- opts: { positional?: boolean } = {},
-): { key: string; value: string } | null {
- const entries = Object.entries(answers)
- const q = norm(question)
- const byText = q ? entries.find(([k]) => norm(k) === q) : undefined
- if (byText && !used.has(byText[0])) return { key: byText[0], value: byText[1] }
- if (!opts.positional) return null
- const next = entries.find(([k]) => !used.has(k))
- return next ? { key: next[0], value: next[1] } : null
-}
-
-/**
- * The id a dialog is announced under — the SAME function the watcher names it with, so the answer's
- * requestId can be checked against the dialog on screen at the moment of typing rather than against
- * whatever the watcher last saw (it polls every 1.5s, and forgets on a reset).
- */
-export function questionRequestId(sessionId: string, view: QuestionView): string {
- return `q_${hash(sessionId + fingerprintOf(view))}`
-}
-
-/** Why an answer was not keyed. Sent back to the client as `question_response_result.error`. */
-export type QuestionAnswerError = 'STALE_QUESTION' | 'AGENT_NOT_FOUND' | 'ANSWER_BUSY' | 'ANSWER_FAILED'
-
-export type QuestionAnswerResult = { ok: true } | { ok: false; error: QuestionAnswerError; detail: string }
-
-const STALE_CHANGED: QuestionAnswerResult = { ok: false, error: 'STALE_QUESTION', detail: 'That question changed before your answer arrived.' }
-const STALE_GONE: QuestionAnswerResult = { ok: false, error: 'STALE_QUESTION', detail: 'That question is no longer open.' }
-const failed = (detail: string): QuestionAnswerResult => ({ ok: false, error: 'ANSWER_FAILED', detail })
-const KEYS_FAILED = failed('The answer could not be typed into the agent\'s terminal.')
-const STUCK = failed('The question did not take the answer.')
-
-export interface AskQuestionDeps {
- getSession: (sessionId: string) => RegisteredSession | undefined
- capture: (terminalTarget: string, historyLines?: number) => Promise<string | null>
- sendText: (terminalTarget: string, text: string) => Promise<boolean>
- sendKey: (terminalTarget: string, key: string) => Promise<boolean>
- /** Pins one backend locator for the whole multi-step dialog drive. */
- acquireControl?: (sessionId: string, opts?: { forAnswer?: boolean }) => (() => void) | null
- /** Injected for tests. */
- wait?: (ms: number) => Promise<void>
-}
-
-export interface QuestionAnswerPayload {
- /** Exact contents reviewed on the device; guarded submissions never use positional fallback. */
- expectedQuestions?: ShapedQuestion[]
- selectedLabels?: Record<string, string[]>
- freeTextKeys?: string[]
- allowPermissions?: boolean
- requestId?: string
- sessionId?: string
- agentId?: string
- answers?: Record<string, string>
-}
-
-/**
- * Owns the OUT side's pending map (requestId → session) and the IN side's pane driving. One answer at a
- * time per agent: a second `question_response` for a dialog already being driven is dropped, not queued.
- */
-export class AskQuestionController {
- private pending = new Map<string, string>() // requestId → sessionId
- private driving = new Set<string>() // sessionIds currently keying a dialog
-
- constructor(private readonly deps: AskQuestionDeps) {}
-
- /** Remember which session a mirrored question belongs to (the device may answer minutes later). */
- remember(requestId: string, sessionId: string): void {
- if (!requestId) return
- if (this.pending.size > 64) this.pending.delete(this.pending.keys().next().value as string)
- this.pending.set(requestId, sessionId)
- }
-
- async answer(payload: QuestionAnswerPayload): Promise<QuestionAnswerResult> {
- const requestId = payload.requestId ?? ''
- if (payload.expectedQuestions !== undefined && (!Array.isArray(payload.expectedQuestions) ||
- payload.expectedQuestions.length < 1 || payload.expectedQuestions.length > 4 ||
- payload.expectedQuestions.some(q => !q || typeof q.key !== 'string' || !q.key ||
- typeof q.q !== 'string' || !q.q || !Array.isArray(q.options) || !q.options.length ||
- q.options.some(option => typeof option !== 'string' || !option) ||
- typeof q.multi !== 'boolean' || (q.multi && !Array.isArray(payload.selectedLabels?.[q.key]))))) {
- return failed('The reviewed question metadata is invalid.')
- }
- if (payload.freeTextKeys !== undefined && (!payload.expectedQuestions || !Array.isArray(payload.freeTextKeys) ||
- payload.freeTextKeys.some(key => typeof key !== 'string' || !payload.expectedQuestions!.some(q =>
- q.key === key && q.canText === true && !q.multi)))) {
- return failed('The reviewed text answer metadata is invalid.')
- }
- const remembered = this.pending.get(requestId)
- const sessionId = payload.sessionId || payload.agentId || remembered || ''
- const answers = payload.answers && typeof payload.answers === 'object' ? payload.answers : null
- if (!sessionId || !answers || Object.keys(answers).length === 0) {
- console.warn(`[question] ignoring answer with no session/answers (req=${requestId})`)
- return failed('The answer named no agent or carried no choice.')
- }
- const session = this.deps.getSession(sessionId)
- const terminalTarget = session?.agentId || session?.sessionId
- if (!terminalTarget) {
- console.warn(`[question] no terminal target for ${sessionId.slice(0, 8)} — answer dropped`)
- return { ok: false, error: 'AGENT_NOT_FOUND', detail: 'That harness is no longer running.' }
- }
- if (remembered) {
- const owner = this.deps.getSession(remembered)
- if ((owner?.agentId || owner?.sessionId) !== terminalTarget) return STALE_CHANGED
- }
- if (this.driving.has(terminalTarget)) {
- console.warn(`[question] ${sessionId.slice(0, 8)} answer dropped · already driving this dialog`)
- return { ok: false, error: 'ANSWER_BUSY', detail: 'Another answer is already being entered for this harness.' }
- }
- // `forAnswer`: a dialog is the engine waiting for input mid-turn, so the open turn must not block it.
- const release = this.deps.acquireControl?.(terminalTarget, { forAnswer: true })
- // Silence here is the failure mode this whole file exists to prevent: the device sends an answer,
- // nothing keys it in, and the pane sits on the dialog looking like a hung agent.
- if (this.deps.acquireControl && !release) {
- console.warn(`[question] ${sessionId.slice(0, 8)} answer dropped · terminal control unavailable`)
- return { ok: false, error: 'ANSWER_BUSY', detail: 'The agent\'s terminal is busy. Try again.' }
- }
- // The ids the watcher could have announced this dialog under: the session it was remembered for, and
- // the session as the registry knows it now.
- const owners = [...new Set([remembered, session?.sessionId].filter((id): id is string => !!id))]
- this.driving.add(terminalTarget)
- try {
- const result = await this.drive(terminalTarget, answers, this.deps.getSession(sessionId)?.engine ?? 'claude', payload.allowPermissions !== false, { requestId, owners }, payload.expectedQuestions ? payload : undefined)
- this.pending.delete(requestId)
- const outcome = result.ok ? 'submitted' : result.error === 'STALE_QUESTION' ? 'refused · STALE_QUESTION, nothing typed' : 'FAILED'
- console.log(`[question] ${sessionId.slice(0, 8)} answered from device · ${outcome} (req=${requestId || 'none'})`)
- return result
- } finally {
- this.driving.delete(terminalTarget)
- release?.()
- }
+export { isApprovalDialog, parsePermissionPane, parseQuestionPane } from '../engines/kit/questionPane.js'
+/** Explicit inline compatibility for direct users and recorded-screen fixtures. */
+export * from './questionController.js'
+export { parseEngineQuestionPane } from '../engines/screens.js'
+import { parseEngineQuestionPane } from '../engines/screens.js'
+import { AskQuestionController as Controller, QuestionWatcher as Watcher } from './questionController.js'
+import type { AskQuestionDeps, QuestionWatcherDeps, PaneView } from './questionController.js'
+const readQuestion: AskQuestionDeps['readQuestion'] = (session, capture) => parseEngineQuestionPane(session.engine ?? 'claude', capture)
+export class AskQuestionController extends Controller {
+ constructor(deps: Omit<AskQuestionDeps, 'readQuestion'> & Partial<Pick<AskQuestionDeps, 'readQuestion'>>) { super({ readQuestion, ...deps }) }
+}
+export class QuestionWatcher extends Watcher {
+ constructor(private readonly inlineDeps: Omit<QuestionWatcherDeps, 'readQuestion'> & Partial<Pick<QuestionWatcherDeps, 'readQuestion'>>) { super({ readQuestion, ...inlineDeps }) }
+ override notePrompt(sessionId: string, capture: string | null, view?: PaneView): void {
+ const session = this.inlineDeps.getSession(sessionId)
+ super.notePrompt(sessionId, capture, view === undefined && capture !== null && session ? parseEngineQuestionPane(session.engine ?? 'claude', capture) : view ?? null)
}
-
- /**
- * Key the answers into the pane's dialog, question by question, ending on the review screen.
- *
- * Nothing is typed until the dialog on screen is shown to be the question the answer was written for:
- * its requestId when the answer carries one, else its own text. An answer can arrive late — the agent
- * moved on, another client answered, the next question of the form is up — and positionally matching
- * it to whatever is showing now is how a person's "Yes" lands on a permission prompt they never saw.
- */
- private async drive(
- terminalTarget: string,
- answers: Record<string, string>,
- engine: AgentEngine,
- allowPermissions: boolean,
- asked: { requestId: string; owners: string[] },
- reviewed?: QuestionAnswerPayload,
- ): Promise<QuestionAnswerResult> {
- const wait = this.deps.wait ?? sleep
- const used = new Set<string>()
- const reviewedComplete = () => !reviewed || used.size === reviewed.expectedQuestions!.length
- let lastQuestion = ''
- let repeats = 0
- let blanks = 0
- let answered = 0
-
- for (let step = 0; step < MAX_STEPS; step++) {
- const capture = await this.deps.capture(terminalTarget, CAPTURE_LINES)
- if (reviewed && capture === null) return failed('The question could not be read.')
- const view = parseEngineQuestionPane(engine, capture ?? '')
- if (!view) {
- // Nothing on screen: either the dialog was already gone, or the last keystroke submitted it.
- return answered > 0 && reviewedComplete() ? { ok: true } : STALE_GONE
- }
- if (!allowPermissions && view.kind === 'question' && view.permission) return failed('Permission prompts cannot be answered from here.')
- if (view.kind === 'question' && view.partial) {
- // The dialog's top is out of the pane. If we have keyed an answer it has not been taken yet;
- // give the TUI a beat. If we have not, there is nothing to match an answer against.
- if (answered === 0) { console.warn('[question] dialog scrolled out of view — cannot key an answer'); return failed('The question is scrolled out of view.') }
- if (++repeats >= 2) { console.warn('[question] dialog stuck (scrolled)'); return STUCK }
- await wait(STEP_MS)
- continue
- }
- if (view.kind === 'review') {
- // Reached by our own keys, this submits the form. Reached first, it means every question was
- // answered somewhere else — submitting would send answers this person never gave.
- if (answered === 0) return STALE_GONE
- if (!reviewedComplete()) return STALE_CHANGED
- return await this.deps.sendKey(terminalTarget, view.submitRow) ? { ok: true } : failed('The answers could not be submitted.')
- }
- // Mid-repaint the question line can read blank for a capture (see parseQuestionPane). Neither its id
- // nor its text can be checked against a blank, so look again rather than judge the dialog by it.
- if (!view.question) {
- if (++blanks > 2) return answered > 0 && reviewedComplete() ? { ok: true } : failed('The question could not be read.')
- await wait(STEP_MS)
- continue
- }
- blanks = 0
- // The same question still showing after we acted on it: give the TUI one more beat to repaint,
- // then treat it as stuck rather than hammering the pane with more keystrokes. Never consume a
- // second answer for it.
- if (view.question === lastQuestion) {
- if (++repeats >= 2) { console.warn(`[question] dialog stuck on "${view.question.slice(0, 60)}"`); return STUCK }
- await wait(STEP_MS)
- continue
- }
- repeats = 0
- lastQuestion = view.question
-
- // Is this the question the answer was for? Only the FIRST one needs the id: every later screen is
- // one our own keys advanced to, and must be named by its text (below).
- let positional = false
- if (answered === 0 && asked.requestId) {
- if (!asked.owners.some((owner) => questionRequestId(owner, view) === asked.requestId)) {
- console.warn(`[question] answer for req=${asked.requestId} arrived after the dialog changed to "${view.question.slice(0, 60)}" — nothing typed`)
- return STALE_CHANGED
- }
- positional = true
- }
-
- // Out of answers with the dialog still up = a multi-QUESTION dialog whose next question the device
- // hasn't been shown yet. Leave it open: the watcher pushes that one and the device answers it next.
- const expected = reviewed?.expectedQuestions?.find(q => q.q === view.question)
- if (reviewed && (!expected || expected.multi !== view.multi ||
- JSON.stringify(expected.options) !== JSON.stringify(view.rows.map(row => row.label)))) {
- // A request id proves the initial dialog; every reviewed screen must also match its full choices.
- return answered > 0 && reviewedComplete() ? { ok: true } : STALE_CHANGED
- }
- const picked = expected
- ? (!used.has(expected.key) && typeof answers[expected.key] === 'string'
- ? { key: expected.key, value: answers[expected.key] } : null)
- : pickAnswer(answers, view.question, used, { positional })
- if (!picked) {
- if (answered > 0 && reviewedComplete()) return { ok: true }
- console.warn(`[question] no answer names "${view.question.slice(0, 60)}" — nothing typed`)
- return STALE_CHANGED
- }
- used.add(picked.key)
- answered++
-
- if (reviewed?.freeTextKeys?.includes(picked.key)) {
- // Spoken words are explicitly text, even when they happen to equal an option label.
- // A permission prompt can never acquire consent through this path.
- if (!expected?.canText || view.permission || view.multi || !view.typeRow ||
- !picked.value.trim() || Buffer.byteLength(picked.value, 'utf8') > 1200 ||
- /[\x00-\x09\x0b-\x1f\x7f]/.test(picked.value)) return failed('The text answer cannot be entered into this question.')
- if (!await this.typeFreeText(terminalTarget, view.typeRow, picked.value, wait)) return KEYS_FAILED
- await wait(STEP_MS)
- continue
- }
-
- if (view.multi) {
- // Device joins the selected labels with ", " (q_done_tap).
- const labels = reviewed ? reviewed.selectedLabels?.[picked.key] ?? []
- : picked.value.split(',').map((s) => s.trim()).filter(Boolean)
- if (reviewed && (!labels.length || labels.some(label => !view.rows.some(row => row.label === label)))) return failed('That answer matches no option.')
- if (reviewed) {
- // Set the exact reviewed set, including clearing choices selected in another client.
- for (const row of view.rows) if (row.checked !== labels.includes(row.label)) {
- if (!await this.deps.sendKey(terminalTarget, row.number)) return KEYS_FAILED
- await wait(TEXT_MS)
- }
- if (!await this.deps.sendKey(terminalTarget, multiSubmitKey(engine))) return KEYS_FAILED
- await wait(STEP_MS)
- continue
- }
- let toggled = 0
- for (const label of labels) {
- const row = matchRow(view.rows, label)
- if (!row || row.checked) continue
- if (!await this.deps.sendKey(terminalTarget, row.number)) return KEYS_FAILED
- await wait(TEXT_MS)
- toggled++
- }
- if (!toggled && view.typeRow
- && !await this.typeFreeText(terminalTarget, view.typeRow, picked.value, wait)) return KEYS_FAILED
- if (!await this.deps.sendKey(terminalTarget, multiSubmitKey(engine))) return KEYS_FAILED // advance to the next question / review
- await wait(STEP_MS)
- continue
- }
-
- const row = reviewed ? view.rows.find(row => row.label === picked.value) ?? null : matchRow(view.rows, picked.value)
- if (row) {
- // One digit selects AND submits — except on Amp, whose rows are unnumbered and reached by
- // walking the list, so this is a short sequence rather than a single key.
- for (const key of rowKeys(engine, row, view)) {
- if (!await this.deps.sendKey(terminalTarget, key)) return KEYS_FAILED
- await wait(TEXT_MS)
- }
- await wait(STEP_MS)
- continue
- }
- if (reviewed) return failed('That answer matches no option.')
- if (!view.typeRow) { console.warn(`[question] no option matched "${picked.value.slice(0, 40)}" and no free-text row`); return failed('That answer matches no option.') }
- if (!await this.typeFreeText(terminalTarget, view.typeRow, picked.value, wait)) return KEYS_FAILED
- await wait(STEP_MS)
- }
- return STUCK
- }
-
- /** True while a dialog is being keyed — the watcher pauses so a half-driven dialog isn't re-announced. */
- isDriving(sessionId: string): boolean {
- const session = this.deps.getSession(sessionId)
- return this.driving.has(session?.agentId || session?.sessionId || sessionId)
- }
-
- /** Free-text answer (a voice answer is always free text): open the "Type something." row, type, Enter. */
- private async typeFreeText(terminalTarget: string, typeRow: QuestionRow, text: string, wait: (ms: number) => Promise<void>): Promise<boolean> {
- if (!await this.deps.sendKey(terminalTarget, typeRow.number)) return false
- await wait(TEXT_MS)
- if (!await this.deps.sendText(terminalTarget, text)) return false
- await wait(TEXT_MS)
- return this.deps.sendKey(terminalTarget, 'Enter')
- }
-}
-
-// ── watching a terminal for an open question ─────────────────────────────────────────────────────
-
-export interface QuestionWatcherDeps {
- getSession: (sessionId: string) => RegisteredSession | undefined
- capture: (terminalTarget: string, historyLines?: number) => Promise<string | null>
- /** Skip the capture entirely when no device is listening — nothing would consume the question. */
- hasDevice: () => boolean
- /** A dialog is open on screen. Fires ONCE per distinct question (until it changes or closes). */
- onQuestion: (sessionId: string, requestId: string, questions: ShapedQuestion[], detail?: { permission: boolean; dialog: string }) => void
- /**
- * An announced dialog LEFT the screen — answered somewhere else, or abandoned.
- *
- * This is the other half of `onQuestion`, and it was missing: a question answered in the app (or in the
- * pane by hand) simply stopped being on screen, the watcher forgot it, and every OTHER client went on
- * showing it. On the dial that is a screen you cannot leave without dismissing a question that no longer
- * exists. The pane is the only source of truth here — there is no server-side question object — so
- * "gone from the pane" is the only signal there is.
- */
- onQuestionGone?: (sessionId: string, requestId: string) => void
-
- /** True while that session's dialog is being keyed by an answer already in flight. */
- isDriving?: (sessionId: string) => boolean
-}
-
-const POLL_MS = 1500
-/** How long the pane read before a prompt stands for the turn it starts: past this, that turn's own start
- * is read instead (a prompt that never became a turn). Turns were seen to start 8 s after their prompt
- * under load. */
-const PROMPT_BASELINE_MS = 120_000
-// Consecutive empty polls before a question is declared gone.
-//
-// NOT 1. A capture taken mid-repaint parses as no-dialog, and announcing a close on that would yank a
-// live question off the dial's screen — the exact failure this feature exists to prevent, inverted.
-// Two ticks costs 1.5s of delay on a real close and makes a flicker unable to cause one.
-const GONE_TICKS = 2
-
-// Amp and codex are here for their PERMISSION prompt, not a question tool — neither has one. That prompt
-// is drawn only in the pane and recorded nowhere, so polling the pane is the only way it is ever seen.
-// Codex needs no parser of its own: it draws numbered rows under a `Press enter to confirm or esc to
-// cancel` footer, which is the shared parser's anchor exactly (`__fixtures__/permission-codex.txt`), and
-// the question it lands on is the command itself. Membership in this set is what starts the poll, so an
-// engine belongs here only once something can actually read its pane.
-const QUESTION_ENGINES = new Set<AgentEngine>(['claude', 'commandcode', 'codex', 'cursor', 'devin', 'hermes', 'opencode', 'muse', 'amp', 'kilo', 'grok', 'agy', 'copilot'])
-
-/** Does this engine ever paint a question dialog? Callers use it to decide whether to watch its pane. */
-export function pollsQuestions(engine: AgentEngine): boolean {
- return QUESTION_ENGINES.has(engine)
-}
-
-/**
- * The OUT half. The obvious source — the transcript's AskUserQuestion `tool_use` line — is useless here:
- * the CLI does not flush that line until the question has been ANSWERED (its JSONL `timestamp` is the
- * message's creation time, not its write time), so a device would only ever learn about a question after
- * it no longer exists. The dialog itself, on the other hand, is on screen the whole time it is waiting.
- *
- * So while a turn is open we read the pane. This also means the question survives an adapter restart and
- * re-announces to a device that attaches mid-question — neither of which a one-shot event could do.
- */
-export class QuestionWatcher {
- private readonly watching = new Set<string>()
- private timer: NodeJS.Timeout | undefined
- private readonly pendingPolls = new Map<string, { cancelled: boolean }>()
- private readonly pendingBaselines = new Map<string, { cancelled: boolean }>()
- private last = new Map<string, string>() // sessionId → fingerprint of the announced question
- private lastId = new Map<string, string>() // sessionId → requestId of the announced question
- private misses = new Map<string, number>() // sessionId → consecutive polls with no dialog
- private readonly blocked = new Map<string, string>()
- /** sessionId → the dialog that was ALREADY on the pane when this turn began. */
- private readonly preTurn = new Map<string, string>()
- /** sessionId → the pane as the daemon read it right before typing a prompt: its dialog, if any, and when. */
- private readonly beforePrompt = new Map<string, { fingerprint: string | null; at: number }>()
-
-
- constructor(private readonly deps: QuestionWatcherDeps) {}
-
- /**
- * A turn just began — remember any dialog already on the pane, so it is not
- * announced as this turn's question.
- *
- * A dialog belonging to a turn cannot have been on screen before that turn's
- * first byte. That invariant is the only way to tell "waiting for an answer"
- * from "answered a moment ago and still drawn", because a pane looks
- * identical either way.
- *
- * MEASURED: the daemon attached to a Codex that had just been asked about an
- * update and answered in the app. Its first capture, 0.8s into the NEXT turn,
- * still held that prompt — so the dial was shown a question nobody was being
- * asked, and it stood there for 25s until the engine's output scrolled it out
- * of the captured window.
- *
- * Deliberately NOT applied on attach: re-announcing a genuinely open dialog to
- * a client that arrives mid-question is a feature, and there is no turn
- * boundary there to reason from.
- */
- noteTurnStart(sessionId: string): void {
- this.preTurn.delete(sessionId)
- this.cancelPending(sessionId)
- // The daemon typed this turn's prompt, and read the pane as it did: what was on it then is the turns
- // before's, and nothing drawn since is (notePrompt says why).
- const typed = this.beforePrompt.get(sessionId)
- this.beforePrompt.delete(sessionId)
- if (typed && Date.now() - typed.at <= PROMPT_BASELINE_MS) {
- if (typed.fingerprint) this.preTurn.set(sessionId, typed.fingerprint)
- return
- }
- const pending = { cancelled: false }
- this.pendingBaselines.set(sessionId, pending)
- void (async () => {
- try {
- const session = this.deps.getSession(sessionId)
- const target = session?.agentId || session?.sessionId
- if (!target || session?.active === false || (session?.engine && !pollsQuestions(session.engine))) return
- const generation = questionCaptureGeneration(session)
- const capture = await this.deps.capture(target, CAPTURE_LINES)
- if (pending.cancelled || generation !== questionCaptureGeneration(this.deps.getSession(sessionId))) return
- const view = parseEngineQuestionPane(session?.engine ?? 'claude', capture ?? '')
- if (!view || view.kind !== 'question' || !view.question || view.rows.length === 0) return
- // Only if nothing has been announced for this turn yet: the capture takes
- // a moment, and a dialog that opened inside that window is this turn's.
- if (this.lastId.has(sessionId)) return
- this.preTurn.set(sessionId, fingerprintOf(view))
- } catch {
- // A failed read supplies no baseline. Keep watching for a fresh dialog.
- } finally {
- if (this.pendingBaselines.get(sessionId) === pending) this.pendingBaselines.delete(sessionId)
- }
- })()
- }
-
- /**
- * The daemon is typing a prompt, and `capture` is the pane as it read it right before (core/input.ts
- * messageWriter): the honest "before this turn" for the turn that prompt starts.
- *
- * A capture taken when the turn is SEEN to start can be too late for that. The turn is seen to start
- * from the transcript, after the prompt is confirmed typed, and an engine can draw its question before
- * that: measured in the soak run (e2e/endurance.e2e.ts) under load, Codex took the prompt at 20:42:54.9
- * (its prompt hook), drew its question a moment later, and the turn was seen to start at 20:43:02.2, 7 s
- * on. The capture at that start held the turn's own question, recorded it as the turns before's, and it
- * was never announced: the agent waited on a question no window or device was shown.
- *
- * A read that failed supplies nothing: the turn's start reads the pane itself, as before.
- */
- notePrompt(sessionId: string, capture: string | null): void {
- const session = this.deps.getSession(sessionId)
- if (capture === null || !session || (session.engine && !pollsQuestions(session.engine))) return
- const view = parseEngineQuestionPane(session.engine ?? 'claude', capture)
- const dialog = view && view.kind === 'question' && view.question && view.rows.length > 0 ? fingerprintOf(view) : null
- this.beforePrompt.set(sessionId, { fingerprint: dialog, at: Date.now() })
- }
-
- /** Poll this session's pane while its turn is open (called on turn_started). */
- start(sessionId: string): void {
- if (this.watching.has(sessionId)) return
- const session = this.deps.getSession(sessionId)
- // Only the engines that actually paint a question dialog: Claude and Command Code share one shape,
- // devin has its own (parseDevinQuestionPane). Polling any other pane would be pure waste.
- if (!session || session.active === false || !(session.agentId || session.sessionId) || !QUESTION_ENGINES.has(session.engine)) return
- this.watching.add(sessionId)
- // One clock also starts the reads in one event-loop turn, allowing the tmux
- // backend to batch them without delaying polls or caching terminal content.
- this.timer ??= setInterval(() => {
- for (const id of this.watching) void this.tick(id)
- }, POLL_MS)
- }
-
- /**
- * Stop polling — and CLOSE any question still outstanding on this session.
- *
- * ⚠️ THIS IS THE COMMON CASE, NOT THE EDGE ONE, and leaving it out made the whole close mechanism look
- * like it did not work. `stop()` is called on turn_ended, and answering the question is precisely what
- * lets the turn end — so the dialog leaving the pane and the watcher being torn down happen within a
- * second or two of each other, far inside the two-tick confirmation. Measured on hardware: the answer
- * landed at ~16:50:19 and the turn ended at 16:50:29 with no close ever announced.
- *
- * No confirmation is needed here and none is wanted: the turn is over, so whatever we announced is
- * definitively not waiting for anybody any more. The same holds for the other callers — an agent that
- * was removed cannot answer either.
- */
- stop(sessionId: string): void {
- this.cancelPending(sessionId)
- this.preTurn.delete(sessionId)
- this.blocked.delete(sessionId)
- this.watching.delete(sessionId)
- if (!this.watching.size && this.timer) { clearInterval(this.timer); this.timer = undefined }
- const requestId = this.lastId.get(sessionId)
- this.forget(sessionId)
- if (requestId) this.deps.onQuestionGone?.(sessionId, requestId)
- }
-
-
- stopAll(): void {
- if (this.timer) clearInterval(this.timer)
- this.timer = undefined
- this.watching.clear()
- for (const pending of this.pendingPolls.values()) pending.cancelled = true
- for (const pending of this.pendingBaselines.values()) pending.cancelled = true
- this.preTurn.clear()
- this.beforePrompt.clear()
- this.blocked.clear()
- this.last.clear()
- this.lastId.clear()
- this.misses.clear()
- }
-
- /** A device (re)joined: forget what we announced so an open question is pushed again. */
- reset(): void {
- this.last.clear()
- this.lastId.clear()
- this.misses.clear()
- }
-
- private forget(sessionId: string): void {
- this.last.delete(sessionId)
- this.lastId.delete(sessionId)
- this.misses.delete(sessionId)
- }
-
- private cancelPending(sessionId: string): void {
- const poll = this.pendingPolls.get(sessionId)
- if (poll) poll.cancelled = true
- const baseline = this.pendingBaselines.get(sessionId)
- if (baseline) baseline.cancelled = true
- }
-
- /**
- * A poll found no dialog. Announce the close once the miss is CONFIRMED, and only if we announced the
- * question in the first place — a session nobody was told about has nothing to take back.
- */
- private noteGone(sessionId: string): void {
- this.last.delete(sessionId) // unchanged: a dialog that comes back announces fresh
- const requestId = this.lastId.get(sessionId)
- if (!requestId) return
- const misses = (this.misses.get(sessionId) ?? 0) + 1
- this.misses.set(sessionId, misses)
- if (misses < GONE_TICKS) return
- this.lastId.delete(sessionId)
- this.misses.delete(sessionId)
- this.deps.onQuestionGone?.(sessionId, requestId)
- }
-
-
- private async tick(sessionId: string): Promise<void> {
- // A capture can take longer than POLL_MS. Keep one outstanding poll per
- // session, including a cancelled capture that has not returned yet.
- if (this.pendingPolls.has(sessionId)) return
- const session = this.deps.getSession(sessionId)
- const terminalTarget = session?.agentId || session?.sessionId
- if (!terminalTarget || session?.active === false || (session?.engine && !pollsQuestions(session.engine))) { this.stop(sessionId); return }
- // Both of these silently do nothing, which is how a live dialog can sit on the terminal with no trace
- // in the log. Say it once per transition rather than every 1.5s tick.
- const blocked = !this.deps.hasDevice() ? 'no device' : this.deps.isDriving?.(sessionId) ? 'driving an answer' : ''
- if (blocked !== (this.blocked.get(sessionId) ?? '')) {
- this.blocked.set(sessionId, blocked)
- console.log(`[question] ${sessionId.slice(0, 8)} watcher ${blocked ? `paused · ${blocked}` : 'polling'}`)
- }
- if (blocked) return
- const pending = { cancelled: false }
- this.pendingPolls.set(sessionId, pending)
- const generation = questionCaptureGeneration(session)
- let capture: string | null
- try {
- capture = await this.deps.capture(terminalTarget, CAPTURE_LINES)
- } catch {
- capture = null
- } finally {
- if (this.pendingPolls.get(sessionId) === pending) this.pendingPolls.delete(sessionId)
- }
- if (pending.cancelled || generation !== questionCaptureGeneration(this.deps.getSession(sessionId))
- || !this.deps.hasDevice() || this.deps.isDriving?.(sessionId)) return
- if (capture === null) {
- // Unavailable is not an empty pane, and breaks a run of confirmed misses.
- this.misses.delete(sessionId)
- return
- }
- const view = parseEngineQuestionPane(session?.engine ?? 'claude', capture)
- if (view?.kind === 'question' && view.partial) {
- // Scrolled so its top is out of the pane: still open, so the client showing it keeps showing it —
- // but there is no question text to announce, and the rows in view are whichever the scroll left.
- this.misses.delete(sessionId)
- return
- }
- if (!view || view.kind !== 'question' || !view.question || view.rows.length === 0) {
- // Dialog closed, or moved to review. Either way it is no longer waiting on anybody, so the clients
- // showing it are told to stop — see noteGone for why this is not announced on the first miss.
- this.noteGone(sessionId)
- return
- }
- this.misses.delete(sessionId) // a dialog on screen ends any run of misses
- const fingerprint = fingerprintOf(view)
- // Already on the pane before this turn started → it belongs to whatever came
- // before, and has been answered. Say nothing until it changes or leaves.
- if (this.preTurn.get(sessionId) === fingerprint) return
- this.preTurn.delete(sessionId)
- if (this.last.get(sessionId) === fingerprint) return
- this.last.set(sessionId, fingerprint)
-
- // A pane-derived question has no tool_use id. The key only has to round-trip through the device and
- // back (the answer is keyed into the pane, not matched to a tool call), so the question's own text
- // serves as both — and the device dedups a repeated push by this id. The answer brings the id back,
- // and AskQuestionController recomputes it off the live pane before typing: a different id there
- // means a different dialog, and the answer is refused (STALE_QUESTION) instead of keyed into it.
- const requestId = questionRequestId(sessionId, view)
- this.lastId.set(sessionId, requestId)
- const baseline = this.pendingBaselines.get(sessionId)
- if (baseline) baseline.cancelled = true
- this.deps.onQuestion(sessionId, requestId, [{
- key: view.question,
- q: view.question,
- options: view.rows.map((r) => r.label),
- multi: view.multi,
- ...(view.typeRow && !view.multi && !view.permission ? { canText: true } : {}),
- }], { permission: isApprovalDialog(view), dialog: view.dialog ?? view.question })
-
- }
-}
-
-/** Snapshot scalar values before awaiting I/O: registry records can change in
- * place when an engine exits, moves, or is replaced in the same pane. */
-function questionCaptureGeneration(session: RegisteredSession | undefined): string {
- return session ? JSON.stringify([session.agentId, session.sessionId, session.engine, session.active,
- session.tmuxPane, session.primaryRuntimeKey, session.runtimes, session.processIdentity]) : ''
-}
-
-/**
- * What makes two captures the SAME dialog: its words, its options, its arity — and, when the parser kept
- * the whole dialog, what it says below its first line (a command that differs only on its second line is
- * another prompt).
- *
- * ⚠️ NOT the raw `dialog`. The id is recomputed every 1.5s poll and again at the moment an answer is typed,
- * so it may only change when the QUESTION does. The raw dialog changes on its own: Hermes and Muse paint a
- * live timer inside it (`(01m30s · ↓ 82 tok)`, `(21s · esc to interrupt)`), and every engine moves its
- * `❯`/`›`/`>` cursor and ticks its `[✔]` boxes in place. Hashed raw, a question was re-announced as new on
- * every poll — the needs-you alert, the sound, the dial push, again and again — and every answer from a
- * dial, a device or the cable was refused as STALE_QUESTION. `dialogSignature` is the dialog with all of
- * that taken out; the raw text still goes, unchanged, to the pair's floor (isApprovalDialog, pair/sensor).
- */
-function fingerprintOf(view: QuestionView): string {
- const base = `${view.question}|${view.rows.map((r) => r.label).join('|')}|${view.multi}`
- return view.dialog === undefined ? base : `${base}|${dialogSignature(view.dialog)}`
-}
-
-// A live status group: an elapsed time (`21s`, `30.5s`, `01m30s`, `1h02m`), a token counter (`↓ 82 tok`,
-// `1.2k tokens`) or `esc to interrupt` inside one pair of parentheses. Units hug their digits, as every
-// engine paints them, so `(see 2 files)` or `(tokens.json)` is never mistaken for one.
-const TIMER_GROUP = String.raw`\([^()\n]*?(?:\b\d+(?:\.\d+)?(?:ms|s|m|h)\b|\b\d+m\d+s\b|\b\d+h\d+m\b|\d+(?:\.\d+)?k?\s*tok(?:en)?s?\b|esc to interrupt)[^()\n]*\)`
-const TIMER_GROUP_RE = new RegExp(TIMER_GROUP, 'i')
-const TIMER_GROUPS_RE = new RegExp(TIMER_GROUP, 'gi')
-// The same, outside parentheses: `↓ 82 tokens · esc to interrupt`.
-const STATUS_BITS_RE = /[↑↓]\s*\d+(?:\.\d+)?k?\s*tok(?:en)?s?\b|\besc to interrupt\b/i
-// A bare elapsed time, with no parentheses and no ` · ` to mark it: a status line that is only a word or
-// three and a duration — a verb in -ing/-ed (`waiting 3s`, `thinking 4s`, `Churned for 4s`) or anything
-// trailing off in an ellipsis (`Waiting… 12s`, `Fetch Bitcoin price… 1m33s`). A command's own number stays:
-// `sleep 30s` and `retry after 30s` are neither.
-const DURATION = String.raw`(?:\d+h\d+m(?:\d+s)?|\d+m\d+s|\d+(?:\.\d+)?m?s)`
-const WORDS = String.raw`(?:\p{L}[\p{L}'’-]*\s+){0,2}\p{L}[\p{L}'’-]*`
-const BARE_TIMER_LINE_RE = new RegExp(String.raw`^${WORDS}(?:(?<=ing|ed)(?:\s+for)?\s+|\s*(?:…|\.{3})\s*)${DURATION}$`, 'iu')
-// …and one hung off the end of a longer line, after an ellipsis or a column gap: grok's right-aligned
-// `Waiting on answers for Which color should I report? 4.2s`. Only the time goes.
-const TRAILING_TIMER_RE = new RegExp(String.raw`(?:(?<=…|\.{3})\s*|\s{2,})${DURATION}$`, 'u')
-// Codex's cursor readout under its request_user_input rows: `option 2/4 | tab to add notes`.
-const CURSOR_READOUT_RE = /^option\s+\d+\s*\/\s*\d+\b/i
-// Whatever leads a line and moves on its own: a cursor (`❯ › > ▶`), a spinner frame (braille, `✻`, `◐`,
-// Muse's `◇`/`◆`) or a box/tab state (`☐ ☒ ✔ ○ ● ◉`).
-const LEAD_MARKS_RE = /^(?:[❯›>▶►▸➤\u2800-\u28ff✻✽✶✳✢✺✹✸✷◐◓◑◒◴◵◶◷◇◆☐☑☒✔✓✗✘○◯●◉◎]\s*)+/u
-// A row's own state right after its number, or at the start of an unnumbered row: `[ ]`, `[✔]`, `(•)`, `◉`.
-const ROW_STATE_RE = /^(\d+[.)]\s+|)(?:\[[^\]\n]?\]|\([^)\n]?\)|[☐☑☒✔✓○◯●◉◎])\s*/u
-// Box drawing: frames and rules redraw to the pane's width.
-const BOX_RE = /[\u2500-\u257f]+/g
-
-/**
- * The dialog as a person reads it, with nothing that changes while it waits: status lines (a live timer,
- * bare or in parentheses, a token counter, `esc to interrupt`) and Codex's cursor readout dropped; cursor marks, spinner frames
- * and checkbox/radio state stripped; frames and whitespace collapsed. Every word of the prompt stays —
- * two commands that differ anywhere are still two signatures.
- */
-function dialogSignature(dialog: string): string {
- const out: string[] = []
- for (const raw of dialog.replace(/\u00a0/g, ' ').split('\n')) {
- let line = raw.replace(BOX_RE, ' ').trim()
- if (CURSOR_READOUT_RE.test(line)) continue
- line = line.replace(LEAD_MARKS_RE, '')
- const row = /^\d+[.)]\s/.test(line)
- // A line that carries a live timer is the engine's status line (Hermes' `💻 curl … (01m30s · ↓ 82 tok)`
- // under its frame, Muse's `◇ Calling tools (21s · esc to interrupt)` above its rule, a bare `waiting 3s`),
- // not the prompt: the prompt is always painted on lines of its own. A ROW keeps its words; only the
- // group goes.
- if (!row && (TIMER_GROUP_RE.test(line) || STATUS_BITS_RE.test(line) || BARE_TIMER_LINE_RE.test(line))) continue
- if (!row) line = line.replace(TRAILING_TIMER_RE, '')
- line = line.replace(TIMER_GROUPS_RE, ' ').replace(ROW_STATE_RE, '$1').replace(/\s+/g, ' ').trim()
- if (line) out.push(line)
- }
- return out.join('\n')
-}
-
-function hash(value: string): string {
- let h = 0x811c9dc5
- for (let i = 0; i < value.length; i++) h = Math.imul(h ^ value.charCodeAt(i), 0x01000193) >>> 0
- return h.toString(16).padStart(8, '0')
}
diff --git a/cli/src/lib/closeAgentService.spec.ts b/cli/src/lib/closeAgentService.spec.ts
index 73323db94..5439a5527 100644
--- a/cli/src/lib/closeAgentService.spec.ts
+++ b/cli/src/lib/closeAgentService.spec.ts
@@ -1,5 +1,6 @@
+import { screenFor } from '../engines/screens.js'
import { afterEach, beforeEach, expect, it, vi } from 'vitest'
-import { CloseAgentService, inspectCloseActivity, type CloseActivity, type CloseAgentServiceDeps, type AgentCloseRequest } from './closeAgentService.js'
+import { CloseAgentService, inspectCloseActivity as inspectCloseReading, type CloseActivity, type CloseAgentServiceDeps, type AgentCloseRequest } from './closeAgentService.js'
import { registry, type RegisteredSession } from './registry.js'
import { mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs'
import { join } from 'node:path'
@@ -373,3 +374,7 @@ it('a concurrent deferred tick never starts a second stop', async () => {
expect(deps.stop).toHaveBeenCalledOnce()
finish(); await closing
})
+
+function inspectCloseActivity(session: Parameters<typeof inspectCloseReading>[0], capture: string | null, turnOpen: boolean | undefined, needsInput: boolean) {
+ return inspectCloseReading(session, capture === null ? null : screenFor(session.engine).inspect(capture), turnOpen, needsInput)
+}
diff --git a/cli/src/lib/closeAgentService.ts b/cli/src/lib/closeAgentService.ts
index d899f6043..3a7e70282 100644
--- a/cli/src/lib/closeAgentService.ts
+++ b/cli/src/lib/closeAgentService.ts
@@ -1,8 +1,6 @@
/** User Close is a disk-backed lifecycle operation. Hiding/switching a tab never calls this. */
-import { stripVTControlCharacters } from 'node:util'
import { randomUUID } from 'node:crypto'
-import { teamWriteHold } from './teamWriteHold.js'
-import { terminalActivity } from './terminalActivity.js'
+import type { ScreenReading } from '../engines/facets/screen.js'
import type { RegisteredSession, registry as liveRegistry } from './registry.js'
import { terminalRouteKey } from './terminalRuntime.js'
import type { StopAgentOptions } from './stopAgentService.js'
@@ -30,21 +28,14 @@ function unusedChat(session: CloseSession): boolean {
/** An empty new chat has no turn state yet. Existing chats still require that state,
* and neither kind is idle without a recognized empty composer. */
-export function inspectCloseActivity(session: CloseSession, screen: string | null,
+export function inspectCloseActivity(session: CloseSession, screen: ScreenReading | null,
turnOpen: boolean | undefined, needsInput: boolean): CloseActivity {
const { engine } = session
if (needsInput) return 'needs_input'
if (turnOpen === true) return 'working'
if (!screen) return 'unknown'
- const footer = stripVTControlCharacters(screen).split('\n').slice(-16).join('\n')
- // Codex can be between turns of an active goal, and Claude can have background tasks. Codex has
- // said an active goal two ways: `◎ /goal active (41m)` in its older footers, and `Pursuing goal (41m)`
- // on its status line since (0.160, tui/src/bottom_pane/footer.rs), where only the older wording was
- // known and an agent between the turns of its goal read as idle, for a close to take. Its other goal
- // states (paused, stalled, unmet, abandoned, achieved) are not work in progress.
- if (/\bgoal\s+active\b|\bpursuing goal\b|\b[1-9]\d*\s+background\s+(?:tasks?|agents?)\b/i.test(footer)
- || terminalActivity(engine, screen)) return 'working'
- const hold = teamWriteHold(engine, screen)
+ if (screen.busy) return 'working'
+ const hold = screen.teamHold
if (hold === 'team_waiting_draft') return 'draft'
if (hold === 'team_waiting_user') return 'needs_input'
if (hold || (turnOpen === undefined && !unusedChat(session))) return 'unknown'
diff --git a/cli/src/lib/codexTurnRecovery.ts b/cli/src/lib/codexTurnRecovery.ts
index 89d208fae..01aeb94fa 100644
--- a/cli/src/lib/codexTurnRecovery.ts
+++ b/cli/src/lib/codexTurnRecovery.ts
@@ -1,20 +1,3 @@
-/** Some Codex goal interruptions never write task_complete/turn_aborted to the
- * rollout. An open transcript is then only historical evidence of work. The
- * live footer is one positive idle reading, never a synthetic turn end. */
-import { stripVTControlCharacters } from 'node:util'
-import { inspectRuntimePane } from './runtimeProfileController.js'
-
-/** Read only the UI below the current empty composer, never output/history.
- * An empty composer alone says nothing: Codex accepts drafts while working. */
-export function codexStoppedGoal(screen: string | null): boolean {
- if (!screen || !inspectRuntimePane('codex', screen).idle) return false
- const lines = stripVTControlCharacters(screen).split(/\r?\n/)
- const prompt = lines.findLastIndex(line => /^\s*›(?:\s|$)/u.test(line))
- if (prompt < 0) return false
- // A live turn can still display the previous goal state during a redraw.
- if (lines.slice(-16).some(line => /\besc to interrupt\b/i.test(line))) return false
- const footer = lines.slice(prompt + 1).filter(line => line.trim())
- return footer.length <= 4
- && footer.some(line => /\bGoal (?:stalled|paused) \(\/goal resume\)\s*$/.test(line))
- && footer.some(line => /\? for shortcuts\b/.test(line))
-}
+import { inspectRuntimePane } from '../engines/codex/screen.js'
+import { stoppedGoal } from '../engines/codex/stoppedGoal.js'
+export function codexStoppedGoal(screen: string | null): boolean { return !!screen && stoppedGoal(screen, inspectRuntimePane(screen)) }
diff --git a/cli/src/lib/composerScreen.ts b/cli/src/lib/composerScreen.ts
index d86903728..59c02f550 100644
--- a/cli/src/lib/composerScreen.ts
+++ b/cli/src/lib/composerScreen.ts
@@ -1,120 +1,7 @@
-/**
- * Whether Claude Code's or Codex's own composer is on screen, ready for a message: what a message is
- * typed into, and nothing else (messageHold.ts). An allowlist, so a screen never seen before is not
- * typed into: it is refused, where a list of screens to refuse would have let it through.
- *
- * Read from the engines' own code (Claude Code 2.1.289's bundle, Codex 0.160's tui/src), not run:
- * - Claude Code draws its prompt in a box ruled above and below (`borderStyle: "round"` with no
- * sides, the rule full width, text in the top rule at times), `❯` and the prompt in it, its further
- * lines under the first; under the box, its footer, or the suggestions for a `/command` or an
- * `@mention` being typed, which take the footer's place (`Kge`, padded to six rows or more).
- * Every dialog, menu and view hides the box.
- * - Codex draws a blank row, then `›` in bold (`»` at its top reasoning effort; dim while input is
- * disabled, `!` in shell mode) and the draft or its dim placeholder at the left edge, its further
- * lines under it, then a blank row and its footer, one or two rows, at the bottom of the pane
- * (chat_composer.rs, snapshots `empty`, `draft_composer`, `status_and_queued_messages`). Its popups
- * are drawn above the composer: the `/command` list (snapshot `slash_popup_footer_wide`) and the
- * mention menu with its `enter/tab insert · esc close` row (`default_unified_mention_popup`). Its
- * approvals, questions, pickers and startup screens replace it; a picker's rows are numbered.
- * A popup open over the composer is no composer to type into: its Enter picks the highlighted row.
- */
-import { stripVTControlCharacters } from 'node:util'
-import type { RegisteredSession } from './registry.js'
-
-export type ComposerState = 'ready' | 'popup' | 'absent'
-
-/** The engines read this way. Every other engine is read only for the screens known to refuse. */
+/** Inline compatibility facade. Supervised core reads the engine screen worker. */
+import type { AgentEngine } from '../engines/types.js'
+import { composerState as claude } from '../engines/claude/composer.js'
+import { composerState as codex } from '../engines/codex/composer.js'
+export type { ComposerState } from '../engines/facets/screen.js'
export const COMPOSER_ENGINES: ReadonlySet<string> = new Set(['claude', 'codex'])
-
-/** A row of Claude Code's rule: all `─`, or `─` at both ends with text set in it. */
-const CLAUDE_RULE = /^─{3,}$|^─{3,}.*─{3,}$/
-
-/** Codex's composer row: `›` (or `»`) at the left edge, never a numbered row of a picker. */
-const CODEX_COMPOSER = /^[›»](?:\s|$)/
-const CODEX_PICKER_ROW = /^[›»]\s*\d+\.\s/
-
-function screenLines(capture: string): { lines: string[]; raw: string[] } {
- const raw = capture.split('\n')
- const lines = raw.map((line) => stripVTControlCharacters(line).replace(/ /g, ' ').trimEnd())
- while (lines.length && !lines.at(-1)) { lines.pop(); raw.pop() }
- return { lines, raw }
-}
-
-/** Whether the first character of a row is drawn bold: the SGR in force when it is written. */
-function startsBold(row: string): boolean {
- let bold = false
- const sgr = /^\u001b\[([0-9;:]*)m/
- let rest = row
- for (let match = sgr.exec(rest); match; match = sgr.exec(rest)) {
- for (const code of match[1].split(/[;:]/)) {
- if (code === '1') bold = true
- else if (code === '' || code === '0' || code === '22') bold = false
- }
- rest = rest.slice(match[0].length)
- }
- return bold
-}
-
-/** The word being typed at the end of a draft, where a `/command` or an `@mention` opens its popup. */
-function typedToken(draft: string): string {
- return draft.split(/\s/).at(-1) ?? ''
-}
-
-export function composerState(engine: RegisteredSession['engine'], capture: string): ComposerState {
- const { lines, raw } = screenLines(capture)
- return engine === 'codex' ? codexComposer(lines, raw) : claudeComposer(lines)
-}
-
-function claudeComposer(lines: string[]): ComposerState {
- let prompt = lines.findLastIndex((line) => /^❯(?:\s|$)/.test(line))
- let bottom = lines.findIndex((line, index) => index > prompt && CLAUDE_RULE.test(line.trim()))
- if (prompt < 1 || !CLAUDE_RULE.test(lines[prompt - 1].trim()) || bottom < 0) {
- // A draft taller than the pane: Claude Code does not cap the prompt's height outside its fullscreen
- // renderer (2.1.289 passes `maxVisibleLines` only there), so the box's top rule and its `❯` row
- // scroll off, and the pane shows the draft's further rows, indented under the `❯`, down to the
- // bottom rule. Every row above that rule is one of them; a dialog's are indented by one column, the
- // conversation's start at the edge.
- bottom = lines.findIndex((line) => CLAUDE_RULE.test(line.trim()))
- if (bottom < 1 || !lines.slice(0, bottom).every((line) => !line || line.startsWith(' '))) return 'absent'
- prompt = -1
- }
- const below = lines.slice(bottom + 1)
- // Nothing ruled under the box: a dialog of Claude Code's own is drawn ruled, in its place.
- if (below.some((line) => CLAUDE_RULE.test(line.trim()))) return 'absent'
- const draft = [prompt < 0 ? '' : lines[prompt].replace(/^❯\s?/, ''), ...lines.slice(prompt + 1, bottom).map((line) => line.trim())].join(' ').trim()
- const token = typedToken(draft)
- const rows = below.map((line) => line.trim()).filter(Boolean)
- if (token.startsWith('/') && rows.some((row) => /^\/[\w:.-]/.test(row))) return 'popup'
- if (token.startsWith('@')) {
- const typed = token.slice(1).toLowerCase()
- // A file or agent suggestion, by what is typed after the `@`; with nothing typed yet, any row but the
- // footer's own (its mode and its hints).
- if (rows.some((row) => typed ? row.replace(/^@/, '').toLowerCase().startsWith(typed) : !/^(?:\?|⏵|esc\b)/.test(row))) return 'popup'
- }
- return 'ready'
-}
-
-function codexComposer(lines: string[], raw: string[]): ComposerState {
- const composer = lines.findLastIndex((line) => CODEX_COMPOSER.test(line) && !CODEX_PICKER_ROW.test(line))
- // Bold, on a row of its own under a blank one: not a line of the conversation, nor a disabled composer.
- if (composer < 0 || !startsBold(raw[composer]) || (composer > 0 && lines[composer - 1].trim())) return 'absent'
- // The draft's further lines, then a blank row and the footer: nothing more is drawn under Codex's
- // composer, so more is some other screen under a line of the conversation.
- let end = composer + 1
- while (end < lines.length && lines[end].trim()) end++
- const footer = lines.slice(end).filter((line) => line.trim())
- if (footer.length > 3) return 'absent'
- const draft = [lines[composer].replace(/^[›»]\s?/, ''), ...lines.slice(composer + 1, end).map((line) => line.trim())].join(' ').trim()
- const token = typedToken(draft)
- // A voice send was refused as popup_open even though the recording had transcribed. Looking through
- // twenty conversation rows also matched command examples and old menu hints. A popup is the block
- // immediately above the composer; a slash menu has a selected row and edits the initial /command.
- let popupEnd = composer - 1
- while (popupEnd >= 0 && !lines[popupEnd].trim()) popupEnd--
- let popupStart = popupEnd
- while (popupStart >= Math.max(0, composer - 20) && lines[popupStart].trim()) popupStart--
- const above = lines.slice(popupStart + 1, popupEnd + 1)
- if (draft.startsWith('/') && token.startsWith('/') && above.some((line) => /^›\s+\/[\w-]+(?:\s{2,}\S|$)/.test(line))) return 'popup'
- if (token.startsWith('@') && above.some((line) => /\benter\/tab insert · esc close\b/.test(line))) return 'popup'
- return 'ready'
-}
+export function composerState(engine: AgentEngine, capture: string) { return engine === 'codex' ? codex(capture) : claude(capture) }
diff --git a/cli/src/lib/legacyPane.ts b/cli/src/lib/legacyPane.ts
new file mode 100644
index 000000000..cf76154aa
--- /dev/null
+++ b/cli/src/lib/legacyPane.ts
@@ -0,0 +1,36 @@
+import type { AgentEngine } from '../engines/types.js'
+import type { PaneInspection } from '../engines/facets/screen.js'
+import { inspectPane, stripAnsi } from '../engines/kit/pane.js'
+import { parsePiFooterProfile } from '../engines/pi/runtimeProfile.js'
+const PI_FOOTER_BUDGET = /\d+(?:\.\d+)?%\s*\/\s*\S+/
+export function inspectRuntimePane(engine: AgentEngine, capture: string): PaneInspection { return engine === 'pi' ? inspectPiPane(capture) : inspectPane(engine, capture) }
+function inspectPiPane(capture: string): PaneInspection {
+ const lines = stripAnsi(capture).split('\n').map((line) => line.replace(/\s+$/, ''))
+ const text = lines.join('\n')
+ // Whether the footer is DRAWN, which is not the same question as whether its profile can be READ —
+ // and conflating the two is what has broken this check twice.
+ //
+ // `parsePiFooterProfile` is strict on purpose: it must return a model AND an effort or nothing, so a
+ // half-drawn redraw is never mistaken for a profile change. Idleness needs far less — only that the
+ // normal view is on screen rather than a picker — and borrowing the strict reader for it meant every
+ // footer shape it could not fully parse read as "still busy".
+ //
+ // The shape it could not parse: a model with no thinking ladder draws no `• <level>` at all, so the
+ // line ends at the model name. EVERY grid model is that shape, because the provider block written in
+ // `gridLaunch.ts` declares `reasoning: false` — which left every Pi agent on a grid permanently
+ // AGENT_BUSY, refusing model switches over an idle pane (measured, pi 0.82, footer
+ // `↑5.4k ↓292 1.5%/200k (auto) Auto`).
+ //
+ // So: the profile still counts when it parses, and the token-budget readout — `0.0%/500k`,
+ // `1.5%/200k`, present in both shapes and drawn by no picker — carries the rest.
+ const footer = parsePiFooterProfile(text) !== null || PI_FOOTER_BUDGET.test(text)
+ const rules: number[] = []
+ lines.forEach((line, index) => { if (/^\s*─{8,}\s*$/.test(line)) rules.push(index) })
+ const dialog = /Thinking Level|Select reasoning depth|Type to search|Enter to select|Only showing models from/i
+ .test(lines.join('\n'))
+ if (rules.length < 2) return { idle: false, plan: false, dialog, draft: false }
+ const band = lines.slice(rules[rules.length - 2] + 1, rules[rules.length - 1]).join('').trim()
+ const draft = band.length > 0
+ return { idle: footer && !dialog && !draft, plan: false, dialog, draft }
+}
+
diff --git a/cli/src/lib/legacyScreen.ts b/cli/src/lib/legacyScreen.ts
new file mode 100644
index 000000000..6b8472eb3
--- /dev/null
+++ b/cli/src/lib/legacyScreen.ts
@@ -0,0 +1,9 @@
+import type { AgentEngine } from '../engines/types.js'
+import { inspectRuntimePane as inspectPane } from './legacyPane.js'
+import { assembleScreen } from '../engines/kit/screen.js'
+import { parseEngineQuestionPane } from './questionPane.js'
+export function legacyScreen(engine: AgentEngine, capture: string | null) {
+ if (capture === null) return { pane: { idle: false, plan: false, dialog: false, draft: false }, question: null, messageHold: null,
+ teamHold: 'team_waiting_unavailable', activity: null, busy: false, stoppedGoal: false }
+ return assembleScreen(engine, capture, inspectPane(engine, capture), parseEngineQuestionPane(engine, capture))
+}
diff --git a/cli/src/lib/messageHold.ts b/cli/src/lib/messageHold.ts
index 13806bb1b..783e66705 100644
--- a/cli/src/lib/messageHold.ts
+++ b/cli/src/lib/messageHold.ts
@@ -1,134 +1,3 @@
-/**
- * Whether a message can be typed into an agent's pane as it stands, and if not, why not and what the
- * person can do about it.
- *
- * A message is a bracketed paste followed by Enter. Typed into the composer, that sends it. Typed into
- * a dialog, it does something else: the dialogs below drop the paste, and the Enter confirms what is
- * highlighted. Read from the engines' own code (Claude Code 2.1.289's bundle, Codex 0.160's tui/src):
- * - a permission prompt approves its highlighted row, the first: Yes, run it (Claude Code's Select has
- * no paste handler; Codex's ApprovalOverlay leaves `handle_paste` to its default, and its own test
- * `enter_sets_last_selected_index_without_dismissing` expects Accept);
- * - a question picks its highlighted option (Claude Code), or takes the paste as notes on it and
- * submits that (Codex's request_user_input);
- * - a menu (model, effort and the other pickers) picks its highlighted row;
- * - a picker for a point to rewind to rewinds, or loses the message (Codex's transcript browser,
- * Claude Code's Rewind menu);
- * - a transcript view loses it (Claude Code's, ctrl+o; Codex's overlay, ctrl+t, in its scrollback
- * mode);
- * - a search takes it as what to search for, and Claude Code's prompt-history search then SENDS the
- * earlier prompt it found (ctrl+r);
- * - the questions asked at startup take the Enter as their highlighted answer: trust the folder
- * (Claude Code's highlights `No, exit`, and quits), run Codex's update, switch Codex's model, start
- * a sign-in.
- * So none of them is typed into. A question is answered through its own path (`question_response`),
- * never by a message.
- *
- * For Claude Code and Codex the screens above are named, to say what is open; anything else is typed
- * into only when the engine's own composer is on screen (composerScreen.ts), so a screen never seen
- * before is refused, never typed into. Other engines are read for their questions and permission
- * prompts only.
- *
- * One function for every route a message takes to a pane, so that none can skip it (core/input.ts).
- */
-import { engineLabel } from './agentNames.js'
-import { isApprovalDialog, parseEngineQuestionPane } from './askQuestion.js'
-import type { RegisteredSession } from './registry.js'
-import { COMPOSER_ENGINES, composerState } from './composerScreen.js'
-import { paneModal } from './runtimeProfileController.js'
-
-/** Why a message was not typed, as the reason its delivery is refused with. */
-export type MessageHold = 'permission_open' | 'question_open' | 'menu_open' | 'rewind_picker_open' | 'transcript_open'
- | 'search_open' | 'trust_open' | 'update_prompt_open' | 'model_prompt_open' | 'sign_in_open' | 'popup_open'
- | 'prompt_hidden' | 'screen_unreadable'
-
-const HOLDS: ReadonlySet<string> = new Set<MessageHold>(['permission_open', 'question_open', 'menu_open', 'rewind_picker_open',
- 'transcript_open', 'search_open', 'trust_open', 'update_prompt_open', 'model_prompt_open', 'sign_in_open', 'popup_open',
- 'prompt_hidden', 'screen_unreadable'])
-
-/**
- * The holds that can be passing: an engine still starting draws nothing yet, a redraw can be caught
- * between frames, a capture can fail once. A message waits a little for these to clear before it is
- * refused (sessionInput.ts), as it does for the pane's control lease.
- */
-const PASSING: ReadonlySet<string> = new Set<MessageHold>(['prompt_hidden', 'screen_unreadable'])
-export function passingHold(reason: string): boolean {
- return PASSING.has(reason)
-}
-
-/** The hold for each of the engines' own screens and modals (runtimeProfileController.ts `paneModal`). */
-const MODAL_HOLDS: Record<NonNullable<ReturnType<typeof paneModal>>, MessageHold> = {
- rewind: 'rewind_picker_open', transcript: 'transcript_open', search: 'search_open', trust: 'trust_open',
- update: 'update_prompt_open', model: 'model_prompt_open', sign_in: 'sign_in_open', permission: 'permission_open', menu: 'menu_open',
-}
-
-/** What the pane shows that a message must not be typed into, or null when it can be typed. */
-export function messageHold(engine: RegisteredSession['engine'], capture: string | null): MessageHold | null {
- const allowlisted = COMPOSER_ENGINES.has(engine)
- if (capture === null) return allowlisted ? 'screen_unreadable' : null
- const view = parseEngineQuestionPane(engine, capture)
- if (view) return view.kind === 'question' && isApprovalDialog(view) ? 'permission_open' : 'question_open'
- if (!allowlisted) return null
- const modal = paneModal(engine, capture)
- if (modal) return MODAL_HOLDS[modal]
- const composer = composerState(engine, capture)
- return composer === 'ready' ? null : composer === 'popup' ? 'popup_open' : 'prompt_hidden'
-}
-
-export function isMessageHold(reason: string): reason is MessageHold {
- return HOLDS.has(reason)
-}
-
-function engineName(engine: string): string {
- return engine === 'claude' ? 'Claude Code' : engineLabel(engine)
-}
-
-/**
- * What the person is told when their message was typed but its Enter not pressed: the engine opened
- * something between the two, which the Enter would have answered. The text waits in the composer.
- */
-export function messageWithheldText(engine: string, hold: string): string {
- const name = engineName(engine)
- const [what, then] = hold === 'permission_open' ? ['asked for permission', 'Answer the request in the app or in its terminal']
- : hold === 'question_open' ? ['asked you a question', 'Answer the question in the app or in its terminal']
- : ['opened another screen', 'Finish what\'s on its screen in its terminal']
- return `${name} ${what} just as your message was typed, so it was not sent; it waits in ${name}'s prompt. ${then}, then press Enter in its terminal to send the message, or clear it there.`
-}
-
-/** What the person is told: why their message was not typed, and what lets it through. */
-export function messageHoldText(engine: string, hold: MessageHold): string {
- const name = engineName(engine)
- switch (hold) {
- case 'permission_open':
- return `${name} is asking for ${engine === 'codex' ? 'approval' : 'permission'}. Answer it first, in the app or in its terminal, then send the message again.`
- case 'question_open':
- return `${name} is asking you a question. Answer it first, in the app or in its terminal, then send the message again.`
- case 'menu_open':
- return `${name} has a menu open, where Enter would pick from it. Close it with Esc in its terminal, then send the message again.`
- case 'rewind_picker_open':
- return engine === 'codex'
- ? 'Codex is browsing its transcript, where Enter would rewind the conversation. Close it with Esc in its terminal, then send the message again.'
- : `${name} has its Rewind menu open, where Enter would pick a point to rewind to. Close it with Esc in its terminal, then send the message again.`
- case 'transcript_open':
- return engine === 'codex'
- ? 'Codex is showing its transcript (ctrl+t), where a message is not typed. Close it with q in its terminal, then send the message again.'
- : `${name} is showing its transcript (ctrl+o), where a message is not typed. Close it with Esc in its terminal, then send the message again.`
- case 'search_open':
- return engine === 'codex'
- ? 'Codex has a search open, where a message would become what it searches for. Close it with Esc in its terminal, then send the message again.'
- : `${name} is searching its prompt history (ctrl+r), where Enter would send an earlier prompt. Close it with ctrl+c in its terminal, then send the message again.`
- case 'trust_open':
- return `${name} is asking whether to trust this folder. Answer it in its terminal, then send the message again.`
- case 'update_prompt_open':
- return `${name} is asking whether to update. Answer it in its terminal, then send the message again.`
- case 'model_prompt_open':
- return `${name} is asking whether to switch to a new model. Answer it in its terminal, then send the message again.`
- case 'sign_in_open':
- return `${name} is asking how to sign in. Sign in in its terminal, then send the message again.`
- case 'popup_open':
- return `${name} message not sent. Close suggestions with Esc in its terminal, then retry.`
- case 'prompt_hidden':
- return `${name} isn't showing its prompt; finish what's on its screen in its terminal, then send the message again.`
- case 'screen_unreadable':
- return `${name}'s screen could not be read, so the message was not typed. Send it again in a moment.`
- }
-}
+/** Inline compatibility API. Core imports the value-only messageHolds module. */
+export * from './messageHolds.js'
+export { messageHold } from '../engines/screens.js'
diff --git a/cli/src/lib/messageHolds.ts b/cli/src/lib/messageHolds.ts
new file mode 100644
index 000000000..c40169aa9
--- /dev/null
+++ b/cli/src/lib/messageHolds.ts
@@ -0,0 +1,76 @@
+import { engineLabel } from './agentNames.js'
+import type { MessageHold } from '../engines/facets/screen.js'
+export type { MessageHold } from '../engines/facets/screen.js'
+
+const HOLDS: ReadonlySet<string> = new Set<MessageHold>(['permission_open', 'question_open', 'menu_open', 'rewind_picker_open',
+ 'transcript_open', 'search_open', 'trust_open', 'update_prompt_open', 'model_prompt_open', 'sign_in_open', 'popup_open',
+ 'prompt_hidden', 'screen_unreadable'])
+
+/**
+ * The holds that can be passing: an engine still starting draws nothing yet, a redraw can be caught
+ * between frames, a capture can fail once. A message waits a little for these to clear before it is
+ * refused (sessionInput.ts), as it does for the pane's control lease.
+ */
+const PASSING: ReadonlySet<string> = new Set<MessageHold>(['prompt_hidden', 'screen_unreadable'])
+export function passingHold(reason: string): boolean {
+ return PASSING.has(reason)
+}
+
+export function isMessageHold(reason: string): reason is MessageHold {
+ return HOLDS.has(reason)
+}
+
+function engineName(engine: string): string {
+ return engine === 'claude' ? 'Claude Code' : engineLabel(engine)
+}
+
+/**
+ * What the person is told when their message was typed but its Enter not pressed: the engine opened
+ * something between the two, which the Enter would have answered. The text waits in the composer.
+ */
+export function messageWithheldText(engine: string, hold: string): string {
+ const name = engineName(engine)
+ const [what, then] = hold === 'permission_open' ? ['asked for permission', 'Answer the request in the app or in its terminal']
+ : hold === 'question_open' ? ['asked you a question', 'Answer the question in the app or in its terminal']
+ : ['opened another screen', 'Finish what\'s on its screen in its terminal']
+ return `${name} ${what} just as your message was typed, so it was not sent; it waits in ${name}'s prompt. ${then}, then press Enter in its terminal to send the message, or clear it there.`
+}
+
+/** What the person is told: why their message was not typed, and what lets it through. */
+export function messageHoldText(engine: string, hold: MessageHold): string {
+ const name = engineName(engine)
+ switch (hold) {
+ case 'permission_open':
+ return `${name} is asking for ${engine === 'codex' ? 'approval' : 'permission'}. Answer it first, in the app or in its terminal, then send the message again.`
+ case 'question_open':
+ return `${name} is asking you a question. Answer it first, in the app or in its terminal, then send the message again.`
+ case 'menu_open':
+ return `${name} has a menu open, where Enter would pick from it. Close it with Esc in its terminal, then send the message again.`
+ case 'rewind_picker_open':
+ return engine === 'codex'
+ ? 'Codex is browsing its transcript, where Enter would rewind the conversation. Close it with Esc in its terminal, then send the message again.'
+ : `${name} has its Rewind menu open, where Enter would pick a point to rewind to. Close it with Esc in its terminal, then send the message again.`
+ case 'transcript_open':
+ return engine === 'codex'
+ ? 'Codex is showing its transcript (ctrl+t), where a message is not typed. Close it with q in its terminal, then send the message again.'
+ : `${name} is showing its transcript (ctrl+o), where a message is not typed. Close it with Esc in its terminal, then send the message again.`
+ case 'search_open':
+ return engine === 'codex'
+ ? 'Codex has a search open, where a message would become what it searches for. Close it with Esc in its terminal, then send the message again.'
+ : `${name} is searching its prompt history (ctrl+r), where Enter would send an earlier prompt. Close it with ctrl+c in its terminal, then send the message again.`
+ case 'trust_open':
+ return `${name} is asking whether to trust this folder. Answer it in its terminal, then send the message again.`
+ case 'update_prompt_open':
+ return `${name} is asking whether to update. Answer it in its terminal, then send the message again.`
+ case 'model_prompt_open':
+ return `${name} is asking whether to switch to a new model. Answer it in its terminal, then send the message again.`
+ case 'sign_in_open':
+ return `${name} is asking how to sign in. Sign in in its terminal, then send the message again.`
+ case 'popup_open':
+ return `${name} message not sent. Close suggestions with Esc in its terminal, then retry.`
+ case 'prompt_hidden':
+ return `${name} isn't showing its prompt; finish what's on its screen in its terminal, then send the message again.`
+ case 'screen_unreadable':
+ return `${name}'s screen could not be read, so the message was not typed. Send it again in a moment.`
+ }
+}
diff --git a/cli/src/lib/questionController.spec.ts b/cli/src/lib/questionController.spec.ts
new file mode 100644
index 000000000..fbdf365de
--- /dev/null
+++ b/cli/src/lib/questionController.spec.ts
@@ -0,0 +1,63 @@
+import { readFileSync } from 'node:fs'
+import { afterEach, describe, expect, it, vi } from 'vitest'
+import { screenFor } from '../engines/screens.js'
+import type { PaneView } from '../engines/facets/screen.js'
+import type { RegisteredSession } from './registry.js'
+import { AskQuestionController, QuestionWatcher } from './questionController.js'
+
+const capture = readFileSync(new URL('./__fixtures__/permission-claude.txt', import.meta.url), 'utf8')
+const view = screenFor('claude').inspect(capture).question
+const row = () => ({ agentId: 'agent', sessionId: 'session', engine: 'claude', active: true }) as RegisteredSession
+const deferred = <T>() => { let resolve!: (value: T) => void; const promise = new Promise<T>(r => { resolve = r }); return { promise, resolve } }
+afterEach(() => { vi.useRealTimers(); vi.restoreAllMocks() })
+
+describe('asynchronous question evidence', () => {
+ it('keeps a previously announced question while the screen reader is unavailable', async () => {
+ vi.useFakeTimers()
+ const current = row(), onQuestion = vi.fn(), onQuestionGone = vi.fn()
+ const readQuestion = vi.fn(async (): Promise<PaneView> => view)
+ const watcher = new QuestionWatcher({ getSession: () => current, capture: async () => capture,
+ hasDevice: () => true, readQuestion, onQuestion, onQuestionGone })
+ watcher.start(current.sessionId)
+ await vi.advanceTimersByTimeAsync(1_500)
+ expect(onQuestion).toHaveBeenCalledOnce()
+ readQuestion.mockRejectedValue(new Error('worker unavailable'))
+ await vi.advanceTimersByTimeAsync(6_000)
+ expect(onQuestionGone).not.toHaveBeenCalled()
+ readQuestion.mockResolvedValue(null)
+ await vi.advanceTimersByTimeAsync(3_000)
+ expect(onQuestionGone).toHaveBeenCalledOnce()
+ watcher.stopAll()
+ })
+ it.each(['replace', 'stop'] as const)('holds one pending interpretation and discards it after %s', async change => {
+ vi.useFakeTimers()
+ const current = row(), pending = deferred<PaneView>(), onQuestion = vi.fn()
+ const readQuestion = vi.fn(() => pending.promise)
+ const watcher = new QuestionWatcher({ getSession: () => current, capture: async () => capture,
+ hasDevice: () => true, readQuestion, onQuestion })
+ watcher.start(current.sessionId)
+ await vi.advanceTimersByTimeAsync(6_000)
+ expect(readQuestion).toHaveBeenCalledOnce()
+ if (change === 'stop') watcher.stop(current.sessionId)
+ else current.sessionId = 'replacement'
+ pending.resolve(view)
+ await vi.advanceTimersByTimeAsync(0)
+ expect(onQuestion).not.toHaveBeenCalled()
+ watcher.stopAll()
+ })
+ it.each(['unavailable', 'rebound', 'capture-rebound'] as const)('types no answer after %s and releases its input lease', async change => {
+ const current = row(), release = vi.fn(), sendKey = vi.fn(async () => true), sendText = vi.fn(async () => true)
+ const controller = new AskQuestionController({ getSession: () => current,
+ capture: async () => { if (change === 'capture-rebound') current.sessionId = 'replacement'; return capture },
+ sendKey, sendText, acquireControl: () => release,
+ readQuestion: async () => {
+ if (change === 'unavailable') throw new Error('worker failed')
+ current.sessionId = 'replacement'
+ return view
+ } })
+ const result = await controller.answer({ agentId: 'agent', answers: { 'question': 'No' } })
+ expect(result).toMatchObject({ ok: false, error: change === 'unavailable' ? 'ANSWER_FAILED' : 'STALE_QUESTION' })
+ expect(sendKey).not.toHaveBeenCalled(); expect(sendText).not.toHaveBeenCalled()
+ expect(release).toHaveBeenCalledOnce()
+ })
+})
diff --git a/cli/src/lib/questionController.ts b/cli/src/lib/questionController.ts
new file mode 100644
index 000000000..a2cc09772
--- /dev/null
+++ b/cli/src/lib/questionController.ts
@@ -0,0 +1,805 @@
+import type { RegisteredSession } from './registry.js'
+import type { AgentEngine } from '../engines/types.js'
+import type { ShapedQuestion, QuestionRow, QuestionView, ReviewView, PaneView } from '../engines/facets/screen.js'
+export type { ShapedQuestion, QuestionRow, QuestionView, ReviewView, PaneView, FoundDialog } from '../engines/facets/screen.js'
+import { ampSelectionKeys } from '../engines/amp/askQuestion.js'
+import { kiloSelectionKeys } from '../engines/kilo/askQuestion.js'
+import { isApprovalDialog } from '../engines/kit/questionPane.js'
+function multiSubmitKey(engine: AgentEngine): string { return engine === 'devin' ? 'Enter' : 'Tab' }
+const STEP_MS = 350 // let the TUI repaint between keystrokes
+const TEXT_MS = 250
+const MAX_STEPS = 14 // hard bound on the drive loop (questions × keys), never spin on a stuck pane
+const CAPTURE_LINES = 60
+
+function norm(value: string): string {
+ return value.toLowerCase().replace(/\s+/g, ' ').replace(/[.…]+$/, '').trim()
+}
+
+function sleep(ms: number): Promise<void> {
+ return new Promise((resolve) => setTimeout(resolve, ms))
+}
+
+/**
+ * Shape a raw AskUserQuestion input into the device form. Port of the hosted runtime’s `commanderQuestions`
+ * (websocket.ts): `key` MUST stay the value the CLI matches answers by — prompt/question first — because
+ * the device echoes it back as the answers-map key.
+ */
+export function shapeQuestions(questions: unknown): ShapedQuestion[] {
+ return (Array.isArray(questions) ? questions : []).map((raw, i) => {
+ const q = (raw ?? {}) as Record<string, unknown>
+ const opts = Array.isArray(q.options) ? q.options : []
+ return {
+ key: (q.prompt as string) || (q.question as string) || (q.id as string) || (q.header as string) || `question_${i}`,
+ q: (q.prompt as string) || (q.question as string) || '',
+ options: opts
+ .map((o) => String((typeof o === 'string' ? o : (o as Record<string, unknown>)?.label) ?? ''))
+ .filter(Boolean),
+ // Three spellings because three CLIs: `allow_multiple` (the hosted runtime), `multiSelect` (claude),
+ // `multi_select` (devin — read from a real `ask_user_question` call in its SQLite store).
+ multi: (q.allow_multiple as boolean) ?? (q.multiSelect as boolean) ?? (q.multi_select as boolean) ?? false,
+ }
+ })
+}
+
+function rowKeys(engine: AgentEngine, row: QuestionRow, view?: QuestionView): string[] {
+ if (engine === 'amp') return ampSelectionKeys(row)
+ if (engine === 'codex') return codexRowKeys(row, view)
+ // Kilo's rows sit side by side, so its walk is horizontal — see engines/kilo/askQuestion.ts.
+ if (engine === 'kilo') return kiloSelectionKeys(row)
+ // Same dialog, different engine: opencode numbers its ask dialog but not its permission prompt, so the
+ // ROW says how it is reached and a per-engine rule would break one of the two.
+ if (row.walk === 'right') return kiloSelectionKeys(row)
+ if (row.walk === 'down') return ampSelectionKeys(row)
+ return [row.number]
+}
+
+/**
+ * Codex, measured 2026-09-15 on 0.149: in its request_user_input dialog a digit only MOVES the highlight
+ * and Enter submits (`enter to submit answer`) — a digit alone left the dialog up and the device's answer
+ * reported as stuck. Its approval prompt is the other way round and was verified earlier: one digit
+ * selects and commits, so that one keeps the single key.
+ */
+export function codexRowKeys(row: QuestionRow, view?: QuestionView): string[] {
+ return view?.enterSubmits ? [row.number, 'Enter'] : [row.number]
+}
+
+export function matchRow(rows: QuestionRow[], answer: string): QuestionRow | null {
+ const a = norm(answer)
+ if (!a) return null
+ return rows.find((r) => norm(r.label) === a)
+ ?? (a.length >= 3 ? rows.find((r) => norm(r.label).startsWith(a)) ?? rows.find((r) => a.startsWith(norm(r.label)) && norm(r.label).length >= 3) : undefined)
+ ?? null
+}
+
+/**
+ * Pick the answer for the question the dialog is currently showing: the entry keyed by its own text.
+ *
+ * `positional` also takes the next unused entry when none names it. Only for a dialog the answer's
+ * requestId proves it was written for: without that proof, an answer that names no question on screen
+ * belongs to one that is gone, and typing it here would answer — or approve — something nobody saw.
+ *
+ * The text must be the question's OWN (case, spacing and a trailing `…` aside), never a prefix either way.
+ * An approval is titled `Approve <header>: <argument>` and the header is shared by every prompt of its
+ * kind: a key left from an earlier prompt — `Approve Bash command` (its argument unread) — was a prefix of
+ * `Approve Bash command: rm -rf ~/projects` and pressed Yes on it, and an old full title named a
+ * header-only prompt the other way round. A client echoes back the key it was announced, so the whole
+ * text is always there to match; one it has cut is answered through its requestId (`positional`).
+ */
+export function pickAnswer(
+ answers: Record<string, string>,
+ question: string,
+ used: Set<string>,
+ opts: { positional?: boolean } = {},
+): { key: string; value: string } | null {
+ const entries = Object.entries(answers)
+ const q = norm(question)
+ const byText = q ? entries.find(([k]) => norm(k) === q) : undefined
+ if (byText && !used.has(byText[0])) return { key: byText[0], value: byText[1] }
+ if (!opts.positional) return null
+ const next = entries.find(([k]) => !used.has(k))
+ return next ? { key: next[0], value: next[1] } : null
+}
+
+/**
+ * The id a dialog is announced under — the SAME function the watcher names it with, so the answer's
+ * requestId can be checked against the dialog on screen at the moment of typing rather than against
+ * whatever the watcher last saw (it polls every 1.5s, and forgets on a reset).
+ */
+export function questionRequestId(sessionId: string, view: QuestionView): string {
+ return `q_${hash(sessionId + fingerprintOf(view))}`
+}
+
+/** Why an answer was not keyed. Sent back to the client as `question_response_result.error`. */
+export type QuestionAnswerError = 'STALE_QUESTION' | 'AGENT_NOT_FOUND' | 'ANSWER_BUSY' | 'ANSWER_FAILED'
+
+export type QuestionAnswerResult = { ok: true } | { ok: false; error: QuestionAnswerError; detail: string }
+
+const STALE_CHANGED: QuestionAnswerResult = { ok: false, error: 'STALE_QUESTION', detail: 'That question changed before your answer arrived.' }
+const STALE_GONE: QuestionAnswerResult = { ok: false, error: 'STALE_QUESTION', detail: 'That question is no longer open.' }
+const failed = (detail: string): QuestionAnswerResult => ({ ok: false, error: 'ANSWER_FAILED', detail })
+const KEYS_FAILED = failed('The answer could not be typed into the agent\'s terminal.')
+const STUCK = failed('The question did not take the answer.')
+
+export interface AskQuestionDeps {
+ readQuestion(session: RegisteredSession, capture: string): PaneView | Promise<PaneView>
+ getSession: (sessionId: string) => RegisteredSession | undefined
+ capture: (terminalTarget: string, historyLines?: number) => Promise<string | null>
+ sendText: (terminalTarget: string, text: string) => Promise<boolean>
+ sendKey: (terminalTarget: string, key: string) => Promise<boolean>
+ /** Pins one backend locator for the whole multi-step dialog drive. */
+ acquireControl?: (sessionId: string, opts?: { forAnswer?: boolean }) => (() => void) | null
+ /** Injected for tests. */
+ wait?: (ms: number) => Promise<void>
+}
+
+export interface QuestionAnswerPayload {
+ /** Exact contents reviewed on the device; guarded submissions never use positional fallback. */
+ expectedQuestions?: ShapedQuestion[]
+ selectedLabels?: Record<string, string[]>
+ freeTextKeys?: string[]
+ allowPermissions?: boolean
+ requestId?: string
+ sessionId?: string
+ agentId?: string
+ answers?: Record<string, string>
+}
+
+/**
+ * Owns the OUT side's pending map (requestId → session) and the IN side's pane driving. One answer at a
+ * time per agent: a second `question_response` for a dialog already being driven is dropped, not queued.
+ */
+export class AskQuestionController {
+ private pending = new Map<string, string>() // requestId → sessionId
+ private driving = new Set<string>() // sessionIds currently keying a dialog
+
+ constructor(private readonly deps: AskQuestionDeps) {}
+
+ /** Remember which session a mirrored question belongs to (the device may answer minutes later). */
+ remember(requestId: string, sessionId: string): void {
+ if (!requestId) return
+ if (this.pending.size > 64) this.pending.delete(this.pending.keys().next().value as string)
+ this.pending.set(requestId, sessionId)
+ }
+
+ async answer(payload: QuestionAnswerPayload): Promise<QuestionAnswerResult> {
+ const requestId = payload.requestId ?? ''
+ if (payload.expectedQuestions !== undefined && (!Array.isArray(payload.expectedQuestions) ||
+ payload.expectedQuestions.length < 1 || payload.expectedQuestions.length > 4 ||
+ payload.expectedQuestions.some(q => !q || typeof q.key !== 'string' || !q.key ||
+ typeof q.q !== 'string' || !q.q || !Array.isArray(q.options) || !q.options.length ||
+ q.options.some(option => typeof option !== 'string' || !option) ||
+ typeof q.multi !== 'boolean' || (q.multi && !Array.isArray(payload.selectedLabels?.[q.key]))))) {
+ return failed('The reviewed question metadata is invalid.')
+ }
+ if (payload.freeTextKeys !== undefined && (!payload.expectedQuestions || !Array.isArray(payload.freeTextKeys) ||
+ payload.freeTextKeys.some(key => typeof key !== 'string' || !payload.expectedQuestions!.some(q =>
+ q.key === key && q.canText === true && !q.multi)))) {
+ return failed('The reviewed text answer metadata is invalid.')
+ }
+ const remembered = this.pending.get(requestId)
+ const sessionId = payload.sessionId || payload.agentId || remembered || ''
+ const answers = payload.answers && typeof payload.answers === 'object' ? payload.answers : null
+ if (!sessionId || !answers || Object.keys(answers).length === 0) {
+ console.warn(`[question] ignoring answer with no session/answers (req=${requestId})`)
+ return failed('The answer named no agent or carried no choice.')
+ }
+ const session = this.deps.getSession(sessionId)
+ const terminalTarget = session?.agentId || session?.sessionId
+ if (!terminalTarget) {
+ console.warn(`[question] no terminal target for ${sessionId.slice(0, 8)} — answer dropped`)
+ return { ok: false, error: 'AGENT_NOT_FOUND', detail: 'That harness is no longer running.' }
+ }
+ if (remembered) {
+ const owner = this.deps.getSession(remembered)
+ if ((owner?.agentId || owner?.sessionId) !== terminalTarget) return STALE_CHANGED
+ }
+ if (this.driving.has(terminalTarget)) {
+ console.warn(`[question] ${sessionId.slice(0, 8)} answer dropped · already driving this dialog`)
+ return { ok: false, error: 'ANSWER_BUSY', detail: 'Another answer is already being entered for this harness.' }
+ }
+ // `forAnswer`: a dialog is the engine waiting for input mid-turn, so the open turn must not block it.
+ const release = this.deps.acquireControl?.(terminalTarget, { forAnswer: true })
+ // Silence here is the failure mode this whole file exists to prevent: the device sends an answer,
+ // nothing keys it in, and the pane sits on the dialog looking like a hung agent.
+ if (this.deps.acquireControl && !release) {
+ console.warn(`[question] ${sessionId.slice(0, 8)} answer dropped · terminal control unavailable`)
+ return { ok: false, error: 'ANSWER_BUSY', detail: 'The agent\'s terminal is busy. Try again.' }
+ }
+ // The ids the watcher could have announced this dialog under: the session it was remembered for, and
+ // the session as the registry knows it now.
+ const owners = [...new Set([remembered, session?.sessionId].filter((id): id is string => !!id))]
+ this.driving.add(terminalTarget)
+ try {
+ const result = await this.drive(terminalTarget, answers, this.deps.getSession(sessionId)?.engine ?? 'claude', payload.allowPermissions !== false, { requestId, owners }, payload.expectedQuestions ? payload : undefined)
+ this.pending.delete(requestId)
+ const outcome = result.ok ? 'submitted' : result.error === 'STALE_QUESTION' ? 'refused · STALE_QUESTION, nothing typed' : 'FAILED'
+ console.log(`[question] ${sessionId.slice(0, 8)} answered from device · ${outcome} (req=${requestId || 'none'})`)
+ return result
+ } finally {
+ this.driving.delete(terminalTarget)
+ release?.()
+ }
+ }
+
+ /**
+ * Key the answers into the pane's dialog, question by question, ending on the review screen.
+ *
+ * Nothing is typed until the dialog on screen is shown to be the question the answer was written for:
+ * its requestId when the answer carries one, else its own text. An answer can arrive late — the agent
+ * moved on, another client answered, the next question of the form is up — and positionally matching
+ * it to whatever is showing now is how a person's "Yes" lands on a permission prompt they never saw.
+ */
+ private async drive(
+ terminalTarget: string,
+ answers: Record<string, string>,
+ engine: AgentEngine,
+ allowPermissions: boolean,
+ asked: { requestId: string; owners: string[] },
+ reviewed?: QuestionAnswerPayload,
+ ): Promise<QuestionAnswerResult> {
+ const wait = this.deps.wait ?? sleep
+ const used = new Set<string>()
+ const reviewedComplete = () => !reviewed || used.size === reviewed.expectedQuestions!.length
+ let lastQuestion = ''
+ let repeats = 0
+ let blanks = 0
+ let answered = 0
+
+ const generation = questionCaptureGeneration(this.deps.getSession(terminalTarget))
+ for (let step = 0; step < MAX_STEPS; step++) {
+ const current = this.deps.getSession(terminalTarget)
+ if (!current || generation !== questionCaptureGeneration(current)) return STALE_CHANGED
+ const capture = await this.deps.capture(terminalTarget, CAPTURE_LINES)
+ if (reviewed && capture === null) return failed('The question could not be read.')
+ if (generation !== questionCaptureGeneration(this.deps.getSession(terminalTarget))) return STALE_CHANGED
+ let view: PaneView
+ try { view = await this.deps.readQuestion(current, capture ?? '') }
+ catch { return failed('The question could not be read.') }
+ if (generation !== questionCaptureGeneration(this.deps.getSession(terminalTarget))) return STALE_CHANGED
+ if (!view) {
+ // Nothing on screen: either the dialog was already gone, or the last keystroke submitted it.
+ return answered > 0 && reviewedComplete() ? { ok: true } : STALE_GONE
+ }
+ if (!allowPermissions && view.kind === 'question' && view.permission) return failed('Permission prompts cannot be answered from here.')
+ if (view.kind === 'question' && view.partial) {
+ // The dialog's top is out of the pane. If we have keyed an answer it has not been taken yet;
+ // give the TUI a beat. If we have not, there is nothing to match an answer against.
+ if (answered === 0) { console.warn('[question] dialog scrolled out of view — cannot key an answer'); return failed('The question is scrolled out of view.') }
+ if (++repeats >= 2) { console.warn('[question] dialog stuck (scrolled)'); return STUCK }
+ await wait(STEP_MS)
+ continue
+ }
+ if (view.kind === 'review') {
+ // Reached by our own keys, this submits the form. Reached first, it means every question was
+ // answered somewhere else — submitting would send answers this person never gave.
+ if (answered === 0) return STALE_GONE
+ if (!reviewedComplete()) return STALE_CHANGED
+ return await this.deps.sendKey(terminalTarget, view.submitRow) ? { ok: true } : failed('The answers could not be submitted.')
+ }
+ // Mid-repaint the question line can read blank for a capture (see parseQuestionPane). Neither its id
+ // nor its text can be checked against a blank, so look again rather than judge the dialog by it.
+ if (!view.question) {
+ if (++blanks > 2) return answered > 0 && reviewedComplete() ? { ok: true } : failed('The question could not be read.')
+ await wait(STEP_MS)
+ continue
+ }
+ blanks = 0
+ // The same question still showing after we acted on it: give the TUI one more beat to repaint,
+ // then treat it as stuck rather than hammering the pane with more keystrokes. Never consume a
+ // second answer for it.
+ if (view.question === lastQuestion) {
+ if (++repeats >= 2) { console.warn(`[question] dialog stuck on "${view.question.slice(0, 60)}"`); return STUCK }
+ await wait(STEP_MS)
+ continue
+ }
+ repeats = 0
+ lastQuestion = view.question
+
+ // Is this the question the answer was for? Only the FIRST one needs the id: every later screen is
+ // one our own keys advanced to, and must be named by its text (below).
+ let positional = false
+ if (answered === 0 && asked.requestId) {
+ if (!asked.owners.some((owner) => questionRequestId(owner, view) === asked.requestId)) {
+ console.warn(`[question] answer for req=${asked.requestId} arrived after the dialog changed to "${view.question.slice(0, 60)}" — nothing typed`)
+ return STALE_CHANGED
+ }
+ positional = true
+ }
+
+ // Out of answers with the dialog still up = a multi-QUESTION dialog whose next question the device
+ // hasn't been shown yet. Leave it open: the watcher pushes that one and the device answers it next.
+ const expected = reviewed?.expectedQuestions?.find(q => q.q === view.question)
+ if (reviewed && (!expected || expected.multi !== view.multi ||
+ JSON.stringify(expected.options) !== JSON.stringify(view.rows.map(row => row.label)))) {
+ // A request id proves the initial dialog; every reviewed screen must also match its full choices.
+ return answered > 0 && reviewedComplete() ? { ok: true } : STALE_CHANGED
+ }
+ const picked = expected
+ ? (!used.has(expected.key) && typeof answers[expected.key] === 'string'
+ ? { key: expected.key, value: answers[expected.key] } : null)
+ : pickAnswer(answers, view.question, used, { positional })
+ if (!picked) {
+ if (answered > 0 && reviewedComplete()) return { ok: true }
+ console.warn(`[question] no answer names "${view.question.slice(0, 60)}" — nothing typed`)
+ return STALE_CHANGED
+ }
+ used.add(picked.key)
+ answered++
+
+ if (reviewed?.freeTextKeys?.includes(picked.key)) {
+ // Spoken words are explicitly text, even when they happen to equal an option label.
+ // A permission prompt can never acquire consent through this path.
+ if (!expected?.canText || view.permission || view.multi || !view.typeRow ||
+ !picked.value.trim() || Buffer.byteLength(picked.value, 'utf8') > 1200 ||
+ /[\x00-\x09\x0b-\x1f\x7f]/.test(picked.value)) return failed('The text answer cannot be entered into this question.')
+ if (!await this.typeFreeText(terminalTarget, view.typeRow, picked.value, wait)) return KEYS_FAILED
+ await wait(STEP_MS)
+ continue
+ }
+
+ if (view.multi) {
+ // Device joins the selected labels with ", " (q_done_tap).
+ const labels = reviewed ? reviewed.selectedLabels?.[picked.key] ?? []
+ : picked.value.split(',').map((s) => s.trim()).filter(Boolean)
+ if (reviewed && (!labels.length || labels.some(label => !view.rows.some(row => row.label === label)))) return failed('That answer matches no option.')
+ if (reviewed) {
+ // Set the exact reviewed set, including clearing choices selected in another client.
+ for (const row of view.rows) if (row.checked !== labels.includes(row.label)) {
+ if (!await this.deps.sendKey(terminalTarget, row.number)) return KEYS_FAILED
+ await wait(TEXT_MS)
+ }
+ if (!await this.deps.sendKey(terminalTarget, multiSubmitKey(engine))) return KEYS_FAILED
+ await wait(STEP_MS)
+ continue
+ }
+ let toggled = 0
+ for (const label of labels) {
+ const row = matchRow(view.rows, label)
+ if (!row || row.checked) continue
+ if (!await this.deps.sendKey(terminalTarget, row.number)) return KEYS_FAILED
+ await wait(TEXT_MS)
+ toggled++
+ }
+ if (!toggled && view.typeRow
+ && !await this.typeFreeText(terminalTarget, view.typeRow, picked.value, wait)) return KEYS_FAILED
+ if (!await this.deps.sendKey(terminalTarget, multiSubmitKey(engine))) return KEYS_FAILED // advance to the next question / review
+ await wait(STEP_MS)
+ continue
+ }
+
+ const row = reviewed ? view.rows.find(row => row.label === picked.value) ?? null : matchRow(view.rows, picked.value)
+ if (row) {
+ // One digit selects AND submits — except on Amp, whose rows are unnumbered and reached by
+ // walking the list, so this is a short sequence rather than a single key.
+ for (const key of rowKeys(engine, row, view)) {
+ if (!await this.deps.sendKey(terminalTarget, key)) return KEYS_FAILED
+ await wait(TEXT_MS)
+ }
+ await wait(STEP_MS)
+ continue
+ }
+ if (reviewed) return failed('That answer matches no option.')
+ if (!view.typeRow) { console.warn(`[question] no option matched "${picked.value.slice(0, 40)}" and no free-text row`); return failed('That answer matches no option.') }
+ if (!await this.typeFreeText(terminalTarget, view.typeRow, picked.value, wait)) return KEYS_FAILED
+ await wait(STEP_MS)
+ }
+ return STUCK
+ }
+
+ /** True while a dialog is being keyed — the watcher pauses so a half-driven dialog isn't re-announced. */
+ isDriving(sessionId: string): boolean {
+ const session = this.deps.getSession(sessionId)
+ return this.driving.has(session?.agentId || session?.sessionId || sessionId)
+ }
+
+ /** Free-text answer (a voice answer is always free text): open the "Type something." row, type, Enter. */
+ private async typeFreeText(terminalTarget: string, typeRow: QuestionRow, text: string, wait: (ms: number) => Promise<void>): Promise<boolean> {
+ if (!await this.deps.sendKey(terminalTarget, typeRow.number)) return false
+ await wait(TEXT_MS)
+ if (!await this.deps.sendText(terminalTarget, text)) return false
+ await wait(TEXT_MS)
+ return this.deps.sendKey(terminalTarget, 'Enter')
+ }
+}
+
+// ── watching a terminal for an open question ─────────────────────────────────────────────────────
+
+export interface QuestionWatcherDeps {
+ readQuestion(session: RegisteredSession, capture: string): PaneView | Promise<PaneView>
+ getSession: (sessionId: string) => RegisteredSession | undefined
+ capture: (terminalTarget: string, historyLines?: number) => Promise<string | null>
+ /** Skip the capture entirely when no device is listening — nothing would consume the question. */
+ hasDevice: () => boolean
+ /** A dialog is open on screen. Fires ONCE per distinct question (until it changes or closes). */
+ onQuestion: (sessionId: string, requestId: string, questions: ShapedQuestion[], detail?: { permission: boolean; dialog: string }) => void
+ /**
+ * An announced dialog LEFT the screen — answered somewhere else, or abandoned.
+ *
+ * This is the other half of `onQuestion`, and it was missing: a question answered in the app (or in the
+ * pane by hand) simply stopped being on screen, the watcher forgot it, and every OTHER client went on
+ * showing it. On the dial that is a screen you cannot leave without dismissing a question that no longer
+ * exists. The pane is the only source of truth here — there is no server-side question object — so
+ * "gone from the pane" is the only signal there is.
+ */
+ onQuestionGone?: (sessionId: string, requestId: string) => void
+
+ /** True while that session's dialog is being keyed by an answer already in flight. */
+ isDriving?: (sessionId: string) => boolean
+}
+
+const POLL_MS = 1500
+/** How long the pane read before a prompt stands for the turn it starts: past this, that turn's own start
+ * is read instead (a prompt that never became a turn). Turns were seen to start 8 s after their prompt
+ * under load. */
+const PROMPT_BASELINE_MS = 120_000
+// Consecutive empty polls before a question is declared gone.
+//
+// NOT 1. A capture taken mid-repaint parses as no-dialog, and announcing a close on that would yank a
+// live question off the dial's screen — the exact failure this feature exists to prevent, inverted.
+// Two ticks costs 1.5s of delay on a real close and makes a flicker unable to cause one.
+const GONE_TICKS = 2
+
+// Amp and codex are here for their PERMISSION prompt, not a question tool — neither has one. That prompt
+// is drawn only in the pane and recorded nowhere, so polling the pane is the only way it is ever seen.
+// Codex needs no parser of its own: it draws numbered rows under a `Press enter to confirm or esc to
+// cancel` footer, which is the shared parser's anchor exactly (`__fixtures__/permission-codex.txt`), and
+// the question it lands on is the command itself. Membership in this set is what starts the poll, so an
+// engine belongs here only once something can actually read its pane.
+const QUESTION_ENGINES = new Set<AgentEngine>(['claude', 'commandcode', 'codex', 'cursor', 'devin', 'hermes', 'opencode', 'muse', 'amp', 'kilo', 'grok', 'agy', 'copilot'])
+
+/** Does this engine ever paint a question dialog? Callers use it to decide whether to watch its pane. */
+export function pollsQuestions(engine: AgentEngine): boolean {
+ return QUESTION_ENGINES.has(engine)
+}
+
+/**
+ * The OUT half. The obvious source — the transcript's AskUserQuestion `tool_use` line — is useless here:
+ * the CLI does not flush that line until the question has been ANSWERED (its JSONL `timestamp` is the
+ * message's creation time, not its write time), so a device would only ever learn about a question after
+ * it no longer exists. The dialog itself, on the other hand, is on screen the whole time it is waiting.
+ *
+ * So while a turn is open we read the pane. This also means the question survives an adapter restart and
+ * re-announces to a device that attaches mid-question — neither of which a one-shot event could do.
+ */
+export class QuestionWatcher {
+ private readonly watching = new Set<string>()
+ private timer: NodeJS.Timeout | undefined
+ private readonly pendingPolls = new Map<string, { cancelled: boolean }>()
+ private readonly pendingBaselines = new Map<string, { cancelled: boolean }>()
+ private last = new Map<string, string>() // sessionId → fingerprint of the announced question
+ private lastId = new Map<string, string>() // sessionId → requestId of the announced question
+ private misses = new Map<string, number>() // sessionId → consecutive polls with no dialog
+ private readonly blocked = new Map<string, string>()
+ /** sessionId → the dialog that was ALREADY on the pane when this turn began. */
+ private readonly preTurn = new Map<string, string>()
+ /** sessionId → the pane as the daemon read it right before typing a prompt: its dialog, if any, and when. */
+ private readonly beforePrompt = new Map<string, { fingerprint: string | null; at: number }>()
+
+
+ constructor(private readonly deps: QuestionWatcherDeps) {}
+
+ /**
+ * A turn just began — remember any dialog already on the pane, so it is not
+ * announced as this turn's question.
+ *
+ * A dialog belonging to a turn cannot have been on screen before that turn's
+ * first byte. That invariant is the only way to tell "waiting for an answer"
+ * from "answered a moment ago and still drawn", because a pane looks
+ * identical either way.
+ *
+ * MEASURED: the daemon attached to a Codex that had just been asked about an
+ * update and answered in the app. Its first capture, 0.8s into the NEXT turn,
+ * still held that prompt — so the dial was shown a question nobody was being
+ * asked, and it stood there for 25s until the engine's output scrolled it out
+ * of the captured window.
+ *
+ * Deliberately NOT applied on attach: re-announcing a genuinely open dialog to
+ * a client that arrives mid-question is a feature, and there is no turn
+ * boundary there to reason from.
+ */
+ noteTurnStart(sessionId: string): void {
+ this.preTurn.delete(sessionId)
+ this.cancelPending(sessionId)
+ // The daemon typed this turn's prompt, and read the pane as it did: what was on it then is the turns
+ // before's, and nothing drawn since is (notePrompt says why).
+ const typed = this.beforePrompt.get(sessionId)
+ this.beforePrompt.delete(sessionId)
+ if (typed && Date.now() - typed.at <= PROMPT_BASELINE_MS) {
+ if (typed.fingerprint) this.preTurn.set(sessionId, typed.fingerprint)
+ return
+ }
+ const pending = { cancelled: false }
+ this.pendingBaselines.set(sessionId, pending)
+ void (async () => {
+ try {
+ const session = this.deps.getSession(sessionId)
+ const target = session?.agentId || session?.sessionId
+ if (!target || session?.active === false || (session?.engine && !pollsQuestions(session.engine))) return
+ const generation = questionCaptureGeneration(session)
+ const capture = await this.deps.capture(target, CAPTURE_LINES)
+ if (pending.cancelled || generation !== questionCaptureGeneration(this.deps.getSession(sessionId))) return
+ const view = await this.deps.readQuestion(session!, capture ?? '')
+ if (pending.cancelled || generation !== questionCaptureGeneration(this.deps.getSession(sessionId))) return
+ if (!view || view.kind !== 'question' || !view.question || view.rows.length === 0) return
+ // Only if nothing has been announced for this turn yet: the capture takes
+ // a moment, and a dialog that opened inside that window is this turn's.
+ if (this.lastId.has(sessionId)) return
+ this.preTurn.set(sessionId, fingerprintOf(view))
+ } catch {
+ // A failed read supplies no baseline. Keep watching for a fresh dialog.
+ } finally {
+ if (this.pendingBaselines.get(sessionId) === pending) this.pendingBaselines.delete(sessionId)
+ }
+ })()
+ }
+
+ /**
+ * The daemon is typing a prompt, and `capture` is the pane as it read it right before (core/input.ts
+ * messageWriter): the honest "before this turn" for the turn that prompt starts.
+ *
+ * A capture taken when the turn is SEEN to start can be too late for that. The turn is seen to start
+ * from the transcript, after the prompt is confirmed typed, and an engine can draw its question before
+ * that: measured in the soak run (e2e/endurance.e2e.ts) under load, Codex took the prompt at 20:42:54.9
+ * (its prompt hook), drew its question a moment later, and the turn was seen to start at 20:43:02.2, 7 s
+ * on. The capture at that start held the turn's own question, recorded it as the turns before's, and it
+ * was never announced: the agent waited on a question no window or device was shown.
+ *
+ * A read that failed supplies nothing: the turn's start reads the pane itself, as before.
+ */
+ notePrompt(sessionId: string, capture: string | null, view: PaneView): void {
+ const session = this.deps.getSession(sessionId)
+ if (capture === null || !session || (session.engine && !pollsQuestions(session.engine))) return
+ const dialog = view && view.kind === 'question' && view.question && view.rows.length > 0 ? fingerprintOf(view) : null
+ this.beforePrompt.set(sessionId, { fingerprint: dialog, at: Date.now() })
+ }
+
+ /** Poll this session's pane while its turn is open (called on turn_started). */
+ start(sessionId: string): void {
+ if (this.watching.has(sessionId)) return
+ const session = this.deps.getSession(sessionId)
+ // Only the engines that actually paint a question dialog: Claude and Command Code share one shape,
+ // devin has its own (parseDevinQuestionPane). Polling any other pane would be pure waste.
+ if (!session || session.active === false || !(session.agentId || session.sessionId) || !QUESTION_ENGINES.has(session.engine)) return
+ this.watching.add(sessionId)
+ // One clock also starts the reads in one event-loop turn, allowing the tmux
+ // backend to batch them without delaying polls or caching terminal content.
+ this.timer ??= setInterval(() => {
+ for (const id of this.watching) void this.tick(id)
+ }, POLL_MS)
+ }
+
+ /**
+ * Stop polling — and CLOSE any question still outstanding on this session.
+ *
+ * ⚠️ THIS IS THE COMMON CASE, NOT THE EDGE ONE, and leaving it out made the whole close mechanism look
+ * like it did not work. `stop()` is called on turn_ended, and answering the question is precisely what
+ * lets the turn end — so the dialog leaving the pane and the watcher being torn down happen within a
+ * second or two of each other, far inside the two-tick confirmation. Measured on hardware: the answer
+ * landed at ~16:50:19 and the turn ended at 16:50:29 with no close ever announced.
+ *
+ * No confirmation is needed here and none is wanted: the turn is over, so whatever we announced is
+ * definitively not waiting for anybody any more. The same holds for the other callers — an agent that
+ * was removed cannot answer either.
+ */
+ stop(sessionId: string): void {
+ this.cancelPending(sessionId)
+ this.preTurn.delete(sessionId)
+ this.blocked.delete(sessionId)
+ this.watching.delete(sessionId)
+ if (!this.watching.size && this.timer) { clearInterval(this.timer); this.timer = undefined }
+ const requestId = this.lastId.get(sessionId)
+ this.forget(sessionId)
+ if (requestId) this.deps.onQuestionGone?.(sessionId, requestId)
+ }
+
+
+ stopAll(): void {
+ if (this.timer) clearInterval(this.timer)
+ this.timer = undefined
+ this.watching.clear()
+ for (const pending of this.pendingPolls.values()) pending.cancelled = true
+ for (const pending of this.pendingBaselines.values()) pending.cancelled = true
+ this.preTurn.clear()
+ this.beforePrompt.clear()
+ this.blocked.clear()
+ this.last.clear()
+ this.lastId.clear()
+ this.misses.clear()
+ }
+
+ /** A device (re)joined: forget what we announced so an open question is pushed again. */
+ reset(): void {
+ this.last.clear()
+ this.lastId.clear()
+ this.misses.clear()
+ }
+
+ private forget(sessionId: string): void {
+ this.last.delete(sessionId)
+ this.lastId.delete(sessionId)
+ this.misses.delete(sessionId)
+ }
+
+ private cancelPending(sessionId: string): void {
+ const poll = this.pendingPolls.get(sessionId)
+ if (poll) poll.cancelled = true
+ const baseline = this.pendingBaselines.get(sessionId)
+ if (baseline) baseline.cancelled = true
+ }
+
+ /**
+ * A poll found no dialog. Announce the close once the miss is CONFIRMED, and only if we announced the
+ * question in the first place — a session nobody was told about has nothing to take back.
+ */
+ private noteGone(sessionId: string): void {
+ this.last.delete(sessionId) // unchanged: a dialog that comes back announces fresh
+ const requestId = this.lastId.get(sessionId)
+ if (!requestId) return
+ const misses = (this.misses.get(sessionId) ?? 0) + 1
+ this.misses.set(sessionId, misses)
+ if (misses < GONE_TICKS) return
+ this.lastId.delete(sessionId)
+ this.misses.delete(sessionId)
+ this.deps.onQuestionGone?.(sessionId, requestId)
+ }
+
+
+ private async tick(sessionId: string): Promise<void> {
+ // A capture can take longer than POLL_MS. Keep one outstanding poll per
+ // session, including a cancelled capture that has not returned yet.
+ if (this.pendingPolls.has(sessionId)) return
+ const session = this.deps.getSession(sessionId)
+ const terminalTarget = session?.agentId || session?.sessionId
+ if (!terminalTarget || session?.active === false || (session?.engine && !pollsQuestions(session.engine))) { this.stop(sessionId); return }
+ // Both of these silently do nothing, which is how a live dialog can sit on the terminal with no trace
+ // in the log. Say it once per transition rather than every 1.5s tick.
+ const blocked = !this.deps.hasDevice() ? 'no device' : this.deps.isDriving?.(sessionId) ? 'driving an answer' : ''
+ if (blocked !== (this.blocked.get(sessionId) ?? '')) {
+ this.blocked.set(sessionId, blocked)
+ console.log(`[question] ${sessionId.slice(0, 8)} watcher ${blocked ? `paused · ${blocked}` : 'polling'}`)
+ }
+ if (blocked) return
+ const pending = { cancelled: false }
+ this.pendingPolls.set(sessionId, pending)
+ const generation = questionCaptureGeneration(session)
+ let capture: string | null
+ let view: PaneView = null
+ try {
+ capture = await this.deps.capture(terminalTarget, CAPTURE_LINES)
+ if (capture !== null) view = await this.deps.readQuestion(session!, capture)
+ } catch {
+ capture = null
+ } finally {
+ if (this.pendingPolls.get(sessionId) === pending) this.pendingPolls.delete(sessionId)
+ }
+ if (pending.cancelled || generation !== questionCaptureGeneration(this.deps.getSession(sessionId))
+ || !this.deps.hasDevice() || this.deps.isDriving?.(sessionId)) return
+ if (capture === null) {
+ // Unavailable is not an empty pane, and breaks a run of confirmed misses.
+ this.misses.delete(sessionId)
+ return
+ }
+ if (view?.kind === 'question' && view.partial) {
+ // Scrolled so its top is out of the pane: still open, so the client showing it keeps showing it —
+ // but there is no question text to announce, and the rows in view are whichever the scroll left.
+ this.misses.delete(sessionId)
+ return
+ }
+ if (!view || view.kind !== 'question' || !view.question || view.rows.length === 0) {
+ // Dialog closed, or moved to review. Either way it is no longer waiting on anybody, so the clients
+ // showing it are told to stop — see noteGone for why this is not announced on the first miss.
+ this.noteGone(sessionId)
+ return
+ }
+ this.misses.delete(sessionId) // a dialog on screen ends any run of misses
+ const fingerprint = fingerprintOf(view)
+ // Already on the pane before this turn started → it belongs to whatever came
+ // before, and has been answered. Say nothing until it changes or leaves.
+ if (this.preTurn.get(sessionId) === fingerprint) return
+ this.preTurn.delete(sessionId)
+ if (this.last.get(sessionId) === fingerprint) return
+ this.last.set(sessionId, fingerprint)
+
+ // A pane-derived question has no tool_use id. The key only has to round-trip through the device and
+ // back (the answer is keyed into the pane, not matched to a tool call), so the question's own text
+ // serves as both — and the device dedups a repeated push by this id. The answer brings the id back,
+ // and AskQuestionController recomputes it off the live pane before typing: a different id there
+ // means a different dialog, and the answer is refused (STALE_QUESTION) instead of keyed into it.
+ const requestId = questionRequestId(sessionId, view)
+ this.lastId.set(sessionId, requestId)
+ const baseline = this.pendingBaselines.get(sessionId)
+ if (baseline) baseline.cancelled = true
+ this.deps.onQuestion(sessionId, requestId, [{
+ key: view.question,
+ q: view.question,
+ options: view.rows.map((r) => r.label),
+ multi: view.multi,
+ ...(view.typeRow && !view.multi && !view.permission ? { canText: true } : {}),
+ }], { permission: isApprovalDialog(view), dialog: view.dialog ?? view.question })
+
+ }
+}
+
+/** Snapshot scalar values before awaiting I/O: registry records can change in
+ * place when an engine exits, moves, or is replaced in the same pane. */
+function questionCaptureGeneration(session: RegisteredSession | undefined): string {
+ return session ? JSON.stringify([session.agentId, session.sessionId, session.engine, session.active,
+ session.tmuxPane, session.primaryRuntimeKey, session.runtimes, session.processIdentity]) : ''
+}
+
+/**
+ * What makes two captures the SAME dialog: its words, its options, its arity — and, when the parser kept
+ * the whole dialog, what it says below its first line (a command that differs only on its second line is
+ * another prompt).
+ *
+ * ⚠️ NOT the raw `dialog`. The id is recomputed every 1.5s poll and again at the moment an answer is typed,
+ * so it may only change when the QUESTION does. The raw dialog changes on its own: Hermes and Muse paint a
+ * live timer inside it (`(01m30s · ↓ 82 tok)`, `(21s · esc to interrupt)`), and every engine moves its
+ * `❯`/`›`/`>` cursor and ticks its `[✔]` boxes in place. Hashed raw, a question was re-announced as new on
+ * every poll — the needs-you alert, the sound, the dial push, again and again — and every answer from a
+ * dial, a device or the cable was refused as STALE_QUESTION. `dialogSignature` is the dialog with all of
+ * that taken out; the raw text still goes, unchanged, to the pair's floor (isApprovalDialog, pair/sensor).
+ */
+function fingerprintOf(view: QuestionView): string {
+ const base = `${view.question}|${view.rows.map((r) => r.label).join('|')}|${view.multi}`
+ return view.dialog === undefined ? base : `${base}|${dialogSignature(view.dialog)}`
+}
+
+// A live status group: an elapsed time (`21s`, `30.5s`, `01m30s`, `1h02m`), a token counter (`↓ 82 tok`,
+// `1.2k tokens`) or `esc to interrupt` inside one pair of parentheses. Units hug their digits, as every
+// engine paints them, so `(see 2 files)` or `(tokens.json)` is never mistaken for one.
+const TIMER_GROUP = String.raw`\([^()\n]*?(?:\b\d+(?:\.\d+)?(?:ms|s|m|h)\b|\b\d+m\d+s\b|\b\d+h\d+m\b|\d+(?:\.\d+)?k?\s*tok(?:en)?s?\b|esc to interrupt)[^()\n]*\)`
+const TIMER_GROUP_RE = new RegExp(TIMER_GROUP, 'i')
+const TIMER_GROUPS_RE = new RegExp(TIMER_GROUP, 'gi')
+// The same, outside parentheses: `↓ 82 tokens · esc to interrupt`.
+const STATUS_BITS_RE = /[↑↓]\s*\d+(?:\.\d+)?k?\s*tok(?:en)?s?\b|\besc to interrupt\b/i
+// A bare elapsed time, with no parentheses and no ` · ` to mark it: a status line that is only a word or
+// three and a duration — a verb in -ing/-ed (`waiting 3s`, `thinking 4s`, `Churned for 4s`) or anything
+// trailing off in an ellipsis (`Waiting… 12s`, `Fetch Bitcoin price… 1m33s`). A command's own number stays:
+// `sleep 30s` and `retry after 30s` are neither.
+const DURATION = String.raw`(?:\d+h\d+m(?:\d+s)?|\d+m\d+s|\d+(?:\.\d+)?m?s)`
+const WORDS = String.raw`(?:\p{L}[\p{L}'’-]*\s+){0,2}\p{L}[\p{L}'’-]*`
+const BARE_TIMER_LINE_RE = new RegExp(String.raw`^${WORDS}(?:(?<=ing|ed)(?:\s+for)?\s+|\s*(?:…|\.{3})\s*)${DURATION}$`, 'iu')
+// …and one hung off the end of a longer line, after an ellipsis or a column gap: grok's right-aligned
+// `Waiting on answers for Which color should I report? 4.2s`. Only the time goes.
+const TRAILING_TIMER_RE = new RegExp(String.raw`(?:(?<=…|\.{3})\s*|\s{2,})${DURATION}$`, 'u')
+// Codex's cursor readout under its request_user_input rows: `option 2/4 | tab to add notes`.
+const CURSOR_READOUT_RE = /^option\s+\d+\s*\/\s*\d+\b/i
+// Whatever leads a line and moves on its own: a cursor (`❯ › > ▶`), a spinner frame (braille, `✻`, `◐`,
+// Muse's `◇`/`◆`) or a box/tab state (`☐ ☒ ✔ ○ ● ◉`).
+const LEAD_MARKS_RE = /^(?:[❯›>▶►▸➤\u2800-\u28ff✻✽✶✳✢✺✹✸✷◐◓◑◒◴◵◶◷◇◆☐☑☒✔✓✗✘○◯●◉◎]\s*)+/u
+// A row's own state right after its number, or at the start of an unnumbered row: `[ ]`, `[✔]`, `(•)`, `◉`.
+const ROW_STATE_RE = /^(\d+[.)]\s+|)(?:\[[^\]\n]?\]|\([^)\n]?\)|[☐☑☒✔✓○◯●◉◎])\s*/u
+// Box drawing: frames and rules redraw to the pane's width.
+const BOX_RE = /[\u2500-\u257f]+/g
+
+/**
+ * The dialog as a person reads it, with nothing that changes while it waits: status lines (a live timer,
+ * bare or in parentheses, a token counter, `esc to interrupt`) and Codex's cursor readout dropped; cursor marks, spinner frames
+ * and checkbox/radio state stripped; frames and whitespace collapsed. Every word of the prompt stays —
+ * two commands that differ anywhere are still two signatures.
+ */
+function dialogSignature(dialog: string): string {
+ const out: string[] = []
+ for (const raw of dialog.replace(/\u00a0/g, ' ').split('\n')) {
+ let line = raw.replace(BOX_RE, ' ').trim()
+ if (CURSOR_READOUT_RE.test(line)) continue
+ line = line.replace(LEAD_MARKS_RE, '')
+ const row = /^\d+[.)]\s/.test(line)
+ // A line that carries a live timer is the engine's status line (Hermes' `💻 curl … (01m30s · ↓ 82 tok)`
+ // under its frame, Muse's `◇ Calling tools (21s · esc to interrupt)` above its rule, a bare `waiting 3s`),
+ // not the prompt: the prompt is always painted on lines of its own. A ROW keeps its words; only the
+ // group goes.
+ if (!row && (TIMER_GROUP_RE.test(line) || STATUS_BITS_RE.test(line) || BARE_TIMER_LINE_RE.test(line))) continue
+ if (!row) line = line.replace(TRAILING_TIMER_RE, '')
+ line = line.replace(TIMER_GROUPS_RE, ' ').replace(ROW_STATE_RE, '$1').replace(/\s+/g, ' ').trim()
+ if (line) out.push(line)
+ }
+ return out.join('\n')
+}
+
+function hash(value: string): string {
+ let h = 0x811c9dc5
+ for (let i = 0; i < value.length; i++) h = Math.imul(h ^ value.charCodeAt(i), 0x01000193) >>> 0
+ return h.toString(16).padStart(8, '0')
+}
diff --git a/cli/src/lib/questionPane.ts b/cli/src/lib/questionPane.ts
new file mode 100644
index 000000000..7f4de65a0
--- /dev/null
+++ b/cli/src/lib/questionPane.ts
@@ -0,0 +1,118 @@
+import type { AgentEngine } from '../engines/types.js'
+import type { FoundDialog, PaneView, QuestionView } from '../engines/facets/screen.js'
+import { locateQuestionPane, parseQuestionPane, parseRow, stripAnsi } from '../engines/kit/questionPane.js'
+import { earlierDialogEnd } from './dialogEnd.js'
+import { locateMuseQuestion } from '../engines/muse/askQuestion.js'
+import { parseAmpQuestionPane } from '../engines/amp/askQuestion.js'
+import { locateKiloQuestion, parseKiloQuestionPane } from '../engines/kilo/askQuestion.js'
+import { parseCursorPermissionPane } from '../engines/cursor/askQuestion.js'
+import { locateDevinPermission, locateDevinQuestion } from '../engines/devin/askQuestion.js'
+import { parseGrokQuestionPane } from '../engines/grok/askQuestion.js'
+import { locateAgyQuestion } from '../engines/agy/askQuestion.js'
+import { withCopilotSubject } from '../engines/copilot/askQuestion.js'
+function permissionView(view: PaneView): PaneView {
+ return view?.kind === 'question' ? { ...view, permission: true } : view
+}
+
+function asPermission(found: FoundDialog | null): FoundDialog | null {
+ return found && { ...found, view: permissionView(found.view) as QuestionView }
+}
+
+/**
+ * The LOWEST dialog any reader found: the live one. A pane keeps an answered dialog in its scrollback, and
+ * an engine with two readers (a question and an approval, say) must never let the first reader's hit on
+ * that one shadow the second reader's hit on the dialog under it: devin's approval under its answered
+ * question, muse's approval under a question, opencode's question under an approval. A tie is one dialog
+ * read twice, and the earlier reader, the engine's own, keeps it.
+ */
+function lowest(...found: Array<FoundDialog | null>): PaneView {
+ let best: FoundDialog | null = null
+ for (const f of found) if (f && (!best || f.at > best.at)) best = f
+ return best?.view ?? null
+}
+
+/**
+ * The dialog on screen, read the way `engine` paints it: the LAST one on the pane, and nothing of it from
+ * above an earlier dialog's end (`dialogEnd.ts`).
+ *
+ * Claude and Command Code share one shape (see parseQuestionPane); devin draws a different one and gets
+ * its own parser rather than more branches in here.
+ */
+export function parseEngineQuestionPane(engine: AgentEngine, capture: string): PaneView {
+ if (engine === 'claude' || engine === 'codex') throw new Error('Native screen reader must be injected')
+ // Devin's two dialogs are told apart by one word in the footer (`↵ select` vs `↵ confirm`), so they can
+ // never both match the same dialog; but an answered one of either kind can sit above the live one.
+ if (engine === 'devin') return lowest(locateDevinQuestion(capture), asPermission(locateDevinPermission(capture)))
+ // Cursor has no ask-the-user tool, so its permission prompt is the ONLY dialog it ever draws — and it
+ // numbers nothing, stating each row's key in the row instead.
+ if (engine === 'cursor') return permissionView(parseCursorPermissionPane(capture))
+ // Muse pairs each option with a description on the same line and floats a live Preview box above the
+ // rows — both confuse the shared parser, so it reads its own. Its PERMISSION prompt is a different
+ // dialog entirely (`Would you like to allow this network access?` over `1. Yes, proceed (y)` rows under
+ // a `Press enter to confirm` footer, `__fixtures__/permission-muse.txt`) and that one the shared parser
+ // reads exactly, so it is read by both and the lower wins.
+ if (engine === 'muse') return lowest(locateMuseQuestion(capture), locateQuestionPane(capture))
+ // Amp's is a permission prompt with unnumbered rows — nothing the shared parser can anchor on.
+ if (engine === 'amp') return permissionView(parseAmpQuestionPane(capture))
+ // Kilo's is the same kind of prompt but laid out HORIZONTALLY, sharing its line with the key hints —
+ // it is a fork of opencode that did not keep opencode's dialog.
+ if (engine === 'kilo') return permissionView(parseKiloQuestionPane(capture))
+ // Grok tells its approval from its questionnaire itself, by the live dialog's own footer.
+ if (engine === 'grok') return parseGrokQuestionPane(capture)
+ // agy's ask-the-user dialog anchors on `Question N/M:` under an `↑/↓ Navigate` footer, which the
+ // shared parser cannot see. Its PERMISSION prompt is numbered rows under `Do you want to proceed?`
+ // and the shared parser reads that one exactly, so it is read by both and the lower wins.
+ if (engine === 'agy') return lowest(locateAgyQuestion(capture), locateQuestionPane(capture))
+ // Hermes, OpenCode and Copilot paint the dialog inside a box; peel the border and the shared parser
+ // fits. Measured on Copilot: framed it returns null, unframed it reads the question, the three
+ // options AND spots `4. Other (type your answer)` as the free-text row rather than an option.
+ if (engine === 'hermes') return parseQuestionPane(unframe(capture))
+ // Copilot boxes its dialog the same way, but names the SUBJECT of a permission prompt above the
+ // question — "attempting to access the following URL:" over a boxed value. Without it the device
+ // shows "Do you want to allow this access?" and a bare "Yes", with nothing to judge.
+ if (engine === 'copilot') {
+ const found = locateQuestionPane(unframe(capture))
+ return found ? withCopilotSubject(found.view, capture, found.at) : null
+ }
+ if (engine === 'opencode') {
+ // OpenCode's PERMISSION prompt is the horizontal one kilo inherited from it — same `△ Permission
+ // required` title, same `⇆ select · enter confirm` footer, same unnumbered rows. Measured: the live
+ // capture in `permission-opencode.txt` parses through kilo's parser unchanged, so it is shared rather
+ // than copied. Listed FIRST because that dialog numbers nothing: on its own dialog the shared parser
+ // still matches the `enter confirm` footer and walks up into whatever numbered rows are above, and
+ // the tie goes to kilo's reader.
+ const plain = unframe(capture)
+ return lowest(asPermission(locateKiloQuestion(capture)), locateOpencodeReview(plain), locateQuestionPane(plain))
+ }
+ return parseQuestionPane(capture)
+}
+
+function locateOpencodeReview(plain: string): FoundDialog | null {
+ const lines = stripAnsi(plain).split('\n')
+ const footer = lines.findLastIndex((l) => /enter\s+submit/i.test(l))
+ if (footer < 0) return null
+ const floor = earlierDialogEnd(lines, footer, 14)
+ let sawReview = false
+ for (let i = footer - 1; i > floor && footer - i <= 14; i--) {
+ if (parseRow(lines[i])) return null // rows above ⇒ still a question, not the review
+ if (/^\s*review\s*$/i.test(lines[i])) { sawReview = true; break }
+ }
+ return sawReview ? { view: { kind: 'review', submitRow: 'Enter' }, at: footer } : null
+}
+
+function unframe(capture: string): string {
+ // stripAnsi FIRST: on a real capture the border is preceded by SGR codes, so matching `^│` against the
+ // raw text silently never fired — hermes read as "no dialog open" with the dialog plainly on screen.
+ return stripAnsi(capture)
+ .split('\n')
+ .map((line) => {
+ const left = line.replace(/^(\s*)[│┃|]\s?/, '$1')
+ // An unanchored whitespace regex retries at every column on blank rows.
+ // Inspect the last non-space character once; keep unframed lines intact.
+ const right = left.trimEnd()
+ return /[│┃|]$/.test(right) ? right.slice(0, -1).trimEnd() : left
+ })
+ .join('\n')
+}
+
+/** How this engine's multi-select dialog is submitted once the boxes are ticked. */
diff --git a/cli/src/lib/runtimeActivity.spec.ts b/cli/src/lib/runtimeActivity.spec.ts
index a684dc912..e90d2dbb3 100644
--- a/cli/src/lib/runtimeActivity.spec.ts
+++ b/cli/src/lib/runtimeActivity.spec.ts
@@ -1,3 +1,4 @@
+import { readInlineScreen } from '../testing/inlineScreen.js'
import { afterEach, describe, expect, it, vi } from 'vitest'
import { CodexActivityReader, RuntimeActivityReader } from './runtimeActivity.js'
import { registry, type RegisteredSession } from './registry.js'
@@ -51,7 +52,7 @@ describe('live terminal fallback', () => {
it('requires changing busy indicators and refuses a frozen footer or prose', async () => {
const row = session()
const capture = vi.fn(async () => '• Working (30s · esc to interrupt)\n› \x1b[2mAsk Codex to do anything\x1b[0m\n? for shortcuts')
- const reader = new RuntimeActivityReader({ codex: async () => 'unknown', capture })
+ const reader = new RuntimeActivityReader({ readScreen: readInlineScreen, codex: async () => 'unknown', capture })
expect(await reader.read(row)).toBe('unknown')
expect(await reader.read(row)).toBe('unknown')
capture.mockResolvedValue('• Working (35s · esc to interrupt)\n› \x1b[2mAsk Codex to do anything\x1b[0m\n? for shortcuts')
@@ -62,7 +63,7 @@ describe('live terminal fallback', () => {
it('recognizes the actual cmd p stopped goal without requiring or generating a transcript end', async () => {
const row = session()
const capture = vi.fn(async () => '■ Conversation interrupted\n› \x1b[2mAsk Codex to do anything\x1b[0m\n GPT-6-Astra max · ~/work Goal stalled (/goal resume)\n? for shortcuts')
- const reader = new RuntimeActivityReader({ codex: async () => 'unknown', capture })
+ const reader = new RuntimeActivityReader({ readScreen: readInlineScreen, codex: async () => 'unknown', capture })
expect(await reader.read(row)).toBe('unknown')
expect(await reader.read(row)).toBe('idle')
row.processIdentity = { ...row.processIdentity!, startMarker: 'replacement' }
@@ -73,9 +74,9 @@ describe('live terminal fallback', () => {
it('prefers structured status and leaves unsupported engines unknown', async () => {
const row = session(); const capture = vi.fn(async () => null)
const codex = vi.fn(async () => 'working' as const)
- const reader = new RuntimeActivityReader({ codex, capture })
+ const reader = new RuntimeActivityReader({ readScreen: readInlineScreen, codex, capture })
expect(await reader.read(row)).toBe('working'); expect(capture).not.toHaveBeenCalled()
- const unsupported = new RuntimeActivityReader({ codex: async () => 'unknown', capture })
+ const unsupported = new RuntimeActivityReader({ readScreen: readInlineScreen, codex: async () => 'unknown', capture })
row.engine = 'pi'; expect(await unsupported.read(row)).toBe('unknown'); expect(capture).not.toHaveBeenCalled()
})
})
diff --git a/cli/src/lib/runtimeActivity.ts b/cli/src/lib/runtimeActivity.ts
index e9e6ac8d4..b28406afc 100644
--- a/cli/src/lib/runtimeActivity.ts
+++ b/cli/src/lib/runtimeActivity.ts
@@ -1,8 +1,7 @@
import { basename } from 'node:path'
import { sessionCodexHome } from './engineHomes.js'
-import { terminalActivityReading } from './terminalActivity.js'
+import type { ScreenReader } from './screenReader.js'
import { connectCodexControl, type CodexControl } from './codexSessionLifecycle.js'
-import { codexStoppedGoal } from './codexTurnRecovery.js'
import { argvTokens, processRows, type ProcessRow } from './tmux.js'
import { sameProcessIdentity } from './terminalRuntime.js'
import type { RegisteredSession } from './registry.js'
@@ -63,6 +62,7 @@ export class CodexActivityReader {
export class RuntimeActivityReader {
private readonly footers = new Map<string, { runtime: string; indicator: string | null; stopped: boolean }>()
constructor(private readonly deps: {
+ readScreen: ScreenReader
codex(session: RegisteredSession): Promise<ActivityState>
capture(session: RegisteredSession): Promise<string | null>
}) {}
@@ -74,8 +74,10 @@ export class RuntimeActivityReader {
const screen = await this.deps.capture(session)
const runtime = activityRuntimeKey(session)
const previous = this.footers.get(session.sessionId)
- const indicator = terminalActivityReading(session.engine, screen)?.indicator ?? null
- const stopped = session.engine === 'codex' && codexStoppedGoal(screen)
+ const reading = await this.deps.readScreen(session, screen)
+ if (!reading) { this.footers.delete(session.sessionId); return 'unknown' }
+ const indicator = reading.activity?.indicator ?? null
+ const stopped = reading.stoppedGoal
this.footers.set(session.sessionId, { runtime, indicator, stopped })
if (previous?.runtime !== runtime) return 'unknown'
if (stopped && previous.stopped) return 'idle'
diff --git a/cli/src/lib/runtimeControl.ts b/cli/src/lib/runtimeControl.ts
new file mode 100644
index 000000000..8446d5a00
--- /dev/null
+++ b/cli/src/lib/runtimeControl.ts
@@ -0,0 +1,892 @@
+import type { ScreenReader } from './screenReader.js'
+import { stripAnsi } from '../engines/kit/pane.js'
+export type { PaneInspection, PaneTakeover } from '../engines/facets/screen.js'
+import { parseRuntimeProfile } from './runtimeProfileWire.js'
+import type { RegisteredSession } from './registry.js'
+import { devinModelCommandResult } from '../engines/devin/runtimeProfile.js'
+import { countCommandcodeRefusals } from '../engines/commandcode/runtimeProfile.js'
+import { parseHermesPickerPage } from '../engines/hermes/runtimeProfile.js'
+import { parsePiFooterProfile, parsePiThinkingSelection, piThinkingSteps } from '../engines/pi/runtimeProfile.js'
+import {
+ countOpencodePickers,
+ opencodeRowMatches,
+ opencodeRowNamesModel,
+ parseOpencodePickerRows,
+ type OpencodeModelTarget,
+ type OpencodePickerRow,
+} from '../engines/opencode/runtimeProfile.js'
+import {
+ chooseCodexRow,
+ codexDigitPressable,
+ codexEffortRows,
+ parseCodexPicker,
+ sameCodexPicker,
+ type CodexCatalogModel,
+ type CodexPicker,
+} from '../engines/codex/modelPicker.js'
+import {
+ type CursorModelTarget,
+ type RuntimeProfile,
+ type RuntimeProfileManager,
+} from './runtimeProfile.js'
+
+const COMMAND_CONFIRM_MS = 8_000
+const PICKER_OPEN_MS = 3_000
+const PICKER_STEP_MS = 2_000
+/**
+ * How far apart the two reads of a Codex model list are, which must agree before a digit is pressed on
+ * it (see reachCodexEfforts): long enough for a redraw in flight to land, short beside the seconds a
+ * switch takes to confirm.
+ */
+const CODEX_SETTLE_MS = 250
+/** Hard bound on ladder keystrokes — twice pi's seven levels, so a desynchronised walk still terminates. */
+const PI_LADDER_MAX_STEPS = 14
+/** Hermes' longest picker page is a provider's model list; twice its size still terminates. */
+const HERMES_PAGE_MAX_STEPS = 60
+/** "Anthropic (13 models)" names provider key `anthropic`; "GitHub Copilot (17 models)" names `copilot`. */
+function hermesProviderMatches(row: string, provider: string): boolean {
+ const name = row.replace(/\s*\([^)]*\)\s*$/, '').trim().toLowerCase()
+ const key = provider.toLowerCase()
+ return name === key || name.replace(/[^a-z0-9]/g, '').includes(key.replace(/[^a-z0-9]/g, ''))
+}
+
+export type RuntimeProfileErrorCode =
+ | 'AGENT_NOT_FOUND'
+ | 'INVALID_RUNTIME_PROFILE'
+ | 'BUSY'
+ | 'UNSUPPORTED_CLI_VERSION'
+ | 'MODEL_UNAVAILABLE'
+ | 'EFFORT_UNSUPPORTED'
+ | 'PLAN_SCOPE_AMBIGUOUS'
+ | 'CONFIRM_TIMEOUT'
+ | 'TMUX_FAILED'
+
+export class RuntimeProfileControlError extends Error {
+ constructor(readonly code: RuntimeProfileErrorCode) {
+ super(code)
+ }
+}
+
+export interface RuntimeProfileControllerDeps {
+ readScreen: ScreenReader
+ manager: Pick<RuntimeProfileManager, 'getState' | 'selectedModel' | 'codexCatalog' | 'modelsForSession'
+ | 'beginControl' | 'finishControl' | 'cancelControl' | 'confirmEffort' | 'waitForProfile' | 'waitForModel'
+ | 'devinTarget' | 'opencodeCatalog' | 'commandcodeTarget' | 'ingestConfig' | 'hermesTarget' | 'cursorTarget' | 'confirmControlProfile'> & {
+ supportsControl(session: RegisteredSession): boolean | Promise<boolean>
+ effortAllowed(session: RegisteredSession, model: string, effort: string, listed: readonly string[] | null): boolean | Promise<boolean>
+ ingestPane(session: RegisteredSession, text: string, silent?: boolean): boolean | Promise<boolean>
+ }
+ getSession: (sessionId: string) => RegisteredSession | undefined
+ validateRuntime: (session: RegisteredSession) => Promise<boolean>
+ capture: (terminalTarget: string, historyLines?: number) => Promise<string | null>
+ sendText: (terminalTarget: string, text: string) => Promise<boolean>
+ /** Type without submitting. */
+ sendLiteral: (terminalTarget: string, text: string) => Promise<boolean>
+ sendKey: (terminalTarget: string, key: string) => Promise<boolean>
+ acquireInput: (sessionId: string) => (() => void) | null
+}
+
+export interface CursorParameterRow {
+ kind: 'context' | 'reasoning' | 'fast' | 'thinking'
+ value: string
+ selected: boolean
+ cursor: boolean
+ index: number
+}
+
+export function parseCursorModelPicker(capture: string): { selectedFamily: string | null } | null {
+ const lines = stripAnsi(capture).split('\n')
+ if (!lines.some((line) => /Available models|Models matching/i.test(line)) || !lines.some((line) => /Tab to edit/i.test(line))) {
+ return null
+ }
+ const selected = lines.findLast((line) => /^\s*→\s+/.test(line))
+ if (!selected) return { selectedFamily: null }
+ const value = selected.replace(/^\s*→\s+/, '').replace(/\s+\(Tab to modify\)\s*$/i, '')
+ return { selectedFamily: value.split(/\s{2,}/)[0]?.trim() || null }
+}
+
+export function parseCursorParameterRows(capture: string): CursorParameterRow[] | null {
+ const lines = stripAnsi(capture).split('\n')
+ if (!lines.some((line) => /— Edit Parameters/i.test(line))) return null
+ const rows: CursorParameterRow[] = []
+ let section: 'context' | 'reasoning' | null = null
+ for (const raw of lines) {
+ const line = raw.replace(/\u00a0/g, ' ')
+ if (/^\s*Context\s*$/.test(line)) { section = 'context'; continue }
+ if (/^\s*Reasoning\s*$/.test(line)) { section = 'reasoning'; continue }
+ const choice = /^\s*(→)?\s*([●○◉◯])\s+(.+?)(?:\s+✓)?\s*$/.exec(line)
+ if (!choice) continue
+ const label = choice[3].trim()
+ let kind: CursorParameterRow['kind']
+ let value: string
+ if (/^Fast$/i.test(label)) {
+ kind = 'fast'
+ value = 'true'
+ } else if (/^Thinking$/i.test(label)) {
+ kind = 'thinking'
+ value = 'true'
+ } else if (section === 'context') {
+ kind = 'context'
+ value = label.toLowerCase()
+ } else if (section === 'reasoning') {
+ kind = 'reasoning'
+ value = label.toLowerCase().replace(/\s+/g, '') === 'extrahigh'
+ ? 'xhigh'
+ : label.toLowerCase()
+ } else {
+ continue
+ }
+ rows.push({
+ kind,
+ value,
+ selected: choice[2] === '●' || choice[2] === '◉' || /✓\s*$/.test(line),
+ cursor: !!choice[1],
+ index: rows.length,
+ })
+ }
+ return rows.length ? rows : null
+}
+
+/** The lists a model is chosen from, as against the reasoning screens that follow them. */
+function isCodexList(picker: CodexPicker): boolean {
+ return picker.kind === 'quick' || picker.kind === 'models' || picker.kind === 'reserve'
+}
+
+function sleep(ms: number): Promise<void> {
+ return new Promise((resolve) => setTimeout(resolve, ms))
+}
+
+export class RuntimeProfileController {
+ constructor(private readonly deps: RuntimeProfileControllerDeps) {}
+
+ async setProfile(sessionId: string, encoded: unknown): Promise<void> {
+ const registeredSession = this.deps.getSession(sessionId)
+ if (!registeredSession) throw new RuntimeProfileControlError('AGENT_NOT_FOUND')
+ const observed = this.deps.manager.getState(registeredSession.sessionId)
+ const session: RegisteredSession = registeredSession.cliVersion || !observed.cliVersion
+ ? registeredSession
+ : { ...registeredSession, cliVersion: observed.cliVersion }
+ const target = parseRuntimeProfile(encoded)
+ // Either id identifies the agent: the encoded profile now carries the agent id, but one minted
+ // before the id cutover (or by a client that still knows the session) must keep working.
+ if (!target || (target.sessionId !== session.agentId && target.sessionId !== session.sessionId) || target.engine !== session.engine) {
+ throw new RuntimeProfileControlError('INVALID_RUNTIME_PROFILE')
+ }
+ const current = parseRuntimeProfile(this.deps.manager.selectedModel(session))
+ if (current?.id === target.id) return
+ if (!await this.deps.manager.supportsControl(session)) {
+ console.warn(`[runtime-profile] unsupported ${session.engine} CLI version ${session.cliVersion ?? 'unknown'} for ${sessionId.slice(0, 8)}`)
+ throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
+ }
+ // Codex's own catalog: which efforts each model takes, and the names its picker shows them by.
+ const codexCatalog = session.engine === 'codex' ? await this.deps.manager.codexCatalog(session) : []
+ if (session.engine === 'codex') {
+ const listed = codexCatalog.find((entry) => entry.slug === target.model)?.efforts ?? null
+ if (!await this.deps.manager.effortAllowed(session, target.model, target.effort, listed)) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
+ }
+ const options = await this.deps.manager.modelsForSession(session)
+ if (!options.some((option) => option.id === target.id)) {
+ const sameModel = options.some((option) => parseRuntimeProfile(option.id)?.model === target.model)
+ throw new RuntimeProfileControlError(sameModel ? 'EFFORT_UNSUPPORTED' : 'MODEL_UNAVAILABLE')
+ }
+ if (session.engine === 'codex' && observed.mode === 'plan') throw new RuntimeProfileControlError('PLAN_SCOPE_AMBIGUOUS')
+ const release = this.deps.acquireInput(session.agentId)
+ if (!release) throw new RuntimeProfileControlError('BUSY')
+ let controlStarted = false
+ let pickerOpen = false
+ try {
+ if (!await this.deps.validateRuntime(session)) throw new RuntimeProfileControlError('TMUX_FAILED')
+ const capture = await this.deps.capture(session.agentId, 100)
+ if (!capture) throw new RuntimeProfileControlError('TMUX_FAILED')
+ const inspection = (await this.deps.readScreen(session, capture))?.pane
+ if (!inspection) throw new RuntimeProfileControlError('BUSY')
+ if (session.engine === 'codex' && inspection.plan) throw new RuntimeProfileControlError('PLAN_SCOPE_AMBIGUOUS')
+ if (!inspection.idle) throw new RuntimeProfileControlError('BUSY')
+ if (!this.deps.manager.beginControl(session, target)) throw new RuntimeProfileControlError('BUSY')
+ controlStarted = true
+ if (session.engine === 'claude') {
+ await this.setClaude(session, target, current, options)
+ } else if (session.engine === 'codex') {
+ pickerOpen = true
+ await this.setCodex(session, target, codexCatalog)
+ pickerOpen = false
+ } else if (session.engine === 'devin') {
+ // No picker is ever opened — `/model <id>` is a single command — so there is nothing to Escape out
+ // of if this throws.
+ await this.setDevin(session, target)
+ } else if (session.engine === 'pi') {
+ pickerOpen = true
+ await this.setPi(session, target, current)
+ pickerOpen = false
+ } else if (session.engine === 'opencode') {
+ pickerOpen = true
+ await this.setOpencode(session, target)
+ pickerOpen = false
+ } else if (session.engine === 'hermes') {
+ pickerOpen = true
+ await this.setHermes(session, target)
+ pickerOpen = false
+ } else if (session.engine === 'commandcode') {
+ // Two plain commands, no dialog — nothing to Escape out of if either throws.
+ await this.setCommandcode(session, target, current)
+ } else {
+ pickerOpen = true
+ await this.setCursor(session, target)
+ pickerOpen = false
+ }
+ this.deps.manager.finishControl(session)
+ controlStarted = false
+ } catch (error) {
+ if (pickerOpen) {
+ for (let attempt = 0; attempt < 3; attempt++) {
+ await this.deps.sendKey(session.agentId, 'Escape').catch(() => false)
+ await sleep(100)
+ const next = await this.deps.capture(session.agentId, 100).catch(() => null)
+ if (!next || !(await this.deps.readScreen(session, next))?.pane.dialog) break
+ }
+ }
+ // By the engine session, as beginControl keyed it. `sessionId` is whichever id the request named,
+ // and the apps name the agent: cancelling under that left the control in place, and every later
+ // switch of the agent answered BUSY until the daemon restarted (found by e2e/models.e2e.ts, a
+ // refused switch followed by one that should have gone through).
+ if (controlStarted) this.deps.manager.cancelControl(session.sessionId)
+ // A failed switch used to leave NOTHING in the log — the picker just flashed open and shut on the
+ // user's terminal and the device said nothing useful. Name the session, engine and reason.
+ const code = error instanceof RuntimeProfileControlError ? error.code : 'TMUX_FAILED'
+ console.warn(`[runtime-profile] ${sessionId.slice(0, 8)} ${session.engine} set ${target.model}@${target.effort} failed: ${code}`)
+ if (error instanceof RuntimeProfileControlError) throw error
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ } finally {
+ release()
+ }
+ }
+
+ private async setClaude(
+ session: RegisteredSession,
+ target: RuntimeProfile,
+ current: RuntimeProfile | null,
+ options: Array<{ id: string }>,
+ ): Promise<void> {
+ if (current?.model !== target.model) {
+ if (!await this.deps.sendText(session.agentId, `/model ${target.model}`)) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ if (!await this.waitClaudeModel(session)) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+ }
+
+ if (current?.effort === target.effort) {
+ this.deps.manager.confirmEffort(session.sessionId, target.effort)
+ return
+ }
+
+ const hasExplicitEffort = options.some((option) => {
+ const profile = parseRuntimeProfile(option.id)
+ return profile?.model === target.model && profile.effort !== 'auto'
+ })
+ if (!hasExplicitEffort && target.effort === 'auto') {
+ this.deps.manager.confirmEffort(session.sessionId, 'auto')
+ return
+ }
+ if (!await this.deps.sendText(session.agentId, `/effort ${target.effort}`)) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ if (!await this.deps.manager.waitForProfile(session.sessionId, COMMAND_CONFIRM_MS)) {
+ await this.deps.sendKey(session.agentId, 'Enter')
+ if (!await this.deps.manager.waitForProfile(session.sessionId, 2_000)) {
+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+ }
+ }
+ }
+
+ private async waitClaudeModel(session: RegisteredSession): Promise<boolean> {
+ if (await this.deps.manager.waitForModel(session.sessionId, 1_200)) return true
+ const capture = await this.deps.capture(session.agentId, 80)
+ if (capture && /switch model|change model|continue.*model|re-read.*history/i.test(stripAnsi(capture))) {
+ await this.deps.sendKey(session.agentId, 'Enter')
+ } else {
+ // Claude and Codex can swallow the first Enter immediately after a terminal literal write.
+ await this.deps.sendKey(session.agentId, 'Enter')
+ }
+ return this.deps.manager.waitForModel(session.sessionId, COMMAND_CONFIRM_MS)
+ }
+
+ /**
+ * Devin takes the whole profile in one command: `/model <id>` accepts the exact id from
+ * `devin models list` (effort included), and answers in the pane — `✓ Model set to <name>` or
+ * `✗ Model not available`. That explicit answer is why devin needs no picker driving at all, and why an
+ * unavailable model (most of them, on a free plan) fails fast instead of timing out.
+ */
+ private async setDevin(session: RegisteredSession, target: RuntimeProfile): Promise<void> {
+ const devin = this.deps.manager.devinTarget(session.sessionId, target.id)
+ if (!devin) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
+ if (!await this.deps.sendText(session.agentId, `/model ${devin.id}`)) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ const answered = await this.waitPane(
+ session.agentId,
+ (value) => devinModelCommandResult(stripAnsi(value)) !== null,
+ COMMAND_CONFIRM_MS,
+ )
+ if (!answered) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+ if (devinModelCommandResult(stripAnsi(answered)) === 'unavailable') {
+ throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
+ }
+ // The acknowledgement line is not the state — the footer is, and only `ingestPane` reads it. Nothing
+ // else feeds the manager during a control, so the wait has to do the feeding itself (same shape as
+ // setCursor); waiting on `waitForProfile` alone timed out on a switch that had already landed.
+ if (!await this.waitObservedProfile(session, target)) {
+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+ }
+ }
+
+ /**
+ * Pi splits the axes: `/model <provider/model>` takes a target directly, while the thinking level lives
+ * only in `/settings` → "Thinking Level", a fixed ladder walked with the arrow keys. Both were driven by
+ * hand first; the ladder's order is what makes the arrow count deterministic (see PI_THINKING_LEVELS).
+ */
+ private async setPi(
+ session: RegisteredSession,
+ target: RuntimeProfile,
+ current: RuntimeProfile | null,
+ ): Promise<void> {
+ if (current?.model !== target.model) {
+ if (!await this.deps.sendText(session.agentId, `/model ${target.model}`)) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ if (!await this.deps.manager.waitForModel(session.sessionId, COMMAND_CONFIRM_MS)) {
+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+ }
+ }
+ if (target.effort === 'auto' || current?.effort === target.effort) {
+ this.deps.manager.confirmEffort(session.sessionId, target.effort)
+ return
+ }
+ await this.setPiThinking(session, target)
+ }
+
+ private async setPiThinking(session: RegisteredSession, target: RuntimeProfile): Promise<void> {
+ const effort = target.effort
+ if (!await this.deps.sendText(session.agentId, '/settings')) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ if (!await this.waitPane(session.agentId, (v) => /Type to search/i.test(stripAnsi(v)), PICKER_OPEN_MS)) {
+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+ }
+ // Typing alone is enough: the settings filter opens a row the moment it narrows to one. Submitting
+ // here would land an Enter on the ladder that just opened and pick the level already highlighted —
+ // which is the CURRENT one, so the switch silently did nothing.
+ if (!await this.deps.sendLiteral(session.agentId, 'thinking level')) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ const ladder = await this.waitPane(
+ session.agentId,
+ (v) => /Select reasoning depth/i.test(stripAnsi(v)),
+ PICKER_OPEN_MS,
+ )
+ if (!ladder) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
+
+ // Walk the ladder ONE key at a time, re-reading the cursor after each. Counting the steps up front and
+ // firing them blind is what an open-loop drive looks like, and it is exactly how this went wrong: a
+ // stale first read sent the cursor to `off` and committed it. Re-reading also means a ladder that
+ // scrolls, clamps at an end, or gains a level cannot desynchronise the walk.
+ let selection = parsePiThinkingSelection(stripAnsi(ladder))
+ if (!selection || piThinkingSteps(selection, effort) === null) {
+ throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
+ }
+ for (let guard = 0; selection !== effort && guard < PI_LADDER_MAX_STEPS; guard++) {
+ const steps = piThinkingSteps(selection as string, effort)
+ if (steps === null || steps === 0) break
+ if (!await this.deps.sendKey(session.agentId, steps > 0 ? 'Down' : 'Up')) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ const moved = await this.waitPane(
+ session.agentId,
+ (value) => {
+ const next = parsePiThinkingSelection(stripAnsi(value))
+ return !!next && next !== selection
+ },
+ PICKER_STEP_MS,
+ )
+ // No movement means the cursor is pinned at an end of the ladder — pressing on cannot reach it.
+ if (!moved) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
+ selection = parsePiThinkingSelection(stripAnsi(moved))
+ }
+ if (selection !== effort) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
+ await this.deps.sendKey(session.agentId, 'Enter')
+ // The ladder returns to the settings list; Escape closes it so the pane is idle again.
+ await this.deps.sendKey(session.agentId, 'Escape')
+ if (!await this.waitObservedProfile(session, target)) {
+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+ }
+ }
+
+ /** Poll the pane INTO the manager until it reports the profile we asked for. */
+ private async waitObservedProfile(session: RegisteredSession, target: RuntimeProfile): Promise<boolean> {
+ const confirmed = await this.waitPane(session.agentId, async (value) => {
+ await this.deps.manager.ingestPane(session, value, true)
+ return this.deps.manager.selectedModel(session) === target.id
+ }, COMMAND_CONFIRM_MS)
+ return !!confirmed
+ }
+
+ /**
+ * OpenCode switches through `/models`, a picker with a typed filter. It is driven by NARROWING, never by
+ * arrowing: type the model's words plus its provider's, and press Enter only once exactly one row is
+ * left and that row is the requested model. If the filter leaves two rows (two providers can carry the
+ * same model name) the switch is refused rather than guessed — picking the wrong model silently is worse
+ * than not switching, and blind arrow-driving is what sank the Command Code attempt.
+ */
+ private async setOpencode(session: RegisteredSession, target: RuntimeProfile): Promise<void> {
+ const entry = (await this.deps.manager.opencodeCatalog()).find((item) => item.id === target.model)
+ if (!entry) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
+ await this.driveOpencodePicker(session, entry)
+ if (!await this.waitObservedProfile(session, target)) {
+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+ }
+ }
+
+ /**
+ * `/models`, filter, act only once exactly one row is left.
+ *
+ * Two attempts, because one filter does not fit both renders. With several providers connected a
+ * row carries its provider (`Big Pickle OpenCode Zen`) and the provider-qualified filter is what
+ * tells two same-named models apart; with one provider the row is bare (`Big Pickle`) and that same
+ * filter matches NOTHING. Measured on a live picker: `big pickle opencode` → empty, `big pickle` →
+ * the row.
+ *
+ * The fallback drops only the PROVIDER half, never the "exactly one row" rule — which is what makes
+ * it safe: if one model answers to the name, there is no second provider to confuse it with, and if
+ * two do, the list does not narrow and this refuses rather than guessing.
+ */
+ private async driveOpencodePicker(session: RegisteredSession, entry: OpencodeModelTarget): Promise<void> {
+ if (await this.pickOpencodeRow(session, entry, entry.filter, opencodeRowMatches)) return
+ if (entry.modelFilter === entry.filter) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
+ if (!await this.pickOpencodeRow(session, entry, entry.modelFilter, opencodeRowNamesModel)) {
+ throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
+ }
+ }
+
+ /**
+ * One attempt at the picker: open it, type `filter`, and press Enter only if exactly one row is left
+ * AND `matches` accepts it. Returns false — with the picker CLOSED — when the list did not narrow to
+ * a row this may act on, so the caller can try a different filter on a pane in its resting state.
+ * Throws only for the states no retry can help: tmux refusing input, or the picker never opening.
+ */
+ private async pickOpencodeRow(
+ session: RegisteredSession,
+ entry: OpencodeModelTarget,
+ filter: string,
+ matches: (target: OpencodeModelTarget, row: OpencodePickerRow) => boolean,
+ ): Promise<boolean> {
+ // "Is a picker open?" cannot be asked of a capture that carries scrollback — an EARLIER picker is
+ // still up there, so the check passes before this one opens and the filter would be typed into the
+ // composer and submitted as a message. Count the openings instead and wait for one more.
+ const before = countOpencodePickers(stripAnsi(await this.deps.capture(session.agentId, 100) ?? ''))
+ if (!await this.deps.sendText(session.agentId, '/models')) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ if (!await this.waitPane(
+ session.agentId,
+ (v) => countOpencodePickers(stripAnsi(v)) > before,
+ PICKER_OPEN_MS,
+ )) {
+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+ }
+ if (!await this.deps.sendLiteral(session.agentId, filter)) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ const narrowed = await this.waitPane(
+ session.agentId,
+ (v) => (parseOpencodePickerRows(stripAnsi(v)) ?? []).length === 1,
+ PICKER_STEP_MS,
+ )
+ const rows = narrowed ? parseOpencodePickerRows(stripAnsi(narrowed)) ?? [] : []
+ if (rows.length !== 1 || !matches(entry, rows[0])) {
+ // Leave the pane as it was found. A picker left open with a dead filter in it swallows whatever
+ // the user types next, and the retry below would type its filter on top of this one.
+ await this.deps.sendKey(session.agentId, 'Escape')
+ return false
+ }
+ if (!await this.deps.sendKey(session.agentId, 'Enter')) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ return true
+ }
+
+ /**
+ * Command Code 1.6.0 takes both axes as arguments: `/model <vendor/id>` and `/effort <level>`. Neither
+ * opens a dialog, which is the whole reason switching is back after the 2026-07-30 revert — the version
+ * gate in supportsNativeRuntimeControl keeps older builds, where only an arrow-driven picker existed,
+ * out of this path.
+ *
+ * Effort is per model and the catalog does not say which models have one, so a refusal is expected
+ * traffic: the CLI answers "Reasoning effort not supported for X." and that becomes EFFORT_UNSUPPORTED.
+ */
+ private async setCommandcode(
+ session: RegisteredSession,
+ target: RuntimeProfile,
+ current: RuntimeProfile | null,
+ ): Promise<void> {
+ const entry = this.deps.manager.commandcodeTarget(session.sessionId, target.id)
+ if (!entry) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
+
+ if (current?.model !== target.model) {
+ if (!await this.deps.sendText(session.agentId, `/model ${entry.id}`)) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ if (!await this.waitObservedModel(session, target)) {
+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+ }
+ }
+
+ if (target.effort === 'auto' || current?.effort === target.effort) {
+ this.deps.manager.confirmEffort(session.sessionId, target.effort)
+ return
+ }
+ // Success prints NOTHING and a refusal prints one line, so both have to be watched at once — and the
+ // refusal has to be a NEW one. Matching the text anywhere in the capture reported failure in 25ms off
+ // a refusal still sitting in the scrollback from an earlier model, while the level had in fact applied.
+ const refusalsBefore = countCommandcodeRefusals(stripAnsi(await this.deps.capture(session.agentId, 100) ?? ''))
+ if (!await this.deps.sendText(session.agentId, `/effort ${target.effort}`)) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ const deadline = Date.now() + COMMAND_CONFIRM_MS
+ while (Date.now() < deadline) {
+ // The level is recorded only in the CLI's global config, so confirming it means re-reading that.
+ await this.deps.manager.ingestConfig(session, true).catch(() => false)
+ if (this.deps.manager.selectedModel(session) === target.id) return
+ const capture = stripAnsi(await this.deps.capture(session.agentId, 100) ?? '')
+ if (countCommandcodeRefusals(capture) > refusalsBefore) {
+ throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
+ }
+ await sleep(150)
+ }
+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+ }
+
+ /**
+ * Hermes' `/model` is a two-level picker — provider page, then that provider's models — with no way to
+ * type or jump. It is walked one arrow at a time, re-reading the `❯` cursor after every key, so a page
+ * that scrolls or reorders cannot desynchronise the walk. If the cursor stops moving before it reaches
+ * the target, the target is not on the page and the switch is refused rather than committed blind.
+ *
+ * Only MODELS are switchable; hermes keeps its reasoning effort in config.yaml with no in-session
+ * command, so hermesModels never offers an effort row to get here (see hermesModels).
+ */
+ private async setHermes(session: RegisteredSession, target: RuntimeProfile): Promise<void> {
+ const entry = this.deps.manager.hermesTarget(session.sessionId, target.id)
+ if (!entry) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
+ if (!await this.deps.sendText(session.agentId, '/model')) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ const opened = await this.waitPane(
+ session.agentId,
+ (v) => !!parseHermesPickerPage(stripAnsi(v)),
+ PICKER_OPEN_MS,
+ )
+ if (!opened) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+
+ // Page 1: the provider. Its row reads "Anthropic (13 models)", so match on the leading name.
+ if (entry.provider) {
+ await this.walkHermesPage(session, (row) => hermesProviderMatches(row, entry.provider))
+ if (!await this.waitPane(
+ session.agentId,
+ (v) => (parseHermesPickerPage(stripAnsi(v))?.rows ?? []).some((row) => row === entry.id),
+ PICKER_OPEN_MS,
+ )) {
+ throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
+ }
+ }
+ // Page 2: the model itself, listed by exact id.
+ await this.walkHermesPage(session, (row) => row === entry.id)
+ if (!await this.waitObservedModel(session, target)) {
+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+ }
+ }
+
+ /** Move the `❯` cursor onto the row `wanted` selects, then commit it with Enter. */
+ private async walkHermesPage(
+ session: RegisteredSession,
+ wanted: (row: string) => boolean,
+ ): Promise<void> {
+ for (let guard = 0; guard < HERMES_PAGE_MAX_STEPS; guard++) {
+ const capture = await this.deps.capture(session.agentId, 100)
+ const page = capture ? parseHermesPickerPage(stripAnsi(capture)) : null
+ if (!page || !page.selected) throw new RuntimeProfileControlError('TMUX_FAILED')
+ if (wanted(page.selected)) {
+ if (!await this.deps.sendKey(session.agentId, 'Enter')) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ return
+ }
+ const at = page.rows.indexOf(page.selected)
+ const to = page.rows.findIndex(wanted)
+ if (to < 0) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
+ if (!await this.deps.sendKey(session.agentId, to > at ? 'Down' : 'Up')) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ const moved = await this.waitPane(session.agentId, (v) => {
+ const next = parseHermesPickerPage(stripAnsi(v))
+ return !!next?.selected && next.selected !== page.selected
+ }, PICKER_STEP_MS)
+ if (!moved) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
+ }
+ throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
+ }
+
+ /** Like waitObservedProfile, but only the model half — used where effort is applied separately. */
+ private async waitObservedModel(session: RegisteredSession, target: RuntimeProfile): Promise<boolean> {
+ const confirmed = await this.waitPane(session.agentId, async (value) => {
+ await this.deps.manager.ingestPane(session, value, true)
+ return parseRuntimeProfile(this.deps.manager.selectedModel(session))?.model === target.model
+ }, COMMAND_CONFIRM_MS)
+ return !!confirmed
+ }
+
+ /**
+ * Codex's `/model`: a quick menu of auto presets with an `All models` row (0.145 and later), the full
+ * list (`Select Model and Effort`), then the chosen model's reasoning picker, each row pressed by its
+ * digit, which selects the row and accepts it. How a row is read and chosen, and why, is in
+ * engines/codex/modelPicker.ts. Confirmed, as it always was, by the `thread_settings_applied` record
+ * Codex writes once the choice is applied (RuntimeProfileManager `ingestCodex`).
+ */
+ private async setCodex(session: RegisteredSession, target: RuntimeProfile, catalog: CodexCatalogModel[]): Promise<void> {
+ if (!await this.deps.sendText(session.agentId, '/model')) throw new RuntimeProfileControlError('TMUX_FAILED')
+ let picker = await this.waitCodexPicker(session, () => true, 900)
+ if (!picker) {
+ await this.deps.sendKey(session.agentId, 'Enter')
+ picker = await this.waitCodexPicker(session, () => true, PICKER_OPEN_MS)
+ }
+ if (!picker) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
+ if (!isCodexList(picker)) {
+ // Opened on a reasoning screen: back to the list it was opened from, so the model is chosen too.
+ if (!await this.deps.sendKey(session.agentId, 'Escape')) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
+ if (!await this.waitCodexPicker(session, isCodexList, PICKER_STEP_MS)) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
+ }
+ const reached = await this.reachCodexEfforts(session, target, catalog)
+ if (reached !== 'applied') await this.pickCodexEffort(session, target, reached)
+ if (!await this.deps.manager.waitForProfile(session.sessionId, COMMAND_CONFIRM_MS)) {
+ throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+ }
+ }
+
+ /**
+ * From the list on screen to the target model's reasoning picker, through `All models` when the quick
+ * menu is first; or 'applied' when the model's row applied it at once (a model with one effort).
+ *
+ * Every digit is pressed from a list read twice, CODEX_SETTLE_MS apart, the same both times. Codex
+ * 0.160 draws the list from its cache and redraws it in place when the server answers, and a model
+ * added or reordered then renumbers the rows: a digit read off the first drawing lands on another
+ * model. The reasoning picker that opens names its model in its title, and a title naming any other
+ * model than the row pressed means the list moved anyway, so nothing more is pressed.
+ */
+ private async reachCodexEfforts(
+ session: RegisteredSession,
+ target: RuntimeProfile,
+ catalog: CodexCatalogModel[],
+ ): Promise<CodexPicker | 'applied'> {
+ let reread = false
+ for (let lists = 0; lists < 2; lists++) {
+ const list = await this.settledCodexList(session)
+ if (!list) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
+ let choice = chooseCodexRow(list, target, catalog)
+ if ('error' in choice && choice.error === 'MODEL_UNAVAILABLE' && !reread) {
+ // Codex saves the catalog its picker refreshed from to models_cache.json, so a model or a name
+ // that came with the refresh is there to read now.
+ reread = true
+ catalog = await this.deps.manager.codexCatalog(session)
+ choice = chooseCodexRow(list, target, catalog)
+ }
+ if ('error' in choice) throw new RuntimeProfileControlError(choice.error)
+ if (!await this.deps.sendKey(session.agentId, String(choice.row.number))) throw new RuntimeProfileControlError('TMUX_FAILED')
+ if (choice.opens === 'list') {
+ if (!await this.waitCodexPicker(session, (next) => next.kind === 'models', PICKER_STEP_MS)) {
+ throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
+ }
+ continue
+ }
+ if (choice.opens === 'applied') return 'applied'
+ const next = await this.waitCodexAfterModelRow(session)
+ if (next === 'closed') return 'applied'
+ if (!next) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
+ if (next.model !== choice.row.name) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
+ return next
+ }
+ throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
+ }
+
+ /** The target's effort in its reasoning picker, through Advanced Reasoning for Max and Ultra. */
+ private async pickCodexEffort(session: RegisteredSession, target: RuntimeProfile, picker: CodexPicker): Promise<void> {
+ const rows = codexEffortRows(picker)
+ let row = target.effort === 'auto' ? rows.defaultRow : rows.efforts.get(target.effort) ?? null
+ if (row === null && (target.effort === 'max' || target.effort === 'ultra') && rows.advancedRow !== null
+ && codexDigitPressable(rows.advancedRow)) {
+ if (!await this.deps.sendKey(session.agentId, String(rows.advancedRow))) throw new RuntimeProfileControlError('TMUX_FAILED')
+ const advanced = await this.waitCodexPicker(session, (next) => next.kind === 'advanced', PICKER_STEP_MS)
+ row = advanced ? codexEffortRows(advanced).efforts.get(target.effort) ?? null : null
+ }
+ if (row === null || !codexDigitPressable(row)) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
+ if (!await this.deps.sendKey(session.agentId, String(row))) throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+
+ /** The first Codex picker on the pane that `wanted` takes, polled until `timeoutMs`. */
+ private async waitCodexPicker(
+ session: RegisteredSession,
+ wanted: (picker: CodexPicker) => boolean,
+ timeoutMs: number,
+ ): Promise<CodexPicker | null> {
+ const capture = await this.waitPane(session.agentId, (value) => {
+ const picker = parseCodexPicker(value)
+ return !!picker && wanted(picker)
+ }, timeoutMs)
+ return capture ? parseCodexPicker(capture) : null
+ }
+
+ /** A model list that reads the same twice, CODEX_SETTLE_MS apart (see reachCodexEfforts). */
+ private async settledCodexList(session: RegisteredSession): Promise<CodexPicker | null> {
+ let previous: CodexPicker | null = null
+ const deadline = Date.now() + PICKER_STEP_MS
+ while (Date.now() < deadline) {
+ const capture = await this.deps.capture(session.agentId, 100)
+ const picker = capture ? parseCodexPicker(capture) : null
+ const list = picker && isCodexList(picker) ? picker : null
+ if (list && previous && sameCodexPicker(previous, list)) return list
+ previous = list
+ await sleep(CODEX_SETTLE_MS)
+ }
+ return null
+ }
+
+ /**
+ * After a model's row: its reasoning picker, or 'closed' when the picker went away and the composer
+ * is back, which is a model the catalog did not describe applying its one effort at once.
+ */
+ private async waitCodexAfterModelRow(session: RegisteredSession): Promise<CodexPicker | 'closed' | null> {
+ const deadline = Date.now() + PICKER_STEP_MS
+ while (Date.now() < deadline) {
+ const capture = await this.deps.capture(session.agentId, 100)
+ if (capture) {
+ const picker = parseCodexPicker(capture)
+ if (picker?.kind === 'efforts') return picker
+ if (!picker && (await this.deps.readScreen(session, capture))?.pane.idle) return 'closed'
+ }
+ await sleep(100)
+ }
+ return null
+ }
+
+ private async setCursor(session: RegisteredSession, target: RuntimeProfile): Promise<void> {
+ const cursorTarget = this.deps.manager.cursorTarget(session.sessionId, target.id)
+ if (!cursorTarget) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
+ if (!await this.deps.sendText(session.agentId, `/model ${cursorTarget.familyLabel}`)) {
+ throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+ let capture = await this.waitPane(session.agentId, (value) => !!parseCursorModelPicker(value), 900)
+ if (!capture) {
+ if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')
+ capture = await this.waitPane(session.agentId, (value) => !!parseCursorModelPicker(value), PICKER_OPEN_MS)
+ }
+ const picker = capture ? parseCursorModelPicker(capture) : null
+ if (!picker || picker.selectedFamily?.toLowerCase() !== cursorTarget.familyLabel.toLowerCase()) {
+ throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
+ }
+
+ if (cursorTarget.rawId !== 'auto') {
+ if (!await this.deps.sendKey(session.agentId, 'Tab')) throw new RuntimeProfileControlError('TMUX_FAILED')
+ capture = await this.waitPane(session.agentId, (value) => !!parseCursorParameterRows(value), PICKER_STEP_MS)
+ if (!capture) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
+
+ if (cursorTarget.context) {
+ await this.setCursorParameter(session, 'context', cursorTarget.context, true)
+ }
+ if (cursorTarget.reasoning) {
+ await this.setCursorParameter(session, 'reasoning', cursorTarget.reasoning, true)
+ }
+ await this.setCursorParameter(session, 'thinking', 'true', cursorTarget.thinking ?? false)
+ await this.setCursorParameter(session, 'fast', 'true', cursorTarget.fast ?? false)
+
+ if (!await this.deps.sendKey(session.agentId, 'Escape')) throw new RuntimeProfileControlError('TMUX_FAILED')
+ if (!await this.waitPane(session.agentId, (value) => !!parseCursorModelPicker(value), PICKER_STEP_MS)) {
+ throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
+ }
+ if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')
+ } else {
+ if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')
+ }
+
+ const confirmed = await this.waitPane(session.agentId, async (value) => {
+ await this.deps.manager.ingestPane(session, value, true)
+ return this.deps.manager.selectedModel(session) === target.id
+ || this.cursorFooterMatches(value, cursorTarget, target.effort)
+ }, COMMAND_CONFIRM_MS)
+ if (!confirmed) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+ if (this.deps.manager.selectedModel(session) !== target.id) {
+ this.deps.manager.confirmControlProfile(target)
+ }
+ }
+
+ private async setCursorParameter(
+ session: RegisteredSession,
+ kind: CursorParameterRow['kind'],
+ value: string,
+ selected: boolean,
+ ): Promise<void> {
+ const capture = await this.deps.capture(session.agentId, 100)
+ const rows = capture ? parseCursorParameterRows(capture) : null
+ if (!rows) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
+ const target = rows.find((row) => row.kind === kind && row.value === value)
+ if (!target) {
+ if (!selected && (kind === 'fast' || kind === 'thinking')) return
+ throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
+ }
+ if (target.selected === selected) return
+ const current = rows.find((row) => row.cursor)
+ if (!current) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
+ const direction = target.index > current.index ? 'Down' : 'Up'
+ for (let i = 0; i < Math.abs(target.index - current.index); i++) {
+ if (!await this.deps.sendKey(session.agentId, direction)) throw new RuntimeProfileControlError('TMUX_FAILED')
+ await sleep(120)
+ }
+ if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')
+ const updated = await this.waitPane(session.agentId, (next) => {
+ const nextRows = parseCursorParameterRows(next)
+ return nextRows?.some((row) => row.kind === kind && row.value === value && row.selected === selected) ?? false
+ }, PICKER_STEP_MS)
+ if (!updated) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
+ }
+
+ private cursorFooterMatches(
+ capture: string,
+ target: CursorModelTarget,
+ effort: string | null,
+ ): boolean {
+ const lines = stripAnsi(capture).split('\n').filter((line) => line.trim()).slice(-15)
+ if (target.rawId === 'auto') return lines.some((line) => /^\s*Auto(?:\s*$|\s*[·│])/i.test(line))
+ return lines.some((line) => {
+ const normalized = line.trim().toLowerCase()
+ if (!normalized.startsWith(target.familyLabel.toLowerCase())) return false
+ const actualFast = /\bfast\b/i.test(line)
+ const actualThinking = !/\bno\s+thinking\b/i.test(line) && /\bthinking\b/i.test(line)
+ if (target.fast != null && actualFast !== target.fast) return false
+ if (target.thinking != null && actualThinking !== target.thinking) return false
+ if (target.context && !normalized.includes(target.context)) return false
+
+ const effortMatch = /\b(extra high|none|low|medium|high|max)\b/i.exec(line)
+ const actualEffort = effortMatch?.[1].toLowerCase().replace(/\s+/g, '') === 'extrahigh'
+ ? 'xhigh'
+ : effortMatch?.[1].toLowerCase() ?? null
+ const footerEffort = target.footerEffort === undefined ? effort : target.footerEffort
+ return footerEffort == null ? actualEffort == null : actualEffort === footerEffort
+ })
+ }
+
+ private async waitPane(terminalTarget: string, predicate: (capture: string) => boolean | Promise<boolean>, timeoutMs: number): Promise<string | null> {
+ const deadline = Date.now() + timeoutMs
+ while (Date.now() < deadline) {
+ const capture = await this.deps.capture(terminalTarget, 100)
+ if (capture && await predicate(capture)) return capture
+ await sleep(100)
+ }
+ return null
+ }
+}
diff --git a/cli/src/lib/runtimeProfileController.ts b/cli/src/lib/runtimeProfileController.ts
index cd56a49db..73b0e72c8 100644
--- a/cli/src/lib/runtimeProfileController.ts
+++ b/cli/src/lib/runtimeProfileController.ts
@@ -1,1442 +1,10 @@
-import { parseRuntimeProfile } from './runtimeProfileWire.js'
-import type { RegisteredSession } from './registry.js'
-import { devinModelCommandResult } from '../engines/devin/runtimeProfile.js'
-import { countCommandcodeRefusals } from '../engines/commandcode/runtimeProfile.js'
-import { parseHermesPickerPage } from '../engines/hermes/runtimeProfile.js'
-import { parsePiFooterProfile, parsePiThinkingSelection, piThinkingSteps } from '../engines/pi/runtimeProfile.js'
-import {
- countOpencodePickers,
- opencodeRowMatches,
- opencodeRowNamesModel,
- parseOpencodePickerRows,
- type OpencodeModelTarget,
- type OpencodePickerRow,
-} from '../engines/opencode/runtimeProfile.js'
-import {
- chooseCodexRow,
- codexDigitPressable,
- codexEffortRows,
- parseCodexPicker,
- sameCodexPicker,
- type CodexCatalogModel,
- type CodexPicker,
-} from '../engines/codex/modelPicker.js'
-import {
- type CursorModelTarget,
- type RuntimeProfile,
- type RuntimeProfileManager,
-} from './runtimeProfile.js'
-
-const COMMAND_CONFIRM_MS = 8_000
-const PICKER_OPEN_MS = 3_000
-const PICKER_STEP_MS = 2_000
-/**
- * How far apart the two reads of a Codex model list are, which must agree before a digit is pressed on
- * it (see reachCodexEfforts): long enough for a redraw in flight to land, short beside the seconds a
- * switch takes to confirm.
- */
-const CODEX_SETTLE_MS = 250
-/** Hard bound on ladder keystrokes — twice pi's seven levels, so a desynchronised walk still terminates. */
-const PI_LADDER_MAX_STEPS = 14
-/** Hermes' longest picker page is a provider's model list; twice its size still terminates. */
-const HERMES_PAGE_MAX_STEPS = 60
-/** "Anthropic (13 models)" names provider key `anthropic`; "GitHub Copilot (17 models)" names `copilot`. */
-function hermesProviderMatches(row: string, provider: string): boolean {
- const name = row.replace(/\s*\([^)]*\)\s*$/, '').trim().toLowerCase()
- const key = provider.toLowerCase()
- return name === key || name.replace(/[^a-z0-9]/g, '').includes(key.replace(/[^a-z0-9]/g, ''))
-}
-
-export type RuntimeProfileErrorCode =
- | 'AGENT_NOT_FOUND'
- | 'INVALID_RUNTIME_PROFILE'
- | 'BUSY'
- | 'UNSUPPORTED_CLI_VERSION'
- | 'MODEL_UNAVAILABLE'
- | 'EFFORT_UNSUPPORTED'
- | 'PLAN_SCOPE_AMBIGUOUS'
- | 'CONFIRM_TIMEOUT'
- | 'TMUX_FAILED'
-
-export class RuntimeProfileControlError extends Error {
- constructor(readonly code: RuntimeProfileErrorCode) {
- super(code)
- }
-}
-
-export interface PaneInspection {
- idle: boolean
- plan: boolean
- dialog: boolean
- draft: boolean
-}
-
-export interface RuntimeProfileControllerDeps {
- manager: Pick<RuntimeProfileManager, 'getState' | 'selectedModel' | 'codexCatalog' | 'modelsForSession'
- | 'beginControl' | 'finishControl' | 'cancelControl' | 'confirmEffort' | 'waitForProfile' | 'waitForModel'
- | 'devinTarget' | 'opencodeCatalog' | 'commandcodeTarget' | 'ingestConfig' | 'hermesTarget' | 'cursorTarget' | 'confirmControlProfile'> & {
- supportsControl(session: RegisteredSession): boolean | Promise<boolean>
- effortAllowed(session: RegisteredSession, model: string, effort: string, listed: readonly string[] | null): boolean | Promise<boolean>
- ingestPane(session: RegisteredSession, text: string, silent?: boolean): boolean | Promise<boolean>
- }
- getSession: (sessionId: string) => RegisteredSession | undefined
- validateRuntime: (session: RegisteredSession) => Promise<boolean>
- capture: (terminalTarget: string, historyLines?: number) => Promise<string | null>
- sendText: (terminalTarget: string, text: string) => Promise<boolean>
- /** Type without submitting. */
- sendLiteral: (terminalTarget: string, text: string) => Promise<boolean>
- sendKey: (terminalTarget: string, key: string) => Promise<boolean>
- acquireInput: (sessionId: string) => (() => void) | null
-}
-
-function stripAnsi(value: string): string {
- return value.replace(/\u001b\[[0-9;:]*[A-Za-z]/g, '')
-}
-
-/**
- * The SGR attributes an escape sequence actually sets, with extended-colour arguments removed.
- *
- * `38`/`48`/`58` introduce a colour whose ARGUMENTS are numbers, not attributes: `5;<n>` for 256 and
- * `2;<r>;<g>;<b>` for truecolor. Without this skip a truecolor foreground (`38;2;…`) reads as SGR 2
- * — dim — and a line the user had typed would be mistaken for an empty composer, which is the one
- * direction this must never be wrong in: the pane would read idle and be respawned under a draft.
- */
-function sgrAttributes(params: readonly string[]): string[] {
- const attributes: string[] = []
- for (let index = 0; index < params.length; index += 1) {
- const code = params[index]
- if (code !== '38' && code !== '48' && code !== '58') {
- attributes.push(code)
- continue
- }
- const kind = params[index + 1]
- index += kind === '5' ? 2 : kind === '2' ? 4 : 1
- }
- return attributes
-}
-
-/**
- * The background an SGR sequence sets, as a comparable string — or null if it sets none.
- *
- * Needed because a composer box is identified by the colour it PAINTS, and the parameters that
- * choose one are positional: `48;5;<n>` and `48;2;<r>;<g>;<b>` carry their arguments inline, so the
- * arguments have to be consumed rather than read as further attributes. `38`/`58` are skipped the
- * same way for the same reason — a truecolor FOREGROUND ends in numbers that would otherwise parse
- * as a background code of their own.
- *
- * SGR 0 counts: a reset puts the default background back, which is a change like any other.
- */
-function sgrBackground(params: readonly string[]): string | null {
- for (let index = 0; index < params.length; index += 1) {
- const code = params[index]
- if (code === '48') {
- const kind = params[index + 1]
- return params.slice(index, index + (kind === '5' ? 3 : kind === '2' ? 5 : 2)).join(';')
- }
- if (code === '49' || /^(?:4[0-7]|10[0-7])$/.test(code)) return code
- if (code === '0' || code === '') return '49'
- if (code === '38' || code === '58') {
- const kind = params[index + 1]
- index += kind === '5' ? 2 : kind === '2' ? 4 : 1
- }
- }
- return null
-}
-
-/**
- * The span of a gutter line the BOX ITSELF paints — everything after the marker was wrong.
- *
- * A TUI draws one row of the terminal at a time, so anything floating OVER the composer lands on the
- * composer's own lines. OpenCode's `Getting started` card is exactly that: a panel on the right-hand
- * side, and its rows are written after the composer's on the same lines. Reading to end-of-line took
- * the card's text as something the user had typed —
- *
- * ┃ <121 spaces of composer> <sidebar> Connect provider /connect
- *
- * so `draft` was true over an EMPTY composer, the pane never read idle, and every model switch came
- * back AGENT_BUSY. Measured on a live pane (`harness-opencode-*`, 2026-09-15) with nothing typed in.
- * Fourth variant of the family above: the marker is found, the strip is dropped, and then the wrong
- * COLUMNS are read.
- *
- * The boundary is the box's own background. Everything a box paints is painted in it, so the
- * interior runs from the background set right after the marker to the next background change — in
- * the capture above, back to the pane's `48;2;10;10;10` where the box ends and the sidebar begins.
- *
- * Two deliberate conservatisms, both pointing the same way — a real draft must never be read as an
- * empty prompt, because that respawns the engine under text the user typed:
- * * A background that appears only AFTER visible text is not the interior's. It is a highlight
- * drawn mid-draft, and treating it as the boundary would cut the draft off at its own styling.
- * * A line that sets no background at all is not clipped. Plain-text panes (and every engine that
- * paints no box) keep the whole line, exactly as before.
- */
-function gutterBoxInterior(line: string, markerIndex: number): string {
- const after = line.slice(markerIndex + 1)
- const sgr = /\u001b\[([0-9;:]*)m/g
- let interior: string | null = null
- let end = after.length
- let match: RegExpExecArray | null
- while ((match = sgr.exec(after)) !== null) {
- const background = sgrBackground(match[1].split(/[;:]/))
- if (background === null) continue
- if (interior === null) {
- if (stripAnsi(after.slice(0, match.index)).trim().length > 0) break
- interior = background
- continue
- }
- if (background !== interior) { end = match.index; break }
- }
- return after.slice(0, end)
-}
-
-/**
- * Is the text after the composer marker the PLACEHOLDER, rather than something the user typed?
- *
- * Read from the STYLING, never the words: the placeholder sentence differs per engine and changes
- * between versions, but every one of these TUIs greys its placeholder out somehow and none of them
- * styles a draft the same way.
- *
- * Two attributes, because two are in use. SGR 2 (dim) is what claude, codex and devin draw; SGR 3
- * (italic) is what hermes draws — its `Ask anything, or type / for commands…` is italic over a
- * 256-colour gold (`ESC[3m ESC[38;5;136m`) and carries no dim at all. Looking only for dim made
- * EVERY hermes pane look like a pane with a draft in it, so `idle` was false for the agent's whole
- * life and every retarget and model switch came back AGENT_BUSY over an engine sitting at an empty
- * prompt. Same failure the comment on `promptMarker` warns about, one step further along: the
- * marker was found, and then the placeholder was not recognised as one.
- */
-function hasPlaceholderSgr(value: string): boolean {
- return [...value.matchAll(/\u001b\[([0-9;:]*)m/g)]
- .some((match) => {
- const attributes = sgrAttributes(match[1].split(/[;:]/))
- return attributes.includes('2') || attributes.includes('3')
- })
-}
-
-/**
- * Is this SGR foreground a MUTED one — the grey a TUI writes hints and placeholders in?
- *
- * Returns null when the sequence sets no foreground, so a caller can keep the colour already in
- * effect rather than treating "no change" as a change.
- *
- * Grey is judged by the colour itself: equal channels, and dark enough to read as secondary next to
- * the near-white a composer draws real text in. OpenCode's placeholder is `38;2;128;128;128` and its
- * own text `38;2;255;255;255`; the 256-colour ramp (232–255) and SGR 90 are the same intent spelled
- * differently.
- */
-function sgrMutedForeground(params: readonly string[]): boolean | null {
- for (let index = 0; index < params.length; index += 1) {
- const code = params[index]
- if (code === '38') {
- const kind = params[index + 1]
- if (kind === '2') {
- const [r, g, b] = [params[index + 2], params[index + 3], params[index + 4]].map(Number)
- if (![r, g, b].every(Number.isFinite)) return null
- return r === g && g === b && r >= 0x40 && r <= 0xb0
- }
- if (kind === '5') {
- const n = Number(params[index + 2])
- if (!Number.isFinite(n)) return null
- // 8 is bright black; 232–255 is the grey ramp, whose middle is the secondary tone.
- return n === 8 || (n >= 236 && n <= 250)
- }
- return null
- }
- if (code === '90') return true
- if (code === '39' || code === '0' || code === '') return false
- if (/^(?:3[0-7]|9[1-7])$/.test(code)) return false
- if (code === '48' || code === '58') {
- const kind = params[index + 1]
- index += kind === '5' ? 2 : kind === '2' ? 4 : 1
- }
- }
- return null
-}
-
-/**
- * Is EVERY visible character here written in a muted colour — i.e. is this a hint rather than a draft?
- *
- * The third placeholder styling this module has had to learn, after dim (claude, codex, devin) and
- * italic (hermes). OpenCode draws `Ask anything… "What is the tech stack of this project?"` as a plain
- * TRUECOLOR GREY with no dim and no italic attribute at all, so [hasPlaceholderSgr] saw nothing and a
- * brand-new pane read as a pane with a draft in it: never idle, and every model switch on an agent
- * with no messages yet refused as AGENT_BUSY over an empty composer. Measured on 1.18.31.
- *
- * ⚠️ EVERY character, not any — which is what keeps this from swallowing a real draft. A composer
- * writes what the user typed in its normal near-white; one such character anywhere means this is
- * text, whatever grey hint may be sitting beside it. Spaces are not evidence of either and carry no
- * colour worth reading, so they are skipped.
- */
-function allVisibleTextIsMuted(value: string): boolean {
- const sgr = /\u001b\[([0-9;:]*)m/g
- let muted: boolean | null = null
- let cursor = 0
- let sawText = false
- let match: RegExpExecArray | null
- const segment = (text: string): boolean => {
- if (!text.trim()) return true
- sawText = true
- return muted === true
- }
- while ((match = sgr.exec(value)) !== null) {
- if (!segment(value.slice(cursor, match.index))) return false
- const next = sgrMutedForeground(match[1].split(/[;:]/))
- if (next !== null) muted = next
- cursor = match.index + match[0].length
- }
- if (!segment(value.slice(cursor))) return false
- return sawText
-}
-
-/**
- * The glyph each CLI puts in front of its composer. They do not share one: cursor uses `→`, devin `❭`
- * (U+276D — close to, but not, claude's `❯` U+276F), and pi draws no marker at all. Getting this wrong is
- * not cosmetic — with no prompt found the pane never reads idle, and every switch is refused as BUSY.
- */
-function promptMarker(engine: RegisteredSession['engine']): RegExp {
- if (engine === 'cursor') return /→/u
- if (engine === 'devin') return /[❭›❯]/u
- if (engine === 'opencode') return /┃/u
- return /[›❯]/u
-}
-
-export function inspectRuntimePane(engine: RegisteredSession['engine'], capture: string): PaneInspection {
- if (engine === 'pi') return inspectPiPane(capture)
- if (engine === 'opencode') return inspectGutterBoxPane(capture, true)
- if (engine === 'copilot') return inspectGutterBoxPane(capture, false)
- if (engine === 'grok') return inspectGrokPane(capture)
- const rawLines = capture.split('\n')
- const marks = promptMarker(engine)
- const promptIndex = latestPromptLine(rawLines, marks)
- const prompt = promptIndex >= 0 ? rawLines[promptIndex] : ''
- const currentUi = currentPaneUi(rawLines, promptIndex)
- const dialog = DIALOG_UI.test(currentUi) || takeoverIn(engine, capture, currentUi) !== null
- const plan = engine === 'codex' ? /\bplan mode\b/i.test(currentUi) : /\bplan mode on\b/i.test(currentUi)
- const marker = stripAnsi(prompt).search(marks)
- let visible = marker >= 0 ? stripAnsi(prompt).slice(marker + 1).replace(/\u00a0/g, ' ').trim() : ''
- const rawMarker = prompt.search(marks)
- const rawTail = rawMarker >= 0 ? prompt.slice(rawMarker + 1) : ''
- const placeholder = hasPlaceholderSgr(rawTail)
- if (placeholder) visible = ''
- const draft = visible.length > 0
- return { idle: !!prompt && !dialog && !draft, plan, dialog, draft }
-}
-
-/** The latest composer prompt line; picker rows use the same glyphs, but numbered rows are not prompts. */
-function latestPromptLine(rawLines: string[], marks: RegExp): number {
- return rawLines.findLastIndex((line) => {
- const visible = stripAnsi(line)
- const marker = visible.search(marks)
- return marker >= 0 && !/^\s*\d+\.\s/.test(visible.slice(marker + 1))
- })
-}
-
-/**
- * Old picker/plan text can remain in tmux history. Only UI below the latest prompt belongs to the
- * current interaction; if no prompt is visible, inspect the whole capture as a conservative fallback.
- */
-function currentPaneUi(rawLines: string[], promptIndex: number): string {
- return stripAnsi((promptIndex >= 0 ? rawLines.slice(promptIndex) : rawLines).join('\n')).replace(/\u00a0/g, ' ')
-}
-
-/** A screen of Claude Code's or Codex's own that takes its composer's place (see [takeoverIn]). */
-export type PaneTakeover = 'rewind' | 'transcript' | 'search' | 'trust' | 'update' | 'model' | 'sign_in'
-
-/**
- * The modal over a Claude Code or Codex pane, read for a message about to be typed into it
- * (messageHold.ts): one of their own screens that takes the composer's place ([takeoverIn]), an
- * approval prompt, or another menu. Every one of them is a dialog to [inspectRuntimePane], except the
- * MCP boot notice: Claude Code takes typing while it shows, so a message sent the moment an agent
- * starts still goes in.
- */
-export function paneModal(engine: RegisteredSession['engine'], capture: string): PaneTakeover | 'permission' | 'menu' | null {
- const rawLines = capture.split('\n')
- const currentUi = currentPaneUi(rawLines, latestPromptLine(rawLines, promptMarker(engine)))
- const takeover = takeoverIn(engine, capture, currentUi)
- if (takeover) return takeover
- // Below the prompt line: that line is the composer, or the echo of the last message, and either can
- // hold anything a person typed.
- if (PERMISSION_UI.test(currentUi.slice(currentUi.indexOf('\n') + 1))) return 'permission'
- return DIALOG_UI.test(currentUi.replace(/Starting MCP servers?/gi, '')) ? 'menu' : null
-}
-
-/**
- * The screens of Claude Code 2.1.289 and Codex 0.160 that take the composer's place, where a pasted
- * message and its Enter do something other than send it (read from their code, not run; the screens
- * are in __fixtures__/rewindPickers.ts and takeoverScreens.ts):
- * - rewind: a picker for a point in the conversation to go back to, whose Enter picks one;
- * - transcript: Claude Code's transcript view (ctrl+o) and Codex's transcript overlay (ctrl+t, in its
- * scrollback mode), which drop the paste and have no Enter: the message is lost;
- * - search: a search through the prompt history (ctrl+r), or Codex's find in its transcript (F3),
- * which takes the paste as what to search for; Claude Code's Enter then SENDS the earlier prompt it
- * found, Codex's puts it in the composer or goes to the next match;
- * - trust, update, model, sign_in: the questions Codex asks at startup (trust this folder, update
- * now, switch to a new model, how to sign in) and Claude Code's trust and sign-in screens, which
- * drop the paste and take the Enter as the highlighted answer: trust the folder (Claude Code's
- * highlights `No, exit`, so it quits), run the update, switch the model, start a sign-in.
- * Each is matched on its own wording and rows, not on a guess at what is missing from the screen.
- */
-function takeoverIn(engine: RegisteredSession['engine'], capture: string, currentUi: string): PaneTakeover | null {
- if (engine !== 'claude' && engine !== 'codex') return null
- const lines = stripAnsi(capture).replace(/\u00a0/g, ' ').split('\n')
- const bottom = lines.filter((line) => line.trim()).slice(-3)
- // A startup screen is the last thing on the pane: one with a composer below it was answered.
- const composer = engine === 'codex'
- ? lines.findLastIndex((line) => /^\s*›/.test(line) && !/^\s*›\s*\d+\.\s/.test(line))
- : lines.findLastIndex((line, index) => /^\s*❯/.test(line) && index > 0 && /^\s*[─━]{8,}\s*$/.test(lines[index - 1]))
- const shown = (...patterns: RegExp[]) => patterns.every((pattern) => lines.findLastIndex((line) => pattern.test(line)) > composer)
- if (engine === 'codex') {
- if (CODEX_TRANSCRIPT_BROWSING.test(currentUi)) return 'rewind'
- if (CODEX_PAGER_HEADER.test(lines.find((line) => line.trim()) ?? '')) return 'transcript'
- if (bottom.some((line) => CODEX_SEARCH_FOOTER.test(line))) return 'search'
- if (shown(CODEX_TRUST_QUESTION, CODEX_TRUST_ROW)) return 'trust'
- if (shown(/^\s*Update available\b/, CODEX_UPDATE_ROW)) return 'update'
- if (shown(/^\s*[›>]?\s*1\. Try new model\s*$/, /^\s*[›>]?\s*2\. Use existing model\s*$/)) return 'model'
- if (shown(/^\s*[›>]?\s*1\. Sign in with ChatGPT\s*$/)) return 'sign_in'
- return null
- }
- if (claudeRewindMenuOpen(capture)) return 'rewind'
- if (bottom.some((line) => CLAUDE_TRANSCRIPT_FOOTER.test(line))) return 'transcript'
- if (bottom.some((line) => CLAUDE_HISTORY_SEARCH.test(line))) return 'search'
- if (shown(/Quick safety check: Is this a project you created or one you trust\?/, /^\s*(?:❯\s*)?Yes, I trust this folder\s*$/)) return 'trust'
- if (shown(/^\s*Select login method:\s*$/, /^\s*(?:❯\s*)?1\. Claude account with subscription\b/)) return 'sign_in'
- return null
-}
-
-/** The header of Codex's pager over the whole pane, its top row: its transcript (ctrl+t) when it is not
- * on the composer's screen (pager_overlay/transcript.rs). Closed with q or ctrl+t; Esc browses prompts. */
-const CODEX_PAGER_HEADER = /^\/ T R A N S C R I P T(?: \/)*\s*$/
-
-/** Codex's search footers: through its prompt history (`reverse-i-search: … enter accept · esc cancel`,
- * chat_composer/history_search.rs) and through its transcript (`Find: …`, transcript_view/search.rs). */
-const CODEX_SEARCH_FOOTER = /^\s*(?:reverse-i-search:|Find: )/
-
-/** Codex's trust question, as 0.160 words it (onboarding/trust_directory.rs) and as 0.147 did. */
-const CODEX_TRUST_QUESTION = /Trust this folder\? Codex can read, edit, and run files here|Do you trust the contents of this directory\?/
-const CODEX_TRUST_ROW = /^\s*[›>]?\s*1\. (?:Trust and continue|Yes, continue|Open restricted|Open existing task)\s*$/
-
-/** Codex's update prompt (update_prompt.rs): its first row runs the update. */
-const CODEX_UPDATE_ROW = /^\s*[›>]?\s*1\. Update now \(runs /
-
-/**
- * Claude Code's transcript view (ctrl+o): the prompt is hidden, and the footer row starts
- * `Showing detailed transcript · ctrl+o to toggle`, after `dialog waiting · ` when a dialog sits behind
- * it (2.1.289). It has no Enter, so a message typed there is lost. Esc, q or ctrl+c close it.
- */
-const CLAUDE_TRANSCRIPT_FOOTER = /^\s*(?:dialog waiting · )?Showing detailed transcript\b/
-
-/** Claude Code's prompt-history search (ctrl+r), under the prompt: `search prompts: <query>`, or
- * `no matching prompt: <query>` (2.1.289). */
-const CLAUDE_HISTORY_SEARCH = /^\s*(?:search prompts|no matching prompt): /
-
-/**
- * A modal drawn over the pane — a picker, a permission prompt, an MCP boot notice.
- *
- * One copy, shared by every reader here: an engine whose composer needs its own function still meets
- * the same dialogs, and two lists of these would drift the first time either gained an entry.
- * No `g` flag, so `test` carries no `lastIndex` between callers.
- */
-const DIALOG_UI = /Select Model(?: and Effort)?|Select Reasoning Level|Advanced Reasoning|Available models|Models matching|Edit Parameters|Type to filter.*Tab to edit|Esc to go back|Press enter to confirm|Do you want to proceed|Allow this action|permission required|Starting MCP servers?/i
-
-/** The approval prompts among them. */
-/**
- * An approval prompt's question. Claude Code asks "Do you want to proceed?", "Do you want to make this
- * edit to <file>?", "Do you want to allow Claude to fetch this content?", "Do you want to <verb> <target>?"
- * for the other tools, and "Would you like to proceed?" over a plan (2.1.289); Codex asks "Would you like
- * to run the following command?" and the like. The rows under it are read too (askQuestion.ts), but a
- * row wrapped in a narrow pane breaks that reading, and the question alone must still hold a message:
- * Enter approves the first row. Asked only below the prompt line, so never of what a person typed.
- */
-const PERMISSION_UI = /\bDo you want to\b|\bWould you like to (?:proceed|run|make|apply|allow)\b|Allow this action|permission required/i
-
-/**
- * Codex browsing its own transcript: Esc twice on an empty composer, and its footer row reads
- * `Browsing transcript · ↑↓/jk scroll · ←→/hl prompts · … · ↵ rewind · esc back`, down to a bare
- * `Browsing` as the pane narrows (0.160, tui/src/app_backtrack/prompt_navigation.rs). In its default
- * fullscreen mode the composer stays, dimmed whole, placeholder and any draft in it alike, so it read
- * as empty and idle; in its scrollback mode (`tui.fullscreen_transcript = false`, no alternate screen,
- * or over SSH) the footer closes a transcript pager drawn over the whole pane. Either way Enter
- * reverts the conversation to the prompt in view. A paste leaves the fullscreen browser and lands in
- * the composer, but the scrollback one drops it, and its Enter then rewinds (app.rs and
- * pager_overlay/transcript.rs).
- */
-const CODEX_TRANSCRIPT_BROWSING = /^\s*Browsing(?: transcript)?(?:\s+·|\s*$)/m
-
-/**
- * Claude Code's Rewind menu (Esc twice on an empty prompt, or `/rewind`), which hides the prompt while
- * it is open. Under its title, `Rewind`, it lists the messages sent so far, `(current)` focused, behind
- * one of these lines, or asks to confirm one picked (2.1.289, MessageSelector). Enter on a message asks
- * to confirm it, and Enter again restores the conversation, the code or both to before it (with file
- * checkpoints off, the first Enter restores the conversation); Enter on `(current)` closes the menu.
- * A message pasted into it is dropped either way. Its focused row is drawn `❯ (current)` in italics,
- * which read as an empty, idle prompt.
- */
-// Their opening words only: a pane narrower than a line wraps the rest onto the next, and the menu went
-// unseen, its Enter picking a message (with checkpoints off, rewinding to it).
-const CLAUDE_REWIND_BODY = /^(?:Restore the code\b|Restore and fork\b|Confirm you want to restore\b|Nothing to rewind to\b)/
-
-function claudeRewindMenuOpen(capture: string): boolean {
- const lines = stripAnsi(capture).replace(/\u00a0/g, ' ').split('\n').map((line) => line.trim())
- // The title, with its body a blank line under it, and no rule after it: the prompt the menu hides
- // comes back between two rules once it closes.
- return lines.some((line, index) => line === 'Rewind'
- && lines.slice(index + 1, index + 4).some((next) => CLAUDE_REWIND_BODY.test(next))
- && !lines.slice(index + 1).some((next) => /[─━]{8,}/u.test(next)))
-}
-
-/** The rule a gutter-box composer is closed with: `╹▀▀▀▀…`. */
-const GUTTER_BOX_RULE = /[─▀▁▔]{8,}/u
-
-/**
- * A composer drawn as a BOX with a `┃` gutter — OpenCode's and Copilot's, which are the same shape.
- *
- * The generic reader cannot read either one. OpenCode's `┃` does not stop at the text: the last
- * gutter line is the box's own STATUS strip — `Build · <model> · <account>`, sitting directly on
- * the rule that closes the box — so taking the last line carrying the marker reads what the BOX says
- * about itself as something the user typed. Copilot fails one step earlier: its composer carries no
- * `›`/`❯` at all, while every message the user has ALREADY SENT is echoed into the transcript as
- * `❯ <text>` in full brightness, so the generic reader walks back to the last of those and reads a
- * sent message as a draft. Either way the pane was permanently "holding a draft": never idle, and
- * every retarget and model switch refused as AGENT_BUSY over an empty composer. Third variant of the
- * failure hermes had — there the placeholder was not recognised, here the wrong line is read.
- *
- * [hasStatusStrip] is the one thing that differs between the two, and it is OpenCode's alone. The
- * strip is identified by WHERE it is — the gutter line the closing rule sits under — rather than by
- * what it says, which is three variable fields that change with the model, the mode and the account.
- * When no rule follows, no strip is assumed and every gutter line counts as composer: a layout we do
- * not recognise must not silently swallow a line the user typed in. Copilot's box has no strip, so
- * its single gutter line IS the composer and dropping it would read a real draft as an empty prompt
- * — the one direction this must never be wrong in.
- *
- * Every gutter line is checked, not just the last: a long message wraps down the box, and reading
- * one line of it would call a full composer empty.
- *
- * A dialog needs no special case in either engine: both REPLACE the composer with the picker or the
- * permission prompt, so no gutter is on screen at all and the pane reads not-idle from the empty run
- * below rather than from [DIALOG_UI].
- */
-function inspectGutterBoxPane(capture: string, hasStatusStrip: boolean): PaneInspection {
- const rawLines = capture.split('\n')
- const marks = /┃/u
- // The LAST contiguous run of gutter lines. Earlier runs are previous composers still in scrollback
- // — the message the user already sent is not a draft.
- const gutter: number[] = []
- for (let index = rawLines.length - 1; index >= 0; index -= 1) {
- if (marks.test(stripAnsi(rawLines[index]))) { gutter.unshift(index); continue }
- if (gutter.length > 0) break
- }
- if (gutter.length === 0) return { idle: false, plan: false, dialog: false, draft: false }
- const last = gutter[gutter.length - 1]
- const closed = last + 1 < rawLines.length && GUTTER_BOX_RULE.test(stripAnsi(rawLines[last + 1]))
- const composer = hasStatusStrip && closed ? gutter.slice(0, -1) : gutter
- const currentUi = stripAnsi(rawLines.slice(gutter[0]).join('\n')).replace(/\u00a0/g, ' ')
- const dialog = DIALOG_UI.test(currentUi)
- const draft = composer.some((index) => {
- const line = rawLines[index]
- const visibleMarker = stripAnsi(line).search(marks)
- if (visibleMarker < 0) return false
- const rawMarker = line.search(marks)
- // The box's interior, not the rest of the row: a panel floating over the composer writes into
- // these same lines, and its text is not a draft. See [gutterBoxInterior]. With no marker in the
- // RAW line there is no interior to bound — read the whole row, the direction that keeps a draft.
- const interior = rawMarker < 0
- ? stripAnsi(line).slice(visibleMarker + 1)
- : gutterBoxInterior(line, rawMarker)
- if (hasPlaceholderSgr(interior) || allVisibleTextIsMuted(interior)) return false
- return stripAnsi(interior).replace(/\u00a0/g, ' ').trim().length > 0
- })
- return { idle: !dialog && !draft, plan: /\bplan mode on\b/i.test(currentUi), dialog, draft }
-}
-
-/**
- * The context readout Pi's footer always carries — `0.0%/500k`, `1.5%/200k`. Present whether or not the
- * model advertises a thinking ladder, and drawn by the normal view alone, which is what makes it the
- * honest "the footer is on screen" signal. No `g` flag, so `test` carries no `lastIndex` between calls.
- */
-const PI_FOOTER_BUDGET = /\d+(?:\.\d+)?%\s*\/\s*\S+/
-
-/**
- * Grok draws its composer as a ROUNDED BOX, and the generic reader counts the box's own right edge
- * as something the user typed.
- *
- * Its empty composer is one line — `│ ❯` … padding … `│` — inside `╭─…─╮` above and
- * `╰─… Grok 4.6 (xhigh) ─╯` below. The shared reader takes everything after `❯`, strips ANSI, trims,
- * and finds `│`: one character, non-empty, so `draft` was true. Measured on a real pane
- * (`harness-grok-*`, 2026-09-09) with nothing typed into it: `visible after marker = "│"`, `draft =
- * true`, `idle = false`. Not a transient — the edge is drawn on every frame, so idle was false for
- * the agent's whole life and EVERY model switch came back AGENT_BUSY over an empty prompt. Same
- * family as the hermes and gutter-box bugs above; here the culprit is the box, not the placeholder.
- *
- * So the composer is the box's INTERIOR: the run of `│` body lines, read between their first and
- * last `│` rather than from the marker to end-of-line. Every body line is checked, not just the one
- * carrying `❯` — a long message wraps down the box and reading one line of it would call a full
- * composer empty.
- *
- * ⚠️ `│` is also Grok's footer separator (`Shift+Tab:mode │ Ctrl+x:shortcuts`), which sits BELOW the
- * box. It is excluded by requiring a body line to both open and close with `│`, which the footer —
- * one separator, no edges — never does.
- */
-function inspectGrokPane(capture: string): PaneInspection {
- const rawLines = capture.split('\n')
- const visibleLines = rawLines.map((line) => stripAnsi(line).replace(/ /g, ' '))
- // The LAST box in the capture. Earlier ones are composers already sent, still in scrollback.
- const bottom = visibleLines.findLastIndex((line) => /^\s*╰[─╌]*.*╯\s*$/u.test(line))
- const top = bottom < 0
- ? -1
- : visibleLines.slice(0, bottom).findLastIndex((line) => /^\s*╭[─╌]+╮\s*$/u.test(line))
- // No recognisable box is "we cannot tell", which the caller reads as busy — the safe direction.
- // Respawning a pane whose layout we do not understand could discard a draft.
- if (top < 0 || bottom - top < 2) return { idle: false, plan: false, dialog: false, draft: false }
- const currentUi = visibleLines.slice(top).join('\n')
- const dialog = DIALOG_UI.test(currentUi)
- const draft = rawLines.slice(top + 1, bottom).some((line, offset) => {
- const visible = visibleLines[top + 1 + offset]
- const first = visible.indexOf('│')
- const last = visible.lastIndexOf('│')
- // Both edges, and not the same character: anything else is not a body line of this box.
- if (first < 0 || last <= first) return false
- const rawMarker = line.search(/[›❯]/u)
- if (rawMarker >= 0 && hasPlaceholderSgr(line.slice(rawMarker + 1))) return false
- // The marker is chrome, not content — it is printed whether or not anything was typed.
- return visible.slice(first + 1, last).replace(/[›❯]/u, '').trim().length > 0
- })
- return { idle: !dialog && !draft, plan: /\bplan mode on\b/i.test(currentUi), dialog, draft }
-}
-
-/**
- * Pi draws no composer marker, so idleness is read from its layout instead: the composer is the band
- * between the last two `───` rules, and the footer (`minimax/minimax-m3 • high`) is only rendered by the
- * normal view. A non-empty band is a draft the user is still typing — injecting there would splice a
- * slash command into their sentence.
- */
-function inspectPiPane(capture: string): PaneInspection {
- const lines = stripAnsi(capture).split('\n').map((line) => line.replace(/\s+$/, ''))
- const text = lines.join('\n')
- // Whether the footer is DRAWN, which is not the same question as whether its profile can be READ —
- // and conflating the two is what has broken this check twice.
- //
- // `parsePiFooterProfile` is strict on purpose: it must return a model AND an effort or nothing, so a
- // half-drawn redraw is never mistaken for a profile change. Idleness needs far less — only that the
- // normal view is on screen rather than a picker — and borrowing the strict reader for it meant every
- // footer shape it could not fully parse read as "still busy".
- //
- // The shape it could not parse: a model with no thinking ladder draws no `• <level>` at all, so the
- // line ends at the model name. EVERY grid model is that shape, because the provider block written in
- // `gridLaunch.ts` declares `reasoning: false` — which left every Pi agent on a grid permanently
- // AGENT_BUSY, refusing model switches over an idle pane (measured, pi 0.82, footer
- // `↑5.4k ↓292 1.5%/200k (auto) Auto`).
- //
- // So: the profile still counts when it parses, and the token-budget readout — `0.0%/500k`,
- // `1.5%/200k`, present in both shapes and drawn by no picker — carries the rest.
- const footer = parsePiFooterProfile(text) !== null || PI_FOOTER_BUDGET.test(text)
- const rules: number[] = []
- lines.forEach((line, index) => { if (/^\s*─{8,}\s*$/.test(line)) rules.push(index) })
- const dialog = /Thinking Level|Select reasoning depth|Type to search|Enter to select|Only showing models from/i
- .test(lines.join('\n'))
- if (rules.length < 2) return { idle: false, plan: false, dialog, draft: false }
- const band = lines.slice(rules[rules.length - 2] + 1, rules[rules.length - 1]).join('').trim()
- const draft = band.length > 0
- return { idle: footer && !dialog && !draft, plan: false, dialog, draft }
-}
-
-export interface CursorParameterRow {
- kind: 'context' | 'reasoning' | 'fast' | 'thinking'
- value: string
- selected: boolean
- cursor: boolean
- index: number
-}
-
-export function parseCursorModelPicker(capture: string): { selectedFamily: string | null } | null {
- const lines = stripAnsi(capture).split('\n')
- if (!lines.some((line) => /Available models|Models matching/i.test(line)) || !lines.some((line) => /Tab to edit/i.test(line))) {
- return null
- }
- const selected = lines.findLast((line) => /^\s*→\s+/.test(line))
- if (!selected) return { selectedFamily: null }
- const value = selected.replace(/^\s*→\s+/, '').replace(/\s+\(Tab to modify\)\s*$/i, '')
- return { selectedFamily: value.split(/\s{2,}/)[0]?.trim() || null }
-}
-
-export function parseCursorParameterRows(capture: string): CursorParameterRow[] | null {
- const lines = stripAnsi(capture).split('\n')
- if (!lines.some((line) => /— Edit Parameters/i.test(line))) return null
- const rows: CursorParameterRow[] = []
- let section: 'context' | 'reasoning' | null = null
- for (const raw of lines) {
- const line = raw.replace(/\u00a0/g, ' ')
- if (/^\s*Context\s*$/.test(line)) { section = 'context'; continue }
- if (/^\s*Reasoning\s*$/.test(line)) { section = 'reasoning'; continue }
- const choice = /^\s*(→)?\s*([●○◉◯])\s+(.+?)(?:\s+✓)?\s*$/.exec(line)
- if (!choice) continue
- const label = choice[3].trim()
- let kind: CursorParameterRow['kind']
- let value: string
- if (/^Fast$/i.test(label)) {
- kind = 'fast'
- value = 'true'
- } else if (/^Thinking$/i.test(label)) {
- kind = 'thinking'
- value = 'true'
- } else if (section === 'context') {
- kind = 'context'
- value = label.toLowerCase()
- } else if (section === 'reasoning') {
- kind = 'reasoning'
- value = label.toLowerCase().replace(/\s+/g, '') === 'extrahigh'
- ? 'xhigh'
- : label.toLowerCase()
- } else {
- continue
- }
- rows.push({
- kind,
- value,
- selected: choice[2] === '●' || choice[2] === '◉' || /✓\s*$/.test(line),
- cursor: !!choice[1],
- index: rows.length,
- })
- }
- return rows.length ? rows : null
-}
-
-/** The lists a model is chosen from, as against the reasoning screens that follow them. */
-function isCodexList(picker: CodexPicker): boolean {
- return picker.kind === 'quick' || picker.kind === 'models' || picker.kind === 'reserve'
-}
-
-function sleep(ms: number): Promise<void> {
- return new Promise((resolve) => setTimeout(resolve, ms))
-}
-
-export class RuntimeProfileController {
- constructor(private readonly deps: RuntimeProfileControllerDeps) {}
-
- async setProfile(sessionId: string, encoded: unknown): Promise<void> {
- const registeredSession = this.deps.getSession(sessionId)
- if (!registeredSession) throw new RuntimeProfileControlError('AGENT_NOT_FOUND')
- const observed = this.deps.manager.getState(registeredSession.sessionId)
- const session: RegisteredSession = registeredSession.cliVersion || !observed.cliVersion
- ? registeredSession
- : { ...registeredSession, cliVersion: observed.cliVersion }
- const target = parseRuntimeProfile(encoded)
- // Either id identifies the agent: the encoded profile now carries the agent id, but one minted
- // before the id cutover (or by a client that still knows the session) must keep working.
- if (!target || (target.sessionId !== session.agentId && target.sessionId !== session.sessionId) || target.engine !== session.engine) {
- throw new RuntimeProfileControlError('INVALID_RUNTIME_PROFILE')
- }
- const current = parseRuntimeProfile(this.deps.manager.selectedModel(session))
- if (current?.id === target.id) return
- if (!await this.deps.manager.supportsControl(session)) {
- console.warn(`[runtime-profile] unsupported ${session.engine} CLI version ${session.cliVersion ?? 'unknown'} for ${sessionId.slice(0, 8)}`)
- throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
- }
- // Codex's own catalog: which efforts each model takes, and the names its picker shows them by.
- const codexCatalog = session.engine === 'codex' ? await this.deps.manager.codexCatalog(session) : []
- if (session.engine === 'codex') {
- const listed = codexCatalog.find((entry) => entry.slug === target.model)?.efforts ?? null
- if (!await this.deps.manager.effortAllowed(session, target.model, target.effort, listed)) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
- }
- const options = await this.deps.manager.modelsForSession(session)
- if (!options.some((option) => option.id === target.id)) {
- const sameModel = options.some((option) => parseRuntimeProfile(option.id)?.model === target.model)
- throw new RuntimeProfileControlError(sameModel ? 'EFFORT_UNSUPPORTED' : 'MODEL_UNAVAILABLE')
- }
- if (session.engine === 'codex' && observed.mode === 'plan') throw new RuntimeProfileControlError('PLAN_SCOPE_AMBIGUOUS')
- const release = this.deps.acquireInput(session.agentId)
- if (!release) throw new RuntimeProfileControlError('BUSY')
- let controlStarted = false
- let pickerOpen = false
- try {
- if (!await this.deps.validateRuntime(session)) throw new RuntimeProfileControlError('TMUX_FAILED')
- const capture = await this.deps.capture(session.agentId, 100)
- if (!capture) throw new RuntimeProfileControlError('TMUX_FAILED')
- const inspection = inspectRuntimePane(session.engine, capture)
- if (session.engine === 'codex' && inspection.plan) throw new RuntimeProfileControlError('PLAN_SCOPE_AMBIGUOUS')
- if (!inspection.idle) throw new RuntimeProfileControlError('BUSY')
- if (!this.deps.manager.beginControl(session, target)) throw new RuntimeProfileControlError('BUSY')
- controlStarted = true
- if (session.engine === 'claude') {
- await this.setClaude(session, target, current, options)
- } else if (session.engine === 'codex') {
- pickerOpen = true
- await this.setCodex(session, target, codexCatalog)
- pickerOpen = false
- } else if (session.engine === 'devin') {
- // No picker is ever opened — `/model <id>` is a single command — so there is nothing to Escape out
- // of if this throws.
- await this.setDevin(session, target)
- } else if (session.engine === 'pi') {
- pickerOpen = true
- await this.setPi(session, target, current)
- pickerOpen = false
- } else if (session.engine === 'opencode') {
- pickerOpen = true
- await this.setOpencode(session, target)
- pickerOpen = false
- } else if (session.engine === 'hermes') {
- pickerOpen = true
- await this.setHermes(session, target)
- pickerOpen = false
- } else if (session.engine === 'commandcode') {
- // Two plain commands, no dialog — nothing to Escape out of if either throws.
- await this.setCommandcode(session, target, current)
- } else {
- pickerOpen = true
- await this.setCursor(session, target)
- pickerOpen = false
- }
- this.deps.manager.finishControl(session)
- controlStarted = false
- } catch (error) {
- if (pickerOpen) {
- for (let attempt = 0; attempt < 3; attempt++) {
- await this.deps.sendKey(session.agentId, 'Escape').catch(() => false)
- await sleep(100)
- const next = await this.deps.capture(session.agentId, 100).catch(() => null)
- if (!next || !inspectRuntimePane(session.engine, next).dialog) break
- }
- }
- // By the engine session, as beginControl keyed it. `sessionId` is whichever id the request named,
- // and the apps name the agent: cancelling under that left the control in place, and every later
- // switch of the agent answered BUSY until the daemon restarted (found by e2e/models.e2e.ts, a
- // refused switch followed by one that should have gone through).
- if (controlStarted) this.deps.manager.cancelControl(session.sessionId)
- // A failed switch used to leave NOTHING in the log — the picker just flashed open and shut on the
- // user's terminal and the device said nothing useful. Name the session, engine and reason.
- const code = error instanceof RuntimeProfileControlError ? error.code : 'TMUX_FAILED'
- console.warn(`[runtime-profile] ${sessionId.slice(0, 8)} ${session.engine} set ${target.model}@${target.effort} failed: ${code}`)
- if (error instanceof RuntimeProfileControlError) throw error
- throw new RuntimeProfileControlError('TMUX_FAILED')
- } finally {
- release()
- }
- }
-
- private async setClaude(
- session: RegisteredSession,
- target: RuntimeProfile,
- current: RuntimeProfile | null,
- options: Array<{ id: string }>,
- ): Promise<void> {
- if (current?.model !== target.model) {
- if (!await this.deps.sendText(session.agentId, `/model ${target.model}`)) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- if (!await this.waitClaudeModel(session)) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
- }
-
- if (current?.effort === target.effort) {
- this.deps.manager.confirmEffort(session.sessionId, target.effort)
- return
- }
-
- const hasExplicitEffort = options.some((option) => {
- const profile = parseRuntimeProfile(option.id)
- return profile?.model === target.model && profile.effort !== 'auto'
- })
- if (!hasExplicitEffort && target.effort === 'auto') {
- this.deps.manager.confirmEffort(session.sessionId, 'auto')
- return
- }
- if (!await this.deps.sendText(session.agentId, `/effort ${target.effort}`)) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- if (!await this.deps.manager.waitForProfile(session.sessionId, COMMAND_CONFIRM_MS)) {
- await this.deps.sendKey(session.agentId, 'Enter')
- if (!await this.deps.manager.waitForProfile(session.sessionId, 2_000)) {
- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
- }
- }
- }
-
- private async waitClaudeModel(session: RegisteredSession): Promise<boolean> {
- if (await this.deps.manager.waitForModel(session.sessionId, 1_200)) return true
- const capture = await this.deps.capture(session.agentId, 80)
- if (capture && /switch model|change model|continue.*model|re-read.*history/i.test(stripAnsi(capture))) {
- await this.deps.sendKey(session.agentId, 'Enter')
- } else {
- // Claude and Codex can swallow the first Enter immediately after a terminal literal write.
- await this.deps.sendKey(session.agentId, 'Enter')
- }
- return this.deps.manager.waitForModel(session.sessionId, COMMAND_CONFIRM_MS)
- }
-
- /**
- * Devin takes the whole profile in one command: `/model <id>` accepts the exact id from
- * `devin models list` (effort included), and answers in the pane — `✓ Model set to <name>` or
- * `✗ Model not available`. That explicit answer is why devin needs no picker driving at all, and why an
- * unavailable model (most of them, on a free plan) fails fast instead of timing out.
- */
- private async setDevin(session: RegisteredSession, target: RuntimeProfile): Promise<void> {
- const devin = this.deps.manager.devinTarget(session.sessionId, target.id)
- if (!devin) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
- if (!await this.deps.sendText(session.agentId, `/model ${devin.id}`)) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- const answered = await this.waitPane(
- session.agentId,
- (value) => devinModelCommandResult(stripAnsi(value)) !== null,
- COMMAND_CONFIRM_MS,
- )
- if (!answered) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
- if (devinModelCommandResult(stripAnsi(answered)) === 'unavailable') {
- throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
- }
- // The acknowledgement line is not the state — the footer is, and only `ingestPane` reads it. Nothing
- // else feeds the manager during a control, so the wait has to do the feeding itself (same shape as
- // setCursor); waiting on `waitForProfile` alone timed out on a switch that had already landed.
- if (!await this.waitObservedProfile(session, target)) {
- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
- }
- }
-
- /**
- * Pi splits the axes: `/model <provider/model>` takes a target directly, while the thinking level lives
- * only in `/settings` → "Thinking Level", a fixed ladder walked with the arrow keys. Both were driven by
- * hand first; the ladder's order is what makes the arrow count deterministic (see PI_THINKING_LEVELS).
- */
- private async setPi(
- session: RegisteredSession,
- target: RuntimeProfile,
- current: RuntimeProfile | null,
- ): Promise<void> {
- if (current?.model !== target.model) {
- if (!await this.deps.sendText(session.agentId, `/model ${target.model}`)) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- if (!await this.deps.manager.waitForModel(session.sessionId, COMMAND_CONFIRM_MS)) {
- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
- }
- }
- if (target.effort === 'auto' || current?.effort === target.effort) {
- this.deps.manager.confirmEffort(session.sessionId, target.effort)
- return
- }
- await this.setPiThinking(session, target)
- }
-
- private async setPiThinking(session: RegisteredSession, target: RuntimeProfile): Promise<void> {
- const effort = target.effort
- if (!await this.deps.sendText(session.agentId, '/settings')) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- if (!await this.waitPane(session.agentId, (v) => /Type to search/i.test(stripAnsi(v)), PICKER_OPEN_MS)) {
- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
- }
- // Typing alone is enough: the settings filter opens a row the moment it narrows to one. Submitting
- // here would land an Enter on the ladder that just opened and pick the level already highlighted —
- // which is the CURRENT one, so the switch silently did nothing.
- if (!await this.deps.sendLiteral(session.agentId, 'thinking level')) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- const ladder = await this.waitPane(
- session.agentId,
- (v) => /Select reasoning depth/i.test(stripAnsi(v)),
- PICKER_OPEN_MS,
- )
- if (!ladder) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
-
- // Walk the ladder ONE key at a time, re-reading the cursor after each. Counting the steps up front and
- // firing them blind is what an open-loop drive looks like, and it is exactly how this went wrong: a
- // stale first read sent the cursor to `off` and committed it. Re-reading also means a ladder that
- // scrolls, clamps at an end, or gains a level cannot desynchronise the walk.
- let selection = parsePiThinkingSelection(stripAnsi(ladder))
- if (!selection || piThinkingSteps(selection, effort) === null) {
- throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
- }
- for (let guard = 0; selection !== effort && guard < PI_LADDER_MAX_STEPS; guard++) {
- const steps = piThinkingSteps(selection as string, effort)
- if (steps === null || steps === 0) break
- if (!await this.deps.sendKey(session.agentId, steps > 0 ? 'Down' : 'Up')) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- const moved = await this.waitPane(
- session.agentId,
- (value) => {
- const next = parsePiThinkingSelection(stripAnsi(value))
- return !!next && next !== selection
- },
- PICKER_STEP_MS,
- )
- // No movement means the cursor is pinned at an end of the ladder — pressing on cannot reach it.
- if (!moved) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
- selection = parsePiThinkingSelection(stripAnsi(moved))
- }
- if (selection !== effort) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
- await this.deps.sendKey(session.agentId, 'Enter')
- // The ladder returns to the settings list; Escape closes it so the pane is idle again.
- await this.deps.sendKey(session.agentId, 'Escape')
- if (!await this.waitObservedProfile(session, target)) {
- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
- }
- }
-
- /** Poll the pane INTO the manager until it reports the profile we asked for. */
- private async waitObservedProfile(session: RegisteredSession, target: RuntimeProfile): Promise<boolean> {
- const confirmed = await this.waitPane(session.agentId, async (value) => {
- await this.deps.manager.ingestPane(session, value, true)
- return this.deps.manager.selectedModel(session) === target.id
- }, COMMAND_CONFIRM_MS)
- return !!confirmed
- }
-
- /**
- * OpenCode switches through `/models`, a picker with a typed filter. It is driven by NARROWING, never by
- * arrowing: type the model's words plus its provider's, and press Enter only once exactly one row is
- * left and that row is the requested model. If the filter leaves two rows (two providers can carry the
- * same model name) the switch is refused rather than guessed — picking the wrong model silently is worse
- * than not switching, and blind arrow-driving is what sank the Command Code attempt.
- */
- private async setOpencode(session: RegisteredSession, target: RuntimeProfile): Promise<void> {
- const entry = (await this.deps.manager.opencodeCatalog()).find((item) => item.id === target.model)
- if (!entry) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
- await this.driveOpencodePicker(session, entry)
- if (!await this.waitObservedProfile(session, target)) {
- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
- }
- }
-
- /**
- * `/models`, filter, act only once exactly one row is left.
- *
- * Two attempts, because one filter does not fit both renders. With several providers connected a
- * row carries its provider (`Big Pickle OpenCode Zen`) and the provider-qualified filter is what
- * tells two same-named models apart; with one provider the row is bare (`Big Pickle`) and that same
- * filter matches NOTHING. Measured on a live picker: `big pickle opencode` → empty, `big pickle` →
- * the row.
- *
- * The fallback drops only the PROVIDER half, never the "exactly one row" rule — which is what makes
- * it safe: if one model answers to the name, there is no second provider to confuse it with, and if
- * two do, the list does not narrow and this refuses rather than guessing.
- */
- private async driveOpencodePicker(session: RegisteredSession, entry: OpencodeModelTarget): Promise<void> {
- if (await this.pickOpencodeRow(session, entry, entry.filter, opencodeRowMatches)) return
- if (entry.modelFilter === entry.filter) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
- if (!await this.pickOpencodeRow(session, entry, entry.modelFilter, opencodeRowNamesModel)) {
- throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
- }
- }
-
- /**
- * One attempt at the picker: open it, type `filter`, and press Enter only if exactly one row is left
- * AND `matches` accepts it. Returns false — with the picker CLOSED — when the list did not narrow to
- * a row this may act on, so the caller can try a different filter on a pane in its resting state.
- * Throws only for the states no retry can help: tmux refusing input, or the picker never opening.
- */
- private async pickOpencodeRow(
- session: RegisteredSession,
- entry: OpencodeModelTarget,
- filter: string,
- matches: (target: OpencodeModelTarget, row: OpencodePickerRow) => boolean,
- ): Promise<boolean> {
- // "Is a picker open?" cannot be asked of a capture that carries scrollback — an EARLIER picker is
- // still up there, so the check passes before this one opens and the filter would be typed into the
- // composer and submitted as a message. Count the openings instead and wait for one more.
- const before = countOpencodePickers(stripAnsi(await this.deps.capture(session.agentId, 100) ?? ''))
- if (!await this.deps.sendText(session.agentId, '/models')) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- if (!await this.waitPane(
- session.agentId,
- (v) => countOpencodePickers(stripAnsi(v)) > before,
- PICKER_OPEN_MS,
- )) {
- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
- }
- if (!await this.deps.sendLiteral(session.agentId, filter)) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- const narrowed = await this.waitPane(
- session.agentId,
- (v) => (parseOpencodePickerRows(stripAnsi(v)) ?? []).length === 1,
- PICKER_STEP_MS,
- )
- const rows = narrowed ? parseOpencodePickerRows(stripAnsi(narrowed)) ?? [] : []
- if (rows.length !== 1 || !matches(entry, rows[0])) {
- // Leave the pane as it was found. A picker left open with a dead filter in it swallows whatever
- // the user types next, and the retry below would type its filter on top of this one.
- await this.deps.sendKey(session.agentId, 'Escape')
- return false
- }
- if (!await this.deps.sendKey(session.agentId, 'Enter')) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- return true
- }
-
- /**
- * Command Code 1.6.0 takes both axes as arguments: `/model <vendor/id>` and `/effort <level>`. Neither
- * opens a dialog, which is the whole reason switching is back after the 2026-07-30 revert — the version
- * gate in supportsNativeRuntimeControl keeps older builds, where only an arrow-driven picker existed,
- * out of this path.
- *
- * Effort is per model and the catalog does not say which models have one, so a refusal is expected
- * traffic: the CLI answers "Reasoning effort not supported for X." and that becomes EFFORT_UNSUPPORTED.
- */
- private async setCommandcode(
- session: RegisteredSession,
- target: RuntimeProfile,
- current: RuntimeProfile | null,
- ): Promise<void> {
- const entry = this.deps.manager.commandcodeTarget(session.sessionId, target.id)
- if (!entry) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
-
- if (current?.model !== target.model) {
- if (!await this.deps.sendText(session.agentId, `/model ${entry.id}`)) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- if (!await this.waitObservedModel(session, target)) {
- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
- }
- }
-
- if (target.effort === 'auto' || current?.effort === target.effort) {
- this.deps.manager.confirmEffort(session.sessionId, target.effort)
- return
- }
- // Success prints NOTHING and a refusal prints one line, so both have to be watched at once — and the
- // refusal has to be a NEW one. Matching the text anywhere in the capture reported failure in 25ms off
- // a refusal still sitting in the scrollback from an earlier model, while the level had in fact applied.
- const refusalsBefore = countCommandcodeRefusals(stripAnsi(await this.deps.capture(session.agentId, 100) ?? ''))
- if (!await this.deps.sendText(session.agentId, `/effort ${target.effort}`)) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- const deadline = Date.now() + COMMAND_CONFIRM_MS
- while (Date.now() < deadline) {
- // The level is recorded only in the CLI's global config, so confirming it means re-reading that.
- await this.deps.manager.ingestConfig(session, true).catch(() => false)
- if (this.deps.manager.selectedModel(session) === target.id) return
- const capture = stripAnsi(await this.deps.capture(session.agentId, 100) ?? '')
- if (countCommandcodeRefusals(capture) > refusalsBefore) {
- throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
- }
- await sleep(150)
- }
- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
- }
-
- /**
- * Hermes' `/model` is a two-level picker — provider page, then that provider's models — with no way to
- * type or jump. It is walked one arrow at a time, re-reading the `❯` cursor after every key, so a page
- * that scrolls or reorders cannot desynchronise the walk. If the cursor stops moving before it reaches
- * the target, the target is not on the page and the switch is refused rather than committed blind.
- *
- * Only MODELS are switchable; hermes keeps its reasoning effort in config.yaml with no in-session
- * command, so hermesModels never offers an effort row to get here (see hermesModels).
- */
- private async setHermes(session: RegisteredSession, target: RuntimeProfile): Promise<void> {
- const entry = this.deps.manager.hermesTarget(session.sessionId, target.id)
- if (!entry) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
- if (!await this.deps.sendText(session.agentId, '/model')) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- const opened = await this.waitPane(
- session.agentId,
- (v) => !!parseHermesPickerPage(stripAnsi(v)),
- PICKER_OPEN_MS,
- )
- if (!opened) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
-
- // Page 1: the provider. Its row reads "Anthropic (13 models)", so match on the leading name.
- if (entry.provider) {
- await this.walkHermesPage(session, (row) => hermesProviderMatches(row, entry.provider))
- if (!await this.waitPane(
- session.agentId,
- (v) => (parseHermesPickerPage(stripAnsi(v))?.rows ?? []).some((row) => row === entry.id),
- PICKER_OPEN_MS,
- )) {
- throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
- }
- }
- // Page 2: the model itself, listed by exact id.
- await this.walkHermesPage(session, (row) => row === entry.id)
- if (!await this.waitObservedModel(session, target)) {
- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
- }
- }
-
- /** Move the `❯` cursor onto the row `wanted` selects, then commit it with Enter. */
- private async walkHermesPage(
- session: RegisteredSession,
- wanted: (row: string) => boolean,
- ): Promise<void> {
- for (let guard = 0; guard < HERMES_PAGE_MAX_STEPS; guard++) {
- const capture = await this.deps.capture(session.agentId, 100)
- const page = capture ? parseHermesPickerPage(stripAnsi(capture)) : null
- if (!page || !page.selected) throw new RuntimeProfileControlError('TMUX_FAILED')
- if (wanted(page.selected)) {
- if (!await this.deps.sendKey(session.agentId, 'Enter')) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- return
- }
- const at = page.rows.indexOf(page.selected)
- const to = page.rows.findIndex(wanted)
- if (to < 0) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
- if (!await this.deps.sendKey(session.agentId, to > at ? 'Down' : 'Up')) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- const moved = await this.waitPane(session.agentId, (v) => {
- const next = parseHermesPickerPage(stripAnsi(v))
- return !!next?.selected && next.selected !== page.selected
- }, PICKER_STEP_MS)
- if (!moved) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
- }
- throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
- }
-
- /** Like waitObservedProfile, but only the model half — used where effort is applied separately. */
- private async waitObservedModel(session: RegisteredSession, target: RuntimeProfile): Promise<boolean> {
- const confirmed = await this.waitPane(session.agentId, async (value) => {
- await this.deps.manager.ingestPane(session, value, true)
- return parseRuntimeProfile(this.deps.manager.selectedModel(session))?.model === target.model
- }, COMMAND_CONFIRM_MS)
- return !!confirmed
- }
-
- /**
- * Codex's `/model`: a quick menu of auto presets with an `All models` row (0.145 and later), the full
- * list (`Select Model and Effort`), then the chosen model's reasoning picker, each row pressed by its
- * digit, which selects the row and accepts it. How a row is read and chosen, and why, is in
- * engines/codex/modelPicker.ts. Confirmed, as it always was, by the `thread_settings_applied` record
- * Codex writes once the choice is applied (RuntimeProfileManager `ingestCodex`).
- */
- private async setCodex(session: RegisteredSession, target: RuntimeProfile, catalog: CodexCatalogModel[]): Promise<void> {
- if (!await this.deps.sendText(session.agentId, '/model')) throw new RuntimeProfileControlError('TMUX_FAILED')
- let picker = await this.waitCodexPicker(session, () => true, 900)
- if (!picker) {
- await this.deps.sendKey(session.agentId, 'Enter')
- picker = await this.waitCodexPicker(session, () => true, PICKER_OPEN_MS)
- }
- if (!picker) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
- if (!isCodexList(picker)) {
- // Opened on a reasoning screen: back to the list it was opened from, so the model is chosen too.
- if (!await this.deps.sendKey(session.agentId, 'Escape')) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
- if (!await this.waitCodexPicker(session, isCodexList, PICKER_STEP_MS)) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
- }
- const reached = await this.reachCodexEfforts(session, target, catalog)
- if (reached !== 'applied') await this.pickCodexEffort(session, target, reached)
- if (!await this.deps.manager.waitForProfile(session.sessionId, COMMAND_CONFIRM_MS)) {
- throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
- }
- }
-
- /**
- * From the list on screen to the target model's reasoning picker, through `All models` when the quick
- * menu is first; or 'applied' when the model's row applied it at once (a model with one effort).
- *
- * Every digit is pressed from a list read twice, CODEX_SETTLE_MS apart, the same both times. Codex
- * 0.160 draws the list from its cache and redraws it in place when the server answers, and a model
- * added or reordered then renumbers the rows: a digit read off the first drawing lands on another
- * model. The reasoning picker that opens names its model in its title, and a title naming any other
- * model than the row pressed means the list moved anyway, so nothing more is pressed.
- */
- private async reachCodexEfforts(
- session: RegisteredSession,
- target: RuntimeProfile,
- catalog: CodexCatalogModel[],
- ): Promise<CodexPicker | 'applied'> {
- let reread = false
- for (let lists = 0; lists < 2; lists++) {
- const list = await this.settledCodexList(session)
- if (!list) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
- let choice = chooseCodexRow(list, target, catalog)
- if ('error' in choice && choice.error === 'MODEL_UNAVAILABLE' && !reread) {
- // Codex saves the catalog its picker refreshed from to models_cache.json, so a model or a name
- // that came with the refresh is there to read now.
- reread = true
- catalog = await this.deps.manager.codexCatalog(session)
- choice = chooseCodexRow(list, target, catalog)
- }
- if ('error' in choice) throw new RuntimeProfileControlError(choice.error)
- if (!await this.deps.sendKey(session.agentId, String(choice.row.number))) throw new RuntimeProfileControlError('TMUX_FAILED')
- if (choice.opens === 'list') {
- if (!await this.waitCodexPicker(session, (next) => next.kind === 'models', PICKER_STEP_MS)) {
- throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
- }
- continue
- }
- if (choice.opens === 'applied') return 'applied'
- const next = await this.waitCodexAfterModelRow(session)
- if (next === 'closed') return 'applied'
- if (!next) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
- if (next.model !== choice.row.name) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
- return next
- }
- throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
- }
-
- /** The target's effort in its reasoning picker, through Advanced Reasoning for Max and Ultra. */
- private async pickCodexEffort(session: RegisteredSession, target: RuntimeProfile, picker: CodexPicker): Promise<void> {
- const rows = codexEffortRows(picker)
- let row = target.effort === 'auto' ? rows.defaultRow : rows.efforts.get(target.effort) ?? null
- if (row === null && (target.effort === 'max' || target.effort === 'ultra') && rows.advancedRow !== null
- && codexDigitPressable(rows.advancedRow)) {
- if (!await this.deps.sendKey(session.agentId, String(rows.advancedRow))) throw new RuntimeProfileControlError('TMUX_FAILED')
- const advanced = await this.waitCodexPicker(session, (next) => next.kind === 'advanced', PICKER_STEP_MS)
- row = advanced ? codexEffortRows(advanced).efforts.get(target.effort) ?? null : null
- }
- if (row === null || !codexDigitPressable(row)) throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
- if (!await this.deps.sendKey(session.agentId, String(row))) throw new RuntimeProfileControlError('TMUX_FAILED')
- }
-
- /** The first Codex picker on the pane that `wanted` takes, polled until `timeoutMs`. */
- private async waitCodexPicker(
- session: RegisteredSession,
- wanted: (picker: CodexPicker) => boolean,
- timeoutMs: number,
- ): Promise<CodexPicker | null> {
- const capture = await this.waitPane(session.agentId, (value) => {
- const picker = parseCodexPicker(value)
- return !!picker && wanted(picker)
- }, timeoutMs)
- return capture ? parseCodexPicker(capture) : null
- }
-
- /** A model list that reads the same twice, CODEX_SETTLE_MS apart (see reachCodexEfforts). */
- private async settledCodexList(session: RegisteredSession): Promise<CodexPicker | null> {
- let previous: CodexPicker | null = null
- const deadline = Date.now() + PICKER_STEP_MS
- while (Date.now() < deadline) {
- const capture = await this.deps.capture(session.agentId, 100)
- const picker = capture ? parseCodexPicker(capture) : null
- const list = picker && isCodexList(picker) ? picker : null
- if (list && previous && sameCodexPicker(previous, list)) return list
- previous = list
- await sleep(CODEX_SETTLE_MS)
- }
- return null
- }
-
- /**
- * After a model's row: its reasoning picker, or 'closed' when the picker went away and the composer
- * is back, which is a model the catalog did not describe applying its one effort at once.
- */
- private async waitCodexAfterModelRow(session: RegisteredSession): Promise<CodexPicker | 'closed' | null> {
- const deadline = Date.now() + PICKER_STEP_MS
- while (Date.now() < deadline) {
- const capture = await this.deps.capture(session.agentId, 100)
- if (capture) {
- const picker = parseCodexPicker(capture)
- if (picker?.kind === 'efforts') return picker
- if (!picker && inspectRuntimePane('codex', capture).idle) return 'closed'
- }
- await sleep(100)
- }
- return null
- }
-
- private async setCursor(session: RegisteredSession, target: RuntimeProfile): Promise<void> {
- const cursorTarget = this.deps.manager.cursorTarget(session.sessionId, target.id)
- if (!cursorTarget) throw new RuntimeProfileControlError('MODEL_UNAVAILABLE')
- if (!await this.deps.sendText(session.agentId, `/model ${cursorTarget.familyLabel}`)) {
- throw new RuntimeProfileControlError('TMUX_FAILED')
- }
- let capture = await this.waitPane(session.agentId, (value) => !!parseCursorModelPicker(value), 900)
- if (!capture) {
- if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')
- capture = await this.waitPane(session.agentId, (value) => !!parseCursorModelPicker(value), PICKER_OPEN_MS)
- }
- const picker = capture ? parseCursorModelPicker(capture) : null
- if (!picker || picker.selectedFamily?.toLowerCase() !== cursorTarget.familyLabel.toLowerCase()) {
- throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
- }
-
- if (cursorTarget.rawId !== 'auto') {
- if (!await this.deps.sendKey(session.agentId, 'Tab')) throw new RuntimeProfileControlError('TMUX_FAILED')
- capture = await this.waitPane(session.agentId, (value) => !!parseCursorParameterRows(value), PICKER_STEP_MS)
- if (!capture) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
-
- if (cursorTarget.context) {
- await this.setCursorParameter(session, 'context', cursorTarget.context, true)
- }
- if (cursorTarget.reasoning) {
- await this.setCursorParameter(session, 'reasoning', cursorTarget.reasoning, true)
- }
- await this.setCursorParameter(session, 'thinking', 'true', cursorTarget.thinking ?? false)
- await this.setCursorParameter(session, 'fast', 'true', cursorTarget.fast ?? false)
-
- if (!await this.deps.sendKey(session.agentId, 'Escape')) throw new RuntimeProfileControlError('TMUX_FAILED')
- if (!await this.waitPane(session.agentId, (value) => !!parseCursorModelPicker(value), PICKER_STEP_MS)) {
- throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
- }
- if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')
- } else {
- if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')
- }
-
- const confirmed = await this.waitPane(session.agentId, async (value) => {
- await this.deps.manager.ingestPane(session, value, true)
- return this.deps.manager.selectedModel(session) === target.id
- || this.cursorFooterMatches(value, cursorTarget, target.effort)
- }, COMMAND_CONFIRM_MS)
- if (!confirmed) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
- if (this.deps.manager.selectedModel(session) !== target.id) {
- this.deps.manager.confirmControlProfile(target)
- }
- }
-
- private async setCursorParameter(
- session: RegisteredSession,
- kind: CursorParameterRow['kind'],
- value: string,
- selected: boolean,
- ): Promise<void> {
- const capture = await this.deps.capture(session.agentId, 100)
- const rows = capture ? parseCursorParameterRows(capture) : null
- if (!rows) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
- const target = rows.find((row) => row.kind === kind && row.value === value)
- if (!target) {
- if (!selected && (kind === 'fast' || kind === 'thinking')) return
- throw new RuntimeProfileControlError('EFFORT_UNSUPPORTED')
- }
- if (target.selected === selected) return
- const current = rows.find((row) => row.cursor)
- if (!current) throw new RuntimeProfileControlError('UNSUPPORTED_CLI_VERSION')
- const direction = target.index > current.index ? 'Down' : 'Up'
- for (let i = 0; i < Math.abs(target.index - current.index); i++) {
- if (!await this.deps.sendKey(session.agentId, direction)) throw new RuntimeProfileControlError('TMUX_FAILED')
- await sleep(120)
- }
- if (!await this.deps.sendKey(session.agentId, 'Enter')) throw new RuntimeProfileControlError('TMUX_FAILED')
- const updated = await this.waitPane(session.agentId, (next) => {
- const nextRows = parseCursorParameterRows(next)
- return nextRows?.some((row) => row.kind === kind && row.value === value && row.selected === selected) ?? false
- }, PICKER_STEP_MS)
- if (!updated) throw new RuntimeProfileControlError('CONFIRM_TIMEOUT')
- }
-
- private cursorFooterMatches(
- capture: string,
- target: CursorModelTarget,
- effort: string | null,
- ): boolean {
- const lines = stripAnsi(capture).split('\n').filter((line) => line.trim()).slice(-15)
- if (target.rawId === 'auto') return lines.some((line) => /^\s*Auto(?:\s*$|\s*[·│])/i.test(line))
- return lines.some((line) => {
- const normalized = line.trim().toLowerCase()
- if (!normalized.startsWith(target.familyLabel.toLowerCase())) return false
- const actualFast = /\bfast\b/i.test(line)
- const actualThinking = !/\bno\s+thinking\b/i.test(line) && /\bthinking\b/i.test(line)
- if (target.fast != null && actualFast !== target.fast) return false
- if (target.thinking != null && actualThinking !== target.thinking) return false
- if (target.context && !normalized.includes(target.context)) return false
-
- const effortMatch = /\b(extra high|none|low|medium|high|max)\b/i.exec(line)
- const actualEffort = effortMatch?.[1].toLowerCase().replace(/\s+/g, '') === 'extrahigh'
- ? 'xhigh'
- : effortMatch?.[1].toLowerCase() ?? null
- const footerEffort = target.footerEffort === undefined ? effort : target.footerEffort
- return footerEffort == null ? actualEffort == null : actualEffort === footerEffort
- })
- }
-
- private async waitPane(terminalTarget: string, predicate: (capture: string) => boolean | Promise<boolean>, timeoutMs: number): Promise<string | null> {
- const deadline = Date.now() + timeoutMs
- while (Date.now() < deadline) {
- const capture = await this.deps.capture(terminalTarget, 100)
- if (capture && await predicate(capture)) return capture
- await sleep(100)
- }
- return null
+/** Explicit inline compatibility for direct controller users. */
+export * from './runtimeControl.js'
+export { inspectRuntimePane, paneModal } from '../engines/screens.js'
+import { screenFor } from '../engines/screens.js'
+import { RuntimeProfileController as Controller, type RuntimeProfileControllerDeps } from './runtimeControl.js'
+export class RuntimeProfileController extends Controller {
+ constructor(deps: Omit<RuntimeProfileControllerDeps, 'readScreen'> & Partial<Pick<RuntimeProfileControllerDeps, 'readScreen'>>) {
+ super({ readScreen: async (session, capture) => capture === null ? null : screenFor(session.engine).inspect(capture), ...deps })
}
}
diff --git a/cli/src/lib/screenReader.ts b/cli/src/lib/screenReader.ts
new file mode 100644
index 000000000..e897339d0
--- /dev/null
+++ b/cli/src/lib/screenReader.ts
@@ -0,0 +1,5 @@
+import type { ScreenReading } from '../engines/facets/screen.js'
+import type { RegisteredSession } from './registry.js'
+
+/** Null means unavailable, not an empty screen or a closed question. */
+export type ScreenReader = (session: RegisteredSession, capture: string | null) => Promise<ScreenReading | null>
diff --git a/cli/src/lib/sessionInput.spec.ts b/cli/src/lib/sessionInput.spec.ts
index 0c01cc21f..03623bc36 100644
--- a/cli/src/lib/sessionInput.spec.ts
+++ b/cli/src/lib/sessionInput.spec.ts
@@ -1,3 +1,4 @@
+import { readInlineScreen } from '../testing/inlineScreen.js'
import { readFileSync } from 'node:fs'
import { dirname, join } from 'node:path'
import { fileURLToPath } from 'node:url'
@@ -38,7 +39,7 @@ describe('SessionInputController', () => {
] as const)('%s, %s: tells the person why and what to do, asks no lease again, and awaits nothing', async (engine, reason, text) => {
const onError = vi.fn(), onSubmitted = vi.fn(), sleep = vi.fn(async () => {})
const inject = vi.fn(async () => held(reason))
- const controller = new SessionInputController({ getSession: () => session(engine), validateRuntime: async () => true,
+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session(engine), validateRuntime: async () => true,
inject, sendKey: async () => true, onError, onSubmitted, sleep })
controller.submit('s1', 'a message from the app')
await vi.waitFor(() => expect(onError).toHaveBeenCalledWith('s1', text))
@@ -54,7 +55,7 @@ describe('SessionInputController', () => {
const onError = vi.fn(), onSubmitted = vi.fn()
const inject = vi.fn<() => Promise<TerminalActionResult>>().mockResolvedValueOnce(held('prompt_hidden'))
.mockResolvedValueOnce(held('screen_unreadable')).mockResolvedValue({ state: 'succeeded', dispatch: 'executed' })
- const controller = new SessionInputController({ getSession: () => session('codex'), validateRuntime: async () => true,
+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('codex'), validateRuntime: async () => true,
inject, sendKey: async () => true, onError, onSubmitted, now: time.now, sleep: time.sleep })
controller.submit('s1', 'sent while it starts')
await vi.waitFor(() => expect(onSubmitted).toHaveBeenCalledWith('s1', 'sent while it starts'))
@@ -67,7 +68,7 @@ describe('SessionInputController', () => {
const time = { at: 0, now: () => time.at, sleep: vi.fn(async (ms: number) => { time.at += ms }) }
const onError = vi.fn()
const inject = vi.fn(async () => held('prompt_hidden'))
- const controller = new SessionInputController({ getSession: () => session('claude'), validateRuntime: async () => true,
+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('claude'), validateRuntime: async () => true,
inject, sendKey: async () => true, onError, now: time.now, sleep: time.sleep })
controller.submit('s1', 'into a screen never seen')
await vi.waitFor(() => expect(onError).toHaveBeenCalledWith('s1', 'Claude Code isn\'t showing its prompt; finish what\'s on its screen in its terminal, then send the message again.'))
@@ -79,7 +80,7 @@ describe('SessionInputController', () => {
const withheld: TerminalActionResult = { state: 'unknown', dispatch: 'possibly_executed', reason: 'enter_withheld:permission_open' }
const onError = vi.fn(), onSubmitted = vi.fn(), sendKey = vi.fn(async () => true), forget = vi.fn(), onDelivery = vi.fn()
vi.useFakeTimers()
- const controller = new SessionInputController({ getSession: () => session('claude'), validateRuntime: async () => true,
+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('claude'), validateRuntime: async () => true,
inject: async () => withheld, sendKey, onError, onSubmitted, onDelivery, beforeSubmit: () => forget, beforeTeamWrite: async () => null })
controller.submit('s1', 'a message from the app')
await vi.advanceTimersByTimeAsync(30_000)
@@ -103,7 +104,7 @@ describe('SessionInputController', () => {
it('rejects a message that wants a receipt with the reason, and says why', async () => {
const onError = vi.fn(), onDelivery = vi.fn()
- const controller = new SessionInputController({ getSession: () => session('codex'), validateRuntime: async () => true,
+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('codex'), validateRuntime: async () => true,
inject: async () => held('permission_open'), sendKey: async () => true, onError, onDelivery })
controller.submit('s1', 'a message from the device', 'd1')
await vi.waitFor(() => expect(onDelivery).toHaveBeenCalledWith({ sessionId: 's1', deliveryId: 'd1', state: 'rejected', reason: 'permission_open' }))
@@ -113,7 +114,7 @@ describe('SessionInputController', () => {
it('rejects a team\'s turn with the reason, and tells no person: its team hears of it', async () => {
const onError = vi.fn(), onDelivery = vi.fn()
- const controller = new SessionInputController({ getSession: () => session('claude'), validateRuntime: async () => true,
+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('claude'), validateRuntime: async () => true,
beforeTeamWrite: async () => null, inject: vi.fn(), injectTeam: async () => held('menu_open'), sendKey: async () => true, onError, onDelivery })
controller.submit('s1', 'a team turn', 'team:1')
await vi.waitFor(() => expect(onDelivery).toHaveBeenCalledWith({ sessionId: 's1', deliveryId: 'team:1', state: 'rejected', reason: 'menu_open' }))
@@ -134,7 +135,7 @@ describe('SessionInputController', () => {
it('waits for the lease — a resume whose new process is not confirmed yet — and then delivers, once', async () => {
const time = clock(), onError = vi.fn(), onSubmitted = vi.fn()
const inject = vi.fn<() => Promise<TerminalActionResult>>().mockResolvedValueOnce(refused).mockResolvedValueOnce(refused).mockResolvedValue(done)
- const controller = new SessionInputController({ getSession: () => session('claude'), validateRuntime: async () => true,
+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('claude'), validateRuntime: async () => true,
inject, sendKey: async () => true, onError, onSubmitted, ...time })
controller.submit('s1', 'right after the resume')
await vi.waitFor(() => expect(onSubmitted).toHaveBeenCalledWith('s1', 'right after the resume'))
@@ -147,7 +148,7 @@ describe('SessionInputController', () => {
it('gives up at fifteen seconds, says the message was not delivered, and wrote nothing', async () => {
const time = clock(), onError = vi.fn()
const inject = vi.fn(async () => refused)
- const controller = new SessionInputController({ getSession: () => session('claude'), validateRuntime: async () => true,
+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('claude'), validateRuntime: async () => true,
inject, sendKey: async () => true, onError, ...time })
controller.submit('s1', 'never leased')
await vi.waitFor(() => expect(onError).toHaveBeenCalledWith('s1', 'The message could not be delivered to the agent.'))
@@ -160,14 +161,14 @@ describe('SessionInputController', () => {
const time = clock(), onError = vi.fn()
let present = true
const inject = vi.fn(async () => { present = false; return refused })
- const gone = new SessionInputController({ getSession: () => (present ? session('claude') : undefined), validateRuntime: async () => true,
+ const gone = new SessionInputController({ readScreen: readInlineScreen, getSession: () => (present ? session('claude') : undefined), validateRuntime: async () => true,
inject, sendKey: async () => true, onError, ...time })
gone.submit('s1', 'to an agent that left')
await vi.waitFor(() => expect(onError).toHaveBeenCalled())
expect(inject).toHaveBeenCalledTimes(1)
const other = vi.fn(async (): Promise<TerminalActionResult> => ({ state: 'failed', dispatch: 'not_started', reason: 'terminal agent is unavailable' }))
- const elsewhere = new SessionInputController({ getSession: () => session('claude'), validateRuntime: async () => true,
+ const elsewhere = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('claude'), validateRuntime: async () => true,
inject: other, sendKey: async () => true, onError: vi.fn(), ...time })
elsewhere.submit('s1', 'to an agent with no terminal')
await vi.waitFor(() => expect(other).toHaveBeenCalledTimes(1))
@@ -178,7 +179,7 @@ describe('SessionInputController', () => {
it('waits the same way for a message that wants a receipt, until it is cancelled', async () => {
const time = clock(), onDelivery = vi.fn()
const inject = vi.fn<() => Promise<TerminalActionResult>>().mockResolvedValueOnce(refused).mockResolvedValue(done)
- const controller = new SessionInputController({ getSession: () => session('codex'), validateRuntime: async () => true,
+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('codex'), validateRuntime: async () => true,
inject, sendKey: async () => true, onError: vi.fn(), onDelivery, ...time })
controller.submit('s1', 'with a receipt', 'delivery-1')
await vi.waitFor(() => expect(inject).toHaveBeenCalledTimes(2))
@@ -186,7 +187,7 @@ describe('SessionInputController', () => {
const cancelled = vi.fn(async () => refused)
let cancel: () => void = () => {}
- const waiting = new SessionInputController({ getSession: () => session('codex'), validateRuntime: async () => true,
+ const waiting = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('codex'), validateRuntime: async () => true,
inject: cancelled, sendKey: async () => true, onError: vi.fn(), onDelivery,
now: time.now, sleep: async (ms) => { time.sleep(ms); cancel() } })
cancel = () => waiting.forget('s1')
@@ -200,7 +201,7 @@ describe('SessionInputController', () => {
it('retains the submitted swarm through the input queue and records it only at dispatch', async () => {
const beforeSubmit = vi.fn(() => vi.fn())
- const controller = new SessionInputController({ getSession: () => session('cursor'), validateRuntime: async () => true,
+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('cursor'), validateRuntime: async () => true,
inject: async () => true, sendKey: async () => true, beforeSubmit, onError: vi.fn() })
controller.setTurnOpen('s1', true)
controller.submit('s1', 'task from A', undefined, 'swarm-a')
@@ -212,7 +213,7 @@ describe('SessionInputController', () => {
it.each(['team_waiting_draft', 'team_waiting_user', 'team_waiting_idle'])('team input preserves the composer on %s', async reason => {
const inject = vi.fn(async () => true), sendKey = vi.fn(async () => true), onDelivery = vi.fn()
- const controller = new SessionInputController({ getSession: () => session('cursor'), validateRuntime: async () => true,
+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('cursor'), validateRuntime: async () => true,
inject, sendKey, onError: vi.fn(), onDelivery, beforeTeamWrite: async () => reason })
controller.submit('s1', 'peer question', 'team:fixture')
await vi.waitFor(() => expect(onDelivery).toHaveBeenCalledWith(expect.objectContaining({ state: 'rejected', reason })))
@@ -223,7 +224,7 @@ describe('SessionInputController', () => {
it('rechecks a queued team message at the actual write boundary', async () => {
const beforeTeamWrite = vi.fn(async () => 'team_waiting_draft'), inject = vi.fn(async () => true), onDelivery = vi.fn()
- const controller = new SessionInputController({ getSession: () => session(), validateRuntime: async () => true,
+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session(), validateRuntime: async () => true,
inject, sendKey: async () => true, onError: vi.fn(), onDelivery, beforeTeamWrite })
controller.setTurnOpen('s1', true)
controller.submit('s1', 'peer question', 'team:fixture')
@@ -237,7 +238,7 @@ describe('SessionInputController', () => {
it('never retries Enter for an automatic team message with uncertain acceptance', async () => {
vi.useFakeTimers()
const sendKey = vi.fn(async () => true), onDelivery = vi.fn()
- const controller = new SessionInputController({ getSession: () => session(), validateRuntime: async () => true,
+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session(), validateRuntime: async () => true,
inject: async () => true, sendKey, onError: vi.fn(), onDelivery, beforeTeamWrite: async () => null })
controller.submit('s1', 'peer question', 'team:fixture')
await vi.advanceTimersByTimeAsync(12_000)
@@ -248,7 +249,7 @@ describe('SessionInputController', () => {
it('requeues a team notice when a draft appears while waiting for the terminal writer', async () => {
const inject = vi.fn(async () => true), onError = vi.fn(), onDelivery = vi.fn()
- const controller = new SessionInputController({ getSession: () => session(), validateRuntime: async () => true,
+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session(), validateRuntime: async () => true,
beforeTeamWrite: async () => null, inject,
injectTeam: async () => ({ state: 'failed', dispatch: 'not_started', reason: 'team_waiting_draft' }),
sendKey: async () => true, onError, onDelivery })
@@ -267,7 +268,7 @@ describe('SessionInputController', () => {
{ engine: 'codex', observed: '<pasted_content id="04e3">\npeer question\n</pasted_content id="04e3">', state: 'unknown' },
] as const)('attributes native paste evidence only to the exact $engine message: $state', async ({ engine, observed, state }) => {
const inject = vi.fn(async () => true), sendKey = vi.fn(async () => true), onDelivery = vi.fn()
- const controller = new SessionInputController({ getSession: () => session(engine), validateRuntime: async () => true,
+ const controller = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session(engine), validateRuntime: async () => true,
inject, sendKey, onError: vi.fn(), onDelivery, beforeTeamWrite: async () => null })
controller.submit('s1', 'peer question', 'team:fixture')
await vi.waitFor(() => expect(onDelivery).toHaveBeenCalledWith(expect.objectContaining({ state: 'delivered' })))
@@ -282,7 +283,7 @@ describe('SessionInputController', () => {
// A voice command spoken while a Codex task ran used to sit in this controller's queue, invisible,
// until the task ended. Codex queues composer input itself, so the daemon types straight away.
const injected: string[] = []
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('codex'), validateRuntime: async () => true,
inject: async (_pane, content) => { injected.push(content); return true },
sendKey: async () => true, onError: vi.fn(),
@@ -303,7 +304,7 @@ describe('SessionInputController', () => {
it('queues Command Code prompts while busy and drains exactly one after turn end', async () => {
const injected: string[] = []
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('commandcode'), validateRuntime: async () => true,
inject: async (_pane, content) => { injected.push(content); return true },
sendKey: async () => true, onError: vi.fn(),
@@ -322,7 +323,7 @@ describe('SessionInputController', () => {
it('clears the echoed prompt from the Cursor composer once the turn starts', async () => {
const sendKey = vi.fn(async (_pane: string, _key: string) => true)
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('cursor'), validateRuntime: async () => true,
inject: async () => true, sendKey,
capture: async () => '⠰ Working\n\n→ hello\nAuto', // the submitted prompt is still on screen
@@ -336,7 +337,7 @@ describe('SessionInputController', () => {
it('leaves a fresh terminal draft alone when the Cursor composer no longer echoes our prompt', async () => {
const sendKey = vi.fn(async (_pane: string, _key: string) => true)
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('cursor'), validateRuntime: async () => true,
inject: async () => true, sendKey,
capture: async () => '⠰ Working\n\n→ something the user just typed\nAuto',
@@ -354,7 +355,7 @@ describe('SessionInputController', () => {
// message is typed onto the end of it and the two are submitted as one run-on prompt — observed as a
// turn starting with the PREVIOUS message's text.
const order: string[] = []
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('cursor'), validateRuntime: async () => true,
inject: async (_pane, content) => { order.push(`inject:${content}`); return true },
sendKey: async (_pane, key) => { order.push(`key:${key}`); return true },
@@ -371,7 +372,7 @@ describe('SessionInputController', () => {
vi.useFakeTimers()
const inject = vi.fn(async () => true)
const sendKey = vi.fn(async () => true)
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session(), validateRuntime: async () => true, inject, sendKey, onError: vi.fn(),
})
controller.submit('s1', 'hello')
@@ -385,7 +386,7 @@ describe('SessionInputController', () => {
vi.useFakeTimers()
const inject = vi.fn(async () => true)
const sendKey = vi.fn(async () => true)
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('claude'), validateRuntime: async () => true, inject, sendKey, onError: vi.fn(),
})
controller.submit('s1', 'hello')
@@ -401,7 +402,7 @@ describe('SessionInputController', () => {
const inject = vi.fn(async () => true)
const sendKey = vi.fn(async (_pane: string, _key: string) => true)
const onError = vi.fn()
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('cursor'),
validateRuntime: async () => true,
inject,
@@ -424,7 +425,7 @@ describe('SessionInputController', () => {
it('retries Enter for Cursor only while the exact draft remains in an idle composer', async () => {
vi.useFakeTimers()
const sendKey = vi.fn(async (_pane: string, _key: string) => true)
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('cursor'),
validateRuntime: async () => true,
inject: async () => true,
@@ -444,7 +445,7 @@ describe('SessionInputController', () => {
it('waits for the Cursor composer to settle before draining the next prompt', async () => {
vi.useFakeTimers()
const injected: string[] = []
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('cursor'),
validateRuntime: async () => true,
inject: async (_pane, content) => { injected.push(content); return true },
@@ -466,7 +467,7 @@ describe('SessionInputController', () => {
vi.useFakeTimers()
const sendKey = vi.fn(async () => true)
const onError = vi.fn()
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('claude'),
validateRuntime: async () => true,
inject: async () => true,
@@ -491,7 +492,7 @@ describe('SessionInputController', () => {
vi.useFakeTimers()
const sendKey = vi.fn(async () => true)
const onError = vi.fn()
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('commandcode'),
validateRuntime: async () => true,
inject: async () => true,
@@ -514,7 +515,7 @@ describe('SessionInputController', () => {
const sendKey = vi.fn(async () => true)
const onError = vi.fn()
const capture = vi.fn(async () => '❯ What are you working on?\n✻ Waiting for 4 background agents to finish\n────\n❯ a different draft I am still writing\n────')
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('claude'), validateRuntime: async () => true,
inject: async () => true, sendKey, capture, onError,
})
@@ -541,7 +542,7 @@ describe('SessionInputController', () => {
vi.useFakeTimers()
const sendKey = vi.fn(async () => true), onError = vi.fn(), onDelivery = vi.fn()
let pasted = false
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('claude'), validateRuntime: async () => true, onDelivery,
inject: async () => { pasted = true; return true }, sendKey, capture: async () => pasted ? screen : '❯ ', onError,
})
@@ -556,7 +557,7 @@ describe('SessionInputController', () => {
vi.useFakeTimers()
const sendKey = vi.fn(async () => true), onError = vi.fn(), onDelivery = vi.fn()
let pasted = false
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('claude'), validateRuntime: async () => true, onDelivery,
inject: async () => { pasted = true; return true }, sendKey, capture: async () => pasted ? screen : '❯ ', onError,
})
@@ -572,7 +573,7 @@ describe('SessionInputController', () => {
const sendKey = vi.fn(async () => true), onError = vi.fn()
// Codex browsing its transcript (0.160): Enter would rewind the conversation to the prompt in view.
const browsing = `› ${message}\n\n Browsing transcript · ↑↓/jk scroll · ←→/hl prompts · ↵ rewind · esc back\n`
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('codex'), validateRuntime: async () => true,
inject: async () => true, sendKey, capture: async () => browsing, onError,
})
@@ -588,7 +589,7 @@ describe('SessionInputController', () => {
vi.useFakeTimers()
const sendKey = vi.fn(async () => true)
const onError = vi.fn()
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('claude'), validateRuntime: async () => true,
inject: async () => true, sendKey, capture: async () => 'Sign in required', onError,
})
@@ -603,7 +604,7 @@ describe('SessionInputController', () => {
vi.useFakeTimers()
const sendKey = vi.fn(async () => true)
const onError = vi.fn()
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('claude'),
validateRuntime: async () => true,
inject: async () => true,
@@ -624,7 +625,7 @@ describe('SessionInputController', () => {
vi.useFakeTimers()
const sendKey = vi.fn(async () => true)
const onError = vi.fn()
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('codex'),
validateRuntime: async () => true,
inject: async () => true,
@@ -645,7 +646,7 @@ describe('SessionInputController', () => {
vi.useFakeTimers()
const sendKey = vi.fn(async () => true)
const onError = vi.fn()
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('codex'),
validateRuntime: async () => true,
inject: async () => true,
@@ -666,7 +667,7 @@ describe('SessionInputController', () => {
vi.useFakeTimers()
const sendKey = vi.fn(async () => true)
const onError = vi.fn()
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('codex'),
validateRuntime: async () => true,
inject: async () => true,
@@ -689,7 +690,7 @@ describe('SessionInputController', () => {
const ambiguous: TerminalActionResult = {
state: 'unknown', dispatch: 'possibly_executed', reason: 'response was lost',
}
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('codex'),
validateRuntime: async () => true,
inject: async () => ambiguous,
@@ -711,7 +712,7 @@ describe('SessionInputController', () => {
const ambiguous: TerminalActionResult = {
state: 'unknown', dispatch: 'possibly_executed', reason: 'response was lost',
}
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('codex'),
validateRuntime: async () => true,
inject: async () => ambiguous,
@@ -734,7 +735,7 @@ describe('SessionInputController', () => {
const ambiguous: TerminalActionResult = {
state: 'unknown', dispatch: 'possibly_executed', reason: 'response was lost',
}
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('codex'),
validateRuntime: async () => true,
inject: async () => ambiguous,
@@ -758,7 +759,7 @@ describe('SessionInputController', () => {
state: 'unknown', dispatch: 'possibly_executed', reason: 'Enter response was lost',
}))
const onError = vi.fn()
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('codex'),
validateRuntime: async () => true,
inject: async () => true,
@@ -782,7 +783,7 @@ describe('SessionInputController', () => {
const ambiguous: TerminalActionResult = {
state: 'unknown', dispatch: 'possibly_executed', reason: 'response was lost',
}
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('cursor'),
validateRuntime: async () => true,
inject: async () => ambiguous,
@@ -806,7 +807,7 @@ describe('SessionInputController', () => {
state: 'unknown', dispatch: 'possibly_executed', reason: 'key response was lost',
}))
const onError = vi.fn()
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('cursor'),
validateRuntime: async () => true,
inject: async () => true,
@@ -825,7 +826,7 @@ describe('SessionInputController', () => {
it('serializes chat input behind a native runtime control lock', async () => {
const injected: string[] = []
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('claude'), validateRuntime: async () => true,
inject: async (_pane, content) => { injected.push(content); return true },
sendKey: async () => true, onError: vi.fn(),
@@ -849,7 +850,7 @@ describe('delivery correlation', () => {
function setup(overrides: Partial<ConstructorParameters<typeof SessionInputController>[0]> = {}) {
const onDelivery = vi.fn()
const inject = vi.fn(async () => true)
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session(), validateRuntime: async () => true,
inject, sendKey: async () => true, onError: vi.fn(), onDelivery, ...overrides,
})
@@ -982,7 +983,7 @@ describe('optional lamp delivery preserves legacy behavior', () => {
// once: the second waits for the first's paste, not for the first's turn.
const validations: Array<(valid: boolean) => void> = []
const inject = vi.fn(async (_target: string, _text: string) => true)
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('claude'),
validateRuntime: () => new Promise<boolean>(resolve => validations.push(resolve)),
inject, sendKey: async () => true, onError: vi.fn(),
@@ -1003,7 +1004,7 @@ describe('optional lamp delivery preserves legacy behavior', () => {
it('writes messages that arrive together in the order they arrived, whichever check finishes first', async () => {
const checks: Array<(valid: boolean) => void> = []
const inject = vi.fn(async (_target: string, _text: string) => true)
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('codex'),
validateRuntime: () => new Promise<boolean>(resolve => checks.push(resolve)),
inject, sendKey: async () => true, onError: vi.fn(),
@@ -1017,7 +1018,7 @@ describe('optional lamp delivery preserves legacy behavior', () => {
expect(inject.mock.calls.map(([, text]) => text)).toEqual(['one', 'two', 'three'])
// A write that fails does not hold up the ones behind it.
const failing = vi.fn(async (_target: string, text: string) => { if (text === 'bad') throw new Error('pane gone'); return true })
- const next = new SessionInputController({ getSession: () => session('codex'), validateRuntime: async () => true,
+ const next = new SessionInputController({ readScreen: readInlineScreen, getSession: () => session('codex'), validateRuntime: async () => true,
inject: failing, sendKey: async () => true, onError: vi.fn() })
next.submit('s1', 'bad')
next.submit('s1', 'good')
@@ -1028,7 +1029,7 @@ describe('optional lamp delivery preserves legacy behavior', () => {
it('keeps native control available during untracked runtime validation', async () => {
let resolve!: (valid: boolean) => void
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('claude'),
validateRuntime: () => new Promise<boolean>(done => { resolve = done }),
inject: async () => true, sendKey: async () => true, onError: vi.fn(),
@@ -1045,7 +1046,7 @@ describe('optional lamp delivery preserves legacy behavior', () => {
it('reports the original legacy cancellation error for a vanished process', async () => {
const onError = vi.fn()
const sendKey = vi.fn(async () => true)
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session(), validateRuntime: async () => false,
inject: async () => true, sendKey, onError,
})
@@ -1059,7 +1060,7 @@ describe('optional lamp delivery preserves legacy behavior', () => {
vi.useFakeTimers()
const onDelivery = vi.fn()
const sendKey = vi.fn(async () => true)
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('commandcode'), validateRuntime: async () => true,
inject: async () => true, sendKey, capture: async () => 'Thinking… esc to interrupt',
onError: vi.fn(), onDelivery,
@@ -1076,7 +1077,7 @@ describe('optional lamp delivery preserves legacy behavior', () => {
it('does not invent rejection when a turn begins during pre-paste validation', async () => {
let resolve!: (valid: boolean) => void
const onDelivery = vi.fn()
- const controller = new SessionInputController({
+ const controller = new SessionInputController({ readScreen: readInlineScreen,
getSession: () => session('claude'),
validateRuntime: () => new Promise<boolean>(done => { resolve = done }),
inject: async () => true, sendKey: async () => true, onError: vi.fn(), onDelivery,
diff --git a/cli/src/lib/sessionInput.ts b/cli/src/lib/sessionInput.ts
index ed54b8b02..d180e117b 100644
--- a/cli/src/lib/sessionInput.ts
+++ b/cli/src/lib/sessionInput.ts
@@ -1,7 +1,8 @@
+import type { ScreenReader } from './screenReader.js'
import { createHash } from 'crypto'
import type { RegisteredSession } from './registry.js'
import { sid } from './log.js'
-import { isMessageHold, messageHold, messageHoldText, messageWithheldText, passingHold, type MessageHold } from './messageHold.js'
+import { isMessageHold, messageHoldText, messageWithheldText, passingHold, type MessageHold } from './messageHolds.js'
import { enterWithheldReason, TERMINAL_LEASE_REFUSED, type TerminalActionResult } from './terminalTypes.js'
const MAX_QUEUE_ITEMS = 8
@@ -105,6 +106,7 @@ interface InputState {
}
export interface SessionInputDeps {
+ readScreen: ScreenReader
/** A team-only preflight at the actual write boundary. A reason proves no paste occurred. */
beforeTeamWrite?: (session: RegisteredSession) => Promise<string | null>
onDelivery?: (event: SessionInputDelivery) => void
@@ -645,7 +647,7 @@ export class SessionInputController {
if (session.engine === 'cursor') {
const capture = await this.deps.capture?.(session.agentId)
if (!state.awaitingFingerprint || state.turnOpen) return
- if (this.dialogOverComposer(sessionId, session, state, capture)) return
+ if (await this.dialogOverComposer(sessionId, session, state, capture)) return
const draftPending = !!capture && cursorComposerContains(capture, state.awaitingContent ?? '')
if (state.ambiguousDispatch && !draftPending) {
this.failAmbiguousSubmission(sessionId, state)
@@ -684,7 +686,7 @@ export class SessionInputController {
// claude/codex/commandcode: verify against the terminal before pressing Enter again or declaring failure.
const capture = await this.deps.capture?.(session.agentId)
if (!state.awaitingFingerprint || state.turnOpen) return
- if (this.dialogOverComposer(sessionId, session, state, capture)) return
+ if (await this.dialogOverComposer(sessionId, session, state, capture)) return
// Command Code writes the user line to its transcript only once the model has finished THINKING, so
// the turn_started this used to wait for can be half a minute late on a real task — and the user
// watched the terminal accept the message and start working while the device claimed it had been
@@ -775,8 +777,11 @@ export class SessionInputController {
* agent asks means the message was taken and its turn has reached a tool; anything else leaves it
* unknown, as does any delivery with a receipt, which the pane alone cannot settle.
*/
- private dialogOverComposer(sessionId: string, session: RegisteredSession, state: InputState, capture: string | null | undefined): boolean {
- const hold = capture ? messageHold(session.engine, capture) : null
+ private async dialogOverComposer(sessionId: string, session: RegisteredSession, state: InputState, capture: string | null | undefined): Promise<boolean> {
+ if (capture === undefined) return false
+ const screen = await this.deps.readScreen(session, capture)
+ if (this.states.get(sessionId) !== state || state.turnOpen || !state.awaitingFingerprint) return true
+ const hold = screen ? screen.messageHold : 'screen_unreadable'
if (!hold) return false
if (state.deliveryId || (hold !== 'permission_open' && hold !== 'question_open')) {
this.failAmbiguousSubmission(sessionId, state)
diff --git a/cli/src/lib/teamWriteHold.ts b/cli/src/lib/teamWriteHold.ts
index d649d4719..74b174d6d 100644
--- a/cli/src/lib/teamWriteHold.ts
+++ b/cli/src/lib/teamWriteHold.ts
@@ -1,38 +1,2 @@
-import { stripVTControlCharacters } from 'node:util'
-import type { RegisteredSession } from './registry.js'
-import { inspectRuntimePane } from './runtimeProfileController.js'
-import { parseEngineQuestionPane } from './askQuestion.js'
-
-/**
- * Conservative, read-only inspection at the serialized input write boundary. The core's: it writes a team's
- * turn only into a ready composer (core/input.ts), and closes an agent only from one (lib/closeAgentService.ts).
- * Here rather than in teams/ since Tab collaboration left the core's process (step 8); teams/preflight.ts says
- * it as it always did.
- */
-export function teamWriteHold(engine: RegisteredSession['engine'], capture: string | null): string | null {
- if (!capture) return 'team_waiting_unavailable'
- const pane = inspectRuntimePane(engine, capture)
- if (pane.dialog || parseEngineQuestionPane(engine, capture)) return 'team_waiting_user'
- if (pane.draft) return 'team_waiting_draft'
- if (!pane.idle) return 'team_waiting_idle'
- const lines = capture.split('\n').map(stripVTControlCharacters)
- if (/esc(?:ape)? to (?:interrupt|cancel|stop)/i.test(lines.slice(-15).join('\n'))) return 'team_waiting_idle'
- // These composers permit an empty first line followed by a multiline human draft.
- // The general profile inspector only needs the first line; team delivery needs every line.
- if (['claude', 'codex', 'cursor', 'hermes'].includes(engine)) {
- const index = lines.findLastIndex(line => /^\s*[›❯→]/u.test(line))
- if (index < 0) return 'team_waiting_idle'
- // Codex shows its styled placeholder only when the whole composer is empty.
- // Its configurable model/cwd/task footer below that placeholder is not draft
- // text. The profile inspector has already proved this visible prompt text is
- // a placeholder; a bare first line still needs the multiline-draft check.
- if (engine === 'codex' && lines[index].replace(/^\s*[›❯→]\s*/u, '').trim()) return null
- for (const line of lines.slice(index + 1)) {
- if (/[─━―]{8,}/u.test(line)) break
- const text = line.trim()
- if (!text || /^(?:\? for shortcuts|\d+% context|context:|ctrl\+|shift\+|tab to|plan mode|accept edits|bypass permissions)/i.test(text)) continue
- return 'team_waiting_draft'
- }
- }
- return null
-}
+/** Inline compatibility API. Supervised core reads the engine worker. */
+export { teamWriteHold } from '../engines/screens.js'
diff --git a/cli/src/lib/terminalActivity.ts b/cli/src/lib/terminalActivity.ts
index 5745860ea..dd5d507ef 100644
--- a/cli/src/lib/terminalActivity.ts
+++ b/cli/src/lib/terminalActivity.ts
@@ -1,33 +1,7 @@
-/** Read only an engine's live footer. No generated status words or transcript inference. */
-export function terminalActivity(engine: string, screen: string | null): string | null {
- return terminalActivityReading(engine, screen)?.label ?? null
-}
-
-/** Keep the live indicator as well as its label: successive changes to its
- * timer/spinner prove the UI is alive; an abandoned static screen does not. */
-export function terminalActivityReading(engine: string, screen: string | null): { label: string; indicator: string } | null {
- if (!screen || (engine !== 'claude' && engine !== 'codex')) return null
- const lines = screen.replace(/\x1b\][^\x07]*(?:\x07|\x1b\\)/g, '')
- .replace(/\x1b\[[0-?]*[ -/]*[@-~]/g, '').split(/\r?\n/)
- let end = lines.length
- if (engine === 'claude') {
- // The expanded agent tree sits BELOW the prompt and can push the live
- // spinner out of the last 16 rows. Anchor to the actual input/footer pair,
- // retaining a short search above it instead of scanning old output.
- for (let i = lines.length - 1; i >= Math.max(0, lines.length - 64); i--) {
- if (/^\s*[❯›>](?:\s|$)/u.test(lines[i]) &&
- lines.slice(i + 1, i + 9).some(line => /shift\+tab to cycle|esc to interrupt/i.test(line))) {
- end = i
- break
- }
- }
- }
- for (let i = end - 1; i >= Math.max(0, end - 16); i--) {
- const line = lines[i].trim()
- const match = engine === 'codex'
- ? /^[•◦⠁-⣿]\s+(.{1,60}?)\s+\([^\n]*\besc to interrupt\b[^\n]*\)\s*$/i.exec(line)
- : /^[✢✳✶✻✽·*]\s+([\p{L}][\p{L}\p{N} '\u2019-]{0,55}(?:…|\.{3}))(?:\s+\([^\n]*\))?\s*$/u.exec(line)
- if (match) return { label: match[1].trim().replace(/…/g, '...'), indicator: line }
- }
- return null
+/** Inline compatibility; supervised core reads engine screen evidence. */
+import { activity as claude } from '../engines/claude/activity.js'
+import { activity as codex } from '../engines/codex/activity.js'
+export function terminalActivityReading(engine: string, screen: string | null) {
+ return !screen ? null : engine === 'claude' ? claude(screen) : engine === 'codex' ? codex(screen) : null
}
+export function terminalActivity(engine: string, screen: string | null): string | null { return terminalActivityReading(engine, screen)?.label ?? null }
diff --git a/cli/src/lib/terminalClose.real.spec.ts b/cli/src/lib/terminalClose.real.spec.ts
index c7bfec041..1715aa3c1 100644
--- a/cli/src/lib/terminalClose.real.spec.ts
+++ b/cli/src/lib/terminalClose.real.spec.ts
@@ -1,10 +1,11 @@
+import { screenFor } from '../engines/screens.js'
import { readFile, readdir, stat } from 'node:fs/promises'
import { join } from 'node:path'
import { expect, it, vi } from 'vitest'
import { env } from '../config/env.js'
import { createTerminalControl } from '../core/terminals/control.js'
import { isolatedTmux } from '../testing/isolatedTmux.js'
-import { CloseAgentService, inspectCloseActivity } from './closeAgentService.js'
+import { CloseAgentService, inspectCloseActivity as inspectCloseReading } from './closeAgentService.js'
import { registry } from './registry.js'
import { AgentRestartCoordinator } from './restartAgent.js'
import { SessionCheckpointStore } from './sessionCheckpoint.js'
@@ -86,3 +87,7 @@ it.runIf(process.env.RUN_REAL_TMUX_DISCOVERY === '1')('saves and closes an inact
await server.close()
}
}, 15_000)
+
+function inspectCloseActivity(session: Parameters<typeof inspectCloseReading>[0], capture: string | null, turnOpen: boolean | undefined, needsInput: boolean) {
+ return inspectCloseReading(session, capture === null ? null : screenFor(session.engine).inspect(capture), turnOpen, needsInput)
+}
diff --git a/cli/src/lib/unusedCodexClose.spec.ts b/cli/src/lib/unusedCodexClose.spec.ts
index 65751180b..574bf60cb 100644
--- a/cli/src/lib/unusedCodexClose.spec.ts
+++ b/cli/src/lib/unusedCodexClose.spec.ts
@@ -1,3 +1,4 @@
+import { screenFor } from '../engines/screens.js'
import { afterEach, beforeEach, expect, it, vi } from 'vitest'
import { mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs'
import { join } from 'node:path'
@@ -5,7 +6,7 @@ import { env } from '../config/env.js'
import { registry, type RegisteredSession } from './registry.js'
import { stoppedAgents } from './stoppedAgents.js'
import { AgentRestartCoordinator } from './restartAgent.js'
-import { CloseAgentService, inspectCloseActivity } from './closeAgentService.js'
+import { CloseAgentService, inspectCloseActivity as inspectCloseReading } from './closeAgentService.js'
import { createStopAgentService } from './stopAgentService.js'
import { SessionCheckpointStore } from './sessionCheckpoint.js'
import { processRows } from './tmux.js'
@@ -151,3 +152,7 @@ it.runIf(process.env.RUN_REAL_TMUX_DISCOVERY === '1')('closes only the exited cl
vi.unstubAllEnvs()
}
}, 15_000)
+
+function inspectCloseActivity(session: Parameters<typeof inspectCloseReading>[0], capture: string | null, turnOpen: boolean | undefined, needsInput: boolean) {
+ return inspectCloseReading(session, capture === null ? null : screenFor(session.engine).inspect(capture), turnOpen, needsInput)
+}
diff --git a/cli/src/services/inline.spec.ts b/cli/src/services/inline.spec.ts
index 11136ad8e..614c95f34 100644
--- a/cli/src/services/inline.spec.ts
+++ b/cli/src/services/inline.spec.ts
@@ -1,3 +1,4 @@
+import { screenFor } from '../engines/screens.js'
import { describe, expect, it } from 'vitest'
import { engineTranscriptFor } from '../engines/transcripts.js'
import { liveFor } from '../engines/live.js'
@@ -25,6 +26,6 @@ import { startWorkspaces } from './workspaces.js'
describe('the services the core runs in its own process only when they do not run in theirs', () => {
it('are their own starts, unchanged: the same services either way', () => {
- expect({ ...inline }).toEqual({ engineTranscriptFor, liveFor, runtimeFor, startCommandBar, startDevices, startHandoff, startGateway, startModels, startMonitor, startOrchestrator, startProjects, startRecaps, startSearch, startSharing, startShell, startStore, startTeamsInCore, startUsage, startViewers, startWifi, startWindowNames, startWorkspaces })
+ expect({ ...inline }).toEqual({ engineTranscriptFor, liveFor, runtimeFor, screenFor, startCommandBar, startDevices, startHandoff, startGateway, startModels, startMonitor, startOrchestrator, startProjects, startRecaps, startSearch, startSharing, startShell, startStore, startTeamsInCore, startUsage, startViewers, startWifi, startWindowNames, startWorkspaces })
})
})
diff --git a/cli/src/services/inline.ts b/cli/src/services/inline.ts
index 9c7086042..2977f7afe 100644
--- a/cli/src/services/inline.ts
+++ b/cli/src/services/inline.ts
@@ -14,6 +14,7 @@
*/
export { engineTranscriptFor } from '../engines/transcripts.js'
export { liveFor } from '../engines/live.js'
+export { screenFor } from '../engines/screens.js'
export { runtimeFor } from '../engines/runtime.js'
export { startGateway } from '../gateway/start.js'
export { startCommandBar } from './commandBar.js'
diff --git a/cli/src/testing/inlineScreen.ts b/cli/src/testing/inlineScreen.ts
new file mode 100644
index 000000000..ffa1a56f4
--- /dev/null
+++ b/cli/src/testing/inlineScreen.ts
@@ -0,0 +1,8 @@
+/** Real native readers injected into unit hosts; production supervision never imports this. */
+import { readerEngine } from '../engines/worker/protocol.js'
+import { screenFor } from '../engines/screens.js'
+import { legacyScreen } from '../lib/legacyScreen.js'
+import type { ScreenReader } from '../lib/screenReader.js'
+export const readInlineScreen: ScreenReader = async (session, capture) => readerEngine(session.engine)
+ ? capture === null ? null : screenFor(session.engine).inspect(capture)
+ : legacyScreen(session.engine, capture)
diff --git a/docs/design/2026-10-08-engine-screen.md b/docs/design/2026-10-08-engine-screen.md
new file mode 100644
index 000000000..08a6e0b86
--- /dev/null
+++ b/docs/design/2026-10-08-engine-screen.md
@@ -0,0 +1,66 @@
+# Engine screen interpretation
+
+Claude Code and Codex interpret terminal captures through `Engine.screen` in their
+existing engine worker. Core captures the terminal and retains the input lease,
+session binding, turn state and question lifecycle. A worker receives text and
+returns facts; it has no terminal handle or authority to type, close or restart.
+
+The facet reports the current prompt, draft, modal, question, message/team hold,
+activity indicator and stopped-goal evidence. The engine folders own native
+composers, takeover screens and activity footers. Shared ANSI and numbered-dialog
+mechanics remain in `engines/kit`; other engines retain their existing readers.
+Recorded screen fixtures are unchanged.
+
+## Consumers
+
+- Message admission checks the screen before a paste and again before Enter.
+ A permission prompt appearing between those operations still withholds Enter.
+- Question polling and answers await engine evidence. An unavailable reader is
+ not an empty pane: existing questions remain open, and no answer is typed.
+- Model control and retargeting use the reported pane for their preflight.
+- Close checks and activity sampling consume reported facts. Core decides whether
+ evidence establishes inactivity and whether a close is still authorized.
+
+The question watcher keeps one pending capture/interpretation per session. Stop,
+rebind and replacement invalidate it. Before a Harness prompt is typed, its
+already-read question view becomes the next turn's baseline; a second late read
+cannot mistake that turn's own question for a preceding one.
+
+## Private transport and failure
+
+`engine_screen_capabilities` and `engine_screen_read` are private methods on the
+existing engine links. They require core's local owner identity with no client
+connection. They are absent from the public client router. Version 1 allows:
+
+| Bound | Value |
+| --- | --- |
+| Capture | 256 KiB of UTF-8 |
+| Reply | 1 MiB including the envelope |
+| Concurrent reads | 8 per engine, with at most 64 waiting in FIFO order |
+| Core deadline | 1 second including queueing, capability negotiation and worker startup |
+
+Core validates reply fields, pane flags, question rows and hold values. Each
+result is fenced to the connection generation and the session/process/terminal
+identity observed before sending it. No screen is cached across calls. A timeout,
+malformed reply or changed identity supplies no evidence. Message and question
+writes fail closed; close/activity checks remain unknown. The existing bounded
+input queue may retry an unreadable screen, without typing anything first.
+
+Worker startup and async loading have a deadline and recycle the worker on a
+timeout. A synchronous parser stall is contained by core's deadline and the
+master's existing heartbeat/restart policy. It can cost that engine's other
+facets, never core's event loop or the other engine worker.
+
+The parent-bound `HARNESSD_ENGINE_SCREEN=<master pid>:1` selects supervision.
+Explicit inline mode and older masters select compatibility during composition.
+A failed worker never selects an inline parser. No worker starts for an empty
+core merely to install this facet.
+
+## Remaining engine isolation
+
+This moves the screen reads described above; the full engine boundary remains in progress. Native model
+picker drivers in `lib/runtimeControl.ts`, answer key selection and fingerprint
+normalization in `lib/questionController.ts`, submission verification, hooks and
+installers, launch/discovery/resume and native control connections still need
+their own engine facets/process boundaries. Core must retain effect authority
+and identity fencing when those drivers move.