The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.
TypeScript 3d-modelingagent-frameworkagent-harnessai-agentscadclaude-codecodexcoding-agentsdeveloper-toolsesp32game-developmentmcp-servermulti-agentopen-hardwareopencodeopenharnesspcbroboticsterminaltmux
Isolate Claude Code and Codex live transcript parsing (#1019)
Dee5eb5c1879c56
diff --git a/cli/AGENTS.md b/cli/AGENTS.mdindex 910f58a2f..1858cea35 100644--- a/cli/AGENTS.md+++ b/cli/AGENTS.md@@ -25,7 +25,7 @@ they show as `harnessd`, `harnessd-core` and `harnessd-<process>` (`src/harnessd|---|---|---|| master | supervision only (`src/harnessd/`) | by `harness start`, `harness start -f`, or launchd or systemd after `harness service install` || core | sessions (`src/core/`) | always |-| engine-claude, engine-codex | history and last-turn readers only | on demand: a read for that engine |+| engine-claude, engine-codex | history, last-turn readers and live transcript parsers | on demand: an attach or read for that engine || search | session search | always || viewers | the harness viewers, their remote streams and rendered surfaces, and the Store | always || edge | the shell service, workspaces, usage, the monitor, the project readers, the change-agent handoff, the recaps | always |diff --git a/cli/e2e/engineReaders.e2e.ts b/cli/e2e/engineReaders.e2e.tsindex c73a3922d..31ce56a08 100644--- a/cli/e2e/engineReaders.e2e.ts+++ b/cli/e2e/engineReaders.e2e.ts@@ -1,5 +1,5 @@/** Private daemon/home/tmux; real supervised reader processes and the engines' recorded wire format. */-import { mkdirSync, readFileSync } from 'node:fs'+import { mkdirSync, readFileSync, writeFileSync } from 'node:fs'import { join } from 'node:path'import { afterEach, describe, expect, it, onTestFailed } from 'vitest'import { LocalClient } from './harness/client.js'@@ -45,7 +45,8 @@ describe('Claude Code and Codex readers in their own processes', () => {const { d, c } = await fresh()expect(readerPid(d, 'claude')).toBeUndefined()expect(readerPid(d, 'codex')).toBeUndefined()- for (const type of ['engine_history_page', 'engine_last_turn']) {+ for (const type of ['engine_history_page', 'engine_last_turn', 'engine_live_capabilities', 'engine_live_prepare',+ 'engine_live_read', 'engine_live_part', 'engine_live_close', 'engine_live_forget']) {const answer = await c.request(type, { version: 1, session: { transcriptPath: '/etc/passwd' } })expect(answer.error).toBe('UNSUPPORTED')}@@ -117,13 +118,74 @@ describe('Claude Code and Codex readers in their own processes', () => {expect(d.coresStarted()).toBe(1)})- it('parks a crashing reader, returns an explicit failure, and leaves the agent working', async () => {- const { d, c } = await fresh({ HARNESSD_TEST_FAULTS: 'engine-codex.crash', HARNESSD_SERVICE_PARK_CRASHES: '3' })+ it.each(['claude', 'codex'] as const)('%s replays a shortened transcript as history and delivers the next turn live once', async engine => {+ const { d, c } = await fresh()+ const agent = await create(d, c, engine)+ await turn(c, agent, 'older turn')+ await turn(c, agent, 'retained history')+ const row = JSON.parse(readFileSync(join(d.dataDir, 'registry.json'), 'utf8')).find((row: any) => row.agentId === agent.id)+ const lines = readFileSync(row.transcriptPath, 'utf8').split('\n')+ let start = lines.findIndex(line => {+ try {+ const record = JSON.parse(line)+ return engine === 'claude'+ ? record.type === 'user' && record.message?.content === 'retained history'+ : record.type === 'event_msg' && record.payload?.type === 'item_completed'+ && record.payload?.item?.type === 'UserMessage'+ && record.payload.item.content.some((part: any) => part.text === 'retained history')+ } catch { return false }+ })+ expect(start).toBeGreaterThan(0)+ if (engine === 'codex') {+ // Keep the real turn's task marker and context before its completed UserMessage item.+ while (start > 0 && JSON.parse(lines[start]).payload?.type !== 'task_started') start--+ expect(start).toBeGreaterThan(0)+ }+ const first = c.frames.length+ const replayed = c.next(frame => frame.type === 'turn_ended' && frame.agentId === agent.id && frame.replay === true, 30_000, 'rewritten history')+ writeFileSync(row.transcriptPath, lines.slice(start).join('\n'))+ await replayed+ expect(c.frames.slice(first).filter(frame => frame.type === 'turn_ended' && frame.agentId === agent.id && !frame.replay)).toEqual([])+ await turn(c, agent, 'after rewrite')+ expect(c.frames.slice(first).filter(frame => frame.type === 'turn_ended' && frame.agentId === agent.id && !frame.replay)).toHaveLength(1)+ expect(d.coresStarted()).toBe(1)+ })++ it('parks a crashing engine worker while the CLI continues, then delivers its queued turn once after recovery', async () => {+ const { d, c } = await fresh({ HARNESSD_SERVICE_PARK_CRASHES: '3', HARNESSD_SERVICE_PARK_RETRY_MS: '30000' })const agent = await create(d, c, 'codex')- await history(c, agent)+ await turn(c, agent, 'before parking')+ const corePid = d.corePid()+ const registered = () => JSON.parse(readFileSync(join(d.dataDir, 'registry.json'), 'utf8')).find((row: any) => row.agentId === agent.id)+ const identity = registered().processIdentity+ for (let index = 0; index < 3; index++) {+ const before = readerPid(d, 'codex')!+ process.kill(before, 'SIGKILL')+ if (index < 2) await until('replacement worker', () => {+ const pid = readerPid(d, 'codex')+ return pid && pid !== before && alive(pid) ? pid : null+ }, 15_000, 100)+ }await until('reader parked', () => d.log().includes('service engine-codex ended 3 times') || null, 60_000, 200)expect(await history(c, agent)).toMatchObject({ error: 'ENGINE_UNAVAILABLE', retryable: true })- await turn(c, agent, 'reader-is-parked')+ const first = c.frames.length+ const ended = c.next(frame => frame.type === 'turn_ended' && frame.agentId === agent.id, 60_000, 'queued turn after worker recovery')+ c.send('message', { agentId: agent.id, content: 'worker-is-parked' })+ await until('the independent CLI to finish writing its turn', () => {+ const file = registered().transcriptPath+ return file && readFileSync(file, 'utf8').split('\n').some(line => {+ try { const row = JSON.parse(line); return row.payload?.type === 'task_complete' && row.payload?.last_agent_message?.includes('worker-is-parked') } catch { return false }+ }) || null+ }, 30_000, 100)+ const parked = readerPid(d, 'codex')+ expect(parked === undefined || !alive(parked)).toBe(true)+ expect(c.frames.slice(first).filter(frame => frame.type === 'turn_ended' && frame.agentId === agent.id)).toEqual([])+ await ended+ expect(c.frames.slice(first).filter(frame => frame.type === 'turn_started' && frame.agentId === agent.id)).toHaveLength(1)+ expect(c.frames.slice(first).filter(frame => frame.type === 'turn_ended' && frame.agentId === agent.id)).toHaveLength(1)+ expect(registered().processIdentity).toEqual(identity)+ expect(alive(identity.pid)).toBe(true)+ expect(d.corePid()).toBe(corePid)expect(d.coresStarted()).toBe(1)})})diff --git a/cli/src/core/AGENTS.md b/cli/src/core/AGENTS.mdindex c7c9dc044..8c43e5203 100644--- a/cli/src/core/AGENTS.md+++ b/cli/src/core/AGENTS.md@@ -26,14 +26,16 @@ questions. Everything else is a service ([../services/AGENTS.md](../services/AGE- `agents/`: create, fork, restart, retarget, stop, resume, close, discovery, adoption, binding, the list.- `transcripts/`: attach (bounded reads from the end), ingest, live tail, relaunch marks, normalizers.Claude Code/Codex live interpretation is injected through `Engine.live`; core keeps parser handles- and immutable turn snapshots, never edits their engine state. Live parsers are still in process- pending the transport in [the live design](../../../docs/design/2026-10-07-engine-live.md).+ and immutable turn snapshots, never edits their engine state. Supervised Claude Code/Codex parsers+ run in workers through [the live transport](../../../docs/design/2026-10-07-engine-streams.md).- `turns/`: working/idle, the event funnel, cancel, heartbeats, hooks, and `recaps.ts`, the core's wholeside of the recaps: the turn lifecycle it tells them, and what it reads back.- `terminals/`: who controls a pane (the control lease), opening a terminal with a literal argv (`open.ts`),and the requests about a terminal itself (`requests.ts`: `terminal_info`, `theme_set`).-- `engines/`: hook coordination and the reader port. Claude Code/Codex history and last-turn readers- run in supervised workers; core passes snapshots, validates replies and owns binding/turn state.+- `engines/`: hook coordination, reader ports and live worker coordination. Claude Code/Codex+ history, last-turn reads and live parsing run in supervised workers; core passes snapshots,+ validates bounded replies, retains acknowledged cursors and owns binding/turn state. The explicit+ inline/older-master compatibility path is the only live parser composition in core.- `input.ts`, `questions.ts`: messages into a pane; an agent's question and its answer.`deviceInput.ts`: the pane writer lock every write takes, and a device's queued turns behind it.`cardText.ts`: an engine's error, rewritten for a device's card.diff --git a/cli/src/core/engines/liveSessions.lifecycle.spec.ts b/cli/src/core/engines/liveSessions.lifecycle.spec.tsnew file mode 100644index 000000000..4faa31ae0--- /dev/null+++ b/cli/src/core/engines/liveSessions.lifecycle.spec.ts@@ -0,0 +1,226 @@+import { afterEach, describe, expect, it, vi } from 'vitest'+import type { RegisteredSession } from '../../lib/registry.js'+import type { LiveFrame, LivePage, LivePull } from '../../engines/worker/liveProtocol.js'+import { createLiveSessions, type LiveSessions } from './liveSessions.js'+import { EngineLiveError } from './liveTransport.js'++const running: LiveSessions[] = []+const deferred = <T>() => { let resolve!: (value: T) => void; let reject!: (error: unknown) => void+ const promise = new Promise<T>((r, j) => { resolve = r; reject = j }); return { promise, resolve, reject } }+const reply = (ask: LivePull, over: Partial<LivePage> = {}): LivePage => ({ frames: [], prepared: true,+ cursor: { serial: (ask.cursor?.serial ?? 0) + 1, origin: 10, offset: 10,+ turn: { identity: `${ask.token}:turn`, turnOpen: true, continued: false }, closed: false,+ prepareEnd: null, completeUntil: null, stamp: null },+ content: false, more: false, records: 0, turnFrom: 0, profileFrom: 0, end: 10, lastStarted: null, ...over })+const frame: LiveFrame = { raw: 'record', profile: true, observe: true, events: [], replay: false,+ turn: { identity: 'turn', turnOpen: true, continued: false } }++function setup(consoleLog = false) {+ const rows = new Map<string, RegisteredSession>()+ const session = (id = 's', path: string | null = '/private/transcript') => {+ const value = { agentId: id, sessionId: id, engine: 'claude', transcriptPath: path,+ boundAt: 1, cwd: '/private', model: null, cliVersion: null } as RegisteredSession+ rows.set(id, value); return value+ }+ const transport = { pull: vi.fn(async (ask: LivePull) => reply(ask)), forget: vi.fn(async () => {}),+ close: vi.fn(), connected: vi.fn(), disconnected: vi.fn() }+ const deps = { handles: (engine: string) => engine === 'claude', transport, bySession: (id: string) => rows.get(id),+ frame: vi.fn(), reattach: vi.fn(async () => {}), watch: vi.fn(), unwatch: vi.fn(),+ ...(consoleLog ? {} : { log: vi.fn() }) }+ const live = createLiveSessions(deps); running.push(live)+ const prepare = (s: RegisteredSession) => live.prepare(s, { live: false, end: 10 }, vi.fn())+ const attach = async (s: RegisteredSession) => { const p = await prepare(s); expect(live.install(p)).toBe(true); await live.follow(s); return p }+ return { live, deps, transport, rows, session, prepare, attach }+}++afterEach(async () => { await Promise.all(running.splice(0).map(live => live.stop())); vi.useRealTimers(); vi.restoreAllMocks() })++describe('live session lifecycle', () => {+ it('ignores absent tails and paths and shares a watched path until its final session leaves', async () => {+ vi.useFakeTimers()+ const p = setup(), a = p.session('a'), b = p.session('b')+ await p.live.follow(a)+ await p.live.pollSession('missing'); p.live.setTail('missing', 3)+ expect(await p.live.hold('missing', '/none')).toBeNull()+ await p.attach(p.session('empty', null))+ expect(p.deps.watch).not.toHaveBeenCalled()+ await p.attach(a); await p.attach(b); await p.attach(p.session('c'))+ p.live.changed('/unrelated')+ expect(await p.live.hold('a', '/wrong')).toBeNull()+ await p.live.follow(a)+ const hold = await p.live.hold('a', a.transcriptPath!)+ a.transcriptPath = '/private/new'+ const replacement = await p.prepare(a)+ expect(p.live.install(replacement)).toBe(true)+ await p.live.follow(a)+ hold!.release()+ expect(p.deps.unwatch).not.toHaveBeenCalled()+ await p.live.removeSession('b')+ expect(p.deps.unwatch).not.toHaveBeenCalled()+ await p.live.removeSession('c')+ expect(p.deps.unwatch).toHaveBeenCalledWith('/private/transcript')+ a.transcriptPath = '/private/third'+ const third = await p.prepare(a)+ expect(p.live.install(third)).toBe(true)+ await p.live.follow(a)+ expect(p.deps.unwatch).toHaveBeenCalledWith('/private/new')+ await p.live.removeSession('a'); await p.live.removeSession('a')+ expect(p.deps.unwatch).toHaveBeenCalledWith('/private/third')+ })++ it('hydrates multiple bounded pages and rejects a candidate after the binding changes', async () => {+ const p = setup(), s = p.session(), observe = vi.fn()+ p.transport.pull.mockImplementationOnce(async ask => reply(ask, { prepared: undefined, frames: [frame], content: true, records: 2 }))+ const prepared = await p.live.prepare(s, { live: false }, observe)+ expect(prepared.records).toBe(2); expect(prepared.content).toBe(true)+ expect(observe).toHaveBeenCalledExactlyOnceWith(frame)+ s.boundAt = (s.boundAt ?? 0) + 1+ expect(p.live.install(prepared)).toBe(false)+ p.live.discard(prepared)+ const pending = deferred<LivePage>()+ p.transport.pull.mockReturnValueOnce(pending.promise)+ const stale = p.prepare(s)+ await vi.waitFor(() => expect(p.transport.pull).toHaveBeenCalledTimes(3))+ s.boundAt = (s.boundAt ?? 0) + 1+ pending.resolve(reply(p.transport.pull.mock.calls.at(-1)![0]))+ await expect(stale).rejects.toMatchObject({ code: 'ENGINE_STALE_REPLY' })+ })++ it('reports repeated failures once, retries a failed poll and contains an observer failure per frame', async () => {+ vi.useFakeTimers()+ const p = setup(), s = p.session()+ await p.attach(s)+ p.transport.pull.mockRejectedValueOnce(new Error('offline')).mockRejectedValueOnce(new Error('offline'))+ await p.live.pollAll()+ await vi.advanceTimersByTimeAsync(1_000)+ expect(p.deps.log).toHaveBeenCalledTimes(1)+ p.transport.pull.mockImplementationOnce(async ask => reply(ask, { frames: [frame, frame] }))+ p.deps.frame.mockImplementationOnce(() => { throw new Error('observer') })+ await vi.advanceTimersByTimeAsync(1_000)+ expect(p.deps.frame).toHaveBeenCalledTimes(2)+ expect(p.deps.log).toHaveBeenCalledTimes(2)+ p.rows.delete(s.sessionId)+ const calls = p.transport.pull.mock.calls.length+ await p.live.pollAll()+ expect(p.transport.pull).toHaveBeenCalledTimes(calls)+ })++ it('reports non-Error failures through the default logger and retries failed activation', async () => {+ vi.useFakeTimers()+ const warn = vi.spyOn(console, 'warn').mockImplementation(() => {})+ const p = setup(true), s = p.session()+ const prepared = await p.prepare(s); p.live.install(prepared)+ p.transport.pull.mockRejectedValueOnce('offline')+ await p.live.follow(s)+ expect(warn).toHaveBeenCalledWith(expect.stringContaining('offline'))+ await vi.advanceTimersByTimeAsync(1_000)+ expect(p.transport.pull).toHaveBeenCalledTimes(3)+ })++ it('coalesces concurrent drains and remembers file changes arriving during a read', async () => {+ vi.useFakeTimers()+ const p = setup(), s = p.session(); await p.attach(s)+ const pending = deferred<LivePage>()+ p.transport.pull.mockReturnValueOnce(pending.promise)+ const first = p.live.pollSession(s.sessionId)+ await vi.advanceTimersByTimeAsync(0)+ const second = p.live.pollSession(s.sessionId)+ p.live.changed(s.transcriptPath!)+ pending.resolve(reply(p.transport.pull.mock.calls.at(-1)![0]))+ await Promise.all([first, second])+ const calls = p.transport.pull.mock.calls.length+ await vi.advanceTimersByTimeAsync(40)+ expect(p.transport.pull).toHaveBeenCalledTimes(calls + 1)+ })++ it('waits out a failed in-flight read before holding, expires holds and ignores late releases after setTail', async () => {+ vi.useFakeTimers()+ const p = setup(), s = p.session(); await p.attach(s)+ const pending = deferred<LivePage>()+ p.transport.pull.mockReturnValueOnce(pending.promise)+ const read = p.live.pollSession(s.sessionId)+ const failure = expect(read).rejects.toThrow('offline')+ await vi.advanceTimersByTimeAsync(0)+ const waiting = p.live.hold(s.sessionId, s.transcriptPath!, 50)+ pending.reject(new Error('offline')); await failure+ const first = (await waiting)!, second = (await p.live.hold(s.sessionId, s.transcriptPath!, 100))!+ expect(first.offset).toBe(10)+ const drain = p.live.pollSession(s.sessionId)+ await vi.advanceTimersByTimeAsync(50)+ expect(first.expired).toBe(true); expect(second.expired).toBe(false)+ p.live.setTail(s.sessionId, 4)+ const moved = (await p.live.hold(s.sessionId, s.transcriptPath!))!+ expect(moved.offset).toBe(4)+ first.release(10); second.release(10); moved.release(10); await drain+ const candidate = await p.prepare(s)+ p.live.setTail(s.sessionId, 5)+ expect(p.live.install(candidate)).toBe(false); p.live.discard(candidate)+ await vi.advanceTimersByTimeAsync(1_000)+ expect(p.deps.reattach).toHaveBeenCalledOnce()+ const accepted = await p.prepare(s)+ expect(p.live.install(accepted)).toBe(true)+ await p.live.pollSession(s.sessionId)+ })++ it('releases a blocked drain when detached and keeps no watch for a removed session', async () => {+ const p = setup(), s = p.session(); await p.attach(s)+ await p.live.hold(s.sessionId, s.transcriptPath!)+ const drain = p.live.pollSession(s.sessionId)+ await p.live.removeSession(s.sessionId)+ await drain+ expect(p.deps.unwatch).toHaveBeenCalledWith(s.transcriptPath)+ })++ it('finishes a hold awaiting a page when the session is removed during that page', async () => {+ const p = setup(), s = p.session(); await p.attach(s)+ const pending = deferred<LivePage>()+ p.transport.pull.mockReturnValueOnce(pending.promise)+ const read = p.live.pollSession(s.sessionId)+ await vi.waitFor(() => expect(p.transport.pull).toHaveBeenCalledTimes(3))+ const hold = p.live.hold(s.sessionId, s.transcriptPath!)+ await p.live.removeSession(s.sessionId)+ pending.resolve(reply(p.transport.pull.mock.calls.at(-1)![0]))+ await read+ expect((await hold)!.offset).toBe(0)+ })++ it('retries reattachment after failure, deduplicates timers and abandons retries for an obsolete binding', async () => {+ vi.useFakeTimers()+ const p = setup(), s = p.session()+ p.deps.reattach.mockRejectedValueOnce(new Error('offline'))+ p.live.retry(s); p.live.retry(s)+ await vi.advanceTimersByTimeAsync(1_000)+ expect(p.deps.reattach).toHaveBeenCalledTimes(1)+ await vi.advanceTimersByTimeAsync(1_000)+ expect(p.deps.reattach).toHaveBeenCalledTimes(2)+ p.live.retry(s); p.rows.delete(s.sessionId)+ await vi.advanceTimersByTimeAsync(1_000)+ expect(p.deps.reattach).toHaveBeenCalledTimes(2)+ await p.live.stop(); p.live.retry(s)+ })++ it('does not resurrect a removed binding when a pending read detects a rewrite', async () => {+ const p = setup(), s = p.session(); await p.attach(s)+ const pending = deferred<LivePage>()+ p.transport.pull.mockReturnValueOnce(pending.promise)+ const read = p.live.pollSession(s.sessionId)+ const failure = expect(read).rejects.toMatchObject({ code: 'ENGINE_TRANSCRIPT_CHANGED' })+ await vi.waitFor(() => expect(p.transport.pull).toHaveBeenCalledTimes(3))+ p.rows.delete(s.sessionId)+ pending.reject(new EngineLiveError('ENGINE_TRANSCRIPT_CHANGED')); await failure+ p.live.setTail(s.sessionId, 3)+ expect(p.deps.reattach).not.toHaveBeenCalled()+ })++ it('bounds its fair queue and rejects queued work on shutdown', async () => {+ const p = setup(), s = p.session(), pending = deferred<LivePage>()+ p.transport.pull.mockReturnValueOnce(pending.promise)+ const requests = Array.from({ length: 256 }, () => p.prepare(s))+ const outcomes = Promise.allSettled(requests)+ await expect(p.prepare(s)).rejects.toMatchObject({ code: 'ENGINE_BUSY' })+ await p.live.stop()+ pending.resolve(reply(p.transport.pull.mock.calls[0][0]))+ expect((await outcomes).every(result => result.status === 'rejected')).toBe(true)+ expect(p.transport.pull).toHaveBeenCalledOnce()+ })+})diff --git a/cli/src/core/engines/liveSessions.spec.ts b/cli/src/core/engines/liveSessions.spec.tsnew file mode 100644index 000000000..4c63bf23b--- /dev/null+++ b/cli/src/core/engines/liveSessions.spec.ts@@ -0,0 +1,208 @@+import { appendFile, mkdtemp, rm, writeFile } from 'node:fs/promises'+import { tmpdir } from 'node:os'+import { join } from 'node:path'+import { afterEach, describe, expect, it, vi } from 'vitest'+import type { RegisteredSession } from '../../lib/registry.js'+import type { LiveFrame } from '../../engines/worker/liveProtocol.js'+import { LIVE_PREPARE, LIVE_READ } from '../../engines/worker/liveProtocol.js'+import { engineLiveRequests } from '../../engines/worker/liveRequests.js'+import { createLiveTransport } from './liveTransport.js'+import { createLiveSessions, type LiveSessions } from './liveSessions.js'++const dirs: string[] = [], sessions: LiveSessions[] = []+const prompt = (message: string) => JSON.stringify({ type: 'user', message: { role: 'user', content: message } }) + '\n'+const done = JSON.stringify({ type: 'assistant', message: { role: 'assistant', content: [{ type: 'text', text: 'finished' }], stop_reason: 'end_turn' } }) + '\n'+const deferred = <T>() => { let resolve!: (value: T) => void; const promise = new Promise<T>(r => { resolve = r }); return { promise, resolve } }++async function setup(content = '') {+ const dir = await mkdtemp(join(tmpdir(), 'core-engine-stream-')); dirs.push(dir)+ const file = join(dir, 'transcript.jsonl'); await writeFile(file, content)+ const session = { agentId: 'agent', sessionId: 'session', engine: 'claude', transcriptPath: file,+ cwd: dir, model: null, cliVersion: null, boundAt: 1 } as RegisteredSession+ let bound: RegisteredSession | undefined = session+ let worker = engineLiveRequests('claude', { fields: () => [] })+ let connected = false+ let intercept: ((method: string, reply: Record<string, unknown>) => Promise<Record<string, unknown>>) | null = null+ const transport = createLiveTransport({ call: async (service, method, payload) => {+ if (!connected) { connected = true; transport.connected(service) }+ const reply = await worker[method](payload, { local: true, owner: true }) as Record<string, unknown>+ return intercept ? intercept(method, reply) : reply+ } })+ const frames: LiveFrame[] = []+ const reattach = vi.fn(async () => {})+ const watch = vi.fn(), unwatch = vi.fn(), log = vi.fn()+ const live = createLiveSessions({ handles: engine => engine === 'claude', transport, bySession: () => bound,+ frame: (_session, frame) => { frames.push(frame) }, reattach, watch, unwatch, log })+ sessions.push(live)+ const prepare = (first = false) => live.prepare(session, { live: first }, () => {})+ const attach = async (first = false) => {+ const prepared = await prepare(first)+ expect(live.install(prepared)).toBe(true)+ await live.follow(session)+ return prepared+ }+ return { file, session, frames, live, prepare, attach, reattach, watch, unwatch, log,+ events: () => frames.flatMap(frame => frame.events),+ unbind: () => { bound = undefined },+ intercept: (fn: typeof intercept) => { intercept = fn },+ restart: () => { transport.disconnected('engine-claude'); worker = engineLiveRequests('claude', { fields: () => [] }); connected = false },+ }+}++afterEach(async () => {+ await Promise.all(sessions.splice(0).map(live => live.stop()))+ vi.useRealTimers()+ await Promise.all(dirs.splice(0).map(dir => rm(dir, { recursive: true, force: true })))+})++describe('core engine stream authority', () => {+ it('remembers first-turn delivery when initial preparation fails and a normal retry follows', async () => {+ const t = await setup(prompt('born after the agent') + done)+ t.intercept(async (method, reply) => { if (method === LIVE_PREPARE) throw new Error('worker disconnected'); return reply })+ await expect(t.prepare(true)).rejects.toThrow('ENGINE_UNAVAILABLE')+ t.intercept(null)+ await t.attach()+ expect(t.events().filter(event => event.type === 'turn_started')).toHaveLength(1)+ expect(t.events().filter(event => event.type === 'turn_ended')).toHaveLength(1)+ })++ it('activates before emitting the first turn and streams it in bounded pages', async () => {+ const t = await setup(prompt('x'.repeat(700_000)) + done)+ const prepared = await t.prepare(true)+ expect(prepared.page.frames).toEqual([])+ expect(prepared.page.cursor.offset).toBe(0)+ expect(t.frames).toEqual([])+ expect(t.live.install(prepared)).toBe(true)+ await t.live.follow(t.session)+ expect(t.events().filter(event => event.type === 'turn_started')).toHaveLength(1)+ expect(t.events().filter(event => event.type === 'turn_ended')).toHaveLength(1)+ expect(prepared.state.handle.turnOpen).toBe(false)+ expect(t.live.current(t.session)).toBe(true)+ expect(t.watch).toHaveBeenCalledWith(t.file)+ await t.live.pollSession(t.session.sessionId)+ expect(t.events().filter(event => event.type === 'turn_started')).toHaveLength(1)+ })++ it('hydrates history silently and resumes from the accepted checkpoint after worker restart', async () => {+ const t = await setup(prompt('old') + done)+ const prepared = await t.attach()+ expect(t.events()).toEqual([])+ await appendFile(t.file, prompt('new'))+ await t.live.pollSession(t.session.sessionId)+ const turn = prepared.state.handle.snapshot()+ expect(turn.turnOpen).toBe(true)+ t.restart()+ await appendFile(t.file, done)+ await t.live.pollSession(t.session.sessionId)+ expect(t.events().map(event => event.type)).toEqual(['turn_started', 'text_delta', 'turn_ended'])+ expect(prepared.state.handle.snapshot()).toEqual({ ...turn, turnOpen: false })+ })++ it('discards a reply for a binding that changed while its worker was reading', async () => {+ const t = await setup()+ const prepared = await t.attach()+ const entered = deferred<void>(), release = deferred<void>()+ t.intercept(async (method, reply) => { if (method === LIVE_READ) { entered.resolve(); await release.promise }; return reply })+ await appendFile(t.file, prompt('late'))+ const read = t.live.pollSession(t.session.sessionId)+ await entered.promise+ t.unbind(); release.resolve(); await read+ expect(t.frames).toEqual([])+ expect(prepared.state.handle.turnOpen).toBe(false)+ expect(t.live.current({ ...t.session, boundAt: 2 })).toBe(false)+ })++ it('fences an in-flight page when cancel closes the observed turn, then accepts the newer turn', async () => {+ const t = await setup(prompt('one'))+ const prepared = await t.attach()+ const first = prepared.state.handle.snapshot()+ const entered = deferred<void>(), release = deferred<void>()+ t.intercept(async (method, reply) => { if (method === LIVE_READ) { entered.resolve(); await release.promise }; return reply })+ await appendFile(t.file, prompt('two'))+ const read = t.live.pollSession(t.session.sessionId)+ await entered.promise+ prepared.state.handle.closeTurn('cancel')+ expect(prepared.state.handle.turnOpen).toBe(false)+ release.resolve(); await read+ expect(t.frames).toEqual([])+ t.intercept(null)+ await t.live.pollSession(t.session.sessionId)+ expect(t.events().filter(event => event.type === 'turn_ended')).toHaveLength(0)+ expect(t.events().filter(event => event.type === 'turn_started')).toHaveLength(1)+ expect(prepared.state.handle.snapshot().identity).not.toBe(first.identity)+ expect(prepared.state.handle.turnOpen).toBe(true)+ })++ it('accepts a read across a Linux clock correction but rejects a changed process start tick', async () => {+ const t = await setup()+ t.session.processIdentity = { pid: 7, executable: 'claude', startMarker: 'one', startTicks: 42 }+ const prepared = await t.attach()+ let entered = deferred<void>(), release = deferred<void>()+ t.intercept(async (method, reply) => { if (method === LIVE_READ) { entered.resolve(); await release.promise }; return reply })+ await appendFile(t.file, prompt('current'))+ const read = t.live.pollSession(t.session.sessionId)+ await entered.promise+ t.session.processIdentity.startMarker = 'two'+ release.resolve(); await read+ expect(t.events().map(event => event.type)).toEqual(['turn_started'])+ expect(prepared.state.handle.turnOpen).toBe(true)+ entered = deferred<void>(); release = deferred<void>()+ await appendFile(t.file, done)+ const stale = t.live.pollSession(t.session.sessionId)+ await entered.promise+ t.session.processIdentity.startTicks = 43+ release.resolve(); await stale+ expect(t.events().map(event => event.type)).toEqual(['turn_started'])+ expect(prepared.state.handle.turnOpen).toBe(true)+ })++ it('holds delivery while a replacement parser hydrates, and cannot install a candidate after cancellation', async () => {+ const t = await setup(prompt('one'))+ const first = await t.attach()+ const hold = await t.live.hold(t.session.sessionId, t.file)+ expect(hold?.offset).toBe(Buffer.byteLength(prompt('one')))+ const candidate = await t.prepare()+ first.state.handle.closeTurn('cancel')+ expect(t.live.install(candidate)).toBe(false)+ t.live.discard(candidate)+ await appendFile(t.file, prompt('two'))+ let finished = false+ const drain = t.live.pollSession(t.session.sessionId).then(() => { finished = true })+ await Promise.resolve()+ expect(finished).toBe(false)+ hold!.release(); await drain+ expect(t.events().filter(event => event.type === 'turn_started')).toHaveLength(1)+ expect(first.state.handle.turnOpen).toBe(true)+ })++ it('preserves the checkpoint on rewrite, schedules recovery, and removes all watches when detached', async () => {+ const t = await setup(prompt('before'))+ const prepared = await t.attach()+ const before = prepared.state.handle.snapshot()+ await writeFile(t.file, prompt('after!'))+ await expect(t.live.pollSession(t.session.sessionId)).rejects.toThrow('ENGINE_TRANSCRIPT_CHANGED')+ expect(prepared.state.handle.snapshot()).toEqual(before)+ expect(t.frames).toEqual([])+ await t.live.removeSession(t.session.sessionId)+ expect(t.unwatch).toHaveBeenCalledWith(t.file)+ expect(t.live.tails(t.session.sessionId, t.file)).toBe(false)+ })++ it('activates a replacement under a hold and replays its history before delivering a fresh turn', async () => {+ const t = await setup(prompt('a much longer original turn'))+ await t.attach()+ await writeFile(t.file, prompt('history') + done)+ await expect(t.live.pollSession(t.session.sessionId)).rejects.toThrow('ENGINE_TRANSCRIPT_CHANGED')+ const hold = await t.live.hold(t.session.sessionId, t.file)+ const replacement = await t.prepare()+ expect(replacement.page.frames).toEqual([])+ expect(t.live.install(replacement)).toBe(true)+ hold!.release()+ await t.live.pollSession(t.session.sessionId)+ expect(t.events().map(event => event.type)).toEqual(['turn_started', 'text_delta', 'turn_ended'])+ expect(t.frames.every(frame => frame.replay)).toBe(true)+ await appendFile(t.file, prompt('fresh'))+ await t.live.pollSession(t.session.sessionId)+ expect(t.frames.at(-1)).toMatchObject({ replay: false, events: [{ type: 'turn_started', payload: { userMessage: 'fresh' } }] })+ })+})diff --git a/cli/src/core/engines/liveSessions.ts b/cli/src/core/engines/liveSessions.tsnew file mode 100644index 000000000..ac42eade6--- /dev/null+++ b/cli/src/core/engines/liveSessions.ts@@ -0,0 +1,283 @@+import { randomUUID } from 'node:crypto'+import type { LiveState } from '../../engines/facets/live.js'+import type { LiveFrame, LivePage, LivePull } from '../../engines/worker/liveProtocol.js'+import type { RegisteredSession } from '../../lib/registry.js'+import { HOLD_TIMEOUT_MS, type TailHold } from '../../watcher/watcher.js'+import { transcriptReadIdentity } from '../transcripts/readIdentity.js'+import { EngineLiveError, type LiveTransport } from './liveTransport.js'++interface SessionState {+ ask: LivePull+ identity: string+ epoch: number+ handle: LiveState+}+interface Tail {+ path: string+ dirty: boolean+ running: Promise<void> | null+ timer: ReturnType<typeof setTimeout> | null+ holds: Set<TailHold>+ unheld: Promise<void> | null+ release: (() => void) | null+ moves: number+ offset: number | null+ rewritten: boolean+}+export interface PreparedLive {+ state: SessionState+ parent: SessionState | undefined+ parentEpoch: number | undefined+ moves: number+ page: LivePage+ records: number+ content: boolean+}+export interface LiveSessionDeps {+ handles(engine: string): boolean+ transport: LiveTransport+ bySession(id: string): RegisteredSession | undefined+ frame(session: RegisteredSession, frame: LiveFrame): void+ reattach(session: RegisteredSession): Promise<unknown>+ watch(path: string): void+ unwatch(path: string): void+ log?(message: string): void+}++/** Core retains only binding facts and the last accepted checkpoint. Unread events stay on disk. */+export function createLiveSessions(deps: LiveSessionDeps) {+ const states = new Map<string, SessionState>()+ const firstLive = new Map<string, string>()+ const tails = new Map<string, Tail>()+ const retries = new Map<string, ReturnType<typeof setTimeout>>()+ const queues = new Map<string, Promise<void>>()+ const queued = new Map<string, number>()+ const errors = new Map<string, string>()+ let stopped = false+ const valid = (s: SessionState): boolean => !stopped && s.identity === transcriptReadIdentity(deps.bySession(s.ask.session.sessionId))+ const report = (id: string, error: unknown): void => {+ const code = error instanceof Error ? error.message : String(error)+ if (errors.get(id) === code) return+ errors.set(id, code)+ ;(deps.log ?? console.warn)(`[engine] ${id} transcript paused at its acknowledged position · ${code}`)+ }+ const slot = async <T>(engine: string, run: () => Promise<T>): Promise<T> => {+ if ((queued.get(engine) ?? 0) >= 256) throw new EngineLiveError('ENGINE_BUSY')+ queued.set(engine, (queued.get(engine) ?? 0) + 1)+ const before = queues.get(engine) ?? Promise.resolve()+ let release!: () => void+ const current = new Promise<void>(resolve => { release = resolve })+ queues.set(engine, current)+ await before+ try { return await run() } finally {+ queued.set(engine, queued.get(engine)! - 1); release()+ if (queues.get(engine) === current) queues.delete(engine)+ }+ }+ const pull = (ask: LivePull) => slot(ask.session.engine, () => {+ if (stopped) throw new EngineLiveError('ENGINE_UNAVAILABLE')+ return deps.transport.pull(ask)+ })+ // Forget shares the worker's one request slot with pulls. Otherwise activating a replacement can+ // make its first read fail ENGINE_BUSY while the old parser is still being released.+ const forget = (engine: string, token: string): void => {+ void slot(engine, () => deps.transport.forget(engine, token)).catch(() => {})+ }+ const makeState = (session: RegisteredSession, live: boolean, end?: number): SessionState => {+ const state: SessionState = { identity: transcriptReadIdentity(session), epoch: 0,+ ask: { token: randomUUID(), session: { agentId: session.agentId, sessionId: session.sessionId,+ engine: session.engine, transcriptPath: session.transcriptPath, cwd: session.cwd, model: session.model,+ cliVersion: session.cliVersion, codexHome: session.codexHome }, cursor: null, fromStart: live,+ replay: false, liveStart: live, ...(end === undefined ? {} : { end }),+ ...(tails.get(session.sessionId)?.rewritten ? { rewritten: true } : {}) },+ handle: {+ engine: session.engine,+ // Handles escape prepare only after the worker has supplied a validated cursor.+ get turnOpen() { return state.ask.cursor!.turn.turnOpen },+ snapshot: () => ({ ...state.ask.cursor!.turn }),+ closeTurn(reason) {+ const cursor = state.ask.cursor!+ // An in-flight reply predates this decision and cannot overwrite it. Reconstructing from this+ // checkpoint applies the engine's own close semantics before reading unseen records.+ state.epoch+++ state.ask.cursor = { ...cursor, closed: reason, turn: { ...cursor.turn, turnOpen: false } }+ schedule(session.sessionId)+ },+ },+ }+ return state+ }+ const retry = (session: RegisteredSession): void => {+ const id = session.sessionId, identity = transcriptReadIdentity(session)+ if (stopped || retries.has(id)) return+ retries.set(id, setTimeout(() => {+ retries.delete(id)+ if (identity !== transcriptReadIdentity(deps.bySession(id))) return+ void deps.reattach(session).catch(error => { report(id, error); retry(session) })+ }, 1_000))+ }+ function schedule(id: string, delay = 40): void {+ const tail = tails.get(id)+ if (!tail || stopped) return+ tail.dirty = true+ if (tail.timer || tail.running || tail.holds.size || tail.offset !== null) return+ tail.timer = setTimeout(() => {+ tail.timer = null+ void pollSession(id).catch(error => { report(id, error); schedule(id, 1_000) })+ }, delay)+ }+ async function pollSession(id: string): Promise<void> {+ const tail = tails.get(id)+ if (!tail || stopped) return+ if (tail.unheld) await tail.unheld+ if (tail.running) return tail.running+ if (tail.offset !== null || tails.get(id) !== tail) return+ tail.dirty = false+ tail.running = (async () => {+ let more = true+ while (more && !stopped && tails.get(id) === tail && !tail.holds.size && tail.offset === null) {+ const state = states.get(id)+ if (!state || !valid(state)) return+ const epoch = state.epoch+ let page: LivePage+ try { page = await pull({ ...state.ask }) }+ catch (error) {+ if (error instanceof EngineLiveError && error.code === 'ENGINE_TRANSCRIPT_CHANGED') {+ // No cursor from the changed file is accepted. Re-attach under a hold and a new identity.+ tail.rewritten = true+ const session = deps.bySession(id)+ if (session) retry(session)+ }+ throw error+ }+ if (!valid(state) || states.get(id) !== state || state.epoch !== epoch) return+ state.ask.cursor = page.cursor+ errors.delete(id)+ const session = deps.bySession(id)!+ for (const frame of page.frames) {+ try { deps.frame(session, frame) } catch (error) { report(id, error) }+ }+ more = page.more+ }+ })()+ try { await tail.running } finally {+ tail.running = null+ if (tail.dirty) schedule(id)+ }+ }+ const removeSession = async (id: string): Promise<void> => {+ firstLive.delete(id)+ const state = states.get(id); states.delete(id)+ if (state) { state.epoch++; forget(state.ask.session.engine, state.ask.token) }+ const tail = tails.get(id); tails.delete(id)+ if (tail) {+ if (tail.timer) clearTimeout(tail.timer)+ for (const hold of [...tail.holds]) hold.release()+ if (![...tails.values()].some(other => other.path === tail.path)) deps.unwatch(tail.path)+ }+ const timer = retries.get(id)+ if (timer) clearTimeout(timer)+ retries.delete(id); errors.delete(id)+ }+ return {+ handles: deps.handles,+ current(session: RegisteredSession): boolean { return states.get(session.sessionId)?.identity === transcriptReadIdentity(session) },+ async prepare(session: RegisteredSession, options: { live: boolean; end?: number }, observe: (frame: LiveFrame) => void): Promise<PreparedLive> {+ // A retry has no original attach flags. Keep first-turn delivery intent until activation;+ // otherwise a failed first prepare turns the user's completed first response into silent history.+ if (options.live) firstLive.set(session.sessionId, transcriptReadIdentity(session))+ const state = makeState(session, firstLive.get(session.sessionId) === transcriptReadIdentity(session), options.end)+ const parent = states.get(session.sessionId), parentEpoch = parent?.epoch+ const moves = tails.get(session.sessionId)?.moves ?? 0+ let records = 0, content = false+ try {+ for (;;) {+ const page = await pull(state.ask)+ if (!valid(state)) throw new EngineLiveError('ENGINE_STALE_REPLY')+ state.ask.cursor = page.cursor; records += page.records; content ||= page.content+ for (const frame of page.frames) observe(frame)+ if (page.prepared) return { state, parent, parentEpoch, moves, page, records, content }+ }+ } catch (error) {+ forget(session.engine, state.ask.token)+ report(session.sessionId, error)+ throw error+ }+ },+ install(prepared: PreparedLive): boolean {+ const { state, parent, parentEpoch, moves } = prepared, id = state.ask.session.sessionId+ if (!valid(state) || states.get(id) !== parent || parent?.epoch !== parentEpoch || (tails.get(id)?.moves ?? 0) !== moves) return false+ states.set(id, state)+ firstLive.delete(id)+ const tail = tails.get(id)+ if (tail) { tail.offset = null; tail.rewritten = false }+ errors.delete(id)+ const timer = retries.get(id)+ if (timer) clearTimeout(timer)+ retries.delete(id)+ if (parent) forget(parent.ask.session.engine, parent.ask.token)+ return true+ },+ discard(prepared: PreparedLive): void {+ forget(prepared.state.ask.session.engine, prepared.state.ask.token)+ },+ retry,+ async follow(session: RegisteredSession): Promise<void> {+ const id = session.sessionId, path = session.transcriptPath+ if (!path || !states.has(id)) return+ const previous = tails.get(id)+ if (previous?.path === path) { schedule(id); return }+ if (previous) {+ tails.delete(id)+ if (previous.timer) clearTimeout(previous.timer)+ for (const hold of [...previous.holds]) hold.release()+ if (![...tails.values()].some(other => other.path === previous.path)) deps.unwatch(previous.path)+ }+ tails.set(id, { path, dirty: false, running: null, timer: null, holds: new Set(), unheld: null,+ release: null, moves: 0, offset: null, rewritten: false })+ deps.watch(path)+ // First-turn delivery follows activation. It never builds one unbounded initialEvents array.+ await pollSession(id).catch(error => { report(id, error); schedule(id, 1_000) })+ },+ changed(path: string): void { for (const [id, tail] of tails) if (tail.path === path) schedule(id) },+ tails(id: string, path: string): boolean { return tails.get(id)?.path === path },+ async hold(id: string, path: string, timeoutMs = HOLD_TIMEOUT_MS): Promise<TailHold | null> {+ const tail = tails.get(id)+ if (!tail || tail.path !== path) return null+ let expired = false, released = false+ let offset = 0+ const hold: TailHold = {+ get offset() { return offset }, get expired() { return expired },+ // Installation already puts the precise new cursor in the session. A released hold cannot+ // move it back after a reset or an explicit setTail changed the transcript underneath it.+ release() {+ if (released) return+ released = true; clearTimeout(timer); tail.holds.delete(hold)+ if (!tail.holds.size) { tail.release?.(); tail.release = null; tail.unheld = null; schedule(id) }+ },+ }+ const timer = setTimeout(() => { expired = true; hold.release() }, timeoutMs)+ if (!tail.holds.size) tail.unheld = new Promise(resolve => { tail.release = resolve })+ tail.holds.add(hold)+ await tail.running?.catch(() => {})+ offset = tail.offset ?? states.get(id)?.ask.cursor?.offset ?? 0+ return hold+ },+ setTail(id: string, offset: number): void {+ const tail = tails.get(id), state = states.get(id)+ if (!tail || !state) return+ state.epoch++; tail.moves++; tail.offset = offset; tail.rewritten = false+ const session = deps.bySession(id)+ if (session) retry(session)+ },+ pollSession,+ async pollAll(): Promise<void> { await Promise.all([...tails.keys()].map(id => pollSession(id).catch(error => { report(id, error); schedule(id, 1_000) }))) },+ removeSession,+ async stop(): Promise<void> {+ stopped = true+ await Promise.all([...new Set([...states.keys(), ...tails.keys(), ...retries.keys(), ...firstLive.keys()])].map(removeSession))+ },+ }+}++export type LiveSessions = ReturnType<typeof createLiveSessions>diff --git a/cli/src/core/engines/liveTransport.spec.ts b/cli/src/core/engines/liveTransport.spec.tsnew file mode 100644index 000000000..222dc7f2f--- /dev/null+++ b/cli/src/core/engines/liveTransport.spec.ts@@ -0,0 +1,184 @@+import { createHash } from 'node:crypto'+import { afterEach, describe, expect, it, vi } from 'vitest'+import { createLiveTransport } from './liveTransport.js'+import {+ LIVE_CAPABILITIES, LIVE_CLOSE, LIVE_FORGET, LIVE_PART, LIVE_PART_BYTES, LIVE_PREPARE, LIVE_READ,+ LIVE_RESULT_BYTES, LIVE_WAIT_MS, type LiveCursor, type LivePage, type LivePull,+} from '../../engines/worker/liveProtocol.js'++const cursor = (changes: Partial<LiveCursor> = {}): LiveCursor => ({ serial: 1, origin: 0, offset: 0,+ turn: { identity: 'binding:empty', turnOpen: false, continued: false }, closed: false,+ prepareEnd: null, completeUntil: null, stamp: null, ...changes })+const page = (changes: Partial<LivePage> = {}): LivePage => ({ frames: [], cursor: cursor(), prepared: true,+ content: false, more: false, records: 0, turnFrom: 0, profileFrom: 0, end: 0, lastStarted: null, ...changes })+const ask = (checkpoint: LiveCursor | null = null, engine = 'claude'): LivePull => ({ token: 'binding', cursor: checkpoint,+ fromStart: false, replay: false, session: { agentId: 'agent', sessionId: 'session', engine,+ transcriptPath: '/private/transcript.jsonl', cwd: '/private', model: null, cliVersion: null } } as LivePull)+const deferred = <T>() => { let resolve!: (value: T) => void; const promise = new Promise<T>(r => { resolve = r }); return { promise, resolve } }+type Reply = Record<string, unknown>++function setup(reply: (type: string, payload: Reply) => Promise<Reply> | Reply = () => ({ version: 1, answer: page() })) {+ const call = vi.fn(async (service: string, type: string, payload: Reply, _wait: number): Promise<Reply> =>+ type === LIVE_CAPABILITIES ? { version: 1, live: 1, engine: service.slice('engine-'.length) } : reply(type, payload))+ const port = createLiveTransport({ call })+ port.connected('engine-claude'); port.connected('engine-codex')+ return { port, call }+}++function fragmented(text: string, size = 29) {+ const bytes = Buffer.from(text), hash = createHash('sha256').update(bytes).digest('hex')+ return (offset = 0): Reply => ({ version: 1, part: 'fragment', offset, bytes: bytes.length, hash,+ data: bytes.subarray(offset, offset + size).toString('base64') })+}++afterEach(() => { vi.useRealTimers(); vi.restoreAllMocks() })++describe('live worker transport', () => {+ it('negotiates a cold connection once, then uses the accepted cursor on the same connection', async () => {+ const p = setup((_method, payload) => ({ version: 1, answer: page({ cursor: cursor({ serial: payload.cursor ? 2 : 1 }) }) }))+ p.port.disconnected('engine-claude')+ const call = p.call.getMockImplementation()!+ p.call.mockImplementationOnce(async (...args) => { p.port.connected('engine-claude'); return call(...args) })+ const first = await p.port.pull(ask())+ expect(p.call.mock.calls.map(call => call[1])).toEqual([LIVE_CAPABILITIES, LIVE_PREPARE])+ expect(p.call.mock.calls[1].slice(0, 3)).toEqual(['engine-claude', LIVE_PREPARE, { ...ask(), version: 1 }])+ expect(p.call.mock.calls[1][3]).toBeGreaterThan(0)+ expect(p.call.mock.calls[1][3]).toBeLessThanOrEqual(LIVE_WAIT_MS)+ await p.port.pull(ask(first.cursor))+ expect(p.call.mock.calls.map(call => call[1])).toEqual([LIVE_CAPABILITIES, LIVE_PREPARE, LIVE_READ])+ p.port.connected('search'); p.port.disconnected('search')+ })++ it('bounds work independently for each engine and frees the slot after a failed read', async () => {+ const waiting = deferred<Reply>()+ const p = setup(() => waiting.promise)+ const first = p.port.pull(ask())+ const settled = expect(first).rejects.toMatchObject({ code: 'ENGINE_UNAVAILABLE' })+ await expect(p.port.pull(ask())).rejects.toMatchObject({ code: 'ENGINE_BUSY' })+ const second = p.port.pull(ask(null, 'codex'))+ const other = expect(second).rejects.toMatchObject({ code: 'ENGINE_UNAVAILABLE' })+ waiting.resolve({ error: 'SERVICE_UNAVAILABLE' })+ await Promise.all([settled, other])+ p.call.mockResolvedValue({ version: 1, answer: page() })+ expect(await p.port.pull(ask())).toEqual(page())+ })++ it.each(['disconnect', 'replacement', 'reconnect'] as const)('rejects a late reply after %s', async kind => {+ const waiting = deferred<Reply>(), p = setup()+ p.call.mockReturnValueOnce(waiting.promise)+ const request = p.port.pull(ask())+ if (kind !== 'replacement') p.port.disconnected('engine-claude')+ if (kind !== 'disconnect') p.port.connected('engine-claude')+ waiting.resolve({ version: 1, live: 1, engine: 'claude' })+ await expect(request).rejects.toMatchObject({ code: 'ENGINE_STALE_REPLY' })+ expect(p.call).toHaveBeenCalledOnce()+ })++ it.each([+ [{ error: 'SERVICE_FAILED' }, 'ENGINE_UNAVAILABLE'],+ [{ error: { message: 'oops' } }, 'ENGINE_UNAVAILABLE'],+ [{ error: 'ENGINE_TRANSCRIPT_CHANGED' }, 'ENGINE_TRANSCRIPT_CHANGED'],+ [{ version: 2 }, 'ENGINE_INVALID_REPLY'],+ ])('contains a failed worker reply %j', async (reply, code) => {+ const p = setup(() => reply)+ await expect(p.port.pull(ask())).rejects.toMatchObject({ code })+ })++ it.each([{ version: 1, live: 2, engine: 'claude' }, { version: 1, live: 1, engine: 'codex' }])('rejects incompatible capabilities %j', async reply => {+ const p = setup(); p.call.mockResolvedValue(reply)+ await expect(p.port.pull(ask())).rejects.toMatchObject({ code: 'ENGINE_INVALID_REPLY' })+ expect(p.call).toHaveBeenCalledOnce()+ })++ it('contains transport exceptions without loading an inline parser', async () => {+ const p = setup(); p.call.mockRejectedValue(new Error('socket closed'))+ await expect(p.port.pull(ask())).rejects.toMatchObject({ code: 'ENGINE_UNAVAILABLE' })+ })++ it.each([+ {}, page({ cursor: cursor({ serial: 3 }) }),+ page({ cursor: cursor({ turn: { identity: 'another:empty', turnOpen: false, continued: false } }) }),+ page({ frames: [{ raw: '', profile: true, observe: true, events: [], replay: false,+ turn: { identity: 'another:empty', turnOpen: false, continued: false } }] }),+ ])('refuses a malformed page or one from another binding', async answer => {+ const p = setup(() => ({ version: 1, answer }))+ await expect(p.port.pull(ask())).rejects.toMatchObject({ code: 'ENGINE_INVALID_REPLY' })+ })++ it.each([+ cursor({ serial: 2, offset: 4, origin: 5 }),+ cursor({ serial: 2, offset: 5, origin: 4 }),+ cursor({ serial: 1, offset: 5, origin: 5 }),+ ])('never moves an accepted cursor backward or changes its recovery window', async checkpoint => {+ const p = setup(() => ({ version: 1, answer: page({ cursor: checkpoint, end: 5 }) }))+ await expect(p.port.pull(ask(cursor({ offset: 5, origin: 5 })))).rejects.toMatchObject({ code: 'ENGINE_INVALID_REPLY' })+ })++ it('reassembles bounded fragments with an exact hash before accepting the page', async () => {+ const fragment = fragmented(JSON.stringify({ version: 1, answer: page() }))+ const p = setup((_method, payload) => fragment(payload.offset as number | undefined))+ expect(await p.port.pull(ask())).toEqual(page())+ expect(p.call.mock.calls.filter(call => call[1] === LIVE_PART).length).toBeGreaterThan(1)+ expect(p.call.mock.calls.slice(2).every(call => call[2].part === 'fragment')).toBe(true)+ })++ it.each([+ { part: 4 }, { part: '' }, { part: 'x'.repeat(201) }, { bytes: '1' }, { bytes: 0 },+ { bytes: 0.5 }, { bytes: LIVE_RESULT_BYTES + 1 }, { hash: 4 }, { hash: 'bad' },+ { offset: 1 }, { data: 1 }, { data: '' }, { data: '?A==' },+ { data: 'a'.repeat(Math.ceil(LIVE_PART_BYTES / 3) * 4 + 1) },+ ])('bounds and validates fragment metadata before accepting it', async changes => {+ const fragment = fragmented(JSON.stringify({ version: 1, answer: page() }))+ const p = setup(() => ({ ...fragment(), ...changes }))+ await expect(p.port.pull(ask())).rejects.toMatchObject({ code: 'ENGINE_INVALID_REPLY' })+ })++ it.each([{ part: 'replacement' }, { bytes: 1 }, { hash: '0'.repeat(64) }, { offset: 0 }])('rejects a changed fragment stream', async changes => {+ const fragment = fragmented(JSON.stringify({ version: 1, answer: page() }))+ const p = setup((method, payload) => ({ ...fragment(payload.offset as number | undefined), ...(method === LIVE_PART ? changes : {}) }))+ await expect(p.port.pull(ask())).rejects.toMatchObject({ code: 'ENGINE_INVALID_REPLY' })+ })++ it.each(['invalid json', 'null', '[]', '{"version":2}'])('rejects invalid assembled content %s', async content => {+ const fragment = fragmented(content)+ const p = setup((_method, payload) => fragment(payload.offset as number | undefined))+ await expect(p.port.pull(ask())).rejects.toMatchObject({ code: 'ENGINE_INVALID_REPLY' })+ })++ it('rejects a corrupt hash, oversized decoded chunk and noncanonical base64', async () => {+ for (const changes of [{ hash: '0'.repeat(64) }, { bytes: 1 }, { bytes: 1, data: 'Zh==' }]) {+ const fragment = fragmented('{}')+ const p = setup(() => ({ ...fragment(), ...changes }))+ await expect(p.port.pull(ask())).rejects.toMatchObject({ code: 'ENGINE_INVALID_REPLY' })+ }+ })++ it('shares one deadline across capability negotiation and every fragment', async () => {+ vi.useFakeTimers({ toFake: ['performance'] })+ const fragment = fragmented(JSON.stringify({ version: 1, answer: page() }))+ const p = setup((_method, payload) => { vi.advanceTimersByTime(6_000); return fragment(payload.offset as number | undefined) })+ await expect(p.port.pull(ask())).rejects.toMatchObject({ code: 'ENGINE_UNAVAILABLE' })+ expect(p.call.mock.calls.map(call => call[3])).toEqual([15_000, 15_000, 9_000, 3_000])+ })++ it('closes only a validated cursor, forgets supported engines, and ignores unrelated services', async () => {+ const p = setup((method) => method === LIVE_CLOSE+ ? { version: 1, answer: { closed: true, cursor: cursor({ closed: 'cancel', serial: 2 }) } }+ : { version: 1, forgotten: true })+ expect(await p.port.close(ask(cursor()), 'binding:empty', 'cancel')).toMatchObject({ closed: true, cursor: { serial: 2 } })+ expect(p.call.mock.calls.at(-1)?.slice(0, 3)).toEqual(['engine-claude', LIVE_CLOSE,+ { ...ask(cursor()), identity: 'binding:empty', reason: 'cancel', version: 1 }])+ await p.port.forget('claude', 'binding')+ expect(p.call.mock.calls.at(-1)?.[1]).toBe(LIVE_FORGET)+ const count = p.call.mock.calls.length+ await p.port.forget('unknown', 'binding')+ await expect(p.port.pull(ask(null, 'unknown'))).rejects.toMatchObject({ code: 'ENGINE_INVALID_REQUEST' })+ await expect(p.port.close(ask(null, 'unknown'), '', 'hook')).rejects.toMatchObject({ code: 'ENGINE_INVALID_REQUEST' })+ expect(p.call).toHaveBeenCalledTimes(count)+ })++ it.each([null, { closed: 'true', cursor: cursor() }, { closed: true, cursor: null }])('rejects invalid close replies %j', async answer => {+ const p = setup(() => ({ version: 1, answer }))+ await expect(p.port.close(ask(cursor()), 'binding:empty', 'hook')).rejects.toMatchObject({ code: 'ENGINE_INVALID_REPLY' })+ })+})diff --git a/cli/src/core/engines/liveTransport.ts b/cli/src/core/engines/liveTransport.tsnew file mode 100644index 000000000..d115abdd2--- /dev/null+++ b/cli/src/core/engines/liveTransport.ts@@ -0,0 +1,98 @@+/** Only protocol code belongs here. Parsers, transcript walks and their recovery execute in workers. */+import { createHash } from 'node:crypto'+import {+ liveCursor, livePage, LIVE_CAPABILITIES, LIVE_CLOSE, LIVE_FORGET, LIVE_PART, LIVE_PART_BYTES,+ LIVE_PREPARE, LIVE_READ, LIVE_RESULT_BYTES, LIVE_VERSION, LIVE_WAIT_MS, LIVE_IN_FLIGHT, type LiveCursor, type LivePage, type LivePull,+} from '../../engines/worker/liveProtocol.js'+import { readerEngine, record, READER_SERVICES, type ReaderEngine } from '../../engines/worker/protocol.js'++export class EngineLiveError extends Error {+ constructor(readonly code: string) { super(code) }+}+export interface LiveTransportDeps {+ call(service: string, type: string, payload: Record<string, unknown>, waitMs: number): Promise<Record<string, unknown>>+}++export function createLiveTransport(deps: LiveTransportDeps) {+ const states = new Map(Object.values(READER_SERVICES).map(service => [service as string, { generation: 0, connected: false, capable: false, pending: 0 }]))+ async function perform(engine: ReaderEngine, type: string, payload: Record<string, unknown>): Promise<Record<string, unknown>> {+ const service = READER_SERVICES[engine]+ const state = states.get(service)!+ const generation = state.generation + (state.connected ? 0 : 1)+ const deadline = performance.now() + LIVE_WAIT_MS+ const one = async (method: string, data: Record<string, unknown>): Promise<Record<string, unknown>> => {+ const remaining = Math.ceil(deadline - performance.now())+ if (remaining <= 0) throw new EngineLiveError('ENGINE_UNAVAILABLE')+ let reply: Record<string, unknown>+ try { reply = await deps.call(service, method, { ...data, version: LIVE_VERSION }, remaining) }+ catch { throw new EngineLiveError('ENGINE_UNAVAILABLE') }+ if (state.generation !== generation) throw new EngineLiveError('ENGINE_STALE_REPLY')+ if (reply.error !== undefined) throw new EngineLiveError(typeof reply.error === 'string' && reply.error.startsWith('ENGINE_') ? reply.error : 'ENGINE_UNAVAILABLE')+ if (reply.version !== LIVE_VERSION) throw new EngineLiveError('ENGINE_INVALID_REPLY')+ return reply+ }+ if (!state.capable) {+ const capabilities = await one(LIVE_CAPABILITIES, {})+ if (capabilities.live !== LIVE_VERSION || capabilities.engine !== engine) throw new EngineLiveError('ENGINE_INVALID_REPLY')+ state.capable = true+ }+ let reply = await one(type, payload)+ if (reply.part === undefined) return reply+ const id = reply.part, size = reply.bytes, hash = reply.hash+ if (typeof id !== 'string' || !id || id.length > 200 || typeof size !== 'number' || !Number.isSafeInteger(size)+ || size < 1 || size > LIVE_RESULT_BYTES || typeof hash !== 'string' || !/^[a-f0-9]{64}$/.test(hash)) throw new EngineLiveError('ENGINE_INVALID_REPLY')+ // Allocate once, after the bound is checked. One session awaits one page, so unread backlog stays+ // on disk rather than queuing raw transcript records in core's heap.+ const body = Buffer.allocUnsafe(size)+ let offset = 0+ while (offset < size) {+ if (reply.part !== id || reply.bytes !== size || reply.hash !== hash || reply.offset !== offset || typeof reply.data !== 'string'+ || reply.data.length > Math.ceil(LIVE_PART_BYTES / 3) * 4) throw new EngineLiveError('ENGINE_INVALID_REPLY')+ const part = Buffer.from(reply.data, 'base64')+ if (!part.length || part.length > size - offset || part.toString('base64') !== reply.data) throw new EngineLiveError('ENGINE_INVALID_REPLY')+ part.copy(body, offset); offset += part.length+ if (offset < size) reply = await one(LIVE_PART, { part: id, offset })+ }+ if (createHash('sha256').update(body).digest('hex') !== hash) throw new EngineLiveError('ENGINE_INVALID_REPLY')+ let result: unknown+ try { result = JSON.parse(body.toString('utf8')) } catch { throw new EngineLiveError('ENGINE_INVALID_REPLY') }+ if (!record(result) || result.version !== LIVE_VERSION) throw new EngineLiveError('ENGINE_INVALID_REPLY')+ return result+ }+ async function request(engine: ReaderEngine, type: string, payload: Record<string, unknown>): Promise<Record<string, unknown>> {+ const state = states.get(READER_SERVICES[engine])!+ if (state.pending >= LIVE_IN_FLIGHT) throw new EngineLiveError('ENGINE_BUSY')+ state.pending+++ try { return await perform(engine, type, payload) } finally { state.pending-- }+ }+ return {+ async pull(ask: LivePull): Promise<LivePage> {+ if (!readerEngine(ask.session.engine)) throw new EngineLiveError('ENGINE_INVALID_REQUEST')+ const reply = await request(ask.session.engine, ask.cursor ? LIVE_READ : LIVE_PREPARE, { ...ask })+ if (!livePage(reply.answer) || reply.answer.cursor.serial !== (ask.cursor?.serial ?? 0) + 1+ || (ask.cursor && (reply.answer.cursor.offset < ask.cursor.offset || reply.answer.cursor.origin !== ask.cursor.origin))+ || !reply.answer.cursor.turn.identity.startsWith(`${ask.token}:`)+ || reply.answer.frames.some(frame => !frame.turn.identity.startsWith(`${ask.token}:`))) throw new EngineLiveError('ENGINE_INVALID_REPLY')+ return reply.answer+ },+ async close(ask: LivePull, identity: string, reason: Exclude<LiveCursor['closed'], false>): Promise<{ closed: boolean; cursor: LiveCursor }> {+ if (!readerEngine(ask.session.engine)) throw new EngineLiveError('ENGINE_INVALID_REQUEST')+ const reply = await request(ask.session.engine, LIVE_CLOSE, { ...ask, identity, reason })+ if (!record(reply.answer) || typeof reply.answer.closed !== 'boolean' || !liveCursor(reply.answer.cursor)) throw new EngineLiveError('ENGINE_INVALID_REPLY')+ return { closed: reply.answer.closed, cursor: reply.answer.cursor }+ },+ async forget(engine: string, token: string): Promise<void> {+ if (readerEngine(engine)) await request(engine, LIVE_FORGET, { token })+ },+ connected(service: string): void {+ const state = states.get(service)+ if (state) { state.connected = true; state.generation++; state.capable = false }+ },+ disconnected(service: string): void {+ const state = states.get(service)+ if (state) { state.connected = false; state.generation++; state.capable = false }+ },+ }+}++export type LiveTransport = ReturnType<typeof createLiveTransport>diff --git a/cli/src/core/engines/liveWatcher.spec.ts b/cli/src/core/engines/liveWatcher.spec.tsnew file mode 100644index 000000000..675336ba2--- /dev/null+++ b/cli/src/core/engines/liveWatcher.spec.ts@@ -0,0 +1,93 @@+import { afterEach, describe, expect, it, vi } from 'vitest'+import type { RegisteredSession } from '../../lib/registry.js'+import type { Watcher } from '../../watcher/watcher.js'+import type { LiveSessionDeps } from './liveSessions.js'+import { createLiveWatcher } from './liveWatcher.js'++const mocks = vi.hoisted(() => ({ factory: vi.fn(), watch: vi.fn() }))+vi.mock('./liveSessions.js', () => ({ createLiveSessions: mocks.factory }))+vi.mock('chokidar', () => ({ default: { watch: mocks.watch } }))++function setup() {+ const local = { on: vi.fn(), start: vi.fn(), stop: vi.fn(async () => {}), addSession: vi.fn(async () => {}),+ removeSession: vi.fn(async () => {}), hold: vi.fn(async () => null), tails: vi.fn(() => false),+ setTail: vi.fn(), pollSession: vi.fn(async () => {}), pollAll: vi.fn(async () => {}) }+ const live = { stop: vi.fn(async () => {}), follow: vi.fn(async () => {}), changed: vi.fn(),+ removeSession: vi.fn(async () => {}), hold: vi.fn(async () => null), tails: vi.fn(() => false),+ setTail: vi.fn(), pollSession: vi.fn(async () => {}), pollAll: vi.fn(async () => {}) }+ const events = new Map<string, (path: string) => void>()+ const signals = { add: vi.fn(), unwatch: vi.fn(async () => {}), close: vi.fn(async () => {}),+ on: vi.fn((name, handler) => { events.set(name, handler); return signals }) }+ const session = { sessionId: 's', engine: 'claude', transcriptPath: '/private/transcript' } as RegisteredSession+ const bySession = vi.fn((): RegisteredSession | undefined => session)+ let deps!: LiveSessionDeps+ mocks.factory.mockImplementation((value: LiveSessionDeps) => { deps = value; return live })+ mocks.watch.mockReturnValue(signals)+ const wrapper = createLiveWatcher(local as unknown as Watcher, {+ handles: engine => engine === 'claude', bySession, transport: {} as LiveSessionDeps['transport'],+ frame: vi.fn(), reattach: vi.fn(),+ })+ return { ...wrapper, local, remote: live, signals, events, session, bySession, deps }+}++afterEach(() => vi.clearAllMocks())++describe('live file signals', () => {+ it('tracks paths before and after starting, forwards file signals and closes both watchers', async () => {+ const p = setup()+ await p.watcher.stop()+ p.deps.watch('/before'); p.deps.watch('/removed'); p.deps.unwatch('/removed')+ p.watcher.start(); p.watcher.start()+ expect(mocks.watch).toHaveBeenCalledExactlyOnceWith(['/before'], { ignoreInitial: true })+ p.deps.watch('/after'); p.deps.unwatch('/before')+ expect(p.signals.add).toHaveBeenCalledWith('/after')+ expect(p.signals.unwatch).toHaveBeenCalledWith('/before')+ for (const name of ['add', 'change', 'unlink']) p.events.get(name)!('/after')+ expect(p.remote.changed.mock.calls).toEqual([['/after'], ['/after'], ['/after']])+ await p.watcher.stop()+ expect(p.signals.close).toHaveBeenCalledOnce()+ expect(p.remote.stop).toHaveBeenCalledTimes(2)+ p.watcher.start()+ expect(mocks.watch).toHaveBeenLastCalledWith(['/after'], { ignoreInitial: true })+ await p.watcher.stop()+ })++ it('routes only the current isolated binding to the worker, preserving legacy tails and event listeners', async () => {+ const p = setup(), options = { fromStart: true }+ const observed = vi.fn()+ p.watcher.on('line', observed)+ expect(p.local.on).toHaveBeenCalledWith('line', observed)+ const legacy = { ...p.session, engine: 'pi' as const, transcriptPath: '/private/legacy' }+ await p.watcher.addSession(legacy, options)+ expect(p.local.addSession).toHaveBeenCalledWith(legacy, options)+ await p.watcher.addSession({ ...p.session, transcriptPath: '/private/transcript' }, options)+ expect(p.local.removeSession).toHaveBeenCalledWith('s')+ expect(p.remote.follow).toHaveBeenCalledExactlyOnceWith(p.session)+ p.bySession.mockReturnValue({ ...p.session, transcriptPath: '/another' })+ await p.watcher.addSession({ ...p.session, transcriptPath: '/private/transcript' })+ p.bySession.mockReturnValue(undefined)+ await p.watcher.addSession({ ...p.session, transcriptPath: '/private/transcript' })+ expect(p.remote.follow).toHaveBeenCalledOnce()+ await p.watcher.removeSession('s')+ expect(p.remote.removeSession).toHaveBeenCalledWith('s')+ })++ it('routes holds by ownership and drains, resets and polls both mechanisms', async () => {+ const p = setup()+ expect(p.watcher.tails('s', '/path')).toBe(false)+ await p.watcher.hold('s', '/path', 100)+ expect(p.local.hold).toHaveBeenCalledWith('s', '/path', 100)+ p.remote.tails.mockReturnValue(true)+ expect(p.watcher.tails('s', '/path')).toBe(true)+ await p.watcher.hold('s', '/path', 200)+ expect(p.remote.hold).toHaveBeenCalledWith('s', '/path', 200)+ p.watcher.setTail('s', 10)+ expect(p.local.setTail).toHaveBeenCalledWith('s', 10)+ expect(p.remote.setTail).toHaveBeenCalledWith('s', 10)+ await p.watcher.pollSession('s'); await p.watcher.pollAll()+ expect(p.local.pollSession).toHaveBeenCalledWith('s')+ expect(p.remote.pollSession).toHaveBeenCalledWith('s')+ expect(p.local.pollAll).toHaveBeenCalledOnce()+ expect(p.remote.pollAll).toHaveBeenCalledOnce()+ })+})diff --git a/cli/src/core/engines/liveWatcher.ts b/cli/src/core/engines/liveWatcher.tsnew file mode 100644index 000000000..22e4083c6--- /dev/null+++ b/cli/src/core/engines/liveWatcher.ts@@ -0,0 +1,35 @@+import chokidar, { type FSWatcher } from 'chokidar'+import type { Watcher } from '../../watcher/watcher.js'+import { createLiveSessions, type LiveSessionDeps } from './liveSessions.js'++/** The core watches file names and asks workers to read. Only legacy engines use the local tailer. */+export function createLiveWatcher(local: Watcher, deps: Omit<LiveSessionDeps, 'watch' | 'unwatch'>) {+ const paths = new Set<string>()+ let signals: FSWatcher | null = null+ const live = createLiveSessions({ ...deps,+ watch(path) { paths.add(path); signals?.add(path) },+ unwatch(path) { paths.delete(path); void signals?.unwatch(path) },+ })+ const watcher: Pick<Watcher, 'on' | 'start' | 'stop' | 'addSession' | 'removeSession' | 'hold' | 'tails' | 'setTail' | 'pollSession' | 'pollAll'> = {+ on: local.on.bind(local),+ start() {+ local.start()+ signals ??= chokidar.watch([...paths], { ignoreInitial: true })+ .on('add', path => live.changed(path)).on('change', path => live.changed(path)).on('unlink', path => live.changed(path))+ },+ async stop() { await Promise.all([local.stop(), live.stop(), signals?.close()]); signals = null },+ async addSession(session, options) {+ if (!deps.handles(session.engine)) return local.addSession(session, options)+ await local.removeSession(session.sessionId)+ const bound = deps.bySession(session.sessionId)+ if (bound?.transcriptPath === session.transcriptPath) await live.follow(bound)+ },+ async removeSession(id) { await Promise.all([local.removeSession(id), live.removeSession(id)]) },+ hold: (id, path, timeout) => live.tails(id, path) ? live.hold(id, path, timeout) : local.hold(id, path, timeout),+ tails: (id, path) => live.tails(id, path) || local.tails(id, path),+ setTail(id, offset) { local.setTail(id, offset); live.setTail(id, offset) },+ async pollSession(id) { await Promise.all([local.pollSession(id), live.pollSession(id)]) },+ async pollAll() { await Promise.all([local.pollAll(), live.pollAll()]) },+ }+ return { live, watcher }+}diff --git a/cli/src/core/main.ts b/cli/src/core/main.tsindex dc655bf29..9882a2b43 100644--- a/cli/src/core/main.ts+++ b/cli/src/core/main.ts@@ -25,7 +25,7 @@ import { registry, projectDisplayName, validTranscriptPath, type RegisteredSessiimport { engineSessionTitle } from '../lib/sessionTitle.js'import { machineNames } from '../lib/machineNames.js'import { engineHooks as engineHookFacets } from '../engines/hooks.js'-import { liveFor } from '../engines/live.js'+import type { LiveFor } from '../engines/facets/live.js'import { DAEMON_LOG_FILE, PID_FILE, daemonPort, isAlive, readPid, LEGACY_LOG_FILE, MACHINE_NAME_FILE, tildify, computerId, thisDeviceLabel } from '../lib/daemonState.js'import { clearSafeModeMarker, safeModeDisposition, safeModeStatusBody, SafeModeRequest, writeSafeModeMarker } from '../lib/daemonSafeMode.js'import { awakeTimeout } from '../lib/sleepAware.js'@@ -89,7 +89,9 @@ import { createQuestions } from './questions.js'import { createTurnActivity } from './turns/activity.js'import { createLastTurnReader } from './transcripts/lastTurn.js'import { createEngineReaders } from './engines/readers.js'-import { READER_SERVICES } from '../engines/worker/protocol.js'+import { createLiveTransport } from './engines/liveTransport.js'+import { createLiveWatcher } from './engines/liveWatcher.js'+import { readerEngine, READER_SERVICES } from '../engines/worker/protocol.js'import { createRecaps } from './turns/recaps.js'import { createUpdateHandoff, handOverOnceReleased, probeStagedMaster, type TeardownStep } from './updateHandoff.js'import { needsUpdaterBeside, startUpdaterBeside } from './updaterBeside.js'@@ -142,7 +144,7 @@ import { createExperimentHooks, wakeExperiments } from './experiments.js'import { answerExperimentQuery, createForExperiment } from './experimentQueries.js'import { createOrchestratorLink } from './orchestratorLink.js'import { daemonCommand } from '../lib/daemonCommand.js'-import { KNOWN_SERVICES, servicesTheMasterRuns } from '../harnessd/services.js'+import { KNOWN_SERVICES, masterRunsLiveEngines, servicesTheMasterRuns } from '../harnessd/services.js'import { createTeamsLink, teamsOutOfProcess } from './teamsLink.js'import { createDevicesLink } from './devicesLink.js'import { CORE_EXIT_STOP, CORE_EXIT_UPDATE, PROBE_COMMAND } from '../harnessd/protocol.js'@@ -779,7 +781,14 @@ async function runForeground(session: AuthSession | null): Promise<void> {const outOfProcess = servicesTheMasterRuns(process.env, KNOWN_SERVICES)// The services' own code, for those that run in this process (services/inline.ts): loaded only then, so// one in its own process, as each is by default, is never loaded here.- const inline = KNOWN_SERVICES.some((name) => !outOfProcess.has(name)) ? await import('../services/inline.js') : null+ // A reader-only older master knows these names but cannot host live parsers. Its explicit version+ // report selects compatibility; a worker timeout or crash never activates an inline fallback.+ const liveHosted = masterRunsLiveEngines(process.env, process.ppid)+ const isolatedLive = (engine: string): boolean => liveHosted+ && readerEngine(engine) && outOfProcess.has(READER_SERVICES[engine])+ const inline = KNOWN_SERVICES.some((name) => !outOfProcess.has(name)) || !liveHosted+ ? await import('../services/inline.js') : null+ const liveFor: LiveFor = (engine) => isolatedLive(engine) ? undefined : inline?.liveFor(engine)// The relay and its E2EE (gateway/): the backend link, the sessions and the keys, which every remote// client's frames go through, held at the same gate. In its own process by default (core/gatewayLink.ts),// or here (gateway/start.ts); the socket hears it through `fromGateway` and speaks to it in the clear.@@ -788,6 +797,9 @@ async function runForeground(session: AuthSession | null): Promise<void> {let serviceLinksRef: ServiceLinks | null = nullconst engineReaders = createEngineReaders({ isolated: outOfProcess, inline: inline?.engineTranscriptFor,call: (service, type, payload, waitMs) => serviceLinksRef?.call(service, type, payload, waitMs) ?? Promise.resolve({ error: 'SERVICE_UNAVAILABLE' }) })+ const liveTransport = createLiveTransport({+ call: (service, type, payload, waitMs) => serviceLinksRef?.call(service, type, payload, waitMs) ?? Promise.resolve({ error: 'SERVICE_UNAVAILABLE' }),+ })const gatewayLink = outOfProcess.has('gateway') ? createGatewayLink({events: backend.fromGateway,notify: (frame) => serviceLinksRef?.notify('gateway', frame) ?? false,@@ -900,7 +912,13 @@ async function runForeground(session: AuthSession | null): Promise<void> {const commandcodeNormalizers = normalizers.commandcodeNormalizersconst sessionTurnState = normalizers.sessionTurnStateconst sessionTurnOpen = normalizers.sessionTurnOpen- const watcher = new Watcher()+ const { watcher, live: engineLive } = createLiveWatcher(new Watcher(), {+ handles: isolatedLive,+ transport: liveTransport,+ bySession: (id) => registry.bySession(id),+ frame: (session, frame) => ingest.acceptFrame(session.sessionId, session.engine, frame),+ reattach: (session) => attachSession(session, true),+ })// Whether a turn is really working, beyond its transcript (core/turns/activity.ts).const activity = createTurnActivity({terminals,@@ -957,6 +975,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {// (core/transcripts/attach.ts).const attach = createAttach({liveFor,+ remoteLive: engineLive,terminalGone: terminalControl.terminalGone,normalizers,watcher,@@ -1170,6 +1189,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {binary: (service, bytes) => { if (service === 'gateway') gatewayLink?.binary(bytes) },connected: (service) => {engineReaders.connected(service)+ liveTransport.connected(service)if (service === 'gateway') gatewayLink?.connected()if (service === 'teams') teamsLink.on()if (service === 'devices') devicesLink.connected()@@ -1178,6 +1198,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {},disconnected: (service) => {engineReaders.disconnected(service)+ liveTransport.disconnected(service)if (service === 'gateway') gatewayLink?.disconnected()if (service === 'devices') devicesLink.disconnected()if (service === 'wifi') wifiCore.stopped()diff --git a/cli/src/core/transcripts/attach.spec.ts b/cli/src/core/transcripts/attach.spec.tsindex 6ae57f522..4e3bdd0ad 100644--- a/cli/src/core/transcripts/attach.spec.ts+++ b/cli/src/core/transcripts/attach.spec.ts@@ -8,6 +8,8 @@ import type { TailHold } from '../../watcher/watcher.js'import { createAttach, type AttachDeps } from './attach.js'import { createSessionNormalizers } from './normalizers.js'import type { RelaunchMark } from './relaunch.js'+import type { PreparedLive } from '../engines/liveSessions.js'+import type { LiveFrame } from '../../engines/worker/liveProtocol.js'/*** The engines' own normalizers and readers are tested with each engine. Here each is a fake that@@ -104,6 +106,24 @@ function setup(over: Partial<AttachDeps> = {}) {const made = (kind: string) => fakes.made.filter((entry) => entry.kind === kind).map((entry) => entry.instance)+function remoteSetup(open = false) {+ const turn = { identity: 'worker:turn', turnOpen: open, continued: false }+ const handle = { engine: 'claude', turnOpen: open, snapshot: () => ({ ...turn }), closeTurn: vi.fn() }+ const candidate = { state: { handle }, page: { cursor: { offset: 20 }, lastStarted: open ? started() : null },+ records: 1, content: true } as unknown as PreparedLive+ const frames: LiveFrame[] = [+ { raw: 'metadata', profile: true, observe: false, events: [], replay: false, turn },+ { raw: 'record', profile: false, observe: true, events: [], replay: false, turn },+ ]+ const remote: NonNullable<AttachDeps['remoteLive']> = {+ handles: engine => engine === 'claude', current: vi.fn(() => true),+ prepare: vi.fn(async (_session, _options, observe) => { frames.forEach(observe); return candidate }),+ install: vi.fn(() => true), discard: vi.fn(), retry: vi.fn(),+ }+ const p = setup({ remoteLive: remote, liveFor: vi.fn(() => { throw new Error('isolated parser must stay in its worker') }) })+ return { ...p, remote, candidate, handle }+}+describe('attaching a session', () => {afterEach(() => {vi.restoreAllMocks()@@ -113,6 +133,71 @@ describe('attaching a session', () => {for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true })})+ it('installs worker state after hydration, keeps device/profile observations and reuses only a current binding', async () => {+ const p = remoteSetup(true), s = session('claude', '/private/transcript')+ p.service.needsTranscript.mockReturnValue(true)+ expect(await p.attach.attachSession(s)).toBe(true)+ expect(p.deps.liveFor).not.toHaveBeenCalled()+ expect(p.remote.prepare).toHaveBeenCalledWith(s, { live: false, end: undefined }, expect.any(Function))+ expect(p.profile.ingest).toHaveBeenCalledExactlyOnceWith('metadata')+ expect(p.service.observeTranscript).toHaveBeenCalledExactlyOnceWith(s.agentId, s.sessionId, s.engine, 'record')+ expect(p.profile.commit).toHaveBeenCalledOnce()+ expect(p.normalizers.liveParsers.get(s.sessionId)).toBe(p.handle)+ expect(p.remote.install).toHaveBeenCalledWith(p.candidate)+ await p.attach.attachSession(s)+ expect(p.remote.prepare).toHaveBeenCalledOnce()+ vi.mocked(p.remote.current).mockReturnValue(false)+ await p.attach.attachSession(s)+ expect(p.remote.prepare).toHaveBeenCalledTimes(2)+ })++ it('activates a first live worker stream without reading locally, including a session whose path is not announced yet', async () => {+ const p = remoteSetup(), s = session('claude')+ expect(await p.attach.attachSession(s, false, false, true)).toBe(true)+ expect(p.remote.prepare).toHaveBeenCalledWith(s, { live: true, end: undefined }, expect.any(Function))+ expect(p.deps.watcher.hold).not.toHaveBeenCalled()+ expect(p.normalizers.liveParsers.get(s.sessionId)).toBe(p.handle)+ })++ it.each([false, true])('retains a binding and retries a failed worker prepare, with an existing tail: %s', async held => {+ const p = remoteSetup(), s = session('claude', '/private/transcript')+ const old = { ...p.handle }+ p.normalizers.liveParsers.set(s.sessionId, old)+ const hold: TailHold = { offset: 12, expired: false, release: vi.fn() }+ if (held) vi.mocked(p.deps.watcher.hold).mockResolvedValue(hold)+ vi.mocked(p.remote.prepare).mockRejectedValueOnce(new Error('ENGINE_UNAVAILABLE'))+ expect(await p.attach.attachSession(s, true)).toBe(true)+ expect(p.remote.retry).toHaveBeenCalledWith(s)+ expect(p.normalizers.liveParsers.get(s.sessionId)).toBe(old)+ expect(p.profile.commit).not.toHaveBeenCalled()+ expect(p.remote.install).not.toHaveBeenCalled()+ if (held) expect(hold.release).toHaveBeenCalled()+ expect(await p.attach.attachSession(s, true)).toBe(true)+ expect(p.normalizers.liveParsers.get(s.sessionId)).toBe(p.handle)+ })++ it.each(['prepare', 'config', 'install'])('discards a superseded candidate during %s and keeps the live state', async stage => {+ const p = remoteSetup(), s = session('claude', '/private/transcript')+ const old = { ...p.handle }+ p.normalizers.liveParsers.set(s.sessionId, old)+ let expired = stage === 'prepare'+ const hold: TailHold = { offset: 12, get expired() { return expired }, release: vi.fn() }+ vi.mocked(p.deps.watcher.hold).mockResolvedValue(hold)+ if (stage === 'config') vi.mocked(p.deps.runtimeProfiles.ingestConfig).mockImplementation(async () => { expired = true; return false })+ if (stage === 'install') vi.mocked(p.remote.install).mockReturnValue(false)+ expect(await p.attach.attachSession(s, true)).toBe(true)+ expect(p.remote.discard).toHaveBeenCalledWith(p.candidate)+ expect(p.normalizers.liveParsers.get(s.sessionId)).toBe(old)+ expect(hold.release).toHaveBeenCalled()+ })++ it('uses the worker only for engines it handles', async () => {+ const p = remoteSetup()+ const local = setup({ remoteLive: p.remote })+ expect(await local.attach.attachSession(session('pi', transcript([])))).toBe(true)+ expect(p.remote.prepare).not.toHaveBeenCalled()+ })+it('refuses a session whose pane is gone', async () => {const { attach, deps } = setup({ terminalGone: vi.fn(async () => true) })expect(await attach.attachSession(session('pi', transcript()))).toBe(false)diff --git a/cli/src/core/transcripts/attach.ts b/cli/src/core/transcripts/attach.tsindex 0fe0e7c50..2bf2a2fc0 100644--- a/cli/src/core/transcripts/attach.ts+++ b/cli/src/core/transcripts/attach.ts@@ -37,9 +37,11 @@ import type { TailHold, Watcher } from '../../watcher/watcher.js'import type { SessionNormalizers } from './normalizers.js'import type { RelaunchMarks } from './relaunch.js'import { createSideReads } from './sideReads.js'+import type { LiveSessions, PreparedLive } from '../engines/liveSessions.js'export interface AttachDeps {liveFor: LiveFor+ remoteLive?: Pick<LiveSessions, 'handles' | 'current' | 'prepare' | 'install' | 'discard' | 'retry'>/** Whether the session's terminal is known to be gone (core/terminals/control.ts `terminalGone`). */terminalGone: (session: RegisteredSession) => Promise<boolean>normalizers: SessionNormalizers@@ -70,7 +72,7 @@ export interface AttachDeps {}export function createAttach({- liveFor, terminalGone, normalizers, watcher, cursorDiscovery, device, runtimeProfiles, captureTerminal, emit,+ liveFor, remoteLive, terminalGone, normalizers, watcher, cursorDiscovery, device, runtimeProfiles, captureTerminal, emit,announceTurnAborted, questionWatcher, terminalLabel, dbs, devinHome, hermesDb, concurrency, relaunchMarks,wholeReadCapBytes = WHOLE_READ_CAP_BYTES, settled,}: AttachDeps) {@@ -131,7 +133,8 @@ export function createAttach({// Taken whichever way this attach goes: a mark is for the next attach of the conversation only.const relaunch = relaunchMarks?.take(session.sessionId)const relaunchedAt = relaunch?.offset- if (!reset && normalizers.hasState(session.sessionId)) {+ const remote = remoteLive?.handles(session.engine) ? remoteLive : undefined+ if (!reset && normalizers.hasState(session.sessionId) && (!remote || remote.current(session))) {if (session.transcriptPath) {const unseen = neverFoldedHistory.delete(session.sessionId)await watcher.addSession(@@ -184,7 +187,7 @@ export function createAttach({// the chips and a continuing /goal still need — never the whole conversation, which on a long session// cost the daemon more memory than it has (lib/attachTranscript.ts). Their normalizers exist before// the read because they fold as the records stream in. The other engines still fold everything.- const adapter = liveFor(session.engine)+ const adapter = remote ? undefined : liveFor(session.engine)const parser = adapter?.create(session)const fields = (line: string): readonly RuntimeField[] => runtimeProfiles.transcriptFields(session, line)const rules = adapter?.attachRules?.(fields)@@ -192,6 +195,7 @@ export function createAttach({? { rules, ingest: (line: string) => parser.ingest(line).events, turnOpen: () => parser.turnOpen }: nulllet fromEnd: AttachRead | null = null+ let prepared: PreparedLive | null = null// A reset keeps the normalizer it meant to replace when its read failed, or outlasted the hold on the// tail — which then let go, and delivery went back to that normalizer: it has seen every record since,// this one has not.@@ -201,7 +205,26 @@ export function createAttach({handover.next = nullreturn true}- if (session.transcriptPath && fromEndFold) {+ if (remote) {+ if (session.transcriptPath) handover.hold = await watcher.hold(session.sessionId, session.transcriptPath)+ const live = replayLive && handover.hold === null+ const profile = runtimeProfiles.beginHydrate(session)+ try {+ prepared = await remote.prepare(session, { live, end: handover.hold?.offset ?? relaunchedAt }, frame => {+ if (frame.profile) sideRead('runtime profile', session.sessionId, () => profile.ingest(frame.raw))+ if (frame.observe && observe) observe(frame.raw)+ })+ } catch {+ remote.retry(session)+ return handover.hold ? keepLiveNormalizer('could not reach its engine worker') : true+ }+ if (handover.hold?.expired) { remote.discard(prepared); return keepLiveNormalizer('outlasted its hold on the tail') }+ profile.commit()+ fromEnd = { next: prepared.page.cursor.offset, records: prepared.records, content: prepared.content }+ handover.next = fromEnd.next+ historyTurnOpen = !live && prepared.state.handle.turnOpen+ if (prepared.page.lastStarted) historyEvents.push(prepared.page.lastStarted)+ } else if (session.transcriptPath && fromEndFold) {// A session already being tailed — a reset — is re-read while its old normalizer is still the one// being fed: hold its tail, so the read stops exactly where delivery stopped and delivery resumes,// into the new normalizer, from where the read stopped (released by `attachSession`).@@ -242,10 +265,16 @@ export function createAttach({await runtimeProfiles.ingestConfig(session, true)// From here to the release in `attachSession` nothing is awaited for a held tail, so the hold cannot// expire between installing the new normalizer and handing it the tail.- if (handover.hold?.expired) return keepLiveNormalizer('outlasted its hold on the tail')+ if (handover.hold?.expired) {+ if (prepared) remote!.discard(prepared)+ return keepLiveNormalizer('outlasted its hold on the tail')+ }const fold = (ingest: (line: string) => LiveEvent[], turnOpenAfter: () => boolean): boolean =>take(foldTranscript(ingest, lines, turnOpenAfter, { live: replayLive }))- if (parser) {+ if (prepared) {+ if (!remote!.install(prepared)) { remote!.discard(prepared); return keepLiveNormalizer('was superseded while reading') }+ liveParsers.set(session.sessionId, prepared.state.handle)+ } else if (parser) {if (!fromEnd) historyTurnOpen = fold((line) => parser.ingest(line).events, () => parser.turnOpen)liveParsers.set(session.sessionId, parser)} else if (session.engine === 'cursor') {diff --git a/cli/src/core/transcripts/ingest.spec.ts b/cli/src/core/transcripts/ingest.spec.tsindex dd875e98a..bfd8f3249 100644--- a/cli/src/core/transcripts/ingest.spec.ts+++ b/cli/src/core/transcripts/ingest.spec.ts@@ -13,6 +13,7 @@ import type { RegisteredSession } from '../../lib/registry.js'import type { HistoryEvent, LineEvent, RewrittenEvent, Watcher } from '../../watcher/watcher.js'import { createIngest, type IngestDeps } from './ingest.js'import { createSessionNormalizers } from './normalizers.js'+import type { LiveFrame } from '../../engines/worker/liveProtocol.js'const CODEX_FAILED = JSON.stringify({ type: 'event_msg', payload: { type: 'task_complete', error: { message: 'rate limited' } } })const COMMANDCODE_FAILED = JSON.stringify({ type: 'message', message: { role: 'user', content: [{ type: 'text', text: 'Error: 500\nTrace ID: 932a' }] } })@@ -44,6 +45,24 @@ const line = (sessionId: string, engine: string, text = '{}'): LineEvent => ({ sdescribe('ingesting a transcript line', () => {afterEach(() => vi.restoreAllMocks())+ it('accepts worker events only for the bound engine, preserving replay and side reads without constructing a parser', () => {+ const p = setup({ s1: 'claude' }, { liveFor: () => { throw new Error('worker events are already normalized') } })+ const frame: LiveFrame = { raw: CLAUDE_PROMPT, events: [{ type: 'turn_started', payload: { userMessage: 'hello' } }],+ profile: true, observe: true, replay: true, turn: { identity: 'worker:turn', turnOpen: true, continued: false } }+ p.ingest.acceptFrame('unknown', 'claude', frame)+ p.ingest.acceptFrame('s1', 'codex', frame)+ expect(p.deps.emit).not.toHaveBeenCalled()+ p.ingest.acceptFrame('s1', 'claude', frame)+ expect(p.deps.emit).toHaveBeenCalledWith('s1', frame.events, { replay: true })+ expect(p.deps.runtimeProfiles.ingest).toHaveBeenCalledWith(p.sessions.get('s1'), frame.raw)+ p.ingest.acceptFrame('s1', 'claude', { ...frame, replay: false, failure: 'rate limited' })+ expect(p.deps.announceTurnAborted).toHaveBeenCalledWith('s1', 'claude', 'rate limited')+ expect(vi.mocked(p.deps.announceTurnAborted).mock.invocationCallOrder[0])+ .toBeLessThan(vi.mocked(p.deps.emit).mock.invocationCallOrder[1])+ expect(p.deps.emit).toHaveBeenLastCalledWith('s1', frame.events, { replay: false })+ expect(p.normalizers.hasState('s1')).toBe(false)+ })+it('takes lines only for a registered session of the engine that wrote them', () => {const { ingest, deps } = setup({ s1: 'claude', s2: 'codex' })expect(ingest.ingestLine(line('nobody', 'claude'))).toBeNull()diff --git a/cli/src/core/transcripts/ingest.ts b/cli/src/core/transcripts/ingest.tsindex 258c29ecc..6fdef44f4 100644--- a/cli/src/core/transcripts/ingest.ts+++ b/cli/src/core/transcripts/ingest.ts@@ -10,7 +10,7 @@*/import { AgyNormalizer } from '../../engines/agy/normalizer.js'import { AmpNormalizer } from '../../engines/amp/normalizer.js'-import type { LiveFor } from '../../engines/facets/live.js'+import type { LiveFor, LiveParser } from '../../engines/facets/live.js'import { CommandCodeNormalizer, commandCodeRunError, commandCodeRunErrorSummary } from '../../engines/commandcode/normalizer.js'import { CopilotNormalizer } from '../../engines/copilot/normalizer.js'import { CursorNormalizer } from '../../engines/cursor/normalizer.js'@@ -25,6 +25,7 @@ import type { RuntimeProfileManager } from '../../lib/runtimeProfile.js'import type { HistoryEvent, LineEvent, RewrittenEvent, Watcher } from '../../watcher/watcher.js'import type { SessionNormalizers } from './normalizers.js'import { createSideReads } from './sideReads.js'+import type { LiveFrame } from '../../engines/worker/liveProtocol.js'export interface IngestDeps {liveFor: LiveFor@@ -52,7 +53,7 @@ export function createIngest({// audiences: web (send, ServerEvents) and device (mirror.ingest → curated commander_event cards)./** One transcript line through its engine's normalizer. The events, not yet emitted — the two* callers below differ only in what they know about the line's age. */- const ingestLine = (evt: LineEvent): ReturnType<CursorNormalizer['ingest']> | null => {+ const observeLine = (evt: Pick<LineEvent, 'sessionId' | 'engine' | 'text'>): RegisteredSession | null => {if (!has(evt.sessionId)) return null // scope to terminal-registered sessionsconst session = bySession(evt.sessionId)if (!session || session.engine !== evt.engine) return null@@ -64,12 +65,17 @@ export function createIngest({}})sideRead('runtime profile', evt.sessionId, () => runtimeProfiles.ingest(session, evt.text))+ return session+ }+ const ingestLine = (evt: LineEvent): ReturnType<CursorNormalizer['ingest']> | null => {+ const session = observeLine(evt)+ if (!session) return nulllet events: LiveEvent[]const adapter = liveFor(session.engine)if (adapter) {let parser = liveParsers.get(evt.sessionId)- if (!parser || parser.engine !== session.engine) { parser = adapter.create(session); liveParsers.set(evt.sessionId, parser) }- const read = parser.ingest(evt.text)+ if (!parser || parser.engine !== session.engine || !('ingest' in parser)) { parser = adapter.create(session); liveParsers.set(evt.sessionId, parser) }+ const read = (parser as LiveParser).ingest(evt.text)events = read.eventsif (read.failure !== undefined) announceTurnAborted(evt.sessionId, session.engine, read.failure)} else if (session.engine === 'cursor') {@@ -159,5 +165,10 @@ export function createIngest({}})}- return { ingestLine, wireWatcher }+ const acceptFrame = (sessionId: string, engine: RegisteredSession['engine'], frame: LiveFrame): void => {+ if (!observeLine({ sessionId, engine, text: frame.raw })) return+ if (frame.failure !== undefined) announceTurnAborted(sessionId, engine, frame.failure)+ emit(sessionId, frame.events, { replay: frame.replay })+ }+ return { ingestLine, wireWatcher, acceptFrame }}diff --git a/cli/src/core/transcripts/normalizers.ts b/cli/src/core/transcripts/normalizers.tsindex ba2cd3887..e48cc2f72 100644--- a/cli/src/core/transcripts/normalizers.ts+++ b/cli/src/core/transcripts/normalizers.ts@@ -11,7 +11,7 @@*/import type { AgyNormalizer } from '../../engines/agy/normalizer.js'import type { AmpNormalizer } from '../../engines/amp/normalizer.js'-import type { LiveParser } from '../../engines/facets/live.js'+import type { LiveState } from '../../engines/facets/live.js'import type { CommandCodeNormalizer } from '../../engines/commandcode/normalizer.js'import type { CopilotNormalizer } from '../../engines/copilot/normalizer.js'import type { CursorNormalizer } from '../../engines/cursor/normalizer.js'@@ -25,7 +25,7 @@ import type { PiNormalizer } from '../../engines/pi/normalizer.js'export function createSessionNormalizers() {// Per-session web turn-lifecycle state; the device mirror keeps its own state + recap.- const liveParsers = new Map<string, LiveParser>()+ const liveParsers = new Map<string, LiveState>()const cursorNormalizers = new Map<string, CursorNormalizer>()const opencodeReaders = new Map<string, OpencodeReader>()const kiloReaders = new Map<string, KiloReader>()diff --git a/cli/src/core/transcripts/readIdentity.spec.ts b/cli/src/core/transcripts/readIdentity.spec.tsindex 270bdfb18..783cd0cd9 100644--- a/cli/src/core/transcripts/readIdentity.spec.ts+++ b/cli/src/core/transcripts/readIdentity.spec.ts@@ -4,12 +4,25 @@ import { transcriptReadIdentity } from './readIdentity.js'it('copies binding identity before yielding, ignoring ordinary activity updates', () => {const session = { agentId: 'a', sessionId: 's', engine: 'claude', transcriptPath: '/t', codexHome: '/home', boundAt: 1,- processIdentity: { pid: 7, startTime: 'one' }, touchedAt: 1 } as unknown as RegisteredSession+ processIdentity: { pid: 7, executable: 'claude', startMarker: 'one' }, touchedAt: 1 } as unknown as RegisteredSessionconst before = transcriptReadIdentity(session)expect(transcriptReadIdentity(undefined)).toBe('')session.touchedAt++expect(transcriptReadIdentity(session)).toBe(before)- for (const field of ['agentId', 'sessionId', 'engine', 'transcriptPath', 'codexHome', 'boundAt', 'processIdentity']) {+ for (const field of ['agentId', 'sessionId', 'engine', 'transcriptPath', 'codexHome', 'boundAt']) {expect(transcriptReadIdentity({ ...session, [field]: 'changed' }), field).not.toBe(before)}+ expect(transcriptReadIdentity({ ...session, processIdentity: { ...session.processIdentity!, pid: 8 } })).not.toBe(before)+ expect(transcriptReadIdentity({ ...session, processIdentity: { ...session.processIdentity!, startMarker: 'two' } })).not.toBe(before)+ expect(transcriptReadIdentity({ ...session, processIdentity: null })).not.toBe(before)+})++it('keeps a Linux process bound across a wall-clock correction, but rejects pid reuse', () => {+ const session = { agentId: 'a', sessionId: 's', engine: 'claude', transcriptPath: '/t',+ processIdentity: { pid: 7, executable: 'claude', startMarker: 'one', startTicks: 42 } } as RegisteredSession+ const before = transcriptReadIdentity(session)+ session.processIdentity!.startMarker = 'two'+ expect(transcriptReadIdentity(session)).toBe(before)+ session.processIdentity!.startTicks = 43+ expect(transcriptReadIdentity(session)).not.toBe(before)})diff --git a/cli/src/core/transcripts/readIdentity.ts b/cli/src/core/transcripts/readIdentity.tsindex 99372cc37..24a50659a 100644--- a/cli/src/core/transcripts/readIdentity.ts+++ b/cli/src/core/transcripts/readIdentity.ts@@ -1,7 +1,9 @@import type { RegisteredSession } from '../../lib/registry.js'+import { processIdentityKey } from '../../lib/terminalRuntime.js'/** Copy before yielding: registry rows can be mutated in place while a worker is reading. */export function transcriptReadIdentity(session: RegisteredSession | undefined): string {return session ? JSON.stringify([session.agentId, session.sessionId, session.engine, session.transcriptPath,- session.codexHome, session.boundAt, session.processIdentity]) : ''+ session.codexHome, session.boundAt,+ session.processIdentity ? processIdentityKey(session.engine, session.processIdentity) : undefined]) : ''}diff --git a/cli/src/engines/facets/live.ts b/cli/src/engines/facets/live.tsindex a5789d764..deb1e4e6f 100644--- a/cli/src/engines/facets/live.ts+++ b/cli/src/engines/facets/live.ts@@ -16,14 +16,18 @@ export interface LiveRead {failure?: string}-/** Stateful interpretation owned by an engine. Core never edits a parser's fields. */-export interface LiveParser {+/** Core's observation and explicit closure authority; the implementation may live in a worker. */+export interface LiveState {/** The binding this parser interprets. A session id reused by another engine needs a fresh parser. */readonly engine: stringreadonly turnOpen: boolean- ingest(line: string): LiveReadsnapshot(): LiveTurncloseTurn(reason: 'cancel' | 'abandoned' | 'hook'): void+}++/** Stateful interpretation owned by an engine. Core never edits a parser's fields. */+export interface LiveParser extends LiveState {+ ingest(line: string): LiveReadwindowStart(offset: number): void}diff --git a/cli/src/engines/worker/liveFiles.ts b/cli/src/engines/worker/liveFiles.tsnew file mode 100644index 000000000..6876221d6--- /dev/null+++ b/cli/src/engines/worker/liveFiles.ts@@ -0,0 +1,25 @@+import { createHash } from 'node:crypto'+import { open } from 'node:fs/promises'+import type { LiveStamp } from './liveProtocol.js'++/** Fingerprint a delivered prefix without rereading the conversation. At byte zero no content exists+ * to preserve: an empty or missing transcript becoming a file is ordinary first-turn delivery. */+export async function stampAt(file: string, offset: number): Promise<LiveStamp | null> {+ if (offset === 0) return null+ const handle = await open(file, 'r')+ try {+ const stat = await handle.stat()+ if (stat.size < offset) throw new Error('ENGINE_TRANSCRIPT_CHANGED')+ const bytes = Math.min(offset, 512)+ const first = Buffer.alloc(bytes), last = Buffer.alloc(bytes)+ const head = await handle.read(first, 0, bytes, 0)+ const tail = await handle.read(last, 0, bytes, offset - bytes)+ if (head.bytesRead !== bytes || tail.bytesRead !== bytes) throw new Error('ENGINE_TRANSCRIPT_CHANGED')+ return { device: stat.dev, inode: stat.ino, bytes, digest: createHash('sha256').update(first).update(last).digest('hex') }+ } finally { await handle.close() }+}++export function sameStamp(a: LiveStamp | null, b: LiveStamp | null): boolean {+ return a === null ? b === null : b !== null && a.device === b.device && a.inode === b.inode+ && a.bytes === b.bytes && a.digest === b.digest+}diff --git a/cli/src/engines/worker/liveProtocol.ts b/cli/src/engines/worker/liveProtocol.tsnew file mode 100644index 000000000..c33db831b--- /dev/null+++ b/cli/src/engines/worker/liveProtocol.ts@@ -0,0 +1,127 @@+/** Private, pull-driven live transcript protocol. A cursor acknowledges the previous complete page. */+import type { LiveEvent } from '../kit/events.js'+import type { LiveTurn } from '../facets/live.js'+import type { RegisteredSession } from '../../lib/registry.js'+import { record, sessionEvent } from './protocol.js'++export const LIVE_VERSION = 1+export const LIVE_CAPABILITIES = 'engine_live_capabilities'+export const LIVE_PREPARE = 'engine_live_prepare'+export const LIVE_READ = 'engine_live_read'+export const LIVE_PART = 'engine_live_part'+export const LIVE_CLOSE = 'engine_live_close'+export const LIVE_FORGET = 'engine_live_forget'+export const LIVE_WAIT_MS = 15_000+export const LIVE_PAGE_BYTES = 1024 * 1024+/** Match the legacy tailer's small-rewrite replay limit; larger rewrites hydrate from their end. */+export const LIVE_HISTORY_BYTES = 32 * 1024 * 1024+/** A single record may exceed a page; transfer it in bounded fragments, never a giant socket frame. */+export const LIVE_RESULT_BYTES = 64 * 1024 * 1024+export const LIVE_PART_BYTES = 256 * 1024+export const LIVE_CACHE_BYTES = 128 * 1024 * 1024+export const LIVE_CACHE_SESSIONS = 256+export const LIVE_IN_FLIGHT = 1++export type LiveSession = Pick<RegisteredSession, 'agentId' | 'sessionId' | 'engine' | 'transcriptPath'+ | 'codexHome' | 'cwd' | 'model' | 'cliVersion'>++/** Bytes at the cursor, including file identity. Appends do not change it; replacement/truncation does. */+export interface LiveStamp { device: number; inode: number; bytes: number; digest: string }+export interface LiveCursor {+ /** Changes even for empty polls: acknowledging an empty page permits observing later appends. */+ serial: number+ /** The frozen end used to choose the original parser window. Recovery folds that same window,+ * including later accepted turns, preserving thinking ids, tool links and continuing goals. */+ origin: number+ offset: number+ turn: LiveTurn+ /** Core's explicit cancellation/relaunch closure, which is not necessarily present in the file. */+ closed: false | 'cancel' | 'abandoned' | 'hook'+ /** A frozen attach end while staging; null once this cursor belongs to a live tail. */+ prepareEnd: number | null+ /** A first turn emitted after activation may include its final complete, unterminated record. */+ completeUntil: number | null+ /** A rewritten file's existing records replay as history, even across pages or worker restart. */+ historyUntil?: number+ stamp: LiveStamp | null+}+export interface LiveFrame {+ raw: string+ /** Profile metadata can begin before the turn's parser window. */+ profile: boolean+ observe: boolean+ events: LiveEvent[]+ failure?: string+ replay: boolean+ turn: LiveTurn+}+export interface LivePage {+ frames: LiveFrame[]+ cursor: LiveCursor+ /** Attach reached its frozen end; later appends belong to a subsequent live read. */+ prepared?: true+ failed?: true+ content: boolean+ more: boolean+ records: number+ turnFrom: number+ profileFrom: number+ end: number+ lastStarted: LiveEvent | null+}++export function liveTurn(value: unknown): value is LiveTurn {+ return record(value) && typeof value.identity === 'string' && value.identity.length <= 512+ && typeof value.turnOpen === 'boolean' && typeof value.continued === 'boolean'+}+export function liveStamp(value: unknown): value is LiveStamp | null {+ return value === null || (record(value) && Number.isSafeInteger(value.device) && Number.isSafeInteger(value.inode)+ && typeof value.bytes === 'number' && Number.isInteger(value.bytes) && value.bytes > 0 && value.bytes <= 512+ && typeof value.digest === 'string' && /^[a-f0-9]{64}$/.test(value.digest))+}+export function liveCursor(value: unknown): value is LiveCursor {+ return record(value) && Number.isSafeInteger(value.serial) && (value.serial as number) >= 0+ && Number.isSafeInteger(value.origin) && (value.origin as number) >= 0+ && typeof value.offset === 'number' && Number.isSafeInteger(value.offset) && value.offset >= 0+ && liveTurn(value.turn) && [false, 'cancel', 'abandoned', 'hook'].includes(value.closed as false | string)+ && (value.prepareEnd === null || (typeof value.prepareEnd === 'number' && Number.isSafeInteger(value.prepareEnd) && value.prepareEnd >= value.offset))+ && (value.completeUntil === null || (typeof value.completeUntil === 'number' && Number.isSafeInteger(value.completeUntil) && value.completeUntil >= value.offset))+ && (value.historyUntil === undefined || (typeof value.historyUntil === 'number' && Number.isSafeInteger(value.historyUntil) && value.historyUntil > value.offset))+ && liveStamp(value.stamp)+}++export function liveEvent(value: unknown): value is LiveEvent {+ if (sessionEvent(value)) return true+ if (!record(value) || !record(value.payload)) return false+ const p = value.payload+ if (value.type === 'turn_started') return typeof p.userMessage === 'string'+ if (value.type === 'turn_ended') return p.aborted === undefined || p.aborted === true+ return value.type === 'subagent_finished' && typeof p.id === 'string' && typeof p.status === 'string'+ && (p.summary === undefined || typeof p.summary === 'string')+}++export function livePage(value: unknown): value is LivePage {+ if (!record(value) || !liveCursor(value.cursor) || !Array.isArray(value.frames)) return false+ if (![value.records, value.turnFrom, value.profileFrom, value.end].every(n => typeof n === 'number' && Number.isSafeInteger(n) && n >= 0)) return false+ return (value.end as number) >= value.cursor.offset && (value.profileFrom as number) <= (value.turnFrom as number)+ && typeof value.content === 'boolean' && typeof value.more === 'boolean' && (value.prepared === undefined || (value.prepared === true && value.cursor.prepareEnd === null))+ && (value.failed === undefined || value.failed === true)+ && (value.lastStarted === null || (liveEvent(value.lastStarted) && value.lastStarted.type === 'turn_started'))+ && value.frames.every(f => record(f) && typeof f.raw === 'string' && typeof f.profile === 'boolean'+ && typeof f.observe === 'boolean' && typeof f.replay === 'boolean' && liveTurn(f.turn)+ && (f.failure === undefined || typeof f.failure === 'string') && Array.isArray(f.events) && f.events.every(liveEvent))+}++export interface LivePull {+ token: string+ session: LiveSession+ /** Null only for the first page of an attach. Each later cursor acknowledges the previous page. */+ cursor: LiveCursor | null+ fromStart: boolean+ replay: boolean+ /** Activate an empty parser first, then stream the first turn with ordinary bounded live pages. */+ liveStart?: boolean+ /** Rebuild after replacement/truncation; stream a small rewritten file as history after activation. */+ rewritten?: boolean+ end?: number+}diff --git a/cli/src/engines/worker/liveRequests.spec.ts b/cli/src/engines/worker/liveRequests.spec.tsnew file mode 100644index 000000000..2d9f49511--- /dev/null+++ b/cli/src/engines/worker/liveRequests.spec.ts@@ -0,0 +1,257 @@+import { appendFile, mkdtemp, readFile, rename, rm, writeFile } from 'node:fs/promises'+import { tmpdir } from 'node:os'+import { join } from 'node:path'+import { afterEach, describe, expect, it, vi } from 'vitest'+import { liveFor } from '../live.js'+import type { ServiceRequests } from '../../core/api.js'+import type { LivePage, LiveSession } from './liveProtocol.js'+import { LIVE_CAPABILITIES, LIVE_CLOSE, LIVE_FORGET, LIVE_PART, LIVE_PREPARE, LIVE_READ, LIVE_WAIT_MS } from './liveProtocol.js'+import { engineLiveRequests, type LiveRequestDeps } from './liveRequests.js'++const dirs: string[] = []+const who = { owner: true, local: true }+const prompt = (engine: string, text: string) => JSON.stringify(engine === 'claude'+ ? { type: 'user', message: { role: 'user', content: text } }+ : { type: 'event_msg', payload: { type: 'user_message', message: text } }) + '\n'+const done = (engine: string) => JSON.stringify(engine === 'claude'+ ? { type: 'assistant', message: { role: 'assistant', content: [{ type: 'text', text: 'done' }], stop_reason: 'end_turn' } }+ : { type: 'event_msg', payload: { type: 'task_complete', last_agent_message: 'done' } }) + '\n'+async function setup(engine: 'claude' | 'codex' = 'claude', lines = '', limits?: LiveRequestDeps['limits']) {+ const dir = await mkdtemp(join(tmpdir(), 'engine-stream-')); dirs.push(dir)+ const file = join(dir, 'transcript.jsonl'); await writeFile(file, lines)+ const session: LiveSession = { agentId: 'agent', sessionId: 'session', engine, transcriptPath: file,+ cwd: dir, model: null, cliVersion: null, codexHome: dir }+ const fresh = () => engineLiveRequests(engine, { limits })+ let worker = fresh()+ const payload = { version: 1, token: 'binding', session, cursor: null as LivePage['cursor'] | null, fromStart: false, replay: false }+ const send = (type = payload.cursor ? LIVE_READ : LIVE_PREPARE, overrides: Record<string, unknown> = {}) => worker[type]({ ...payload, ...overrides }, who) as Promise<Record<string, any>>+ const page = async (overrides: Record<string, unknown> = {}): Promise<LivePage> => {+ const reply = await send(undefined, overrides)+ expect(reply.error).toBeUndefined()+ expect(reply.answer).toBeDefined()+ payload.cursor = reply.answer.cursor+ return reply.answer+ }+ return { file, session, payload, fresh, send, page, worker: () => worker, restart: () => { worker = fresh() } }+}+afterEach(async () => { vi.useRealTimers(); vi.restoreAllMocks(); await Promise.all(dirs.splice(0).map(dir => rm(dir, { recursive: true, force: true }))) })++describe('engine live checkpoints', () => {+ it.each(['claude', 'codex'] as const)('%s preserves thinking identities and cross-turn state across a restart', async engine => {+ const thinking = (text: string) => JSON.stringify(engine === 'claude'+ ? { type: 'assistant', message: { role: 'assistant', content: [{ type: 'thinking', thinking: text }] } }+ : { type: 'response_item', payload: { type: 'reasoning', summary: [{ text }] } }) + '\n'+ const t = await setup(engine, prompt(engine, 'old') + thinking('earlier') + done(engine))+ const continuous = t.fresh()+ const firstAsk = { ...t.payload }+ await t.page()+ await continuous[LIVE_PREPARE](firstAsk, who)+ await appendFile(t.file, prompt(engine, 'current') + thinking('before restart'))+ const secondAsk = { ...t.payload }+ await t.page()+ await continuous[LIVE_READ](secondAsk, who)+ const finalAsk = { ...t.payload }+ t.restart()+ await appendFile(t.file, thinking('after restart') + done(engine))+ const recovered = await t.page()+ const expected = await continuous[LIVE_READ](finalAsk, who) as { answer: LivePage }+ expect(recovered.frames.flatMap(frame => frame.events)).toEqual(expected.answer.frames.flatMap(frame => frame.events))+ expect(recovered.cursor.turn).toEqual(expected.answer.cursor.turn)+ })++ it('remembers a Codex goal across an ordinary turn and worker restart', async () => {+ const goal = JSON.stringify({ type: 'response_item', payload: { type: 'message', role: 'user', content: [{ type: 'input_text',+ text: '<codex_internal_context source="goal"><objective>Finish the task</objective></codex_internal_context>' }] } }) + '\n'+ const t = await setup('codex', goal + done('codex'))+ await t.page()+ await appendFile(t.file, prompt('codex', 'ordinary') + done('codex'))+ await t.page()+ t.restart()+ await appendFile(t.file, goal)+ expect((await t.page()).frames.flatMap(frame => frame.events).find(event => event.type === 'turn_started')?.payload)+ .toEqual({ userMessage: 'Continuing goal: Finish the task' })+ })++ it.each(['claude', 'codex'] as const)('%s reconstructs after a worker restart without redelivering accepted turns', async engine => {+ const t = await setup(engine, prompt(engine, 'history') + done(engine))+ const first = await t.page()+ expect(first.prepared).toBe(true)+ expect(first.frames.flatMap(f => f.events)).toEqual([])+ expect(first.cursor.turn.turnOpen).toBe(false)+ await appendFile(t.file, prompt(engine, 'live'))+ const live = await t.page()+ expect(live.frames.flatMap(f => f.events).filter(e => e.type === 'turn_started')).toHaveLength(1)+ expect(live.cursor.turn.turnOpen).toBe(true)+ t.restart()+ const caughtUp = await t.page()+ expect(caughtUp.frames).toEqual([])+ expect(caughtUp.cursor.turn).toEqual(live.cursor.turn)+ await appendFile(t.file, done(engine))+ const end = await t.page()+ expect(end.frames.flatMap(f => f.events).filter(e => e.type === 'turn_ended')).toHaveLength(1)+ expect(end.cursor.turn.identity).toBe(live.cursor.turn.identity)+ expect(end.cursor.turn.turnOpen).toBe(false)+ })++ it('returns the exact unacknowledged page on a retry, even after new file content arrives', async () => {+ const t = await setup('claude', prompt('claude', 'first'))+ t.payload.fromStart = true; t.payload.replay = true+ const first = await t.send()+ await appendFile(t.file, done('claude'))+ expect(await t.send()).toEqual(first)+ t.payload.cursor = first.answer.cursor+ const next = await t.page()+ expect(next.frames.flatMap(f => f.events).some(e => e.type === 'turn_ended')).toBe(true)+ })++ it('keeps a frozen attach end across pages and a worker restart, then delivers subsequent appends live', async () => {+ const t = await setup('claude', prompt('claude', 'x'.repeat(700_000)) + done('claude'))+ t.payload.fromStart = true; t.payload.replay = true+ const first = await t.page()+ expect(first.prepared).toBeUndefined()+ expect(first.cursor.prepareEnd).toBe(first.end)+ const frozen = first.end+ await appendFile(t.file, prompt('claude', 'after freeze'))+ t.restart()+ const final = await t.page()+ expect(final.prepared).toBe(true)+ expect(final.end).toBe(frozen)+ expect(final.frames.flatMap(f => f.events).some(e => e.type === 'turn_started')).toBe(false)+ const next = await t.page()+ expect(next.frames.flatMap(f => f.events).find(e => e.type === 'turn_started')?.payload).toEqual({ userMessage: 'after freeze' })+ })++ it('holds partial UTF-8 records on disk until a newline completes them', async () => {+ const t = await setup()+ await t.page()+ const record = Buffer.from(prompt('claude', '😺 partial'))+ const split = record.indexOf(Buffer.from('😺')) + 2+ await appendFile(t.file, record.subarray(0, split))+ const partial = await t.page()+ expect(partial.frames).toEqual([])+ expect(partial.cursor.offset).toBe(0)+ await appendFile(t.file, record.subarray(split))+ const whole = await t.page()+ expect(whole.frames.flatMap(f => f.events).find(e => e.type === 'turn_started')?.payload).toEqual({ userMessage: '😺 partial' })+ expect(whole.cursor.offset).toBe(record.length)+ })++ it.each(['claude', 'codex'] as const)('%s replays a small rewritten file as history across pages and restart, then resumes live delivery', async engine => {+ const t = await setup(engine, prompt(engine, 'x'.repeat(700_000)) + done(engine))+ const active = await t.page({ rewritten: true, end: 1 })+ expect(active.prepared).toBe(true)+ expect(active.frames).toEqual([])+ expect(active.cursor.historyUntil).toBe(active.end)+ const first = await t.page({ rewritten: true })+ expect(first.more).toBe(true)+ expect(first.frames.every(frame => frame.replay)).toBe(true)+ t.restart()+ await appendFile(t.file, prompt(engine, 'fresh'))+ const rest = await t.page({ rewritten: true })+ expect(rest.frames.flatMap(frame => frame.events.map(event => [event.type, frame.replay])))+ .toContainEqual(['turn_ended', true])+ expect(rest.frames.flatMap(frame => frame.events.map(event => [event.type, frame.replay])))+ .toContainEqual(['turn_started', false])+ expect(rest.cursor.historyUntil).toBeUndefined()+ })++ it('accepts complete attach records without a trailing newline and finishes after trailing blank bytes', async () => {+ const t = await setup('claude', prompt('claude', 'attached').trimEnd())+ expect((await t.page()).prepared).toBe(true)+ const empty = await setup('claude', ' ')+ expect((await empty.page()).prepared).toBe(true)+ })++ it('detects replacement, truncation and a rewritten acknowledged boundary without acknowledging any new bytes', async () => {+ for (const change of ['replace', 'truncate', 'rewrite']) {+ const t = await setup('claude', prompt('claude', 'one'))+ const first = await t.page()+ if (change === 'replace') { await writeFile(t.file + '.new', await readFile(t.file)); await rename(t.file + '.new', t.file) }+ else await writeFile(t.file, change === 'truncate' ? '' : prompt('claude', 'two'))+ expect(await t.send()).toMatchObject({ error: 'ENGINE_TRANSCRIPT_CHANGED' })+ expect(t.payload.cursor).toEqual(first.cursor)+ }+ })++ it('closes only the expected turn and preserves explicit closure through restart', async () => {+ const t = await setup('claude', prompt('claude', 'one'))+ const first = await t.page()+ expect((await t.send(LIVE_CLOSE, { identity: 'stale', reason: 'hook' })).answer.closed).toBe(false)+ const close = await t.send(LIVE_CLOSE, { identity: first.cursor.turn.identity, reason: 'hook' })+ expect(close.answer.closed).toBe(true)+ expect(await t.send(LIVE_CLOSE, { identity: first.cursor.turn.identity, reason: 'hook' })).toEqual(close)+ t.payload.cursor = close.answer.cursor+ t.restart()+ expect((await t.page()).cursor.turn.turnOpen).toBe(false)+ await appendFile(t.file, prompt('claude', 'two'))+ const second = await t.page()+ expect(second.cursor.turn.identity).not.toBe(first.cursor.turn.identity)+ expect(second.cursor.closed).toBe(false)+ expect((await t.send(LIVE_CLOSE, { identity: first.cursor.turn.identity, reason: 'hook' })).answer.closed).toBe(false)+ })++ it('recovers evicted parser state from core checkpoints and forgets detached candidates', async () => {+ const t = await setup('claude', prompt('claude', 'one'), { sessions: 1 })+ const first = await t.page()+ await t.send(LIVE_PREPARE, { token: 'second', cursor: null })+ await appendFile(t.file, done('claude'))+ expect((await t.page()).frames.flatMap(f => f.events).filter(e => e.type === 'turn_ended')).toHaveLength(1)+ expect(await t.send(LIVE_FORGET)).toMatchObject({ forgotten: true })+ expect((await t.page()).cursor.turn.identity).toBe(first.cursor.turn.identity)+ })++ it('fragments large results with a stable digest and refuses expired or invalid fragments', async () => {+ const t = await setup('claude', prompt('claude', 'first'), { inline: 100, part: 123 })+ const first = await t.send()+ expect(first.part).toBeTypeOf('string')+ const chunks: Buffer[] = [Buffer.from(first.data, 'base64')]+ for (let offset = chunks[0].length; offset < first.bytes;) {+ const part = await t.send(LIVE_PART, { part: first.part, offset })+ expect(part.hash).toBe(first.hash)+ expect(part.offset).toBe(offset)+ const chunk = Buffer.from(part.data, 'base64'); chunks.push(chunk); offset += chunk.length+ }+ const decoded = JSON.parse(Buffer.concat(chunks).toString('utf8'))+ expect(decoded.answer.prepared).toBe(true)+ expect(await t.send(LIVE_PART, { part: first.part, offset: -1 })).toMatchObject({ error: 'ENGINE_INVALID_REQUEST' })+ await t.send(LIVE_FORGET)+ expect(await t.send(LIVE_PART, { part: first.part, offset: 0 })).toMatchObject({ error: 'ENGINE_STALE_REPLY' })+ })++ it('rejects oversized results explicitly without advancing the caller checkpoint', async () => {+ const t = await setup('claude', prompt('claude', 'first'), { reply: 100 })+ expect(await t.send()).toMatchObject({ error: 'ENGINE_REPLY_TOO_LARGE', retryable: false })+ expect(t.payload.cursor).toBeNull()+ })++ it('limits work and recycles a hung worker, without any capability to kill core or an engine CLI', async () => {+ vi.useFakeTimers()+ let resolve!: (value: NonNullable<ReturnType<typeof liveFor>>) => void+ const adapter = new Promise<NonNullable<ReturnType<typeof liveFor>>>(r => { resolve = r })+ const recycle = vi.fn()+ const requests = engineLiveRequests('claude', { load: () => adapter, recycle })+ const payload = { version: 1, token: 'one', cursor: null, fromStart: false, replay: false,+ session: { agentId: 'agent', sessionId: 'one', engine: 'claude', transcriptPath: null, cwd: null, model: null, cliVersion: null } }+ const first = requests[LIVE_PREPARE](payload, who)+ expect(await requests[LIVE_PREPARE](payload, who)).toMatchObject({ error: 'ENGINE_BUSY' })+ await vi.advanceTimersByTimeAsync(LIVE_WAIT_MS)+ expect(await first).toMatchObject({ error: 'ENGINE_UNAVAILABLE' })+ expect(recycle).toHaveBeenCalledOnce()+ resolve(liveFor('claude')!)+ })++ it('rejects public calls, wrong versions, wrong engines and malformed cursors before loading a parser', async () => {+ const t = await setup()+ const load = vi.fn(async () => liveFor('claude')!)+ const requests: ServiceRequests = engineLiveRequests('claude', { load })+ expect(await requests[LIVE_CAPABILITIES]({ version: 1 }, who)).toEqual({ version: 1, live: 1, engine: 'claude' })+ for (const caller of [{ ...who, owner: false }, { ...who, local: false }, { ...who, connection: 'public' }]) {+ expect(await requests[LIVE_PREPARE](t.payload, caller)).toMatchObject({ error: 'ENGINE_INVALID_REQUEST' })+ }+ for (const fields of [{ version: 2 }, { token: '' }, { cursor: {} }, { session: { ...t.session, engine: 'codex' } },+ { session: { ...t.session, transcriptPath: 'relative' } }, { fromStart: 'yes' }, { end: -1 }]) {+ expect(await requests[LIVE_PREPARE]({ ...t.payload, ...fields }, who)).toMatchObject({ error: 'ENGINE_INVALID_REQUEST' })+ }+ expect(load).not.toHaveBeenCalled()+ })+})diff --git a/cli/src/engines/worker/liveRequests.ts b/cli/src/engines/worker/liveRequests.tsnew file mode 100644index 000000000..db694ab04--- /dev/null+++ b/cli/src/engines/worker/liveRequests.ts@@ -0,0 +1,145 @@+import { createHash, randomUUID } from 'node:crypto'+import { isAbsolute } from 'node:path'+import type { ServiceRequests } from '../../core/api.js'+import type { RegisteredSession } from '../../lib/registry.js'+import type { RuntimeField } from '../../lib/runtimeProfile.js'+import type { EngineLive } from '../facets/live.js'+import { record, READER_REPLY_BYTES, type ReaderEngine } from './protocol.js'+import { LiveStreams } from './liveStreams.js'+import {+ liveCursor, LIVE_CACHE_BYTES, LIVE_CACHE_SESSIONS, LIVE_CAPABILITIES, LIVE_CLOSE, LIVE_FORGET,+ LIVE_IN_FLIGHT, LIVE_PART, LIVE_PART_BYTES, LIVE_PREPARE, LIVE_READ, LIVE_RESULT_BYTES, LIVE_VERSION, LIVE_WAIT_MS,+ type LiveSession, type LivePull,+} from './liveProtocol.js'++const loadLive = {+ claude: async () => (await import('../claude/live.js')).live,+ codex: async () => (await import('../codex/live.js')).live,+}+const path = (value: unknown): value is string => typeof value === 'string' && value.length <= 32_768+ && !value.includes('\0') && isAbsolute(value)+const label = (value: unknown): value is string => typeof value === 'string' && value.length > 0 && value.length <= 200+const nullableText = (value: unknown): value is string | null => value === null || (typeof value === 'string' && value.length <= 2_000)+const position = (value: unknown): value is number => typeof value === 'number' && Number.isSafeInteger(value) && value >= 0++function sessionOf(value: unknown, engine: ReaderEngine): LiveSession | null {+ if (!record(value) || !label(value.agentId) || !label(value.sessionId) || value.engine !== engine+ || (value.transcriptPath !== null && !path(value.transcriptPath))+ || (value.codexHome != null && !path(value.codexHome)) || (value.cwd !== null && !path(value.cwd))+ || !nullableText(value.model) || !nullableText(value.cliVersion)) return null+ return { agentId: value.agentId, sessionId: value.sessionId, engine, transcriptPath: value.transcriptPath,+ codexHome: value.codexHome as string | null | undefined, cwd: value.cwd, model: value.model, cliVersion: value.cliVersion }+}++function pullOf(payload: Record<string, unknown>, engine: ReaderEngine): LivePull | null {+ const session = sessionOf(payload.session, engine)+ if (!session || !label(payload.token) || (payload.cursor !== null && !liveCursor(payload.cursor))+ || typeof payload.fromStart !== 'boolean' || typeof payload.replay !== 'boolean'+ || (payload.end !== undefined && !position(payload.end)) || (payload.liveStart !== undefined && typeof payload.liveStart !== 'boolean')+ || (payload.rewritten !== undefined && typeof payload.rewritten !== 'boolean')) return null+ return { token: payload.token, session, cursor: payload.cursor, fromStart: payload.fromStart,+ replay: payload.replay, ...(payload.liveStart === undefined ? {} : { liveStart: payload.liveStart }),+ ...(payload.rewritten === undefined ? {} : { rewritten: payload.rewritten }),+ ...(payload.end === undefined ? {} : { end: payload.end }) }+}++interface Reply {+ input: string+ id: string+ bytes: Buffer+ hash: string+}++export interface LiveRequestDeps {+ load?: () => Promise<EngineLive>+ fields?: (session: LiveSession, raw: string) => readonly RuntimeField[]+ recycle?: () => void+ /** Small budgets let tests exercise real eviction/fragmentation without allocating giant fixtures. */+ limits?: { reply?: number; cache?: number; sessions?: number; inline?: number; part?: number }+}++/** Private methods, available only on core's authenticated owner link. One active read per worker,+ * one cached response per token, and bounded LRU storage. Retrying an unacknowledged cursor is exact. */+export function engineLiveRequests(engine: ReaderEngine, deps: LiveRequestDeps = {}): ServiceRequests {+ const replies = new Map<string, Reply>()+ const parts = new Map<string, Reply>()+ const replyLimit = deps.limits?.reply ?? LIVE_RESULT_BYTES+ const cacheLimit = deps.limits?.cache ?? LIVE_CACHE_BYTES+ const sessionLimit = deps.limits?.sessions ?? LIVE_CACHE_SESSIONS+ const inlineLimit = deps.limits?.inline ?? READER_REPLY_BYTES+ const partSize = deps.limits?.part ?? LIVE_PART_BYTES+ let bytes = 0, pending = 0+ let streams: Promise<LiveStreams> | null = null+ const get = (): Promise<LiveStreams> => streams ??= (async () => {+ const adapter = await (deps.load ?? loadLive[engine])()+ // The runtime-profile facet is the next boundary: reuse its exact field probe during this+ // migration rather than introducing a second, disagreeing interpretation of model metadata.+ const profiles = deps.fields ? null : new (await import('../../lib/runtimeProfile.js')).RuntimeProfileManager()+ const fields = deps.fields ?? ((session: LiveSession, raw: string) => profiles!.transcriptFields(session as RegisteredSession, raw))+ return new LiveStreams(adapter, fields)+ })().catch((error) => { streams = null; throw error })+ const forget = async (token: string): Promise<void> => {+ const old = replies.get(token)+ if (old) { bytes -= old.bytes.length; replies.delete(token); parts.delete(old.id) }+ if (streams) (await streams).forget(token)+ }+ const failure = (error: string) => ({ version: LIVE_VERSION, error,+ retryable: error !== 'ENGINE_INVALID_REQUEST' && error !== 'ENGINE_REPLY_TOO_LARGE' })+ const fragment = (reply: Reply, offset: number): Record<string, unknown> => ({ version: LIVE_VERSION,+ part: reply.id, offset, bytes: reply.bytes.length, hash: reply.hash,+ data: reply.bytes.subarray(offset, offset + partSize).toString('base64') })+ const answer = (reply: Reply): Record<string, unknown> => reply.bytes.length <= inlineLimit+ ? JSON.parse(reply.bytes.toString('utf8')) as Record<string, unknown> : fragment(reply, 0)++ const handler = (type: string): ServiceRequests[string] => async (payload, asker, closed) => {+ if (!asker.owner || !asker.local || asker.connection !== undefined || payload.version !== LIVE_VERSION) return failure('ENGINE_INVALID_REQUEST')+ if (closed?.aborted) return failure('ENGINE_UNAVAILABLE')+ if (type === LIVE_CAPABILITIES) return { version: LIVE_VERSION, live: LIVE_VERSION, engine }+ if (type === LIVE_PART) {+ if (!label(payload.part) || !position(payload.offset)) return failure('ENGINE_INVALID_REQUEST')+ const reply = parts.get(payload.part)+ return reply && payload.offset < reply.bytes.length ? fragment(reply, payload.offset) : failure('ENGINE_STALE_REPLY')+ }+ if (!label(payload.token)) return failure('ENGINE_INVALID_REQUEST')+ if (pending >= LIVE_IN_FLIGHT) return failure('ENGINE_BUSY')+ if (type === LIVE_FORGET) { await forget(payload.token); return { version: LIVE_VERSION, forgotten: true } }+ const ask = pullOf(payload, engine)+ if (!ask || (type === LIVE_PREPARE && ask.cursor !== null) || (type === LIVE_READ && ask.cursor === null)+ || (type === LIVE_CLOSE && (!ask.cursor || typeof payload.identity !== 'string'+ || !['cancel', 'abandoned', 'hook'].includes(payload.reason as string)))) return failure('ENGINE_INVALID_REQUEST')+ const input = JSON.stringify([type, ask, payload.identity, payload.reason])+ const previous = replies.get(ask.token)+ if (previous?.input === input) {+ replies.delete(ask.token); replies.set(ask.token, previous)+ return answer(previous)+ }+ pending+++ let timer: ReturnType<typeof setTimeout> | undefined+ const deadline = new Promise<Record<string, unknown>>((resolve) => {+ timer = setTimeout(() => { resolve(failure('ENGINE_UNAVAILABLE')); (deps.recycle ?? (() => process.exit(1)))() }, LIVE_WAIT_MS)+ })+ try {+ return await Promise.race([deadline, (async () => {+ const worker = await get()+ const value = type === LIVE_CLOSE+ ? await worker.close(ask, payload.identity as string, payload.reason as 'cancel' | 'abandoned' | 'hook')+ : await worker.pull(ask)+ if (closed?.aborted) { await forget(ask.token); return failure('ENGINE_UNAVAILABLE') }+ const result = Buffer.from(JSON.stringify({ version: LIVE_VERSION, answer: value }))+ if (result.length > replyLimit || result.length > cacheLimit) {+ await forget(ask.token)+ return failure('ENGINE_REPLY_TOO_LARGE')+ }+ if (previous) { bytes -= previous.bytes.length; replies.delete(ask.token); parts.delete(previous.id) }+ while (bytes + result.length > cacheLimit || replies.size >= sessionLimit) await forget(replies.keys().next().value!)+ const reply: Reply = { input, id: randomUUID(), bytes: result, hash: createHash('sha256').update(result).digest('hex') }+ replies.set(ask.token, reply); parts.set(reply.id, reply); bytes += result.length+ return answer(reply)+ })()])+ } catch (error) {+ await forget(ask.token)+ return failure(error instanceof Error && error.message === 'ENGINE_TRANSCRIPT_CHANGED' ? error.message : 'ENGINE_UNAVAILABLE')+ } finally { clearTimeout(timer); pending-- }+ }+ return Object.fromEntries([LIVE_CAPABILITIES, LIVE_PREPARE, LIVE_READ, LIVE_PART, LIVE_CLOSE, LIVE_FORGET].map(type => [type, handler(type)]))+}diff --git a/cli/src/engines/worker/liveStreams.ts b/cli/src/engines/worker/liveStreams.tsnew file mode 100644index 000000000..f57521a8f--- /dev/null+++ b/cli/src/engines/worker/liveStreams.ts@@ -0,0 +1,176 @@+/** Engine-owned parsers with core-owned checkpoints. No stream progresses until core pulls it. */+import { stat } from 'node:fs/promises'+import type { EngineLive, LiveParser, LiveTurn } from '../facets/live.js'+import type { LiveEvent } from '../kit/events.js'+import { isWholeRecord, locateAttachSpan, type AttachSpan } from '../../lib/attachTranscript.js'+import type { RuntimeField } from '../../lib/runtimeProfile.js'+import { streamRecords } from '../../lib/transcriptTail.js'+import { sameStamp, stampAt } from './liveFiles.js'+import { LIVE_HISTORY_BYTES, LIVE_PAGE_BYTES, type LiveCursor, type LiveFrame, type LivePage, type LiveSession, type LivePull } from './liveProtocol.js'++interface Stream {+ session: LiveSession+ parser: LiveParser+ span: AttachSpan+ turn: LiveTurn+ closed: LiveCursor['closed']+ cursor: LiveCursor | null+ lastStarted: LiveEvent | null+ head: boolean+ activation: { end: number; history: boolean } | null+}++const emptySpan = (): AttachSpan => ({ end: 0, turnFrom: 0, profileFrom: 0, head: null, seeds: [] })+const sameCursor = (a: LiveCursor | null, b: LiveCursor | null): boolean => JSON.stringify(a) === JSON.stringify(b)+const binding = (s: LiveSession): string => JSON.stringify([s.agentId, s.sessionId, s.engine, s.transcriptPath, s.codexHome])+function changed(): never { throw new Error('ENGINE_TRANSCRIPT_CHANGED') }++export class LiveStreams {+ private readonly streams = new Map<string, Stream>()+ constructor(private readonly adapter: EngineLive, private readonly fields: (session: LiveSession, raw: string) => readonly RuntimeField[]) {}++ forget(token: string): void { this.streams.delete(token) }++ private ingest(stream: Stream, token: string, raw: string, offset: number): ReturnType<LiveParser['ingest']> {+ const before = stream.parser.snapshot().identity+ const result = stream.parser.ingest(raw)+ const after = stream.parser.snapshot()+ if (after.identity !== before) {+ stream.turn = { ...after, identity: `${token}:${offset.toString(36)}` }+ stream.closed = false+ } else {+ stream.turn = { ...after, identity: stream.turn.identity }+ if (after.turnOpen) stream.closed = false+ }+ for (const event of result.events) if (event.type === 'turn_started') stream.lastStarted = event+ return result+ }++ private async create(ask: LivePull): Promise<Stream> {+ const parser = this.adapter.create(ask.session)+ const stream: Stream = { session: { ...ask.session }, parser, span: emptySpan(),+ turn: { ...parser.snapshot(), identity: `${ask.token}:empty` }, closed: false,+ cursor: null, lastStarted: null, head: true, activation: null }+ const file = ask.session.transcriptPath+ if (!ask.cursor && (ask.liveStart || ask.rewritten)) {+ let end = 0+ try { if (file) end = (await stat(file)).size }+ catch (error) { if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error }+ if (ask.liveStart || end <= LIVE_HISTORY_BYTES) {+ stream.activation = { end, history: !!ask.rewritten }+ return stream+ }+ }+ if (!file) return stream+ const cursor = ask.cursor+ if (cursor && !sameStamp(cursor.stamp, await stampAt(file, cursor.offset))) changed()+ const end = cursor ? cursor.origin : ask.rewritten ? undefined : ask.end+ const rules = this.adapter.attachRules!((line) => this.fields(ask.session, line))+ try {+ stream.span = await locateAttachSpan(file, rules, { end, fromStart: ask.fromStart })+ ?? changed()+ } catch (error) {+ // An announced path can precede its file. It is an empty stream only before any byte is accepted.+ if ((error as NodeJS.ErrnoException).code !== 'ENOENT' || (cursor && cursor.offset > 0)) throw error+ }+ parser.windowStart(stream.span.turnFrom)+ for (const seed of stream.span.seeds) this.ingest(stream, ask.token, seed, stream.span.turnFrom)+ if (cursor) {+ // Rebuild only through core's acknowledged position. No recovered event is delivered again.+ const start = Math.min(stream.span.turnFrom, cursor.offset)+ if (cursor.offset > start) {+ const read = await streamRecords(file, start, cursor.offset, (raw, offset) => {+ this.ingest(stream, ask.token, raw, offset)+ }, isWholeRecord)+ if (!read || read.next !== cursor.offset) changed()+ }+ if (!sameStamp(cursor.stamp, await stampAt(file, cursor.offset))) changed()+ if (cursor.closed) parser.closeTurn(cursor.closed)+ stream.turn = { ...cursor.turn }+ stream.closed = cursor.closed+ stream.cursor = cursor+ stream.head = false+ }+ return stream+ }++ private async get(ask: LivePull): Promise<Stream> {+ let stream = this.streams.get(ask.token)+ if (!stream || binding(stream.session) !== binding(ask.session) || !sameCursor(stream.cursor, ask.cursor)) {+ stream = await this.create(ask)+ this.streams.set(ask.token, stream)+ }+ return stream+ }++ async pull(ask: LivePull): Promise<LivePage> {+ const stream = await this.get(ask)+ const file = ask.session.transcriptPath+ if (!ask.cursor && stream.activation) {+ const { end, history } = stream.activation+ const cursor: LiveCursor = { serial: 1, origin: 0, offset: 0, turn: stream.turn, closed: false,+ prepareEnd: null, completeUntil: history ? null : end || null, stamp: null,+ ...(history && end > 0 ? { historyUntil: end } : {}) }+ stream.cursor = cursor+ return { frames: [], cursor, prepared: true, content: end > 0, more: false, records: 0,+ turnFrom: 0, profileFrom: 0, end, lastStarted: null }+ }+ const preparing = !ask.cursor || ask.cursor.prepareEnd !== null+ const start = ask.cursor?.offset ?? stream.span.profileFrom+ let end = preparing ? stream.span.end : start+ if (file) {+ if (ask.cursor && !sameStamp(ask.cursor.stamp, await stampAt(file, start))) changed()+ try { if (!preparing) end = ask.cursor?.completeUntil ?? (await stat(file)).size }+ catch (error) { if ((error as NodeJS.ErrnoException).code !== 'ENOENT' || start > 0) throw error }+ }+ if (end < start) changed()+ const frames: LiveFrame[] = []+ let bytes = 0+ let records = 0+ if (preparing && stream.head && stream.span.head !== null) {+ const raw = stream.span.head+ frames.push({ raw, profile: true, observe: false, events: [], replay: false, turn: { ...stream.turn } })+ bytes += Buffer.byteLength(raw)+ }+ stream.head = false+ let stopped = false+ const read = file && end > start ? await streamRecords(file, start, end, (raw, offset) => {+ const fold = !preparing || offset >= stream.span.turnFrom+ const result = fold ? this.ingest(stream, ask.token, raw, offset) : { events: [] }+ if (fold) records+++ const events = preparing && !ask.replay ? [] : result.events+ frames.push({ raw, profile: true, observe: fold, events, replay: offset < (ask.cursor?.historyUntil ?? 0), turn: { ...stream.turn },+ ...(!preparing && result.failure !== undefined ? { failure: result.failure } : {}) })+ // Account for normalized output as well as raw input. A large record occupies its own page;+ // the reply transport fragments it and enforces the independent hard result bound.+ bytes += Buffer.byteLength(raw) + Buffer.byteLength(JSON.stringify(events))+ return stopped = bytes >= LIVE_PAGE_BYTES+ }, preparing || ask.cursor?.completeUntil != null ? isWholeRecord : () => false) : { next: start, partial: false, records: 0 }+ if (!read) changed()+ const prepared = preparing && (!stopped || read.next >= end)+ const cursor: LiveCursor = { serial: (ask.cursor?.serial ?? 0) + 1, origin: ask.cursor?.origin ?? stream.span.end,+ offset: read.next, turn: { ...stream.turn }, closed: stream.closed,+ prepareEnd: preparing && !prepared ? end : null,+ completeUntil: stopped && read.next < end ? ask.cursor?.completeUntil ?? null : null,+ ...(ask.cursor?.historyUntil !== undefined && read.next < ask.cursor.historyUntil ? { historyUntil: ask.cursor.historyUntil } : {}),+ stamp: file ? await stampAt(file, read.next) : null }+ // Verify the old boundary again before publishing; a concurrent rewrite invalidates this page.+ if (file && ask.cursor && !sameStamp(ask.cursor.stamp, await stampAt(file, start))) changed()+ stream.cursor = cursor+ return { frames, cursor, ...(prepared ? { prepared: true } : {}), content: records > 0 || read.partial,+ more: stopped && read.next < end || ask.cursor?.completeUntil != null && cursor.completeUntil === null,+ records, turnFrom: stream.span.turnFrom, profileFrom: stream.span.profileFrom, end,+ lastStarted: prepared ? stream.lastStarted : null }+ }++ async close(ask: LivePull, identity: string, reason: Exclude<LiveCursor['closed'], false>): Promise<{ closed: boolean; cursor: LiveCursor }> {+ if (!ask.cursor) throw new Error('ENGINE_INVALID_REQUEST')+ const stream = await this.get(ask)+ if (stream.turn.identity !== identity || !stream.turn.turnOpen) return { closed: false, cursor: ask.cursor }+ stream.parser.closeTurn(reason)+ stream.closed = reason+ stream.turn = { ...stream.turn, turnOpen: false }+ stream.cursor = { ...ask.cursor, serial: ask.cursor.serial + 1, closed: reason, turn: stream.turn }+ return { closed: true, cursor: stream.cursor }+ }+}diff --git a/cli/src/engines/worker/process.ts b/cli/src/engines/worker/process.tsindex d09bd30cb..d63332115 100644--- a/cli/src/engines/worker/process.ts+++ b/cli/src/engines/worker/process.ts@@ -5,6 +5,7 @@ import { TranscriptPager } from '../../lib/transcriptPages.js'import { runServiceProcess, type ServiceProcess } from '../../services/process.js'import type { ServiceProcessOptions } from '../../serviceProcess.js'import type { EngineTranscript, HistoryAsk, TranscriptSession } from '../facets/transcript.js'+import { engineLiveRequests } from './liveRequests.js'import {EngineReadError, record, READER_HISTORY, READER_IN_FLIGHT, READER_LAST_TURN, READER_REPLY_BYTES,READER_SERVICES, READER_VERSION, READER_WAIT_MS, type ReaderEngine, type ReaderErrorCode,@@ -84,6 +85,6 @@ export interface EngineProcessOptions extends ServiceProcessOptions {export function runEngineReader(engine: ReaderEngine, options: EngineProcessOptions): ServiceProcess {return (options.run ?? runServiceProcess)({name: READER_SERVICES[engine], socketPath: options.socketPath, machineId: options.machineId,- token: options.token, requests: options.requests ?? engineReaderRequests(engine),+ token: options.token, requests: options.requests ?? { ...engineReaderRequests(engine), ...engineLiveRequests(engine) },})}diff --git a/cli/src/engines/worker/protocol.ts b/cli/src/engines/worker/protocol.tsindex e067136ab..f78602e69 100644--- a/cli/src/engines/worker/protocol.ts+++ b/cli/src/engines/worker/protocol.ts@@ -37,7 +37,7 @@ function subagent(value: unknown): boolean {}/** Reject malformed adapter output before handing it to a window; never accept live turn mutations. */-function event(value: unknown): value is SessionEvent {+export function sessionEvent(value: unknown): value is SessionEvent {if (!record(value) || !record(value.payload)) return falseconst p = value.payloadswitch (value.type) {@@ -60,7 +60,7 @@ function event(value: unknown): value is SessionEvent {export function historyAnswer(value: unknown): value is HistoryAnswer {return record(value) && Object.keys(value).every(key => ['events', 'timestamp', 'hasMore', 'oldestCursor', 'staleCursor'].includes(key))&& typeof value.timestamp === 'string' && Number.isFinite(Date.parse(value.timestamp))- && Array.isArray(value.events) && value.events.every(event)+ && Array.isArray(value.events) && value.events.every(sessionEvent)&& (value.hasMore === undefined || typeof value.hasMore === 'boolean')&& (value.oldestCursor === undefined || value.oldestCursor === null || typeof value.oldestCursor === 'string')&& (value.staleCursor === undefined || value.staleCursor === true)diff --git a/cli/src/harnessd/master.ts b/cli/src/harnessd/master.tsindex 4d2f2e491..702cf20b6 100644--- a/cli/src/harnessd/master.ts+++ b/cli/src/harnessd/master.ts@@ -345,7 +345,7 @@ export function runMaster(config: MasterConfig): Supervisor {// Told which services this master runs, so it routes to exactly those and runs the rest itself;// and, from the lean bundle, which cli.js is its CLI (leanCoreEntry.ts).env: {- ...env, ...extra, HARNESSD_SERVICE_TOKEN: token, ...serviceProcessesEnv(specs),+ ...env, ...extra, HARNESSD_SERVICE_TOKEN: token, ...serviceProcessesEnv(specs, process.pid),...(script === config.scriptPath ? {} : { [LEAN_CORE_SCRIPT_ENV]: config.scriptPath }),},stdio: ['ignore', 'inherit', 'inherit', 'ipc'],@@ -409,4 +409,3 @@ export function runMaster(config: MasterConfig): Supervisor {if (!reexec?.replacing()) services.start()return supervisor}-diff --git a/cli/src/harnessd/services.spec.ts b/cli/src/harnessd/services.spec.tsindex a89da7629..08ae000aa 100644--- a/cli/src/harnessd/services.spec.ts+++ b/cli/src/harnessd/services.spec.ts@@ -1,6 +1,6 @@import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'import {- DEFAULT_SERVICE_OPTIONS, KNOWN_SERVICES, SERVICE_HOSTS, SERVICE_PROCESSES_ENV, ServiceSupervisor, UPDATER_HOST, serviceOptions, serviceProcessesEnv, serviceSpecs, servicesTheMasterRuns,+ DEFAULT_SERVICE_OPTIONS, ENGINE_LIVE_ENV, KNOWN_SERVICES, SERVICE_HOSTS, SERVICE_PROCESSES_ENV, ServiceSupervisor, UPDATER_HOST, masterRunsLiveEngines, serviceOptions, serviceProcessesEnv, serviceSpecs, servicesTheMasterRuns,type ServiceSpec, type ServiceSupervisorOptions,} from './services.js'import type { CoreHandle } from './supervisor.js'@@ -472,17 +472,17 @@ describe('which services the core leaves to its master', () => {it('is what the master says it runs, told in a form a core from before the list reads too', () => {const specs = serviceSpecs({ HARNESSD_SERVICES: 'search,workspaces' }, SERVICE_HOSTS)- expect(serviceProcessesEnv(specs)).toEqual({ [SERVICE_PROCESSES_ENV]: 'search,workspaces', HARNESSD_SERVICES: 'search,workspaces' })+ expect(serviceProcessesEnv(specs, 42)).toEqual({ [SERVICE_PROCESSES_ENV]: 'search,workspaces', HARNESSD_SERVICES: 'search,workspaces', [ENGINE_LIVE_ENV]: '42:1' })// A process named as one of its services is named whole: the viewers' process runs the Store beside them.- expect(serviceProcessesEnv(serviceSpecs({ HARNESSD_SERVICES: 'viewers' }, SERVICE_HOSTS))).toMatchObject({ [SERVICE_PROCESSES_ENV]: 'viewers,store' })- expect(serviceProcessesEnv(serviceSpecs({ HARNESSD_SERVICES: 'store' }, SERVICE_HOSTS))).toMatchObject({ [SERVICE_PROCESSES_ENV]: 'store' })+ expect(serviceProcessesEnv(serviceSpecs({ HARNESSD_SERVICES: 'viewers' }, SERVICE_HOSTS), 42)).toMatchObject({ [SERVICE_PROCESSES_ENV]: 'viewers,store' })+ expect(serviceProcessesEnv(serviceSpecs({ HARNESSD_SERVICES: 'store' }, SERVICE_HOSTS), 42)).toMatchObject({ [SERVICE_PROCESSES_ENV]: 'store' })// By service, not by process: a core from before the edge host routes the services it knows of it.const hosted = serviceSpecs({ HARNESSD_SERVICES: 'edge' }, SERVICE_HOSTS)- expect(serviceProcessesEnv(hosted)).toEqual({ [SERVICE_PROCESSES_ENV]: 'workspaces,usage,monitor,projects,handoff,recaps,windowNames,shell', HARNESSD_SERVICES: 'workspaces,usage,monitor,projects,handoff,recaps,windowNames,shell' })- expect([...servicesTheMasterRuns({ ...supervised, ...serviceProcessesEnv(hosted) }, known)]).toEqual(['workspaces'])- expect(serviceProcessesEnv([])).toEqual({ [SERVICE_PROCESSES_ENV]: '', HARNESSD_SERVICES: 'none' })- expect([...servicesTheMasterRuns({ ...supervised, ...serviceProcessesEnv(specs) }, known)]).toEqual(['search', 'workspaces'])- expect([...servicesTheMasterRuns({ ...supervised, ...serviceProcessesEnv([]) }, known)]).toEqual([])+ expect(serviceProcessesEnv(hosted, 42)).toEqual({ [SERVICE_PROCESSES_ENV]: 'workspaces,usage,monitor,projects,handoff,recaps,windowNames,shell', HARNESSD_SERVICES: 'workspaces,usage,monitor,projects,handoff,recaps,windowNames,shell', [ENGINE_LIVE_ENV]: '42:1' })+ expect([...servicesTheMasterRuns({ ...supervised, ...serviceProcessesEnv(hosted, 42) }, known)]).toEqual(['workspaces'])+ expect(serviceProcessesEnv([], 42)).toEqual({ [SERVICE_PROCESSES_ENV]: '', HARNESSD_SERVICES: 'none', [ENGINE_LIVE_ENV]: '42:1' })+ expect([...servicesTheMasterRuns({ ...supervised, ...serviceProcessesEnv(specs, 42) }, known)]).toEqual(['search', 'workspaces'])+ expect([...servicesTheMasterRuns({ ...supervised, ...serviceProcessesEnv([], 42) }, known)]).toEqual([])// What the master says wins over whatever HARNESSD_SERVICES the core inherited; names it does not know are its own.expect([...servicesTheMasterRuns({ ...supervised, [SERVICE_PROCESSES_ENV]: ' teams, nothing ', HARNESSD_SERVICES: 'search' }, known)]).toEqual(['teams'])})@@ -494,6 +494,17 @@ describe('which services the core leaves to its master', () => {expect([...servicesTheMasterRuns({ ...supervised, HARNESSD_SERVICES: 'search' }, known)]).toEqual(['search'])})+ it('accepts live engine support only from the actual supervising parent', () => {+ const env = { ...supervised, ...serviceProcessesEnv([], 42) }+ expect(masterRunsLiveEngines(env, 42)).toBe(true)+ expect(masterRunsLiveEngines(env, 43)).toBe(false)+ expect(masterRunsLiveEngines(supervised, 42)).toBe(false)+ expect(masterRunsLiveEngines({ ...env, [ENGINE_LIVE_ENV]: '1' }, 42)).toBe(false)+ expect(masterRunsLiveEngines({ ...env, [ENGINE_LIVE_ENV]: '42:2' }, 42)).toBe(false)+ expect(masterRunsLiveEngines({ ...env, HARNESSD_SUPERVISED: undefined }, 42)).toBe(false)+ expect(masterRunsLiveEngines({ ...env, HARNESSD_SERVICE_TOKEN: undefined }, 42)).toBe(false)+ })+it('is none without a master, or without the token its services would connect with', () => {expect([...servicesTheMasterRuns({ [SERVICE_PROCESSES_ENV]: 'search' }, known)]).toEqual([])expect([...servicesTheMasterRuns({ HARNESSD_SUPERVISED: '1', [SERVICE_PROCESSES_ENV]: 'search' }, known)]).toEqual([])diff --git a/cli/src/harnessd/services.ts b/cli/src/harnessd/services.tsindex cef1df4f6..86ffd54b0 100644--- a/cli/src/harnessd/services.ts+++ b/cli/src/harnessd/services.ts@@ -455,6 +455,8 @@ export function serviceOptions(env: NodeJS.ProcessEnv): ServiceSupervisorOptions/** How a master tells the core it starts which services it runs in their own processes. */export const SERVICE_PROCESSES_ENV = 'HARNESSD_SERVICE_PROCESSES'+/** Version of the live engine methods hosted by this master, independent of reader-only hosts. */+export const ENGINE_LIVE_ENV = 'HARNESSD_ENGINE_LIVE'/*** What a master puts in its core's environment about the services it runs in their own processes: the@@ -462,11 +464,17 @@ export const SERVICE_PROCESSES_ENV = 'HARNESSD_SERVICE_PROCESSES'* one this master finds back on disk after a rollback, reads the same answer for the services it knows* and runs none of them a second time.*/-export function serviceProcessesEnv(specs: readonly ServiceSpec[]): Record<string, string> {+export function serviceProcessesEnv(specs: readonly ServiceSpec[], masterPid: number): Record<string, string> {// The services, not the processes: a core knows what it routes by service, and one from before the// edge host still finds the services it knows here (workspaces) and runs the rest itself.const names = specs.flatMap((spec) => spec.services).join(',')- return { [SERVICE_PROCESSES_ENV]: names, HARNESSD_SERVICES: names || 'none' }+ return { [SERVICE_PROCESSES_ENV]: names, HARNESSD_SERVICES: names || 'none', [ENGINE_LIVE_ENV]: `${masterPid}:1` }+}++/** An older master may inherit a newer master's environment after rollback. Trust only this parent. */+export function masterRunsLiveEngines(env: NodeJS.ProcessEnv, parentPid: number): boolean {+ return env.HARNESSD_SUPERVISED === '1' && !!env.HARNESSD_SERVICE_TOKEN+ && env[ENGINE_LIVE_ENV] === `${parentPid}:1`}/**diff --git a/cli/src/services/AGENTS.md b/cli/src/services/AGENTS.mdindex 43c0c570e..8f88e956d 100644--- a/cli/src/services/AGENTS.md+++ b/cli/src/services/AGENTS.md@@ -51,7 +51,7 @@ that process. A process per risk, not per feature (`SERVICE_HOSTS` in `src/harne| Process | Services | Why its own | Started ||---|---|---|---|-| `engine-claude`, `engine-codex` | history/last-turn readers (`src/engines/worker/`) | engine parsing outside core; read-only snapshots and no core capabilities | on demand |+| `engine-claude`, `engine-codex` | history/last-turn readers and live parsers (`src/engines/worker/`) | engine parsing outside core; read-only snapshots and no core capabilities | on demand || `search` | search | native `node:sqlite`, the index's memory | always || `viewers` | viewers, store | the viewer servers, the remote viewer streams and rendered surfaces, minutes-long installs | always || `edge` | workspaces, usage, monitor, projects, handoff, recaps, windowNames, shell | light pure-JS services: isolated from the core, not from each other | always |diff --git a/cli/src/services/inline.spec.ts b/cli/src/services/inline.spec.tsindex fc05a7908..1c061f523 100644--- a/cli/src/services/inline.spec.ts+++ b/cli/src/services/inline.spec.ts@@ -1,5 +1,6 @@import { describe, expect, it } from 'vitest'import { engineTranscriptFor } from '../engines/transcripts.js'+import { liveFor } from '../engines/live.js'import { startGateway } from '../gateway/start.js'import { startCommandBar } from './commandBar.js'import { startDevices } from './devices.js'@@ -23,6 +24,6 @@ import { startWorkspaces } from './workspaces.js'describe('the services the core runs in its own process only when they do not run in theirs', () => {it('are their own starts, unchanged: the same services either way', () => {- expect({ ...inline }).toEqual({ engineTranscriptFor, startCommandBar, startDevices, startHandoff, startGateway, startModels, startMonitor, startOrchestrator, startProjects, startRecaps, startSearch, startSharing, startShell, startStore, startTeamsInCore, startUsage, startViewers, startWifi, startWindowNames, startWorkspaces })+ expect({ ...inline }).toEqual({ engineTranscriptFor, liveFor, startCommandBar, startDevices, startHandoff, startGateway, startModels, startMonitor, startOrchestrator, startProjects, startRecaps, startSearch, startSharing, startShell, startStore, startTeamsInCore, startUsage, startViewers, startWifi, startWindowNames, startWorkspaces })})})diff --git a/cli/src/services/inline.ts b/cli/src/services/inline.tsindex 8eac27305..2639d20bb 100644--- a/cli/src/services/inline.ts+++ b/cli/src/services/inline.ts@@ -13,6 +13,7 @@* declared in core/api.ts, which it loads either way.*/export { engineTranscriptFor } from '../engines/transcripts.js'+export { liveFor } from '../engines/live.js'export { startGateway } from '../gateway/start.js'export { startCommandBar } from './commandBar.js'export { startDevices } from './devices.js'diff --git a/docs/design/2026-10-07-engine-live.md b/docs/design/2026-10-07-engine-live.mdindex 02842f29b..856a2de93 100644--- a/docs/design/2026-10-07-engine-live.md+++ b/docs/design/2026-10-07-engine-live.md@@ -21,10 +21,12 @@ A file line for a database engine cannot create a Claude parser for that engine.The old plain-terminal fallback remains explicit in the temporary live lookup;it does not acquire end-first attach rules.-**This increment still runs live parsers in core's process.** `engines/live.ts`+**The interface-only increment ran live parsers in core's process.** The following+[live worker increment](2026-10-07-engine-streams.md) moves supervised parsing+across the process boundary and records its remaining validation. `engines/live.ts`is temporary composition while the live transport is built. `lib/normalize.ts`is a compatibility export used by consumers still being migrated. The existing-supervised workers still isolate history and last-turn reads only.+supervised workers initially isolated history and last-turn reads only.## Preserved requirementsdiff --git a/docs/design/2026-10-07-engine-streams.md b/docs/design/2026-10-07-engine-streams.mdnew file mode 100644index 000000000..ae6f32336--- /dev/null+++ b/docs/design/2026-10-07-engine-streams.md@@ -0,0 +1,96 @@+# Live transcript workers++Tracking: [#1016](https://github.com/autonomous-ai/openharness/issues/1016).+Follows the [live interface](2026-10-07-engine-live.md) and+[reader workers](2026-10-07-engine-readers.md). This change is being validated;+it does not complete the full Claude Code/Codex isolation effort.++## Ownership++The existing supervised `engine-claude` and `engine-codex` processes now host+live parsers alongside history readers. They locate transcript windows, fold+records, retain parser state, and propose normalized events. Core owns session+bindings, accepted transcript checkpoints, explicit cancellation/closure, and+the event funnel. Generic file notifications stay in core; it neither tails nor+normalizes Claude Code/Codex JSONL through the legacy watcher in this mode.++`LiveState` is a core observation and closure handle. `LiveParser` adds the+engine's ingestion methods and exists inside its worker. Other engines keep+their current tailers and parsers.++## Checkpoints and delivery++- **ES-001 — Pull and acknowledgement.** One live operation runs per engine+ worker. Core serializes requests fairly across that engine's sessions. A+ request carries the last accepted cursor; retrying the same request returns+ the same cached page. Every successful page advances a serial, including empty+ polls, so acknowledging an empty response permits observing later appends.+- **ES-002 — Bounded delivery.** Pages target 1 MiB including raw records and+ normalized events. A single larger record can exceed that target. Results over+ the socket envelope travel in 256 KiB fragments with a digest, up to a 64 MiB+ serialized result limit. A worker's response cache is bounded to 128 MiB and+ 256 sessions. Oversized results fail explicitly without accepting their bytes.+ Remaining transcript data stays on disk. Capability negotiation and all reply+ fragments share one 15-second core deadline. Parser cleanup uses the same+ per-engine queue as reads, so retiring a parser cannot displace its replacement's+ first read.+- **ES-003 — Recovery equivalence.** Core retains the original attach end as+ well as the accepted offset. A replacement worker reconstructs the same+ original parser window through the accepted offset, silently, then reads new+ records. Reconstructing only the latest turn changed thinking ids and forgot+ a Codex goal across ordinary turns; regression tests compare recovery with a+ continuously running parser. Recovery streams records rather than retaining+ the whole file, but its elapsed work grows with bytes accepted since attach.+ The 15-second operation deadline and supervisor budgets still apply; long+ recovery is measured separately from paired steady-state CPU, memory and latency.+- **ES-004 — Binding and cancellation.** Replies are fenced by service connection+ generation, copied binding identity, parser handle, and core decision epoch.+ Closing an observed turn updates its checkpoint and invalidates an in-flight+ page. The worker applies its own closure behavior when reconstructing that+ checkpoint; an unseen newer turn can still open normally. Stop's identity+ check remains synchronous in core. Linux process start ticks preserve a binding+ across wall-clock corrections; a changed pid or start tick still invalidates it.+- **ES-005 — Attach handover.** A hold stops new pulls and waits for the current+ page. Replacement hydration is staged to that acknowledged byte. Installation+ is synchronous, and an expired hold, changed binding, explicit tail move or+ intervening cancellation rejects the candidate. Initial live content activates+ an empty parser first and streams afterward; core does not collect the whole+ first turn in an `initialEvents` array. Core retains first-turn delivery intent+ across a failed initial prepare until activation succeeds for that binding.+- **ES-006 — File changes.** A checkpoint fingerprints file identity and bytes at+ the accepted boundary. Replacement, truncation or a changed boundary rejects+ the page and schedules a held re-attach. Files up to 32 MiB activate a fresh+ parser and replay existing records as history in bounded pages; subsequent+ appends are live. The history boundary survives worker restart. Larger files+ hydrate from their new end. A trusted explicit tail move hydrates at the new+ offset without replay. This is not a checksum of every byte in the transcript.++## Availability and compatibility++Worker failure pauses its engine's transcript delivery. Core and the engine CLI+remain alive; the CLI keeps writing its transcript. Recovery delivers the+unaccepted records once. An engine worker that remains parked cannot deliver+live events until it runs again. History requests report explicit unavailability.+Other engines and core operations remain independent.++Private live methods are not public routes and accept only core's authenticated+owner connection. Workers retain no core query capabilities. Capability replies+are checked before use. A master's explicit live-host version selects the new+path only when its report names the core's actual parent pid, so a stale inherited+flag does not select an older master's reader-only worker. An older master or explicit inline configuration uses the+compatibility module. Runtime errors never select an inline parser.++## Remaining work and evidence++Validation covers the core and supervisor per-file gates, protocol and compatibility+contracts, private-daemon crash/freeze/memory/parking recovery, rewrite replay,+turns, compaction and history. Results and source-specific receipts are recorded in+the PR and the working branch's ignored `.harness`. Paired performance measurements,+a separate code review and ready-PR CI still precede merge.++Raw records still reach the runtime-profile and device-evidence consumers, and+the worker currently reuses the runtime-profile field probe for attach. These+are explicit migration bridges. Hook policy/admission/installers, launch and+session discovery/resume, pane/input interpretation, runtime models/profiles,+and the remaining shared helpers still need process boundaries. The complete+normal core import closure has not yet been cleared of Claude Code/Codex code.