Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

harnessd: a floor under every engine's whole transcript reads; compared with v0.3.57, nothing differs; round 16 (#760)

Dee6e33a8080b80
diff --git a/cli/e2e/compat.e2e.ts b/cli/e2e/compat.e2e.ts
new file mode 100644
index 000000000..c43747723
--- /dev/null
+++ b/cli/e2e/compat.e2e.ts
@@ -0,0 +1,301 @@
+/**
+ * What the apps see, compared with the released build. A daemon of the released build (`COMPAT_FROM`,
+ * its bundled `cli.js`) and one of this checkout's bundle run the same scenario: a Claude Code agent
+ * and a Codex agent created, a turn each, a question and a permission answered, search, every request
+ * the apps send (well formed and malformed), then the lifecycle: rename, cancel, fork, restart, stop,
+ * resume, close. Their replies, and the shape of the frames a turn, a question and a permission push,
+ * are compared once ids, times, paths and counters are made comparable.
+ *
+ * A field this build adds is reported and allowed: the apps ignore what they do not read. Any other
+ * difference (a field gone, a value changed, an error instead of an answer) is either listed in CHANGED
+ * with why it changed on purpose, or a regression. This is the check behind "a release breaks no app":
+ * the desktop app, `hn` and the phone were written against the released daemon's answers.
+ *
+ * Skipped unless COMPAT_FROM names a released bundle, so CI does not build old releases. Before a
+ * release: build the last released tag's bundle (`node build-bundle.mjs` in a checkout of it) and run
+ * `COMPAT_FROM=<that>/dist/cli.js npm run test:e2e -- compat`. `COMPAT_REPORT=<file>` writes both
+ * sides' answers and every difference, for reading one by one.
+ */
+import { execFileSync } from 'node:child_process'
+import { copyFileSync, mkdirSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { dirname, join } from 'node:path'
+import { afterAll, beforeAll, describe, expect, it } from 'vitest'
+import { LocalClient, type Frame } from './harness/client.js'
+import { CLI_ROOT, IsolatedDaemon, until } from './harness/daemon.js'
+
+const FROM = process.env.COMPAT_FROM
+
+/**
+ * Differences on purpose, by the path that differs (a prefix covers everything under it), each with
+ * why. Keep this honest: a difference nobody can explain is a regression until shown otherwise.
+ */
+const CHANGED: Record<string, string> = {}
+
+type Engine = 'claude' | 'codex'
+type Answers = Record<string, unknown>
+const ENGINES: Engine[] = ['claude', 'codex']
+
+/** Makes one daemon's answers comparable with another's: its own ids, times, paths and counters. */
+function comparable(d: IsolatedDaemon, known: Map<string, string>) {
+ const roots = [...new Set([realpathSync(d.root), d.root])].sort((a, b) => b.length - a.length)
+ const UUID = /[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/gi
+ const VOLATILE = new Set(['pid', 'port', 'tookMs', 'durationMs', 'elapsedMs', 'uptimeMs', 'mtimeMs', 'requestId', 'version', 'cliVersion', 'daemonVersion', 'nonce', 'revision', 'seq', 'offset', 'bytes', 'size'])
+ let unknown = 0
+ const name = (id: string): string => {
+ const lower = id.toLowerCase()
+ if (!known.has(lower)) known.set(lower, `<id:${++unknown}>`)
+ return known.get(lower)!
+ }
+ const walk = (value: unknown, key = ''): unknown => {
+ if (typeof value === 'string') {
+ if (VOLATILE.has(key)) return '<v>'
+ let text = value
+ for (const root of roots) text = text.split(root).join('<root>')
+ return text.replace(UUID, name).replace(/\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d+)?Z/g, '<iso>')
+ // A new agent's default name carries the minute it was made: "Claude harness 10-4 23:09".
+ .replace(/\b\d{1,2}-\d{1,2} \d{1,2}:\d{2}\b/g, '<when>')
+ // A question's id is a hash of its session and its dialog, so it differs with the session id.
+ .replace(/\bq_[0-9a-f]{8}\b/g, '<question>')
+ // This machine's network name, which can change between the two runs.
+ .replace(/^[\w-]+\.(lan|local|home)$/, '<host>')
+ }
+ if (typeof value === 'number') {
+ if (VOLATILE.has(key)) return '<n>'
+ if (value > 1e12 && value < 3e12) return '<ms>'
+ if (value > 1.5e9 && value < 3e9) return '<s>'
+ return value
+ }
+ if (Array.isArray(value)) return value.map((item) => walk(item, key))
+ if (value && typeof value === 'object') return Object.fromEntries(Object.entries(value).map(([k, v]) => [k, walk(v, k)]))
+ return value
+ }
+ return walk
+}
+
+/** The frames a turn pushed, as their shape: each type in order (repeats folded), with its payload's keys. */
+function shape(frames: Frame[]): string[] {
+ const out: string[] = []
+ for (const frame of frames) {
+ const keys = Object.keys(frame.payload ?? {}).sort().join(',')
+ const line = `${frame.type} {${keys}}`
+ if (out.at(-1) !== line) out.push(line)
+ }
+ return out
+}
+
+/** Every path where two answers differ, with both values. */
+function differences(a: unknown, b: unknown, path = ''): Array<{ path: string; from: unknown; to: unknown }> {
+ if (JSON.stringify(a) === JSON.stringify(b)) return []
+ const objects = a && b && typeof a === 'object' && typeof b === 'object'
+ if (objects && !Array.isArray(a) && !Array.isArray(b)) {
+ const keys = [...new Set([...Object.keys(a as object), ...Object.keys(b as object)])].sort()
+ return keys.flatMap((key) => differences((a as Answers)[key], (b as Answers)[key], `${path}.${key}`))
+ }
+ if (Array.isArray(a) && Array.isArray(b) && a.length === b.length) return a.flatMap((item, i) => differences(item, b[i], `${path}[${i}]`))
+ return [{ path, from: a, to: b }]
+}
+
+/** The scenario, on one daemon: what it answered, made comparable. */
+async function scenario(d: IsolatedDaemon): Promise<Answers> {
+ const known = new Map<string, string>()
+ const walk = comparable(d, known)
+ const answers: Answers = {}
+ const client = await LocalClient.connect(d)
+ // A request some daemons never answer (`cancel` is fire-and-forget) is recorded as such: that is
+ // part of what the apps see too.
+ const ask = async (step: string, type: string, payload: Record<string, unknown> = {}, ms = 60_000): Promise<Record<string, any>> => {
+ const answer = await client.request(type, payload, ms).catch(() => ({ '<no reply>': true }))
+ answers[step] = walk(answer)
+ return answer
+ }
+ const rows = async () => (await client.request<{ agents: Array<Record<string, any>> }>('agents_list', { includeStopped: true }, 30_000)).agents
+ const agent: Record<Engine, string> = { claude: '', codex: '' }
+ const session: Record<Engine, string> = { claude: '', codex: '' }
+ const cwd = (engine: Engine) => join(d.projectsDir, `compat-${engine}`)
+
+ for (const engine of ENGINES) {
+ mkdirSync(cwd(engine), { recursive: true })
+ writeFileSync(join(cwd(engine), 'README.md'), `# compat ${engine}\n`)
+ // Asked until the daemon has wired its handlers: a released build answers before it has.
+ const created = await until(`a ${engine} agent`, async () => {
+ const answer = await client.request('agent_create', { engine, cwd: cwd(engine), bypassPermission: true }, 90_000)
+ return answer.error ? null : answer
+ }, 60_000, 1_000)
+ agent[engine] = created.agent.id
+ known.set(agent[engine].toLowerCase(), `<agent:${engine}>`)
+ const bound = await until(`${engine} to bind`, async () => {
+ const row = (await rows()).find((one) => one.id === agent[engine])
+ return row?.sessionId && row.status === 'active' ? row : null
+ }, 60_000, 500)
+ session[engine] = bound.sessionId
+ known.set(session[engine].toLowerCase(), `<session:${engine}>`)
+ // How far the launch had got when the reply was built is a race with the engine starting, on either
+ // build; what a launch still starting carries (`bypassPermission`) goes with it.
+ const { launch: _launch, bypassPermission: _bypass, ...agentRow } = created.agent
+ answers[`agent_create ${engine}`] = walk({ ...created, agent: agentRow })
+ }
+
+ for (const engine of ENGINES) {
+ const from = client.frames.length
+ const ended = client.next((frame) => frame.type === 'turn_ended' && frame.agentId === agent[engine], 60_000, `turn_ended (${engine})`)
+ client.send('message', { agentId: agent[engine], content: `compat turn on ${engine}` })
+ await ended
+ await new Promise((done) => setTimeout(done, 1_000))
+ answers[`frames of a turn on ${engine}`] = shape(client.frames.slice(from).filter((frame) => frame.agentId === agent[engine] || (frame.payload as Answers | undefined)?.agentId === agent[engine]))
+ }
+
+ // A question and a permission, answered as the window answers them: the frames they push, and what
+ // the answer gets back (under the question's own request id).
+ for (const engine of ENGINES) {
+ for (const [step, content, pick] of [['a question', '!ask', 'Coffee'], ['a permission', '!permit printf hi', 0]] as const) {
+ const from = client.frames.length
+ const asked = client.next((frame) => frame.type === 'commander_question' && frame.agentId === agent[engine], 30_000, `${step} (${engine})`)
+ client.send('message', { agentId: agent[engine], content })
+ const question = await asked
+ const shaped = question.payload?.questions?.[0]
+ answers[`${step} on ${engine}`] = walk(question.payload)
+ const ended = client.next((frame) => frame.type === 'turn_ended' && frame.agentId === agent[engine], 60_000, `turn_ended after ${step}`)
+ const replied = client.next((frame) => frame.type === 'question_response_result' && frame.payload?.requestId === question.payload.requestId, 45_000, `${step} answered`)
+ client.send('question_response', { requestId: question.payload.requestId, agentId: agent[engine], answers: { [shaped.q]: typeof pick === 'number' ? shaped.options[pick] : pick } })
+ answers[`${step} answered on ${engine}`] = walk((await replied).payload)
+ await ended
+ await new Promise((done) => setTimeout(done, 1_000))
+ answers[`frames of ${step} on ${engine}`] = shape(client.frames.slice(from).filter((frame) => frame.agentId === agent[engine] || (frame.payload as Answers | undefined)?.agentId === agent[engine]))
+ }
+ }
+
+ // What search finds, once it has indexed both conversations.
+ for (const engine of ENGINES) {
+ await until(`search to find the ${engine} conversation`, async () => {
+ const found = await client.request('session_search', { query: `compat turn on ${engine}` }, 30_000)
+ return JSON.stringify(found).includes(session[engine]) ? found : null
+ }, 60_000, 1_000)
+ await ask(`session_search ${engine}`, 'session_search', { query: `compat turn on ${engine}`, limit: 5 })
+ await ask(`session_tail ${engine}`, 'session_tail', { sessionId: session[engine], maxChars: 4_000 })
+ }
+ await ask('session_search for nothing', 'session_search', { query: 'nothing anyone said here' })
+
+ // Every read the apps make, once the rows have settled. Where an agent worked is read from its
+ // transcript at most every 15 s (lib/agentTokenUsage.ts REFRESH_MS), on either build: one refresh is
+ // let pass after the last turn, and the rows are read until two reads agree.
+ await new Promise((done) => setTimeout(done, 16_000))
+ await rows()
+ await new Promise((done) => setTimeout(done, 2_000))
+ let previous = ''
+ await until('the agents\' rows to settle', async () => {
+ const now = JSON.stringify(walk(await rows()))
+ const settled = now === previous
+ previous = now
+ return settled
+ }, 30_000, 1_500)
+ await ask('agents_list', 'agents_list')
+ await ask('agents_list with stopped', 'agents_list', { includeStopped: true })
+ for (const engine of ENGINES) {
+ await ask(`sessions_list ${engine}`, 'sessions_list', { agentId: agent[engine] })
+ await ask(`session_get ${engine}`, 'session_get', { sessionId: session[engine] })
+ await ask(`session_get ${engine}, one turn`, 'session_get', { sessionId: session[engine], limit: 1 })
+ await ask(`agent_recent ${engine}`, 'agent_recent', { agentId: agent[engine] })
+ await ask(`terminal_info ${engine}`, 'terminal_info', { agentId: agent[engine] })
+ await ask(`agent_read_file ${engine}`, 'agent_read_file', { agentId: agent[engine], path: 'README.md' })
+ }
+ await ask('models_list', 'models_list')
+ await ask('git_project_info', 'git_project_info', { path: cwd('claude') })
+ await ask('fs_list_dir', 'fs_list_dir', { path: d.projectsDir })
+ await ask('project_preview', 'project_preview', { path: cwd('claude') })
+ await ask('codex_profiles_list', 'codex_profiles_list')
+ await ask('claude_login_status', 'claude_login_status')
+ await ask('agents_cleanup_preview', 'agents_cleanup_preview')
+ await ask('harness_devices_list', 'harness_devices_list')
+ await ask('theme_set', 'theme_set', { background: '#000000', foreground: '#ffffff' })
+ await ask('a request nobody answers', 'compat_unknown_request')
+
+ // The same requests, malformed: what each refuses with.
+ const malformed = ['session_get', 'sessions_list', 'agent_recent', 'terminal_info', 'agent_read_file', 'agent_update', 'agent_fork',
+ 'agent_restart', 'agent_delete', 'agent_resume', 'agent_purge', 'agent_retarget', 'agent_close', 'cancel', 'question_response',
+ 'git_project_info', 'fs_list_dir', 'project_preview', 'session_tail', 'dsh_remove', 'dsh_install', 'dsh_update', 'theme_set']
+ for (const type of malformed) {
+ const ms = type === 'cancel' ? 5_000 : 60_000
+ await ask(`${type} with nothing`, type, {}, ms)
+ await ask(`${type} for no such agent`, type, { agentId: 'compat-no-such-agent', sessionId: 'compat-no-such-session', path: '/compat/no/such/path' }, ms)
+ }
+
+ // The lifecycle, as the apps drive it.
+ await ask('agent_update rename', 'agent_update', { agentId: agent.claude, name: 'Renamed in compat' })
+ await ask('cancel while idle', 'cancel', { agentId: agent.claude }, 5_000)
+ const forked = await ask('agent_fork', 'agent_fork', { agentId: agent.claude })
+ if (typeof forked.agent?.id === 'string') known.set(forked.agent.id.toLowerCase(), '<agent:fork>')
+ await ask('agent_restart', 'agent_restart', { agentId: agent.codex })
+ await until('codex to be back after its restart', async () => {
+ const row = (await rows()).find((one) => one.id === agent.codex)
+ return row?.sessionId && row.status === 'active' ? row : null
+ }, 60_000, 500)
+ await ask('agent_delete (stop)', 'agent_delete', { agentId: agent.codex })
+ await ask('agents_list after a stop', 'agents_list', { includeStopped: true })
+ await ask('agent_resume', 'agent_resume', { agentId: agent.codex })
+ await until('codex to be back after its resume', async () => {
+ const row = (await rows()).find((one) => one.id === agent.codex)
+ return row?.sessionId && row.status === 'active' ? row : null
+ }, 60_000, 500)
+ const closing = (await rows()).find((one) => one.id === agent.claude)
+ await ask('agent_close', 'agent_close', { agentId: agent.claude, sessionId: closing?.sessionId, createdAt: closing?.createdAt, mode: 'now' })
+ await ask('agents_list at the end', 'agents_list', { includeStopped: true })
+ client.close()
+ return answers
+}
+
+describe.skipIf(!FROM)('what the apps see, compared with the released build', () => {
+ let scratch = ''
+ const sides: Partial<Record<'released' | 'this', { version: string; answers: Answers }>> = {}
+
+ beforeAll(async () => {
+ scratch = mkdtempSync(join(tmpdir(), 'harnessd-compat-'))
+ // This checkout as it ships: its bundle, under harnessd's master.
+ const build = join(scratch, 'this')
+ execFileSync(process.execPath, ['build-bundle.mjs'], { cwd: CLI_ROOT, env: { ...process.env, BUNDLE_OUT_DIR: build }, stdio: 'pipe' })
+ // The released build as it runs: its bundle, its core on its own (it has no master).
+ const released = join(scratch, 'released')
+ mkdirSync(released)
+ copyFileSync(FROM!, join(released, 'cli.js'))
+ copyFileSync(join(dirname(FROM!), 'notify.mjs'), join(released, 'notify.mjs'))
+ for (const dir of [build, released]) writeFileSync(join(dir, 'package.json'), '{"type":"module"}\n')
+
+ for (const [side, options] of [
+ ['released', { scriptPath: join(released, 'cli.js'), noMaster: true, env: { ADAPTER_CLI_DIR: released } }],
+ ['this', { scriptPath: join(build, 'cli.js'), env: { ADAPTER_CLI_DIR: build } }],
+ ] as const) {
+ const d = await IsolatedDaemon.create(options)
+ try {
+ await d.start({ ready: 'port' })
+ const version = execFileSync(process.execPath, [options.scriptPath, 'version'], { encoding: 'utf8' }).trim()
+ sides[side] = { version, answers: await scenario(d) }
+ } catch (error) {
+ console.error(`---- ${side} daemon log\n${d.log().split('\n').slice(-120).join('\n')}`)
+ throw error
+ } finally {
+ await d.close()
+ }
+ }
+ }, 900_000)
+
+ afterAll(() => { if (scratch) rmSync(scratch, { recursive: true, force: true }) })
+
+ it('answers every request the way the released build did, but for the changes made on purpose', () => {
+ const released = sides.released!
+ const current = sides.this!
+ const found = differences(released.answers, current.answers)
+ const explained = (path: string) => Object.keys(CHANGED).find((prefix) => path === prefix || path.startsWith(`${prefix}.`) || path.startsWith(`${prefix}[`))
+ const added = found.filter((difference) => difference.from === undefined)
+ const unexplained = found.filter((difference) => difference.from !== undefined && !explained(difference.path))
+ if (process.env.COMPAT_REPORT) {
+ writeFileSync(process.env.COMPAT_REPORT, JSON.stringify({
+ released: released.version, this: current.version, unexplained, added,
+ explained: found.filter((difference) => difference.from !== undefined && explained(difference.path)),
+ answers: { released: released.answers, this: current.answers },
+ }, null, 2))
+ }
+ expect(unexplained, `${unexplained.length} differences from ${released.version}; see COMPAT_REPORT`).toEqual([])
+ // An explanation nothing needs any more is removed.
+ for (const prefix of Object.keys(CHANGED)) expect(found.some((difference) => explained(difference.path) === prefix), `${prefix} no longer differs`).toBe(true)
+ })
+})
diff --git a/cli/e2e/ends.e2e.ts b/cli/e2e/ends.e2e.ts
new file mode 100644
index 000000000..fac8dd9be
--- /dev/null
+++ b/cli/e2e/ends.e2e.ts
@@ -0,0 +1,189 @@
+/**
+ * How an agent's life ends, for Claude Code and Codex: closed now, closed only when idle, closed after
+ * its task (and that intent cancelled, or carried across a daemon restart), and purged with its history
+ * — and the search that finds a conversation until it is purged. A close never takes a turn that is
+ * still working unless it was asked to; a purge leaves nothing of the conversation behind.
+ */
+import { existsSync, mkdirSync, readdirSync } from 'node:fs'
+import { join } from 'node:path'
+import { afterEach, describe, expect, it, onTestFailed } from 'vitest'
+import { LocalClient, type Frame } from './harness/client.js'
+import { IsolatedDaemon, until } from './harness/daemon.js'
+
+type Engine = 'claude' | 'codex'
+const engines: Engine[] = ['claude', 'codex']
+
+const rows = async (client: LocalClient) =>
+ (await client.request<{ agents: Array<Record<string, any>> }>('agents_list', { includeStopped: true }, 30_000)).agents
+const row = async (client: LocalClient, agentId: string) => (await rows(client)).find((agent) => agent.id === agentId)
+async function create(daemon: IsolatedDaemon, client: LocalClient, engine: Engine, folder: string): Promise<Record<string, any>> {
+ const cwd = join(daemon.projectsDir, folder)
+ mkdirSync(cwd, { recursive: true })
+ const created = await client.request('agent_create', { engine, cwd, bypassPermission: true }, 90_000)
+ expect(created.error, `${folder}: ${JSON.stringify(created)}`).toBeUndefined()
+ return until(`${folder} to bind its conversation`, async () => {
+ const agent = await row(client, created.agent.id)
+ return agent?.sessionId && agent.status === 'active' ? agent : null
+ }, 60_000, 500)
+}
+const isTurn = (type: string, agentId: string) => (frame: Frame) => frame.type === type && frame.agentId === agentId
+async function turn(client: LocalClient, agentId: string, content: string): Promise<void> {
+ const ended = client.next(isTurn('turn_ended', agentId), 45_000, `turn_ended (${content})`)
+ client.send('message', { agentId, content })
+ await ended
+}
+const close = (client: LocalClient, agent: Record<string, any>, mode: string) =>
+ client.request('agent_close', { agentId: agent.id, sessionId: agent.sessionId, createdAt: agent.createdAt, mode }, 90_000)
+const stopped = (client: LocalClient, agentId: string, ms = 45_000) =>
+ until(`${agentId.slice(0, 8)} to stop`, async () => (await row(client, agentId))?.status === 'stopped' || null, ms, 500)
+/** The engine's own transcript files for a conversation, where the fake engines write them. */
+function transcriptsOf(daemon: IsolatedDaemon, sessionId: string): string[] {
+ const found: string[] = []
+ const walk = (dir: string): void => {
+ let entries: Array<{ name: string; isDirectory(): boolean }>
+ try { entries = readdirSync(dir, { withFileTypes: true }) as never } catch { return }
+ for (const entry of entries) {
+ const path = join(dir, entry.name)
+ if (entry.isDirectory()) walk(path)
+ else if (entry.name.endsWith('.jsonl') && entry.name.includes(sessionId)) found.push(path)
+ }
+ }
+ walk(join(daemon.root, 'claude', 'projects'))
+ walk(join(daemon.root, 'codex', 'sessions'))
+ return found
+}
+
+describe('how an agent\'s life ends', () => {
+ let daemon: IsolatedDaemon | undefined
+ afterEach(async () => { await daemon?.close(); daemon = undefined })
+ const fresh = async () => {
+ const d = await IsolatedDaemon.create()
+ daemon = d
+ onTestFailed(() => { console.log(`---- daemon log\n${d.log().split('\n').slice(-150).join('\n')}`) })
+ await d.start()
+ return d
+ }
+
+ it.each(engines)('%s: closed now it stops with its conversation kept, and opening it again goes on with it', async (engine) => {
+ const d = await fresh()
+ const client = await LocalClient.connect(d)
+ const agent = await create(d, client, engine, `close-now-${engine}`)
+ await turn(client, agent.id, 'before the close')
+ expect(await close(client, agent, 'inspect')).toMatchObject({ activity: 'idle' })
+ expect(await close(client, agent, 'now')).toMatchObject({ closed: true })
+ await stopped(client, agent.id)
+ expect((await client.request('agent_resume', { agentId: agent.id }, 90_000)).error).toBeUndefined()
+ await until('the agent to be back', async () => {
+ const now = await row(client, agent.id)
+ return now?.status === 'active' && now.sessionId === agent.sessionId ? now : null
+ }, 60_000, 500)
+ await turn(client, agent.id, 'after opening it again')
+ client.close()
+ })
+
+ it.each(engines)('%s: closed only when idle, a working agent is left working and says so', async (engine) => {
+ const d = await fresh()
+ const client = await LocalClient.connect(d)
+ const agent = await create(d, client, engine, `close-idle-${engine}`)
+ const ended = client.next(isTurn('turn_ended', agent.id), 45_000, 'turn_ended')
+ client.send('message', { agentId: agent.id, content: '!slow 4000' })
+ await client.next(isTurn('turn_started', agent.id), 30_000, 'turn_started')
+ await until('the agent to read as working', async () => (await row(client, agent.id))?.activity?.state === 'working' || null, 15_000, 250)
+ expect(await close(client, agent, 'inspect')).toMatchObject({ activity: 'working' })
+ expect(await close(client, agent, 'idle')).toMatchObject({ error: 'SESSION_NOT_IDLE', activity: 'working' })
+ await ended
+ expect((await row(client, agent.id))?.status).toBe('active')
+ await turn(client, agent.id, 'still here')
+ client.close()
+ })
+
+ it.each(engines)('%s: closed after its task, it finishes the turn, then closes on its own', async (engine) => {
+ const d = await fresh()
+ const client = await LocalClient.connect(d)
+ const agent = await create(d, client, engine, `close-after-${engine}`)
+ const ended = client.next(isTurn('turn_ended', agent.id), 45_000, 'turn_ended')
+ client.send('message', { agentId: agent.id, content: '!slow 3000' })
+ await client.next(isTurn('turn_started', agent.id), 30_000, 'turn_started')
+ expect(await close(client, agent, 'after_task')).toMatchObject({ deferred: true })
+ // Asking twice is the same plan.
+ expect(await close(client, agent, 'after_task')).toMatchObject({ deferred: true })
+ await ended
+ expect((await row(client, agent.id))?.status).toBe('active')
+ // Idle for a while (a turn boundary alone is not completion), then closed.
+ await stopped(client, agent.id, 45_000)
+ client.close()
+ })
+
+ it.each(engines)('%s: a close after its task, cancelled, leaves the agent open', async (engine) => {
+ const d = await fresh()
+ const client = await LocalClient.connect(d)
+ const agent = await create(d, client, engine, `close-cancel-${engine}`)
+ const ended = client.next(isTurn('turn_ended', agent.id), 45_000, 'turn_ended')
+ client.send('message', { agentId: agent.id, content: '!slow 2000' })
+ await client.next(isTurn('turn_started', agent.id), 30_000, 'turn_started')
+ expect(await close(client, agent, 'after_task')).toMatchObject({ deferred: true })
+ expect(await close(client, agent, 'cancel')).toMatchObject({ cancelled: true })
+ await ended
+ await new Promise((resolve) => setTimeout(resolve, 15_000))
+ expect((await row(client, agent.id))?.status).toBe('active')
+ await turn(client, agent.id, 'still open')
+ client.close()
+ })
+
+ it('a close after its task outlives a daemon restart, and is carried out once the agent is idle', async () => {
+ const d = await fresh()
+ let client = await LocalClient.connect(d)
+ const agent = await create(d, client, 'claude', 'close-across-restart')
+ client.send('message', { agentId: agent.id, content: '!slow 4000' })
+ await client.next(isTurn('turn_started', agent.id), 30_000, 'turn_started')
+ expect(await close(client, agent, 'after_task')).toMatchObject({ deferred: true })
+ client.close()
+ await d.restart()
+ client = await LocalClient.connect(d)
+ await stopped(client, agent.id, 60_000)
+ client.close()
+ })
+
+ it('a close for an agent that is not the one asked about changes nothing', async () => {
+ const d = await fresh()
+ const client = await LocalClient.connect(d)
+ const agent = await create(d, client, 'codex', 'close-changed')
+ expect(await close(client, { ...agent, sessionId: 'another-conversation' }, 'now')).toMatchObject({ error: 'AGENT_CHANGED' })
+ expect(await close(client, { ...agent, createdAt: new Date(0).toISOString() }, 'now')).toMatchObject({ error: 'AGENT_CHANGED' })
+ expect((await client.request('agent_close', { agentId: agent.id, mode: 'now' }, 30_000)).error).toBe('INVALID_CLOSE_REQUEST')
+ await turn(client, agent.id, 'untouched')
+ client.close()
+ })
+
+ it.each(engines)('%s: found by search until purged; purged, nothing of the conversation is left', async (engine) => {
+ const d = await fresh()
+ const client = await LocalClient.connect(d)
+ const agent = await create(d, client, engine, `purge-${engine}`)
+ const word = `zebrafish${engine}`
+ await turn(client, agent.id, `remember the ${word}`)
+ await until('search to find the conversation', async () => {
+ const found = await client.request('session_search', { query: word }, 30_000)
+ return JSON.stringify(found).includes(agent.sessionId) ? found : null
+ }, 60_000, 1_000)
+ const files = transcriptsOf(d, agent.sessionId)
+ expect(files.length).toBeGreaterThan(0)
+ const createdAt = Date.parse(agent.createdAt)
+ const review = await client.request('agent_purge', { agentId: agent.id, sessionId: agent.sessionId, createdAt, mode: 'inspect' }, 60_000)
+ expect(review.error, JSON.stringify(review)).toBeUndefined()
+ expect(review.reviewId).toBeTruthy()
+ // A review is used once: a delete with a made-up one is refused, and nothing goes.
+ const forged = await client.request('agent_purge', { agentId: agent.id, sessionId: agent.sessionId, createdAt, mode: 'delete', reviewId: 'not-a-review' }, 60_000)
+ expect(forged.error).toBe('DELETE_REFUSED')
+ expect((await row(client, agent.id))?.status).toBe('active')
+ const deleted = await client.request('agent_purge', { agentId: agent.id, sessionId: agent.sessionId, createdAt, mode: 'delete', reviewId: review.reviewId }, 90_000)
+ expect(deleted, JSON.stringify(deleted)).toMatchObject({ deleted: true, sessionDeleted: true })
+ for (const file of files) expect(existsSync(file), file).toBe(false)
+ await until('the agent to be gone', async () => !(await row(client, agent.id)) || null, 30_000, 500)
+ expect((await client.request('session_get', { sessionId: agent.sessionId, limit: 10 }, 30_000)).error).toBe('NOT_FOUND')
+ await until('search to forget it', async () => {
+ const found = await client.request('session_search', { query: word }, 30_000)
+ return JSON.stringify(found).includes(agent.sessionId) ? null : found
+ }, 30_000, 1_000)
+ client.close()
+ })
+})
diff --git a/cli/src/backendSocket.cappedHistory.spec.ts b/cli/src/backendSocket.cappedHistory.spec.ts
new file mode 100644
index 000000000..bb186787d
--- /dev/null
+++ b/cli/src/backendSocket.cappedHistory.spec.ts
@@ -0,0 +1,91 @@
+// `session_get` for an engine with no pages of its own reads its transcript from the end, bounded
+// (lib/transcriptTail.ts `tailFileCapped`): a transcript past the cap shows its newest history, and every
+// reply says so. A real file over the cap is 64 MB; the reader is made to cut short instead.
+import { copyFileSync, mkdirSync, rmSync, writeFileSync } from 'node:fs'
+import { join } from 'node:path'
+import { fileURLToPath } from 'node:url'
+import { afterAll, afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
+
+// Transcripts are only served from the engines' own folders, pointed at throwaway ones before the
+// config is read.
+const roots = vi.hoisted(() => {
+ const base = `${process.env.TMPDIR || '/tmp'}/capped-history-${process.pid}-${Date.now()}`
+ process.env.MUSE_HOME = `${base}/muse`
+ process.env.PI_HOME = `${base}/pi`
+ return { base, muse: `${base}/muse/sessions`, pi: `${base}/pi/agent/sessions` }
+})
+const reader = vi.hoisted(() => ({ cut: false }))
+vi.mock('./lib/transcriptTail.js', async (importOriginal) => {
+ const real = await importOriginal<typeof import('./lib/transcriptTail.js')>()
+ return {
+ ...real,
+ tailFileCapped: vi.fn(async (path: string, maxBytes?: number) => {
+ const read = await real.tailFileCapped(path, maxBytes)
+ return reader.cut ? { lines: read.lines.slice(-2), truncated: true } : read
+ }),
+ }
+})
+
+import { BackendSocket } from './backendSocket.js'
+import { registry } from './lib/registry.js'
+
+afterAll(() => rmSync(roots.base, { recursive: true, force: true }))
+
+describe('session_get past the cap, for an engine without pages', () => {
+ let socket: BackendSocket
+ let frames: Array<{ type: string; payload: Record<string, any> }>
+ const agents: string[] = []
+ let pane = 8800
+ let requests = 0
+
+ beforeEach(() => {
+ socket = new BackendSocket('fixture')
+ frames = []
+ socket.registerLocalClient('local:capped', { sendFrame: (frame) => { frames.push(frame as never); return true }, sendBinary: () => true })
+ vi.spyOn(console, 'warn').mockImplementation(() => {})
+ })
+ afterEach(async () => {
+ reader.cut = false
+ for (const agentId of agents.splice(0)) registry.removeAgent(agentId)
+ await socket.stop()
+ vi.restoreAllMocks()
+ })
+
+ const bind = (engine: 'muse' | 'pi', root: string, write: (file: string) => void): string => {
+ const dir = join(root, `capped-${++pane}`)
+ mkdirSync(dir, { recursive: true })
+ const file = join(dir, 'session.jsonl')
+ write(file)
+ const paneId = `%${pane}`
+ agents.push(registry.openPendingAgent({ engine, runtimes: [{ backend: 'tmux', paneId }], cwd: dir })!.agentId)
+ const sessionId = `${engine}-capped-${pane}`
+ registry.register({ engine, sessionId, transcriptPath: file, tmuxPane: paneId, cwd: dir })
+ expect(registry.resolve(sessionId)?.transcriptPath, 'bound to the transcript under test').toBe(file)
+ return sessionId
+ }
+ const ask = async (payload: Record<string, unknown>): Promise<Record<string, any>> => {
+ const requestId = `c${++requests}`
+ await (socket as any).dispatchDown({ type: 'session_get', payload: { requestId, ...payload } }, 'local:capped', 'local')
+ return frames.find((frame) => frame.type === 'session_get_result' && frame.payload.requestId === requestId)!.payload
+ }
+
+ it('says so in the whole read and the whole page, and leaves a transcript under the cap as it was', async () => {
+ const fixture = fileURLToPath(new URL('./lib/__fixtures__/muse-session.jsonl', import.meta.url))
+ const sessionId = bind('muse', roots.muse, (file) => copyFileSync(fixture, file))
+ const whole = await ask({ sessionId })
+ expect(whole.truncated).toBeUndefined()
+ reader.cut = true
+ const cut = await ask({ sessionId })
+ expect(cut).toMatchObject({ id: sessionId, engine: 'muse', truncated: true })
+ expect(cut.events.length).toBeLessThan(whole.events.length)
+ expect(await ask({ sessionId, limit: 50 })).toMatchObject({ truncated: true, hasMore: false })
+ expect(console.warn).toHaveBeenCalledWith(expect.stringMatching(/^\[backend\] session_get .*: the transcript is over 64 MB · its oldest history is not shown$/))
+ })
+
+ it('says so in a window and in a stale cursor\'s answer', async () => {
+ const sessionId = bind('pi', roots.pi, (file) => writeFileSync(file, Array.from({ length: 6 }, (_, i) => JSON.stringify({ type: 'message', id: `m${i}` })).join('\n') + '\n'))
+ reader.cut = true
+ expect(await ask({ sessionId, limit: 10 })).toMatchObject({ engine: 'pi', truncated: true })
+ expect(await ask({ sessionId, limit: 10, before: 'no-such-cursor' })).toMatchObject({ staleCursor: true, truncated: true })
+ })
+})
diff --git a/cli/src/backendSocket.ts b/cli/src/backendSocket.ts
index f123f7c81..50f7d816b 100644
--- a/cli/src/backendSocket.ts
+++ b/cli/src/backendSocket.ts
@@ -88,11 +88,10 @@ import { engineLabel } from './lib/agentNames.js'
import { DSH_ID_RE, dshSupportedEngines } from './dsh/manifest.js'
import { terminalHandoffRequest } from './lib/terminalHandoff.js'
import { routeVoiceTask } from './lib/voiceRouter.js'
-import { tailFile } from './lib/sessions.js'
import { stoppedAgents } from './lib/stoppedAgents.js'
import { messagesToEvents, windowRawLines, SubagentStats, type SessionEvent } from './lib/normalize.js'
import { TranscriptPager } from './lib/transcriptPages.js'
-import { streamRecords } from './lib/transcriptTail.js'
+import { streamRecords, tailFileCapped, WHOLE_READ_CAP_BYTES } from './lib/transcriptTail.js'
import { MediaPreviewError, readMediaPreviewChunk } from './lib/mediaPreview.js'
import { ViewerForwarder } from './lib/viewerForwarder.js'
import { InteractiveViewers } from './lib/interactiveViewer.js'
@@ -2358,7 +2357,12 @@ export class BackendSocket {
})
return
}
- const lines = await tailFile(s.transcriptPath, Infinity)
+ // Read from the end and bounded: these engines have no pages of their own yet, and one huge
+ // transcript read whole would take the whole daemon down (lib/transcriptTail.ts). History past
+ // the cap is not shown, and the reply says so.
+ const { lines, truncated: capped } = await tailFileCapped(s.transcriptPath)
+ if (capped) console.warn(`[backend] session_get ${sid(sessionId)}: the transcript is over ${WHOLE_READ_CAP_BYTES / 1024 / 1024} MB · its oldest history is not shown`)
+ const truncated = capped ? { truncated: true } : {}
const st = await stat(s.transcriptPath).catch(() => null)
const timestamp = new Date(st?.mtimeMs ?? Date.now()).toISOString()
@@ -2387,6 +2391,7 @@ export class BackendSocket {
events: fullEvents,
timestamp,
engine: s.engine,
+ ...truncated,
})
return
}
@@ -2418,6 +2423,7 @@ export class BackendSocket {
engine: s.engine,
hasMore: wholePage?.hasMore ?? false,
oldestCursor: wholePage?.oldestCursor ?? null,
+ ...truncated,
})
return
}
@@ -2429,7 +2435,7 @@ export class BackendSocket {
? windowCommandCodeLines(lines, { limit, before })
: windowRawLines(lines, { limit, before })
if (w.staleCursor) {
- reply(type, requestId, { id: sessionId, title: projectDisplayName(s), events: [], timestamp, engine: s.engine, hasMore: false, oldestCursor: null, staleCursor: true })
+ reply(type, requestId, { id: sessionId, title: projectDisplayName(s), events: [], timestamp, engine: s.engine, hasMore: false, oldestCursor: null, staleCursor: true, ...truncated })
return
}
const events = s.engine === 'cursor'
@@ -2457,6 +2463,7 @@ export class BackendSocket {
engine: s.engine,
hasMore: w.hasMore,
oldestCursor: w.oldestCursor,
+ ...truncated,
})
return
}
diff --git a/cli/src/core/transcripts/attach.spec.ts b/cli/src/core/transcripts/attach.spec.ts
index d8ad9f4c2..4ea8ebd3b 100644
--- a/cli/src/core/transcripts/attach.spec.ts
+++ b/cli/src/core/transcripts/attach.spec.ts
@@ -379,6 +379,21 @@ describe('attaching a session', () => {
expect(run.normalizers.turnStates.has('terminal-s')).toBe(true)
expect(run.deps.questionWatcher.start).not.toHaveBeenCalled()
})
+
+ it('folds only the newest part of a transcript over the cap, and says so', async () => {
+ vi.spyOn(console, 'log').mockImplementation(() => {})
+ const warn = vi.spyOn(console, 'warn').mockImplementation(() => {})
+ const records = Array.from({ length: 20 }, (_, i) => JSON.stringify({ n: i, pad: 'x'.repeat(40) }))
+ const capBytes = records.slice(-5).reduce((sum, record) => sum + Buffer.byteLength(record), 0)
+ const run = setup({ wholeReadCapBytes: capBytes })
+ await run.attach.attachSession(session('terminal', transcript(records)))
+ expect(warn).toHaveBeenCalledWith('[agent] terminal transcript over 0 MB · folded from its newest 0 MB')
+ // The profile was hydrated from what was read: the newest records that fit, in order.
+ expect(run.deps.runtimeProfiles.hydrate).toHaveBeenCalledWith(expect.objectContaining({ agentId: 'terminal-agent' }), records.slice(-5))
+ warn.mockClear()
+ await run.attach.attachSession(session('terminal', transcript(records.slice(0, 3)), { agentId: 'small', sessionId: 'small-s' }))
+ expect(warn).not.toHaveBeenCalled()
+ })
})
it('says when an attach is slow', async () => {
diff --git a/cli/src/core/transcripts/attach.ts b/cli/src/core/transcripts/attach.ts
index 34e0e4777..a51ee1a55 100644
--- a/cli/src/core/transcripts/attach.ts
+++ b/cli/src/core/transcripts/attach.ts
@@ -32,7 +32,7 @@ import { sid } from '../../lib/log.js'
import { foldTranscript, lineToEvents, newTurnState, TranscriptFold, type LiveEvent } from '../../lib/normalize.js'
import type { RegisteredSession } from '../../lib/registry.js'
import type { RuntimeField, RuntimeProfileManager } from '../../lib/runtimeProfile.js'
-import { tailFile } from '../../lib/transcriptTail.js'
+import { tailFileCapped, WHOLE_READ_CAP_BYTES } from '../../lib/transcriptTail.js'
import type { TailHold, Watcher } from '../../watcher/watcher.js'
import type { SessionNormalizers } from './normalizers.js'
import type { RelaunchMarks } from './relaunch.js'
@@ -60,11 +60,14 @@ export interface AttachDeps {
concurrency: number
/** Where an engine's writing became live again after a relaunch (core/transcripts/relaunch.ts): the fold stops there. */
relaunchMarks?: Pick<RelaunchMarks, 'take'>
+ /** The most of a transcript an engine without its own reader from the end folds, from its end. */
+ wholeReadCapBytes?: number
}
export function createAttach({
validateTerminal, normalizers, watcher, cursorDiscovery, device, runtimeProfiles, captureTerminal, emit,
announceTurnAborted, questionWatcher, terminalLabel, dbs, devinHome, hermesDb, concurrency, relaunchMarks,
+ wholeReadCapBytes = WHOLE_READ_CAP_BYTES,
}: AttachDeps) {
const {
turnStates, codexNormalizers, cursorNormalizers, opencodeReaders, kiloReaders, museNormalizers, ampNormalizers,
@@ -209,7 +212,15 @@ export function createAttach({
historyTurnOpen = take(stream.finish())
console.log(`[agent] ${sid(session.agentId)} read the transcript from its end · turn @${read.turnFrom} · profile @${read.profileFrom} · ${read.end} bytes${handover.hold ? ' · took over its tail' : ''}`)
}
- const lines = session.transcriptPath && !fromEnd ? await tailFile(session.transcriptPath, Infinity) : []
+ // An engine without a reader from the end folds its transcript from the start, bounded: one huge
+ // transcript read whole would take every agent's daemon down (the October 3 crash, on Codex). Its
+ // oldest history past the cap is not folded; its last turns, which say whether it is working, are.
+ let lines: string[] = []
+ if (session.transcriptPath && !fromEnd) {
+ const read = await tailFileCapped(session.transcriptPath, wholeReadCapBytes)
+ if (read.truncated) console.warn(`[agent] ${sid(session.agentId)} transcript over ${Math.round(wholeReadCapBytes / 1024 / 1024)} MB · folded from its newest ${Math.round(wholeReadCapBytes / 1024 / 1024)} MB`)
+ lines = read.lines
+ }
if (!fromEnd) {
if (observe) for (const line of lines) observe(line)
runtimeProfiles.hydrate(session, lines)
diff --git a/cli/src/core/transcripts/lastTurn.spec.ts b/cli/src/core/transcripts/lastTurn.spec.ts
index 6804ca01d..d8def278e 100644
--- a/cli/src/core/transcripts/lastTurn.spec.ts
+++ b/cli/src/core/transcripts/lastTurn.spec.ts
@@ -14,7 +14,7 @@ vi.mock('../../engines/devin/reader.js', () => ({ readDevinMessages: vi.fn(async
vi.mock('../../engines/devin/normalizer.js', () => ({ lastDevinTurnText: vi.fn((rows: string[]) => ({ text: rows[0] })) }))
vi.mock('../../engines/codex/lastTurn.js', () => ({ readLastCodexTurnText: vi.fn(async (path: string) => ({ text: `codex ${path}` })) }))
vi.mock('../../lib/transcriptTail.js', () => ({
- tailFile: vi.fn(async (path: string, n: number) => [`${path} last ${n}`]),
+ tailFileCapped: vi.fn(async (path: string) => ({ lines: [`${path} capped`], truncated: false })),
tailFileUntil: vi.fn(async (path: string) => [`${path} back to the last turn`]),
}))
vi.mock('../../lib/normalize.js', () => ({
@@ -67,9 +67,9 @@ describe('the last turn of each engine', () => {
expect(await read('claude-s')).toEqual(text('raw: /t/claude.jsonl back to the last turn'))
expect(await read('codex-s')).toEqual(text('codex /t/codex.jsonl'))
for (const engine of engines) {
- expect(await read(`${engine}-s`), engine).toEqual(text(`${engine}: /t/${engine}.jsonl last Infinity`))
+ expect(await read(`${engine}-s`), engine).toEqual(text(`${engine}: /t/${engine}.jsonl capped`))
}
// An engine with no reader of its own: its raw lines.
- expect(await read('terminal-s')).toEqual(text('raw: /t/shell.log last Infinity'))
+ expect(await read('terminal-s')).toEqual(text('raw: /t/shell.log capped'))
})
})
diff --git a/cli/src/core/transcripts/lastTurn.ts b/cli/src/core/transcripts/lastTurn.ts
index d70ec1b36..2c6182b5b 100644
--- a/cli/src/core/transcripts/lastTurn.ts
+++ b/cli/src/core/transcripts/lastTurn.ts
@@ -26,7 +26,7 @@ import { readOpencodeMessages } from '../../engines/opencode/reader.js'
import { lastPiTurnText } from '../../engines/pi/normalizer.js'
import { lastTurnTextFromRawLines, selectClaudeRecapLine, type LastTurnText } from '../../lib/normalize.js'
import type { RegisteredSession } from '../../lib/registry.js'
-import { tailFile, tailFileUntil } from '../../lib/transcriptTail.js'
+import { tailFileCapped, tailFileUntil } from '../../lib/transcriptTail.js'
export interface LastTurnDeps {
bySession: (sessionId: string) => RegisteredSession | undefined
@@ -48,7 +48,10 @@ export function createLastTurnReader({ bySession, dbs, hermesDb }: LastTurnDeps)
if (s.engine === 'codex') return readLastCodexTurnText(s.transcriptPath)
// Its last turn, read backward — not the whole conversation once per turn end.
if (s.engine === 'claude') return lastTurnTextFromRawLines(await tailFileUntil(s.transcriptPath, selectClaudeRecapLine))
- const lines = await tailFile(s.transcriptPath, Infinity)
+ // Bounded from the end like every whole read of an engine without pages (lib/transcriptTail.ts):
+ // the last turn is at the end, and a transcript past the cap would otherwise be read whole at
+ // every turn's end.
+ const { lines } = await tailFileCapped(s.transcriptPath)
if (s.engine === 'cursor') return lastCursorTurnText(lines)
if (s.engine === 'muse') return lastMuseTurnText(lines)
if (s.engine === 'amp') return lastAmpTurnText(lines)
diff --git a/cli/src/lib/transcriptTail.spec.ts b/cli/src/lib/transcriptTail.spec.ts
index 3d754e45f..94469312e 100644
--- a/cli/src/lib/transcriptTail.spec.ts
+++ b/cli/src/lib/transcriptTail.spec.ts
@@ -2,7 +2,7 @@ import { afterEach, beforeEach, describe, expect, it } from 'vitest'
import { appendFileSync, mkdtempSync, rmSync, truncateSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
-import { scanRecordsBackward, streamRecords, tailFile, tailFileUntil } from './transcriptTail.js'
+import { scanRecordsBackward, streamRecords, tailFile, tailFileCapped, tailFileUntil, WHOLE_READ_CAP_BYTES } from './transcriptTail.js'
describe('backward transcript suffix', () => {
let directory: string, file: string
@@ -192,3 +192,31 @@ describe('tailFile', () => {
expect(await tailFile(file, 1.9)).toEqual(['b'])
})
})
+
+describe('tailFileCapped', () => {
+ let directory: string, file: string
+ beforeEach(() => { directory = mkdtempSync(join(tmpdir(), 'tail-capped-')); file = join(directory, 't.jsonl') })
+ afterEach(() => rmSync(directory, { recursive: true, force: true }))
+
+ it('reads a file under the cap whole, as tailFile does, with the same line rules', async () => {
+ writeFileSync(file, '{"a":1}\r\n\n{"b":"é"}\r{"c":3}\n \n{"d":4}')
+ expect(await tailFileCapped(file)).toEqual({ lines: await tailFile(file, Infinity), truncated: false })
+ expect((await tailFileCapped(file)).lines).toEqual(['{"a":1}', '{"b":"é"}', '{"c":3}', '{"d":4}'])
+ expect(WHOLE_READ_CAP_BYTES).toBe(64 * 1024 * 1024)
+ })
+
+ it('keeps only the newest records that fit, in order, and says older ones were left', async () => {
+ const records = Array.from({ length: 10 }, (_, i) => JSON.stringify({ n: i, pad: 'x'.repeat(90) }))
+ writeFileSync(file, records.join('\n') + '\n')
+ const each = Buffer.byteLength(records[0])
+ expect(await tailFileCapped(file, each * 3)).toEqual({ lines: records.slice(-3), truncated: true })
+ expect(await tailFileCapped(file, each * 3 + 1)).toEqual({ lines: records.slice(-3), truncated: true })
+ expect(await tailFileCapped(file, each * 10)).toEqual({ lines: records, truncated: false })
+ // A newest record bigger than the cap: nothing that fits is newer, so nothing is read.
+ expect(await tailFileCapped(file, each - 1)).toEqual({ lines: [], truncated: true })
+ })
+
+ it('reads nothing of a file that is not there', async () => {
+ expect(await tailFileCapped(join(directory, 'gone.jsonl'))).toEqual({ lines: [], truncated: false })
+ })
+})
diff --git a/cli/src/lib/transcriptTail.ts b/cli/src/lib/transcriptTail.ts
index f4117d33a..919b61830 100644
--- a/cli/src/lib/transcriptTail.ts
+++ b/cli/src/lib/transcriptTail.ts
@@ -181,6 +181,35 @@ export async function tailFileUntil(filePath: string, select: (line: string) =>
}
}
+/** The most of a transcript read at once by a reader that has no pages of its own: its newest records
+ * up to this many bytes. The October 3 crash was an 803 MB transcript read whole: past what the heap
+ * holds, one conversation's read takes every agent's daemon down with it. Claude Code and Codex read
+ * pages instead (lib/transcriptPages.ts); this is the floor under every other engine until they do. */
+export const WHOLE_READ_CAP_BYTES = 64 * 1024 * 1024
+
+/**
+ * A transcript's newest records, up to `maxBytes` of them, in order: what `tailFile(path, Infinity)`
+ * returns for a file under the cap, with `truncated` saying older records were left unread. A record
+ * that alone is bigger than what is left of the cap ends the read there. Empty for a file that cannot
+ * be read.
+ */
+export async function tailFileCapped(filePath: string, maxBytes = WHOLE_READ_CAP_BYTES): Promise<{ lines: string[]; truncated: boolean }> {
+ try {
+ const { size } = await stat(filePath)
+ const lines: string[] = []
+ let bytes = 0
+ let truncated = false
+ await scanRecordsBackward(filePath, size, (record) => {
+ if (bytes + record.length > maxBytes) { truncated = true; return true }
+ bytes += record.length
+ lines.push(record.toString('utf8'))
+ })
+ return { lines: lines.reverse(), truncated }
+ } catch {
+ return { lines: [], truncated: false }
+ }
+}
+
/** Return the last `n` non-empty raw lines of a specific transcript file. Prefer this over
* `tailLines` when the caller already holds a trusted, registered `transcriptPath` — it takes no
* request-controlled id, so there is no path to traverse. */
diff --git a/docs/design/2026-10-03-harnessd.md b/docs/design/2026-10-03-harnessd.md
index a8d10dc67..1eccb344a 100644
--- a/docs/design/2026-10-03-harnessd.md
+++ b/docs/design/2026-10-03-harnessd.md
@@ -195,6 +195,42 @@ input, answers and agent actions out. Firmware push, the dial↔window bridge an
shared utilities drivers use. Adding a device type adds a driver and its manifest capabilities;
the core does not change.
+**From today's code to there** (mapped on 2026-10-04). The device code is about 7,200 lines:
+`cable/` 5,800, `device/` 1,200 and `devices/` 100, beside about 6,000 lines of tests. Another 400
+lines wire it inside `runForeground`. Every call the core makes into it is guarded already
+(`outsideConsumers('devices')`), so an exception there costs one call. A native crash in the serial
+port code, or a leak, would still cost the whole core. Three couplings decide the order:
+
+1. **The dial's host also routes turns between machines.** ⌘K and the window's voice route send a
+ turn through `DaemonCableHost.sendTurn`. It knows a local agent from a remote one by the agent
+ lists it learned from the fleet (`DeviceFleet` over `DeviceLink`, the lane to the owner's other
+ machines). With the dial off, ⌘K would answer "no agent list yet". So the router, and the fleet
+ and machine list it reads, leave the dial's host first, as a fleet service of their own that ⌘K
+ and the dial both use.
+2. **The Wi-Fi device rides the relay's encrypted sessions.** `AutonomousDeviceService` and
+ `AutonomousDeviceDirect` pair, attach and receive through `BackendSocket`, so they move with the
+ relay, after it has a boundary of its own.
+3. **The window bridges are the desk's, not a driver's.** Selection, visit, form and the voice route
+ are shared by the dial, the Wi-Fi device and ⌘K. They stay in the core, and drivers reach them
+ through `CoreApi`.
+
+The steps, each a pull request with its end-to-end proof:
+
+- **D0, the fleet on its own.** The machine list, peers, `DeviceLink`, `DeviceFleet` and the turn
+ router across machines become `services/fleet.ts`. ⌘K, the voice route and the dial all send
+ turns through it.
+- **D1, the dial behind the boundary, in process.** `services/dial.ts` owns the serial ports, the
+ dial's host, its log and the `harness_devices_*` requests. The core reaches it through a `DialPort`, each member with its fallback: the desk, the
+ swarms, what is unread, focus, what was seen, the turn and question events teed to it, settings,
+ status and stop. The dial reaches the core through `CoreApi`: activity text, recent turns, sending
+ and stopping a turn, answering, the runtime profile and models, fork, the window frames (`dial_*`)
+ and the window bridges.
+- **D2, the dial in its own process.** The serial ports belong to `harness __service dial`; the
+ same handlers answer. A crash in the native serial code then costs the dial alone, and the master
+ starts it again. It is proven end to end with a fake dial on a pseudo-terminal, speaking the cable
+ protocol from `cable/vectors/`.
+- **D3, the Wi-Fi device**, with the relay.
+
### Models
Grid is how Harness runs local AI models with Codex, Claude Code and the other engines, from the
@@ -446,6 +482,17 @@ Done so far:
`dsh_remove` (`services/store.ts`, a service with no port). Five `BackendSocket` slots went with
them.
- Adding a request to a feature now touches only that feature's service.
+- **A floor under every engine's whole reads** (lib/transcriptTail.ts `tailFileCapped`). Claude Code
+ and Codex read pages. Every other engine still read its transcript whole, in three places that run
+ for everyone: attach (at every daemon start), the last-turn read (at every turn's end) and
+ `session_get`. Each now reads at most the newest 64 MB, from the end. One huge transcript of any
+ engine could otherwise run the core out of memory at start, again at every restart, and take every
+ agent's daemon down with it: the October 3 crash, on another engine.
+ - Past the cap, attach folds the newest part and says so in the log. The last turn is read from
+ the newest part. `session_get` shows the newest history, and its reply carries `truncated: true`.
+ - Known limit: Cursor's watcher still re-reads Cursor's transcript whole on each change. Cursor
+ rewrites it in place, and the watcher finds what is new by comparing whole files, so a cap there
+ would replay everything. It needs pages of its own, as in the rollout's step 2.
- **The walls, written down and checked.** The rules live in `AGENTS.md` files that coding agents read:
`cli/` and one per layer (`src/core/`, `src/services/`, `src/harnessd/`). A `CLAUDE.md` beside each
imports it, so Claude Code reads the same rules as Codex. On its own, Claude Code reads `AGENTS.md`
@@ -657,6 +704,36 @@ Found while mapping, and fixed on their own rather than inside a move:
that release does, by spawning the new `__run` and exiting. The new core comes up on its own,
without a master, with both agents and their conversations, and they take turns. The next start
brings up the master over the same agents. Passed from v0.3.57, three runs of three.
+ - What the apps see, compared with the released build (`e2e/compat.e2e.ts`, run with
+ `COMPAT_FROM` set to a released bundle). A daemon of the released build and one of this
+ checkout's bundle run the same scenario, and every answer is compared once ids, times, paths and
+ counters are made comparable. The scenario covers:
+ - a Claude Code and a Codex agent created, with a turn each;
+ - a question and a permission, answered;
+ - search;
+ - every read the apps make, well formed and malformed;
+ - rename, cancel, fork, restart, stop, resume and close.
+
+ A field this build adds is allowed, because the apps ignore what they do not read. Anything else is
+ explained or a regression. From v0.3.57 nothing differs, across 90 answers and the shape of
+ every frame a turn, a question and a permission push. The only addition is `ready` on
+ `session_search`'s reply. Two differences turned out to be timing on both builds, and the test now
+ waits them out:
+ - where an agent worked is read at most every 15 s;
+ - how far a launch has got when `agent_create` replies is a race with the engine starting.
+
+ One finding: the switch's `terminal_info` case can never run, because every `terminal_*` frame
+ goes to the terminal streams first. Neither build answers it.
+ - The sixteenth round (`e2e/ends.e2e.ts`), how an agent's life ends, for both engines:
+ - closed now, then opened again with its conversation;
+ - closed only when idle, which leaves a working agent working;
+ - closed after its task, which finishes the turn and then closes;
+ - closed after its task and cancelled, which leaves it open;
+ - closed after its task across a daemon restart;
+ - closed against an identity that changed, which changes nothing;
+ - purged, after which search no longer finds it and its transcripts are gone.
+
+ No defect was found.
## Testing