Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

Every service in its own process by default, each loading only its own code (#829)

Dee8851e7a3af2f
diff --git a/cli/build-bundle.mjs b/cli/build-bundle.mjs
index 783fa7a57..7c507d9b5 100644
--- a/cli/build-bundle.mjs
+++ b/cli/build-bundle.mjs
@@ -11,11 +11,13 @@
* `require` via the banner so ws's try/catch fallback works without them.
*/
import * as esbuild from 'esbuild'
-import { readFileSync, copyFileSync, rmSync } from 'fs'
+import { appendFileSync, readFileSync, copyFileSync, rmSync } from 'fs'
import { readDshRegistry } from './scripts/lib/dshRegistry.mjs'
import { readBuiltinBundle, readHarnessMonitorBundle, readModelManagerBundle } from './scripts/lib/modelManagerBundle.mjs'
import { readProcessImageBundle } from './scripts/lib/processImageBundle.mjs'
+import { leanBlock } from './scripts/lib/leanBlock.mjs'
import { fileURLToPath } from 'node:url'
+import { basename } from 'node:path'
const modelManagerBundle = JSON.stringify(readModelManagerBundle(fileURLToPath(new URL('../store/agents/autonomous-grid', import.meta.url))))
const devicesBundle = JSON.stringify(readBuiltinBundle(fileURLToPath(new URL('../store/agents/devices', import.meta.url)), ['harness.json', 'AGENTS.md', 'LICENSE', 'template']))
const harnessMonitorBundle = JSON.stringify(readHarnessMonitorBundle(fileURLToPath(new URL('../store/agents/harness-monitor', import.meta.url))))
@@ -37,9 +39,7 @@ const outDir = process.env.BUNDLE_OUT_DIR || 'dist'
// Start clean so no stale per-file `dist/*.js` / sourcemaps leak into the release artifact.
rmSync(outDir, { recursive: true, force: true })
-await esbuild.build({
- entryPoints: ['src/cli.ts'],
- outfile: `${outDir}/cli.js`,
+const options = {
bundle: true,
platform: 'node',
format: 'esm',
@@ -65,7 +65,32 @@ await esbuild.build({
keepNames: true,
legalComments: 'eof',
logLevel: 'info',
+}
+
+// harnessd's master and its services, bundled on their own (src/leanEntry.ts): Node parses all of the
+// file a process starts on, and the whole CLI's cost each of them about 45 MiB at idle. Split, so that a
+// process parses only the files its own code is in: the master never the services', search never the
+// viewers'. Carried inside cli.js, at its end, as a comment Node only skims (scripts/lib/leanBlock.mjs).
+const lean = await esbuild.build({
+ ...options,
+ entryPoints: { harnessd: 'src/leanEntry.ts' },
+ outdir: 'lean',
+ splitting: true,
+ chunkNames: '[name]-[hash]',
+ outExtension: { '.js': '.mjs' },
+ write: false,
+ logLevel: 'warning',
+})
+const leanFiles = Object.fromEntries(lean.outputFiles.map((file) => [basename(file.path), file.contents]))
+
+await esbuild.build({
+ ...options,
+ // Still one file, which the self-updater downloads, verifies and swaps whole. Its entry decides what a
+ // process loads (src/entry.ts), and the master starts itself and the services from the lean bundle.
+ entryPoints: ['src/entry.ts'],
+ outfile: `${outDir}/cli.js`,
})
+appendFileSync(`${outDir}/cli.js`, leanBlock(leanFiles))
copyFileSync('hook/notify.mjs', `${outDir}/notify.mjs`)
diff --git a/cli/e2e/harness/release.ts b/cli/e2e/harness/release.ts
new file mode 100644
index 000000000..42630f814
--- /dev/null
+++ b/cli/e2e/harness/release.ts
@@ -0,0 +1,28 @@
+/**
+ * A release of this checkout at a made-up version, as the update and re-execution tests serve them: the
+ * bundle built once, with its version swapped. The version is baked in at build time, in cli.js and in
+ * the lean bundle cli.js carries for harnessd's master and its services (src/harnessd/leanBundle.ts), so
+ * it is swapped in both: a master reporting the build before is not the build it was handed.
+ */
+import { readLeanBundle } from '../../src/harnessd/leanBundle.js'
+
+const { LEAN_MARKER, leanBlock } = await import('../../scripts/lib/leanBlock.mjs' as string) as {
+ LEAN_MARKER: string
+ leanBlock: (files: Record<string, string | Uint8Array>) => string
+}
+
+/** [bundle] (cli.js's text) with every [from] made [to], in the lean bundle it carries too. */
+export function atVersion(bundle: string, from: string, to: string): string {
+ const lean = readLeanBundle(Buffer.from(bundle))
+ if (!lean) return bundle.replaceAll(from, to)
+ const head = bundle.slice(0, bundle.lastIndexOf(LEAN_MARKER)).replace(/\n$/, '')
+ const files = Object.fromEntries([...lean.files].map(([name, code]) => [name, code.toString('utf8').replaceAll(from, to)]))
+ return head.replaceAll(from, to) + leanBlock(files)
+}
+
+/** [bundle] carrying [files] as its lean bundle instead of its own: a lean bundle that misbehaves. */
+export function withLean(bundle: string, files: Record<string, string>): string {
+ const at = bundle.lastIndexOf(LEAN_MARKER)
+ const head = at < 0 ? bundle : bundle.slice(0, at).replace(/\n$/, '')
+ return head + leanBlock(files)
+}
diff --git a/cli/e2e/lean.e2e.ts b/cli/e2e/lean.e2e.ts
new file mode 100644
index 000000000..d270d7d2b
--- /dev/null
+++ b/cli/e2e/lean.e2e.ts
@@ -0,0 +1,174 @@
+/**
+ * harnessd's master and its services lean, on a real release bundle: Node parses all of the file a
+ * process starts on, and started on the whole 4.4 MB cli.js each paid about 45 MiB for that alone. So
+ * the master re-executes, same pid, on the lean bundle cli.js carries, and starts every service from it
+ * (src/harnessd/leanBundle.ts); the core runs from cli.js as always. Every service still runs in its own
+ * process and does its work, for Claude Code and Codex agents alike. A lean bundle that cannot start a
+ * master is never handed the daemon, and `HARNESSD_LEAN=off` runs everything from cli.js as before.
+ *
+ * The bundle is built from this checkout (the run's own with `E2E_BUNDLE=1`).
+ */
+import { execFileSync } from 'node:child_process'
+import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { dirname, join } from 'node:path'
+import { afterAll, afterEach, beforeAll, describe, expect, it, onTestFailed } from 'vitest'
+import { LocalClient, type Frame } from './harness/client.js'
+import { CLI_ROOT, IsolatedDaemon, until } from './harness/daemon.js'
+import { withLean } from './harness/release.js'
+import { readLeanBundle } from '../src/harnessd/leanBundle.js'
+
+type Engine = 'claude' | 'codex'
+const SERVICES = ['search', 'viewers', 'workspaces', 'teams']
+
+const commandOf = (pid: number): string => execFileSync('ps', ['-o', 'command=', '-p', String(pid)], { encoding: 'utf8' }).trim()
+const rssMiB = (pid: number): number => Number(execFileSync('ps', ['-o', 'rss=', '-p', String(pid)], { encoding: 'utf8' }).trim()) / 1024
+const servicePids = (d: IsolatedDaemon): Map<string, number> => new Map(SERVICES.map((name) => {
+ const started = [...d.log().matchAll(new RegExp(`\\[harnessd\\] service ${name} started \\(pid (\\d+)\\)`, 'g'))]
+ return [name, Number(started.at(-1)?.[1] ?? 0)]
+}))
+const isTurn = (type: string, agentId: string) => (frame: Frame) => frame.type === type && frame.agentId === agentId
+
+describe('harnessd\'s master and services lean', () => {
+ let scratch = ''
+ let bundle = ''
+ let daemon: IsolatedDaemon | undefined
+ beforeAll(() => {
+ scratch = mkdtempSync(join(tmpdir(), 'harnessd-lean-'))
+ bundle = process.env.E2E_BUNDLE_PATH ?? join(scratch, 'build', 'cli.js')
+ if (!process.env.E2E_BUNDLE_PATH) {
+ execFileSync(process.execPath, ['build-bundle.mjs'], { cwd: CLI_ROOT, env: { ...process.env, BUNDLE_OUT_DIR: join(scratch, 'build') }, stdio: 'pipe' })
+ }
+ }, 120_000)
+ afterAll(() => { if (scratch) rmSync(scratch, { recursive: true, force: true }) })
+ afterEach(async () => { await daemon?.close(); daemon = undefined })
+ const fresh = async (env: Record<string, string> = {}, scriptPath = bundle) => {
+ const d = await IsolatedDaemon.create({ scriptPath, env })
+ daemon = d
+ onTestFailed(() => { console.log(`---- daemon log\n${d.log().split('\n').slice(-120).join('\n')}`) })
+ await d.start()
+ await until('every service', () => SERVICES.every((name) => new RegExp(`service ${name} started`).test(d.log())) || null, 60_000, 200)
+ return d
+ }
+ async function agentWorks(d: IsolatedDaemon, engine: Engine): Promise<void> {
+ const client = await LocalClient.connect(d)
+ try {
+ const cwd = join(d.projectsDir, `lean-${engine}`)
+ mkdirSync(cwd, { recursive: true })
+ const created = await client.request('agent_create', { engine, cwd, bypassPermission: true }, 90_000)
+ expect(created.error, JSON.stringify(created)).toBeUndefined()
+ const agent = await until(`the ${engine} agent to bind`, async () => {
+ const rows = (await client.request<{ agents: Array<Record<string, any>> }>('agents_list', {}, 30_000)).agents
+ const row = rows.find((one) => one.id === created.agent.id)
+ return row?.sessionId && row.status === 'active' ? row : null
+ }, 60_000, 500)
+ const word = `lean${engine}${Date.now().toString(36)}`
+ const ended = client.next(isTurn('turn_ended', agent.id), 45_000, 'turn_ended')
+ client.send('message', { agentId: agent.id, content: `remember ${word}` })
+ await ended
+ // Search, in its own process from the lean bundle, indexes the turn and finds it.
+ await until('search to find the turn', async () =>
+ JSON.stringify(await client.request('session_search', { query: word }, 30_000)).includes(agent.sessionId) || null, 30_000, 500)
+ } finally { client.close() }
+ }
+
+ it('re-executes the master, same pid, on the lean bundle, and runs every service from it; the core from cli.js', async () => {
+ const d = await fresh()
+ // The master titles itself `harnessd`, so its log says what it runs on: the lean bundle, written
+ // into the data folder, which only a master running on it says it shares with the services.
+ const lean = /\[harnessd\] services run from (\S+), as this master does/.exec(d.log())?.[1]
+ expect(lean, 'the master re-executed on the lean bundle').toMatch(/[\\/]lean[\\/][0-9a-f]{16}[\\/]harnessd\.mjs$/)
+ expect(lean!.startsWith(join(d.dataDir, 'lean'))).toBe(true)
+ expect(existsSync(lean!)).toBe(true)
+ // The same process `harness start` started: re-executed in place.
+ const master = d.pid!
+ expect(commandOf(master)).toBe('harnessd')
+ expect(commandOf(d.corePid()!)).toContain(`${bundle} __run`)
+ for (const [name, pid] of servicePids(d)) expect(commandOf(pid), name).toBe(`harnessd ${name}`)
+ // Lean: under the cost of parsing the whole CLI, which every one of them paid before (at idle,
+ // 115 to 160 MiB each), and a long way under it at that (55 to 80).
+ for (const [name, pid] of [['master', master], ...servicePids(d)] as Array<[string, number]>) {
+ expect(rssMiB(pid), `${name} resident MiB`).toBeLessThan(100)
+ }
+ // A service's lines are stamped, as the core's and the master's are in the log they share.
+ for (const name of SERVICES) {
+ await until(`${name} to say it is connected`, () => new RegExp(`\\d{4}-\\d\\d-\\d\\d \\d\\d:\\d\\d:\\d\\d\\.\\d{3} \\[service ${name}\\] connected to the core`).test(d.log()) || null, 30_000, 200)
+ }
+ await agentWorks(d, 'claude')
+ await agentWorks(d, 'codex')
+ expect(d.coresStarted()).toBe(1)
+ expect(d.log()).not.toMatch(/service \w+ (exited|ended)/)
+ // The master claims the folder it runs from while it lives, and gives the claim up as it stops.
+ const claim = join(dirname(lean!), `.claim-${master}`)
+ expect(existsSync(claim)).toBe(true)
+ await d.stop()
+ expect(existsSync(claim)).toBe(false)
+ })
+
+ it('starts a service from cli.js once the lean bundle is gone from under its master, and the service works', async () => {
+ // A master lives for weeks; its lean folder, gone (someone tidying up), failed every restart of every
+ // service with MODULE_NOT_FOUND until the master itself restarted. The lean bundle is only ever an
+ // optimisation.
+ const d = await fresh()
+ const ours = /\[harnessd\] services run from (\S+), as this master does/.exec(d.log())![1]
+ rmSync(dirname(ours), { recursive: true })
+ const search = servicePids(d).get('search')!
+ const restarts = (): number => [...d.log().matchAll(/\[harnessd\] service search started/g)].length
+ const before = restarts()
+ process.kill(search, 'SIGKILL')
+ await until('search started again', () => restarts() > before || null, 60_000, 200)
+ // From cli.js, as the core: its path as the master resolved it.
+ expect(d.log()).toMatch(new RegExp(`\\[harnessd\\] the lean bundle \\S+ cannot be used \\(it is gone\\): services start from \\S+cli\\.js`))
+ expect(d.log()).toContain(`the lean bundle ${ours} cannot be used`)
+ await agentWorks(d, 'codex')
+ expect(d.log()).not.toMatch(/MODULE_NOT_FOUND|Cannot find module/)
+ expect(d.coresStarted()).toBe(1)
+ })
+
+ it('keeps a master\'s lean bundle while it lives: another build\'s master on the same data folder leaves it, and its services still restart', async () => {
+ // Two builds on one computer can share a data folder, and a master restarts its services from its
+ // lean bundle for as long as it lives. The other master writes its own bundle and clears the folder
+ // of the ones no live master claims, then leaves: a daemon is already serving.
+ const d = await fresh()
+ const ours = /\[harnessd\] services run from (\S+), as this master does/.exec(d.log())![1]
+ const lean = readLeanBundle(readFileSync(bundle))!
+ const files = Object.fromEntries([...lean.files].map(([name, code]) => [name, code.toString('utf8')]))
+ files['harnessd.mjs'] += '\n// another build\n'
+ const other = join(scratch, 'other', 'cli.js')
+ mkdirSync(join(scratch, 'other'), { recursive: true })
+ writeFileSync(other, withLean(readFileSync(bundle, 'utf8'), files), { mode: 0o755 })
+ const second = await IsolatedDaemon.create({ beside: d, dataDir: d.dataDir, port: d.port, scriptPath: other })
+ try {
+ await second.start({ ready: 'none' })
+ await until('the other master to leave', () => second.child === null || null, 90_000, 250)
+ expect(second.log()).toMatch(/\[harnessd\] services run from \S+, as this master does/)
+ expect(existsSync(ours), 'the running master\'s lean bundle').toBe(true)
+ } finally { await second.close() }
+ // Its services come back from it: search, killed outright, is started again and finds a turn.
+ const [, search] = [...servicePids(d)].find(([name]) => name === 'search')!
+ const restarts = (): number => [...d.log().matchAll(/\[harnessd\] service search started/g)].length
+ const before = restarts()
+ process.kill(search, 'SIGKILL')
+ await until('search started again', () => restarts() > before || null, 60_000, 200)
+ await agentWorks(d, 'claude')
+ expect(d.coresStarted()).toBe(1)
+ })
+
+ it('runs everything from cli.js, as before, with HARNESSD_LEAN=off', async () => {
+ const d = await fresh({ HARNESSD_LEAN: 'off' })
+ expect(existsSync(join(d.dataDir, 'lean'))).toBe(false)
+ expect(d.log()).not.toMatch(/\[harnessd\] services run from/)
+ await agentWorks(d, 'codex')
+ })
+
+ it('never hands the daemon a lean bundle that cannot start a master: everything runs from cli.js', async () => {
+ const broken = join(scratch, 'broken', 'cli.js')
+ mkdirSync(join(scratch, 'broken'), { recursive: true })
+ writeFileSync(broken, withLean(readFileSync(bundle, 'utf8'), { 'harnessd.mjs': 'process.exit(5)\n' }), { mode: 0o755 })
+ const d = await fresh({}, broken)
+ expect(d.log()).toMatch(/\[harnessd\] the lean bundle \S+ did not answer its probe \(exit 5\): the master and the services run from /)
+ expect(d.log()).not.toMatch(/\[harnessd\] services run from/)
+ await agentWorks(d, 'claude')
+ expect(d.coresStarted()).toBe(1)
+ })
+})
diff --git a/cli/e2e/reexec.e2e.ts b/cli/e2e/reexec.e2e.ts
index 13edb5bf7..f55c9bc2f 100644
--- a/cli/e2e/reexec.e2e.ts
+++ b/cli/e2e/reexec.e2e.ts
@@ -21,6 +21,7 @@ import { dirname, join } from 'node:path'
import { afterAll, beforeAll, describe, expect, it, onTestFailed } from 'vitest'
import { LocalClient, type Frame } from './harness/client.js'
import { CLI_ROOT, IsolatedDaemon, until } from './harness/daemon.js'
+import { atVersion } from './harness/release.js'
const FIRST = '42.0.1'
const sha = (bytes: Buffer): string => createHash('sha256').update(bytes).digest('hex')
@@ -96,7 +97,7 @@ describe('the master re-executing itself on an update', () => {
const release = (version: string, inject = ''): void => {
// The version is baked in at build time; the others are the same bytes with it swapped, and a
// fault put in at the top where a test needs one.
- const source = first.replaceAll(FIRST, version).replace('\n', `\n${inject}\n`)
+ const source = atVersion(first, FIRST, version).replace('\n', `\n${inject}\n`)
const file = join(out, `cli-${version}.js`)
writeFileSync(file, source)
expect(execFileSync(process.execPath, [file, 'version'], { encoding: 'utf8' }).trim()).toBe(version)
diff --git a/cli/e2e/services.e2e.ts b/cli/e2e/services.e2e.ts
index 8599571a4..b5469258c 100644
--- a/cli/e2e/services.e2e.ts
+++ b/cli/e2e/services.e2e.ts
@@ -1,7 +1,10 @@
/**
- * A failing service never takes the core down (core/serviceHost.ts), proven on the real daemon: every
- * service made to fail as it starts, and services made to fail on every call, while a client starts an
- * agent, the agent binds, messages become turns that start and end, and the daemon restarts.
+ * A failing service never takes the core down, proven on the real daemon: every service made to fail as
+ * it starts, and services made to fail on every call, while a client starts an agent, the agent binds,
+ * messages become turns that start and end, and the daemon restarts. Both ways a service runs: in its
+ * own process (the default for search, the viewers, workspaces and the teams, harnessd/services.ts),
+ * and inside the core's process (`HARNESSD_SERVICES=none`, and always for models, the Store and the
+ * fleet), where the core's host guards it (core/serviceHost.ts).
*/
import { randomUUID } from 'node:crypto'
import { mkdirSync } from 'node:fs'
@@ -43,12 +46,77 @@ async function turn(client: LocalClient, agentId: string, content: string): Prom
await ended
}
+/** Every service inside the core's process, where the core's host guards it. */
+const IN_THE_CORE = { HARNESSD_SERVICES: 'none' }
+/** A master quick to give up on a service that keeps failing as it starts. */
+const QUICK_TO_PARK = { HARNESSD_SERVICE_PARK_CRASHES: '3', HARNESSD_SERVICE_INITIAL_BACKOFF_MS: '200', HARNESSD_SERVICE_MAX_BACKOFF_MS: '1000' }
+const PROCESS_SERVICES = ['search', 'viewers', 'workspaces', 'teams'] as const
+
describe('a failing service never takes the core down', () => {
let daemon: IsolatedDaemon | undefined
afterEach(async () => { await daemon?.close(); daemon = undefined })
- it('with every service failing to start, the core starts, runs an agent through turns and a restart, and says each service is off', async () => {
- daemon = await IsolatedDaemon.create({ env: { HARNESSD_TEST_FAULTS: 'search,viewers,models,workspaces,store' } })
+ it('with every service failing to start, in its own process or in the core, the core starts, runs an agent through turns and a restart, and says each one is unavailable', async () => {
+ daemon = await IsolatedDaemon.create({ env: { HARNESSD_TEST_FAULTS: 'search,viewers,workspaces,teams,models,store', ...QUICK_TO_PARK } })
+ const d = daemon
+ onTestFailed(() => { console.log(`---- daemon log\n${d.log().split('\n').slice(-120).join('\n')}`) })
+ await d.start()
+ // Each process fails as it starts, again and again, until the master parks it; the core never waits for one.
+ for (const service of PROCESS_SERVICES) {
+ await until(`the master to park ${service}`, () => d.log().includes(`[harnessd] service ${service} ended 3 times`) || null, 60_000, 250)
+ }
+ for (const service of ['models', 'store']) {
+ expect(d.log()).toContain(`[services] ${service} did not start · injected fault: ${service} · the core runs without it`)
+ }
+ const client = await LocalClient.connect(d)
+ const agentId = await boundAgent(d, client, 'no-service-processes')
+ await turn(client, agentId, 'first, with every service down')
+ expect(await client.request('session_search', { query: 'first' })).toMatchObject({ error: 'SERVICE_UNAVAILABLE', service: 'search' })
+ expect(await client.request('dsh_list', {})).toMatchObject({ error: 'SERVICE_UNAVAILABLE', service: 'store' })
+ expect(d.coresStarted()).toBe(1)
+
+ await d.restart()
+ const again = await LocalClient.connect(d)
+ await until('the agent to be back after a restart', async () => {
+ const agent = await row(again, agentId)
+ return agent?.sessionId && agent.status !== 'stopped' ? agent : null
+ }, 45_000, 250)
+ await turn(again, agentId, 'second, after a restart')
+ again.close()
+ client.close()
+ })
+
+ it('with services in their own processes failing on every request and event, each request is answered failed, every turn reaches the client, and nothing restarts', async () => {
+ daemon = await IsolatedDaemon.create({ env: { HARNESSD_TEST_FAULTS: 'search.touch,search.session_search,workspaces.nameBranches,workspaces.sweep' } })
+ const d = daemon
+ onTestFailed(() => { console.log(`---- daemon log\n${d.log().split('\n').slice(-120).join('\n')}`) })
+ await d.start()
+ const client = await LocalClient.connect(d)
+ const agentId = await boundAgent(d, client, 'failing-processes')
+ for (let i = 1; i <= 4; i++) await turn(client, agentId, `turn ${i}`)
+ // (The viewers hear only of agents that have a viewer: e2e/viewersProcess.e2e.ts runs one.)
+ // In its own process a failing service costs only what failed: nothing is switched off, and each
+ // request is answered for itself, as often as it is asked.
+ for (let i = 0; i < 6; i++) {
+ expect(await client.request('session_search', { query: `ask ${i}` }), `ask ${i}`).toMatchObject({ error: 'SERVICE_FAILED', service: 'search' })
+ }
+ for (const line of [
+ '[service search] touch failed · injected fault: search.touch',
+ '[service search] session_search failed · injected fault: search.session_search',
+ '[service workspaces] nameBranches failed · injected fault: workspaces.nameBranches',
+ ]) await until(line, () => d.log().includes(line) || null, 15_000, 250)
+ const agent = await row(client, agentId)
+ expect(agent?.sessionId).toBeTruthy()
+ expect(agent?.status).not.toBe('stopped')
+ await turn(client, agentId, 'after six failed searches')
+ expect(d.log()).not.toMatch(/\[harnessd\] service \w+ started .* restart \d+/)
+ expect(d.log()).not.toContain('switched off')
+ expect(d.coresStarted()).toBe(1)
+ client.close()
+ })
+
+ it('with every service in the core failing to start, the core starts, runs an agent through turns and a restart, and says each service is off', async () => {
+ daemon = await IsolatedDaemon.create({ env: { HARNESSD_TEST_FAULTS: 'search,viewers,models,workspaces,store', ...IN_THE_CORE } })
onTestFailed(() => { console.log(`---- daemon log\n${daemon?.log().split('\n').slice(-80).join('\n')}`) })
await daemon.start()
for (const service of ['search', 'viewers', 'models', 'workspaces', 'store']) {
@@ -80,9 +148,9 @@ describe('a failing service never takes the core down', () => {
client.close()
})
- it('with services failing on every call, they are switched off and every turn still reaches the client', async () => {
+ it('with services in the core failing on every call, they are switched off and every turn still reaches the client', async () => {
daemon = await IsolatedDaemon.create({
- env: { HARNESSD_TEST_FAULTS: 'search.touch,search.session_search,viewers.frameContext,viewers.attach,workspaces.nameBranches' },
+ env: { HARNESSD_TEST_FAULTS: 'search.touch,search.session_search,viewers.frameContext,viewers.attach,workspaces.nameBranches', ...IN_THE_CORE },
})
onTestFailed(() => { console.log(`---- daemon log\n${daemon?.log().split('\n').slice(-120).join('\n')}`) })
await daemon.start()
@@ -109,9 +177,9 @@ describe('a failing service never takes the core down', () => {
client.close()
})
- it('with the teams failing on every call, or not starting at all, messages are written and every turn reaches the client', async () => {
+ it('with the teams in the core failing on every call, or not starting at all, messages are written and every turn reaches the client', async () => {
for (const faults of ['teams.prepare,teams.started,teams.raw,teams.forget', 'teams']) {
- daemon = await IsolatedDaemon.create({ env: { HARNESSD_TEST_FAULTS: faults } })
+ daemon = await IsolatedDaemon.create({ env: { HARNESSD_TEST_FAULTS: faults, ...IN_THE_CORE } })
const d = daemon
onTestFailed(() => { console.log(`---- daemon log (${faults})\n${d.log().split('\n').slice(-80).join('\n')}`) })
await d.start()
@@ -126,8 +194,8 @@ describe('a failing service never takes the core down', () => {
}
})
- it('a search request that fails before search is switched off answers that request, and the next one too', async () => {
- daemon = await IsolatedDaemon.create({ env: { HARNESSD_TEST_FAULTS: 'search.session_search' } })
+ it('a search request that fails in the core before search is switched off answers that request, and the next one too', async () => {
+ daemon = await IsolatedDaemon.create({ env: { HARNESSD_TEST_FAULTS: 'search.session_search', ...IN_THE_CORE } })
onTestFailed(() => { console.log(`---- daemon log\n${daemon?.log().split('\n').slice(-80).join('\n')}`) })
await daemon.start()
const client = await LocalClient.connect(daemon)
diff --git a/cli/e2e/teamsProcess.e2e.ts b/cli/e2e/teamsProcess.e2e.ts
index 5d892c4bc..88a9e2c38 100644
--- a/cli/e2e/teamsProcess.e2e.ts
+++ b/cli/e2e/teamsProcess.e2e.ts
@@ -70,8 +70,8 @@ describe('the teams\' prompt scopes in their own process', () => {
return d
}
- it('by default the scopes stay in the core\'s process, as before', async () => {
- const d = await fresh({ HARNESSD_SERVICES: '' }, false)
+ it('with HARNESSD_SERVICES=none the scopes stay in the core\'s process, as before', async () => {
+ const d = await fresh({ HARNESSD_SERVICES: 'none' }, false)
const client = await LocalClient.connect(d)
const agent = await create(d, client, 'scopes-default')
await say(client, agent.id, 'hello from the alpha tab', 'tab-alpha')
diff --git a/cli/e2e/update.e2e.ts b/cli/e2e/update.e2e.ts
index 83c5552d0..e40777d66 100644
--- a/cli/e2e/update.e2e.ts
+++ b/cli/e2e/update.e2e.ts
@@ -15,6 +15,7 @@ import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { afterAll, beforeAll, describe, expect, it, onTestFailed } from 'vitest'
import { CLI_ROOT, IsolatedDaemon, until } from './harness/daemon.js'
+import { atVersion } from './harness/release.js'
const FIRST = '41.0.1'
const sha = (bytes: Buffer): string => createHash('sha256').update(bytes).digest('hex')
@@ -49,7 +50,7 @@ describe('a self-update under harnessd', () => {
// The version is baked in at build time; the others are the same bytes with it swapped.
expect(count(first, FIRST)).toBeGreaterThanOrEqual(2)
const release = (version: string, crashesOnStart = false): void => {
- let source = first.replaceAll(FIRST, version)
+ let source = atVersion(first, FIRST, version)
// Passes the updater's canary (`cli.js version`), then dies on every start as a core.
if (crashesOnStart) source = source.replace('\n', '\nif(process.argv[2]==="__run")process.exit(3);\n')
const file = join(out, `cli-${version}.js`)
diff --git a/cli/e2e/updateHostile.e2e.ts b/cli/e2e/updateHostile.e2e.ts
index f2df725b0..4eca6ae42 100644
--- a/cli/e2e/updateHostile.e2e.ts
+++ b/cli/e2e/updateHostile.e2e.ts
@@ -24,6 +24,7 @@ import { join } from 'node:path'
import { afterAll, afterEach, beforeAll, describe, expect, it, onTestFailed } from 'vitest'
import { LocalClient, type Frame } from './harness/client.js'
import { CLI_ROOT, IsolatedDaemon, until } from './harness/daemon.js'
+import { atVersion } from './harness/release.js'
const FIRST = '43.0.1'
const DISKFULL = process.env.DISKFULL === '1' && process.platform === 'darwin'
@@ -66,7 +67,8 @@ class Releases {
readonly cli = new Map<string, Buffer>()
readonly notify = new Map<string, Buffer>()
add(version: string, inject = ''): void {
- const source = built.replaceAll(FIRST, version).replace('\n', `\n${inject}\n`)
+ // In the lean bundle cli.js carries for the master and the services too (e2e/harness/release.ts).
+ const source = atVersion(built, FIRST, version).replace('\n', `\n${inject}\n`)
const file = join(buildDir, `cli-${version}.js`)
writeFileSync(file, source)
expect(execFileSync(process.execPath, [file, 'version'], { encoding: 'utf8' }).trim()).toBe(version)
diff --git a/cli/e2e/viewersProcess.e2e.ts b/cli/e2e/viewersProcess.e2e.ts
index cf188a06c..f773dccc1 100644
--- a/cli/e2e/viewersProcess.e2e.ts
+++ b/cli/e2e/viewersProcess.e2e.ts
@@ -122,8 +122,8 @@ describe('the DSH viewers in their own process', () => {
return d
}
- it('by default the viewers stay in the core\'s process, as before', async () => {
- const d = await fresh('')
+ it('with HARNESSD_SERVICES=none the viewers stay in the core\'s process, as before', async () => {
+ const d = await fresh('none')
const client = await LocalClient.connect(d)
const sketch = await create(d, client, 'sketch-default', { dsh: HARNESS })
const answer = await serves(await viewerUrl(client, sketch.id))
diff --git a/cli/scripts/check-lean-bundle.mjs b/cli/scripts/check-lean-bundle.mjs
new file mode 100644
index 000000000..deec74b13
--- /dev/null
+++ b/cli/scripts/check-lean-bundle.mjs
@@ -0,0 +1,48 @@
+/**
+ * Before a release is uploaded (scripts/upload-cli.sh): the lean bundle cli.js carries for harnessd's
+ * master and its services (src/harnessd/leanBundle.ts) must start a master and load every service.
+ *
+ * A master started on cli.js runs only a lean bundle that answers its probe, and a service that cannot
+ * start from it is started from cli.js, so a broken one costs memory, not the daemon. This keeps a broken
+ * one from shipping at all: it writes the files out, asks the master's probe (`harnessd.mjs
+ * __harnessd-probe`), and imports the module each process starts on, each in a Node of its own, in a
+ * throwaway home and data folder.
+ *
+ * node scripts/check-lean-bundle.mjs dist/cli.js
+ */
+import { execFileSync } from 'node:child_process'
+import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { join, resolve } from 'node:path'
+import { pathToFileURL } from 'node:url'
+import { readLeanBlock } from './lib/leanBlock.mjs'
+
+const cli = resolve(process.argv[2] ?? 'dist/cli.js')
+const files = readLeanBlock(readFileSync(cli))
+const fail = (why) => { console.error(`error: ${cli}: ${why}`); process.exit(1) }
+if (!files) fail('carries no lean bundle, or one that does not match its checksum')
+if (!files['harnessd.mjs']) fail('its lean bundle has no harnessd.mjs')
+const root = mkdtempSync(join(tmpdir(), 'lean-check-'))
+try {
+ const dir = join(root, 'lean')
+ mkdirSync(dir)
+ for (const [name, code] of Object.entries(files)) writeFileSync(join(dir, name), code)
+ const env = { PATH: process.env.PATH, HOME: join(root, 'home'), ADAPTER_DATA_DIR: join(root, 'data'), ADAPTER_COMPUTER_ID_FILE: join(root, 'computer-id') }
+ const run = (args) => execFileSync(process.execPath, args, { cwd: root, env, encoding: 'utf8', timeout: 30_000, stdio: ['ignore', 'pipe', 'pipe'] })
+ let answer
+ try { answer = run([join(dir, 'harnessd.mjs'), '__harnessd-probe']) } catch (error) { fail(`its master does not answer its probe: ${String(error.stderr || error.message).trim()}`) }
+ if (!answer.includes('harnessd-probe ok')) fail(`its master answered its probe with: ${answer.trim()}`)
+ // Each process's own module: the master's, and every service's (serviceProcess.ts loads the one it is named).
+ const modules = Object.keys(files).filter((name) => /^(masterProcess|serviceProcess|\w+Process)-[A-Z0-9]+\.mjs$/.test(name))
+ if (!modules.some((name) => name.startsWith('masterProcess-')) || !modules.some((name) => name.startsWith('serviceProcess-'))) {
+ fail(`its lean bundle is missing the master's or the services' module (${modules.join(', ') || 'none'})`)
+ }
+ for (const name of modules) {
+ try { run(['--input-type=module', '-e', `await import(${JSON.stringify(pathToFileURL(join(dir, name)).href)})`]) } catch (error) {
+ fail(`${name} does not load: ${String(error.stderr || error.message).trim()}`)
+ }
+ }
+ console.log(`>> lean bundle: the master answers its probe, and ${modules.length} modules load`)
+} finally {
+ rmSync(root, { recursive: true, force: true })
+}
diff --git a/cli/scripts/lib/leanBlock.mjs b/cli/scripts/lib/leanBlock.mjs
new file mode 100644
index 000000000..454ac4bbb
--- /dev/null
+++ b/cli/scripts/lib/leanBlock.mjs
@@ -0,0 +1,48 @@
+/**
+ * The lean bundle cli.js carries (src/harnessd/leanBundle.ts reads it back).
+ *
+ * harnessd's master and its services each run in a process of their own, and a process started on the
+ * whole 4.4 MB cli.js paid about 45 MiB just for Node to parse it, whatever it ran. So the build bundles
+ * the master and the services a second time on their own (src/leanEntry.ts), split so that each process
+ * parses only the files its own code is in, and appends those files to cli.js as a comment, which Node
+ * only skims. The master writes them out and runs itself and the services from them. The release is
+ * still the one cli.js the updater downloads, verifies and swaps, so a lean bundle is always the one
+ * built with the cli.js that carries it.
+ *
+ * The comment holds the sha256 of the files, as one JSON object of name to code, and that JSON,
+ * brotli-compressed and base64-encoded: base64 never contains the `*` and `/` that would end a comment.
+ */
+import { createHash } from 'node:crypto'
+import { brotliCompressSync, brotliDecompressSync, constants } from 'node:zlib'
+
+/** Where the lean bundle starts, at the very end of cli.js. src/harnessd/leanBundle.ts looks for it. */
+export const LEAN_MARKER = '/*@harness-lean:'
+
+/** The comment that carries [files] (file name to code), to append to cli.js. */
+export function leanBlock(files) {
+ const named = Object.fromEntries(Object.entries(files).map(([name, code]) => [name, Buffer.from(code).toString('utf8')]))
+ const payload = Buffer.from(JSON.stringify(named))
+ const sha256 = createHash('sha256').update(payload).digest('hex')
+ const packed = brotliCompressSync(payload, { params: { [constants.BROTLI_PARAM_QUALITY]: 11 } }).toString('base64')
+ return `\n${LEAN_MARKER}${sha256}:${packed}*/\n`
+}
+
+/**
+ * The files the comment at the end of [bundle] (cli.js's bytes) carries, by name; null when it carries
+ * none or one that does not match its checksum. For the release script's check
+ * (scripts/check-lean-bundle.mjs); the daemon reads it with src/harnessd/leanBundle.ts.
+ */
+export function readLeanBlock(bundle) {
+ const text = Buffer.from(bundle).toString('latin1')
+ const start = text.lastIndexOf(LEAN_MARKER)
+ if (start < 0) return null
+ const end = text.indexOf('*/', start)
+ if (end < 0 || text.slice(end + 2).trim() !== '') return null
+ const body = text.slice(start + LEAN_MARKER.length, end)
+ const [sha256, packed] = body.split(':')
+ if (!/^[0-9a-f]{64}$/.test(sha256 ?? '') || !packed) return null
+ let payload
+ try { payload = brotliDecompressSync(Buffer.from(packed, 'base64')) } catch { return null }
+ if (createHash('sha256').update(payload).digest('hex') !== sha256) return null
+ return JSON.parse(payload.toString('utf8'))
+}
diff --git a/cli/scripts/upload-cli.sh b/cli/scripts/upload-cli.sh
index 7982a2fc8..bd1959e23 100755
--- a/cli/scripts/upload-cli.sh
+++ b/cli/scripts/upload-cli.sh
@@ -134,6 +134,7 @@ fi
head -1 "$CLI" | grep -q '^#!' || { echo "error: dist/cli.js lost its shebang on line 1 (esbuild change?)" >&2; exit 1; }
[ "$(node "$CLI" version)" = "$VER" ] || { echo "error: bundled version != $VER (ADAPTER_VERSION not injected)" >&2; exit 1; }
node "$ADAPTER_DIR/scripts/test-core-harness-upgrade.mjs" "$CLI"
+node "$ADAPTER_DIR/scripts/check-lean-bundle.mjs" "$CLI"
# --- Step 3: upload both artifacts + merge the manifest ---
CLI_GCS="harness/cli/${VER}/cli.js"
diff --git a/cli/src/cli.ts b/cli/src/cli.ts
index b052b58e8..589fa8db4 100644
--- a/cli/src/cli.ts
+++ b/cli/src/cli.ts
@@ -179,18 +179,16 @@ import { createServiceLinks } from './core/serviceLinks.js'
import { createViewersLink } from './core/viewersLink.js'
import { createWorkspacesLink } from './core/workspacesLink.js'
import { KNOWN_SERVICES, servicesTheMasterRuns } from './harnessd/services.js'
-import { runSearchService } from './services/searchProcess.js'
-import { runViewersService } from './services/viewersProcess.js'
-import { runWorkspacesService } from './services/workspacesProcess.js'
import { createTeamsLink } from './core/teamsLink.js'
-import { runTeamsService } from './services/teamsProcess.js'
import { SEARCH_REQUESTS, startSearch } from './services/search.js'
import { STORE_REQUESTS, startStore } from './services/store.js'
import { startViewers } from './services/viewers.js'
import { MODELS_REQUESTS, startModels } from './services/models.js'
import { startWorkspaces } from './services/workspaces.js'
import { startFleet } from './services/fleet.js'
-import { describeMasterStatus, probeMaster, readStatusFile, runMaster } from './harnessd/master.js'
+import { describeMasterStatus, readStatusFile } from './harnessd/master.js'
+import { probeThisMaster, startMaster } from './masterProcess.js'
+import { startServiceProcess } from './serviceProcess.js'
import { CORE_EXIT_STOP, CORE_EXIT_UPDATE } from './harnessd/protocol.js'
import { isLocalSocketName, localSocketPath, refuseServedDataFolder, type LocalSocketServer } from './lib/localSocket.js'
import { legacyDaemonStatus, localDaemonStatus, saveDaemonPort } from './lib/daemonEndpoint.js'
@@ -220,7 +218,7 @@ import { DeviceLogSyncer, type DeviceLogFetched } from './lib/e2ee/deviceLogSync
import { DeviceLogStore } from './lib/e2ee/deviceLogStore.js'
import { TrustGroupStore, type GroupMember } from './lib/e2ee/trustGroup.js'
import {
- startSelfUpdater, restore as restoreUpdate, confirm as confirmUpdate, unjudgedUpdate,
+ startSelfUpdater, restore as restoreUpdate, confirm as confirmUpdate,
fetchManifest, downloadVerified, canary, stage, semverGt, isLocalDevBuild,
type Poller, type UpdateEntry,
} from './lib/selfUpdate.js'
@@ -257,8 +255,6 @@ import {
// Daemon stdout/stderr. Capped at LOG_MAX_BYTES — see prepareLogFile/trimLogFile in lib/log.ts.
const LOG_FILE = DAEMON_LOG_FILE
-/** Where a master re-executing on a new bundle leaves word of it, until that master's core is up. */
-const HARNESSD_REEXEC_FILE = join(env.ADAPTER_DATA_DIR, 'harnessd-reexec.json')
// Pre-rename name. Adopted (renamed, keeping the inode) the first time a daemon opens the log, so a
// machine that updates mid-run keeps its history instead of stranding it in a file nobody tails.
// The log has had three names; this slot holds the OLDEST. The middle one (`machine.log`) is adopted
@@ -5488,36 +5484,16 @@ switch (cmd) {
case 'join':
console.error('`harness join` has been removed. Run `harness login`, then `harness start`.')
process.exit(1)
+ // The bundle starts these three without loading this file (entry.ts); from the sources they come here.
case '__harnessd': // internal: the master `harness start` launches; it runs and supervises `__run`
- runMaster({
- nodePath: process.execPath,
- execArgv: process.execArgv,
- scriptPath: SCRIPT_PATH,
- pidFile: PID_FILE,
- statusFile: HARNESSD_STATUS_FILE,
- logFile: LOG_FILE,
- restoreUpdate: () => restoreUpdate(env.ADAPTER_CLI_DIR),
- confirmUpdate: () => confirmUpdate(env.ADAPTER_CLI_DIR),
- version: VERSION,
- reexecMarkerFile: HARNESSD_REEXEC_FILE,
- unjudgedUpdate: (bundle) => unjudgedUpdate(env.ADAPTER_CLI_DIR, bundle),
- })
+ startMaster({ scriptPath: SCRIPT_PATH })
break
case '__harnessd-probe': // internal: a master about to re-execute on this bundle asks it first (harnessd/reexec.ts)
- process.exitCode = probeMaster({ env: process.env, execArgv: process.execArgv, version: VERSION })
+ process.exitCode = probeThisMaster()
break
- case '__service': { // internal: a service harnessd's master runs in its own process (services/process.ts)
- const socketPath = localSocketPath(env.ADAPTER_DATA_DIR, env.PORT)
- const runService = rest[0] === 'search' ? runSearchService : rest[0] === 'viewers' ? runViewersService
- : rest[0] === 'workspaces' ? runWorkspacesService : rest[0] === 'teams' ? runTeamsService : null
- if (!runService || !socketPath) {
- console.error(`[service] ${rest[0] ?? '(none)'}: ${socketPath ? 'no such service in this build' : 'the core has no local socket to reach'}`)
- process.exit(2)
- }
- process.title = `harnessd ${rest[0]}`
- runService({ dataDir: env.ADAPTER_DATA_DIR, socketPath, machineId: computerId(), token: process.env.HARNESSD_SERVICE_TOKEN ?? '' })
+ case '__service': // internal: a service harnessd's master runs in its own process (serviceProcess.ts)
+ void startServiceProcess(rest[0])
break
- }
case '__run': // internal: the detached daemon child reads the durable SSO session — or runs without one
// Inert unless the end-to-end suite asks for its event loop to be held (core/stall.ts).
startStalls(testFaults(process.env.HARNESSD_TEST_FAULTS))
diff --git a/cli/src/config/env.spec.ts b/cli/src/config/env.spec.ts
new file mode 100644
index 000000000..791aeea4f
--- /dev/null
+++ b/cli/src/config/env.spec.ts
@@ -0,0 +1,64 @@
+import { spawnSync } from 'node:child_process'
+import { mkdtempSync, rmSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+import { fileURLToPath } from 'node:url'
+import { afterEach, describe, expect, it } from 'vitest'
+
+// env.ts reads the environment once, as it loads: each case is a process of its own, in a throwaway
+// home, so the legacy-state adoption it runs on load never sees a real ~/.harness.
+const tsx = fileURLToPath(new URL('../../node_modules/tsx/dist/cli.mjs', import.meta.url))
+const module = fileURLToPath(new URL('./env.ts', import.meta.url))
+const roots: string[] = []
+afterEach(() => { for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }) })
+
+const READ = ['NODE_ENV', 'PORT', 'WEB_URL', 'HARNESS_STORE_REF', 'HARNESS_STORE_CATALOG_URL', 'HOOK_INSTALL_ENGINES', 'TERMINAL_BACKENDS', 'RECAP_WITHOUT_DEVICE', 'DISABLE_HOOK_INSTALL', 'TERMINAL_RECONCILE_INTERVAL_MS', 'ADAPTER_DATA_DIR']
+
+function load(variables: Record<string, string>) {
+ const home = mkdtempSync(join(tmpdir(), 'harness-env-'))
+ roots.push(home)
+ const childEnv: NodeJS.ProcessEnv = { PATH: process.env.PATH, HOME: home, ADAPTER_DATA_DIR: join(home, 'data'), ...variables }
+ const script = `import { env } from ${JSON.stringify(module)}; console.log(JSON.stringify(Object.fromEntries(${JSON.stringify(READ)}.map((k) => [k, env[k] instanceof Set ? [...env[k]] : env[k] ?? null]))))`
+ const run = spawnSync(process.execPath, [tsx, '--eval', script], { cwd: home, env: childEnv, encoding: 'utf8', timeout: 20_000 })
+ return { status: run.status, stderr: run.stderr, env: run.status === 0 ? JSON.parse(run.stdout) as Record<string, unknown> : null, home }
+}
+
+describe('the environment, as env.ts reads it', () => {
+ it('fills what is unset with its default, and reads what is set as its kind', () => {
+ const plain = load({})
+ expect(plain.status, plain.stderr).toBe(0)
+ expect(plain.env).toEqual({
+ NODE_ENV: 'development', PORT: 18473, WEB_URL: 'https://harness.autonomous.ai', HARNESS_STORE_REF: null, HARNESS_STORE_CATALOG_URL: null,
+ HOOK_INSTALL_ENGINES: null, TERMINAL_BACKENDS: null, RECAP_WITHOUT_DEVICE: true, DISABLE_HOOK_INSTALL: false,
+ TERMINAL_RECONCILE_INTERVAL_MS: null, ADAPTER_DATA_DIR: join(plain.home, 'data'),
+ })
+ const set = load({
+ NODE_ENV: 'test', PORT: '9123', BACKEND_WS_URL: 'ws://localhost:8090', HARNESS_STORE_REF: 'feature/x', HARNESS_STORE_CATALOG_URL: ' http://127.0.0.1:9/c ',
+ HOOK_INSTALL_ENGINES: 'claude, codex,', TERMINAL_BACKENDS: 'tmux', RECAP_WITHOUT_DEVICE: 'false', DISABLE_HOOK_INSTALL: 'true', TERMINAL_RECONCILE_INTERVAL_MS: '6000',
+ })
+ expect(set.status, set.stderr).toBe(0)
+ expect(set.env).toMatchObject({
+ NODE_ENV: 'test', PORT: 9123, WEB_URL: 'http://localhost:3000', HARNESS_STORE_REF: 'feature/x', HARNESS_STORE_CATALOG_URL: 'http://127.0.0.1:9/c',
+ HOOK_INSTALL_ENGINES: ['claude', 'codex'], TERMINAL_BACKENDS: ['tmux'], RECAP_WITHOUT_DEVICE: false, DISABLE_HOOK_INSTALL: true,
+ TERMINAL_RECONCILE_INTERVAL_MS: 6000,
+ })
+ })
+
+ it('ignores a store setting it cannot use, which under zod stopped every `harness` command as it loaded', () => {
+ const loaded = load({ HARNESS_STORE_CATALOG_URL: 'not a url', HARNESS_STORE_REF: 'not a ref!' })
+ expect(loaded.status, loaded.stderr).toBe(0)
+ expect(loaded.env).toMatchObject({ HARNESS_STORE_CATALOG_URL: null, HARNESS_STORE_REF: null })
+ })
+
+ it('stops, naming every variable it cannot use and why', () => {
+ const loaded = load({ NODE_ENV: 'staging', TERMINAL_BACKENDS: 'nonsense' })
+ expect(loaded.status).toBe(1)
+ expect(loaded.stderr).toContain('Invalid environment variables:')
+ expect(loaded.stderr).toContain('NODE_ENV')
+ expect(loaded.stderr).toContain('expected one of "development"|"production"|"test"')
+ expect(loaded.stderr).toContain('TERMINAL_BACKENDS')
+ const tooOften = load({ TERMINAL_RECONCILE_INTERVAL_MS: '100' })
+ expect(tooOften.status).toBe(1)
+ expect(tooOften.stderr).toContain('TERMINAL_RECONCILE_INTERVAL_MS must be at least 5000')
+ })
+})
diff --git a/cli/src/config/env.ts b/cli/src/config/env.ts
index 646c94745..4b987ed0c 100644
--- a/cli/src/config/env.ts
+++ b/cli/src/config/env.ts
@@ -2,7 +2,7 @@ import './loadEnv.js'
import { existsSync, mkdirSync, readdirSync, renameSync, rmSync } from 'fs'
import { homedir } from 'os'
import { join } from 'path'
-import { z } from 'zod'
+import { EnvIssue, flag, matching, maybe, maybeNumber, number, oneOf, parseEnv, text, unlessFalse, url, type EnvField, type EnvOf } from './envFields.js'
import { parseTerminalBackends } from './terminalConfig.js'
import { adoptComputerId } from '../lib/computerIdentity.js'
@@ -115,12 +115,12 @@ function migrateLegacyAdapterState(): void {
migrateLegacyAdapterState()
-const envSchema = z.object({
- NODE_ENV: z.enum(['development', 'production', 'test']).default('development'),
+const fields = {
+ NODE_ENV: oneOf(['development', 'production', 'test'], 'development'),
// Preferred localhost control port, also the stable name of this user's private daemon socket.
// When another OS user holds it, Unix daemons record a separate TCP port in this user's data dir;
// engine hooks and CLI commands use that actual port while native clients use the private socket.
- PORT: z.string().default('18473').transform(Number),
+ PORT: number('18473'),
// The loopback port `harness login` listens on for the SSO redirect. 0 (the default) takes whatever
// the OS gives, which is right on a real computer: the browser and the listener are the same
// loopback, so the port never has to be known in advance.
@@ -129,89 +129,73 @@ const envSchema = z.object({
// host and the listener is not, so the only way the redirect ever arrives is a published port — and
// a port cannot be published before it is known. Set this, publish the same number, and the real
// SSO flow works from a box with no browser of its own. See cli/docker/remote-machine/.
- ADAPTER_LOGIN_CALLBACK_PORT: z.string().default('0').transform(Number),
+ ADAPTER_LOGIN_CALLBACK_PORT: number('0'),
// The backend the CLI dials (`/api/adapter-ws`). Local dev: ws://localhost:8090.
- BACKEND_WS_URL: z.string().default('wss://harness-api.autonomous.ai'),
+ BACKEND_WS_URL: text('wss://harness-api.autonomous.ai'),
// SSO account plane used by native-loopback login and the adapter WebSocket.
- AUTONOMOUS_ENV: z.enum(['prod', 'stag']).default('prod'),
+ AUTONOMOUS_ENV: oneOf(['prod', 'stag'], 'prod'),
// Web app base URL — used to print the agent's chat link on `adapter start`. Local: http://localhost:3000.
- WEB_URL: z.string().default('https://harness.autonomous.ai'),
+ WEB_URL: text('https://harness.autonomous.ai'),
// Set to '1' to lift the path fences shared by lib/pathContainment.ts: the New Agent folder browser
// (fs_list_dir), the project preview, git_project_info, and media previews all stop measuring what
// they were asked for against the folders they are allowed to read.
// Off by default so a fat-fingered path or a compromised relay hop can't walk the whole filesystem.
- HARNESS_FS_BROWSE_UNRESTRICTED: z.string().optional(),
+ HARNESS_FS_BROWSE_UNRESTRICTED: maybe,
// Where Claude Code writes its per-session JSONL transcripts.
- CLAUDE_PROJECTS_DIR: z.string().default(join(homedir(), '.claude', 'projects')),
+ CLAUDE_PROJECTS_DIR: text(join(homedir(), '.claude', 'projects')),
// Codex state root. Only hook-registered rollout files beneath <CODEX_HOME>/sessions are exposed.
- CODEX_HOME: z.string().default(join(homedir(), '.codex')),
+ CODEX_HOME: text(join(homedir(), '.codex')),
// Grok state root. Conversation records live below <GROK_HOME>/sessions/<encoded-cwd>/<uuid>/updates.jsonl.
- GROK_HOME: z.string().default(join(homedir(), '.grok')),
+ GROK_HOME: text(join(homedir(), '.grok')),
// Antigravity CLI state root. Conversation transcripts live below
// <AGY_HOME>/brain/<conversationId>/.system_generated/logs/transcript_full.jsonl — the `_full`
// sibling is the one to read: plain `transcript.jsonl` re-quotes tool arguments and truncates
// long content. The hook payload names the same path, so this default is only the fallback.
- AGY_HOME: z.string().default(join(homedir(), '.gemini', 'antigravity-cli')),
+ AGY_HOME: text(join(homedir(), '.gemini', 'antigravity-cli')),
// Where the Antigravity CLI reads its shared customization root (hooks.json lives here).
- AGY_CONFIG_DIR: z.string().default(join(homedir(), '.gemini', 'config')),
+ AGY_CONFIG_DIR: text(join(homedir(), '.gemini', 'config')),
// GitHub Copilot CLI state root. Its per-session event stream lives at
// <COPILOT_HOME>/session-state/<sessionId>/events.jsonl, and hooks are read from
// <COPILOT_HOME>/hooks/*.json — a directory, so Harness drops in its own file.
- COPILOT_HOME: z.string().default(join(homedir(), '.copilot')),
+ COPILOT_HOME: text(join(homedir(), '.copilot')),
// Legacy Cursor fallback. cursor/home.ts resolves its distinct config (chats/hooks) and data
// (projects/transcripts) roots using CURSOR_CONFIG_DIR, XDG_CONFIG_HOME and CURSOR_DATA_DIR.
- CURSOR_HOME: z.string().default(join(homedir(), '.cursor')),
+ CURSOR_HOME: text(join(homedir(), '.cursor')),
// OpenCode state root — the SQLite store lives at <OPENCODE_DATA_DIR>/opencode.db (honors
// XDG_DATA_HOME). Sessions are polled from that DB (no per-session transcript file).
- OPENCODE_DATA_DIR: z
- .string()
- .default(join(process.env.XDG_DATA_HOME || join(homedir(), '.local', 'share'), 'opencode')),
+ OPENCODE_DATA_DIR: text(join(process.env.XDG_DATA_HOME || join(homedir(), '.local', 'share'), 'opencode')),
// OpenCode plugin dir the adapter drops its discovery plugin into (honors XDG_CONFIG_HOME).
- OPENCODE_PLUGIN_DIR: z
- .string()
- .default(join(process.env.XDG_CONFIG_HOME || join(homedir(), '.config'), 'opencode', 'plugin')),
+ OPENCODE_PLUGIN_DIR: text(join(process.env.XDG_CONFIG_HOME || join(homedir(), '.config'), 'opencode', 'plugin')),
// Kilo state root — the SQLite store lives at <KILO_DATA_DIR>/kilo.db. Kilo is an opencode fork and
// keeps the same layout, but NOT the same overrides: measured on 7.4.20 via `kilo debug paths`, it
// honours XDG_DATA_HOME and ignores both `KILO_DATA_DIR` and `OPENCODE_DATA_DIR`. So this variable
// steers the ADAPTER's reads only; anything that has to move KILO's own writes (the recap one-shot)
// must set XDG_DATA_HOME on the child instead — see `KiloWorker` in lib/oneshot.ts.
- KILO_DATA_DIR: z
- .string()
- .default(join(process.env.XDG_DATA_HOME || join(homedir(), '.local', 'share'), 'kilo')),
+ KILO_DATA_DIR: text(join(process.env.XDG_DATA_HOME || join(homedir(), '.local', 'share'), 'kilo')),
// Kilo plugin dir the adapter drops its discovery plugin into (honors XDG_CONFIG_HOME — measured).
- KILO_PLUGIN_DIR: z
- .string()
- .default(join(process.env.XDG_CONFIG_HOME || join(homedir(), '.config'), 'kilo', 'plugin')),
+ KILO_PLUGIN_DIR: text(join(process.env.XDG_CONFIG_HOME || join(homedir(), '.config'), 'kilo', 'plugin')),
// Pi state root. Session transcripts live under <PI_HOME>/agent/sessions/--<mangled-cwd>--/*.jsonl and
// the adapter's discovery extension is installed into <PI_HOME>/agent/extensions.
- PI_HOME: z.string().default(join(homedir(), '.pi')),
+ PI_HOME: text(join(homedir(), '.pi')),
// Hermes state root — the SQLite store is <HERMES_HOME>/state.db and the shell hooks the adapter
// installs live in <HERMES_HOME>/config.yaml (+ shell-hooks-allowlist.json).
- HERMES_HOME: z.string().default(join(homedir(), '.hermes')),
+ HERMES_HOME: text(join(homedir(), '.hermes')),
// Command Code state root — transcripts live under <COMMANDCODE_HOME>/projects/<cwd-slug>/<id>.jsonl
// and the adapter installs its shell hooks into <COMMANDCODE_HOME>/settings.json.
- COMMANDCODE_HOME: z.string().default(join(homedir(), '.commandcode')),
+ COMMANDCODE_HOME: text(join(homedir(), '.commandcode')),
// Devin CLI state root — history is the SQLite store <DEVIN_HOME>/sessions.db (WAL, no transcript file
// unless the user passes --export) and <DEVIN_HOME>/session_locks/<id>.lock holds the owning PID.
- DEVIN_HOME: z
- .string()
- .default(join(process.env.XDG_DATA_HOME || join(homedir(), '.local', 'share'), 'devin', 'cli')),
+ DEVIN_HOME: text(join(process.env.XDG_DATA_HOME || join(homedir(), '.local', 'share'), 'devin', 'cli')),
// Muse Code state root. Transcripts are JSONL but the layout is DATE-SHARDED, not hashed by project
// path: <MUSE_HOME>/sessions/YYYY/MM/DD/<session-uuid>/session.jsonl, with sub-agents one level deeper
// under `subagent/<child-uuid>/`. The only link back to a project is `workspace_root`, carried in the
// FIRST record of each file — which is why this engine is discovered by scanning rather than by a hook.
- MUSE_HOME: z
- .string()
- .default(join(process.env.XDG_DATA_HOME || join(homedir(), '.local', 'share'), 'muse')),
- MUSE_CONFIG_DIR: z
- .string()
- .default(join(process.env.XDG_CONFIG_HOME || join(homedir(), '.config'), 'muse')),
+ MUSE_HOME: text(join(process.env.XDG_DATA_HOME || join(homedir(), '.local', 'share'), 'muse')),
+ MUSE_CONFIG_DIR: text(join(process.env.XDG_CONFIG_HOME || join(homedir(), '.config'), 'muse')),
// Amp plugin dir the adapter drops its discovery plugin into (honors XDG_CONFIG_HOME). Amp calls these
// "system plugins" and loads every `*.ts` there for EVERY thread, which is what makes one install cover
// all projects — the project-local `.amp/plugins/` alternative would need one copy per repo.
- AMP_PLUGIN_DIR: z
- .string()
- .default(join(process.env.XDG_CONFIG_HOME || join(homedir(), '.config'), 'amp', 'plugins')),
+ AMP_PLUGIN_DIR: text(join(process.env.XDG_CONFIG_HOME || join(homedir(), '.config'), 'amp', 'plugins')),
// Where the Amp plugin writes the transcripts this adapter tails.
//
// Amp is the only supported engine that keeps NO conversation on disk: threads live on the server and
@@ -219,190 +203,176 @@ const envSchema = z.object({
// `session.json`. `amp threads export` can fetch the content, but it costs a ~1.5s network round trip
// per read and never shows a message before it is complete. So the plugin writes the transcript we tail,
// and this directory — ours, not Amp's — is the trusted root for it.
- AMP_SESSIONS_DIR: z.string().default(join(adapterDataDir, 'amp-sessions')),
+ AMP_SESSIONS_DIR: text(join(adapterDataDir, 'amp-sessions')),
// Amp's own state dir, read-only for us: `session.json` there maps a tmux pane to the thread started in
// it, which is how a re-attaching daemon re-binds a pane without guessing.
- AMP_STATE_DIR: z
- .string()
- .default(join(process.env.XDG_DATA_HOME || join(homedir(), '.local', 'share'), 'amp')),
+ AMP_STATE_DIR: text(join(process.env.XDG_DATA_HOME || join(homedir(), '.local', 'share'), 'amp')),
// Devin's user-level config, where the adapter merges its hooks under a "hooks" key. Devin reads
// Claude's hook schema verbatim, so the installed block is shaped exactly like ~/.claude/settings.json.
- DEVIN_CONFIG_PATH: z
- .string()
- .default(join(process.env.XDG_CONFIG_HOME || join(homedir(), '.config'), 'devin', 'config.json')),
+ DEVIN_CONFIG_PATH: text(join(process.env.XDG_CONFIG_HOME || join(homedir(), '.config'), 'devin', 'config.json')),
// Where the tmux-session registry and daemon-local state are persisted.
- ADAPTER_DATA_DIR: z.string().default(adapterDataDir),
+ ADAPTER_DATA_DIR: text(adapterDataDir),
// Logs meant to be READ and sent — the dial's `dial-YYYYMMDD.log` — beside the desktop app's own
// `app-*.log`/`cli-*.log`, so one directory holds everything a bug report needs. Not the data dir:
// `harness.log` there is the daemon's console, and `harness reset` wipes it.
- HARNESS_LOGS_DIR: z.string().default(join(adapterRootDir, 'logs')),
+ HARNESS_LOGS_DIR: text(join(adapterRootDir, 'logs')),
// Where domain-specific harnesses are installed (`harness dsh install`): one directory per
// `<owner>/<name>` plus `installed.json`. Product-root state like the SSO session, not daemon data.
- DSH_DIR: z.string().default(join(adapterRootDir, 'dsh')),
+ DSH_DIR: text(join(adapterRootDir, 'dsh')),
// The ref the built-in shelf (`store/*` of the Harness monorepo) installs from, instead of the one
// its registry entries name (`main`). For trying a store change end to end BEFORE it merges: push
// the branch, run the daemon with HARNESS_STORE_REF=<branch>, and Get in the store fetches from it.
- HARNESS_STORE_REF: z.string().regex(/^[A-Za-z0-9._\/-]{1,200}$/).optional().catch(undefined),
- // Optional catalog mirror. Public HTTPS in production; loopback HTTP supports isolated tests.
- HARNESS_STORE_CATALOG_URL: z.string().url().refine((value) => {
- const url = new URL(value)
- return !url.username && !url.password && (url.protocol === 'https:'
- || (url.protocol === 'http:' && ['127.0.0.1', 'localhost', '[::1]'].includes(url.hostname)))
- }).optional().catch(undefined),
+ HARNESS_STORE_REF: matching(/^[A-Za-z0-9._\/-]{1,200}$/),
+ // Optional catalog mirror. Public HTTPS in production; loopback HTTP supports isolated tests. Anything
+ // else is ignored, as if unset (under zod, a value that was not a URL at all threw as env.ts loaded,
+ // and every `harness` command with it).
+ HARNESS_STORE_CATALOG_URL: url((address) => !address.username && !address.password && (address.protocol === 'https:'
+ || (address.protocol === 'http:' && ['127.0.0.1', 'localhost', '[::1]'].includes(address.hostname)))),
// This computer's stable id, minted once and never regenerated (see computerIdFile above). Pin it
// explicitly on a box with no durable home — a container or CI job that gets a fresh ~/.harness on
// every boot would otherwise look like a NEW computer each time and collect a machine per start.
// Setting this is a pin, not a regeneration: it is never written to disk, so unsetting it returns
// you to the file's id. Any 16-64 hex (dashes allowed, the backend de-dashes).
- ADAPTER_COMPUTER_ID: z.string().optional(),
- ADAPTER_COMPUTER_ID_FILE: z.string().default(computerIdFile),
+ ADAPTER_COMPUTER_ID: maybe,
+ ADAPTER_COMPUTER_ID_FILE: text(computerIdFile),
// Set to 'true' to skip auto-installing lifecycle hooks for every supported engine.
- DISABLE_HOOK_INSTALL: z.string().default('false').transform((v) => v === 'true'),
+ DISABLE_HOOK_INSTALL: flag('false'),
// Install only these engines' hooks (comma-separated, e.g. `claude,codex`); every engine's when unset.
// For a machine where Harness should leave the other tools' settings alone, and for the end-to-end
// suite, which installs Claude Code's and Codex's alone, inside its throwaway home.
- HOOK_INSTALL_ENGINES: z.string().optional().transform((v) => v === undefined
+ HOOK_INSTALL_ENGINES: (raw: string | undefined): Set<string> | null => raw === undefined
? null
- : new Set(v.split(',').map((name) => name.trim()).filter(Boolean))),
+ : new Set(raw.split(',').map((name) => name.trim()).filter(Boolean)),
// `harness start` and `harness login` install the `grid` CLI when the machine has none (see
// lib/gridInstall.ts). Off for tests and for a machine whose grid is managed some other way.
- DISABLE_GRID_INSTALL: z.string().default('false').transform((v) => v === 'true'),
+ DISABLE_GRID_INSTALL: flag('false'),
// Terminal backends to watch. UNSET MEANS AUTO — every backend usable here, which is tmux. A value
// pins the set; a retired name still in someone's environment is dropped with a warning.
- TERMINAL_BACKENDS: z.string().optional().transform((value, context) => {
- if (value === undefined || value === '') return undefined
- try { return parseTerminalBackends(value) } catch (error) {
- context.addIssue({ code: 'custom', message: error instanceof Error ? error.message : 'invalid terminal backends' })
- return z.NEVER
+ TERMINAL_BACKENDS: (raw: string | undefined): ReturnType<typeof parseTerminalBackends> | undefined => {
+ if (raw === undefined || raw === '') return undefined
+ try { return parseTerminalBackends(raw) } catch (error) {
+ throw new EnvIssue(error instanceof Error ? error.message : 'invalid terminal backends')
}
- }),
+ },
// Neutral discovery interval. The legacy tmux name remains a one-release fallback.
- TERMINAL_RECONCILE_INTERVAL_MS: z.string().optional().transform((value) => value === undefined ? undefined : Number(value)),
+ TERMINAL_RECONCILE_INTERVAL_MS: maybeNumber,
// How often (ms) the reaper checks tmux panes and drops dead sessions.
- TMUX_REAP_INTERVAL_MS: z.string().default('5000').transform(Number),
+ TMUX_REAP_INTERVAL_MS: number('5000'),
// Path to the `claude` CLI for the device turn-recap one-shot (else resolved from PATH).
- CLAUDE_PATH: z.string().optional(),
+ CLAUDE_PATH: maybe,
// Path to the Cursor Agent CLI for recap one-shots (else a verified `agent`/`cursor-agent` is used).
- CURSOR_PATH: z.string().optional(),
+ CURSOR_PATH: maybe,
// Path to the `opencode` CLI for OpenCode recap one-shots (else `opencode` is resolved from PATH).
- OPENCODE_PATH: z.string().optional(),
+ OPENCODE_PATH: maybe,
// Path to the `pi` CLI for Pi recap one-shots (else `pi` is resolved from PATH).
- PI_PATH: z.string().optional(),
+ PI_PATH: maybe,
// Path to the `hermes` CLI for Hermes recap one-shots (else `hermes` is resolved from PATH).
- HERMES_PATH: z.string().optional(),
+ HERMES_PATH: maybe,
// Path to the `commandcode` CLI for Command Code recap one-shots (else resolved from PATH).
- COMMANDCODE_PATH: z.string().optional(),
+ COMMANDCODE_PATH: maybe,
// Path to the `devin` CLI for Devin recap one-shots (else `devin` is resolved from PATH).
- DEVIN_PATH: z.string().optional(),
- MUSE_PATH: z.string().optional(),
+ DEVIN_PATH: maybe,
+ MUSE_PATH: maybe,
// Path to the `amp` CLI for Amp recap one-shots (else `amp` is resolved from PATH).
- AMP_PATH: z.string().optional(),
+ AMP_PATH: maybe,
// Path to the `kilo` CLI for Kilo recap one-shots (else `kilo` is resolved from PATH). `@kilocode/cli`
// also installs it as `kilocode`; both are the same file.
- KILO_PATH: z.string().optional(),
+ KILO_PATH: maybe,
// Path to the xAI Grok CLI for interactive sessions and voice-router one-shots.
- GROK_PATH: z.string().optional(),
+ GROK_PATH: maybe,
// Path to the GitHub Copilot CLI. Two builds answer to `copilot` on a typical machine — a compiled
// binary and the npm loader script — so an override is worth having.
- COPILOT_PATH: z.string().optional(),
+ COPILOT_PATH: maybe,
// Path to the Antigravity CLI. Two binaries answer to `agy` on a typical PATH — the CLI itself and
// the Antigravity IDE launcher — so an override is worth having.
- AGY_PATH: z.string().optional(),
- RECAP_WITHOUT_DEVICE: z.string().default('true').transform((v) => v !== 'false'),
+ AGY_PATH: maybe,
+ RECAP_WITHOUT_DEVICE: unlessFalse,
// Model for the voice router one-shot classifier (Overview voice → pick the agent). Small/fast by default.
- VOICE_ROUTE_MODEL: z.string().default('haiku'),
+ VOICE_ROUTE_MODEL: text('haiku'),
// Test override: run the recap even with no device connected (mirrors node isRecapForced()).
- RECAP_FORCE: z.string().default('false').transform((v) => v === 'true'),
+ RECAP_FORCE: flag('false'),
// Log one line per backend frame (type + audience + a few opaque ids). OFF by default: it is noisy,
// and it is the only place frames are visible in the clear — every content-bearing frame is E2EE
// encrypted on the wire, so a packet capture cannot answer "what did the adapter actually send".
// Never prints message text, transcripts, tokens or payload bodies.
- LOG_FRAMES: z.string().default('false').transform((v) => v === 'true'),
+ LOG_FRAMES: flag('false'),
// TEMPORARY / diagnostic. Forces iceTransportPolicy:'relay', so the ONLY candidate either peer can
// offer is a Cloudflare TURN allocation — the way to exercise the relay path on a pair that would
// otherwise always connect directly. Costs money per GB while it is on; never ship it enabled.
- TERMINAL_P2P_FORCE_RELAY: z.string().default('false').transform((v) => v === 'true'),
+ TERMINAL_P2P_FORCE_RELAY: flag('false'),
// ── OpenRouter gateway agents (`ori claude`, `ori codex`, …) ───────────────────────────────────
// An agent whose CLI is pointed at OpenRouter has no vendor credential to spend, so its voice route
// is served by ONE direct chat/completions call instead of a vendor one-shot: spawning a whole coding
// agent for it would bill an account the user may not even have. A cheap, fast model by default; ''
// disables that call (the VOICE_ROUTE_MODEL convention) and falls back to the engine path.
- ORI_VOICE_ROUTE_MODEL: z.string().default('deepseek/deepseek-v4-flash'),
+ ORI_VOICE_ROUTE_MODEL: text('deepseek/deepseek-v4-flash'),
// Where `ori login` stores the key ({ createdAt, key, userId }). Only read when neither the daemon
// env nor the agent's own process supplies one.
- ORI_CREDENTIALS_PATH: z.string().default(join(homedir(), '.ori', 'credentials.json')),
+ ORI_CREDENTIALS_PATH: text(join(homedir(), '.ori', 'credentials.json')),
// ── self-update (the daemon polls a GCS manifest and swaps its own bundle) ──────────────────────
// Manifest URL (same GCS bucket + metadata.json shape as the device OTA; key = ADAPTER_UPDATE_KEY).
- ADAPTER_UPDATE_URL: z
- .string()
- .default('https://storage.googleapis.com/s3-autonomous-upgrade-3/harness/cli/metadata.json'),
- ADAPTER_UPDATE_KEY: z.string().default('cli'),
+ ADAPTER_UPDATE_URL: text('https://storage.googleapis.com/s3-autonomous-upgrade-3/harness/cli/metadata.json'),
+ ADAPTER_UPDATE_KEY: text('cli'),
// How often the daemon checks for a newer build (ms). The poll is a tiny no-cache metadata fetch;
// the artifact is downloaded only when the manifest version is strictly newer.
- ADAPTER_UPDATE_CHECK_MS: z.string().default('60000').transform(Number),
+ ADAPTER_UPDATE_CHECK_MS: number('60000'),
// The wall-clock second each check lands on, when the interval divides a minute. The desktop app
// spawns `harness start` only around :15 (see desktop/lib/ws/local_cli_discovery.dart `inSpawnSlot`);
// keeping the update handoff 30s away from that is what stops the two from fighting over the spawn
// lock. Set to a negative number to keep the plain interval.
- ADAPTER_UPDATE_SLOT_SEC: z.string().default('45').transform(Number),
+ ADAPTER_UPDATE_SLOT_SEC: number('45'),
// How long an update's download may go without a byte, and take in all, before the check gives up and
// the next one tries again (lib/selfUpdate.ts `TransferLimits`). The manifest gets at most a minute.
- ADAPTER_UPDATE_IDLE_MS: z.string().default('60000').transform(Number),
- ADAPTER_UPDATE_DEADLINE_MS: z.string().default('900000').transform(Number),
+ ADAPTER_UPDATE_IDLE_MS: number('60000'),
+ ADAPTER_UPDATE_DEADLINE_MS: number('900000'),
// Set 'true' to disable self-update entirely.
- ADAPTER_UPDATE_DISABLE: z.string().default('false').transform((v) => v === 'true'),
+ ADAPTER_UPDATE_DISABLE: flag('false'),
/** How long a daemon whose start-up failed stays up serving nothing but its updater, before it
* gives a clean process a turn. ~15 update slots; `0` keeps it up for ever. */
- ADAPTER_SAFE_MODE_MS: z.string().default('900000').transform(Number),
+ ADAPTER_SAFE_MODE_MS: number('900000'),
// Install dir holding the packaged cli.js + notify.mjs that the self-updater swaps in place.
- ADAPTER_CLI_DIR: z.string().default(adapterCliDir),
+ ADAPTER_CLI_DIR: text(adapterCliDir),
// Where the managed Node runtime lives. Read (never written) by this process: the hook command
// lines have to name an interpreter by absolute path, because a hook fires in a shell whose PATH
// we do not control — see managedNodePath() in lib/nodeRuntime.ts.
- ADAPTER_RUNTIME_DIR: z.string().default(adapterRuntimeDir),
+ ADAPTER_RUNTIME_DIR: text(adapterRuntimeDir),
// The runtime manifest — a DIFFERENT document from ADAPTER_UPDATE_URL's: keyed by platform
// (`darwin-arm64`, `linux-x64`, …) with url/sha256/size/archiveRoot per entry. Deliberately the
// same URL `install.sh` and Desktop Harness read, so all three land on identical bytes.
- ADAPTER_RUNTIME_METADATA_URL: z
- .string()
- .default('https://storage.googleapis.com/s3-autonomous-upgrade-3/harness/runtime/metadata.json'),
+ ADAPTER_RUNTIME_METADATA_URL: text('https://storage.googleapis.com/s3-autonomous-upgrade-3/harness/runtime/metadata.json'),
// The managed grid's manifest — its own document, as tmux's is (harness/runtime/tmux/metadata.json):
// install.sh slices a manifest by the FIRST platform key it finds, and Node's already has one. The
// same entry shape (version/url/sha256/size/archiveRoot), and its version is the PIN: the grid this
// build of the CLI drives, moved on purpose by a release and never by grid's own updater — see
// ensureManagedGrid() in lib/runtimeInstall.ts, which follows it on every daemon start.
- ADAPTER_GRID_RUNTIME_METADATA_URL: z
- .string()
- .default('https://storage.googleapis.com/s3-autonomous-upgrade-3/harness/runtime/grid/metadata.json'),
+ ADAPTER_GRID_RUNTIME_METADATA_URL: text('https://storage.googleapis.com/s3-autonomous-upgrade-3/harness/runtime/grid/metadata.json'),
// Where the `harness` launcher lives. Same name (and default) `scripts/install-cli.sh` uses, so a
// sandboxed install and this process agree on which launcher they are talking about.
- HARNESS_BIN_DIR: z.string().default(adapterBinDir),
+ HARNESS_BIN_DIR: text(adapterBinDir),
// ── the dial on the USB cable ──────────────────────────────────────────────────────────────────
// Set 'true' to leave the serial port alone entirely. The port is exclusive, so this is what a
// developer flips before running esptool or a serial monitor against the dial.
- CABLE_DISABLE: z.string().default('false').transform((v) => v === 'true'),
+ CABLE_DISABLE: flag('false'),
// Transcription for voice captured on the dial. The device holds no cloud credential and never talks to
// one; the daemon has an account, and this call is authenticated by the SSO session `harness login`
// leaves behind — no shared secret, because this repository is public. Path only: the host comes from
// BACKEND_WS_URL, so a daemon pointed at a local backend transcribes against that one too.
- CABLE_STT_PATH: z.string().default('/api/voice/stt'),
+ CABLE_STT_PATH: text('/api/voice/stt'),
// Firmware for the dial: the SAME published manifest the device used to poll before it lost WiFi, so
// `make upload-circle` remains the only way a release happens — this only changes who downloads.
- CABLE_FW_MANIFEST_URL: z
- .string()
- .default('https://storage.googleapis.com/s3-autonomous-upgrade-3/harness/esp32/ota/metadata.json'),
- CABLE_FW_DISABLE: z.string().default('false').transform((v) => v === 'true'),
-})
+ CABLE_FW_MANIFEST_URL: text('https://storage.googleapis.com/s3-autonomous-upgrade-3/harness/esp32/ota/metadata.json'),
+ CABLE_FW_DISABLE: flag('false'),
+} satisfies Record<string, EnvField<unknown>>
-export type Env = z.infer<typeof envSchema>
+export type Env = EnvOf<typeof fields>
function validateEnv(): Env {
- const parsed = envSchema.safeParse(process.env)
+ const parsed = parseEnv(fields, process.env)
- if (!parsed.success) {
- console.error('Invalid environment variables:', parsed.error.flatten().fieldErrors)
+ if (!parsed.ok) {
+ console.error('Invalid environment variables:', parsed.issues)
process.exit(1)
}
diff --git a/cli/src/config/envFields.spec.ts b/cli/src/config/envFields.spec.ts
new file mode 100644
index 000000000..19dc0a030
--- /dev/null
+++ b/cli/src/config/envFields.spec.ts
@@ -0,0 +1,71 @@
+import { describe, expect, it } from 'vitest'
+import { z } from 'zod'
+import { EnvIssue, flag, matching, maybe, maybeNumber, number, oneOf, parseEnv, text, unlessFalse, url } from './envFields.js'
+
+// The values a variable can arrive with: unset, empty, padded, and the ones each kind is about.
+const VALUES = [undefined, '', ' ', 'true', 'false', 'TRUE', '0', '42', ' 7 ', 'x', 'test', 'prod', 'stag',
+ 'https://mirror.example/catalog', ' https://mirror.example/x ', 'http://127.0.0.1:9/c', 'http://localhost/c', 'http://[::1]/c',
+ 'http://mirror.example/c', 'https://user:[email protected]/c', 'ftp://mirror.example', 'http:mirror.example', 'feature/x-1.2']
+
+/** What zod's schema made of [value]: the value, or `issue` when it refused it. */
+function viaZod(schema: z.ZodType, value: string | undefined): unknown {
+ const parsed = z.object({ V: schema }).safeParse(value === undefined ? {} : { V: value })
+ return parsed.success ? (parsed.data as { V: unknown }).V : 'issue'
+}
+function viaField(field: (raw: string | undefined) => unknown, value: string | undefined): unknown {
+ try { return field(value) } catch (error) { if (error instanceof EnvIssue) return 'issue'; throw error }
+}
+
+describe('an environment variable read as zod read it', () => {
+ const accept = (address: URL) => !address.username && !address.password && (address.protocol === 'https:'
+ || (address.protocol === 'http:' && ['127.0.0.1', 'localhost', '[::1]'].includes(address.hostname)))
+ const kinds: Array<[string, z.ZodType, (raw: string | undefined) => unknown]> = [
+ ['a string with a default', z.string().default('d'), text('d')],
+ ['a string or nothing', z.string().optional(), maybe],
+ ['a number with a default', z.string().default('5000').transform(Number), number('5000')],
+ ['a number or nothing', z.string().optional().transform((value) => value === undefined ? undefined : Number(value)), maybeNumber],
+ ['a flag, off by default', z.string().default('false').transform((v) => v === 'true'), flag('false')],
+ ['a flag, on by default', z.string().default('true').transform((v) => v === 'true'), flag('true')],
+ ['on unless false', z.string().default('true').transform((v) => v !== 'false'), unlessFalse],
+ ['one of a few', z.enum(['development', 'production', 'test']).default('development'), oneOf(['development', 'production', 'test'], 'development')],
+ ['a pattern or nothing', z.string().regex(/^[A-Za-z0-9._/-]{1,200}$/).optional().catch(undefined), matching(/^[A-Za-z0-9._/-]{1,200}$/)],
+ ]
+ it.each(kinds)('%s', (_kind, schema, field) => {
+ for (const value of VALUES) expect(viaField(field, value), JSON.stringify(value)).toEqual(viaZod(schema, value))
+ })
+
+ it('a URL or nothing, as zod read every URL, and nothing for what zod threw on as it loaded', () => {
+ const schema = z.string().url().refine((value) => accept(new URL(value))).optional().catch(undefined)
+ for (const value of VALUES) {
+ let expected: unknown
+ // zod ran the refinement on a value it had already refused as a URL, and `new URL` threw out of it.
+ try { expected = viaZod(schema, value) } catch { expected = undefined }
+ expect(viaField(url(accept), value), JSON.stringify(value)).toEqual(expected)
+ }
+ expect(url(accept)(' https://mirror.example/x ')).toBe('https://mirror.example/x')
+ expect(url(accept)('not a url')).toBeUndefined()
+ })
+
+ it('says what one of a few must be', () => {
+ expect(() => oneOf(['prod', 'stag'], 'prod')('dev')).toThrow('Invalid option: expected one of "prod"|"stag"')
+ })
+})
+
+describe('the environment read whole', () => {
+ const fields = { PORT: number('18473'), NODE_ENV: oneOf(['development', 'test'], 'development'), MODE: oneOf(['a', 'b'], 'a'), NAME: maybe }
+
+ it('is every variable\'s value, unknown variables left out', () => {
+ expect(parseEnv(fields, { PORT: '9', NODE_ENV: 'test', OTHER: 'x' })).toEqual({ ok: true, data: { PORT: 9, NODE_ENV: 'test', MODE: 'a', NAME: undefined } })
+ })
+
+ it('is every issue, by variable, when any has one', () => {
+ expect(parseEnv(fields, { NODE_ENV: 'prod', MODE: 'c' })).toEqual({
+ ok: false,
+ issues: { NODE_ENV: ['Invalid option: expected one of "development"|"test"'], MODE: ['Invalid option: expected one of "a"|"b"'] },
+ })
+ })
+
+ it('lets anything but an issue through: a bug is not a bad variable', () => {
+ expect(() => parseEnv({ BROKEN: () => { throw new TypeError('a bug') } }, {})).toThrow('a bug')
+ })
+})
diff --git a/cli/src/config/envFields.ts b/cli/src/config/envFields.ts
new file mode 100644
index 000000000..d9556990a
--- /dev/null
+++ b/cli/src/config/envFields.ts
@@ -0,0 +1,65 @@
+/**
+ * How the environment's variables become `env` (./env.ts): one small function per variable.
+ *
+ * The schema was zod's. Every process loads env.ts at start, harnessd's master and each service in its
+ * own process among them, and zod with a schema of a hundred variables cost each of them 15 to 19 MiB
+ * resident, against 3 for the master's own code (measured 2026-10-05). These are the few kinds of
+ * variable the schema had, with zod's meaning: a default stands in only for a variable that is unset (an
+ * empty one is kept), and a value that cannot be what its variable must be is an issue, reported with
+ * every other before the process exits.
+ */
+
+/** Why a variable's value cannot be used; collected per variable, as zod's `fieldErrors` were. */
+export class EnvIssue extends Error {}
+
+/** One variable: its value from the environment (undefined when unset) to what `env` holds. */
+export type EnvField<T> = (raw: string | undefined) => T
+
+/** A string, [fallback] when unset. */
+export const text = (fallback: string): EnvField<string> => (raw) => raw ?? fallback
+/** A string, or undefined when unset. */
+export const maybe: EnvField<string | undefined> = (raw) => raw
+/** A number (`Number`, so not a number is NaN, as before), [fallback] when unset. */
+export const number = (fallback: string): EnvField<number> => (raw) => Number(raw ?? fallback)
+/** A number, or undefined when unset. */
+export const maybeNumber: EnvField<number | undefined> = (raw) => raw === undefined ? undefined : Number(raw)
+/** True only for `true`, [fallback] when unset. */
+export const flag = (fallback: 'true' | 'false'): EnvField<boolean> => (raw) => (raw ?? fallback) === 'true'
+/** True for anything but `false`: on unless turned off. */
+export const unlessFalse: EnvField<boolean> = (raw) => raw !== 'false'
+/** One of [values], [fallback] when unset; anything else is an issue. */
+export function oneOf<const T extends string>(values: readonly T[], fallback: T): EnvField<T> {
+ return (raw) => {
+ const value = raw ?? fallback
+ if (!(values as readonly string[]).includes(value)) {
+ throw new EnvIssue(`Invalid option: expected one of ${values.map((one) => JSON.stringify(one)).join('|')}`)
+ }
+ return value as T
+ }
+}
+/** A string that matches [pattern], or undefined: unset, or a value that does not. */
+export const matching = (pattern: RegExp): EnvField<string | undefined> => (raw) => raw !== undefined && pattern.test(raw) ? raw : undefined
+/** A URL [accept] takes, trimmed, or undefined: unset, not a URL, or one it refuses. */
+export const url = (accept: (url: URL) => boolean): EnvField<string | undefined> => (raw) => {
+ if (raw === undefined) return undefined
+ const value = raw.trim()
+ try { return accept(new URL(value)) ? value : undefined } catch { return undefined }
+}
+
+export type EnvOf<F extends Record<string, EnvField<unknown>>> = { [K in keyof F]: ReturnType<F[K]> }
+
+/** [source] read through [fields]: every variable's value, or the issues, by variable, if any has one. */
+export function parseEnv<F extends Record<string, EnvField<unknown>>>(fields: F, source: NodeJS.ProcessEnv):
+ { ok: true; data: EnvOf<F> } | { ok: false; issues: Record<string, string[]> } {
+ const data: Record<string, unknown> = {}
+ const issues: Record<string, string[]> = {}
+ for (const [name, field] of Object.entries(fields)) {
+ try {
+ data[name] = field(source[name])
+ } catch (error) {
+ if (!(error instanceof EnvIssue)) throw error
+ issues[name] = [error.message]
+ }
+ }
+ return Object.keys(issues).length ? { ok: false, issues } : { ok: true, data: data as EnvOf<F> }
+}
diff --git a/cli/src/dsh/id.ts b/cli/src/dsh/id.ts
new file mode 100644
index 000000000..d4eb7c249
--- /dev/null
+++ b/cli/src/dsh/id.ts
@@ -0,0 +1,9 @@
+/**
+ * A domain-specific harness's id, `<owner>/<name>`.
+ *
+ * Apart from the manifest (./manifest.ts), which holds the zod schemas and builds them as it loads: the
+ * registry checks ids, and every process that reads the registry (the core, and the search and
+ * workspaces services in their own processes) would otherwise load and build those schemas too, about
+ * 8 MiB each, to read one pattern (measured 2026-10-05).
+ */
+export const DSH_ID_RE = /^[a-z0-9][a-z0-9-]{0,63}\/[a-z0-9][a-z0-9-]{0,63}$/
diff --git a/cli/src/dsh/manifest.ts b/cli/src/dsh/manifest.ts
index bb2a297d7..d7015f6e8 100644
--- a/cli/src/dsh/manifest.ts
+++ b/cli/src/dsh/manifest.ts
@@ -14,8 +14,9 @@ import { homedir } from 'node:os'
import { isAbsolute } from 'node:path'
import { z } from 'zod'
import { ENGINES } from '../engines/types.js'
+import { DSH_ID_RE } from './id.js'
-export const DSH_ID_RE = /^[a-z0-9][a-z0-9-]{0,63}\/[a-z0-9][a-z0-9-]{0,63}$/
+export { DSH_ID_RE }
export const DSH_MANIFEST_FILE = 'harness.json'
export const DEFAULT_VERDICT_PATH = '.harness/verdict.json'
export { compatibleHarnessEngines as dshSupportedEngines } from './compatibility.js'
diff --git a/cli/src/entry.ts b/cli/src/entry.ts
new file mode 100644
index 000000000..0099d3d14
--- /dev/null
+++ b/cli/src/entry.ts
@@ -0,0 +1,29 @@
+#!/usr/bin/env node
+/**
+ * The bundle's entry (build-bundle.mjs): which process this is decides what it loads.
+ *
+ * Every daemon process runs the one `cli.js` the updater downloads and swaps, and every one used to
+ * evaluate all of it at start: harnessd's master, each service in its own process and the CLI alike.
+ * Measured from the bundle at idle on 2026-10-05, the master took 160 MiB resident and each service 115
+ * to 160, so isolating the services cost more memory than everything else the daemon does. Here the
+ * master and the services load only their own modules: esbuild, bundling without code splitting, turns
+ * each dynamic import below into a module initialised the first time it is imported, so the code a
+ * process never imports is parsed but never run. Everything else, the core (`__run`) included, is the
+ * CLI as it always was.
+ *
+ * Nothing heavy may be imported statically here: it would be evaluated by every process.
+ */
+import { fileURLToPath } from 'node:url'
+
+const [, , command, name] = process.argv
+if (command === '__harnessd') {
+ // This file is the bundle: the master starts the core from it, as cli.ts's own `SCRIPT_PATH` names it
+ // for a master started from the sources, and itself and the services from the lean bundle it carries.
+ (await import('./masterProcess.js')).startMasterFromBundle(fileURLToPath(import.meta.url))
+} else if (command === '__harnessd-probe') {
+ process.exitCode = (await import('./masterProcess.js')).probeThisMaster()
+} else if (command === '__service') {
+ await (await import('./serviceProcess.js')).startServiceProcess(name)
+} else {
+ await import('./cli.js')
+}
diff --git a/cli/src/harnessd/AGENTS.md b/cli/src/harnessd/AGENTS.md
index 41a63ec79..a7fc83b52 100644
--- a/cli/src/harnessd/AGENTS.md
+++ b/cli/src/harnessd/AGENTS.md
@@ -22,4 +22,13 @@ It keeps the core and the services running, and nothing else.
job ends every agent. `stop()` signals the master instead. The desktop app does not use it yet.
6. **The master replaces itself only in the gap before a core starts** (`reexec.ts`): no core running,
services stopped and reaped first, in place so the pid stays. Never exec with a child running: it
- would lose its channel, and become an orphan or a zombie no one reaps.
+ would lose its channel, and become an orphan or a zombie no one reaps. A master started on cli.js
+ does it once more, before it has started anything: onto the lean bundle cli.js carries, so the
+ master parses its own code and not the whole CLI's (`../masterProcess.ts`).
+7. **The lean bundle is only ever an optimisation** (`leanBundle.ts`, `leanServices.ts`). A service is
+ started from it only while its files are the ones its master started with and cli.js is the bundle
+ they came from, and not after it died twice from it before beating; from cli.js otherwise. Other
+ masters (another build, a second `harness start`) may share the data folder: each claims the folder
+ it runs from, and one is removed only once no live master claims it. `lean-off` in the data folder
+ turns it off for masters launchd or systemd start. Never exec onto a file without checking it is
+ there: a failed exec cannot be caught (`reexec.ts`).
diff --git a/cli/src/harnessd/leanBundle.spec.ts b/cli/src/harnessd/leanBundle.spec.ts
new file mode 100644
index 000000000..c1b5ed487
--- /dev/null
+++ b/cli/src/harnessd/leanBundle.spec.ts
@@ -0,0 +1,331 @@
+import { spawn } from 'node:child_process'
+import { createHash } from 'node:crypto'
+import { chmodSync, existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, statSync, utimesSync, writeFileSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+import { brotliCompressSync } from 'node:zlib'
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
+import {
+ LEAN_ENTRY, LEAN_RECENT_MS, folderFingerprint, leanFingerprint, processAlive, readLeanBundle, releaseLeanClaim, writeLeanBundle, type LeanBundle,
+} from './leanBundle.js'
+
+// The real file system, with a rename a test can step in front of (another master's, at the same moment),
+// a read it can watch and a write it can fail (a disk that fills part-way).
+const fsHooks = vi.hoisted(() => ({
+ beforeRename: null as null | ((from: string, to: string) => void),
+ onRead: null as null | ((path: string) => void),
+ failWrite: null as null | ((path: string) => boolean),
+ beforeLstat: null as null | ((path: string) => void),
+}))
+const renames = { set before(step: null | ((from: string, to: string) => void)) { fsHooks.beforeRename = step } }
+vi.mock('node:fs', async (real) => {
+ const fs = await real<typeof import('node:fs')>()
+ return {
+ ...fs,
+ renameSync: (from: string, to: string) => {
+ const step = fsHooks.beforeRename
+ fsHooks.beforeRename = null
+ step?.(from, to)
+ fs.renameSync(from, to)
+ },
+ readFileSync: ((path: string, ...rest: unknown[]) => {
+ fsHooks.onRead?.(String(path))
+ return (fs.readFileSync as (...args: unknown[]) => unknown)(path, ...rest)
+ }) as typeof fs.readFileSync,
+ lstatSync: ((path: string, ...rest: unknown[]) => {
+ fsHooks.beforeLstat?.(String(path))
+ return (fs.lstatSync as (...args: unknown[]) => unknown)(path, ...rest)
+ }) as typeof fs.lstatSync,
+ writeFileSync: ((path: string, ...rest: unknown[]) => {
+ if (fsHooks.failWrite?.(String(path))) throw Object.assign(new Error('ENOSPC: no space left on device, write'), { code: 'ENOSPC' })
+ return (fs.writeFileSync as (...args: unknown[]) => unknown)(path, ...rest)
+ }) as typeof fs.writeFileSync,
+ }
+})
+
+// The writer the build uses (it cannot import TypeScript): what is read back here is what the release
+// carries.
+// @ts-expect-error — plain ESM with no declaration file, imported to hold the build to the runtime
+const { LEAN_MARKER, leanBlock } = await import('../../scripts/lib/leanBlock.mjs') as {
+ LEAN_MARKER: string
+ leanBlock: (files: Record<string, string | Uint8Array>) => string
+}
+
+const sha256 = (bytes: Buffer | string): string => createHash('sha256').update(bytes).digest('hex')
+const FILES = { [LEAN_ENTRY]: 'await import("./master-AB12.mjs")\n', 'master-AB12.mjs': 'console.log("the master — and only it")\n' }
+const CLI = '#!/usr/bin/env node\nconsole.log("the whole CLI")\n/*! a dependency\'s notice */\n'
+/** A block whose payload is [payload], as the build would write it if it wrote that. */
+const blockOf = (payload: string): string => `\n${LEAN_MARKER}${sha256(payload)}:${brotliCompressSync(Buffer.from(payload)).toString('base64')}*/\n`
+
+describe('the lean bundle cli.js carries', () => {
+ it('is read back as the build wrote it, file by file, with the sha256 of the whole cli.js it came from', () => {
+ const bundle = Buffer.from(CLI + leanBlock(FILES))
+ const lean = readLeanBundle(bundle)!
+ expect(Object.fromEntries([...lean.files].map(([name, code]) => [name, code.toString('utf8')]))).toEqual(FILES)
+ expect(lean.sha256).toBe(sha256(JSON.stringify(FILES)))
+ expect(lean.bundleSha256).toBe(sha256(bundle))
+ })
+
+ it('is none in a cli.js that carries none, or anything else that looks like its start', () => {
+ expect(readLeanBundle(Buffer.from(CLI))).toBeNull()
+ // The marker in the code, as a minified bundle folds the reader's own two pieces back together.
+ expect(readLeanBundle(Buffer.from(`const marker = "${LEAN_MARKER}"; /* a comment */ console.log(marker)\n`))).toBeNull()
+ expect(readLeanBundle(Buffer.from(`${CLI}${LEAN_MARKER}unterminated`))).toBeNull()
+ })
+
+ it('is none when it does not match its checksum, or cannot be unpacked', () => {
+ const block = leanBlock(FILES)
+ const [, sum, packed] = /:([0-9a-f]{64}):([^*]+)\*\//.exec(block)!
+ expect(readLeanBundle(Buffer.from(CLI + block.replace(sum, sha256('something else'))))).toBeNull()
+ expect(readLeanBundle(Buffer.from(CLI + block.replace(packed, Buffer.from('not brotli').toString('base64'))))).toBeNull()
+ expect(readLeanBundle(Buffer.from(`${CLI}${LEAN_MARKER}not-a-sum:${packed}*/\n`))).toBeNull()
+ expect(readLeanBundle(Buffer.from(`${CLI}${LEAN_MARKER}${sum}*/\n`))).toBeNull()
+ })
+
+ it('is none when its files are not files the master can start: no entry, a name that leaves the folder, not code', () => {
+ expect(readLeanBundle(Buffer.from(CLI + blockOf('not json')))).toBeNull()
+ expect(readLeanBundle(Buffer.from(CLI + blockOf('["a list"]')))).toBeNull()
+ expect(readLeanBundle(Buffer.from(CLI + blockOf('null')))).toBeNull()
+ expect(readLeanBundle(Buffer.from(CLI + leanBlock({ 'master-AB12.mjs': 'x' })))).toBeNull()
+ for (const name of ['../escape.mjs', 'sub/dir.mjs', '.claim-1.mjs', 'not-a-module.js']) {
+ expect(readLeanBundle(Buffer.from(CLI + leanBlock({ ...FILES, [name]: 'x' }))), name).toBeNull()
+ }
+ expect(readLeanBundle(Buffer.from(CLI + blockOf(JSON.stringify({ [LEAN_ENTRY]: 42 }))))).toBeNull()
+ })
+})
+
+describe('writing the lean bundle out', () => {
+ let dir: string
+ beforeEach(() => { dir = join(mkdtempSync(join(tmpdir(), 'lean-')), 'lean') })
+ afterEach(() => {
+ fsHooks.beforeRename = null
+ fsHooks.onRead = null
+ fsHooks.failWrite = null
+ fsHooks.beforeLstat = null
+ rmSync(join(dir, '..'), { recursive: true, force: true })
+ })
+ /** A cleaner's clock, past the minute it leaves anything that just changed alone. */
+ const later = () => Date.now() + LEAN_RECENT_MS + 1_000
+ const lean = (files: Record<string, string> = FILES): LeanBundle => ({
+ files: new Map(Object.entries(files).map(([name, code]) => [name, Buffer.from(code)])),
+ sha256: sha256(JSON.stringify(files)),
+ bundleSha256: 'b',
+ })
+ const folderOf = (bundle: LeanBundle): string => join(dir, bundle.sha256.slice(0, 16))
+ const alive = (...pids: number[]) => (pid: number) => pids.includes(pid)
+
+ it('writes its files into a folder of its own, for its owner alone, claimed by this master, and returns its entry', () => {
+ const bundle = lean()
+ const entry = writeLeanBundle(dir, bundle, { pid: 100, alive: alive(100) })
+ expect(entry).toBe(join(folderOf(bundle), LEAN_ENTRY))
+ expect(readdirSync(folderOf(bundle)).sort()).toEqual(['.claim-100', LEAN_ENTRY, 'master-AB12.mjs'])
+ for (const [name, code] of Object.entries(FILES)) {
+ expect(readFileSync(join(folderOf(bundle), name), 'utf8')).toBe(code)
+ expect(statSync(join(folderOf(bundle), name)).mode & 0o777).toBe(0o600)
+ }
+ expect(statSync(dir).mode & 0o777).toBe(0o700)
+ expect(statSync(folderOf(bundle)).mode & 0o777).toBe(0o700)
+ })
+
+ it('uses a folder that holds its bytes as it is, and replaces one that does not', () => {
+ const bundle = lean()
+ writeLeanBundle(dir, bundle, { pid: 100, alive: alive(100) })
+ const written = statSync(join(folderOf(bundle), LEAN_ENTRY)).mtimeMs
+ // Another master of the same build: the folder is shared, each claims it.
+ writeLeanBundle(dir, bundle, { pid: 200, alive: alive(100, 200) })
+ expect(statSync(join(folderOf(bundle), LEAN_ENTRY)).mtimeMs).toBe(written)
+ expect(readdirSync(folderOf(bundle)).filter((name) => name.startsWith('.claim')).sort()).toEqual(['.claim-100', '.claim-200'])
+ // Changed on disk, or a file gone: replaced whole, and the claims of masters gone with it.
+ writeFileSync(join(folderOf(bundle), 'master-AB12.mjs'), 'changed on disk')
+ writeLeanBundle(dir, bundle, { pid: 300, alive: alive(300) })
+ expect(readFileSync(join(folderOf(bundle), 'master-AB12.mjs'), 'utf8')).toBe(FILES['master-AB12.mjs'])
+ rmSync(join(folderOf(bundle), LEAN_ENTRY))
+ writeLeanBundle(dir, bundle, { pid: 300, alive: alive(300) })
+ expect(readdirSync(folderOf(bundle)).sort()).toEqual(['.claim-300', LEAN_ENTRY, 'master-AB12.mjs'])
+ expect(readdirSync(dir)).toEqual([bundle.sha256.slice(0, 16)])
+ })
+
+ it('takes the folder another master of the same build wrote while it was writing its own', () => {
+ // The two masters of a second `harness start`, at once: the folder appears between this master's
+ // look and its rename, written by the other, and the rename fails over it.
+ const bundle = lean()
+ renames.before = () => { writeLeanBundle(dir, bundle, { pid: 100, alive: alive(100) }) }
+ expect(writeLeanBundle(dir, bundle, { pid: 200, alive: alive(100, 200) })).toBe(join(folderOf(bundle), LEAN_ENTRY))
+ expect(readdirSync(folderOf(bundle)).sort()).toEqual(['.claim-100', '.claim-200', LEAN_ENTRY, 'master-AB12.mjs'])
+ // Its own write is gone: nothing but the one folder.
+ expect(readdirSync(dir)).toEqual([bundle.sha256.slice(0, 16)])
+ })
+
+ it('says why its write could not be put in place, and leaves the next master to clear it', () => {
+ const bundle = lean()
+ renames.before = () => { throw Object.assign(new Error('EXDEV: the rename failed'), { code: 'EXDEV' }) }
+ expect(() => writeLeanBundle(dir, bundle, { pid: 100, alive: alive(100) })).toThrow('EXDEV: the rename failed')
+ expect(existsSync(folderOf(bundle))).toBe(false)
+ writeLeanBundle(dir, bundle, { pid: 200, alive: alive(200) })
+ expect(readdirSync(dir)).toEqual([bundle.sha256.slice(0, 16)])
+ })
+
+ it('removes every folder no live master claims, and every write a crash cut short, but never one a live master runs from', () => {
+ const older = lean({ [LEAN_ENTRY]: 'older', 'master-OLD1.mjs': 'older' })
+ const another = lean({ [LEAN_ENTRY]: 'another build', 'master-OTH1.mjs': 'another' })
+ const unclaimed = lean({ [LEAN_ENTRY]: 'unclaimed' })
+ writeLeanBundle(dir, older, { pid: 100, alive: alive(100) })
+ writeLeanBundle(dir, another, { pid: 200, alive: alive(100, 200) })
+ writeLeanBundle(dir, unclaimed, { pid: 250, alive: alive(100, 200, 250) })
+ rmSync(join(folderOf(unclaimed), '.claim-250'))
+ mkdirSync(join(dir, `${'c'.repeat(16)}.4242.tmp`))
+ mkdirSync(join(dir, `${'d'.repeat(16)}.200.old`))
+ writeFileSync(join(dir, 'notes.txt'), 'not a lean bundle')
+ // This master (100) moves to a new build; 200 still runs from its own; nothing else is alive.
+ const bundle = lean()
+ writeLeanBundle(dir, bundle, { pid: 100, alive: alive(100, 200), now: later })
+ expect(readdirSync(dir).sort()).toEqual([
+ another.sha256.slice(0, 16), bundle.sha256.slice(0, 16), `${'d'.repeat(16)}.200.old`, 'notes.txt',
+ ].sort())
+ // Once that master is gone, its folder goes with the next master's start, and so does its scratch.
+ writeLeanBundle(dir, bundle, { pid: 300, alive: alive(300), now: later })
+ expect(readdirSync(dir).sort()).toEqual([bundle.sha256.slice(0, 16), 'notes.txt'])
+ expect(readdirSync(folderOf(bundle)).filter((name) => name.startsWith('.claim'))).toEqual(['.claim-300'])
+ })
+
+ it('leaves what it cannot remove, and what it cannot read, for a later master', () => {
+ const stuck = lean({ [LEAN_ENTRY]: 'stuck' })
+ writeLeanBundle(dir, stuck, { pid: 100, alive: alive(100) })
+ chmodSync(folderOf(stuck), 0o500)
+ // Named like a folder, but a file: no claims to read in it, so it goes.
+ writeFileSync(join(dir, 'e'.repeat(16)), 'not a folder')
+ try {
+ writeLeanBundle(dir, lean(), { pid: 200, alive: alive(200), now: later })
+ expect(readdirSync(dir).sort()).toEqual([lean().sha256.slice(0, 16), stuck.sha256.slice(0, 16)].sort())
+ } finally { chmodSync(folderOf(stuck), 0o700) }
+ })
+
+ it('leaves alone, for a minute, any folder or write that just changed: another master may be at it now', () => {
+ const unclaimed = lean({ [LEAN_ENTRY]: 'unclaimed' })
+ writeLeanBundle(dir, unclaimed, { pid: 250, alive: alive(250) })
+ rmSync(join(folderOf(unclaimed), '.claim-250'))
+ mkdirSync(join(dir, `${'c'.repeat(16)}.4242.tmp`))
+ const bundle = lean()
+ writeLeanBundle(dir, bundle, { pid: 100, alive: alive(100) })
+ expect(readdirSync(dir).sort()).toEqual([`${'c'.repeat(16)}.4242.tmp`, bundle.sha256.slice(0, 16), unclaimed.sha256.slice(0, 16)].sort())
+ writeLeanBundle(dir, bundle, { pid: 100, alive: alive(100), now: later })
+ expect(readdirSync(dir)).toEqual([bundle.sha256.slice(0, 16)])
+ })
+
+ it('claims a folder before it checks its bytes, and claims its own write before it puts it in place', () => {
+ // From the moment the bytes are found good there is no instant a cleaner could find the folder
+ // unclaimed: not between a check and a claim, nor between a rename and a claim.
+ const bundle = lean()
+ let inScratch: string[] = []
+ fsHooks.beforeRename = (from) => { inScratch = readdirSync(from) }
+ writeLeanBundle(dir, bundle, { pid: 100, alive: alive(100) })
+ expect(inScratch).toContain('.claim-100')
+ const claimedWhenRead: boolean[] = []
+ fsHooks.onRead = (path) => { if (path.startsWith(folderOf(bundle))) claimedWhenRead.push(existsSync(join(folderOf(bundle), '.claim-200'))) }
+ writeLeanBundle(dir, bundle, { pid: 200, alive: alive(100, 200) })
+ expect(claimedWhenRead.length).toBeGreaterThan(0)
+ expect(claimedWhenRead.every(Boolean)).toBe(true)
+ })
+
+ it('keeps the claims of the live masters that run from a folder it replaces', () => {
+ const bundle = lean()
+ writeLeanBundle(dir, bundle, { pid: 100, alive: alive(100) })
+ writeLeanBundle(dir, bundle, { pid: 200, alive: alive(100, 200) })
+ writeFileSync(join(folderOf(bundle), 'master-AB12.mjs'), 'changed on disk')
+ // 100 still runs from it; 200 is gone.
+ writeLeanBundle(dir, bundle, { pid: 300, alive: alive(100, 300) })
+ expect(readFileSync(join(folderOf(bundle), 'master-AB12.mjs'), 'utf8')).toBe(FILES['master-AB12.mjs'])
+ expect(readdirSync(folderOf(bundle)).filter((name) => name.startsWith('.claim')).sort()).toEqual(['.claim-100', '.claim-300'])
+ })
+
+ it('takes a claim made before this computer started for a dead master\'s, whoever has its pid now', () => {
+ const older = lean({ [LEAN_ENTRY]: 'older' })
+ writeLeanBundle(dir, older, { pid: 100, alive: alive(100) })
+ const bootedAt = Date.now() - 1_000
+ utimesSync(join(folderOf(older), '.claim-100'), new Date('2026-01-01'), new Date('2026-01-01'))
+ const bundle = lean()
+ writeLeanBundle(dir, bundle, { pid: 300, alive: alive(100, 300), now: later, bootedAt })
+ // pid 100 is alive again, but it is not the master that claimed the folder before the restart.
+ expect(readdirSync(dir)).toEqual([bundle.sha256.slice(0, 16)])
+ writeLeanBundle(dir, bundle, { pid: 100, alive: alive(100, 300), bootedAt })
+ utimesSync(join(folderOf(bundle), '.claim-100'), new Date('2026-01-01'), new Date('2026-01-01'))
+ writeLeanBundle(dir, bundle, { pid: 300, alive: alive(100, 300), bootedAt })
+ expect(readdirSync(folderOf(bundle)).filter((name) => name.startsWith('.claim'))).toEqual(['.claim-300'])
+ })
+
+ it('takes in its stride a write another master clears while this one looks at it', () => {
+ const gone = join(dir, `${'c'.repeat(16)}.4242.tmp`)
+ mkdirSync(gone, { recursive: true })
+ fsHooks.beforeLstat = (path) => { if (path === gone) rmSync(path, { recursive: true }) }
+ const bundle = lean()
+ writeLeanBundle(dir, bundle, { pid: 100, alive: alive(100) })
+ expect(readdirSync(dir)).toEqual([bundle.sha256.slice(0, 16)])
+ })
+
+ it('removes its write when the disk fills part-way, and says why', () => {
+ let writes = 0
+ fsHooks.failWrite = () => ++writes === 3
+ expect(() => writeLeanBundle(dir, lean(), { pid: 100, alive: alive(100) })).toThrow('ENOSPC')
+ expect(readdirSync(dir)).toEqual([])
+ })
+
+ it('gives up a master\'s claim as it exits', () => {
+ const entry = writeLeanBundle(dir, lean(), { pid: 100, alive: alive(100) })
+ releaseLeanClaim(entry, 100)
+ expect(readdirSync(folderOf(lean())).filter((name) => name.startsWith('.claim'))).toEqual([])
+ releaseLeanClaim(entry, 100)
+ releaseLeanClaim(join(dir, 'nowhere', LEAN_ENTRY))
+ })
+
+ it('says why when the folder cannot be written, and leaves nothing half-written in place', () => {
+ writeFileSync(join(dir, '..', 'blocked'), 'a file where the folder would go')
+ expect(() => writeLeanBundle(join(dir, '..', 'blocked'), lean())).toThrow()
+ mkdirSync(dir)
+ chmodSync(dir, 0o500)
+ try {
+ expect(() => writeLeanBundle(dir, lean(), { pid: 100, alive: alive(100) })).toThrow()
+ expect(readdirSync(dir)).toEqual([])
+ } finally { chmodSync(dir, 0o700) }
+ expect(existsSync(folderOf(lean()))).toBe(false)
+ })
+})
+
+describe('the fingerprint of a lean bundle', () => {
+ let dir: string
+ beforeEach(() => { dir = mkdtempSync(join(tmpdir(), 'lean-print-')) })
+ afterEach(() => rmSync(dir, { recursive: true, force: true }))
+ const bundle = (): LeanBundle => readLeanBundle(Buffer.from(CLI + leanBlock(FILES)))!
+
+ it('is the same for the bundle and the folder written from it, claims aside, and changes with any file', () => {
+ const entry = writeLeanBundle(join(dir, 'lean'), bundle(), { pid: 100, alive: () => true })
+ const folder = join(entry, '..')
+ expect(folderFingerprint(folder)).toBe(leanFingerprint(bundle()))
+ writeFileSync(join(folder, '.claim-200'), '')
+ expect(folderFingerprint(folder)).toBe(leanFingerprint(bundle()))
+ writeFileSync(join(folder, 'master-AB12.mjs'), 'changed')
+ expect(folderFingerprint(folder)).not.toBe(leanFingerprint(bundle()))
+ })
+
+ it('is none for a folder with no entry, or one that cannot be read', () => {
+ expect(folderFingerprint(join(dir, 'missing'))).toBeNull()
+ writeFileSync(join(dir, 'master-AB12.mjs'), 'x')
+ expect(folderFingerprint(dir)).toBeNull()
+ writeFileSync(join(dir, LEAN_ENTRY), 'x')
+ chmodSync(join(dir, LEAN_ENTRY), 0o000)
+ try { expect(folderFingerprint(dir)).toBeNull() } finally { chmodSync(join(dir, LEAN_ENTRY), 0o600) }
+ })
+})
+
+describe('whether a master is alive', () => {
+ it('is yes for a running process, no for one that has exited', async () => {
+ expect(processAlive(process.pid)).toBe(true)
+ const child = spawn(process.execPath, ['-e', ''])
+ await new Promise((done) => child.once('exit', done))
+ expect(processAlive(child.pid!)).toBe(false)
+ })
+
+ it('is yes for one this user may not signal: a process of another user is still running', () => {
+ // pid 1 is launchd or init, never this user's.
+ expect(processAlive(1)).toBe(true)
+ })
+})
diff --git a/cli/src/harnessd/leanBundle.ts b/cli/src/harnessd/leanBundle.ts
new file mode 100644
index 000000000..9c8ed5a35
--- /dev/null
+++ b/cli/src/harnessd/leanBundle.ts
@@ -0,0 +1,233 @@
+/**
+ * The lean bundle a release's cli.js carries, written out for the master and the services to run from.
+ *
+ * Node parses the whole file a process is started on. Started on the 4.4 MB cli.js, harnessd's master
+ * and each service paid about 45 MiB for that alone, at idle, before running a line of their own code
+ * (measured 2026-10-05). The build therefore bundles the master and the services a second time on their
+ * own, split into files so that each process parses only the ones its own code is in, and appends those
+ * files to cli.js as a comment, which Node only skims (scripts/lib/leanBlock.mjs). This reads them back
+ * and writes them where the master can start processes from them: `lean/<sha>/` in the data folder.
+ *
+ * Nothing here is needed for the daemon to run. A cli.js without the block (one built from the
+ * sources, or a test's), a block that does not match its checksum, or a folder that cannot be written
+ * leaves every process running from cli.js, as before; and a service is started from cli.js whenever
+ * its master's lean files are gone or changed, or cli.js is no longer the bundle they came from
+ * (./leanServices.ts).
+ *
+ * Several masters can share one data folder (a second `harness start`, two builds on one computer), and
+ * a master restarts its services from its folder for as long as it lives. So each master claims the
+ * folder it runs from (`.claim-<pid>`) before it is in place or checked, gives the claim up as it exits
+ * cleanly, and a folder is removed only once no live master claims it and it has not changed for a
+ * minute. A claim older than the computer's last start is a dead master's, whatever has its pid now.
+ */
+import { createHash } from 'node:crypto'
+import { lstatSync, mkdirSync, readdirSync, readFileSync, renameSync, rmSync, writeFileSync } from 'node:fs'
+import { uptime } from 'node:os'
+import { dirname, join } from 'node:path'
+import { brotliDecompressSync } from 'node:zlib'
+
+/** As scripts/lib/leanBlock.mjs writes it. Built in two pieces, so this file's own text never matches. */
+const MARKER = Buffer.from('/*@harness-' + 'lean:')
+const END = Buffer.from('*/')
+/** The file the master and the services start on (src/leanEntry.ts); the others are what it imports. */
+export const LEAN_ENTRY = 'harnessd.mjs'
+const FILE_NAME = /^[A-Za-z0-9_-][A-Za-z0-9_.-]*\.mjs$/
+const FOLDER = /^[0-9a-f]{16}$/
+const CLAIM = /^\.claim-(\d+)$/
+const SCRATCH = /^[0-9a-f]{16}\.(\d+)\.(tmp|old)$/
+
+export interface LeanBundle {
+ /** Its files, by name: the entry and the chunks it imports. */
+ files: ReadonlyMap<string, Buffer>
+ /** The sha256 of its files, as the build recorded it. */
+ sha256: string
+ /** The sha256 of the whole cli.js it was read from: the bundle the master runs, for re-execution. */
+ bundleSha256: string
+}
+
+const sha256 = (bytes: Buffer): string => createHash('sha256').update(bytes).digest('hex')
+
+/** The lean bundle [bundle] carries; null when it carries none, or one that does not match its checksum. */
+export function readLeanBundle(bundle: Buffer): LeanBundle | null {
+ const start = bundle.lastIndexOf(MARKER)
+ if (start < 0) return null
+ const end = bundle.indexOf(END, start + MARKER.length)
+ // The block is the last thing in the file: anything else that looks like its start is not it.
+ if (end < 0 || bundle.subarray(end + END.length).toString('latin1').trim() !== '') return null
+ const body = bundle.subarray(start + MARKER.length, end).toString('latin1')
+ const colon = body.indexOf(':')
+ const expected = body.slice(0, colon)
+ if (colon < 0 || !/^[0-9a-f]{64}$/.test(expected)) return null
+ let payload: Buffer
+ try { payload = brotliDecompressSync(Buffer.from(body.slice(colon + 1), 'base64')) } catch { return null }
+ if (sha256(payload) !== expected) return null
+ let named: unknown
+ try { named = JSON.parse(payload.toString('utf8')) } catch { return null }
+ if (!named || typeof named !== 'object' || Array.isArray(named)) return null
+ const files = new Map<string, Buffer>()
+ for (const [name, code] of Object.entries(named)) {
+ // Names become paths: nothing that could leave the folder, or be taken for a claim or scratch.
+ if (!FILE_NAME.test(name) || typeof code !== 'string') return null
+ files.set(name, Buffer.from(code, 'utf8'))
+ }
+ if (!files.has(LEAN_ENTRY)) return null
+ return { files, sha256: expected, bundleSha256: sha256(bundle) }
+}
+
+/** Whether a process is alive; one of another user's is (EPERM). */
+export function processAlive(pid: number): boolean {
+ try { process.kill(pid, 0); return true } catch (error) { return (error as NodeJS.ErrnoException).code === 'EPERM' }
+}
+
+/**
+ * A fingerprint of the lean bundle's code, file by file in name order, which a folder written from it
+ * reproduces (`folderFingerprint`): how a master knows, before each service it starts, that the files it
+ * would start it from are still the ones it was started with.
+ */
+export function leanFingerprint(lean: LeanBundle): string {
+ const hash = createHash('sha256')
+ for (const name of [...lean.files.keys()].sort()) hash.update(`${name}\0${sha256(lean.files.get(name)!)}\n`)
+ return hash.digest('hex')
+}
+
+/** The fingerprint of the lean bundle written in [folder] (claims aside); null when it holds none. */
+export function folderFingerprint(folder: string): string | null {
+ const names = safeList(folder).filter((name) => FILE_NAME.test(name)).sort()
+ if (!names.includes(LEAN_ENTRY)) return null
+ const hash = createHash('sha256')
+ try {
+ for (const name of names) hash.update(`${name}\0${sha256(readFileSync(join(folder, name)))}\n`)
+ } catch {
+ return null
+ }
+ return hash.digest('hex')
+}
+
+/** The claim [pid] holds on the lean bundle whose entry is [entry], given up as its master exits. */
+export function releaseLeanClaim(entry: string, pid: number = process.pid): void {
+ removeQuietly(join(dirname(entry), `.claim-${pid}`))
+}
+
+/** What a cleaner leaves alone: a folder or file changed this recently may be one a master is writing,
+ * or has just claimed, in another process at this moment. */
+export const LEAN_RECENT_MS = 60_000
+
+export interface LeanWrite {
+ /** This master: what its claim is under. */
+ pid?: number
+ alive?: (pid: number) => boolean
+ /** The wall clock, ms. */
+ now?: () => number
+ /** When this computer started, ms: a claim older than that was made by a process that is gone,
+ * whatever runs under its pid now. */
+ bootedAt?: number
+}
+
+/** Whether [folder] holds every file of [lean], byte for byte. */
+function holds(folder: string, lean: LeanBundle): boolean {
+ for (const [name, code] of lean.files) {
+ try { if (!readFileSync(join(folder, name)).equals(code)) return false } catch { return false }
+ }
+ return true
+}
+
+const removeQuietly = (path: string): void => { try { rmSync(path, { recursive: true, force: true }) } catch { /* the next master tries again */ } }
+const changedAt = (path: string): number => { try { return lstatSync(path).mtimeMs } catch { return 0 } }
+
+/** The pids that claim [folder], with when each claimed it. */
+function claimsIn(folder: string): Array<{ pid: number; at: number }> {
+ return safeList(folder).flatMap((file) => {
+ const claim = CLAIM.exec(file)
+ return claim ? [{ pid: Number(claim[1]), at: changedAt(join(folder, file)) }] : []
+ })
+}
+
+/** Claim [folder] for [claim], then check it holds [lean]: from the moment its bytes are found good, a
+ * cleaner in another process sees the claim. False when there is no folder, or it does not hold them. */
+function claimed(folder: string, claim: string, lean: LeanBundle): boolean {
+ try { writeFileSync(join(folder, claim), '', { mode: 0o600 }) } catch { return false }
+ return holds(folder, lean)
+}
+
+/**
+ * Write [lean] into `[dir]/<sha>/`, claim it for this master and return the path of its entry. A folder
+ * already there is used only if it holds the bundle's bytes; one that does not is replaced whole, with
+ * the claims of the live masters that run from it. Every other lean folder no live master claims, and
+ * every write a crash cut short, is removed, unless it changed in the last minute: a process running
+ * from one has read it already, and the next master writes its own. Throws when the folder cannot be
+ * written, and leaves nothing half-written.
+ */
+export function writeLeanBundle(dir: string, lean: LeanBundle, options: LeanWrite = {}): string {
+ const pid = options.pid ?? process.pid
+ const alive = options.alive ?? processAlive
+ const now = options.now ?? Date.now
+ const bootedAt = options.bootedAt ?? Date.now() - uptime() * 1000
+ // A claim made before this computer started is a dead master's, whatever runs under its pid now.
+ const live = (claim: { pid: number; at: number }): boolean => claim.at >= bootedAt && alive(claim.pid)
+ const claim = `.claim-${pid}`
+ const name = lean.sha256.slice(0, 16)
+ const folder = join(dir, name)
+ mkdirSync(dir, { recursive: true, mode: 0o700 })
+ if (!claimed(folder, claim, lean)) {
+ // Written whole, claimed, then renamed into place: a folder there is always complete and never
+ // unclaimed, so two masters of one build starting at once (a second `harness start`) never see each
+ // other's half, and no cleaner takes it between the rename and the claim.
+ const scratch = join(dir, `${name}.${pid}.tmp`)
+ removeQuietly(scratch)
+ try {
+ mkdirSync(scratch, { mode: 0o700 })
+ for (const [file, code] of lean.files) writeFileSync(join(scratch, file), code, { mode: 0o600 })
+ writeFileSync(join(scratch, claim), '', { mode: 0o600 })
+ } catch (error) {
+ // A full disk, part-way: nothing half-written is left for the next master to trip on.
+ removeQuietly(scratch)
+ throw error
+ }
+ try {
+ renameSync(scratch, folder)
+ } catch (error) {
+ // Not over a folder with files in it. Another master of this build, written first: the same bytes.
+ if (claimed(folder, claim, lean)) {
+ removeQuietly(scratch)
+ } else {
+ // One that changed on disk is replaced, and keeps the claims of the masters still running from
+ // it. One starting a service from it in that instant finds no file and starts it from cli.js.
+ for (const other of claimsIn(folder)) {
+ if (other.pid !== pid && live(other)) writeFileSync(join(scratch, `.claim-${other.pid}`), '', { mode: 0o600 })
+ }
+ const aside = join(dir, `${name}.${pid}.old`)
+ try {
+ renameSync(folder, aside)
+ renameSync(scratch, folder)
+ } catch {
+ removeQuietly(scratch)
+ throw error
+ } finally {
+ removeQuietly(aside)
+ }
+ }
+ }
+ }
+ const settled = (path: string): boolean => now() - changedAt(path) >= LEAN_RECENT_MS
+ for (const entry of safeList(dir)) {
+ const path = join(dir, entry)
+ const scratch = SCRATCH.exec(entry)
+ if (scratch) {
+ if (Number(scratch[1]) === pid || (!alive(Number(scratch[1])) && settled(path))) removeQuietly(path)
+ continue
+ }
+ if (!FOLDER.test(entry)) continue
+ const claims = claimsIn(path)
+ if (entry === name) {
+ for (const other of claims) if (other.pid !== pid && !live(other)) removeQuietly(join(folder, `.claim-${other.pid}`))
+ } else if (!claims.some((other) => other.pid !== pid && live(other)) && settled(path)) {
+ // A claim of this master's own on another folder is one it held before it re-executed on this one.
+ removeQuietly(path)
+ }
+ }
+ return join(folder, LEAN_ENTRY)
+}
+
+function safeList(dir: string): string[] {
+ try { return readdirSync(dir) } catch { return [] }
+}
diff --git a/cli/src/harnessd/leanServices.spec.ts b/cli/src/harnessd/leanServices.spec.ts
new file mode 100644
index 000000000..95cc8766c
--- /dev/null
+++ b/cli/src/harnessd/leanServices.spec.ts
@@ -0,0 +1,124 @@
+import { describe, expect, it } from 'vitest'
+import { LEAN_EARLY_DEATHS, leanServices, type LeanServicesDeps } from './leanServices.js'
+import type { CoreHandle } from './supervisor.js'
+
+const CLI = '/cli/cli.js'
+const LEAN = '/data/lean/aaaa/harnessd.mjs'
+
+function fakeProcess() {
+ const messages: Array<(message: unknown) => void> = []
+ const exits: Array<(code: number | null, signal: NodeJS.Signals | null) => void> = []
+ const handle = {
+ pid: 1, send: () => {}, kill: () => {},
+ onMessage: (listener: (message: unknown) => void) => { messages.push(listener) },
+ onExit: (listener: (code: number | null, signal: NodeJS.Signals | null) => void) => { exits.push(listener) },
+ } as unknown as CoreHandle
+ return {
+ handle,
+ beat: () => { for (const listener of messages) listener({ type: 'harnessd:heartbeat', rssBytes: 1, heapUsedBytes: 1 }) },
+ say: (message: unknown) => { for (const listener of messages) listener(message) },
+ exit: (code: number | null, signal: NodeJS.Signals | null = null) => { for (const listener of exits) listener(code, signal) },
+ }
+}
+
+function make(over: Partial<LeanServicesDeps> = {}) {
+ const logs: string[] = []
+ const world = { fingerprint: 'f' as string | null, same: true, exists: true }
+ const lean = leanServices({
+ scriptPath: CLI, leanPath: LEAN, leanFingerprint: 'f',
+ folderFingerprint: (folder) => { expect(folder).toBe('/data/lean/aaaa'); return world.fingerprint },
+ exists: () => world.exists,
+ sameBundle: () => world.same,
+ log: (line) => logs.push(line),
+ ...over,
+ })
+ return { lean, logs, world }
+}
+
+describe('the file a service starts from', () => {
+ it('is the lean bundle while its files are the ones the master started with, and cli.js is the bundle they came from', () => {
+ const { lean, logs } = make()
+ expect(lean.scriptFor('search')).toBe(LEAN)
+ expect(logs).toEqual([])
+ })
+
+ it('is cli.js when there is no lean bundle, or it is cli.js itself (from the sources)', () => {
+ expect(make({ leanPath: undefined }).lean.scriptFor('search')).toBe(CLI)
+ expect(make({ leanPath: CLI }).lean.scriptFor('search')).toBe(CLI)
+ })
+
+ it('is cli.js once the lean bundle is gone or changed, saying why once, and the lean bundle again once it is back', () => {
+ // A folder gone from under a master that lives for weeks failed every restart with MODULE_NOT_FOUND.
+ const { lean, logs, world } = make()
+ world.fingerprint = null
+ expect(lean.scriptFor('search')).toBe(CLI)
+ expect(lean.scriptFor('viewers')).toBe(CLI)
+ world.fingerprint = 'changed'
+ expect(lean.scriptFor('search')).toBe(CLI)
+ expect(logs).toEqual([
+ `[harnessd] the lean bundle ${LEAN} cannot be used (it is gone): services start from ${CLI}`,
+ `[harnessd] the lean bundle ${LEAN} cannot be used (its files changed): services start from ${CLI}`,
+ ])
+ world.fingerprint = 'f'
+ expect(lean.scriptFor('search')).toBe(LEAN)
+ world.fingerprint = null
+ expect(lean.scriptFor('search')).toBe(CLI)
+ expect(logs).toHaveLength(3)
+ })
+
+ it('is cli.js once cli.js is another bundle than the one the lean bundle came from', () => {
+ // An update this master did not re-execute on (no `process.execve`, or a re-execution kept back): the
+ // services must not run the old lean code against the new core.
+ const { lean, logs, world } = make()
+ world.same = false
+ expect(lean.scriptFor('search')).toBe(CLI)
+ expect(logs).toEqual([`[harnessd] the lean bundle ${LEAN} cannot be used (${CLI} is no longer the bundle it came from): services start from ${CLI}`])
+ })
+
+ it('checks only that the entry is there when it has no fingerprint to check against', () => {
+ const { lean, world } = make({ leanFingerprint: undefined })
+ expect(lean.scriptFor('search')).toBe(LEAN)
+ world.exists = false
+ expect(lean.scriptFor('search')).toBe(CLI)
+ })
+
+ it('is cli.js for a service that died twice running from the lean bundle before it ever beat', () => {
+ const { lean, logs } = make()
+ for (let death = 1; death <= LEAN_EARLY_DEATHS; death++) {
+ expect(lean.scriptFor('search')).toBe(LEAN)
+ const child = fakeProcess()
+ lean.started('search', LEAN, child.handle)
+ child.say({ type: 'not a beat' })
+ child.exit(1)
+ }
+ expect(logs).toEqual([`[harnessd] service search died ${LEAN_EARLY_DEATHS} times from the lean bundle before it beat: it starts from ${CLI} from now on`])
+ expect(lean.scriptFor('search')).toBe(CLI)
+ // Only that service: the others still start from the lean bundle.
+ expect(lean.scriptFor('viewers')).toBe(LEAN)
+ })
+
+ it('counts only deaths in a row before a beat, and never the master\'s own stop', () => {
+ const { lean } = make()
+ const early = fakeProcess()
+ lean.started('search', LEAN, early.handle)
+ early.exit(1)
+ // A process that beat, then crashed: not the bundle's fault, and the count starts again.
+ const beat = fakeProcess()
+ lean.started('search', LEAN, beat.handle)
+ beat.beat()
+ beat.beat()
+ beat.exit(1)
+ const stopped = fakeProcess()
+ lean.started('search', LEAN, stopped.handle)
+ stopped.exit(null, 'SIGTERM')
+ const again = fakeProcess()
+ lean.started('search', LEAN, again.handle)
+ again.exit(null, 'SIGKILL')
+ expect(lean.scriptFor('search')).toBe(LEAN)
+ // One started from cli.js is not watched.
+ const fromCli = fakeProcess()
+ lean.started('search', CLI, fromCli.handle)
+ fromCli.exit(1)
+ expect(lean.scriptFor('search')).toBe(LEAN)
+ })
+})
diff --git a/cli/src/harnessd/leanServices.ts b/cli/src/harnessd/leanServices.ts
new file mode 100644
index 000000000..18ca3ec69
--- /dev/null
+++ b/cli/src/harnessd/leanServices.ts
@@ -0,0 +1,87 @@
+/**
+ * Which file a service's process is started from: the lean bundle (./leanBundle.ts), only while it is
+ * purely an optimisation, and cli.js otherwise.
+ *
+ * The lean bundle is code the master wrote into the data folder as it started, and a master lives for
+ * weeks. Started from it unconditionally, a service failed on every restart once the folder went (a
+ * cleaner, a person tidying up): MODULE_NOT_FOUND until the master itself restarted. And once an update
+ * installed a new cli.js that this master did not re-execute on (a Node without `process.execve`, or a
+ * re-execution it kept back), every service ran the old lean code against the new core for the rest of
+ * the master's life. So before each start: the lean bundle's files must still be the ones this master
+ * was started with, and cli.js must still be the bundle they came from; and a service that dies twice
+ * running from it before it ever beats starts from cli.js from then on. cli.js is always there to fall
+ * back on: it is what the core runs.
+ */
+import { dirname } from 'node:path'
+import { isCoreMessage } from './protocol.js'
+import type { CoreHandle } from './supervisor.js'
+
+/** This many deaths in a row before a first heartbeat, started from the lean bundle, and a service is
+ * started from cli.js for the rest of this master's life. */
+export const LEAN_EARLY_DEATHS = 2
+
+export interface LeanServicesDeps {
+ /** cli.js (or the sources): what the core runs. */
+ scriptPath: string
+ /** The lean bundle's entry, when this master has one. */
+ leanPath?: string
+ /** What the lean bundle's folder should fingerprint to (./leanBundle.ts `leanFingerprint`); without
+ * it, only that the entry is there is checked. */
+ leanFingerprint?: string
+ /** The fingerprint of the lean bundle in a folder, null when it holds none (./leanBundle.ts). */
+ folderFingerprint(folder: string): string | null
+ exists(path: string): boolean
+ /** Whether the bundle on disk is still the one this master runs: the one its lean bundle came from. */
+ sameBundle(): boolean
+ log(line: string): void
+}
+
+export interface LeanServices {
+ /** The file to start service [name] from now. */
+ scriptFor(name: string): string
+ /** Watch a process started from [script] for a death before its first heartbeat. */
+ started(name: string, script: string, handle: CoreHandle): void
+}
+
+export function leanServices(deps: LeanServicesDeps): LeanServices {
+ const { leanPath, scriptPath } = deps
+ const earlyDeaths = new Map<string, number>()
+ let said: string | null = null
+ /** Why the lean bundle cannot be used now; null when it can. */
+ const unusable = (lean: string): string | null => {
+ if (!deps.sameBundle()) return `${scriptPath} is no longer the bundle it came from`
+ if (deps.leanFingerprint === undefined) return deps.exists(lean) ? null : 'it is gone'
+ const found = deps.folderFingerprint(dirname(lean))
+ return found === deps.leanFingerprint ? null : found === null ? 'it is gone' : 'its files changed'
+ }
+ return {
+ scriptFor: (name) => {
+ if (!leanPath || leanPath === scriptPath) return scriptPath
+ if ((earlyDeaths.get(name) ?? 0) >= LEAN_EARLY_DEATHS) return scriptPath
+ const why = unusable(leanPath)
+ if (why === null) { said = null; return leanPath }
+ // Said once per reason, not on every restart.
+ if (said !== why) deps.log(`[harnessd] the lean bundle ${leanPath} cannot be used (${why}): services start from ${scriptPath}`)
+ said = why
+ return scriptPath
+ },
+ started: (name, script, handle) => {
+ if (script === scriptPath) return
+ let beat = false
+ handle.onMessage((message) => {
+ if (beat || !isCoreMessage(message) || message.type !== 'harnessd:heartbeat') return
+ beat = true
+ earlyDeaths.delete(name)
+ })
+ handle.onExit((_code, signal) => {
+ // SIGTERM is the master's own stop (the daemon stopping, a memory restart): not the bundle's fault.
+ if (beat || signal === 'SIGTERM') return
+ const deaths = (earlyDeaths.get(name) ?? 0) + 1
+ earlyDeaths.set(name, deaths)
+ if (deaths === LEAN_EARLY_DEATHS) {
+ deps.log(`[harnessd] service ${name} died ${deaths} times from the lean bundle before it beat: it starts from ${scriptPath} from now on`)
+ }
+ })
+ },
+ }
+}
diff --git a/cli/src/harnessd/master.spec.ts b/cli/src/harnessd/master.spec.ts
index ad47a5858..a90baca9d 100644
--- a/cli/src/harnessd/master.spec.ts
+++ b/cli/src/harnessd/master.spec.ts
@@ -1,5 +1,5 @@
import { EventEmitter } from 'node:events'
-import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
+import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { spawn, type ChildProcess } from 'node:child_process'
@@ -8,6 +8,7 @@ import {
coreExecArgv, coreHandle, describeMasterStatus, heapLimitInArgv, masterDefaults, onProcessSignal, probeMaster, processExecve, processExit,
readStatusFile, runMaster, supervisorOptions, trimLogEvery, writeStatusFile, type MasterStatusFile,
} from './master.js'
+import { folderFingerprint } from './leanBundle.js'
import { PROBE_ANSWER, RESUME_ENV, decodeResume, encodeResume, fingerprint, readMarker, writeMarker } from './reexec.js'
import { DEFAULT_SUPERVISOR_OPTIONS, type ResumeState } from './supervisor.js'
@@ -256,7 +257,7 @@ describe('runMaster', () => {
pidFile,
restoreUpdate: () => updates.push('restore'),
confirmUpdate: () => updates.push('confirm'),
- env: { ...process.env, HARNESSD_INITIAL_BACKOFF_MS: '10' },
+ env: { ...process.env, HARNESSD_SERVICES: 'none', HARNESSD_INITIAL_BACKOFF_MS: '10' },
exit: (code) => exits.push(code),
onSignal: (signal, listener) => signals.set(signal, listener),
})
@@ -289,7 +290,7 @@ describe('runMaster', () => {
runMaster({
nodePath: process.execPath, execArgv: [], scriptPath: core, pidFile, statusFile, logFile: join(dir, 'harness.log'),
restoreUpdate: () => {}, confirmUpdate: () => {},
- env: { ...process.env, HARNESSD_HEAP_LIMIT_MIB: '300' },
+ env: { ...process.env, HARNESSD_SERVICES: 'none', HARNESSD_HEAP_LIMIT_MIB: '300' },
exit: (code) => exits.push(code), onSignal: (signal, listener) => signals.set(signal, listener),
})
await until('the core to be ready', () => readStatusFile(statusFile, process.pid)?.state === 'running')
@@ -317,7 +318,7 @@ describe('runMaster', () => {
runMaster({
nodePath: process.execPath, execArgv: [], scriptPath: core, pidFile, statusFile,
restoreUpdate: () => {}, confirmUpdate: () => {},
- env: { ...process.env, HARNESSD_PLATFORM: 'launchd' },
+ env: { ...process.env, HARNESSD_SERVICES: 'none', HARNESSD_PLATFORM: 'launchd' },
exit: (code) => exits.push(code), onSignal: (signal, listener) => signals.set(signal, listener),
})
await until('the core to be ready', () => readStatusFile(statusFile, process.pid)?.state === 'running')
@@ -366,6 +367,94 @@ describe('runMaster', () => {
expect(exits).toEqual([0])
})
+ it('starts the services from the lean bundle when given one, and the core from the CLI entry', async () => {
+ // The lean bundle cli.js carries (./leanBundle.ts): each service parses its own code, not the CLI's.
+ const pidFile = join(dir, 'adapter.pid')
+ const seen = join(dir, 'seen')
+ const body = (who: string) => `
+ const { appendFileSync } = require('node:fs')
+ const role = process.argv[2] === '__service' ? 'service:' + process.argv[3] : process.argv[2]
+ appendFileSync(${JSON.stringify(seen)}, JSON.stringify({ role, from: ${JSON.stringify(who)} }) + '\\n')
+ if (role === '__run') { process.send({ type: 'harnessd:bound', protocol: 2, port: 1 }); process.send({ type: 'harnessd:ready' }) }
+ setInterval(() => process.send({ type: 'harnessd:heartbeat', rssBytes: 1, heapUsedBytes: 1, loopDelayMs: 0 }), 50)
+ process.on('SIGTERM', () => process.exit(0))
+ `
+ const cli = join(dir, 'cli.cjs')
+ const lean = join(dir, 'lean.cjs')
+ writeFileSync(cli, body('cli'))
+ writeFileSync(lean, body('lean'))
+ const lines: string[] = []
+ const log = vi.spyOn(console, 'log').mockImplementation((line: string) => { lines.push(line) })
+ const exits: number[] = []
+ const signals = new Map<string, () => void>()
+ try {
+ runMaster({
+ nodePath: process.execPath, execArgv: [], scriptPath: cli, serviceScriptPath: lean, pidFile,
+ restoreUpdate: () => {}, confirmUpdate: () => {},
+ env: { ...process.env, HARNESSD_SERVICES: 'search' },
+ exit: (code) => exits.push(code), onSignal: (signal, listener) => signals.set(signal, listener),
+ })
+ const roles = (): Array<Record<string, string>> => existsSync(seen) ? readFileSync(seen, 'utf8').trim().split('\n').map((line) => JSON.parse(line)) : []
+ await until('the core and the service', () => roles().length >= 2)
+ expect(roles().sort((a, b) => a.role.localeCompare(b.role))).toEqual([{ role: '__run', from: 'cli' }, { role: 'service:search', from: 'lean' }])
+ expect(lines.some((line) => line.endsWith(`[harnessd] services run from ${lean}`))).toBe(true)
+ signals.get('SIGTERM')!()
+ await until('the master to finish', () => exits.length > 0)
+ } finally { log.mockRestore() }
+ })
+
+ it('starts a service from cli.js once its lean bundle is gone, or cli.js is no longer the bundle it came from', async () => {
+ // A master lives for weeks: a lean folder gone from under it failed every restart of every service
+ // with MODULE_NOT_FOUND, and after an update it did not re-execute on, the services ran the old lean
+ // code against the new core. The lean bundle is only an optimisation (./leanServices.ts).
+ const pidFile = join(dir, 'adapter.pid')
+ const seen = join(dir, 'seen')
+ // Each service process beats, then ends: the master starts it again at once, from wherever it may.
+ const body = (who: string) => `
+ const { appendFileSync } = require('node:fs')
+ const role = process.argv[2] === '__service' ? 'service:' + process.argv[3] : process.argv[2]
+ appendFileSync(${JSON.stringify(seen)}, JSON.stringify({ role, from: ${JSON.stringify(who)} }) + '\\n')
+ if (role === '__run') { process.send({ type: 'harnessd:bound', protocol: 2, port: 1 }); process.send({ type: 'harnessd:ready' }) }
+ process.send({ type: 'harnessd:heartbeat', rssBytes: 1, heapUsedBytes: 1, loopDelayMs: 0 })
+ setInterval(() => process.send({ type: 'harnessd:heartbeat', rssBytes: 1, heapUsedBytes: 1, loopDelayMs: 0 }), 50)
+ if (role !== '__run') setTimeout(() => process.exit(1), 100)
+ process.on('SIGTERM', () => process.exit(0))
+ `
+ const cli = join(dir, 'cli.cjs')
+ const leanDir = join(dir, 'lean')
+ const lean = join(leanDir, 'harnessd.mjs')
+ writeFileSync(cli, body('cli'))
+ mkdirSync(leanDir)
+ writeFileSync(lean, body('lean').replace("require('node:fs')", "await import('node:fs')"))
+ const lines: string[] = []
+ const log = vi.spyOn(console, 'log').mockImplementation((line: string) => { lines.push(line) })
+ const exits: number[] = []
+ const signals = new Map<string, () => void>()
+ const services = (): string[] => (existsSync(seen) ? readFileSync(seen, 'utf8').trim().split('\n').map((line) => JSON.parse(line)) : [])
+ .filter((line) => line.role === 'service:search').map((line) => line.from)
+ try {
+ runMaster({
+ nodePath: process.execPath, execArgv: [], scriptPath: cli, serviceScriptPath: lean, leanFingerprint: folderFingerprint(leanDir)!, pidFile,
+ restoreUpdate: () => {}, confirmUpdate: () => {},
+ env: { ...process.env, HARNESSD_SERVICES: 'search', HARNESSD_SERVICE_INITIAL_BACKOFF_MS: '0', HARNESSD_SERVICE_MAX_BACKOFF_MS: '0', HARNESSD_SERVICE_PARK_CRASHES: '1000' },
+ exit: (code) => exits.push(code), onSignal: (signal, listener) => signals.set(signal, listener),
+ })
+ await until('a service from the lean bundle', () => services().includes('lean'))
+ rmSync(leanDir, { recursive: true })
+ await until('a service from cli.js', () => services().at(-1) === 'cli')
+ expect(lines.some((line) => line.endsWith(`[harnessd] the lean bundle ${lean} cannot be used (it is gone): services start from ${cli}`))).toBe(true)
+ // Back as it was, and then cli.js replaced by an update this master runs no code of.
+ mkdirSync(leanDir)
+ writeFileSync(lean, body('lean').replace("require('node:fs')", "await import('node:fs')"))
+ await until('the service from the lean bundle again', () => services().at(-1) === 'lean')
+ writeFileSync(cli, `${body('cli')}\n// the next release\n`)
+ await until('the service from the new cli.js', () => services().at(-1) === 'cli')
+ expect(lines.some((line) => line.endsWith(`[harnessd] the lean bundle ${lean} cannot be used (${cli} is no longer the bundle it came from): services start from ${cli}`))).toBe(true)
+ signals.get('SIGTERM')!()
+ await until('the master to finish', () => exits.length > 0)
+ } finally { log.mockRestore() }
+ })
+
it.each([
['is gone', (pidFile: string) => rmSync(pidFile, { force: true })],
['holds no number', (pidFile: string) => writeFileSync(pidFile, 'garbage\n')],
@@ -411,7 +500,7 @@ describe('runMaster', () => {
runMaster({
nodePath: '/nonexistent/node', execArgv: [], scriptPath: '/nonexistent/core.js',
pidFile: ${JSON.stringify(join(dir, 'adapter.pid'))}, restoreUpdate: () => {}, confirmUpdate: () => {},
- env: { ...process.env, HARNESSD_INITIAL_BACKOFF_MS: '20' },
+ env: { ...process.env, HARNESSD_SERVICES: 'none', HARNESSD_INITIAL_BACKOFF_MS: '20' },
})
console.log('READY')
`)
@@ -452,7 +541,7 @@ describe('runMaster', () => {
const supervisor = runMaster({
nodePath: process.execPath, execArgv: [], scriptPath, pidFile: join(dir, 'adapter.pid'), statusFile: join(dir, 'harnessd-status.json'),
restoreUpdate: () => calls.push('restore'), confirmUpdate: () => calls.push('confirm'),
- env: { ...process.env, HARNESSD_INITIAL_BACKOFF_MS: '10', HARNESSD_UPDATE_PROBATION_MS: '100' },
+ env: { ...process.env, HARNESSD_SERVICES: 'none', HARNESSD_INITIAL_BACKOFF_MS: '10', HARNESSD_UPDATE_PROBATION_MS: '100' },
exit: (code) => exits.push(code), onSignal: (signal, listener) => signals.set(signal, listener),
version: '1.0.0', reexecMarkerFile: join(dir, 'harnessd-reexec.json'),
execve: (file: string, args: string[], env: NodeJS.ProcessEnv) => { execs.push({ file, args, env }) },
@@ -515,20 +604,39 @@ describe('runMaster', () => {
const lines: string[] = []
const log = vi.spyOn(console, 'log').mockImplementation((line: string) => { lines.push(line) })
try {
- const master = start(script, { env: { ...process.env, HARNESSD_UPDATE_PROBATION_MS: '100', [RESUME_ENV]: encodeResume(resume) } })
+ const master = start(script, { env: { ...process.env, HARNESSD_SERVICES: 'none', HARNESSD_UPDATE_PROBATION_MS: '100', [RESUME_ENV]: encodeResume(resume) } })
await until('the update to be kept', () => master.calls.includes('confirm'))
expect(JSON.parse(readFileSync(seen, 'utf8').trim().split('\n')[0])).toEqual({ restarts: '3', resume: null })
expect(readStatusFile(join(dir, 'harnessd-status.json'), process.pid)).toMatchObject({ masterVersion: '1.0.0', reexecs: 1, restarts: 3 })
expect(lines.some((line) => line.endsWith(`[harnessd] master re-executed (pid ${process.pid}) · now v1.0.0`))).toBe(true)
await master.stop()
// Without a version it says so without one.
- const plain = start(script, { version: undefined, env: { ...process.env, [RESUME_ENV]: encodeResume(resume) } })
+ const plain = start(script, { version: undefined, env: { ...process.env, HARNESSD_SERVICES: 'none', [RESUME_ENV]: encodeResume(resume) } })
await until('its core', () => plain.supervisor.status().state === 'running')
expect(lines.some((line) => line.endsWith(`[harnessd] master re-executed (pid ${process.pid})`))).toBe(true)
await plain.stop()
} finally { log.mockRestore() }
})
+ it('judges the bundle on disk against the one its lean bundle was read from, when it runs on one', async () => {
+ // A master re-executed on the lean bundle is handed the sha256 of the cli.js it was read from: the
+ // code it runs is that bundle's, whatever is on disk by the time it starts.
+ const script = bundle('')
+ const lines: string[] = []
+ const log = vi.spyOn(console, 'log').mockImplementation((line: string) => { lines.push(line) })
+ try {
+ const same = start(script, { serviceScriptPath: join(dir, 'lean.mjs'), bundleFingerprint: fingerprint(script) })
+ await until('its core', () => same.supervisor.status().state === 'running')
+ expect(same.execs).toEqual([])
+ expect(lines.some((line) => line.endsWith(`[harnessd] services run from ${join(dir, 'lean.mjs')}, as this master does`))).toBe(true)
+ await same.stop()
+ const older = start(script, { bundleFingerprint: 'the bundle before' })
+ await until('the master to re-execute on the bundle on disk', () => older.execs.length > 0)
+ expect(older.execs[0].args).toEqual([process.execPath, script, '__harnessd'])
+ await older.stop()
+ } finally { log.mockRestore() }
+ })
+
it('rolls back a re-execution that never brought a core up, then moves onto the bundle restored', async () => {
const script = bundle('')
writeMarker(marker(), { pid: DEAD_PID, from: 'the one before', to: fingerprint(script)!, at: 1 })
diff --git a/cli/src/harnessd/master.ts b/cli/src/harnessd/master.ts
index edb9320ef..34dfb0d15 100644
--- a/cli/src/harnessd/master.ts
+++ b/cli/src/harnessd/master.ts
@@ -5,9 +5,11 @@
*/
import { spawn, type ChildProcess } from 'node:child_process'
import { randomBytes } from 'node:crypto'
-import { readFileSync, renameSync, rmSync, writeFileSync } from 'node:fs'
+import { existsSync, readFileSync, renameSync, rmSync, writeFileSync } from 'node:fs'
import { performance } from 'node:perf_hooks'
-import { trimLogFile } from '../lib/log.js'
+import { trimLogFile, ts } from '../lib/log.js'
+import { folderFingerprint } from './leanBundle.js'
+import { leanServices } from './leanServices.js'
import { platformFromEnv, type PlatformName } from './platform.js'
import type { MasterMessage } from './protocol.js'
import {
@@ -25,8 +27,20 @@ export interface MasterConfig {
/** The node binary and flags the core runs with. */
nodePath: string
execArgv: string[]
- /** The CLI entry (`cli.js`, or `src/cli.ts` under tsx). */
+ /** The CLI entry (`cli.js`, or `src/cli.ts` under tsx): what the core runs, and what an update
+ * replaces on disk. */
scriptPath: string
+ /** What the services run instead, when it is not the CLI entry: the lean bundle the entry carries
+ * (./leanBundle.ts), so each service parses its own code and not the whole CLI's. Only ever an
+ * optimisation: a service starts from `scriptPath` whenever it cannot be used (./leanServices.ts). */
+ serviceScriptPath?: string
+ /** What the lean bundle's folder fingerprints to as this master starts (./leanBundle.ts
+ * `leanFingerprint`): checked before every service is started from it. */
+ leanFingerprint?: string
+ /** The sha256 of the bundle this master's code came from, when the master was not started on
+ * `scriptPath` itself but on the lean bundle read from it: the bundle it runs is the one it was read
+ * from, whatever is on disk by the time this master starts. */
+ bundleFingerprint?: string
pidFile: string
/** Where the master records its status for `harness status` (see `writeStatusFile`). */
statusFile?: string
@@ -229,14 +243,19 @@ export function runMaster(config: MasterConfig): Supervisor {
// Trimmed here rather than by the core: the master outlives every core, and two trimmers rewriting
// one file in place would race.
const stopTrimming = trimLogEvery(config.logFile)
- const log = (line: string) => console.log(`${new Date().toISOString().replace('T', ' ').slice(0, 23)} ${line}`)
+ // The core's stamp, local time (lib/log.ts): in UTC, the master's lines sat hours away from the core's
+ // lines around them in the one log they share.
+ const log = (line: string) => console.log(`${ts()} ${line}`)
// Set by the launchd agent or systemd unit `harness service install` writes (./platform.ts).
const platform = platformFromEnv(env)
if (platform) log(`[harnessd] run by ${platform}, which starts this master again if it dies`)
if (resume) log(`[harnessd] master re-executed (pid ${process.pid})${config.version ? ` · now v${config.version}` : ''}`)
+ if (config.serviceScriptPath && config.serviceScriptPath !== config.scriptPath) {
+ log(`[harnessd] services run from ${config.serviceScriptPath}${config.bundleFingerprint ? ', as this master does' : ''}`)
+ }
// The bundle this master's code came from, read once, as it starts; the bundle on disk is read
// again before every core starts.
- const own = fingerprint(config.scriptPath)
+ const own = config.bundleFingerprint ?? fingerprint(config.scriptPath)
const bundle = () => fingerprint(config.scriptPath)
const markerFile = config.reexecMarkerFile
const alive = (pid: number): boolean => { try { process.kill(pid, 0); return true } catch { return false } }
@@ -260,12 +279,21 @@ export function runMaster(config: MasterConfig): Supervisor {
// A new one every boot, given to the core and to each service: how the core knows a service
// connection is one this master started, and no other local process.
const token = randomBytes(24).toString('hex')
+ const lean = leanServices({
+ scriptPath: config.scriptPath, leanPath: config.serviceScriptPath, leanFingerprint: config.leanFingerprint,
+ folderFingerprint, exists: existsSync, sameBundle: () => bundle() === own, log,
+ })
const specs = serviceSpecs(env, KNOWN_SERVICES)
const services = new ServiceSupervisor(specs, {
- spawnService: (spec, extra) => coreHandle(spawn(config.nodePath, [...coreExecArgv(config.execArgv, spec.heapLimitMiB), config.scriptPath, '__service', spec.name], {
- env: { ...env, ...extra },
- stdio: ['ignore', 'inherit', 'inherit', 'ipc'],
- })),
+ spawnService: (spec, extra) => {
+ const script = lean.scriptFor(spec.name)
+ const handle = coreHandle(spawn(config.nodePath, [...coreExecArgv(config.execArgv, spec.heapLimitMiB), script, '__service', spec.name], {
+ env: { ...env, ...extra },
+ stdio: ['ignore', 'inherit', 'inherit', 'ipc'],
+ }))
+ lean.started(spec.name, script, handle)
+ return handle
+ },
now: () => performance.now(),
wallClock: () => Date.now(),
setTimer: (run, ms) => setTimeout(run, ms),
@@ -274,7 +302,7 @@ export function runMaster(config: MasterConfig): Supervisor {
}, serviceOptions(env), { HARNESSD_SERVICE_TOKEN: token })
const reexec = markerFile ? createReexec({
own, current: bundle, nodePath: config.nodePath, execArgv: config.execArgv, scriptPath: config.scriptPath, env, pid: process.pid,
- execve,
+ execve, exists: existsSync,
probe: (args, probeEnv) => runProbe(config.nodePath, args, probeEnv),
stopChildren: (done) => services.stop(done),
startChildren: () => services.start(),
diff --git a/cli/src/harnessd/reexec.spec.ts b/cli/src/harnessd/reexec.spec.ts
index fcb89795f..9e957f7db 100644
--- a/cli/src/harnessd/reexec.spec.ts
+++ b/cli/src/harnessd/reexec.spec.ts
@@ -173,6 +173,7 @@ describe('re-executing', () => {
own: 'old', current: () => onDisk, nodePath: '/opt/node', execArgv: ['--flag'], scriptPath: '/opt/cli.js',
env: { KEEP: '1' }, pid: 4242,
execve: (file, args, env) => { execs.push({ file, args, env }) },
+ exists: () => true,
probe: (args, env) => {
let settle!: (result: ProbeResult) => void
const entry = { args, env, settle: (result: ProbeResult) => settle(result), cancelled: false }
@@ -279,6 +280,21 @@ describe('re-executing', () => {
expect(calls).toContain('[harnessd] could not re-execute this master (odd) — keeping it')
})
+ it('never execs onto a node binary or a bundle that is not there: that exec could not be caught', async () => {
+ // On Node 22.23 an exec of a node binary that is gone aborts the master (exit 134), and one onto a bundle
+ // that is gone ends it in the new image: neither reaches the catch, and the next master rolls back.
+ for (const [gone, said] of [['/opt/node', '/opt/node'], ['/opt/cli.js', '/opt/cli.js']]) {
+ calls = []
+ const missing = make({ exists: (path) => path !== gone })
+ missing.reexec(state(), (outcome) => outcomes.push(outcome))
+ stopped.at(-1)!()
+ await settle({ ok: true, detail: PROBE_ANSWER })
+ expect(execs).toEqual([])
+ expect(calls.slice(-3)).toEqual(['remove marker', `[harnessd] ${said} is not there to re-execute on — keeping this master`, 'start children'])
+ }
+ expect(outcomes).toEqual(['kept', 'kept'])
+ })
+
it('carries on as itself, and does not end, when its marker cannot be written (a full disk)', async () => {
const full = make({ writeMarker: () => { throw new Error('ENOSPC: no space left on device, write') } })
full.reexec(state(), (outcome) => outcomes.push(outcome))
diff --git a/cli/src/harnessd/reexec.ts b/cli/src/harnessd/reexec.ts
index 215f375df..0ce264298 100644
--- a/cli/src/harnessd/reexec.ts
+++ b/cli/src/harnessd/reexec.ts
@@ -184,6 +184,8 @@ export interface ReexecDeps {
pid: number
/** `process.execve`, where this Node has it (22.15 and 23.11 on). */
execve: ((file: string, args: string[], env: NodeJS.ProcessEnv) => void) | null
+ /** Whether a file is there: the node binary and the bundle, checked right before the exec. */
+ exists(path: string): boolean
probe(args: string[], env: NodeJS.ProcessEnv): { result: Promise<ProbeResult>; cancel(): void }
/** Stop the services and wait for them to be gone, so none is left a zombie or an orphan. */
stopChildren(done: () => void): void
@@ -257,10 +259,22 @@ export function createReexec(deps: ReexecDeps): {
carryOn('kept')
return
}
+ // An exec that fails cannot be caught once it has begun: on Node 22.23 a node binary that is not
+ // there aborts this process (exit 134), and a bundle that is not there ends it in the new image
+ // (MODULE_NOT_FOUND), the marker left behind for the next master to roll the update back. Both
+ // were there for the probe a moment ago; they are checked again here, right before.
+ const missing = [deps.nodePath, deps.scriptPath].find((path) => !deps.exists(path))
+ if (missing) {
+ deps.removeMarker()
+ deps.log(`[harnessd] ${missing} is not there to re-execute on — keeping this master`)
+ carryOn('kept')
+ return
+ }
try {
// Never returns: from here this process is the new bundle's master, with the same pid.
deps.execve!(deps.nodePath, [deps.nodePath, ...args, '__harnessd'], env)
} catch (error) {
+ // Only what `process.execve` refuses before it begins: arguments it cannot take.
deps.removeMarker()
deps.log(`[harnessd] could not re-execute this master (${error instanceof Error ? error.message : String(error)}) — keeping it`)
carryOn('kept')
diff --git a/cli/src/harnessd/services.spec.ts b/cli/src/harnessd/services.spec.ts
index 074534a8a..3bd6fc16a 100644
--- a/cli/src/harnessd/services.spec.ts
+++ b/cli/src/harnessd/services.spec.ts
@@ -295,7 +295,11 @@ describe('ServiceSupervisor', () => {
it('reads the services to run from the environment, keeping only known ones, once each', () => {
const known = { search: { heapLimitMiB: 512, rssLimitMiB: 1_024 }, devices: { heapLimitMiB: 256, rssLimitMiB: 0 } }
expect(serviceSpecs({ HARNESSD_SERVICES: ' search, nope ,devices,search,' }, known)).toEqual([search, devices])
- expect(serviceSpecs({}, known)).toEqual([])
+ // Every service in its own process unless told otherwise: isolation is the default.
+ expect(serviceSpecs({}, known)).toEqual([search, devices])
+ expect(serviceSpecs({ HARNESSD_SERVICES: 'none' }, known)).toEqual([])
+ expect(serviceSpecs({ HARNESSD_SERVICES: '' }, known)).toEqual([])
+ expect(serviceSpecs({ HARNESSD_SERVICES: 'none,search' }, known)).toEqual([search])
expect(serviceSpecs({ HARNESSD_SERVICES: 'toString,constructor' }, known)).toEqual([])
// One heap limit for every service, when given as a whole number of MiB.
expect(serviceSpecs({ HARNESSD_SERVICES: 'search', HARNESSD_SERVICE_HEAP_LIMIT_MIB: '96' }, known)).toEqual([{ ...search, heapLimitMiB: 96 }])
diff --git a/cli/src/harnessd/services.ts b/cli/src/harnessd/services.ts
index 240fbed0a..ce3df9678 100644
--- a/cli/src/harnessd/services.ts
+++ b/cli/src/harnessd/services.ts
@@ -287,9 +287,9 @@ export class ServiceSupervisor {
}
/**
- * The services this build can run in their own processes, with their memory budgets. Which of them do
- * is `HARNESSD_SERVICES` (`search,viewers,workspaces,teams`): off until named, while each one beds in,
- * and the core runs a service that is not out here in its own process, as before.
+ * The services this build runs in their own processes, with their memory budgets: every one, unless
+ * `HARNESSD_SERVICES` names a subset (`search,viewers`) or `none` (see `serviceSpecs`). A service that
+ * is not out here runs inside the core's process, behind the service host's guard, as before.
*/
export const KNOWN_SERVICES: Readonly<Record<string, Omit<ServiceSpec, 'name'>>> = {
search: { heapLimitMiB: 1_024, rssLimitMiB: 2_048 },
@@ -351,10 +351,17 @@ export function servicesTheMasterRuns(env: NodeJS.ProcessEnv, known: Readonly<Re
return new Set(said.split(',').map((name) => name.trim()).filter((name) => name && Object.hasOwn(known, name)))
}
-/** The services to run, from `HARNESSD_SERVICES` (`search,devices`): only names this build knows.
- * `HARNESSD_SERVICE_HEAP_LIMIT_MIB` gives every one the same heap limit instead (tests, support). */
+/**
+ * The services to run in their own processes: every one this build knows, unless `HARNESSD_SERVICES`
+ * names a subset; `none` (or empty) runs them all inside the core's process, for debugging or a quick
+ * way back. Isolation is the point of the split (one service failing costs only itself), so it is the
+ * default, not an opt-in. Only names this build knows. `HARNESSD_SERVICE_HEAP_LIMIT_MIB` gives every
+ * one the same heap limit instead (tests, support).
+ */
export function serviceSpecs(env: NodeJS.ProcessEnv, known: Readonly<Record<string, Omit<ServiceSpec, 'name'>>>): ServiceSpec[] {
- const names = (env.HARNESSD_SERVICES ?? '').split(',').map((name) => name.trim()).filter(Boolean)
+ const names = env.HARNESSD_SERVICES === undefined
+ ? Object.keys(known)
+ : env.HARNESSD_SERVICES.split(',').map((name) => name.trim()).filter((name) => name && name !== 'none')
const heap = Number(env.HARNESSD_SERVICE_HEAP_LIMIT_MIB)
return [...new Set(names)].filter((name) => Object.hasOwn(known, name)).map((name) => ({
name, ...known[name], ...(Number.isInteger(heap) && heap > 0 ? { heapLimitMiB: heap } : {}),
diff --git a/cli/src/leanEntry.spec.ts b/cli/src/leanEntry.spec.ts
new file mode 100644
index 000000000..4e75e51bc
--- /dev/null
+++ b/cli/src/leanEntry.spec.ts
@@ -0,0 +1,93 @@
+/**
+ * The lean bundle a release carries, as the build writes it: what each process of harnessd loads from
+ * it. A process parses every file it imports, so the memory each costs is the code it imports, and
+ * that is what this holds: the master none of the services' code, a service none of another's, and the
+ * master, search and workspaces no zod (`config/env.ts` and `lib/registry.ts` once brought it to every
+ * one of them, 8 to 19 MiB each, measured 2026-10-05).
+ */
+import { execFileSync, spawnSync } from 'node:child_process'
+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+import { fileURLToPath } from 'node:url'
+import { afterAll, beforeAll, describe, expect, it } from 'vitest'
+import { LEAN_ENTRY, readLeanBundle, writeLeanBundle } from './harnessd/leanBundle.js'
+import { PROBE_ANSWER } from './harnessd/reexec.js'
+import { KNOWN_SERVICES } from './harnessd/services.js'
+
+const CLI_ROOT = fileURLToPath(new URL('..', import.meta.url))
+const ROLES = ['master', ...Object.keys(KNOWN_SERVICES)]
+const RUNNER: Record<string, string> = { master: 'masterProcess' }
+for (const name of Object.keys(KNOWN_SERVICES)) RUNNER[name] = `${name}Process`
+
+describe('the lean bundle a release carries', () => {
+ let scratch = ''
+ let entry = ''
+ let files = new Map<string, string>()
+ beforeAll(() => {
+ scratch = mkdtempSync(join(tmpdir(), 'lean-entry-'))
+ execFileSync(process.execPath, ['build-bundle.mjs'], { cwd: CLI_ROOT, env: { ...process.env, BUNDLE_OUT_DIR: join(scratch, 'build') }, stdio: 'pipe' })
+ const lean = readLeanBundle(readFileSync(join(scratch, 'build', 'cli.js')))!
+ files = new Map([...lean.files].map(([name, code]) => [name, code.toString('utf8')]))
+ entry = writeLeanBundle(join(scratch, 'lean'), lean)
+ }, 120_000)
+ afterAll(() => { rmSync(scratch, { recursive: true, force: true }) })
+
+ /** The files a process loads: the entry, its runner's file, and what they import, but not what they
+ * import only when asked (another role's runner). */
+ function loads(role: string): Set<string> {
+ const runner = [...files.keys()].find((name) => name.startsWith(`${RUNNER[role]}-`))
+ expect(runner, `${role}'s own file`).toBeDefined()
+ const roots = role === 'master' ? [LEAN_ENTRY, runner!] : [LEAN_ENTRY, [...files.keys()].find((name) => name.startsWith('serviceProcess-'))!, runner!]
+ const seen = new Set<string>()
+ const visit = (name: string): void => {
+ if (seen.has(name)) return
+ seen.add(name)
+ for (const match of files.get(name)!.matchAll(/(?:from|import)\s*"\.\/([^"]+\.mjs)"/g)) visit(match[1])
+ }
+ for (const root of roots) visit(root)
+ return seen
+ }
+ const runnerOf = (name: string): string | undefined => Object.entries(RUNNER).find(([, runner]) => name.startsWith(`${runner}-`))?.[0]
+ const hasZod = (names: Set<string>): boolean => [...names].some((name) => files.get(name)!.includes('$ZodType'))
+
+ it('is files of their own for the master and each service, the code it runs and no other role\'s', () => {
+ for (const role of ROLES) {
+ const others = [...loads(role)].map(runnerOf).filter((owner) => owner && owner !== role)
+ expect(others, `${role} loads another's code`).toEqual([])
+ }
+ })
+
+ it('brings no zod to the master, search or workspaces', () => {
+ expect(hasZod(new Set(files.keys())), 'the bundle still holds zod, for the services whose own code uses it').toBe(true)
+ for (const role of ['master', 'search', 'workspaces']) expect(hasZod(loads(role)), role).toBe(false)
+ })
+
+ it('passes the release script\'s check, which a lean bundle that breaks a process does not', async () => {
+ // @ts-expect-error — plain ESM with no declaration file
+ const { leanBlock } = await import('../scripts/lib/leanBlock.mjs') as { leanBlock: (files: Record<string, string>) => string }
+ const check = (cli: string) => spawnSync(process.execPath, ['scripts/check-lean-bundle.mjs', cli], { cwd: CLI_ROOT, encoding: 'utf8', timeout: 120_000 })
+ const good = check(join(scratch, 'build', 'cli.js'))
+ expect(good.status, good.stderr).toBe(0)
+ expect(good.stdout).toMatch(/the master answers its probe, and \d+ modules load/)
+ const search = [...files.keys()].find((name) => name.startsWith('searchProcess-'))!
+ const broken = join(scratch, 'broken.js')
+ writeFileSync(broken, `console.log("cli")\n${leanBlock({ ...Object.fromEntries(files), [search]: 'throw new Error("a broken service")' })}`)
+ const bad = check(broken)
+ expect(bad.status).toBe(1)
+ expect(bad.stderr).toContain(`${search} does not load`)
+ expect(bad.stderr).toContain('a broken service')
+ writeFileSync(broken, `console.log("cli")\n${leanBlock({ ...Object.fromEntries(files), [LEAN_ENTRY]: 'process.exit(5)' })}`)
+ expect(check(broken).stderr).toContain('its master does not answer its probe')
+ writeFileSync(broken, 'console.log("no lean bundle")\n')
+ expect(check(broken).stderr).toContain('carries no lean bundle')
+ }, 120_000)
+
+ it('starts as written out: its master answers the probe a re-executing master asks', () => {
+ const run = spawnSync(process.execPath, [entry, '__harnessd-probe'], {
+ env: { PATH: process.env.PATH, HOME: join(scratch, 'home'), ADAPTER_DATA_DIR: join(scratch, 'data') }, encoding: 'utf8', timeout: 30_000,
+ })
+ expect(run.status, run.stderr).toBe(0)
+ expect(run.stdout).toContain(PROBE_ANSWER)
+ })
+})
diff --git a/cli/src/leanEntry.ts b/cli/src/leanEntry.ts
new file mode 100644
index 000000000..20a89c114
--- /dev/null
+++ b/cli/src/leanEntry.ts
@@ -0,0 +1,43 @@
+/**
+ * The lean bundle's entry: harnessd's master and its services, and nothing else (harnessd/leanBundle.ts).
+ *
+ * Built a second time, on its own, and carried inside cli.js, so that the master and each service start
+ * on their own code instead of the whole CLI's 4.4 MB: Node parses every byte of the files a process
+ * loads, and the whole CLI cost each of them about 45 MiB at idle. Built split, each dynamic import below
+ * a file of its own with what only it uses, so a process loads this file, its own and the shared ones it
+ * imports: the master never the services' code, a service never the master's nor another service's
+ * (leanEntry.spec.ts holds it to that). A master started on cli.js re-executes on this file
+ * (masterProcess.ts `startMasterFromBundle`) and starts the services from it. The core and the CLI
+ * always run from cli.js.
+ */
+import { fileURLToPath } from 'node:url'
+
+const [, , command, name] = process.argv
+if (command === '__harnessd') {
+ const master = await import('./masterProcess.js')
+ const bundle = process.env[master.BUNDLE_ENV]
+ if (!bundle) {
+ console.error('[harnessd] the lean bundle runs a master only for the cli.js it was read from, which hands over its path')
+ process.exit(2)
+ }
+ const bundleFingerprint = process.env[master.BUNDLE_SHA256_ENV]
+ const leanFingerprint = process.env[master.LEAN_FINGERPRINT_ENV]
+ // Not for the core and the services: neither starts a master, and a master started from cli.js again,
+ // after an update, hands over its own.
+ delete process.env[master.BUNDLE_ENV]
+ delete process.env[master.BUNDLE_SHA256_ENV]
+ delete process.env[master.LEAN_FINGERPRINT_ENV]
+ master.startMaster({
+ scriptPath: bundle,
+ serviceScriptPath: fileURLToPath(import.meta.url),
+ ...(bundleFingerprint ? { bundleFingerprint } : {}),
+ ...(leanFingerprint ? { leanFingerprint } : {}),
+ })
+} else if (command === '__harnessd-probe') {
+ process.exitCode = (await import('./masterProcess.js')).probeThisMaster()
+} else if (command === '__service') {
+ await (await import('./serviceProcess.js')).startServiceProcess(name)
+} else {
+ console.error(`[harnessd] the lean bundle runs harnessd's master and its services; ${command ?? 'nothing'} is the CLI's (cli.js)`)
+ process.exit(2)
+}
diff --git a/cli/src/lib/registry.ts b/cli/src/lib/registry.ts
index 070481aaa..dd20abfc6 100644
--- a/cli/src/lib/registry.ts
+++ b/cli/src/lib/registry.ts
@@ -13,7 +13,7 @@
* Module singleton (like the ws `clients` set) — imported by routes + reaper.
*/
-import { DSH_ID_RE } from '../dsh/manifest.js'
+import { DSH_ID_RE } from '../dsh/id.js'
import { AGENT_NAME_RE } from './engineLaunch.js'
import { resumesConversation } from './resumeCapability.js'
import { namingTitle } from './sessionTitle.js'
diff --git a/cli/src/masterProcess.spec.ts b/cli/src/masterProcess.spec.ts
new file mode 100644
index 000000000..efb029cc1
--- /dev/null
+++ b/cli/src/masterProcess.spec.ts
@@ -0,0 +1,195 @@
+import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
+import { PROBE_ANSWER } from './harnessd/reexec.js'
+import { LEAN_ENTRY, leanFingerprint, type LeanBundle } from './harnessd/leanBundle.js'
+
+const runMaster = vi.hoisted(() => vi.fn((_config: Record<string, unknown>) => 'supervisor'))
+vi.mock('./harnessd/master.js', async (real) => ({ ...await real<object>(), runMaster }))
+const leanBundle = vi.hoisted(() => ({ read: vi.fn((_bundle: Buffer): LeanBundle | null => null) }))
+vi.mock('./harnessd/leanBundle.js', async (real) => ({ ...await real<object>(), readLeanBundle: (bundle: Buffer) => leanBundle.read(bundle) }))
+
+const {
+ BUNDLE_ENV, BUNDLE_SHA256_ENV, LEAN_DIR, LEAN_FINGERPRINT_ENV, LEAN_OFF_FILE, probeLean, probeThisMaster, processBundleDeps, startMaster, startMasterFromBundle,
+} = await import('./masterProcess.js')
+type Deps = Parameters<typeof startMasterFromBundle>[1] & object
+
+const LEAN: LeanBundle = { files: new Map([[LEAN_ENTRY, Buffer.from('lean')]]), sha256: 'a'.repeat(64), bundleSha256: 'b'.repeat(64) }
+const PRINT = leanFingerprint(LEAN)
+
+function deps(over: Partial<Deps> = {}) {
+ const calls = { starts: [] as unknown[], execs: [] as Array<{ file: string; args: string[]; env: NodeJS.ProcessEnv }>, logs: [] as string[], probes: [] as Array<{ path: string; env: NodeJS.ProcessEnv }> }
+ const given: Deps = {
+ env: { HOME: '/home/someone' },
+ leanOff: () => false,
+ exists: () => true,
+ read: () => Buffer.from('cli.js'),
+ write: () => '/data/lean/harnessd-aaaa.mjs',
+ probe: (path, env) => { calls.probes.push({ path, env }); return { ok: true, detail: PROBE_ANSWER } },
+ execve: (file, args, env) => { calls.execs.push({ file, args, env }) },
+ start: (start) => { calls.starts.push(start) },
+ log: (line) => { calls.logs.push(line) },
+ ...over,
+ }
+ return { given, calls }
+}
+
+describe('a master started on cli.js', () => {
+ beforeEach(() => { leanBundle.read.mockReset().mockReturnValue(LEAN) })
+
+ it('re-executes, same pid, on the lean bundle cli.js carries, once that bundle answers its probe', () => {
+ const { given, calls } = deps()
+ startMasterFromBundle('/cli/cli.js', given)
+ const handed = { HOME: '/home/someone', [BUNDLE_ENV]: '/cli/cli.js', [BUNDLE_SHA256_ENV]: 'b'.repeat(64), [LEAN_FINGERPRINT_ENV]: PRINT }
+ expect(calls.probes).toEqual([{ path: '/data/lean/harnessd-aaaa.mjs', env: handed }])
+ expect(calls.execs).toEqual([{ file: process.execPath, args: [process.execPath, ...process.execArgv, '/data/lean/harnessd-aaaa.mjs', '__harnessd'], env: handed }])
+ expect(calls.starts).toEqual([])
+ expect(calls.logs).toEqual([])
+ })
+
+ it('runs from cli.js, with the services still from the lean bundle, where this Node cannot re-execute', () => {
+ const { given, calls } = deps({ execve: null })
+ startMasterFromBundle('/cli/cli.js', given)
+ expect(calls.starts).toEqual([{ scriptPath: '/cli/cli.js', serviceScriptPath: '/data/lean/harnessd-aaaa.mjs', leanFingerprint: PRINT }])
+ expect(calls.logs).toEqual(['[harnessd] this Node cannot re-execute the master: it runs from /cli/cli.js, the services from /data/lean/harnessd-aaaa.mjs'])
+ const failing = deps({ execve: () => { throw new Error('E2BIG') } })
+ startMasterFromBundle('/cli/cli.js', failing.given)
+ expect(failing.calls.starts).toEqual([{ scriptPath: '/cli/cli.js', serviceScriptPath: '/data/lean/harnessd-aaaa.mjs', leanFingerprint: PRINT }])
+ expect(failing.calls.logs[0]).toContain('could not re-execute on /data/lean/harnessd-aaaa.mjs (E2BIG)')
+ const thrown = deps({ execve: () => { throw 'refused' } })
+ startMasterFromBundle('/cli/cli.js', thrown.given)
+ expect(thrown.calls.logs[0]).toContain('(refused)')
+ })
+
+ it('never execs onto a node binary or a lean bundle that is not there: that exec could not be caught', () => {
+ // On Node 22.23 an exec of a missing node binary aborts the master (exit 134) and one onto a missing
+ // script ends it in the new image (MODULE_NOT_FOUND): neither reaches the catch.
+ for (const gone of [process.execPath, '/data/lean/harnessd-aaaa.mjs']) {
+ const { given, calls } = deps({ exists: (path) => path !== gone })
+ startMasterFromBundle('/cli/cli.js', given)
+ expect(calls.execs).toEqual([])
+ expect(calls.starts).toEqual([{ scriptPath: '/cli/cli.js' }])
+ expect(calls.logs).toEqual([`[harnessd] ${gone} is not there to re-execute on: the master and the services run from /cli/cli.js`])
+ }
+ })
+
+ it('runs everything from cli.js while the lean-off file is in the data folder: a switch a launchd or systemd master sees', () => {
+ const { given, calls } = deps({ leanOff: () => true })
+ startMasterFromBundle('/cli/cli.js', given)
+ expect(calls.starts).toEqual([{ scriptPath: '/cli/cli.js' }])
+ expect(calls.probes).toEqual([])
+ expect(calls.logs).toEqual([`[harnessd] ${LEAN_OFF_FILE} is there: the master and the services run from /cli/cli.js`])
+ })
+
+ it('runs everything from cli.js when the lean bundle is not there to use, and says why', () => {
+ const cases: Array<[Partial<Deps>, string]> = [
+ [{ probe: () => ({ ok: false, detail: 'SyntaxError' }) }, 'did not answer its probe (SyntaxError)'],
+ [{ write: () => { throw new Error('ENOSPC') } }, 'could not be written out (ENOSPC)'],
+ [{ read: () => { throw 'EACCES' } }, 'could not be written out (EACCES)'],
+ ]
+ for (const [over, said] of cases) {
+ const { given, calls } = deps(over)
+ startMasterFromBundle('/cli/cli.js', given)
+ expect(calls.starts).toEqual([{ scriptPath: '/cli/cli.js' }])
+ expect(calls.execs).toEqual([])
+ expect(calls.logs[0]).toContain(said)
+ }
+ leanBundle.read.mockReturnValue(null)
+ const none = deps()
+ startMasterFromBundle('/cli/cli.js', none.given)
+ expect(none.calls.starts).toEqual([{ scriptPath: '/cli/cli.js' }])
+ expect(none.calls.logs).toEqual(['[harnessd] no lean bundle in /cli/cli.js: the master and the services run from it'])
+ })
+
+ it('runs everything from cli.js, quietly, with HARNESSD_LEAN=off', () => {
+ const { given, calls } = deps({ env: { HARNESSD_LEAN: 'off' } })
+ startMasterFromBundle('/cli/cli.js', given)
+ expect(calls.starts).toEqual([{ scriptPath: '/cli/cli.js' }])
+ expect(calls.logs).toEqual([])
+ expect(calls.probes).toEqual([])
+ })
+
+ it('acts on this process by default', () => {
+ const real = processBundleDeps()
+ expect(real.env).toBe(process.env)
+ expect(real.leanOff()).toBe(false)
+ expect(real.exists(process.execPath)).toBe(true)
+ expect(real.start).toBe(startMaster)
+ expect(real.read(__filename).length).toBeGreaterThan(0)
+ // The data folder is a test's own (vitest.setup.ts).
+ const written = real.write(LEAN)
+ expect(written.startsWith(LEAN_DIR)).toBe(true)
+ expect(real.probe(written, process.env).ok).toBe(false)
+ const log = vi.spyOn(console, 'log').mockImplementation(() => {})
+ real.log('[harnessd] a line')
+ expect(log.mock.calls[0][0]).toMatch(/^\d{4}-\d\d-\d\d \d\d:\d\d:\d\d\.\d{3} \[harnessd\] a line$/)
+ log.mockRestore()
+ })
+})
+
+describe('the probe of a lean bundle', () => {
+ let dir: string
+ beforeEach(() => { dir = mkdtempSync(join(tmpdir(), 'lean-probe-')) })
+ afterEach(() => rmSync(dir, { recursive: true, force: true }))
+ const script = (body: string): string => {
+ const path = join(dir, `${Math.random().toString(36).slice(2)}.mjs`)
+ writeFileSync(path, body)
+ return path
+ }
+
+ it('passes when it answers as a master would, and says why it did not otherwise', () => {
+ expect(probeLean(script(`if (process.argv[2] === '__harnessd-probe' && process.env.HANDED === 'yes') console.log(${JSON.stringify(PROBE_ANSWER)})`), { ...process.env, HANDED: 'yes' }))
+ .toEqual({ ok: true, detail: PROBE_ANSWER })
+ expect(probeLean(script(`console.error('it broke'); process.exit(3)`), process.env)).toEqual({ ok: false, detail: 'it broke' })
+ expect(probeLean(script(`console.log('something else')`), process.env)).toEqual({ ok: false, detail: 'something else' })
+ expect(probeLean(script(''), process.env)).toEqual({ ok: false, detail: 'exit 0' })
+ expect(probeLean(script(`process.kill(process.pid, 'SIGKILL')`), process.env)).toEqual({ ok: false, detail: 'signal SIGKILL' })
+ expect(probeLean(script('setInterval(() => {}, 1000)'), process.env, 200).ok).toBe(false)
+ })
+})
+
+describe('starting the master', () => {
+ it('runs it on the paths it is given, with the daemon\'s files and update backups', () => {
+ runMaster.mockClear()
+ expect(startMaster({ scriptPath: '/cli/cli.js', serviceScriptPath: '/lean.mjs', bundleFingerprint: 'f' })).toBe('supervisor')
+ const config = runMaster.mock.calls[0][0] as Record<string, unknown> & { restoreUpdate(): void; confirmUpdate(): void }
+ expect(config).toMatchObject({ scriptPath: '/cli/cli.js', serviceScriptPath: '/lean.mjs', bundleFingerprint: 'f', nodePath: process.execPath })
+ expect(String(config.reexecMarkerFile)).toMatch(/harnessd-reexec\.json$/)
+ expect(LEAN_DIR).toMatch(/[\\/]lean$/)
+ expect(LEAN_OFF_FILE).toMatch(/[\\/]lean-off$/)
+ })
+
+ it('gives up its claim on the lean bundle as it exits cleanly', () => {
+ const folder = mkdtempSync(join(tmpdir(), 'lean-claim-'))
+ try {
+ const claim = join(folder, `.claim-${process.pid}`)
+ writeFileSync(claim, '')
+ const exits: number[] = []
+ runMaster.mockClear()
+ startMaster({ scriptPath: '/cli/cli.js', serviceScriptPath: join(folder, LEAN_ENTRY) }, (code) => exits.push(code))
+ const config = runMaster.mock.calls[0][0] as { exit(code: number): void }
+ config.exit(0)
+ expect(exits).toEqual([0])
+ expect(existsSync(claim)).toBe(false)
+ // Without a lean bundle there is no claim to give up.
+ runMaster.mockClear()
+ startMaster({ scriptPath: '/cli/cli.js' }, (code) => exits.push(code))
+ ;(runMaster.mock.calls[0][0] as { exit(code: number): void }).exit(3)
+ expect(exits).toEqual([0, 3])
+ const processExit = vi.spyOn(process, 'exit').mockImplementation((() => undefined) as never)
+ runMaster.mockClear()
+ startMaster({ scriptPath: '/cli/cli.js' })
+ ;(runMaster.mock.calls[0][0] as { exit(code: number): void }).exit(4)
+ expect(processExit).toHaveBeenCalledWith(4)
+ processExit.mockRestore()
+ } finally { rmSync(folder, { recursive: true, force: true }) }
+ })
+
+ it('answers the probe a re-executing master asks', () => {
+ const log = vi.spyOn(console, 'log').mockImplementation(() => {})
+ expect(probeThisMaster()).toBe(0)
+ expect(String(log.mock.calls[0][0])).toContain(PROBE_ANSWER)
+ log.mockRestore()
+ })
+})
diff --git a/cli/src/masterProcess.ts b/cli/src/masterProcess.ts
new file mode 100644
index 000000000..a4cb88d94
--- /dev/null
+++ b/cli/src/masterProcess.ts
@@ -0,0 +1,175 @@
+/**
+ * harnessd's master as a process: what `harness start` runs (`__harnessd`), and the probe a master about
+ * to re-execute on a new bundle asks first (`__harnessd-probe`, harnessd/reexec.ts).
+ *
+ * The master holds no feature code (harnessd/AGENTS.md), yet weighed what the whole CLI weighs: every
+ * process evaluated the whole bundle, and Node parses all of the file a process is started on whatever
+ * it runs. Measured from the bundle at idle (2026-10-05): the master 160 MiB resident, each service 115
+ * to 160. So a master started on cli.js re-executes itself, same pid, on the lean bundle cli.js carries
+ * (harnessd/leanBundle.ts), and starts the services from it too: each then parses its own code and not
+ * the whole CLI's 4.4 MB.
+ */
+import { spawnSync } from 'node:child_process'
+import { existsSync, readFileSync } from 'node:fs'
+import { join } from 'node:path'
+import { env } from './config/env.js'
+import { processExecve, probeMaster, runMaster, type Execve } from './harnessd/master.js'
+import { PROBE_ANSWER, PROBE_TIMEOUT_MS } from './harnessd/reexec.js'
+import { ensureUtf8Locale } from './lib/childLocale.js'
+import { DAEMON_LOG_FILE, HARNESSD_STATUS_FILE, PID_FILE } from './lib/daemonState.js'
+import { leanFingerprint, readLeanBundle, releaseLeanClaim, writeLeanBundle, type LeanBundle } from './harnessd/leanBundle.js'
+import { ts } from './lib/log.js'
+import { confirm as confirmUpdate, restore as restoreUpdate, unjudgedUpdate } from './lib/selfUpdate.js'
+import { VERSION } from './version.js'
+
+/** Where a master re-executing on a new bundle leaves word of it, until that master's core is up. */
+export const HARNESSD_REEXEC_FILE = join(env.ADAPTER_DATA_DIR, 'harnessd-reexec.json')
+/** Where the lean bundle is written out for the master and the services to start from. */
+export const LEAN_DIR = join(env.ADAPTER_DATA_DIR, 'lean')
+/** Handed by a master started on cli.js to itself re-executed on the lean bundle: the bundle it read the
+ * lean bundle from, and that bundle's sha256. */
+export const BUNDLE_ENV = 'HARNESSD_BUNDLE'
+export const BUNDLE_SHA256_ENV = 'HARNESSD_BUNDLE_SHA256'
+/** And what the lean bundle's folder fingerprints to: checked before every service is started from it. */
+export const LEAN_FINGERPRINT_ENV = 'HARNESSD_LEAN_FINGERPRINT'
+/** While this file is in the data folder, masters start on cli.js alone, as with `HARNESSD_LEAN=off`: the
+ * way to turn the lean bundle off for a master launchd or systemd starts, whose environment is the
+ * unit's (`touch ~/.harness/cli/data/lean-off`, then `harness restart`). */
+export const LEAN_OFF_FILE = join(env.ADAPTER_DATA_DIR, 'lean-off')
+
+export interface MasterStart {
+ /** What the core runs and an update replaces: cli.js, or src/cli.ts from the sources. */
+ scriptPath: string
+ /** What the services run, when not `scriptPath`: the lean bundle. */
+ serviceScriptPath?: string
+ /** The sha256 of the bundle the lean bundle was read from, for a master running on the lean bundle. */
+ bundleFingerprint?: string
+ /** What the lean bundle's folder fingerprints to (harnessd/leanBundle.ts `leanFingerprint`). */
+ leanFingerprint?: string
+}
+
+/** Run the master. */
+export function startMaster(start: MasterStart, exit: (code: number) => void = (code) => process.exit(code)): ReturnType<typeof runMaster> {
+ // Before it starts anything: on Linux an absent locale makes tmux and ps mangle their output, and the
+ // core and the services inherit this environment (lib/childLocale.ts). launchd and systemd start the
+ // master with no `harness start` before it to have set it.
+ ensureUtf8Locale()
+ const { serviceScriptPath } = start
+ return runMaster({
+ nodePath: process.execPath,
+ execArgv: process.execArgv,
+ ...start,
+ pidFile: PID_FILE,
+ statusFile: HARNESSD_STATUS_FILE,
+ logFile: DAEMON_LOG_FILE,
+ restoreUpdate: () => restoreUpdate(env.ADAPTER_CLI_DIR),
+ confirmUpdate: () => confirmUpdate(env.ADAPTER_CLI_DIR),
+ version: VERSION,
+ reexecMarkerFile: HARNESSD_REEXEC_FILE,
+ unjudgedUpdate: (bundle) => unjudgedUpdate(env.ADAPTER_CLI_DIR, bundle),
+ // A master that ends cleanly gives up its claim on its lean bundle, so the next one to start can
+ // clear the folder; one that dies leaves a claim whose pid is gone, which counts for nothing.
+ exit: (code) => {
+ if (serviceScriptPath) releaseLeanClaim(serviceScriptPath)
+ exit(code)
+ },
+ })
+}
+
+/** Whether this bundle can run a master that takes over from the running one; the exit code. */
+export function probeThisMaster(): number {
+ return probeMaster({ env: process.env, execArgv: process.execArgv, version: VERSION })
+}
+
+export interface BundleMasterDeps {
+ env: NodeJS.ProcessEnv
+ /** Whether `LEAN_OFF_FILE` is there. */
+ leanOff: () => boolean
+ exists: (path: string) => boolean
+ read: (path: string) => Buffer
+ /** Writes the lean bundle out and returns where (harnessd/leanBundle.ts `writeLeanBundle`). */
+ write: (lean: LeanBundle) => string
+ /** Runs `<lean> __harnessd-probe` as the re-executed master would start: whether it answered. */
+ probe: (leanPath: string, env: NodeJS.ProcessEnv) => { ok: boolean; detail: string }
+ execve: Execve | null
+ start: (start: MasterStart) => unknown
+ log: (line: string) => void
+}
+
+/** Run `<leanPath> __harnessd-probe` with this process's Node and flags, as the master would start on it. */
+export function probeLean(leanPath: string, probeEnv: NodeJS.ProcessEnv, timeoutMs = PROBE_TIMEOUT_MS): { ok: boolean; detail: string } {
+ const run = spawnSync(process.execPath, [...process.execArgv, leanPath, '__harnessd-probe'], {
+ env: probeEnv, encoding: 'utf8', timeout: timeoutMs, stdio: ['ignore', 'pipe', 'pipe'],
+ })
+ // Judged as the re-execution's own probe judges a bundle (harnessd/reexec.ts `runProbe`).
+ const out = String(run.stdout ?? '')
+ const lastLine = (text: string): string => text.trim().split('\n').pop()!.trim()
+ return {
+ ok: run.status === 0 && out.includes(PROBE_ANSWER),
+ detail: run.error?.message ?? (lastLine(String(run.stderr ?? '')) || lastLine(out) || (run.signal ? `signal ${run.signal}` : `exit ${run.status}`)),
+ }
+}
+
+export const processBundleDeps = (): BundleMasterDeps => ({
+ env: process.env,
+ leanOff: () => existsSync(LEAN_OFF_FILE),
+ exists: existsSync,
+ read: (path) => readFileSync(path),
+ write: (lean) => writeLeanBundle(LEAN_DIR, lean),
+ probe: (leanPath, probeEnv) => probeLean(leanPath, probeEnv),
+ execve: processExecve(),
+ start: startMaster,
+ log: (line) => console.log(`${ts()} ${line}`),
+})
+
+/**
+ * The master, started on [bundlePath] (cli.js): re-executed on the lean bundle cli.js carries when this
+ * Node can (22.15 and 23.11 on), with the services started from it; run here, from cli.js, with the
+ * services still started from the lean bundle when it cannot; and as before, from cli.js alone, when
+ * there is no lean bundle to use, `HARNESSD_LEAN=off` or `LEAN_OFF_FILE`. A lean bundle is used only once
+ * it has answered the probe a master about to re-execute asks (harnessd/reexec.ts): one that cannot start
+ * a master is never handed the daemon.
+ */
+export function startMasterFromBundle(bundlePath: string, deps: BundleMasterDeps = processBundleDeps()): void {
+ const fromBundle = (reason: string | null, lean?: { path: string; fingerprint: string }): void => {
+ if (reason) deps.log(`[harnessd] ${reason}`)
+ deps.start({ scriptPath: bundlePath, ...(lean ? { serviceScriptPath: lean.path, leanFingerprint: lean.fingerprint } : {}) })
+ }
+ if (deps.env.HARNESSD_LEAN === 'off') { fromBundle(null); return }
+ if (deps.leanOff()) { fromBundle(`${LEAN_OFF_FILE} is there: the master and the services run from ${bundlePath}`); return }
+ let written: { lean: LeanBundle; path: string } | null
+ try {
+ const lean = readLeanBundle(deps.read(bundlePath))
+ written = lean ? { lean, path: deps.write(lean) } : null
+ } catch (error) {
+ fromBundle(`the lean bundle could not be written out (${error instanceof Error ? error.message : String(error)}): the master and the services run from ${bundlePath}`)
+ return
+ }
+ if (!written) { fromBundle(`no lean bundle in ${bundlePath}: the master and the services run from it`); return }
+ const { lean, path: leanPath } = written
+ const fingerprint = leanFingerprint(lean)
+ const leanEnv = { ...deps.env, [BUNDLE_ENV]: bundlePath, [BUNDLE_SHA256_ENV]: lean.bundleSha256, [LEAN_FINGERPRINT_ENV]: fingerprint }
+ const probed = deps.probe(leanPath, leanEnv)
+ if (!probed.ok) {
+ fromBundle(`the lean bundle ${leanPath} did not answer its probe (${probed.detail}): the master and the services run from ${bundlePath}`)
+ return
+ }
+ if (!deps.execve) {
+ fromBundle(`this Node cannot re-execute the master: it runs from ${bundlePath}, the services from ${leanPath}`, { path: leanPath, fingerprint })
+ return
+ }
+ // An exec that fails cannot be caught once it has begun: on Node 22.23 a node binary that is not there
+ // aborts this process (exit 134), and a script that is not there ends it in the new image
+ // (MODULE_NOT_FOUND). The probe ran both a moment ago; they are checked again right before.
+ const missing = [process.execPath, leanPath].find((path) => !deps.exists(path))
+ if (missing) {
+ fromBundle(`${missing} is not there to re-execute on: the master and the services run from ${bundlePath}`)
+ return
+ }
+ try {
+ deps.execve(process.execPath, [process.execPath, ...process.execArgv, leanPath, '__harnessd'], leanEnv)
+ } catch (error) {
+ // Only what `process.execve` refuses before it begins: arguments it cannot take.
+ fromBundle(`the master could not re-execute on ${leanPath} (${error instanceof Error ? error.message : String(error)}): it runs from ${bundlePath}, the services from ${leanPath}`, { path: leanPath, fingerprint })
+ }
+}
diff --git a/cli/src/serviceProcess.spec.ts b/cli/src/serviceProcess.spec.ts
new file mode 100644
index 000000000..2c3a55a81
--- /dev/null
+++ b/cli/src/serviceProcess.spec.ts
@@ -0,0 +1,56 @@
+import { afterEach, describe, expect, it, vi } from 'vitest'
+import { SERVICE_RUNNERS, startServiceProcess, type ServiceProcessOptions } from './serviceProcess.js'
+import { KNOWN_SERVICES } from './harnessd/services.js'
+
+const local = vi.hoisted(() => ({ socket: '/data/daemon-18473.sock' as string | null }))
+vi.mock('./lib/localSocket.js', async (real) => ({ ...await real<object>(), localSocketPath: () => local.socket }))
+// The test's own console stays as it is: it is shared with every other test in this worker.
+const stamped = vi.hoisted(() => ({ times: 0 }))
+vi.mock('./lib/log.js', async (real) => ({ ...await real<object>(), installTimestampedConsole: () => { stamped.times++ } }))
+
+describe('a service in its own process', () => {
+ const title = process.title
+ afterEach(() => { process.title = title; local.socket = '/data/daemon-18473.sock'; vi.restoreAllMocks() })
+
+ it('can run every service the master knows, and loads each one\'s runner on its own', async () => {
+ expect([...SERVICE_RUNNERS.keys()].sort()).toEqual(Object.keys(KNOWN_SERVICES).sort())
+ for (const [name, load] of SERVICE_RUNNERS) expect(typeof await load(), name).toBe('function')
+ })
+
+ it('runs the one it is named, against the core\'s socket, with the master\'s token', async () => {
+ const seen: ServiceProcessOptions[] = []
+ const handle = { stop: () => {} }
+ const loaded: string[] = []
+ const runners = new Map([
+ ['search', async () => { loaded.push('search'); return (options: ServiceProcessOptions) => { seen.push(options); return handle } }],
+ ['viewers', async () => { loaded.push('viewers'); return () => handle }],
+ ])
+ process.env.HARNESSD_SERVICE_TOKEN = 'token'
+ try {
+ expect(await startServiceProcess('search', { runners })).toBe(handle)
+ } finally { delete process.env.HARNESSD_SERVICE_TOKEN }
+ expect(loaded).toEqual(['search'])
+ expect(process.title).toBe('harnessd search')
+ // Its lines are stamped like the core's and the master's in the log they share.
+ expect(stamped.times).toBe(1)
+ expect(seen).toEqual([{ dataDir: expect.any(String), socketPath: '/data/daemon-18473.sock', machineId: expect.any(String), token: 'token' }])
+ expect(await startServiceProcess('viewers', { runners })).toBe(handle)
+ })
+
+ it('refuses, with exit 2 and why, a service this build does not know or a core with no socket', async () => {
+ const error = vi.spyOn(console, 'error').mockImplementation(() => {})
+ const exit = vi.fn((code: number): never => { throw new Error(`exit ${code}`) })
+ await expect(startServiceProcess('nope', { runners: new Map(), exit })).rejects.toThrow('exit 2')
+ await expect(startServiceProcess(undefined, { runners: new Map(), exit })).rejects.toThrow('exit 2')
+ local.socket = null
+ await expect(startServiceProcess('search', { exit })).rejects.toThrow('exit 2')
+ expect(error.mock.calls.map((call) => call[0])).toEqual([
+ '[service] nope: no such service in this build',
+ '[service] (none): no such service in this build',
+ '[service] search: the core has no local socket to reach',
+ ])
+ const processExit = vi.spyOn(process, 'exit').mockImplementation(((code: number) => { throw new Error(`process.exit ${code}`) }) as never)
+ await expect(startServiceProcess('nope', { runners: new Map() })).rejects.toThrow('process.exit 2')
+ expect(processExit).toHaveBeenCalledWith(2)
+ })
+})
diff --git a/cli/src/serviceProcess.ts b/cli/src/serviceProcess.ts
new file mode 100644
index 000000000..57d2d9851
--- /dev/null
+++ b/cli/src/serviceProcess.ts
@@ -0,0 +1,61 @@
+/**
+ * A service in its own process (`__service <name>`), as harnessd's master starts it.
+ *
+ * Its own module, so the bundle's entry (entry.ts) can start a service that evaluates only that service:
+ * each runner is imported by the process that runs it, and a search process never evaluates the
+ * viewers. Isolation is the point of the split, and it cost the whole CLI per process while every one
+ * evaluated the whole bundle: 115 to 160 MiB resident each at idle, measured from the bundle
+ * (2026-10-05).
+ */
+import { env } from './config/env.js'
+import { ensureUtf8Locale } from './lib/childLocale.js'
+import { readOrMintComputerId } from './lib/computerIdentity.js'
+import { localSocketPath } from './lib/localSocket.js'
+import { installTimestampedConsole } from './lib/log.js'
+import type { ServiceProcess } from './services/process.js'
+
+export interface ServiceProcessOptions {
+ dataDir: string
+ socketPath: string
+ machineId: string
+ token: string
+}
+type Runner = (options: ServiceProcessOptions) => ServiceProcess
+
+/** Every service this build can run in its own process, and how to load its runner alone. */
+export const SERVICE_RUNNERS: ReadonlyMap<string, () => Promise<Runner>> = new Map([
+ ['search', async () => (await import('./services/searchProcess.js')).runSearchService],
+ ['viewers', async () => (await import('./services/viewersProcess.js')).runViewersService],
+ ['workspaces', async () => (await import('./services/workspacesProcess.js')).runWorkspacesService],
+ ['teams', async () => (await import('./services/teamsProcess.js')).runTeamsService],
+])
+
+export interface ServiceProcessDeps {
+ runners?: ReadonlyMap<string, () => Promise<Runner>>
+ exit?: (code: number) => never
+}
+
+/** Run the service [name] until the master stops it; exits 2 for a name this build does not know. */
+export async function startServiceProcess(name: string | undefined, deps: ServiceProcessDeps = {}): Promise<ServiceProcess> {
+ const runners = deps.runners ?? SERVICE_RUNNERS
+ const exit = deps.exit ?? ((code: number) => process.exit(code))
+ const socketPath = localSocketPath(env.ADAPTER_DATA_DIR, env.PORT)
+ const load = name === undefined ? undefined : runners.get(name)
+ if (!load || !socketPath) {
+ console.error(`[service] ${name ?? '(none)'}: ${socketPath ? 'no such service in this build' : 'the core has no local socket to reach'}`)
+ return exit(2)
+ }
+ process.title = `harnessd ${name}`
+ // As every daemon process did at load: a service that runs `ps` or `git` must not get mangled output.
+ ensureUtf8Locale()
+ // Its lines go to the daemon's log between the core's and the master's, which are stamped: unstamped,
+ // a service's said nothing of when.
+ installTimestampedConsole()
+ const run = await load()
+ return run({
+ dataDir: env.ADAPTER_DATA_DIR,
+ socketPath,
+ machineId: readOrMintComputerId(env.ADAPTER_COMPUTER_ID_FILE, env.ADAPTER_COMPUTER_ID),
+ token: process.env.HARNESSD_SERVICE_TOKEN ?? '',
+ })
+}
diff --git a/cli/src/services/AGENTS.md b/cli/src/services/AGENTS.md
index 54f20dd0b..71ec5ac10 100644
--- a/cli/src/services/AGENTS.md
+++ b/cli/src/services/AGENTS.md
@@ -31,7 +31,8 @@ let several people build features at once without touching the core or each othe
5. **Test it to 100%** with `fakeCore()` (`src/testing/fakeCore.ts`); `npm run test:core` covers this
folder. Prove failure isolation end to end with `HARNESSD_TEST_FAULTS=<name>` (its start fails),
`<name>.<member>` (one port call fails) and `<name>.<request type>` (one request fails); see
- `e2e/services.e2e.ts`.
+ `e2e/services.e2e.ts`. The same names work in a service's own process, where `<name>.<event kind>`
+ fails an event; `<name>.crash` and `<name>.leak` exist only there (`src/services/process.ts`).
`store.ts` (the Harness Store: no port, four requests) is the example to copy for a feature;
`search.ts` for a service the core also calls.
@@ -39,11 +40,25 @@ let several people build features at once without touching the core or each othe
## Running in its own process
A service that can crash natively, hang or leak should run in its own process, where it costs only
-itself. Search, the viewers, workspaces and the teams' prompt scopes do:
-`HARNESSD_SERVICES=search,viewers,workspaces,teams` (any of them alone, too).
+itself. Search, the viewers, workspaces and the teams' prompt scopes do, by default: every service in
+`KNOWN_SERVICES` runs in its own process unless `HARNESSD_SERVICES` names a subset
+(`search,viewers`), and `HARNESSD_SERVICES=none` runs them all inside the core's process (for debugging
+or a quick way back).
- `src/harnessd/services.ts` runs it (`KNOWN_SERVICES`, with its memory budget). The core routes the
requests it declared (its `<NAME>_REQUESTS`) to its process, and the same handlers answer them there.
+- `src/serviceProcess.ts` starts it (`SERVICE_RUNNERS`, which must name every service in
+ `KNOWN_SERVICES`): a process imports only the runner it is named. From a release, it runs on the lean
+ bundle cli.js carries for the master and the services (`src/harnessd/leanBundle.ts`), split so that a
+ service loads its own code and nothing else: 61 to 77 MiB resident at idle (20 to 35 MiB physical
+ footprint), against 118 to 131 (54 to 90) when each started on cli.js (2026-10-05). It is only an
+ optimisation: the master starts a service from cli.js whenever the lean bundle cannot be used
+ (`src/harnessd/leanServices.ts`), and the release script refuses one that does not load
+ (`scripts/check-lean-bundle.mjs`).
+- **What a service imports is what its process costs.** Import from small modules: one schema module
+ pulled in for a constant brought zod to search and workspaces, 8 MiB each (`src/dsh/id.ts`). A
+ failing import fails the service's start, loudly, and the master parks it. `src/leanEntry.spec.ts`
+ holds each process to its own code, and the master, search and workspaces to no zod.
- `src/services/process.ts` is the process's side: heartbeats to the master, the connection to the
core with the master's token, reconnecting after core restarts.
- `src/core/serviceLinks.ts` is the core's side: it routes those requests to the process, answers
diff --git a/cli/src/services/process.spec.ts b/cli/src/services/process.spec.ts
index 4efe2de8a..b70c30207 100644
--- a/cli/src/services/process.spec.ts
+++ b/cli/src/services/process.spec.ts
@@ -102,9 +102,11 @@ describe('a service in its own process', () => {
await vi.waitFor(() => expect(socket().sent).toHaveLength(4))
expect(socket().sent.slice(1)).toEqual(expect.arrayContaining([
{ type: 'session_search_result', payload: { hits: ['zebra'], requestId: 'r1' } },
- { type: 'session_tail_result', payload: { error: 'SERVICE_FAILED', detail: 'index gone', requestId: 'r2' } },
- { type: 'session_other_result', payload: { error: 'SERVICE_FAILED', detail: 'not an error', requestId: 'r3' } },
+ { type: 'session_tail_result', payload: { error: 'SERVICE_FAILED', service: 'search', requestId: 'r2' } },
+ { type: 'session_other_result', payload: { error: 'SERVICE_FAILED', service: 'search', requestId: 'r3' } },
]))
+ // What went wrong is logged here, as the core's host logs it, and not told to whoever asked.
+ expect(lines).toEqual(expect.arrayContaining(['[service search] session_tail failed · index gone', '[service search] session_other failed · not an error']))
})
it('tells each handler who asked, as the core established it, and the least it could be when that is missing', async () => {
@@ -190,9 +192,57 @@ describe('a service in its own process', () => {
expect(socket().sent).toEqual([])
})
- it('reads its test faults from the environment, its own only', () => {
- expect([...serviceFaults({ HARNESSD_TEST_FAULTS: 'search.crash, search.hang,search.leak,devices.crash,search.other,search' }, 'search')]).toEqual(['crash', 'leak'])
- expect(serviceFaults({}, 'search').size).toBe(0)
+ it('reads its test faults from the environment, its own only, in the names the core\'s host takes', () => {
+ expect(serviceFaults({ HARNESSD_TEST_FAULTS: 'search.crash, search.hang,search.leak,devices.crash,search.session_search,search' }, 'search'))
+ .toEqual({ start: true, crash: true, leak: true, calls: new Set(['hang', 'session_search']) })
+ expect(serviceFaults({ HARNESSD_TEST_FAULTS: 'devices,searching' }, 'search')).toEqual({ start: false, crash: false, leak: false, calls: new Set() })
+ expect(serviceFaults({}, 'search')).toEqual({ start: false, crash: false, leak: false, calls: new Set() })
+ })
+
+ it('fails its start when a test asks it to, before it beats or connects, as a service whose start throws', () => {
+ const channel = new FakeChannel()
+ expect(() => run({ channel, env: { HARNESSD_TEST_FAULTS: 'search' } })).toThrow('injected fault: search')
+ expect(channel.beats).toEqual([])
+ expect(sockets).toEqual([])
+ })
+
+ it('fails a request or an event on every call when a test asks it to, and answers and hears the rest', async () => {
+ const onEvent = vi.fn()
+ const requests = { session_search: vi.fn(() => ({ hits: [] })), session_tail: vi.fn(() => ({ tail: [] })) }
+ run({ requests, onEvent, env: { HARNESSD_TEST_FAULTS: 'search.session_search,search.touch' } })
+ socket().open()
+ for (const id of ['r1', 'r2']) socket().say({ type: 'session_search', payload: { requestId: id } })
+ socket().say({ type: 'session_tail', payload: { requestId: 'r3' } })
+ socket().say({ type: 'service_event', payload: { kind: 'touch', sessionId: 's1' } })
+ socket().say({ type: 'service_event', payload: { kind: 'deleteHistory', sessionId: 's1' } })
+ await vi.waitFor(() => expect(socket().sent).toHaveLength(4))
+ expect(socket().sent.slice(1)).toEqual(expect.arrayContaining([
+ { type: 'session_search_result', payload: { error: 'SERVICE_FAILED', service: 'search', requestId: 'r1' } },
+ { type: 'session_search_result', payload: { error: 'SERVICE_FAILED', service: 'search', requestId: 'r2' } },
+ { type: 'session_tail_result', payload: { tail: [], requestId: 'r3' } },
+ ]))
+ expect(requests.session_search).not.toHaveBeenCalled()
+ expect(onEvent.mock.calls).toEqual([[{ kind: 'deleteHistory', sessionId: 's1' }]])
+ expect(lines).toEqual(expect.arrayContaining([
+ '[service search] session_search failed · injected fault: search.session_search',
+ '[service search] touch failed · injected fault: search.touch',
+ ]))
+ })
+
+ it('logs an event that throws or rejects, and goes on to the next one', async () => {
+ const heard: unknown[] = []
+ run({ onEvent: (payload) => {
+ heard.push(payload.kind)
+ if (payload.kind === 'throws') throw new Error('bad row')
+ if (payload.kind === 'rejects') return Promise.reject(new Error('index closed'))
+ if (payload.kind === undefined) throw 'not an error'
+ } })
+ socket().open()
+ for (const kind of ['throws', 'rejects', undefined, 'fine']) socket().say({ type: 'service_event', payload: kind ? { kind } : {} })
+ await vi.waitFor(() => expect(lines).toContain('[service search] rejects failed · index closed'))
+ expect(heard).toEqual(['throws', 'rejects', undefined, 'fine'])
+ expect(lines).toEqual(expect.arrayContaining(['[service search] throws failed · bad row', '[service search] event failed · not an error']))
+ expect(exits).toEqual([])
})
it('crashes or leaks on purpose when a test asks it to', () => {
diff --git a/cli/src/services/process.ts b/cli/src/services/process.ts
index f69182aa6..674429040 100644
--- a/cli/src/services/process.ts
+++ b/cli/src/services/process.ts
@@ -30,8 +30,8 @@ export interface ServiceProcessOptions {
token: string
/** The requests the core routes here, by type: each answered under `<type>_result`. */
requests: ServiceRequests
- /** What the core tells this service (`service_event`). */
- onEvent?: (payload: Payload) => void
+ /** What the core tells this service (`service_event`). A throw, or a rejection of what it returns, is logged. */
+ onEvent?: (payload: Payload) => void | Promise<unknown>
/** Each time it is connected to a core: the first time, and after every core restart. */
onConnected?: (core: CoreConnection) => void
/** The master's spawn channel; this process when absent. */
@@ -46,20 +46,37 @@ export interface ServiceProcessOptions {
maxBackoffMs?: number
}
+/** What a test asked this service's process to do wrong. */
+export interface ServiceFaults {
+ start: boolean
+ crash: boolean
+ leak: boolean
+ /** The requests and events that fail on every call. */
+ calls: ReadonlySet<string>
+}
+
/**
- * `HARNESSD_TEST_FAULTS` for a service's own process, as the end-to-end suite uses them: `<name>.crash`
- * exits soon after start, `<name>.leak` keeps allocating memory it never lets go. Each is what the master
- * must survive. (A hang needs no fault: a stopped process — SIGSTOP — beats no more than a hung one.)
+ * `HARNESSD_TEST_FAULTS` for a service's own process, as the end-to-end suite uses them. The names the
+ * core's host takes mean the same here (core/serviceHost.ts), so one test proves a guarantee whichever
+ * process the service runs in: `<name>` fails its start, `<name>.<request or event>` fails that request
+ * or event on every call. Two only a process has: `<name>.crash` exits soon after start, `<name>.leak`
+ * keeps allocating memory it never lets go. Each is what the master, or the core, must survive. (A hang
+ * needs no fault: a stopped process — SIGSTOP — beats no more than a hung one.)
*/
-export function serviceFaults(env: NodeJS.ProcessEnv, name: string): Set<'crash' | 'leak'> {
- const faults = new Set<'crash' | 'leak'>()
+export function serviceFaults(env: NodeJS.ProcessEnv, name: string): ServiceFaults {
+ const faults = { start: false, crash: false, leak: false, calls: new Set<string>() }
for (const entry of (env.HARNESSD_TEST_FAULTS ?? '').split(',')) {
const [service, fault] = entry.trim().split('.')
- if (service === name && (fault === 'crash' || fault === 'leak')) faults.add(fault)
+ if (service !== name) continue
+ if (fault === undefined) faults.start = true
+ else if (fault === 'crash' || fault === 'leak') faults[fault] = true
+ else faults.calls.add(fault)
}
return faults
}
+const describe = (error: unknown): string => (error instanceof Error ? error.message : String(error))
+
export interface ServiceProcess {
stop(): void
}
@@ -68,6 +85,9 @@ export interface ServiceProcess {
export function runServiceProcess(options: ServiceProcessOptions): ServiceProcess {
const channel: MasterChannel = options.channel ?? (process as unknown as MasterChannel)
const env = options.env ?? process.env
+ const faults = serviceFaults(env, options.name)
+ // As a service whose start throws: this process ends, and the master decides whether to try again.
+ if (faults.start) throw new Error(`injected fault: ${options.name}`)
const connect = options.connect ?? ((url: string) => new WebSocket(url))
const exit = options.exit ?? ((code: number) => process.exit(code))
const log = options.log ?? ((line: string) => console.log(line))
@@ -88,10 +108,9 @@ export function runServiceProcess(options: ServiceProcessOptions): ServiceProces
}
const beating = channel.send ? setInterval(beat, heartbeatInterval(env)) : null
if (channel.send) beat()
- const faults = serviceFaults(env, options.name)
const leaked: Buffer[] = []
- if (faults.has('crash')) setTimeout(() => exit(1), 200)
- const leaking = faults.has('leak') ? setInterval(() => { leaked.push(Buffer.alloc(8 * 1024 * 1024, 1)); leaked.push(Buffer.from(new Array(200_000).fill('x').join(''))) }, 100) : null
+ if (faults.crash) setTimeout(() => exit(1), 200)
+ const leaking = faults.leak ? setInterval(() => { leaked.push(Buffer.alloc(8 * 1024 * 1024, 1)); leaked.push(Buffer.from(new Array(200_000).fill('x').join(''))) }, 100) : null
// The master is gone: so is this service, rather than an orphan holding what it holds.
channel.once('disconnect', () => { stop(); exit(0) })
@@ -118,7 +137,17 @@ export function runServiceProcess(options: ServiceProcessOptions): ServiceProces
options.onConnected?.(core)
return
}
- if (frame.type === 'service_event') { options.onEvent?.(payload); return }
+ if (frame.type === 'service_event') {
+ // An event that fails is this service's alone: logged, and the next one handled as usual. Thrown
+ // out of the socket's listener it ended the process, and every request in flight with it.
+ const kind = typeof payload.kind === 'string' ? payload.kind : 'event'
+ const failed = (error: unknown): void => log(`[service ${options.name}] ${kind} failed · ${describe(error)}`)
+ try {
+ if (faults.calls.has(kind)) throw new Error(`injected fault: ${options.name}.${kind}`)
+ void Promise.resolve(options.onEvent?.(payload)).catch(failed)
+ } catch (error) { failed(error) }
+ return
+ }
if (frame.type === 'service_query_result') {
const waiting = pending.get(String(payload.requestId))
if (!waiting) return
@@ -133,10 +162,17 @@ export function runServiceProcess(options: ServiceProcessOptions): ServiceProces
const requestId = payload.requestId
// Who asked, as the core established it; read as the least it could be if it is missing.
const asker = { local: frame.asker?.local === true, owner: frame.asker?.owner === true }
- // A request that fails here is answered as failed, never left for the core's timeout.
+ // A request that fails here is answered as failed, never left for the core's timeout, and in the
+ // words the core's host uses: what went wrong goes to the log, not to whoever asked.
void Promise.resolve()
- .then(() => handle(payload, asker))
- .catch((error: unknown) => ({ error: 'SERVICE_FAILED', detail: error instanceof Error ? error.message : String(error) }))
+ .then(() => {
+ if (faults.calls.has(type)) throw new Error(`injected fault: ${options.name}.${type}`)
+ return handle(payload, asker)
+ })
+ .catch((error: unknown) => {
+ log(`[service ${options.name}] ${type} failed · ${describe(error)}`)
+ return { error: 'SERVICE_FAILED', service: options.name }
+ })
.then((result) => send({ type: `${type}_result`, payload: { ...result, requestId } }))
}
diff --git a/cli/src/services/searchProcess.ts b/cli/src/services/searchProcess.ts
index de48fd4e2..08e95a079 100644
--- a/cli/src/services/searchProcess.ts
+++ b/cli/src/services/searchProcess.ts
@@ -98,9 +98,10 @@ export function runSearchService(options: SearchServiceOptions): ServiceProcess
const sessionId = typeof payload.sessionId === 'string' ? payload.sessionId : ''
if (!sessionId || !index) return
// A turn boundary: the agents may have changed (a new one, a new conversation), and this
- // session has new turns to index.
- if (payload.kind === 'touch') void refresh().then(() => index.touch(sessionId))
- else if (payload.kind === 'deleteHistory') index.deleteHistory(sessionId)
+ // session has new turns to index. Returned, so a failure is logged rather than left unhandled,
+ // which would end this process.
+ if (payload.kind === 'touch') return refresh().then(() => index.touch(sessionId))
+ if (payload.kind === 'deleteHistory') index.deleteHistory(sessionId)
},
onConnected: (connection) => {
core = connection
diff --git a/docs/design/2026-10-03-harnessd.md b/docs/design/2026-10-03-harnessd.md
index 6a8fd046c..1ad8d53cf 100644
--- a/docs/design/2026-10-03-harnessd.md
+++ b/docs/design/2026-10-03-harnessd.md
@@ -508,8 +508,31 @@ Done so far:
The funnel calls the teams and the device service the same way. `HARNESSD_TEST_FAULTS` injects
faults, and `e2e/services.e2e.ts` proves on the real daemon that agents bind, run their turns and
come back after a restart with every service failing to start or failing on every call.
-- **Services in their own processes** (step 4, started; off until `HARNESSD_SERVICES` names one). The
- master runs each named service as `harness __service <name>` beside the core
+- **Lean processes** (2026-10-05). Node parses all of the file a process starts on, and every daemon
+ process started on the whole 4.4 MB cli.js: at idle, from the bundle, the master took 164 MiB resident
+ and each service 118 to 131, 839 MiB with the core. So the build bundles the master and the services a
+ second time on their own (`cli/src/leanEntry.ts`), split into files so that each process loads only
+ its own code and the shared code it imports, and appends them to cli.js as a comment
+ (`cli/scripts/lib/leanBlock.mjs`). A master started on cli.js writes them into `lean/<sha>/` in the
+ data folder, checks they answer the probe a re-executing master asks, and re-executes on them, same
+ pid; it starts the services from them too (`cli/src/masterProcess.ts`, `cli/src/harnessd/leanBundle.ts`).
+ Each master claims the folder it runs from, and a folder is removed only once no live master claims
+ it, so two builds on one data folder never take each other's. It is only ever an optimisation: a
+ service starts from cli.js whenever its master's lean files are gone or changed, cli.js is no longer
+ the bundle they came from (an update the master did not re-execute on), or it died twice from them
+ before beating (`cli/src/harnessd/leanServices.ts`); `lean-off` in the data folder turns it off for a
+ master launchd or systemd starts; and the release script refuses a lean bundle whose master does not
+ answer its probe or whose modules do not load (`cli/scripts/check-lean-bundle.mjs`). The environment is read without zod
+ (`cli/src/config/envFields.ts`), which every process loads and which cost each 15 to 19 MiB. The core
+ and the CLI run from cli.js. The release is still the one cli.js the updater downloads, verifies and
+ swaps. Where Node cannot re-execute (before 22.15), the master runs from cli.js and the services still
+ from the lean bundle; without a lean bundle that answers, everything runs from cli.js, as with
+ `HARNESSD_LEAN=off`. Measured at idle, the median of three runs: the master 55 MiB resident (16 MiB
+ physical footprint, from 113), search 77 (35), the viewers 69 (29), workspaces 61 (20), the teams 66
+ (26); 465 MiB with the core, against 839. `e2e/lean.e2e.ts` proves it on a real release bundle.
+- **Services in their own processes** (step 4; on by default since 2026-10-05, `HARNESSD_SERVICES=none`
+ to run them inside the core's process). The master runs each service as `harness __service <name>`
+ beside the core
(`cli/src/harnessd/services.ts`): it restarts one that crashes, with backoff from 1 s to 60 s; kills
one that stops beating (hung) or outgrows its memory budget (leaking); and parks one that ends five
times in ten minutes, trying it again after thirty. A service reaches the core through its local