Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

harnessd: services answer their own requests; the store becomes a service (#758)

Dee1ef1f3480ad3
diff --git a/cli/AGENTS.md b/cli/AGENTS.md
index 3f3788152..e92c155f7 100644
--- a/cli/AGENTS.md
+++ b/cli/AGENTS.md
@@ -23,6 +23,7 @@ together. `src/backendSocket.ts` is the transport: it receives frames and dispat
| You are adding | Put it in | Not in |
|---|---|---|
| A new feature (anything a session can run without) | a new service, `src/services/<name>.ts` | the core, `cli.ts`, `backendSocket.ts` |
+| A request the apps send to a feature | the service's start returns its handler ([src/services/AGENTS.md](src/services/AGENTS.md)) | a case in `backendSocket.ts`, a slot on `BackendSocket` |
| Behaviour of agents, terminals, transcripts, turns, input or questions | the module under `src/core/` that owns it | `cli.ts` |
| Support for an engine (Claude Code, Codex, …) | `src/engines/<engine>/` | the core |
| A pure helper with no daemon state | `src/lib/` | the core |
diff --git a/cli/e2e/services.e2e.ts b/cli/e2e/services.e2e.ts
index 80769fa5c..76efef685 100644
--- a/cli/e2e/services.e2e.ts
+++ b/cli/e2e/services.e2e.ts
@@ -38,17 +38,21 @@ describe('a failing service never takes the core down', () => {
let daemon: IsolatedDaemon | undefined
afterEach(async () => { await daemon?.close(); daemon = undefined })
- it('with every service failing to start, the core starts, runs an agent through turns and a restart, and says search is off', async () => {
- daemon = await IsolatedDaemon.create({ env: { HARNESSD_TEST_FAULTS: 'search,viewers,models,workspaces' } })
+ it('with every service failing to start, the core starts, runs an agent through turns and a restart, and says each service is off', async () => {
+ daemon = await IsolatedDaemon.create({ env: { HARNESSD_TEST_FAULTS: 'search,viewers,models,workspaces,store' } })
onTestFailed(() => { console.log(`---- daemon log\n${daemon?.log().split('\n').slice(-80).join('\n')}`) })
await daemon.start()
- for (const service of ['search', 'viewers', 'models', 'workspaces']) {
+ for (const service of ['search', 'viewers', 'models', 'workspaces', 'store']) {
expect(daemon.log()).toContain(`[services] ${service} did not start · injected fault: ${service} · the core runs without it`)
}
const client = await LocalClient.connect(daemon)
const agentId = await boundAgent(daemon, client, 'no-services')
await turn(client, agentId, 'first, with no services')
- expect((await client.request('session_search', { query: 'first' })).error).toBe('SEARCH_UNAVAILABLE')
+ // Off, never UNSUPPORTED: the apps read that as "update the CLI".
+ expect(await client.request('session_search', { query: 'first' })).toMatchObject({ error: 'SERVICE_UNAVAILABLE', service: 'search', retryable: false })
+ for (const type of ['dsh_list', 'dsh_install', 'dsh_update', 'dsh_remove']) {
+ expect(await client.request(type, { id: 'acme/thing' }), type).toMatchObject({ error: 'SERVICE_UNAVAILABLE', service: 'store', retryable: false })
+ }
await daemon.restart()
const again = await LocalClient.connect(daemon)
@@ -63,7 +67,7 @@ describe('a failing service never takes the core down', () => {
it('with services failing on every call, they are switched off and every turn still reaches the client', async () => {
daemon = await IsolatedDaemon.create({
- env: { HARNESSD_TEST_FAULTS: 'search.touch,search.search,viewers.frameContext,viewers.attach,workspaces.nameBranches' },
+ env: { HARNESSD_TEST_FAULTS: 'search.touch,search.session_search,viewers.frameContext,viewers.attach,workspaces.nameBranches' },
})
onTestFailed(() => { console.log(`---- daemon log\n${daemon?.log().split('\n').slice(-120).join('\n')}`) })
await daemon.start()
@@ -79,7 +83,7 @@ describe('a failing service never takes the core down', () => {
expect(daemon.log()).toContain('[services] viewers.frameContext failed · injected fault: viewers.frameContext')
// Off, search says so; the agent's row still carries everything the core owns.
- expect((await client.request('session_search', { query: 'turn' })).error).toBe('SEARCH_UNAVAILABLE')
+ expect(await client.request('session_search', { query: 'turn' })).toMatchObject({ error: 'SERVICE_UNAVAILABLE', service: 'search' })
const agent = await row(client, agentId)
expect(agent?.sessionId).toBeTruthy()
expect(agent?.status).not.toBe('stopped')
@@ -108,18 +112,43 @@ describe('a failing service never takes the core down', () => {
})
it('a search request that fails before search is switched off answers that request, and the next one too', async () => {
- daemon = await IsolatedDaemon.create({ env: { HARNESSD_TEST_FAULTS: 'search.search' } })
+ daemon = await IsolatedDaemon.create({ env: { HARNESSD_TEST_FAULTS: 'search.session_search' } })
onTestFailed(() => { console.log(`---- daemon log\n${daemon?.log().split('\n').slice(-80).join('\n')}`) })
await daemon.start()
const client = await LocalClient.connect(daemon)
- const first = await client.request('session_search', { query: 'anything' })
- expect(first).toMatchObject({ error: 'SERVICE_UNAVAILABLE', service: 'search', retryable: true })
const answers = []
- for (let i = 0; i < 5; i++) answers.push((await client.request('session_search', { query: `again ${i}` })).error)
- // The fifth failure switched search off: from then on the request says search is off.
- expect(answers.slice(0, 3)).toEqual(['SERVICE_UNAVAILABLE', 'SERVICE_UNAVAILABLE', 'SERVICE_UNAVAILABLE'])
- expect(answers.at(-1)).toBe('SEARCH_UNAVAILABLE')
+ for (let i = 0; i < 6; i++) answers.push(await client.request('session_search', { query: `ask ${i}` }))
+ // Each failure answers its own request; the fifth switches search off, and from then on the
+ // request says search is off. Its other request, and the core's, go on.
+ expect(answers.slice(0, 5)).toEqual(Array(5).fill(expect.objectContaining({ error: 'SERVICE_FAILED', service: 'search' })))
+ expect(answers[5]).toMatchObject({ error: 'SERVICE_UNAVAILABLE', service: 'search', retryable: false })
+ expect(daemon.log()).toContain('[services] search.session_search failed · injected fault: search.session_search')
expect((await client.request('agents_list', {})).agents).toEqual([])
client.close()
})
+
+ it('the store answers from its own service; one of its requests failing leaves its others and the core alone', async () => {
+ daemon = await IsolatedDaemon.create({ env: { HARNESSD_TEST_FAULTS: 'store.dsh_list' } })
+ onTestFailed(() => { console.log(`---- daemon log\n${daemon?.log().split('\n').slice(-80).join('\n')}`) })
+ await daemon.start()
+ const client = await LocalClient.connect(daemon)
+ expect(await client.request('dsh_list', {})).toMatchObject({ error: 'SERVICE_FAILED', service: 'store' })
+ // Its other requests are still the store's own answers, refusals included.
+ expect(await client.request('dsh_remove', { id: '../../etc' })).toMatchObject({ error: 'INVALID_DSH', detail: 'dsh_remove needs an id' })
+ expect(await client.request('dsh_install', { url: 'https://example.com/\n' })).toMatchObject({ error: 'INVALID_DSH' })
+ const agentId = await boundAgent(daemon, client, 'store-failing')
+ await turn(client, agentId, 'with the store failing')
+ expect(daemon.coresStarted()).toBe(1)
+ client.close()
+ })
+
+ it('the store lists what is installed here and what the registry offers', async () => {
+ daemon = await IsolatedDaemon.create()
+ await daemon.start()
+ const client = await LocalClient.connect(daemon)
+ const listed = await client.request('dsh_list', {}, 60_000)
+ expect(listed.error, JSON.stringify(listed).slice(0, 400)).toBeUndefined()
+ expect(Array.isArray(listed.dsh)).toBe(true)
+ client.close()
+ })
})
diff --git a/cli/src/architecture.spec.ts b/cli/src/architecture.spec.ts
index 41a3f8680..68bc1543f 100644
--- a/cli/src/architecture.spec.ts
+++ b/cli/src/architecture.spec.ts
@@ -57,11 +57,12 @@ function runForegroundLines(): number {
}
/**
- * The most each may grow to. Today's size and a little room for wiring. Raising a budget needs a reason
- * a reviewer agrees with; the usual one is wrong, and the code belongs in a module or a service.
+ * The most each may grow to: its size when it last shrank, and a little room for wiring. Lower a budget
+ * when you move code out; raising one needs a reason a reviewer agrees with, and the usual one is wrong:
+ * the code belongs in a module or a service.
*/
-const RUN_FOREGROUND_BUDGET = 2_700
-const BACKEND_SOCKET_BUDGET = 3_600
+const RUN_FOREGROUND_BUDGET = 2_690
+const BACKEND_SOCKET_BUDGET = 3_500
/** Exceptions, each with its reason. Keep this short. */
const SERVICE_MAY_IMPORT: Record<string, string> = {
diff --git a/cli/src/backendSocket.dsh.spec.ts b/cli/src/backendSocket.dsh.spec.ts
index babc8ca0c..59a5c01e8 100644
--- a/cli/src/backendSocket.dsh.spec.ts
+++ b/cli/src/backendSocket.dsh.spec.ts
@@ -1,19 +1,36 @@
-// dsh_list, dsh_install and dsh_remove through the socket a local client (the desktop app) talks to:
-// the replies it gets, the status it is pushed, and what it gets when this daemon cannot do it.
+// dsh_list, dsh_install and dsh_remove through the socket a local client (the desktop app) talks to,
+// answered by the store service (services/store.ts) the way the daemon runs it: the replies the client
+// gets, the status it is pushed, and what it gets while the store is off.
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { BackendSocket } from './backendSocket.js'
import { env } from './config/env.js'
+import { emptyPorts } from './core/api.js'
+import { createServiceHost } from './core/serviceHost.js'
+import { refreshDshRegistry } from './dsh/catalog.js'
import { dshInstallDir, invalidateInstalledDsh, upsertInstalledRecord } from './dsh/installed.js'
import { resetBundledDshRegistry } from './dsh/registry.js'
+import { dshListRows } from './dsh/wire.js'
+import { STORE_REQUESTS, startStore, type StoreDeps } from './services/store.js'
+import { fakeCore } from './testing/fakeCore.js'
describe('the DSH requests on the local socket', () => {
let socket: BackendSocket
let frames: Array<{ type: string; payload: Record<string, unknown> }>
let root: string
let savedDshDir: string
+ // The store's install and remove, swapped per test; listing reads the real catalog and installs.
+ let mutate: StoreDeps['mutate']
+ let remove: StoreDeps['remove']
+ const serveStore = (faults: ReadonlySet<string> = new Set()): void => {
+ const host = createServiceHost(emptyPorts(), { log: () => {}, faults })
+ const deps: StoreDeps = { refresh: refreshDshRegistry, rows: dshListRows, mutate: (input, progress) => mutate(input, progress), remove: (id) => remove(id) }
+ const core = fakeCore({ clients: { dshInstallStatus: (status) => socket.send({ type: 'dsh_install_status', payload: status }) } })
+ host.serve('store', (api) => startStore(api, deps), core, STORE_REQUESTS)
+ socket.serviceRouter = (type, payload, asker, reply) => host.route(type, payload, asker, reply)
+ }
beforeEach(() => {
root = mkdtempSync(join(tmpdir(), 'dsh-socket-'))
savedDshDir = env.DSH_DIR
@@ -22,6 +39,9 @@ describe('the DSH requests on the local socket', () => {
socket = new BackendSocket('token')
frames = []
socket.registerLocalClient('local:store', { sendFrame: (frame) => { frames.push(frame as (typeof frames)[number]); return true }, sendBinary: () => true })
+ mutate = vi.fn(async () => ({ ok: true as const, id: 'acme/thing' }))
+ remove = vi.fn(() => ({ ok: true as const }))
+ serveStore()
})
afterEach(async () => {
await socket.unregisterLocalClient('local:store')
@@ -71,27 +91,45 @@ describe('the DSH requests on the local socket', () => {
.toEqual([['acme/thing', 'Thing here', true, true, 0], ['acme/other', 'Other', false, false, 1]])
})
- it('dsh_remove and dsh_install are refused on a daemon that cannot do them', async () => {
+ it('dsh_remove: uninstalls through the store, and refuses a malformed id before it hears of it', async () => {
+ const removed: string[] = []
+ remove = (id) => { removed.push(id); return id === 'autonomous/marp' ? { ok: true } : { ok: false, error: 'NOT_INSTALLED', detail: `${id} is not installed` } }
+ ask('dsh_remove', { requestId: 'rm-1', id: 'autonomous/marp' })
+ ask('dsh_remove', { requestId: 'rm-2', id: 'autonomous/none' })
+ ask('dsh_remove', { requestId: 'rm-3', id: '../../etc' })
+ await vi.waitFor(() => expect(replies('dsh_remove')).toHaveLength(3))
+ expect(replies('dsh_remove')).toEqual([
+ expect.objectContaining({ requestId: 'rm-1', ok: true, id: 'autonomous/marp' }),
+ expect.objectContaining({ requestId: 'rm-2', error: 'NOT_INSTALLED' }),
+ expect.objectContaining({ requestId: 'rm-3', error: 'INVALID_DSH' }),
+ ])
+ expect(removed).toEqual(['autonomous/marp', 'autonomous/none'])
+ })
+
+ it('dsh_remove, dsh_install and dsh_update say the store is unavailable while it is off, never UNSUPPORTED', async () => {
+ serveStore(new Set(['store']))
ask('dsh_remove', { requestId: 'rm-1', id: 'acme/thing' })
ask('dsh_install', { requestId: 'in-1', id: 'acme/thing' })
ask('dsh_update', { requestId: 'up-1', id: 'acme/thing' })
- await vi.waitFor(() => expect(replies('dsh_install')).toHaveLength(1))
- expect(replies('dsh_remove')).toEqual([expect.objectContaining({ requestId: 'rm-1', error: 'UNSUPPORTED_ON_REMOTE' })])
- expect(replies('dsh_install')).toEqual([expect.objectContaining({ requestId: 'in-1', error: 'UNSUPPORTED_ON_REMOTE' })])
- expect(replies('dsh_update')).toEqual([expect.objectContaining({ requestId: 'up-1', error: 'UNSUPPORTED_ON_REMOTE' })])
+ await vi.waitFor(() => expect(replies('dsh_update')).toHaveLength(1))
+ const off = { error: 'SERVICE_UNAVAILABLE', service: 'store', retryable: false }
+ expect(replies('dsh_remove')).toEqual([{ requestId: 'rm-1', ...off }])
+ expect(replies('dsh_install')).toEqual([{ requestId: 'in-1', ...off }])
+ expect(replies('dsh_update')).toEqual([{ requestId: 'up-1', ...off }])
+ expect(mutate).not.toHaveBeenCalled()
})
it('dsh_update validates identity, streams progress, and returns update failures', async () => {
- const update = vi.fn<NonNullable<BackendSocket['onDshUpdate']>>(async (id, progress) => {
+ const update = vi.fn<StoreDeps['mutate']>(async ({ id }, progress) => {
progress({ id: null, phase: 'clone' })
- progress({ id, phase: 'done' })
- return { ok: true, id }
+ progress({ id: id ?? null, phase: 'done' })
+ return { ok: true, id: id ?? '' }
})
- socket.onDshUpdate = update
+ mutate = update
ask('dsh_update', { requestId: 'invalid', id: '../../etc' })
ask('dsh_update', { requestId: 'update', id: 'acme/thing', url: 'ignored', ref: 'ignored' })
await vi.waitFor(() => expect(replies('dsh_update')).toHaveLength(2))
- expect(update).toHaveBeenCalledTimes(1)
+ expect(update).toHaveBeenCalledExactlyOnceWith({ id: 'acme/thing', update: true }, expect.any(Function))
expect(replies('dsh_update')).toEqual([
expect.objectContaining({ requestId: 'invalid', error: 'INVALID_DSH' }),
expect.objectContaining({ requestId: 'update', ok: true, id: 'acme/thing' }),
@@ -111,7 +149,7 @@ describe('the DSH requests on the local socket', () => {
it('dsh_install: refuses a request with no usable id or url, before the daemon hears of it', async () => {
const asked: unknown[] = []
- socket.onDshInstall = async (input) => { asked.push(input); return { ok: true, id: 'acme/thing' } }
+ mutate = async (input) => { asked.push(input); return { ok: true, id: 'acme/thing' } }
ask('dsh_install', { requestId: 'in-1', id: '../../etc', url: 'https://example.com/\n' })
await vi.waitFor(() => expect(replies('dsh_install')).toHaveLength(1))
expect(replies('dsh_install')[0]).toMatchObject({ requestId: 'in-1', error: 'INVALID_DSH', detail: 'dsh_install needs an id or a url' })
@@ -120,7 +158,7 @@ describe('the DSH requests on the local socket', () => {
it('dsh_install: pushes each phase under the id asked for, then replies with what was installed', async () => {
const asked: unknown[] = []
- socket.onDshInstall = async (input, progress) => {
+ mutate = async (input, progress) => {
asked.push(input)
progress({ id: null, phase: 'clone', detail: 'cloning' })
progress({ id: 'acme/thing', phase: 'doctor' })
@@ -137,13 +175,13 @@ describe('the DSH requests on the local socket', () => {
})
it('dsh_install: a failed install is its error and detail; one that throws is INTERNAL with what was thrown', async () => {
- socket.onDshInstall = async () => ({ ok: false, error: 'SETUP_FAILED', detail: 'setup exited 1' })
+ mutate = async () => ({ ok: false, error: 'SETUP_FAILED', detail: 'setup exited 1' })
ask('dsh_install', { requestId: 'in-3', url: 'https://example.com/thing.git' })
await vi.waitFor(() => expect(replies('dsh_install')).toHaveLength(1))
- socket.onDshInstall = async () => { throw new Error('disk full') }
+ mutate = async () => { throw new Error('disk full') }
ask('dsh_install', { requestId: 'in-4', url: 'https://example.com/thing.git' })
await vi.waitFor(() => expect(replies('dsh_install')).toHaveLength(2))
- socket.onDshInstall = () => Promise.reject('not an Error')
+ mutate = () => Promise.reject('not an Error')
ask('dsh_install', { requestId: 'in-5', url: 'https://example.com/thing.git' })
await vi.waitFor(() => expect(replies('dsh_install')).toHaveLength(3))
expect(replies('dsh_install')).toEqual([
diff --git a/cli/src/backendSocket.spec.ts b/cli/src/backendSocket.spec.ts
index dcdbc573b..3861584fe 100644
--- a/cli/src/backendSocket.spec.ts
+++ b/cli/src/backendSocket.spec.ts
@@ -22,7 +22,6 @@ import * as machineResources from './lib/machineResources.js'
import * as projectFolder from './lib/projectFolder.js'
import * as claudeTrust from './lib/claudeTrust.js'
import * as projectPreview from './lib/projectPreview.js'
-import * as storeCatalog from './dsh/catalog.js'
import * as opencodeVersion from './engines/opencode/version.js'
import { randomUUID } from 'node:crypto'
import { fakeGridAnswers, installFakeGrid, type FakeGrid } from './lib/__fixtures__/fakeGrid.js'
@@ -807,84 +806,69 @@ describe('BackendSocket outbound queue', () => {
await socket.stop()
})
- it('answers session_search from the index, sealed to the requester', async () => {
+ it('hands a request a service declared to the service router with who asked, and seals its answer to the requester', async () => {
const socket = new BackendSocket('token')
- const asked: Array<[string, number | undefined]> = []
- socket.sessionSearchProvider = (query, options) => {
- asked.push([query, options.limit])
- return { hits: [], indexed: 3, pending: 0, ready: true, tookMs: 1 }
+ const routed: Array<{ type: string; payload: Record<string, unknown>; asker: unknown }> = []
+ socket.serviceRouter = (type, payload, asker, reply) => {
+ routed.push({ type, payload, asker })
+ if (type !== 'session_search') return false
+ reply({ hits: [], indexed: 3, pending: 0, ready: true, tookMs: 1 })
+ return true
}
socket.connect()
const ws = wsMock.instances[0]
ws.open()
const unwrap = vi.spyOn(socket.e2ee, 'unwrapDown')
vi.spyOn(socket.e2ee, 'hasSession').mockReturnValue(true)
- const wrapReply = vi.spyOn(socket.e2ee, 'wrapRpcReply').mockReturnValue({
- type: 'session_search_result', payload: { __e2e: { v: 1, k: 's', n: 1, ct: 'ciphertext' } },
- })
+ const role = vi.spyOn(socket.e2ee, 'sessionRole').mockReturnValue('web')
+ const wrapReply = vi.spyOn(socket.e2ee, 'wrapRpcReply').mockImplementation((_connId, type) => ({
+ type, payload: { __e2e: { v: 1, k: 's', n: 1, ct: 'ciphertext' } },
+ }))
const envelope = { __e2e: { v: 1, k: 's', n: 1, ct: 'ciphertext' } }
- unwrap.mockReturnValueOnce({ type: 'session_search', payload: { requestId: 's-1', query: 'dial scroll', limit: 12 } })
- ws.message({ t: 'down', connId: 'web-1', frame: { type: 'session_search', payload: envelope } })
+ const ask = (type: string, payload: Record<string, unknown>) => {
+ unwrap.mockReturnValueOnce({ type, payload })
+ ws.message({ t: 'down', connId: 'web-1', frame: { type, payload: envelope } })
+ }
+
+ // The owner's paired app, over the relay: sealed back to it, under its own request id.
+ ask('session_search', { requestId: 's-1', query: 'dial scroll', limit: 12 })
await vi.waitFor(() => {
expect(wrapReply).toHaveBeenCalledWith('web-1', 'session_search_result', 's-1', { hits: [], indexed: 3, pending: 0, ready: true, tookMs: 1 })
})
- expect(asked).toEqual([['dial scroll', 12]])
+ expect(routed[0]).toEqual({ type: 'session_search', payload: { requestId: 's-1', query: 'dial scroll', limit: 12 }, asker: { local: false, owner: true } })
- // A machine whose Node has no node:sqlite says so rather than going silent.
- socket.sessionSearchProvider = null
- unwrap.mockReturnValueOnce({ type: 'session_search', payload: { requestId: 's-2', query: 'dial' } })
- ws.message({ t: 'down', connId: 'web-1', frame: { type: 'session_search', payload: envelope } })
- await vi.waitFor(() => {
- expect(wrapReply).toHaveBeenCalledWith('web-1', 'session_search_result', 's-2', { error: 'SEARCH_UNAVAILABLE' })
- })
+ // A device's session may not act as the owner, and the service is told so.
+ role.mockReturnValue('device')
+ ask('session_search', { requestId: 's-2', query: 'dial' })
+ await vi.waitFor(() => expect(routed).toHaveLength(2))
+ expect(routed[1].asker).toEqual({ local: false, owner: false })
- // A search service that failed (core/serviceHost.ts) answers this one request, and says the
- // client may ask again — never INTERNAL, and never the whole socket.
- const warn = vi.spyOn(console, 'warn').mockImplementation(() => {})
- socket.sessionSearchProvider = () => { throw new ServiceUnavailableError('search', new Error('disk I/O error')) }
- unwrap.mockReturnValueOnce({ type: 'session_search', payload: { requestId: 's-3', query: 'dial' } })
- ws.message({ t: 'down', connId: 'web-1', frame: { type: 'session_search', payload: envelope } })
- await vi.waitFor(() => {
- expect(wrapReply).toHaveBeenCalledWith('web-1', 'session_search_result', 's-3', { error: 'SERVICE_UNAVAILABLE', service: 'search', retryable: true })
- })
- expect(warn).toHaveBeenCalledWith('[backend] session_search: the search service is unavailable')
+ // A type no service declared is the socket's own: an unknown one is refused at once, in the clear,
+ // as a reply that carries nothing is.
+ role.mockReturnValue('web')
+ ask('nobody_answers', { requestId: 'n-1' })
+ await vi.waitFor(() => expect(parseSent(ws)).toContainEqual({ t: 'up', frame: { type: 'nobody_answers_result', payload: { requestId: 'n-1', error: 'UNSUPPORTED' } } }))
+ expect(routed.at(-1)?.type).toBe('nobody_answers')
+
+ // A process on this machine asks as itself, and as the owner.
+ const frames: Array<Record<string, unknown>> = []
+ socket.registerLocalClient('local:app', { sendFrame: (frame) => { frames.push(frame as Record<string, unknown>); return true }, sendBinary: () => true })
+ socket.handleLocalFrame('local:app', { type: 'session_search', payload: { requestId: 'l-1', query: 'dial' } })
+ await vi.waitFor(() => expect(frames).toContainEqual(expect.objectContaining({ type: 'session_search_result' })))
+ expect(routed.at(-1)?.asker).toEqual({ local: true, owner: true })
await socket.stop()
})
- it('answers session_tail from the index, sealed to the requester, and says what it cannot', async () => {
+ it('answers a request whose service is unavailable SERVICE_UNAVAILABLE, retryable, and goes on', async () => {
const socket = new BackendSocket('token')
- const asked: Array<[string, number | undefined, number | undefined]> = []
- const tail = { sessionId: 'sess-1', rows: [{ turn: 4, at: 1, ask: 'fix the dial', answer: 'Done.', tools: '' }], hasMore: true, total: 5, lastAt: 1 }
- socket.sessionTailProvider = async (sessionId, options) => {
- asked.push([sessionId, options.beforeTurn, options.maxChars])
- return sessionId === 'sess-1' ? tail : null
- }
- socket.connect()
- const ws = wsMock.instances[0]
- ws.open()
- const unwrap = vi.spyOn(socket.e2ee, 'unwrapDown')
- vi.spyOn(socket.e2ee, 'hasSession').mockReturnValue(true)
- const wrapReply = vi.spyOn(socket.e2ee, 'wrapRpcReply').mockReturnValue({
- type: 'session_tail_result', payload: { __e2e: { v: 1, k: 's', n: 1, ct: 'ciphertext' } },
- })
- const envelope = { __e2e: { v: 1, k: 's', n: 1, ct: 'ciphertext' } }
- const ask = (payload: Record<string, unknown>) => {
- unwrap.mockReturnValueOnce({ type: 'session_tail', payload })
- ws.message({ t: 'down', connId: 'web-1', frame: { type: 'session_tail', payload: envelope } })
- }
- ask({ requestId: 't-1', sessionId: 'sess-1', beforeTurn: 9, maxChars: 8000 })
- await vi.waitFor(() => expect(wrapReply).toHaveBeenCalledWith('web-1', 'session_tail_result', 't-1', tail))
- // Only whole numbers page; anything else asks for the last rows.
- ask({ requestId: 't-2', sessionId: 'sess-1', beforeTurn: '9', maxChars: 1.5 })
- await vi.waitFor(() => expect(asked).toHaveLength(2))
- expect(asked).toEqual([['sess-1', 9, 8000], ['sess-1', undefined, undefined]])
- ask({ requestId: 't-3', sessionId: 'nope' })
- await vi.waitFor(() => expect(wrapReply).toHaveBeenCalledWith('web-1', 'session_tail_result', 't-3', { error: 'NOT_INDEXED', sessionId: 'nope' }))
- ask({ requestId: 't-4' })
- await vi.waitFor(() => expect(wrapReply).toHaveBeenCalledWith('web-1', 'session_tail_result', 't-4', { error: 'BAD_SESSION' }))
- socket.sessionTailProvider = null
- ask({ requestId: 't-5', sessionId: 'sess-1' })
- await vi.waitFor(() => expect(wrapReply).toHaveBeenCalledWith('web-1', 'session_tail_result', 't-5', { error: 'SEARCH_UNAVAILABLE' }))
+ // A slot that reaches a service through its port, whose fallback is to fail (core/serviceHost.ts).
+ socket.hostThemeSink = () => { throw new ServiceUnavailableError('models', new Error('disk I/O error')) }
+ const warn = vi.spyOn(console, 'warn').mockImplementation(() => {})
+ const frames: Array<Record<string, unknown>> = []
+ socket.registerLocalClient('local:app', { sendFrame: (frame) => { frames.push(frame as Record<string, unknown>); return true }, sendBinary: () => true })
+ socket.handleLocalFrame('local:app', { type: 'theme_set', payload: { requestId: 't-1', foreground: '#ffffff', background: '#000000' } })
+ await vi.waitFor(() => expect(frames).toContainEqual({ type: 'theme_set_result', payload: { requestId: 't-1', error: 'SERVICE_UNAVAILABLE', service: 'models', retryable: true } }))
+ expect(warn).toHaveBeenCalledWith('[backend] theme_set: the models service is unavailable')
await socket.stop()
})
@@ -1381,44 +1365,23 @@ describe('BackendSocket outbound queue', () => {
await socket.stop()
})
- it('dsh_remove uninstalls through the daemon and refuses a malformed id', async () => {
- const socket = new BackendSocket('token')
- const frames: Array<Record<string, unknown>> = []
- socket.registerLocalClient('local:store', { sendFrame: (frame) => { frames.push(frame); return true }, sendBinary: () => true })
- const removed: string[] = []
- socket.onDshRemove = (id) => { removed.push(id); return id === 'autonomous/marp' ? { ok: true } : { ok: false, error: 'NOT_INSTALLED', detail: `${id} is not installed` } }
-
- socket.handleLocalFrame('local:store', { type: 'dsh_remove', payload: { requestId: 'rm-1', id: 'autonomous/marp' } })
- socket.handleLocalFrame('local:store', { type: 'dsh_remove', payload: { requestId: 'rm-2', id: 'autonomous/none' } })
- socket.handleLocalFrame('local:store', { type: 'dsh_remove', payload: { requestId: 'rm-3', id: '../../etc' } })
-
- await vi.waitFor(() => expect(frames.filter((f) => f.type === 'dsh_remove_result')).toHaveLength(3))
- const results = frames.filter((f) => f.type === 'dsh_remove_result').map((f) => f.payload as Record<string, unknown>)
- expect(results).toEqual([
- expect.objectContaining({ requestId: 'rm-1', ok: true, id: 'autonomous/marp' }),
- expect.objectContaining({ requestId: 'rm-2', error: 'NOT_INSTALLED' }),
- expect.objectContaining({ requestId: 'rm-3', error: 'INVALID_DSH' }),
- ])
- expect(removed).toEqual(['autonomous/marp', 'autonomous/none'])
- await socket.unregisterLocalClient('local:store')
- await socket.stop()
- })
-
- it('serves live catalog entries without making unrelated RPCs wait for catalog I/O', async () => {
- let finish!: (entries: Awaited<ReturnType<typeof storeCatalog.refreshDshRegistry>>) => void
- vi.spyOn(storeCatalog, 'refreshDshRegistry').mockImplementation(() => new Promise(resolve => { finish = resolve }))
+ it('never holds a connection\'s next request for a service still answering one', async () => {
const socket = new BackendSocket('token')
socket.runtimeModelsProvider = async () => []
+ let answerCatalog!: (result: Record<string, unknown>) => void
+ socket.serviceRouter = (type, _payload, _asker, reply) => {
+ if (type !== 'dsh_list') return false
+ answerCatalog = reply
+ return true
+ }
const frames: Array<Record<string, unknown>> = []
socket.registerLocalClient('local:catalog', { sendFrame: frame => { frames.push(frame); return true }, sendBinary: () => true })
socket.handleLocalFrame('local:catalog', { type: 'dsh_list', payload: { requestId: 'catalog' } })
socket.handleLocalFrame('local:catalog', { type: 'models_list', payload: { requestId: 'models' } })
await vi.waitFor(() => expect(frames.some(frame => frame.type === 'models_list_result')).toBe(true))
expect(frames.some(frame => frame.type === 'dsh_list_result')).toBe(false)
- finish([{ id: 'acme/published-today', name: 'Published today', engine: 'claude', repo: 'https://example.test/project', tier: 2, viewerUse: 'acme/viewer' }])
- await vi.waitFor(() => expect(frames).toContainEqual({
- type: 'dsh_list_result', payload: { requestId: 'catalog', dsh: [expect.objectContaining({ id: 'acme/published-today', installed: false, viewerUse: 'acme/viewer' })] },
- }))
+ answerCatalog({ dsh: [] })
+ await vi.waitFor(() => expect(frames).toContainEqual({ type: 'dsh_list_result', payload: { requestId: 'catalog', dsh: [] } }))
await socket.unregisterLocalClient('local:catalog')
await socket.stop()
})
@@ -3447,23 +3410,23 @@ describe('relay down-frames are default-deny: sealed, or the backend\'s own', ()
expect(onMessage).not.toHaveBeenCalled()
})
- it.each([...STRICT_DOWN_TYPES])('refuses a plaintext %s from the relay with E2EE_REQUIRED', async (type) => {
+ it.each([...STRICT_DOWN_TYPES])('refuses a plaintext %s from the relay with E2EE_REQUIRED, before any service hears it', async (type) => {
const { socket, replies, dispatch } = harness()
- const onDshInstall = vi.fn(async () => ({ ok: true }) as never)
- socket.onDshInstall = onDshInstall
+ const serviceRouter = vi.fn(() => true)
+ socket.serviceRouter = serviceRouter
socket.onCancel = vi.fn()
await dispatch({ type, payload: { requestId: 'r', url: 'https://example.invalid/evil.git', agentId: 'a1' } }, 'web-1')
- expect(onDshInstall).not.toHaveBeenCalled()
+ expect(serviceRouter).not.toHaveBeenCalled()
expect(socket.onCancel).not.toHaveBeenCalled()
expect(replies).toEqual([{ connId: 'web-1', type, payload: { error: 'E2EE_REQUIRED' } }])
})
- it('runs a sealed dsh_install from a paired client', async () => {
+ it('hands a sealed dsh_install from a paired client to the service that answers it', async () => {
const { socket, dispatch } = harness()
- const onDshInstall = vi.fn(async () => ({ ok: true }) as never)
- socket.onDshInstall = onDshInstall
+ const serviceRouter = vi.fn(() => true)
+ socket.serviceRouter = serviceRouter
await dispatch(sealedDown(socket, 'web-1', 'dsh_install', { requestId: 'r', id: 'acme/some-dsh' }).frame, 'web-1')
- expect(onDshInstall).toHaveBeenCalledWith(expect.objectContaining({ id: 'acme/some-dsh' }), expect.any(Function))
+ expect(serviceRouter).toHaveBeenCalledWith('dsh_install', expect.objectContaining({ id: 'acme/some-dsh' }), { local: false, owner: expect.any(Boolean) }, expect.any(Function))
})
it('refuses a plaintext RPC even from the backend itself (connId \'\'), which would read the reply', async () => {
diff --git a/cli/src/backendSocket.ts b/cli/src/backendSocket.ts
index d94b61ba0..f123f7c81 100644
--- a/cli/src/backendSocket.ts
+++ b/cli/src/backendSocket.ts
@@ -1,4 +1,5 @@
import type { PurgeAgentService } from './lib/purgeAgentService.js'
+import type { Asker } from './core/api.js'
import { ServiceUnavailableError } from './core/serviceHost.js'
import type { ActivityFrame } from './lib/turnActivity.js'
import { readSessionGitPullRequest } from './lib/sessionGitPullRequest.js'
@@ -85,9 +86,6 @@ import type { SessionInputDelivery } from './lib/sessionInput.js'
import type { QuestionAnswerResult } from './lib/askQuestion.js'
import { engineLabel } from './lib/agentNames.js'
import { DSH_ID_RE, dshSupportedEngines } from './dsh/manifest.js'
-import { refreshDshRegistry } from './dsh/catalog.js'
-import type { DshInstallProgress } from './dsh/install.js'
-import { dshInstallReply, dshInstallRequest, dshInstallStatus, dshListRows, dshRemoveId, dshRemoveReply } from './dsh/wire.js'
import { terminalHandoffRequest } from './lib/terminalHandoff.js'
import { routeVoiceTask } from './lib/voiceRouter.js'
import { tailFile } from './lib/sessions.js'
@@ -141,8 +139,6 @@ import { shouldReplayCommander } from './lib/commanderReplay.js'
import { RuntimeProfileControlError, type RuntimeProfileErrorCode } from './lib/runtimeProfileController.js'
import { parseRuntimeProfile, type RuntimeModelOption } from './lib/runtimeProfile.js'
import { sid, preview, logFrame } from './lib/log.js'
-import type { SessionSearchResult } from './lib/sessionSearch/indexer.js'
-import type { SessionTail } from './lib/sessionSearch/store.js'
import {
TerminalP2pResponderPool,
TERMINAL_P2P_DOWN_TYPES,
@@ -569,14 +565,6 @@ export class BackendSocket {
takeOver?: 'idle' | 'now' | 'wait' | null
}) =>
Promise<{ ok: true; session: RegisteredSession } | { ok: false; error: string; detail?: string }>) | null = null
- /** Called on `dsh_install` — cli.ts clones/sets up/doctors the harness and reports each phase. */
- onDshInstall: ((input: { id?: string; url?: string; ref?: string }, progress: (p: DshInstallProgress) => void) =>
- Promise<{ ok: true; id: string } | { ok: false; error: string; detail: string }>) | null = null
- /** An explicit update follows the installed source and retains the previous package on failure. */
- onDshUpdate: ((id: string, progress: (p: DshInstallProgress) => void) =>
- Promise<{ ok: true; id: string } | { ok: false; error: string; detail: string }>) | null = null
- /** Called on `dsh_remove` — cli.ts uninstalls the harness from this machine. */
- onDshRemove: ((id: string) => { ok: true } | { ok: false; error: string; detail: string }) | null = null
/** Called on `remote_terminal_handoff` — cli.ts names the agent whose tile is that tmux pane, or null. */
onTerminalHandoff: ((tmuxPane: string) => string | null) | null = null
/** What the daemon knows about an agent's DSH companions (viewer URL, verdict); null when nothing. */
@@ -792,11 +780,6 @@ export class BackendSocket {
Promise<{ file: string | null; gitRepo: boolean; cwd: string; degraded: string[] }>) | null = null
monitorActivityProvider: ((sessionId: string) => MonitorActivity) | null = null
private readonly monitorCompletions = new MonitorCompletions()
- /** Answers `session_search` from this machine's transcript index (lib/sessionSearch/). Null when
- * this Node has no `node:sqlite`. */
- sessionSearchProvider: ((query: string, options: { limit?: number; from?: number; to?: number }) => SessionSearchResult) | null = null
- /** The end of one session from the same index, for Cmd-P's preview (`session_tail`). */
- sessionTailProvider: ((sessionId: string, options: { beforeTurn?: number; maxChars?: number }) => Promise<SessionTail | null>) | null = null
/** Runtime Model/Effort integration, wired by cli.ts for registered tmux sessions. */
runtimeModelsProvider: ((sessionId?: string) => Promise<RuntimeModelOption[]>) | null = null
/** Answers `usage_read` — this machine's own agent-account usage (lib/accountUsage.ts). A field
@@ -1435,10 +1418,10 @@ export class BackendSocket {
/** The windows attached right now — for a listener that arrives after some of them did. */
localClientIds(): string[] { return [...this.localClients.keys()].filter((connId) => !this.toolClients.has(connId)) }
+ /** Routes a request to the service that answers it, in its own process (core/serviceLinks.ts) or in
+ * this one (core/serviceHost.ts): false when none does and the socket answers it itself. */
+ serviceRouter: ((type: string, payload: Record<string, unknown>, asker: Asker, reply: (result: Record<string, unknown>) => void) => boolean) | null = null
/** A window (or `hn`) on this computer attached or went away — the pair brain thinks only while one is here. */
- /** Routes a request to the out-of-process service that answers it (core/serviceLinks.ts `route`):
- * false when none does and the socket answers it itself. */
- serviceRouter: ((type: string, payload: Record<string, unknown>, reply: (result: Record<string, unknown>) => void) => boolean) | null = null
onLocalClient: ((connId: string, attached: boolean) => void) | null = null
/** Release all connection-scoped state when the loopback WebSocket closes. */
@@ -2067,9 +2050,11 @@ export class BackendSocket {
return
}
- // A request a service answers in its own process (core/serviceLinks.ts): routed to it, or answered
- // SERVICE_UNAVAILABLE while it is down. Never waited on in line: the next frame is not held for it.
- if (this.serviceRouter?.(type, payload, (result) => reply(type, requestId, result))) return
+ // A request a service answers, in its own process or in this one: routed to it, or answered
+ // SERVICE_UNAVAILABLE while it is off. Never waited on in line: the next frame is not held for it.
+ // Who asked is established here, after the gates above, and the service trusts only that.
+ const asker: Asker = { local, owner: local || this.e2ee.sessionRole(connId) === 'web' }
+ if (this.serviceRouter?.(type, payload, asker, (result) => reply(type, requestId, result))) return
try {
switch (type) {
@@ -2540,26 +2525,6 @@ export class BackendSocket {
return
}
- case 'dsh_list': {
- // Keep catalog I/O off this connection's ordered RPC queue.
- void refreshDshRegistry()
- .then(catalog => reply(type, requestId, { dsh: dshListRows(undefined, catalog) }))
- .catch(error => reply(type, requestId, { error: 'INTERNAL', detail: error instanceof Error ? error.message : String(error) }))
- return
- }
-
- case 'dsh_remove': {
- // Uninstall a harness from THIS machine: the clone under ~/.harness/dsh goes (a linked
- // install loses only its link), the index forgets it, and a `dsh_list` after this no
- // longer says installed. Agents already running from it keep running — their processes
- // hold what they need — and the store is what asks; it refreshes the list itself.
- if (!this.onDshRemove) { reply(type, requestId, { error: 'UNSUPPORTED_ON_REMOTE' }); return }
- const id = dshRemoveId(payload)
- if (!id) { reply(type, requestId, { error: 'INVALID_DSH', detail: 'dsh_remove needs an id' }); return }
- reply(type, requestId, dshRemoveReply(id, this.onDshRemove(id)))
- return
- }
-
case 'remote_terminal_handoff': {
// `harness remote`, typed INSIDE one of this machine's terminal tiles, opened a terminal on
// another machine and asks the window showing the tile to swap it over: the tile it was
@@ -2581,29 +2546,6 @@ export class BackendSocket {
return
}
- case 'dsh_update': {
- if (!this.onDshUpdate) { reply(type, requestId, { error: 'UNSUPPORTED_ON_REMOTE' }); return }
- const id = dshRemoveId(payload)
- if (!id) { reply(type, requestId, { error: 'INVALID_DSH', detail: 'dsh_update needs an id' }); return }
- void this.onDshUpdate(id, p => this.send({ type: 'dsh_install_status', payload: dshInstallStatus(p, { id }) }))
- .then(result => reply(type, requestId, dshInstallReply(result)))
- .catch(error => reply(type, requestId, { error: 'INTERNAL', detail: error instanceof Error ? error.message : String(error) }))
- return
- }
-
- case 'dsh_install': {
- // Clone, set up and doctor a harness on THIS machine. Long — minutes, for a toolchain — so
- // it is detached from the ordered RPC chain like `engines_probe`, and progress travels as
- // `dsh_install_status` pushes the app renders in the create dialog.
- if (!this.onDshInstall) { reply(type, requestId, { error: 'UNSUPPORTED_ON_REMOTE' }); return }
- const request = dshInstallRequest(payload)
- if (!request) { reply(type, requestId, { error: 'INVALID_DSH', detail: 'dsh_install needs an id or a url' }); return }
- void this.onDshInstall(request, (p) => this.send({ type: 'dsh_install_status', payload: dshInstallStatus(p, request) }))
- .then((result) => reply(type, requestId, dshInstallReply(result)))
- .catch((error) => reply(type, requestId, { error: 'INTERNAL', detail: error instanceof Error ? error.message : String(error) }))
- return
- }
-
case 'engines_probe': {
// Which engines this machine has, asked BEFORE a create rather than discovered by one
// failing. Answered here — on the machine in question — because a Mac and the Docker rig
@@ -3443,32 +3385,6 @@ export class BackendSocket {
return
}
- // Every conversation on this machine, searched by what was said in it (lib/sessionSearch/).
- // Synchronous and a few milliseconds: the index is local SQLite FTS5. The words searched for
- // arrive sealed and the hits leave sealed — the relay reads neither.
- case 'session_search': {
- if (!this.sessionSearchProvider) { reply(type, requestId, { error: 'SEARCH_UNAVAILABLE' }); return }
- const query = typeof payload.query === 'string' ? payload.query.slice(0, 500) : ''
- const number = (value: unknown) => typeof value === 'number' && Number.isFinite(value) ? value : undefined
- // `from`/`to`: only sessions worked on in that window (epoch ms) — "the dial one from last
- // week". The client reads the time words, so every machine searches the same window.
- reply(type, requestId, { ...this.sessionSearchProvider(query, { limit: number(payload.limit), from: number(payload.from), to: number(payload.to) }) })
- return
- }
-
- // A session's latest rows, newest last; `beforeTurn` pages up from the first row the client
- // has. The same index as `session_search`, so it reads no transcript for a preview.
- case 'session_tail': {
- if (!this.sessionTailProvider) { reply(type, requestId, { error: 'SEARCH_UNAVAILABLE' }); return }
- const sessionId = typeof payload.sessionId === 'string' ? payload.sessionId.slice(0, 200) : ''
- if (!sessionId) { reply(type, requestId, { error: 'BAD_SESSION' }); return }
- const integer = (value: unknown) => typeof value === 'number' && Number.isInteger(value) ? value : undefined
- const tail = await this.sessionTailProvider(sessionId, { beforeTurn: integer(payload.beforeTurn), maxChars: integer(payload.maxChars) })
- if (!tail) { reply(type, requestId, { error: 'NOT_INDEXED', sessionId }); return }
- reply(type, requestId, { ...tail })
- return
- }
-
// Physical devices belong to this machine; only its owner or loopback tools may manage them.
case 'harness_devices_list':
case 'harness_device_settings': {
diff --git a/cli/src/cli.ts b/cli/src/cli.ts
index e638afddb..b458c7928 100644
--- a/cli/src/cli.ts
+++ b/cli/src/cli.ts
@@ -3,7 +3,6 @@ import './config/loadEnv.js'
import { ensureBundledCoreHarnesses } from './dsh/builtins.js'
import { runDevicesCommand } from './devices/client.js'
import { createDeviceStore, deviceStoreAgents } from './lib/autonomous-device/storeRuntime.js'
-import { mutateDsh } from './dsh/service.js'
import { HarnessShareOwner } from './sharing/owner.js'
import { HarnessGrantStore } from './sharing/grants.js'
import { HarnessCollaborationStore } from './sharing/collaboration.js'
@@ -103,7 +102,6 @@ import { type LaunchOverridesDeps } from './lib/launchOverrides.js'
import { buildHarnessSessionLabel } from './lib/harnessSessionLabel.js'
import { adoptLegacyHarnessSessions, listTmuxPanes } from './lib/tmuxAgentDiscovery.js'
import { installedDsh } from './dsh/installed.js'
-import { removeDsh } from './dsh/install.js'
import { prepareHarnessLaunch } from './dsh/runtime.js'
import { dshCommand, dshUsage } from './dsh/command.js'
import { ApiConnections } from './lib/apiConnections.js'
@@ -165,9 +163,10 @@ import { databaseHistory } from './core/transcripts/databaseHistory.js'
import { createCoreApi, emptyPorts, MODELS_FALLBACKS, SEARCH_FALLBACKS, TEAMS_FALLBACKS, VIEWERS_FALLBACKS, WORKSPACES_FALLBACKS, type TeamsPort } from './core/api.js'
import { createServiceHost, testFaults } from './core/serviceHost.js'
import { createServiceLinks } from './core/serviceLinks.js'
-import { KNOWN_SERVICES, SERVICE_REQUESTS, serviceSpecs } from './harnessd/services.js'
+import { KNOWN_SERVICES, serviceSpecs } from './harnessd/services.js'
import { runSearchService } from './services/searchProcess.js'
-import { startSearch } from './services/search.js'
+import { SEARCH_REQUESTS, startSearch } from './services/search.js'
+import { STORE_REQUESTS, startStore } from './services/store.js'
import { startViewers } from './services/viewers.js'
import { startModels } from './services/models.js'
import { startWorkspaces } from './services/workspaces.js'
@@ -1797,6 +1796,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {
backendRef?.interactiveViewers.refresh(agentId)
},
gridNamed: (name) => backendRef?.setHarnessGridName(name),
+ dshInstallStatus: (status) => backendRef?.send({ type: 'dsh_install_status', payload: status }),
// The backend mints and remembers the account's grid name; this CLI holds neither the account's
// email nor its id. An older backend (no route) answers nothing, which the grid reconcile treats as
// "no grid yet". Bounded so a stalled control-plane connection cannot hold the attempt open.
@@ -2178,15 +2178,19 @@ async function runForeground(session: AuthSession | null): Promise<void> {
// routed to them, and answered SERVICE_UNAVAILABLE while they are down (core/serviceLinks.ts).
const serviceToken = process.env.HARNESSD_SUPERVISED === '1' ? process.env.HARNESSD_SERVICE_TOKEN : undefined
const outOfProcess = new Set(serviceToken ? serviceSpecs(process.env, KNOWN_SERVICES).map((spec) => spec.name) : [])
+ // The requests each service that can run in its own process answers (its own module declares them).
+ const requestsOf: Record<string, readonly string[]> = { search: SEARCH_REQUESTS }
const serviceLinks = createServiceLinks({
token: serviceToken,
- owned: Object.fromEntries([...outOfProcess].map((name) => [name, SERVICE_REQUESTS[name] ?? []])),
+ owned: Object.fromEntries([...outOfProcess].map((name) => [name, requestsOf[name] ?? []])),
// Every agent, live then stopped, with the name the apps show for it: what search indexes.
answer: (_service, query) => query === 'agents'
? { agents: coreApi.agents.all().map((session) => ({ ...session, displayName: coreApi.agents.displayName(session) })) }
: { error: 'UNKNOWN_QUERY' },
})
- backend.serviceRouter = (type, payload, reply) => serviceLinks.route(type, payload, reply)
+ // A request a service declared goes to it: in its own process, or in this one (core/serviceHost.ts).
+ backend.serviceRouter = (type, payload, asker, reply) =>
+ serviceLinks.route(type, payload, asker, reply) || serviceHost.route(type, payload, asker, reply)
// Session search (services/search.ts): in this process, or in its own (services/searchProcess.ts),
// where the core tells it what changed. A purge's forgetting waits for it if it is down.
@@ -2195,20 +2199,15 @@ async function runForeground(session: AuthSession | null): Promise<void> {
touch: (sessionId) => { serviceLinks.notify('search', { type: 'service_event', payload: { kind: 'touch', sessionId } }) },
deleteHistory: (sessionId) => { serviceLinks.notify('search', { type: 'service_event', payload: { kind: 'deleteHistory', sessionId } }, { untilDelivered: true }) },
session: () => undefined,
- // Never asked: its requests are routed to its process before they reach a handler here.
- search: () => { throw new Error('search runs in its own process') },
- tail: () => Promise.reject(new Error('search runs in its own process')),
stop: () => {},
}
} else {
- serviceHost.start('search', startSearch, coreApi, SEARCH_FALLBACKS)
+ serviceHost.start('search', startSearch, coreApi, SEARCH_FALLBACKS, SEARCH_REQUESTS)
}
+ // What the core calls search through: guarded, so it answers its fallbacks once search is switched off.
const sessionSearch = ports.search
- const searchHere = sessionSearch && !outOfProcess.has('search') ? sessionSearch : null
- backend.sessionSearchProvider = searchHere ? (query, options) => searchHere.search(query, options) : null
- backend.sessionTailProvider = searchHere ? (sessionId, options) => searchHere.tail(sessionId, options) : null
- // Search switched off: both requests say search is unavailable, as on a Node without an index.
- serviceHost.onOff('search', () => { backend.sessionSearchProvider = null; backend.sessionTailProvider = null })
+ // The harnesses installed here, and installing, updating and removing one (services/store.ts).
+ serviceHost.serve('store', startStore, coreApi, STORE_REQUESTS)
const runtimeController = new RuntimeProfileController({
manager: runtimeProfiles,
@@ -2235,13 +2234,10 @@ async function runForeground(session: AuthSession | null): Promise<void> {
}
backend.runtimeProfileProvider = (session) => runtimeProfiles.selectedModel(session)
backend.dshFrameProvider = (s) => ports.viewers?.frameContext(s) ?? null
- backend.onDshRemove = (id) => removeDsh(id)
// `harness remote` names the tile it was typed in by its tmux pane; the registry knows whose it is.
backend.onTerminalHandoff = (tmuxPane) => registry.advertised()
.find((session) => session.tmuxPane === tmuxPane
|| session.runtimes.some((runtime) => runtime.backend === 'tmux' && runtime.paneId === tmuxPane))?.agentId ?? null
- backend.onDshInstall = (input, progress) => mutateDsh(input, progress)
- backend.onDshUpdate = (id, progress) => mutateDsh({ id, update: true }, progress)
backend.onAgentRename = (session, name) => { void terminals.setTitle(session, name) }
backend.onRuntimeProfileUpdate = (sessionId, selectedModel) => runtimeController.setProfile(sessionId, selectedModel)
runtimeProfiles.onChanged = (sessionId) => {
diff --git a/cli/src/core/api.spec.ts b/cli/src/core/api.spec.ts
index 970bb718f..ebc60908e 100644
--- a/cli/src/core/api.spec.ts
+++ b/cli/src/core/api.spec.ts
@@ -21,6 +21,7 @@ describe('the core API services stand on', () => {
syncSession: vi.fn(),
viewerChanged: vi.fn(),
gridNamed: vi.fn(),
+ dshInstallStatus: vi.fn(),
mintGridName: vi.fn(async () => 'grid-1'),
accessToken: vi.fn(async () => 'token'),
}
@@ -37,6 +38,7 @@ describe('the core API services stand on', () => {
expect(core.agents.terminalAvailable('live')).toBe(true)
expect(core.agents.sync).toBe(deps.syncSession)
expect(core.clients.viewerChanged).toBe(deps.viewerChanged)
+ expect(core.clients.dshInstallStatus).toBe(deps.dshInstallStatus)
expect(core.agents.advertised().map((s) => s.agentId)).toEqual(['live'])
expect(core.clients.gridNamed).toBe(deps.gridNamed)
expect(core.account.mintGridName).toBe(deps.mintGridName)
diff --git a/cli/src/core/api.ts b/cli/src/core/api.ts
index 39302db78..c07209141 100644
--- a/cli/src/core/api.ts
+++ b/cli/src/core/api.ts
@@ -6,6 +6,9 @@
* become requests on the local socket and its port becomes a proxy that sends them; the service's
* code stays as it is. Members are added as services move behind it, each a call that exists today,
* never a generic `call(name, args)`.
+ *
+ * The apps reach a service through the requests it answers (`ServiceRequests`), which its start
+ * returns. The core routes them to it; a port is only for what the core itself must ask.
*/
import type { AgentDshContext } from '../lib/agentFrame.js'
import type { GridAccess } from '../lib/gridAttach.js'
@@ -60,18 +63,37 @@ export interface CoreApi {
viewerChanged(agentId: string): void
/** The account's grid has a name: the models picker answers with it at once. */
gridNamed(name: string): void
+ /** A harness being installed or updated moved on (`dsh_install_status`): the apps show it in the
+ * create dialog. */
+ dshInstallStatus(status: Record<string, unknown>): void
}
}
+/** Who sent a request, as the core established it. A service trusts this, never a field of the
+ * payload: a payload says whatever its sender wrote. */
+export interface Asker {
+ /** A process on this machine (the desktop app, `hn`, a script), over the local socket. */
+ local: boolean
+ /** Whether it may act as this machine's owner: a local process, or the owner's paired app over the
+ * relay. A device or an observer may not. */
+ owner: boolean
+}
+
+/** A request a service answers for the apps: the reply, or a promise of it. A throw or a rejection is
+ * answered `SERVICE_FAILED` by the host and counted against the service. */
+export type ServiceRequest = (payload: Record<string, unknown>, asker: Asker) => Record<string, unknown> | Promise<Record<string, unknown>>
+
+/** The requests a service answers, by frame type: what its start returns. */
+export type ServiceRequests = Readonly<Record<string, ServiceRequest>>
+
/** The core's calls into session search: index a session at its turn boundaries, forget a purged
- * conversation, the title it indexed for one being adopted, the two requests it answers
- * (`session_search`, `session_tail`), and stopping its sweeps. */
-export type SearchPort = Pick<SessionSearchIndex, 'touch' | 'deleteHistory' | 'session' | 'search' | 'tail' | 'stop'>
+ * conversation, the title it indexed for one being adopted, and stopping its sweeps. The apps' own
+ * requests (`session_search`, `session_tail`) are its `ServiceRequests`, not the core's calls. */
+export type SearchPort = Pick<SessionSearchIndex, 'touch' | 'deleteHistory' | 'session' | 'stop'>
-/** What the core gets when search fails: nothing indexed, no title, and the requests answered with
- * an error. */
+/** What the core gets when search fails: nothing indexed and no title. */
export const SEARCH_FALLBACKS: PortFallbacks<SearchPort> = {
- touch: undefined, deleteHistory: undefined, session: undefined, search: FAIL, tail: later(FAIL), stop: undefined,
+ touch: undefined, deleteHistory: undefined, session: undefined, stop: undefined,
}
/** The core's calls into the DSH viewers: each harness agent's viewer server and verdict watch. */
@@ -156,13 +178,14 @@ export interface CoreApiDeps {
syncSession: CoreApi['agents']['sync']
viewerChanged: CoreApi['clients']['viewerChanged']
gridNamed: CoreApi['clients']['gridNamed']
+ dshInstallStatus: CoreApi['clients']['dshInstallStatus']
mintGridName: CoreApi['account']['mintGridName']
accessToken: CoreApi['account']['accessToken']
}
export function createCoreApi({
dataDir, registry, stoppedAgents, databaseHistory, externalSessions, openSessions, syncSession, viewerChanged,
- gridNamed, mintGridName, accessToken,
+ gridNamed, dshInstallStatus, mintGridName, accessToken,
}: CoreApiDeps): CoreApi {
return {
dataDir,
@@ -178,6 +201,6 @@ export function createCoreApi({
transcripts: { databaseHistory },
external: { sessions: externalSessions, open: openSessions },
account: { mintGridName, accessToken },
- clients: { viewerChanged, gridNamed },
+ clients: { viewerChanged, gridNamed, dshInstallStatus },
}
}
diff --git a/cli/src/core/serviceHost.spec.ts b/cli/src/core/serviceHost.spec.ts
index 67bddcb78..7e916384a 100644
--- a/cli/src/core/serviceHost.spec.ts
+++ b/cli/src/core/serviceHost.spec.ts
@@ -13,9 +13,13 @@ class FakeSearch {
async tail(sessionId: string) { return { sessionId } as never }
stop(): void { this.calls.push('stop') }
}
-const SEARCH: PortFallbacks<SearchPort> = {
+// The search port as these tests drive it: two more members, whose fallback is to fail, sync and async —
+// the shapes of a member a request needs answered. The host guards whatever the fallbacks name.
+type TestPort = SearchPort & { search(query: string, options: object): unknown; tail(sessionId: string, options: object): Promise<unknown> }
+const SEARCH = {
touch: undefined, deleteHistory: undefined, session: undefined, search: FAIL, tail: later(FAIL), stop: undefined,
-}
+} as unknown as PortFallbacks<SearchPort>
+const testPort = (h: { ports: CorePorts }): TestPort => h.ports.search as unknown as TestPort
const VIEWERS: PortFallbacks<ViewersPort> = {
attach: undefined, detach: undefined, frameContext: null, forwardingUrl: null, stop: later(undefined),
}
@@ -27,9 +31,9 @@ function host(options: Parameters<typeof createServiceHost>[1] = {}) {
const services = createServiceHost(ports, { log: (line) => lines.push(line), now: () => at, ...options })
return { ports, lines, services, advance: (ms: number) => { at += ms } }
}
-function startSearch(h: ReturnType<typeof host>, port: Partial<SearchPort> = new FakeSearch()) {
+function startSearch(h: ReturnType<typeof host>, port: Partial<TestPort> = new FakeSearch()) {
h.services.start('search', (_core, ports) => { ports.search = port as SearchPort }, fakeCore(), SEARCH)
- return port as SearchPort
+ return port as TestPort
}
describe('hosting services so one that fails cannot take the core down', () => {
@@ -40,7 +44,7 @@ describe('hosting services so one that fails cannot take the core down', () => {
const h = host()
const real = new FakeSearch()
startSearch(h, real)
- const port = h.ports.search!
+ const port = testPort(h)
expect(port).not.toBe(real)
port.touch('s1')
port.deleteHistory('s2')
@@ -93,8 +97,8 @@ describe('hosting services so one that fails cannot take the core down', () => {
touch: () => { throw new Error('store closed') },
session: () => { throw 'bad row' },
})
- expect(h.ports.search!.touch('s1')).toBeUndefined()
- expect(h.ports.search!.session('s1')).toBeUndefined()
+ expect(testPort(h).touch('s1')).toBeUndefined()
+ expect(testPort(h).session('s1')).toBeUndefined()
expect(port).toBeDefined()
expect(h.lines).toEqual(['[services] search.touch failed · store closed', '[services] search.session failed · bad row'])
})
@@ -104,10 +108,10 @@ describe('hosting services so one that fails cannot take the core down', () => {
const cause = new Error('disk I/O error')
startSearch(h, { ...new FakeSearch(), search: () => { throw cause }, tail: async () => { throw cause } })
let thrown: unknown
- try { h.ports.search!.search('x', {}) } catch (error) { thrown = error }
+ try { testPort(h).search('x', {}) } catch (error) { thrown = error }
expect(thrown).toBeInstanceOf(ServiceUnavailableError)
expect(thrown).toMatchObject({ service: 'search', cause, name: 'ServiceUnavailableError', message: 'the search service is unavailable' })
- await expect(h.ports.search!.tail('s1', {})).rejects.toMatchObject({ service: 'search', cause })
+ await expect(testPort(h).tail('s1', {})).rejects.toMatchObject({ service: 'search', cause })
})
it('resolves an async member to its fallback whether it rejects or throws before its promise', async () => {
@@ -138,7 +142,7 @@ describe('hosting services so one that fails cannot take the core down', () => {
const real = new FakeSearch()
real.touch = () => { throw new Error('store closed') }
startSearch(h, real)
- const captured = h.ports.search!
+ const captured = testPort(h)
const unbind = vi.fn()
h.services.onOff('search', unbind)
for (let i = 0; i < 4; i++) { captured.touch('s1'); h.advance(1_000) }
@@ -161,7 +165,7 @@ describe('hosting services so one that fails cannot take the core down', () => {
it('forgets failures older than the window, so a slow trickle never switches a service off', () => {
const h = host({ maxFailures: 3, windowMs: 10_000 })
startSearch(h, { ...new FakeSearch(), touch: () => { throw new Error('flaky') } })
- for (let i = 0; i < 10; i++) { h.ports.search!.touch('s1'); h.advance(6_000) }
+ for (let i = 0; i < 10; i++) { testPort(h).touch('s1'); h.advance(6_000) }
expect(h.services.isOff('search')).toBe(false)
})
@@ -169,7 +173,7 @@ describe('hosting services so one that fails cannot take the core down', () => {
const h = host({ maxFailures: 2 })
const rejects: Array<(error: Error) => void> = []
startSearch(h, { ...new FakeSearch(), tail: () => new Promise((_resolve, reject) => { rejects.push(reject) }) })
- const calls = [h.ports.search!.tail('a', {}), h.ports.search!.tail('b', {}), h.ports.search!.tail('c', {})]
+ const calls = [testPort(h).tail('a', {}), testPort(h).tail('b', {}), testPort(h).tail('c', {})]
for (const reject of rejects) reject(new Error('timeout'))
await Promise.allSettled(calls)
expect(h.lines.filter((line) => line.includes('switched off'))).toHaveLength(1)
@@ -181,7 +185,7 @@ describe('hosting services so one that fails cannot take the core down', () => {
startSearch(h, { ...new FakeSearch(), touch: () => { throw new Error('x') }, stop: () => { throw new Error('stop failed') } })
h.services.onOff('search', () => { throw new Error('unbind failed') })
h.services.onOff('search', late)
- h.ports.search!.touch('s1')
+ testPort(h).touch('s1')
expect(late).toHaveBeenCalled()
expect(h.lines).toContain('[services] switching search off: stop failed')
expect(h.lines).toContain('[services] switching search off: unbind failed')
@@ -213,6 +217,120 @@ describe('hosting services so one that fails cannot take the core down', () => {
})
})
+ describe('requests a service answers for the apps', () => {
+ const ASKER = { local: true, owner: true }
+ const collect = () => {
+ const got: Array<Record<string, unknown>> = []
+ return { got, reply: (result: Record<string, unknown>) => { got.push(result) } }
+ }
+
+ it('routes a declared request to its handler with who asked, and leaves the socket everything else', async () => {
+ const h = host()
+ const seen: unknown[] = []
+ h.services.start('search', (_core, ports) => {
+ ports.search = new FakeSearch() as unknown as SearchPort
+ return { session_search: (payload, asker) => { seen.push([payload, asker]); return { hits: [] } } }
+ }, fakeCore(), SEARCH, ['session_search'])
+ const { got, reply } = collect()
+ expect(h.services.route('session_search', { query: 'x' }, ASKER, reply)).toBe(true)
+ await vi.waitFor(() => expect(got).toEqual([{ hits: [] }]))
+ expect(seen).toEqual([[{ query: 'x' }, ASKER]])
+ expect(h.services.route('agents_list', {}, ASKER, reply)).toBe(false)
+ })
+
+ it('returns at once, and replies when a promised answer comes', async () => {
+ const h = host()
+ let answer!: (value: Record<string, unknown>) => void
+ h.services.serve('store', () => ({ dsh_list: () => new Promise((resolve) => { answer = resolve }) }), fakeCore(), ['dsh_list'])
+ const { got, reply } = collect()
+ expect(h.services.route('dsh_list', {}, ASKER, reply)).toBe(true)
+ expect(got).toEqual([])
+ answer({ dsh: [] })
+ await vi.waitFor(() => expect(got).toEqual([{ dsh: [] }]))
+ })
+
+ it('answers SERVICE_UNAVAILABLE, never UNSUPPORTED, for a service that did not start, left its port empty, or was switched off', async () => {
+ const h = host({ maxFailures: 1 })
+ h.services.start('search', () => { throw new Error('index locked') }, fakeCore(), SEARCH, ['session_search'])
+ h.services.start('viewers', () => {}, fakeCore(), VIEWERS, ['viewer_list'])
+ const switchedOff = vi.fn()
+ h.services.serve('store', () => ({ dsh_list: () => { throw new Error('catalog corrupt') } }), fakeCore(), ['dsh_list'])
+ h.services.onOff('store', switchedOff)
+ const { got, reply } = collect()
+ h.services.route('session_search', {}, ASKER, reply)
+ h.services.route('viewer_list', {}, ASKER, reply)
+ h.services.route('dsh_list', {}, ASKER, reply)
+ h.services.route('dsh_list', {}, ASKER, reply)
+ expect(got).toEqual([
+ { error: 'SERVICE_UNAVAILABLE', service: 'search', retryable: false },
+ { error: 'SERVICE_UNAVAILABLE', service: 'viewers', retryable: false },
+ { error: 'SERVICE_FAILED', service: 'store' },
+ { error: 'SERVICE_UNAVAILABLE', service: 'store', retryable: false },
+ ])
+ expect(h.services.isOff('store')).toBe(true)
+ expect(switchedOff).toHaveBeenCalledOnce()
+ expect(h.lines.at(-1)).toBe('[services] store switched off after 1 failures in 60s · it stays off until the daemon restarts')
+ })
+
+ it('answers a handler that throws, rejects or replies with nothing SERVICE_FAILED, and counts each against its service', async () => {
+ const h = host({ maxFailures: 3 })
+ h.services.serve('store', () => ({
+ a: () => { throw new Error('bad row') },
+ b: async () => { throw 'closed' },
+ c: () => undefined as unknown as Record<string, unknown>,
+ }), fakeCore(), ['a', 'b', 'c'])
+ const { got, reply } = collect()
+ for (const type of ['a', 'b', 'c']) h.services.route(type, {}, ASKER, reply)
+ await vi.waitFor(() => expect(got).toHaveLength(3))
+ expect(got).toEqual([{ error: 'SERVICE_FAILED', service: 'store' }, { error: 'SERVICE_FAILED', service: 'store' }, { error: 'SERVICE_FAILED', service: 'store' }])
+ expect(h.lines).toEqual([
+ '[services] store.a failed · bad row',
+ '[services] store.b failed · closed',
+ '[services] store.c failed · c was answered with no reply',
+ '[services] store switched off after 3 failures in 60s · it stays off until the daemon restarts',
+ ])
+ })
+
+ it('leaves off a service that claims a request another answers, or that answers other than it declared', () => {
+ const h = host()
+ h.services.serve('store', () => ({ dsh_list: () => ({ dsh: [] }) }), fakeCore(), ['dsh_list'])
+ h.services.serve('rival', () => ({ dsh_list: () => ({}) }), fakeCore(), ['dsh_list'])
+ h.services.serve('loose', () => ({ a: () => ({}), b: () => ({}), c: () => ({}) }), fakeCore(), ['a'])
+ h.services.serve('looser', () => ({ d: () => ({}), e: () => ({}) }), fakeCore(), ['d'])
+ h.services.serve('short', () => ({}), fakeCore(), ['x', 'y'])
+ h.services.serve('shorter', () => ({}), fakeCore(), ['z'])
+ expect(['rival', 'loose', 'looser', 'short', 'shorter'].every((name) => h.services.isOff(name))).toBe(true)
+ expect(h.services.isOff('store')).toBe(false)
+ expect(h.lines).toEqual([
+ '[services] rival did not start · dsh_list is answered by store · the core runs without it',
+ '[services] loose did not start · it answers b, c without declaring them · the core runs without it',
+ '[services] looser did not start · it answers e without declaring it · the core runs without it',
+ '[services] short did not start · it declares x, y without answering them · the core runs without it',
+ '[services] shorter did not start · it declares z without answering it · the core runs without it',
+ ])
+ // The store keeps its own requests.
+ const { got, reply } = collect()
+ h.services.route('dsh_list', {}, ASKER, reply)
+ return vi.waitFor(() => expect(got).toEqual([{ dsh: [] }]))
+ })
+
+ it('a service with a port that answers other than it declared is off, its port empty', () => {
+ const h = host()
+ h.services.start('search', (_core, ports) => { ports.search = new FakeSearch() as unknown as SearchPort; return {} }, fakeCore(), SEARCH, ['session_search'])
+ expect(h.ports.search).toBeNull()
+ expect(h.services.isOff('search')).toBe(true)
+ expect(h.lines).toEqual(['[services] search did not start · it declares session_search without answering it · the core runs without it'])
+ })
+
+ it('serves a service with no port, and will not serve one whose name is a port\'s', () => {
+ const h = host()
+ h.services.serve('search', () => ({}), fakeCore(), [])
+ expect(h.services.isOff('search')).toBe(true)
+ expect(h.ports.search).toBeNull()
+ expect(h.lines).toEqual(['[services] search did not start · it has a port: start it with start() · the core runs without it'])
+ })
+ })
+
describe('faults injected for the end-to-end suite', () => {
beforeEach(() => { vi.spyOn(console, 'warn').mockImplementation(() => {}) })
@@ -234,5 +352,16 @@ describe('hosting services so one that fails cannot take the core down', () => {
expect(real.calls).toEqual(['delete s1'])
expect(member.lines).toEqual(['[services] search.touch failed · injected fault: search.touch'])
})
+
+ it('fails one request on every ask, and leaves the service\'s others alone', async () => {
+ const h = host({ faults: testFaults('store.dsh_list') })
+ h.services.serve('store', () => ({ dsh_list: () => ({ dsh: [] }), dsh_remove: () => ({ ok: true }) }), fakeCore(), ['dsh_list', 'dsh_remove'])
+ const got: Array<Record<string, unknown>> = []
+ h.services.route('dsh_list', {}, { local: true, owner: true }, (result) => got.push(result))
+ h.services.route('dsh_remove', {}, { local: true, owner: true }, (result) => got.push(result))
+ await vi.waitFor(() => expect(got).toHaveLength(2))
+ expect(got).toEqual([{ error: 'SERVICE_FAILED', service: 'store' }, { ok: true }])
+ expect(h.lines).toEqual(['[services] store.dsh_list failed · injected fault: store.dsh_list'])
+ })
})
})
diff --git a/cli/src/core/serviceHost.ts b/cli/src/core/serviceHost.ts
index c307b6eba..db30d13f0 100644
--- a/cli/src/core/serviceHost.ts
+++ b/cli/src/core/serviceHost.ts
@@ -10,12 +10,19 @@
* and answered with that member's fallback, so the core's caller (the event funnel, restore, a
* frame being built) carries on. A member whose fallback is `FAIL` answers a request: its failure
* goes back to that one request, which the socket answers with an error.
+ * - **Requests.** A service answers the apps through the requests it declares when it is started;
+ * its start returns their handlers, and the socket asks `route` before its own switch, so a feature
+ * adds a request without touching the socket. While the service is off they are answered
+ * `SERVICE_UNAVAILABLE`, never `UNSUPPORTED`, which the apps read as "update the CLI": that is why
+ * the types are declared before the start that could fail. A handler that throws or rejects is
+ * answered `SERVICE_FAILED`, and counts as a failure of the service like a call into its port.
* - **Switching off.** A service that fails `maxFailures` times within `windowMs` is switched off
* for the rest of the daemon's life: it is stopped when its port can stop, its port goes null, the
- * core's calls answer their fallbacks without reaching it, and the `onOff` handlers unbind what
- * clients reach it through. A restart of the daemon starts it again.
+ * core's calls answer their fallbacks without reaching it, its requests are answered unavailable,
+ * and the `onOff` handlers unbind what clients reach it through. A restart of the daemon starts it
+ * again.
*/
-import type { CoreApi, CorePorts } from './api.js'
+import type { Asker, CoreApi, CorePorts, ServiceRequest, ServiceRequests } from './api.js'
/** The fallback of a member whose failure belongs to its caller: it throws (or rejects). */
export const FAIL = Symbol('fail')
@@ -72,26 +79,28 @@ export function createServiceHost(ports: CorePorts, options: ServiceHostOptions
const now = options.now ?? Date.now
const log = options.log ?? ((line: string) => console.warn(line))
const faults = options.faults ?? new Set<string>()
- const off = new Set<PortName>()
- const failures = new Map<PortName, number[]>()
- const offHandlers = new Map<PortName, Array<() => void>>()
- const stoppers = new Map<PortName, () => void>()
+ const off = new Set<string>()
+ const failures = new Map<string, number[]>()
+ const offHandlers = new Map<string, Array<() => void>>()
+ const stoppers = new Map<string, () => void>()
+ /** The service that answers each request type, and its handler once it has started. */
+ const owners = new Map<string, { service: string; answer: ServiceRequest | null }>()
const inject = (target: string): void => {
if (faults.has(target)) throw new Error(`injected fault: ${target}`)
}
- const switchOff = (name: PortName, failed: number): void => {
+ const switchOff = (name: string, failed: number): void => {
if (off.has(name)) return
off.add(name)
- ports[name] = null
+ if (Object.hasOwn(ports, name)) ports[name as PortName] = null
log(`[services] ${name} switched off after ${failed} failures in ${Math.round(windowMs / 1000)}s · it stays off until the daemon restarts`)
for (const handler of [stoppers.get(name), ...offHandlers.get(name) ?? []]) {
try { handler?.() } catch (error) { log(`[services] switching ${name} off: ${describe(error)}`) }
}
}
- const failed = (name: PortName, member: string, error: unknown): void => {
+ const failed = (name: string, member: string, error: unknown): void => {
log(`[services] ${name}.${member} failed · ${describe(error)}`)
const at = now()
const recent = (failures.get(name) ?? []).filter((t) => at - t < windowMs)
@@ -133,21 +142,54 @@ export function createServiceHost(ports: CorePorts, options: ServiceHostOptions
return guarded as unknown as Port<K>
}
+ /** Take the request types a service declares, before its start: a type another service answers is a
+ * mistake in the code, and leaves this one off rather than taking the other's requests. */
+ const claim = (name: string, requests: readonly string[]): void => {
+ for (const type of requests) {
+ const owner = owners.get(type)
+ if (owner) throw new Error(`${type} is answered by ${owner.service}`)
+ }
+ for (const type of requests) owners.set(type, { service: name, answer: null })
+ }
+
+ /** Install a started service's handlers: exactly the types it declared, no more and no fewer. */
+ const install = (name: string, requests: readonly string[], answers: ServiceRequests | void): void => {
+ const given = Object.keys(answers ?? {})
+ const undeclared = given.filter((type) => !requests.includes(type))
+ const unanswered = requests.filter((type) => !given.includes(type))
+ if (undeclared.length) throw new Error(`it answers ${undeclared.join(', ')} without declaring ${undeclared.length > 1 ? 'them' : 'it'}`)
+ if (unanswered.length) throw new Error(`it declares ${unanswered.join(', ')} without answering ${unanswered.length > 1 ? 'them' : 'it'}`)
+ for (const type of requests) owners.set(type, { service: name, answer: (answers as ServiceRequests)[type] })
+ }
+
+ const didNotStart = (name: string, error: unknown): void => {
+ off.add(name)
+ if (Object.hasOwn(ports, name)) ports[name as PortName] = null
+ log(`[services] ${name} did not start · ${describe(error)} · the core runs without it`)
+ }
+
return {
/**
* Start one service. It fills its own port (`ports[name]`) in a staging copy; what it filled is
* installed guarded, and anything else it wrote is ignored. A service that leaves its port null
- * has said why itself and is simply off.
+ * has said why itself and is simply off. `requests` are the types it answers for the apps; its
+ * start returns their handlers.
*/
- start<K extends PortName>(name: K, start: (core: CoreApi, ports: CorePorts) => void, core: CoreApi, fallbacks: PortFallbacks<Port<K>>): void {
+ start<K extends PortName>(
+ name: K,
+ start: (core: CoreApi, ports: CorePorts) => ServiceRequests | void,
+ core: CoreApi,
+ fallbacks: PortFallbacks<Port<K>>,
+ requests: readonly string[] = [],
+ ): void {
const staging: CorePorts = { ...ports, [name]: null }
+ let answers: ServiceRequests | void
try {
+ claim(name, requests)
inject(name)
- start(core, staging)
+ answers = start(core, staging)
} catch (error) {
- off.add(name)
- ports[name] = null
- log(`[services] ${name} did not start · ${describe(error)} · the core runs without it`)
+ didNotStart(name, error)
return
}
const port = staging[name] as Port<K> | null
@@ -156,16 +198,66 @@ export function createServiceHost(ports: CorePorts, options: ServiceHostOptions
ports[name] = null
return
}
+ try {
+ install(name, requests, answers)
+ } catch (error) {
+ didNotStart(name, error)
+ return
+ }
const stop = (port as unknown as { stop?: () => unknown }).stop
if (typeof stop === 'function') stoppers.set(name, () => { void Promise.resolve(stop.call(port)).catch(() => {}) })
ports[name] = guard(name, port, fallbacks) as CorePorts[K]
},
+ /**
+ * Start a service the core never calls: it has no port, and only answers the apps' `requests`.
+ * Most features are this.
+ */
+ serve(name: string, start: (core: CoreApi) => ServiceRequests, core: CoreApi, requests: readonly string[]): void {
+ try {
+ if (Object.hasOwn(ports, name)) throw new Error('it has a port: start it with start()')
+ claim(name, requests)
+ inject(name)
+ install(name, requests, start(core))
+ } catch (error) {
+ didNotStart(name, error)
+ }
+ },
+ /**
+ * Answer a request a service in this process declared: false when none did, and the socket answers
+ * it. Never waits in line: the reply goes when the handler's promise settles.
+ */
+ route(type: string, payload: Record<string, unknown>, asker: Asker, reply: (result: Record<string, unknown>) => void): boolean {
+ const owner = owners.get(type)
+ if (!owner) return false
+ const { service, answer } = owner
+ if (off.has(service) || !answer) {
+ reply({ error: 'SERVICE_UNAVAILABLE', service, retryable: false })
+ return true
+ }
+ const failedWith = (error: unknown): void => {
+ failed(service, type, error)
+ reply({ error: 'SERVICE_FAILED', service })
+ }
+ let result: ReturnType<ServiceRequest>
+ try {
+ inject(`${service}.${type}`)
+ result = answer(payload, asker)
+ } catch (error) {
+ failedWith(error)
+ return true
+ }
+ void Promise.resolve(result).then((value) => {
+ if (typeof value === 'object' && value !== null) reply(value)
+ else failedWith(new Error(`${type} was answered with no reply`))
+ }, failedWith)
+ return true
+ },
/** Run `handler` once, when `name` is switched off. */
- onOff(name: PortName, handler: () => void): void {
+ onOff(name: string, handler: () => void): void {
offHandlers.set(name, [...offHandlers.get(name) ?? [], handler])
},
/** Whether `name` is off: it did not start, or it was switched off. */
- isOff(name: PortName): boolean {
+ isOff(name: string): boolean {
return off.has(name)
},
}
diff --git a/cli/src/core/serviceLinks.spec.ts b/cli/src/core/serviceLinks.spec.ts
index c9a533b5d..259688403 100644
--- a/cli/src/core/serviceLinks.spec.ts
+++ b/cli/src/core/serviceLinks.spec.ts
@@ -2,6 +2,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { createServiceLinks, HELD_MAX, type ServiceFrame } from './serviceLinks.js'
const TOKEN = 'a'.repeat(48)
+const ASKER = { local: false, owner: true }
function sink(accepting = true) {
const sent: ServiceFrame[] = []
@@ -43,8 +44,9 @@ describe('service links', () => {
const search = sink()
const link = links.accept('search', TOKEN, search, vi.fn())!
const reply = vi.fn()
- expect(links.route('session_search', { query: 'zebra', requestId: 'client-1' }, reply)).toBe(true)
- expect(search.sent).toEqual([{ type: 'session_search', payload: { query: 'zebra', requestId: 'route-1' } }])
+ // Who asked goes beside the payload, so a payload naming its own asker stands for nothing.
+ expect(links.route('session_search', { query: 'zebra', requestId: 'client-1', asker: { local: true } }, ASKER, reply)).toBe(true)
+ expect(search.sent).toEqual([{ type: 'session_search', payload: { query: 'zebra', requestId: 'route-1', asker: { local: true } }, asker: ASKER }])
// Answers that are not this request's are ignored: another type, an unknown id, no id.
link.receive({ type: 'session_tail_result', payload: { requestId: 'route-1', rows: [] } })
link.receive({ type: 'session_search_result', payload: { requestId: 'route-9', hits: [] } })
@@ -56,17 +58,17 @@ describe('service links', () => {
link.receive({ type: 'session_search_result', payload: { requestId: 'route-1', hits: ['again'] } })
expect(reply).toHaveBeenCalledOnce()
// What no service owns is the core's own.
- expect(links.route('agents_list', {}, reply)).toBe(false)
+ expect(links.route('agents_list', {}, ASKER, reply)).toBe(false)
})
it('answers SERVICE_UNAVAILABLE at once while the service is down, and when it cannot be sent to', () => {
const links = make()
const reply = vi.fn()
- expect(links.route('session_tail', {}, reply)).toBe(true)
+ expect(links.route('session_tail', {}, ASKER, reply)).toBe(true)
expect(reply).toHaveBeenCalledWith({ error: 'SERVICE_UNAVAILABLE', service: 'search', retryable: true })
links.accept('search', TOKEN, sink(false), vi.fn())
const again = vi.fn()
- links.route('session_tail', {}, again)
+ links.route('session_tail', {}, ASKER, again)
expect(again).toHaveBeenCalledWith({ error: 'SERVICE_UNAVAILABLE', service: 'search', retryable: true })
})
@@ -74,7 +76,7 @@ describe('service links', () => {
const links = make()
links.accept('search', TOKEN, sink(), vi.fn())
const reply = vi.fn()
- links.route('session_search', {}, reply)
+ links.route('session_search', {}, ASKER, reply)
vi.advanceTimersByTime(4_999)
expect(reply).not.toHaveBeenCalled()
vi.advanceTimersByTime(1)
@@ -85,11 +87,11 @@ describe('service links', () => {
const links = make({ owned: { search: ['session_search'], devices: ['harness_devices_list'] } })
links.accept('devices', TOKEN, sink(), vi.fn())
const elsewhere = vi.fn()
- links.route('harness_devices_list', {}, elsewhere)
+ links.route('harness_devices_list', {}, ASKER, elsewhere)
const closeOld = vi.fn()
const old = links.accept('search', TOKEN, sink(), closeOld)!
const waiting = vi.fn()
- links.route('session_search', {}, waiting)
+ links.route('session_search', {}, ASKER, waiting)
const newer = links.accept('search', TOKEN, sink(), vi.fn())!
expect(closeOld).toHaveBeenCalledWith(4409, 'replaced by a newer connection')
// The old connection's end does not take the newer one with it.
@@ -97,7 +99,7 @@ describe('service links', () => {
expect(links.connected('search')).toBe(true)
expect(waiting).not.toHaveBeenCalled()
const pending = vi.fn()
- links.route('session_search', {}, pending)
+ links.route('session_search', {}, ASKER, pending)
newer.closed()
expect(links.connected('search')).toBe(false)
expect(pending).toHaveBeenCalledWith({ error: 'SERVICE_UNAVAILABLE', service: 'search', retryable: true })
@@ -173,7 +175,7 @@ describe('service links', () => {
const search = sink()
links.accept('search', TOKEN, search, vi.fn())
const reply = vi.fn()
- links.route('session_search', {}, reply)
+ links.route('session_search', {}, ASKER, reply)
expect(String(search.sent[0].payload?.requestId)).toMatch(/^[0-9a-f-]{36}$/)
expect(warn).toHaveBeenCalledWith('[services] search connected')
// Answered, so its timer is cleared rather than left to fire.
diff --git a/cli/src/core/serviceLinks.ts b/cli/src/core/serviceLinks.ts
index 0e9d476df..0f76a46e4 100644
--- a/cli/src/core/serviceLinks.ts
+++ b/cli/src/core/serviceLinks.ts
@@ -10,6 +10,7 @@
* the few questions a service may ask it (`service_query`), nothing more.
*/
import { randomUUID, timingSafeEqual } from 'node:crypto'
+import type { Asker } from './api.js'
export interface ServiceFrame {
type: string
@@ -129,8 +130,9 @@ export function createServiceLinks(options: ServiceLinksOptions) {
}
},
- /** Route a request a service owns: false when no service owns `type`, and the core answers it itself. */
- route(type: string, payload: Record<string, unknown>, reply: (result: Record<string, unknown>) => void): boolean {
+ /** Route a request a service owns: false when no service owns `type`, and the core answers it itself.
+ * The asker goes beside the payload, never in it, so nothing a client writes can stand for it. */
+ route(type: string, payload: Record<string, unknown>, asker: Asker, reply: (result: Record<string, unknown>) => void): boolean {
const service = ownerOf.get(type)
if (!service) return false
const link = links.get(service)
@@ -140,7 +142,7 @@ export function createServiceLinks(options: ServiceLinksOptions) {
// Cleared whenever the entry is settled, so it only ever fires for one still waiting.
entry.timer = setTimer(() => settle(id, entry, unavailable(service)), timeoutMs)
waiting.set(id, entry)
- if (!link.sink.sendFrame({ type, payload: { ...payload, requestId: id } })) settle(id, entry, unavailable(service))
+ if (!link.sink.sendFrame({ type, payload: { ...payload, requestId: id }, asker })) settle(id, entry, unavailable(service))
return true
},
diff --git a/cli/src/harnessd/services.ts b/cli/src/harnessd/services.ts
index 2c182407c..d11686d52 100644
--- a/cli/src/harnessd/services.ts
+++ b/cli/src/harnessd/services.ts
@@ -289,11 +289,6 @@ export const KNOWN_SERVICES: Readonly<Record<string, Omit<ServiceSpec, 'name'>>>
search: { heapLimitMiB: 1_024, rssLimitMiB: 2_048 },
}
-/** The requests each of them answers, which the core routes to its process (core/serviceLinks.ts). */
-export const SERVICE_REQUESTS: Readonly<Record<string, readonly string[]>> = {
- search: ['session_search', 'session_tail'],
-}
-
/** Service timings from the environment (for tests and support); anything unset or invalid keeps its default. */
export function serviceOptions(env: NodeJS.ProcessEnv): ServiceSupervisorOptions {
const read = (name: string, fallback: number, min: number): number => {
diff --git a/cli/src/services/AGENTS.md b/cli/src/services/AGENTS.md
index 8737d653e..5a5d31167 100644
--- a/cli/src/services/AGENTS.md
+++ b/cli/src/services/AGENTS.md
@@ -9,24 +9,40 @@ let several people build features at once without touching the core or each othe
only through `core` (`src/core/api.ts`). If `CoreApi` lacks something you need, add it to `CoreApi`
in a separate, reviewed change; never import a core module, the registry, `cli.ts` or
`backendSocket.ts`.
-2. **If the core must call the service, give it a port.** Add `<Name>Port` and its fallbacks
+2. **If the apps call it, declare its requests and answer them from start.** Export the frame types,
+ `export const <NAME>_REQUESTS = ['<name>_list', …] as const`, and return their handlers from start:
+ `{ <name>_list: (payload, asker) => reply }`. A handler returns the reply (or a promise of it); the
+ core routes the request to it, replies under the asker's request id, and never waits in line for it.
+ Never add a case to `backendSocket.ts`'s switch or a slot to `BackendSocket`.
+ - Trust `asker`, never the payload: `if (!asker.owner) return { error: 'OWNER_REQUIRED' }`.
+ - While the service is off, its requests are answered `SERVICE_UNAVAILABLE`, never `UNSUPPORTED`
+ (the apps read that as "update the CLI"). That is why the types are declared up front.
+ - A handler that throws or rejects is answered `SERVICE_FAILED` and counts against the service.
+3. **If the core must call the service, give it a port.** Add `<Name>Port` and its fallbacks
(`<NAME>_FALLBACKS`) to `src/core/api.ts`. A fallback is what the core does while the service is
off or failing: `undefined` for nothing, a value, or `FAIL` to answer that one request
- `SERVICE_UNAVAILABLE`. The core only ever calls the port, never the service.
-3. **Start it in `src/cli.ts` with `serviceHost.start('<name>', start<Name>, coreApi, <NAME>_FALLBACKS)`.**
- Never call `start<Name>` directly: the host leaves a service off when its start throws, guards every
- call, and switches it off after five failures in a minute.
-4. **Test it to 100%** with `fakeCore()` (`src/testing/fakeCore.ts`); `npm run test:core` covers this
- folder. Prove failure isolation end to end with `HARNESSD_TEST_FAULTS=<name>` (its start fails) and
- `<name>.<member>` (one call fails); see `e2e/services.e2e.ts`.
+ `SERVICE_UNAVAILABLE`. The core only ever calls the port, never the service. Most features need no
+ port: only the apps call them.
+4. **Start it in `src/cli.ts` through the host, never directly:**
+ `serviceHost.serve('<name>', start<Name>, coreApi, <NAME>_REQUESTS)` for a service with no port, or
+ `serviceHost.start('<name>', start<Name>, coreApi, <NAME>_FALLBACKS, <NAME>_REQUESTS)` for one with a
+ port. The host leaves a service off when its start throws, guards every call and request, and
+ switches it off after five failures in a minute.
+5. **Test it to 100%** with `fakeCore()` (`src/testing/fakeCore.ts`); `npm run test:core` covers this
+ folder. Prove failure isolation end to end with `HARNESSD_TEST_FAULTS=<name>` (its start fails),
+ `<name>.<member>` (one port call fails) and `<name>.<request type>` (one request fails); see
+ `e2e/services.e2e.ts`.
+
+`store.ts` (the Harness Store: no port, four requests) is the example to copy for a feature;
+`search.ts` for a service the core also calls.
## Running in its own process
A service that can crash natively, hang or leak should run in its own process, where it costs only
itself. Search does: `HARNESSD_SERVICES=search`.
-- `src/harnessd/services.ts` runs it (`KNOWN_SERVICES`, with its memory budget) and lists the
- requests it answers (`SERVICE_REQUESTS`).
+- `src/harnessd/services.ts` runs it (`KNOWN_SERVICES`, with its memory budget). The core routes the
+ requests it declared (its `<NAME>_REQUESTS`) to its process, and the same handlers answer them there.
- `src/services/process.ts` is the process's side: heartbeats to the master, the connection to the
core with the master's token, reconnecting after core restarts.
- `src/core/serviceLinks.ts` is the core's side: it routes those requests to the process, answers
diff --git a/cli/src/services/process.spec.ts b/cli/src/services/process.spec.ts
index 8b818377b..4efe2de8a 100644
--- a/cli/src/services/process.spec.ts
+++ b/cli/src/services/process.spec.ts
@@ -107,6 +107,17 @@ describe('a service in its own process', () => {
]))
})
+ it('tells each handler who asked, as the core established it, and the least it could be when that is missing', async () => {
+ const asked: unknown[] = []
+ run({ requests: { session_search: (_payload: Record<string, unknown>, asker: unknown) => { asked.push(asker); return {} } } })
+ socket().open()
+ socket().say({ type: 'session_search', payload: { requestId: 'r1' }, asker: { local: true, owner: true } })
+ socket().say({ type: 'session_search', payload: { requestId: 'r2' }, asker: { local: 'yes', owner: 1 } })
+ socket().say({ type: 'session_search', payload: { requestId: 'r3' } })
+ await vi.waitFor(() => expect(asked).toHaveLength(3))
+ expect(asked).toEqual([{ local: true, owner: true }, { local: false, owner: false }, { local: false, owner: false }])
+ })
+
it('hears what the core tells it, and asks the core what it needs to know', async () => {
const onEvent = vi.fn()
let core: Parameters<NonNullable<ServiceProcessOptions['onConnected']>>[0] | null = null
diff --git a/cli/src/services/process.ts b/cli/src/services/process.ts
index ad0ae2fef..f69182aa6 100644
--- a/cli/src/services/process.ts
+++ b/cli/src/services/process.ts
@@ -10,6 +10,7 @@
* needs to know.
*/
import WebSocket from 'ws'
+import type { ServiceRequests } from '../core/api.js'
import { heartbeatInterval, processLoopDelay, type LoopDelay, type MasterChannel } from '../harnessd/coreLink.js'
type Payload = Record<string, unknown>
@@ -28,7 +29,7 @@ export interface ServiceProcessOptions {
machineId: string
token: string
/** The requests the core routes here, by type: each answered under `<type>_result`. */
- requests: Readonly<Record<string, (payload: Payload) => Payload | Promise<Payload>>>
+ requests: ServiceRequests
/** What the core tells this service (`service_event`). */
onEvent?: (payload: Payload) => void
/** Each time it is connected to a core: the first time, and after every core restart. */
@@ -108,7 +109,7 @@ export function runServiceProcess(options: ServiceProcessOptions): ServiceProces
}
const onFrame = (raw: WebSocket.RawData): void => {
- let frame: { type?: unknown; payload?: Payload }
+ let frame: { type?: unknown; payload?: Payload; asker?: { local?: unknown; owner?: unknown } }
try { frame = JSON.parse(raw.toString()) as typeof frame } catch { return }
const payload = frame.payload ?? {}
if (frame.type === 'connected') {
@@ -130,9 +131,11 @@ export function runServiceProcess(options: ServiceProcessOptions): ServiceProces
const handle = Object.hasOwn(options.requests, type) ? options.requests[type] : undefined
if (!handle) return
const requestId = payload.requestId
+ // Who asked, as the core established it; read as the least it could be if it is missing.
+ const asker = { local: frame.asker?.local === true, owner: frame.asker?.owner === true }
// A request that fails here is answered as failed, never left for the core's timeout.
void Promise.resolve()
- .then(() => handle(payload))
+ .then(() => handle(payload, asker))
.catch((error: unknown) => ({ error: 'SERVICE_FAILED', detail: error instanceof Error ? error.message : String(error) }))
.then((result) => send({ type: `${type}_result`, payload: { ...result, requestId } }))
}
diff --git a/cli/src/services/search.spec.ts b/cli/src/services/search.spec.ts
index b6d2faabc..25721561c 100644
--- a/cli/src/services/search.spec.ts
+++ b/cli/src/services/search.spec.ts
@@ -6,7 +6,7 @@ import type { SessionSearchIndexOptions } from '../lib/sessionSearch/indexer.js'
import { fakeCore } from '../testing/fakeCore.js'
import { SESSION_SEARCH_FILE } from '../lib/sessionSearch/command.js'
import { SessionSearchStore } from '../lib/sessionSearch/store.js'
-import { startSearch } from './search.js'
+import { SEARCH_REQUESTS, searchRequests, startSearch } from './search.js'
vi.mock('../lib/sessionSearch/store.js', () => ({ SessionSearchStore: { open: vi.fn() } }))
// The index, recording what it was built with so its callbacks can be driven directly.
@@ -27,6 +27,8 @@ const row = (over: Partial<RegisteredSession>) =>
({ agentId: 'a1', sessionId: 's1', engine: 'claude', registeredAt: 1, ...over }) as RegisteredSession
const reader = async () => []
+const ASKER = { local: false, owner: true }
+
function setup(agents: RegisteredSession[] = [], external: unknown[] = [], owned: string[] = []) {
const store = { ownedSessionIds: vi.fn(() => new Set(owned)) }
vi.mocked(SessionSearchStore.open).mockReturnValue(store as never)
@@ -39,8 +41,8 @@ function setup(agents: RegisteredSession[] = [], external: unknown[] = [], owned
},
})
const ports = emptyPorts()
- startSearch(core, ports)
- return { core, ports, store, index: ports.search as unknown as Recorded }
+ const requests = startSearch(core, ports)
+ return { core, ports, store, requests, index: ports.search as unknown as Recorded }
}
describe('the session search service', () => {
@@ -52,18 +54,19 @@ describe('the session search service', () => {
afterEach(() => { vi.restoreAllMocks(); vi.clearAllMocks() })
describe('starting', () => {
- it('opens the index in the core\'s data folder, starts it, and answers the core through its port', () => {
- const { core, index } = setup()
+ it('opens the index in the core\'s data folder, starts it, and answers the core through its port and the apps through its requests', () => {
+ const { core, index, requests } = setup()
expect(SessionSearchStore.open).toHaveBeenCalledWith(join('/data', SESSION_SEARCH_FILE))
expect(index.started).toBe(true)
expect(index.opts.openSessions).toBe(core.external.open)
+ expect(Object.keys(requests ?? {})).toEqual([...SEARCH_REQUESTS])
})
it('leaves the port empty, and says so, on a Node without node:sqlite', () => {
const { core } = setup()
vi.mocked(SessionSearchStore.open).mockReturnValueOnce(null)
const ports = emptyPorts()
- startSearch(core, ports)
+ expect(startSearch(core, ports)).toBeUndefined()
expect(ports.search).toBeNull()
expect(console.warn).toHaveBeenCalledWith('[search] node:sqlite is not available on this Node — session search is off')
})
@@ -81,6 +84,48 @@ describe('the session search service', () => {
})
})
+ describe('the requests it answers', () => {
+ const index = () => {
+ const searched: Array<[string, unknown]> = []
+ const tailed: Array<[string, unknown]> = []
+ const tail = { sessionId: 'sess-1', rows: [{ turn: 4, at: 1, ask: 'fix the dial', answer: 'Done.', tools: '' }], hasMore: true, total: 5, lastAt: 1 }
+ return {
+ searched,
+ tailed,
+ tail,
+ index: {
+ search: (query: string, options: unknown) => { searched.push([query, options]); return { hits: [], indexed: 3, pending: 0, ready: true, tookMs: 1 } },
+ tail: async (sessionId: string, options: unknown) => { tailed.push([sessionId, options]); return sessionId === 'sess-1' ? tail : null },
+ },
+ }
+ }
+
+ it('session_search: the words cut to 500 characters, and only finite numbers for the limit and the window', () => {
+ const { index: idx, searched } = index()
+ const requests = searchRequests(idx as never)
+ expect(requests.session_search({ query: 'dial scroll', limit: 12, from: 5, to: 9 }, ASKER)).toEqual({ hits: [], indexed: 3, pending: 0, ready: true, tookMs: 1 })
+ requests.session_search({ query: 'x'.repeat(600), limit: Number.NaN, from: '5', to: Infinity }, ASKER)
+ requests.session_search({ query: 7 }, ASKER)
+ expect(searched).toEqual([
+ ['dial scroll', { limit: 12, from: 5, to: 9 }],
+ ['x'.repeat(500), { limit: undefined, from: undefined, to: undefined }],
+ ['', { limit: undefined, from: undefined, to: undefined }],
+ ])
+ })
+
+ it('session_tail: a session\'s rows, paged by whole numbers only, and says what it cannot', async () => {
+ const { index: idx, tailed, tail } = index()
+ const requests = searchRequests(idx as never)
+ expect(await requests.session_tail({ sessionId: 'sess-1', beforeTurn: 9, maxChars: 8000 }, ASKER)).toEqual(tail)
+ await requests.session_tail({ sessionId: 'sess-1', beforeTurn: '9', maxChars: 1.5 }, ASKER)
+ expect(tailed).toEqual([['sess-1', { beforeTurn: 9, maxChars: 8000 }], ['sess-1', { beforeTurn: undefined, maxChars: undefined }]])
+ expect(await requests.session_tail({ sessionId: 'nope' }, ASKER)).toEqual({ error: 'NOT_INDEXED', sessionId: 'nope' })
+ expect(await requests.session_tail({}, ASKER)).toEqual({ error: 'BAD_SESSION' })
+ await requests.session_tail({ sessionId: 's'.repeat(300) }, ASKER)
+ expect(tailed.at(-1)?.[0]).toBe('s'.repeat(200))
+ })
+ })
+
describe('what it indexes', () => {
it('every agent with a transcript or a database history, named and stamped by its conversation', () => {
const agents = [
diff --git a/cli/src/services/search.ts b/cli/src/services/search.ts
index 47332ec0c..f57571801 100644
--- a/cli/src/services/search.ts
+++ b/cli/src/services/search.ts
@@ -1,68 +1,102 @@
/**
* Session search: every turn of every conversation on this machine, live and stopped, indexed from
* its transcript and searched by `session_search` (lib/sessionSearch/). Nothing leaves the machine
- * but the hits for a query. A Node without `node:sqlite` has no index; the RPC then says so.
+ * but the hits for a query. A Node without `node:sqlite` has no index, and search is off.
*
* A service on the core boundary (docs/design/2026-10-03-harnessd.md, step 13): it reads the core
- * only through `CoreApi`, and the core reaches it only through `ports.search`, which stays null when
- * there is no index.
+ * only through `CoreApi`, the core reaches it only through `ports.search`, which stays null when
+ * there is no index, and the apps through the two requests it answers.
*/
import { join } from 'node:path'
-import type { CoreApi, CorePorts } from '../core/api.js'
+import type { CoreApi, CorePorts, ServiceRequests } from '../core/api.js'
import { SESSION_SEARCH_FILE } from '../lib/sessionSearch/command.js'
import { SessionSearchIndex, folderWords, type SearchSource } from '../lib/sessionSearch/indexer.js'
import { SessionSearchStore } from '../lib/sessionSearch/store.js'
-export function startSearch(core: CoreApi, ports: CorePorts): void {
- ports.search = (() => {
- try {
- const store = SessionSearchStore.open(join(core.dataDir, SESSION_SEARCH_FILE))
- if (!store) {
- console.warn('[search] node:sqlite is not available on this Node — session search is off')
- return null
- }
- const index = new SessionSearchIndex({
- store,
- sources: () => {
- const own = core.agents.all()
- const sources = own.flatMap((s): SearchSource[] => {
- const readHistory = s.transcriptPath ? undefined : core.transcripts.databaseHistory(s)
- if (!s.sessionId || (!s.transcriptPath && !readHistory)) return []
- return [{
- agentId: s.agentId,
- sessionId: s.sessionId,
- engine: s.engine,
- transcriptPath: s.transcriptPath || null,
- header: [core.agents.displayName(s), s.title, folderWords(s.cwd)].filter(Boolean).join(' · '),
- // Conversation stamps only: the row's `touchedAt` includes discovery bookkeeping.
- changedAt: Math.max(s.lastTranscriptAt || 0, s.lastHookAt || 0) || s.boundAt || s.registeredAt || 0,
- readHistory,
- }]
- })
- // Conversations Harness did not start — any Harness agent's, earlier ones included, are not.
- const known = store.ownedSessionIds()
- for (const s of own) if (s.sessionId) known.add(s.sessionId)
- for (const e of core.external.sessions.list()) {
- // A conversation Harness holds under any of its ids is Harness's.
- if (known.has(e.sessionId) || e.aliases?.some((id) => known.has(id)) || (!e.transcriptPath && !e.readHistory)) continue
- sources.push({
- agentId: '', sessionId: e.sessionId, engine: e.engine, transcriptPath: e.transcriptPath,
- header: '', changedAt: e.mtime, external: { cwd: e.cwd, origin: e.origin, title: e.title },
- ...(e.readHistory ? { readHistory: e.readHistory } : {}),
- })
- }
- return sources
- },
- agents: () => core.agents.all().map((s) => s.agentId),
- discover: () => core.external.sessions.scan(),
- openSessions: core.external.open,
- log: (line) => console.log(line),
- })
- index.start()
- return index
- } catch (error) {
- console.error('[search] could not open the session index:', error instanceof Error ? error.message : error)
+/** The requests search answers for the apps. */
+export const SEARCH_REQUESTS = ['session_search', 'session_tail'] as const
+
+const number = (value: unknown) => typeof value === 'number' && Number.isFinite(value) ? value : undefined
+const integer = (value: unknown) => typeof value === 'number' && Number.isInteger(value) ? value : undefined
+
+/** The two requests, on an index: the same answers in the core's process and in search's own. */
+export function searchRequests(index: Pick<SessionSearchIndex, 'search' | 'tail'>): ServiceRequests {
+ return {
+ // Every conversation on this machine, searched by what was said in it. Synchronous and a few
+ // milliseconds: the index is local SQLite FTS5. The words searched for arrive sealed and the hits
+ // leave sealed — the relay reads neither. `from`/`to`: only sessions worked on in that window
+ // (epoch ms) — "the dial one from last week". The client reads the time words, so every machine
+ // searches the same window.
+ session_search: (payload) => {
+ const query = typeof payload.query === 'string' ? payload.query.slice(0, 500) : ''
+ return { ...index.search(query, { limit: number(payload.limit), from: number(payload.from), to: number(payload.to) }) }
+ },
+ // A session's latest rows, newest last; `beforeTurn` pages up from the first row the client has.
+ // The same index as `session_search`, so it reads no transcript for a preview.
+ session_tail: async (payload) => {
+ const sessionId = typeof payload.sessionId === 'string' ? payload.sessionId.slice(0, 200) : ''
+ if (!sessionId) return { error: 'BAD_SESSION' }
+ const tail = await index.tail(sessionId, { beforeTurn: integer(payload.beforeTurn), maxChars: integer(payload.maxChars) })
+ return tail ? { ...tail } : { error: 'NOT_INDEXED', sessionId }
+ },
+ }
+}
+
+/** Start search: its port for the core, and its requests for the apps, both on one index. */
+export function startSearch(core: CoreApi, ports: CorePorts): ServiceRequests | void {
+ const index = openIndex(core)
+ ports.search = index
+ if (index) return searchRequests(index)
+}
+
+function openIndex(core: CoreApi): SessionSearchIndex | null {
+ try {
+ const store = SessionSearchStore.open(join(core.dataDir, SESSION_SEARCH_FILE))
+ if (!store) {
+ console.warn('[search] node:sqlite is not available on this Node — session search is off')
return null
}
- })()
+ const index = new SessionSearchIndex({
+ store,
+ sources: () => {
+ const own = core.agents.all()
+ const sources = own.flatMap((s): SearchSource[] => {
+ const readHistory = s.transcriptPath ? undefined : core.transcripts.databaseHistory(s)
+ if (!s.sessionId || (!s.transcriptPath && !readHistory)) return []
+ return [{
+ agentId: s.agentId,
+ sessionId: s.sessionId,
+ engine: s.engine,
+ transcriptPath: s.transcriptPath || null,
+ header: [core.agents.displayName(s), s.title, folderWords(s.cwd)].filter(Boolean).join(' · '),
+ // Conversation stamps only: the row's `touchedAt` includes discovery bookkeeping.
+ changedAt: Math.max(s.lastTranscriptAt || 0, s.lastHookAt || 0) || s.boundAt || s.registeredAt || 0,
+ readHistory,
+ }]
+ })
+ // Conversations Harness did not start — any Harness agent's, earlier ones included, are not.
+ const known = store.ownedSessionIds()
+ for (const s of own) if (s.sessionId) known.add(s.sessionId)
+ for (const e of core.external.sessions.list()) {
+ // A conversation Harness holds under any of its ids is Harness's.
+ if (known.has(e.sessionId) || e.aliases?.some((id) => known.has(id)) || (!e.transcriptPath && !e.readHistory)) continue
+ sources.push({
+ agentId: '', sessionId: e.sessionId, engine: e.engine, transcriptPath: e.transcriptPath,
+ header: '', changedAt: e.mtime, external: { cwd: e.cwd, origin: e.origin, title: e.title },
+ ...(e.readHistory ? { readHistory: e.readHistory } : {}),
+ })
+ }
+ return sources
+ },
+ agents: () => core.agents.all().map((s) => s.agentId),
+ discover: () => core.external.sessions.scan(),
+ openSessions: core.external.open,
+ log: (line) => console.log(line),
+ })
+ index.start()
+ return index
+ } catch (error) {
+ console.error('[search] could not open the session index:', error instanceof Error ? error.message : error)
+ return null
+ }
}
diff --git a/cli/src/services/searchProcess.spec.ts b/cli/src/services/searchProcess.spec.ts
index e5792b7f1..4098f3728 100644
--- a/cli/src/services/searchProcess.spec.ts
+++ b/cli/src/services/searchProcess.spec.ts
@@ -11,6 +11,8 @@ vi.mock('./process.js', () => ({ runServiceProcess: vi.fn(() => ({ stop: vi.fn()
const SESSION = '0b0c1d2e-3f40-4a5b-8c6d-7e8f90a1b2c3'
+const ASKER = { local: false, owner: true }
+
describe('search in its own process', () => {
const dirs: string[] = []
const services: Array<{ stop(): void }> = []
@@ -57,22 +59,22 @@ describe('search in its own process', () => {
options.onConnected!(core([agent]))
options.onEvent!({ kind: 'touch', sessionId: SESSION })
await vi.waitFor(async () => {
- expect(JSON.stringify(await options.requests.session_search({ query: 'zebrafish' }))).toContain(SESSION)
+ expect(JSON.stringify(await options.requests.session_search({ query: 'zebrafish' }, ASKER))).toContain(SESSION)
}, { timeout: 15_000, interval: 250 })
- const tail = await options.requests.session_tail({ sessionId: SESSION, beforeTurn: 5, maxChars: 2_000 })
+ const tail = await options.requests.session_tail({ sessionId: SESSION, beforeTurn: 5, maxChars: 2_000 }, ASKER)
expect(tail.error).toBeUndefined()
// Something it is told that it does not act on changes nothing.
options.onEvent!({ kind: 'renamed', sessionId: SESSION })
- expect(JSON.stringify(await options.requests.session_search({ query: 'zebrafish' }))).toContain(SESSION)
+ expect(JSON.stringify(await options.requests.session_search({ query: 'zebrafish' }, ASKER))).toContain(SESSION)
options.onEvent!({ kind: 'deleteHistory', sessionId: SESSION })
- expect(JSON.stringify(await options.requests.session_search({ query: 'zebrafish', limit: 5 }))).not.toContain(SESSION)
+ expect(JSON.stringify(await options.requests.session_search({ query: 'zebrafish', limit: 5 }, ASKER))).not.toContain(SESSION)
})
it('answers what it cannot serve the way the core does', async () => {
const { options } = setup()
- expect(await options.requests.session_tail({})).toEqual({ error: 'BAD_SESSION' })
- expect(await options.requests.session_tail({ sessionId: 'never-indexed' })).toEqual({ error: 'NOT_INDEXED', sessionId: 'never-indexed' })
- expect((await options.requests.session_search({ query: 7, limit: 'x', from: Number.NaN })).error).toBeUndefined()
+ expect(await options.requests.session_tail({}, ASKER)).toEqual({ error: 'BAD_SESSION' })
+ expect(await options.requests.session_tail({ sessionId: 'never-indexed' }, ASKER)).toEqual({ error: 'NOT_INDEXED', sessionId: 'never-indexed' })
+ expect((await options.requests.session_search({ query: 7, limit: 'x', from: Number.NaN }, ASKER)).error).toBeUndefined()
// An event naming no session is nothing to act on.
options.onEvent!({ kind: 'touch' })
})
@@ -85,14 +87,15 @@ describe('search in its own process', () => {
options.onConnected!({ query: vi.fn(async () => ({ error: 'UNKNOWN_QUERY' })) })
options.onEvent!({ kind: 'touch', sessionId: SESSION })
await vi.waitFor(async () => {
- expect(JSON.stringify(await options.requests.session_search({ query: 'heron' }))).toContain(SESSION)
+ expect(JSON.stringify(await options.requests.session_search({ query: 'heron' }, ASKER))).toContain(SESSION)
}, { timeout: 15_000, interval: 250 })
})
- it('without an index (no node:sqlite) says search is unavailable, as the core would', async () => {
+ it('without an index (no node:sqlite) says search is off, as the core would', async () => {
const { options } = setup({ start: () => {} })
- expect(await options.requests.session_search({ query: 'anything' })).toEqual({ error: 'SEARCH_UNAVAILABLE' })
- expect(await options.requests.session_tail({ sessionId: SESSION })).toEqual({ error: 'SEARCH_UNAVAILABLE' })
+ const off = { error: 'SERVICE_UNAVAILABLE', service: 'search', retryable: false }
+ expect(await options.requests.session_search({ query: 'anything' }, ASKER)).toEqual(off)
+ expect(await options.requests.session_tail({ sessionId: SESSION }, ASKER)).toEqual(off)
options.onEvent!({ kind: 'touch', sessionId: SESSION })
})
@@ -124,6 +127,7 @@ describe('search in its own process', () => {
await expect(api.account.accessToken()).rejects.toThrow('search holds no credential')
api.clients.viewerChanged('live')
api.clients.gridNamed('grid')
+ api.clients.dshInstallStatus({ phase: 'clone' })
})
it('runs as a real service process, over the conversations this machine has, by default', () => {
diff --git a/cli/src/services/searchProcess.ts b/cli/src/services/searchProcess.ts
index 7b6110eb2..8468b733e 100644
--- a/cli/src/services/searchProcess.ts
+++ b/cli/src/services/searchProcess.ts
@@ -14,11 +14,7 @@ import { ExternalSessions, OpenSessions } from '../lib/sessionSearch/external.js
import { externalProviders } from '../lib/sessionSearch/externals/index.js'
import type { RegisteredSession } from '../lib/registry.js'
import { runServiceProcess, type CoreConnection, type ServiceProcess } from './process.js'
-import { startSearch } from './search.js'
-
-type Payload = Record<string, unknown>
-const number = (value: unknown) => typeof value === 'number' && Number.isFinite(value) ? value : undefined
-const integer = (value: unknown) => typeof value === 'number' && Number.isInteger(value) ? value : undefined
+import { SEARCH_REQUESTS, startSearch } from './search.js'
export interface SearchServiceOptions {
dataDir: string
@@ -60,7 +56,7 @@ export function searchCoreApi(
},
// Search holds no credential and talks to no window: these are never asked of it.
account: { mintGridName: async () => null, accessToken: () => Promise.reject(new Error('search holds no credential')) },
- clients: { viewerChanged: () => {}, gridNamed: () => {} },
+ clients: { viewerChanged: () => {}, gridNamed: () => {}, dshInstallStatus: () => {} },
}
}
@@ -74,27 +70,18 @@ export function runSearchService(options: SearchServiceOptions): ServiceProcess
}
const api = searchCoreApi(options.dataDir, () => agents, options.providers ?? externalProviders())
const ports = emptyPorts()
- ;(options.start ?? startSearch)(api, ports)
+ const answers = (options.start ?? startSearch)(api, ports)
const index = ports.search
+ // No index here (a Node without `node:sqlite`): search is off, as it would be in the core's process.
+ const off = { error: 'SERVICE_UNAVAILABLE', service: 'search', retryable: false }
const run = options.run ?? runServiceProcess
const service = run({
name: 'search',
socketPath: options.socketPath,
machineId: options.machineId,
token: options.token,
- // The same answers the core's own handlers give (backendSocket.ts `session_search`, `session_tail`).
- requests: {
- session_search: (payload: Payload) => index
- ? { ...index.search(typeof payload.query === 'string' ? payload.query.slice(0, 500) : '', { limit: number(payload.limit), from: number(payload.from), to: number(payload.to) }) }
- : { error: 'SEARCH_UNAVAILABLE' },
- session_tail: async (payload: Payload) => {
- if (!index) return { error: 'SEARCH_UNAVAILABLE' }
- const sessionId = typeof payload.sessionId === 'string' ? payload.sessionId.slice(0, 200) : ''
- if (!sessionId) return { error: 'BAD_SESSION' }
- const tail = await index.tail(sessionId, { beforeTurn: integer(payload.beforeTurn), maxChars: integer(payload.maxChars) })
- return tail ? { ...tail } : { error: 'NOT_INDEXED', sessionId }
- },
- },
+ // The same handlers as in the core's process (services/search.ts `searchRequests`).
+ requests: answers ?? Object.fromEntries(SEARCH_REQUESTS.map((type) => [type, () => off])),
onEvent: (payload) => {
const sessionId = typeof payload.sessionId === 'string' ? payload.sessionId : ''
if (!sessionId || !index) return
diff --git a/cli/src/services/store.spec.ts b/cli/src/services/store.spec.ts
new file mode 100644
index 000000000..df38f09aa
--- /dev/null
+++ b/cli/src/services/store.spec.ts
@@ -0,0 +1,77 @@
+import { describe, expect, it, vi } from 'vitest'
+import type { DshInstallProgress } from '../dsh/install.js'
+import { fakeCore } from '../testing/fakeCore.js'
+import { STORE_REQUESTS, startStore, type StoreDeps } from './store.js'
+
+const ASKER = { local: true, owner: true }
+
+function setup(over: Partial<StoreDeps> = {}) {
+ const deps: StoreDeps = {
+ refresh: vi.fn(async () => [{ id: 'acme/thing' }] as never),
+ rows: vi.fn((_installed, catalog) => (catalog ?? []).map((entry: { id: string }) => ({ id: entry.id, installed: false }))),
+ remove: vi.fn(() => ({ ok: true as const })),
+ mutate: vi.fn(async ({ id }: { id?: string }) => ({ ok: true as const, id: id ?? 'acme/from-url' })),
+ ...over,
+ }
+ const core = fakeCore()
+ return { core, deps, requests: startStore(core, deps) }
+}
+
+describe('the store service', () => {
+ it('answers exactly the requests it declares', () => {
+ expect(Object.keys(setup().requests).sort()).toEqual([...STORE_REQUESTS].sort())
+ // Started as the daemon starts it, with the real catalog and installs behind it.
+ expect(Object.keys(startStore(fakeCore())).sort()).toEqual([...STORE_REQUESTS].sort())
+ })
+
+ it('dsh_list: the rows for the catalog it refreshed, or what went wrong reading it', async () => {
+ const { requests, deps } = setup()
+ expect(await requests.dsh_list({}, ASKER)).toEqual({ dsh: [{ id: 'acme/thing', installed: false }] })
+ expect(deps.rows).toHaveBeenCalledWith(undefined, [{ id: 'acme/thing' }])
+ vi.mocked(deps.refresh).mockRejectedValueOnce(new Error('registry unreadable'))
+ expect(await requests.dsh_list({}, ASKER)).toEqual({ error: 'INTERNAL', detail: 'registry unreadable' })
+ vi.mocked(deps.refresh).mockRejectedValueOnce('offline')
+ expect(await requests.dsh_list({}, ASKER)).toEqual({ error: 'INTERNAL', detail: 'offline' })
+ })
+
+ it('dsh_remove: uninstalls a well-formed id and says what happened; refuses any other before removing anything', async () => {
+ const { requests, deps } = setup()
+ expect(await requests.dsh_remove({ id: 'acme/thing' }, ASKER)).toEqual({ ok: true, id: 'acme/thing' })
+ vi.mocked(deps.remove).mockReturnValueOnce({ ok: false, error: 'NOT_INSTALLED', detail: 'acme/none is not installed' })
+ expect(await requests.dsh_remove({ id: 'acme/none' }, ASKER)).toEqual({ error: 'NOT_INSTALLED', detail: 'acme/none is not installed' })
+ expect(await requests.dsh_remove({ id: '../../etc' }, ASKER)).toEqual({ error: 'INVALID_DSH', detail: 'dsh_remove needs an id' })
+ expect(await requests.dsh_remove({}, ASKER)).toEqual({ error: 'INVALID_DSH', detail: 'dsh_remove needs an id' })
+ expect(deps.remove).toHaveBeenCalledTimes(2)
+ })
+
+ it('dsh_update: updates a well-formed id, pushing its progress under that id, and says how it ended', async () => {
+ const progressed: DshInstallProgress[] = [{ id: null, phase: 'clone' } as DshInstallProgress, { id: 'acme/thing', phase: 'done' } as DshInstallProgress]
+ const { requests, deps, core } = setup({
+ mutate: vi.fn(async (_input, progress) => { for (const p of progressed) progress(p); return { ok: true as const, id: 'acme/thing' } }),
+ })
+ expect(await requests.dsh_update({ id: 'acme/thing', url: 'ignored' }, ASKER)).toEqual({ ok: true, id: 'acme/thing' })
+ expect(deps.mutate).toHaveBeenCalledWith({ id: 'acme/thing', update: true }, expect.any(Function))
+ expect(vi.mocked(core.clients.dshInstallStatus).mock.calls).toEqual([[{ id: 'acme/thing', phase: 'clone' }], [{ id: 'acme/thing', phase: 'done' }]])
+ vi.mocked(deps.mutate).mockResolvedValueOnce({ ok: false, error: 'LINKED_INSTALL', detail: 'Update the checkout' })
+ expect(await requests.dsh_update({ id: 'acme/thing' }, ASKER)).toEqual({ error: 'LINKED_INSTALL', detail: 'Update the checkout' })
+ vi.mocked(deps.mutate).mockRejectedValueOnce(new Error('disk full'))
+ expect(await requests.dsh_update({ id: 'acme/thing' }, ASKER)).toEqual({ error: 'INTERNAL', detail: 'disk full' })
+ expect(await requests.dsh_update({ id: '../../etc' }, ASKER)).toEqual({ error: 'INVALID_DSH', detail: 'dsh_update needs an id' })
+ expect(deps.mutate).toHaveBeenCalledTimes(3)
+ })
+
+ it('dsh_install: installs what a usable id or url names, pushing each phase under the id asked for', async () => {
+ const { requests, deps, core } = setup({
+ mutate: vi.fn(async (_input, progress) => { progress({ id: null, phase: 'clone', detail: 'cloning' } as DshInstallProgress); return { ok: true as const, id: 'acme/thing' } }),
+ })
+ expect(await requests.dsh_install({ id: 'acme/thing', ref: 'v2' }, ASKER)).toEqual({ ok: true, id: 'acme/thing' })
+ expect(deps.mutate).toHaveBeenCalledWith({ id: 'acme/thing', url: undefined, ref: 'v2' }, expect.any(Function))
+ expect(core.clients.dshInstallStatus).toHaveBeenCalledWith({ id: 'acme/thing', phase: 'clone', detail: 'cloning' })
+ vi.mocked(deps.mutate).mockResolvedValueOnce({ ok: false, error: 'SETUP_FAILED', detail: 'setup exited 1' })
+ expect(await requests.dsh_install({ url: 'https://example.com/thing.git' }, ASKER)).toEqual({ error: 'SETUP_FAILED', detail: 'setup exited 1' })
+ vi.mocked(deps.mutate).mockRejectedValueOnce('not an Error')
+ expect(await requests.dsh_install({ url: 'https://example.com/thing.git' }, ASKER)).toEqual({ error: 'INTERNAL', detail: 'not an Error' })
+ expect(await requests.dsh_install({ id: '../../etc', url: 'https://example.com/\n' }, ASKER)).toEqual({ error: 'INVALID_DSH', detail: 'dsh_install needs an id or a url' })
+ expect(deps.mutate).toHaveBeenCalledTimes(3)
+ })
+})
diff --git a/cli/src/services/store.ts b/cli/src/services/store.ts
new file mode 100644
index 000000000..d6c56b196
--- /dev/null
+++ b/cli/src/services/store.ts
@@ -0,0 +1,57 @@
+/**
+ * The Harness Store on this machine: the harnesses (DSHs) installed here, and installing, updating and
+ * removing one (`dsh_list`, `dsh_install`, `dsh_update`, `dsh_remove`; store/spec/README.md § Wire).
+ * The core never calls it, so it has no port: it only answers the apps. What each request reads off its
+ * payload and replies is dsh/wire.ts.
+ *
+ * Install and update take minutes for a toolchain. Their handlers return a promise, so the socket never
+ * holds anything else for them, and they say how they are going as `dsh_install_status` pushes, which
+ * the apps show in the create dialog. Agents already running from a harness that is removed keep running:
+ * their processes hold what they need.
+ */
+import type { CoreApi, ServiceRequests } from '../core/api.js'
+import { refreshDshRegistry } from '../dsh/catalog.js'
+import { removeDsh } from '../dsh/install.js'
+import { mutateDsh } from '../dsh/service.js'
+import { dshInstallReply, dshInstallRequest, dshInstallStatus, dshListRows, dshRemoveId, dshRemoveReply } from '../dsh/wire.js'
+
+/** The requests the store answers for the apps. */
+export const STORE_REQUESTS = ['dsh_list', 'dsh_install', 'dsh_update', 'dsh_remove'] as const
+
+export interface StoreDeps {
+ refresh: typeof refreshDshRegistry
+ remove: typeof removeDsh
+ mutate: typeof mutateDsh
+ rows: typeof dshListRows
+}
+
+const DEFAULTS: StoreDeps = { refresh: refreshDshRegistry, remove: removeDsh, mutate: mutateDsh, rows: dshListRows }
+
+const internal = (error: unknown): Record<string, unknown> =>
+ ({ error: 'INTERNAL', detail: error instanceof Error ? error.message : String(error) })
+
+export function startStore(core: CoreApi, deps: StoreDeps = DEFAULTS): ServiceRequests {
+ return {
+ dsh_list: () => deps.refresh().then((catalog) => ({ dsh: deps.rows(undefined, catalog) }), internal),
+ // The clone under ~/.harness/dsh goes (a linked install loses only its link), and the index forgets
+ // it, so a `dsh_list` after this no longer says installed. The store asks, and refreshes its list.
+ dsh_remove: (payload) => {
+ const id = dshRemoveId(payload)
+ if (!id) return { error: 'INVALID_DSH', detail: 'dsh_remove needs an id' }
+ return dshRemoveReply(id, deps.remove(id))
+ },
+ dsh_update: (payload) => {
+ const id = dshRemoveId(payload)
+ if (!id) return { error: 'INVALID_DSH', detail: 'dsh_update needs an id' }
+ return deps.mutate({ id, update: true }, (progress) => core.clients.dshInstallStatus(dshInstallStatus(progress, { id })))
+ .then(dshInstallReply, internal)
+ },
+ // Clone, set up and doctor a harness on this machine.
+ dsh_install: (payload) => {
+ const request = dshInstallRequest(payload)
+ if (!request) return { error: 'INVALID_DSH', detail: 'dsh_install needs an id or a url' }
+ return deps.mutate(request, (progress) => core.clients.dshInstallStatus(dshInstallStatus(progress, request)))
+ .then(dshInstallReply, internal)
+ },
+ }
+}
diff --git a/cli/src/socketBindings.spec.ts b/cli/src/socketBindings.spec.ts
index 97e312b14..d47c58ff4 100644
--- a/cli/src/socketBindings.spec.ts
+++ b/cli/src/socketBindings.spec.ts
@@ -57,7 +57,8 @@ describe('the socket\'s slots into the core', () => {
// Both kinds: one that starts null, and an optional callback with no initializer.
expect(slots).toContain('onCreateAgent')
expect(slots).toContain('onOutboundCommander')
- expect(slots.length).toBeGreaterThan(50)
+ // Dozens, and fewer as requests move into the services that answer them (core/serviceHost.ts).
+ expect(slots.length).toBeGreaterThan(30)
for (const name of Object.keys(NOT_BOUND_ON_PURPOSE)) expect(slots, name).toContain(name)
})
})
diff --git a/cli/src/testing/fakeCore.ts b/cli/src/testing/fakeCore.ts
index 777ec4f40..c696b2c0a 100644
--- a/cli/src/testing/fakeCore.ts
+++ b/cli/src/testing/fakeCore.ts
@@ -28,6 +28,6 @@ export function fakeCore(over: Overrides = {}): CoreApi {
...over.external,
},
account: { mintGridName: vi.fn(async () => null), accessToken: vi.fn(async () => 'token'), ...over.account },
- clients: { viewerChanged: vi.fn(), gridNamed: vi.fn(), ...over.clients },
+ clients: { viewerChanged: vi.fn(), gridNamed: vi.fn(), dshInstallStatus: vi.fn(), ...over.clients },
}
}
diff --git a/docs/design/2026-10-03-harnessd.md b/docs/design/2026-10-03-harnessd.md
index 3eb40f940..a8d10dc67 100644
--- a/docs/design/2026-10-03-harnessd.md
+++ b/docs/design/2026-10-03-harnessd.md
@@ -433,6 +433,19 @@ Done so far:
`cli/src/services/searchProcess.ts` runs the same index on a core API built in its own process.
`e2e/serviceProcesses.e2e.ts` proves on the real daemon that search killed, hung (stopped), leaking
or crashing on every start costs search alone: the core never restarts and agents keep working.
+- **Services answer their own requests** (core/serviceHost.ts `route`). A service declares the frame
+ types it answers (`<NAME>_REQUESTS`) and returns their handlers from its start. The socket hands
+ such a request to it before its own switch, with who asked (`Asker`: local, owner), and never
+ waits in line for it. While the service is off, its requests are answered `SERVICE_UNAVAILABLE`,
+ not `UNSUPPORTED`, which the desktop reads as "update the CLI": that is why the types are declared
+ before the start that could fail. A handler that throws, rejects or replies with nothing is
+ answered `SERVICE_FAILED` and counts toward switching the service off. The same handlers answer in
+ a service's own process, where the asker travels beside the payload, never in it.
+ - Moved out of the socket: search's `session_search` and `session_tail`, whose port lost the two
+ members only the apps called, and the store's `dsh_list`, `dsh_install`, `dsh_update` and
+ `dsh_remove` (`services/store.ts`, a service with no port). Five `BackendSocket` slots went with
+ them.
+ - Adding a request to a feature now touches only that feature's service.
- **The walls, written down and checked.** The rules live in `AGENTS.md` files that coding agents read:
`cli/` and one per layer (`src/core/`, `src/services/`, `src/harnessd/`). A `CLAUDE.md` beside each
imports it, so Claude Code reads the same rules as Codex. On its own, Claude Code reads `AGENTS.md`
diff --git a/docs/design/2026-10-04-harnessd-before-after.md b/docs/design/2026-10-04-harnessd-before-after.md
index 77b98ed8c..bf1bcc999 100644
--- a/docs/design/2026-10-04-harnessd-before-after.md
+++ b/docs/design/2026-10-04-harnessd-before-after.md
@@ -15,10 +15,10 @@ its plan are in [2026-10-03-harnessd.md](2026-10-03-harnessd.md). Paths are unde
│ cli.ts runForeground() one function, 5,306 lines, shared variables │
│ backendSocket.ts ~70 request types, 53 callbacks wired by hand │
│ │
-│ agents · tmux · discovery · hooks · transcripts · turns · input · questions│
+│ agents · tmux · discovery · hooks · transcripts · turns · input · questions│
│ relay · E2EE · sharing · dial · Wi-Fi device · search · viewers · grid │
│ workspaces · teams · recap model pool · … │
-│ everything inside one function: every part can touch every other │
+│ everything inside one function: every part can touch every other │
│ │
│ ✗ one bug anywhere (exception, hang, memory) takes the whole daemon down │
│ ✗ no supervisor: back only when the desktop app respawned it (up to ~70 s) │
@@ -45,7 +45,7 @@ and it crash-looped: every agent on the machine lost its daemon at once.
┌──────────────────────────────────────┴──────────────────────────────────────┐
│ ② CORE: must never go down src/core/ harness __run │
│ │
-│ core/agents/ create · fork · restart · stop · resume · close · │
+│ core/agents/ create · fork · restart · stop · resume · close · │
│ discovery · bind 15 files │
│ core/transcripts/ attach · live tail · relaunch marks · normalizers 6 │
│ core/turns/ working/idle · event funnel · cancel · recaps 7 │
@@ -54,7 +54,7 @@ and it crash-looped: every agent on the machine lost its daemon at once.
│ core/input.ts typed messages into the pane, in order │
│ core/questions.ts an agent's question to you, and your answer back │
│ │
-│ cli.ts runForeground() only wires these together: 5,306 → 2,636 lines │
+│ cli.ts runForeground() only wires these together: 5,306 → 2,663 lines │
│ each module: one factory, explicit dependencies, 100% test coverage │
│ │
│ core/api.ts the contract between core and services: │
@@ -67,10 +67,12 @@ and it crash-looped: every agent on the machine lost its daemon at once.
│ only through CoreApi / CorePorts
┌──────────────────────────────────────┴──────────────────────────────────────┐
│ ③ SERVICES: the extendable part src/services/ │
-│ in the core's process behind the host, or each in its own process │
+│ in the core's process behind the host, or each in its own process │
│ (HARNESSD_SERVICES=search: harnessd/services.ts runs it, the core routes │
│ to it through core/serviceLinks.ts) — search is the first │
+│ each answers its own requests from the apps: declared, routed to it │
│ │
+│ services/store.ts the Harness Store uses dsh/ │
│ services/search.ts session search uses lib/sessionSearch/ │
│ services/viewers.ts harness viewers (DSH) uses dsh/ │
│ services/models.ts Grid and local models uses lib/grid* │
@@ -100,12 +102,20 @@ and it crash-looped: every agent on the machine lost its daemon at once.
1. Put it in `services/<name>.ts` as `start<Name>(core: CoreApi, ports: CorePorts)`. It may only
ask the core what `CoreApi` offers.
-2. If the core must call it, add a port to `CorePorts` in `core/api.ts`, with fallbacks beside it:
- what the core does while the service is off or failing.
-3. Start it in `cli.ts` through `serviceHost.start(...)`, never directly. The host guards its start
- and every call, and switches it off when it keeps failing.
-4. Hold the file to 100% (`npm run test:core` covers `src/services/`). For the end-to-end suite,
- `HARNESSD_TEST_FAULTS=<name>` makes its start fail, and `<name>.<member>` makes one call fail.
+2. If the apps call it, declare its requests (`<NAME>_REQUESTS`) and return their handlers from its
+ start. The core routes each request to it, with who asked. Nothing changes in `backendSocket.ts`
+ or `cli.ts` but the one line that starts the service. While it is off, its requests are answered
+ `SERVICE_UNAVAILABLE`.
+3. If the core must call it, add a port to `CorePorts` in `core/api.ts`, with fallbacks beside it:
+ what the core does while the service is off or failing. Most features need no port.
+4. Start it in `cli.ts` through `serviceHost.serve(...)`, or `serviceHost.start(...)` when it has a
+ port; never directly. The host guards its start, every call and every request, and switches it off
+ when it keeps failing.
+5. Hold the file to 100% (`npm run test:core` covers `src/services/`). For the end-to-end suite,
+ `HARNESSD_TEST_FAULTS=<name>` makes its start fail, and `<name>.<member>` or `<name>.<request>`
+ makes one call or one request fail.
+
+`services/store.ts`, the Harness Store, is the example: no port, four requests, about 60 lines.
The same rules are written for coding agents in `AGENTS.md` files, one per layer: `cli/`, `src/core/`,
`src/services/` and `src/harnessd/`. Codex reads them directly. Claude Code reads them through a