Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

feat(web): enable linked-machine tools and interactive viewers

deea84e6c6b27b3
diff --git a/cli/scripts/share-harness-e2e.ts b/cli/scripts/share-harness-e2e.ts
index d8676ab12..a27096cf6 100644
--- a/cli/scripts/share-harness-e2e.ts
+++ b/cli/scripts/share-harness-e2e.ts
@@ -150,7 +150,7 @@ try {
run('backend', join(repo, 'backend'), ['--import', 'tsx', 'src/server.ts'], backendEnv)
await until('backend', async () => fetch(`${backendUrl}/api/health`).then(r => r.ok).catch(() => false))
const engine = join(root, 'codex')
- await writeFile(engine, `#!${process.execPath}\nif (process.argv.includes('--version')) { console.log('codex-cli 1.0.0'); process.exit(0) }\nif (process.argv.includes('--help')) { console.log('--approve-for-me --dangerously-bypass-approvals-and-sandbox --ask-for-approval'); process.exit(0) }\nprocess.title = 'codex'; process.stdin.setRawMode?.(true); console.log('SHARING_READY'); process.stdin.on('data', x => process.stdout.write('ECHO:' + x + '\\r\\n')); setInterval(() => {}, 1000);\n`, { mode: 0o700 })
+ await writeFile(engine, `#!${process.execPath}\nif (process.argv.includes('--version')) { console.log('codex-cli 1.0.0'); process.exit(0) }\nif (process.argv.includes('--help')) { console.log('--approve-for-me --dangerously-bypass-approvals-and-sandbox --ask-for-approval'); process.exit(0) }\nconst { appendFileSync } = require('node:fs'); process.title = 'codex'; process.stdin.setRawMode?.(true); console.log('SHARING_READY'); process.stdin.on('data', x => { appendFileSync(${JSON.stringify(join(root, 'fixture-input.bin'))}, x); process.stdout.write('ECHO:' + x + '\\r\\n'); }); setInterval(() => {}, 1000);\n`, { mode: 0o700 })
const fixture = join(root, 'harness-source')
await mkdir(join(fixture, 'template'), { recursive: true })
await writeFile(join(fixture, 'harness.json'), JSON.stringify({ spec: 1, id: 'fixture/sharing', name: 'Sharing demo', engine: 'codex',
@@ -158,7 +158,8 @@ try {
viewer: { command: './viewer.mjs', url: 'http://127.0.0.1:${port}/' }, verdict: '.harness/verdict.json' }))
await writeFile(join(fixture, 'AGENTS.md'), 'Share harness test fixture.\n')
await writeFile(join(fixture, 'template/result.txt'), 'Sharing demo')
- await writeFile(join(fixture, 'viewer.mjs'), `#!${process.execPath}\nimport { createServer } from 'node:http';\ncreateServer((req, res) => { res.setHeader('Content-Type', 'text/html'); res.end('<html><body style="background:#152238;color:white;font:40px sans-serif"><h1>Live harness</h1><p id="clock"></p><script>setInterval(() => document.getElementById("clock").textContent=Date.now(), 200)</script></body></html>'); }).listen(Number(process.env.HARNESS_VIEWER_PORT), '127.0.0.1');\n`, { mode: 0o700 })
+ const viewerPage = `<html><body style="margin:0;background:#152238;color:white;font:40px sans-serif"><h1 style="position:absolute;left:20px;top:20px;margin:0;font-size:36px">Live harness</h1><button style="position:absolute;left:20px;top:100px;width:200px;height:56px;font-size:24px" onclick="fetch('/event?click=1')">Click me</button><input aria-label="Viewer text" style="position:absolute;left:20px;top:180px;width:400px;height:56px;font-size:24px" oninput="fetch('/event?text='+encodeURIComponent(this.value))"><p id="clock" style="position:absolute;left:20px;top:250px"></p><script>setInterval(() => document.getElementById('clock').textContent=Date.now(), 200)</script></body></html>`
+ await writeFile(join(fixture, 'viewer.mjs'), `#!${process.execPath}\nimport { createServer } from 'node:http';\nimport { appendFileSync } from 'node:fs';\ncreateServer((req, res) => { if (req.url.startsWith('/event?')) { appendFileSync(${JSON.stringify(join(root, 'viewer-events.log'))}, req.url + '\\n'); res.end('ok'); return; } res.setHeader('Content-Type', 'text/html'); res.end(${JSON.stringify(viewerPage)}); }).listen(Number(process.env.HARNESS_VIEWER_PORT), '127.0.0.1');\n`, { mode: 0o700 })
const machines: Array<{ machineId: string; port: number; env: NodeJS.ProcessEnv; child: ChildProcess; socket: string; workspace: string }> = []
for (const account of accounts.slice(0, 3)) {
const folder = join(root, account.name), data = join(folder, 'data'), auth = join(folder, 'auth'), workspace = join(folder, 'project')
@@ -197,6 +198,23 @@ try {
await until('fixture model ready', () => owner.text.includes('SHARING_READY'))
owner.input('owner-before-sharing')
await until('owner terminal input', () => owner.text.includes('ECHO:owner-before-sharing'))
+ const workspaceCheck = process.env.HARNESS_WORKSPACE_BROWSER_CHECK
+ if (workspaceCheck) {
+ const password = `Fixture-only-${randomUUID()}`
+ const response = await fetch(`http://127.0.0.1:${host.port}/api/remote-password/set`, {
+ method: 'POST', headers: { 'content-type': 'application/json', 'x-adapter-local': '1' },
+ body: JSON.stringify({ password }),
+ })
+ assert.equal(response.status, 200, await response.text())
+ const path = join(root, 'browser-workspace.json')
+ await writeFile(path, JSON.stringify({ origin: process.env.HARNESS_SHARE_BROWSER_ORIGIN,
+ backendUrl, token: accounts[0].token, machineId: host.machineId, agentId, password, root }), { mode: 0o600 })
+ const result = await exec(process.execPath, [workspaceCheck, path], { env: process.env, timeout: 240_000 })
+ console.log(result.stdout)
+ // The browser took control and resized this terminal. Reclaim it explicitly before the
+ // observer assertions below; viewers must inherit the controller's dimensions.
+ await owner.open(agentId, 110, 33)
+ }
console.log('Owner input verified; inviting Ken and Diego')
const invitation = await owner.rpc('harness_share_invite', { agentId, emails: [accounts[1].email.toUpperCase(), accounts[2].email], days: 30 })
assert.equal(invitation.shares.length, 2, JSON.stringify(invitation))
diff --git a/cli/src/backendSocket.spec.ts b/cli/src/backendSocket.spec.ts
index e9866d161..32d8ea15d 100644
--- a/cli/src/backendSocket.spec.ts
+++ b/cli/src/backendSocket.spec.ts
@@ -203,6 +203,53 @@ describe('agent_update opened: one "last used" for every app', () => {
})
describe('viewer forwarding authentication', () => {
+ it.each(['command_bar', 'route_task', 'route_send'])('requires a sealed owner session for %s', async type => {
+ const socket = new BackendSocket('token'), internals = socket as any
+ const request = vi.spyOn(socket.ownerCommands, 'request').mockResolvedValue({ ok: true })
+ vi.spyOn(internals.e2ee, 'hasSession').mockReturnValue(true)
+ const role = vi.spyOn(internals.e2ee, 'sessionRole').mockReturnValue('web')
+ const clear = { type, payload: { requestId: 'one', text: 'fixture task' } }
+ vi.spyOn(internals.e2ee, 'unwrapDown').mockReturnValue(clear)
+ const sealedReply = vi.spyOn(internals.e2ee, 'wrapRpcReply').mockReturnValue({ type: `${type}_result`, payload: { __e2e: 'sealed' } })
+ await internals.dispatchDown(clear, 'remote')
+ expect(request).not.toHaveBeenCalled()
+ const sealed = { type, payload: { __e2e: { v: 1, k: 'p', n: 1, ct: 'fixture' } } }
+ role.mockReturnValue('device')
+ await internals.dispatchDown(sealed, 'remote')
+ expect(request).not.toHaveBeenCalled()
+ role.mockReturnValue('web')
+ await internals.dispatchDown(sealed, 'remote')
+ expect(request).toHaveBeenCalledWith('remote', type, clear.payload)
+ expect(sealedReply).toHaveBeenCalledWith('remote', `${type}_result`, 'one', { ok: true })
+ await socket.stop()
+ })
+
+ it('allows interactive viewers only on a sealed owner web connection or trusted loopback', async () => {
+ const socket = new BackendSocket('token')
+ const internals = socket as any
+ const request = vi.spyOn(socket.interactiveViewers, 'request').mockResolvedValue({ data: 'jpeg' })
+ vi.spyOn(internals.e2ee, 'hasSession').mockReturnValue(true)
+ const role = vi.spyOn(internals.e2ee, 'sessionRole').mockReturnValue('web')
+ const clear = { type: 'viewer_surface', payload: { requestId: 'one', surfaceId: 'surface', agentId: 'a', op: 'frame' } }
+ vi.spyOn(internals.e2ee, 'unwrapDown').mockReturnValue(clear)
+ const reply = vi.spyOn(internals.e2ee, 'wrapRpcReply').mockReturnValue({ type: 'viewer_surface_result', payload: { __e2e: 'sealed' } })
+ await internals.dispatchDown(clear, 'remote')
+ expect(request).not.toHaveBeenCalled()
+ const sealed = { type: 'viewer_surface', payload: { __e2e: { v: 1, k: 'p', n: 1, ct: 'fixture' } } }
+ role.mockReturnValue('device')
+ await internals.dispatchDown(sealed, 'remote')
+ expect(request).not.toHaveBeenCalled()
+ role.mockReturnValue('web')
+ await internals.dispatchDown(sealed, 'remote')
+ expect(request).toHaveBeenCalledWith('remote', clear.payload)
+ expect(reply).toHaveBeenCalledWith('remote', 'viewer_surface_result', 'one', { data: 'jpeg' })
+ socket.registerLocalClient('local:viewer', { sendFrame: () => true, sendBinary: () => true })
+ await internals.dispatchDown(clear, 'local:viewer')
+ expect(request).toHaveBeenCalledWith('local:viewer', clear.payload)
+ await socket.unregisterLocalClient('local:viewer')
+ await socket.stop()
+ })
+
it('requires encryption and a web-role session remotely, while permitting trusted local clients', async () => {
const socket = new BackendSocket('token')
const internals = socket as any
diff --git a/cli/src/backendSocket.ts b/cli/src/backendSocket.ts
index 8396c7352..8cc01d629 100644
--- a/cli/src/backendSocket.ts
+++ b/cli/src/backendSocket.ts
@@ -74,6 +74,8 @@ import { messagesToEvents, windowRawLines, subagentStatsFromRawLines, type Sessi
import { listFileTree, readProjectFile } from './lib/files.js'
import { MediaPreviewError, readMediaPreviewChunk } from './lib/mediaPreview.js'
import { ViewerForwarder } from './lib/viewerForwarder.js'
+import { InteractiveViewers } from './lib/interactiveViewer.js'
+import { OwnerCommands, OWNER_COMMAND_TYPES } from './lib/ownerCommands.js'
import { VIEWER_DOWN_TYPES } from './lib/viewerWire.js'
import { codexMessagesToEvents, windowCodexLines } from './engines/codex/normalizer.js'
import { codexSubagentResolverFor } from './engines/codex/subagent.js'
@@ -521,6 +523,7 @@ export class BackendSocket {
harnessSharing: HarnessShareOwner | null = null
dshFrameProvider: ((session: RegisteredSession) => AgentDshContext | null) | null = null
viewerTargetProvider: ((agentId: string) => string | null) | null = null
+ readonly interactiveViewers = new InteractiveViewers(agentId => this.viewerTargetProvider?.(agentId) ?? null)
readonly viewerForwarder = new ViewerForwarder({
target: (agentId) => this.viewerTargetProvider?.(agentId) ?? null,
send: (connId, type, payload) => {
@@ -537,6 +540,7 @@ export class BackendSocket {
engineProbeProvider: typeof probeEngines = probeEngines
/** Entrypoint override for isolated integration fixtures; never a wire option. */
orchestratorCommand: string | null = null
+ readonly ownerCommands = new OwnerCommands()
onCancelOrchestratorMessage: ((deliveryId: string) => boolean) | null = null
orchestratorDelivery(event: SessionInputDelivery): void {
this.orchestratorService?.delivery(event)
@@ -775,7 +779,7 @@ export class BackendSocket {
isConnectionAvailable: connId => this.directDeviceSinks.has(connId) || this.isConnected(),
onIdentityPaired: (connId, pub) => { if (this.directDeviceSinks.has(connId)) this.directDevicePins.set(connId, pub) },
onIdentityRevoked: identity => { this.autonomousDeviceRelay?.revoke(identity); this.onDirectDeviceRevoked?.(fingerprint(b64d(identity))) },
- onSessionDropped: (connId) => this.viewerForwarder.closeConnection(connId),
+ onSessionDropped: (connId) => { this.viewerForwarder.closeConnection(connId); this.interactiveViewers.closeConnection(connId); this.ownerCommands.closeConnection(connId) },
})
this.terminalP2p = new TerminalP2pResponderPool({
sendSignal: (connId, type, payload) => this.sendP2pSignal(connId, type, payload),
@@ -956,6 +960,7 @@ export class BackendSocket {
this.harnessSharing?.closeAll()
this.setCommanderCount(0, null) // active count is unknown until the next __clients snapshot
this.viewerForwarder.closeAll()
+ this.interactiveViewers.closeAll(); this.ownerCommands.closeAll()
void this.terminalStreams?.closeConnectionsWhere(
(connId) => !isLocalClientId(connId),
'backend disconnected',
@@ -1028,6 +1033,7 @@ export class BackendSocket {
this.stopGridModelsPush()
this.orchestratorService?.stop()
this.viewerForwarder.closeAll()
+ this.interactiveViewers.closeAll(); this.ownerCommands.closeAll()
if (this.heartbeat) this.heartbeat.stop()
if (this.appPing) clearInterval(this.appPing)
await this.terminalStreams?.stop()
@@ -1193,6 +1199,7 @@ export class BackendSocket {
if (!this.localClients.delete(connId)) return
this.rowStateWindows.delete(connId)
this.viewerForwarder.closeConnection(connId)
+ this.interactiveViewers.closeConnection(connId); this.ownerCommands.closeConnection(connId)
// The window left before any link could hear it attach: nothing happened, as far as the backend
// is concerned, and a later link must not be told otherwise.
if (this.localClients.size === 0) this.appOpenOwed = false
@@ -1404,7 +1411,7 @@ export class BackendSocket {
const resultType = `${type}_result`
if (connId && deviceDump.enabled && this.e2ee.sessionRole(connId) === 'device') deviceDump.record('out', 'legacy', connId, { type: resultType, payload: { requestId, ...payload } })
// Before the E2EE wrap: an RPC reply is only readable here.
- if (env.LOG_FRAMES && !type.startsWith('grid_fleet_') && type !== 'agent_read_file' && type !== 'project_preview' && type !== 'git_project_info' && type !== 'git_pull_request' && !SHARE_REQUEST_TYPES.has(type)) logFrame('→', connId ? `conn:${sid(connId)}` : 'backend', { type: resultType, payload: { requestId, ...payload } })
+ if (env.LOG_FRAMES && !OWNER_COMMAND_TYPES.has(type) && !type.startsWith('viewer_') && type !== 'phone_pair' && type !== 'api_connections' && type !== 'orchestrator' && !type.startsWith('grid_fleet_') && type !== 'agent_read_file' && type !== 'project_preview' && type !== 'git_project_info' && type !== 'git_pull_request' && !SHARE_REQUEST_TYPES.has(type)) logFrame('→', connId ? `conn:${sid(connId)}` : 'backend', { type: resultType, payload: { requestId, ...payload } })
if (this.localClients.has(connId)) {
this.sendTo(connId, { type: resultType, payload: { requestId, ...payload } })
return
@@ -1531,7 +1538,7 @@ export class BackendSocket {
// than as an opaque __e2e envelope.
// Terminal frames contain raw keystrokes, paste text and screen bytes after
// unwrap. Never pass them to the frame logger, even in diagnostic mode.
- if (env.LOG_FRAMES && !type.startsWith('terminal_') && !type.startsWith('viewer_') && !type.startsWith('grid_fleet_') && type !== 'agent_read_file' && type !== 'project_preview' && type !== 'git_project_info' && type !== 'git_pull_request' && type !== 'orchestrator' && !SHARE_REQUEST_TYPES.has(type)) {
+ if (env.LOG_FRAMES && !OWNER_COMMAND_TYPES.has(type) && !type.startsWith('terminal_') && !type.startsWith('viewer_') && !type.startsWith('grid_fleet_') && type !== 'agent_read_file' && type !== 'project_preview' && type !== 'git_project_info' && type !== 'git_pull_request' && type !== 'orchestrator' && type !== 'api_connections' && type !== 'phone_pair' && !SHARE_REQUEST_TYPES.has(type)) {
logFrame('←', connId ? `conn:${sid(connId)}` : 'backend', frame)
}
const reply = (t: string, rid: unknown, p: Record<string, unknown>): void => this.emitReply(connId, t, rid, p)
@@ -1572,6 +1579,7 @@ export class BackendSocket {
this.autonomousDeviceRelay?.drop(connId)
this.e2ee.dropSession(connId)
this.viewerForwarder.closeConnection(connId)
+ this.interactiveViewers.closeConnection(connId); this.ownerCommands.closeConnection(connId)
await this.terminalP2p.closeConnection(connId, 'client_disconnected', false)
this.p2pPendingOpens.delete(connId)
this.p2pStreams.delete(connId)
@@ -1641,15 +1649,21 @@ export class BackendSocket {
const requestId = payload.requestId
if (type === 'api_connections') {
- if (!local) { reply(type, requestId, { error: 'LOCAL_ONLY', detail: 'Manage APIs on this computer.' }); return }
+ if (!local && this.e2ee.sessionRole(connId) !== 'web') { reply(type, requestId, { error: 'OWNER_REQUIRED' }); return }
reply(type, requestId, apiConnectionsRequest(this.apiConnections, payload))
return
}
- // Same-host only until remote viewer transport and remote task ownership exist.
- // Refuse before parsing project content; never send it to the relay as plaintext.
+ // A paired owner can run the machine's orchestrator; observers and device sessions cannot.
+ // Both requests and replies are encrypted, including project artifacts.
+ if (OWNER_COMMAND_TYPES.has(type)) {
+ if (!local && this.e2ee.sessionRole(connId) !== 'web') { reply(type, requestId, { error: 'OWNER_REQUIRED' }); return }
+ void this.ownerCommands.request(connId, type, payload).then(result => reply(type, requestId, result))
+ return
+ }
+
if (type === 'orchestrator') {
- if (!local) { reply(type, requestId, { error: 'LOCAL_ONLY', detail: 'Orchestrator projects run on the local machine.' }); return }
+ if (!local && this.e2ee.sessionRole(connId) !== 'web') { reply(type, requestId, { error: 'OWNER_REQUIRED' }); return }
// Detached: a large artifact snapshot must not block cancel/status on this connection.
void orchestratorRequest(this.orchestration(), payload)
.then(result => reply(type, requestId, result))
@@ -1657,6 +1671,15 @@ export class BackendSocket {
return
}
+ if (type === 'viewer_surface') {
+ if (!local && this.e2ee.sessionRole(connId) !== 'web') return
+ // Rendering and input never hold up terminal traffic on the ordered machine queue.
+ void this.interactiveViewers.request(connId, payload)
+ .then(result => reply(type, requestId, result))
+ .catch(() => reply(type, requestId, { error: 'VIEWER_UNAVAILABLE' }))
+ return
+ }
+
if (type.startsWith('viewer_')) {
if (VIEWER_DOWN_TYPES.has(type) && (local || this.e2ee.sessionRole(connId) === 'web')) {
this.viewerForwarder.handle(connId, type, payload)
@@ -1709,6 +1732,10 @@ export class BackendSocket {
case 'device_e2ee_pair':
await this.e2ee.pairDeviceFromTrustedWeb(connId, payload)
return
+ case 'phone_pair':
+ // CPace waits for another connection. Do not block this browser's terminal queue.
+ void this.e2ee.pairPhoneFromTrustedWeb(connId, payload)
+ return
case 'e2ee_pairings_list':
reply(type, requestId, { pairs: this.e2ee.listPaired(connId) })
diff --git a/cli/src/cli.ts b/cli/src/cli.ts
index 4e983f630..b920da118 100644
--- a/cli/src/cli.ts
+++ b/cli/src/cli.ts
@@ -1867,6 +1867,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {
onUrl: (agentId, url) => {
dshFrameFor(agentId).viewerUrl = url
backendRef?.viewerForwarder.refresh(agentId)
+ backendRef?.interactiveViewers.refresh(agentId)
syncCompanion(agentId)
},
log: (line) => console.log(line),
@@ -4246,7 +4247,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {
// first fifteen they can SEE. An earlier cut put open tiles first, on the theory that working on
// something is a statement about relevance; it is, but it also made the fifteen unpredictable from
// the screen, and predictable beat clever here (owner's call).
- onRouteTask: async (text) => {
+ onRouteTask: backend.ownerCommands.onRouteTask = async (text) => {
const host = cableHostRef
if (!host) return { agentId: '', machineId: '', name: '', confidence: 0, reason: 'no agent list yet', candidates: [], weighed: 0, machines: 0, via: '' }
// Whatever the daemon knows right now. This also kicks a refresh of the remote machines, so a list
@@ -4364,7 +4365,7 @@ async function runForeground(session: AuthSession | null): Promise<void> {
// one the dial has been using for every voice turn.
// A window that connects after the dial did has missed the `dial_status` that announced it.
dialStatus: () => cableHostRef?.currentDialStatus() ?? { attached: false },
- onRouteSend: (agentId, text) => {
+ onRouteSend: backend.ownerCommands.onRouteSend = (agentId, text) => {
const sent = cableHostRef?.sendTurn(agentId, text) ?? { ok: false as const, machine: '', reason: 'no agent list yet' }
console.log(`[route] ⌘K → ${sid(agentId)} · bytes=${Buffer.byteLength(text, 'utf8')}`
+ (sent.ok ? '' : ` · REFUSED: ${sent.reason}${sent.machine ? ` (${sent.machine})` : ''}`))
diff --git a/cli/src/lib/apiConnectionsRpc.spec.ts b/cli/src/lib/apiConnectionsRpc.spec.ts
index 839b5a6bb..59de0ca55 100644
--- a/cli/src/lib/apiConnectionsRpc.spec.ts
+++ b/cli/src/lib/apiConnectionsRpc.spec.ts
@@ -21,7 +21,7 @@ describe('API connection RPC', () => {
await socket.stop()
})
- it('refuses remote access before touching the credential store', async () => {
+ it('refuses unsealed remote access before touching the credential store', async () => {
const socket = new BackendSocket('fixture')
const list = vi.spyOn(ApiConnections.prototype, 'list')
const save = vi.spyOn(ApiConnections.prototype, 'save')
@@ -32,16 +32,58 @@ describe('API connection RPC', () => {
await socket.stop()
})
- it('refuses authenticated encrypted remote management too', async () => {
+ it('refuses a sealed device session before touching the credential store', async () => {
const socket = new BackendSocket('fixture')
const save = vi.spyOn(ApiConnections.prototype, 'save')
const reply = vi.spyOn(socket as any, 'emitReply').mockImplementation(() => {})
+ vi.spyOn((socket as any).e2ee, 'sessionRole').mockReturnValue('device')
vi.spyOn((socket as any).e2ee, 'unwrapDown').mockReturnValue({
type: 'api_connections', payload: { action: 'save', requestId: 'remote', connection: { provider: 'fal', apiKey: 'must-stay-local' } },
})
await (socket as any).dispatchDown({ type: 'api_connections', payload: { __e2e: { v: 1, k: 'p', n: 1, ct: 'fixture' } } }, 'remote')
expect(save).not.toHaveBeenCalled()
- expect(reply).toHaveBeenCalledWith('remote', 'api_connections', 'remote', { error: 'LOCAL_ONLY', detail: 'Manage APIs on this computer.' })
+ expect(reply).toHaveBeenCalledWith('remote', 'api_connections', 'remote', { error: 'OWNER_REQUIRED' })
+ await socket.stop()
+ })
+
+ it.each(['list', 'save', 'remove'])('allows sealed owner web %s with a sealed metadata-only reply', async action => {
+ const socket = new BackendSocket('fixture'), internals = socket as any
+ vi.spyOn(ApiConnections.prototype, 'list').mockReturnValue([])
+ const save = vi.spyOn(ApiConnections.prototype, 'save').mockReturnValue({ id: 'fixture' } as any)
+ const remove = vi.spyOn(ApiConnections.prototype, 'remove').mockImplementation(() => {})
+ vi.spyOn(internals.e2ee, 'sessionRole').mockReturnValue('web')
+ vi.spyOn(internals.e2ee, 'hasSession').mockReturnValue(true)
+ vi.spyOn(internals.e2ee, 'unwrapDown').mockReturnValue({ type: 'api_connections', payload: {
+ requestId: 'owner', action, id: 'fixture', connection: { provider: 'fal', apiKey: 'fixture-private-key' },
+ } })
+ const sealedReply = vi.spyOn(internals.e2ee, 'wrapRpcReply').mockReturnValue({ type: 'api_connections_result', payload: { __e2e: 'sealed' } })
+ await internals.dispatchDown({ type: 'api_connections', payload: { __e2e: { v: 1, k: 'p', n: 1, ct: 'fixture' } } }, 'remote')
+ expect(sealedReply).toHaveBeenCalledWith('remote', 'api_connections_result', 'owner', expect.objectContaining({ connections: [] }))
+ expect(JSON.stringify(sealedReply.mock.calls)).not.toContain('fixture-private-key')
+ if (action === 'save') expect(save).toHaveBeenCalledWith({ provider: 'fal', apiKey: 'fixture-private-key' })
+ if (action === 'remove') expect(remove).toHaveBeenCalledWith('fixture')
+ await socket.stop()
+ })
+
+ it('requires a sealed owner web session for remote orchestration', async () => {
+ const socket = new BackendSocket('fixture'), internals = socket as any
+ const list = vi.fn().mockReturnValue([])
+ const service = vi.spyOn(internals, 'orchestration').mockReturnValue({ list })
+ vi.spyOn(internals.e2ee, 'hasSession').mockReturnValue(true)
+ const role = vi.spyOn(internals.e2ee, 'sessionRole').mockReturnValue('web')
+ const clear = { type: 'orchestrator', payload: { requestId: 'owner', action: 'list' } }
+ vi.spyOn(internals.e2ee, 'unwrapDown').mockReturnValue(clear)
+ const sealedReply = vi.spyOn(internals.e2ee, 'wrapRpcReply').mockReturnValue({ type: 'orchestrator_result', payload: { __e2e: 'sealed' } })
+ await internals.dispatchDown(clear, 'remote')
+ expect(service).not.toHaveBeenCalled()
+ const sealed = { type: 'orchestrator', payload: { __e2e: { v: 1, k: 'p', n: 1, ct: 'fixture' } } }
+ role.mockReturnValue('device')
+ await internals.dispatchDown(sealed, 'remote')
+ expect(service).not.toHaveBeenCalled()
+ role.mockReturnValue('web')
+ await internals.dispatchDown(sealed, 'remote')
+ await vi.waitFor(() => expect(list).toHaveBeenCalledOnce())
+ expect(sealedReply).toHaveBeenCalledWith('remote', 'orchestrator_result', 'owner', { projects: [] })
await socket.stop()
})
})
diff --git a/cli/src/lib/e2ee/applicationFrames.ts b/cli/src/lib/e2ee/applicationFrames.ts
index a9bd0b167..5e8c134d4 100644
--- a/cli/src/lib/e2ee/applicationFrames.ts
+++ b/cli/src/lib/e2ee/applicationFrames.ts
@@ -10,11 +10,12 @@
import { ENCRYPTED_RPC_RESULT_TYPES, isEncryptedDownType } from './core.js'
import { SHARE_REQUEST_TYPES, SHARE_RESULT_TYPES } from '../../sharing/protocol.js'
import { VIEWER_DOWN_TYPES } from '../viewerWire.js'
+export const OWNER_COMMAND_TYPES = new Set(['command_bar', 'route_task', 'route_send'])
const FLEET_REQUESTS = new Set(['grid_fleet_capabilities', 'grid_fleet_run', 'grid_fleet_cancel',
'grid_fleet_models_list', 'grid_fleet_model_download', 'grid_fleet_model_start', 'grid_fleet_model_stop'])
const FLEET_RESULTS = new Set([...FLEET_REQUESTS].map(type => `${type}_result`))
-const MACHINE_REQUESTS = new Set(['git_project_info', 'git_pull_request', 'machine_resources', 'api_connections'])
+const MACHINE_REQUESTS = new Set(['git_project_info', 'git_pull_request', 'machine_resources', 'api_connections', 'phone_pair', 'viewer_surface', 'orchestrator', ...OWNER_COMMAND_TYPES])
const MACHINE_RESULTS = new Set([...MACHINE_REQUESTS].map(type => `${type}_result`))
export const encryptDownFrame = (type: string): boolean =>
isEncryptedDownType(type) || MACHINE_REQUESTS.has(type) || FLEET_REQUESTS.has(type) || SHARE_REQUEST_TYPES.has(type) || VIEWER_DOWN_TYPES.has(type)
diff --git a/cli/src/lib/e2ee/manager.test.ts b/cli/src/lib/e2ee/manager.test.ts
index bb3c6350d..47c49355a 100644
--- a/cli/src/lib/e2ee/manager.test.ts
+++ b/cli/src/lib/e2ee/manager.test.ts
@@ -38,6 +38,7 @@ beforeEach(() => {
/** A minimal web peer that runs the CPace responder + session handshake against the manager. */
class WebPeer {
+ constructor(readonly role: 'web' | 'device' = 'web') {}
identity = C.newIdentity()
session?: { c2s: Uint8Array; s2c: Uint8Array; groupKey: Uint8Array; epoch: string; myEph: import('./core.js').Ephemeral }
adapterPub?: Uint8Array
@@ -45,7 +46,7 @@ class WebPeer {
intent(pairId: Uint8Array): Frame {
this.pr = { code: '', pairId, pairIdB64: C.b64e(pairId) }
- return { type: 'e2e_pair_intent', payload: { requestId: 'r1', pairId: this.pr.pairIdB64, label: 'Chrome · macOS' } }
+ return { type: 'e2e_pair_intent', payload: { requestId: 'r1', pairId: this.pr.pairIdB64, label: 'Chrome · macOS', role: this.role } }
}
setCode(code: string): void { if (this.pr) this.pr.code = code }
@@ -53,7 +54,7 @@ class WebPeer {
onPake(frame: Frame): Frame | null {
const p = frame.payload as Record<string, unknown>
const pr = this.pr!
- const ci = C.pairContext(AGENT)
+ const ci = C.pairContext(AGENT, this.role)
const round = Number(p.round)
if (round === 1) {
const g = C.cpaceGenerator(pr.code, pr.pairId, ci)
@@ -168,8 +169,8 @@ function machine(extra: Partial<ConstructorParameters<typeof makeManager>[0]> =
}
/** Drive a full pairing + session for one connection; returns the established web peer. */
-async function fullPair(h: ReturnType<typeof machine>, conn: string): Promise<WebPeer> {
- const web = new WebPeer()
+async function fullPair(h: ReturnType<typeof machine>, conn: string, role: 'web' | 'device' = 'web'): Promise<WebPeer> {
+ const web = new WebPeer(role)
h.mgr.handleFrame(conn, web.intent(C.newPairId()))
const code = C.newPairCode(); web.setCode(code)
const pairP = h.mgr.onPair(code)
@@ -182,6 +183,57 @@ async function fullPair(h: ReturnType<typeof machine>, conn: string): Promise<We
}
describe('E2eeManager pairing', () => {
+ it('a trusted browser pairs a phone and receives an encrypted result', async () => {
+ const h = machine()
+ const browser = await fullPair(h, 'browser')
+ const phone = new WebPeer()
+ const code = 'ABCDEFGHJKMNPQRS'
+ h.mgr.handleFrame('phone', phone.intent(C.newPairId()))
+ phone.setCode(code)
+ const pairing = h.mgr.pairPhoneFromTrustedWeb('browser', { requestId: 'phone-link', code })
+ h.mgr.handleFrame('phone', phone.onPake(h.lastFor('phone', 'e2e_pake')!)!)
+ h.mgr.handleFrame('phone', phone.onPake(h.lastFor('phone', 'e2e_pake')!)!)
+ await pairing
+ const answer = h.lastFor('browser', 'phone_pair_result')!
+ expect(C.isWrapped(answer.payload)).toBe(true)
+ const opened = C.unwrapPayload(browser.session!.s2c,
+ (answer.payload as import('./core.js').WrappedPayload).__e2e, 'phone_pair_result', undefined)
+ expect(opened).toMatchObject({ requestId: 'phone-link', ok: true, label: 'Chrome · macOS' })
+ h.mgr.handleFrame('phone', phone.hello())
+ phone.onWelcome(h.lastFor('phone', 'e2e_welcome')!, phone.adapterPub!)
+ expect(h.mgr.hasSession('phone')).toBe(true)
+ })
+
+ it('unpaired visitors and hardware devices cannot authorize a phone', async () => {
+ const h = machine()
+ await fullPair(h, 'device', 'device')
+ const phone = new WebPeer()
+ h.mgr.handleFrame('phone', phone.intent(C.newPairId()))
+ const before = h.sent.length
+ for (const conn of ['anonymous', 'device']) {
+ await h.mgr.pairPhoneFromTrustedWeb(conn, { requestId: 'r', code: 'ABCDEFGHJKMNPQRS' })
+ }
+ expect(h.sent).toHaveLength(before)
+ expect(h.mgr.hasSession('phone')).toBe(false)
+ h.mgr.dropSession('phone')
+ })
+
+ it('phone authorization does not arm hardware intents or accept malformed codes', async () => {
+ const h = machine()
+ const browser = await fullPair(h, 'browser')
+ const read = () => C.unwrapPayload(browser.session!.s2c,
+ (h.lastFor('browser', 'phone_pair_result')!.payload as import('./core.js').WrappedPayload).__e2e, 'phone_pair_result', undefined)
+ await h.mgr.pairPhoneFromTrustedWeb('browser', { code: 'short' })
+ expect(read()).toMatchObject({ error: 'BAD_CODE' })
+ await h.mgr.pairPhoneFromTrustedWeb('browser', { code: 'ABCDEFGHJKMNPQRS' })
+ expect(read()).toMatchObject({ error: 'NO_INTENT' })
+ h.mgr.handleFrame('device', new WebPeer('device').intent(C.newPairId()))
+ await h.mgr.pairPhoneFromTrustedWeb('browser', { code: 'ABCDEFGHJKMNPQRS' })
+ expect(read()).toMatchObject({ error: 'NO_INTENT' })
+ expect(h.lastFor('device', 'e2e_pake')).toBeUndefined()
+ h.mgr.dropSession('device')
+ })
+
it('pairs nobody from an e2e_setup_claim — browser setup links were removed with the web client', () => {
const { mgr, sent } = machine()
const web = new WebPeer()
diff --git a/cli/src/lib/e2ee/manager.ts b/cli/src/lib/e2ee/manager.ts
index bb384f648..6fe561a81 100644
--- a/cli/src/lib/e2ee/manager.ts
+++ b/cli/src/lib/e2ee/manager.ts
@@ -495,6 +495,25 @@ export class E2eeManager {
if (reply) this.deps.sendTo(connId, reply)
}
+ /** A linked browser can arm phone pairing on its own machine, just as the local desktop can.
+ * The code travels only inside the existing encrypted session. Possessing an account token or
+ * an observer link is not enough; the caller must already hold a paired web identity. */
+ async pairPhoneFromTrustedWeb(connId: string, p: Record<string, unknown>): Promise<void> {
+ if (this.sessions.get(connId)?.role !== 'web') return
+ const reply = (payload: Record<string, unknown>): void => {
+ const frame = this.wrapRpcReply(connId, 'phone_pair_result', p.requestId, payload)
+ if (frame) this.deps.sendTo(connId, frame)
+ }
+ const code = C.normalizeCode(typeof p.code === 'string' ? p.code : '')
+ if (!/^[A-Z2-9]{16}$/.test(code)) { reply({ error: 'BAD_CODE' }); return }
+ // A phone uses the web role. Never consume a hardware pairing or the caller's own intent.
+ if (!this.slot || this.slot.role !== 'web' || this.slot.connId === connId) {
+ reply({ error: 'NO_INTENT' }); return
+ }
+ const result = await this.onPair(code)
+ reply(result.ok ? { ok: true, label: result.label, fingerprint: result.fingerprint } : { error: result.error })
+ }
+
/** Called by the hook server when the user runs `harness pair <code>`. Resolves when done/failed. */
onPair(code: string): Promise<PairResult> {
return new Promise<PairResult>((resolve) => {
diff --git a/cli/src/lib/interactiveViewer.spec.ts b/cli/src/lib/interactiveViewer.spec.ts
new file mode 100644
index 000000000..da0d5a3fc
--- /dev/null
+++ b/cli/src/lib/interactiveViewer.spec.ts
@@ -0,0 +1,117 @@
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
+import { InteractiveViewerCapture, InteractiveViewers, surfaceFrame } from './interactiveViewer.js'
+
+const frame = (extra = {}) => ({ surfaceId: 'one', agentId: 'agent', op: 'frame', width: 800, height: 600, dark: true, ...extra })
+const pointer = { type: 'pointer', event: 'mousePressed', x: .5, y: .25, button: 'left', buttons: 1, clickCount: 1, modifiers: 0 }
+
+describe('interactive viewer input boundary', () => {
+ it('maps only ordinary pointer, keyboard, and text input to CDP', () => {
+ const parsed = surfaceFrame(frame({ events: [pointer,
+ { type: 'key', event: 'keyDown', key: 'Enter', code: 'Enter', keyCode: 13, modifiers: 0 },
+ { type: 'text', text: 'hello 世界' },
+ { ...pointer, event: 'mouseWheel', deltaX: 0, deltaY: 80 },
+ ] }))!
+ expect(parsed.commands).toEqual([
+ { method: 'Input.dispatchMouseEvent', params: { type: 'mousePressed', x: 399.5, y: 149.75, button: 'left', buttons: 1, clickCount: 1, modifiers: 0 } },
+ { method: 'Input.dispatchKeyEvent', params: { type: 'keyDown', key: 'Enter', code: 'Enter', windowsVirtualKeyCode: 13, modifiers: 0, text: '\r' } },
+ { method: 'Input.insertText', params: { text: 'hello 世界' } },
+ { method: 'Input.dispatchMouseEvent', params: { type: 'mouseWheel', x: 399.5, y: 149.75, button: 'left', buttons: 1, clickCount: 1, modifiers: 0, deltaX: 0, deltaY: 80 } },
+ ])
+ })
+ it.each([
+ { width: Infinity }, { height: 9000 }, { dark: 'yes' }, { reload: 'yes' }, { events: {} },
+ { events: Array(65).fill(pointer) }, { events: [null] }, { events: [{ type: 'Runtime.evaluate', expression: 'secrets' }] },
+ { events: [{ ...pointer, x: NaN }] }, { events: [{ ...pointer, x: -1 }] },
+ { events: [{ ...pointer, event: 'mouseWheel', deltaX: 0, deltaY: 9000 }] },
+ { events: [{ type: 'text', text: 'a'.repeat(4097) }] },
+ { events: [{ type: 'key', event: 'keyDown', key: 'x', code: 'KeyX', keyCode: 88, modifiers: 999 }] },
+ ])('rejects malformed or excessive input without executing it: %j', extra => {
+ expect(surfaceFrame(frame(extra))).toBeNull()
+ })
+ it('applies viewport/theme changes once, with input before the image', async () => {
+ const capture = new InteractiveViewerCapture()
+ const command = vi.spyOn(capture as any, 'call').mockResolvedValue({})
+ const screenshot = vi.spyOn(capture, 'capture').mockResolvedValue('jpeg')
+ await capture.frame(surfaceFrame(frame())!)
+ await capture.frame(surfaceFrame(frame({ events: [{ type: 'text', text: 'hi' }] }))!)
+ expect(command.mock.calls.map(call => call[0])).toEqual([
+ 'Emulation.setDeviceMetricsOverride', 'Emulation.setEmulatedMedia', 'Input.insertText',
+ ])
+ expect(screenshot).toHaveBeenCalledTimes(2)
+ await capture.frame(surfaceFrame(frame({ width: 900, reload: true }))!)
+ expect(command.mock.calls.slice(-3).map(call => call[0])).toEqual([
+ 'Emulation.setDeviceMetricsOverride', 'Page.reload', 'Emulation.setEmulatedMedia',
+ ])
+ })
+})
+
+describe('owner viewer sessions', () => {
+ let viewers: InteractiveViewers, target: string | null
+ let captures: Array<{ start: ReturnType<typeof vi.fn>; frame: ReturnType<typeof vi.fn>; stop: ReturnType<typeof vi.fn> }>
+ beforeEach(() => {
+ vi.useFakeTimers(); captures = []; target = 'http://127.0.0.1:8000/'
+ viewers = new InteractiveViewers(() => target, () => {
+ const capture = { start: vi.fn(async () => {}), frame: vi.fn(async () => 'jpeg'), stop: vi.fn(async () => {}) }
+ captures.push(capture)
+ return capture as unknown as InteractiveViewerCapture
+ })
+ })
+ afterEach(() => { viewers.closeAll(); vi.useRealTimers() })
+
+ it('reuses a connection-owned renderer and closes only the caller’s session', async () => {
+ expect(await viewers.request('one', frame())).toMatchObject({ data: 'jpeg', width: 800 })
+ await viewers.request('one', frame())
+ expect(captures[0].start).toHaveBeenCalledTimes(1)
+ expect(captures[0].start).toHaveBeenCalledWith(target)
+ await viewers.request('two', frame())
+ await viewers.request('one', frame({ op: 'close' }))
+ expect(captures[0].stop).toHaveBeenCalledOnce()
+ expect(captures[1].stop).not.toHaveBeenCalled()
+ viewers.closeConnection('two')
+ expect(captures[1].stop).toHaveBeenCalledOnce()
+ })
+ it('never accepts an arbitrary URL, another agent, or an invalid action', async () => {
+ expect(await viewers.request('one', frame({ surfaceId: '../bad' }))).toHaveProperty('error', 'INVALID_VIEWER_REQUEST')
+ await viewers.request('one', frame({ url: 'http://attacker.invalid' }))
+ expect(captures[0].start).toHaveBeenCalledWith(target)
+ expect(await viewers.request('one', frame({ agentId: 'other' }))).toHaveProperty('error', 'INVALID_VIEWER_REQUEST')
+ expect(await viewers.request('one', frame({ op: 'evaluate' }))).toHaveProperty('error', 'INVALID_VIEWER_REQUEST')
+ expect(await viewers.request('one', frame({ width: -1 }))).toHaveProperty('error', 'INVALID_VIEWER_REQUEST')
+ target = 'https://external.invalid'
+ expect(await viewers.request('one', frame())).toHaveProperty('error', 'VIEWER_UNAVAILABLE')
+ expect(captures[0].stop).toHaveBeenCalledOnce()
+ })
+ it('bounds per-client renderers and expires abandoned surfaces', async () => {
+ for (let i = 0; i < 4; i++) await viewers.request('one', frame({ surfaceId: `s-${i}` }))
+ expect(await viewers.request('one', frame({ surfaceId: 'fifth' }))).toHaveProperty('error', 'VIEWER_LIMIT')
+ for (let i = 0; i < 4; i++) await viewers.request('two', frame({ surfaceId: `s-${i}` }))
+ expect(await viewers.request('three', frame())).toHaveProperty('error', 'VIEWER_LIMIT')
+ await vi.advanceTimersByTimeAsync(30_000)
+ expect(captures.every(capture => capture.stop.mock.calls.length === 1)).toBe(true)
+ })
+ it('discards an old target before another process can reuse its port', async () => {
+ await viewers.request('one', frame())
+ target = 'http://127.0.0.1:9000/'
+ viewers.refresh('agent')
+ expect(captures[0].stop).toHaveBeenCalledOnce()
+ await viewers.request('one', frame())
+ expect(captures[1].start).toHaveBeenCalledWith(target)
+ })
+ it('refuses overlapping captures and never returns a frame after disconnect', async () => {
+ await viewers.request('one', frame())
+ let finish!: (value: string) => void
+ captures[0].frame.mockImplementationOnce(() => new Promise<string>(resolve => { finish = resolve }))
+ const pending = viewers.request('one', frame())
+ expect(await viewers.request('one', frame())).toHaveProperty('error', 'VIEWER_BUSY')
+ viewers.closeConnection('one'); finish('old frame')
+ expect(await pending).toHaveProperty('error', 'VIEWER_CLOSED')
+ })
+ it('releases a failed renderer and reports a useful error', async () => {
+ await viewers.request('one', frame())
+ captures[0].frame.mockRejectedValueOnce(new Error('Chrome stopped'))
+ expect(await viewers.request('one', frame())).toMatchObject({ error: 'VIEWER_UNAVAILABLE', detail: 'Chrome stopped' })
+ expect(captures[0].stop).toHaveBeenCalledOnce()
+ await viewers.request('one', frame())
+ expect(captures).toHaveLength(2)
+ })
+})
diff --git a/cli/src/lib/interactiveViewer.ts b/cli/src/lib/interactiveViewer.ts
new file mode 100644
index 000000000..9cca51c87
--- /dev/null
+++ b/cli/src/lib/interactiveViewer.ts
@@ -0,0 +1,141 @@
+import { ViewerCapture, viewerTarget } from '../sharing/viewer.js'
+
+type Payload = Record<string, unknown>
+type Command = { method: string; params: Payload }
+type Frame = { width: number; height: number; dark: boolean; reload: boolean; commands: Command[] }
+const integer = (value: unknown, min: number, max: number): value is number =>
+ Number.isInteger(value) && Number(value) >= min && Number(value) <= max
+const coordinate = (value: unknown): value is number =>
+ typeof value === 'number' && Number.isFinite(value) && value >= 0 && value <= 1
+
+/** Deliberately not a general CDP bridge: a browser can only send ordinary viewer input. */
+export function surfaceFrame(payload: Payload): Frame | null {
+ if (!integer(payload.width, 160, 1920) || !integer(payload.height, 120, 1200)
+ || typeof payload.dark !== 'boolean' || (payload.reload !== undefined && typeof payload.reload !== 'boolean')) return null
+ const events = payload.events ?? []
+ if (!Array.isArray(events) || events.length > 64) return null
+ const commands: Command[] = []
+ for (const event of events) {
+ if (!event || typeof event !== 'object' || Array.isArray(event)) return null
+ if (event.type === 'text') {
+ if (typeof event.text !== 'string' || !event.text.length || event.text.length > 4096) return null
+ commands.push({ method: 'Input.insertText', params: { text: event.text } })
+ } else if (event.type === 'key') {
+ if (!['keyDown', 'keyUp'].includes(event.event) || typeof event.key !== 'string'
+ || event.key.length < 1 || event.key.length > 64 || typeof event.code !== 'string' || event.code.length > 64
+ || !integer(event.keyCode, 0, 255) || !integer(event.modifiers, 0, 15)) return null
+ commands.push({ method: 'Input.dispatchKeyEvent', params: {
+ type: event.event, key: event.key, code: event.code, windowsVirtualKeyCode: event.keyCode,
+ modifiers: event.modifiers, ...(event.event === 'keyDown' && event.key === 'Enter' ? { text: '\r' } : {}),
+ } })
+ } else if (event.type === 'pointer') {
+ if (!['mousePressed', 'mouseReleased', 'mouseMoved', 'mouseWheel'].includes(event.event)
+ || !coordinate(event.x) || !coordinate(event.y) || !integer(event.buttons, 0, 7)
+ || !['none', 'left', 'right', 'middle'].includes(event.button) || !integer(event.modifiers, 0, 15)
+ || !integer(event.clickCount, 0, 2)) return null
+ const params: Payload = { type: event.event, x: event.x * (payload.width - 1), y: event.y * (payload.height - 1),
+ buttons: event.buttons, button: event.button, modifiers: event.modifiers, clickCount: event.clickCount }
+ if (event.event === 'mouseWheel') {
+ if (typeof event.deltaX !== 'number' || !Number.isFinite(event.deltaX) || Math.abs(event.deltaX) > 2000
+ || typeof event.deltaY !== 'number' || !Number.isFinite(event.deltaY) || Math.abs(event.deltaY) > 2000) return null
+ params.deltaX = event.deltaX; params.deltaY = event.deltaY
+ }
+ commands.push({ method: 'Input.dispatchMouseEvent', params })
+ } else return null
+ }
+ return { width: payload.width, height: payload.height, dark: payload.dark, reload: payload.reload === true, commands }
+}
+
+/** Separate from the read-only observer renderer. No cookies, profile, or app credentials. */
+export class InteractiveViewerCapture extends ViewerCapture {
+ private geometry = ''
+ private dark: boolean | null = null
+ async frame(frame: Frame): Promise<string> {
+ const geometry = `${frame.width}x${frame.height}`
+ if (geometry !== this.geometry) {
+ await this.call('Emulation.setDeviceMetricsOverride', {
+ width: frame.width, height: frame.height, deviceScaleFactor: 1, mobile: false,
+ })
+ this.geometry = geometry
+ }
+ if (frame.reload) await this.call('Page.reload')
+ if (frame.dark !== this.dark || frame.reload) {
+ await this.call('Emulation.setEmulatedMedia', { features: [{ name: 'prefers-color-scheme', value: frame.dark ? 'dark' : 'light' }] })
+ this.dark = frame.dark
+ }
+ for (const command of frame.commands) await this.call(command.method, command.params)
+ return this.capture()
+ }
+}
+
+interface Surface {
+ connId: string
+ agentId: string
+ target: string
+ capture: InteractiveViewerCapture
+ started: boolean
+ busy: boolean
+ timer: NodeJS.Timeout
+}
+
+/** Owner-only, bounded renderers. Frames are pulled, so a slow client cannot accumulate JPEGs. */
+export class InteractiveViewers {
+ private readonly surfaces = new Map<string, Surface>()
+ constructor(private readonly target: (agentId: string) => string | null,
+ private readonly create = () => new InteractiveViewerCapture()) {}
+
+ async request(connId: string, payload: Payload): Promise<Payload> {
+ const { surfaceId, agentId, op } = payload
+ if (typeof surfaceId !== 'string' || !/^[a-zA-Z0-9-]{1,80}$/.test(surfaceId)
+ || typeof agentId !== 'string' || !agentId.length || agentId.length > 160
+ || !['frame', 'close'].includes(String(op))) return { error: 'INVALID_VIEWER_REQUEST' }
+ const key = `${connId}/${surfaceId}`
+ let surface = this.surfaces.get(key)
+ if (surface && surface.agentId !== agentId) return { error: 'INVALID_VIEWER_REQUEST' }
+ if (op === 'close') { this.remove(key); return { closed: true } }
+ const frame = surfaceFrame(payload)
+ if (!frame) return { error: 'INVALID_VIEWER_REQUEST' }
+ const target = viewerTarget(this.target(agentId))
+ if (!target) { this.remove(key); return { error: 'VIEWER_UNAVAILABLE', detail: 'This harness has no live viewer yet.' } }
+ if (surface && surface.target !== target) { this.remove(key); surface = undefined }
+ if (!surface) {
+ if (this.surfaces.size >= 8 || [...this.surfaces.values()].filter(s => s.connId === connId).length >= 4) {
+ return { error: 'VIEWER_LIMIT', detail: 'Close another viewer to open this one.' }
+ }
+ surface = { connId, agentId, target, capture: this.create(), started: false, busy: false,
+ timer: setTimeout(() => this.remove(key), 30_000) }
+ surface.timer.unref()
+ this.surfaces.set(key, surface)
+ }
+ if (surface.busy) return { error: 'VIEWER_BUSY' }
+ surface.timer.refresh()
+ surface.busy = true
+ try {
+ if (!surface.started) { await surface.capture.start(target); surface.started = true }
+ if (this.surfaces.get(key) !== surface) return { error: 'VIEWER_CLOSED' }
+ const data = await surface.capture.frame(frame)
+ if (this.surfaces.get(key) !== surface) return { error: 'VIEWER_CLOSED' }
+ return { data, mime: 'image/jpeg', width: frame.width, height: frame.height }
+ } catch (error) {
+ if (this.surfaces.get(key) === surface) this.remove(key)
+ return { error: 'VIEWER_UNAVAILABLE', detail: error instanceof Error ? error.message : 'The viewer could not start.' }
+ } finally { surface.busy = false }
+ }
+
+ private remove(key: string): void {
+ const surface = this.surfaces.get(key)
+ if (!surface) return
+ this.surfaces.delete(key); clearTimeout(surface.timer)
+ void surface.capture.stop().catch(() => {})
+ }
+ refresh(agentId: string): void {
+ const target = viewerTarget(this.target(agentId))
+ for (const [key, surface] of this.surfaces) {
+ if (surface.agentId === agentId && surface.target !== target) this.remove(key)
+ }
+ }
+ closeConnection(connId: string): void {
+ for (const [key, surface] of this.surfaces) if (surface.connId === connId) this.remove(key)
+ }
+ closeAll(): void { for (const key of this.surfaces.keys()) this.remove(key) }
+}
diff --git a/cli/src/lib/ownerCommands.spec.ts b/cli/src/lib/ownerCommands.spec.ts
new file mode 100644
index 000000000..6b6b443e6
--- /dev/null
+++ b/cli/src/lib/ownerCommands.spec.ts
@@ -0,0 +1,45 @@
+import { describe, expect, it, vi } from 'vitest'
+import { OwnerCommands } from './ownerCommands.js'
+import { CommandBarError } from './commandBar.js'
+
+describe('owner browser commands', () => {
+ it('uses the desktop decision service without executing its choice', async () => {
+ const decide = vi.fn().mockResolvedValue({ selectedId: 'send:a', autoExecute: false })
+ const commands = new OwnerCommands({ decide }), send = vi.fn()
+ commands.onRouteSend = send
+ const request = { prompt: 'Send a task', candidates: [] }
+ expect(await commands.request('one', 'command_bar', { request })).toEqual({ selectedId: 'send:a', autoExecute: false })
+ expect(decide).toHaveBeenCalledWith(request, expect.any(AbortSignal))
+ expect(send).not.toHaveBeenCalled()
+ })
+
+ it('validates task delivery and preserves the daemon refusal', async () => {
+ const commands = new OwnerCommands({ decide: vi.fn() })
+ const send = vi.fn().mockReturnValue({ ok: false, machine: 'fixture', reason: 'offline' })
+ commands.onRouteSend = send
+ for (const payload of [{}, { agentId: 'a', text: '' }, { agentId: 'a', text: 'x'.repeat(16_001) }]) {
+ expect(await commands.request('one', 'route_send', payload)).toEqual({ error: 'INVALID_REQUEST' })
+ }
+ expect(send).not.toHaveBeenCalled()
+ expect(await commands.request('one', 'route_send', { agentId: 'a', text: 'hello' })).toEqual({ ok: false, machine: 'fixture', reason: 'offline' })
+ expect(send).toHaveBeenCalledWith('a', 'hello')
+ expect(await commands.request('one', 'route_task', { text: 'hello' })).toEqual({ error: 'UNSUPPORTED' })
+ })
+
+ it('bounds pending decisions and aborts only the disconnecting connection', async () => {
+ const decide = vi.fn().mockImplementation((_raw, signal: AbortSignal) => new Promise((_, reject) => {
+ signal.addEventListener('abort', () => reject(new CommandBarError(499, 'CANCELLED', 'Command cancelled.')))
+ }))
+ const commands = new OwnerCommands({ decide })
+ const one = commands.request('one', 'command_bar', { request: {} })
+ const two = commands.request('one', 'command_bar', { request: {} })
+ const other = commands.request('other', 'command_bar', { request: {} })
+ expect(await commands.request('one', 'command_bar', { request: {} })).toEqual({ error: 'BUSY' })
+ commands.closeConnection('one')
+ expect(await one).toMatchObject({ error: 'CANCELLED' })
+ expect(await two).toMatchObject({ error: 'CANCELLED' })
+ expect(decide.mock.calls[2][1].aborted).toBe(false)
+ commands.closeAll()
+ expect(await other).toMatchObject({ error: 'CANCELLED' })
+ })
+})
diff --git a/cli/src/lib/ownerCommands.ts b/cli/src/lib/ownerCommands.ts
new file mode 100644
index 000000000..9640016ed
--- /dev/null
+++ b/cli/src/lib/ownerCommands.ts
@@ -0,0 +1,42 @@
+import { CommandBarError, commandBarService, type CommandBarService } from './commandBar.js'
+import type { LocalWsServerOptions } from '../localWsServer.js'
+import { OWNER_COMMAND_TYPES } from './e2ee/applicationFrames.js'
+export { OWNER_COMMAND_TYPES } from './e2ee/applicationFrames.js'
+
+/** The same decisions and task delivery as the desktop, behind the paired-owner boundary.
+ * A decision never executes an action. The client still reviews and commits the selected action. */
+export class OwnerCommands {
+ onRouteTask?: LocalWsServerOptions['onRouteTask']
+ onRouteSend?: LocalWsServerOptions['onRouteSend']
+ private readonly pending = new Map<AbortController, string>()
+ constructor(private readonly commands: Pick<CommandBarService, 'decide'> = commandBarService) {}
+
+ async request(connId: string, type: string, payload: Record<string, unknown>): Promise<Record<string, unknown>> {
+ if (!OWNER_COMMAND_TYPES.has(type)) return { error: 'UNSUPPORTED' }
+ if (this.pending.size >= 8 || [...this.pending.values()].filter(id => id === connId).length >= 2) return { error: 'BUSY' }
+ const abort = new AbortController()
+ this.pending.set(abort, connId)
+ try {
+ if (type === 'command_bar') return await this.commands.decide(payload.request, abort.signal)
+ const text = typeof payload.text === 'string' ? payload.text.trim() : ''
+ if (!text || text.length > 16_000) return { error: 'INVALID_REQUEST' }
+ if (type === 'route_task') {
+ if (!this.onRouteTask) return { error: 'UNSUPPORTED' }
+ return { ...await this.onRouteTask(text) }
+ }
+ const agentId = payload.agentId
+ if (typeof agentId !== 'string' || !agentId.length || agentId.length > 160) return { error: 'INVALID_REQUEST' }
+ if (!this.onRouteSend) return { error: 'UNSUPPORTED' }
+ return this.onRouteSend(agentId, text)
+ } catch (error) {
+ return error instanceof CommandBarError
+ ? { error: error.code, detail: error.message }
+ : { error: 'COMMAND_UNAVAILABLE', detail: 'This machine could not complete the command. Try again.' }
+ } finally { this.pending.delete(abort) }
+ }
+
+ closeConnection(connId: string): void {
+ for (const [abort, id] of this.pending) if (id === connId) abort.abort()
+ }
+ closeAll(): void { for (const abort of this.pending.keys()) abort.abort() }
+}
diff --git a/cli/src/sharing/viewer.ts b/cli/src/sharing/viewer.ts
index 446b6b986..2cb814c70 100644
--- a/cli/src/sharing/viewer.ts
+++ b/cli/src/sharing/viewer.ts
@@ -103,7 +103,7 @@ export class ViewerCapture {
this.loaded = null
}
private loaded: (() => void) | null = null
- private call(method: string, params: Payload = {}): Promise<Payload> {
+ protected call(method: string, params: Payload = {}): Promise<Payload> {
return new Promise((resolve, reject) => {
if (this.closed || this.ws?.readyState !== WebSocket.OPEN) { reject(new Error('The viewer renderer disconnected.')); return }
const id = ++this.id
diff --git a/desktop/README.md b/desktop/README.md
index 6dd793772..1d473ec27 100644
--- a/desktop/README.md
+++ b/desktop/README.md
@@ -3,7 +3,8 @@
Harness Desktop is the native Flutter client for browsing Harness machines and
interacting with their terminal-backed agents. **macOS is the primary supported and tested
experience.** Linux builds exist, with feature parity still in progress; Windows support is
-planned and its runner is unexercised. Embedded harness viewers currently require macOS.
+planned and its runner is unexercised. Native embedded harness viewers require macOS;
+the browser renders managed viewers on their connected machine.
The browser target uses this same Flutter package and `lib/main.dart`: workspace,
tabs, pickers, settings, state, and the patched xterm renderer are shared. Browser
@@ -98,10 +99,37 @@ For a local production build, run `bash scripts/build-web-release.sh X.Y.Z`.
Control keys keep their usual behavior. The shared shortcut sheet and welcome
hints show the active bindings.
- Agent processes and files stay on their host machines. Local provisioning,
- desktop updates, device pairing, local usage ledgers, keyboard config files,
- native file previews/image clipboard, and embedded native webviews remain
- desktop capabilities. Remote terminals and streamed image viewers reuse the
- shared UI. Closing the browser does not stop a running agent.
+ desktop updates, hardware firmware, local usage ledgers, keyboard config files,
+ system notifications, and native image clipboard remain desktop capabilities. Closing the browser
+ does not stop a running agent.
+- **Work from your phone** on the welcome page shows the same QR setup as
+ desktop. In a browser it pairs the phone with the selected linked computer,
+ over an encrypted owner connection. The computer name stays visible and fixed
+ while the QR is open. A browser without a linked computer offers the machine
+ picker first.
+- API connections, model controls and orchestrator projects run on a linked
+ computer. Editors keep their destination while open; reopening model controls
+ selects the current computer. API keys are saved on that computer, not in
+ browser preferences. The command bar uses the same daemon decision service
+ and requires the same provider configuration; local navigation still works
+ without it. Sending a task remains a separate confirmed action.
+- Managed viewer panes accept mouse, keyboard and text input through an
+ encrypted owner connection. Their isolated Chromium renderer runs on the
+ agent machine and must be installed there. Shared-link viewers remain read
+ only. Streams are bounded to four interactive surfaces per connection/eight
+ per daemon, with an idle timeout; native dialogs, browser downloads, audio and
+ OS clipboard bridging are not provided by this stream. These capabilities
+ require an updated daemon; older hosts receive update guidance.
+
+The disposable full-stack fixture also accepts
+`HARNESS_WORKSPACE_BROWSER_CHECK=$PWD/desktop/scripts/check-workspace.cjs` and
+`HARNESS_SHARE_BROWSER_CHECK=$PWD/desktop/scripts/check-sharing.cjs` when running
+`npm run test:sharing-e2e` from `cli/` (set the paths from the repository root).
+It launches fixture accounts and daemons, signs into the browser through an SSO
+stand-in, then uses real password linking, encrypted RPCs, terminal/viewer input,
+API storage and shared links. No real user home or credentials are used. See
+[the readiness record](../docs/plans/2026-09-27-008-web-release-readiness.md)
+for the build origin, service prerequisites and current verification evidence.
Keep product changes in the existing shared screens and state. Add platform
adapters only for browser/native capabilities, following the conditional stores,
diff --git a/desktop/design/images/web-interactive-viewer.png b/desktop/design/images/web-interactive-viewer.png
new file mode 100644
index 000000000..ebbf60c39
Binary files /dev/null and b/desktop/design/images/web-interactive-viewer.png differ
diff --git a/desktop/lib/api/access_token_source.dart b/desktop/lib/api/access_token_source.dart
index f0c9f6dd2..fbf9e7ae6 100644
--- a/desktop/lib/api/access_token_source.dart
+++ b/desktop/lib/api/access_token_source.dart
@@ -4,3 +4,8 @@
abstract interface class AccessTokenSource {
Future<String> accessToken({bool force = false, String? failedToken});
}
+
+/// Distinguishes an expired session from a temporary failure to renew it.
+abstract interface class AccessTokenFailure implements Exception {
+ bool get signedOut;
+}
diff --git a/desktop/lib/api/api_client.dart b/desktop/lib/api/api_client.dart
index 93e6bb36a..9a6767cb6 100644
--- a/desktop/lib/api/api_client.dart
+++ b/desktop/lib/api/api_client.dart
@@ -354,7 +354,11 @@ class ApiException implements Exception {
}
bool isUnauthorizedError(Object error) =>
- error is DioException && error.response?.statusCode == 401 ||
+ error is AccessTokenFailure && error.signedOut ||
+ error is DioException &&
+ (error.response?.statusCode == 401 ||
+ error.error is AccessTokenFailure &&
+ (error.error as AccessTokenFailure).signedOut) ||
error is ApiException && error.status == 401;
/// Unwraps the backend's `{success, data, error}` envelope, which both legs
diff --git a/desktop/lib/api/bearer_auth_interceptor.dart b/desktop/lib/api/bearer_auth_interceptor.dart
index 24f5f4f7f..666eec70a 100644
--- a/desktop/lib/api/bearer_auth_interceptor.dart
+++ b/desktop/lib/api/bearer_auth_interceptor.dart
@@ -48,9 +48,20 @@ class BearerAuthInterceptor extends Interceptor {
);
options.extra[_retriedKey] = true;
handler.resolve(await _dio.fetch<dynamic>(options));
- } catch (_) {
- // The refresh itself failed, so the 401 is the honest answer to hand back.
- handler.next(response);
+ } catch (error) {
+ // The old access token's 401 does not mean a refresh service outage revoked the session.
+ // Preserve the refresh failure so a browser can retry without deleting valid credentials.
+ final expired = error is AccessTokenFailure && error.signedOut;
+ handler.reject(
+ DioException(
+ requestOptions: options,
+ response: expired ? response : null,
+ error: error,
+ type: expired
+ ? DioExceptionType.badResponse
+ : DioExceptionType.connectionError,
+ ),
+ );
}
}
}
diff --git a/desktop/lib/e2ee/envelope.dart b/desktop/lib/e2ee/envelope.dart
index 7cfb758f6..01543dd85 100644
--- a/desktop/lib/e2ee/envelope.dart
+++ b/desktop/lib/e2ee/envelope.dart
@@ -25,6 +25,13 @@ const Set<String> encryptedDownTypes = {
'grid_fleet_run',
'grid_fleet_cancel',
'machine_resources',
+ 'phone_pair',
+ 'viewer_surface',
+ 'api_connections',
+ 'orchestrator',
+ 'command_bar',
+ 'route_task',
+ 'route_send',
'grid_fleet_models_list',
'grid_fleet_model_start',
'grid_fleet_model_download',
diff --git a/desktop/lib/models/api_connections_controller.dart b/desktop/lib/models/api_connections_controller.dart
index f3add784e..2cd328ea6 100644
--- a/desktop/lib/models/api_connections_controller.dart
+++ b/desktop/lib/models/api_connections_controller.dart
@@ -6,7 +6,7 @@ import '../core/models.dart';
import '../state/app_state.dart';
import '../ws/ws_conn.dart';
-/// The desktop retains public metadata only. Keys travel once to the local CLI;
+/// The UI retains public metadata only. Keys travel once to the selected daemon;
/// replies, preferences, and provider rows never contain them.
class ApiConnection {
const ApiConnection(this.data);
@@ -40,12 +40,38 @@ class ApiConnection {
}
class ApiConnectionsController extends ChangeNotifier {
- ApiConnectionsController(this.app) {
+ ApiConnectionsController(this.app, {String? machineId})
+ : _machineId =
+ machineId ??
+ (app.viewer == null
+ ? null
+ : app.ownedActionMachine?.machine.machineId) {
app.addListener(_observe);
- _owner = app.localMachineState;
+ _owner = _target;
_connected = available;
}
final AppNotifier app;
+ String? _machineId;
+ MachineState? get _target {
+ if (_machineId != null) return app.stateOf(_machineId!);
+ if (app.viewer == null) return app.localMachineState;
+ final machine = app.ownedActionMachine;
+ _machineId = machine?.machine.machineId;
+ return machine;
+ }
+
+ String get hostLabel => app.viewer == null && _machineId == null
+ ? 'this computer'
+ : _owner?.machine.displayName ?? 'a connected machine';
+ String get _connectMessage => 'Connect $hostLabel to manage APIs.';
+
+ /// Called between panels, never while an API editor is open.
+ void useMachine(String machineId) {
+ if (_disposed || saving || machineId == _machineId) return;
+ _machineId = machineId;
+ _observe();
+ }
+
MachineState? _owner;
bool _disposed = false;
bool _connected = false;
@@ -54,22 +80,27 @@ class ApiConnectionsController extends ChangeNotifier {
bool loading = false, saving = false, loaded = false;
String? error;
- bool get available => _owner?.connectionStatus == ConnectionStatus.connected;
+ bool get available =>
+ _owner != null &&
+ !_owner!.machine.isShared &&
+ !_owner!.needsLink &&
+ _owner!.nodeOnline != false &&
+ _owner!.connectionStatus == ConnectionStatus.connected;
void _changed() {
if (!_disposed) notifyListeners();
}
void _observe() {
- if (identical(_owner, app.localMachineState)) {
+ if (identical(_owner, _target)) {
if (_connected != available) {
_connected = available;
- if (!_connected) error = 'Connect this computer to manage APIs.';
+ if (!_connected) error = _connectMessage;
_changed();
if (_connected) unawaited(refresh());
}
return;
}
- _owner = app.localMachineState;
+ _owner = _target;
_connected = available;
_revision++;
connections = [];
@@ -88,7 +119,7 @@ class ApiConnectionsController extends ChangeNotifier {
if (_disposed || saving) return false;
final owner = _owner;
if (owner == null || !available) {
- error = 'Connect this computer to manage APIs.';
+ error = _connectMessage;
_changed();
return false;
}
@@ -99,9 +130,7 @@ class ApiConnectionsController extends ChangeNotifier {
error = null;
_changed();
bool current() =>
- !_disposed &&
- revision == _revision &&
- identical(owner, app.localMachineState);
+ !_disposed && revision == _revision && identical(owner, _target);
try {
final answer = await app.apiConnections(owner.machine.machineId, payload);
if (!current()) return false;
@@ -111,7 +140,7 @@ class ApiConnectionsController extends ChangeNotifier {
return false;
}
if (answer['connections'] is! List || answer['presets'] is! List) {
- error = 'Update Harness on this computer to connect APIs.';
+ error = 'Update Harness on $hostLabel to connect APIs.';
return false;
}
List<ApiConnection> rows(String key) => (answer[key] as List)
diff --git a/desktop/lib/models/api_connections_panel.dart b/desktop/lib/models/api_connections_panel.dart
index a2ff83024..cce642d0b 100644
--- a/desktop/lib/models/api_connections_panel.dart
+++ b/desktop/lib/models/api_connections_panel.dart
@@ -377,6 +377,13 @@ class _ApiEditorState extends State<_ApiEditor> {
],
),
const SizedBox(height: 12),
+ if (controller.app.viewer != null) ...[
+ Text(
+ 'Saved on ${controller.hostLabel}',
+ style: AppType.monoMeta(color: AppPalette.textSecondary),
+ ),
+ const SizedBox(height: 12),
+ ],
if (custom || _advanced) ...[
_field(
_name,
@@ -463,7 +470,7 @@ class _ApiEditorState extends State<_ApiEditor> {
children: [
Expanded(
child: Text(
- 'Stored on this computer.',
+ 'Stored on ${controller.hostLabel}.',
style: AppType.monoMeta(
color: AppPalette.textSecondary,
),
diff --git a/desktop/lib/models/model_manager_controller.dart b/desktop/lib/models/model_manager_controller.dart
index 7c4cc34e4..768e8f65a 100644
--- a/desktop/lib/models/model_manager_controller.dart
+++ b/desktop/lib/models/model_manager_controller.dart
@@ -21,17 +21,30 @@ class ModelManagerController extends ChangeNotifier {
this.app, {
LocalKeyValueStore? storage,
this.poll = true,
- this.targetMachineId,
- }) : _storage = storage ?? (kUnderTest ? null : HarnessFileStore.shared);
+ String? targetMachineId,
+ }) : _targetMachineId =
+ targetMachineId ??
+ (app.viewer == null
+ ? null
+ : app.ownedActionMachine?.machine.machineId),
+ _followsBrowserChoice = targetMachineId == null && app.viewer != null,
+ _storage = storage ?? (kUnderTest ? null : HarnessFileStore.shared);
final AppNotifier app;
final LocalKeyValueStore? _storage;
final bool poll;
/// A remote host uses the same inventory and lifecycle RPCs, without setting
/// up a Model Manager harness or reading another copy of the shared catalog.
- final String? targetMachineId;
+ String? _targetMachineId;
+ final bool _followsBrowserChoice;
+ String? get targetMachineId {
+ if (_targetMachineId != null || app.viewer == null) return _targetMachineId;
+ return _targetMachineId = app.ownedActionMachine?.machine.machineId;
+ }
+
ApiConnectionsController? _apis;
- ApiConnectionsController get apis => _apis ??= ApiConnectionsController(app);
+ ApiConnectionsController get apis =>
+ _apis ??= ApiConnectionsController(app, machineId: targetMachineId);
static const introKey = 'models.introduction.dismissed';
MachineState? _machine;
Future<void>? _preparing, _opening, _refreshing;
@@ -57,7 +70,7 @@ class ModelManagerController extends ChangeNotifier {
bool pendingDownload = false;
MachineState? get machine => targetMachineId == null
- ? app.localMachineState
+ ? app.ownedActionMachine
: app.stateOf(targetMachineId!);
Agent? get manager => machine?.agents
.where((a) => a.dsh == AppNotifier.gridHarness)
@@ -348,6 +361,18 @@ class ModelManagerController extends ChangeNotifier {
}
void setPanelVisible(bool visible) {
+ if (visible &&
+ !_panelVisible &&
+ _followsBrowserChoice &&
+ !busy &&
+ !(_apis?.saving ?? false)) {
+ final next = app.ownedActionMachine?.machine.machineId;
+ if (next != null && next != targetMachineId) {
+ _targetMachineId = next;
+ _apis?.useMachine(next);
+ _observe();
+ }
+ }
_panelVisible = visible;
}
diff --git a/desktop/lib/models/models_panel.dart b/desktop/lib/models/models_panel.dart
index 5042c8b43..d628ad1e1 100644
--- a/desktop/lib/models/models_panel.dart
+++ b/desktop/lib/models/models_panel.dart
@@ -398,7 +398,7 @@ class _ModelsPanelState extends State<ModelsPanel> with SectionWakes {
Expanded(
child: Text(
_selectedTab == ModelsTab.apis || _editingApi
- ? 'Available to harness tools on this computer.'
+ ? 'Available to harness tools on ${controller.apis.hostLabel}.'
: 'Select models in a session’s model picker.',
style: AppType.monoMeta(
color: AppPalette.textSecondary,
diff --git a/desktop/lib/orchestrator/orchestrator_launcher.dart b/desktop/lib/orchestrator/orchestrator_launcher.dart
index 44afb73ee..47e27fecd 100644
--- a/desktop/lib/orchestrator/orchestrator_launcher.dart
+++ b/desktop/lib/orchestrator/orchestrator_launcher.dart
@@ -3,6 +3,7 @@ import 'dart:async';
import '../shared/widgets/labeled_field.dart';
import 'package:flutter/material.dart';
+import 'package:flutter/foundation.dart' show kIsWeb;
import 'package:flutter/services.dart';
import 'package:lucide_icons_flutter/lucide_icons.dart';
@@ -39,7 +40,12 @@ class _OrchestratorLauncherState extends State<OrchestratorLauncher> {
bool _starting = false, _automatic = false, _advanced = false;
Map<String, dynamic>? _attempt;
List<Map<String, dynamic>> _recent = [];
- String? get _machine => widget.notifier.localMachineState?.machine.machineId;
+ String? _machineId;
+ String? get _machine =>
+ _machineId ??= widget.notifier.ownedActionMachine?.machine.machineId;
+ String get _host =>
+ widget.notifier.stateOf(_machine ?? '')?.machine.displayName ??
+ 'a connected machine';
@override
void initState() {
@@ -74,7 +80,7 @@ class _OrchestratorLauncherState extends State<OrchestratorLauncher> {
if (_starting || _prompt.text.trim().isEmpty) return;
final machine = _machine;
if (machine == null) {
- setState(() => _error = 'Connect this computer’s Harness daemon first.');
+ setState(() => _error = 'Connect a machine before starting a project.');
return;
}
_attempt ??= {
@@ -154,8 +160,11 @@ class _OrchestratorLauncherState extends State<OrchestratorLauncher> {
padding: const EdgeInsets.all(28),
child: CallbackShortcuts(
bindings: {
- const SingleActivator(LogicalKeyboardKey.enter, meta: true):
- _start,
+ const SingleActivator(
+ LogicalKeyboardKey.enter,
+ meta: !kIsWeb,
+ alt: kIsWeb,
+ ): _start,
},
child: Column(
crossAxisAlignment: CrossAxisAlignment.start,
@@ -254,7 +263,7 @@ class _OrchestratorLauncherState extends State<OrchestratorLauncher> {
_starting
? 'Starting…'
: _attempt == null
- ? 'Start creating ⌘↵'
+ ? 'Start creating ${kIsWeb ? 'Alt+↵' : '⌘↵'}'
: 'Check launch',
),
),
@@ -287,7 +296,7 @@ class _OrchestratorLauncherState extends State<OrchestratorLauncher> {
],
const SizedBox(height: 18),
Text(
- 'Runs on this computer with installed harnesses. Each specialist gets its own folder; no automatic installs.',
+ 'Runs on $_host with installed harnesses. Each specialist gets its own folder; no automatic installs.',
style: grid.AppType.body(
color: grid.AppPalette.textSecondary,
),
diff --git a/desktop/lib/screens/swarm_screen.dart b/desktop/lib/screens/swarm_screen.dart
index f4e38e3fb..8e71370cb 100644
--- a/desktop/lib/screens/swarm_screen.dart
+++ b/desktop/lib/screens/swarm_screen.dart
@@ -178,7 +178,7 @@ class _SwarmScreenState extends State<SwarmScreen> {
bool _commandBarOpen = false;
bool _commandActionInFlight = false;
FocusNode? _commandReturnFocus;
- bool get _hasCommandBar => widget.commandBarEnabled && app.viewer == null;
+ bool get _hasCommandBar => widget.commandBarEnabled;
late final _commandBar = CommandBarController(
catalog: () => buildCommandBarCatalog(
app,
@@ -191,7 +191,7 @@ class _SwarmScreenState extends State<SwarmScreen> {
resolve:
widget.commandResolver ??
(request, cancel) =>
- app.api.resolveCommandBar(request, cancelToken: cancel),
+ app.resolveCommandBar(request, cancelToken: cancel),
)..addListener(_commandChanged);
final _canvasFocus = FocusNode(
debugLabel: 'Swarm canvas',
@@ -1721,8 +1721,14 @@ class _SwarmScreenState extends State<SwarmScreen> {
/// Harness ▸ Add Phone… and `> add phone`: the QR a phone scans to sign in
/// and pair with this computer. See `widgets/add_phone_dialog.dart`.
- Future<void> _addPhone() =>
- _dialog(() => showAddPhoneDialog(context, app, keymap: _keymap));
+ Future<void> _addPhone() => _dialog(
+ () => showAddPhoneDialog(
+ context,
+ app,
+ keymap: _keymap,
+ onConnectMachine: () => unawaited(_openMachines()),
+ ),
+ );
/// Settings, by section, as rows of the box: `> usage` goes straight to
/// Settings ▸ Usage. A palette that finds a setting by name is how an editor
@@ -3755,11 +3761,7 @@ class _SwarmScreenState extends State<SwarmScreen> {
!machine.machine.isShared &&
!isTerminalEngine(pane!.session!.engineId);
}
- if (id == 'keyboard.quick_start' ||
- id == 'keyboard.practice' ||
- id == 'app.onboarding_review' ||
- // A viewer has no daemon of its own to pair a phone with.
- id == 'app.add_phone') {
+ if (id == 'app.onboarding_review') {
return app.viewer == null;
}
if (id == 'agent.rename' ||
@@ -4105,7 +4107,7 @@ class _SwarmScreenState extends State<SwarmScreen> {
WorkspacePaneContext.focused(app),
menuOnly: true,
),
- if (_learning.active && app.viewer == null)
+ if (_learning.active)
WorkspaceQuickStart(
learning: _learning,
onCommand: _runShortcut,
@@ -4277,15 +4279,11 @@ class _SwarmScreenState extends State<SwarmScreen> {
onCommands:
_showSearchCommands,
onQuickStart:
- _learning.offer &&
- app.viewer ==
- null
+ _learning.offer
? _startQuickStart
: null,
onPractice:
- app.viewer == null
- ? _practiceKeyboard
- : null,
+ _practiceKeyboard,
onNew: () => _newAgent(
placement:
HarnessPlacement
diff --git a/desktop/lib/state/app_state.dart b/desktop/lib/state/app_state.dart
index cf84a9ee9..cc6989397 100644
--- a/desktop/lib/state/app_state.dart
+++ b/desktop/lib/state/app_state.dart
@@ -916,13 +916,22 @@ class AppNotifier extends ChangeNotifier {
String machineId,
Map<String, dynamic> payload,
) async {
- if (machineId != localMachineState?.machine.machineId) {
+ final machine = stateOf(machineId);
+ if (machine == null ||
+ machine.machine.isShared ||
+ machine.needsLink ||
+ machine.nodeOnline == false ||
+ machine.connectionStatus != ConnectionStatus.connected) {
throw StateError(
- 'Orchestrator projects currently run on this computer only.',
+ 'Connect this machine to start an orchestrator project.',
);
}
+ final connection = _conn(machineId);
+ if (!connection.isReady) {
+ throw StateError('Reconnect this machine to manage its projects.');
+ }
try {
- return await _conn(machineId).request(
+ return await connection.request(
'orchestrator',
payload: payload,
timeout: const Duration(seconds: 35),
@@ -930,7 +939,7 @@ class AppNotifier extends ChangeNotifier {
} on WsRequestFailure catch (e) {
if (e.code == 'UNSUPPORTED') {
throw StateError(
- 'Update the local Harness CLI to use the orchestrator.',
+ 'Update Harness on this machine to use the orchestrator.',
);
}
rethrow;
@@ -1696,6 +1705,9 @@ class AppNotifier extends ChangeNotifier {
bool signingIn = false;
bool _sessionExpired = false;
bool get sessionExpired => _sessionExpired;
+
+ @visibleForTesting
+ void handleAuthFailureForTest(String message) => _signedOutAtRuntime(message);
bool signingOut = false;
String? signOutError;
Future<void>? _logoutInFlight;
@@ -2415,6 +2427,27 @@ class AppNotifier extends ChangeNotifier {
return null;
}
+ /// Default host for machine-scoped tools. A dialog freezes this choice while someone edits it.
+ /// Native windows retain their local host; a browser uses the focused linked host first.
+ MachineState? get ownedActionMachine {
+ if (viewer == null) return localMachineState;
+ for (final id in [
+ focusedPane?.machineId,
+ selectedMachineId,
+ ...machineStates.keys,
+ ]) {
+ final machine = machineStates[id];
+ if (machine != null &&
+ !machine.machine.isShared &&
+ !machine.needsLink &&
+ machine.nodeOnline != false &&
+ machine.connectionStatus == ConnectionStatus.connected) {
+ return machine;
+ }
+ }
+ return null;
+ }
+
/// Ask the daemon which agent a typed task belongs to. Sends nothing.
///
/// Rides the app's own rpc convention (`ws_conn.request`), so the pending map, the timeout and the
@@ -2446,9 +2479,13 @@ class AppNotifier extends ChangeNotifier {
}
Future<RouteAnswer?> routeTask(String text) async {
- final machineId = localMachineState?.machine.machineId;
- final connection = machineId == null ? null : _pool?[machineId];
- if (connection == null) return null;
+ final machineId = ownedActionMachine?.machine.machineId;
+ final connection = machineId == null
+ ? null
+ : viewer == null
+ ? (_pool?[machineId])
+ : _conn(machineId);
+ if (connection == null || !connection.isReady) return null;
try {
final reply = await connection.request(
'route_task',
@@ -2486,13 +2523,27 @@ class AppNotifier extends ChangeNotifier {
String agentMachineId,
String text,
) async {
- final localId = localMachineState?.machine.machineId;
- if (localId == null) return 'No local machine is connected.';
- final connection = _pool?[localId];
- if (connection == null) return 'No local machine is connected.';
- // The task goes to the LOCAL daemon whichever machine the agent is on: it owns the dispatch that
- // knows the difference (its own registry, or the fleet link to the other computer). Sending it down
- // the remote machine's own socket would be a second delivery path for the same thing.
+ final localId = viewer == null
+ ? localMachineState?.machine.machineId
+ : agentMachineId.isNotEmpty
+ ? agentMachineId
+ : ownedActionMachine?.machine.machineId;
+ if (localId == null) return 'Connect a machine to send a task.';
+ final targetMachine = stateOf(localId);
+ if (viewer != null &&
+ (targetMachine == null ||
+ targetMachine.machine.isShared ||
+ targetMachine.needsLink ||
+ targetMachine.nodeOnline == false ||
+ targetMachine.connectionStatus != ConnectionStatus.connected)) {
+ return 'Reconnect this machine to send the task.';
+ }
+ final connection = viewer == null ? (_pool?[localId]) : _conn(localId);
+ if (connection == null || !connection.isReady) {
+ return 'Reconnect this machine to send the task.';
+ }
+ // Desktop dispatches through its local daemon; a browser uses the agent's linked owner.
+ // Both enter the same daemon task-delivery path, including refusal acknowledgements.
try {
final reply = await connection.request(
'route_send',
@@ -2517,6 +2568,47 @@ class AppNotifier extends ChangeNotifier {
return null;
}
+ Future<Map<String, dynamic>> resolveCommandBar(
+ Map<String, dynamic> request, {
+ required CancelToken cancelToken,
+ }) async {
+ if (viewer == null) {
+ return api.resolveCommandBar(request, cancelToken: cancelToken);
+ }
+ if (cancelToken.isCancelled) throw cancelToken.cancelError!;
+ final machine = ownedActionMachine;
+ if (machine == null) {
+ throw ApiException('Connect a machine, or choose an action below.');
+ }
+ final connection = _conn(machine.machine.machineId);
+ if (!connection.isReady) {
+ throw ApiException('Reconnect this machine, or choose an action below.');
+ }
+ try {
+ return await Future.any([
+ connection.request(
+ 'command_bar',
+ payload: {'request': request},
+ timeout: const Duration(seconds: 16),
+ ),
+ cancelToken.whenCancel.then<Map<String, dynamic>>(
+ (error) => throw error,
+ ),
+ ]);
+ } on WsRequestFailure catch (failure) {
+ throw ApiException(
+ failure.code == 'UNSUPPORTED'
+ ? 'Update Harness on this machine to use smart commands.'
+ : failure.detail ??
+ 'This machine could not answer. Choose an action below.',
+ );
+ } on WsRequestTimeout {
+ throw ApiException(
+ 'This machine did not answer. Update Harness or choose an action below.',
+ );
+ }
+ }
+
MachineState? get activeMachineState {
final terminal = activeTerminal;
if (terminal != null) return machineStates[terminal.machineId];
@@ -4113,6 +4205,36 @@ class AppNotifier extends ChangeNotifier {
);
}
+ /// The browser controls only a linked, owned machine's managed viewer. Requests fail while
+ /// disconnected rather than replaying old clicks after a new connection takes over.
+ Future<Map<String, dynamic>> viewerSurface(
+ String machineId,
+ String agentId,
+ Map<String, dynamic> payload,
+ ) {
+ final machine = machineStates[machineId];
+ if (machine == null ||
+ machine.machine.isShared ||
+ machine.needsLink ||
+ machine.nodeOnline == false ||
+ machine.connectionStatus != ConnectionStatus.connected) {
+ return Future.error(
+ StateError('Reconnect this machine to open its viewer.'),
+ );
+ }
+ final connection = _conn(machineId);
+ if (!connection.isReady) {
+ return Future.error(
+ StateError('Reconnect this machine to open its viewer.'),
+ );
+ }
+ return connection.request(
+ 'viewer_surface',
+ payload: {...payload, 'agentId': agentId},
+ timeout: const Duration(seconds: 25),
+ );
+ }
+
int _machineInventoryRevision = 0;
bool _machineInventoryCurrent(int auth, int request) =>
@@ -4163,6 +4285,10 @@ class AppNotifier extends ChangeNotifier {
return null;
} catch (error) {
if (!_machineInventoryCurrent(revision, request)) return null;
+ if (viewer != null && isUnauthorizedError(error)) {
+ _signedOutAtRuntime('Your sign-in expired. Sign in again.');
+ return null;
+ }
if (isTransientApiError(error)) {
_scheduleMachineRecovery(revision);
} else {
@@ -5443,11 +5569,25 @@ class AppNotifier extends ChangeNotifier {
Future<Map<String, dynamic>> apiConnections(
String machineId,
Map<String, dynamic> payload,
- ) => _conn(machineId).request(
- 'api_connections',
- payload: payload,
- timeout: const Duration(seconds: 10),
- );
+ ) async {
+ final machine = stateOf(machineId);
+ if (machine == null ||
+ machine.machine.isShared ||
+ machine.needsLink ||
+ machine.nodeOnline == false ||
+ machine.connectionStatus != ConnectionStatus.connected) {
+ throw StateError('Reconnect this machine to manage its APIs.');
+ }
+ final connection = _conn(machineId);
+ if (!connection.isReady) {
+ throw StateError('Reconnect this machine to manage its APIs.');
+ }
+ return connection.request(
+ 'api_connections',
+ payload: payload,
+ timeout: const Duration(seconds: 10),
+ );
+ }
Future<Map<String, dynamic>> controlLocalModel(
String machineId,
@@ -6963,6 +7103,26 @@ class AppNotifier extends ChangeNotifier {
await _loadMachineData(machine, force: true);
}
+ /// Arm the selected owner's daemon through an already linked browser. Never queue a QR code
+ /// across reconnection: closing the dialog must end future pairing attempts.
+ Future<Map<String, dynamic>> pairPhone(String machineId, String code) {
+ final machine = stateOf(machineId);
+ if (machine == null ||
+ machine.machine.isShared ||
+ machine.needsLink ||
+ machine.nodeOnline == false ||
+ machine.connectionStatus != ConnectionStatus.connected) {
+ return Future.value({'error': 'BACKEND_DOWN'});
+ }
+ final connection = _conn(machineId);
+ if (!connection.isReady) return Future.value({'error': 'BACKEND_DOWN'});
+ return connection.request(
+ 'phone_pair',
+ payload: {'code': code},
+ timeout: const Duration(seconds: 60),
+ );
+ }
+
/// What every connected REMOTE machine's agent accounts have spent, asked in
/// parallel and read there with that machine's own credentials (`usage_read`).
///
diff --git a/desktop/lib/state/new_harness.dart b/desktop/lib/state/new_harness.dart
index 7ca42a732..e87440c3e 100644
--- a/desktop/lib/state/new_harness.dart
+++ b/desktop/lib/state/new_harness.dart
@@ -33,7 +33,8 @@ import 'pane_arrangement.dart';
/// The app, including Store Open/Try, uses the box with inline advanced options.
/// Legacy form tests leave this off; dock
/// journeys enable it explicitly.
-bool newHarnessOpensInBox = !kUnderTest;
+bool newHarnessOpensInBox =
+ !kUnderTest || const bool.fromEnvironment('HARNESS_CURRENT_WORKSPACE');
/// A launch command with inherited arguments. Arrows select a launch argument;
/// a focused prompt edits one argument at a time. Enter
diff --git a/desktop/lib/state/terminal_pane.dart b/desktop/lib/state/terminal_pane.dart
index 06a66fd99..c50fd62cf 100644
--- a/desktop/lib/state/terminal_pane.dart
+++ b/desktop/lib/state/terminal_pane.dart
@@ -74,6 +74,9 @@ class TerminalPane {
bool claimOnFirstAttach = false;
TerminalSession? session;
+
+ /// Browser viewer input, installed only while its renderer is mounted.
+ bool Function()? focusViewerInput;
SharedHarness? sharedHarness;
String? sharedOwnerName;
diff --git a/desktop/lib/viewer/direct_auth_api.dart b/desktop/lib/viewer/direct_auth_api.dart
index aec3046d0..e969f4fba 100644
--- a/desktop/lib/viewer/direct_auth_api.dart
+++ b/desktop/lib/viewer/direct_auth_api.dart
@@ -1,16 +1,18 @@
import 'package:dio/dio.dart';
import '../api/api_client.dart';
+import '../api/access_token_source.dart';
import '../core/config.dart';
import '../logging/http_log.dart';
/// A sign-in or refresh that produced no usable session.
-class DirectAuthException implements Exception {
+class DirectAuthException implements AccessTokenFailure {
const DirectAuthException(this.message, {this.signedOut = false});
final String message;
/// The session is gone for good: retrying cannot help, only signing in again.
+ @override
final bool signedOut;
@override
diff --git a/desktop/lib/viewer/interactive_viewer.dart b/desktop/lib/viewer/interactive_viewer.dart
new file mode 100644
index 000000000..ed0015747
--- /dev/null
+++ b/desktop/lib/viewer/interactive_viewer.dart
@@ -0,0 +1,389 @@
+import 'dart:async';
+import 'dart:convert';
+
+import 'package:flutter/gestures.dart';
+import 'package:flutter/material.dart';
+import 'package:flutter/services.dart';
+// The same web/IME input adapter used by our vendored terminal.
+// ignore: implementation_imports
+import 'package:xterm/src/ui/custom_text_edit.dart';
+
+import '../e2ee/bytes.dart';
+import '../shared/theme/app_theme.dart' as grid;
+import '../terminal/terminal_text.dart';
+import '../widgets/terminal_text_action.dart';
+import '../ws/ws_conn.dart';
+
+typedef ViewerSurfaceRequest = Future<Map<String, dynamic>> Function(
+ Map<String, dynamic> payload,
+);
+
+/// Pulling the next frame only after the previous reply bounds both rendering and network work.
+/// Input is connection-local: a failed request is never replayed into a replacement viewer.
+class InteractiveViewerSession extends ChangeNotifier {
+ InteractiveViewerSession(this.request);
+ final ViewerSurfaceRequest request;
+ final String id = hexOf(secureRandomBytes(16));
+ Uint8List? image;
+ String? error;
+ bool Function()? focusInput;
+ final _events = <Map<String, dynamic>>[];
+ Timer? _timer;
+ bool _disposed = false, _busy = false, _reload = false;
+ int _width = 0, _height = 0;
+ bool _dark = true;
+
+ void configure(Size size, bool dark) {
+ if (_disposed || size.isEmpty) return;
+ _width = size.width.round().clamp(160, 1920);
+ _height = size.height.round().clamp(120, 1200);
+ _dark = dark;
+ if (!_busy && _timer == null) _schedule(Duration.zero);
+ }
+
+ void input(Map<String, dynamic> event) {
+ if (_disposed || error != null || image == null) return;
+ // Mouse motion can be coalesced, but never across a press/release or key event.
+ if (event['event'] == 'mouseMoved' &&
+ _events.isNotEmpty &&
+ _events.last['event'] == 'mouseMoved') {
+ _events[_events.length - 1] = event;
+ } else if (_events.length < 64) {
+ _events.add(event);
+ } else {
+ _events.clear();
+ error = 'The connection is too slow. Reconnect the viewer and try again.';
+ _timer?.cancel();
+ _timer = null;
+ notifyListeners();
+ return;
+ }
+ if (!_busy) _schedule(Duration.zero);
+ }
+
+ void reload() {
+ if (_disposed) return;
+ error = null;
+ _events.clear();
+ _reload = true;
+ notifyListeners();
+ if (!_busy) _schedule(Duration.zero);
+ }
+
+ void _schedule(Duration delay) {
+ _timer?.cancel();
+ _timer = Timer(delay, () {
+ _timer = null;
+ unawaited(_frame());
+ });
+ }
+
+ Future<void> _frame() async {
+ if (_disposed || _busy || _width == 0 || error != null) return;
+ _busy = true;
+ final events = List<Map<String, dynamic>>.of(_events);
+ _events.clear();
+ final reload = _reload;
+ _reload = false;
+ try {
+ final reply = await request({
+ 'surfaceId': id,
+ 'op': 'frame',
+ 'width': _width,
+ 'height': _height,
+ 'dark': _dark,
+ 'reload': reload,
+ 'events': events,
+ });
+ if (_disposed) return;
+ if (reply['error'] != null) {
+ error =
+ reply['detail'] as String? ??
+ 'The viewer is unavailable. Try again.';
+ } else if (reply['mime'] != 'image/jpeg' ||
+ reply['data'] is! String ||
+ (reply['data'] as String).length > 2 * 1024 * 1024) {
+ error = 'The viewer sent an invalid image.';
+ } else {
+ image = base64Decode(reply['data'] as String);
+ }
+ } on WsRequestTimeout {
+ if (!_disposed) {
+ error =
+ 'Update Harness on this machine to use its viewer in the browser.';
+ }
+ } on WsRequestFailure catch (failure) {
+ if (!_disposed) {
+ error = failure.code == 'UNSUPPORTED'
+ ? 'Update Harness on this machine to use its viewer in the browser.'
+ : 'The viewer disconnected. Reconnect and try again.';
+ }
+ } catch (_) {
+ if (!_disposed) {
+ error = 'The viewer disconnected. Reconnect and try again.';
+ }
+ } finally {
+ _busy = false;
+ if (!_disposed) {
+ notifyListeners();
+ if (error == null) {
+ _schedule(
+ _events.isEmpty && !_reload
+ ? const Duration(milliseconds: 160)
+ : Duration.zero,
+ );
+ }
+ }
+ }
+ }
+
+ @override
+ void dispose() {
+ _disposed = true;
+ _timer?.cancel();
+ _events.clear();
+ unawaited(
+ request({'surfaceId': id, 'op': 'close'})
+ .catchError((_) => <String, dynamic>{}),
+ );
+ super.dispose();
+ }
+}
+
+class RemoteViewerSurface extends StatefulWidget {
+ const RemoteViewerSurface({super.key, required this.session});
+ final InteractiveViewerSession session;
+ @override
+ State<RemoteViewerSurface> createState() => _InteractiveViewerState();
+}
+
+class _InteractiveViewerState extends State<RemoteViewerSurface> {
+ final _focus = FocusNode();
+ final _editor = GlobalKey<CustomTextEditState>();
+ Size _size = Size.zero;
+ int _pressed = 0;
+ int _lastPress = 0, _clicks = 0;
+ Offset _lastPosition = Offset.zero;
+
+ bool _focusInput() {
+ final editor = _editor.currentState;
+ if (editor == null) return false;
+ editor.requestKeyboard();
+ return true;
+ }
+
+ @override
+ void initState() {
+ super.initState();
+ widget.session.focusInput = _focusInput;
+ }
+
+ @override
+ void didUpdateWidget(RemoteViewerSurface oldWidget) {
+ super.didUpdateWidget(oldWidget);
+ if (oldWidget.session != widget.session) {
+ oldWidget.session.focusInput = null;
+ widget.session.focusInput = _focusInput;
+ }
+ }
+
+ int get _modifiers {
+ final keys = HardwareKeyboard.instance;
+ return (keys.isAltPressed ? 1 : 0) |
+ (keys.isControlPressed ? 2 : 0) |
+ (keys.isMetaPressed ? 4 : 0) |
+ (keys.isShiftPressed ? 8 : 0);
+ }
+
+ String _button(int buttons) => buttons & kSecondaryButton != 0
+ ? 'right'
+ : buttons & kMiddleMouseButton != 0
+ ? 'middle'
+ : buttons & kPrimaryButton != 0
+ ? 'left'
+ : 'none';
+
+ void _pointer(PointerEvent event, String type, {int? buttons}) {
+ if (_size.isEmpty) return;
+ final down = buttons ?? event.buttons;
+ widget.session.input({
+ 'type': 'pointer',
+ 'event': type,
+ 'x': (event.localPosition.dx / _size.width).clamp(0.0, 1.0),
+ 'y': (event.localPosition.dy / _size.height).clamp(0.0, 1.0),
+ 'buttons': event.buttons & 7,
+ 'button': _button(down),
+ 'modifiers': _modifiers,
+ 'clickCount': type == 'mouseMoved' || type == 'mouseWheel' ? 0 : _clicks,
+ if (event is PointerScrollEvent) ...{
+ 'deltaX': event.scrollDelta.dx.clamp(-2000.0, 2000.0),
+ 'deltaY': event.scrollDelta.dy.clamp(-2000.0, 2000.0),
+ },
+ });
+ }
+
+ static final _keys = <LogicalKeyboardKey, (String, String, int)>{
+ LogicalKeyboardKey.enter: ('Enter', 'Enter', 13),
+ LogicalKeyboardKey.numpadEnter: ('Enter', 'NumpadEnter', 13),
+ LogicalKeyboardKey.backspace: ('Backspace', 'Backspace', 8),
+ LogicalKeyboardKey.delete: ('Delete', 'Delete', 46),
+ LogicalKeyboardKey.tab: ('Tab', 'Tab', 9),
+ LogicalKeyboardKey.escape: ('Escape', 'Escape', 27),
+ LogicalKeyboardKey.arrowUp: ('ArrowUp', 'ArrowUp', 38),
+ LogicalKeyboardKey.arrowDown: ('ArrowDown', 'ArrowDown', 40),
+ LogicalKeyboardKey.arrowLeft: ('ArrowLeft', 'ArrowLeft', 37),
+ LogicalKeyboardKey.arrowRight: ('ArrowRight', 'ArrowRight', 39),
+ LogicalKeyboardKey.home: ('Home', 'Home', 36),
+ LogicalKeyboardKey.end: ('End', 'End', 35),
+ LogicalKeyboardKey.pageUp: ('PageUp', 'PageUp', 33),
+ LogicalKeyboardKey.pageDown: ('PageDown', 'PageDown', 34),
+ };
+
+ void _key((String, String, int) key, bool down) => widget.session.input({
+ 'type': 'key',
+ 'event': down ? 'keyDown' : 'keyUp',
+ 'key': key.$1,
+ 'code': key.$2,
+ 'keyCode': key.$3,
+ 'modifiers': _modifiers,
+ });
+
+ KeyEventResult _onKey(FocusNode _, KeyEvent event) {
+ // Alt/Command shortcuts remain workspace navigation, as in a terminal pane.
+ if (HardwareKeyboard.instance.isAltPressed ||
+ HardwareKeyboard.instance.isMetaPressed) {
+ return KeyEventResult.ignored;
+ }
+ var key = _keys[event.logicalKey];
+ if (key == null && HardwareKeyboard.instance.isControlPressed) {
+ final label = event.logicalKey.keyLabel;
+ if (label.length == 1) {
+ key = (
+ label.toLowerCase(),
+ 'Key${label.toUpperCase()}',
+ label.toUpperCase().codeUnitAt(0),
+ );
+ }
+ }
+ if (key == null) return KeyEventResult.ignored;
+ final down = event is! KeyUpEvent;
+ _key(key, down);
+ if (down) _editor.currentState?.resetEditingState();
+ return KeyEventResult.handled;
+ }
+
+ @override
+ void dispose() {
+ widget.session.focusInput = null;
+ _focus.dispose();
+ super.dispose();
+ }
+
+ @override
+ Widget build(BuildContext context) {
+ grid.AppTheme.watch(context);
+ return LayoutBuilder(
+ builder: (context, constraints) {
+ _size = constraints.biggest;
+ widget.session.configure(
+ _size,
+ grid.AppTheme.brightness.value == Brightness.dark,
+ );
+ return ListenableBuilder(
+ listenable: widget.session,
+ builder: (context, _) {
+ final error = widget.session.error;
+ if (error != null) {
+ return Center(
+ child: Padding(
+ padding: const EdgeInsets.all(24),
+ child: Column(
+ mainAxisSize: MainAxisSize.min,
+ children: [
+ Text(
+ error,
+ style: terminalContentStyle(),
+ textAlign: TextAlign.center,
+ ),
+ const SizedBox(height: 16),
+ TerminalTextAction(
+ label: 'Retry',
+ onPressed: widget.session.reload,
+ ),
+ ],
+ ),
+ ),
+ );
+ }
+ final bytes = widget.session.image;
+ if (bytes == null) {
+ return Center(
+ child: Text('Opening viewer…', style: terminalContentStyle()),
+ );
+ }
+ return CustomTextEdit(
+ key: _editor,
+ focusNode: _focus,
+ semanticLabel: 'Viewer input',
+ onInsert: (text) =>
+ widget.session.input({'type': 'text', 'text': text}),
+ onDelete: (count) {
+ for (var i = 0; i < count.clamp(0, 32); i++) {
+ _key(_keys[LogicalKeyboardKey.backspace]!, true);
+ _key(_keys[LogicalKeyboardKey.backspace]!, false);
+ }
+ },
+ onComposing: (_, _) {},
+ onKeyEvent: _onKey,
+ onAction: (_) {
+ _key(_keys[LogicalKeyboardKey.enter]!, true);
+ _key(_keys[LogicalKeyboardKey.enter]!, false);
+ _editor.currentState?.resetEditingState();
+ },
+ child: Listener(
+ behavior: HitTestBehavior.opaque,
+ onPointerDown: (event) {
+ _editor.currentState?.requestKeyboard();
+ final now = DateTime.now().millisecondsSinceEpoch;
+ _clicks =
+ now - _lastPress < 400 &&
+ (event.localPosition - _lastPosition).distance < 5
+ ? 2
+ : 1;
+ _lastPress = now;
+ _lastPosition = event.localPosition;
+ _pressed = event.buttons;
+ _pointer(event, 'mousePressed');
+ },
+ onPointerUp: (event) {
+ _pointer(event, 'mouseReleased', buttons: _pressed);
+ _pressed = 0;
+ },
+ onPointerCancel: (event) {
+ _pointer(event, 'mouseReleased', buttons: _pressed);
+ _pressed = 0;
+ },
+ onPointerMove: (event) => _pointer(event, 'mouseMoved'),
+ onPointerHover: (event) => _pointer(event, 'mouseMoved'),
+ onPointerSignal: (event) {
+ if (event is PointerScrollEvent) {
+ GestureBinding.instance.pointerSignalResolver.register(
+ event,
+ (event) => _pointer(event, 'mouseWheel'),
+ );
+ }
+ },
+ child: Image.memory(
+ bytes,
+ fit: BoxFit.fill,
+ gaplessPlayback: true,
+ excludeFromSemantics: true,
+ ),
+ ),
+ );
+ },
+ );
+ },
+ );
+ }
+}
diff --git a/desktop/lib/widgets/add_phone_dialog.dart b/desktop/lib/widgets/add_phone_dialog.dart
index 11a7700d1..a3e4a2544 100644
--- a/desktop/lib/widgets/add_phone_dialog.dart
+++ b/desktop/lib/widgets/add_phone_dialog.dart
@@ -12,11 +12,13 @@ import '../api/api_client.dart';
import '../shared/theme/app_theme.dart' as grid;
import '../shortcuts/app_keymap.dart';
import '../state/app_state.dart';
+import '../ws/ws_conn.dart' show WsRequestFailure, WsRequestTimeout;
import '../terminal/terminal_text.dart';
import '../terminal/terminal_theme.dart';
import '../terminal/terminal_theme_store.dart';
import 'box_chrome.dart';
import 'terminal_prompt.dart';
+import 'terminal_text_action.dart';
/// Harness ▸ Add Phone… — a QR the phone scans to sign in to this account AND
/// pair with this computer, end to end encrypted, with no password typed.
@@ -45,13 +47,15 @@ Future<void> showAddPhoneDialog(
AppKeymap? keymap,
PhonePairCall? pair,
PhoneSignInCodeCall? signInCode,
+ VoidCallback? onConnectMachine,
}) => showTerminalPrompt<void>(
context,
keymap: keymap,
builder: (_) => AddPhoneDialog(
app: app,
- pair: pair ?? phonePairOverDaemon(app.api),
+ pair: pair,
signInCode: signInCode ?? app.api.phoneSignInCode,
+ onConnectMachine: onConnectMachine,
),
);
@@ -189,6 +193,41 @@ PhonePairCall phonePairOverDaemon(ApiClient api) => (code, cancel) async {
}
};
+/// The browser already holds the machine's trusted identity. Arm that machine over its encrypted
+/// connection; the account service never receives the QR's pairing secret.
+PhonePairCall phonePairOverRelay(AppNotifier app, String machineId) =>
+ (code, cancel) async {
+ if (cancel.isCancelled) return const PhonePairAnswer.failed('CANCELLED');
+ try {
+ final answer = await Future.any([
+ app.pairPhone(machineId, code),
+ cancel.whenCancel.then(
+ (_) => <String, dynamic>{'error': 'CANCELLED'},
+ ),
+ ]);
+ if (answer['ok'] == true) {
+ final label = answer['label'];
+ return PhonePairAnswer.paired(
+ label is String && label.trim().isNotEmpty ? label.trim() : 'phone',
+ );
+ }
+ final error = answer['error'];
+ return PhonePairAnswer.failed(
+ error == 'UNSUPPORTED' ? PhonePairAnswer.unavailable : '$error',
+ );
+ } on WsRequestTimeout {
+ return const PhonePairAnswer.failed(PhonePairAnswer.unavailable);
+ } on WsRequestFailure catch (failure) {
+ return PhonePairAnswer.failed(
+ failure.code == 'UNSUPPORTED'
+ ? PhonePairAnswer.unavailable
+ : failure.code,
+ );
+ } catch (_) {
+ return const PhonePairAnswer.failed('BACKEND_DOWN');
+ }
+ };
+
/// A fresh one-time sign-in code for the QR and how long it stays good, or
/// null when there is none to be had (see [ApiClient.phoneSignInCode]).
typedef PhoneSignInCodeCall = Future<({String code, Duration ttl})?> Function();
@@ -197,13 +236,15 @@ class AddPhoneDialog extends StatefulWidget {
const AddPhoneDialog({
super.key,
required this.app,
- required this.pair,
+ this.pair,
required this.signInCode,
+ this.onConnectMachine,
});
final AppNotifier app;
- final PhonePairCall pair;
+ final PhonePairCall? pair;
final PhoneSignInCodeCall signInCode;
+ final VoidCallback? onConnectMachine;
@override
State<AddPhoneDialog> createState() => _AddPhoneDialogState();
@@ -230,6 +271,17 @@ class _AddPhoneDialogState extends State<AddPhoneDialog> {
AppNotifier get app => widget.app;
+ // Freeze the destination while a QR is visible, even if focus or inventory changes.
+ late final String? _remoteMachineId = app.viewer == null
+ ? null
+ : app.ownedActionMachine?.machine.machineId;
+
+ late final PhonePairCall _pair =
+ widget.pair ??
+ (_remoteMachineId != null
+ ? phonePairOverRelay(app, _remoteMachineId)
+ : phonePairOverDaemon(app.api));
+
/// A fresh code each time the dialog opens, and a fresh one again after a
/// scan that did not match — see [_loop].
late String _code = newPhonePairCode();
@@ -298,7 +350,9 @@ class _AddPhoneDialogState extends State<AddPhoneDialog> {
({String email, String machineId})? get _target {
if (app.isGuest) return null;
final email = app.currentUser?.email.trim();
- final machineId = app.localMachineState?.machine.machineId;
+ final machineId = app.viewer == null
+ ? app.localMachineState?.machine.machineId
+ : _remoteMachineId;
// `CurrentUserProfile.local()` says "local terminal": not an address.
if (email == null || !email.contains('@')) return null;
if (machineId == null || machineId.isEmpty) return null;
@@ -321,7 +375,7 @@ class _AddPhoneDialogState extends State<AddPhoneDialog> {
final code = _code;
PhonePairAnswer answer;
try {
- answer = await widget.pair(code, _cancel);
+ answer = await _pair(code, _cancel);
} catch (error) {
answer = PhonePairAnswer.failed('$error');
}
@@ -355,7 +409,11 @@ class _AddPhoneDialogState extends State<AddPhoneDialog> {
_say('Too many tries. Wait a minute.');
wait = _rateLimited;
case 'BACKEND_DOWN':
- _say("This ${_thisComputer()} can't reach Harness right now.");
+ _say(
+ app.viewer == null
+ ? "This ${_thisComputer()} can't reach Harness right now."
+ : 'The computer is offline. Reconnect it to add your phone.',
+ );
wait = _poll;
// The phone started and went quiet, or said no. The QR is still
// good: scanning it again starts over.
@@ -366,7 +424,11 @@ class _AddPhoneDialogState extends State<AddPhoneDialog> {
_say('Cancelled on your phone. Scan again.', sticky: true);
wait = _poll;
case PhonePairAnswer.unavailable:
- _say('Update Harness on this ${_thisComputer()} to add a phone.');
+ _say(
+ app.viewer == null
+ ? 'Update Harness on this ${_thisComputer()} to add a phone.'
+ : 'Update Harness on the computer to add a phone.',
+ );
setState(() => _stopped = true);
return;
case final other:
@@ -504,6 +566,21 @@ class _AddPhoneDialogState extends State<AddPhoneDialog> {
}
final target = _target;
if (target == null) {
+ if (app.viewer != null && app.currentUser?.email.contains('@') == true) {
+ return [
+ Text('Connect a computer to add your phone.', style: _ink()),
+ if (widget.onConnectMachine case final connect?) ...[
+ SizedBox(height: row),
+ TerminalTextAction(
+ label: 'Connect a machine',
+ onPressed: () {
+ Navigator.of(context).pop();
+ connect();
+ },
+ ),
+ ],
+ ];
+ }
return [
Text(
app.currentUser?.email.contains('@') == true
@@ -531,6 +608,14 @@ class _AddPhoneDialogState extends State<AddPhoneDialog> {
),
),
SizedBox(height: row),
+ if (_remoteMachineId case final machineId?) ...[
+ Text(
+ app.stateOf(machineId)?.machine.displayName ?? 'Computer',
+ textAlign: TextAlign.center,
+ style: _ink(_faint),
+ ),
+ SizedBox(height: row),
+ ],
// One line, and it is the status too: what to do, then that it worked.
// The phone's own screen says the rest (Yes — scan to connect).
_status(),
diff --git a/desktop/lib/widgets/api_picker_form.dart b/desktop/lib/widgets/api_picker_form.dart
index 0486e0ec0..10abc3c0a 100644
--- a/desktop/lib/widgets/api_picker_form.dart
+++ b/desktop/lib/widgets/api_picker_form.dart
@@ -429,6 +429,8 @@ class ApiPickerFormState extends State<ApiPickerForm> {
: _editing?.name ?? 'Add API',
style: style,
),
+ if (controller.app.viewer != null)
+ Text('Saved on ${controller.hostLabel}', style: muted),
SizedBox(height: cell.height),
if (_editing != null &&
!widget.removing &&
@@ -500,7 +502,10 @@ class ApiPickerFormState extends State<ApiPickerForm> {
),
),
if (_editing != null && !widget.removing) ...[
- Text('Stored on this computer.', style: muted),
+ Text(
+ 'Stored on ${controller.hostLabel}.',
+ style: muted,
+ ),
SizedBox(height: cell.height),
],
if (_error ?? controller.error case final error?) ...[
diff --git a/desktop/lib/widgets/pane_grid.dart b/desktop/lib/widgets/pane_grid.dart
index 485a2ed5c..519595636 100644
--- a/desktop/lib/widgets/pane_grid.dart
+++ b/desktop/lib/widgets/pane_grid.dart
@@ -330,7 +330,8 @@ class _SwarmCanvasState extends State<_SwarmCanvas> {
ModalRoute.of(context)?.isCurrent == false) {
return;
}
- if (app.focusedPane?.session?.focusInput() != true) {
+ if (app.focusedPane?.session?.focusInput() != true &&
+ app.focusedPane?.focusViewerInput?.call() != true) {
// Blank pages and not-yet-mounted destinations must release the old
// terminal's text client immediately, without focusing welcome search.
_idleFocus.requestFocus();
diff --git a/desktop/lib/widgets/task_palette.dart b/desktop/lib/widgets/task_palette.dart
index cf1356327..801dc3e5f 100644
--- a/desktop/lib/widgets/task_palette.dart
+++ b/desktop/lib/widgets/task_palette.dart
@@ -291,16 +291,22 @@ class _TaskPaletteState extends State<_TaskPalette> {
if (answer == null) {
setState(() {
_stage = _Stage.empty;
- _note = widget.notifier.localMachineState == null
- ? 'No local machine is connected yet.'
- : 'Could not reach the router on this computer.';
+ _note = widget.notifier.ownedActionMachine == null
+ ? (widget.notifier.viewer == null
+ ? 'No local machine is connected yet.'
+ : 'Connect a machine first.')
+ : (widget.notifier.viewer == null
+ ? 'Could not reach the router on this computer.'
+ : 'Could not reach the router on the connected machine.');
});
return;
}
if (answer.isEmpty) {
setState(() {
_stage = _Stage.empty;
- _note = 'There is no agent on this computer to send that to.';
+ _note = widget.notifier.viewer == null
+ ? 'There is no agent on this computer to send that to.'
+ : 'There is no agent on the connected machine to send that to.';
});
return;
}
diff --git a/desktop/lib/widgets/web_pane_panel.dart b/desktop/lib/widgets/web_pane_panel.dart
index 4232effdd..e169eddb8 100644
--- a/desktop/lib/widgets/web_pane_panel.dart
+++ b/desktop/lib/widgets/web_pane_panel.dart
@@ -1,3 +1,4 @@
+import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import 'package:lucide_icons_flutter/lucide_icons.dart';
import 'package:webview_flutter/webview_flutter.dart';
@@ -11,6 +12,7 @@ import '../shared/theme/workspace_bar_style.dart';
import '../state/app_state.dart';
import '../state/terminal_pane.dart';
import '../theme/app_theme.dart';
+import '../viewer/interactive_viewer.dart';
import 'engine_identity.dart';
import '../terminal/terminal_text.dart';
import 'verdict_marks.dart';
@@ -78,6 +80,8 @@ class WebPanePanel extends StatefulWidget {
class _WebPanePanelState extends State<WebPanePanel> {
WebViewController? _controller;
+ InteractiveViewerSession? _remote;
+ String? _remoteIdentity;
String? _loadedUrl;
bool _loading = false;
String? _failure;
@@ -88,9 +92,36 @@ class _WebPanePanelState extends State<WebPanePanel> {
@override
void initState() {
super.initState();
+ _mountRemote();
if (WebPanePanel.webviewAvailable) _mountController();
}
+ void _mountRemote() {
+ if (!kIsWeb) return;
+ final pane = widget.pane;
+ final identity = '${pane.machineId}/${pane.ownerAgentId}/${pane.url}';
+ if (_remoteIdentity == identity) return;
+ _remote?.dispose();
+ _remoteIdentity = identity;
+ final notifier = widget.notifier;
+ final machineId = pane.machineId, agentId = pane.ownerAgentId!;
+ _remote = InteractiveViewerSession(
+ (payload) => notifier.viewerSurface(machineId, agentId, payload),
+ );
+ pane.focusViewerInput = _focusRemote;
+ }
+
+ bool _focusRemote() => _remote?.focusInput?.call() ?? false;
+
+ @override
+ void dispose() {
+ if (widget.pane.focusViewerInput == _focusRemote) {
+ widget.pane.focusViewerInput = null;
+ }
+ _remote?.dispose();
+ super.dispose();
+ }
+
void _mountController() {
// WebKit's default media policy wants a click before any playback, which
// leaves a viewer's muted video sitting at 00:00 with a play button; a
@@ -179,6 +210,10 @@ class _WebPanePanelState extends State<WebPanePanel> {
}
void _reload() {
+ if (_remote case final remote?) {
+ remote.reload();
+ return;
+ }
if (_controller == null) return;
if (_loadedUrl != widget.pane.url) {
_load();
@@ -192,6 +227,7 @@ class _WebPanePanelState extends State<WebPanePanel> {
@override
void didUpdateWidget(WebPanePanel oldWidget) {
super.didUpdateWidget(oldWidget);
+ _mountRemote();
// The daemon named a different page — the newest artifact, a viewer
// restarted on another port. Navigate in place; the tile stays.
if (widget.pane.url != _loadedUrl) _load();
@@ -283,7 +319,7 @@ class _WebPanePanelState extends State<WebPanePanel> {
// coverage:ignore-end
_ViewerActions(
zoomed: widget.zoomed,
- onReload: _controller == null ? null : _reload,
+ onReload: _controller == null && _remote == null ? null : _reload,
onZoom: widget.onToggleZoom,
onClose: widget.onClose,
),
@@ -302,6 +338,7 @@ class _WebPanePanelState extends State<WebPanePanel> {
detail: error,
);
}
+ if (_remote case final remote?) return RemoteViewerSurface(session: remote);
final controller = _controller;
final url = widget.pane.url;
if (controller == null) {
diff --git a/desktop/lib/ws/ws_conn.dart b/desktop/lib/ws/ws_conn.dart
index 9e920f203..9331a04db 100644
--- a/desktop/lib/ws/ws_conn.dart
+++ b/desktop/lib/ws/ws_conn.dart
@@ -592,6 +592,12 @@ class WsConn {
static bool _worthLogging(String type) =>
!_unlogged.contains(type) &&
+ !type.startsWith('phone_pair') &&
+ !type.startsWith('viewer_surface') &&
+ !type.startsWith('api_connections') &&
+ !type.startsWith('orchestrator') &&
+ !type.startsWith('command_bar') &&
+ !type.startsWith('route_') &&
!type.startsWith('harness_share_') &&
!type.startsWith('observer_');
diff --git a/desktop/scripts/check-workspace.cjs b/desktop/scripts/check-workspace.cjs
new file mode 100644
index 000000000..44e93e9f4
--- /dev/null
+++ b/desktop/scripts/check-workspace.cjs
@@ -0,0 +1,212 @@
+// Real browser workspace check. Only the SSO redirect/exchange is mocked; the backend and
+// owner daemon are disposable processes launched by cli/scripts/share-harness-e2e.ts.
+const { chromium } = require(process.env.HARNESS_PLAYWRIGHT_MODULE || 'playwright');
+const assert = require('node:assert/strict');
+const fs = require('node:fs');
+const path = require('node:path');
+const fixture = JSON.parse(fs.readFileSync(process.argv[2], 'utf8'));
+const { origin, backendUrl, token, root, password } = fixture;
+assert.equal(new URL(origin).hostname, '127.0.0.1');
+assert.equal(new URL(backendUrl).hostname, '127.0.0.1');
+
+(async () => {
+ const browser = await chromium.launch({ channel: 'chrome', headless: true });
+ const context = await browser.newContext({ viewport: { width: 1440, height: 900 } });
+ let page = await context.newPage();
+ const errors = [], calls = [];
+ let cdp = await context.newCDPSession(page);
+ async function editorReady(label, selector = `textarea[aria-label="${label}"]`) {
+ // Accessibility creates the DOM textbox before Flutter attaches its text-input listener.
+ // Wait for the real editor, as opposed to typing into that unbound DOM placeholder.
+ const deadline = Date.now() + 5000;
+ while (true) {
+ const { result } = await cdp.send('Runtime.evaluate', {
+ expression: `(() => { const selector = ${JSON.stringify(selector)}; function find(root) { const found = root.querySelector(selector); if (found) return found; for (const el of root.querySelectorAll('*')) { if (el.shadowRoot) { const nested = find(el.shadowRoot); if (nested) return nested; } } return null; } return find(document); })()`,
+ });
+ if (result.objectId) {
+ const { listeners } = await cdp.send('DOMDebugger.getEventListeners', { objectId: result.objectId });
+ await cdp.send('Runtime.releaseObject', { objectId: result.objectId });
+ if (listeners.some(listener => listener.type === 'input')) return;
+ }
+ if (Date.now() > deadline) throw new Error(`${label} never attached its browser input listener`);
+ await page.evaluate(() => new Promise(requestAnimationFrame));
+ }
+ }
+ page.on('pageerror', error => errors.push(error.message));
+ const cors = { 'access-control-allow-origin': origin, 'access-control-allow-headers': '*', 'access-control-allow-methods': 'GET,POST,OPTIONS' };
+ async function semantics() {
+ await page.locator('flutter-view').waitFor({ state: 'attached', timeout: 45000 });
+ const placeholder = page.locator('flt-semantics-placeholder');
+ if (await placeholder.count()) await placeholder.evaluate(el => el.click());
+ }
+ async function capture(name) {
+ fs.writeFileSync(path.join(root, `workspace-${name}.txt`), await page.locator('body').ariaSnapshot());
+ fs.writeFileSync(path.join(root, `workspace-${name}.html`), await page.content());
+ await page.screenshot({ path: path.join(root, `workspace-${name}.png`) });
+ fs.writeFileSync(path.join(root, `workspace-${name}-inputs.json`), JSON.stringify(await page.evaluate(() => ({
+ active: document.activeElement?.outerHTML, inputs: [...document.querySelectorAll('input,textarea')].map(el => ({label: el.getAttribute('aria-label'), value: el.value, active: el === document.activeElement})),
+ trace: window.workspaceInputTrace,
+ })), null, 2));
+ }
+ try {
+ await context.addInitScript(() => {
+ window.workspaceInputTrace = [];
+ for (const type of ['focusin', 'focusout', 'input', 'keydown', 'keyup']) document.addEventListener(type, e => {
+ const label = e.target?.getAttribute?.('aria-label');
+ if (label === 'Viewer input' || label === 'Terminal input') window.workspaceInputTrace.push({type, label, key: e.key, value: e.target.value});
+ }, true);
+ });
+ await context.route('**/*', async route => {
+ const url = new URL(route.request().url());
+ if (![origin, backendUrl].includes(url.origin)) return route.abort();
+ calls.push(url.pathname);
+ if (url.pathname === '/api/auth/authorize-native') return route.fulfill({ headers: cors,
+ contentType: 'application/json', body: JSON.stringify({ success: true, data: {
+ authorizeUrl: `${origin}/fixture-authorize?state=workspace-fixture&redirect_uri=${encodeURIComponent(origin + '/callback')}`,
+ tx: 'workspace-transaction',
+ } }) });
+ if (url.pathname === '/fixture-authorize') return route.fulfill({ status: 302,
+ headers: { location: origin + '/callback?code=workspace-code&state=workspace-fixture' }, body: '' });
+ if (url.pathname === '/api/auth/exchange') {
+ assert.deepEqual(route.request().postDataJSON(), { code: 'workspace-code', state: 'workspace-fixture', tx: 'workspace-transaction' });
+ return route.fulfill({ headers: cors, contentType: 'application/json', body: JSON.stringify({ success: true, data: {
+ token, refreshToken: 'fixture-refresh', expiresIn: 3600, autonomousEnv: 'prod',
+ } }) });
+ }
+ return route.continue();
+ });
+ await page.goto(origin); await semantics();
+ await page.getByRole('button', { name: 'Sign in', exact: true }).click();
+ await page.waitForURL('**/callback?**'); await semantics();
+ // The initial link-required machine may open the picker during discovery.
+ await page.getByRole('button', { name: 'Connect', exact: true }).waitFor({ timeout: 30000 });
+ await page.keyboard.press('Escape');
+ await page.getByText('Harness like a boss.', { exact: true }).waitFor({ timeout: 30000 });
+ await capture('welcome');
+ await page.keyboard.press('Alt+m');
+ await page.getByRole('button', { name: 'Connect', exact: true }).waitFor({ timeout: 30000 });
+ await capture('machines');
+ await page.getByRole('button', { name: 'Connect', exact: true }).click();
+ await capture('link');
+ await page.locator('input[type="password"]').fill(password);
+ await page.getByRole('button', { name: 'Connect', exact: true }).click();
+ await page.waitForFunction(machineId => {
+ const peers = JSON.parse(localStorage.getItem('harness.web.v1.viewer_e2ee_machine_peers') || '[]');
+ return peers.some(peer => peer.machineId === machineId);
+ }, fixture.machineId, { timeout: 60000 });
+ await page.locator('input[type="password"]').waitFor({ state: 'detached', timeout: 30000 });
+ await page.getByText(/Connected/).first().waitFor({ timeout: 30000 });
+ await capture('linked');
+ await page.keyboard.press('Escape');
+ if (await page.getByRole('textbox', { name: 'Search results Search machines', exact: true }).count()) await page.keyboard.press('Escape');
+ await page.getByRole('button', { name: 'Work from your phone', exact: true }).click();
+ await page.getByRole('img', { name: 'QR code to add your phone', exact: true }).waitFor({ timeout: 15000 });
+ await page.getByText('owner test machine', { exact: true }).waitFor();
+ await page.getByRole('dialog').getByText('Scan with Harness on your iPhone', { exact: true }).first().waitFor();
+ await capture('phone');
+ await page.keyboard.press('Escape');
+ await page.getByRole('dialog').waitFor({ state: 'detached' });
+ await page.getByRole('button', { name: 'Alt+I Deploy a local model', exact: true }).click();
+ await page.getByRole('textbox').first().fill(':api');
+ await page.getByRole('button', { name: /^\[ Add \] Connect an API/ }).click();
+ await page.getByRole('button', { name: 'fal.ai', exact: true }).click();
+ await page.locator('input[type="password"]').click();
+ await editorReady('API key', 'input[type="password"]');
+ await page.locator('input[type="password"]').fill('browser-fixture-api-key');
+ await page.getByRole('button', { name: 'Save', exact: true }).click();
+ const apiFile = path.join(root, 'owner/data/api-connections/connections.json');
+ const apiDeadline = Date.now() + 10000;
+ while (!fs.existsSync(apiFile) || !fs.readFileSync(apiFile, 'utf8').includes('browser-fixture-api-key')) {
+ if (Date.now() > apiDeadline) throw new Error('The selected daemon did not save the fixture API key');
+ await new Promise(resolve => setTimeout(resolve, 50));
+ }
+ assert.ok(!JSON.stringify(await page.evaluate(() => ({...localStorage}))).includes('browser-fixture-api-key'));
+ await capture('api-saved');
+ await page.keyboard.press('Escape');
+ await page.keyboard.press('Alt+Shift+p');
+ await page.getByRole('textbox').first().fill('>orchestrator');
+ await page.getByRole('button', { name: /Create with the orchestrator/ }).click();
+ await page.getByRole('group', { name: /Orchestrator What would you like to make/ }).waitFor();
+ assert.match(await page.locator('body').ariaSnapshot(), /Runs on owner test machine/);
+ await capture('orchestrator');
+ await page.getByRole('button', { name: 'Close', exact: true }).click();
+ await page.getByRole('dialog').waitFor({ state: 'detached' });
+ await page.keyboard.press('Alt+p');
+ await page.getByRole('textbox').first().fill('Sharing demo');
+ await page.getByRole('button', { name: /^Sharing demo/ }).click();
+ await page.getByRole('button', { name: 'Connecting', exact: true }).waitFor({ state: 'detached', timeout: 30000 });
+ // Prove the viewer is interactive on the remote machine, rather than a local placeholder.
+ const viewer = page.getByRole('textbox', { name: 'Viewer input', exact: true });
+ await viewer.waitFor({ timeout: 45000 });
+ await viewer.click({ position: { x: 70, y: 125 } });
+ async function viewerReceived(marker) {
+ const deadline = Date.now() + 15000;
+ while (!fs.existsSync(path.join(root, 'viewer-events.log')) ||
+ !fs.readFileSync(path.join(root, 'viewer-events.log'), 'utf8').includes(marker)) {
+ if (Date.now() > deadline) throw new Error(`The remote viewer did not receive ${marker}`);
+ await new Promise(resolve => setTimeout(resolve, 50));
+ }
+ }
+ await viewerReceived('click=1');
+ await viewer.click({ position: { x: 70, y: 205 } });
+ await editorReady('Viewer input');
+ await page.keyboard.type('browser-viewer');
+ await viewerReceived('text=browser-viewer');
+ await capture('viewer');
+ // The fixture's specialized harness opens a viewer beside its terminal on the right.
+ await page.getByRole('textbox', { name: 'Terminal input', exact: true }).click();
+ await editorReady('Terminal input');
+ await page.keyboard.type('browser-workspace-input');
+ await page.keyboard.press('Enter');
+ // The raw-mode fixture gets one chunk per keyboard event, just like a real TUI. Check
+ // its received bytes, not an assumption that all letters arrive in one output frame.
+ const inputDeadline = Date.now() + 15000;
+ while (!fs.readFileSync(path.join(root, 'fixture-input.bin'), 'utf8').includes('browser-workspace-input\r')) {
+ if (Date.now() > inputDeadline) throw new Error('The terminal did not receive the typed command');
+ await new Promise(resolve => setTimeout(resolve, 50));
+ }
+ await capture('agent');
+ const peers = await page.evaluate(() => localStorage.getItem('harness.web.v1.viewer_e2ee_machine_peers'));
+ assert.ok(JSON.parse(peers).some(peer => peer.machineId === fixture.machineId));
+ await page.reload(); await semantics();
+ await page.getByRole('link', { name: 'Download app', exact: true }).waitFor({ timeout: 30000 });
+ assert.equal(await page.getByRole('button', { name: 'Sign in', exact: true }).count(), 0);
+ assert.equal(await page.evaluate(() => localStorage.getItem('harness.web.v1.viewer_e2ee_machine_peers')), peers);
+ assert.equal(calls.filter(call => call === '/api/auth/exchange').length, 1);
+ await capture('reloaded');
+ const terminalBounds = await page.getByRole('textbox', { name: 'Terminal input', exact: true }).boundingBox();
+ assert.ok(terminalBounds);
+ await page.close();
+ page = await context.newPage();
+ cdp = await context.newCDPSession(page);
+ page.on('pageerror', error => errors.push(error.message));
+ // Test ordinary pointer/keyboard input too. This tab never enables accessibility semantics.
+ let terminalReady = false;
+ page.on('websocket', socket => socket.on('framereceived', ({ payload }) => {
+ if (typeof payload !== 'string') return;
+ try { if (JSON.parse(payload).type === 'terminal_ready') terminalReady = true; } catch {}
+ }));
+ await page.goto(origin);
+ const readyDeadline = Date.now() + 30000;
+ while (!terminalReady) {
+ if (Date.now() > readyDeadline) throw new Error('The ordinary browser tab did not attach its terminal');
+ await new Promise(resolve => setTimeout(resolve, 50));
+ }
+ await page.mouse.click(terminalBounds.x + 24, terminalBounds.y + 48);
+ await editorReady('ordinary terminal', 'flt-text-editing-host textarea, flt-text-editing-host input');
+ await page.keyboard.type('browser-default-input');
+ await page.keyboard.press('Enter');
+ const normalDeadline = Date.now() + 10000;
+ while (!fs.readFileSync(path.join(root, 'fixture-input.bin'), 'utf8').includes('browser-default-input\r')) {
+ if (Date.now() > normalDeadline) throw new Error('Ordinary browser input did not reach the terminal');
+ await new Promise(resolve => setTimeout(resolve, 50));
+ }
+ await page.screenshot({ path: path.join(root, 'workspace-ordinary-input.png') });
+ assert.deepEqual(errors, []);
+ console.log('PASS browser workspace: sign-in, machine linking, phone QR, API persistence, agent picker, ordinary/accessibility terminal input, interactive viewer, and reload');
+ } catch (error) {
+ await capture('failure');
+ fs.writeFileSync(path.join(root, 'workspace-failure-details.txt'), `${error.stack}\n${JSON.stringify({ errors, calls })}`);
+ throw error;
+ } finally { await browser.close(); }
+})().catch(error => { console.error(error); process.exitCode = 1; });
diff --git a/desktop/test/add_phone_dialog_test.dart b/desktop/test/add_phone_dialog_test.dart
index 14a142f3e..445e4fb12 100644
--- a/desktop/test/add_phone_dialog_test.dart
+++ b/desktop/test/add_phone_dialog_test.dart
@@ -6,13 +6,17 @@ import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:harness/core/models.dart';
+import 'package:harness/core/config.dart';
+import 'package:harness/auth/auth_session.dart';
+import 'package:harness/viewer/viewer_services.dart';
+import 'package:harness/viewer/viewer_key_store.dart';
import 'package:harness/shared/theme/app_theme.dart' as grid;
import 'package:harness/state/app_state.dart';
import 'package:harness/widgets/add_phone_dialog.dart';
import 'swarm_interactions_test.dart' show chord;
import 'swarm_screen_test.dart' show mount;
-import 'swarm_state_test.dart' show createApp;
+import 'swarm_state_test.dart' show createApp, MemoryStore;
/// The alphabet the Add Phone contract names. The code must come from it —
/// and, see [kPhonePairCodeAlphabet], from the part both normalisers agree on.
@@ -65,6 +69,7 @@ Future<void> _open(
AppNotifier app,
PhonePairCall pair, {
PhoneSignInCodeCall? signInCode,
+ VoidCallback? onConnectMachine,
}) async {
tester.view.devicePixelRatio = 1;
tester.view.physicalSize = const Size(1280, 800);
@@ -82,6 +87,7 @@ Future<void> _open(
app,
pair: pair,
signInCode: signInCode ?? _FakeSignIn().call,
+ onConnectMachine: onConnectMachine,
),
);
await tester.pump();
@@ -96,6 +102,81 @@ String _status(WidgetTester tester) =>
tester.widget<Text>(find.byKey(const ValueKey('add-phone-status'))).data!;
void main() {
+ AppNotifier browserApp() {
+ final storage = MemoryStore();
+ final session = AuthSession(storage: storage);
+ return AppNotifier(
+ config: AppConfig.dev,
+ authSession: session,
+ viewer: ViewerServices(
+ config: AppConfig.dev,
+ session: session,
+ keys: ViewerKeyStore(storage: storage),
+ ),
+ )
+ ..status = AppStatus.authenticated
+ ..signedIn = true
+ ..currentUser = const CurrentUserProfile(email: '[email protected]');
+ }
+
+ testWidgets(
+ 'browser QR targets a linked computer and keeps that destination',
+ (tester) async {
+ final app = browserApp();
+ addTearDown(app.dispose);
+ for (final id in ['a', 'b']) {
+ app.machineStates[id] =
+ MachineState(
+ Machine(
+ machineId: id,
+ name: 'Computer $id',
+ authMode: MachineAuthMode.remote,
+ ),
+ )
+ ..connectionStatus = ConnectionStatus.connected
+ ..nodeOnline = true;
+ }
+ app.selectedMachineId = 'b';
+ final daemon = _FakeDaemon(List.filled(3, _failed('NO_INTENT')));
+ await _open(tester, app, daemon.call);
+ final before = _qrData(tester);
+ expect(Uri.splitQueryString(Uri.parse(before).fragment)['m'], 'b');
+ expect(find.text('Computer b'), findsOneWidget);
+ app.selectedMachineId = 'a';
+ app.notifyListeners();
+ await tester.pump();
+ expect(_qrData(tester), before);
+ expect(daemon.codes, hasLength(1));
+ await tester.pumpWidget(const SizedBox());
+ expect(daemon.tokens.single.isCancelled, isTrue);
+ },
+ );
+
+ testWidgets('browser without a linked computer offers the machine picker', (
+ tester,
+ ) async {
+ final app = browserApp();
+ addTearDown(app.dispose);
+ app.machineStates['unlinked'] = MachineState(
+ const Machine(machineId: 'unlinked', authMode: MachineAuthMode.remote),
+ )..needsLink = true;
+ var opened = false;
+ final daemon = _FakeDaemon([]);
+ await _open(
+ tester,
+ app,
+ daemon.call,
+ onConnectMachine: () => opened = true,
+ );
+ expect(find.byType(PhonePairQr), findsNothing);
+ expect(daemon.codes, isEmpty);
+ await tester.tap(find.text('[ Connect a machine ]'));
+ await tester.pump();
+ expect(opened, isTrue);
+ expect(find.byType(AddPhoneDialog), findsNothing);
+ await tester.pumpWidget(const SizedBox());
+ });
+
group('the QR payload', () {
test('is the /pair link with everything in the fragment', () {
final link = phonePairLink(
diff --git a/desktop/test/bearer_auth_interceptor_test.dart b/desktop/test/bearer_auth_interceptor_test.dart
new file mode 100644
index 000000000..3585b34ee
--- /dev/null
+++ b/desktop/test/bearer_auth_interceptor_test.dart
@@ -0,0 +1,117 @@
+import 'dart:typed_data';
+
+import 'package:dio/dio.dart';
+import 'package:flutter_test/flutter_test.dart';
+import 'package:harness/api/access_token_source.dart';
+import 'package:harness/api/api_client.dart';
+import 'package:harness/api/bearer_auth_interceptor.dart';
+import 'package:harness/viewer/direct_auth_api.dart';
+
+class _Auth implements AccessTokenSource {
+ Object? initialFailure, refreshFailure;
+ String token = 'old';
+ int refreshes = 0;
+ @override
+ Future<String> accessToken({bool force = false, String? failedToken}) async {
+ if (!force && initialFailure != null) throw initialFailure!;
+ if (force) {
+ refreshes++;
+ expect(failedToken, 'old');
+ if (refreshFailure != null) throw refreshFailure!;
+ token = 'renewed';
+ }
+ return token;
+ }
+}
+
+class _Adapter implements HttpClientAdapter {
+ final seen = <String>[];
+ @override
+ Future<ResponseBody> fetch(
+ RequestOptions options,
+ Stream<Uint8List>? requestStream,
+ Future<void>? cancelFuture,
+ ) async {
+ final token = options.headers['authorization'] as String;
+ seen.add(token);
+ return ResponseBody.fromString(
+ '{}',
+ token == 'Bearer renewed' ? 200 : 401,
+ headers: {
+ 'content-type': ['application/json'],
+ },
+ );
+ }
+
+ @override
+ void close({bool force = false}) {}
+}
+
+void main() {
+ late _Auth auth;
+ late _Adapter adapter;
+ late Dio dio;
+ setUp(() {
+ auth = _Auth();
+ adapter = _Adapter();
+ dio = Dio(
+ BaseOptions(
+ baseUrl: 'https://fixture.invalid',
+ validateStatus: (_) => true,
+ ),
+ );
+ dio.httpClientAdapter = adapter;
+ dio.interceptors.add(
+ BearerAuthInterceptor(auth, dio, autonomousEnv: 'prod'),
+ );
+ });
+ tearDown(() => dio.close());
+
+ test('renews one refused token and retries with the replacement', () async {
+ expect((await dio.get('/api/machines')).statusCode, 200);
+ expect(adapter.seen, ['Bearer old', 'Bearer renewed']);
+ expect(auth.refreshes, 1);
+ });
+
+ for (final expired in [false, true]) {
+ test(
+ 'refresh failure distinguishes ${expired ? 'expiry' : 'service outage'}',
+ () async {
+ auth.refreshFailure = DirectAuthException(
+ 'refresh failed',
+ signedOut: expired,
+ );
+ await expectLater(
+ dio.get('/api/machines'),
+ throwsA(
+ isA<DioException>()
+ .having(isUnauthorizedError, 'requires sign-in', expired)
+ .having(isTransientApiError, 'can retry', !expired),
+ ),
+ );
+ expect(adapter.seen, ['Bearer old']);
+ },
+ );
+ }
+
+ test(
+ 'an expired session is recognized even before an HTTP request can be sent',
+ () async {
+ auth.initialFailure = const DirectAuthException(
+ 'Not signed in.',
+ signedOut: true,
+ );
+ await expectLater(
+ dio.get('/api/machines'),
+ throwsA(
+ isA<DioException>().having(
+ isUnauthorizedError,
+ 'requires sign-in',
+ true,
+ ),
+ ),
+ );
+ expect(adapter.seen, isEmpty);
+ },
+ );
+}
diff --git a/desktop/test/boot_flow_widget_test.dart b/desktop/test/boot_flow_widget_test.dart
index 14a18d5a0..6bfec0351 100644
--- a/desktop/test/boot_flow_widget_test.dart
+++ b/desktop/test/boot_flow_widget_test.dart
@@ -1,4 +1,5 @@
import 'support/workspace_tools.dart';
+import 'support/resource_picker.dart';
import 'swarm_interactions_test.dart' show chord;
import 'package:flutter/services.dart';
@@ -910,17 +911,18 @@ void main() {
await tester.sendKeyEvent(LogicalKeyboardKey.enter);
await chord(tester, LogicalKeyboardKey.keyN);
await tester.pumpAndSettle();
- // With no machine to open an agent on, the start page's New goes to
- // the Machines panel, with desktop and server instructions one click away.
- expect(find.byKey(const ValueKey('machines-panel')), findsOneWidget);
- await tester.tap(find.text('Add a second machine'));
+ // Creation without a machine opens the same picker as Cmd-M.
+ expect(resourceScope('@'), findsOneWidget);
+ expect(resourceSearch(tester).rows.last.title, 'Add machine');
+ await tester.sendKeyEvent(LogicalKeyboardKey.arrowUp);
+ await tester.sendKeyEvent(LogicalKeyboardKey.enter);
+ await tester.pump();
+ await tester.sendKeyEvent(LogicalKeyboardKey.enter);
await tester.pumpAndSettle();
- expect(
- find.text('1. Open Harness on your other computer.'),
- findsOneWidget,
- );
- expect(find.text('Set up a server…'), findsOneWidget);
- await tester.tap(find.byTooltip('Close Machines'));
+ expect(find.text('Add machine · App'), findsOneWidget);
+ await tester.sendKeyEvent(LogicalKeyboardKey.escape);
+ await tester.sendKeyEvent(LogicalKeyboardKey.escape);
+ await tester.sendKeyEvent(LogicalKeyboardKey.escape);
await tester.pumpAndSettle();
expect(app.panes, isEmpty);
await tester.pumpWidget(const SizedBox());
@@ -1122,7 +1124,7 @@ void main() {
// The panel and its password field receive focus after the first frame.
await tester.pumpAndSettle();
- expect(find.byKey(const ValueKey('machines-panel')), findsOneWidget);
+ expect(resourceScope('@'), findsOneWidget);
expect(
find.byWidgetPredicate(
(widget) =>
@@ -1131,13 +1133,8 @@ void main() {
),
findsNothing,
);
- expect(
- find.byWidgetPredicate(
- (widget) =>
- widget is Icon && widget.semanticLabel == 'remote-mac: Offline',
- ),
- findsOneWidget,
- );
+ expect(resourceSearch(tester).selected?.machineId, machine.machineId);
+ expect(find.text('Offline'), findsWidgets);
expect(find.text('Harness is offline'), findsNothing);
expect(find.text('harness start'), findsNothing);
await tester.pumpWidget(const SizedBox());
@@ -1188,20 +1185,16 @@ void main() {
await openWorkspaceManagement(tester, 'machines');
await tester.pumpAndSettle();
- await tester.ensureVisible(find.text('link-mac'));
+ await selectResource(tester, 'machine:link-machine');
await tester.tap(
- find.byKey(const ValueKey('connect-machine-link-machine')),
+ find.byKey(const ValueKey('resource-action:picker.resource_connect')),
);
// Same popup-transition reasoning as above.
await tester.pumpAndSettle();
- expect(find.byKey(const ValueKey('machines-panel')), findsOneWidget);
+ expect(resourceScope('@'), findsOneWidget);
expect(
- find.byWidgetPredicate(
- (widget) =>
- widget is TextField &&
- widget.decoration?.hintText == 'Harness password',
- ),
+ find.byKey(const ValueKey('remote-password-connect-field')),
findsOneWidget,
);
await tester.pumpWidget(const SizedBox());
@@ -1272,13 +1265,13 @@ void main() {
await openWorkspaceManagement(tester, 'machines');
await tester.pumpAndSettle();
- await tester.ensureVisible(find.text('link-mac'));
+ await selectResource(tester, 'machine:link-machine');
await tester.tap(
- find.byKey(const ValueKey('connect-machine-link-machine')),
+ find.byKey(const ValueKey('resource-action:picker.resource_connect')),
);
await tester.pumpAndSettle();
- expect(find.byKey(const ValueKey('machines-panel')), findsOneWidget);
+ expect(resourceScope('@'), findsOneWidget);
// No second pane was opened for the popup — just the one terminal pane that was
// already there.
expect(app.allPanes, hasLength(1));
diff --git a/desktop/test/browser_machine_actions_test.dart b/desktop/test/browser_machine_actions_test.dart
new file mode 100644
index 000000000..95cb2dc85
--- /dev/null
+++ b/desktop/test/browser_machine_actions_test.dart
@@ -0,0 +1,251 @@
+import 'package:flutter/material.dart';
+import 'package:flutter/foundation.dart' show kIsWeb;
+import 'package:flutter/services.dart';
+import 'package:dio/dio.dart';
+import 'package:flutter_test/flutter_test.dart';
+import 'package:lucide_icons_flutter/lucide_icons.dart';
+import 'package:harness/auth/auth_session.dart';
+import 'package:harness/core/config.dart';
+import 'package:harness/core/models.dart';
+import 'package:harness/models/api_connections_controller.dart';
+import 'package:harness/models/model_manager_controller.dart';
+import 'package:harness/orchestrator/orchestrator_launcher.dart';
+import 'package:harness/state/app_state.dart';
+import 'package:harness/viewer/viewer_key_store.dart';
+import 'package:harness/viewer/viewer_services.dart';
+import 'package:harness/ws/ws_conn.dart';
+import 'package:harness/widgets/add_phone_dialog.dart';
+
+import 'swarm_state_test.dart' show MemoryStore;
+
+class _Connection extends WsConn {
+ _Connection(String id)
+ : super(
+ wsBaseUrl: 'ws://fixture.invalid',
+ autonomousEnv: 'test',
+ machineId: id,
+ accessTokenProvider: (_, _) async => '',
+ onAuthFailure: (_) {},
+ onEvent: (_) {},
+ onStatus: (_) {},
+ );
+ bool ready = true;
+ Object? failure;
+ final requests = <({String type, Map<String, dynamic> payload})>[];
+ @override
+ bool get isReady => ready;
+ @override
+ Future<Map<String, dynamic>> request(
+ String type, {
+ Map<String, dynamic> payload = const {},
+ Duration timeout = const Duration(seconds: 20),
+ }) async {
+ requests.add((type: type, payload: payload));
+ if (failure != null) throw failure!;
+ if (type == 'command_bar') return {'selectedId': null, 'suggestions': []};
+ if (type == 'route_send') return {'ok': false, 'reason': 'fixture refused'};
+ if (type == 'api_connections') return {'connections': [], 'presets': []};
+ if (payload['action'] == 'list') return {'projects': []};
+ return {'error': 'FIXTURE_STOP', 'detail': 'Fixture launch recorded'};
+ }
+}
+
+void main() {
+ // Initialize the generated icon library before entering the deep widget-build stack in DDC.
+ setUpAll(() => expect(LucideIcons.sparkles.codePoint, greaterThan(0)));
+ late AppNotifier app;
+ late Map<String, _Connection> connections;
+ setUp(() {
+ final storage = MemoryStore(),
+ session = AuthSession(storage: MemoryStore());
+ connections = {
+ for (final id in ['a', 'b', 'shared']) id: _Connection(id),
+ };
+ app = AppNotifier(
+ config: AppConfig.dev,
+ authSession: session,
+ configStore: null,
+ viewer: ViewerServices(
+ config: AppConfig.dev,
+ session: session,
+ keys: ViewerKeyStore(storage: storage),
+ ),
+ connectionForTest: (id) => connections[id]!,
+ );
+ for (final id in connections.keys) {
+ app.machineStates[id] =
+ MachineState(
+ Machine(
+ machineId: id,
+ authMode: MachineAuthMode.remote,
+ name: 'Computer $id',
+ isShared: id == 'shared',
+ ),
+ )
+ ..connectionStatus = ConnectionStatus.connected
+ ..nodeOnline = true;
+ }
+ app.selectedMachineId = 'b';
+ });
+ tearDown(() async {
+ app.dispose();
+ for (final connection in connections.values) {
+ await connection.close();
+ }
+ });
+
+ test(
+ 'browser tools select a linked owner and exclude shared or offline hosts',
+ () {
+ expect(app.ownedActionMachine?.machine.machineId, 'b');
+ app.machineStates['b']!.needsLink = true;
+ expect(app.ownedActionMachine?.machine.machineId, 'a');
+ app.machineStates['a']!.nodeOnline = false;
+ app.selectedMachineId = 'shared';
+ expect(app.ownedActionMachine, isNull);
+ },
+ );
+
+ test('API edits and model controls keep their initial destination', () async {
+ final apis = ApiConnectionsController(app);
+ final models = ModelManagerController(app, poll: false);
+ addTearDown(apis.dispose);
+ addTearDown(models.dispose);
+ expect(apis.hostLabel, 'Computer b');
+ expect(models.machine?.machine.machineId, 'b');
+ app.selectedMachineId = 'a';
+ app.notifyListeners();
+ expect(
+ await apis.save({'provider': 'fal', 'apiKey': 'fixture-only'}),
+ isTrue,
+ );
+ expect(models.machine?.machine.machineId, 'b');
+ expect(connections['a']!.requests, isEmpty);
+ expect(connections['b']!.requests.single.type, 'api_connections');
+ app.machineStates['b']!.connectionStatus = ConnectionStatus.disconnected;
+ app.notifyListeners();
+ expect(await apis.save({'provider': 'fal', 'apiKey': 'not-sent'}), isFalse);
+ expect(connections['b']!.requests, hasLength(1));
+ expect(apis.hostLabel, 'Computer b');
+ });
+
+ test(
+ 'reopening models selects the current host without replacing API listeners',
+ () {
+ final models = ModelManagerController(app, poll: false);
+ addTearDown(models.dispose);
+ models.setPanelVisible(true);
+ final apis = models.apis;
+ app.selectedMachineId = 'a';
+ models.setPanelVisible(true);
+ expect(models.targetMachineId, 'b');
+ models.setPanelVisible(false);
+ models.setPanelVisible(true);
+ expect(models.targetMachineId, 'a');
+ expect(models.apis, same(apis));
+ expect(apis.hostLabel, 'Computer a');
+ },
+ );
+
+ test('API and orchestrator mutations are never queued on an unready or shared connection', () async {
+ connections['b']!.ready = false;
+ for (final id in ['b', 'shared']) {
+ await expectLater(
+ app.apiConnections(id, {'action': 'save'}),
+ throwsStateError,
+ );
+ await expectLater(
+ app.orchestratorRequest(id, {'action': 'start'}),
+ throwsStateError,
+ );
+ expect(connections[id]!.requests, isEmpty);
+ }
+ });
+
+ test('phone relay preserves protocol failures instead of reporting every reply as offline', () async {
+ for (final code in [
+ 'NO_INTENT',
+ 'EXPIRED',
+ 'BUSY',
+ 'CODE_MISMATCH',
+ 'RATE_LIMITED',
+ 'TIMEOUT',
+ 'UNSUPPORTED',
+ ]) {
+ connections['b']!.failure = WsRequestFailure(
+ responseType: 'phone_pair_result',
+ code: code,
+ );
+ final answer = await phonePairOverRelay(app, 'b')(
+ 'ABCDEFGHJKMNPQRS',
+ CancelToken(),
+ );
+ expect(
+ answer.error,
+ code == 'UNSUPPORTED' ? PhonePairAnswer.unavailable : code,
+ );
+ }
+ });
+
+ test('browser commands use the linked host and never send after cancellation or to a shared host', () async {
+ expect(
+ await app.resolveCommandBar({
+ 'prompt': 'fixture',
+ }, cancelToken: CancelToken()),
+ {'selectedId': null, 'suggestions': []},
+ );
+ expect(connections['b']!.requests.single.type, 'command_bar');
+ final cancelled = CancelToken()..cancel();
+ await expectLater(
+ app.resolveCommandBar({'prompt': 'cancelled'}, cancelToken: cancelled),
+ throwsA(isA<DioException>()),
+ );
+ expect(connections['b']!.requests, hasLength(1));
+ expect(
+ await app.sendRoutedTask('a', 'shared', 'not sent'),
+ contains('Reconnect'),
+ );
+ expect(connections['shared']!.requests, isEmpty);
+ expect(
+ await app.sendRoutedTask('a', 'b', 'fixture'),
+ contains('fixture refused'),
+ );
+ expect(connections['b']!.requests.last.type, 'route_send');
+ });
+
+ testWidgets('orchestrator launch keeps the host shown when it opened', (
+ tester,
+ ) async {
+ tester.view.physicalSize = const Size(1400, 1000);
+ tester.view.devicePixelRatio = 1;
+ addTearDown(tester.view.resetPhysicalSize);
+ addTearDown(tester.view.resetDevicePixelRatio);
+ await tester.pumpWidget(
+ MaterialApp(home: OrchestratorLauncher(notifier: app)),
+ );
+ await tester.pumpAndSettle();
+ expect(find.textContaining('Computer b'), findsWidgets);
+ app.selectedMachineId = 'a';
+ app.notifyListeners();
+ await tester.enterText(
+ find.byKey(const ValueKey('orchestrator-prompt')),
+ 'Fixture project',
+ );
+ await tester.pump();
+ if (kIsWeb) {
+ await tester.sendKeyDownEvent(LogicalKeyboardKey.altLeft);
+ await tester.sendKeyEvent(LogicalKeyboardKey.enter);
+ await tester.sendKeyUpEvent(LogicalKeyboardKey.altLeft);
+ } else {
+ await tester.tap(find.byKey(const ValueKey('orchestrator-start')));
+ }
+ await tester.pumpAndSettle();
+ expect(connections['a']!.requests, isEmpty);
+ expect(connections['b']!.requests.map((r) => r.payload['action']), [
+ 'list',
+ 'start',
+ ]);
+ expect(find.textContaining('Fixture launch recorded'), findsOneWidget);
+ await tester.pumpWidget(const SizedBox());
+ });
+}
diff --git a/desktop/test/environment_recheck_timer_test.dart b/desktop/test/environment_recheck_timer_test.dart
index cf9fcb53d..801ca5b4f 100644
--- a/desktop/test/environment_recheck_timer_test.dart
+++ b/desktop/test/environment_recheck_timer_test.dart
@@ -1,17 +1,10 @@
import 'package:flutter_test/flutter_test.dart';
-import 'package:harness/auth/auth_session.dart';
-import 'package:harness/auth/cli_login.dart';
import 'package:harness/bootstrap/environment_provisioner.dart';
-import 'package:harness/core/config.dart';
import 'package:harness/core/local_key_value_store.dart';
import 'package:harness/settings/config_store.dart';
import 'package:harness/state/app_state.dart';
-class _FakeCliLogin extends CliLogin {
- @override
- Future<CliAuthStatus> checkStatus() async =>
- const CliAuthStatus(loggedIn: false);
-}
+import 'environment_setup_screen_test.dart' show SetupWorkspaceApp;
class _FakeKeyValueStore implements LocalKeyValueStore {
final Map<String, String> values = {};
@@ -97,13 +90,7 @@ void main() {
),
);
final provisioner = _ScriptedProvisioner([review, waiting, failed]);
- final app = AppNotifier(
- config: AppConfig.dev,
- authSession: AuthSession(),
- configStore: null,
- cliLogin: _FakeCliLogin(),
- environmentProvisioner: provisioner,
- );
+ final app = SetupWorkspaceApp(provisioner);
await app.bootstrap();
await app.startEnvironmentSetup();
await app.recheckEnvironmentStep(EnvironmentStep.tmux);
@@ -140,12 +127,9 @@ void main() {
mode: EnvironmentSetupMode.automatic,
);
final provisioner = _ScriptedProvisioner([review, waiting, ready]);
- final app = AppNotifier(
- config: AppConfig.dev,
- authSession: AuthSession(),
+ final app = SetupWorkspaceApp(
+ provisioner,
configStore: ConfigStore(storage: _FakeKeyValueStore()),
- cliLogin: _FakeCliLogin(),
- environmentProvisioner: provisioner,
);
await app.bootstrap();
@@ -160,7 +144,9 @@ void main() {
expect(provisioner.installCalls, [false, true, false]);
expect(app.environmentReadiness.isReady, isTrue);
expect(app.environmentRecheckPending, isFalse);
- expect(app.status, AppStatus.unauthenticated);
+ await tester.pumpAndSettle();
+ expect(app.status, AppStatus.authenticated);
+ expect(app.isGuest, isTrue);
app.dispose();
});
}
diff --git a/desktop/test/environment_setup_screen_test.dart b/desktop/test/environment_setup_screen_test.dart
index 248efac1f..f4d09a6d3 100644
--- a/desktop/test/environment_setup_screen_test.dart
+++ b/desktop/test/environment_setup_screen_test.dart
@@ -11,6 +11,8 @@ import 'package:harness/shared/theme/app_theme.dart' as grid;
import 'package:harness/state/app_state.dart';
import 'package:harness/widgets/environment_setup_screen.dart';
+import 'swarm_state_test.dart' show MemoryStore;
+
const setupReview = EnvironmentReadiness(
steps: {
EnvironmentStep.clipboard: EnvironmentStepStatus.notApplicable,
@@ -28,6 +30,21 @@ class SetupLogin extends CliLogin {
const CliAuthStatus(loggedIn: false);
}
+/// Installer tests stop at the workspace boundary; they must never start a real daemon.
+class SetupWorkspaceApp extends AppNotifier {
+ SetupWorkspaceApp(EnvironmentProvisioner provisioner, {super.configStore})
+ : super(
+ config: AppConfig.dev,
+ authSession: AuthSession(storage: MemoryStore()),
+ cliLogin: SetupLogin(),
+ environmentProvisioner: provisioner,
+ );
+ @override
+ Future<void> ensureCliDaemonReady() async {}
+ @override
+ Future<bool> refreshMachines() async => true;
+}
+
class SetupAttempt {
SetupAttempt(this.install, this.progress);
final bool install;
@@ -84,8 +101,8 @@ Future<void> _mount(
),
home: ListenableBuilder(
listenable: app,
- builder: (_, _) => app.status == AppStatus.unauthenticated
- ? const Scaffold(body: Text('Sign-in reached'))
+ builder: (_, _) => app.status == AppStatus.authenticated
+ ? const Scaffold(body: Text('Guest workspace reached'))
: EnvironmentSetupScreen(notifier: app),
),
),
@@ -93,16 +110,9 @@ Future<void> _mount(
await tester.pump();
}
-AppNotifier _app(SetupProvisioner provisioner) =>
- AppNotifier(
- config: AppConfig.dev,
- authSession: AuthSession(),
- configStore: null,
- cliLogin: SetupLogin(),
- environmentProvisioner: provisioner,
- )
- ..status = AppStatus.preparingEnvironment
- ..environmentReadiness = setupReview;
+AppNotifier _app(SetupProvisioner provisioner) => SetupWorkspaceApp(provisioner)
+ ..status = AppStatus.preparingEnvironment
+ ..environmentReadiness = setupReview;
void main() {
testWidgets('Retry after a launch check failure only checks the computer', (
@@ -353,52 +363,55 @@ void main() {
});
}
- testWidgets('Enter installs and retries once before reaching sign-in', (
- tester,
- ) async {
- final provisioner = SetupProvisioner();
- final app = _app(provisioner);
- await _mount(tester, app);
- expect(provisioner.attempts, isEmpty);
- await tester.sendKeyEvent(LogicalKeyboardKey.enter);
- await tester.pump();
- expect(provisioner.attempts, hasLength(1));
- expect(provisioner.attempts.single.install, isTrue);
- await tester.sendKeyEvent(LogicalKeyboardKey.enter);
- await tester.pump();
- expect(provisioner.attempts, hasLength(1));
- provisioner.attempts.single.finish(
- setupReview.copyWith(
- phase: EnvironmentSetupPhase.failed,
- failure: const EnvironmentFailure(
- title: 'Could not install Harness',
- detail: 'Check your connection, then retry setup.',
+ testWidgets(
+ 'Enter installs and retries once before reaching the guest workspace',
+ (tester) async {
+ final provisioner = SetupProvisioner();
+ final app = _app(provisioner);
+ await _mount(tester, app);
+ expect(provisioner.attempts, isEmpty);
+ await tester.sendKeyEvent(LogicalKeyboardKey.enter);
+ await tester.pump();
+ expect(provisioner.attempts, hasLength(1));
+ expect(provisioner.attempts.single.install, isTrue);
+ await tester.sendKeyEvent(LogicalKeyboardKey.enter);
+ await tester.pump();
+ expect(provisioner.attempts, hasLength(1));
+ provisioner.attempts.single.finish(
+ setupReview.copyWith(
+ phase: EnvironmentSetupPhase.failed,
+ failure: const EnvironmentFailure(
+ title: 'Could not install Harness',
+ detail: 'Check your connection, then retry setup.',
+ ),
),
- ),
- );
- await tester.pump();
- await tester.pump();
- await tester.sendKeyEvent(LogicalKeyboardKey.enter);
- await tester.pump();
- expect(provisioner.attempts, hasLength(2));
- provisioner.attempts.last.finish(
- const EnvironmentReadiness(
- steps: {
- EnvironmentStep.clipboard: EnvironmentStepStatus.notApplicable,
- EnvironmentStep.tmux: EnvironmentStepStatus.ready,
- EnvironmentStep.harness: EnvironmentStepStatus.ready,
- },
- phase: EnvironmentSetupPhase.ready,
- ),
- );
- await tester.pump();
- await tester.pump();
- expect(find.text('Sign-in reached'), findsOneWidget);
- expect(provisioner.attempts.map((attempt) => attempt.install), [
- true,
- true,
- ]);
- await tester.pumpWidget(const SizedBox());
- app.dispose();
- });
+ );
+ await tester.pump();
+ await tester.pump();
+ await tester.sendKeyEvent(LogicalKeyboardKey.enter);
+ await tester.pump();
+ expect(provisioner.attempts, hasLength(2));
+ provisioner.attempts.last.finish(
+ const EnvironmentReadiness(
+ steps: {
+ EnvironmentStep.clipboard: EnvironmentStepStatus.notApplicable,
+ EnvironmentStep.tmux: EnvironmentStepStatus.ready,
+ EnvironmentStep.harness: EnvironmentStepStatus.ready,
+ },
+ phase: EnvironmentSetupPhase.ready,
+ ),
+ );
+ await tester.pump();
+ await tester.pump();
+ await tester.pumpAndSettle();
+ expect(find.text('Guest workspace reached'), findsOneWidget);
+ expect(app.isGuest, isTrue);
+ expect(provisioner.attempts.map((attempt) => attempt.install), [
+ true,
+ true,
+ ]);
+ await tester.pumpWidget(const SizedBox());
+ app.dispose();
+ },
+ );
}
diff --git a/desktop/test/first_workspace_test.dart b/desktop/test/first_workspace_test.dart
index 7c4d2ff08..031329b87 100644
--- a/desktop/test/first_workspace_test.dart
+++ b/desktop/test/first_workspace_test.dart
@@ -529,7 +529,7 @@ void main() {
find.byKey(const ValueKey('swarm-new-pane-button')),
findsNothing,
);
- expect(find.byTooltip('Harnesses'), findsOneWidget);
+ expect(find.byTooltip('Harnesses'), findsNothing);
expect(
find.byKey(const ValueKey('swarm-new-harness-button')),
findsNothing,
diff --git a/desktop/test/grid_model_picker_test.dart b/desktop/test/grid_model_picker_test.dart
index abe8f42e7..1f08a36bd 100644
--- a/desktop/test/grid_model_picker_test.dart
+++ b/desktop/test/grid_model_picker_test.dart
@@ -296,15 +296,12 @@ void main() {
of: control,
matching: find.byType(ColoredBox),
);
- expect(fill, findsOneWidget);
- expect(tester.getSize(fill).height, 28);
expect(
- tester.widget<ColoredBox>(fill).color,
- tester
- .widget<WorkspaceBarControl>(control)
- .selection
- .withValues(alpha: .5),
+ fill,
+ findsNothing,
+ reason: 'Status controls emphasize text on hover without a fill.',
);
+ expect(tester.getSize(control).height, 28);
await hover.moveTo(Offset.zero);
await tester.pumpAndSettle();
await mount('GPT-5.6 Sol');
diff --git a/desktop/test/interactive_viewer_test.dart b/desktop/test/interactive_viewer_test.dart
new file mode 100644
index 000000000..0d21e5d40
--- /dev/null
+++ b/desktop/test/interactive_viewer_test.dart
@@ -0,0 +1,118 @@
+import 'dart:async';
+import 'dart:convert';
+
+import 'package:flutter/material.dart';
+import 'package:flutter_test/flutter_test.dart';
+import 'package:harness/viewer/interactive_viewer.dart';
+
+Map<String, dynamic> picture() => {
+ 'data': base64Encode([1, 2, 3]),
+ 'mime': 'image/jpeg',
+ 'width': 800,
+ 'height': 600,
+};
+
+void main() {
+ testWidgets('one frame in flight; queued input arrives once and in order', (
+ tester,
+ ) async {
+ final requests = <Map<String, dynamic>>[];
+ final replies = <Completer<Map<String, dynamic>>>[];
+ final session = InteractiveViewerSession((payload) {
+ requests.add(payload);
+ if (payload['op'] == 'close') return Future.value({'closed': true});
+ final reply = Completer<Map<String, dynamic>>();
+ replies.add(reply);
+ return reply.future;
+ });
+ session.configure(const Size(800, 600), true);
+ await tester.pump(const Duration(milliseconds: 1));
+ expect(requests, hasLength(1));
+ await tester.pump(const Duration(seconds: 2));
+ expect(requests, hasLength(1));
+ replies[0].complete(picture());
+ await tester.pump(const Duration(milliseconds: 1));
+ expect(session.image, [1, 2, 3]);
+ session.input({'type': 'text', 'text': 'first'});
+ await tester.pump(const Duration(milliseconds: 1));
+ session.input({'type': 'text', 'text': 'second'});
+ expect(requests, hasLength(2));
+ expect(requests[1]['events'], [
+ {'type': 'text', 'text': 'first'},
+ ]);
+ replies[1].complete(picture());
+ await tester.pump(const Duration(milliseconds: 1));
+ await tester.pump(const Duration(milliseconds: 1));
+ expect(requests, hasLength(3));
+ expect(requests[2]['events'], [
+ {'type': 'text', 'text': 'second'},
+ ]);
+ session.dispose();
+ expect(requests.last['op'], 'close');
+ expect(requests.last['surfaceId'], requests.first['surfaceId']);
+ replies[2].complete(picture());
+ await tester.pump(const Duration(milliseconds: 1));
+ expect(tester.takeException(), isNull);
+ });
+
+ testWidgets('an interrupted request never replays stale input on Retry', (
+ tester,
+ ) async {
+ final requests = <Map<String, dynamic>>[];
+ var fail = false;
+ final session = InteractiveViewerSession((payload) async {
+ requests.add(payload);
+ if (fail) throw StateError('disconnected');
+ return picture();
+ });
+ session.configure(const Size(800, 600), false);
+ await tester.pump(const Duration(milliseconds: 1));
+ fail = true;
+ session.input({'type': 'text', 'text': 'do not replay'});
+ await tester.pump(const Duration(milliseconds: 1));
+ expect(session.error, contains('disconnected'));
+ final before = requests.length;
+ await tester.pump(const Duration(seconds: 10));
+ expect(requests, hasLength(before));
+ fail = false;
+ session.reload();
+ await tester.pump(const Duration(milliseconds: 1));
+ expect(requests.last['events'], isEmpty);
+ expect(requests.last['reload'], true);
+ expect(session.error, isNull);
+ session.dispose();
+ });
+
+ testWidgets(
+ 'bounds geometry, coalesces motion, and stops after invalid replies',
+ (tester) async {
+ final requests = <Map<String, dynamic>>[];
+ var invalid = false;
+ final session = InteractiveViewerSession((payload) async {
+ requests.add(payload);
+ return invalid ? {'data': 'bad!', 'mime': 'image/jpeg'} : picture();
+ });
+ session.configure(const Size(9000, 40), true);
+ await tester.pump(const Duration(milliseconds: 1));
+ expect(requests.first['width'], 1920);
+ expect(requests.first['height'], 120);
+ for (var i = 0; i < 100; i++) {
+ session.input({'type': 'pointer', 'event': 'mouseMoved', 'x': i});
+ }
+ session.input({'type': 'pointer', 'event': 'mousePressed'});
+ session.input({'type': 'pointer', 'event': 'mouseMoved', 'x': 100});
+ invalid = true;
+ await tester.pump(const Duration(milliseconds: 1));
+ expect(requests.last['events'], [
+ {'type': 'pointer', 'event': 'mouseMoved', 'x': 99},
+ {'type': 'pointer', 'event': 'mousePressed'},
+ {'type': 'pointer', 'event': 'mouseMoved', 'x': 100},
+ ]);
+ expect(session.error, isNotNull);
+ final before = requests.length;
+ await tester.pump(const Duration(seconds: 10));
+ expect(requests, hasLength(before));
+ session.dispose();
+ },
+ );
+}
diff --git a/desktop/test/local_cli_discovery_test.dart b/desktop/test/local_cli_discovery_test.dart
index a4c46d862..577397703 100644
--- a/desktop/test/local_cli_discovery_test.dart
+++ b/desktop/test/local_cli_discovery_test.dart
@@ -767,11 +767,9 @@ void main() {
expect(spawnCount, lessThan(6));
});
- // The bug this closes: a daemon that signed ITSELF out (its machine was deleted from another
- // machine) exits, and the supervisor respawned it forever — every replacement starting without a
- // session and exiting again, silently, for the app's whole lifetime.
+ // Signed-out daemons now serve local agents in guest mode, so supervision must continue.
test(
- 'startSupervising stops respawning once the CLI reports it is signed out',
+ 'startSupervising keeps the guest daemon alive after sign-out',
() async {
const computerId = '0123456789abcdef0123456789abcdef';
final identityFile = File('${scratch.path}/computer-id')
@@ -806,13 +804,18 @@ void main() {
await Future.delayed(const Duration(milliseconds: 300));
+ expect(spawnCount, greaterThan(0));
expect(
spawnCount,
- 0,
- reason: 'a signed-out daemon must never be respawned',
+ lessThan(6),
+ reason: 'guest restarts retain the backoff',
+ );
+ expect(
+ signedOutCalls,
+ spawnCount,
+ reason: 'auth is checked once per spawn attempt',
);
- expect(signedOutCalls, 1, reason: 'the caller is told exactly once');
- expect(timer.isActive, isFalse, reason: 'supervision stops for good');
+ expect(timer.isActive, isTrue);
},
);
diff --git a/desktop/test/machines_manager_test.dart b/desktop/test/machines_manager_test.dart
index 593620a81..9d0bac97b 100644
--- a/desktop/test/machines_manager_test.dart
+++ b/desktop/test/machines_manager_test.dart
@@ -105,7 +105,7 @@ void main() {
await openWorkspaceManagement(tester, 'machines');
await tester.pumpAndSettle();
expect(find.byType(HarnessSessionManager), findsNothing);
- expect(find.byKey(const ValueKey('machines-panel')), findsOneWidget);
+ expect(resourceScope('@'), findsOneWidget);
await openWorkspaceManagement(tester, 'models');
await tester.pumpAndSettle();
expect(find.byKey(const ValueKey('machines-panel')), findsNothing);
@@ -414,8 +414,8 @@ void main() {
await tester.pumpAndSettle();
}
- expect(find.byKey(const ValueKey('machines-panel')), findsOneWidget);
- expect(find.text('Add a second machine'), findsOneWidget);
+ expect(resourceScope('@'), findsOneWidget);
+ expect(resourceSearch(tester).rows.last.title, 'Add machine');
expect(app.activeSwarm, same(tab));
expect(app.panes, isEmpty);
expect(tester.takeException(), isNull);
diff --git a/desktop/test/open_picker_rendering_test.dart b/desktop/test/open_picker_rendering_test.dart
index 62642c24a..7ac32998e 100644
--- a/desktop/test/open_picker_rendering_test.dart
+++ b/desktop/test/open_picker_rendering_test.dart
@@ -296,91 +296,101 @@ void main() {
},
);
- testWidgets('Open Harness builds a small window and Tab reaches later rows', (
- tester,
- ) async {
- final app = createApp();
- app.machineStates['m']!.nodeOnline = true;
- app.adoptSessionForTest(terminal('a0', []));
- final map = MemoryKeymap();
- await mount(tester, app, map);
- await openHarnessPicker(tester);
- final results = find.byType(SwarmSearchResults);
- final search = tester.widget<SwarmSearchResults>(results).search;
- final rowIds = search.rows.map((row) => row.id).toSet();
- final rows = find.descendant(
- of: results,
- matching: find.byWidgetPredicate(
- (widget) =>
- widget is InkWell &&
- widget.key is ValueKey<String> &&
- rowIds.contains((widget.key! as ValueKey<String>).value),
- ),
- );
- final input = find.byKey(const ValueKey('swarm-search-input'));
- final cell = terminalCellSizeOf(tester.element(input));
- for (final row in rows.evaluate()) {
- final id = (row.widget.key! as ValueKey<String>).value;
- final line = find.byKey(ValueKey('swarm-search-line:$id'));
- expect(
- tester.getSize(line).height,
- closeTo(cell.height, .01),
- reason: 'Selection occupies exactly one terminal line.',
+ testWidgets(
+ 'Open Harness builds a small window and Down reaches later rows',
+ (tester) async {
+ final app = createApp();
+ app.machineStates['m']!.nodeOnline = true;
+ app.adoptSessionForTest(terminal('a0', []));
+ final map = MemoryKeymap();
+ await mount(tester, app, map);
+ await openHarnessPicker(tester);
+ final results = find.byType(SwarmSearchResults);
+ final search = tester.widget<SwarmSearchResults>(results).search;
+ final rowIds = search.rows.map((row) => row.id).toSet();
+ final rows = find.descendant(
+ of: results,
+ matching: find.byWidgetPredicate(
+ (widget) =>
+ widget is InkWell &&
+ widget.key is ValueKey<String> &&
+ rowIds.contains((widget.key! as ValueKey<String>).value),
+ ),
);
+ final input = find.byKey(const ValueKey('swarm-search-input'));
+ final cell = terminalCellSizeOf(tester.element(input));
+ for (final row in rows.evaluate()) {
+ final id = (row.widget.key! as ValueKey<String>).value;
+ final line = find.byKey(ValueKey('swarm-search-line:$id'));
+ expect(
+ tester.getSize(line).height,
+ closeTo(cell.height, .01),
+ reason: 'Selection occupies exactly one terminal line.',
+ );
+ expect(
+ tester.getSize(find.byKey(ValueKey(id))).height,
+ closeTo(cell.height, .01),
+ reason: 'The entire result occupies one line, with no spacer row.',
+ );
+ final title = find
+ .descendant(of: line, matching: find.byType(Text))
+ .first;
+ expect(
+ tester.getTopLeft(title).dx,
+ closeTo(tester.getTopLeft(input).dx, .01),
+ );
+ }
+ final first = find.byKey(ValueKey(search.rows[0].id));
+ final second = find.byKey(ValueKey(search.rows[1].id));
expect(
- tester.getSize(find.byKey(ValueKey(id))).height,
+ tester.getTopLeft(second).dy - tester.getTopLeft(first).dy,
closeTo(cell.height, .01),
- reason: 'The entire result occupies one line, with no spacer row.',
);
- final title = find
- .descendant(of: line, matching: find.byType(Text))
- .first;
- expect(
- tester.getTopLeft(title).dx,
- closeTo(tester.getTopLeft(input).dx, .01),
- );
- }
- final first = find.byKey(ValueKey(search.rows[0].id));
- final second = find.byKey(ValueKey(search.rows[1].id));
- expect(
- tester.getTopLeft(second).dy - tester.getTopLeft(first).dy,
- closeTo(cell.height, .01),
- );
- for (final type in [ListTile, Icon, Image, EngineMark, PromptContextView]) {
- expect(
- find.descendant(of: results, matching: find.byType(type)),
- findsNothing,
- );
- }
- expect(search.rows.length, greaterThan(50));
- final list = find.byKey(const ValueKey('swarm-search-result-list'));
- final capacity = (tester.getSize(list).height / cell.height).ceil();
- expect(rows.evaluate().length, lessThanOrEqualTo(capacity + 2));
- expect(rows.evaluate().length, lessThan(search.rows.length));
- final visited = <String>{};
- for (var step = 0; step < 35; step++) {
- await key(tester, LogicalKeyboardKey.tab);
- await tester.pumpAndSettle();
- final selected = search.selected;
- if (selected != null) {
- visited.add(selected.id);
- expect(find.byKey(ValueKey(selected.id)).hitTestable(), findsOneWidget);
+ for (final type in [
+ ListTile,
+ Icon,
+ Image,
+ EngineMark,
+ PromptContextView,
+ ]) {
expect(
- tester
- .widget<TextField>(
- find.byKey(const ValueKey('swarm-search-input')),
- )
- .focusNode!
- .hasFocus,
- isTrue,
+ find.descendant(of: results, matching: find.byType(type)),
+ findsNothing,
);
}
- }
- // Focus crosses the initial viewport repeatedly as new rows are built.
- expect(visited.length, greaterThan(15));
- expect(tester.takeException(), isNull);
- await tester.pumpWidget(const SizedBox());
- app.dispose();
- map.dispose();
- });
+ expect(search.rows.length, greaterThan(50));
+ final list = find.byKey(const ValueKey('swarm-search-result-list'));
+ final capacity = (tester.getSize(list).height / cell.height).ceil();
+ expect(rows.evaluate().length, lessThanOrEqualTo(capacity + 2));
+ expect(rows.evaluate().length, lessThan(search.rows.length));
+ final visited = <String>{};
+ for (var step = 0; step < 35; step++) {
+ await key(tester, LogicalKeyboardKey.arrowDown);
+ await tester.pumpAndSettle();
+ final selected = search.selected;
+ if (selected != null) {
+ visited.add(selected.id);
+ expect(
+ find.byKey(ValueKey(selected.id)).hitTestable(),
+ findsOneWidget,
+ );
+ expect(
+ tester
+ .widget<TextField>(
+ find.byKey(const ValueKey('swarm-search-input')),
+ )
+ .focusNode!
+ .hasFocus,
+ isTrue,
+ );
+ }
+ }
+ // Focus crosses the initial viewport repeatedly as new rows are built.
+ expect(visited.length, greaterThan(15));
+ expect(tester.takeException(), isNull);
+ await tester.pumpWidget(const SizedBox());
+ app.dispose();
+ map.dispose();
+ },
+ );
}
diff --git a/desktop/test/orchestrator_controls_test.dart b/desktop/test/orchestrator_controls_test.dart
index a682e8681..6ec623b8c 100644
--- a/desktop/test/orchestrator_controls_test.dart
+++ b/desktop/test/orchestrator_controls_test.dart
@@ -24,6 +24,8 @@ class _Connection extends WsConn {
onEvent: (_) {},
onStatus: (_) {},
);
+ @override
+ bool get isReady => true;
final OrchestratorRequest reply;
@override
Future<Map<String, dynamic>> request(
@@ -66,6 +68,7 @@ void main() {
(tester) async {
largeSurface(tester);
final app = createApp(
+ connected: true,
connectionForTest: (_) => _Connection((p) async {
if (p['action'] == 'list') throw StateError('Saved projects offline');
return {
@@ -113,6 +116,7 @@ void main() {
(tester) async {
largeSurface(tester);
final app = createApp(
+ connected: true,
connectionForTest: (_) => _Connection(
(p) async => p['action'] == 'list'
? {'projects': []}
@@ -293,7 +297,10 @@ void main() {
'project': {...project(), 'prompt': p['prompt']},
};
});
- final app = createApp(connectionForTest: (_) => connection);
+ final app = createApp(
+ connected: true,
+ connectionForTest: (_) => connection,
+ );
app.machineStates['m']!.localOnly = true;
addTearDown(app.dispose);
await tester.pumpWidget(
@@ -362,6 +369,7 @@ void main() {
largeSurface(tester);
final requests = <String>[];
final app = createApp(
+ connected: true,
connectionForTest: (_) => _Connection((p) async {
requests.add(p['action'] as String);
return {
@@ -410,7 +418,7 @@ void main() {
await tester.tap(find.byKey(const ValueKey('orchestrator-start')));
await tester.pump();
expect(
- find.text('Connect this computer’s Harness daemon first.'),
+ find.text('Connect a machine before starting a project.'),
findsOneWidget,
);
await tester.pumpWidget(const SizedBox());
diff --git a/desktop/test/orchestrator_test.dart b/desktop/test/orchestrator_test.dart
index ba0521c6a..c35f87e16 100644
--- a/desktop/test/orchestrator_test.dart
+++ b/desktop/test/orchestrator_test.dart
@@ -57,6 +57,8 @@ class _LaunchConnection extends WsConn {
onEvent: (_) {},
onStatus: (_) {},
);
+ @override
+ bool get isReady => true;
final Exception failure;
final starts = <Map<String, dynamic>>[];
@override
@@ -85,7 +87,10 @@ void main() {
)
: const WsRequestTimeout('orchestrator'),
);
- final app = createApp(connectionForTest: (_) => connection);
+ final app = createApp(
+ connected: true,
+ connectionForTest: (_) => connection,
+ );
app.machineStates['m']!.localOnly = true;
addTearDown(app.dispose);
await tester.pumpWidget(
@@ -119,25 +124,28 @@ void main() {
);
}
- test('Cmd-P opens a pane and the orchestrator remains a palette command', () {
- final shortcut = appShortcuts()
- .singleWhere((s) => s.action == ShortcutAction.addAgent)
- .activator;
- expect(shortcut.trigger, LogicalKeyboardKey.keyP);
- expect(shortcut.meta, isTrue);
- expect(shortcut.shift, isFalse);
- expect(
- kAppShortcuts.where((s) => s.action == ShortcutAction.orchestrate),
- isEmpty,
- );
- expect(
- kAppShortcuts
- .singleWhere((s) => s.action == ShortcutAction.routeTask)
- .activator
- .trigger,
- LogicalKeyboardKey.keyB,
- );
- });
+ test(
+ 'Cmd-O opens projects and the orchestrator remains a palette command',
+ () {
+ final shortcut = appShortcuts()
+ .singleWhere((s) => s.action == ShortcutAction.addAgent)
+ .activator;
+ expect(shortcut.trigger, LogicalKeyboardKey.keyO);
+ expect(shortcut.meta, isTrue);
+ expect(shortcut.shift, isFalse);
+ expect(
+ kAppShortcuts.where((s) => s.action == ShortcutAction.orchestrate),
+ isEmpty,
+ );
+ expect(
+ kAppShortcuts
+ .singleWhere((s) => s.action == ShortcutAction.routeTask)
+ .activator
+ .trigger,
+ LogicalKeyboardKey.keyB,
+ );
+ },
+ );
test(
'transient chat failure retries the same receipt and retains the draft',
diff --git a/desktop/test/signout_recovery_test.dart b/desktop/test/signout_recovery_test.dart
index 022e1f37a..39b30e3be 100644
--- a/desktop/test/signout_recovery_test.dart
+++ b/desktop/test/signout_recovery_test.dart
@@ -13,6 +13,8 @@ import 'package:harness/screens/login_screen.dart';
import 'package:harness/shared/theme/app_theme.dart' as grid;
import 'package:harness/state/app_state.dart';
import 'package:harness/terminal/terminal_session.dart';
+import 'package:harness/viewer/viewer_services.dart';
+import 'package:harness/viewer/viewer_key_store.dart';
import 'support/real_fonts.dart';
import 'swarm_state_test.dart' show MemoryStore;
@@ -63,6 +65,11 @@ AppNotifier signOutApp(SignOutFixture cli, {bool local = false}) => AppNotifier(
configStore: null,
authSession: AuthSession(storage: MemoryStore()),
cliLogin: cli,
+ viewer: ViewerServices(
+ config: AppConfig.dev,
+ session: AuthSession(storage: MemoryStore()),
+ keys: ViewerKeyStore(storage: MemoryStore()),
+ ),
localManualFixture: local
? const LocalManualFixture(
apiBaseUrl: 'http://127.0.0.1:1',
diff --git a/desktop/test/terminal_panel_presentation_test.dart b/desktop/test/terminal_panel_presentation_test.dart
index 7820eb5eb..1f5625acc 100644
--- a/desktop/test/terminal_panel_presentation_test.dart
+++ b/desktop/test/terminal_panel_presentation_test.dart
@@ -49,7 +49,12 @@ void main() {
await tester.pump();
revision.value = 1;
await tester.pump();
- for (final label in ['Close Pane', 'Zoom Pane', 'Stop Harness']) {
+ final mouse = await tester.createGesture(kind: PointerDeviceKind.mouse);
+ await mouse.addPointer(location: const Offset(1, 100));
+ await mouse.moveTo(tester.getCenter(find.text(session.agentName)));
+ await tester.pump();
+ expect(find.byTooltip('Close Pane').hitTestable(), findsOneWidget);
+ for (final label in ['Zoom Pane', 'Stop Harness']) {
expect(find.byTooltip(label), findsNothing);
}
expect(closed, isEmpty);
@@ -190,6 +195,7 @@ void main() {
tester.widget<Text>(find.text('Renamed terminal')).style!.fontFamily,
workspaceBarTextStyle().fontFamily,
);
+ await mouse.removePointer();
await tester.pumpWidget(const SizedBox());
revision.dispose();
session.dispose();
@@ -198,7 +204,7 @@ void main() {
);
for (final local in [true, false]) {
testWidgets(
- '${local ? 'local' : 'remote'} compact model selectors stay visible without moving the title or terminal',
+ '${local ? 'local' : 'remote'} compact headers keep the title and close control stable',
(tester) async {
final app = createApp();
app.stateOf('m')!.localOnly = local;
@@ -259,8 +265,13 @@ void main() {
.length,
0,
);
- expect(find.text('OpenAI').hitTestable(), findsOneWidget);
- for (final label in ['Close Pane', 'Zoom Pane', 'Stop Harness']) {
+ expect(
+ find.byType(GridModelPicker),
+ findsNothing,
+ reason: 'The focused model is in the workspace status bar.',
+ );
+ expect(find.byTooltip('Close Pane').hitTestable(), findsNothing);
+ for (final label in ['Zoom Pane', 'Stop Harness']) {
expect(find.byTooltip(label).hitTestable(), findsNothing);
}
final controlsBounds = tester.getRect(controls);
@@ -268,14 +279,15 @@ void main() {
await mouse.addPointer(location: const Offset(1, 100));
await mouse.moveTo(tester.getCenter(title));
await tester.pump();
- for (final label in ['Close Pane', 'Zoom Pane', 'Stop Harness']) {
+ expect(find.byTooltip('Close Pane').hitTestable(), findsOneWidget);
+ for (final label in ['Zoom Pane', 'Stop Harness']) {
expect(find.byTooltip(label), findsNothing);
}
expect(tester.getRect(title), titleBounds);
expect(tester.getRect(controls), controlsBounds);
await mouse.moveTo(tester.getCenter(find.byType(TerminalView)));
await tester.pump();
- for (final label in ['Close Pane', 'Zoom Pane', 'Stop Harness']) {
+ for (final label in ['Zoom Pane', 'Stop Harness']) {
expect(find.byTooltip(label).hitTestable(), findsNothing);
}
expect(find.byTooltip('Share harness'), findsNothing);
@@ -297,7 +309,7 @@ void main() {
);
}
testWidgets(
- 'narrow headers preserve identity and model selection at large text',
+ 'narrow headers preserve identity and close control at large text',
(tester) async {
final app = createApp();
final session = terminal('a0', []);
@@ -344,8 +356,13 @@ void main() {
expect(tester.getSize(title).width, greaterThan(64));
expect(tester.takeException(), isNull);
final titleBefore = tester.getRect(title);
- final picker = find.byType(GridModelPicker);
- expect(picker.hitTestable(), findsOneWidget);
+ expect(find.byType(GridModelPicker), findsNothing);
+ final mouse = await tester.createGesture(kind: PointerDeviceKind.mouse);
+ await mouse.addPointer(location: const Offset(1, 100));
+ await mouse.moveTo(tester.getCenter(title));
+ await tester.pump();
+ expect(find.byTooltip('Close Pane').hitTestable(), findsOneWidget);
+ await mouse.removePointer();
expect(find.byTooltip('Stop Harness'), findsNothing);
expect(find.byTooltip('Zoom Pane'), findsNothing);
expect(tester.getRect(title), titleBefore);
diff --git a/desktop/test/terminal_session_test.dart b/desktop/test/terminal_session_test.dart
index 4db29d1a4..7cf915e94 100644
--- a/desktop/test/terminal_session_test.dart
+++ b/desktop/test/terminal_session_test.dart
@@ -1337,7 +1337,13 @@ void main() {
Future<void> acceptBegin() async {
await pump();
- expect(sent.single.type, 'terminal_chunked_upload_begin');
+ expect(
+ sent
+ .where((frame) => frame.type == 'terminal_chunked_upload_begin')
+ .single
+ .type,
+ 'terminal_chunked_upload_begin',
+ );
await session.handleFrame('terminal_chunked_upload_begin_result', {
'streamId': streamId,
'accepted': true,
@@ -1362,9 +1368,27 @@ void main() {
final future = session.pasteImage(png);
await acceptBegin();
- expect(sent.single.payload, containsPair('uploadKind', 'image'));
- expect(sent.single.payload, containsPair('totalBytes', png.length));
- expect(sent.single.payload, isNot(contains('filename')));
+ expect(
+ sent
+ .where((frame) => frame.type == 'terminal_chunked_upload_begin')
+ .single
+ .payload,
+ containsPair('uploadKind', 'image'),
+ );
+ expect(
+ sent
+ .where((frame) => frame.type == 'terminal_chunked_upload_begin')
+ .single
+ .payload,
+ containsPair('totalBytes', png.length),
+ );
+ expect(
+ sent
+ .where((frame) => frame.type == 'terminal_chunked_upload_begin')
+ .single
+ .payload,
+ isNot(contains('filename')),
+ );
await pump();
expect(imagePastes(), hasLength(1));
@@ -1406,8 +1430,20 @@ void main() {
final future = session.pasteFile('report.pdf', content);
await acceptBegin();
- expect(sent.single.payload, containsPair('uploadKind', 'file'));
- expect(sent.single.payload, containsPair('filename', 'report.pdf'));
+ expect(
+ sent
+ .where((frame) => frame.type == 'terminal_chunked_upload_begin')
+ .single
+ .payload,
+ containsPair('uploadKind', 'file'),
+ );
+ expect(
+ sent
+ .where((frame) => frame.type == 'terminal_chunked_upload_begin')
+ .single
+ .payload,
+ containsPair('filename', 'report.pdf'),
+ );
await pump();
expect(filePastes(), hasLength(1));
diff --git a/desktop/test/workspace_account_lifecycle_test.dart b/desktop/test/workspace_account_lifecycle_test.dart
index 66e73b2df..555e5baa4 100644
--- a/desktop/test/workspace_account_lifecycle_test.dart
+++ b/desktop/test/workspace_account_lifecycle_test.dart
@@ -1,6 +1,7 @@
import 'dart:async';
import 'dart:convert';
+import 'package:dio/dio.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:harness/api/api_client.dart';
import 'package:harness/auth/auth_session.dart';
@@ -12,6 +13,9 @@ import 'package:harness/state/pane_layout_store.dart';
import 'package:harness/state/pane_preset.dart';
import 'package:harness/terminal/terminal_session.dart';
import 'package:harness/ws/local_cli_discovery.dart';
+import 'package:harness/viewer/viewer_services.dart';
+import 'package:harness/viewer/viewer_key_store.dart';
+import 'package:harness/viewer/direct_auth_api.dart';
import 'swarm_screen_test.dart' show terminal;
import 'swarm_state_test.dart' show MemoryStore;
@@ -51,6 +55,7 @@ class _Api extends ApiClient {
session: AuthSession(storage: MemoryStore()),
);
var inventoryRequests = 0;
+ Object? inventoryFailure;
@override
Future<Map<String, dynamic>?> me() async => {
'user': {'id': 'fixture-account', 'email': '[email protected]'},
@@ -58,6 +63,7 @@ class _Api extends ApiClient {
@override
Future<List<Machine>> machines() async {
inventoryRequests++;
+ if (inventoryFailure case final error?) throw error;
return [];
}
}
@@ -69,6 +75,13 @@ class WorkspaceAccountFixture extends AppNotifier {
authSession: AuthSession(storage: MemoryStore()),
cliLogin: cli,
localCliDiscovery: _Discovery(),
+ // A browser must clear its workspace on sign-out/expiry. Native desktop deliberately
+ // keeps a guest workspace now; guest_window_test.dart covers that separate contract.
+ viewer: ViewerServices(
+ config: AppConfig.dev,
+ session: AuthSession(storage: MemoryStore()),
+ keys: ViewerKeyStore(storage: MemoryStore()),
+ ),
paneLayoutStore: PaneLayoutStore(storage: storage),
) {
api = _Api();
@@ -76,16 +89,11 @@ class WorkspaceAccountFixture extends AppNotifier {
}
final WorkspaceAccountLogin cli;
int get inventoryRequests => (api as _Api).inventoryRequests;
- var expired = false;
@override
- Future<void> ensureCliDaemonReady() async {
- if (expired) await super.ensureCliDaemonReady();
- }
+ Future<void> ensureCliDaemonReady() async {}
Future<void> expire() async {
- expired = true;
- await ensureCliDaemonReady();
- expired = false;
+ handleAuthFailureForTest('You were signed out. Sign in again.');
}
}
@@ -119,6 +127,30 @@ Future<void> arrangeAccountWorkspace(WorkspaceAccountFixture app) async {
}
void main() {
+ test('REST expiry returns an unconnected browser to sign-in', () async {
+ final app = WorkspaceAccountFixture(MemoryStore(), WorkspaceAccountLogin());
+ addTearDown(app.dispose);
+ (app.api as _Api).inventoryFailure = DioException(
+ requestOptions: RequestOptions(path: '/api/machines'),
+ error: const DirectAuthException('Expired', signedOut: true),
+ );
+ expect(app.machineStates, isEmpty);
+ expect(await app.refreshMachines(), false);
+ expect(app.status, AppStatus.unauthenticated);
+ });
+
+ test('REST outage keeps the browser signed in and retryable', () async {
+ final app = WorkspaceAccountFixture(MemoryStore(), WorkspaceAccountLogin());
+ addTearDown(app.dispose);
+ (app.api as _Api).inventoryFailure = DioException(
+ requestOptions: RequestOptions(path: '/api/machines'),
+ type: DioExceptionType.connectionError,
+ error: const DirectAuthException('Sign-in service unavailable'),
+ );
+ await expectLater(app.refreshMachines(), throwsA(isA<DioException>()));
+ expect(app.status, AppStatus.authenticated);
+ });
+
for (final expires in [false, true]) {
test(
'${expires ? 'expiry' : 'sign-out'} restores all tabs and their arrangement on sign-in',
diff --git a/desktop/third_party/xterm/README.autonomous.md b/desktop/third_party/xterm/README.autonomous.md
index f984052fe..2e37b1389 100644
--- a/desktop/third_party/xterm/README.autonomous.md
+++ b/desktop/third_party/xterm/README.autonomous.md
@@ -13,6 +13,17 @@ fix and the regression in `test/terminal_session_test.dart` passes against it.
## Local patches
+- **Browser accessibility input and editor switching**
+ (`lib/src/ui/custom_text_edit.dart`). The browser's text-input strategy needs
+ an editable semantics node when accessibility is enabled. The custom adapter
+ now publishes its editing value and focus through one such node, while the
+ enclosing `Focus` omits duplicate semantics. That prevents switching between
+ terminal and remote-viewer editors from deactivating the new DOM editor.
+ Read-only surfaces stay read-only; native input uses its existing path.
+ `semanticLabel` lets the viewer reuse the IME adapter. Validation includes
+ the native input regressions and `desktop/scripts/check-workspace.cjs`, which
+ checks actual received terminal bytes and remote viewer input after switching.
+
- **Linux clipboard and Meta keys leave shell editing intact**
(`lib/src/ui/shortcut/shortcuts.dart`, `lib/src/terminal_view.dart`,
`lib/src/core/input/handler.dart`). Linux uses Ctrl-Shift-C/V/A for clipboard
diff --git a/desktop/third_party/xterm/lib/src/ui/custom_text_edit.dart b/desktop/third_party/xterm/lib/src/ui/custom_text_edit.dart
index f2c4cf46b..46aaf2141 100644
--- a/desktop/third_party/xterm/lib/src/ui/custom_text_edit.dart
+++ b/desktop/third_party/xterm/lib/src/ui/custom_text_edit.dart
@@ -6,7 +6,7 @@ import 'package:flutter/services.dart';
import 'package:xterm/src/utils/unicode_v11.dart';
class CustomTextEdit extends StatefulWidget {
- CustomTextEdit({
+ const CustomTextEdit({
super.key,
required this.child,
required this.onInsert,
@@ -22,6 +22,7 @@ class CustomTextEdit extends StatefulWidget {
this.inputAction = TextInputAction.newline,
this.keyboardAppearance = Brightness.light,
this.deleteDetection = false,
+ this.semanticLabel,
});
final Widget child;
@@ -49,6 +50,7 @@ class CustomTextEdit extends StatefulWidget {
final Brightness keyboardAppearance;
final bool deleteDetection;
+ final String? semanticLabel;
@override
CustomTextEditState createState() => CustomTextEditState();
@@ -85,6 +87,7 @@ class CustomTextEditState extends State<CustomTextEdit> with TextInputClient {
void dispose() {
widget.focusNode.removeListener(_onFocusChange);
_closeInputConnectionIfNeeded();
+ _semanticEditingState.dispose();
super.dispose();
}
@@ -92,9 +95,46 @@ class CustomTextEditState extends State<CustomTextEdit> with TextInputClient {
Widget build(BuildContext context) {
return Focus(
focusNode: widget.focusNode,
+ // The web text-field node below owns focus semantics. A second focused ancestor can
+ // deactivate its DOM editor when input moves between two custom editors.
+ includeSemantics: !kIsWeb,
autofocus: widget.autofocus,
onKeyEvent: _onKeyEvent,
- child: widget.child,
+ // Flutter web's accessibility input strategy creates its DOM editor from a semantic
+ // text field. A bare TextInputClient receives arrows/Enter but loses every printable key
+ // when a screen reader (or accessibility automation) enables semantics.
+ child: kIsWeb
+ ? ListenableBuilder(
+ listenable:
+ Listenable.merge([widget.focusNode, _semanticEditingState]),
+ builder: (context, _) => Semantics(
+ container: true,
+ excludeSemantics: true,
+ // Read-only observers are output surfaces, not browser editors. The web engine
+ // creates a writable DOM textarea even for a semantic readOnly text field.
+ textField: !widget.readOnly,
+ enabled: true,
+ focusable: !widget.readOnly,
+ focused: !widget.readOnly && widget.focusNode.hasFocus,
+ multiline: true,
+ label: widget.semanticLabel ??
+ (widget.readOnly ? 'Terminal output' : 'Terminal input'),
+ value: _semanticEditingState.value.text,
+ onTap: widget.readOnly ? null : requestKeyboard,
+ onFocus: widget.readOnly ? null : requestKeyboard,
+ onSetText: widget.readOnly
+ ? null
+ : (text) {
+ updateEditingValue(TextEditingValue(
+ text: text,
+ selection:
+ TextSelection.collapsed(offset: text.length),
+ ));
+ },
+ child: widget.child,
+ ),
+ )
+ : widget.child,
);
}
@@ -117,6 +157,7 @@ class CustomTextEditState extends State<CustomTextEdit> with TextInputClient {
void setEditingState(TextEditingValue value) {
_cancelPendingDeletes();
_currentEditingState = value;
+ _semanticEditingState.value = value;
_terminalText = value.text;
_connection?.setEditingState(value);
}
@@ -127,6 +168,7 @@ class CustomTextEditState extends State<CustomTextEdit> with TextInputClient {
void resetEditingState() {
_cancelPendingDeletes();
_currentEditingState = _initEditingState.copyWith();
+ _semanticEditingState.value = _currentEditingState;
_terminalText = _currentEditingState.text;
widget.onComposing(null, 0);
_connection?.setEditingState(_currentEditingState);
@@ -245,6 +287,7 @@ class CustomTextEditState extends State<CustomTextEdit> with TextInputClient {
);
late var _currentEditingState = _initEditingState.copyWith();
+ late final _semanticEditingState = ValueNotifier(_currentEditingState);
/// Text that has already been mirrored to the PTY. This deliberately stays
/// separate from [_currentEditingState], whose composing range can contain
@@ -311,6 +354,7 @@ class CustomTextEditState extends State<CustomTextEdit> with TextInputClient {
}) {
final wasComposing = !_currentEditingState.composing.isCollapsed;
_currentEditingState = value;
+ _semanticEditingState.value = value;
final isComposing = !_currentEditingState.composing.isCollapsed;
if (hasTextMutation || wasComposing || isComposing) {
@@ -423,6 +467,7 @@ class CustomTextEditState extends State<CustomTextEdit> with TextInputClient {
composing: TextRange.empty,
);
_currentEditingState = next;
+ _semanticEditingState.value = next;
_connection?.setEditingState(next);
_syncTerminalText(next.text);
return;
diff --git a/docs/plans/2026-09-27-008-web-release-readiness.md b/docs/plans/2026-09-27-008-web-release-readiness.md
new file mode 100644
index 000000000..09c58b8a3
--- /dev/null
+++ b/docs/plans/2026-09-27-008-web-release-readiness.md
@@ -0,0 +1,67 @@
+# Web release readiness
+
+User objective: bring the latest desktop features to the web, test the complete browser experience, and prepare the website for a public release. Continue using the shared Flutter source; do not replace feature parity with a separate, reduced web interface.
+
+## Current source and evidence
+
+- Fetched `origin/main` on September 27: `93f148d6`, including the latest desktop welcome/new-tab work (`767c1e07`) and phone setup (`244cfe71`).
+- Sharing links/comments are complete in PR #394, with hosting support in autonomous-code PR #5. This readiness branch builds on those commits; it has not shipped them.
+- Chrome regression run: 48 tests pass for workspace/pickers, login, persistent credentials/machine keys, concurrent refresh/logout, password stretching, encrypted observers and binary terminal framing. This is a starting gate, not evidence for every product workflow.
+- The initial full desktop suite exposed 35 stale expectations after recent desktop changes. Corrected those fixtures and expectations. The final complete rerun passed **3,785 tests, 12 existing skips** (`/private/tmp/harness-web-readiness-native-complete.jsonl`), including all browser-machine control changes.
+- The final Chrome workspace/machine-control run passed **12 tests**, including Alt-Enter orchestration launch. CLI TypeScript checking passed. Full Flutter analysis has no errors or new findings; its 16 existing findings are one unused test import and 15 test/vendor lint notices.
+
+## Release audit
+
+The completion audit must contain current code and runtime evidence for each of these areas:
+
+1. Latest welcome screen/new tab, working visible actions, keyboard hints, responsive layouts and Download app.
+2. Sign-in/callback, private-link return, sign-out/account changes, refresh, browser restart and multiple tabs.
+3. Machine discovery, password linking, saved pins, connection errors/retry, settings, rename/remove and offline/reconnect behavior.
+4. Agent creation/opening, project selection, engine/model configuration, terminal input/clipboard, control ownership, stop/resume/restart, rename/fork/clone where supported by the engine.
+5. Tabs and panes, layouts/zoom/focus, persisted workspace and account desk synchronization, search/commands and browser shortcuts.
+6. Store installation, previews/viewers, customization, local models on connected machines and API connections.
+7. Public/private sharing, comments, identity verification, revocation and browser reload (full-stack evidence in the sharing plan).
+8. Settings, notifications, keyboard practice, usage and phone handoff. Native OS or hardware operations need an explicit functioning browser route or handoff, not a visible dead control.
+9. Production hosting routes, asset/cache behavior, startup speed, production build and deployment readiness. Evidence must distinguish local implementation, merged code and the deployed website.
+
+## Implemented and verified
+
+- Fixed the welcome command gate: keyboard practice, quick start, and Work from your phone are available in the browser. Phone setup freezes the selected linked machine and pairs through an authenticated, encrypted owner RPC. Phone widget/transport tests (18) and CLI pairing/crypto tests (41) pass. The real browser welcome action exposed `NO_INTENT` being mislabeled as offline; preserve `WsRequestFailure.code` through the relay adapter. The QR now stays ready to scan, with dedicated regression coverage.
+- Added `desktop/scripts/check-workspace.cjs`, run by the isolated backend/daemon fixture. The final expanded run passed in `harness-share-e2e-keODGG`: real browser sign-in, password linking, saved machine identity, phone QR, API-key persistence on the daemon, orchestrator screen, agent selection, ordinary and accessibility terminal typing, interactive viewer clicks/typing, and reload. It then passed public/private/denied browser links and comments, followed by the daemon restart, moderation and revocation checks. The test records actual fixture PTY input rather than assuming keystrokes arrive in one output chunk. The SSO redirect/exchange and model are the only stand-ins; machine transport and encryption are real.
+- Fixed browser terminal accessibility input by supplying one editable semantics node in the vendored input adapter. Disabling the enclosing Focus's duplicate semantics fixes switching between custom editors. Read-only terminals publish output semantics instead: the web engine otherwise creates a writable-looking textarea even with its semantic `readOnly` flag. The public-link regression now proves there is no editor. Real viewer-to-terminal typing and a fresh ordinary browser tab both pass; temporary production diagnostic logs have been removed.
+- Implemented an owner-only encrypted interactive viewer using the existing isolated Chromium renderer. It is separate from read-only shared observers, scoped to a managed agent viewer, bounded to four surfaces per connection/eight total, and released on disconnect or idle timeout. CLI permission/lifecycle/input tests (187 with existing backend/viewer tests) and Flutter controller/pane tests (17) pass. Real browser mouse and keyboard interaction pass. Chromium must be installed on the agent machine.
+- Fixed REST-only browser expiry: invalid credentials return to sign-in even without an open machine WebSocket. Refresh-service outages stay retryable and preserve credentials. Auth/client tests (27) and workspace expiry tests (17) pass.
+- API connections, model controls and orchestration now target the selected linked owner machine. Targets remain fixed while editing, disconnected mutations fail instead of queuing, and model controls select the current host when reopened. API metadata persists in UI state; secrets stay on the chosen daemon. Focused Flutter checks: 106 pass, followed by 77 after the panel-switching/learning changes. CLI owner permissions and viewer checks: 200 pass.
+- Browser command decisions and task delivery now reuse the desktop services through sealed owner RPCs. The daemon bounds in-flight decisions and aborts them on disconnect; decisions never execute actions themselves. Flutter command/phone/connection checks: 69 pass. CLI command, permission and viewer checks: 189 pass. Chrome workspace/machine controls: 12 pass. The widget fixture initializes the generated Lucide icon library before building the deep widget tree to avoid a DDC module-initialization stack overflow. The production orchestrator dialog also passes the real release-build browser check.
+- Reviewed native-only workspace commands and settings against the scope below. Browser controls use a connected-machine route or the existing desktop handoff. Native OS operations and untested engine/provider integrations are listed explicitly instead of being counted as browser runtime coverage.
+
+![Interactive browser viewer and terminal, using disposable fixture data](../../desktop/design/images/web-interactive-viewer.png)
+
+## Reproducible fixture
+
+The current local fixture uses backend port `54861` and a Flutter preview on
+`54862`. Mongo/Redis are disposable fixture services described by
+`/private/tmp/harness-sharing-services/services.json`; they are not production
+services. Build with `HARNESS_API_URL=http://127.0.0.1:54861`, `HARNESS_TEST=true`,
+`HARNESS_CURRENT_WORKSPACE=true`, and `HARNESS_CLEAN_PREVIEW=true`. Run
+`cli/scripts/share-harness-e2e.ts` with both browser-check script paths, the
+preview origin, and `HARNESS_SHARE_E2E_SERVICES` pointing to that service file.
+The fixture redirects each daemon's data/runtime/auth/project paths into its
+own temporary directory and blocks external traffic. It does not alter HOME,
+CODEX_HOME, the installed daemon, or real account credentials.
+
+Production remains on the previously released web build. Sharing PR #394,
+hosting PR #5 and this readiness branch have not been deployed.
+
+## Platform scope and evidence
+
+| Area | Browser implementation and verification |
+| --- | --- |
+| Welcome, tabs, panes, pickers and shortcuts | Shared desktop widgets; Chrome workspace tests plus the real new-tab, machines, models, agents and command-picker flows. Browser bindings use Alt/Option, including orchestration's Alt-Enter. |
+| Auth and persistence | Real fixture sign-in/reload/new tab; Chrome storage, concurrent refresh/logout, private-link return and owner-key tests. REST expiry/outage regressions also pass without a machine socket. |
+| Agent lifecycle and workspace state | Shared controllers and daemon RPCs; full native unit/widget suite, encrypted browser transport tests and real managed-agent terminal/viewer connection. Individual engine integrations retain their existing daemon coverage; this audit does not claim a live model run for every engine. |
+| Models, APIs, Store and orchestration | Machine-scoped owner RPCs; explicit frozen host in editors; model lifecycle/controller tests and real browser API persistence/orchestration screen. The fixture installs its managed harness locally and opens its real viewer. It does not download multi-GB models or install paid third-party tools. |
+| Sharing and comments | Actual anonymous, invited and denied browser accounts; encrypted output/comments, responsive layout, reload, moderation, restart persistence, revocation and rejected observer controls. |
+| Settings and phone | Shared preferences, learning/practice, browser activity and remote account usage. Real welcome phone QR, protocol-error regression and cryptographic phone pairing tests. |
+| Native platform operations | Local daemon installation, firmware, OS notifications, native image clipboard, keyboard dotfiles and local transcript ledgers stay in desktop; browser settings already omit or explain these controls, with Download app as the handoff. Managed-viewer streaming does not forward native dialogs, downloads or audio. |
+| Hosting and release | Existing public preview and hosting pipeline remain unchanged. Shared-link hosting is PR #5 against `deploy/flutter-web`; source is PR #394 plus this branch. Final production artifact results are recorded below after packaging committed source. |