Skip to content

An agentic browser harness: an AI side panel & orb in your own logged-in Chrome, Edge, Arc, Brave or even Firefox run by the agent/subscription you already use (Claude Code, Codex, Antigravity). Free, open source, no API key. Also an MCP server.

feat(android): protocol 23 adds the Android frames, sent only to a peer that speaks 23; the Bridge keeps each extension's protocol, the extension keeps the Bridge's, and the control socket gets an android op

Shahriarcc134de1b97f
diff --git a/docs/internals/transport.md b/docs/internals/transport.md
index 44fbf0b..1be4264 100644
--- a/docs/internals/transport.md
+++ b/docs/internals/transport.md
@@ -113,7 +113,7 @@ dialling. It never deletes the stored key; only pairing again or `disconnect` re
## Protocol version
-Both sides compile in `SOCKET_PROTOCOL_VERSION` (currently **22**). They used to have to match
+Both sides compile in `SOCKET_PROTOCOL_VERSION` (currently **23**). They used to have to match
exactly. A store copy updates when its browser decides to, not when the daemon does, so the
extension and the daemon are routinely a version apart, and the daemon accepts a **window**:
@@ -133,6 +133,31 @@ a change neither side can do without raises a minimum. A refusal must never stra
v0.8.0 store build unpairs on any refusal and does not retry, so no daemon may refuse protocol 22
until a store build that retries has replaced it.
+### Protocol 23: Android
+
+Protocol 23 adds the frames that let a browser drive Chrome on an Android phone through the Bridge.
+The Bridge keeps the number each extension's `hello` claimed and sends these frames only to an
+extension that speaks 23. The extension keeps the number the `welcome` carried (`daemonSpeaks`) and
+sends them only to a Bridge that speaks 23. An extension that speaks 22 is never sent one, and a phone
+request from it is answered `UNSUPPORTED`. The shapes are in `src/lib/phone/types.ts`.
+
+| Frame | From | What it carries |
+| --- | --- | --- |
+| `androidInfo` | Bridge | The phones and whether one is ready (`AndroidState`). Pushed with an empty `id` on connect and on every change, and the answer to `phoneLaunch` |
+| `phoneOpen` | extension | `serial`: start a session with Chrome on that phone. Answered by `phoneOpened` |
+| `phoneOpened` | Bridge | The device, its open tabs and Chrome's version, or an error |
+| `phoneClose` | extension | `serial`: end the session. Not answered |
+| `phoneLaunch` | extension | `serial` and an optional `url`: open Chrome on the phone |
+| `cdp` | extension | One DevTools command for the phone's Chrome: `method`, `params` and `sessionId`. Answered by `cdpResult` |
+| `cdpResult` | Bridge | Chrome's result, or `CDP_ERROR`, `TIMEOUT`, `PHONE_GONE` or `NOT_OWNER` |
+| `cdpEvent` | Bridge | Every DevTools event from the phone, screencast frames included, sent only to the extension that opened the session |
+| `phoneClosed` | Bridge | `serial` and why the session ended: `unplugged`, `chrome-exited`, `bridge-stopping` or `closed` |
+
+The control socket has an `android` op that reads the same state, subscribes to `android-changed`
+with `watch`, and opens Chrome on a phone with `launch` (refused to an agent run's connection). It
+has no way to send a DevTools command. Every action on the phone goes through the extension, which
+is where the blocked-sites list is checked.
+
---
## What this does not protect against
diff --git a/src/daemon/android/service.ts b/src/daemon/android/service.ts
new file mode 100644
index 0000000..0aa2979
--- /dev/null
+++ b/src/daemon/android/service.ts
@@ -0,0 +1,39 @@
+import { failure, type ActionResult } from '@/lib/actions/protocol';
+import type { AndroidProblem, AndroidState } from '@/lib/phone/types';
+
+export interface Android {
+ state(): Promise<AndroidState>;
+ /**
+ * A watcher hears the state as soon as it is known, then every change. The Bridge looks for phones
+ * only while someone holds a watch, and the returned function lets go of it.
+ */
+ watch(listener: (state: AndroidState) => void): () => void;
+ launch(serial: string, url?: string): Promise<ActionResult<AndroidState>>;
+ stop(): Promise<void>;
+}
+
+const SWITCHED_OFF: AndroidProblem = {
+ code: 'ANDROID_OFF',
+ message: 'Android is switched off for this computer.',
+ fix: 'Set "android": { "enabled": true } in ~/.browsentic/config.json',
+};
+
+export const ANDROID_OFF_STATE: AndroidState = {
+ enabled: false,
+ ready: false,
+ adb: { found: false },
+ devices: [],
+ problem: SWITCHED_OFF,
+};
+
+export function androidOff(): Android {
+ return {
+ state: async () => ANDROID_OFF_STATE,
+ watch: (listener) => {
+ queueMicrotask(() => listener(ANDROID_OFF_STATE));
+ return () => {};
+ },
+ launch: async () => failure(SWITCHED_OFF.code, SWITCHED_OFF.message),
+ stop: async () => {},
+ };
+}
diff --git a/src/daemon/control.ts b/src/daemon/control.ts
index 4d9588d..777609f 100644
--- a/src/daemon/control.ts
+++ b/src/daemon/control.ts
@@ -1,6 +1,7 @@
import type { ActionResult } from '@/lib/actions/protocol';
import type { ToolDescriptor } from '@/lib/actions/manifest';
import type { AgentKind, AgentState } from '@/lib/agents/catalog';
+import type { AndroidState } from '@/lib/phone/types';
import type { Preferences } from '@/lib/settings/preferences';
import type { Source } from '@/lib/stores';
@@ -49,7 +50,9 @@ export type ControlRequest =
| { id: string; op: 'agent'; set?: AgentKind; grant?: AgentKind; models?: AgentKind }
/** `watch` also subscribes this caller to `settings-changed`, which no other caller is sent. */
| { id: string; op: 'preferences'; watch?: boolean }
- | { id: string; op: 'setPreference'; change: unknown };
+ | { id: string; op: 'setPreference'; change: unknown }
+ /** `launch` opens Chrome on that phone. `watch` subscribes this caller to `android-changed`. */
+ | { id: string; op: 'android'; launch?: string; watch?: boolean };
export type ControlMessage =
| { id: string; op: 'describe'; tools: ToolDescriptor[]; reserved?: string[]; withheld?: ToolDescriptor[] }
@@ -61,9 +64,11 @@ export type ControlMessage =
| { id: string; op: 'revoke'; revoked: number }
| { id: string; op: 'agent'; state: AgentState }
| { id: string; op: 'preferences'; result: ActionResult<Preferences> }
+ | { id: string; op: 'android'; result: ActionResult<AndroidState> }
| { event: 'manifest-changed' }
/** config.json changed, from any side: read the preferences and the agent state again. */
- | { event: 'settings-changed' };
+ | { event: 'settings-changed' }
+ | { event: 'android-changed'; state: AndroidState };
export interface Described {
tools: ToolDescriptor[];
diff --git a/src/daemon/daemon.ts b/src/daemon/daemon.ts
index df2d251..4756105 100644
--- a/src/daemon/daemon.ts
+++ b/src/daemon/daemon.ts
@@ -4,6 +4,7 @@ import type { AddressInfo } from 'node:net';
import type { Duplex } from 'node:stream';
import { WebSocketServer, type WebSocket } from 'ws';
import {
+ ANDROID_PROTOCOL,
EXTERNAL_RUN_ID,
MIN_EXTENSION_PROTOCOL,
SOCKET_PROTOCOL_VERSION,
@@ -12,10 +13,12 @@ import {
parseFrame,
success,
type ActionResult,
+ type ExtensionRequest,
type RunEvent,
type SkillCatalog,
type SocketFrame,
} from '@/lib/actions/protocol';
+import type { AndroidState } from '@/lib/phone/types';
import type { TaskList } from '@/lib/schedules/task';
import { hashManifest, type ToolDescriptor } from '@/lib/actions/manifest';
import { solveCaptcha } from '@/lib/actions/page/solve-captcha';
@@ -50,6 +53,7 @@ import {
type Session,
} from './auth-store';
import { AgentSession } from './agent/service';
+import { androidOff, type Android } from './android/service';
import { solveCaptchaWithAnalyst } from './agent/captcha-solver';
import { FileAnalyses } from './agent/file-analyst';
import { analyzeRecording } from './agent/recording';
@@ -90,6 +94,18 @@ import { daemonPorts } from './ports';
type AgentFrame = Extract<SocketFrame, { t: 'agentState' | 'setAgent' | 'setAgentModel' | 'grantAgent' }>;
type TaskFrame = Extract<SocketFrame, { t: 'tasks' | 'saveTask' | 'deleteTask' | 'pauseTasks' }>;
+type PhoneRequest = Extract<ExtensionRequest, { t: 'phoneOpen' | 'phoneClose' | 'phoneLaunch' | 'cdp' }>;
+
+const PHONE_REQUESTS: ReadonlySet<string> = new Set<PhoneRequest['t']>(['phoneOpen', 'phoneClose', 'phoneLaunch', 'cdp']);
+
+const isPhoneRequest = (request: ExtensionRequest): request is PhoneRequest => PHONE_REQUESTS.has(request.t);
+
+function phoneReply(request: PhoneRequest, result: ActionResult<never>): SocketFrame | null {
+ if (request.t === 'phoneOpen') return { t: 'phoneOpened', id: request.id, result };
+ if (request.t === 'phoneLaunch') return { t: 'androidInfo', id: request.id, result };
+ if (request.t === 'cdp') return { t: 'cdpResult', id: request.id, result };
+ return null;
+}
const IDLE_EXIT_MS = 30 * 60 * 1000;
const BINDING_IDLE_MS = 2 * 60 * 1000;
@@ -128,6 +144,7 @@ export function protocolRefusal(hello: { protocolVersion: unknown; minDaemonProt
export interface DaemonOptions {
version: string;
idleExit?: boolean;
+ android?: Android;
}
export interface Daemon extends Bridge {
@@ -177,7 +194,7 @@ function persistDownload(action: string, result: ActionResult, hosts?: readonly
return { ok: true, data: { downloadId: id, name, mime, size, host, notes, savedTo } };
}
-export async function startDaemon({ version, idleExit = true }: DaemonOptions): Promise<Daemon> {
+export async function startDaemon({ version, idleExit = true, android = androidOff() }: DaemonOptions): Promise<Daemon> {
const swept = sweepDownloads();
if (swept) log(`swept ${swept} expired download${swept === 1 ? '' : 's'}`);
@@ -196,6 +213,7 @@ export async function startDaemon({ version, idleExit = true }: DaemonOptions):
const analyses = new FileAnalyses();
const controls = new Set<WebSocket>();
const settingsWatchers = new Set<WebSocket>();
+ const androidWatches = new Map<ExtensionLink | WebSocket, () => void>();
let configSeen = markConfig();
const unwatchConfig = watchConfig(() => configChanged());
let controlSeq = 0;
@@ -405,6 +423,7 @@ export async function startDaemon({ version, idleExit = true }: DaemonOptions):
agents.delete(closing);
scheduler.linkClosed(closing.id);
analyses.cancelOwnedBy(closing);
+ unwatchAndroid(closing);
if (links.get(closing.id) === closing) links.delete(closing.id);
log(`${closing.label} disconnected`);
scheduleIdleExit();
@@ -500,6 +519,7 @@ export async function startDaemon({ version, idleExit = true }: DaemonOptions):
if (request.t === 'discardSiteMap') {
return source.send({ t: 'skillResult', id: request.id, result: discardStaging(request.stagingId) });
}
+ if (isPhoneRequest(request)) return settlePhone(request, source);
session(source).handle(request);
},
);
@@ -526,9 +546,37 @@ export async function startDaemon({ version, idleExit = true }: DaemonOptions):
accepted.send({ t: 'preferencesInfo', id: '', result: success(preferencesNow()) });
accepted.send({ t: 'taskList', id: '', result: success(scheduler.list()) });
scheduler.linkOpened();
+ if (accepted.speaks(ANDROID_PROTOCOL)) watchAndroid(accepted);
if (!known) await adoptExtensionManifest(accepted);
}
+ function watchAndroid(link: ExtensionLink): void {
+ const push = (state: AndroidState) => {
+ if (link.isOpen) link.send({ t: 'androidInfo', id: '', result: success(state) });
+ };
+ androidWatches.set(link, android.watch(push));
+ }
+
+ function unwatchAndroid(watcher: ExtensionLink | WebSocket): void {
+ androidWatches.get(watcher)?.();
+ androidWatches.delete(watcher);
+ }
+
+ function settlePhone(request: PhoneRequest, source: ExtensionLink): void {
+ const answer = (result: ActionResult<never>) => {
+ const reply = phoneReply(request, result);
+ if (reply) source.send(reply);
+ };
+ if (!source.speaks(ANDROID_PROTOCOL)) {
+ return answer(failure('UNSUPPORTED', `Android needs an extension that speaks protocol ${ANDROID_PROTOCOL}.`));
+ }
+ if (request.t === 'phoneLaunch') {
+ void android.launch(request.serial, request.url).then((result) => source.send({ t: 'androidInfo', id: request.id, result }));
+ return;
+ }
+ answer(failure('UNSUPPORTED', 'This Bridge cannot drive a phone yet.'));
+ }
+
function openLinks(): ExtensionLink[] {
return [...links.values()].filter((link) => link.isOpen);
}
@@ -821,6 +869,21 @@ export async function startDaemon({ version, idleExit = true }: DaemonOptions):
if (request.watch) settingsWatchers.add(ws);
return send(ws, { id: request.id, op: 'preferences', result: success(preferencesNow()) });
}
+ if (request.op === 'android') {
+ if (request.launch && speaksForRun) {
+ log(`control ${client} tried to open Chrome on a phone from inside an agent run; refused`);
+ return send(ws, {
+ id: request.id,
+ op: 'android',
+ result: failure('BLOCKED', 'An agent run cannot open apps on the phone. Open Chrome from the side panel or the desktop app.'),
+ });
+ }
+ if (request.watch && !androidWatches.has(ws)) {
+ androidWatches.set(ws, android.watch((state) => send(ws, { event: 'android-changed', state })));
+ }
+ const result = request.launch ? await android.launch(request.launch) : success(await android.state());
+ return send(ws, { id: request.id, op: 'android', result });
+ }
if (request.op === 'setPreference') {
if (speaksForRun) {
log(`control ${client} tried to change a setting from inside an agent run; refused`);
@@ -848,6 +911,7 @@ export async function startDaemon({ version, idleExit = true }: DaemonOptions):
for (const call of inflight.values()) call.abort();
controls.delete(ws);
settingsWatchers.delete(ws);
+ unwatchAndroid(ws);
scheduleIdleExit();
};
ws.on('close', drop);
@@ -965,6 +1029,8 @@ export async function startDaemon({ version, idleExit = true }: DaemonOptions):
if (idleTimer) clearTimeout(idleTimer);
scheduler.stop();
unwatchConfig();
+ for (const watcher of [...androidWatches.keys()]) unwatchAndroid(watcher);
+ await android.stop();
for (const link of [...links.values()]) link.close('daemon shutting down');
for (const ws of controls) ws.close(1001, 'daemon shutting down');
wss.close();
diff --git a/src/daemon/extension-link.test.ts b/src/daemon/extension-link.test.ts
index e902f48..4891cc4 100644
--- a/src/daemon/extension-link.test.ts
+++ b/src/daemon/extension-link.test.ts
@@ -22,7 +22,7 @@ class FakeSocket extends EventEmitter {
}
}
-const hello = { extensionVersion: '0.6.2', manifestHash: 'abc', origin: 'chrome-extension://abcdefghijklmnop', installId: 'install-chrome-0001' };
+const hello = { protocolVersion: 22, extensionVersion: '0.6.2', manifestHash: 'abc', origin: 'chrome-extension://abcdefghijklmnop', installId: 'install-chrome-0001' };
let socket: FakeSocket;
let closed: ExtensionLink[];
diff --git a/src/daemon/extension-link.ts b/src/daemon/extension-link.ts
index c263071..9052615 100644
--- a/src/daemon/extension-link.ts
+++ b/src/daemon/extension-link.ts
@@ -38,6 +38,7 @@ export class ExtensionLink {
readonly origin: string;
readonly id: string;
readonly browser?: string;
+ readonly protocolVersion: number;
/** Which connected browser the user was last in, so a caller that names none reaches that one. */
lastActive = touched();
/** What this browser offers: a Firefox build lists fewer tools than a Chromium one, and a drifted build its own. */
@@ -48,7 +49,14 @@ export class ExtensionLink {
constructor(
private readonly socket: WebSocket,
- hello: { extensionVersion: string; manifestHash: string; origin: string; installId: string; browser?: string },
+ hello: {
+ protocolVersion: number;
+ extensionVersion: string;
+ manifestHash: string;
+ origin: string;
+ installId: string;
+ browser?: string;
+ },
private readonly onClose: (link: ExtensionLink) => void,
private readonly onRequest?: (request: ExtensionRequest, link: ExtensionLink) => void,
) {
@@ -57,6 +65,7 @@ export class ExtensionLink {
this.origin = hello.origin;
this.id = hello.installId;
this.browser = hello.browser;
+ this.protocolVersion = hello.protocolVersion;
socket.on('message', (raw) => this.receive(String(raw)));
socket.on('close', () => this.dispose('socket closed'));
socket.on('error', (error) => {
@@ -70,6 +79,10 @@ export class ExtensionLink {
return this.browser ?? this.origin;
}
+ speaks(version: number): boolean {
+ return this.protocolVersion >= version;
+ }
+
get isOpen(): boolean {
return !this.closed && this.socket.readyState === this.socket.OPEN;
}
diff --git a/src/daemon/remote-bridge.ts b/src/daemon/remote-bridge.ts
index ce6c323..05c7fbc 100644
--- a/src/daemon/remote-bridge.ts
+++ b/src/daemon/remote-bridge.ts
@@ -6,6 +6,7 @@ import { SOLVE_CAPTCHA_TIMEOUT_MS, solveCaptcha } from '@/lib/actions/page/solve
import { startMonitor } from '@/lib/actions/page/start-monitor';
import { failure, type ActionResult } from '@/lib/actions/protocol';
import type { AgentKind, AgentState } from '@/lib/agents/catalog';
+import type { AndroidState } from '@/lib/phone/types';
import { AWAIT_DEFAULT_TIMEOUT_MS } from '@/lib/monitor/events';
import type { Bridge, BridgeStatus, ControlMessage, ControlRequest, Described, SessionSummary } from './control';
@@ -14,6 +15,7 @@ const REQUEST_TIMEOUT_MS = 60_000;
export class RemoteBridge implements Bridge {
private readonly pending = new Map<string, (message: ControlMessage) => void>();
private readonly manifestListeners = new Set<() => void>();
+ private readonly androidListeners = new Set<(state: AndroidState) => void>();
private constructor(
private readonly socket: WebSocket,
@@ -67,10 +69,20 @@ export class RemoteBridge implements Bridge {
async agent(change?: { set?: AgentKind; grant?: AgentKind; models?: AgentKind }): Promise<AgentState> {
const reply = await this.request({ id: randomUUID(), op: 'agent', ...change });
- if (reply && 'state' in reply) return reply.state;
+ if (reply && 'op' in reply && reply.op === 'agent') return reply.state;
throw new Error('The Browsentic daemon did not answer about its agent');
}
+ async android(request?: { launch?: string; watch?: boolean }): Promise<ActionResult<AndroidState>> {
+ const reply = await this.request({ id: randomUUID(), op: 'android', ...request });
+ if (reply && 'op' in reply && reply.op === 'android') return reply.result;
+ return failure('DAEMON_UNREACHABLE', 'The Browsentic daemon did not answer about Android');
+ }
+
+ onAndroidChanged(listener: (state: AndroidState) => void): void {
+ this.androidListeners.add(listener);
+ }
+
/** A browser is named by its session id; an origin still unpairs every browser presenting it. */
async revoke(browser?: string): Promise<number> {
const named = browser?.includes('://') ? { origin: browser } : { session: browser };
@@ -120,6 +132,7 @@ export class RemoteBridge implements Bridge {
}
if ('event' in message) {
if (message.event === 'manifest-changed') for (const listener of this.manifestListeners) listener();
+ if (message.event === 'android-changed') for (const listener of this.androidListeners) listener(message.state);
return;
}
this.pending.get(message.id)?.(message);
diff --git a/src/daemon/test/fake-android.ts b/src/daemon/test/fake-android.ts
new file mode 100644
index 0000000..a30ba54
--- /dev/null
+++ b/src/daemon/test/fake-android.ts
@@ -0,0 +1,61 @@
+import { success, type ActionResult } from '@/lib/actions/protocol';
+import type { AndroidState } from '@/lib/phone/types';
+import type { Android } from '../android/service';
+
+export const NO_PHONE: AndroidState = {
+ enabled: true,
+ ready: false,
+ adb: { found: true, path: '/fake/adb', version: '1.0.41' },
+ devices: [],
+ problem: { code: 'NO_DEVICE', message: 'No phone is connected.' },
+};
+
+export const READY_PHONE: AndroidState = {
+ enabled: true,
+ ready: true,
+ adb: { found: true, path: '/fake/adb', version: '1.0.41' },
+ devices: [
+ {
+ serial: 'emulator-5554',
+ transport: 'usb',
+ state: 'ready',
+ model: 'Pixel Fold',
+ chrome: { installed: true, version: '150.0.7871.186', running: true, debuggable: true },
+ },
+ ],
+};
+
+/** The Bridge's view of the phones, set by the test instead of read from adb. */
+export class FakeAndroid implements Android {
+ current: AndroidState = NO_PHONE;
+ readonly launched: string[] = [];
+ private readonly listeners = new Set<(state: AndroidState) => void>();
+
+ get watchers(): number {
+ return this.listeners.size;
+ }
+
+ async state(): Promise<AndroidState> {
+ return this.current;
+ }
+
+ watch(listener: (state: AndroidState) => void): () => void {
+ this.listeners.add(listener);
+ queueMicrotask(() => this.listeners.has(listener) && listener(this.current));
+ return () => this.listeners.delete(listener);
+ }
+
+ async launch(serial: string): Promise<ActionResult<AndroidState>> {
+ this.launched.push(serial);
+ return success(this.current);
+ }
+
+ async stop(): Promise<void> {
+ this.listeners.clear();
+ }
+
+ change(next: AndroidState): void {
+ this.current = next;
+ for (const listener of this.listeners) listener(next);
+ }
+}
diff --git a/src/daemon/test/fake-browser.ts b/src/daemon/test/fake-browser.ts
index b3fdb47..454412f 100644
--- a/src/daemon/test/fake-browser.ts
+++ b/src/daemon/test/fake-browser.ts
@@ -44,6 +44,9 @@ export class FakeBrowser {
preferences: Preferences | null = null;
/** The agent state the daemon last pushed or answered with. */
agent: AgentState | null = null;
+ /** Every frame the daemon sent after the welcome, in arrival order. */
+ readonly received: SocketFrame[] = [];
+ private readonly arrivals = new Set<() => void>();
/** While set, every invoke is answered only once it resolves. */
holdInvokes: Promise<void> | null = null;
private readonly replies = new Map<string, (frame: SocketFrame) => void>();
@@ -122,6 +125,13 @@ export class FakeBrowser {
this.socket.close(1000, 'test over');
}
+ /** Resolves once the daemon has sent this many frames of a kind, with every one of them. */
+ async frames<T extends SocketFrame['t']>(t: T, count = 1): Promise<Extract<SocketFrame, { t: T }>[]> {
+ const ofKind = () => this.received.filter((frame): frame is Extract<SocketFrame, { t: T }> => frame.t === t);
+ while (ofKind().length < count) await new Promise<void>((resolve) => this.arrivals.add(resolve));
+ return ofKind();
+ }
+
/** Sends what the side panel would, and resolves with the daemon's answer to it. */
ask(request: Extract<ExtensionRequest, { id: string }>): Promise<SocketFrame> {
const answered = new Promise<SocketFrame>((resolve) => this.replies.set(request.id, resolve));
@@ -134,6 +144,9 @@ export class FakeBrowser {
}
private receive(frame: SocketFrame | null): void {
+ if (frame) this.received.push(frame);
+ for (const arrived of this.arrivals) arrived();
+ this.arrivals.clear();
if (frame?.t === 'taskList' && frame.result.ok) this.tasks = frame.result.data;
if (frame?.t === 'preferencesInfo' && frame.result.ok) this.preferences = frame.result.data;
if (frame?.t === 'agentInfo' && frame.result.ok) this.agent = frame.result.data;
diff --git a/src/daemon/test/protocol-window.test.ts b/src/daemon/test/protocol-window.test.ts
index 38447d8..0096047 100644
--- a/src/daemon/test/protocol-window.test.ts
+++ b/src/daemon/test/protocol-window.test.ts
@@ -1,10 +1,11 @@
import { afterAll, afterEach, beforeAll, describe, expect, test } from 'vitest';
-import { MIN_EXTENSION_PROTOCOL, SOCKET_PROTOCOL_VERSION } from '@/lib/actions/protocol';
+import { ANDROID_PROTOCOL, MIN_EXTENSION_PROTOCOL, SOCKET_PROTOCOL_VERSION, type SocketFrame } from '@/lib/actions/protocol';
import { CHROME_WEB_STORE } from '@/lib/stores';
import { clearAuth } from '../auth-store';
import { startDaemon, type Daemon } from '../daemon';
import { readLockfile } from '../lockfile';
import { RemoteBridge } from '../remote-bridge';
+import { FakeAndroid, NO_PHONE, READY_PHONE } from './fake-android';
import { FakeBrowser, type Profile } from './fake-browser';
const store: Profile = {
@@ -14,10 +15,12 @@ const store: Profile = {
};
let daemon: Daemon;
+let android: FakeAndroid;
let opened: { close(): unknown }[] = [];
beforeAll(async () => {
- daemon = await startDaemon({ version: '0.0.0-test', idleExit: false });
+ android = new FakeAndroid();
+ daemon = await startDaemon({ version: '0.0.0-test', idleExit: false, android });
});
afterAll(async () => {
@@ -29,6 +32,8 @@ afterEach(async () => {
opened = [];
await new Promise((resolve) => setTimeout(resolve, 20));
clearAuth();
+ android.current = NO_PHONE;
+ android.launched.length = 0;
});
async function control(): Promise<RemoteBridge> {
@@ -76,3 +81,76 @@ describe('the protocol window', () => {
expect([session.source, session.connected]).toEqual(['chrome-web-store', true]);
});
});
+
+const ANDROID_FRAMES: ReadonlySet<SocketFrame['t']> = new Set(['androidInfo', 'phoneOpened', 'cdpResult', 'cdpEvent', 'phoneClosed']);
+
+describe('Android across the protocol window', () => {
+ const store22 = { ...store, protocolVersion: ANDROID_PROTOCOL - 1 };
+
+ test('a protocol-22 store copy is sent no Android frame, and its phone requests are refused with an answer', async () => {
+ const browser = await pair(store22);
+ android.change(READY_PHONE);
+ await browser.focus();
+ expect(browser.received.map(({ t }) => t).filter((t) => ANDROID_FRAMES.has(t))).toEqual([]);
+ expect(android.watchers).toBe(0);
+
+ const refused = { ok: false, error: { code: 'UNSUPPORTED' } };
+ expect(await browser.ask({ t: 'phoneOpen', id: 'open-1', serial: 'emulator-5554' })).toMatchObject({ t: 'phoneOpened', result: refused });
+ expect(await browser.ask({ t: 'cdp', id: 'cdp-1', serial: 'emulator-5554', method: 'Browser.getVersion' })).toMatchObject({
+ t: 'cdpResult',
+ result: refused,
+ });
+ expect(await browser.ask({ t: 'phoneLaunch', id: 'launch-1', serial: 'emulator-5554' })).toMatchObject({ t: 'androidInfo', result: refused });
+ expect(android.launched).toEqual([]);
+ });
+
+ test('a protocol-23 extension is sent the state on connect and on every change, and stops watching when it leaves', async () => {
+ const browser = await pair({ ...store, protocolVersion: ANDROID_PROTOCOL });
+ const [first] = await browser.frames('androidInfo');
+ expect(first).toEqual({ t: 'androidInfo', id: '', result: { ok: true, data: NO_PHONE } });
+ expect(android.watchers).toBe(1);
+
+ android.change(READY_PHONE);
+ const [, second] = await browser.frames('androidInfo', 2);
+ expect(second.result).toEqual({ ok: true, data: READY_PHONE });
+
+ browser.close();
+ await browser.closed;
+ await new Promise((resolve) => setTimeout(resolve, 20));
+ expect(android.watchers).toBe(0);
+ });
+
+ test('a protocol-23 extension can ask the Bridge to open Chrome on the phone', async () => {
+ android.current = READY_PHONE;
+ const browser = await pair({ ...store, protocolVersion: ANDROID_PROTOCOL });
+ expect(await browser.ask({ t: 'phoneLaunch', id: 'launch-2', serial: 'emulator-5554' })).toEqual({
+ t: 'androidInfo',
+ id: 'launch-2',
+ result: { ok: true, data: READY_PHONE },
+ });
+ expect(android.launched).toEqual(['emulator-5554']);
+ });
+
+ test('the control op answers the state, and a watcher hears it and each change', async () => {
+ const bridge = await control();
+ const heard: unknown[] = [];
+ bridge.onAndroidChanged((state) => heard.push(state));
+ expect(await bridge.android({ watch: true })).toEqual({ ok: true, data: NO_PHONE });
+ android.change(READY_PHONE);
+ await bridge.status();
+ expect(heard).toEqual([NO_PHONE, READY_PHONE]);
+ bridge.close();
+ await new Promise((resolve) => setTimeout(resolve, 20));
+ expect(android.watchers).toBe(0);
+ });
+
+ test('an agent run cannot open Chrome on the phone through the control socket', async () => {
+ const run = await RemoteBridge.connect(daemon.port, readLockfile()!.token, 'run-1');
+ opened.push(run);
+ await run.describe();
+ expect(await run.android({ launch: 'emulator-5554' })).toMatchObject({ ok: false, error: { code: 'BLOCKED' } });
+ expect(android.launched).toEqual([]);
+ expect(await (await control()).android({ launch: 'emulator-5554' })).toEqual({ ok: true, data: NO_PHONE });
+ expect(android.launched).toEqual(['emulator-5554']);
+ });
+});
diff --git a/src/lib/actions/protocol.ts b/src/lib/actions/protocol.ts
index 5738033..3ddf475 100644
--- a/src/lib/actions/protocol.ts
+++ b/src/lib/actions/protocol.ts
@@ -4,6 +4,7 @@ import type { FileReport, FileVerdict } from '@/lib/files/report';
import type { MonitorSample } from '@/lib/monitor/events';
import type { RecordedEvent } from '@/lib/recordings/events';
import type { RecordingWorkflow } from '@/lib/recordings/workflow';
+import type { AndroidState, PhoneClosedReason, PhoneContext, PhoneOpened } from '@/lib/phone/types';
import type { PreferenceChange, Preferences } from '@/lib/settings/preferences';
import type { SkillCategory, SkillDraft } from '@/lib/skills/format';
import type { SiteMapDraft } from '@/lib/skills/site-map';
@@ -12,7 +13,10 @@ import type { TaskContext, TaskList, TaskOrder, TaskResult } from '@/lib/schedul
export const ACTION_CHANNEL = 'browsentic/action';
export const BRIDGE_CHANNEL = 'browsentic/bridge';
-export const SOCKET_PROTOCOL_VERSION = 22;
+export const SOCKET_PROTOCOL_VERSION = 23;
+
+/** The first protocol with the Android frames: `androidInfo`, the phone requests, `cdpEvent` and `phoneClosed`. */
+export const ANDROID_PROTOCOL = 23;
/**
* The oldest extension the daemon lets in. A store copy updates when its browser decides to, so it
@@ -167,6 +171,8 @@ export interface RunContext {
liveTools?: boolean;
/** Set when a schedule started this run: nobody is watching it, and it should end on one line saying what it found. */
task?: TaskContext;
+ /** Set when the run drives Chrome on an Android phone through the mirror tab. */
+ phone?: PhoneContext;
}
/** One skill from the active agent CLI's own library. Title and handle only — the file's path and content stay on the daemon's side. */
@@ -276,7 +282,16 @@ export type SocketFrame =
| { t: 'pauseTasks'; id: string; paused: boolean }
| { t: 'taskDone'; id: string; taskId: string; result: TaskResult }
| { t: 'taskList'; id: string; result: ActionResult<TaskList> }
- | { t: 'runTask'; id: string; order: TaskOrder };
+ | { t: 'runTask'; id: string; order: TaskOrder }
+ | { t: 'androidInfo'; id: string; result: ActionResult<AndroidState> }
+ | { t: 'phoneOpen'; id: string; serial: string }
+ | { t: 'phoneOpened'; id: string; result: ActionResult<PhoneOpened> }
+ | { t: 'phoneClose'; id: string; serial: string }
+ | { t: 'phoneLaunch'; id: string; serial: string; url?: string }
+ | { t: 'cdp'; id: string; serial: string; method: string; params?: Record<string, unknown>; sessionId?: string }
+ | { t: 'cdpResult'; id: string; result: ActionResult<Record<string, unknown>> }
+ | { t: 'cdpEvent'; serial: string; method: string; params: Record<string, unknown>; sessionId?: string }
+ | { t: 'phoneClosed'; serial: string; reason: PhoneClosedReason };
export interface RecordingAnalysis {
workflow: RecordingWorkflow;
@@ -316,6 +331,10 @@ export const EXTENSION_REQUEST_FRAMES = [
'runTaskNow',
'pauseTasks',
'taskDone',
+ 'phoneOpen',
+ 'phoneClose',
+ 'phoneLaunch',
+ 'cdp',
] as const;
export type ExtensionRequest = Extract<SocketFrame, { t: (typeof EXTENSION_REQUEST_FRAMES)[number] }>;
diff --git a/src/lib/bridge/socket-window.test.ts b/src/lib/bridge/socket-window.test.ts
new file mode 100644
index 0000000..a9fbedf
--- /dev/null
+++ b/src/lib/bridge/socket-window.test.ts
@@ -0,0 +1,84 @@
+import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, test, vi } from 'vitest';
+import { fakeBrowser } from 'wxt/testing';
+import { WebSocketServer, type WebSocket } from 'ws';
+import { newNonce, serverProof, type Transcript } from '@/lib/actions/handshake';
+import { ANDROID_PROTOCOL, parseFrame, type SocketFrame } from '@/lib/actions/protocol';
+import type { DaemonState } from './socket';
+
+const ports = vi.hoisted(() => [] as number[]);
+vi.mock('@/lib/actions/protocol', async (importOriginal) => ({
+ ...(await importOriginal<object>()),
+ DAEMON_PORTS: ports,
+}));
+
+const SESSION_KEY = 'session-key-of-a-paired-browser';
+const RELEASED_BRIDGE = { daemonVersion: '0.8.1', protocolVersion: 22 };
+/** A fresh import of the worker is slow once the whole suite shares the machine, as in socket.test.ts. */
+const TEST_TIMEOUT_MS = 20_000;
+
+let bridge: WebSocketServer;
+const heard: SocketFrame[] = [];
+
+/** Browsentic Bridge 0.8.1 as far as the handshake goes: it speaks 22 and proves it holds the session key. */
+function released(ws: WebSocket): void {
+ let transcript: Transcript | undefined;
+ const send = (frame: SocketFrame) => ws.send(JSON.stringify(frame));
+ ws.on('message', async (raw) => {
+ const frame = parseFrame(String(raw));
+ if (frame) heard.push(frame);
+ if (frame?.t === 'hello') {
+ transcript = {
+ protocolVersion: frame.protocolVersion,
+ extensionVersion: frame.extensionVersion,
+ manifestHash: frame.manifestHash,
+ clientNonce: frame.nonce,
+ serverNonce: newNonce(),
+ };
+ send({ t: 'challenge', nonce: transcript.serverNonce });
+ }
+ if (frame?.t === 'prove' && transcript) {
+ const welcome = { daemonVersion: RELEASED_BRIDGE.daemonVersion, manifestHash: transcript.manifestHash, manifestInSync: true };
+ const proof = await serverProof(SESSION_KEY, transcript, welcome);
+ send({ t: 'welcome', ...welcome, protocolVersion: RELEASED_BRIDGE.protocolVersion, proof });
+ }
+ });
+}
+
+beforeAll(async () => {
+ bridge = new WebSocketServer({ host: '127.0.0.1', port: 0 });
+ bridge.on('connection', released);
+ await new Promise((resolve) => bridge.once('listening', resolve));
+ const address = bridge.address();
+ if (typeof address === 'object' && address) ports.push(address.port);
+});
+
+afterAll(() => new Promise((resolve) => bridge.close(resolve)));
+
+beforeEach(() => {
+ fakeBrowser.reset();
+ vi.resetModules();
+ vi.spyOn(fakeBrowser.runtime, 'getManifest').mockReturnValue({ version: '0.0.0-test' } as never);
+});
+
+let worker: typeof import('./socket') | undefined;
+afterEach(() => worker?.disconnectDaemon());
+
+async function daemonState(): Promise<DaemonState | undefined> {
+ return (await fakeBrowser.storage.session.get('browsentic/daemon'))['browsentic/daemon'] as DaemonState | undefined;
+}
+
+describe('an extension against a Bridge from before Android', () => {
+ test('connects, and knows the Bridge does not speak Android', async () => {
+ await fakeBrowser.storage.local.set({ 'browsentic/sessionKey': SESSION_KEY });
+ worker = await import('./socket');
+ await worker.connectDaemon();
+ for (let tries = 0; !(await daemonState())?.connected; tries++) {
+ if (tries > 150) throw new Error('the link never came online');
+ await new Promise((resolve) => setTimeout(resolve, 20));
+ }
+ expect(await daemonState()).toMatchObject({ connected: true, ...RELEASED_BRIDGE });
+ expect(worker.daemonSpeaks(RELEASED_BRIDGE.protocolVersion)).toBe(true);
+ expect(worker.daemonSpeaks(ANDROID_PROTOCOL)).toBe(false);
+ expect(heard.find((frame) => frame.t === 'hello')).toMatchObject({ protocolVersion: ANDROID_PROTOCOL });
+ }, TEST_TIMEOUT_MS);
+});
diff --git a/src/lib/bridge/socket.test.ts b/src/lib/bridge/socket.test.ts
index 6239926..dc74f8d 100644
--- a/src/lib/bridge/socket.test.ts
+++ b/src/lib/bridge/socket.test.ts
@@ -5,6 +5,7 @@ import { fakeBrowser } from 'wxt/testing';
import { startDaemon, type Daemon } from '@/daemon/daemon';
import { logPath, readLockfile } from '@/daemon/lockfile';
import { RemoteBridge } from '@/daemon/remote-bridge';
+import { ANDROID_PROTOCOL, SOCKET_PROTOCOL_VERSION } from '@/lib/actions/protocol';
import type { DaemonState } from './socket';
const ports = vi.hoisted(() => [] as number[]);
@@ -90,4 +91,14 @@ describe('keeping one link to the daemon', () => {
expect((await daemonState())?.connected).toBe(true);
}, TEST_TIMEOUT_MS);
+ test('the welcome says which protocol the Bridge speaks, and this one speaks Android', async () => {
+ const socket = await revivedWorker();
+ expect(socket.daemonSpeaks(ANDROID_PROTOCOL)).toBe(false);
+ await socket.connectDaemon();
+ await online();
+ expect((await daemonState())?.protocolVersion).toBe(SOCKET_PROTOCOL_VERSION);
+ expect(socket.daemonSpeaks(ANDROID_PROTOCOL)).toBe(true);
+ await socket.disconnectDaemon();
+ expect(socket.daemonSpeaks(ANDROID_PROTOCOL)).toBe(false);
+ }, TEST_TIMEOUT_MS);
});
diff --git a/src/lib/bridge/socket.ts b/src/lib/bridge/socket.ts
index 4c2fdcb..24e7dfc 100644
--- a/src/lib/bridge/socket.ts
+++ b/src/lib/bridge/socket.ts
@@ -74,6 +74,8 @@ export interface DaemonState {
paired: boolean;
port?: number;
daemonVersion?: string;
+ /** What the Bridge speaks. Absent from a Bridge before 0.8, which speaks 22 or less. */
+ protocolVersion?: number;
manifestInSync?: boolean;
/** Which agent CLI the daemon runs, and what each one's state is. Pushed on connect. */
agent?: AgentState;
@@ -87,6 +89,7 @@ export interface DaemonState {
let socket: WebSocket | null = null;
let welcomedSocket: WebSocket | null = null;
+let daemonProtocol = 0;
let attempts = 0;
let connecting: Promise<void> | null = null;
let reconnectTimer: ReturnType<typeof setTimeout> | undefined;
@@ -588,6 +591,7 @@ async function handle(ws: WebSocket, raw: string, attempt: Attempt): Promise<voi
attempt.done();
welcomedSocket = ws;
+ daemonProtocol = frame.protocolVersion ?? 0;
// A link that is welcomed and then dropped at once has not recovered, and forgetting the
// backoff on the welcome alone lets two peers take the link from each other every second.
clearTimeout(stableTimer);
@@ -599,6 +603,7 @@ async function handle(ws: WebSocket, raw: string, attempt: Attempt): Promise<voi
paired: true,
port: attempt.port,
daemonVersion: frame.daemonVersion,
+ protocolVersion: frame.protocolVersion,
manifestInSync: frame.manifestInSync,
error: undefined,
lastChangeAt: Date.now(),
@@ -692,7 +697,15 @@ async function handle(ws: WebSocket, raw: string, attempt: Attempt): Promise<voi
/** Held back until the daemon has welcomed us: mid-handshake it would be read as the proof. */
export function reportFocus(): void {
- if (socket && socket === welcomedSocket) post({ t: 'focus' });
+ if (isWelcomed()) post({ t: 'focus' });
+}
+
+export function daemonSpeaks(version: number): boolean {
+ return isWelcomed() && daemonProtocol >= version;
+}
+
+function isWelcomed(): boolean {
+ return !!socket && socket === welcomedSocket;
}
function send(ws: WebSocket, frame: SocketFrame): void {
diff --git a/src/lib/phone/types.ts b/src/lib/phone/types.ts
new file mode 100644
index 0000000..e55188a
--- /dev/null
+++ b/src/lib/phone/types.ts
@@ -0,0 +1,78 @@
+export type AndroidProblemCode =
+ | 'ANDROID_OFF'
+ | 'ADB_MISSING'
+ | 'ADB_BROKEN'
+ | 'NO_DEVICE'
+ | 'DEVICE_UNAUTHORIZED'
+ | 'DEVICE_OFFLINE'
+ | 'DEVICE_BOOTING'
+ | 'NO_PERMISSIONS'
+ | 'CHROME_MISSING'
+ | 'CHROME_NOT_RUNNING'
+ | 'SCREEN_OFF';
+
+export type AndroidFixAction = 'launchChrome' | 'installChrome';
+
+export interface AndroidProblem {
+ code: AndroidProblemCode;
+ message: string;
+ fix?: string;
+ action?: AndroidFixAction;
+}
+
+export type AndroidDeviceState = 'ready' | 'unauthorized' | 'offline' | 'no-permissions' | 'other';
+
+export interface AndroidDevice {
+ serial: string;
+ transport: 'usb' | 'wifi';
+ state: AndroidDeviceState;
+ model?: string;
+ manufacturer?: string;
+ android?: string;
+ sdk?: number;
+ /** Physical pixels, after any override the phone renders at. */
+ screen?: { width: number; height: number; density: number; awake: boolean };
+ chrome: {
+ installed: boolean;
+ version?: string;
+ /** Android starts Chrome's process at boot, so a running Chrome is not yet one that can be driven. */
+ running: boolean;
+ debuggable: boolean;
+ };
+ /** An advisory such as `SCREEN_OFF` can sit on a phone that is ready. */
+ problem?: AndroidProblem;
+}
+
+export interface AndroidState {
+ /** Off when config.json says `android.enabled: false`, and then the Bridge runs no adb at all. */
+ enabled: boolean;
+ ready: boolean;
+ adb: { found: boolean; path?: string; version?: string; problem?: AndroidProblem };
+ devices: AndroidDevice[];
+ /** The first thing between the user and a ready phone, or an advisory once one is ready. */
+ problem?: AndroidProblem;
+ session?: { serial: string; since: string };
+}
+
+export interface PhoneTarget {
+ targetId: string;
+ url: string;
+ title: string;
+}
+
+export interface PhoneOpened {
+ device: AndroidDevice;
+ targets: PhoneTarget[];
+ browserVersion: string;
+}
+
+export type PhoneClosedReason = 'unplugged' | 'chrome-exited' | 'bridge-stopping' | 'closed';
+
+export interface PhoneContext {
+ serial: string;
+ model: string;
+ android: string;
+ chrome: string;
+ /** CSS px of the visual viewport. */
+ viewport: { width: number; height: number; dpr: number };
+}