Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

feat(firefox): add optional BiDi trusted automation companion

Emre Sokullu644eed37d111
diff --git a/firefox-companion/README.md b/firefox-companion/README.md
new file mode 100644
index 000000000..9b717bd6d
--- /dev/null
+++ b/firefox-companion/README.md
@@ -0,0 +1,47 @@
+# Firefox trusted automation (experimental)
+
+The optional local companion connects WebBrain to Firefox WebDriver BiDi. Standard Firefox extension operation does not need it. Chrome continues to use its existing backend.
+
+## Setup
+
+Requires Node.js 22 or newer with the built-in WebSocket API, and a current Firefox. Tested on Firefox 155.0.1 on macOS. macOS and Linux installation is supported; Windows is not yet validated.
+
+1. From this checkout, run `node firefox-companion/install.mjs`. This installs a native-messaging manifest and launcher for the current user. Keep the checkout and Node executable in their installed locations; rerun the installer after moving either.
+2. Quit Firefox, then start it with remote automation enabled:
+
+ - macOS: `/Applications/Firefox.app/Contents/MacOS/firefox --remote-debugging-port 9222`
+ - Linux: `firefox --remote-debugging-port 9222`
+
+ Use a separate profile if desired. An already running Firefox process may ignore startup flags. Keep the debugging endpoint on loopback; it grants browser control to local clients.
+3. Load this branch's `src/firefox/manifest.json` from `about:debugging` → This Firefox → Load Temporary Add-on (or install a build containing these changes).
+4. In WebBrain Settings → General → Advanced, enable **Firefox trusted automation**, keep the port at 9222 (or match your startup port), and select **Test connection**.
+5. Start a new Act or Dev task. Connection failures fail the task rather than silently falling back. Ask mode does not open an automation run. Tasks can start on blank/internal tabs; document binding is deferred until authorized navigation reaches a web page.
+
+The connection test checks the real BiDi session. It does not submit forms or send input. Disabling the setting or changing the port disconnects the helper; restart any active task afterward.
+
+## First supported scope
+
+- Session connection through Firefox native messaging to loopback BiDi; no arbitrary protocol or script tool exposed to the model.
+- Trusted clicks (`click`, `click_ax`), hover, text (`type_text`, `type_ax`, `set_field`), and existing supported `press_keys` keys/repeats. Native selects retain their existing semantic selection path. Drag/drop and dedicated iframe tools retain their existing implementations.
+- Typing dispatches one character at a time, checking run ownership, deadline and focus before each step. Stop prevents the next step; a single already-dispatched character can still arrive. Newlines use a literal editing command, never Enter; single-line fields reject multiline payloads before clearing.
+- Exact value readback after typing. `set_field` submits with Enter only after verification. An uncertain dispatch is not safe to repeat automatically.
+- Active-run dialogs: acknowledge alerts; dismiss confirmations and prompts; stay on beforeunload unless an explicit navigate action is in progress. A Leave allowance is single use. This prevents prompts from hanging input without blanket approval of destructive confirmations. Idle tabs are not auto-handled. Pending dialog events are retained and handled when a task takes ownership; manually closed dialogs are discarded. Browser permission dialogs and arbitrary operating-system dialogs are outside this scope.
+- Uploads keep the existing WebBrain attachment/download/picker authorization. Bytes are written to a private temporary file, passed to `input.setFiles`, and the file input's name and size are checked. The input must be unique, enabled, and in the current document's light DOM. Attachment is not proof the site has uploaded or submitted the file. Arbitrary model-supplied filesystem paths are not accepted.
+
+Target markers bind the extension-selected page element to a BiDi shared node. Duplicate or stale bindings fail. Existing permission, recipient, toolbar and dispatch guards run before target preparation. Native input rechecks connection, run ownership, and target visibility before dispatch. Page DOM is not a security boundary against a hostile page rearranging itself between validation and input; existing prompt-injection and action-authorization checks remain necessary.
+
+The host stores temporary upload files only for its connection lifetime and deletes them on normal shutdown. A forced process kill can leave `webbrain-bidi-*` directories in the OS temporary directory. It does not log file bytes or typed text. Only the extension ID `[email protected]` is allowed to connect to the native host.
+
+## Testing
+
+- `npm run test:firefox-bidi`: deterministic cancellation, disconnection, dialog policy and binding regressions (also in `npm test`).
+- `npm run test:firefox-bidi:e2e`: starts an isolated headless Firefox profile and local fixture, exercises the production BiDi session, verifies `isTrusted`, dialog dismissal, exact typing, and uploaded bytes, then removes the profile. Set `FIREFOX_BINARY` if Firefox is not in the macOS default location. It also temporarily installs the packaged extension and exercises its content-script target preparation. The isolated test enables system access solely to inspect its extension Settings page; normal companion use does not need that flag. It does not install the native host in your normal browser profile.
+
+## Uninstall
+
+Disable trusted automation in Settings and close active tasks. Remove `one.webbrain.bidi.json` and `webbrain-bidi` from:
+
+- macOS: `~/Library/Application Support/Mozilla/NativeMessagingHosts/`
+- Linux: `~/.mozilla/native-messaging-hosts/`
+
+Restart Firefox without the remote-debugging flag when it is no longer needed.
diff --git a/firefox-companion/host.mjs b/firefox-companion/host.mjs
new file mode 100644
index 000000000..ef03d5086
--- /dev/null
+++ b/firefox-companion/host.mjs
@@ -0,0 +1,34 @@
+#!/usr/bin/env node
+import { BidiSession } from './session.mjs';
+const session = new BidiSession();
+let buffer = Buffer.alloc(0);
+function reply(value) {
+ const data = Buffer.from(JSON.stringify(value));
+ const header = Buffer.alloc(4); header.writeUInt32LE(data.length);
+ process.stdout.write(Buffer.concat([header, data]));
+}
+async function dispatch(message) {
+ const { id, command, ...args } = message;
+ try {
+ let result;
+ switch (command) {
+ case 'connect': result = await session.connect(args.port); break;
+ case 'openRun': result = await session.openRun(args.runId, args.token, args.url); break;
+ case 'closeRun': result = await session.closeRun(args.runId); break;
+ case 'perform': result = await session.perform(args.runId, args.action, args.payload || {}); break;
+ default: throw new Error('Unknown companion command');
+ }
+ reply({ id, result });
+ } catch (error) { reply({ id, error: error.message }); }
+}
+process.stdin.on('data', data => {
+ buffer = Buffer.concat([buffer, data]);
+ while (buffer.length >= 4) {
+ const length = buffer.readUInt32LE(0);
+ if (length > 40 * 1024 * 1024) process.exit(1);
+ if (buffer.length < length + 4) break;
+ const data = buffer.subarray(4, length + 4); buffer = buffer.subarray(length + 4);
+ try { void dispatch(JSON.parse(data)); } catch { process.exit(1); }
+ }
+});
+process.stdin.on('end', () => { void session.close().finally(() => process.exit()); });
diff --git a/firefox-companion/install.mjs b/firefox-companion/install.mjs
new file mode 100644
index 000000000..22ce05eaa
--- /dev/null
+++ b/firefox-companion/install.mjs
@@ -0,0 +1,19 @@
+import { mkdir, writeFile } from 'node:fs/promises';
+import { homedir } from 'node:os';
+import { dirname, join, resolve } from 'node:path';
+import { fileURLToPath } from 'node:url';
+const root = dirname(fileURLToPath(import.meta.url));
+const target = process.argv[2] || (process.platform === 'darwin'
+ ? join(homedir(), 'Library/Application Support/Mozilla/NativeMessagingHosts')
+ : process.platform === 'linux' ? join(homedir(), '.mozilla/native-messaging-hosts') : null);
+if (!target) throw new Error('On Windows, pass an installation directory and register its manifest under HKCU\\Software\\Mozilla\\NativeMessagingHosts\\one.webbrain.bidi');
+await mkdir(target, { recursive: true });
+const wrapper = resolve(target, process.platform === 'win32' ? 'webbrain-bidi.cmd' : 'webbrain-bidi');
+const quote = text => `'${text.replaceAll("'", "'\\''")}'`;
+await writeFile(wrapper, process.platform === 'win32'
+ ? `@echo off\r\n"${process.execPath}" "${join(root, 'host.mjs')}"\r\n`
+ : `#!/bin/sh\nexec ${quote(process.execPath)} ${quote(join(root, 'host.mjs'))}\n`, { mode: 0o700 });
+const manifest = { name: 'one.webbrain.bidi', description: 'WebBrain Firefox BiDi companion', path: wrapper, type: 'stdio', allowed_extensions: ['[email protected]'] };
+const path = resolve(target, 'one.webbrain.bidi.json');
+await writeFile(path, JSON.stringify(manifest, null, 2), { mode: 0o600 });
+console.log(`Installed companion manifest: ${path}`);
diff --git a/firefox-companion/session.mjs b/firefox-companion/session.mjs
new file mode 100644
index 000000000..2037ed133
--- /dev/null
+++ b/firefox-companion/session.mjs
@@ -0,0 +1,242 @@
+import { mkdtemp, writeFile, rm } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join, basename } from 'node:path';
+
+export class BidiSession {
+ constructor(WebSocketImpl = WebSocket) {
+ this.WebSocketImpl = WebSocketImpl;
+ this.pending = new Map(); this.runs = new Map(); this.sequence = 0; this.files = []; this.closedRuns = new Set(); this.dialogs = new Map();
+ }
+ connect(port = 9222) {
+ if (!this.connection) this.connection = this.connectSession(port).catch(error => {
+ this.socket?.close(); this.connection = null; throw error;
+ });
+ return this.connection;
+ }
+ async connectSession(port) {
+ if (!Number.isInteger(port) || port < 1024 || port > 65535) throw new Error('Invalid local Firefox port');
+ if (this.socket?.readyState === 1) return { connected: true };
+ const socket = this.socket = new this.WebSocketImpl(`ws://127.0.0.1:${port}/session`);
+ socket.addEventListener('message', event => this.receive(JSON.parse(event.data)));
+ socket.addEventListener('close', () => {
+ for (const entry of this.pending.values()) entry.reject(new Error('Firefox disconnected; action outcome may be unknown'));
+ this.pending.clear(); this.runs.clear(); this.dialogs.clear(); this.connection = null;
+ });
+ await new Promise((resolve, reject) => {
+ const timer = setTimeout(() => { socket.close(); reject(new Error('Start Firefox with --remote-debugging-port')); }, 5000);
+ socket.addEventListener('open', () => { clearTimeout(timer); resolve(); }, { once: true });
+ socket.addEventListener('error', () => { clearTimeout(timer); reject(new Error('Cannot connect to Firefox BiDi')); }, { once: true });
+ });
+ await this.send('session.new', { capabilities: { alwaysMatch: { unhandledPromptBehavior: { default: 'ignore' } } } });
+ await this.send('session.subscribe', { events: ['browsingContext.userPromptOpened', 'browsingContext.userPromptClosed', 'browsingContext.contextDestroyed'] });
+ return { connected: true };
+ }
+ send(method, params) {
+ if (this.socket?.readyState !== 1) return Promise.reject(new Error('Firefox BiDi is disconnected'));
+ const id = ++this.sequence;
+ return new Promise((resolve, reject) => {
+ const timer = setTimeout(() => { this.pending.delete(id); reject(new Error(`${method} timed out; do not repeat an uncertain action`)); }, 10000);
+ this.pending.set(id, { resolve: value => { clearTimeout(timer); resolve(value); }, reject: error => { clearTimeout(timer); reject(error); } });
+ try { this.socket.send(JSON.stringify({ id, method, params })); }
+ catch (error) { this.pending.get(id).reject(error); this.pending.delete(id); }
+ });
+ }
+ receive(message) {
+ if (message.id != null) {
+ const entry = this.pending.get(message.id); if (!entry) return;
+ this.pending.delete(message.id);
+ if (message.type === 'error') entry.reject(new Error(message.error || 'BiDi command failed'));
+ else entry.resolve(message.result);
+ return;
+ }
+ const params = message.params || {};
+ if (message.method === 'browsingContext.contextDestroyed') {
+ for (const [id, run] of this.runs) if (run.context === params.context) this.runs.delete(id);
+ }
+ if (message.method === 'browsingContext.userPromptClosed' || message.method === 'browsingContext.contextDestroyed') this.dialogs.delete(params.context);
+ if (message.method === 'browsingContext.userPromptOpened') {
+ this.dialogs.set(params.context, { type: params.type });
+ const run = [...this.runs.values()].find(item => item.context === params.context);
+ if (run) void this.handleDialog(run).catch(() => {});
+ }
+ }
+ async handleDialog(run) {
+ const dialog = this.dialogs.get(run.context);
+ if (!dialog) return;
+ if (dialog.handling) return dialog.handling;
+ const accept = dialog.type === 'alert' || (dialog.type === 'beforeunload' && run.navigation === true);
+ if (dialog.type === 'beforeunload') run.navigation = false;
+ dialog.handling = this.send('browsingContext.handleUserPrompt', { context: run.context, accept });
+ try { await dialog.handling; }
+ finally { if (this.dialogs.get(run.context) === dialog) this.dialogs.delete(run.context); }
+ }
+ async locate(token, url, context = null) {
+ if (!/^[a-f0-9-]{36}$/.test(token) || !/^https?:\/\//.test(url)) throw new Error('Invalid tab binding');
+ const tree = await this.send('browsingContext.getTree', context ? { root: context } : {});
+ const contexts = [];
+ const visit = nodes => { for (const node of nodes || []) { contexts.push(node); visit(node.children); } };
+ visit(tree.contexts);
+ if (contexts.length > 128) throw new Error('Too many Firefox contexts to bind safely');
+ const matches = [];
+ for (const frame of contexts) {
+ if (frame.url !== url) continue;
+ const found = await this.send('browsingContext.locateNodes', {
+ context: frame.context, locator: { type: 'css', value: `[data-webbrain-bidi="${token}"]` },
+ });
+ for (const node of found.nodes || []) matches.push({ context: frame.context, node });
+ }
+ if (matches.length !== 1) throw new Error('Tab or element binding is stale or ambiguous; re-read the page');
+ return matches[0];
+ }
+ async openRun(id, token, url) {
+ if (!/^[a-f0-9-]{36}$/.test(id) || this.runs.has(id) || this.closedRuns.has(id)) throw new Error('Invalid run');
+ const match = await this.locate(token, url);
+ if (this.closedRuns.has(id)) throw new Error('Run stopped during binding');
+ this.runs.set(id, { context: match.context, navigation: false });
+ await this.handleDialog(this.runs.get(id));
+ if (this.closedRuns.has(id)) throw new Error('Run stopped during binding');
+ await this.call(match, '(el) => { el.removeAttribute("data-webbrain-bidi"); return true; }');
+ return { connected: true };
+ }
+ call(match, functionDeclaration, args = []) {
+ return this.send('script.callFunction', { target: { context: match.context }, functionDeclaration,
+ arguments: [{ sharedId: match.node.sharedId }, ...args], awaitPromise: true });
+ }
+ async closeRun(id) {
+ this.closedRuns.add(id);
+ if (this.closedRuns.size > 4096) this.closedRuns.delete(this.closedRuns.values().next().value);
+ const run = this.runs.get(id); this.runs.delete(id);
+ if (run) await this.send('input.releaseActions', { context: run.context }).catch(() => {});
+ return {};
+ }
+ async perform(id, action, payload) {
+ if (!['navigate', 'click', 'hover', 'type', 'field', 'key', 'upload'].includes(action)) throw new Error('Unsupported BiDi action');
+ if (payload.modifiers) throw new Error('Key modifiers are not supported by this tool');
+ const run = this.runs.get(id); if (!run) throw new Error('Run stopped or disconnected');
+ const assertLive = () => {
+ if (this.runs.get(id) !== run) throw new Error('Run stopped');
+ if (payload.deadlineAt && Date.now() >= payload.deadlineAt) throw new Error('Action deadline expired');
+ };
+ assertLive();
+ if (action === 'navigate') {
+ if (!/^https?:\/\//.test(payload.url)) throw new Error('Invalid navigation URL');
+ run.navigation = true;
+ try { await this.send('browsingContext.navigate', { context: run.context, url: payload.url, wait: 'interactive' }); }
+ finally { run.navigation = false; }
+ return { success: true, dispatched: true };
+ }
+ const match = await this.locate(payload.token, payload.url, run.context);
+ assertLive();
+ const check = await this.call(match, `(el, token, action) => {
+ if (!el.isConnected || el.getAttribute('data-webbrain-bidi') !== token || el.disabled) return false;
+ el.removeAttribute('data-webbrain-bidi');
+ if (action === 'upload') return el.tagName === 'INPUT' && el.type === 'file';
+ const r = el.getBoundingClientRect();
+ const hit = document.elementFromPoint(r.x+r.width/2, r.y+r.height/2);
+ if (!r.width || !r.height || !(hit === el || el.contains(hit))) return false;
+ if (action === 'type' || action === 'field' || action === 'key') {
+ el.focus({preventScroll:true});
+ if (el.getRootNode().activeElement !== el) return false;
+ }
+ return true;
+ }`, [{ type: 'string', value: payload.token }, { type: 'string', value: action }]);
+ if (check.result?.value !== true) throw new Error('Target changed or is covered; no input sent');
+ assertLive();
+ if (action === 'upload') {
+ if (typeof payload.base64 !== 'string' || payload.base64.length > 36 * 1024 * 1024) throw new Error('Upload exceeds 25MB');
+ const bytes = Buffer.from(payload.base64, 'base64');
+ if (bytes.length > 25 * 1024 * 1024) throw new Error('Upload exceeds 25MB');
+ const dir = await mkdtemp(join(tmpdir(), 'webbrain-bidi-')); this.files.push(dir);
+ const name = basename(String(payload.filename || 'attachment')).replace(/[^a-zA-Z0-9._-]/g, '_');
+ const path = join(dir, name === '.' || name === '..' ? 'attachment' : name);
+ await writeFile(path, bytes, { mode: 0o600 });
+ assertLive();
+ await this.send('input.setFiles', { context: match.context, element: { sharedId: match.node.sharedId }, files: [path] });
+ const attached = await this.call(match, '(el, name, size) => el.isConnected && el.files?.length === 1 && el.files[0].name === name && el.files[0].size === size', [{ type: 'string', value: name }, { type: 'number', value: bytes.length }]);
+ if (attached.result?.value !== true) return { success: false, dispatched: true, outcomeUnknown: true, retryable: false, error: 'File input changed after attachment; inspect the page before retrying.' };
+ return { success: true, dispatched: true, attachmentState: 'input_attached', file: name, size: bytes.length };
+ }
+ const typing = action === 'type' || action === 'field';
+ const clear = action === 'field' ? payload.clear !== false : payload.clear === true;
+ const before = typing ? await this.call(match, '(el) => el.isContentEditable ? el.innerText : el.value') : null;
+ const assertFocus = async () => {
+ assertLive();
+ const focused = await this.call(match, '(el) => el.isConnected && !el.disabled && el.getRootNode().activeElement === el');
+ if (focused.result?.value !== true) throw new Error('Focus changed; no further keys sent');
+ assertLive();
+ };
+ const press = async (value, modifier = null, followTabFocus = false) => {
+ if (followTabFocus) assertLive();
+ else await assertFocus();
+ const actions = [
+ ...(modifier ? [{ type: 'keyDown', value: modifier }] : []),
+ { type: 'keyDown', value }, { type: 'keyUp', value },
+ ...(modifier ? [{ type: 'keyUp', value: modifier }] : []),
+ ];
+ try { await this.send('input.performActions', { context: match.context, actions: [{ type: 'key', id: 'webbrain-keyboard', actions }] }); }
+ finally { await this.send('input.releaseActions', { context: match.context }).catch(() => {}); }
+ assertLive();
+ };
+ if (action === 'click' || action === 'hover') {
+ const source = { type: 'pointer', id: 'webbrain-pointer', parameters: { pointerType: 'mouse' }, actions: [
+ { type: 'pointerMove', x: 0, y: 0, origin: { type: 'element', element: { sharedId: match.node.sharedId } } },
+ ...(action === 'click' ? [{ type: 'pointerDown', button: 0 }, { type: 'pointerUp', button: 0 }] : []),
+ ] };
+ assertLive();
+ try { await this.send('input.performActions', { context: match.context, actions: [source] }); }
+ finally { await this.send('input.releaseActions', { context: match.context }).catch(() => {}); }
+ assertLive();
+ } else if (typing) {
+ if (typeof payload.text !== 'string' || payload.text.length > 10000) throw new Error('Trusted text input is limited to 10000 characters per action');
+ if (/[\r\n]/.test(payload.text)) {
+ const multiline = await this.call(match, '(el) => el.tagName === "TEXTAREA" || el.isContentEditable');
+ if (multiline.result?.value !== true) throw new Error('Newlines require a multiline editor; no text sent');
+ }
+ if (clear) {
+ const platform = await this.call(match, '() => navigator.platform');
+ await press('a', /Mac/.test(platform.result?.value || '') ? '\uE03D' : '\uE009');
+ await press('\uE003');
+ }
+ // One character per dispatch bounds work after Stop and catches focus changes between keys.
+ for (const char of payload.text) {
+ if (char === '\n' || char === '\r') {
+ await assertFocus();
+ // Enter is a submit shortcut on many editors, even with Shift. Insert a literal
+ // newline through the editing command instead; never synthesize a submit key.
+ const inserted = await this.call(match, `(el, text) => {
+ if (!el.isConnected || el.getRootNode().activeElement !== el) return false;
+ return document.execCommand('insertText', false, text);
+ }`, [{ type: 'string', value: char }]);
+ if (inserted.result?.value !== true) throw new Error('Editor rejected literal newline insertion');
+ assertLive();
+ } else await press(char);
+ }
+ } else if (action === 'key') {
+ const keys = { Enter: '\uE007', Tab: '\uE004', Escape: '\uE00C', Backspace: '\uE003', Delete: '\uE017', ArrowLeft: '\uE012', ArrowRight: '\uE014', ArrowUp: '\uE013', ArrowDown: '\uE015', Home: '\uE011', End: '\uE010' };
+ const value = keys[payload.key] || (String(payload.key).length === 1 ? payload.key : null);
+ if (!value) throw new Error('Unsupported trusted key');
+ const repeat = Math.max(1, Math.min(3, Math.floor(Number(payload.repeat) || 1)));
+ // Repeated Tab intentionally follows the focus moved by the previous Tab.
+ for (let i = 0; i < repeat; i++) await press(value, null, payload.key === 'Tab' && i > 0);
+ }
+ if (typing) {
+ await new Promise(resolve => setTimeout(resolve, 100));
+ assertLive();
+ const expected = clear ? payload.text : String(before.result?.value || '') + payload.text;
+ const verified = await this.call(match, '(el, expected) => el.isConnected && (el.isContentEditable ? el.innerText : el.value) === expected', [{ type: 'string', value: expected }]);
+ if (verified.result?.value !== true) return { success: false, dispatched: true, verified: false, submitted: false, outcomeUnknown: true, retryable: false, error: 'Trusted typing did not match the requested value; inspect before another write.' };
+ if (action === 'field' && payload.submit === true) {
+ assertLive();
+ await press('\uE007');
+ }
+ return { success: true, dispatched: true, verified: true, submitted: action === 'field' && payload.submit === true, method: 'firefox-bidi' };
+ }
+ return { success: true, dispatched: true, verified: false, method: 'firefox-bidi' };
+ }
+ async close() {
+ for (const id of this.runs.keys()) await this.closeRun(id);
+ this.socket?.close();
+ for (const dir of this.files) await rm(dir, { recursive: true, force: true });
+ this.files = [];
+ }
+}
diff --git a/package.json b/package.json
index 7139af282..c29328f53 100644
--- a/package.json
+++ b/package.json
@@ -5,7 +5,7 @@
"private": true,
"type": "module",
"scripts": {
- "test": "node --test test/browser-dialogs.mjs && npm run test:runtime-lifecycle && npm run test:provider-limits && npm run test:toolbar-guard && npm run test:pdf-read && npm run test:pdf-selection && npm run test:social-contract && node test/run.js && node test/selection-scope-restoration.mjs && node scripts/benchmark-offline-relevance.mjs && npm run test:security",
+ "test": "node --test test/browser-dialogs.mjs && npm run test:firefox-bidi && npm run test:runtime-lifecycle && npm run test:provider-limits && npm run test:toolbar-guard && npm run test:pdf-read && npm run test:pdf-selection && npm run test:social-contract && node test/run.js && node test/selection-scope-restoration.mjs && node scripts/benchmark-offline-relevance.mjs && npm run test:security",
"test:provider-limits": "node test/provider-model-limits.mjs",
"test:security": "node test/security/injection-corpus.mjs",
"test:toolbar-guard": "node test/rich-text-toolbar-guard.mjs",
@@ -39,7 +39,9 @@
"test:social-contract": "node --test test/social-publish-contract.mjs",
"test:social-contract:dom": "node test/social-publish-contract-dom.mjs",
"test:runtime-lifecycle": "node --test test/agent-lifecycle.mjs test/runtime-lifecycle.mjs test/network-lifecycle.mjs test/text-write-cancellation.mjs && node test/ui-content-lifecycle.mjs",
- "test:rich-text-dom": "node test/ui-content-lifecycle.mjs --dom"
+ "test:rich-text-dom": "node test/ui-content-lifecycle.mjs --dom",
+ "test:firefox-bidi": "node --test test/firefox-bidi.mjs",
+ "test:firefox-bidi:e2e": "node test/firefox-bidi-e2e.mjs"
},
"devDependencies": {
"playwright": "^1.48.0",
diff --git a/src/chrome/src/ui/locales/ar.js b/src/chrome/src/ui/locales/ar.js
index c81b90b30..d8450b108 100644
--- a/src/chrome/src/ui/locales/ar.js
+++ b/src/chrome/src/ui/locales/ar.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Arabic (ar).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'تكبير واجهة الإضافة',
'sp.ui_scale.decrease': 'تصغير واجهة الإضافة',
'sp.ui_scale.increase': 'تكبير واجهة الإضافة',
diff --git a/src/chrome/src/ui/locales/bidi-copy.mjs b/src/chrome/src/ui/locales/bidi-copy.mjs
new file mode 100644
index 000000000..0c7a591da
--- /dev/null
+++ b/src/chrome/src/ui/locales/bidi-copy.mjs
@@ -0,0 +1,16 @@
+// English fallback for experimental Firefox-only setup; shared to keep locale keys aligned.
+export default {
+ "st.bidi.title": "Firefox trusted automation (experimental)",
+ "st.bidi.desc": "Use a local companion for browser dialogs, trusted clicks and typing, and file uploads. Requires Node.js 22+ and Firefox started with remote automation enabled.",
+ "st.bidi.setup_html": "Install the companion using <code>node firefox-companion/install.mjs</code> from the WebBrain checkout. Restart Firefox with <code>--remote-debugging-port 9222</code>. Keep the connection on this computer.",
+ "st.bidi.enable": "Use trusted Firefox automation",
+ "st.bidi.port": "Local Firefox port",
+ "st.bidi.test": "Test connection",
+ "st.bidi.invalid_port": "Choose a port from 1024 to 65535.",
+ "st.bidi.enabled": "Enabled for new Act/Dev tasks.",
+ "st.bidi.disabled": "Using standard Firefox automation.",
+ "st.bidi.connecting": "Connecting…",
+ "st.bidi.connected": "Connected to Firefox.",
+ "st.bidi.failed": "Connection failed.",
+
+};
diff --git a/src/chrome/src/ui/locales/bn.js b/src/chrome/src/ui/locales/bn.js
index 4d2136321..214072580 100644
--- a/src/chrome/src/ui/locales/bn.js
+++ b/src/chrome/src/ui/locales/bn.js
@@ -1,8 +1,10 @@
+import bidiCopy from './bidi-copy.mjs';
// Bengali — translated from the canonical English locale.
import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'এক্সটেনশনের UI জুম',
'sp.ui_scale.decrease': 'এক্সটেনশনের UI ছোট করুন',
'sp.ui_scale.increase': 'এক্সটেনশনের UI বড় করুন',
diff --git a/src/chrome/src/ui/locales/de.js b/src/chrome/src/ui/locales/de.js
index ee150c2d0..d59a33874 100644
--- a/src/chrome/src/ui/locales/de.js
+++ b/src/chrome/src/ui/locales/de.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// German (de).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Erweiterungsoberfläche zoomen',
'sp.ui_scale.decrease': 'Erweiterungsoberfläche verkleinern',
'sp.ui_scale.increase': 'Erweiterungsoberfläche vergrößern',
diff --git a/src/chrome/src/ui/locales/en.js b/src/chrome/src/ui/locales/en.js
index 23f0f691f..29d3f9691 100644
--- a/src/chrome/src/ui/locales/en.js
+++ b/src/chrome/src/ui/locales/en.js
@@ -1,8 +1,10 @@
+import bidiCopy from './bidi-copy.mjs';
// English — canonical locale. Other locales inherit key names from this file.
import apocalypseModeCopy from './apocalypse-copy.mjs';
import emergencyCopy from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Extension UI zoom',
'sp.ui_scale.decrease': 'Zoom extension UI out',
'sp.ui_scale.increase': 'Zoom extension UI in',
diff --git a/src/chrome/src/ui/locales/es.js b/src/chrome/src/ui/locales/es.js
index fc995b01e..9970f8410 100644
--- a/src/chrome/src/ui/locales/es.js
+++ b/src/chrome/src/ui/locales/es.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Spanish (es).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Zoom de la interfaz de la extensión',
'sp.ui_scale.decrease': 'Reducir el zoom de la interfaz',
'sp.ui_scale.increase': 'Aumentar el zoom de la interfaz',
diff --git a/src/chrome/src/ui/locales/fa.js b/src/chrome/src/ui/locales/fa.js
index 6f5d51f04..317fd8cbb 100644
--- a/src/chrome/src/ui/locales/fa.js
+++ b/src/chrome/src/ui/locales/fa.js
@@ -1,8 +1,10 @@
+import bidiCopy from './bidi-copy.mjs';
// Persian — translated from the canonical English locale.
import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'بزرگ‌نمایی رابط کاربری افزونه',
'sp.ui_scale.decrease': 'کاهش بزرگ‌نمایی رابط افزونه',
'sp.ui_scale.increase': 'افزایش بزرگ‌نمایی رابط افزونه',
diff --git a/src/chrome/src/ui/locales/fr.js b/src/chrome/src/ui/locales/fr.js
index c9607b1dc..257f0d229 100644
--- a/src/chrome/src/ui/locales/fr.js
+++ b/src/chrome/src/ui/locales/fr.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// French (fr).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Zoom de l’interface de l’extension',
'sp.ui_scale.decrease': 'Réduire le zoom de l’interface',
'sp.ui_scale.increase': 'Augmenter le zoom de l’interface',
diff --git a/src/chrome/src/ui/locales/he.js b/src/chrome/src/ui/locales/he.js
index 71d0312fe..5ffe3963b 100644
--- a/src/chrome/src/ui/locales/he.js
+++ b/src/chrome/src/ui/locales/he.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Hebrew (he).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'הגדלת ממשק התוסף',
'sp.ui_scale.decrease': 'הקטנת תצוגת ממשק התוסף',
'sp.ui_scale.increase': 'הגדלת תצוגת ממשק התוסף',
diff --git a/src/chrome/src/ui/locales/hi.js b/src/chrome/src/ui/locales/hi.js
index 04f7b92eb..5e8d35271 100644
--- a/src/chrome/src/ui/locales/hi.js
+++ b/src/chrome/src/ui/locales/hi.js
@@ -1,8 +1,10 @@
+import bidiCopy from './bidi-copy.mjs';
// Hindi — translated from the canonical English locale.
import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'एक्सटेंशन UI ज़ूम',
'sp.ui_scale.decrease': 'एक्सटेंशन UI को छोटा करें',
'sp.ui_scale.increase': 'एक्सटेंशन UI को बड़ा करें',
diff --git a/src/chrome/src/ui/locales/id.js b/src/chrome/src/ui/locales/id.js
index 9cc3cec09..3d050a529 100644
--- a/src/chrome/src/ui/locales/id.js
+++ b/src/chrome/src/ui/locales/id.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Indonesian (id).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Zoom UI ekstensi',
'sp.ui_scale.decrease': 'Perkecil UI ekstensi',
'sp.ui_scale.increase': 'Perbesar UI ekstensi',
diff --git a/src/chrome/src/ui/locales/ja.js b/src/chrome/src/ui/locales/ja.js
index 514039701..7558d8930 100644
--- a/src/chrome/src/ui/locales/ja.js
+++ b/src/chrome/src/ui/locales/ja.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Japanese (ja).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': '拡張機能 UI のズーム',
'sp.ui_scale.decrease': '拡張機能 UI を縮小',
'sp.ui_scale.increase': '拡張機能 UI を拡大',
diff --git a/src/chrome/src/ui/locales/ko.js b/src/chrome/src/ui/locales/ko.js
index 1bf2d6956..7d0569794 100644
--- a/src/chrome/src/ui/locales/ko.js
+++ b/src/chrome/src/ui/locales/ko.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Korean (ko).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': '확장 프로그램 UI 확대/축소',
'sp.ui_scale.decrease': '확장 프로그램 UI 축소',
'sp.ui_scale.increase': '확장 프로그램 UI 확대',
diff --git a/src/chrome/src/ui/locales/ms.js b/src/chrome/src/ui/locales/ms.js
index a7969f12a..95d5bade1 100644
--- a/src/chrome/src/ui/locales/ms.js
+++ b/src/chrome/src/ui/locales/ms.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Malay (ms).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Zum UI sambungan',
'sp.ui_scale.decrease': 'Kecilkan UI sambungan',
'sp.ui_scale.increase': 'Besarkan UI sambungan',
diff --git a/src/chrome/src/ui/locales/nl.js b/src/chrome/src/ui/locales/nl.js
index bdce860eb..493fc2d2c 100644
--- a/src/chrome/src/ui/locales/nl.js
+++ b/src/chrome/src/ui/locales/nl.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Dutch (nl).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Zoom van extensie-interface',
'sp.ui_scale.decrease': 'Extensie-interface uitzoomen',
'sp.ui_scale.increase': 'Extensie-interface inzoomen',
diff --git a/src/chrome/src/ui/locales/pl.js b/src/chrome/src/ui/locales/pl.js
index 130ed06ae..15e34b23e 100644
--- a/src/chrome/src/ui/locales/pl.js
+++ b/src/chrome/src/ui/locales/pl.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Polski — translated from en.js. Keys mirror the English canonical file.
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Powiększenie interfejsu rozszerzenia',
'sp.ui_scale.decrease': 'Pomniejsz interfejs rozszerzenia',
'sp.ui_scale.increase': 'Powiększ interfejs rozszerzenia',
diff --git a/src/chrome/src/ui/locales/pt.js b/src/chrome/src/ui/locales/pt.js
index 9e61fc25f..c40378490 100644
--- a/src/chrome/src/ui/locales/pt.js
+++ b/src/chrome/src/ui/locales/pt.js
@@ -1,8 +1,10 @@
+import bidiCopy from './bidi-copy.mjs';
// Portuguese — translated from the canonical English locale.
import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Zoom da interface da extensão',
'sp.ui_scale.decrease': 'Reduzir o zoom da interface',
'sp.ui_scale.increase': 'Aumentar o zoom da interface',
diff --git a/src/chrome/src/ui/locales/ru.js b/src/chrome/src/ui/locales/ru.js
index 9643a6de1..ef3465f03 100644
--- a/src/chrome/src/ui/locales/ru.js
+++ b/src/chrome/src/ui/locales/ru.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Russian (ru).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Масштаб интерфейса расширения',
'sp.ui_scale.decrease': 'Уменьшить интерфейс расширения',
'sp.ui_scale.increase': 'Увеличить интерфейс расширения',
diff --git a/src/chrome/src/ui/locales/th.js b/src/chrome/src/ui/locales/th.js
index c3cda8251..a75111952 100644
--- a/src/chrome/src/ui/locales/th.js
+++ b/src/chrome/src/ui/locales/th.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Thai (th).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'ซูม UI ของส่วนขยาย',
'sp.ui_scale.decrease': 'ย่อ UI ของส่วนขยาย',
'sp.ui_scale.increase': 'ขยาย UI ของส่วนขยาย',
diff --git a/src/chrome/src/ui/locales/tl.js b/src/chrome/src/ui/locales/tl.js
index c81505058..4e437b618 100644
--- a/src/chrome/src/ui/locales/tl.js
+++ b/src/chrome/src/ui/locales/tl.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Filipino / Tagalog (tl).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Pag-zoom ng UI ng extension',
'sp.ui_scale.decrease': 'Bawasan ang zoom ng UI ng extension',
'sp.ui_scale.increase': 'Dagdagan ang zoom ng UI ng extension',
diff --git a/src/chrome/src/ui/locales/tr.js b/src/chrome/src/ui/locales/tr.js
index 61df15a3b..522eb8710 100644
--- a/src/chrome/src/ui/locales/tr.js
+++ b/src/chrome/src/ui/locales/tr.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Turkish (tr).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Uzantı arayüzü yakınlaştırması',
'sp.ui_scale.decrease': 'Uzantı arayüzünü uzaklaştır',
'sp.ui_scale.increase': 'Uzantı arayüzünü yakınlaştır',
diff --git a/src/chrome/src/ui/locales/uk.js b/src/chrome/src/ui/locales/uk.js
index ba2fd1318..45a3b4c22 100644
--- a/src/chrome/src/ui/locales/uk.js
+++ b/src/chrome/src/ui/locales/uk.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Ukrainian (uk).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Масштаб інтерфейсу розширення',
'sp.ui_scale.decrease': 'Зменшити інтерфейс розширення',
'sp.ui_scale.increase': 'Збільшити інтерфейс розширення',
diff --git a/src/chrome/src/ui/locales/vi.js b/src/chrome/src/ui/locales/vi.js
index 17592ed6c..6f57ee118 100644
--- a/src/chrome/src/ui/locales/vi.js
+++ b/src/chrome/src/ui/locales/vi.js
@@ -1,8 +1,10 @@
+import bidiCopy from './bidi-copy.mjs';
// Vietnamese — translated from the canonical English locale.
import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Thu phóng giao diện tiện ích',
'sp.ui_scale.decrease': 'Thu nhỏ giao diện tiện ích',
'sp.ui_scale.increase': 'Phóng to giao diện tiện ích',
diff --git a/src/chrome/src/ui/locales/zh.js b/src/chrome/src/ui/locales/zh.js
index 3c7c32943..3734ed5a7 100644
--- a/src/chrome/src/ui/locales/zh.js
+++ b/src/chrome/src/ui/locales/zh.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Simplified Chinese (zh).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': '插件界面缩放',
'sp.ui_scale.decrease': '缩小插件界面',
'sp.ui_scale.increase': '放大插件界面',
diff --git a/src/firefox/ARCHITECTURE.md b/src/firefox/ARCHITECTURE.md
index 207921bde..7892b7bff 100644
--- a/src/firefox/ARCHITECTURE.md
+++ b/src/firefox/ARCHITECTURE.md
@@ -6,7 +6,7 @@
Firefox uses Manifest V2 (background page, not service worker) and has **no access to the Chrome DevTools Protocol (CDP)**. Starting with v3.6.x, the Firefox build has been brought to functional parity with Chrome for the accessibility-tree (AX) subsystem — the same tree builder, the same four AX tools (`get_accessibility_tree`, `click_ax`, `type_ax`, `set_field`), and the same ref_id registry. What Firefox still lacks:
-- **No trusted events** — clicks and key presses are synthetic (`el.click()`, `new KeyboardEvent()`), and some sites reject `event.isTrusted === false`. All AX-tool click/type paths use synthetic dispatch in Firefox; the CDP-backed trusted-event path in Chrome has no Firefox equivalent.
+- **Optional trusted events** — standard mode uses synthetic events. The experimental [BiDi companion](../../firefox-companion/README.md) adds trusted click, hover, text, supported keys, file-input attachment, and native JavaScript-dialog handling. It requires a local helper and Firefox remote automation; other tools retain their existing implementations. This is a Firefox-specific transport, not a Chrome CDP adapter.
- **No pixel-perfect / full-page screenshots** — uses `browser.tabs.captureTab()` instead of CDP `Page.captureScreenshot`; it can capture the run tab while that tab is inactive. Firefox has exposed `tabs.captureTab()` since Firefox 59, before WebBrain's current minimum, and the manifest declares the required `<all_urls>` permission.
- **No shadow DOM piercing** — content script can read open shadow roots via `element.shadowRoot`, but cannot pierce closed roots.
- **No offscreen document** — no HTTP fetch proxy for localhost LLM servers with Private Network Access / CORS issues. User must ensure their local LLM server sends permissive CORS headers.
diff --git a/src/firefox/manifest.json b/src/firefox/manifest.json
index 034514268..89fa8c069 100644
--- a/src/firefox/manifest.json
+++ b/src/firefox/manifest.json
@@ -5,6 +5,7 @@
"description": "Open-source AI browser agent — chat with pages, automate tasks, multi-provider LLM support.",
"permissions": [
"activeTab",
+ "nativeMessaging",
"menus",
"webNavigation",
"webRequest",
diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.js
index fa89aa611..83222666d 100644
--- a/src/firefox/src/agent/agent.js
+++ b/src/firefox/src/agent/agent.js
@@ -1,3 +1,4 @@
+import { firefoxBidi } from '../bidi/client.js';
import { SOCIAL_PLATFORMS, normalizePublicationContract, publicationProgress, exactPublicationText, publicationMediaMatches, publicationContractMessages, publicationAuditMessages, publicationAuditAccepted } from './social-publish-contract.js';
import { AGENT_TOOLS, AGENT_TOOL_NAMES, RESERVED_AGENT_TOOL_NAMES, getToolsForMode, SYSTEM_PROMPT_ASK, SYSTEM_PROMPT_ACT, SYSTEM_PROMPT_ACT_COMPACT, SYSTEM_PROMPT_ACT_MID, SYSTEM_PROMPT_DEV_APPENDIX } from './tools.js';
import { validateToolArguments } from './tool-arguments.js';
@@ -5688,6 +5689,7 @@ export class Agent extends LoopDetector {
}
_releaseRunEntry(tabId) {
+ firefoxBidi.stopRun(tabId);
this._runAbortStates.get(tabId)?.dispose();
this._runAbortStates.delete(tabId);
this.abortFlags.delete(tabId);
@@ -12117,7 +12119,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
const send = () => {
throwIfAborted();
dispatchState.started = true;
- return browser.tabs.sendMessage(tabId, {
+ return firefoxBidi.sendContent(tabId, {
target: 'content',
action: 'click_ax',
params: contentArgs,
@@ -29029,6 +29031,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
this._prepareClarificationAuthorizationForRun(tabId);
this.permissions.beginTurn(tabId);
this.conversationModes.set(tabId, 'act');
+ await firefoxBidi.startRun(tabId, this._runAbortSignal(tabId));
completionRunToken = this._beginCompletionInvariant(tabId);
startUrl = await this._currentUrl(tabId);
const conversationId = await this.ensureConversationId(tabId, 'act');
@@ -30170,13 +30173,15 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
};
try {
- await browser.tabs.update(tabId, { url: rawUrl });
+ if (firefoxBidi.runs.has(tabId)) await firefoxBidi.perform(tabId, 'navigate', { url: rawUrl });
+ else await browser.tabs.update(tabId, { url: rawUrl });
} catch (e) {
removeNavigationListener();
return {
success: false,
dispatched: false,
noDispatch: true,
+ ...(firefoxBidi.runs.has(tabId) ? { dispatched: true, noDispatch: false, outcomeUnknown: true, retryable: false } : {}),
error: `navigate: browser rejected the navigation: ${e?.message || String(e)}`,
};
}
@@ -31109,6 +31114,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
return await downloadFiles(args);
}
if (name === 'upload_file') {
+ const bidiUploadOwner = firefoxBidi.runs.get(tabId);
const UPLOAD_MAX_BYTES = 25 * 1024 * 1024;
try {
args = args || {};
@@ -31338,6 +31344,14 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
return { success: false, error: 'No file data available to attach' };
}
+ if (bidiUploadOwner) {
+ if (firefoxBidi.runs.get(tabId) !== bidiUploadOwner || bidiUploadOwner.signal?.aborted) return { success: false, dispatched: false, noDispatch: true, error: 'Upload task stopped before attachment.' };
+ return await firefoxBidi.sendContent(tabId, {
+ target: 'content', action: 'bidi_prepare_upload',
+ params: { selector: args.selector, base64, filename, mimeType },
+ });
+ }
+
const targetProbeCode = `
(function() {
// WebBrain file attachment target probe.
@@ -33113,7 +33127,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
.map(signal => Number(CONTENT_ACTION_SIGNAL_DEADLINES.get(signal)?.deadlineAt))
.filter(value => Number.isFinite(value) && value > 0);
const actionDeadlineAt = deadlines.length ? Math.min(...deadlines) : 0;
- return browser.tabs.sendMessage(tabId, {
+ return firefoxBidi.sendContent(tabId, {
target: 'content',
action,
params: contentArgs,
@@ -33409,6 +33423,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
this.cloudRunContexts.set(tabId, { outputSchema: runOptions.outputSchema ?? null, schemaRepairUsed: false });
}
try {
+ if ((mode === 'act' || mode === 'dev') && !this._isStandaloneChatRun(runOptions)) await firefoxBidi.startRun(tabId, this._runAbortSignal(tabId));
const result = await this._processMessageInner(tabId, userMessage, onUpdate, mode, attachments, runOptions);
void this._maybeEmitAskModeHandoff(tabId, mode, userMessage, result, onUpdate, runOptions);
return result;
@@ -34837,6 +34852,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
this.cloudRunContexts.set(tabId, { outputSchema: runOptions.outputSchema ?? null, schemaRepairUsed: false });
}
try {
+ if ((mode === 'act' || mode === 'dev') && !this._isStandaloneChatRun(runOptions)) await firefoxBidi.startRun(tabId, this._runAbortSignal(tabId));
const result = await this._processMessageStreamInner(tabId, userMessage, onUpdate, mode, runOptions);
void this._maybeEmitAskModeHandoff(tabId, mode, userMessage, result, onUpdate, runOptions);
return result;
diff --git a/src/firefox/src/agent/tools.js b/src/firefox/src/agent/tools.js
index e4db67484..09484f7de 100644
--- a/src/firefox/src/agent/tools.js
+++ b/src/firefox/src/agent/tools.js
@@ -147,7 +147,7 @@ export const AGENT_TOOLS = [
type: 'function',
function: {
name: 'hover',
- description: 'Hover the mouse over an element by its ref_id. Use this for menus, tooltips, and "More actions" overlays that only reveal on hover. On Firefox this is synthetic (mouseenter/mouseover/pointerover events) — works on most sites but sites that gate hover-reveal on event.isTrusted will not respond. Re-read the accessibility tree after to find the now-visible items.',
+ description: 'Hover the mouse over an element by its ref_id. Use this for menus, tooltips, and "More actions" overlays that only reveal on hover. On Firefox standard mode this is synthetic (the optional BiDi companion supplies trusted pointer input) — works on most sites but sites that gate hover-reveal on event.isTrusted will not respond. Re-read the accessibility tree after to find the now-visible items.',
parameters: {
type: 'object',
properties: {
diff --git a/src/firefox/src/background.js b/src/firefox/src/background.js
index 831dc313d..4f166102b 100644
--- a/src/firefox/src/background.js
+++ b/src/firefox/src/background.js
@@ -1,3 +1,4 @@
+import { firefoxBidi } from './bidi/client.js';
import { ProviderManager } from './providers/manager.js';
import { Agent } from './agent/agent.js';
import {
@@ -3640,3 +3641,14 @@ browser.commands.onCommand.addListener(async (command, tab) => {
console.error('[WebBrain] failed to dispatch command:', command, err);
}
});
+
+// Connection controls belong only to the packaged Settings page, never a tab.
+browser.runtime.onMessage.addListener((message, sender) => {
+ if (message?.type !== 'WB_BIDI_CONNECT' && message?.type !== 'WB_BIDI_DISCONNECT') return;
+ if (sender.id !== browser.runtime.id || sender.url !== browser.runtime.getURL('src/ui/settings.html')) return;
+ if (message.type === 'WB_BIDI_DISCONNECT') { firefoxBidi.disconnect(); return Promise.resolve({ success: true }); }
+ return firefoxBidi.connect().then(() => ({ success: true }), error => ({ success: false, error: error.message }));
+});
+browser.storage.onChanged.addListener((changes, area) => {
+ if (area === 'local' && ['firefoxBidiEnabled', 'firefoxBidiPort'].some(key => changes[key] && changes[key].oldValue !== changes[key].newValue)) firefoxBidi.disconnect();
+});
diff --git a/src/firefox/src/bidi/client.js b/src/firefox/src/bidi/client.js
new file mode 100644
index 000000000..cf7655084
--- /dev/null
+++ b/src/firefox/src/bidi/client.js
@@ -0,0 +1,113 @@
+/** Optional native-messaging transport. No raw BiDi commands are exposed to pages. */
+export class FirefoxBidiClient {
+ constructor(api) { this.apiOverride = api; this.pending = new Map(); this.runs = new Map(); this.sequence = 0; }
+ get api() { return this.apiOverride || globalThis.browser; }
+ request(command, args = {}) {
+ if (!this.port) {
+ this.port = this.api.runtime.connectNative('one.webbrain.bidi');
+ this.port.onMessage.addListener(message => {
+ const entry = this.pending.get(message.id); if (!entry) return;
+ this.pending.delete(message.id);
+ message.error ? entry.reject(new Error(message.error)) : entry.resolve(message.result);
+ });
+ const port = this.port;
+ this.port.onDisconnect.addListener(() => {
+ if (this.port !== port) return;
+ this.port = null; this.connection = null;
+ for (const entry of this.pending.values()) entry.reject(new Error('Firefox companion disconnected; do not retry an uncertain action'));
+ this.pending.clear();
+ for (const owner of this.runs.values()) owner.disconnected = true;
+ });
+ }
+ const id = ++this.sequence;
+ return new Promise((resolve, reject) => {
+ const timer = setTimeout(() => { this.pending.delete(id); reject(new Error('Firefox companion timed out; action outcome may be unknown')); }, 15000);
+ this.pending.set(id, { resolve: value => { clearTimeout(timer); resolve(value); }, reject: error => { clearTimeout(timer); reject(error); } });
+ try { this.port.postMessage({ id, command, ...args }); }
+ catch (error) { this.pending.get(id).reject(error); this.pending.delete(id); }
+ });
+ }
+ async connect() {
+ const settings = await this.api.storage.local.get('firefoxBidiPort');
+ if (!this.connection) this.connection = this.request('connect', { port: Number(settings.firefoxBidiPort) || 9222 }).catch(error => { this.connection = null; throw error; });
+ return this.connection;
+ }
+ async startRun(tabId, signal) {
+ if (!this.api?.storage?.local) return;
+ const settings = await this.api.storage.local.get('firefoxBidiEnabled');
+ if (!settings?.firefoxBidiEnabled || signal?.aborted) return;
+ this.stopRun(tabId);
+ const runId = crypto.randomUUID();
+ const owner = { runId, signal, abort: () => this.stopRun(tabId) };
+ this.runs.set(tabId, owner);
+ signal?.addEventListener('abort', owner.abort, { once: true });
+ try {
+ await this.connect();
+ if (this.runs.get(tabId) !== owner) throw new Error('Run stopped');
+ await this.bindRun(tabId, owner);
+ } catch (error) { if (this.runs.get(tabId) === owner) this.stopRun(tabId); throw error; }
+ }
+ async bindRun(tabId, owner) {
+ if (owner.bound) return;
+ if (this.runs.get(tabId) !== owner || owner.disconnected || owner.signal?.aborted) throw new Error('Run stopped');
+ const tab = await this.api.tabs.get(tabId);
+ // Internal/new tabs cannot accept content scripts. Keep ownership but defer
+ // document binding so the normal authorized navigation can leave that page.
+ if (!/^https?:\/\//.test(tab.url || '')) return;
+ const token = crypto.randomUUID();
+ const [url] = await this.api.tabs.executeScript(tabId, { frameId: 0, code: `(() => { const el = document.documentElement; const token = ${JSON.stringify(token)}; el.setAttribute('data-webbrain-bidi', token); setTimeout(() => { if (el.getAttribute('data-webbrain-bidi') === token) el.removeAttribute('data-webbrain-bidi'); }, 10000); return location.href; })()` });
+ if (this.runs.get(tabId) !== owner || owner.disconnected || owner.signal?.aborted) throw new Error('Run stopped');
+ await this.request('openRun', { runId: owner.runId, token, url });
+ if (this.runs.get(tabId) !== owner || owner.disconnected || owner.signal?.aborted) {
+ if (this.port) void this.request('closeRun', { runId: owner.runId }).catch(() => {});
+ throw new Error('Run stopped');
+ }
+ owner.bound = true;
+ }
+ stopRun(tabId) {
+ const owner = this.runs.get(tabId); if (!owner) return;
+ this.runs.delete(tabId); owner.signal?.removeEventListener('abort', owner.abort);
+ if (this.port) void this.request('closeRun', { runId: owner.runId }).catch(() => {});
+ }
+ disconnect() {
+ // Retain failed owners until their task ends; never switch a live task to synthetic input.
+ for (const owner of this.runs.values()) owner.disconnected = true;
+ const port = this.port; this.port = null; this.connection = null;
+ for (const entry of this.pending.values()) entry.reject(new Error('Firefox companion disconnected'));
+ this.pending.clear(); port?.disconnect();
+ }
+ async perform(tabId, action, payload) {
+ const owner = this.runs.get(tabId);
+ if (!owner || owner.signal?.aborted || owner.disconnected) throw new Error('No connected Firefox BiDi run; restart the task after reconnecting');
+ await this.bindRun(tabId, owner);
+ if (this.runs.get(tabId) !== owner || owner.disconnected || owner.signal?.aborted) throw new Error('Run stopped');
+ if (!owner.bound) {
+ if (action !== 'navigate' || !/^https?:\/\//.test(payload.url || '')) throw new Error('Navigate to a web page before trusted input');
+ await this.api.tabs.update(tabId, { url: payload.url });
+ return { success: true, dispatched: true, bindingDeferred: true };
+ }
+ return this.request('perform', { runId: owner.runId, action, payload });
+ }
+ async sendContent(tabId, message, options) {
+ const actions = { click: 'click', click_ax: 'click', type: 'type', type_ax: 'type', set_field: 'field', press_keys: 'key', hover: 'hover', bidi_prepare_upload: 'upload' };
+ const action = actions[message.action];
+ const owner = this.runs.get(tabId);
+ if (owner?.disconnected || owner?.signal?.aborted) return { success: false, dispatched: false, noDispatch: true, retryable: false, error: 'Firefox connection stopped; restart the task.' };
+ if (!owner || !action) return this.api.tabs.sendMessage(tabId, message, options);
+ await this.bindRun(tabId, owner);
+ if (!owner.bound) return { success: false, dispatched: false, noDispatch: true, error: 'Navigate to a web page before trusted input.' };
+ const token = crypto.randomUUID();
+ const prepared = await this.api.tabs.sendMessage(tabId, { ...message, params: { ...message.params, _bidiPrepare: token } }, options);
+ if (!prepared?.bidiPrepared) return prepared;
+ if (this.runs.get(tabId) !== owner || owner.disconnected || owner.signal?.aborted) return { success: false, dispatched: false, noDispatch: true, retryable: false, error: 'Run stopped before trusted input.' };
+ if (message.actionDeadlineAt && Date.now() >= message.actionDeadlineAt) return { success: false, dispatched: false, noDispatch: true, deadlineExpired: true };
+ const metadata = { fieldMeta: prepared.fieldMeta, ...(message.params?.ref_id ? { ref_id: message.params.ref_id } : {}) };
+ try {
+ return { ...metadata, ...await this.perform(tabId, action, { ...message.params, token, url: prepared.url, deadlineAt: message.actionDeadlineAt || 0 }), ...(prepared.rect ? { rect: prepared.rect } : {}), ...(prepared._filePickerGuardId ? { _filePickerGuardId: prepared._filePickerGuardId } : {}) };
+ } catch (error) {
+ // A transport failure can occur after trusted input was delivered.
+ return { ...metadata, success: false, dispatched: true, outcomeUnknown: true, retryable: false, error: error.message };
+ }
+ }
+}
+export const firefoxBidi = new FirefoxBidiClient();
diff --git a/src/firefox/src/content/content.js b/src/firefox/src/content/content.js
index ac0ef1437..2ae2e96bd 100644
--- a/src/firefox/src/content/content.js
+++ b/src/firefox/src/content/content.js
@@ -2097,6 +2097,7 @@
if (actionDeadlineExpired()) return deadlineFailure();
const clickedRect = rememberInteractionPoint(el, 'click');
if (actionDeadlineExpired()) return deadlineFailure();
+ if (params._bidiPrepare) return { ...prepareBidiTarget(el, params._bidiPrepare), _filePickerGuardId: clickWithoutNativeFilePicker(() => {}).guardId };
dispatched = true;
const filePickerGuard = clickWithoutNativeFilePicker(() => el.click());
if (filePickerGuard.blocked) {
@@ -2392,6 +2393,8 @@
const beforeValue = String(el.isContentEditable ? (el.textContent || '') : (el.value || ''));
const routeHrefBeforeType = location.href;
+ if (params._bidiPrepare && !(el instanceof HTMLSelectElement)) return prepareBidiTarget(el, params._bidiPrepare);
+
// Rich editors must retain their native node structure and editing events.
if (el.isContentEditable) {
return _insertContentEditableText(el, typedText, params.clear === true, actionDeadlineExpired);
@@ -2661,6 +2664,8 @@
? focusedTarget
: document;
+ if (params._bidiPrepare) return prepareBidiTarget(focusedTarget || document.body, params._bidiPrepare);
+
const moveTabFocus = () => {
const focusables = Array.from(document.querySelectorAll(
'a[href], button:not([disabled]), input:not([disabled]):not([type="hidden"]), select:not([disabled]), textarea:not([disabled]), [tabindex]:not([tabindex="-1"])'
@@ -4059,6 +4064,17 @@
// conversation before a message can be sent. It deliberately ignores input
// values and ordinary page text: a searched recipient name is not proof that
// the corresponding conversation is active.
+ function prepareBidiTarget(el, token) {
+ if (!/^[a-f0-9-]{36}$/.test(token || '') || !el?.isConnected) {
+ return { success: false, dispatched: false, noDispatch: true, error: 'Invalid trusted-input target' };
+ }
+ el.setAttribute('data-webbrain-bidi', token);
+ setTimeout(() => { if (el.getAttribute('data-webbrain-bidi') === token) el.removeAttribute('data-webbrain-bidi'); }, 10000);
+ const rect = el.getBoundingClientRect();
+ return { bidiPrepared: true, success: false, dispatched: false, noDispatch: true, url: location.href,
+ fieldMeta: _fieldMeta(el), rect: { x: rect.x, y: rect.y, w: rect.width, h: rect.height } };
+ }
+
function _probeMessageRecipientGuard(params = {}) {
try {
const tool = String(params.tool || '');
@@ -5293,6 +5309,7 @@
{ deadlineExpired: true, retryable: true },
);
}
+ if (msg.params?._bidiPrepare) return { ...prepareBidiTarget(el, msg.params._bidiPrepare), _filePickerGuardId: clickWithoutNativeFilePicker(() => {}).guardId };
dispatched = true;
const filePickerGuard = clickWithoutNativeFilePicker(() => el.click());
if (filePickerGuard.blocked) {
@@ -5545,6 +5562,15 @@
return failure(e && e.message || String(e));
}
},
+ 'bidi_prepare_upload': () => {
+ const params = msg.params || {};
+ let matches;
+ try { matches = document.querySelectorAll(params.selector); } catch { return { success: false, dispatched: false, noDispatch: true, error: 'Invalid file selector' }; }
+ if (matches.length !== 1 || matches[0].tagName !== 'INPUT' || matches[0].type !== 'file' || matches[0].disabled) {
+ return { success: false, dispatched: false, noDispatch: true, error: 'Choose one enabled file input', ambiguous: matches.length > 1, matchCount: matches.length };
+ }
+ return prepareBidiTarget(matches[0], params._bidiPrepare);
+ },
'type_ax': async () => {
let dispatched = false;
const failure = (error, extra = {}) => ({
@@ -5585,6 +5611,10 @@
return { x: Math.round(r.x), y: Math.round(r.y), w: Math.round(r.width), h: Math.round(r.height) };
} catch { return null; }
})();
+ if (msg.params?._bidiPrepare && el.tagName !== 'SELECT') {
+ if (!_isTypeableElement(el)) return failure('Target is not editable');
+ return prepareBidiTarget(el, msg.params._bidiPrepare);
+ }
const fieldMeta = _fieldMeta(el);
let previous = '';
let method = '';
@@ -5773,6 +5803,13 @@
} else if (!el.isContentEditable && el.tagName !== 'TEXTAREA' && el.tagName !== 'INPUT') {
return failure(`ref_id ${ref_id} is not a text field (tag=${el.tagName}). set_field works on input/textarea/contenteditable only.`);
}
+ if (msg.params?._bidiPrepare) {
+ if (submit && msg.params.messageRecipientGuardRequired) {
+ const validation = _consumeMessageRecipientDispatchBinding(msg.params, el);
+ if (validation.success !== true) return validation;
+ }
+ return prepareBidiTarget(el, msg.params._bidiPrepare);
+ }
let prevValue = '';
if (actionDeadlineExpired()) return deadlineFailure();
dispatched = true;
@@ -6128,6 +6165,7 @@
const r = el.getBoundingClientRect();
const cx = r.left + r.width / 2;
const cy = r.top + r.height / 2;
+ if (msg.params?._bidiPrepare) return prepareBidiTarget(el, msg.params._bidiPrepare);
const eventInit = { bubbles: true, cancelable: true, view: window, clientX: cx, clientY: cy };
const dispatchHover = (EventType, type) => {
if (actionDeadlineExpired()) return false;
diff --git a/src/firefox/src/ui/locales/ar.js b/src/firefox/src/ui/locales/ar.js
index 5b4579cdb..bbdff12a6 100644
--- a/src/firefox/src/ui/locales/ar.js
+++ b/src/firefox/src/ui/locales/ar.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Arabic (ar).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'تكبير واجهة الإضافة',
'sp.ui_scale.decrease': 'تصغير واجهة الإضافة',
'sp.ui_scale.increase': 'تكبير واجهة الإضافة',
diff --git a/src/firefox/src/ui/locales/bidi-copy.mjs b/src/firefox/src/ui/locales/bidi-copy.mjs
new file mode 100644
index 000000000..0c7a591da
--- /dev/null
+++ b/src/firefox/src/ui/locales/bidi-copy.mjs
@@ -0,0 +1,16 @@
+// English fallback for experimental Firefox-only setup; shared to keep locale keys aligned.
+export default {
+ "st.bidi.title": "Firefox trusted automation (experimental)",
+ "st.bidi.desc": "Use a local companion for browser dialogs, trusted clicks and typing, and file uploads. Requires Node.js 22+ and Firefox started with remote automation enabled.",
+ "st.bidi.setup_html": "Install the companion using <code>node firefox-companion/install.mjs</code> from the WebBrain checkout. Restart Firefox with <code>--remote-debugging-port 9222</code>. Keep the connection on this computer.",
+ "st.bidi.enable": "Use trusted Firefox automation",
+ "st.bidi.port": "Local Firefox port",
+ "st.bidi.test": "Test connection",
+ "st.bidi.invalid_port": "Choose a port from 1024 to 65535.",
+ "st.bidi.enabled": "Enabled for new Act/Dev tasks.",
+ "st.bidi.disabled": "Using standard Firefox automation.",
+ "st.bidi.connecting": "Connecting…",
+ "st.bidi.connected": "Connected to Firefox.",
+ "st.bidi.failed": "Connection failed.",
+
+};
diff --git a/src/firefox/src/ui/locales/bn.js b/src/firefox/src/ui/locales/bn.js
index 634dc397f..4498b336e 100644
--- a/src/firefox/src/ui/locales/bn.js
+++ b/src/firefox/src/ui/locales/bn.js
@@ -1,8 +1,10 @@
+import bidiCopy from './bidi-copy.mjs';
// Bengali — translated from the canonical English locale.
import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'এক্সটেনশনের UI জুম',
'sp.ui_scale.decrease': 'এক্সটেনশনের UI ছোট করুন',
'sp.ui_scale.increase': 'এক্সটেনশনের UI বড় করুন',
diff --git a/src/firefox/src/ui/locales/de.js b/src/firefox/src/ui/locales/de.js
index efd6907b6..6766d7dc2 100644
--- a/src/firefox/src/ui/locales/de.js
+++ b/src/firefox/src/ui/locales/de.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// German (de).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Erweiterungsoberfläche zoomen',
'sp.ui_scale.decrease': 'Erweiterungsoberfläche verkleinern',
'sp.ui_scale.increase': 'Erweiterungsoberfläche vergrößern',
diff --git a/src/firefox/src/ui/locales/en.js b/src/firefox/src/ui/locales/en.js
index 4b6a25ccc..a41a9ee9e 100644
--- a/src/firefox/src/ui/locales/en.js
+++ b/src/firefox/src/ui/locales/en.js
@@ -1,8 +1,10 @@
+import bidiCopy from './bidi-copy.mjs';
// English — canonical locale. Other locales inherit key names from this file.
import apocalypseModeCopy from './apocalypse-copy.mjs';
import emergencyCopy from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Extension UI zoom',
'sp.ui_scale.decrease': 'Zoom extension UI out',
'sp.ui_scale.increase': 'Zoom extension UI in',
diff --git a/src/firefox/src/ui/locales/es.js b/src/firefox/src/ui/locales/es.js
index 851562c3f..605d114f1 100644
--- a/src/firefox/src/ui/locales/es.js
+++ b/src/firefox/src/ui/locales/es.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Spanish (es).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Zoom de la interfaz de la extensión',
'sp.ui_scale.decrease': 'Reducir el zoom de la interfaz',
'sp.ui_scale.increase': 'Aumentar el zoom de la interfaz',
diff --git a/src/firefox/src/ui/locales/fa.js b/src/firefox/src/ui/locales/fa.js
index 4e1aa4f9f..d02f1bb11 100644
--- a/src/firefox/src/ui/locales/fa.js
+++ b/src/firefox/src/ui/locales/fa.js
@@ -1,8 +1,10 @@
+import bidiCopy from './bidi-copy.mjs';
// Persian — translated from the canonical English locale.
import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'بزرگ‌نمایی رابط کاربری افزونه',
'sp.ui_scale.decrease': 'کاهش بزرگ‌نمایی رابط افزونه',
'sp.ui_scale.increase': 'افزایش بزرگ‌نمایی رابط افزونه',
diff --git a/src/firefox/src/ui/locales/fr.js b/src/firefox/src/ui/locales/fr.js
index 88e6ead43..b1dedec49 100644
--- a/src/firefox/src/ui/locales/fr.js
+++ b/src/firefox/src/ui/locales/fr.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// French (fr).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Zoom de l’interface de l’extension',
'sp.ui_scale.decrease': 'Réduire le zoom de l’interface',
'sp.ui_scale.increase': 'Augmenter le zoom de l’interface',
diff --git a/src/firefox/src/ui/locales/he.js b/src/firefox/src/ui/locales/he.js
index 4a81ac99a..515bb0c7e 100644
--- a/src/firefox/src/ui/locales/he.js
+++ b/src/firefox/src/ui/locales/he.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Hebrew (he).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'הגדלת ממשק התוסף',
'sp.ui_scale.decrease': 'הקטנת תצוגת ממשק התוסף',
'sp.ui_scale.increase': 'הגדלת תצוגת ממשק התוסף',
diff --git a/src/firefox/src/ui/locales/hi.js b/src/firefox/src/ui/locales/hi.js
index 368eb6a53..e21ce3bf6 100644
--- a/src/firefox/src/ui/locales/hi.js
+++ b/src/firefox/src/ui/locales/hi.js
@@ -1,8 +1,10 @@
+import bidiCopy from './bidi-copy.mjs';
// Hindi — translated from the canonical English locale.
import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'एक्सटेंशन UI ज़ूम',
'sp.ui_scale.decrease': 'एक्सटेंशन UI को छोटा करें',
'sp.ui_scale.increase': 'एक्सटेंशन UI को बड़ा करें',
diff --git a/src/firefox/src/ui/locales/id.js b/src/firefox/src/ui/locales/id.js
index 8a03d99d7..64fb34c31 100644
--- a/src/firefox/src/ui/locales/id.js
+++ b/src/firefox/src/ui/locales/id.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Indonesian (id).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Zoom UI ekstensi',
'sp.ui_scale.decrease': 'Perkecil UI ekstensi',
'sp.ui_scale.increase': 'Perbesar UI ekstensi',
diff --git a/src/firefox/src/ui/locales/ja.js b/src/firefox/src/ui/locales/ja.js
index d0e470c69..35dfef70d 100644
--- a/src/firefox/src/ui/locales/ja.js
+++ b/src/firefox/src/ui/locales/ja.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Japanese (ja).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': '拡張機能 UI のズーム',
'sp.ui_scale.decrease': '拡張機能 UI を縮小',
'sp.ui_scale.increase': '拡張機能 UI を拡大',
diff --git a/src/firefox/src/ui/locales/ko.js b/src/firefox/src/ui/locales/ko.js
index 9fdc8e453..f8f8be97e 100644
--- a/src/firefox/src/ui/locales/ko.js
+++ b/src/firefox/src/ui/locales/ko.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Korean (ko).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': '확장 프로그램 UI 확대/축소',
'sp.ui_scale.decrease': '확장 프로그램 UI 축소',
'sp.ui_scale.increase': '확장 프로그램 UI 확대',
diff --git a/src/firefox/src/ui/locales/ms.js b/src/firefox/src/ui/locales/ms.js
index 6e11c252b..78917b8ba 100644
--- a/src/firefox/src/ui/locales/ms.js
+++ b/src/firefox/src/ui/locales/ms.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Malay (ms).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Zum UI sambungan',
'sp.ui_scale.decrease': 'Kecilkan UI sambungan',
'sp.ui_scale.increase': 'Besarkan UI sambungan',
diff --git a/src/firefox/src/ui/locales/nl.js b/src/firefox/src/ui/locales/nl.js
index a3f16e97a..92000c9c9 100644
--- a/src/firefox/src/ui/locales/nl.js
+++ b/src/firefox/src/ui/locales/nl.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Dutch (nl).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Zoom van extensie-interface',
'sp.ui_scale.decrease': 'Extensie-interface uitzoomen',
'sp.ui_scale.increase': 'Extensie-interface inzoomen',
diff --git a/src/firefox/src/ui/locales/pl.js b/src/firefox/src/ui/locales/pl.js
index 4e45f0b16..4466fce17 100644
--- a/src/firefox/src/ui/locales/pl.js
+++ b/src/firefox/src/ui/locales/pl.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Polski — translated from en.js. Keys mirror the English canonical file.
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Powiększenie interfejsu rozszerzenia',
'sp.ui_scale.decrease': 'Pomniejsz interfejs rozszerzenia',
'sp.ui_scale.increase': 'Powiększ interfejs rozszerzenia',
diff --git a/src/firefox/src/ui/locales/pt.js b/src/firefox/src/ui/locales/pt.js
index 7dbfe6ad9..db554bc02 100644
--- a/src/firefox/src/ui/locales/pt.js
+++ b/src/firefox/src/ui/locales/pt.js
@@ -1,8 +1,10 @@
+import bidiCopy from './bidi-copy.mjs';
// Portuguese — translated from the canonical English locale.
import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Zoom da interface da extensão',
'sp.ui_scale.decrease': 'Reduzir o zoom da interface',
'sp.ui_scale.increase': 'Aumentar o zoom da interface',
diff --git a/src/firefox/src/ui/locales/ru.js b/src/firefox/src/ui/locales/ru.js
index 1e1e9cb06..8104d803d 100644
--- a/src/firefox/src/ui/locales/ru.js
+++ b/src/firefox/src/ui/locales/ru.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Russian (ru).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Масштаб интерфейса расширения',
'sp.ui_scale.decrease': 'Уменьшить интерфейс расширения',
'sp.ui_scale.increase': 'Увеличить интерфейс расширения',
diff --git a/src/firefox/src/ui/locales/th.js b/src/firefox/src/ui/locales/th.js
index a90a28319..da910287d 100644
--- a/src/firefox/src/ui/locales/th.js
+++ b/src/firefox/src/ui/locales/th.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Thai (th).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'ซูม UI ของส่วนขยาย',
'sp.ui_scale.decrease': 'ย่อ UI ของส่วนขยาย',
'sp.ui_scale.increase': 'ขยาย UI ของส่วนขยาย',
diff --git a/src/firefox/src/ui/locales/tl.js b/src/firefox/src/ui/locales/tl.js
index 20eccbc92..b8e54de6b 100644
--- a/src/firefox/src/ui/locales/tl.js
+++ b/src/firefox/src/ui/locales/tl.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Filipino / Tagalog (tl).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Pag-zoom ng UI ng extension',
'sp.ui_scale.decrease': 'Bawasan ang zoom ng UI ng extension',
'sp.ui_scale.increase': 'Dagdagan ang zoom ng UI ng extension',
diff --git a/src/firefox/src/ui/locales/tr.js b/src/firefox/src/ui/locales/tr.js
index e57214aec..c06bbb69e 100644
--- a/src/firefox/src/ui/locales/tr.js
+++ b/src/firefox/src/ui/locales/tr.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Turkish (tr).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Uzantı arayüzü yakınlaştırması',
'sp.ui_scale.decrease': 'Uzantı arayüzünü uzaklaştır',
'sp.ui_scale.increase': 'Uzantı arayüzünü yakınlaştır',
diff --git a/src/firefox/src/ui/locales/uk.js b/src/firefox/src/ui/locales/uk.js
index 325dbeb3d..6bf23381e 100644
--- a/src/firefox/src/ui/locales/uk.js
+++ b/src/firefox/src/ui/locales/uk.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Ukrainian (uk).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Масштаб інтерфейсу розширення',
'sp.ui_scale.decrease': 'Зменшити інтерфейс розширення',
'sp.ui_scale.increase': 'Збільшити інтерфейс розширення',
diff --git a/src/firefox/src/ui/locales/vi.js b/src/firefox/src/ui/locales/vi.js
index 455be9310..1d12c33c8 100644
--- a/src/firefox/src/ui/locales/vi.js
+++ b/src/firefox/src/ui/locales/vi.js
@@ -1,8 +1,10 @@
+import bidiCopy from './bidi-copy.mjs';
// Vietnamese — translated from the canonical English locale.
import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': 'Thu phóng giao diện tiện ích',
'sp.ui_scale.decrease': 'Thu nhỏ giao diện tiện ích',
'sp.ui_scale.increase': 'Phóng to giao diện tiện ích',
diff --git a/src/firefox/src/ui/locales/zh.js b/src/firefox/src/ui/locales/zh.js
index 94670d70c..2599c87bb 100644
--- a/src/firefox/src/ui/locales/zh.js
+++ b/src/firefox/src/ui/locales/zh.js
@@ -1,3 +1,4 @@
+import bidiCopy from './bidi-copy.mjs';
// Simplified Chinese (zh).
import chromeWebStoreLocale from './chrome-web-store.mjs';
@@ -5,6 +6,7 @@ import { getApocalypseModeCopy } from './apocalypse-copy.mjs';
import { getEmergencyBoxCopy } from './emergency-copy.mjs';
export default {
+ ...bidiCopy,
'sp.ui_scale.label': '插件界面缩放',
'sp.ui_scale.decrease': '缩小插件界面',
'sp.ui_scale.increase': '放大插件界面',
diff --git a/src/firefox/src/ui/settings.html b/src/firefox/src/ui/settings.html
index c4fdeafa1..34f4409ba 100644
--- a/src/firefox/src/ui/settings.html
+++ b/src/firefox/src/ui/settings.html
@@ -297,6 +297,10 @@
transition: border-color 0.2s;
}
.provider-card.selected { border-color: var(--accent); }
+ #firefox-bidi-card h3 { font-size: 14px; margin-bottom: 8px; }
+ #firefox-bidi-card p { font-size: 12px; line-height: 1.55; color: var(--text2); margin: 8px 0; }
+ #firefox-bidi-card > label { display: block; font-size: 12px; margin: 12px 0; }
+ #firefox-bidi-card code { overflow-wrap: anywhere; }
.provider-card.collapsed { padding: 12px 16px; }
.provider-card.collapsed .provider-header { margin-bottom: 0; }
@@ -1595,6 +1599,15 @@
</label>
</div>
+ <div class="provider-card" id="firefox-bidi-card">
+ <h3 data-i18n="st.bidi.title">Firefox trusted automation (experimental)</h3>
+ <p data-i18n="st.bidi.desc">Use a local companion for browser dialogs, trusted clicks and typing, and file uploads. Requires Node.js 22+ and Firefox started with remote automation enabled.</p>
+ <p data-i18n-html="st.bidi.setup_html">Install the companion using <code>node firefox-companion/install.mjs</code> from the WebBrain checkout. Restart Firefox with <code>--remote-debugging-port 9222</code>. Keep the connection on this computer.</p>
+ <label><input type="checkbox" id="firefox-bidi-enabled"> <span data-i18n="st.bidi.enable">Use trusted Firefox automation</span></label>
+ <div class="field"><label for="firefox-bidi-port" data-i18n="st.bidi.port">Local Firefox port</label><input id="firefox-bidi-port" type="number" min="1024" max="65535" value="9222"></div>
+ <button type="button" id="firefox-bidi-connect" class="btn-secondary" data-i18n="st.bidi.test">Test connection</button>
+ <p id="firefox-bidi-status" role="status"></p>
+ </div>
<div class="provider-card" id="captcha-card">
<div style="font-size:12px;color:var(--text2);margin-bottom:14px;line-height:1.55;" data-i18n-html="st.captcha.desc_html"></div>
diff --git a/src/firefox/src/ui/settings.js b/src/firefox/src/ui/settings.js
index 4eb24edbb..637443d4d 100644
--- a/src/firefox/src/ui/settings.js
+++ b/src/firefox/src/ui/settings.js
@@ -3564,3 +3564,31 @@ async function sendToBackground(action, data = {}) {
}
init();
+
+// Firefox-only companion controls; kept off by default for extension-only users.
+const bidiEnabled = document.getElementById('firefox-bidi-enabled');
+const bidiPort = document.getElementById('firefox-bidi-port');
+const bidiStatus = document.getElementById('firefox-bidi-status');
+if (bidiEnabled && bidiPort) {
+ browser.storage.local.get(['firefoxBidiEnabled', 'firefoxBidiPort']).then(values => {
+ bidiEnabled.checked = values.firefoxBidiEnabled === true;
+ bidiPort.value = values.firefoxBidiPort || 9222;
+ });
+ const saveBidi = async () => {
+ const port = Number(bidiPort.value);
+ if (!Number.isInteger(port) || port < 1024 || port > 65535) { bidiStatus.textContent = t('st.bidi.invalid_port'); return false; }
+ await browser.storage.local.set({ firefoxBidiEnabled: bidiEnabled.checked, firefoxBidiPort: port });
+ bidiStatus.textContent = bidiEnabled.checked ? t('st.bidi.enabled') : t('st.bidi.disabled');
+ return true;
+ };
+ bidiEnabled.addEventListener('change', saveBidi);
+ bidiPort.addEventListener('change', saveBidi);
+ document.getElementById('firefox-bidi-connect').addEventListener('click', async () => {
+ if (!await saveBidi()) return;
+ bidiStatus.textContent = t('st.bidi.connecting');
+ try {
+ const result = await browser.runtime.sendMessage({ type: 'WB_BIDI_CONNECT' });
+ bidiStatus.textContent = result?.success ? t('st.bidi.connected') : result?.error || t('st.bidi.failed');
+ } catch (error) { bidiStatus.textContent = error.message; }
+ });
+}
diff --git a/test/firefox-bidi-e2e.mjs b/test/firefox-bidi-e2e.mjs
new file mode 100644
index 000000000..ae288bdf5
--- /dev/null
+++ b/test/firefox-bidi-e2e.mjs
@@ -0,0 +1,161 @@
+import assert from 'node:assert/strict';
+import { spawn } from 'node:child_process';
+import { mkdtemp, rm, writeFile } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { createServer } from 'node:http';
+import { fileURLToPath } from 'node:url';
+import { BidiSession } from '../firefox-companion/session.mjs';
+const profile = await mkdtemp(join(tmpdir(), 'webbrain-firefox-test-'));
+const server = createServer((_req, res) => {
+ res.setHeader('Content-Type', 'text/html');
+ res.end(`<button id="click">Click</button><input id="text"><input id="file" type="file"><script>
+ window.events=[]; document.addEventListener('click',e=>events.push(['click',e.isTrusted]));
+ document.addEventListener('input',e=>events.push(['input',e.isTrusted]));
+ document.querySelector('#click').onclick=()=>{window.answer=confirm('Continue?')};
+ </script>`);
+});
+await new Promise(resolve => server.listen(0, '127.0.0.1', resolve));
+const url = `http://127.0.0.1:${server.address().port}/`;
+const firefox = spawn(process.env.FIREFOX_BINARY || '/Applications/Firefox.app/Contents/MacOS/firefox', ['--headless', '--remote-allow-system-access', '--no-remote', '--profile', profile, '--remote-debugging-port', '0'], { stdio: ['ignore', 'pipe', 'pipe'] });
+let session;
+try {
+ const port = await new Promise((resolve, reject) => {
+ const timer = setTimeout(() => reject(new Error('Firefox did not start BiDi')), 20000);
+ let output = '';
+ firefox.stderr.on('data', data => { output += data; const match = output.match(/WebDriver BiDi listening on ws:\/\/127\.0\.0\.1:(\d+)/); if (match) { clearTimeout(timer); resolve(Number(match[1])); } });
+ firefox.on('error', reject);
+ });
+ session = new BidiSession();
+ await session.connect(port);
+ await session.send('webExtension.install', { extensionData: { type: 'path', path: fileURLToPath(new URL('../src/firefox/', import.meta.url)) } });
+ const { context } = await session.send('browsingContext.create', { type: 'tab' });
+ await session.send('browsingContext.navigate', { context, url, wait: 'complete' });
+ const evaluate = async expression => (await session.send('script.evaluate', { target: { context }, expression, awaitPromise: true })).result;
+ const mark = async selector => {
+ const token = crypto.randomUUID();
+ await evaluate(`document.querySelector(${JSON.stringify(selector)}).setAttribute('data-webbrain-bidi', ${JSON.stringify(token)})`);
+ return { token, url };
+ };
+ const runId = crypto.randomUUID();
+ const binding = await mark('html'); await session.openRun(runId, binding.token, binding.url);
+ assert.equal((await session.perform(runId, 'click', await mark('#click'))).success, true);
+ assert.equal((await evaluate('window.answer')).value, false, 'unapproved confirm must be dismissed');
+ assert.equal((await evaluate('events.some(e => e[0] === "click" && e[1])')).value, true);
+ await evaluate(`document.querySelector('#click').onclick=()=>{window.answer=prompt('Input?')}`);
+ await session.perform(runId, 'click', await mark('#click'));
+ assert.equal((await evaluate('window.answer')).type, 'null');
+ await evaluate(`document.querySelector('#click').onclick=()=>{alert('Notice');window.acknowledged=true}`);
+ await session.perform(runId, 'click', await mark('#click'));
+ assert.equal((await evaluate('window.acknowledged')).value, true);
+ const typed = await session.perform(runId, 'field', { ...await mark('#text'), text: 'Hello Firefox', clear: true });
+ assert.equal(typed.verified, true);
+ assert.equal((await evaluate('document.querySelector("#text").value')).value, 'Hello Firefox');
+ assert.equal((await evaluate('events.some(e => e[0] === "input" && e[1])')).value, true);
+ const uploaded = await session.perform(runId, 'upload', { ...await mark('#file'), filename: 'sample.txt', base64: Buffer.from('upload test').toString('base64') });
+ assert.equal(uploaded.success, true);
+ assert.equal((await evaluate('document.querySelector("#file").files[0].text()')).value, 'upload test');
+ const tree = await session.send('browsingContext.getTree', {});
+ const extensionContext = tree.contexts.find(item => item.url?.startsWith('moz-extension://'));
+ assert.ok(extensionContext, 'installed extension opens onboarding');
+ const extensionOrigin = new URL(extensionContext.url).origin === 'null' ? extensionContext.url.split('/').slice(0, 3).join('/') : new URL(extensionContext.url).origin;
+ const extensionEval = async expression => {
+ const result = await session.send('script.evaluate', { target: { context: extensionContext.context }, expression, awaitPromise: true });
+ if (result.type === 'exception') throw new Error(result.exceptionDetails.text);
+ return result.result;
+ };
+ const tabId = (await extensionEval(`browser.tabs.query({}).then(tabs => tabs.find(tab => tab.url === ${JSON.stringify(url)}).id)`)).value;
+ const token = crypto.randomUUID();
+ const prepared = await extensionEval(`browser.tabs.sendMessage(${tabId}, {target:'content', action:'type', params:{selector:'#text', text:'Through extension', clear:true, _bidiPrepare:${JSON.stringify(token)}}}).then(value => JSON.stringify(value))`);
+ assert.equal(JSON.parse(prepared.value).bidiPrepared, true, 'packaged content script prepares a target');
+ assert.equal((await session.perform(runId, 'type', { token, url, text: 'Through extension', clear: true })).verified, true);
+ // A newline is literal editing, never an Enter-to-send shortcut.
+ await evaluate(`document.body.insertAdjacentHTML('beforeend', '<textarea id="multiline"></textarea><input id="other">'); window.enters=0; document.addEventListener('keydown', e=>{if(e.key==='Enter'){window.enters++;e.preventDefault()}});`);
+ const multiline = await session.perform(runId, 'field', { ...await mark('#multiline'), text: 'first\nsecond', clear: true, submit: false });
+ assert.equal(multiline.verified, true);
+ assert.equal((await evaluate('window.enters')).value, 0);
+ await assert.rejects(session.perform(runId, 'field', { ...await mark('#text'), text: 'first\nsecond', clear: true, submit: false }), /multiline/);
+ assert.equal((await evaluate('document.querySelector("#text").value')).value, 'Through extension');
+ // Focus handlers must not redirect a secret into another input.
+ await evaluate(`document.querySelector('#other').focus(); document.querySelector('#text').onfocus=()=>document.querySelector('#other').focus();`);
+ await assert.rejects(session.perform(runId, 'field', { ...await mark('#text'), text: 'private payload', clear: true }), /Target|Focus/);
+ assert.equal((await evaluate('document.querySelector("#other").value')).value, '');
+ await evaluate(`document.querySelector('#text').onfocus=null;`);
+ await evaluate(`document.querySelector('#text').oninput=()=>{if(document.querySelector('#text').value==='ready')document.querySelector('#other').focus()};`);
+ await assert.rejects(session.perform(runId, 'field', { ...await mark('#text'), text: 'ready', clear: true, submit: true }), /Focus changed/);
+ assert.equal((await evaluate('window.enters')).value, 0);
+ await evaluate(`document.querySelector('#text').oninput=null; document.body.insertAdjacentHTML('beforeend','<div id="rich" contenteditable="true" style="width:300px;height:80px;border:1px solid"></div>');`);
+ assert.equal((await session.perform(runId, 'field', { ...await mark('#rich'), text: 'first\nsecond', clear: true, submit: false })).verified, true);
+ assert.equal((await evaluate('window.enters')).value, 0);
+
+ // Invoke the real client in the packaged extension with a transport stub; no
+ // native manifest is installed in the user's normal Firefox configuration.
+ await session.perform(runId, 'key', { ...await mark('#text'), key: 'Tab', repeat: 2 });
+ assert.equal((await evaluate('document.activeElement.id')).value, 'multiline');
+ const deferred = await extensionEval(`(async () => {
+ const {FirefoxBidiClient} = await import(${JSON.stringify(extensionOrigin + '/src/bidi/client.js')});
+ const client = new FirefoxBidiClient(browser);
+ client.connect = async () => ({});
+ const calls = []; client.request = async (command) => { calls.push(command); return {}; };
+ await browser.storage.local.set({firefoxBidiEnabled:true});
+ const blank = await browser.tabs.create({url:'about:blank'});
+ await client.startRun(blank.id);
+ const initial = client.runs.get(blank.id).bound === true;
+ await client.perform(blank.id, 'navigate', {url:${JSON.stringify(url)}});
+ await new Promise(resolve => setTimeout(resolve, 500));
+ await client.bindRun(blank.id, client.runs.get(blank.id));
+ const bound = client.runs.get(blank.id).bound;
+ client.stopRun(blank.id); await browser.tabs.remove(blank.id);
+ await browser.storage.local.set({firefoxBidiEnabled:false});
+ return JSON.stringify({initial,bound,calls});
+ })()`);
+ const deferredResult = JSON.parse(deferred.value);
+ assert.equal(deferredResult.initial, false);
+ assert.equal(deferredResult.bound, true);
+ assert.ok(deferredResult.calls.includes('openRun'));
+ const settings = await session.send('browsingContext.create', { type: 'tab' });
+ await session.send('browsingContext.navigate', { context: settings.context, url: extensionOrigin + '/src/ui/settings.html', wait: 'complete' });
+ await session.send('script.evaluate', { target: { context: settings.context }, expression: "(async () => { document.querySelector('[data-tab=display]').click(); await new Promise(r => setTimeout(r, 500)); document.getElementById('firefox-bidi-card').closest('details').open = true; document.getElementById('firefox-bidi-card').scrollIntoView({behavior:'instant', block:'center'}); await new Promise(r => setTimeout(r, 250)); })()", awaitPromise: true });
+ const screenshot = await extensionEval(`browser.tabs.query({}).then(tabs => browser.tabs.captureTab(tabs.find(tab => tab.url.endsWith('/src/ui/settings.html')).id, {format:'png'}))`);
+ await writeFile('/tmp/webbrain3-firefox-bidi-settings.png', Buffer.from(screenshot.value.split(',')[1], 'base64'));
+ await evaluate(`window.addEventListener('beforeunload', event => { event.preventDefault(); event.returnValue = ''; })`);
+ assert.equal((await session.perform(runId, 'navigate', { url: url + 'next' })).success, true);
+ assert.equal((await evaluate('location.href')).value, url + 'next');
+ await session.closeRun(runId);
+ // Remember an idle tab's dialog, but handle it only once a run owns that tab.
+ await session.send('browsingContext.navigate', {context, url, wait:'complete'});
+ const bind = await mark('html');
+ const opened = new Promise(resolve => {
+ const receive = session.receive.bind(session);
+ session.receive = message => { receive(message); if (message.method === 'browsingContext.userPromptOpened') resolve(); };
+ });
+ await evaluate('setTimeout(()=>alert("Pending notice"),0)'); await opened;
+ assert.ok(session.dialogs.has(context));
+ const resumed = crypto.randomUUID();
+ await session.openRun(resumed, bind.token, bind.url);
+ assert.equal(session.dialogs.has(context), false);
+ assert.equal((await session.perform(resumed, 'field', { ...await mark('#text'), text: 'resumed', clear: true })).verified, true);
+ // Stop after the first real text key command has been sent. At most that
+ // single in-flight character may land; the rest must never be queued.
+ const originalSend = session.send.bind(session); let stopped = false;
+ session.send = (method, params) => {
+ const result = originalSend(method, params);
+ if (method === 'input.performActions' && !stopped) {
+ stopped = true; void session.closeRun(resumed);
+ }
+ return result;
+ };
+ await assert.rejects(session.perform(resumed, 'type', { ...await mark('#text'), text: 'a'.repeat(500) }), /stopped/);
+ const afterStop = (await evaluate('document.querySelector("#text").value')).value;
+ assert.ok(afterStop.length <= 'resumed'.length + 1, afterStop);
+ await new Promise(resolve => setTimeout(resolve, 100));
+ assert.equal((await evaluate('document.querySelector("#text").value')).value, afterStop);
+ await assert.rejects(session.perform(resumed, 'click', await mark('#click')), /stopped|disconnected/);
+ console.log('PASS: Firefox connection, dialogs, uploads, literal newlines, focus guards, interrupted typing, deferred blank-tab binding, and packaged extension integration');
+} finally {
+ await session?.close().catch(() => {});
+ firefox.kill('SIGTERM');
+ await new Promise(resolve => { if (firefox.exitCode != null) resolve(); else { firefox.once('exit', resolve); setTimeout(resolve, 3000); } });
+ server.close();
+ await rm(profile, { recursive: true, force: true });
+}
diff --git a/test/firefox-bidi.mjs b/test/firefox-bidi.mjs
new file mode 100644
index 000000000..e330a5c36
--- /dev/null
+++ b/test/firefox-bidi.mjs
@@ -0,0 +1,172 @@
+import { test } from 'node:test';
+import assert from 'node:assert/strict';
+import { FirefoxBidiClient } from '../src/firefox/src/bidi/client.js';
+import { BidiSession } from '../firefox-companion/session.mjs';
+
+const id = () => crypto.randomUUID();
+test('connection loss cannot fall through to synthetic input', async () => {
+ let messages = 0;
+ const client = new FirefoxBidiClient({ tabs: { sendMessage() { messages++; } } });
+ client.runs.set(1, { runId: id(), bound: true });
+ client.disconnect();
+ const result = await client.sendContent(1, { action: 'click' });
+ assert.equal(result.noDispatch, true);
+ assert.equal(messages, 0);
+});
+test('a replacement run cannot inherit prepared input from its predecessor', async () => {
+ let finish;
+ const client = new FirefoxBidiClient({ tabs: { sendMessage: () => new Promise(resolve => { finish = resolve; }) } });
+ client.runs.set(1, { runId: id(), bound: true });
+ let dispatches = 0; client.request = async () => { dispatches++; };
+ const pending = client.sendContent(1, { action: 'click' });
+ await Promise.resolve();
+ client.runs.set(1, { runId: id(), bound: true });
+ finish({ bidiPrepared: true, url: 'https://example.com/' });
+ assert.equal((await pending).noDispatch, true);
+ assert.equal(dispatches, 0);
+});
+test('stopping during native tab binding prevents late ownership', async () => {
+ const session = new BidiSession(); const runId = id();
+ let finish;
+ session.locate = () => new Promise(resolve => { finish = resolve; });
+ session.call = async () => ({});
+ const opening = session.openRun(runId, id(), 'https://example.com/');
+ await session.closeRun(runId);
+ finish({ context: 'tab' });
+ await assert.rejects(opening, /stopped/);
+ assert.equal(session.runs.size, 0);
+});
+test('dialogs are scoped to a live run and navigation permission is single use', () => {
+ const session = new BidiSession(); const replies = [];
+ session.send = async (method, params) => { replies.push({ method, ...params }); };
+ const event = (context, type) => session.receive({ method: 'browsingContext.userPromptOpened', params: { context, type } });
+ event('idle', 'alert'); assert.equal(replies.length, 0);
+ const run = { context: 'owned', navigation: false }; session.runs.set(id(), run);
+ for (const type of ['confirm', 'prompt', 'beforeunload', 'alert']) event('owned', type);
+ assert.deepEqual(replies.map(item => item.accept), [false, false, false, true]);
+ run.navigation = true;
+ event('owned', 'beforeunload'); event('owned', 'beforeunload');
+ assert.deepEqual(replies.slice(-2).map(item => item.accept), [true, false]);
+});
+test('target lookup rejects ambiguity even when URLs match', async () => {
+ const session = new BidiSession();
+ session.send = async method => method === 'browsingContext.getTree'
+ ? { contexts: [{ context: 'a', url: 'https://example.com/' }, { context: 'b', url: 'https://example.com/' }] }
+ : { nodes: [{ sharedId: 'node' }] };
+ await assert.rejects(session.locate(id(), 'https://example.com/'), /ambiguous/);
+});
+test('stop during target preparation prevents input dispatch', async () => {
+ const session = new BidiSession(); const runId = id(); const sent = [];
+ session.runs.set(runId, { context: 'a' });
+ session.locate = async () => ({ context: 'a', node: { sharedId: 'el' } });
+ session.call = async () => { await session.closeRun(runId); return { result: { value: true } }; };
+ session.send = async method => { sent.push(method); return {}; };
+ await assert.rejects(session.perform(runId, 'click', {}), /stopped/);
+ assert.equal(sent.includes('input.performActions'), false);
+});
+test('unsupported commands and modifiers fail before touching the page', async () => {
+ const session = new BidiSession(); const runId = id();
+ session.runs.set(runId, { context: 'a' });
+ await assert.rejects(session.perform(runId, 'rawScript', {}), /Unsupported/);
+ await assert.rejects(session.perform(runId, 'key', { modifiers: ['Control'] }), /modifiers/);
+});
+
+test('installer and native launcher preserve Firefox message framing and command whitelist', async () => {
+ const { mkdtemp, readFile, rm } = await import('node:fs/promises');
+ const { tmpdir } = await import('node:os');
+ const { join } = await import('node:path');
+ const { spawnSync } = await import('node:child_process');
+ const directory = await mkdtemp(join(tmpdir(), 'webbrain-native-test-'));
+ try {
+ const install = spawnSync(process.execPath, ['firefox-companion/install.mjs', directory], { encoding: 'utf8' });
+ assert.equal(install.status, 0, install.stderr);
+ const manifest = JSON.parse(await readFile(join(directory, 'one.webbrain.bidi.json'), 'utf8'));
+ assert.deepEqual(manifest.allowed_extensions, ['[email protected]']);
+ const message = Buffer.from(JSON.stringify({ id: 12, command: 'script.evaluate', expression: 'arbitrary code' }));
+ const header = Buffer.alloc(4); header.writeUInt32LE(message.length);
+ const result = spawnSync(manifest.path, [], { input: Buffer.concat([header, message]), timeout: 5000 });
+ assert.equal(result.status, 0, result.stderr?.toString());
+ assert.equal(result.stdout.readUInt32LE(0), result.stdout.length - 4);
+ assert.deepEqual(JSON.parse(result.stdout.subarray(4)), { id: 12, error: 'Unknown companion command' });
+ } finally { await rm(directory, { recursive: true, force: true }); }
+});
+
+test('expired action deadline fails before target lookup', async () => {
+ const session = new BidiSession(); const runId = id();
+ session.runs.set(runId, { context: 'a' });
+ session.locate = () => assert.fail('expired actions must not locate a target');
+ await assert.rejects(session.perform(runId, 'click', { deadlineAt: Date.now() - 1 }), /deadline/);
+});
+
+test('blank-tab runs defer binding and retain ownership through navigation', async () => {
+ let url = 'about:blank', injections = 0; const calls = [];
+ const client = new FirefoxBidiClient({
+ storage: { local: { get: async () => ({firefoxBidiEnabled:true}) } },
+ tabs: {
+ get: async () => ({url}),
+ update: async (_id, value) => { url = value.url; },
+ executeScript: async () => { injections++; return [url]; },
+ },
+ });
+ client.connect = async () => ({});
+ client.request = async command => { calls.push(command); return {}; };
+ await client.startRun(1);
+ assert.equal(injections, 0);
+ assert.ok(client.runs.has(1));
+ assert.equal((await client.perform(1, 'navigate', {url:'https://example.com/'})).bindingDeferred, true);
+ await client.bindRun(1, client.runs.get(1));
+ assert.equal(injections, 1);
+ assert.deepEqual(calls, ['openRun']);
+});
+test('pending dialogs are handled only after binding their owner', async () => {
+ const session = new BidiSession(); const calls = [];
+ session.send = async (method, params) => { calls.push({method,params}); return {}; };
+ session.locate = async () => ({context:'tab', node:{sharedId:'html'}});
+ session.receive({method:'browsingContext.userPromptOpened',params:{context:'tab',type:'confirm'}});
+ assert.equal(calls.length, 0);
+ await session.openRun(id(), id(), 'https://example.com/');
+ assert.equal(calls[0].method, 'browsingContext.handleUserPrompt');
+ assert.equal(calls[0].params.accept, false);
+ assert.equal(session.dialogs.size, 0);
+});
+test('closed idle dialogs are not replayed when a task binds', async () => {
+ const session = new BidiSession(); const calls = [];
+ session.send = async method => { calls.push(method); return {}; };
+ session.locate = async () => ({context:'tab', node:{sharedId:'html'}});
+ session.receive({method:'browsingContext.userPromptOpened',params:{context:'tab',type:'alert'}});
+ session.receive({method:'browsingContext.userPromptClosed',params:{context:'tab'}});
+ await session.openRun(id(), id(), 'https://example.com/');
+ assert.deepEqual(calls, ['script.callFunction']);
+});
+test('a focus change blocks key dispatch before any text can leak', async () => {
+ const session = new BidiSession(); const runId = id(); const sent = [];
+ session.runs.set(runId, {context:'tab'});
+ session.locate = async () => ({context:'tab',node:{sharedId:'input'}});
+ session.call = async (_match, fn) => ({result:{value:fn.includes('activeElement === el') ? false : true}});
+ session.send = async method => { sent.push(method); return {}; };
+ await assert.rejects(session.perform(runId, 'type', {text:'secret'}), /Focus changed/);
+ assert.equal(sent.includes('input.performActions'), false);
+});
+test('Stop after one character never queues the remaining text', async () => {
+ const session = new BidiSession(); const runId = id(); const dispatched = [];
+ session.runs.set(runId, {context:'tab'});
+ session.locate = async () => ({context:'tab',node:{sharedId:'input'}});
+ session.call = async () => ({result:{value:true}});
+ session.send = async (method, params) => {
+ if (method === 'input.performActions') { dispatched.push(params); await session.closeRun(runId); }
+ return {};
+ };
+ await assert.rejects(session.perform(runId, 'type', {text:'hello'}), /stopped/);
+ assert.equal(dispatched.length, 1);
+ assert.equal(dispatched[0].actions[0].actions.length, 2);
+});
+test('literal newline insertion never dispatches an Enter key', async () => {
+ const session = new BidiSession(); const runId = id(); const dispatched = []; const scripts = [];
+ session.runs.set(runId, {context:'tab'});
+ session.locate = async () => ({context:'tab',node:{sharedId:'input'}});
+ session.call = async (_match, fn) => { scripts.push(fn); return {result:{value:true}}; };
+ session.send = async (method, params) => { if(method==='input.performActions') dispatched.push(params); return {}; };
+ await session.perform(runId, 'type', {text:'a\nb'});
+ assert.ok(scripts.some(fn=>fn.includes("execCommand('insertText'")));
+ assert.deepEqual(dispatched.flatMap(p=>p.actions[0].actions.map(a=>a.value)), ['a','a','b','b']);
+});