Open-source AI browser agent for Chrome and Firefox (monorepo) π§
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
Support Firefox full-page screenshots through BiDi
Emre Sokullu23dd9c994fef
diff --git a/assets/store-listing-firefox.txt b/assets/store-listing-firefox.txtindex a17711bf3..c4a1856a6 100644--- a/assets/store-listing-firefox.txt+++ b/assets/store-listing-firefox.txt@@ -19,7 +19,7 @@ READ, RESEARCH AND GET THINGS DONEPRACTICAL TOOLS IN THE SAME SIDEBAR-β’ Page screenshots: capture the visible part of a tab and save the image.+β’ Full-page screenshots: capture complete web pages with the optional Firefox trusted automation companion. Visible-tab screenshots work without it.β’ Bulk file downloads: download multiple linked files in one task, including documents and images.β’ Article and video summaries: read New York Times articles and summarize YouTube transcripts through the included reading tools. Availability depends on the source and connected service.β’ CAPTCHA assistance: connect a supported solving service to help continue tasks blocked by supported challenges. Optional setup and a separate service account are required; charges may apply.@@ -49,4 +49,4 @@ Source and support: https://github.com/webbrain-one/webbrainEND DESCRIPTIONEDITORIAL NOTE - not part of the store description:-This is proposed copy, not a published store update. The title and summary are separate metadata; Chrome also requires manifest changes in a future package. See store-listing-review.txt for evidence and publication notes. Full-page capture is not yet wired through the Firefox BiDi companion in this checkout. Correct the separate AMO license field to GPL-3.0-or-later as well.+This is proposed copy, not a published store update. The title and summary are separate metadata; Chrome also requires manifest changes in a future package. See store-listing-review.txt for evidence and publication notes. Full-page capture requires the local Firefox trusted automation companion. Publish this feature claim with a release containing the implementation. Correct the separate AMO license field to GPL-3.0-or-later as well.diff --git a/assets/store-listing-review.txt b/assets/store-listing-review.txtindex 1d89b2a34..1003b4f03 100644--- a/assets/store-listing-review.txt+++ b/assets/store-listing-review.txt@@ -13,7 +13,7 @@ Chrome: https://chromewebstore.google.com/detail/webbrain/ljhijonmfahplgbbacgcfnDisplayed version: 36.1.0; updated September 15, 2026.Firefox: https://addons.mozilla.org/en-US/firefox/addon/webbrain/Displayed version: 36.8.0; updated September 20, 2026.-Local source: a34843d22; both manifests report 38.0.1.+Source for the initial copy audit: a34843d22; both manifests reported 38.0.1. The Firefox implementation follow-up started from 38.0.13.The public descriptions are nearly identical. They omit scheduling, workflows, explicit PDF analysis, bulk downloads, screenshots, article/transcript reading, CAPTCHA integration and Dev mode. The Firefox opening incorrectly says Chrome. Both descriptions say MIT; Firefox's separate license metadata also says MIT. LICENSE specifies GPL-3.0-or-later for 33.0.0 onward.@@ -41,14 +41,14 @@ Source evidence and claim boundariesβ’ New York Times reading and YouTube transcripts: src/{chrome,firefox}/skills/freeskillz-xyz.md, agent/skills.js, docs/skills.md, docs/privacy-and-data-flow.md. The service-backed skill is enabled by default and removable. NYTimes support entered history in July 2026 (834e4c524, 0b68a52bf). Do not imply publisher affiliation, universal article access or a universal paywall bypass. Source/service availability is explicit in the draft.β’ Scheduling/watches: schedule_task in both agent/tools.js files, agent/scheduler.js, docs/slash-commands.md. One-shot and fixed-interval recurrence exist; calendar/cron recurrence does not. Browser alarms require the browser to run. Do not promise execution with the browser/computer shut down.β’ Workflows/teaching: agent/workflows.js, agent/teacher-mode.js, content/teacher-capture.js, docs/export-and-workflow-formats.md. Teacher mode is documented in 27.1.0. Avoid claiming every demonstrated action can be captured/replayed.-β’ Chrome full-page screenshots: agent/agent.js captureFullPageScreenshotForUser and background.js capture_full_page_screenshot. Firefox ui/sidepanel.js:712 still marks --full-page unsupported; its background only routes viewport capture. Shared retired tool names and rendering helpers do not prove a working capture path.-β’ Firefox BiDi: firefox-companion/session.mjs:179 allows navigate, click, hover, type, field, key and upload, with no screenshot operation. src/firefox/src/bidi/client.js does not route screenshot capture. The protocol supports full-document screenshots, but this checkout has not wired that capability into WebBrain. Keep viewport wording until the feature is implemented and verified. This is an implementation gap, not an inherent Firefox limitation.+β’ Chrome full-page screenshots: agent/agent.js captureFullPageScreenshotForUser and background.js capture_full_page_screenshot. The Firefox follow-up now routes --full-page through the local companion, with preview/save and capture-time privacy geometry.+β’ Firefox BiDi: firefox-companion/session.mjs now provides a document-bound full-page capture command; src/firefox/src/bidi/client.js transfers large PNGs in chunks. The updated Firefox draft states the optional companion requirement. This feature is implemented locally and must accompany a containing extension release before it is advertised on the live store.β’ CAPTCHA: agent/captcha-solver.js, ui/captcha-settings.js, docs/captcha-provider-coverage.md. Opt-in integration dates to 7.3.0. The expanded seven-service catalog is newer than the displayed store releases. Advertise generic optional assistance, not that new catalog, particular challenge families or guaranteed acceptance. Disclose separate setup/account/charges.β’ Offline mode: docs/apocalypse-mode.md and the live listings. Keep prior-download and WebGPU qualifications attached to on-device inference, not all local model servers.β’ Model setup: README.md and docs/providers-and-models.md. The built-in default requires no API key. Compatibility depends on the selected model and setup; avoid any-LLM guarantees and rapidly aging model-version lists.β’ Privacy: docs/privacy-and-data-flow.md. Relevant context goes to the selected AI service; local inference and online skill calls are distinct. Help Improve WebBrain is on by default for the built-in model and can be disabled. Avoid all-data-stays-local and API-keys-never-leave claims.-This was a source and listing review, not a live end-to-end test of paid solvers or reading services. Store versions lag the checkout. The draft avoids newly expanded solver counts. Verify package-specific behavior when preparing a submission; source presence alone is not proof of a published feature.+The initial work was a source and listing review. The Firefox follow-up also includes a real-browser test of full-page pixels, large image transfer, below-fold privacy geometry and PNG saving. Paid solvers and reading services were not tested end to end. Store versions lag the checkout. The draft avoids newly expanded solver counts. Verify package-specific behavior when preparing a submission; source presence alone is not proof of a published feature.Search and conversiondiff --git a/docs/privacy-and-data-flow.md b/docs/privacy-and-data-flow.mdindex e105d9fa1..e9124fe12 100644--- a/docs/privacy-and-data-flow.md+++ b/docs/privacy-and-data-flow.md@@ -697,6 +697,7 @@ are present and identical to Chrome (`src/firefox/src/trace/recorder.js`). Alldata-flow patterns are otherwise the same, except:- No dedicated vision sub-call (screenshots go directly to the main provider if vision is supported)+- Full-page screenshots use the optional local Firefox trusted-automation companion over native messaging and loopback BiDi. The PNG stays local until the user attaches it to a model request. Capture-time page-coordinate redaction scans must agree before a privacy-enabled capture can be staged; otherwise only local preview/save is offered.- No slash-driven tab/screen recording- Conversation, rendered chat, and detached-run UI journals use`browser.storage.session`, matching Chrome's session-scoped persistence.diff --git a/docs/slash-commands.md b/docs/slash-commands.mdindex aca173a1b..a5881e23e 100644--- a/docs/slash-commands.md+++ b/docs/slash-commands.md@@ -40,7 +40,7 @@ for its available flags.| `/verbose` | Toggle verbose/compact tool display || `/reset` | Clear the conversation and all per-conversation flags || `/print` | Open the current page's native print dialog |-| `/screenshot [--full-page]` | Capture the visible tab, or the full scrollable page with `--full-page` (Chrome only) |+| `/screenshot [--full-page]` | Capture the visible tab, or the full scrollable page with `--full-page` (Firefox requires the optional trusted-automation companion) || `/record [--full-screen] [--hide-recording-indicator] [--transcribe]` | Record the current tab, or a selected screen/window with `--full-screen` (Chrome only); add `--hide-recording-indicator` to hide the banner or `--transcribe` to save a transcript after stop || `/export [--traces [--full] \| --config]` | Download version-stamped conversation Markdown, export the version-stamped tool chain with `--traces`, add `--full` for the complete stored session JSON including screenshots, or export a Settings snapshot with `--config` || `/import <json>` | Import a Settings snapshot pasted inline |@@ -48,6 +48,10 @@ for its available flags.| `/profile` | Toggle profile auto-fill on/off without opening Settings || `/vision` | Toggle vision mode (screenshot understanding) on the active provider |+## Full-page screenshots on Firefox++Install the [local companion](../firefox-companion/README.md) and enable Firefox trusted automation in Settings before using `/screenshot --full-page`. The capture stays local until you attach it to a message, and Save as exports a PNG. It captures the currently loaded document, including content outside the viewport; it does not scroll to load an infinite feed. Very large pages return an explicit size error. If the privacy scan cannot verify stable sensitive-field geometry, the preview remains saveable but is not attached to the composer. Ordinary `/screenshot` needs no companion.+## `/watch``/watch` performs its first check immediately and then polls every 60 seconds bydiff --git a/firefox-companion/README.md b/firefox-companion/README.mdindex 00908346c..bb3d35f4c 100644--- a/firefox-companion/README.md+++ b/firefox-companion/README.md@@ -15,12 +15,13 @@ Requires Node.js 22 or newer with the built-in WebSocket API, and a current FireUse a separate profile if desired. An already running Firefox process may ignore startup flags. Keep the debugging endpoint on loopback; it grants browser control to local clients.3. Load this branch's `src/firefox/manifest.json` from `about:debugging` β This Firefox β Load Temporary Add-on (or install a build containing these changes).4. In WebBrain Settings β General β Advanced, enable **Firefox trusted automation**, keep the port at 9222 (or match your startup port), and select **Test connection**.-5. Start a new Act or Dev task. Connection failures fail the task rather than silently falling back. Ask mode does not open an automation run. Tasks can start on blank/internal tabs; document binding is deferred until authorized navigation reaches a web page.+5. Use `/screenshot --full-page` to capture a complete web page, or start a new Act or Dev task. Connection failures fail the task rather than silently falling back. Ask mode does not open an automation run. Tasks can start on blank/internal tabs; document binding is deferred until authorized navigation reaches a web page.The connection test checks the real BiDi session. It does not submit forms or send input. Disabling the setting or changing the port disconnects the helper; restart any active task afterward.## First supported scope+- User-driven full-page PNG capture through `/screenshot --full-page`, including English/Turkish screenshot-only requests. It binds the current document independently of agent runs and does not change their ownership or handle dialogs. Navigation, layout/scroll changes, oversized pages and truncated captures fail explicitly. The capture covers the loaded document without scrolling to load more content. Large images use chunked native replies below Firefox's per-message limit. Capture-time privacy scans gate model attachment; an unscannable page can still be previewed/saved locally.- Session connection through Firefox native messaging to loopback BiDi; no arbitrary protocol or script tool exposed to the model.- Coordinate clicks preserve the validated CSS viewport point and recheck that point against the bound element before dispatch. Pre-dispatch validation failures retain `noDispatch:true`; transport failures and failures after input remain uncertain and unsafe to replay.- Trusted clicks (`click`, `click_ax`), hover, text (`type_text`, `type_ax`, `set_field`), and existing supported `press_keys` keys/repeats. Native selects retain their existing semantic selection path. Drag/drop and dedicated iframe tools retain their existing implementations.@@ -35,8 +36,8 @@ The host stores temporary upload files only for its connection lifetime and dele## Testing-- `npm run test:firefox-bidi`: deterministic cancellation, disconnection, dialog policy and binding regressions (also in `npm test`).-- `npm run test:firefox-bidi:e2e`: starts an isolated headless Firefox profile and local fixture, exercises the production BiDi session, verifies `isTrusted`, dialog dismissal, exact typing, and uploaded bytes, then removes the profile. Set `FIREFOX_BINARY` if Firefox is not in the macOS default location. It also temporarily installs the packaged extension and exercises its content-script target preparation. The isolated test enables system access solely to inspect its extension Settings page; normal companion use does not need that flag. It does not install the native host in your normal browser profile.+- `npm run test:firefox-bidi`: deterministic screenshot transfer, document binding, privacy gating, cancellation, disconnection and dialog regressions (also in `npm test`).+- `npm run test:firefox-bidi:e2e`: starts an isolated headless Firefox profile and local fixture, exercises the production BiDi session, verifies full-page pixels, large image transfers, below-fold privacy geometry, PNG saving, `isTrusted`, dialog dismissal, exact typing, and uploaded bytes, then removes the profile. Set `FIREFOX_BINARY` if Firefox is not in the macOS default location. It also temporarily installs the packaged extension and exercises its content-script target preparation. The isolated test enables system access solely to inspect its extension Settings page; normal companion use does not need that flag. It does not install the native host in your normal browser profile.## Uninstalldiff --git a/firefox-companion/host.mjs b/firefox-companion/host.mjsindex 6d278bab4..da8823dba 100644--- a/firefox-companion/host.mjs+++ b/firefox-companion/host.mjs@@ -1,11 +1,14 @@#!/usr/bin/env nodeimport { BidiSession } from './session.mjs';+import { nativeReplyMessages } from './native-messages.mjs';const session = new BidiSession();let buffer = Buffer.alloc(0);function reply(value) {- const data = Buffer.from(JSON.stringify(value));- const header = Buffer.alloc(4); header.writeUInt32LE(data.length);- process.stdout.write(Buffer.concat([header, data]));+ for (const message of nativeReplyMessages(value)) {+ const data = Buffer.from(JSON.stringify(message));+ const header = Buffer.alloc(4); header.writeUInt32LE(data.length);+ process.stdout.write(Buffer.concat([header, data]));+ }}async function dispatch(message) {const { id, command, ...args } = message;@@ -16,6 +19,7 @@ async function dispatch(message) {case 'openRun': result = await session.openRun(args.runId, args.token, args.url); break;case 'closeRun': result = await session.closeRun(args.runId); break;case 'perform': result = await session.perform(args.runId, args.action, args.payload || {}); break;+ case 'captureFullPage': result = await session.captureFullPage(args.token, args.url); break;default: throw new Error('Unknown companion command');}reply({ id, result });diff --git a/firefox-companion/native-messages.mjs b/firefox-companion/native-messages.mjsnew file mode 100644index 000000000..a374f916e--- /dev/null+++ b/firefox-companion/native-messages.mjs@@ -0,0 +1,14 @@+// Firefox caps each native-host message at 1 MiB. PNGs routinely exceed it.+// Chunk the serialized reply, keeping framing independent of screenshot content.+export function* nativeReplyMessages(value) {+ const serialized = JSON.stringify(value);+ if (serialized.length > 32 * 1024 * 1024) throw new Error('Screenshot exceeds the 32 MiB transfer limit');+ if (Buffer.byteLength(serialized) < 512 * 1024) {+ yield value;+ return;+ }+ const size = 128 * 1024;+ for (let offset = 0, index = 0; offset < serialized.length; offset += size, index++) {+ yield { id: value.id, chunk: serialized.slice(offset, offset + size), index, last: offset + size >= serialized.length };+ }+}diff --git a/firefox-companion/session.mjs b/firefox-companion/session.mjsindex 7199ebacc..64471bdd6 100644--- a/firefox-companion/session.mjs+++ b/firefox-companion/session.mjs@@ -109,6 +109,44 @@ export class BidiSession {await this.call(match, '(el) => { el.removeAttribute("data-webbrain-bidi"); return true; }');return { connected: true };}+ async captureFullPage(token, url) {+ // A user capture has its own document binding. It never takes over an+ // automation run, dismisses its dialogs, or releases its pressed keys.+ const match = await this.locate(token, url);+ const readDocument = async () => {+ const result = await this.call(match, `(el) => {+ if (!el.isConnected || el !== document.documentElement || window !== window.top) return null;+ return JSON.stringify({ url: location.href, timeOrigin: performance.timeOrigin,+ width: Math.max(el.scrollWidth, innerWidth), height: Math.max(el.scrollHeight, innerHeight),+ dpr: devicePixelRatio, scrollX, scrollY });+ }`);+ if (result.type === 'exception' || typeof result.result?.value !== 'string') throw new Error('Screenshot document changed; capture again');+ return JSON.parse(result.result.value);+ };+ try {+ const before = await readDocument();+ if (before.url !== url) throw new Error('Screenshot document changed; capture again');+ const { width, height, dpr } = before;+ if (![width, height, dpr].every(value => Number.isFinite(value) && value > 0)+ || width * dpr > 32767 || height * dpr > 32767 || width * height * dpr * dpr > 50_000_000) {+ throw new Error('This page is too large for a full-page screenshot; use /screenshot for the visible area');+ }+ const result = await this.send('browsingContext.captureScreenshot', {+ context: match.context, origin: 'document', format: { type: 'image/png' },+ });+ if (JSON.stringify(before) !== JSON.stringify(await readDocument())) throw new Error('Page changed during screenshot capture; capture again');+ if (typeof result.data !== 'string' || result.data.length > 30 * 1024 * 1024) throw new Error('Screenshot is empty or exceeds the transfer limit');+ const png = Buffer.from(result.data, 'base64');+ if (png.length < 24 || !png.subarray(0, 8).equals(Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]))) throw new Error('Firefox returned an invalid screenshot');+ const imageWidth = png.readUInt32BE(16), imageHeight = png.readUInt32BE(20);+ if (Math.abs(imageWidth - width * dpr) > 1 || Math.abs(imageHeight - height * dpr) > 1) {+ throw new Error('Firefox could not capture the whole document; use /screenshot for the visible area');+ }+ return { dataUrl: `data:image/png;base64,${result.data}`, captureBounds: { x: 0, y: 0, width, height } };+ } finally {+ await this.call(match, '(el, token) => { if (el.getAttribute("data-webbrain-bidi") === token) el.removeAttribute("data-webbrain-bidi"); }', [{ type: 'string', value: token }]).catch(() => {});+ }+ }call(match, functionDeclaration, args = []) {return this.send('script.callFunction', { target: { context: match.context }, functionDeclaration,arguments: [{ sharedId: match.node.sharedId }, ...args], awaitPromise: true });diff --git a/package.json b/package.jsonindex cc3cb6ee2..4473ac773 100644--- a/package.json+++ b/package.json@@ -51,7 +51,7 @@"test:social-contract:dom": "node test/social-publish-contract-dom.mjs","test:runtime-lifecycle": "node --test test/agent-lifecycle.mjs test/runtime-lifecycle.mjs test/network-lifecycle.mjs test/text-write-cancellation.mjs && node test/ui-content-lifecycle.mjs","test:rich-text-dom": "node test/ui-content-lifecycle.mjs --dom",- "test:firefox-bidi": "node --test test/firefox-bidi.mjs",+ "test:firefox-bidi": "node --test test/firefox-bidi.mjs test/firefox-full-page-screenshot.mjs","test:firefox-bidi:e2e": "node test/firefox-bidi-e2e.mjs","test:systemone": "node --test test/systemone.mjs","test:systemone:ui": "node test/systemone-ui.mjs",diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.jsindex a998e342c..3c07b7d77 100644--- a/src/firefox/src/agent/agent.js+++ b/src/firefox/src/agent/agent.js@@ -13597,6 +13597,39 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d}}+ async captureFullPageScreenshotForUser(tabId) {+ if (!tabId) return { ok: false, error: 'No tab ID' };+ try {+ return await this._withIndicatorsHidden(tabId, async () => {+ let before;+ const capture = await firefoxBidi.captureFullPage(tabId, async () => {+ before = await this.captureScreenshotRedactionSnapshotForUser(tabId, { coordinateSpace: 'page' });+ });+ if (!capture?.dataUrl) return { ok: false, error: 'Full-page screenshot returned no image data' };+ const after = await this.captureScreenshotRedactionSnapshotForUser(tabId, { coordinateSpace: 'page' });+ const stableSnapshot = before?.ok === true && after?.ok === true+ && JSON.stringify(before.snapshot) === JSON.stringify(after.snapshot)+ && before.snapshot.viewport.width === capture.captureBounds?.width+ && before.snapshot.viewport.height === capture.captureBounds?.height;+ // Retain the local preview/save path if redaction cannot be prepared,+ // but never stage an unscanned or moving document for a model request.+ const redactionUnavailable = this.screenshotRedaction === true && !stableSnapshot;+ return {+ ok: true,+ dataUrl: capture.dataUrl,+ captureBounds: capture.captureBounds,+ ...(stableSnapshot ? { redactionSnapshotReady: true, redactionSnapshot: before.snapshot } : {}),+ ...(redactionUnavailable ? {+ redactionUnavailable: true,+ warning: 'Screenshot redaction could not verify a stable page. You can save this image locally, but it cannot be attached to a message. Capture again after the page settles.',+ } : {}),+ };+ });+ } catch (error) {+ return { ok: false, error: error?.message || String(error) };+ }+ }+async captureViewportScreenshotForUser(tabId) {if (!tabId) return { ok: false, error: 'No tab ID' };try {diff --git a/src/firefox/src/agent/tools.js b/src/firefox/src/agent/tools.jsindex 074bee3c4..6b078bac3 100644--- a/src/firefox/src/agent/tools.js+++ b/src/firefox/src/agent/tools.js@@ -1808,7 +1808,7 @@ RULES:10. For loop tasks, keep using tools in this run; never say "I'll continue" unless you are actually making more tool calls.11. You cannot schedule, sleep, set timers, or check back later in compact mode. If something must wait for an external event, call done({summary:"...", outcome:"partial"}) with the current state and ask the user to re-invoke you.12. When the task is complete, call done({summary:"...", outcome:"success"}). Verify success first.-13. Call \`inspect_viewport\` when rendered pixels matter. Mention \`/screenshot\` only when the user explicitly wants to capture, save, or attach a page image; never require it just so the agent can see.+13. Call \`inspect_viewport\` when rendered pixels matter. Mention \`/screenshot\` only when the user explicitly wants to capture, save, or attach a page image; never require it just so the agent can see. Full-page image requests can use \`/screenshot --full-page\` with Firefox trusted automation enabled and the local companion installed.14. Recording is not supported in the Firefox build. Do not call or invent recording tools.15. CAPTCHA: stop and ask the user to complete verification manually. Never dismiss, close, or resubmit the challenge. Report blocked work with done({outcome:"partial"}).16. Before filling an external email/message/post composer, formulate the exact recipient, subject, and body. For more than a one-line body, save the complete text as \`[pending draft]\` with scratchpad_write first so it can be recovered if the UI fails; never mark it sent until verified.@@ -1869,7 +1869,7 @@ ${BROWSER_TAB_LIMITATION_ASK}CHAT IMAGES:- When the answer depends on appearance, an advertisement, an image/canvas/chart, visual layout, or visually rendered text that page reads miss, call \`inspect_viewport\` yourself. It is read-only and works in Ask mode; never ask the user to type \`/screenshot\` merely so you can see the page.-- If the user explicitly wants to capture, save, or attach a page image in the chat UI, tell them to type \`/screenshot\` for the visible viewport. The captured image is staged for their next message.+- If the user explicitly wants to capture, save, or attach a page image in the chat UI, tell them to type \`/screenshot\` for the visible viewport. The captured image is staged for their next message. Full-page image requests can use \`/screenshot --full-page\` with Firefox trusted automation enabled and the local companion installed.RECORDING:- Recording is not supported in the Firefox build. Do not call or invent recording tools.@@ -1972,7 +1972,7 @@ ${BROWSER_TAB_LIMITATION}CHAT IMAGES:- Call \`inspect_viewport\` yourself when appearance, an ad, image/canvas/chart, visual layout, or rendered pixels matter. Do not ask the user for \`/screenshot\` just to give the agent vision.-- Reserve \`/screenshot\` for an explicit user request to capture, save, or attach a page image; the slash command stages the capture for their next message.+- Reserve \`/screenshot\` for an explicit user request to capture, save, or attach a page image; the slash command stages the capture for their next message. Full-page image requests can use \`/screenshot --full-page\` with Firefox trusted automation enabled and the local companion installed.RECORDING:- Recording is not supported in the Firefox build. Do not call or invent recording tools.@@ -2200,7 +2200,7 @@ ${BROWSER_TAB_LIMITATION}- done({summary, outcome}): signal completion; use outcome:"success" only after verifying success.CHAT IMAGES:-- Call \`inspect_viewport\` yourself when appearance, an ad, image/canvas/chart, visual layout, or rendered pixels matter. Do not ask the user for \`/screenshot\` just to give the agent vision; mention it only when they explicitly want to capture, save, or attach a page image. The slash command stages it for their next message.+- Call \`inspect_viewport\` yourself when appearance, an ad, image/canvas/chart, visual layout, or rendered pixels matter. Do not ask the user for \`/screenshot\` just to give the agent vision; mention it only when they explicitly want to capture, save, or attach a page image. The slash command stages it for their next message. Full-page image requests can use \`/screenshot --full-page\` with Firefox trusted automation enabled and the local companion installed.RECORDING:- Recording is not supported in the Firefox build. Do not call or invent recording tools.diff --git a/src/firefox/src/background.js b/src/firefox/src/background.jsindex 684439d62..48f1901ec 100644--- a/src/firefox/src/background.js+++ b/src/firefox/src/background.js@@ -3575,6 +3575,10 @@ async function handleMessage(msg, sender) {case 'get_recording_state':return { ok: true, state: { recording: false, supported: false } };+ case 'capture_full_page_screenshot': {+ const tabId = msg.tabId || sender.tab?.id;+ return await agent.captureFullPageScreenshotForUser(tabId);+ }case 'capture_viewport_screenshot': {const tabId = msg.tabId || sender.tab?.id;return await agent.captureViewportScreenshotForUser(tabId);diff --git a/src/firefox/src/bidi/client.js b/src/firefox/src/bidi/client.jsindex 02f077ddf..f8faa7203 100644--- a/src/firefox/src/bidi/client.js+++ b/src/firefox/src/bidi/client.js@@ -1,12 +1,32 @@/** Optional native-messaging transport. No raw BiDi commands are exposed to pages. */export class FirefoxBidiClient {- constructor(api) { this.apiOverride = api; this.pending = new Map(); this.runs = new Map(); this.sequence = 0; }+ constructor(api) { this.apiOverride = api; this.pending = new Map(); this.runs = new Map(); this.captures = new Map(); this.sequence = 0; this.connectionEpoch = 0; }get api() { return this.apiOverride || globalThis.browser; }request(command, args = {}) {if (!this.port) {this.port = this.api.runtime.connectNative('one.webbrain.bidi');this.port.onMessage.addListener(message => {const entry = this.pending.get(message.id); if (!entry) return;+ if (typeof message.chunk === 'string') {+ entry.chunks ||= [];+ entry.chunkLength = (entry.chunkLength || 0) + message.chunk.length;+ if (message.index !== entry.chunks.length || entry.chunkLength > 32 * 1024 * 1024) {+ this.pending.delete(message.id);+ entry.reject(new Error('Invalid screenshot transfer; capture again'));+ return;+ }+ entry.chunks.push(message.chunk);+ if (!message.last) return;+ try {+ const parsed = JSON.parse(entry.chunks.join(''));+ if (parsed?.id !== message.id) throw new Error('Invalid screenshot reply');+ message = parsed;+ } catch (error) {+ this.pending.delete(message.id);+ entry.reject(error);+ return;+ }+ }this.pending.delete(message.id);if (message.error) {const error = new Error(message.error);@@ -20,6 +40,7 @@ export class FirefoxBidiClient {this.port.onDisconnect.addListener(() => {if (this.port !== port) return;this.port = null; this.connection = null;+ this.connectionEpoch++;for (const entry of this.pending.values()) entry.reject(new Error('Firefox companion disconnected; do not retry an uncertain action'));this.pending.clear();for (const owner of this.runs.values()) owner.disconnected = true;@@ -76,12 +97,43 @@ export class FirefoxBidiClient {if (this.port) void this.request('closeRun', { runId: owner.runId }).catch(() => {});}disconnect() {+ this.connectionEpoch++;// Retain failed owners until their task ends; never switch a live task to synthetic input.for (const owner of this.runs.values()) owner.disconnected = true;const port = this.port; this.port = null; this.connection = null;for (const entry of this.pending.values()) entry.reject(new Error('Firefox companion disconnected'));this.pending.clear(); port?.disconnect();}+ async captureFullPage(tabId, beforeCapture = async () => {}) {+ const epoch = this.connectionEpoch;+ const settings = await this.api.storage.local.get('firefoxBidiEnabled');+ if (!settings.firefoxBidiEnabled) throw new Error('Full-page screenshots require Firefox trusted automation. Install the local companion and enable it in Settings β General β Advanced.');+ if (this.captures.has(tabId)) throw new Error('A full-page screenshot is already in progress for this tab');+ const owner = {};+ this.captures.set(tabId, owner);+ const assertCurrent = () => {+ if (this.connectionEpoch !== epoch || this.captures.get(tabId) !== owner) throw new Error('Firefox connection changed during screenshot capture; capture again');+ };+ try {+ assertCurrent();+ await this.connect();+ assertCurrent();+ const tab = await this.api.tabs.get(tabId);+ if (!/^https?:\/\//.test(tab.url || '')) throw new Error('Full-page screenshots require a regular web page');+ const [binding] = await this.api.tabs.executeScript(tabId, { frameId: 0, file: '/src/bidi/bind.js' });+ assertCurrent();+ if (binding?.url !== tab.url) throw new Error('Page changed before screenshot capture; capture again');+ await beforeCapture();+ assertCurrent();+ const result = await this.request('captureFullPage', binding);+ assertCurrent();+ if ((await this.api.tabs.get(tabId)).url !== binding.url) throw new Error('Page changed during screenshot capture; capture again');+ assertCurrent();+ return result;+ } finally {+ if (this.captures.get(tabId) === owner) this.captures.delete(tabId);+ }+ }async perform(tabId, action, payload) {const owner = this.runs.get(tabId);if (!owner || owner.signal?.aborted || owner.disconnected) throw new Error('No connected Firefox BiDi run; restart the task after reconnecting');diff --git a/src/firefox/src/ui/screenshot-download.js b/src/firefox/src/ui/screenshot-download.jsnew file mode 100644index 000000000..85f9c3932--- /dev/null+++ b/src/firefox/src/ui/screenshot-download.js@@ -0,0 +1,27 @@+// Firefox rejects data: URLs in downloads.download. Use an extension-owned+// blob URL and retain it until the download completes (or fails).+export async function saveScreenshot(dataUrl, filename, { saveAs = true, api = browser } = {}) {+ if (!/^data:image\/png;base64,/i.test(dataUrl)) throw new Error('Invalid screenshot image');+ const url = URL.createObjectURL(await (await fetch(dataUrl)).blob());+ let downloadId, timer;+ const release = () => {+ clearTimeout(timer);+ api.downloads.onChanged.removeListener(onChanged);+ URL.revokeObjectURL(url);+ };+ const onChanged = delta => {+ if (delta.id === downloadId && ['complete', 'interrupted'].includes(delta.state?.current)) release();+ };+ api.downloads.onChanged.addListener(onChanged);+ try {+ downloadId = await api.downloads.download({ url, filename, saveAs, conflictAction: 'uniquify' });+ timer = setTimeout(release, 60_000);+ // A small local image can finish before download() resolves.+ const [download] = await api.downloads.search({ id: downloadId });+ if (['complete', 'interrupted'].includes(download?.state)) release();+ return downloadId;+ } catch (error) {+ release();+ throw error;+ }+}diff --git a/src/firefox/src/ui/sidepanel.js b/src/firefox/src/ui/sidepanel.jsindex 4b3ae47c0..3edd81c37 100644--- a/src/firefox/src/ui/sidepanel.js+++ b/src/firefox/src/ui/sidepanel.js@@ -1,3 +1,4 @@+import { saveScreenshot } from './screenshot-download.js';/*** WebBrain Side Panel β Chat UI logic.* Default: compact history in chat plus the live label; click for status-only mode.@@ -704,12 +705,12 @@ const SLASH_COMMANDS = [{ value: '/print', usage: '/print', descriptionKey: 'sp.slash.print', action: 'print' },{value: '/screenshot',- usage: '/screenshot',+ usage: '/screenshot [--full-page]',descriptionKey: 'sp.slash.screenshot',action: 'viewport',outOfBand: true,options: [- { value: '--full-page', descriptionKey: 'sp.slash.full_page_screenshot', action: 'full-page', unsupported: true, unsupportedUsage: '/screenshot --full-page', disallowPayload: true },+ { value: '--full-page', descriptionKey: 'sp.slash.full_page_screenshot', action: 'full-page', outOfBand: false, disallowPayload: true },],},{@@ -1022,7 +1023,17 @@ function isPlainScreenshotRequest(text) {|| /^(?:lutfen )?(?:bu |mevcut |aktif )?(?:sekmenin|sayfanin|ekranin) ekran goruntusunu (?:al|cek|goster|at)$/.test(s);}+function isPlainFullPageScreenshotRequest(text) {+ const s = normalizeScreenshotRequestText(text);+ if (!s || s.startsWith('/')) return false;+ return /^(?:please |pls )?(?:(?:full|whole|entire|complete) page|fullpage|long) (?:screenshot|screen ?shot)(?: (?:please|pls))?$/.test(s)+ || /^(?:please |pls |can you |could you |would you )?(?:take|capture|grab|show|get) (?:a |the |this )?(?:(?:full|whole|entire|complete) page|fullpage|long) (?:screenshot|screen ?shot)(?: (?:of|for) (?:the |this |current )?(?:page|tab|screen|window))?$/.test(s)+ || /^(?:lutfen )?(?:tam sayfa|butun sayfa|tum sayfa|uzun) ekran goruntusu(?: (?:al|cek|goster|at))?$/.test(s)+ || /^(?:lutfen )?(?:bu |mevcut |aktif )?(?:sayfanin|sekmenin) (?:tam|butun|tum) ekran goruntusunu (?:al|cek|goster|at)$/.test(s);+}+function normalizeScreenshotCommandText(text) {+ if (isPlainFullPageScreenshotRequest(text)) return '/screenshot --full-page';if (isPlainScreenshotRequest(text)) return '/screenshot';return text;}@@ -7827,15 +7838,10 @@ function bindScreenshotSaveButton(btn) {btn.setAttribute('aria-busy', 'true');if (label) label.textContent = t('sp.screenshot.saving');try {- await browser.downloads.download({- url: dataUrl,- filename: btn.dataset.filename || screenshotDownloadFilename(- '',- result?.querySelector('.screenshot-result-image-full-page') != null,- ),- saveAs: true,- conflictAction: 'uniquify',- });+ await saveScreenshot(dataUrl, btn.dataset.filename || screenshotDownloadFilename(+ '',+ result?.querySelector('.screenshot-result-image-full-page') != null,+ ));} catch (error) {const message = error?.message || String(error);// Closing the native Save As dialog is an ordinary user choice.@@ -8227,6 +8233,44 @@ async function parseSlashCommands(text, tabId = currentTabId, options = {}) {return '';}+ if (command.value === '/screenshot' && action === 'full-page') {+ try {+ const pageUrl = tabId == null+ ? ''+ : await browser.tabs.get(tabId).then(tab => tab?.url || '').catch(() => '');+ const res = await sendToBackground('capture_full_page_screenshot', { tabId });+ if (currentTabId !== tabId) return '';+ if (!res?.ok || !res.dataUrl) {+ addPersistentSlashMessage(systemHtml(tSystemHtml('sp.screenshot.error', { msg: res?.error || 'unknown error' })));+ return '';+ }+ // An enabled privacy pass that could not prepare this capture makes it+ // undeliverable: _applyAttachments rejects a full-page screenshot with no+ // capture-time snapshot. Keep the preview and save button, but do not+ // stage a chip the user would have to remove before sending anything.+ const stagedAttachment = res.redactionUnavailable === true+ ? null+ : await stageScreenshotAttachment(tabId, res.dataUrl, {+ fullPage: true,+ pageUrl,+ captureBounds: res.captureBounds,+ redactionSnapshotReady: res.redactionSnapshotReady === true,+ redactionSnapshot: res.redactionSnapshot,+ });+ if (currentTabId !== tabId) return '';+ addScreenshotResultMessage(res.dataUrl, {+ fullPage: true,+ warning: res.warning,+ pageUrl,+ stagedAttachment,+ });+ } catch (e) {+ if (currentTabId !== tabId) return '';+ addPersistentSlashMessage(systemHtml(tSystemHtml('sp.screenshot.error', { msg: e.message })));+ }+ return '';+ }+if (command.value === '/import' && action === 'file') {requestConfigurationFile(tabId);return '';diff --git a/test/firefox-bidi-e2e.mjs b/test/firefox-bidi-e2e.mjsindex a87264ace..f6686aed1 100644--- a/test/firefox-bidi-e2e.mjs+++ b/test/firefox-bidi-e2e.mjs@@ -6,8 +6,24 @@ import { join } from 'node:path';import { createServer } from 'node:http';import { fileURLToPath } from 'node:url';import { BidiSession } from '../firefox-companion/session.mjs';+import { nativeReplyMessages } from '../firefox-companion/native-messages.mjs';const profile = await mkdtemp(join(tmpdir(), 'webbrain-firefox-test-'));-const server = createServer((_req, res) => {+await writeFile(join(profile, 'user.js'), `user_pref("browser.download.folderList", 2); user_pref("browser.download.dir", ${JSON.stringify(profile)}); user_pref("browser.download.useDownloadDir", true);`);+if (process.env.FIREFOX_TEST_DPR) await writeFile(join(profile, 'user.js'), `user_pref("layout.css.devPixelsPerPx", ${JSON.stringify(process.env.FIREFOX_TEST_DPR)});\n`, { flag: 'a' });+const server = createServer(async (_req, res) => {+ if (_req.url === '/native-capture' && _req.method === 'POST') {+ let body = '';+ for await (const chunk of _req) body += chunk;+ const message = JSON.parse(body);+ try {+ const result = message.command === 'connect' ? { connected: true }+ : message.command === 'captureFullPage' ? await session.captureFullPage(message.token, message.url)+ : (() => { throw new Error('Unexpected test command'); })();+ res.setHeader('Content-Type', 'application/json');+ res.end(JSON.stringify([...nativeReplyMessages({ id: message.id, result })]));+ } catch (error) { res.end(JSON.stringify([{ id: message.id, error: error.message }])); }+ return;+ }res.setHeader('Content-Type', 'text/html');res.end(`<button id="click">Click</button><input id="text"><input id="file" type="file"><script>window.events=[]; document.addEventListener('click',e=>events.push(['click',e.isTrusted]));@@ -65,7 +81,7 @@ try {const extensionOrigin = new URL(extensionContext.url).origin === 'null' ? extensionContext.url.split('/').slice(0, 3).join('/') : new URL(extensionContext.url).origin;const extensionEval = async expression => {const result = await session.send('script.evaluate', { target: { context: extensionContext.context }, expression, awaitPromise: true });- if (result.type === 'exception') throw new Error(result.exceptionDetails.text);+ if (result.type === 'exception') throw new Error(result.exceptionDetails.text.replace(/data:image[^\s)]+/g, '[image omitted]').slice(0, 600));return result.result;};const tabId = (await extensionEval(`browser.tabs.query({}).then(tabs => tabs.find(tab => tab.url === ${JSON.stringify(url)}).id)`)).value;@@ -106,10 +122,61 @@ try {assert.equal(JSON.parse(belowFoldPrepared.value).bidiPrepared, true);assert.equal((await session.perform(runId, 'click', { token: belowFoldToken, url })).success, true);assert.equal((await evaluate('window.belowFoldTrusted')).value, true);+ // Capture the complete document from a scrolled viewport. A noisy canvas+ // makes the PNG exceed the native messaging limit so the chunk path runs.+ await evaluate(`(() => {+ const canvas=document.createElement('canvas'); canvas.id='capture-noise'; canvas.width=1000; canvas.height=450;+ const ctx=canvas.getContext('2d'), pixels=ctx.createImageData(canvas.width,canvas.height);+ let seed=123; for(let i=0;i<pixels.data.length;i+=4){seed=(Math.imul(seed,1664525)+1013904223)>>>0;pixels.data[i]=seed&255;pixels.data[i+1]=(seed>>>8)&255;pixels.data[i+2]=(seed>>>16)&255;pixels.data[i+3]=255;}+ ctx.putImageData(pixels,0,0); document.body.append(canvas);+ document.body.insertAdjacentHTML('beforeend','<input id="capture-private" value="private fixture value" style="position:absolute;top:2750px;left:20px;width:200px"><div id="capture-footer" style="position:absolute;top:2900px;left:20px;width:200px;height:50px;background:rgb(17,119,51)">FULL PAGE FOOTER</div>');+ })()`);+ const fullPage = JSON.parse((await extensionEval(`(async () => {+ const {firefoxBidi} = await import(${JSON.stringify(extensionOrigin + '/src/bidi/client.js')});+ const {Agent} = await import(${JSON.stringify(extensionOrigin + '/src/agent/agent.js')});+ let transferredChunks=0;+ firefoxBidi.apiOverride = { tabs:browser.tabs, storage:browser.storage, runtime:{ connectNative() {+ let receive;+ return { onMessage:{addListener(fn){receive=fn}}, onDisconnect:{addListener(){}}, disconnect(){},+ postMessage(message) { fetch(${JSON.stringify(url + 'native-capture')},{method:'POST',body:JSON.stringify(message)}).then(r=>r.json()).then(messages=>{transferredChunks+=messages.filter(m=>typeof m.chunk==='string').length;messages.forEach(receive)}); }+ };+ }}};+ await browser.storage.local.set({firefoxBidiEnabled:true});+ const agent=Object.create(Agent.prototype); agent.screenshotRedaction=true;+ const capture=await agent.captureFullPageScreenshotForUser(${tabId});+ if(!capture.ok) throw new Error(capture.error);+ const image=await createImageBitmap(await (await fetch(capture.dataUrl)).blob());+ const canvas=new OffscreenCanvas(image.width,image.height); const ctx=canvas.getContext('2d');ctx.drawImage(image,0,0);+ const scale=image.width/capture.captureBounds.width;+ const footerPixel=[...ctx.getImageData(Math.round(30*scale),Math.round(2940*scale),1,1).data];+ const redacted=await agent._redactScreenshotDataUrl(${tabId},capture.dataUrl,{coordinateSpace:'page',redactionSnapshot:capture.redactionSnapshot,capturedCssBounds:capture.captureBounds});+ const {saveScreenshot}=await import(${JSON.stringify(extensionOrigin + '/src/ui/screenshot-download.js')});+ const downloadId=await saveScreenshot(capture.dataUrl,'webbrain-bidi-test.png',{saveAs:false});+ let download; for(let attempt=0;attempt<100;attempt++){[download]=await browser.downloads.search({id:downloadId});if(download.state==='complete')break;await new Promise(resolve=>setTimeout(resolve,20));}+ if(download.state!=='complete')throw new Error('Screenshot download did not complete');+ await browser.downloads.removeFile(downloadId); await browser.downloads.erase({id:downloadId});+ const result={...capture,width:image.width,height:image.height,footerPixel,transferredChunks,redacted:redacted!==capture.dataUrl,saved:download.filename.endsWith('webbrain-bidi-test.png')};+ image.close(); firefoxBidi.disconnect(); firefoxBidi.apiOverride=undefined;+ await browser.storage.local.set({firefoxBidiEnabled:false});+ return JSON.stringify(result);+ })()`)).value);+ assert.equal(fullPage.redactionSnapshotReady, true, 'full-page privacy scan should be available');+ assert.equal(fullPage.redactionUnavailable, undefined);+ assert.ok(fullPage.redactionSnapshot.regions.some(region => region.rect.y > 2700), 'privacy scan covers fields below the viewport');+ assert.ok(fullPage.height >= 3000);+ if (process.env.FIREFOX_TEST_DPR) assert.ok(Math.abs(fullPage.width / fullPage.captureBounds.width - Number(process.env.FIREFOX_TEST_DPR)) < 0.01, 'full-page bounds use CSS pixels at high DPI');+ assert.ok(fullPage.transferredChunks > 1, 'large PNG uses chunked native replies');+ assert.deepEqual(fullPage.footerPixel, [17,119,51,255], 'image contains the document footer');+ assert.equal(fullPage.redacted, true, 'model-facing image redacts sensitive regions');+ assert.equal(fullPage.saved, true, 'captured PNG can be saved through Firefox downloads');+ assert.equal(session.runs.has(runId), true, 'standalone capture preserves active automation');+ console.log(`PASS: full-page PNG ${fullPage.width}x${fullPage.height}, ${fullPage.transferredChunks} native chunks, footer pixels, below-fold privacy and local save`);+ await writeFile('/tmp/webbrain3-firefox-full-page.png', Buffer.from(fullPage.dataUrl.split(',')[1], 'base64'));+ await evaluate(`document.querySelector('#capture-noise').remove();document.querySelector('#capture-private').remove();document.querySelector('#capture-footer').remove();`);await evaluate(`document.activeElement.blur()`);assert.equal((await session.perform(runId, 'key', { ...await mark('body'), key: 'Escape' })).success, true);const shadowToken = crypto.randomUUID();- await evaluate(`const host=document.createElement('div'); host.id='shadow-host'; host.style.cssText='position:fixed;left:20px;top:300px'; host.attachShadow({mode:'open'}).innerHTML='<button id="shadow-button">Shadow button</button>'; host.shadowRoot.querySelector('button').setAttribute('data-webbrain-bidi', ${JSON.stringify(shadowToken)}); host.shadowRoot.querySelector('button').onclick=e=>window.shadowTrusted=e.isTrusted; document.body.append(host);`);+ await evaluate(`const host=document.createElement('div'); host.id='shadow-host'; host.style.cssText='position:fixed;left:20px;top:100px'; host.attachShadow({mode:'open'}).innerHTML='<button id="shadow-button">Shadow button</button>'; host.shadowRoot.querySelector('button').setAttribute('data-webbrain-bidi', ${JSON.stringify(shadowToken)}); host.shadowRoot.querySelector('button').onclick=e=>window.shadowTrusted=e.isTrusted; document.body.append(host);`);const shadowNode = await session.send('script.evaluate', { target: { context }, expression: "document.querySelector('#shadow-host').shadowRoot.querySelector('#shadow-button')", awaitPromise: true });const locate = session.locate;session.locate = async () => ({ context, node: { sharedId: shadowNode.result.sharedId } });@@ -206,7 +273,7 @@ try {await new Promise(resolve => setTimeout(resolve, 100));assert.equal((await evaluate('document.querySelector("#text").value')).value, afterStop);await assert.rejects(session.perform(resumed, 'click', await mark('#click')), /stopped|disconnected/);- console.log(`PASS: Firefox connection, dialogs, uploads, literal newlines, focus guards, interrupted typing, deferred blank-tab binding, and packaged extension integration; checkable state event woke in ${delayedChecked._checkableObservationMs}ms (80ms cap)`);+ console.log(`PASS: Firefox full-page screenshots, chunked PNG transfer, below-fold privacy, local PNG save, connection, dialogs, uploads, literal newlines, focus guards, interrupted typing, deferred blank-tab binding, and packaged extension integration; checkable state event woke in ${delayedChecked._checkableObservationMs}ms (80ms cap)`);} finally {await session?.close().catch(() => {});firefox.kill('SIGTERM');diff --git a/test/firefox-full-page-screenshot.mjs b/test/firefox-full-page-screenshot.mjsnew file mode 100644index 000000000..20b98110f--- /dev/null+++ b/test/firefox-full-page-screenshot.mjs@@ -0,0 +1,171 @@+import { test } from 'node:test';+import assert from 'node:assert/strict';+import { readFileSync } from 'node:fs';+import { BidiSession } from '../firefox-companion/session.mjs';+import { nativeReplyMessages } from '../firefox-companion/native-messages.mjs';+import { FirefoxBidiClient } from '../src/firefox/src/bidi/client.js';+import { saveScreenshot } from '../src/firefox/src/ui/screenshot-download.js';++function pngHeader(width, height) {+ const png = Buffer.alloc(24);+ Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]).copy(png);+ png.writeUInt32BE(width, 16); png.writeUInt32BE(height, 20);+ return png.toString('base64');+}+function captureSession() {+ const session = new BidiSession(), calls = [];+ const document = { url: 'https://example.com/', timeOrigin: 123, width: 800, height: 3000, dpr: 2, scrollX: 0, scrollY: 700 };+ session.locate = async () => ({ context: 'selected', node: { sharedId: 'document' } });+ session.call = async (_match, fn) => ({ result: { value: fn.includes('JSON.stringify') ? JSON.stringify(document) : true } });+ session.send = async (method, params) => { calls.push({ method, params }); return { data: pngHeader(1600, 6000) }; };+ return { session, calls, document };+}++test('full-page capture binds the document, preserves CSS bounds and leaves the active run alone', async () => {+ const { session, calls } = captureSession();+ const owner = { context: 'selected', navigation: false };+ session.runs.set('existing', owner);+ const result = await session.captureFullPage(crypto.randomUUID(), 'https://example.com/');+ assert.deepEqual(calls, [{ method: 'browsingContext.captureScreenshot', params: { context: 'selected', origin: 'document', format: { type: 'image/png' } } }]);+ assert.deepEqual(result.captureBounds, { x: 0, y: 0, width: 800, height: 3000 });+ assert.equal(session.runs.get('existing'), owner);+});+test('navigation or layout changes during capture discard the image', async () => {+ for (const changed of [{ timeOrigin: 456 }, { width: 900 }, { scrollY: 0 }]) {+ const { session, document } = captureSession();+ session.send = async () => { Object.assign(document, changed); return { data: pngHeader(1600, 6000) }; };+ await assert.rejects(session.captureFullPage(crypto.randomUUID(), document.url), /changed/);+ }+});+test('oversized pages and truncated native images fail explicitly', async () => {+ const { session, calls, document } = captureSession();+ document.height = 1e9;+ await assert.rejects(session.captureFullPage(crypto.randomUUID(), document.url), /too large/);+ assert.equal(calls.length, 0);+ document.height = 3000;+ session.send = async () => ({ data: pngHeader(1600, 900) });+ await assert.rejects(session.captureFullPage(crypto.randomUUID(), document.url), /whole document/);+});+test('a stale document binding never triggers native capture', async () => {+ const { session, calls } = captureSession();+ session.call = async () => ({ result: { type: 'null' } });+ await assert.rejects(session.captureFullPage(crypto.randomUUID(), 'https://example.com/'), /document changed/);+ assert.equal(calls.length, 0);+});++function clientHarness({ enabled = true, reply } = {}) {+ let receive;+ const sent = [];+ const api = {+ storage: { local: { get: async () => ({ firefoxBidiEnabled: enabled }) } },+ tabs: { get: async () => ({ url: 'https://example.com/' }), executeScript: async () => [{ token: crypto.randomUUID(), url: 'https://example.com/' }] },+ runtime: { connectNative: () => ({+ onMessage: { addListener: fn => { receive = fn; } }, onDisconnect: { addListener() {} },+ postMessage: request => {+ sent.push(request);+ const result = reply?.(request) || {};+ queueMicrotask(() => { for (const message of nativeReplyMessages({ id: request.id, result })) receive(message); });+ }, disconnect() {},+ }) },+ };+ return { client: new FirefoxBidiClient(api), api, sent, receive: message => receive(message) };+}+test('native replies larger than 1 MiB are reassembled without oversized messages', async () => {+ const dataUrl = 'data:image/png;base64,' + 'a'.repeat(2 * 1024 * 1024);+ for (const message of nativeReplyMessages({ id: 1, result: { dataUrl } })) {+ assert.ok(Buffer.byteLength(JSON.stringify(message)) < 1024 * 1024);+ }+ const { client } = clientHarness({ reply: () => ({ dataUrl }) });+ assert.deepEqual(await client.request('captureFullPage'), { dataUrl });+});+test('native framing remains below the cap for escaped Unicode data', () => {+ const result = '\u0000π'.repeat(200000);+ const messages = [...nativeReplyMessages({ id: 5, result })];+ assert.ok(messages.length > 1);+ assert.ok(messages.every(message => Buffer.byteLength(JSON.stringify(message)) < 1024 * 1024));+ assert.equal(JSON.parse(messages.map(message => message.chunk).join('')).result, result);+});+test('out-of-order chunks reject and release the pending request', async () => {+ const { client, api } = clientHarness();+ api.runtime.connectNative = () => ({ onMessage: { addListener: fn => { api.deliver = fn; } }, onDisconnect: { addListener() {} }, postMessage() {} });+ const promise = client.request('captureFullPage');+ api.deliver({ id: 1, index: 1, chunk: 'bad', last: true });+ await assert.rejects(promise, /Invalid screenshot transfer/);+ assert.equal(client.pending.size, 0);+});+test('disabled BiDi fails before connecting or inspecting a tab', async () => {+ const { client, sent, api } = clientHarness({ enabled: false });+ api.tabs.get = () => assert.fail('must not inspect the tab');+ await assert.rejects(client.captureFullPage(7), /require Firefox trusted automation/);+ assert.equal(sent.length, 0);+});+test('standalone capture cannot replace or end an active automation run', async () => {+ const { client, sent } = clientHarness({ reply: () => ({ dataUrl: 'data:image/png;base64,example' }) });+ const owner = { runId: 'active', bound: true };+ client.runs.set(7, owner);+ let prepared = false;+ await client.captureFullPage(7, async () => { prepared = true; });+ assert.equal(prepared, true);+ assert.equal(client.runs.get(7), owner);+ assert.deepEqual(sent.map(item => item.command), ['connect', 'captureFullPage']);+ assert.equal(client.captures.size, 0);+});+test('disabling the connection during privacy preparation prevents screenshot dispatch', async () => {+ const { client, sent } = clientHarness();+ await assert.rejects(client.captureFullPage(7, async () => client.disconnect()), /connection changed/);+ assert.deepEqual(sent.map(item => item.command), ['connect']);+ assert.equal(client.captures.size, 0);+});+test('Firefox saving uses a live blob URL and releases it on completion', async () => {+ let listener, savedUrl;+ const api = { downloads: {+ onChanged: { addListener(fn) { listener = fn; }, removeListener(fn) { assert.equal(fn, listener); listener = null; } },+ download: async options => {+ savedUrl = options.url;+ assert.ok(savedUrl.startsWith('blob:'));+ assert.equal(options.saveAs, true);+ assert.equal(options.conflictAction, 'uniquify');+ assert.equal((await fetch(savedUrl)).status, 200);+ return 4;+ },+ search: async () => [{ state: 'in_progress' }],+ } };+ assert.equal(await saveScreenshot(`data:image/png;base64,${pngHeader(10, 10)}`, 'page.png', { api }), 4);+ assert.equal((await fetch(savedUrl)).status, 200);+ listener({ id: 4, state: { current: 'complete' } });+ assert.equal(listener, null);+ await assert.rejects(fetch(savedUrl));+});+test('cancelling Save as immediately releases its blob and download listener', async () => {+ let savedUrl, listener;+ const api = { downloads: {+ onChanged: { addListener(fn) { listener = fn; }, removeListener() { listener = null; } },+ download: async ({ url }) => { savedUrl = url; throw new Error('cancelled'); },+ } };+ await assert.rejects(saveScreenshot(`data:image/png;base64,${pngHeader(10, 10)}`, 'page.png', { api }), /cancelled/);+ assert.equal(listener, null);+ await assert.rejects(fetch(savedUrl));+});++// Exercise the production Agent method with deterministic capture/scan results.+const agentSource = readFileSync(new URL('../src/firefox/src/agent/agent.js', import.meta.url), 'utf8');+const captureMethod = agentSource.slice(agentSource.indexOf(' async captureFullPageScreenshotForUser('), agentSource.indexOf(' async captureViewportScreenshotForUser('));+test('full-page captures stage only stable, complete privacy geometry', async () => {+ const snapshot = { coordinateSpace: 'page', viewport: { width: 800, height: 3000 }, regions: [{ kind: 'input', rect: { x: 1, y: 2400, w: 50, h: 20 } }] };+ for (const after of [{ ok: true, snapshot }, { ok: false }, { ok: true, snapshot: { ...snapshot, regions: [] } }]) {+ const method = Function('firefoxBidi', `return ({${captureMethod}}).captureFullPageScreenshotForUser` )({+ captureFullPage: async (_tab, prepare) => { await prepare(); return { dataUrl: 'original', captureBounds: { x: 0, y: 0, width: 800, height: 3000 } }; },+ });+ let scans = 0;+ const result = await method.call({ screenshotRedaction: true, _withIndicatorsHidden: (_tab, fn) => fn(), captureScreenshotRedactionSnapshotForUser: async () => ++scans === 1 ? { ok: true, snapshot } : after }, 7);+ assert.equal(result.ok, true);+ assert.equal(result.dataUrl, 'original', 'local preview remains available');+ if (after.snapshot === snapshot) {+ assert.equal(result.redactionSnapshotReady, true);+ assert.equal(result.redactionUnavailable, undefined);+ } else {+ assert.equal(result.redactionUnavailable, true);+ assert.equal(result.redactionSnapshot, undefined);+ }+ }+});diff --git a/test/run.js b/test/run.jsindex 3091f3e96..76e8dd734 100644--- a/test/run.js+++ b/test/run.js@@ -29506,7 +29506,7 @@ test('getToolsForMode: retired tools are not model-callable', () => {assert.match(prompt, /\/record\b/, `[${label}] ${promptLabel} prompt should direct recording requests to the /record slash command`);assert.match(prompt, /\/record --full-screen\b/, `[${label}] ${promptLabel} prompt should direct screen recording requests to /record --full-screen`);} else {- assert.doesNotMatch(prompt, /\/screenshot --full-page\b/, `[${label}] ${promptLabel} prompt must not mention the Chrome-only full-page screenshot flag`);+ assert.match(prompt, /\/screenshot --full-page\b/, `[${label}] ${promptLabel} prompt should explain full-page capture with the Firefox companion`);assert.doesNotMatch(prompt, /\/record --full-screen\b/, `[${label}] ${promptLabel} prompt must not mention the Chrome-only full-screen recording flag`);}}@@ -44231,11 +44231,11 @@ test('canonical slash parser handles flags, values, casing, termination, and har}assert.equal(firefox.parseSlashInvocation('/record --full-screen --transcribe').unsupported, true, 'Firefox should reject canonical recording locally');- assert.equal(firefox.parseSlashInvocation('/screenshot --full-page').unsupported, true, 'Firefox should reject the unsupported canonical screenshot flag locally');+ assert.equal(firefox.parseSlashInvocation('/screenshot --full-page').action, 'full-page', 'Firefox should route full-page captures through the companion');assert.equal(firefox.SLASH_COMMANDS.find((command) => command.value === '/record').unsupported, true, 'Firefox recording should be hidden from discovery');const firefoxHelp = firefox.buildSlashCommandHelpHtml();assert.doesNotMatch(firefoxHelp, /\/record/, 'Firefox help should omit unsupported recording');- assert.doesNotMatch(firefoxHelp, /--full-page/, 'Firefox help should omit the unsupported full-page flag');+ assert.match(firefoxHelp, /--full-page/, 'Firefox help should expose the full-page flag');const firefoxEnglish = fs.readFileSync(path.join(ROOT, 'src/firefox/src/ui/locales/en.js'), 'utf8');assert.doesNotMatch(firefoxEnglish, /Stop an active recording/, 'Firefox help shortcuts should not advertise unsupported recording');@@ -44702,7 +44702,7 @@ test('slash autocomplete progressively suggests only available unused flags', ()assert.equal(firefox.getMatches(firefox.getContext('/scratchpad '))[0]?.kind, 'base-action', 'Firefox should mirror the base-action row');assert.equal(firefox.getMatches(firefox.getContext('/scratchpad --clear '))[0]?.kind, 'base-action', 'Firefox should mirror Enter after a selected flag');assert.equal(firefox.getMatches(firefox.getContext('/schedule --help '))[0]?.descriptionKey, 'sp.slash.help', 'Firefox should describe the completed help action accurately');- assert.deepEqual(optionMatches(firefox, firefox.getContext('/screenshot ')), ['--help'], 'Firefox should omit unsupported flags but still offer command help');+ assert.deepEqual(optionMatches(firefox, firefox.getContext('/screenshot ')), ['--full-page', '--help'], 'Firefox should offer full-page capture and command help');assert.deepEqual(optionMatches(firefox, firefox.getContext('/scratchpad --clear ')), [], 'Firefox should not suggest conflicting scratchpad actions');});@@ -50342,7 +50342,11 @@ test('sidepanel scopes async tab commands to the original tab', () => {assert.match(screenshotBody, /if \(currentTabId !== tabId \|\| !tab\?\.active\) return '';[\s\S]*?sendToBackground\('capture_viewport_screenshot', \{ tabId \}\);[\s\S]*?if \(currentTabId !== tabId\) return '';[\s\S]*?stageScreenshotAttachment\(tabId, res\.dataUrl, \{[\s\S]*?pageUrl: tab\.url,[\s\S]*?redactionSnapshotReady: res\.redactionSnapshotReady === true,[\s\S]*?modelRedactionReady: res\.modelRedactionReady === true,[\s\S]*?modelDataUrl: res\.modelDataUrl,[\s\S]*?addScreenshotResultMessage\(res\.dataUrl, \{ pageUrl: tab\.url, stagedAttachment \}\);/, `${label}: /screenshot should retain the background-created model copy, reject stale-tab completions, and keep the raw URL for local preview`);assert.doesNotMatch(screenshotBody, /Promise\.all|captureVisibleTab|capture_screenshot_redaction_snapshot/, `${label}: the side panel must not race viewport pixels against privacy geometry`);assert.match(panel, /function renderScreenshotResult\(dataUrl,[\s\S]*?screenshot-save-btn[\s\S]*?sp\.screenshot\.save_as/, `${label}: screenshot messages should render a visible Save As action`);- assert.match(panel, /function bindScreenshotSaveButton\(btn\)[\s\S]*?downloads\.download\(\{[\s\S]*?url: dataUrl,[\s\S]*?saveAs: true,[\s\S]*?conflictAction: 'uniquify'/, `${label}: screenshot Save As should use the browser Downloads API and native picker`);+ if (label === 'chrome') {+ assert.match(panel, /function bindScreenshotSaveButton\(btn\)[\s\S]*?downloads\.download\(\{[\s\S]*?url: dataUrl,[\s\S]*?saveAs: true,[\s\S]*?conflictAction: 'uniquify'/, `${label}: screenshot Save As should use the browser Downloads API and native picker`);+ } else {+ assert.match(panel, /saveScreenshot\(dataUrl, btn\.dataset\.filename/, 'Firefox Save As should use the blob-backed screenshot downloader');+ }assert.match(panel, /function rebindRestoredMessageControls\(\)[\s\S]*?rebindScreenshotSaveButtons\(\);/, `${label}: restored screenshot messages should regain their Save As behavior`);assert.match(style, /\.screenshot-save-btn \{[\s\S]*?cursor: pointer;[\s\S]*?\}[\s\S]*?\.screenshot-save-btn:hover,[\s\S]*?\.screenshot-save-btn:focus-visible/, `${label}: screenshot Save As should be styled for pointer and keyboard interaction`);assert.match(locale, /'sp\.screenshot\.save_as': 'Save asβ¦'/, `${label}: screenshot Save As should have an English label`);@@ -50350,19 +50354,12 @@ test('sidepanel scopes async tab commands to the original tab', () => {const fullPageIdx = panel.indexOf("if (command.value === '/screenshot' && action === 'full-page')");assert.match(panel, /function normalizeScreenshotRequestText\(text\) \{[\s\S]*?\.normalize\('NFKD'\)[\s\S]*?\.replace\(\/\[\\u0300-\\u036f\]\/g, ''\)[\s\S]*?\.replace\(\/\\u0131\/g, 'i'\)/, `${label}: plain screenshot request normalization should handle accented Turkish text`);assert.match(panel, /function isPlainScreenshotRequest\(text\) \{[\s\S]*?const s = normalizeScreenshotRequestText\(text\);[\s\S]*?s\.startsWith\('\/'\)[\s\S]*?ekran goruntusu[\s\S]*?ekran goruntusunu/, `${label}: plain screenshot request routing should cover English and Turkish screenshot-only requests`);- if (label === 'chrome') {- assert.notEqual(fullPageIdx, -1, `${label}: /screenshot --full-page parser missing`);- const fullPageBody = panel.slice(fullPageIdx, panel.indexOf("if (command.value === '/record'", fullPageIdx));- assert.match(fullPageBody, /tabs\.get\(tabId\)[\s\S]*?sendToBackground\('capture_full_page_screenshot', \{ tabId \}\);[\s\S]*?if \(currentTabId !== tabId\) return '';[\s\S]*?stageScreenshotAttachment\(tabId, res\.dataUrl, \{[\s\S]*?fullPage: true,[\s\S]*?pageUrl,[\s\S]*?captureBounds: res\.captureBounds,[\s\S]*?redactionSnapshotReady: res\.redactionSnapshotReady === true,[\s\S]*?redactionSnapshot: res\.redactionSnapshot,[\s\S]*?\}\);[\s\S]*?addScreenshotResultMessage\(res\.dataUrl, \{[\s\S]*?fullPage: true,[\s\S]*?warning: res\.warning,[\s\S]*?pageUrl,[\s\S]*?stagedAttachment,[\s\S]*?\}\);/, `${label}: /screenshot --full-page should stage capture-time privacy geometry and render only in the initiating tab with URL-aware Save As`);- assert.match(panel, /function renderScreenshotResult\(dataUrl,[\s\S]*?warningHtml = warning[\s\S]*?escapeHtml\(warning\)/, `${label}: fallback full-page images should display their escaped assembly warning`);- assert.match(panel, /function isPlainFullPageScreenshotRequest\(text\) \{[\s\S]*?full\|whole\|entire\|complete[\s\S]*?tam sayfa[\s\S]*?ekran goruntusu/, `${label}: plain full-page screenshot request routing should cover English and Turkish requests`);- assert.match(panel, /function normalizeScreenshotCommandText\(text\) \{[\s\S]*?isPlainFullPageScreenshotRequest\(text\)[\s\S]*?return '\/screenshot --full-page';[\s\S]*?isPlainScreenshotRequest\(text\)[\s\S]*?return '\/screenshot';/, `${label}: screenshot normalization should route full-page requests before viewport screenshots`);- } else {- assert.equal(fullPageIdx, -1, `${label}: /screenshot --full-page action should stay Chrome-only`);- assert.doesNotMatch(panel, /capture_full_page_screenshot/, `${label}: should not call the Chrome-only full-page screenshot route`);- assert.doesNotMatch(panel, /isPlainFullPageScreenshotRequest/, `${label}: should not normalize full-page screenshot text into an unsupported slash command`);- assert.match(panel, /function normalizeScreenshotCommandText\(text\) \{[\s\S]*?if \(isPlainScreenshotRequest\(text\)\) return '\/screenshot';[\s\S]*?return text;[\s\S]*?\}/, `${label}: screenshot normalization should only route viewport screenshots`);- }+ assert.notEqual(fullPageIdx, -1, `${label}: /screenshot --full-page parser missing`);+ const fullPageBody = panel.slice(fullPageIdx, panel.indexOf("if (command.value === '/record'", fullPageIdx));+ assert.match(fullPageBody, /tabs\.get\(tabId\)[\s\S]*?sendToBackground\('capture_full_page_screenshot', \{ tabId \}\);[\s\S]*?if \(currentTabId !== tabId\) return '';[\s\S]*?stageScreenshotAttachment\(tabId, res\.dataUrl, \{[\s\S]*?fullPage: true,[\s\S]*?pageUrl,[\s\S]*?captureBounds: res\.captureBounds,[\s\S]*?redactionSnapshotReady: res\.redactionSnapshotReady === true,[\s\S]*?redactionSnapshot: res\.redactionSnapshot,[\s\S]*?\}\);[\s\S]*?addScreenshotResultMessage\(res\.dataUrl, \{[\s\S]*?fullPage: true,[\s\S]*?warning: res\.warning,[\s\S]*?pageUrl,[\s\S]*?stagedAttachment,[\s\S]*?\}\);/, `${label}: /screenshot --full-page should stage capture-time privacy geometry and render only in the initiating tab with URL-aware Save As`);+ assert.match(panel, /function renderScreenshotResult\(dataUrl,[\s\S]*?warningHtml = warning[\s\S]*?escapeHtml\(warning\)/, `${label}: fallback full-page images should display their escaped assembly warning`);+ assert.match(panel, /function isPlainFullPageScreenshotRequest\(text\) \{[\s\S]*?full\|whole\|entire\|complete[\s\S]*?tam sayfa[\s\S]*?ekran goruntusu/, `${label}: plain full-page screenshot request routing should cover English and Turkish requests`);+ assert.match(panel, /function normalizeScreenshotCommandText\(text\) \{[\s\S]*?isPlainFullPageScreenshotRequest\(text\)[\s\S]*?return '\/screenshot --full-page';[\s\S]*?isPlainScreenshotRequest\(text\)[\s\S]*?return '\/screenshot';/, `${label}: screenshot normalization should route full-page requests before viewport screenshots`);const sendIdx = panel.search(/async function sendMessage\(extraChatParams(?: = \{\})?\)/);assert.notEqual(sendIdx, -1, `${label}: sendMessage missing`);const sendBody = panel.slice(sendIdx, panel.indexOf('let assistantEl = null;', sendIdx));@@ -50657,8 +50654,8 @@ test('sidepanel allows safe slash commands and queues normal messages while busyfor (const command of ['/schedule task', '/scratchpad --append note', '/scratchpad --clear', '/memory --add note', '/memory --forget id']) {assert.equal(slash.slashInvocationIsOutOfBand(slash.parseSlashInvocation(command)), false, `${label}: ${command} should stay gated while busy`);}+ assert.equal(slash.slashInvocationIsOutOfBand(slash.parseSlashInvocation('/screenshot --full-page')), false, `${label}: full-page capture should stay gated while busy`);if (label === 'chrome') {- assert.equal(slash.slashInvocationIsOutOfBand(slash.parseSlashInvocation('/screenshot --full-page')), false, `${label}: full-page capture should stay gated while busy`);assert.equal(slash.slashInvocationIsOutOfBand(slash.parseSlashInvocation('/record --full-screen')), false, `${label}: recording should stay gated while busy`);assert.equal(slash.slashInvocationIsOutOfBand(slash.parseSlashInvocation('/record --stop')), true, `${label}: recording Stop should stay available while busy`);}