Open-source AI browser agent for Chrome and Firefox (monorepo) π§
WebBrain Firefox Extension β Architecture
Version 45.0.4 Β· Manifest V2 Β· Background Page
How Firefox Differs from Chrome
Firefox uses Manifest V2 (background page, not service worker) and has no access to the Chrome DevTools Protocol (CDP). Starting with v3.6.x, the Firefox build has been brought to functional parity with Chrome for the accessibility-tree (AX) subsystem β the same tree builder, the same four AX tools (get_accessibility_tree, click_ax, type_ax, set_field), and the same ref_id registry. What Firefox still lacks:
- Optional trusted events β standard mode uses synthetic events. The experimental BiDi companion adds trusted click, hover, text, supported keys, file-input attachment, and native JavaScript-dialog handling. It requires a local helper and Firefox remote automation; other tools retain their existing implementations. This is a Firefox-specific transport, not a Chrome CDP adapter.
- No pixel-perfect / full-page screenshots β uses
browser.tabs.captureTab()instead of CDPPage.captureScreenshot; it can capture the run tab while that tab is inactive. Firefox has exposedtabs.captureTab()since Firefox 59, before WebBrain's current minimum, and the manifest declares the required<all_urls>permission. - No shadow DOM piercing β content script can read open shadow roots via
element.shadowRoot, but cannot pierce closed roots. - Cloud Bridge lives in the background page β with no offscreen document,
src/cloud-bridge.jsholds the outbound WebSocket and handscloud_*commands straight tohandleMessage;src/cloud-runs.jsmirrors the Chrome controller and receives the bridge as an injected dependency. The protocol and optional browser approval are identical to Chrome (docs/cloud-bridge-browser-approval.md). - No offscreen document β no HTTP fetch proxy for localhost LLM servers with Private Network Access / CORS issues. User must ensure their local LLM server sends permissive CORS headers.
- Some Chrome-only tools/features remain absent β no CDP full-page screenshot, CDP upload automation, tab recording, offscreen fetch proxy, Chrome-only
shadow_dom_query, or closed-shadow-root traversal.
Everything else β the agent loop, LLM providers, site adapters, Ask/Act/Dev mode routing, Plan before Act, loop detection, API shortcut observer, trace recorder, scheduler, context management β is architecturally identical to Chrome unless noted below.
PDF handling is an intentional platform exception: Firefox has no equivalent
to Chrome's global mime_types_handler/chrome.mimeHandler route in this
extension. Firefox therefore keeps its native PDF viewer as the default and
uses an explicit WebBrain PDF viewer context-menu entry when the user chooses
it. The Chrome-only automatic PDF viewer setting (on by default) does not apply to
Firefox; the explicit Firefox entry remains available independently.
The explicit Firefox route is URL/GET based. It cannot replay an arbitrary POST navigation because the context-menu event does not expose the original request body; those documents remain in the native viewer unless a stable PDF URL is available. Chrome's MIME-handler route receives the browser's one-time stream and therefore also covers PDF responses that originated from POST navigations.
High-Level Overview
ββββββββββββββ messages βββββββββββββββ HTTP/JSON ββββββββββββββββ
β Sidebar β ββββββββββββββββΊ β Background β ββββββββββββββββΊ β LLM Providerβ
β (UI) β browser.runtime β Page β fetch() β (OpenAI / β
β sidepanel β .sendMessage β agent.js β β Anthropic /β
β .js β β background β β llama.cpp) β
ββββββββ¬ββββββ β .js β ββββββββββββββββ
β ββββββββ¬βββββββ
β β
β browser.tabs.executeScript / sendMessage
β β
β βΌ
β ββββββββββββββββββββββββββββββββ
β β Content Scripts (injected) β
β β β’ accessibility-tree.js β
β β β’ content.js β
β ββββββββββββββββββββββββββββββββ
β β
βββββββββββββββββββββββββββββββββ
DOM / Page
Key differences from Chrome: No CDP client. No offscreen document. All DOM interaction happens through content script injection only, and all HTTP requests happen directly from the background page.
Directory Structure
src/firefox/
βββ manifest.json # Manifest V2 config
βββ src/
β βββ background.html # Background page (MV2 requirement)
β βββ background.js # Message router
β βββ run-ui-journal.js # Detached-run replay + streamed-text snapshots
β βββ agent/
β β βββ agent.js # Core agent loop
β β βββ loop-detector.js # Browser-free loop detection, directly unit-tested
β β βββ image-budget.js # Browser-free screenshot sizing, directly unit-tested
β β βββ mutation-tools.js # This build's state-change + mutating tool sets
β β βββ tools.js # Tool schemas + system prompts (incl. 4 AX tools)
β β βββ skills.js # Settings skills + dynamic skill tool manifests
β β βββ planner.js # Plan-before-Act structured planner
β β βββ permission-gate.js # Capability x origin permission gate
β β βββ adapters.js # Per-site guidance (identical to Chrome)
β β βββ scheduler.js # ScheduledJobManager β alarms-backed deferred tasks
β βββ content/
β β βββ accessibility-tree.js # AX tree builder + ref_id registry (NEW in 3.6.8)
β β βββ content.js # DOM reader / typer / clicker + AX handlers
β βββ network/
β β βββ network-tools.js # fetch_url, research_url, skill HTTP tools
β βββ providers/
β β βββ base.js # Provider interface
β β βββ manager.js # Provider lifecycle
β β βββ openai.js # OpenAI-compatible
β β βββ azure-openai.js # Azure OpenAI deployments
β β βββ aws-bedrock.js # AWS Bedrock Converse
β β βββ anthropic.js # Anthropic Claude
β β βββ llamacpp.js # Local llama.cpp server
β β βββ fetch-timeout.js # Direct provider fetch timeout wrapper
β βββ trace/
β β βββ recorder.js # Optional IndexedDB run recorder
β βββ ui/
β βββ sidepanel.html
β βββ sidepanel.js # Chat UI, verbose mode, deep verbose
β βββ settings.html # Settings: General, Budgets, Memory, Skills, CAPTCHA, Bridge, Advanced
β βββ providers.html # Providers: LLM, Vision, Speech, Generative, Decision, SafeSocial
β βββ permissions.html # Site permissions
β βββ settings.css # Shared by the three settings pages
β βββ settings.js # One controller for all three settings pages
β βββ traces.html
β βββ traces.js
βββ skills/ # Packaged default skills (removable after seeding)
βββ icons/
Notable absences vs Chrome: no cdp/, no offscreen/, no recorder/, no providers/fetch-with-fallback.js. The Cloud Bridge (cloud-bridge.js, cloud-runs.js) sits next to background.js instead of under offscreen/.
Permissions
{
"permissions": [
"activeTab",
"menus",
"webNavigation",
"webRequest",
"dns",
"storage",
"unlimitedStorage",
"tabs",
"tabGroups",
"downloads",
"alarms",
"clipboardWrite",
"clipboardRead",
"<all_urls>"
]
}
Notably missing vs Chrome: debugger, sidePanel, scripting, offscreen, privateNetworkAccess, tabCapture.
- No
debuggerβ no CDP, no trusted events - No
offscreenβ no HTTP fetch proxy; direct fetch from background page only - No
privateNetworkAccessβ localhost LLM servers must send CORS headers themselves webRequestis used for the same in-memory API shortcut observer as Chrome. The setting is on by default and can be disabled in Settings.dns(Firefox only) letsfetch_url,read_page_sourceandresearch_urlresolve the target hostname and refuse a blocked answer before sending; both builds also check the server address webRequest reports for the response. See Local network blocking indocs/security-model.md.- Uses
sidebar_action(MV2) instead ofside_panel(MV3) - Uses
browser.tabs.executeScript()/browser.tabs.sendMessage()instead ofchrome.scripting.executeScript()
unlimitedStorage supports the optional IndexedDB trace recorder, matching Chrome's trace storage model.
The Accessibility-Tree System (v3.6.x)
As of v3.6.8, Firefox ships content/accessibility-tree.js β the same DOM-walker that Chrome uses to produce a compact, indexed, semantic snapshot of the page for the LLM. The file was ported verbatim from Chrome; it uses only standard DOM APIs and works unchanged in Firefox.
What the tree is
A flattened outline of the page's interactive and informative nodes, each assigned a stable ref_id:
[1] button "Sign in"
[2] textbox "Email" value="emre@..."
[3] textbox "Password"
[4] searchbox "Search" aria-controls="listbox-1"
[5] listbox
[6] option "Every month"
[7] option "Custom"
The tree is produced by a single DOM walk with these rules:
- Filtering:
display:none,visibility:hidden,aria-hidden=true, and zero-dimension nodes are skipped. Overlay containers (fixed/absolute with very high z-index) are hoisted so portaled dropdowns show up near the input that opened them, not at the end of the tree. - Accessible name resolution β priority order:
aria-labelledbyresolved to referenced elements' textaria-label<label for>/ wrapping<label>placeholder(for inputs/textareas, as a weaker hint)- For submit/button/reset inputs: the
valueattribute - For buttons / links /
<summary>with no inner text: the fullinnerText - New in 3.6.8: for unlabeled inputs/textareas/selects/textboxes/searchboxes/spinbuttons/comboboxes, scan preceding siblings and parent's preceding siblings for adjacent text nodes that look like labels
- Value attribute (v3.6.8) β
<input>and<textarea>render their live.valueas a separatevalue="..."attribute on the tree line. Values are truncated to 60 chars. Skipped input types:submit, button, reset, file, checkbox, radio, image, hidden, color, range, password. This cleanly separates "what this field is called" from "what it currently contains" β a fix for the v3.6.7 Stripe bug wheretextbox "1"was ambiguous between a textbox named "1" and one containing "1". - ref_id registry β each emitted element is stored in
window.__wbElementMapas aWeakRef, keyed by ref_id. The map is cleared at the start of every tree build so IDs don't leak between turns. All AX tools resolve ref_id through this map, getting fast O(1) lookup and automatic garbage collection of stale entries. - Soft truncation β the tree is capped at ~3000 chars. If it overflows, later elements are dropped with a truncation marker so the LLM knows the snapshot is incomplete.
AX tools
| Tool | Purpose |
|---|---|
get_accessibility_tree |
Returns the rendered tree (string) plus metadata β used as the LLM's primary page-understanding surface for v3.6.x |
click_ax |
Click an element by ref_id. Resolves through __wbElementMap, scrolls into view, dispatches el.click() |
type_ax |
Type into an input/textarea/contenteditable by ref_id. Uses the native value setter (bypasses React controlled-input wrappers), dispatches input + change |
set_field |
Combined type-and-submit for fields that participate in a combobox/autocomplete. Types the value, detects whether the field is a combobox (role=searchbox/combobox, aria-autocomplete, aria-controls pointing at a listbox, or any visible listbox on the page), and if so dispatches ArrowDown β Enter with small delays to commit the highlighted option. Otherwise falls back to form.requestSubmit(). This is the main fix for the Stripe "Every N months" bug β it lets the agent close combobox selections deterministically |
Firefox's AX tools use synthetic events only β there is no trusted-event path. Sites that check event.isTrusted will reject these the same way they reject legacy click / type_text.
What was intentionally skipped in the Firefox port
These Chrome v3.6.x features depend on CDP and were not ported β they can be re-evaluated later:
- CDP-enriched
click_axfrontmost resolution β whenclick_axlands on a node that overlaps many candidates, Chrome re-queries via CDP to pick the frontmost hit. Firefox relies on the initial ref_id resolution plus the v4.0.1 occlusion hit-test (see below). - Offscreen fetch fallback β Chrome falls through to an offscreen-document proxy when direct fetch fails (common for localhost LLM servers and private-network destinations). Firefox has no offscreen API; local servers must handle CORS themselves.
Overlay defenses (v4.0.1+)
Brought to Chrome parity in v4.0.1 β same three layers, synthetic-event-safe so they work without CDP:
- Modal-scoped text click.
click({text: ...})incontent.jsresolves_findTopmostModal()(native<dialog open>,aria-modal,[role=dialog], common overlay patterns) and scopes itsquerySelectorAllto that subtree β plus the labelβinput map, the 3Γscroll-down retry, and the contenteditable/[role]/[tabindex] fallback. Closes the GitHub "dimmed Publish button behind Create-tag dialog" class of failures. - Post-click occlusion hit-test. For text/selector/index clicks (skipped for x,y and SELECT), after
scrollIntoViewbut before.click(), the resolver callselementFromPoint(cx, cy)at the target center. If topmost is neither the target nor a DOM ancestor/descendant, the click is refused with{occluded: true, occludedBy}and a force-click hint. - Rich ambiguity payload. Ambiguous text matches now return Chrome's
{index, tag, role, text, cx, cy, rect, ancestor}candidates. Theancestorstring identifies the containing dialog/form so the model can disambiguate "Cancel in alertdialog" vs "Cancel in form" by location.
blockedDone heuristic (ported in v4.0.1). Firefox's done now probes open dialogs, visible forms, and live-region messages via browser.tabs.executeScript (MV2 equivalent of Chrome's CDP probe). If the summary claims completion while a modal or form is still visible, returns {blockedDone: true} up to 2Γ per tab before letting done through with a loud verification note. Block count cleared on clearConversation.
System prompt has a new "MODALS & DIALOGS" section describing the intended flow and the "dialog still open" failure pattern.
Duplicate-submit guard
Submit-like text clicks use the same browser-free guard as Chrome. The first
click is recorded by tab, normalized label, and current URL; another matching
click within 45 seconds is blocked unless _allowResubmit explicitly
acknowledges the retry. An acknowledged retry re-arms the window, so a further
rapid duplicate needs its own acknowledgement. Navigation, expired entries,
ordinary labels, and validation-rejected submits remain eligible for a fresh
click.
Skills and Dynamic Skill Tools
Settings -> Skills stores enabled skills under customSkills. On first run,
background.js seeds packaged skills from skills/*; FreeSkillz.xyz is enabled
by default but is just a stored built-in skill, so the user can remove it. If a
packaged built-in skill changes and the user still has it enabled, startup
refreshes the stored copy without re-adding deleted skills.
agent/skills.js splits each skill into two surfaces:
- prompt instructions appended by
buildCustomSkillsPrompt(); - optional tool schemas declared in fenced
webbrain-toolsJSON blocks.
The manifest fence is stripped before prompt injection. Declared skill tools are
appended to getToolsForMode(...) at LLM-call time and executed through
executeHttpSkillTool() in network-tools.js. Current skill tools support
read-only HTTPS GET/POST integrations and HTTPS download-job integrations that
poll a same-origin status URL, save through browser Downloads, and clean up the
provider job. Requests use credentials: "omit", optional URL input allowlists,
and optional response limits.
Importing/enabling a skill is the trust boundary. After import, the declared
tool can contact its declared endpoint without a per-call permission prompt.
Download-job tools still run in action modes and use the normal Downloads
permission gate before saving files. Third-party results should use
resultPolicy: "untrusted" so the agent wraps and digests them like page
content instead of trusted instructions.
The exact packaged Wikipedia skill uses agent/wikipedia-offline.js to fall
back to user-installed Kiwix/ZIM archives after a live request fails.
agent/apocalypse-mode.js owns the opt-in archive manager, resumable verified
downloads, durable IndexedDB state, OPFS bytes, and local openZIM title lookup.
No archive is downloaded by enabling the skill. Local passages retain their
canonical URL, language, archive date, and license metadata and stay untrusted.
Agent Loop
The agent loop is structurally identical to Chrome. The same Ask/Act/Dev
processMessage() / processMessageStream() flow runs:
User message
β
βΌ
_enrichFirstUserMessage() β same as Chrome (URL/title + screenshot + adapter)
β
βΌ
Main Loop (max steps from Settings, default 130)
β
ββ chatMainTurn(messages, {tools, temp, maxTokens:4096})
β ββ provider.chat() by default
β ββ official OpenAI Responses streaming for interactive Ask only
ββ If tool_calls β _executeToolBatch() β push results β continue
ββ If text only β return as final answer
ββ If done() β return summary
Sequential browser batches
Mid/Full Act and Dev expose browser_batch (6 actions for Mid, 10 for Full),
unless the internal browserBatchEnabled switch is false. Ask and Compact do
not expose it. _executeToolBatch validates the whole wrapper, then expands
its allowlisted actions into the existing execution loop. Every action retains
normal permission, targeting, submit, cancellation and verification checks.
An invalid later action prevents all dispatch. Navigation continues only into
trailing observations; failures, unverified actions and submits stop remaining
actions and sibling calls. Mid's single-action boundary is suppressed only
between actions of the same batch. Navigation-prone actions use the existing
bounded settling helpers before the next action.
The page monitor can derive same-batch continuation evidence after verified
native input/textarea edits by set_field, type_ax, or type_text. It updates only the
completed control's value while retaining original nodes, labels, ancestors,
form/recipient bindings, collateral values and human authority. Other batches
and top-level siblings cannot borrow this evidence. Select/contenteditable and
click transitions keep normal context-change stops; no model snapshot recapture
is used to accept a changed page.
The original assistant call and provider replay state remain intact. Results
fold into one parent tool message before persistence and provider conversion,
with internal sub-call IDs excluded and supplemental images following parent
results. Each sub-result keeps its own untrusted-content wrapper; aggregate
digests contain only app-owned execution metadata. UI and trace show the parent
and ordinary sub-call rows, with optional trace batch metadata for grouping.
Automatic capture remains once per executed turn, not once per sub-action.
Ask-only OpenAI Responses streaming
Firefox uses the same narrow integration as Chrome inside the production
processMessage() loop. Normal sidebar sends remain detached chat_start
runs owned and journaled by the background page, including reconnect replay and
Ask attachments. Only an interactive Ask run using the official OpenAI
Responses route can call provider.chatStream(), and the default-on Advanced
kill switch can force new chats back to provider.chat() immediately. Act,
Dev, scheduled, cloud, and Continue runs are non-streaming.
Text deltas may render before completion, but tool calls, usage, reasoning, and
Responses output Items are withheld from the loop until response.completed.
Transport/protocol interruptions clear partial text, disable streaming for the
rest of that run, and retry the generation through provider.chat(); terminal
HTTP/API and response.incomplete errors propagate without a duplicate
fallback request. No incomplete assistant turn or tool call is persisted or
executed. Live deltas remain immediate, while reconnect snapshots coalesce on
a 200 ms trailing interval; terminal updates and pre-tool durability
checkpoints flush immediately. The older full processMessageStream()
lifecycle remains separate; this feature does not move normal chat traffic
into it. The journal also keeps separately bounded accumulated streamed text,
so a reopened sidebar can rebuild in-progress Markdown after early delta events
have been acknowledged or trimmed from the replay window.
Conversation persistence (parity with Chrome)
Per-tab agent conversations, sidebar chat HTML, and detached-run UI snapshots
are mirrored to browser.storage.session as agentConv:<tabId>,
tabChat:<tabId>, and runUi:<tabId>, the same key shapes as Chrome. The
background page hydrates provider history before the next message; the sidebar
restores the visible transcript and reconnects to active runs on open/reopen.
The UI snapshot has a 256-event replay window plus separately bounded
accumulated streamed text, so an in-progress Markdown response survives a
sidebar close without duplicated or plaintext fragments. Delta writes
coalesce for 200 ms; non-delta, terminal, and pre-tool checkpoints persist
immediately.
// Both browsers:
this.conversations = new Map(); // + _persist() + _hydrate() β storage.session
Tools
Complete tool list
browser_batch groups advertised, allowlisted actions with optional trailing
observations. See Agent Tools
for its limits, stop rules and exclusions. Each action uses the existing Firefox
implementation, including its synthetic input limitations.
The model-facing tool surface is selected by conversation mode plus provider tier:
- Ask: semantic/read-only page tools only. Ask intentionally excludes
clarify,read_page_source,inspect_element_styles,execute_js, and action tools. - Act: the selected provider tier's normal browser-agent tools.
- Dev: Mid/Full only. Uses the selected Act tier, then adds source/style tools and Dev-extended shadow/frame inspection. Compact Dev is blocked.
AX tools (preferred in 3.6.x):
| Tool | Description |
|---|---|
get_accessibility_tree |
Indexed semantic snapshot + ref_id map |
click_ax |
Click by ref_id |
type_ax |
Type into field by ref_id |
set_field |
Type + combobox-aware commit (ArrowDown+Enter) or form submit |
hover |
Full Act only. Synthetic hover (mouseenter/mouseover/pointerover events). isTrusted: false β sites that gate hover-reveal on event trust will not respond. Works on most React/Vue handlers that listen to the standard events. |
drag_drop |
Full Act only. Synthetic drag: pointerdown/move/up + HTML5 dragstart/dragover/drop with constructed DataTransfer. Less reliable than Chrome's CDP-trusted path; verify by re-reading the tree. |
Legacy tier (kept for compatibility with older prompts and for non-AX flows):
| Tool | Description |
|---|---|
read_page, screenshot, get_interactive_elements |
Page content / image / indexed elements |
click, type_text, press_keys |
Text/selector/index-based interaction |
scroll, navigate, go_back, go_forward, promote_iframe, wait_for_element, wait_for_stable |
Page control. promote_iframe resolves one child frame and navigates the current run tab to its standalone URL. wait_for_stable polls MutationObserver + in-flight fetch/XHR β works identically to Chrome. |
extract_data, get_selection |
Data extraction / selected text |
get_shadow_dom, get_frames, iframe_read, iframe_click, iframe_type |
Frame / shadow DOM. Iframe reads enumerate labels, values, and per-selector matchIndex values; mutations fail before dispatch on ambiguity. An iframe type must be followed by same-scope verify_form({urlFilter}) before successful completion. get_shadow_dom and get_frames are Full Act and Dev-extended for Mid Dev. |
fetch_url, research_url |
HTTP / open-and-read |
list_downloads, read_downloaded_file, download_resource_from_page, download_files |
Download helpers via browser.downloads where available |
verify_form |
Reads form field values + viewport screenshot before submit |
done |
Signal completion |
Dev-only Firefox add-ons: read_page_source, inspect_element_styles, and execute_js. execute_js is intentionally absent from normal Act and is only exposed when the user selects Dev mode on a Mid/Full provider.
Chrome-only (absent in Firefox): full_page_screenshot, shadow_dom_query (CDP shadow-pierce variant), and slash-driven tab/screen recording.
Firefox upload_file supports downloadId re-fetch and a sidepanel user file picker; arbitrary local filePath uploads remain Chrome-only (CDP).
Click β content script implementation
Firefox's click implementations (both legacy click and new click_ax) live entirely in the content script. No CDP fallback:
// Text-based click: auto-fallback matching (legacy click tool)
const modes = explicit ? [explicit] : ['exact', 'prefix', 'contains'];
for (const m of modes) {
matches = tryMode(m);
if (matches.length === 1) break;
if (matches.length > 1) break;
}
// If found: synthetic click
el.scrollIntoView({ behavior: 'smooth', block: 'center' });
el.click(); // β NOT trusted
click_ax skips the text matching entirely and resolves the element via window.__wbElementMap[ref_id]?.deref(), then runs the same scroll + synthetic-click dispatch.
Type β three paths
Both type_text and type_ax end up in the same typing core:
- ContentEditable: sets
textContent, dispatchesbeforeinput+input+change - Select elements: matches option by value or visible text
- Input/Textarea: uses the native property setter via
Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, 'value').set.call(el, value)to bypass React/Vue controlled-component wrappers, then dispatchesinput+change
set_field β combobox-aware submit
'set_field': async () => {
// ... type the value, verify it landed ...
if (submit) {
const roleAttr = (el.getAttribute('role') || '').toLowerCase();
const controls = el.getAttribute('aria-controls');
let isCombobox = roleAttr === 'searchbox' || roleAttr === 'combobox'
|| el.getAttribute('aria-autocomplete')
|| el.getAttribute('aria-expanded') === 'true';
// controls-id lookup + visible-listbox fallback...
if (isCombobox) {
await new Promise(r => setTimeout(r, 80));
dispatchKey('keydown', 'ArrowDown', 40);
dispatchKey('keyup', 'ArrowDown', 40);
await new Promise(r => setTimeout(r, 30));
}
dispatchKey('keydown', 'Enter', 13);
// ... + form.requestSubmit() fallback for non-combobox
}
}
Press keys
No CDP, so Firefox dispatches synthetic KeyboardEvent and Tab is implemented via manual focus advancement:
const ev = new KeyboardEvent('keydown', {
key: 'Escape', code: 'Escape', keyCode: 27, which: 27,
bubbles: true, cancelable: true
});
target.dispatchEvent(ev);
document.dispatchEvent(ev);
Verify form
Reads all form field values via browser.tabs.executeScript() and captures a viewport screenshot via browser.tabs.captureTab(). The system prompt guides the LLM to call this before submitting important multi-field forms.
Content Script
The content script is the only way Firefox interacts with the page. content.js now imports accessibility-tree.js (as a separate file loaded first in the content_scripts list) and exposes the following handlers relevant to v3.6.x:
get_accessibility_tree β window.buildAccessibilityTree() β { tree, metadata }
click_ax β resolveRef(id) β scrollIntoView + el.click()
type_ax β resolveRef(id) β native setter + input/change
set_field β type + combobox detect β ArrowDown/Enter or submit
Plus the legacy handlers: read_page, click, type_text, press_keys, scroll, extract_data, get_selection, get_shadow_dom, get_frames, iframe_*, and Dev-only read/debug handlers.
Manifest wiring
"content_scripts": [{
"matches": ["<all_urls>"],
"js": ["src/content/accessibility-tree.js", "src/content/content.js"],
"run_at": "document_idle"
}]
accessibility-tree.js is loaded first so its window.buildAccessibilityTree and window.__wbElementMap are available by the time content.js wires up the message handlers.
Provider System
Identical to Chrome at the provider-class and configuration layer:
WebBrain Compass, nine local endpoints, Azure OpenAI, AWS Bedrock, Anthropic, and
the current direct-cloud/router OpenAI-compatible configs use the same message
format and conversion logic. The canonical current ID and default-model table
is maintained in
docs/providers-and-models.md.
Uses browser.storage.local instead of chrome.storage.local for config persistence.
No fetch-with-fallback. Chrome has a providers/fetch-with-fallback.js layer that catches direct-fetch failures (typically CORS/PNA on localhost LLM servers) and retries through an offscreen document. Firefox has no equivalent β all provider fetches go directly from the background page, and the local LLM server must set Access-Control-Allow-Origin: * (or the extension's moz-extension://β¦ origin) itself.
Scheduled Tasks (scheduler.js)
Firefox ships the same ScheduledJobManager class (src/firefox/src/agent/scheduler.js), using browser.alarms instead of chrome.alarms. Feature parity with the Chrome build except for one difference:
- No service-worker keepalive. Chrome pings
chrome.runtime.getPlatformInfoevery 20 s during a job run to prevent the MV3 service worker from dying mid-run. Firefox has a persistent background page (MV2) that is always alive, so no keepalive is needed.
Like Chrome, Firefox opens URL-target task tabs inactive and does not activate an existing target tab before a scheduled run.
All job kinds (resume, task), lifecycle states, retry/deferral logic, schedule types (once, recurring), LLM tools (schedule_resume, schedule_task), and storage key (wb_scheduled_jobs) are identical to Chrome. See docs/architecture.md Β§ Scheduled Tasks for the full reference.
Loop Detection, Context Management, Verbose Mode, Site Adapters
All identical to Chrome:
- Rich-text toolbar guard β
content/rich-text-toolbar-heuristic.jsis byte-identical to the Chrome copy and is loaded ahead ofcontent.jsby the manifest. Detection, the recovery contract, and the positive-proofverifiedsemantics for text-entry tools match Chrome exactly; seesrc/chrome/ARCHITECTURE.md Β§ Rich-Text Toolbar Guard. The only difference is adjudication: Chrome can also route a selector probe through CDP, which Firefox has no equivalent for, so Firefox always scores through the content script - Loop detection β
agent/loop-detector.jsis inherited byAgentand imported directly by the unit suite; it contains the three detectors (general repeat, coordinate click, navigation) with the same thresholds and nudge/stop behavior. It is byte-identical to the Chrome copy; the tool sets it classifies against differ and live inagent/mutation-tools.js - Context management β auto-trim at >50 messages or >80,000 chars, LLM-powered summarization, emergency trim on context overflow, image pruning (last 4 only), tool-result cap at 8,000 chars
- Verbose mode β three levels: Normal / Verbose ON / Deep verbose (Shift+click dumps the LLM-payload ring buffer to DevTools console). Deep verbose works identically; there's just no persisted trace UI to browse it from
- Site adapters β same 110+ adapter set as Chrome across code/dev, productivity, social, messaging, e-commerce, travel, finance, news paywalls, job portals, and other regional surfaces; same
getActiveAdapter(url)matching and mid-conversation re-injection. Only ONE adapter fires at a time so prompt cost is fixed regardless of total count. Every match emits content-free adapter/revision/notes-injected trace metadata. Selected high-evidence adapters also expose identicalwebbrain-adapter-workflow/2jobs: both planner variants receive bounded app-owned IDs/descriptions, the binding is revalidated against the live URL immediately before execution, and trusted Continue fallback retains it only for the same adapter/revision/schema/job. The executor receives the selected stages/evidence contract. Required submissions need job-bound terminal evidence after dispatch (for example paid/ticket-issued transaction state or recipient-bound sent-message state); repeated jobs must exactly reconcile terminal ledger IDs against a complete app-owned accessibility-tree or seeded inventory. Edited reviews clear hidden routing, and selected jobs additionally retain only adapter/revision/job/template identity. - Recipient guard β same structured planner target and URL-scoped runtime policy as Chrome. On Douyin
/chat, Firefox pins anactive_conversationrequest to exactly one strong visible header before any page tool runs, then uses a read-only content-script probe immediately before send-like dispatch. Only one unique exact identity from the narrow, non-scrollable header above a lower-page layout composer can authorize the send. Enter in another editable such as recipient search is non-message, and a structurally verified conversation row in the separate left rail remains selectable even when a short list does not overflow, while distant controls and nested row actions remain inconclusive. Protected composer Enter dispatch is limited to one keypress per verification. Send-capable clicks, accessibility clicks, submitted fields, and Enter presses carry a one-use binding to the action target, composer, URL, and identity set and consume it immediately before the consequential click or key event. Ordinary message text, mismatches, unresolved controls/composers, ambiguity, and dispatch paths that cannot bind their effects to the verified recipient all fail closed.upload_fileis included because attaching a file can trigger an immediate page-side send. Saved workflows cannot inherit a planner recipient target, so any potentially dispatching step scoped to a protected messaging route stops before deterministic replay and must be run as a normal Act task with a freshly named recipient.
Side Panel UI
| Feature | Chrome | Firefox |
|---|---|---|
| Panel type | Side panel (MV3 API) | Sidebar action (MV2) |
| Chat persistence | Survives panel close | Survives sidebar close (browser.storage.session) |
| Long-reply navigation | Reading-first anchors + floating navigation | Same behavior and localized controls |
| Tab tracking | chrome.tabs.onActivated + session storage |
browser.tabs.onActivated + session storage |
| Background comms | chrome.runtime.sendMessage |
browser.runtime.sendMessage (Promise-based) |
| Trace viewer | Yes (ui/traces.html) |
No |
Message Flow β Walkthrough (Stripe "Every 2 months" example)
1. User: "create a product priced at 500 CNY billed every 2 months"
in a Stripe product-creation tab.
2. sidepanel.js β sendRunWithReconnect('chat_start', {
text, mode: 'act', tabId, requestId
})
3. background.js begins/persists `runUi:<tabId>`, starts the detached `chat`
lifecycle, then calls `agent.processMessage(tabId, text, onUpdate, 'act')`.
4. Agent builds messages and calls `provider.chat(...)`.
5. LLM: get_accessibility_tree {}
β content.js runs buildAccessibilityTree
β tree returned:
[12] textbox "Name"
[17] textbox "Price" value="500"
[22] combobox "Currency" value="CNY"
[31] combobox "Billing period"
...
6. LLM: set_field {ref_id: 31, value: "Custom", submit: true}
β content.js types "Custom", detects combobox (role=combobox +
aria-controls listbox), fires ArrowDown+Enter β option commits.
7. Agent auto-screenshots, pushes tree+screenshot back.
8. LLM: get_accessibility_tree {}
β tree now shows the newly-rendered "Every N months" row with
[45] spinbutton "Every" value="1" and adjacent labels resolved
by the preceding-sibling scan.
9. LLM: type_ax {ref_id: 45, value: "2"}
β native setter writes 2, input event fires.
10. LLM: click_ax {ref_id: <Save button>} β synthetic click.
11. LLM: done({summary: "Product created..."})
β sidepanel renders final message.
Same end-to-end shape as Chrome, minus the CDP-trusted-event path and the offscreen fetch fallback.
Planner prompts follow Chrome's token-minimal gating: the base planner prompt
includes general repeated-task pacing, while API replay guidance is appended only
when API mutations are authorized by the persistent setting or the tab's
/allow-api override. Both compact and full planner schemas carry a
language-neutral messaging target only when the trusted request authorizes an
external message.
Limitations vs Chrome
| Limitation | Impact | Workaround |
|---|---|---|
No CDP (no debugger permission) |
Clicks are synthetic (isTrusted: false) |
Most sites work; some banking/captcha sites may reject |
| No offscreen document | Localhost LLM fetches fail on CORS | Configure server CORS headers |
| No trusted keyboard events | press_keys may not land on all sites |
Dispatched to both activeElement and document |
| No full-page screenshot | Only visible viewport | Scroll + multiple captures |
| No shadow-root piercing (closed) | Can't read closed shadow roots | Dev-mode execute_js with manual traversal |
| No arbitrary-path/CDP upload | Cannot attach an arbitrary local path silently | Use a prior downloadId re-fetch or WebBrain's user file picker |
| No ambiguous-click CDP enrichment | Overlapping hit-target ambiguity resolved by ref_id only | Prompting / adapter guidance |
| MV2 background page | Less efficient than MV3 service worker | persistent: false helps |
Security Model
Same as Chrome, minus CDP:
- Extension runs with user's full browser permissions
<all_urls>host permission allows content-script injection anywhere- Cross-origin iframes accessible via extension privilege
- Ask is read-only; Act and Dev are action modes. Dev adds source/style/page-inspection tools and is blocked for Compact-tier providers.
- Plan before Act can require user approval before any action-mode tool executes
- API shortcut observer is on by default and can be disabled in Settings; when enabled, it records bounded same-tab XHR/fetch replay metadata in memory only
- The persistent Always allow API mutations setting (on by default) or the conversation's
/allow-apioverride waives permission prompts for API mutations (POST/PUT/PATCH/DELETE viafetch_url/research_url). If the persistent setting cannot be read from storage, it grants no authorization. - Finance adapters get extra safety warnings
- Tool results capped at 8KB
- No remote code execution: all providers called via
fetch()with user-supplied keys; no eval of LLM responses
Versioning & Port Status
Firefox remains a self-contained MV2 build with mirrored agent/provider/UI code where the browser APIs allow it. The historical v3.6.8 port brought the AX subsystem to parity; current 25.7.12 parity includes Plan before Act, scheduled tasks, trace recording, the API shortcut observer, browser-history tools, reading-first navigation, and reconnect-safe streamed Markdown. The remaining gaps are browser-platform gaps listed above, mostly CDP/offscreen features.