Open-source AI browser agent for Chrome and Firefox (monorepo) ðŸ§
WebBrain vs Claude Chrome Extension
This note compares the local WebBrain checkout with ../webbrain-claude, a
deobfuscated Claude Chrome extension tree. It focuses on architecture,
model-callable tools, and website-specific adapter behavior.
Sources Inspected
WebBrain:
docs/architecture.mddocs/adding-a-tool.mddocs/site-adapters.mddocs/accessibility-tree-and-refs.mddocs/webbrain-tool-tiers.xlsxsrc/chrome/ARCHITECTURE.mdsrc/chrome/src/agent/tools.jssrc/chrome/src/agent/skills.jssrc/chrome/src/agent/adapters.jssrc/chrome/skills/freeskillz-xyz.md- Matching Firefox files where parity matters
Claude Chrome:
../webbrain-claude/manifest.json../webbrain-claude/settings.html../webbrain-claude/settings.js../webbrain-claude/assets/service-worker.js../webbrain-claude/assets/mcpPermissions.js../webbrain-claude/assets/PermissionManager.js../webbrain-claude/assets/sidepanel.js../webbrain-claude/assets/accessibility-tree.js
The Claude tree is bundled/minified in places. Tool names below were recovered
from toAnthropicSchema() definitions, native-message dispatch, and the
sidepanel quick-command prompt.
Architecture
Claude in Chrome 1.0.100: sequential batching
The 1.0.100 behavior summary supplied for this implementation adds
browser_batch: several browser actions run sequentially within one model
round trip. WebBrain now implements this with its own schemas, prompts and
existing execution guards in both browser builds. Mid/Full Act and Dev receive
the tool by default (6/10 action limits); Ask and Compact exclude it. The
internal browserBatchEnabled switch removes exposure and optional guidance.
WebBrain validates every action before dispatch and gates each action at its normal capability and origin boundary. Failure, unverified application and submit boundaries require a fresh turn; navigation can continue into trailing observations. One original parent call receives the folded results, preserving per-action untrusted-content boundaries and provider replay metadata. Trace sub-calls carry optional batch grouping metadata. No recurring nudge is enabled.
The same 1.0.100 reference summary describes Anthropic cache breakpoints and region/scale screenshot controls. This update implements only batching; it does not establish those other features as shipped or verified in WebBrain. The earlier tree comparison below remains a historical snapshot where its tool counts or bundled paths differ from the current source.
| Area | WebBrain | Claude Chrome extension |
|---|---|---|
| Browser support | Two mirrored extension builds: Chrome/Edge MV3 and Firefox MV2. | Chrome MV3 only in this tree. |
| Agent location | Extension owns the full agent loop in agent.js; providers are local extension modules. |
Two paths: a normal Anthropic tool-calling loop in the sidepanel, plus a native-host/MCP bridge in the service worker. |
| LLM provider model | Provider abstraction supports OpenAI-compatible, Anthropic, local llama.cpp/LM Studio/Ollama-style endpoints, and provider settings. | Primarily Anthropic Messages API; sidepanel quick mode points at http://localhost:4000 when configured, and native host names target Claude desktop / Claude Code integration. |
| Tool dispatch | getToolsForMode() returns OpenAI-style function schemas from tools.js, optionally extended by enabled skill tools. agent.js dispatches tools to content scripts, Chrome APIs, CDP, network helpers, or skill executors. |
MCP-style schemas live in mcpPermissions.js. Sidepanel executes tool calls by name. Quick mode uses a compact command DSL with no Anthropic tools, then converts commands into synthetic tool-use/results. |
| Page reading | Preferred AX tree tool with stable ref_ids plus prose/page-source/PDF readers. |
AX tree reader also exists and uses window.__wbElementMap / ref_ IDs, but the primary browser action tool is more coordinate/computer oriented. |
| Trusted browser events | Chrome uses CDP for trusted mouse/keyboard events, screenshots, closed shadow-root access, and some file-upload paths. Firefox uses synthetic events. | Chrome uses debugger/CDP for computer actions, screenshots, JavaScript evaluation, uploads, console/network tracking, and zoom screenshots. |
| Conversation controls | Ask/Act modes, plan-before-act, scratchpad, progress ledger, scheduled tasks/resumes, optional traces. | Permission modes, plan approval via update_plan, domain transition prompts, tab groups, compaction, native host/MCP status. |
| Dynamic extension model | User/imported skills can inject prompt text and declare webbrain-tools runtime tools. |
Native/MCP and shortcuts are the extension points visible in the deobfuscated tree; no equivalent user-editable Markdown tool manifest was found. |
WebBrain Tool Surface
Current static core tools from the local source:
- Chrome: 62 core tools.
- Firefox: 53 core tools.
- Chrome-only core tools:
shadow_dom_queryplus the reversible/diagnostic Dev toolsinject_css,remove_injected_css,patch_element,revert_patch,read_console,inspect_network_requests,inspect_event_listeners, andhighlight_element. execute_jsis shared: Chrome Dev uses CDP evaluation and Firefox Dev uses its MV2 content-script evaluator.- Dynamic skill tools can add more schemas at runtime and are not included in those counts.
Ask-mode core tools:
get_accessibility_tree, read_page, read_pdf,
get_window_info, get_interactive_elements, scroll, extract_data,
get_selection, find_text, done, wait_for_stable,
fetch_url, research_url, list_downloads
Chrome static core tool list (the union across modes and tiers):
get_accessibility_tree, click_ax, type_ax, set_field, set_checked, hover, drag_drop,
read_page, read_pdf, read_page_source, get_window_info, resize_window,
get_interactive_elements, click, type_text, press_keys, scroll, navigate,
go_back, go_forward, extract_data, inspect_element_styles, wait_for_element,
inject_css, remove_injected_css, patch_element, revert_patch, execute_js,
read_console, inspect_network_requests, inspect_event_listeners,
highlight_element, wait_for_stable, schedule_resume, schedule_task, get_selection, find_text,
promote_iframe,
done, clarify, get_shadow_dom, shadow_dom_query, get_frames, iframe_read,
iframe_click, iframe_type, fetch_url, research_url, list_downloads,
list_webmcp_tools, execute_webmcp_tool,
read_downloaded_file, download_resource_from_page, download_files,
upload_file, scratchpad_write, progress_update, progress_read,
verify_form, download_social_media, solve_captcha
Firefox omits the Chrome-only Dev tools and shadow_dom_query; the remaining
core surface, including Dev-only execute_js, is shared.
WebBrain Tool Families
| Family | Tools |
|---|---|
| AX-first DOM control | get_accessibility_tree, click_ax, type_ax, set_field, set_checked, hover, drag_drop |
| Legacy DOM fallback | get_interactive_elements, click, type_text, press_keys, scroll, wait_for_element, wait_for_stable |
| Navigation | navigate, go_back, go_forward, promote_iframe |
| Reading/extraction | read_page, read_pdf, read_page_source, extract_data, inspect_element_styles, get_selection, find_text |
| Dev editing and diagnostics | inject_css, remove_injected_css, patch_element, revert_patch, execute_js, read_console, inspect_network_requests, inspect_event_listeners, highlight_element |
| Shadow DOM and frames | get_shadow_dom, shadow_dom_query on Chrome, get_frames, iframe_read, iframe_click, iframe_type, promote_iframe |
| WebMCP (experimental, on by default) | list_webmcp_tools, execute_webmcp_tool on supported Chrome builds when enabled |
| Network and files | fetch_url, research_url, list_downloads, read_downloaded_file, download_resource_from_page, download_files, upload_file |
| Long-running work | schedule_resume, schedule_task, scratchpad_write, progress_update, progress_read |
| Safety/workflow | verify_form, clarify, done, solve_captcha |
| Media | download_social_media, plus dynamic skill tools when enabled |
WebBrain Dynamic Skill Tools
WebBrain has two tool classes declared inside skill Markdown fences:
kind: "http": read-only HTTPS GET/POST tools, available in Ask and Act according to their manifestmodes.kind: "httpDownloadJob": Act-only HTTPS POST job tools that create a job, poll status, fetch the file, save through browser Downloads, and clean up.
The bundled FreeSkillz.xyz skill exposes:
read_youtube_transcript
resolve_public_media
download_public_media
These are not /allow-api mutations. They are trusted at skill import/enable
time, use credentials: "omit", and should mark third-party results as
untrusted.
Claude Chrome Tool Surface
Recovered MCP/tool schemas:
computer, javascript_tool, file_upload, find, form_input, get_page_text,
gif_creator, navigate, read_console_messages, read_network_requests,
read_page, resize_window, tabs_context, tabs_create, turn_answer_start,
update_plan, upload_image, tabs_context_mcp, tabs_create_mcp, tabs_close_mcp,
shortcuts_list, shortcuts_execute
The biggest design difference is that Claude collapses many browser actions into
computer, with an action enum:
left_click, right_click, type, screenshot, wait, scroll, key,
left_click_drag, double_click, triple_click, zoom, scroll_to, hover
The sidepanel quick mode has a separate command DSL:
ST tabId select tab
NT url open new tab
LT list tabs
C x y click
RC x y right-click
DC x y double-click
TC x y triple-click
H x y hover
T text type
K keys press keys
S dir amt x y scroll
D x1 y1 x2 y2 drag
Z x1 y1 x2 y2 zoom screenshot region
N url navigate, back, or forward
J code execute JavaScript
W wait for page settle
Quick mode sends Anthropic tools: [], parses these textual commands, executes
them locally, then appends synthetic tool_use / tool_result messages with a
fresh screenshot.
Claude Tool Families
| Family | Tools |
|---|---|
| Browser/computer actions | computer, navigate, resize_window |
| Page reading/search | read_page, get_page_text, find |
| DOM/form/file work | form_input, file_upload, upload_image, javascript_tool |
| Debugging | read_console_messages, read_network_requests |
| Tabs and MCP tab groups | tabs_context, tabs_create, tabs_context_mcp, tabs_create_mcp, tabs_close_mcp |
| Workflow and permissions | update_plan, turn_answer_start |
| Reuse/export | gif_creator, shortcuts_list, shortcuts_execute |
Tool Differences
| Capability | WebBrain | Claude Chrome |
|---|---|---|
| Tool granularity | Many narrow tools: separate AX click/type/set-field, network, downloads, scheduler, iframe, PDF, source, progress tools. | Fewer high-level tools; browser input is mostly one computer tool plus action enum. |
| Primary reading path | get_accessibility_tree is the preferred first read and returns stable refs with pagination/auto-degradation behavior. |
read_page also returns an accessibility tree, but screenshot-driven coordinate control is more central, especially in quick mode. |
| Natural-language element lookup | No standalone model-powered find; the model usually reads the AX tree and chooses refs. |
Has find, which runs a small model call over the accessibility tree and returns matching refs. |
| Page text | read_page is prose/article-oriented; get_accessibility_tree is UI-oriented. |
Splits read_page as AX tree and get_page_text as raw/article text. |
| PDF reading | read_pdf extracts PDF text directly. |
No equivalent recovered. |
| Raw source reading | read_page_source exposes server-delivered HTML and asset URLs. |
No equivalent recovered. |
| Network fetch | fetch_url / research_url, with WebBrain-specific API mutation rules and /allow-api for mutating methods. |
No generic fetch tool recovered. Debug network logs exist through read_network_requests. |
| Console/network inspection | Chrome Dev exposes read_console, inspect_network_requests, and inspect_event_listeners; Firefox does not expose these Chrome-only diagnostics. |
Dedicated read_console_messages and read_network_requests. |
| Downloads | Several browser download/file tools plus dynamic download-job skill tools. | downloads permission exists and gif_creator can download exports, but no general download manager equivalent was recovered. |
| Media download | download_public_media skill first; download_social_media browser fallback. |
No public-media download equivalent recovered. |
| File upload | Both browsers have upload_file. Chrome supports downloadId and absolute filePath; Firefox supports downloadId re-fetch and a sidepanel user file picker (no arbitrary local paths without CDP). |
file_upload directly sets local absolute paths on a file input; upload_image uploads captured/user images by ref or coordinate. |
| Scheduler | schedule_resume and schedule_task. |
Scheduled task UI/prompt strings exist, but no equivalent model-callable scheduler schema was recovered in the visible tool list. |
| CAPTCHA | solve_captcha when CapSolver is configured. |
Explicit security prompt says respect CAPTCHAs and never bypass; no solver tool recovered. |
| Persistent agent memory | scratchpad_write, progress_update, progress_read. |
Conversation compaction exists; no equivalent scratchpad/progress tools recovered. |
| Form safety | verify_form for important forms. |
form_input can set values; no dedicated verify-form tool recovered. |
| Iframes | Dedicated get_frames, iframe_read, iframe_click, and iframe_type, plus promote_iframe to move a discovered child frame into the current tab as a standalone page with ambiguity and unsaved-draft checks. |
No dedicated iframe tools recovered; actions are likely through coordinates/JS where permitted. |
| Shortcuts/workflows | Custom skills are Markdown plus optional tool manifests. | shortcuts_list / shortcuts_execute expose saved shortcuts/workflows. |
| GIF/video workflow | Slash-driven recording exists in WebBrain Chrome, but not as model-callable tools. | gif_creator is model-callable and can record/export browser automation sessions as GIF. |
Website-Specific Adapters
WebBrain
WebBrain has a real site-adapter system:
- Adapter files live in both
src/chrome/src/agent/adapters.jsandsrc/firefox/src/agent/adapters.js. getActiveAdapter(url)returns the first matching adapter.- Only one adapter fires at a time.
- Adapter notes inject into the first user message.
- If navigation moves to a different matching adapter mid-conversation, WebBrain
injects a new
[Site context changed ...]user message. UNIVERSAL_PREAMBLEis added to the system prompt when adapters are enabled. It covers cookie/consent banners, paywalls, and PDF-tab behavior.- Finance adapters carry high-stakes wording and must precede
finance-generic. - Chrome and Firefox adapter changes should stay mirrored.
Current adapter inventory from listAdapters():
github, gitlab, stackoverflow, hackernews, gmail, google-docs,
google-calendar, slack, notion, jira, twitter, linkedin, reddit, youtube,
medium, substack, wordpress, amazon, aws, gcp, cloudflare, vercel, nytimes,
wsj, ft, bloomberg, economist, washingtonpost, stripe, coinbase, robinhood,
tradingview, finance-generic, airbnb, booking, expedia, google-maps,
google-flights, kayak, opentable, ebay, walmart, target, etsy, sahibinden,
trendyol, apple, outlook, google-sheets, trello, instagram, tiktok, facebook,
leetcode, hackerrank, greenhouse, workday, discord, whatsapp-web, telegram,
mastodon
Notable adapter behavior:
- Adapters encode page-shape guidance, not brittle selectors.
- WordPress is host-agnostic over
/wp-adminand/wp-login.php. - Mastodon uses a large known-host set plus conservative URL matching to avoid
claiming every generic
/@userpath on the web. - Finance precedence matters: broad finance matching can shadow specific adapters unless exclusions/order are maintained.
Claude Chrome
The deobfuscated Claude tree has settings UI that claims "Site adapters":
settings.htmlexposes a "Site adapters" toggle.settings.jsreads and writesuseSiteAdapters.
However, no backing adapter registry, getActiveAdapter equivalent, universal
preamble injection, or site-specific guidance injection path was found in the
inspected Claude tree. Searches for WebBrain-style adapter markers only found
the settings label/storage path.
The closest Claude equivalents are not website adapters:
- Domain permission modes and domain-transition prompts.
- Managed policy / domain block handling.
- Domain skills metadata in tab-context reminders.
- MCP/Gmail/Google-style integration labels elsewhere in the UI bundle.
So the practical adapter difference is:
- WebBrain has website-specific prompt augmentation as a first-class browser agent feature.
- Claude Chrome, in this deobfuscated tree, appears to rely on screenshots,
find, domain permissions, and tab/domain context rather than per-site adapter notes.
Ideas Worth Borrowing
Potentially useful Claude ideas for WebBrain:
- A
findtool that uses a small/fast model over the AX tree to return candidate refs for vague element descriptions. - Model-facing console and network request readers for debugging web apps.
- A GIF/workflow export tool if model-callable recording/export is desired.
- Shortcut/workflow list and execution primitives, if WebBrain wants a reusable workflow layer separate from Markdown skills.
- Direct image upload by captured screenshot/image ID, if sidepanel image attachment workflows grow.
Ideas to avoid copying directly:
- A settings-only "site adapters" surface without a backing registry and injection path.
- Collapsing too many deterministic browser operations into one
computerschema if WebBrain wants to preserve its current narrow, auditable tool semantics. - Treating screenshot/coordinate control as the primary path when stable AX refs are available.
Documentation Follow-Ups
If this comparison becomes user-facing docs:
- Update counts from
src/chrome/src/agent/tools.jsandsrc/firefox/src/agent/tools.jsbefore publishing. - Re-run
listAdapters()to avoid stale adapter inventory. - Treat Claude details as local reverse-engineering, not an upstream product contract.
- If the Claude tree is refreshed, re-check whether
useSiteAdaptersgained a backing adapter registry.