Skip to content

Open-source AI browser agent for Chrome and Firefox (monorepo) 🧠

10.3 KB
Raw Blame

WebBrain

Open-source AI browser agent for Chrome and Firefox. Chat with any web page, automate browser tasks, and run multi-step agent workflows β€” powered by your choice of LLM.

Features

  • Page Reading β€” Extracts text, links, forms, tables, and interactive elements from any page
  • Browser Actions β€” Click, type, scroll, navigate, and interact with page elements
  • Ask / Act / Dev Modes β€” Read-only by default, normal browser actions on request, and Mid/Full Dev tools for source/style/page debugging
  • Multi-Step Agent β€” Autonomous task execution with tool-use loops (configurable, default 130 steps)
  • Continue from Limit β€” At the step limit, WebBrain first delivers a context-only partial result or explicit blocker; click Continue to keep going
  • Multi-Provider LLM β€” WebBrain Compass plus local llama.cpp/Ollama/LM Studio/Jan/vLLM/SGLang/LocalAI and major direct cloud providers
  • Reliable Compass improvement traces β€” when Help Improve WebBrain is enabled, terminal tool outcomes are durably queued and retried without delaying the visible answer
  • Side Panel UI β€” Clean chat interface that lives alongside your browsing
  • Reading-first long replies β€” Questions stay visible while answers grow, with controls to follow, jump to the latest content, or return to the question
  • Per-Tab Conversations β€” Each tab has its own chat history
  • Ask streaming β€” Eligible interactive Ask chats stream official OpenAI Responses text; tools/history wait for terminal completion and interrupted transports fall back safely
  • Smart Context β€” Automatic context trimming, tool result limits, and emergency overflow recovery
  • Copy Support β€” Copy buttons on code blocks and full messages
  • Page Inspection Banner β€” Visual indicator when the agent is interacting with the page
  • Stop Button β€” Abort the agent mid-execution at any time
  • Deterministic Action Modes β€” Act and Dev use temperature 0.15 for browser-control decisions; Ask uses 0.3, and dedicated vision screenshot descriptions use 0

Quick Start

Chrome

git clone https://github.com/webbrain-one/webbrain.git
  1. Open Chrome β†’ chrome://extensions/
  2. Enable Developer mode (top right)
  3. Click Load unpacked β†’ select the webbrain/src/chrome folder

Firefox

git clone https://github.com/webbrain-one/webbrain.git
  1. Open Firefox β†’ about:debugging#/runtime/this-firefox
  2. Click Load Temporary Add-on
  3. Navigate to webbrain/src/firefox/ and select manifest.json

Note: Temporary add-ons are removed when Firefox restarts. For permanent installation, the extension needs to be signed via addons.mozilla.org.

Start a local LLM (optional)

# Using llama.cpp
llama-server -m your-model.gguf --port 8080

# Or using Ollama (OpenAI-compatible)
ollama serve
# Then set base URL to http://localhost:11434/v1 in settings
# Or run: ollama launch webbrain --model <model>
# Vision defaults to Auto and is detected from Ollama /api/show metadata

# Or using Jan, vLLM, SGLang, or LocalAI (OpenAI-compatible)
# Jan: http://localhost:1337/v1
# vLLM: http://localhost:8000/v1
# SGLang: http://localhost:30000/v1
# LocalAI: http://localhost:8080/v1

For llama.cpp, LM Studio, and LocalAI, Vision defaults to Auto. WebBrain reads the selected model's server metadata before a turn and sends screenshots only when image input is reported. Settings also offers Force on and Off overrides; metadata failures remain text-only for that turn.

Use it

Click the WebBrain icon β†’ the side panel opens. Type a message like:

  • "Summarize this page"
  • "Find all links about pricing"
  • "Fill in the search box with 'AI agents' and click Search"
  • "Navigate to github.com and find trending repositories"

Configuration

Click the gear icon or go to the extension's Options page to configure:

Display Settings:

  • Verbose Mode β€” Show full tool call JSON (off by default)
  • Auto-screenshot β€” Provide visual context when DOM/page reads are insufficient
  • Max Agent Steps β€” Configurable step limit (5-195 or unlimited, default 130)
  • Plan before Act β€” Try by default; optionally review a structured Act/Dev plan before browser tools run

Providers:

Base URLs are pre-filled in Settings. The complete, current provider/default model table lives in the root README; provider internals are documented in providers-and-models.md.

Architecture

src/chrome/                        src/firefox/
β”œβ”€β”€ manifest.json (MV3)            β”œβ”€β”€ manifest.json (MV2)
β”œβ”€β”€ src/                           β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ background.js              β”‚   β”œβ”€β”€ background.js (+ background.html)
β”‚   β”œβ”€β”€ run-ui-journal.js          β”‚   β”œβ”€β”€ run-ui-journal.js
β”‚   β”œβ”€β”€ agent/                     β”‚   β”œβ”€β”€ agent/
β”‚   β”‚   β”œβ”€β”€ agent.js               β”‚   β”‚   β”œβ”€β”€ agent.js
β”‚   β”‚   └── tools.js               β”‚   β”‚   └── tools.js
β”‚   β”œβ”€β”€ content/                   β”‚   β”œβ”€β”€ content/
β”‚   β”‚   └── content.js             β”‚   β”‚   └── content.js
β”‚   β”œβ”€β”€ providers/                 β”‚   β”œβ”€β”€ providers/
β”‚   β”‚   β”œβ”€β”€ base.js                β”‚   β”‚   β”œβ”€β”€ base.js
β”‚   β”‚   β”œβ”€β”€ llamacpp.js            β”‚   β”‚   β”œβ”€β”€ llamacpp.js
β”‚   β”‚   β”œβ”€β”€ openai.js              β”‚   β”‚   β”œβ”€β”€ openai.js
β”‚   β”‚   β”œβ”€β”€ anthropic.js           β”‚   β”‚   β”œβ”€β”€ anthropic.js
β”‚   β”‚   └── manager.js             β”‚   β”‚   └── manager.js
β”‚   └── ui/                        β”‚   └── ui/
β”‚       β”œβ”€β”€ sidepanel.html         β”‚       β”œβ”€β”€ sidepanel.html
β”‚       β”œβ”€β”€ sidepanel.js           β”‚       β”œβ”€β”€ sidepanel.js
β”‚       β”œβ”€β”€ settings.html          β”‚       β”œβ”€β”€ settings.html
β”‚       └── settings.js            β”‚       └── settings.js
β”œβ”€β”€ styles/                        β”œβ”€β”€ styles/
β”‚   └── sidepanel.css              β”‚   └── sidepanel.css
└── icons/                         └── icons/

Key difference: Chrome uses Manifest V3 (service worker, chrome.scripting, sidePanel API), Firefox uses Manifest V2 (background page, browser.tabs.executeScript, sidebar_action).

Agent Tools

Tool Ask Mode Act Mode Description
read_page Yes Yes Extract page text, links, forms
screenshot Yes Yes Capture visible tab
get_interactive_elements Yes Yes List all clickable/interactive elements
scroll Yes Yes Scroll the page
extract_data Yes Yes Extract tables, headings, images
get_selection Yes Yes Get highlighted text
click No Yes Click elements by selector, index, or coordinates
type_text No Yes Type into input fields
navigate No Yes Go to a URL
wait_for_element No Yes Wait for a selector to appear
execute_js No Dev only Run one JavaScript function body through the MV2 content-script evaluator
done Yes Yes Signal task completion

Browser-tab creation, enumeration, activation, and run retargeting are not general model-callable capabilities. Use a URL reader for another page or navigate the current run tab for interaction; an explicit separate-tab request is not silently converted into current-tab navigation. After the bundled OTP skill is active, Mid/Full expose one narrow exception: read_email_verification_message can inspect an already-open signed-in supported webmail tab, directly read a verified already-open message route, or open one opaque inspected inbox item in a temporary inactive duplicate that it closes. Candidate disclosure requires the full normalized service identity or all sufficiently discriminative service tokens. inspect remains read-only in Ask; because opening can mark mail read, open_message requires Act/Dev plus normal click permission on the mailbox host. Message reads consume exact bounded continuations to completion or fail closed. It does not expose a tab catalog, mailbox URL, or internal accessibility refs; Compact never receives it.

Dev-mode difference

Firefox keeps Dev-only execute_js, but does not expose Chrome's eight CDP-backed reversible/diagnostic additions: inject_css, remove_injected_css, patch_element, revert_patch, read_console, inspect_network_requests, inspect_event_listeners, or highlight_element. execute_js is absent from Ask and normal Act in both builds.

Known Issues

  • No Chrome DevTools Protocol (CDP) β€” Firefox uses synthetic content-script interaction, visible-viewport screenshots, and open-shadow access only. It cannot provide Chrome's trusted events, closed-shadow traversal, full-page CDP screenshots, or CDP diagnostics.
  • File upload limits β€” Firefox can re-fetch a prior downloadId or open WebBrain's own user file picker, but cannot silently attach an arbitrary local path.
  • Localhost CORS β€” Firefox has no offscreen fetch proxy; local model servers must allow extension origins.
  • No tab/screen recording β€” Slash-driven recording remains Chrome-only.
  • Firefox temporary add-on β€” Firefox requires the extension to be loaded as a temporary add-on during development, which is removed on restart.

Project Status

Downloads, downloadId/user-picker uploads, conversation/config export, dynamic skill tools, keyboard shortcuts, screenshots/vision, and the official Firefox Add-ons package are implemented. Use the changelog for shipped changes and the repository issue tracker for future work.

Adding a New Provider

  1. Create a new class extending BaseLLMProvider in src/providers/
  2. Implement chat() and optionally chatStream()
  3. Register it in src/providers/manager.js

All providers normalize to a common response format:

{ content: string, toolCalls: Array|null, usage: Object|null }

Website

The repository web/ folder contains the landing page for webbrain.one, deployable to Vercel:

cd web
vercel dev    # local preview
vercel        # deploy

License

MIT β€” built by Emre Sokullu