Open-source AI browser agent for Chrome and Firefox (monorepo) π§
JavaScript ai-agentai-agentsai-toolsbrowser-aibrowser-automationbrowser-extensionchrome-aichrome-devtoolschrome-extensionchrome-extensionsfirefox-addonfirefox-addonsfirefox-extensionfirefox-extensionsgemmagemma4minimaxoffline-aioffline-llmqwen
chore: uncomment the .gitignore
azghr7bc135ad1f4a
diff --git a/.claude/context.md b/.claude/context.mddeleted file mode 100644index 0b0a7f5f3..000000000--- a/.claude/context.md+++ /dev/null@@ -1,701 +0,0 @@-# WebBrain Project Context--## What Is This Project?--WebBrain is an **open-source AI browser agent** β a browser extension for Chrome and Firefox that lets users chat with web pages, automate tasks, and run multi-step workflows using their choice of LLM (local or cloud).--The user types a natural-language instruction in a side panel, and an autonomous agent loop calls the LLM, executes tool calls (click, type, navigate, read page state, etc.), feeds results back to the LLM, and repeats until the task is done.--## Core Identity--- **Language**: Vanilla JavaScript (ES modules), no frameworks, no build step for the extension itself-- **License**: GPL-3.0-or-later (was MIT; change effective from 33.0.0. Releases before 33.0.0 remain MIT β see `LICENSES/MIT.txt`)-- **Version**: 36.8.0-- **Repository**: https://github.com/webbrain-one/webbrain.git-- **Website**: https://webbrain.one--## Architecture Overview--There are **two browser builds** that share almost all code but are duplicated so each is self-contained and can be loaded directly without a build step:--### Chrome Build (`src/chrome/`)--- **Manifest V3** β service worker background, `chrome.scripting`, `sidePanel` API-- CDP-backed trusted events (real `isTrusted=true` mouse/keyboard events)-- Offscreen document for fetch proxy and tab recording-- Shadow DOM piercing via CDP for closed roots-- Conversation persistence across service worker restarts via `chrome.storage.session`--### Firefox Build (`src/firefox/`)--- **Manifest V2** β background page, `browser.tabs.executeScript`, `sidebar_action`-- Synthetic events by default (`el.click()`, `new KeyboardEvent()`); optional `firefox-companion/` gives BiDi trusted input-- No offscreen document, no CDP, no closed shadow root support-- In-memory conversation only (no persistence across background restarts)--### Shared Components--- Agent loop, prompts, tools, adapters, providers, loop detection, context management-- Site adapters (138 sites)-- Provider system (local + cloud LLMs)-- Accessibility tree and ref-based interaction-- Trace recording (IndexedDB)--### MCP Server (`mcp-server/`)--- `@webbrain/mcp-server` (v0.1.0, TypeScript, MIT)-- Local stdio bridge letting any MCP client (Claude Code, Codex, Cursor, OpenClaw) drive the user's real authenticated browser via the running extension-- Client β `webbrain-mcp` (stdio) β `ws://127.0.0.1:17374/extension` β WebBrain extension-- Chromium only (Firefox build has no offscreen document bridge)-- `src/`: `bridge.ts`, `config.ts`, `index.ts`, `runs.ts`; tests in `test/` (6 `.mjs` suites)--### Firefox Companion (`firefox-companion/`)--- Optional local companion providing **WebDriver BiDi trusted-automation** for Firefox (Node β₯22)-- Native-messaging host (`webbrain-bidi`) + `ws://localhost:9222`; only extension ID `[email protected]` may connect-- Gives Firefox CDP-like `isTrusted` input (coordinate clicks with post-validation, char-by-char trusted typing, native selects, sandboxed uploads via `input.setFiles`) without CDP-- Not required for standard Firefox operation; extra files: `host.mjs`, `install.mjs`, `session.mjs`--### Marketing Site (`web/`)--- Pure HTML/CSS, built from `web/build/template.html` + `web/build/locales/*.json`-- Multi-language (ar, bn, de, en, es, fa, fr, he, hi, id, ja, ko, ms, nl, pl, pt, ru, th, tl, tr, uk, vi, zh)-- Deployed via Vercel--### LM Studio Plugin (`lmstudio-plugin/`)--- TypeScript, standalone tool provider for LM Studio-- Ports `fetch_url` and `research_url` as pure Node tools (no browser dependency)--## Directory Structure--```-webbrain/-βββ src/-β βββ chrome/ # Chrome MV3 extension-β β βββ manifest.json # MV3 manifest-β β βββ skills/ # Packaged default skills (markdown)-β β βββ icons/ # Extension icons-β β βββ styles/ # Extension CSS-β β βββ vendor/ # Third-party libs (pdfjs, turkish-deasciifier)-β β βββ src/-β β βββ background.js # Service worker (message router, agent lifecycle)-β β βββ agent/ # Agent loop, tools, adapters, skills, planner-β β β βββ agent.js # Main agent loop + executeTool()-β β β βββ tools.js # Tool schemas + system prompts-β β β βββ planner.js # Plan-before-Act JSON planner-β β β βββ adapters.js # Per-site guidance (138 adapters)-β β β βββ skills.js # Skill loading, catalog, tool exposure-β β β βββ permission-gate.js # Capability Γ origin gating-β β β βββ credential-fields.js # Secret detection-β β β βββ user-memory.js # Local preference memory-β β β βββ loop-bucket.js # URL-family loop detection bucketing-β β β βββ loop-detector.js # Extracted loop-detection helpers (repeat/coord/nav)-β β β βββ scheduler.js # Scheduled task management-β β β βββ pdf-tools.js # PDF text extraction-β β β βββ pdf-extraction.js # PDF parsing-β β β βββ pdf-ocr.js # PDF OCR-β β β βββ pdf-stream.js # PDF streaming-β β β βββ captcha-solver.js # CapSolver integration-β β β βββ captcha-gate.js # Frame-aware CAPTCHA detection/gate-β β β βββ captcha-frame-runtime.js # CAPTCHA frame runtime helpers-β β β βββ capsolver-config.js # CapSolver config-β β β βββ screenshot-redaction.js # Privacy redaction-β β β βββ sheets-tools.js # A1 range parsing, TSV roundtrip-β β β βββ progress-ledger.js # Progress tracking-β β β βββ progress-intent.js # Progress intent tracking-β β β βββ json-extract.js # JSON extraction helpers-β β β βββ text-sanitize.js # Text sanitization (canonical)-β β β βββ image-budget.js # Auto-screenshot budget helpers-β β β βββ read-page-window.js # read_page pagination windows-β β β βββ tool-call-parser.js # Text tool-call parsing-β β β βββ tool-arguments.js # Tool argument validation-β β β βββ submit-click-guard.js # Rapid duplicate-submit guard-β β β βββ mutation-tools.js # DOM mutation tools-β β β βββ trace-export.js # Trace to Markdown export-β β β βββ transcribe.js # Audio transcription-β β β βββ cloud-output.js # Cloud run output handling-β β β βββ runtime-context.js # Trusted runtime context-β β β βββ workflows.js # Workflow orchestration-β β β βββ chat-workflow.js # Chat workflow handling-β β β βββ adapter-workflow.js / adapter-workflow-evidence.js # Adapter workflow evidence-β β β βββ completion-invariant.js # Completion invariant checks-β β β βββ conversation-persistence.js # Conversation persistence-β β β βββ message-recipient-guard.js # Recipient guard (Gmail/X/social)-β β β βββ systemone-fast.js / systemone-judge.js / systemone-evidence.js # Jev system-β β β βββ otp-email-tool.js / otp-* # OTP email mailbox reader-β β β βββ research-escalation.js # Research escalation-β β β βββ social-media-downloader.js # Social media content downloading-β β β βββ social-publish-contract.js # Social publication contract-β β β βββ public-media-url.js # Public media URL resolution-β β β βββ offline-*.js # Offline RAG, retrieval, semantic, reranker, stopwords-β β β βββ zim-xapian.js / zim-xapian-runtime.js # Offline Xapian search-β β β βββ apocalypse-mode.js / emergency-*.js # Apocalypse Mode + Emergency Box-β β β βββ teacher-mode.js / rich-text-toolbar-guard.js / rich-text-toolbar-probe.js-β β β βββ model-output-diagnostics.js # Model output analysis-β β β βββ read-completeness.js # Read completeness tracking-β β β βββ archive-opfs-writer-worker.js # OPFS archiving worker-β β β βββ observers/ # Site-specific observers (github, mastodon)-β β βββ providers/ # LLM provider abstraction-β β β βββ base.js # BaseLLMProvider-β β β βββ manager.js # ProviderManager (registry + config)-β β β βββ provider-catalog.js # Provider catalog/registry-β β β βββ openai.js # OpenAI-compatible providers-β β β βββ anthropic.js # Anthropic Claude-β β β βββ llamacpp.js # llama.cpp local provider-β β β βββ azure-openai.js # Azure OpenAI-β β β βββ aws-bedrock.js # AWS Bedrock-β β β βββ vertex-anthropic.js # Google Vertex AI via Anthropic-β β β βββ context-windows.js # Context window detection/normalization-β β β βββ provider-compatibility.js # Provider feature flags-β β β βββ fetch-with-fallback.js # HTTP fetch with fallback-β β β βββ oauth-claude.js # Claude OAuth flow-β β β βββ oauth-subscriptions.js # OAuth subscription management-β β β βββ deepseek.js / deepseek-config.js # Dedicated DeepSeek provider-β β β βββ vision-capabilities.js # Vision support detection-β β β βββ connection-test-assets.js # Connection test fixtures-β β βββ content/ # Content scripts (injected into pages)-β β β βββ accessibility-tree.js # AX tree builder + ref_ids-β β β βββ content.js # DOM reader, clicker, typer-β β β βββ agent-visual-indicator.js # Pulsing border + Stop-β β β βββ selection-shortcut.js # Selection handling-β β β βββ redaction-regions.js # PII region collection-β β β βββ file-picker-guard-page.js # File picker guard page-β β β βββ ollama-launch-handoff.js # Ollama launch handoff-β β βββ network/ # Network tools-β β β βββ network-tools.js # fetch_url, downloads, URL validation-β β βββ cdp/ # Chrome DevTools Protocol (Chrome only)-β β β βββ cdp-client.js # chrome.debugger wrapper-β β β βββ image-utils.js # Image processing-β β βββ offscreen/ # Offscreen document (Chrome only)-β β βββ recorder/ # Tab/screen recording orchestration-β β βββ trace/ # IndexedDB trace recorder-β β βββ ui/ # Side panel UI-β β βββ sidepanel.html / sidepanel.js / sidepanel-window-scope.js-β β βββ settings.html / settings.js / settings-tabs.js-β β βββ traces.html / traces.js # Trace viewer-β β βββ history.html / history.js / history-text.js # Chat history-β β βββ i18n.js / locales/ # Internationalization (23 locales)-β β βββ markdown-render.js / markdown-link.js / skill-markdown.js-β β βββ theme.js / theme-bootstrap.js / ui-scale.js-β β βββ recommended-actions.js / store-review-prompt.js / run-error-dedupe.js-β β βββ context-menu-prompts.js / chat-history-store.js / tab-chat-persistence.js-β β βββ provider-icons.js / message-info.js-β β βββ attachment-drop.js / attachment-file.js / staged-screenshot-store.js-β β βββ selection-quote.js / watch-command.js / coupon-domains.js-β β βββ install.html / install.js / install.css / install-assets/-β β βββ mic-permission.html / mic-permission.js-β β βββ pdf-handler.html/js/css # PDF viewer-β β βββ wikipedia-*.js/html/css # Wikipedia reader/library-β β βββ apocalypse-*.js/html/css # Apocalypse Mode UI-β β βββ emergency-*.js/html/css # Emergency Box/comm/PDF/text UI-β β βββ offline-rag-readiness.js/css # Offline RAG readiness UI-β β βββ safesocial-settings.js # SafeSocial settings-β β βββ download-tracker.js/css # Download tracker-β β βββ utils.js # Shared UI utilities (escapeHtml, etc.)-β β βββ cloud-runs.js # Cloud run persistence-β β βββ run-capture.js # Run capture (screenshots/recordings)-β β βββ run-ui-journal.js # UI journal for runs-β β βββ context-menu-storage.js # Context menu storage (claim/ownership races)-β β βββ profile-sync.js # Profile synchronization-β β βββ run-reconnect.js # Run reconnect helpers-β β βββ config-transfer.js # Settings/config transfer-β β βββ chrome-protected-pages.js # Chrome restricted-page handling-β β βββ chrome-web-store-release.js # Web Store release automation-β β βββ download-directory.js # Download directory helpers-β β βββ download-result.js # Download result tracking-β β βββ error-format.js # Error formatting-β β βββ tab-group-preference.js # Tab group preferences-β β βββ selection-shortcut-i18n.js # Selection shortcut i18n-β β βββ ollama-handoff.js # Ollama launch handoff-β β-β βββ firefox/ # Firefox MV2 extension (mirrors Chrome structure)-β βββ manifest.json # MV2 manifest-β βββ skills/ # Packaged default skills (same as Chrome)-β βββ icons/ # Extension icons-β βββ styles/ # Extension CSS-β βββ vendor/ # Third-party libs (same as Chrome)-β βββ src/ # Same structure as Chrome, minus cdp/, offscreen/, recorder/-β # Firefox-exclusive: bidi/ (bind.js, client.js),-β # background.html, firefox-restricted-domains.js,-β # shortcut-command.js, watch-alert.js, smd-loader.js,-β # content/file-picker-guard-loader.js-β-βββ web/ # Marketing site-β βββ index.html # Generated landing page-β βββ privacy.html # Privacy policy-β βββ vercel.json # Vercel config-β βββ build/ # Build system-β β βββ build.mjs # Site generator (pure Node ESM)-β β βββ template.html # HTML template with {{t:key}} markers-β β βββ locales/ # Locale JSON files-β β βββ plausible.mjs # Analytics partial-β βββ assets/ # Site images-β βββ blog/ # Blog content-β βββ docs/ # Documentation site-β βββ {locale}/ # Generated locale pages (es, fr, zh, etc.)-β-βββ lmstudio-plugin/ # LM Studio plugin (TypeScript)-β βββ package.json # @webbrain/lmstudio-web-tools-β βββ tsconfig.json # TypeScript config-β βββ src/ # Plugin source-β βββ dist/ # Compiled output-β-βββ mcp-server/ # MCP server (TypeScript, `@webbrain/mcp-server`)-β βββ package.json # stdio bridge β extension via ws://127.0.0.1:17374-β βββ tsconfig.json-β βββ src/ # bridge.ts, config.ts, index.ts, runs.ts-β βββ test/ # 6 .mjs test suites-β-βββ firefox-companion/ # Firefox WebDriver BiDi trusted-automation companion-β βββ README.md # Node 22+, native messaging + ws://localhost:9222-β βββ host.mjs # webbrain-bidi native-messaging host-β βββ install.mjs # Companion installer-β βββ session.mjs # BiDi session logic-β-βββ LICENSES/ # Historical license texts (MIT.txt for pre-33.0.0)-β-βββ test/ # Tests (pure Node, no framework)-β βββ run.js # Main test runner (loads Chrome + Firefox modules)-β βββ README.md # Test documentation-β βββ security/ # Security/injection tests-β βββ fixtures/ # Fixture-based tests-β βββ anonymous/ # Anonymous usage tests-β βββ llm/ # LLM scenario tests-β βββ memory/ # User memory tests-β βββ smd-tests/ # Social media downloader tests-β βββ jev/ # Jev classifier tests-β βββ llm-tiny/ + llm-tiny-v2/ # Small-model test suites-β βββ vision/ + vision-results/ # Vision probe tests-β βββ systemone*.mjs # Jev fast-classifier tests-β βββ safesocial*.mjs # SafeSocial classifier tests-β βββ firefox-bidi*.mjs # Firefox BiDi companion tests-β βββ social-publish-contract*.mjs # Social publication contract tests-β βββ pdf-read.mjs / pdf-selection.mjs / pdf-mime-handler-e2e.mjs # PDF tests-β βββ provider-model-limits.mjs # Provider model limits-β βββ attachment-drop.mjs / build-unpacked.mjs / rich-text-toolbar-guard.mjs-β βββ browser-dialogs*.mjs / agent-lifecycle.mjs / runtime-lifecycle.mjs-β βββ webmcp-e2e.mjs # WebMCP end-to-end test-β βββ vision-probe.mjs # Vision probe test-β βββ manual-permissions.md # Manual permissions testing guide-β βββ manual-screenshot-redaction.md # Manual screenshot redaction testing guide-β-βββ ci/ # CI/CD pipeline-β βββ run.mjs # CI runner-β βββ test.mjs # CI test runner-β βββ README.md # CI documentation-β βββ cloud-capture.test.mjs # Cloud capture tests-β βββ lib/ # CI shared libraries-β β βββ webbrain-client.mjs # WebBrain client-β β βββ grader.mjs # Test grader-β β βββ suite.mjs # Test suite-β βββ catalog/ # Test catalogs-β βββ scenarios.json # Scenario definitions-β-βββ scripts/ # Build/dev scripts-β βββ build-blog.mjs # Blog builder-β βββ build-docs.mjs # Docs builder-β βββ build-zip.mjs # Release zip builder-β βββ build-zim-xapian.mjs # ZIM/Xapian index builder-β βββ build-unpacked.mjs # Unpacked build helper-β βββ bump-version.mjs # Version bumper-β βββ update-changelog.mjs # Changelog updater-β βββ update-coupon-domains.mjs # Coupon domain list updater-β βββ trace-to-otlp.mjs # Trace β OTLP exporter-β βββ preview-web.mjs # Web preview server-β βββ i18n-perm-translations.mjs # i18n permission translations-β βββ gen-store-promos.py # Store promo generator-β βββ sync-logo-assets.py # Logo sync-β-βββ docs/ # Documentation-β βββ architecture.md # System architecture-β βββ adding-a-tool.md # New tool checklist-β βββ accessibility-tree-and-refs.md # AX refs and page reads-β βββ site-adapters.md # Adapter guide-β βββ providers-and-models.md # Provider config-β βββ localization.md # i18n workflow-β βββ privacy-and-data-flow.md # Data handling-β βββ security-model.md # Permissions and risk-β βββ prompt-injection-defense.md # Injection defense-β βββ THREAT-MODEL.md # Threat model-β βββ test-scenarios.md # Test scenarios-β βββ claude-chrome-comparison.md # Comparison with Claude Chrome-β βββ agent-tools.md / skills.md / slash-commands.md # Tooling references-β βββ offline-rag.md / offline-rag-licensing.md / offline-rag-release-checklist.md-β βββ apocalypse-mode.md / remote-downloads.md / community.md / discord-setup.md-β βββ export-and-workflow-formats.md / social-publication-contract.md-β βββ selection-context-verification.md / trace-format-compatibility.md-β βββ accessibility-tree-benchmark.md / browser-agent-interface-findings.md-β βββ vision-models/ # Vision model docs-β βββ zh-CN/ # Chinese documentation-β βββ fr/ # French documentation-β-βββ assets/ # Logo and promo assets-βββ dist/ # Built release zips-βββ package.json # Root package.json-βββ AGENT.md # Agent guidelines (MUST READ)-βββ README.md # Project README-βββ README.es.md # Spanish README-βββ README.fr.md # French README-βββ README.zh-CN.md # Chinese README-βββ CHANGELOG.md # Version history-βββ CONTRIBUTING.md # Contribution guide-βββ CODE_OF_CONDUCT.md # Code of conduct-βββ GOVERNANCE.md # Project governance-βββ LICENSE # GPL-3.0-or-later-βββ LICENSES/ # Historical licenses (MIT.txt for pre-33.0.0)-βββ SECURITY.md # Security policy-βββ TODOs.md # Feature backlog-βββ .gitattributes # Git attributes-```--## Conversation Modes--WebBrain separates **model tier** from **conversation mode**:--### Tiers (`compact | mid | full`)--Control how many normal browser-agent tools a model sees:--- **Compact**: Reduced tool set + shorter system prompt for smaller local models-- **Mid**: Common task tools, iframe support, downloads, scheduling, form verification-- **Full**: Advanced browser-operation tools (hover, drag-drop, frames, shadow DOM)--### Modes (`ask | act | dev`)--Control what kind of task the user is allowing:--- **Ask**: Read-only. Agent can read, analyze, summarize but never click, type, or navigate-- **Act**: Exposes the selected tier's normal browser-agent tools-- **Dev**: Requires Mid/Full provider. Adds source/style/debug tools, page inspection, execute_js--## Agent Tools--Key tools organized by category (full list ~80 tools in `src/chrome/src/agent/tools.js`):--### Read-only (Ask + all modes)--- `get_accessibility_tree` β AX tree with ref_ids (preferred reader; auto-slices with continuation metadata, returns structured `pageGate` on rendered login/registration/paywall surfaces, Gmail `conversationRootRefId` support)-- `read_page` β Prose fallback for long-form articles; windowed with `continuationArgs`/`accessState`/`pageGate`-- `read_pdf` β PDF text extraction (pdfjs-dist; `hasExtractableText` flag; paginate with fromPage/toPage)-- `get_window_info` β Page metadata-- `get_interactive_elements` β Interactive elements-- `scroll` β Scroll the page (pane-aware via ref_id/x-y)-- `extract_data` β Structured data extraction-- `get_selection` β Selected text-- `fetch_url` β HTTP requests-- `research_url` β Extract readable article body-- `delegate_research` β Escalate to a research specialist runs-- `find_text` β Locate text on the page-- `gmail_count_results` β Gmail result counts-- `list_webmcp_tools` / `execute_webmcp_tool` β WebMCP tool discovery/execution-- `done` β Complete the task--### Action (Compact+)--- `click` / `click_ax` β Click elements-- `type_text` / `type_ax` / `set_field` / `set_checked` β Form interaction-- `press_keys` β Keyboard shortcuts-- `navigate` / `new_tab` / `list_tabs` / `activate_tab` β Navigation and tabs-- `carousel_navigate` β Carousel stepping-- `wait_for_element` / `wait_for_stable` β Wait conditions-- `scratchpad_write` β Write to scratchpad-- `progress_update` / `progress_read` β Progress tracking-- `chat_observe` / `chat_send` β Chat workflow observation + send-- `clarify` β Pause and ask the user (mid-call)-- `beep` β Audible notification-- `upload_file` β File uploads--### Advanced (Mid+)--- `go_back` / `go_forward` β Browser history-- `download_files` / `list_downloads` / `read_downloaded_file` / `download_resource_from_page` β Downloads-- `download_social_media` β Social media content download-- `schedule_task` / `schedule_resume` β Scheduled tasks-- `iframe_read` / `iframe_click` / `iframe_type` / `promote_iframe` β iframe interaction-- `solve_captcha` β CAPTCHA solving-- `verify_form` β Form verification-- `resize_window` / `inspect_viewport` β Window/viewport control--### Full-tier--- `hover` β Mouse hover (CDP-trusted)-- `drag_drop` β Drag and drop-- `get_shadow_dom` / `shadow_dom_query` β Shadow DOM access-- `get_frames` β Frame enumeration--### Dev-only--- `read_page_source` β View page source-- `inspect_element_styles` β Inspect CSS-- `execute_js` β Run JavaScript-- `inject_css` / `remove_injected_css` β Reversible CSS-- `patch_element` / `revert_patch` β Reversible DOM edits-- `highlight_element` β Visual overlay-- `read_console` β Console logs-- `inspect_network_requests` β Network requests-- `inspect_event_listeners` β Event listeners--## Provider System--All providers extend `BaseLLMProvider` and normalize to:--```js-{ content: string, toolCalls: Array|null, usage: Object|null }-```--### Local Providers (no API key needed)--- **llama.cpp** (port 8080)-- **Ollama** (port 11434/v1)-- **LM Studio** (port 1234/v1)-- **Jan** (port 1337/v1)-- **vLLM** (port 8000/v1)-- **SGLang** (port 30000/v1)-- **LocalAI** (port 8080/v1)-- **GPT4All** (local OpenAI-compatible)-- **WebGPU** β in-browser LLM inference (Compass Tiny v2.1, MiniCPM5-2B, LFM2.5, Bonsai 27B, Nanbeige4.2-3B presets); Chrome-only--### Cloud Providers (API key required)--- **WebBrain Compass** β Managed cloud (default; formerly "WebBrain Cloud 1.0", renamed 2026-09-01)-- **OpenAI** β GPT-5.6, etc.-- **Anthropic Claude** β Native API-- **Google Gemini**, **Mistral AI**, **DeepSeek** (dedicated provider), **xAI Grok**, **Groq**-- **MiniMax**, **Alibaba Cloud (Qwen)**-- **Cloudflare Workers AI**, **Nvidia NIM**-- **OpenRouter** β 100+ models (default: `openrouter/free`) + routing variants-- **Azure OpenAI**, **AWS Bedrock**-- **Unsloth Studio** (added 2026-08-25), **Pollinations AI** (2026-09-09), **NEAR AI Cloud** (2026-09-17)-- Plus **76 additional provider cards** in `provider-catalog.js` (104 built-in cards total, pre-filled base URLs and defaults β see `docs/providers-and-models.md`) β now with configurable model limits--## Skills System--Skills are optional prompt guidance + tool extensions:--- Stored in `chrome.storage.local` / `browser.storage.local`-- Packaged defaults ship in `src/{chrome,firefox}/skills/`-- Current defaults (12): FreeSkillz.xyz, OTP helper, Disposable email (Mail.tm), Open-Meteo weather, Open Library, Litterbox, Frankfurter FX, Wikipedia, Humanizer, Phonr calls, Turkish deasciifier, Chrome Web Store release-- Skills can expose HTTP tools via `webbrain-tools` manifest-- Skills are loaded per-run based on user intent (routed by planner or model)-- Agent Skills trust boundary enforced (structural plain scalars rejected, frontmatter validated)--## Site Adapters--138 site-specific adapters inject guidance into the first user message:--- Only ONE adapter fires at a time (first match)-- Adapters inject selectors, URL patterns, visible text, traps, success indicators-- 29+ added since 2026-08: India bundle (Swiggy, IRCTC, Paytm, Snapdeal), CIS/MENA (Wildberries, Avito, VK, Noon), LATAM (OLX, Despegar), East Asia/RU (Mercari, Yahoo JP, Naver, Yandex Market), Africa/MENA (Jumia, Kilimall, Careem, Talabat), EU/SEA (Bol, Otto, Willhaben, Tokopedia), baidu-tieba expansion-- Site adapters are high-leverage product work (short, concrete notes)--## Key Subsystems--### Plan before Act--- Optional action-mode planning gate-- Runs before first browser tool call-- Returns structured JSON with summary, steps, skill_ids, risks-- User approves/rejects before execution--### Loop Detection--Three independent detectors:--1. **General repeat** β Last 6 tool calls by (name + args hash + outcome)-2. **Coordinate click** β 5px-bucketed clicks-3. **Navigation** β URL snapshot before/after actions--### Context Management--- **Auto-compaction** β Summarizes older turns when nearing context window-- **Emergency trim** β Keeps only last 6 messages on overflow-- **Image pruning** β Strips base64 images from all but last 4 messages-- **Tool result cap** β 8KB per result--### User Memory--- Local, user-stated durable preferences-- Stored in `wb_user_memory_v1`-- Injected into system prompt as bounded block-- Optional auto-learning (off by default)--### Scheduling--- Deferred work using browser `alarms` API-- Job kinds: `resume` (continue conversation) and `task` (standalone prompt)-- Supports one-shot and recurring schedules-- Jobs persist in `chrome.storage.local`--### Offline RAG + Xapian (Apocalypse Mode)--- Offline-first question answering over locally-synced corpora (Wikipedia etc.)-- Semantic runtime, retrieval + reranker, query stopwords, ZIM/Xapian index (`scripts/build-zim-xapian.mjs`)-- Apocalypse Mode + Emergency Box UIs; teacher mode; OPFS archiving worker-- `docs/offline-rag.md` for details--### Jev Fast-Classifier System (systemone)--- Uses small/fast classifiers ("safe-stop", "systemone") to decide when an agent run should stop/pause before expensive slow-path inference-- TypeSafe scheduler judge β Assistive Models; fast-flow screenshot handoffs + event-driven wakeups-- `test/systemone*.mjs` suites (Some require Playwright browsers)--### Social Recipient Guard--- Pre-send recipient verification for Gmail/X/social DM composition-- X group DM recipient binding, message-history delivery proofs; generic-first guard on any site-- User-granted send authorization, fail-open guard, silent-reply recovery-- `message-recipient-guard.js` + site adapters--### PDF Tooling--- `read_pdf` agent tool (pdfjs-dist, per-page extraction, vision fallback note)-- PDF extraction/OCR/stream modules; PDF viewer + Wikipedia reader UI in the sidepanel--## Security Model--- MV3 manifest permissions: `sidePanel`, `activeTab`, `contextMenus`, `tabs`, `tabGroups`, `scripting`, `storage`, `notifications`, `webNavigation`, `webRequest`, `debugger`, `downloads`, `alarms`, `unlimitedStorage`, `offscreen`, `privateNetworkAccess`, `tabCapture`, `clipboardWrite/Read` + `<all_urls>`-- Ask is read-only; Act/Dev are action modes-- Plan before Act can require human approval-- `/allow-api` flag gates destructive HTTP methods-- Tool results capped at 8KB to limit injection surface-- `strictSecretMode` prevents credential quoting-- Trace data is local-only (IndexedDB)-- Untrusted content wrapped with `_wrapUntrusted` boundaries-- Screenshot redaction + URL redaction hardening for PII-- Social recipient guard: pre-send verification for Gmail/X/social DMs--## Build System--### Extension--**No build step** β load directly from `src/chrome/` or `src/firefox/`--`npm run build:chrome|build:firefox|build:all` β optional `scripts/build-unpacked.mjs` packaging--### Marketing Site--```bash-npm run build:web # Build landing page + blog + docs-npm run build:web:landing # Build landing page only-npm run build:blog # Build blog only-npm run build:docs # Build docs only-npm run preview:web # Preview site-```--### Release--```bash-npm run build:zip # Build release zip-npm run bump # Bump version-npm run release # Bump + release-```--## Testing--- `npm test` β Full suite (systemone, firefox-bidi, lifecycle, provider-limits, benchmark, toolbar-guard, pdf, social, safesocial, build-unpacked, attachment-drop, `test/run.js`, selection-scope, security); several suites need Playwright browsers (`npx playwright install`)-- `node test/run.js` β Core regression suite (pure Node, no framework, no chrome.\* APIs) β **2396 tests**-- `npm run test:security` β Security/injection tests-- `npm run test:injection-bench` β Prompt injection benchmarks-- `npm run test:safety-report` β Safety report-- `npm run test:fixtures` β Fixture tests-- `npm run test:anonymous` β Anonymous usage tests-- `npm run test:systemone[:fast|:ui]` β Jev classifier suite (+ DOM/UI variants)-- `npm run test:safesocial[:ui|:extension]` β SafeSocial suite-- `npm run test:firefox-bidi` / `:e2e` β Firefox BiDi companion tests-- `npm run test:social-contract[:dom]` β Social publication contract tests-- `npm run test:provider-limits` β Provider model limits-- `npm run test:pdf-read` / `test:pdf-selection` β PDF tooling tests-- `npm run test:attachment-drop` β Drag-and-drop attachment tests-- `npm run test:vision` / `test:webmcp` β Vision / WebMCP tests-- `node --check <file>` β Syntax-only JS check on touched files--Tests import Chrome and Firefox modules directly (no browser needed) and verify:--- Tool classifications and parity-- Provider config and compatibility-- Adapter matching-- Loop detection-- Permission gating-- Skills catalog and loading-- Markdown rendering-- Version bumping-- Changelog management-- And more--## Code Conventions--- **No frameworks** β Vanilla JS/CSS unless very strong reason-- **No build step for extension** β Direct load for development-- **Mirror changes** β Changes should be mirrored across Chrome and Firefox unless platform makes parity impossible-- **Keep tool schemas narrow** β Stable tool names, arguments, result shapes-- **Comments explain why** β Browser quirks, prompt rules, permission gates, context choices-- **Adapter notes imperative** β Name selectors, visible text, URL patterns, traps, success indicators-- **No broad dependencies** β Extension is intentionally simple to load and inspect--## Key Files to Read First--1. `AGENT.md` β Agent guidelines and goals-2. `docs/architecture.md` β System overview-3. `docs/adding-a-tool.md` β New tool checklist-4. `docs/site-adapters.md` β Adapter guide-5. `docs/providers-and-models.md` β Provider config-6. `docs/prompt-injection-defense.md` β Injection defense-7. `docs/security-model.md` β Permissions and risk-8. `docs/offline-rag.md` β Offline RAG/Apocalypse Mode-9. `docs/skills.md` / `docs/slash-commands.md` β Skills and slash commands-10. `src/chrome/src/agent/tools.js` β Tool definitions-11. `src/chrome/src/agent/agent.js` β Agent loop-12. `src/chrome/src/providers/manager.js` β Provider registry--## Current Status (as of 2026-09-22)--### Test Suite--- **2395 passed, 1 failed** (2396 total) via `node test/run.js`-- Failing test:- 1. `version 33-and-later licensing boundary is consistent across project metadata and FAQ copy` β **active regression**: commit `0b37aab73` (2026-09-21) removed the "releases before 33.0.0 remain MIT" paragraph from `LICENSE` that this test greps for. Fix either test or LICENSE/FAQ copy.-- Note: `npm test` runs additional Playwright suites (`test:systemone`, etc.) that need `npx playwright install` β environment, not code.--### Code Health--- **0 syntax errors** across key files (verified via `node --check`)-- **0 TODO/FIXME/HACK/BUG comments** in source `.js` files-- **11 engineering TODOs** tracked in `TODOs.md` (3 resolved, several partially resolved, rest open)-- Full issues report: `.claude/issues-report.md`--### Recent Changes--- **Licensing**: MIT β GPL-3.0-or-later effective 33.0.0; `LICENSES/MIT.txt` retains historical MIT text; README (incl. fr/zh-CN) license statements synced (commit `0b37aab73`)-- **Recipient guard / social-messaging hardening** (~30 commits, dominant theme): X group DM recipient binding, message-history delivery proofs, LinkedIn post composer allowance + localized controls, generic-first recipient guard on any site, fail-open guard + user-granted send authorization, silent-reply recovery, screenshot budget, shadow-host `aria-hidden` traversal fix in `_hasVisibleBox`-- **SafeSocial**: Instagram classifier + settings export (Chrome+Firefox), opt-in Jev classifiers with guarded browser decisions β v36.8.0-- **Jev fast-classifier system** (systemone): TypeSafe scheduler judge moved to Assistive Models, fast-flow screenshot handoffs, event-driven wakeups, malformed-response/reconciliation fixes-- **MCP server** (`mcp-server/`): stdio bridge so Claude Code/Codex/Cursor/OpenClaw can drive the running extension (`ws://127.0.0.1:17374`); made browser-optional and Chromium-focused-- **Firefox BiDi companion** (`firefox-companion/`): WebDriver BiDi native-messaging host giving Firefox trusted `isTrusted` input (clicks, typing, selects, uploads)-- **Providers**: dedicated DeepSeek provider; Unsloth Studio, Pollinations AI, NEAR AI Cloud added; WebGPU provider restored (Compass Tiny v2.1, MiniCPM5-2B, LFM2.5, Bonsai 27B, Nanbeige4.2-3B presets); managed default renamed WebBrain Cloud 1.0 β **WebBrain Compass**; configurable model limits-- **Offline RAG + Xapian**: offline retrieval/semantic/reranker/runtime, ZIM/Xapian index builder (`scripts/build-zim-xapian.mjs`, `zim-xapian.js`, `zim-xapian-runtime.js`), offline stopword handling-- **PDF viewer + OCR**: `pdf-extraction.js`, `pdf-ocr.js`, `pdf-stream.js`, PDF read/selection test suites, OPFS archiver worker-- **Theme/content**: Apocalypse Mode + Emergency Box, teacher mode, rich-text toolbar guard/probe, OTP skill-gated mailbox reader (`otp-email-tool.js`), drag-and-drop attachments, localized tool-action labels, URL redaction hardening-- **Site adapters**: +29 since 2026-08 β **138 total** (India, CIS/MENA, LATAM, East Asia/RU, Africa/MENA, EU/SEA bundles, baidu-tieba expansion)-- **Skills**: Humanizer and Phonr calls added to packaged defaults (now 12)-- Releases: v33.0.0 (2026-08-20) β¦ v35.0.0, v36.0.0 (2026-09-09) β¦ **v36.8.0 (2026-09-20)**, current dev version 36.8.0--### Known Firefox Parity Gaps--- `upload_file` β available in Firefox (user-picker resolve path)-- `full_page_screenshot` β retired as an agent tool in both builds (`RETIRED_AGENT_TOOL_NAMES`); `/screenshot --full-page` unsupported in Firefox (no scroll-and-stitch fallback)-- `get_shadow_dom` β dispatched in Firefox with open roots only ("Closed shadow roots are not accessible in Firefox" per tool schema); `shadow_dom_query` β **not implemented** in Firefox (no tool def, no content dispatch); still referenced in dead sets (`COMPLETION_DOCUMENT_OBSERVATION_TOOLS`, `WORKFLOW_CONTENT_READ_TOOLS`). `firefox-companion/` BiDi layer mitigates other CDP gaps.--### Chrome/Firefox Source Drift (verified 2026-09-22)--- **Chrome-only agent files:** `offline-answer-copy.js`, `offline-retrieval-offscreen.js`, `pdf-extraction.js`, `transcribe.js` (Chrome agent dir 80 files vs Firefox 77)-- **Firefox-only agent files:** `smd-loader.js`-- **Chrome-only infra:** `cdp/`, `offscreen/*`, `cloud-runs.js`, `recorder/host.js`, `providers/webgpu.js`-- **Firefox-only:** `bidi/` (bind.js, client.js), `background.html`, `firefox-restricted-domains.js`, `shortcut-command.js`, `watch-alert.js`, `content/file-picker-guard-loader.js`-- **Size drift:** `agent.js` Chrome 44,081 vs Firefox 36,714 (+20%), `content.js` Chrome 8,345 vs Firefox 7,291, `oauth-claude.js` Chrome +35%--### Prompt System TODO--- Compact-vs-full prompt contradiction is **partially resolved** (compact routing is per-provider opt-in); remaining work is prompt quality / model-tier selection-- Proposed: three-tier system (frontier/mid/small) instead of binary-- See `TODOs.md` item #1 for detailed analysisdiff --git a/.claude/setup.md b/.claude/setup.mddeleted file mode 100644index ccd53f483..000000000--- a/.claude/setup.md+++ /dev/null@@ -1,305 +0,0 @@-# WebBrain Setup & Run Flow--## Prerequisites--- **Node.js** >= 18 (for tests, scripts, web build, lmstudio-plugin)-- **Chrome** or **Firefox** browser-- **LLM access** (one of):- - Local: llama.cpp, Ollama, LM Studio, Jan, vLLM, SGLang, LocalAI, or GPT4All- - Cloud: API key for OpenAI, Anthropic, Google, OpenRouter, etc.- - Or use WebBrain Compass (managed cloud, no setup needed β the default provider)-------## 1. Clone the Repository--```bash-git clone https://github.com/webbrain-one/webbrain.git-cd webbrain-```--## 2. Install Dependencies--```bash-npm install-```--This installs `playwright` (dev dependency for tests). The extension itself has zero dependencies.--For the LM Studio plugin (optional):--```bash-cd lmstudio-plugin-npm install-cd ..-```-------## 3. Load the Extension--### Chrome (Recommended)--1. Open Chrome β `chrome://extensions/`-2. Enable **Developer mode** (top right toggle)-3. Click **Load unpacked**-4. Select the `src/chrome` folder from the cloned repo-5. The WebBrain icon appears in the toolbar--### Firefox--1. Open Firefox β `about:debugging#/runtime/this-firefox`-2. Click **Load Temporary Add-on**-3. Navigate to `src/firefox/` and select `manifest.json`-4. The WebBrain icon appears in the toolbar--> Note: Firefox temporary add-ons are removed on restart. For permanent installation, the extension needs signing via addons.mozilla.org.-------## 4. Set Up an LLM Provider--### Option A: WebBrain Cloud (No Setup)--- Default provider, no API key needed-- Works immediately after loading the extension--### Option B: Local Model--Start your local LLM server:--```bash-# llama.cpp-llama-server -m your-model.gguf --port 8080--# Ollama-ollama serve-# Then in WebBrain settings, set base URL to http://localhost:11434/v1--# LM Studio-# Start LM Studio's local API server (default port 1234)--# Jan-# Start Jan's local API server (default port 1337)--# GPT4All-# Start GPT4All's local server (OpenAI-compatible)--# vLLM-vllm serve your-model --port 8000--# SGLang-python -m sglang.launch_server --model-path your-model --port 30000-```--Then in WebBrain settings:--1. Click the gear icon (or extension Options page)-2. Select your provider (llama.cpp, Ollama, etc.)-3. The base URL is pre-filled for known providers-4. Click **Test Connection** to verify--> Context window: Use a model with at least 16k tokens for reliable agent runs. 8k works with Compact mode. 4k is too small.--### Option C: Cloud Provider--1. Click the gear icon in WebBrain-2. Select your provider (OpenAI, Anthropic, etc.)-3. Enter your API key-4. Select a model-5. Click **Test Connection**-------## 5. Use WebBrain--1. Click the WebBrain icon β the side panel opens-2. Type a message like:- - "Summarize this page"- - "Find all links about pricing"- - "Fill in the search box with 'AI agents' and click Search"- - "Navigate to github.com and find trending repositories"--### Conversation Modes--- **Ask** (default) β Read-only, safe by default-- **Act** β Browser actions (click, type, navigate)-- **Dev** β Page debugging and HTML/CSS inspection--### Slash Commands--Type `/` in the input to see available commands:--- `/help` β Show all commands-- `/ask` β Switch to Ask mode-- `/act` β Switch to Act mode-- `/dev` β Switch to Dev mode-- `/compact` β Force context compaction-- `/reset` β Clear conversation-- `/memory` β Manage user memory-- `/export` β Download conversation--### Keyboard Shortcuts--- `Ctrl+/` or `Cmd+/` β Focus input-- `Ctrl+Shift+A` or `Cmd+Shift+A` β Ask mode-- `Ctrl+Shift+X` or `Cmd+Shift+X` β Act mode-- `Ctrl+Shift+D` or `Cmd+Shift+D` β Dev mode-- `Escape` β Stop active run-------## 6. Run Tests--```bash-# Main regression suite (pure Node, no browser needed)-npm test--# Security/injection tests-npm run test:security--# Prompt injection benchmarks-npm run test:injection-bench--# Safety report-npm run test:safety-report--# Fixture tests-npm run test:fixtures--# Anonymous usage tests-npm run test:anonymous--# Syntax-only check on a specific file-node --check src/chrome/src/agent/agent.js-```--The test runner (`test/run.js`) imports Chrome and Firefox modules directly under Node (no chrome.\* APIs needed for most tests) and verifies parity between browsers.-------## 7. Build the Marketing Site--After editing files in `web/`:--```bash-# Build everything (landing page + blog + docs)-npm run build:web--# Build landing page only-npm run build:web:landing--# Build blog only-npm run build:blog--# Build docs only-npm run build:docs-```--The site generator reads `web/build/template.html` + `web/build/locales/*.json` and writes generated HTML files. **Do not hand-edit** `web/index.html` or `web/*/index.html` directly β the next build will overwrite them.-------## 8. Build the LM Studio Plugin (Optional)--```bash-cd lmstudio-plugin-npm install-npm run build # Compiles TypeScript to dist/-npm run dev # Watch mode-cd ..-```-------## 9. Release Process--```bash-# Bump version in package.json, manifest.json, etc.-npm run bump--# Build release zip for Chrome Web Store-npm run build:zip--# Bump + release-npm run release-```-------## 10. Development Workflow--### Making Changes--1. Edit files in `src/chrome/` or `src/firefox/`-2. Mirror changes across browsers unless platform makes parity impossible-3. Reload the extension in the browser:- - Chrome: `chrome://extensions/` β click refresh icon on WebBrain card- - Firefox: `about:debugging` β click "Reload"-4. Test manually in the browser--### Adding a New Tool--1. Read `docs/adding-a-tool.md` for the checklist-2. Add tool schema in `src/*/src/agent/tools.js`-3. Add implementation in `src/*/src/agent/agent.js` or content script-4. Mirror across Chrome and Firefox-5. Add tests in `test/run.js`-6. Run `npm test` to verify--### Adding a New Provider--1. Create a new class extending `BaseLLMProvider` in `src/*/src/providers/`-2. Implement `chat()` and optionally `chatStream()`-3. Register it in `src/*/src/providers/manager.js`-4. Mirror across Chrome and Firefox--### Adding a Site Adapter--1. Read `docs/site-adapters.md` for the guide-2. Add adapter in `src/*/src/agent/adapters.js`-3. Test on the target site-------## Quick Reference--| Task | Command |-| ---------------------- | ----------------------------------------------------------------------- |-| Load Chrome extension | `chrome://extensions/` β Load unpacked β `src/chrome` |-| Load Firefox extension | `about:debugging` β Load Temporary Add-on β `src/firefox/manifest.json` |-| Run tests | `npm test` |-| Build web | `npm run build:web` |-| Build LM Studio plugin | `cd lmstudio-plugin && npm run build` |-| Build release zip | `npm run build:zip` |-| Bump version | `npm run bump` |-| Syntax check | `node --check <file>` |-------## Troubleshooting--### Extension won't load--- Ensure Developer mode is enabled (Chrome)-- Check browser console for errors-- Verify you selected the correct folder (`src/chrome` or `src/firefox`)--### Local model not connecting--- Ensure the local server is running-- Check the port matches (default ports listed above)-- Click "Test Connection" in WebBrain settings-- Verify CORS settings if using Firefox (local providers need CORS headers)--### Tests failing--- Ensure Node.js >= 18 is installed-- Run `npm install` first-- Check if the failing test is browser-specific (Chrome-only or Firefox-only)--### Web build not working--- Ensure you edited files in `web/build/` (not `web/index.html` directly)-- Run `npm run build:web` after changes-- Check `web/build/locales/*.json` for translation issuesdiff --git a/.gitignore b/.gitignoreindex 70eb110cc..1d7725c51 100644--- a/.gitignore+++ b/.gitignore@@ -1,4 +1,4 @@-# .claude/+.claude/.DS_Storenode_modules/test/anonymous/.test-profile/