Agent Zero AI framework
Slim browser prompt with progressive skills
Alessandroc56b65803bb0
diff --git a/plugins/_browser/AGENTS.md b/plugins/_browser/AGENTS.mdindex 96d9f905c..515ff382c 100644--- a/plugins/_browser/AGENTS.md+++ b/plugins/_browser/AGENTS.md@@ -22,10 +22,13 @@## Work Guidance- Coordinate tool, helper, and panel changes so browser state shown in the UI matches tool behavior.+- Keep `prompts/agent.system.tool.browser.md` as a compact callable contract; move detailed browser workflows into `skills/browser-automation/SKILL.md`.+- Keep fragile form guidance progressively disclosed through `skills/browser-form-workflows/SKILL.md`, linked from the browser prompt through `browser-automation`.## Verification- Smoke-test browser launch, navigation, DOM capture, and WebUI viewer after runtime changes.+- Run browser prompt/skill regression tests after changing browser prompt or Browser plugin skills.## Child DOX Indexdiff --git a/plugins/_browser/prompts/agent.system.tool.browser.md b/plugins/_browser/prompts/agent.system.tool.browser.mdindex 4ecfd4660..2f1323281 100644--- a/plugins/_browser/prompts/agent.system.tool.browser.md+++ b/plugins/_browser/prompts/agent.system.tool.browser.md@@ -3,36 +3,15 @@ Rendered browser automation for pages that need interaction, JavaScript, forms,Prefer `search_engine` or `document_query` for plain text research. The tool must not open a Browser surface automatically. Use the tool headlessly unless the user opens the Browser surface or asks for the optional visible WebUI viewer.-The browser may run in Docker container mode or A0 CLI host-browser mode depending on settings. Container-mode paths resolve inside Agent Zero; host-mode paths resolve on the connected A0 CLI host.+When the user asks for "my browser", "host browser", "local browser", local Chrome, or opening a URL in their host browser, use this `browser` tool. Do not substitute `computer_use_remote`, `code_execution_remote`, `xdg-open`, `sensible-browser`, or Python `webbrowser.open`. If setup fails and mentions remote debugging, tell the user to open `chrome://inspect/#remote-debugging`, enable "Allow remote debugging for this browser instance", run `/browser host on`, and retry.-When the user asks for "my browser", "host browser", "local browser", local Chrome, or opening a URL in their host browser, use this `browser` tool. Do not substitute `computer_use_remote`, `code_execution_remote`, or host shell launchers such as `xdg-open`, `sensible-browser`, or Python `webbrowser.open`. If host-browser setup fails and mentions remote debugging, stop and tell the user to open `chrome://inspect/#remote-debugging`, enable "Allow remote debugging for this browser instance", run `/browser host on`, and retry.+For rendered browsing workflows, multi-step interaction, screenshots, downloads, uploads, forms, or host/container mode decisions, first load `browser-automation` with `skills_tool:load`, then call this tool using the loaded instructions. For fragile forms, `browser-automation` links to `browser-form-workflows`; load it when selects, checkboxes, radios, uploads, contenteditable fields, validation, or submission state are central.-For complex browser workflows, load skill `browser-automation`. For fragile forms, load skill `browser-form-workflows`.+Actions: tabs `open`, `list`, `state`, `set_active`, `navigate`, `back`, `forward`, `reload`, `close`, `close_all`; inspect `content`, `detail`, `screenshot`; interact `click`, `hover`, `double_click`, `right_click`, `drag`, `type`, `submit`, `type_submit`, `scroll`, `select_option`, `set_checked`, `upload_file`; advanced `evaluate`, `key_chord`, `mouse`, `wheel`, `keyboard`, `clipboard`, `set_viewport`, `multi`.-Actions: `open`, `list`, `state`, `set_active`, `navigate`, `back`, `forward`, `reload`, `content`, `detail`, `screenshot`, `click`, `hover`, `double_click`, `right_click`, `drag`, `type`, `submit`, `type_submit`, `scroll`, `evaluate`, `key_chord`, `mouse`, `wheel`, `keyboard`, `clipboard`, `set_viewport`, `select_option`, `set_checked`, `upload_file`, `multi`, `close`, `close_all`.--Common args: `action`, `browser_id`, `url`, `ref`, `target_ref`, `text`, `selector`, `selectors`, `script`, `modifiers`, `keys`, `key`, `include_content`, `focus_popup`, `event_type`, `x`, `y`, `to_x`, `to_y`, `delta_x`, `delta_y`, `button`, `quality`, `full_page`, `path`, `paths`, `value`, `values`, `checked`, `width`, `height`, `calls`.--Workflow:-- `open` creates a tab and returns id/state. In host-browser mode, if the requested URL is already open, the host may reuse and activate that existing tab.-- If the user asks for an existing tab, page title, or already-open URL, call `list` first, match by `title` or `currentUrl`, then use `set_active` or `navigate` on that `browser_id` instead of opening a new tab.-- `content` returns markdown with refs like `[link 3]`, `[button 6]`, `[input text 8]`.-- Interactions use refs from the latest `content` capture.-- For same-page controls that are easier to identify structurally, `click`, `type`, `submit`, `type_submit`, `scroll`, `select_option`, `set_checked`, and `upload_file` may use `selector` instead of `ref`; the tool resolves the selector through `content` first.-- `click` with `x`/`y` and no `ref` is treated as a coordinate mouse click. `type` with text and no `ref` types into the currently focused element. `key_chord` accepts either `["Control", "A"]` or `"CTRL+A"`.-- `navigate` reuses an existing `browser_id` and is preferred for serial browsing.-- Screenshots are explicit only; the browser does not automatically load screenshots. Call `vision_load` with the returned `vision_load.tool_args.paths` value before reasoning visually. When no `path` is requested, browser screenshots are saved as chat-scoped artifacts; explicit `path` requests remain user-owned files.+Rules:+- If the user asks for an existing tab, page title, or already-open URL, call `list` first, match by `title` or `currentUrl`, then use `set_active` or `navigate` on that `browser_id`.+- Use refs from the latest `content`; same-page controls may use `selector`.+- Screenshots are explicit only; the browser does not automatically load screenshots. Call `vision_load` with the returned `vision_load.tool_args.paths` value before reasoning visually.- Keep the tab set small; close pages after extracting what you need.--`multi` is only a browser action: use `tool_name: "browser"` with `tool_args.action: "multi"`. Never use `tool_name: "multi"`.--Example:-~~~json-{- "tool_name": "browser",- "tool_args": {- "action": "open",- "url": "https://example.com"- }-}-~~~+- `multi` is only a browser action: use `tool_name: "browser"` with `tool_args.action: "multi"`. Never use `tool_name: "multi"`.diff --git a/plugins/_browser/skills/browser-automation/SKILL.md b/plugins/_browser/skills/browser-automation/SKILL.mdindex b79d0e42b..96a2cf590 100644--- a/plugins/_browser/skills/browser-automation/SKILL.md+++ b/plugins/_browser/skills/browser-automation/SKILL.md@@ -5,7 +5,9 @@ description: Use for complex Agent Zero browser automation, including multi-tab# Browser Tool-Use the `browser` tool for rendered pages, forms, logins, downloads, JavaScript-heavy sites, screenshots, and visual inspection. Prefer `search_engine` or `document_query` for plain text research.+Use this skill after the compact `browser` tool prompt points you here. It is the progressive-disclosure workflow guide for rendered pages, multi-step browser work, logins, downloads, JavaScript-heavy sites, screenshots, host/container browser mode, and visual inspection. Prefer `search_engine` or `document_query` for plain text research.++For fragile forms, load `browser-form-workflows` with `skills_tool:load` before acting when selects, checkboxes, radios, file uploads, contenteditable fields, validation, or final submission state are central to the task.## Core Workflow@@ -14,6 +16,7 @@ Use the `browser` tool for rendered pages, forms, logins, downloads, JavaScript-3. Interact with refs using `click`, `type`, `submit`, `scroll`, etc.4. Use `navigate` on an existing `browser_id` for serial browsing.5. Keep only a small working tab set; close pages when finished.+6. If the user asks for an existing tab, page title, or already-open URL, call `list` first, match by `title` or `currentUrl`, then use `set_active` or `navigate` on that `browser_id` instead of opening a new tab.## Modes@@ -39,7 +42,7 @@ Screenshot args include `quality`, `full_page`, and optional `path`. Without `pa- `select_option` works for native selects and detectable ARIA listbox/combobox controls.- `set_checked` works for checkbox, radio, switch, and toggle-like refs.- `upload_file` works for file input refs or associated labels; verify the file exists in the active browser environment.-- For fragile forms, load skill `browser-form-workflows`.+- For fragile forms, call `skills_tool` with `action: "load"` and `skill_name: "browser-form-workflows"`, then follow that form-specific workflow before filling or submitting.## Pointer And Keyboarddiff --git a/plugins/_browser/skills/browser-form-workflows/SKILL.md b/plugins/_browser/skills/browser-form-workflows/SKILL.mdindex 4dffd483d..879902f09 100644--- a/plugins/_browser/skills/browser-form-workflows/SKILL.md+++ b/plugins/_browser/skills/browser-form-workflows/SKILL.md@@ -5,7 +5,7 @@ description: Use for complex Agent Zero Browser form workflows involving selects# Browser Forms-Use this skill for complex Browser form workflows where the page state may depend on selects, checkboxes, radios, file uploads, contenteditable fields, validation, or visual confirmation.+Use this skill as the form-specific extension of `browser-automation`. Load it after, or alongside, `browser-automation` when the page state may depend on selects, checkboxes, radios, file uploads, contenteditable fields, validation, or visual confirmation.Start with `browser:content` to capture current refs, then use `browser:detail` on ambiguous fields before acting. Prefer ref-based form actions before coordinates.diff --git a/plugins/_browser/webui/config.html b/plugins/_browser/webui/config.htmlindex 06a711046..bcbaa2b22 100644--- a/plugins/_browser/webui/config.html+++ b/plugins/_browser/webui/config.html@@ -37,7 +37,7 @@class="browser-config-field-help"x-show="$store.browserConfig.config.runtime_backend === 'host_required'">- Uses Chrome, Edge, or Chromium on the A0 CLI host. Keep A0 CLI connected; the browser opens when the agent first needs it.+ Uses Chrome, Edge, or Chromium on the A0 CLI host. Keep A0 CLI connected; the browser opens when the agent first needs it. For an already-open browser, use chrome://inspect/#remote-debugging and enable "Allow remote debugging for this browser instance".</span></label>diff --git a/tests/test_browser_agent_regressions.py b/tests/test_browser_agent_regressions.pyindex 160e1bde8..16dd6857b 100644--- a/tests/test_browser_agent_regressions.py+++ b/tests/test_browser_agent_regressions.py@@ -896,6 +896,8 @@ def test_browser_tool_does_not_auto_open_canvas_policy_is_documented():prompt = (PROJECT_ROOT / "plugins" / "_browser" / "prompts" / "agent.system.tool.browser.md").read_text(encoding="utf-8")+ from helpers import tokens+config = (PROJECT_ROOT / "plugins" / "_browser" / "default_config.yaml").read_text(encoding="utf-8")@@ -912,21 +914,33 @@ def test_browser_tool_does_not_auto_open_canvas_policy_is_documented():assert "set_checked" in promptassert "upload_file" in promptassert "browser-form-workflows" in prompt+ assert "first load `browser-automation` with `skills_tool:load`" in prompt+ assert "`browser-automation` links to `browser-form-workflows`" in promptassert "does not automatically load screenshots" in promptassert "chrome://inspect/#remote-debugging" in prompt+ assert tokens.approximate_tokens(prompt) <= 650assert "already open" in configassert "already-open Browser surface" in config_htmlassert "chrome://inspect/#remote-debugging" in config_html-def test_browser_forms_skill_is_plugin_owned_and_discoverable():+def test_browser_skills_are_plugin_owned_and_progressively_linked():+ automation_path = (+ PROJECT_ROOT / "plugins" / "_browser" / "skills" / "browser-automation" / "SKILL.md"+ )skill_path = PROJECT_ROOT / "plugins" / "_browser" / "skills" / "browser-form-workflows" / "SKILL.md"+ assert automation_path.exists()assert skill_path.exists()+ automation = automation_path.read_text(encoding="utf-8")skill = skill_path.read_text(encoding="utf-8")assert skill.startswith("---\n")frontmatter = skill.split("---", 2)[1]assert "name: browser-form-workflows" in frontmatterassert "description:" in frontmatter+ assert "progressive-disclosure workflow guide" in automation+ assert "`browser-form-workflows` with `skills_tool:load`" in automation+ assert 'skill_name: "browser-form-workflows"' in automation+ assert "form-specific extension of `browser-automation`" in skillassert "select_option" in skillassert "set_checked" in skillassert "upload_file" in skill