An agentic browser harness: an AI side panel & orb in your own logged-in Chrome, Edge, Arc, Brave or even Firefox run by the agent/subscription you already use (Claude Code, Codex, Antigravity). Free, open source, no API key. Also an MCP server.
TypeScript agentic-browseragentic-browsingagy-cliaiai-agentai-browser-extensionai-toolsbrave-extensionbrowser-agentbrowser-automationbrowser-extensionbrowser-usebrowserskillchrome-extensionclaude-codecodexfirefox-addonmcp-servernanobrowserweb-automation
feat(tools): less common actions belong to a group, browsentic_openSkill opens a group's guide with its skill and the schemas of the tools this caller is offered, and browsentic_use runs a page action by name through the same path as its own tool
Shahriarc8e7c07811f6
diff --git a/docs/guide/mcp-clients.md b/docs/guide/mcp-clients.mdindex 77e7895..9aee521 100644--- a/docs/guide/mcp-clients.md+++ b/docs/guide/mcp-clients.md@@ -49,6 +49,9 @@ surprise here.- **`browsentic_status`**: whether the extension is connected, its version, the active tab, anyrunning monitors, and a `hint` naming the fix when something is wrong. Call it first when a pagetool fails.+- **`browsentic_openSkill`**: the guide to a less common capability (captcha, theming,+ diagnostics, monitoring, downloads, drag) and the tools for it. The server's instructions list+ the guides, so the agent knows what to open.- **Three read-only resources**, which return page context without spending a tool call:| Resource | Use when |diff --git a/docs/internals/registry.md b/docs/internals/registry.mdindex c480752..9b1aba7 100644--- a/docs/internals/registry.md+++ b/docs/internals/registry.md@@ -24,6 +24,12 @@ export const clickElement = defineAction({});```+An action that only some tasks need also names its `group`: `captcha`, `theming`, `diagnostics`,+`monitoring`, `downloads` or `drag`. [`src/lib/actions/groups.ts`](../../src/lib/actions/groups.ts)+gives each group its line in the index and the skills that teach it, and `browsentic_openSkill`+opens a group as one guide. Ungrouped actions are the core. The group never leaves the daemon's+own copy of the registry, so it changes neither the descriptors nor the manifest hash.+`describeActions()` turns that array into tool descriptors by converting each zod schema to JSONSchema. The daemon serves those descriptors to MCP clients, so **a tool cannot describe a capabilitythe browser does not have**: both are generated from the same source.@@ -75,7 +81,9 @@ ordinary MCP client. `browsentic_saveSiteMap` is published as a tool **only** tospawned inside an agent run.`browsentic_status` is not in the registry either: the MCP server answers it directly by combining-daemon state with a one-element `page.getPageInfo` and the current monitor list.+daemon state with a one-element `page.getPageInfo` and the current monitor list. Nor is+`browsentic_openSkill`, which the MCP server answers from the groups, the skills in effect (a+user's own copy of a skill wins) and the tools this caller is offered.---diff --git a/docs/reference/tools.md b/docs/reference/tools.mdindex 0992d32..d738fc4 100644--- a/docs/reference/tools.md+++ b/docs/reference/tools.md@@ -15,14 +15,14 @@ human-readable copy; `yarn daemon:manifest` prints the machine listing (see**Names.** An action is namespaced with dots, a tool with underscores: action `page.fillInput` ispublished as tool `page_fillInput`. The mapping is mechanical, so this page names each tool onceand the action is implied. Actions under the reserved `browsentic.` prefix are daemon verbs, not-registry actions. Of those, only `browsentic_status` (always) and `browsentic_saveSiteMap`-(mapping runs only) become tools.+registry actions. Of those, `browsentic_status` and `browsentic_openSkill` are always tools, and+`browsentic_saveSiteMap` becomes one in mapping runs only.Tools by group:| Group | Tools || --- | --- |-| [Status](#status) | `browsentic_status` |+| [Status and guides](#status-and-guides) | `browsentic_status`, `browsentic_openSkill` || [Reading](#reading) | `page_getPageInfo`, `page_extractText`, `page_waitForElement`, `page_findProgress`, `page_findSearch`, `page_pickElement`, `page_screenshot` || [Acting](#acting) | `page_clickElement`, `page_trustedClick`, `page_findCaptcha`, `page_solveCaptcha`, `page_hoverElement`, `page_dragElement`, `page_focusInput`, `page_fillInput`, `page_typeText`, `page_selectOption`, `page_selectText`, `page_pressKey`, `page_submitForm`, `page_highlightElement` || [Scripting](#scripting) | `page_injectCode`, `page_runCode` |@@ -61,7 +61,7 @@ Below, a parameter typed *[target](#element-targets)* is exactly this object.----## Status+## Status and guides### browsentic_status@@ -69,6 +69,19 @@ Report whether the extension is connected, its version, the active tab, any runna `hint` naming the fix when something is wrong. Call it first when a page tool fails. Noparameters.+### browsentic_openSkill++Open the guide to a less common capability: the skill that teaches it, then each of its tools with+its description and input schema. The guides are `captcha`, `theming`, `diagnostics`, `monitoring`+(monitors and timers), `downloads` and `drag`. Each has a one-line entry in the tool's description+and in the server's instructions, so an agent can tell which one the page calls for. A guide lists+only the tools this caller is offered and names the rest; one with nothing to offer, such as+`captcha` in Firefox, is refused with `UNAVAILABLE`.++| Parameter | Type | Default | Purpose |+| --- | --- | --- | --- |+| `name` | string | required | The guide to open. The name of its skill (`page-theming`) or of one of its tools (`page_applyTheme`) opens it too |+---## Readingdiff --git a/src/daemon/server.test.ts b/src/daemon/server.test.tsindex 5eb6828..f956368 100644--- a/src/daemon/server.test.ts+++ b/src/daemon/server.test.ts@@ -1,4 +1,5 @@-import { rmSync, writeFileSync } from 'node:fs';+import { mkdirSync, rmSync, writeFileSync } from 'node:fs';+import { join } from 'node:path';import { Client } from '@modelcontextprotocol/sdk/client/index.js';import { InMemoryTransport } from '@modelcontextprotocol/sdk/inMemory.js';import { ToolListChangedNotificationSchema } from '@modelcontextprotocol/sdk/types.js';@@ -11,6 +12,7 @@ import { toolNameFor } from '@/lib/actions/tool-names';import { configPath } from './agent/config';import type { Bridge, BridgeStatus } from './control';import { FENCE_NOTE, IMAGE_NOTE } from './guardrails';+import { stateDir } from './lockfile';import { createMcpServer } from './server';const KEY = 'sk-ant-api03-AbCdEfGhIjKlMnOpQrStUvWxYz012345';@@ -64,26 +66,55 @@ afterEach(() => {});describe('the tool list', () => {- test('mirrors the action registry, plus the status tool', async () => {+ test('mirrors the action registry, plus the status tool and the guide opener', async () => {const { tools } = await (await connect(fakeBridge().bridge)).listTools();expect(tools.map(({ name, description, inputSchema }) => ({ name, description, inputSchema }))).toEqual([...describeActions().map((action) => ({ name: toolNameFor(action.name), description: action.description, inputSchema: action.inputSchema })),expect.objectContaining({ name: 'browsentic_status' }),+ expect.objectContaining({ name: 'browsentic_openSkill' }),]);});+ test('the guide opener names every guide, in the words of the need, with a schema that takes only those names', async () => {+ const { tools } = await (await connect(fakeBridge().bridge)).listTools();+ const opener = tools.find((tool) => tool.name === 'browsentic_openSkill')!;+ expect({+ lines: opener.description!.split('\n').slice(1).map((line) => line.split(':')[0]),+ captcha: opener.description!.includes('- captcha: get past a “verify you are human” check'),+ names: (opener.inputSchema.properties as { name: { enum: string[] } }).name.enum,+ }).toEqual({+ lines: ['- captcha', '- theming', '- diagnostics', '- monitoring', '- downloads', '- drag'],+ captcha: true,+ names: ['captcha', 'theming', 'diagnostics', 'monitoring', 'downloads', 'drag'],+ });+ });++ test('the server instructions carry the same index, well inside what a client keeps of them', async () => {+ const instructions = (await connect(fakeBridge().bridge)).getInstructions() ?? '';+ expect({+ opener: instructions.includes('opened with browsentic_openSkill'),+ chromium: instructions.includes('- diagnostics: read the console errors, exceptions and failed requests behind a page that misbehaves (Chromium browsers only)'),+ kept: instructions.length < 4_096,+ }).toEqual({ opener: true, chromium: true, kept: true });+ });+test('a reserved tool is listed only when the daemon offers it to this caller', async () => {const offered = (reserved: string[]) => fakeBridge({}, {}, describeActions(), reserved).bridge;const names = async (bridge: Bridge) => (await (await connect(bridge, { agentRun: true })).listTools()).tools.slice(-2).map((tool) => tool.name);expect([await names(offered([FOCUS_SHOT_ACTION])), await names(offered([]))]).toEqual([- ['browsentic_status', toolNameFor(FOCUS_SHOT_ACTION)],- [toolNameFor(describeActions().at(-1)!.name), 'browsentic_status'],+ ['browsentic_openSkill', toolNameFor(FOCUS_SHOT_ACTION)],+ ['browsentic_status', 'browsentic_openSkill'],]);});test('a daemon too old to say what it offers leaves an agent run with both reserved tools', async () => {const { tools } = await (await connect(fakeBridge().bridge, { agentRun: true })).listTools();- expect(tools.slice(-3).map((tool) => tool.name)).toEqual(['browsentic_status', toolNameFor(SAVE_SITE_MAP_ACTION), toolNameFor(FOCUS_SHOT_ACTION)]);+ expect(tools.slice(-4).map((tool) => tool.name)).toEqual([+ 'browsentic_status',+ 'browsentic_openSkill',+ toolNameFor(SAVE_SITE_MAP_ACTION),+ toolNameFor(FOCUS_SHOT_ACTION),+ ]);});test('an extension that describes an action no MCP client could call is refused, not passed on', async () => {@@ -285,6 +316,147 @@ describe('a picked element', () => {});});+describe('a guide', () => {+ const open = async (name: unknown, tools?: ToolDescriptor[]) => call(await connect(fakeBridge({}, {}, tools).bridge), 'browsentic_openSkill', { name });++ test("opens with its skill, then each of its tools with the description and schema it is listed with", async () => {+ const [text] = texts(await open('drag'));+ const drag = describeActions().find((action) => action.name === 'page.dragElement')!;+ expect(text).toBe(+ `# The drag guide\n\n## Its tools\n\nCall them by name, as you would any other tool.\n\n` ++ `### page_dragElement\n${drag.description}\nInput schema: ${JSON.stringify(drag.inputSchema)}`,+ );+ });++ test('monitoring teaches watching and timers, in that order, and lists all eight tools', async () => {+ const [text] = texts(await open('monitoring'));+ expect({+ order: text.indexOf('You are setting up a watch') < text.indexOf('The extension keeps the clock'),+ tools: [...text.matchAll(/^### (\S+)$/gm)].map((match) => match[1]),+ }).toEqual({+ order: true,+ tools: ['page_findProgress', 'page_startMonitor', 'page_monitorStatus', 'page_awaitMonitor', 'page_stopMonitor', 'page_startTimer', 'page_timerStatus', 'page_stopTimer'],+ });+ });++ test('a skill the user rewrote is the one the guide opens with', async () => {+ mkdirSync(join(stateDir, 'skills'), { recursive: true });+ writeFileSync(join(stateDir, 'skills', 'captcha.md'), '---\nname: captcha\ndescription: mine\n---\nAsk me before any captcha.\n');+ try {+ const [text] = texts(await open('captcha'));+ expect(text.startsWith('# The captcha guide\n\nAsk me before any captcha.\n\n## Its tools')).toBe(true);+ } finally {+ rmSync(join(stateDir, 'skills'), { recursive: true, force: true });+ }+ });++ test('opens by its skill or any of its tools as well as by its own name', async () => {+ const heading = async (name: string) => texts(await open(name))[0].split('\n')[0];+ expect(await Promise.all(['theming', 'page-theming', 'page_applyTheme', 'page.readTheme', 'auditContrast', 'scheduled-jobs'].map(heading))).toEqual([+ '# The theming guide',+ '# The theming guide',+ '# The theming guide',+ '# The theming guide',+ '# The theming guide',+ '# The monitoring guide',+ ]);+ });++ test('names what this run is not offered, and refuses a guide with nothing to offer', async () => {+ const phone = describeActions().filter((action) => ['page.findProgress', 'page.startTimer', 'page.timerStatus', 'page.stopTimer'].includes(action.name));+ const monitoring = await open('monitoring', phone);+ const firefox = await open('captcha', describeActions('firefox'));+ expect({+ offered: [...texts(monitoring)[0].matchAll(/^### (\S+)$/gm)].map((match) => match[1]),+ missing: texts(monitoring)[0].endsWith('Not offered in this run: page_startMonitor, page_monitorStatus, page_awaitMonitor, page_stopMonitor.'),+ firefox: [firefox.isError, texts(firefox)[0]],+ }).toEqual({+ offered: ['page_findProgress', 'page_startTimer', 'page_timerStatus', 'page_stopTimer'],+ missing: true,+ firefox: [true, 'UNAVAILABLE: None of the captcha tools are offered in this run; they run only in a Chromium browser such as Chrome or Edge, on a computer.'],+ });+ });++ test('an unknown name is answered with every guide there is', async () => {+ const reply = await open('sliders');+ expect([reply.isError, texts(reply)[0].split('\n').slice(0, 2)]).toEqual([+ true,+ ['UNKNOWN_SKILL: There is no guide named "sliders". The guides are:', '- captcha: get past a “verify you are human” check: reCAPTCHA, hCaptcha, Turnstile (Chromium browsers only)'],+ ]);+ });+});++describe('a call through browsentic_use', () => {+ test('reaches the bridge exactly as the listed tool would, and reads back the same', async () => {+ const answers = { 'page.dragElement': success({ dropped: true }) };+ const direct = fakeBridge(answers);+ const through = fakeBridge(answers);+ const input = { from: { text: 'Card' }, to: { text: 'Done' } };+ const replies = [+ await call(await connect(direct.bridge), 'page_dragElement', input),+ await call(await connect(through.bridge), 'browsentic_use', { action: 'page_dragElement', input }),+ ];+ expect([through.asked, replies[1].content.length, texts(replies[1])[0].replace(/[0-9a-f]{8,}/g, '')]).toEqual([+ direct.asked,+ replies[0].content.length,+ texts(replies[0])[0].replace(/[0-9a-f]{8,}/g, ''),+ ]);+ });++ test('takes the action as its tool, its action or bare, and the input as an object, a JSON string or flat', async () => {+ const { bridge, asked } = fakeBridge({ 'page.solveCaptcha': success({ state: 'solved' }) });+ const client = await connect(bridge);+ await call(client, 'browsentic_use', { action: 'page_solveCaptcha', input: { tiles: [1] } });+ await call(client, 'browsentic_use', { action: 'page.solveCaptcha', input: '{"tiles":[2]}' });+ await call(client, 'browsentic_use', { action: 'solveCaptcha', tiles: [3] });+ await call(client, 'browsentic_use', { action: 'solveCaptcha' });+ expect(asked).toEqual([+ ['page.solveCaptcha', { tiles: [1] }],+ ['page.solveCaptcha', { tiles: [2] }],+ ['page.solveCaptcha', { tiles: [3] }],+ ['page.solveCaptcha', {}],+ ]);+ });++ test("renders a result as its own tool does: an open captcha comes back as its picture", async () => {+ const challenge = { kind: 'tiles', prompt: 'Select all buses', image: PNG, imageWidth: 300, imageHeight: 300 };+ const client = await connect(fakeBridge({ 'page.solveCaptcha': success({ state: 'challenge', challenge }) }).bridge);+ const reply = await call(client, 'browsentic_use', { action: 'solveCaptcha' });+ expect(reply.content.map((part) => part.type)).toEqual(['text', 'image', 'text']);+ });++ test('invalid input comes back with the schema to correct it by', async () => {+ const client = await connect(fakeBridge({ 'page.dragElement': failure('INVALID_INPUT', '✖ Invalid input') }).bridge);+ const [text] = texts(await call(client, 'browsentic_use', { action: 'dragElement', input: { from: 3 } }));+ const schema = describeActions().find((action) => action.name === 'page.dragElement')!.inputSchema;+ expect(text).toBe(`INVALID_INPUT: ✖ Invalid input\npage_dragElement takes: ${JSON.stringify(schema)}`);+ });++ test('input that is not an object is refused before it reaches the browser', async () => {+ const { bridge, asked } = fakeBridge();+ const reply = await call(await connect(bridge), 'browsentic_use', { action: 'dragElement', input: '[1, 2' });+ expect([reply.isError, texts(reply)[0], asked]).toEqual([+ true,+ "INVALID_INPUT: input must be an object, or a JSON string of one, holding page_dragElement's parameters.",+ [],+ ]);+ });++ test('an action that does not exist, or is not a page action, is answered with the guides and never invoked', async () => {+ const { bridge, asked } = fakeBridge();+ const client = await connect(bridge);+ const replies = await Promise.all(['page_sliders', 'browsentic_status', 'browsentic_use'].map((action) => call(client, 'browsentic_use', { action })));+ expect([replies.map((reply) => texts(reply)[0].split('\n')[0]), asked]).toEqual([+ [+ 'UNKNOWN_ACTION: There is no browser action named "page_sliders". Open the guide that covers what you need with browsentic_openSkill:',+ 'UNKNOWN_ACTION: There is no browser action named "browsentic_status". Open the guide that covers what you need with browsentic_openSkill:',+ 'UNKNOWN_ACTION: There is no browser action named "browsentic_use". Open the guide that covers what you need with browsentic_openSkill:',+ ],+ [],+ ]);+ });+});+describe('the status tool', () => {const statusOf = async (answers: Record<string, ActionResult>, status: Partial<BridgeStatus> = {}) =>JSON.parse(texts(await call(await connect(fakeBridge(answers, status).bridge), 'browsentic_status'))[0]) as Record<string, unknown>;diff --git a/src/daemon/server.ts b/src/daemon/server.tsindex d86402f..b85bc2e 100644--- a/src/daemon/server.ts+++ b/src/daemon/server.ts@@ -10,6 +10,7 @@ import { readAgentConfig } from './agent/config';import type { Bridge } from './control';import { fence, fenceTag, policyFrom, sealSecrets } from './guardrails';import { log } from './log';+import { GROUPS_NOTE } from './tool-groups';import { toolHost } from './tool-host';const RESOURCES = [@@ -55,7 +56,8 @@ export function createMcpServer(bridge: Bridge, version: string, opts: { agentRu'⟦password:[email protected]⟧; the real value stays in the browser. Pass a placeholder through unchanged as ' +'page_fillInput’s value or page_typeText’s text and it becomes the credential at the moment it reaches the field. ' +'Anywhere else it is refused, and it is never yours to read, rebuild or repeat. ' +- 'SITE_BLOCKED means the user has put that site off-limits to Browsentic: do not retry it or reach it another way — tell the user.',+ 'SITE_BLOCKED means the user has put that site off-limits to Browsentic: do not retry it or reach it another way — tell the user.\n\n' ++ GROUPS_NOTE,},);diff --git a/src/daemon/tool-groups.test.ts b/src/daemon/tool-groups.test.tsnew file mode 100644index 0000000..b9e8266--- /dev/null+++ b/src/daemon/tool-groups.test.ts@@ -0,0 +1,35 @@+import { describe, expect, test } from 'vitest';+import { TOOL_GROUPS } from '@/lib/actions/groups';+import { actions } from '@/lib/actions/registry';+import { bundledSkillNames } from './agent/skills';++const grouped = Object.fromEntries(+ Object.keys(TOOL_GROUPS).map((group) => [group, [...actions.values()].filter((action) => action.group === group).map((action) => action.name)]),+);++describe('the tool groups', () => {+ test('hold the less common actions, and every other action stays in the core', () => {+ expect(grouped).toEqual({+ captcha: ['page.findCaptcha', 'page.solveCaptcha'],+ theming: ['page.readTheme', 'page.auditContrast', 'page.applyTheme'],+ diagnostics: ['page.startDiagnostics', 'page.readConsole', 'page.readNetwork', 'page.stopDiagnostics'],+ monitoring: [+ 'page.findProgress',+ 'page.startMonitor',+ 'page.monitorStatus',+ 'page.awaitMonitor',+ 'page.stopMonitor',+ 'page.startTimer',+ 'page.timerStatus',+ 'page.stopTimer',+ ],+ downloads: ['page.captureDownload', 'page.listDownloads'],+ drag: ['page.dragElement'],+ });+ });++ test('open with skills Browsentic bundles', () => {+ const skills = Object.values(TOOL_GROUPS).flatMap((group) => group.skills);+ expect(skills.filter((name) => !bundledSkillNames().includes(name))).toEqual([]);+ });+});diff --git a/src/daemon/tool-groups.ts b/src/daemon/tool-groups.tsnew file mode 100644index 0000000..844d46e--- /dev/null+++ b/src/daemon/tool-groups.ts@@ -0,0 +1,63 @@+import { TOOL_GROUPS, type ToolGroupName } from '@/lib/actions/groups';+import type { ToolDescriptor } from '@/lib/actions/manifest';+import { failure, success, type ActionResult } from '@/lib/actions/protocol';+import { actions } from '@/lib/actions/registry';+import { OPEN_SKILL_TOOL, toolNameFor } from '@/lib/actions/tool-names';+import type { Skill } from './agent/skills';++const GROUP_NAMES = Object.keys(TOOL_GROUPS) as ToolGroupName[];++const membersOf = (group: ToolGroupName) => [...actions.values()].filter((action) => action.group === group);++const chromiumOnly = (group: ToolGroupName) => membersOf(group).every((action) => action.chromiumOnly);++export const GROUP_INDEX = GROUP_NAMES.map(+ (group) => `- ${group}: ${TOOL_GROUPS[group].index}${chromiumOnly(group) ? ' (Chromium browsers only)' : ''}`,+).join('\n');++export const GROUPS_NOTE =+ `Less common capabilities each have a guide, opened with ${OPEN_SKILL_TOOL}: how to go about it, then its tools with their ` ++ `input schemas. Open one when the page calls for it:\n${GROUP_INDEX}`;++/** A page action the registry knows, named as an action, as its tool, or bare: page.dragElement, page_dragElement, dragElement. */+export function pageActionNamed(name: string): string | undefined {+ const action = `page.${name.trim().replace(/^page[._]/, '')}`;+ return actions.has(action) ? action : undefined;+}++/** A group named as itself, by one of its skills, or by one of its actions. */+export function groupNamed(name: string): ToolGroupName | undefined {+ const wanted = name.trim();+ const action = pageActionNamed(wanted);+ return GROUP_NAMES.find(+ (group) =>+ group === wanted.toLowerCase() ||+ (TOOL_GROUPS[group].skills as readonly string[]).includes(wanted) ||+ (action !== undefined && actions.get(action)?.group === group),+ );+}++/** The group's skills, then each of its tools this caller is offered, with the schema to call it by. */+export function groupGuide(group: ToolGroupName, offered: readonly ToolDescriptor[], skills: readonly Skill[]): ActionResult<string> {+ const members = membersOf(group).map((action) => action.name);+ const tools = offered.filter((tool) => members.includes(tool.name));+ const missing = members.filter((name) => !tools.some((tool) => tool.name === name)).map(toolNameFor);+ if (!tools.length) {+ return failure(+ 'UNAVAILABLE',+ `None of the ${group} tools are offered in this run${chromiumOnly(group) ? '; they run only in a Chromium browser such as Chrome or Edge, on a computer' : ''}.`,+ );+ }++ const bodies = TOOL_GROUPS[group].skills.flatMap((name) => skills.find((skill) => skill.name === name)?.body.trim() ?? []);+ return success(+ [+ `# The ${group} guide`,+ ...bodies,+ '## Its tools',+ 'Call them by name, as you would any other tool.',+ ...tools.map((tool) => `### ${toolNameFor(tool.name)}\n${tool.description}\nInput schema: ${JSON.stringify(tool.inputSchema)}`),+ ...(missing.length ? [`Not offered in this run: ${missing.join(', ')}.`] : []),+ ].join('\n\n'),+ );+}diff --git a/src/daemon/tool-host.ts b/src/daemon/tool-host.tsindex f80633c..7e3e283 100644--- a/src/daemon/tool-host.ts+++ b/src/daemon/tool-host.ts@@ -1,14 +1,32 @@+import { TOOL_GROUPS } from '@/lib/actions/groups';import type { ToolDescriptor } from '@/lib/actions/manifest';-import type { ActionResult } from '@/lib/actions/protocol';+import { failure, type ActionResult } from '@/lib/actions/protocol';import { FOCUS_SHOT_ACTION, RESERVED_ACTIONS, SAVE_SITE_MAP_ACTION } from '@/lib/actions/reserved';-import { actionNameFor, assertToolNamesRoundTrip, STATUS_TOOL, toolNameFor } from '@/lib/actions/tool-names';+import { actionNameFor, assertToolNamesRoundTrip, OPEN_SKILL_TOOL, STATUS_TOOL, toolNameFor, USE_TOOL } from '@/lib/actions/tool-names';+import { loadSkills } from './agent/skills';import type { Bridge } from './control';import { IMAGE_NOTE, fence, sealSecrets, shouldFence, type Policy } from './guardrails';+import { GROUP_INDEX, groupGuide, groupNamed, pageActionNamed } from './tool-groups';-const SCREENSHOT_TOOL = 'page_screenshot';-const PICK_TOOL = 'page_pickElement';-const CAPTCHA_TOOL = 'page_solveCaptcha';-const EXTRACT_TOOL = 'page_extractText';+const EXTRACT_ACTION = 'page.extractText';++const RENDERERS: Record<string, (result: ActionResult, fenceWith?: string) => ToolReply> = {+ 'page.screenshot': renderScreenshot,+ 'page.pickElement': renderPick,+ 'page.solveCaptcha': renderCaptcha,+ [FOCUS_SHOT_ACTION]: renderFocusShot,+};++const OPEN_SKILL = {+ name: OPEN_SKILL_TOOL,+ description: `Open the guide to one of these less common capabilities: how to go about it, then its tools with their input schemas.\n${GROUP_INDEX}`,+ inputSchema: {+ type: 'object' as const,+ properties: { name: { type: 'string', enum: Object.keys(TOOL_GROUPS), description: 'The capability whose guide to open.' } },+ required: ['name'],+ additionalProperties: false,+ },+};const FOCUS_SHOT_TOOL = {name: toolNameFor(FOCUS_SHOT_ACTION),@@ -132,16 +150,46 @@ interface HostOptions {}export function toolHost(bridge: Bridge, { agentRun = false, resultBytes, policy, tag }: HostOptions): ToolHost {+ const invoke = (action: string, args: Record<string, unknown>, signal?: AbortSignal) =>+ bridge.invoke(action, resultBytes && action === EXTRACT_ACTION ? fitRead(args, resultBytes) : args, signal);++ const present = (action: string, result: ActionResult) =>+ (RENDERERS[action] ?? render)(result, shouldFence(action, policy) ? tag : undefined);++ const openSkill = async ({ name }: Record<string, unknown>): Promise<ToolReply> => {+ const group = typeof name === 'string' ? groupNamed(name) : undefined;+ if (!group) return render(failure('UNKNOWN_SKILL', `There is no guide named "${String(name ?? '')}". The guides are:\n${GROUP_INDEX}`));+ const guide = groupGuide(group, (await bridge.describe()).tools, loadSkills());+ return guide.ok ? { content: [{ type: 'text', text: guide.data }] } : render(guide);+ };++ const withSchema = async (action: string, result: ActionResult): Promise<ActionResult> => {+ if (result.ok || result.error.code !== 'INVALID_INPUT') return result;+ const schema = (await bridge.describe()).tools.find((tool) => tool.name === action)?.inputSchema;+ return schema ? failure('INVALID_INPUT', `${result.error.message}\n${toolNameFor(action)} takes: ${JSON.stringify(schema)}`) : result;+ };++ const use = async ({ action: named, input, ...flat }: Record<string, unknown>, signal?: AbortSignal): Promise<ToolReply> => {+ const action = typeof named === 'string' ? pageActionNamed(named) : undefined;+ if (!action) {+ return render(+ failure(+ 'UNKNOWN_ACTION',+ `There is no browser action named "${String(named ?? '')}". Open the guide that covers what you need with ${OPEN_SKILL_TOOL}:\n${GROUP_INDEX}`,+ ),+ );+ }+ const args = inputFrom(input ?? flat);+ if (!args) return render(failure('INVALID_INPUT', `input must be an object, or a JSON string of one, holding ${toolNameFor(action)}'s parameters.`));+ return present(action, await withSchema(action, await invoke(action, args, signal)));+ };+const answer = async (name: string, args: Record<string, unknown>, signal?: AbortSignal): Promise<ToolReply> => {if (name === STATUS_TOOL) return render(await status(bridge));+ if (name === OPEN_SKILL_TOOL) return openSkill(args);+ if (name === USE_TOOL) return use(args, signal);const action = actionNameFor(name);- const result = await bridge.invoke(action, resultBytes && name === EXTRACT_TOOL ? fitRead(args, resultBytes) : args, signal);- const fenceWith = shouldFence(action, policy) ? tag : undefined;- if (name === SCREENSHOT_TOOL) return renderScreenshot(result);- if (name === FOCUS_SHOT_TOOL.name) return renderFocusShot(result);- if (name === PICK_TOOL) return renderPick(result, fenceWith);- if (name === CAPTCHA_TOOL) return renderCaptcha(result, fenceWith);- return render(result, fenceWith);+ return present(action, await invoke(action, args, signal));};return {@@ -157,6 +205,7 @@ export function toolHost(bridge: Bridge, { agentRun = false, resultBytes, policy'Report whether the Browsentic browser extension is connected, its version, and the active tab. Use this first if a page tool fails.',inputSchema: { type: 'object' as const, properties: {}, additionalProperties: false },},+ OPEN_SKILL,...RESERVED_TOOLS.filter((tool) => reserved.includes(tool.action)).map((tool) => tool.descriptor),],withheld: withheld.map(listed),@@ -204,6 +253,21 @@ function withinLimit(reply: ToolReply, resultBytes: number): ToolReply {};}+/** An action's input as a model sends it: an object, or that object as a JSON string, which some CLIs pass instead. */+function inputFrom(input: unknown): Record<string, unknown> | undefined {+ const value = typeof input === 'string' ? parseJson(input) : input;+ return value && typeof value === 'object' && !Array.isArray(value) ? (value as Record<string, unknown>) : undefined;+}++function parseJson(text: string): unknown {+ if (!text.trim()) return {};+ try {+ return JSON.parse(text);+ } catch {+ return undefined;+ }+}+const listed = (action: ToolDescriptor): ListedTool => ({name: toolNameFor(action.name),description: action.description,diff --git a/src/lib/actions/core.ts b/src/lib/actions/core.tsindex 23c80de..b16027a 100644--- a/src/lib/actions/core.ts+++ b/src/lib/actions/core.ts@@ -1,4 +1,5 @@import type { z } from 'zod';+import type { ToolGroupName } from './groups';export interface Action<Input extends z.ZodType = z.ZodType, Output = unknown> {readonly name: string;@@ -7,6 +8,8 @@ export interface Action<Input extends z.ZodType = z.ZodType, Output = unknown> {readonly execute: (input: z.output<Input>) => Output | Promise<Output>;/** Needs Chrome's debugger, which Firefox has no counterpart for, so a Firefox build leaves it off the list it offers. */readonly chromiumOnly?: true;+ /** A less common capability, taught by its group's guide; ungrouped actions are the core every run is offered. */+ readonly group?: ToolGroupName;}export type AnyAction = Action<any, any>;diff --git a/src/lib/actions/groups.ts b/src/lib/actions/groups.tsnew file mode 100644index 0000000..3e0ca1f--- /dev/null+++ b/src/lib/actions/groups.ts@@ -0,0 +1,20 @@+export interface ToolGroup {+ /** Its line in the index, in the words someone would use for the need. */+ readonly index: string;+ /** Bundled skills whose bodies open the group's guide, in order. */+ readonly skills: readonly string[];+}++export const TOOL_GROUPS = {+ captcha: { index: 'get past a “verify you are human” check: reCAPTCHA, hCaptcha, Turnstile', skills: ['captcha'] },+ theming: { index: 'measure the colours and contrast a page paints, and retheme it: dark mode, easier on the eyes', skills: ['page-theming'] },+ diagnostics: { index: 'read the console errors, exceptions and failed requests behind a page that misbehaves', skills: ['page-diagnostics'] },+ monitoring: {+ index: 'watch a long upload, build or deploy and tell the user when it ends, or come back to something later on a timer',+ skills: ['monitor-progress', 'scheduled-jobs'],+ },+ downloads: { index: 'make the page download a file and keep it, and list the files kept so far', skills: [] },+ drag: { index: 'drag one thing onto another: reorder a list, move a card, pull a slider, draw on a canvas', skills: [] },+} as const satisfies Record<string, ToolGroup>;++export type ToolGroupName = keyof typeof TOOL_GROUPS;diff --git a/src/lib/actions/page/apply-theme.ts b/src/lib/actions/page/apply-theme.tsindex 1313757..bd57170 100644--- a/src/lib/actions/page/apply-theme.ts+++ b/src/lib/actions/page/apply-theme.ts@@ -30,6 +30,7 @@ const UNINVERT = 'invert(1) hue-rotate(180deg)';export const applyTheme = defineAction({name: 'page.applyTheme',+ group: 'theming',description:'Retheme the page, or put it back. Prefers the page’s own terms — it switches on the dark/light hook its stylesheets ' +'already define (a ".dark" class, a [data-theme] attribute), sets color-scheme, and overrides the design tokens you name. ' +diff --git a/src/lib/actions/page/audit-contrast.ts b/src/lib/actions/page/audit-contrast.tsindex a193bea..da1f153 100644--- a/src/lib/actions/page/audit-contrast.ts+++ b/src/lib/actions/page/audit-contrast.ts@@ -31,6 +31,7 @@ interface Check {export const auditContrast = defineAction({name: 'page.auditContrast',+ group: 'theming',description:'Score the readability of the page against WCAG contrast rules. Walks the visible text, resolves each run’s ' +'foreground against the real background painted behind it — blending translucent layers up the ancestor chain — ' +diff --git a/src/lib/actions/page/await-monitor.ts b/src/lib/actions/page/await-monitor.tsindex a8c32ad..e861552 100644--- a/src/lib/actions/page/await-monitor.ts+++ b/src/lib/actions/page/await-monitor.ts@@ -4,6 +4,7 @@ import { ActionError, defineAction } from '../core';export const awaitMonitor = defineAction({name: 'page.awaitMonitor',+ group: 'monitoring',description:'Block until a background monitor completes, then return its final state with the full log. ' +'A reply with settled: false means the timeout passed while the watch continues — call again to keep waiting; that is normal, not an error. ' +diff --git a/src/lib/actions/page/capture-download.ts b/src/lib/actions/page/capture-download.tsindex d2e11fd..6879daa 100644--- a/src/lib/actions/page/capture-download.ts+++ b/src/lib/actions/page/capture-download.ts@@ -6,6 +6,7 @@ export const CAPTURE_TIMEOUT_MS = 60_000;export const captureDownload = defineAction({name: 'page.captureDownload',+ group: 'downloads',description:'Make the page download a file and keep it. Either click something that produces a download — an “Export CSV” button, a “Download invoice” link — or give a direct url, which is fetched in the browser’s own logged-in session rather than anonymously. The file lands in the user’s ~/browsentic/download/ folder and the result reports the path and notes about what arrived; you get the notes, never the bytes. Hand the returned downloadId to page.attachFile to upload it somewhere else without the file ever passing through you.',input: z.object({diff --git a/src/lib/actions/page/drag-element.ts b/src/lib/actions/page/drag-element.tsindex 838c1f8..603f658 100644--- a/src/lib/actions/page/drag-element.ts+++ b/src/lib/actions/page/drag-element.ts@@ -119,6 +119,7 @@ async function nativeDrag(source: HTMLElement, plan: DragPath) {export const dragElement = defineAction({name: 'page.dragElement',+ group: 'drag',description:'Drag one thing onto another — reorder a list, move a card between columns, pull a slider handle, draw on a canvas. Give a "from" element (or "fromPoint") to pick up and a "to" element (or "toPoint") to drop on. Both ends must be on screen at the same moment, because nothing auto-scrolls mid-drag. The web has two unrelated drag mechanisms and "auto" chooses between them by reading the grabbed element: "pointer" presses, moves and releases a pointer, which is what dnd-kit, react-beautiful-dnd, sliders and canvases listen for; "native" fires the HTML5 dragstart/dragover/drop sequence with a DataTransfer, which is what an element carrying draggable="true" expects. The drop point is measured before the drag starts, so a list that reflows as you pass over it can land a slot out — raise "steps" and "settleMs", then read the result back. Set "trusted" when the page ignores synthetic events; that path is pointer-only, Chrome-only, and shows the “Browsentic is debugging this browser” bar. If nothing moved, check "accepted" and try the other mode.',input: z.object({diff --git a/src/lib/actions/page/find-captcha.ts b/src/lib/actions/page/find-captcha.tsindex d5f6189..8a282e9 100644--- a/src/lib/actions/page/find-captcha.ts+++ b/src/lib/actions/page/find-captcha.ts@@ -4,6 +4,7 @@ import { ActionError, defineAction } from '../core';export const findCaptcha = defineAction({name: 'page.findCaptcha',chromiumOnly: true,+ group: 'captcha',description:'Report what captcha is on the page, without touching it: the vendor (Cloudflare Turnstile, reCAPTCHA v2/v3, hCaptcha, GeeTest, Arkose, AWS WAF), how many frames deep it sits, its on-screen bounds, whether it is already satisfied, and — while an image challenge is open — its prompt and grid. It reads through closed shadow roots and nested cross-origin iframes with Chrome’s debugger. Read-only. To get past a captcha, call page.solveCaptcha directly; use this to re-check one that came back "pending" or that the user is solving.',input: z.object({}),diff --git a/src/lib/actions/page/find-progress.ts b/src/lib/actions/page/find-progress.tsindex 601d1ad..7324679 100644--- a/src/lib/actions/page/find-progress.ts+++ b/src/lib/actions/page/find-progress.ts@@ -18,6 +18,7 @@ interface Candidate {export const findProgress = defineAction({name: 'page.findProgress',+ group: 'monitoring',description:'Scan the page for progress signals worth monitoring: progress bars, percent readouts, spinners and busy regions, each with a selector for page.startMonitor. ' +'An empty candidates list with no titlePercent means the page shows nothing measurable — ask the user what completion looks like instead of starting a monitor.',diff --git a/src/lib/actions/page/list-downloads.ts b/src/lib/actions/page/list-downloads.tsindex 21f677d..bc22426 100644--- a/src/lib/actions/page/list-downloads.ts+++ b/src/lib/actions/page/list-downloads.ts@@ -3,6 +3,7 @@ import { ActionError, defineAction } from '../core';export const listDownloads = defineAction({name: 'page.listDownloads',+ group: 'downloads',description:'List the files Browsentic has captured with page.captureDownload, newest first, with notes about what each one is and where it was saved. Use a downloadId from here with page.attachFile to upload one to another page.',input: z.object({diff --git a/src/lib/actions/page/monitor-status.ts b/src/lib/actions/page/monitor-status.tsindex c521d70..117cf64 100644--- a/src/lib/actions/page/monitor-status.ts+++ b/src/lib/actions/page/monitor-status.ts@@ -3,6 +3,7 @@ import { ActionError, defineAction } from '../core';export const monitorStatus = defineAction({name: 'page.monitorStatus',+ group: 'monitoring',description:'Report on background monitors started with page.startMonitor: phase, percent, ETA, how long since anything changed, and the latest log lines.',input: z.object({diff --git a/src/lib/actions/page/read-console.ts b/src/lib/actions/page/read-console.tsindex a5aa725..4380159 100644--- a/src/lib/actions/page/read-console.ts+++ b/src/lib/actions/page/read-console.ts@@ -5,6 +5,7 @@ import { ActionError, defineAction } from '../core';export const readConsole = defineAction({name: 'page.readConsole',chromiumOnly: true,+ group: 'diagnostics',description:'Read the console messages and uncaught exceptions a page has reported since page.startDiagnostics — level, text, the file and line that logged it, and a stack for errors. ' +'Newest last. Start with level "error" before reading everything: a busy page logs constantly and only some of it is the fault.',diff --git a/src/lib/actions/page/read-network.ts b/src/lib/actions/page/read-network.tsindex 691b80d..1b7fad3 100644--- a/src/lib/actions/page/read-network.ts+++ b/src/lib/actions/page/read-network.ts@@ -5,6 +5,7 @@ import { ActionError, defineAction } from '../core';export const readNetwork = defineAction({name: 'page.readNetwork',chromiumOnly: true,+ group: 'diagnostics',description:'Read the requests a page has made since page.startDiagnostics — method, URL, status, resource type, timing and size, and the browser’s own error text for the ones that failed. ' +'Newest last. This is how a button that “did nothing” turns into a 500 or a CORS refusal. Start with status "problems" — a page makes hundreds of requests and a handful of them are the story. ' +diff --git a/src/lib/actions/page/read-theme.ts b/src/lib/actions/page/read-theme.tsindex 4d27721..e322d66 100644--- a/src/lib/actions/page/read-theme.ts+++ b/src/lib/actions/page/read-theme.ts@@ -46,6 +46,7 @@ interface Surface {export const readTheme = defineAction({name: 'page.readTheme',+ group: 'theming',description:'Measure the page’s theme: the relative luminance of its background and text, whether it is rendering light or dark, ' +'the palette actually painted on screen grouped into surface, text, border and accent colours with how much area each covers, ' +diff --git a/src/lib/actions/page/solve-captcha.ts b/src/lib/actions/page/solve-captcha.tsindex 38ec6c0..7900528 100644--- a/src/lib/actions/page/solve-captcha.ts+++ b/src/lib/actions/page/solve-captcha.ts@@ -6,6 +6,7 @@ export const SOLVE_CAPTCHA_TIMEOUT_MS = 120_000;export const solveCaptcha = defineAction({name: 'page.solveCaptcha',chromiumOnly: true,+ group: 'captcha',description:'Get past a captcha. Call it as soon as page.getPageInfo reports a "captcha", or a page stalls on “verify you are human” — nothing else reaches the widget, so do not try clicks or selectors first. It finds the widget in any frame at any depth, through closed shadow roots and cross-origin iframes, waits out one that is still loading, ticks the checkbox with a real browser-level click, and sees an image challenge through: Browsentic’s own vision analyst answers reCAPTCHA and hCaptcha challenges round by round when it can. State "solved" means carry on. State "challenge" means a challenge is still open and waiting on you: the result carries it as an image plus its prompt — a tile grid with each tile numbered (answer with "tiles") or a picture to tap on (answer with "points") — look at it and call this again with your answer, repeating until "solved". "invisible" is a scoring captcha with nothing to click; "needsHuman" is a puzzle it cannot answer (Arkose, AWS WAF) — hand that one to the user. Shows the debugger bar while it runs, and is gated for approval because it acts on another site’s security control.',input: z.object({diff --git a/src/lib/actions/page/start-diagnostics.ts b/src/lib/actions/page/start-diagnostics.tsindex 8497764..b3cde0c 100644--- a/src/lib/actions/page/start-diagnostics.ts+++ b/src/lib/actions/page/start-diagnostics.ts@@ -5,6 +5,7 @@ import { ActionError, defineAction } from '../core';export const startDiagnostics = defineAction({name: 'page.startDiagnostics',chromiumOnly: true,+ group: 'diagnostics',description:'Start recording what a page reports rather than what it shows — console messages, uncaught exceptions and every request the tab makes. ' +'Console and network events only exist while Chrome’s debugger is attached, so this has to be running before the thing you are diagnosing happens: start it, then reload or click, then read. ' +diff --git a/src/lib/actions/page/start-monitor.ts b/src/lib/actions/page/start-monitor.tsindex f99374f..ec23b87 100644--- a/src/lib/actions/page/start-monitor.ts+++ b/src/lib/actions/page/start-monitor.ts@@ -5,6 +5,7 @@ import { targetSchema } from './dom';export const startMonitor = defineAction({name: 'page.startMonitor',+ group: 'monitoring',description:'Watch one tab in the background until a progress condition completes — an upload reaching 100%, a build log announcing success, a spinner disappearing. ' +'Returns a monitorId immediately; the extension pins the tab, keeps watching with no further tool calls even while the user works elsewhere, and notifies them on completion. ' +diff --git a/src/lib/actions/page/start-timer.ts b/src/lib/actions/page/start-timer.tsindex 1666fdb..8c23973 100644--- a/src/lib/actions/page/start-timer.ts+++ b/src/lib/actions/page/start-timer.ts@@ -10,6 +10,7 @@ import { ActionError, defineAction } from '../core';export const startTimer = defineAction({name: 'page.startTimer',+ group: 'monitoring',description:'Schedule work for later — “in ten minutes check whether the build finished”, “every two minutes refresh the queue and tell me when something lands”. ' +'Returns a timerId immediately; the extension keeps the schedule with no further tool calls and wakes this conversation with your prompt each time it fires, so set it and end your turn. ' +diff --git a/src/lib/actions/page/stop-diagnostics.ts b/src/lib/actions/page/stop-diagnostics.tsindex f1ac5f2..3af63fa 100644--- a/src/lib/actions/page/stop-diagnostics.ts+++ b/src/lib/actions/page/stop-diagnostics.ts@@ -4,6 +4,7 @@ import { ActionError, defineAction } from '../core';export const stopDiagnostics = defineAction({name: 'page.stopDiagnostics',chromiumOnly: true,+ group: 'diagnostics',description:'Detach the debugger and take Chrome’s “Browsentic is debugging this browser” bar away. What was collected stays readable by page.readConsole and page.readNetwork afterwards, minus response bodies, which only exist while attached. ' +'Call this as soon as you have what you need rather than leaving the bar up.',diff --git a/src/lib/actions/page/stop-monitor.ts b/src/lib/actions/page/stop-monitor.tsindex dcd4674..5bdfe64 100644--- a/src/lib/actions/page/stop-monitor.ts+++ b/src/lib/actions/page/stop-monitor.ts@@ -3,6 +3,7 @@ import { ActionError, defineAction } from '../core';export const stopMonitor = defineAction({name: 'page.stopMonitor',+ group: 'monitoring',description:'Stop a background monitor before it completes. The tab is unpinned again if the monitor pinned it. No notification is shown — the stop was asked for.',input: z.object({diff --git a/src/lib/actions/page/stop-timer.ts b/src/lib/actions/page/stop-timer.tsindex b33bc0a..e84576c 100644--- a/src/lib/actions/page/stop-timer.ts+++ b/src/lib/actions/page/stop-timer.ts@@ -3,6 +3,7 @@ import { ActionError, defineAction } from '../core';export const stopTimer = defineAction({name: 'page.stopTimer',+ group: 'monitoring',description:'Cancel a scheduled job before it has run out. Nothing further fires and no notification is shown — the stop was asked for.',input: z.object({diff --git a/src/lib/actions/page/timer-status.ts b/src/lib/actions/page/timer-status.tsindex 0686243..b86c20d 100644--- a/src/lib/actions/page/timer-status.ts+++ b/src/lib/actions/page/timer-status.ts@@ -3,6 +3,7 @@ import { ActionError, defineAction } from '../core';export const timerStatus = defineAction({name: 'page.timerStatus',+ group: 'monitoring',description:'Report on scheduled jobs started with page.startTimer: how many times each has fired, how many fires were skipped because the conversation was still busy, when the next one is due, and the latest log lines.',input: z.object({diff --git a/src/lib/actions/tool-names.ts b/src/lib/actions/tool-names.tsindex 230749a..5d26cbd 100644--- a/src/lib/actions/tool-names.ts+++ b/src/lib/actions/tool-names.ts@@ -8,10 +8,19 @@ export function toolNameFor(actionName: string): string {}export const STATUS_TOOL = toolNameFor(`${RESERVED_PREFIX}status`);+export const OPEN_SKILL_TOOL = toolNameFor(`${RESERVED_PREFIX}openSkill`);+export const USE_TOOL = toolNameFor(`${RESERVED_PREFIX}use`);/** Every tool a side-panel run is offered, for a CLI that has to be told each one by name. */export function agentRunToolNames(actionNames: Iterable<string>): string[] {- return [...[...actionNames].map(toolNameFor), STATUS_TOOL, toolNameFor(SAVE_SITE_MAP_ACTION), toolNameFor(FOCUS_SHOT_ACTION)];+ return [+ ...[...actionNames].map(toolNameFor),+ STATUS_TOOL,+ OPEN_SKILL_TOOL,+ USE_TOOL,+ toolNameFor(SAVE_SITE_MAP_ACTION),+ toolNameFor(FOCUS_SHOT_ACTION),+ ];}export function actionNameFor(toolName: string): string {