Skip to content

Automate browser based workflows with AI

Control your own browser from Skyvern MCP via a Chrome extension bridge (SKY-13370) (#8070)

Suchintan37e5a145b947
diff --git a/docs/developers/optimization/chrome-extension.mdx b/docs/developers/optimization/chrome-extension.mdx
new file mode 100644
index 000000000..d68d74b97
--- /dev/null
+++ b/docs/developers/optimization/chrome-extension.mdx
@@ -0,0 +1,215 @@
+---
+title: Control Your Chrome with Skyvern Agent
+subtitle: Let local Skyvern MCP use your real Chrome profile, tabs, and logged-in sessions
+description: Use the Skyvern Agent Chrome extension to automate explicitly shared tabs from a local MCP server.
+slug: developers/optimization/chrome-extension
+keywords:
+ - Chrome extension
+ - MCP
+ - local browser
+ - browser session
+ - authentication
+ - cookies
+ - Playwright
+---
+
+Want Skyvern MCP to automate sites in the Chrome browser you already use? The **Skyvern Agent** Chrome extension lets a local
+MCP server drive your real Chrome profile, including its logged-in sessions and cookies. Unlike
+`skyvern browser serve`, which starts a separate Chrome browser with a copied profile, the extension controls
+explicitly shared tabs in your actual browser. Chrome 136 and later ignores `--remote-debugging-port` for the default
+profile, so an extension is the only way to automate that browser directly.
+
+---
+
+## Requirements
+
+- Chrome 125 or later
+- A local Skyvern MCP server connected over stdio
+
+Hosted MCP is not supported yet.
+
+---
+
+## Install the extension
+
+The recommended setup command prints the unpacked extension directory, opens Chrome's extensions page when possible,
+and prints the remaining steps:
+
+```bash
+skyvern browser extension-install
+```
+
+Then enable **Developer mode**, click **Load unpacked**, and select the printed directory. If your MCP server is already
+running with the extension bridge, the command also opens the one-click pairing page. Otherwise, start the MCP server
+and run `skyvern browser extension-pair`.
+
+For manual setup:
+
+1. Print the directory containing the extension:
+
+ ```bash
+ skyvern browser extension-path
+ ```
+
+2. Open `chrome://extensions` in Chrome.
+3. Enable **Developer mode**.
+4. Click **Load unpacked**.
+5. Select the directory printed by the command.
+
+---
+
+## Pair the extension
+
+With your MCP server running, start the recommended one-click flow:
+
+```bash
+skyvern browser extension-pair
+```
+
+Click **Approve** in the pairing page, then approve the pairing again in the **Skyvern Agent** confirmation tab that
+opens. The pairing link expires after two minutes and can be used only once; rerun the command if it expires.
+
+As a manual fallback, copy the pairing token to your clipboard:
+
+```bash
+skyvern browser extension-token
+```
+
+Then open the **Skyvern Agent** popup, paste the token, and click **Connect**.
+
+Treat the pairing token like a password. The one-click URL contains only a short-lived nonce in its fragment; Skyvern
+does not put the pairing token in a URL or expose it through MCP tools.
+
+---
+
+## Configure MCP
+
+Set `BROWSER_TYPE=extension-connect` in the environment for your local MCP server. For example, use this stdio
+configuration in an MCP client that accepts JSON:
+
+```json
+{
+ "mcpServers": {
+ "skyvern": {
+ "command": "skyvern",
+ "args": ["run", "mcp"],
+ "env": {
+ "BROWSER_TYPE": "extension-connect"
+ }
+ }
+ }
+}
+```
+
+Alternatively, start the local MCP server with the browser extension bridge enabled:
+
+```bash
+skyvern run mcp --browser-extension
+```
+
+The environment setting starts the bridge lazily when `skyvern_browser_session_create` needs it. The flag starts the
+bridge eagerly with the MCP server.
+
+Check the local setup without exposing the pairing token:
+
+```bash
+skyvern browser extension-status
+```
+
+The command reports the extension directory, whether token configuration and file permissions are present, and whether
+the configured loopback port is accepting connections. The default port is `19777`; a custom port is configured with
+`SKYVERN_BROWSER_EXTENSION_PORT` for the MCP server and under **Advanced settings** in the popup.
+
+---
+
+## Use your browser
+
+After the extension is connected, a typical MCP flow is:
+
+1. Call `skyvern_browser_session_create` to connect Skyvern to the extension.
+2. Call `skyvern_navigate` to open a page.
+3. Call `skyvern_observe` to inspect the page and identify actions.
+4. Call `skyvern_execute` or `skyvern_click` to interact with the page.
+
+Selectors remain the primary way to target elements. `skyvern_click` and `skyvern_type` also accept `x` and `y`
+coordinates in viewport CSS pixels, measured from the top-left of the web content. Provide both coordinates together
+and do not combine them with a selector. Coordinates must be finite and non-negative. `skyvern_execute` can pass the
+same coordinates to click and type actions. Coordinates align with `skyvern_screenshot(full_page=False)` and are not
+scaled by the device pixel ratio.
+
+---
+
+## Consent and security
+
+Membership in the Chrome tab group named **Skyvern Controlled** is the consent boundary. Dragging a tab into that group
+shares it with Skyvern; dragging it out immediately revokes access. The **Add to Skyvern Controlled** and
+**Remove from Skyvern Controlled** buttons in the **Skyvern Agent** popup make the same group-membership changes. The
+extension never discloses tabs outside the group to Skyvern.
+
+Skyvern's scope includes:
+
+- Existing tabs you explicitly add to **Skyvern Controlled**, by dragging or with the popup
+- Tabs Skyvern creates and adds to **Skyvern Controlled**
+- Popups opened by a controlled tab and added to **Skyvern Controlled**
+
+When Skyvern attaches to a controlled tab, Chrome displays its debugger infobar; Skyvern never hides it. Clicking
+**Cancel** in that infobar immediately revokes access to the tab, and Skyvern does not automatically reattach. Skyvern
+also never closes or restarts your Chrome browser.
+
+The extension connects outbound to a bridge bound only to `127.0.0.1`. The default endpoint is
+`ws://127.0.0.1:19777/extension/v1`. Pairing-token authentication verifies both the extension and the local MCP bridge,
+and the token is never included in the URL.
+
+The extension refuses to control `chrome://`, `chrome-untrusted://`, `chrome-extension://`, `devtools://`, `edge://`,
+`file://`, Chrome Web Store pages on `chromewebstore.google.com`, and `about:` pages other than `about:blank`.
+
+---
+
+## Limitations
+
+- Downloads and file-chooser management are not supported yet.
+- Incognito windows are not supported.
+- Opening DevTools for a controlled tab detaches Skyvern and revokes access to that tab.
+- Only one local Skyvern MCP process can control the extension at a time.
+- Hosted Skyvern MCP cannot use the extension yet.
+- Chrome internal pages, extension pages, DevTools, local files, the Chrome Web Store, and the other restricted targets
+ listed above cannot be controlled.
+
+---
+
+## Troubleshooting
+
+<AccordionGroup>
+ <Accordion title="Skyvern browser extension is not connected">
+ Follow the steps in order:
+
+ - Run `skyvern browser extension-install` and follow its numbered setup instructions.
+ - Start the MCP server with `--browser-extension` or `BROWSER_TYPE=extension-connect`.
+ - Run `skyvern browser extension-status` and confirm it reports the bridge listening on the configured port.
+ - Run `skyvern browser extension-pair`, approve the browser page, then approve the Skyvern Agent confirmation tab.
+ - If one-click pairing fails, open the **Skyvern Agent** popup and use `skyvern browser extension-token` as the
+ manual paste fallback.
+ - Add at least one controllable tab to **Skyvern Controlled**, then retry.
+ </Accordion>
+
+ <Accordion title="Port already in use">
+ The bridge uses port `19777` by default. Set `SKYVERN_BROWSER_EXTENSION_PORT` in the MCP server environment,
+ configure the same port under **Advanced settings** in the popup, then restart the MCP server and reconnect the
+ extension.
+ </Accordion>
+
+ <Accordion title="Rotate the pairing token">
+ Stop the MCP server, delete `~/.skyvern/browser_extension_token`, and restart the server. Run
+ `skyvern browser extension-pair` and approve both browser steps. Use `skyvern browser extension-token` and paste the
+ new token into the **Skyvern Agent** popup only as a manual fallback.
+
+ If `SKYVERN_BROWSER_EXTENSION_TOKEN` is set in the MCP environment, remove it before rotating the token file
+ because the environment value takes precedence.
+ </Accordion>
+
+ <Accordion title="DevTools or the debugger infobar disconnected a tab">
+ Close DevTools if it is open, then add the tab to **Skyvern Controlled** again by dragging it into the group or using
+ the popup. If you clicked **Cancel** in Chrome's debugger infobar, adding the tab again restores consent; Skyvern
+ never reattaches automatically.
+ </Accordion>
+</AccordionGroup>
diff --git a/docs/docs.json b/docs/docs.json
index e435b6da2..7be5d45bd 100644
--- a/docs/docs.json
+++ b/docs/docs.json
@@ -181,6 +181,8 @@
"pages": [
"developers/optimization/browser-sessions",
"developers/optimization/browser-profiles",
+ "developers/optimization/browser-tunneling",
+ "developers/optimization/chrome-extension",
"developers/optimization/cost-control"
]
},
diff --git a/docs/snippets/mcp-content.mdx b/docs/snippets/mcp-content.mdx
index 71b3808d4..23115abd5 100644
--- a/docs/snippets/mcp-content.mdx
+++ b/docs/snippets/mcp-content.mdx
@@ -529,6 +529,10 @@ Replace `/usr/bin/python3` with the output of `which python3`. For Skyvern Cloud
You can point your AI assistant at a browser running on your machine instead of a cloud browser. This lets you automate internal tools on localhost, reuse existing login sessions, and verify local dev changes, all through the same MCP tools.
+To drive the Chrome profile and tabs you already use from a local stdio MCP server, install the
+[Skyvern Chrome extension](/developers/optimization/chrome-extension). You explicitly choose which tabs to share, and
+the connection stays on your machine.
+
Start a local browser with a tunnel so Skyvern Cloud can reach it:
```bash
diff --git a/skyvern/browser_extension/__init__.py b/skyvern/browser_extension/__init__.py
new file mode 100644
index 000000000..9d74c5036
--- /dev/null
+++ b/skyvern/browser_extension/__init__.py
@@ -0,0 +1,49 @@
+from skyvern.browser_extension.auth import (
+ build_challenge,
+ compute_ext_proof,
+ compute_server_proof,
+ load_or_create_pairing_token,
+ verify_ext_proof,
+)
+from skyvern.browser_extension.errors import (
+ BrowserExtensionError,
+ BrowserExtensionNotConnectedError,
+ ExtensionRequestError,
+)
+from skyvern.browser_extension.protocol import (
+ ALLOWED_CDP_METHOD_PREFIXES,
+ ALLOWED_EVENTS,
+ ALLOWED_OPS,
+ ERROR_CODES,
+ PROTOCOL_VERSION,
+ RESTRICTED_URL_PREFIXES,
+ ParsedMessage,
+ build_request,
+ is_cdp_method_allowed,
+ is_restricted_url,
+ parse_extension_message,
+)
+from skyvern.browser_extension.target_registry import VirtualTargetRegistry
+
+__all__ = [
+ "ALLOWED_CDP_METHOD_PREFIXES",
+ "ALLOWED_EVENTS",
+ "ALLOWED_OPS",
+ "ERROR_CODES",
+ "PROTOCOL_VERSION",
+ "RESTRICTED_URL_PREFIXES",
+ "BrowserExtensionError",
+ "BrowserExtensionNotConnectedError",
+ "ExtensionRequestError",
+ "ParsedMessage",
+ "VirtualTargetRegistry",
+ "build_challenge",
+ "build_request",
+ "compute_ext_proof",
+ "compute_server_proof",
+ "is_cdp_method_allowed",
+ "is_restricted_url",
+ "load_or_create_pairing_token",
+ "parse_extension_message",
+ "verify_ext_proof",
+]
diff --git a/skyvern/browser_extension/auth.py b/skyvern/browser_extension/auth.py
new file mode 100644
index 000000000..6d648f790
--- /dev/null
+++ b/skyvern/browser_extension/auth.py
@@ -0,0 +1,134 @@
+from __future__ import annotations
+
+import base64
+import hashlib
+import hmac
+import json
+import os
+import secrets
+import stat
+import tempfile
+from pathlib import Path
+
+from skyvern.browser_extension.errors import BrowserExtensionError
+from skyvern.browser_extension.protocol import PROTOCOL_VERSION
+
+_TOKEN_ENV = "SKYVERN_BROWSER_EXTENSION_TOKEN"
+
+
+def load_or_create_pairing_token() -> str:
+ environment_token = os.environ.get(_TOKEN_ENV)
+ if environment_token is not None:
+ environment_token = environment_token.strip()
+ if environment_token:
+ return environment_token
+
+ token_dir = Path.home() / ".skyvern"
+ token_path = token_dir / "browser_extension_token"
+ existing_token = _read_token_file(token_path)
+ if existing_token:
+ return existing_token
+ if existing_token == "":
+ token_path.unlink()
+
+ token_dir_existed = token_dir.exists()
+ token_dir.mkdir(parents=True, exist_ok=True, mode=0o700)
+ if not token_dir_existed:
+ token_dir.chmod(0o700)
+
+ token = secrets.token_urlsafe(32)
+ return _publish_token(token_path, token)
+
+
+def _read_token_file(token_path: Path) -> str | None:
+ flags = os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0)
+ try:
+ file_descriptor = os.open(token_path, flags)
+ except FileNotFoundError:
+ return None
+ except OSError as exc:
+ raise BrowserExtensionError("Pairing token path must be a regular file owned by the current user") from exc
+
+ try:
+ file_stat = os.fstat(file_descriptor)
+ if not stat.S_ISREG(file_stat.st_mode):
+ raise BrowserExtensionError("Pairing token path must be a regular file")
+ if hasattr(os, "getuid") and file_stat.st_uid != os.getuid():
+ raise BrowserExtensionError("Pairing token file must be owned by the current user")
+ if stat.S_IMODE(file_stat.st_mode) & 0o177:
+ os.fchmod(file_descriptor, 0o600)
+ with os.fdopen(file_descriptor, "r", encoding="utf-8") as token_file:
+ file_descriptor = -1
+ return token_file.read().strip()
+ finally:
+ if file_descriptor >= 0:
+ os.close(file_descriptor)
+
+
+def _publish_token(token_path: Path, token: str) -> str:
+ file_descriptor, temporary_name = tempfile.mkstemp(
+ dir=token_path.parent,
+ prefix=f".{token_path.name}.",
+ text=True,
+ )
+ temporary_path = Path(temporary_name)
+ try:
+ os.fchmod(file_descriptor, 0o600)
+ with os.fdopen(file_descriptor, "w", encoding="utf-8") as token_file:
+ token_file.write(token)
+ token_file.flush()
+ os.fsync(token_file.fileno())
+ try:
+ os.link(temporary_path, token_path)
+ except FileExistsError:
+ try:
+ existing_token = _read_token_file(token_path)
+ except BrowserExtensionError:
+ existing_token = None
+ if existing_token:
+ return existing_token
+ try:
+ token_path.unlink(missing_ok=True)
+ os.link(temporary_path, token_path)
+ except OSError as exc:
+ raise BrowserExtensionError("Failed to publish a valid pairing token after a creation race") from exc
+ except OSError as exc:
+ raise BrowserExtensionError("Failed to publish pairing token") from exc
+ return token
+ finally:
+ temporary_path.unlink(missing_ok=True)
+
+
+def build_challenge() -> tuple[str, str]:
+ server_nonce = _b64url(secrets.token_bytes(32))
+ challenge = json.dumps(
+ {"v": PROTOCOL_VERSION, "type": "auth.challenge", "serverNonce": server_nonce}, separators=(",", ":")
+ )
+ return server_nonce, challenge
+
+
+def compute_ext_proof(token: str, server_nonce: str, client_nonce: str) -> str:
+ message = f"skyvern-ext-v1|{server_nonce}|{client_nonce}"
+ return _compute_proof(token, message)
+
+
+def compute_server_proof(token: str, client_nonce: str, server_nonce: str) -> str:
+ message = f"skyvern-srv-v1|{client_nonce}|{server_nonce}"
+ return _compute_proof(token, message)
+
+
+def verify_ext_proof(token: str, server_nonce: str, client_nonce: str, proof: str) -> bool:
+ expected_proof = compute_ext_proof(token, server_nonce, client_nonce)
+ try:
+ return hmac.compare_digest(expected_proof, proof)
+ except TypeError:
+ return False
+
+
+def _compute_proof(token: str, message: str) -> str:
+ digest = hmac.new(token.encode("utf-8"), message.encode("utf-8"), hashlib.sha256).digest()
+ return _b64url(digest)
+
+
+def _b64url(value: bytes) -> str:
+ return base64.urlsafe_b64encode(value).rstrip(b"=").decode("ascii")
diff --git a/skyvern/browser_extension/cdp_adapter.py b/skyvern/browser_extension/cdp_adapter.py
new file mode 100644
index 000000000..5ab18a801
--- /dev/null
+++ b/skyvern/browser_extension/cdp_adapter.py
@@ -0,0 +1,883 @@
+from __future__ import annotations
+
+import asyncio
+import json
+import secrets
+from collections.abc import Coroutine
+from typing import TYPE_CHECKING
+
+import structlog
+from aiohttp import WSMsgType, web
+
+from skyvern.browser_extension.errors import BrowserExtensionError, ExtensionRequestError
+from skyvern.browser_extension.protocol import is_cdp_method_allowed
+from skyvern.browser_extension.relay import _MAX_WS_MESSAGE_BYTES
+from skyvern.browser_extension.target_registry import VirtualTargetRegistry
+
+if TYPE_CHECKING:
+ from skyvern.browser_extension.relay import ExtensionRelayServer
+
+
+LOG = structlog.get_logger()
+
+_VERSION_RESULT = {
+ "protocolVersion": "1.3",
+ "product": "Chrome/999.0.0.0",
+ "revision": "",
+ "userAgent": "Skyvern-Extension-Bridge",
+ "jsVersion": "",
+}
+_WINDOW_BOUNDS = {"left": 0, "top": 0, "width": 1280, "height": 720, "windowState": "normal"}
+_BROWSER_TARGET_INFO = {
+ "targetId": "skyvern-browser",
+ "type": "browser",
+ "title": "",
+ "url": "",
+ "attached": True,
+ "canAccessOpener": False,
+}
+_CHILD_AUTO_ATTACH_PARAMS = {
+ "flatten": True,
+ "autoAttach": True,
+ "waitForDebuggerOnStart": False,
+}
+
+
+class ExtensionCdpAdapter:
+ def __init__(self, registry: VirtualTargetRegistry, relay: ExtensionRelayServer) -> None:
+ self._registry = registry
+ self._relay = relay
+ self._capability = secrets.token_urlsafe(32)
+ self._runner: web.AppRunner | None = None
+ self._site: web.TCPSite | None = None
+ self._port: int | None = None
+ self._client_ws: web.WebSocketResponse | None = None
+ self._client_guard = asyncio.Lock()
+ self._send_lock = asyncio.Lock()
+ self._auto_attach = False
+ self._discover_targets = False
+ self._attached_tabs: set[int] = set()
+ self._attach_locks: dict[int, asyncio.Lock] = {}
+ self._opener_ids: dict[int, str] = {}
+ self._scope_generations: dict[int, int] = {}
+ self._scope_tombstones: set[int] = set()
+ self._background_tasks: set[asyncio.Task[None]] = set()
+
+ async def start(self) -> None:
+ if self._runner is not None:
+ return
+ app = web.Application()
+ app.router.add_get(f"/cdp/{self._capability}", self._handle_websocket)
+ runner = web.AppRunner(app)
+ await runner.setup()
+ site = web.TCPSite(runner, "127.0.0.1", 0)
+ await site.start()
+ addresses = runner.addresses
+ if not addresses:
+ await runner.cleanup()
+ raise RuntimeError("CDP adapter failed to bind")
+ self._runner = runner
+ self._site = site
+ self._port = int(addresses[0][1])
+
+ async def stop(self) -> None:
+ async with self._client_guard:
+ ws = self._client_ws
+ self._client_ws = None
+ self._reset_connection_state()
+ if ws is not None and not ws.closed:
+ await ws.close(code=1001, message=b"adapter stopped")
+ if self._runner is not None:
+ await self._runner.cleanup()
+ self._runner = None
+ self._site = None
+ self._port = None
+
+ @property
+ def cdp_ws_url(self) -> str:
+ if self._port is None:
+ raise RuntimeError("CDP adapter is not started")
+ return f"ws://127.0.0.1:{self._port}/cdp/{self._capability}"
+
+ async def handle_extension_event(self, event: str, params: dict) -> None:
+ if event == "debugger.event":
+ await self._handle_debugger_event(params)
+ elif event == "debugger.detached":
+ await self._remove_tab_with_events(params.get("tabId"))
+ elif event == "scope.tabRemoved":
+ await self._remove_tab_with_events(params.get("tabId"))
+ elif event in {"scope.tabAdded", "tabs.created"}:
+ tab_id = params.get("tabId")
+ if type(tab_id) is not int:
+ return
+ if event == "scope.tabAdded":
+ try:
+ self._registry.target_id_for_tab(tab_id)
+ except KeyError:
+ pass
+ else:
+ return
+ generation = self._begin_tab_scope(tab_id)
+ if self._auto_attach:
+ self._spawn(
+ self._handle_tab_added(
+ params,
+ include_opener=event == "tabs.created",
+ generation=generation,
+ )
+ )
+ await asyncio.sleep(0)
+ else:
+ await self._handle_tab_added(
+ params,
+ include_opener=event == "tabs.created",
+ generation=generation,
+ )
+ elif event == "extension.hello":
+ scoped_tabs = params.get("scopedTabs")
+ if isinstance(scoped_tabs, list):
+ tabs = [
+ (tab, self._begin_tab_scope(tab["tabId"]))
+ for tab in scoped_tabs
+ if isinstance(tab, dict) and type(tab.get("tabId")) is int
+ ]
+ if self._auto_attach:
+ self._spawn(self._handle_hello_tabs(tabs))
+ await asyncio.sleep(0)
+ else:
+ await self._handle_hello_tabs(tabs)
+
+ async def on_extension_disconnect(self) -> None:
+ async with self._client_guard:
+ ws = self._client_ws
+ self._client_ws = None
+ self._reset_connection_state()
+ if ws is not None and not ws.closed:
+ await ws.close(code=1001, message=b"extension disconnected")
+
+ async def _handle_websocket(self, request: web.Request) -> web.WebSocketResponse:
+ ws = web.WebSocketResponse(max_msg_size=_MAX_WS_MESSAGE_BYTES)
+ await ws.prepare(request)
+ async with self._client_guard:
+ if self._client_ws is not None:
+ rejected = True
+ else:
+ rejected = False
+ self._client_ws = ws
+ if rejected:
+ await ws.close(code=4409, message=b"CDP client already connected")
+ return ws
+
+ try:
+ async for message in ws:
+ if message.type == WSMsgType.TEXT:
+ await self._handle_client_text(ws, message.data)
+ elif message.type == WSMsgType.ERROR:
+ break
+ elif message.type == WSMsgType.BINARY:
+ await ws.close(code=1003, message=b"text frames required")
+ break
+ finally:
+ async with self._client_guard:
+ if self._client_ws is ws:
+ self._client_ws = None
+ self._reset_connection_state()
+ return ws
+
+ async def _handle_client_text(self, ws: web.WebSocketResponse, raw: str) -> None:
+ try:
+ payload = json.loads(raw)
+ except json.JSONDecodeError:
+ await self._send(ws, {"error": {"code": -32700, "message": "Parse error"}})
+ return
+ if not isinstance(payload, dict):
+ await self._send(ws, {"error": {"code": -32600, "message": "Invalid Request"}})
+ return
+ request_id = payload.get("id")
+ method = payload.get("method")
+ params = payload.get("params", {})
+ if not isinstance(method, str) or not isinstance(params, dict):
+ await self._send(ws, {"id": request_id, "error": {"code": -32600, "message": "Invalid Request"}})
+ return
+ LOG.debug("browser_extension_cdp_message", method=method, size_bytes=len(raw.encode()))
+ session_id = payload.get("sessionId")
+ try:
+ if isinstance(session_id, str):
+ if self._registry.is_browser_session_alias(session_id):
+ await self._handle_root_command(ws, request_id, method, params, session_id)
+ else:
+ await self._handle_session_command(ws, request_id, session_id, method, params)
+ else:
+ await self._handle_root_command(ws, request_id, method, params)
+ except ExtensionRequestError as exc:
+ error = {"code": -32000, "message": f"{exc.code}: {exc.message}"}
+ response = {"id": request_id, "error": error}
+ if isinstance(session_id, str):
+ response["sessionId"] = session_id
+ await self._send(ws, response)
+
+ async def _handle_session_command(
+ self, ws: web.WebSocketResponse, request_id: object, session_id: str, method: str, params: dict
+ ) -> None:
+ try:
+ tab_id, chrome_session_id = self._registry.resolve_session(session_id)
+ except KeyError:
+ await self._send(
+ ws,
+ {
+ "id": request_id,
+ "sessionId": session_id,
+ "error": {"code": -32001, "message": "session not found"},
+ },
+ )
+ return
+ if not is_cdp_method_allowed(method, params):
+ raise ExtensionRequestError("CDP_METHOD_NOT_ALLOWED", "The requested CDP method is not allowed.")
+ args = {"tabId": tab_id, "method": method, "params": params}
+ if chrome_session_id is not None:
+ args["sessionId"] = chrome_session_id
+ relay_result = await self._relay.request("debugger.send", args)
+ await self._send(
+ ws,
+ {"id": request_id, "sessionId": session_id, "result": relay_result.get("result", {})},
+ )
+
+ async def _handle_root_command(
+ self,
+ ws: web.WebSocketResponse,
+ request_id: object,
+ method: str,
+ params: dict,
+ response_session_id: str | None = None,
+ ) -> None:
+ if method == "Browser.getVersion":
+ await self._reply(ws, request_id, dict(_VERSION_RESULT), response_session_id)
+ elif method == "Browser.setDownloadBehavior":
+ await self._reply(ws, request_id, {}, response_session_id)
+ elif method == "Browser.close":
+ await self._reply(ws, request_id, {}, response_session_id)
+ await self._detach_all_tabs()
+ async with self._client_guard:
+ if self._client_ws is ws:
+ self._client_ws = None
+ self._reset_connection_state()
+ await ws.close()
+ elif method == "Browser.getWindowForTarget":
+ await self._reply(
+ ws,
+ request_id,
+ {"windowId": 1, "bounds": dict(_WINDOW_BOUNDS)},
+ response_session_id,
+ )
+ elif method == "Browser.setWindowBounds":
+ await self._reply(ws, request_id, {}, response_session_id)
+ elif method == "Browser.getWindowBounds":
+ await self._reply(ws, request_id, {"bounds": dict(_WINDOW_BOUNDS)}, response_session_id)
+ elif method == "Target.setAutoAttach":
+ await self._set_auto_attach(ws, request_id, params, response_session_id)
+ elif method == "Target.setDiscoverTargets":
+ await self._set_discover_targets(ws, request_id, params, response_session_id)
+ elif method == "Target.getTargets":
+ self._register_scoped_tabs()
+ await self._reply(
+ ws,
+ request_id,
+ {"targetInfos": self._page_target_infos()},
+ response_session_id,
+ )
+ elif method == "Target.getTargetInfo":
+ await self._get_target_info(ws, request_id, params, response_session_id)
+ elif method == "Target.createTarget":
+ await self._create_target(ws, request_id, params, response_session_id)
+ elif method == "Target.closeTarget":
+ await self._close_target(ws, request_id, params, response_session_id)
+ elif method == "Target.activateTarget":
+ await self._activate_target(ws, request_id, params, response_session_id)
+ elif method == "Target.attachToTarget":
+ await self._attach_to_target(ws, request_id, params, response_session_id)
+ elif method == "Target.attachToBrowserTarget":
+ await self._attach_to_browser_target(ws, request_id, response_session_id)
+ elif method == "Target.detachFromTarget":
+ await self._detach_from_target(ws, request_id, params, response_session_id)
+ else:
+ await self._send_error(
+ ws,
+ request_id,
+ -32601,
+ f"'{method}' wasn't found",
+ response_session_id,
+ )
+
+ async def _set_auto_attach(
+ self,
+ ws: web.WebSocketResponse,
+ request_id: object,
+ params: dict,
+ response_session_id: str | None,
+ ) -> None:
+ self._auto_attach = params.get("autoAttach") is True
+ await self._reply(ws, request_id, {}, response_session_id)
+ if not self._auto_attach:
+ return
+ tabs = [tab for tab in self._relay.scoped_tabs if isinstance(tab, dict)]
+ if not tabs:
+ try:
+ created = await self._relay.request("tabs.create", {"url": "about:blank"})
+ tab_id = created["tabId"]
+ if type(tab_id) is not int:
+ raise BrowserExtensionError("Created tab id is invalid")
+ tabs = [{"tabId": tab_id, "url": "about:blank", "title": ""}]
+ self._begin_tab_scope(tab_id)
+ except Exception as exc:
+ LOG.debug(
+ "browser_extension_auto_attach_target_skipped",
+ error_type=type(exc).__name__,
+ )
+ return
+ for tab in tabs:
+ tab_id = tab.get("tabId")
+ if type(tab_id) is not int:
+ continue
+ generation = self._active_scope_generation(tab_id)
+ if generation is None:
+ continue
+ try:
+ attached = await self._ensure_attached(tab, generation=generation)
+ if attached is not None:
+ target_id, is_new = attached
+ if is_new:
+ await self._emit_attached(tab_id, target_id, generation)
+ except Exception as exc:
+ LOG.debug(
+ "browser_extension_auto_attach_target_skipped",
+ tab_id=tab_id,
+ error_type=type(exc).__name__,
+ )
+
+ async def _set_discover_targets(
+ self,
+ ws: web.WebSocketResponse,
+ request_id: object,
+ params: dict,
+ response_session_id: str | None,
+ ) -> None:
+ self._discover_targets = params.get("discover") is True
+ self._register_scoped_tabs()
+ await self._reply(ws, request_id, {}, response_session_id)
+ if self._discover_targets:
+ for target_info in self._page_target_infos():
+ await self._emit(
+ "Target.targetCreated",
+ {"targetInfo": target_info},
+ response_session_id,
+ )
+ await self._emit(
+ "Target.targetCreated",
+ {"targetInfo": dict(_BROWSER_TARGET_INFO)},
+ response_session_id,
+ )
+
+ async def _get_target_info(
+ self,
+ ws: web.WebSocketResponse,
+ request_id: object,
+ params: dict,
+ response_session_id: str | None,
+ ) -> None:
+ target_id = params.get("targetId")
+ if target_id is None:
+ target_info = dict(_BROWSER_TARGET_INFO)
+ else:
+ self._register_scoped_tabs()
+ try:
+ if not isinstance(target_id, str):
+ raise KeyError(target_id)
+ tab_id = self._registry.tab_for_target(target_id)
+ target_info = self._registry.target_info(target_id)
+ if target_id == self._registry.target_id_for_tab(tab_id):
+ target_info = self._target_info(tab_id)
+ except KeyError:
+ await self._send_error(ws, request_id, -32000, "target not found", response_session_id)
+ return
+ await self._reply(ws, request_id, {"targetInfo": target_info}, response_session_id)
+
+ async def _create_target(
+ self,
+ ws: web.WebSocketResponse,
+ request_id: object,
+ params: dict,
+ response_session_id: str | None,
+ ) -> None:
+ url = params.get("url", "about:blank")
+ if not isinstance(url, str):
+ url = "about:blank"
+ created = await self._relay.request("tabs.create", {"url": url})
+ tab = {"tabId": created["tabId"], "url": url, "title": ""}
+ tab_id = tab["tabId"]
+ if type(tab_id) is not int:
+ await self._send_error(ws, request_id, -32000, "created target is invalid", response_session_id)
+ return
+ generation = self._begin_tab_scope(tab_id)
+ attached = await self._ensure_attached(tab, generation=generation)
+ if attached is None:
+ await self._send_error(ws, request_id, -32000, "target was revoked", response_session_id)
+ return
+ target_id, _ = attached
+ await self._emit_attached(tab_id, target_id, generation)
+ await self._reply(ws, request_id, {"targetId": target_id}, response_session_id)
+
+ async def _close_target(
+ self,
+ ws: web.WebSocketResponse,
+ request_id: object,
+ params: dict,
+ response_session_id: str | None,
+ ) -> None:
+ target_id = params.get("targetId")
+ if not isinstance(target_id, str):
+ await self._send_error(ws, request_id, -32000, "target not found", response_session_id)
+ return
+ try:
+ tab_id = self._registry.tab_for_target(target_id)
+ except KeyError:
+ await self._send_error(ws, request_id, -32000, "target not found", response_session_id)
+ return
+ await self._relay.request("tabs.remove", {"tabId": tab_id})
+ await self._reply(ws, request_id, {"success": True}, response_session_id)
+
+ async def _activate_target(
+ self,
+ ws: web.WebSocketResponse,
+ request_id: object,
+ params: dict,
+ response_session_id: str | None,
+ ) -> None:
+ target_id = params.get("targetId")
+ if not isinstance(target_id, str):
+ await self._send_error(ws, request_id, -32000, "target not found", response_session_id)
+ return
+ try:
+ tab_id = self._registry.tab_for_target(target_id)
+ except KeyError:
+ await self._send_error(ws, request_id, -32000, "target not found", response_session_id)
+ return
+ await self._relay.request("tabs.activate", {"tabId": tab_id})
+ await self._reply(ws, request_id, {}, response_session_id)
+
+ async def _attach_to_target(
+ self,
+ ws: web.WebSocketResponse,
+ request_id: object,
+ params: dict,
+ response_session_id: str | None,
+ ) -> None:
+ self._register_scoped_tabs()
+ requested_target_id = params.get("targetId")
+ if not isinstance(requested_target_id, str):
+ await self._send_error(ws, request_id, -32000, "target not found", response_session_id)
+ return
+ try:
+ tab_id = self._registry.tab_for_target(requested_target_id)
+ info = self._target_info(tab_id)
+ except KeyError:
+ await self._send_error(ws, request_id, -32000, "target not found", response_session_id)
+ return
+ tab = {"tabId": tab_id, "url": info["url"], "title": info["title"]}
+ generation = self._active_scope_generation(tab_id)
+ if generation is None:
+ await self._send_error(ws, request_id, -32000, "target was revoked", response_session_id)
+ return
+ attached = await self._ensure_attached(tab, generation=generation)
+ if attached is None:
+ await self._send_error(ws, request_id, -32000, "target was revoked", response_session_id)
+ return
+ target_id, is_new = attached
+ session_id = self._registry.create_root_session_alias(tab_id)
+ await self._reply(ws, request_id, {"sessionId": session_id}, response_session_id)
+ if is_new:
+ await self._emit_attached(tab_id, target_id, generation)
+
+ async def _attach_to_browser_target(
+ self,
+ ws: web.WebSocketResponse,
+ request_id: object,
+ response_session_id: str | None,
+ ) -> None:
+ session_id = self._registry.create_browser_session_alias()
+ await self._reply(ws, request_id, {"sessionId": session_id}, response_session_id)
+
+ async def _detach_from_target(
+ self,
+ ws: web.WebSocketResponse,
+ request_id: object,
+ params: dict,
+ response_session_id: str | None,
+ ) -> None:
+ session_id = params.get("sessionId")
+ if isinstance(session_id, str):
+ if self._registry.remove_browser_session_alias(session_id):
+ await self._reply(ws, request_id, {}, response_session_id)
+ return
+ if self._registry.remove_root_session_alias(session_id):
+ await self._reply(ws, request_id, {}, response_session_id)
+ return
+ try:
+ tab_id, chrome_session_id = self._registry.resolve_session(session_id)
+ except KeyError:
+ tab_id = None
+ chrome_session_id = None
+ if tab_id is not None and chrome_session_id is None:
+ await self._relay.request("debugger.detach", {"tabId": tab_id})
+ self._forget_tab(tab_id)
+ await self._reply(ws, request_id, {}, response_session_id)
+
+ async def _handle_debugger_event(self, payload: dict) -> None:
+ tab_id = payload.get("tabId")
+ method = payload.get("method")
+ event_params = payload.get("params")
+ if type(tab_id) is not int or not isinstance(method, str) or not isinstance(event_params, dict):
+ return
+ try:
+ if isinstance(payload.get("sessionId"), str):
+ outer_session_ids = [payload["sessionId"]]
+ self._registry.resolve_session(outer_session_ids[0])
+ else:
+ outer_session_ids = self._registry.root_session_ids(tab_id)
+ except KeyError:
+ return
+
+ if method == "Page.frameNavigated":
+ self._update_main_frame(tab_id, event_params)
+ if method == "Target.attachedToTarget":
+ child_session_id = event_params.get("sessionId")
+ target_info = event_params.get("targetInfo")
+ if isinstance(child_session_id, str) and isinstance(target_info, dict):
+ # The tab's debugger session reports its own top-level document as a
+ # "page" child on every cross-process swap; a real browser endpoint
+ # never surfaces that, and forwarding it duplicates the tab target.
+ if target_info.get("type") == "page":
+ return
+ self._registry.register_child_session(tab_id, child_session_id, target_info)
+ await self._emit_to_sessions(method, event_params, outer_session_ids)
+ if self._auto_attach:
+ self._spawn(self._set_child_auto_attach(tab_id, child_session_id))
+ await asyncio.sleep(0)
+ return
+ elif method == "Target.detachedFromTarget":
+ child_session_id = event_params.get("sessionId")
+ if not isinstance(child_session_id, str):
+ return
+ try:
+ self._registry.resolve_session(child_session_id)
+ except KeyError:
+ return
+ await self._emit_to_sessions(method, event_params, outer_session_ids)
+ self._registry.remove_child_session(child_session_id)
+ return
+ await self._emit_to_sessions(method, event_params, outer_session_ids)
+
+ async def _set_child_auto_attach(self, tab_id: int, child_session_id: str) -> None:
+ try:
+ await self._relay.request(
+ "debugger.send",
+ {
+ "tabId": tab_id,
+ "sessionId": child_session_id,
+ "method": "Target.setAutoAttach",
+ "params": dict(_CHILD_AUTO_ATTACH_PARAMS),
+ },
+ )
+ except ExtensionRequestError as exc:
+ LOG.debug(
+ "browser_extension_child_auto_attach_failed",
+ method="Target.setAutoAttach",
+ error_code=exc.code,
+ )
+
+ async def _handle_hello_tabs(self, tabs: list[tuple[dict, int]]) -> None:
+ for tab, generation in tabs:
+ await self._handle_tab_added(tab, include_opener=False, generation=generation)
+
+ async def _handle_tab_added(self, params: dict, include_opener: bool, generation: int) -> None:
+ tab_id = params.get("tabId")
+ if type(tab_id) is not int:
+ return
+ opener_id = None
+ opener_tab_id = params.get("openerTabId")
+ if include_opener and self._auto_attach and type(opener_tab_id) is int:
+ try:
+ opener_id = self._registry.target_id_for_tab(opener_tab_id)
+ except KeyError:
+ opener_id = None
+ tab = {
+ "tabId": tab_id,
+ "url": params.get("url") if isinstance(params.get("url"), str) else "about:blank",
+ "title": params.get("title") if isinstance(params.get("title"), str) else "",
+ }
+ if self._auto_attach:
+ attached = await self._ensure_attached(tab, opener_id, generation)
+ if attached is None:
+ return
+ target_id, is_new = attached
+ if is_new:
+ await self._emit_attached(tab_id, target_id, generation)
+ else:
+ if not self._scope_is_current(tab_id, generation):
+ return
+ target_id = self._register_tab(tab, opener_id)
+ if self._discover_targets and self._scope_is_current(tab_id, generation):
+ await self._emit(
+ "Target.targetCreated",
+ {"targetInfo": self._target_info(tab_id)},
+ scope_guard=(tab_id, generation),
+ )
+
+ async def _remove_tab_with_events(self, tab_id: object) -> None:
+ if type(tab_id) is not int:
+ return
+ self._revoke_tab_scope(tab_id)
+ lock = self._attach_locks.setdefault(tab_id, asyncio.Lock())
+ async with lock:
+ try:
+ session_ids = self._registry.root_session_ids(tab_id)
+ target_id = self._registry.target_id_for_tab(tab_id)
+ except KeyError:
+ return
+ self._forget_tab(tab_id)
+ for session_id in session_ids:
+ await self._emit(
+ "Target.detachedFromTarget",
+ {"sessionId": session_id, "targetId": target_id},
+ )
+ await self._emit("Target.targetDestroyed", {"targetId": target_id})
+
+ async def _ensure_attached(
+ self,
+ tab: dict,
+ opener_id: str | None = None,
+ generation: int | None = None,
+ ) -> tuple[str, bool] | None:
+ tab_id = tab.get("tabId")
+ if type(tab_id) is not int:
+ raise BrowserExtensionError("Scoped tab id is invalid")
+ if generation is None:
+ generation = self._active_scope_generation(tab_id)
+ if generation is None:
+ return None
+ lock = self._attach_locks.setdefault(tab_id, asyncio.Lock())
+ async with lock:
+ if not self._scope_is_current(tab_id, generation):
+ return None
+ is_new = tab_id not in self._attached_tabs
+ real_target_id: str | None = None
+ if is_new:
+ try:
+ await self._relay.request("debugger.attach", {"tabId": tab_id})
+ except ExtensionRequestError as exc:
+ if "already attached" not in exc.message.lower():
+ raise
+ real_target_id = await self._fetch_main_frame_id(tab_id)
+ if not self._scope_is_current(tab_id, generation):
+ return None
+ target_id = self._register_tab(tab, opener_id, real_target_id)
+ self._attached_tabs.add(tab_id)
+ return target_id, is_new
+
+ async def _fetch_main_frame_id(self, tab_id: int) -> str | None:
+ # Playwright resolves the main frame's session by targetId, so the exposed
+ # page targetId must equal Chrome's real main-frame id for this tab.
+ try:
+ tree = await self._relay.request(
+ "debugger.send",
+ {"tabId": tab_id, "method": "Page.getFrameTree", "params": {}},
+ )
+ except ExtensionRequestError:
+ return None
+ frame = tree.get("result", {}).get("frameTree", {}).get("frame", {})
+ frame_id = frame.get("id") if isinstance(frame, dict) else None
+ return frame_id if isinstance(frame_id, str) and frame_id else None
+
+ def _register_tab(self, tab: dict, opener_id: str | None = None, target_id_override: str | None = None) -> str:
+ tab_id = tab["tabId"]
+ raw_url = tab.get("url")
+ url = raw_url if isinstance(raw_url, str) else "about:blank"
+ raw_title = tab.get("title")
+ title = raw_title if isinstance(raw_title, str) else ""
+ try:
+ target_id = self._registry.target_id_for_tab(tab_id)
+ if target_id_override is not None and target_id != target_id_override:
+ target_id = self._registry.register_tab(tab_id, url, title, target_id_override)
+ else:
+ self._registry.update_tab(tab_id, url, title)
+ except KeyError:
+ target_id = self._registry.register_tab(tab_id, url, title, target_id_override)
+ if opener_id is not None:
+ self._opener_ids[tab_id] = opener_id
+ return target_id
+
+ def _register_scoped_tabs(self) -> None:
+ for tab in self._relay.scoped_tabs:
+ if isinstance(tab, dict) and type(tab.get("tabId")) is int:
+ tab_id = tab["tabId"]
+ if self._active_scope_generation(tab_id) is not None:
+ self._register_tab(tab)
+
+ def _target_info(self, tab_id: int) -> dict:
+ target_info = self._registry.target_info_for_tab(tab_id)
+ opener_id = self._opener_ids.get(tab_id)
+ if opener_id is not None:
+ target_info["openerId"] = opener_id
+ return target_info
+
+ def _page_target_infos(self) -> list[dict]:
+ return [
+ self._target_info(self._registry.tab_for_target(info["targetId"]))
+ for info in self._registry.list_page_targets()
+ ]
+
+ async def _emit_attached(self, tab_id: int, target_id: str, generation: int) -> None:
+ if not self._scope_is_current(tab_id, generation):
+ return
+ await self._emit(
+ "Target.attachedToTarget",
+ {
+ "sessionId": self._registry.root_session_id(tab_id),
+ "targetInfo": self._target_info(tab_id),
+ "waitingForDebugger": False,
+ },
+ scope_guard=(tab_id, generation),
+ )
+
+ def _update_main_frame(self, tab_id: int, params: dict) -> None:
+ frame = params.get("frame")
+ if not isinstance(frame, dict) or frame.get("parentId") is not None:
+ return
+ url = frame.get("url")
+ if not isinstance(url, str):
+ return
+ try:
+ current = self._target_info(tab_id)
+ except KeyError:
+ return
+ raw_title = frame.get("title")
+ title = raw_title if isinstance(raw_title, str) else str(current["title"])
+ self._registry.update_tab(tab_id, url, title)
+
+ async def _detach_all_tabs(self) -> None:
+ tab_ids = [self._registry.tab_for_target(info["targetId"]) for info in self._registry.list_page_targets()]
+ for tab_id in tab_ids:
+ try:
+ await self._relay.request("debugger.detach", {"tabId": tab_id}, timeout=2.0)
+ except Exception as exc:
+ LOG.debug(
+ "browser_extension_debugger_detach_failed", method="debugger.detach", error_type=type(exc).__name__
+ )
+
+ def _forget_tab(self, tab_id: int) -> None:
+ self._attached_tabs.discard(tab_id)
+ self._opener_ids.pop(tab_id, None)
+ self._registry.remove_tab(tab_id)
+
+ def _begin_tab_scope(self, tab_id: int) -> int:
+ generation = self._scope_generations.get(tab_id, 0) + 1
+ self._scope_generations[tab_id] = generation
+ self._scope_tombstones.discard(tab_id)
+ return generation
+
+ def _revoke_tab_scope(self, tab_id: int) -> int:
+ generation = self._scope_generations.get(tab_id, 0) + 1
+ self._scope_generations[tab_id] = generation
+ self._scope_tombstones.add(tab_id)
+ return generation
+
+ def _active_scope_generation(self, tab_id: int) -> int | None:
+ if tab_id in self._scope_tombstones:
+ return None
+ generation = self._scope_generations.get(tab_id)
+ if generation is None:
+ generation = self._begin_tab_scope(tab_id)
+ return generation
+
+ def _scope_is_current(self, tab_id: int, generation: int) -> bool:
+ return self._scope_generations.get(tab_id) == generation and tab_id not in self._scope_tombstones
+
+ def _reset_connection_state(self) -> None:
+ for task in self._background_tasks:
+ task.cancel()
+ self._auto_attach = False
+ self._discover_targets = False
+ self._attached_tabs.clear()
+ self._attach_locks.clear()
+ self._opener_ids.clear()
+ self._scope_generations.clear()
+ self._scope_tombstones.clear()
+ self._registry.clear()
+
+ def _spawn(self, coroutine: Coroutine[object, object, None]) -> None:
+ task = asyncio.create_task(coroutine)
+ self._background_tasks.add(task)
+ task.add_done_callback(self._background_task_done)
+
+ def _background_task_done(self, task: asyncio.Task[None]) -> None:
+ self._background_tasks.discard(task)
+ if task.cancelled():
+ return
+ error = task.exception()
+ if error is not None:
+ LOG.debug("browser_extension_event_task_failed", error_type=type(error).__name__)
+
+ async def _reply(
+ self,
+ ws: web.WebSocketResponse,
+ request_id: object,
+ result: dict,
+ session_id: str | None = None,
+ ) -> None:
+ payload = {"id": request_id, "result": result}
+ if session_id is not None:
+ payload["sessionId"] = session_id
+ await self._send(ws, payload)
+
+ async def _send_error(
+ self,
+ ws: web.WebSocketResponse,
+ request_id: object,
+ code: int,
+ message: str,
+ session_id: str | None = None,
+ ) -> None:
+ payload = {"id": request_id, "error": {"code": code, "message": message}}
+ if session_id is not None:
+ payload["sessionId"] = session_id
+ await self._send(ws, payload)
+
+ async def _emit_to_sessions(self, method: str, params: dict, session_ids: list[str]) -> None:
+ for session_id in session_ids:
+ await self._emit(method, params, session_id)
+
+ async def _emit(
+ self,
+ method: str,
+ params: dict,
+ session_id: str | None = None,
+ scope_guard: tuple[int, int] | None = None,
+ ) -> None:
+ ws = self._client_ws
+ if ws is None or ws.closed:
+ return
+ payload = {"method": method, "params": params}
+ if session_id is not None:
+ payload["sessionId"] = session_id
+ await self._send(ws, payload, scope_guard)
+
+ async def _send(
+ self,
+ ws: web.WebSocketResponse,
+ payload: dict,
+ scope_guard: tuple[int, int] | None = None,
+ ) -> None:
+ async with self._send_lock:
+ if scope_guard is not None and not self._scope_is_current(*scope_guard):
+ return
+ if not ws.closed:
+ await ws.send_json(payload)
diff --git a/skyvern/browser_extension/errors.py b/skyvern/browser_extension/errors.py
new file mode 100644
index 000000000..7a3f95716
--- /dev/null
+++ b/skyvern/browser_extension/errors.py
@@ -0,0 +1,13 @@
+class BrowserExtensionError(Exception):
+ pass
+
+
+class BrowserExtensionNotConnectedError(BrowserExtensionError):
+ pass
+
+
+class ExtensionRequestError(BrowserExtensionError):
+ def __init__(self, code: str, message: str) -> None:
+ self.code = code
+ self.message = message
+ super().__init__(f"{code}: {message}")
diff --git a/skyvern/browser_extension/extension/README.md b/skyvern/browser_extension/extension/README.md
new file mode 100644
index 000000000..c071b8511
--- /dev/null
+++ b/skyvern/browser_extension/extension/README.md
@@ -0,0 +1,57 @@
+# Install Skyvern Agent for development
+
+The **Skyvern Agent** extension is plain JavaScript and does not require a build step. The recommended setup is:
+
+```bash
+skyvern browser extension-install
+```
+
+This prints the unpacked extension directory, best-effort opens `chrome://extensions`, and prints the remaining
+numbered steps. If the MCP extension bridge is already running, it also starts one-click pairing.
+
+To set it up manually:
+
+1. Print the unpacked extension directory:
+
+ ```bash
+ skyvern browser extension-path
+ ```
+
+2. Open `chrome://extensions`, enable **Developer mode**, and click **Load unpacked**.
+3. Select the directory printed by the command.
+4. Start the local MCP server with the extension bridge enabled:
+
+ ```bash
+ skyvern run mcp --browser-extension
+ ```
+
+5. Start the recommended one-click pairing flow:
+
+ ```bash
+ skyvern browser extension-pair
+ ```
+
+6. Click **Approve** in the pairing page, then approve the pairing in the **Skyvern Agent** confirmation tab that opens.
+ The link expires after two minutes and is single-use, so rerun the command when needed.
+7. Add any tab you want Skyvern to control to the **Skyvern Controlled** tab group. Dragging a tab into the group grants
+ access and dragging it out revokes access. The popup's **Add to Skyvern Controlled** and
+ **Remove from Skyvern Controlled** buttons perform the same actions.
+
+If one-click pairing is unavailable, run `skyvern browser extension-token`, open the **Skyvern Agent** popup, paste the
+token, and click **Connect**. This popup-paste path is the manual fallback.
+
+The local Skyvern MCP process runs the bridge on `ws://127.0.0.1:19777/extension/v1` by default, and the extension
+connects outbound to it. To use another port, set `SKYVERN_BROWSER_EXTENSION_PORT` for the MCP process and enter the
+same port under **Advanced settings** in the popup.
+
+Check the token configuration, token-file permissions, and bridge status without displaying the token:
+
+```bash
+skyvern browser extension-status
+```
+
+Chrome's debugger infobar remains visible while Skyvern controls a tab. Clicking **Cancel**, removing the tab with the
+popup, or dragging the tab out of **Skyvern Controlled** revokes access.
+
+For the complete setup, security model, and limitations, see
+[Control Your Chrome with Skyvern Agent](../../../docs/developers/optimization/chrome-extension.mdx).
diff --git a/skyvern/browser_extension/extension/bridge_connection.js b/skyvern/browser_extension/extension/bridge_connection.js
new file mode 100644
index 000000000..2ea2e8086
--- /dev/null
+++ b/skyvern/browser_extension/extension/bridge_connection.js
@@ -0,0 +1,463 @@
+import {
+ BRIDGE_ALARM_NAME,
+ DEFAULT_BRIDGE_PORT,
+ ERROR_CODES,
+ MESSAGE_TYPES,
+ PROTOCOL_VERSION,
+ ProtocolError,
+ protocolErrorEnvelope,
+} from "./protocol.js";
+
+const PING_INTERVAL_MS = 20_000;
+const SILENCE_TIMEOUT_MS = 45_000;
+const RECONNECT_MAX_MS = 30_000;
+const AUTH_EXT_CONTEXT = "skyvern-ext-v1|";
+const AUTH_SERVER_CONTEXT = "skyvern-srv-v1|";
+
+function bytesToBase64Url(bytes) {
+ let binary = "";
+ for (const byte of bytes) {
+ binary += String.fromCharCode(byte);
+ }
+ return btoa(binary)
+ .replaceAll("+", "-")
+ .replaceAll("/", "_")
+ .replace(/=+$/, "");
+}
+
+function base64UrlToBytes(value) {
+ if (typeof value !== "string" || !/^[A-Za-z0-9_-]+$/.test(value)) {
+ throw new ProtocolError(
+ ERROR_CODES.AUTH_FAILED,
+ "Bridge authentication failed.",
+ );
+ }
+ const padding = "=".repeat((4 - (value.length % 4)) % 4);
+ try {
+ const binary = atob(
+ value.replaceAll("-", "+").replaceAll("_", "/") + padding,
+ );
+ return Uint8Array.from(binary, (character) => character.charCodeAt(0));
+ } catch {
+ throw new ProtocolError(
+ ERROR_CODES.AUTH_FAILED,
+ "Bridge authentication failed.",
+ );
+ }
+}
+
+function constantTimeEqual(left, right) {
+ const length = Math.max(left.length, right.length);
+ let difference = left.length ^ right.length;
+ for (let index = 0; index < length; index += 1) {
+ difference |= (left[index] ?? 0) ^ (right[index] ?? 0);
+ }
+ return difference === 0;
+}
+
+async function importHmacKey(token) {
+ return crypto.subtle.importKey(
+ "raw",
+ new TextEncoder().encode(token),
+ { name: "HMAC", hash: "SHA-256" },
+ false,
+ ["sign"],
+ );
+}
+
+async function signHmac(key, message) {
+ const signature = await crypto.subtle.sign(
+ "HMAC",
+ key,
+ new TextEncoder().encode(message),
+ );
+ return new Uint8Array(signature);
+}
+
+export class BridgeConnection {
+ constructor({ onRequest, onAuthenticated, onStateChange }) {
+ this.onRequest = onRequest;
+ this.onAuthenticated = onAuthenticated;
+ this.onStateChange = onStateChange;
+ this.socket = null;
+ this.authenticated = false;
+ this.enabled = false;
+ this.port = DEFAULT_BRIDGE_PORT;
+ this.token = "";
+ this.lastError = "";
+ this.lastMessageAt = 0;
+ this.reconnectDelayMs = 1_000;
+ this.reconnectTimer = null;
+ this.pingTimer = null;
+ this.silenceTimer = null;
+ this.connectionGeneration = 0;
+ this.clientNonce = null;
+ this.serverNonce = null;
+ this.hmacKey = null;
+
+ chrome.alarms.onAlarm.addListener((alarm) => {
+ if (alarm.name === BRIDGE_ALARM_NAME) {
+ void this.kick();
+ }
+ });
+ }
+
+ async initialize() {
+ const stored = await chrome.storage.local.get({
+ bridgePort: DEFAULT_BRIDGE_PORT,
+ pairingToken: "",
+ enabled: false,
+ });
+ this.port = this.normalizePort(stored.bridgePort);
+ this.token =
+ typeof stored.pairingToken === "string" ? stored.pairingToken : "";
+ this.enabled = stored.enabled === true;
+ await chrome.alarms.create(BRIDGE_ALARM_NAME, { periodInMinutes: 1 });
+ this.notifyState();
+ if (this.enabled && this.token) {
+ await this.kick();
+ }
+ }
+
+ getStatus() {
+ return {
+ connected: this.authenticated,
+ enabled: this.enabled,
+ bridgePort: this.port,
+ pairingToken: this.token,
+ lastError: this.lastError,
+ };
+ }
+
+ async setConfig({ bridgePort, pairingToken }) {
+ const port = this.normalizePort(bridgePort);
+ if (typeof pairingToken !== "string") {
+ throw new ProtocolError(
+ ERROR_CODES.AUTH_FAILED,
+ "A pairing token is required.",
+ );
+ }
+ const token = pairingToken.trim();
+ const changed = port !== this.port || token !== this.token;
+ this.port = port;
+ this.token = token;
+ await chrome.storage.local.set({ bridgePort: port, pairingToken: token });
+ if (changed && this.enabled) {
+ this.closeSocket();
+ await this.kick();
+ }
+ return {};
+ }
+
+ async connect() {
+ if (!this.token) {
+ throw new ProtocolError(
+ ERROR_CODES.AUTH_FAILED,
+ "A pairing token is required.",
+ );
+ }
+ this.enabled = true;
+ this.lastError = "";
+ this.reconnectDelayMs = 1_000;
+ await chrome.storage.local.set({ enabled: true });
+ await this.kick();
+ return {};
+ }
+
+ async disconnect() {
+ this.enabled = false;
+ this.lastError = "";
+ await chrome.storage.local.set({ enabled: false });
+ this.clearReconnectTimer();
+ this.closeSocket();
+ this.notifyState();
+ return {};
+ }
+
+ async kick() {
+ if (!this.enabled || !this.token || this.socket !== null) {
+ return;
+ }
+
+ const generation = ++this.connectionGeneration;
+ let socket;
+ try {
+ socket = new WebSocket(`ws://127.0.0.1:${this.port}/extension/v1`);
+ } catch {
+ this.scheduleReconnect();
+ return;
+ }
+ this.socket = socket;
+ this.authenticated = false;
+ this.clientNonce = null;
+ this.serverNonce = null;
+ this.hmacKey = null;
+ this.notifyState();
+
+ socket.addEventListener("open", () => {
+ if (generation === this.connectionGeneration) {
+ this.lastMessageAt = Date.now();
+ this.startHealthTimers();
+ }
+ });
+ socket.addEventListener("message", (event) => {
+ if (generation === this.connectionGeneration) {
+ void this.handleMessage(event.data);
+ }
+ });
+ socket.addEventListener("error", () => {
+ if (generation === this.connectionGeneration) {
+ this.lastError = "Could not reach the local Skyvern bridge.";
+ this.notifyState();
+ }
+ });
+ socket.addEventListener("close", () => {
+ if (generation !== this.connectionGeneration) {
+ return;
+ }
+ this.socket = null;
+ this.authenticated = false;
+ this.stopHealthTimers();
+ this.notifyState();
+ this.scheduleReconnect();
+ });
+ }
+
+ async handleMessage(rawMessage) {
+ this.lastMessageAt = Date.now();
+ let message;
+ try {
+ message = JSON.parse(rawMessage);
+ } catch {
+ this.failConnection("The local bridge sent an invalid message.");
+ return;
+ }
+
+ if (message?.v !== PROTOCOL_VERSION || typeof message.type !== "string") {
+ this.failConnection("The local bridge protocol is incompatible.");
+ return;
+ }
+
+ try {
+ if (!this.authenticated) {
+ await this.handleAuthMessage(message);
+ return;
+ }
+ if (message.type === MESSAGE_TYPES.PING) {
+ this.sendRaw({ v: PROTOCOL_VERSION, type: MESSAGE_TYPES.PONG });
+ return;
+ }
+ if (message.type === MESSAGE_TYPES.PONG) {
+ return;
+ }
+ if (message.type === MESSAGE_TYPES.REQUEST) {
+ await this.handleRequest(message);
+ return;
+ }
+ this.failConnection("The local bridge sent an unexpected message.");
+ } catch (error) {
+ if (
+ error instanceof ProtocolError &&
+ error.code === ERROR_CODES.AUTH_FAILED
+ ) {
+ this.failConnection(error.message, 4403);
+ } else {
+ this.failConnection("The local bridge connection failed.");
+ }
+ }
+ }
+
+ async handleAuthMessage(message) {
+ if (
+ message.type === MESSAGE_TYPES.AUTH_CHALLENGE &&
+ this.serverNonce === null
+ ) {
+ const serverNonceBytes = base64UrlToBytes(message.serverNonce);
+ if (serverNonceBytes.length !== 32) {
+ throw new ProtocolError(
+ ERROR_CODES.AUTH_FAILED,
+ "Bridge authentication failed.",
+ );
+ }
+ this.serverNonce = message.serverNonce;
+ const nonceBytes = crypto.getRandomValues(new Uint8Array(32));
+ this.clientNonce = bytesToBase64Url(nonceBytes);
+ this.hmacKey = await importHmacKey(this.token);
+ const proof = bytesToBase64Url(
+ await signHmac(
+ this.hmacKey,
+ `${AUTH_EXT_CONTEXT}${this.serverNonce}|${this.clientNonce}`,
+ ),
+ );
+ this.sendRaw({
+ v: PROTOCOL_VERSION,
+ type: MESSAGE_TYPES.AUTH_PROOF,
+ clientNonce: this.clientNonce,
+ proof,
+ });
+ return;
+ }
+
+ if (
+ message.type === MESSAGE_TYPES.AUTH_OK &&
+ this.clientNonce !== null &&
+ this.serverNonce !== null
+ ) {
+ const receivedProof = base64UrlToBytes(message.serverProof);
+ const expectedProof = await signHmac(
+ this.hmacKey,
+ `${AUTH_SERVER_CONTEXT}${this.clientNonce}|${this.serverNonce}`,
+ );
+ if (!constantTimeEqual(receivedProof, expectedProof)) {
+ throw new ProtocolError(
+ ERROR_CODES.AUTH_FAILED,
+ "Bridge authentication failed.",
+ );
+ }
+ this.authenticated = true;
+ this.lastError = "";
+ this.reconnectDelayMs = 1_000;
+ this.notifyState();
+ await this.onAuthenticated();
+ return;
+ }
+
+ throw new ProtocolError(
+ ERROR_CODES.AUTH_FAILED,
+ "Bridge authentication failed.",
+ );
+ }
+
+ async handleRequest(message) {
+ if (typeof message.id !== "string" || typeof message.op !== "string") {
+ this.failConnection("The local bridge sent an invalid request.");
+ return;
+ }
+
+ try {
+ const result = await this.onRequest(message.op, message.args);
+ this.sendRaw({
+ v: PROTOCOL_VERSION,
+ type: MESSAGE_TYPES.RESPONSE,
+ id: message.id,
+ ok: true,
+ result,
+ });
+ } catch (error) {
+ this.sendRaw({
+ v: PROTOCOL_VERSION,
+ type: MESSAGE_TYPES.RESPONSE,
+ id: message.id,
+ ok: false,
+ error: protocolErrorEnvelope(error),
+ });
+ }
+ }
+
+ sendEvent(event, params) {
+ if (!this.authenticated) {
+ return false;
+ }
+ return this.sendRaw({
+ v: PROTOCOL_VERSION,
+ type: MESSAGE_TYPES.EVENT,
+ event,
+ params,
+ });
+ }
+
+ sendRaw(message) {
+ if (this.socket?.readyState !== WebSocket.OPEN) {
+ return false;
+ }
+ this.socket.send(JSON.stringify(message));
+ return true;
+ }
+
+ normalizePort(value) {
+ const port = Number(value);
+ if (!Number.isInteger(port) || port < 1 || port > 65_535) {
+ throw new ProtocolError(
+ ERROR_CODES.INTERNAL,
+ "Bridge port must be between 1 and 65535.",
+ );
+ }
+ return port;
+ }
+
+ startHealthTimers() {
+ this.stopHealthTimers();
+ this.pingTimer = setInterval(() => {
+ if (this.authenticated) {
+ this.sendRaw({ v: PROTOCOL_VERSION, type: MESSAGE_TYPES.PING });
+ }
+ }, PING_INTERVAL_MS);
+ this.silenceTimer = setInterval(() => {
+ if (
+ this.lastMessageAt > 0 &&
+ Date.now() - this.lastMessageAt > SILENCE_TIMEOUT_MS
+ ) {
+ this.failConnection("The local bridge stopped responding.");
+ }
+ }, 5_000);
+ }
+
+ stopHealthTimers() {
+ if (this.pingTimer !== null) {
+ clearInterval(this.pingTimer);
+ this.pingTimer = null;
+ }
+ if (this.silenceTimer !== null) {
+ clearInterval(this.silenceTimer);
+ this.silenceTimer = null;
+ }
+ }
+
+ scheduleReconnect() {
+ if (!this.enabled || !this.token || this.reconnectTimer !== null) {
+ return;
+ }
+ const delay = this.reconnectDelayMs;
+ this.reconnectDelayMs = Math.min(
+ this.reconnectDelayMs * 2,
+ RECONNECT_MAX_MS,
+ );
+ this.reconnectTimer = setTimeout(() => {
+ this.reconnectTimer = null;
+ void this.kick();
+ }, delay);
+ }
+
+ clearReconnectTimer() {
+ if (this.reconnectTimer !== null) {
+ clearTimeout(this.reconnectTimer);
+ this.reconnectTimer = null;
+ }
+ }
+
+ closeSocket() {
+ this.connectionGeneration += 1;
+ const socket = this.socket;
+ this.socket = null;
+ this.authenticated = false;
+ this.stopHealthTimers();
+ if (socket !== null && socket.readyState < WebSocket.CLOSING) {
+ socket.close(1000, "Disconnected");
+ }
+ this.notifyState();
+ }
+
+ failConnection(message, code = 4400) {
+ this.lastError = message;
+ this.authenticated = false;
+ const socket = this.socket;
+ if (socket !== null && socket.readyState < WebSocket.CLOSING) {
+ socket.close(code, message.slice(0, 120));
+ }
+ this.notifyState();
+ }
+
+ notifyState() {
+ this.onStateChange(this.getStatus());
+ }
+}
diff --git a/skyvern/browser_extension/extension/debugger_router.js b/skyvern/browser_extension/extension/debugger_router.js
new file mode 100644
index 000000000..8ce4340cd
--- /dev/null
+++ b/skyvern/browser_extension/extension/debugger_router.js
@@ -0,0 +1,217 @@
+import {
+ ALLOWED_CDP_PREFIXES,
+ DENIED_CDP_METHODS,
+ ERROR_CODES,
+ EVENTS,
+ ProtocolError,
+ requireArgs,
+ requireTabId,
+} from "./protocol.js";
+
+export class DebuggerRouter {
+ constructor({ tabScope, sendEvent, onAttachedChange }) {
+ this.tabScope = tabScope;
+ this.sendEvent = sendEvent;
+ this.onAttachedChange = onAttachedChange;
+ this.attachedTabs = new Set();
+
+ chrome.debugger.onEvent.addListener((source, method, params) => {
+ void this.handleDebuggerEvent(source, method, params);
+ });
+ chrome.debugger.onDetach.addListener((source, reason) => {
+ void this.handleDebuggerDetach(source, reason);
+ });
+ }
+
+ hasAttachedTabs() {
+ return this.attachedTabs.size > 0;
+ }
+
+ async attach(args) {
+ const values = requireArgs(args);
+ const tabId = requireTabId(values.tabId);
+ return this.tabScope.runTabOperation(tabId, async () => {
+ await this.ensureStillControllableLocked(tabId);
+ if (this.attachedTabs.has(tabId)) {
+ return {};
+ }
+ try {
+ await chrome.debugger.attach({ tabId }, "1.3");
+ } catch {
+ await this.ensureStillControllableLocked(tabId);
+ throw new ProtocolError(
+ ERROR_CODES.CDP_ERROR,
+ "Chrome could not attach the debugger to this tab.",
+ );
+ }
+ this.attachedTabs.add(tabId);
+ this.onAttachedChange();
+ await this.ensureStillControllableLocked(tabId);
+ return {};
+ });
+ }
+
+ async detach(args) {
+ const values = requireArgs(args);
+ const tabId = requireTabId(values.tabId);
+ return this.tabScope.runTabOperation(tabId, async () => {
+ await this.tabScope.assertScoped(tabId);
+ if (!this.attachedTabs.has(tabId)) {
+ throw new ProtocolError(
+ ERROR_CODES.DEBUGGER_DETACHED,
+ "The debugger is not attached to this tab.",
+ );
+ }
+ try {
+ await chrome.debugger.detach({ tabId });
+ } catch {
+ throw new ProtocolError(
+ ERROR_CODES.DEBUGGER_DETACHED,
+ "The debugger is not attached to this tab.",
+ );
+ }
+ this.attachedTabs.delete(tabId);
+ this.onAttachedChange();
+ return {};
+ });
+ }
+
+ async send(args) {
+ const values = requireArgs(args);
+ const tabId = requireTabId(values.tabId);
+ return this.tabScope.runTabOperation(tabId, async () => {
+ await this.ensureStillControllableLocked(tabId);
+ if (!this.attachedTabs.has(tabId)) {
+ throw new ProtocolError(
+ ERROR_CODES.DEBUGGER_DETACHED,
+ "The debugger is not attached to this tab.",
+ );
+ }
+ if (
+ typeof values.method !== "string" ||
+ !this.isMethodAllowed(values.method, values.params)
+ ) {
+ throw new ProtocolError(
+ ERROR_CODES.CDP_METHOD_NOT_ALLOWED,
+ "The requested CDP method is not allowed.",
+ );
+ }
+ if (
+ values.sessionId !== undefined &&
+ typeof values.sessionId !== "string"
+ ) {
+ throw new ProtocolError(
+ ERROR_CODES.CDP_ERROR,
+ "The CDP session identifier is invalid.",
+ );
+ }
+ if (
+ values.params !== undefined &&
+ (values.params === null ||
+ typeof values.params !== "object" ||
+ Array.isArray(values.params))
+ ) {
+ throw new ProtocolError(
+ ERROR_CODES.CDP_ERROR,
+ "CDP parameters must be an object.",
+ );
+ }
+
+ const target =
+ values.sessionId === undefined
+ ? { tabId }
+ : { tabId, sessionId: values.sessionId };
+ let result;
+ try {
+ result = await chrome.debugger.sendCommand(
+ target,
+ values.method,
+ values.params ?? {},
+ );
+ } catch {
+ await this.ensureStillControllableLocked(tabId);
+ throw new ProtocolError(
+ ERROR_CODES.CDP_ERROR,
+ "Chrome rejected the CDP command.",
+ );
+ }
+ await this.ensureStillControllableLocked(tabId);
+ return { result: result ?? {} };
+ });
+ }
+
+ async detachIfAttached(tabId) {
+ return this.tabScope.runTabOperation(tabId, () =>
+ this.detachIfAttachedLocked(tabId),
+ );
+ }
+
+ async detachIfAttachedLocked(tabId) {
+ if (!this.attachedTabs.has(tabId)) {
+ return;
+ }
+ try {
+ await chrome.debugger.detach({ tabId });
+ } catch {
+ this.attachedTabs.delete(tabId);
+ this.onAttachedChange();
+ return;
+ }
+ this.attachedTabs.delete(tabId);
+ this.onAttachedChange();
+ }
+
+ async ensureStillControllableLocked(tabId) {
+ try {
+ await this.tabScope.assertControllableLocked(tabId);
+ } catch (error) {
+ await this.detachIfAttachedLocked(tabId);
+ throw error;
+ }
+ }
+
+ isMethodAllowed(method, params) {
+ if (
+ DENIED_CDP_METHODS.has(method) ||
+ (method === "Network.getCookies" &&
+ params !== null &&
+ typeof params === "object" &&
+ Object.prototype.hasOwnProperty.call(params, "urls"))
+ ) {
+ return false;
+ }
+ const separator = method.indexOf(".");
+ return (
+ separator > 0 && ALLOWED_CDP_PREFIXES.has(method.slice(0, separator))
+ );
+ }
+
+ async handleDebuggerEvent(source, method, params) {
+ if (
+ !Number.isInteger(source.tabId) ||
+ !this.tabScope.isScoped(source.tabId)
+ ) {
+ return;
+ }
+ const eventParams = { tabId: source.tabId, method, params: params ?? {} };
+ if (typeof source.sessionId === "string") {
+ eventParams.sessionId = source.sessionId;
+ }
+ this.sendEvent(EVENTS.DEBUGGER_EVENT, eventParams);
+ }
+
+ async handleDebuggerDetach(source, reason) {
+ if (!Number.isInteger(source.tabId)) {
+ return;
+ }
+ await this.tabScope.runTabOperation(source.tabId, async () => {
+ this.attachedTabs.delete(source.tabId);
+ this.onAttachedChange();
+ this.sendEvent(EVENTS.DEBUGGER_DETACHED, {
+ tabId: source.tabId,
+ reason: typeof reason === "string" ? reason : "unknown",
+ });
+ await this.tabScope.handleDebuggerDetachLocked(source.tabId);
+ });
+ }
+}
diff --git a/skyvern/browser_extension/extension/icons/icon128.png b/skyvern/browser_extension/extension/icons/icon128.png
new file mode 100644
index 000000000..bc5624dc3
Binary files /dev/null and b/skyvern/browser_extension/extension/icons/icon128.png differ
diff --git a/skyvern/browser_extension/extension/icons/icon16.png b/skyvern/browser_extension/extension/icons/icon16.png
new file mode 100644
index 000000000..539ac0c32
Binary files /dev/null and b/skyvern/browser_extension/extension/icons/icon16.png differ
diff --git a/skyvern/browser_extension/extension/icons/icon48.png b/skyvern/browser_extension/extension/icons/icon48.png
new file mode 100644
index 000000000..98820aa21
Binary files /dev/null and b/skyvern/browser_extension/extension/icons/icon48.png differ
diff --git a/skyvern/browser_extension/extension/manifest.json b/skyvern/browser_extension/extension/manifest.json
new file mode 100644
index 000000000..77686c502
--- /dev/null
+++ b/skyvern/browser_extension/extension/manifest.json
@@ -0,0 +1,36 @@
+{
+ "manifest_version": 3,
+ "name": "Skyvern Agent",
+ "version": "0.1.0",
+ "description": "Connects explicitly shared Chrome tabs to a local Skyvern browser session.",
+ "minimum_chrome_version": "125",
+ "key": "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoSTVDWJdVfrESGVbDh+BU9SLzY0aSnGb3HmW8ZnlRm43Mo9NmIoU4zhB/pbVcoK//0QMIiGzj22242cdwCfcl7X70bJHWWYG33US9FOTPdkxqviv73Jgi0z+XM/TDGe1dUU0ztz1QsfuMFu2MT9Yk1AT5r1507oERuNXo32BzNr5V7S/Ja8OZeRlYW4x7m3hA+OAQiU/0jE2FfuJVa/xB5Gm54LpxpJ7RxM421EZShp6yJyFxxgAEEMiFuNGXfNVY6kMLlpsQ1QiKTo+8I8RJ4DLcb2/bKteCk1U0JqoV+X9nj5sZBA7UmkBmp2+HTkCEIAlKgV+fUjoi3QNrXqPOQIDAQAB",
+ "permissions": [
+ "debugger",
+ "tabs",
+ "tabGroups",
+ "storage",
+ "alarms"
+ ],
+ "background": {
+ "service_worker": "service_worker.js",
+ "type": "module"
+ },
+ "externally_connectable": {
+ "matches": ["http://127.0.0.1/*", "http://localhost/*"]
+ },
+ "action": {
+ "default_title": "Skyvern Agent",
+ "default_popup": "popup.html",
+ "default_icon": {
+ "16": "icons/icon16.png",
+ "48": "icons/icon48.png",
+ "128": "icons/icon128.png"
+ }
+ },
+ "icons": {
+ "16": "icons/icon16.png",
+ "48": "icons/icon48.png",
+ "128": "icons/icon128.png"
+ }
+}
diff --git a/skyvern/browser_extension/extension/pairing_confirm.css b/skyvern/browser_extension/extension/pairing_confirm.css
new file mode 100644
index 000000000..a71375c89
--- /dev/null
+++ b/skyvern/browser_extension/extension/pairing_confirm.css
@@ -0,0 +1,437 @@
+:root {
+ color-scheme: dark light;
+ font-family:
+ ui-sans-serif,
+ system-ui,
+ -apple-system,
+ BlinkMacSystemFont,
+ "Segoe UI",
+ sans-serif;
+ color: #f4f4f5;
+ background: #0b0c0f;
+ font-synthesis: none;
+ --accent: #6d6cf6;
+ --accent-hover: #5f5eea;
+ --surface: #141519;
+ --surface-subtle: #1a1b20;
+ --border: #2b2c33;
+ --border-strong: #3a3b43;
+ --text: #f4f4f5;
+ --muted: #b2b3ba;
+ --subtle: #85868f;
+ --radius: 0.5rem;
+}
+
+* {
+ box-sizing: border-box;
+}
+
+body {
+ min-height: 100vh;
+ margin: 0;
+ display: grid;
+ overflow-x: hidden;
+ color: var(--text);
+ background: #0b0c0f;
+ place-items: center;
+}
+
+.page-shell {
+ width: 100%;
+ padding: 2rem 1rem;
+}
+
+.pairing-card {
+ width: min(38rem, 100%);
+ margin: 0 auto;
+ overflow: hidden;
+ border: 1px solid var(--border);
+ border-radius: var(--radius);
+ background: var(--surface);
+ box-shadow: 0 1rem 2.5rem rgb(0 0 0 / 28%);
+}
+
+.card-header {
+ display: flex;
+ align-items: center;
+ gap: 0.75rem;
+ padding: 1.5rem 1.75rem 0;
+}
+
+.brand-mark {
+ display: grid;
+ flex: none;
+ width: 2rem;
+ height: 2rem;
+ overflow: hidden;
+ border-radius: var(--radius);
+ place-items: center;
+}
+
+.brand-mark img {
+ width: 100%;
+ height: 100%;
+ object-fit: cover;
+}
+
+.brand-copy {
+ display: grid;
+}
+
+.brand-name {
+ font-size: 0.9rem;
+ font-weight: 650;
+ letter-spacing: -0.01em;
+}
+
+.card-body {
+ padding: 2.5rem 1.75rem 1.75rem;
+}
+
+h1 {
+ margin: 0;
+ font-size: clamp(2rem, 6vw, 2.35rem);
+ font-weight: 680;
+ line-height: 1.15;
+ letter-spacing: -0.035em;
+}
+
+.question {
+ max-width: 32rem;
+ min-height: 2.9rem;
+ margin: 0.85rem 0 0;
+ color: var(--muted);
+ font-size: 0.95rem;
+ line-height: 1.55;
+}
+
+.pairing-details {
+ display: grid;
+ grid-template-columns: repeat(2, minmax(0, 1fr));
+ gap: 0.75rem;
+ margin-top: 2rem;
+}
+
+.detail-chip {
+ display: grid;
+ min-width: 0;
+ gap: 0.4rem;
+ padding: 0.9rem 1rem;
+ border: 1px solid var(--border);
+ border-radius: var(--radius);
+ background: var(--surface-subtle);
+}
+
+.detail-copy {
+ display: contents;
+}
+
+.detail-label {
+ color: var(--subtle);
+ font-size: 0.72rem;
+ font-weight: 600;
+ line-height: 1.3;
+}
+
+.detail-chip code {
+ overflow: hidden;
+ color: var(--text);
+ font:
+ 600 0.78rem/1.35 ui-monospace,
+ SFMono-Regular,
+ Menlo,
+ monospace;
+ text-overflow: ellipsis;
+ white-space: nowrap;
+}
+
+#pairing-fingerprint {
+ letter-spacing: 0.12em;
+}
+
+.security-note {
+ margin: 0.8rem 0 1.75rem;
+ color: var(--subtle);
+}
+
+.security-note p {
+ margin: 0;
+ font-size: 0.72rem;
+ line-height: 1.5;
+}
+
+.actions {
+ display: flex;
+ justify-content: flex-end;
+ gap: 0.65rem;
+}
+
+button {
+ font: inherit;
+}
+
+.primary-button,
+.secondary-button {
+ display: inline-flex;
+ width: auto;
+ min-height: 2.75rem;
+ align-items: center;
+ justify-content: center;
+ gap: 0.55rem;
+ padding: 0.7rem 1rem;
+ border-radius: var(--radius);
+ font-size: 0.875rem;
+ font-weight: 650;
+ cursor: pointer;
+ transition:
+ border-color 150ms ease,
+ background 150ms ease,
+ box-shadow 150ms ease;
+}
+
+.primary-button {
+ border: 1px solid var(--accent);
+ color: white;
+ background: var(--accent);
+ box-shadow: 0 0.25rem 0.7rem rgb(0 0 0 / 24%);
+}
+
+.secondary-button {
+ border: 1px solid var(--border-strong);
+ color: var(--text);
+ background: transparent;
+}
+
+.primary-button:hover:not(:disabled) {
+ background: var(--accent-hover);
+ box-shadow: 0 0.3rem 0.85rem rgb(0 0 0 / 28%);
+}
+
+.secondary-button:hover:not(:disabled) {
+ background: var(--surface-subtle);
+}
+
+.primary-button:focus-visible,
+.secondary-button:focus-visible,
+.copy-chip:focus-visible {
+ outline: 2px solid var(--text);
+ outline-offset: 3px;
+}
+
+.primary-button:disabled,
+.secondary-button:disabled {
+ cursor: not-allowed;
+ opacity: 0.55;
+}
+
+[data-state="approving"] .primary-button:disabled {
+ cursor: wait;
+ opacity: 0.8;
+}
+
+.spinner {
+ display: none;
+ width: 1rem;
+ height: 1rem;
+ border: 2px solid rgb(255 255 255 / 35%);
+ border-top-color: white;
+ border-radius: 50%;
+ animation: spin 700ms linear infinite;
+}
+
+[data-state="approving"] .spinner {
+ display: block;
+}
+
+.result-panel {
+ display: grid;
+ gap: 0.7rem;
+ margin-top: 1.5rem;
+ padding-top: 1rem;
+ border-top: 1px solid var(--border);
+}
+
+.result-panel[hidden] {
+ display: none;
+}
+
+.result-heading {
+ display: flex;
+ align-items: center;
+ gap: 0.65rem;
+}
+
+.result-icon {
+ display: grid;
+ flex: none;
+ width: 1.5rem;
+ height: 1.5rem;
+ color: var(--muted);
+ place-items: center;
+}
+
+.result-icon svg {
+ width: 1rem;
+ height: 1rem;
+}
+
+.result-icon .success-icon,
+.result-icon .neutral-icon {
+ display: none;
+}
+
+.result-title {
+ margin: 0;
+ font-size: 0.9rem;
+ font-weight: 650;
+}
+
+.result-message {
+ margin: 0;
+ color: var(--muted);
+ font-size: 0.78rem;
+ line-height: 1.55;
+}
+
+[data-state="success"] .result-icon .error-icon,
+[data-state="cancelled"] .result-icon .error-icon {
+ display: none;
+}
+
+[data-state="success"] .result-icon .success-icon,
+[data-state="cancelled"] .result-icon .neutral-icon {
+ display: block;
+}
+
+[data-state="success"] .actions,
+[data-state="cancelled"] .actions,
+[data-state="error"] .actions[data-terminal="true"] {
+ display: none;
+}
+
+.recovery {
+ display: grid;
+ gap: 0.4rem;
+}
+
+.recovery[hidden] {
+ display: none;
+}
+
+.recovery-label {
+ color: var(--subtle);
+ font-size: 0.7rem;
+ font-weight: 600;
+}
+
+.copy-chip {
+ display: flex;
+ width: fit-content;
+ max-width: 100%;
+ min-width: 0;
+ align-items: center;
+ justify-content: space-between;
+ gap: 0.75rem;
+ padding: 0.65rem 0.75rem;
+ border: 1px solid var(--border);
+ border-radius: var(--radius);
+ color: var(--text);
+ background: var(--surface-subtle);
+ cursor: copy;
+}
+
+.copy-chip:hover {
+ border-color: var(--border-strong);
+ background: #202126;
+}
+
+.copy-chip code {
+ overflow: hidden;
+ font:
+ 600 0.7rem/1.4 ui-monospace,
+ SFMono-Regular,
+ Menlo,
+ monospace;
+ text-overflow: ellipsis;
+ white-space: nowrap;
+}
+
+.copy-label {
+ flex: none;
+ color: var(--muted);
+ font-size: 0.7rem;
+ font-weight: 650;
+}
+
+.fine-print {
+ margin: 1.25rem 0 0;
+ color: var(--subtle);
+ font-size: 0.7rem;
+ line-height: 1.5;
+ text-align: left;
+}
+
+@keyframes spin {
+ to {
+ transform: rotate(360deg);
+ }
+}
+
+@media (max-width: 34rem) {
+ .page-shell {
+ padding: 1rem;
+ }
+
+ .card-header {
+ padding: 1rem 1.1rem 0;
+ }
+
+ .card-body {
+ padding: 2rem 1.1rem 1.1rem;
+ }
+
+ .pairing-details {
+ grid-template-columns: 1fr;
+ }
+
+ .actions {
+ flex-wrap: wrap;
+ }
+}
+
+@media (prefers-reduced-motion: reduce) {
+ *,
+ *::before,
+ *::after {
+ scroll-behavior: auto !important;
+ transition: none !important;
+ animation-duration: 0.01ms !important;
+ animation-iteration-count: 1 !important;
+ }
+}
+
+@media (prefers-color-scheme: light) {
+ :root {
+ color: #202124;
+ background: #f5f5f7;
+ --surface: #ffffff;
+ --surface-subtle: #f5f5f7;
+ --border: #dedfe3;
+ --border-strong: #c9cbd1;
+ --text: #202124;
+ --muted: #5f6169;
+ --subtle: #777982;
+ }
+
+ body {
+ background: #f5f5f7;
+ }
+
+ .pairing-card {
+ box-shadow: 0 1rem 2.5rem rgb(32 33 36 / 10%);
+ }
+
+ .copy-chip:hover {
+ background: #ececef;
+ }
+}
diff --git a/skyvern/browser_extension/extension/pairing_confirm.html b/skyvern/browser_extension/extension/pairing_confirm.html
new file mode 100644
index 000000000..cf737c3cb
--- /dev/null
+++ b/skyvern/browser_extension/extension/pairing_confirm.html
@@ -0,0 +1,139 @@
+<!doctype html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1" />
+ <title>Pair Skyvern Agent</title>
+ <link rel="stylesheet" href="pairing_confirm.css" />
+ </head>
+ <body data-state="loading">
+ <div class="page-shell">
+ <main class="pairing-card" aria-labelledby="pairing-title">
+ <header class="card-header">
+ <span class="brand-mark" aria-hidden="true">
+ <img src="icons/icon128.png" alt="" />
+ </span>
+ <span class="brand-copy">
+ <span class="brand-name">Skyvern Agent</span>
+ </span>
+ </header>
+
+ <div class="card-body">
+ <section class="intro">
+ <h1 id="pairing-title">Pair Skyvern Agent</h1>
+ <p id="pairing-question" class="question">
+ Loading the local pairing offer…
+ </p>
+ </section>
+
+ <div class="pairing-details" aria-label="Pairing details">
+ <div class="detail-chip">
+ <span class="detail-copy">
+ <span class="detail-label">Local server</span>
+ <code id="pairing-port">127.0.0.1:-----</code>
+ </span>
+ </div>
+
+ <div class="detail-chip">
+ <span class="detail-copy">
+ <span class="detail-label">Token fingerprint</span>
+ <code id="pairing-fingerprint">--------</code>
+ </span>
+ </div>
+ </div>
+
+ <div class="security-note">
+ <p>Confirm the fingerprint matches the local server you started.</p>
+ </div>
+
+ <div class="actions">
+ <button
+ id="cancel-button"
+ class="secondary-button"
+ type="button"
+ disabled
+ >
+ Cancel
+ </button>
+ <button
+ id="approve-button"
+ class="primary-button"
+ type="button"
+ disabled
+ >
+ <span class="spinner" aria-hidden="true"></span>
+ <span id="approve-label">Approve pairing</span>
+ </button>
+ </div>
+
+ <section
+ id="pairing-result"
+ class="result-panel"
+ role="status"
+ aria-live="polite"
+ hidden
+ >
+ <div class="result-heading">
+ <span class="result-icon" aria-hidden="true">
+ <svg
+ class="success-icon"
+ viewBox="0 0 24 24"
+ fill="none"
+ stroke="currentColor"
+ stroke-width="2.4"
+ stroke-linecap="round"
+ stroke-linejoin="round"
+ >
+ <path d="m7 12.5 3.2 3.2L17.5 8.5" />
+ </svg>
+ <svg
+ class="error-icon"
+ viewBox="0 0 24 24"
+ fill="none"
+ stroke="currentColor"
+ stroke-width="2"
+ stroke-linecap="round"
+ >
+ <path d="M12 8v5M12 17h.01" />
+ <circle cx="12" cy="12" r="9" />
+ </svg>
+ <svg
+ class="neutral-icon"
+ viewBox="0 0 24 24"
+ fill="none"
+ stroke="currentColor"
+ stroke-width="2"
+ stroke-linecap="round"
+ >
+ <path d="M7 12h10" />
+ <circle cx="12" cy="12" r="9" />
+ </svg>
+ </span>
+ <h2 id="result-title" class="result-title"></h2>
+ </div>
+ <p id="result-message" class="result-message"></p>
+ <div id="recovery" class="recovery" hidden>
+ <span class="recovery-label">Start a new pairing link</span>
+ <button
+ id="copy-command"
+ class="copy-chip"
+ type="button"
+ aria-label="Copy skyvern browser extension-pair"
+ >
+ <code id="recovery-command"
+ >skyvern browser extension-pair</code
+ >
+ <span id="copy-label" class="copy-label">Copy</span>
+ </button>
+ </div>
+ </section>
+
+ <p class="fine-print">
+ This offer is local to your computer and can be approved once.
+ </p>
+ </div>
+ </main>
+ </div>
+ <script type="module" src="pairing_confirm.js"></script>
+ </body>
+</html>
diff --git a/skyvern/browser_extension/extension/pairing_confirm.js b/skyvern/browser_extension/extension/pairing_confirm.js
new file mode 100644
index 000000000..cf8a0d941
--- /dev/null
+++ b/skyvern/browser_extension/extension/pairing_confirm.js
@@ -0,0 +1,213 @@
+const PENDING_PAIRING_STORAGE_KEY = "pendingPairingOffer";
+const RECOVERY_COMMAND = "skyvern browser extension-pair";
+
+const elements = {
+ question: document.querySelector("#pairing-question"),
+ port: document.querySelector("#pairing-port"),
+ fingerprint: document.querySelector("#pairing-fingerprint"),
+ approveButton: document.querySelector("#approve-button"),
+ approveLabel: document.querySelector("#approve-label"),
+ cancelButton: document.querySelector("#cancel-button"),
+ actions: document.querySelector(".actions"),
+ result: document.querySelector("#pairing-result"),
+ resultTitle: document.querySelector("#result-title"),
+ resultMessage: document.querySelector("#result-message"),
+ recovery: document.querySelector("#recovery"),
+ recoveryCommand: document.querySelector("#recovery-command"),
+ copyButton: document.querySelector("#copy-command"),
+ copyLabel: document.querySelector("#copy-label"),
+};
+
+let renderGeneration = 0;
+let operationInProgress = false;
+
+async function sendMessage(message) {
+ const response = await chrome.runtime.sendMessage(message);
+ if (!response?.ok) {
+ throw new Error(
+ response?.error?.message || "The extension request failed.",
+ );
+ }
+ return response.result;
+}
+
+function isPendingOffer(offer) {
+ return (
+ offer !== null &&
+ typeof offer === "object" &&
+ !Array.isArray(offer) &&
+ Number.isInteger(offer.port) &&
+ offer.port >= 1 &&
+ offer.port <= 65_535 &&
+ typeof offer.token === "string" &&
+ offer.token.length > 0
+ );
+}
+
+async function fingerprint(token) {
+ const digest = await crypto.subtle.digest(
+ "SHA-256",
+ new TextEncoder().encode(token),
+ );
+ return Array.from(new Uint8Array(digest), (byte) =>
+ byte.toString(16).padStart(2, "0"),
+ )
+ .join("")
+ .slice(0, 8);
+}
+
+function showResult({
+ state,
+ title,
+ message,
+ recoverable = false,
+ terminal = false,
+}) {
+ document.body.dataset.state = state;
+ elements.actions.dataset.terminal = String(terminal);
+ elements.result.hidden = false;
+ elements.resultTitle.textContent = title;
+ elements.resultMessage.textContent = message;
+ elements.recovery.hidden = !recoverable;
+}
+
+function clearResult() {
+ document.body.dataset.state = "idle";
+ elements.actions.dataset.terminal = "false";
+ elements.result.hidden = true;
+ elements.resultTitle.textContent = "";
+ elements.resultMessage.textContent = "";
+ elements.recovery.hidden = true;
+}
+
+async function copyRecoveryCommand() {
+ try {
+ await navigator.clipboard.writeText(RECOVERY_COMMAND);
+ } catch (_error) {
+ const range = document.createRange();
+ range.selectNodeContents(elements.recoveryCommand);
+ const selection = globalThis.getSelection();
+ selection.removeAllRanges();
+ selection.addRange(range);
+ document.execCommand("copy");
+ selection.removeAllRanges();
+ }
+ elements.copyLabel.textContent = "Copied";
+ setTimeout(() => {
+ elements.copyLabel.textContent = "Copy";
+ }, 1600);
+}
+
+async function renderPendingOffer() {
+ const generation = ++renderGeneration;
+ const stored = await chrome.storage.session.get(PENDING_PAIRING_STORAGE_KEY);
+ const offer = stored[PENDING_PAIRING_STORAGE_KEY];
+ if (!isPendingOffer(offer)) {
+ elements.question.textContent =
+ "This one-time pairing offer is no longer available.";
+ elements.port.textContent = "Unavailable";
+ elements.fingerprint.textContent = "--------";
+ elements.approveButton.disabled = true;
+ elements.cancelButton.disabled = true;
+ showResult({
+ state: "error",
+ title: "Pairing request expired",
+ message: "Start a new pairing request from your terminal to continue.",
+ recoverable: true,
+ terminal: true,
+ });
+ return;
+ }
+ const tokenFingerprint = await fingerprint(offer.token);
+ if (generation !== renderGeneration) {
+ return;
+ }
+ elements.question.textContent =
+ "Skyvern on this computer wants to drive Chrome through this extension.";
+ elements.port.textContent = `127.0.0.1:${offer.port}`;
+ elements.fingerprint.textContent = tokenFingerprint;
+ elements.approveButton.disabled = false;
+ elements.cancelButton.disabled = false;
+ clearResult();
+}
+
+function disableActions() {
+ elements.approveButton.disabled = true;
+ elements.cancelButton.disabled = true;
+}
+
+function finishBusyState() {
+ elements.approveButton.removeAttribute("aria-busy");
+ elements.approveLabel.textContent = "Approve pairing";
+}
+
+elements.approveButton.addEventListener("click", () => {
+ operationInProgress = true;
+ disableActions();
+ document.body.dataset.state = "approving";
+ elements.approveButton.setAttribute("aria-busy", "true");
+ elements.approveLabel.textContent = "Connecting…";
+ elements.result.hidden = true;
+ void sendMessage({ type: "applyPendingPairing" })
+ .then(() => {
+ finishBusyState();
+ showResult({
+ state: "success",
+ title: "Connected",
+ message:
+ "Skyvern Agent is now paired with your local MCP server. You can close this tab.",
+ terminal: true,
+ });
+ })
+ .catch((error) => {
+ operationInProgress = false;
+ finishBusyState();
+ showResult({
+ state: "error",
+ title: "Connection failed",
+ message: error.message,
+ });
+ elements.approveButton.disabled = false;
+ elements.cancelButton.disabled = false;
+ });
+});
+
+elements.cancelButton.addEventListener("click", () => {
+ operationInProgress = true;
+ disableActions();
+ elements.result.hidden = true;
+ void sendMessage({ type: "cancelPendingPairing" })
+ .then(() => {
+ showResult({
+ state: "cancelled",
+ title: "Pairing cancelled",
+ message: "No connection was made. You can close this tab.",
+ terminal: true,
+ });
+ })
+ .catch((error) => {
+ operationInProgress = false;
+ showResult({
+ state: "error",
+ title: "Couldn’t cancel pairing",
+ message: error.message,
+ });
+ elements.cancelButton.disabled = false;
+ });
+});
+
+elements.copyButton.addEventListener("click", () => {
+ void copyRecoveryCommand();
+});
+
+chrome.storage.onChanged.addListener((changes, areaName) => {
+ if (
+ !operationInProgress &&
+ areaName === "session" &&
+ PENDING_PAIRING_STORAGE_KEY in changes
+ ) {
+ void renderPendingOffer();
+ }
+});
+
+await renderPendingOffer();
diff --git a/skyvern/browser_extension/extension/popup.css b/skyvern/browser_extension/extension/popup.css
new file mode 100644
index 000000000..f6cbd627d
--- /dev/null
+++ b/skyvern/browser_extension/extension/popup.css
@@ -0,0 +1,274 @@
+:root {
+ color-scheme: light dark;
+ font-family:
+ ui-sans-serif,
+ system-ui,
+ -apple-system,
+ BlinkMacSystemFont,
+ "Segoe UI",
+ sans-serif;
+ color: #202033;
+ background: #ffffff;
+}
+
+body {
+ width: 340px;
+ margin: 0;
+}
+
+main {
+ padding: 16px;
+}
+
+header,
+.status,
+li {
+ display: flex;
+ align-items: center;
+}
+
+header,
+li {
+ justify-content: space-between;
+ gap: 12px;
+}
+
+h1 {
+ margin: 0;
+ font-size: 18px;
+}
+
+h2 {
+ margin: 0 0 10px;
+ font-size: 13px;
+}
+
+section {
+ margin-top: 16px;
+ padding-top: 14px;
+ border-top: 1px solid #dddded;
+}
+
+.status {
+ gap: 6px;
+ font-size: 12px;
+}
+
+.status-dot {
+ width: 8px;
+ height: 8px;
+ border-radius: 50%;
+ background: #8a8a9a;
+}
+
+.status-dot.connected {
+ background: #35a66f;
+}
+
+.settings-row {
+ display: grid;
+ grid-template-columns: 88px 1fr;
+ gap: 8px;
+ align-items: center;
+}
+
+.settings {
+ display: grid;
+ gap: 10px;
+}
+
+.advanced-settings {
+ font-size: 12px;
+}
+
+.advanced-settings summary {
+ color: #77778b;
+ cursor: pointer;
+ user-select: none;
+}
+
+.advanced-settings summary:hover {
+ color: #6d6cf6;
+}
+
+.advanced-settings-row {
+ margin-top: 10px;
+}
+
+label {
+ font-size: 12px;
+}
+
+input,
+button {
+ box-sizing: border-box;
+ border: 1px solid #c9c9dc;
+ border-radius: 6px;
+ font: inherit;
+}
+
+input {
+ min-width: 0;
+ padding: 7px 8px;
+ background: transparent;
+}
+
+.token-field {
+ position: relative;
+ min-width: 0;
+}
+
+.token-field input {
+ width: 100%;
+ padding-right: 34px;
+}
+
+.token-visibility {
+ position: absolute;
+ top: 50%;
+ right: 3px;
+ display: grid;
+ width: 28px;
+ height: 28px;
+ padding: 5px;
+ border: 0;
+ background: transparent;
+ place-items: center;
+ transform: translateY(-50%);
+}
+
+.token-visibility svg {
+ width: 16px;
+ height: 16px;
+}
+
+.token-visibility [hidden] {
+ display: none;
+}
+
+button {
+ padding: 7px 10px;
+ color: inherit;
+ background: #f3f3fb;
+ cursor: pointer;
+}
+
+button:hover:not(:disabled) {
+ border-color: #6d6cf6;
+}
+
+button:disabled {
+ cursor: not-allowed;
+ opacity: 0.55;
+}
+
+.primary {
+ color: #ffffff;
+ border-color: #6d6cf6;
+ background: #6d6cf6;
+}
+
+.message {
+ min-height: 16px;
+ margin: 6px 0 0;
+ color: #b43b4a;
+ font-size: 11px;
+}
+
+.settings .message {
+ margin: 0;
+}
+
+ul {
+ display: grid;
+ gap: 8px;
+ margin: 0;
+ padding: 0;
+ list-style: none;
+}
+
+li {
+ min-width: 0;
+}
+
+.tab-title {
+ overflow: hidden;
+ font-size: 12px;
+ text-overflow: ellipsis;
+ white-space: nowrap;
+}
+
+.muted {
+ margin: 0;
+ color: #77778b;
+ font-size: 12px;
+}
+
+.tab-group-hint {
+ margin: 10px 0 0;
+ color: #77778b;
+ font-size: 10px;
+ white-space: nowrap;
+}
+
+.coming-soon {
+ padding-top: 12px;
+}
+
+.coming-soon h2 {
+ margin-bottom: 7px;
+ color: #77778b;
+ font-size: 11px;
+ letter-spacing: 0.04em;
+}
+
+.coming-soon-list {
+ gap: 4px;
+ color: #9999a8;
+ font-size: 11px;
+ cursor: not-allowed;
+ opacity: 0.65;
+}
+
+.coming-soon-list li {
+ justify-content: flex-start;
+ gap: 6px;
+}
+
+@media (prefers-color-scheme: dark) {
+ :root {
+ color: #f2f2fa;
+ background: #1c1c26;
+ }
+
+ section {
+ border-color: #383848;
+ }
+
+ input,
+ button {
+ border-color: #4c4c61;
+ }
+
+ button {
+ background: #30303f;
+ }
+
+ .primary {
+ background: #6d6cf6;
+ }
+
+ .muted {
+ color: #a7a7ba;
+ }
+
+ .advanced-settings summary,
+ .tab-group-hint,
+ .coming-soon h2 {
+ color: #a7a7ba;
+ }
+
+ .coming-soon-list {
+ color: #b9b9c8;
+ }
+}
diff --git a/skyvern/browser_extension/extension/popup.html b/skyvern/browser_extension/extension/popup.html
new file mode 100644
index 000000000..4a98079b6
--- /dev/null
+++ b/skyvern/browser_extension/extension/popup.html
@@ -0,0 +1,119 @@
+<!doctype html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1" />
+ <title>Skyvern Agent</title>
+ <link rel="stylesheet" href="popup.css" />
+ </head>
+ <body>
+ <main>
+ <header>
+ <h1>Skyvern Agent</h1>
+ <div class="status" aria-live="polite">
+ <span id="status-dot" class="status-dot"></span>
+ <span id="status-label">Disconnected</span>
+ </div>
+ </header>
+
+ <section class="settings" aria-label="Agent settings">
+ <div class="settings-row">
+ <label for="pairing-token">Pairing token</label>
+ <div class="token-field">
+ <input
+ id="pairing-token"
+ type="password"
+ autocomplete="off"
+ spellcheck="false"
+ />
+ <button
+ id="pairing-token-visibility"
+ class="token-visibility"
+ type="button"
+ aria-label="Show pairing token"
+ aria-pressed="false"
+ >
+ <svg
+ class="eye-icon"
+ aria-hidden="true"
+ viewBox="0 0 24 24"
+ fill="none"
+ stroke="currentColor"
+ stroke-width="2"
+ >
+ <path d="M2 12s3.5-7 10-7 10 7 10 7-3.5 7-10 7S2 12 2 12Z" />
+ <circle cx="12" cy="12" r="3" />
+ </svg>
+ <svg
+ class="eye-off-icon"
+ aria-hidden="true"
+ viewBox="0 0 24 24"
+ fill="none"
+ stroke="currentColor"
+ stroke-width="2"
+ hidden
+ >
+ <path d="m3 3 18 18" />
+ <path
+ d="M10.6 5.2A11.4 11.4 0 0 1 12 5c6.5 0 10 7 10 7a18 18 0 0 1-2.1 3.2"
+ />
+ <path
+ d="M6.2 6.2C3.5 8.1 2 12 2 12s3.5 7 10 7a10.7 10.7 0 0 0 4.1-.8"
+ />
+ </svg>
+ </button>
+ </div>
+ </div>
+ <details id="advanced-settings" class="advanced-settings">
+ <summary>Advanced settings</summary>
+ <div class="settings-row advanced-settings-row">
+ <label for="bridge-port">Port</label>
+ <input
+ id="bridge-port"
+ type="number"
+ min="1"
+ max="65535"
+ inputmode="numeric"
+ />
+ </div>
+ </details>
+ <button id="connection-button" class="primary" type="button">
+ Connect
+ </button>
+ <p id="connection-error" class="message" role="alert"></p>
+ </section>
+
+ <section aria-labelledby="current-tab-title">
+ <h2 id="current-tab-title">Current tab</h2>
+ <button id="share-button" type="button">
+ Add to Skyvern Controlled
+ </button>
+ <p id="share-reason" class="message"></p>
+ </section>
+
+ <section aria-labelledby="shared-tabs-title">
+ <h2 id="shared-tabs-title">Skyvern Controlled tabs</h2>
+ <ul id="shared-tabs"></ul>
+ <p id="empty-tabs" class="muted">No tabs in Skyvern Controlled.</p>
+ <p class="tab-group-hint">
+ Tabs in the Skyvern Controlled group can be driven by Skyvern.
+ </p>
+ </section>
+
+ <section class="coming-soon" aria-labelledby="coming-soon-title">
+ <h2 id="coming-soon-title">Coming soon</h2>
+ <ul class="coming-soon-list" aria-disabled="true">
+ <li>
+ <span aria-hidden="true">🎬</span> Record actions → create workflows
+ </li>
+ <li><span aria-hidden="true">📋</span> Record SOPs</li>
+ <li><span aria-hidden="true">👤</span> Sync browser profiles</li>
+ <li>
+ <span aria-hidden="true">🔑</span> Push credentials to Skyvern
+ </li>
+ </ul>
+ </section>
+ </main>
+ <script type="module" src="popup.js"></script>
+ </body>
+</html>
diff --git a/skyvern/browser_extension/extension/popup.js b/skyvern/browser_extension/extension/popup.js
new file mode 100644
index 000000000..d60185689
--- /dev/null
+++ b/skyvern/browser_extension/extension/popup.js
@@ -0,0 +1,186 @@
+import { DEFAULT_BRIDGE_PORT, isRestrictedUrl } from "./protocol.js";
+
+const elements = {
+ statusDot: document.querySelector("#status-dot"),
+ statusLabel: document.querySelector("#status-label"),
+ advancedSettings: document.querySelector("#advanced-settings"),
+ port: document.querySelector("#bridge-port"),
+ token: document.querySelector("#pairing-token"),
+ tokenVisibility: document.querySelector("#pairing-token-visibility"),
+ eyeIcon: document.querySelector(".eye-icon"),
+ eyeOffIcon: document.querySelector(".eye-off-icon"),
+ connectionButton: document.querySelector("#connection-button"),
+ connectionError: document.querySelector("#connection-error"),
+ shareButton: document.querySelector("#share-button"),
+ shareReason: document.querySelector("#share-reason"),
+ sharedTabs: document.querySelector("#shared-tabs"),
+ emptyTabs: document.querySelector("#empty-tabs"),
+};
+
+let status = { connected: false, clientAttached: false, enabled: false };
+let activeTab = null;
+let scopedTabs = [];
+let configTimer = null;
+let configLoaded = false;
+
+async function sendMessage(message) {
+ const response = await chrome.runtime.sendMessage(message);
+ if (!response?.ok) {
+ throw new Error(
+ response?.error?.message || "The extension request failed.",
+ );
+ }
+ return response.result;
+}
+
+function loadConfig(config) {
+ elements.port.value = String(config.bridgePort);
+ elements.advancedSettings.open =
+ Number(config.bridgePort) !== DEFAULT_BRIDGE_PORT;
+ elements.token.value =
+ typeof config.pairingToken === "string" ? config.pairingToken : "";
+ configLoaded = true;
+}
+
+async function saveConfig() {
+ window.clearTimeout(configTimer);
+ configTimer = null;
+ await sendMessage({
+ type: "setConfig",
+ config: {
+ bridgePort: Number(elements.port.value),
+ pairingToken: elements.token.value,
+ },
+ });
+}
+
+function scheduleConfigSave() {
+ window.clearTimeout(configTimer);
+ configTimer = window.setTimeout(() => {
+ void saveConfig().catch(showConnectionError);
+ }, 250);
+}
+
+function showConnectionError(error) {
+ elements.connectionError.textContent =
+ error instanceof Error ? error.message : "The extension request failed.";
+}
+
+function setTokenVisibility(visible) {
+ elements.token.type = visible ? "text" : "password";
+ elements.tokenVisibility.setAttribute("aria-pressed", String(visible));
+ elements.tokenVisibility.setAttribute(
+ "aria-label",
+ visible ? "Hide pairing token" : "Show pairing token",
+ );
+ elements.eyeIcon.hidden = visible;
+ elements.eyeOffIcon.hidden = !visible;
+}
+
+function renderStatus() {
+ const label = status.clientAttached
+ ? "Client attached"
+ : status.connected
+ ? "Connected"
+ : "Disconnected";
+ elements.statusLabel.textContent = label;
+ elements.statusDot.className = `status-dot ${status.connected ? "connected" : ""}`;
+ elements.connectionButton.textContent = status.enabled
+ ? "Disconnect"
+ : "Connect";
+}
+
+function renderCurrentTab() {
+ const restricted =
+ activeTab === null ||
+ isRestrictedUrl(activeTab.pendingUrl || activeTab.url || "");
+ const alreadyShared =
+ activeTab !== null && scopedTabs.some((tab) => tab.tabId === activeTab.id);
+ elements.shareButton.disabled = restricted || alreadyShared;
+ if (restricted) {
+ elements.shareReason.textContent =
+ "Chrome does not allow adding this page.";
+ } else if (alreadyShared) {
+ elements.shareReason.textContent =
+ "This tab is already in Skyvern Controlled.";
+ } else {
+ elements.shareReason.textContent = "";
+ }
+}
+
+function renderScopedTabs() {
+ elements.sharedTabs.replaceChildren();
+ elements.emptyTabs.hidden = scopedTabs.length > 0;
+ for (const tab of scopedTabs) {
+ const item = document.createElement("li");
+ const text = document.createElement("span");
+ const button = document.createElement("button");
+ text.className = "tab-title";
+ text.textContent = tab.title || tab.url || `Tab ${tab.tabId}`;
+ text.title = tab.url || "";
+ button.type = "button";
+ button.textContent = "Remove from Skyvern Controlled";
+ button.addEventListener("click", () => {
+ button.disabled = true;
+ void sendMessage({ type: "unshareTab", tabId: tab.tabId })
+ .then(refresh)
+ .catch((error) => {
+ button.disabled = false;
+ showConnectionError(error);
+ });
+ });
+ item.append(text, button);
+ elements.sharedTabs.append(item);
+ }
+}
+
+async function refresh() {
+ const [nextStatus, scopedResult, activeTabs] = await Promise.all([
+ sendMessage({ type: "getStatus" }),
+ sendMessage({ type: "listScoped" }),
+ chrome.tabs.query({ active: true, currentWindow: true }),
+ ]);
+ status = nextStatus;
+ if (!configLoaded) {
+ loadConfig(nextStatus);
+ }
+ scopedTabs = scopedResult.tabs;
+ activeTab = activeTabs[0] ?? null;
+ elements.connectionError.textContent = status.lastError || "";
+ renderStatus();
+ renderCurrentTab();
+ renderScopedTabs();
+}
+
+elements.port.addEventListener("input", scheduleConfigSave);
+elements.token.addEventListener("input", scheduleConfigSave);
+elements.tokenVisibility.addEventListener("click", () => {
+ setTokenVisibility(elements.token.type === "password");
+});
+elements.connectionButton.addEventListener("click", () => {
+ elements.connectionButton.disabled = true;
+ elements.connectionError.textContent = "";
+ const action = status.enabled
+ ? sendMessage({ type: "disconnect" })
+ : saveConfig().then(() => sendMessage({ type: "connect" }));
+ void action
+ .then(refresh)
+ .catch(showConnectionError)
+ .finally(() => {
+ elements.connectionButton.disabled = false;
+ });
+});
+elements.shareButton.addEventListener("click", () => {
+ if (activeTab === null || !Number.isInteger(activeTab.id)) {
+ return;
+ }
+ elements.shareButton.disabled = true;
+ void sendMessage({ type: "shareTab", tabId: activeTab.id })
+ .then(refresh)
+ .catch(showConnectionError);
+});
+
+await refresh();
+window.setInterval(() => {
+ void refresh().catch(showConnectionError);
+}, 2_000);
diff --git a/skyvern/browser_extension/extension/protocol.js b/skyvern/browser_extension/extension/protocol.js
new file mode 100644
index 000000000..c21a5132b
--- /dev/null
+++ b/skyvern/browser_extension/extension/protocol.js
@@ -0,0 +1,153 @@
+export const PROTOCOL_VERSION = 1;
+export const DEFAULT_BRIDGE_PORT = 19777;
+export const BRIDGE_ALARM_NAME = "skyvern-bridge-reconnect";
+
+export const MESSAGE_TYPES = Object.freeze({
+ AUTH_CHALLENGE: "auth.challenge",
+ AUTH_PROOF: "auth.proof",
+ AUTH_OK: "auth.ok",
+ REQUEST: "request",
+ RESPONSE: "response",
+ EVENT: "event",
+ PING: "ping",
+ PONG: "pong",
+});
+
+export const OPS = Object.freeze({
+ DEBUGGER_ATTACH: "debugger.attach",
+ DEBUGGER_DETACH: "debugger.detach",
+ DEBUGGER_SEND: "debugger.send",
+ SHARE_TAB: "shareTab",
+ UNSHARE_TAB: "unshareTab",
+ TABS_CREATE: "tabs.create",
+ TABS_REMOVE: "tabs.remove",
+ TABS_ACTIVATE: "tabs.activate",
+ TABS_LIST: "tabs.list",
+});
+
+export const EVENTS = Object.freeze({
+ EXTENSION_HELLO: "extension.hello",
+ DEBUGGER_EVENT: "debugger.event",
+ DEBUGGER_DETACHED: "debugger.detached",
+ SCOPE_TAB_ADDED: "scope.tabAdded",
+ SCOPE_TAB_REMOVED: "scope.tabRemoved",
+ TABS_CREATED: "tabs.created",
+});
+
+export const ERROR_CODES = Object.freeze({
+ AUTH_FAILED: "AUTH_FAILED",
+ OP_NOT_ALLOWED: "OP_NOT_ALLOWED",
+ TAB_NOT_FOUND: "TAB_NOT_FOUND",
+ TAB_NOT_SCOPED: "TAB_NOT_SCOPED",
+ RESTRICTED_URL: "RESTRICTED_URL",
+ DEBUGGER_DETACHED: "DEBUGGER_DETACHED",
+ CDP_METHOD_NOT_ALLOWED: "CDP_METHOD_NOT_ALLOWED",
+ CDP_ERROR: "CDP_ERROR",
+ INTERNAL: "INTERNAL",
+});
+
+export const ALLOWED_CDP_PREFIXES = new Set([
+ "Accessibility",
+ "Animation",
+ "CSS",
+ "Console",
+ "DOM",
+ "DOMDebugger",
+ "DOMSnapshot",
+ "DOMStorage",
+ "Debugger",
+ "Emulation",
+ "Fetch",
+ "IO",
+ "Input",
+ "Inspector",
+ "Log",
+ "Network",
+ "Overlay",
+ "Page",
+ "Performance",
+ "Profiler",
+ "Runtime",
+ "Security",
+ "Storage",
+ "Target",
+]);
+
+export const DENIED_CDP_METHODS = new Set([
+ "Network.getAllCookies",
+ "Network.clearBrowserCookies",
+ "Network.clearBrowserCache",
+ "Storage.getCookies",
+ "Storage.setCookies",
+ "Storage.clearCookies",
+]);
+
+export class ProtocolError extends Error {
+ constructor(code, message) {
+ super(message);
+ this.name = "ProtocolError";
+ this.code = code;
+ }
+}
+
+export function isRestrictedUrl(url) {
+ if (typeof url !== "string" || url.length === 0) {
+ return false;
+ }
+
+ const normalized = url.trim().toLowerCase();
+ if (normalized === "about:blank") {
+ return false;
+ }
+ if (
+ normalized.startsWith("chrome://") ||
+ normalized.startsWith("chrome-untrusted://") ||
+ normalized.startsWith("chrome-extension://") ||
+ normalized.startsWith("devtools://") ||
+ normalized.startsWith("edge://") ||
+ normalized.startsWith("about:") ||
+ normalized.startsWith("file://")
+ ) {
+ return true;
+ }
+
+ try {
+ const hostname = new URL(url).hostname.toLowerCase().replace(/\.$/, "");
+ return hostname === "chromewebstore.google.com";
+ } catch {
+ return false;
+ }
+}
+
+export function requireArgs(args) {
+ if (args === null || typeof args !== "object" || Array.isArray(args)) {
+ throw new ProtocolError(
+ ERROR_CODES.INTERNAL,
+ "Request arguments must be an object.",
+ );
+ }
+ return args;
+}
+
+export function requireTabId(value) {
+ if (!Number.isInteger(value) || value < 0) {
+ throw new ProtocolError(
+ ERROR_CODES.TAB_NOT_FOUND,
+ "The requested tab was not found.",
+ );
+ }
+ return value;
+}
+
+export function protocolErrorEnvelope(error) {
+ if (
+ error instanceof ProtocolError &&
+ Object.values(ERROR_CODES).includes(error.code)
+ ) {
+ return { code: error.code, message: error.message };
+ }
+ return {
+ code: ERROR_CODES.INTERNAL,
+ message: "The extension operation failed.",
+ };
+}
diff --git a/skyvern/browser_extension/extension/service_worker.js b/skyvern/browser_extension/extension/service_worker.js
new file mode 100644
index 000000000..f170ae943
--- /dev/null
+++ b/skyvern/browser_extension/extension/service_worker.js
@@ -0,0 +1,274 @@
+import { BridgeConnection } from "./bridge_connection.js";
+import { DebuggerRouter } from "./debugger_router.js";
+import {
+ ERROR_CODES,
+ EVENTS,
+ OPS,
+ PROTOCOL_VERSION,
+ ProtocolError,
+ requireArgs,
+} from "./protocol.js";
+import { TabScope } from "./tab_scope.js";
+
+const PENDING_PAIRING_STORAGE_KEY = "pendingPairingOffer";
+const PAIRING_CONFIRM_PATH = "pairing_confirm.html";
+
+let debuggerRouter;
+let pendingPairingOffer = null;
+let pairingOfferQueue = Promise.resolve();
+
+const bridge = new BridgeConnection({
+ onRequest: (op, args) => dispatchRequest(op, args),
+ onAuthenticated: () => sendHello(),
+ onStateChange: () => updateActionState(),
+});
+
+const tabScope = new TabScope({
+ sendEvent: (event, params) => bridge.sendEvent(event, params),
+});
+
+debuggerRouter = new DebuggerRouter({
+ tabScope,
+ sendEvent: (event, params) => bridge.sendEvent(event, params),
+ onAttachedChange: () => updateActionState(),
+});
+tabScope.setDebuggerRouter(debuggerRouter);
+
+const handlers = new Map([
+ [OPS.DEBUGGER_ATTACH, (args) => debuggerRouter.attach(args)],
+ [OPS.DEBUGGER_DETACH, (args) => debuggerRouter.detach(args)],
+ [OPS.DEBUGGER_SEND, (args) => debuggerRouter.send(args)],
+ [OPS.TABS_CREATE, (args) => tabScope.create(args)],
+ [OPS.TABS_REMOVE, (args) => tabScope.remove(args)],
+ [OPS.TABS_ACTIVATE, (args) => tabScope.activate(args)],
+ [
+ OPS.TABS_LIST,
+ (args) => {
+ requireArgs(args);
+ return tabScope.list();
+ },
+ ],
+]);
+
+async function dispatchRequest(op, args) {
+ const handler = handlers.get(op);
+ if (handler === undefined) {
+ throw new ProtocolError(
+ ERROR_CODES.OP_NOT_ALLOWED,
+ "The requested operation is not allowed.",
+ );
+ }
+ return handler(args);
+}
+
+async function sendHello() {
+ bridge.sendEvent(EVENTS.EXTENSION_HELLO, {
+ extensionVersion: chrome.runtime.getManifest().version,
+ scopedTabs: await tabScope.helloTabs(),
+ });
+}
+
+async function updateActionState() {
+ const connected = bridge.getStatus().connected;
+ await chrome.action.setBadgeText({ text: connected ? "ON" : "" });
+ await chrome.action.setBadgeBackgroundColor({ color: "#6D6CF6" });
+}
+
+async function popupStatus() {
+ const status = bridge.getStatus();
+ return {
+ ...status,
+ clientAttached: status.connected && debuggerRouter.hasAttachedTabs(),
+ protocolVersion: PROTOCOL_VERSION,
+ };
+}
+
+function parsePairingOffer(message, sender) {
+ if (
+ message === null ||
+ typeof message !== "object" ||
+ Array.isArray(message) ||
+ message.type !== "skyvern.pairingOffer" ||
+ message.v !== 1 ||
+ !Number.isInteger(message.port) ||
+ message.port < 1 ||
+ message.port > 65_535 ||
+ typeof message.token !== "string" ||
+ message.token.length === 0
+ ) {
+ return null;
+ }
+
+ let senderUrl;
+ try {
+ senderUrl = new URL(sender.url);
+ } catch {
+ return null;
+ }
+ if (
+ senderUrl.protocol !== "http:" ||
+ (senderUrl.hostname !== "127.0.0.1" &&
+ senderUrl.hostname !== "localhost") ||
+ senderUrl.port === "" ||
+ Number(senderUrl.port) !== message.port
+ ) {
+ return null;
+ }
+ return { port: message.port, token: message.token };
+}
+
+async function focusPairingConfirmation() {
+ const confirmationUrl = chrome.runtime.getURL(PAIRING_CONFIRM_PATH);
+ const tabs = await chrome.tabs.query({});
+ const existingTab = tabs.find((tab) => tab.url === confirmationUrl);
+ if (Number.isInteger(existingTab?.id)) {
+ await chrome.tabs.update(existingTab.id, { active: true });
+ if (
+ Number.isInteger(existingTab.windowId) &&
+ typeof chrome.windows?.update === "function"
+ ) {
+ await chrome.windows.update(existingTab.windowId, { focused: true });
+ }
+ return;
+ }
+ await chrome.tabs.create({ active: true, url: confirmationUrl });
+}
+
+async function acceptPairingOffer(offer) {
+ pendingPairingOffer = offer;
+ await chrome.storage.session.set({ [PENDING_PAIRING_STORAGE_KEY]: offer });
+ await focusPairingConfirmation();
+}
+
+function enqueuePairingOperation(operation) {
+ const result = pairingOfferQueue
+ .catch(() => undefined)
+ .then(() => operation());
+ pairingOfferQueue = result;
+ return result;
+}
+
+async function readPendingPairingOffer() {
+ if (pendingPairingOffer !== null) {
+ return pendingPairingOffer;
+ }
+ const stored = await chrome.storage.session.get(PENDING_PAIRING_STORAGE_KEY);
+ const offer = stored[PENDING_PAIRING_STORAGE_KEY];
+ if (
+ offer === null ||
+ typeof offer !== "object" ||
+ Array.isArray(offer) ||
+ !Number.isInteger(offer.port) ||
+ offer.port < 1 ||
+ offer.port > 65_535 ||
+ typeof offer.token !== "string" ||
+ offer.token.length === 0
+ ) {
+ return null;
+ }
+ pendingPairingOffer = { port: offer.port, token: offer.token };
+ return pendingPairingOffer;
+}
+
+async function clearPendingPairingOffer() {
+ pendingPairingOffer = null;
+ await chrome.storage.session.remove(PENDING_PAIRING_STORAGE_KEY);
+}
+
+async function applyPendingPairing() {
+ const offer = await readPendingPairingOffer();
+ if (offer === null) {
+ throw new ProtocolError(
+ ERROR_CODES.INTERNAL,
+ "No pending pairing offer is available.",
+ );
+ }
+ await bridge.setConfig({
+ bridgePort: offer.port,
+ pairingToken: offer.token,
+ });
+ await clearPendingPairingOffer();
+ return bridge.connect();
+}
+
+async function handlePopupMessage(message) {
+ if (
+ message === null ||
+ typeof message !== "object" ||
+ Array.isArray(message)
+ ) {
+ throw new ProtocolError(
+ ERROR_CODES.INTERNAL,
+ "The popup request is invalid.",
+ );
+ }
+ switch (message.type) {
+ case "getStatus":
+ return popupStatus();
+ case "setConfig":
+ return bridge.setConfig(requireArgs(message.config));
+ case "applyPendingPairing":
+ return enqueuePairingOperation(applyPendingPairing);
+ case "cancelPendingPairing":
+ return enqueuePairingOperation(clearPendingPairingOffer);
+ case "connect":
+ return bridge.connect();
+ case "disconnect":
+ return bridge.disconnect();
+ case "shareTab":
+ return tabScope.shareTab(message.tabId);
+ case "unshareTab":
+ return tabScope.unshareTab(message.tabId);
+ case "listScoped":
+ return tabScope.list();
+ default:
+ throw new ProtocolError(
+ ERROR_CODES.OP_NOT_ALLOWED,
+ "The popup operation is not allowed.",
+ );
+ }
+}
+
+chrome.runtime.onMessageExternal.addListener(
+ (message, sender, sendResponse) => {
+ const offer = parsePairingOffer(message, sender);
+ if (offer === null) {
+ sendResponse({ ok: false, error: "invalid_offer" });
+ return false;
+ }
+
+ const acceptance = enqueuePairingOperation(() => acceptPairingOffer(offer));
+ void acceptance
+ .then(() => sendResponse({ ok: true, pending: true }))
+ .catch(() => sendResponse({ ok: false, error: "pairing_failed" }));
+ return true;
+ },
+);
+
+// Chrome fails the whole service worker if module evaluation uses top-level await
+// (event dispatch never starts), so initialization must run as a tracked promise.
+const initialized = (async () => {
+ await tabScope.initialize();
+ await bridge.initialize();
+ await updateActionState();
+})();
+initialized.catch((error) =>
+ console.error("Skyvern bridge initialization failed", error),
+);
+
+chrome.runtime.onMessage.addListener((message, _sender, sendResponse) => {
+ void initialized
+ .catch(() => undefined)
+ .then(() => handlePopupMessage(message))
+ .then((result) => sendResponse({ ok: true, result }))
+ .catch((error) => {
+ const code =
+ error instanceof ProtocolError ? error.code : ERROR_CODES.INTERNAL;
+ const messageText =
+ error instanceof ProtocolError
+ ? error.message
+ : "The extension operation failed.";
+ sendResponse({ ok: false, error: { code, message: messageText } });
+ });
+ return true;
+});
diff --git a/skyvern/browser_extension/extension/tab_scope.js b/skyvern/browser_extension/extension/tab_scope.js
new file mode 100644
index 000000000..ac8677802
--- /dev/null
+++ b/skyvern/browser_extension/extension/tab_scope.js
@@ -0,0 +1,613 @@
+import {
+ ERROR_CODES,
+ EVENTS,
+ ProtocolError,
+ isRestrictedUrl,
+ requireArgs,
+ requireTabId,
+} from "./protocol.js";
+
+const SCOPED_TAB_IDS_KEY = "scopedTabIds";
+const SCOPED_GROUP_IDS_KEY = "scopedTabGroupIds";
+const SKYVERN_GROUP_TITLE = "Skyvern Controlled";
+const SKYVERN_GROUP_COLOR = "purple";
+const TAB_GROUP_ID_NONE = -1;
+const ANY_GROUP_ID = Symbol("anyGroupId");
+
+function tabUrl(tab) {
+ return tab.pendingUrl || tab.url || "";
+}
+
+function isTabRestricted(tab) {
+ return (
+ isRestrictedUrl(tab.pendingUrl || "") || isRestrictedUrl(tab.url || "")
+ );
+}
+
+export class TabScope {
+ constructor({ sendEvent }) {
+ this.sendEvent = sendEvent;
+ this.scopedTabIds = new Set();
+ this.scopedGroupIds = new Map();
+ this.expectedGroupTransitions = new Map();
+ this.tabOperations = new Map();
+ this.debuggerRouter = null;
+ this.ready = new Promise((resolve) => {
+ this.resolveReady = resolve;
+ });
+
+ chrome.tabs.onCreated.addListener((tab) => {
+ void this.handleTabCreated(tab);
+ });
+ chrome.tabs.onRemoved.addListener((tabId) => {
+ void this.handleTabRemoved(tabId);
+ });
+ chrome.tabs.onUpdated.addListener((tabId, changeInfo) => {
+ if (
+ Object.hasOwn(changeInfo, "url") ||
+ Object.hasOwn(changeInfo, "groupId")
+ ) {
+ const expectedGroupTransition = Object.hasOwn(changeInfo, "groupId")
+ ? this.getExpectedGroupTransition(tabId, changeInfo.groupId)
+ : null;
+ void this.handleTabUpdated(tabId, changeInfo, expectedGroupTransition);
+ }
+ });
+ }
+
+ setDebuggerRouter(debuggerRouter) {
+ this.debuggerRouter = debuggerRouter;
+ }
+
+ async initialize() {
+ const stored = await chrome.storage.session.get({
+ [SCOPED_TAB_IDS_KEY]: [],
+ [SCOPED_GROUP_IDS_KEY]: {},
+ });
+ const storedIds = Array.isArray(stored[SCOPED_TAB_IDS_KEY])
+ ? stored[SCOPED_TAB_IDS_KEY]
+ : [];
+ const storedGroups = stored[SCOPED_GROUP_IDS_KEY];
+ for (const tabId of storedIds) {
+ if (Number.isInteger(tabId) && tabId >= 0) {
+ this.scopedTabIds.add(tabId);
+ }
+ }
+ if (
+ storedGroups !== null &&
+ typeof storedGroups === "object" &&
+ !Array.isArray(storedGroups)
+ ) {
+ for (const [tabId, groupId] of Object.entries(storedGroups)) {
+ const numericTabId = Number(tabId);
+ if (this.scopedTabIds.has(numericTabId) && Number.isInteger(groupId)) {
+ this.scopedGroupIds.set(numericTabId, groupId);
+ }
+ }
+ }
+ this.resolveReady();
+ await this.reconcileStoredTabs();
+ }
+
+ isScoped(tabId) {
+ return this.scopedTabIds.has(tabId);
+ }
+
+ async assertScoped(tabId) {
+ await this.ready;
+ if (!this.scopedTabIds.has(tabId)) {
+ throw new ProtocolError(
+ ERROR_CODES.TAB_NOT_SCOPED,
+ "The requested tab is not shared.",
+ );
+ }
+ }
+
+ async assertControllableLocked(tabId) {
+ await this.assertScoped(tabId);
+ let tab;
+ try {
+ tab = await this.getTab(tabId);
+ } catch (error) {
+ if (this.scopedTabIds.has(tabId)) {
+ await this.removeFromScopeLocked(tabId, "closed", true);
+ }
+ throw error;
+ }
+ await this.assertScoped(tabId);
+ if (isTabRestricted(tab)) {
+ await this.removeFromScopeLocked(tabId, "unshared", true);
+ throw new ProtocolError(
+ ERROR_CODES.RESTRICTED_URL,
+ "Chrome does not allow controlling this URL.",
+ );
+ }
+ return tab;
+ }
+
+ async shareTab(tabId) {
+ await this.ready;
+ const validTabId = requireTabId(tabId);
+ return this.runTabOperation(validTabId, async () => {
+ const tab = await this.getTab(validTabId);
+ if (isTabRestricted(tab)) {
+ throw new ProtocolError(
+ ERROR_CODES.RESTRICTED_URL,
+ "Chrome does not allow sharing this URL.",
+ );
+ }
+ if (this.scopedTabIds.has(validTabId)) {
+ return {};
+ }
+ const scopedTab = await this.addToScopeLocked(tab);
+ this.sendEvent(EVENTS.SCOPE_TAB_ADDED, this.publicTab(scopedTab, false));
+ return {};
+ });
+ }
+
+ async unshareTab(tabId) {
+ await this.ready;
+ const validTabId = requireTabId(tabId);
+ return this.runTabOperation(validTabId, async () => {
+ await this.assertScoped(validTabId);
+ await this.removeFromScopeLocked(validTabId, "unshared", true);
+ return {};
+ });
+ }
+
+ async create(args) {
+ const values = requireArgs(args);
+ const url =
+ values.url === undefined || values.url === ""
+ ? "about:blank"
+ : values.url;
+ if (typeof url !== "string" || isRestrictedUrl(url)) {
+ throw new ProtocolError(
+ ERROR_CODES.RESTRICTED_URL,
+ "Chrome does not allow creating this URL.",
+ );
+ }
+ let tab;
+ try {
+ tab = await chrome.tabs.create({ url });
+ } catch {
+ throw new ProtocolError(
+ ERROR_CODES.INTERNAL,
+ "Chrome could not create the tab.",
+ );
+ }
+ if (!Number.isInteger(tab.id)) {
+ throw new ProtocolError(
+ ERROR_CODES.TAB_NOT_FOUND,
+ "Chrome did not return a tab identifier.",
+ );
+ }
+ return this.runTabOperation(tab.id, async () => {
+ const scopedTab = await this.addToScopeLocked(tab);
+ this.sendEvent(EVENTS.SCOPE_TAB_ADDED, this.publicTab(scopedTab, false));
+ return { tabId: tab.id };
+ });
+ }
+
+ async remove(args) {
+ const values = requireArgs(args);
+ const tabId = requireTabId(values.tabId);
+ return this.runTabOperation(tabId, async () => {
+ await this.assertControllableLocked(tabId);
+ try {
+ await chrome.tabs.remove(tabId);
+ } catch {
+ if (!this.scopedTabIds.has(tabId)) {
+ return {};
+ }
+ throw new ProtocolError(
+ ERROR_CODES.TAB_NOT_FOUND,
+ "The requested tab was not found.",
+ );
+ }
+ if (this.scopedTabIds.has(tabId)) {
+ await this.removeFromScopeLocked(tabId, "closed", false);
+ }
+ return {};
+ });
+ }
+
+ async activate(args) {
+ const values = requireArgs(args);
+ const tabId = requireTabId(values.tabId);
+ return this.runTabOperation(tabId, async () => {
+ const tab = await this.assertControllableLocked(tabId);
+ try {
+ await chrome.tabs.update(tabId, { active: true });
+ if (Number.isInteger(tab.windowId)) {
+ await chrome.windows.update(tab.windowId, { focused: true });
+ }
+ } catch {
+ throw new ProtocolError(
+ ERROR_CODES.TAB_NOT_FOUND,
+ "The requested tab was not found.",
+ );
+ }
+ await this.assertControllableLocked(tabId);
+ return {};
+ });
+ }
+
+ async list() {
+ await this.ready;
+ const tabs = await this.collectScopedTabs(true);
+ return { tabs };
+ }
+
+ async helloTabs() {
+ await this.ready;
+ return this.collectScopedTabs(false);
+ }
+
+ async handleDebuggerDetachLocked(tabId) {
+ await this.ready;
+ if (this.scopedTabIds.has(tabId)) {
+ await this.removeFromScopeLocked(tabId, "detached", false);
+ }
+ }
+
+ async handleTabCreated(tab) {
+ await this.ready;
+ if (!Number.isInteger(tab.id) || !Number.isInteger(tab.openerTabId)) {
+ return;
+ }
+ await this.runTabOperation(tab.id, async () => {
+ if (!this.scopedTabIds.has(tab.openerTabId) || isTabRestricted(tab)) {
+ return;
+ }
+ const scopedTab = await this.addToScopeLocked(tab);
+ this.sendEvent(EVENTS.TABS_CREATED, {
+ tabId: scopedTab.id,
+ openerTabId: tab.openerTabId,
+ url: tabUrl(scopedTab),
+ });
+ });
+ }
+
+ async handleTabRemoved(tabId) {
+ await this.ready;
+ await this.runTabOperation(tabId, async () => {
+ this.expectedGroupTransitions.delete(tabId);
+ if (this.scopedTabIds.has(tabId)) {
+ await this.removeFromScopeLocked(tabId, "closed", false);
+ }
+ });
+ }
+
+ async handleTabUpdated(tabId, changeInfo, expectedGroupTransition = null) {
+ await this.ready;
+ await this.runTabOperation(tabId, async () => {
+ try {
+ if (
+ this.scopedTabIds.has(tabId) &&
+ Object.hasOwn(changeInfo, "url") &&
+ isRestrictedUrl(changeInfo.url)
+ ) {
+ await this.removeFromScopeLocked(tabId, "unshared", true);
+ return;
+ }
+ if (
+ !Object.hasOwn(changeInfo, "groupId") ||
+ expectedGroupTransition !== null
+ ) {
+ return;
+ }
+
+ let tab;
+ try {
+ tab = await chrome.tabs.get(tabId);
+ } catch {
+ return;
+ }
+ if (tab.groupId !== changeInfo.groupId) {
+ return;
+ }
+
+ const controlledGroup = await this.getControlledGroup(tab.groupId);
+ if (this.scopedTabIds.has(tabId)) {
+ const expectedGroupId = this.scopedGroupIds.get(tabId);
+ if (tab.groupId === expectedGroupId) {
+ return;
+ }
+ if (controlledGroup !== null) {
+ this.scopedGroupIds.set(tabId, tab.groupId);
+ await this.persistScope();
+ await this.updateControlledGroup(tab.groupId);
+ return;
+ }
+ if (expectedGroupId !== undefined) {
+ await this.removeFromScopeLocked(tabId, "unshared", true);
+ }
+ return;
+ }
+
+ if (controlledGroup === null) {
+ return;
+ }
+ if (isTabRestricted(tab)) {
+ await this.ungroupTabLocked(tabId, tab.groupId);
+ return;
+ }
+
+ await this.updateControlledGroup(tab.groupId);
+ const scopedTab = await this.addGroupedTabToScopeLocked(
+ tab,
+ tab.groupId,
+ );
+ this.sendEvent(
+ EVENTS.SCOPE_TAB_ADDED,
+ this.publicTab(scopedTab, false),
+ );
+ } finally {
+ if (expectedGroupTransition !== null) {
+ this.clearExpectedGroupTransition(tabId, expectedGroupTransition);
+ }
+ }
+ });
+ }
+
+ async addToScopeLocked(tab) {
+ if (!Number.isInteger(tab.id)) {
+ throw new ProtocolError(
+ ERROR_CODES.TAB_NOT_FOUND,
+ "The requested tab was not found.",
+ );
+ }
+ this.scopedTabIds.add(tab.id);
+ await this.persistScope();
+ await this.groupTabLocked(tab);
+ return this.assertControllableLocked(tab.id);
+ }
+
+ async addGroupedTabToScopeLocked(tab, groupId) {
+ if (!Number.isInteger(tab.id)) {
+ throw new ProtocolError(
+ ERROR_CODES.TAB_NOT_FOUND,
+ "The requested tab was not found.",
+ );
+ }
+ this.scopedTabIds.add(tab.id);
+ this.scopedGroupIds.set(tab.id, groupId);
+ await this.persistScope();
+ return this.assertControllableLocked(tab.id);
+ }
+
+ async removeFromScopeLocked(tabId, reason, detach) {
+ if (!this.scopedTabIds.delete(tabId)) {
+ return;
+ }
+ this.expectedGroupTransitions.delete(tabId);
+ const scopedGroupId = this.scopedGroupIds.get(tabId);
+ this.scopedGroupIds.delete(tabId);
+ try {
+ await this.persistScope();
+ } finally {
+ await this.ungroupTabLocked(tabId, scopedGroupId);
+ try {
+ if (detach && this.debuggerRouter !== null) {
+ await this.debuggerRouter.detachIfAttachedLocked(tabId);
+ }
+ } finally {
+ this.sendEvent(EVENTS.SCOPE_TAB_REMOVED, { tabId, reason });
+ }
+ }
+ }
+
+ async groupTabLocked(tab) {
+ const tabId = tab.id;
+ if (!Number.isInteger(tabId) || !Number.isInteger(tab.windowId)) {
+ return;
+ }
+ let groupId;
+ try {
+ const groups = await chrome.tabGroups.query({ windowId: tab.windowId });
+ const existingGroup = groups.find(
+ (group) => group.title === SKYVERN_GROUP_TITLE,
+ );
+ const expectedGroupId = existingGroup?.id ?? ANY_GROUP_ID;
+ const transition = this.expectGroupTransition(tabId, expectedGroupId);
+ let grouped = false;
+ try {
+ groupId = existingGroup
+ ? await chrome.tabs.group({
+ groupId: existingGroup.id,
+ tabIds: [tabId],
+ })
+ : await chrome.tabs.group({ tabIds: [tabId] });
+ grouped = true;
+ } finally {
+ if (!grouped) {
+ this.clearExpectedGroupTransition(tabId, transition);
+ }
+ }
+ } catch {
+ this.scopedGroupIds.delete(tabId);
+ await this.persistScope();
+ return;
+ }
+ this.scopedGroupIds.set(tabId, groupId);
+ await this.persistScope();
+ await this.updateControlledGroup(groupId);
+ }
+
+ async ungroupTabLocked(tabId, scopedGroupId) {
+ if (!Number.isInteger(scopedGroupId)) {
+ return;
+ }
+ let tab;
+ try {
+ tab = await chrome.tabs.get(tabId);
+ } catch {
+ return;
+ }
+ if (tab.groupId !== scopedGroupId) {
+ return;
+ }
+ const transition = this.expectGroupTransition(tabId, TAB_GROUP_ID_NONE);
+ let ungrouped = false;
+ try {
+ await chrome.tabs.ungroup([tabId]);
+ ungrouped = true;
+ } catch {
+ return;
+ } finally {
+ if (!ungrouped) {
+ this.clearExpectedGroupTransition(tabId, transition);
+ }
+ }
+ }
+
+ async getControlledGroup(groupId) {
+ if (!Number.isInteger(groupId) || groupId === TAB_GROUP_ID_NONE) {
+ return null;
+ }
+ try {
+ const group = await chrome.tabGroups.get(groupId);
+ return group.title === SKYVERN_GROUP_TITLE ? group : null;
+ } catch {
+ return null;
+ }
+ }
+
+ async updateControlledGroup(groupId) {
+ try {
+ await chrome.tabGroups.update(groupId, {
+ title: SKYVERN_GROUP_TITLE,
+ color: SKYVERN_GROUP_COLOR,
+ });
+ } catch {
+ return;
+ }
+ }
+
+ expectGroupTransition(tabId, groupId) {
+ const transition = { groupId };
+ this.expectedGroupTransitions.set(tabId, transition);
+ return transition;
+ }
+
+ getExpectedGroupTransition(tabId, groupId) {
+ const transition = this.expectedGroupTransitions.get(tabId);
+ if (
+ transition === undefined ||
+ (transition.groupId !== ANY_GROUP_ID && transition.groupId !== groupId) ||
+ (transition.groupId === ANY_GROUP_ID && groupId === TAB_GROUP_ID_NONE)
+ ) {
+ return null;
+ }
+ return transition;
+ }
+
+ clearExpectedGroupTransition(tabId, transition) {
+ if (this.expectedGroupTransitions.get(tabId) === transition) {
+ this.expectedGroupTransitions.delete(tabId);
+ }
+ }
+
+ async collectScopedTabs(includeActive) {
+ const tabs = [];
+ for (const tabId of [...this.scopedTabIds]) {
+ const tab = await this.runTabOperation(tabId, async () => {
+ if (!this.scopedTabIds.has(tabId)) {
+ return null;
+ }
+ let scopedTab;
+ try {
+ scopedTab = await chrome.tabs.get(tabId);
+ } catch {
+ await this.removeFromScopeLocked(tabId, "closed", false);
+ return null;
+ }
+ if (isTabRestricted(scopedTab)) {
+ await this.removeFromScopeLocked(tabId, "unshared", true);
+ return null;
+ }
+ return this.publicTab(scopedTab, includeActive);
+ });
+ if (tab !== null) {
+ tabs.push(tab);
+ }
+ }
+ return tabs;
+ }
+
+ publicTab(tab, includeActive) {
+ const result = {
+ tabId: tab.id,
+ url: tabUrl(tab),
+ title: typeof tab.title === "string" ? tab.title : "",
+ };
+ if (includeActive) {
+ result.active = tab.active === true;
+ }
+ return result;
+ }
+
+ async getTab(tabId) {
+ try {
+ return await chrome.tabs.get(tabId);
+ } catch {
+ throw new ProtocolError(
+ ERROR_CODES.TAB_NOT_FOUND,
+ "The requested tab was not found.",
+ );
+ }
+ }
+
+ async reconcileStoredTabs() {
+ for (const tabId of [...this.scopedTabIds]) {
+ await this.runTabOperation(tabId, async () => {
+ let tab;
+ try {
+ tab = await chrome.tabs.get(tabId);
+ } catch {
+ await this.removeFromScopeLocked(tabId, "closed", false);
+ return;
+ }
+ if (isTabRestricted(tab)) {
+ await this.removeFromScopeLocked(tabId, "unshared", true);
+ return;
+ }
+ const expectedGroupId = this.scopedGroupIds.get(tabId);
+ if (expectedGroupId !== undefined && tab.groupId !== expectedGroupId) {
+ const controlledGroup = await this.getControlledGroup(tab.groupId);
+ if (controlledGroup === null) {
+ await this.removeFromScopeLocked(tabId, "unshared", true);
+ } else {
+ this.scopedGroupIds.set(tabId, tab.groupId);
+ await this.persistScope();
+ await this.updateControlledGroup(tab.groupId);
+ }
+ } else if (expectedGroupId === undefined) {
+ await this.groupTabLocked(tab);
+ }
+ });
+ }
+ }
+
+ async runTabOperation(tabId, operation) {
+ const previous = this.tabOperations.get(tabId) ?? Promise.resolve();
+ const current = previous.catch(() => undefined).then(operation);
+ this.tabOperations.set(tabId, current);
+ try {
+ return await current;
+ } finally {
+ if (this.tabOperations.get(tabId) === current) {
+ this.tabOperations.delete(tabId);
+ }
+ }
+ }
+
+ async persistScope() {
+ const groupIds = Object.fromEntries(this.scopedGroupIds);
+ await chrome.storage.session.set({
+ [SCOPED_TAB_IDS_KEY]: [...this.scopedTabIds],
+ [SCOPED_GROUP_IDS_KEY]: groupIds,
+ });
+ }
+}
diff --git a/skyvern/browser_extension/protocol.py b/skyvern/browser_extension/protocol.py
new file mode 100644
index 000000000..5c13789aa
--- /dev/null
+++ b/skyvern/browser_extension/protocol.py
@@ -0,0 +1,219 @@
+from __future__ import annotations
+
+import json
+from dataclasses import dataclass
+from typing import Literal
+from urllib.parse import urlsplit
+
+from skyvern.browser_extension.errors import BrowserExtensionError
+
+PROTOCOL_VERSION = 1
+EXTENSION_ID = "fmamdhmfeihjjaiheideemihnbpnokin"
+
+ALLOWED_OPS = frozenset(
+ {
+ "debugger.attach",
+ "debugger.detach",
+ "debugger.send",
+ "tabs.create",
+ "tabs.remove",
+ "tabs.activate",
+ "tabs.list",
+ }
+)
+
+ALLOWED_EVENTS = frozenset(
+ {
+ "extension.hello",
+ "debugger.event",
+ "debugger.detached",
+ "scope.tabAdded",
+ "scope.tabRemoved",
+ "tabs.created",
+ }
+)
+
+ERROR_CODES = frozenset(
+ {
+ "AUTH_FAILED",
+ "OP_NOT_ALLOWED",
+ "TAB_NOT_FOUND",
+ "TAB_NOT_SCOPED",
+ "RESTRICTED_URL",
+ "DEBUGGER_DETACHED",
+ "CDP_METHOD_NOT_ALLOWED",
+ "CDP_ERROR",
+ "INTERNAL",
+ }
+)
+
+ALLOWED_CDP_METHOD_PREFIXES = (
+ "Accessibility.",
+ "Animation.",
+ "CSS.",
+ "Console.",
+ "DOM.",
+ "DOMDebugger.",
+ "DOMSnapshot.",
+ "DOMStorage.",
+ "Debugger.",
+ "Emulation.",
+ "Fetch.",
+ "IO.",
+ "Input.",
+ "Inspector.",
+ "Log.",
+ "Network.",
+ "Overlay.",
+ "Page.",
+ "Performance.",
+ "Profiler.",
+ "Runtime.",
+ "Security.",
+ "Storage.",
+ "Target.",
+)
+
+DENIED_CDP_METHODS = frozenset(
+ {
+ "Network.getAllCookies",
+ "Network.clearBrowserCookies",
+ "Network.clearBrowserCache",
+ "Storage.getCookies",
+ "Storage.setCookies",
+ "Storage.clearCookies",
+ }
+)
+
+RESTRICTED_URL_PREFIXES = (
+ "chrome://",
+ "chrome-untrusted://",
+ "chrome-extension://",
+ "devtools://",
+ "edge://",
+ "about:",
+ "file://",
+)
+
+MessageKind = Literal["response", "event", "pong", "ping", "auth.proof"]
+
+
+@dataclass(frozen=True, slots=True)
+class ParsedMessage:
+ kind: MessageKind
+ request_id: str | None = None
+ ok: bool | None = None
+ result: dict | None = None
+ error: dict | None = None
+ error_code: str | None = None
+ error_message: str | None = None
+ event: str | None = None
+ params: dict | None = None
+ client_nonce: str | None = None
+ proof: str | None = None
+
+
+def build_request(request_id: str, op: str, args: dict) -> dict:
+ if not isinstance(request_id, str) or not request_id:
+ raise BrowserExtensionError("Request id must be a non-empty string")
+ if not isinstance(op, str) or op not in ALLOWED_OPS:
+ raise BrowserExtensionError(f"Operation is not allowed: {op}")
+ if not isinstance(args, dict):
+ raise BrowserExtensionError("Request args must be an object")
+ return {"v": PROTOCOL_VERSION, "type": "request", "id": request_id, "op": op, "args": args}
+
+
+def parse_extension_message(raw: str) -> ParsedMessage:
+ try:
+ payload = json.loads(raw)
+ except (json.JSONDecodeError, TypeError) as exc:
+ raise BrowserExtensionError("Extension frame is not valid JSON") from exc
+
+ if not isinstance(payload, dict):
+ raise BrowserExtensionError("Extension frame must be an object")
+ if type(payload.get("v")) is not int or payload["v"] != PROTOCOL_VERSION:
+ raise BrowserExtensionError("Unsupported extension protocol version")
+
+ frame_type = payload.get("type")
+ if frame_type == "response":
+ return _parse_response(payload)
+ if frame_type == "event":
+ return _parse_event(payload)
+ if frame_type == "ping":
+ return ParsedMessage(kind="ping")
+ if frame_type == "pong":
+ return ParsedMessage(kind="pong")
+ if frame_type == "auth.proof":
+ return ParsedMessage(
+ kind="auth.proof",
+ client_nonce=_required_string(payload, "clientNonce"),
+ proof=_required_string(payload, "proof"),
+ )
+ raise BrowserExtensionError("Unknown extension frame type")
+
+
+def is_cdp_method_allowed(method: str, params: dict | None = None) -> bool:
+ if method in DENIED_CDP_METHODS:
+ return False
+ if method == "Network.getCookies" and isinstance(params, dict) and "urls" in params:
+ return False
+ return any(method.startswith(prefix) and len(method) > len(prefix) for prefix in ALLOWED_CDP_METHOD_PREFIXES)
+
+
+def is_restricted_url(url: str) -> bool:
+ normalized_url = url.strip().lower()
+ if normalized_url == "about:blank":
+ return False
+ if normalized_url.startswith(RESTRICTED_URL_PREFIXES):
+ return True
+ try:
+ hostname = urlsplit(normalized_url).hostname
+ return hostname is not None and hostname.removesuffix(".") == "chromewebstore.google.com"
+ except ValueError:
+ return False
+
+
+def _parse_response(payload: dict) -> ParsedMessage:
+ request_id = _required_string(payload, "id")
+ ok = payload.get("ok")
+ if type(ok) is not bool:
+ raise BrowserExtensionError("Response ok must be a boolean")
+
+ if ok:
+ result = _required_dict(payload, "result")
+ return ParsedMessage(kind="response", request_id=request_id, ok=True, result=result)
+
+ error = _required_dict(payload, "error")
+ error_code = _required_string(error, "code")
+ error_message = _required_string(error, "message")
+ if error_code not in ERROR_CODES:
+ raise BrowserExtensionError("Response contains an unknown error code")
+ return ParsedMessage(
+ kind="response",
+ request_id=request_id,
+ ok=False,
+ error=error,
+ error_code=error_code,
+ error_message=error_message,
+ )
+
+
+def _parse_event(payload: dict) -> ParsedMessage:
+ event = _required_string(payload, "event")
+ if event not in ALLOWED_EVENTS:
+ raise BrowserExtensionError("Unknown extension event")
+ return ParsedMessage(kind="event", event=event, params=_required_dict(payload, "params"))
+
+
+def _required_string(payload: dict, field: str) -> str:
+ value = payload.get(field)
+ if not isinstance(value, str) or not value:
+ raise BrowserExtensionError(f"Extension frame field {field} must be a non-empty string")
+ return value
+
+
+def _required_dict(payload: dict, field: str) -> dict:
+ value = payload.get(field)
+ if not isinstance(value, dict):
+ raise BrowserExtensionError(f"Extension frame field {field} must be an object")
+ return value
diff --git a/skyvern/browser_extension/relay.py b/skyvern/browser_extension/relay.py
new file mode 100644
index 000000000..5316e7fbe
--- /dev/null
+++ b/skyvern/browser_extension/relay.py
@@ -0,0 +1,947 @@
+from __future__ import annotations
+
+import asyncio
+import base64
+import binascii
+import hashlib
+import hmac
+import itertools
+import json
+import secrets
+import time
+from collections.abc import Awaitable, Callable
+from contextlib import suppress
+from urllib.parse import urlsplit
+
+import structlog
+from aiohttp import WSMsgType, web
+
+from skyvern.browser_extension.auth import build_challenge, compute_server_proof, verify_ext_proof
+from skyvern.browser_extension.errors import (
+ BrowserExtensionError,
+ BrowserExtensionNotConnectedError,
+ ExtensionRequestError,
+)
+from skyvern.browser_extension.protocol import (
+ EXTENSION_ID,
+ PROTOCOL_VERSION,
+ ParsedMessage,
+ build_request,
+ parse_extension_message,
+)
+
+LOG = structlog.get_logger(__name__)
+
+_AUTH_TIMEOUT_SECONDS = 10.0
+_PING_INTERVAL_SECONDS = 20.0
+_INBOUND_TIMEOUT_SECONDS = 45.0
+_MAX_WS_MESSAGE_BYTES = 256 * 1024 * 1024
+_AUTH_CLOSE_CODE = 4403
+_REPLACED_CLOSE_CODE = 4000
+_PAIRING_NONCE_TTL_SECONDS = 120.0
+_PAIR_BEGIN_MESSAGE = b"skyvern-pair-begin-v1"
+
+_PAIR_PAGE_TEMPLATE = """<!doctype html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width,initial-scale=1">
+ <title>Pair Skyvern Agent</title>
+ <style>
+ :root {
+ color-scheme: dark light;
+ font-family: ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
+ color: #f4f4f5;
+ background: #0b0c0f;
+ font-synthesis: none;
+ --accent: #6d6cf6;
+ --accent-hover: #5f5eea;
+ --surface: #141519;
+ --surface-subtle: #1a1b20;
+ --border: #2b2c33;
+ --border-strong: #3a3b43;
+ --text: #f4f4f5;
+ --muted: #b2b3ba;
+ --subtle: #85868f;
+ --radius: .5rem;
+ }
+
+ * { box-sizing: border-box; }
+
+ body {
+ min-height: 100vh;
+ margin: 0;
+ display: grid;
+ overflow-x: hidden;
+ color: var(--text);
+ background: #0b0c0f;
+ place-items: center;
+ }
+
+ .page-shell {
+ position: relative;
+ width: 100%;
+ padding: 2rem 1rem;
+ }
+
+ .pairing-card {
+ width: min(38rem, 100%);
+ margin: 0 auto;
+ overflow: hidden;
+ border: 1px solid var(--border);
+ border-radius: var(--radius);
+ background: var(--surface);
+ box-shadow: 0 1rem 2.5rem rgb(0 0 0 / 28%);
+ }
+
+ .card-header {
+ display: flex;
+ align-items: center;
+ gap: .75rem;
+ padding: 1.5rem 1.75rem 0;
+ }
+
+ .brand-mark,
+ .endpoint-mark {
+ display: grid;
+ flex: none;
+ overflow: hidden;
+ place-items: center;
+ }
+
+ .brand-mark {
+ width: 2rem;
+ height: 2rem;
+ border-radius: var(--radius);
+ }
+
+ .brand-mark img,
+ .endpoint-mark img {
+ width: 100%;
+ height: 100%;
+ object-fit: cover;
+ }
+
+ .brand-copy { display: grid; }
+ .brand-name { font-size: .9rem; font-weight: 650; letter-spacing: -.01em; }
+
+ .card-body { padding: 2.5rem 1.75rem 1.75rem; }
+ .intro { text-align: left; }
+
+ h1 {
+ margin: 0;
+ font-size: clamp(2rem, 6vw, 2.35rem);
+ font-weight: 680;
+ line-height: 1.15;
+ letter-spacing: -.035em;
+ }
+
+ .supporting {
+ max-width: 32rem;
+ margin: .85rem 0 0;
+ color: var(--muted);
+ font-size: .95rem;
+ line-height: 1.55;
+ }
+
+ .connection {
+ display: grid;
+ grid-template-columns: minmax(0, 1fr) 3.5rem minmax(0, 1fr);
+ align-items: center;
+ margin: 2.25rem 0;
+ }
+
+ .endpoint {
+ display: grid;
+ grid-template-columns: 2rem minmax(0, 1fr);
+ grid-template-rows: auto auto;
+ min-width: 0;
+ align-items: center;
+ column-gap: .7rem;
+ row-gap: .2rem;
+ }
+
+ .endpoint-top {
+ display: contents;
+ }
+
+ .endpoint-mark {
+ grid-row: 1 / 3;
+ width: 2rem;
+ height: 2rem;
+ border-radius: var(--radius);
+ color: var(--muted);
+ }
+
+ .endpoint-mark svg { width: 1rem; height: 1rem; }
+ .endpoint-title { overflow: hidden; font-size: .8rem; font-weight: 650; text-overflow: ellipsis; white-space: nowrap; }
+ .endpoint-meta { grid-column: 2; margin: 0; overflow: hidden; color: var(--subtle); font-size: .7rem; line-height: 1.4; text-overflow: ellipsis; white-space: nowrap; }
+ #server-address { font-family: ui-monospace, SFMono-Regular, Menlo, monospace; }
+
+ .connection-rail {
+ position: relative;
+ height: 1px;
+ background: var(--border-strong);
+ }
+
+ .link-glyph {
+ position: absolute;
+ top: 50%;
+ left: 50%;
+ display: grid;
+ width: 1.75rem;
+ height: 1.75rem;
+ color: var(--subtle);
+ background: var(--surface);
+ place-items: center;
+ transform: translate(-50%, -50%);
+ }
+
+ .link-glyph svg { width: .95rem; height: .95rem; }
+
+ .approval-controls { display: flex; flex-direction: column; align-items: flex-start; gap: .8rem; }
+
+ .approve-button {
+ display: inline-flex;
+ width: auto;
+ min-height: 2.75rem;
+ align-items: center;
+ justify-content: center;
+ gap: .6rem;
+ margin: 0;
+ padding: .72rem 1.15rem;
+ border: 1px solid var(--accent);
+ border-radius: var(--radius);
+ color: white;
+ background: var(--accent);
+ box-shadow: 0 .25rem .7rem rgb(0 0 0 / 24%);
+ font: inherit;
+ font-size: .9rem;
+ font-weight: 650;
+ cursor: pointer;
+ transition: background 150ms ease, box-shadow 150ms ease;
+ }
+
+ .approve-button:hover:not(:disabled) {
+ background: var(--accent-hover);
+ box-shadow: 0 .3rem .85rem rgb(0 0 0 / 28%);
+ }
+
+ .approve-button:focus-visible,
+ .copy-chip:focus-visible { outline: 2px solid var(--text); outline-offset: 3px; }
+ .approve-button:disabled { cursor: wait; opacity: .78; }
+
+ .spinner {
+ display: none;
+ width: 1rem;
+ height: 1rem;
+ border: 2px solid rgb(255 255 255 / 35%);
+ border-top-color: white;
+ border-radius: 50%;
+ animation: spin 700ms linear infinite;
+ }
+
+ [data-state="approving"] .spinner { display: block; }
+ .fine-print {
+ margin: 0;
+ color: var(--subtle);
+ font-size: .7rem;
+ line-height: 1.5;
+ text-align: center;
+ }
+
+ .status-panel {
+ display: grid;
+ gap: .7rem;
+ margin-top: .35rem;
+ padding: 1rem 0 0;
+ text-align: left;
+ }
+
+ .status-panel[hidden] { display: none; }
+ .status-heading { display: flex; align-items: center; gap: .7rem; }
+
+ .status-icon {
+ display: grid;
+ flex: none;
+ width: 2rem;
+ height: 2rem;
+ color: var(--muted);
+ place-items: center;
+ }
+
+ .status-icon svg { width: 1rem; height: 1rem; }
+ .status-icon .success-icon { display: none; }
+ .status-title { margin: 0; font-size: .9rem; font-weight: 650; }
+ .status-message { margin: 0; color: var(--muted); font-size: .78rem; line-height: 1.55; }
+
+ [data-state="success"] .status-icon .error-icon { display: none; }
+ [data-state="success"] .status-icon .success-icon { display: block; }
+ [data-state="success"] .approval-controls,
+ [data-state="error"] .approval-controls { display: none; }
+
+ .recovery { display: grid; gap: .4rem; }
+ .recovery[hidden] { display: none; }
+ .recovery-label { color: var(--subtle); font-size: .67rem; font-weight: 650; }
+
+ .copy-chip {
+ display: flex;
+ width: fit-content;
+ max-width: 100%;
+ min-width: 0;
+ align-items: center;
+ justify-content: space-between;
+ gap: .75rem;
+ padding: .7rem .8rem;
+ border: 1px solid var(--border);
+ border-radius: var(--radius);
+ color: var(--text);
+ background: var(--surface-subtle);
+ font: inherit;
+ cursor: copy;
+ }
+
+ .copy-chip:hover { border-color: var(--border-strong); background: #202126; }
+ .copy-chip code { overflow: hidden; font: 600 .68rem/1.4 ui-monospace, SFMono-Regular, Menlo, monospace; text-overflow: ellipsis; white-space: nowrap; }
+ .copy-label { flex: none; color: var(--muted); font-size: .66rem; font-weight: 650; }
+
+ @keyframes spin { to { transform: rotate(360deg); } }
+ @media (max-width: 34rem) {
+ .page-shell { padding: 1rem; }
+ .card-header { padding: 1rem 1.1rem; }
+ .card-body { padding: 1.7rem 1.2rem 1.35rem; }
+ h1 { font-size: 1.9rem; }
+ .connection { grid-template-columns: 1fr; gap: .85rem; }
+ .connection-rail { width: 1px; height: 1.6rem; margin-left: 1rem; }
+ .endpoint-title { overflow: visible; font-size: .78rem; line-height: 1.25; text-overflow: clip; white-space: normal; }
+ .endpoint-meta { font-size: .67rem; }
+ .link-glyph { width: 1.5rem; height: 1.5rem; }
+ .supporting { font-size: .84rem; }
+ }
+
+ @media (prefers-reduced-motion: reduce) {
+ *, *::before, *::after { scroll-behavior: auto !important; transition: none !important; animation-duration: .01ms !important; animation-iteration-count: 1 !important; }
+ }
+
+ @media (prefers-color-scheme: light) {
+ :root {
+ color: #202124;
+ background: #f5f5f7;
+ --surface: #ffffff;
+ --surface-subtle: #f5f5f7;
+ --border: #dedfe3;
+ --border-strong: #c9cbd1;
+ --text: #202124;
+ --muted: #5f6169;
+ --subtle: #777982;
+ }
+
+ body { background: #f5f5f7; }
+ .pairing-card { box-shadow: 0 1rem 2.5rem rgb(32 33 36 / 10%); }
+ .copy-chip:hover { background: #ececef; }
+ }
+ </style>
+</head>
+<body data-state="idle">
+ <div class="page-shell">
+ <main class="pairing-card" aria-labelledby="pairing-title">
+ <header class="card-header">
+ <span class="brand-mark" aria-hidden="true">
+ <img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAhGVYSWZNTQAqAAAACAAFARIAAwAAAAEAAQAAARoABQAAAAEAAABKARsABQAAAAEAAABSASgAAwAAAAEAAgAAh2kABAAAAAEAAABaAAAAAAAAAEgAAAABAAAASAAAAAEAA6ABAAMAAAABAAEAAKACAAQAAAABAAAAMKADAAQAAAABAAAAMAAAAAAoDQEPAAAACXBIWXMAAAsTAAALEwEAmpwYAAACymlUWHRYTUw6Y29tLmFkb2JlLnhtcAAAAAAAPHg6eG1wbWV0YSB4bWxuczp4PSJhZG9iZTpuczptZXRhLyIgeDp4bXB0az0iWE1QIENvcmUgNi4wLjAiPgogICA8cmRmOlJERiB4bWxuczpyZGY9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjItcmRmLXN5bnRheC1ucyMiPgogICAgICA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0iIgogICAgICAgICAgICB4bWxuczp0aWZmPSJodHRwOi8vbnMuYWRvYmUuY29tL3RpZmYvMS4wLyIKICAgICAgICAgICAgeG1sbnM6ZXhpZj0iaHR0cDovL25zLmFkb2JlLmNvbS9leGlmLzEuMC8iPgogICAgICAgICA8dGlmZjpZUmVzb2x1dGlvbj43MjwvdGlmZjpZUmVzb2x1dGlvbj4KICAgICAgICAgPHRpZmY6UmVzb2x1dGlvblVuaXQ+MjwvdGlmZjpSZXNvbHV0aW9uVW5pdD4KICAgICAgICAgPHRpZmY6WFJlc29sdXRpb24+NzI8L3RpZmY6WFJlc29sdXRpb24+CiAgICAgICAgIDx0aWZmOk9yaWVudGF0aW9uPjE8L3RpZmY6T3JpZW50YXRpb24+CiAgICAgICAgIDxleGlmOlBpeGVsWERpbWVuc2lvbj4yNTY8L2V4aWY6UGl4ZWxYRGltZW5zaW9uPgogICAgICAgICA8ZXhpZjpDb2xvclNwYWNlPjE8L2V4aWY6Q29sb3JTcGFjZT4KICAgICAgICAgPGV4aWY6UGl4ZWxZRGltZW5zaW9uPjI1NjwvZXhpZjpQaXhlbFlEaW1lbnNpb24+CiAgICAgIDwvcmRmOkRlc2NyaXB0aW9uPgogICA8L3JkZjpSREY+CjwveDp4bXBtZXRhPgrkVyGCAAANkElEQVRoBdVaCXCU5Rl+stnd3Pd9AoGES0KAIAmXQUAOJYJCQfAAES22nTr2mPEYtcfojLXO1Oloa6ejTp2KI4paCqhcIQkISThyJ+S+yJ3NtbvZze72eb8kEjQX0HbgY5Zs/n/3/97jeZ/3eT9w0sy5x4HbaTk5wYkvh92urNbcTrY7HA64urhAr9dB3su6vRxg1GNiouE0LOq3jQN2mx0hIUGYGTcZpl6jgtFtkwGFFkb/sR0bYejshoPODK3bIgOOvj6sXbcCKUsSceFSEaB1HrL/1q8Be58FC5MT8dwzu5CVk4fWplZoNFfjrv3OlVvsjbCMw2LFkmWL8MIvnoBep8Hh46dB+rnG0lvSAbvNpozcvPlePL37R3Bz1aGwtAZ5+SVw0l5r8rW/XePb//+XoagHBAfiKRq+Yc0yWJkFi6UfZ7Pz0csC1rAHDF//EwfEED0bjtViGb7XqO/tAgsaqnNzxZr1y7DnsQcRGRGC7t5eXraiv9+O3MLSEb//X3NAjJbisnHDsLAQ9JPqWlpMcGbK5d5IS0HF2g9XL0+kpCRj59b1SJgzAz0mCzq6etCvjO+HiYXc3m5g2x3ewgaeeNMOqLQTs4FBAYyYEdHhIZgyKRInM7IQEOCPPhrY29PzXeOxi4bhNXqL6KhwrF2RhG2pqzB7+lR0mcxo6OiEmbRppfE2ftbWb4ODvdfXzwew/zAQN+WAnWl31usxd+506HRahAf6I4SOpJ29gOXJC1Df3ILyylplvF02p1H+Qf5ISZqPzWvuwt2L5yHIwwMNRjPyKxvQ1tWtjO+z9BH3FvTyuoGZaGpqZxCMcPVwpw8OQooBGFxON6JGVeoZmaWL5mH3lvXQ6vToY9TSs3Nx4Jt0hIcGo83QiZbWdgUribq7qwv27tiEJ/n5uMgwlNZfwZcnv0XauUsoqqhFj9GkFKaN0JPPawgXnU4Hf0Y+iIFxUIFW1zSgoaERtkGWEh+uOwPy8EkRofjZw5uwM3UN6lvb8PHhkygqr4ahuxsRZJDK2npG0qqMVwXt7Iy3XnoGuwmVRmL58ZffxKdHTsGVRTsjdjKS7pwLPx8vqkw9g6Fj3TjDw90Vnoz4lcZWZJzJwaW8YvT09KpnDkX/uh2wm/vw4IZVuI8Ft3FFMq60tGHv796Cj5c7Pnr9eeicNTjAqL697yDOZF9S+ziI9zsXL1DG5xSV4aFf/h4aZuyFZ/cgfvZU6N1cYKSzfYSjldCwDmK+sroeH358EJcoHax9ZsX/wzvwkBPOTiFTXxn6ZayfYnxc3BS8+sxu2LnJvBkxLDIHjXZGQXkNymrqkbJwLqaR/jyJ68tMdzMdZL6xa8sGxESH494fP49ZM6bitRd+ygKOQGt3L660GRTjCGUKjEx0JCunEH/7+0eoYlYdJB4N95AhZqQ1IQgJbHZuux8v7d2Ozi4jqhua8ZdPDuFw+llk5ZXAQoeOnjjN1Ovw2588jARG1tPTQ22qYT+YHz8Tr7zzD0iDSr13BQ6nfYuq+mY4KGkE4/7+vnAnZJzITC0tHfj4k3+hh8X7/aZ1ww4w1IibFIEPvzyK1/76T6XHlSaRYmJ0QAYSQ/cdPoFd5HJn4jiUbCNZm0J6NJBePzuSBi8fb/zqlT+pvqAlzqVXSLOTWoiKjsIdc6azDhhTYcsROP+GHdBws5f//L5iB+FlhhZ6Fti8hFlYdGc8Gacb2edy4ePhhrbOLuRfrkFuSYXaLyV5PvYfOkanTUhMjMfc+Fnw9mZ2WC9WmwNNhNmliwXIycpFfm4BoidHIWpKNOqqamE0GkdtgkPOTAhCYrBwr+qo5OHkpYl4lCzkzCilpWcj7dRZbFqbgiriXuOkQSc1S0VJOcInRzITATh4NBM7H9+GKTFRMLLLmu3kccKytLQC2VkX0dzYwmfb1fMryyqJeS3ZSKsgOFoXvz4HBj/tTIzu3L0Zmzak4MAXJ3Dkm0zUs9CiOacmzIzFZTYtYZXM83l0xAlbUu/BwWMZCGJfqK5rQA05PDg0iBnwQikzdPTICZVVqQ1re4faxUkgSQyJjJjImlAGVBTIODv3bMO6NUvx7nufUzaYBhoKobB7aypyiy5j+rRo1Dc2I5vUt+ORTSitoAQuuIwowqKeUe4n/AooyiIiwmCgZFANUeBIhRkYHISmK40q6qMxzkgOXR1tRro7eE24fPnKJVhFefvJ50cRRLFWSk5vJHUKZhNmx+LM+QIkzZuJ1o4u7N21Bd1sOmmnc7By/UqsZe9YlJSImGkx8PL2Id4voq62gcWuhc5FTx6gDGF3DqITHqyt8WAz3NRxMyAHSL6BAUh9YC3O5xYjhDx/6IuvUV9dp4p5YfwMlLHp+LKT+pBl5PPfZGTjyLEzWLZiKU8SAlBdXoussznoEJiQYTTMWi8hIpEWqPj4eiM0LBit1E6h4cHIO5/PxnV17h1u8Pffj+8A057ITqonw3h4+6LodDbK8ktJm3rIvBoWEogKOhMdFUqHgIysAuz77CuER0WitPgyzmddQBeZyWoyqW7qNEiPQzARhw0GA+aHzIfJdB6xM4NZR67ol1mCDo63xoSQpFKGjFlsREZ2SPIEvs04d82pgIkS2EKI6VjgJpOVc2smbNy8rroWpq4uKlRfzJw6CeHEvRhkZ8SHQ0SaV6ehi+zUpwpax2box8amhpzxrOf9MTPgYOH68mHe7JaWvn5UknHaKN6ktatFKOQWl2H18iSUkIE6e81obh1gk9S1d+GBdcvhzY6s1TmrsbCssh7/PpqBdMrtAcU5ED+RJr3dRkXVFk5fnt7eaCJjTWSNmQECGh6enmC/gZldtfkK2780ssEllHeRcLKwR4guMjGKAXRYBo9JUWHwI7b1Ljq4MKo+Xh5YunAOXn/+abz48yeUXhoa3gV6XcyChZDst9p47HPt3Du030g/xxRzgs8AHufFzJzGWcSGmooq1JRVqS4qDxMc2wifBp7VJC9MQBfF2KTIEORcLFSOVdQ2oYNdWpSBH6PqyR6hobUJs6ZhKtkr/exFZbQbxZ/e1XWgiCMi0E1Z3tHWyuePHV+xYRwHHPDy9UFkbIzi8E5SZGVJmcKwGnMZOWGUjg4DWqgqJerxNM7Pzxf5pNkKvnJyi3AsMxvnLhapQX86mx5hr+rChQ6lU+t7+vrBzCK3MZOBISGsiQ5mpFMFSIwca43tAL+p45ARFTsNhlYD8nJyoeEmkeys7ixuIwtYClbOamToLqR86GGDWzB3NpZwSLEy9E2kTivh19bWgTQa29zeheQFc3hQpcVsyvNsNroa9oTe3h7SqR+ZzgftrS0wso8MMdUNOyBflEi7unmg8FwWHmVhPvfkdjyweim2b1iNzevugkynhZerFEyk05ZRy2RSmDW3tCOWouyOGbHoZ020kSpFYRazE/fynGfF4gR4urspZjrCwpY6CAgiFTOj7c3Nqj/ctAMK44x4S30DXnxqh1K4f3x/P9478BVn31McNjT49Z6HEEdYnBI80zANsyHCr5GSIq+whONlA0LZYeWEoo3ZsNOJ0rJqJM2fg8msFxcy1Gdfp5MoCFcfP1KyBZ1tHIQURseK/cC9MSEkHxHeXpw4F8EcrN/ef5j1MA1uHp6MZAmysi7hOKO9e+sGxNKJE5QOwt/iuPC7sJQcTDVQ44j29ySjiUS2mc08hvHHSp5c9Ntt+PTQSaVgpc9IFs093Soz45vPpI77IaY/KiyMCrMQU1jMDg4xrm5uiJkeR2zpUUgR9/RLb5Ii4+lIKkQ3DV/ijGSlnRDqIrs4Sw/htaZmRllIgH9UsEnZMmuYWQvCWhNd4zvAzbo4r5oYtbamNsLEirqaGrgzmmERkXCikizIK8Ib7+7Dnu3384RhnpIY3zdA5LUcvUgDE4t9KaldObk1sbjb2QOECCQDaoDnZye6xneARVVSSc0fGY7KsnJytBGePv5oqKtXkQwMCYWWCvLAoeP46lQWfvvsE5hEjreLlhlhKRnBLCzhdOaiccaJjBxCpldlxmrm2dAEsT/06HEdENlQVlENF3bUqTwyLC8uRp/ZgsDQcB6F9MHI7Li5e6jTg9ff+YBHLe146zfPIo7SWWZi6bZilLzUe1Lv+tXLsHHVEuSSsd7bf0gpT3VPsnOda9wilufJ5jV1V7By+RIOIgZUV1WzKbnC3ctLYdgskWMxmligmewVSfPuwJMPpULLHtLa3kn49QnsEUwmemTrfXiDcsLQ3YOnXvgDSjiZadgTbnRN+GhRIuRNg5MSE0iH7Sjn0O3Q6OHC6POfnSkJqEoJGxOLUE9q3Mo+8eimtdRAHPQ5fUnBRoeH8hmeOJ55Hq++/QGKisvHPToZq6ClUibsgERIFSB/ytlnIIWaFF87a0JDLMsxulCn/HOoZMRCaPn4eVP3xGIG5bQHZUMj4XWBnbeIekoOvMaLvJbWj4ZxccxKD67LAX5HrSFHtDRaYZvYlWO/qydoA3LbajGTVjmckyIVN0rIWFPSHybSZV35Nd0oKZBq6eU2NwS+oTPKIUfEGHFExNiApeKn/J8G/k2avZk1iv18sNxhj7mZh4/8XYXMkW9d99WRzXei8fKSpZVp6FZd/cyq4ucfGMjrdMDOl3bj3ck/uH2rXBgM8qjmyMn1fwCpJWsaG/VU1QAAAABJRU5ErkJggg==" alt="">
+ </span>
+ <span class="brand-copy">
+ <span class="brand-name">Skyvern</span>
+ </span>
+ </header>
+
+ <div class="card-body">
+ <section class="intro">
+ <h1 id="pairing-title">Pair Skyvern Agent</h1>
+ <p class="supporting">Skyvern on this computer wants to drive Chrome through the Skyvern Agent extension.</p>
+ </section>
+
+ <div class="connection" aria-label="Skyvern MCP server connecting to Skyvern Agent extension">
+ <div class="endpoint">
+ <div class="endpoint-top">
+ <span class="endpoint-mark" aria-hidden="true">
+ <img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAhGVYSWZNTQAqAAAACAAFARIAAwAAAAEAAQAAARoABQAAAAEAAABKARsABQAAAAEAAABSASgAAwAAAAEAAgAAh2kABAAAAAEAAABaAAAAAAAAAEgAAAABAAAASAAAAAEAA6ABAAMAAAABAAEAAKACAAQAAAABAAAAMKADAAQAAAABAAAAMAAAAAAoDQEPAAAACXBIWXMAAAsTAAALEwEAmpwYAAACymlUWHRYTUw6Y29tLmFkb2JlLnhtcAAAAAAAPHg6eG1wbWV0YSB4bWxuczp4PSJhZG9iZTpuczptZXRhLyIgeDp4bXB0az0iWE1QIENvcmUgNi4wLjAiPgogICA8cmRmOlJERiB4bWxuczpyZGY9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjItcmRmLXN5bnRheC1ucyMiPgogICAgICA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0iIgogICAgICAgICAgICB4bWxuczp0aWZmPSJodHRwOi8vbnMuYWRvYmUuY29tL3RpZmYvMS4wLyIKICAgICAgICAgICAgeG1sbnM6ZXhpZj0iaHR0cDovL25zLmFkb2JlLmNvbS9leGlmLzEuMC8iPgogICAgICAgICA8dGlmZjpZUmVzb2x1dGlvbj43MjwvdGlmZjpZUmVzb2x1dGlvbj4KICAgICAgICAgPHRpZmY6UmVzb2x1dGlvblVuaXQ+MjwvdGlmZjpSZXNvbHV0aW9uVW5pdD4KICAgICAgICAgPHRpZmY6WFJlc29sdXRpb24+NzI8L3RpZmY6WFJlc29sdXRpb24+CiAgICAgICAgIDx0aWZmOk9yaWVudGF0aW9uPjE8L3RpZmY6T3JpZW50YXRpb24+CiAgICAgICAgIDxleGlmOlBpeGVsWERpbWVuc2lvbj4yNTY8L2V4aWY6UGl4ZWxYRGltZW5zaW9uPgogICAgICAgICA8ZXhpZjpDb2xvclNwYWNlPjE8L2V4aWY6Q29sb3JTcGFjZT4KICAgICAgICAgPGV4aWY6UGl4ZWxZRGltZW5zaW9uPjI1NjwvZXhpZjpQaXhlbFlEaW1lbnNpb24+CiAgICAgIDwvcmRmOkRlc2NyaXB0aW9uPgogICA8L3JkZjpSREY+CjwveDp4bXBtZXRhPgrkVyGCAAANkElEQVRoBdVaCXCU5Rl+stnd3Pd9AoGES0KAIAmXQUAOJYJCQfAAES22nTr2mPEYtcfojLXO1Oloa6ejTp2KI4paCqhcIQkISThyJ+S+yJ3NtbvZze72eb8kEjQX0HbgY5Zs/n/3/97jeZ/3eT9w0sy5x4HbaTk5wYkvh92urNbcTrY7HA64urhAr9dB3su6vRxg1GNiouE0LOq3jQN2mx0hIUGYGTcZpl6jgtFtkwGFFkb/sR0bYejshoPODK3bIgOOvj6sXbcCKUsSceFSEaB1HrL/1q8Be58FC5MT8dwzu5CVk4fWplZoNFfjrv3OlVvsjbCMw2LFkmWL8MIvnoBep8Hh46dB+rnG0lvSAbvNpozcvPlePL37R3Bz1aGwtAZ5+SVw0l5r8rW/XePb//+XoagHBAfiKRq+Yc0yWJkFi6UfZ7Pz0csC1rAHDF//EwfEED0bjtViGb7XqO/tAgsaqnNzxZr1y7DnsQcRGRGC7t5eXraiv9+O3MLSEb//X3NAjJbisnHDsLAQ9JPqWlpMcGbK5d5IS0HF2g9XL0+kpCRj59b1SJgzAz0mCzq6etCvjO+HiYXc3m5g2x3ewgaeeNMOqLQTs4FBAYyYEdHhIZgyKRInM7IQEOCPPhrY29PzXeOxi4bhNXqL6KhwrF2RhG2pqzB7+lR0mcxo6OiEmbRppfE2ftbWb4ODvdfXzwew/zAQN+WAnWl31usxd+506HRahAf6I4SOpJ29gOXJC1Df3ILyylplvF02p1H+Qf5ISZqPzWvuwt2L5yHIwwMNRjPyKxvQ1tWtjO+z9BH3FvTyuoGZaGpqZxCMcPVwpw8OQooBGFxON6JGVeoZmaWL5mH3lvXQ6vToY9TSs3Nx4Jt0hIcGo83QiZbWdgUribq7qwv27tiEJ/n5uMgwlNZfwZcnv0XauUsoqqhFj9GkFKaN0JPPawgXnU4Hf0Y+iIFxUIFW1zSgoaERtkGWEh+uOwPy8EkRofjZw5uwM3UN6lvb8PHhkygqr4ahuxsRZJDK2npG0qqMVwXt7Iy3XnoGuwmVRmL58ZffxKdHTsGVRTsjdjKS7pwLPx8vqkw9g6Fj3TjDw90Vnoz4lcZWZJzJwaW8YvT09KpnDkX/uh2wm/vw4IZVuI8Ft3FFMq60tGHv796Cj5c7Pnr9eeicNTjAqL697yDOZF9S+ziI9zsXL1DG5xSV4aFf/h4aZuyFZ/cgfvZU6N1cYKSzfYSjldCwDmK+sroeH358EJcoHax9ZsX/wzvwkBPOTiFTXxn6ZayfYnxc3BS8+sxu2LnJvBkxLDIHjXZGQXkNymrqkbJwLqaR/jyJ68tMdzMdZL6xa8sGxESH494fP49ZM6bitRd+ygKOQGt3L660GRTjCGUKjEx0JCunEH/7+0eoYlYdJB4N95AhZqQ1IQgJbHZuux8v7d2Ozi4jqhua8ZdPDuFw+llk5ZXAQoeOnjjN1Ovw2588jARG1tPTQ22qYT+YHz8Tr7zzD0iDSr13BQ6nfYuq+mY4KGkE4/7+vnAnZJzITC0tHfj4k3+hh8X7/aZ1ww4w1IibFIEPvzyK1/76T6XHlSaRYmJ0QAYSQ/cdPoFd5HJn4jiUbCNZm0J6NJBePzuSBi8fb/zqlT+pvqAlzqVXSLOTWoiKjsIdc6azDhhTYcsROP+GHdBws5f//L5iB+FlhhZ6Fti8hFlYdGc8Gacb2edy4ePhhrbOLuRfrkFuSYXaLyV5PvYfOkanTUhMjMfc+Fnw9mZ2WC9WmwNNhNmliwXIycpFfm4BoidHIWpKNOqqamE0GkdtgkPOTAhCYrBwr+qo5OHkpYl4lCzkzCilpWcj7dRZbFqbgiriXuOkQSc1S0VJOcInRzITATh4NBM7H9+GKTFRMLLLmu3kccKytLQC2VkX0dzYwmfb1fMryyqJeS3ZSKsgOFoXvz4HBj/tTIzu3L0Zmzak4MAXJ3Dkm0zUs9CiOacmzIzFZTYtYZXM83l0xAlbUu/BwWMZCGJfqK5rQA05PDg0iBnwQikzdPTICZVVqQ1re4faxUkgSQyJjJjImlAGVBTIODv3bMO6NUvx7nufUzaYBhoKobB7aypyiy5j+rRo1Dc2I5vUt+ORTSitoAQuuIwowqKeUe4n/AooyiIiwmCgZFANUeBIhRkYHISmK40q6qMxzkgOXR1tRro7eE24fPnKJVhFefvJ50cRRLFWSk5vJHUKZhNmx+LM+QIkzZuJ1o4u7N21Bd1sOmmnc7By/UqsZe9YlJSImGkx8PL2Id4voq62gcWuhc5FTx6gDGF3DqITHqyt8WAz3NRxMyAHSL6BAUh9YC3O5xYjhDx/6IuvUV9dp4p5YfwMlLHp+LKT+pBl5PPfZGTjyLEzWLZiKU8SAlBdXoussznoEJiQYTTMWi8hIpEWqPj4eiM0LBit1E6h4cHIO5/PxnV17h1u8Pffj+8A057ITqonw3h4+6LodDbK8ktJm3rIvBoWEogKOhMdFUqHgIysAuz77CuER0WitPgyzmddQBeZyWoyqW7qNEiPQzARhw0GA+aHzIfJdB6xM4NZR67ol1mCDo63xoSQpFKGjFlsREZ2SPIEvs04d82pgIkS2EKI6VjgJpOVc2smbNy8rroWpq4uKlRfzJw6CeHEvRhkZ8SHQ0SaV6ehi+zUpwpax2box8amhpzxrOf9MTPgYOH68mHe7JaWvn5UknHaKN6ktatFKOQWl2H18iSUkIE6e81obh1gk9S1d+GBdcvhzY6s1TmrsbCssh7/PpqBdMrtAcU5ED+RJr3dRkXVFk5fnt7eaCJjTWSNmQECGh6enmC/gZldtfkK2780ssEllHeRcLKwR4guMjGKAXRYBo9JUWHwI7b1Ljq4MKo+Xh5YunAOXn/+abz48yeUXhoa3gV6XcyChZDst9p47HPt3Du030g/xxRzgs8AHufFzJzGWcSGmooq1JRVqS4qDxMc2wifBp7VJC9MQBfF2KTIEORcLFSOVdQ2oYNdWpSBH6PqyR6hobUJs6ZhKtkr/exFZbQbxZ/e1XWgiCMi0E1Z3tHWyuePHV+xYRwHHPDy9UFkbIzi8E5SZGVJmcKwGnMZOWGUjg4DWqgqJerxNM7Pzxf5pNkKvnJyi3AsMxvnLhapQX86mx5hr+rChQ6lU+t7+vrBzCK3MZOBISGsiQ5mpFMFSIwca43tAL+p45ARFTsNhlYD8nJyoeEmkeys7ixuIwtYClbOamToLqR86GGDWzB3NpZwSLEy9E2kTivh19bWgTQa29zeheQFc3hQpcVsyvNsNroa9oTe3h7SqR+ZzgftrS0wso8MMdUNOyBflEi7unmg8FwWHmVhPvfkdjyweim2b1iNzevugkynhZerFEyk05ZRy2RSmDW3tCOWouyOGbHoZ020kSpFYRazE/fynGfF4gR4urspZjrCwpY6CAgiFTOj7c3Nqj/ctAMK44x4S30DXnxqh1K4f3x/P9478BVn31McNjT49Z6HEEdYnBI80zANsyHCr5GSIq+whONlA0LZYeWEoo3ZsNOJ0rJqJM2fg8msFxcy1Gdfp5MoCFcfP1KyBZ1tHIQURseK/cC9MSEkHxHeXpw4F8EcrN/ef5j1MA1uHp6MZAmysi7hOKO9e+sGxNKJE5QOwt/iuPC7sJQcTDVQ44j29ySjiUS2mc08hvHHSp5c9Ntt+PTQSaVgpc9IFs093Soz45vPpI77IaY/KiyMCrMQU1jMDg4xrm5uiJkeR2zpUUgR9/RLb5Ii4+lIKkQ3DV/ijGSlnRDqIrs4Sw/htaZmRllIgH9UsEnZMmuYWQvCWhNd4zvAzbo4r5oYtbamNsLEirqaGrgzmmERkXCikizIK8Ib7+7Dnu3384RhnpIY3zdA5LUcvUgDE4t9KaldObk1sbjb2QOECCQDaoDnZye6xneARVVSSc0fGY7KsnJytBGePv5oqKtXkQwMCYWWCvLAoeP46lQWfvvsE5hEjreLlhlhKRnBLCzhdOaiccaJjBxCpldlxmrm2dAEsT/06HEdENlQVlENF3bUqTwyLC8uRp/ZgsDQcB6F9MHI7Li5e6jTg9ff+YBHLe146zfPIo7SWWZi6bZilLzUe1Lv+tXLsHHVEuSSsd7bf0gpT3VPsnOda9wilufJ5jV1V7By+RIOIgZUV1WzKbnC3ctLYdgskWMxmligmewVSfPuwJMPpULLHtLa3kn49QnsEUwmemTrfXiDcsLQ3YOnXvgDSjiZadgTbnRN+GhRIuRNg5MSE0iH7Sjn0O3Q6OHC6POfnSkJqEoJGxOLUE9q3Mo+8eimtdRAHPQ5fUnBRoeH8hmeOJ55Hq++/QGKisvHPToZq6ClUibsgERIFSB/ytlnIIWaFF87a0JDLMsxulCn/HOoZMRCaPn4eVP3xGIG5bQHZUMj4XWBnbeIekoOvMaLvJbWj4ZxccxKD67LAX5HrSFHtDRaYZvYlWO/qydoA3LbajGTVjmckyIVN0rIWFPSHybSZV35Nd0oKZBq6eU2NwS+oTPKIUfEGHFExNiApeKn/J8G/k2avZk1iv18sNxhj7mZh4/8XYXMkW9d99WRzXei8fKSpZVp6FZd/cyq4ucfGMjrdMDOl3bj3ck/uH2rXBgM8qjmyMn1fwCpJWsaG/VU1QAAAABJRU5ErkJggg==" alt="">
+ </span>
+ <span class="endpoint-title">Skyvern MCP Server</span>
+ </div>
+ <p id="server-address" class="endpoint-meta">127.0.0.1</p>
+ </div>
+
+ <div class="connection-rail" aria-hidden="true">
+ <span class="link-glyph">
+ <svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round">
+ <path d="M10 13a5 5 0 0 0 7.1.1l2-2a5 5 0 0 0-7.1-7.1l-1.1 1.1"/>
+ <path d="M14 11a5 5 0 0 0-7.1-.1l-2 2A5 5 0 0 0 12 20l1.1-1.1"/>
+ </svg>
+ </span>
+ </div>
+
+ <div class="endpoint">
+ <div class="endpoint-top">
+ <span class="endpoint-mark" aria-hidden="true">
+ <svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8">
+ <rect x="3.5" y="4.5" width="17" height="15" rx="2.5"/>
+ <path d="M3.5 8.5h17"/>
+ <circle cx="6.5" cy="6.5" r=".5" fill="currentColor" stroke="none"/>
+ <circle cx="9" cy="6.5" r=".5" fill="currentColor" stroke="none"/>
+ </svg>
+ </span>
+ <span class="endpoint-title">Chrome</span>
+ </div>
+ <p class="endpoint-meta">Skyvern Agent extension</p>
+ </div>
+ </div>
+
+ <div class="approval-controls">
+ <button id="approve" class="approve-button" type="button">
+ <span class="spinner" aria-hidden="true"></span>
+ <span id="approve-label">Approve connection</span>
+ </button>
+ <p class="fine-print">This link expires in 2 minutes and can be used once.</p>
+ </div>
+
+ <section id="status" class="status-panel" role="status" aria-live="polite" hidden>
+ <div class="status-heading">
+ <span class="status-icon" aria-hidden="true">
+ <svg class="error-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round">
+ <path d="M12 8v5M12 17h.01"/>
+ <circle cx="12" cy="12" r="9"/>
+ </svg>
+ <svg class="success-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.4" stroke-linecap="round" stroke-linejoin="round">
+ <path d="m7 12.5 3.2 3.2L17.5 8.5"/>
+ </svg>
+ </span>
+ <h2 id="status-title" class="status-title"></h2>
+ </div>
+ <p id="status-message" class="status-message"></p>
+ <div id="recovery" class="recovery" hidden>
+ <span class="recovery-label">Start a new pairing link</span>
+ <button id="copy-command" class="copy-chip" type="button" aria-label="Copy skyvern browser extension-pair">
+ <code id="recovery-command">skyvern browser extension-pair</code>
+ <span id="copy-label" class="copy-label">Copy</span>
+ </button>
+ </div>
+ </section>
+ </div>
+ </main>
+ </div>
+ <script>
+ (() => {
+ const nonce = location.hash.slice(1);
+ const command = "skyvern browser extension-pair";
+ const button = document.getElementById("approve");
+ const buttonLabel = document.getElementById("approve-label");
+ const copyButton = document.getElementById("copy-command");
+ const copyLabel = document.getElementById("copy-label");
+ const recovery = document.getElementById("recovery");
+ const serverAddress = document.getElementById("server-address");
+ const status = document.getElementById("status");
+ const statusTitle = document.getElementById("status-title");
+ const statusMessage = document.getElementById("status-message");
+
+ serverAddress.textContent = `127.0.0.1:${location.port}`;
+
+ function showState(kind, title, message, showRecovery = false) {
+ document.body.dataset.state = kind;
+ status.hidden = false;
+ statusTitle.textContent = title;
+ statusMessage.textContent = message;
+ recovery.hidden = !showRecovery;
+ }
+
+ function showExpired() {
+ showState(
+ "error",
+ "Pairing link expired",
+ "This one-time link is no longer valid. Start a new pairing request to continue.",
+ true,
+ );
+ }
+
+ function showInstallHint() {
+ showState(
+ "error",
+ "Skyvern Agent isn’t available",
+ "Install or enable the Skyvern Agent Chrome extension, then start a new pairing request.",
+ true,
+ );
+ }
+
+ async function copyRecoveryCommand() {
+ try {
+ await navigator.clipboard.writeText(command);
+ } catch (_error) {
+ const range = document.createRange();
+ range.selectNodeContents(document.getElementById("recovery-command"));
+ const selection = globalThis.getSelection();
+ selection.removeAllRanges();
+ selection.addRange(range);
+ document.execCommand("copy");
+ selection.removeAllRanges();
+ }
+ copyLabel.textContent = "Copied";
+ setTimeout(() => { copyLabel.textContent = "Copy"; }, 1600);
+ }
+
+ button.addEventListener("click", async () => {
+ button.disabled = true;
+ button.setAttribute("aria-busy", "true");
+ buttonLabel.textContent = "Approving…";
+ document.body.dataset.state = "approving";
+ if (!nonce) {
+ showExpired();
+ return;
+ }
+ try {
+ const response = await fetch("/pair/claim", {
+ method: "POST",
+ headers: {"Content-Type": "application/json"},
+ body: JSON.stringify({v: 1, nonce}),
+ });
+ if (response.status === 403) {
+ showExpired();
+ return;
+ }
+ if (!response.ok) {
+ throw new Error("claim failed");
+ }
+ const offer = await response.json();
+ if (!globalThis.chrome?.runtime?.sendMessage) {
+ showInstallHint();
+ return;
+ }
+ chrome.runtime.sendMessage(
+ "__EXTENSION_ID__",
+ {type: "skyvern.pairingOffer", v: 1, port: offer.port, token: offer.token},
+ (result) => {
+ if (chrome.runtime.lastError) {
+ showInstallHint();
+ } else if (result?.ok === true && result?.pending === true) {
+ showState(
+ "success",
+ "Approved — finish in the Skyvern Agent tab",
+ "Review the local server details in the confirmation tab that just opened.",
+ );
+ } else {
+ showState(
+ "error",
+ "Pairing request wasn’t accepted",
+ "The extension could not open the confirmation step. Start a new pairing request to try again.",
+ true,
+ );
+ }
+ },
+ );
+ } catch (_error) {
+ showState(
+ "error",
+ "Pairing couldn’t be completed",
+ "Check that the local MCP server is running, then start a new pairing request.",
+ true,
+ );
+ }
+ });
+
+ copyButton.addEventListener("click", () => { void copyRecoveryCommand(); });
+ })();
+ </script>
+</body>
+</html>
+"""
+_PAIR_PAGE = _PAIR_PAGE_TEMPLATE.replace("__EXTENSION_ID__", EXTENSION_ID)
+
+
+class ExtensionRelayServer:
+ def __init__(
+ self,
+ token: str,
+ port: int,
+ on_event: Callable[[str, dict], Awaitable[None]],
+ on_disconnect: Callable[[], Awaitable[None]] | None = None,
+ ) -> None:
+ self._token = token
+ self._port = port
+ self._on_event = on_event
+ self._on_disconnect = on_disconnect
+ self._app = web.Application()
+ self._app.router.add_get("/extension/v1", self._handle_websocket)
+ self._app.router.add_get("/pair", self._handle_pair_page)
+ self._app.router.add_post("/pair/begin", self._handle_pair_begin)
+ self._app.router.add_post("/pair/claim", self._handle_pair_claim)
+ self._runner: web.AppRunner | None = None
+ self._site: web.TCPSite | None = None
+ self._websocket: web.WebSocketResponse | None = None
+ self._connection_lock = asyncio.Lock()
+ self._connected_event = asyncio.Event()
+ self._request_ids = itertools.count(1)
+ self._pending: dict[str, asyncio.Future[dict]] = {}
+ self._pairing_nonce: str | None = None
+ self._pairing_nonce_created_at: float | None = None
+ self.bound_port = port
+ self.scoped_tabs: list[dict] = []
+
+ def create_pairing_nonce(self) -> str:
+ nonce = secrets.token_urlsafe(32)
+ self._pairing_nonce = nonce
+ self._pairing_nonce_created_at = time.monotonic()
+ return nonce
+
+ async def start(self) -> None:
+ if self._runner is not None:
+ return
+
+ runner = web.AppRunner(self._app)
+ await runner.setup()
+ site = web.TCPSite(runner, "127.0.0.1", self._port)
+ try:
+ await site.start()
+ except BaseException:
+ await runner.cleanup()
+ raise
+
+ server = site._server
+ if server is None or not server.sockets:
+ await runner.cleanup()
+ raise BrowserExtensionError("Browser extension relay did not bind a loopback port")
+
+ self.bound_port = int(server.sockets[0].getsockname()[1])
+ self._runner = runner
+ self._site = site
+
+ async def stop(self) -> None:
+ runner = self._runner
+ if runner is None:
+ return
+
+ websocket = self._websocket
+ if websocket is not None and not websocket.closed:
+ await websocket.close(code=1001, message=b"relay stopped")
+ await runner.cleanup()
+ if websocket is not None:
+ await self._handle_disconnect(websocket)
+ self._runner = None
+ self._site = None
+
+ @property
+ def connected(self) -> bool:
+ return self._connected_event.is_set() and self._websocket is not None and not self._websocket.closed
+
+ async def wait_connected(self, timeout: float) -> bool:
+ if self.connected:
+ return True
+ try:
+ await asyncio.wait_for(self._connected_event.wait(), timeout)
+ except TimeoutError:
+ return False
+ return self.connected
+
+ async def request(self, op: str, args: dict, timeout: float = 30.0) -> dict:
+ websocket = self._websocket
+ if not self.connected or websocket is None:
+ raise BrowserExtensionNotConnectedError("Skyvern browser extension is not connected")
+
+ request_id = f"r-{next(self._request_ids)}"
+ frame = build_request(request_id, op, args)
+ future: asyncio.Future[dict] = asyncio.get_running_loop().create_future()
+ self._pending[request_id] = future
+ try:
+ await websocket.send_json(frame)
+ except (ConnectionError, RuntimeError):
+ pending = self._pending.pop(request_id, None)
+ if pending is not None:
+ pending.cancel()
+ raise BrowserExtensionNotConnectedError("Skyvern browser extension is not connected") from None
+
+ try:
+ return await asyncio.wait_for(asyncio.shield(future), timeout)
+ except TimeoutError:
+ pending = self._pending.pop(request_id, None)
+ if pending is not None:
+ pending.cancel()
+ raise ExtensionRequestError("INTERNAL", f"extension request timed out: {op}") from None
+ except asyncio.CancelledError:
+ pending = self._pending.pop(request_id, None)
+ if pending is not None:
+ pending.cancel()
+ raise
+
+ async def _handle_pair_page(self, _request: web.Request) -> web.Response:
+ return web.Response(
+ text=_PAIR_PAGE,
+ content_type="text/html",
+ headers={
+ "Cache-Control": "no-store",
+ "Content-Security-Policy": (
+ "default-src 'none'; script-src 'unsafe-inline'; style-src 'unsafe-inline'; "
+ "img-src data:; connect-src 'self'"
+ ),
+ "Referrer-Policy": "no-referrer",
+ "X-Content-Type-Options": "nosniff",
+ },
+ )
+
+ async def _handle_pair_begin(self, request: web.Request) -> web.Response:
+ payload = await _read_json_object(request)
+ proof = payload.get("proof") if payload is not None else None
+ expected = hmac.new(self._token.encode("utf-8"), _PAIR_BEGIN_MESSAGE, hashlib.sha256).hexdigest()
+ supplied = proof if isinstance(proof, str) else ""
+ valid = hmac.compare_digest(expected, supplied)
+ if payload is None or type(payload.get("v")) is not int or payload["v"] != PROTOCOL_VERSION or not valid:
+ return web.json_response({"error": "invalid_proof"}, status=403, headers={"Cache-Control": "no-store"})
+ return web.json_response(
+ {"v": PROTOCOL_VERSION, "nonce": self.create_pairing_nonce()},
+ headers={"Cache-Control": "no-store"},
+ )
+
+ async def _handle_pair_claim(self, request: web.Request) -> web.Response:
+ active_nonce = self._pairing_nonce
+ created_at = self._pairing_nonce_created_at
+ self._pairing_nonce = None
+ self._pairing_nonce_created_at = None
+
+ payload = await _read_json_object(request)
+ supplied_nonce = payload.get("nonce") if payload is not None else None
+ nonce_matches = secrets.compare_digest(
+ active_nonce or ("0" * 43),
+ supplied_nonce if isinstance(supplied_nonce, str) else "",
+ )
+ expired = created_at is None or time.monotonic() - created_at >= _PAIRING_NONCE_TTL_SECONDS
+ if (
+ payload is None
+ or type(payload.get("v")) is not int
+ or payload["v"] != PROTOCOL_VERSION
+ or not nonce_matches
+ or expired
+ ):
+ return web.json_response({"error": "invalid_nonce"}, status=403, headers={"Cache-Control": "no-store"})
+ return web.json_response(
+ {"v": PROTOCOL_VERSION, "port": self.bound_port, "token": self._token},
+ headers={"Cache-Control": "no-store"},
+ )
+
+ async def _handle_websocket(self, request: web.Request) -> web.WebSocketResponse:
+ websocket = web.WebSocketResponse(max_msg_size=_MAX_WS_MESSAGE_BYTES)
+ await websocket.prepare(request)
+
+ origin = request.headers.get("Origin")
+ if origin is not None and not _is_extension_origin(origin):
+ await websocket.close(code=_AUTH_CLOSE_CODE, message=b"authentication failed")
+ return websocket
+
+ server_nonce, challenge = build_challenge()
+ await websocket.send_str(challenge)
+ try:
+ proof_frame = await websocket.receive(timeout=_AUTH_TIMEOUT_SECONDS)
+ if proof_frame.type is not WSMsgType.TEXT:
+ raise BrowserExtensionError("Authentication proof must be a text frame")
+ proof = parse_extension_message(proof_frame.data)
+ if proof.kind != "auth.proof" or proof.client_nonce is None or proof.proof is None:
+ raise BrowserExtensionError("Authentication proof is missing")
+ if not _is_valid_client_nonce(proof.client_nonce):
+ raise BrowserExtensionError("Authentication nonce is invalid")
+ if not verify_ext_proof(self._token, server_nonce, proof.client_nonce, proof.proof):
+ raise BrowserExtensionError("Authentication proof is invalid")
+ except (BrowserExtensionError, TimeoutError):
+ await websocket.close(code=_AUTH_CLOSE_CODE, message=b"authentication failed")
+ return websocket
+
+ await websocket.send_json(
+ {
+ "v": 1,
+ "type": "auth.ok",
+ "serverProof": compute_server_proof(self._token, proof.client_nonce, server_nonce),
+ }
+ )
+ await self._activate_connection(websocket)
+
+ loop = asyncio.get_running_loop()
+ last_inbound = loop.time()
+ keepalive_task = asyncio.create_task(self._run_keepalive(websocket, lambda: last_inbound))
+ try:
+ async for message in websocket:
+ last_inbound = loop.time()
+ if self._websocket is not websocket:
+ break
+ if message.type is WSMsgType.TEXT:
+ await self._handle_text_frame(websocket, message.data)
+ elif message.type in {WSMsgType.CLOSE, WSMsgType.CLOSED, WSMsgType.ERROR}:
+ break
+ else:
+ LOG.warning("browser extension sent a non-text frame")
+ finally:
+ keepalive_task.cancel()
+ with suppress(asyncio.CancelledError):
+ await keepalive_task
+ await self._handle_disconnect(websocket)
+ return websocket
+
+ async def _activate_connection(self, websocket: web.WebSocketResponse) -> None:
+ async with self._connection_lock:
+ previous = self._websocket
+ self._websocket = websocket
+ self._connected_event.clear()
+
+ if previous is not None and previous is not websocket:
+ self._fail_pending_requests()
+ self.scoped_tabs = []
+ await self._call_on_disconnect()
+ if not previous.closed:
+ await previous.close(code=_REPLACED_CLOSE_CODE, message=b"replaced")
+
+ async def _handle_text_frame(self, websocket: web.WebSocketResponse, raw: str) -> None:
+ try:
+ message = parse_extension_message(raw)
+ except BrowserExtensionError:
+ LOG.warning("browser extension sent an invalid protocol frame")
+ return
+
+ if message.kind == "response":
+ self._handle_response(message)
+ elif message.kind == "event":
+ await self._handle_event(message)
+ if message.event == "extension.hello":
+ await self._mark_hello_processed(websocket)
+ elif message.kind == "ping":
+ await websocket.send_json({"v": 1, "type": "pong"})
+
+ async def _mark_hello_processed(self, websocket: web.WebSocketResponse) -> None:
+ async with self._connection_lock:
+ if self._websocket is websocket and not websocket.closed:
+ self._connected_event.set()
+
+ def _handle_response(self, message: ParsedMessage) -> None:
+ if message.request_id is None:
+ return
+ future = self._pending.pop(message.request_id, None)
+ if future is None or future.done():
+ return
+ if message.ok:
+ future.set_result(message.result or {})
+ return
+ if message.error_code is None or message.error_message is None:
+ future.set_exception(ExtensionRequestError("INTERNAL", "extension returned an invalid error"))
+ return
+ future.set_exception(ExtensionRequestError(message.error_code, message.error_message))
+
+ async def _handle_event(self, message: ParsedMessage) -> None:
+ if message.event is None or message.params is None:
+ return
+ self._update_scoped_tabs(message.event, message.params)
+ try:
+ await self._on_event(message.event, message.params)
+ except Exception:
+ LOG.exception("browser extension event callback failed", event=message.event)
+
+ def _update_scoped_tabs(self, event: str, params: dict) -> None:
+ if event == "extension.hello":
+ tabs = params.get("scopedTabs")
+ if not isinstance(tabs, list):
+ self.scoped_tabs = []
+ return
+ self.scoped_tabs = [snapshot for tab in tabs if (snapshot := _tab_snapshot(tab)) is not None]
+ return
+ if event in {"scope.tabAdded", "tabs.created"}:
+ snapshot = _tab_snapshot(params)
+ if snapshot is None:
+ return
+ self.scoped_tabs = [tab for tab in self.scoped_tabs if tab.get("tabId") != snapshot["tabId"]]
+ self.scoped_tabs.append(snapshot)
+ return
+ if event == "scope.tabRemoved":
+ tab_id = params.get("tabId")
+ if type(tab_id) is int:
+ self.scoped_tabs = [tab for tab in self.scoped_tabs if tab.get("tabId") != tab_id]
+
+ async def _run_keepalive(
+ self,
+ websocket: web.WebSocketResponse,
+ last_inbound: Callable[[], float],
+ ) -> None:
+ loop = asyncio.get_running_loop()
+ next_ping = loop.time() + _PING_INTERVAL_SECONDS
+ while not websocket.closed:
+ now = loop.time()
+ silence = now - last_inbound()
+ if silence >= _INBOUND_TIMEOUT_SECONDS:
+ await websocket.close(code=1001, message=b"inbound timeout")
+ return
+ if now >= next_ping:
+ try:
+ await websocket.send_json({"v": 1, "type": "ping"})
+ except (ConnectionError, RuntimeError):
+ return
+ next_ping = now + _PING_INTERVAL_SECONDS
+ delay = min(next_ping - now, _INBOUND_TIMEOUT_SECONDS - silence)
+ await asyncio.sleep(max(delay, 0.01))
+
+ async def _handle_disconnect(self, websocket: web.WebSocketResponse) -> None:
+ async with self._connection_lock:
+ if self._websocket is not websocket:
+ return
+ self._websocket = None
+ self._connected_event.clear()
+
+ self._fail_pending_requests()
+ self.scoped_tabs = []
+ await self._call_on_disconnect()
+
+ async def _call_on_disconnect(self) -> None:
+ if self._on_disconnect is not None:
+ try:
+ await self._on_disconnect()
+ except Exception:
+ LOG.exception("browser extension disconnect callback failed")
+
+ def _fail_pending_requests(self) -> None:
+ pending = list(self._pending.values())
+ self._pending.clear()
+ for future in pending:
+ if not future.done():
+ future.set_exception(BrowserExtensionNotConnectedError("Skyvern browser extension is not connected"))
+
+
+def _is_extension_origin(origin: str) -> bool:
+ try:
+ parsed = urlsplit(origin)
+ except ValueError:
+ return False
+ return parsed.scheme == "chrome-extension" and bool(parsed.netloc)
+
+
+async def _read_json_object(request: web.Request) -> dict | None:
+ try:
+ payload = await request.json(loads=json.loads)
+ except (json.JSONDecodeError, UnicodeDecodeError):
+ return None
+ return payload if isinstance(payload, dict) else None
+
+
+def _is_valid_client_nonce(client_nonce: str) -> bool:
+ if not client_nonce or "=" in client_nonce:
+ return False
+ padding = "=" * (-len(client_nonce) % 4)
+ try:
+ decoded = base64.b64decode(client_nonce + padding, altchars=b"-_", validate=True)
+ except (binascii.Error, ValueError):
+ return False
+ canonical = base64.urlsafe_b64encode(decoded).rstrip(b"=").decode("ascii")
+ return len(decoded) == 32 and canonical == client_nonce
+
+
+def _tab_snapshot(value: object) -> dict | None:
+ if not isinstance(value, dict):
+ return None
+ tab_id = value.get("tabId")
+ if type(tab_id) is not int:
+ return None
+ url = value.get("url")
+ title = value.get("title")
+ return {
+ "tabId": tab_id,
+ "url": url if isinstance(url, str) else "",
+ "title": title if isinstance(title, str) else "",
+ }
diff --git a/skyvern/browser_extension/runtime.py b/skyvern/browser_extension/runtime.py
new file mode 100644
index 000000000..5eb36afb6
--- /dev/null
+++ b/skyvern/browser_extension/runtime.py
@@ -0,0 +1,164 @@
+from __future__ import annotations
+
+import asyncio
+import errno
+import os
+from collections.abc import Awaitable, Callable
+from pathlib import Path
+from typing import Protocol
+
+import structlog
+
+from skyvern.browser_extension.auth import load_or_create_pairing_token
+from skyvern.browser_extension.errors import BrowserExtensionError
+from skyvern.browser_extension.relay import ExtensionRelayServer
+from skyvern.browser_extension.target_registry import VirtualTargetRegistry
+
+LOG = structlog.get_logger(__name__)
+
+_PORT_ENV = "SKYVERN_BROWSER_EXTENSION_PORT"
+_DEFAULT_PORT = 19777
+
+
+class _Adapter(Protocol):
+ @property
+ def cdp_ws_url(self) -> str: ...
+
+ async def start(self) -> None: ...
+
+ async def stop(self) -> None: ...
+
+ async def handle_extension_event(self, event: str, params: dict) -> None: ...
+
+ async def on_extension_disconnect(self) -> None: ...
+
+
+_relay_factory: Callable[
+ [str, int, Callable[[str, dict], Awaitable[None]], Callable[[], Awaitable[None]] | None],
+ ExtensionRelayServer,
+] = ExtensionRelayServer
+_adapter_factory: Callable[[VirtualTargetRegistry, ExtensionRelayServer], _Adapter] | None = None
+
+
+class BrowserExtensionRuntime:
+ _instance: BrowserExtensionRuntime | None = None
+ _lock = asyncio.Lock()
+
+ def __init__(self, relay: ExtensionRelayServer, adapter: _Adapter) -> None:
+ self._relay = relay
+ self._adapter = adapter
+ self._stopped = False
+
+ @classmethod
+ async def get_or_start(cls, port: int | None = None) -> BrowserExtensionRuntime:
+ async with cls._lock:
+ if cls._instance is not None:
+ return cls._instance
+
+ resolved_port = _resolve_port(port)
+ token = load_or_create_pairing_token()
+ registry = VirtualTargetRegistry()
+ adapter_holder: list[_Adapter] = []
+
+ async def handle_event(event: str, params: dict) -> None:
+ await adapter_holder[0].handle_extension_event(event, params)
+
+ async def on_disconnect() -> None:
+ await adapter_holder[0].on_extension_disconnect()
+
+ relay = _relay_factory(token, resolved_port, handle_event, on_disconnect)
+ adapter = _create_adapter(registry, relay)
+ adapter_holder.append(adapter)
+
+ try:
+ await adapter.start()
+ await relay.start()
+ except OSError as exc:
+ await _cleanup_failed_start(relay, adapter)
+ if exc.errno == errno.EADDRINUSE:
+ raise BrowserExtensionError(
+ f"Browser extension relay port {resolved_port} is already in use; "
+ f"set {_PORT_ENV} to a free port"
+ ) from exc
+ raise
+ except BaseException:
+ await _cleanup_failed_start(relay, adapter)
+ raise
+
+ runtime = cls(relay, adapter)
+ cls._instance = runtime
+ return runtime
+
+ @classmethod
+ def instance(cls) -> BrowserExtensionRuntime | None:
+ return cls._instance
+
+ @property
+ def cdp_ws_url(self) -> str:
+ return self._adapter.cdp_ws_url
+
+ @property
+ def extension_connected(self) -> bool:
+ return self._relay.connected
+
+ async def wait_for_extension(self, timeout: float = 10.0) -> bool:
+ return await self._relay.wait_connected(timeout)
+
+ async def shutdown(self) -> None:
+ cls = type(self)
+ async with cls._lock:
+ if self._stopped:
+ return
+ self._stopped = True
+ try:
+ await self._relay.stop()
+ finally:
+ try:
+ await self._adapter.stop()
+ finally:
+ if cls._instance is self:
+ cls._instance = None
+
+ @staticmethod
+ def extension_dir() -> Path:
+ return Path(__file__).resolve().parent / "extension"
+
+ @staticmethod
+ def configured_port() -> int:
+ return _resolve_port(None)
+
+
+def _create_adapter(registry: VirtualTargetRegistry, relay: ExtensionRelayServer) -> _Adapter:
+ if _adapter_factory is not None:
+ return _adapter_factory(registry, relay)
+ from skyvern.browser_extension.cdp_adapter import ExtensionCdpAdapter
+
+ return ExtensionCdpAdapter(registry, relay)
+
+
+def _resolve_port(port: int | None) -> int:
+ if port is not None:
+ resolved_port = port
+ else:
+ environment_port = os.environ.get(_PORT_ENV)
+ if environment_port is None:
+ resolved_port = _DEFAULT_PORT
+ else:
+ try:
+ resolved_port = int(environment_port)
+ except ValueError as exc:
+ raise BrowserExtensionError(f"{_PORT_ENV} must be an integer port") from exc
+ if type(resolved_port) is not int or not 1 <= resolved_port <= 65535:
+ raise BrowserExtensionError(f"{_PORT_ENV} must be a port between 1 and 65535")
+ return resolved_port
+
+
+async def _cleanup_failed_start(relay: ExtensionRelayServer, adapter: _Adapter) -> None:
+ try:
+ await relay.stop()
+ except Exception:
+ LOG.exception("failed to stop browser extension relay after startup error")
+ try:
+ await adapter.stop()
+ except Exception:
+ LOG.exception("failed to stop browser extension adapter after startup error")
diff --git a/skyvern/browser_extension/target_registry.py b/skyvern/browser_extension/target_registry.py
new file mode 100644
index 000000000..2f47f803a
--- /dev/null
+++ b/skyvern/browser_extension/target_registry.py
@@ -0,0 +1,173 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+
+@dataclass(frozen=True, slots=True)
+class _TabRecord:
+ target_id: str
+ root_session_id: str
+
+
+@dataclass(frozen=True, slots=True)
+class _ChildSessionRecord:
+ tab_id: int
+ target_id: str
+
+
+class VirtualTargetRegistry:
+ def __init__(self) -> None:
+ self._attach_counter = 0
+ self._alias_counter = 0
+ self._tabs: dict[int, _TabRecord] = {}
+ self._root_sessions: dict[str, int] = {}
+ self._tab_root_aliases: dict[int, list[str]] = {}
+ self._browser_session_aliases: set[str] = set()
+ self._child_sessions: dict[str, _ChildSessionRecord] = {}
+ self._child_target_sessions: dict[str, str] = {}
+ self._target_tabs: dict[str, int] = {}
+ self._target_infos: dict[str, dict] = {}
+
+ def register_tab(self, tab_id: int, url: str, title: str, target_id: str | None = None) -> str:
+ self.remove_tab(tab_id)
+ self._attach_counter += 1
+ # Playwright requires the page targetId to equal Chrome's main-frame id;
+ # callers pass the real frame id once the debugger is attached.
+ target_id = target_id or f"tab-{tab_id}"
+ root_session_id = f"sess-tab-{tab_id}-{self._attach_counter}"
+ self._tabs[tab_id] = _TabRecord(target_id=target_id, root_session_id=root_session_id)
+ self._root_sessions[root_session_id] = tab_id
+ self._target_tabs[target_id] = tab_id
+ self._target_infos[target_id] = {
+ "targetId": target_id,
+ "type": "page",
+ "title": title,
+ "url": url,
+ "attached": True,
+ "canAccessOpener": False,
+ "browserContextId": "skyvern-default",
+ }
+ return target_id
+
+ def remove_tab(self, tab_id: int) -> None:
+ tab = self._tabs.pop(tab_id, None)
+ if tab is not None:
+ self._root_sessions.pop(tab.root_session_id, None)
+ for alias_session_id in self._tab_root_aliases.pop(tab_id, []):
+ self._root_sessions.pop(alias_session_id, None)
+ self._target_tabs.pop(tab.target_id, None)
+ self._target_infos.pop(tab.target_id, None)
+
+ child_session_ids = [
+ session_id for session_id, record in self._child_sessions.items() if record.tab_id == tab_id
+ ]
+ for session_id in child_session_ids:
+ self.remove_child_session(session_id)
+
+ def root_session_id(self, tab_id: int) -> str:
+ return self._tabs[tab_id].root_session_id
+
+ def root_session_ids(self, tab_id: int) -> list[str]:
+ return [self.root_session_id(tab_id), *self._tab_root_aliases.get(tab_id, [])]
+
+ def create_root_session_alias(self, tab_id: int) -> str:
+ if tab_id not in self._tabs:
+ raise KeyError(tab_id)
+ self._alias_counter += 1
+ session_id = f"alias-tab-{tab_id}-{self._alias_counter}"
+ self._root_sessions[session_id] = tab_id
+ self._tab_root_aliases.setdefault(tab_id, []).append(session_id)
+ return session_id
+
+ def remove_root_session_alias(self, session_id: str) -> bool:
+ tab_id = self._root_sessions.get(session_id)
+ if tab_id is None:
+ return False
+ aliases = self._tab_root_aliases.get(tab_id)
+ if aliases is None or session_id not in aliases:
+ return False
+ aliases.remove(session_id)
+ if not aliases:
+ self._tab_root_aliases.pop(tab_id, None)
+ self._root_sessions.pop(session_id, None)
+ return True
+
+ def create_browser_session_alias(self) -> str:
+ self._alias_counter += 1
+ session_id = f"browser-alias-{self._alias_counter}"
+ self._browser_session_aliases.add(session_id)
+ return session_id
+
+ def is_browser_session_alias(self, session_id: str) -> bool:
+ return session_id in self._browser_session_aliases
+
+ def remove_browser_session_alias(self, session_id: str) -> bool:
+ if session_id not in self._browser_session_aliases:
+ return False
+ self._browser_session_aliases.remove(session_id)
+ return True
+
+ def target_id_for_tab(self, tab_id: int) -> str:
+ return self._tabs[tab_id].target_id
+
+ def target_info_for_tab(self, tab_id: int) -> dict:
+ return self.target_info(self._tabs[tab_id].target_id)
+
+ def register_child_session(self, tab_id: int, chrome_session_id: str, target_info: dict) -> None:
+ if tab_id not in self._tabs:
+ raise KeyError(tab_id)
+ target_id = target_info["targetId"]
+ if not isinstance(target_id, str):
+ raise KeyError("targetId")
+
+ self.remove_child_session(chrome_session_id)
+ existing_session_id = self._child_target_sessions.get(target_id)
+ if existing_session_id is not None:
+ self.remove_child_session(existing_session_id)
+
+ self._child_sessions[chrome_session_id] = _ChildSessionRecord(tab_id=tab_id, target_id=target_id)
+ self._child_target_sessions[target_id] = chrome_session_id
+ self._target_tabs[target_id] = tab_id
+ self._target_infos[target_id] = dict(target_info)
+
+ def remove_child_session(self, chrome_session_id: str) -> None:
+ record = self._child_sessions.pop(chrome_session_id, None)
+ if record is None:
+ return
+ if self._child_target_sessions.get(record.target_id) == chrome_session_id:
+ self._child_target_sessions.pop(record.target_id, None)
+ self._target_tabs.pop(record.target_id, None)
+ self._target_infos.pop(record.target_id, None)
+
+ def resolve_session(self, session_id: str) -> tuple[int, str | None]:
+ root_tab_id = self._root_sessions.get(session_id)
+ if root_tab_id is not None:
+ return root_tab_id, None
+ child = self._child_sessions.get(session_id)
+ if child is None:
+ raise KeyError(session_id)
+ return child.tab_id, session_id
+
+ def target_info(self, target_id: str) -> dict:
+ return dict(self._target_infos[target_id])
+
+ def list_page_targets(self) -> list[dict]:
+ return [self.target_info(tab.target_id) for tab in self._tabs.values()]
+
+ def tab_for_target(self, target_id: str) -> int:
+ return self._target_tabs[target_id]
+
+ def update_tab(self, tab_id: int, url: str, title: str) -> None:
+ target_id = self._tabs[tab_id].target_id
+ self._target_infos[target_id]["url"] = url
+ self._target_infos[target_id]["title"] = title
+
+ def clear(self) -> None:
+ self._tabs.clear()
+ self._root_sessions.clear()
+ self._tab_root_aliases.clear()
+ self._browser_session_aliases.clear()
+ self._child_sessions.clear()
+ self._child_target_sessions.clear()
+ self._target_tabs.clear()
+ self._target_infos.clear()
diff --git a/skyvern/cli/commands/browser.py b/skyvern/cli/commands/browser.py
index 37a57748d..7d069312c 100644
--- a/skyvern/cli/commands/browser.py
+++ b/skyvern/cli/commands/browser.py
@@ -1,7 +1,14 @@
from __future__ import annotations
import asyncio
+import hashlib
+import hmac
+import http.client
import json
+import os
+import shutil
+import socket
+import stat
import subprocess
import sys
from dataclasses import asdict, dataclass
@@ -14,6 +21,9 @@ if TYPE_CHECKING:
import typer
+from skyvern.browser_extension.auth import load_or_create_pairing_token
+from skyvern.browser_extension.errors import BrowserExtensionError
+from skyvern.browser_extension.runtime import BrowserExtensionRuntime
from skyvern.cli.commands._output import (
console,
)
@@ -182,6 +192,254 @@ def _handle_tool_error(e: Exception, *, tool: str, hint: str, json_output: bool)
output_error(str(e), hint=hint, json_mode=json_output)
+def _clipboard_command() -> list[str] | None:
+ if sys.platform == "darwin":
+ executable = shutil.which("pbcopy")
+ return [executable] if executable else None
+ if sys.platform.startswith("linux"):
+ executable = shutil.which("xclip")
+ if executable:
+ return [executable, "-selection", "clipboard"]
+ executable = shutil.which("xsel")
+ if executable:
+ return [executable, "--clipboard", "--input"]
+ return None
+
+
+def _copy_pairing_token_to_clipboard(token: str) -> bool:
+ command = _clipboard_command()
+ if command is None:
+ return False
+ try:
+ subprocess.run(
+ command,
+ input=token,
+ text=True,
+ check=True,
+ stdout=subprocess.DEVNULL,
+ stderr=subprocess.DEVNULL,
+ )
+ except (OSError, subprocess.CalledProcessError):
+ return False
+ return True
+
+
+def _open_chrome_extensions() -> bool:
+ url = "chrome://extensions"
+ command: list[str] | None = None
+ if sys.platform == "darwin":
+ executable = shutil.which("open")
+ if executable:
+ command = [executable, "-a", "Google Chrome", url]
+ elif sys.platform.startswith("linux"):
+ for browser_name in ("google-chrome", "chromium", "chromium-browser"):
+ executable = shutil.which(browser_name)
+ if executable:
+ command = [executable, url]
+ break
+ if command is None:
+ return False
+ try:
+ subprocess.run(command, check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
+ except (OSError, subprocess.CalledProcessError):
+ return False
+ return True
+
+
+def _pairing_token_file_status() -> tuple[bool, str]:
+ token_path = Path.home() / ".skyvern" / "browser_extension_token"
+ try:
+ file_status = token_path.lstat()
+ except FileNotFoundError:
+ return False, "not applicable (file does not exist)"
+ except OSError:
+ return False, "NOT OK (file cannot be inspected)"
+
+ permissions_ok = stat.S_ISREG(file_status.st_mode) and not (stat.S_IMODE(file_status.st_mode) & 0o177)
+ if hasattr(os, "getuid"):
+ permissions_ok = permissions_ok and file_status.st_uid == os.getuid()
+ return True, "OK" if permissions_ok else "NOT OK"
+
+
+def _bridge_is_listening(port: int) -> bool:
+ try:
+ connection = socket.create_connection(("127.0.0.1", port), timeout=0.5)
+ except OSError:
+ return False
+ try:
+ connection.close()
+ except OSError:
+ pass
+ return True
+
+
+def _request_pairing_nonce(port: int, token: str) -> str:
+ proof = hmac.new(token.encode("utf-8"), b"skyvern-pair-begin-v1", hashlib.sha256).hexdigest()
+ body = json.dumps({"v": 1, "proof": proof}, separators=(",", ":"))
+ connection = http.client.HTTPConnection("127.0.0.1", port, timeout=2.0)
+ try:
+ connection.request("POST", "/pair/begin", body=body, headers={"Content-Type": "application/json"})
+ response = connection.getresponse()
+ response_body = response.read(65_537)
+ except OSError as exc:
+ raise BrowserExtensionError("The browser extension bridge did not accept the pairing request") from exc
+ finally:
+ connection.close()
+ if response.status != 200:
+ raise BrowserExtensionError("The browser extension bridge rejected the pairing request")
+ if len(response_body) > 65_536:
+ raise BrowserExtensionError("The browser extension bridge returned an invalid pairing response")
+ try:
+ payload = json.loads(response_body)
+ except (json.JSONDecodeError, UnicodeDecodeError) as exc:
+ raise BrowserExtensionError("The browser extension bridge returned an invalid pairing response") from exc
+ nonce = (
+ payload.get("nonce")
+ if isinstance(payload, dict) and type(payload.get("v")) is int and payload["v"] == 1
+ else None
+ )
+ if not isinstance(nonce, str) or not nonce:
+ raise BrowserExtensionError("The browser extension bridge returned an invalid pairing response")
+ return nonce
+
+
+def _open_pairing_url(url: str) -> bool:
+ command: list[str] | None = None
+ if sys.platform == "darwin":
+ executable = shutil.which("open")
+ if executable:
+ command = [executable, url]
+ elif sys.platform.startswith("linux"):
+ executable = shutil.which("xdg-open")
+ if executable:
+ command = [executable, url]
+ if command is None:
+ return False
+ try:
+ subprocess.run(command, check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
+ except (OSError, subprocess.CalledProcessError):
+ return False
+ return True
+
+
+def _launch_extension_pairing(port: int) -> None:
+ token = load_or_create_pairing_token()
+ nonce = _request_pairing_nonce(port, token)
+ url = f"http://127.0.0.1:{port}/pair#{nonce}"
+ if not _open_pairing_url(url):
+ console.print(url, markup=False, soft_wrap=True)
+ console.print("Approve the pairing in your browser.")
+
+
+@browser_app.command("extension-path")
+def extension_path() -> None:
+ console.print(str(BrowserExtensionRuntime.extension_dir().resolve()), markup=False, soft_wrap=True)
+
+
+@browser_app.command("extension-token")
+def extension_token() -> None:
+ """Copy the pairing token for manual popup-paste fallback."""
+ token = load_or_create_pairing_token()
+ if _copy_pairing_token_to_clipboard(token):
+ console.print("Pairing token copied to clipboard.")
+ else:
+ console.print(
+ "Pairing token was not printed. Copy it from ~/.skyvern/browser_extension_token "
+ "or SKYVERN_BROWSER_EXTENSION_TOKEN."
+ )
+ console.print("Paste this token into the Skyvern browser extension popup.")
+ console.print("Click Connect.")
+
+
+@browser_app.command("extension-status")
+def extension_status() -> None:
+ console.print(
+ f"extension directory: {BrowserExtensionRuntime.extension_dir().resolve()}",
+ markup=False,
+ soft_wrap=True,
+ )
+
+ environment_configured = bool(os.environ.get("SKYVERN_BROWSER_EXTENSION_TOKEN", "").strip())
+ token_file_exists, permissions = _pairing_token_file_status()
+ if environment_configured:
+ console.print("pairing token: configured (environment override)")
+ elif token_file_exists:
+ console.print("pairing token: configured (file exists)")
+ else:
+ console.print("pairing token: not configured")
+ console.print(f"pairing token file permissions: {permissions}")
+
+ try:
+ port = BrowserExtensionRuntime.configured_port()
+ except BrowserExtensionError as exc:
+ console.print(f"bridge configuration invalid ({exc})")
+ return
+
+ if not _bridge_is_listening(port):
+ console.print("bridge not running (start your MCP server with --browser-extension)")
+ else:
+ console.print(f"bridge listening on {port}")
+
+
+@browser_app.command("extension-pair")
+def extension_pair() -> None:
+ try:
+ port = BrowserExtensionRuntime.configured_port()
+ except BrowserExtensionError:
+ console.print("Start your MCP server first: skyvern run mcp --browser-extension")
+ raise typer.Exit(code=1) from None
+ if not _bridge_is_listening(port):
+ console.print("Start your MCP server first: skyvern run mcp --browser-extension")
+ raise typer.Exit(code=1)
+ try:
+ _launch_extension_pairing(port)
+ except BrowserExtensionError as exc:
+ console.print(str(exc))
+ raise typer.Exit(code=1) from None
+
+
+@browser_app.command("extension-install")
+def extension_install() -> None:
+ extension_dir = BrowserExtensionRuntime.extension_dir().resolve()
+ console.print(str(extension_dir), markup=False, soft_wrap=True)
+
+ try:
+ port = BrowserExtensionRuntime.configured_port()
+ bridge_listening = _bridge_is_listening(port)
+ except BrowserExtensionError:
+ port = None
+ bridge_listening = False
+
+ if not bridge_listening:
+ token = load_or_create_pairing_token()
+ if _copy_pairing_token_to_clipboard(token):
+ console.print("Pairing token copied to clipboard.")
+ else:
+ console.print("No clipboard tool found. Use skyvern browser extension-token for the manual fallback.")
+
+ if _open_chrome_extensions():
+ console.print("Opened chrome://extensions.")
+ else:
+ console.print("chrome://extensions", markup=False)
+
+ console.print("1. Enable Developer mode.")
+ console.print("2. Click Load unpacked.")
+ console.print(f"3. Select {extension_dir}.", markup=False, soft_wrap=True)
+ if bridge_listening and port is not None:
+ console.print("4. Click Approve in the pairing page.")
+ console.print("5. Approve the pairing in the Skyvern Agent confirmation tab.")
+ console.print('6. Add tabs to the "Skyvern Controlled" group.')
+ try:
+ _launch_extension_pairing(port)
+ except BrowserExtensionError:
+ console.print("Automatic pairing could not start. Run skyvern browser extension-pair after setup.")
+ else:
+ console.print("4. Open the Skyvern Agent popup.")
+ console.print("5. Paste the pairing token and click Connect.")
+ console.print('6. Add tabs to the "Skyvern Controlled" group.')
+ console.print("When your MCP server is running, pair with: skyvern browser extension-pair")
+
+
# ---------------------------------------------------------------------------
# Session commands
# ---------------------------------------------------------------------------
diff --git a/skyvern/cli/core/browser_ops.py b/skyvern/cli/core/browser_ops.py
index d267a6a82..72be6460c 100644
--- a/skyvern/cli/core/browser_ops.py
+++ b/skyvern/cli/core/browser_ops.py
@@ -23,10 +23,44 @@ from skyvern.forge.sdk.settings_manager import SettingsManager
from skyvern.utils.url_validators import validate_fetch_url
from skyvern.webeye.utils.page import JS_FUNCTION_DEFS, SkyvernFrame
-from .guards import GuardError
+from .guards import CREDENTIAL_HINT, GuardError
LOG = structlog.get_logger(__name__)
+TYPE_PASSWORD_REFUSAL_MESSAGE = "Cannot type into password fields — credentials must not be passed through tool calls"
+COORDINATE_TYPE_TARGET_REFUSAL_MESSAGE = "could not verify the coordinate target; refusing to type"
+
+_COORDINATE_PASSWORD_TARGET_JS = """
+() => {
+ let element = document.activeElement;
+ while (element) {
+ const tag = (element.tagName || "").toLowerCase();
+ if (tag === "iframe" || tag === "frame") {
+ try {
+ element = element.contentDocument?.activeElement || null;
+ } catch {
+ return null;
+ }
+ if (!element) return null;
+ continue;
+ }
+ const shadowRoot = element.shadowRoot;
+ const shadowActiveElement = shadowRoot?.activeElement;
+ if (shadowActiveElement) {
+ element = shadowActiveElement;
+ continue;
+ }
+ if (tag.includes("-") && !shadowRoot) return null;
+ break;
+ }
+ if (!element || typeof element.getAttribute !== "function") return null;
+ const tag = (element.tagName || "").toLowerCase();
+ const type = (element.getAttribute("type") || "").toLowerCase();
+ const autocomplete = (element.getAttribute("autocomplete") || "").toLowerCase();
+ return tag === "input" && (type === "password" || autocomplete.includes("password"));
+}
+"""
+
LOCALHOST_RECOVERY_HINT = (
"Run `pip install skyvern && skyvern browser serve --tunnel` to bridge "
"your local dev server to a cloud browser via ngrok. "
@@ -138,6 +172,49 @@ async def do_extract(
return ExtractResult(extracted=extracted)
+async def do_click_at(
+ page: Any,
+ x: float,
+ y: float,
+ button: str = "left",
+ click_count: int = 1,
+) -> None:
+ raw_page = page.page if hasattr(page, "page") else page
+ await raw_page.mouse.click(x, y, button=button, click_count=click_count)
+
+
+async def do_type_at(
+ page: Any,
+ x: float,
+ y: float,
+ text: str,
+ clear_first: bool = False,
+ press_enter: bool = False,
+) -> None:
+ raw_page = page.page if hasattr(page, "page") else page
+ await raw_page.mouse.click(x, y)
+ try:
+ is_password = await raw_page.evaluate(_COORDINATE_PASSWORD_TARGET_JS)
+ except Exception as exc:
+ raise GuardError(
+ COORDINATE_TYPE_TARGET_REFUSAL_MESSAGE,
+ "Use a selector or intent to target a verified non-password input",
+ ) from exc
+ if is_password is True:
+ raise GuardError(TYPE_PASSWORD_REFUSAL_MESSAGE, CREDENTIAL_HINT)
+ if is_password is not False:
+ raise GuardError(
+ COORDINATE_TYPE_TARGET_REFUSAL_MESSAGE,
+ "Use a selector or intent to target a verified non-password input",
+ )
+ if clear_first:
+ await raw_page.keyboard.press("ControlOrMeta+A")
+ await raw_page.keyboard.press("Backspace")
+ await raw_page.keyboard.type(text)
+ if press_enter:
+ await raw_page.keyboard.press("Enter")
+
+
# -- Semantic locators --
diff --git a/skyvern/cli/core/session_manager.py b/skyvern/cli/core/session_manager.py
index 213258e66..18ad1e9e7 100644
--- a/skyvern/cli/core/session_manager.py
+++ b/skyvern/cli/core/session_manager.py
@@ -27,6 +27,7 @@ _BODY_SEMAPHORE_LIMIT = 5 # concurrent CDP body downloads (worst case: 5 * 10s
if TYPE_CHECKING:
from playwright.async_api import Frame, Page
+ from skyvern.browser_extension.runtime import BrowserExtensionRuntime
from skyvern.library.skyvern_browser import SkyvernBrowser
from skyvern.library.skyvern_browser_page import SkyvernBrowserPage
@@ -150,6 +151,8 @@ def _session_ref_key(
if resolved_session_id:
return (organization_id, "cloud_session", resolved_session_id, api_key_hash)
+ if context is not None and context.mode == "extension":
+ return (organization_id, "extension", "own-browser", api_key_hash)
if resolved_cdp_url:
return (organization_id, "cdp", resolved_cdp_url, api_key_hash)
return None
@@ -395,6 +398,7 @@ def _matches_current(
*,
session_id: str | None = None,
cdp_url: str | None = None,
+ extension_runtime: BrowserExtensionRuntime | None = None,
local: bool = False,
) -> bool:
if current.browser is None or current.context is None:
@@ -404,6 +408,8 @@ def _matches_current(
if session_id:
return current.context.mode == "cloud_session" and current.context.session_id == session_id
+ if extension_runtime is not None:
+ return current.context.mode == "extension"
if cdp_url:
return current.context.mode == "cdp" and current.context.cdp_url == cdp_url
if local:
@@ -411,6 +417,14 @@ def _matches_current(
return False
+def _extension_browser_is_connected(browser: SkyvernBrowser) -> bool:
+ try:
+ playwright_browser = browser.browser
+ return playwright_browser is not None and playwright_browser.is_connected()
+ except Exception:
+ return False
+
+
async def resolve_browser(
session_id: str | None = None,
cdp_url: str | None = None,
@@ -418,6 +432,8 @@ async def resolve_browser(
create_session: bool = False,
timeout: int | None = None,
headless: bool = False,
+ *,
+ extension_runtime: BrowserExtensionRuntime | None = None,
) -> tuple[SkyvernBrowser, BrowserContext]:
"""Resolve browser from parameters or current session.
@@ -428,13 +444,27 @@ async def resolve_browser(
skyvern = get_skyvern()
current = get_current_session()
- if _stateless_http_mode and not (session_id or cdp_url or local or create_session):
+ if _stateless_http_mode and not (session_id or cdp_url or extension_runtime or local or create_session):
raise BrowserNotAvailableError()
- if _matches_current(current, session_id=session_id, cdp_url=cdp_url, local=local):
+ if _matches_current(
+ current,
+ session_id=session_id,
+ cdp_url=cdp_url,
+ extension_runtime=extension_runtime,
+ local=local,
+ ):
if current.browser is None or current.context is None:
raise RuntimeError("Expected active browser and context for matching session")
- return current.browser, current.context
+ if extension_runtime is None or _extension_browser_is_connected(current.browser):
+ return current.browser, current.context
+ try:
+ await _close_session_state(current, close_via_active_client=False)
+ except Exception:
+ pass
+ finally:
+ set_current_session(SessionState())
+ current = get_current_session()
# Cloud sessions created by the MCP session tool intentionally do not open a
# second CDP connection. Connect lazily when a browser tool is used; this
@@ -483,6 +513,12 @@ async def resolve_browser(
set_current_session(SessionState(browser=browser, context=ctx, api_key_hash=active_api_key_hash))
return browser, ctx
+ if extension_runtime is not None:
+ browser = await skyvern.connect_to_browser_extension(extension_runtime)
+ ctx = BrowserContext(mode="extension", can_access_localhost=True)
+ set_current_session(SessionState(browser=browser, context=ctx, api_key_hash=active_api_key_hash))
+ return browser, ctx
+
if cdp_url:
browser = await skyvern.connect_to_browser_over_cdp(cdp_url)
ctx = BrowserContext(mode="cdp", cdp_url=cdp_url)
diff --git a/skyvern/cli/mcp_tools/browser.py b/skyvern/cli/mcp_tools/browser.py
index 3150bbcf9..121eee3ac 100644
--- a/skyvern/cli/mcp_tools/browser.py
+++ b/skyvern/cli/mcp_tools/browser.py
@@ -3,6 +3,7 @@ from __future__ import annotations
import asyncio
import base64
import json
+import math
import mimetypes
import os
import re
@@ -23,6 +24,7 @@ from skyvern.cli.core.browser_ops import (
_ALLOWED_EXECUTE_TOOLS,
LOCALHOST_RECOVERY_HINT,
MAX_EXECUTE_STEPS,
+ TYPE_PASSWORD_REFUSAL_MESSAGE,
CustomSelectClassifyError,
CustomSelectMatchError,
CustomSelectOpenError,
@@ -31,6 +33,7 @@ from skyvern.cli.core.browser_ops import (
ObserveFrameError,
ToolStepError,
do_act,
+ do_click_at,
do_execute,
do_extract,
do_find,
@@ -41,6 +44,7 @@ from skyvern.cli.core.browser_ops import (
do_observe,
do_screenshot,
do_select_option,
+ do_type_at,
parse_extract_schema,
ref_map_from_elements,
ref_to_selector,
@@ -207,6 +211,9 @@ def _record_trajectory_entry(
typed_text: str | None = None,
value: str | None = None,
key: str | None = None,
+ x: float | None = None,
+ y: float | None = None,
+ sdk_equivalent: str | None = None,
) -> None:
try:
if ctx.mode != "cloud_session" or not ctx.session_id:
@@ -231,6 +238,9 @@ def _record_trajectory_entry(
"source_url": projected.source_url,
"value": value,
"key": key,
+ "x": x,
+ "y": y,
+ "sdk_equivalent": sdk_equivalent,
}
if tool_name == "type_text":
entry["typed_length"] = len(typed_text or "")
@@ -284,6 +294,27 @@ def _blank_to_none(value: str | None) -> str | None:
return value if value is None or value.strip() else None
+def _coordinate_target(x: float | None, y: float | None, selector: str | None) -> str | None:
+ if (x is None) != (y is None):
+ raise GuardError(
+ "x and y must be provided together",
+ "Provide both viewport coordinates, or omit both and use selector or intent",
+ )
+ if x is None or y is None:
+ return None
+ if selector is not None:
+ raise GuardError(
+ "Coordinates x/y are mutually exclusive with selector",
+ "Use either x and y coordinates or selector, not both",
+ )
+ if isinstance(x, bool) or isinstance(y, bool) or not math.isfinite(x) or not math.isfinite(y) or x < 0 or y < 0:
+ raise GuardError(
+ "Coordinates x and y must be finite numbers greater than or equal to 0",
+ "Use non-negative viewport CSS pixel coordinates",
+ )
+ return f"coordinates ({x}, {y})"
+
+
def _add_timing_prefix(timing_ms: dict[str, int], elapsed_ms: int) -> dict[str, int]:
return {name: elapsed_ms + duration for name, duration in timing_ms.items()}
@@ -540,6 +571,20 @@ async def skyvern_click(
"Use standard CSS: 'button.class', 'a[href*=\"pdf\"]', '#id', ':nth-of-type()'."
),
] = None,
+ x: Annotated[
+ float | None,
+ Field(
+ description=f"{DIRECT_TARGET_DESCRIPTION} Viewport x-coordinate in CSS pixels from the left edge of "
+ "the web content. Provide with y and without selector."
+ ),
+ ] = None,
+ y: Annotated[
+ float | None,
+ Field(
+ description=f"{DIRECT_TARGET_DESCRIPTION} Viewport y-coordinate in CSS pixels from the top edge of "
+ "the web content. Provide with x and without selector."
+ ),
+ ] = None,
selector_mode: _SelectorMode = "resilient",
session_id: Annotated[str | None, Field(description="Browser session ID (pbs_...)")] = None,
cdp_url: Annotated[str | None, Field(description="CDP WebSocket URL")] = None,
@@ -551,7 +596,7 @@ async def skyvern_click(
click_count: Annotated[int | None, Field(description="Number of clicks (2 for double-click)")] = None,
intent: Annotated[str | None, Field(description=AI_FALLBACK_DESCRIPTION)] = None,
) -> dict[str, Any]:
- """Click an element using AI intent, CSS/XPath selector, or both.
+ """Click using viewport coordinates, AI intent, CSS/XPath selector, or a selector with intent fallback.
For text input use skyvern_type. For dropdowns use skyvern_select_option. For multiple actions prefer skyvern_act.
"""
if button is not None and button not in ("left", "right", "middle"):
@@ -563,17 +608,28 @@ async def skyvern_click(
selector = _blank_to_none(selector)
intent = _blank_to_none(intent)
- ai_mode, err = _resolve_ai_mode(selector, intent)
- if err:
+ try:
+ coordinate_target = _coordinate_target(x, y, selector)
+ except GuardError as e:
return make_result(
"skyvern_click",
ok=False,
- error=make_error(
- ErrorCode.INVALID_INPUT,
- "Must provide intent, selector, or both",
- "Use intent='describe what to click' for AI-powered clicking, or selector='#css-selector' for precise targeting",
- ),
+ error=make_error(ErrorCode.INVALID_INPUT, str(e), e.hint),
)
+ if coordinate_target is None:
+ ai_mode, err = _resolve_ai_mode(selector, intent)
+ if err:
+ return make_result(
+ "skyvern_click",
+ ok=False,
+ error=make_error(
+ ErrorCode.INVALID_INPUT,
+ "Must provide intent, selector, or both",
+ "Use intent='describe what to click' for AI-powered clicking, or selector='#css-selector' for precise targeting",
+ ),
+ )
+ else:
+ ai_mode = None
try:
page, ctx = await get_page(session_id=session_id, cdp_url=cdp_url)
@@ -598,25 +654,51 @@ async def skyvern_click(
if click_count is not None:
kwargs["click_count"] = click_count
- if selector is not None and (deterministic or ai_mode is None or ai_mode == "fallback"):
- native_option_selection = await select_native_option_if_targeted(page, selector, timeout=action_timeout)
-
- if native_option_selection is not None:
- resolved = native_option_selection.select_selector
- elif deterministic:
- # selector_mode="direct": pin the selector, no overlay-dismiss or AI re-target, so a
- # missed target fails fast and the agent re-derives it instead of AI scout-scrolling.
- resolved = await page.click(selector=selector, mode="direct", **kwargs)
- elif ai_mode is not None:
- used_ai_path = True
- resolved = await page.click(selector=selector, prompt=intent, ai=ai_mode, **kwargs) # type: ignore[arg-type]
+ if coordinate_target is not None:
+ assert x is not None and y is not None
+ await do_click_at(
+ page,
+ x,
+ y,
+ button=button or "left",
+ click_count=click_count if click_count is not None else 1,
+ )
+ resolved = coordinate_target
else:
- assert selector is not None
- if isinstance(page, SkyvernPage):
- kwargs["_skip_element_prep"] = skip_element_prep
- resolved = await page.click(selector=selector, **kwargs)
+ if selector is not None and (deterministic or ai_mode is None or ai_mode == "fallback"):
+ native_option_selection = await select_native_option_if_targeted(
+ page,
+ selector,
+ timeout=action_timeout,
+ )
+ if native_option_selection is not None:
+ resolved = native_option_selection.select_selector
+ elif deterministic:
+ # selector_mode="direct": pin the selector, no overlay-dismiss or AI re-target, so a
+ # missed target fails fast and the agent re-derives it instead of AI scout-scrolling.
+ resolved = await page.click(selector=selector, mode="direct", **kwargs)
+ elif ai_mode is not None:
+ used_ai_path = True
+ resolved = await page.click(selector=selector, prompt=intent, ai=ai_mode, **kwargs) # type: ignore[arg-type]
+ else:
+ assert selector is not None
+ if isinstance(page, SkyvernPage):
+ kwargs["_skip_element_prep"] = skip_element_prep
+ resolved = await page.click(selector=selector, **kwargs)
timer.mark("sdk")
except PlaywrightTimeoutError as e:
+ if coordinate_target is not None:
+ return action_result(
+ "skyvern_click",
+ ok=False,
+ browser_context=ctx,
+ timing_ms=timer.timing_ms,
+ error=make_error(
+ ErrorCode.ACTION_FAILED,
+ str(e),
+ "Check that the coordinates are within the current viewport",
+ ),
+ )
if direct_action and selector is not None:
return await _direct_failure_result("skyvern_click", ctx, timer, page, selector, e, action_timeout)
return action_result(
@@ -647,7 +729,13 @@ async def skyvern_click(
),
)
- data: dict[str, Any] = {"selector": selector, "intent": intent, "ai_mode": ai_mode}
+ data: dict[str, Any] = {
+ "selector": selector,
+ "intent": intent,
+ "ai_mode": ai_mode,
+ }
+ if coordinate_target is not None:
+ data.update({"x": x, "y": y, "resolved_target": coordinate_target})
if native_option_selection is not None:
data["selected_option"] = {
"select_selector": native_option_selection.select_selector,
@@ -659,13 +747,20 @@ async def skyvern_click(
data["selected_option"]["value"] = native_option_selection.value
if native_option_selection.label is not None:
data["selected_option"]["label"] = native_option_selection.label
- if resolved and resolved != selector:
+ if coordinate_target is None and resolved and resolved != selector:
data["resolved_selector"] = resolved
# Build sdk_equivalent: prefer hybrid selector+prompt for production scripts.
# resolved_selector already contains the "xpath=" prefix (e.g. "xpath=//button[@id='x']"),
# so pass it directly as the selector positional arg.
resolved_sel = resolved if resolved and resolved != selector else selector
- if native_option_selection is not None:
+ coordinate_sdk_equivalent: str | None = None
+ if coordinate_target is not None:
+ coordinate_sdk_equivalent = (
+ f"await page.mouse.click({x!r}, {y!r}, button={(button or 'left')!r}, "
+ f"click_count={click_count if click_count is not None else 1})"
+ )
+ data["sdk_equivalent"] = coordinate_sdk_equivalent
+ elif native_option_selection is not None:
if native_option_selection.selected_by == "label":
data["sdk_equivalent"] = (
f"await page.select_option({native_option_selection.select_selector!r}, "
@@ -699,14 +794,25 @@ async def skyvern_click(
value=native_option_selection.value,
)
elif button in (None, "left") and click_count in (None, 1):
- replayable_selector = resolved if used_ai_path else resolved or selector
- if replayable_selector:
+ if coordinate_target is not None:
_record_trajectory_entry(
ctx,
tool_name="click",
- selector=replayable_selector,
+ selector=None,
source_url=source_url,
+ x=x,
+ y=y,
+ sdk_equivalent=coordinate_sdk_equivalent,
)
+ else:
+ replayable_selector = resolved if used_ai_path else resolved or selector
+ if replayable_selector:
+ _record_trajectory_entry(
+ ctx,
+ tool_name="click",
+ selector=replayable_selector,
+ source_url=source_url,
+ )
return action_result(
"skyvern_click",
browser_context=ctx,
@@ -1117,6 +1223,20 @@ async def skyvern_type(
selector: Annotated[
str | None, Field(description=f"{DIRECT_TARGET_DESCRIPTION} CSS selector or XPath for the input element.")
] = None,
+ x: Annotated[
+ float | None,
+ Field(
+ description=f"{DIRECT_TARGET_DESCRIPTION} Viewport x-coordinate in CSS pixels from the left edge of "
+ "the web content. Provide with y and without selector."
+ ),
+ ] = None,
+ y: Annotated[
+ float | None,
+ Field(
+ description=f"{DIRECT_TARGET_DESCRIPTION} Viewport y-coordinate in CSS pixels from the top edge of "
+ "the web content. Provide with x and without selector."
+ ),
+ ] = None,
selector_mode: _SelectorMode = "resilient",
session_id: Annotated[str | None, Field(description="Browser session ID (pbs_...)")] = None,
cdp_url: Annotated[str | None, Field(description="CDP WebSocket URL")] = None,
@@ -1127,43 +1247,65 @@ async def skyvern_type(
clear: Annotated[bool, Field(description="Clear existing content before typing")] = True,
delay: Annotated[int | None, Field(description="Delay between keystrokes in ms")] = None,
intent: Annotated[str | None, Field(description=AI_FALLBACK_DESCRIPTION)] = None,
+ clear_first: Annotated[
+ bool | None,
+ Field(description="Clear existing content before coordinate typing; defaults to clear"),
+ ] = None,
+ press_enter: Annotated[bool, Field(description="Press Enter after typing")] = False,
) -> dict[str, Any]:
- """Type text into an input field using AI intent, CSS/XPath selector, or both. Clears field by default (set clear=false to append).
+ """Type using viewport coordinates, AI intent, CSS/XPath selector, or a selector with intent fallback.
+ Clears field by default (set clear=false to append).
NEVER use for passwords — use skyvern_login instead. For dropdowns use skyvern_select_option.
"""
- # Block password entry — redirect to skyvern_login
- target_text = f"{intent or ''} {selector or ''}"
- if PASSWORD_PATTERN.search(target_text):
+ selector = _blank_to_none(selector)
+ intent = _blank_to_none(intent)
+ try:
+ coordinate_target = _coordinate_target(x, y, selector)
+ except GuardError as e:
return make_result(
"skyvern_type",
ok=False,
- error=make_error(
- ErrorCode.INVALID_INPUT,
- "Cannot type into password fields — credentials must not be passed through tool calls",
- CREDENTIAL_HINT,
- ),
+ error=make_error(ErrorCode.INVALID_INPUT, str(e), e.hint),
)
- selector = _blank_to_none(selector)
- intent = _blank_to_none(intent)
- ai_mode, err = _resolve_ai_mode(selector, intent)
- if err:
+ target_text = f"{intent or ''} {selector or ''}"
+ if PASSWORD_PATTERN.search(target_text):
return make_result(
"skyvern_type",
ok=False,
error=make_error(
ErrorCode.INVALID_INPUT,
- "Must provide intent, selector, or both",
- "Use intent='describe the input field' for AI-powered targeting, or selector='#css-selector' for precise targeting",
+ TYPE_PASSWORD_REFUSAL_MESSAGE,
+ CREDENTIAL_HINT,
),
)
+ if coordinate_target is None:
+ ai_mode, err = _resolve_ai_mode(selector, intent)
+ if err:
+ return make_result(
+ "skyvern_type",
+ ok=False,
+ error=make_error(
+ ErrorCode.INVALID_INPUT,
+ "Must provide intent, selector, or both",
+ "Use intent='describe the input field' for AI-powered targeting, or selector='#css-selector' for precise targeting",
+ ),
+ )
+ else:
+ ai_mode = None
+
try:
page, ctx = await get_page(session_id=session_id, cdp_url=cdp_url)
except BrowserNotAvailableError:
return make_result("skyvern_type", ok=False, error=no_browser_error())
- action_result = _action_result_factory(ctx=ctx, page=page, selector=selector, typed_text=text)
+ action_result = _action_result_factory(
+ ctx=ctx,
+ page=page,
+ selector=selector,
+ typed_text=text,
+ )
source_url = _trajectory_source_url(page)
# DOM-level guard: check if the target element is a password field
@@ -1184,11 +1326,12 @@ async def skyvern_type(
ok=False,
error=make_error(
ErrorCode.INVALID_INPUT,
- "Cannot type into password fields — credentials must not be passed through tool calls",
+ TYPE_PASSWORD_REFUSAL_MESSAGE,
CREDENTIAL_HINT,
),
)
+ clear_content = clear if clear_first is None else clear_first
deterministic = selector is not None and selector_mode == "direct"
direct_action = is_direct_action(selector, ai_mode, deterministic=deterministic)
action_timeout = resolve_action_timeout_ms(timeout, direct_action=direct_action)
@@ -1199,7 +1342,17 @@ async def skyvern_type(
# selector_mode="direct" pins the selector with no AI fall-back. Resilient (default) and
# intent-only calls keep AI; emitted scripts keep the selector+prompt fallback via
# sdk_equivalent for DOM-drift resilience.
- if clear:
+ if coordinate_target is not None:
+ assert x is not None and y is not None
+ await do_type_at(
+ page,
+ x,
+ y,
+ text,
+ clear_first=clear_content,
+ press_enter=press_enter,
+ )
+ elif clear_content:
if deterministic:
assert selector is not None
await page.fill(selector, text, mode="direct", timeout=action_timeout)
@@ -1225,8 +1378,31 @@ async def skyvern_type(
if isinstance(page, SkyvernPage):
kwargs["_skip_element_prep"] = skip_element_prep
await page.type(selector, text, **kwargs)
+ if coordinate_target is None and press_enter:
+ raw_page = page.page if hasattr(page, "page") else page
+ await raw_page.keyboard.press("Enter")
timer.mark("sdk")
+ except GuardError as e:
+ return action_result(
+ "skyvern_type",
+ ok=False,
+ browser_context=ctx,
+ timing_ms=timer.timing_ms,
+ error=make_error(ErrorCode.INVALID_INPUT, str(e), e.hint),
+ )
except PlaywrightTimeoutError as e:
+ if coordinate_target is not None:
+ return action_result(
+ "skyvern_type",
+ ok=False,
+ browser_context=ctx,
+ timing_ms=timer.timing_ms,
+ error=make_error(
+ ErrorCode.ACTION_FAILED,
+ str(e),
+ "Check that the coordinates are within the current viewport",
+ ),
+ )
if direct_action and selector is not None:
return await _direct_failure_result(
"skyvern_type", ctx, timer, page, selector, e, action_timeout, typed_text=text
@@ -1265,21 +1441,48 @@ async def skyvern_type(
# Unlike click(), we cannot return resolved_selector here. SKY-7905 will
# update the SDK to return element metadata from all action methods.
data: dict[str, Any] = {"selector": selector, "intent": intent, "ai_mode": ai_mode, "text_length": len(text)}
+ if coordinate_target is not None:
+ data.update({"x": x, "y": y, "resolved_target": coordinate_target})
# Build sdk_equivalent: prefer hybrid selector+prompt for production scripts
- if selector and intent:
+ coordinate_sdk_equivalent: str | None = None
+ if coordinate_target is not None:
+ clear_snippet = (
+ "await page.keyboard.press('ControlOrMeta+A'); await page.keyboard.press('Backspace'); "
+ if clear_content
+ else ""
+ )
+ enter_snippet = "; await page.keyboard.press('Enter')" if press_enter else ""
+ coordinate_sdk_equivalent = (
+ f"await page.mouse.click({x!r}, {y!r}); {clear_snippet}await page.keyboard.type({text!r}){enter_snippet}"
+ )
+ data["sdk_equivalent"] = coordinate_sdk_equivalent
+ elif selector and intent:
data["sdk_equivalent"] = f"await page.fill({selector!r}, {text!r}, prompt={intent!r})"
elif ai_mode:
data["sdk_equivalent"] = f"await page.fill(prompt={intent!r}, value={text!r})"
elif selector:
data["sdk_equivalent"] = f"await page.fill({selector!r}, {text!r})"
- if clear and selector is not None and (ai_mode is None or deterministic):
- _record_trajectory_entry(
- ctx,
- tool_name="type_text",
- selector=selector,
- source_url=source_url,
- typed_text=text,
- )
+ replayable_type = coordinate_target is not None or ai_mode is None or deterministic
+ if clear_content and replayable_type:
+ if coordinate_target is not None:
+ _record_trajectory_entry(
+ ctx,
+ tool_name="type_text",
+ selector=None,
+ source_url=source_url,
+ typed_text=text,
+ x=x,
+ y=y,
+ sdk_equivalent=coordinate_sdk_equivalent,
+ )
+ elif selector is not None:
+ _record_trajectory_entry(
+ ctx,
+ tool_name="type_text",
+ selector=selector,
+ source_url=source_url,
+ typed_text=text,
+ )
return action_result(
"skyvern_type",
browser_context=ctx,
@@ -3323,8 +3526,8 @@ _TOOL_NAME_MAP: dict[str, str] = {
# Accepted user-facing params for each dispatched tool (excludes session_id/cdp_url).
_TOOL_ACCEPTED_PARAMS: dict[str, frozenset[str]] = {
"navigate": frozenset({"url", "timeout", "wait_until"}),
- "click": frozenset({"intent", "selector", "timeout", "click_count", "button"}),
- "type": frozenset({"text", "intent", "selector", "clear_first", "press_enter", "timeout"}),
+ "click": frozenset({"intent", "selector", "x", "y", "timeout", "click_count", "button"}),
+ "type": frozenset({"text", "intent", "selector", "x", "y", "clear_first", "press_enter", "timeout"}),
"press_key": frozenset({"key", "intent", "selector", "timeout"}),
"select_option": frozenset({"value", "intent", "selector", "timeout", "by_label"}),
"hover": frozenset({"intent", "selector", "timeout"}),
diff --git a/skyvern/cli/mcp_tools/session.py b/skyvern/cli/mcp_tools/session.py
index 087c3766c..69ec41af0 100644
--- a/skyvern/cli/mcp_tools/session.py
+++ b/skyvern/cli/mcp_tools/session.py
@@ -5,6 +5,7 @@ from typing import Annotated, Any
from pydantic import Field
+from skyvern.browser_extension.runtime import BrowserExtensionRuntime
from skyvern.cli.core.action_log import drain_action_log_events
from skyvern.cli.core.api_key_hash import hash_api_key_for_cache
from skyvern.cli.core.client import get_active_api_key
@@ -32,6 +33,12 @@ from ._session import (
set_current_session,
)
+_EXTENSION_NOT_CONNECTED_GUIDANCE = (
+ "Skyvern browser extension is not connected. Install it: run `skyvern browser extension-path`, open "
+ "chrome://extensions, enable Developer mode, Load unpacked, and select that directory. Then run "
+ "`skyvern browser extension-token`, paste the token into the extension popup, and click Connect. Then retry."
+)
+
def _session_api_key_hash() -> str | None:
api_key = get_active_api_key()
@@ -49,6 +56,10 @@ def _should_default_to_cdp() -> tuple[bool, str | None]:
return False, None
+def _should_default_to_extension() -> bool:
+ return os.environ.get("BROWSER_TYPE", "") == "extension-connect"
+
+
def _session_create_data(
session_id: str | None,
timeout_minutes: int | None,
@@ -114,10 +125,74 @@ async def skyvern_browser_session_create(
Use local=true for a local Chromium instance.
The session persists across tool calls until explicitly closed.
+ When the server runs in browser-extension mode, the session is implicit: no session_id is
+ returned and browser tools are called without one.
"""
- # When BROWSER_TYPE=cdp-connect, auto-connect to the user's local browser via CDP.
- # resolve_browser() stores the browser in session state via set_current_session()
- # internally, so we don't need to call it again here.
+ if _should_default_to_extension() and not local:
+ with Timer() as timer:
+ if is_stateless_http_mode():
+ return make_result(
+ "skyvern_browser_session_create",
+ ok=False,
+ timing_ms=timer.timing_ms,
+ error=make_error(
+ ErrorCode.SDK_ERROR,
+ "The Skyvern browser extension requires the MCP server to run on the stdio transport. "
+ "Restart with: skyvern mcp --browser-extension",
+ "",
+ ),
+ )
+ try:
+ runtime = await BrowserExtensionRuntime.get_or_start()
+ if not await runtime.wait_for_extension(10.0):
+ timer.mark("sdk")
+ return make_result(
+ "skyvern_browser_session_create",
+ ok=False,
+ timing_ms=timer.timing_ms,
+ error=make_error(
+ ErrorCode.BROWSER_NOT_FOUND,
+ _EXTENSION_NOT_CONNECTED_GUIDANCE,
+ "",
+ ),
+ )
+ except Exception as e:
+ return make_result(
+ "skyvern_browser_session_create",
+ ok=False,
+ timing_ms=timer.timing_ms,
+ error=make_error(
+ ErrorCode.SDK_ERROR,
+ str(e),
+ "Failed to start or connect to the Skyvern browser extension",
+ ),
+ )
+
+ try:
+ _browser, ctx = await resolve_browser(extension_runtime=runtime)
+ timer.mark("sdk")
+ except Exception:
+ return make_result(
+ "skyvern_browser_session_create",
+ ok=False,
+ timing_ms=timer.timing_ms,
+ error=make_error(
+ ErrorCode.BROWSER_NOT_FOUND,
+ _EXTENSION_NOT_CONNECTED_GUIDANCE,
+ "",
+ ),
+ )
+ return make_result(
+ "skyvern_browser_session_create",
+ browser_context=ctx,
+ data={
+ "browser": "extension",
+ "session": "implicit",
+ "note": "Extension sessions are implicit; call browser tools without a session_id.",
+ },
+ timing_ms=timer.timing_ms,
+ )
+
use_cdp, cdp_url = _should_default_to_cdp()
if use_cdp and not local and cdp_url:
if browser_profile_id is not None or generate_browser_profile:
diff --git a/skyvern/cli/run_commands.py b/skyvern/cli/run_commands.py
index 4a601dd92..a331b2bad 100644
--- a/skyvern/cli/run_commands.py
+++ b/skyvern/cli/run_commands.py
@@ -227,16 +227,29 @@ def _handle_mcp_shutdown_signal(_signum: int, _frame: Any) -> None:
_mcp_main_task.cancel()
-async def _run_mcp_with_cleanup(run_async: Any, **kwargs: Any) -> None:
+async def _run_mcp_with_cleanup(run_async: Any, *, browser_extension: bool = False, **kwargs: Any) -> None:
global _mcp_cleanup_done, _mcp_cleanup_in_progress, _mcp_main_task
current_task = asyncio.current_task()
_mcp_main_task = current_task
+ extension_runtime: BrowserExtensionRuntime | None = None
try:
+ if browser_extension:
+ from skyvern.browser_extension.runtime import BrowserExtensionRuntime # noqa: PLC0415
+
+ extension_runtime = await BrowserExtensionRuntime.get_or_start()
await run_async(**kwargs)
finally:
_mcp_cleanup_in_progress = True
try:
- await _cleanup_mcp_resources()
+ try:
+ await _cleanup_mcp_resources()
+ finally:
+ if extension_runtime is None:
+ from skyvern.browser_extension.runtime import BrowserExtensionRuntime # noqa: PLC0415
+
+ extension_runtime = BrowserExtensionRuntime.instance()
+ if extension_runtime is not None:
+ await extension_runtime.shutdown()
finally:
_mcp_cleanup_done = True
_mcp_cleanup_in_progress = False
@@ -607,11 +620,22 @@ def run_mcp(
help="Return full tool responses including sdk_equivalent, browser_context, and timing.",
),
] = False,
+ browser_extension: Annotated[
+ bool,
+ typer.Option(
+ "--browser-extension/--no-browser-extension",
+ help="Start the relay for controlling Chrome through the Skyvern browser extension.",
+ ),
+ ] = False,
) -> None:
"""Run the MCP server with configurable transport for local or remote hosting."""
global _mcp_eof_shutdown_requested, _mcp_shutdown_exit_code
_mcp_eof_shutdown_requested = False
_mcp_shutdown_exit_code = None
+ if browser_extension and transport != "stdio":
+ raise typer.BadParameter("--browser-extension requires --transport stdio", param_hint="--browser-extension")
+ if browser_extension:
+ os.environ.setdefault("BROWSER_TYPE", "extension-connect")
prepare_cli_runtime(intent=EnvIntent.CLOUD)
try:
from skyvern.library.local_browser_profile import ( # noqa: PLC0415
@@ -646,7 +670,13 @@ def run_mcp(
original_signal_handlers[signal.SIGTERM] = signal.signal(signal.SIGTERM, _handle_mcp_shutdown_signal)
eof_watcher_stop, shutdown_complete = _start_stdin_eof_watcher()
try:
- asyncio.run(_run_mcp_with_cleanup(mcp.run_async, transport="stdio"))
+ asyncio.run(
+ _run_mcp_with_cleanup(
+ mcp.run_async,
+ browser_extension=browser_extension,
+ transport="stdio",
+ )
+ )
except asyncio.CancelledError:
if _mcp_shutdown_exit_code is None:
raise
@@ -660,6 +690,7 @@ def run_mcp(
asyncio.run(
_run_mcp_with_cleanup(
mcp.run_async,
+ browser_extension=browser_extension,
transport=transport,
host=host,
port=port,
diff --git a/skyvern/library/skyvern.py b/skyvern/library/skyvern.py
index aa7a0fb1e..c3a6e8748 100644
--- a/skyvern/library/skyvern.py
+++ b/skyvern/library/skyvern.py
@@ -26,6 +26,7 @@ _DEVTOOLS_ACTIVE_PORT_TIMEOUT_SECONDS = 2.0
if TYPE_CHECKING:
from playwright.async_api import Playwright
+ from skyvern.browser_extension.runtime import BrowserExtensionRuntime
from skyvern.library.skyvern_browser import SkyvernBrowser
from skyvern.schemas.llm import LLMConfig, LLMRouterConfig
@@ -589,6 +590,14 @@ class Skyvern(AsyncSkyvern):
browser_context = browser.contexts[0] if browser.contexts else await browser.new_context()
return SkyvernBrowser(self, browser_context, browser_address=cdp_url)
+ async def connect_to_browser_extension(self, runtime: BrowserExtensionRuntime) -> SkyvernBrowser:
+ from skyvern.library.skyvern_browser import SkyvernBrowser # noqa: PLC0415
+
+ playwright = await self._get_playwright()
+ browser = await playwright.chromium.connect_over_cdp(runtime.cdp_ws_url)
+ browser_context = browser.contexts[0] if browser.contexts else await browser.new_context()
+ return SkyvernBrowser(self, browser_context)
+
async def connect_to_cloud_browser_session(self, browser_session_id: str) -> SkyvernBrowser:
"""Connect to an existing cloud-hosted browser session by ID.
diff --git a/tests/smoke_tests/browser_extension/__init__.py b/tests/smoke_tests/browser_extension/__init__.py
new file mode 100644
index 000000000..e69de29bb
diff --git a/tests/smoke_tests/browser_extension/run_extension_smoke.py b/tests/smoke_tests/browser_extension/run_extension_smoke.py
new file mode 100644
index 000000000..b5d7940a1
--- /dev/null
+++ b/tests/smoke_tests/browser_extension/run_extension_smoke.py
@@ -0,0 +1,508 @@
+from __future__ import annotations
+
+import argparse
+import asyncio
+import base64
+import hashlib
+import inspect
+import json
+import os
+import re
+import secrets
+import socket
+import tempfile
+from collections.abc import Awaitable, Callable, Sequence
+from dataclasses import dataclass
+from pathlib import Path
+from urllib.parse import urlsplit
+
+from aiohttp import web
+from playwright.async_api import Browser, BrowserContext, Page, Playwright, async_playwright
+
+from skyvern.browser_extension.protocol import EXTENSION_ID
+from skyvern.browser_extension.runtime import BrowserExtensionRuntime
+
+EXPECTED_EXTENSION_ID = EXTENSION_ID
+TOKEN_ENV = "SKYVERN_BROWSER_EXTENSION_TOKEN"
+BROWSER_TYPE_ENV = "BROWSER_TYPE"
+MCP_BROWSER_TYPE = "extension-connect"
+SMOKE_TEXT = "Skyvern extension bridge smoke"
+CHECKS = (
+ "setup",
+ "extension paired",
+ "navigate",
+ "screenshot",
+ "trusted selector click",
+ "coordinate click",
+ "type",
+ "iframe evaluate",
+ "new tab",
+ "close tab",
+ "chrome survives disconnect",
+ "mcp session/navigate/observe",
+)
+CDP_URL_PATTERN = re.compile(r"ws://127\.0\.0\.1:\d+/cdp/[A-Za-z0-9_-]+")
+
+
+class McpUnavailableError(RuntimeError):
+ pass
+
+
+class SmokeReport:
+ def __init__(self) -> None:
+ self.passed = 0
+ self.failed = 0
+ self.skipped = 0
+ self.recorded: set[str] = set()
+
+ def pass_check(self, label: str, details: str | None = None) -> None:
+ self.passed += 1
+ self.recorded.add(label)
+ print(_status_line("PASS", label, details))
+
+ def fail_check(self, label: str, details: str | None = None) -> None:
+ self.failed += 1
+ self.recorded.add(label)
+ print(_status_line("FAIL", label, details))
+
+ def skip_check(self, label: str, details: str | None = None) -> None:
+ self.skipped += 1
+ self.recorded.add(label)
+ print(_status_line("SKIP", label, details))
+
+ def skip_unrecorded(self, reason: str) -> None:
+ for label in CHECKS:
+ if label not in self.recorded:
+ self.skip_check(label, reason)
+
+ def print_summary(self) -> None:
+ print()
+ print(f"SMOKE RESULT: {self.passed} passed, {self.failed} failed, {self.skipped} skipped")
+
+
+@dataclass
+class FixtureServers:
+ outer_runner: web.AppRunner
+ iframe_runner: web.AppRunner
+ outer_url: str
+
+ async def close(self) -> None:
+ try:
+ await self.outer_runner.cleanup()
+ finally:
+ await self.iframe_runner.cleanup()
+
+
+def _status_line(status: str, label: str, details: str | None) -> str:
+ suffix = f": {details}" if details else ""
+ return f"{status} {label}{suffix}"
+
+
+def _reserve_free_port() -> tuple[socket.socket, int]:
+ listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ try:
+ listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+ listener.bind(("127.0.0.1", 0))
+ listener.listen(1)
+ except BaseException:
+ listener.close()
+ raise
+ return listener, int(listener.getsockname()[1])
+
+
+def _extension_id_from_key(manifest_key: str) -> str:
+ public_key = base64.b64decode(manifest_key, validate=True)
+ digest_prefix = hashlib.sha256(public_key).hexdigest()[:32]
+ return "".join(chr(ord("a") + int(nibble, 16)) for nibble in digest_prefix)
+
+
+def _extension_id_from_manifest(extension_dir: Path) -> str:
+ manifest_value = json.loads((extension_dir / "manifest.json").read_text())
+ if not isinstance(manifest_value, dict):
+ raise ValueError("extension manifest must contain a JSON object")
+ manifest_key = manifest_value.get("key")
+ if not isinstance(manifest_key, str) or not manifest_key:
+ raise ValueError("extension manifest is missing its key")
+ return _extension_id_from_key(manifest_key)
+
+
+def _safe_text(value: object, redactions: Sequence[str]) -> str:
+ text = CDP_URL_PATTERN.sub("<redacted-cdp-url>", str(value))
+ for secret_value in redactions:
+ if secret_value:
+ text = text.replace(secret_value, "<redacted>")
+ return " ".join(text.split())[:600]
+
+
+async def _discover_extension_id(context: BrowserContext, timeout_seconds: float = 15.0) -> str:
+ loop = asyncio.get_running_loop()
+ deadline = loop.time() + timeout_seconds
+ while loop.time() < deadline:
+ parsed_urls = (urlsplit(worker.url) for worker in context.service_workers)
+ extension_ids = {
+ parsed.netloc for parsed in parsed_urls if parsed.scheme == "chrome-extension" and parsed.netloc
+ }
+ if EXPECTED_EXTENSION_ID in extension_ids:
+ return EXPECTED_EXTENSION_ID
+ if extension_ids:
+ return sorted(extension_ids)[0]
+ await asyncio.sleep(0.1)
+ raise TimeoutError("no chrome-extension service worker appeared within 15 seconds")
+
+
+async def _wait_for_pairing(runtime: BrowserExtensionRuntime, timeout_seconds: float = 15.0) -> bool:
+ loop = asyncio.get_running_loop()
+ deadline = loop.time() + timeout_seconds
+ while loop.time() < deadline:
+ if runtime.extension_connected:
+ return True
+ await asyncio.sleep(0.1)
+ return runtime.extension_connected
+
+
+async def _start_runner(
+ handler: Callable[[web.Request], Awaitable[web.StreamResponse]],
+) -> tuple[web.AppRunner, int]:
+ app = web.Application()
+ app.router.add_get("/", handler)
+ runner = web.AppRunner(app)
+ await runner.setup()
+ site = web.TCPSite(runner, "127.0.0.1", 0)
+ try:
+ await site.start()
+ except BaseException:
+ await runner.cleanup()
+ raise
+ addresses = runner.addresses
+ if not addresses:
+ await runner.cleanup()
+ raise RuntimeError("fixture server failed to bind")
+ return runner, int(addresses[0][1])
+
+
+async def _start_fixture_servers() -> FixtureServers:
+ async def iframe_handler(_request: web.Request) -> web.Response:
+ return web.Response(
+ text='<!doctype html><html><body><div id="frame-marker">iframe-ready</div></body></html>',
+ content_type="text/html",
+ headers={"Cache-Control": "no-store"},
+ )
+
+ iframe_runner, iframe_port = await _start_runner(iframe_handler)
+ iframe_url = f"http://127.0.0.1:{iframe_port}/"
+
+ async def outer_handler(_request: web.Request) -> web.Response:
+ html = f"""<!doctype html><html><head><meta charset="utf-8">
+<title>Extension Bridge Fixture</title></head><body>
+<button id="smoke-button" type="button">Smoke button</button>
+<input id="text-input" type="text"><input id="password-input" type="password">
+<iframe id="smoke-frame" src="{iframe_url}"></iframe><script>
+window.smokeState = {{clickCount: 0, lastTrusted: false}};
+document.querySelector("#smoke-button").addEventListener("click", (event) => {{
+ window.smokeState.clickCount += 1; window.smokeState.lastTrusted = event.isTrusted;
+}});</script></body></html>"""
+ return web.Response(text=html, content_type="text/html", headers={"Cache-Control": "no-store"})
+
+ try:
+ outer_runner, outer_port = await _start_runner(outer_handler)
+ except BaseException:
+ await iframe_runner.cleanup()
+ raise
+ return FixtureServers(
+ outer_runner=outer_runner,
+ iframe_runner=iframe_runner,
+ outer_url=f"http://localhost:{outer_port}/",
+ )
+
+
+def _tool_error(result: dict[str, object]) -> str:
+ error = result.get("error")
+ if not isinstance(error, dict):
+ return "tool returned ok=false"
+ code = error.get("code")
+ message = error.get("message")
+ return f"{code}: {message}"
+
+
+async def _run_mcp_phase(fixture_url: str) -> str:
+ from skyvern.cli.mcp_tools import browser as browser_tools
+ from skyvern.cli.mcp_tools import session as session_tools
+
+ if MCP_BROWSER_TYPE not in inspect.getsource(session_tools):
+ raise McpUnavailableError("the current MCP session layer has no extension-connect branch")
+
+ previous_browser_type = os.environ.get(BROWSER_TYPE_ENV)
+ os.environ[BROWSER_TYPE_ENV] = MCP_BROWSER_TYPE
+ session_created = False
+ try:
+ session_result = await session_tools.skyvern_browser_session_create()
+ if session_result.get("ok") is not True:
+ raise McpUnavailableError(f"session=FAIL; unavailable in-process ({_tool_error(session_result)})")
+ session_created = True
+
+ navigate_result = await browser_tools.skyvern_navigate(url=fixture_url)
+ observe_result = await browser_tools.skyvern_observe(selector="body", interactive_only=True)
+ statuses = (
+ f"session={'PASS' if session_result.get('ok') is True else 'FAIL'} "
+ f"navigate={'PASS' if navigate_result.get('ok') is True else 'FAIL'} "
+ f"observe={'PASS' if observe_result.get('ok') is True else 'FAIL'}"
+ )
+ if navigate_result.get("ok") is not True:
+ raise AssertionError(f"{statuses}; navigate error={_tool_error(navigate_result)}")
+ if observe_result.get("ok") is not True:
+ raise AssertionError(f"{statuses}; observe error={_tool_error(observe_result)}")
+ return statuses
+ finally:
+ if session_created:
+ try:
+ await session_tools.skyvern_browser_session_close()
+ except Exception:
+ pass
+ if previous_browser_type is None:
+ os.environ.pop(BROWSER_TYPE_ENV, None)
+ else:
+ os.environ[BROWSER_TYPE_ENV] = previous_browser_type
+
+
+async def _pair_extension(
+ context: BrowserContext,
+ runtime: BrowserExtensionRuntime,
+ extension_id: str,
+ relay_port: int,
+ pairing_token: str,
+) -> Page:
+ popup_page = await context.new_page()
+ await popup_page.goto(f"chrome-extension://{extension_id}/popup.html", wait_until="domcontentloaded")
+ await popup_page.locator("#advanced-settings summary").click()
+ await popup_page.locator("#bridge-port").fill(str(relay_port))
+ await popup_page.locator("#pairing-token").fill(pairing_token)
+ await popup_page.locator("#connection-button").click()
+ if not await _wait_for_pairing(runtime):
+ popup_error = await popup_page.locator("#connection-error").text_content()
+ raise TimeoutError(f"extension did not pair within 15 seconds; popup error: {popup_error or 'none'}")
+ return popup_page
+
+
+async def _exercise_page(page: Page, report: SmokeReport, screenshot_path: Path) -> FixtureServers:
+ servers = await _start_fixture_servers()
+ try:
+ try:
+ await page.goto("https://example.com", wait_until="domcontentloaded")
+ title = await page.title()
+ assert "Example" in title, f"unexpected example.com title: {title!r}"
+ except Exception:
+ await page.goto(servers.outer_url, wait_until="domcontentloaded")
+ report.pass_check("navigate", "loopback fallback")
+ else:
+ report.pass_check("navigate", f"title={title!r}")
+
+ await page.screenshot(path=screenshot_path)
+ assert screenshot_path.is_file() and screenshot_path.stat().st_size > 0
+ report.pass_check("screenshot", str(screenshot_path))
+
+ await page.goto(servers.outer_url, wait_until="domcontentloaded")
+ await page.click("#smoke-button")
+ selector_state = await page.evaluate("() => ({...window.smokeState})")
+ assert isinstance(selector_state, dict)
+ assert selector_state.get("clickCount") == 1
+ assert selector_state.get("lastTrusted") is True
+ report.pass_check("trusted selector click")
+
+ button = page.locator("#smoke-button")
+ bounds = await button.bounding_box()
+ assert bounds is not None, "button has no bounding box"
+ x = bounds["x"] + bounds["width"] / 2
+ y = bounds["y"] + bounds["height"] / 2
+ await page.mouse.click(x, y)
+ coordinate_state = await page.evaluate("() => ({...window.smokeState})")
+ assert isinstance(coordinate_state, dict)
+ assert coordinate_state.get("clickCount") == 2
+ assert coordinate_state.get("lastTrusted") is True
+ report.pass_check("coordinate click", f"x={x:.1f}, y={y:.1f}")
+
+ text_input = page.locator("#text-input")
+ await text_input.focus()
+ await page.keyboard.type(SMOKE_TEXT)
+ assert await text_input.input_value() == SMOKE_TEXT
+ assert await page.locator("#password-input").count() == 1
+ report.pass_check("type")
+
+ iframe_element = await page.wait_for_selector("#smoke-frame")
+ frame = await iframe_element.content_frame()
+ assert frame is not None, "iframe did not expose a Playwright frame"
+ await frame.wait_for_selector("#frame-marker")
+ marker = await frame.evaluate("document.querySelector('#frame-marker')?.textContent")
+ outer_origin = await page.evaluate("location.origin")
+ iframe_origin = await frame.evaluate("location.origin")
+ assert marker == "iframe-ready"
+ assert iframe_origin != outer_origin
+ report.pass_check("iframe evaluate", f"cross-origin frame={iframe_origin}")
+ except BaseException:
+ await servers.close()
+ raise
+ return servers
+
+
+async def _cleanup(
+ bridge_browser: Browser | None,
+ fixture_servers: FixtureServers | None,
+ persistent_context: BrowserContext | None,
+ runtime: BrowserExtensionRuntime | None,
+ playwright: Playwright | None,
+ profile_dir: tempfile.TemporaryDirectory[str] | None,
+) -> list[str]:
+ errors: list[str] = []
+ cleanup_steps: list[tuple[str, Callable[[], Awaitable[object]]]] = []
+ if bridge_browser is not None and bridge_browser.is_connected():
+ cleanup_steps.append(("bridge browser", bridge_browser.close))
+ if fixture_servers is not None:
+ cleanup_steps.append(("fixture servers", fixture_servers.close))
+ if persistent_context is not None:
+ cleanup_steps.append(("persistent context", persistent_context.close))
+ if runtime is not None:
+ cleanup_steps.append(("extension runtime", runtime.shutdown))
+ if playwright is not None:
+ cleanup_steps.append(("playwright", playwright.stop))
+ for label, cleanup_step in cleanup_steps:
+ try:
+ await cleanup_step()
+ except Exception as exc:
+ errors.append(f"{label}: {type(exc).__name__}: {exc}")
+ if profile_dir is not None:
+ try:
+ profile_dir.cleanup()
+ except Exception as exc:
+ errors.append(f"temporary profile: {type(exc).__name__}: {exc}")
+ return errors
+
+
+async def run_smoke() -> int:
+ report = SmokeReport()
+ pairing_token = secrets.token_urlsafe(32)
+ previous_token = os.environ.get(TOKEN_ENV)
+ os.environ[TOKEN_ENV] = pairing_token
+ redactions = [pairing_token]
+ runtime: BrowserExtensionRuntime | None = None
+ playwright: Playwright | None = None
+ persistent_context: BrowserContext | None = None
+ bridge_browser: Browser | None = None
+ fixture_servers: FixtureServers | None = None
+ profile_dir: tempfile.TemporaryDirectory[str] | None = None
+ screenshot_dir = Path(tempfile.mkdtemp(prefix="skyvern-extension-smoke-evidence-"))
+ screenshot_path = (screenshot_dir / "smoke_navigate.png").resolve()
+
+ try:
+ port_reservation, relay_port = _reserve_free_port()
+ port_reservation.close()
+ runtime = await BrowserExtensionRuntime.get_or_start(port=relay_port)
+ redactions.append(runtime.cdp_ws_url)
+ extension_dir = BrowserExtensionRuntime.extension_dir().resolve()
+ derived_extension_id = _extension_id_from_manifest(extension_dir)
+ assert derived_extension_id == EXPECTED_EXTENSION_ID, (
+ f"manifest key derived {derived_extension_id}, expected {EXPECTED_EXTENSION_ID}"
+ )
+
+ playwright = await async_playwright().start()
+ profile_dir = tempfile.TemporaryDirectory(prefix="skyvern-extension-smoke-profile-")
+ extension_path = str(extension_dir)
+ persistent_context = await playwright.chromium.launch_persistent_context(
+ profile_dir.name,
+ channel="chromium",
+ headless=False,
+ args=[
+ f"--disable-extensions-except={extension_path}",
+ f"--load-extension={extension_path}",
+ ],
+ )
+ discovered_extension_id = await _discover_extension_id(persistent_context)
+ if discovered_extension_id != EXPECTED_EXTENSION_ID:
+ print(f"WARN extension ID mismatch: expected {EXPECTED_EXTENSION_ID}, discovered {discovered_extension_id}")
+ assert discovered_extension_id == EXPECTED_EXTENSION_ID
+ extension_id = discovered_extension_id
+ report.pass_check("setup", f"relay port={relay_port}, extension ID={extension_id}")
+
+ await _pair_extension(persistent_context, runtime, extension_id, relay_port, pairing_token)
+ assert runtime.extension_connected
+ report.pass_check("extension paired")
+
+ bridge_browser = await playwright.chromium.connect_over_cdp(runtime.cdp_ws_url)
+ assert bridge_browser.contexts, "bridge connection exposed no browser context"
+ bridge_context = bridge_browser.contexts[0]
+ bridge_page = bridge_context.pages[0] if bridge_context.pages else await bridge_context.new_page()
+ fixture_servers = await _exercise_page(bridge_page, report, screenshot_path)
+
+ page_count = len(bridge_context.pages)
+ new_page = await bridge_context.new_page()
+ assert len(bridge_context.pages) == page_count + 1
+ report.pass_check("new tab")
+ await new_page.close()
+ assert new_page.is_closed()
+ assert len(bridge_context.pages) == page_count
+ report.pass_check("close tab")
+
+ await bridge_browser.close()
+ assert not bridge_browser.is_connected()
+ assert persistent_context.pages, "persistent Chrome context has no pages after bridge disconnect"
+ surviving_page = next((page for page in persistent_context.pages if not page.is_closed()), None)
+ assert surviving_page is not None
+ assert await surviving_page.evaluate("1 + 1") == 2
+ assert runtime.extension_connected
+ report.pass_check("chrome survives disconnect")
+
+ try:
+ mcp_statuses = await _run_mcp_phase(fixture_servers.outer_url)
+ except McpUnavailableError as exc:
+ report.skip_check("mcp session/navigate/observe", _safe_text(exc, redactions))
+ else:
+ report.pass_check("mcp session/navigate/observe", mcp_statuses)
+ except Exception as exc:
+ failed_check = next((label for label in CHECKS if label not in report.recorded), "smoke")
+ report.fail_check(failed_check, _safe_text(f"{type(exc).__name__}: {exc}", redactions))
+ report.skip_unrecorded(f"blocked by {failed_check}")
+ finally:
+ cleanup_errors = await _cleanup(
+ bridge_browser,
+ fixture_servers,
+ persistent_context,
+ runtime,
+ playwright,
+ profile_dir,
+ )
+ if cleanup_errors:
+ report.fail_check("cleanup", _safe_text("; ".join(cleanup_errors), redactions))
+ if previous_token is None:
+ os.environ.pop(TOKEN_ENV, None)
+ else:
+ os.environ[TOKEN_ENV] = previous_token
+
+ report.print_summary()
+ return 1 if report.failed else 0
+
+
+def _dry_run_helpers() -> int:
+ extension_dir = BrowserExtensionRuntime.extension_dir().resolve()
+ derived_extension_id = _extension_id_from_manifest(extension_dir)
+ if derived_extension_id != EXPECTED_EXTENSION_ID:
+ print(f"HELPER FAIL extension ID: expected {EXPECTED_EXTENSION_ID}, derived {derived_extension_id}")
+ return 1
+ print(f"HELPER PASS extension ID: {derived_extension_id}")
+ port_reservation, free_port = _reserve_free_port()
+ port_reservation.close()
+ if not 0 < free_port <= 65535:
+ print(f"HELPER FAIL free port: {free_port}")
+ return 1
+ print(f"HELPER PASS free port: {free_port}")
+ return 0
+
+
+def main(argv: Sequence[str] | None = None) -> int:
+ parser = argparse.ArgumentParser(description="Run the real Chrome extension bridge smoke test")
+ parser.add_argument("--dry-run-helpers", action="store_true", help="validate pure-Python helpers only")
+ args = parser.parse_args(argv)
+ if args.dry_run_helpers:
+ return _dry_run_helpers()
+ return asyncio.run(run_smoke())
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/tests/unit/browser_extension/__init__.py b/tests/unit/browser_extension/__init__.py
new file mode 100644
index 000000000..e69de29bb
diff --git a/tests/unit/browser_extension/test_auth.py b/tests/unit/browser_extension/test_auth.py
new file mode 100644
index 000000000..4a90265bc
--- /dev/null
+++ b/tests/unit/browser_extension/test_auth.py
@@ -0,0 +1,168 @@
+from __future__ import annotations
+
+import base64
+import json
+import os
+import stat
+from pathlib import Path
+
+import pytest
+
+import skyvern.browser_extension.auth as auth_module
+from skyvern.browser_extension.auth import (
+ build_challenge,
+ compute_ext_proof,
+ compute_server_proof,
+ load_or_create_pairing_token,
+ verify_ext_proof,
+)
+from skyvern.browser_extension.errors import BrowserExtensionError
+
+
+def test_extension_proof_round_trip_and_tampering() -> None:
+ token = "pairing-token"
+ server_nonce = "server-nonce"
+ client_nonce = "client-nonce"
+ proof = compute_ext_proof(token, server_nonce, client_nonce)
+
+ assert verify_ext_proof(token, server_nonce, client_nonce, proof)
+ assert not verify_ext_proof(token, "tampered-server", client_nonce, proof)
+ assert not verify_ext_proof(token, server_nonce, "tampered-client", proof)
+ assert not verify_ext_proof(token, server_nonce, client_nonce, f"{proof}x")
+ assert not verify_ext_proof(token, server_nonce, client_nonce, "not-ascii-\N{SNOWMAN}")
+
+
+def test_server_proof_uses_distinct_domain_separator() -> None:
+ token = "pairing-token"
+ server_nonce = "server-nonce"
+ client_nonce = "client-nonce"
+
+ assert compute_server_proof(token, client_nonce, server_nonce) != compute_ext_proof(
+ token, server_nonce, client_nonce
+ )
+
+
+def test_build_challenge_contains_a_32_byte_b64url_nonce() -> None:
+ server_nonce, raw_challenge = build_challenge()
+ padding = "=" * (-len(server_nonce) % 4)
+
+ assert len(base64.urlsafe_b64decode(server_nonce + padding)) == 32
+ assert "=" not in server_nonce
+ assert json.loads(raw_challenge) == {"v": 1, "type": "auth.challenge", "serverNonce": server_nonce}
+
+
+def test_environment_token_wins_over_existing_file(tmp_path: Path, monkeypatch) -> None:
+ token_dir = tmp_path / ".skyvern"
+ token_dir.mkdir()
+ (token_dir / "browser_extension_token").write_text("file-token")
+ monkeypatch.setattr(Path, "home", classmethod(lambda cls: tmp_path))
+ monkeypatch.setenv("SKYVERN_BROWSER_EXTENSION_TOKEN", " environment-token\n")
+
+ assert load_or_create_pairing_token() == "environment-token"
+
+
+def test_token_file_is_created_securely_and_reused(tmp_path: Path, monkeypatch) -> None:
+ monkeypatch.setattr(Path, "home", classmethod(lambda cls: tmp_path))
+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_TOKEN", raising=False)
+
+ first_token = load_or_create_pairing_token()
+ token_dir = tmp_path / ".skyvern"
+ token_file = token_dir / "browser_extension_token"
+
+ assert token_file.read_text() == first_token
+ assert stat.S_IMODE(token_dir.stat().st_mode) == 0o700
+ assert stat.S_IMODE(token_file.stat().st_mode) == 0o600
+
+ monkeypatch.setattr("skyvern.browser_extension.auth.secrets.token_urlsafe", lambda _: "different-token")
+ assert load_or_create_pairing_token() == first_token
+
+
+def test_existing_token_file_is_trimmed_and_hardened(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ token_dir = tmp_path / ".skyvern"
+ token_dir.mkdir()
+ token_file = token_dir / "browser_extension_token"
+ token_file.write_text(" file-token\n")
+ token_file.chmod(0o644)
+ monkeypatch.setattr(Path, "home", classmethod(lambda cls: tmp_path))
+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_TOKEN", raising=False)
+
+ assert load_or_create_pairing_token() == "file-token"
+ assert stat.S_IMODE(token_file.stat().st_mode) == 0o600
+
+
+def test_empty_environment_and_file_tokens_are_regenerated(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ token_dir = tmp_path / ".skyvern"
+ token_dir.mkdir()
+ token_file = token_dir / "browser_extension_token"
+ token_file.write_text(" \n")
+ monkeypatch.setattr(Path, "home", classmethod(lambda cls: tmp_path))
+ monkeypatch.setenv("SKYVERN_BROWSER_EXTENSION_TOKEN", " \n")
+ monkeypatch.setattr("skyvern.browser_extension.auth.secrets.token_urlsafe", lambda _: "regenerated-token")
+
+ assert load_or_create_pairing_token() == "regenerated-token"
+ assert token_file.read_text() == "regenerated-token"
+
+
+def test_token_path_must_be_a_regular_file(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ token_dir = tmp_path / ".skyvern"
+ token_dir.mkdir()
+ (token_dir / "browser_extension_token").mkdir()
+ monkeypatch.setattr(Path, "home", classmethod(lambda cls: tmp_path))
+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_TOKEN", raising=False)
+
+ with pytest.raises(BrowserExtensionError, match="regular file"):
+ load_or_create_pairing_token()
+
+
+def test_token_file_must_be_owned_by_current_user(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ token_dir = tmp_path / ".skyvern"
+ token_dir.mkdir()
+ (token_dir / "browser_extension_token").write_text("file-token")
+ monkeypatch.setattr(Path, "home", classmethod(lambda cls: tmp_path))
+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_TOKEN", raising=False)
+ actual_user_id = os.getuid()
+ monkeypatch.setattr("skyvern.browser_extension.auth.os.getuid", lambda: actual_user_id + 1)
+
+ with pytest.raises(BrowserExtensionError, match="owned"):
+ load_or_create_pairing_token()
+
+
+def test_publish_token_writes_secure_file_and_returns_new_token(tmp_path: Path) -> None:
+ token_path = tmp_path / "browser_extension_token"
+
+ assert auth_module._publish_token(token_path, "new-token") == "new-token"
+ assert token_path.read_text() == "new-token"
+ assert stat.S_IMODE(token_path.stat().st_mode) == 0o600
+
+
+def test_publish_token_collision_adopts_existing_token(tmp_path: Path) -> None:
+ token_path = tmp_path / "browser_extension_token"
+ token_path.write_text("winning-token")
+ token_path.chmod(0o600)
+
+ assert auth_module._publish_token(token_path, "losing-token") == "winning-token"
+ assert token_path.read_text() == "winning-token"
+
+
+def test_publish_token_replaces_corrupt_collision_and_retries_link_once(
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ token_path = tmp_path / "browser_extension_token"
+ real_link = os.link
+ link_calls = 0
+
+ def collide_once(source: str | Path, destination: str | Path) -> None:
+ nonlocal link_calls
+ link_calls += 1
+ if link_calls == 1:
+ token_path.write_text("\n")
+ token_path.chmod(0o600)
+ raise FileExistsError
+ real_link(source, destination)
+
+ monkeypatch.setattr(auth_module.os, "link", collide_once)
+
+ assert auth_module._publish_token(token_path, "new-token") == "new-token"
+ assert token_path.read_text() == "new-token"
+ assert link_calls == 2
diff --git a/tests/unit/browser_extension/test_cdp_adapter.py b/tests/unit/browser_extension/test_cdp_adapter.py
new file mode 100644
index 000000000..e2e0bd12d
--- /dev/null
+++ b/tests/unit/browser_extension/test_cdp_adapter.py
@@ -0,0 +1,642 @@
+from __future__ import annotations
+
+import asyncio
+from collections.abc import AsyncIterator
+from unittest.mock import patch
+
+import pytest
+import pytest_asyncio
+from aiohttp import ClientSession, ClientWebSocketResponse, WSMsgType
+
+from skyvern.browser_extension.cdp_adapter import ExtensionCdpAdapter
+from skyvern.browser_extension.errors import ExtensionRequestError
+from skyvern.browser_extension.target_registry import VirtualTargetRegistry
+
+
+class StubRelay:
+ def __init__(self, scoped_tabs: list[dict] | None = None) -> None:
+ self.scoped_tabs = scoped_tabs or []
+ self.calls: list[tuple[str, dict]] = []
+ self.next_tab_id = 100
+ self.fail_next: ExtensionRequestError | None = None
+ self.fail_attach_tab_ids: set[int] = set()
+ self.block_attach_tab_id: int | None = None
+ self.attach_started = asyncio.Event()
+ self.release_attach = asyncio.Event()
+
+ async def request(self, op: str, args: dict, timeout: float = 30.0) -> dict:
+ self.calls.append((op, args))
+ if self.fail_next is not None:
+ error = self.fail_next
+ self.fail_next = None
+ raise error
+ if op == "debugger.attach":
+ tab_id = args["tabId"]
+ if tab_id in self.fail_attach_tab_ids:
+ raise ExtensionRequestError("CDP_ERROR", "attach failed")
+ if tab_id == self.block_attach_tab_id:
+ self.attach_started.set()
+ await self.release_attach.wait()
+ if op == "tabs.create":
+ tab_id = self.next_tab_id
+ self.next_tab_id += 1
+ return {"tabId": tab_id}
+ if op == "debugger.send":
+ return {"result": {"forwardedMethod": args["method"]}}
+ return {}
+
+
+@pytest_asyncio.fixture
+async def adapter_server() -> AsyncIterator[tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry]]:
+ relay = StubRelay()
+ registry = VirtualTargetRegistry()
+ adapter = ExtensionCdpAdapter(registry, relay)
+ await adapter.start()
+ try:
+ yield adapter, relay, registry
+ finally:
+ await adapter.stop()
+
+
+async def receive_response(ws: ClientWebSocketResponse, request_id: int) -> dict:
+ while True:
+ message = await ws.receive_json(timeout=2)
+ if message.get("id") == request_id:
+ return message
+
+
+async def receive_event(ws: ClientWebSocketResponse, method: str) -> dict:
+ while True:
+ message = await ws.receive_json(timeout=2)
+ if message.get("method") == method:
+ return message
+
+
+def assert_subset(actual: dict, expected: dict) -> None:
+ for key, value in expected.items():
+ if isinstance(value, dict):
+ assert_subset(actual[key], value)
+ else:
+ assert actual[key] == value
+
+
+# Derived from chromium/crBrowser.js, chromium/crConnection.js, and chromium/chromium.js.
+ROOT_CONNECT_CONTRACT = [
+ (
+ "Browser.getVersion",
+ {"product": "Chrome/999.0.0.0", "userAgent": "Skyvern-Extension-Bridge"},
+ ),
+ ("Target.setAutoAttach", {}),
+ ("Browser.setDownloadBehavior", {}),
+ ("Target.getTargetInfo", {"targetInfo": {"targetId": "skyvern-browser", "type": "browser"}}),
+]
+
+ROOT_CONNECT_PARAMS = {
+ "Browser.getVersion": {},
+ "Target.setAutoAttach": {"autoAttach": True, "waitForDebuggerOnStart": True, "flatten": True},
+ "Browser.setDownloadBehavior": {"behavior": "allowAndName", "eventsEnabled": True},
+ "Target.getTargetInfo": {},
+}
+
+
+async def test_capability_path_is_required(
+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],
+) -> None:
+ adapter, _, _ = adapter_server
+ wrong_url = adapter.cdp_ws_url.rsplit("/", 1)[0] + "/wrong"
+
+ async with ClientSession() as client:
+ response = await client.get(wrong_url.replace("ws://", "http://"))
+ assert response.status == 404
+ response.release()
+
+ ws = await client.ws_connect(adapter.cdp_ws_url)
+ assert not ws.closed
+ await ws.close()
+
+
+async def test_root_connect_contract_fixture(
+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],
+) -> None:
+ adapter, _, _ = adapter_server
+
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ for request_id, (method, minimal_reply) in enumerate(ROOT_CONNECT_CONTRACT, start=1):
+ await ws.send_json({"id": request_id, "method": method, "params": ROOT_CONNECT_PARAMS[method]})
+ response = await receive_response(ws, request_id)
+ assert "error" not in response
+ assert_subset(response["result"], minimal_reply)
+
+
+async def test_auto_attach_creates_blank_tab_when_scope_is_empty(
+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],
+) -> None:
+ adapter, relay, registry = adapter_server
+
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ await ws.send_json({"id": 1, "method": "Target.setAutoAttach", "params": {"autoAttach": True}})
+ assert await receive_response(ws, 1) == {"id": 1, "result": {}}
+ attached = await receive_event(ws, "Target.attachedToTarget")
+ root_session_id = registry.root_session_id(100)
+
+ assert relay.calls[:2] == [
+ ("tabs.create", {"url": "about:blank"}),
+ ("debugger.attach", {"tabId": 100}),
+ ]
+ assert attached["params"] == {
+ "sessionId": root_session_id,
+ "targetInfo": {
+ "targetId": "tab-100",
+ "type": "page",
+ "title": "",
+ "url": "about:blank",
+ "attached": True,
+ "canAccessOpener": False,
+ "browserContextId": "skyvern-default",
+ },
+ "waitingForDebugger": False,
+ }
+
+
+async def test_auto_attach_existing_and_later_scoped_tabs() -> None:
+ relay = StubRelay(
+ [
+ {"tabId": 7, "url": "https://one.example", "title": "One"},
+ {"tabId": 8, "url": "https://two.example", "title": "Two"},
+ ]
+ )
+ relay.fail_next = ExtensionRequestError("CDP_ERROR", "Another debugger is already attached")
+ registry = VirtualTargetRegistry()
+ adapter = ExtensionCdpAdapter(registry, relay)
+ await adapter.start()
+ try:
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ await ws.send_json({"id": 1, "method": "Target.setAutoAttach", "params": {"autoAttach": True}})
+ await receive_response(ws, 1)
+ events = [await receive_event(ws, "Target.attachedToTarget") for _ in range(2)]
+
+ await adapter.handle_extension_event(
+ "scope.tabAdded", {"tabId": 9, "url": "https://three.example", "title": "Three"}
+ )
+ later = await receive_event(ws, "Target.attachedToTarget")
+
+ await adapter.handle_extension_event(
+ "tabs.created", {"tabId": 10, "openerTabId": 7, "url": "https://popup.example"}
+ )
+ popup = await receive_event(ws, "Target.attachedToTarget")
+ finally:
+ await adapter.stop()
+
+ assert [call for call in relay.calls if call[0] == "debugger.attach"] == [
+ ("debugger.attach", {"tabId": 7}),
+ ("debugger.attach", {"tabId": 8}),
+ ("debugger.attach", {"tabId": 9}),
+ ("debugger.attach", {"tabId": 10}),
+ ]
+ assert [event["params"]["targetInfo"]["targetId"] for event in events] == ["tab-7", "tab-8"]
+ assert later["params"]["targetInfo"]["targetId"] == "tab-9"
+ assert popup["params"]["targetInfo"]["openerId"] == "tab-7"
+
+
+async def test_auto_attach_post_reply_failure_skips_failed_tab_without_second_response() -> None:
+ relay = StubRelay(
+ [
+ {"tabId": 30, "url": "https://bad.example", "title": "Bad"},
+ {"tabId": 31, "url": "https://good.example", "title": "Good"},
+ ]
+ )
+ relay.fail_attach_tab_ids.add(30)
+ registry = VirtualTargetRegistry()
+ adapter = ExtensionCdpAdapter(registry, relay)
+ await adapter.start()
+ try:
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ await ws.send_json({"id": 1, "method": "Target.setAutoAttach", "params": {"autoAttach": True}})
+
+ assert await receive_response(ws, 1) == {"id": 1, "result": {}}
+ attached = await receive_event(ws, "Target.attachedToTarget")
+ assert attached["params"]["targetInfo"]["targetId"] == "tab-31"
+ with pytest.raises(TimeoutError):
+ await ws.receive_json(timeout=0.05)
+ finally:
+ await adapter.stop()
+
+
+async def test_tab_removal_tombstones_in_flight_background_attach() -> None:
+ relay = StubRelay([{"tabId": 40, "url": "https://existing.example", "title": "Existing"}])
+ registry = VirtualTargetRegistry()
+ adapter = ExtensionCdpAdapter(registry, relay)
+ await adapter.start()
+ try:
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ await ws.send_json({"id": 1, "method": "Target.setAutoAttach", "params": {"autoAttach": True}})
+ await receive_response(ws, 1)
+ await receive_event(ws, "Target.attachedToTarget")
+
+ relay.block_attach_tab_id = 41
+ await adapter.handle_extension_event(
+ "scope.tabAdded", {"tabId": 41, "url": "https://racy.example", "title": "Racy"}
+ )
+ await asyncio.wait_for(relay.attach_started.wait(), 1)
+ removal_task = asyncio.create_task(
+ adapter.handle_extension_event("scope.tabRemoved", {"tabId": 41, "reason": "unshared"})
+ )
+ await asyncio.sleep(0)
+ relay.release_attach.set()
+ await removal_task
+
+ with pytest.raises(KeyError):
+ registry.root_session_id(41)
+ with pytest.raises(TimeoutError):
+ await ws.receive_json(timeout=0.05)
+ finally:
+ await adapter.stop()
+
+
+async def test_session_command_routes_to_relay_and_preserves_session_id(
+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],
+) -> None:
+ adapter, relay, registry = adapter_server
+ registry.register_tab(42, "https://example.com", "Example")
+ session_id = registry.root_session_id(42)
+
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ await ws.send_json(
+ {"id": 5, "sessionId": session_id, "method": "Runtime.evaluate", "params": {"expression": "1+1"}}
+ )
+ response = await receive_response(ws, 5)
+
+ await ws.send_json({"id": 6, "sessionId": "missing", "method": "Runtime.enable", "params": {}})
+ missing = await receive_response(ws, 6)
+
+ assert relay.calls[-1] == (
+ "debugger.send",
+ {"tabId": 42, "method": "Runtime.evaluate", "params": {"expression": "1+1"}},
+ )
+ assert response == {"id": 5, "sessionId": session_id, "result": {"forwardedMethod": "Runtime.evaluate"}}
+ assert missing == {
+ "id": 6,
+ "sessionId": "missing",
+ "error": {"code": -32001, "message": "session not found"},
+ }
+
+
+ ("method", "params"),
+ [
+ ("Network.getAllCookies", {}),
+ ("Network.getCookies", {"urls": ["https://unshared.example"]}),
+ ],
+)
+async def test_session_command_rejects_denied_cdp_methods(
+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],
+ method: str,
+ params: dict,
+) -> None:
+ adapter, relay, registry = adapter_server
+ registry.register_tab(42, "https://example.com", "Example")
+ session_id = registry.root_session_id(42)
+
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ await ws.send_json({"id": 1, "sessionId": session_id, "method": method, "params": params})
+ response = await receive_response(ws, 1)
+
+ assert response == {
+ "id": 1,
+ "sessionId": session_id,
+ "error": {
+ "code": -32000,
+ "message": "CDP_METHOD_NOT_ALLOWED: The requested CDP method is not allowed.",
+ },
+ }
+ assert relay.calls == []
+
+
+async def test_page_attach_aliases_are_unique_route_commands_and_receive_root_events() -> None:
+ relay = StubRelay([{"tabId": 42, "url": "https://example.com", "title": "Example"}])
+ registry = VirtualTargetRegistry()
+ adapter = ExtensionCdpAdapter(registry, relay)
+ await adapter.start()
+ try:
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ await ws.send_json({"id": 1, "method": "Target.setAutoAttach", "params": {"autoAttach": True}})
+ await receive_response(ws, 1)
+ primary_attached = await receive_event(ws, "Target.attachedToTarget")
+ primary_session_id = primary_attached["params"]["sessionId"]
+
+ for request_id in (2, 3):
+ await ws.send_json(
+ {
+ "id": request_id,
+ "method": "Target.attachToTarget",
+ "params": {"targetId": "tab-42", "flatten": True},
+ }
+ )
+ first_alias = (await receive_response(ws, 2))["result"]["sessionId"]
+ second_alias = (await receive_response(ws, 3))["result"]["sessionId"]
+
+ assert len({primary_session_id, first_alias, second_alias}) == 3
+ await ws.send_json({"id": 4, "sessionId": first_alias, "method": "Runtime.enable", "params": {}})
+ assert (await receive_response(ws, 4))["sessionId"] == first_alias
+
+ event_params = {"name": "networkAlmostIdle", "frameId": "main"}
+ await adapter.handle_extension_event(
+ "debugger.event",
+ {"tabId": 42, "method": "Page.lifecycleEvent", "params": event_params},
+ )
+ routed_events = [await receive_event(ws, "Page.lifecycleEvent") for _ in range(3)]
+ assert {event["sessionId"] for event in routed_events} == {
+ primary_session_id,
+ first_alias,
+ second_alias,
+ }
+
+ await ws.send_json({"id": 5, "method": "Target.detachFromTarget", "params": {"sessionId": first_alias}})
+ assert await receive_response(ws, 5) == {"id": 5, "result": {}}
+ assert all(op != "debugger.detach" for op, _ in relay.calls)
+ with pytest.raises(KeyError):
+ registry.resolve_session(first_alias)
+ assert registry.resolve_session(second_alias) == (42, None)
+ finally:
+ await adapter.stop()
+
+
+async def test_browser_alias_routes_root_commands_and_detaches_independently(
+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],
+) -> None:
+ adapter, _, registry = adapter_server
+
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ await ws.send_json({"id": 1, "method": "Target.attachToBrowserTarget", "params": {}})
+ first_alias = (await receive_response(ws, 1))["result"]["sessionId"]
+ await ws.send_json({"id": 2, "method": "Target.attachToBrowserTarget", "params": {}})
+ second_alias = (await receive_response(ws, 2))["result"]["sessionId"]
+
+ assert first_alias != second_alias
+ await ws.send_json({"id": 3, "sessionId": first_alias, "method": "Browser.getVersion", "params": {}})
+ version = await receive_response(ws, 3)
+ assert version["sessionId"] == first_alias
+ assert version["result"]["protocolVersion"] == "1.3"
+
+ await ws.send_json({"id": 4, "method": "Target.detachFromTarget", "params": {"sessionId": first_alias}})
+ assert await receive_response(ws, 4) == {"id": 4, "result": {}}
+ assert not registry.is_browser_session_alias(first_alias)
+ assert registry.is_browser_session_alias(second_alias)
+
+ await ws.send_json({"id": 5, "sessionId": first_alias, "method": "Browser.getVersion", "params": {}})
+ assert (await receive_response(ws, 5))["error"]["code"] == -32001
+
+
+async def test_child_attach_recurses_and_detach_unregisters(
+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],
+) -> None:
+ adapter, relay, registry = adapter_server
+ registry.register_tab(12, "https://example.com", "Example")
+ root_session_id = registry.root_session_id(12)
+
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ await ws.send_json({"id": 1, "method": "Target.setAutoAttach", "params": {"autoAttach": True}})
+ await receive_response(ws, 1)
+ await receive_event(ws, "Target.attachedToTarget")
+ relay.calls.clear()
+
+ target_info = {
+ "targetId": "frame-12",
+ "type": "iframe",
+ "title": "",
+ "url": "https://example.com/frame",
+ "attached": True,
+ "canAccessOpener": False,
+ "browserContextId": "skyvern-default",
+ }
+ await adapter.handle_extension_event(
+ "debugger.event",
+ {
+ "tabId": 12,
+ "method": "Target.attachedToTarget",
+ "params": {"sessionId": "child-12", "targetInfo": target_info, "waitingForDebugger": False},
+ },
+ )
+ attached = await receive_event(ws, "Target.attachedToTarget")
+
+ await ws.send_json({"id": 2, "sessionId": "child-12", "method": "Runtime.enable", "params": {}})
+ await receive_response(ws, 2)
+
+ await adapter.handle_extension_event(
+ "debugger.event",
+ {"tabId": 12, "method": "Target.detachedFromTarget", "params": {"sessionId": "child-12"}},
+ )
+ detached = await receive_event(ws, "Target.detachedFromTarget")
+
+ assert attached["sessionId"] == root_session_id
+ assert relay.calls[0] == (
+ "debugger.send",
+ {
+ "tabId": 12,
+ "sessionId": "child-12",
+ "method": "Target.setAutoAttach",
+ "params": {"flatten": True, "autoAttach": True, "waitForDebuggerOnStart": False},
+ },
+ )
+ assert relay.calls[1][1]["sessionId"] == "child-12"
+ assert detached["params"] == {"sessionId": "child-12"}
+ with pytest.raises(KeyError):
+ registry.resolve_session("child-12")
+
+
+async def test_get_target_info_returns_registered_child_target(
+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],
+) -> None:
+ adapter, _, registry = adapter_server
+ registry.register_tab(13, "https://example.com", "Example")
+ child_target_info = {
+ "targetId": "frame-13",
+ "type": "iframe",
+ "title": "Frame",
+ "url": "https://example.com/frame",
+ "attached": True,
+ "canAccessOpener": False,
+ "browserContextId": "skyvern-default",
+ }
+ registry.register_child_session(13, "child-13", child_target_info)
+
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ await ws.send_json({"id": 1, "method": "Target.getTargetInfo", "params": {"targetId": "frame-13"}})
+ response = await receive_response(ws, 1)
+
+ assert response == {"id": 1, "result": {"targetInfo": child_target_info}}
+
+
+async def test_create_and_close_target(
+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],
+) -> None:
+ adapter, relay, _ = adapter_server
+
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ await ws.send_json({"id": 1, "method": "Target.createTarget", "params": {"url": "https://example.com"}})
+ attached = await receive_event(ws, "Target.attachedToTarget")
+ created = await receive_response(ws, 1)
+
+ await ws.send_json({"id": 2, "method": "Target.closeTarget", "params": {"targetId": "tab-100"}})
+ closed = await receive_response(ws, 2)
+ await adapter.handle_extension_event("scope.tabRemoved", {"tabId": 100, "reason": "closed"})
+ detached = await receive_event(ws, "Target.detachedFromTarget")
+ destroyed = await receive_event(ws, "Target.targetDestroyed")
+
+ assert created == {"id": 1, "result": {"targetId": "tab-100"}}
+ assert attached["params"]["targetInfo"]["url"] == "https://example.com"
+ assert closed == {"id": 2, "result": {"success": True}}
+ assert detached["params"]["sessionId"].startswith("sess-tab-100-")
+ assert destroyed["params"] == {"targetId": "tab-100"}
+ assert ("tabs.remove", {"tabId": 100}) in relay.calls
+
+
+async def test_create_target_and_scope_event_register_tab_once(
+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],
+) -> None:
+ adapter, relay, registry = adapter_server
+ original_request = relay.request
+
+ async def request_with_scope_event(op: str, args: dict, timeout: float = 30.0) -> dict:
+ result = await original_request(op, args, timeout)
+ if op == "tabs.create":
+ await adapter.handle_extension_event(
+ "scope.tabAdded",
+ {"tabId": result["tabId"], "url": args["url"], "title": ""},
+ )
+ return result
+
+ relay.request = request_with_scope_event
+
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ await ws.send_json({"id": 1, "method": "Target.setDiscoverTargets", "params": {"discover": True}})
+ assert await receive_response(ws, 1) == {"id": 1, "result": {}}
+ browser_created = await receive_event(ws, "Target.targetCreated")
+ assert browser_created["params"]["targetInfo"]["targetId"] == "skyvern-browser"
+
+ with patch.object(registry, "register_tab", wraps=registry.register_tab) as register_tab:
+ await ws.send_json({"id": 2, "method": "Target.createTarget", "params": {"url": "https://example.com"}})
+ target_created = await receive_event(ws, "Target.targetCreated")
+ await receive_event(ws, "Target.attachedToTarget")
+ created = await receive_response(ws, 2)
+
+ assert register_tab.call_count == 1
+ assert target_created["params"]["targetInfo"]["targetId"] == "tab-100"
+ assert created == {"id": 2, "result": {"targetId": "tab-100"}}
+ assert len(registry.list_page_targets()) == 1
+ await adapter.handle_extension_event(
+ "scope.tabAdded",
+ {"tabId": 100, "url": "https://example.com", "title": ""},
+ )
+ with pytest.raises(TimeoutError):
+ await ws.receive_json(timeout=0.05)
+
+
+async def test_debugger_detached_emits_destroyed_and_navigation_updates_target(
+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],
+) -> None:
+ adapter, _, registry = adapter_server
+ registry.register_tab(15, "https://old.example", "Old")
+ session_id = registry.root_session_id(15)
+
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ await adapter.handle_extension_event(
+ "debugger.event",
+ {
+ "tabId": 15,
+ "method": "Page.frameNavigated",
+ "params": {"frame": {"id": "main", "url": "https://new.example"}},
+ },
+ )
+ navigated = await receive_event(ws, "Page.frameNavigated")
+ assert navigated["sessionId"] == session_id
+ assert registry.target_info("tab-15")["url"] == "https://new.example"
+
+ await adapter.handle_extension_event("debugger.detached", {"tabId": 15, "reason": "canceled_by_user"})
+ detached = await receive_event(ws, "Target.detachedFromTarget")
+ destroyed = await receive_event(ws, "Target.targetDestroyed")
+
+ assert detached["params"] == {"sessionId": session_id, "targetId": "tab-15"}
+ assert destroyed["params"] == {"targetId": "tab-15"}
+ with pytest.raises(KeyError):
+ registry.root_session_id(15)
+
+
+async def test_unknown_root_and_extension_request_errors(
+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],
+) -> None:
+ adapter, relay, registry = adapter_server
+ registry.register_tab(18, "https://example.com", "Example")
+ session_id = registry.root_session_id(18)
+
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ await ws.send_json({"id": 1, "method": "Nope.missing", "params": {}})
+ unknown = await receive_response(ws, 1)
+
+ relay.fail_next = ExtensionRequestError("CDP_ERROR", "command failed")
+ await ws.send_json({"id": 2, "sessionId": session_id, "method": "Runtime.enable", "params": {}})
+ failed = await receive_response(ws, 2)
+
+ assert unknown == {"id": 1, "error": {"code": -32601, "message": "'Nope.missing' wasn't found"}}
+ assert failed == {
+ "id": 2,
+ "sessionId": session_id,
+ "error": {"code": -32000, "message": "CDP_ERROR: command failed"},
+ }
+
+
+async def test_second_client_rejected_and_disconnect_allows_future_client(
+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],
+) -> None:
+ adapter, _, registry = adapter_server
+ registry.register_tab(20, "https://example.com", "Example")
+
+ async with ClientSession() as client:
+ first = await client.ws_connect(adapter.cdp_ws_url)
+ second = await client.ws_connect(adapter.cdp_ws_url)
+ await second.receive(timeout=2)
+ assert second.close_code == 4409
+
+ await adapter.on_extension_disconnect()
+ await first.receive(timeout=2)
+ assert first.close_code == 1001
+ assert registry.list_page_targets() == []
+
+ future = await client.ws_connect(adapter.cdp_ws_url)
+ await future.send_json({"id": 1, "method": "Browser.getVersion", "params": {}})
+ assert (await receive_response(future, 1))["result"]["protocolVersion"] == "1.3"
+ await future.close()
+
+
+async def test_browser_close_replies_then_closes_without_removing_tabs(
+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],
+) -> None:
+ adapter, relay, registry = adapter_server
+ registry.register_tab(22, "https://example.com", "Example")
+
+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:
+ await ws.send_json({"id": 9, "method": "Browser.close", "params": {}})
+ assert await receive_response(ws, 9) == {"id": 9, "result": {}}
+ close_message = await ws.receive(timeout=2)
+ assert close_message.type in {WSMsgType.CLOSE, WSMsgType.CLOSED}
+
+ assert ("debugger.detach", {"tabId": 22}) in relay.calls
+ assert all(op != "tabs.remove" for op, _ in relay.calls)
diff --git a/tests/unit/browser_extension/test_mcp_glue.py b/tests/unit/browser_extension/test_mcp_glue.py
new file mode 100644
index 000000000..58d29b905
--- /dev/null
+++ b/tests/unit/browser_extension/test_mcp_glue.py
@@ -0,0 +1,591 @@
+from __future__ import annotations
+
+import hashlib
+import hmac
+import json
+import sys
+import types
+from pathlib import Path
+from types import SimpleNamespace
+from unittest.mock import AsyncMock, MagicMock, Mock, call
+
+import pytest
+import typer
+from typer.testing import CliRunner
+
+from skyvern.browser_extension.errors import BrowserExtensionError
+from skyvern.browser_extension.runtime import BrowserExtensionRuntime
+from skyvern.cli import run_commands
+from skyvern.cli.commands import browser as browser_commands
+from skyvern.cli.commands.browser import browser_app
+from skyvern.cli.core import session_manager
+from skyvern.cli.core.result import BrowserContext
+from skyvern.cli.mcp_tools import session as mcp_session
+
+_NOT_CONNECTED_GUIDANCE = (
+ "Skyvern browser extension is not connected. Install it: run `skyvern browser extension-path`, open "
+ "chrome://extensions, enable Developer mode, Load unpacked, and select that directory. Then run "
+ "`skyvern browser extension-token`, paste the token into the extension popup, and click Connect. Then retry."
+)
+
+
[email protected](autouse=True)
+def _use_stdio_transport(monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(session_manager, "_stateless_http_mode", False)
+
+
+ ("browser_type", "expected"),
+ [
+ (None, False),
+ ("", False),
+ ("cdp-connect", False),
+ ("extension-connect", True),
+ ("Extension-Connect", False),
+ ],
+)
+def test_should_default_to_extension_env_matrix(
+ monkeypatch: pytest.MonkeyPatch,
+ browser_type: str | None,
+ expected: bool,
+) -> None:
+ if browser_type is None:
+ monkeypatch.setenv("BROWSER_TYPE", "leak-guard-sentinel")
+ monkeypatch.delenv("BROWSER_TYPE", raising=False)
+ else:
+ monkeypatch.setenv("BROWSER_TYPE", browser_type)
+
+ assert mcp_session._should_default_to_extension() is expected
+
+
+async def test_explicit_local_session_does_not_start_extension_runtime(monkeypatch: pytest.MonkeyPatch) -> None:
+ browser = MagicMock(app_url=None)
+ do_session_create = AsyncMock(return_value=(browser, SimpleNamespace(local=True, headless=False)))
+ get_or_start = AsyncMock()
+ monkeypatch.setenv("BROWSER_TYPE", "extension-connect")
+ monkeypatch.setattr(mcp_session, "get_skyvern", MagicMock())
+ monkeypatch.setattr(mcp_session, "do_session_create", do_session_create)
+ monkeypatch.setattr(mcp_session, "set_current_session", MagicMock())
+ monkeypatch.setattr(mcp_session.BrowserExtensionRuntime, "get_or_start", get_or_start)
+
+ result = await mcp_session.skyvern_browser_session_create(local=True)
+
+ assert result["ok"] is True
+ assert result["data"] == {"local": True, "headless": False}
+ get_or_start.assert_not_awaited()
+ do_session_create.assert_awaited_once()
+
+
+async def test_extension_session_rejects_stateless_http_mode(monkeypatch: pytest.MonkeyPatch) -> None:
+ get_or_start = AsyncMock()
+ monkeypatch.setenv("BROWSER_TYPE", "extension-connect")
+ monkeypatch.setattr(mcp_session.BrowserExtensionRuntime, "get_or_start", get_or_start)
+ session_manager.set_stateless_http_mode(True)
+ try:
+ result = await mcp_session.skyvern_browser_session_create()
+ finally:
+ session_manager.set_stateless_http_mode(False)
+
+ assert result["ok"] is False
+ assert result["error"]["code"] == mcp_session.ErrorCode.SDK_ERROR
+ assert result["error"]["message"] == (
+ "The Skyvern browser extension requires the MCP server to run on the stdio transport. "
+ "Restart with: skyvern mcp --browser-extension"
+ )
+ get_or_start.assert_not_awaited()
+
+
+async def test_session_create_extension_takes_precedence_over_cdp(monkeypatch: pytest.MonkeyPatch) -> None:
+ runtime = SimpleNamespace(wait_for_extension=AsyncMock(return_value=True))
+ get_or_start = AsyncMock(return_value=runtime)
+ resolve_browser = AsyncMock(return_value=(MagicMock(), BrowserContext(mode="extension")))
+ monkeypatch.setattr(mcp_session, "_should_default_to_extension", lambda: True)
+ monkeypatch.setattr(mcp_session, "_should_default_to_cdp", lambda: (True, "ws://cdp.example.test"))
+ monkeypatch.setattr(mcp_session.BrowserExtensionRuntime, "get_or_start", get_or_start)
+ monkeypatch.setattr(mcp_session, "resolve_browser", resolve_browser)
+
+ result = await mcp_session.skyvern_browser_session_create()
+
+ assert result["ok"] is True
+ assert result["data"]["browser"] == "extension"
+ assert result["data"]["session"] == "implicit"
+ get_or_start.assert_awaited_once_with()
+ runtime.wait_for_extension.assert_awaited_once_with(10.0)
+ resolve_browser.assert_awaited_once_with(extension_runtime=runtime)
+
+
+async def test_session_create_extension_not_connected_returns_pinned_guidance(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ runtime = SimpleNamespace(wait_for_extension=AsyncMock(return_value=False))
+ get_or_start = AsyncMock(return_value=runtime)
+ resolve_browser = AsyncMock()
+ monkeypatch.setenv("BROWSER_TYPE", "extension-connect")
+ monkeypatch.setattr(mcp_session.BrowserExtensionRuntime, "get_or_start", get_or_start)
+ monkeypatch.setattr(mcp_session, "resolve_browser", resolve_browser)
+
+ result = await mcp_session.skyvern_browser_session_create()
+
+ assert result["ok"] is False
+ assert result["error"]["message"] == _NOT_CONNECTED_GUIDANCE
+ get_or_start.assert_awaited_once_with()
+ runtime.wait_for_extension.assert_awaited_once_with(10.0)
+ resolve_browser.assert_not_awaited()
+
+
+async def test_session_create_extension_connected_returns_safe_success_payload(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ runtime = SimpleNamespace(
+ cdp_ws_url="ws://127.0.0.1/private-capability",
+ pairing_token="private-pairing-token",
+ wait_for_extension=AsyncMock(return_value=True),
+ )
+ monkeypatch.setenv("BROWSER_TYPE", "extension-connect")
+ monkeypatch.setattr(mcp_session.BrowserExtensionRuntime, "get_or_start", AsyncMock(return_value=runtime))
+ monkeypatch.setattr(
+ mcp_session,
+ "resolve_browser",
+ AsyncMock(return_value=(MagicMock(), BrowserContext(mode="extension"))),
+ )
+
+ result = await mcp_session.skyvern_browser_session_create()
+
+ assert result["ok"] is True
+ assert result["browser_context"]["mode"] == "extension"
+ assert result["data"]["browser"] == "extension"
+ assert result["data"]["session"] == "implicit"
+ assert runtime.cdp_ws_url not in repr(result)
+ assert runtime.pairing_token not in repr(result)
+
+
+async def test_session_create_extension_runtime_error_is_structured(monkeypatch: pytest.MonkeyPatch) -> None:
+ message = "Browser extension relay port 19777 is already in use"
+ monkeypatch.setenv("BROWSER_TYPE", "extension-connect")
+ monkeypatch.setattr(
+ mcp_session.BrowserExtensionRuntime,
+ "get_or_start",
+ AsyncMock(side_effect=BrowserExtensionError(message)),
+ )
+
+ result = await mcp_session.skyvern_browser_session_create()
+
+ assert result["ok"] is False
+ assert result["error"]["message"] == message
+
+
+async def test_session_create_extension_connection_error_redacts_capability_url(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ from skyvern.library.skyvern import Skyvern
+
+ capability_token = "fake-secret-capability"
+ capability_url = f"ws://127.0.0.1:43210/cdp/{capability_token}"
+ connect_over_cdp = AsyncMock(
+ side_effect=RuntimeError(f"BrowserType.connect_over_cdp failed while connecting to {capability_url}")
+ )
+ playwright = SimpleNamespace(chromium=SimpleNamespace(connect_over_cdp=connect_over_cdp))
+ skyvern = object.__new__(Skyvern)
+ skyvern._get_playwright = AsyncMock(return_value=playwright)
+ runtime = SimpleNamespace(cdp_ws_url=capability_url, wait_for_extension=AsyncMock(return_value=True))
+ current_token = session_manager._current_session.set(None)
+ monkeypatch.setattr(session_manager, "_global_session", None)
+ monkeypatch.setattr(session_manager, "get_skyvern", lambda: skyvern)
+ monkeypatch.setenv("BROWSER_TYPE", "extension-connect")
+ monkeypatch.setattr(mcp_session.BrowserExtensionRuntime, "get_or_start", AsyncMock(return_value=runtime))
+
+ try:
+ result = await mcp_session.skyvern_browser_session_create()
+ finally:
+ session_manager._current_session.reset(current_token)
+
+ assert result["ok"] is False
+ assert result["error"]["message"] == _NOT_CONNECTED_GUIDANCE
+ assert "/cdp/" not in repr(result)
+ assert capability_token not in repr(result)
+ connect_over_cdp.assert_awaited_once_with(capability_url)
+
+
+async def test_resolve_browser_extension_connects_before_cdp(monkeypatch: pytest.MonkeyPatch) -> None:
+ current_token = session_manager._current_session.set(None)
+ monkeypatch.setattr(session_manager, "_global_session", None)
+ runtime = SimpleNamespace()
+ browser = MagicMock()
+ fake_skyvern = MagicMock()
+ fake_skyvern.connect_to_browser_extension = AsyncMock(return_value=browser)
+ fake_skyvern.connect_to_browser_over_cdp = AsyncMock()
+ monkeypatch.setattr(session_manager, "get_skyvern", lambda: fake_skyvern)
+
+ try:
+ resolved_browser, context = await session_manager.resolve_browser(
+ cdp_url="ws://cdp.example.test",
+ extension_runtime=runtime,
+ )
+ finally:
+ session_manager._current_session.reset(current_token)
+
+ assert resolved_browser is browser
+ assert context == BrowserContext(mode="extension", can_access_localhost=True)
+ fake_skyvern.connect_to_browser_extension.assert_awaited_once_with(runtime)
+ fake_skyvern.connect_to_browser_over_cdp.assert_not_awaited()
+
+
+async def test_resolve_browser_reconnects_disconnected_extension_session(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ current_token = session_manager._current_session.set(None)
+ monkeypatch.setattr(session_manager, "_global_session", None)
+ playwright_browser = SimpleNamespace(is_connected=Mock(side_effect=[True, False]))
+ stale_browser = SimpleNamespace(browser=playwright_browser, close=AsyncMock())
+ fresh_browser = SimpleNamespace(browser=SimpleNamespace(is_connected=Mock(return_value=True)))
+ runtime = SimpleNamespace()
+ fake_skyvern = MagicMock()
+ fake_skyvern.connect_to_browser_extension = AsyncMock(return_value=fresh_browser)
+ monkeypatch.setattr(session_manager, "get_active_api_key", lambda: None)
+ monkeypatch.setattr(session_manager, "get_skyvern", lambda: fake_skyvern)
+ session_manager.set_current_session(
+ session_manager.SessionState(browser=stale_browser, context=BrowserContext(mode="extension"))
+ )
+
+ try:
+ first_browser, first_context = await session_manager.resolve_browser(extension_runtime=runtime)
+ second_browser, second_context = await session_manager.resolve_browser(extension_runtime=runtime)
+ finally:
+ session_manager._current_session.reset(current_token)
+
+ assert first_browser is stale_browser
+ assert first_context.mode == "extension"
+ assert second_browser is fresh_browser
+ assert second_context == BrowserContext(mode="extension", can_access_localhost=True)
+ assert playwright_browser.is_connected.call_count == 2
+ stale_browser.close.assert_awaited_once_with()
+ fake_skyvern.connect_to_browser_extension.assert_awaited_once_with(runtime)
+
+
+def test_extension_session_ref_key_is_stable_and_distinct(monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(session_manager, "get_active_api_key", lambda: None)
+ extension_state = session_manager.SessionState(context=BrowserContext(mode="extension"))
+ local_state = session_manager.SessionState(context=BrowserContext(mode="local"))
+
+ first = session_manager._session_ref_key(extension_state)
+ second = session_manager._session_ref_key(extension_state)
+
+ assert first == second == (None, "extension", "own-browser", None)
+ assert first != session_manager._session_ref_key(local_state)
+
+
[email protected]("has_existing_context", [True, False])
+async def test_connect_to_browser_extension_uses_runtime_cdp_url_and_context_fallback(
+ monkeypatch: pytest.MonkeyPatch,
+ has_existing_context: bool,
+) -> None:
+ from skyvern.library.skyvern import Skyvern
+
+ existing_context = MagicMock()
+ new_context = MagicMock()
+ browser = SimpleNamespace(
+ contexts=[existing_context] if has_existing_context else [],
+ new_context=AsyncMock(return_value=new_context),
+ )
+ connect_over_cdp = AsyncMock(return_value=browser)
+ playwright = SimpleNamespace(chromium=SimpleNamespace(connect_over_cdp=connect_over_cdp))
+ skyvern = object.__new__(Skyvern)
+ skyvern._get_playwright = AsyncMock(return_value=playwright)
+ runtime = SimpleNamespace(cdp_ws_url="ws://127.0.0.1/private-capability")
+
+ wrapper_module = types.ModuleType("skyvern.library.skyvern_browser")
+
+ class FakeSkyvernBrowser:
+ def __init__(self, client: object, browser_context: object, **kwargs: object) -> None:
+ self.client = client
+ self.browser_context = browser_context
+ self.kwargs = kwargs
+
+ wrapper_module.SkyvernBrowser = FakeSkyvernBrowser
+ monkeypatch.setitem(sys.modules, "skyvern.library.skyvern_browser", wrapper_module)
+
+ connected = await skyvern.connect_to_browser_extension(runtime)
+
+ connect_over_cdp.assert_awaited_once_with(runtime.cdp_ws_url)
+ assert connected.browser_context is (existing_context if has_existing_context else new_context)
+ if has_existing_context:
+ browser.new_context.assert_not_awaited()
+ else:
+ browser.new_context.assert_awaited_once_with()
+
+
+def test_browser_extension_path_command_prints_real_absolute_directory() -> None:
+ result = CliRunner().invoke(browser_app, ["extension-path"])
+
+ assert result.exit_code == 0
+ assert result.stdout.strip() == str(BrowserExtensionRuntime.extension_dir().resolve())
+
+
+def test_browser_extension_token_command_creates_but_does_not_print_token(
+ monkeypatch: pytest.MonkeyPatch,
+ tmp_path: Path,
+) -> None:
+ monkeypatch.setenv("HOME", str(tmp_path))
+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_TOKEN", raising=False)
+ monkeypatch.setattr(browser_commands.shutil, "which", lambda _name: None)
+
+ result = CliRunner().invoke(browser_app, ["extension-token"])
+
+ token_path = tmp_path / ".skyvern" / "browser_extension_token"
+ token = token_path.read_text()
+ assert result.exit_code == 0
+ assert token_path.is_file()
+ assert token not in result.stdout
+ assert result.stdout.splitlines()[0].startswith("Pairing token was not printed.")
+ assert "SKYVERN_BROWSER_EXTENSION_TOKEN" in result.stdout
+ assert "Paste this token into the Skyvern browser extension popup." in result.stdout
+
+
+def test_browser_extension_token_command_copies_token_when_clipboard_is_available(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ token = "pairing-token-clipboard-sentinel"
+ run = MagicMock()
+ monkeypatch.setattr(browser_commands, "load_or_create_pairing_token", lambda: token)
+ monkeypatch.setattr(browser_commands.sys, "platform", "darwin")
+ monkeypatch.setattr(browser_commands.shutil, "which", lambda name: f"/usr/bin/{name}")
+ monkeypatch.setattr(browser_commands.subprocess, "run", run)
+
+ result = CliRunner().invoke(browser_app, ["extension-token"])
+
+ assert result.exit_code == 0
+ assert result.stdout.splitlines() == [
+ "Pairing token copied to clipboard.",
+ "Paste this token into the Skyvern browser extension popup.",
+ "Click Connect.",
+ ]
+ run.assert_called_once_with(
+ ["/usr/bin/pbcopy"],
+ input=token,
+ text=True,
+ check=True,
+ stdout=browser_commands.subprocess.DEVNULL,
+ stderr=browser_commands.subprocess.DEVNULL,
+ )
+
+
+def test_browser_extension_status_reports_configuration_without_printing_token(
+ monkeypatch: pytest.MonkeyPatch,
+ tmp_path: Path,
+) -> None:
+ token = "status-must-not-print-this-token"
+ token_dir = tmp_path / ".skyvern"
+ token_dir.mkdir()
+ token_path = token_dir / "browser_extension_token"
+ token_path.write_text(token)
+ token_path.chmod(0o600)
+ connection = MagicMock()
+ connect = MagicMock(return_value=connection)
+ monkeypatch.setenv("HOME", str(tmp_path))
+ monkeypatch.setenv("SKYVERN_BROWSER_EXTENSION_PORT", "20123")
+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_TOKEN", raising=False)
+ monkeypatch.setattr(browser_commands.socket, "create_connection", connect)
+
+ result = CliRunner().invoke(browser_app, ["extension-status"])
+
+ assert result.exit_code == 0
+ assert str(BrowserExtensionRuntime.extension_dir().resolve()) in result.stdout
+ assert "pairing token: configured (file exists)" in result.stdout
+ assert "pairing token file permissions: OK" in result.stdout
+ assert "bridge listening on 20123" in result.stdout
+ assert token not in result.stdout
+ connect.assert_called_once_with(("127.0.0.1", 20123), timeout=0.5)
+ connection.close.assert_called_once_with()
+
+
+def test_browser_extension_status_is_informational_when_bridge_is_not_running(
+ monkeypatch: pytest.MonkeyPatch,
+ tmp_path: Path,
+) -> None:
+ monkeypatch.setenv("HOME", str(tmp_path))
+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_TOKEN", raising=False)
+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_PORT", raising=False)
+ monkeypatch.setattr(browser_commands.socket, "create_connection", MagicMock(side_effect=OSError))
+
+ result = CliRunner().invoke(browser_app, ["extension-status"])
+
+ assert result.exit_code == 0
+ assert "pairing token: not configured" in result.stdout
+ assert "bridge not running (start your MCP server with --browser-extension)" in result.stdout
+
+
+def test_browser_extension_pair_exits_with_guidance_when_bridge_is_not_running(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ monkeypatch.setattr(browser_commands.socket, "create_connection", MagicMock(side_effect=OSError))
+
+ result = CliRunner().invoke(browser_app, ["extension-pair"])
+
+ assert result.exit_code == 1
+ assert "Start your MCP server first: skyvern run mcp --browser-extension" in result.stdout
+
+
+def test_browser_extension_pair_opens_fragment_url_without_printing_token(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ token = "pair-command-token-must-stay-private"
+ open_pairing_url = MagicMock(return_value=True)
+ monkeypatch.setattr(browser_commands.BrowserExtensionRuntime, "configured_port", lambda: 20123)
+ monkeypatch.setattr(browser_commands, "_bridge_is_listening", lambda _port: True)
+ monkeypatch.setattr(browser_commands, "load_or_create_pairing_token", lambda: token)
+ monkeypatch.setattr(browser_commands, "_request_pairing_nonce", lambda _port, _token: "pairing-nonce")
+ monkeypatch.setattr(browser_commands, "_open_pairing_url", open_pairing_url)
+
+ result = CliRunner().invoke(browser_app, ["extension-pair"])
+
+ assert result.exit_code == 0
+ assert result.stdout.strip() == "Approve the pairing in your browser."
+ assert token not in result.stdout
+ open_pairing_url.assert_called_once_with("http://127.0.0.1:20123/pair#pairing-nonce")
+
+
+def test_browser_extension_pair_begin_uses_hex_hmac_proof(monkeypatch: pytest.MonkeyPatch) -> None:
+ token = "pair-begin-proof-token"
+ response = SimpleNamespace(status=200, read=MagicMock(return_value=b'{"v":1,"nonce":"pairing-nonce"}'))
+ connection = MagicMock()
+ connection.getresponse.return_value = response
+ http_connection = MagicMock(return_value=connection)
+ monkeypatch.setattr(browser_commands.http.client, "HTTPConnection", http_connection)
+
+ nonce = browser_commands._request_pairing_nonce(20123, token)
+
+ assert nonce == "pairing-nonce"
+ http_connection.assert_called_once_with("127.0.0.1", 20123, timeout=2.0)
+ request = connection.request.call_args
+ payload = json.loads(request.kwargs["body"])
+ expected_proof = hmac.new(token.encode(), b"skyvern-pair-begin-v1", hashlib.sha256).hexdigest()
+ assert payload == {"v": 1, "proof": expected_proof}
+ connection.close.assert_called_once_with()
+
+
+def test_browser_extension_install_copies_token_opens_chrome_and_prints_steps(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ token = "install-token-must-stay-private"
+ extension_dir = BrowserExtensionRuntime.extension_dir().resolve()
+ run = MagicMock()
+ monkeypatch.setattr(browser_commands, "load_or_create_pairing_token", lambda: token)
+ monkeypatch.setattr(browser_commands.sys, "platform", "darwin")
+ monkeypatch.setattr(browser_commands.shutil, "which", lambda name: f"/usr/bin/{name}")
+ monkeypatch.setattr(browser_commands.subprocess, "run", run)
+ monkeypatch.setattr(browser_commands, "_bridge_is_listening", lambda _port: False)
+
+ result = CliRunner().invoke(browser_app, ["extension-install"])
+
+ assert result.exit_code == 0
+ assert result.stdout.splitlines()[0] == str(extension_dir)
+ assert "Pairing token copied to clipboard." in result.stdout
+ assert token not in result.stdout
+ assert "1. Enable Developer mode." in result.stdout
+ assert "2. Click Load unpacked." in result.stdout
+ assert f"3. Select {extension_dir}." in result.stdout
+ assert "4. Open the Skyvern Agent popup." in result.stdout
+ assert "5. Paste the pairing token and click Connect." in result.stdout
+ assert '6. Add tabs to the "Skyvern Controlled" group.' in result.stdout
+ assert "When your MCP server is running, pair with: skyvern browser extension-pair" in result.stdout
+ assert run.call_args_list == [
+ call(
+ ["/usr/bin/pbcopy"],
+ input=token,
+ text=True,
+ check=True,
+ stdout=browser_commands.subprocess.DEVNULL,
+ stderr=browser_commands.subprocess.DEVNULL,
+ ),
+ call(
+ ["/usr/bin/open", "-a", "Google Chrome", "chrome://extensions"],
+ check=True,
+ stdout=browser_commands.subprocess.DEVNULL,
+ stderr=browser_commands.subprocess.DEVNULL,
+ ),
+ ]
+
+
+def test_browser_extension_install_attempts_one_click_pairing_when_bridge_is_listening(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ launch_pairing = MagicMock()
+ load_token = MagicMock()
+ monkeypatch.setattr(browser_commands.BrowserExtensionRuntime, "configured_port", lambda: 20123)
+ monkeypatch.setattr(browser_commands, "_bridge_is_listening", lambda _port: True)
+ monkeypatch.setattr(browser_commands, "_open_chrome_extensions", lambda: False)
+ monkeypatch.setattr(browser_commands, "_launch_extension_pairing", launch_pairing)
+ monkeypatch.setattr(browser_commands, "load_or_create_pairing_token", load_token)
+
+ result = CliRunner().invoke(browser_app, ["extension-install"])
+
+ assert result.exit_code == 0
+ assert "4. Click Approve in the pairing page." in result.stdout
+ assert "5. Approve the pairing in the Skyvern Agent confirmation tab." in result.stdout
+ assert "pairing token" not in result.stdout.lower()
+ launch_pairing.assert_called_once_with(20123)
+ load_token.assert_not_called()
+
+
+def test_run_mcp_browser_extension_flag_starts_and_stops_runtime(monkeypatch: pytest.MonkeyPatch) -> None:
+ events: list[str] = []
+ runtime = SimpleNamespace(shutdown=AsyncMock(side_effect=lambda: events.append("shutdown")))
+
+ async def get_or_start() -> object:
+ events.append("start")
+ return runtime
+
+ async def run_async(**_kwargs: object) -> None:
+ assert run_commands.os.environ["BROWSER_TYPE"] == "extension-connect"
+ events.append("serve")
+
+ async def cleanup() -> None:
+ events.append("cleanup")
+
+ # delenv on an absent var records no undo, so run_mcp's os.environ.setdefault
+ # would leak into later tests; setenv first guarantees restoration.
+ monkeypatch.setenv("BROWSER_TYPE", "leak-guard-sentinel")
+ monkeypatch.delenv("BROWSER_TYPE", raising=False)
+ monkeypatch.setattr(run_commands, "prepare_cli_runtime", lambda **_kwargs: None)
+ monkeypatch.setattr(run_commands.atexit, "register", lambda _callback: None)
+ monkeypatch.setattr(run_commands.signal, "signal", lambda *_args: None)
+ monkeypatch.setattr(run_commands, "_start_stdin_eof_watcher", lambda: (MagicMock(), MagicMock()))
+ monkeypatch.setattr(run_commands, "_cleanup_mcp_resources", cleanup)
+ monkeypatch.setattr(
+ "skyvern.library.local_browser_profile.sweep_local_browser_profiles_with_budget",
+ lambda: None,
+ )
+ monkeypatch.setattr("skyvern.cli.mcp_tools.mcp.run_async", run_async)
+ monkeypatch.setattr(BrowserExtensionRuntime, "get_or_start", AsyncMock(side_effect=get_or_start))
+ monkeypatch.setattr(run_commands, "_mcp_cleanup_done", False)
+ monkeypatch.setattr(run_commands, "_mcp_cleanup_in_progress", False)
+
+ run_commands.run_mcp(browser_extension=True)
+
+ assert run_commands.os.environ["BROWSER_TYPE"] == "extension-connect"
+ assert events == ["start", "serve", "cleanup", "shutdown"]
+ runtime.shutdown.assert_awaited_once_with()
+
+
[email protected]("transport", ["sse", "streamable-http"])
+def test_run_mcp_rejects_browser_extension_with_http_transport(
+ monkeypatch: pytest.MonkeyPatch,
+ transport: str,
+) -> None:
+ monkeypatch.setenv("BROWSER_TYPE", "leak-guard-sentinel")
+ monkeypatch.delenv("BROWSER_TYPE", raising=False)
+
+ with pytest.raises(typer.BadParameter, match="--browser-extension requires --transport stdio"):
+ run_commands.run_mcp(transport=transport, browser_extension=True)
+
+ assert "BROWSER_TYPE" not in run_commands.os.environ
diff --git a/tests/unit/browser_extension/test_protocol.py b/tests/unit/browser_extension/test_protocol.py
new file mode 100644
index 000000000..e2c0bfc33
--- /dev/null
+++ b/tests/unit/browser_extension/test_protocol.py
@@ -0,0 +1,157 @@
+from __future__ import annotations
+
+import json
+
+import pytest
+
+from skyvern.browser_extension.errors import BrowserExtensionError
+from skyvern.browser_extension.protocol import (
+ ALLOWED_CDP_METHOD_PREFIXES,
+ ALLOWED_EVENTS,
+ ALLOWED_OPS,
+ DENIED_CDP_METHODS,
+ ERROR_CODES,
+ EXTENSION_ID,
+ PROTOCOL_VERSION,
+ build_request,
+ is_cdp_method_allowed,
+ is_restricted_url,
+ parse_extension_message,
+)
+
+
+def test_parse_valid_response() -> None:
+ parsed = parse_extension_message(
+ json.dumps({"v": 1, "type": "response", "id": "r-1", "ok": True, "result": {"tabId": 12}})
+ )
+
+ assert parsed.kind == "response"
+ assert parsed.request_id == "r-1"
+ assert parsed.ok is True
+ assert parsed.result == {"tabId": 12}
+ assert parsed.error_code is None
+
+
+def test_parse_valid_event() -> None:
+ parsed = parse_extension_message(
+ json.dumps(
+ {
+ "v": 1,
+ "type": "event",
+ "event": "scope.tabAdded",
+ "params": {"tabId": 12, "url": "https://example.com", "title": "Example"},
+ }
+ )
+ )
+
+ assert parsed.kind == "event"
+ assert parsed.event == "scope.tabAdded"
+ assert parsed.params == {"tabId": 12, "url": "https://example.com", "title": "Example"}
+
+
+def test_parse_valid_ping() -> None:
+ parsed = parse_extension_message('{"v":1,"type":"ping"}')
+
+ assert parsed.kind == "ping"
+
+
+def test_unknown_message_type_raises() -> None:
+ with pytest.raises(BrowserExtensionError):
+ parse_extension_message('{"v":1,"type":"auth.ok"}')
+
+
+def test_protocol_allowlists_match_contract() -> None:
+ assert PROTOCOL_VERSION == 1
+ assert EXTENSION_ID == "fmamdhmfeihjjaiheideemihnbpnokin"
+ assert ALLOWED_OPS == frozenset(
+ {
+ "debugger.attach",
+ "debugger.detach",
+ "debugger.send",
+ "tabs.create",
+ "tabs.remove",
+ "tabs.activate",
+ "tabs.list",
+ }
+ )
+ assert ALLOWED_EVENTS == frozenset(
+ {
+ "extension.hello",
+ "debugger.event",
+ "debugger.detached",
+ "scope.tabAdded",
+ "scope.tabRemoved",
+ "tabs.created",
+ }
+ )
+ assert ERROR_CODES == frozenset(
+ {
+ "AUTH_FAILED",
+ "OP_NOT_ALLOWED",
+ "TAB_NOT_FOUND",
+ "TAB_NOT_SCOPED",
+ "RESTRICTED_URL",
+ "DEBUGGER_DETACHED",
+ "CDP_METHOD_NOT_ALLOWED",
+ "CDP_ERROR",
+ "INTERNAL",
+ }
+ )
+
+
+def test_build_request_checks_operation_allowlist() -> None:
+ assert build_request("r-3", "tabs.list", {}) == {
+ "v": 1,
+ "type": "request",
+ "id": "r-3",
+ "op": "tabs.list",
+ "args": {},
+ }
+
+ with pytest.raises(BrowserExtensionError):
+ build_request("r-4", "cookies.read", {})
+ with pytest.raises(BrowserExtensionError):
+ build_request(4, "tabs.list", {}) # type: ignore[arg-type]
+
+
+def test_cdp_method_allowlist() -> None:
+ assert all(prefix.endswith(".") for prefix in ALLOWED_CDP_METHOD_PREFIXES)
+ assert is_cdp_method_allowed("Network.enable")
+ assert is_cdp_method_allowed("Storage.getUsageAndQuota")
+ assert is_cdp_method_allowed("Network.getCookies")
+ assert not is_cdp_method_allowed("Network.getCookies", {"urls": []})
+ assert not is_cdp_method_allowed("Browser.close")
+ assert not is_cdp_method_allowed("SystemInfo.getInfo")
+ assert not is_cdp_method_allowed("PageX.navigate")
+
+
+ "method",
+ [
+ "Network.getAllCookies",
+ "Network.clearBrowserCookies",
+ "Network.clearBrowserCache",
+ "Storage.getCookies",
+ "Storage.setCookies",
+ "Storage.clearCookies",
+ ],
+)
+def test_cdp_method_denylist(method: str) -> None:
+ assert method in DENIED_CDP_METHODS
+ assert not is_cdp_method_allowed(method)
+
+
+ ("url", "restricted"),
+ [
+ ("chrome://settings", True),
+ ("about:blank", False),
+ ("about:config", True),
+ ("https://chromewebstore.google.com/detail/example", True),
+ ("https://chromewebstore.google.com./detail/x", True),
+ ("https://example.com", False),
+ ("file:///tmp/example.html", True),
+ ],
+)
+def test_restricted_url_matrix(url: str, restricted: bool) -> None:
+ assert is_restricted_url(url) is restricted
diff --git a/tests/unit/browser_extension/test_relay.py b/tests/unit/browser_extension/test_relay.py
new file mode 100644
index 000000000..21a43dcb2
--- /dev/null
+++ b/tests/unit/browser_extension/test_relay.py
@@ -0,0 +1,457 @@
+from __future__ import annotations
+
+import asyncio
+import base64
+import hashlib
+import hmac
+import secrets
+from collections.abc import AsyncGenerator
+
+import pytest
+import pytest_asyncio
+from aiohttp import ClientSession, ClientWebSocketResponse, WSMsgType
+
+import skyvern.browser_extension.relay as relay_module
+from skyvern.browser_extension.auth import compute_ext_proof, compute_server_proof
+from skyvern.browser_extension.errors import BrowserExtensionNotConnectedError, ExtensionRequestError
+from skyvern.browser_extension.relay import ExtensionRelayServer
+
+TOKEN = "test-pairing-token"
+
+
+class RelayHarness:
+ def __init__(self) -> None:
+ self.events: list[tuple[str, dict]] = []
+ self.event_received = asyncio.Event()
+ self.disconnect_called = asyncio.Event()
+ self.server = ExtensionRelayServer(TOKEN, 0, self.on_event, self.on_disconnect)
+
+ async def on_event(self, event: str, params: dict) -> None:
+ self.events.append((event, params))
+ self.event_received.set()
+
+ async def on_disconnect(self) -> None:
+ self.disconnect_called.set()
+
+
+@pytest_asyncio.fixture
+async def relay_harness() -> AsyncGenerator[RelayHarness]:
+ harness = RelayHarness()
+ await harness.server.start()
+ yield harness
+ await harness.server.stop()
+
+
+def relay_url(harness: RelayHarness) -> str:
+ return f"ws://127.0.0.1:{harness.server.bound_port}/extension/v1"
+
+
+def http_url(harness: RelayHarness, path: str) -> str:
+ return f"http://127.0.0.1:{harness.server.bound_port}{path}"
+
+
+def pair_begin_proof(token: str) -> str:
+ return hmac.new(token.encode(), b"skyvern-pair-begin-v1", hashlib.sha256).hexdigest()
+
+
+async def authenticate(
+ session: ClientSession,
+ harness: RelayHarness,
+ *,
+ origin: str | None = "chrome-extension://abcdefghijklmnop",
+ send_hello: bool = True,
+) -> ClientWebSocketResponse:
+ headers = {"Origin": origin} if origin is not None else None
+ websocket = await session.ws_connect(relay_url(harness), headers=headers)
+ challenge = await websocket.receive_json()
+ client_nonce = secrets.token_urlsafe(32)
+ await websocket.send_json(
+ {
+ "v": 1,
+ "type": "auth.proof",
+ "clientNonce": client_nonce,
+ "proof": compute_ext_proof(TOKEN, challenge["serverNonce"], client_nonce),
+ }
+ )
+ auth_ok = await websocket.receive_json()
+ assert auth_ok == {
+ "v": 1,
+ "type": "auth.ok",
+ "serverProof": compute_server_proof(TOKEN, client_nonce, challenge["serverNonce"]),
+ }
+ if send_hello:
+ event_count = len(harness.events)
+ await websocket.send_json(
+ {
+ "v": 1,
+ "type": "event",
+ "event": "extension.hello",
+ "params": {"extensionVersion": "1.0.0", "scopedTabs": []},
+ }
+ )
+
+ async def hello_processed() -> None:
+ while len(harness.events) == event_count or not harness.server.connected:
+ await asyncio.sleep(0)
+
+ await asyncio.wait_for(hello_processed(), 1)
+ return websocket
+
+
+async def test_pair_begin_claim_happy_path_and_pair_page_never_contains_token(
+ relay_harness: RelayHarness,
+) -> None:
+ async with ClientSession() as session:
+ begin = await session.post(
+ http_url(relay_harness, "/pair/begin"),
+ json={"v": 1, "proof": pair_begin_proof(TOKEN)},
+ )
+ assert begin.status == 200
+ assert begin.headers["Cache-Control"] == "no-store"
+ begin_payload = await begin.json()
+ assert begin_payload["v"] == 1
+ assert isinstance(begin_payload["nonce"], str)
+ assert begin_payload["nonce"]
+
+ page = await session.get(http_url(relay_harness, "/pair"))
+ page_body = await page.text()
+ assert page.status == 200
+ assert TOKEN not in page_body
+ assert "fmamdhmfeihjjaiheideemihnbpnokin" in page_body
+
+ claim = await session.post(
+ http_url(relay_harness, "/pair/claim"),
+ json={"v": 1, "nonce": begin_payload["nonce"]},
+ )
+ assert claim.status == 200
+ assert claim.headers["Cache-Control"] == "no-store"
+ assert await claim.json() == {"v": 1, "port": relay_harness.server.bound_port, "token": TOKEN}
+
+
+async def test_pair_claim_wrong_nonce_is_forbidden_and_consumes_active_nonce(
+ relay_harness: RelayHarness,
+) -> None:
+ nonce = relay_harness.server.create_pairing_nonce()
+ async with ClientSession() as session:
+ wrong = await session.post(
+ http_url(relay_harness, "/pair/claim"),
+ json={"v": 1, "nonce": secrets.token_urlsafe(32)},
+ )
+ assert wrong.status == 403
+ assert await wrong.json() == {"error": "invalid_nonce"}
+
+ consumed = await session.post(
+ http_url(relay_harness, "/pair/claim"),
+ json={"v": 1, "nonce": nonce},
+ )
+ assert consumed.status == 403
+
+
+async def test_pair_claim_nonce_is_single_use(relay_harness: RelayHarness) -> None:
+ nonce = relay_harness.server.create_pairing_nonce()
+ async with ClientSession() as session:
+ first = await session.post(
+ http_url(relay_harness, "/pair/claim"),
+ json={"v": 1, "nonce": nonce},
+ )
+ second = await session.post(
+ http_url(relay_harness, "/pair/claim"),
+ json={"v": 1, "nonce": nonce},
+ )
+
+ assert first.status == 200
+ assert second.status == 403
+ assert await second.json() == {"error": "invalid_nonce"}
+
+
+async def test_pair_claim_expired_nonce_is_forbidden(
+ relay_harness: RelayHarness,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ now = 10_000.0
+ monkeypatch.setattr(relay_module.time, "monotonic", lambda: now)
+ nonce = relay_harness.server.create_pairing_nonce()
+ now += 121.0
+
+ async with ClientSession() as session:
+ response = await session.post(
+ http_url(relay_harness, "/pair/claim"),
+ json={"v": 1, "nonce": nonce},
+ )
+
+ assert response.status == 403
+ assert await response.json() == {"error": "invalid_nonce"}
+
+
+async def test_pair_begin_bad_proof_is_forbidden(relay_harness: RelayHarness) -> None:
+ async with ClientSession() as session:
+ response = await session.post(
+ http_url(relay_harness, "/pair/begin"),
+ json={"v": 1, "proof": "not-the-proof"},
+ )
+
+ assert response.status == 403
+
+
+async def test_authentication_and_hello_update_scoped_tabs(relay_harness: RelayHarness) -> None:
+ async with ClientSession() as session:
+ websocket = await authenticate(session, relay_harness, send_hello=False)
+
+ assert not relay_harness.server.connected
+ assert not await relay_harness.server.wait_connected(0.01)
+
+ params = {
+ "extensionVersion": "1.0.0",
+ "scopedTabs": [{"tabId": 17, "url": "https://example.com", "title": "Example"}],
+ }
+ await websocket.send_json({"v": 1, "type": "event", "event": "extension.hello", "params": params})
+ await asyncio.wait_for(relay_harness.event_received.wait(), 1)
+
+ assert relay_harness.server.connected
+ assert await relay_harness.server.wait_connected(0.1)
+ assert relay_harness.server.scoped_tabs == [{"tabId": 17, "url": "https://example.com", "title": "Example"}]
+ assert relay_harness.events == [("extension.hello", params)]
+
+
+async def test_wrong_proof_is_closed_with_4403(relay_harness: RelayHarness) -> None:
+ async with ClientSession() as session:
+ websocket = await session.ws_connect(relay_url(relay_harness))
+ challenge = await websocket.receive_json()
+ assert challenge["type"] == "auth.challenge"
+ await websocket.send_json(
+ {"v": 1, "type": "auth.proof", "clientNonce": secrets.token_urlsafe(32), "proof": "wrong-proof"}
+ )
+
+ message = await websocket.receive()
+
+ assert message.type in {WSMsgType.CLOSE, WSMsgType.CLOSED}
+ assert websocket.close_code == 4403
+ assert not relay_harness.server.connected
+
+
+ "client_nonce",
+ [
+ "short",
+ secrets.token_urlsafe(31),
+ secrets.token_urlsafe(32) + "=",
+ secrets.token_urlsafe(33),
+ "!" * 43,
+ base64.urlsafe_b64encode(b"\0" * 32).rstrip(b"=").decode("ascii")[:-1] + "B",
+ ],
+)
+async def test_client_nonce_must_be_unpadded_base64url_for_exactly_32_bytes(
+ relay_harness: RelayHarness,
+ client_nonce: str,
+) -> None:
+ async with ClientSession() as session:
+ websocket = await session.ws_connect(relay_url(relay_harness))
+ challenge = await websocket.receive_json()
+ await websocket.send_json(
+ {
+ "v": 1,
+ "type": "auth.proof",
+ "clientNonce": client_nonce,
+ "proof": compute_ext_proof(TOKEN, challenge["serverNonce"], client_nonce),
+ }
+ )
+
+ message = await websocket.receive()
+
+ assert message.type in {WSMsgType.CLOSE, WSMsgType.CLOSED}
+ assert websocket.close_code == 4403
+ assert not relay_harness.server.connected
+
+
+async def test_missing_proof_times_out_with_4403(
+ relay_harness: RelayHarness,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ monkeypatch.setattr(relay_module, "_AUTH_TIMEOUT_SECONDS", 0.01)
+ async with ClientSession() as session:
+ websocket = await session.ws_connect(relay_url(relay_harness))
+ challenge = await websocket.receive_json()
+ assert challenge["type"] == "auth.challenge"
+
+ message = await websocket.receive()
+
+ assert message.type in {WSMsgType.CLOSE, WSMsgType.CLOSED}
+ assert websocket.close_code == 4403
+
+
+async def test_origin_validation_rejects_web_origin_and_allows_missing_origin(relay_harness: RelayHarness) -> None:
+ async with ClientSession() as session:
+ rejected = await session.ws_connect(relay_url(relay_harness), headers={"Origin": "https://example.com"})
+
+ message = await rejected.receive()
+
+ assert message.type in {WSMsgType.CLOSE, WSMsgType.CLOSED}
+ assert rejected.close_code == 4403
+
+ allowed = await session.ws_connect(relay_url(relay_harness))
+ challenge = await allowed.receive_json()
+
+ assert challenge["type"] == "auth.challenge"
+ await allowed.close()
+
+
+async def test_request_response_error_and_timeout_paths(relay_harness: RelayHarness) -> None:
+ async with ClientSession() as session:
+ websocket = await authenticate(session, relay_harness)
+
+ success_task = asyncio.create_task(relay_harness.server.request("tabs.list", {}))
+ success_request = await websocket.receive_json()
+ assert success_request == {"v": 1, "type": "request", "id": "r-1", "op": "tabs.list", "args": {}}
+ await websocket.send_json(
+ {"v": 1, "type": "response", "id": success_request["id"], "ok": True, "result": {"tabs": []}}
+ )
+ assert await success_task == {"tabs": []}
+
+ error_task = asyncio.create_task(relay_harness.server.request("debugger.attach", {"tabId": 17}))
+ error_request = await websocket.receive_json()
+ await websocket.send_json(
+ {
+ "v": 1,
+ "type": "response",
+ "id": error_request["id"],
+ "ok": False,
+ "error": {"code": "TAB_NOT_SCOPED", "message": "tab is not shared"},
+ }
+ )
+ with pytest.raises(ExtensionRequestError) as error_info:
+ await error_task
+ assert error_info.value.code == "TAB_NOT_SCOPED"
+ assert error_info.value.message == "tab is not shared"
+
+ timeout_task = asyncio.create_task(relay_harness.server.request("tabs.list", {}, timeout=0.01))
+ timeout_request = await websocket.receive_json()
+ assert timeout_request["id"] == "r-3"
+ with pytest.raises(ExtensionRequestError) as timeout_info:
+ await timeout_task
+ assert timeout_info.value.code == "INTERNAL"
+ assert timeout_info.value.message == "extension request timed out: tabs.list"
+
+
+async def test_response_larger_than_default_aiohttp_limit_reaches_requester(relay_harness: RelayHarness) -> None:
+ async with ClientSession() as session:
+ websocket = await authenticate(session, relay_harness)
+ response_task = asyncio.create_task(relay_harness.server.request("debugger.send", {}))
+ request = await websocket.receive_json()
+ large_payload = "x" * (6 * 1024 * 1024)
+
+ await websocket.send_json(
+ {"v": 1, "type": "response", "id": request["id"], "ok": True, "result": {"data": large_payload}}
+ )
+
+ assert await response_task == {"data": large_payload}
+
+
+async def test_new_authenticated_connection_replaces_old_connection(relay_harness: RelayHarness) -> None:
+ async with ClientSession() as session:
+ first = await authenticate(session, relay_harness)
+ second_task = asyncio.create_task(authenticate(session, relay_harness))
+ first_message = await first.receive()
+ second = await second_task
+
+ assert first_message.type in {WSMsgType.CLOSE, WSMsgType.CLOSED}
+ assert first.close_code == 4000
+ assert relay_harness.server.connected
+ assert not second.closed
+ assert relay_harness.disconnect_called.is_set()
+
+
+async def test_scope_events_add_create_and_remove_scoped_tabs(relay_harness: RelayHarness) -> None:
+ async with ClientSession() as session:
+ websocket = await authenticate(session, relay_harness)
+ await websocket.send_json(
+ {
+ "v": 1,
+ "type": "event",
+ "event": "scope.tabAdded",
+ "params": {"tabId": 21, "url": "https://example.com/one", "title": "One"},
+ }
+ )
+ await websocket.send_json(
+ {
+ "v": 1,
+ "type": "event",
+ "event": "tabs.created",
+ "params": {"tabId": 22, "openerTabId": 21, "url": "https://example.com/two"},
+ }
+ )
+ await websocket.send_json(
+ {
+ "v": 1,
+ "type": "event",
+ "event": "scope.tabRemoved",
+ "params": {"tabId": 21, "reason": "unshared"},
+ }
+ )
+
+ async def all_events_received() -> None:
+ while len(relay_harness.events) < 4:
+ await asyncio.sleep(0)
+
+ await asyncio.wait_for(all_events_received(), 1)
+ assert relay_harness.server.scoped_tabs == [{"tabId": 22, "url": "https://example.com/two", "title": ""}]
+
+
+async def test_disconnect_fails_pending_requests_clears_tabs_and_calls_callback(
+ relay_harness: RelayHarness,
+) -> None:
+ async with ClientSession() as session:
+ websocket = await authenticate(session, relay_harness)
+ relay_harness.event_received.clear()
+ await websocket.send_json(
+ {
+ "v": 1,
+ "type": "event",
+ "event": "extension.hello",
+ "params": {
+ "extensionVersion": "1.0.0",
+ "scopedTabs": [{"tabId": 17, "url": "about:blank", "title": ""}],
+ },
+ }
+ )
+ await asyncio.wait_for(relay_harness.event_received.wait(), 1)
+
+ pending_request = asyncio.create_task(relay_harness.server.request("tabs.list", {}))
+ await websocket.receive_json()
+ await websocket.close()
+
+ with pytest.raises(BrowserExtensionNotConnectedError):
+ await pending_request
+ await asyncio.wait_for(relay_harness.disconnect_called.wait(), 1)
+ assert not relay_harness.server.connected
+ assert relay_harness.server.scoped_tabs == []
+
+
+async def test_extension_ping_receives_pong(relay_harness: RelayHarness) -> None:
+ async with ClientSession() as session:
+ websocket = await authenticate(session, relay_harness)
+
+ await websocket.send_json({"v": 1, "type": "ping"})
+
+ assert await websocket.receive_json() == {"v": 1, "type": "pong"}
+
+
+async def test_request_without_extension_fails_immediately(relay_harness: RelayHarness) -> None:
+ with pytest.raises(BrowserExtensionNotConnectedError):
+ await relay_harness.server.request("tabs.list", {})
diff --git a/tests/unit/browser_extension/test_runtime.py b/tests/unit/browser_extension/test_runtime.py
new file mode 100644
index 000000000..44170ced3
--- /dev/null
+++ b/tests/unit/browser_extension/test_runtime.py
@@ -0,0 +1,210 @@
+from __future__ import annotations
+
+import asyncio
+import errno
+from collections.abc import AsyncGenerator, Awaitable, Callable
+from pathlib import Path
+
+import pytest
+import pytest_asyncio
+
+import skyvern.browser_extension.runtime as runtime_module
+from skyvern.browser_extension.errors import BrowserExtensionError
+from skyvern.browser_extension.runtime import BrowserExtensionRuntime
+
+
+class StubRelay:
+ def __init__(
+ self,
+ token: str,
+ port: int,
+ on_event: Callable[[str, dict], Awaitable[None]],
+ on_disconnect: Callable[[], Awaitable[None]] | None = None,
+ *,
+ calls: list[str],
+ start_error: OSError | None = None,
+ ) -> None:
+ self.token = token
+ self.port = port
+ self.on_event = on_event
+ self.on_disconnect = on_disconnect
+ self.calls = calls
+ self.start_error = start_error
+ self.connected = True
+ self.stop_count = 0
+
+ async def start(self) -> None:
+ self.calls.append("relay.start")
+ if self.start_error is not None:
+ raise self.start_error
+
+ async def stop(self) -> None:
+ self.calls.append("relay.stop")
+ self.stop_count += 1
+
+ async def wait_connected(self, timeout: float) -> bool:
+ return self.connected
+
+
+class StubAdapter:
+ def __init__(self, registry, relay: StubRelay, *, calls: list[str]) -> None:
+ self.registry = registry
+ self.relay = relay
+ self.calls = calls
+ self.events: list[tuple[str, dict]] = []
+ self.disconnect_count = 0
+ self.stop_count = 0
+ self.cdp_ws_url = "ws://127.0.0.1:23456/cdp/test-capability"
+
+ async def start(self) -> None:
+ self.calls.append("adapter.start")
+
+ async def stop(self) -> None:
+ self.calls.append("adapter.stop")
+ self.stop_count += 1
+
+ async def handle_extension_event(self, event: str, params: dict) -> None:
+ self.events.append((event, params))
+
+ async def on_extension_disconnect(self) -> None:
+ self.disconnect_count += 1
+
+
+@pytest_asyncio.fixture(autouse=True)
+async def reset_runtime() -> AsyncGenerator[None]:
+ BrowserExtensionRuntime._instance = None
+ BrowserExtensionRuntime._lock = asyncio.Lock()
+ yield
+ instance = BrowserExtensionRuntime.instance()
+ if instance is not None:
+ await instance.shutdown()
+
+
+def install_stubs(
+ monkeypatch: pytest.MonkeyPatch,
+ *,
+ relay_start_error: OSError | None = None,
+) -> tuple[list[StubRelay], list[StubAdapter], list[str]]:
+ relays: list[StubRelay] = []
+ adapters: list[StubAdapter] = []
+ calls: list[str] = []
+
+ def relay_factory(token, port, on_event, on_disconnect) -> StubRelay:
+ relay = StubRelay(
+ token,
+ port,
+ on_event,
+ on_disconnect,
+ calls=calls,
+ start_error=relay_start_error,
+ )
+ relays.append(relay)
+ return relay
+
+ def adapter_factory(registry, relay) -> StubAdapter:
+ adapter = StubAdapter(registry, relay, calls=calls)
+ adapters.append(adapter)
+ return adapter
+
+ monkeypatch.setattr(runtime_module, "_relay_factory", relay_factory)
+ monkeypatch.setattr(runtime_module, "_adapter_factory", adapter_factory)
+ monkeypatch.setattr(runtime_module, "load_or_create_pairing_token", lambda: "runtime-test-token")
+ return relays, adapters, calls
+
+
+async def test_singleton_is_idempotent_and_late_binds_adapter_callbacks(monkeypatch: pytest.MonkeyPatch) -> None:
+ relays, adapters, calls = install_stubs(monkeypatch)
+
+ first = await BrowserExtensionRuntime.get_or_start(21001)
+ second = await BrowserExtensionRuntime.get_or_start(21002)
+
+ assert first is second
+ assert BrowserExtensionRuntime.instance() is first
+ assert len(relays) == len(adapters) == 1
+ assert relays[0].port == 21001
+ assert calls == ["adapter.start", "relay.start"]
+ assert first.cdp_ws_url == adapters[0].cdp_ws_url
+ assert first.extension_connected
+ assert await first.wait_for_extension(0.01)
+
+ event_params = {"tabId": 17}
+ await relays[0].on_event("scope.tabAdded", event_params)
+ assert adapters[0].events == [("scope.tabAdded", event_params)]
+ assert relays[0].on_disconnect is not None
+ await relays[0].on_disconnect()
+ assert adapters[0].disconnect_count == 1
+
+
+async def test_port_resolution_prefers_explicit_then_environment_then_default(monkeypatch: pytest.MonkeyPatch) -> None:
+ relays, _, _ = install_stubs(monkeypatch)
+ monkeypatch.setenv("SKYVERN_BROWSER_EXTENSION_PORT", "22001")
+
+ environment_runtime = await BrowserExtensionRuntime.get_or_start()
+ assert relays[-1].port == 22001
+ await environment_runtime.shutdown()
+
+ explicit_runtime = await BrowserExtensionRuntime.get_or_start(22002)
+ assert relays[-1].port == 22002
+ await explicit_runtime.shutdown()
+
+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_PORT")
+ default_runtime = await BrowserExtensionRuntime.get_or_start()
+ assert relays[-1].port == 19777
+ await default_runtime.shutdown()
+
+
+async def test_port_in_use_has_actionable_browser_extension_error(monkeypatch: pytest.MonkeyPatch) -> None:
+ _, adapters, _ = install_stubs(monkeypatch, relay_start_error=OSError(errno.EADDRINUSE, "address in use"))
+
+ with pytest.raises(BrowserExtensionError, match="SKYVERN_BROWSER_EXTENSION_PORT") as error_info:
+ await BrowserExtensionRuntime.get_or_start(23001)
+
+ assert "23001" in str(error_info.value)
+ assert adapters[0].stop_count == 1
+ assert BrowserExtensionRuntime.instance() is None
+
+
[email protected]("port_value", [0, "0"])
+async def test_zero_port_is_rejected(monkeypatch: pytest.MonkeyPatch, port_value: int | str) -> None:
+ install_stubs(monkeypatch)
+ if isinstance(port_value, str):
+ monkeypatch.setenv("SKYVERN_BROWSER_EXTENSION_PORT", port_value)
+ port = None
+ else:
+ port = port_value
+
+ with pytest.raises(BrowserExtensionError, match="between 1 and 65535"):
+ await BrowserExtensionRuntime.get_or_start(port)
+
+
+async def test_shutdown_stops_relay_before_adapter_is_idempotent_and_resets_singleton(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ relays, adapters, calls = install_stubs(monkeypatch)
+ runtime = await BrowserExtensionRuntime.get_or_start(24001)
+
+ await runtime.shutdown()
+ await runtime.shutdown()
+
+ assert calls == ["adapter.start", "relay.start", "relay.stop", "adapter.stop"]
+ assert relays[0].stop_count == adapters[0].stop_count == 1
+ assert BrowserExtensionRuntime.instance() is None
+
+ restarted = await BrowserExtensionRuntime.get_or_start(24002)
+ assert restarted is not runtime
+ assert relays[-1].port == 24002
+
+
+def test_extension_dir_points_to_packaged_manifest() -> None:
+ directory = BrowserExtensionRuntime.extension_dir()
+ expected_directory = Path(runtime_module.__file__).resolve().parent / "extension"
+
+ assert directory == expected_directory
+ if not (directory / "manifest.json").exists():
+ pytest.skip("extension manifest is owned by another build stream")
+ assert directory.is_dir()
diff --git a/tests/unit/browser_extension/test_target_registry.py b/tests/unit/browser_extension/test_target_registry.py
new file mode 100644
index 000000000..3f2272f7e
--- /dev/null
+++ b/tests/unit/browser_extension/test_target_registry.py
@@ -0,0 +1,168 @@
+from __future__ import annotations
+
+import pytest
+
+from skyvern.browser_extension.target_registry import VirtualTargetRegistry
+
+
+def test_register_list_and_update_tab_target_info() -> None:
+ registry = VirtualTargetRegistry()
+
+ target_id = registry.register_tab(42, "https://example.com/start", "Start")
+ root_session_id = registry.root_session_id(42)
+
+ assert target_id == "tab-42"
+ assert registry.root_session_id(42) == root_session_id
+ assert registry.tab_for_target(target_id) == 42
+ assert registry.target_info(target_id) == {
+ "targetId": "tab-42",
+ "type": "page",
+ "title": "Start",
+ "url": "https://example.com/start",
+ "attached": True,
+ "canAccessOpener": False,
+ "browserContextId": "skyvern-default",
+ }
+
+ registry.update_tab(42, "https://example.com/next", "Next")
+
+ assert registry.list_page_targets() == [
+ {
+ "targetId": "tab-42",
+ "type": "page",
+ "title": "Next",
+ "url": "https://example.com/next",
+ "attached": True,
+ "canAccessOpener": False,
+ "browserContextId": "skyvern-default",
+ }
+ ]
+
+
+def test_root_session_changes_after_remove_and_reregister() -> None:
+ registry = VirtualTargetRegistry()
+ registry.register_tab(7, "about:blank", "")
+ first_session_id = registry.root_session_id(7)
+
+ registry.remove_tab(7)
+ registry.register_tab(7, "about:blank", "")
+
+ assert registry.root_session_id(7) != first_session_id
+
+
+def test_root_session_aliases_are_unique_resolvable_and_independently_removable() -> None:
+ registry = VirtualTargetRegistry()
+ registry.register_tab(7, "about:blank", "")
+ primary_session_id = registry.root_session_id(7)
+
+ first_alias = registry.create_root_session_alias(7)
+ second_alias = registry.create_root_session_alias(7)
+
+ assert len({primary_session_id, first_alias, second_alias}) == 3
+ assert registry.root_session_ids(7) == [primary_session_id, first_alias, second_alias]
+ assert registry.resolve_session(first_alias) == (7, None)
+ assert registry.resolve_session(second_alias) == (7, None)
+
+ assert registry.remove_root_session_alias(first_alias)
+ assert not registry.remove_root_session_alias(primary_session_id)
+ assert registry.root_session_ids(7) == [primary_session_id, second_alias]
+ with pytest.raises(KeyError):
+ registry.resolve_session(first_alias)
+
+
+def test_browser_session_aliases_are_unique_and_independently_removable() -> None:
+ registry = VirtualTargetRegistry()
+
+ first_alias = registry.create_browser_session_alias()
+ second_alias = registry.create_browser_session_alias()
+
+ assert first_alias != second_alias
+ assert registry.is_browser_session_alias(first_alias)
+ assert registry.is_browser_session_alias(second_alias)
+ assert registry.remove_browser_session_alias(first_alias)
+ assert not registry.is_browser_session_alias(first_alias)
+ assert registry.is_browser_session_alias(second_alias)
+
+
+def test_child_session_register_resolve_and_remove() -> None:
+ registry = VirtualTargetRegistry()
+ registry.register_tab(9, "https://example.com", "Example")
+ child_target_info = {
+ "targetId": "frame-1",
+ "type": "iframe",
+ "title": "",
+ "url": "https://example.com/frame",
+ "attached": True,
+ "canAccessOpener": False,
+ "browserContextId": "skyvern-default",
+ }
+
+ registry.register_child_session(9, "chrome-child-1", child_target_info)
+
+ assert registry.resolve_session(registry.root_session_id(9)) == (9, None)
+ assert registry.resolve_session("chrome-child-1") == (9, "chrome-child-1")
+ assert registry.target_info("frame-1") == child_target_info
+
+ registry.remove_child_session("chrome-child-1")
+
+ with pytest.raises(KeyError):
+ registry.resolve_session("chrome-child-1")
+ with pytest.raises(KeyError):
+ registry.target_info("frame-1")
+
+
+def test_remove_tab_removes_root_and_child_sessions() -> None:
+ registry = VirtualTargetRegistry()
+ target_id = registry.register_tab(5, "https://example.com", "Example")
+ root_session_id = registry.root_session_id(5)
+ root_alias = registry.create_root_session_alias(5)
+ registry.register_child_session(5, "chrome-child", {"targetId": "frame-5"})
+
+ registry.remove_tab(5)
+
+ for session_id in (root_session_id, root_alias, "chrome-child"):
+ with pytest.raises(KeyError):
+ registry.resolve_session(session_id)
+ with pytest.raises(KeyError):
+ registry.target_info(target_id)
+ with pytest.raises(KeyError):
+ registry.tab_for_target(target_id)
+
+
+def test_unknown_registry_entries_raise_key_error() -> None:
+ registry = VirtualTargetRegistry()
+
+ with pytest.raises(KeyError):
+ registry.root_session_id(99)
+ with pytest.raises(KeyError):
+ registry.resolve_session("missing-session")
+ with pytest.raises(KeyError):
+ registry.target_info("missing-target")
+ with pytest.raises(KeyError):
+ registry.tab_for_target("missing-target")
+ with pytest.raises(KeyError):
+ registry.update_tab(99, "about:blank", "")
+ with pytest.raises(KeyError):
+ registry.register_child_session(99, "missing-child", {"targetId": "frame-missing"})
+
+
+def test_clear_removes_all_registry_state() -> None:
+ registry = VirtualTargetRegistry()
+ target_id = registry.register_tab(3, "https://example.com", "Example")
+ root_session_id = registry.root_session_id(3)
+ root_alias = registry.create_root_session_alias(3)
+ browser_alias = registry.create_browser_session_alias()
+ registry.register_child_session(3, "chrome-child", {"targetId": "frame-3"})
+
+ registry.clear()
+
+ assert registry.list_page_targets() == []
+ with pytest.raises(KeyError):
+ registry.resolve_session(root_session_id)
+ with pytest.raises(KeyError):
+ registry.resolve_session(root_alias)
+ with pytest.raises(KeyError):
+ registry.resolve_session("chrome-child")
+ with pytest.raises(KeyError):
+ registry.tab_for_target(target_id)
+ assert not registry.is_browser_session_alias(browser_alias)
diff --git a/tests/unit/test_mcp_browser_tools.py b/tests/unit/test_mcp_browser_tools.py
index 7bbcd3539..38a57ee29 100644
--- a/tests/unit/test_mcp_browser_tools.py
+++ b/tests/unit/test_mcp_browser_tools.py
@@ -24,11 +24,13 @@ from skyvern.cli.mcp_tools import cdp_input as mcp_cdp_input
from skyvern.cli.mcp_tools import mcp
from skyvern.client.errors import InternalServerError, UnprocessableEntityError
from tests.unit._mcp_browser_fakes import (
+ make_mock_page,
make_probe_locator,
make_real_wait_for_timeout,
make_select_like_page,
make_select_option_page,
make_skyvern_page,
+ patch_get_page,
)
@@ -205,11 +207,11 @@ async def test_skyvern_run_task_timeout_returns_timeout_code(monkeypatch: pytest
async def test_browser_tool_targeting_schema_prefers_direct_params() -> None:
tools_by_name = {tool.name: tool for tool in await mcp.list_tools()}
expectations = {
- "skyvern_click": (("selector",), ("intent",)),
+ "skyvern_click": (("selector", "x", "y"), ("intent",)),
"skyvern_drag": (("source_selector", "target_selector"), ("source_intent", "target_intent")),
"skyvern_file_upload": (("selector",), ("intent",)),
"skyvern_hover": (("selector",), ("intent",)),
- "skyvern_type": (("selector",), ("intent",)),
+ "skyvern_type": (("selector", "x", "y"), ("intent",)),
"skyvern_screenshot": (("selector",), ()),
"skyvern_scroll": (("selector",), ("intent",)),
"skyvern_select_option": (("selector",), ("intent",)),
@@ -297,6 +299,212 @@ def _direct_type_page(
return fill, locator
+async def test_skyvern_click_coordinates_dispatches_raw_mouse_click(monkeypatch: pytest.MonkeyPatch) -> None:
+ page = make_mock_page()
+ page.mouse.click = AsyncMock()
+ context = BrowserContext(mode="local")
+ patch_get_page(monkeypatch, mcp_browser, page, context)
+ resolve_ai_mode = Mock(side_effect=AssertionError("coordinate clicks must not resolve AI mode"))
+ select_native_option = AsyncMock(side_effect=AssertionError("coordinate clicks must not resolve selectors"))
+ monkeypatch.setattr(mcp_browser, "_resolve_ai_mode", resolve_ai_mode)
+ monkeypatch.setattr(mcp_browser, "select_native_option_if_targeted", select_native_option)
+
+ result = await mcp_browser.skyvern_click(x=12.5, y=24.0, button="right", click_count=2)
+
+ assert result["ok"] is True, result
+ page.mouse.click.assert_awaited_once_with(12.5, 24.0, button="right", click_count=2)
+ assert result["data"]["resolved_target"] == "coordinates (12.5, 24.0)"
+ assert result["data"]["sdk_equivalent"] == ("await page.mouse.click(12.5, 24.0, button='right', click_count=2)")
+ resolve_ai_mode.assert_not_called()
+ select_native_option.assert_not_awaited()
+
+
+async def test_skyvern_type_coordinates_focuses_then_types(monkeypatch: pytest.MonkeyPatch) -> None:
+ page = make_mock_page()
+ page.mouse.click = AsyncMock()
+ page.keyboard.type = AsyncMock()
+ page.keyboard.press = AsyncMock()
+ page.evaluate = AsyncMock(return_value=False)
+ context = BrowserContext(mode="local")
+ patch_get_page(monkeypatch, mcp_browser, page, context)
+
+ result = await mcp_browser.skyvern_type(text="Noor", x=18.0, y=36.5, clear=False)
+
+ assert result["ok"] is True, result
+ page.mouse.click.assert_awaited_once_with(18.0, 36.5)
+ page.keyboard.type.assert_awaited_once_with("Noor")
+ page.keyboard.press.assert_not_awaited()
+ assert result["data"]["resolved_target"] == "coordinates (18.0, 36.5)"
+
+
+async def test_skyvern_type_coordinates_refuses_password_input(monkeypatch: pytest.MonkeyPatch) -> None:
+ page = make_mock_page()
+ page.mouse.click = AsyncMock()
+ page.keyboard.type = AsyncMock()
+ page.keyboard.press = AsyncMock()
+ page.evaluate = AsyncMock(return_value=True)
+ context = BrowserContext(mode="local")
+ patch_get_page(monkeypatch, mcp_browser, page, context)
+
+ result = await mcp_browser.skyvern_type(text="not-a-secret", x=18.0, y=36.5, clear=False)
+
+ assert result["ok"] is False
+ assert result["error"]["code"] == mcp_browser.ErrorCode.INVALID_INPUT
+ assert "password" in result["error"]["message"].lower()
+ page.keyboard.type.assert_not_awaited()
+
+
+async def test_skyvern_type_coordinates_refuses_unverifiable_target(monkeypatch: pytest.MonkeyPatch) -> None:
+ page = make_mock_page()
+ page.mouse.click = AsyncMock()
+ page.keyboard.type = AsyncMock()
+ page.keyboard.press = AsyncMock()
+ page.evaluate = AsyncMock(return_value=None)
+ context = BrowserContext(mode="local")
+ patch_get_page(monkeypatch, mcp_browser, page, context)
+
+ result = await mcp_browser.skyvern_type(text="Noor", x=18.0, y=36.5, clear=False)
+
+ assert result["ok"] is False
+ assert result["error"]["code"] == mcp_browser.ErrorCode.INVALID_INPUT
+ assert result["error"]["message"] == browser_ops.COORDINATE_TYPE_TARGET_REFUSAL_MESSAGE
+ page.keyboard.type.assert_not_awaited()
+
+
+async def test_skyvern_click_requires_both_coordinates_before_session(monkeypatch: pytest.MonkeyPatch) -> None:
+ get_page = AsyncMock(side_effect=AssertionError("get_page should not be called for invalid coordinates"))
+ monkeypatch.setattr(mcp_browser, "get_page", get_page)
+
+ result = await mcp_browser.skyvern_click(x=12.5)
+
+ assert result["ok"] is False
+ assert result["error"]["code"] == mcp_browser.ErrorCode.INVALID_INPUT
+ assert "x" in result["error"]["message"] and "y" in result["error"]["message"]
+ get_page.assert_not_awaited()
+
+
+ ("x", "y"),
+ [
+ pytest.param(-1.0, 0.0, id="negative"),
+ pytest.param(float("inf"), 0.0, id="infinite"),
+ pytest.param(float("nan"), 0.0, id="nan"),
+ ],
+)
+async def test_skyvern_click_rejects_invalid_coordinates_before_session(
+ monkeypatch: pytest.MonkeyPatch,
+ x: float,
+ y: float,
+) -> None:
+ get_page = AsyncMock(side_effect=AssertionError("get_page should not be called for invalid coordinates"))
+ monkeypatch.setattr(mcp_browser, "get_page", get_page)
+
+ result = await mcp_browser.skyvern_click(x=x, y=y)
+
+ assert result["ok"] is False
+ assert result["error"]["code"] == mcp_browser.ErrorCode.INVALID_INPUT
+ get_page.assert_not_awaited()
+
+
+async def test_skyvern_type_rejects_selector_with_coordinates_before_session(monkeypatch: pytest.MonkeyPatch) -> None:
+ get_page = AsyncMock(side_effect=AssertionError("get_page should not be called for conflicting targets"))
+ monkeypatch.setattr(mcp_browser, "get_page", get_page)
+
+ result = await mcp_browser.skyvern_type(text="Noor", selector="#first_name", x=18.0, y=36.5)
+
+ assert result["ok"] is False
+ assert result["error"]["code"] == mcp_browser.ErrorCode.INVALID_INPUT
+ assert "selector" in result["error"]["message"].lower()
+ assert "coordinates" in result["error"]["message"].lower()
+ get_page.assert_not_awaited()
+
+
+async def test_skyvern_execute_click_coordinates_flow_through_dispatch(monkeypatch: pytest.MonkeyPatch) -> None:
+ page = make_mock_page()
+ page.mouse.click = AsyncMock()
+ context = BrowserContext(mode="local")
+ get_page = patch_get_page(monkeypatch, mcp_browser, page, context)
+ get_page.return_value[0]._working_frame = None
+
+ result = await mcp_browser.skyvern_execute(
+ steps=[{"tool": "click", "params": {"x": 12.5, "y": 24.0, "button": "middle"}}]
+ )
+
+ assert result["ok"] is True, result
+ assert result["data"]["steps_completed"] == 1
+ page.mouse.click.assert_awaited_once_with(12.5, 24.0, button="middle", click_count=1)
+
+
+async def test_skyvern_execute_type_coordinates_flow_through_dispatch(monkeypatch: pytest.MonkeyPatch) -> None:
+ page = make_mock_page()
+ page.mouse.click = AsyncMock()
+ page.keyboard.type = AsyncMock()
+ page.keyboard.press = AsyncMock()
+ page.evaluate = AsyncMock(return_value=False)
+ context = BrowserContext(mode="local")
+ get_page = patch_get_page(monkeypatch, mcp_browser, page, context)
+ get_page.return_value[0]._working_frame = None
+
+ result = await mcp_browser.skyvern_execute(
+ steps=[
+ {
+ "tool": "type",
+ "params": {
+ "text": "Noor",
+ "x": 18.0,
+ "y": 36.5,
+ "clear_first": False,
+ "press_enter": True,
+ },
+ }
+ ]
+ )
+
+ assert result["ok"] is True, result
+ page.mouse.click.assert_awaited_once_with(18.0, 36.5)
+ page.keyboard.type.assert_awaited_once_with("Noor")
+ page.keyboard.press.assert_awaited_once_with("Enter")
+
+
+async def test_coordinate_actions_record_explicit_trajectory_coordinates(monkeypatch: pytest.MonkeyPatch) -> None:
+ page = make_mock_page()
+ page.mouse.click = AsyncMock()
+ page.keyboard.type = AsyncMock()
+ page.keyboard.press = AsyncMock()
+ page.evaluate = AsyncMock(return_value=False)
+ context = BrowserContext(mode="cloud_session", session_id="pbs_test")
+ patch_get_page(monkeypatch, mcp_browser, page, context)
+ append_trajectory = Mock()
+ monkeypatch.setattr(mcp_browser, "append_trajectory_entry", append_trajectory)
+ monkeypatch.setattr(mcp_browser, "current_api_key_hash", lambda: None)
+
+ click_result = await mcp_browser.skyvern_click(x=12.5, y=24.0)
+ type_result = await mcp_browser.skyvern_type(text="Noor", x=18.0, y=36.5)
+
+ assert click_result["ok"] is True, click_result
+ assert type_result["ok"] is True, type_result
+ click_entry, type_entry = (entry.kwargs["entry"] for entry in append_trajectory.call_args_list)
+ assert "selector" not in click_entry
+ assert (click_entry["x"], click_entry["y"]) == (12.5, 24.0)
+ assert click_entry["sdk_equivalent"] == click_result["data"]["sdk_equivalent"]
+ assert "locator" not in click_entry["sdk_equivalent"]
+ assert "selector" not in type_entry
+ assert (type_entry["x"], type_entry["y"]) == (18.0, 36.5)
+ assert type_entry["sdk_equivalent"] == type_result["data"]["sdk_equivalent"]
+ assert "locator" not in type_entry["sdk_equivalent"]
+ assert "coordinates (" not in repr([click_entry, type_entry])
+
+
# Default (resilient) keeps the shared MCP surface unchanged for external callers: a selector
# tries first, then dismisses overlays and AI-falls-back. selector_mode="direct" (which the
# copilot binds via forced_args) opts into deterministic, no-AI behavior.