Automate browser based workflows with AI
Python aiapiautomationbrowserbrowser-automationcomputergptllmplaywrightpowerautomatepuppeteerpythonrpaseleniumvisionworkflow
Control your own browser from Skyvern MCP via a Chrome extension bridge (SKY-13370) (#8070)
Suchintan37e5a145b947
diff --git a/docs/developers/optimization/chrome-extension.mdx b/docs/developers/optimization/chrome-extension.mdxnew file mode 100644index 000000000..d68d74b97--- /dev/null+++ b/docs/developers/optimization/chrome-extension.mdx@@ -0,0 +1,215 @@+---+title: Control Your Chrome with Skyvern Agent+subtitle: Let local Skyvern MCP use your real Chrome profile, tabs, and logged-in sessions+description: Use the Skyvern Agent Chrome extension to automate explicitly shared tabs from a local MCP server.+slug: developers/optimization/chrome-extension+keywords:+ - Chrome extension+ - MCP+ - local browser+ - browser session+ - authentication+ - cookies+ - Playwright+---++Want Skyvern MCP to automate sites in the Chrome browser you already use? The **Skyvern Agent** Chrome extension lets a local+MCP server drive your real Chrome profile, including its logged-in sessions and cookies. Unlike+`skyvern browser serve`, which starts a separate Chrome browser with a copied profile, the extension controls+explicitly shared tabs in your actual browser. Chrome 136 and later ignores `--remote-debugging-port` for the default+profile, so an extension is the only way to automate that browser directly.++---++## Requirements++- Chrome 125 or later+- A local Skyvern MCP server connected over stdio++Hosted MCP is not supported yet.++---++## Install the extension++The recommended setup command prints the unpacked extension directory, opens Chrome's extensions page when possible,+and prints the remaining steps:++```bash+skyvern browser extension-install+```++Then enable **Developer mode**, click **Load unpacked**, and select the printed directory. If your MCP server is already+running with the extension bridge, the command also opens the one-click pairing page. Otherwise, start the MCP server+and run `skyvern browser extension-pair`.++For manual setup:++1. Print the directory containing the extension:++ ```bash+ skyvern browser extension-path+ ```++2. Open `chrome://extensions` in Chrome.+3. Enable **Developer mode**.+4. Click **Load unpacked**.+5. Select the directory printed by the command.++---++## Pair the extension++With your MCP server running, start the recommended one-click flow:++```bash+skyvern browser extension-pair+```++Click **Approve** in the pairing page, then approve the pairing again in the **Skyvern Agent** confirmation tab that+opens. The pairing link expires after two minutes and can be used only once; rerun the command if it expires.++As a manual fallback, copy the pairing token to your clipboard:++```bash+skyvern browser extension-token+```++Then open the **Skyvern Agent** popup, paste the token, and click **Connect**.++Treat the pairing token like a password. The one-click URL contains only a short-lived nonce in its fragment; Skyvern+does not put the pairing token in a URL or expose it through MCP tools.++---++## Configure MCP++Set `BROWSER_TYPE=extension-connect` in the environment for your local MCP server. For example, use this stdio+configuration in an MCP client that accepts JSON:++```json+{+ "mcpServers": {+ "skyvern": {+ "command": "skyvern",+ "args": ["run", "mcp"],+ "env": {+ "BROWSER_TYPE": "extension-connect"+ }+ }+ }+}+```++Alternatively, start the local MCP server with the browser extension bridge enabled:++```bash+skyvern run mcp --browser-extension+```++The environment setting starts the bridge lazily when `skyvern_browser_session_create` needs it. The flag starts the+bridge eagerly with the MCP server.++Check the local setup without exposing the pairing token:++```bash+skyvern browser extension-status+```++The command reports the extension directory, whether token configuration and file permissions are present, and whether+the configured loopback port is accepting connections. The default port is `19777`; a custom port is configured with+`SKYVERN_BROWSER_EXTENSION_PORT` for the MCP server and under **Advanced settings** in the popup.++---++## Use your browser++After the extension is connected, a typical MCP flow is:++1. Call `skyvern_browser_session_create` to connect Skyvern to the extension.+2. Call `skyvern_navigate` to open a page.+3. Call `skyvern_observe` to inspect the page and identify actions.+4. Call `skyvern_execute` or `skyvern_click` to interact with the page.++Selectors remain the primary way to target elements. `skyvern_click` and `skyvern_type` also accept `x` and `y`+coordinates in viewport CSS pixels, measured from the top-left of the web content. Provide both coordinates together+and do not combine them with a selector. Coordinates must be finite and non-negative. `skyvern_execute` can pass the+same coordinates to click and type actions. Coordinates align with `skyvern_screenshot(full_page=False)` and are not+scaled by the device pixel ratio.++---++## Consent and security++Membership in the Chrome tab group named **Skyvern Controlled** is the consent boundary. Dragging a tab into that group+shares it with Skyvern; dragging it out immediately revokes access. The **Add to Skyvern Controlled** and+**Remove from Skyvern Controlled** buttons in the **Skyvern Agent** popup make the same group-membership changes. The+extension never discloses tabs outside the group to Skyvern.++Skyvern's scope includes:++- Existing tabs you explicitly add to **Skyvern Controlled**, by dragging or with the popup+- Tabs Skyvern creates and adds to **Skyvern Controlled**+- Popups opened by a controlled tab and added to **Skyvern Controlled**++When Skyvern attaches to a controlled tab, Chrome displays its debugger infobar; Skyvern never hides it. Clicking+**Cancel** in that infobar immediately revokes access to the tab, and Skyvern does not automatically reattach. Skyvern+also never closes or restarts your Chrome browser.++The extension connects outbound to a bridge bound only to `127.0.0.1`. The default endpoint is+`ws://127.0.0.1:19777/extension/v1`. Pairing-token authentication verifies both the extension and the local MCP bridge,+and the token is never included in the URL.++The extension refuses to control `chrome://`, `chrome-untrusted://`, `chrome-extension://`, `devtools://`, `edge://`,+`file://`, Chrome Web Store pages on `chromewebstore.google.com`, and `about:` pages other than `about:blank`.++---++## Limitations++- Downloads and file-chooser management are not supported yet.+- Incognito windows are not supported.+- Opening DevTools for a controlled tab detaches Skyvern and revokes access to that tab.+- Only one local Skyvern MCP process can control the extension at a time.+- Hosted Skyvern MCP cannot use the extension yet.+- Chrome internal pages, extension pages, DevTools, local files, the Chrome Web Store, and the other restricted targets+ listed above cannot be controlled.++---++## Troubleshooting++<AccordionGroup>+ <Accordion title="Skyvern browser extension is not connected">+ Follow the steps in order:++ - Run `skyvern browser extension-install` and follow its numbered setup instructions.+ - Start the MCP server with `--browser-extension` or `BROWSER_TYPE=extension-connect`.+ - Run `skyvern browser extension-status` and confirm it reports the bridge listening on the configured port.+ - Run `skyvern browser extension-pair`, approve the browser page, then approve the Skyvern Agent confirmation tab.+ - If one-click pairing fails, open the **Skyvern Agent** popup and use `skyvern browser extension-token` as the+ manual paste fallback.+ - Add at least one controllable tab to **Skyvern Controlled**, then retry.+ </Accordion>++ <Accordion title="Port already in use">+ The bridge uses port `19777` by default. Set `SKYVERN_BROWSER_EXTENSION_PORT` in the MCP server environment,+ configure the same port under **Advanced settings** in the popup, then restart the MCP server and reconnect the+ extension.+ </Accordion>++ <Accordion title="Rotate the pairing token">+ Stop the MCP server, delete `~/.skyvern/browser_extension_token`, and restart the server. Run+ `skyvern browser extension-pair` and approve both browser steps. Use `skyvern browser extension-token` and paste the+ new token into the **Skyvern Agent** popup only as a manual fallback.++ If `SKYVERN_BROWSER_EXTENSION_TOKEN` is set in the MCP environment, remove it before rotating the token file+ because the environment value takes precedence.+ </Accordion>++ <Accordion title="DevTools or the debugger infobar disconnected a tab">+ Close DevTools if it is open, then add the tab to **Skyvern Controlled** again by dragging it into the group or using+ the popup. If you clicked **Cancel** in Chrome's debugger infobar, adding the tab again restores consent; Skyvern+ never reattaches automatically.+ </Accordion>+</AccordionGroup>diff --git a/docs/docs.json b/docs/docs.jsonindex e435b6da2..7be5d45bd 100644--- a/docs/docs.json+++ b/docs/docs.json@@ -181,6 +181,8 @@"pages": ["developers/optimization/browser-sessions","developers/optimization/browser-profiles",+ "developers/optimization/browser-tunneling",+ "developers/optimization/chrome-extension","developers/optimization/cost-control"]},diff --git a/docs/snippets/mcp-content.mdx b/docs/snippets/mcp-content.mdxindex 71b3808d4..23115abd5 100644--- a/docs/snippets/mcp-content.mdx+++ b/docs/snippets/mcp-content.mdx@@ -529,6 +529,10 @@ Replace `/usr/bin/python3` with the output of `which python3`. For Skyvern CloudYou can point your AI assistant at a browser running on your machine instead of a cloud browser. This lets you automate internal tools on localhost, reuse existing login sessions, and verify local dev changes, all through the same MCP tools.+To drive the Chrome profile and tabs you already use from a local stdio MCP server, install the+[Skyvern Chrome extension](/developers/optimization/chrome-extension). You explicitly choose which tabs to share, and+the connection stays on your machine.+Start a local browser with a tunnel so Skyvern Cloud can reach it:```bashdiff --git a/skyvern/browser_extension/__init__.py b/skyvern/browser_extension/__init__.pynew file mode 100644index 000000000..9d74c5036--- /dev/null+++ b/skyvern/browser_extension/__init__.py@@ -0,0 +1,49 @@+from skyvern.browser_extension.auth import (+ build_challenge,+ compute_ext_proof,+ compute_server_proof,+ load_or_create_pairing_token,+ verify_ext_proof,+)+from skyvern.browser_extension.errors import (+ BrowserExtensionError,+ BrowserExtensionNotConnectedError,+ ExtensionRequestError,+)+from skyvern.browser_extension.protocol import (+ ALLOWED_CDP_METHOD_PREFIXES,+ ALLOWED_EVENTS,+ ALLOWED_OPS,+ ERROR_CODES,+ PROTOCOL_VERSION,+ RESTRICTED_URL_PREFIXES,+ ParsedMessage,+ build_request,+ is_cdp_method_allowed,+ is_restricted_url,+ parse_extension_message,+)+from skyvern.browser_extension.target_registry import VirtualTargetRegistry++__all__ = [+ "ALLOWED_CDP_METHOD_PREFIXES",+ "ALLOWED_EVENTS",+ "ALLOWED_OPS",+ "ERROR_CODES",+ "PROTOCOL_VERSION",+ "RESTRICTED_URL_PREFIXES",+ "BrowserExtensionError",+ "BrowserExtensionNotConnectedError",+ "ExtensionRequestError",+ "ParsedMessage",+ "VirtualTargetRegistry",+ "build_challenge",+ "build_request",+ "compute_ext_proof",+ "compute_server_proof",+ "is_cdp_method_allowed",+ "is_restricted_url",+ "load_or_create_pairing_token",+ "parse_extension_message",+ "verify_ext_proof",+]diff --git a/skyvern/browser_extension/auth.py b/skyvern/browser_extension/auth.pynew file mode 100644index 000000000..6d648f790--- /dev/null+++ b/skyvern/browser_extension/auth.py@@ -0,0 +1,134 @@+from __future__ import annotations++import base64+import hashlib+import hmac+import json+import os+import secrets+import stat+import tempfile+from pathlib import Path++from skyvern.browser_extension.errors import BrowserExtensionError+from skyvern.browser_extension.protocol import PROTOCOL_VERSION++_TOKEN_ENV = "SKYVERN_BROWSER_EXTENSION_TOKEN"+++def load_or_create_pairing_token() -> str:+ environment_token = os.environ.get(_TOKEN_ENV)+ if environment_token is not None:+ environment_token = environment_token.strip()+ if environment_token:+ return environment_token++ token_dir = Path.home() / ".skyvern"+ token_path = token_dir / "browser_extension_token"+ existing_token = _read_token_file(token_path)+ if existing_token:+ return existing_token+ if existing_token == "":+ token_path.unlink()++ token_dir_existed = token_dir.exists()+ token_dir.mkdir(parents=True, exist_ok=True, mode=0o700)+ if not token_dir_existed:+ token_dir.chmod(0o700)++ token = secrets.token_urlsafe(32)+ return _publish_token(token_path, token)+++def _read_token_file(token_path: Path) -> str | None:+ flags = os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0)+ try:+ file_descriptor = os.open(token_path, flags)+ except FileNotFoundError:+ return None+ except OSError as exc:+ raise BrowserExtensionError("Pairing token path must be a regular file owned by the current user") from exc++ try:+ file_stat = os.fstat(file_descriptor)+ if not stat.S_ISREG(file_stat.st_mode):+ raise BrowserExtensionError("Pairing token path must be a regular file")+ if hasattr(os, "getuid") and file_stat.st_uid != os.getuid():+ raise BrowserExtensionError("Pairing token file must be owned by the current user")+ if stat.S_IMODE(file_stat.st_mode) & 0o177:+ os.fchmod(file_descriptor, 0o600)+ with os.fdopen(file_descriptor, "r", encoding="utf-8") as token_file:+ file_descriptor = -1+ return token_file.read().strip()+ finally:+ if file_descriptor >= 0:+ os.close(file_descriptor)+++def _publish_token(token_path: Path, token: str) -> str:+ file_descriptor, temporary_name = tempfile.mkstemp(+ dir=token_path.parent,+ prefix=f".{token_path.name}.",+ text=True,+ )+ temporary_path = Path(temporary_name)+ try:+ os.fchmod(file_descriptor, 0o600)+ with os.fdopen(file_descriptor, "w", encoding="utf-8") as token_file:+ token_file.write(token)+ token_file.flush()+ os.fsync(token_file.fileno())+ try:+ os.link(temporary_path, token_path)+ except FileExistsError:+ try:+ existing_token = _read_token_file(token_path)+ except BrowserExtensionError:+ existing_token = None+ if existing_token:+ return existing_token+ try:+ token_path.unlink(missing_ok=True)+ os.link(temporary_path, token_path)+ except OSError as exc:+ raise BrowserExtensionError("Failed to publish a valid pairing token after a creation race") from exc+ except OSError as exc:+ raise BrowserExtensionError("Failed to publish pairing token") from exc+ return token+ finally:+ temporary_path.unlink(missing_ok=True)+++def build_challenge() -> tuple[str, str]:+ server_nonce = _b64url(secrets.token_bytes(32))+ challenge = json.dumps(+ {"v": PROTOCOL_VERSION, "type": "auth.challenge", "serverNonce": server_nonce}, separators=(",", ":")+ )+ return server_nonce, challenge+++def compute_ext_proof(token: str, server_nonce: str, client_nonce: str) -> str:+ message = f"skyvern-ext-v1|{server_nonce}|{client_nonce}"+ return _compute_proof(token, message)+++def compute_server_proof(token: str, client_nonce: str, server_nonce: str) -> str:+ message = f"skyvern-srv-v1|{client_nonce}|{server_nonce}"+ return _compute_proof(token, message)+++def verify_ext_proof(token: str, server_nonce: str, client_nonce: str, proof: str) -> bool:+ expected_proof = compute_ext_proof(token, server_nonce, client_nonce)+ try:+ return hmac.compare_digest(expected_proof, proof)+ except TypeError:+ return False+++def _compute_proof(token: str, message: str) -> str:+ digest = hmac.new(token.encode("utf-8"), message.encode("utf-8"), hashlib.sha256).digest()+ return _b64url(digest)+++def _b64url(value: bytes) -> str:+ return base64.urlsafe_b64encode(value).rstrip(b"=").decode("ascii")diff --git a/skyvern/browser_extension/cdp_adapter.py b/skyvern/browser_extension/cdp_adapter.pynew file mode 100644index 000000000..5ab18a801--- /dev/null+++ b/skyvern/browser_extension/cdp_adapter.py@@ -0,0 +1,883 @@+from __future__ import annotations++import asyncio+import json+import secrets+from collections.abc import Coroutine+from typing import TYPE_CHECKING++import structlog+from aiohttp import WSMsgType, web++from skyvern.browser_extension.errors import BrowserExtensionError, ExtensionRequestError+from skyvern.browser_extension.protocol import is_cdp_method_allowed+from skyvern.browser_extension.relay import _MAX_WS_MESSAGE_BYTES+from skyvern.browser_extension.target_registry import VirtualTargetRegistry++if TYPE_CHECKING:+ from skyvern.browser_extension.relay import ExtensionRelayServer+++LOG = structlog.get_logger()++_VERSION_RESULT = {+ "protocolVersion": "1.3",+ "product": "Chrome/999.0.0.0",+ "revision": "",+ "userAgent": "Skyvern-Extension-Bridge",+ "jsVersion": "",+}+_WINDOW_BOUNDS = {"left": 0, "top": 0, "width": 1280, "height": 720, "windowState": "normal"}+_BROWSER_TARGET_INFO = {+ "targetId": "skyvern-browser",+ "type": "browser",+ "title": "",+ "url": "",+ "attached": True,+ "canAccessOpener": False,+}+_CHILD_AUTO_ATTACH_PARAMS = {+ "flatten": True,+ "autoAttach": True,+ "waitForDebuggerOnStart": False,+}+++class ExtensionCdpAdapter:+ def __init__(self, registry: VirtualTargetRegistry, relay: ExtensionRelayServer) -> None:+ self._registry = registry+ self._relay = relay+ self._capability = secrets.token_urlsafe(32)+ self._runner: web.AppRunner | None = None+ self._site: web.TCPSite | None = None+ self._port: int | None = None+ self._client_ws: web.WebSocketResponse | None = None+ self._client_guard = asyncio.Lock()+ self._send_lock = asyncio.Lock()+ self._auto_attach = False+ self._discover_targets = False+ self._attached_tabs: set[int] = set()+ self._attach_locks: dict[int, asyncio.Lock] = {}+ self._opener_ids: dict[int, str] = {}+ self._scope_generations: dict[int, int] = {}+ self._scope_tombstones: set[int] = set()+ self._background_tasks: set[asyncio.Task[None]] = set()++ async def start(self) -> None:+ if self._runner is not None:+ return+ app = web.Application()+ app.router.add_get(f"/cdp/{self._capability}", self._handle_websocket)+ runner = web.AppRunner(app)+ await runner.setup()+ site = web.TCPSite(runner, "127.0.0.1", 0)+ await site.start()+ addresses = runner.addresses+ if not addresses:+ await runner.cleanup()+ raise RuntimeError("CDP adapter failed to bind")+ self._runner = runner+ self._site = site+ self._port = int(addresses[0][1])++ async def stop(self) -> None:+ async with self._client_guard:+ ws = self._client_ws+ self._client_ws = None+ self._reset_connection_state()+ if ws is not None and not ws.closed:+ await ws.close(code=1001, message=b"adapter stopped")+ if self._runner is not None:+ await self._runner.cleanup()+ self._runner = None+ self._site = None+ self._port = None++ @property+ def cdp_ws_url(self) -> str:+ if self._port is None:+ raise RuntimeError("CDP adapter is not started")+ return f"ws://127.0.0.1:{self._port}/cdp/{self._capability}"++ async def handle_extension_event(self, event: str, params: dict) -> None:+ if event == "debugger.event":+ await self._handle_debugger_event(params)+ elif event == "debugger.detached":+ await self._remove_tab_with_events(params.get("tabId"))+ elif event == "scope.tabRemoved":+ await self._remove_tab_with_events(params.get("tabId"))+ elif event in {"scope.tabAdded", "tabs.created"}:+ tab_id = params.get("tabId")+ if type(tab_id) is not int:+ return+ if event == "scope.tabAdded":+ try:+ self._registry.target_id_for_tab(tab_id)+ except KeyError:+ pass+ else:+ return+ generation = self._begin_tab_scope(tab_id)+ if self._auto_attach:+ self._spawn(+ self._handle_tab_added(+ params,+ include_opener=event == "tabs.created",+ generation=generation,+ )+ )+ await asyncio.sleep(0)+ else:+ await self._handle_tab_added(+ params,+ include_opener=event == "tabs.created",+ generation=generation,+ )+ elif event == "extension.hello":+ scoped_tabs = params.get("scopedTabs")+ if isinstance(scoped_tabs, list):+ tabs = [+ (tab, self._begin_tab_scope(tab["tabId"]))+ for tab in scoped_tabs+ if isinstance(tab, dict) and type(tab.get("tabId")) is int+ ]+ if self._auto_attach:+ self._spawn(self._handle_hello_tabs(tabs))+ await asyncio.sleep(0)+ else:+ await self._handle_hello_tabs(tabs)++ async def on_extension_disconnect(self) -> None:+ async with self._client_guard:+ ws = self._client_ws+ self._client_ws = None+ self._reset_connection_state()+ if ws is not None and not ws.closed:+ await ws.close(code=1001, message=b"extension disconnected")++ async def _handle_websocket(self, request: web.Request) -> web.WebSocketResponse:+ ws = web.WebSocketResponse(max_msg_size=_MAX_WS_MESSAGE_BYTES)+ await ws.prepare(request)+ async with self._client_guard:+ if self._client_ws is not None:+ rejected = True+ else:+ rejected = False+ self._client_ws = ws+ if rejected:+ await ws.close(code=4409, message=b"CDP client already connected")+ return ws++ try:+ async for message in ws:+ if message.type == WSMsgType.TEXT:+ await self._handle_client_text(ws, message.data)+ elif message.type == WSMsgType.ERROR:+ break+ elif message.type == WSMsgType.BINARY:+ await ws.close(code=1003, message=b"text frames required")+ break+ finally:+ async with self._client_guard:+ if self._client_ws is ws:+ self._client_ws = None+ self._reset_connection_state()+ return ws++ async def _handle_client_text(self, ws: web.WebSocketResponse, raw: str) -> None:+ try:+ payload = json.loads(raw)+ except json.JSONDecodeError:+ await self._send(ws, {"error": {"code": -32700, "message": "Parse error"}})+ return+ if not isinstance(payload, dict):+ await self._send(ws, {"error": {"code": -32600, "message": "Invalid Request"}})+ return+ request_id = payload.get("id")+ method = payload.get("method")+ params = payload.get("params", {})+ if not isinstance(method, str) or not isinstance(params, dict):+ await self._send(ws, {"id": request_id, "error": {"code": -32600, "message": "Invalid Request"}})+ return+ LOG.debug("browser_extension_cdp_message", method=method, size_bytes=len(raw.encode()))+ session_id = payload.get("sessionId")+ try:+ if isinstance(session_id, str):+ if self._registry.is_browser_session_alias(session_id):+ await self._handle_root_command(ws, request_id, method, params, session_id)+ else:+ await self._handle_session_command(ws, request_id, session_id, method, params)+ else:+ await self._handle_root_command(ws, request_id, method, params)+ except ExtensionRequestError as exc:+ error = {"code": -32000, "message": f"{exc.code}: {exc.message}"}+ response = {"id": request_id, "error": error}+ if isinstance(session_id, str):+ response["sessionId"] = session_id+ await self._send(ws, response)++ async def _handle_session_command(+ self, ws: web.WebSocketResponse, request_id: object, session_id: str, method: str, params: dict+ ) -> None:+ try:+ tab_id, chrome_session_id = self._registry.resolve_session(session_id)+ except KeyError:+ await self._send(+ ws,+ {+ "id": request_id,+ "sessionId": session_id,+ "error": {"code": -32001, "message": "session not found"},+ },+ )+ return+ if not is_cdp_method_allowed(method, params):+ raise ExtensionRequestError("CDP_METHOD_NOT_ALLOWED", "The requested CDP method is not allowed.")+ args = {"tabId": tab_id, "method": method, "params": params}+ if chrome_session_id is not None:+ args["sessionId"] = chrome_session_id+ relay_result = await self._relay.request("debugger.send", args)+ await self._send(+ ws,+ {"id": request_id, "sessionId": session_id, "result": relay_result.get("result", {})},+ )++ async def _handle_root_command(+ self,+ ws: web.WebSocketResponse,+ request_id: object,+ method: str,+ params: dict,+ response_session_id: str | None = None,+ ) -> None:+ if method == "Browser.getVersion":+ await self._reply(ws, request_id, dict(_VERSION_RESULT), response_session_id)+ elif method == "Browser.setDownloadBehavior":+ await self._reply(ws, request_id, {}, response_session_id)+ elif method == "Browser.close":+ await self._reply(ws, request_id, {}, response_session_id)+ await self._detach_all_tabs()+ async with self._client_guard:+ if self._client_ws is ws:+ self._client_ws = None+ self._reset_connection_state()+ await ws.close()+ elif method == "Browser.getWindowForTarget":+ await self._reply(+ ws,+ request_id,+ {"windowId": 1, "bounds": dict(_WINDOW_BOUNDS)},+ response_session_id,+ )+ elif method == "Browser.setWindowBounds":+ await self._reply(ws, request_id, {}, response_session_id)+ elif method == "Browser.getWindowBounds":+ await self._reply(ws, request_id, {"bounds": dict(_WINDOW_BOUNDS)}, response_session_id)+ elif method == "Target.setAutoAttach":+ await self._set_auto_attach(ws, request_id, params, response_session_id)+ elif method == "Target.setDiscoverTargets":+ await self._set_discover_targets(ws, request_id, params, response_session_id)+ elif method == "Target.getTargets":+ self._register_scoped_tabs()+ await self._reply(+ ws,+ request_id,+ {"targetInfos": self._page_target_infos()},+ response_session_id,+ )+ elif method == "Target.getTargetInfo":+ await self._get_target_info(ws, request_id, params, response_session_id)+ elif method == "Target.createTarget":+ await self._create_target(ws, request_id, params, response_session_id)+ elif method == "Target.closeTarget":+ await self._close_target(ws, request_id, params, response_session_id)+ elif method == "Target.activateTarget":+ await self._activate_target(ws, request_id, params, response_session_id)+ elif method == "Target.attachToTarget":+ await self._attach_to_target(ws, request_id, params, response_session_id)+ elif method == "Target.attachToBrowserTarget":+ await self._attach_to_browser_target(ws, request_id, response_session_id)+ elif method == "Target.detachFromTarget":+ await self._detach_from_target(ws, request_id, params, response_session_id)+ else:+ await self._send_error(+ ws,+ request_id,+ -32601,+ f"'{method}' wasn't found",+ response_session_id,+ )++ async def _set_auto_attach(+ self,+ ws: web.WebSocketResponse,+ request_id: object,+ params: dict,+ response_session_id: str | None,+ ) -> None:+ self._auto_attach = params.get("autoAttach") is True+ await self._reply(ws, request_id, {}, response_session_id)+ if not self._auto_attach:+ return+ tabs = [tab for tab in self._relay.scoped_tabs if isinstance(tab, dict)]+ if not tabs:+ try:+ created = await self._relay.request("tabs.create", {"url": "about:blank"})+ tab_id = created["tabId"]+ if type(tab_id) is not int:+ raise BrowserExtensionError("Created tab id is invalid")+ tabs = [{"tabId": tab_id, "url": "about:blank", "title": ""}]+ self._begin_tab_scope(tab_id)+ except Exception as exc:+ LOG.debug(+ "browser_extension_auto_attach_target_skipped",+ error_type=type(exc).__name__,+ )+ return+ for tab in tabs:+ tab_id = tab.get("tabId")+ if type(tab_id) is not int:+ continue+ generation = self._active_scope_generation(tab_id)+ if generation is None:+ continue+ try:+ attached = await self._ensure_attached(tab, generation=generation)+ if attached is not None:+ target_id, is_new = attached+ if is_new:+ await self._emit_attached(tab_id, target_id, generation)+ except Exception as exc:+ LOG.debug(+ "browser_extension_auto_attach_target_skipped",+ tab_id=tab_id,+ error_type=type(exc).__name__,+ )++ async def _set_discover_targets(+ self,+ ws: web.WebSocketResponse,+ request_id: object,+ params: dict,+ response_session_id: str | None,+ ) -> None:+ self._discover_targets = params.get("discover") is True+ self._register_scoped_tabs()+ await self._reply(ws, request_id, {}, response_session_id)+ if self._discover_targets:+ for target_info in self._page_target_infos():+ await self._emit(+ "Target.targetCreated",+ {"targetInfo": target_info},+ response_session_id,+ )+ await self._emit(+ "Target.targetCreated",+ {"targetInfo": dict(_BROWSER_TARGET_INFO)},+ response_session_id,+ )++ async def _get_target_info(+ self,+ ws: web.WebSocketResponse,+ request_id: object,+ params: dict,+ response_session_id: str | None,+ ) -> None:+ target_id = params.get("targetId")+ if target_id is None:+ target_info = dict(_BROWSER_TARGET_INFO)+ else:+ self._register_scoped_tabs()+ try:+ if not isinstance(target_id, str):+ raise KeyError(target_id)+ tab_id = self._registry.tab_for_target(target_id)+ target_info = self._registry.target_info(target_id)+ if target_id == self._registry.target_id_for_tab(tab_id):+ target_info = self._target_info(tab_id)+ except KeyError:+ await self._send_error(ws, request_id, -32000, "target not found", response_session_id)+ return+ await self._reply(ws, request_id, {"targetInfo": target_info}, response_session_id)++ async def _create_target(+ self,+ ws: web.WebSocketResponse,+ request_id: object,+ params: dict,+ response_session_id: str | None,+ ) -> None:+ url = params.get("url", "about:blank")+ if not isinstance(url, str):+ url = "about:blank"+ created = await self._relay.request("tabs.create", {"url": url})+ tab = {"tabId": created["tabId"], "url": url, "title": ""}+ tab_id = tab["tabId"]+ if type(tab_id) is not int:+ await self._send_error(ws, request_id, -32000, "created target is invalid", response_session_id)+ return+ generation = self._begin_tab_scope(tab_id)+ attached = await self._ensure_attached(tab, generation=generation)+ if attached is None:+ await self._send_error(ws, request_id, -32000, "target was revoked", response_session_id)+ return+ target_id, _ = attached+ await self._emit_attached(tab_id, target_id, generation)+ await self._reply(ws, request_id, {"targetId": target_id}, response_session_id)++ async def _close_target(+ self,+ ws: web.WebSocketResponse,+ request_id: object,+ params: dict,+ response_session_id: str | None,+ ) -> None:+ target_id = params.get("targetId")+ if not isinstance(target_id, str):+ await self._send_error(ws, request_id, -32000, "target not found", response_session_id)+ return+ try:+ tab_id = self._registry.tab_for_target(target_id)+ except KeyError:+ await self._send_error(ws, request_id, -32000, "target not found", response_session_id)+ return+ await self._relay.request("tabs.remove", {"tabId": tab_id})+ await self._reply(ws, request_id, {"success": True}, response_session_id)++ async def _activate_target(+ self,+ ws: web.WebSocketResponse,+ request_id: object,+ params: dict,+ response_session_id: str | None,+ ) -> None:+ target_id = params.get("targetId")+ if not isinstance(target_id, str):+ await self._send_error(ws, request_id, -32000, "target not found", response_session_id)+ return+ try:+ tab_id = self._registry.tab_for_target(target_id)+ except KeyError:+ await self._send_error(ws, request_id, -32000, "target not found", response_session_id)+ return+ await self._relay.request("tabs.activate", {"tabId": tab_id})+ await self._reply(ws, request_id, {}, response_session_id)++ async def _attach_to_target(+ self,+ ws: web.WebSocketResponse,+ request_id: object,+ params: dict,+ response_session_id: str | None,+ ) -> None:+ self._register_scoped_tabs()+ requested_target_id = params.get("targetId")+ if not isinstance(requested_target_id, str):+ await self._send_error(ws, request_id, -32000, "target not found", response_session_id)+ return+ try:+ tab_id = self._registry.tab_for_target(requested_target_id)+ info = self._target_info(tab_id)+ except KeyError:+ await self._send_error(ws, request_id, -32000, "target not found", response_session_id)+ return+ tab = {"tabId": tab_id, "url": info["url"], "title": info["title"]}+ generation = self._active_scope_generation(tab_id)+ if generation is None:+ await self._send_error(ws, request_id, -32000, "target was revoked", response_session_id)+ return+ attached = await self._ensure_attached(tab, generation=generation)+ if attached is None:+ await self._send_error(ws, request_id, -32000, "target was revoked", response_session_id)+ return+ target_id, is_new = attached+ session_id = self._registry.create_root_session_alias(tab_id)+ await self._reply(ws, request_id, {"sessionId": session_id}, response_session_id)+ if is_new:+ await self._emit_attached(tab_id, target_id, generation)++ async def _attach_to_browser_target(+ self,+ ws: web.WebSocketResponse,+ request_id: object,+ response_session_id: str | None,+ ) -> None:+ session_id = self._registry.create_browser_session_alias()+ await self._reply(ws, request_id, {"sessionId": session_id}, response_session_id)++ async def _detach_from_target(+ self,+ ws: web.WebSocketResponse,+ request_id: object,+ params: dict,+ response_session_id: str | None,+ ) -> None:+ session_id = params.get("sessionId")+ if isinstance(session_id, str):+ if self._registry.remove_browser_session_alias(session_id):+ await self._reply(ws, request_id, {}, response_session_id)+ return+ if self._registry.remove_root_session_alias(session_id):+ await self._reply(ws, request_id, {}, response_session_id)+ return+ try:+ tab_id, chrome_session_id = self._registry.resolve_session(session_id)+ except KeyError:+ tab_id = None+ chrome_session_id = None+ if tab_id is not None and chrome_session_id is None:+ await self._relay.request("debugger.detach", {"tabId": tab_id})+ self._forget_tab(tab_id)+ await self._reply(ws, request_id, {}, response_session_id)++ async def _handle_debugger_event(self, payload: dict) -> None:+ tab_id = payload.get("tabId")+ method = payload.get("method")+ event_params = payload.get("params")+ if type(tab_id) is not int or not isinstance(method, str) or not isinstance(event_params, dict):+ return+ try:+ if isinstance(payload.get("sessionId"), str):+ outer_session_ids = [payload["sessionId"]]+ self._registry.resolve_session(outer_session_ids[0])+ else:+ outer_session_ids = self._registry.root_session_ids(tab_id)+ except KeyError:+ return++ if method == "Page.frameNavigated":+ self._update_main_frame(tab_id, event_params)+ if method == "Target.attachedToTarget":+ child_session_id = event_params.get("sessionId")+ target_info = event_params.get("targetInfo")+ if isinstance(child_session_id, str) and isinstance(target_info, dict):+ # The tab's debugger session reports its own top-level document as a+ # "page" child on every cross-process swap; a real browser endpoint+ # never surfaces that, and forwarding it duplicates the tab target.+ if target_info.get("type") == "page":+ return+ self._registry.register_child_session(tab_id, child_session_id, target_info)+ await self._emit_to_sessions(method, event_params, outer_session_ids)+ if self._auto_attach:+ self._spawn(self._set_child_auto_attach(tab_id, child_session_id))+ await asyncio.sleep(0)+ return+ elif method == "Target.detachedFromTarget":+ child_session_id = event_params.get("sessionId")+ if not isinstance(child_session_id, str):+ return+ try:+ self._registry.resolve_session(child_session_id)+ except KeyError:+ return+ await self._emit_to_sessions(method, event_params, outer_session_ids)+ self._registry.remove_child_session(child_session_id)+ return+ await self._emit_to_sessions(method, event_params, outer_session_ids)++ async def _set_child_auto_attach(self, tab_id: int, child_session_id: str) -> None:+ try:+ await self._relay.request(+ "debugger.send",+ {+ "tabId": tab_id,+ "sessionId": child_session_id,+ "method": "Target.setAutoAttach",+ "params": dict(_CHILD_AUTO_ATTACH_PARAMS),+ },+ )+ except ExtensionRequestError as exc:+ LOG.debug(+ "browser_extension_child_auto_attach_failed",+ method="Target.setAutoAttach",+ error_code=exc.code,+ )++ async def _handle_hello_tabs(self, tabs: list[tuple[dict, int]]) -> None:+ for tab, generation in tabs:+ await self._handle_tab_added(tab, include_opener=False, generation=generation)++ async def _handle_tab_added(self, params: dict, include_opener: bool, generation: int) -> None:+ tab_id = params.get("tabId")+ if type(tab_id) is not int:+ return+ opener_id = None+ opener_tab_id = params.get("openerTabId")+ if include_opener and self._auto_attach and type(opener_tab_id) is int:+ try:+ opener_id = self._registry.target_id_for_tab(opener_tab_id)+ except KeyError:+ opener_id = None+ tab = {+ "tabId": tab_id,+ "url": params.get("url") if isinstance(params.get("url"), str) else "about:blank",+ "title": params.get("title") if isinstance(params.get("title"), str) else "",+ }+ if self._auto_attach:+ attached = await self._ensure_attached(tab, opener_id, generation)+ if attached is None:+ return+ target_id, is_new = attached+ if is_new:+ await self._emit_attached(tab_id, target_id, generation)+ else:+ if not self._scope_is_current(tab_id, generation):+ return+ target_id = self._register_tab(tab, opener_id)+ if self._discover_targets and self._scope_is_current(tab_id, generation):+ await self._emit(+ "Target.targetCreated",+ {"targetInfo": self._target_info(tab_id)},+ scope_guard=(tab_id, generation),+ )++ async def _remove_tab_with_events(self, tab_id: object) -> None:+ if type(tab_id) is not int:+ return+ self._revoke_tab_scope(tab_id)+ lock = self._attach_locks.setdefault(tab_id, asyncio.Lock())+ async with lock:+ try:+ session_ids = self._registry.root_session_ids(tab_id)+ target_id = self._registry.target_id_for_tab(tab_id)+ except KeyError:+ return+ self._forget_tab(tab_id)+ for session_id in session_ids:+ await self._emit(+ "Target.detachedFromTarget",+ {"sessionId": session_id, "targetId": target_id},+ )+ await self._emit("Target.targetDestroyed", {"targetId": target_id})++ async def _ensure_attached(+ self,+ tab: dict,+ opener_id: str | None = None,+ generation: int | None = None,+ ) -> tuple[str, bool] | None:+ tab_id = tab.get("tabId")+ if type(tab_id) is not int:+ raise BrowserExtensionError("Scoped tab id is invalid")+ if generation is None:+ generation = self._active_scope_generation(tab_id)+ if generation is None:+ return None+ lock = self._attach_locks.setdefault(tab_id, asyncio.Lock())+ async with lock:+ if not self._scope_is_current(tab_id, generation):+ return None+ is_new = tab_id not in self._attached_tabs+ real_target_id: str | None = None+ if is_new:+ try:+ await self._relay.request("debugger.attach", {"tabId": tab_id})+ except ExtensionRequestError as exc:+ if "already attached" not in exc.message.lower():+ raise+ real_target_id = await self._fetch_main_frame_id(tab_id)+ if not self._scope_is_current(tab_id, generation):+ return None+ target_id = self._register_tab(tab, opener_id, real_target_id)+ self._attached_tabs.add(tab_id)+ return target_id, is_new++ async def _fetch_main_frame_id(self, tab_id: int) -> str | None:+ # Playwright resolves the main frame's session by targetId, so the exposed+ # page targetId must equal Chrome's real main-frame id for this tab.+ try:+ tree = await self._relay.request(+ "debugger.send",+ {"tabId": tab_id, "method": "Page.getFrameTree", "params": {}},+ )+ except ExtensionRequestError:+ return None+ frame = tree.get("result", {}).get("frameTree", {}).get("frame", {})+ frame_id = frame.get("id") if isinstance(frame, dict) else None+ return frame_id if isinstance(frame_id, str) and frame_id else None++ def _register_tab(self, tab: dict, opener_id: str | None = None, target_id_override: str | None = None) -> str:+ tab_id = tab["tabId"]+ raw_url = tab.get("url")+ url = raw_url if isinstance(raw_url, str) else "about:blank"+ raw_title = tab.get("title")+ title = raw_title if isinstance(raw_title, str) else ""+ try:+ target_id = self._registry.target_id_for_tab(tab_id)+ if target_id_override is not None and target_id != target_id_override:+ target_id = self._registry.register_tab(tab_id, url, title, target_id_override)+ else:+ self._registry.update_tab(tab_id, url, title)+ except KeyError:+ target_id = self._registry.register_tab(tab_id, url, title, target_id_override)+ if opener_id is not None:+ self._opener_ids[tab_id] = opener_id+ return target_id++ def _register_scoped_tabs(self) -> None:+ for tab in self._relay.scoped_tabs:+ if isinstance(tab, dict) and type(tab.get("tabId")) is int:+ tab_id = tab["tabId"]+ if self._active_scope_generation(tab_id) is not None:+ self._register_tab(tab)++ def _target_info(self, tab_id: int) -> dict:+ target_info = self._registry.target_info_for_tab(tab_id)+ opener_id = self._opener_ids.get(tab_id)+ if opener_id is not None:+ target_info["openerId"] = opener_id+ return target_info++ def _page_target_infos(self) -> list[dict]:+ return [+ self._target_info(self._registry.tab_for_target(info["targetId"]))+ for info in self._registry.list_page_targets()+ ]++ async def _emit_attached(self, tab_id: int, target_id: str, generation: int) -> None:+ if not self._scope_is_current(tab_id, generation):+ return+ await self._emit(+ "Target.attachedToTarget",+ {+ "sessionId": self._registry.root_session_id(tab_id),+ "targetInfo": self._target_info(tab_id),+ "waitingForDebugger": False,+ },+ scope_guard=(tab_id, generation),+ )++ def _update_main_frame(self, tab_id: int, params: dict) -> None:+ frame = params.get("frame")+ if not isinstance(frame, dict) or frame.get("parentId") is not None:+ return+ url = frame.get("url")+ if not isinstance(url, str):+ return+ try:+ current = self._target_info(tab_id)+ except KeyError:+ return+ raw_title = frame.get("title")+ title = raw_title if isinstance(raw_title, str) else str(current["title"])+ self._registry.update_tab(tab_id, url, title)++ async def _detach_all_tabs(self) -> None:+ tab_ids = [self._registry.tab_for_target(info["targetId"]) for info in self._registry.list_page_targets()]+ for tab_id in tab_ids:+ try:+ await self._relay.request("debugger.detach", {"tabId": tab_id}, timeout=2.0)+ except Exception as exc:+ LOG.debug(+ "browser_extension_debugger_detach_failed", method="debugger.detach", error_type=type(exc).__name__+ )++ def _forget_tab(self, tab_id: int) -> None:+ self._attached_tabs.discard(tab_id)+ self._opener_ids.pop(tab_id, None)+ self._registry.remove_tab(tab_id)++ def _begin_tab_scope(self, tab_id: int) -> int:+ generation = self._scope_generations.get(tab_id, 0) + 1+ self._scope_generations[tab_id] = generation+ self._scope_tombstones.discard(tab_id)+ return generation++ def _revoke_tab_scope(self, tab_id: int) -> int:+ generation = self._scope_generations.get(tab_id, 0) + 1+ self._scope_generations[tab_id] = generation+ self._scope_tombstones.add(tab_id)+ return generation++ def _active_scope_generation(self, tab_id: int) -> int | None:+ if tab_id in self._scope_tombstones:+ return None+ generation = self._scope_generations.get(tab_id)+ if generation is None:+ generation = self._begin_tab_scope(tab_id)+ return generation++ def _scope_is_current(self, tab_id: int, generation: int) -> bool:+ return self._scope_generations.get(tab_id) == generation and tab_id not in self._scope_tombstones++ def _reset_connection_state(self) -> None:+ for task in self._background_tasks:+ task.cancel()+ self._auto_attach = False+ self._discover_targets = False+ self._attached_tabs.clear()+ self._attach_locks.clear()+ self._opener_ids.clear()+ self._scope_generations.clear()+ self._scope_tombstones.clear()+ self._registry.clear()++ def _spawn(self, coroutine: Coroutine[object, object, None]) -> None:+ task = asyncio.create_task(coroutine)+ self._background_tasks.add(task)+ task.add_done_callback(self._background_task_done)++ def _background_task_done(self, task: asyncio.Task[None]) -> None:+ self._background_tasks.discard(task)+ if task.cancelled():+ return+ error = task.exception()+ if error is not None:+ LOG.debug("browser_extension_event_task_failed", error_type=type(error).__name__)++ async def _reply(+ self,+ ws: web.WebSocketResponse,+ request_id: object,+ result: dict,+ session_id: str | None = None,+ ) -> None:+ payload = {"id": request_id, "result": result}+ if session_id is not None:+ payload["sessionId"] = session_id+ await self._send(ws, payload)++ async def _send_error(+ self,+ ws: web.WebSocketResponse,+ request_id: object,+ code: int,+ message: str,+ session_id: str | None = None,+ ) -> None:+ payload = {"id": request_id, "error": {"code": code, "message": message}}+ if session_id is not None:+ payload["sessionId"] = session_id+ await self._send(ws, payload)++ async def _emit_to_sessions(self, method: str, params: dict, session_ids: list[str]) -> None:+ for session_id in session_ids:+ await self._emit(method, params, session_id)++ async def _emit(+ self,+ method: str,+ params: dict,+ session_id: str | None = None,+ scope_guard: tuple[int, int] | None = None,+ ) -> None:+ ws = self._client_ws+ if ws is None or ws.closed:+ return+ payload = {"method": method, "params": params}+ if session_id is not None:+ payload["sessionId"] = session_id+ await self._send(ws, payload, scope_guard)++ async def _send(+ self,+ ws: web.WebSocketResponse,+ payload: dict,+ scope_guard: tuple[int, int] | None = None,+ ) -> None:+ async with self._send_lock:+ if scope_guard is not None and not self._scope_is_current(*scope_guard):+ return+ if not ws.closed:+ await ws.send_json(payload)diff --git a/skyvern/browser_extension/errors.py b/skyvern/browser_extension/errors.pynew file mode 100644index 000000000..7a3f95716--- /dev/null+++ b/skyvern/browser_extension/errors.py@@ -0,0 +1,13 @@+class BrowserExtensionError(Exception):+ pass+++class BrowserExtensionNotConnectedError(BrowserExtensionError):+ pass+++class ExtensionRequestError(BrowserExtensionError):+ def __init__(self, code: str, message: str) -> None:+ self.code = code+ self.message = message+ super().__init__(f"{code}: {message}")diff --git a/skyvern/browser_extension/extension/README.md b/skyvern/browser_extension/extension/README.mdnew file mode 100644index 000000000..c071b8511--- /dev/null+++ b/skyvern/browser_extension/extension/README.md@@ -0,0 +1,57 @@+# Install Skyvern Agent for development++The **Skyvern Agent** extension is plain JavaScript and does not require a build step. The recommended setup is:++```bash+skyvern browser extension-install+```++This prints the unpacked extension directory, best-effort opens `chrome://extensions`, and prints the remaining+numbered steps. If the MCP extension bridge is already running, it also starts one-click pairing.++To set it up manually:++1. Print the unpacked extension directory:++ ```bash+ skyvern browser extension-path+ ```++2. Open `chrome://extensions`, enable **Developer mode**, and click **Load unpacked**.+3. Select the directory printed by the command.+4. Start the local MCP server with the extension bridge enabled:++ ```bash+ skyvern run mcp --browser-extension+ ```++5. Start the recommended one-click pairing flow:++ ```bash+ skyvern browser extension-pair+ ```++6. Click **Approve** in the pairing page, then approve the pairing in the **Skyvern Agent** confirmation tab that opens.+ The link expires after two minutes and is single-use, so rerun the command when needed.+7. Add any tab you want Skyvern to control to the **Skyvern Controlled** tab group. Dragging a tab into the group grants+ access and dragging it out revokes access. The popup's **Add to Skyvern Controlled** and+ **Remove from Skyvern Controlled** buttons perform the same actions.++If one-click pairing is unavailable, run `skyvern browser extension-token`, open the **Skyvern Agent** popup, paste the+token, and click **Connect**. This popup-paste path is the manual fallback.++The local Skyvern MCP process runs the bridge on `ws://127.0.0.1:19777/extension/v1` by default, and the extension+connects outbound to it. To use another port, set `SKYVERN_BROWSER_EXTENSION_PORT` for the MCP process and enter the+same port under **Advanced settings** in the popup.++Check the token configuration, token-file permissions, and bridge status without displaying the token:++```bash+skyvern browser extension-status+```++Chrome's debugger infobar remains visible while Skyvern controls a tab. Clicking **Cancel**, removing the tab with the+popup, or dragging the tab out of **Skyvern Controlled** revokes access.++For the complete setup, security model, and limitations, see+[Control Your Chrome with Skyvern Agent](../../../docs/developers/optimization/chrome-extension.mdx).diff --git a/skyvern/browser_extension/extension/bridge_connection.js b/skyvern/browser_extension/extension/bridge_connection.jsnew file mode 100644index 000000000..2ea2e8086--- /dev/null+++ b/skyvern/browser_extension/extension/bridge_connection.js@@ -0,0 +1,463 @@+import {+ BRIDGE_ALARM_NAME,+ DEFAULT_BRIDGE_PORT,+ ERROR_CODES,+ MESSAGE_TYPES,+ PROTOCOL_VERSION,+ ProtocolError,+ protocolErrorEnvelope,+} from "./protocol.js";++const PING_INTERVAL_MS = 20_000;+const SILENCE_TIMEOUT_MS = 45_000;+const RECONNECT_MAX_MS = 30_000;+const AUTH_EXT_CONTEXT = "skyvern-ext-v1|";+const AUTH_SERVER_CONTEXT = "skyvern-srv-v1|";++function bytesToBase64Url(bytes) {+ let binary = "";+ for (const byte of bytes) {+ binary += String.fromCharCode(byte);+ }+ return btoa(binary)+ .replaceAll("+", "-")+ .replaceAll("/", "_")+ .replace(/=+$/, "");+}++function base64UrlToBytes(value) {+ if (typeof value !== "string" || !/^[A-Za-z0-9_-]+$/.test(value)) {+ throw new ProtocolError(+ ERROR_CODES.AUTH_FAILED,+ "Bridge authentication failed.",+ );+ }+ const padding = "=".repeat((4 - (value.length % 4)) % 4);+ try {+ const binary = atob(+ value.replaceAll("-", "+").replaceAll("_", "/") + padding,+ );+ return Uint8Array.from(binary, (character) => character.charCodeAt(0));+ } catch {+ throw new ProtocolError(+ ERROR_CODES.AUTH_FAILED,+ "Bridge authentication failed.",+ );+ }+}++function constantTimeEqual(left, right) {+ const length = Math.max(left.length, right.length);+ let difference = left.length ^ right.length;+ for (let index = 0; index < length; index += 1) {+ difference |= (left[index] ?? 0) ^ (right[index] ?? 0);+ }+ return difference === 0;+}++async function importHmacKey(token) {+ return crypto.subtle.importKey(+ "raw",+ new TextEncoder().encode(token),+ { name: "HMAC", hash: "SHA-256" },+ false,+ ["sign"],+ );+}++async function signHmac(key, message) {+ const signature = await crypto.subtle.sign(+ "HMAC",+ key,+ new TextEncoder().encode(message),+ );+ return new Uint8Array(signature);+}++export class BridgeConnection {+ constructor({ onRequest, onAuthenticated, onStateChange }) {+ this.onRequest = onRequest;+ this.onAuthenticated = onAuthenticated;+ this.onStateChange = onStateChange;+ this.socket = null;+ this.authenticated = false;+ this.enabled = false;+ this.port = DEFAULT_BRIDGE_PORT;+ this.token = "";+ this.lastError = "";+ this.lastMessageAt = 0;+ this.reconnectDelayMs = 1_000;+ this.reconnectTimer = null;+ this.pingTimer = null;+ this.silenceTimer = null;+ this.connectionGeneration = 0;+ this.clientNonce = null;+ this.serverNonce = null;+ this.hmacKey = null;++ chrome.alarms.onAlarm.addListener((alarm) => {+ if (alarm.name === BRIDGE_ALARM_NAME) {+ void this.kick();+ }+ });+ }++ async initialize() {+ const stored = await chrome.storage.local.get({+ bridgePort: DEFAULT_BRIDGE_PORT,+ pairingToken: "",+ enabled: false,+ });+ this.port = this.normalizePort(stored.bridgePort);+ this.token =+ typeof stored.pairingToken === "string" ? stored.pairingToken : "";+ this.enabled = stored.enabled === true;+ await chrome.alarms.create(BRIDGE_ALARM_NAME, { periodInMinutes: 1 });+ this.notifyState();+ if (this.enabled && this.token) {+ await this.kick();+ }+ }++ getStatus() {+ return {+ connected: this.authenticated,+ enabled: this.enabled,+ bridgePort: this.port,+ pairingToken: this.token,+ lastError: this.lastError,+ };+ }++ async setConfig({ bridgePort, pairingToken }) {+ const port = this.normalizePort(bridgePort);+ if (typeof pairingToken !== "string") {+ throw new ProtocolError(+ ERROR_CODES.AUTH_FAILED,+ "A pairing token is required.",+ );+ }+ const token = pairingToken.trim();+ const changed = port !== this.port || token !== this.token;+ this.port = port;+ this.token = token;+ await chrome.storage.local.set({ bridgePort: port, pairingToken: token });+ if (changed && this.enabled) {+ this.closeSocket();+ await this.kick();+ }+ return {};+ }++ async connect() {+ if (!this.token) {+ throw new ProtocolError(+ ERROR_CODES.AUTH_FAILED,+ "A pairing token is required.",+ );+ }+ this.enabled = true;+ this.lastError = "";+ this.reconnectDelayMs = 1_000;+ await chrome.storage.local.set({ enabled: true });+ await this.kick();+ return {};+ }++ async disconnect() {+ this.enabled = false;+ this.lastError = "";+ await chrome.storage.local.set({ enabled: false });+ this.clearReconnectTimer();+ this.closeSocket();+ this.notifyState();+ return {};+ }++ async kick() {+ if (!this.enabled || !this.token || this.socket !== null) {+ return;+ }++ const generation = ++this.connectionGeneration;+ let socket;+ try {+ socket = new WebSocket(`ws://127.0.0.1:${this.port}/extension/v1`);+ } catch {+ this.scheduleReconnect();+ return;+ }+ this.socket = socket;+ this.authenticated = false;+ this.clientNonce = null;+ this.serverNonce = null;+ this.hmacKey = null;+ this.notifyState();++ socket.addEventListener("open", () => {+ if (generation === this.connectionGeneration) {+ this.lastMessageAt = Date.now();+ this.startHealthTimers();+ }+ });+ socket.addEventListener("message", (event) => {+ if (generation === this.connectionGeneration) {+ void this.handleMessage(event.data);+ }+ });+ socket.addEventListener("error", () => {+ if (generation === this.connectionGeneration) {+ this.lastError = "Could not reach the local Skyvern bridge.";+ this.notifyState();+ }+ });+ socket.addEventListener("close", () => {+ if (generation !== this.connectionGeneration) {+ return;+ }+ this.socket = null;+ this.authenticated = false;+ this.stopHealthTimers();+ this.notifyState();+ this.scheduleReconnect();+ });+ }++ async handleMessage(rawMessage) {+ this.lastMessageAt = Date.now();+ let message;+ try {+ message = JSON.parse(rawMessage);+ } catch {+ this.failConnection("The local bridge sent an invalid message.");+ return;+ }++ if (message?.v !== PROTOCOL_VERSION || typeof message.type !== "string") {+ this.failConnection("The local bridge protocol is incompatible.");+ return;+ }++ try {+ if (!this.authenticated) {+ await this.handleAuthMessage(message);+ return;+ }+ if (message.type === MESSAGE_TYPES.PING) {+ this.sendRaw({ v: PROTOCOL_VERSION, type: MESSAGE_TYPES.PONG });+ return;+ }+ if (message.type === MESSAGE_TYPES.PONG) {+ return;+ }+ if (message.type === MESSAGE_TYPES.REQUEST) {+ await this.handleRequest(message);+ return;+ }+ this.failConnection("The local bridge sent an unexpected message.");+ } catch (error) {+ if (+ error instanceof ProtocolError &&+ error.code === ERROR_CODES.AUTH_FAILED+ ) {+ this.failConnection(error.message, 4403);+ } else {+ this.failConnection("The local bridge connection failed.");+ }+ }+ }++ async handleAuthMessage(message) {+ if (+ message.type === MESSAGE_TYPES.AUTH_CHALLENGE &&+ this.serverNonce === null+ ) {+ const serverNonceBytes = base64UrlToBytes(message.serverNonce);+ if (serverNonceBytes.length !== 32) {+ throw new ProtocolError(+ ERROR_CODES.AUTH_FAILED,+ "Bridge authentication failed.",+ );+ }+ this.serverNonce = message.serverNonce;+ const nonceBytes = crypto.getRandomValues(new Uint8Array(32));+ this.clientNonce = bytesToBase64Url(nonceBytes);+ this.hmacKey = await importHmacKey(this.token);+ const proof = bytesToBase64Url(+ await signHmac(+ this.hmacKey,+ `${AUTH_EXT_CONTEXT}${this.serverNonce}|${this.clientNonce}`,+ ),+ );+ this.sendRaw({+ v: PROTOCOL_VERSION,+ type: MESSAGE_TYPES.AUTH_PROOF,+ clientNonce: this.clientNonce,+ proof,+ });+ return;+ }++ if (+ message.type === MESSAGE_TYPES.AUTH_OK &&+ this.clientNonce !== null &&+ this.serverNonce !== null+ ) {+ const receivedProof = base64UrlToBytes(message.serverProof);+ const expectedProof = await signHmac(+ this.hmacKey,+ `${AUTH_SERVER_CONTEXT}${this.clientNonce}|${this.serverNonce}`,+ );+ if (!constantTimeEqual(receivedProof, expectedProof)) {+ throw new ProtocolError(+ ERROR_CODES.AUTH_FAILED,+ "Bridge authentication failed.",+ );+ }+ this.authenticated = true;+ this.lastError = "";+ this.reconnectDelayMs = 1_000;+ this.notifyState();+ await this.onAuthenticated();+ return;+ }++ throw new ProtocolError(+ ERROR_CODES.AUTH_FAILED,+ "Bridge authentication failed.",+ );+ }++ async handleRequest(message) {+ if (typeof message.id !== "string" || typeof message.op !== "string") {+ this.failConnection("The local bridge sent an invalid request.");+ return;+ }++ try {+ const result = await this.onRequest(message.op, message.args);+ this.sendRaw({+ v: PROTOCOL_VERSION,+ type: MESSAGE_TYPES.RESPONSE,+ id: message.id,+ ok: true,+ result,+ });+ } catch (error) {+ this.sendRaw({+ v: PROTOCOL_VERSION,+ type: MESSAGE_TYPES.RESPONSE,+ id: message.id,+ ok: false,+ error: protocolErrorEnvelope(error),+ });+ }+ }++ sendEvent(event, params) {+ if (!this.authenticated) {+ return false;+ }+ return this.sendRaw({+ v: PROTOCOL_VERSION,+ type: MESSAGE_TYPES.EVENT,+ event,+ params,+ });+ }++ sendRaw(message) {+ if (this.socket?.readyState !== WebSocket.OPEN) {+ return false;+ }+ this.socket.send(JSON.stringify(message));+ return true;+ }++ normalizePort(value) {+ const port = Number(value);+ if (!Number.isInteger(port) || port < 1 || port > 65_535) {+ throw new ProtocolError(+ ERROR_CODES.INTERNAL,+ "Bridge port must be between 1 and 65535.",+ );+ }+ return port;+ }++ startHealthTimers() {+ this.stopHealthTimers();+ this.pingTimer = setInterval(() => {+ if (this.authenticated) {+ this.sendRaw({ v: PROTOCOL_VERSION, type: MESSAGE_TYPES.PING });+ }+ }, PING_INTERVAL_MS);+ this.silenceTimer = setInterval(() => {+ if (+ this.lastMessageAt > 0 &&+ Date.now() - this.lastMessageAt > SILENCE_TIMEOUT_MS+ ) {+ this.failConnection("The local bridge stopped responding.");+ }+ }, 5_000);+ }++ stopHealthTimers() {+ if (this.pingTimer !== null) {+ clearInterval(this.pingTimer);+ this.pingTimer = null;+ }+ if (this.silenceTimer !== null) {+ clearInterval(this.silenceTimer);+ this.silenceTimer = null;+ }+ }++ scheduleReconnect() {+ if (!this.enabled || !this.token || this.reconnectTimer !== null) {+ return;+ }+ const delay = this.reconnectDelayMs;+ this.reconnectDelayMs = Math.min(+ this.reconnectDelayMs * 2,+ RECONNECT_MAX_MS,+ );+ this.reconnectTimer = setTimeout(() => {+ this.reconnectTimer = null;+ void this.kick();+ }, delay);+ }++ clearReconnectTimer() {+ if (this.reconnectTimer !== null) {+ clearTimeout(this.reconnectTimer);+ this.reconnectTimer = null;+ }+ }++ closeSocket() {+ this.connectionGeneration += 1;+ const socket = this.socket;+ this.socket = null;+ this.authenticated = false;+ this.stopHealthTimers();+ if (socket !== null && socket.readyState < WebSocket.CLOSING) {+ socket.close(1000, "Disconnected");+ }+ this.notifyState();+ }++ failConnection(message, code = 4400) {+ this.lastError = message;+ this.authenticated = false;+ const socket = this.socket;+ if (socket !== null && socket.readyState < WebSocket.CLOSING) {+ socket.close(code, message.slice(0, 120));+ }+ this.notifyState();+ }++ notifyState() {+ this.onStateChange(this.getStatus());+ }+}diff --git a/skyvern/browser_extension/extension/debugger_router.js b/skyvern/browser_extension/extension/debugger_router.jsnew file mode 100644index 000000000..8ce4340cd--- /dev/null+++ b/skyvern/browser_extension/extension/debugger_router.js@@ -0,0 +1,217 @@+import {+ ALLOWED_CDP_PREFIXES,+ DENIED_CDP_METHODS,+ ERROR_CODES,+ EVENTS,+ ProtocolError,+ requireArgs,+ requireTabId,+} from "./protocol.js";++export class DebuggerRouter {+ constructor({ tabScope, sendEvent, onAttachedChange }) {+ this.tabScope = tabScope;+ this.sendEvent = sendEvent;+ this.onAttachedChange = onAttachedChange;+ this.attachedTabs = new Set();++ chrome.debugger.onEvent.addListener((source, method, params) => {+ void this.handleDebuggerEvent(source, method, params);+ });+ chrome.debugger.onDetach.addListener((source, reason) => {+ void this.handleDebuggerDetach(source, reason);+ });+ }++ hasAttachedTabs() {+ return this.attachedTabs.size > 0;+ }++ async attach(args) {+ const values = requireArgs(args);+ const tabId = requireTabId(values.tabId);+ return this.tabScope.runTabOperation(tabId, async () => {+ await this.ensureStillControllableLocked(tabId);+ if (this.attachedTabs.has(tabId)) {+ return {};+ }+ try {+ await chrome.debugger.attach({ tabId }, "1.3");+ } catch {+ await this.ensureStillControllableLocked(tabId);+ throw new ProtocolError(+ ERROR_CODES.CDP_ERROR,+ "Chrome could not attach the debugger to this tab.",+ );+ }+ this.attachedTabs.add(tabId);+ this.onAttachedChange();+ await this.ensureStillControllableLocked(tabId);+ return {};+ });+ }++ async detach(args) {+ const values = requireArgs(args);+ const tabId = requireTabId(values.tabId);+ return this.tabScope.runTabOperation(tabId, async () => {+ await this.tabScope.assertScoped(tabId);+ if (!this.attachedTabs.has(tabId)) {+ throw new ProtocolError(+ ERROR_CODES.DEBUGGER_DETACHED,+ "The debugger is not attached to this tab.",+ );+ }+ try {+ await chrome.debugger.detach({ tabId });+ } catch {+ throw new ProtocolError(+ ERROR_CODES.DEBUGGER_DETACHED,+ "The debugger is not attached to this tab.",+ );+ }+ this.attachedTabs.delete(tabId);+ this.onAttachedChange();+ return {};+ });+ }++ async send(args) {+ const values = requireArgs(args);+ const tabId = requireTabId(values.tabId);+ return this.tabScope.runTabOperation(tabId, async () => {+ await this.ensureStillControllableLocked(tabId);+ if (!this.attachedTabs.has(tabId)) {+ throw new ProtocolError(+ ERROR_CODES.DEBUGGER_DETACHED,+ "The debugger is not attached to this tab.",+ );+ }+ if (+ typeof values.method !== "string" ||+ !this.isMethodAllowed(values.method, values.params)+ ) {+ throw new ProtocolError(+ ERROR_CODES.CDP_METHOD_NOT_ALLOWED,+ "The requested CDP method is not allowed.",+ );+ }+ if (+ values.sessionId !== undefined &&+ typeof values.sessionId !== "string"+ ) {+ throw new ProtocolError(+ ERROR_CODES.CDP_ERROR,+ "The CDP session identifier is invalid.",+ );+ }+ if (+ values.params !== undefined &&+ (values.params === null ||+ typeof values.params !== "object" ||+ Array.isArray(values.params))+ ) {+ throw new ProtocolError(+ ERROR_CODES.CDP_ERROR,+ "CDP parameters must be an object.",+ );+ }++ const target =+ values.sessionId === undefined+ ? { tabId }+ : { tabId, sessionId: values.sessionId };+ let result;+ try {+ result = await chrome.debugger.sendCommand(+ target,+ values.method,+ values.params ?? {},+ );+ } catch {+ await this.ensureStillControllableLocked(tabId);+ throw new ProtocolError(+ ERROR_CODES.CDP_ERROR,+ "Chrome rejected the CDP command.",+ );+ }+ await this.ensureStillControllableLocked(tabId);+ return { result: result ?? {} };+ });+ }++ async detachIfAttached(tabId) {+ return this.tabScope.runTabOperation(tabId, () =>+ this.detachIfAttachedLocked(tabId),+ );+ }++ async detachIfAttachedLocked(tabId) {+ if (!this.attachedTabs.has(tabId)) {+ return;+ }+ try {+ await chrome.debugger.detach({ tabId });+ } catch {+ this.attachedTabs.delete(tabId);+ this.onAttachedChange();+ return;+ }+ this.attachedTabs.delete(tabId);+ this.onAttachedChange();+ }++ async ensureStillControllableLocked(tabId) {+ try {+ await this.tabScope.assertControllableLocked(tabId);+ } catch (error) {+ await this.detachIfAttachedLocked(tabId);+ throw error;+ }+ }++ isMethodAllowed(method, params) {+ if (+ DENIED_CDP_METHODS.has(method) ||+ (method === "Network.getCookies" &&+ params !== null &&+ typeof params === "object" &&+ Object.prototype.hasOwnProperty.call(params, "urls"))+ ) {+ return false;+ }+ const separator = method.indexOf(".");+ return (+ separator > 0 && ALLOWED_CDP_PREFIXES.has(method.slice(0, separator))+ );+ }++ async handleDebuggerEvent(source, method, params) {+ if (+ !Number.isInteger(source.tabId) ||+ !this.tabScope.isScoped(source.tabId)+ ) {+ return;+ }+ const eventParams = { tabId: source.tabId, method, params: params ?? {} };+ if (typeof source.sessionId === "string") {+ eventParams.sessionId = source.sessionId;+ }+ this.sendEvent(EVENTS.DEBUGGER_EVENT, eventParams);+ }++ async handleDebuggerDetach(source, reason) {+ if (!Number.isInteger(source.tabId)) {+ return;+ }+ await this.tabScope.runTabOperation(source.tabId, async () => {+ this.attachedTabs.delete(source.tabId);+ this.onAttachedChange();+ this.sendEvent(EVENTS.DEBUGGER_DETACHED, {+ tabId: source.tabId,+ reason: typeof reason === "string" ? reason : "unknown",+ });+ await this.tabScope.handleDebuggerDetachLocked(source.tabId);+ });+ }+}diff --git a/skyvern/browser_extension/extension/icons/icon128.png b/skyvern/browser_extension/extension/icons/icon128.pngnew file mode 100644index 000000000..bc5624dc3Binary files /dev/null and b/skyvern/browser_extension/extension/icons/icon128.png differdiff --git a/skyvern/browser_extension/extension/icons/icon16.png b/skyvern/browser_extension/extension/icons/icon16.pngnew file mode 100644index 000000000..539ac0c32Binary files /dev/null and b/skyvern/browser_extension/extension/icons/icon16.png differdiff --git a/skyvern/browser_extension/extension/icons/icon48.png b/skyvern/browser_extension/extension/icons/icon48.pngnew file mode 100644index 000000000..98820aa21Binary files /dev/null and b/skyvern/browser_extension/extension/icons/icon48.png differdiff --git a/skyvern/browser_extension/extension/manifest.json b/skyvern/browser_extension/extension/manifest.jsonnew file mode 100644index 000000000..77686c502--- /dev/null+++ b/skyvern/browser_extension/extension/manifest.json@@ -0,0 +1,36 @@+{+ "manifest_version": 3,+ "name": "Skyvern Agent",+ "version": "0.1.0",+ "description": "Connects explicitly shared Chrome tabs to a local Skyvern browser session.",+ "minimum_chrome_version": "125",+ "key": "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoSTVDWJdVfrESGVbDh+BU9SLzY0aSnGb3HmW8ZnlRm43Mo9NmIoU4zhB/pbVcoK//0QMIiGzj22242cdwCfcl7X70bJHWWYG33US9FOTPdkxqviv73Jgi0z+XM/TDGe1dUU0ztz1QsfuMFu2MT9Yk1AT5r1507oERuNXo32BzNr5V7S/Ja8OZeRlYW4x7m3hA+OAQiU/0jE2FfuJVa/xB5Gm54LpxpJ7RxM421EZShp6yJyFxxgAEEMiFuNGXfNVY6kMLlpsQ1QiKTo+8I8RJ4DLcb2/bKteCk1U0JqoV+X9nj5sZBA7UmkBmp2+HTkCEIAlKgV+fUjoi3QNrXqPOQIDAQAB",+ "permissions": [+ "debugger",+ "tabs",+ "tabGroups",+ "storage",+ "alarms"+ ],+ "background": {+ "service_worker": "service_worker.js",+ "type": "module"+ },+ "externally_connectable": {+ "matches": ["http://127.0.0.1/*", "http://localhost/*"]+ },+ "action": {+ "default_title": "Skyvern Agent",+ "default_popup": "popup.html",+ "default_icon": {+ "16": "icons/icon16.png",+ "48": "icons/icon48.png",+ "128": "icons/icon128.png"+ }+ },+ "icons": {+ "16": "icons/icon16.png",+ "48": "icons/icon48.png",+ "128": "icons/icon128.png"+ }+}diff --git a/skyvern/browser_extension/extension/pairing_confirm.css b/skyvern/browser_extension/extension/pairing_confirm.cssnew file mode 100644index 000000000..a71375c89--- /dev/null+++ b/skyvern/browser_extension/extension/pairing_confirm.css@@ -0,0 +1,437 @@+:root {+ color-scheme: dark light;+ font-family:+ ui-sans-serif,+ system-ui,+ -apple-system,+ BlinkMacSystemFont,+ "Segoe UI",+ sans-serif;+ color: #f4f4f5;+ background: #0b0c0f;+ font-synthesis: none;+ --accent: #6d6cf6;+ --accent-hover: #5f5eea;+ --surface: #141519;+ --surface-subtle: #1a1b20;+ --border: #2b2c33;+ --border-strong: #3a3b43;+ --text: #f4f4f5;+ --muted: #b2b3ba;+ --subtle: #85868f;+ --radius: 0.5rem;+}++* {+ box-sizing: border-box;+}++body {+ min-height: 100vh;+ margin: 0;+ display: grid;+ overflow-x: hidden;+ color: var(--text);+ background: #0b0c0f;+ place-items: center;+}++.page-shell {+ width: 100%;+ padding: 2rem 1rem;+}++.pairing-card {+ width: min(38rem, 100%);+ margin: 0 auto;+ overflow: hidden;+ border: 1px solid var(--border);+ border-radius: var(--radius);+ background: var(--surface);+ box-shadow: 0 1rem 2.5rem rgb(0 0 0 / 28%);+}++.card-header {+ display: flex;+ align-items: center;+ gap: 0.75rem;+ padding: 1.5rem 1.75rem 0;+}++.brand-mark {+ display: grid;+ flex: none;+ width: 2rem;+ height: 2rem;+ overflow: hidden;+ border-radius: var(--radius);+ place-items: center;+}++.brand-mark img {+ width: 100%;+ height: 100%;+ object-fit: cover;+}++.brand-copy {+ display: grid;+}++.brand-name {+ font-size: 0.9rem;+ font-weight: 650;+ letter-spacing: -0.01em;+}++.card-body {+ padding: 2.5rem 1.75rem 1.75rem;+}++h1 {+ margin: 0;+ font-size: clamp(2rem, 6vw, 2.35rem);+ font-weight: 680;+ line-height: 1.15;+ letter-spacing: -0.035em;+}++.question {+ max-width: 32rem;+ min-height: 2.9rem;+ margin: 0.85rem 0 0;+ color: var(--muted);+ font-size: 0.95rem;+ line-height: 1.55;+}++.pairing-details {+ display: grid;+ grid-template-columns: repeat(2, minmax(0, 1fr));+ gap: 0.75rem;+ margin-top: 2rem;+}++.detail-chip {+ display: grid;+ min-width: 0;+ gap: 0.4rem;+ padding: 0.9rem 1rem;+ border: 1px solid var(--border);+ border-radius: var(--radius);+ background: var(--surface-subtle);+}++.detail-copy {+ display: contents;+}++.detail-label {+ color: var(--subtle);+ font-size: 0.72rem;+ font-weight: 600;+ line-height: 1.3;+}++.detail-chip code {+ overflow: hidden;+ color: var(--text);+ font:+ 600 0.78rem/1.35 ui-monospace,+ SFMono-Regular,+ Menlo,+ monospace;+ text-overflow: ellipsis;+ white-space: nowrap;+}++#pairing-fingerprint {+ letter-spacing: 0.12em;+}++.security-note {+ margin: 0.8rem 0 1.75rem;+ color: var(--subtle);+}++.security-note p {+ margin: 0;+ font-size: 0.72rem;+ line-height: 1.5;+}++.actions {+ display: flex;+ justify-content: flex-end;+ gap: 0.65rem;+}++button {+ font: inherit;+}++.primary-button,+.secondary-button {+ display: inline-flex;+ width: auto;+ min-height: 2.75rem;+ align-items: center;+ justify-content: center;+ gap: 0.55rem;+ padding: 0.7rem 1rem;+ border-radius: var(--radius);+ font-size: 0.875rem;+ font-weight: 650;+ cursor: pointer;+ transition:+ border-color 150ms ease,+ background 150ms ease,+ box-shadow 150ms ease;+}++.primary-button {+ border: 1px solid var(--accent);+ color: white;+ background: var(--accent);+ box-shadow: 0 0.25rem 0.7rem rgb(0 0 0 / 24%);+}++.secondary-button {+ border: 1px solid var(--border-strong);+ color: var(--text);+ background: transparent;+}++.primary-button:hover:not(:disabled) {+ background: var(--accent-hover);+ box-shadow: 0 0.3rem 0.85rem rgb(0 0 0 / 28%);+}++.secondary-button:hover:not(:disabled) {+ background: var(--surface-subtle);+}++.primary-button:focus-visible,+.secondary-button:focus-visible,+.copy-chip:focus-visible {+ outline: 2px solid var(--text);+ outline-offset: 3px;+}++.primary-button:disabled,+.secondary-button:disabled {+ cursor: not-allowed;+ opacity: 0.55;+}++[data-state="approving"] .primary-button:disabled {+ cursor: wait;+ opacity: 0.8;+}++.spinner {+ display: none;+ width: 1rem;+ height: 1rem;+ border: 2px solid rgb(255 255 255 / 35%);+ border-top-color: white;+ border-radius: 50%;+ animation: spin 700ms linear infinite;+}++[data-state="approving"] .spinner {+ display: block;+}++.result-panel {+ display: grid;+ gap: 0.7rem;+ margin-top: 1.5rem;+ padding-top: 1rem;+ border-top: 1px solid var(--border);+}++.result-panel[hidden] {+ display: none;+}++.result-heading {+ display: flex;+ align-items: center;+ gap: 0.65rem;+}++.result-icon {+ display: grid;+ flex: none;+ width: 1.5rem;+ height: 1.5rem;+ color: var(--muted);+ place-items: center;+}++.result-icon svg {+ width: 1rem;+ height: 1rem;+}++.result-icon .success-icon,+.result-icon .neutral-icon {+ display: none;+}++.result-title {+ margin: 0;+ font-size: 0.9rem;+ font-weight: 650;+}++.result-message {+ margin: 0;+ color: var(--muted);+ font-size: 0.78rem;+ line-height: 1.55;+}++[data-state="success"] .result-icon .error-icon,+[data-state="cancelled"] .result-icon .error-icon {+ display: none;+}++[data-state="success"] .result-icon .success-icon,+[data-state="cancelled"] .result-icon .neutral-icon {+ display: block;+}++[data-state="success"] .actions,+[data-state="cancelled"] .actions,+[data-state="error"] .actions[data-terminal="true"] {+ display: none;+}++.recovery {+ display: grid;+ gap: 0.4rem;+}++.recovery[hidden] {+ display: none;+}++.recovery-label {+ color: var(--subtle);+ font-size: 0.7rem;+ font-weight: 600;+}++.copy-chip {+ display: flex;+ width: fit-content;+ max-width: 100%;+ min-width: 0;+ align-items: center;+ justify-content: space-between;+ gap: 0.75rem;+ padding: 0.65rem 0.75rem;+ border: 1px solid var(--border);+ border-radius: var(--radius);+ color: var(--text);+ background: var(--surface-subtle);+ cursor: copy;+}++.copy-chip:hover {+ border-color: var(--border-strong);+ background: #202126;+}++.copy-chip code {+ overflow: hidden;+ font:+ 600 0.7rem/1.4 ui-monospace,+ SFMono-Regular,+ Menlo,+ monospace;+ text-overflow: ellipsis;+ white-space: nowrap;+}++.copy-label {+ flex: none;+ color: var(--muted);+ font-size: 0.7rem;+ font-weight: 650;+}++.fine-print {+ margin: 1.25rem 0 0;+ color: var(--subtle);+ font-size: 0.7rem;+ line-height: 1.5;+ text-align: left;+}++@keyframes spin {+ to {+ transform: rotate(360deg);+ }+}++@media (max-width: 34rem) {+ .page-shell {+ padding: 1rem;+ }++ .card-header {+ padding: 1rem 1.1rem 0;+ }++ .card-body {+ padding: 2rem 1.1rem 1.1rem;+ }++ .pairing-details {+ grid-template-columns: 1fr;+ }++ .actions {+ flex-wrap: wrap;+ }+}++@media (prefers-reduced-motion: reduce) {+ *,+ *::before,+ *::after {+ scroll-behavior: auto !important;+ transition: none !important;+ animation-duration: 0.01ms !important;+ animation-iteration-count: 1 !important;+ }+}++@media (prefers-color-scheme: light) {+ :root {+ color: #202124;+ background: #f5f5f7;+ --surface: #ffffff;+ --surface-subtle: #f5f5f7;+ --border: #dedfe3;+ --border-strong: #c9cbd1;+ --text: #202124;+ --muted: #5f6169;+ --subtle: #777982;+ }++ body {+ background: #f5f5f7;+ }++ .pairing-card {+ box-shadow: 0 1rem 2.5rem rgb(32 33 36 / 10%);+ }++ .copy-chip:hover {+ background: #ececef;+ }+}diff --git a/skyvern/browser_extension/extension/pairing_confirm.html b/skyvern/browser_extension/extension/pairing_confirm.htmlnew file mode 100644index 000000000..cf737c3cb--- /dev/null+++ b/skyvern/browser_extension/extension/pairing_confirm.html@@ -0,0 +1,139 @@+<!doctype html>+<html lang="en">+ <head>+ <meta charset="utf-8" />+ <meta name="viewport" content="width=device-width, initial-scale=1" />+ <title>Pair Skyvern Agent</title>+ <link rel="stylesheet" href="pairing_confirm.css" />+ </head>+ <body data-state="loading">+ <div class="page-shell">+ <main class="pairing-card" aria-labelledby="pairing-title">+ <header class="card-header">+ <span class="brand-mark" aria-hidden="true">+ <img src="icons/icon128.png" alt="" />+ </span>+ <span class="brand-copy">+ <span class="brand-name">Skyvern Agent</span>+ </span>+ </header>++ <div class="card-body">+ <section class="intro">+ <h1 id="pairing-title">Pair Skyvern Agent</h1>+ <p id="pairing-question" class="question">+ Loading the local pairing offer…+ </p>+ </section>++ <div class="pairing-details" aria-label="Pairing details">+ <div class="detail-chip">+ <span class="detail-copy">+ <span class="detail-label">Local server</span>+ <code id="pairing-port">127.0.0.1:-----</code>+ </span>+ </div>++ <div class="detail-chip">+ <span class="detail-copy">+ <span class="detail-label">Token fingerprint</span>+ <code id="pairing-fingerprint">--------</code>+ </span>+ </div>+ </div>++ <div class="security-note">+ <p>Confirm the fingerprint matches the local server you started.</p>+ </div>++ <div class="actions">+ <button+ id="cancel-button"+ class="secondary-button"+ type="button"+ disabled+ >+ Cancel+ </button>+ <button+ id="approve-button"+ class="primary-button"+ type="button"+ disabled+ >+ <span class="spinner" aria-hidden="true"></span>+ <span id="approve-label">Approve pairing</span>+ </button>+ </div>++ <section+ id="pairing-result"+ class="result-panel"+ role="status"+ aria-live="polite"+ hidden+ >+ <div class="result-heading">+ <span class="result-icon" aria-hidden="true">+ <svg+ class="success-icon"+ viewBox="0 0 24 24"+ fill="none"+ stroke="currentColor"+ stroke-width="2.4"+ stroke-linecap="round"+ stroke-linejoin="round"+ >+ <path d="m7 12.5 3.2 3.2L17.5 8.5" />+ </svg>+ <svg+ class="error-icon"+ viewBox="0 0 24 24"+ fill="none"+ stroke="currentColor"+ stroke-width="2"+ stroke-linecap="round"+ >+ <path d="M12 8v5M12 17h.01" />+ <circle cx="12" cy="12" r="9" />+ </svg>+ <svg+ class="neutral-icon"+ viewBox="0 0 24 24"+ fill="none"+ stroke="currentColor"+ stroke-width="2"+ stroke-linecap="round"+ >+ <path d="M7 12h10" />+ <circle cx="12" cy="12" r="9" />+ </svg>+ </span>+ <h2 id="result-title" class="result-title"></h2>+ </div>+ <p id="result-message" class="result-message"></p>+ <div id="recovery" class="recovery" hidden>+ <span class="recovery-label">Start a new pairing link</span>+ <button+ id="copy-command"+ class="copy-chip"+ type="button"+ aria-label="Copy skyvern browser extension-pair"+ >+ <code id="recovery-command"+ >skyvern browser extension-pair</code+ >+ <span id="copy-label" class="copy-label">Copy</span>+ </button>+ </div>+ </section>++ <p class="fine-print">+ This offer is local to your computer and can be approved once.+ </p>+ </div>+ </main>+ </div>+ <script type="module" src="pairing_confirm.js"></script>+ </body>+</html>diff --git a/skyvern/browser_extension/extension/pairing_confirm.js b/skyvern/browser_extension/extension/pairing_confirm.jsnew file mode 100644index 000000000..cf8a0d941--- /dev/null+++ b/skyvern/browser_extension/extension/pairing_confirm.js@@ -0,0 +1,213 @@+const PENDING_PAIRING_STORAGE_KEY = "pendingPairingOffer";+const RECOVERY_COMMAND = "skyvern browser extension-pair";++const elements = {+ question: document.querySelector("#pairing-question"),+ port: document.querySelector("#pairing-port"),+ fingerprint: document.querySelector("#pairing-fingerprint"),+ approveButton: document.querySelector("#approve-button"),+ approveLabel: document.querySelector("#approve-label"),+ cancelButton: document.querySelector("#cancel-button"),+ actions: document.querySelector(".actions"),+ result: document.querySelector("#pairing-result"),+ resultTitle: document.querySelector("#result-title"),+ resultMessage: document.querySelector("#result-message"),+ recovery: document.querySelector("#recovery"),+ recoveryCommand: document.querySelector("#recovery-command"),+ copyButton: document.querySelector("#copy-command"),+ copyLabel: document.querySelector("#copy-label"),+};++let renderGeneration = 0;+let operationInProgress = false;++async function sendMessage(message) {+ const response = await chrome.runtime.sendMessage(message);+ if (!response?.ok) {+ throw new Error(+ response?.error?.message || "The extension request failed.",+ );+ }+ return response.result;+}++function isPendingOffer(offer) {+ return (+ offer !== null &&+ typeof offer === "object" &&+ !Array.isArray(offer) &&+ Number.isInteger(offer.port) &&+ offer.port >= 1 &&+ offer.port <= 65_535 &&+ typeof offer.token === "string" &&+ offer.token.length > 0+ );+}++async function fingerprint(token) {+ const digest = await crypto.subtle.digest(+ "SHA-256",+ new TextEncoder().encode(token),+ );+ return Array.from(new Uint8Array(digest), (byte) =>+ byte.toString(16).padStart(2, "0"),+ )+ .join("")+ .slice(0, 8);+}++function showResult({+ state,+ title,+ message,+ recoverable = false,+ terminal = false,+}) {+ document.body.dataset.state = state;+ elements.actions.dataset.terminal = String(terminal);+ elements.result.hidden = false;+ elements.resultTitle.textContent = title;+ elements.resultMessage.textContent = message;+ elements.recovery.hidden = !recoverable;+}++function clearResult() {+ document.body.dataset.state = "idle";+ elements.actions.dataset.terminal = "false";+ elements.result.hidden = true;+ elements.resultTitle.textContent = "";+ elements.resultMessage.textContent = "";+ elements.recovery.hidden = true;+}++async function copyRecoveryCommand() {+ try {+ await navigator.clipboard.writeText(RECOVERY_COMMAND);+ } catch (_error) {+ const range = document.createRange();+ range.selectNodeContents(elements.recoveryCommand);+ const selection = globalThis.getSelection();+ selection.removeAllRanges();+ selection.addRange(range);+ document.execCommand("copy");+ selection.removeAllRanges();+ }+ elements.copyLabel.textContent = "Copied";+ setTimeout(() => {+ elements.copyLabel.textContent = "Copy";+ }, 1600);+}++async function renderPendingOffer() {+ const generation = ++renderGeneration;+ const stored = await chrome.storage.session.get(PENDING_PAIRING_STORAGE_KEY);+ const offer = stored[PENDING_PAIRING_STORAGE_KEY];+ if (!isPendingOffer(offer)) {+ elements.question.textContent =+ "This one-time pairing offer is no longer available.";+ elements.port.textContent = "Unavailable";+ elements.fingerprint.textContent = "--------";+ elements.approveButton.disabled = true;+ elements.cancelButton.disabled = true;+ showResult({+ state: "error",+ title: "Pairing request expired",+ message: "Start a new pairing request from your terminal to continue.",+ recoverable: true,+ terminal: true,+ });+ return;+ }+ const tokenFingerprint = await fingerprint(offer.token);+ if (generation !== renderGeneration) {+ return;+ }+ elements.question.textContent =+ "Skyvern on this computer wants to drive Chrome through this extension.";+ elements.port.textContent = `127.0.0.1:${offer.port}`;+ elements.fingerprint.textContent = tokenFingerprint;+ elements.approveButton.disabled = false;+ elements.cancelButton.disabled = false;+ clearResult();+}++function disableActions() {+ elements.approveButton.disabled = true;+ elements.cancelButton.disabled = true;+}++function finishBusyState() {+ elements.approveButton.removeAttribute("aria-busy");+ elements.approveLabel.textContent = "Approve pairing";+}++elements.approveButton.addEventListener("click", () => {+ operationInProgress = true;+ disableActions();+ document.body.dataset.state = "approving";+ elements.approveButton.setAttribute("aria-busy", "true");+ elements.approveLabel.textContent = "Connecting…";+ elements.result.hidden = true;+ void sendMessage({ type: "applyPendingPairing" })+ .then(() => {+ finishBusyState();+ showResult({+ state: "success",+ title: "Connected",+ message:+ "Skyvern Agent is now paired with your local MCP server. You can close this tab.",+ terminal: true,+ });+ })+ .catch((error) => {+ operationInProgress = false;+ finishBusyState();+ showResult({+ state: "error",+ title: "Connection failed",+ message: error.message,+ });+ elements.approveButton.disabled = false;+ elements.cancelButton.disabled = false;+ });+});++elements.cancelButton.addEventListener("click", () => {+ operationInProgress = true;+ disableActions();+ elements.result.hidden = true;+ void sendMessage({ type: "cancelPendingPairing" })+ .then(() => {+ showResult({+ state: "cancelled",+ title: "Pairing cancelled",+ message: "No connection was made. You can close this tab.",+ terminal: true,+ });+ })+ .catch((error) => {+ operationInProgress = false;+ showResult({+ state: "error",+ title: "Couldn’t cancel pairing",+ message: error.message,+ });+ elements.cancelButton.disabled = false;+ });+});++elements.copyButton.addEventListener("click", () => {+ void copyRecoveryCommand();+});++chrome.storage.onChanged.addListener((changes, areaName) => {+ if (+ !operationInProgress &&+ areaName === "session" &&+ PENDING_PAIRING_STORAGE_KEY in changes+ ) {+ void renderPendingOffer();+ }+});++await renderPendingOffer();diff --git a/skyvern/browser_extension/extension/popup.css b/skyvern/browser_extension/extension/popup.cssnew file mode 100644index 000000000..f6cbd627d--- /dev/null+++ b/skyvern/browser_extension/extension/popup.css@@ -0,0 +1,274 @@+:root {+ color-scheme: light dark;+ font-family:+ ui-sans-serif,+ system-ui,+ -apple-system,+ BlinkMacSystemFont,+ "Segoe UI",+ sans-serif;+ color: #202033;+ background: #ffffff;+}++body {+ width: 340px;+ margin: 0;+}++main {+ padding: 16px;+}++header,+.status,+li {+ display: flex;+ align-items: center;+}++header,+li {+ justify-content: space-between;+ gap: 12px;+}++h1 {+ margin: 0;+ font-size: 18px;+}++h2 {+ margin: 0 0 10px;+ font-size: 13px;+}++section {+ margin-top: 16px;+ padding-top: 14px;+ border-top: 1px solid #dddded;+}++.status {+ gap: 6px;+ font-size: 12px;+}++.status-dot {+ width: 8px;+ height: 8px;+ border-radius: 50%;+ background: #8a8a9a;+}++.status-dot.connected {+ background: #35a66f;+}++.settings-row {+ display: grid;+ grid-template-columns: 88px 1fr;+ gap: 8px;+ align-items: center;+}++.settings {+ display: grid;+ gap: 10px;+}++.advanced-settings {+ font-size: 12px;+}++.advanced-settings summary {+ color: #77778b;+ cursor: pointer;+ user-select: none;+}++.advanced-settings summary:hover {+ color: #6d6cf6;+}++.advanced-settings-row {+ margin-top: 10px;+}++label {+ font-size: 12px;+}++input,+button {+ box-sizing: border-box;+ border: 1px solid #c9c9dc;+ border-radius: 6px;+ font: inherit;+}++input {+ min-width: 0;+ padding: 7px 8px;+ background: transparent;+}++.token-field {+ position: relative;+ min-width: 0;+}++.token-field input {+ width: 100%;+ padding-right: 34px;+}++.token-visibility {+ position: absolute;+ top: 50%;+ right: 3px;+ display: grid;+ width: 28px;+ height: 28px;+ padding: 5px;+ border: 0;+ background: transparent;+ place-items: center;+ transform: translateY(-50%);+}++.token-visibility svg {+ width: 16px;+ height: 16px;+}++.token-visibility [hidden] {+ display: none;+}++button {+ padding: 7px 10px;+ color: inherit;+ background: #f3f3fb;+ cursor: pointer;+}++button:hover:not(:disabled) {+ border-color: #6d6cf6;+}++button:disabled {+ cursor: not-allowed;+ opacity: 0.55;+}++.primary {+ color: #ffffff;+ border-color: #6d6cf6;+ background: #6d6cf6;+}++.message {+ min-height: 16px;+ margin: 6px 0 0;+ color: #b43b4a;+ font-size: 11px;+}++.settings .message {+ margin: 0;+}++ul {+ display: grid;+ gap: 8px;+ margin: 0;+ padding: 0;+ list-style: none;+}++li {+ min-width: 0;+}++.tab-title {+ overflow: hidden;+ font-size: 12px;+ text-overflow: ellipsis;+ white-space: nowrap;+}++.muted {+ margin: 0;+ color: #77778b;+ font-size: 12px;+}++.tab-group-hint {+ margin: 10px 0 0;+ color: #77778b;+ font-size: 10px;+ white-space: nowrap;+}++.coming-soon {+ padding-top: 12px;+}++.coming-soon h2 {+ margin-bottom: 7px;+ color: #77778b;+ font-size: 11px;+ letter-spacing: 0.04em;+}++.coming-soon-list {+ gap: 4px;+ color: #9999a8;+ font-size: 11px;+ cursor: not-allowed;+ opacity: 0.65;+}++.coming-soon-list li {+ justify-content: flex-start;+ gap: 6px;+}++@media (prefers-color-scheme: dark) {+ :root {+ color: #f2f2fa;+ background: #1c1c26;+ }++ section {+ border-color: #383848;+ }++ input,+ button {+ border-color: #4c4c61;+ }++ button {+ background: #30303f;+ }++ .primary {+ background: #6d6cf6;+ }++ .muted {+ color: #a7a7ba;+ }++ .advanced-settings summary,+ .tab-group-hint,+ .coming-soon h2 {+ color: #a7a7ba;+ }++ .coming-soon-list {+ color: #b9b9c8;+ }+}diff --git a/skyvern/browser_extension/extension/popup.html b/skyvern/browser_extension/extension/popup.htmlnew file mode 100644index 000000000..4a98079b6--- /dev/null+++ b/skyvern/browser_extension/extension/popup.html@@ -0,0 +1,119 @@+<!doctype html>+<html lang="en">+ <head>+ <meta charset="utf-8" />+ <meta name="viewport" content="width=device-width, initial-scale=1" />+ <title>Skyvern Agent</title>+ <link rel="stylesheet" href="popup.css" />+ </head>+ <body>+ <main>+ <header>+ <h1>Skyvern Agent</h1>+ <div class="status" aria-live="polite">+ <span id="status-dot" class="status-dot"></span>+ <span id="status-label">Disconnected</span>+ </div>+ </header>++ <section class="settings" aria-label="Agent settings">+ <div class="settings-row">+ <label for="pairing-token">Pairing token</label>+ <div class="token-field">+ <input+ id="pairing-token"+ type="password"+ autocomplete="off"+ spellcheck="false"+ />+ <button+ id="pairing-token-visibility"+ class="token-visibility"+ type="button"+ aria-label="Show pairing token"+ aria-pressed="false"+ >+ <svg+ class="eye-icon"+ aria-hidden="true"+ viewBox="0 0 24 24"+ fill="none"+ stroke="currentColor"+ stroke-width="2"+ >+ <path d="M2 12s3.5-7 10-7 10 7 10 7-3.5 7-10 7S2 12 2 12Z" />+ <circle cx="12" cy="12" r="3" />+ </svg>+ <svg+ class="eye-off-icon"+ aria-hidden="true"+ viewBox="0 0 24 24"+ fill="none"+ stroke="currentColor"+ stroke-width="2"+ hidden+ >+ <path d="m3 3 18 18" />+ <path+ d="M10.6 5.2A11.4 11.4 0 0 1 12 5c6.5 0 10 7 10 7a18 18 0 0 1-2.1 3.2"+ />+ <path+ d="M6.2 6.2C3.5 8.1 2 12 2 12s3.5 7 10 7a10.7 10.7 0 0 0 4.1-.8"+ />+ </svg>+ </button>+ </div>+ </div>+ <details id="advanced-settings" class="advanced-settings">+ <summary>Advanced settings</summary>+ <div class="settings-row advanced-settings-row">+ <label for="bridge-port">Port</label>+ <input+ id="bridge-port"+ type="number"+ min="1"+ max="65535"+ inputmode="numeric"+ />+ </div>+ </details>+ <button id="connection-button" class="primary" type="button">+ Connect+ </button>+ <p id="connection-error" class="message" role="alert"></p>+ </section>++ <section aria-labelledby="current-tab-title">+ <h2 id="current-tab-title">Current tab</h2>+ <button id="share-button" type="button">+ Add to Skyvern Controlled+ </button>+ <p id="share-reason" class="message"></p>+ </section>++ <section aria-labelledby="shared-tabs-title">+ <h2 id="shared-tabs-title">Skyvern Controlled tabs</h2>+ <ul id="shared-tabs"></ul>+ <p id="empty-tabs" class="muted">No tabs in Skyvern Controlled.</p>+ <p class="tab-group-hint">+ Tabs in the Skyvern Controlled group can be driven by Skyvern.+ </p>+ </section>++ <section class="coming-soon" aria-labelledby="coming-soon-title">+ <h2 id="coming-soon-title">Coming soon</h2>+ <ul class="coming-soon-list" aria-disabled="true">+ <li>+ <span aria-hidden="true">🎬</span> Record actions → create workflows+ </li>+ <li><span aria-hidden="true">📋</span> Record SOPs</li>+ <li><span aria-hidden="true">👤</span> Sync browser profiles</li>+ <li>+ <span aria-hidden="true">🔑</span> Push credentials to Skyvern+ </li>+ </ul>+ </section>+ </main>+ <script type="module" src="popup.js"></script>+ </body>+</html>diff --git a/skyvern/browser_extension/extension/popup.js b/skyvern/browser_extension/extension/popup.jsnew file mode 100644index 000000000..d60185689--- /dev/null+++ b/skyvern/browser_extension/extension/popup.js@@ -0,0 +1,186 @@+import { DEFAULT_BRIDGE_PORT, isRestrictedUrl } from "./protocol.js";++const elements = {+ statusDot: document.querySelector("#status-dot"),+ statusLabel: document.querySelector("#status-label"),+ advancedSettings: document.querySelector("#advanced-settings"),+ port: document.querySelector("#bridge-port"),+ token: document.querySelector("#pairing-token"),+ tokenVisibility: document.querySelector("#pairing-token-visibility"),+ eyeIcon: document.querySelector(".eye-icon"),+ eyeOffIcon: document.querySelector(".eye-off-icon"),+ connectionButton: document.querySelector("#connection-button"),+ connectionError: document.querySelector("#connection-error"),+ shareButton: document.querySelector("#share-button"),+ shareReason: document.querySelector("#share-reason"),+ sharedTabs: document.querySelector("#shared-tabs"),+ emptyTabs: document.querySelector("#empty-tabs"),+};++let status = { connected: false, clientAttached: false, enabled: false };+let activeTab = null;+let scopedTabs = [];+let configTimer = null;+let configLoaded = false;++async function sendMessage(message) {+ const response = await chrome.runtime.sendMessage(message);+ if (!response?.ok) {+ throw new Error(+ response?.error?.message || "The extension request failed.",+ );+ }+ return response.result;+}++function loadConfig(config) {+ elements.port.value = String(config.bridgePort);+ elements.advancedSettings.open =+ Number(config.bridgePort) !== DEFAULT_BRIDGE_PORT;+ elements.token.value =+ typeof config.pairingToken === "string" ? config.pairingToken : "";+ configLoaded = true;+}++async function saveConfig() {+ window.clearTimeout(configTimer);+ configTimer = null;+ await sendMessage({+ type: "setConfig",+ config: {+ bridgePort: Number(elements.port.value),+ pairingToken: elements.token.value,+ },+ });+}++function scheduleConfigSave() {+ window.clearTimeout(configTimer);+ configTimer = window.setTimeout(() => {+ void saveConfig().catch(showConnectionError);+ }, 250);+}++function showConnectionError(error) {+ elements.connectionError.textContent =+ error instanceof Error ? error.message : "The extension request failed.";+}++function setTokenVisibility(visible) {+ elements.token.type = visible ? "text" : "password";+ elements.tokenVisibility.setAttribute("aria-pressed", String(visible));+ elements.tokenVisibility.setAttribute(+ "aria-label",+ visible ? "Hide pairing token" : "Show pairing token",+ );+ elements.eyeIcon.hidden = visible;+ elements.eyeOffIcon.hidden = !visible;+}++function renderStatus() {+ const label = status.clientAttached+ ? "Client attached"+ : status.connected+ ? "Connected"+ : "Disconnected";+ elements.statusLabel.textContent = label;+ elements.statusDot.className = `status-dot ${status.connected ? "connected" : ""}`;+ elements.connectionButton.textContent = status.enabled+ ? "Disconnect"+ : "Connect";+}++function renderCurrentTab() {+ const restricted =+ activeTab === null ||+ isRestrictedUrl(activeTab.pendingUrl || activeTab.url || "");+ const alreadyShared =+ activeTab !== null && scopedTabs.some((tab) => tab.tabId === activeTab.id);+ elements.shareButton.disabled = restricted || alreadyShared;+ if (restricted) {+ elements.shareReason.textContent =+ "Chrome does not allow adding this page.";+ } else if (alreadyShared) {+ elements.shareReason.textContent =+ "This tab is already in Skyvern Controlled.";+ } else {+ elements.shareReason.textContent = "";+ }+}++function renderScopedTabs() {+ elements.sharedTabs.replaceChildren();+ elements.emptyTabs.hidden = scopedTabs.length > 0;+ for (const tab of scopedTabs) {+ const item = document.createElement("li");+ const text = document.createElement("span");+ const button = document.createElement("button");+ text.className = "tab-title";+ text.textContent = tab.title || tab.url || `Tab ${tab.tabId}`;+ text.title = tab.url || "";+ button.type = "button";+ button.textContent = "Remove from Skyvern Controlled";+ button.addEventListener("click", () => {+ button.disabled = true;+ void sendMessage({ type: "unshareTab", tabId: tab.tabId })+ .then(refresh)+ .catch((error) => {+ button.disabled = false;+ showConnectionError(error);+ });+ });+ item.append(text, button);+ elements.sharedTabs.append(item);+ }+}++async function refresh() {+ const [nextStatus, scopedResult, activeTabs] = await Promise.all([+ sendMessage({ type: "getStatus" }),+ sendMessage({ type: "listScoped" }),+ chrome.tabs.query({ active: true, currentWindow: true }),+ ]);+ status = nextStatus;+ if (!configLoaded) {+ loadConfig(nextStatus);+ }+ scopedTabs = scopedResult.tabs;+ activeTab = activeTabs[0] ?? null;+ elements.connectionError.textContent = status.lastError || "";+ renderStatus();+ renderCurrentTab();+ renderScopedTabs();+}++elements.port.addEventListener("input", scheduleConfigSave);+elements.token.addEventListener("input", scheduleConfigSave);+elements.tokenVisibility.addEventListener("click", () => {+ setTokenVisibility(elements.token.type === "password");+});+elements.connectionButton.addEventListener("click", () => {+ elements.connectionButton.disabled = true;+ elements.connectionError.textContent = "";+ const action = status.enabled+ ? sendMessage({ type: "disconnect" })+ : saveConfig().then(() => sendMessage({ type: "connect" }));+ void action+ .then(refresh)+ .catch(showConnectionError)+ .finally(() => {+ elements.connectionButton.disabled = false;+ });+});+elements.shareButton.addEventListener("click", () => {+ if (activeTab === null || !Number.isInteger(activeTab.id)) {+ return;+ }+ elements.shareButton.disabled = true;+ void sendMessage({ type: "shareTab", tabId: activeTab.id })+ .then(refresh)+ .catch(showConnectionError);+});++await refresh();+window.setInterval(() => {+ void refresh().catch(showConnectionError);+}, 2_000);diff --git a/skyvern/browser_extension/extension/protocol.js b/skyvern/browser_extension/extension/protocol.jsnew file mode 100644index 000000000..c21a5132b--- /dev/null+++ b/skyvern/browser_extension/extension/protocol.js@@ -0,0 +1,153 @@+export const PROTOCOL_VERSION = 1;+export const DEFAULT_BRIDGE_PORT = 19777;+export const BRIDGE_ALARM_NAME = "skyvern-bridge-reconnect";++export const MESSAGE_TYPES = Object.freeze({+ AUTH_CHALLENGE: "auth.challenge",+ AUTH_PROOF: "auth.proof",+ AUTH_OK: "auth.ok",+ REQUEST: "request",+ RESPONSE: "response",+ EVENT: "event",+ PING: "ping",+ PONG: "pong",+});++export const OPS = Object.freeze({+ DEBUGGER_ATTACH: "debugger.attach",+ DEBUGGER_DETACH: "debugger.detach",+ DEBUGGER_SEND: "debugger.send",+ SHARE_TAB: "shareTab",+ UNSHARE_TAB: "unshareTab",+ TABS_CREATE: "tabs.create",+ TABS_REMOVE: "tabs.remove",+ TABS_ACTIVATE: "tabs.activate",+ TABS_LIST: "tabs.list",+});++export const EVENTS = Object.freeze({+ EXTENSION_HELLO: "extension.hello",+ DEBUGGER_EVENT: "debugger.event",+ DEBUGGER_DETACHED: "debugger.detached",+ SCOPE_TAB_ADDED: "scope.tabAdded",+ SCOPE_TAB_REMOVED: "scope.tabRemoved",+ TABS_CREATED: "tabs.created",+});++export const ERROR_CODES = Object.freeze({+ AUTH_FAILED: "AUTH_FAILED",+ OP_NOT_ALLOWED: "OP_NOT_ALLOWED",+ TAB_NOT_FOUND: "TAB_NOT_FOUND",+ TAB_NOT_SCOPED: "TAB_NOT_SCOPED",+ RESTRICTED_URL: "RESTRICTED_URL",+ DEBUGGER_DETACHED: "DEBUGGER_DETACHED",+ CDP_METHOD_NOT_ALLOWED: "CDP_METHOD_NOT_ALLOWED",+ CDP_ERROR: "CDP_ERROR",+ INTERNAL: "INTERNAL",+});++export const ALLOWED_CDP_PREFIXES = new Set([+ "Accessibility",+ "Animation",+ "CSS",+ "Console",+ "DOM",+ "DOMDebugger",+ "DOMSnapshot",+ "DOMStorage",+ "Debugger",+ "Emulation",+ "Fetch",+ "IO",+ "Input",+ "Inspector",+ "Log",+ "Network",+ "Overlay",+ "Page",+ "Performance",+ "Profiler",+ "Runtime",+ "Security",+ "Storage",+ "Target",+]);++export const DENIED_CDP_METHODS = new Set([+ "Network.getAllCookies",+ "Network.clearBrowserCookies",+ "Network.clearBrowserCache",+ "Storage.getCookies",+ "Storage.setCookies",+ "Storage.clearCookies",+]);++export class ProtocolError extends Error {+ constructor(code, message) {+ super(message);+ this.name = "ProtocolError";+ this.code = code;+ }+}++export function isRestrictedUrl(url) {+ if (typeof url !== "string" || url.length === 0) {+ return false;+ }++ const normalized = url.trim().toLowerCase();+ if (normalized === "about:blank") {+ return false;+ }+ if (+ normalized.startsWith("chrome://") ||+ normalized.startsWith("chrome-untrusted://") ||+ normalized.startsWith("chrome-extension://") ||+ normalized.startsWith("devtools://") ||+ normalized.startsWith("edge://") ||+ normalized.startsWith("about:") ||+ normalized.startsWith("file://")+ ) {+ return true;+ }++ try {+ const hostname = new URL(url).hostname.toLowerCase().replace(/\.$/, "");+ return hostname === "chromewebstore.google.com";+ } catch {+ return false;+ }+}++export function requireArgs(args) {+ if (args === null || typeof args !== "object" || Array.isArray(args)) {+ throw new ProtocolError(+ ERROR_CODES.INTERNAL,+ "Request arguments must be an object.",+ );+ }+ return args;+}++export function requireTabId(value) {+ if (!Number.isInteger(value) || value < 0) {+ throw new ProtocolError(+ ERROR_CODES.TAB_NOT_FOUND,+ "The requested tab was not found.",+ );+ }+ return value;+}++export function protocolErrorEnvelope(error) {+ if (+ error instanceof ProtocolError &&+ Object.values(ERROR_CODES).includes(error.code)+ ) {+ return { code: error.code, message: error.message };+ }+ return {+ code: ERROR_CODES.INTERNAL,+ message: "The extension operation failed.",+ };+}diff --git a/skyvern/browser_extension/extension/service_worker.js b/skyvern/browser_extension/extension/service_worker.jsnew file mode 100644index 000000000..f170ae943--- /dev/null+++ b/skyvern/browser_extension/extension/service_worker.js@@ -0,0 +1,274 @@+import { BridgeConnection } from "./bridge_connection.js";+import { DebuggerRouter } from "./debugger_router.js";+import {+ ERROR_CODES,+ EVENTS,+ OPS,+ PROTOCOL_VERSION,+ ProtocolError,+ requireArgs,+} from "./protocol.js";+import { TabScope } from "./tab_scope.js";++const PENDING_PAIRING_STORAGE_KEY = "pendingPairingOffer";+const PAIRING_CONFIRM_PATH = "pairing_confirm.html";++let debuggerRouter;+let pendingPairingOffer = null;+let pairingOfferQueue = Promise.resolve();++const bridge = new BridgeConnection({+ onRequest: (op, args) => dispatchRequest(op, args),+ onAuthenticated: () => sendHello(),+ onStateChange: () => updateActionState(),+});++const tabScope = new TabScope({+ sendEvent: (event, params) => bridge.sendEvent(event, params),+});++debuggerRouter = new DebuggerRouter({+ tabScope,+ sendEvent: (event, params) => bridge.sendEvent(event, params),+ onAttachedChange: () => updateActionState(),+});+tabScope.setDebuggerRouter(debuggerRouter);++const handlers = new Map([+ [OPS.DEBUGGER_ATTACH, (args) => debuggerRouter.attach(args)],+ [OPS.DEBUGGER_DETACH, (args) => debuggerRouter.detach(args)],+ [OPS.DEBUGGER_SEND, (args) => debuggerRouter.send(args)],+ [OPS.TABS_CREATE, (args) => tabScope.create(args)],+ [OPS.TABS_REMOVE, (args) => tabScope.remove(args)],+ [OPS.TABS_ACTIVATE, (args) => tabScope.activate(args)],+ [+ OPS.TABS_LIST,+ (args) => {+ requireArgs(args);+ return tabScope.list();+ },+ ],+]);++async function dispatchRequest(op, args) {+ const handler = handlers.get(op);+ if (handler === undefined) {+ throw new ProtocolError(+ ERROR_CODES.OP_NOT_ALLOWED,+ "The requested operation is not allowed.",+ );+ }+ return handler(args);+}++async function sendHello() {+ bridge.sendEvent(EVENTS.EXTENSION_HELLO, {+ extensionVersion: chrome.runtime.getManifest().version,+ scopedTabs: await tabScope.helloTabs(),+ });+}++async function updateActionState() {+ const connected = bridge.getStatus().connected;+ await chrome.action.setBadgeText({ text: connected ? "ON" : "" });+ await chrome.action.setBadgeBackgroundColor({ color: "#6D6CF6" });+}++async function popupStatus() {+ const status = bridge.getStatus();+ return {+ ...status,+ clientAttached: status.connected && debuggerRouter.hasAttachedTabs(),+ protocolVersion: PROTOCOL_VERSION,+ };+}++function parsePairingOffer(message, sender) {+ if (+ message === null ||+ typeof message !== "object" ||+ Array.isArray(message) ||+ message.type !== "skyvern.pairingOffer" ||+ message.v !== 1 ||+ !Number.isInteger(message.port) ||+ message.port < 1 ||+ message.port > 65_535 ||+ typeof message.token !== "string" ||+ message.token.length === 0+ ) {+ return null;+ }++ let senderUrl;+ try {+ senderUrl = new URL(sender.url);+ } catch {+ return null;+ }+ if (+ senderUrl.protocol !== "http:" ||+ (senderUrl.hostname !== "127.0.0.1" &&+ senderUrl.hostname !== "localhost") ||+ senderUrl.port === "" ||+ Number(senderUrl.port) !== message.port+ ) {+ return null;+ }+ return { port: message.port, token: message.token };+}++async function focusPairingConfirmation() {+ const confirmationUrl = chrome.runtime.getURL(PAIRING_CONFIRM_PATH);+ const tabs = await chrome.tabs.query({});+ const existingTab = tabs.find((tab) => tab.url === confirmationUrl);+ if (Number.isInteger(existingTab?.id)) {+ await chrome.tabs.update(existingTab.id, { active: true });+ if (+ Number.isInteger(existingTab.windowId) &&+ typeof chrome.windows?.update === "function"+ ) {+ await chrome.windows.update(existingTab.windowId, { focused: true });+ }+ return;+ }+ await chrome.tabs.create({ active: true, url: confirmationUrl });+}++async function acceptPairingOffer(offer) {+ pendingPairingOffer = offer;+ await chrome.storage.session.set({ [PENDING_PAIRING_STORAGE_KEY]: offer });+ await focusPairingConfirmation();+}++function enqueuePairingOperation(operation) {+ const result = pairingOfferQueue+ .catch(() => undefined)+ .then(() => operation());+ pairingOfferQueue = result;+ return result;+}++async function readPendingPairingOffer() {+ if (pendingPairingOffer !== null) {+ return pendingPairingOffer;+ }+ const stored = await chrome.storage.session.get(PENDING_PAIRING_STORAGE_KEY);+ const offer = stored[PENDING_PAIRING_STORAGE_KEY];+ if (+ offer === null ||+ typeof offer !== "object" ||+ Array.isArray(offer) ||+ !Number.isInteger(offer.port) ||+ offer.port < 1 ||+ offer.port > 65_535 ||+ typeof offer.token !== "string" ||+ offer.token.length === 0+ ) {+ return null;+ }+ pendingPairingOffer = { port: offer.port, token: offer.token };+ return pendingPairingOffer;+}++async function clearPendingPairingOffer() {+ pendingPairingOffer = null;+ await chrome.storage.session.remove(PENDING_PAIRING_STORAGE_KEY);+}++async function applyPendingPairing() {+ const offer = await readPendingPairingOffer();+ if (offer === null) {+ throw new ProtocolError(+ ERROR_CODES.INTERNAL,+ "No pending pairing offer is available.",+ );+ }+ await bridge.setConfig({+ bridgePort: offer.port,+ pairingToken: offer.token,+ });+ await clearPendingPairingOffer();+ return bridge.connect();+}++async function handlePopupMessage(message) {+ if (+ message === null ||+ typeof message !== "object" ||+ Array.isArray(message)+ ) {+ throw new ProtocolError(+ ERROR_CODES.INTERNAL,+ "The popup request is invalid.",+ );+ }+ switch (message.type) {+ case "getStatus":+ return popupStatus();+ case "setConfig":+ return bridge.setConfig(requireArgs(message.config));+ case "applyPendingPairing":+ return enqueuePairingOperation(applyPendingPairing);+ case "cancelPendingPairing":+ return enqueuePairingOperation(clearPendingPairingOffer);+ case "connect":+ return bridge.connect();+ case "disconnect":+ return bridge.disconnect();+ case "shareTab":+ return tabScope.shareTab(message.tabId);+ case "unshareTab":+ return tabScope.unshareTab(message.tabId);+ case "listScoped":+ return tabScope.list();+ default:+ throw new ProtocolError(+ ERROR_CODES.OP_NOT_ALLOWED,+ "The popup operation is not allowed.",+ );+ }+}++chrome.runtime.onMessageExternal.addListener(+ (message, sender, sendResponse) => {+ const offer = parsePairingOffer(message, sender);+ if (offer === null) {+ sendResponse({ ok: false, error: "invalid_offer" });+ return false;+ }++ const acceptance = enqueuePairingOperation(() => acceptPairingOffer(offer));+ void acceptance+ .then(() => sendResponse({ ok: true, pending: true }))+ .catch(() => sendResponse({ ok: false, error: "pairing_failed" }));+ return true;+ },+);++// Chrome fails the whole service worker if module evaluation uses top-level await+// (event dispatch never starts), so initialization must run as a tracked promise.+const initialized = (async () => {+ await tabScope.initialize();+ await bridge.initialize();+ await updateActionState();+})();+initialized.catch((error) =>+ console.error("Skyvern bridge initialization failed", error),+);++chrome.runtime.onMessage.addListener((message, _sender, sendResponse) => {+ void initialized+ .catch(() => undefined)+ .then(() => handlePopupMessage(message))+ .then((result) => sendResponse({ ok: true, result }))+ .catch((error) => {+ const code =+ error instanceof ProtocolError ? error.code : ERROR_CODES.INTERNAL;+ const messageText =+ error instanceof ProtocolError+ ? error.message+ : "The extension operation failed.";+ sendResponse({ ok: false, error: { code, message: messageText } });+ });+ return true;+});diff --git a/skyvern/browser_extension/extension/tab_scope.js b/skyvern/browser_extension/extension/tab_scope.jsnew file mode 100644index 000000000..ac8677802--- /dev/null+++ b/skyvern/browser_extension/extension/tab_scope.js@@ -0,0 +1,613 @@+import {+ ERROR_CODES,+ EVENTS,+ ProtocolError,+ isRestrictedUrl,+ requireArgs,+ requireTabId,+} from "./protocol.js";++const SCOPED_TAB_IDS_KEY = "scopedTabIds";+const SCOPED_GROUP_IDS_KEY = "scopedTabGroupIds";+const SKYVERN_GROUP_TITLE = "Skyvern Controlled";+const SKYVERN_GROUP_COLOR = "purple";+const TAB_GROUP_ID_NONE = -1;+const ANY_GROUP_ID = Symbol("anyGroupId");++function tabUrl(tab) {+ return tab.pendingUrl || tab.url || "";+}++function isTabRestricted(tab) {+ return (+ isRestrictedUrl(tab.pendingUrl || "") || isRestrictedUrl(tab.url || "")+ );+}++export class TabScope {+ constructor({ sendEvent }) {+ this.sendEvent = sendEvent;+ this.scopedTabIds = new Set();+ this.scopedGroupIds = new Map();+ this.expectedGroupTransitions = new Map();+ this.tabOperations = new Map();+ this.debuggerRouter = null;+ this.ready = new Promise((resolve) => {+ this.resolveReady = resolve;+ });++ chrome.tabs.onCreated.addListener((tab) => {+ void this.handleTabCreated(tab);+ });+ chrome.tabs.onRemoved.addListener((tabId) => {+ void this.handleTabRemoved(tabId);+ });+ chrome.tabs.onUpdated.addListener((tabId, changeInfo) => {+ if (+ Object.hasOwn(changeInfo, "url") ||+ Object.hasOwn(changeInfo, "groupId")+ ) {+ const expectedGroupTransition = Object.hasOwn(changeInfo, "groupId")+ ? this.getExpectedGroupTransition(tabId, changeInfo.groupId)+ : null;+ void this.handleTabUpdated(tabId, changeInfo, expectedGroupTransition);+ }+ });+ }++ setDebuggerRouter(debuggerRouter) {+ this.debuggerRouter = debuggerRouter;+ }++ async initialize() {+ const stored = await chrome.storage.session.get({+ [SCOPED_TAB_IDS_KEY]: [],+ [SCOPED_GROUP_IDS_KEY]: {},+ });+ const storedIds = Array.isArray(stored[SCOPED_TAB_IDS_KEY])+ ? stored[SCOPED_TAB_IDS_KEY]+ : [];+ const storedGroups = stored[SCOPED_GROUP_IDS_KEY];+ for (const tabId of storedIds) {+ if (Number.isInteger(tabId) && tabId >= 0) {+ this.scopedTabIds.add(tabId);+ }+ }+ if (+ storedGroups !== null &&+ typeof storedGroups === "object" &&+ !Array.isArray(storedGroups)+ ) {+ for (const [tabId, groupId] of Object.entries(storedGroups)) {+ const numericTabId = Number(tabId);+ if (this.scopedTabIds.has(numericTabId) && Number.isInteger(groupId)) {+ this.scopedGroupIds.set(numericTabId, groupId);+ }+ }+ }+ this.resolveReady();+ await this.reconcileStoredTabs();+ }++ isScoped(tabId) {+ return this.scopedTabIds.has(tabId);+ }++ async assertScoped(tabId) {+ await this.ready;+ if (!this.scopedTabIds.has(tabId)) {+ throw new ProtocolError(+ ERROR_CODES.TAB_NOT_SCOPED,+ "The requested tab is not shared.",+ );+ }+ }++ async assertControllableLocked(tabId) {+ await this.assertScoped(tabId);+ let tab;+ try {+ tab = await this.getTab(tabId);+ } catch (error) {+ if (this.scopedTabIds.has(tabId)) {+ await this.removeFromScopeLocked(tabId, "closed", true);+ }+ throw error;+ }+ await this.assertScoped(tabId);+ if (isTabRestricted(tab)) {+ await this.removeFromScopeLocked(tabId, "unshared", true);+ throw new ProtocolError(+ ERROR_CODES.RESTRICTED_URL,+ "Chrome does not allow controlling this URL.",+ );+ }+ return tab;+ }++ async shareTab(tabId) {+ await this.ready;+ const validTabId = requireTabId(tabId);+ return this.runTabOperation(validTabId, async () => {+ const tab = await this.getTab(validTabId);+ if (isTabRestricted(tab)) {+ throw new ProtocolError(+ ERROR_CODES.RESTRICTED_URL,+ "Chrome does not allow sharing this URL.",+ );+ }+ if (this.scopedTabIds.has(validTabId)) {+ return {};+ }+ const scopedTab = await this.addToScopeLocked(tab);+ this.sendEvent(EVENTS.SCOPE_TAB_ADDED, this.publicTab(scopedTab, false));+ return {};+ });+ }++ async unshareTab(tabId) {+ await this.ready;+ const validTabId = requireTabId(tabId);+ return this.runTabOperation(validTabId, async () => {+ await this.assertScoped(validTabId);+ await this.removeFromScopeLocked(validTabId, "unshared", true);+ return {};+ });+ }++ async create(args) {+ const values = requireArgs(args);+ const url =+ values.url === undefined || values.url === ""+ ? "about:blank"+ : values.url;+ if (typeof url !== "string" || isRestrictedUrl(url)) {+ throw new ProtocolError(+ ERROR_CODES.RESTRICTED_URL,+ "Chrome does not allow creating this URL.",+ );+ }+ let tab;+ try {+ tab = await chrome.tabs.create({ url });+ } catch {+ throw new ProtocolError(+ ERROR_CODES.INTERNAL,+ "Chrome could not create the tab.",+ );+ }+ if (!Number.isInteger(tab.id)) {+ throw new ProtocolError(+ ERROR_CODES.TAB_NOT_FOUND,+ "Chrome did not return a tab identifier.",+ );+ }+ return this.runTabOperation(tab.id, async () => {+ const scopedTab = await this.addToScopeLocked(tab);+ this.sendEvent(EVENTS.SCOPE_TAB_ADDED, this.publicTab(scopedTab, false));+ return { tabId: tab.id };+ });+ }++ async remove(args) {+ const values = requireArgs(args);+ const tabId = requireTabId(values.tabId);+ return this.runTabOperation(tabId, async () => {+ await this.assertControllableLocked(tabId);+ try {+ await chrome.tabs.remove(tabId);+ } catch {+ if (!this.scopedTabIds.has(tabId)) {+ return {};+ }+ throw new ProtocolError(+ ERROR_CODES.TAB_NOT_FOUND,+ "The requested tab was not found.",+ );+ }+ if (this.scopedTabIds.has(tabId)) {+ await this.removeFromScopeLocked(tabId, "closed", false);+ }+ return {};+ });+ }++ async activate(args) {+ const values = requireArgs(args);+ const tabId = requireTabId(values.tabId);+ return this.runTabOperation(tabId, async () => {+ const tab = await this.assertControllableLocked(tabId);+ try {+ await chrome.tabs.update(tabId, { active: true });+ if (Number.isInteger(tab.windowId)) {+ await chrome.windows.update(tab.windowId, { focused: true });+ }+ } catch {+ throw new ProtocolError(+ ERROR_CODES.TAB_NOT_FOUND,+ "The requested tab was not found.",+ );+ }+ await this.assertControllableLocked(tabId);+ return {};+ });+ }++ async list() {+ await this.ready;+ const tabs = await this.collectScopedTabs(true);+ return { tabs };+ }++ async helloTabs() {+ await this.ready;+ return this.collectScopedTabs(false);+ }++ async handleDebuggerDetachLocked(tabId) {+ await this.ready;+ if (this.scopedTabIds.has(tabId)) {+ await this.removeFromScopeLocked(tabId, "detached", false);+ }+ }++ async handleTabCreated(tab) {+ await this.ready;+ if (!Number.isInteger(tab.id) || !Number.isInteger(tab.openerTabId)) {+ return;+ }+ await this.runTabOperation(tab.id, async () => {+ if (!this.scopedTabIds.has(tab.openerTabId) || isTabRestricted(tab)) {+ return;+ }+ const scopedTab = await this.addToScopeLocked(tab);+ this.sendEvent(EVENTS.TABS_CREATED, {+ tabId: scopedTab.id,+ openerTabId: tab.openerTabId,+ url: tabUrl(scopedTab),+ });+ });+ }++ async handleTabRemoved(tabId) {+ await this.ready;+ await this.runTabOperation(tabId, async () => {+ this.expectedGroupTransitions.delete(tabId);+ if (this.scopedTabIds.has(tabId)) {+ await this.removeFromScopeLocked(tabId, "closed", false);+ }+ });+ }++ async handleTabUpdated(tabId, changeInfo, expectedGroupTransition = null) {+ await this.ready;+ await this.runTabOperation(tabId, async () => {+ try {+ if (+ this.scopedTabIds.has(tabId) &&+ Object.hasOwn(changeInfo, "url") &&+ isRestrictedUrl(changeInfo.url)+ ) {+ await this.removeFromScopeLocked(tabId, "unshared", true);+ return;+ }+ if (+ !Object.hasOwn(changeInfo, "groupId") ||+ expectedGroupTransition !== null+ ) {+ return;+ }++ let tab;+ try {+ tab = await chrome.tabs.get(tabId);+ } catch {+ return;+ }+ if (tab.groupId !== changeInfo.groupId) {+ return;+ }++ const controlledGroup = await this.getControlledGroup(tab.groupId);+ if (this.scopedTabIds.has(tabId)) {+ const expectedGroupId = this.scopedGroupIds.get(tabId);+ if (tab.groupId === expectedGroupId) {+ return;+ }+ if (controlledGroup !== null) {+ this.scopedGroupIds.set(tabId, tab.groupId);+ await this.persistScope();+ await this.updateControlledGroup(tab.groupId);+ return;+ }+ if (expectedGroupId !== undefined) {+ await this.removeFromScopeLocked(tabId, "unshared", true);+ }+ return;+ }++ if (controlledGroup === null) {+ return;+ }+ if (isTabRestricted(tab)) {+ await this.ungroupTabLocked(tabId, tab.groupId);+ return;+ }++ await this.updateControlledGroup(tab.groupId);+ const scopedTab = await this.addGroupedTabToScopeLocked(+ tab,+ tab.groupId,+ );+ this.sendEvent(+ EVENTS.SCOPE_TAB_ADDED,+ this.publicTab(scopedTab, false),+ );+ } finally {+ if (expectedGroupTransition !== null) {+ this.clearExpectedGroupTransition(tabId, expectedGroupTransition);+ }+ }+ });+ }++ async addToScopeLocked(tab) {+ if (!Number.isInteger(tab.id)) {+ throw new ProtocolError(+ ERROR_CODES.TAB_NOT_FOUND,+ "The requested tab was not found.",+ );+ }+ this.scopedTabIds.add(tab.id);+ await this.persistScope();+ await this.groupTabLocked(tab);+ return this.assertControllableLocked(tab.id);+ }++ async addGroupedTabToScopeLocked(tab, groupId) {+ if (!Number.isInteger(tab.id)) {+ throw new ProtocolError(+ ERROR_CODES.TAB_NOT_FOUND,+ "The requested tab was not found.",+ );+ }+ this.scopedTabIds.add(tab.id);+ this.scopedGroupIds.set(tab.id, groupId);+ await this.persistScope();+ return this.assertControllableLocked(tab.id);+ }++ async removeFromScopeLocked(tabId, reason, detach) {+ if (!this.scopedTabIds.delete(tabId)) {+ return;+ }+ this.expectedGroupTransitions.delete(tabId);+ const scopedGroupId = this.scopedGroupIds.get(tabId);+ this.scopedGroupIds.delete(tabId);+ try {+ await this.persistScope();+ } finally {+ await this.ungroupTabLocked(tabId, scopedGroupId);+ try {+ if (detach && this.debuggerRouter !== null) {+ await this.debuggerRouter.detachIfAttachedLocked(tabId);+ }+ } finally {+ this.sendEvent(EVENTS.SCOPE_TAB_REMOVED, { tabId, reason });+ }+ }+ }++ async groupTabLocked(tab) {+ const tabId = tab.id;+ if (!Number.isInteger(tabId) || !Number.isInteger(tab.windowId)) {+ return;+ }+ let groupId;+ try {+ const groups = await chrome.tabGroups.query({ windowId: tab.windowId });+ const existingGroup = groups.find(+ (group) => group.title === SKYVERN_GROUP_TITLE,+ );+ const expectedGroupId = existingGroup?.id ?? ANY_GROUP_ID;+ const transition = this.expectGroupTransition(tabId, expectedGroupId);+ let grouped = false;+ try {+ groupId = existingGroup+ ? await chrome.tabs.group({+ groupId: existingGroup.id,+ tabIds: [tabId],+ })+ : await chrome.tabs.group({ tabIds: [tabId] });+ grouped = true;+ } finally {+ if (!grouped) {+ this.clearExpectedGroupTransition(tabId, transition);+ }+ }+ } catch {+ this.scopedGroupIds.delete(tabId);+ await this.persistScope();+ return;+ }+ this.scopedGroupIds.set(tabId, groupId);+ await this.persistScope();+ await this.updateControlledGroup(groupId);+ }++ async ungroupTabLocked(tabId, scopedGroupId) {+ if (!Number.isInteger(scopedGroupId)) {+ return;+ }+ let tab;+ try {+ tab = await chrome.tabs.get(tabId);+ } catch {+ return;+ }+ if (tab.groupId !== scopedGroupId) {+ return;+ }+ const transition = this.expectGroupTransition(tabId, TAB_GROUP_ID_NONE);+ let ungrouped = false;+ try {+ await chrome.tabs.ungroup([tabId]);+ ungrouped = true;+ } catch {+ return;+ } finally {+ if (!ungrouped) {+ this.clearExpectedGroupTransition(tabId, transition);+ }+ }+ }++ async getControlledGroup(groupId) {+ if (!Number.isInteger(groupId) || groupId === TAB_GROUP_ID_NONE) {+ return null;+ }+ try {+ const group = await chrome.tabGroups.get(groupId);+ return group.title === SKYVERN_GROUP_TITLE ? group : null;+ } catch {+ return null;+ }+ }++ async updateControlledGroup(groupId) {+ try {+ await chrome.tabGroups.update(groupId, {+ title: SKYVERN_GROUP_TITLE,+ color: SKYVERN_GROUP_COLOR,+ });+ } catch {+ return;+ }+ }++ expectGroupTransition(tabId, groupId) {+ const transition = { groupId };+ this.expectedGroupTransitions.set(tabId, transition);+ return transition;+ }++ getExpectedGroupTransition(tabId, groupId) {+ const transition = this.expectedGroupTransitions.get(tabId);+ if (+ transition === undefined ||+ (transition.groupId !== ANY_GROUP_ID && transition.groupId !== groupId) ||+ (transition.groupId === ANY_GROUP_ID && groupId === TAB_GROUP_ID_NONE)+ ) {+ return null;+ }+ return transition;+ }++ clearExpectedGroupTransition(tabId, transition) {+ if (this.expectedGroupTransitions.get(tabId) === transition) {+ this.expectedGroupTransitions.delete(tabId);+ }+ }++ async collectScopedTabs(includeActive) {+ const tabs = [];+ for (const tabId of [...this.scopedTabIds]) {+ const tab = await this.runTabOperation(tabId, async () => {+ if (!this.scopedTabIds.has(tabId)) {+ return null;+ }+ let scopedTab;+ try {+ scopedTab = await chrome.tabs.get(tabId);+ } catch {+ await this.removeFromScopeLocked(tabId, "closed", false);+ return null;+ }+ if (isTabRestricted(scopedTab)) {+ await this.removeFromScopeLocked(tabId, "unshared", true);+ return null;+ }+ return this.publicTab(scopedTab, includeActive);+ });+ if (tab !== null) {+ tabs.push(tab);+ }+ }+ return tabs;+ }++ publicTab(tab, includeActive) {+ const result = {+ tabId: tab.id,+ url: tabUrl(tab),+ title: typeof tab.title === "string" ? tab.title : "",+ };+ if (includeActive) {+ result.active = tab.active === true;+ }+ return result;+ }++ async getTab(tabId) {+ try {+ return await chrome.tabs.get(tabId);+ } catch {+ throw new ProtocolError(+ ERROR_CODES.TAB_NOT_FOUND,+ "The requested tab was not found.",+ );+ }+ }++ async reconcileStoredTabs() {+ for (const tabId of [...this.scopedTabIds]) {+ await this.runTabOperation(tabId, async () => {+ let tab;+ try {+ tab = await chrome.tabs.get(tabId);+ } catch {+ await this.removeFromScopeLocked(tabId, "closed", false);+ return;+ }+ if (isTabRestricted(tab)) {+ await this.removeFromScopeLocked(tabId, "unshared", true);+ return;+ }+ const expectedGroupId = this.scopedGroupIds.get(tabId);+ if (expectedGroupId !== undefined && tab.groupId !== expectedGroupId) {+ const controlledGroup = await this.getControlledGroup(tab.groupId);+ if (controlledGroup === null) {+ await this.removeFromScopeLocked(tabId, "unshared", true);+ } else {+ this.scopedGroupIds.set(tabId, tab.groupId);+ await this.persistScope();+ await this.updateControlledGroup(tab.groupId);+ }+ } else if (expectedGroupId === undefined) {+ await this.groupTabLocked(tab);+ }+ });+ }+ }++ async runTabOperation(tabId, operation) {+ const previous = this.tabOperations.get(tabId) ?? Promise.resolve();+ const current = previous.catch(() => undefined).then(operation);+ this.tabOperations.set(tabId, current);+ try {+ return await current;+ } finally {+ if (this.tabOperations.get(tabId) === current) {+ this.tabOperations.delete(tabId);+ }+ }+ }++ async persistScope() {+ const groupIds = Object.fromEntries(this.scopedGroupIds);+ await chrome.storage.session.set({+ [SCOPED_TAB_IDS_KEY]: [...this.scopedTabIds],+ [SCOPED_GROUP_IDS_KEY]: groupIds,+ });+ }+}diff --git a/skyvern/browser_extension/protocol.py b/skyvern/browser_extension/protocol.pynew file mode 100644index 000000000..5c13789aa--- /dev/null+++ b/skyvern/browser_extension/protocol.py@@ -0,0 +1,219 @@+from __future__ import annotations++import json+from dataclasses import dataclass+from typing import Literal+from urllib.parse import urlsplit++from skyvern.browser_extension.errors import BrowserExtensionError++PROTOCOL_VERSION = 1+EXTENSION_ID = "fmamdhmfeihjjaiheideemihnbpnokin"++ALLOWED_OPS = frozenset(+ {+ "debugger.attach",+ "debugger.detach",+ "debugger.send",+ "tabs.create",+ "tabs.remove",+ "tabs.activate",+ "tabs.list",+ }+)++ALLOWED_EVENTS = frozenset(+ {+ "extension.hello",+ "debugger.event",+ "debugger.detached",+ "scope.tabAdded",+ "scope.tabRemoved",+ "tabs.created",+ }+)++ERROR_CODES = frozenset(+ {+ "AUTH_FAILED",+ "OP_NOT_ALLOWED",+ "TAB_NOT_FOUND",+ "TAB_NOT_SCOPED",+ "RESTRICTED_URL",+ "DEBUGGER_DETACHED",+ "CDP_METHOD_NOT_ALLOWED",+ "CDP_ERROR",+ "INTERNAL",+ }+)++ALLOWED_CDP_METHOD_PREFIXES = (+ "Accessibility.",+ "Animation.",+ "CSS.",+ "Console.",+ "DOM.",+ "DOMDebugger.",+ "DOMSnapshot.",+ "DOMStorage.",+ "Debugger.",+ "Emulation.",+ "Fetch.",+ "IO.",+ "Input.",+ "Inspector.",+ "Log.",+ "Network.",+ "Overlay.",+ "Page.",+ "Performance.",+ "Profiler.",+ "Runtime.",+ "Security.",+ "Storage.",+ "Target.",+)++DENIED_CDP_METHODS = frozenset(+ {+ "Network.getAllCookies",+ "Network.clearBrowserCookies",+ "Network.clearBrowserCache",+ "Storage.getCookies",+ "Storage.setCookies",+ "Storage.clearCookies",+ }+)++RESTRICTED_URL_PREFIXES = (+ "chrome://",+ "chrome-untrusted://",+ "chrome-extension://",+ "devtools://",+ "edge://",+ "about:",+ "file://",+)++MessageKind = Literal["response", "event", "pong", "ping", "auth.proof"]+++@dataclass(frozen=True, slots=True)+class ParsedMessage:+ kind: MessageKind+ request_id: str | None = None+ ok: bool | None = None+ result: dict | None = None+ error: dict | None = None+ error_code: str | None = None+ error_message: str | None = None+ event: str | None = None+ params: dict | None = None+ client_nonce: str | None = None+ proof: str | None = None+++def build_request(request_id: str, op: str, args: dict) -> dict:+ if not isinstance(request_id, str) or not request_id:+ raise BrowserExtensionError("Request id must be a non-empty string")+ if not isinstance(op, str) or op not in ALLOWED_OPS:+ raise BrowserExtensionError(f"Operation is not allowed: {op}")+ if not isinstance(args, dict):+ raise BrowserExtensionError("Request args must be an object")+ return {"v": PROTOCOL_VERSION, "type": "request", "id": request_id, "op": op, "args": args}+++def parse_extension_message(raw: str) -> ParsedMessage:+ try:+ payload = json.loads(raw)+ except (json.JSONDecodeError, TypeError) as exc:+ raise BrowserExtensionError("Extension frame is not valid JSON") from exc++ if not isinstance(payload, dict):+ raise BrowserExtensionError("Extension frame must be an object")+ if type(payload.get("v")) is not int or payload["v"] != PROTOCOL_VERSION:+ raise BrowserExtensionError("Unsupported extension protocol version")++ frame_type = payload.get("type")+ if frame_type == "response":+ return _parse_response(payload)+ if frame_type == "event":+ return _parse_event(payload)+ if frame_type == "ping":+ return ParsedMessage(kind="ping")+ if frame_type == "pong":+ return ParsedMessage(kind="pong")+ if frame_type == "auth.proof":+ return ParsedMessage(+ kind="auth.proof",+ client_nonce=_required_string(payload, "clientNonce"),+ proof=_required_string(payload, "proof"),+ )+ raise BrowserExtensionError("Unknown extension frame type")+++def is_cdp_method_allowed(method: str, params: dict | None = None) -> bool:+ if method in DENIED_CDP_METHODS:+ return False+ if method == "Network.getCookies" and isinstance(params, dict) and "urls" in params:+ return False+ return any(method.startswith(prefix) and len(method) > len(prefix) for prefix in ALLOWED_CDP_METHOD_PREFIXES)+++def is_restricted_url(url: str) -> bool:+ normalized_url = url.strip().lower()+ if normalized_url == "about:blank":+ return False+ if normalized_url.startswith(RESTRICTED_URL_PREFIXES):+ return True+ try:+ hostname = urlsplit(normalized_url).hostname+ return hostname is not None and hostname.removesuffix(".") == "chromewebstore.google.com"+ except ValueError:+ return False+++def _parse_response(payload: dict) -> ParsedMessage:+ request_id = _required_string(payload, "id")+ ok = payload.get("ok")+ if type(ok) is not bool:+ raise BrowserExtensionError("Response ok must be a boolean")++ if ok:+ result = _required_dict(payload, "result")+ return ParsedMessage(kind="response", request_id=request_id, ok=True, result=result)++ error = _required_dict(payload, "error")+ error_code = _required_string(error, "code")+ error_message = _required_string(error, "message")+ if error_code not in ERROR_CODES:+ raise BrowserExtensionError("Response contains an unknown error code")+ return ParsedMessage(+ kind="response",+ request_id=request_id,+ ok=False,+ error=error,+ error_code=error_code,+ error_message=error_message,+ )+++def _parse_event(payload: dict) -> ParsedMessage:+ event = _required_string(payload, "event")+ if event not in ALLOWED_EVENTS:+ raise BrowserExtensionError("Unknown extension event")+ return ParsedMessage(kind="event", event=event, params=_required_dict(payload, "params"))+++def _required_string(payload: dict, field: str) -> str:+ value = payload.get(field)+ if not isinstance(value, str) or not value:+ raise BrowserExtensionError(f"Extension frame field {field} must be a non-empty string")+ return value+++def _required_dict(payload: dict, field: str) -> dict:+ value = payload.get(field)+ if not isinstance(value, dict):+ raise BrowserExtensionError(f"Extension frame field {field} must be an object")+ return valuediff --git a/skyvern/browser_extension/relay.py b/skyvern/browser_extension/relay.pynew file mode 100644index 000000000..5316e7fbe--- /dev/null+++ b/skyvern/browser_extension/relay.py@@ -0,0 +1,947 @@+from __future__ import annotations++import asyncio+import base64+import binascii+import hashlib+import hmac+import itertools+import json+import secrets+import time+from collections.abc import Awaitable, Callable+from contextlib import suppress+from urllib.parse import urlsplit++import structlog+from aiohttp import WSMsgType, web++from skyvern.browser_extension.auth import build_challenge, compute_server_proof, verify_ext_proof+from skyvern.browser_extension.errors import (+ BrowserExtensionError,+ BrowserExtensionNotConnectedError,+ ExtensionRequestError,+)+from skyvern.browser_extension.protocol import (+ EXTENSION_ID,+ PROTOCOL_VERSION,+ ParsedMessage,+ build_request,+ parse_extension_message,+)++LOG = structlog.get_logger(__name__)++_AUTH_TIMEOUT_SECONDS = 10.0+_PING_INTERVAL_SECONDS = 20.0+_INBOUND_TIMEOUT_SECONDS = 45.0+_MAX_WS_MESSAGE_BYTES = 256 * 1024 * 1024+_AUTH_CLOSE_CODE = 4403+_REPLACED_CLOSE_CODE = 4000+_PAIRING_NONCE_TTL_SECONDS = 120.0+_PAIR_BEGIN_MESSAGE = b"skyvern-pair-begin-v1"++_PAIR_PAGE_TEMPLATE = """<!doctype html>+<html lang="en">+<head>+ <meta charset="utf-8">+ <meta name="viewport" content="width=device-width,initial-scale=1">+ <title>Pair Skyvern Agent</title>+ <style>+ :root {+ color-scheme: dark light;+ font-family: ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;+ color: #f4f4f5;+ background: #0b0c0f;+ font-synthesis: none;+ --accent: #6d6cf6;+ --accent-hover: #5f5eea;+ --surface: #141519;+ --surface-subtle: #1a1b20;+ --border: #2b2c33;+ --border-strong: #3a3b43;+ --text: #f4f4f5;+ --muted: #b2b3ba;+ --subtle: #85868f;+ --radius: .5rem;+ }++ * { box-sizing: border-box; }++ body {+ min-height: 100vh;+ margin: 0;+ display: grid;+ overflow-x: hidden;+ color: var(--text);+ background: #0b0c0f;+ place-items: center;+ }++ .page-shell {+ position: relative;+ width: 100%;+ padding: 2rem 1rem;+ }++ .pairing-card {+ width: min(38rem, 100%);+ margin: 0 auto;+ overflow: hidden;+ border: 1px solid var(--border);+ border-radius: var(--radius);+ background: var(--surface);+ box-shadow: 0 1rem 2.5rem rgb(0 0 0 / 28%);+ }++ .card-header {+ display: flex;+ align-items: center;+ gap: .75rem;+ padding: 1.5rem 1.75rem 0;+ }++ .brand-mark,+ .endpoint-mark {+ display: grid;+ flex: none;+ overflow: hidden;+ place-items: center;+ }++ .brand-mark {+ width: 2rem;+ height: 2rem;+ border-radius: var(--radius);+ }++ .brand-mark img,+ .endpoint-mark img {+ width: 100%;+ height: 100%;+ object-fit: cover;+ }++ .brand-copy { display: grid; }+ .brand-name { font-size: .9rem; font-weight: 650; letter-spacing: -.01em; }++ .card-body { padding: 2.5rem 1.75rem 1.75rem; }+ .intro { text-align: left; }++ h1 {+ margin: 0;+ font-size: clamp(2rem, 6vw, 2.35rem);+ font-weight: 680;+ line-height: 1.15;+ letter-spacing: -.035em;+ }++ .supporting {+ max-width: 32rem;+ margin: .85rem 0 0;+ color: var(--muted);+ font-size: .95rem;+ line-height: 1.55;+ }++ .connection {+ display: grid;+ grid-template-columns: minmax(0, 1fr) 3.5rem minmax(0, 1fr);+ align-items: center;+ margin: 2.25rem 0;+ }++ .endpoint {+ display: grid;+ grid-template-columns: 2rem minmax(0, 1fr);+ grid-template-rows: auto auto;+ min-width: 0;+ align-items: center;+ column-gap: .7rem;+ row-gap: .2rem;+ }++ .endpoint-top {+ display: contents;+ }++ .endpoint-mark {+ grid-row: 1 / 3;+ width: 2rem;+ height: 2rem;+ border-radius: var(--radius);+ color: var(--muted);+ }++ .endpoint-mark svg { width: 1rem; height: 1rem; }+ .endpoint-title { overflow: hidden; font-size: .8rem; font-weight: 650; text-overflow: ellipsis; white-space: nowrap; }+ .endpoint-meta { grid-column: 2; margin: 0; overflow: hidden; color: var(--subtle); font-size: .7rem; line-height: 1.4; text-overflow: ellipsis; white-space: nowrap; }+ #server-address { font-family: ui-monospace, SFMono-Regular, Menlo, monospace; }++ .connection-rail {+ position: relative;+ height: 1px;+ background: var(--border-strong);+ }++ .link-glyph {+ position: absolute;+ top: 50%;+ left: 50%;+ display: grid;+ width: 1.75rem;+ height: 1.75rem;+ color: var(--subtle);+ background: var(--surface);+ place-items: center;+ transform: translate(-50%, -50%);+ }++ .link-glyph svg { width: .95rem; height: .95rem; }++ .approval-controls { display: flex; flex-direction: column; align-items: flex-start; gap: .8rem; }++ .approve-button {+ display: inline-flex;+ width: auto;+ min-height: 2.75rem;+ align-items: center;+ justify-content: center;+ gap: .6rem;+ margin: 0;+ padding: .72rem 1.15rem;+ border: 1px solid var(--accent);+ border-radius: var(--radius);+ color: white;+ background: var(--accent);+ box-shadow: 0 .25rem .7rem rgb(0 0 0 / 24%);+ font: inherit;+ font-size: .9rem;+ font-weight: 650;+ cursor: pointer;+ transition: background 150ms ease, box-shadow 150ms ease;+ }++ .approve-button:hover:not(:disabled) {+ background: var(--accent-hover);+ box-shadow: 0 .3rem .85rem rgb(0 0 0 / 28%);+ }++ .approve-button:focus-visible,+ .copy-chip:focus-visible { outline: 2px solid var(--text); outline-offset: 3px; }+ .approve-button:disabled { cursor: wait; opacity: .78; }++ .spinner {+ display: none;+ width: 1rem;+ height: 1rem;+ border: 2px solid rgb(255 255 255 / 35%);+ border-top-color: white;+ border-radius: 50%;+ animation: spin 700ms linear infinite;+ }++ [data-state="approving"] .spinner { display: block; }+ .fine-print {+ margin: 0;+ color: var(--subtle);+ font-size: .7rem;+ line-height: 1.5;+ text-align: center;+ }++ .status-panel {+ display: grid;+ gap: .7rem;+ margin-top: .35rem;+ padding: 1rem 0 0;+ text-align: left;+ }++ .status-panel[hidden] { display: none; }+ .status-heading { display: flex; align-items: center; gap: .7rem; }++ .status-icon {+ display: grid;+ flex: none;+ width: 2rem;+ height: 2rem;+ color: var(--muted);+ place-items: center;+ }++ .status-icon svg { width: 1rem; height: 1rem; }+ .status-icon .success-icon { display: none; }+ .status-title { margin: 0; font-size: .9rem; font-weight: 650; }+ .status-message { margin: 0; color: var(--muted); font-size: .78rem; line-height: 1.55; }++ [data-state="success"] .status-icon .error-icon { display: none; }+ [data-state="success"] .status-icon .success-icon { display: block; }+ [data-state="success"] .approval-controls,+ [data-state="error"] .approval-controls { display: none; }++ .recovery { display: grid; gap: .4rem; }+ .recovery[hidden] { display: none; }+ .recovery-label { color: var(--subtle); font-size: .67rem; font-weight: 650; }++ .copy-chip {+ display: flex;+ width: fit-content;+ max-width: 100%;+ min-width: 0;+ align-items: center;+ justify-content: space-between;+ gap: .75rem;+ padding: .7rem .8rem;+ border: 1px solid var(--border);+ border-radius: var(--radius);+ color: var(--text);+ background: var(--surface-subtle);+ font: inherit;+ cursor: copy;+ }++ .copy-chip:hover { border-color: var(--border-strong); background: #202126; }+ .copy-chip code { overflow: hidden; font: 600 .68rem/1.4 ui-monospace, SFMono-Regular, Menlo, monospace; text-overflow: ellipsis; white-space: nowrap; }+ .copy-label { flex: none; color: var(--muted); font-size: .66rem; font-weight: 650; }++ @keyframes spin { to { transform: rotate(360deg); } }+ @media (max-width: 34rem) {+ .page-shell { padding: 1rem; }+ .card-header { padding: 1rem 1.1rem; }+ .card-body { padding: 1.7rem 1.2rem 1.35rem; }+ h1 { font-size: 1.9rem; }+ .connection { grid-template-columns: 1fr; gap: .85rem; }+ .connection-rail { width: 1px; height: 1.6rem; margin-left: 1rem; }+ .endpoint-title { overflow: visible; font-size: .78rem; line-height: 1.25; text-overflow: clip; white-space: normal; }+ .endpoint-meta { font-size: .67rem; }+ .link-glyph { width: 1.5rem; height: 1.5rem; }+ .supporting { font-size: .84rem; }+ }++ @media (prefers-reduced-motion: reduce) {+ *, *::before, *::after { scroll-behavior: auto !important; transition: none !important; animation-duration: .01ms !important; animation-iteration-count: 1 !important; }+ }++ @media (prefers-color-scheme: light) {+ :root {+ color: #202124;+ background: #f5f5f7;+ --surface: #ffffff;+ --surface-subtle: #f5f5f7;+ --border: #dedfe3;+ --border-strong: #c9cbd1;+ --text: #202124;+ --muted: #5f6169;+ --subtle: #777982;+ }++ body { background: #f5f5f7; }+ .pairing-card { box-shadow: 0 1rem 2.5rem rgb(32 33 36 / 10%); }+ .copy-chip:hover { background: #ececef; }+ }+ </style>+</head>+<body data-state="idle">+ <div class="page-shell">+ <main class="pairing-card" aria-labelledby="pairing-title">+ <header class="card-header">+ <span class="brand-mark" aria-hidden="true">+ <img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAhGVYSWZNTQAqAAAACAAFARIAAwAAAAEAAQAAARoABQAAAAEAAABKARsABQAAAAEAAABSASgAAwAAAAEAAgAAh2kABAAAAAEAAABaAAAAAAAAAEgAAAABAAAASAAAAAEAA6ABAAMAAAABAAEAAKACAAQAAAABAAAAMKADAAQAAAABAAAAMAAAAAAoDQEPAAAACXBIWXMAAAsTAAALEwEAmpwYAAACymlUWHRYTUw6Y29tLmFkb2JlLnhtcAAAAAAAPHg6eG1wbWV0YSB4bWxuczp4PSJhZG9iZTpuczptZXRhLyIgeDp4bXB0az0iWE1QIENvcmUgNi4wLjAiPgogICA8cmRmOlJERiB4bWxuczpyZGY9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjItcmRmLXN5bnRheC1ucyMiPgogICAgICA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0iIgogICAgICAgICAgICB4bWxuczp0aWZmPSJodHRwOi8vbnMuYWRvYmUuY29tL3RpZmYvMS4wLyIKICAgICAgICAgICAgeG1sbnM6ZXhpZj0iaHR0cDovL25zLmFkb2JlLmNvbS9leGlmLzEuMC8iPgogICAgICAgICA8dGlmZjpZUmVzb2x1dGlvbj43MjwvdGlmZjpZUmVzb2x1dGlvbj4KICAgICAgICAgPHRpZmY6UmVzb2x1dGlvblVuaXQ+MjwvdGlmZjpSZXNvbHV0aW9uVW5pdD4KICAgICAgICAgPHRpZmY6WFJlc29sdXRpb24+NzI8L3RpZmY6WFJlc29sdXRpb24+CiAgICAgICAgIDx0aWZmOk9yaWVudGF0aW9uPjE8L3RpZmY6T3JpZW50YXRpb24+CiAgICAgICAgIDxleGlmOlBpeGVsWERpbWVuc2lvbj4yNTY8L2V4aWY6UGl4ZWxYRGltZW5zaW9uPgogICAgICAgICA8ZXhpZjpDb2xvclNwYWNlPjE8L2V4aWY6Q29sb3JTcGFjZT4KICAgICAgICAgPGV4aWY6UGl4ZWxZRGltZW5zaW9uPjI1NjwvZXhpZjpQaXhlbFlEaW1lbnNpb24+CiAgICAgIDwvcmRmOkRlc2NyaXB0aW9uPgogICA8L3JkZjpSREY+CjwveDp4bXBtZXRhPgrkVyGCAAANkElEQVRoBdVaCXCU5Rl+stnd3Pd9AoGES0KAIAmXQUAOJYJCQfAAES22nTr2mPEYtcfojLXO1Oloa6ejTp2KI4paCqhcIQkISThyJ+S+yJ3NtbvZze72eb8kEjQX0HbgY5Zs/n/3/97jeZ/3eT9w0sy5x4HbaTk5wYkvh92urNbcTrY7HA64urhAr9dB3su6vRxg1GNiouE0LOq3jQN2mx0hIUGYGTcZpl6jgtFtkwGFFkb/sR0bYejshoPODK3bIgOOvj6sXbcCKUsSceFSEaB1HrL/1q8Be58FC5MT8dwzu5CVk4fWplZoNFfjrv3OlVvsjbCMw2LFkmWL8MIvnoBep8Hh46dB+rnG0lvSAbvNpozcvPlePL37R3Bz1aGwtAZ5+SVw0l5r8rW/XePb//+XoagHBAfiKRq+Yc0yWJkFi6UfZ7Pz0csC1rAHDF//EwfEED0bjtViGb7XqO/tAgsaqnNzxZr1y7DnsQcRGRGC7t5eXraiv9+O3MLSEb//X3NAjJbisnHDsLAQ9JPqWlpMcGbK5d5IS0HF2g9XL0+kpCRj59b1SJgzAz0mCzq6etCvjO+HiYXc3m5g2x3ewgaeeNMOqLQTs4FBAYyYEdHhIZgyKRInM7IQEOCPPhrY29PzXeOxi4bhNXqL6KhwrF2RhG2pqzB7+lR0mcxo6OiEmbRppfE2ftbWb4ODvdfXzwew/zAQN+WAnWl31usxd+506HRahAf6I4SOpJ29gOXJC1Df3ILyylplvF02p1H+Qf5ISZqPzWvuwt2L5yHIwwMNRjPyKxvQ1tWtjO+z9BH3FvTyuoGZaGpqZxCMcPVwpw8OQooBGFxON6JGVeoZmaWL5mH3lvXQ6vToY9TSs3Nx4Jt0hIcGo83QiZbWdgUribq7qwv27tiEJ/n5uMgwlNZfwZcnv0XauUsoqqhFj9GkFKaN0JPPawgXnU4Hf0Y+iIFxUIFW1zSgoaERtkGWEh+uOwPy8EkRofjZw5uwM3UN6lvb8PHhkygqr4ahuxsRZJDK2npG0qqMVwXt7Iy3XnoGuwmVRmL58ZffxKdHTsGVRTsjdjKS7pwLPx8vqkw9g6Fj3TjDw90Vnoz4lcZWZJzJwaW8YvT09KpnDkX/uh2wm/vw4IZVuI8Ft3FFMq60tGHv796Cj5c7Pnr9eeicNTjAqL697yDOZF9S+ziI9zsXL1DG5xSV4aFf/h4aZuyFZ/cgfvZU6N1cYKSzfYSjldCwDmK+sroeH358EJcoHax9ZsX/wzvwkBPOTiFTXxn6ZayfYnxc3BS8+sxu2LnJvBkxLDIHjXZGQXkNymrqkbJwLqaR/jyJ68tMdzMdZL6xa8sGxESH494fP49ZM6bitRd+ygKOQGt3L660GRTjCGUKjEx0JCunEH/7+0eoYlYdJB4N95AhZqQ1IQgJbHZuux8v7d2Ozi4jqhua8ZdPDuFw+llk5ZXAQoeOnjjN1Ovw2588jARG1tPTQ22qYT+YHz8Tr7zzD0iDSr13BQ6nfYuq+mY4KGkE4/7+vnAnZJzITC0tHfj4k3+hh8X7/aZ1ww4w1IibFIEPvzyK1/76T6XHlSaRYmJ0QAYSQ/cdPoFd5HJn4jiUbCNZm0J6NJBePzuSBi8fb/zqlT+pvqAlzqVXSLOTWoiKjsIdc6azDhhTYcsROP+GHdBws5f//L5iB+FlhhZ6Fti8hFlYdGc8Gacb2edy4ePhhrbOLuRfrkFuSYXaLyV5PvYfOkanTUhMjMfc+Fnw9mZ2WC9WmwNNhNmliwXIycpFfm4BoidHIWpKNOqqamE0GkdtgkPOTAhCYrBwr+qo5OHkpYl4lCzkzCilpWcj7dRZbFqbgiriXuOkQSc1S0VJOcInRzITATh4NBM7H9+GKTFRMLLLmu3kccKytLQC2VkX0dzYwmfb1fMryyqJeS3ZSKsgOFoXvz4HBj/tTIzu3L0Zmzak4MAXJ3Dkm0zUs9CiOacmzIzFZTYtYZXM83l0xAlbUu/BwWMZCGJfqK5rQA05PDg0iBnwQikzdPTICZVVqQ1re4faxUkgSQyJjJjImlAGVBTIODv3bMO6NUvx7nufUzaYBhoKobB7aypyiy5j+rRo1Dc2I5vUt+ORTSitoAQuuIwowqKeUe4n/AooyiIiwmCgZFANUeBIhRkYHISmK40q6qMxzkgOXR1tRro7eE24fPnKJVhFefvJ50cRRLFWSk5vJHUKZhNmx+LM+QIkzZuJ1o4u7N21Bd1sOmmnc7By/UqsZe9YlJSImGkx8PL2Id4voq62gcWuhc5FTx6gDGF3DqITHqyt8WAz3NRxMyAHSL6BAUh9YC3O5xYjhDx/6IuvUV9dp4p5YfwMlLHp+LKT+pBl5PPfZGTjyLEzWLZiKU8SAlBdXoussznoEJiQYTTMWi8hIpEWqPj4eiM0LBit1E6h4cHIO5/PxnV17h1u8Pffj+8A057ITqonw3h4+6LodDbK8ktJm3rIvBoWEogKOhMdFUqHgIysAuz77CuER0WitPgyzmddQBeZyWoyqW7qNEiPQzARhw0GA+aHzIfJdB6xM4NZR67ol1mCDo63xoSQpFKGjFlsREZ2SPIEvs04d82pgIkS2EKI6VjgJpOVc2smbNy8rroWpq4uKlRfzJw6CeHEvRhkZ8SHQ0SaV6ehi+zUpwpax2box8amhpzxrOf9MTPgYOH68mHe7JaWvn5UknHaKN6ktatFKOQWl2H18iSUkIE6e81obh1gk9S1d+GBdcvhzY6s1TmrsbCssh7/PpqBdMrtAcU5ED+RJr3dRkXVFk5fnt7eaCJjTWSNmQECGh6enmC/gZldtfkK2780ssEllHeRcLKwR4guMjGKAXRYBo9JUWHwI7b1Ljq4MKo+Xh5YunAOXn/+abz48yeUXhoa3gV6XcyChZDst9p47HPt3Du030g/xxRzgs8AHufFzJzGWcSGmooq1JRVqS4qDxMc2wifBp7VJC9MQBfF2KTIEORcLFSOVdQ2oYNdWpSBH6PqyR6hobUJs6ZhKtkr/exFZbQbxZ/e1XWgiCMi0E1Z3tHWyuePHV+xYRwHHPDy9UFkbIzi8E5SZGVJmcKwGnMZOWGUjg4DWqgqJerxNM7Pzxf5pNkKvnJyi3AsMxvnLhapQX86mx5hr+rChQ6lU+t7+vrBzCK3MZOBISGsiQ5mpFMFSIwca43tAL+p45ARFTsNhlYD8nJyoeEmkeys7ixuIwtYClbOamToLqR86GGDWzB3NpZwSLEy9E2kTivh19bWgTQa29zeheQFc3hQpcVsyvNsNroa9oTe3h7SqR+ZzgftrS0wso8MMdUNOyBflEi7unmg8FwWHmVhPvfkdjyweim2b1iNzevugkynhZerFEyk05ZRy2RSmDW3tCOWouyOGbHoZ020kSpFYRazE/fynGfF4gR4urspZjrCwpY6CAgiFTOj7c3Nqj/ctAMK44x4S30DXnxqh1K4f3x/P9478BVn31McNjT49Z6HEEdYnBI80zANsyHCr5GSIq+whONlA0LZYeWEoo3ZsNOJ0rJqJM2fg8msFxcy1Gdfp5MoCFcfP1KyBZ1tHIQURseK/cC9MSEkHxHeXpw4F8EcrN/ef5j1MA1uHp6MZAmysi7hOKO9e+sGxNKJE5QOwt/iuPC7sJQcTDVQ44j29ySjiUS2mc08hvHHSp5c9Ntt+PTQSaVgpc9IFs093Soz45vPpI77IaY/KiyMCrMQU1jMDg4xrm5uiJkeR2zpUUgR9/RLb5Ii4+lIKkQ3DV/ijGSlnRDqIrs4Sw/htaZmRllIgH9UsEnZMmuYWQvCWhNd4zvAzbo4r5oYtbamNsLEirqaGrgzmmERkXCikizIK8Ib7+7Dnu3384RhnpIY3zdA5LUcvUgDE4t9KaldObk1sbjb2QOECCQDaoDnZye6xneARVVSSc0fGY7KsnJytBGePv5oqKtXkQwMCYWWCvLAoeP46lQWfvvsE5hEjreLlhlhKRnBLCzhdOaiccaJjBxCpldlxmrm2dAEsT/06HEdENlQVlENF3bUqTwyLC8uRp/ZgsDQcB6F9MHI7Li5e6jTg9ff+YBHLe146zfPIo7SWWZi6bZilLzUe1Lv+tXLsHHVEuSSsd7bf0gpT3VPsnOda9wilufJ5jV1V7By+RIOIgZUV1WzKbnC3ctLYdgskWMxmligmewVSfPuwJMPpULLHtLa3kn49QnsEUwmemTrfXiDcsLQ3YOnXvgDSjiZadgTbnRN+GhRIuRNg5MSE0iH7Sjn0O3Q6OHC6POfnSkJqEoJGxOLUE9q3Mo+8eimtdRAHPQ5fUnBRoeH8hmeOJ55Hq++/QGKisvHPToZq6ClUibsgERIFSB/ytlnIIWaFF87a0JDLMsxulCn/HOoZMRCaPn4eVP3xGIG5bQHZUMj4XWBnbeIekoOvMaLvJbWj4ZxccxKD67LAX5HrSFHtDRaYZvYlWO/qydoA3LbajGTVjmckyIVN0rIWFPSHybSZV35Nd0oKZBq6eU2NwS+oTPKIUfEGHFExNiApeKn/J8G/k2avZk1iv18sNxhj7mZh4/8XYXMkW9d99WRzXei8fKSpZVp6FZd/cyq4ucfGMjrdMDOl3bj3ck/uH2rXBgM8qjmyMn1fwCpJWsaG/VU1QAAAABJRU5ErkJggg==" alt="">+ </span>+ <span class="brand-copy">+ <span class="brand-name">Skyvern</span>+ </span>+ </header>++ <div class="card-body">+ <section class="intro">+ <h1 id="pairing-title">Pair Skyvern Agent</h1>+ <p class="supporting">Skyvern on this computer wants to drive Chrome through the Skyvern Agent extension.</p>+ </section>++ <div class="connection" aria-label="Skyvern MCP server connecting to Skyvern Agent extension">+ <div class="endpoint">+ <div class="endpoint-top">+ <span class="endpoint-mark" aria-hidden="true">+ <img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAhGVYSWZNTQAqAAAACAAFARIAAwAAAAEAAQAAARoABQAAAAEAAABKARsABQAAAAEAAABSASgAAwAAAAEAAgAAh2kABAAAAAEAAABaAAAAAAAAAEgAAAABAAAASAAAAAEAA6ABAAMAAAABAAEAAKACAAQAAAABAAAAMKADAAQAAAABAAAAMAAAAAAoDQEPAAAACXBIWXMAAAsTAAALEwEAmpwYAAACymlUWHRYTUw6Y29tLmFkb2JlLnhtcAAAAAAAPHg6eG1wbWV0YSB4bWxuczp4PSJhZG9iZTpuczptZXRhLyIgeDp4bXB0az0iWE1QIENvcmUgNi4wLjAiPgogICA8cmRmOlJERiB4bWxuczpyZGY9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjItcmRmLXN5bnRheC1ucyMiPgogICAgICA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0iIgogICAgICAgICAgICB4bWxuczp0aWZmPSJodHRwOi8vbnMuYWRvYmUuY29tL3RpZmYvMS4wLyIKICAgICAgICAgICAgeG1sbnM6ZXhpZj0iaHR0cDovL25zLmFkb2JlLmNvbS9leGlmLzEuMC8iPgogICAgICAgICA8dGlmZjpZUmVzb2x1dGlvbj43MjwvdGlmZjpZUmVzb2x1dGlvbj4KICAgICAgICAgPHRpZmY6UmVzb2x1dGlvblVuaXQ+MjwvdGlmZjpSZXNvbHV0aW9uVW5pdD4KICAgICAgICAgPHRpZmY6WFJlc29sdXRpb24+NzI8L3RpZmY6WFJlc29sdXRpb24+CiAgICAgICAgIDx0aWZmOk9yaWVudGF0aW9uPjE8L3RpZmY6T3JpZW50YXRpb24+CiAgICAgICAgIDxleGlmOlBpeGVsWERpbWVuc2lvbj4yNTY8L2V4aWY6UGl4ZWxYRGltZW5zaW9uPgogICAgICAgICA8ZXhpZjpDb2xvclNwYWNlPjE8L2V4aWY6Q29sb3JTcGFjZT4KICAgICAgICAgPGV4aWY6UGl4ZWxZRGltZW5zaW9uPjI1NjwvZXhpZjpQaXhlbFlEaW1lbnNpb24+CiAgICAgIDwvcmRmOkRlc2NyaXB0aW9uPgogICA8L3JkZjpSREY+CjwveDp4bXBtZXRhPgrkVyGCAAANkElEQVRoBdVaCXCU5Rl+stnd3Pd9AoGES0KAIAmXQUAOJYJCQfAAES22nTr2mPEYtcfojLXO1Oloa6ejTp2KI4paCqhcIQkISThyJ+S+yJ3NtbvZze72eb8kEjQX0HbgY5Zs/n/3/97jeZ/3eT9w0sy5x4HbaTk5wYkvh92urNbcTrY7HA64urhAr9dB3su6vRxg1GNiouE0LOq3jQN2mx0hIUGYGTcZpl6jgtFtkwGFFkb/sR0bYejshoPODK3bIgOOvj6sXbcCKUsSceFSEaB1HrL/1q8Be58FC5MT8dwzu5CVk4fWplZoNFfjrv3OlVvsjbCMw2LFkmWL8MIvnoBep8Hh46dB+rnG0lvSAbvNpozcvPlePL37R3Bz1aGwtAZ5+SVw0l5r8rW/XePb//+XoagHBAfiKRq+Yc0yWJkFi6UfZ7Pz0csC1rAHDF//EwfEED0bjtViGb7XqO/tAgsaqnNzxZr1y7DnsQcRGRGC7t5eXraiv9+O3MLSEb//X3NAjJbisnHDsLAQ9JPqWlpMcGbK5d5IS0HF2g9XL0+kpCRj59b1SJgzAz0mCzq6etCvjO+HiYXc3m5g2x3ewgaeeNMOqLQTs4FBAYyYEdHhIZgyKRInM7IQEOCPPhrY29PzXeOxi4bhNXqL6KhwrF2RhG2pqzB7+lR0mcxo6OiEmbRppfE2ftbWb4ODvdfXzwew/zAQN+WAnWl31usxd+506HRahAf6I4SOpJ29gOXJC1Df3ILyylplvF02p1H+Qf5ISZqPzWvuwt2L5yHIwwMNRjPyKxvQ1tWtjO+z9BH3FvTyuoGZaGpqZxCMcPVwpw8OQooBGFxON6JGVeoZmaWL5mH3lvXQ6vToY9TSs3Nx4Jt0hIcGo83QiZbWdgUribq7qwv27tiEJ/n5uMgwlNZfwZcnv0XauUsoqqhFj9GkFKaN0JPPawgXnU4Hf0Y+iIFxUIFW1zSgoaERtkGWEh+uOwPy8EkRofjZw5uwM3UN6lvb8PHhkygqr4ahuxsRZJDK2npG0qqMVwXt7Iy3XnoGuwmVRmL58ZffxKdHTsGVRTsjdjKS7pwLPx8vqkw9g6Fj3TjDw90Vnoz4lcZWZJzJwaW8YvT09KpnDkX/uh2wm/vw4IZVuI8Ft3FFMq60tGHv796Cj5c7Pnr9eeicNTjAqL697yDOZF9S+ziI9zsXL1DG5xSV4aFf/h4aZuyFZ/cgfvZU6N1cYKSzfYSjldCwDmK+sroeH358EJcoHax9ZsX/wzvwkBPOTiFTXxn6ZayfYnxc3BS8+sxu2LnJvBkxLDIHjXZGQXkNymrqkbJwLqaR/jyJ68tMdzMdZL6xa8sGxESH494fP49ZM6bitRd+ygKOQGt3L660GRTjCGUKjEx0JCunEH/7+0eoYlYdJB4N95AhZqQ1IQgJbHZuux8v7d2Ozi4jqhua8ZdPDuFw+llk5ZXAQoeOnjjN1Ovw2588jARG1tPTQ22qYT+YHz8Tr7zzD0iDSr13BQ6nfYuq+mY4KGkE4/7+vnAnZJzITC0tHfj4k3+hh8X7/aZ1ww4w1IibFIEPvzyK1/76T6XHlSaRYmJ0QAYSQ/cdPoFd5HJn4jiUbCNZm0J6NJBePzuSBi8fb/zqlT+pvqAlzqVXSLOTWoiKjsIdc6azDhhTYcsROP+GHdBws5f//L5iB+FlhhZ6Fti8hFlYdGc8Gacb2edy4ePhhrbOLuRfrkFuSYXaLyV5PvYfOkanTUhMjMfc+Fnw9mZ2WC9WmwNNhNmliwXIycpFfm4BoidHIWpKNOqqamE0GkdtgkPOTAhCYrBwr+qo5OHkpYl4lCzkzCilpWcj7dRZbFqbgiriXuOkQSc1S0VJOcInRzITATh4NBM7H9+GKTFRMLLLmu3kccKytLQC2VkX0dzYwmfb1fMryyqJeS3ZSKsgOFoXvz4HBj/tTIzu3L0Zmzak4MAXJ3Dkm0zUs9CiOacmzIzFZTYtYZXM83l0xAlbUu/BwWMZCGJfqK5rQA05PDg0iBnwQikzdPTICZVVqQ1re4faxUkgSQyJjJjImlAGVBTIODv3bMO6NUvx7nufUzaYBhoKobB7aypyiy5j+rRo1Dc2I5vUt+ORTSitoAQuuIwowqKeUe4n/AooyiIiwmCgZFANUeBIhRkYHISmK40q6qMxzkgOXR1tRro7eE24fPnKJVhFefvJ50cRRLFWSk5vJHUKZhNmx+LM+QIkzZuJ1o4u7N21Bd1sOmmnc7By/UqsZe9YlJSImGkx8PL2Id4voq62gcWuhc5FTx6gDGF3DqITHqyt8WAz3NRxMyAHSL6BAUh9YC3O5xYjhDx/6IuvUV9dp4p5YfwMlLHp+LKT+pBl5PPfZGTjyLEzWLZiKU8SAlBdXoussznoEJiQYTTMWi8hIpEWqPj4eiM0LBit1E6h4cHIO5/PxnV17h1u8Pffj+8A057ITqonw3h4+6LodDbK8ktJm3rIvBoWEogKOhMdFUqHgIysAuz77CuER0WitPgyzmddQBeZyWoyqW7qNEiPQzARhw0GA+aHzIfJdB6xM4NZR67ol1mCDo63xoSQpFKGjFlsREZ2SPIEvs04d82pgIkS2EKI6VjgJpOVc2smbNy8rroWpq4uKlRfzJw6CeHEvRhkZ8SHQ0SaV6ehi+zUpwpax2box8amhpzxrOf9MTPgYOH68mHe7JaWvn5UknHaKN6ktatFKOQWl2H18iSUkIE6e81obh1gk9S1d+GBdcvhzY6s1TmrsbCssh7/PpqBdMrtAcU5ED+RJr3dRkXVFk5fnt7eaCJjTWSNmQECGh6enmC/gZldtfkK2780ssEllHeRcLKwR4guMjGKAXRYBo9JUWHwI7b1Ljq4MKo+Xh5YunAOXn/+abz48yeUXhoa3gV6XcyChZDst9p47HPt3Du030g/xxRzgs8AHufFzJzGWcSGmooq1JRVqS4qDxMc2wifBp7VJC9MQBfF2KTIEORcLFSOVdQ2oYNdWpSBH6PqyR6hobUJs6ZhKtkr/exFZbQbxZ/e1XWgiCMi0E1Z3tHWyuePHV+xYRwHHPDy9UFkbIzi8E5SZGVJmcKwGnMZOWGUjg4DWqgqJerxNM7Pzxf5pNkKvnJyi3AsMxvnLhapQX86mx5hr+rChQ6lU+t7+vrBzCK3MZOBISGsiQ5mpFMFSIwca43tAL+p45ARFTsNhlYD8nJyoeEmkeys7ixuIwtYClbOamToLqR86GGDWzB3NpZwSLEy9E2kTivh19bWgTQa29zeheQFc3hQpcVsyvNsNroa9oTe3h7SqR+ZzgftrS0wso8MMdUNOyBflEi7unmg8FwWHmVhPvfkdjyweim2b1iNzevugkynhZerFEyk05ZRy2RSmDW3tCOWouyOGbHoZ020kSpFYRazE/fynGfF4gR4urspZjrCwpY6CAgiFTOj7c3Nqj/ctAMK44x4S30DXnxqh1K4f3x/P9478BVn31McNjT49Z6HEEdYnBI80zANsyHCr5GSIq+whONlA0LZYeWEoo3ZsNOJ0rJqJM2fg8msFxcy1Gdfp5MoCFcfP1KyBZ1tHIQURseK/cC9MSEkHxHeXpw4F8EcrN/ef5j1MA1uHp6MZAmysi7hOKO9e+sGxNKJE5QOwt/iuPC7sJQcTDVQ44j29ySjiUS2mc08hvHHSp5c9Ntt+PTQSaVgpc9IFs093Soz45vPpI77IaY/KiyMCrMQU1jMDg4xrm5uiJkeR2zpUUgR9/RLb5Ii4+lIKkQ3DV/ijGSlnRDqIrs4Sw/htaZmRllIgH9UsEnZMmuYWQvCWhNd4zvAzbo4r5oYtbamNsLEirqaGrgzmmERkXCikizIK8Ib7+7Dnu3384RhnpIY3zdA5LUcvUgDE4t9KaldObk1sbjb2QOECCQDaoDnZye6xneARVVSSc0fGY7KsnJytBGePv5oqKtXkQwMCYWWCvLAoeP46lQWfvvsE5hEjreLlhlhKRnBLCzhdOaiccaJjBxCpldlxmrm2dAEsT/06HEdENlQVlENF3bUqTwyLC8uRp/ZgsDQcB6F9MHI7Li5e6jTg9ff+YBHLe146zfPIo7SWWZi6bZilLzUe1Lv+tXLsHHVEuSSsd7bf0gpT3VPsnOda9wilufJ5jV1V7By+RIOIgZUV1WzKbnC3ctLYdgskWMxmligmewVSfPuwJMPpULLHtLa3kn49QnsEUwmemTrfXiDcsLQ3YOnXvgDSjiZadgTbnRN+GhRIuRNg5MSE0iH7Sjn0O3Q6OHC6POfnSkJqEoJGxOLUE9q3Mo+8eimtdRAHPQ5fUnBRoeH8hmeOJ55Hq++/QGKisvHPToZq6ClUibsgERIFSB/ytlnIIWaFF87a0JDLMsxulCn/HOoZMRCaPn4eVP3xGIG5bQHZUMj4XWBnbeIekoOvMaLvJbWj4ZxccxKD67LAX5HrSFHtDRaYZvYlWO/qydoA3LbajGTVjmckyIVN0rIWFPSHybSZV35Nd0oKZBq6eU2NwS+oTPKIUfEGHFExNiApeKn/J8G/k2avZk1iv18sNxhj7mZh4/8XYXMkW9d99WRzXei8fKSpZVp6FZd/cyq4ucfGMjrdMDOl3bj3ck/uH2rXBgM8qjmyMn1fwCpJWsaG/VU1QAAAABJRU5ErkJggg==" alt="">+ </span>+ <span class="endpoint-title">Skyvern MCP Server</span>+ </div>+ <p id="server-address" class="endpoint-meta">127.0.0.1</p>+ </div>++ <div class="connection-rail" aria-hidden="true">+ <span class="link-glyph">+ <svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round">+ <path d="M10 13a5 5 0 0 0 7.1.1l2-2a5 5 0 0 0-7.1-7.1l-1.1 1.1"/>+ <path d="M14 11a5 5 0 0 0-7.1-.1l-2 2A5 5 0 0 0 12 20l1.1-1.1"/>+ </svg>+ </span>+ </div>++ <div class="endpoint">+ <div class="endpoint-top">+ <span class="endpoint-mark" aria-hidden="true">+ <svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8">+ <rect x="3.5" y="4.5" width="17" height="15" rx="2.5"/>+ <path d="M3.5 8.5h17"/>+ <circle cx="6.5" cy="6.5" r=".5" fill="currentColor" stroke="none"/>+ <circle cx="9" cy="6.5" r=".5" fill="currentColor" stroke="none"/>+ </svg>+ </span>+ <span class="endpoint-title">Chrome</span>+ </div>+ <p class="endpoint-meta">Skyvern Agent extension</p>+ </div>+ </div>++ <div class="approval-controls">+ <button id="approve" class="approve-button" type="button">+ <span class="spinner" aria-hidden="true"></span>+ <span id="approve-label">Approve connection</span>+ </button>+ <p class="fine-print">This link expires in 2 minutes and can be used once.</p>+ </div>++ <section id="status" class="status-panel" role="status" aria-live="polite" hidden>+ <div class="status-heading">+ <span class="status-icon" aria-hidden="true">+ <svg class="error-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round">+ <path d="M12 8v5M12 17h.01"/>+ <circle cx="12" cy="12" r="9"/>+ </svg>+ <svg class="success-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.4" stroke-linecap="round" stroke-linejoin="round">+ <path d="m7 12.5 3.2 3.2L17.5 8.5"/>+ </svg>+ </span>+ <h2 id="status-title" class="status-title"></h2>+ </div>+ <p id="status-message" class="status-message"></p>+ <div id="recovery" class="recovery" hidden>+ <span class="recovery-label">Start a new pairing link</span>+ <button id="copy-command" class="copy-chip" type="button" aria-label="Copy skyvern browser extension-pair">+ <code id="recovery-command">skyvern browser extension-pair</code>+ <span id="copy-label" class="copy-label">Copy</span>+ </button>+ </div>+ </section>+ </div>+ </main>+ </div>+ <script>+ (() => {+ const nonce = location.hash.slice(1);+ const command = "skyvern browser extension-pair";+ const button = document.getElementById("approve");+ const buttonLabel = document.getElementById("approve-label");+ const copyButton = document.getElementById("copy-command");+ const copyLabel = document.getElementById("copy-label");+ const recovery = document.getElementById("recovery");+ const serverAddress = document.getElementById("server-address");+ const status = document.getElementById("status");+ const statusTitle = document.getElementById("status-title");+ const statusMessage = document.getElementById("status-message");++ serverAddress.textContent = `127.0.0.1:${location.port}`;++ function showState(kind, title, message, showRecovery = false) {+ document.body.dataset.state = kind;+ status.hidden = false;+ statusTitle.textContent = title;+ statusMessage.textContent = message;+ recovery.hidden = !showRecovery;+ }++ function showExpired() {+ showState(+ "error",+ "Pairing link expired",+ "This one-time link is no longer valid. Start a new pairing request to continue.",+ true,+ );+ }++ function showInstallHint() {+ showState(+ "error",+ "Skyvern Agent isn’t available",+ "Install or enable the Skyvern Agent Chrome extension, then start a new pairing request.",+ true,+ );+ }++ async function copyRecoveryCommand() {+ try {+ await navigator.clipboard.writeText(command);+ } catch (_error) {+ const range = document.createRange();+ range.selectNodeContents(document.getElementById("recovery-command"));+ const selection = globalThis.getSelection();+ selection.removeAllRanges();+ selection.addRange(range);+ document.execCommand("copy");+ selection.removeAllRanges();+ }+ copyLabel.textContent = "Copied";+ setTimeout(() => { copyLabel.textContent = "Copy"; }, 1600);+ }++ button.addEventListener("click", async () => {+ button.disabled = true;+ button.setAttribute("aria-busy", "true");+ buttonLabel.textContent = "Approving…";+ document.body.dataset.state = "approving";+ if (!nonce) {+ showExpired();+ return;+ }+ try {+ const response = await fetch("/pair/claim", {+ method: "POST",+ headers: {"Content-Type": "application/json"},+ body: JSON.stringify({v: 1, nonce}),+ });+ if (response.status === 403) {+ showExpired();+ return;+ }+ if (!response.ok) {+ throw new Error("claim failed");+ }+ const offer = await response.json();+ if (!globalThis.chrome?.runtime?.sendMessage) {+ showInstallHint();+ return;+ }+ chrome.runtime.sendMessage(+ "__EXTENSION_ID__",+ {type: "skyvern.pairingOffer", v: 1, port: offer.port, token: offer.token},+ (result) => {+ if (chrome.runtime.lastError) {+ showInstallHint();+ } else if (result?.ok === true && result?.pending === true) {+ showState(+ "success",+ "Approved — finish in the Skyvern Agent tab",+ "Review the local server details in the confirmation tab that just opened.",+ );+ } else {+ showState(+ "error",+ "Pairing request wasn’t accepted",+ "The extension could not open the confirmation step. Start a new pairing request to try again.",+ true,+ );+ }+ },+ );+ } catch (_error) {+ showState(+ "error",+ "Pairing couldn’t be completed",+ "Check that the local MCP server is running, then start a new pairing request.",+ true,+ );+ }+ });++ copyButton.addEventListener("click", () => { void copyRecoveryCommand(); });+ })();+ </script>+</body>+</html>+"""+_PAIR_PAGE = _PAIR_PAGE_TEMPLATE.replace("__EXTENSION_ID__", EXTENSION_ID)+++class ExtensionRelayServer:+ def __init__(+ self,+ token: str,+ port: int,+ on_event: Callable[[str, dict], Awaitable[None]],+ on_disconnect: Callable[[], Awaitable[None]] | None = None,+ ) -> None:+ self._token = token+ self._port = port+ self._on_event = on_event+ self._on_disconnect = on_disconnect+ self._app = web.Application()+ self._app.router.add_get("/extension/v1", self._handle_websocket)+ self._app.router.add_get("/pair", self._handle_pair_page)+ self._app.router.add_post("/pair/begin", self._handle_pair_begin)+ self._app.router.add_post("/pair/claim", self._handle_pair_claim)+ self._runner: web.AppRunner | None = None+ self._site: web.TCPSite | None = None+ self._websocket: web.WebSocketResponse | None = None+ self._connection_lock = asyncio.Lock()+ self._connected_event = asyncio.Event()+ self._request_ids = itertools.count(1)+ self._pending: dict[str, asyncio.Future[dict]] = {}+ self._pairing_nonce: str | None = None+ self._pairing_nonce_created_at: float | None = None+ self.bound_port = port+ self.scoped_tabs: list[dict] = []++ def create_pairing_nonce(self) -> str:+ nonce = secrets.token_urlsafe(32)+ self._pairing_nonce = nonce+ self._pairing_nonce_created_at = time.monotonic()+ return nonce++ async def start(self) -> None:+ if self._runner is not None:+ return++ runner = web.AppRunner(self._app)+ await runner.setup()+ site = web.TCPSite(runner, "127.0.0.1", self._port)+ try:+ await site.start()+ except BaseException:+ await runner.cleanup()+ raise++ server = site._server+ if server is None or not server.sockets:+ await runner.cleanup()+ raise BrowserExtensionError("Browser extension relay did not bind a loopback port")++ self.bound_port = int(server.sockets[0].getsockname()[1])+ self._runner = runner+ self._site = site++ async def stop(self) -> None:+ runner = self._runner+ if runner is None:+ return++ websocket = self._websocket+ if websocket is not None and not websocket.closed:+ await websocket.close(code=1001, message=b"relay stopped")+ await runner.cleanup()+ if websocket is not None:+ await self._handle_disconnect(websocket)+ self._runner = None+ self._site = None++ @property+ def connected(self) -> bool:+ return self._connected_event.is_set() and self._websocket is not None and not self._websocket.closed++ async def wait_connected(self, timeout: float) -> bool:+ if self.connected:+ return True+ try:+ await asyncio.wait_for(self._connected_event.wait(), timeout)+ except TimeoutError:+ return False+ return self.connected++ async def request(self, op: str, args: dict, timeout: float = 30.0) -> dict:+ websocket = self._websocket+ if not self.connected or websocket is None:+ raise BrowserExtensionNotConnectedError("Skyvern browser extension is not connected")++ request_id = f"r-{next(self._request_ids)}"+ frame = build_request(request_id, op, args)+ future: asyncio.Future[dict] = asyncio.get_running_loop().create_future()+ self._pending[request_id] = future+ try:+ await websocket.send_json(frame)+ except (ConnectionError, RuntimeError):+ pending = self._pending.pop(request_id, None)+ if pending is not None:+ pending.cancel()+ raise BrowserExtensionNotConnectedError("Skyvern browser extension is not connected") from None++ try:+ return await asyncio.wait_for(asyncio.shield(future), timeout)+ except TimeoutError:+ pending = self._pending.pop(request_id, None)+ if pending is not None:+ pending.cancel()+ raise ExtensionRequestError("INTERNAL", f"extension request timed out: {op}") from None+ except asyncio.CancelledError:+ pending = self._pending.pop(request_id, None)+ if pending is not None:+ pending.cancel()+ raise++ async def _handle_pair_page(self, _request: web.Request) -> web.Response:+ return web.Response(+ text=_PAIR_PAGE,+ content_type="text/html",+ headers={+ "Cache-Control": "no-store",+ "Content-Security-Policy": (+ "default-src 'none'; script-src 'unsafe-inline'; style-src 'unsafe-inline'; "+ "img-src data:; connect-src 'self'"+ ),+ "Referrer-Policy": "no-referrer",+ "X-Content-Type-Options": "nosniff",+ },+ )++ async def _handle_pair_begin(self, request: web.Request) -> web.Response:+ payload = await _read_json_object(request)+ proof = payload.get("proof") if payload is not None else None+ expected = hmac.new(self._token.encode("utf-8"), _PAIR_BEGIN_MESSAGE, hashlib.sha256).hexdigest()+ supplied = proof if isinstance(proof, str) else ""+ valid = hmac.compare_digest(expected, supplied)+ if payload is None or type(payload.get("v")) is not int or payload["v"] != PROTOCOL_VERSION or not valid:+ return web.json_response({"error": "invalid_proof"}, status=403, headers={"Cache-Control": "no-store"})+ return web.json_response(+ {"v": PROTOCOL_VERSION, "nonce": self.create_pairing_nonce()},+ headers={"Cache-Control": "no-store"},+ )++ async def _handle_pair_claim(self, request: web.Request) -> web.Response:+ active_nonce = self._pairing_nonce+ created_at = self._pairing_nonce_created_at+ self._pairing_nonce = None+ self._pairing_nonce_created_at = None++ payload = await _read_json_object(request)+ supplied_nonce = payload.get("nonce") if payload is not None else None+ nonce_matches = secrets.compare_digest(+ active_nonce or ("0" * 43),+ supplied_nonce if isinstance(supplied_nonce, str) else "",+ )+ expired = created_at is None or time.monotonic() - created_at >= _PAIRING_NONCE_TTL_SECONDS+ if (+ payload is None+ or type(payload.get("v")) is not int+ or payload["v"] != PROTOCOL_VERSION+ or not nonce_matches+ or expired+ ):+ return web.json_response({"error": "invalid_nonce"}, status=403, headers={"Cache-Control": "no-store"})+ return web.json_response(+ {"v": PROTOCOL_VERSION, "port": self.bound_port, "token": self._token},+ headers={"Cache-Control": "no-store"},+ )++ async def _handle_websocket(self, request: web.Request) -> web.WebSocketResponse:+ websocket = web.WebSocketResponse(max_msg_size=_MAX_WS_MESSAGE_BYTES)+ await websocket.prepare(request)++ origin = request.headers.get("Origin")+ if origin is not None and not _is_extension_origin(origin):+ await websocket.close(code=_AUTH_CLOSE_CODE, message=b"authentication failed")+ return websocket++ server_nonce, challenge = build_challenge()+ await websocket.send_str(challenge)+ try:+ proof_frame = await websocket.receive(timeout=_AUTH_TIMEOUT_SECONDS)+ if proof_frame.type is not WSMsgType.TEXT:+ raise BrowserExtensionError("Authentication proof must be a text frame")+ proof = parse_extension_message(proof_frame.data)+ if proof.kind != "auth.proof" or proof.client_nonce is None or proof.proof is None:+ raise BrowserExtensionError("Authentication proof is missing")+ if not _is_valid_client_nonce(proof.client_nonce):+ raise BrowserExtensionError("Authentication nonce is invalid")+ if not verify_ext_proof(self._token, server_nonce, proof.client_nonce, proof.proof):+ raise BrowserExtensionError("Authentication proof is invalid")+ except (BrowserExtensionError, TimeoutError):+ await websocket.close(code=_AUTH_CLOSE_CODE, message=b"authentication failed")+ return websocket++ await websocket.send_json(+ {+ "v": 1,+ "type": "auth.ok",+ "serverProof": compute_server_proof(self._token, proof.client_nonce, server_nonce),+ }+ )+ await self._activate_connection(websocket)++ loop = asyncio.get_running_loop()+ last_inbound = loop.time()+ keepalive_task = asyncio.create_task(self._run_keepalive(websocket, lambda: last_inbound))+ try:+ async for message in websocket:+ last_inbound = loop.time()+ if self._websocket is not websocket:+ break+ if message.type is WSMsgType.TEXT:+ await self._handle_text_frame(websocket, message.data)+ elif message.type in {WSMsgType.CLOSE, WSMsgType.CLOSED, WSMsgType.ERROR}:+ break+ else:+ LOG.warning("browser extension sent a non-text frame")+ finally:+ keepalive_task.cancel()+ with suppress(asyncio.CancelledError):+ await keepalive_task+ await self._handle_disconnect(websocket)+ return websocket++ async def _activate_connection(self, websocket: web.WebSocketResponse) -> None:+ async with self._connection_lock:+ previous = self._websocket+ self._websocket = websocket+ self._connected_event.clear()++ if previous is not None and previous is not websocket:+ self._fail_pending_requests()+ self.scoped_tabs = []+ await self._call_on_disconnect()+ if not previous.closed:+ await previous.close(code=_REPLACED_CLOSE_CODE, message=b"replaced")++ async def _handle_text_frame(self, websocket: web.WebSocketResponse, raw: str) -> None:+ try:+ message = parse_extension_message(raw)+ except BrowserExtensionError:+ LOG.warning("browser extension sent an invalid protocol frame")+ return++ if message.kind == "response":+ self._handle_response(message)+ elif message.kind == "event":+ await self._handle_event(message)+ if message.event == "extension.hello":+ await self._mark_hello_processed(websocket)+ elif message.kind == "ping":+ await websocket.send_json({"v": 1, "type": "pong"})++ async def _mark_hello_processed(self, websocket: web.WebSocketResponse) -> None:+ async with self._connection_lock:+ if self._websocket is websocket and not websocket.closed:+ self._connected_event.set()++ def _handle_response(self, message: ParsedMessage) -> None:+ if message.request_id is None:+ return+ future = self._pending.pop(message.request_id, None)+ if future is None or future.done():+ return+ if message.ok:+ future.set_result(message.result or {})+ return+ if message.error_code is None or message.error_message is None:+ future.set_exception(ExtensionRequestError("INTERNAL", "extension returned an invalid error"))+ return+ future.set_exception(ExtensionRequestError(message.error_code, message.error_message))++ async def _handle_event(self, message: ParsedMessage) -> None:+ if message.event is None or message.params is None:+ return+ self._update_scoped_tabs(message.event, message.params)+ try:+ await self._on_event(message.event, message.params)+ except Exception:+ LOG.exception("browser extension event callback failed", event=message.event)++ def _update_scoped_tabs(self, event: str, params: dict) -> None:+ if event == "extension.hello":+ tabs = params.get("scopedTabs")+ if not isinstance(tabs, list):+ self.scoped_tabs = []+ return+ self.scoped_tabs = [snapshot for tab in tabs if (snapshot := _tab_snapshot(tab)) is not None]+ return+ if event in {"scope.tabAdded", "tabs.created"}:+ snapshot = _tab_snapshot(params)+ if snapshot is None:+ return+ self.scoped_tabs = [tab for tab in self.scoped_tabs if tab.get("tabId") != snapshot["tabId"]]+ self.scoped_tabs.append(snapshot)+ return+ if event == "scope.tabRemoved":+ tab_id = params.get("tabId")+ if type(tab_id) is int:+ self.scoped_tabs = [tab for tab in self.scoped_tabs if tab.get("tabId") != tab_id]++ async def _run_keepalive(+ self,+ websocket: web.WebSocketResponse,+ last_inbound: Callable[[], float],+ ) -> None:+ loop = asyncio.get_running_loop()+ next_ping = loop.time() + _PING_INTERVAL_SECONDS+ while not websocket.closed:+ now = loop.time()+ silence = now - last_inbound()+ if silence >= _INBOUND_TIMEOUT_SECONDS:+ await websocket.close(code=1001, message=b"inbound timeout")+ return+ if now >= next_ping:+ try:+ await websocket.send_json({"v": 1, "type": "ping"})+ except (ConnectionError, RuntimeError):+ return+ next_ping = now + _PING_INTERVAL_SECONDS+ delay = min(next_ping - now, _INBOUND_TIMEOUT_SECONDS - silence)+ await asyncio.sleep(max(delay, 0.01))++ async def _handle_disconnect(self, websocket: web.WebSocketResponse) -> None:+ async with self._connection_lock:+ if self._websocket is not websocket:+ return+ self._websocket = None+ self._connected_event.clear()++ self._fail_pending_requests()+ self.scoped_tabs = []+ await self._call_on_disconnect()++ async def _call_on_disconnect(self) -> None:+ if self._on_disconnect is not None:+ try:+ await self._on_disconnect()+ except Exception:+ LOG.exception("browser extension disconnect callback failed")++ def _fail_pending_requests(self) -> None:+ pending = list(self._pending.values())+ self._pending.clear()+ for future in pending:+ if not future.done():+ future.set_exception(BrowserExtensionNotConnectedError("Skyvern browser extension is not connected"))+++def _is_extension_origin(origin: str) -> bool:+ try:+ parsed = urlsplit(origin)+ except ValueError:+ return False+ return parsed.scheme == "chrome-extension" and bool(parsed.netloc)+++async def _read_json_object(request: web.Request) -> dict | None:+ try:+ payload = await request.json(loads=json.loads)+ except (json.JSONDecodeError, UnicodeDecodeError):+ return None+ return payload if isinstance(payload, dict) else None+++def _is_valid_client_nonce(client_nonce: str) -> bool:+ if not client_nonce or "=" in client_nonce:+ return False+ padding = "=" * (-len(client_nonce) % 4)+ try:+ decoded = base64.b64decode(client_nonce + padding, altchars=b"-_", validate=True)+ except (binascii.Error, ValueError):+ return False+ canonical = base64.urlsafe_b64encode(decoded).rstrip(b"=").decode("ascii")+ return len(decoded) == 32 and canonical == client_nonce+++def _tab_snapshot(value: object) -> dict | None:+ if not isinstance(value, dict):+ return None+ tab_id = value.get("tabId")+ if type(tab_id) is not int:+ return None+ url = value.get("url")+ title = value.get("title")+ return {+ "tabId": tab_id,+ "url": url if isinstance(url, str) else "",+ "title": title if isinstance(title, str) else "",+ }diff --git a/skyvern/browser_extension/runtime.py b/skyvern/browser_extension/runtime.pynew file mode 100644index 000000000..5eb36afb6--- /dev/null+++ b/skyvern/browser_extension/runtime.py@@ -0,0 +1,164 @@+from __future__ import annotations++import asyncio+import errno+import os+from collections.abc import Awaitable, Callable+from pathlib import Path+from typing import Protocol++import structlog++from skyvern.browser_extension.auth import load_or_create_pairing_token+from skyvern.browser_extension.errors import BrowserExtensionError+from skyvern.browser_extension.relay import ExtensionRelayServer+from skyvern.browser_extension.target_registry import VirtualTargetRegistry++LOG = structlog.get_logger(__name__)++_PORT_ENV = "SKYVERN_BROWSER_EXTENSION_PORT"+_DEFAULT_PORT = 19777+++class _Adapter(Protocol):+ @property+ def cdp_ws_url(self) -> str: ...++ async def start(self) -> None: ...++ async def stop(self) -> None: ...++ async def handle_extension_event(self, event: str, params: dict) -> None: ...++ async def on_extension_disconnect(self) -> None: ...+++_relay_factory: Callable[+ [str, int, Callable[[str, dict], Awaitable[None]], Callable[[], Awaitable[None]] | None],+ ExtensionRelayServer,+] = ExtensionRelayServer+_adapter_factory: Callable[[VirtualTargetRegistry, ExtensionRelayServer], _Adapter] | None = None+++class BrowserExtensionRuntime:+ _instance: BrowserExtensionRuntime | None = None+ _lock = asyncio.Lock()++ def __init__(self, relay: ExtensionRelayServer, adapter: _Adapter) -> None:+ self._relay = relay+ self._adapter = adapter+ self._stopped = False++ @classmethod+ async def get_or_start(cls, port: int | None = None) -> BrowserExtensionRuntime:+ async with cls._lock:+ if cls._instance is not None:+ return cls._instance++ resolved_port = _resolve_port(port)+ token = load_or_create_pairing_token()+ registry = VirtualTargetRegistry()+ adapter_holder: list[_Adapter] = []++ async def handle_event(event: str, params: dict) -> None:+ await adapter_holder[0].handle_extension_event(event, params)++ async def on_disconnect() -> None:+ await adapter_holder[0].on_extension_disconnect()++ relay = _relay_factory(token, resolved_port, handle_event, on_disconnect)+ adapter = _create_adapter(registry, relay)+ adapter_holder.append(adapter)++ try:+ await adapter.start()+ await relay.start()+ except OSError as exc:+ await _cleanup_failed_start(relay, adapter)+ if exc.errno == errno.EADDRINUSE:+ raise BrowserExtensionError(+ f"Browser extension relay port {resolved_port} is already in use; "+ f"set {_PORT_ENV} to a free port"+ ) from exc+ raise+ except BaseException:+ await _cleanup_failed_start(relay, adapter)+ raise++ runtime = cls(relay, adapter)+ cls._instance = runtime+ return runtime++ @classmethod+ def instance(cls) -> BrowserExtensionRuntime | None:+ return cls._instance++ @property+ def cdp_ws_url(self) -> str:+ return self._adapter.cdp_ws_url++ @property+ def extension_connected(self) -> bool:+ return self._relay.connected++ async def wait_for_extension(self, timeout: float = 10.0) -> bool:+ return await self._relay.wait_connected(timeout)++ async def shutdown(self) -> None:+ cls = type(self)+ async with cls._lock:+ if self._stopped:+ return+ self._stopped = True+ try:+ await self._relay.stop()+ finally:+ try:+ await self._adapter.stop()+ finally:+ if cls._instance is self:+ cls._instance = None++ @staticmethod+ def extension_dir() -> Path:+ return Path(__file__).resolve().parent / "extension"++ @staticmethod+ def configured_port() -> int:+ return _resolve_port(None)+++def _create_adapter(registry: VirtualTargetRegistry, relay: ExtensionRelayServer) -> _Adapter:+ if _adapter_factory is not None:+ return _adapter_factory(registry, relay)+ from skyvern.browser_extension.cdp_adapter import ExtensionCdpAdapter++ return ExtensionCdpAdapter(registry, relay)+++def _resolve_port(port: int | None) -> int:+ if port is not None:+ resolved_port = port+ else:+ environment_port = os.environ.get(_PORT_ENV)+ if environment_port is None:+ resolved_port = _DEFAULT_PORT+ else:+ try:+ resolved_port = int(environment_port)+ except ValueError as exc:+ raise BrowserExtensionError(f"{_PORT_ENV} must be an integer port") from exc+ if type(resolved_port) is not int or not 1 <= resolved_port <= 65535:+ raise BrowserExtensionError(f"{_PORT_ENV} must be a port between 1 and 65535")+ return resolved_port+++async def _cleanup_failed_start(relay: ExtensionRelayServer, adapter: _Adapter) -> None:+ try:+ await relay.stop()+ except Exception:+ LOG.exception("failed to stop browser extension relay after startup error")+ try:+ await adapter.stop()+ except Exception:+ LOG.exception("failed to stop browser extension adapter after startup error")diff --git a/skyvern/browser_extension/target_registry.py b/skyvern/browser_extension/target_registry.pynew file mode 100644index 000000000..2f47f803a--- /dev/null+++ b/skyvern/browser_extension/target_registry.py@@ -0,0 +1,173 @@+from __future__ import annotations++from dataclasses import dataclass+++@dataclass(frozen=True, slots=True)+class _TabRecord:+ target_id: str+ root_session_id: str+++@dataclass(frozen=True, slots=True)+class _ChildSessionRecord:+ tab_id: int+ target_id: str+++class VirtualTargetRegistry:+ def __init__(self) -> None:+ self._attach_counter = 0+ self._alias_counter = 0+ self._tabs: dict[int, _TabRecord] = {}+ self._root_sessions: dict[str, int] = {}+ self._tab_root_aliases: dict[int, list[str]] = {}+ self._browser_session_aliases: set[str] = set()+ self._child_sessions: dict[str, _ChildSessionRecord] = {}+ self._child_target_sessions: dict[str, str] = {}+ self._target_tabs: dict[str, int] = {}+ self._target_infos: dict[str, dict] = {}++ def register_tab(self, tab_id: int, url: str, title: str, target_id: str | None = None) -> str:+ self.remove_tab(tab_id)+ self._attach_counter += 1+ # Playwright requires the page targetId to equal Chrome's main-frame id;+ # callers pass the real frame id once the debugger is attached.+ target_id = target_id or f"tab-{tab_id}"+ root_session_id = f"sess-tab-{tab_id}-{self._attach_counter}"+ self._tabs[tab_id] = _TabRecord(target_id=target_id, root_session_id=root_session_id)+ self._root_sessions[root_session_id] = tab_id+ self._target_tabs[target_id] = tab_id+ self._target_infos[target_id] = {+ "targetId": target_id,+ "type": "page",+ "title": title,+ "url": url,+ "attached": True,+ "canAccessOpener": False,+ "browserContextId": "skyvern-default",+ }+ return target_id++ def remove_tab(self, tab_id: int) -> None:+ tab = self._tabs.pop(tab_id, None)+ if tab is not None:+ self._root_sessions.pop(tab.root_session_id, None)+ for alias_session_id in self._tab_root_aliases.pop(tab_id, []):+ self._root_sessions.pop(alias_session_id, None)+ self._target_tabs.pop(tab.target_id, None)+ self._target_infos.pop(tab.target_id, None)++ child_session_ids = [+ session_id for session_id, record in self._child_sessions.items() if record.tab_id == tab_id+ ]+ for session_id in child_session_ids:+ self.remove_child_session(session_id)++ def root_session_id(self, tab_id: int) -> str:+ return self._tabs[tab_id].root_session_id++ def root_session_ids(self, tab_id: int) -> list[str]:+ return [self.root_session_id(tab_id), *self._tab_root_aliases.get(tab_id, [])]++ def create_root_session_alias(self, tab_id: int) -> str:+ if tab_id not in self._tabs:+ raise KeyError(tab_id)+ self._alias_counter += 1+ session_id = f"alias-tab-{tab_id}-{self._alias_counter}"+ self._root_sessions[session_id] = tab_id+ self._tab_root_aliases.setdefault(tab_id, []).append(session_id)+ return session_id++ def remove_root_session_alias(self, session_id: str) -> bool:+ tab_id = self._root_sessions.get(session_id)+ if tab_id is None:+ return False+ aliases = self._tab_root_aliases.get(tab_id)+ if aliases is None or session_id not in aliases:+ return False+ aliases.remove(session_id)+ if not aliases:+ self._tab_root_aliases.pop(tab_id, None)+ self._root_sessions.pop(session_id, None)+ return True++ def create_browser_session_alias(self) -> str:+ self._alias_counter += 1+ session_id = f"browser-alias-{self._alias_counter}"+ self._browser_session_aliases.add(session_id)+ return session_id++ def is_browser_session_alias(self, session_id: str) -> bool:+ return session_id in self._browser_session_aliases++ def remove_browser_session_alias(self, session_id: str) -> bool:+ if session_id not in self._browser_session_aliases:+ return False+ self._browser_session_aliases.remove(session_id)+ return True++ def target_id_for_tab(self, tab_id: int) -> str:+ return self._tabs[tab_id].target_id++ def target_info_for_tab(self, tab_id: int) -> dict:+ return self.target_info(self._tabs[tab_id].target_id)++ def register_child_session(self, tab_id: int, chrome_session_id: str, target_info: dict) -> None:+ if tab_id not in self._tabs:+ raise KeyError(tab_id)+ target_id = target_info["targetId"]+ if not isinstance(target_id, str):+ raise KeyError("targetId")++ self.remove_child_session(chrome_session_id)+ existing_session_id = self._child_target_sessions.get(target_id)+ if existing_session_id is not None:+ self.remove_child_session(existing_session_id)++ self._child_sessions[chrome_session_id] = _ChildSessionRecord(tab_id=tab_id, target_id=target_id)+ self._child_target_sessions[target_id] = chrome_session_id+ self._target_tabs[target_id] = tab_id+ self._target_infos[target_id] = dict(target_info)++ def remove_child_session(self, chrome_session_id: str) -> None:+ record = self._child_sessions.pop(chrome_session_id, None)+ if record is None:+ return+ if self._child_target_sessions.get(record.target_id) == chrome_session_id:+ self._child_target_sessions.pop(record.target_id, None)+ self._target_tabs.pop(record.target_id, None)+ self._target_infos.pop(record.target_id, None)++ def resolve_session(self, session_id: str) -> tuple[int, str | None]:+ root_tab_id = self._root_sessions.get(session_id)+ if root_tab_id is not None:+ return root_tab_id, None+ child = self._child_sessions.get(session_id)+ if child is None:+ raise KeyError(session_id)+ return child.tab_id, session_id++ def target_info(self, target_id: str) -> dict:+ return dict(self._target_infos[target_id])++ def list_page_targets(self) -> list[dict]:+ return [self.target_info(tab.target_id) for tab in self._tabs.values()]++ def tab_for_target(self, target_id: str) -> int:+ return self._target_tabs[target_id]++ def update_tab(self, tab_id: int, url: str, title: str) -> None:+ target_id = self._tabs[tab_id].target_id+ self._target_infos[target_id]["url"] = url+ self._target_infos[target_id]["title"] = title++ def clear(self) -> None:+ self._tabs.clear()+ self._root_sessions.clear()+ self._tab_root_aliases.clear()+ self._browser_session_aliases.clear()+ self._child_sessions.clear()+ self._child_target_sessions.clear()+ self._target_tabs.clear()+ self._target_infos.clear()diff --git a/skyvern/cli/commands/browser.py b/skyvern/cli/commands/browser.pyindex 37a57748d..7d069312c 100644--- a/skyvern/cli/commands/browser.py+++ b/skyvern/cli/commands/browser.py@@ -1,7 +1,14 @@from __future__ import annotationsimport asyncio+import hashlib+import hmac+import http.clientimport json+import os+import shutil+import socket+import statimport subprocessimport sysfrom dataclasses import asdict, dataclass@@ -14,6 +21,9 @@ if TYPE_CHECKING:import typer+from skyvern.browser_extension.auth import load_or_create_pairing_token+from skyvern.browser_extension.errors import BrowserExtensionError+from skyvern.browser_extension.runtime import BrowserExtensionRuntimefrom skyvern.cli.commands._output import (console,)@@ -182,6 +192,254 @@ def _handle_tool_error(e: Exception, *, tool: str, hint: str, json_output: bool)output_error(str(e), hint=hint, json_mode=json_output)+def _clipboard_command() -> list[str] | None:+ if sys.platform == "darwin":+ executable = shutil.which("pbcopy")+ return [executable] if executable else None+ if sys.platform.startswith("linux"):+ executable = shutil.which("xclip")+ if executable:+ return [executable, "-selection", "clipboard"]+ executable = shutil.which("xsel")+ if executable:+ return [executable, "--clipboard", "--input"]+ return None+++def _copy_pairing_token_to_clipboard(token: str) -> bool:+ command = _clipboard_command()+ if command is None:+ return False+ try:+ subprocess.run(+ command,+ input=token,+ text=True,+ check=True,+ stdout=subprocess.DEVNULL,+ stderr=subprocess.DEVNULL,+ )+ except (OSError, subprocess.CalledProcessError):+ return False+ return True+++def _open_chrome_extensions() -> bool:+ url = "chrome://extensions"+ command: list[str] | None = None+ if sys.platform == "darwin":+ executable = shutil.which("open")+ if executable:+ command = [executable, "-a", "Google Chrome", url]+ elif sys.platform.startswith("linux"):+ for browser_name in ("google-chrome", "chromium", "chromium-browser"):+ executable = shutil.which(browser_name)+ if executable:+ command = [executable, url]+ break+ if command is None:+ return False+ try:+ subprocess.run(command, check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)+ except (OSError, subprocess.CalledProcessError):+ return False+ return True+++def _pairing_token_file_status() -> tuple[bool, str]:+ token_path = Path.home() / ".skyvern" / "browser_extension_token"+ try:+ file_status = token_path.lstat()+ except FileNotFoundError:+ return False, "not applicable (file does not exist)"+ except OSError:+ return False, "NOT OK (file cannot be inspected)"++ permissions_ok = stat.S_ISREG(file_status.st_mode) and not (stat.S_IMODE(file_status.st_mode) & 0o177)+ if hasattr(os, "getuid"):+ permissions_ok = permissions_ok and file_status.st_uid == os.getuid()+ return True, "OK" if permissions_ok else "NOT OK"+++def _bridge_is_listening(port: int) -> bool:+ try:+ connection = socket.create_connection(("127.0.0.1", port), timeout=0.5)+ except OSError:+ return False+ try:+ connection.close()+ except OSError:+ pass+ return True+++def _request_pairing_nonce(port: int, token: str) -> str:+ proof = hmac.new(token.encode("utf-8"), b"skyvern-pair-begin-v1", hashlib.sha256).hexdigest()+ body = json.dumps({"v": 1, "proof": proof}, separators=(",", ":"))+ connection = http.client.HTTPConnection("127.0.0.1", port, timeout=2.0)+ try:+ connection.request("POST", "/pair/begin", body=body, headers={"Content-Type": "application/json"})+ response = connection.getresponse()+ response_body = response.read(65_537)+ except OSError as exc:+ raise BrowserExtensionError("The browser extension bridge did not accept the pairing request") from exc+ finally:+ connection.close()+ if response.status != 200:+ raise BrowserExtensionError("The browser extension bridge rejected the pairing request")+ if len(response_body) > 65_536:+ raise BrowserExtensionError("The browser extension bridge returned an invalid pairing response")+ try:+ payload = json.loads(response_body)+ except (json.JSONDecodeError, UnicodeDecodeError) as exc:+ raise BrowserExtensionError("The browser extension bridge returned an invalid pairing response") from exc+ nonce = (+ payload.get("nonce")+ if isinstance(payload, dict) and type(payload.get("v")) is int and payload["v"] == 1+ else None+ )+ if not isinstance(nonce, str) or not nonce:+ raise BrowserExtensionError("The browser extension bridge returned an invalid pairing response")+ return nonce+++def _open_pairing_url(url: str) -> bool:+ command: list[str] | None = None+ if sys.platform == "darwin":+ executable = shutil.which("open")+ if executable:+ command = [executable, url]+ elif sys.platform.startswith("linux"):+ executable = shutil.which("xdg-open")+ if executable:+ command = [executable, url]+ if command is None:+ return False+ try:+ subprocess.run(command, check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)+ except (OSError, subprocess.CalledProcessError):+ return False+ return True+++def _launch_extension_pairing(port: int) -> None:+ token = load_or_create_pairing_token()+ nonce = _request_pairing_nonce(port, token)+ url = f"http://127.0.0.1:{port}/pair#{nonce}"+ if not _open_pairing_url(url):+ console.print(url, markup=False, soft_wrap=True)+ console.print("Approve the pairing in your browser.")+++@browser_app.command("extension-path")+def extension_path() -> None:+ console.print(str(BrowserExtensionRuntime.extension_dir().resolve()), markup=False, soft_wrap=True)+++@browser_app.command("extension-token")+def extension_token() -> None:+ """Copy the pairing token for manual popup-paste fallback."""+ token = load_or_create_pairing_token()+ if _copy_pairing_token_to_clipboard(token):+ console.print("Pairing token copied to clipboard.")+ else:+ console.print(+ "Pairing token was not printed. Copy it from ~/.skyvern/browser_extension_token "+ "or SKYVERN_BROWSER_EXTENSION_TOKEN."+ )+ console.print("Paste this token into the Skyvern browser extension popup.")+ console.print("Click Connect.")+++@browser_app.command("extension-status")+def extension_status() -> None:+ console.print(+ f"extension directory: {BrowserExtensionRuntime.extension_dir().resolve()}",+ markup=False,+ soft_wrap=True,+ )++ environment_configured = bool(os.environ.get("SKYVERN_BROWSER_EXTENSION_TOKEN", "").strip())+ token_file_exists, permissions = _pairing_token_file_status()+ if environment_configured:+ console.print("pairing token: configured (environment override)")+ elif token_file_exists:+ console.print("pairing token: configured (file exists)")+ else:+ console.print("pairing token: not configured")+ console.print(f"pairing token file permissions: {permissions}")++ try:+ port = BrowserExtensionRuntime.configured_port()+ except BrowserExtensionError as exc:+ console.print(f"bridge configuration invalid ({exc})")+ return++ if not _bridge_is_listening(port):+ console.print("bridge not running (start your MCP server with --browser-extension)")+ else:+ console.print(f"bridge listening on {port}")+++@browser_app.command("extension-pair")+def extension_pair() -> None:+ try:+ port = BrowserExtensionRuntime.configured_port()+ except BrowserExtensionError:+ console.print("Start your MCP server first: skyvern run mcp --browser-extension")+ raise typer.Exit(code=1) from None+ if not _bridge_is_listening(port):+ console.print("Start your MCP server first: skyvern run mcp --browser-extension")+ raise typer.Exit(code=1)+ try:+ _launch_extension_pairing(port)+ except BrowserExtensionError as exc:+ console.print(str(exc))+ raise typer.Exit(code=1) from None+++@browser_app.command("extension-install")+def extension_install() -> None:+ extension_dir = BrowserExtensionRuntime.extension_dir().resolve()+ console.print(str(extension_dir), markup=False, soft_wrap=True)++ try:+ port = BrowserExtensionRuntime.configured_port()+ bridge_listening = _bridge_is_listening(port)+ except BrowserExtensionError:+ port = None+ bridge_listening = False++ if not bridge_listening:+ token = load_or_create_pairing_token()+ if _copy_pairing_token_to_clipboard(token):+ console.print("Pairing token copied to clipboard.")+ else:+ console.print("No clipboard tool found. Use skyvern browser extension-token for the manual fallback.")++ if _open_chrome_extensions():+ console.print("Opened chrome://extensions.")+ else:+ console.print("chrome://extensions", markup=False)++ console.print("1. Enable Developer mode.")+ console.print("2. Click Load unpacked.")+ console.print(f"3. Select {extension_dir}.", markup=False, soft_wrap=True)+ if bridge_listening and port is not None:+ console.print("4. Click Approve in the pairing page.")+ console.print("5. Approve the pairing in the Skyvern Agent confirmation tab.")+ console.print('6. Add tabs to the "Skyvern Controlled" group.')+ try:+ _launch_extension_pairing(port)+ except BrowserExtensionError:+ console.print("Automatic pairing could not start. Run skyvern browser extension-pair after setup.")+ else:+ console.print("4. Open the Skyvern Agent popup.")+ console.print("5. Paste the pairing token and click Connect.")+ console.print('6. Add tabs to the "Skyvern Controlled" group.')+ console.print("When your MCP server is running, pair with: skyvern browser extension-pair")++# ---------------------------------------------------------------------------# Session commands# ---------------------------------------------------------------------------diff --git a/skyvern/cli/core/browser_ops.py b/skyvern/cli/core/browser_ops.pyindex d267a6a82..72be6460c 100644--- a/skyvern/cli/core/browser_ops.py+++ b/skyvern/cli/core/browser_ops.py@@ -23,10 +23,44 @@ from skyvern.forge.sdk.settings_manager import SettingsManagerfrom skyvern.utils.url_validators import validate_fetch_urlfrom skyvern.webeye.utils.page import JS_FUNCTION_DEFS, SkyvernFrame-from .guards import GuardError+from .guards import CREDENTIAL_HINT, GuardErrorLOG = structlog.get_logger(__name__)+TYPE_PASSWORD_REFUSAL_MESSAGE = "Cannot type into password fields — credentials must not be passed through tool calls"+COORDINATE_TYPE_TARGET_REFUSAL_MESSAGE = "could not verify the coordinate target; refusing to type"++_COORDINATE_PASSWORD_TARGET_JS = """+() => {+ let element = document.activeElement;+ while (element) {+ const tag = (element.tagName || "").toLowerCase();+ if (tag === "iframe" || tag === "frame") {+ try {+ element = element.contentDocument?.activeElement || null;+ } catch {+ return null;+ }+ if (!element) return null;+ continue;+ }+ const shadowRoot = element.shadowRoot;+ const shadowActiveElement = shadowRoot?.activeElement;+ if (shadowActiveElement) {+ element = shadowActiveElement;+ continue;+ }+ if (tag.includes("-") && !shadowRoot) return null;+ break;+ }+ if (!element || typeof element.getAttribute !== "function") return null;+ const tag = (element.tagName || "").toLowerCase();+ const type = (element.getAttribute("type") || "").toLowerCase();+ const autocomplete = (element.getAttribute("autocomplete") || "").toLowerCase();+ return tag === "input" && (type === "password" || autocomplete.includes("password"));+}+"""+LOCALHOST_RECOVERY_HINT = ("Run `pip install skyvern && skyvern browser serve --tunnel` to bridge ""your local dev server to a cloud browser via ngrok. "@@ -138,6 +172,49 @@ async def do_extract(return ExtractResult(extracted=extracted)+async def do_click_at(+ page: Any,+ x: float,+ y: float,+ button: str = "left",+ click_count: int = 1,+) -> None:+ raw_page = page.page if hasattr(page, "page") else page+ await raw_page.mouse.click(x, y, button=button, click_count=click_count)+++async def do_type_at(+ page: Any,+ x: float,+ y: float,+ text: str,+ clear_first: bool = False,+ press_enter: bool = False,+) -> None:+ raw_page = page.page if hasattr(page, "page") else page+ await raw_page.mouse.click(x, y)+ try:+ is_password = await raw_page.evaluate(_COORDINATE_PASSWORD_TARGET_JS)+ except Exception as exc:+ raise GuardError(+ COORDINATE_TYPE_TARGET_REFUSAL_MESSAGE,+ "Use a selector or intent to target a verified non-password input",+ ) from exc+ if is_password is True:+ raise GuardError(TYPE_PASSWORD_REFUSAL_MESSAGE, CREDENTIAL_HINT)+ if is_password is not False:+ raise GuardError(+ COORDINATE_TYPE_TARGET_REFUSAL_MESSAGE,+ "Use a selector or intent to target a verified non-password input",+ )+ if clear_first:+ await raw_page.keyboard.press("ControlOrMeta+A")+ await raw_page.keyboard.press("Backspace")+ await raw_page.keyboard.type(text)+ if press_enter:+ await raw_page.keyboard.press("Enter")++# -- Semantic locators --diff --git a/skyvern/cli/core/session_manager.py b/skyvern/cli/core/session_manager.pyindex 213258e66..18ad1e9e7 100644--- a/skyvern/cli/core/session_manager.py+++ b/skyvern/cli/core/session_manager.py@@ -27,6 +27,7 @@ _BODY_SEMAPHORE_LIMIT = 5 # concurrent CDP body downloads (worst case: 5 * 10sif TYPE_CHECKING:from playwright.async_api import Frame, Page+ from skyvern.browser_extension.runtime import BrowserExtensionRuntimefrom skyvern.library.skyvern_browser import SkyvernBrowserfrom skyvern.library.skyvern_browser_page import SkyvernBrowserPage@@ -150,6 +151,8 @@ def _session_ref_key(if resolved_session_id:return (organization_id, "cloud_session", resolved_session_id, api_key_hash)+ if context is not None and context.mode == "extension":+ return (organization_id, "extension", "own-browser", api_key_hash)if resolved_cdp_url:return (organization_id, "cdp", resolved_cdp_url, api_key_hash)return None@@ -395,6 +398,7 @@ def _matches_current(*,session_id: str | None = None,cdp_url: str | None = None,+ extension_runtime: BrowserExtensionRuntime | None = None,local: bool = False,) -> bool:if current.browser is None or current.context is None:@@ -404,6 +408,8 @@ def _matches_current(if session_id:return current.context.mode == "cloud_session" and current.context.session_id == session_id+ if extension_runtime is not None:+ return current.context.mode == "extension"if cdp_url:return current.context.mode == "cdp" and current.context.cdp_url == cdp_urlif local:@@ -411,6 +417,14 @@ def _matches_current(return False+def _extension_browser_is_connected(browser: SkyvernBrowser) -> bool:+ try:+ playwright_browser = browser.browser+ return playwright_browser is not None and playwright_browser.is_connected()+ except Exception:+ return False++async def resolve_browser(session_id: str | None = None,cdp_url: str | None = None,@@ -418,6 +432,8 @@ async def resolve_browser(create_session: bool = False,timeout: int | None = None,headless: bool = False,+ *,+ extension_runtime: BrowserExtensionRuntime | None = None,) -> tuple[SkyvernBrowser, BrowserContext]:"""Resolve browser from parameters or current session.@@ -428,13 +444,27 @@ async def resolve_browser(skyvern = get_skyvern()current = get_current_session()- if _stateless_http_mode and not (session_id or cdp_url or local or create_session):+ if _stateless_http_mode and not (session_id or cdp_url or extension_runtime or local or create_session):raise BrowserNotAvailableError()- if _matches_current(current, session_id=session_id, cdp_url=cdp_url, local=local):+ if _matches_current(+ current,+ session_id=session_id,+ cdp_url=cdp_url,+ extension_runtime=extension_runtime,+ local=local,+ ):if current.browser is None or current.context is None:raise RuntimeError("Expected active browser and context for matching session")- return current.browser, current.context+ if extension_runtime is None or _extension_browser_is_connected(current.browser):+ return current.browser, current.context+ try:+ await _close_session_state(current, close_via_active_client=False)+ except Exception:+ pass+ finally:+ set_current_session(SessionState())+ current = get_current_session()# Cloud sessions created by the MCP session tool intentionally do not open a# second CDP connection. Connect lazily when a browser tool is used; this@@ -483,6 +513,12 @@ async def resolve_browser(set_current_session(SessionState(browser=browser, context=ctx, api_key_hash=active_api_key_hash))return browser, ctx+ if extension_runtime is not None:+ browser = await skyvern.connect_to_browser_extension(extension_runtime)+ ctx = BrowserContext(mode="extension", can_access_localhost=True)+ set_current_session(SessionState(browser=browser, context=ctx, api_key_hash=active_api_key_hash))+ return browser, ctx+if cdp_url:browser = await skyvern.connect_to_browser_over_cdp(cdp_url)ctx = BrowserContext(mode="cdp", cdp_url=cdp_url)diff --git a/skyvern/cli/mcp_tools/browser.py b/skyvern/cli/mcp_tools/browser.pyindex 3150bbcf9..121eee3ac 100644--- a/skyvern/cli/mcp_tools/browser.py+++ b/skyvern/cli/mcp_tools/browser.py@@ -3,6 +3,7 @@ from __future__ import annotationsimport asyncioimport base64import json+import mathimport mimetypesimport osimport re@@ -23,6 +24,7 @@ from skyvern.cli.core.browser_ops import (_ALLOWED_EXECUTE_TOOLS,LOCALHOST_RECOVERY_HINT,MAX_EXECUTE_STEPS,+ TYPE_PASSWORD_REFUSAL_MESSAGE,CustomSelectClassifyError,CustomSelectMatchError,CustomSelectOpenError,@@ -31,6 +33,7 @@ from skyvern.cli.core.browser_ops import (ObserveFrameError,ToolStepError,do_act,+ do_click_at,do_execute,do_extract,do_find,@@ -41,6 +44,7 @@ from skyvern.cli.core.browser_ops import (do_observe,do_screenshot,do_select_option,+ do_type_at,parse_extract_schema,ref_map_from_elements,ref_to_selector,@@ -207,6 +211,9 @@ def _record_trajectory_entry(typed_text: str | None = None,value: str | None = None,key: str | None = None,+ x: float | None = None,+ y: float | None = None,+ sdk_equivalent: str | None = None,) -> None:try:if ctx.mode != "cloud_session" or not ctx.session_id:@@ -231,6 +238,9 @@ def _record_trajectory_entry("source_url": projected.source_url,"value": value,"key": key,+ "x": x,+ "y": y,+ "sdk_equivalent": sdk_equivalent,}if tool_name == "type_text":entry["typed_length"] = len(typed_text or "")@@ -284,6 +294,27 @@ def _blank_to_none(value: str | None) -> str | None:return value if value is None or value.strip() else None+def _coordinate_target(x: float | None, y: float | None, selector: str | None) -> str | None:+ if (x is None) != (y is None):+ raise GuardError(+ "x and y must be provided together",+ "Provide both viewport coordinates, or omit both and use selector or intent",+ )+ if x is None or y is None:+ return None+ if selector is not None:+ raise GuardError(+ "Coordinates x/y are mutually exclusive with selector",+ "Use either x and y coordinates or selector, not both",+ )+ if isinstance(x, bool) or isinstance(y, bool) or not math.isfinite(x) or not math.isfinite(y) or x < 0 or y < 0:+ raise GuardError(+ "Coordinates x and y must be finite numbers greater than or equal to 0",+ "Use non-negative viewport CSS pixel coordinates",+ )+ return f"coordinates ({x}, {y})"++def _add_timing_prefix(timing_ms: dict[str, int], elapsed_ms: int) -> dict[str, int]:return {name: elapsed_ms + duration for name, duration in timing_ms.items()}@@ -540,6 +571,20 @@ async def skyvern_click("Use standard CSS: 'button.class', 'a[href*=\"pdf\"]', '#id', ':nth-of-type()'."),] = None,+ x: Annotated[+ float | None,+ Field(+ description=f"{DIRECT_TARGET_DESCRIPTION} Viewport x-coordinate in CSS pixels from the left edge of "+ "the web content. Provide with y and without selector."+ ),+ ] = None,+ y: Annotated[+ float | None,+ Field(+ description=f"{DIRECT_TARGET_DESCRIPTION} Viewport y-coordinate in CSS pixels from the top edge of "+ "the web content. Provide with x and without selector."+ ),+ ] = None,selector_mode: _SelectorMode = "resilient",session_id: Annotated[str | None, Field(description="Browser session ID (pbs_...)")] = None,cdp_url: Annotated[str | None, Field(description="CDP WebSocket URL")] = None,@@ -551,7 +596,7 @@ async def skyvern_click(click_count: Annotated[int | None, Field(description="Number of clicks (2 for double-click)")] = None,intent: Annotated[str | None, Field(description=AI_FALLBACK_DESCRIPTION)] = None,) -> dict[str, Any]:- """Click an element using AI intent, CSS/XPath selector, or both.+ """Click using viewport coordinates, AI intent, CSS/XPath selector, or a selector with intent fallback.For text input use skyvern_type. For dropdowns use skyvern_select_option. For multiple actions prefer skyvern_act."""if button is not None and button not in ("left", "right", "middle"):@@ -563,17 +608,28 @@ async def skyvern_click(selector = _blank_to_none(selector)intent = _blank_to_none(intent)- ai_mode, err = _resolve_ai_mode(selector, intent)- if err:+ try:+ coordinate_target = _coordinate_target(x, y, selector)+ except GuardError as e:return make_result("skyvern_click",ok=False,- error=make_error(- ErrorCode.INVALID_INPUT,- "Must provide intent, selector, or both",- "Use intent='describe what to click' for AI-powered clicking, or selector='#css-selector' for precise targeting",- ),+ error=make_error(ErrorCode.INVALID_INPUT, str(e), e.hint),)+ if coordinate_target is None:+ ai_mode, err = _resolve_ai_mode(selector, intent)+ if err:+ return make_result(+ "skyvern_click",+ ok=False,+ error=make_error(+ ErrorCode.INVALID_INPUT,+ "Must provide intent, selector, or both",+ "Use intent='describe what to click' for AI-powered clicking, or selector='#css-selector' for precise targeting",+ ),+ )+ else:+ ai_mode = Nonetry:page, ctx = await get_page(session_id=session_id, cdp_url=cdp_url)@@ -598,25 +654,51 @@ async def skyvern_click(if click_count is not None:kwargs["click_count"] = click_count- if selector is not None and (deterministic or ai_mode is None or ai_mode == "fallback"):- native_option_selection = await select_native_option_if_targeted(page, selector, timeout=action_timeout)-- if native_option_selection is not None:- resolved = native_option_selection.select_selector- elif deterministic:- # selector_mode="direct": pin the selector, no overlay-dismiss or AI re-target, so a- # missed target fails fast and the agent re-derives it instead of AI scout-scrolling.- resolved = await page.click(selector=selector, mode="direct", **kwargs)- elif ai_mode is not None:- used_ai_path = True- resolved = await page.click(selector=selector, prompt=intent, ai=ai_mode, **kwargs) # type: ignore[arg-type]+ if coordinate_target is not None:+ assert x is not None and y is not None+ await do_click_at(+ page,+ x,+ y,+ button=button or "left",+ click_count=click_count if click_count is not None else 1,+ )+ resolved = coordinate_targetelse:- assert selector is not None- if isinstance(page, SkyvernPage):- kwargs["_skip_element_prep"] = skip_element_prep- resolved = await page.click(selector=selector, **kwargs)+ if selector is not None and (deterministic or ai_mode is None or ai_mode == "fallback"):+ native_option_selection = await select_native_option_if_targeted(+ page,+ selector,+ timeout=action_timeout,+ )+ if native_option_selection is not None:+ resolved = native_option_selection.select_selector+ elif deterministic:+ # selector_mode="direct": pin the selector, no overlay-dismiss or AI re-target, so a+ # missed target fails fast and the agent re-derives it instead of AI scout-scrolling.+ resolved = await page.click(selector=selector, mode="direct", **kwargs)+ elif ai_mode is not None:+ used_ai_path = True+ resolved = await page.click(selector=selector, prompt=intent, ai=ai_mode, **kwargs) # type: ignore[arg-type]+ else:+ assert selector is not None+ if isinstance(page, SkyvernPage):+ kwargs["_skip_element_prep"] = skip_element_prep+ resolved = await page.click(selector=selector, **kwargs)timer.mark("sdk")except PlaywrightTimeoutError as e:+ if coordinate_target is not None:+ return action_result(+ "skyvern_click",+ ok=False,+ browser_context=ctx,+ timing_ms=timer.timing_ms,+ error=make_error(+ ErrorCode.ACTION_FAILED,+ str(e),+ "Check that the coordinates are within the current viewport",+ ),+ )if direct_action and selector is not None:return await _direct_failure_result("skyvern_click", ctx, timer, page, selector, e, action_timeout)return action_result(@@ -647,7 +729,13 @@ async def skyvern_click(),)- data: dict[str, Any] = {"selector": selector, "intent": intent, "ai_mode": ai_mode}+ data: dict[str, Any] = {+ "selector": selector,+ "intent": intent,+ "ai_mode": ai_mode,+ }+ if coordinate_target is not None:+ data.update({"x": x, "y": y, "resolved_target": coordinate_target})if native_option_selection is not None:data["selected_option"] = {"select_selector": native_option_selection.select_selector,@@ -659,13 +747,20 @@ async def skyvern_click(data["selected_option"]["value"] = native_option_selection.valueif native_option_selection.label is not None:data["selected_option"]["label"] = native_option_selection.label- if resolved and resolved != selector:+ if coordinate_target is None and resolved and resolved != selector:data["resolved_selector"] = resolved# Build sdk_equivalent: prefer hybrid selector+prompt for production scripts.# resolved_selector already contains the "xpath=" prefix (e.g. "xpath=//button[@id='x']"),# so pass it directly as the selector positional arg.resolved_sel = resolved if resolved and resolved != selector else selector- if native_option_selection is not None:+ coordinate_sdk_equivalent: str | None = None+ if coordinate_target is not None:+ coordinate_sdk_equivalent = (+ f"await page.mouse.click({x!r}, {y!r}, button={(button or 'left')!r}, "+ f"click_count={click_count if click_count is not None else 1})"+ )+ data["sdk_equivalent"] = coordinate_sdk_equivalent+ elif native_option_selection is not None:if native_option_selection.selected_by == "label":data["sdk_equivalent"] = (f"await page.select_option({native_option_selection.select_selector!r}, "@@ -699,14 +794,25 @@ async def skyvern_click(value=native_option_selection.value,)elif button in (None, "left") and click_count in (None, 1):- replayable_selector = resolved if used_ai_path else resolved or selector- if replayable_selector:+ if coordinate_target is not None:_record_trajectory_entry(ctx,tool_name="click",- selector=replayable_selector,+ selector=None,source_url=source_url,+ x=x,+ y=y,+ sdk_equivalent=coordinate_sdk_equivalent,)+ else:+ replayable_selector = resolved if used_ai_path else resolved or selector+ if replayable_selector:+ _record_trajectory_entry(+ ctx,+ tool_name="click",+ selector=replayable_selector,+ source_url=source_url,+ )return action_result("skyvern_click",browser_context=ctx,@@ -1117,6 +1223,20 @@ async def skyvern_type(selector: Annotated[str | None, Field(description=f"{DIRECT_TARGET_DESCRIPTION} CSS selector or XPath for the input element.")] = None,+ x: Annotated[+ float | None,+ Field(+ description=f"{DIRECT_TARGET_DESCRIPTION} Viewport x-coordinate in CSS pixels from the left edge of "+ "the web content. Provide with y and without selector."+ ),+ ] = None,+ y: Annotated[+ float | None,+ Field(+ description=f"{DIRECT_TARGET_DESCRIPTION} Viewport y-coordinate in CSS pixels from the top edge of "+ "the web content. Provide with x and without selector."+ ),+ ] = None,selector_mode: _SelectorMode = "resilient",session_id: Annotated[str | None, Field(description="Browser session ID (pbs_...)")] = None,cdp_url: Annotated[str | None, Field(description="CDP WebSocket URL")] = None,@@ -1127,43 +1247,65 @@ async def skyvern_type(clear: Annotated[bool, Field(description="Clear existing content before typing")] = True,delay: Annotated[int | None, Field(description="Delay between keystrokes in ms")] = None,intent: Annotated[str | None, Field(description=AI_FALLBACK_DESCRIPTION)] = None,+ clear_first: Annotated[+ bool | None,+ Field(description="Clear existing content before coordinate typing; defaults to clear"),+ ] = None,+ press_enter: Annotated[bool, Field(description="Press Enter after typing")] = False,) -> dict[str, Any]:- """Type text into an input field using AI intent, CSS/XPath selector, or both. Clears field by default (set clear=false to append).+ """Type using viewport coordinates, AI intent, CSS/XPath selector, or a selector with intent fallback.+ Clears field by default (set clear=false to append).NEVER use for passwords — use skyvern_login instead. For dropdowns use skyvern_select_option."""- # Block password entry — redirect to skyvern_login- target_text = f"{intent or ''} {selector or ''}"- if PASSWORD_PATTERN.search(target_text):+ selector = _blank_to_none(selector)+ intent = _blank_to_none(intent)+ try:+ coordinate_target = _coordinate_target(x, y, selector)+ except GuardError as e:return make_result("skyvern_type",ok=False,- error=make_error(- ErrorCode.INVALID_INPUT,- "Cannot type into password fields — credentials must not be passed through tool calls",- CREDENTIAL_HINT,- ),+ error=make_error(ErrorCode.INVALID_INPUT, str(e), e.hint),)- selector = _blank_to_none(selector)- intent = _blank_to_none(intent)- ai_mode, err = _resolve_ai_mode(selector, intent)- if err:+ target_text = f"{intent or ''} {selector or ''}"+ if PASSWORD_PATTERN.search(target_text):return make_result("skyvern_type",ok=False,error=make_error(ErrorCode.INVALID_INPUT,- "Must provide intent, selector, or both",- "Use intent='describe the input field' for AI-powered targeting, or selector='#css-selector' for precise targeting",+ TYPE_PASSWORD_REFUSAL_MESSAGE,+ CREDENTIAL_HINT,),)+ if coordinate_target is None:+ ai_mode, err = _resolve_ai_mode(selector, intent)+ if err:+ return make_result(+ "skyvern_type",+ ok=False,+ error=make_error(+ ErrorCode.INVALID_INPUT,+ "Must provide intent, selector, or both",+ "Use intent='describe the input field' for AI-powered targeting, or selector='#css-selector' for precise targeting",+ ),+ )+ else:+ ai_mode = None+try:page, ctx = await get_page(session_id=session_id, cdp_url=cdp_url)except BrowserNotAvailableError:return make_result("skyvern_type", ok=False, error=no_browser_error())- action_result = _action_result_factory(ctx=ctx, page=page, selector=selector, typed_text=text)+ action_result = _action_result_factory(+ ctx=ctx,+ page=page,+ selector=selector,+ typed_text=text,+ )source_url = _trajectory_source_url(page)# DOM-level guard: check if the target element is a password field@@ -1184,11 +1326,12 @@ async def skyvern_type(ok=False,error=make_error(ErrorCode.INVALID_INPUT,- "Cannot type into password fields — credentials must not be passed through tool calls",+ TYPE_PASSWORD_REFUSAL_MESSAGE,CREDENTIAL_HINT,),)+ clear_content = clear if clear_first is None else clear_firstdeterministic = selector is not None and selector_mode == "direct"direct_action = is_direct_action(selector, ai_mode, deterministic=deterministic)action_timeout = resolve_action_timeout_ms(timeout, direct_action=direct_action)@@ -1199,7 +1342,17 @@ async def skyvern_type(# selector_mode="direct" pins the selector with no AI fall-back. Resilient (default) and# intent-only calls keep AI; emitted scripts keep the selector+prompt fallback via# sdk_equivalent for DOM-drift resilience.- if clear:+ if coordinate_target is not None:+ assert x is not None and y is not None+ await do_type_at(+ page,+ x,+ y,+ text,+ clear_first=clear_content,+ press_enter=press_enter,+ )+ elif clear_content:if deterministic:assert selector is not Noneawait page.fill(selector, text, mode="direct", timeout=action_timeout)@@ -1225,8 +1378,31 @@ async def skyvern_type(if isinstance(page, SkyvernPage):kwargs["_skip_element_prep"] = skip_element_prepawait page.type(selector, text, **kwargs)+ if coordinate_target is None and press_enter:+ raw_page = page.page if hasattr(page, "page") else page+ await raw_page.keyboard.press("Enter")timer.mark("sdk")+ except GuardError as e:+ return action_result(+ "skyvern_type",+ ok=False,+ browser_context=ctx,+ timing_ms=timer.timing_ms,+ error=make_error(ErrorCode.INVALID_INPUT, str(e), e.hint),+ )except PlaywrightTimeoutError as e:+ if coordinate_target is not None:+ return action_result(+ "skyvern_type",+ ok=False,+ browser_context=ctx,+ timing_ms=timer.timing_ms,+ error=make_error(+ ErrorCode.ACTION_FAILED,+ str(e),+ "Check that the coordinates are within the current viewport",+ ),+ )if direct_action and selector is not None:return await _direct_failure_result("skyvern_type", ctx, timer, page, selector, e, action_timeout, typed_text=text@@ -1265,21 +1441,48 @@ async def skyvern_type(# Unlike click(), we cannot return resolved_selector here. SKY-7905 will# update the SDK to return element metadata from all action methods.data: dict[str, Any] = {"selector": selector, "intent": intent, "ai_mode": ai_mode, "text_length": len(text)}+ if coordinate_target is not None:+ data.update({"x": x, "y": y, "resolved_target": coordinate_target})# Build sdk_equivalent: prefer hybrid selector+prompt for production scripts- if selector and intent:+ coordinate_sdk_equivalent: str | None = None+ if coordinate_target is not None:+ clear_snippet = (+ "await page.keyboard.press('ControlOrMeta+A'); await page.keyboard.press('Backspace'); "+ if clear_content+ else ""+ )+ enter_snippet = "; await page.keyboard.press('Enter')" if press_enter else ""+ coordinate_sdk_equivalent = (+ f"await page.mouse.click({x!r}, {y!r}); {clear_snippet}await page.keyboard.type({text!r}){enter_snippet}"+ )+ data["sdk_equivalent"] = coordinate_sdk_equivalent+ elif selector and intent:data["sdk_equivalent"] = f"await page.fill({selector!r}, {text!r}, prompt={intent!r})"elif ai_mode:data["sdk_equivalent"] = f"await page.fill(prompt={intent!r}, value={text!r})"elif selector:data["sdk_equivalent"] = f"await page.fill({selector!r}, {text!r})"- if clear and selector is not None and (ai_mode is None or deterministic):- _record_trajectory_entry(- ctx,- tool_name="type_text",- selector=selector,- source_url=source_url,- typed_text=text,- )+ replayable_type = coordinate_target is not None or ai_mode is None or deterministic+ if clear_content and replayable_type:+ if coordinate_target is not None:+ _record_trajectory_entry(+ ctx,+ tool_name="type_text",+ selector=None,+ source_url=source_url,+ typed_text=text,+ x=x,+ y=y,+ sdk_equivalent=coordinate_sdk_equivalent,+ )+ elif selector is not None:+ _record_trajectory_entry(+ ctx,+ tool_name="type_text",+ selector=selector,+ source_url=source_url,+ typed_text=text,+ )return action_result("skyvern_type",browser_context=ctx,@@ -3323,8 +3526,8 @@ _TOOL_NAME_MAP: dict[str, str] = {# Accepted user-facing params for each dispatched tool (excludes session_id/cdp_url)._TOOL_ACCEPTED_PARAMS: dict[str, frozenset[str]] = {"navigate": frozenset({"url", "timeout", "wait_until"}),- "click": frozenset({"intent", "selector", "timeout", "click_count", "button"}),- "type": frozenset({"text", "intent", "selector", "clear_first", "press_enter", "timeout"}),+ "click": frozenset({"intent", "selector", "x", "y", "timeout", "click_count", "button"}),+ "type": frozenset({"text", "intent", "selector", "x", "y", "clear_first", "press_enter", "timeout"}),"press_key": frozenset({"key", "intent", "selector", "timeout"}),"select_option": frozenset({"value", "intent", "selector", "timeout", "by_label"}),"hover": frozenset({"intent", "selector", "timeout"}),diff --git a/skyvern/cli/mcp_tools/session.py b/skyvern/cli/mcp_tools/session.pyindex 087c3766c..69ec41af0 100644--- a/skyvern/cli/mcp_tools/session.py+++ b/skyvern/cli/mcp_tools/session.py@@ -5,6 +5,7 @@ from typing import Annotated, Anyfrom pydantic import Field+from skyvern.browser_extension.runtime import BrowserExtensionRuntimefrom skyvern.cli.core.action_log import drain_action_log_eventsfrom skyvern.cli.core.api_key_hash import hash_api_key_for_cachefrom skyvern.cli.core.client import get_active_api_key@@ -32,6 +33,12 @@ from ._session import (set_current_session,)+_EXTENSION_NOT_CONNECTED_GUIDANCE = (+ "Skyvern browser extension is not connected. Install it: run `skyvern browser extension-path`, open "+ "chrome://extensions, enable Developer mode, Load unpacked, and select that directory. Then run "+ "`skyvern browser extension-token`, paste the token into the extension popup, and click Connect. Then retry."+)+def _session_api_key_hash() -> str | None:api_key = get_active_api_key()@@ -49,6 +56,10 @@ def _should_default_to_cdp() -> tuple[bool, str | None]:return False, None+def _should_default_to_extension() -> bool:+ return os.environ.get("BROWSER_TYPE", "") == "extension-connect"++def _session_create_data(session_id: str | None,timeout_minutes: int | None,@@ -114,10 +125,74 @@ async def skyvern_browser_session_create(Use local=true for a local Chromium instance.The session persists across tool calls until explicitly closed.+ When the server runs in browser-extension mode, the session is implicit: no session_id is+ returned and browser tools are called without one."""- # When BROWSER_TYPE=cdp-connect, auto-connect to the user's local browser via CDP.- # resolve_browser() stores the browser in session state via set_current_session()- # internally, so we don't need to call it again here.+ if _should_default_to_extension() and not local:+ with Timer() as timer:+ if is_stateless_http_mode():+ return make_result(+ "skyvern_browser_session_create",+ ok=False,+ timing_ms=timer.timing_ms,+ error=make_error(+ ErrorCode.SDK_ERROR,+ "The Skyvern browser extension requires the MCP server to run on the stdio transport. "+ "Restart with: skyvern mcp --browser-extension",+ "",+ ),+ )+ try:+ runtime = await BrowserExtensionRuntime.get_or_start()+ if not await runtime.wait_for_extension(10.0):+ timer.mark("sdk")+ return make_result(+ "skyvern_browser_session_create",+ ok=False,+ timing_ms=timer.timing_ms,+ error=make_error(+ ErrorCode.BROWSER_NOT_FOUND,+ _EXTENSION_NOT_CONNECTED_GUIDANCE,+ "",+ ),+ )+ except Exception as e:+ return make_result(+ "skyvern_browser_session_create",+ ok=False,+ timing_ms=timer.timing_ms,+ error=make_error(+ ErrorCode.SDK_ERROR,+ str(e),+ "Failed to start or connect to the Skyvern browser extension",+ ),+ )++ try:+ _browser, ctx = await resolve_browser(extension_runtime=runtime)+ timer.mark("sdk")+ except Exception:+ return make_result(+ "skyvern_browser_session_create",+ ok=False,+ timing_ms=timer.timing_ms,+ error=make_error(+ ErrorCode.BROWSER_NOT_FOUND,+ _EXTENSION_NOT_CONNECTED_GUIDANCE,+ "",+ ),+ )+ return make_result(+ "skyvern_browser_session_create",+ browser_context=ctx,+ data={+ "browser": "extension",+ "session": "implicit",+ "note": "Extension sessions are implicit; call browser tools without a session_id.",+ },+ timing_ms=timer.timing_ms,+ )+use_cdp, cdp_url = _should_default_to_cdp()if use_cdp and not local and cdp_url:if browser_profile_id is not None or generate_browser_profile:diff --git a/skyvern/cli/run_commands.py b/skyvern/cli/run_commands.pyindex 4a601dd92..a331b2bad 100644--- a/skyvern/cli/run_commands.py+++ b/skyvern/cli/run_commands.py@@ -227,16 +227,29 @@ def _handle_mcp_shutdown_signal(_signum: int, _frame: Any) -> None:_mcp_main_task.cancel()-async def _run_mcp_with_cleanup(run_async: Any, **kwargs: Any) -> None:+async def _run_mcp_with_cleanup(run_async: Any, *, browser_extension: bool = False, **kwargs: Any) -> None:global _mcp_cleanup_done, _mcp_cleanup_in_progress, _mcp_main_taskcurrent_task = asyncio.current_task()_mcp_main_task = current_task+ extension_runtime: BrowserExtensionRuntime | None = Nonetry:+ if browser_extension:+ from skyvern.browser_extension.runtime import BrowserExtensionRuntime # noqa: PLC0415++ extension_runtime = await BrowserExtensionRuntime.get_or_start()await run_async(**kwargs)finally:_mcp_cleanup_in_progress = Truetry:- await _cleanup_mcp_resources()+ try:+ await _cleanup_mcp_resources()+ finally:+ if extension_runtime is None:+ from skyvern.browser_extension.runtime import BrowserExtensionRuntime # noqa: PLC0415++ extension_runtime = BrowserExtensionRuntime.instance()+ if extension_runtime is not None:+ await extension_runtime.shutdown()finally:_mcp_cleanup_done = True_mcp_cleanup_in_progress = False@@ -607,11 +620,22 @@ def run_mcp(help="Return full tool responses including sdk_equivalent, browser_context, and timing.",),] = False,+ browser_extension: Annotated[+ bool,+ typer.Option(+ "--browser-extension/--no-browser-extension",+ help="Start the relay for controlling Chrome through the Skyvern browser extension.",+ ),+ ] = False,) -> None:"""Run the MCP server with configurable transport for local or remote hosting."""global _mcp_eof_shutdown_requested, _mcp_shutdown_exit_code_mcp_eof_shutdown_requested = False_mcp_shutdown_exit_code = None+ if browser_extension and transport != "stdio":+ raise typer.BadParameter("--browser-extension requires --transport stdio", param_hint="--browser-extension")+ if browser_extension:+ os.environ.setdefault("BROWSER_TYPE", "extension-connect")prepare_cli_runtime(intent=EnvIntent.CLOUD)try:from skyvern.library.local_browser_profile import ( # noqa: PLC0415@@ -646,7 +670,13 @@ def run_mcp(original_signal_handlers[signal.SIGTERM] = signal.signal(signal.SIGTERM, _handle_mcp_shutdown_signal)eof_watcher_stop, shutdown_complete = _start_stdin_eof_watcher()try:- asyncio.run(_run_mcp_with_cleanup(mcp.run_async, transport="stdio"))+ asyncio.run(+ _run_mcp_with_cleanup(+ mcp.run_async,+ browser_extension=browser_extension,+ transport="stdio",+ )+ )except asyncio.CancelledError:if _mcp_shutdown_exit_code is None:raise@@ -660,6 +690,7 @@ def run_mcp(asyncio.run(_run_mcp_with_cleanup(mcp.run_async,+ browser_extension=browser_extension,transport=transport,host=host,port=port,diff --git a/skyvern/library/skyvern.py b/skyvern/library/skyvern.pyindex aa7a0fb1e..c3a6e8748 100644--- a/skyvern/library/skyvern.py+++ b/skyvern/library/skyvern.py@@ -26,6 +26,7 @@ _DEVTOOLS_ACTIVE_PORT_TIMEOUT_SECONDS = 2.0if TYPE_CHECKING:from playwright.async_api import Playwright+ from skyvern.browser_extension.runtime import BrowserExtensionRuntimefrom skyvern.library.skyvern_browser import SkyvernBrowserfrom skyvern.schemas.llm import LLMConfig, LLMRouterConfig@@ -589,6 +590,14 @@ class Skyvern(AsyncSkyvern):browser_context = browser.contexts[0] if browser.contexts else await browser.new_context()return SkyvernBrowser(self, browser_context, browser_address=cdp_url)+ async def connect_to_browser_extension(self, runtime: BrowserExtensionRuntime) -> SkyvernBrowser:+ from skyvern.library.skyvern_browser import SkyvernBrowser # noqa: PLC0415++ playwright = await self._get_playwright()+ browser = await playwright.chromium.connect_over_cdp(runtime.cdp_ws_url)+ browser_context = browser.contexts[0] if browser.contexts else await browser.new_context()+ return SkyvernBrowser(self, browser_context)+async def connect_to_cloud_browser_session(self, browser_session_id: str) -> SkyvernBrowser:"""Connect to an existing cloud-hosted browser session by ID.diff --git a/tests/smoke_tests/browser_extension/__init__.py b/tests/smoke_tests/browser_extension/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/tests/smoke_tests/browser_extension/run_extension_smoke.py b/tests/smoke_tests/browser_extension/run_extension_smoke.pynew file mode 100644index 000000000..b5d7940a1--- /dev/null+++ b/tests/smoke_tests/browser_extension/run_extension_smoke.py@@ -0,0 +1,508 @@+from __future__ import annotations++import argparse+import asyncio+import base64+import hashlib+import inspect+import json+import os+import re+import secrets+import socket+import tempfile+from collections.abc import Awaitable, Callable, Sequence+from dataclasses import dataclass+from pathlib import Path+from urllib.parse import urlsplit++from aiohttp import web+from playwright.async_api import Browser, BrowserContext, Page, Playwright, async_playwright++from skyvern.browser_extension.protocol import EXTENSION_ID+from skyvern.browser_extension.runtime import BrowserExtensionRuntime++EXPECTED_EXTENSION_ID = EXTENSION_ID+TOKEN_ENV = "SKYVERN_BROWSER_EXTENSION_TOKEN"+BROWSER_TYPE_ENV = "BROWSER_TYPE"+MCP_BROWSER_TYPE = "extension-connect"+SMOKE_TEXT = "Skyvern extension bridge smoke"+CHECKS = (+ "setup",+ "extension paired",+ "navigate",+ "screenshot",+ "trusted selector click",+ "coordinate click",+ "type",+ "iframe evaluate",+ "new tab",+ "close tab",+ "chrome survives disconnect",+ "mcp session/navigate/observe",+)+CDP_URL_PATTERN = re.compile(r"ws://127\.0\.0\.1:\d+/cdp/[A-Za-z0-9_-]+")+++class McpUnavailableError(RuntimeError):+ pass+++class SmokeReport:+ def __init__(self) -> None:+ self.passed = 0+ self.failed = 0+ self.skipped = 0+ self.recorded: set[str] = set()++ def pass_check(self, label: str, details: str | None = None) -> None:+ self.passed += 1+ self.recorded.add(label)+ print(_status_line("PASS", label, details))++ def fail_check(self, label: str, details: str | None = None) -> None:+ self.failed += 1+ self.recorded.add(label)+ print(_status_line("FAIL", label, details))++ def skip_check(self, label: str, details: str | None = None) -> None:+ self.skipped += 1+ self.recorded.add(label)+ print(_status_line("SKIP", label, details))++ def skip_unrecorded(self, reason: str) -> None:+ for label in CHECKS:+ if label not in self.recorded:+ self.skip_check(label, reason)++ def print_summary(self) -> None:+ print()+ print(f"SMOKE RESULT: {self.passed} passed, {self.failed} failed, {self.skipped} skipped")+++@dataclass+class FixtureServers:+ outer_runner: web.AppRunner+ iframe_runner: web.AppRunner+ outer_url: str++ async def close(self) -> None:+ try:+ await self.outer_runner.cleanup()+ finally:+ await self.iframe_runner.cleanup()+++def _status_line(status: str, label: str, details: str | None) -> str:+ suffix = f": {details}" if details else ""+ return f"{status} {label}{suffix}"+++def _reserve_free_port() -> tuple[socket.socket, int]:+ listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)+ try:+ listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)+ listener.bind(("127.0.0.1", 0))+ listener.listen(1)+ except BaseException:+ listener.close()+ raise+ return listener, int(listener.getsockname()[1])+++def _extension_id_from_key(manifest_key: str) -> str:+ public_key = base64.b64decode(manifest_key, validate=True)+ digest_prefix = hashlib.sha256(public_key).hexdigest()[:32]+ return "".join(chr(ord("a") + int(nibble, 16)) for nibble in digest_prefix)+++def _extension_id_from_manifest(extension_dir: Path) -> str:+ manifest_value = json.loads((extension_dir / "manifest.json").read_text())+ if not isinstance(manifest_value, dict):+ raise ValueError("extension manifest must contain a JSON object")+ manifest_key = manifest_value.get("key")+ if not isinstance(manifest_key, str) or not manifest_key:+ raise ValueError("extension manifest is missing its key")+ return _extension_id_from_key(manifest_key)+++def _safe_text(value: object, redactions: Sequence[str]) -> str:+ text = CDP_URL_PATTERN.sub("<redacted-cdp-url>", str(value))+ for secret_value in redactions:+ if secret_value:+ text = text.replace(secret_value, "<redacted>")+ return " ".join(text.split())[:600]+++async def _discover_extension_id(context: BrowserContext, timeout_seconds: float = 15.0) -> str:+ loop = asyncio.get_running_loop()+ deadline = loop.time() + timeout_seconds+ while loop.time() < deadline:+ parsed_urls = (urlsplit(worker.url) for worker in context.service_workers)+ extension_ids = {+ parsed.netloc for parsed in parsed_urls if parsed.scheme == "chrome-extension" and parsed.netloc+ }+ if EXPECTED_EXTENSION_ID in extension_ids:+ return EXPECTED_EXTENSION_ID+ if extension_ids:+ return sorted(extension_ids)[0]+ await asyncio.sleep(0.1)+ raise TimeoutError("no chrome-extension service worker appeared within 15 seconds")+++async def _wait_for_pairing(runtime: BrowserExtensionRuntime, timeout_seconds: float = 15.0) -> bool:+ loop = asyncio.get_running_loop()+ deadline = loop.time() + timeout_seconds+ while loop.time() < deadline:+ if runtime.extension_connected:+ return True+ await asyncio.sleep(0.1)+ return runtime.extension_connected+++async def _start_runner(+ handler: Callable[[web.Request], Awaitable[web.StreamResponse]],+) -> tuple[web.AppRunner, int]:+ app = web.Application()+ app.router.add_get("/", handler)+ runner = web.AppRunner(app)+ await runner.setup()+ site = web.TCPSite(runner, "127.0.0.1", 0)+ try:+ await site.start()+ except BaseException:+ await runner.cleanup()+ raise+ addresses = runner.addresses+ if not addresses:+ await runner.cleanup()+ raise RuntimeError("fixture server failed to bind")+ return runner, int(addresses[0][1])+++async def _start_fixture_servers() -> FixtureServers:+ async def iframe_handler(_request: web.Request) -> web.Response:+ return web.Response(+ text='<!doctype html><html><body><div id="frame-marker">iframe-ready</div></body></html>',+ content_type="text/html",+ headers={"Cache-Control": "no-store"},+ )++ iframe_runner, iframe_port = await _start_runner(iframe_handler)+ iframe_url = f"http://127.0.0.1:{iframe_port}/"++ async def outer_handler(_request: web.Request) -> web.Response:+ html = f"""<!doctype html><html><head><meta charset="utf-8">+<title>Extension Bridge Fixture</title></head><body>+<button id="smoke-button" type="button">Smoke button</button>+<input id="text-input" type="text"><input id="password-input" type="password">+<iframe id="smoke-frame" src="{iframe_url}"></iframe><script>+window.smokeState = {{clickCount: 0, lastTrusted: false}};+document.querySelector("#smoke-button").addEventListener("click", (event) => {{+ window.smokeState.clickCount += 1; window.smokeState.lastTrusted = event.isTrusted;+}});</script></body></html>"""+ return web.Response(text=html, content_type="text/html", headers={"Cache-Control": "no-store"})++ try:+ outer_runner, outer_port = await _start_runner(outer_handler)+ except BaseException:+ await iframe_runner.cleanup()+ raise+ return FixtureServers(+ outer_runner=outer_runner,+ iframe_runner=iframe_runner,+ outer_url=f"http://localhost:{outer_port}/",+ )+++def _tool_error(result: dict[str, object]) -> str:+ error = result.get("error")+ if not isinstance(error, dict):+ return "tool returned ok=false"+ code = error.get("code")+ message = error.get("message")+ return f"{code}: {message}"+++async def _run_mcp_phase(fixture_url: str) -> str:+ from skyvern.cli.mcp_tools import browser as browser_tools+ from skyvern.cli.mcp_tools import session as session_tools++ if MCP_BROWSER_TYPE not in inspect.getsource(session_tools):+ raise McpUnavailableError("the current MCP session layer has no extension-connect branch")++ previous_browser_type = os.environ.get(BROWSER_TYPE_ENV)+ os.environ[BROWSER_TYPE_ENV] = MCP_BROWSER_TYPE+ session_created = False+ try:+ session_result = await session_tools.skyvern_browser_session_create()+ if session_result.get("ok") is not True:+ raise McpUnavailableError(f"session=FAIL; unavailable in-process ({_tool_error(session_result)})")+ session_created = True++ navigate_result = await browser_tools.skyvern_navigate(url=fixture_url)+ observe_result = await browser_tools.skyvern_observe(selector="body", interactive_only=True)+ statuses = (+ f"session={'PASS' if session_result.get('ok') is True else 'FAIL'} "+ f"navigate={'PASS' if navigate_result.get('ok') is True else 'FAIL'} "+ f"observe={'PASS' if observe_result.get('ok') is True else 'FAIL'}"+ )+ if navigate_result.get("ok") is not True:+ raise AssertionError(f"{statuses}; navigate error={_tool_error(navigate_result)}")+ if observe_result.get("ok") is not True:+ raise AssertionError(f"{statuses}; observe error={_tool_error(observe_result)}")+ return statuses+ finally:+ if session_created:+ try:+ await session_tools.skyvern_browser_session_close()+ except Exception:+ pass+ if previous_browser_type is None:+ os.environ.pop(BROWSER_TYPE_ENV, None)+ else:+ os.environ[BROWSER_TYPE_ENV] = previous_browser_type+++async def _pair_extension(+ context: BrowserContext,+ runtime: BrowserExtensionRuntime,+ extension_id: str,+ relay_port: int,+ pairing_token: str,+) -> Page:+ popup_page = await context.new_page()+ await popup_page.goto(f"chrome-extension://{extension_id}/popup.html", wait_until="domcontentloaded")+ await popup_page.locator("#advanced-settings summary").click()+ await popup_page.locator("#bridge-port").fill(str(relay_port))+ await popup_page.locator("#pairing-token").fill(pairing_token)+ await popup_page.locator("#connection-button").click()+ if not await _wait_for_pairing(runtime):+ popup_error = await popup_page.locator("#connection-error").text_content()+ raise TimeoutError(f"extension did not pair within 15 seconds; popup error: {popup_error or 'none'}")+ return popup_page+++async def _exercise_page(page: Page, report: SmokeReport, screenshot_path: Path) -> FixtureServers:+ servers = await _start_fixture_servers()+ try:+ try:+ await page.goto("https://example.com", wait_until="domcontentloaded")+ title = await page.title()+ assert "Example" in title, f"unexpected example.com title: {title!r}"+ except Exception:+ await page.goto(servers.outer_url, wait_until="domcontentloaded")+ report.pass_check("navigate", "loopback fallback")+ else:+ report.pass_check("navigate", f"title={title!r}")++ await page.screenshot(path=screenshot_path)+ assert screenshot_path.is_file() and screenshot_path.stat().st_size > 0+ report.pass_check("screenshot", str(screenshot_path))++ await page.goto(servers.outer_url, wait_until="domcontentloaded")+ await page.click("#smoke-button")+ selector_state = await page.evaluate("() => ({...window.smokeState})")+ assert isinstance(selector_state, dict)+ assert selector_state.get("clickCount") == 1+ assert selector_state.get("lastTrusted") is True+ report.pass_check("trusted selector click")++ button = page.locator("#smoke-button")+ bounds = await button.bounding_box()+ assert bounds is not None, "button has no bounding box"+ x = bounds["x"] + bounds["width"] / 2+ y = bounds["y"] + bounds["height"] / 2+ await page.mouse.click(x, y)+ coordinate_state = await page.evaluate("() => ({...window.smokeState})")+ assert isinstance(coordinate_state, dict)+ assert coordinate_state.get("clickCount") == 2+ assert coordinate_state.get("lastTrusted") is True+ report.pass_check("coordinate click", f"x={x:.1f}, y={y:.1f}")++ text_input = page.locator("#text-input")+ await text_input.focus()+ await page.keyboard.type(SMOKE_TEXT)+ assert await text_input.input_value() == SMOKE_TEXT+ assert await page.locator("#password-input").count() == 1+ report.pass_check("type")++ iframe_element = await page.wait_for_selector("#smoke-frame")+ frame = await iframe_element.content_frame()+ assert frame is not None, "iframe did not expose a Playwright frame"+ await frame.wait_for_selector("#frame-marker")+ marker = await frame.evaluate("document.querySelector('#frame-marker')?.textContent")+ outer_origin = await page.evaluate("location.origin")+ iframe_origin = await frame.evaluate("location.origin")+ assert marker == "iframe-ready"+ assert iframe_origin != outer_origin+ report.pass_check("iframe evaluate", f"cross-origin frame={iframe_origin}")+ except BaseException:+ await servers.close()+ raise+ return servers+++async def _cleanup(+ bridge_browser: Browser | None,+ fixture_servers: FixtureServers | None,+ persistent_context: BrowserContext | None,+ runtime: BrowserExtensionRuntime | None,+ playwright: Playwright | None,+ profile_dir: tempfile.TemporaryDirectory[str] | None,+) -> list[str]:+ errors: list[str] = []+ cleanup_steps: list[tuple[str, Callable[[], Awaitable[object]]]] = []+ if bridge_browser is not None and bridge_browser.is_connected():+ cleanup_steps.append(("bridge browser", bridge_browser.close))+ if fixture_servers is not None:+ cleanup_steps.append(("fixture servers", fixture_servers.close))+ if persistent_context is not None:+ cleanup_steps.append(("persistent context", persistent_context.close))+ if runtime is not None:+ cleanup_steps.append(("extension runtime", runtime.shutdown))+ if playwright is not None:+ cleanup_steps.append(("playwright", playwright.stop))+ for label, cleanup_step in cleanup_steps:+ try:+ await cleanup_step()+ except Exception as exc:+ errors.append(f"{label}: {type(exc).__name__}: {exc}")+ if profile_dir is not None:+ try:+ profile_dir.cleanup()+ except Exception as exc:+ errors.append(f"temporary profile: {type(exc).__name__}: {exc}")+ return errors+++async def run_smoke() -> int:+ report = SmokeReport()+ pairing_token = secrets.token_urlsafe(32)+ previous_token = os.environ.get(TOKEN_ENV)+ os.environ[TOKEN_ENV] = pairing_token+ redactions = [pairing_token]+ runtime: BrowserExtensionRuntime | None = None+ playwright: Playwright | None = None+ persistent_context: BrowserContext | None = None+ bridge_browser: Browser | None = None+ fixture_servers: FixtureServers | None = None+ profile_dir: tempfile.TemporaryDirectory[str] | None = None+ screenshot_dir = Path(tempfile.mkdtemp(prefix="skyvern-extension-smoke-evidence-"))+ screenshot_path = (screenshot_dir / "smoke_navigate.png").resolve()++ try:+ port_reservation, relay_port = _reserve_free_port()+ port_reservation.close()+ runtime = await BrowserExtensionRuntime.get_or_start(port=relay_port)+ redactions.append(runtime.cdp_ws_url)+ extension_dir = BrowserExtensionRuntime.extension_dir().resolve()+ derived_extension_id = _extension_id_from_manifest(extension_dir)+ assert derived_extension_id == EXPECTED_EXTENSION_ID, (+ f"manifest key derived {derived_extension_id}, expected {EXPECTED_EXTENSION_ID}"+ )++ playwright = await async_playwright().start()+ profile_dir = tempfile.TemporaryDirectory(prefix="skyvern-extension-smoke-profile-")+ extension_path = str(extension_dir)+ persistent_context = await playwright.chromium.launch_persistent_context(+ profile_dir.name,+ channel="chromium",+ headless=False,+ args=[+ f"--disable-extensions-except={extension_path}",+ f"--load-extension={extension_path}",+ ],+ )+ discovered_extension_id = await _discover_extension_id(persistent_context)+ if discovered_extension_id != EXPECTED_EXTENSION_ID:+ print(f"WARN extension ID mismatch: expected {EXPECTED_EXTENSION_ID}, discovered {discovered_extension_id}")+ assert discovered_extension_id == EXPECTED_EXTENSION_ID+ extension_id = discovered_extension_id+ report.pass_check("setup", f"relay port={relay_port}, extension ID={extension_id}")++ await _pair_extension(persistent_context, runtime, extension_id, relay_port, pairing_token)+ assert runtime.extension_connected+ report.pass_check("extension paired")++ bridge_browser = await playwright.chromium.connect_over_cdp(runtime.cdp_ws_url)+ assert bridge_browser.contexts, "bridge connection exposed no browser context"+ bridge_context = bridge_browser.contexts[0]+ bridge_page = bridge_context.pages[0] if bridge_context.pages else await bridge_context.new_page()+ fixture_servers = await _exercise_page(bridge_page, report, screenshot_path)++ page_count = len(bridge_context.pages)+ new_page = await bridge_context.new_page()+ assert len(bridge_context.pages) == page_count + 1+ report.pass_check("new tab")+ await new_page.close()+ assert new_page.is_closed()+ assert len(bridge_context.pages) == page_count+ report.pass_check("close tab")++ await bridge_browser.close()+ assert not bridge_browser.is_connected()+ assert persistent_context.pages, "persistent Chrome context has no pages after bridge disconnect"+ surviving_page = next((page for page in persistent_context.pages if not page.is_closed()), None)+ assert surviving_page is not None+ assert await surviving_page.evaluate("1 + 1") == 2+ assert runtime.extension_connected+ report.pass_check("chrome survives disconnect")++ try:+ mcp_statuses = await _run_mcp_phase(fixture_servers.outer_url)+ except McpUnavailableError as exc:+ report.skip_check("mcp session/navigate/observe", _safe_text(exc, redactions))+ else:+ report.pass_check("mcp session/navigate/observe", mcp_statuses)+ except Exception as exc:+ failed_check = next((label for label in CHECKS if label not in report.recorded), "smoke")+ report.fail_check(failed_check, _safe_text(f"{type(exc).__name__}: {exc}", redactions))+ report.skip_unrecorded(f"blocked by {failed_check}")+ finally:+ cleanup_errors = await _cleanup(+ bridge_browser,+ fixture_servers,+ persistent_context,+ runtime,+ playwright,+ profile_dir,+ )+ if cleanup_errors:+ report.fail_check("cleanup", _safe_text("; ".join(cleanup_errors), redactions))+ if previous_token is None:+ os.environ.pop(TOKEN_ENV, None)+ else:+ os.environ[TOKEN_ENV] = previous_token++ report.print_summary()+ return 1 if report.failed else 0+++def _dry_run_helpers() -> int:+ extension_dir = BrowserExtensionRuntime.extension_dir().resolve()+ derived_extension_id = _extension_id_from_manifest(extension_dir)+ if derived_extension_id != EXPECTED_EXTENSION_ID:+ print(f"HELPER FAIL extension ID: expected {EXPECTED_EXTENSION_ID}, derived {derived_extension_id}")+ return 1+ print(f"HELPER PASS extension ID: {derived_extension_id}")+ port_reservation, free_port = _reserve_free_port()+ port_reservation.close()+ if not 0 < free_port <= 65535:+ print(f"HELPER FAIL free port: {free_port}")+ return 1+ print(f"HELPER PASS free port: {free_port}")+ return 0+++def main(argv: Sequence[str] | None = None) -> int:+ parser = argparse.ArgumentParser(description="Run the real Chrome extension bridge smoke test")+ parser.add_argument("--dry-run-helpers", action="store_true", help="validate pure-Python helpers only")+ args = parser.parse_args(argv)+ if args.dry_run_helpers:+ return _dry_run_helpers()+ return asyncio.run(run_smoke())+++if __name__ == "__main__":+ raise SystemExit(main())diff --git a/tests/unit/browser_extension/__init__.py b/tests/unit/browser_extension/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/tests/unit/browser_extension/test_auth.py b/tests/unit/browser_extension/test_auth.pynew file mode 100644index 000000000..4a90265bc--- /dev/null+++ b/tests/unit/browser_extension/test_auth.py@@ -0,0 +1,168 @@+from __future__ import annotations++import base64+import json+import os+import stat+from pathlib import Path++import pytest++import skyvern.browser_extension.auth as auth_module+from skyvern.browser_extension.auth import (+ build_challenge,+ compute_ext_proof,+ compute_server_proof,+ load_or_create_pairing_token,+ verify_ext_proof,+)+from skyvern.browser_extension.errors import BrowserExtensionError+++def test_extension_proof_round_trip_and_tampering() -> None:+ token = "pairing-token"+ server_nonce = "server-nonce"+ client_nonce = "client-nonce"+ proof = compute_ext_proof(token, server_nonce, client_nonce)++ assert verify_ext_proof(token, server_nonce, client_nonce, proof)+ assert not verify_ext_proof(token, "tampered-server", client_nonce, proof)+ assert not verify_ext_proof(token, server_nonce, "tampered-client", proof)+ assert not verify_ext_proof(token, server_nonce, client_nonce, f"{proof}x")+ assert not verify_ext_proof(token, server_nonce, client_nonce, "not-ascii-\N{SNOWMAN}")+++def test_server_proof_uses_distinct_domain_separator() -> None:+ token = "pairing-token"+ server_nonce = "server-nonce"+ client_nonce = "client-nonce"++ assert compute_server_proof(token, client_nonce, server_nonce) != compute_ext_proof(+ token, server_nonce, client_nonce+ )+++def test_build_challenge_contains_a_32_byte_b64url_nonce() -> None:+ server_nonce, raw_challenge = build_challenge()+ padding = "=" * (-len(server_nonce) % 4)++ assert len(base64.urlsafe_b64decode(server_nonce + padding)) == 32+ assert "=" not in server_nonce+ assert json.loads(raw_challenge) == {"v": 1, "type": "auth.challenge", "serverNonce": server_nonce}+++def test_environment_token_wins_over_existing_file(tmp_path: Path, monkeypatch) -> None:+ token_dir = tmp_path / ".skyvern"+ token_dir.mkdir()+ (token_dir / "browser_extension_token").write_text("file-token")+ monkeypatch.setattr(Path, "home", classmethod(lambda cls: tmp_path))+ monkeypatch.setenv("SKYVERN_BROWSER_EXTENSION_TOKEN", " environment-token\n")++ assert load_or_create_pairing_token() == "environment-token"+++def test_token_file_is_created_securely_and_reused(tmp_path: Path, monkeypatch) -> None:+ monkeypatch.setattr(Path, "home", classmethod(lambda cls: tmp_path))+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_TOKEN", raising=False)++ first_token = load_or_create_pairing_token()+ token_dir = tmp_path / ".skyvern"+ token_file = token_dir / "browser_extension_token"++ assert token_file.read_text() == first_token+ assert stat.S_IMODE(token_dir.stat().st_mode) == 0o700+ assert stat.S_IMODE(token_file.stat().st_mode) == 0o600++ monkeypatch.setattr("skyvern.browser_extension.auth.secrets.token_urlsafe", lambda _: "different-token")+ assert load_or_create_pairing_token() == first_token+++def test_existing_token_file_is_trimmed_and_hardened(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:+ token_dir = tmp_path / ".skyvern"+ token_dir.mkdir()+ token_file = token_dir / "browser_extension_token"+ token_file.write_text(" file-token\n")+ token_file.chmod(0o644)+ monkeypatch.setattr(Path, "home", classmethod(lambda cls: tmp_path))+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_TOKEN", raising=False)++ assert load_or_create_pairing_token() == "file-token"+ assert stat.S_IMODE(token_file.stat().st_mode) == 0o600+++def test_empty_environment_and_file_tokens_are_regenerated(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:+ token_dir = tmp_path / ".skyvern"+ token_dir.mkdir()+ token_file = token_dir / "browser_extension_token"+ token_file.write_text(" \n")+ monkeypatch.setattr(Path, "home", classmethod(lambda cls: tmp_path))+ monkeypatch.setenv("SKYVERN_BROWSER_EXTENSION_TOKEN", " \n")+ monkeypatch.setattr("skyvern.browser_extension.auth.secrets.token_urlsafe", lambda _: "regenerated-token")++ assert load_or_create_pairing_token() == "regenerated-token"+ assert token_file.read_text() == "regenerated-token"+++def test_token_path_must_be_a_regular_file(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:+ token_dir = tmp_path / ".skyvern"+ token_dir.mkdir()+ (token_dir / "browser_extension_token").mkdir()+ monkeypatch.setattr(Path, "home", classmethod(lambda cls: tmp_path))+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_TOKEN", raising=False)++ with pytest.raises(BrowserExtensionError, match="regular file"):+ load_or_create_pairing_token()+++def test_token_file_must_be_owned_by_current_user(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:+ token_dir = tmp_path / ".skyvern"+ token_dir.mkdir()+ (token_dir / "browser_extension_token").write_text("file-token")+ monkeypatch.setattr(Path, "home", classmethod(lambda cls: tmp_path))+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_TOKEN", raising=False)+ actual_user_id = os.getuid()+ monkeypatch.setattr("skyvern.browser_extension.auth.os.getuid", lambda: actual_user_id + 1)++ with pytest.raises(BrowserExtensionError, match="owned"):+ load_or_create_pairing_token()+++def test_publish_token_writes_secure_file_and_returns_new_token(tmp_path: Path) -> None:+ token_path = tmp_path / "browser_extension_token"++ assert auth_module._publish_token(token_path, "new-token") == "new-token"+ assert token_path.read_text() == "new-token"+ assert stat.S_IMODE(token_path.stat().st_mode) == 0o600+++def test_publish_token_collision_adopts_existing_token(tmp_path: Path) -> None:+ token_path = tmp_path / "browser_extension_token"+ token_path.write_text("winning-token")+ token_path.chmod(0o600)++ assert auth_module._publish_token(token_path, "losing-token") == "winning-token"+ assert token_path.read_text() == "winning-token"+++def test_publish_token_replaces_corrupt_collision_and_retries_link_once(+ tmp_path: Path,+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ token_path = tmp_path / "browser_extension_token"+ real_link = os.link+ link_calls = 0++ def collide_once(source: str | Path, destination: str | Path) -> None:+ nonlocal link_calls+ link_calls += 1+ if link_calls == 1:+ token_path.write_text("\n")+ token_path.chmod(0o600)+ raise FileExistsError+ real_link(source, destination)++ monkeypatch.setattr(auth_module.os, "link", collide_once)++ assert auth_module._publish_token(token_path, "new-token") == "new-token"+ assert token_path.read_text() == "new-token"+ assert link_calls == 2diff --git a/tests/unit/browser_extension/test_cdp_adapter.py b/tests/unit/browser_extension/test_cdp_adapter.pynew file mode 100644index 000000000..e2e0bd12d--- /dev/null+++ b/tests/unit/browser_extension/test_cdp_adapter.py@@ -0,0 +1,642 @@+from __future__ import annotations++import asyncio+from collections.abc import AsyncIterator+from unittest.mock import patch++import pytest+import pytest_asyncio+from aiohttp import ClientSession, ClientWebSocketResponse, WSMsgType++from skyvern.browser_extension.cdp_adapter import ExtensionCdpAdapter+from skyvern.browser_extension.errors import ExtensionRequestError+from skyvern.browser_extension.target_registry import VirtualTargetRegistry+++class StubRelay:+ def __init__(self, scoped_tabs: list[dict] | None = None) -> None:+ self.scoped_tabs = scoped_tabs or []+ self.calls: list[tuple[str, dict]] = []+ self.next_tab_id = 100+ self.fail_next: ExtensionRequestError | None = None+ self.fail_attach_tab_ids: set[int] = set()+ self.block_attach_tab_id: int | None = None+ self.attach_started = asyncio.Event()+ self.release_attach = asyncio.Event()++ async def request(self, op: str, args: dict, timeout: float = 30.0) -> dict:+ self.calls.append((op, args))+ if self.fail_next is not None:+ error = self.fail_next+ self.fail_next = None+ raise error+ if op == "debugger.attach":+ tab_id = args["tabId"]+ if tab_id in self.fail_attach_tab_ids:+ raise ExtensionRequestError("CDP_ERROR", "attach failed")+ if tab_id == self.block_attach_tab_id:+ self.attach_started.set()+ await self.release_attach.wait()+ if op == "tabs.create":+ tab_id = self.next_tab_id+ self.next_tab_id += 1+ return {"tabId": tab_id}+ if op == "debugger.send":+ return {"result": {"forwardedMethod": args["method"]}}+ return {}+++@pytest_asyncio.fixture+async def adapter_server() -> AsyncIterator[tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry]]:+ relay = StubRelay()+ registry = VirtualTargetRegistry()+ adapter = ExtensionCdpAdapter(registry, relay)+ await adapter.start()+ try:+ yield adapter, relay, registry+ finally:+ await adapter.stop()+++async def receive_response(ws: ClientWebSocketResponse, request_id: int) -> dict:+ while True:+ message = await ws.receive_json(timeout=2)+ if message.get("id") == request_id:+ return message+++async def receive_event(ws: ClientWebSocketResponse, method: str) -> dict:+ while True:+ message = await ws.receive_json(timeout=2)+ if message.get("method") == method:+ return message+++def assert_subset(actual: dict, expected: dict) -> None:+ for key, value in expected.items():+ if isinstance(value, dict):+ assert_subset(actual[key], value)+ else:+ assert actual[key] == value+++# Derived from chromium/crBrowser.js, chromium/crConnection.js, and chromium/chromium.js.+ROOT_CONNECT_CONTRACT = [+ (+ "Browser.getVersion",+ {"product": "Chrome/999.0.0.0", "userAgent": "Skyvern-Extension-Bridge"},+ ),+ ("Target.setAutoAttach", {}),+ ("Browser.setDownloadBehavior", {}),+ ("Target.getTargetInfo", {"targetInfo": {"targetId": "skyvern-browser", "type": "browser"}}),+]++ROOT_CONNECT_PARAMS = {+ "Browser.getVersion": {},+ "Target.setAutoAttach": {"autoAttach": True, "waitForDebuggerOnStart": True, "flatten": True},+ "Browser.setDownloadBehavior": {"behavior": "allowAndName", "eventsEnabled": True},+ "Target.getTargetInfo": {},+}+++async def test_capability_path_is_required(+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],+) -> None:+ adapter, _, _ = adapter_server+ wrong_url = adapter.cdp_ws_url.rsplit("/", 1)[0] + "/wrong"++ async with ClientSession() as client:+ response = await client.get(wrong_url.replace("ws://", "http://"))+ assert response.status == 404+ response.release()++ ws = await client.ws_connect(adapter.cdp_ws_url)+ assert not ws.closed+ await ws.close()+++async def test_root_connect_contract_fixture(+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],+) -> None:+ adapter, _, _ = adapter_server++ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ for request_id, (method, minimal_reply) in enumerate(ROOT_CONNECT_CONTRACT, start=1):+ await ws.send_json({"id": request_id, "method": method, "params": ROOT_CONNECT_PARAMS[method]})+ response = await receive_response(ws, request_id)+ assert "error" not in response+ assert_subset(response["result"], minimal_reply)+++async def test_auto_attach_creates_blank_tab_when_scope_is_empty(+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],+) -> None:+ adapter, relay, registry = adapter_server++ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ await ws.send_json({"id": 1, "method": "Target.setAutoAttach", "params": {"autoAttach": True}})+ assert await receive_response(ws, 1) == {"id": 1, "result": {}}+ attached = await receive_event(ws, "Target.attachedToTarget")+ root_session_id = registry.root_session_id(100)++ assert relay.calls[:2] == [+ ("tabs.create", {"url": "about:blank"}),+ ("debugger.attach", {"tabId": 100}),+ ]+ assert attached["params"] == {+ "sessionId": root_session_id,+ "targetInfo": {+ "targetId": "tab-100",+ "type": "page",+ "title": "",+ "url": "about:blank",+ "attached": True,+ "canAccessOpener": False,+ "browserContextId": "skyvern-default",+ },+ "waitingForDebugger": False,+ }+++async def test_auto_attach_existing_and_later_scoped_tabs() -> None:+ relay = StubRelay(+ [+ {"tabId": 7, "url": "https://one.example", "title": "One"},+ {"tabId": 8, "url": "https://two.example", "title": "Two"},+ ]+ )+ relay.fail_next = ExtensionRequestError("CDP_ERROR", "Another debugger is already attached")+ registry = VirtualTargetRegistry()+ adapter = ExtensionCdpAdapter(registry, relay)+ await adapter.start()+ try:+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ await ws.send_json({"id": 1, "method": "Target.setAutoAttach", "params": {"autoAttach": True}})+ await receive_response(ws, 1)+ events = [await receive_event(ws, "Target.attachedToTarget") for _ in range(2)]++ await adapter.handle_extension_event(+ "scope.tabAdded", {"tabId": 9, "url": "https://three.example", "title": "Three"}+ )+ later = await receive_event(ws, "Target.attachedToTarget")++ await adapter.handle_extension_event(+ "tabs.created", {"tabId": 10, "openerTabId": 7, "url": "https://popup.example"}+ )+ popup = await receive_event(ws, "Target.attachedToTarget")+ finally:+ await adapter.stop()++ assert [call for call in relay.calls if call[0] == "debugger.attach"] == [+ ("debugger.attach", {"tabId": 7}),+ ("debugger.attach", {"tabId": 8}),+ ("debugger.attach", {"tabId": 9}),+ ("debugger.attach", {"tabId": 10}),+ ]+ assert [event["params"]["targetInfo"]["targetId"] for event in events] == ["tab-7", "tab-8"]+ assert later["params"]["targetInfo"]["targetId"] == "tab-9"+ assert popup["params"]["targetInfo"]["openerId"] == "tab-7"+++async def test_auto_attach_post_reply_failure_skips_failed_tab_without_second_response() -> None:+ relay = StubRelay(+ [+ {"tabId": 30, "url": "https://bad.example", "title": "Bad"},+ {"tabId": 31, "url": "https://good.example", "title": "Good"},+ ]+ )+ relay.fail_attach_tab_ids.add(30)+ registry = VirtualTargetRegistry()+ adapter = ExtensionCdpAdapter(registry, relay)+ await adapter.start()+ try:+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ await ws.send_json({"id": 1, "method": "Target.setAutoAttach", "params": {"autoAttach": True}})++ assert await receive_response(ws, 1) == {"id": 1, "result": {}}+ attached = await receive_event(ws, "Target.attachedToTarget")+ assert attached["params"]["targetInfo"]["targetId"] == "tab-31"+ with pytest.raises(TimeoutError):+ await ws.receive_json(timeout=0.05)+ finally:+ await adapter.stop()+++async def test_tab_removal_tombstones_in_flight_background_attach() -> None:+ relay = StubRelay([{"tabId": 40, "url": "https://existing.example", "title": "Existing"}])+ registry = VirtualTargetRegistry()+ adapter = ExtensionCdpAdapter(registry, relay)+ await adapter.start()+ try:+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ await ws.send_json({"id": 1, "method": "Target.setAutoAttach", "params": {"autoAttach": True}})+ await receive_response(ws, 1)+ await receive_event(ws, "Target.attachedToTarget")++ relay.block_attach_tab_id = 41+ await adapter.handle_extension_event(+ "scope.tabAdded", {"tabId": 41, "url": "https://racy.example", "title": "Racy"}+ )+ await asyncio.wait_for(relay.attach_started.wait(), 1)+ removal_task = asyncio.create_task(+ adapter.handle_extension_event("scope.tabRemoved", {"tabId": 41, "reason": "unshared"})+ )+ await asyncio.sleep(0)+ relay.release_attach.set()+ await removal_task++ with pytest.raises(KeyError):+ registry.root_session_id(41)+ with pytest.raises(TimeoutError):+ await ws.receive_json(timeout=0.05)+ finally:+ await adapter.stop()+++async def test_session_command_routes_to_relay_and_preserves_session_id(+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],+) -> None:+ adapter, relay, registry = adapter_server+ registry.register_tab(42, "https://example.com", "Example")+ session_id = registry.root_session_id(42)++ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ await ws.send_json(+ {"id": 5, "sessionId": session_id, "method": "Runtime.evaluate", "params": {"expression": "1+1"}}+ )+ response = await receive_response(ws, 5)++ await ws.send_json({"id": 6, "sessionId": "missing", "method": "Runtime.enable", "params": {}})+ missing = await receive_response(ws, 6)++ assert relay.calls[-1] == (+ "debugger.send",+ {"tabId": 42, "method": "Runtime.evaluate", "params": {"expression": "1+1"}},+ )+ assert response == {"id": 5, "sessionId": session_id, "result": {"forwardedMethod": "Runtime.evaluate"}}+ assert missing == {+ "id": 6,+ "sessionId": "missing",+ "error": {"code": -32001, "message": "session not found"},+ }+++ ("method", "params"),+ [+ ("Network.getAllCookies", {}),+ ("Network.getCookies", {"urls": ["https://unshared.example"]}),+ ],+)+async def test_session_command_rejects_denied_cdp_methods(+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],+ method: str,+ params: dict,+) -> None:+ adapter, relay, registry = adapter_server+ registry.register_tab(42, "https://example.com", "Example")+ session_id = registry.root_session_id(42)++ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ await ws.send_json({"id": 1, "sessionId": session_id, "method": method, "params": params})+ response = await receive_response(ws, 1)++ assert response == {+ "id": 1,+ "sessionId": session_id,+ "error": {+ "code": -32000,+ "message": "CDP_METHOD_NOT_ALLOWED: The requested CDP method is not allowed.",+ },+ }+ assert relay.calls == []+++async def test_page_attach_aliases_are_unique_route_commands_and_receive_root_events() -> None:+ relay = StubRelay([{"tabId": 42, "url": "https://example.com", "title": "Example"}])+ registry = VirtualTargetRegistry()+ adapter = ExtensionCdpAdapter(registry, relay)+ await adapter.start()+ try:+ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ await ws.send_json({"id": 1, "method": "Target.setAutoAttach", "params": {"autoAttach": True}})+ await receive_response(ws, 1)+ primary_attached = await receive_event(ws, "Target.attachedToTarget")+ primary_session_id = primary_attached["params"]["sessionId"]++ for request_id in (2, 3):+ await ws.send_json(+ {+ "id": request_id,+ "method": "Target.attachToTarget",+ "params": {"targetId": "tab-42", "flatten": True},+ }+ )+ first_alias = (await receive_response(ws, 2))["result"]["sessionId"]+ second_alias = (await receive_response(ws, 3))["result"]["sessionId"]++ assert len({primary_session_id, first_alias, second_alias}) == 3+ await ws.send_json({"id": 4, "sessionId": first_alias, "method": "Runtime.enable", "params": {}})+ assert (await receive_response(ws, 4))["sessionId"] == first_alias++ event_params = {"name": "networkAlmostIdle", "frameId": "main"}+ await adapter.handle_extension_event(+ "debugger.event",+ {"tabId": 42, "method": "Page.lifecycleEvent", "params": event_params},+ )+ routed_events = [await receive_event(ws, "Page.lifecycleEvent") for _ in range(3)]+ assert {event["sessionId"] for event in routed_events} == {+ primary_session_id,+ first_alias,+ second_alias,+ }++ await ws.send_json({"id": 5, "method": "Target.detachFromTarget", "params": {"sessionId": first_alias}})+ assert await receive_response(ws, 5) == {"id": 5, "result": {}}+ assert all(op != "debugger.detach" for op, _ in relay.calls)+ with pytest.raises(KeyError):+ registry.resolve_session(first_alias)+ assert registry.resolve_session(second_alias) == (42, None)+ finally:+ await adapter.stop()+++async def test_browser_alias_routes_root_commands_and_detaches_independently(+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],+) -> None:+ adapter, _, registry = adapter_server++ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ await ws.send_json({"id": 1, "method": "Target.attachToBrowserTarget", "params": {}})+ first_alias = (await receive_response(ws, 1))["result"]["sessionId"]+ await ws.send_json({"id": 2, "method": "Target.attachToBrowserTarget", "params": {}})+ second_alias = (await receive_response(ws, 2))["result"]["sessionId"]++ assert first_alias != second_alias+ await ws.send_json({"id": 3, "sessionId": first_alias, "method": "Browser.getVersion", "params": {}})+ version = await receive_response(ws, 3)+ assert version["sessionId"] == first_alias+ assert version["result"]["protocolVersion"] == "1.3"++ await ws.send_json({"id": 4, "method": "Target.detachFromTarget", "params": {"sessionId": first_alias}})+ assert await receive_response(ws, 4) == {"id": 4, "result": {}}+ assert not registry.is_browser_session_alias(first_alias)+ assert registry.is_browser_session_alias(second_alias)++ await ws.send_json({"id": 5, "sessionId": first_alias, "method": "Browser.getVersion", "params": {}})+ assert (await receive_response(ws, 5))["error"]["code"] == -32001+++async def test_child_attach_recurses_and_detach_unregisters(+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],+) -> None:+ adapter, relay, registry = adapter_server+ registry.register_tab(12, "https://example.com", "Example")+ root_session_id = registry.root_session_id(12)++ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ await ws.send_json({"id": 1, "method": "Target.setAutoAttach", "params": {"autoAttach": True}})+ await receive_response(ws, 1)+ await receive_event(ws, "Target.attachedToTarget")+ relay.calls.clear()++ target_info = {+ "targetId": "frame-12",+ "type": "iframe",+ "title": "",+ "url": "https://example.com/frame",+ "attached": True,+ "canAccessOpener": False,+ "browserContextId": "skyvern-default",+ }+ await adapter.handle_extension_event(+ "debugger.event",+ {+ "tabId": 12,+ "method": "Target.attachedToTarget",+ "params": {"sessionId": "child-12", "targetInfo": target_info, "waitingForDebugger": False},+ },+ )+ attached = await receive_event(ws, "Target.attachedToTarget")++ await ws.send_json({"id": 2, "sessionId": "child-12", "method": "Runtime.enable", "params": {}})+ await receive_response(ws, 2)++ await adapter.handle_extension_event(+ "debugger.event",+ {"tabId": 12, "method": "Target.detachedFromTarget", "params": {"sessionId": "child-12"}},+ )+ detached = await receive_event(ws, "Target.detachedFromTarget")++ assert attached["sessionId"] == root_session_id+ assert relay.calls[0] == (+ "debugger.send",+ {+ "tabId": 12,+ "sessionId": "child-12",+ "method": "Target.setAutoAttach",+ "params": {"flatten": True, "autoAttach": True, "waitForDebuggerOnStart": False},+ },+ )+ assert relay.calls[1][1]["sessionId"] == "child-12"+ assert detached["params"] == {"sessionId": "child-12"}+ with pytest.raises(KeyError):+ registry.resolve_session("child-12")+++async def test_get_target_info_returns_registered_child_target(+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],+) -> None:+ adapter, _, registry = adapter_server+ registry.register_tab(13, "https://example.com", "Example")+ child_target_info = {+ "targetId": "frame-13",+ "type": "iframe",+ "title": "Frame",+ "url": "https://example.com/frame",+ "attached": True,+ "canAccessOpener": False,+ "browserContextId": "skyvern-default",+ }+ registry.register_child_session(13, "child-13", child_target_info)++ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ await ws.send_json({"id": 1, "method": "Target.getTargetInfo", "params": {"targetId": "frame-13"}})+ response = await receive_response(ws, 1)++ assert response == {"id": 1, "result": {"targetInfo": child_target_info}}+++async def test_create_and_close_target(+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],+) -> None:+ adapter, relay, _ = adapter_server++ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ await ws.send_json({"id": 1, "method": "Target.createTarget", "params": {"url": "https://example.com"}})+ attached = await receive_event(ws, "Target.attachedToTarget")+ created = await receive_response(ws, 1)++ await ws.send_json({"id": 2, "method": "Target.closeTarget", "params": {"targetId": "tab-100"}})+ closed = await receive_response(ws, 2)+ await adapter.handle_extension_event("scope.tabRemoved", {"tabId": 100, "reason": "closed"})+ detached = await receive_event(ws, "Target.detachedFromTarget")+ destroyed = await receive_event(ws, "Target.targetDestroyed")++ assert created == {"id": 1, "result": {"targetId": "tab-100"}}+ assert attached["params"]["targetInfo"]["url"] == "https://example.com"+ assert closed == {"id": 2, "result": {"success": True}}+ assert detached["params"]["sessionId"].startswith("sess-tab-100-")+ assert destroyed["params"] == {"targetId": "tab-100"}+ assert ("tabs.remove", {"tabId": 100}) in relay.calls+++async def test_create_target_and_scope_event_register_tab_once(+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],+) -> None:+ adapter, relay, registry = adapter_server+ original_request = relay.request++ async def request_with_scope_event(op: str, args: dict, timeout: float = 30.0) -> dict:+ result = await original_request(op, args, timeout)+ if op == "tabs.create":+ await adapter.handle_extension_event(+ "scope.tabAdded",+ {"tabId": result["tabId"], "url": args["url"], "title": ""},+ )+ return result++ relay.request = request_with_scope_event++ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ await ws.send_json({"id": 1, "method": "Target.setDiscoverTargets", "params": {"discover": True}})+ assert await receive_response(ws, 1) == {"id": 1, "result": {}}+ browser_created = await receive_event(ws, "Target.targetCreated")+ assert browser_created["params"]["targetInfo"]["targetId"] == "skyvern-browser"++ with patch.object(registry, "register_tab", wraps=registry.register_tab) as register_tab:+ await ws.send_json({"id": 2, "method": "Target.createTarget", "params": {"url": "https://example.com"}})+ target_created = await receive_event(ws, "Target.targetCreated")+ await receive_event(ws, "Target.attachedToTarget")+ created = await receive_response(ws, 2)++ assert register_tab.call_count == 1+ assert target_created["params"]["targetInfo"]["targetId"] == "tab-100"+ assert created == {"id": 2, "result": {"targetId": "tab-100"}}+ assert len(registry.list_page_targets()) == 1+ await adapter.handle_extension_event(+ "scope.tabAdded",+ {"tabId": 100, "url": "https://example.com", "title": ""},+ )+ with pytest.raises(TimeoutError):+ await ws.receive_json(timeout=0.05)+++async def test_debugger_detached_emits_destroyed_and_navigation_updates_target(+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],+) -> None:+ adapter, _, registry = adapter_server+ registry.register_tab(15, "https://old.example", "Old")+ session_id = registry.root_session_id(15)++ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ await adapter.handle_extension_event(+ "debugger.event",+ {+ "tabId": 15,+ "method": "Page.frameNavigated",+ "params": {"frame": {"id": "main", "url": "https://new.example"}},+ },+ )+ navigated = await receive_event(ws, "Page.frameNavigated")+ assert navigated["sessionId"] == session_id+ assert registry.target_info("tab-15")["url"] == "https://new.example"++ await adapter.handle_extension_event("debugger.detached", {"tabId": 15, "reason": "canceled_by_user"})+ detached = await receive_event(ws, "Target.detachedFromTarget")+ destroyed = await receive_event(ws, "Target.targetDestroyed")++ assert detached["params"] == {"sessionId": session_id, "targetId": "tab-15"}+ assert destroyed["params"] == {"targetId": "tab-15"}+ with pytest.raises(KeyError):+ registry.root_session_id(15)+++async def test_unknown_root_and_extension_request_errors(+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],+) -> None:+ adapter, relay, registry = adapter_server+ registry.register_tab(18, "https://example.com", "Example")+ session_id = registry.root_session_id(18)++ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ await ws.send_json({"id": 1, "method": "Nope.missing", "params": {}})+ unknown = await receive_response(ws, 1)++ relay.fail_next = ExtensionRequestError("CDP_ERROR", "command failed")+ await ws.send_json({"id": 2, "sessionId": session_id, "method": "Runtime.enable", "params": {}})+ failed = await receive_response(ws, 2)++ assert unknown == {"id": 1, "error": {"code": -32601, "message": "'Nope.missing' wasn't found"}}+ assert failed == {+ "id": 2,+ "sessionId": session_id,+ "error": {"code": -32000, "message": "CDP_ERROR: command failed"},+ }+++async def test_second_client_rejected_and_disconnect_allows_future_client(+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],+) -> None:+ adapter, _, registry = adapter_server+ registry.register_tab(20, "https://example.com", "Example")++ async with ClientSession() as client:+ first = await client.ws_connect(adapter.cdp_ws_url)+ second = await client.ws_connect(adapter.cdp_ws_url)+ await second.receive(timeout=2)+ assert second.close_code == 4409++ await adapter.on_extension_disconnect()+ await first.receive(timeout=2)+ assert first.close_code == 1001+ assert registry.list_page_targets() == []++ future = await client.ws_connect(adapter.cdp_ws_url)+ await future.send_json({"id": 1, "method": "Browser.getVersion", "params": {}})+ assert (await receive_response(future, 1))["result"]["protocolVersion"] == "1.3"+ await future.close()+++async def test_browser_close_replies_then_closes_without_removing_tabs(+ adapter_server: tuple[ExtensionCdpAdapter, StubRelay, VirtualTargetRegistry],+) -> None:+ adapter, relay, registry = adapter_server+ registry.register_tab(22, "https://example.com", "Example")++ async with ClientSession() as client, client.ws_connect(adapter.cdp_ws_url) as ws:+ await ws.send_json({"id": 9, "method": "Browser.close", "params": {}})+ assert await receive_response(ws, 9) == {"id": 9, "result": {}}+ close_message = await ws.receive(timeout=2)+ assert close_message.type in {WSMsgType.CLOSE, WSMsgType.CLOSED}++ assert ("debugger.detach", {"tabId": 22}) in relay.calls+ assert all(op != "tabs.remove" for op, _ in relay.calls)diff --git a/tests/unit/browser_extension/test_mcp_glue.py b/tests/unit/browser_extension/test_mcp_glue.pynew file mode 100644index 000000000..58d29b905--- /dev/null+++ b/tests/unit/browser_extension/test_mcp_glue.py@@ -0,0 +1,591 @@+from __future__ import annotations++import hashlib+import hmac+import json+import sys+import types+from pathlib import Path+from types import SimpleNamespace+from unittest.mock import AsyncMock, MagicMock, Mock, call++import pytest+import typer+from typer.testing import CliRunner++from skyvern.browser_extension.errors import BrowserExtensionError+from skyvern.browser_extension.runtime import BrowserExtensionRuntime+from skyvern.cli import run_commands+from skyvern.cli.commands import browser as browser_commands+from skyvern.cli.commands.browser import browser_app+from skyvern.cli.core import session_manager+from skyvern.cli.core.result import BrowserContext+from skyvern.cli.mcp_tools import session as mcp_session++_NOT_CONNECTED_GUIDANCE = (+ "Skyvern browser extension is not connected. Install it: run `skyvern browser extension-path`, open "+ "chrome://extensions, enable Developer mode, Load unpacked, and select that directory. Then run "+ "`skyvern browser extension-token`, paste the token into the extension popup, and click Connect. Then retry."+)++[email protected](autouse=True)+def _use_stdio_transport(monkeypatch: pytest.MonkeyPatch) -> None:+ monkeypatch.setattr(session_manager, "_stateless_http_mode", False)+++ ("browser_type", "expected"),+ [+ (None, False),+ ("", False),+ ("cdp-connect", False),+ ("extension-connect", True),+ ("Extension-Connect", False),+ ],+)+def test_should_default_to_extension_env_matrix(+ monkeypatch: pytest.MonkeyPatch,+ browser_type: str | None,+ expected: bool,+) -> None:+ if browser_type is None:+ monkeypatch.setenv("BROWSER_TYPE", "leak-guard-sentinel")+ monkeypatch.delenv("BROWSER_TYPE", raising=False)+ else:+ monkeypatch.setenv("BROWSER_TYPE", browser_type)++ assert mcp_session._should_default_to_extension() is expected+++async def test_explicit_local_session_does_not_start_extension_runtime(monkeypatch: pytest.MonkeyPatch) -> None:+ browser = MagicMock(app_url=None)+ do_session_create = AsyncMock(return_value=(browser, SimpleNamespace(local=True, headless=False)))+ get_or_start = AsyncMock()+ monkeypatch.setenv("BROWSER_TYPE", "extension-connect")+ monkeypatch.setattr(mcp_session, "get_skyvern", MagicMock())+ monkeypatch.setattr(mcp_session, "do_session_create", do_session_create)+ monkeypatch.setattr(mcp_session, "set_current_session", MagicMock())+ monkeypatch.setattr(mcp_session.BrowserExtensionRuntime, "get_or_start", get_or_start)++ result = await mcp_session.skyvern_browser_session_create(local=True)++ assert result["ok"] is True+ assert result["data"] == {"local": True, "headless": False}+ get_or_start.assert_not_awaited()+ do_session_create.assert_awaited_once()+++async def test_extension_session_rejects_stateless_http_mode(monkeypatch: pytest.MonkeyPatch) -> None:+ get_or_start = AsyncMock()+ monkeypatch.setenv("BROWSER_TYPE", "extension-connect")+ monkeypatch.setattr(mcp_session.BrowserExtensionRuntime, "get_or_start", get_or_start)+ session_manager.set_stateless_http_mode(True)+ try:+ result = await mcp_session.skyvern_browser_session_create()+ finally:+ session_manager.set_stateless_http_mode(False)++ assert result["ok"] is False+ assert result["error"]["code"] == mcp_session.ErrorCode.SDK_ERROR+ assert result["error"]["message"] == (+ "The Skyvern browser extension requires the MCP server to run on the stdio transport. "+ "Restart with: skyvern mcp --browser-extension"+ )+ get_or_start.assert_not_awaited()+++async def test_session_create_extension_takes_precedence_over_cdp(monkeypatch: pytest.MonkeyPatch) -> None:+ runtime = SimpleNamespace(wait_for_extension=AsyncMock(return_value=True))+ get_or_start = AsyncMock(return_value=runtime)+ resolve_browser = AsyncMock(return_value=(MagicMock(), BrowserContext(mode="extension")))+ monkeypatch.setattr(mcp_session, "_should_default_to_extension", lambda: True)+ monkeypatch.setattr(mcp_session, "_should_default_to_cdp", lambda: (True, "ws://cdp.example.test"))+ monkeypatch.setattr(mcp_session.BrowserExtensionRuntime, "get_or_start", get_or_start)+ monkeypatch.setattr(mcp_session, "resolve_browser", resolve_browser)++ result = await mcp_session.skyvern_browser_session_create()++ assert result["ok"] is True+ assert result["data"]["browser"] == "extension"+ assert result["data"]["session"] == "implicit"+ get_or_start.assert_awaited_once_with()+ runtime.wait_for_extension.assert_awaited_once_with(10.0)+ resolve_browser.assert_awaited_once_with(extension_runtime=runtime)+++async def test_session_create_extension_not_connected_returns_pinned_guidance(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ runtime = SimpleNamespace(wait_for_extension=AsyncMock(return_value=False))+ get_or_start = AsyncMock(return_value=runtime)+ resolve_browser = AsyncMock()+ monkeypatch.setenv("BROWSER_TYPE", "extension-connect")+ monkeypatch.setattr(mcp_session.BrowserExtensionRuntime, "get_or_start", get_or_start)+ monkeypatch.setattr(mcp_session, "resolve_browser", resolve_browser)++ result = await mcp_session.skyvern_browser_session_create()++ assert result["ok"] is False+ assert result["error"]["message"] == _NOT_CONNECTED_GUIDANCE+ get_or_start.assert_awaited_once_with()+ runtime.wait_for_extension.assert_awaited_once_with(10.0)+ resolve_browser.assert_not_awaited()+++async def test_session_create_extension_connected_returns_safe_success_payload(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ runtime = SimpleNamespace(+ cdp_ws_url="ws://127.0.0.1/private-capability",+ pairing_token="private-pairing-token",+ wait_for_extension=AsyncMock(return_value=True),+ )+ monkeypatch.setenv("BROWSER_TYPE", "extension-connect")+ monkeypatch.setattr(mcp_session.BrowserExtensionRuntime, "get_or_start", AsyncMock(return_value=runtime))+ monkeypatch.setattr(+ mcp_session,+ "resolve_browser",+ AsyncMock(return_value=(MagicMock(), BrowserContext(mode="extension"))),+ )++ result = await mcp_session.skyvern_browser_session_create()++ assert result["ok"] is True+ assert result["browser_context"]["mode"] == "extension"+ assert result["data"]["browser"] == "extension"+ assert result["data"]["session"] == "implicit"+ assert runtime.cdp_ws_url not in repr(result)+ assert runtime.pairing_token not in repr(result)+++async def test_session_create_extension_runtime_error_is_structured(monkeypatch: pytest.MonkeyPatch) -> None:+ message = "Browser extension relay port 19777 is already in use"+ monkeypatch.setenv("BROWSER_TYPE", "extension-connect")+ monkeypatch.setattr(+ mcp_session.BrowserExtensionRuntime,+ "get_or_start",+ AsyncMock(side_effect=BrowserExtensionError(message)),+ )++ result = await mcp_session.skyvern_browser_session_create()++ assert result["ok"] is False+ assert result["error"]["message"] == message+++async def test_session_create_extension_connection_error_redacts_capability_url(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ from skyvern.library.skyvern import Skyvern++ capability_token = "fake-secret-capability"+ capability_url = f"ws://127.0.0.1:43210/cdp/{capability_token}"+ connect_over_cdp = AsyncMock(+ side_effect=RuntimeError(f"BrowserType.connect_over_cdp failed while connecting to {capability_url}")+ )+ playwright = SimpleNamespace(chromium=SimpleNamespace(connect_over_cdp=connect_over_cdp))+ skyvern = object.__new__(Skyvern)+ skyvern._get_playwright = AsyncMock(return_value=playwright)+ runtime = SimpleNamespace(cdp_ws_url=capability_url, wait_for_extension=AsyncMock(return_value=True))+ current_token = session_manager._current_session.set(None)+ monkeypatch.setattr(session_manager, "_global_session", None)+ monkeypatch.setattr(session_manager, "get_skyvern", lambda: skyvern)+ monkeypatch.setenv("BROWSER_TYPE", "extension-connect")+ monkeypatch.setattr(mcp_session.BrowserExtensionRuntime, "get_or_start", AsyncMock(return_value=runtime))++ try:+ result = await mcp_session.skyvern_browser_session_create()+ finally:+ session_manager._current_session.reset(current_token)++ assert result["ok"] is False+ assert result["error"]["message"] == _NOT_CONNECTED_GUIDANCE+ assert "/cdp/" not in repr(result)+ assert capability_token not in repr(result)+ connect_over_cdp.assert_awaited_once_with(capability_url)+++async def test_resolve_browser_extension_connects_before_cdp(monkeypatch: pytest.MonkeyPatch) -> None:+ current_token = session_manager._current_session.set(None)+ monkeypatch.setattr(session_manager, "_global_session", None)+ runtime = SimpleNamespace()+ browser = MagicMock()+ fake_skyvern = MagicMock()+ fake_skyvern.connect_to_browser_extension = AsyncMock(return_value=browser)+ fake_skyvern.connect_to_browser_over_cdp = AsyncMock()+ monkeypatch.setattr(session_manager, "get_skyvern", lambda: fake_skyvern)++ try:+ resolved_browser, context = await session_manager.resolve_browser(+ cdp_url="ws://cdp.example.test",+ extension_runtime=runtime,+ )+ finally:+ session_manager._current_session.reset(current_token)++ assert resolved_browser is browser+ assert context == BrowserContext(mode="extension", can_access_localhost=True)+ fake_skyvern.connect_to_browser_extension.assert_awaited_once_with(runtime)+ fake_skyvern.connect_to_browser_over_cdp.assert_not_awaited()+++async def test_resolve_browser_reconnects_disconnected_extension_session(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ current_token = session_manager._current_session.set(None)+ monkeypatch.setattr(session_manager, "_global_session", None)+ playwright_browser = SimpleNamespace(is_connected=Mock(side_effect=[True, False]))+ stale_browser = SimpleNamespace(browser=playwright_browser, close=AsyncMock())+ fresh_browser = SimpleNamespace(browser=SimpleNamespace(is_connected=Mock(return_value=True)))+ runtime = SimpleNamespace()+ fake_skyvern = MagicMock()+ fake_skyvern.connect_to_browser_extension = AsyncMock(return_value=fresh_browser)+ monkeypatch.setattr(session_manager, "get_active_api_key", lambda: None)+ monkeypatch.setattr(session_manager, "get_skyvern", lambda: fake_skyvern)+ session_manager.set_current_session(+ session_manager.SessionState(browser=stale_browser, context=BrowserContext(mode="extension"))+ )++ try:+ first_browser, first_context = await session_manager.resolve_browser(extension_runtime=runtime)+ second_browser, second_context = await session_manager.resolve_browser(extension_runtime=runtime)+ finally:+ session_manager._current_session.reset(current_token)++ assert first_browser is stale_browser+ assert first_context.mode == "extension"+ assert second_browser is fresh_browser+ assert second_context == BrowserContext(mode="extension", can_access_localhost=True)+ assert playwright_browser.is_connected.call_count == 2+ stale_browser.close.assert_awaited_once_with()+ fake_skyvern.connect_to_browser_extension.assert_awaited_once_with(runtime)+++def test_extension_session_ref_key_is_stable_and_distinct(monkeypatch: pytest.MonkeyPatch) -> None:+ monkeypatch.setattr(session_manager, "get_active_api_key", lambda: None)+ extension_state = session_manager.SessionState(context=BrowserContext(mode="extension"))+ local_state = session_manager.SessionState(context=BrowserContext(mode="local"))++ first = session_manager._session_ref_key(extension_state)+ second = session_manager._session_ref_key(extension_state)++ assert first == second == (None, "extension", "own-browser", None)+ assert first != session_manager._session_ref_key(local_state)++[email protected]("has_existing_context", [True, False])+async def test_connect_to_browser_extension_uses_runtime_cdp_url_and_context_fallback(+ monkeypatch: pytest.MonkeyPatch,+ has_existing_context: bool,+) -> None:+ from skyvern.library.skyvern import Skyvern++ existing_context = MagicMock()+ new_context = MagicMock()+ browser = SimpleNamespace(+ contexts=[existing_context] if has_existing_context else [],+ new_context=AsyncMock(return_value=new_context),+ )+ connect_over_cdp = AsyncMock(return_value=browser)+ playwright = SimpleNamespace(chromium=SimpleNamespace(connect_over_cdp=connect_over_cdp))+ skyvern = object.__new__(Skyvern)+ skyvern._get_playwright = AsyncMock(return_value=playwright)+ runtime = SimpleNamespace(cdp_ws_url="ws://127.0.0.1/private-capability")++ wrapper_module = types.ModuleType("skyvern.library.skyvern_browser")++ class FakeSkyvernBrowser:+ def __init__(self, client: object, browser_context: object, **kwargs: object) -> None:+ self.client = client+ self.browser_context = browser_context+ self.kwargs = kwargs++ wrapper_module.SkyvernBrowser = FakeSkyvernBrowser+ monkeypatch.setitem(sys.modules, "skyvern.library.skyvern_browser", wrapper_module)++ connected = await skyvern.connect_to_browser_extension(runtime)++ connect_over_cdp.assert_awaited_once_with(runtime.cdp_ws_url)+ assert connected.browser_context is (existing_context if has_existing_context else new_context)+ if has_existing_context:+ browser.new_context.assert_not_awaited()+ else:+ browser.new_context.assert_awaited_once_with()+++def test_browser_extension_path_command_prints_real_absolute_directory() -> None:+ result = CliRunner().invoke(browser_app, ["extension-path"])++ assert result.exit_code == 0+ assert result.stdout.strip() == str(BrowserExtensionRuntime.extension_dir().resolve())+++def test_browser_extension_token_command_creates_but_does_not_print_token(+ monkeypatch: pytest.MonkeyPatch,+ tmp_path: Path,+) -> None:+ monkeypatch.setenv("HOME", str(tmp_path))+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_TOKEN", raising=False)+ monkeypatch.setattr(browser_commands.shutil, "which", lambda _name: None)++ result = CliRunner().invoke(browser_app, ["extension-token"])++ token_path = tmp_path / ".skyvern" / "browser_extension_token"+ token = token_path.read_text()+ assert result.exit_code == 0+ assert token_path.is_file()+ assert token not in result.stdout+ assert result.stdout.splitlines()[0].startswith("Pairing token was not printed.")+ assert "SKYVERN_BROWSER_EXTENSION_TOKEN" in result.stdout+ assert "Paste this token into the Skyvern browser extension popup." in result.stdout+++def test_browser_extension_token_command_copies_token_when_clipboard_is_available(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ token = "pairing-token-clipboard-sentinel"+ run = MagicMock()+ monkeypatch.setattr(browser_commands, "load_or_create_pairing_token", lambda: token)+ monkeypatch.setattr(browser_commands.sys, "platform", "darwin")+ monkeypatch.setattr(browser_commands.shutil, "which", lambda name: f"/usr/bin/{name}")+ monkeypatch.setattr(browser_commands.subprocess, "run", run)++ result = CliRunner().invoke(browser_app, ["extension-token"])++ assert result.exit_code == 0+ assert result.stdout.splitlines() == [+ "Pairing token copied to clipboard.",+ "Paste this token into the Skyvern browser extension popup.",+ "Click Connect.",+ ]+ run.assert_called_once_with(+ ["/usr/bin/pbcopy"],+ input=token,+ text=True,+ check=True,+ stdout=browser_commands.subprocess.DEVNULL,+ stderr=browser_commands.subprocess.DEVNULL,+ )+++def test_browser_extension_status_reports_configuration_without_printing_token(+ monkeypatch: pytest.MonkeyPatch,+ tmp_path: Path,+) -> None:+ token = "status-must-not-print-this-token"+ token_dir = tmp_path / ".skyvern"+ token_dir.mkdir()+ token_path = token_dir / "browser_extension_token"+ token_path.write_text(token)+ token_path.chmod(0o600)+ connection = MagicMock()+ connect = MagicMock(return_value=connection)+ monkeypatch.setenv("HOME", str(tmp_path))+ monkeypatch.setenv("SKYVERN_BROWSER_EXTENSION_PORT", "20123")+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_TOKEN", raising=False)+ monkeypatch.setattr(browser_commands.socket, "create_connection", connect)++ result = CliRunner().invoke(browser_app, ["extension-status"])++ assert result.exit_code == 0+ assert str(BrowserExtensionRuntime.extension_dir().resolve()) in result.stdout+ assert "pairing token: configured (file exists)" in result.stdout+ assert "pairing token file permissions: OK" in result.stdout+ assert "bridge listening on 20123" in result.stdout+ assert token not in result.stdout+ connect.assert_called_once_with(("127.0.0.1", 20123), timeout=0.5)+ connection.close.assert_called_once_with()+++def test_browser_extension_status_is_informational_when_bridge_is_not_running(+ monkeypatch: pytest.MonkeyPatch,+ tmp_path: Path,+) -> None:+ monkeypatch.setenv("HOME", str(tmp_path))+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_TOKEN", raising=False)+ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_PORT", raising=False)+ monkeypatch.setattr(browser_commands.socket, "create_connection", MagicMock(side_effect=OSError))++ result = CliRunner().invoke(browser_app, ["extension-status"])++ assert result.exit_code == 0+ assert "pairing token: not configured" in result.stdout+ assert "bridge not running (start your MCP server with --browser-extension)" in result.stdout+++def test_browser_extension_pair_exits_with_guidance_when_bridge_is_not_running(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ monkeypatch.setattr(browser_commands.socket, "create_connection", MagicMock(side_effect=OSError))++ result = CliRunner().invoke(browser_app, ["extension-pair"])++ assert result.exit_code == 1+ assert "Start your MCP server first: skyvern run mcp --browser-extension" in result.stdout+++def test_browser_extension_pair_opens_fragment_url_without_printing_token(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ token = "pair-command-token-must-stay-private"+ open_pairing_url = MagicMock(return_value=True)+ monkeypatch.setattr(browser_commands.BrowserExtensionRuntime, "configured_port", lambda: 20123)+ monkeypatch.setattr(browser_commands, "_bridge_is_listening", lambda _port: True)+ monkeypatch.setattr(browser_commands, "load_or_create_pairing_token", lambda: token)+ monkeypatch.setattr(browser_commands, "_request_pairing_nonce", lambda _port, _token: "pairing-nonce")+ monkeypatch.setattr(browser_commands, "_open_pairing_url", open_pairing_url)++ result = CliRunner().invoke(browser_app, ["extension-pair"])++ assert result.exit_code == 0+ assert result.stdout.strip() == "Approve the pairing in your browser."+ assert token not in result.stdout+ open_pairing_url.assert_called_once_with("http://127.0.0.1:20123/pair#pairing-nonce")+++def test_browser_extension_pair_begin_uses_hex_hmac_proof(monkeypatch: pytest.MonkeyPatch) -> None:+ token = "pair-begin-proof-token"+ response = SimpleNamespace(status=200, read=MagicMock(return_value=b'{"v":1,"nonce":"pairing-nonce"}'))+ connection = MagicMock()+ connection.getresponse.return_value = response+ http_connection = MagicMock(return_value=connection)+ monkeypatch.setattr(browser_commands.http.client, "HTTPConnection", http_connection)++ nonce = browser_commands._request_pairing_nonce(20123, token)++ assert nonce == "pairing-nonce"+ http_connection.assert_called_once_with("127.0.0.1", 20123, timeout=2.0)+ request = connection.request.call_args+ payload = json.loads(request.kwargs["body"])+ expected_proof = hmac.new(token.encode(), b"skyvern-pair-begin-v1", hashlib.sha256).hexdigest()+ assert payload == {"v": 1, "proof": expected_proof}+ connection.close.assert_called_once_with()+++def test_browser_extension_install_copies_token_opens_chrome_and_prints_steps(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ token = "install-token-must-stay-private"+ extension_dir = BrowserExtensionRuntime.extension_dir().resolve()+ run = MagicMock()+ monkeypatch.setattr(browser_commands, "load_or_create_pairing_token", lambda: token)+ monkeypatch.setattr(browser_commands.sys, "platform", "darwin")+ monkeypatch.setattr(browser_commands.shutil, "which", lambda name: f"/usr/bin/{name}")+ monkeypatch.setattr(browser_commands.subprocess, "run", run)+ monkeypatch.setattr(browser_commands, "_bridge_is_listening", lambda _port: False)++ result = CliRunner().invoke(browser_app, ["extension-install"])++ assert result.exit_code == 0+ assert result.stdout.splitlines()[0] == str(extension_dir)+ assert "Pairing token copied to clipboard." in result.stdout+ assert token not in result.stdout+ assert "1. Enable Developer mode." in result.stdout+ assert "2. Click Load unpacked." in result.stdout+ assert f"3. Select {extension_dir}." in result.stdout+ assert "4. Open the Skyvern Agent popup." in result.stdout+ assert "5. Paste the pairing token and click Connect." in result.stdout+ assert '6. Add tabs to the "Skyvern Controlled" group.' in result.stdout+ assert "When your MCP server is running, pair with: skyvern browser extension-pair" in result.stdout+ assert run.call_args_list == [+ call(+ ["/usr/bin/pbcopy"],+ input=token,+ text=True,+ check=True,+ stdout=browser_commands.subprocess.DEVNULL,+ stderr=browser_commands.subprocess.DEVNULL,+ ),+ call(+ ["/usr/bin/open", "-a", "Google Chrome", "chrome://extensions"],+ check=True,+ stdout=browser_commands.subprocess.DEVNULL,+ stderr=browser_commands.subprocess.DEVNULL,+ ),+ ]+++def test_browser_extension_install_attempts_one_click_pairing_when_bridge_is_listening(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ launch_pairing = MagicMock()+ load_token = MagicMock()+ monkeypatch.setattr(browser_commands.BrowserExtensionRuntime, "configured_port", lambda: 20123)+ monkeypatch.setattr(browser_commands, "_bridge_is_listening", lambda _port: True)+ monkeypatch.setattr(browser_commands, "_open_chrome_extensions", lambda: False)+ monkeypatch.setattr(browser_commands, "_launch_extension_pairing", launch_pairing)+ monkeypatch.setattr(browser_commands, "load_or_create_pairing_token", load_token)++ result = CliRunner().invoke(browser_app, ["extension-install"])++ assert result.exit_code == 0+ assert "4. Click Approve in the pairing page." in result.stdout+ assert "5. Approve the pairing in the Skyvern Agent confirmation tab." in result.stdout+ assert "pairing token" not in result.stdout.lower()+ launch_pairing.assert_called_once_with(20123)+ load_token.assert_not_called()+++def test_run_mcp_browser_extension_flag_starts_and_stops_runtime(monkeypatch: pytest.MonkeyPatch) -> None:+ events: list[str] = []+ runtime = SimpleNamespace(shutdown=AsyncMock(side_effect=lambda: events.append("shutdown")))++ async def get_or_start() -> object:+ events.append("start")+ return runtime++ async def run_async(**_kwargs: object) -> None:+ assert run_commands.os.environ["BROWSER_TYPE"] == "extension-connect"+ events.append("serve")++ async def cleanup() -> None:+ events.append("cleanup")++ # delenv on an absent var records no undo, so run_mcp's os.environ.setdefault+ # would leak into later tests; setenv first guarantees restoration.+ monkeypatch.setenv("BROWSER_TYPE", "leak-guard-sentinel")+ monkeypatch.delenv("BROWSER_TYPE", raising=False)+ monkeypatch.setattr(run_commands, "prepare_cli_runtime", lambda **_kwargs: None)+ monkeypatch.setattr(run_commands.atexit, "register", lambda _callback: None)+ monkeypatch.setattr(run_commands.signal, "signal", lambda *_args: None)+ monkeypatch.setattr(run_commands, "_start_stdin_eof_watcher", lambda: (MagicMock(), MagicMock()))+ monkeypatch.setattr(run_commands, "_cleanup_mcp_resources", cleanup)+ monkeypatch.setattr(+ "skyvern.library.local_browser_profile.sweep_local_browser_profiles_with_budget",+ lambda: None,+ )+ monkeypatch.setattr("skyvern.cli.mcp_tools.mcp.run_async", run_async)+ monkeypatch.setattr(BrowserExtensionRuntime, "get_or_start", AsyncMock(side_effect=get_or_start))+ monkeypatch.setattr(run_commands, "_mcp_cleanup_done", False)+ monkeypatch.setattr(run_commands, "_mcp_cleanup_in_progress", False)++ run_commands.run_mcp(browser_extension=True)++ assert run_commands.os.environ["BROWSER_TYPE"] == "extension-connect"+ assert events == ["start", "serve", "cleanup", "shutdown"]+ runtime.shutdown.assert_awaited_once_with()++[email protected]("transport", ["sse", "streamable-http"])+def test_run_mcp_rejects_browser_extension_with_http_transport(+ monkeypatch: pytest.MonkeyPatch,+ transport: str,+) -> None:+ monkeypatch.setenv("BROWSER_TYPE", "leak-guard-sentinel")+ monkeypatch.delenv("BROWSER_TYPE", raising=False)++ with pytest.raises(typer.BadParameter, match="--browser-extension requires --transport stdio"):+ run_commands.run_mcp(transport=transport, browser_extension=True)++ assert "BROWSER_TYPE" not in run_commands.os.environdiff --git a/tests/unit/browser_extension/test_protocol.py b/tests/unit/browser_extension/test_protocol.pynew file mode 100644index 000000000..e2c0bfc33--- /dev/null+++ b/tests/unit/browser_extension/test_protocol.py@@ -0,0 +1,157 @@+from __future__ import annotations++import json++import pytest++from skyvern.browser_extension.errors import BrowserExtensionError+from skyvern.browser_extension.protocol import (+ ALLOWED_CDP_METHOD_PREFIXES,+ ALLOWED_EVENTS,+ ALLOWED_OPS,+ DENIED_CDP_METHODS,+ ERROR_CODES,+ EXTENSION_ID,+ PROTOCOL_VERSION,+ build_request,+ is_cdp_method_allowed,+ is_restricted_url,+ parse_extension_message,+)+++def test_parse_valid_response() -> None:+ parsed = parse_extension_message(+ json.dumps({"v": 1, "type": "response", "id": "r-1", "ok": True, "result": {"tabId": 12}})+ )++ assert parsed.kind == "response"+ assert parsed.request_id == "r-1"+ assert parsed.ok is True+ assert parsed.result == {"tabId": 12}+ assert parsed.error_code is None+++def test_parse_valid_event() -> None:+ parsed = parse_extension_message(+ json.dumps(+ {+ "v": 1,+ "type": "event",+ "event": "scope.tabAdded",+ "params": {"tabId": 12, "url": "https://example.com", "title": "Example"},+ }+ )+ )++ assert parsed.kind == "event"+ assert parsed.event == "scope.tabAdded"+ assert parsed.params == {"tabId": 12, "url": "https://example.com", "title": "Example"}+++def test_parse_valid_ping() -> None:+ parsed = parse_extension_message('{"v":1,"type":"ping"}')++ assert parsed.kind == "ping"+++def test_unknown_message_type_raises() -> None:+ with pytest.raises(BrowserExtensionError):+ parse_extension_message('{"v":1,"type":"auth.ok"}')+++def test_protocol_allowlists_match_contract() -> None:+ assert PROTOCOL_VERSION == 1+ assert EXTENSION_ID == "fmamdhmfeihjjaiheideemihnbpnokin"+ assert ALLOWED_OPS == frozenset(+ {+ "debugger.attach",+ "debugger.detach",+ "debugger.send",+ "tabs.create",+ "tabs.remove",+ "tabs.activate",+ "tabs.list",+ }+ )+ assert ALLOWED_EVENTS == frozenset(+ {+ "extension.hello",+ "debugger.event",+ "debugger.detached",+ "scope.tabAdded",+ "scope.tabRemoved",+ "tabs.created",+ }+ )+ assert ERROR_CODES == frozenset(+ {+ "AUTH_FAILED",+ "OP_NOT_ALLOWED",+ "TAB_NOT_FOUND",+ "TAB_NOT_SCOPED",+ "RESTRICTED_URL",+ "DEBUGGER_DETACHED",+ "CDP_METHOD_NOT_ALLOWED",+ "CDP_ERROR",+ "INTERNAL",+ }+ )+++def test_build_request_checks_operation_allowlist() -> None:+ assert build_request("r-3", "tabs.list", {}) == {+ "v": 1,+ "type": "request",+ "id": "r-3",+ "op": "tabs.list",+ "args": {},+ }++ with pytest.raises(BrowserExtensionError):+ build_request("r-4", "cookies.read", {})+ with pytest.raises(BrowserExtensionError):+ build_request(4, "tabs.list", {}) # type: ignore[arg-type]+++def test_cdp_method_allowlist() -> None:+ assert all(prefix.endswith(".") for prefix in ALLOWED_CDP_METHOD_PREFIXES)+ assert is_cdp_method_allowed("Network.enable")+ assert is_cdp_method_allowed("Storage.getUsageAndQuota")+ assert is_cdp_method_allowed("Network.getCookies")+ assert not is_cdp_method_allowed("Network.getCookies", {"urls": []})+ assert not is_cdp_method_allowed("Browser.close")+ assert not is_cdp_method_allowed("SystemInfo.getInfo")+ assert not is_cdp_method_allowed("PageX.navigate")+++ "method",+ [+ "Network.getAllCookies",+ "Network.clearBrowserCookies",+ "Network.clearBrowserCache",+ "Storage.getCookies",+ "Storage.setCookies",+ "Storage.clearCookies",+ ],+)+def test_cdp_method_denylist(method: str) -> None:+ assert method in DENIED_CDP_METHODS+ assert not is_cdp_method_allowed(method)+++ ("url", "restricted"),+ [+ ("chrome://settings", True),+ ("about:blank", False),+ ("about:config", True),+ ("https://chromewebstore.google.com/detail/example", True),+ ("https://chromewebstore.google.com./detail/x", True),+ ("https://example.com", False),+ ("file:///tmp/example.html", True),+ ],+)+def test_restricted_url_matrix(url: str, restricted: bool) -> None:+ assert is_restricted_url(url) is restricteddiff --git a/tests/unit/browser_extension/test_relay.py b/tests/unit/browser_extension/test_relay.pynew file mode 100644index 000000000..21a43dcb2--- /dev/null+++ b/tests/unit/browser_extension/test_relay.py@@ -0,0 +1,457 @@+from __future__ import annotations++import asyncio+import base64+import hashlib+import hmac+import secrets+from collections.abc import AsyncGenerator++import pytest+import pytest_asyncio+from aiohttp import ClientSession, ClientWebSocketResponse, WSMsgType++import skyvern.browser_extension.relay as relay_module+from skyvern.browser_extension.auth import compute_ext_proof, compute_server_proof+from skyvern.browser_extension.errors import BrowserExtensionNotConnectedError, ExtensionRequestError+from skyvern.browser_extension.relay import ExtensionRelayServer++TOKEN = "test-pairing-token"+++class RelayHarness:+ def __init__(self) -> None:+ self.events: list[tuple[str, dict]] = []+ self.event_received = asyncio.Event()+ self.disconnect_called = asyncio.Event()+ self.server = ExtensionRelayServer(TOKEN, 0, self.on_event, self.on_disconnect)++ async def on_event(self, event: str, params: dict) -> None:+ self.events.append((event, params))+ self.event_received.set()++ async def on_disconnect(self) -> None:+ self.disconnect_called.set()+++@pytest_asyncio.fixture+async def relay_harness() -> AsyncGenerator[RelayHarness]:+ harness = RelayHarness()+ await harness.server.start()+ yield harness+ await harness.server.stop()+++def relay_url(harness: RelayHarness) -> str:+ return f"ws://127.0.0.1:{harness.server.bound_port}/extension/v1"+++def http_url(harness: RelayHarness, path: str) -> str:+ return f"http://127.0.0.1:{harness.server.bound_port}{path}"+++def pair_begin_proof(token: str) -> str:+ return hmac.new(token.encode(), b"skyvern-pair-begin-v1", hashlib.sha256).hexdigest()+++async def authenticate(+ session: ClientSession,+ harness: RelayHarness,+ *,+ origin: str | None = "chrome-extension://abcdefghijklmnop",+ send_hello: bool = True,+) -> ClientWebSocketResponse:+ headers = {"Origin": origin} if origin is not None else None+ websocket = await session.ws_connect(relay_url(harness), headers=headers)+ challenge = await websocket.receive_json()+ client_nonce = secrets.token_urlsafe(32)+ await websocket.send_json(+ {+ "v": 1,+ "type": "auth.proof",+ "clientNonce": client_nonce,+ "proof": compute_ext_proof(TOKEN, challenge["serverNonce"], client_nonce),+ }+ )+ auth_ok = await websocket.receive_json()+ assert auth_ok == {+ "v": 1,+ "type": "auth.ok",+ "serverProof": compute_server_proof(TOKEN, client_nonce, challenge["serverNonce"]),+ }+ if send_hello:+ event_count = len(harness.events)+ await websocket.send_json(+ {+ "v": 1,+ "type": "event",+ "event": "extension.hello",+ "params": {"extensionVersion": "1.0.0", "scopedTabs": []},+ }+ )++ async def hello_processed() -> None:+ while len(harness.events) == event_count or not harness.server.connected:+ await asyncio.sleep(0)++ await asyncio.wait_for(hello_processed(), 1)+ return websocket+++async def test_pair_begin_claim_happy_path_and_pair_page_never_contains_token(+ relay_harness: RelayHarness,+) -> None:+ async with ClientSession() as session:+ begin = await session.post(+ http_url(relay_harness, "/pair/begin"),+ json={"v": 1, "proof": pair_begin_proof(TOKEN)},+ )+ assert begin.status == 200+ assert begin.headers["Cache-Control"] == "no-store"+ begin_payload = await begin.json()+ assert begin_payload["v"] == 1+ assert isinstance(begin_payload["nonce"], str)+ assert begin_payload["nonce"]++ page = await session.get(http_url(relay_harness, "/pair"))+ page_body = await page.text()+ assert page.status == 200+ assert TOKEN not in page_body+ assert "fmamdhmfeihjjaiheideemihnbpnokin" in page_body++ claim = await session.post(+ http_url(relay_harness, "/pair/claim"),+ json={"v": 1, "nonce": begin_payload["nonce"]},+ )+ assert claim.status == 200+ assert claim.headers["Cache-Control"] == "no-store"+ assert await claim.json() == {"v": 1, "port": relay_harness.server.bound_port, "token": TOKEN}+++async def test_pair_claim_wrong_nonce_is_forbidden_and_consumes_active_nonce(+ relay_harness: RelayHarness,+) -> None:+ nonce = relay_harness.server.create_pairing_nonce()+ async with ClientSession() as session:+ wrong = await session.post(+ http_url(relay_harness, "/pair/claim"),+ json={"v": 1, "nonce": secrets.token_urlsafe(32)},+ )+ assert wrong.status == 403+ assert await wrong.json() == {"error": "invalid_nonce"}++ consumed = await session.post(+ http_url(relay_harness, "/pair/claim"),+ json={"v": 1, "nonce": nonce},+ )+ assert consumed.status == 403+++async def test_pair_claim_nonce_is_single_use(relay_harness: RelayHarness) -> None:+ nonce = relay_harness.server.create_pairing_nonce()+ async with ClientSession() as session:+ first = await session.post(+ http_url(relay_harness, "/pair/claim"),+ json={"v": 1, "nonce": nonce},+ )+ second = await session.post(+ http_url(relay_harness, "/pair/claim"),+ json={"v": 1, "nonce": nonce},+ )++ assert first.status == 200+ assert second.status == 403+ assert await second.json() == {"error": "invalid_nonce"}+++async def test_pair_claim_expired_nonce_is_forbidden(+ relay_harness: RelayHarness,+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ now = 10_000.0+ monkeypatch.setattr(relay_module.time, "monotonic", lambda: now)+ nonce = relay_harness.server.create_pairing_nonce()+ now += 121.0++ async with ClientSession() as session:+ response = await session.post(+ http_url(relay_harness, "/pair/claim"),+ json={"v": 1, "nonce": nonce},+ )++ assert response.status == 403+ assert await response.json() == {"error": "invalid_nonce"}+++async def test_pair_begin_bad_proof_is_forbidden(relay_harness: RelayHarness) -> None:+ async with ClientSession() as session:+ response = await session.post(+ http_url(relay_harness, "/pair/begin"),+ json={"v": 1, "proof": "not-the-proof"},+ )++ assert response.status == 403+++async def test_authentication_and_hello_update_scoped_tabs(relay_harness: RelayHarness) -> None:+ async with ClientSession() as session:+ websocket = await authenticate(session, relay_harness, send_hello=False)++ assert not relay_harness.server.connected+ assert not await relay_harness.server.wait_connected(0.01)++ params = {+ "extensionVersion": "1.0.0",+ "scopedTabs": [{"tabId": 17, "url": "https://example.com", "title": "Example"}],+ }+ await websocket.send_json({"v": 1, "type": "event", "event": "extension.hello", "params": params})+ await asyncio.wait_for(relay_harness.event_received.wait(), 1)++ assert relay_harness.server.connected+ assert await relay_harness.server.wait_connected(0.1)+ assert relay_harness.server.scoped_tabs == [{"tabId": 17, "url": "https://example.com", "title": "Example"}]+ assert relay_harness.events == [("extension.hello", params)]+++async def test_wrong_proof_is_closed_with_4403(relay_harness: RelayHarness) -> None:+ async with ClientSession() as session:+ websocket = await session.ws_connect(relay_url(relay_harness))+ challenge = await websocket.receive_json()+ assert challenge["type"] == "auth.challenge"+ await websocket.send_json(+ {"v": 1, "type": "auth.proof", "clientNonce": secrets.token_urlsafe(32), "proof": "wrong-proof"}+ )++ message = await websocket.receive()++ assert message.type in {WSMsgType.CLOSE, WSMsgType.CLOSED}+ assert websocket.close_code == 4403+ assert not relay_harness.server.connected+++ "client_nonce",+ [+ "short",+ secrets.token_urlsafe(31),+ secrets.token_urlsafe(32) + "=",+ secrets.token_urlsafe(33),+ "!" * 43,+ base64.urlsafe_b64encode(b"\0" * 32).rstrip(b"=").decode("ascii")[:-1] + "B",+ ],+)+async def test_client_nonce_must_be_unpadded_base64url_for_exactly_32_bytes(+ relay_harness: RelayHarness,+ client_nonce: str,+) -> None:+ async with ClientSession() as session:+ websocket = await session.ws_connect(relay_url(relay_harness))+ challenge = await websocket.receive_json()+ await websocket.send_json(+ {+ "v": 1,+ "type": "auth.proof",+ "clientNonce": client_nonce,+ "proof": compute_ext_proof(TOKEN, challenge["serverNonce"], client_nonce),+ }+ )++ message = await websocket.receive()++ assert message.type in {WSMsgType.CLOSE, WSMsgType.CLOSED}+ assert websocket.close_code == 4403+ assert not relay_harness.server.connected+++async def test_missing_proof_times_out_with_4403(+ relay_harness: RelayHarness,+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ monkeypatch.setattr(relay_module, "_AUTH_TIMEOUT_SECONDS", 0.01)+ async with ClientSession() as session:+ websocket = await session.ws_connect(relay_url(relay_harness))+ challenge = await websocket.receive_json()+ assert challenge["type"] == "auth.challenge"++ message = await websocket.receive()++ assert message.type in {WSMsgType.CLOSE, WSMsgType.CLOSED}+ assert websocket.close_code == 4403+++async def test_origin_validation_rejects_web_origin_and_allows_missing_origin(relay_harness: RelayHarness) -> None:+ async with ClientSession() as session:+ rejected = await session.ws_connect(relay_url(relay_harness), headers={"Origin": "https://example.com"})++ message = await rejected.receive()++ assert message.type in {WSMsgType.CLOSE, WSMsgType.CLOSED}+ assert rejected.close_code == 4403++ allowed = await session.ws_connect(relay_url(relay_harness))+ challenge = await allowed.receive_json()++ assert challenge["type"] == "auth.challenge"+ await allowed.close()+++async def test_request_response_error_and_timeout_paths(relay_harness: RelayHarness) -> None:+ async with ClientSession() as session:+ websocket = await authenticate(session, relay_harness)++ success_task = asyncio.create_task(relay_harness.server.request("tabs.list", {}))+ success_request = await websocket.receive_json()+ assert success_request == {"v": 1, "type": "request", "id": "r-1", "op": "tabs.list", "args": {}}+ await websocket.send_json(+ {"v": 1, "type": "response", "id": success_request["id"], "ok": True, "result": {"tabs": []}}+ )+ assert await success_task == {"tabs": []}++ error_task = asyncio.create_task(relay_harness.server.request("debugger.attach", {"tabId": 17}))+ error_request = await websocket.receive_json()+ await websocket.send_json(+ {+ "v": 1,+ "type": "response",+ "id": error_request["id"],+ "ok": False,+ "error": {"code": "TAB_NOT_SCOPED", "message": "tab is not shared"},+ }+ )+ with pytest.raises(ExtensionRequestError) as error_info:+ await error_task+ assert error_info.value.code == "TAB_NOT_SCOPED"+ assert error_info.value.message == "tab is not shared"++ timeout_task = asyncio.create_task(relay_harness.server.request("tabs.list", {}, timeout=0.01))+ timeout_request = await websocket.receive_json()+ assert timeout_request["id"] == "r-3"+ with pytest.raises(ExtensionRequestError) as timeout_info:+ await timeout_task+ assert timeout_info.value.code == "INTERNAL"+ assert timeout_info.value.message == "extension request timed out: tabs.list"+++async def test_response_larger_than_default_aiohttp_limit_reaches_requester(relay_harness: RelayHarness) -> None:+ async with ClientSession() as session:+ websocket = await authenticate(session, relay_harness)+ response_task = asyncio.create_task(relay_harness.server.request("debugger.send", {}))+ request = await websocket.receive_json()+ large_payload = "x" * (6 * 1024 * 1024)++ await websocket.send_json(+ {"v": 1, "type": "response", "id": request["id"], "ok": True, "result": {"data": large_payload}}+ )++ assert await response_task == {"data": large_payload}+++async def test_new_authenticated_connection_replaces_old_connection(relay_harness: RelayHarness) -> None:+ async with ClientSession() as session:+ first = await authenticate(session, relay_harness)+ second_task = asyncio.create_task(authenticate(session, relay_harness))+ first_message = await first.receive()+ second = await second_task++ assert first_message.type in {WSMsgType.CLOSE, WSMsgType.CLOSED}+ assert first.close_code == 4000+ assert relay_harness.server.connected+ assert not second.closed+ assert relay_harness.disconnect_called.is_set()+++async def test_scope_events_add_create_and_remove_scoped_tabs(relay_harness: RelayHarness) -> None:+ async with ClientSession() as session:+ websocket = await authenticate(session, relay_harness)+ await websocket.send_json(+ {+ "v": 1,+ "type": "event",+ "event": "scope.tabAdded",+ "params": {"tabId": 21, "url": "https://example.com/one", "title": "One"},+ }+ )+ await websocket.send_json(+ {+ "v": 1,+ "type": "event",+ "event": "tabs.created",+ "params": {"tabId": 22, "openerTabId": 21, "url": "https://example.com/two"},+ }+ )+ await websocket.send_json(+ {+ "v": 1,+ "type": "event",+ "event": "scope.tabRemoved",+ "params": {"tabId": 21, "reason": "unshared"},+ }+ )++ async def all_events_received() -> None:+ while len(relay_harness.events) < 4:+ await asyncio.sleep(0)++ await asyncio.wait_for(all_events_received(), 1)+ assert relay_harness.server.scoped_tabs == [{"tabId": 22, "url": "https://example.com/two", "title": ""}]+++async def test_disconnect_fails_pending_requests_clears_tabs_and_calls_callback(+ relay_harness: RelayHarness,+) -> None:+ async with ClientSession() as session:+ websocket = await authenticate(session, relay_harness)+ relay_harness.event_received.clear()+ await websocket.send_json(+ {+ "v": 1,+ "type": "event",+ "event": "extension.hello",+ "params": {+ "extensionVersion": "1.0.0",+ "scopedTabs": [{"tabId": 17, "url": "about:blank", "title": ""}],+ },+ }+ )+ await asyncio.wait_for(relay_harness.event_received.wait(), 1)++ pending_request = asyncio.create_task(relay_harness.server.request("tabs.list", {}))+ await websocket.receive_json()+ await websocket.close()++ with pytest.raises(BrowserExtensionNotConnectedError):+ await pending_request+ await asyncio.wait_for(relay_harness.disconnect_called.wait(), 1)+ assert not relay_harness.server.connected+ assert relay_harness.server.scoped_tabs == []+++async def test_extension_ping_receives_pong(relay_harness: RelayHarness) -> None:+ async with ClientSession() as session:+ websocket = await authenticate(session, relay_harness)++ await websocket.send_json({"v": 1, "type": "ping"})++ assert await websocket.receive_json() == {"v": 1, "type": "pong"}+++async def test_request_without_extension_fails_immediately(relay_harness: RelayHarness) -> None:+ with pytest.raises(BrowserExtensionNotConnectedError):+ await relay_harness.server.request("tabs.list", {})diff --git a/tests/unit/browser_extension/test_runtime.py b/tests/unit/browser_extension/test_runtime.pynew file mode 100644index 000000000..44170ced3--- /dev/null+++ b/tests/unit/browser_extension/test_runtime.py@@ -0,0 +1,210 @@+from __future__ import annotations++import asyncio+import errno+from collections.abc import AsyncGenerator, Awaitable, Callable+from pathlib import Path++import pytest+import pytest_asyncio++import skyvern.browser_extension.runtime as runtime_module+from skyvern.browser_extension.errors import BrowserExtensionError+from skyvern.browser_extension.runtime import BrowserExtensionRuntime+++class StubRelay:+ def __init__(+ self,+ token: str,+ port: int,+ on_event: Callable[[str, dict], Awaitable[None]],+ on_disconnect: Callable[[], Awaitable[None]] | None = None,+ *,+ calls: list[str],+ start_error: OSError | None = None,+ ) -> None:+ self.token = token+ self.port = port+ self.on_event = on_event+ self.on_disconnect = on_disconnect+ self.calls = calls+ self.start_error = start_error+ self.connected = True+ self.stop_count = 0++ async def start(self) -> None:+ self.calls.append("relay.start")+ if self.start_error is not None:+ raise self.start_error++ async def stop(self) -> None:+ self.calls.append("relay.stop")+ self.stop_count += 1++ async def wait_connected(self, timeout: float) -> bool:+ return self.connected+++class StubAdapter:+ def __init__(self, registry, relay: StubRelay, *, calls: list[str]) -> None:+ self.registry = registry+ self.relay = relay+ self.calls = calls+ self.events: list[tuple[str, dict]] = []+ self.disconnect_count = 0+ self.stop_count = 0+ self.cdp_ws_url = "ws://127.0.0.1:23456/cdp/test-capability"++ async def start(self) -> None:+ self.calls.append("adapter.start")++ async def stop(self) -> None:+ self.calls.append("adapter.stop")+ self.stop_count += 1++ async def handle_extension_event(self, event: str, params: dict) -> None:+ self.events.append((event, params))++ async def on_extension_disconnect(self) -> None:+ self.disconnect_count += 1+++@pytest_asyncio.fixture(autouse=True)+async def reset_runtime() -> AsyncGenerator[None]:+ BrowserExtensionRuntime._instance = None+ BrowserExtensionRuntime._lock = asyncio.Lock()+ yield+ instance = BrowserExtensionRuntime.instance()+ if instance is not None:+ await instance.shutdown()+++def install_stubs(+ monkeypatch: pytest.MonkeyPatch,+ *,+ relay_start_error: OSError | None = None,+) -> tuple[list[StubRelay], list[StubAdapter], list[str]]:+ relays: list[StubRelay] = []+ adapters: list[StubAdapter] = []+ calls: list[str] = []++ def relay_factory(token, port, on_event, on_disconnect) -> StubRelay:+ relay = StubRelay(+ token,+ port,+ on_event,+ on_disconnect,+ calls=calls,+ start_error=relay_start_error,+ )+ relays.append(relay)+ return relay++ def adapter_factory(registry, relay) -> StubAdapter:+ adapter = StubAdapter(registry, relay, calls=calls)+ adapters.append(adapter)+ return adapter++ monkeypatch.setattr(runtime_module, "_relay_factory", relay_factory)+ monkeypatch.setattr(runtime_module, "_adapter_factory", adapter_factory)+ monkeypatch.setattr(runtime_module, "load_or_create_pairing_token", lambda: "runtime-test-token")+ return relays, adapters, calls+++async def test_singleton_is_idempotent_and_late_binds_adapter_callbacks(monkeypatch: pytest.MonkeyPatch) -> None:+ relays, adapters, calls = install_stubs(monkeypatch)++ first = await BrowserExtensionRuntime.get_or_start(21001)+ second = await BrowserExtensionRuntime.get_or_start(21002)++ assert first is second+ assert BrowserExtensionRuntime.instance() is first+ assert len(relays) == len(adapters) == 1+ assert relays[0].port == 21001+ assert calls == ["adapter.start", "relay.start"]+ assert first.cdp_ws_url == adapters[0].cdp_ws_url+ assert first.extension_connected+ assert await first.wait_for_extension(0.01)++ event_params = {"tabId": 17}+ await relays[0].on_event("scope.tabAdded", event_params)+ assert adapters[0].events == [("scope.tabAdded", event_params)]+ assert relays[0].on_disconnect is not None+ await relays[0].on_disconnect()+ assert adapters[0].disconnect_count == 1+++async def test_port_resolution_prefers_explicit_then_environment_then_default(monkeypatch: pytest.MonkeyPatch) -> None:+ relays, _, _ = install_stubs(monkeypatch)+ monkeypatch.setenv("SKYVERN_BROWSER_EXTENSION_PORT", "22001")++ environment_runtime = await BrowserExtensionRuntime.get_or_start()+ assert relays[-1].port == 22001+ await environment_runtime.shutdown()++ explicit_runtime = await BrowserExtensionRuntime.get_or_start(22002)+ assert relays[-1].port == 22002+ await explicit_runtime.shutdown()++ monkeypatch.delenv("SKYVERN_BROWSER_EXTENSION_PORT")+ default_runtime = await BrowserExtensionRuntime.get_or_start()+ assert relays[-1].port == 19777+ await default_runtime.shutdown()+++async def test_port_in_use_has_actionable_browser_extension_error(monkeypatch: pytest.MonkeyPatch) -> None:+ _, adapters, _ = install_stubs(monkeypatch, relay_start_error=OSError(errno.EADDRINUSE, "address in use"))++ with pytest.raises(BrowserExtensionError, match="SKYVERN_BROWSER_EXTENSION_PORT") as error_info:+ await BrowserExtensionRuntime.get_or_start(23001)++ assert "23001" in str(error_info.value)+ assert adapters[0].stop_count == 1+ assert BrowserExtensionRuntime.instance() is None++[email protected]("port_value", [0, "0"])+async def test_zero_port_is_rejected(monkeypatch: pytest.MonkeyPatch, port_value: int | str) -> None:+ install_stubs(monkeypatch)+ if isinstance(port_value, str):+ monkeypatch.setenv("SKYVERN_BROWSER_EXTENSION_PORT", port_value)+ port = None+ else:+ port = port_value++ with pytest.raises(BrowserExtensionError, match="between 1 and 65535"):+ await BrowserExtensionRuntime.get_or_start(port)+++async def test_shutdown_stops_relay_before_adapter_is_idempotent_and_resets_singleton(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ relays, adapters, calls = install_stubs(monkeypatch)+ runtime = await BrowserExtensionRuntime.get_or_start(24001)++ await runtime.shutdown()+ await runtime.shutdown()++ assert calls == ["adapter.start", "relay.start", "relay.stop", "adapter.stop"]+ assert relays[0].stop_count == adapters[0].stop_count == 1+ assert BrowserExtensionRuntime.instance() is None++ restarted = await BrowserExtensionRuntime.get_or_start(24002)+ assert restarted is not runtime+ assert relays[-1].port == 24002+++def test_extension_dir_points_to_packaged_manifest() -> None:+ directory = BrowserExtensionRuntime.extension_dir()+ expected_directory = Path(runtime_module.__file__).resolve().parent / "extension"++ assert directory == expected_directory+ if not (directory / "manifest.json").exists():+ pytest.skip("extension manifest is owned by another build stream")+ assert directory.is_dir()diff --git a/tests/unit/browser_extension/test_target_registry.py b/tests/unit/browser_extension/test_target_registry.pynew file mode 100644index 000000000..3f2272f7e--- /dev/null+++ b/tests/unit/browser_extension/test_target_registry.py@@ -0,0 +1,168 @@+from __future__ import annotations++import pytest++from skyvern.browser_extension.target_registry import VirtualTargetRegistry+++def test_register_list_and_update_tab_target_info() -> None:+ registry = VirtualTargetRegistry()++ target_id = registry.register_tab(42, "https://example.com/start", "Start")+ root_session_id = registry.root_session_id(42)++ assert target_id == "tab-42"+ assert registry.root_session_id(42) == root_session_id+ assert registry.tab_for_target(target_id) == 42+ assert registry.target_info(target_id) == {+ "targetId": "tab-42",+ "type": "page",+ "title": "Start",+ "url": "https://example.com/start",+ "attached": True,+ "canAccessOpener": False,+ "browserContextId": "skyvern-default",+ }++ registry.update_tab(42, "https://example.com/next", "Next")++ assert registry.list_page_targets() == [+ {+ "targetId": "tab-42",+ "type": "page",+ "title": "Next",+ "url": "https://example.com/next",+ "attached": True,+ "canAccessOpener": False,+ "browserContextId": "skyvern-default",+ }+ ]+++def test_root_session_changes_after_remove_and_reregister() -> None:+ registry = VirtualTargetRegistry()+ registry.register_tab(7, "about:blank", "")+ first_session_id = registry.root_session_id(7)++ registry.remove_tab(7)+ registry.register_tab(7, "about:blank", "")++ assert registry.root_session_id(7) != first_session_id+++def test_root_session_aliases_are_unique_resolvable_and_independently_removable() -> None:+ registry = VirtualTargetRegistry()+ registry.register_tab(7, "about:blank", "")+ primary_session_id = registry.root_session_id(7)++ first_alias = registry.create_root_session_alias(7)+ second_alias = registry.create_root_session_alias(7)++ assert len({primary_session_id, first_alias, second_alias}) == 3+ assert registry.root_session_ids(7) == [primary_session_id, first_alias, second_alias]+ assert registry.resolve_session(first_alias) == (7, None)+ assert registry.resolve_session(second_alias) == (7, None)++ assert registry.remove_root_session_alias(first_alias)+ assert not registry.remove_root_session_alias(primary_session_id)+ assert registry.root_session_ids(7) == [primary_session_id, second_alias]+ with pytest.raises(KeyError):+ registry.resolve_session(first_alias)+++def test_browser_session_aliases_are_unique_and_independently_removable() -> None:+ registry = VirtualTargetRegistry()++ first_alias = registry.create_browser_session_alias()+ second_alias = registry.create_browser_session_alias()++ assert first_alias != second_alias+ assert registry.is_browser_session_alias(first_alias)+ assert registry.is_browser_session_alias(second_alias)+ assert registry.remove_browser_session_alias(first_alias)+ assert not registry.is_browser_session_alias(first_alias)+ assert registry.is_browser_session_alias(second_alias)+++def test_child_session_register_resolve_and_remove() -> None:+ registry = VirtualTargetRegistry()+ registry.register_tab(9, "https://example.com", "Example")+ child_target_info = {+ "targetId": "frame-1",+ "type": "iframe",+ "title": "",+ "url": "https://example.com/frame",+ "attached": True,+ "canAccessOpener": False,+ "browserContextId": "skyvern-default",+ }++ registry.register_child_session(9, "chrome-child-1", child_target_info)++ assert registry.resolve_session(registry.root_session_id(9)) == (9, None)+ assert registry.resolve_session("chrome-child-1") == (9, "chrome-child-1")+ assert registry.target_info("frame-1") == child_target_info++ registry.remove_child_session("chrome-child-1")++ with pytest.raises(KeyError):+ registry.resolve_session("chrome-child-1")+ with pytest.raises(KeyError):+ registry.target_info("frame-1")+++def test_remove_tab_removes_root_and_child_sessions() -> None:+ registry = VirtualTargetRegistry()+ target_id = registry.register_tab(5, "https://example.com", "Example")+ root_session_id = registry.root_session_id(5)+ root_alias = registry.create_root_session_alias(5)+ registry.register_child_session(5, "chrome-child", {"targetId": "frame-5"})++ registry.remove_tab(5)++ for session_id in (root_session_id, root_alias, "chrome-child"):+ with pytest.raises(KeyError):+ registry.resolve_session(session_id)+ with pytest.raises(KeyError):+ registry.target_info(target_id)+ with pytest.raises(KeyError):+ registry.tab_for_target(target_id)+++def test_unknown_registry_entries_raise_key_error() -> None:+ registry = VirtualTargetRegistry()++ with pytest.raises(KeyError):+ registry.root_session_id(99)+ with pytest.raises(KeyError):+ registry.resolve_session("missing-session")+ with pytest.raises(KeyError):+ registry.target_info("missing-target")+ with pytest.raises(KeyError):+ registry.tab_for_target("missing-target")+ with pytest.raises(KeyError):+ registry.update_tab(99, "about:blank", "")+ with pytest.raises(KeyError):+ registry.register_child_session(99, "missing-child", {"targetId": "frame-missing"})+++def test_clear_removes_all_registry_state() -> None:+ registry = VirtualTargetRegistry()+ target_id = registry.register_tab(3, "https://example.com", "Example")+ root_session_id = registry.root_session_id(3)+ root_alias = registry.create_root_session_alias(3)+ browser_alias = registry.create_browser_session_alias()+ registry.register_child_session(3, "chrome-child", {"targetId": "frame-3"})++ registry.clear()++ assert registry.list_page_targets() == []+ with pytest.raises(KeyError):+ registry.resolve_session(root_session_id)+ with pytest.raises(KeyError):+ registry.resolve_session(root_alias)+ with pytest.raises(KeyError):+ registry.resolve_session("chrome-child")+ with pytest.raises(KeyError):+ registry.tab_for_target(target_id)+ assert not registry.is_browser_session_alias(browser_alias)diff --git a/tests/unit/test_mcp_browser_tools.py b/tests/unit/test_mcp_browser_tools.pyindex 7bbcd3539..38a57ee29 100644--- a/tests/unit/test_mcp_browser_tools.py+++ b/tests/unit/test_mcp_browser_tools.py@@ -24,11 +24,13 @@ from skyvern.cli.mcp_tools import cdp_input as mcp_cdp_inputfrom skyvern.cli.mcp_tools import mcpfrom skyvern.client.errors import InternalServerError, UnprocessableEntityErrorfrom tests.unit._mcp_browser_fakes import (+ make_mock_page,make_probe_locator,make_real_wait_for_timeout,make_select_like_page,make_select_option_page,make_skyvern_page,+ patch_get_page,)@@ -205,11 +207,11 @@ async def test_skyvern_run_task_timeout_returns_timeout_code(monkeypatch: pytestasync def test_browser_tool_targeting_schema_prefers_direct_params() -> None:tools_by_name = {tool.name: tool for tool in await mcp.list_tools()}expectations = {- "skyvern_click": (("selector",), ("intent",)),+ "skyvern_click": (("selector", "x", "y"), ("intent",)),"skyvern_drag": (("source_selector", "target_selector"), ("source_intent", "target_intent")),"skyvern_file_upload": (("selector",), ("intent",)),"skyvern_hover": (("selector",), ("intent",)),- "skyvern_type": (("selector",), ("intent",)),+ "skyvern_type": (("selector", "x", "y"), ("intent",)),"skyvern_screenshot": (("selector",), ()),"skyvern_scroll": (("selector",), ("intent",)),"skyvern_select_option": (("selector",), ("intent",)),@@ -297,6 +299,212 @@ def _direct_type_page(return fill, locator+async def test_skyvern_click_coordinates_dispatches_raw_mouse_click(monkeypatch: pytest.MonkeyPatch) -> None:+ page = make_mock_page()+ page.mouse.click = AsyncMock()+ context = BrowserContext(mode="local")+ patch_get_page(monkeypatch, mcp_browser, page, context)+ resolve_ai_mode = Mock(side_effect=AssertionError("coordinate clicks must not resolve AI mode"))+ select_native_option = AsyncMock(side_effect=AssertionError("coordinate clicks must not resolve selectors"))+ monkeypatch.setattr(mcp_browser, "_resolve_ai_mode", resolve_ai_mode)+ monkeypatch.setattr(mcp_browser, "select_native_option_if_targeted", select_native_option)++ result = await mcp_browser.skyvern_click(x=12.5, y=24.0, button="right", click_count=2)++ assert result["ok"] is True, result+ page.mouse.click.assert_awaited_once_with(12.5, 24.0, button="right", click_count=2)+ assert result["data"]["resolved_target"] == "coordinates (12.5, 24.0)"+ assert result["data"]["sdk_equivalent"] == ("await page.mouse.click(12.5, 24.0, button='right', click_count=2)")+ resolve_ai_mode.assert_not_called()+ select_native_option.assert_not_awaited()+++async def test_skyvern_type_coordinates_focuses_then_types(monkeypatch: pytest.MonkeyPatch) -> None:+ page = make_mock_page()+ page.mouse.click = AsyncMock()+ page.keyboard.type = AsyncMock()+ page.keyboard.press = AsyncMock()+ page.evaluate = AsyncMock(return_value=False)+ context = BrowserContext(mode="local")+ patch_get_page(monkeypatch, mcp_browser, page, context)++ result = await mcp_browser.skyvern_type(text="Noor", x=18.0, y=36.5, clear=False)++ assert result["ok"] is True, result+ page.mouse.click.assert_awaited_once_with(18.0, 36.5)+ page.keyboard.type.assert_awaited_once_with("Noor")+ page.keyboard.press.assert_not_awaited()+ assert result["data"]["resolved_target"] == "coordinates (18.0, 36.5)"+++async def test_skyvern_type_coordinates_refuses_password_input(monkeypatch: pytest.MonkeyPatch) -> None:+ page = make_mock_page()+ page.mouse.click = AsyncMock()+ page.keyboard.type = AsyncMock()+ page.keyboard.press = AsyncMock()+ page.evaluate = AsyncMock(return_value=True)+ context = BrowserContext(mode="local")+ patch_get_page(monkeypatch, mcp_browser, page, context)++ result = await mcp_browser.skyvern_type(text="not-a-secret", x=18.0, y=36.5, clear=False)++ assert result["ok"] is False+ assert result["error"]["code"] == mcp_browser.ErrorCode.INVALID_INPUT+ assert "password" in result["error"]["message"].lower()+ page.keyboard.type.assert_not_awaited()+++async def test_skyvern_type_coordinates_refuses_unverifiable_target(monkeypatch: pytest.MonkeyPatch) -> None:+ page = make_mock_page()+ page.mouse.click = AsyncMock()+ page.keyboard.type = AsyncMock()+ page.keyboard.press = AsyncMock()+ page.evaluate = AsyncMock(return_value=None)+ context = BrowserContext(mode="local")+ patch_get_page(monkeypatch, mcp_browser, page, context)++ result = await mcp_browser.skyvern_type(text="Noor", x=18.0, y=36.5, clear=False)++ assert result["ok"] is False+ assert result["error"]["code"] == mcp_browser.ErrorCode.INVALID_INPUT+ assert result["error"]["message"] == browser_ops.COORDINATE_TYPE_TARGET_REFUSAL_MESSAGE+ page.keyboard.type.assert_not_awaited()+++async def test_skyvern_click_requires_both_coordinates_before_session(monkeypatch: pytest.MonkeyPatch) -> None:+ get_page = AsyncMock(side_effect=AssertionError("get_page should not be called for invalid coordinates"))+ monkeypatch.setattr(mcp_browser, "get_page", get_page)++ result = await mcp_browser.skyvern_click(x=12.5)++ assert result["ok"] is False+ assert result["error"]["code"] == mcp_browser.ErrorCode.INVALID_INPUT+ assert "x" in result["error"]["message"] and "y" in result["error"]["message"]+ get_page.assert_not_awaited()+++ ("x", "y"),+ [+ pytest.param(-1.0, 0.0, id="negative"),+ pytest.param(float("inf"), 0.0, id="infinite"),+ pytest.param(float("nan"), 0.0, id="nan"),+ ],+)+async def test_skyvern_click_rejects_invalid_coordinates_before_session(+ monkeypatch: pytest.MonkeyPatch,+ x: float,+ y: float,+) -> None:+ get_page = AsyncMock(side_effect=AssertionError("get_page should not be called for invalid coordinates"))+ monkeypatch.setattr(mcp_browser, "get_page", get_page)++ result = await mcp_browser.skyvern_click(x=x, y=y)++ assert result["ok"] is False+ assert result["error"]["code"] == mcp_browser.ErrorCode.INVALID_INPUT+ get_page.assert_not_awaited()+++async def test_skyvern_type_rejects_selector_with_coordinates_before_session(monkeypatch: pytest.MonkeyPatch) -> None:+ get_page = AsyncMock(side_effect=AssertionError("get_page should not be called for conflicting targets"))+ monkeypatch.setattr(mcp_browser, "get_page", get_page)++ result = await mcp_browser.skyvern_type(text="Noor", selector="#first_name", x=18.0, y=36.5)++ assert result["ok"] is False+ assert result["error"]["code"] == mcp_browser.ErrorCode.INVALID_INPUT+ assert "selector" in result["error"]["message"].lower()+ assert "coordinates" in result["error"]["message"].lower()+ get_page.assert_not_awaited()+++async def test_skyvern_execute_click_coordinates_flow_through_dispatch(monkeypatch: pytest.MonkeyPatch) -> None:+ page = make_mock_page()+ page.mouse.click = AsyncMock()+ context = BrowserContext(mode="local")+ get_page = patch_get_page(monkeypatch, mcp_browser, page, context)+ get_page.return_value[0]._working_frame = None++ result = await mcp_browser.skyvern_execute(+ steps=[{"tool": "click", "params": {"x": 12.5, "y": 24.0, "button": "middle"}}]+ )++ assert result["ok"] is True, result+ assert result["data"]["steps_completed"] == 1+ page.mouse.click.assert_awaited_once_with(12.5, 24.0, button="middle", click_count=1)+++async def test_skyvern_execute_type_coordinates_flow_through_dispatch(monkeypatch: pytest.MonkeyPatch) -> None:+ page = make_mock_page()+ page.mouse.click = AsyncMock()+ page.keyboard.type = AsyncMock()+ page.keyboard.press = AsyncMock()+ page.evaluate = AsyncMock(return_value=False)+ context = BrowserContext(mode="local")+ get_page = patch_get_page(monkeypatch, mcp_browser, page, context)+ get_page.return_value[0]._working_frame = None++ result = await mcp_browser.skyvern_execute(+ steps=[+ {+ "tool": "type",+ "params": {+ "text": "Noor",+ "x": 18.0,+ "y": 36.5,+ "clear_first": False,+ "press_enter": True,+ },+ }+ ]+ )++ assert result["ok"] is True, result+ page.mouse.click.assert_awaited_once_with(18.0, 36.5)+ page.keyboard.type.assert_awaited_once_with("Noor")+ page.keyboard.press.assert_awaited_once_with("Enter")+++async def test_coordinate_actions_record_explicit_trajectory_coordinates(monkeypatch: pytest.MonkeyPatch) -> None:+ page = make_mock_page()+ page.mouse.click = AsyncMock()+ page.keyboard.type = AsyncMock()+ page.keyboard.press = AsyncMock()+ page.evaluate = AsyncMock(return_value=False)+ context = BrowserContext(mode="cloud_session", session_id="pbs_test")+ patch_get_page(monkeypatch, mcp_browser, page, context)+ append_trajectory = Mock()+ monkeypatch.setattr(mcp_browser, "append_trajectory_entry", append_trajectory)+ monkeypatch.setattr(mcp_browser, "current_api_key_hash", lambda: None)++ click_result = await mcp_browser.skyvern_click(x=12.5, y=24.0)+ type_result = await mcp_browser.skyvern_type(text="Noor", x=18.0, y=36.5)++ assert click_result["ok"] is True, click_result+ assert type_result["ok"] is True, type_result+ click_entry, type_entry = (entry.kwargs["entry"] for entry in append_trajectory.call_args_list)+ assert "selector" not in click_entry+ assert (click_entry["x"], click_entry["y"]) == (12.5, 24.0)+ assert click_entry["sdk_equivalent"] == click_result["data"]["sdk_equivalent"]+ assert "locator" not in click_entry["sdk_equivalent"]+ assert "selector" not in type_entry+ assert (type_entry["x"], type_entry["y"]) == (18.0, 36.5)+ assert type_entry["sdk_equivalent"] == type_result["data"]["sdk_equivalent"]+ assert "locator" not in type_entry["sdk_equivalent"]+ assert "coordinates (" not in repr([click_entry, type_entry])++# Default (resilient) keeps the shared MCP surface unchanged for external callers: a selector# tries first, then dismisses overlays and AI-falls-back. selector_mode="direct" (which the# copilot binds via forced_args) opts into deterministic, no-AI behavior.