Skip to content

Automate browser based workflows with AI

Remove the CDP proxy service and proxy configuration from the package (#7583)

Aaron Pereza20c56f1e9ae
diff --git a/docs/developers/browser-automations/handle-browsers.mdx b/docs/developers/browser-automations/handle-browsers.mdx
index 527bffe3c..22de8db35 100644
--- a/docs/developers/browser-automations/handle-browsers.mdx
+++ b/docs/developers/browser-automations/handle-browsers.mdx
@@ -226,8 +226,6 @@ Available locations:
| `RESIDENTIAL_CA` | Canada |
| `NONE` | No proxy (direct connection) |
-For self-hosted proxy configuration, see [Proxy Setup](/developers/self-hosted/proxy).
-
---
## Session persistence
diff --git a/docs/developers/features/proxy-and-geo-targeting.mdx b/docs/developers/features/proxy-and-geo-targeting.mdx
index c2e8358d3..7b8aecf07 100644
--- a/docs/developers/features/proxy-and-geo-targeting.mdx
+++ b/docs/developers/features/proxy-and-geo-targeting.mdx
@@ -53,10 +53,4 @@ Region-locked pricing, inventory, or UX becomes visible when you route through t
>
Lock a proxy location across multiple operations in one session.
</Card>
- <Card
- title="Self-hosted proxy"
- href="/developers/self-hosted/proxy"
- >
- Bring your own proxy on self-hosted deployments.
- </Card>
</CardGroup>
diff --git a/docs/developers/self-hosted/browser.mdx b/docs/developers/self-hosted/browser.mdx
index 9c8027b15..6e507964f 100644
--- a/docs/developers/self-hosted/browser.mdx
+++ b/docs/developers/self-hosted/browser.mdx
@@ -435,9 +435,6 @@ BROWSER_REMOTE_DEBUGGING_URL=http://browserless:3000/
## Next steps
<CardGroup cols={2}>
- <Card title="Proxy Setup" icon="shield" href="/developers/self-hosted/proxy">
- Configure proxies to avoid bot detection
- </Card>
<Card title="Storage Configuration" icon="hard-drive" href="/developers/self-hosted/storage">
Store recordings and artifacts in S3 or Azure Blob
</Card>
diff --git a/docs/developers/self-hosted/docker.mdx b/docs/developers/self-hosted/docker.mdx
index 040a136d6..a4fa35bb3 100644
--- a/docs/developers/self-hosted/docker.mdx
+++ b/docs/developers/self-hosted/docker.mdx
@@ -398,7 +398,4 @@ The page content is too large for the LLM. Try:
<Card title="Storage Configuration" icon="hard-drive" href="/developers/self-hosted/storage">
Store artifacts in S3 or Azure Blob instead of local disk
</Card>
- <Card title="Proxy Setup" icon="shield" href="/developers/self-hosted/proxy">
- Configure proxies to avoid bot detection
- </Card>
</CardGroup>
diff --git a/docs/developers/self-hosted/proxy.mdx b/docs/developers/self-hosted/proxy.mdx
deleted file mode 100644
index 9b4d0347f..000000000
--- a/docs/developers/self-hosted/proxy.mdx
+++ /dev/null
@@ -1,245 +0,0 @@
----
-title: Proxy Setup
-subtitle: Configure proxies to avoid bot detection
-description: Configure residential, ISP, or datacenter proxies for self-hosted Skyvern deployments to avoid bot detection, bypass geo-restrictions, and prevent rate limiting on target websites.
-slug: developers/self-hosted/proxy
-keywords:
- - residential proxy
- - ISP proxy
- - datacenter proxy
- - rate limiting
- - geo-restriction
- - PROXY_URL
- - configuration
----
-
-Many websites block requests from datacenter IPs or detect automated browser patterns. Skyvern Cloud includes managed residential proxies that handle this automatically. Self-hosted deployments require you to configure your own proxy provider.
-
-## Why you need proxies
-
-Without proxies, your browser automation traffic originates from your server's IP address. This causes issues when:
-
-- **Target sites block datacenter IPs**: Many sites automatically block traffic from known hosting providers (AWS, GCP, Azure)
-- **Rate limiting**: Repeated requests from one IP trigger rate limits
-- **Geo-restrictions**: Sites serve different content based on location
-- **Bot detection**: Some sites fingerprint datacenter traffic patterns
-
-<Note>
-If you're automating internal tools or sites that don't have bot detection, you may not need proxies at all. Test without proxies first.
-</Note>
-
----
-
-## Proxy types
-
-### Residential proxies
-
-Traffic appears to come from real home internet connections. Most expensive but least likely to be blocked. Recommended for browser automation. Start here unless cost is a primary concern.
-
-**Providers:**
-- [Bright Data](https://brightdata.com/)
-- [Oxylabs](https://oxylabs.io/)
-- [Smartproxy](https://smartproxy.com/)
-- [IPRoyal](https://iproyal.com/)
-
-### ISP proxies
-
-Static IPs from internet service providers. Good balance between cost and detection avoidance.
-
-### Datacenter proxies
-
-IPs from cloud providers. Cheapest but most likely to be blocked.
-
-### Rotating vs. static
-
-See [Rotating proxies vs. sticky sessions](#rotating-proxies-vs-sticky-sessions) for guidance on which to use.
-
----
-
-## Configuration
-
-Skyvern supports proxy configuration at the browser level through Playwright.
-
-### Environment variable approach
-
-Set proxy configuration in your `.env` file:
-
-```bash .env
-ENABLE_PROXY=true
-
-# Single proxy
-HOSTED_PROXY_POOL=http://user:[email protected]:8080
-
-# Multiple proxies: Skyvern randomly selects one per browser session
-HOSTED_PROXY_POOL=http://user:[email protected]:8080,http://user:[email protected]:8080
-```
-
-<Note>
-Skyvern Cloud supports a `proxy_location` parameter on task requests for geographic targeting (e.g., `RESIDENTIAL_US`). This feature is not available in self-hosted deployments. All tasks use the proxy configured in `HOSTED_PROXY_POOL`.
-</Note>
-
----
-
-## Setting up a proxy provider
-
-### Step 1: Choose a provider
-
-For browser automation, residential proxies work best. See [proxy types](#proxy-types) above.
-
-### Step 2: Configure Skyvern
-
-Add your proxy to the environment:
-
-```bash .env
-ENABLE_PROXY=true
-HOSTED_PROXY_POOL=http://username:[email protected]:8080
-```
-
-### Step 3: Test the connection
-
-Run a simple task that checks your IP:
-
-```bash
-curl -s http://localhost:8000/v1/tasks \
- -H "x-api-key: YOUR_API_KEY" \
- -H "Content-Type: application/json" \
- -d '{
- "prompt": "What is the IP address shown on this page?",
- "url": "https://whatismyipaddress.com"
- }'
-```
-
-The task result should show an IP from your proxy provider, not your server's IP.
-
----
-
-## Proxy authentication methods
-
-### Basic auth (most common)
-
-Include credentials in the URL:
-
-```bash
-http://username:[email protected]:8080
-```
-
-### IP whitelist
-
-Some providers allow you to whitelist your server's IP instead of using credentials:
-
-1. Get your server's public IP: `curl ifconfig.me`
-2. Add it to your proxy provider's whitelist
-3. Use the proxy without credentials:
-
-```bash
-http://proxy.example.com:8080
-```
-
----
-
-## Geographic targeting
-
-If your proxy provider supports geographic targeting, configure it in your proxy URL. The exact format depends on the provider.
-
-### Bright Data example
-
-```bash
-# Target US residential
-http://user-country-us:[email protected]:8080
-
-# Target specific US state
-http://user-country-us-state-california:[email protected]:8080
-```
-
-### Oxylabs example
-
-```bash
-# Target UK
-http://user-country-gb:[email protected]:8080
-```
-
-Check your provider's documentation for the exact format.
-
----
-
-## Rotating proxies vs. sticky sessions
-
-### Rotating (new IP per request)
-
-Good for:
-- High-volume scraping
-- Avoiding per-IP rate limits
-- Tasks that don't need session persistence
-
-### Sticky sessions (same IP for duration)
-
-Good for:
-- Multi-step automations where the site tracks your session
-- Login flows
-- Sites that block IP changes mid-session
-
-Most providers support sticky sessions via a session ID parameter:
-
-```bash
-# Bright Data sticky session
-http://user-session-abc123:[email protected]:8080
-```
-
----
-
-## Troubleshooting
-
-### "Connection refused" or timeout errors
-
-- Verify your proxy endpoint and credentials are correct
-- Check if your server can reach the proxy: `curl -x http://user:pass@proxy:port http://example.com`
-- Ensure your provider hasn't blocked your IP
-
-### Target site still blocking requests
-
-- Try a different proxy location
-- Use residential instead of datacenter proxies
-- Enable sticky sessions if the site tracks session changes
-- Verify the proxy is actually being used (check the IP)
-
-### Slow performance
-
-- Proxy overhead adds 100-500ms per request
-- Choose a proxy location geographically close to the target site
-- Use datacenter proxies for sites that allow them (faster than residential)
-
-### High proxy costs
-
-Residential proxy bandwidth is expensive. To reduce costs:
-- Disable video recording (reduces bandwidth)
-- Use datacenter proxies for sites that allow them
-- Cache resources where possible
-- Minimize unnecessary page loads
-
----
-
-## Running without proxies
-
-For internal tools or development, proxies aren't always necessary:
-
-```bash .env
-ENABLE_PROXY=false
-```
-
-Your browser traffic will originate directly from your server's IP. This works well for:
-- Internal applications
-- Development and testing
-- Sites that don't block datacenter traffic
-
----
-
-## Next steps
-
-<CardGroup cols={2}>
- <Card title="Storage Configuration" icon="hard-drive" href="/developers/self-hosted/storage">
- Store recordings and artifacts in S3 or Azure Blob
- </Card>
- <Card title="Kubernetes Deployment" icon="dharmachakra" href="/developers/self-hosted/kubernetes">
- Deploy Skyvern at scale with Kubernetes
- </Card>
-</CardGroup>
diff --git a/docs/docs.json b/docs/docs.json
index 953758b48..e435b6da2 100644
--- a/docs/docs.json
+++ b/docs/docs.json
@@ -209,7 +209,6 @@
"developers/self-hosted/docker",
"developers/self-hosted/llm-configuration",
"developers/self-hosted/browser",
- "developers/self-hosted/proxy",
"developers/self-hosted/kubernetes",
"developers/self-hosted/storage",
"developers/self-hosted/google-integrations",
diff --git a/docs/llms.txt b/docs/llms.txt
index cd3d070ab..31997ac89 100644
--- a/docs/llms.txt
+++ b/docs/llms.txt
@@ -62,7 +62,6 @@
- [Docker Setup](https://skyvern.com/docs/developers/self-hosted/docker.md): Get Skyvern running in 10 minutes with Docker Compose
- [LLM Configuration](https://skyvern.com/docs/developers/self-hosted/llm-configuration.md): Connect OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Ollama, or any OpenAI-compatible LLM provider
- [Browser Configuration](https://skyvern.com/docs/developers/self-hosted/browser.md): Configure browser modes, display settings, headless vs headed, and external Chrome connections via Playwright
-- [Proxy Setup](https://skyvern.com/docs/developers/self-hosted/proxy.md): Configure residential proxies and geolocation for bot detection avoidance in self-hosted deployments
- [Kubernetes Deployment](https://skyvern.com/docs/developers/self-hosted/kubernetes.md): Deploy Skyvern at scale with Kubernetes manifests and Helm charts
- [Storage Configuration](https://skyvern.com/docs/developers/self-hosted/storage.md): Configure where Skyvern stores artifacts, recordings, and screenshots; local filesystem or S3-compatible storage
- [Local LLMs](https://skyvern.com/docs/integrations/local-llms.md): Run Skyvern with Ollama, LiteLLM, or any OpenAI-compatible endpoint for self-hosted LLM inference
diff --git a/pyproject.toml b/pyproject.toml
index 25af8934c..2c88baa06 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -357,47 +357,5 @@ skip = ["webeye/actions/__init__.py", "forge/sdk/__init__.py"]
[tool.mypy]
plugins = "sqlalchemy.ext.mypy.plugin"
-[tool.importlinter]
-root_packages = ["skyvern"]
-include_external_packages = true
-
-[[tool.importlinter.contracts]]
-name = "proxy core and ports import no framework or I/O libraries"
-type = "forbidden"
-source_modules = ["skyvern.proxy.core", "skyvern.proxy.ports"]
-forbidden_modules = [
- "aiohttp",
- "fastapi",
- "httpx",
- "playwright",
- "redis",
- "requests",
- "sqlalchemy",
- "starlette",
- "structlog",
- "uvicorn",
- "websockets",
-]
-
-[[tool.importlinter.contracts]]
-name = "proxy adapters depend on ports and core, never the reverse"
-type = "layers"
-layers = ["skyvern.proxy.adapters", "skyvern.proxy.ports", "skyvern.proxy.core"]
-
-[[tool.importlinter.contracts]]
-name = "proxy never imports runner code or playwright"
-type = "forbidden"
-source_modules = ["skyvern.proxy"]
-forbidden_modules = [
- "cloud",
- "codeblock",
- "playwright",
- "skyvern.cli",
- "skyvern.config",
- "skyvern.forge",
- "skyvern.webeye",
- "workers",
-]
-
[project.scripts]
skyvern = "skyvern.__main__:main"
diff --git a/skyvern-frontend/src/api/types.ts b/skyvern-frontend/src/api/types.ts
index 782720309..a8fc75b26 100644
--- a/skyvern-frontend/src/api/types.ts
+++ b/skyvern-frontend/src/api/types.ts
@@ -701,14 +701,59 @@ export interface EvalBrowserSession extends Eval {
arm?: string | null;
model?: string | null;
task_id?: string | null;
+ task_title?: string | null;
perfect?: boolean | null;
rubric_avg?: number | null;
+ started_at?: string | null;
+ finished_at?: string | null;
+ duration_s?: number | null;
}
export type EvalApiResponse = Array<
EvalWorkflow | EvalTask | EvalBrowserSession
>;
+export interface EvalModelSummary {
+ model: string | null;
+ total: number;
+ perfect_count: number;
+ graded_count: number;
+ pass_rate: number | null;
+ rubric_avg: number | null;
+ duration_avg_s: number | null;
+ last_started_at: string | null;
+}
+
+export interface EvalSummaryResponse {
+ models: EvalModelSummary[];
+ total_runs: number;
+}
+
+export interface RuntimeCostTierMetrics {
+ steps: number | null;
+ duration_min: number | null;
+ tokens: number | null;
+ cost: number | null;
+}
+
+export interface RuntimeCostModelSummary {
+ model: string;
+ runs: number;
+ steps: number | null;
+ duration_min: number | null;
+ tokens: number | null;
+ output_tokens: number | null;
+ cost: number | null;
+ token_coverage: number;
+ tiers: Partial<Record<"easy" | "medium" | "hard", RuntimeCostTierMetrics>>;
+}
+
+export interface RuntimeCostSummaryResponse {
+ task_set_size: number;
+ captured_at: string | null;
+ models: RuntimeCostModelSummary[];
+}
+
export type DebugSessionApiResponse = {
debug_session_id: string;
browser_session_id: string;
diff --git a/skyvern-frontend/src/util/timeFormat.test.ts b/skyvern-frontend/src/util/timeFormat.test.ts
index a4dcdebc4..e36c90aed 100644
--- a/skyvern-frontend/src/util/timeFormat.test.ts
+++ b/skyvern-frontend/src/util/timeFormat.test.ts
@@ -4,6 +4,7 @@ import { describe, test, expect } from "vitest";
import {
basicLocalTimeFormat,
basicTimeFormat,
+ formatDurationSeconds,
formatExecutionTime,
normalizeUtcTimestamp,
} from "./timeFormat";
@@ -115,6 +116,17 @@ describe("formatExecutionTime", () => {
});
});
+describe("formatDurationSeconds", () => {
+ test("formats valid durations and leaves unavailable values blank", () => {
+ expect(formatDurationSeconds(42.4)).toBe("42s");
+ expect(formatDurationSeconds(125.2)).toBe("2m 5s");
+ expect(formatDurationSeconds(119.6)).toBe("2m 0s");
+ expect(formatDurationSeconds(59.6)).toBe("1m 0s");
+ expect(formatDurationSeconds(null)).toBe("—");
+ expect(formatDurationSeconds(-1)).toBe("—");
+ });
+});
+
describe("basicTimeFormat", () => {
test("appends UTC label to the formatted string", () => {
const result = basicTimeFormat("2026-03-04T18:30:00Z");
diff --git a/skyvern-frontend/src/util/timeFormat.ts b/skyvern-frontend/src/util/timeFormat.ts
index bdb0428db..0484d9211 100644
--- a/skyvern-frontend/src/util/timeFormat.ts
+++ b/skyvern-frontend/src/util/timeFormat.ts
@@ -134,6 +134,19 @@ function formatExecutionTime(
return `${seconds}s`;
}
+function formatDurationSeconds(seconds: number | null | undefined): string {
+ if (seconds == null || seconds < 0 || !Number.isFinite(seconds)) {
+ return "—";
+ }
+
+ const total = Math.round(seconds);
+ if (total < 60) {
+ return `${total}s`;
+ }
+
+ return `${Math.floor(total / 60)}m ${total % 60}s`;
+}
+
export {
normalizeUtcTimestamp,
basicLocalTimeFormat,
@@ -143,4 +156,5 @@ export {
localTimeFormatWithShortDate,
formatTimeRemaining,
formatExecutionTime,
+ formatDurationSeconds,
};
diff --git a/skyvern/cli/run_commands.py b/skyvern/cli/run_commands.py
index 007fa0225..1e997e2e7 100644
--- a/skyvern/cli/run_commands.py
+++ b/skyvern/cli/run_commands.py
@@ -1,3 +1,5 @@
+from __future__ import annotations
+
import _thread
import asyncio
import atexit
@@ -10,12 +12,15 @@ import signal
import subprocess
import sys
import threading
-from typing import TYPE_CHECKING, Annotated, Any, List, Literal, Optional, cast
+from typing import TYPE_CHECKING, Annotated, Any, Literal, cast
if TYPE_CHECKING:
from starlette.types import ASGIApp, Receive, Scope, Send
+ from skyvern.library.local_browser_profile import LocalBrowserProfile
+
import psutil
+import structlog
import typer
import uvicorn
from dotenv import set_key
@@ -38,11 +43,11 @@ from skyvern.utils.env_paths import (
)
run_app = typer.Typer(help="Commands to run Skyvern services such as the API server or UI.")
+LOG = structlog.get_logger(__name__)
_mcp_cleanup_done = False
_mcp_cleanup_in_progress = False
_mcp_eof_shutdown_requested = False
_MCP_GRACEFUL_CLEANUP_TIMEOUT_SECONDS = 5.0
-_MCP_PROCESS_KILL_TIMEOUT_SECONDS = 2.0
_MCP_NATIVE_EOF_GRACE_SECONDS = 0.25
# The EOF watcher's os._exit(0) preempts cleanup unconditionally, so this must exceed
# the cloud path's graceful join plus process-kill wait.
@@ -76,25 +81,18 @@ def _cleanup_mcp_resources_blocking() -> None:
_mcp_cleanup_in_progress = True
try:
- logger = logging.getLogger(__name__)
try:
local_browser_identity = _current_local_browser_identity()
except Exception:
- logger.warning("Failed to identify the local MCP browser", exc_info=True)
+ LOG.warning("Failed to identify the local MCP browser", exc_info=True)
local_browser_identity = None
# In stdio, this main-thread read and a cleanup thread both resolve _global_session because
# ContextVars are not inherited. The exclusive mode means local identity cannot hide a cloud close.
if local_browser_identity is not None:
try:
- # Playwright objects belong to mcp.run's closed loop; a fresh-loop close hangs forever.
- # The owned profile is an anonymous mkdtemp throwaway, so clean it up directly.
- try:
- _kill_local_browser_process_tree(local_browser_identity[0], local_browser_identity[1])
- finally:
- if local_browser_identity[2] and local_browser_identity[1]:
- shutil.rmtree(local_browser_identity[1], ignore_errors=True)
+ _cleanup_local_browser_from_identity(local_browser_identity, event_prefix="mcp_local_browser")
except Exception:
- logger.warning("MCP local browser cleanup failed", exc_info=True)
+ LOG.warning("MCP local browser cleanup failed", exc_info=True)
else:
cleanup_errors: list[BaseException] = []
@@ -108,10 +106,10 @@ def _cleanup_mcp_resources_blocking() -> None:
cleanup_thread.start()
cleanup_thread.join(_MCP_GRACEFUL_CLEANUP_TIMEOUT_SECONDS)
if cleanup_thread.is_alive():
- logger.warning("MCP graceful cleanup timed out")
+ LOG.warning("MCP graceful cleanup timed out")
elif cleanup_errors:
error = cleanup_errors[0]
- logger.warning("MCP graceful cleanup failed", exc_info=(type(error), error, error.__traceback__))
+ LOG.warning("MCP graceful cleanup failed", exc_info=(type(error), error, error.__traceback__))
finally:
_mcp_cleanup_done = True
_mcp_cleanup_in_progress = False
@@ -122,68 +120,32 @@ def _cleanup_mcp_resources_sync() -> None:
_cleanup_mcp_resources_blocking()
-def _current_local_browser_identity() -> tuple[int | None, str | None, bool] | None:
+def _current_local_browser_identity() -> tuple[str | None, bool, LocalBrowserProfile | None] | None:
from skyvern.cli.core.session_manager import get_current_session # noqa: PLC0415
current = get_current_session()
if current.context is None or current.context.mode != "local" or current.browser is None:
return None
return (
- current.browser.local_cdp_port,
current.browser.local_user_data_dir,
current.browser.local_user_data_dir_owned,
+ current.browser.local_browser_profile,
)
-def _find_local_browser_processes(port: int | None, user_data_dir: str | None) -> list[psutil.Process]:
- if user_data_dir is None:
- return []
-
- processes: dict[int, psutil.Process] = {}
- user_data_arg = f"--user-data-dir={user_data_dir}"
- try:
- for process in psutil.process_iter(["pid", "cmdline"]):
- try:
- if user_data_arg in ((process.info or {}).get("cmdline") or []):
- processes[process.pid] = process
- except (psutil.NoSuchProcess, psutil.AccessDenied, psutil.ZombieProcess):
- pass
- except Exception:
- logging.getLogger(__name__).warning("Failed to inspect local browser processes", exc_info=True)
- return list(processes.values())
-
-
-def _local_browser_process_tree(port: int | None, user_data_dir: str | None) -> list[psutil.Process]:
- processes: dict[int, psutil.Process] = {}
- # The Playwright node driver is Chromium's parent, not its descendant; stdin EOF reaps that driver.
- for root in _find_local_browser_processes(port, user_data_dir):
- processes[root.pid] = root
- try:
- processes.update({process.pid: process for process in root.children(recursive=True)})
- except (psutil.NoSuchProcess, psutil.AccessDenied, psutil.ZombieProcess):
- pass
- return list(processes.values())
-
-
-def _kill_local_browser_process_tree(
- port: int | None,
- user_data_dir: str | None,
+def _cleanup_local_browser_from_identity(
+ identity: tuple[str | None, bool, LocalBrowserProfile | None],
*,
- known_processes: list[psutil.Process] | None = None,
+ event_prefix: str,
) -> None:
- if user_data_dir is None:
- return
+ from skyvern.library import local_browser_profile # noqa: PLC0415
- processes = {process.pid: process for process in known_processes or []}
- processes.update({process.pid: process for process in _local_browser_process_tree(port, user_data_dir)})
-
- for process in processes.values():
- try:
- process.kill()
- except (psutil.NoSuchProcess, psutil.AccessDenied, psutil.ZombieProcess):
- pass
- if processes:
- psutil.wait_procs(list(processes.values()), timeout=_MCP_PROCESS_KILL_TIMEOUT_SECONDS)
+ user_data_dir, owned, profile = identity
+ if owned and user_data_dir:
+ if not local_browser_profile.cleanup_local_browser_profile(profile or user_data_dir):
+ LOG.warning(f"{event_prefix}_profile_cleanup_deferred", user_data_dir=user_data_dir)
+ elif user_data_dir and not local_browser_profile.terminate_local_browser_processes(user_data_dir):
+ LOG.warning(f"{event_prefix}_termination_incomplete", user_data_dir=user_data_dir)
def _watch_stdin_eof(
@@ -207,13 +169,9 @@ def _watch_stdin_eof(
try:
local_browser_identity = _current_local_browser_identity()
if local_browser_identity is not None:
- try:
- _kill_local_browser_process_tree(local_browser_identity[0], local_browser_identity[1])
- finally:
- if local_browser_identity[2] and local_browser_identity[1]:
- shutil.rmtree(local_browser_identity[1], ignore_errors=True)
+ _cleanup_local_browser_from_identity(local_browser_identity, event_prefix="mcp_eof_local_browser")
except Exception:
- logging.getLogger(__name__).warning("MCP EOF fallback cleanup failed", exc_info=True)
+ LOG.warning("MCP EOF fallback cleanup failed", exc_info=True)
(force_exit or os._exit)(0)
try:
@@ -236,7 +194,7 @@ def _watch_stdin_eof(
return
stop.wait(0.05)
except (AttributeError, OSError, ValueError):
- logging.getLogger(__name__).warning("MCP stdin EOF watcher failed", exc_info=True)
+ LOG.warning("MCP stdin EOF watcher failed", exc_info=True)
def _start_stdin_eof_watcher() -> tuple[threading.Event, threading.Event]:
@@ -261,7 +219,7 @@ def _handle_mcp_shutdown_signal(_signum: int, _frame: Any) -> None:
os._exit(0 if eof_initiated else 128 + _signum)
-def get_pids_on_port(port: int) -> List[int]:
+def get_pids_on_port(port: int) -> list[int]:
"""Return a list of PIDs listening on the given port."""
pids = []
try:
@@ -273,7 +231,7 @@ def get_pids_on_port(port: int) -> List[int]:
return list(set(pids))
-def kill_pids(pids: List[int]) -> None:
+def kill_pids(pids: list[int]) -> None:
"""Kill the given list of PIDs in a cross-platform way."""
host_system = detect_os()
for pid in pids:
@@ -569,7 +527,7 @@ def run_dev() -> None:
class _ServerCardMiddleware:
"""Serve /.well-known/mcp/server-card.json for HTTP MCP transports."""
- def __init__(self, app: "ASGIApp", transport_type: str, host: str, port: int, mcp_path: str = "/mcp") -> None:
+ def __init__(self, app: ASGIApp, transport_type: str, host: str, port: int, mcp_path: str = "/mcp") -> None:
from skyvern.cli.core.server_card import build_server_card # noqa: PLC0415
self.app = app
@@ -579,7 +537,7 @@ class _ServerCardMiddleware:
endpoint_url = os.environ.get("SKYVERN_MCP_PUBLIC_URL") or f"http://{host_part}:{port}{mcp_path}"
self.card = build_server_card(self.transport_type, endpoint_url)
- async def __call__(self, scope: "Scope", receive: "Receive", send: "Send") -> None:
+ async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
if scope["type"] == "http" and scope["path"] == "/.well-known/mcp/server-card.json":
cors_headers = {
"Access-Control-Allow-Origin": "*",
@@ -629,6 +587,14 @@ def run_mcp(
global _mcp_eof_shutdown_requested
_mcp_eof_shutdown_requested = False
prepare_cli_runtime(intent=EnvIntent.CLOUD)
+ try:
+ from skyvern.library.local_browser_profile import ( # noqa: PLC0415
+ sweep_local_browser_profiles_with_budget,
+ )
+
+ sweep_local_browser_profiles_with_budget()
+ except Exception:
+ LOG.warning("local_browser_profile_startup_sweep_failed", exc_info=True)
from skyvern.cli.core.mcp_http_auth import MCPAPIKeyMiddleware # noqa: PLC0415
from skyvern.cli.core.session_manager import set_stateless_http_mode # noqa: PLC0415
from skyvern.cli.mcp_tools import mcp # noqa: PLC0415
@@ -693,10 +659,10 @@ def _normalize_mcp_path(path: str) -> str:
)
def run_code(
script_path: str = typer.Argument(..., help="Path to the Python script to run"),
- params: List[str] = typer.Option([], "-p", help="Parameters in format param=value (without leading dash)"),
+ params: list[str] = typer.Option([], "-p", help="Parameters in format param=value (without leading dash)"),
params_json: str = typer.Option(None, "--params", help="JSON string of parameters"),
params_file: str = typer.Option(None, "--params-file", help="Path to JSON file with parameters"),
- ai: Optional[str] = typer.Option(
+ ai: str | None = typer.Option(
"fallback", "--ai", help="AI mode to use for the script. Options: fallback, proactive or None"
),
) -> None:
diff --git a/skyvern/config.py b/skyvern/config.py
index df7423fe5..5f8912fa3 100644
--- a/skyvern/config.py
+++ b/skyvern/config.py
@@ -235,10 +235,6 @@ class Settings(BaseSettings):
# blocked by default; self-hosted deployments with internal SFTP targets can enable.
ALLOW_SFTP_INTERNAL_HOSTS: bool = False
- # Format: "http://<username>:<password>@host:port, http://<username>:<password>@host:port, ...."
- HOSTED_PROXY_POOL: str = ""
- ENABLE_PROXY: bool = False
-
# Secret key for JWT. Please generate your own secret key in production
SECRET_KEY: str = "PLACEHOLDER"
# Algorithm used to sign the JWT
diff --git a/skyvern/library/local_browser_profile.py b/skyvern/library/local_browser_profile.py
new file mode 100644
index 000000000..071458c66
--- /dev/null
+++ b/skyvern/library/local_browser_profile.py
@@ -0,0 +1,791 @@
+from __future__ import annotations
+
+import os
+import shutil
+import signal
+import stat
+import subprocess
+import sys
+import tempfile
+import threading
+import time
+from _thread import LockType
+from dataclasses import dataclass, field
+from pathlib import Path
+from uuid import uuid4
+
+import psutil
+import structlog
+
+LOG = structlog.get_logger(__name__)
+
+PROFILE_ROOT_PREFIX = "skyvern-local-browsers-"
+PROFILE_DIR_PREFIX = "profile-"
+PROFILE_LOCK_FILENAME = ".skyvern-profile.lock"
+PROFILE_ROOT_MODE = 0o700
+PROFILE_DIR_MODE = 0o700
+PROFILE_LOCK_MODE = 0o600
+PROCESS_KILL_TIMEOUT_SECONDS = 2.0
+PROFILE_DELETE_TIMEOUT_SECONDS = 1.0
+# One orphan with a live browser can cost ~3s (kill wait + delete) — reclamation converges across startups.
+PROFILE_SWEEP_STARTUP_BUDGET_SECONDS = 1.0
+PROFILE_PROCESS_STOP_RESERVE_SECONDS = 0.1
+
+_sweep_triggered = False
+_sweep_trigger_lock = threading.Lock()
+
+_Identity = tuple[int, int]
+
+
+class _ProfileIdentityChangedError(RuntimeError):
+ pass
+
+
+def _identity(result: os.stat_result) -> _Identity:
+ return result.st_dev, result.st_ino
+
+
+def _validate_directory(path: Path, uid: int, mode: int) -> os.stat_result:
+ result = os.lstat(path)
+ if not stat.S_ISDIR(result.st_mode) or result.st_uid != uid or stat.S_IMODE(result.st_mode) != mode:
+ raise RuntimeError(f"Unsafe local browser profile directory: {path}")
+ return result
+
+
+def _validate_lock_path(path: Path, fd: int, uid: int) -> os.stat_result:
+ path_result = os.lstat(path)
+ fd_result = os.fstat(fd)
+ if (
+ not stat.S_ISREG(path_result.st_mode)
+ or not stat.S_ISREG(fd_result.st_mode)
+ or path_result.st_uid != uid
+ or fd_result.st_uid != uid
+ or stat.S_IMODE(path_result.st_mode) != PROFILE_LOCK_MODE
+ or stat.S_IMODE(fd_result.st_mode) != PROFILE_LOCK_MODE
+ or _identity(path_result) != _identity(fd_result)
+ ):
+ raise RuntimeError(f"Unsafe local browser profile lock: {path}")
+ return fd_result
+
+
+def _lock_exclusive_nonblocking(fd: int) -> None:
+ import fcntl # noqa: PLC0415
+
+ fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
+
+
+def _unlock(fd: int) -> None:
+ import fcntl # noqa: PLC0415
+
+ fcntl.flock(fd, fcntl.LOCK_UN)
+
+
+def _lock_open_flags(*, create: bool) -> int:
+ flags = os.O_RDWR
+ if create:
+ flags |= os.O_CREAT | os.O_EXCL
+ flags |= getattr(os, "O_CLOEXEC", 0)
+ flags |= getattr(os, "O_NOFOLLOW", 0)
+ return flags
+
+
+def _directory_open_flags() -> int:
+ return os.O_RDONLY | getattr(os, "O_DIRECTORY", 0) | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_CLOEXEC", 0)
+
+
+def _matches_directory(
+ result: os.stat_result,
+ *,
+ uid: int,
+ mode: int,
+ expected_identity: _Identity,
+) -> bool:
+ return (
+ stat.S_ISDIR(result.st_mode)
+ and result.st_uid == uid
+ and stat.S_IMODE(result.st_mode) == mode
+ and _identity(result) == expected_identity
+ )
+
+
+def _profile_root(uid: int) -> Path:
+ return Path(tempfile.gettempdir()) / f"{PROFILE_ROOT_PREFIX}{uid}"
+
+
+def _prepare_profile_root() -> tuple[Path, int, os.stat_result]:
+ uid = os.getuid()
+ root = _profile_root(uid)
+ try:
+ root.mkdir(mode=PROFILE_ROOT_MODE)
+ except FileExistsError:
+ pass
+ else:
+ root.chmod(PROFILE_ROOT_MODE)
+ return root, uid, _validate_directory(root, uid, PROFILE_ROOT_MODE)
+
+
+@dataclass(slots=True)
+class LocalBrowserProfile:
+ path: Path
+ _root: Path
+ _uid: int
+ _root_identity: _Identity
+ _profile_identity: _Identity
+ _lock_identity: _Identity
+ _lock_fd: int | None
+ _state_lock: LockType = field(default_factory=threading.Lock, repr=False)
+ _cleanup_started: bool = field(default=False, repr=False)
+
+ def revalidate(self) -> bool:
+ with self._state_lock:
+ fd = self._lock_fd
+ if fd is None or self.path.parent != self._root:
+ return False
+ try:
+ root_result = _validate_directory(self._root, self._uid, PROFILE_ROOT_MODE)
+ profile_result = _validate_directory(self.path, self._uid, PROFILE_DIR_MODE)
+ lock_result = _validate_lock_path(self.path / PROFILE_LOCK_FILENAME, fd, self._uid)
+ _lock_exclusive_nonblocking(fd)
+ except (OSError, RuntimeError):
+ return False
+ return (
+ _identity(root_result) == self._root_identity
+ and _identity(profile_result) == self._profile_identity
+ and _identity(lock_result) == self._lock_identity
+ )
+
+ def release(self) -> None:
+ with self._state_lock:
+ fd = self._lock_fd
+ self._lock_fd = None
+ if fd is None:
+ return
+ try:
+ try:
+ _unlock(fd)
+ except OSError:
+ pass
+ finally:
+ try:
+ os.close(fd)
+ except OSError:
+ pass
+
+ def _begin_cleanup(self) -> bool:
+ with self._state_lock:
+ if self._cleanup_started:
+ return False
+ self._cleanup_started = True
+ return True
+
+
+def sweep_local_browser_profiles_once_in_background() -> threading.Thread | None:
+ if sys.platform == "win32":
+ return None
+
+ global _sweep_triggered
+ with _sweep_trigger_lock:
+ if _sweep_triggered:
+ return None
+ _sweep_triggered = True
+
+ thread = threading.Thread(
+ target=sweep_local_browser_profiles_with_budget,
+ daemon=True,
+ name="skyvern-profile-sweep",
+ )
+ thread.start()
+ return thread
+
+
+def create_local_browser_profile() -> LocalBrowserProfile | None:
+ if sys.platform == "win32":
+ return None
+
+ sweep_local_browser_profiles_once_in_background()
+
+ # A concurrent startup sweep can win the race in the mkdtemp->flock window: reaping the
+ # still-lockless dir (FileNotFoundError) or acquiring the just-published lock first
+ # (BlockingIOError — nobody else can hold a lock we created O_EXCL). Retry with a fresh dir.
+ last_error: OSError | None = None
+ for _ in range(3):
+ try:
+ return _create_local_browser_profile_once()
+ except (FileNotFoundError, BlockingIOError) as exc:
+ last_error = exc
+ raise RuntimeError("Local browser profile creation kept losing its directory") from last_error
+
+
+def _create_local_browser_profile_once() -> LocalBrowserProfile:
+ root, uid, root_result = _prepare_profile_root()
+ profile_path = Path(tempfile.mkdtemp(prefix=PROFILE_DIR_PREFIX, dir=root))
+ lock_fd: int | None = None
+ lock_acquired = False
+ try:
+ profile_path.chmod(PROFILE_DIR_MODE)
+ profile_result = _validate_directory(profile_path, uid, PROFILE_DIR_MODE)
+ lock_path = profile_path / PROFILE_LOCK_FILENAME
+ lock_fd = os.open(lock_path, _lock_open_flags(create=True), PROFILE_LOCK_MODE)
+ os.fchmod(lock_fd, PROFILE_LOCK_MODE)
+ lock_result = _validate_lock_path(lock_path, lock_fd, uid)
+ _lock_exclusive_nonblocking(lock_fd)
+ lock_acquired = True
+ profile = LocalBrowserProfile(
+ path=profile_path,
+ _root=root,
+ _uid=uid,
+ _root_identity=_identity(root_result),
+ _profile_identity=_identity(profile_result),
+ _lock_identity=_identity(lock_result),
+ _lock_fd=lock_fd,
+ )
+ lock_fd = None
+ if not profile.revalidate():
+ profile.release()
+ raise _ProfileIdentityChangedError("Local browser profile identity changed during creation")
+ return profile
+ except BlockingIOError:
+ if lock_fd is not None:
+ os.close(lock_fd)
+ raise
+ except _ProfileIdentityChangedError:
+ raise
+ except Exception:
+ if lock_fd is not None:
+ try:
+ if lock_acquired:
+ _unlock(lock_fd)
+ finally:
+ os.close(lock_fd)
+ try:
+ _remove_profile_directory_bounded(profile_path)
+ except Exception:
+ LOG.warning("local_browser_profile_creation_rollback_failed", path=str(profile_path), exc_info=True)
+ raise
+
+
+def _matching_processes(user_data_dir: Path) -> list[psutil.Process] | None:
+ user_data_arg = f"--user-data-dir={user_data_dir}"
+ matches: dict[int, psutil.Process] = {}
+ try:
+ for process in psutil.process_iter(["pid", "cmdline"]):
+ info = process.info
+ cmdline = None if info is None else info.get("cmdline")
+ if not cmdline:
+ # Our Chromium is an ordinary same-uid child, so its args are readable; blank same-uid entries are
+ # zombies or arg-restricted binaries, while other users cannot traverse the 0700 profile root.
+ continue
+ if user_data_arg not in cmdline:
+ continue
+ matches[process.pid] = process
+ for child in process.children(recursive=True):
+ matches[child.pid] = child
+ except Exception:
+ return None
+ return list(matches.values())
+
+
+def terminate_local_browser_processes(user_data_dir: str | Path) -> bool:
+ path = Path(user_data_dir)
+ scanned = _matching_processes(path)
+ if scanned is None:
+ return False
+ processes = {process.pid: process for process in scanned}
+ for process in processes.values():
+ try:
+ process.kill()
+ except (psutil.NoSuchProcess, psutil.ZombieProcess):
+ continue
+ except Exception:
+ return False
+ if not processes:
+ return True
+ try:
+ _gone, alive = psutil.wait_procs(list(processes.values()), timeout=PROCESS_KILL_TIMEOUT_SECONDS)
+ except Exception:
+ return False
+ return not alive
+
+
+def _wait_for_process_with_budget(
+ process: subprocess.Popen[bytes],
+ *,
+ started_at: float,
+ budget: float,
+ process_group: bool = False,
+) -> int | None:
+ deadline = started_at + max(budget, 0.0)
+ stop_reserve = min(PROFILE_PROCESS_STOP_RESERVE_SECONDS, max(budget / 2, 0.0))
+ work_timeout = max(0.0, deadline - time.monotonic() - stop_reserve)
+ try:
+ return process.wait(timeout=work_timeout)
+ except subprocess.TimeoutExpired:
+ pass
+
+ try:
+ if process_group:
+ os.killpg(process.pid, signal.SIGTERM)
+ else:
+ process.terminate()
+ except ProcessLookupError:
+ return None
+
+ terminate_timeout = min(stop_reserve / 2, max(0.0, deadline - time.monotonic()))
+ try:
+ process.wait(timeout=terminate_timeout)
+ except subprocess.TimeoutExpired:
+ pass
+
+ if process_group:
+ try:
+ os.killpg(process.pid, signal.SIGKILL)
+ except ProcessLookupError:
+ return None
+ elif process.poll() is None:
+ try:
+ process.kill()
+ except ProcessLookupError:
+ return None
+
+ if process.poll() is None:
+ try:
+ process.wait(timeout=max(0.0, deadline - time.monotonic()))
+ except subprocess.TimeoutExpired:
+ return None
+ return None
+
+
+def _remove_verified_directory_via_tombstone(
+ root_fd: int,
+ name: str,
+ *,
+ expected_identity: _Identity,
+ uid: int,
+) -> bool:
+ tombstone = f".reap-{uuid4().hex}"
+ try:
+ os.rename(name, tombstone, src_dir_fd=root_fd, dst_dir_fd=root_fd)
+ except OSError:
+ return False
+
+ def restore_original_name() -> None:
+ try:
+ os.rename(tombstone, name, src_dir_fd=root_fd, dst_dir_fd=root_fd)
+ except OSError:
+ pass
+
+ try:
+ tombstone_fd = os.open(tombstone, _directory_open_flags(), dir_fd=root_fd)
+ except OSError:
+ restore_original_name()
+ return False
+ try:
+ try:
+ result = os.fstat(tombstone_fd)
+ except OSError:
+ restore_original_name()
+ return False
+
+ if _identity(result) == expected_identity:
+ try:
+ os.rmdir(tombstone, dir_fd=root_fd)
+ except OSError:
+ restore_original_name()
+ return False
+ # APFS keeps st_nlink on the held fd after rmdir, so confirm removal by absence:
+ # the verified inode still existing under any root entry means a same-name swap
+ # won the race and rmdir removed the impostor instead. This scans the whole root
+ # per removal — fine at local-dev profile counts; revisit if the root grows large.
+ try:
+ for entry in os.listdir(root_fd):
+ if _identity(os.lstat(entry, dir_fd=root_fd)) == expected_identity:
+ return False
+ except OSError:
+ return False
+ return True
+
+ if stat.S_ISDIR(result.st_mode) and result.st_uid == uid:
+ try:
+ os.rmdir(tombstone, dir_fd=root_fd)
+ except OSError:
+ restore_original_name()
+ return False
+
+ restore_original_name()
+ return False
+ finally:
+ os.close(tombstone_fd)
+
+
+def _delete_profile_path(path: Path, *, preserve_lock: bool, expected_identity: _Identity | None = None) -> bool:
+ if sys.platform != "win32":
+ if expected_identity is None or not shutil.rmtree.avoids_symlink_attacks:
+ return False
+ flags = _directory_open_flags()
+ try:
+ parent_fd = os.open(path.parent, flags)
+ except FileNotFoundError:
+ return True
+ except OSError:
+ return False
+ try:
+ try:
+ profile_fd = os.open(path.name, flags, dir_fd=parent_fd)
+ except FileNotFoundError:
+ return True
+ except OSError:
+ return False
+ try:
+ if _identity(os.fstat(profile_fd)) != expected_identity:
+ return False
+ for name in os.listdir(profile_fd):
+ if preserve_lock and name == PROFILE_LOCK_FILENAME:
+ continue
+ try:
+ result = os.lstat(name, dir_fd=profile_fd)
+ if stat.S_ISDIR(result.st_mode):
+ shutil.rmtree(name, dir_fd=profile_fd)
+ else:
+ os.unlink(name, dir_fd=profile_fd)
+ except FileNotFoundError:
+ continue
+ if preserve_lock:
+ return True
+ return _remove_verified_directory_via_tombstone(
+ parent_fd,
+ path.name,
+ expected_identity=expected_identity,
+ uid=os.getuid(),
+ )
+ except OSError:
+ return False
+ finally:
+ os.close(profile_fd)
+ finally:
+ os.close(parent_fd)
+
+ try:
+ for entry in path.iterdir():
+ if preserve_lock and entry.name == PROFILE_LOCK_FILENAME:
+ continue
+ try:
+ if entry.is_dir() and not entry.is_symlink():
+ shutil.rmtree(entry)
+ else:
+ entry.unlink()
+ except FileNotFoundError:
+ continue
+ if not preserve_lock:
+ path.rmdir()
+ except FileNotFoundError:
+ return True
+ except OSError:
+ return False
+ return True
+
+
+def _remove_profile_directory_bounded(
+ path: Path,
+ *,
+ preserve_lock: bool = False,
+ expected_identity: _Identity | None = None,
+) -> bool:
+ started_at = time.monotonic()
+ mode = "--delete-profile-contents" if preserve_lock else "--delete-profile-tree"
+ command = [sys.executable, "-m", "skyvern.library.local_browser_profile", mode, str(path)]
+ if sys.platform != "win32":
+ if expected_identity is None:
+ try:
+ expected_identity = _identity(os.lstat(path))
+ except FileNotFoundError:
+ return True
+ except OSError:
+ return False
+ command.extend((str(expected_identity[0]), str(expected_identity[1])))
+ process = subprocess.Popen(
+ command,
+ stdin=subprocess.DEVNULL,
+ stdout=subprocess.DEVNULL,
+ stderr=subprocess.DEVNULL,
+ )
+ return_code = _wait_for_process_with_budget(
+ process,
+ started_at=started_at,
+ budget=PROFILE_DELETE_TIMEOUT_SECONDS,
+ )
+ return return_code == 0 and (path.exists() if preserve_lock else not path.exists())
+
+
+def _finalize_managed_profile_removal(profile: LocalBrowserProfile) -> bool:
+ if not profile.revalidate():
+ return False
+ flags = _directory_open_flags()
+ try:
+ root_fd = os.open(profile._root, flags)
+ except OSError:
+ return False
+ try:
+ if not _matches_directory(
+ os.fstat(root_fd),
+ uid=profile._uid,
+ mode=PROFILE_ROOT_MODE,
+ expected_identity=profile._root_identity,
+ ):
+ return False
+ try:
+ profile_fd = os.open(profile.path.name, flags, dir_fd=root_fd)
+ except OSError:
+ return False
+ try:
+ if not _matches_directory(
+ os.fstat(profile_fd),
+ uid=profile._uid,
+ mode=PROFILE_DIR_MODE,
+ expected_identity=profile._profile_identity,
+ ):
+ return False
+ try:
+ os.unlink(PROFILE_LOCK_FILENAME, dir_fd=profile_fd)
+ except OSError:
+ return False
+ finally:
+ os.close(profile_fd)
+
+ return _remove_verified_directory_via_tombstone(
+ root_fd,
+ profile.path.name,
+ expected_identity=profile._profile_identity,
+ uid=profile._uid,
+ )
+ finally:
+ os.close(root_fd)
+
+
+def cleanup_local_browser_profile(profile: LocalBrowserProfile | str | Path) -> bool:
+ managed = profile if isinstance(profile, LocalBrowserProfile) else None
+ path = managed.path if managed is not None else Path(profile)
+ if managed is not None and not managed._begin_cleanup():
+ return not path.exists()
+ try:
+ if managed is not None and not managed.revalidate():
+ return False
+ terminated = terminate_local_browser_processes(path)
+ if not terminated:
+ return False
+ if managed is not None and not managed.revalidate():
+ return False
+ if managed is None:
+ return _remove_profile_directory_bounded(path)
+ if not _remove_profile_directory_bounded(
+ path,
+ preserve_lock=True,
+ expected_identity=managed._profile_identity,
+ ):
+ return False
+ return _finalize_managed_profile_removal(managed)
+ except Exception:
+ LOG.warning("local_browser_profile_cleanup_failed", path=str(path), exc_info=True)
+ return False
+ finally:
+ if managed is not None:
+ managed.release()
+
+
+def _remove_lockless_empty_sweep_candidate(
+ root: Path,
+ uid: int,
+ root_result: os.stat_result,
+ path: Path,
+ profile_result: os.stat_result,
+) -> bool:
+ flags = _directory_open_flags()
+ try:
+ root_fd = os.open(root, flags)
+ except OSError:
+ return False
+ try:
+ if not _matches_directory(
+ os.fstat(root_fd),
+ uid=uid,
+ mode=PROFILE_ROOT_MODE,
+ expected_identity=_identity(root_result),
+ ):
+ return False
+ return _remove_verified_directory_via_tombstone(
+ root_fd,
+ path.name,
+ expected_identity=_identity(profile_result),
+ uid=uid,
+ )
+ except OSError:
+ return False
+ finally:
+ os.close(root_fd)
+
+
+def _remove_leftover_tombstone(
+ root: Path,
+ uid: int,
+ root_result: os.stat_result,
+ path: Path,
+) -> bool:
+ flags = _directory_open_flags()
+ try:
+ root_fd = os.open(root, flags)
+ except OSError:
+ return False
+ try:
+ if not _matches_directory(
+ os.fstat(root_fd),
+ uid=uid,
+ mode=PROFILE_ROOT_MODE,
+ expected_identity=_identity(root_result),
+ ):
+ return False
+ tombstone_fd = os.open(path.name, flags, dir_fd=root_fd)
+ try:
+ result = os.fstat(tombstone_fd)
+ if not stat.S_ISDIR(result.st_mode) or result.st_uid != uid:
+ return False
+ finally:
+ os.close(tombstone_fd)
+ os.rmdir(path.name, dir_fd=root_fd)
+ return True
+ except OSError:
+ return False
+ finally:
+ os.close(root_fd)
+
+
+def _open_sweep_candidate(
+ root: Path,
+ uid: int,
+ root_result: os.stat_result,
+ path: Path,
+) -> tuple[LocalBrowserProfile | None, bool]:
+ lock_fd: int | None = None
+ try:
+ profile_result = _validate_directory(path, uid, PROFILE_DIR_MODE)
+ lock_path = path / PROFILE_LOCK_FILENAME
+ try:
+ lock_fd = os.open(lock_path, _lock_open_flags(create=False))
+ except FileNotFoundError:
+ return None, _remove_lockless_empty_sweep_candidate(root, uid, root_result, path, profile_result)
+ lock_result = _validate_lock_path(lock_path, lock_fd, uid)
+ try:
+ _lock_exclusive_nonblocking(lock_fd)
+ except BlockingIOError:
+ os.close(lock_fd)
+ return None, False
+ profile = LocalBrowserProfile(
+ path=path,
+ _root=root,
+ _uid=uid,
+ _root_identity=_identity(root_result),
+ _profile_identity=_identity(profile_result),
+ _lock_identity=_identity(lock_result),
+ _lock_fd=lock_fd,
+ )
+ if not profile.revalidate():
+ profile.release()
+ return None, False
+ return profile, False
+ except (OSError, RuntimeError):
+ if lock_fd is not None:
+ os.close(lock_fd)
+ return None, False
+
+
+def sweep_orphaned_local_browser_profiles() -> int:
+ if sys.platform == "win32":
+ return 0
+ try:
+ root, uid, root_result = _prepare_profile_root()
+ candidates = list(root.iterdir())
+ except (OSError, RuntimeError):
+ LOG.warning("local_browser_profile_sweep_root_unsafe", exc_info=True)
+ return 0
+
+ removed = 0
+ for path in candidates:
+ if path.name.startswith("."):
+ if path.name.startswith(".reap-") and _remove_leftover_tombstone(root, uid, root_result, path):
+ removed += 1
+ continue
+ profile, recovered = _open_sweep_candidate(root, uid, root_result, path)
+ if recovered or (profile is not None and cleanup_local_browser_profile(profile)):
+ removed += 1
+ return removed
+
+
+def sweep_local_browser_profiles_with_budget() -> None:
+ if sys.platform == "win32":
+ return
+
+ global _sweep_triggered
+ with _sweep_trigger_lock:
+ _sweep_triggered = True
+
+ with tempfile.TemporaryFile() as capture_file:
+ started_at = time.monotonic()
+ process = subprocess.Popen(
+ [sys.executable, "-m", "skyvern.library.local_browser_profile", "--sweep"],
+ stdin=subprocess.DEVNULL,
+ stdout=capture_file,
+ stderr=capture_file,
+ start_new_session=True,
+ )
+ return_code = _wait_for_process_with_budget(
+ process,
+ started_at=started_at,
+ budget=PROFILE_SWEEP_STARTUP_BUDGET_SECONDS,
+ process_group=True,
+ )
+ capture_file.seek(0, os.SEEK_END)
+ capture_file.seek(max(capture_file.tell() - 2048, 0))
+ output_tail = capture_file.read(2048).decode(errors="replace").strip()
+
+ if return_code is None:
+ LOG.warning("local_browser_profile_sweep_timed_out", output_tail=output_tail)
+ return
+
+ if return_code != 0:
+ LOG.warning(
+ "local_browser_profile_sweep_failed",
+ return_code=return_code,
+ output_tail=output_tail,
+ )
+ elif output_tail:
+ LOG.warning("local_browser_profile_sweep_output", return_code=return_code, output_tail=output_tail)
+
+
+if __name__ == "__main__":
+ if sys.argv[1:] == ["--sweep"]:
+ sweep_orphaned_local_browser_profiles()
+ elif (
+ sys.platform == "win32"
+ and len(sys.argv) == 3
+ and sys.argv[1] in {"--delete-profile-contents", "--delete-profile-tree"}
+ ):
+ deleted = _delete_profile_path(
+ Path(sys.argv[2]),
+ preserve_lock=sys.argv[1] == "--delete-profile-contents",
+ )
+ raise SystemExit(0 if deleted else 1)
+ elif (
+ sys.platform != "win32"
+ and len(sys.argv) == 5
+ and sys.argv[1] in {"--delete-profile-contents", "--delete-profile-tree"}
+ ):
+ try:
+ expected_identity = int(sys.argv[3]), int(sys.argv[4])
+ except ValueError:
+ raise SystemExit(2) from None
+ deleted = _delete_profile_path(
+ Path(sys.argv[2]),
+ preserve_lock=sys.argv[1] == "--delete-profile-contents",
+ expected_identity=expected_identity,
+ )
+ raise SystemExit(0 if deleted else 1)
+ else:
+ raise SystemExit(2)
diff --git a/skyvern/library/skyvern.py b/skyvern/library/skyvern.py
index 36881439b..a2f27648c 100644
--- a/skyvern/library/skyvern.py
+++ b/skyvern/library/skyvern.py
@@ -3,7 +3,6 @@ from __future__ import annotations
import asyncio
import os
import pathlib
-import shutil
import tempfile
from typing import TYPE_CHECKING, Any
@@ -505,13 +504,20 @@ class Skyvern(AsyncSkyvern):
SkyvernBrowser: A browser instance with Skyvern capabilities.
"""
+ from skyvern.library import local_browser_profile # noqa: PLC0415
from skyvern.library.skyvern_browser import SkyvernBrowser # noqa: PLC0415
playwright = await self._get_playwright()
use_instance_defaults = user_data_dir is None and port is None
+ managed_profile: local_browser_profile.LocalBrowserProfile | None = None
if use_instance_defaults:
- user_data_path = pathlib.Path(tempfile.mkdtemp(prefix="skyvern-browser-"))
+ managed_profile = local_browser_profile.create_local_browser_profile()
+ user_data_path = (
+ managed_profile.path
+ if managed_profile is not None
+ else pathlib.Path(tempfile.mkdtemp(prefix="skyvern-browser-"))
+ )
launch_port = 0
elif user_data_dir:
user_data_path = pathlib.Path(user_data_dir)
@@ -526,6 +532,12 @@ class Skyvern(AsyncSkyvern):
if args:
launch_args.extend(args)
+ if managed_profile is not None and not managed_profile.revalidate():
+ deleted = await asyncio.to_thread(local_browser_profile.cleanup_local_browser_profile, managed_profile)
+ if not deleted:
+ LOG.warning("local_browser_profile_cleanup_deferred", user_data_dir=str(user_data_path))
+ raise RuntimeError("Local browser profile identity changed before Chromium launch")
+
browser_context = None
try:
browser_context = await playwright.chromium.launch_persistent_context(
@@ -534,23 +546,29 @@ class Skyvern(AsyncSkyvern):
args=launch_args,
)
resolved_port = await _read_devtools_active_port(user_data_path) if use_instance_defaults else launch_port
- except Exception:
- try:
- if browser_context is not None:
+ browser_address = f"http://localhost:{resolved_port}"
+ return SkyvernBrowser(
+ self,
+ browser_context,
+ browser_address=browser_address,
+ local_cdp_port=resolved_port,
+ local_user_data_dir=str(user_data_path),
+ local_user_data_dir_owned=use_instance_defaults,
+ local_browser_profile=managed_profile,
+ )
+ except BaseException:
+ if browser_context is not None:
+ try:
await browser_context.close()
- finally:
- if use_instance_defaults:
- shutil.rmtree(user_data_path, ignore_errors=True)
+ except BaseException:
+ LOG.warning("local_browser_context_close_rollback_failed", exc_info=True)
+ if use_instance_defaults:
+ deleted = await asyncio.to_thread(
+ local_browser_profile.cleanup_local_browser_profile, managed_profile or user_data_path
+ )
+ if not deleted:
+ LOG.warning("local_browser_profile_cleanup_deferred", user_data_dir=str(user_data_path))
raise
- browser_address = f"http://localhost:{resolved_port}"
- return SkyvernBrowser(
- self,
- browser_context,
- browser_address=browser_address,
- local_cdp_port=resolved_port,
- local_user_data_dir=str(user_data_path),
- local_user_data_dir_owned=use_instance_defaults,
- )
async def connect_to_browser_over_cdp(self, cdp_url: str) -> SkyvernBrowser:
"""Connect to an existing browser instance via Chrome DevTools Protocol (CDP).
diff --git a/skyvern/library/skyvern_browser.py b/skyvern/library/skyvern_browser.py
index 870a18ff0..fc45046f0 100644
--- a/skyvern/library/skyvern_browser.py
+++ b/skyvern/library/skyvern_browser.py
@@ -1,15 +1,20 @@
# ruff: noqa: E402
-import shutil
+import asyncio
from typing import TYPE_CHECKING, Any
+import structlog
+
from skyvern.exceptions import require_local_extra_modules
require_local_extra_modules("skyvern.library.skyvern_browser")
from playwright.async_api import BrowserContext, Page
+from skyvern.library import local_browser_profile
from skyvern.library.skyvern_browser_page import SkyvernBrowserPage
+LOG = structlog.get_logger(__name__)
+
if TYPE_CHECKING:
from skyvern.library.skyvern import Skyvern
@@ -51,6 +56,7 @@ class SkyvernBrowser(BrowserContext):
local_cdp_port: int | None = None,
local_user_data_dir: str | None = None,
local_user_data_dir_owned: bool = False,
+ local_browser_profile: local_browser_profile.LocalBrowserProfile | None = None,
):
super().__init__(browser_context)
self._skyvern = skyvern
@@ -61,6 +67,9 @@ class SkyvernBrowser(BrowserContext):
self._local_cdp_port = local_cdp_port
self._local_user_data_dir = local_user_data_dir
self._local_user_data_dir_owned = local_user_data_dir_owned
+ self._local_browser_profile = local_browser_profile
+ self._close_task: asyncio.Task[None] | None = None
+ self._closed = False
self.workflow_run_id: None | str = None
@@ -96,6 +105,10 @@ class SkyvernBrowser(BrowserContext):
def local_user_data_dir_owned(self) -> bool:
return self._local_user_data_dir_owned
+ @property
+ def local_browser_profile(self) -> local_browser_profile.LocalBrowserProfile | None:
+ return self._local_browser_profile
+
@property
def app_url(self) -> str | None:
return self._app_url
@@ -160,9 +173,38 @@ class SkyvernBrowser(BrowserContext):
await browser.close() # Closes both browser and cloud session
```
"""
- await self._browser_context.close(**kwargs)
- if self._local_user_data_dir_owned and self._local_user_data_dir:
- shutil.rmtree(self._local_user_data_dir, ignore_errors=True)
-
- if self._browser_session_id:
- await self._skyvern.close_browser_session(self._browser_session_id)
+ if self._closed:
+ return
+ close_task = self._close_task
+ if close_task is None:
+
+ async def close_sequence() -> None:
+ try:
+ await self._browser_context.close(**kwargs)
+ finally:
+ if self._local_user_data_dir_owned and self._local_user_data_dir:
+ deleted = await asyncio.to_thread(
+ local_browser_profile.cleanup_local_browser_profile,
+ self._local_browser_profile or self._local_user_data_dir,
+ )
+ if not deleted:
+ LOG.warning(
+ "local_browser_profile_cleanup_deferred",
+ user_data_dir=self._local_user_data_dir,
+ )
+
+ if self._browser_session_id:
+ await self._skyvern.close_browser_session(self._browser_session_id)
+
+ close_task = asyncio.create_task(close_sequence())
+ self._close_task = close_task
+
+ def _finish(task: asyncio.Task[None]) -> None:
+ if not task.cancelled() and task.exception() is None:
+ self._closed = True
+ self._close_task = None
+
+ close_task.add_done_callback(_finish)
+
+ # Shield: a cancelled waiter must not cancel the shared close for every other caller.
+ await asyncio.shield(close_task)
diff --git a/skyvern/proxy/__init__.py b/skyvern/proxy/__init__.py
deleted file mode 100644
index e8aff01aa..000000000
--- a/skyvern/proxy/__init__.py
+++ /dev/null
@@ -1,5 +0,0 @@
-"""CDP proxy service: a hexagonal core with pluggable ports and adapters.
-
-Layering is enforced by import-linter (see pyproject.toml): `core` and `ports`
-are pure (no framework or I/O imports); `adapters` depend on them, never the reverse.
-"""
diff --git a/skyvern/proxy/__main__.py b/skyvern/proxy/__main__.py
deleted file mode 100644
index 821212a08..000000000
--- a/skyvern/proxy/__main__.py
+++ /dev/null
@@ -1,130 +0,0 @@
-from __future__ import annotations
-
-import asyncio
-import os
-import time
-
-import structlog
-
-from skyvern.proxy.adapters.local_chrome import LocalChromeUpstreamBrowser
-from skyvern.proxy.adapters.memory import (
- AllowAllAuth,
- ForwardAllEventPolicy,
- InMemorySessionRegistry,
- InMemoryUpstreamBrowser,
- NoOpMetrics,
-)
-from skyvern.proxy.adapters.websocket_server import CdpProxyServer
-from skyvern.proxy.adapters.websocket_upstream import WebSocketUpstreamBrowser
-from skyvern.proxy.core.denylist import MethodPatternSet, org_denylist_interceptor
-from skyvern.proxy.core.interception_demo import demo_get_version_interceptor
-from skyvern.proxy.core.pipeline import CommandInterceptor, MiddlewarePipeline
-from skyvern.proxy.core.policy import EventPolicyEngine
-from skyvern.proxy.core.policy_pack import NOISY_EVENT_PACK_V1
-from skyvern.proxy.core.screencast import SCREENCAST_PACK_V1, screencast_pipeline
-from skyvern.proxy.core.session import ProxySession
-from skyvern.proxy.ports import EventPolicyPort, UpstreamBrowserPort
-
-LOG = structlog.get_logger(__name__)
-
-
-def build_upstream(kind: str) -> UpstreamBrowserPort:
- """Config-only adapter switch: swapping upstreams never touches the core."""
- if kind == "websocket":
- return WebSocketUpstreamBrowser()
- if kind == "local-chrome":
- return LocalChromeUpstreamBrowser()
- if kind == "memory":
- return InMemoryUpstreamBrowser()
- raise ValueError(f"unknown CDP_PROXY_UPSTREAM adapter: {kind!r}")
-
-
-def build_event_policy(kind: str) -> EventPolicyPort:
- """Config-only policy switch. Filtering is opt-in: the default forwards every
- event unchanged, and turning a pack on for real traffic is a rollout decision
- (SKY-12537), not a consequence of shipping the rules.
-
- Throttle windows run on a monotonic clock — a wall-clock step would otherwise
- hand out a free budget or stall one.
- """
- if kind == "forward-all":
- return ForwardAllEventPolicy()
- if kind == "noisy-v1":
- return EventPolicyEngine(config=NOISY_EVENT_PACK_V1, clock=time.monotonic)
- if kind == "screencast-v1":
- return EventPolicyEngine(config=SCREENCAST_PACK_V1, clock=time.monotonic)
- raise ValueError(f"unknown CDP_PROXY_EVENT_POLICY: {kind!r}")
-
-
-def build_interceptors(kind: str, denylist: str = "") -> tuple[CommandInterceptor, ...]:
- """Config-only interceptor switch (SKY-12535). The default is none: with
- interception disabled the command path is exactly the pre-seam pass-through.
-
- `denylist` (CDP_PROXY_DENYLIST, comma-separated method patterns) is the
- generic wiring for SKY-12538: one static list applied to every session this
- proxy serves, compiled at startup so a bad pattern fails the boot rather than
- a command. The per-organization source (cloud/cdp_proxy/denylist.py) replaces
- the lookup in the cloud composition root. The denylist runs FIRST: a denied
- method is denied before any other interceptor can see or rewrite it.
- """
- interceptors: list[CommandInterceptor] = []
- if denylist:
- patterns = MethodPatternSet.compile([part.strip() for part in denylist.split(",") if part.strip()])
-
- async def deny_all_sessions(session: ProxySession) -> MethodPatternSet | None:
- return patterns
-
- interceptors.append(org_denylist_interceptor(deny_all_sessions))
- if kind == "demo-get-version":
- interceptors.append(demo_get_version_interceptor)
- elif kind:
- raise ValueError(f"unknown CDP_PROXY_INTERCEPTORS: {kind!r}")
- return tuple(interceptors)
-
-
-def build_pipeline(kind: str, interceptors_kind: str = "", denylist: str = "") -> MiddlewarePipeline:
- """The command half of the selected policy, keyed off the same switch.
-
- Only the screencast policy has one: bounding what a client asks Page.startScreencast
- for is a command rewrite, and EventPolicyPort decides events only, so the two halves
- of that one policy reach the proxy through different seams and must be turned on
- together. A pack combining screencast with noisy-v1 would have to select both halves
- here too — the packs' rules do not overlap, so nothing but this switch prevents it.
-
- Interceptors are orthogonal to the policy pack and compose into the same pipeline.
- """
- interceptors = build_interceptors(interceptors_kind, denylist)
- if kind == "screencast-v1":
- return screencast_pipeline(interceptors)
- return MiddlewarePipeline(interceptors=interceptors)
-
-
-def main() -> None:
- # Dev wiring: in-memory registry, allow-all auth. Production adapters are
- # injected here in follow-up issues.
- upstream_kind = os.environ.get("CDP_PROXY_UPSTREAM", "websocket")
- policy_kind = os.environ.get("CDP_PROXY_EVENT_POLICY", "forward-all")
- interceptors_kind = os.environ.get("CDP_PROXY_INTERCEPTORS", "")
- denylist = os.environ.get("CDP_PROXY_DENYLIST", "")
- server = CdpProxyServer(
- upstream=build_upstream(upstream_kind),
- sessions=InMemorySessionRegistry(),
- auth=AllowAllAuth(),
- metrics=NoOpMetrics(),
- event_policy=build_event_policy(policy_kind),
- pipeline=build_pipeline(policy_kind, interceptors_kind, denylist),
- host=os.environ.get("CDP_PROXY_HOST", "0.0.0.0"),
- port=int(os.environ.get("CDP_PROXY_PORT", "9223")),
- )
- LOG.info(
- "starting CDP proxy",
- upstream=upstream_kind,
- event_policy=policy_kind,
- interceptors=interceptors_kind,
- denylist_patterns=len([part for part in denylist.split(",") if part.strip()]),
- )
- asyncio.run(server.serve_forever())
-
-
-if __name__ == "__main__":
- main()
diff --git a/skyvern/proxy/adapters/__init__.py b/skyvern/proxy/adapters/__init__.py
deleted file mode 100644
index 6ec680d05..000000000
--- a/skyvern/proxy/adapters/__init__.py
+++ /dev/null
@@ -1,5 +0,0 @@
-"""Generic adapters for the CDP proxy ports.
-
-Skyvern-cloud-specific adapters (DB-backed session registry, API-key auth,
-vendor-specific upstream configs) live in cloud/, never here.
-"""
diff --git a/skyvern/proxy/adapters/caching.py b/skyvern/proxy/adapters/caching.py
deleted file mode 100644
index 938d0b832..000000000
--- a/skyvern/proxy/adapters/caching.py
+++ /dev/null
@@ -1,87 +0,0 @@
-"""TTL-caching decorator over any SessionRegistryPort."""
-
-from __future__ import annotations
-
-import time
-from typing import Callable
-
-from skyvern.proxy.core.session import SessionResolution, SessionResolutionStatus
-from skyvern.proxy.ports import SessionRegistryPort
-
-DEFAULT_TTL_SECONDS = 15.0
-DEFAULT_MAX_ENTRIES = 1024
-
-_UNCACHEABLE = frozenset({SessionResolutionStatus.UNKNOWN, SessionResolutionStatus.PENDING})
-
-
-class TtlCachingSessionRegistry:
- """Caches resolutions so the connect path doesn't hit the backing store every time.
-
- ACTIVE, CLOSED, and EXPIRED results are cached for ttl_seconds; CLOSED and
- EXPIRED are terminal states that never revert, so caching those negatives is
- safe. UNKNOWN and PENDING are not cached, so a session created or published
- moments later becomes visible immediately. An ACTIVE entry is additionally
- capped at the resolution's expires_in_seconds so a session is never served
- past its own deadline. invalidate() drops the entry, orphans in-flight
- resolves for the id (so a read started before the close cannot re-insert a
- stale entry after it), and forwards to the inner registry.
- """
-
- def __init__(
- self,
- inner: SessionRegistryPort,
- ttl_seconds: float = DEFAULT_TTL_SECONDS,
- clock: Callable[[], float] = time.monotonic,
- max_entries: int = DEFAULT_MAX_ENTRIES,
- ) -> None:
- self._inner = inner
- self._ttl_seconds = ttl_seconds
- self._clock = clock
- self._max_entries = max_entries
- self._cache: dict[str, tuple[float, SessionResolution]] = {}
- self._inflight: dict[str, set[object]] = {}
-
- async def resolve(self, session_id: str) -> SessionResolution:
- now = self._clock()
- cached = self._cache.get(session_id)
- if cached is not None:
- deadline, resolution = cached
- if now < deadline:
- return resolution
- del self._cache[session_id]
- token = object()
- self._inflight.setdefault(session_id, set()).add(token)
- try:
- resolution = await self._inner.resolve(session_id)
- finally:
- tokens = self._inflight.get(session_id)
- flight_valid = tokens is not None and token in tokens
- if tokens is not None:
- tokens.discard(token)
- if not tokens:
- self._inflight.pop(session_id, None)
- if resolution.status in _UNCACHEABLE or not flight_valid:
- return resolution
- entry_ttl = self._ttl_seconds
- if resolution.expires_in_seconds is not None:
- entry_ttl = min(entry_ttl, resolution.expires_in_seconds)
- if entry_ttl <= 0:
- return resolution
- now = self._clock()
- self._prune(now)
- if len(self._cache) >= self._max_entries:
- # ponytail: FIFO eviction of the oldest insert; swap for real LRU if hit rates ever matter
- self._cache.pop(next(iter(self._cache)))
- self._cache[session_id] = (now + entry_ttl, resolution)
- return resolution
-
- async def invalidate(self, session_id: str) -> None:
- self._inflight.pop(session_id, None)
- self._cache.pop(session_id, None)
- await self._inner.invalidate(session_id)
-
- def _prune(self, now: float) -> None:
- # ponytail: O(n) sweep on insert; swap for a heap if per-node session counts ever make it show up
- stale = [key for key, (deadline, _) in self._cache.items() if deadline <= now]
- for key in stale:
- del self._cache[key]
diff --git a/skyvern/proxy/adapters/local_chrome.py b/skyvern/proxy/adapters/local_chrome.py
deleted file mode 100644
index ea10daf4a..000000000
--- a/skyvern/proxy/adapters/local_chrome.py
+++ /dev/null
@@ -1,180 +0,0 @@
-"""Local headless-Chrome upstream adapter for dev and CI.
-
-Launches a throwaway headless Chrome per connection with --remote-debugging-port=0,
-discovers the browser websocket endpoint from DevToolsActivePort, and dials it with
-the generic WebSocket adapter so both share one dial-and-classify path. Ignores
-session.upstream_ws_url — each connection gets its own browser.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import os
-import shutil
-import tempfile
-from pathlib import Path
-
-import structlog
-
-from skyvern.proxy.adapters.websocket_upstream import WebSocketUpstreamBrowser
-from skyvern.proxy.core.errors import LaunchEnvironmentError, LaunchTimeoutError
-from skyvern.proxy.core.session import ProxySession
-from skyvern.proxy.ports import UpstreamConnection
-
-LOG = structlog.get_logger(__name__)
-
-_EXECUTABLE_CANDIDATES = (
- "chromium",
- "chromium-browser",
- "google-chrome",
- "google-chrome-stable",
- "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
-)
-_DEVTOOLS_PORT_FILE = "DevToolsActivePort"
-_POLL_INTERVAL_SECONDS = 0.05
-_TERMINATE_GRACE_SECONDS = 5.0
-
-
-def find_local_chrome_executable() -> str | None:
- # Env var directly, not skyvern.config: the proxy image ships zero runner deps.
- if configured := os.environ.get("CHROME_EXECUTABLE_PATH"):
- return configured
- for candidate in _EXECUTABLE_CANDIDATES:
- if os.path.sep in candidate:
- if os.path.exists(candidate):
- return candidate
- elif resolved := shutil.which(candidate):
- return resolved
- return None
-
-
-def _read_devtools_endpoint(user_data_dir: Path) -> tuple[int, str] | None:
- """None while the port file is absent or still being written; raises
- LaunchEnvironmentError when its contents are garbage."""
- try:
- lines = (user_data_dir / _DEVTOOLS_PORT_FILE).read_text().splitlines()
- except OSError:
- return None
- except UnicodeDecodeError as exc:
- raise LaunchEnvironmentError("devtools port file is not valid UTF-8") from exc
- if len(lines) < 2:
- return None
- try:
- port = int(lines[0])
- except ValueError as exc:
- raise LaunchEnvironmentError("devtools port file did not contain a port number") from exc
- if not 1 <= port <= 65535:
- raise LaunchEnvironmentError(f"devtools port file contains an out-of-range port ({port})")
- return port, lines[1]
-
-
-# Accepted gap: SIGKILL to the parent cannot cascade to Chrome's renderer/zygote
-# children; dev/CI-only adapter, so stray children are bounded by the sandbox lifetime.
-async def _terminate_process(process: asyncio.subprocess.Process) -> None:
- if process.returncode is not None:
- return
- try:
- process.terminate()
- except ProcessLookupError:
- return
- try:
- await asyncio.wait_for(process.wait(), timeout=_TERMINATE_GRACE_SECONDS)
- except asyncio.TimeoutError:
- process.kill()
- await process.wait()
-
-
-class LocalChromeUpstreamConnection:
- def __init__(
- self, connection: UpstreamConnection, process: asyncio.subprocess.Process, user_data_dir: Path
- ) -> None:
- self._connection = connection
- self._process = process
- self._user_data_dir = user_data_dir
-
- async def send(self, raw: str) -> None:
- await self._connection.send(raw)
-
- async def receive(self) -> str:
- return await self._connection.receive()
-
- async def close(self) -> None:
- try:
- await self._connection.close()
- finally:
- try:
- await _terminate_process(self._process)
- finally:
- shutil.rmtree(self._user_data_dir, ignore_errors=True)
-
-
-class LocalChromeUpstreamBrowser:
- def __init__(
- self,
- executable_path: str | None = None,
- launch_timeout_seconds: float = 30.0,
- ) -> None:
- self._executable_path = executable_path
- self._launch_timeout_seconds = launch_timeout_seconds
- self._dialer = WebSocketUpstreamBrowser()
-
- async def connect(self, session: ProxySession) -> UpstreamConnection:
- executable = self._executable_path or find_local_chrome_executable()
- if executable is None:
- raise LaunchEnvironmentError("no Chrome/Chromium executable found; set CHROME_EXECUTABLE_PATH")
- try:
- user_data_dir = Path(tempfile.mkdtemp(prefix="skyvern-cdp-proxy-chrome-"))
- except OSError as exc:
- raise LaunchEnvironmentError(
- f"failed to create a browser profile directory ({type(exc).__name__})"
- ) from exc
- process: asyncio.subprocess.Process | None = None
- try:
- process = await self._launch(executable, user_data_dir)
- port, browser_path = await self._wait_for_devtools_endpoint(process, user_data_dir)
- connection = await self._dialer.dial(f"ws://127.0.0.1:{port}{browser_path}")
- return LocalChromeUpstreamConnection(connection, process, user_data_dir)
- except BaseException:
- # BaseException: a cancelled connect must still reap the process and profile dir.
- try:
- if process is not None:
- await _terminate_process(process)
- finally:
- shutil.rmtree(user_data_dir, ignore_errors=True)
- raise
-
- async def _launch(self, executable: str, user_data_dir: Path) -> asyncio.subprocess.Process:
- try:
- return await asyncio.create_subprocess_exec(
- executable,
- "--headless=new",
- "--remote-debugging-port=0",
- f"--user-data-dir={user_data_dir}",
- "--no-first-run",
- "--no-default-browser-check",
- # Dev/CI-only adapter: containerized CI has no user namespaces for the sandbox.
- "--no-sandbox",
- "--disable-gpu",
- stdout=asyncio.subprocess.DEVNULL,
- stderr=asyncio.subprocess.DEVNULL,
- )
- except OSError as exc:
- raise LaunchEnvironmentError(f"failed to launch local browser ({type(exc).__name__})") from exc
-
- async def _wait_for_devtools_endpoint(
- self, process: asyncio.subprocess.Process, user_data_dir: Path
- ) -> tuple[int, str]:
- deadline = asyncio.get_running_loop().time() + self._launch_timeout_seconds
- while True:
- if process.returncode is not None:
- raise LaunchEnvironmentError(
- f"local browser exited with code {process.returncode} before its devtools endpoint became ready"
- )
- endpoint = _read_devtools_endpoint(user_data_dir)
- if endpoint is not None:
- return endpoint
- if asyncio.get_running_loop().time() >= deadline:
- raise LaunchTimeoutError(
- f"local browser did not expose a devtools endpoint within {self._launch_timeout_seconds}s"
- )
- await asyncio.sleep(_POLL_INTERVAL_SECONDS)
diff --git a/skyvern/proxy/adapters/memory.py b/skyvern/proxy/adapters/memory.py
deleted file mode 100644
index f8b7d140c..000000000
--- a/skyvern/proxy/adapters/memory.py
+++ /dev/null
@@ -1,134 +0,0 @@
-"""In-memory adapters: local dev defaults and contract-test references."""
-
-from __future__ import annotations
-
-import asyncio
-from typing import Mapping
-
-from skyvern.proxy.core.frames import CdpCommand, CdpEvent
-from skyvern.proxy.core.policy import FORWARD, PolicyDecision
-from skyvern.proxy.core.session import (
- Principal,
- ProxySession,
- ResolvedSession,
- SessionResolution,
- UpstreamClosedError,
- principal_owns_resolution,
-)
-from skyvern.proxy.ports import UpstreamConnection
-
-
-class InMemoryUpstreamConnection:
- """Loopback upstream: frames sent are echoed back to receive()."""
-
- def __init__(self) -> None:
- self._queue: asyncio.Queue[str | None] = asyncio.Queue()
- self._closed = False
-
- async def send(self, raw: str) -> None:
- if self._closed:
- raise UpstreamClosedError("connection is closed")
- await self._queue.put(raw)
-
- async def receive(self) -> str:
- if self._closed and self._queue.empty():
- raise UpstreamClosedError("connection is closed")
- raw = await self._queue.get()
- if raw is None:
- raise UpstreamClosedError("connection is closed")
- return raw
-
- async def close(self) -> None:
- self._closed = True
- await self._queue.put(None)
-
-
-class InMemoryUpstreamBrowser:
- async def connect(self, session: ProxySession) -> UpstreamConnection:
- return InMemoryUpstreamConnection()
-
-
-class InMemorySessionRegistry:
- """Authoritative in-memory registry for dev and tests; seed with put()/mark_*()."""
-
- def __init__(self) -> None:
- self._resolutions: dict[str, SessionResolution] = {}
-
- async def resolve(self, session_id: str) -> SessionResolution:
- return self._resolutions.get(session_id, SessionResolution.unknown())
-
- async def invalidate(self, session_id: str) -> None:
- """No-op: this store is authoritative, there is no cached state to drop."""
- return None
-
- def put(self, session: ResolvedSession, expires_in_seconds: float | None = None) -> None:
- self._resolutions[session.session_id] = SessionResolution.active(session, expires_in_seconds)
-
- def mark_pending(self, session_id: str, organization_id: str | None = None) -> None:
- self._resolutions[session_id] = SessionResolution.pending(organization_id)
-
- def mark_closed(self, session_id: str, organization_id: str | None = None) -> None:
- self._resolutions[session_id] = SessionResolution.closed(organization_id)
-
- def mark_expired(self, session_id: str, organization_id: str | None = None) -> None:
- self._resolutions[session_id] = SessionResolution.expired(organization_id)
-
- def remove(self, session_id: str) -> None:
- self._resolutions.pop(session_id, None)
-
-
-class AllowAllAuth:
- async def authenticate(self, credentials: Mapping[str, str]) -> Principal | None:
- return Principal(principal_id="local-dev")
-
- def authorize(self, principal: Principal, resolution: SessionResolution) -> bool:
- return principal_owns_resolution(principal, resolution)
-
-
-class StaticKeyAuth:
- """Dev/test AuthPort: a fixed api-key -> Principal table.
-
- The driving adapter normalizes header/query/path credentials into the same
- mapping, so a key presented any of those ways matches here. Key values are
- never logged.
- """
-
- def __init__(self, keys: Mapping[str, Principal]) -> None:
- self._keys = dict(keys)
-
- async def authenticate(self, credentials: Mapping[str, str]) -> Principal | None:
- api_key = credentials.get("x-api-key")
- if not api_key:
- return None
- return self._keys.get(api_key)
-
- def authorize(self, principal: Principal, resolution: SessionResolution) -> bool:
- return principal_owns_resolution(principal, resolution)
-
-
-class NoOpMetrics:
- def increment(self, name: str, amount: int = 1, tags: Mapping[str, str] | None = None) -> None:
- return None
-
- def observe(self, name: str, value: float, tags: Mapping[str, str] | None = None) -> None:
- return None
-
- def gauge(self, name: str, amount: int, tags: Mapping[str, str] | None = None) -> None:
- return None
-
-
-class ForwardAllEventPolicy:
- """The default policy: every event forwarded unchanged, nothing tracked.
-
- Deliberately stateless — with no policy configured the proxy must behave exactly
- as it would with no policy engine at all.
- """
-
- def decide(self, event: CdpEvent, session: ProxySession) -> PolicyDecision:
- return FORWARD
-
- def observe_command(self, command: CdpCommand, session: ProxySession) -> None:
- return None
-
- def forget(self, session_id: str) -> None:
- return None
diff --git a/skyvern/proxy/adapters/websocket_server.py b/skyvern/proxy/adapters/websocket_server.py
deleted file mode 100644
index 6e600189e..000000000
--- a/skyvern/proxy/adapters/websocket_server.py
+++ /dev/null
@@ -1,1763 +0,0 @@
-"""Driving adapter: client-facing CDP WebSocket endpoint.
-
-Accepts a client connection, resolves its session, and relays frames both ways
-through the core pipeline. Also serves the HTTP discovery endpoints CDP clients
-(Playwright, puppeteer) probe before the WebSocket upgrade — /json/version,
-/json/list, /json — returning proxy-scoped debugger URLs so the upstream browser
-URL is never exposed. The client-bound relay is decoupled by a bounded outbound
-queue so a slow client cannot back up the proxy or the upstream.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import contextlib
-import email.utils
-import json
-import logging
-import os
-import signal
-import time
-from dataclasses import dataclass, field
-from http import HTTPStatus
-from typing import Mapping, NamedTuple
-from urllib.parse import parse_qs, urlsplit
-
-import structlog
-import websockets
-from websockets.datastructures import Headers, MultipleValuesError
-from websockets.http11 import Request, Response
-
-from skyvern.proxy.core.errors import UpstreamConnectError
-from skyvern.proxy.core.frames import (
- KNOWN_CDP_DOMAINS,
- LIFECYCLE_EVENTS,
- PROXY_CLIENT_KEY,
- TARGET_ATTACHED_EVENT,
- TARGET_DETACHED_EVENT,
- CdpCommand,
- CdpEvent,
- CdpFrame,
- CdpResponse,
- FrameDecodeError,
- RemapperFullError,
- RequestIdRemapper,
- decode_frame,
- encode_frame,
- params_session_id,
-)
-from skyvern.proxy.core.pipeline import (
- INTERCEPTOR_FAILURE_REASON,
- Direction,
- InterceptContext,
- MiddlewarePipeline,
- SynthesizedResponse,
- interceptor_failure_response,
-)
-from skyvern.proxy.core.policy import Drop, Rewrite
-from skyvern.proxy.core.policy_pack import POLICY_PACK_EVENT_METHODS
-from skyvern.proxy.core.screencast import (
- SCREENCAST_PACK_EVENT_METHODS,
- is_screencast_frame,
- screencast_frame_ack,
-)
-from skyvern.proxy.core.session import ProxySession, SessionResolutionStatus, UpstreamClosedError
-from skyvern.proxy.ports import (
- AuthPort,
- EventPolicyPort,
- MetricsPort,
- SessionRegistryPort,
- UpstreamBrowserPort,
- UpstreamConnection,
-)
-
-LOG = structlog.get_logger(__name__)
-
-# websockets DEBUG-logs handshake URIs and headers, which can carry a client's
-# URL-borne API key; pin its logger above DEBUG so a directly-embedded driving
-# adapter self-suppresses regardless of import order (never rely on a sibling).
-logging.getLogger("websockets").setLevel(logging.INFO)
-
-_UNAUTHORIZED_CLOSE_CODE = 4401
-_UNKNOWN_CLOSE_CODE = 4404
-_UNKNOWN_CLOSE_REASON = "unknown session"
-_UPSTREAM_UNAVAILABLE_CLOSE_CODE = 1011
-# WS 1013 "try again later" — the client cannot drain fast enough and is closed
-# so its backpressure never reaches the upstream browser (the runner-CPU goal).
-_BACKPRESSURE_CLOSE_CODE = 1013
-# Cap on in-flight commands per connection, applied to both the id-remap table and
-# the command start-times; a stalled/unanswered command must never grow either map
-# unbounded. Oldest-eviction FIFO — well above real CDP concurrency, so a legitimate
-# client's response or latency is never dropped.
-_MAX_PENDING_COMMANDS = 4096
-
-# CDP's own code/message for a sessionId the connection cannot address. Reused
-# verbatim when refusing a client's command for a session another client owns, so a
-# foreign session is indistinguishable from a nonexistent one (no ownership oracle).
-_SESSION_NOT_FOUND_CODE = -32001
-_SESSION_NOT_FOUND_MESSAGE = "Session with given id not found."
-
-_SET_AUTO_ATTACH_METHOD = "Target.setAutoAttach"
-_ATTACH_TO_TARGET_METHOD = "Target.attachToTarget"
-# Commands that address a session inside params rather than by the frame's sessionId.
-_SESSION_PARAM_METHODS = frozenset({"Target.detachFromTarget", "Target.sendMessageToTarget"})
-
-# Client-bound outbound queue bounds. A dedicated writer drains this queue to the
-# client socket, so a slow client blocks only the writer — never the upstream read
-# loop. Overflow of EITHER the frame-count or the total-queued-byte bound tail-drops
-# droppable events; a client that stays overflowed for close_after consecutive
-# frames (or cannot even take a command response) is closed with
-# _BACKPRESSURE_CLOSE_CODE. The byte bound stops a slow client pinning a few huge
-# frames (large bodies, screencast) from exhausting proxy memory under the count
-# bound. Env-tunable per the proxy's own knobs (zero-dep image, no skyvern.config).
-_DEFAULT_CLIENT_QUEUE_MAXSIZE = 1024
-_DEFAULT_BACKPRESSURE_CLOSE_AFTER = 512
-_DEFAULT_CLIENT_QUEUE_MAXBYTES = 64 * 1024 * 1024
-_MAX_CLIENT_QUEUE_BOUND = 1_000_000
-_MAX_CLIENT_QUEUE_MAXBYTES = 8 * 1024 * 1024 * 1024
-# Bounded window to flush queued frames (e.g. a final command response) to the
-# client on a clean upstream close before teardown; a client that cannot take the
-# drain in time is closed with _BACKPRESSURE_CLOSE_CODE.
-_OUTBOUND_DRAIN_TIMEOUT_SECONDS = 5.0
-
-# HTTP discovery targets a client GETs before the WS upgrade, longest suffix first
-# so /json/version and /json/list match before the bare /json alias.
-_DISCOVERY_SUFFIXES = ("/json/version", "/json/list", "/json")
-
-# Unauthenticated HTTP health target for orchestrator probes (httpGet, not
-# tcpSocket: a wedged event loop still accepts TCP via the listen backlog).
-_HEALTH_PATH = "/healthz"
-# After SIGTERM/SIGINT the listener closes and existing relays drain. Past this
-# budget remaining connections are force-closed (1001) so the process exits with
-# clean WS closes instead of the supervisor's SIGKILL cutting sockets mid-frame.
-_DEFAULT_DRAIN_TIMEOUT_SECONDS = 3600
-_MAX_DRAIN_TIMEOUT_SECONDS = 24 * 3600
-
-# Enqueued by the drain path so the writer flushes all real frames then stops.
-_DRAIN_SENTINEL = object()
-
-
-def _positive_env_int(name: str, default: int, maximum: int) -> int:
- """Env int in [1, maximum], falling back to default on any invalid value so a
- misconfig can neither zero the bound nor make it unbounded."""
- try:
- value = int(os.environ.get(name, ""))
- except ValueError:
- return default
- return value if 1 <= value <= maximum else default
-
-
-def resolve_client_queue_limits() -> tuple[int, int, int]:
- return (
- _positive_env_int("CDP_PROXY_CLIENT_QUEUE_MAXSIZE", _DEFAULT_CLIENT_QUEUE_MAXSIZE, _MAX_CLIENT_QUEUE_BOUND),
- _positive_env_int(
- "CDP_PROXY_CLIENT_BACKPRESSURE_CLOSE_AFTER", _DEFAULT_BACKPRESSURE_CLOSE_AFTER, _MAX_CLIENT_QUEUE_BOUND
- ),
- _positive_env_int(
- "CDP_PROXY_CLIENT_QUEUE_MAXBYTES", _DEFAULT_CLIENT_QUEUE_MAXBYTES, _MAX_CLIENT_QUEUE_MAXBYTES
- ),
- )
-
-
-def _clamp_client_bound(value: int, maximum: int = _MAX_CLIENT_QUEUE_BOUND) -> int:
- """Force an explicit constructor bound into [1, maximum]; a <=0 maxsize would
- make asyncio.Queue unbounded and silently defeat tail-drop / 1013-close."""
- return max(1, min(value, maximum))
-
-
-class _OutboundQueue:
- """Client-bound queue bounded by BOTH frame count and total queued bytes.
-
- put_nowait raises asyncio.QueueFull when either bound would be exceeded, so the
- relay's existing tail-drop/1013 policy fires on byte pressure too. A lone frame
- is always admitted onto an empty queue (mirroring the count bound accepting one
- frame) so an over-budget frame can never permanently wedge delivery.
- """
-
- def __init__(self, max_frames: int, max_bytes: int) -> None:
- self._queue: asyncio.Queue[str | object] = asyncio.Queue(maxsize=max_frames)
- self._max_bytes = max_bytes
- self._queued_bytes = 0
-
- def put_nowait(self, wire: str) -> None:
- if self._queued_bytes and self._queued_bytes + len(wire) > self._max_bytes:
- raise asyncio.QueueFull
- self._queue.put_nowait(wire)
- self._queued_bytes += len(wire)
-
- async def get(self) -> str | object:
- item = await self._queue.get()
- if isinstance(item, str):
- self._queued_bytes -= len(item)
- return item
-
- def get_nowait(self) -> str:
- item = self._queue.get_nowait()
- assert isinstance(item, str)
- self._queued_bytes -= len(item)
- return item
-
- def empty(self) -> bool:
- return self._queue.empty()
-
- async def put_sentinel(self) -> None:
- # Bypasses the byte bound (the sentinel carries none) and waits for a count
- # slot, which the draining writer frees; the outer drain timeout bounds it.
- await self._queue.put(_DRAIN_SENTINEL)
-
-
-class _ClientChannel:
- """One client's delivery seat on a shared upstream connection.
-
- The single upstream reader routes frames into `outbound` (drained by the
- client's own writer), so per-client backpressure stays isolated: one slow
- client is tail-dropped and closed without stalling the shared read loop or the
- other clients. `closed` is the teardown signal the reader raises for this
- client (backpressure, or a deterministic close when the upstream dies).
- """
-
- def __init__(self, client_key: str, outbound: _OutboundQueue) -> None:
- self.client_key = client_key
- self.outbound = outbound
- self.overflow_streak = 0
- self.closed = asyncio.Event()
- self.drain = False
- self.close_code: int | None = None
- self.close_reason = ""
-
- def signal(self, *, drain: bool, code: int | None, reason: str) -> None:
- if self.closed.is_set():
- return
- self.drain = drain
- self.close_code = code
- self.close_reason = reason
- self.closed.set()
-
- async def wait_closed(self) -> None:
- await self.closed.wait()
-
-
-class _AttachIntent(NamedTuple):
- """A client's attach request, recorded when it is sent and settled when its response
- arrives. Tentative on purpose: the announcement it causes precedes the response, so
- the intent is what routes that announcement — but a command that ERRORS is never
- announced, and its intent must not survive to be paired with another client's later
- attach to the same target."""
-
- upstream_id: int
- client_key: str
- target_id: str | None # set for an explicit Target.attachToTarget
- enabled_autoattach: bool # this command is what turned browser-level autoAttach on
- disabled_autoattach: bool # this command is what turned this client's autoAttach off
-
-
-@dataclass
-class _SharedUpstream:
- """One upstream browser connection shared by every client on the same browser.
-
- Keyed by the resolved upstream target and ref-counted by the owning server: the
- connection opens on the first client and closes when the last leaves. The
- request-id remapper, per-command latency starts, and the cdp-sessionId owner map
- are shared so one reader can multiplex several clients over the single socket.
-
- Implements the send half of `UpstreamConnection` (a serialized send so two
- clients' frames never interleave on the wire); the shared reader owns the
- receive half directly. Reconnection/session-resumption would slot into this
- owner without touching clients — out of scope for v1 (seam only, SKY-12499).
- """
-
- session: ProxySession
- connection: UpstreamConnection | None = None
- connect_task: asyncio.Future[None] | None = None
- reader: asyncio.Task[None] | None = None
- remapper: RequestIdRemapper = field(default_factory=RequestIdRemapper)
- command_starts: dict[int, tuple[str, float]] = field(default_factory=dict)
- session_owner: dict[str, str] = field(default_factory=dict)
- # Clients that enabled browser-level autoAttach, in the order they asked. Chrome
- # auto-attaches once per target on this one shared connection, so the sessions it
- # creates are owned by the first still-connected asker.
- autoattach_clients: list[str] = field(default_factory=list)
- # Attach requests still in flight, oldest first. Chrome announces an attach with an
- # event that lands BEFORE the command response, so the reply cannot be what tells us
- # who to hand that event to — the intent has to be recorded up front and settled
- # afterwards. Ordered, and not keyed by targetId, because two clients may attach to
- # the SAME target: Chrome opens a session per attach and announces them in the order
- # asked, so the announcements pair off against this queue in order.
- attach_intents: list[_AttachIntent] = field(default_factory=list)
- channels: dict[str, _ClientChannel] = field(default_factory=dict)
- send_lock: asyncio.Lock = field(default_factory=asyncio.Lock)
- refcount: int = 0
- # Set once the reader has exited: the socket is dead and this entry must never be
- # joined by a new client (they dial fresh). Guards the teardown-window race.
- defunct: bool = False
- _client_seq: int = 0
- _closing: bool = False
-
- def next_client_key(self) -> str:
- self._client_seq += 1
- return f"c{self._client_seq}"
-
- async def send(self, raw: str) -> None:
- if self.connection is None:
- raise UpstreamClosedError("shared upstream is not connected")
- async with self.send_lock:
- await self.connection.send(raw)
-
- async def receive(self) -> str:
- raise RuntimeError("the shared connection is read only by its reader task")
-
- async def close(self) -> None:
- await self.aclose()
-
- async def aclose(self) -> None:
- """Reap the reader and close the socket. Idempotent, teardown-safe."""
- if self._closing:
- return
- self._closing = True
- if self.reader is not None:
- self.reader.cancel()
- with contextlib.suppress(asyncio.CancelledError):
- await self.reader
- if self.connection is not None:
- await self.connection.close()
-
-
-# Lifecycle codes are only ever revealed to the owning organization.
-_LIFECYCLE_CLOSE_CODES = {
- SessionResolutionStatus.PENDING: (4409, "session not ready"),
- SessionResolutionStatus.CLOSED: (4410, "session closed"),
- SessionResolutionStatus.EXPIRED: (4408, "session expired"),
-}
-
-# Metric names (namespace skyvern.cdp_proxy.*). Emitted through the injected
-# MetricsPort only — this module never imports a telemetry SDK. Tags carry only
-# safe dims (org_id, session_id, cdp method/domain, direction, reason); the
-# upstream URL, connect headers, and any credential never become a label.
-_METRIC_CLIENT_CONNECTED = "skyvern.cdp_proxy.client_connected"
-_METRIC_CLIENT_DISCONNECTED = "skyvern.cdp_proxy.client_disconnected"
-_METRIC_ACTIVE_SESSIONS = "skyvern.cdp_proxy.active_sessions"
-_METRIC_CONNECTION_REJECTED = "skyvern.cdp_proxy.connection_rejected"
-_METRIC_FRAMES_RELAYED = "skyvern.cdp_proxy.frames_relayed"
-_METRIC_BYTES_RELAYED = "skyvern.cdp_proxy.bytes_relayed"
-_METRIC_COMMAND_LATENCY = "skyvern.cdp_proxy.command_latency_seconds"
-_METRIC_FRAMES_DROPPED = "skyvern.cdp_proxy.frames_dropped"
-_METRIC_FRAME_DECODE_ERRORS = "skyvern.cdp_proxy.frame_decode_errors"
-_METRIC_CLIENT_BACKPRESSURE_CLOSED = "skyvern.cdp_proxy.client_backpressure_closed"
-_METRIC_COMMANDS_INTERCEPTED = "skyvern.cdp_proxy.commands_intercepted"
-
-
-def _org_tag(session: ProxySession) -> dict[str, str]:
- org_id = session.principal.organization_id if session.principal else None
- return {"org_id": org_id} if org_id else {}
-
-
-# Latency labels are derived from client-supplied method names, so BOTH the method
-# and the domain must be bucketed against allowlists — otherwise an authenticated
-# client could mint unbounded histogram series or smuggle a token/URL into a label
-# (e.g. "Page.<a-token>"). A method is exported verbatim only if it is in the
-# method allowlist; otherwise it collapses to "other" (keeping its domain when that
-# domain is known, else "other"). The allowlists cover the common CDP surface that
-# Playwright/puppeteer/CDP clients drive; anything else buckets to "other" — safe
-# degradation, never a client-controlled string. Kept for benchmark (SKY-12503) /
-# policy (SKY-12532).
-#
-# Commands, plus the EVENT methods the policy pack names (SKY-12501): a drop counter
-# tagged cdp_method="other" cannot say which stream a reduction came from. Only the
-# pack's own methods are listed — the set stays static and small, so an event method
-# never reaches a label unless it is written here.
-
-_KNOWN_CDP_METHODS = frozenset(
- {
- *POLICY_PACK_EVENT_METHODS,
- *SCREENCAST_PACK_EVENT_METHODS,
- "Accessibility.disable",
- "Accessibility.enable",
- "Accessibility.getFullAXTree",
- "Accessibility.getPartialAXTree",
- "Accessibility.queryAXTree",
- "Browser.close",
- "Browser.getVersion",
- "Browser.getWindowBounds",
- "Browser.getWindowForTarget",
- "Browser.grantPermissions",
- "Browser.resetPermissions",
- "Browser.setDownloadBehavior",
- "Browser.setWindowBounds",
- "CSS.disable",
- "CSS.enable",
- "CSS.getComputedStyleForNode",
- "CSS.getMatchedStylesForNode",
- "Debugger.disable",
- "Debugger.enable",
- "Debugger.evaluateOnCallFrame",
- "Debugger.removeBreakpoint",
- "Debugger.resume",
- "Debugger.setBreakpointByUrl",
- "Debugger.setPauseOnExceptions",
- "Debugger.stepOver",
- "DOM.describeNode",
- "DOM.disable",
- "DOM.enable",
- "DOM.focus",
- "DOM.getAttributes",
- "DOM.getBoxModel",
- "DOM.getContentQuads",
- "DOM.getDocument",
- "DOM.getOuterHTML",
- "DOM.querySelector",
- "DOM.querySelectorAll",
- "DOM.removeAttribute",
- "DOM.requestNode",
- "DOM.resolveNode",
- "DOM.scrollIntoViewIfNeeded",
- "DOM.setAttributeValue",
- "DOM.setFileInputFiles",
- "DOMSnapshot.captureSnapshot",
- "DOMSnapshot.disable",
- "DOMSnapshot.enable",
- "Emulation.clearDeviceMetricsOverride",
- "Emulation.clearGeolocationOverride",
- "Emulation.setCPUThrottlingRate",
- "Emulation.setDefaultBackgroundColorOverride",
- "Emulation.setDeviceMetricsOverride",
- "Emulation.setEmulatedMedia",
- "Emulation.setGeolocationOverride",
- "Emulation.setLocaleOverride",
- "Emulation.setScriptExecutionDisabled",
- "Emulation.setTimezoneOverride",
- "Emulation.setTouchEmulationEnabled",
- "Emulation.setUserAgentOverride",
- "Fetch.continueRequest",
- "Fetch.continueResponse",
- "Fetch.continueWithAuth",
- "Fetch.disable",
- "Fetch.enable",
- "Fetch.failRequest",
- "Fetch.fulfillRequest",
- "Fetch.getResponseBody",
- "Fetch.takeResponseBodyAsStream",
- "IO.close",
- "IO.read",
- "IO.resolveBlob",
- "Input.dispatchDragEvent",
- "Input.dispatchKeyEvent",
- "Input.dispatchMouseEvent",
- "Input.dispatchTouchEvent",
- "Input.insertText",
- "Input.setInterceptDrags",
- "Input.synthesizeScrollGesture",
- "Input.synthesizeTapGesture",
- "Log.clear",
- "Log.disable",
- "Log.enable",
- "Log.startViolationsReport",
- "Network.clearBrowserCache",
- "Network.clearBrowserCookies",
- "Network.continueInterceptedRequest",
- "Network.deleteCookies",
- "Network.disable",
- "Network.emulateNetworkConditions",
- "Network.enable",
- "Network.getAllCookies",
- "Network.getCookies",
- "Network.getRequestPostData",
- "Network.getResponseBody",
- "Network.getResponseBodyForInterception",
- "Network.setBlockedURLs",
- "Network.setCacheDisabled",
- "Network.setCookie",
- "Network.setCookies",
- "Network.setExtraHTTPHeaders",
- "Network.setRequestInterception",
- "Network.setUserAgentOverride",
- "Page.addScriptToEvaluateOnNewDocument",
- "Page.bringToFront",
- "Page.captureScreenshot",
- "Page.close",
- "Page.createIsolatedWorld",
- "Page.crash",
- "Page.disable",
- "Page.enable",
- "Page.getFrameTree",
- "Page.getLayoutMetrics",
- "Page.getNavigationHistory",
- "Page.handleJavaScriptDialog",
- "Page.navigate",
- "Page.navigateToHistoryEntry",
- "Page.printToPDF",
- "Page.reload",
- "Page.removeScriptToEvaluateOnNewDocument",
- "Page.screencastFrameAck",
- "Page.setBypassCSP",
- "Page.setDocumentContent",
- "Page.setDownloadBehavior",
- "Page.setInterceptFileChooserDialog",
- "Page.setLifecycleEventsEnabled",
- "Page.startScreencast",
- "Page.stopLoading",
- "Page.stopScreencast",
- "Performance.disable",
- "Performance.enable",
- "Performance.getMetrics",
- "Profiler.disable",
- "Profiler.enable",
- "Profiler.start",
- "Profiler.stop",
- "Runtime.addBinding",
- "Runtime.awaitPromise",
- "Runtime.callFunctionOn",
- "Runtime.compileScript",
- "Runtime.disable",
- "Runtime.discardConsoleEntries",
- "Runtime.enable",
- "Runtime.evaluate",
- "Runtime.getProperties",
- "Runtime.globalLexicalScopeNames",
- "Runtime.releaseObject",
- "Runtime.releaseObjectGroup",
- "Runtime.removeBinding",
- "Runtime.runScript",
- "Security.disable",
- "Security.enable",
- "Security.setIgnoreCertificateErrors",
- "Storage.clearCookies",
- "Storage.clearDataForOrigin",
- "Storage.getCookies",
- "Storage.getUsageAndQuota",
- "Storage.setCookies",
- "Target.activateTarget",
- "Target.attachToTarget",
- "Target.closeTarget",
- "Target.createBrowserContext",
- "Target.createTarget",
- "Target.detachFromTarget",
- "Target.disposeBrowserContext",
- "Target.getBrowserContexts",
- "Target.getTargetInfo",
- "Target.getTargets",
- "Target.sendMessageToTarget",
- "Target.setAutoAttach",
- "Target.setDiscoverTargets",
- }
-)
-
-
-def _cdp_method_tags(method: str) -> tuple[str, str]:
- if method in _KNOWN_CDP_METHODS:
- return method, method.split(".", 1)[0]
- domain = method.split(".", 1)[0]
- return "other", domain if domain in KNOWN_CDP_DOMAINS else "other"
-
-
-def _track_command(command_starts: dict[int, tuple[str, float]], upstream_id: int, method: str) -> None:
- if len(command_starts) >= _MAX_PENDING_COMMANDS and upstream_id not in command_starts:
- command_starts.pop(next(iter(command_starts)), None)
- command_starts[upstream_id] = (method, time.monotonic())
-
-
-def _announcement_target_id(event: CdpFrame) -> str | None:
- target_info = (event.params or {}).get("targetInfo") if isinstance(event, (CdpCommand, CdpEvent)) else None
- target_id = target_info.get("targetId") if isinstance(target_info, dict) else None
- return target_id if isinstance(target_id, str) and target_id else None
-
-
-def _pending_attach_owner(shared: _SharedUpstream, event: CdpFrame) -> str | None:
- """The client whose in-flight explicit attach this announcement answers: the oldest
- outstanding attach for that target. Consumes that intent."""
- target_id = _announcement_target_id(event)
- if target_id is None:
- return None
- for index, intent in enumerate(shared.attach_intents):
- if intent.target_id == target_id:
- shared.attach_intents.pop(index)
- return intent.client_key
- return None
-
-
-def _has_pending_explicit_attach(shared: _SharedUpstream, event: CdpFrame) -> bool:
- """Whether an explicit attach for this announcement's target is still in flight,
- WITHOUT consuming its intent — detects the autoAttach/explicit ambiguity."""
- target_id = _announcement_target_id(event)
- if target_id is None:
- return False
- return any(intent.target_id == target_id for intent in shared.attach_intents)
-
-
-def _owns_session(shared: _SharedUpstream, client_key: str, cdp_session_id: str | None) -> bool:
- """Whether this client may drive that cdp session. An unowned session is left to
- the browser to accept or reject, exactly as it would without the proxy."""
- if cdp_session_id is None:
- return True
- owner_key = shared.session_owner.get(cdp_session_id)
- return owner_key is None or owner_key == client_key
-
-
-def _owns_addressed_sessions(shared: _SharedUpstream, client_key: str, command: CdpCommand) -> bool:
- """Whether this client may send this command at all. A session is addressed either
- by the frame's own sessionId or, for the target-plumbing commands, inside params —
- both routes reach a co-tenant's session and both are checked."""
- if not _owns_session(shared, client_key, command.session_id):
- return False
- if command.method in _SESSION_PARAM_METHODS:
- return _owns_session(shared, client_key, params_session_id(command))
- return True
-
-
-def _record_attach_intent(shared: _SharedUpstream, intent: _AttachIntent) -> None:
- # An intent whose response never arrives would otherwise sit here until the client
- # leaves. At the cap, reclaim a slot at the CALLER's own expense — its oldest intent,
- # never a co-tenant's unreconciled announcement — mirroring the remapper's
- # owner-scoped _evict_one_for. A caller with nothing of its own to drop yields the
- # new intent (its announcement falls back to autoAttach/unowned routing) rather than
- # evicting another client's.
- while len(shared.attach_intents) >= _MAX_PENDING_COMMANDS:
- own = next((i for i, held in enumerate(shared.attach_intents) if held.client_key == intent.client_key), None)
- if own is None:
- return
- shared.attach_intents.pop(own)
- shared.attach_intents.append(intent)
-
-
-def _track_attach_intent(shared: _SharedUpstream, client_key: str, command: CdpCommand, upstream_id: int) -> None:
- """Record who asked for the sessions Chrome is about to announce.
-
- Both routes matter, because the attach EVENT arrives before any reply that could
- identify its owner: an explicit attachToTarget is matched back by targetId, while
- browser-level autoAttach is credited to the first client that enabled it. A
- session-scoped setAutoAttach needs no record — its children inherit that session's
- owner.
- """
- if command.session_id is not None:
- return
- if command.method == _ATTACH_TO_TARGET_METHOD:
- target_id = (command.params or {}).get("targetId")
- if isinstance(target_id, str) and target_id:
- _record_attach_intent(shared, _AttachIntent(upstream_id, client_key, target_id, False, False))
- return
- if command.method != _SET_AUTO_ATTACH_METHOD:
- return
- if (command.params or {}).get("autoAttach"):
- enabling = client_key not in shared.autoattach_clients
- if enabling:
- shared.autoattach_clients.append(client_key)
- _record_attach_intent(shared, _AttachIntent(upstream_id, client_key, None, enabling, False))
- else:
- # Optimistically drop the client's autoAttach ownership, but record the disable so
- # a rejected command can restore it: without the intent the browser would stay in
- # autoAttach while the proxy believed the client opted out, dropping/misrouting its
- # later attach announcements.
- disabling = client_key in shared.autoattach_clients
- if disabling:
- shared.autoattach_clients.remove(client_key)
- _record_attach_intent(shared, _AttachIntent(upstream_id, client_key, None, False, disabling))
-
-
-def _reconcile_attach_intent(shared: _SharedUpstream, upstream_id: int, *, failed: bool) -> None:
- """Settle a tentative intent against its response.
-
- A command that errored was never announced, so its intent must go — left behind, it
- would pair with another client's later attach to the same target and hand them each
- other's session. A successful intent is simply spent (its announcement either
- consumed it already, or autoAttach explained the announcement instead).
- """
- for index, intent in enumerate(shared.attach_intents):
- if intent.upstream_id != upstream_id:
- continue
- shared.attach_intents.pop(index)
- if failed:
- # A rejected command never took effect on the browser, so undo the ownership
- # change THIS command made and nothing else — a co-tenant's (or this client's
- # own later, successful) state must survive: drop an enable, restore a disable.
- if intent.enabled_autoattach and intent.client_key in shared.autoattach_clients:
- shared.autoattach_clients.remove(intent.client_key)
- elif intent.disabled_autoattach and intent.client_key not in shared.autoattach_clients:
- shared.autoattach_clients.append(intent.client_key)
- return
-
-
-# Query param and path-marker names a client may use to carry the API key when
-# it cannot set a WS header (e.g. puppeteer). Path scheme: /<marker>/<key>/<session_id>.
-_API_KEY_QUERY_PARAMS = ("x-api-key", "api-key", "api_key", "apikey", "token", "access_token")
-_PATH_CREDENTIAL_MARKERS = ("apikey", "api-key", "api_key", "key", "token")
-
-
-def _parse_request(headers: Mapping[str, str], request_target: str) -> tuple[str, dict[str, str]]:
- """Split a client WS request into (session_id, credentials).
-
- Credentials may ride in a header, the query string, or a marked path segment;
- the raw request target can therefore carry a secret and must never be logged.
- The session id is always the final path segment.
- """
- split = urlsplit(request_target)
- credentials: dict[str, str] = {}
- lowered = {str(name).lower(): value for name, value in headers.items()}
- if lowered.get("x-api-key"):
- credentials["x-api-key"] = lowered["x-api-key"]
- if lowered.get("authorization"):
- credentials["authorization"] = lowered["authorization"]
-
- query = parse_qs(split.query)
- for name in _API_KEY_QUERY_PARAMS:
- values = query.get(name)
- if values and values[0]:
- credentials.setdefault("x-api-key", values[0])
- break
-
- segments = [segment for segment in split.path.split("/") if segment]
- credential_segment: int | None = None
- for index in range(len(segments) - 1):
- if segments[index].lower() in _PATH_CREDENTIAL_MARKERS:
- credentials.setdefault("x-api-key", segments[index + 1])
- credential_segment = index + 1
- break
- # The session id is the final path segment — unless that segment was itself
- # consumed as a credential (a path with no distinct trailing session id), in
- # which case the secret must never land in the session-id argument.
- if segments and credential_segment != len(segments) - 1:
- session_id = segments[-1]
- else:
- session_id = ""
- return session_id, credentials
-
-
-def _split_discovery_target(request_target: str) -> tuple[str, str] | None:
- """Split a discovery GET into (base_target, suffix) or None if not discovery.
-
- base_target is the request target with the /json* suffix (and one optional
- trailing slash — Playwright appends `json/version/`) removed, query string
- preserved, so it re-parses through _parse_request exactly like a WS target.
- """
- split = urlsplit(request_target)
- path = split.path[:-1] if split.path.endswith("/") and split.path != "/" else split.path
- for suffix in _DISCOVERY_SUFFIXES:
- if path == suffix or path.endswith(suffix):
- base_path = path[: -len(suffix)] or "/"
- base_target = base_path if not split.query else f"{base_path}?{split.query}"
- return base_target, suffix
- return None
-
-
-def _proxy_scoped_ws_url(host: str, forwarded_proto: str | None, base_target: str) -> str:
- """Point the client back at this proxy — never the upstream URL. Preserves the
- client's own credential-bearing path/query so a header-less client (puppeteer)
- keeps its URL-carried key on the subsequent WS connect."""
- # XFP is a comma-chain from proxies ("https, http"); the first hop is the client
- # scheme. Compare case-insensitively so "HTTPS"/" https" still yield wss.
- first_proto = (forwarded_proto or "").split(",", 1)[0].strip().lower()
- scheme = "wss" if first_proto == "https" else "ws"
- return f"{scheme}://{host}{base_target}"
-
-
-def _discovery_payload(suffix: str, ws_url: str, session_id: str) -> dict[str, object] | list[dict[str, object]]:
- if suffix == "/json/version":
- return {"Browser": "Skyvern-CDP-Proxy", "Protocol-Version": "1.3", "webSocketDebuggerUrl": ws_url}
- return [
- {
- "id": session_id or "proxy",
- "type": "page",
- "title": "Skyvern CDP Proxy",
- "url": "about:blank",
- "webSocketDebuggerUrl": ws_url,
- }
- ]
-
-
-def _json_response(body_text: str) -> Response:
- body = body_text.encode()
- # no-store: the webSocketDebuggerUrl can carry a header-less client's own
- # URL-borne API key, so a shared intermediary must never retain the body.
- headers = Headers(
- [
- ("Date", email.utils.formatdate(usegmt=True)),
- ("Connection", "close"),
- ("Cache-Control", "no-store"),
- ("Pragma", "no-cache"),
- ("Content-Length", str(len(body))),
- ("Content-Type", "application/json; charset=utf-8"),
- ]
- )
- return Response(HTTPStatus.OK.value, HTTPStatus.OK.phrase, headers, body)
-
-
-class CdpProxyServer:
- def __init__(
- self,
- upstream: UpstreamBrowserPort,
- sessions: SessionRegistryPort,
- auth: AuthPort,
- metrics: MetricsPort,
- event_policy: EventPolicyPort,
- pipeline: MiddlewarePipeline | None = None,
- host: str = "0.0.0.0",
- port: int = 9223,
- client_queue_maxsize: int | None = None,
- client_backpressure_close_after: int | None = None,
- client_queue_maxbytes: int | None = None,
- ) -> None:
- self._upstream = upstream
- self._sessions = sessions
- self._auth = auth
- self._metrics = metrics
- self._event_policy = event_policy
- self._pipeline = pipeline or MiddlewarePipeline()
- self._host = host
- self._port = port
- env_maxsize, env_close_after, env_maxbytes = resolve_client_queue_limits()
- self._client_queue_maxsize = (
- _clamp_client_bound(client_queue_maxsize) if client_queue_maxsize is not None else env_maxsize
- )
- self._client_backpressure_close_after = (
- _clamp_client_bound(client_backpressure_close_after)
- if client_backpressure_close_after is not None
- else env_close_after
- )
- self._client_queue_maxbytes = (
- _clamp_client_bound(client_queue_maxbytes, _MAX_CLIENT_QUEUE_MAXBYTES)
- if client_queue_maxbytes is not None
- else env_maxbytes
- )
- self._max_pending_requests = _MAX_PENDING_COMMANDS
- # One shared upstream connection per resolved browser target, ref-counted
- # across clients. The lock guards only the dict + refcount (never held across
- # a dial), so first-client dials of different browsers never serialize.
- self._shared_upstreams: dict[str, _SharedUpstream] = {}
- self._shared_upstreams_lock = asyncio.Lock()
-
- async def serve_forever(self) -> None:
- loop = asyncio.get_running_loop()
- stop = asyncio.Event()
- installed: list[signal.Signals] = []
- # Handlers go in BEFORE the listener opens so no signal can land in a
- # window where connections are being accepted but drain is not armed.
- for sig in (signal.SIGTERM, signal.SIGINT):
- try:
- loop.add_signal_handler(sig, stop.set)
- installed.append(sig)
- except (NotImplementedError, RuntimeError, ValueError):
- # Platforms/embeddings without loop signal support (Windows,
- # non-main threads) still serve; drain then relies on cancellation.
- pass
- server = await websockets.serve(
- self._handle_client, self._host, self._port, max_size=None, process_request=self._process_request
- )
- LOG.info("CDP proxy listening", host=self._host, port=self._port)
- try:
- await stop.wait()
- # Drain: close the listener so no new client lands here, keep every
- # established relay running, and only force-close stragglers once the
- # drain budget is spent. The orchestrator's grace period is the outer
- # bound; this inner one exists so a healthy drain ends with clean
- # 1001 closes rather than a SIGKILL.
- drain_timeout = _positive_env_int(
- "CDP_PROXY_DRAIN_TIMEOUT_SECONDS", _DEFAULT_DRAIN_TIMEOUT_SECONDS, _MAX_DRAIN_TIMEOUT_SECONDS
- )
- LOG.info("CDP proxy draining", drain_timeout_seconds=drain_timeout)
- server.close(close_connections=False)
- # Re-arm the handlers: a second signal ends the drain early, so a
- # dev Ctrl-C (or an operator's repeated stop) stays interruptible
- # instead of sitting out the full drain budget.
- force = asyncio.Event()
- for sig in installed:
- loop.remove_signal_handler(sig)
- loop.add_signal_handler(sig, force.set)
- drained = asyncio.create_task(server.wait_closed())
- forced = asyncio.create_task(force.wait())
- try:
- done, _ = await asyncio.wait(
- {drained, forced}, timeout=drain_timeout, return_when=asyncio.FIRST_COMPLETED
- )
- if drained not in done:
- # Server.close() is idempotent — a second call cannot upgrade the
- # drain to a force-close — so stragglers are closed directly. Each
- # close() bounds its own handshake, so this always terminates.
- # `handlers` is an undocumented websockets.asyncio Server attribute
- # (its _close() iterates the same dict); the lifecycle tests here
- # exercise it, so a websockets bump that renames it fails loudly.
- reason = "second signal" if forced in done else "drain timeout expired"
- LOG.warning("CDP proxy force-closing remaining connections", reason=reason)
- stragglers = [asyncio.create_task(conn.close(1001)) for conn in list(server.handlers)]
- if stragglers:
- await asyncio.wait(stragglers)
- finally:
- for task in (drained, forced):
- task.cancel()
- with contextlib.suppress(asyncio.CancelledError):
- await task
- finally:
- for sig in installed:
- loop.remove_signal_handler(sig)
- server.close(close_connections=True)
- await server.wait_closed()
- LOG.info("CDP proxy stopped")
-
- async def _process_request(self, connection: websockets.ServerConnection, request: Request) -> Response | None:
- """Serve the pre-upgrade CDP discovery GETs; return None to let a real WS
- upgrade proceed. Authenticated the same way as the WS connection."""
- try:
- is_health_probe = urlsplit(request.path).path == _HEALTH_PATH
- except ValueError:
- is_health_probe = False
- if is_health_probe:
- # Unauthenticated on purpose: probes carry no credentials, and the
- # response reveals nothing but liveness. Draining needs no branch —
- # the closed listener refuses the probe's connect outright.
- return connection.respond(HTTPStatus.OK, "ok")
- try:
- split = _split_discovery_target(request.path)
- if split is None:
- return None
- base_target, suffix = split
- headers = dict(request.headers)
- session_id, credentials = _parse_request(headers, base_target)
- except (ValueError, MultipleValuesError):
- self._metrics.increment(
- _METRIC_CONNECTION_REJECTED, tags={"reason": "malformed_request", "phase": "discovery"}
- )
- return connection.respond(HTTPStatus.UNAUTHORIZED, "unauthorized")
- if await self._auth.authenticate(credentials) is None:
- self._metrics.increment(_METRIC_CONNECTION_REJECTED, tags={"reason": "unauthorized", "phase": "discovery"})
- return connection.respond(HTTPStatus.UNAUTHORIZED, "unauthorized")
- ws_url = _proxy_scoped_ws_url(
- headers.get("host", f"{self._host}:{self._port}"), headers.get("x-forwarded-proto"), base_target
- )
- return _json_response(json.dumps(_discovery_payload(suffix, ws_url, session_id)))
-
- async def _handle_client(self, ws: websockets.ServerConnection) -> None:
- try:
- headers = dict(ws.request.headers) if ws.request else {}
- request_target = ws.request.path if ws.request else "/"
- session_id, credentials = _parse_request(headers, request_target)
- except (ValueError, MultipleValuesError):
- # A malformed request target (urlsplit) or duplicate credential
- # headers (dict(Headers)) are rejected as an auth failure — a clean
- # close, never an uncaught error, and before any session lookup.
- self._metrics.increment(_METRIC_CONNECTION_REJECTED, tags={"reason": "malformed_request"})
- await ws.close(code=_UNAUTHORIZED_CLOSE_CODE, reason="unauthorized")
- return
- # The client credential is consumed here and never reaches the upstream
- # dial below (which uses the registry's operator routing/headers only).
- principal = await self._auth.authenticate(credentials)
- if principal is None:
- self._metrics.increment(_METRIC_CONNECTION_REJECTED, tags={"reason": "unauthorized"})
- await ws.close(code=_UNAUTHORIZED_CLOSE_CODE, reason="unauthorized")
- return
- # Resolved once per connection — routing never runs per message.
- resolution = await self._sessions.resolve(session_id)
- # authorize() rejects a non-owning org (and an unknown session) with the
- # exact same close as a truly unknown id (the 404-not-403 convention), so
- # a foreign session is indistinguishable from a nonexistent one.
- if not self._auth.authorize(principal, resolution):
- self._metrics.increment(_METRIC_CONNECTION_REJECTED, tags={"reason": "unknown_session"})
- await ws.close(code=_UNKNOWN_CLOSE_CODE, reason=_UNKNOWN_CLOSE_REASON)
- return
- resolved = resolution.session
- if resolved is None:
- code, reason = _LIFECYCLE_CLOSE_CODES.get(resolution.status, (_UNKNOWN_CLOSE_CODE, _UNKNOWN_CLOSE_REASON))
- self._metrics.increment(_METRIC_CONNECTION_REJECTED, tags={"reason": resolution.status.value})
- await ws.close(code=code, reason=reason)
- return
- # resolved.upstream_adapter selects among configured upstream ports once
- # multi-adapter wiring lands later in the epic; a single port is injected today.
- session = ProxySession(
- session_id=resolved.session_id,
- upstream_ws_url=resolved.upstream_ws_url,
- principal=principal,
- connect_headers=resolved.connect_headers,
- )
- session_tags = {**_org_tag(session), "session_id": session.session_id}
- # client_connected/client_disconnected are attempt/teardown counters: on an
- # upstream-connect failure below, client_connected has already fired but no
- # disconnected follows, so they can skew by 1. active_sessions (the gauge) is
- # the authoritative currently-connected signal and stays balanced regardless.
- self._metrics.increment(_METRIC_CLIENT_CONNECTED, tags=session_tags)
- try:
- # Ref-counted: the shared upstream opens on the first client for this
- # browser and is reused (not re-dialed) by every later client on it.
- shared = await self._acquire_shared(session)
- except UpstreamConnectError:
- # Upstream dial failed (vendor down, transient, config): reject the
- # client cleanly and record it. Reconnection is out of scope (SKY-12499).
- self._metrics.increment(_METRIC_CONNECTION_REJECTED, tags={"reason": "upstream_unavailable"})
- await ws.close(code=_UPSTREAM_UNAVAILABLE_CLOSE_CODE, reason="upstream unavailable")
- return
- # A ref is now held on `shared`; every exit path below (including cancellation
- # at the attach seam) must release it, or the entry + socket + reader leak.
- client_key = shared.next_client_key()
- relays: tuple[asyncio.Task[None], ...] = ()
- gauge_up = False
- try:
- # The upstream read loop enqueues here; a dedicated writer drains it to the
- # client socket, so a slow client blocks only its own writer, never the
- # shared read loop or the other clients on this browser.
- outbound = _OutboundQueue(self._client_queue_maxsize, self._client_queue_maxbytes)
- channel = _ClientChannel(client_key, outbound)
- await self._attach_client(shared, channel)
- self._metrics.gauge(_METRIC_ACTIVE_SESSIONS, 1, tags=session_tags)
- gauge_up = True
- writer = asyncio.create_task(self._drain_client_outbound(ws, session, outbound))
- # This client's uplink shares the connection's remapper (keyed by
- # client_key, so responses route back to the right client) and its
- # serialized send.
- relays = (
- asyncio.create_task(
- self._relay_client_to_upstream(
- ws, shared, session, shared.remapper, shared.command_starts, client_key, shared
- )
- ),
- writer,
- asyncio.create_task(channel.wait_closed()),
- )
- await asyncio.wait(relays, return_when=asyncio.FIRST_COMPLETED)
- if channel.closed.is_set():
- if channel.drain:
- # Clean upstream close: flush queued frames (e.g. a final command
- # response) before teardown; a client that cannot take the drain
- # in time is closed 1013 rather than losing the response silently.
- await self._drain_then_close(ws, outbound, writer)
- elif channel.close_code is not None:
- with contextlib.suppress(websockets.ConnectionClosed):
- await ws.close(code=channel.close_code, reason=channel.close_reason)
- except UpstreamConnectError:
- # _attach_client found the shared upstream defunct (its reader had already
- # exited): reject cleanly like a failed dial rather than joining a dead reader.
- self._metrics.increment(_METRIC_CONNECTION_REJECTED, tags={"reason": "upstream_unavailable"})
- with contextlib.suppress(websockets.ConnectionClosed):
- await ws.close(code=_UPSTREAM_UNAVAILABLE_CLOSE_CODE, reason="upstream unavailable")
- finally:
- # Cancellation-safe teardown: whatever exit path (a relay finishing,
- # _handle_client cancelled, or the attach rejection above), cancel and reap
- # EVERY child so none leak and every child exception is retrieved.
- for task in relays:
- task.cancel()
- if relays:
- with contextlib.suppress(asyncio.CancelledError):
- await asyncio.gather(*relays, return_exceptions=True)
- shared.channels.pop(client_key, None)
- # Drop everything keyed to this client: pending id mappings, learned
- # cdp-session owners, and its autoAttach claim (so a co-tenant that also
- # asked inherits the sessions Chrome attaches from here on). No await runs
- # between these, so the shared reader cannot observe a half-cleaned client.
- # The command-start map is not exactly evicted here; it stays bounded by
- # _MAX_PENDING_COMMANDS.
- shared.remapper.clear_client(client_key)
- for cdp_session_id in [sid for sid, owner in shared.session_owner.items() if owner == client_key]:
- del shared.session_owner[cdp_session_id]
- shared.attach_intents[:] = [intent for intent in shared.attach_intents if intent.client_key != client_key]
- if client_key in shared.autoattach_clients:
- shared.autoattach_clients.remove(client_key)
- if gauge_up:
- # Decrement the gauge before the upstream close I/O — a real adapter's
- # close() can raise, and active_sessions must still net back to 0.
- self._metrics.gauge(_METRIC_ACTIVE_SESSIONS, -1, tags=session_tags)
- self._metrics.increment(_METRIC_CLIENT_DISCONNECTED, tags=session_tags)
- await self._release_shared(session, shared)
-
- async def _acquire_shared(self, session: ProxySession) -> _SharedUpstream:
- """Return the shared connection for this browser, opening it on the first
- client and ref-counting every later one. Concurrent first-clients for the
- same target await one dial via a single-flight connect task; a failed dial
- removes the entry and re-raises UpstreamConnectError to the caller."""
- # LOAD-BEARING INVARIANT: the key is the resolved per-browser upstream cdp_url,
- # so same key ⟹ same browser ⟹ same org (auth authorizes org-per-session before
- # this, and distinct vendor/own-infra browsers are distinct endpoints). A future
- # SHARED-GATEWAY adapter (one URL, per-browser routing in a header/selector) would
- # have to fold that routing identity into the key, or reuse would cross browsers.
- key = session.upstream_ws_url
- async with self._shared_upstreams_lock:
- shared = self._shared_upstreams.get(key)
- if shared is None or shared.defunct:
- # No live entry (or a dead one whose reader exited): dial fresh. A
- # defunct entry is overwritten; its own clients still tear it down.
- shared = _SharedUpstream(
- session=session, remapper=RequestIdRemapper(max_pending=self._max_pending_requests)
- )
- shared.connect_task = asyncio.ensure_future(self._open_shared(session, shared))
- self._shared_upstreams[key] = shared
- shared.refcount += 1
- assert shared.connect_task is not None
- connect_task = shared.connect_task
- try:
- # Shield: one waiter's cancellation must not abort the dial the other
- # co-waiters are still legitimately waiting on. The last waiter out (below)
- # is the one that cancels the dial task.
- await asyncio.shield(connect_task)
- except BaseException:
- async with self._shared_upstreams_lock:
- shared.refcount -= 1
- removed = shared.refcount <= 0 and self._shared_upstreams.get(key) is shared
- if removed:
- del self._shared_upstreams[key]
- if removed:
- # Last waiter out: cancel the (possibly still in-flight, shielded) dial
- # and close any connection it produced, so nothing is left orphaned.
- connect_task.cancel()
- with contextlib.suppress(asyncio.CancelledError):
- await connect_task
- await shared.aclose()
- raise
- return shared
-
- async def _open_shared(self, session: ProxySession, shared: _SharedUpstream) -> None:
- shared.connection = await self._upstream.connect(session)
-
- async def _attach_client(self, shared: _SharedUpstream, channel: _ClientChannel) -> None:
- """Register a client and start the shared reader on the first one. The reader
- must not start before a channel exists, or a burst of upstream frames could
- drain to no one and the client would never be signaled on teardown."""
- async with self._shared_upstreams_lock:
- if shared.defunct:
- # The reader exited (upstream died) after we acquired: never join a dead
- # reader — reject so the caller dials fresh instead of hanging.
- raise UpstreamConnectError("shared upstream is no longer connected")
- shared.channels[channel.client_key] = channel
- if shared.reader is None:
- shared.reader = asyncio.create_task(self._read_shared_upstream(shared))
-
- async def _evict_shared(self, shared: _SharedUpstream) -> None:
- """Mark a connection defunct and drop it from the pool (by identity) so no new
- client can join its dead reader. Called on every reader exit; idempotent."""
- shared.defunct = True
- key = shared.session.upstream_ws_url
- async with self._shared_upstreams_lock:
- if self._shared_upstreams.get(key) is shared:
- del self._shared_upstreams[key]
- # The session is gone for good; a policy holding per-session state would
- # otherwise keep one entry per session the proxy has ever served.
- # Forgetting only the first client's session_id relies on one upstream_ws_url
- # == one session_id (multi-client fan-in shares it), which is what the shared
- # connection is keyed on. A topology that put distinct session_ids on one url
- # would strand the others' policy state here — see the shared-gateway keying
- # carry-forward (SKY-12535); it would have to forget per owning session.
- self._event_policy.forget(shared.session.session_id)
-
- async def _release_shared(self, session: ProxySession, shared: _SharedUpstream) -> None:
- key = session.upstream_ws_url
- async with self._shared_upstreams_lock:
- shared.refcount -= 1
- if shared.refcount > 0:
- return
- if self._shared_upstreams.get(key) is shared:
- del self._shared_upstreams[key]
- # Last client left: reap the reader and close the socket. aclose is idempotent.
- await shared.aclose()
-
- async def _read_shared_upstream(self, shared: _SharedUpstream) -> None:
- """The single reader for one shared upstream: decode each frame, process it
- once, and route it to the owning client(s). On upstream death every client
- sharing this browser is closed deterministically (drain-then-close)."""
- assert shared.connection is not None
- session = shared.session
- direction = Direction.UPSTREAM_TO_CLIENT
- try:
- while True:
- try:
- frame = decode_frame(await shared.connection.receive())
- except FrameDecodeError:
- self._metrics.increment(
- _METRIC_FRAME_DECODE_ERRORS, tags={**_org_tag(session), "direction": direction.value}
- )
- # A garbled upstream stream is fatal for every multiplexed client.
- self._shutdown_clients(shared, drain=False, code=1003, reason="invalid CDP frame")
- return
- result = await self._process_upstream_frame(
- frame, session, shared.remapper, shared.command_starts, shared, upstream=shared
- )
- if result is None:
- continue
- client_key, processed = result
- self._route_upstream_frame(shared, client_key, processed)
- except UpstreamClosedError:
- # Deterministic teardown: every client on this browser drains its queued
- # frames then closes. Reconnection/resumption would re-dial + re-attach
- # here instead — out of scope for v1 (SKY-12499), seam only.
- self._shutdown_clients(shared, drain=True, code=None, reason="")
- except Exception as exc:
- # An unexpected reader failure still tears every client down (rather than
- # leaving them hung on a dead socket). Log the type only — a frame's
- # content must never reach logs. Exhaustive error taxonomy is SKY-12500.
- LOG.warning("shared upstream reader failed", error_type=type(exc).__name__)
- self._shutdown_clients(shared, drain=False, code=_UPSTREAM_UNAVAILABLE_CLOSE_CODE, reason="upstream error")
- finally:
- # Reader is gone: evict so no new client joins its dead socket. Nothing
- # between the reader's exit and _evict_shared setting `defunct` yields, so an
- # acquirer racing this teardown always sees it. Existing clients were signaled
- # above; a new one is rejected at _attach_client's defunct check.
- await self._evict_shared(shared)
-
- def _route_upstream_frame(self, shared: _SharedUpstream, client_key: str | None, processed: CdpFrame) -> None:
- if client_key == PROXY_CLIENT_KEY:
- # The proxy's own command came back: consumed here. A client must never see
- # a response for an id it did not send.
- self._metrics.increment(
- _METRIC_FRAMES_DROPPED,
- tags={
- **_org_tag(shared.session),
- "direction": Direction.UPSTREAM_TO_CLIENT.value,
- "reason": "proxy_command",
- },
- )
- return
- if client_key is not None:
- # A response for a specific client. Flat-session attach: an attach reply
- # carries the new cdp sessionId, so learn which client owns that session
- # (explicit Target.attachToTarget).
- if isinstance(processed, CdpResponse) and processed.result:
- cdp_session_id = processed.result.get("sessionId")
- if isinstance(cdp_session_id, str) and cdp_session_id:
- shared.session_owner[cdp_session_id] = client_key
- channel = shared.channels.get(client_key)
- targets: tuple[_ClientChannel, ...] = (channel,) if channel is not None else ()
- else:
- targets = self._event_targets(shared, processed)
- if not targets:
- return
- wire = encode_frame(processed)
- for channel in targets:
- self._deliver_to_channel(shared, channel, processed, wire)
-
- def _event_targets(self, shared: _SharedUpstream, processed: CdpFrame) -> tuple[_ClientChannel, ...]:
- """Route an upstream-initiated frame to the client that owns its session.
-
- A session-scoped frame is delivered ONLY to its owner: on one shared
- connection every co-tenant would otherwise see another client's page traffic.
- """
- method = processed.method if isinstance(processed, CdpEvent) else None
- if method == TARGET_ATTACHED_EVENT:
- owner_key = self._learn_attached_session(shared, processed)
- elif method == TARGET_DETACHED_EVENT:
- # The session is gone: hand the notice to its owner and forget it. This is
- # what keeps session_owner flat under a long-lived client's attach/detach
- # churn. Target.targetDestroyed needs no separate GC — Chrome detaches a
- # target's sessions before destroying it — and a client's remaining entries
- # are dropped wholesale when it disconnects.
- detached = params_session_id(processed)
- owner_key = shared.session_owner.pop(detached, None) if detached else None
- elif processed.session_id:
- owner_key = shared.session_owner.get(processed.session_id)
- else:
- # Genuinely browser-level (no sessionId, e.g. Target.targetCreated): it
- # describes the browser rather than one client's session, so the fan-out to
- # every client on this connection is intentional.
- return tuple(shared.channels.values())
- if owner_key is None:
- # Nobody owns this session, so there is no recipient it could go to without
- # handing it to a non-owner. Dropping is the only safe routing.
- self._metrics.increment(
- _METRIC_FRAMES_DROPPED,
- tags={
- **_org_tag(shared.session),
- "direction": Direction.UPSTREAM_TO_CLIENT.value,
- "reason": "unowned_session",
- },
- )
- return ()
- owner = shared.channels.get(owner_key)
- return (owner,) if owner is not None else ()
-
- def _learn_attached_session(self, shared: _SharedUpstream, event: CdpFrame) -> str | None:
- """Learn who owns a newly attached session: for a nested attach the parent
- session's owner, at browser level the client that enabled autoAttach, else the
- client whose explicit attach this answers. Returns the owner to deliver the
- announcement to, or None to withhold it (still recording durable ownership).
-
- An announcement carries nothing that distinguishes an autoAttach from an explicit
- attach, so while autoAttach is live the two are indistinguishable here and
- autoAttach wins the DURABLE assignment: the autoAttach half has no response that
- could ever correct it, whereas an explicit attach's response is authoritative and
- reassigns its own session (see `_route_upstream_frame`). Getting that backwards
- durably hands one client's auto-attached session to a co-tenant. But when a
- co-tenant has an explicit attach IN FLIGHT for this same target, the announcement
- itself could be theirs — so it is withheld (fail-closed) rather than delivered to
- the autoAttach client, closing the cross-tenant announcement window. Telling the
- two apart to deliver correctly needs the interception seam (SKY-12535).
- """
- deliver = True
- if event.session_id is not None:
- owner_key = shared.session_owner.get(event.session_id)
- elif shared.autoattach_clients:
- owner_key = shared.autoattach_clients[0]
- if _has_pending_explicit_attach(shared, event):
- deliver = False
- else:
- owner_key = _pending_attach_owner(shared, event)
- attached = params_session_id(event)
- if attached is not None and owner_key is not None:
- shared.session_owner[attached] = owner_key
- return owner_key if deliver else None
-
- def _deliver_to_channel(
- self, shared: _SharedUpstream, channel: _ClientChannel, processed: CdpFrame, wire: str
- ) -> None:
- if channel.closed.is_set():
- # Already signaled for teardown; don't queue or re-count it.
- return
- try:
- channel.outbound.put_nowait(wire)
- channel.overflow_streak = 0
- return
- except asyncio.QueueFull:
- pass
- session = shared.session
- direction = Direction.UPSTREAM_TO_CLIENT
- # An ordinary event is droppable (clients tolerate event loss); a command
- # response and a session lifecycle event are not — losing either desyncs the
- # client. A client too slow to take one, or one that stays overflowed, is closed
- # 1013 without stalling the shared reader or the other clients.
- if isinstance(processed, CdpEvent) and processed.method not in LIFECYCLE_EVENTS:
- # ponytail: a screencast frame tail-dropped HERE is never acked (unlike a
- # policy-withheld one, which is — SKY-12502), so a client that overflows
- # without hitting the close threshold can leave its own stream stalled on the
- # frames it never saw. Pre-existing and self-inflicted by a client too slow to
- # drain; acking from this sync path needs an owned task. Fix when a real slow
- # client is seen wedging a screencast, not before.
- channel.overflow_streak += 1
- self._metrics.increment(
- _METRIC_FRAMES_DROPPED,
- tags={**_org_tag(session), "direction": direction.value, "reason": "backpressure"},
- )
- if channel.overflow_streak < self._client_backpressure_close_after:
- return
- self._metrics.increment(
- _METRIC_CLIENT_BACKPRESSURE_CLOSED, tags={**_org_tag(session), "session_id": session.session_id}
- )
- channel.signal(drain=False, code=_BACKPRESSURE_CLOSE_CODE, reason="client too slow")
-
- def _shutdown_clients(self, shared: _SharedUpstream, *, drain: bool, code: int | None, reason: str) -> None:
- for channel in list(shared.channels.values()):
- channel.signal(drain=drain, code=code, reason=reason)
-
- async def _drain_then_close(
- self, ws: websockets.ServerConnection, outbound: _OutboundQueue, writer: asyncio.Task[None]
- ) -> None:
- if writer.done():
- return
-
- async def _flush() -> None:
- await outbound.put_sentinel()
- await writer # FIFO: the writer sends every queued frame, then the sentinel stops it
-
- try:
- await asyncio.wait_for(_flush(), timeout=_OUTBOUND_DRAIN_TIMEOUT_SECONDS)
- except (asyncio.TimeoutError, websockets.ConnectionClosed, UpstreamClosedError):
- with contextlib.suppress(websockets.ConnectionClosed):
- await ws.close(code=_BACKPRESSURE_CLOSE_CODE, reason="client too slow")
-
- def _record_relayed(self, session: ProxySession, direction: Direction, wire: str) -> None:
- tags = {**_org_tag(session), "direction": direction.value}
- self._metrics.increment(_METRIC_FRAMES_RELAYED, tags=tags)
- self._metrics.increment(_METRIC_BYTES_RELAYED, amount=len(wire), tags=tags)
-
- def _record_command_latency(self, session: ProxySession, started: tuple[str, float] | None) -> None:
- if started is None:
- return
- method, start = started
- cdp_method, cdp_domain = _cdp_method_tags(method)
- self._metrics.observe(
- _METRIC_COMMAND_LATENCY,
- time.monotonic() - start,
- tags={**_org_tag(session), "cdp_method": cdp_method, "cdp_domain": cdp_domain},
- )
-
- async def _relay_client_to_upstream(
- self,
- ws: websockets.ServerConnection,
- connection: UpstreamConnection,
- session: ProxySession,
- remapper: RequestIdRemapper,
- command_starts: dict[int, tuple[str, float]],
- client_key: str = "client",
- shared: _SharedUpstream | None = None,
- ) -> None:
- # shared is None only on the pinned single-client relay contract, where there
- # is no co-tenant to steer another client's session or race its autoAttach.
- direction = Direction.CLIENT_TO_UPSTREAM
- context: InterceptContext | None = None
- if self._pipeline.has_interceptors:
-
- async def send_proxy_command(command: CdpCommand) -> None:
- # The proxy's reserved id lane: the response is consumed by the
- # upstream reader's PROXY_CLIENT_KEY branch, never sent to a client.
- upstream_command = remapper.to_upstream_as_proxy(command)
- try:
- await connection.send(encode_frame(upstream_command))
- except BaseException:
- # ANY post-allocation failure (encode, send, cancellation) must
- # free the mapping: proxy-lane entries are never evictable by
- # clients, so a leaked one is a slot lost to co-tenants forever.
- remapper.discard(upstream_command.id)
- raise
-
- context = InterceptContext(send_proxy_command=send_proxy_command)
- async for raw in ws:
- try:
- frame = decode_frame(raw)
- except FrameDecodeError:
- self._metrics.increment(
- _METRIC_FRAME_DECODE_ERRORS, tags={**_org_tag(session), "direction": direction.value}
- )
- await ws.close(code=1003, reason="invalid CDP frame")
- return
- processed = await self._pipeline.process(frame, direction, session)
- if processed is None:
- self._metrics.increment(
- _METRIC_FRAMES_DROPPED,
- tags={**_org_tag(session), "direction": direction.value, "reason": "pipeline"},
- )
- continue
- if isinstance(processed, CdpCommand):
- if shared is not None and not _owns_addressed_sessions(shared, client_key, processed):
- self._refuse_foreign_session(shared, session, client_key, processed)
- continue
- if context is not None:
- # After the ownership refusal (an interceptor never sees a
- # foreign-session command) and before remapping (a synthesized
- # response reuses the client's own id; only a forwarded command
- # ever allocates an upstream one).
- verdict = await self._intercept_command(processed, session, context)
- if isinstance(verdict, CdpResponse):
- await self._deliver_intercept_response(ws, shared, session, client_key, verdict)
- continue
- processed = verdict
- try:
- upstream_command = remapper.to_upstream(client_key, processed)
- except RemapperFullError:
- # This client has flooded the shared pending table and has nothing of
- # its own left to reclaim. Close it rather than admit the command by
- # dropping a co-tenant's promised response.
- self._metrics.increment(
- _METRIC_CLIENT_BACKPRESSURE_CLOSED,
- tags={**_org_tag(session), "session_id": session.session_id},
- )
- await ws.close(code=_BACKPRESSURE_CLOSE_CODE, reason="too many in-flight commands")
- return
- if shared is not None:
- # No await runs between this and the serialized send below, so the
- # queue is ordered exactly as the attaches reach the wire — which is
- # what lets each announcement pair off with the attach that caused it.
- _track_attach_intent(shared, client_key, processed, upstream_command.id)
- # Declared interest is read from what the client actually asked for, so
- # this sees the client's own command before the id is remapped upstream.
- self._event_policy.observe_command(processed, session)
- _track_command(command_starts, upstream_command.id, processed.method)
- processed = upstream_command
- elif isinstance(processed, (CdpEvent, CdpResponse)):
- # A CDP client only ever sends COMMANDS (method + id) to the browser;
- # events and responses flow the other way. A method-bearing frame
- # sent without an id decodes as an event (frames.py), so forwarding a
- # client event raw would let a client smuggle a command past command
- # interception — e.g. a denied Target.sendMessageToTarget stripped of
- # its id would tunnel a denied inner method past the denylist. Dropped,
- # never forwarded: closing the bypass here rather than trusting the
- # browser to reject an id-less command.
- self._metrics.increment(
- _METRIC_FRAMES_DROPPED,
- tags={**_org_tag(session), "direction": direction.value, "reason": "non_command_upstream"},
- )
- continue
- wire = encode_frame(processed)
- await connection.send(wire)
- self._record_relayed(session, direction, wire)
-
- def _refuse_foreign_session(
- self, shared: _SharedUpstream, session: ProxySession, client_key: str, command: CdpCommand
- ) -> None:
- """Answer a command for a session another client owns with CDP's own
- session-not-found error, so the client fails at once instead of hanging on a
- command that was never sent upstream."""
- self._metrics.increment(
- _METRIC_FRAMES_DROPPED,
- tags={
- **_org_tag(session),
- "direction": Direction.CLIENT_TO_UPSTREAM.value,
- "reason": "session_not_owned",
- },
- )
- channel = shared.channels.get(client_key)
- if channel is None:
- return
- refusal = CdpResponse(
- id=command.id,
- error={"code": _SESSION_NOT_FOUND_CODE, "message": _SESSION_NOT_FOUND_MESSAGE},
- session_id=command.session_id,
- )
- self._deliver_to_channel(shared, channel, refusal, encode_frame(refusal))
-
- async def _intercept_command(
- self, command: CdpCommand, session: ProxySession, context: InterceptContext
- ) -> CdpCommand | CdpResponse:
- """Run the interceptor chain over a client command, fail-closed: a raising
- interceptor (or a contract violation) answers the client with a deterministic
- internal error and forwards nothing — a policy hook that fails must never
- fail open into the browser."""
- try:
- outcome = await self._pipeline.intercept(command, session, context)
- except Exception as exc:
- # Type only — a frame's content must never reach logs.
- LOG.warning("command interceptor failed", error_type=type(exc).__name__, cdp_method=command.method)
- self._record_intercepted(session, command, INTERCEPTOR_FAILURE_REASON)
- return interceptor_failure_response(command)
- if isinstance(outcome, SynthesizedResponse):
- self._record_intercepted(session, command, outcome.reason, audit_method=outcome.audit_method)
- return outcome.to_response(command)
- return outcome
-
- def _record_intercepted(
- self, session: ProxySession, command: CdpCommand, reason: str, audit_method: str | None = None
- ) -> None:
- """The audit trail for a command answered at the proxy (SKY-12538): a counter
- under bounded labels plus one structured line naming the exact method. The
- synthesis's audit_method wins over the wrapper it arrived in, so a denial
- tunneled through Target.sendMessageToTarget is recorded against the method
- actually blocked."""
- method = audit_method or command.method
- cdp_method, cdp_domain = _cdp_method_tags(method)
- self._metrics.increment(
- _METRIC_COMMANDS_INTERCEPTED,
- tags={**_org_tag(session), "reason": reason, "cdp_method": cdp_method, "cdp_domain": cdp_domain},
- )
- LOG.info(
- "CDP command intercepted",
- session_id=session.session_id,
- cdp_method=method,
- reason=reason,
- **_org_tag(session),
- )
-
- async def _deliver_intercept_response(
- self,
- ws: websockets.ServerConnection,
- shared: _SharedUpstream | None,
- session: ProxySession,
- client_key: str,
- response: CdpResponse,
- ) -> None:
- wire = encode_frame(response)
- if shared is not None:
- channel = shared.channels.get(client_key)
- if channel is not None:
- # A response is non-droppable in _deliver_to_channel: a client too
- # slow to take it is closed rather than left hanging on the command.
- self._deliver_to_channel(shared, channel, response, wire)
- return
- await ws.send(wire)
- self._record_relayed(session, Direction.UPSTREAM_TO_CLIENT, wire)
-
- async def _pay_screencast_ack(
- self,
- upstream: UpstreamConnection | None,
- remapper: RequestIdRemapper,
- ack: CdpCommand | None,
- ) -> bool:
- """Send an ack the client will not, so the browser releases the next frame. False
- if it could not be sent at all — the caller must then deliver the ORIGINAL frame
- rather than withhold one nothing will ever ack.
-
- The ack rides the proxy's own lane, so its reply is consumed by the reader and a
- client never sees a response for a command it did not send. It is not latency
- tracked: the histogram is meant to describe what CLIENTS asked for.
- """
- if ack is None or upstream is None:
- return False
- try:
- upstream_ack = remapper.to_upstream_as_proxy(ack)
- except RemapperFullError:
- # The pending table is full of responses already promised to clients, and the
- # proxy may not evict one of those to make room for its own command.
- return False
- try:
- await upstream.send(encode_frame(upstream_ack))
- except UpstreamClosedError:
- # The socket died under us; the reader's next receive tears every client down.
- remapper.discard(upstream_ack.id)
- return False
- return True
-
- async def _process_upstream_frame(
- self,
- frame: CdpFrame,
- session: ProxySession,
- remapper: RequestIdRemapper,
- command_starts: dict[int, tuple[str, float]],
- shared: _SharedUpstream | None = None,
- upstream: UpstreamConnection | None = None,
- ) -> tuple[str | None, CdpFrame] | None:
- """Run one upstream frame through the pipeline, remap + latency-account a
- response, and apply the event policy. Returns the routing owner and the
- frame to deliver, or None if the frame was dropped (metric already emitted).
- The owner is the response's client key; None for events/commands (the caller
- routes those by sessionId or broadcast). Shared by the single-client relay
- and the shared multi-client reader so the two never drift.
-
- `upstream` is the sender a withheld screencast frame's ack goes back out on
- (SKY-12502); both callers pass their own, since the ack is owed on whichever
- connection the frame arrived on."""
- direction = Direction.UPSTREAM_TO_CLIENT
- # What the browser is waiting on for THIS frame, read from the frame as it
- # ARRIVED — before any middleware or policy can suppress or alter it. The ack is
- # owed by the original or the stream stalls, so it cannot be derived from whatever
- # comes out the far end: a rewrite would ack a frame the browser never sent, and a
- # suppression would ack nothing at all (SKY-12502).
- owed_ack = screencast_frame_ack(frame)
- awaits_ack = is_screencast_frame(frame)
- processed = await self._pipeline.process(frame, direction, session)
- if processed is None:
- if awaits_ack and not await self._pay_screencast_ack(upstream, remapper, owed_ack):
- # Nothing can ack it, so it must not be suppressed: deliver the original
- # and let the client's own ack keep the stream alive.
- return None, frame
- # A response that will never be delivered still has to free every table its
- # upstream id sits in, or the mapping outlives the exchange.
- if isinstance(frame, CdpResponse):
- command_starts.pop(frame.id, None)
- remapper.discard(frame.id)
- if shared is not None:
- # The suppressed frame still carries its own verdict — settle the
- # intent by it, or a dropped enable rejection would leave the client
- # credited and the next browser-level attach handed to it.
- _reconcile_attach_intent(shared, frame.id, failed=frame.error is not None)
- self._metrics.increment(
- _METRIC_FRAMES_DROPPED, tags={**_org_tag(session), "direction": direction.value, "reason": "pipeline"}
- )
- return None
- if isinstance(processed, CdpResponse):
- if shared is not None:
- # Before to_client rewrites the id back to the client's own.
- _reconcile_attach_intent(shared, processed.id, failed=processed.error is not None)
- mapped = remapper.to_client(processed)
- if mapped is None:
- command_starts.pop(processed.id, None)
- self._metrics.increment(
- _METRIC_FRAMES_DROPPED,
- tags={**_org_tag(session), "direction": direction.value, "reason": "unmatched_response"},
- )
- return None
- self._record_command_latency(session, command_starts.pop(processed.id, None))
- return mapped
- if isinstance(processed, CdpEvent):
- # On the shared path `session` is the first client's, while interest is
- # recorded under each client's own session (see observe_command below).
- # The two agree only because one upstream_ws_url == one session_id, so an
- # interest-relaxable rule (SKY-12501) reads the interest it meant to. A
- # topology putting distinct session_ids on one url must resolve the event
- # owner's session first — see the shared-gateway carry-forward (SKY-12535).
- decision = self._event_policy.decide(processed, session)
- if isinstance(decision, Drop):
- if awaits_ack and not await self._pay_screencast_ack(upstream, remapper, owed_ack):
- # A screencast frame may only be withheld once the browser has been
- # acked for it: the stream is ack-driven, so an unacked drop stops it
- # for good rather than thinning it (SKY-12502). Nothing can ack this
- # one, so deliver the ORIGINAL and let the client's own ack keep the
- # stream alive — fail open, the way the engine treats unobserved
- # interest.
- return None, frame
- # The decision trace. The reason is a closed set, but the method is
- # not — the policy is a port, so an adapter may drop any method at all,
- # and an arbitrary one would export a series per value and put the
- # string itself in a label. Bucketed like command latency (SKY-12510).
- # _KNOWN_CDP_METHODS is a command allowlist, so an event buckets to
- # "other" and only its domain carries through today; SKY-12501 can name
- # the events its rules gate to get them back per-method.
- cdp_method, cdp_domain = _cdp_method_tags(processed.method)
- self._metrics.increment(
- _METRIC_FRAMES_DROPPED,
- tags={
- **_org_tag(session),
- "direction": direction.value,
- "reason": decision.reason.value,
- "cdp_method": cdp_method,
- "cdp_domain": cdp_domain,
- },
- )
- return None
- if isinstance(decision, Rewrite):
- processed = decision.event
- if awaits_ack and screencast_frame_ack(processed) != owed_ack:
- # Something is delivered, but not the frame the browser is waiting on: a
- # middleware or a policy Rewrite replaced it, so the client's ack (if it sends
- # one at all) names the replacement and the original stays unacked. Pay it
- # here. An equal ack means the replacement still names the same frame on the
- # same session — the client settles that one itself, and a second ack would
- # spend the browser's in-flight allowance twice.
- await self._pay_screencast_ack(upstream, remapper, owed_ack)
- return None, processed
-
- async def _relay_upstream_to_client(
- self,
- ws: websockets.ServerConnection,
- connection: UpstreamConnection,
- session: ProxySession,
- remapper: RequestIdRemapper,
- command_starts: dict[int, tuple[str, float]],
- outbound: _OutboundQueue | None = None,
- ) -> None:
- direction = Direction.UPSTREAM_TO_CLIENT
- # ponytail: single-subscriber relay retained as the pinned per-client
- # contract (metrics + backpressure). The served path multiplexes many
- # clients over one connection via _read_shared_upstream; both share
- # _process_upstream_frame. outbound=None is the direct-send test fallback.
- overflow_streak = 0
- while True:
- try:
- frame = decode_frame(await connection.receive())
- except FrameDecodeError:
- self._metrics.increment(
- _METRIC_FRAME_DECODE_ERRORS, tags={**_org_tag(session), "direction": direction.value}
- )
- await ws.close(code=1003, reason="invalid CDP frame")
- return
- result = await self._process_upstream_frame(frame, session, remapper, command_starts, upstream=connection)
- if result is None:
- continue
- client_key, processed = result
- if client_key == PROXY_CLIENT_KEY:
- # The proxy's own command came back (a synthesized screencast ack):
- # consumed here, exactly as the shared reader does. A client must never
- # see a response for an id it did not send — this relay hands frames
- # straight to one client, so an unconsumed proxy reply would arrive as a
- # response to whatever it has in flight under that id.
- self._metrics.increment(
- _METRIC_FRAMES_DROPPED,
- tags={**_org_tag(session), "direction": direction.value, "reason": "proxy_command"},
- )
- continue
- wire = encode_frame(processed)
- if outbound is None:
- await ws.send(wire)
- self._record_relayed(session, direction, wire)
- continue
- try:
- outbound.put_nowait(wire)
- overflow_streak = 0
- except asyncio.QueueFull:
- # An ordinary event is droppable (clients tolerate event loss); a command
- # response and a session lifecycle event are not — losing either desyncs
- # the client, so a client too slow to take one, or one that stays
- # overflowed, is closed cleanly.
- # ponytail: tail-drop; head-drop for screencast freshness is EventPolicy (SKY-12532).
- if isinstance(processed, CdpEvent) and processed.method not in LIFECYCLE_EVENTS:
- overflow_streak += 1
- self._metrics.increment(
- _METRIC_FRAMES_DROPPED,
- tags={**_org_tag(session), "direction": direction.value, "reason": "backpressure"},
- )
- if overflow_streak < self._client_backpressure_close_after:
- continue
- await self._close_slow_client(ws, session)
- return
-
- async def _drain_client_outbound(
- self, ws: websockets.ServerConnection, session: ProxySession, outbound: _OutboundQueue
- ) -> None:
- direction = Direction.UPSTREAM_TO_CLIENT
- while True:
- item = await outbound.get()
- if not isinstance(item, str): # the drain sentinel: all real frames sent, stop
- return
- await ws.send(item)
- self._record_relayed(session, direction, item)
-
- async def _close_slow_client(self, ws: websockets.ServerConnection, session: ProxySession) -> None:
- self._metrics.increment(
- _METRIC_CLIENT_BACKPRESSURE_CLOSED, tags={**_org_tag(session), "session_id": session.session_id}
- )
- await ws.close(code=_BACKPRESSURE_CLOSE_CODE, reason="client too slow")
diff --git a/skyvern/proxy/adapters/websocket_upstream.py b/skyvern/proxy/adapters/websocket_upstream.py
deleted file mode 100644
index 6097616c1..000000000
--- a/skyvern/proxy/adapters/websocket_upstream.py
+++ /dev/null
@@ -1,258 +0,0 @@
-"""Generic CDP-over-WebSocket upstream adapter.
-
-Dials any hosted-browser vendor or own-infra browser that exposes a CDP ws/wss
-endpoint. Connect headers (vendor credentials) are injected from proxy-operator
-configuration — never taken from client input.
-
-The transient-error predicate, retry-budget knobs, and browser-address fragment
-strip deliberately duplicate ~15 lines of skyvern/webeye/cdp_retry.py and
-cdp_connection.py rather than importing them: those modules are Playwright-bound
-runner code and the proxy image ships zero runner dependencies (ADR-0010).
-"""
-
-from __future__ import annotations
-
-import asyncio
-import logging
-import math
-import os
-import re
-import ssl
-from typing import Mapping
-from urllib.parse import urlparse, urlunparse
-
-import structlog
-import websockets
-from websockets.exceptions import InvalidHandshake, InvalidStatus, InvalidURI
-
-from skyvern.proxy.core.errors import (
- ProtocolConfigurationError,
- TransientConnectionError,
- UpstreamConnectError,
- VendorAuthError,
- VendorRateLimitError,
-)
-from skyvern.proxy.core.session import ProxySession, UpstreamClosedError
-from skyvern.proxy.ports import UpstreamConnection
-
-LOG = structlog.get_logger(__name__)
-
-
-class _CredentialRedactingFilter(logging.Filter):
- """Scrub the CDP session token and injected vendor credential out of websockets'
- DEBUG handshake logs so a later level change cannot surface them."""
-
- _REQUEST_TARGET = re.compile(r"(?im)^([<>]\s*[A-Z]+\s+)\S+(\s+HTTP/\d)")
- _SENSITIVE_HEADER = re.compile(
- r"(?im)^([<>]\s*(?:authorization|proxy-authorization|cookie|set-cookie|x-api-key"
- r"|sec-websocket-key|sec-websocket-accept)\s*:\s*).+$"
- )
-
- def filter(self, record: logging.LogRecord) -> bool:
- try:
- message = record.getMessage()
- except Exception:
- record.msg, record.args = "websockets log suppressed (unrenderable)", None
- return True
- redacted = self._SENSITIVE_HEADER.sub(r"\1[REDACTED]", self._REQUEST_TARGET.sub(r"\1[REDACTED]\2", message))
- if redacted != message:
- record.msg, record.args = redacted, None
- return True
-
-
-# websockets DEBUG-logs the handshake target and headers, which carry the CDP session token
-# and the injected vendor credential; the filter (always run because dial() pins the
-# connection to this logger) scrubs them even if the level is later lowered to DEBUG.
-_WEBSOCKETS_LOGGER = logging.getLogger("websockets")
-_WEBSOCKETS_LOGGER.setLevel(logging.INFO)
-_WEBSOCKETS_LOGGER.addFilter(_CredentialRedactingFilter())
-
-_AUTH_HTTP_STATUSES = frozenset({401, 403, 407})
-_RATE_LIMIT_HTTP_STATUS = 429
-DEFAULT_OPEN_TIMEOUT_SECONDS = 30.0
-_DEFAULT_RETRY_ATTEMPTS = 6
-_DEFAULT_BACKOFF_SCHEDULE = (1.0, 2.0, 3.0, 4.0, 5.0)
-_MAX_RETRY_ATTEMPTS = 100
-_MAX_BACKOFF_SECONDS = 300.0
-
-
-def resolve_retry_budget() -> tuple[int, tuple[float, ...]]:
- """Resolve (attempts, backoff) from the proxy's env knobs, falling back to the
- defaults when a value is invalid (non-numeric, NaN/inf, or out of bounds) so a
- misconfig cannot shrink the budget or hang the retry loop."""
- try:
- attempts = int(os.environ.get("CDP_PROXY_CONNECT_RETRY_ATTEMPTS", ""))
- except ValueError:
- attempts = _DEFAULT_RETRY_ATTEMPTS
- if not 1 <= attempts <= _MAX_RETRY_ATTEMPTS:
- attempts = _DEFAULT_RETRY_ATTEMPTS
- raw_backoff = os.environ.get("CDP_PROXY_CONNECT_BACKOFF_SECONDS", "")
- try:
- backoff = tuple(float(part) for part in raw_backoff.split(",")) if raw_backoff else ()
- except ValueError:
- backoff = ()
- if not backoff or any(
- not math.isfinite(seconds) or not 0 <= seconds <= _MAX_BACKOFF_SECONDS for seconds in backoff
- ):
- backoff = _DEFAULT_BACKOFF_SCHEDULE
- return attempts, backoff
-
-
-def merge_connect_headers(operator_headers: Mapping[str, str], session_headers: Mapping[str, str]) -> dict[str, str]:
- """Case-insensitive merge (HTTP header names are case-insensitive); the
- operator-configured value wins so registry data can never override proxy
- credentials or duplicate a header under a different casing."""
- operator_names = {name.lower() for name in operator_headers}
- merged = {name: value for name, value in session_headers.items() if name.lower() not in operator_names}
- merged.update(operator_headers)
- return merged
-
-
-def strip_browser_address_discriminator(url: str) -> str:
- """Remove local PBS URL fragments used only for browser_address DB uniqueness."""
- parsed = urlparse(url)
- if parsed.fragment.startswith("pbs_"):
- return urlunparse(parsed._replace(fragment=""))
- return url
-
-
-def endpoint_label(url: str) -> str:
- """Scheme and host only — upstream paths and query strings can carry session tokens."""
- parsed = urlparse(url)
- return f"{parsed.scheme}://{parsed.netloc.rsplit('@', 1)[-1]}"
-
-
-def classify_connect_error(exc: Exception, label: str) -> UpstreamConnectError:
- if isinstance(exc, InvalidStatus):
- status = exc.response.status_code
- if status in _AUTH_HTTP_STATUSES:
- return VendorAuthError(f"upstream at {label} rejected the proxy's credentials (HTTP {status})")
- if status == _RATE_LIMIT_HTTP_STATUS:
- return VendorRateLimitError(f"upstream at {label} is rate limiting connections (HTTP {status})")
- if status >= 500:
- return TransientConnectionError(f"upstream at {label} failed the websocket handshake (HTTP {status})")
- return ProtocolConfigurationError(f"upstream at {label} is not a CDP websocket endpoint (HTTP {status})")
- # Mid-handshake connection drops (LB draining, browser warmup, half-open reset) surface
- # as InvalidHandshake subclasses (InvalidMessage, InvalidProxyMessage, ...) with an
- # EOFError cause — transient, unlike other handshake failures.
- if isinstance(exc, EOFError) or (isinstance(exc, InvalidHandshake) and isinstance(exc.__cause__, EOFError)):
- return TransientConnectionError(f"upstream at {label} dropped the websocket handshake ({type(exc).__name__})")
- if isinstance(exc, (InvalidURI, InvalidHandshake)):
- return ProtocolConfigurationError(
- f"upstream at {label} is not a usable CDP websocket endpoint ({type(exc).__name__})"
- )
- # TLS failures (expired/untrusted/hostname-mismatch cert) subclass OSError but are a
- # config problem, not a transient network one — classify before the broad OSError check.
- if isinstance(exc, ssl.SSLError):
- return ProtocolConfigurationError(f"upstream at {label} failed TLS verification ({type(exc).__name__})")
- # Remaining transients a raw websocket dial can raise: refused/reset/DNS (OSError)
- # and open-timeout (asyncio.TimeoutError).
- if isinstance(exc, (OSError, asyncio.TimeoutError)):
- return TransientConnectionError(f"connection to upstream at {label} failed ({type(exc).__name__})")
- return UpstreamConnectError(f"connecting to upstream at {label} failed ({type(exc).__name__})")
-
-
-def _refuse_redirect(exc: Exception) -> Exception:
- # websockets treats a returned exception (rather than a new URI) as "not a redirect" and
- # re-raises it without following, so the injected credential never reaches the redirect target.
- return exc
-
-
-class WebSocketUpstreamConnection:
- def __init__(self, ws: websockets.ClientConnection) -> None:
- self._ws = ws
-
- async def send(self, raw: str) -> None:
- try:
- await self._ws.send(raw)
- except websockets.ConnectionClosed as exc:
- raise UpstreamClosedError("upstream websocket closed") from exc
-
- async def receive(self) -> str:
- try:
- message = await self._ws.recv()
- except websockets.ConnectionClosed as exc:
- raise UpstreamClosedError("upstream websocket closed") from exc
- return message if isinstance(message, str) else message.decode()
-
- async def close(self) -> None:
- await self._ws.close()
-
-
-class WebSocketUpstreamBrowser:
- """Connects to any CDP-over-WebSocket endpoint named by the session's upstream URL.
-
- Retries transient failures on the proxy's own retry budget with a per-attempt
- open timeout, and maps failures onto the port error taxonomy.
- """
-
- def __init__(
- self,
- connect_headers: Mapping[str, str] | None = None,
- open_timeout_seconds: float = DEFAULT_OPEN_TIMEOUT_SECONDS,
- ) -> None:
- self._connect_headers = dict(connect_headers or {})
- self._open_timeout_seconds = open_timeout_seconds
-
- async def connect(self, session: ProxySession) -> UpstreamConnection:
- return await self.dial(
- session.upstream_ws_url, merge_connect_headers(self._connect_headers, session.connect_headers)
- )
-
- async def dial(self, url: str, connect_headers: Mapping[str, str] | None = None) -> UpstreamConnection:
- # Taxonomy errors are always raised OUTSIDE the except suite: raising inside
- # would re-attach the raw transport exception as __context__ at raise time,
- # and its text can echo the token-bearing URL.
- parsed: tuple[str, str] | None = None
- try:
- stripped = strip_browser_address_discriminator(url)
- parsed = (stripped, endpoint_label(stripped))
- except ValueError:
- pass
- if parsed is None:
- raise ProtocolConfigurationError("upstream URL is malformed")
- url, label = parsed
- headers = dict(connect_headers) if connect_headers is not None else self._connect_headers
- max_attempts, backoff_schedule = resolve_retry_budget()
- for attempt in range(1, max_attempts + 1):
- try:
- connector = websockets.connect(
- url,
- additional_headers=headers or None,
- max_size=None,
- open_timeout=self._open_timeout_seconds,
- logger=_WEBSOCKETS_LOGGER,
- )
- # A CDP endpoint answering with a 3xx is not one; refusing to follow the
- # redirect stops websockets re-sending the vendor credential to another host.
- connector.process_redirect = _refuse_redirect
- ws = await connector
- except Exception as exc:
- error = classify_connect_error(exc, label)
- error_type = type(exc).__name__
- else:
- if attempt > 1:
- LOG.info(
- "upstream CDP connection recovered after retry", endpoint=label, successful_attempt=attempt
- )
- return WebSocketUpstreamConnection(ws)
- if not isinstance(error, TransientConnectionError) or attempt == max_attempts:
- LOG.warning(
- "upstream CDP connection failed",
- endpoint=label,
- attempt=attempt,
- error_type=error_type,
- category=type(error).__name__,
- )
- raise error
- backoff = backoff_schedule[min(attempt - 1, len(backoff_schedule) - 1)]
- LOG.warning(
- "upstream CDP connection failed, retrying",
- endpoint=label,
- attempt=attempt,
- max_attempts=max_attempts,
- backoff_seconds=backoff,
- error_type=error_type,
- )
- await asyncio.sleep(backoff)
- raise TransientConnectionError(f"connection to upstream at {label} failed")
diff --git a/skyvern/proxy/core/__init__.py b/skyvern/proxy/core/__init__.py
deleted file mode 100644
index 13907e357..000000000
--- a/skyvern/proxy/core/__init__.py
+++ /dev/null
@@ -1,71 +0,0 @@
-from skyvern.proxy.core.frames import (
- KNOWN_CDP_DOMAINS,
- LIFECYCLE_EVENTS,
- TARGET_ATTACHED_EVENT,
- TARGET_DETACHED_EVENT,
- CdpCommand,
- CdpEvent,
- CdpFrame,
- CdpResponse,
- FrameDecodeError,
- RequestIdRemapper,
- decode_frame,
- encode_frame,
- params_session_id,
-)
-from skyvern.proxy.core.pipeline import Direction, Middleware, MiddlewarePipeline
-from skyvern.proxy.core.policy import (
- FORWARD,
- Clock,
- Drop,
- DropReason,
- EventPolicyConfig,
- EventPolicyEngine,
- Forward,
- PolicyDecision,
- RateRule,
- Rewrite,
-)
-from skyvern.proxy.core.policy_pack import (
- CONSOLE_BURST_PER_SECOND,
- NOISY_EVENT_PACK_V1,
- POLICY_PACK_EVENT_METHODS,
- UNCONSUMED_EVENTS,
-)
-from skyvern.proxy.core.session import Principal, ProxySession, UpstreamClosedError
-
-__all__ = [
- "CONSOLE_BURST_PER_SECOND",
- "FORWARD",
- "KNOWN_CDP_DOMAINS",
- "LIFECYCLE_EVENTS",
- "NOISY_EVENT_PACK_V1",
- "POLICY_PACK_EVENT_METHODS",
- "TARGET_ATTACHED_EVENT",
- "TARGET_DETACHED_EVENT",
- "UNCONSUMED_EVENTS",
- "CdpCommand",
- "CdpEvent",
- "CdpFrame",
- "CdpResponse",
- "Clock",
- "Direction",
- "Drop",
- "DropReason",
- "EventPolicyConfig",
- "EventPolicyEngine",
- "Forward",
- "FrameDecodeError",
- "Middleware",
- "MiddlewarePipeline",
- "PolicyDecision",
- "Principal",
- "ProxySession",
- "RateRule",
- "RequestIdRemapper",
- "Rewrite",
- "UpstreamClosedError",
- "decode_frame",
- "encode_frame",
- "params_session_id",
-]
diff --git a/skyvern/proxy/core/denylist.py b/skyvern/proxy/core/denylist.py
deleted file mode 100644
index 35cef8b19..000000000
--- a/skyvern/proxy/core/denylist.py
+++ /dev/null
@@ -1,157 +0,0 @@
-"""Org-level CDP operation denylists (SKY-12538), as an interceptor on the
-command-interception seam (SKY-12535).
-
-A denied command is answered with a deterministic synthesized CDP error under the
-client's own request id — never a silent drop, which would hang the client, and
-never a forward, which would defeat the policy. Running as an interceptor keeps
-the denylist inside the policy engine's decision path rather than a bypass: the
-audit trail (the commands_intercepted counter plus the adapter's structured log)
-comes with the seam.
-
-Matching covers the legacy `Target.sendMessageToTarget` tunnel: the inner
-serialized command's method is matched too (see `denied_method`), so wrapping a
-denied method does not bypass the policy. A tunneled denial surfaces as an error
-on the WRAPPER id — the same in-protocol outcome as Chrome refusing the send
-itself (invalid or closed session), which legacy clients must and do handle by
-failing the inner call (pre-flat puppeteer rejects the inner callback on a
-wrapper error). Synthesizing the target-side `Target.receivedMessageFromTarget`
-reply instead would need an event-synthesis verb the interception seam
-deliberately does not have (SKY-12535 v1), for a client shape no worse off than
-against a real browser refusing the send.
-
-The METHOD-PATTERN GRAMMAR is a strict superset of the policy packs' exact method
-names, so any method string a pack rule names is a valid pattern here:
-
-- exact: ``Page.navigate`` matches only that method;
-- trailing ``*``: a prefix wildcard — ``Network.*`` denies a domain, ``*`` denies
- everything. ``*`` anywhere else is rejected at compile time, not silently
- ignored: a pattern that quietly matched nothing would read as protection.
-
-WHO IS DENIED comes from `DenylistLookup`: the session's organization resolves to
-its compiled pattern set (the cloud ConfigStore-backed source lives in
-cloud/cdp_proxy/denylist.py; its cache TTL is the propagation window). A lookup
-returning None means no denylist — the source absorbs config-infrastructure
-failures by failing OPEN, matching the event policy's forwarding-is-the-safe
-default, while a lookup that raises fails the command CLOSED via the seam's rule.
-"""
-
-from __future__ import annotations
-
-import json
-from collections.abc import Awaitable, Callable, Iterable
-from dataclasses import dataclass
-from typing import Any
-
-from skyvern.proxy.core.frames import CdpCommand
-from skyvern.proxy.core.pipeline import CommandInterceptor, InterceptContext, InterceptOutcome, SynthesizedResponse
-from skyvern.proxy.core.session import ProxySession
-
-DENYLIST_REASON = "org_denylist"
-
-# JSON-RPC generic server error; distinct from CDP's -32001 session-not-found and
-# the seam's -32603 interceptor failure, so a denial is tellable from both.
-DENIED_ERROR_CODE = -32000
-
-
-class MethodPatternError(ValueError):
- pass
-
-
-def denied_error(method: str) -> dict[str, Any]:
- return {"code": DENIED_ERROR_CODE, "message": f"'{method}' is not allowed by organization policy"}
-
-
-@dataclass(frozen=True)
-class MethodPatternSet:
- """A compiled denylist: exact methods plus '*'-terminated prefixes."""
-
- exact: frozenset[str]
- prefixes: tuple[str, ...]
-
- @classmethod
- def compile(cls, patterns: Iterable[str]) -> MethodPatternSet:
- exact: set[str] = set()
- prefixes: list[str] = []
- for pattern in patterns:
- if not isinstance(pattern, str) or not pattern or any(c.isspace() for c in pattern):
- raise MethodPatternError(f"invalid method pattern: {pattern!r}")
- star = pattern.find("*")
- if star == -1:
- exact.add(pattern)
- elif star == len(pattern) - 1:
- prefixes.append(pattern[:-1])
- else:
- raise MethodPatternError(f"'*' is only valid as the final character: {pattern!r}")
- return cls(frozenset(exact), tuple(prefixes))
-
- def matches(self, method: str) -> bool:
- return method in self.exact or any(method.startswith(prefix) for prefix in self.prefixes)
-
- def __bool__(self) -> bool:
- return bool(self.exact or self.prefixes)
-
-
-DenylistLookup = Callable[[ProxySession], Awaitable["MethodPatternSet | None"]]
-
-_SEND_MESSAGE_METHOD = "Target.sendMessageToTarget"
-# Legitimate clients do not nest the legacy tunnel at all; a chain deeper than
-# this is denied outright rather than left unverifiable.
-_MAX_TUNNEL_DEPTH = 8
-
-
-def denied_method(patterns: MethodPatternSet, command: CdpCommand) -> str | None:
- """The method this command would run that the denylist matches, or None.
-
- Target.sendMessageToTarget is the legacy non-flat tunnel: its `message` param
- carries a serialized inner command that Chrome unwraps and dispatches, so the
- inner method must be matched too — recursively, since the tunnel can wrap
- itself — or denying a method is bypassed by wrapping it. An undecodable
- message passes (Chrome cannot dispatch it either, so there is nothing to
- deny); a chain past the depth cap is denied as-is.
- """
- method = command.method
- params: dict[str, Any] = command.params or {}
- for _ in range(_MAX_TUNNEL_DEPTH + 1):
- if patterns.matches(method):
- return method
- if method != _SEND_MESSAGE_METHOD:
- return None
- message = params.get("message")
- if not isinstance(message, str):
- return None
- try:
- inner = json.loads(message)
- except ValueError:
- return None
- if not isinstance(inner, dict) or not isinstance(inner.get("method"), str):
- return None
- method = inner["method"]
- inner_params = inner.get("params")
- params = inner_params if isinstance(inner_params, dict) else {}
- return method
-
-
-def org_denylist_interceptor(lookup: DenylistLookup) -> CommandInterceptor:
- async def intercept(command: CdpCommand, session: ProxySession, context: InterceptContext) -> InterceptOutcome:
- patterns = await lookup(session)
- if patterns is not None:
- matched = denied_method(patterns, command)
- if matched is not None:
- # audit_method: a tunneled denial is audited against the method it
- # blocked, not the Target.sendMessageToTarget wrapper it rode in.
- return SynthesizedResponse(error=denied_error(matched), reason=DENYLIST_REASON, audit_method=matched)
- return command
-
- return intercept
-
-
-__all__ = [
- "DENIED_ERROR_CODE",
- "DENYLIST_REASON",
- "DenylistLookup",
- "MethodPatternError",
- "MethodPatternSet",
- "denied_error",
- "denied_method",
- "org_denylist_interceptor",
-]
diff --git a/skyvern/proxy/core/errors.py b/skyvern/proxy/core/errors.py
deleted file mode 100644
index 9155ba688..000000000
--- a/skyvern/proxy/core/errors.py
+++ /dev/null
@@ -1,37 +0,0 @@
-"""Shared error taxonomy for upstream browser connections.
-
-Pure exception classes only: adapters translate transport- and vendor-specific
-failures into these categories so the core and its callers never branch on
-framework exception types. Messages must never contain raw upstream URLs —
-their paths and query strings can carry session tokens.
-"""
-
-from __future__ import annotations
-
-
-class UpstreamConnectError(Exception):
- """Establishing a connection to the upstream browser failed."""
-
-
-class TransientConnectionError(UpstreamConnectError):
- """Network-level failure worth retrying: refused, reset, timeout, DNS."""
-
-
-class ProtocolConfigurationError(UpstreamConnectError):
- """The endpoint is reachable but is not a usable CDP websocket endpoint."""
-
-
-class LaunchEnvironmentError(UpstreamConnectError):
- """A locally launched browser could not start in this environment."""
-
-
-class LaunchTimeoutError(UpstreamConnectError):
- """A locally launched browser started but never became connectable in time."""
-
-
-class VendorRateLimitError(UpstreamConnectError):
- """A hosted-browser provider rejected the connection due to rate limiting."""
-
-
-class VendorAuthError(UpstreamConnectError):
- """A hosted-browser provider rejected the credentials attached by the proxy."""
diff --git a/skyvern/proxy/core/frames.py b/skyvern/proxy/core/frames.py
deleted file mode 100644
index 4820982f8..000000000
--- a/skyvern/proxy/core/frames.py
+++ /dev/null
@@ -1,409 +0,0 @@
-from __future__ import annotations
-
-import json
-import math
-from dataclasses import dataclass, field
-from typing import Any, Union
-
-
-class FrameDecodeError(ValueError):
- pass
-
-
-class RemapperFullError(RuntimeError):
- """The pending-request table is full and nothing this caller owns can be evicted
- to make room. Admitting the command would mean discarding a mapping already
- promised to another client (or to the proxy), so the caller is refused instead."""
-
-
-@dataclass(frozen=True)
-class CdpCommand:
- id: int
- method: str
- params: dict[str, Any] | None = None
- session_id: str | None = None
-
-
-@dataclass(frozen=True)
-class CdpResponse:
- id: int
- result: dict[str, Any] | None = None
- error: dict[str, Any] | None = None
- session_id: str | None = None
-
-
-@dataclass(frozen=True)
-class CdpEvent:
- method: str
- params: dict[str, Any] | None = None
- session_id: str | None = None
-
-
-CdpFrame = Union[CdpCommand, CdpResponse, CdpEvent]
-
-# Flat-session lifecycle events. Both carry the affected session in params.sessionId
-# (NOT the frame's own sessionId, which is the parent for a nested attach), so they are
-# routed by ownership of that session rather than by their own scope.
-TARGET_ATTACHED_EVENT = "Target.attachedToTarget"
-TARGET_DETACHED_EVENT = "Target.detachedFromTarget"
-# A client builds and retires its session objects from these two, and the proxy learns
-# session ownership from them, so unlike ordinary events they are never droppable:
-# losing one leaves the client silently desynced from the browser (a session it can
-# never use, or one it thinks is still live) and strands that session's owner. A client
-# too slow to take one is closed instead. Delivery, routing, and the event policy all
-# key off this one set — two copies could drift, and a drift is invisible until a
-# session is already stranded.
-LIFECYCLE_EVENTS = frozenset({TARGET_ATTACHED_EVENT, TARGET_DETACHED_EVENT})
-
-# The CDP domains this proxy recognizes. Used to bucket a method to bounded metric
-# labels and to bound what counts as real, declared client interest — an unknown
-# domain is neither. Core-owned so the metric bucketing and the event policy agree.
-KNOWN_CDP_DOMAINS = frozenset(
- {
- "Accessibility",
- "Animation",
- "Audits",
- "BackgroundService",
- "Browser",
- "CacheStorage",
- "Cast",
- "Console",
- "CSS",
- "Database",
- "Debugger",
- "DeviceOrientation",
- "DOM",
- "DOMDebugger",
- "DOMSnapshot",
- "DOMStorage",
- "Emulation",
- "Fetch",
- "HeadlessExperimental",
- "HeapProfiler",
- "IndexedDB",
- "Input",
- "Inspector",
- "IO",
- "LayerTree",
- "Log",
- "Media",
- "Memory",
- "Network",
- "Overlay",
- "Page",
- "Performance",
- "Profiler",
- "Runtime",
- "Schema",
- "Security",
- "ServiceWorker",
- "Storage",
- "SystemInfo",
- "Target",
- "Tethering",
- "Tracing",
- "WebAudio",
- "WebAuthn",
- }
-)
-
-_FRAME_KEYS = frozenset({"error", "id", "method", "params", "result", "sessionId"})
-_ERROR_KEYS = frozenset({"code", "data", "message"})
-
-# The lane proxy-issued commands ride in the remapper. Client keys are allocated by
-# the adapter (c1, c2, ...), so this can never be a real client; the reserved-key
-# guards below keep it that way even if that scheme changes.
-PROXY_CLIENT_KEY = "__proxy__"
-
-# An in-flight command whose response never arrives would otherwise pin its mapping
-# forever. Mirrors the adapter's own pending-command bound.
-_MAX_PENDING_REQUESTS = 4096
-
-
-def _require_id(value: object, field_name: str = "id") -> int:
- if isinstance(value, bool) or not isinstance(value, int):
- raise FrameDecodeError(f"CDP {field_name} must be an integer")
- return value
-
-
-def _require_text(value: object, field_name: str) -> str:
- if not isinstance(value, str) or not value.strip():
- raise FrameDecodeError(f"CDP {field_name} must be a non-empty string")
- try:
- value.encode("utf-8")
- except UnicodeEncodeError as exc:
- raise FrameDecodeError(f"CDP {field_name} must be valid UTF-8 text") from exc
- return value
-
-
-def _optional_object(payload: dict[str, Any], field_name: str) -> dict[str, Any] | None:
- if field_name not in payload:
- return None
- value = payload[field_name]
- if not isinstance(value, dict):
- raise FrameDecodeError(f"CDP {field_name} must be an object")
- return value
-
-
-def _optional_session_id(payload: dict[str, Any]) -> str | None:
- if "sessionId" not in payload:
- return None
- return _require_text(payload["sessionId"], "sessionId")
-
-
-def _validate_error(value: object) -> dict[str, Any]:
- if not isinstance(value, dict):
- raise FrameDecodeError("CDP error must be an object")
- if set(value) - _ERROR_KEYS or "code" not in value or "message" not in value:
- raise FrameDecodeError("CDP error must contain only code, message, and optional data")
- _require_id(value["code"], "error code")
- _require_text(value["message"], "error message")
- return value
-
-
-def _reject_json_constant(value: str) -> Any:
- raise ValueError(f"invalid JSON constant: {value}")
-
-
-def _parse_float(value: str) -> float:
- parsed = float(value)
- if not math.isfinite(parsed):
- raise ValueError(f"non-finite JSON number: {value}")
- return parsed
-
-
-def _object_pairs_hook(pairs: list[tuple[str, Any]]) -> dict[str, Any]:
- payload: dict[str, Any] = {}
- for key, value in pairs:
- if key in payload:
- raise FrameDecodeError(f"duplicate JSON field: {key}")
- payload[key] = value
- return payload
-
-
-def _parse_json(raw: str | bytes) -> object:
- if isinstance(raw, bytes):
- try:
- text = raw.decode("utf-8")
- except UnicodeDecodeError as exc:
- raise FrameDecodeError(f"invalid UTF-8 frame: {exc}") from exc
- elif isinstance(raw, str):
- text = raw
- else:
- raise FrameDecodeError("CDP frame must be text or UTF-8 bytes")
-
- try:
- return json.loads(
- text,
- object_pairs_hook=_object_pairs_hook,
- parse_constant=_reject_json_constant,
- parse_float=_parse_float,
- )
- except FrameDecodeError:
- raise
- except (json.JSONDecodeError, RecursionError, TypeError, ValueError) as exc:
- raise FrameDecodeError(f"invalid JSON frame: {exc}") from exc
-
-
-def _decode_payload(payload: object) -> CdpFrame:
- if not isinstance(payload, dict):
- raise FrameDecodeError("CDP frame must be a JSON object")
- unexpected_keys = set(payload) - _FRAME_KEYS
- if unexpected_keys:
- raise FrameDecodeError(f"CDP frame has unexpected fields: {sorted(unexpected_keys)!r}")
-
- has_id = "id" in payload
- has_method = "method" in payload
- if not has_id and not has_method:
- raise FrameDecodeError("CDP frame must contain id or method")
-
- session_id = _optional_session_id(payload)
- if has_id:
- frame_id = _require_id(payload["id"])
- if has_method:
- if "result" in payload or "error" in payload:
- raise FrameDecodeError("CDP command cannot contain result or error")
- return CdpCommand(
- id=frame_id,
- method=_require_text(payload["method"], "method"),
- params=_optional_object(payload, "params"),
- session_id=session_id,
- )
-
- if "method" in payload or "params" in payload:
- raise FrameDecodeError("CDP response cannot contain method or params")
- has_result = "result" in payload
- has_error = "error" in payload
- if has_result == has_error:
- raise FrameDecodeError("CDP response must contain exactly one of result or error")
- result = _optional_object(payload, "result") if has_result else None
- error = _validate_error(payload["error"]) if has_error else None
- return CdpResponse(id=frame_id, result=result, error=error, session_id=session_id)
-
- if "result" in payload or "error" in payload:
- raise FrameDecodeError("CDP event cannot contain result or error")
- return CdpEvent(
- method=_require_text(payload["method"], "method"),
- params=_optional_object(payload, "params"),
- session_id=session_id,
- )
-
-
-def params_session_id(frame: CdpFrame) -> str | None:
- """The session a lifecycle frame is ABOUT (params.sessionId), as opposed to the
- session it is scoped to (the frame's own sessionId)."""
- params = frame.params if isinstance(frame, (CdpCommand, CdpEvent)) else None
- value = params.get("sessionId") if params else None
- return value if isinstance(value, str) and value else None
-
-
-def decode_frame(raw: str | bytes) -> CdpFrame:
- return _decode_payload(_parse_json(raw))
-
-
-def _frame_payload(frame: CdpFrame) -> dict[str, Any]:
- if isinstance(frame, CdpCommand):
- payload: dict[str, Any] = {
- "id": _require_id(frame.id),
- "method": _require_text(frame.method, "method"),
- }
- if frame.params is not None:
- if not isinstance(frame.params, dict):
- raise FrameDecodeError("CDP params must be an object")
- payload["params"] = frame.params
- elif isinstance(frame, CdpResponse):
- payload = {"id": _require_id(frame.id)}
- if frame.result is not None and frame.error is not None:
- raise FrameDecodeError("CDP response cannot contain both result and error")
- if frame.error is not None:
- payload["error"] = _validate_error(frame.error)
- else:
- result = frame.result if frame.result is not None else {}
- if not isinstance(result, dict):
- raise FrameDecodeError("CDP result must be an object")
- payload["result"] = result
- elif isinstance(frame, CdpEvent):
- payload = {"method": _require_text(frame.method, "method")}
- if frame.params is not None:
- if not isinstance(frame.params, dict):
- raise FrameDecodeError("CDP params must be an object")
- payload["params"] = frame.params
- else:
- raise FrameDecodeError("unsupported CDP frame type")
-
- if frame.session_id is not None:
- payload["sessionId"] = _require_text(frame.session_id, "sessionId")
- return payload
-
-
-def encode_frame(frame: CdpFrame) -> str:
- payload = _frame_payload(frame)
- try:
- return json.dumps(payload, ensure_ascii=True, separators=(",", ":"), allow_nan=False)
- except (TypeError, ValueError, OverflowError, RecursionError) as exc:
- raise FrameDecodeError(f"CDP frame is not JSON serializable: {exc}") from exc
-
-
-@dataclass
-class RequestIdRemapper:
- """Remaps client request IDs onto one upstream ID space.
-
- Proxy-issued commands share that ID space under `PROXY_CLIENT_KEY`, so a proxy
- command can never collide with a client's in-flight ID (which would hand one
- party the other's response).
- """
-
- max_pending: int = _MAX_PENDING_REQUESTS
- _next_id: int = 1
- _upstream_to_client: dict[int, tuple[str, int]] = field(default_factory=dict)
-
- def __post_init__(self) -> None:
- if self.max_pending < 1:
- raise ValueError("max_pending must be positive")
-
- @property
- def pending_count(self) -> int:
- return len(self._upstream_to_client)
-
- def __len__(self) -> int:
- return self.pending_count
-
- def _allocate_upstream_id(self) -> int:
- upstream_id = self._next_id
- while upstream_id in self._upstream_to_client:
- upstream_id += 1
- self._next_id = upstream_id + 1
- return upstream_id
-
- def _evict_one_for(self, client_key: str) -> None:
- """Reclaim one slot on behalf of `client_key`, at its OWN expense.
-
- A client at the cap may only drop its own oldest mapping — never a co-tenant's
- and never the proxy's, since those responses were already promised and losing
- one silently desyncs that party. A caller with nothing of its own to reclaim is
- refused rather than served at someone else's cost.
- """
- for upstream_id, (owner, _) in self._upstream_to_client.items():
- if owner == client_key and owner != PROXY_CLIENT_KEY:
- del self._upstream_to_client[upstream_id]
- return
- raise RemapperFullError("no pending request of this caller's own can be evicted")
-
- def _remap(self, client_key: str, command: CdpCommand) -> CdpCommand:
- if not isinstance(command, CdpCommand):
- raise FrameDecodeError("only CDP commands can be remapped upstream")
- _frame_payload(command)
- # A response for an evicted ID arrives unmatched (dropped) rather than
- # mis-routed, since the ID is only recycled once the table wraps.
- while len(self._upstream_to_client) >= self.max_pending:
- self._evict_one_for(client_key)
- upstream_id = self._allocate_upstream_id()
- self._upstream_to_client[upstream_id] = (client_key, command.id)
- return CdpCommand(id=upstream_id, method=command.method, params=command.params, session_id=command.session_id)
-
- def to_upstream(self, client_key: str, command: CdpCommand) -> CdpCommand:
- if not isinstance(client_key, str) or not client_key.strip():
- raise ValueError("client_key must be a non-empty string")
- if client_key == PROXY_CLIENT_KEY:
- raise ValueError("client_key is reserved for proxy-issued commands")
- return self._remap(client_key, command)
-
- def to_upstream_as_proxy(self, command: CdpCommand) -> CdpCommand:
- """Remap a command the proxy itself issues; its response routes to
- `PROXY_CLIENT_KEY` and is consumed by the proxy, never sent to a client."""
- return self._remap(PROXY_CLIENT_KEY, command)
-
- def to_client(self, response: CdpResponse) -> tuple[str, CdpResponse] | None:
- if not isinstance(response, CdpResponse):
- raise FrameDecodeError("only CDP responses can be remapped to a client")
- _frame_payload(response)
- mapping = self._upstream_to_client.pop(response.id, None)
- if mapping is None:
- return None
- client_key, client_id = mapping
- return client_key, CdpResponse(
- id=client_id, result=response.result, error=response.error, session_id=response.session_id
- )
-
- def discard(self, upstream_id: int) -> None:
- """Forget a pending mapping whose response will never be delivered. Every
- terminal path for a response must clear this table, not only `to_client`."""
- self._upstream_to_client.pop(upstream_id, None)
-
- def clear_client(self, client_key: str) -> int:
- if not isinstance(client_key, str) or not client_key.strip():
- raise ValueError("client_key must be a non-empty string")
- if client_key == PROXY_CLIENT_KEY:
- # A departing client must never drop the proxy's own in-flight commands.
- raise ValueError("client_key is reserved for proxy-issued commands")
- pending_ids = [
- upstream_id for upstream_id, mapping in self._upstream_to_client.items() if mapping[0] == client_key
- ]
- for upstream_id in pending_ids:
- del self._upstream_to_client[upstream_id]
- return len(pending_ids)
-
- def clear(self) -> int:
- pending_count = len(self._upstream_to_client)
- self._upstream_to_client.clear()
- return pending_count
diff --git a/skyvern/proxy/core/interception_demo.py b/skyvern/proxy/core/interception_demo.py
deleted file mode 100644
index bf8d2ff13..000000000
--- a/skyvern/proxy/core/interception_demo.py
+++ /dev/null
@@ -1,54 +0,0 @@
-"""Flag-gated demonstrator for the command-interception seam (SKY-12535).
-
-One interceptor proving both halves of interception against a live upstream,
-without any behavior a real client depends on:
-
-- REQUEST synthesis: a proxy-originated Browser.getVersion is sent upstream on the
- proxy's reserved id lane, so the upstream still sees the traffic and its reply is
- consumed by the proxy — never delivered to any client, never colliding with a
- client's in-flight ids.
-- RESPONSE synthesis: the client's own command is answered locally with the proxy's
- identity (matching the /json/version discovery payload), under the client's own
- request id and session scope.
-
-Enabled only by `CDP_PROXY_INTERCEPTORS=demo-get-version`; never a default. If the
-proxy-lane send fails (remapper full, upstream gone) the interceptor raises and the
-command fails closed — the deterministic behavior the seam guarantees.
-"""
-
-from __future__ import annotations
-
-from skyvern.proxy.core.frames import CdpCommand
-from skyvern.proxy.core.pipeline import InterceptContext, InterceptOutcome, SynthesizedResponse
-from skyvern.proxy.core.session import ProxySession
-
-GET_VERSION_METHOD = "Browser.getVersion"
-DEMO_INTERCEPT_REASON = "demo_get_version"
-
-# Mirrors the discovery endpoint's identity, so an intercepted response is
-# self-evidently the proxy's rather than a mangled browser reply.
-DEMO_GET_VERSION_RESULT = {
- "protocolVersion": "1.3",
- "product": "Skyvern-CDP-Proxy",
- "revision": "@proxy",
- "userAgent": "Skyvern-CDP-Proxy",
- "jsVersion": "0",
-}
-
-
-async def demo_get_version_interceptor(
- command: CdpCommand, session: ProxySession, context: InterceptContext
-) -> InterceptOutcome:
- if command.method != GET_VERSION_METHOD:
- return command
- # id=0 is a placeholder: to_upstream_as_proxy allocates the real upstream id.
- await context.send_proxy_command(CdpCommand(id=0, method=GET_VERSION_METHOD))
- return SynthesizedResponse(result=dict(DEMO_GET_VERSION_RESULT), reason=DEMO_INTERCEPT_REASON)
-
-
-__all__ = [
- "DEMO_GET_VERSION_RESULT",
- "DEMO_INTERCEPT_REASON",
- "GET_VERSION_METHOD",
- "demo_get_version_interceptor",
-]
diff --git a/skyvern/proxy/core/pipeline.py b/skyvern/proxy/core/pipeline.py
deleted file mode 100644
index e4b7aa141..000000000
--- a/skyvern/proxy/core/pipeline.py
+++ /dev/null
@@ -1,225 +0,0 @@
-"""The command-side seam of the proxy core: an ordered middleware chain over frames,
-plus command interception (SKY-12535).
-
-Two hooks with different contracts, run in this order on the client→upstream leg:
-
-1. FRAME MIDDLEWARES (`Middleware`) — pure rewrites over any frame in either
- direction, in registration order; returning None drops the frame. This is the
- right hook for bounding or normalizing what rides through (see
- `bound_screencast_middleware`).
-2. COMMAND INTERCEPTORS (`CommandInterceptor`) — client→upstream COMMANDS only,
- after the middlewares and after the driving adapter's session-ownership refusal
- (an interceptor never sees a command addressed to a session its client does not
- own). This is the hook for implementing a CDP operation at the proxy: rewrite
- it, delegate it, or answer it without the browser.
-
-Writing an interceptor — the composition and correlation rules:
-
-- Interceptors run in registration order; each receives the command as the
- previous one left it.
-- Return the command (same or rewritten) to continue the chain; the final command
- is forwarded upstream through request-id remapping exactly as an untouched one.
-- Return a `SynthesizedResponse` to answer the client at the proxy: the chain
- short-circuits, nothing is forwarded, and the response is delivered carrying the
- command's own id and sessionId — stamped by `SynthesizedResponse.to_response`,
- never hand-built, so a synthesis cannot mis-correlate. Because the command never
- reaches the remapper, no upstream id, latency record, or attach intent exists to
- leak or leave dangling.
-- There is no drop verb. A silently dropped command hangs its client forever, so
- blocking a command means synthesizing a deterministic error response (SKY-12538).
-- A rewrite must preserve the request id and the session addressing (the frame's
- sessionId and a params-addressed sessionId): ownership was checked on what
- arrived, and the response must reach the id the client is waiting on. Both are
- enforced — a violation raises `InterceptorContractError`.
-- An interceptor that raises is FAIL-CLOSED: the driving adapter answers the client
- with `interceptor_failure_response` (a deterministic internal error) and forwards
- nothing. A policy hook that fails must never fail open into the browser.
-- `InterceptContext.send_proxy_command` synthesizes a REQUEST: a proxy-originated
- command sent upstream on the proxy's reserved id lane
- (`RequestIdRemapper.to_upstream_as_proxy`), whose response is consumed by the
- proxy and can never collide with, or be delivered as, a client's. Fire-and-forget
- in v1; awaiting the response (full delegation) is a planned extension of this
- context and needs no interceptor signature change.
-- Upstream→client frames never reach interceptors; responses can be rewritten by a
- frame middleware, and events belong to `EventPolicyPort`.
-
-A general plugin system is explicitly out of scope: interceptors are proxy-operator
-code selected by deployment configuration (see `skyvern.proxy.__main__`).
-"""
-
-from __future__ import annotations
-
-import enum
-import re
-from collections.abc import Awaitable, Callable, Iterable
-from dataclasses import dataclass
-from typing import Any, Union
-
-from skyvern.proxy.core.frames import (
- CdpCommand,
- CdpFrame,
- CdpResponse,
- FrameDecodeError,
- encode_frame,
- params_session_id,
-)
-from skyvern.proxy.core.session import ProxySession
-
-
-class Direction(enum.Enum):
- CLIENT_TO_UPSTREAM = "client_to_upstream"
- UPSTREAM_TO_CLIENT = "upstream_to_client"
-
-
-Middleware = Callable[[CdpFrame, Direction, ProxySession], Awaitable[CdpFrame | None]]
-
-
-class InterceptorContractError(RuntimeError):
- """An interceptor broke a correlation invariant (changed the request id or the
- session addressing, or returned something that is neither a command nor a
- synthesis). Surfaced to the driving adapter, which fails the command closed."""
-
-
-# JSON-RPC internal error: the deterministic verdict for a command whose
-# interceptor failed. Distinct from CDP's own -32001 session-not-found.
-INTERCEPTOR_FAILURE_CODE = -32603
-_INTERCEPTOR_FAILURE_MESSAGE = "CDP command interception failed"
-
-# The audit reason for the fail-closed path; SynthesizedResponse reasons share the
-# same closed-set discipline (they become metric labels, never free-form).
-INTERCEPTOR_FAILURE_REASON = "interceptor_error"
-
-
-# Reasons become metric labels and log fields, so they are constrained to short
-# label-safe identifiers: a reason derived from request data (a URL, an id, error
-# text) cannot pass this, which is what bounds cardinality and rules out content
-# leakage without a central registry.
-_REASON_PATTERN = re.compile(r"[a-z0-9_]{1,64}")
-
-
-@dataclass(frozen=True, slots=True)
-class SynthesizedResponse:
- """An interceptor's local answer to a command: exactly one of result/error,
- validated eagerly — through the real frame encoder — so a malformed or
- non-encodable synthesis fails at construction (inside the interceptor, where
- the fail-closed rule applies) rather than at delivery, where it would tear the
- relay down instead of answering the client.
-
- `reason` is the audit tag for the decision trace — a label-safe identifier,
- since it becomes a metric label. `audit_method` names the method the decision
- was actually about when that differs from the intercepted command's own — a
- denied method tunneled inside Target.sendMessageToTarget (SKY-12538) — so the
- audit trail records the blocked method, not the wrapper it arrived in.
- """
-
- result: dict[str, Any] | None = None
- error: dict[str, Any] | None = None
- reason: str = "intercepted"
- audit_method: str | None = None
-
- def __post_init__(self) -> None:
- if (self.result is None) == (self.error is None):
- raise ValueError("synthesized response must carry exactly one of result or error")
- try:
- # The id is a placeholder; to_response stamps the real one. What this
- # probes is everything else: error shape, and that every value in
- # result/error (including error.data) survives CDP JSON encoding.
- encode_frame(CdpResponse(id=0, result=self.result, error=self.error))
- except FrameDecodeError as exc:
- raise ValueError(f"synthesized response is not CDP-encodable: {exc}") from exc
- if not isinstance(self.reason, str) or not _REASON_PATTERN.fullmatch(self.reason):
- raise ValueError("reason must be a short lowercase identifier ([a-z0-9_]{1,64})")
- if self.audit_method is not None and (
- not isinstance(self.audit_method, str)
- or not self.audit_method
- or any(c.isspace() for c in self.audit_method)
- ):
- raise ValueError("audit_method must be a non-empty method name")
-
- def to_response(self, command: CdpCommand) -> CdpResponse:
- """Stamp the synthesis with the intercepted command's own id and session
- scope — the correlation guarantee, by construction rather than convention."""
- return CdpResponse(id=command.id, result=self.result, error=self.error, session_id=command.session_id)
-
-
-@dataclass(frozen=True)
-class InterceptContext:
- """Per-connection capabilities the driving adapter hands an interceptor.
-
- `send_proxy_command` sends a proxy-originated command upstream on the reserved
- proxy id lane; its response is consumed by the proxy and never reaches a
- client. It raises (`RemapperFullError`, `UpstreamClosedError`) when the send
- cannot happen — an interceptor that lets that propagate fails closed.
- """
-
- send_proxy_command: Callable[[CdpCommand], Awaitable[None]]
-
-
-InterceptOutcome = Union[CdpCommand, SynthesizedResponse]
-CommandInterceptor = Callable[[CdpCommand, ProxySession, InterceptContext], Awaitable[InterceptOutcome]]
-
-
-def interceptor_failure_response(command: CdpCommand) -> CdpResponse:
- """The deterministic fail-closed answer for a command whose interceptor raised."""
- return CdpResponse(
- id=command.id,
- error={"code": INTERCEPTOR_FAILURE_CODE, "message": _INTERCEPTOR_FAILURE_MESSAGE},
- session_id=command.session_id,
- )
-
-
-class MiddlewarePipeline:
- """Runs each frame through an ordered middleware chain; returning None drops the
- frame. Command interceptors are a separate, stricter chain — see the module
- docstring for the ordering and composition rules."""
-
- def __init__(
- self,
- middlewares: Iterable[Middleware] | None = None,
- interceptors: Iterable[CommandInterceptor] | None = None,
- ) -> None:
- self._middlewares = list(middlewares or ())
- self._interceptors = list(interceptors or ())
-
- def add(self, middleware: Middleware) -> None:
- self._middlewares.append(middleware)
-
- @property
- def has_interceptors(self) -> bool:
- """False means the adapter's pass-through path runs with zero interception
- overhead — disabled is byte-identical to the pipeline before the seam."""
- return bool(self._interceptors)
-
- async def process(self, frame: CdpFrame, direction: Direction, session: ProxySession) -> CdpFrame | None:
- current: CdpFrame | None = frame
- for middleware in self._middlewares:
- if current is None:
- return None
- current = await middleware(current, direction, session)
- return current
-
- async def intercept(
- self, command: CdpCommand, session: ProxySession, context: InterceptContext
- ) -> InterceptOutcome:
- """Run a client→upstream command through the interceptor chain.
-
- Returns the final command to forward, or the first SynthesizedResponse
- (short-circuit). Contract violations raise InterceptorContractError; an
- interceptor's own exception propagates — the caller fails the command
- closed either way (`interceptor_failure_response`).
- """
- current = command
- for interceptor in self._interceptors:
- outcome = await interceptor(current, session, context)
- if isinstance(outcome, SynthesizedResponse):
- return outcome
- if not isinstance(outcome, CdpCommand):
- raise InterceptorContractError("interceptor must return a command or a synthesized response")
- if (
- outcome.id != command.id
- or outcome.session_id != command.session_id
- or params_session_id(outcome) != params_session_id(command)
- ):
- raise InterceptorContractError("interceptor must preserve the request id and session addressing")
- current = outcome
- return current
diff --git a/skyvern/proxy/core/policy.py b/skyvern/proxy/core/policy.py
deleted file mode 100644
index b8b012f2c..000000000
--- a/skyvern/proxy/core/policy.py
+++ /dev/null
@@ -1,336 +0,0 @@
-"""The event-policy engine: decides, per upstream CDP event, what the client sees.
-
-Pure and I/O-free. Policy arrives as a config VALUE and time as an injected clock, so
-a decision is a function of (config, clock, observed traffic) alone — which is what
-makes throttling testable without real time. The engine emits no telemetry itself:
-MetricsPort lives in `skyvern.proxy.ports`, which imports this module, so the decision
-trace is emitted by the caller from the returned decision (see the driving adapter).
-
-This module is the MECHANISM. The noisy-domain rule pack (SKY-12501) and org-level
-denylists (SKY-12538) are configuration built on top of it and live elsewhere.
-"""
-
-from __future__ import annotations
-
-from collections.abc import Callable, Sequence
-from dataclasses import dataclass, field
-from enum import Enum
-from typing import Union
-
-from skyvern.proxy.core.frames import (
- KNOWN_CDP_DOMAINS,
- LIFECYCLE_EVENTS,
- TARGET_DETACHED_EVENT,
- CdpCommand,
- CdpEvent,
- params_session_id,
-)
-from skyvern.proxy.core.session import ProxySession
-
-Clock = Callable[[], float]
-
-# Bound on the per-session window table. A rule keyed on a client-controlled param
-# (e.g. requestId) would otherwise grow one entry per distinct value for the life of
-# the session. Mirrors the adapter's pending-command bound. Overflow evicts, which
-# refills that key's budget early — fail-open, since forwarding is the safe default.
-_MAX_WINDOWS_PER_SESSION = 4096
-
-# Bound on tracked child CDP sessions per proxy session. A client may address a
-# sessionId the proxy has never seen attached (the driving adapter leaves an unowned
-# session for the browser to accept or reject), so the child id on a command is not
-# trustworthy input and cannot be allowed to grow this table without limit.
-_MAX_TRACKED_CHILD_SESSIONS = 512
-
-# Placeholder window for a zero-budget (always-drop) rule, which never reaches the
-# window table. Only present because a rule's window must be positive.
-_UNUSED_WINDOW_SECONDS = 1.0
-
-
-class DropReason(str, Enum):
- """Why an event was not delivered. Closed set: these become metric tags, and a
- free-form reason would blow up label cardinality (SKY-12510).
-
- POLICY keeps the `event_policy` wire value the frames_dropped counter already
- emitted before this engine existed, so the existing metric contract holds and
- only the new reasons are additive.
- """
-
- POLICY = "event_policy"
- THROTTLED = "throttled"
-
-
-@dataclass(frozen=True, slots=True)
-class Forward:
- """Deliver the event as-is.
-
- Carries no payload on purpose: the caller keeps the frame it already has and its
- existing encode path, so a forwarded event is byte-identical to what the proxy
- delivers with no policy at all. A Forward that carried an event would invite a
- re-serialization the default path must never have.
- """
-
-
-@dataclass(frozen=True, slots=True)
-class Drop:
- reason: DropReason
-
-
-@dataclass(frozen=True, slots=True)
-class Rewrite:
- """Deliver `event` in place of the original (SKY-12501/12538 synthesized errors)."""
-
- event: CdpEvent
-
-
-PolicyDecision = Union[Forward, Drop, Rewrite]
-
-# The default decision is a singleton, so the pass-through path allocates nothing per
-# event no matter how much traffic runs through it.
-FORWARD = Forward()
-
-
-@dataclass(frozen=True, slots=True)
-class RateRule:
- """At most `max_per_window` events of `method` per `window_seconds`, per session.
-
- First-wins within a window: the events that open the budget are delivered and the
- rest of the burst is dropped. That is what a rate cap means, and it is the only
- honest thing this mechanism does today.
-
- A budget of zero (`drop()`) never forwards, which is how the pack expresses an
- event no client consumes at all. `relax_when_enabled` lifts a cap entirely for a
- session observed to have enabled the event's domain — interest may only ever
- RELAX a rule, never tighten one, because unobserved interest is indistinguishable
- from absent interest (see `is_domain_enabled`).
-
- There is deliberately no `coalesce()` here. Collapsing a burst to its LATEST value
- (screencast frames, target-info churn) is the useful shape for freshness, but it
- cannot be done from a per-event decision: at the moment an event arrives, nothing
- knows a newer one is coming, so a budget of one per window delivers the OLDEST and
- drops the newest — stale state, the opposite of coalescing. True latest-wins needs
- to reach into the delivery queue (replace a still-queued event with the newer one)
- or defer a flush, both of which belong with the backpressure code rather than in a
- pure decision function. SKY-12501 shipped no rule that needs it: the queue it would
- reach into is empty on the healthy path, so keyed replacement would collapse almost
- nothing that the client's tail-drop does not already handle.
- """
-
- method: str
- max_per_window: int
- window_seconds: float
- key_params: tuple[str, ...] = ()
- reason: DropReason = DropReason.THROTTLED
- relax_when_enabled: bool = False
-
- def __post_init__(self) -> None:
- if not isinstance(self.method, str) or not self.method.strip():
- raise ValueError("method must be a non-empty string")
- if self.max_per_window < 0:
- raise ValueError("max_per_window must not be negative")
- if self.window_seconds <= 0:
- raise ValueError("window_seconds must be positive")
-
- @classmethod
- def throttle(
- cls,
- method: str,
- max_per_window: int,
- window_seconds: float,
- key_params: Sequence[str] = (),
- relax_when_enabled: bool = False,
- ) -> RateRule:
- return cls(method, max_per_window, window_seconds, tuple(key_params), DropReason.THROTTLED, relax_when_enabled)
-
- @classmethod
- def drop(cls, method: str) -> RateRule:
- """Never deliver `method`, whatever the client has enabled.
-
- The window is immaterial at a zero budget (it is never consulted) and only
- satisfies the positive-window invariant the throttle path relies on.
- """
- return cls(method, 0, _UNUSED_WINDOW_SECONDS, (), DropReason.POLICY)
-
-
-@dataclass(frozen=True, slots=True)
-class EventPolicyConfig:
- """Versioned deployment policy, as data. No rules means pass-through."""
-
- version: int = 1
- rules: tuple[RateRule, ...] = ()
-
- def __post_init__(self) -> None:
- methods = [rule.method for rule in self.rules]
- duplicates = {method for method in methods if methods.count(method) > 1}
- if duplicates:
- # Two rules for one method would make the winner depend on tuple order.
- raise ValueError(f"duplicate rules for methods: {sorted(duplicates)}")
-
-
-@dataclass(slots=True)
-class _SessionState:
- # Interest is per CHILD cdp session (None = the browser-level session), because
- # that is the scope a client actually enables a domain on. Keying it by the outer
- # session alone would let a disable on one child silently clear another's.
- enabled_domains: dict[str | None, set[str]] = field(default_factory=dict)
- windows: dict[object, tuple[float, int]] = field(default_factory=dict)
-
-
-class EventPolicyEngine:
- """Applies an EventPolicyConfig to one proxy's event stream.
-
- Stateful across calls (throttle windows, per-session domain interest) but pure in
- the sense that matters: no I/O, no ambient clock, no telemetry. `forget()` releases
- a session's state; the caller must invoke it when the session goes away.
- """
-
- def __init__(self, config: EventPolicyConfig, clock: Clock) -> None:
- self._config = config
- self._clock = clock
- self._rules: dict[str, RateRule] = {rule.method: rule for rule in config.rules}
- self._max_window_seconds = max((rule.window_seconds for rule in config.rules), default=0.0)
- self._sessions: dict[str, _SessionState] = {}
-
- @property
- def config(self) -> EventPolicyConfig:
- return self._config
-
- def decide(self, event: CdpEvent, session: ProxySession) -> PolicyDecision:
- if event.method in LIFECYCLE_EVENTS:
- # Structural, never droppable: the caller learns session ownership from
- # these downstream of this gate, so dropping one strands the owner
- # (SKY-12500). No rule can express it, whatever the config says. The detach
- # notice is also where a child session's tracked interest is retired — it
- # is the only signal that the child is gone.
- if event.method == TARGET_DETACHED_EVENT:
- self._forget_child(session, params_session_id(event))
- return FORWARD
- if not self._rules:
- # No policy configured: the engine is a transparent no-op and the caller
- # delivers exactly the bytes it would with no engine at all.
- return FORWARD
- rule = self._rules.get(event.method)
- if rule is None:
- return FORWARD
- if rule.relax_when_enabled and self._declared_interest(session, event):
- # The client asked for this domain outright, so it gets the stream in
- # full. Only True is actionable: False means "never seen enabled", which
- # a rule must never read as disinterest (see is_domain_enabled).
- return FORWARD
- return self._spend_budget(rule, event, session)
-
- def _declared_interest(self, session: ProxySession, event: CdpEvent) -> bool:
- return self.is_domain_enabled(session, event.method.partition(".")[0], event.session_id)
-
- def _spend_budget(self, rule: RateRule, event: CdpEvent, session: ProxySession) -> PolicyDecision:
- if rule.max_per_window == 0:
- # A budget that can never be spent needs no window: tracking one would
- # cost an entry per key for a rule whose answer is always the same.
- return Drop(rule.reason)
- state = self._sessions.get(session.session_id)
- if state is None:
- state = self._sessions[session.session_id] = _SessionState()
- now = self._clock()
- key = self._window_key(rule, event)
- window = state.windows.get(key)
- if window is None:
- self._make_room(state, now)
- start, count = now, 0
- elif now - window[0] >= rule.window_seconds:
- start, count = now, 0
- else:
- start, count = window
- if count < rule.max_per_window:
- state.windows[key] = (start, count + 1)
- return FORWARD
- state.windows[key] = (start, count)
- return Drop(rule.reason)
-
- @staticmethod
- def _window_key(rule: RateRule, event: CdpEvent) -> object:
- if not rule.key_params:
- return rule.method
- params = event.params or {}
- return (rule.method, tuple(str(params.get(name)) for name in rule.key_params))
-
- def _make_room(self, state: _SessionState, now: float) -> None:
- """Keep the window table bounded before admitting a new key."""
- if len(state.windows) < _MAX_WINDOWS_PER_SESSION:
- return
- # A window older than the longest configured window has expired under every
- # rule, so dropping it only hands that key a fresh budget it was owed anyway.
- cutoff = now - self._max_window_seconds
- for key in [key for key, (start, _) in state.windows.items() if start <= cutoff]:
- del state.windows[key]
- if len(state.windows) < _MAX_WINDOWS_PER_SESSION:
- return
- del state.windows[min(state.windows, key=lambda key: state.windows[key][0])]
-
- def observe_command(self, command: CdpCommand, session: ProxySession) -> None:
- """Track which CDP domains a session actually asked for, per child session.
-
- A rule can then gate on the client's declared interest instead of assuming it
- (SKY-12501). Only `<Domain>.enable`/`<Domain>.disable` move this, and only for
- a domain the protocol actually has: every other command is ignored, which is
- also what keeps an invented `<anything>.enable` out of this table.
- """
- domain, separator, verb = command.method.partition(".")
- if not separator or domain not in KNOWN_CDP_DOMAINS:
- return
- if verb not in ("enable", "disable"):
- return
- child = command.session_id
- if verb == "disable":
- state = self._sessions.get(session.session_id)
- if state is not None and child in state.enabled_domains:
- state.enabled_domains[child].discard(domain)
- return
- state = self._sessions.get(session.session_id)
- if state is None:
- state = self._sessions[session.session_id] = _SessionState()
- interest = state.enabled_domains.get(child)
- if interest is None:
- if len(state.enabled_domains) >= _MAX_TRACKED_CHILD_SESSIONS:
- # Oldest-out. A consumer must therefore treat "not enabled" as "not
- # known", never as proof of disinterest, and fail open — see
- # is_domain_enabled.
- del state.enabled_domains[next(iter(state.enabled_domains))]
- interest = state.enabled_domains[child] = set()
- interest.add(domain)
-
- def is_domain_enabled(self, session: ProxySession, domain: str, cdp_session_id: str | None = None) -> bool:
- """Whether `cdp_session_id` (None = browser level) has enabled `domain`.
-
- False also means "never seen it enabled", which is not the same as knowing the
- client is uninterested: interest is only observed while the proxy is watching,
- and the tables above are bounded. A rule that drops on False is fail-closed and
- will eventually eat real traffic — gate on True to relax a drop, not to cause
- one (SKY-12501).
- """
- state = self._sessions.get(session.session_id)
- if state is None:
- return False
- return domain in state.enabled_domains.get(cdp_session_id, frozenset())
-
- def _forget_child(self, session: ProxySession, cdp_session_id: str | None) -> None:
- if cdp_session_id is None:
- return
- state = self._sessions.get(session.session_id)
- if state is not None:
- state.enabled_domains.pop(cdp_session_id, None)
-
- def forget(self, session_id: str) -> None:
- """Release a session's state. Idempotent; safe for a session never seen."""
- self._sessions.pop(session_id, None)
-
- def tracked_sessions(self) -> int:
- return len(self._sessions)
-
- def tracked_children(self, session: ProxySession) -> int:
- state = self._sessions.get(session.session_id)
- return len(state.enabled_domains) if state is not None else 0
-
- def enabled_domain_count(self, session: ProxySession) -> int:
- state = self._sessions.get(session.session_id)
- if state is None:
- return 0
- return sum(len(domains) for domains in state.enabled_domains.values())
diff --git a/skyvern/proxy/core/policy_pack.py b/skyvern/proxy/core/policy_pack.py
deleted file mode 100644
index 49974e3df..000000000
--- a/skyvern/proxy/core/policy_pack.py
+++ /dev/null
@@ -1,79 +0,0 @@
-"""The first event-filtering policy pack (SKY-12501), as data for the engine.
-
-Two kinds of rule, and the difference matters:
-
-- An UNCONDITIONAL drop, for an event no CDP client reads at all. Safe for everyone
- precisely because it does not depend on knowing anything about the client.
-- An interest-relaxable THROTTLE, for an event clients do read but which a page can
- emit without bound. Observed `<Domain>.enable` lifts the cap; never seeing one only
- leaves the cap in place. Interest may only RELAX a rule (see `is_domain_enabled`) —
- a rule that tightened on False would eventually eat traffic a client asked for,
- because "not observed" and "not wanted" are the same value.
-
-The reduction comes from the drop set. A throttle is a ceiling on pathological volume,
-not a lever: every client library enables the domains it uses, so the relax path is the
-normal path and the cap is what remains for a stream nobody claimed.
-
-Enabling this pack is a deployment choice (`CDP_PROXY_EVENT_POLICY=noisy-v1`); the
-proxy still defaults to forwarding everything unchanged.
-"""
-
-from __future__ import annotations
-
-from skyvern.proxy.core.policy import EventPolicyConfig, RateRule
-
-# Events that neither the pinned Playwright driver nor puppeteer-core references
-# ANYWHERE — verified by enumerating every `Network.*` symbol in both sources rather
-# than by sampling the ones we expected to find:
-#
-# Network.dataReceived per-chunk byte progress (requestId, dataLength,
-# encodedDataLength, timestamp). A body's real size
-# arrives on responseReceived/loadingFinished, which
-# is what clients actually read; this only reports
-# how the transfer was split up on the way.
-# Network.resourceChangedPriority the scheduler re-ranked a pending request.
-# Diagnostic; no client models request priority.
-#
-# Both fire only for a session that enabled Network — i.e. they are noise WITHIN a
-# stream the client wanted, which is why interest cannot be what gates them.
-#
-# Adding to this list means proving the same thing about the new method: grep the
-# pinned client sources, and keep tests/unit/proxy/test_policy_pack.py's consumed-set
-# guard green. Everything a client reads is off limits no matter how noisy it looks —
-# Network.loadingFinished and the *ExtraInfo pair are high-frequency AND load-bearing.
-UNCONSUMED_EVENTS = (
- "Network.dataReceived",
- "Network.resourceChangedPriority",
-)
-
-# A page in a console loop can emit console events as fast as it can run, and each one
-# carries its serialized arguments. Console output is diagnostic and already lossy, so
-# a ceiling is safe where one on a structural stream would not be. Set well above what
-# a page emits normally: this is a runaway guard, not a budget anyone should feel.
-CONSOLE_BURST_PER_SECOND = 100
-
-NOISY_EVENT_PACK_V1 = EventPolicyConfig(
- version=1,
- rules=(
- *(RateRule.drop(method) for method in UNCONSUMED_EVENTS),
- RateRule.throttle(
- "Runtime.consoleAPICalled",
- max_per_window=CONSOLE_BURST_PER_SECOND,
- window_seconds=1.0,
- relax_when_enabled=True,
- ),
- ),
-)
-
-# The event methods this pack gates, for the driving adapter's metric-label allowlist.
-# Derived from the rules so the two cannot drift: a rule whose method is missing from
-# the allowlist reports its drops as cdp_method="other" and the reduction becomes
-# invisible per method (SKY-12510 bounds labels; this list is static, not per-request).
-POLICY_PACK_EVENT_METHODS = tuple(rule.method for rule in NOISY_EVENT_PACK_V1.rules)
-
-__all__ = [
- "CONSOLE_BURST_PER_SECOND",
- "NOISY_EVENT_PACK_V1",
- "POLICY_PACK_EVENT_METHODS",
- "UNCONSUMED_EVENTS",
-]
diff --git a/skyvern/proxy/core/screencast.py b/skyvern/proxy/core/screencast.py
deleted file mode 100644
index 70935c3f7..000000000
--- a/skyvern/proxy/core/screencast.py
+++ /dev/null
@@ -1,207 +0,0 @@
-"""Screencast protection at the proxy boundary (SKY-12502), as data plus two pure rules.
-
-Chrome's screencast is ACK-DRIVEN: the browser holds a small number of frames in flight
-and emits the next one only once an earlier one comes back acked. The ack, not the event,
-is the flow-control signal — which is what makes a rate cap here different in kind from
-every other rule in this package. A frame the proxy withholds is a frame the client never
-sees and so never acks, so once the in-flight allowance is spent the browser stops
-emitting altogether: an unacked cap does not throttle the stream, it stalls it to 0 FPS
-and never recovers. The proxy therefore owes the browser an ack for every frame it
-withholds (`screencast_frame_ack`), and the driving adapter must not withhold a frame it
-cannot ack.
-
-The two halves bound different legs, and neither covers the other:
-
-- The rate cap bounds what the CLIENT is delivered. It cannot make the browser cheaper:
- acking a withheld frame immediately is what keeps the stream alive, and that same ack
- releases the browser to encode the next frame at once.
-- `bound_start_screencast` bounds what the BROWSER produces, by clamping the capture
- params the client asks for. It is the only lever here that makes the upstream leg
- cheaper, which is why the cap alone is not the whole ticket.
-
-Scope is the EXTERNAL proxied path: clients driving Page.startScreencast through the
-proxy. Cloud live-view is VNC + Page.captureScreenshot and never touches this.
-"""
-
-from __future__ import annotations
-
-from collections.abc import Iterable
-
-from skyvern.proxy.core.frames import CdpCommand, CdpEvent, CdpFrame
-from skyvern.proxy.core.pipeline import CommandInterceptor, Direction, MiddlewarePipeline
-from skyvern.proxy.core.policy import EventPolicyConfig, RateRule
-from skyvern.proxy.core.session import ProxySession
-
-SCREENCAST_FRAME_EVENT = "Page.screencastFrame"
-SCREENCAST_FRAME_ACK_METHOD = "Page.screencastFrameAck"
-START_SCREENCAST_METHOD = "Page.startScreencast"
-
-# Frames delivered per session per second. A remote client reads a screencast to watch a
-# page, not to sample every composited frame, and the frontend consumers of a screencast
-# already tolerate frame loss. Deliberately a ceiling on the pathological case rather
-# than a target anyone should feel.
-SCREENCAST_MAX_FRAMES_PER_SECOND = 10
-
-# Ceiling on the frame the browser encodes. Chrome scales to fit maxWidth/maxHeight and
-# still reports the true page size in each frame's metadata, so a client mapping frame
-# coordinates back to the page can. Applied when absent too: Chrome reads a missing bound
-# as no bound and encodes at full viewport size, which is the expensive default this
-# exists to stop.
-SCREENCAST_MAX_DIMENSION = 1280
-
-# Ceiling on JPEG quality, which is what most directly decides a frame's size. Bounds
-# bytes where the dimension bound cannot: a full-size-but-cheap frame and a
-# small-but-pristine one cost very differently. 60 is what this product's own live-view
-# asks for, so it is a known-legible picture rather than a guessed number. `format` is
-# deliberately NOT rewritten: quality trades fidelity within a codec the client already
-# chose, whereas swapping the codec changes what its decoder is handed.
-SCREENCAST_MAX_QUALITY = 60
-
-# Floor on everyNthFrame: the browser skips capture entirely for the frames in between,
-# so this is what actually spends less CPU upstream. Coarse on purpose — it divides the
-# page's own composite rate, which the proxy cannot see, so it cannot be solved for a
-# target FPS. A page composing below twice the cap is delivered under the cap as a
-# result; the cap is a ceiling, not a promise, and this is the knob to raise if the
-# upstream leg needs to be cheaper still.
-SCREENCAST_MIN_EVERY_NTH_FRAME = 2
-
-# Keyed on the method alone, so the budget is per PROXY SESSION and a client screencasting
-# several pages shares one budget across them (AC: cap delivered frame volume per session).
-# The engine keys windows on params, and a frame's only distinguishing param is its
-# ever-changing frame number, so per-page budgets would need the engine to key on the
-# frame's CDP session — worth doing if one busy page is ever seen starving a quiet one.
-SCREENCAST_PACK_V1 = EventPolicyConfig(
- version=1,
- rules=(
- RateRule.throttle(
- SCREENCAST_FRAME_EVENT,
- max_per_window=SCREENCAST_MAX_FRAMES_PER_SECOND,
- window_seconds=1.0,
- ),
- ),
-)
-
-# For the driving adapter's metric-label allowlist, derived from the rules so the two
-# cannot drift (SKY-12510 bounds labels; this list is static, not per-request).
-SCREENCAST_PACK_EVENT_METHODS = tuple(rule.method for rule in SCREENCAST_PACK_V1.rules)
-
-
-def screencast_frame_ack(frame: CdpFrame | None) -> CdpCommand | None:
- """The ack `frame` owes the browser, or None if it owes none — because it is not a
- screencast frame at all, or is one carrying no frame number to name.
-
- Total on purpose, so one question ("what ack does this frame owe?") can be asked of
- any frame. Asking it of the frame that arrived and of the frame actually delivered,
- and comparing, is what tells a caller whether the CLIENT's ack will cover the
- original or whether the proxy still owes it (equal commands mean the delivered frame
- names the same frame number on the same session, so the client's own ack settles it).
-
- `params.sessionId` on a screencast frame is a FRAME NUMBER, not a CDP session id — the
- screencast domain reuses the name for an unrelated integer. The session this ack
- belongs to is the frame's OWN session_id. Reading params.sessionId as a session id
- (`params_session_id`, which requires a string) returns None for every real frame and
- would ack nothing at all.
-
- `id` is a placeholder: the caller allocates the real one when it remaps the command
- onto the proxy's own lane (`RequestIdRemapper.to_upstream_as_proxy`). It is fixed
- rather than incidental so that two acks for the same frame compare equal.
- """
- if not isinstance(frame, CdpEvent) or frame.method != SCREENCAST_FRAME_EVENT:
- return None
- number = (frame.params or {}).get("sessionId")
- # bool is an int in Python, and True would ack frame 1 — a frame the browser may
- # genuinely still be waiting on.
- if not isinstance(number, int) or isinstance(number, bool):
- return None
- return CdpCommand(
- id=0,
- method=SCREENCAST_FRAME_ACK_METHOD,
- params={"sessionId": number},
- session_id=frame.session_id,
- )
-
-
-def is_screencast_frame(frame: CdpFrame | None) -> bool:
- """Whether the browser is waiting on an ack for this frame.
-
- Distinct from `screencast_frame_ack(frame) is not None`: a screencast frame with no
- usable frame number owes an ack that cannot be built, and the two cases have opposite
- dispositions — one is withheld once paid, the other must never be withheld at all.
- """
- return isinstance(frame, CdpEvent) and frame.method == SCREENCAST_FRAME_EVENT
-
-
-def _bounded_dimension(value: object) -> int:
- # A non-positive or non-integer value is not a modest request: Chrome ignores it and
- # encodes unbounded, so it lands on the ceiling rather than being honoured as-is.
- if isinstance(value, int) and not isinstance(value, bool) and value > 0:
- return min(value, SCREENCAST_MAX_DIMENSION)
- return SCREENCAST_MAX_DIMENSION
-
-
-def _bounded_every_nth_frame(value: object) -> int:
- if isinstance(value, int) and not isinstance(value, bool) and value > 0:
- return max(value, SCREENCAST_MIN_EVERY_NTH_FRAME)
- return SCREENCAST_MIN_EVERY_NTH_FRAME
-
-
-def _bounded_quality(value: object) -> int:
- if isinstance(value, int) and not isinstance(value, bool) and 0 <= value <= 100:
- return min(value, SCREENCAST_MAX_QUALITY)
- return SCREENCAST_MAX_QUALITY
-
-
-def bound_start_screencast(command: CdpCommand) -> CdpCommand:
- """Clamp a client's Page.startScreencast into the range the proxy will serve.
-
- Every bound is applied whether or not the client asked, because each param's absent
- default is its expensive one: no dimension bound means full viewport, no
- everyNthFrame means every frame, and no quality means the browser's own high default.
- Absent is the common case, so it cannot be the unbounded one. `format` rides through
- untouched (see SCREENCAST_MAX_QUALITY).
- """
- params = dict(command.params or {})
- params["maxWidth"] = _bounded_dimension(params.get("maxWidth"))
- params["maxHeight"] = _bounded_dimension(params.get("maxHeight"))
- params["everyNthFrame"] = _bounded_every_nth_frame(params.get("everyNthFrame"))
- params["quality"] = _bounded_quality(params.get("quality"))
- return CdpCommand(id=command.id, method=command.method, params=params, session_id=command.session_id)
-
-
-async def bound_screencast_middleware(frame: CdpFrame, direction: Direction, session: ProxySession) -> CdpFrame:
- if (
- direction is Direction.CLIENT_TO_UPSTREAM
- and isinstance(frame, CdpCommand)
- and frame.method == START_SCREENCAST_METHOD
- ):
- return bound_start_screencast(frame)
- return frame
-
-
-def screencast_pipeline(interceptors: Iterable[CommandInterceptor] | None = None) -> MiddlewarePipeline:
- """The command half of the screencast policy.
-
- Param bounding is a command rewrite, and EventPolicyPort decides events only, so the
- two halves of this one policy are wired through different seams and a deployment must
- turn both on together (see `skyvern.proxy.__main__`). Rewriting commands in general is
- the interception seam's job (SKY-12535); this is the one rule that cannot wait for it.
- """
- return MiddlewarePipeline([bound_screencast_middleware], interceptors)
-
-
-__all__ = [
- "SCREENCAST_FRAME_ACK_METHOD",
- "SCREENCAST_FRAME_EVENT",
- "SCREENCAST_MAX_DIMENSION",
- "SCREENCAST_MAX_FRAMES_PER_SECOND",
- "SCREENCAST_MAX_QUALITY",
- "SCREENCAST_MIN_EVERY_NTH_FRAME",
- "SCREENCAST_PACK_EVENT_METHODS",
- "SCREENCAST_PACK_V1",
- "START_SCREENCAST_METHOD",
- "bound_screencast_middleware",
- "bound_start_screencast",
- "is_screencast_frame",
- "screencast_frame_ack",
- "screencast_pipeline",
-]
diff --git a/skyvern/proxy/core/session.py b/skyvern/proxy/core/session.py
deleted file mode 100644
index 18526fe9a..000000000
--- a/skyvern/proxy/core/session.py
+++ /dev/null
@@ -1,171 +0,0 @@
-from __future__ import annotations
-
-from collections.abc import Mapping
-from dataclasses import dataclass, field
-from enum import Enum
-
-
-def _require_non_empty(value: object, field_name: str) -> str:
- if not isinstance(value, str) or not value.strip():
- raise ValueError(f"{field_name} must be a non-empty string")
- return value
-
-
-class UpstreamClosedError(Exception):
- """Raised by upstream connections when the browser side is gone."""
-
-
-@dataclass(frozen=True)
-class Principal:
- """Authenticated caller identity; opaque to the core beyond these fields."""
-
- principal_id: str
- organization_id: str | None = None
-
- def __post_init__(self) -> None:
- _require_non_empty(self.principal_id, "principal_id")
- if self.organization_id is not None:
- _require_non_empty(self.organization_id, "organization_id")
-
-
-class SessionResolutionStatus(Enum):
- ACTIVE = "active"
- UNKNOWN = "unknown"
- PENDING = "pending"
- CLOSED = "closed"
- EXPIRED = "expired"
-
-
-@dataclass(frozen=True)
-class ResolvedSession:
- """Connect-time routing for one session: which adapter dials which endpoint.
-
- upstream_ws_url and connect_headers are excluded from repr — the URL's path
- and query can carry session tokens and headers can carry credentials, so
- neither may reach logs or clients.
- """
-
- session_id: str
- upstream_adapter: str
- upstream_ws_url: str = field(repr=False)
- organization_id: str | None = None
- connect_headers: Mapping[str, str] = field(default_factory=dict, repr=False)
-
- def __post_init__(self) -> None:
- _require_non_empty(self.session_id, "session_id")
- _require_non_empty(self.upstream_adapter, "upstream_adapter")
- _require_non_empty(self.upstream_ws_url, "upstream_ws_url")
- if self.organization_id is not None:
- _require_non_empty(self.organization_id, "organization_id")
- if not isinstance(self.connect_headers, Mapping):
- raise ValueError("connect_headers must be a mapping")
-
-
-@dataclass(frozen=True)
-class SessionResolution:
- """Outcome of resolving a session id; `session` is set exactly when ACTIVE.
-
- Negative statuses are explicit so callers reject the client before any
- upstream dial ever happens. organization_id names the owning organization
- when the registry knows it — including on negatives, so callers can hide
- lifecycle state from non-owners. expires_in_seconds is an optional hint of
- how long an ACTIVE resolution stays valid (caches must not serve it longer).
- """
-
- status: SessionResolutionStatus
- session: ResolvedSession | None = None
- organization_id: str | None = None
- expires_in_seconds: float | None = None
-
- def __post_init__(self) -> None:
- if (self.status is SessionResolutionStatus.ACTIVE) != (self.session is not None):
- raise ValueError("session must be set exactly when status is ACTIVE")
- if self.session is not None and self.organization_id != self.session.organization_id:
- raise ValueError("organization_id must match the resolved session's organization")
- if self.expires_in_seconds is not None:
- if self.status is not SessionResolutionStatus.ACTIVE:
- raise ValueError("expires_in_seconds is only valid on ACTIVE resolutions")
- if self.expires_in_seconds <= 0:
- raise ValueError("expires_in_seconds must be positive")
-
- @classmethod
- def active(cls, session: ResolvedSession, expires_in_seconds: float | None = None) -> SessionResolution:
- return cls(SessionResolutionStatus.ACTIVE, session, session.organization_id, expires_in_seconds)
-
- @classmethod
- def unknown(cls) -> SessionResolution:
- return cls(SessionResolutionStatus.UNKNOWN)
-
- @classmethod
- def pending(cls, organization_id: str | None = None) -> SessionResolution:
- """The session exists but has not published a dialable upstream yet."""
- return cls(SessionResolutionStatus.PENDING, organization_id=organization_id)
-
- @classmethod
- def closed(cls, organization_id: str | None = None) -> SessionResolution:
- return cls(SessionResolutionStatus.CLOSED, organization_id=organization_id)
-
- @classmethod
- def expired(cls, organization_id: str | None = None) -> SessionResolution:
- return cls(SessionResolutionStatus.EXPIRED, organization_id=organization_id)
-
-
-def principal_owns_resolution(principal: Principal, resolution: SessionResolution) -> bool:
- """The org-vs-session authz rule shared by every AuthPort adapter.
-
- False for both an unknown session and one owned by another organization, so
- callers emit a single uniform rejection: a foreign session is
- indistinguishable from a nonexistent one, leaking neither existence nor
- lifecycle state across organizations. A resolution with no owning
- organization is dev-only and open to any principal.
- """
- if resolution.status is SessionResolutionStatus.UNKNOWN:
- return False
- owner = resolution.organization_id
- return owner is None or principal.organization_id == owner
-
-
-@dataclass
-class ProxySession:
- """Routable browser session state owned by the proxy core.
-
- upstream_ws_url and connect_headers are excluded from repr for the same
- reason as on ResolvedSession: they can carry session tokens and credentials.
- """
-
- session_id: str
- upstream_ws_url: str = field(repr=False)
- principal: Principal | None = None
- attached_cdp_session_ids: set[str] = field(default_factory=set)
- connect_headers: Mapping[str, str] = field(default_factory=dict, repr=False)
-
- def __post_init__(self) -> None:
- _require_non_empty(self.session_id, "session_id")
- _require_non_empty(self.upstream_ws_url, "upstream_ws_url")
- if not isinstance(self.attached_cdp_session_ids, set):
- raise ValueError("attached_cdp_session_ids must be a set")
- for cdp_session_id in self.attached_cdp_session_ids:
- _require_non_empty(cdp_session_id, "cdp_session_id")
- if not isinstance(self.connect_headers, Mapping):
- raise ValueError("connect_headers must be a mapping")
-
- def allows_principal(self, principal: Principal) -> bool:
- return self.principal is None or self.principal == principal
-
- def attach_cdp_session(self, cdp_session_id: str) -> bool:
- _require_non_empty(cdp_session_id, "cdp_session_id")
- if cdp_session_id in self.attached_cdp_session_ids:
- return False
- self.attached_cdp_session_ids.add(cdp_session_id)
- return True
-
- def detach_cdp_session(self, cdp_session_id: str) -> bool:
- _require_non_empty(cdp_session_id, "cdp_session_id")
- if cdp_session_id not in self.attached_cdp_session_ids:
- return False
- self.attached_cdp_session_ids.remove(cdp_session_id)
- return True
-
- def has_cdp_session(self, cdp_session_id: str) -> bool:
- _require_non_empty(cdp_session_id, "cdp_session_id")
- return cdp_session_id in self.attached_cdp_session_ids
diff --git a/skyvern/proxy/ports/__init__.py b/skyvern/proxy/ports/__init__.py
deleted file mode 100644
index 2dbcf97c1..000000000
--- a/skyvern/proxy/ports/__init__.py
+++ /dev/null
@@ -1,168 +0,0 @@
-"""Driven ports for the CDP proxy core.
-
-Each port is a Protocol; adapters implement them without the core knowing which.
-Adding a new hosted-browser vendor means implementing UpstreamBrowserPort alone —
-zero core changes. Every adapter must pass the reusable contract suite for its
-port (see tests/unit/proxy/).
-"""
-
-from __future__ import annotations
-
-from typing import Mapping, Protocol, runtime_checkable
-
-from skyvern.proxy.core.errors import (
- LaunchEnvironmentError,
- LaunchTimeoutError,
- ProtocolConfigurationError,
- TransientConnectionError,
- UpstreamConnectError,
- VendorAuthError,
- VendorRateLimitError,
-)
-from skyvern.proxy.core.frames import CdpCommand, CdpEvent
-from skyvern.proxy.core.policy import PolicyDecision
-from skyvern.proxy.core.session import (
- Principal,
- ProxySession,
- ResolvedSession,
- SessionResolution,
- SessionResolutionStatus,
-)
-
-
-@runtime_checkable
-class UpstreamConnection(Protocol):
- """One live browser-side connection; frames are raw CDP JSON strings."""
-
- async def send(self, raw: str) -> None:
- """Sends one raw frame; raises UpstreamClosedError once the upstream is gone."""
- ...
-
- async def receive(self) -> str:
- """Returns the next raw frame; raises UpstreamClosedError once the upstream is gone."""
- ...
-
- async def close(self) -> None:
- """Releases the connection and any resources behind it. Idempotent."""
- ...
-
-
-@runtime_checkable
-class UpstreamBrowserPort(Protocol):
- """Connects to the browser side of a session. The only port a new vendor needs.
-
- Failures raise UpstreamConnectError subclasses (skyvern.proxy.core.errors) so
- callers never branch on transport exception types. Any credentials the upstream
- needs are attached by the adapter from proxy-operator configuration — never
- taken from client input.
- """
-
- async def connect(self, session: ProxySession) -> UpstreamConnection: ...
-
-
-@runtime_checkable
-class SessionRegistryPort(Protocol):
- """Resolves a session id into upstream routing at connect/attach time.
-
- Resolution runs once per client connection, never per message. Unknown,
- pending, closed, and expired sessions come back as explicit negative
- results so callers reject the client before any upstream dial. Session
- lifecycle writes stay with the application; the registry only reads
- (and caches — see TtlCachingSessionRegistry).
- """
-
- async def resolve(self, session_id: str) -> SessionResolution: ...
-
- async def invalidate(self, session_id: str) -> None:
- """Drops any cached resolution for the session; called when it closes."""
- ...
-
-
-@runtime_checkable
-class AuthPort(Protocol):
- async def authenticate(self, credentials: Mapping[str, str]) -> Principal | None:
- """Returns the caller's principal, or None to reject the connection.
-
- credentials is a normalized mapping the driving adapter builds from the
- request header, query string, and URL path — a WS client that cannot set
- headers (e.g. puppeteer) can still carry the key in the URL. The client
- credential is consumed here and never forwarded upstream.
- """
- ...
-
- def authorize(self, principal: Principal, resolution: SessionResolution) -> bool:
- """Whether `principal` may use the resolved session; runs before any dial.
-
- A non-owning organization (or an unknown session) must be rejected
- identically to a nonexistent one — no cross-org existence or lifecycle
- oracle.
- """
- ...
-
-
-@runtime_checkable
-class MetricsPort(Protocol):
- """Thin telemetry seam: the core emits through these three ops and never
- imports a telemetry SDK. Adapters (NoOp in OSS, OTel in cloud) map them onto
- counters/histograms/up-down instruments. Tags carry only safe dims (org_id,
- session_id, cdp method/domain, direction, reason) — never URLs or credentials.
- """
-
- def increment(self, name: str, amount: int = 1, tags: Mapping[str, str] | None = None) -> None:
- """Adds `amount` to a monotonic counter (frames, bytes, connects, rejects)."""
- ...
-
- def observe(self, name: str, value: float, tags: Mapping[str, str] | None = None) -> None:
- """Records `value` into a histogram (e.g. command round-trip latency, seconds)."""
- ...
-
- def gauge(self, name: str, amount: int, tags: Mapping[str, str] | None = None) -> None:
- """Shifts an up-down counter by `amount` (+1/-1 for active session count)."""
- ...
-
-
-@runtime_checkable
-class EventPolicyPort(Protocol):
- """Decides what a client sees of the upstream event stream.
-
- The default (ForwardAllEventPolicy) forwards everything unchanged; a policy is
- what a deployment opts into. Target lifecycle events are never droppable — the
- driving adapter learns session ownership from them downstream of this gate.
- """
-
- def decide(self, event: CdpEvent, session: ProxySession) -> PolicyDecision:
- """Forward the event as-is, drop it (with a reason for the decision trace),
- or replace it. Forward carries no payload so the caller delivers the frame it
- already holds — an unchanged event is never re-serialized on the way through.
- """
- ...
-
- def observe_command(self, command: CdpCommand, session: ProxySession) -> None:
- """Feed a client command in, so a policy can gate on the CDP domains the
- session actually enabled rather than assuming its interest."""
- ...
-
- def forget(self, session_id: str) -> None:
- """Release any state held for the session. Idempotent."""
- ...
-
-
-__all__ = [
- "AuthPort",
- "EventPolicyPort",
- "LaunchEnvironmentError",
- "LaunchTimeoutError",
- "MetricsPort",
- "PolicyDecision",
- "ProtocolConfigurationError",
- "ResolvedSession",
- "SessionRegistryPort",
- "SessionResolution",
- "SessionResolutionStatus",
- "TransientConnectionError",
- "UpstreamBrowserPort",
- "UpstreamConnectError",
- "UpstreamConnection",
- "VendorAuthError",
- "VendorRateLimitError",
-]
diff --git a/skyvern/webeye/browser_factory.py b/skyvern/webeye/browser_factory.py
index def99102b..3b84da223 100644
--- a/skyvern/webeye/browser_factory.py
+++ b/skyvern/webeye/browser_factory.py
@@ -4,7 +4,6 @@ import asyncio
import os
import pathlib
import platform
-import random
import re
import shutil
import socket
@@ -485,21 +484,6 @@ class BrowserContextFactory:
if settings.BROWSER_LOCALE:
args["locale"] = settings.BROWSER_LOCALE
- # Custom per-request proxy URL takes precedence over the global proxy pool.
- # Users may pass proxy_location={"url": "http://user:pass@host:port"} to
- # route this specific task/session through their own proxy server.
- if isinstance(proxy_location, dict) and "url" in proxy_location:
- proxy_url = proxy_location["url"]
- if _is_valid_proxy_url(proxy_url):
- args["proxy"] = {"server": proxy_url}
- LOG.info("Using custom per-request proxy URL", proxy_url=_redact_proxy_url(proxy_url))
- else:
- LOG.warning("Invalid custom proxy URL provided, ignoring", proxy_url=_redact_proxy_url(proxy_url))
- elif settings.ENABLE_PROXY:
- proxy_config = setup_proxy()
- if proxy_config:
- args["proxy"] = proxy_config
-
if isinstance(proxy_location, ProxyLocation):
if tz_info := get_tzinfo_from_proxy(proxy_location=proxy_location):
args["timezone_id"] = tz_info.key
@@ -562,79 +546,6 @@ class BrowserContextFactory:
raise UnknownErrorWhileCreatingBrowserContext(browser_type, e) from e
-def setup_proxy() -> dict | None:
- if not settings.HOSTED_PROXY_POOL or settings.HOSTED_PROXY_POOL.strip() == "":
- LOG.warning("No proxy server value found. Continuing without using proxy...")
- return None
-
- proxy_servers = [server.strip() for server in settings.HOSTED_PROXY_POOL.split(",") if server.strip()]
-
- if not proxy_servers:
- LOG.warning("Proxy pool contains only empty values. Continuing without proxy...")
- return None
-
- valid_proxies = []
- for proxy in proxy_servers:
- if _is_valid_proxy_url(proxy):
- valid_proxies.append(proxy)
- else:
- LOG.warning(f"Invalid proxy URL format: {proxy}")
-
- if not valid_proxies:
- LOG.warning("No valid proxy URLs found. Continuing without proxy...")
- return None
-
- try:
- proxy_server = random.choice(valid_proxies)
- proxy_creds = _get_proxy_server_creds(proxy_server)
-
- LOG.info("Found proxy server creds, using them...")
-
- return {
- "server": proxy_server,
- "username": proxy_creds.get("username", ""),
- "password": proxy_creds.get("password", ""),
- }
- except Exception as e:
- LOG.warning(f"Error setting up proxy: {e}. Continuing without proxy...")
- return None
-
-
-def _redact_proxy_url(url: str) -> str:
- try:
- parsed = urlparse(url)
- if not parsed.hostname:
- return "<redacted>"
- host = parsed.hostname
- if parsed.port:
- host = f"{host}:{parsed.port}"
- userinfo = ""
- if parsed.username:
- userinfo = f"{parsed.username}:***@" if parsed.password else f"{parsed.username}@"
- return f"{parsed.scheme}://{userinfo}{host}"
- except Exception:
- return "<redacted>"
-
-
-def _is_valid_proxy_url(url: str) -> bool:
- PROXY_PATTERN = re.compile(r"^(http|https|socks5):\/\/([^:@]+(:[^@]*)?@)?[^\s:\/]+(:\d+)?$")
- try:
- parsed = urlparse(url)
- if not parsed.scheme or not parsed.netloc:
- return False
- return bool(PROXY_PATTERN.match(url))
- except Exception:
- return False
-
-
-def _get_proxy_server_creds(proxy: str) -> dict:
- parsed_url = urlparse(proxy)
- if parsed_url.username and parsed_url.password:
- return {"username": parsed_url.username, "password": parsed_url.password}
- LOG.warning("No credentials found in the proxy URL.")
- return {}
-
-
def _is_display_server_error(error: Exception) -> bool:
"""Return True if the worker cannot initialize the browser display/graphics stack.
diff --git a/tests/unit/proxy/__init__.py b/tests/unit/proxy/__init__.py
deleted file mode 100644
index e69de29bb..000000000
diff --git a/tests/unit/proxy/_e2e_gate.py b/tests/unit/proxy/_e2e_gate.py
deleted file mode 100644
index 822fb633a..000000000
--- a/tests/unit/proxy/_e2e_gate.py
+++ /dev/null
@@ -1,66 +0,0 @@
-"""Whether an optional E2E dependency is present, and whether its absence may skip.
-
-The compatibility suite needs things a bare unit-test runner has no reason to carry:
-a real Chromium, node, the pinned JS clients. Skipping when they are missing keeps the
-suite usable on a laptop; skipping SILENTLY in CI turns the merge gate into decoration.
-
-So CI sets CDP_PROXY_E2E_REQUIRED=1 and every one of those skips becomes a failure.
-The suite is a gate only on a runner that actually has to run it.
-"""
-
-from __future__ import annotations
-
-import importlib.util
-import os
-import shutil
-import subprocess
-from functools import lru_cache
-from pathlib import Path
-
-import pytest
-
-E2E_REQUIRED_ENV = "CDP_PROXY_E2E_REQUIRED"
-
-JS_CLIENT_DIR = Path(__file__).parent / "fixtures" / "js_clients"
-
-
-def e2e_required() -> bool:
- return os.environ.get(E2E_REQUIRED_ENV) == "1"
-
-
-def require(available: bool, reason: str) -> None:
- """Skip when a dependency is missing — unless CI declared it mandatory."""
- if available:
- return
- if e2e_required():
- pytest.fail(f"{reason} — required because {E2E_REQUIRED_ENV}=1")
- pytest.skip(reason)
-
-
-@lru_cache(maxsize=1)
-def find_node() -> str | None:
- return shutil.which("node")
-
-
-@lru_cache(maxsize=1)
-def playwright_python_available() -> bool:
- return importlib.util.find_spec("playwright.async_api") is not None
-
-
-@lru_cache(maxsize=1)
-def js_clients_installed() -> bool:
- """The pinned clients are vendored by `npm ci` in fixtures/js_clients."""
- return (JS_CLIENT_DIR / "node_modules").is_dir()
-
-
-def run_js_client(script: str, endpoint: str, timeout: float = 60.0) -> subprocess.CompletedProcess[str]:
- """Drives one JS CDP client against `endpoint`. Bounded: a hung client fails the
- test rather than hanging the job."""
- return subprocess.run(
- [find_node() or "node", script, endpoint],
- cwd=JS_CLIENT_DIR,
- capture_output=True,
- text=True,
- timeout=timeout,
- check=False,
- )
diff --git a/tests/unit/proxy/fixtures/__init__.py b/tests/unit/proxy/fixtures/__init__.py
deleted file mode 100644
index e69de29bb..000000000
diff --git a/tests/unit/proxy/fixtures/browser_client_a.jsonl b/tests/unit/proxy/fixtures/browser_client_a.jsonl
deleted file mode 100644
index f4ca434be..000000000
--- a/tests/unit/proxy/fixtures/browser_client_a.jsonl
+++ /dev/null
@@ -1,15 +0,0 @@
-{"id":1,"method":"Target.setDiscoverTargets","params":{"discover":true}}
-{"id":2,"method":"Target.attachToTarget","params":{"targetId":"page-a","flatten":true}}
-{"id":2,"result":{"sessionId":"attached-a"}}
-{"method":"Target.attachedToTarget","params":{"sessionId":"attached-a","targetInfo":{"targetId":"page-a","type":"page","title":"","url":"about:blank","attached":true,"canAccessOpener":false}}}
-{"id":3,"method":"Page.enable","sessionId":"attached-a"}
-{"id":3,"result":{},"sessionId":"attached-a"}
-{"id":4,"method":"Network.enable","params":{"maxTotalBufferSize":1000000},"sessionId":"attached-a"}
-{"id":4,"result":{},"sessionId":"attached-a"}
-{"id":5,"method":"Runtime.enable","sessionId":"attached-a"}
-{"id":5,"result":{},"sessionId":"attached-a"}
-{"method":"Runtime.executionContextCreated","params":{"context":{"id":1,"origin":"","name":"","auxData":{"isDefault":true,"type":"default"}}},"sessionId":"attached-a"}
-{"method":"Page.loadEventFired","params":{"timestamp":123.45},"sessionId":"attached-a"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"request-a","request":{"url":"about:blank","method":"GET","headers":{}},"type":"Document","timestamp":123.46,"wallTime":1700000000.0},"sessionId":"attached-a"}
-{"id":6,"method":"Runtime.evaluate","params":{"expression":"document.title","returnByValue":true},"sessionId":"attached-a"}
-{"id":6,"result":{"result":{"type":"string","value":"","description":""}},"sessionId":"attached-a"}
diff --git a/tests/unit/proxy/fixtures/browser_client_b.jsonl b/tests/unit/proxy/fixtures/browser_client_b.jsonl
deleted file mode 100644
index 3a2d0fb08..000000000
--- a/tests/unit/proxy/fixtures/browser_client_b.jsonl
+++ /dev/null
@@ -1,15 +0,0 @@
-{"id":11,"method":"Target.setDiscoverTargets","params":{"discover":true}}
-{"id":12,"method":"Target.attachToTarget","params":{"targetId":"page-b","flatten":true}}
-{"id":12,"result":{"sessionId":"attached-b"}}
-{"method":"Target.attachedToTarget","params":{"sessionId":"attached-b","targetInfo":{"targetId":"page-b","type":"page","title":"","url":"about:blank","attached":true,"canAccessOpener":false}}}
-{"id":13,"method":"Page.enable","sessionId":"attached-b"}
-{"id":13,"result":{},"sessionId":"attached-b"}
-{"id":14,"method":"Network.enable","sessionId":"attached-b"}
-{"id":14,"result":{},"sessionId":"attached-b"}
-{"id":15,"method":"Runtime.enable","sessionId":"attached-b"}
-{"id":15,"result":{},"sessionId":"attached-b"}
-{"method":"Runtime.executionContextCreated","params":{"context":{"id":2,"origin":"","name":"","auxData":{"isDefault":true,"type":"default"}}},"sessionId":"attached-b"}
-{"method":"Network.responseReceived","params":{"requestId":"request-b","response":{"url":"about:blank","status":200,"statusText":"OK","mimeType":"text/html","headers":{}}},"sessionId":"attached-b"}
-{"id":16,"method":"Runtime.evaluate","params":{"expression":"document.readyState","returnByValue":true},"sessionId":"attached-b"}
-{"id":16,"result":{"result":{"type":"string","value":"complete","description":"complete"}},"sessionId":"attached-b"}
-{"method":"Page.loadEventFired","params":{"timestamp":223.45},"sessionId":"attached-b"}
diff --git a/tests/unit/proxy/fixtures/js_clients/.npmrc b/tests/unit/proxy/fixtures/js_clients/.npmrc
deleted file mode 100644
index 5c429893e..000000000
--- a/tests/unit/proxy/fixtures/js_clients/.npmrc
+++ /dev/null
@@ -1,5 +0,0 @@
-# Supply chain protection: do not run lifecycle scripts (preinstall, install,
-# postinstall) on npm install. Blocks worms like "Shai-Hulud" from executing
-# on a compromised dependency before we notice. If a package genuinely needs
-# its install script, use @lavamoat/allow-scripts to allowlist it.
-ignore-scripts=true
diff --git a/tests/unit/proxy/fixtures/js_clients/package-lock.json b/tests/unit/proxy/fixtures/js_clients/package-lock.json
deleted file mode 100644
index b917946d6..000000000
--- a/tests/unit/proxy/fixtures/js_clients/package-lock.json
+++ /dev/null
@@ -1,1010 +0,0 @@
-{
- "name": "cdp-proxy-js-compat-clients",
- "lockfileVersion": 3,
- "requires": true,
- "packages": {
- "": {
- "name": "cdp-proxy-js-compat-clients",
- "dependencies": {
- "playwright-core": "1.46.0",
- "puppeteer-core": "23.11.1"
- }
- },
- "node_modules/@puppeteer/browsers": {
- "version": "2.6.1",
- "resolved": "https://registry.npmjs.org/@puppeteer/browsers/-/browsers-2.6.1.tgz",
- "integrity": "sha512-aBSREisdsGH890S2rQqK82qmQYU3uFpSH8wcZWHgHzl3LfzsxAKbLNiAG9mO8v1Y0UICBeClICxPJvyr0rcuxg==",
- "license": "Apache-2.0",
- "dependencies": {
- "debug": "^4.4.0",
- "extract-zip": "^2.0.1",
- "progress": "^2.0.3",
- "proxy-agent": "^6.5.0",
- "semver": "^7.6.3",
- "tar-fs": "^3.0.6",
- "unbzip2-stream": "^1.4.3",
- "yargs": "^17.7.2"
- },
- "bin": {
- "browsers": "lib/cjs/main-cli.js"
- },
- "engines": {
- "node": ">=18"
- }
- },
- "node_modules/@tootallnate/quickjs-emscripten": {
- "version": "0.23.0",
- "resolved": "https://registry.npmjs.org/@tootallnate/quickjs-emscripten/-/quickjs-emscripten-0.23.0.tgz",
- "integrity": "sha512-C5Mc6rdnsaJDjO3UpGW/CQTHtCKaYlScZTly4JIu97Jxo/odCiH0ITnDXSJPTOrEKk/ycSZ0AOgTmkDtkOsvIA==",
- "license": "MIT"
- },
- "node_modules/@types/node": {
- "version": "26.1.1",
- "resolved": "https://registry.npmjs.org/@types/node/-/node-26.1.1.tgz",
- "integrity": "sha512-nxAkRSVkN1Y0JC1W8ky/fTfkGsMmcrRsbx+3XoZE+rMOX71kLYTV7fLXpqud1GpbpP5TuffXFqfX7fH2GgZREw==",
- "license": "MIT",
- "optional": true,
- "dependencies": {
- "undici-types": "~8.3.0"
- }
- },
- "node_modules/@types/yauzl": {
- "version": "2.10.3",
- "resolved": "https://registry.npmjs.org/@types/yauzl/-/yauzl-2.10.3.tgz",
- "integrity": "sha512-oJoftv0LSuaDZE3Le4DbKX+KS9G36NzOeSap90UIK0yMA/NhKJhqlSGtNDORNRaIbQfzjXDrQa0ytJ6mNRGz/Q==",
- "license": "MIT",
- "optional": true,
- "dependencies": {
- "@types/node": "*"
- }
- },
- "node_modules/agent-base": {
- "version": "7.1.4",
- "resolved": "https://registry.npmjs.org/agent-base/-/agent-base-7.1.4.tgz",
- "integrity": "sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ==",
- "license": "MIT",
- "engines": {
- "node": ">= 14"
- }
- },
- "node_modules/ansi-regex": {
- "version": "5.0.1",
- "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz",
- "integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==",
- "license": "MIT",
- "engines": {
- "node": ">=8"
- }
- },
- "node_modules/ansi-styles": {
- "version": "4.3.0",
- "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz",
- "integrity": "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==",
- "license": "MIT",
- "dependencies": {
- "color-convert": "^2.0.1"
- },
- "engines": {
- "node": ">=8"
- },
- "funding": {
- "url": "https://github.com/chalk/ansi-styles?sponsor=1"
- }
- },
- "node_modules/ast-types": {
- "version": "0.13.4",
- "resolved": "https://registry.npmjs.org/ast-types/-/ast-types-0.13.4.tgz",
- "integrity": "sha512-x1FCFnFifvYDDzTaLII71vG5uvDwgtmDTEVWAxrgeiR8VjMONcCXJx7E+USjDtHlwFmt9MysbqgF9b9Vjr6w+w==",
- "license": "MIT",
- "dependencies": {
- "tslib": "^2.0.1"
- },
- "engines": {
- "node": ">=4"
- }
- },
- "node_modules/b4a": {
- "version": "1.8.1",
- "resolved": "https://registry.npmjs.org/b4a/-/b4a-1.8.1.tgz",
- "integrity": "sha512-aiqre1Nr0B/6DgE2N5vwTc+2/oQZ4Wh1t4NznYY4E00y8LCt6NqdRv81so00oo27D8MVKTpUa/MwUUtBLXCoDw==",
- "license": "Apache-2.0",
- "peerDependencies": {
- "react-native-b4a": "*"
- },
- "peerDependenciesMeta": {
- "react-native-b4a": {
- "optional": true
- }
- }
- },
- "node_modules/bare-events": {
- "version": "2.9.1",
- "resolved": "https://registry.npmjs.org/bare-events/-/bare-events-2.9.1.tgz",
- "integrity": "sha512-Z0oHEHAFDZkffN8Qc39zNZjQlMDkPJRyyyZieU1VH7u8c5S+qHZ2S8ixdKIAxEjfHO7FJxXmJWgteOghVanIsg==",
- "license": "Apache-2.0",
- "peerDependencies": {
- "bare-abort-controller": "*"
- },
- "peerDependenciesMeta": {
- "bare-abort-controller": {
- "optional": true
- }
- }
- },
- "node_modules/bare-fs": {
- "version": "4.7.4",
- "resolved": "https://registry.npmjs.org/bare-fs/-/bare-fs-4.7.4.tgz",
- "integrity": "sha512-y1kC+ffIx/tPLdTE693uNjHfzTfr+ravR5tvWlMXe25nELbkqV400S71qHDwbkAQ1FVEZobB1NFRzFbCCcyBCQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "bare-events": "^2.5.4",
- "bare-path": "^3.0.0",
- "bare-stream": "^2.6.4",
- "bare-url": "^2.2.2",
- "fast-fifo": "^1.3.2"
- },
- "engines": {
- "bare": ">=1.16.0"
- },
- "peerDependencies": {
- "bare-buffer": "*"
- },
- "peerDependenciesMeta": {
- "bare-buffer": {
- "optional": true
- }
- }
- },
- "node_modules/bare-path": {
- "version": "3.1.1",
- "resolved": "https://registry.npmjs.org/bare-path/-/bare-path-3.1.1.tgz",
- "integrity": "sha512-JprUlveX3QjApC1cTpsUOiscADftCGVWkzitbHsRqv84hzYwYHw2mbluddsq5TvI8mH/8Ov1f4BiMAdcB0oYnQ==",
- "license": "Apache-2.0"
- },
- "node_modules/bare-stream": {
- "version": "2.13.3",
- "resolved": "https://registry.npmjs.org/bare-stream/-/bare-stream-2.13.3.tgz",
- "integrity": "sha512-Kc+brLqvEqGkjyfiwJmImAOqLZL7OsoLKuavx+hJjgVV3nLTOjloJyPMFxjUPerGGHrNH0fLU06jjykMLWrERQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "b4a": "^1.8.1",
- "streamx": "^2.25.0",
- "teex": "^1.0.1"
- },
- "peerDependencies": {
- "bare-abort-controller": "*",
- "bare-buffer": "*",
- "bare-events": "*"
- },
- "peerDependenciesMeta": {
- "bare-abort-controller": {
- "optional": true
- },
- "bare-buffer": {
- "optional": true
- },
- "bare-events": {
- "optional": true
- }
- }
- },
- "node_modules/bare-url": {
- "version": "2.4.5",
- "resolved": "https://registry.npmjs.org/bare-url/-/bare-url-2.4.5.tgz",
- "integrity": "sha512-K+y9xF1tN+CdPu4qWwr0QiK1Al07eFPGYK5M2pDXcmHdMdgC/tT/bpmMe1hrmRHaidKLkXrC+cRNYf3XVDUhSQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "bare-path": "^3.0.0"
- }
- },
- "node_modules/base64-js": {
- "version": "1.5.1",
- "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz",
- "integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==",
- "funding": [
- {
- "type": "github",
- "url": "https://github.com/sponsors/feross"
- },
- {
- "type": "patreon",
- "url": "https://www.patreon.com/feross"
- },
- {
- "type": "consulting",
- "url": "https://feross.org/support"
- }
- ],
- "license": "MIT"
- },
- "node_modules/basic-ftp": {
- "version": "5.3.1",
- "resolved": "https://registry.npmjs.org/basic-ftp/-/basic-ftp-5.3.1.tgz",
- "integrity": "sha512-bopVNp6ugyA150DDuZfPFdt1KZ5a94ZDiwX4hMgZDzF+GttD80lEy8kj98kbyhLXnPvhtIo93mdnLIjpCAeeOw==",
- "license": "MIT",
- "engines": {
- "node": ">=10.0.0"
- }
- },
- "node_modules/buffer": {
- "version": "5.7.1",
- "resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",
- "integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",
- "funding": [
- {
- "type": "github",
- "url": "https://github.com/sponsors/feross"
- },
- {
- "type": "patreon",
- "url": "https://www.patreon.com/feross"
- },
- {
- "type": "consulting",
- "url": "https://feross.org/support"
- }
- ],
- "license": "MIT",
- "dependencies": {
- "base64-js": "^1.3.1",
- "ieee754": "^1.1.13"
- }
- },
- "node_modules/buffer-crc32": {
- "version": "0.2.13",
- "resolved": "https://registry.npmjs.org/buffer-crc32/-/buffer-crc32-0.2.13.tgz",
- "integrity": "sha512-VO9Ht/+p3SN7SKWqcrgEzjGbRSJYTx+Q1pTQC0wrWqHx0vpJraQ6GtHx8tvcg1rlK1byhU5gccxgOgj7B0TDkQ==",
- "license": "MIT",
- "engines": {
- "node": "*"
- }
- },
- "node_modules/chromium-bidi": {
- "version": "0.11.0",
- "resolved": "https://registry.npmjs.org/chromium-bidi/-/chromium-bidi-0.11.0.tgz",
- "integrity": "sha512-6CJWHkNRoyZyjV9Rwv2lYONZf1Xm0IuDyNq97nwSsxxP3wf5Bwy15K5rOvVKMtJ127jJBmxFUanSAOjgFRxgrA==",
- "license": "Apache-2.0",
- "dependencies": {
- "mitt": "3.0.1",
- "zod": "3.23.8"
- },
- "peerDependencies": {
- "devtools-protocol": "*"
- }
- },
- "node_modules/cliui": {
- "version": "8.0.1",
- "resolved": "https://registry.npmjs.org/cliui/-/cliui-8.0.1.tgz",
- "integrity": "sha512-BSeNnyus75C4//NQ9gQt1/csTXyo/8Sb+afLAkzAptFuMsod9HFokGNudZpi/oQV73hnVK+sR+5PVRMd+Dr7YQ==",
- "license": "ISC",
- "dependencies": {
- "string-width": "^4.2.0",
- "strip-ansi": "^6.0.1",
- "wrap-ansi": "^7.0.0"
- },
- "engines": {
- "node": ">=12"
- }
- },
- "node_modules/color-convert": {
- "version": "2.0.1",
- "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz",
- "integrity": "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==",
- "license": "MIT",
- "dependencies": {
- "color-name": "~1.1.4"
- },
- "engines": {
- "node": ">=7.0.0"
- }
- },
- "node_modules/color-name": {
- "version": "1.1.4",
- "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz",
- "integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==",
- "license": "MIT"
- },
- "node_modules/data-uri-to-buffer": {
- "version": "6.0.2",
- "resolved": "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-6.0.2.tgz",
- "integrity": "sha512-7hvf7/GW8e86rW0ptuwS3OcBGDjIi6SZva7hCyWC0yYry2cOPmLIjXAUHI6DK2HsnwJd9ifmt57i8eV2n4YNpw==",
- "license": "MIT",
- "engines": {
- "node": ">= 14"
- }
- },
- "node_modules/debug": {
- "version": "4.4.3",
- "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
- "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
- "license": "MIT",
- "dependencies": {
- "ms": "^2.1.3"
- },
- "engines": {
- "node": ">=6.0"
- },
- "peerDependenciesMeta": {
- "supports-color": {
- "optional": true
- }
- }
- },
- "node_modules/degenerator": {
- "version": "5.0.1",
- "resolved": "https://registry.npmjs.org/degenerator/-/degenerator-5.0.1.tgz",
- "integrity": "sha512-TllpMR/t0M5sqCXfj85i4XaAzxmS5tVA16dqvdkMwGmzI+dXLXnw3J+3Vdv7VKw+ThlTMboK6i9rnZ6Nntj5CQ==",
- "license": "MIT",
- "dependencies": {
- "ast-types": "^0.13.4",
- "escodegen": "^2.1.0",
- "esprima": "^4.0.1"
- },
- "engines": {
- "node": ">= 14"
- }
- },
- "node_modules/devtools-protocol": {
- "version": "0.0.1367902",
- "resolved": "https://registry.npmjs.org/devtools-protocol/-/devtools-protocol-0.0.1367902.tgz",
- "integrity": "sha512-XxtPuC3PGakY6PD7dG66/o8KwJ/LkH2/EKe19Dcw58w53dv4/vSQEkn/SzuyhHE2q4zPgCkxQBxus3VV4ql+Pg==",
- "license": "BSD-3-Clause"
- },
- "node_modules/emoji-regex": {
- "version": "8.0.0",
- "resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-8.0.0.tgz",
- "integrity": "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==",
- "license": "MIT"
- },
- "node_modules/end-of-stream": {
- "version": "1.4.5",
- "resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz",
- "integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==",
- "license": "MIT",
- "dependencies": {
- "once": "^1.4.0"
- }
- },
- "node_modules/escalade": {
- "version": "3.2.0",
- "resolved": "https://registry.npmjs.org/escalade/-/escalade-3.2.0.tgz",
- "integrity": "sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA==",
- "license": "MIT",
- "engines": {
- "node": ">=6"
- }
- },
- "node_modules/escodegen": {
- "version": "2.1.0",
- "resolved": "https://registry.npmjs.org/escodegen/-/escodegen-2.1.0.tgz",
- "integrity": "sha512-2NlIDTwUWJN0mRPQOdtQBzbUHvdGY2P1VXSyU83Q3xKxM7WHX2Ql8dKq782Q9TgQUNOLEzEYu9bzLNj1q88I5w==",
- "license": "BSD-2-Clause",
- "dependencies": {
- "esprima": "^4.0.1",
- "estraverse": "^5.2.0",
- "esutils": "^2.0.2"
- },
- "bin": {
- "escodegen": "bin/escodegen.js",
- "esgenerate": "bin/esgenerate.js"
- },
- "engines": {
- "node": ">=6.0"
- },
- "optionalDependencies": {
- "source-map": "~0.6.1"
- }
- },
- "node_modules/esprima": {
- "version": "4.0.1",
- "resolved": "https://registry.npmjs.org/esprima/-/esprima-4.0.1.tgz",
- "integrity": "sha512-eGuFFw7Upda+g4p+QHvnW0RyTX/SVeJBDM/gCtMARO0cLuT2HcEKnTPvhjV6aGeqrCB/sbNop0Kszm0jsaWU4A==",
- "license": "BSD-2-Clause",
- "bin": {
- "esparse": "bin/esparse.js",
- "esvalidate": "bin/esvalidate.js"
- },
- "engines": {
- "node": ">=4"
- }
- },
- "node_modules/estraverse": {
- "version": "5.3.0",
- "resolved": "https://registry.npmjs.org/estraverse/-/estraverse-5.3.0.tgz",
- "integrity": "sha512-MMdARuVEQziNTeJD8DgMqmhwR11BRQ/cBP+pLtYdSTnf3MIO8fFeiINEbX36ZdNlfU/7A9f3gUw49B3oQsvwBA==",
- "license": "BSD-2-Clause",
- "engines": {
- "node": ">=4.0"
- }
- },
- "node_modules/esutils": {
- "version": "2.0.3",
- "resolved": "https://registry.npmjs.org/esutils/-/esutils-2.0.3.tgz",
- "integrity": "sha512-kVscqXk4OCp68SZ0dkgEKVi6/8ij300KBWTJq32P/dYeWTSwK41WyTxalN1eRmA5Z9UU/LX9D7FWSmV9SAYx6g==",
- "license": "BSD-2-Clause",
- "engines": {
- "node": ">=0.10.0"
- }
- },
- "node_modules/events-universal": {
- "version": "1.0.1",
- "resolved": "https://registry.npmjs.org/events-universal/-/events-universal-1.0.1.tgz",
- "integrity": "sha512-LUd5euvbMLpwOF8m6ivPCbhQeSiYVNb8Vs0fQ8QjXo0JTkEHpz8pxdQf0gStltaPpw0Cca8b39KxvK9cfKRiAw==",
- "license": "Apache-2.0",
- "dependencies": {
- "bare-events": "^2.7.0"
- }
- },
- "node_modules/extract-zip": {
- "version": "2.0.1",
- "resolved": "https://registry.npmjs.org/extract-zip/-/extract-zip-2.0.1.tgz",
- "integrity": "sha512-GDhU9ntwuKyGXdZBUgTIe+vXnWj0fppUEtMDL0+idd5Sta8TGpHssn/eusA9mrPr9qNDym6SxAYZjNvCn/9RBg==",
- "license": "BSD-2-Clause",
- "dependencies": {
- "debug": "^4.1.1",
- "get-stream": "^5.1.0",
- "yauzl": "^2.10.0"
- },
- "bin": {
- "extract-zip": "cli.js"
- },
- "engines": {
- "node": ">= 10.17.0"
- },
- "optionalDependencies": {
- "@types/yauzl": "^2.9.1"
- }
- },
- "node_modules/fast-fifo": {
- "version": "1.3.2",
- "resolved": "https://registry.npmjs.org/fast-fifo/-/fast-fifo-1.3.2.tgz",
- "integrity": "sha512-/d9sfos4yxzpwkDkuN7k2SqFKtYNmCTzgfEpz82x34IM9/zc8KGxQoXg1liNC/izpRM/MBdt44Nmx41ZWqk+FQ==",
- "license": "MIT"
- },
- "node_modules/fd-slicer": {
- "version": "1.1.0",
- "resolved": "https://registry.npmjs.org/fd-slicer/-/fd-slicer-1.1.0.tgz",
- "integrity": "sha512-cE1qsB/VwyQozZ+q1dGxR8LBYNZeofhEdUNGSMbQD3Gw2lAzX9Zb3uIU6Ebc/Fmyjo9AWWfnn0AUCHqtevs/8g==",
- "license": "MIT",
- "dependencies": {
- "pend": "~1.2.0"
- }
- },
- "node_modules/get-caller-file": {
- "version": "2.0.5",
- "resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz",
- "integrity": "sha512-DyFP3BM/3YHTQOCUL/w0OZHR0lpKeGrxotcHWcqNEdnltqFwXVfhEBQ94eIo34AfQpo0rGki4cyIiftY06h2Fg==",
- "license": "ISC",
- "engines": {
- "node": "6.* || 8.* || >= 10.*"
- }
- },
- "node_modules/get-stream": {
- "version": "5.2.0",
- "resolved": "https://registry.npmjs.org/get-stream/-/get-stream-5.2.0.tgz",
- "integrity": "sha512-nBF+F1rAZVCu/p7rjzgA+Yb4lfYXrpl7a6VmJrU8wF9I1CKvP/QwPNZHnOlwbTkY6dvtFIzFMSyQXbLoTQPRpA==",
- "license": "MIT",
- "dependencies": {
- "pump": "^3.0.0"
- },
- "engines": {
- "node": ">=8"
- },
- "funding": {
- "url": "https://github.com/sponsors/sindresorhus"
- }
- },
- "node_modules/get-uri": {
- "version": "6.0.5",
- "resolved": "https://registry.npmjs.org/get-uri/-/get-uri-6.0.5.tgz",
- "integrity": "sha512-b1O07XYq8eRuVzBNgJLstU6FYc1tS6wnMtF1I1D9lE8LxZSOGZ7LhxN54yPP6mGw5f2CkXY2BQUL9Fx41qvcIg==",
- "license": "MIT",
- "dependencies": {
- "basic-ftp": "^5.0.2",
- "data-uri-to-buffer": "^6.0.2",
- "debug": "^4.3.4"
- },
- "engines": {
- "node": ">= 14"
- }
- },
- "node_modules/http-proxy-agent": {
- "version": "7.0.2",
- "resolved": "https://registry.npmjs.org/http-proxy-agent/-/http-proxy-agent-7.0.2.tgz",
- "integrity": "sha512-T1gkAiYYDWYx3V5Bmyu7HcfcvL7mUrTWiM6yOfa3PIphViJ/gFPbvidQ+veqSOHci/PxBcDabeUNCzpOODJZig==",
- "license": "MIT",
- "dependencies": {
- "agent-base": "^7.1.0",
- "debug": "^4.3.4"
- },
- "engines": {
- "node": ">= 14"
- }
- },
- "node_modules/https-proxy-agent": {
- "version": "7.0.6",
- "resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-7.0.6.tgz",
- "integrity": "sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw==",
- "license": "MIT",
- "dependencies": {
- "agent-base": "^7.1.2",
- "debug": "4"
- },
- "engines": {
- "node": ">= 14"
- }
- },
- "node_modules/ieee754": {
- "version": "1.2.1",
- "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",
- "integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",
- "funding": [
- {
- "type": "github",
- "url": "https://github.com/sponsors/feross"
- },
- {
- "type": "patreon",
- "url": "https://www.patreon.com/feross"
- },
- {
- "type": "consulting",
- "url": "https://feross.org/support"
- }
- ],
- "license": "BSD-3-Clause"
- },
- "node_modules/ip-address": {
- "version": "10.2.0",
- "resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.2.0.tgz",
- "integrity": "sha512-/+S6j4E9AHvW9SWMSEY9Xfy66O5PWvVEJ08O0y5JGyEKQpojb0K0GKpz/v5HJ/G0vi3D2sjGK78119oXZeE0qA==",
- "license": "MIT",
- "engines": {
- "node": ">= 12"
- }
- },
- "node_modules/is-fullwidth-code-point": {
- "version": "3.0.0",
- "resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz",
- "integrity": "sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg==",
- "license": "MIT",
- "engines": {
- "node": ">=8"
- }
- },
- "node_modules/lru-cache": {
- "version": "7.18.3",
- "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-7.18.3.tgz",
- "integrity": "sha512-jumlc0BIUrS3qJGgIkWZsyfAM7NCWiBcCDhnd+3NNM5KbBmLTgHVfWBcg6W+rLUsIpzpERPsvwUP7CckAQSOoA==",
- "license": "ISC",
- "engines": {
- "node": ">=12"
- }
- },
- "node_modules/mitt": {
- "version": "3.0.1",
- "resolved": "https://registry.npmjs.org/mitt/-/mitt-3.0.1.tgz",
- "integrity": "sha512-vKivATfr97l2/QBCYAkXYDbrIWPM2IIKEl7YPhjCvKlG3kE2gm+uBo6nEXK3M5/Ffh/FLpKExzOQ3JJoJGFKBw==",
- "license": "MIT"
- },
- "node_modules/ms": {
- "version": "2.1.3",
- "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
- "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
- "license": "MIT"
- },
- "node_modules/netmask": {
- "version": "2.1.1",
- "resolved": "https://registry.npmjs.org/netmask/-/netmask-2.1.1.tgz",
- "integrity": "sha512-eonl3sLUha+S1GzTPxychyhnUzKyeQkZ7jLjKrBagJgPla13F+uQ71HgpFefyHgqrjEbCPkDArxYsjY8/+gLKA==",
- "license": "MIT",
- "engines": {
- "node": ">= 0.4.0"
- }
- },
- "node_modules/once": {
- "version": "1.4.0",
- "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
- "integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",
- "license": "ISC",
- "dependencies": {
- "wrappy": "1"
- }
- },
- "node_modules/pac-proxy-agent": {
- "version": "7.2.0",
- "resolved": "https://registry.npmjs.org/pac-proxy-agent/-/pac-proxy-agent-7.2.0.tgz",
- "integrity": "sha512-TEB8ESquiLMc0lV8vcd5Ql/JAKAoyzHFXaStwjkzpOpC5Yv+pIzLfHvjTSdf3vpa2bMiUQrg9i6276yn8666aA==",
- "license": "MIT",
- "dependencies": {
- "@tootallnate/quickjs-emscripten": "^0.23.0",
- "agent-base": "^7.1.2",
- "debug": "^4.3.4",
- "get-uri": "^6.0.1",
- "http-proxy-agent": "^7.0.0",
- "https-proxy-agent": "^7.0.6",
- "pac-resolver": "^7.0.1",
- "socks-proxy-agent": "^8.0.5"
- },
- "engines": {
- "node": ">= 14"
- }
- },
- "node_modules/pac-resolver": {
- "version": "7.0.1",
- "resolved": "https://registry.npmjs.org/pac-resolver/-/pac-resolver-7.0.1.tgz",
- "integrity": "sha512-5NPgf87AT2STgwa2ntRMr45jTKrYBGkVU36yT0ig/n/GMAa3oPqhZfIQ2kMEimReg0+t9kZViDVZ83qfVUlckg==",
- "license": "MIT",
- "dependencies": {
- "degenerator": "^5.0.0",
- "netmask": "^2.0.2"
- },
- "engines": {
- "node": ">= 14"
- }
- },
- "node_modules/pend": {
- "version": "1.2.0",
- "resolved": "https://registry.npmjs.org/pend/-/pend-1.2.0.tgz",
- "integrity": "sha512-F3asv42UuXchdzt+xXqfW1OGlVBe+mxa2mqI0pg5yAHZPvFmY3Y6drSf/GQ1A86WgWEN9Kzh/WrgKa6iGcHXLg==",
- "license": "MIT"
- },
- "node_modules/playwright-core": {
- "version": "1.46.0",
- "resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.46.0.tgz",
- "integrity": "sha512-9Y/d5UIwuJk8t3+lhmMSAJyNP1BUC/DqP3cQJDQQL/oWqAiuPTLgy7Q5dzglmTLwcBRdetzgNM/gni7ckfTr6A==",
- "license": "Apache-2.0",
- "bin": {
- "playwright-core": "cli.js"
- },
- "engines": {
- "node": ">=18"
- }
- },
- "node_modules/progress": {
- "version": "2.0.3",
- "resolved": "https://registry.npmjs.org/progress/-/progress-2.0.3.tgz",
- "integrity": "sha512-7PiHtLll5LdnKIMw100I+8xJXR5gW2QwWYkT6iJva0bXitZKa/XMrSbdmg3r2Xnaidz9Qumd0VPaMrZlF9V9sA==",
- "license": "MIT",
- "engines": {
- "node": ">=0.4.0"
- }
- },
- "node_modules/proxy-agent": {
- "version": "6.5.0",
- "resolved": "https://registry.npmjs.org/proxy-agent/-/proxy-agent-6.5.0.tgz",
- "integrity": "sha512-TmatMXdr2KlRiA2CyDu8GqR8EjahTG3aY3nXjdzFyoZbmB8hrBsTyMezhULIXKnC0jpfjlmiZ3+EaCzoInSu/A==",
- "license": "MIT",
- "dependencies": {
- "agent-base": "^7.1.2",
- "debug": "^4.3.4",
- "http-proxy-agent": "^7.0.1",
- "https-proxy-agent": "^7.0.6",
- "lru-cache": "^7.14.1",
- "pac-proxy-agent": "^7.1.0",
- "proxy-from-env": "^1.1.0",
- "socks-proxy-agent": "^8.0.5"
- },
- "engines": {
- "node": ">= 14"
- }
- },
- "node_modules/proxy-from-env": {
- "version": "1.1.0",
- "resolved": "https://registry.npmjs.org/proxy-from-env/-/proxy-from-env-1.1.0.tgz",
- "integrity": "sha512-D+zkORCbA9f1tdWRK0RaCR3GPv50cMxcrz4X8k5LTSUD1Dkw47mKJEZQNunItRTkWwgtaUSo1RVFRIG9ZXiFYg==",
- "license": "MIT"
- },
- "node_modules/pump": {
- "version": "3.0.4",
- "resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz",
- "integrity": "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==",
- "license": "MIT",
- "dependencies": {
- "end-of-stream": "^1.1.0",
- "once": "^1.3.1"
- }
- },
- "node_modules/puppeteer-core": {
- "version": "23.11.1",
- "resolved": "https://registry.npmjs.org/puppeteer-core/-/puppeteer-core-23.11.1.tgz",
- "integrity": "sha512-3HZ2/7hdDKZvZQ7dhhITOUg4/wOrDRjyK2ZBllRB0ZCOi9u0cwq1ACHDjBB+nX+7+kltHjQvBRdeY7+W0T+7Gg==",
- "license": "Apache-2.0",
- "dependencies": {
- "@puppeteer/browsers": "2.6.1",
- "chromium-bidi": "0.11.0",
- "debug": "^4.4.0",
- "devtools-protocol": "0.0.1367902",
- "typed-query-selector": "^2.12.0",
- "ws": "^8.18.0"
- },
- "engines": {
- "node": ">=18"
- }
- },
- "node_modules/require-directory": {
- "version": "2.1.1",
- "resolved": "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz",
- "integrity": "sha512-fGxEI7+wsG9xrvdjsrlmL22OMTTiHRwAMroiEeMgq8gzoLC/PQr7RsRDSTLUg/bZAZtF+TVIkHc6/4RIKrui+Q==",
- "license": "MIT",
- "engines": {
- "node": ">=0.10.0"
- }
- },
- "node_modules/semver": {
- "version": "7.8.5",
- "resolved": "https://registry.npmjs.org/semver/-/semver-7.8.5.tgz",
- "integrity": "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==",
- "license": "ISC",
- "bin": {
- "semver": "bin/semver.js"
- },
- "engines": {
- "node": ">=10"
- }
- },
- "node_modules/smart-buffer": {
- "version": "4.2.0",
- "resolved": "https://registry.npmjs.org/smart-buffer/-/smart-buffer-4.2.0.tgz",
- "integrity": "sha512-94hK0Hh8rPqQl2xXc3HsaBoOXKV20MToPkcXvwbISWLEs+64sBq5kFgn2kJDHb1Pry9yrP0dxrCI9RRci7RXKg==",
- "license": "MIT",
- "engines": {
- "node": ">= 6.0.0",
- "npm": ">= 3.0.0"
- }
- },
- "node_modules/socks": {
- "version": "2.8.9",
- "resolved": "https://registry.npmjs.org/socks/-/socks-2.8.9.tgz",
- "integrity": "sha512-LJhUYUvItdQ0LkJTmPeaEObWXAqFyfmP85x0tch/ez9cahmhlBBLbIqDFnvBnUJGagb0JbIQrkBs1wJ+yRYpEw==",
- "license": "MIT",
- "dependencies": {
- "ip-address": "^10.1.1",
- "smart-buffer": "^4.2.0"
- },
- "engines": {
- "node": ">= 10.0.0",
- "npm": ">= 3.0.0"
- }
- },
- "node_modules/socks-proxy-agent": {
- "version": "8.0.5",
- "resolved": "https://registry.npmjs.org/socks-proxy-agent/-/socks-proxy-agent-8.0.5.tgz",
- "integrity": "sha512-HehCEsotFqbPW9sJ8WVYB6UbmIMv7kUUORIF2Nncq4VQvBfNBLibW9YZR5dlYCSUhwcD628pRllm7n+E+YTzJw==",
- "license": "MIT",
- "dependencies": {
- "agent-base": "^7.1.2",
- "debug": "^4.3.4",
- "socks": "^2.8.3"
- },
- "engines": {
- "node": ">= 14"
- }
- },
- "node_modules/source-map": {
- "version": "0.6.1",
- "resolved": "https://registry.npmjs.org/source-map/-/source-map-0.6.1.tgz",
- "integrity": "sha512-UjgapumWlbMhkBgzT7Ykc5YXUT46F0iKu8SGXq0bcwP5dz/h0Plj6enJqjz1Zbq2l5WaqYnrVbwWOWMyF3F47g==",
- "license": "BSD-3-Clause",
- "optional": true,
- "engines": {
- "node": ">=0.10.0"
- }
- },
- "node_modules/streamx": {
- "version": "2.28.0",
- "resolved": "https://registry.npmjs.org/streamx/-/streamx-2.28.0.tgz",
- "integrity": "sha512-1Yowhzjf0ivGMrTIkY9hav5TxobO9qIVqUE41fiCGMGgc3CLlf4MY+9AHmZqBWgDTue0fY9zWjYFVyf6Diuobw==",
- "license": "MIT",
- "dependencies": {
- "events-universal": "^1.0.0",
- "fast-fifo": "^1.3.2",
- "text-decoder": "^1.1.0"
- }
- },
- "node_modules/string-width": {
- "version": "4.2.3",
- "resolved": "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz",
- "integrity": "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==",
- "license": "MIT",
- "dependencies": {
- "emoji-regex": "^8.0.0",
- "is-fullwidth-code-point": "^3.0.0",
- "strip-ansi": "^6.0.1"
- },
- "engines": {
- "node": ">=8"
- }
- },
- "node_modules/strip-ansi": {
- "version": "6.0.1",
- "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz",
- "integrity": "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==",
- "license": "MIT",
- "dependencies": {
- "ansi-regex": "^5.0.1"
- },
- "engines": {
- "node": ">=8"
- }
- },
- "node_modules/tar-fs": {
- "version": "3.1.3",
- "resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-3.1.3.tgz",
- "integrity": "sha512-/hU4AXnIdZu+Gvl1pk0oI5f5HxWsCJRtY2aFaJdk9VvyL48DWU6iU5WAIPG+wIi1YvWA6eTJvIviP/tMAZZNwQ==",
- "license": "MIT",
- "dependencies": {
- "pump": "^3.0.0",
- "tar-stream": "^3.1.5"
- },
- "optionalDependencies": {
- "bare-fs": "^4.0.1",
- "bare-path": "^3.0.0"
- }
- },
- "node_modules/tar-stream": {
- "version": "3.2.0",
- "resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-3.2.0.tgz",
- "integrity": "sha512-ojzvCvVaNp6aOTFmG7jaRD0meowIAuPc3cMMhSgKiVWws1GyHbGd/xvnyuRKcKlMpt3qvxx6r0hreCNITP9hIg==",
- "license": "MIT",
- "dependencies": {
- "b4a": "^1.6.4",
- "bare-fs": "^4.5.5",
- "fast-fifo": "^1.2.0",
- "streamx": "^2.15.0"
- }
- },
- "node_modules/teex": {
- "version": "1.0.1",
- "resolved": "https://registry.npmjs.org/teex/-/teex-1.0.1.tgz",
- "integrity": "sha512-eYE6iEI62Ni1H8oIa7KlDU6uQBtqr4Eajni3wX7rpfXD8ysFx8z0+dri+KWEPWpBsxXfxu58x/0jvTVT1ekOSg==",
- "license": "MIT",
- "dependencies": {
- "streamx": "^2.12.5"
- }
- },
- "node_modules/text-decoder": {
- "version": "1.2.7",
- "resolved": "https://registry.npmjs.org/text-decoder/-/text-decoder-1.2.7.tgz",
- "integrity": "sha512-vlLytXkeP4xvEq2otHeJfSQIRyWxo/oZGEbXrtEEF9Hnmrdly59sUbzZ/QgyWuLYHctCHxFF4tRQZNQ9k60ExQ==",
- "license": "Apache-2.0",
- "dependencies": {
- "b4a": "^1.6.4"
- }
- },
- "node_modules/through": {
- "version": "2.3.8",
- "resolved": "https://registry.npmjs.org/through/-/through-2.3.8.tgz",
- "integrity": "sha512-w89qg7PI8wAdvX60bMDP+bFoD5Dvhm9oLheFp5O4a2QF0cSBGsBX4qZmadPMvVqlLJBBci+WqGGOAPvcDeNSVg==",
- "license": "MIT"
- },
- "node_modules/tslib": {
- "version": "2.8.1",
- "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
- "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
- "license": "0BSD"
- },
- "node_modules/typed-query-selector": {
- "version": "2.12.2",
- "resolved": "https://registry.npmjs.org/typed-query-selector/-/typed-query-selector-2.12.2.tgz",
- "integrity": "sha512-EOPFbyIub4ngnEdqi2yOcNeDLaX/0jcE1JoAXQDDMIthap7FoN795lc/SHfIq2d416VufXpM8z/lD+WRm2gfOQ==",
- "license": "MIT"
- },
- "node_modules/unbzip2-stream": {
- "version": "1.4.3",
- "resolved": "https://registry.npmjs.org/unbzip2-stream/-/unbzip2-stream-1.4.3.tgz",
- "integrity": "sha512-mlExGW4w71ebDJviH16lQLtZS32VKqsSfk80GCfUlwT/4/hNRFsoscrF/c++9xinkMzECL1uL9DDwXqFWkruPg==",
- "license": "MIT",
- "dependencies": {
- "buffer": "^5.2.1",
- "through": "^2.3.8"
- }
- },
- "node_modules/undici-types": {
- "version": "8.3.0",
- "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-8.3.0.tgz",
- "integrity": "sha512-j375ScV60dom+YkPFIfTLcOiPxkN/buHz5GobjLhixFuANaNs3C9l4GmrWqejgXWJ7BbJcFYpTEUkS1Ge8bpZQ==",
- "license": "MIT",
- "optional": true
- },
- "node_modules/wrap-ansi": {
- "version": "7.0.0",
- "resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-7.0.0.tgz",
- "integrity": "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==",
- "license": "MIT",
- "dependencies": {
- "ansi-styles": "^4.0.0",
- "string-width": "^4.1.0",
- "strip-ansi": "^6.0.0"
- },
- "engines": {
- "node": ">=10"
- },
- "funding": {
- "url": "https://github.com/chalk/wrap-ansi?sponsor=1"
- }
- },
- "node_modules/wrappy": {
- "version": "1.0.2",
- "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
- "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
- "license": "ISC"
- },
- "node_modules/ws": {
- "version": "8.21.1",
- "resolved": "https://registry.npmjs.org/ws/-/ws-8.21.1.tgz",
- "integrity": "sha512-+0NTnW77fFN/DjQi6k/Sq/Yvk4Sgajw7urW8V+asjXnRgDs9gyGkdb7EzgfhA4goXsRIZKE28fzIXBHEzhuiWw==",
- "license": "MIT",
- "engines": {
- "node": ">=10.0.0"
- },
- "peerDependencies": {
- "bufferutil": "^4.0.1",
- "utf-8-validate": ">=5.0.2"
- },
- "peerDependenciesMeta": {
- "bufferutil": {
- "optional": true
- },
- "utf-8-validate": {
- "optional": true
- }
- }
- },
- "node_modules/y18n": {
- "version": "5.0.8",
- "resolved": "https://registry.npmjs.org/y18n/-/y18n-5.0.8.tgz",
- "integrity": "sha512-0pfFzegeDWJHJIAmTLRP2DwHjdF5s7jo9tuztdQxAhINCdvS+3nGINqPd00AphqJR/0LhANUS6/+7SCb98YOfA==",
- "license": "ISC",
- "engines": {
- "node": ">=10"
- }
- },
- "node_modules/yargs": {
- "version": "17.7.3",
- "resolved": "https://registry.npmjs.org/yargs/-/yargs-17.7.3.tgz",
- "integrity": "sha512-GZtjxm/J/4TSxuL3FNYjCmLktBTnIw/rVmKSIyKeYAZpmJB2ig9VauCC5xsa82GNKVKDAqpOn3KVzNt0zmrU0g==",
- "license": "MIT",
- "dependencies": {
- "cliui": "^8.0.1",
- "escalade": "^3.1.1",
- "get-caller-file": "^2.0.5",
- "require-directory": "^2.1.1",
- "string-width": "^4.2.3",
- "y18n": "^5.0.5",
- "yargs-parser": "^21.1.1"
- },
- "engines": {
- "node": ">=12"
- }
- },
- "node_modules/yargs-parser": {
- "version": "21.1.1",
- "resolved": "https://registry.npmjs.org/yargs-parser/-/yargs-parser-21.1.1.tgz",
- "integrity": "sha512-tVpsJW7DdjecAiFpbIB1e3qxIQsE6NoPc5/eTdrbbIC4h0LVsWhnoa3g+m2HclBIujHzsxZ4VJVA+GUuc2/LBw==",
- "license": "ISC",
- "engines": {
- "node": ">=12"
- }
- },
- "node_modules/yauzl": {
- "version": "2.10.0",
- "resolved": "https://registry.npmjs.org/yauzl/-/yauzl-2.10.0.tgz",
- "integrity": "sha512-p4a9I6X6nu6IhoGmBqAcbJy1mlC4j27vEPZX9F4L4/vZT3Lyq1VkFHw/V/PUcB9Buo+DG3iHkT0x3Qya58zc3g==",
- "license": "MIT",
- "dependencies": {
- "buffer-crc32": "~0.2.3",
- "fd-slicer": "~1.1.0"
- }
- },
- "node_modules/zod": {
- "version": "3.23.8",
- "resolved": "https://registry.npmjs.org/zod/-/zod-3.23.8.tgz",
- "integrity": "sha512-XBx9AXhXktjUqnepgTiE5flcKIYWi/rme0Eaj+5Y0lftuGBq+jyRu/md4WnuxqgP1ubdpNCsYEYPxrzVHD8d6g==",
- "license": "MIT",
- "funding": {
- "url": "https://github.com/sponsors/colinhacks"
- }
- }
- }
-}
diff --git a/tests/unit/proxy/fixtures/js_clients/package.json b/tests/unit/proxy/fixtures/js_clients/package.json
deleted file mode 100644
index 6ccd4de4e..000000000
--- a/tests/unit/proxy/fixtures/js_clients/package.json
+++ /dev/null
@@ -1,16 +0,0 @@
-{
- "name": "cdp-proxy-js-compat-clients",
- "private": true,
- "type": "module",
- "description": "Pinned JS CDP clients driven against the proxy by tests/unit/proxy/test_client_compat.py.",
- "comment": [
- "Exact pins, no ranges: a CI gate that silently follows a client's latest release",
- "is not reproducible. playwright-core matches the repo's pinned playwright.",
- "Both packages are the -core builds and connect to an already-running browser, so",
- "npm ci downloads no Chromium at all — the browser comes from the upstream adapter."
- ],
- "dependencies": {
- "playwright-core": "1.46.0",
- "puppeteer-core": "23.11.1"
- }
-}
diff --git a/tests/unit/proxy/fixtures/js_clients/playwright_client.mjs b/tests/unit/proxy/fixtures/js_clients/playwright_client.mjs
deleted file mode 100644
index 398a3bac0..000000000
--- a/tests/unit/proxy/fixtures/js_clients/playwright_client.mjs
+++ /dev/null
@@ -1,16 +0,0 @@
-// Playwright for JS, driving a page through the proxy. Prints one JSON line for the
-// test to assert on. Hermetic: the page is a data: URL, so no network and no site.
-import { chromium } from "playwright-core";
-
-const endpoint = process.argv[2];
-const browser = await chromium.connectOverCDP(endpoint);
-try {
- const context = browser.contexts()[0] ?? (await browser.newContext());
- const page = await context.newPage();
- await page.goto("data:text/html,<title>proxied</title><h1>hello</h1>");
- const title = await page.title();
- const heading = await page.textContent("h1");
- console.log(JSON.stringify({ client: "playwright-js", title, heading }));
-} finally {
- await browser.close();
-}
diff --git a/tests/unit/proxy/fixtures/js_clients/puppeteer_client.mjs b/tests/unit/proxy/fixtures/js_clients/puppeteer_client.mjs
deleted file mode 100644
index 21a772cd9..000000000
--- a/tests/unit/proxy/fixtures/js_clients/puppeteer_client.mjs
+++ /dev/null
@@ -1,23 +0,0 @@
-// Puppeteer, driving a page through the proxy. Prints one JSON line to assert on.
-//
-// A ws:// endpoint connects via browserWSEndpoint, an http:// one via browserURL —
-// the test drives both, because only the first can address a path-scoped session:
-// browserURL resolves discovery as `new URL('/json/version', browserURL)`
-// (BrowserConnector.js), an ABSOLUTE path that discards the session prefix.
-import puppeteer from "puppeteer-core";
-
-const endpoint = process.argv[2];
-const connectOptions = endpoint.startsWith("ws")
- ? { browserWSEndpoint: endpoint }
- : { browserURL: endpoint };
-
-const browser = await puppeteer.connect(connectOptions);
-try {
- const page = await browser.newPage();
- await page.goto("data:text/html,<title>proxied</title><h1>hello</h1>");
- const title = await page.title();
- const heading = await page.$eval("h1", (node) => node.textContent);
- console.log(JSON.stringify({ client: "puppeteer", title, heading }));
-} finally {
- await browser.disconnect();
-}
diff --git a/tests/unit/proxy/fixtures/record_workload.py b/tests/unit/proxy/fixtures/record_workload.py
deleted file mode 100644
index 3aeeeafb9..000000000
--- a/tests/unit/proxy/fixtures/record_workload.py
+++ /dev/null
@@ -1,164 +0,0 @@
-"""Regenerate the workload_page_load_*.jsonl fixtures from a real Chrome.
-
- CHROME=/path/to/chrome python tests/unit/proxy/fixtures/record_workload.py out.jsonl
- THROTTLE=1 CHROME=... python .../record_workload.py out_broadband.jsonl
-
-Not a test (pytest collects test_*.py only) and never run in CI: it needs a local
-Chrome. It exists so the recorded fixtures are reproducible evidence rather than a
-data blob someone has to take on faith — the policy pack's whole claim is measured
-against them, so how they were produced has to be checkable.
-
-Drives Chrome over RAW websockets the way a client library does (flat auto-attach,
-then Runtime/Network/Page/Log enable per page session) and records every upstream
-frame verbatim. The page is served from a throwaway local HTTP server: no external
-host, so the fixture carries no third-party address.
-
-THROTTLE=1 emulates ordinary broadband. This matters more than it looks: over
-loopback Chrome takes each response body in a handful of huge reads, so
-Network.dataReceived fires ~14x less than it does over a real network. Both
-recordings are kept — the throttled one is what a runner crossing a real network
-looks like, the loopback one is the floor.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import json
-import os
-import shutil
-import subprocess
-import sys
-import tempfile
-import threading
-import urllib.request
-from functools import partial
-from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
-from pathlib import Path
-from typing import Any
-
-import websockets
-
-SETTLE_SECONDS = 14.0
-
-# The recorded workload's definition, importable so a replay drives the same
-# stream: any drift between these constants and a replayer silently changes
-# what "the same workload" means.
-BROADBAND_CONDITIONS = {
- "offline": False,
- "latency": 40,
- "downloadThroughput": 5_000_000 // 8,
- "uploadThroughput": 1_000_000 // 8,
-}
-AUTO_ATTACH_PARAMS = {"autoAttach": True, "waitForDebuggerOnStart": False, "flatten": True}
-WORKLOAD_DOMAINS = ("Runtime.enable", "Network.enable", "Page.enable", "Log.enable")
-
-
-def build_site(root: Path) -> None:
- """A page shaped like a real one: stylesheet, script, images, an xhr, console."""
- (root / "app.js").write_text("console.log('boot');\n" + "// pad\n" * 8000)
- (root / "styles.css").write_text("body{margin:0}\n" + "/* pad */\n" * 6000)
- for i in range(4):
- # Incompressible, so the transfer size is real rather than gzipped away.
- (root / f"img{i}.bin").write_bytes(os.urandom(220_000))
- (root / "data.json").write_text(json.dumps({"rows": [{"i": i, "v": "x" * 200} for i in range(900)]}))
- (root / "index.html").write_text(
- """<!doctype html><html><head>
-<link rel="stylesheet" href="styles.css">
-</head><body>
-<h1>local test page</h1>
-<img src="img0.bin"><img src="img1.bin"><img src="img2.bin"><img src="img3.bin">
-<script src="app.js"></script>
-<script>
-for (let i = 0; i < 5; i++) console.log('render pass', i);
-fetch('data.json').then(r => r.json()).then(d => console.log('rows', d.rows.length));
-</script>
-</body></html>"""
- )
-
-
-async def main() -> None:
- chrome = os.environ.get("CHROME")
- if not chrome:
- raise SystemExit("set CHROME to a chrome/chromium binary")
- out = Path(sys.argv[1])
-
- root = Path(tempfile.mkdtemp(prefix="site-"))
- build_site(root)
- httpd = ThreadingHTTPServer(("127.0.0.1", 0), partial(SimpleHTTPRequestHandler, directory=str(root)))
- threading.Thread(target=httpd.serve_forever, daemon=True).start()
- page_url = f"http://127.0.0.1:{httpd.server_address[1]}/index.html"
-
- profile = tempfile.mkdtemp(prefix="prof-")
- proc = subprocess.Popen(
- [chrome, "--remote-debugging-port=0", f"--user-data-dir={profile}", "--no-sandbox", "about:blank"],
- stderr=subprocess.DEVNULL,
- stdout=subprocess.DEVNULL,
- )
- port_file = Path(profile) / "DevToolsActivePort"
- for _ in range(100):
- if port_file.exists() and port_file.read_text().splitlines():
- break
- await asyncio.sleep(0.1)
- port = port_file.read_text().splitlines()[0]
- ws_url = json.loads(urllib.request.urlopen(f"http://127.0.0.1:{port}/json/version").read())["webSocketDebuggerUrl"]
-
- frames: list[str] = []
- next_id = iter(range(1, 10_000))
- recording = True
-
- async with websockets.connect(ws_url, max_size=None) as ws:
-
- async def reader() -> None:
- while recording:
- try:
- raw = await asyncio.wait_for(ws.recv(), timeout=1.0)
- except (asyncio.TimeoutError, websockets.ConnectionClosed):
- continue
- frames.append(raw if isinstance(raw, str) else raw.decode())
-
- task = asyncio.create_task(reader())
-
- async def send(method: str, params: dict[str, Any] | None = None, session: str | None = None) -> None:
- msg: dict[str, Any] = {"id": next(next_id), "method": method}
- if params:
- msg["params"] = params
- if session:
- msg["sessionId"] = session
- await ws.send(json.dumps(msg))
-
- # Chrome was launched on about:blank, so flat auto-attach picks up the existing
- # page target (Chrome rejects GET on /json/new).
- await send("Target.setAutoAttach", AUTO_ATTACH_PARAMS)
- await send("Target.setDiscoverTargets", {"discover": True})
- await asyncio.sleep(2.0)
-
- session_id = None
- for raw in frames:
- frame = json.loads(raw)
- if frame.get("method") == "Target.attachedToTarget":
- if frame["params"]["targetInfo"].get("type") == "page":
- session_id = frame["params"]["sessionId"]
- if not session_id:
- raise SystemExit("no page session attached")
-
- for method in WORKLOAD_DOMAINS:
- await send(method, None, session_id)
- if os.environ.get("THROTTLE"):
- await send("Network.emulateNetworkConditions", BROADBAND_CONDITIONS, session_id)
- await asyncio.sleep(0.3)
- await send("Page.navigate", {"url": page_url}, session_id)
- await asyncio.sleep(SETTLE_SECONDS)
-
- recording = False
- await task
-
- proc.terminate()
- httpd.shutdown()
- shutil.rmtree(root, ignore_errors=True)
- shutil.rmtree(profile, ignore_errors=True)
- out.write_text("\n".join(frames) + "\n")
- print(f"recorded {len(frames)} frames -> {out}")
-
-
-if __name__ == "__main__":
- asyncio.run(main())
diff --git a/tests/unit/proxy/fixtures/workload_page_load_broadband.jsonl b/tests/unit/proxy/fixtures/workload_page_load_broadband.jsonl
deleted file mode 100644
index 15aa58f03..000000000
--- a/tests/unit/proxy/fixtures/workload_page_load_broadband.jsonl
+++ /dev/null
@@ -1,259 +0,0 @@
-{"method":"Target.attachedToTarget","params":{"sessionId":"194DB96D0F674F1BD4D295AD56547FE3","targetInfo":{"targetId":"C8D09C5863610CBA19F8F06A912FDC75","type":"page","title":"about:blank","url":"about:blank","attached":true,"canAccessOpener":false,"browserContextId":"79B22624E78EE021FA2609DF916CB5C6"},"waitingForDebugger":false}}
-{"id":1,"result":{}}
-{"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"C8D09C5863610CBA19F8F06A912FDC75","type":"page","title":"about:blank","url":"about:blank","attached":true,"canAccessOpener":false,"browserContextId":"79B22624E78EE021FA2609DF916CB5C6"}}}
-{"id":2,"result":{}}
-{"method":"Runtime.executionContextCreated","params":{"context":{"id":1,"origin":"://","name":"","uniqueId":"-5931964147432654505.-4638959688055148640","auxData":{"isDefault":true,"type":"default","frameId":"C8D09C5863610CBA19F8F06A912FDC75"}}},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"id":3,"result":{},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"id":4,"result":{},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"id":5,"result":{},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"id":6,"result":{},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"id":7,"result":{},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Page.frameStartedNavigating","params":{"frameId":"C8D09C5863610CBA19F8F06A912FDC75","url":"http://127.0.0.1:62700/index.html","loaderId":"E9D3CB25E796E8C762542F1098345B73","navigationType":"differentDocument"},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Page.frameStartedLoading","params":{"frameId":"C8D09C5863610CBA19F8F06A912FDC75"},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"E9D3CB25E796E8C762542F1098345B73","loaderId":"E9D3CB25E796E8C762542F1098345B73","documentURL":"http://127.0.0.1:62700/index.html","request":{"url":"http://127.0.0.1:62700/index.html","method":"GET","headers":{"Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"mixedContentType":"none","initialPriority":"VeryHigh","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191138.903347,"wallTime":1784244177.913465,"initiator":{"type":"other"},"redirectHasExtraInfo":false,"type":"Document","frameId":"C8D09C5863610CBA19F8F06A912FDC75","hasUserGesture":false},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"E9D3CB25E796E8C762542F1098345B73","associatedCookies":[],"headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62700","Sec-Fetch-Dest":"document","Sec-Fetch-Mode":"navigate","Sec-Fetch-Site":"none","Sec-Fetch-User":"?1","Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191138.904663},"siteHasCookieInOtherPartition":false},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"E9D3CB25E796E8C762542F1098345B73","blockedCookies":[],"headers":{"Content-Length":"399","Content-type":"text/html","Date":"Thu, 16 Jul 2026 23:22:57 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:57 GMT\r\nContent-type: text/html\r\nContent-Length: 399\r\nLast-Modified: Thu, 16 Jul 2026 23:22:55 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceived","params":{"requestId":"E9D3CB25E796E8C762542F1098345B73","loaderId":"E9D3CB25E796E8C762542F1098345B73","timestamp":191139.100952,"type":"Document","response":{"url":"http://127.0.0.1:62700/index.html","status":200,"statusText":"OK","headers":{"Content-Length":"399","Content-type":"text/html","Date":"Thu, 16 Jul 2026 23:22:57 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"text/html","charset":"","connectionReused":false,"connectionId":21,"remoteIPAddress":"127.0.0.1","remotePort":62700,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":187,"timing":{"requestTime":191138.904663,"proxyStart":-1,"proxyEnd":-1,"dnsStart":7.527,"dnsEnd":7.882,"connectStart":7.882,"connectEnd":8.182,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":8.341,"sendEnd":8.409,"pushStart":0,"pushEnd":0,"receiveHeadersStart":11.472,"receiveHeadersEnd":195.514},"responseTime":1.784244177926231e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"C8D09C5863610CBA19F8F06A912FDC75"},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"id":8,"result":{"frameId":"C8D09C5863610CBA19F8F06A912FDC75","loaderId":"E9D3CB25E796E8C762542F1098345B73","isDownload":false},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Runtime.executionContextsCleared","params":{},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Runtime.executionContextsCleared","params":{},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Page.frameNavigated","params":{"frame":{"id":"C8D09C5863610CBA19F8F06A912FDC75","loaderId":"E9D3CB25E796E8C762542F1098345B73","url":"http://127.0.0.1:62700/index.html","domainAndRegistry":"","securityOrigin":"http://127.0.0.1:62700","securityOriginDetails":{"isLocalhost":true},"mimeType":"text/html","adFrameStatus":{"adFrameType":"none"},"secureContextType":"SecureLocalhost","crossOriginIsolatedContextType":"NotIsolated","gatedAPIFeatures":[]},"type":"Navigation"},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.policyUpdated","params":{},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"C8D09C5863610CBA19F8F06A912FDC75","type":"page","title":"127.0.0.1:62700/index.html","url":"http://127.0.0.1:62700/index.html","attached":true,"canAccessOpener":false,"browserContextId":"79B22624E78EE021FA2609DF916CB5C6"}}}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"75491.2","associatedCookies":[],"headers":{"Accept":"text/css,*/*;q=0.1","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62700","Referer":"http://127.0.0.1:62700/index.html","Sec-Fetch-Dest":"style","Sec-Fetch-Mode":"no-cors","Sec-Fetch-Site":"same-origin","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191139.106187},"clientSecurityState":{"initiatorIsSecureContext":true,"initiatorIPAddressSpace":"Loopback","localNetworkAccessRequestPolicy":"PermissionBlock"},"siteHasCookieInOtherPartition":false},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"75491.3","associatedCookies":[],"headers":{"Accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62700","Referer":"http://127.0.0.1:62700/index.html","Sec-Fetch-Dest":"image","Sec-Fetch-Mode":"no-cors","Sec-Fetch-Site":"same-origin","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191139.106682},"clientSecurityState":{"initiatorIsSecureContext":true,"initiatorIPAddressSpace":"Loopback","localNetworkAccessRequestPolicy":"PermissionBlock"},"siteHasCookieInOtherPartition":false},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"75491.4","associatedCookies":[],"headers":{"Accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62700","Referer":"http://127.0.0.1:62700/index.html","Sec-Fetch-Dest":"image","Sec-Fetch-Mode":"no-cors","Sec-Fetch-Site":"same-origin","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191139.107037},"clientSecurityState":{"initiatorIsSecureContext":true,"initiatorIPAddressSpace":"Loopback","localNetworkAccessRequestPolicy":"PermissionBlock"},"siteHasCookieInOtherPartition":false},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"E9D3CB25E796E8C762542F1098345B73","timestamp":191139.104536,"dataLength":399,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"75491.2","loaderId":"E9D3CB25E796E8C762542F1098345B73","documentURL":"http://127.0.0.1:62700/index.html","request":{"url":"http://127.0.0.1:62700/styles.css","method":"GET","headers":{"sec-ch-ua-platform":"\"macOS\"","Referer":"http://127.0.0.1:62700/index.html","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0"},"mixedContentType":"none","initialPriority":"VeryHigh","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191139.105296,"wallTime":1784244178.115434,"initiator":{"type":"parser","url":"http://127.0.0.1:62700/index.html","lineNumber":1,"columnNumber":41},"redirectHasExtraInfo":false,"type":"Stylesheet","frameId":"C8D09C5863610CBA19F8F06A912FDC75","hasUserGesture":false,"renderBlockingBehavior":"Blocking"},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"75491.3","loaderId":"E9D3CB25E796E8C762542F1098345B73","documentURL":"http://127.0.0.1:62700/index.html","request":{"url":"http://127.0.0.1:62700/img0.bin","method":"GET","headers":{"sec-ch-ua-platform":"\"macOS\"","Referer":"http://127.0.0.1:62700/index.html","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0"},"mixedContentType":"none","initialPriority":"Medium","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191139.105534,"wallTime":1784244178.115643,"initiator":{"type":"parser","url":"http://127.0.0.1:62700/index.html","lineNumber":4,"columnNumber":20},"redirectHasExtraInfo":false,"type":"Image","frameId":"C8D09C5863610CBA19F8F06A912FDC75","hasUserGesture":false},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"75491.4","loaderId":"E9D3CB25E796E8C762542F1098345B73","documentURL":"http://127.0.0.1:62700/index.html","request":{"url":"http://127.0.0.1:62700/img1.bin","method":"GET","headers":{"sec-ch-ua-platform":"\"macOS\"","Referer":"http://127.0.0.1:62700/index.html","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0"},"mixedContentType":"none","initialPriority":"Medium","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191139.105604,"wallTime":1784244178.115718,"initiator":{"type":"parser","url":"http://127.0.0.1:62700/index.html","lineNumber":4,"columnNumber":40},"redirectHasExtraInfo":false,"type":"Image","frameId":"C8D09C5863610CBA19F8F06A912FDC75","hasUserGesture":false},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"75491.5","loaderId":"E9D3CB25E796E8C762542F1098345B73","documentURL":"http://127.0.0.1:62700/index.html","request":{"url":"http://127.0.0.1:62700/img2.bin","method":"GET","headers":{"sec-ch-ua-platform":"\"macOS\"","Referer":"http://127.0.0.1:62700/index.html","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0"},"mixedContentType":"none","initialPriority":"Medium","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191139.105665,"wallTime":1784244178.115773,"initiator":{"type":"parser","url":"http://127.0.0.1:62700/index.html","lineNumber":4,"columnNumber":60},"redirectHasExtraInfo":false,"type":"Image","frameId":"C8D09C5863610CBA19F8F06A912FDC75","hasUserGesture":false},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"75491.6","loaderId":"E9D3CB25E796E8C762542F1098345B73","documentURL":"http://127.0.0.1:62700/index.html","request":{"url":"http://127.0.0.1:62700/img3.bin","method":"GET","headers":{"sec-ch-ua-platform":"\"macOS\"","Referer":"http://127.0.0.1:62700/index.html","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0"},"mixedContentType":"none","initialPriority":"Medium","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191139.10572,"wallTime":1784244178.115862,"initiator":{"type":"parser","url":"http://127.0.0.1:62700/index.html","lineNumber":4,"columnNumber":80},"redirectHasExtraInfo":false,"type":"Image","frameId":"C8D09C5863610CBA19F8F06A912FDC75","hasUserGesture":false},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"75491.7","loaderId":"E9D3CB25E796E8C762542F1098345B73","documentURL":"http://127.0.0.1:62700/index.html","request":{"url":"http://127.0.0.1:62700/app.js","method":"GET","headers":{"sec-ch-ua-platform":"\"macOS\"","Referer":"http://127.0.0.1:62700/index.html","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0"},"mixedContentType":"none","initialPriority":"Medium","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191139.105863,"wallTime":1784244178.115973,"initiator":{"type":"parser","url":"http://127.0.0.1:62700/index.html","lineNumber":5,"columnNumber":21},"redirectHasExtraInfo":false,"type":"Script","frameId":"C8D09C5863610CBA19F8F06A912FDC75","hasUserGesture":false,"renderBlockingBehavior":"InBodyParserBlocking"},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Runtime.executionContextCreated","params":{"context":{"id":2,"origin":"http://127.0.0.1:62700","name":"","uniqueId":"-8030694222046454219.7525240474905787859","auxData":{"isDefault":true,"type":"default","frameId":"C8D09C5863610CBA19F8F06A912FDC75"}}},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"75491.5","associatedCookies":[],"headers":{"Accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62700","Referer":"http://127.0.0.1:62700/index.html","Sec-Fetch-Dest":"image","Sec-Fetch-Mode":"no-cors","Sec-Fetch-Site":"same-origin","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191139.108359},"clientSecurityState":{"initiatorIsSecureContext":true,"initiatorIPAddressSpace":"Loopback","localNetworkAccessRequestPolicy":"PermissionBlock"},"siteHasCookieInOtherPartition":false},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"75491.6","associatedCookies":[],"headers":{"Accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62700","Referer":"http://127.0.0.1:62700/index.html","Sec-Fetch-Dest":"image","Sec-Fetch-Mode":"no-cors","Sec-Fetch-Site":"same-origin","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191139.108634},"clientSecurityState":{"initiatorIsSecureContext":true,"initiatorIPAddressSpace":"Loopback","localNetworkAccessRequestPolicy":"PermissionBlock"},"siteHasCookieInOtherPartition":false},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"75491.7","associatedCookies":[],"headers":{"Accept":"*/*","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62700","Referer":"http://127.0.0.1:62700/index.html","Sec-Fetch-Dest":"script","Sec-Fetch-Mode":"no-cors","Sec-Fetch-Site":"same-origin","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191139.109042},"clientSecurityState":{"initiatorIsSecureContext":true,"initiatorIPAddressSpace":"Loopback","localNetworkAccessRequestPolicy":"PermissionBlock"},"siteHasCookieInOtherPartition":false},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.resourceChangedPriority","params":{"requestId":"75491.5","newPriority":"High","timestamp":191139.112936},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.resourceChangedPriority","params":{"requestId":"75491.4","newPriority":"High","timestamp":191139.112939},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.resourceChangedPriority","params":{"requestId":"75491.3","newPriority":"High","timestamp":191139.11294},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.resourceChangedPriority","params":{"requestId":"75491.6","newPriority":"High","timestamp":191139.112941},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.loadingFinished","params":{"requestId":"E9D3CB25E796E8C762542F1098345B73","timestamp":191139.100673,"encodedDataLength":586},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"75491.2","blockedCookies":[],"headers":{"Content-Length":"60015","Content-type":"text/css","Date":"Thu, 16 Jul 2026 23:22:58 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:58 GMT\r\nContent-type: text/css\r\nContent-Length: 60015\r\nLast-Modified: Thu, 16 Jul 2026 23:22:55 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceived","params":{"requestId":"75491.2","loaderId":"E9D3CB25E796E8C762542F1098345B73","timestamp":191139.253806,"type":"Stylesheet","response":{"url":"http://127.0.0.1:62700/styles.css","status":200,"statusText":"OK","headers":{"Content-Length":"60015","Date":"Thu, 16 Jul 2026 23:22:58 GMT","Content-type":"text/css","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"text/css","charset":"","connectionReused":false,"connectionId":29,"remoteIPAddress":"127.0.0.1","remotePort":62700,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":188,"timing":{"requestTime":191139.106187,"proxyStart":-1,"proxyEnd":-1,"dnsStart":0.179,"dnsEnd":0.287,"connectStart":0.287,"connectEnd":1.071,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":1.137,"sendEnd":1.17,"pushStart":0,"pushEnd":0,"receiveHeadersStart":2.754,"receiveHeadersEnd":146.993},"responseTime":1.784244178119036e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"C8D09C5863610CBA19F8F06A912FDC75"},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"75491.3","blockedCookies":[],"headers":{"Content-Length":"220000","Content-type":"application/octet-stream","Date":"Thu, 16 Jul 2026 23:22:58 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:58 GMT\r\nContent-type: application/octet-stream\r\nContent-Length: 220000\r\nLast-Modified: Thu, 16 Jul 2026 23:22:55 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"75491.5","blockedCookies":[],"headers":{"Content-Length":"220000","Content-type":"application/octet-stream","Date":"Thu, 16 Jul 2026 23:22:58 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:58 GMT\r\nContent-type: application/octet-stream\r\nContent-Length: 220000\r\nLast-Modified: Thu, 16 Jul 2026 23:22:55 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"75491.6","blockedCookies":[],"headers":{"Content-Length":"220000","Content-type":"application/octet-stream","Date":"Thu, 16 Jul 2026 23:22:58 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:58 GMT\r\nContent-type: application/octet-stream\r\nContent-Length: 220000\r\nLast-Modified: Thu, 16 Jul 2026 23:22:55 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"75491.4","blockedCookies":[],"headers":{"Content-Length":"220000","Content-type":"application/octet-stream","Date":"Thu, 16 Jul 2026 23:22:58 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:58 GMT\r\nContent-type: application/octet-stream\r\nContent-Length: 220000\r\nLast-Modified: Thu, 16 Jul 2026 23:22:55 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"75491.7","blockedCookies":[],"headers":{"Content-Length":"56021","Content-type":"text/javascript","Date":"Thu, 16 Jul 2026 23:22:58 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:58 GMT\r\nContent-type: text/javascript\r\nContent-Length: 56021\r\nLast-Modified: Thu, 16 Jul 2026 23:22:55 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.267268,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceived","params":{"requestId":"75491.3","loaderId":"E9D3CB25E796E8C762542F1098345B73","timestamp":191139.277944,"type":"Image","response":{"url":"http://127.0.0.1:62700/img0.bin","status":200,"statusText":"OK","headers":{"Content-Length":"220000","Date":"Thu, 16 Jul 2026 23:22:58 GMT","Content-type":"application/octet-stream","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"application/octet-stream","charset":"","connectionReused":false,"connectionId":37,"remoteIPAddress":"127.0.0.1","remotePort":62700,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":205,"timing":{"requestTime":191139.106682,"proxyStart":-1,"proxyEnd":-1,"dnsStart":0.105,"dnsEnd":0.17,"connectStart":0.17,"connectEnd":0.586,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":0.698,"sendEnd":0.713,"pushStart":0,"pushEnd":0,"receiveHeadersStart":2.733,"receiveHeadersEnd":147.599},"responseTime":1.784244178119509e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"C8D09C5863610CBA19F8F06A912FDC75"},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.3","timestamp":191139.277973,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.loadingFinished","params":{"requestId":"75491.3","timestamp":191139.27804,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceived","params":{"requestId":"75491.5","loaderId":"E9D3CB25E796E8C762542F1098345B73","timestamp":191139.278406,"type":"Image","response":{"url":"http://127.0.0.1:62700/img2.bin","status":200,"statusText":"OK","headers":{"Content-Length":"220000","Date":"Thu, 16 Jul 2026 23:22:58 GMT","Content-type":"application/octet-stream","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"application/octet-stream","charset":"","connectionReused":false,"connectionId":53,"remoteIPAddress":"127.0.0.1","remotePort":62700,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":205,"timing":{"requestTime":191139.108359,"proxyStart":-1,"proxyEnd":-1,"dnsStart":0.071,"dnsEnd":0.145,"connectStart":0.145,"connectEnd":0.902,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":0.99,"sendEnd":1.016,"pushStart":0,"pushEnd":0,"receiveHeadersStart":2.216,"receiveHeadersEnd":158.364},"responseTime":1.784244178120669e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"C8D09C5863610CBA19F8F06A912FDC75"},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.5","timestamp":191139.278423,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.loadingFinished","params":{"requestId":"75491.5","timestamp":191139.278437,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceived","params":{"requestId":"75491.6","loaderId":"E9D3CB25E796E8C762542F1098345B73","timestamp":191139.278709,"type":"Image","response":{"url":"http://127.0.0.1:62700/img3.bin","status":200,"statusText":"OK","headers":{"Content-Length":"220000","Date":"Thu, 16 Jul 2026 23:22:58 GMT","Content-type":"application/octet-stream","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"application/octet-stream","charset":"","connectionReused":false,"connectionId":61,"remoteIPAddress":"127.0.0.1","remotePort":62700,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":205,"timing":{"requestTime":191139.108634,"proxyStart":-1,"proxyEnd":-1,"dnsStart":0.058,"dnsEnd":0.201,"connectStart":0.201,"connectEnd":0.633,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":0.758,"sendEnd":0.773,"pushStart":0,"pushEnd":0,"receiveHeadersStart":2.086,"receiveHeadersEnd":158.206},"responseTime":1.784244178120813e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"C8D09C5863610CBA19F8F06A912FDC75"},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.6","timestamp":191139.278744,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.loadingFinished","params":{"requestId":"75491.6","timestamp":191139.278763,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceived","params":{"requestId":"75491.4","loaderId":"E9D3CB25E796E8C762542F1098345B73","timestamp":191139.278974,"type":"Image","response":{"url":"http://127.0.0.1:62700/img1.bin","status":200,"statusText":"OK","headers":{"Content-Length":"220000","Date":"Thu, 16 Jul 2026 23:22:58 GMT","Content-type":"application/octet-stream","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"application/octet-stream","charset":"","connectionReused":false,"connectionId":45,"remoteIPAddress":"127.0.0.1","remotePort":62700,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":205,"timing":{"requestTime":191139.107037,"proxyStart":-1,"proxyEnd":-1,"dnsStart":0.059,"dnsEnd":0.117,"connectStart":0.117,"connectEnd":0.234,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":0.373,"sendEnd":0.401,"pushStart":0,"pushEnd":0,"receiveHeadersStart":3.775,"receiveHeadersEnd":159.898},"responseTime":1.784244178120906e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"C8D09C5863610CBA19F8F06A912FDC75"},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.4","timestamp":191139.27899,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.loadingFinished","params":{"requestId":"75491.4","timestamp":191139.279003,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.280656,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.281012,"dataLength":908,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.290094,"dataLength":1500,"encodedDataLength":908},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.290676,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.300167,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.301306,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.317156,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.318507,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.333641,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.334549,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.334677,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.336202,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.345904,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.346598,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.351512,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.358348,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.359165,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.370605,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.371207,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.385279,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.386874,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.391143,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.400178,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.400574,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.412454,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.412825,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.412841,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.424636,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.425104,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.433471,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.433946,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.450143,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.450613,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.450665,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.467085,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.467627,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.468083,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceived","params":{"requestId":"75491.7","loaderId":"E9D3CB25E796E8C762542F1098345B73","timestamp":191139.468856,"type":"Script","response":{"url":"http://127.0.0.1:62700/app.js","status":200,"statusText":"OK","headers":{"Content-Length":"56021","Date":"Thu, 16 Jul 2026 23:22:58 GMT","Content-type":"text/javascript","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"text/javascript","charset":"","connectionReused":false,"connectionId":69,"remoteIPAddress":"127.0.0.1","remotePort":62700,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":195,"timing":{"requestTime":191139.109042,"proxyStart":-1,"proxyEnd":-1,"dnsStart":0.077,"dnsEnd":0.144,"connectStart":0.144,"connectEnd":0.253,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":0.401,"sendEnd":0.416,"pushStart":0,"pushEnd":0,"receiveHeadersStart":2.148,"receiveHeadersEnd":157.97},"responseTime":1.784244178121284e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"C8D09C5863610CBA19F8F06A912FDC75"},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468885,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468887,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468887,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468888,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468889,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.46889,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468891,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468892,"dataLength":3000,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468893,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468895,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468896,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468897,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468898,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468899,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468902,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468903,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468904,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468906,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468906,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468907,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468908,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468909,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468911,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468913,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468914,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468915,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468916,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468918,"dataLength":3000,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468919,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.46892,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468922,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468923,"dataLength":3000,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468923,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.468924,"dataLength":2021,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.7","timestamp":191139.469034,"dataLength":0,"encodedDataLength":56021},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.loadingFinished","params":{"requestId":"75491.7","timestamp":191139.467772,"encodedDataLength":56216},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.483463,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.483976,"dataLength":607,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.2","timestamp":191139.484429,"dataLength":0,"encodedDataLength":607},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.loadingFinished","params":{"requestId":"75491.2","timestamp":191139.483986,"encodedDataLength":60203},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"boot"}],"executionContextId":2,"timestamp":1.78424417849501e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"3","url":"http://127.0.0.1:62700/app.js","lineNumber":0,"columnNumber":8}]}},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"render pass"},{"type":"number","value":0,"description":"0"}],"executionContextId":2,"timestamp":1.784244178497628e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"4","url":"http://127.0.0.1:62700/index.html","lineNumber":7,"columnNumber":36}]}},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"render pass"},{"type":"number","value":1,"description":"1"}],"executionContextId":2,"timestamp":1.784244178497678e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"4","url":"http://127.0.0.1:62700/index.html","lineNumber":7,"columnNumber":36}]}},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"render pass"},{"type":"number","value":2,"description":"2"}],"executionContextId":2,"timestamp":1.784244178497743e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"4","url":"http://127.0.0.1:62700/index.html","lineNumber":7,"columnNumber":36}]}},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"render pass"},{"type":"number","value":3,"description":"3"}],"executionContextId":2,"timestamp":1.784244178497774e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"4","url":"http://127.0.0.1:62700/index.html","lineNumber":7,"columnNumber":36}]}},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"render pass"},{"type":"number","value":4,"description":"4"}],"executionContextId":2,"timestamp":1.784244178497795e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"4","url":"http://127.0.0.1:62700/index.html","lineNumber":7,"columnNumber":36}]}},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"75491.14","loaderId":"E9D3CB25E796E8C762542F1098345B73","documentURL":"http://127.0.0.1:62700/index.html","request":{"url":"http://127.0.0.1:62700/data.json","method":"GET","headers":{"sec-ch-ua-platform":"\"macOS\"","Referer":"http://127.0.0.1:62700/index.html","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0"},"mixedContentType":"none","initialPriority":"High","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191139.48789,"wallTime":1784244178.497997,"initiator":{"type":"script","stack":{"callFrames":[{"functionName":"","scriptId":"4","url":"http://127.0.0.1:62700/index.html","lineNumber":8,"columnNumber":0}]}},"redirectHasExtraInfo":false,"type":"Fetch","frameId":"C8D09C5863610CBA19F8F06A912FDC75","hasUserGesture":false},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Page.domContentEventFired","params":{"timestamp":191139.488027},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Page.loadEventFired","params":{"timestamp":191139.488058},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Page.frameStoppedLoading","params":{"frameId":"C8D09C5863610CBA19F8F06A912FDC75"},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"75491.14","associatedCookies":[],"headers":{"Accept":"*/*","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62700","Referer":"http://127.0.0.1:62700/index.html","Sec-Fetch-Dest":"empty","Sec-Fetch-Mode":"cors","Sec-Fetch-Site":"same-origin","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191139.488114},"clientSecurityState":{"initiatorIsSecureContext":true,"initiatorIPAddressSpace":"Loopback","localNetworkAccessRequestPolicy":"PermissionBlock"},"siteHasCookieInOtherPartition":false},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"75491.14","blockedCookies":[],"headers":{"Content-Length":"198800","Content-type":"application/json","Date":"Thu, 16 Jul 2026 23:22:58 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:58 GMT\r\nContent-type: application/json\r\nContent-Length: 198800\r\nLast-Modified: Thu, 16 Jul 2026 23:22:55 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.responseReceived","params":{"requestId":"75491.14","loaderId":"E9D3CB25E796E8C762542F1098345B73","timestamp":191139.68401,"type":"Fetch","response":{"url":"http://127.0.0.1:62700/data.json","status":200,"statusText":"OK","headers":{"Content-Length":"198800","Date":"Thu, 16 Jul 2026 23:22:58 GMT","Content-type":"application/json","Last-Modified":"Thu, 16 Jul 2026 23:22:55 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"application/json","charset":"","connectionReused":false,"connectionId":77,"remoteIPAddress":"127.0.0.1","remotePort":62700,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":197,"timing":{"requestTime":191139.488114,"proxyStart":-1,"proxyEnd":-1,"dnsStart":0.061,"dnsEnd":0.123,"connectStart":0.123,"connectEnd":0.223,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":0.251,"sendEnd":0.269,"pushStart":0,"pushEnd":0,"receiveHeadersStart":0.665,"receiveHeadersEnd":195.342},"responseTime":1.784244178498868e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"C8D09C5863610CBA19F8F06A912FDC75"},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.684414,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.684559,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.684592,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.684989,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.685313,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.685547,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.700305,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.700982,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.701636,"dataLength":3000,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.70211,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.703502,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.712406,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.712731,"dataLength":3000,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.713105,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.716759,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.717133,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.733488,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.7339,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.734218,"dataLength":3000,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.734458,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.745678,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.746006,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.746299,"dataLength":3000,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.757912,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.758408,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.758723,"dataLength":3000,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.766833,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.767176,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.7835,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.784409,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.784486,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.785082,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.785156,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.785798,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.790021,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.790701,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.794484,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.79515,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.800117,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.800897,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.818646,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.819904,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.820659,"dataLength":3000,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.821165,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.828556,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.829022,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.832884,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.833428,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.839622,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.840095,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.850171,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.850723,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.851368,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.867079,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.868028,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.868643,"dataLength":3000,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.869433,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.874281,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.874816,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.876786,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.879265,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.883451,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.884102,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.88661,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.891181,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.89179,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.891902,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.912033,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.91315,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.913285,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.914177,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.914987,"dataLength":3000,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.915817,"dataLength":1500,"encodedDataLength":4500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.916078,"dataLength":1500,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.933578,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.93459,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.935426,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.935512,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.936312,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.939764,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.940102,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.950326,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.951528,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.952415,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.966851,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.967123,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.967407,"dataLength":3000,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.967705,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.971733,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.971948,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.983518,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.984032,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.984512,"dataLength":3000,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.986566,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.99583,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.996392,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191139.99688,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191140.000068,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191140.000422,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191140.016825,"dataLength":1500,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191140.017427,"dataLength":3000,"encodedDataLength":1500},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191140.017812,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191140.017857,"dataLength":1500,"encodedDataLength":3000},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191140.018479,"dataLength":800,"encodedDataLength":0},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.dataReceived","params":{"requestId":"75491.14","timestamp":191140.019123,"dataLength":0,"encodedDataLength":800},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Network.loadingFinished","params":{"requestId":"75491.14","timestamp":191140.01843,"encodedDataLength":198997},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
-{"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"rows"},{"type":"number","value":900,"description":"900"}],"executionContextId":2,"timestamp":1.784244179029513e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"4","url":"http://127.0.0.1:62700/index.html","lineNumber":8,"columnNumber":57}]}},"sessionId":"194DB96D0F674F1BD4D295AD56547FE3"}
diff --git a/tests/unit/proxy/fixtures/workload_page_load_loopback.jsonl b/tests/unit/proxy/fixtures/workload_page_load_loopback.jsonl
deleted file mode 100644
index bfa3a3271..000000000
--- a/tests/unit/proxy/fixtures/workload_page_load_loopback.jsonl
+++ /dev/null
@@ -1,83 +0,0 @@
-{"method":"Target.attachedToTarget","params":{"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD","targetInfo":{"targetId":"4DCEEB232DED9A65C4C075B27C710EAD","type":"page","title":"about:blank","url":"about:blank","attached":true,"canAccessOpener":false,"browserContextId":"03E787F50066BC1605FE0452B5277D80"},"waitingForDebugger":false}}
-{"id":1,"result":{}}
-{"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"4DCEEB232DED9A65C4C075B27C710EAD","type":"page","title":"about:blank","url":"about:blank","attached":true,"canAccessOpener":false,"browserContextId":"03E787F50066BC1605FE0452B5277D80"}}}
-{"id":2,"result":{}}
-{"method":"Runtime.executionContextCreated","params":{"context":{"id":1,"origin":"://","name":"","uniqueId":"8777511098671345158.5905806312560495113","auxData":{"isDefault":true,"type":"default","frameId":"4DCEEB232DED9A65C4C075B27C710EAD"}}},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"id":3,"result":{},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"id":4,"result":{},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"id":5,"result":{},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"id":6,"result":{},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Page.frameStartedNavigating","params":{"frameId":"4DCEEB232DED9A65C4C075B27C710EAD","url":"http://127.0.0.1:62680/index.html","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","navigationType":"differentDocument"},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Page.frameStartedLoading","params":{"frameId":"4DCEEB232DED9A65C4C075B27C710EAD"},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"A8B88DDE85271025AAB7716F78FCCFB9","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","documentURL":"http://127.0.0.1:62680/index.html","request":{"url":"http://127.0.0.1:62680/index.html","method":"GET","headers":{"Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"mixedContentType":"none","initialPriority":"VeryHigh","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191085.701804,"wallTime":1784244124.712865,"initiator":{"type":"other"},"redirectHasExtraInfo":false,"type":"Document","frameId":"4DCEEB232DED9A65C4C075B27C710EAD","hasUserGesture":false},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"A8B88DDE85271025AAB7716F78FCCFB9","associatedCookies":[],"headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62680","Sec-Fetch-Dest":"document","Sec-Fetch-Mode":"navigate","Sec-Fetch-Site":"none","Sec-Fetch-User":"?1","Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191085.703565},"siteHasCookieInOtherPartition":false},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"A8B88DDE85271025AAB7716F78FCCFB9","blockedCookies":[],"headers":{"Content-Length":"399","Content-type":"text/html","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:04 GMT\r\nContent-type: text/html\r\nContent-Length: 399\r\nLast-Modified: Thu, 16 Jul 2026 23:22:02 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceived","params":{"requestId":"A8B88DDE85271025AAB7716F78FCCFB9","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","timestamp":191085.714887,"type":"Document","response":{"url":"http://127.0.0.1:62680/index.html","status":200,"statusText":"OK","headers":{"Content-Length":"399","Content-type":"text/html","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"text/html","charset":"","connectionReused":false,"connectionId":21,"remoteIPAddress":"127.0.0.1","remotePort":62680,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":187,"timing":{"requestTime":191085.703565,"proxyStart":-1,"proxyEnd":-1,"dnsStart":5.664,"dnsEnd":6.085,"connectStart":6.085,"connectEnd":6.417,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":6.667,"sendEnd":6.786,"pushStart":0,"pushEnd":0,"receiveHeadersStart":10.393,"receiveHeadersEnd":10.585},"responseTime":1.784244124724887e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"4DCEEB232DED9A65C4C075B27C710EAD"},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"id":7,"result":{"frameId":"4DCEEB232DED9A65C4C075B27C710EAD","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","isDownload":false},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Runtime.executionContextsCleared","params":{},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Runtime.executionContextsCleared","params":{},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Page.frameNavigated","params":{"frame":{"id":"4DCEEB232DED9A65C4C075B27C710EAD","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","url":"http://127.0.0.1:62680/index.html","domainAndRegistry":"","securityOrigin":"http://127.0.0.1:62680","securityOriginDetails":{"isLocalhost":true},"mimeType":"text/html","adFrameStatus":{"adFrameType":"none"},"secureContextType":"SecureLocalhost","crossOriginIsolatedContextType":"NotIsolated","gatedAPIFeatures":[]},"type":"Navigation"},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.policyUpdated","params":{},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"4DCEEB232DED9A65C4C075B27C710EAD","type":"page","title":"127.0.0.1:62680/index.html","url":"http://127.0.0.1:62680/index.html","attached":true,"canAccessOpener":false,"browserContextId":"03E787F50066BC1605FE0452B5277D80"}}}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"75174.2","associatedCookies":[],"headers":{"Accept":"text/css,*/*;q=0.1","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62680","Referer":"http://127.0.0.1:62680/index.html","Sec-Fetch-Dest":"style","Sec-Fetch-Mode":"no-cors","Sec-Fetch-Site":"same-origin","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191085.720973},"clientSecurityState":{"initiatorIsSecureContext":true,"initiatorIPAddressSpace":"Loopback","localNetworkAccessRequestPolicy":"PermissionBlock"},"siteHasCookieInOtherPartition":false},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"75174.3","associatedCookies":[],"headers":{"Accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62680","Referer":"http://127.0.0.1:62680/index.html","Sec-Fetch-Dest":"image","Sec-Fetch-Mode":"no-cors","Sec-Fetch-Site":"same-origin","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191085.721186},"clientSecurityState":{"initiatorIsSecureContext":true,"initiatorIPAddressSpace":"Loopback","localNetworkAccessRequestPolicy":"PermissionBlock"},"siteHasCookieInOtherPartition":false},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"75174.4","associatedCookies":[],"headers":{"Accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62680","Referer":"http://127.0.0.1:62680/index.html","Sec-Fetch-Dest":"image","Sec-Fetch-Mode":"no-cors","Sec-Fetch-Site":"same-origin","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191085.721307},"clientSecurityState":{"initiatorIsSecureContext":true,"initiatorIPAddressSpace":"Loopback","localNetworkAccessRequestPolicy":"PermissionBlock"},"siteHasCookieInOtherPartition":false},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.dataReceived","params":{"requestId":"A8B88DDE85271025AAB7716F78FCCFB9","timestamp":191085.71816,"dataLength":399,"encodedDataLength":0},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"75174.2","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","documentURL":"http://127.0.0.1:62680/index.html","request":{"url":"http://127.0.0.1:62680/styles.css","method":"GET","headers":{"sec-ch-ua-platform":"\"macOS\"","Referer":"http://127.0.0.1:62680/index.html","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0"},"mixedContentType":"none","initialPriority":"VeryHigh","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191085.720545,"wallTime":1784244124.731566,"initiator":{"type":"parser","url":"http://127.0.0.1:62680/index.html","lineNumber":1,"columnNumber":41},"redirectHasExtraInfo":false,"type":"Stylesheet","frameId":"4DCEEB232DED9A65C4C075B27C710EAD","hasUserGesture":false,"renderBlockingBehavior":"Blocking"},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"75174.3","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","documentURL":"http://127.0.0.1:62680/index.html","request":{"url":"http://127.0.0.1:62680/img0.bin","method":"GET","headers":{"sec-ch-ua-platform":"\"macOS\"","Referer":"http://127.0.0.1:62680/index.html","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0"},"mixedContentType":"none","initialPriority":"Medium","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191085.720801,"wallTime":1784244124.731739,"initiator":{"type":"parser","url":"http://127.0.0.1:62680/index.html","lineNumber":4,"columnNumber":20},"redirectHasExtraInfo":false,"type":"Image","frameId":"4DCEEB232DED9A65C4C075B27C710EAD","hasUserGesture":false},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"75174.4","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","documentURL":"http://127.0.0.1:62680/index.html","request":{"url":"http://127.0.0.1:62680/img1.bin","method":"GET","headers":{"sec-ch-ua-platform":"\"macOS\"","Referer":"http://127.0.0.1:62680/index.html","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0"},"mixedContentType":"none","initialPriority":"Medium","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191085.720863,"wallTime":1784244124.731824,"initiator":{"type":"parser","url":"http://127.0.0.1:62680/index.html","lineNumber":4,"columnNumber":40},"redirectHasExtraInfo":false,"type":"Image","frameId":"4DCEEB232DED9A65C4C075B27C710EAD","hasUserGesture":false},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"75174.5","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","documentURL":"http://127.0.0.1:62680/index.html","request":{"url":"http://127.0.0.1:62680/img2.bin","method":"GET","headers":{"sec-ch-ua-platform":"\"macOS\"","Referer":"http://127.0.0.1:62680/index.html","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0"},"mixedContentType":"none","initialPriority":"Medium","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191085.72093,"wallTime":1784244124.731866,"initiator":{"type":"parser","url":"http://127.0.0.1:62680/index.html","lineNumber":4,"columnNumber":60},"redirectHasExtraInfo":false,"type":"Image","frameId":"4DCEEB232DED9A65C4C075B27C710EAD","hasUserGesture":false},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"75174.6","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","documentURL":"http://127.0.0.1:62680/index.html","request":{"url":"http://127.0.0.1:62680/img3.bin","method":"GET","headers":{"sec-ch-ua-platform":"\"macOS\"","Referer":"http://127.0.0.1:62680/index.html","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0"},"mixedContentType":"none","initialPriority":"Medium","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191085.720965,"wallTime":1784244124.731902,"initiator":{"type":"parser","url":"http://127.0.0.1:62680/index.html","lineNumber":4,"columnNumber":80},"redirectHasExtraInfo":false,"type":"Image","frameId":"4DCEEB232DED9A65C4C075B27C710EAD","hasUserGesture":false},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"75174.7","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","documentURL":"http://127.0.0.1:62680/index.html","request":{"url":"http://127.0.0.1:62680/app.js","method":"GET","headers":{"sec-ch-ua-platform":"\"macOS\"","Referer":"http://127.0.0.1:62680/index.html","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0"},"mixedContentType":"none","initialPriority":"Medium","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191085.721001,"wallTime":1784244124.731938,"initiator":{"type":"parser","url":"http://127.0.0.1:62680/index.html","lineNumber":5,"columnNumber":21},"redirectHasExtraInfo":false,"type":"Script","frameId":"4DCEEB232DED9A65C4C075B27C710EAD","hasUserGesture":false,"renderBlockingBehavior":"InBodyParserBlocking"},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Runtime.executionContextCreated","params":{"context":{"id":2,"origin":"http://127.0.0.1:62680","name":"","uniqueId":"2792145801372266464.-5744135212519671797","auxData":{"isDefault":true,"type":"default","frameId":"4DCEEB232DED9A65C4C075B27C710EAD"}}},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"75174.3","blockedCookies":[],"headers":{"Content-Length":"220000","Content-type":"application/octet-stream","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:04 GMT\r\nContent-type: application/octet-stream\r\nContent-Length: 220000\r\nLast-Modified: Thu, 16 Jul 2026 23:22:02 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"75174.4","blockedCookies":[],"headers":{"Content-Length":"220000","Content-type":"application/octet-stream","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:04 GMT\r\nContent-type: application/octet-stream\r\nContent-Length: 220000\r\nLast-Modified: Thu, 16 Jul 2026 23:22:02 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"75174.2","blockedCookies":[],"headers":{"Content-Length":"60015","Content-type":"text/css","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:04 GMT\r\nContent-type: text/css\r\nContent-Length: 60015\r\nLast-Modified: Thu, 16 Jul 2026 23:22:02 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"75174.5","associatedCookies":[],"headers":{"Accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62680","Referer":"http://127.0.0.1:62680/index.html","Sec-Fetch-Dest":"image","Sec-Fetch-Mode":"no-cors","Sec-Fetch-Site":"same-origin","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191085.72251},"clientSecurityState":{"initiatorIsSecureContext":true,"initiatorIPAddressSpace":"Loopback","localNetworkAccessRequestPolicy":"PermissionBlock"},"siteHasCookieInOtherPartition":false},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"75174.6","associatedCookies":[],"headers":{"Accept":"image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62680","Referer":"http://127.0.0.1:62680/index.html","Sec-Fetch-Dest":"image","Sec-Fetch-Mode":"no-cors","Sec-Fetch-Site":"same-origin","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191085.72266},"clientSecurityState":{"initiatorIsSecureContext":true,"initiatorIPAddressSpace":"Loopback","localNetworkAccessRequestPolicy":"PermissionBlock"},"siteHasCookieInOtherPartition":false},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"75174.7","associatedCookies":[],"headers":{"Accept":"*/*","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62680","Referer":"http://127.0.0.1:62680/index.html","Sec-Fetch-Dest":"script","Sec-Fetch-Mode":"no-cors","Sec-Fetch-Site":"same-origin","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191085.722778},"clientSecurityState":{"initiatorIsSecureContext":true,"initiatorIPAddressSpace":"Loopback","localNetworkAccessRequestPolicy":"PermissionBlock"},"siteHasCookieInOtherPartition":false},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceived","params":{"requestId":"75174.3","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","timestamp":191085.722601,"type":"Image","response":{"url":"http://127.0.0.1:62680/img0.bin","status":200,"statusText":"OK","headers":{"Content-Length":"220000","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Content-type":"application/octet-stream","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"application/octet-stream","charset":"","connectionReused":false,"connectionId":37,"remoteIPAddress":"127.0.0.1","remotePort":62680,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":205,"timing":{"requestTime":191085.721186,"proxyStart":-1,"proxyEnd":-1,"dnsStart":0.036,"dnsEnd":0.061,"connectStart":0.061,"connectEnd":0.202,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":0.314,"sendEnd":0.37,"pushStart":0,"pushEnd":0,"receiveHeadersStart":1.137,"receiveHeadersEnd":1.163},"responseTime":1.784244124733252e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"4DCEEB232DED9A65C4C075B27C710EAD"},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.dataReceived","params":{"requestId":"75174.3","timestamp":191085.722759,"dataLength":65536,"encodedDataLength":0},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.loadingFinished","params":{"requestId":"75174.3","timestamp":191085.723653,"encodedDataLength":65536},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceived","params":{"requestId":"75174.4","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","timestamp":191085.723908,"type":"Image","response":{"url":"http://127.0.0.1:62680/img1.bin","status":200,"statusText":"OK","headers":{"Content-Length":"220000","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Content-type":"application/octet-stream","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"application/octet-stream","charset":"","connectionReused":false,"connectionId":45,"remoteIPAddress":"127.0.0.1","remotePort":62680,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":205,"timing":{"requestTime":191085.721307,"proxyStart":-1,"proxyEnd":-1,"dnsStart":0.029,"dnsEnd":0.051,"connectStart":0.051,"connectEnd":0.134,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":0.285,"sendEnd":0.304,"pushStart":0,"pushEnd":0,"receiveHeadersStart":1.56,"receiveHeadersEnd":1.572},"responseTime":1.784244124733797e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"4DCEEB232DED9A65C4C075B27C710EAD"},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.dataReceived","params":{"requestId":"75174.4","timestamp":191085.723915,"dataLength":212940,"encodedDataLength":0},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.loadingFinished","params":{"requestId":"75174.4","timestamp":191085.724014,"encodedDataLength":212940},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"75174.6","blockedCookies":[],"headers":{"Content-Length":"220000","Content-type":"application/octet-stream","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:04 GMT\r\nContent-type: application/octet-stream\r\nContent-Length: 220000\r\nLast-Modified: Thu, 16 Jul 2026 23:22:02 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceived","params":{"requestId":"75174.2","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","timestamp":191085.724152,"type":"Stylesheet","response":{"url":"http://127.0.0.1:62680/styles.css","status":200,"statusText":"OK","headers":{"Content-Length":"60015","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Content-type":"text/css","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"text/css","charset":"","connectionReused":false,"connectionId":29,"remoteIPAddress":"127.0.0.1","remotePort":62680,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":188,"timing":{"requestTime":191085.720973,"proxyStart":-1,"proxyEnd":-1,"dnsStart":0.077,"dnsEnd":0.136,"connectStart":0.136,"connectEnd":0.41,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":0.441,"sendEnd":0.462,"pushStart":0,"pushEnd":0,"receiveHeadersStart":2.183,"receiveHeadersEnd":2.192},"responseTime":1.784244124734085e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"4DCEEB232DED9A65C4C075B27C710EAD"},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.dataReceived","params":{"requestId":"75174.2","timestamp":191085.724162,"dataLength":60015,"encodedDataLength":0},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.dataReceived","params":{"requestId":"75174.2","timestamp":191085.724444,"dataLength":0,"encodedDataLength":60015},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.loadingFinished","params":{"requestId":"75174.2","timestamp":191085.723683,"encodedDataLength":60203},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"75174.5","blockedCookies":[],"headers":{"Content-Length":"220000","Content-type":"application/octet-stream","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:04 GMT\r\nContent-type: application/octet-stream\r\nContent-Length: 220000\r\nLast-Modified: Thu, 16 Jul 2026 23:22:02 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"75174.7","blockedCookies":[],"headers":{"Content-Length":"56021","Content-type":"text/javascript","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:04 GMT\r\nContent-type: text/javascript\r\nContent-Length: 56021\r\nLast-Modified: Thu, 16 Jul 2026 23:22:02 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.resourceChangedPriority","params":{"requestId":"75174.5","newPriority":"High","timestamp":191085.737871},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.resourceChangedPriority","params":{"requestId":"75174.6","newPriority":"High","timestamp":191085.737874},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceived","params":{"requestId":"75174.5","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","timestamp":191085.738352,"type":"Image","response":{"url":"http://127.0.0.1:62680/img2.bin","status":200,"statusText":"OK","headers":{"Content-Length":"220000","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Content-type":"application/octet-stream","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"application/octet-stream","charset":"","connectionReused":false,"connectionId":53,"remoteIPAddress":"127.0.0.1","remotePort":62680,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":205,"timing":{"requestTime":191085.72251,"proxyStart":-1,"proxyEnd":-1,"dnsStart":0.042,"dnsEnd":0.086,"connectStart":0.086,"connectEnd":0.354,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":1.051,"sendEnd":1.066,"pushStart":0,"pushEnd":0,"receiveHeadersStart":2.044,"receiveHeadersEnd":2.062},"responseTime":1.784244124735483e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"4DCEEB232DED9A65C4C075B27C710EAD"},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.dataReceived","params":{"requestId":"75174.5","timestamp":191085.738363,"dataLength":220000,"encodedDataLength":0},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.loadingFinished","params":{"requestId":"75174.5","timestamp":191085.73839,"encodedDataLength":220000},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceived","params":{"requestId":"75174.6","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","timestamp":191085.738488,"type":"Image","response":{"url":"http://127.0.0.1:62680/img3.bin","status":200,"statusText":"OK","headers":{"Content-Length":"220000","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Content-type":"application/octet-stream","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"application/octet-stream","charset":"","connectionReused":false,"connectionId":61,"remoteIPAddress":"127.0.0.1","remotePort":62680,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":205,"timing":{"requestTime":191085.72266,"proxyStart":-1,"proxyEnd":-1,"dnsStart":0.029,"dnsEnd":0.05,"connectStart":0.05,"connectEnd":0.493,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":0.938,"sendEnd":0.948,"pushStart":0,"pushEnd":0,"receiveHeadersStart":1.79,"receiveHeadersEnd":1.8},"responseTime":1.784244124735379e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"4DCEEB232DED9A65C4C075B27C710EAD"},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.dataReceived","params":{"requestId":"75174.6","timestamp":191085.738492,"dataLength":220000,"encodedDataLength":0},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.loadingFinished","params":{"requestId":"75174.6","timestamp":191085.738511,"encodedDataLength":220000},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.loadingFinished","params":{"requestId":"A8B88DDE85271025AAB7716F78FCCFB9","timestamp":191085.714673,"encodedDataLength":586},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceived","params":{"requestId":"75174.7","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","timestamp":191085.738969,"type":"Script","response":{"url":"http://127.0.0.1:62680/app.js","status":200,"statusText":"OK","headers":{"Content-Length":"56021","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Content-type":"text/javascript","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"text/javascript","charset":"","connectionReused":false,"connectionId":69,"remoteIPAddress":"127.0.0.1","remotePort":62680,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":195,"timing":{"requestTime":191085.722778,"proxyStart":-1,"proxyEnd":-1,"dnsStart":0.038,"dnsEnd":0.06,"connectStart":0.06,"connectEnd":0.433,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":0.951,"sendEnd":0.963,"pushStart":0,"pushEnd":0,"receiveHeadersStart":2.331,"receiveHeadersEnd":2.35},"responseTime":1.784244124736038e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"4DCEEB232DED9A65C4C075B27C710EAD"},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.dataReceived","params":{"requestId":"75174.7","timestamp":191085.738974,"dataLength":56021,"encodedDataLength":0},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.dataReceived","params":{"requestId":"75174.7","timestamp":191085.73901,"dataLength":0,"encodedDataLength":56021},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.loadingFinished","params":{"requestId":"75174.7","timestamp":191085.725237,"encodedDataLength":56216},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"boot"}],"executionContextId":2,"timestamp":1.784244124750335e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"3","url":"http://127.0.0.1:62680/app.js","lineNumber":0,"columnNumber":8}]}},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"render pass"},{"type":"number","value":0,"description":"0"}],"executionContextId":2,"timestamp":1.784244124750657e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"4","url":"http://127.0.0.1:62680/index.html","lineNumber":7,"columnNumber":36}]}},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"render pass"},{"type":"number","value":1,"description":"1"}],"executionContextId":2,"timestamp":1.784244124750685e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"4","url":"http://127.0.0.1:62680/index.html","lineNumber":7,"columnNumber":36}]}},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"render pass"},{"type":"number","value":2,"description":"2"}],"executionContextId":2,"timestamp":1.784244124750694e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"4","url":"http://127.0.0.1:62680/index.html","lineNumber":7,"columnNumber":36}]}},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"render pass"},{"type":"number","value":3,"description":"3"}],"executionContextId":2,"timestamp":1.7842441247507e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"4","url":"http://127.0.0.1:62680/index.html","lineNumber":7,"columnNumber":36}]}},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"render pass"},{"type":"number","value":4,"description":"4"}],"executionContextId":2,"timestamp":1.784244124750704e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"4","url":"http://127.0.0.1:62680/index.html","lineNumber":7,"columnNumber":36}]}},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.requestWillBeSent","params":{"requestId":"75174.14","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","documentURL":"http://127.0.0.1:62680/index.html","request":{"url":"http://127.0.0.1:62680/data.json","method":"GET","headers":{"sec-ch-ua-platform":"\"macOS\"","Referer":"http://127.0.0.1:62680/index.html","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0"},"mixedContentType":"none","initialPriority":"High","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":191085.739991,"wallTime":1784244124.750932,"initiator":{"type":"script","stack":{"callFrames":[{"functionName":"","scriptId":"4","url":"http://127.0.0.1:62680/index.html","lineNumber":8,"columnNumber":0}]}},"redirectHasExtraInfo":false,"type":"Fetch","frameId":"4DCEEB232DED9A65C4C075B27C710EAD","hasUserGesture":false},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Page.domContentEventFired","params":{"timestamp":191085.740087},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Page.loadEventFired","params":{"timestamp":191085.740184},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Page.frameStoppedLoading","params":{"frameId":"4DCEEB232DED9A65C4C075B27C710EAD"},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"75174.14","associatedCookies":[],"headers":{"Accept":"*/*","Accept-Encoding":"gzip, deflate, br, zstd","Connection":"keep-alive","Host":"127.0.0.1:62680","Referer":"http://127.0.0.1:62680/index.html","Sec-Fetch-Dest":"empty","Sec-Fetch-Mode":"cors","Sec-Fetch-Site":"same-origin","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/149.0.7827.55 Safari/537.36","sec-ch-ua":"\"HeadlessChrome\";v=\"149\", \"Chromium\";v=\"149\", \"Not)A;Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\""},"connectTiming":{"requestTime":191085.740106},"clientSecurityState":{"initiatorIsSecureContext":true,"initiatorIPAddressSpace":"Loopback","localNetworkAccessRequestPolicy":"PermissionBlock"},"siteHasCookieInOtherPartition":false},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceivedExtraInfo","params":{"requestId":"75174.14","blockedCookies":[],"headers":{"Content-Length":"198800","Content-type":"application/json","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"resourceIPAddressSpace":"Loopback","statusCode":200,"headersText":"HTTP/1.0 200 OK\r\nServer: SimpleHTTP/0.6 Python/3.13.12\r\nDate: Thu, 16 Jul 2026 23:22:04 GMT\r\nContent-type: application/json\r\nContent-Length: 198800\r\nLast-Modified: Thu, 16 Jul 2026 23:22:02 GMT\r\n\r\n","cookiePartitionKey":{"topLevelSite":"http://127.0.0.1","hasCrossSiteAncestor":false},"cookiePartitionKeyOpaque":false,"exemptedCookies":[]},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.responseReceived","params":{"requestId":"75174.14","loaderId":"A8B88DDE85271025AAB7716F78FCCFB9","timestamp":191085.740981,"type":"Fetch","response":{"url":"http://127.0.0.1:62680/data.json","status":200,"statusText":"OK","headers":{"Content-Length":"198800","Date":"Thu, 16 Jul 2026 23:22:04 GMT","Content-type":"application/json","Last-Modified":"Thu, 16 Jul 2026 23:22:02 GMT","Server":"SimpleHTTP/0.6 Python/3.13.12"},"mimeType":"application/json","charset":"","connectionReused":false,"connectionId":77,"remoteIPAddress":"127.0.0.1","remotePort":62680,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":197,"timing":{"requestTime":191085.740106,"proxyStart":-1,"proxyEnd":-1,"dnsStart":0.056,"dnsEnd":0.119,"connectStart":0.119,"connectEnd":0.252,"sslStart":-1,"sslEnd":-1,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":0.282,"sendEnd":0.303,"pushStart":0,"pushEnd":0,"receiveHeadersStart":0.747,"receiveHeadersEnd":0.769},"responseTime":1.784244124751782e+12,"protocol":"http/1.0","alternateProtocolUsage":"unspecifiedReason","securityState":"secure"},"hasExtraInfo":true,"frameId":"4DCEEB232DED9A65C4C075B27C710EAD"},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.dataReceived","params":{"requestId":"75174.14","timestamp":191085.741235,"dataLength":65536,"encodedDataLength":65536},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.dataReceived","params":{"requestId":"75174.14","timestamp":191085.741321,"dataLength":131072,"encodedDataLength":0},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.dataReceived","params":{"requestId":"75174.14","timestamp":191085.741519,"dataLength":2192,"encodedDataLength":131072},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.dataReceived","params":{"requestId":"75174.14","timestamp":191085.741759,"dataLength":0,"encodedDataLength":2192},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Network.loadingFinished","params":{"requestId":"75174.14","timestamp":191085.741527,"encodedDataLength":198997},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
-{"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"rows"},{"type":"number","value":900,"description":"900"}],"executionContextId":2,"timestamp":1.784244124752842e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"4","url":"http://127.0.0.1:62680/index.html","lineNumber":8,"columnNumber":57}]}},"sessionId":"BED1D63FAE7CF7494D93E082011E8DBD"}
diff --git a/tests/unit/proxy/test_adapter_contract_coverage.py b/tests/unit/proxy/test_adapter_contract_coverage.py
deleted file mode 100644
index 606808ed1..000000000
--- a/tests/unit/proxy/test_adapter_contract_coverage.py
+++ /dev/null
@@ -1,139 +0,0 @@
-"""Every shipped adapter must be bound to its port's contract suite.
-
-A contract is only "required for adapter changes" if shipping an adapter WITHOUT one
-fails. Participation was a hand-maintained list of subclasses, so a new adapter with no
-binding sailed through green — not because it satisfied the contract, but because the
-contract never ran against it. Nothing announced the gap, because the gap was silence.
-
-So the shipped implementations are discovered by walking the package rather than listed
-here: a new adapter is covered the moment it exists, and stays red until someone binds
-it to its port's contract.
-
-Cloud adapters get the same treatment in tests/cloud/ — this file cannot import them
-(tests/unit is OSS-synced, and skyvern/ may not know cloud/ exists).
-"""
-
-from __future__ import annotations
-
-import importlib
-import inspect
-import pkgutil
-
-import pytest
-
-import skyvern.proxy.adapters
-from skyvern.proxy.adapters.memory import AllowAllAuth
-from skyvern.proxy.ports import AuthPort, EventPolicyPort, MetricsPort, SessionRegistryPort, UpstreamBrowserPort
-from tests.unit.proxy.test_auth_contract import AuthPortContract
-from tests.unit.proxy.test_event_policy_contract import EventPolicyPortContract
-from tests.unit.proxy.test_metrics_contract import MetricsPortContract
-from tests.unit.proxy.test_session_registry_contract import SessionRegistryContract
-from tests.unit.proxy.test_upstream_browser_port_contract import UpstreamBrowserPortContract
-
-# Each driven port, the reusable contract every adapter of it must pass, and the factory
-# that contract calls to get one.
-PORT_CONTRACTS = (
- (UpstreamBrowserPort, UpstreamBrowserPortContract, "make_port"),
- (SessionRegistryPort, SessionRegistryContract, "make_registry"),
- (AuthPort, AuthPortContract, "make_auth"),
- (MetricsPort, MetricsPortContract, "make_metrics"),
- (EventPolicyPort, EventPolicyPortContract, "make_policy"),
-)
-
-
-# Adapters that deliberately do NOT satisfy their port's contract, each with the reason.
-# An entry here is a decision a reviewer can see and argue with; what this file replaces
-# was silence. It is not a way to make a red go away — every exemption owes a test below
-# proving its stated reason is still true, so an adapter cannot quietly drift in here.
-CONTRACT_EXEMPT: dict[type, str] = {
- AllowAllAuth: (
- "A deliberate authentication bypass for the local-dev entrypoint "
- "(skyvern/proxy/__main__.py): it returns a principal for every caller, including one "
- "presenting no credential at all, so it cannot satisfy AuthPortContract's authenticate "
- "clauses and must never be reachable from the cloud wiring."
- ),
-}
-
-
-def _adapter_classes() -> list[type]:
- """Every class the adapters package defines itself (not what it imports)."""
- found: list[type] = []
- for module_info in pkgutil.iter_modules(skyvern.proxy.adapters.__path__, "skyvern.proxy.adapters."):
- module = importlib.import_module(module_info.name)
- for _, cls in inspect.getmembers(module, inspect.isclass):
- if cls.__module__ == module_info.name and not cls.__name__.startswith("_"):
- found.append(cls)
- return found
-
-
-def _implements(cls: type, port: type) -> bool:
- # The ports are runtime_checkable method-only Protocols, so a structural check is
- # exactly the question being asked: does this class present the port's surface?
- try:
- return issubclass(cls, port)
- except TypeError:
- return False
-
-
-def _all_subclasses(base: type) -> set[type]:
- direct = set(base.__subclasses__())
- return direct.union(*(_all_subclasses(child) for child in direct)) if direct else set()
-
-
-def _bound_adapters(contract: type, factory: str) -> set[type]:
- """The adapter types actually exercised by a collected contract suite.
-
- Derived from the suites themselves rather than a list, so an adapter can only be
- "covered" by really being run through its contract.
- """
- bound: set[type] = set()
- for subclass in _all_subclasses(contract):
- # Only pytest-collected suites count; the deliberately broken fixtures in
- # test_contract_teeth.py subclass these contracts too and prove nothing here.
- if not subclass.__name__.startswith("Test"):
- continue
- bound.add(type(getattr(subclass(), factory)()))
- return bound
-
-
- ("port", "contract", "factory"), PORT_CONTRACTS, ids=[entry[0].__name__ for entry in PORT_CONTRACTS]
-)
-def test_every_shipped_adapter_is_bound_to_its_port_contract(port: type, contract: type, factory: str) -> None:
- shipped = {cls for cls in _adapter_classes() if _implements(cls, port)}
- unbound = shipped - _bound_adapters(contract, factory) - set(CONTRACT_EXEMPT)
- assert not unbound, (
- f"{sorted(cls.__name__ for cls in unbound)} implement {port.__name__} but no collected "
- f"{contract.__name__} subclass runs them. Bind each to its contract — an adapter no "
- f"contract runs against is not covered, it is merely unexamined. If it genuinely cannot "
- f"conform, add it to CONTRACT_EXEMPT with the reason and a test pinning that reason."
- )
-
-
-async def test_the_allow_all_auth_exemption_still_describes_what_it_does() -> None:
- """An exemption is only honest while its stated reason is true.
-
- AllowAllAuth is exempt because it authenticates everyone by design. The day it starts
- checking credentials, that reason is false and it belongs under the contract like any
- other adapter — so this fails and says so, rather than leaving a stale excuse behind.
- """
- assert await AllowAllAuth().authenticate({}) is not None, (
- "AllowAllAuth no longer authenticates a caller presenting no credential, so its "
- "CONTRACT_EXEMPT reason is stale: bind it to AuthPortContract instead."
- )
-
-
-def test_the_coverage_check_can_actually_find_an_adapter() -> None:
- """Guard on the guard: if the package walk returned nothing, every assertion above
- would pass vacuously and the whole file would be decoration."""
- assert len(_adapter_classes()) > 5
- assert any(_implements(cls, UpstreamBrowserPort) for cls in _adapter_classes())
-
-
-def test_every_exemption_names_an_adapter_that_still_exists() -> None:
- """A stale exemption silently widens the escape hatch for a class nobody ships."""
- shipped = set(_adapter_classes())
- assert set(CONTRACT_EXEMPT) <= shipped, (
- f"exemptions for adapters that no longer exist: {sorted(cls.__name__ for cls in set(CONTRACT_EXEMPT) - shipped)}"
- )
diff --git a/tests/unit/proxy/test_auth_contract.py b/tests/unit/proxy/test_auth_contract.py
deleted file mode 100644
index 15f0aa2c1..000000000
--- a/tests/unit/proxy/test_auth_contract.py
+++ /dev/null
@@ -1,233 +0,0 @@
-"""Reusable contract suite for AuthPort adapters.
-
-Any adapter (the generic static-keys adapter here, the cloud Skyvern API-key
-adapter under tests/cloud/) subclasses AuthPortContract and overrides the four
-hooks; every behavioral guarantee of the port — credential extraction from
-header/query/path, authentication, org-vs-session authorization, clean rejection,
-no upstream credential leak, and token redaction — is asserted once, here.
-"""
-
-from __future__ import annotations
-
-import json
-from types import SimpleNamespace
-from typing import Mapping
-
-import pytest
-from structlog.testing import capture_logs
-
-from skyvern.proxy.adapters.memory import (
- ForwardAllEventPolicy,
- InMemorySessionRegistry,
- InMemoryUpstreamBrowser,
- NoOpMetrics,
- StaticKeyAuth,
-)
-from skyvern.proxy.adapters.websocket_server import CdpProxyServer
-from skyvern.proxy.core.session import Principal, ProxySession, ResolvedSession, SessionResolution
-from skyvern.proxy.ports import AuthPort, UpstreamConnection
-
-SESSION_ID = "s1"
-
-
-class FakeClientWebSocket:
- def __init__(self, path: str, headers: Mapping[str, str] | None = None) -> None:
- self.request = SimpleNamespace(headers=dict(headers or {}), path=path)
- self.sent: list[str] = []
- self.close_code: int | None = None
- self.close_reason: str | None = None
-
- def __aiter__(self) -> FakeClientWebSocket:
- return self
-
- async def __anext__(self) -> str:
- raise StopAsyncIteration
-
- async def send(self, raw: str) -> None:
- self.sent.append(raw)
-
- async def close(self, code: int = 1000, reason: str = "") -> None:
- self.close_code = code
- self.close_reason = reason
-
-
-class RecordingUpstreamBrowser(InMemoryUpstreamBrowser):
- def __init__(self) -> None:
- self.sessions: list[ProxySession] = []
-
- async def connect(self, session: ProxySession) -> UpstreamConnection:
- self.sessions.append(session)
- return await super().connect(session)
-
-
-class AuthPortContract:
- def make_auth(self) -> AuthPort:
- raise NotImplementedError
-
- def valid_credential(self) -> str:
- raise NotImplementedError
-
- def credential_org_id(self) -> str:
- raise NotImplementedError
-
- def invalid_credential(self) -> str:
- raise NotImplementedError
-
- def _server_for_org(self, org_id: str) -> tuple[CdpProxyServer, RecordingUpstreamBrowser]:
- upstream = RecordingUpstreamBrowser()
- sessions = InMemorySessionRegistry()
- sessions.put(
- ResolvedSession(
- session_id=SESSION_ID,
- upstream_adapter="memory",
- upstream_ws_url="ws://upstream.internal:9222/x",
- organization_id=org_id,
- connect_headers={"x-routing": "operator-only"},
- )
- )
- server = CdpProxyServer(
- upstream=upstream, # type: ignore[arg-type]
- sessions=sessions,
- auth=self.make_auth(),
- metrics=NoOpMetrics(),
- event_policy=ForwardAllEventPolicy(),
- )
- return server, upstream
-
- @pytest.mark.asyncio
- async def test_authenticate_valid_credential_returns_principal_for_its_org(self) -> None:
- principal = await self.make_auth().authenticate({"x-api-key": self.valid_credential()})
- assert principal is not None
- assert principal.organization_id == self.credential_org_id()
-
- @pytest.mark.asyncio
- async def test_authenticate_invalid_credential_returns_none(self) -> None:
- assert await self.make_auth().authenticate({"x-api-key": self.invalid_credential()}) is None
-
- @pytest.mark.asyncio
- async def test_authenticate_missing_credential_returns_none(self) -> None:
- assert await self.make_auth().authenticate({}) is None
-
- def test_authorize_owning_org_is_allowed(self) -> None:
- principal = Principal(principal_id="p", organization_id=self.credential_org_id())
- resolution = SessionResolution.active(
- ResolvedSession(
- session_id=SESSION_ID,
- upstream_adapter="memory",
- upstream_ws_url="ws://x",
- organization_id=self.credential_org_id(),
- )
- )
- assert self.make_auth().authorize(principal, resolution) is True
-
- def test_authorize_foreign_org_is_rejected(self) -> None:
- principal = Principal(principal_id="p", organization_id=self.credential_org_id())
- resolution = SessionResolution.active(
- ResolvedSession(
- session_id=SESSION_ID,
- upstream_adapter="memory",
- upstream_ws_url="ws://x",
- organization_id="o_someone_else",
- )
- )
- assert self.make_auth().authorize(principal, resolution) is False
-
- def test_authorize_unknown_session_is_rejected(self) -> None:
- principal = Principal(principal_id="p", organization_id=self.credential_org_id())
- assert self.make_auth().authorize(principal, SessionResolution.unknown()) is False
-
- @pytest.mark.asyncio
- async def test_header_credential_authenticates_and_dials(self) -> None:
- server, upstream = self._server_for_org(self.credential_org_id())
- ws = FakeClientWebSocket(path=f"/{SESSION_ID}", headers={"x-api-key": self.valid_credential()})
- await server._handle_client(ws) # type: ignore[arg-type]
- assert ws.close_code is None
- assert len(upstream.sessions) == 1
-
- @pytest.mark.asyncio
- async def test_query_credential_authenticates_and_dials(self) -> None:
- server, upstream = self._server_for_org(self.credential_org_id())
- ws = FakeClientWebSocket(path=f"/{SESSION_ID}?token={self.valid_credential()}")
- await server._handle_client(ws) # type: ignore[arg-type]
- assert ws.close_code is None
- assert len(upstream.sessions) == 1
-
- @pytest.mark.asyncio
- async def test_path_credential_authenticates_and_dials(self) -> None:
- server, upstream = self._server_for_org(self.credential_org_id())
- ws = FakeClientWebSocket(path=f"/key/{self.valid_credential()}/{SESSION_ID}")
- await server._handle_client(ws) # type: ignore[arg-type]
- assert ws.close_code is None
- assert len(upstream.sessions) == 1
-
- @pytest.mark.asyncio
- async def test_invalid_credential_closes_cleanly_before_dial(self) -> None:
- server, upstream = self._server_for_org(self.credential_org_id())
- ws = FakeClientWebSocket(path=f"/{SESSION_ID}", headers={"x-api-key": self.invalid_credential()})
- await server._handle_client(ws) # type: ignore[arg-type]
- assert ws.close_code == 4401
- assert not upstream.sessions
-
- @pytest.mark.asyncio
- async def test_wrong_org_gets_uniform_4404_before_dial(self) -> None:
- server, upstream = self._server_for_org("o_a_different_org")
- ws = FakeClientWebSocket(path=f"/{SESSION_ID}", headers={"x-api-key": self.valid_credential()})
- await server._handle_client(ws) # type: ignore[arg-type]
- assert (ws.close_code, ws.close_reason) == (4404, "unknown session")
- assert not upstream.sessions
-
- @pytest.mark.asyncio
- async def test_wrong_org_is_indistinguishable_from_unknown_session(self) -> None:
- foreign_server, foreign_upstream = self._server_for_org("o_a_different_org")
- foreign_ws = FakeClientWebSocket(path=f"/{SESSION_ID}", headers={"x-api-key": self.valid_credential()})
- await foreign_server._handle_client(foreign_ws) # type: ignore[arg-type]
-
- unknown_server, unknown_upstream = self._server_for_org(self.credential_org_id())
- unknown_ws = FakeClientWebSocket(path="/s_missing", headers={"x-api-key": self.valid_credential()})
- await unknown_server._handle_client(unknown_ws) # type: ignore[arg-type]
-
- assert (foreign_ws.close_code, foreign_ws.close_reason) == (unknown_ws.close_code, unknown_ws.close_reason)
- assert not foreign_upstream.sessions
- assert not unknown_upstream.sessions
-
- @pytest.mark.asyncio
- async def test_client_credential_is_never_forwarded_upstream(self) -> None:
- server, upstream = self._server_for_org(self.credential_org_id())
- ws = FakeClientWebSocket(
- path=f"/{SESSION_ID}?token={self.valid_credential()}",
- headers={"x-api-key": self.valid_credential()},
- )
- await server._handle_client(ws) # type: ignore[arg-type]
- assert len(upstream.sessions) == 1
- dialed = upstream.sessions[0]
- blob = dialed.upstream_ws_url + "".join(f"{k}={v}" for k, v in dialed.connect_headers.items())
- assert self.valid_credential() not in blob
-
- @pytest.mark.asyncio
- async def test_credential_value_is_redacted_from_logs(self) -> None:
- secret = self.valid_credential()
- with capture_logs() as logs:
- server, _ = self._server_for_org(self.credential_org_id())
- ok_ws = FakeClientWebSocket(path=f"/{SESSION_ID}", headers={"x-api-key": secret})
- await server._handle_client(ok_ws) # type: ignore[arg-type]
- bad_server, _ = self._server_for_org(self.credential_org_id())
- bad_ws = FakeClientWebSocket(path=f"/{SESSION_ID}", headers={"x-api-key": self.invalid_credential()})
- await bad_server._handle_client(bad_ws) # type: ignore[arg-type]
- assert secret not in json.dumps(logs, default=str)
-
-
-class TestStaticKeyAuthContract(AuthPortContract):
- _VALID = "static-key-abc123"
- _ORG = "o_static"
-
- def make_auth(self) -> AuthPort:
- return StaticKeyAuth({self._VALID: Principal(principal_id=self._ORG, organization_id=self._ORG)})
-
- def valid_credential(self) -> str:
- return self._VALID
-
- def credential_org_id(self) -> str:
- return self._ORG
-
- def invalid_credential(self) -> str:
- return "static-key-does-not-exist"
diff --git a/tests/unit/proxy/test_client_compat.py b/tests/unit/proxy/test_client_compat.py
deleted file mode 100644
index e89ca6a3e..000000000
--- a/tests/unit/proxy/test_client_compat.py
+++ /dev/null
@@ -1,239 +0,0 @@
-"""Real CDP-client compatibility through the proxy in pass-through mode.
-
-Runs a real CdpProxyServer over a real headless Chrome (the local-Chrome upstream
-adapter) and drives it with the clients customers actually point at us: Playwright for
-Python, Playwright for JS, puppeteer, and a raw CDP websocket client. Each one really
-opens a page and reads it back, so "compatible" means a client's own protocol code was
-satisfied end to end — not that a hand-rolled fake of it was.
-
-Hermetic on purpose: the page under test is a data: URL, so a run touches no network
-and no third-party site, and the only browser involved is the pinned one the CI job
-installs. Nothing here sleeps to synchronize; every wait is a bounded wait-for-condition.
-
-Missing Chrome/node/JS clients skip locally and FAIL in CI (see _e2e_gate) — this suite
-is the merge gate for the proxy, and a gate that skips itself is not a gate.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import contextlib
-import json
-import urllib.request
-from typing import Any, AsyncIterator
-
-import pytest
-import websockets
-
-from skyvern.proxy.adapters.local_chrome import LocalChromeUpstreamBrowser, find_local_chrome_executable
-from skyvern.proxy.adapters.memory import AllowAllAuth, ForwardAllEventPolicy, InMemorySessionRegistry, NoOpMetrics
-from skyvern.proxy.adapters.websocket_server import CdpProxyServer
-from skyvern.proxy.core.session import ResolvedSession
-from tests.unit.proxy._e2e_gate import (
- JS_CLIENT_DIR,
- find_node,
- js_clients_installed,
- playwright_python_available,
- require,
- run_js_client,
-)
-
-SESSION_ID = "s1"
-
-# The one page every client loads. A data: URL keeps the suite hermetic and keeps any
-# real site's name out of an OSS-synced test.
-TEST_PAGE = "data:text/html,<title>proxied</title><h1>hello</h1>"
-
-FRAME_TIMEOUT_SECONDS = 30.0
-
-# Resolves on the page's own load event (or immediately, if it already fired), so the
-# raw client synchronizes on the browser rather than on the clock.
-_TITLE_AFTER_LOAD = """
-new Promise((resolve) => {
- if (document.readyState === 'complete') { resolve(document.title); return; }
- addEventListener('load', () => resolve(document.title), {once: true});
-})
-"""
-
-
[email protected](autouse=True)
-def _require_chrome() -> None:
- require(find_local_chrome_executable() is not None, "no local Chrome/Chromium for the upstream adapter")
-
-
-def _require_js_clients() -> None:
- require(find_node() is not None, "node is not installed")
- require(js_clients_installed(), f"pinned JS clients not installed: run `npm ci` in {JS_CLIENT_DIR}")
-
-
-async def _running_proxy() -> AsyncIterator[int]:
- sessions = InMemorySessionRegistry()
- sessions.put(
- ResolvedSession(session_id=SESSION_ID, upstream_adapter="local-chrome", upstream_ws_url="ws://ignored")
- )
- server = CdpProxyServer(
- upstream=LocalChromeUpstreamBrowser(),
- sessions=sessions,
- auth=AllowAllAuth(),
- metrics=NoOpMetrics(),
- event_policy=ForwardAllEventPolicy(),
- )
- async with websockets.serve(
- server._handle_client, "127.0.0.1", 0, max_size=None, process_request=server._process_request
- ) as ws_server:
- yield ws_server.sockets[0].getsockname()[1]
-
-
-def _http_get_json(url: str) -> object:
- with urllib.request.urlopen(url, timeout=30) as response: # noqa: S310 - localhost test fixture
- return json.loads(response.read().decode())
-
-
- ("discovery_path", "client"),
- [
- ("/json/version", "generic-cdp"), # a client asking for the browser version
- ("/json/version/", "playwright"), # playwright appends /json/version/ (chromium.js)
- ("/json/list", "raw-cdp"), # a raw client enumerating targets
- ],
-)
-async def test_discovery_handshake_reaches_upstream(discovery_path: str, client: str) -> None:
- async with _running_proxy() as port:
- discovery = await asyncio.to_thread(_http_get_json, f"http://127.0.0.1:{port}/{SESSION_ID}{discovery_path}")
- entry = discovery if isinstance(discovery, dict) else discovery[0]
- ws_url = entry["webSocketDebuggerUrl"]
- # Proxy-scoped: points back at the proxy, never at the upstream browser.
- assert ws_url == f"ws://127.0.0.1:{port}/{SESSION_ID}"
- async with websockets.connect(ws_url, max_size=None) as ws:
- await ws.send(json.dumps({"id": 1, "method": "Browser.getVersion"}))
- response = json.loads(await asyncio.wait_for(ws.recv(), timeout=FRAME_TIMEOUT_SECONDS))
- assert response["id"] == 1
- assert "product" in response["result"] # a real upstream Chrome answered
-
-
-async def test_real_playwright_python_drives_a_page_through_the_proxy() -> None:
- # require() first: importorskip alone would skip straight past the E2E gate, which
- # is the failure this suite exists to make impossible.
- require(playwright_python_available(), "playwright for python is not installed")
- playwright_api = pytest.importorskip("playwright.async_api")
- async with _running_proxy() as port:
- async with playwright_api.async_playwright() as p:
- browser = await p.chromium.connect_over_cdp(f"http://127.0.0.1:{port}/{SESSION_ID}")
- try:
- context = browser.contexts[0] if browser.contexts else await browser.new_context()
- page = await context.new_page()
- await page.goto(TEST_PAGE)
- assert await page.title() == "proxied"
- finally:
- await browser.close()
-
-
- ("script", "client", "scheme"),
- [
- # playwright-js resolves discovery relative to the endpoint, so it takes the
- # http URL; puppeteer must be given the ws endpoint (see the browserURL test).
- ("playwright_client.mjs", "playwright-js", "http"),
- ("puppeteer_client.mjs", "puppeteer", "ws"),
- ],
-)
-async def test_real_js_client_drives_a_page_through_the_proxy(script: str, client: str, scheme: str) -> None:
- """The pinned JS clients run their own protocol code against the proxy, rather than
- a Python imitation of it — which is the only way a claim of compatibility means
- anything to someone pointing that client at us."""
- _require_js_clients()
- async with _running_proxy() as port:
- result = await asyncio.to_thread(run_js_client, script, f"{scheme}://127.0.0.1:{port}/{SESSION_ID}")
-
- assert result.returncode == 0, f"{client} failed:\nstdout={result.stdout}\nstderr={result.stderr}"
- payload = json.loads(result.stdout.strip().splitlines()[-1])
- assert payload == {"client": client, "title": "proxied", "heading": "hello"}
-
-
-async def test_puppeteer_browser_url_cannot_address_a_path_scoped_session() -> None:
- """A proven client limitation, pinned so nobody mistakes it for a proxy bug.
-
- puppeteer resolves discovery as `new URL('/json/version', browserURL)`
- (BrowserConnector.js) — an absolute path, so the session prefix in the URL is
- discarded and it asks the proxy root for a session it cannot name. Callers must use
- browserWSEndpoint, which keeps the path. If a future puppeteer resolves discovery
- relatively, this test fails and browserURL becomes supportable.
- """
- _require_js_clients()
- async with _running_proxy() as port:
- result = await asyncio.to_thread(run_js_client, "puppeteer_client.mjs", f"http://127.0.0.1:{port}/{SESSION_ID}")
-
- # Asserting the REASON, not just a non-zero exit: any crash would satisfy "it
- # failed", including one that has nothing to do with the discovery path. A future
- # puppeteer that resolves discovery relatively lands here as a clear signal that
- # browserURL has become supportable, rather than as a vague red.
- assert result.returncode != 0, (
- "puppeteer browserURL unexpectedly reached the session — if puppeteer now resolves "
- f"/json/version relatively, browserURL is supportable and this test should go.\nstdout={result.stdout}"
- )
- assert "TargetCloseError" in result.stderr or "Protocol error" in result.stderr, (
- f"puppeteer browserURL failed for an unexpected reason (expected the proxy to close a "
- f"session-less connection):\nstdout={result.stdout}\nstderr={result.stderr}"
- )
-
-
-async def _command(
- ws: websockets.ClientConnection,
- command_id: int,
- method: str,
- params: dict[str, Any] | None = None,
- session_id: str | None = None,
-) -> dict[str, Any]:
- """Sends one CDP command and returns its response, skipping the events that arrive
- first. Waits on the matching id rather than on a duration."""
- message: dict[str, Any] = {"id": command_id, "method": method}
- if params is not None:
- message["params"] = params
- if session_id is not None:
- message["sessionId"] = session_id
- await ws.send(json.dumps(message))
-
- async def until_response() -> dict[str, Any]:
- while True:
- frame: dict[str, Any] = json.loads(await ws.recv())
- if frame.get("id") == command_id:
- return frame
-
- return await asyncio.wait_for(until_response(), timeout=FRAME_TIMEOUT_SECONDS)
-
-
-async def test_raw_cdp_client_drives_a_page_through_the_proxy() -> None:
- """The flat-session path no client library exposes: create a target, attach to it,
- and evaluate in the attached session. Exercises the proxy's own session routing and
- request-id remapping (SKY-12500), which Playwright otherwise hides."""
- async with _running_proxy() as port:
- async with websockets.connect(f"ws://127.0.0.1:{port}/{SESSION_ID}", max_size=None) as ws:
- # Opened blank and navigated explicitly: createTarget answers as soon as the
- # target exists, so a target created straight at the page can still be on
- # about:blank (readyState already 'complete', title already '') when the
- # evaluate below lands. Navigating separately means the document under test
- # is the committed one, with no ordering left to get lucky with.
- created = await _command(ws, 1, "Target.createTarget", {"url": "about:blank"})
- target_id = created["result"]["targetId"]
-
- attached = await _command(ws, 2, "Target.attachToTarget", {"targetId": target_id, "flatten": True})
- cdp_session = attached["result"]["sessionId"]
-
- await _command(ws, 3, "Page.enable", session_id=cdp_session)
- await _command(ws, 4, "Page.navigate", {"url": TEST_PAGE}, session_id=cdp_session)
-
- evaluated = await _command(
- ws,
- 5,
- "Runtime.evaluate",
- {"expression": _TITLE_AFTER_LOAD, "awaitPromise": True},
- session_id=cdp_session,
- )
-
- assert evaluated["result"]["result"]["value"] == "proxied"
diff --git a/tests/unit/proxy/test_contract_teeth.py b/tests/unit/proxy/test_contract_teeth.py
deleted file mode 100644
index e29f42903..000000000
--- a/tests/unit/proxy/test_contract_teeth.py
+++ /dev/null
@@ -1,674 +0,0 @@
-"""Proof that the reusable port-contract suites actually reject a bad adapter.
-
-A contract suite that asserts nothing passes exactly as green as one that asserts
-everything, so "the adapter passes its contract" is worth only as much as the
-contract's ability to fail. Every adapter here is deliberately broken in exactly one
-way, and MUTANTS pairs each with the clause that must catch it.
-
-Two properties are enforced, and the second is the one that makes the first mean
-something:
-
-* every mutant's clause catches it (test_clause_catches_its_mutant), and
-* every clause of every covered contract HAS a mutant (test_every_claimed_clause_has_a_
- mutant). Without that, a clause with no mutant could be gutted or deleted with this
- file still green — the covered subset would look like the whole.
-
-Covered contracts are listed in TEETH_COVERED_CONTRACTS. AuthPortContract and
-MetricsPortContract are NOT covered here: their clauses drive a whole proxy server
-rather than swapping one adapter, so mutating them needs a different harness than this
-file's. That is a real gap, named rather than papered over.
-
-These adapters are fixtures, never shipped: nothing outside this file imports them, and
-the leading underscore keeps pytest from collecting them as suites of their own.
-"""
-
-from __future__ import annotations
-
-import inspect
-from dataclasses import dataclass
-from typing import Callable, Sequence
-
-import pytest
-
-from skyvern.proxy.adapters.memory import InMemorySessionRegistry, InMemoryUpstreamConnection
-from skyvern.proxy.core.frames import KNOWN_CDP_DOMAINS, LIFECYCLE_EVENTS, CdpCommand, CdpEvent
-from skyvern.proxy.core.policy import FORWARD, Drop, DropReason, PolicyDecision, Rewrite
-from skyvern.proxy.core.session import (
- ProxySession,
- ResolvedSession,
- SessionResolution,
- SessionResolutionStatus,
-)
-from skyvern.proxy.ports import UpstreamBrowserPort, UpstreamConnection
-from tests.unit.proxy.test_event_policy_contract import EventPolicyPortContract
-from tests.unit.proxy.test_session_registry_contract import SessionRegistryContract
-from tests.unit.proxy.test_upstream_browser_port_contract import (
- DialingUpstreamBrowserPortContract,
- UpstreamBrowserPortContract,
-)
-
-SECRET_UPSTREAM_URL = "ws://upstream.internal:9222/devtools/browser/abc?token=secret-token"
-
-
-def _resolve_clause(contract: object, clause: str) -> Callable[[], object]:
- """Look the clause up BEFORE it is run, so a deleted one cannot masquerade as a
- working one.
-
- Resolving inside the try below would turn a renamed or removed clause into an
- AttributeError that reads exactly like the contract rejecting the adapter — the
- suite would go green precisely because its assertion had stopped existing. That is
- the failure this whole file exists to make impossible, so it is checked first.
- """
- found = getattr(contract, clause, None)
- if not callable(found):
- raise AssertionError(
- f"{type(contract).__name__} has no clause {clause!r} — a clause that no longer exists cannot catch anything"
- )
- return found
-
-
-async def _run_clause(contract: object, clause: str) -> None:
- outcome = _resolve_clause(contract, clause)()
- if inspect.isawaitable(outcome):
- await outcome
-
-
-async def _assert_contract_catches(contract: object, clause: str) -> None:
- """`clause` must fail against the broken adapter behind `contract`."""
- invoke = _resolve_clause(contract, clause)
- try:
- outcome = invoke()
- if inspect.isawaitable(outcome):
- await outcome
- except pytest.skip.Exception as skipped:
- raise AssertionError(f"{clause} SKIPPED instead of failing — an unrun contract is a false green") from skipped
- except (Exception, pytest.fail.Exception):
- return
- raise AssertionError(f"{clause} PASSED against a deliberately broken adapter — the clause has no teeth")
-
-
-async def _assert_contract_allows(contract: object, clause: str) -> None:
- """`clause` must pass — used to show a narrower clause misses what a wider one catches."""
- await _run_clause(contract, clause)
-
-
-# --- EventPolicyPort mutants -------------------------------------------------------
-
-
-class _CompliantPolicy:
- """The baseline each EventPolicy mutant below breaks in exactly one way."""
-
- def decide(self, event: CdpEvent, session: ProxySession) -> PolicyDecision:
- return FORWARD
-
- def observe_command(self, command: CdpCommand, session: ProxySession) -> None:
- return None
-
- def forget(self, session_id: str) -> None:
- return None
-
-
-class _ReturnsNonDecision(_CompliantPolicy):
- """Returns something that is not a PolicyDecision at all."""
-
- def decide(self, event: CdpEvent, session: ProxySession) -> PolicyDecision:
- return None # type: ignore[return-value]
-
-
-class _DropsLifecycleEvents(_CompliantPolicy):
- """Violates the never-droppable lifecycle rule (SKY-12500): a dropped attach or
- detach strands that session's owner."""
-
- def decide(self, event: CdpEvent, session: ProxySession) -> PolicyDecision:
- if event.method in LIFECYCLE_EVENTS:
- return Drop(DropReason.POLICY)
- return FORWARD
-
-
-class _ThrottlesLifecycleAfterABurst(_CompliantPolicy):
- """Forwards the first few lifecycle events, then throttles them.
-
- Invisible to a single-event check — which is exactly why the contract bursts.
- """
-
- def __init__(self) -> None:
- self._seen = 0
-
- def decide(self, event: CdpEvent, session: ProxySession) -> PolicyDecision:
- if event.method in LIFECYCLE_EVENTS:
- self._seen += 1
- if self._seen > 10:
- return Drop(DropReason.THROTTLED)
- return FORWARD
-
-
-class _StripsLifecycleIdentity(_CompliantPolicy):
- """Delivers every lifecycle event, with the sessionId the router needs removed.
-
- Never drops anything, so a contract that only rejects Drop calls this compliant —
- while the client is stranded exactly as if the event had been dropped.
- """
-
- def decide(self, event: CdpEvent, session: ProxySession) -> PolicyDecision:
- if event.method in LIFECYCLE_EVENTS:
- stripped = {key: value for key, value in (event.params or {}).items() if key != "sessionId"}
- return Rewrite(CdpEvent(method=event.method, params=stripped, session_id=event.session_id))
- return FORWARD
-
-
-class _FreeFormDropReason(_CompliantPolicy):
- """Reason taken from the wire instead of the closed set: one metric label per
- distinct CDP method, which is the cardinality blowup SKY-12510 bounds."""
-
- def decide(self, event: CdpEvent, session: ProxySession) -> PolicyDecision:
- if event.method in LIFECYCLE_EVENTS:
- return FORWARD
- return Drop(f"unbounded-{event.method}") # type: ignore[arg-type]
-
-
-class _RejectsOddCommands(_CompliantPolicy):
- """Trusts the client's command to be well-formed — a client can send anything."""
-
- def observe_command(self, command: CdpCommand, session: ProxySession) -> None:
- domain = command.method.partition(".")[0]
- if domain not in KNOWN_CDP_DOMAINS:
- raise ValueError(f"unknown domain: {domain}")
-
-
-class _NonIdempotentForget(_CompliantPolicy):
- """forget() is a teardown path; a second call must not explode."""
-
- def __init__(self) -> None:
- self._forgotten: set[str] = set()
-
- def forget(self, session_id: str) -> None:
- if session_id in self._forgotten:
- raise RuntimeError("forget() called twice")
- self._forgotten.add(session_id)
-
-
-class _PoisonedAfterForget(_CompliantPolicy):
- """forget() leaves the port unusable, so a late event for a torn-down session
- raises instead of being decided."""
-
- def __init__(self) -> None:
- self._forgotten = False
-
- def decide(self, event: CdpEvent, session: ProxySession) -> PolicyDecision:
- if self._forgotten:
- raise RuntimeError("decide() after forget()")
- return FORWARD
-
- def forget(self, session_id: str) -> None:
- self._forgotten = True
-
-
-class _BrokenPolicyContract(EventPolicyPortContract):
- """Binds one broken policy instance to the contract. The instance is shared across
- make_policy() calls because a mutant like the burst-throttler accumulates state the
- clause is meant to notice."""
-
- def __init__(self, policy: object) -> None:
- self._policy = policy
-
- def make_policy(self) -> object: # type: ignore[override]
- return self._policy
-
-
-# --- UpstreamBrowserPort mutants ---------------------------------------------------
-
-
-class _NotAConnection:
- """Missing receive(), so it does not satisfy the UpstreamConnection protocol."""
-
- async def send(self, raw: str) -> None:
- return None
-
- async def close(self) -> None:
- return None
-
-
-class _NoMatchingResponseConnection(InMemoryUpstreamConnection):
- """Answers, but never with the id that was asked for."""
-
- async def receive(self) -> str:
- await super().receive()
- return '{"id": 999, "result": {}}'
-
-
-class _NonIdempotentCloseConnection(InMemoryUpstreamConnection):
- """close() is a teardown path: a second call must not explode."""
-
- async def close(self) -> None:
- if self._closed:
- raise RuntimeError("close() called twice")
- await super().close()
-
-
-class _SilentReceiveAfterCloseConnection(InMemoryUpstreamConnection):
- """Returns an empty frame after close instead of raising, so a relay loop reads
- silence forever rather than learning the upstream is gone."""
-
- async def receive(self) -> str:
- if self._closed:
- return ""
- return await super().receive()
-
-
-class _SilentSendAfterCloseConnection(InMemoryUpstreamConnection):
- """Accepts writes into the void after close instead of raising."""
-
- async def send(self, raw: str) -> None:
- if self._closed:
- return None
- await super().send(raw)
-
-
-class _RawTransportErrorConnection:
- """Leaks the transport's own exception once the remote goes away, instead of the
- port taxonomy every caller branches on."""
-
- def __init__(self, break_send: bool, break_receive: bool) -> None:
- self.broken = False
- self._break_send = break_send
- self._break_receive = break_receive
-
- async def send(self, raw: str) -> None:
- if self.broken and self._break_send:
- raise ConnectionResetError("raw transport error")
-
- async def receive(self) -> str:
- if self.broken and self._break_receive:
- raise ConnectionResetError("raw transport error")
- return '{"id": 42, "result": {}}'
-
- async def close(self) -> None:
- return None
-
-
-class _FixedConnectionBrowser:
- def __init__(self, connection: object) -> None:
- self._connection = connection
-
- async def connect(self, session: ProxySession) -> UpstreamConnection:
- return self._connection # type: ignore[return-value]
-
-
-class _ConnectionFactoryBrowser:
- def __init__(self, factory: Callable[[], object]) -> None:
- self._factory = factory
-
- async def connect(self, session: ProxySession) -> UpstreamConnection:
- return self._factory() # type: ignore[return-value]
-
-
-class _RefusingBrowser:
- """Leaks the raw dial failure instead of the port's UpstreamConnectError taxonomy."""
-
- async def connect(self, session: ProxySession) -> UpstreamConnection:
- raise ConnectionRefusedError("connection refused")
-
-
-class _BrokenUpstreamContract(UpstreamBrowserPortContract):
- def __init__(self, port: object) -> None:
- self._port = port
-
- def make_port(self) -> UpstreamBrowserPort:
- return self._port # type: ignore[return-value]
-
-
-class _BrokenDialingContract(DialingUpstreamBrowserPortContract):
- """A dialing adapter that leaks raw transport errors rather than the taxonomy."""
-
- def __init__(self, *, refusing: bool = False, break_send: bool = False, break_receive: bool = False) -> None:
- self._refusing = refusing
- self._connection = _RawTransportErrorConnection(break_send=break_send, break_receive=break_receive)
-
- def make_port(self) -> UpstreamBrowserPort:
- return _FixedConnectionBrowser(self._connection) # type: ignore[return-value]
-
- def make_unreachable_port_and_session(self) -> tuple[UpstreamBrowserPort, ProxySession]:
- return _RefusingBrowser(), self.make_session() # type: ignore[return-value]
-
- async def break_remote(self, connection: UpstreamConnection) -> None:
- self._connection.broken = True
-
-
-# --- SessionRegistryPort mutants ---------------------------------------------------
-
-
-def _seed_memory_registry(
- active: Sequence[ResolvedSession],
- pending: Sequence[str],
- closed: Sequence[str],
- expired: Sequence[str],
-) -> InMemorySessionRegistry:
- registry = InMemorySessionRegistry()
- for session in active:
- registry.put(session)
- for session_id in pending:
- registry.mark_pending(session_id)
- for session_id in closed:
- registry.mark_closed(session_id)
- for session_id in expired:
- registry.mark_expired(session_id)
- return registry
-
-
-@dataclass(frozen=True)
-class _LeakyResolution:
- """A registry's own resolution type that forgets to keep the upstream URL out of
- its repr, so one log line leaks the session token in the URL's query."""
-
- status: SessionResolutionStatus
- session: ResolvedSession | None = None
- organization_id: str | None = None
- expires_in_seconds: float | None = None
-
- def __repr__(self) -> str:
- upstream = self.session.upstream_ws_url if self.session else None
- return f"_LeakyResolution(status={self.status.value}, upstream={upstream})"
-
-
-class _LeakyRegistry:
- """Correct routing, leaky repr — exactly one clause broken."""
-
- def __init__(self, inner: InMemorySessionRegistry) -> None:
- self._inner = inner
-
- async def resolve(self, session_id: str) -> SessionResolution:
- resolution = await self._inner.resolve(session_id)
- return _LeakyResolution( # type: ignore[return-value]
- resolution.status, resolution.session, resolution.organization_id, resolution.expires_in_seconds
- )
-
- async def invalidate(self, session_id: str) -> None:
- await self._inner.invalidate(session_id)
-
-
-class _RoutesStatusRegistry:
- """Hands back full routing for a session the control plane says is not usable, so a
- client reaches a browser that is pending, closed or already expired."""
-
- def __init__(self, inner: InMemorySessionRegistry, status: SessionResolutionStatus) -> None:
- self._inner = inner
- self._status = status
-
- async def resolve(self, session_id: str) -> SessionResolution:
- resolution = await self._inner.resolve(session_id)
- if resolution.status is self._status:
- return SessionResolution.active(
- ResolvedSession(
- session_id=session_id,
- upstream_adapter="websocket",
- upstream_ws_url=SECRET_UPSTREAM_URL,
- organization_id="o_1",
- )
- )
- return resolution
-
- async def invalidate(self, session_id: str) -> None:
- await self._inner.invalidate(session_id)
-
-
-class _RoutesUnknownRegistry(_RoutesStatusRegistry):
- def __init__(self, inner: InMemorySessionRegistry) -> None:
- super().__init__(inner, SessionResolutionStatus.UNKNOWN)
-
-
-class _LosesRoutingOnActiveRegistry:
- """Reports ACTIVE but without the routing the caller needs to dial anything."""
-
- def __init__(self, inner: InMemorySessionRegistry) -> None:
- self._inner = inner
-
- async def resolve(self, session_id: str) -> SessionResolution:
- resolution = await self._inner.resolve(session_id)
- if resolution.status is SessionResolutionStatus.ACTIVE:
- return SessionResolution(SessionResolutionStatus.ACTIVE, None, resolution.organization_id)
- return resolution
-
- async def invalidate(self, session_id: str) -> None:
- await self._inner.invalidate(session_id)
-
-
-class _NonIdempotentInvalidateRegistry:
- """invalidate() is a teardown path fired on close; a repeat must not explode."""
-
- def __init__(self, inner: InMemorySessionRegistry) -> None:
- self._inner = inner
- self._invalidated: set[str] = set()
-
- async def resolve(self, session_id: str) -> SessionResolution:
- return await self._inner.resolve(session_id)
-
- async def invalidate(self, session_id: str) -> None:
- if session_id in self._invalidated:
- raise RuntimeError("invalidate() called twice")
- self._invalidated.add(session_id)
-
-
-class _BrokenRegistryContract(SessionRegistryContract):
- def __init__(self, wrap: Callable[[InMemorySessionRegistry], object]) -> None:
- self._wrap = wrap
-
- def make_registry(
- self,
- *,
- active: tuple[ResolvedSession, ...] = (),
- pending: tuple[str, ...] = (),
- closed: tuple[str, ...] = (),
- expired: tuple[str, ...] = (),
- ) -> object: # type: ignore[override]
- return self._wrap(_seed_memory_registry(active, pending, closed, expired))
-
-
-# --- the registry ------------------------------------------------------------------
-
-
-@dataclass(frozen=True)
-class Mutant:
- """One adapter broken in exactly one way, and the clause that must catch it."""
-
- contract: type
- clause: str
- make: Callable[[], object]
-
-
-MUTANTS: tuple[Mutant, ...] = (
- # EventPolicyPortContract
- Mutant(
- EventPolicyPortContract,
- "test_decide_returns_a_policy_decision",
- lambda: _BrokenPolicyContract(_ReturnsNonDecision()),
- ),
- Mutant(
- EventPolicyPortContract,
- "test_lifecycle_events_are_never_dropped",
- lambda: _BrokenPolicyContract(_DropsLifecycleEvents()),
- ),
- Mutant(
- EventPolicyPortContract,
- "test_lifecycle_survives_a_burst_whatever_the_rate_budget",
- lambda: _BrokenPolicyContract(_ThrottlesLifecycleAfterABurst()),
- ),
- Mutant(
- EventPolicyPortContract,
- "test_lifecycle_events_keep_their_routing_identity",
- lambda: _BrokenPolicyContract(_StripsLifecycleIdentity()),
- ),
- Mutant(
- EventPolicyPortContract,
- "test_drop_decisions_carry_a_closed_set_reason",
- lambda: _BrokenPolicyContract(_FreeFormDropReason()),
- ),
- Mutant(
- EventPolicyPortContract,
- "test_observe_command_accepts_whatever_a_client_sends",
- lambda: _BrokenPolicyContract(_RejectsOddCommands()),
- ),
- Mutant(
- EventPolicyPortContract,
- "test_forget_is_idempotent_and_safe_for_unseen_sessions",
- lambda: _BrokenPolicyContract(_NonIdempotentForget()),
- ),
- Mutant(
- EventPolicyPortContract,
- "test_decide_still_answers_after_the_session_is_forgotten",
- lambda: _BrokenPolicyContract(_PoisonedAfterForget()),
- ),
- # UpstreamBrowserPortContract
- Mutant(
- UpstreamBrowserPortContract,
- "test_connect_returns_open_connection",
- lambda: _BrokenUpstreamContract(_FixedConnectionBrowser(_NotAConnection())),
- ),
- Mutant(
- UpstreamBrowserPortContract,
- "test_command_receives_frame_with_matching_id",
- lambda: _BrokenUpstreamContract(_ConnectionFactoryBrowser(_NoMatchingResponseConnection)),
- ),
- Mutant(
- UpstreamBrowserPortContract,
- "test_send_after_close_raises_upstream_closed",
- lambda: _BrokenUpstreamContract(_ConnectionFactoryBrowser(_SilentSendAfterCloseConnection)),
- ),
- Mutant(
- UpstreamBrowserPortContract,
- "test_receive_after_close_raises_upstream_closed",
- lambda: _BrokenUpstreamContract(_ConnectionFactoryBrowser(_SilentReceiveAfterCloseConnection)),
- ),
- Mutant(
- UpstreamBrowserPortContract,
- "test_close_is_idempotent",
- lambda: _BrokenUpstreamContract(_ConnectionFactoryBrowser(_NonIdempotentCloseConnection)),
- ),
- # DialingUpstreamBrowserPortContract
- Mutant(
- DialingUpstreamBrowserPortContract,
- "test_failed_dial_raises_the_port_error_taxonomy",
- lambda: _BrokenDialingContract(refusing=True),
- ),
- Mutant(
- DialingUpstreamBrowserPortContract,
- "test_remote_close_surfaces_as_upstream_closed_on_receive",
- lambda: _BrokenDialingContract(break_receive=True),
- ),
- Mutant(
- DialingUpstreamBrowserPortContract,
- "test_remote_close_surfaces_as_upstream_closed_on_send",
- lambda: _BrokenDialingContract(break_send=True),
- ),
- # SessionRegistryContract
- Mutant(
- SessionRegistryContract,
- "test_unknown_session_is_an_explicit_negative",
- lambda: _BrokenRegistryContract(_RoutesUnknownRegistry),
- ),
- Mutant(
- SessionRegistryContract,
- "test_active_session_resolves_full_routing",
- lambda: _BrokenRegistryContract(_LosesRoutingOnActiveRegistry),
- ),
- Mutant(
- SessionRegistryContract,
- "test_pending_session_is_rejected_without_routing_data",
- lambda: _BrokenRegistryContract(lambda inner: _RoutesStatusRegistry(inner, SessionResolutionStatus.PENDING)),
- ),
- Mutant(
- SessionRegistryContract,
- "test_closed_session_is_rejected_without_routing_data",
- lambda: _BrokenRegistryContract(lambda inner: _RoutesStatusRegistry(inner, SessionResolutionStatus.CLOSED)),
- ),
- Mutant(
- SessionRegistryContract,
- "test_expired_session_is_rejected_without_routing_data",
- lambda: _BrokenRegistryContract(lambda inner: _RoutesStatusRegistry(inner, SessionResolutionStatus.EXPIRED)),
- ),
- Mutant(
- SessionRegistryContract,
- "test_resolution_repr_never_leaks_the_upstream_url",
- lambda: _BrokenRegistryContract(_LeakyRegistry),
- ),
- Mutant(
- SessionRegistryContract,
- "test_invalidate_is_safe_and_idempotent_for_unknown_sessions",
- lambda: _BrokenRegistryContract(_NonIdempotentInvalidateRegistry),
- ),
-)
-
-# The contracts this file claims to give teeth. Every clause each one declares must have
-# a mutant below — see test_every_claimed_clause_has_a_mutant.
-TEETH_COVERED_CONTRACTS: tuple[type, ...] = (
- EventPolicyPortContract,
- UpstreamBrowserPortContract,
- DialingUpstreamBrowserPortContract,
- SessionRegistryContract,
-)
-
-
-def _declared_clauses(contract: type) -> set[str]:
- """Clauses the contract declares ITSELF — an inherited one belongs to the parent
- that declared it and is covered by the parent's own mutants."""
- return {name for name in vars(contract) if name.startswith("test_")}
-
-
[email protected]("mutant", MUTANTS, ids=lambda m: f"{m.contract.__name__}.{m.clause}")
-async def test_clause_catches_its_mutant(mutant: Mutant) -> None:
- await _assert_contract_catches(mutant.make(), mutant.clause)
-
-
-def test_every_claimed_clause_has_a_mutant() -> None:
- """The guard that makes this file's claim true rather than partial.
-
- Without it, teeth cover whatever subset happened to get a mutant, and a clause added
- later — or one already here — can be gutted with everything still green.
- """
- for contract in TEETH_COVERED_CONTRACTS:
- declared = _declared_clauses(contract)
- covered = {mutant.clause for mutant in MUTANTS if mutant.contract is contract}
- assert declared == covered, (
- f"{contract.__name__}: clauses without a mutant: {sorted(declared - covered)}; "
- f"mutants for clauses that no longer exist: {sorted(covered - declared)}"
- )
-
-
-async def test_burst_clause_catches_lifecycle_throttling_a_single_event_check_misses() -> None:
- # The narrow clause passes: the first attach/detach still gets through. Only the
- # burst finds it, which is what earns that clause its place in the contract.
- contract = _BrokenPolicyContract(_ThrottlesLifecycleAfterABurst())
- await _assert_contract_allows(contract, "test_lifecycle_events_are_never_dropped")
- await _assert_contract_catches(contract, "test_lifecycle_survives_a_burst_whatever_the_rate_budget")
-
-
-async def test_identity_clause_catches_a_rewrite_no_drop_check_would_notice() -> None:
- # The stripping policy drops nothing, so the never-dropped clause is satisfied while
- # the client is stranded anyway. That gap is why the identity clause exists.
- contract = _BrokenPolicyContract(_StripsLifecycleIdentity())
- await _assert_contract_allows(contract, "test_lifecycle_events_are_never_dropped")
- await _assert_contract_catches(contract, "test_lifecycle_events_keep_their_routing_identity")
-
-
-async def test_a_deleted_clause_is_reported_rather_than_read_as_a_catch() -> None:
- """Deleting a clause is the cheapest way to make a broken adapter pass its contract.
-
- It must not be mistaken for the contract working: the lookup fails the same way a
- rejected adapter does, so without an explicit check this file goes green exactly
- when its assertions stop existing.
- """
- with pytest.raises(AssertionError, match="no clause"):
- await _assert_contract_catches(_BrokenPolicyContract(_CompliantPolicy()), "test_clause_someone_renamed_away")
-
-
-async def test_the_teeth_helper_itself_fails_when_an_adapter_is_actually_compliant() -> None:
- """The guard on the guard: if _assert_contract_catches treated a passing clause as
- caught, every test above would pass no matter how toothless the contracts got."""
- with pytest.raises(AssertionError, match="no teeth"):
- await _assert_contract_catches(
- _BrokenPolicyContract(_CompliantPolicy()), "test_lifecycle_events_are_never_dropped"
- )
diff --git a/tests/unit/proxy/test_denylist.py b/tests/unit/proxy/test_denylist.py
deleted file mode 100644
index d052b9927..000000000
--- a/tests/unit/proxy/test_denylist.py
+++ /dev/null
@@ -1,368 +0,0 @@
-"""Org-level CDP operation denylists (SKY-12538) on the interception seam.
-
-A denied command is answered with a deterministic synthesized CDP error under the
-client's own request id — never a silent drop — and never reaches the browser.
-The pattern grammar is a superset of the policy packs' exact method names: exact
-("Page.navigate") or trailing-'*' prefix ("Network.*", "*").
-"""
-
-from __future__ import annotations
-
-import asyncio
-import json
-
-import pytest
-
-from skyvern.proxy.__main__ import build_interceptors, build_pipeline
-from skyvern.proxy.adapters.memory import ForwardAllEventPolicy, InMemorySessionRegistry, StaticKeyAuth
-from skyvern.proxy.adapters.websocket_server import CdpProxyServer
-from skyvern.proxy.core.denylist import (
- DENIED_ERROR_CODE,
- DENYLIST_REASON,
- MethodPatternError,
- MethodPatternSet,
- denied_error,
- denied_method,
- org_denylist_interceptor,
-)
-from skyvern.proxy.core.frames import CdpCommand, CdpEvent, decode_frame
-from skyvern.proxy.core.pipeline import InterceptContext, MiddlewarePipeline, SynthesizedResponse
-from skyvern.proxy.core.session import Principal, ProxySession, ResolvedSession
-from tests.unit.proxy.test_interception import _responses, _upstream_methods
-from tests.unit.proxy.test_metrics_contract import _RecordingMetrics
-from tests.unit.proxy.test_shared_connection import (
- UPSTREAM_URL,
- _command,
- _ControllableUpstream,
- _ScriptClient,
- _SharedBrowser,
- _until,
-)
-
-
-def test_exact_patterns_match_only_their_method() -> None:
- patterns = MethodPatternSet.compile(["Page.navigate"])
-
- assert patterns.matches("Page.navigate")
- assert not patterns.matches("Page.navigateToHistoryEntry")
- assert not patterns.matches("Page.enable")
- assert bool(patterns)
-
-
-def test_trailing_star_is_a_prefix_wildcard() -> None:
- patterns = MethodPatternSet.compile(["Network.*", "Page.nav*"])
-
- assert patterns.matches("Network.enable")
- assert patterns.matches("Network.setCookie")
- assert patterns.matches("Page.navigate")
- assert patterns.matches("Page.navigateToHistoryEntry")
- assert not patterns.matches("Page.enable")
- assert not patterns.matches("Runtime.evaluate")
-
-
-def test_bare_star_matches_every_method() -> None:
- patterns = MethodPatternSet.compile(["*"])
-
- assert patterns.matches("Page.enable")
- assert patterns.matches("anything")
-
-
-def test_empty_pattern_set_matches_nothing() -> None:
- patterns = MethodPatternSet.compile([])
-
- assert not patterns.matches("Page.enable")
- assert not bool(patterns)
-
-
[email protected]("bad", ["", " ", "Page .navigate", "Page.*.x", "*.enable", "Pa*ge", "**", None, 7])
-def test_invalid_patterns_are_rejected(bad: object) -> None:
- with pytest.raises(MethodPatternError):
- MethodPatternSet.compile([bad]) # type: ignore[list-item]
-
-
-def test_denied_error_is_deterministic() -> None:
- # Exact on purpose: the AC promises a deterministic error, so the message is
- # contract, not prose.
- assert denied_error("Page.navigate") == {
- "code": DENIED_ERROR_CODE,
- "message": "'Page.navigate' is not allowed by organization policy",
- }
-
-
-def _session(organization_id: str | None = "org-a") -> ProxySession:
- principal = Principal(principal_id="p", organization_id=organization_id) if organization_id else None
- return ProxySession(session_id="s1", upstream_ws_url="ws://x/y", principal=principal)
-
-
-def _context() -> InterceptContext:
- async def send_proxy_command(command: CdpCommand) -> None:
- raise AssertionError("a denylist never issues upstream traffic")
-
- return InterceptContext(send_proxy_command=send_proxy_command)
-
-
-async def test_denied_method_synthesizes_the_deterministic_error() -> None:
- async def lookup(session: ProxySession) -> MethodPatternSet | None:
- return MethodPatternSet.compile(["Network.*"])
-
- interceptor = org_denylist_interceptor(lookup)
-
- outcome = await interceptor(CdpCommand(id=1, method="Network.enable"), _session(), _context())
-
- assert isinstance(outcome, SynthesizedResponse)
- assert outcome.error == denied_error("Network.enable")
- assert outcome.reason == DENYLIST_REASON
-
-
-async def test_unmatched_and_unconfigured_commands_pass_through() -> None:
- async def some(session: ProxySession) -> MethodPatternSet | None:
- return MethodPatternSet.compile(["Network.*"])
-
- async def none(session: ProxySession) -> MethodPatternSet | None:
- return None
-
- command = CdpCommand(id=1, method="Page.enable")
-
- assert await org_denylist_interceptor(some)(command, _session(), _context()) is command
- assert await org_denylist_interceptor(none)(command, _session(), _context()) is command
-
-
-def _tunneled(method: str, params: dict | None = None, depth: int = 1) -> CdpCommand:
- inner: dict = {"id": 1, "method": method, "params": params or {}}
- for _ in range(depth):
- inner = {"id": 1, "method": "Target.sendMessageToTarget", "params": {"message": json.dumps(inner)}}
- return CdpCommand(id=1, method=inner["method"], params=inner["params"])
-
-
-def test_the_legacy_send_message_tunnel_cannot_bypass_the_denylist() -> None:
- patterns = MethodPatternSet.compile(["Page.navigate"])
-
- # The inner method is what Chrome dispatches, so it is what gets matched.
- assert denied_method(patterns, _tunneled("Page.navigate")) == "Page.navigate"
- assert denied_method(patterns, _tunneled("Page.navigate", depth=3)) == "Page.navigate"
- assert denied_method(patterns, _tunneled("Page.enable")) is None
- # Denying the tunnel itself still works by pattern.
- assert denied_method(MethodPatternSet.compile(["Target.*"]), _tunneled("Page.enable")) == (
- "Target.sendMessageToTarget"
- )
-
-
-def test_an_undispatchable_tunnel_message_passes() -> None:
- patterns = MethodPatternSet.compile(["Page.navigate"])
- for params in (
- None,
- {},
- {"message": 7},
- {"message": "not json"},
- {"message": json.dumps(["not", "an", "object"])},
- {"message": json.dumps({"id": 1})},
- ):
- command = CdpCommand(id=1, method="Target.sendMessageToTarget", params=params)
- # Chrome cannot dispatch any of these either, so there is nothing to deny.
- assert denied_method(patterns, command) is None
-
-
-def test_a_tunnel_nested_past_the_depth_cap_is_denied() -> None:
- patterns = MethodPatternSet.compile(["Page.navigate"])
-
- assert denied_method(patterns, _tunneled("Page.enable", depth=20)) == "Target.sendMessageToTarget"
-
-
-async def test_the_interceptor_denies_the_tunneled_inner_method() -> None:
- async def lookup(session: ProxySession) -> MethodPatternSet | None:
- return MethodPatternSet.compile(["Page.navigate"])
-
- interceptor = org_denylist_interceptor(lookup)
-
- outcome = await interceptor(_tunneled("Page.navigate"), _session(), _context())
-
- assert isinstance(outcome, SynthesizedResponse)
- # The error names the method that was actually denied, not the wrapper.
- assert outcome.error == denied_error("Page.navigate")
-
-
-async def test_a_tunneled_denial_is_audited_against_the_inner_method() -> None:
- connection = _ControllableUpstream()
- sessions = InMemorySessionRegistry()
- sessions.put(ResolvedSession(session_id="s1", upstream_adapter="memory", upstream_ws_url=UPSTREAM_URL))
- metrics = _RecordingMetrics()
- server = CdpProxyServer(
- upstream=_SharedBrowser(connection), # type: ignore[arg-type]
- sessions=sessions,
- auth=StaticKeyAuth({"key": Principal(principal_id="p", organization_id="org")}),
- metrics=metrics, # type: ignore[arg-type]
- event_policy=ForwardAllEventPolicy(),
- pipeline=build_pipeline("forward-all", denylist="Page.navigate"),
- )
- tunneled = _tunneled("Page.navigate")
- client = _ScriptClient([json.dumps({"id": 1, "method": tunneled.method, "params": tunneled.params})])
- client.request.headers = {"x-api-key": "key"}
-
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: client.sent)
-
- [denied] = _responses(client)
- assert denied.error == denied_error("Page.navigate")
- # An org monitoring Page.navigate denials sees the tunneled attempt too: the
- # audit records the blocked method, not the wrapper it rode in.
- [tags] = [tags for op, name, _, tags in metrics.calls if name.endswith("commands_intercepted")]
- assert tags["cdp_method"] == "Page.navigate"
- assert tags["cdp_domain"] == "Page"
- assert tags["reason"] == DENYLIST_REASON
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_an_id_less_tunnel_cannot_bypass_the_denylist() -> None:
- # A denied Target.sendMessageToTarget stripped of its top-level id decodes as a
- # CdpEvent, not a CdpCommand — the interception path is command-only, so without
- # this guard the frame would be forwarded upstream raw, tunneling the denied
- # inner method past the denylist. It must be dropped, never forwarded.
- connection = _ControllableUpstream()
- sessions = InMemorySessionRegistry()
- sessions.put(ResolvedSession(session_id="s1", upstream_adapter="memory", upstream_ws_url=UPSTREAM_URL))
- server = CdpProxyServer(
- upstream=_SharedBrowser(connection), # type: ignore[arg-type]
- sessions=sessions,
- auth=StaticKeyAuth({"key": Principal(principal_id="p", organization_id="org")}),
- metrics=_RecordingMetrics(), # type: ignore[arg-type]
- event_policy=ForwardAllEventPolicy(),
- pipeline=build_pipeline("forward-all", denylist="Page.navigate"),
- )
- inner = json.dumps({"id": 1, "method": "Page.navigate", "params": {"url": "about:blank"}})
- # No top-level "id": this frame decodes as an event and skips command interception.
- id_less_tunnel = json.dumps({"method": "Target.sendMessageToTarget", "params": {"message": inner}})
- # A following command with an id proves the connection stayed usable.
- client = _ScriptClient([id_less_tunnel, _command(2, "Runtime.enable")])
- client.request.headers = {"x-api-key": "key"}
-
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: client.echoes())
-
- # The id-less tunnel never reached the browser (checked over ALL forwarded
- # frames, since the tunnel decodes as an event, not a command); only the
- # legitimate command did. Without the guard this frame is forwarded raw and
- # "Target.sendMessageToTarget" appears here.
- upstream = [decode_frame(wire) for wire in connection.sent]
- upstream_methods = [f.method for f in upstream if isinstance(f, (CdpCommand, CdpEvent))]
- assert "Target.sendMessageToTarget" not in upstream_methods
- assert upstream_methods == ["Runtime.enable"]
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_the_static_env_denylist_wiring_denies_through_a_live_server() -> None:
- pipeline = build_pipeline("forward-all", denylist="Network.*, Page.navigate")
- assert pipeline.has_interceptors
-
- connection = _ControllableUpstream()
- sessions = InMemorySessionRegistry()
- sessions.put(ResolvedSession(session_id="s1", upstream_adapter="memory", upstream_ws_url=UPSTREAM_URL))
- server = CdpProxyServer(
- upstream=_SharedBrowser(connection), # type: ignore[arg-type]
- sessions=sessions,
- auth=StaticKeyAuth({"key": Principal(principal_id="p", organization_id="org")}),
- metrics=_RecordingMetrics(), # type: ignore[arg-type]
- event_policy=ForwardAllEventPolicy(),
- pipeline=pipeline,
- )
- client = _ScriptClient([_command(1, "Network.enable"), _command(2, "Runtime.enable")])
- client.request.headers = {"x-api-key": "key"}
-
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: len(client.sent) >= 2)
-
- [denied] = [r for r in _responses(client) if r.error is not None]
- assert denied.id == 1
- assert denied.error == denied_error("Network.enable")
- assert _upstream_methods(connection) == ["Runtime.enable"]
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-def test_a_bad_env_denylist_pattern_fails_the_boot_not_a_command() -> None:
- with pytest.raises(MethodPatternError):
- build_interceptors("", "Network.*,bad*pattern")
- assert build_interceptors("", "") == ()
-
-
-async def test_denylists_apply_per_organization_across_all_its_sessions() -> None:
- """AC: an org with a denylist gets deterministic errors on banned methods on
- every session it owns, while another org's identical command passes."""
- denylists = {"org-a": MethodPatternSet.compile(["Runtime.*"])}
-
- async def lookup(session: ProxySession) -> MethodPatternSet | None:
- organization_id = session.principal.organization_id if session.principal else None
- return denylists.get(organization_id) if organization_id else None
-
- connection_a, connection_b = _ControllableUpstream(), _ControllableUpstream()
-
- class _TwoBrowsers:
- async def connect(self, session: ProxySession) -> _ControllableUpstream:
- return connection_a if "org-a" in session.upstream_ws_url else connection_b
-
- sessions = InMemorySessionRegistry()
- sessions.put(
- ResolvedSession(
- session_id="sa", upstream_adapter="memory", upstream_ws_url="ws://org-a/b", organization_id="org-a"
- )
- )
- sessions.put(
- ResolvedSession(
- session_id="sb", upstream_adapter="memory", upstream_ws_url="ws://org-b/b", organization_id="org-b"
- )
- )
- metrics = _RecordingMetrics()
- server = CdpProxyServer(
- upstream=_TwoBrowsers(), # type: ignore[arg-type]
- sessions=sessions,
- auth=StaticKeyAuth(
- {
- "key-a": Principal(principal_id="a", organization_id="org-a"),
- "key-b": Principal(principal_id="b", organization_id="org-b"),
- }
- ),
- metrics=metrics, # type: ignore[arg-type]
- event_policy=ForwardAllEventPolicy(),
- pipeline=MiddlewarePipeline(interceptors=[org_denylist_interceptor(lookup)]),
- )
-
- client_a = _ScriptClient([_command(1, "Runtime.evaluate"), _command(2, "Page.enable")])
- client_a.request.path = "/sa"
- client_a.request.headers = {"x-api-key": "key-a"}
- client_b = _ScriptClient([_command(1, "Runtime.evaluate")])
- client_b.request.path = "/sb"
- client_b.request.headers = {"x-api-key": "key-b"}
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: len(client_a.sent) >= 2 and client_b.sent)
-
- # org-a: the banned method is answered with the deterministic error, its other
- # command still reaches the browser; the banned one never does.
- [denied] = [r for r in _responses(client_a) if r.error is not None]
- assert denied.id == 1
- assert denied.error == denied_error("Runtime.evaluate")
- assert _upstream_methods(connection_a) == ["Page.enable"]
- # org-b: the same method sails through.
- assert client_b.echoes() == {"Runtime.evaluate"}
- assert _upstream_methods(connection_b) == ["Runtime.evaluate"]
- # AC: denied attempts are visible in the audit trail, attributed to the org.
- audits = [tags for op, name, _, tags in metrics.calls if name.endswith("commands_intercepted")]
- assert audits == [
- {"org_id": "org-a", "reason": DENYLIST_REASON, "cdp_method": "Runtime.evaluate", "cdp_domain": "Runtime"}
- ]
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
diff --git a/tests/unit/proxy/test_discovery.py b/tests/unit/proxy/test_discovery.py
deleted file mode 100644
index 704b328c8..000000000
--- a/tests/unit/proxy/test_discovery.py
+++ /dev/null
@@ -1,263 +0,0 @@
-"""HTTP discovery endpoint tests for the client-facing CDP proxy.
-
-CDP clients (Playwright, puppeteer) GET /json/version — /json/list, /json — to
-learn the browser websocket url before the WS upgrade. The proxy must serve these
-authenticated and return PROXY-SCOPED debugger urls, never the upstream url.
-"""
-
-from __future__ import annotations
-
-import json
-
-import pytest
-from websockets.datastructures import Headers
-from websockets.http11 import Request, Response
-
-from skyvern.proxy.adapters.memory import (
- ForwardAllEventPolicy,
- InMemorySessionRegistry,
- InMemoryUpstreamBrowser,
- NoOpMetrics,
- StaticKeyAuth,
-)
-from skyvern.proxy.adapters.websocket_server import (
- CdpProxyServer,
- _discovery_payload,
- _proxy_scoped_ws_url,
- _split_discovery_target,
-)
-from skyvern.proxy.core.session import Principal, ResolvedSession
-
-VALID_KEY = "disco-key-abc123"
-SECRET = "upstream-secret-token-do-not-leak"
-
-
-class _FakeConnection:
- """Stands in for websockets.ServerConnection.respond during process_request."""
-
- def respond(self, status: object, text: str) -> Response:
- body = text.encode()
- headers = Headers([("Content-Type", "text/plain; charset=utf-8"), ("Content-Length", str(len(body)))])
- return Response(int(status), "Unauthorized", headers, body)
-
-
-class _RecordingMetrics:
- def __init__(self) -> None:
- self.calls: list[tuple[str, str, dict[str, str]]] = []
-
- def increment(self, name: str, amount: int = 1, tags: object = None) -> None:
- self.calls.append(("increment", name, dict(tags or {})))
-
- def observe(self, name: str, value: float, tags: object = None) -> None:
- self.calls.append(("observe", name, dict(tags or {})))
-
- def gauge(self, name: str, amount: int, tags: object = None) -> None:
- self.calls.append(("gauge", name, dict(tags or {})))
-
-
-class _CountingRegistry(InMemorySessionRegistry):
- def __init__(self) -> None:
- super().__init__()
- self.resolve_calls = 0
-
- async def resolve(self, session_id: str): # type: ignore[no-untyped-def]
- self.resolve_calls += 1
- return await super().resolve(session_id)
-
-
-def _server(auth: object, metrics: object | None = None, sessions: object | None = None) -> CdpProxyServer:
- return CdpProxyServer(
- upstream=InMemoryUpstreamBrowser(),
- sessions=sessions or InMemorySessionRegistry(),
- auth=auth, # type: ignore[arg-type]
- metrics=metrics or NoOpMetrics(),
- event_policy=ForwardAllEventPolicy(),
- )
-
-
-def _request(path: str, headers: dict[str, str] | None = None) -> Request:
- hdrs = Headers()
- hdrs["Host"] = "proxy.example:9223"
- for name, value in (headers or {}).items():
- hdrs[name] = value
- return Request(path=path, headers=hdrs)
-
-
-async def _discover(server: CdpProxyServer, path: str, headers: dict[str, str] | None = None) -> Response | None:
- return await server._process_request(_FakeConnection(), _request(path, headers)) # type: ignore[arg-type]
-
-
-# ---- pure splitter ----------------------------------------------------------
-
-
- ("target", "expected"),
- [
- ("/s1/json/version", ("/s1", "/json/version")),
- ("/s1/json/version/", ("/s1", "/json/version")), # Playwright appends a trailing slash
- ("/s1/json/list", ("/s1", "/json/list")),
- ("/s1/json", ("/s1", "/json")),
- ("/json/version", ("/", "/json/version")), # session-less
- ("/key/mykey/s1/json/version", ("/key/mykey/s1", "/json/version")),
- ("/s1/json/version?token=x", ("/s1?token=x", "/json/version")),
- ("/s1", None), # a plain WS target, not discovery
- ("/s1/json/other", None),
- ("/", None),
- ],
-)
-def test_split_discovery_target(target: str, expected: tuple[str, str] | None) -> None:
- assert _split_discovery_target(target) == expected
-
-
-def test_proxy_scoped_ws_url_never_points_at_upstream() -> None:
- url = _proxy_scoped_ws_url("proxy.example:9223", None, "/s1")
- assert url == "ws://proxy.example:9223/s1"
-
-
- ("forwarded_proto", "scheme"),
- [
- ("https", "wss"),
- ("HTTPS", "wss"), # case-insensitive
- (" https ", "wss"), # whitespace-padded
- ("https, http", "wss"), # proxy comma-chain: first hop is the client scheme
- ("http", "ws"),
- (None, "ws"),
- ("", "ws"),
- ],
-)
-def test_proxy_scoped_ws_url_parses_forwarded_proto(forwarded_proto: str | None, scheme: str) -> None:
- assert _proxy_scoped_ws_url("proxy.example", forwarded_proto, "/s1").startswith(f"{scheme}://")
-
-
-def test_websocket_server_import_suppresses_websockets_debug_logging() -> None:
- # The driving adapter must self-suppress the credential-bearing websockets DEBUG
- # logs at import, not lean on a sibling module's import side effect.
- import importlib
- import logging
-
- from skyvern.proxy.adapters import websocket_server
-
- logging.getLogger("websockets").setLevel(logging.DEBUG)
- importlib.reload(websocket_server)
- assert logging.getLogger("websockets").level == logging.INFO
-
-
-def test_discovery_payload_version_carries_the_ws_debugger_url() -> None:
- payload = _discovery_payload("/json/version", "ws://p/s1", "s1")
- assert payload["webSocketDebuggerUrl"] == "ws://p/s1"
-
-
-def test_discovery_payload_list_is_an_array_of_targets() -> None:
- payload = _discovery_payload("/json/list", "ws://p/s1", "s1")
- assert isinstance(payload, list) and payload[0]["webSocketDebuggerUrl"] == "ws://p/s1"
-
-
-# ---- process_request --------------------------------------------------------
-
-
-def _body(response: Response) -> object:
- return json.loads(bytes(response.body).decode())
-
-
[email protected]("suffix", ["/json/version", "/json/version/", "/json/list", "/json"])
-async def test_authenticated_discovery_returns_proxy_scoped_url(suffix: str) -> None:
- server = _server(StaticKeyAuth({VALID_KEY: Principal(principal_id="p")}))
- response = await _discover(server, f"/s1{suffix}", headers={"x-api-key": VALID_KEY})
- assert response is not None and response.status_code == 200
- body = _body(response)
- url = body["webSocketDebuggerUrl"] if isinstance(body, dict) else body[0]["webSocketDebuggerUrl"]
- # Points back at the proxy (Host header), scoped to the requested session id.
- assert url == "ws://proxy.example:9223/s1"
-
-
-async def test_non_discovery_target_is_passed_through_to_the_ws_handshake() -> None:
- server = _server(StaticKeyAuth({VALID_KEY: Principal(principal_id="p")}))
- assert await _discover(server, "/s1", headers={"x-api-key": VALID_KEY}) is None
-
-
-async def test_unauthenticated_discovery_is_rejected_401() -> None:
- metrics = _RecordingMetrics()
- server = _server(StaticKeyAuth({VALID_KEY: Principal(principal_id="p")}), metrics=metrics)
- response = await _discover(server, "/s1/json/version") # no key
- assert response is not None and response.status_code == 401
- reasons = {tags.get("reason") for _op, name, tags in metrics.calls if name.endswith("connection_rejected")}
- assert "unauthorized" in reasons
-
-
-async def test_malformed_discovery_target_is_rejected_401() -> None:
- server = _server(StaticKeyAuth({VALID_KEY: Principal(principal_id="p")}))
- response = await _discover(server, "//[/json/version", headers={"x-api-key": VALID_KEY})
- assert response is not None and response.status_code == 401
-
-
-async def test_discovery_preserves_url_credentials_for_headerless_clients() -> None:
- # A puppeteer-style client carries its key in the path (it cannot set headers);
- # the returned ws url must keep it so the follow-up WS connect still authenticates.
- server = _server(StaticKeyAuth({VALID_KEY: Principal(principal_id="p")}))
- response = await _discover(server, f"/key/{VALID_KEY}/s1/json/version")
- assert response is not None and response.status_code == 200
- assert _body(response)["webSocketDebuggerUrl"] == f"ws://proxy.example:9223/key/{VALID_KEY}/s1"
-
-
-async def test_discovery_response_is_never_cached() -> None:
- # The webSocketDebuggerUrl can carry a header-less client's own API key, so a
- # shared intermediary must never retain the body.
- server = _server(StaticKeyAuth({VALID_KEY: Principal(principal_id="p")}))
- response = await _discover(server, "/s1/json/version", headers={"x-api-key": VALID_KEY})
- assert response is not None
- assert response.headers["Cache-Control"] == "no-store"
- assert response.headers["Pragma"] == "no-cache"
-
-
-async def test_discovery_never_resolves_or_leaks_the_upstream() -> None:
- # Discovery must not dial or even resolve the session — so it cannot leak the
- # upstream url and cannot become a cross-org existence oracle.
- sessions = _CountingRegistry()
- sessions.put(
- ResolvedSession(
- session_id="s1",
- upstream_adapter="memory",
- upstream_ws_url=f"ws://vendor.internal:9222/live?token={SECRET}",
- organization_id="o_owner",
- )
- )
- server = _server(StaticKeyAuth({VALID_KEY: Principal(principal_id="p")}), sessions=sessions)
- response = await _discover(server, "/s1/json/version", headers={"x-api-key": VALID_KEY})
- assert response is not None
- assert sessions.resolve_calls == 0
- text = bytes(response.body).decode()
- assert SECRET not in text and "vendor.internal" not in text
-
-
-# ---- health endpoint --------------------------------------------------------
-
-
-async def test_healthz_answers_without_credentials() -> None:
- server = _server(StaticKeyAuth({VALID_KEY: Principal(principal_id="p")}))
-
- response = await _discover(server, "/healthz")
-
- assert response is not None
- assert response.status_code == 200
- assert response.body == b"ok"
-
-
-async def test_healthz_never_reaches_discovery_or_ws_upgrade() -> None:
- # A probe target is answered directly: not None (which would hand the GET to
- # the WS handshake) and not gated on the discovery auth path.
- server = _server(StaticKeyAuth({VALID_KEY: Principal(principal_id="p")}))
-
- response = await _discover(server, "/healthz?probe=1")
-
- assert response is not None
- assert response.status_code == 200
diff --git a/tests/unit/proxy/test_event_policy.py b/tests/unit/proxy/test_event_policy.py
deleted file mode 100644
index b6b6072e1..000000000
--- a/tests/unit/proxy/test_event_policy.py
+++ /dev/null
@@ -1,567 +0,0 @@
-from __future__ import annotations
-
-import asyncio
-import time
-from pathlib import Path
-
-import pytest
-
-from skyvern.proxy.adapters.memory import ForwardAllEventPolicy
-from skyvern.proxy.core.frames import (
- KNOWN_CDP_DOMAINS,
- LIFECYCLE_EVENTS,
- TARGET_ATTACHED_EVENT,
- TARGET_DETACHED_EVENT,
- CdpCommand,
- CdpEvent,
- decode_frame,
- encode_frame,
-)
-from skyvern.proxy.core.policy import (
- _MAX_TRACKED_CHILD_SESSIONS,
- FORWARD,
- Drop,
- DropReason,
- EventPolicyConfig,
- EventPolicyEngine,
- Forward,
- RateRule,
- Rewrite,
-)
-from skyvern.proxy.core.session import Principal, ProxySession
-
-# Reusing the shared-connection harness rather than pasting a fourth copy of a proxy
-# server double: these two tests must exercise the very routing/ownership flow those
-# tests pin, so they have to run against the same fakes.
-from tests.unit.proxy.test_shared_connection import (
- _AttachEventUpstream,
- _command,
- _ControllableUpstream,
- _ScriptClient,
- _server,
- _SharedBrowser,
- _until,
-)
-
-FIXTURES_DIR = Path(__file__).parent / "fixtures"
-SESSION_ID = "s_1"
-ORG_ID = "o_1"
-
-
-class FakeClock:
- """Injected time source: tests advance it explicitly, so a throttle window is
- decided by the test rather than by how long the test took to run."""
-
- def __init__(self, now: float = 1000.0) -> None:
- self._now = now
-
- def __call__(self) -> float:
- return self._now
-
- def advance(self, seconds: float) -> None:
- self._now += seconds
-
-
-def _session(session_id: str = SESSION_ID) -> ProxySession:
- return ProxySession(
- session_id=session_id,
- upstream_ws_url="ws://upstream.internal:9222/x?token=secret-token",
- principal=Principal(principal_id=ORG_ID, organization_id=ORG_ID),
- connect_headers={"authorization": "Bearer secret-token"},
- )
-
-
-def _engine(config: EventPolicyConfig | None = None, clock: FakeClock | None = None) -> EventPolicyEngine:
- return EventPolicyEngine(config=config or EventPolicyConfig(), clock=clock or FakeClock())
-
-
-def _recorded_events() -> list[CdpEvent]:
- events: list[CdpEvent] = []
- for fixture in sorted(FIXTURES_DIR.glob("*.jsonl")):
- for line in fixture.read_text().splitlines():
- if not line.strip():
- continue
- frame = decode_frame(line)
- if isinstance(frame, CdpEvent):
- events.append(frame)
- return events
-
-
-# --- AC2: no policy configured -> the engine is a transparent no-op -------------
-
-
-def test_no_policy_forwards_every_recorded_event_unchanged() -> None:
- engine = _engine()
- session = _session()
- events = _recorded_events()
-
- assert events, "fixtures must contain events for this to prove anything"
- for event in events:
- assert engine.decide(event, session) is FORWARD
-
-
-def test_no_policy_leaves_wire_bytes_byte_identical() -> None:
- """AC2: with no policy, the bytes the call site would deliver are byte-for-byte
- what it delivers today. Forward carries no payload, so the call site keeps its
- own already-processed frame and its existing encode path — the engine cannot
- perturb the wire even in principle."""
- engine = _engine()
- session = _session()
-
- for event in _recorded_events():
- baseline = encode_frame(event)
- decision = engine.decide(event, session)
-
- assert isinstance(decision, Forward)
- assert encode_frame(event) == baseline
-
-
-def test_forward_is_a_singleton_carrying_no_payload() -> None:
- engine = _engine()
- event = CdpEvent(method="Page.loadEventFired", params={})
-
- assert engine.decide(event, _session()) is FORWARD
- assert not hasattr(FORWARD, "event")
-
-
-# --- AC3: throttle, on an injected clock ---------------------------------------
-
-
-def test_throttle_forwards_up_to_the_cap_then_drops_within_the_window() -> None:
- clock = FakeClock()
- config = EventPolicyConfig(
- rules=(RateRule.throttle(method="Network.dataReceived", max_per_window=2, window_seconds=10.0),)
- )
- engine = _engine(config, clock)
- session = _session()
- event = CdpEvent(method="Network.dataReceived", params={"encodedDataLength": 1})
-
- decisions = [engine.decide(event, session) for _ in range(5)]
-
- assert decisions[0] is FORWARD
- assert decisions[1] is FORWARD
- assert all(decision == Drop(DropReason.THROTTLED) for decision in decisions[2:])
-
-
-def test_throttle_window_reopens_only_after_the_clock_advances() -> None:
- clock = FakeClock()
- config = EventPolicyConfig(
- rules=(RateRule.throttle(method="Network.dataReceived", max_per_window=1, window_seconds=10.0),)
- )
- engine = _engine(config, clock)
- session = _session()
- event = CdpEvent(method="Network.dataReceived", params={})
-
- assert engine.decide(event, session) is FORWARD
- assert engine.decide(event, session) == Drop(DropReason.THROTTLED)
-
- clock.advance(9.9)
- assert engine.decide(event, session) == Drop(DropReason.THROTTLED)
-
- clock.advance(0.1)
- assert engine.decide(event, session) is FORWARD
-
-
-def test_throttle_does_not_touch_other_methods() -> None:
- config = EventPolicyConfig(
- rules=(RateRule.throttle(method="Network.dataReceived", max_per_window=1, window_seconds=10.0),)
- )
- engine = _engine(config)
- session = _session()
-
- engine.decide(CdpEvent(method="Network.dataReceived", params={}), session)
-
- for _ in range(10):
- assert engine.decide(CdpEvent(method="Page.loadEventFired", params={}), session) is FORWARD
-
-
-def test_throttle_budget_is_per_session_not_global() -> None:
- """Two sessions must not spend each other's budget — a busy tenant would
- otherwise throttle a quiet co-tenant on the same shared upstream."""
- config = EventPolicyConfig(
- rules=(RateRule.throttle(method="Network.dataReceived", max_per_window=1, window_seconds=10.0),)
- )
- engine = _engine(config)
- event = CdpEvent(method="Network.dataReceived", params={})
- first, second = _session("s_1"), _session("s_2")
-
- assert engine.decide(event, first) is FORWARD
- assert engine.decide(event, first) == Drop(DropReason.THROTTLED)
- assert engine.decide(event, second) is FORWARD
-
-
-def test_throttle_keys_split_the_budget_by_param() -> None:
- config = EventPolicyConfig(
- rules=(
- RateRule.throttle(
- method="Network.dataReceived", max_per_window=1, window_seconds=10.0, key_params=("requestId",)
- ),
- )
- )
- engine = _engine(config)
- session = _session()
-
- assert engine.decide(CdpEvent(method="Network.dataReceived", params={"requestId": "a"}), session) is FORWARD
- assert engine.decide(CdpEvent(method="Network.dataReceived", params={"requestId": "a"}), session) == Drop(
- DropReason.THROTTLED
- )
- assert engine.decide(CdpEvent(method="Network.dataReceived", params={"requestId": "b"}), session) is FORWARD
-
-
-# --- no coalesce: a rate cap is first-wins, and says so ------------------------
-
-
-def test_a_rate_cap_is_first_wins_and_there_is_no_coalesce_claiming_otherwise() -> None:
- """A budget of one per window delivers the FIRST event of a burst and drops the
- newer ones — so it must never be dressed up as 'collapse to the latest'. Latest-wins
- coalescing cannot be decided per-event and is deferred to SKY-12501; this pins that
- the misleading shape is not back.
- """
- engine = _engine(EventPolicyConfig(rules=(RateRule.throttle("Target.targetInfoChanged", 1, 5.0),)))
- session = _session()
- burst = [CdpEvent(method="Target.targetInfoChanged", params={"targetInfo": {"n": n}}) for n in range(5)]
-
- decisions = [engine.decide(event, session) for event in burst]
-
- # The oldest survives; that is a rate cap, not coalescing.
- assert decisions[0] is FORWARD
- assert all(decision == Drop(DropReason.THROTTLED) for decision in decisions[1:])
- assert not hasattr(RateRule, "coalesce")
- assert not any(reason.name == "COALESCED" for reason in DropReason)
-
-
-# --- carry-forward #11: lifecycle events are structurally non-droppable --------
-
-
[email protected]("method", sorted(LIFECYCLE_EVENTS))
-def test_lifecycle_events_cannot_be_dropped_by_any_rule(method: str) -> None:
- """SKY-12500 learns session ownership from these two events downstream of the
- policy gate. A policy that dropped one would skip the ownership transition and
- strand the session's owner, so the engine must not be able to express it."""
- config = EventPolicyConfig(rules=(RateRule.throttle(method=method, max_per_window=0, window_seconds=10.0),))
- engine = _engine(config)
- session = _session()
- event = CdpEvent(method=method, params={"sessionId": "cdp_1"})
-
- for _ in range(10):
- assert engine.decide(event, session) is FORWARD
-
-
-def test_a_zero_budget_rule_still_drops_an_ordinary_event() -> None:
- """Guards the test above: a max_per_window=0 rule really does drop, so the
- lifecycle result proves non-droppability rather than a broken rule."""
- config = EventPolicyConfig(
- rules=(RateRule.throttle(method="Network.dataReceived", max_per_window=0, window_seconds=10.0),)
- )
- engine = _engine(config)
-
- assert engine.decide(CdpEvent(method="Network.dataReceived", params={}), _session()) == Drop(DropReason.THROTTLED)
-
-
-def test_lifecycle_events_are_exactly_the_ownership_events() -> None:
- assert LIFECYCLE_EVENTS == frozenset({TARGET_ATTACHED_EVENT, TARGET_DETACHED_EVENT})
-
-
-# --- per-session domain enablement (consumed by SKY-12501) ---------------------
-
-
-def test_domain_starts_disabled_and_tracks_enable_then_disable() -> None:
- engine = _engine()
- session = _session()
-
- assert not engine.is_domain_enabled(session, "Network")
-
- engine.observe_command(CdpCommand(id=1, method="Network.enable"), session)
- assert engine.is_domain_enabled(session, "Network")
-
- engine.observe_command(CdpCommand(id=2, method="Network.disable"), session)
- assert not engine.is_domain_enabled(session, "Network")
-
-
-def test_domain_enablement_is_tracked_per_session() -> None:
- engine = _engine()
- first, second = _session("s_1"), _session("s_2")
-
- engine.observe_command(CdpCommand(id=1, method="Network.enable"), first)
-
- assert engine.is_domain_enabled(first, "Network")
- assert not engine.is_domain_enabled(second, "Network")
-
-
-def test_non_enable_commands_do_not_enable_a_domain() -> None:
- engine = _engine()
- session = _session()
-
- engine.observe_command(CdpCommand(id=1, method="Network.getCookies"), session)
- engine.observe_command(CdpCommand(id=2, method="Runtime.evaluate", params={"expression": "1"}), session)
-
- assert not engine.is_domain_enabled(session, "Network")
- assert not engine.is_domain_enabled(session, "Runtime")
-
-
-def test_interest_is_tracked_per_child_session_not_per_connection() -> None:
- """A client enables a domain on ONE attached page. Keying interest by the outer
- session would let a disable on a sibling child clear it, and a rule reading this
- would then mis-drop that page's traffic while it is still subscribed.
- """
- engine = _engine()
- session = _session()
-
- engine.observe_command(CdpCommand(id=1, method="Network.enable", session_id="child_a"), session)
- engine.observe_command(CdpCommand(id=2, method="Network.disable", session_id="child_b"), session)
-
- assert engine.is_domain_enabled(session, "Network", "child_a")
- assert not engine.is_domain_enabled(session, "Network", "child_b")
-
-
-def test_browser_level_interest_is_distinct_from_a_child_session() -> None:
- engine = _engine()
- session = _session()
-
- engine.observe_command(CdpCommand(id=1, method="Target.enable"), session)
-
- assert engine.is_domain_enabled(session, "Target")
- assert not engine.is_domain_enabled(session, "Target", "child_a")
-
-
-def test_detaching_a_child_retires_its_interest() -> None:
- engine = _engine()
- session = _session()
- engine.observe_command(CdpCommand(id=1, method="Network.enable", session_id="child_a"), session)
-
- engine.decide(CdpEvent(method=TARGET_DETACHED_EVENT, params={"sessionId": "child_a"}), session)
-
- assert not engine.is_domain_enabled(session, "Network", "child_a")
- assert engine.tracked_children(session) == 0
-
-
-def test_detaching_one_child_leaves_a_siblings_interest_alone() -> None:
- engine = _engine()
- session = _session()
- engine.observe_command(CdpCommand(id=1, method="Network.enable", session_id="child_a"), session)
- engine.observe_command(CdpCommand(id=2, method="Network.enable", session_id="child_b"), session)
-
- engine.decide(CdpEvent(method=TARGET_DETACHED_EVENT, params={"sessionId": "child_b"}), session)
-
- assert engine.is_domain_enabled(session, "Network", "child_a")
- assert not engine.is_domain_enabled(session, "Network", "child_b")
-
-
-# --- interest tracking is bounded on both axes ---------------------------------
-
-
-def test_an_invented_domain_is_never_tracked() -> None:
- """`<anything>.enable` is client input. Only real CDP domains count as interest,
- which is also what bounds the set."""
- engine = _engine()
- session = _session()
-
- for n in range(5_000):
- engine.observe_command(CdpCommand(id=n, method=f"NotADomain{n}.enable"), session)
-
- assert not engine.is_domain_enabled(session, "NotADomain0")
- assert engine.tracked_children(session) == 0
-
-
-def test_enabled_domains_stay_bounded_by_the_protocol_surface() -> None:
- engine = _engine()
- session = _session()
-
- for n, domain in enumerate(sorted(KNOWN_CDP_DOMAINS) * 3):
- engine.observe_command(CdpCommand(id=n, method=f"{domain}.enable"), session)
-
- assert engine.tracked_children(session) == 1
- assert engine.enabled_domain_count(session) == len(KNOWN_CDP_DOMAINS)
-
-
-def test_child_session_churn_stays_bounded() -> None:
- """A client may address a sessionId that was never attached, so the child id is
- untrusted input and cannot grow this table without limit."""
- engine = _engine()
- session = _session()
-
- for n in range(5_000):
- engine.observe_command(CdpCommand(id=n, method="Network.enable", session_id=f"fabricated_{n}"), session)
-
- assert engine.tracked_children(session) <= _MAX_TRACKED_CHILD_SESSIONS
-
-
-def test_forget_session_releases_its_state() -> None:
- """Per-session state must not outlive the session, or a long-lived proxy leaks
- one entry per session it has ever seen."""
- config = EventPolicyConfig(
- rules=(RateRule.throttle(method="Network.dataReceived", max_per_window=1, window_seconds=10.0),)
- )
- engine = _engine(config)
- session = _session()
- engine.observe_command(CdpCommand(id=1, method="Network.enable"), session)
- engine.decide(CdpEvent(method="Network.dataReceived", params={}), session)
-
- assert engine.tracked_sessions() == 1
-
- engine.forget(session.session_id)
-
- assert engine.tracked_sessions() == 0
- assert not engine.is_domain_enabled(session, "Network")
- assert engine.decide(CdpEvent(method="Network.dataReceived", params={}), session) is FORWARD
-
-
-def test_forget_is_idempotent_for_an_unknown_session() -> None:
- engine = _engine()
-
- engine.forget("never-seen")
-
- assert engine.tracked_sessions() == 0
-
-
-# --- decision traces: bounded tags, no credentials -----------------------------
-
-
-def test_drop_reasons_are_a_closed_bounded_set() -> None:
- """Decision reasons become metric tags; a free-form reason would blow up label
- cardinality (SKY-12510), so the type itself has to bound them."""
- assert len(DropReason) <= 8
- for reason in DropReason:
- assert reason.value.replace("_", "").isalnum()
-
-
-def test_decision_traces_never_carry_credentials_or_urls() -> None:
- config = EventPolicyConfig(
- rules=(RateRule.throttle(method="Network.dataReceived", max_per_window=0, window_seconds=10.0),)
- )
- engine = _engine(config)
- session = _session()
- event = CdpEvent(method="Network.dataReceived", params={"url": "https://host.example/p?token=secret-token"})
-
- decision = engine.decide(event, session)
-
- assert isinstance(decision, Drop)
- rendered = repr(decision) + decision.reason.value
- for leak in ("secret-token", "upstream.internal", "host.example", "Bearer", "ws://", "https://"):
- assert leak not in rendered
-
-
-def test_rewrite_carries_the_replacement_event() -> None:
- """The Rewrite variant exists for SKY-12501/12538 (synthesized errors); the
- engine ships the type, not the rules that return it."""
- replacement = CdpEvent(method="Page.loadEventFired", params={"redacted": True})
- decision = Rewrite(replacement)
-
- assert decision.event is replacement
-
-
-# --- AC1: the no-policy path is allocation-light -------------------------------
-
-
-def test_no_policy_decide_is_allocation_light_over_a_large_stream() -> None:
- """AC1 smell test, not a microbenchmark: the default path must not allocate a
- decision per event. FORWARD is a singleton, so a long stream adds no objects."""
- engine = _engine()
- session = _session()
- event = CdpEvent(method="Network.dataReceived", params={"encodedDataLength": 1})
-
- decisions = {id(engine.decide(event, session)) for _ in range(10_000)}
-
- assert decisions == {id(FORWARD)}
-
-
-def test_engine_keeps_up_with_a_high_volume_stream() -> None:
- """AC1: a coarse ceiling that catches an accidental O(rules) rescan or a
- per-event parse. Generous enough not to flake on a loaded CI box."""
- config = EventPolicyConfig(
- rules=(RateRule.throttle(method="Network.dataReceived", max_per_window=100, window_seconds=1.0),)
- )
- engine = _engine(config)
- session = _session()
- events = [CdpEvent(method="Network.dataReceived", params={"n": n}) for n in range(20_000)]
-
- started = time.perf_counter()
- for event in events:
- engine.decide(event, session)
- elapsed = time.perf_counter() - started
-
- assert elapsed < 2.0
-
-
-# --- wired into the proxy: the two properties that must survive the engine --------
-
-
-def _engine_policy(*rules: RateRule) -> EventPolicyEngine:
- return EventPolicyEngine(config=EventPolicyConfig(rules=rules), clock=FakeClock())
-
-
-async def test_engine_with_no_policy_delivers_the_same_bytes_as_forward_all() -> None:
- """AC2, end to end: swap the default ForwardAll for an unconfigured engine and the
- client's wire bytes are unchanged. This is the property the whole ticket rests on —
- turning the engine on must be invisible until a rule is configured."""
-
- # Browser-level (no sessionId), so every one reaches the lone client instead of
- # being dropped as unowned — the recorded events are replayed at browser scope for
- # exactly that reason. The non-ASCII frame is the case where the codec's
- # normalization actually bites, so it is the one most likely to expose a Forward
- # path that re-serializes.
- # Lifecycle events are excluded: they route by ownership rather than broadcast, so
- # a lone client with nothing attached would never receive them. The test above
- # covers them directly.
- stream = [
- CdpEvent(method=event.method, params=event.params)
- for event in _recorded_events()
- if event.method not in LIFECYCLE_EVENTS
- ]
- stream.append(CdpEvent(method="Runtime.consoleAPICalled", params={"text": "café ünicode"}))
- sentinel = CdpEvent(method="Runtime.executionContextsCleared", params={})
-
- async def deliver(policy: object) -> list[str]:
- connection = _ControllableUpstream()
- server = _server(_SharedBrowser(connection))
- server._event_policy = policy # type: ignore[assignment]
- client = _ScriptClient([_command(1, "Page.enable")])
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: bool(client.sent))
- for event in stream:
- connection.emit(event)
- connection.emit(sentinel)
- assert await _until(lambda: any(sentinel.method in raw for raw in client.sent))
- client.release()
- await asyncio.wait_for(task, timeout=5)
- return client.sent
-
- baseline = await deliver(ForwardAllEventPolicy())
- with_engine = await deliver(_engine_policy())
-
- assert with_engine == baseline
- # Guards against a vacuous pass: every event really did reach the client, so the
- # comparison covers the whole stream rather than an empty one.
- assert len(baseline) == len(stream) + 2 # + the Page.enable response + the sentinel
-
-
-async def test_a_policy_that_would_drop_the_attach_notice_still_learns_ownership() -> None:
- """Carry-forward #11 (SKY-12500 gate): the policy gate runs upstream of the
- ownership bookkeeping, so a dropped Target.attachedToTarget would strand the
- session — its owner never learned, its traffic never routable. The engine treats
- the attach notice as structural, so a rule aimed straight at it changes nothing.
- """
- connection = _AttachEventUpstream()
- server = _server(_SharedBrowser(connection))
- server._event_policy = _engine_policy( # type: ignore[assignment]
- RateRule.throttle(method=TARGET_ATTACHED_EVENT, max_per_window=0, window_seconds=3600.0)
- )
- client_a = _ScriptClient([_command(1, "Target.attachToTarget", {"targetId": "tA", "flatten": True})])
- client_b = _ScriptClient([_command(1, "Page.enable")])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: client_a.events(TARGET_ATTACHED_EVENT) and client_b.echoes())
-
- # The notice survived the rule, and ownership was learned from it: the attached
- # session's later traffic still routes to its owner and to nobody else.
- assert client_b.events(TARGET_ATTACHED_EVENT) == []
- connection.emit(CdpEvent(method="Page.loadEventFired", session_id="tA"))
- assert await _until(lambda: "tA" in client_a.event_sessions())
- assert client_b.event_sessions() == set()
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
diff --git a/tests/unit/proxy/test_event_policy_contract.py b/tests/unit/proxy/test_event_policy_contract.py
deleted file mode 100644
index ebee42e3f..000000000
--- a/tests/unit/proxy/test_event_policy_contract.py
+++ /dev/null
@@ -1,159 +0,0 @@
-"""Reusable contract suite for EventPolicyPort adapters.
-
-Any adapter (generic here, cloud-specific under tests/cloud/) subclasses
-EventPolicyPortContract and overrides make_policy(); every behavioral guarantee of
-the port is asserted once, here. Each shipped policy config is an adapter of this
-port and gets a subclass, so a new rule pack cannot skip the invariants.
-
-tests/unit/proxy/test_contract_teeth.py proves these clauses actually catch a
-violation; without it a weakened clause here would stay green forever.
-"""
-
-from __future__ import annotations
-
-from skyvern.proxy.adapters.memory import ForwardAllEventPolicy
-from skyvern.proxy.core.frames import TARGET_ATTACHED_EVENT, TARGET_DETACHED_EVENT, CdpCommand, CdpEvent
-from skyvern.proxy.core.policy import (
- Clock,
- Drop,
- DropReason,
- EventPolicyConfig,
- EventPolicyEngine,
- Forward,
- Rewrite,
-)
-from skyvern.proxy.core.policy_pack import NOISY_EVENT_PACK_V1
-from skyvern.proxy.core.screencast import SCREENCAST_PACK_V1
-from skyvern.proxy.core.session import ProxySession
-from skyvern.proxy.ports import EventPolicyPort
-
-# Events every adapter must survive: gated-noisy, ordinary, unknown domain, and the
-# shapes a client can put on the wire that no protocol doc promises.
-SAMPLE_EVENTS = (
- CdpEvent(method="Network.dataReceived", params={"requestId": "r1"}),
- CdpEvent(method="Runtime.consoleAPICalled", params={"type": "log"}),
- CdpEvent(method="Page.screencastFrame", params={"sessionId": 1}),
- CdpEvent(method="Page.loadEventFired", params={}),
- CdpEvent(method="Unknown.somethingNobodyDefined", params=None),
- CdpEvent(method="NoDomainSeparator", params={"a": 1}, session_id="child-1"),
- CdpEvent(method="", params=None),
-)
-
-LIFECYCLE_SAMPLES = (
- CdpEvent(method=TARGET_ATTACHED_EVENT, params={"sessionId": "child-1"}),
- CdpEvent(method=TARGET_DETACHED_EVENT, params={"sessionId": "child-1"}),
-)
-
-SAMPLE_COMMANDS = (
- CdpCommand(id=1, method="Network.enable"),
- CdpCommand(id=2, method="Network.disable", session_id="child-1"),
- CdpCommand(id=3, method="Unknown.enable"),
- CdpCommand(id=4, method="NoDomainSeparator"),
- CdpCommand(id=5, method="", params={"weird": True}),
-)
-
-
-def frozen_clock() -> Clock:
- """Time never advances, so a throttle budget never refills: the harshest clock a
- rule can face, and deterministic — a contract suite may never race a real clock.
- """
- return lambda: 0.0
-
-
-class EventPolicyPortContract:
- def make_policy(self) -> EventPolicyPort:
- raise NotImplementedError
-
- def make_session(self) -> ProxySession:
- return ProxySession(session_id="s1", upstream_ws_url="ws://localhost:0/devtools/browser/test")
-
- def test_decide_returns_a_policy_decision(self) -> None:
- policy, session = self.make_policy(), self.make_session()
- for event in SAMPLE_EVENTS:
- assert isinstance(policy.decide(event, session), (Forward, Drop, Rewrite))
-
- def test_lifecycle_events_are_never_dropped(self) -> None:
- """SKY-12500: the driving adapter learns session ownership downstream of this
- gate, so a dropped attach/detach strands that session's owner for good."""
- policy, session = self.make_policy(), self.make_session()
- for event in LIFECYCLE_SAMPLES:
- assert not isinstance(policy.decide(event, session), Drop)
-
- def test_lifecycle_survives_a_burst_whatever_the_rate_budget(self) -> None:
- """A frozen clock never refills a budget, so a rule that could ever throttle
- lifecycle drops one here."""
- policy, session = self.make_policy(), self.make_session()
- for _ in range(200):
- for event in LIFECYCLE_SAMPLES:
- assert not isinstance(policy.decide(event, session), Drop)
-
- def test_lifecycle_events_keep_their_routing_identity(self) -> None:
- """Not dropping a lifecycle event is not enough for the client to stay attached.
-
- The driving adapter learns which session an attach/detach belongs to from the
- event's method and params.sessionId, and it reads them AFTER this gate. An event
- delivered with that identity rewritten away is undeliverable in exactly the way a
- dropped one is: the owner is stranded either way (SKY-12500). So a policy may
- rewrite a lifecycle event, but never into something the router can no longer
- place.
- """
- policy, session = self.make_policy(), self.make_session()
- for event in LIFECYCLE_SAMPLES:
- decision = policy.decide(event, session)
- assert not isinstance(decision, Drop)
- if isinstance(decision, Rewrite):
- assert decision.event.method == event.method
- assert (decision.event.params or {}).get("sessionId") == (event.params or {}).get("sessionId")
- assert decision.event.session_id == event.session_id
-
- def test_drop_decisions_carry_a_closed_set_reason(self) -> None:
- """SKY-12510: a reason becomes a metric tag, so a free-form one is unbounded
- label cardinality."""
- policy, session = self.make_policy(), self.make_session()
- for event in SAMPLE_EVENTS:
- decision = policy.decide(event, session)
- if isinstance(decision, Drop):
- assert isinstance(decision.reason, DropReason)
-
- def test_observe_command_accepts_whatever_a_client_sends(self) -> None:
- policy, session = self.make_policy(), self.make_session()
- for command in SAMPLE_COMMANDS:
- policy.observe_command(command, session)
-
- def test_forget_is_idempotent_and_safe_for_unseen_sessions(self) -> None:
- policy, session = self.make_policy(), self.make_session()
- policy.forget("never-seen-at-all")
- policy.decide(SAMPLE_EVENTS[0], session)
- policy.observe_command(SAMPLE_COMMANDS[0], session)
- policy.forget(session.session_id)
- policy.forget(session.session_id)
-
- def test_decide_still_answers_after_the_session_is_forgotten(self) -> None:
- """forget() releases state, it does not poison the port: a late event may
- still arrive for a session already torn down."""
- policy, session = self.make_policy(), self.make_session()
- policy.decide(SAMPLE_EVENTS[0], session)
- policy.forget(session.session_id)
- assert isinstance(policy.decide(SAMPLE_EVENTS[0], session), (Forward, Drop, Rewrite))
-
-
-class TestForwardAllEventPolicyContract(EventPolicyPortContract):
- def make_policy(self) -> EventPolicyPort:
- return ForwardAllEventPolicy()
-
-
-class TestPassThroughEngineContract(EventPolicyPortContract):
- """The engine with no rules configured — the default deployment shape."""
-
- def make_policy(self) -> EventPolicyPort:
- return EventPolicyEngine(EventPolicyConfig(), frozen_clock())
-
-
-class TestNoisyPackEngineContract(EventPolicyPortContract):
- def make_policy(self) -> EventPolicyPort:
- return EventPolicyEngine(NOISY_EVENT_PACK_V1, frozen_clock())
-
-
-class TestScreencastPackEngineContract(EventPolicyPortContract):
- def make_policy(self) -> EventPolicyPort:
- return EventPolicyEngine(SCREENCAST_PACK_V1, frozen_clock())
diff --git a/tests/unit/proxy/test_frames.py b/tests/unit/proxy/test_frames.py
deleted file mode 100644
index 663bed66c..000000000
--- a/tests/unit/proxy/test_frames.py
+++ /dev/null
@@ -1,364 +0,0 @@
-from __future__ import annotations
-
-import json
-import random
-from pathlib import Path
-
-import pytest
-
-from skyvern.proxy.core.frames import (
- PROXY_CLIENT_KEY,
- CdpCommand,
- CdpEvent,
- CdpResponse,
- FrameDecodeError,
- RemapperFullError,
- RequestIdRemapper,
- decode_frame,
- encode_frame,
-)
-
-FIXTURES_DIR = Path(__file__).parent / "fixtures"
-
-
-def test_decode_command_response_event() -> None:
- assert decode_frame('{"id": 1, "method": "Page.enable", "sessionId": "s1"}') == CdpCommand(
- id=1, method="Page.enable", session_id="s1"
- )
- assert decode_frame('{"id": 1, "result": {"ok": true}}') == CdpResponse(id=1, result={"ok": True})
- assert decode_frame('{"method": "Page.loadEventFired", "params": {"timestamp": 1}}') == CdpEvent(
- method="Page.loadEventFired", params={"timestamp": 1}
- )
-
-
- "raw",
- [r'{"id":1,"method":"\ud800"}', r'{"method":"Page.loadEventFired","sessionId":"\ud800"}'],
-)
-def test_decode_rejects_lone_utf16_surrogates(raw: str) -> None:
- with pytest.raises(FrameDecodeError):
- decode_frame(raw)
-
-
- "raw",
- [
- '{"id":1,"method":"Page.enable","params":{"value":1e400}}',
- '{"id":1,"method":"Page.enable","params":{"value":-1e400}}',
- '{"id":1,"result":{"value":1e400}}',
- '{"id":1,"result":{"value":-1e400}}',
- ],
-)
-def test_decode_rejects_float_overflow(raw: str) -> None:
- with pytest.raises(FrameDecodeError):
- decode_frame(raw)
-
-
- "raw",
- [
- "",
- " \n",
- b"",
- b"\x80\x81",
- "not json",
- "[1]",
- "null",
- "true",
- "{}",
- '{"id": 1}',
- '{"id": null, "method": "Page.enable"}',
- '{"id": true, "method": "Page.enable"}',
- '{"id": "1", "method": "Page.enable"}',
- '{"id": 1.5, "method": "Page.enable"}',
- '{"id": 1, "method": ""}',
- '{"id": 1, "method": 7}',
- '{"id": 1, "method": "Page.enable", "params": null}',
- '{"id": 1, "method": "Page.enable", "params": []}',
- '{"id": 1, "method": "Page.enable", "sessionId": null}',
- '{"id": 1, "method": "Page.enable", "sessionId": ""}',
- '{"id": 1, "method": "Page.enable", "result": {}}',
- '{"id": 1, "result": null}',
- '{"id": 1, "result": []}',
- '{"id": 1, "result": {}, "error": {"code": -1, "message": "bad"}}',
- '{"id": 1, "error": {"code": "-1", "message": "bad"}}',
- '{"id": 1, "error": {"code": -1}}',
- '{"method": "Page.loadEventFired", "params": null}',
- '{"method": "Page.loadEventFired", "params": []}',
- '{"method": "Page.loadEventFired", "sessionId": 1}',
- '{"id": 1, "result": {}, "unexpected": true}',
- '{"id": 1, "method": "Page.enable", "id": 2}',
- ],
-)
-def test_decode_rejects_malformed_frames(raw: str | bytes) -> None:
- with pytest.raises(FrameDecodeError):
- decode_frame(raw)
-
-
[email protected]("fixture_name", ["browser_client_a.jsonl", "browser_client_b.jsonl"])
-def test_recorded_frames_round_trip_semantically(fixture_name: str) -> None:
- raw_frames = [line for line in (FIXTURES_DIR / fixture_name).read_text().splitlines() if line]
-
- assert raw_frames
- for raw in raw_frames:
- decoded = decode_frame(raw)
- encoded = encode_frame(decoded)
- encoded.encode("utf-8")
- assert decode_frame(encoded) == decoded
-
-
- "frame",
- [
- CdpCommand(id=1, method="Runtime.evaluate", params={"expression": "café"}),
- CdpResponse(id=1, result={"value": "café"}),
- CdpEvent(method="Runtime.consoleAPICalled", params={"text": "café"}),
- ],
-)
-def test_generated_valid_frames_round_trip(frame: CdpCommand | CdpResponse | CdpEvent) -> None:
- encoded = encode_frame(frame)
- encoded.encode("utf-8")
- assert decode_frame(encoded) == frame
-
-
-def test_mutated_bytes_only_raise_frame_decode_error() -> None:
- randomizer = random.Random(12506)
- seeds = [
- b"\x80",
- b"{}",
- b'{"id": "one"}',
- b'{"method": null}',
- b'{"id": 1, "method": "Page.enable", "params": []}',
- ]
- mutations: list[bytes] = []
-
- for seed in seeds:
- for _ in range(32):
- mutated = bytearray(seed)
- index = randomizer.randrange(len(mutated))
- mutated[index] ^= randomizer.randrange(1, 256)
- mutations.append(bytes(mutated))
-
- cut = randomizer.randrange(len(seed) + 1)
- mutations.append(seed[:cut])
- insert_at = randomizer.randrange(len(seed) + 1)
- inserted = bytes((randomizer.randrange(256),))
- mutations.append(seed[:insert_at] + inserted + seed[insert_at:])
-
- base = b'{"id": 1, "method": "Runtime.enable", "params": {}}'
- mutations.extend(
- [
- base.replace(b'"id": 1', b'"id": null'),
- base.replace(b'"id": 1', b'"id": "1"'),
- base.replace(b'"method": "Runtime.enable"', b'"method": null'),
- base.replace(b'"params": {}', b'"params": []'),
- base.replace(b'"params": {}', b'"params": null'),
- base[:-1],
- base[:-1] + b",}",
- b'{"id":1,"result":{},"error":{"code":-1,"message":"bad"}}',
- b'{"id":1,"method":"Runtime.enable","method":"Page.enable"}',
- b'{"id":1,"method":"Runtime.enable","unexpected":true}',
- b'{"id":1}',
- b'{"method":""}',
- ]
- )
-
- for raw in mutations:
- try:
- decoded = decode_frame(raw)
- except FrameDecodeError:
- continue
- except Exception as exc:
- pytest.fail(f"unexpected exception for mutated frame {raw!r}: {exc!r}")
- assert isinstance(decoded, (CdpCommand, CdpResponse, CdpEvent))
-
-
-def test_encode_roundtrip() -> None:
- for frame in (
- CdpCommand(id=3, method="Target.attachToTarget", params={"targetId": "t", "flatten": True}, session_id="s"),
- CdpResponse(id=3, result={"sessionId": "s2"}),
- CdpEvent(method="Target.attachedToTarget", params={}, session_id="s"),
- ):
- assert decode_frame(encode_frame(frame)) == frame
-
-
-def test_error_response_omits_result() -> None:
- payload = json.loads(encode_frame(CdpResponse(id=4, error={"code": -32000, "message": "nope"})))
- assert payload == {"id": 4, "error": {"code": -32000, "message": "nope"}}
-
-
-def test_request_id_remapper_roundtrip_and_unknown_ids() -> None:
- remapper = RequestIdRemapper()
- upstream_a = remapper.to_upstream("client-a", CdpCommand(id=1, method="Page.enable"))
- upstream_b = remapper.to_upstream("client-b", CdpCommand(id=1, method="Page.enable"))
- assert upstream_a.id != upstream_b.id
-
- mapped = remapper.to_client(CdpResponse(id=upstream_b.id, result={}))
- assert mapped is not None
- client_key, response = mapped
- assert client_key == "client-b"
- assert response.id == 1
-
- assert remapper.to_client(CdpResponse(id=upstream_b.id, result={})) is None
- assert remapper.to_client(CdpResponse(id=9999, result={})) is None
-
-
-def test_request_id_remapper_cleanup_is_client_scoped() -> None:
- remapper = RequestIdRemapper()
- first = remapper.to_upstream("client-a", CdpCommand(id=1, method="Page.enable"))
- second = remapper.to_upstream("client-a", CdpCommand(id=2, method="Runtime.enable"))
- remapper.to_upstream("client-b", CdpCommand(id=1, method="Network.enable"))
-
- assert remapper.pending_count == 3
- assert remapper.clear_client("client-a") == 2
- assert remapper.pending_count == 1
- assert remapper.to_client(CdpResponse(id=first.id, result={})) is None
- assert remapper.to_client(CdpResponse(id=second.id, result={})) is None
- assert remapper.clear_client("client-a") == 0
- remapper.clear()
- assert remapper.pending_count == 0
-
-
-def test_request_id_remapper_bounds_pending_mappings() -> None:
- # SKY-12500 (AC5): a command whose response never arrives must not pin an entry
- # forever. The oldest pending mapping is evicted once the cap is reached, so a
- # long-lived client cannot grow the table without bound.
- remapper = RequestIdRemapper(max_pending=3)
- oldest = remapper.to_upstream("client-a", CdpCommand(id=1, method="Page.enable"))
- for client_id in range(2, 6):
- remapper.to_upstream("client-a", CdpCommand(id=client_id, method="Page.enable"))
-
- assert remapper.pending_count == 3
- # The evicted command's late response is unmatched (dropped), never mis-routed.
- assert remapper.to_client(CdpResponse(id=oldest.id, result={})) is None
-
-
-def test_a_client_at_the_cap_evicts_only_its_own_pending_request() -> None:
- # SKY-12500: the flooder pays its own tax. At the cap a client reclaims its own
- # oldest mapping — a co-tenant's admitted response is never the thing dropped.
- remapper = RequestIdRemapper(max_pending=2)
- victim = remapper.to_upstream("client-a", CdpCommand(id=1, method="Page.enable"))
- flooder_first = remapper.to_upstream("client-b", CdpCommand(id=1, method="Network.enable"))
- flooder_second = remapper.to_upstream("client-b", CdpCommand(id=2, method="Runtime.enable"))
-
- assert len({victim.id, flooder_first.id, flooder_second.id}) == 3
- # B's oldest went, not A's.
- assert remapper.to_client(CdpResponse(id=flooder_first.id, result={})) is None
- assert remapper.to_client(CdpResponse(id=victim.id, result={})) == ("client-a", CdpResponse(id=1, result={}))
-
-
-def test_a_full_table_refuses_a_client_with_nothing_of_its_own_to_evict() -> None:
- # Rather than silently drop a co-tenant's live mapping, the newcomer is refused
- # (its caller closes it) — a lost response would desync the victim invisibly.
- remapper = RequestIdRemapper(max_pending=2)
- kept_a = remapper.to_upstream("client-a", CdpCommand(id=1, method="Page.enable"))
- kept_b = remapper.to_upstream("client-b", CdpCommand(id=1, method="Network.enable"))
-
- with pytest.raises(RemapperFullError):
- remapper.to_upstream("client-c", CdpCommand(id=1, method="Runtime.enable"))
-
- # Both incumbents still resolve to their own owners.
- assert remapper.to_client(CdpResponse(id=kept_a.id, result={})) == ("client-a", CdpResponse(id=1, result={}))
- assert remapper.to_client(CdpResponse(id=kept_b.id, result={})) == ("client-b", CdpResponse(id=1, result={}))
-
-
-def test_a_flooding_client_never_evicts_the_proxy_lane() -> None:
- # The proxy's own in-flight command must survive any client's flood; losing it
- # would strand proxy state on a response that never correlates.
- remapper = RequestIdRemapper(max_pending=2)
- proxy = remapper.to_upstream_as_proxy(CdpCommand(id=1, method="Target.setAutoAttach"))
- remapper.to_upstream("client-a", CdpCommand(id=1, method="Page.enable"))
- for client_id in range(2, 8):
- remapper.to_upstream("client-a", CdpCommand(id=client_id, method="Page.enable"))
-
- mapped = remapper.to_client(CdpResponse(id=proxy.id, result={}))
- assert mapped is not None and mapped[0] == PROXY_CLIENT_KEY
-
-
-def test_the_proxy_lane_is_refused_rather_than_evicting_a_client() -> None:
- remapper = RequestIdRemapper(max_pending=1)
- kept = remapper.to_upstream("client-a", CdpCommand(id=1, method="Page.enable"))
-
- with pytest.raises(RemapperFullError):
- remapper.to_upstream_as_proxy(CdpCommand(id=1, method="Target.setAutoAttach"))
-
- assert remapper.to_client(CdpResponse(id=kept.id, result={})) == ("client-a", CdpResponse(id=1, result={}))
-
-
-def test_discard_frees_a_pending_mapping_whose_response_is_suppressed() -> None:
- # SKY-12500 (AC5): a response the pipeline swallows must still free its mapping.
- remapper = RequestIdRemapper()
- upstream = remapper.to_upstream("client-a", CdpCommand(id=1, method="Page.enable"))
- assert remapper.pending_count == 1
-
- remapper.discard(upstream.id)
- assert remapper.pending_count == 0
- assert remapper.to_client(CdpResponse(id=upstream.id, result={})) is None
- remapper.discard(upstream.id) # idempotent
-
-
-def test_proxy_lane_is_reserved_and_never_claimed_by_a_client() -> None:
- # SKY-12500 (AC2): proxy-issued commands ride a lane no client can claim or observe.
- remapper = RequestIdRemapper()
- with pytest.raises(ValueError):
- remapper.to_upstream(PROXY_CLIENT_KEY, CdpCommand(id=1, method="Target.setAutoAttach"))
-
- upstream = remapper.to_upstream_as_proxy(CdpCommand(id=1, method="Target.setAutoAttach"))
- mapped = remapper.to_client(CdpResponse(id=upstream.id, result={}))
- assert mapped is not None
- assert mapped[0] == PROXY_CLIENT_KEY
-
-
-def test_proxy_and_client_commands_share_one_upstream_id_space() -> None:
- # A proxy command must never collide with a client's in-flight id: a collision
- # would hand the proxy's response to a client (or swallow the client's).
- remapper = RequestIdRemapper()
- client = remapper.to_upstream("client-a", CdpCommand(id=1, method="Page.enable"))
- proxy = remapper.to_upstream_as_proxy(CdpCommand(id=1, method="Target.setAutoAttach"))
- assert client.id != proxy.id
-
- assert remapper.to_client(CdpResponse(id=client.id, result={})) == (
- "client-a",
- CdpResponse(id=1, result={}),
- )
- proxy_mapped = remapper.to_client(CdpResponse(id=proxy.id, result={}))
- assert proxy_mapped is not None and proxy_mapped[0] == PROXY_CLIENT_KEY
-
-
-def test_clear_client_cannot_evict_the_proxy_lane() -> None:
- remapper = RequestIdRemapper()
- proxy = remapper.to_upstream_as_proxy(CdpCommand(id=1, method="Target.setAutoAttach"))
- with pytest.raises(ValueError):
- remapper.clear_client(PROXY_CLIENT_KEY)
-
- mapped = remapper.to_client(CdpResponse(id=proxy.id, result={}))
- assert mapped is not None and mapped[0] == PROXY_CLIENT_KEY
-
-
-def test_request_id_remapper_seeded_multi_client_property() -> None:
- randomizer = random.Random(12506)
- remapper = RequestIdRemapper()
- pending: dict[int, tuple[str, int]] = {}
-
- for client_index in range(24):
- client_key = f"client-{client_index}"
- for _ in range(19):
- original_id = randomizer.randint(1, 7)
- command = CdpCommand(id=original_id, method="Runtime.enable")
- upstream = remapper.to_upstream(client_key, command)
- assert upstream.id not in pending
- pending[upstream.id] = (client_key, original_id)
-
- assert remapper.pending_count == len(pending)
- upstream_ids = list(pending)
- randomizer.shuffle(upstream_ids)
- for upstream_id in upstream_ids:
- client_key, original_id = pending[upstream_id]
- assert remapper.to_client(CdpResponse(id=upstream_id, result={"request": upstream_id})) == (
- client_key,
- CdpResponse(id=original_id, result={"request": upstream_id}),
- )
-
- assert remapper.pending_count == 0
- assert len(remapper) == 0
- assert remapper.to_client(CdpResponse(id=upstream_ids[0], result={})) is None
diff --git a/tests/unit/proxy/test_interception.py b/tests/unit/proxy/test_interception.py
deleted file mode 100644
index 4f5b19471..000000000
--- a/tests/unit/proxy/test_interception.py
+++ /dev/null
@@ -1,332 +0,0 @@
-"""Command interception through the live relay (SKY-12535).
-
-The seam's promises, proven against the shared-connection server: disabled means
-byte-identical pass-through; a synthesized response answers the client under its
-own request id with nothing forwarded; a proxy-synthesized request rides the
-reserved id lane and its reply never reaches a client; rewrites forward through
-remapping; a failing interceptor fails the command closed; and an interceptor
-never sees a command the ownership check refuses.
-"""
-
-from __future__ import annotations
-
-import asyncio
-
-import pytest
-
-from skyvern.proxy.__main__ import build_interceptors, build_pipeline
-from skyvern.proxy.adapters.memory import AllowAllAuth, ForwardAllEventPolicy, InMemorySessionRegistry, NoOpMetrics
-from skyvern.proxy.adapters.websocket_server import CdpProxyServer
-from skyvern.proxy.core.frames import CdpCommand, CdpResponse, decode_frame
-from skyvern.proxy.core.interception_demo import (
- DEMO_GET_VERSION_RESULT,
- DEMO_INTERCEPT_REASON,
- GET_VERSION_METHOD,
- demo_get_version_interceptor,
-)
-from skyvern.proxy.core.pipeline import (
- INTERCEPTOR_FAILURE_CODE,
- InterceptContext,
- MiddlewarePipeline,
- SynthesizedResponse,
-)
-from skyvern.proxy.core.session import ProxySession, ResolvedSession
-from tests.unit.proxy.test_metrics_contract import _RecordingMetrics
-from tests.unit.proxy.test_shared_connection import (
- UPSTREAM_URL,
- _command,
- _ControllableUpstream,
- _ScriptClient,
- _SharedBrowser,
- _until,
-)
-
-
-def _intercept_server(
- browser: object, pipeline: MiddlewarePipeline | None = None, metrics: object | None = None
-) -> CdpProxyServer:
- sessions = InMemorySessionRegistry()
- sessions.put(ResolvedSession(session_id="s1", upstream_adapter="memory", upstream_ws_url=UPSTREAM_URL))
- return CdpProxyServer(
- upstream=browser, # type: ignore[arg-type]
- sessions=sessions,
- auth=AllowAllAuth(),
- metrics=metrics or NoOpMetrics(), # type: ignore[arg-type]
- event_policy=ForwardAllEventPolicy(),
- pipeline=pipeline,
- )
-
-
-def _demo_pipeline() -> MiddlewarePipeline:
- return MiddlewarePipeline(interceptors=[demo_get_version_interceptor])
-
-
-def _upstream_methods(connection: _ControllableUpstream) -> list[str]:
- return [f.method for f in map(decode_frame, connection.sent) if isinstance(f, CdpCommand)]
-
-
-def _responses(client: _ScriptClient) -> list[CdpResponse]:
- return [f for f in map(decode_frame, client.sent) if isinstance(f, CdpResponse)]
-
-
-def test_build_interceptors_is_a_closed_switch() -> None:
- assert build_interceptors("") == ()
- assert build_interceptors("demo-get-version") == (demo_get_version_interceptor,)
- assert build_pipeline("forward-all").has_interceptors is False
- assert build_pipeline("forward-all", "demo-get-version").has_interceptors is True
- assert build_pipeline("screencast-v1", "demo-get-version").has_interceptors is True
- with pytest.raises(ValueError):
- build_interceptors("bogus")
-
-
-async def test_demo_interceptor_passes_other_methods_untouched() -> None:
- sent: list[CdpCommand] = []
-
- async def send_proxy_command(command: CdpCommand) -> None:
- sent.append(command)
-
- command = CdpCommand(id=1, method="Page.enable")
- session = ProxySession(session_id="s1", upstream_ws_url=UPSTREAM_URL)
-
- outcome = await demo_get_version_interceptor(command, session, InterceptContext(send_proxy_command))
-
- assert outcome is command
- assert sent == []
-
-
-async def test_disabled_interception_forwards_get_version_upstream() -> None:
- connection = _ControllableUpstream()
- server = _intercept_server(_SharedBrowser(connection))
- client = _ScriptClient([_command(1, GET_VERSION_METHOD)])
-
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: client.echoes())
-
- # Pass-through unchanged: the browser answered, the proxy synthesized nothing.
- assert client.echoes() == {GET_VERSION_METHOD}
- assert _upstream_methods(connection) == [GET_VERSION_METHOD]
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_demo_synthesizes_the_response_under_the_clients_own_id() -> None:
- connection = _ControllableUpstream()
- metrics = _RecordingMetrics()
- server = _intercept_server(_SharedBrowser(connection), _demo_pipeline(), metrics)
- client = _ScriptClient([_command(5, GET_VERSION_METHOD)])
-
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: client.sent)
-
- # RESPONSE synthesis: answered locally, under the client's own id.
- [response] = _responses(client)
- assert response.id == 5
- assert response.result == DEMO_GET_VERSION_RESULT
- # REQUEST synthesis: the upstream still saw one getVersion — the proxy's own,
- # whose echo reply was consumed on the proxy lane and never sent to the client.
- assert _upstream_methods(connection) == [GET_VERSION_METHOD]
- assert len(client.sent) == 1
- # Remapping is unbroken: once the proxy-lane reply lands, no mapping dangles.
- shared = server._shared_upstreams[UPSTREAM_URL]
- assert await _until(lambda: shared.remapper.pending_count == 0)
- # The audit trail names the interception.
- reasons = [tags.get("reason") for op, name, _, tags in metrics.calls if name.endswith("commands_intercepted")]
- assert reasons == [DEMO_INTERCEPT_REASON]
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_interception_does_not_disturb_a_co_tenants_remapping() -> None:
- connection = _ControllableUpstream()
- server = _intercept_server(_SharedBrowser(connection), _demo_pipeline())
- # Both clients use id 1: one is answered by the proxy, one by the browser.
- client_a = _ScriptClient([_command(1, GET_VERSION_METHOD)])
- client_b = _ScriptClient([_command(1, "Page.enable")])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: client_a.sent and client_b.echoes())
-
- [synthesized] = _responses(client_a)
- assert synthesized.id == 1
- assert synthesized.result == DEMO_GET_VERSION_RESULT
- assert client_b.echoes() == {"Page.enable"}
- assert all(decode_frame(wire).id == 1 for wire in client_b.sent)
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
-
-async def test_a_rewriting_interceptor_forwards_through_remapping() -> None:
- async def bound_params(command: CdpCommand, session: ProxySession, context: InterceptContext) -> CdpCommand:
- if command.method != "Page.navigate":
- return command
- return CdpCommand(id=command.id, method=command.method, params={**(command.params or {}), "bounded": True})
-
- connection = _ControllableUpstream()
- server = _intercept_server(_SharedBrowser(connection), MiddlewarePipeline(interceptors=[bound_params]))
- client = _ScriptClient([_command(3, "Page.navigate", {"url": "about:blank"})])
-
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: client.echoes())
-
- [forwarded] = [f for f in map(decode_frame, connection.sent) if isinstance(f, CdpCommand)]
- assert forwarded.params == {"url": "about:blank", "bounded": True}
- [response] = _responses(client)
- assert response.id == 3
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_a_failing_interceptor_fails_the_command_closed() -> None:
- async def broken(command: CdpCommand, session: ProxySession, context: InterceptContext) -> CdpCommand:
- raise RuntimeError("interceptor bug")
-
- connection = _ControllableUpstream()
- metrics = _RecordingMetrics()
- server = _intercept_server(_SharedBrowser(connection), MiddlewarePipeline(interceptors=[broken]), metrics)
- client = _ScriptClient([_command(4, "Page.enable")])
-
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: client.sent)
-
- # Deterministic error to the client, nothing forwarded upstream.
- [response] = _responses(client)
- assert response.id == 4
- assert response.error is not None
- assert response.error["code"] == INTERCEPTOR_FAILURE_CODE
- assert connection.sent == []
- reasons = [tags.get("reason") for op, name, _, tags in metrics.calls if name.endswith("commands_intercepted")]
- assert reasons == ["interceptor_error"]
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_an_invalid_synthesis_fails_closed_without_killing_the_relay() -> None:
- async def bad_synthesis(command: CdpCommand, session: ProxySession, context: InterceptContext) -> object:
- if command.method == "Page.enable":
- return SynthesizedResponse(result={"blob": b"not-json"})
- return command
-
- connection = _ControllableUpstream()
- server = _intercept_server(_SharedBrowser(connection), MiddlewarePipeline(interceptors=[bad_synthesis]))
- client = _ScriptClient([_command(1, "Page.enable"), _command(2, "Runtime.enable")])
-
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: len(client.sent) >= 2)
-
- # The bad synthesis became the deterministic error, and the relay survived to
- # serve the next command normally.
- [failure] = [r for r in _responses(client) if r.error is not None]
- assert failure.id == 1
- assert failure.error["code"] == INTERCEPTOR_FAILURE_CODE
- assert client.echoes() == {"Runtime.enable"}
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_a_failed_proxy_lane_send_leaves_no_mapping_behind() -> None:
- async def sends_unencodable(command: CdpCommand, session: ProxySession, context: InterceptContext) -> object:
- if command.method == "Page.enable":
- # Encoding fails after the mapping is allocated; the closure must
- # discard it or repeated failures eat the shared table for good.
- await context.send_proxy_command(CdpCommand(id=0, method="Proxy.probe", params={"blob": b"x"}))
- return command
-
- connection = _ControllableUpstream()
- server = _intercept_server(_SharedBrowser(connection), MiddlewarePipeline(interceptors=[sends_unencodable]))
- client = _ScriptClient([_command(1, "Page.enable")])
-
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: client.sent)
-
- [failure] = _responses(client)
- assert failure.id == 1
- assert failure.error is not None
- assert failure.error["code"] == INTERCEPTOR_FAILURE_CODE
- shared = server._shared_upstreams[UPSTREAM_URL]
- assert shared.remapper.pending_count == 0
- assert connection.sent == []
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_an_interceptor_never_sees_a_foreign_session_command() -> None:
- seen: list[tuple[str, str | None]] = []
-
- async def recorder(command: CdpCommand, session: ProxySession, context: InterceptContext) -> CdpCommand:
- seen.append((command.method, command.session_id))
- return command
-
- connection = _ControllableUpstream()
- server = _intercept_server(_SharedBrowser(connection), MiddlewarePipeline(interceptors=[recorder]))
- client_a = _ScriptClient([_command(1, "Target.attachToTarget", {"targetId": "tA", "flatten": True})])
- client_b = _ScriptClient([])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- # Wait until A owns session tA (its attach response landed), then B intrudes.
- assert await _until(lambda: client_a.sent)
- client_b.feed(_command(2, "Runtime.evaluate", {"expression": "1"}))
- # Give B's command a distinct marker frame so we know it was processed.
- client_b.feed(_command(3, "Runtime.evaluate"))
- shared = server._shared_upstreams[UPSTREAM_URL]
- assert await _until(lambda: len(seen) >= 2)
- seen.clear()
-
- client_b.feed(
- # Addressed to A's session: refused with CDP's own session-not-found,
- # without the interceptor ever seeing it.
- '{"id": 9, "method": "Runtime.evaluate", "sessionId": "tA"}'
- )
- assert await _until(lambda: any(r.id == 9 for r in _responses(client_b)))
-
- [refusal] = [r for r in _responses(client_b) if r.id == 9]
- assert refusal.error is not None
- assert refusal.error["code"] == -32001
- assert seen == []
- assert shared.session_owner.get("tA") == "c1"
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
-
-async def test_a_synthesizing_interceptor_leaves_no_upstream_state_behind() -> None:
- async def answer(command: CdpCommand, session: ProxySession, context: InterceptContext) -> SynthesizedResponse:
- return SynthesizedResponse(result={"ok": True}, reason="test")
-
- connection = _ControllableUpstream()
- server = _intercept_server(_SharedBrowser(connection), MiddlewarePipeline(interceptors=[answer]))
- client = _ScriptClient([_command(1, "Page.enable"), _command(2, "Page.enable")])
-
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: len(client.sent) == 2)
-
- shared = server._shared_upstreams[UPSTREAM_URL]
- # No forward, no remap entry, no latency start, no attach intent: the command
- # was answered before any of those tables could learn about it.
- assert connection.sent == []
- assert shared.remapper.pending_count == 0
- assert shared.command_starts == {}
- assert shared.attach_intents == []
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
diff --git a/tests/unit/proxy/test_local_chrome.py b/tests/unit/proxy/test_local_chrome.py
deleted file mode 100644
index 1e7fedb44..000000000
--- a/tests/unit/proxy/test_local_chrome.py
+++ /dev/null
@@ -1,111 +0,0 @@
-"""Launch-failure taxonomy tests for the local headless-Chrome adapter.
-
-These run without a real browser; the full port contract (needs Chrome) lives in
-test_upstream_browser_port_contract.py behind a skipif.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import os
-import tempfile
-from pathlib import Path
-
-import pytest
-
-from skyvern.proxy.adapters import local_chrome
-from skyvern.proxy.adapters.local_chrome import LocalChromeUpstreamBrowser
-from skyvern.proxy.core.errors import LaunchEnvironmentError, LaunchTimeoutError
-from skyvern.proxy.core.session import ProxySession
-
-
-def _session() -> ProxySession:
- return ProxySession(session_id="test-session", upstream_ws_url="ws://localhost:0/unused")
-
-
-def _fake_chrome(tmp_path: Path, body: str) -> str:
- script = tmp_path / "fake-chrome"
- script.write_text(f"#!/bin/sh\n{body}\n")
- script.chmod(0o755)
- return str(script)
-
-
-async def test_no_executable_discovered_raises_launch_environment(monkeypatch: pytest.MonkeyPatch) -> None:
- monkeypatch.setattr(local_chrome, "find_local_chrome_executable", lambda: None)
- with pytest.raises(LaunchEnvironmentError):
- await LocalChromeUpstreamBrowser().connect(_session())
-
-
-async def test_missing_executable_raises_launch_environment(tmp_path: Path) -> None:
- port = LocalChromeUpstreamBrowser(executable_path=str(tmp_path / "no-such-chrome"))
- with pytest.raises(LaunchEnvironmentError):
- await port.connect(_session())
-
-
-async def test_early_exit_raises_launch_environment(tmp_path: Path) -> None:
- port = LocalChromeUpstreamBrowser(executable_path=_fake_chrome(tmp_path, "exit 3"))
- with pytest.raises(LaunchEnvironmentError):
- await port.connect(_session())
-
-
-async def test_never_ready_process_raises_launch_timeout(tmp_path: Path) -> None:
- port = LocalChromeUpstreamBrowser(
- executable_path=_fake_chrome(tmp_path, "exec sleep 30"), launch_timeout_seconds=0.2
- )
- with pytest.raises(LaunchTimeoutError):
- await port.connect(_session())
-
-
-async def test_mkdtemp_failure_raises_launch_environment(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
- def failing_mkdtemp(*args: object, **kwargs: object) -> str:
- raise OSError("disk full")
-
- monkeypatch.setattr(tempfile, "mkdtemp", failing_mkdtemp)
- port = LocalChromeUpstreamBrowser(executable_path=str(tmp_path / "unused-chrome"))
- with pytest.raises(LaunchEnvironmentError):
- await port.connect(_session())
-
-
[email protected]("port_line", ["notaport", "99999"])
-async def test_garbage_devtools_port_file_raises_launch_environment(tmp_path: Path, port_line: str) -> None:
- body = f'udd="${{3#--user-data-dir=}}"\nprintf \'{port_line}\\n/devtools/browser/x\\n\' > "$udd/DevToolsActivePort"\nexec sleep 30'
- port = LocalChromeUpstreamBrowser(executable_path=_fake_chrome(tmp_path, body), launch_timeout_seconds=5)
- with pytest.raises(LaunchEnvironmentError):
- await port.connect(_session())
-
-
-async def test_cancelled_connect_reaps_process_and_tmpdir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
- pid_file = tmp_path / "chrome.pid"
- executable = _fake_chrome(tmp_path, f'echo $$ > "{pid_file}"\nexec sleep 30')
- created_dirs: list[str] = []
- real_mkdtemp = tempfile.mkdtemp
-
- def tracking_mkdtemp(*args: object, **kwargs: object) -> str:
- path = real_mkdtemp(*args, **kwargs) # type: ignore[arg-type]
- created_dirs.append(path)
- return path
-
- monkeypatch.setattr(tempfile, "mkdtemp", tracking_mkdtemp)
- port = LocalChromeUpstreamBrowser(executable_path=executable, launch_timeout_seconds=30)
- task = asyncio.create_task(port.connect(_session()))
- for _ in range(500):
- if pid_file.exists() and pid_file.read_text().strip():
- break
- await asyncio.sleep(0.01)
- else:
- task.cancel()
- pytest.fail("fake chrome never started")
- task.cancel()
- with pytest.raises(asyncio.CancelledError):
- await task
- with pytest.raises(ProcessLookupError):
- os.kill(int(pid_file.read_text()), 0)
- assert created_dirs
- assert not Path(created_dirs[0]).exists()
diff --git a/tests/unit/proxy/test_metrics_contract.py b/tests/unit/proxy/test_metrics_contract.py
deleted file mode 100644
index 36c712d69..000000000
--- a/tests/unit/proxy/test_metrics_contract.py
+++ /dev/null
@@ -1,633 +0,0 @@
-"""Reusable contract suite for MetricsPort adapters.
-
-Any adapter (the OSS NoOp here, the cloud OTel adapter under tests/cloud/)
-subclasses MetricsPortContract and overrides make_metrics()/collect(); every
-guarantee the proxy relies on — a pass-through session vs a filtered/errored one
-is distinguishable via metrics, an active-session gauge that balances, and no
-token or upstream URL in any label — is asserted once, here.
-
-The NoOp subclass cannot record, so collect() returns None and the recording
-assertions are skipped; what still runs against NoOp is that driving every
-instrumented path calls the port cleanly (no crash, all ops accepted).
-"""
-
-from __future__ import annotations
-
-import asyncio
-from collections import namedtuple
-from types import SimpleNamespace
-from typing import Iterable, Iterator, Mapping
-
-import pytest
-
-from skyvern.proxy.adapters.memory import ForwardAllEventPolicy, InMemorySessionRegistry, NoOpMetrics, StaticKeyAuth
-from skyvern.proxy.adapters.websocket_server import (
- _MAX_PENDING_COMMANDS,
- CdpProxyServer,
- _track_command,
-)
-from skyvern.proxy.core.errors import UpstreamConnectError
-from skyvern.proxy.core.frames import CdpCommand, CdpEvent, CdpResponse, RequestIdRemapper, decode_frame, encode_frame
-from skyvern.proxy.core.pipeline import MiddlewarePipeline
-from skyvern.proxy.core.policy import Drop, DropReason, PolicyDecision
-from skyvern.proxy.core.session import Principal, ProxySession, ResolvedSession, UpstreamClosedError
-from skyvern.proxy.ports import MetricsPort
-
-SESSION_ID = "pbs_metrics_session"
-ORG_ID = "o_metrics"
-VALID_KEY = "metrics-key-abc123"
-SECRET = "upstream-secret-token-do-not-leak"
-
-Record = namedtuple("Record", "name value attributes")
-
-FRAMES_RELAYED = "skyvern.cdp_proxy.frames_relayed"
-BYTES_RELAYED = "skyvern.cdp_proxy.bytes_relayed"
-FRAMES_DROPPED = "skyvern.cdp_proxy.frames_dropped"
-FRAME_DECODE_ERRORS = "skyvern.cdp_proxy.frame_decode_errors"
-COMMAND_LATENCY = "skyvern.cdp_proxy.command_latency_seconds"
-ACTIVE_SESSIONS = "skyvern.cdp_proxy.active_sessions"
-CLIENT_CONNECTED = "skyvern.cdp_proxy.client_connected"
-CLIENT_DISCONNECTED = "skyvern.cdp_proxy.client_disconnected"
-CONNECTION_REJECTED = "skyvern.cdp_proxy.connection_rejected"
-
-
-class _FakeClientWebSocket:
- def __init__(
- self,
- path: str,
- incoming: list[str] | None = None,
- headers: Mapping[str, str] | None = None,
- block: bool = True,
- ) -> None:
- self.request = SimpleNamespace(headers=dict(headers or {}), path=path)
- self.sent: list[str] = []
- self.close_code: int | None = None
- self.close_reason: str | None = None
- self._incoming = list(incoming or [])
- self._block = block
- self._drained = asyncio.Event()
-
- def __aiter__(self) -> _FakeClientWebSocket:
- return self
-
- async def __anext__(self) -> str:
- if self._incoming:
- return self._incoming.pop(0)
- # Stay open like a real client until the connection is torn down (block),
- # or end the client stream once drained (block=False) so the client side
- # can terminate a session the upstream never will (e.g. a pipeline drop).
- if self._block:
- await self._drained.wait()
- raise StopAsyncIteration
-
- async def send(self, raw: str) -> None:
- self.sent.append(raw)
-
- async def close(self, code: int = 1000, reason: str = "") -> None:
- self.close_code = code
- self.close_reason = reason
- self._drained.set()
-
-
-class _EchoThenCloseUpstreamConnection:
- """Answers each client command with a matching response, then closes so the
- upstream relay finishes only after the response is processed (deterministic)."""
-
- def __init__(self) -> None:
- self._queue: asyncio.Queue[str | None] = asyncio.Queue()
- self.received: list[str] = []
- self._closed = False
-
- async def send(self, raw: str) -> None:
- if self._closed:
- raise UpstreamClosedError("closed")
- self.received.append(raw)
- frame = decode_frame(raw)
- self._queue.put_nowait(encode_frame(CdpResponse(id=frame.id, result={})))
- self._queue.put_nowait(None)
-
- async def receive(self) -> str:
- if self._closed and self._queue.empty():
- raise UpstreamClosedError("closed")
- raw = await self._queue.get()
- if raw is None:
- raise UpstreamClosedError("closed")
- return raw
-
- async def close(self) -> None:
- self._closed = True
- self._queue.put_nowait(None)
-
-
-class _ScriptedEventUpstreamConnection:
- """Emits pre-scripted upstream frames then closes; ignores client sends."""
-
- def __init__(self, scripted: list[str]) -> None:
- self._queue: asyncio.Queue[str | None] = asyncio.Queue()
- for raw in scripted:
- self._queue.put_nowait(raw)
- self._queue.put_nowait(None)
- self._closed = False
-
- async def send(self, raw: str) -> None:
- return None
-
- async def receive(self) -> str:
- raw = await self._queue.get()
- if raw is None:
- raise UpstreamClosedError("closed")
- return raw
-
- async def close(self) -> None:
- self._closed = True
- self._queue.put_nowait(None)
-
-
-class _RaisingCloseUpstreamConnection(_EchoThenCloseUpstreamConnection):
- """Echoes like its parent but raises from close() — stands in for a real
- upstream adapter whose close() does I/O and can fail."""
-
- async def close(self) -> None:
- raise RuntimeError("upstream close failed")
-
-
-class _Upstream:
- def __init__(self, connection: object) -> None:
- self._connection = connection
-
- async def connect(self, session: ProxySession) -> object:
- return self._connection
-
-
-class _DropAllEvents:
- def decide(self, event: CdpEvent, session: ProxySession) -> PolicyDecision:
- return Drop(DropReason.POLICY)
-
- def observe_command(self, command: CdpCommand, session: ProxySession) -> None:
- return None
-
- def forget(self, session_id: str) -> None:
- return None
-
-
-class _FailingUpstream:
- async def connect(self, session: ProxySession) -> object:
- raise UpstreamConnectError("upstream dial failed")
-
-
-class _BlockingUpstreamConnection:
- """Never yields a frame; receive() blocks until close(), keeping the session
- live so the active_sessions gauge can be observed mid-flight."""
-
- def __init__(self) -> None:
- self._closed = asyncio.Event()
-
- async def send(self, raw: str) -> None:
- return None
-
- async def receive(self) -> str:
- await self._closed.wait()
- raise UpstreamClosedError("closed")
-
- async def close(self) -> None:
- self._closed.set()
-
-
-class _CaptureUpstreamConnection:
- """Records what the client->upstream leg sends; never yields upstream frames."""
-
- def __init__(self) -> None:
- self.received: list[str] = []
- self._closed = asyncio.Event()
-
- async def send(self, raw: str) -> None:
- self.received.append(raw)
-
- async def receive(self) -> str:
- await self._closed.wait()
- raise UpstreamClosedError("closed")
-
- async def close(self) -> None:
- self._closed.set()
-
-
-class _RecordingMetrics:
- """MetricsPort that records the full call sequence (order + amount + tags),
- so a test can prove a live +1 gauge, not just a final net-0."""
-
- def __init__(self) -> None:
- self.calls: list[tuple[str, str, float, dict[str, str]]] = []
-
- def increment(self, name: str, amount: int = 1, tags: Mapping[str, str] | None = None) -> None:
- self.calls.append(("increment", name, amount, dict(tags or {})))
-
- def observe(self, name: str, value: float, tags: Mapping[str, str] | None = None) -> None:
- self.calls.append(("observe", name, value, dict(tags or {})))
-
- def gauge(self, name: str, amount: int, tags: Mapping[str, str] | None = None) -> None:
- self.calls.append(("gauge", name, amount, dict(tags or {})))
-
-
-def _bare_session() -> ProxySession:
- return ProxySession(
- session_id=SESSION_ID,
- upstream_ws_url="ws://x/y",
- principal=Principal(principal_id=ORG_ID, organization_id=ORG_ID),
- )
-
-
-def _relay_server(metrics: MetricsPort, pipeline: MiddlewarePipeline | None = None) -> CdpProxyServer:
- return CdpProxyServer(
- upstream=_Upstream(None), # type: ignore[arg-type]
- sessions=InMemorySessionRegistry(),
- auth=StaticKeyAuth({}),
- metrics=metrics,
- event_policy=ForwardAllEventPolicy(), # type: ignore[arg-type]
- pipeline=pipeline,
- )
-
-
-def _sum(records: Iterable[Record], name: str) -> float:
- return sum(record.value for record in records if record.name == name)
-
-
-def _has(records: Iterable[Record], name: str) -> bool:
- return any(record.name == name for record in records)
-
-
-def _label_values(records: Iterable[Record]) -> Iterator[str]:
- for record in records:
- for value in record.attributes.values():
- yield str(value)
-
-
-class MetricsPortContract:
- def make_metrics(self) -> MetricsPort:
- raise NotImplementedError
-
- def collect(self, metrics: MetricsPort) -> list[Record] | None:
- """Emitted metrics for `metrics`, or None if the adapter cannot record."""
- return None
-
- def _server(
- self,
- metrics: MetricsPort,
- *,
- upstream: object,
- event_policy: object | None = None,
- pipeline: MiddlewarePipeline | None = None,
- sessions: InMemorySessionRegistry | None = None,
- ) -> CdpProxyServer:
- if sessions is None:
- sessions = InMemorySessionRegistry()
- sessions.put(
- ResolvedSession(
- session_id=SESSION_ID,
- upstream_adapter="memory",
- upstream_ws_url=f"ws://vendor.internal:9222/live?token={SECRET}",
- organization_id=ORG_ID,
- connect_headers={"authorization": f"Bearer {SECRET}"},
- )
- )
- return CdpProxyServer(
- upstream=upstream, # type: ignore[arg-type]
- sessions=sessions,
- auth=StaticKeyAuth({VALID_KEY: Principal(principal_id=ORG_ID, organization_id=ORG_ID)}),
- metrics=metrics,
- event_policy=event_policy or ForwardAllEventPolicy(), # type: ignore[arg-type]
- pipeline=pipeline,
- )
-
- def _client(self, incoming: list[str] | None = None, block: bool = True) -> _FakeClientWebSocket:
- return _FakeClientWebSocket(
- path=f"/{SESSION_ID}", incoming=incoming, headers={"x-api-key": VALID_KEY}, block=block
- )
-
- async def _run_passthrough(
- self, metrics: MetricsPort, method: str = "Page.enable"
- ) -> tuple[_FakeClientWebSocket, _EchoThenCloseUpstreamConnection]:
- connection = _EchoThenCloseUpstreamConnection()
- client = self._client([encode_frame(CdpCommand(id=7, method=method))])
- server = self._server(metrics, upstream=_Upstream(connection))
- await server._handle_client(client) # type: ignore[arg-type]
- return client, connection
-
- async def _run_filtered(self, metrics: MetricsPort) -> None:
- event = encode_frame(CdpEvent(method="Network.requestWillBeSent", params={}))
- server = self._server(
- metrics, upstream=_Upstream(_ScriptedEventUpstreamConnection([event])), event_policy=_DropAllEvents()
- )
- await server._handle_client(self._client()) # type: ignore[arg-type]
-
- async def _run_unmatched_response(self, metrics: MetricsPort) -> None:
- # A response whose id was never mapped by the remapper — a genuinely
- # reachable path (late/duplicate upstream reply).
- response = encode_frame(CdpResponse(id=999, result={}))
- server = self._server(metrics, upstream=_Upstream(_ScriptedEventUpstreamConnection([response])))
- await server._handle_client(self._client()) # type: ignore[arg-type]
-
- async def _run_pipeline_dropped(self, metrics: MetricsPort) -> None:
- async def drop(frame: object, direction: object, session: object) -> None:
- return None
-
- # The pipeline drops the only client frame, so no upstream reply ever
- # arrives; a non-blocking client ends the stream to tear the session down.
- server = self._server(
- metrics, upstream=_Upstream(_EchoThenCloseUpstreamConnection()), pipeline=MiddlewarePipeline([drop])
- )
- await server._handle_client(self._client([encode_frame(CdpCommand(id=1, method="Page.enable"))], block=False)) # type: ignore[arg-type]
-
- async def _run_errored(self, metrics: MetricsPort) -> None:
- server = self._server(metrics, upstream=_Upstream(_EchoThenCloseUpstreamConnection()))
- await server._handle_client(self._client(["}not a frame{"])) # type: ignore[arg-type]
-
- async def _run_rejected(self, metrics: MetricsPort, path: str, key: str, sessions=None) -> None: # type: ignore[no-untyped-def]
- server = self._server(metrics, upstream=_Upstream(_EchoThenCloseUpstreamConnection()), sessions=sessions)
- ws = _FakeClientWebSocket(path=path, headers={"x-api-key": key})
- await server._handle_client(ws) # type: ignore[arg-type]
-
- async def _run_connect_failure(self, metrics: MetricsPort) -> _FakeClientWebSocket:
- server = self._server(metrics, upstream=_FailingUpstream())
- ws = self._client([encode_frame(CdpCommand(id=1, method="Page.enable"))])
- await server._handle_client(ws) # type: ignore[arg-type]
- return ws
-
- @pytest.mark.asyncio
- async def test_passthrough_session_relays_frames_in_both_directions(self) -> None:
- metrics = self.make_metrics()
- await self._run_passthrough(metrics)
- records = self.collect(metrics)
- if records is None:
- return
- assert _sum(records, FRAMES_RELAYED) >= 2
- assert _sum(records, FRAMES_DROPPED) == 0
- assert _has(records, COMMAND_LATENCY)
-
- @pytest.mark.asyncio
- async def test_bytes_relayed_matches_wire_size_by_direction(self) -> None:
- metrics = self.make_metrics()
- client, connection = await self._run_passthrough(metrics)
- records = self.collect(metrics)
- if records is None:
- return
- upstream_bytes = sum(len(wire) for wire in connection.received)
- client_bytes = sum(len(wire) for wire in client.sent)
- assert upstream_bytes > 0 and client_bytes > 0
- by_direction: dict[str | None, float] = {}
- for record in records:
- if record.name == BYTES_RELAYED:
- key = record.attributes.get("direction")
- by_direction[key] = by_direction.get(key, 0) + record.value
- assert by_direction.get("client_to_upstream") == upstream_bytes
- assert by_direction.get("upstream_to_client") == client_bytes
-
- @pytest.mark.asyncio
- async def test_unknown_cdp_domain_and_method_bucket_to_other(self) -> None:
- metrics = self.make_metrics()
- await self._run_passthrough(metrics, method="Bogus.doThing")
- records = self.collect(metrics)
- if records is None:
- return
- latency = [r for r in records if r.name == COMMAND_LATENCY]
- assert latency
- assert all(r.attributes.get("cdp_domain") == "other" for r in latency)
- assert all(r.attributes.get("cdp_method") == "other" for r in latency)
-
- @pytest.mark.asyncio
- async def test_known_domain_unknown_method_buckets_method_and_hides_client_string(self) -> None:
- # A method under a known domain carrying a token: the domain is kept, but
- # the method (client-controlled) must collapse to "other" so it can neither
- # mint unbounded series nor smuggle the token into a label.
- metrics = self.make_metrics()
- await self._run_passthrough(metrics, method=f"Page.{SECRET}")
- records = self.collect(metrics)
- if records is None:
- return
- latency = [r for r in records if r.name == COMMAND_LATENCY]
- assert latency
- assert all(r.attributes.get("cdp_domain") == "Page" for r in latency)
- assert all(r.attributes.get("cdp_method") == "other" for r in latency)
- for value in _label_values(records):
- assert SECRET not in value
-
- @pytest.mark.asyncio
- async def test_upstream_connect_failure_is_rejected_and_recorded(self) -> None:
- metrics = self.make_metrics()
- ws = await self._run_connect_failure(metrics)
- assert ws.close_code == 1011
- records = self.collect(metrics)
- if records is None:
- return
- rejected = [r for r in records if r.name == CONNECTION_REJECTED]
- assert any(r.attributes.get("reason") == "upstream_unavailable" for r in rejected)
- # The gauge never went up (the dial failed), so it must not have gone down.
- assert _sum(records, ACTIVE_SESSIONS) == 0
-
- @pytest.mark.asyncio
- async def test_filtered_session_is_distinguishable_from_passthrough(self) -> None:
- metrics = self.make_metrics()
- await self._run_filtered(metrics)
- records = self.collect(metrics)
- if records is None:
- return
- dropped = [r for r in records if r.name == FRAMES_DROPPED]
- assert _sum(dropped, FRAMES_DROPPED) >= 1
- assert any(r.attributes.get("reason") == "event_policy" for r in dropped)
-
- @pytest.mark.asyncio
- async def test_frames_dropped_reasons_are_labelled(self) -> None:
- metrics = self.make_metrics()
- await self._run_filtered(metrics)
- await self._run_unmatched_response(metrics)
- await self._run_pipeline_dropped(metrics)
- records = self.collect(metrics)
- if records is None:
- return
- reasons = {r.attributes.get("reason") for r in records if r.name == FRAMES_DROPPED}
- assert {"event_policy", "unmatched_response", "pipeline"} <= reasons
-
- @pytest.mark.asyncio
- async def test_errored_session_records_a_frame_decode_error(self) -> None:
- metrics = self.make_metrics()
- await self._run_errored(metrics)
- records = self.collect(metrics)
- if records is None:
- return
- assert _sum(records, FRAME_DECODE_ERRORS) >= 1
-
- @pytest.mark.asyncio
- async def test_every_rejection_reason_is_labelled(self) -> None:
- metrics = self.make_metrics()
- await self._run_rejected(metrics, path="//[", key=VALID_KEY) # urlsplit raises -> malformed_request
- await self._run_rejected(metrics, path=f"/{SESSION_ID}", key="wrong-key") # bad key -> unauthorized
- await self._run_rejected(metrics, path="/s_unknown", key=VALID_KEY) # unseeded id -> unknown_session
- for status in ("pending", "closed", "expired"): # owned but not routable -> lifecycle reason
- registry = InMemorySessionRegistry()
- getattr(registry, f"mark_{status}")(SESSION_ID, organization_id=ORG_ID)
- await self._run_rejected(metrics, path=f"/{SESSION_ID}", key=VALID_KEY, sessions=registry)
- await self._run_connect_failure(metrics) # dial fails -> upstream_unavailable
- records = self.collect(metrics)
- if records is None:
- return
- reasons = {r.attributes.get("reason") for r in records if r.name == CONNECTION_REJECTED}
- assert {
- "malformed_request",
- "unauthorized",
- "unknown_session",
- "pending",
- "closed",
- "expired",
- "upstream_unavailable",
- } <= reasons
-
- @pytest.mark.asyncio
- async def test_active_session_gauge_is_recorded_and_balances(self) -> None:
- metrics = self.make_metrics()
- await self._run_passthrough(metrics)
- records = self.collect(metrics)
- if records is None:
- return
- assert _sum(records, CLIENT_CONNECTED) == 1
- assert _sum(records, CLIENT_DISCONNECTED) == 1
- assert _has(records, ACTIVE_SESSIONS)
- assert _sum(records, ACTIVE_SESSIONS) == 0
-
- @pytest.mark.asyncio
- async def test_active_session_gauge_balances_when_upstream_close_raises(self) -> None:
- metrics = self.make_metrics()
- connection = _RaisingCloseUpstreamConnection()
- server = self._server(metrics, upstream=_Upstream(connection))
- with pytest.raises(RuntimeError):
- await server._handle_client(self._client([encode_frame(CdpCommand(id=7, method="Page.enable"))])) # type: ignore[arg-type]
- records = self.collect(metrics)
- if records is None:
- return
- assert _has(records, ACTIVE_SESSIONS)
- assert _sum(records, ACTIVE_SESSIONS) == 0
-
- @pytest.mark.asyncio
- async def test_no_token_or_upstream_url_appears_in_any_label(self) -> None:
- metrics = self.make_metrics()
- # Drive every emission path so redaction is checked across all of them.
- await self._run_passthrough(metrics)
- await self._run_filtered(metrics)
- await self._run_unmatched_response(metrics)
- await self._run_errored(metrics)
- await self._run_rejected(metrics, path="/s_unknown", key=VALID_KEY)
- records = self.collect(metrics)
- if records is None:
- return
- labels = list(_label_values(records))
- assert labels # the driven paths emitted labelled metrics
- for value in labels:
- assert SECRET not in value
- assert "vendor.internal" not in value
-
-
-class TestNoOpMetricsContract(MetricsPortContract):
- def make_metrics(self) -> MetricsPort:
- return NoOpMetrics()
-
-
-def _gauge_amounts(metrics: _RecordingMetrics) -> list[float]:
- return [amount for op, name, amount, _tags in metrics.calls if op == "gauge" and name == ACTIVE_SESSIONS]
-
-
-async def test_active_gauge_emits_a_live_plus_one_before_the_teardown_minus_one() -> None:
- metrics = _RecordingMetrics()
- harness = TestNoOpMetricsContract()
- server = harness._server(metrics, upstream=_Upstream(_EchoThenCloseUpstreamConnection()))
- await server._handle_client(harness._client([encode_frame(CdpCommand(id=7, method="Page.enable"))])) # type: ignore[arg-type]
- assert _gauge_amounts(metrics) == [1, -1]
-
-
-async def test_active_gauge_decrements_on_midflight_teardown() -> None:
- metrics = _RecordingMetrics()
- harness = TestNoOpMetricsContract()
- connection = _BlockingUpstreamConnection()
- client = harness._client(block=True) # stays live until we end the stream
- server = harness._server(metrics, upstream=_Upstream(connection))
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- for _ in range(100):
- if 1 in _gauge_amounts(metrics):
- break
- await asyncio.sleep(0)
- assert _gauge_amounts(metrics) == [1] # live +1, no -1 yet
- client._drained.set() # client stream ends -> finally cancels the pending relay
- await task
- assert _gauge_amounts(metrics) == [1, -1]
-
-
-def test_track_command_is_bounded_by_the_cap() -> None:
- command_starts: dict[int, tuple[str, float]] = {}
- for upstream_id in range(_MAX_PENDING_COMMANDS + 50):
- _track_command(command_starts, upstream_id, "Page.enable")
- assert len(command_starts) == _MAX_PENDING_COMMANDS
- assert (_MAX_PENDING_COMMANDS + 49) in command_starts # newest kept
- assert 0 not in command_starts # oldest evicted
-
-
-async def test_out_of_order_responses_match_and_drain_command_starts() -> None:
- metrics = _RecordingMetrics()
- server = _relay_server(metrics)
- session = _bare_session()
- remapper = RequestIdRemapper()
- command_starts: dict[int, tuple[str, float]] = {}
- commands = _FakeClientWebSocket(
- "/x",
- incoming=[
- encode_frame(CdpCommand(id=10, method="Page.enable")),
- encode_frame(CdpCommand(id=11, method="Network.enable")),
- encode_frame(CdpCommand(id=12, method="Runtime.enable")),
- ],
- block=False,
- )
- await server._relay_client_to_upstream(commands, _CaptureUpstreamConnection(), session, remapper, command_starts)
- assert len(command_starts) == 3
- # Respond out of order (3, 1, 2) then close.
- responses = _ScriptedEventUpstreamConnection([encode_frame(CdpResponse(id=i, result={})) for i in (3, 1, 2)])
- try:
- await server._relay_upstream_to_client(_FakeClientWebSocket("/x"), responses, session, remapper, command_starts)
- except UpstreamClosedError:
- pass
- assert command_starts == {} # empty after quiescence
- assert sum(1 for op, name, *_ in metrics.calls if op == "observe" and name == COMMAND_LATENCY) == 3
-
-
-async def test_dropped_and_unmatched_responses_evict_their_command_starts() -> None:
- metrics = _RecordingMetrics()
- session = _bare_session()
-
- async def drop(frame: object, direction: object, sess: object) -> None:
- return None
-
- # A pipeline-dropped response frees its pending latency entry AND its id mapping:
- # the response will never be delivered, so every table its upstream id sits in has
- # to let go of it (SKY-12500 AC5).
- dropping = _relay_server(metrics, pipeline=MiddlewarePipeline([drop]))
- remapper = RequestIdRemapper()
- suppressed = remapper.to_upstream("client", CdpCommand(id=1, method="Page.enable"))
- command_starts: dict[int, tuple[str, float]] = {suppressed.id: ("Page.enable", 0.0)}
- dropped = _ScriptedEventUpstreamConnection([encode_frame(CdpResponse(id=suppressed.id, result={}))])
- try:
- await dropping._relay_upstream_to_client(_FakeClientWebSocket("/x"), dropped, session, remapper, command_starts)
- except UpstreamClosedError:
- pass
- assert suppressed.id not in command_starts
- assert remapper.pending_count == 0
-
- # An unmatched response (id never remapped) also evicts and is labelled.
- passthrough = _relay_server(metrics)
- unmatched_starts: dict[int, tuple[str, float]] = {7: ("Page.enable", 0.0)}
- unmatched = _ScriptedEventUpstreamConnection([encode_frame(CdpResponse(id=7, result={}))])
- try:
- await passthrough._relay_upstream_to_client(
- _FakeClientWebSocket("/x"), unmatched, session, RequestIdRemapper(), unmatched_starts
- )
- except UpstreamClosedError:
- pass
- assert 7 not in unmatched_starts
diff --git a/tests/unit/proxy/test_pipeline.py b/tests/unit/proxy/test_pipeline.py
deleted file mode 100644
index fff2d7707..000000000
--- a/tests/unit/proxy/test_pipeline.py
+++ /dev/null
@@ -1,246 +0,0 @@
-from __future__ import annotations
-
-import pytest
-
-from skyvern.proxy.core.frames import CdpCommand, CdpFrame
-from skyvern.proxy.core.pipeline import (
- Direction,
- InterceptContext,
- InterceptorContractError,
- MiddlewarePipeline,
- SynthesizedResponse,
- interceptor_failure_response,
-)
-from skyvern.proxy.core.session import Principal, ProxySession
-
-
-def make_session() -> ProxySession:
- return ProxySession(
- session_id="session",
- upstream_ws_url="ws://localhost:1",
- principal=Principal(principal_id="owner"),
- )
-
-
-def make_context(sent: list[CdpCommand] | None = None) -> InterceptContext:
- async def send_proxy_command(command: CdpCommand) -> None:
- if sent is not None:
- sent.append(command)
-
- return InterceptContext(send_proxy_command=send_proxy_command)
-
-
-async def test_pipeline_composes_middlewares_in_order() -> None:
- calls: list[tuple[str, Direction, str]] = []
-
- async def add_first_param(frame: CdpFrame, direction: Direction, session: ProxySession) -> CdpFrame:
- calls.append(("first", direction, session.session_id))
- assert isinstance(frame, CdpCommand)
- return CdpCommand(id=frame.id, method=frame.method, params={"first": True})
-
- async def add_second_param(frame: CdpFrame, direction: Direction, session: ProxySession) -> CdpFrame:
- calls.append(("second", direction, session.session_id))
- assert isinstance(frame, CdpCommand)
- return CdpCommand(id=frame.id, method=frame.method, params={**(frame.params or {}), "second": True})
-
- pipeline = MiddlewarePipeline([add_first_param])
- pipeline.add(add_second_param)
-
- result = await pipeline.process(
- CdpCommand(id=1, method="Runtime.enable"), Direction.CLIENT_TO_UPSTREAM, make_session()
- )
-
- assert result == CdpCommand(id=1, method="Runtime.enable", params={"first": True, "second": True})
- assert calls == [
- ("first", Direction.CLIENT_TO_UPSTREAM, "session"),
- ("second", Direction.CLIENT_TO_UPSTREAM, "session"),
- ]
-
-
-async def test_pipeline_stops_after_middleware_drops_frame() -> None:
- calls: list[str] = []
-
- async def drop(frame: CdpFrame, direction: Direction, session: ProxySession) -> None:
- calls.append("drop")
- return None
-
- async def should_not_run(frame: CdpFrame, direction: Direction, session: ProxySession) -> CdpFrame:
- calls.append("unexpected")
- return frame
-
- pipeline = MiddlewarePipeline([drop, should_not_run])
-
- result = await pipeline.process(
- CdpCommand(id=1, method="Runtime.enable"), Direction.UPSTREAM_TO_CLIENT, make_session()
- )
-
- assert result is None
- assert calls == ["drop"]
-
-
-async def test_empty_pipeline_returns_original_frame() -> None:
- frame = CdpCommand(id=1, method="Runtime.enable")
-
- assert await MiddlewarePipeline().process(frame, Direction.CLIENT_TO_UPSTREAM, make_session()) == frame
-
-
-async def test_interceptors_run_in_order_and_rewrite_the_command() -> None:
- calls: list[str] = []
-
- async def first(command: CdpCommand, session: ProxySession, context: InterceptContext) -> CdpCommand:
- calls.append("first")
- return CdpCommand(id=command.id, method=command.method, params={"first": True})
-
- async def second(command: CdpCommand, session: ProxySession, context: InterceptContext) -> CdpCommand:
- calls.append("second")
- return CdpCommand(id=command.id, method=command.method, params={**(command.params or {}), "second": True})
-
- pipeline = MiddlewarePipeline(interceptors=[first, second])
-
- result = await pipeline.intercept(CdpCommand(id=7, method="Page.navigate"), make_session(), make_context())
-
- assert result == CdpCommand(id=7, method="Page.navigate", params={"first": True, "second": True})
- assert calls == ["first", "second"]
-
-
-async def test_synthesized_response_short_circuits_later_interceptors() -> None:
- calls: list[str] = []
-
- async def answer(command: CdpCommand, session: ProxySession, context: InterceptContext) -> SynthesizedResponse:
- calls.append("answer")
- return SynthesizedResponse(result={"ok": True}, reason="test")
-
- async def never_runs(command: CdpCommand, session: ProxySession, context: InterceptContext) -> CdpCommand:
- calls.append("never")
- return command
-
- pipeline = MiddlewarePipeline(interceptors=[answer, never_runs])
- command = CdpCommand(id=9, method="Page.navigate", session_id="sess-1")
-
- outcome = await pipeline.intercept(command, make_session(), make_context())
-
- assert isinstance(outcome, SynthesizedResponse)
- assert calls == ["answer"]
- # Stamping is the correlation guarantee: the response reuses the command's own
- # id and session scope, so an interceptor cannot mis-correlate a synthesis.
- response = outcome.to_response(command)
- assert response.id == 9
- assert response.session_id == "sess-1"
- assert response.result == {"ok": True}
- assert response.error is None
-
-
-def test_synthesized_response_requires_exactly_one_of_result_and_error() -> None:
- with pytest.raises(ValueError):
- SynthesizedResponse()
- with pytest.raises(ValueError):
- SynthesizedResponse(result={}, error={"code": 1, "message": "x"})
-
-
-def test_synthesized_error_must_be_a_valid_cdp_error() -> None:
- with pytest.raises(ValueError):
- SynthesizedResponse(error={"code": 1})
- with pytest.raises(ValueError):
- SynthesizedResponse(error={"message": "no code"})
- with pytest.raises(ValueError):
- SynthesizedResponse(error={"code": 1, "message": "x", "extra": True})
-
-
-def test_synthesized_response_requires_a_reason() -> None:
- with pytest.raises(ValueError):
- SynthesizedResponse(result={}, reason="")
-
-
-def test_synthesized_payloads_must_be_cdp_encodable() -> None:
- # A synthesis that cannot be JSON-encoded must fail at construction — inside
- # the interceptor, where the fail-closed rule converts it into a deterministic
- # error — never at delivery, where it would tear the relay down instead.
- with pytest.raises(ValueError):
- SynthesizedResponse(result={"blob": b"bytes"})
- with pytest.raises(ValueError):
- SynthesizedResponse(result={"nan": float("nan")})
- with pytest.raises(ValueError):
- SynthesizedResponse(result={"obj": object()})
- with pytest.raises(ValueError):
- SynthesizedResponse(error={"code": 1, "message": "x", "data": {"blob": b"bytes"}})
-
-
-def test_reason_must_be_a_label_safe_identifier() -> None:
- # The reason becomes a metric label: request-derived content (URLs, ids,
- # error text) must be unrepresentable, which is what bounds cardinality.
- for bad in ("has space", "https://example.com", "UPPER", "x" * 65, "dash-ed"):
- with pytest.raises(ValueError):
- SynthesizedResponse(result={}, reason=bad)
- assert SynthesizedResponse(result={}, reason="org_denylist_v2").reason == "org_denylist_v2"
-
-
-async def test_interceptor_must_preserve_the_request_id() -> None:
- async def renumber(command: CdpCommand, session: ProxySession, context: InterceptContext) -> CdpCommand:
- return CdpCommand(id=command.id + 1, method=command.method)
-
- pipeline = MiddlewarePipeline(interceptors=[renumber])
-
- with pytest.raises(InterceptorContractError):
- await pipeline.intercept(CdpCommand(id=1, method="Page.navigate"), make_session(), make_context())
-
-
-async def test_interceptor_must_preserve_session_addressing() -> None:
- async def rescope(command: CdpCommand, session: ProxySession, context: InterceptContext) -> CdpCommand:
- return CdpCommand(id=command.id, method=command.method, session_id="other-session")
-
- async def readdress(command: CdpCommand, session: ProxySession, context: InterceptContext) -> CdpCommand:
- return CdpCommand(id=command.id, method=command.method, params={"sessionId": "other-session"})
-
- with pytest.raises(InterceptorContractError):
- await MiddlewarePipeline(interceptors=[rescope]).intercept(
- CdpCommand(id=1, method="Page.navigate", session_id="mine"), make_session(), make_context()
- )
- with pytest.raises(InterceptorContractError):
- await MiddlewarePipeline(interceptors=[readdress]).intercept(
- CdpCommand(id=1, method="Target.detachFromTarget", params={"sessionId": "mine"}),
- make_session(),
- make_context(),
- )
-
-
-async def test_interceptor_must_return_a_command_or_a_synthesis() -> None:
- async def drops(command: CdpCommand, session: ProxySession, context: InterceptContext) -> None:
- return None
-
- pipeline = MiddlewarePipeline(interceptors=[drops])
-
- # A silently dropped command would hang its client forever, so an interceptor
- # has no drop verb at all — blocking means synthesizing an error response.
- with pytest.raises(InterceptorContractError):
- await pipeline.intercept(CdpCommand(id=1, method="Page.navigate"), make_session(), make_context())
-
-
-async def test_no_interceptors_is_identity() -> None:
- command = CdpCommand(id=1, method="Page.navigate")
- pipeline = MiddlewarePipeline()
-
- assert pipeline.has_interceptors is False
- assert await pipeline.intercept(command, make_session(), make_context()) is command
- assert MiddlewarePipeline(interceptors=[]).has_interceptors is False
-
-
-def test_interceptor_failure_response_is_deterministic() -> None:
- command = CdpCommand(id=13, method="Page.navigate", session_id="sess-1")
-
- response = interceptor_failure_response(command)
-
- assert response.id == 13
- assert response.session_id == "sess-1"
- assert response.result is None
- assert response.error is not None
- assert response.error["code"] == -32603
- assert response == interceptor_failure_response(command)
diff --git a/tests/unit/proxy/test_policy_pack.py b/tests/unit/proxy/test_policy_pack.py
deleted file mode 100644
index c26bcad27..000000000
--- a/tests/unit/proxy/test_policy_pack.py
+++ /dev/null
@@ -1,286 +0,0 @@
-from __future__ import annotations
-
-from pathlib import Path
-
-import pytest
-
-from skyvern.proxy.adapters.websocket_server import _cdp_method_tags
-from skyvern.proxy.core.frames import (
- LIFECYCLE_EVENTS,
- CdpCommand,
- CdpEvent,
- decode_frame,
- encode_frame,
-)
-from skyvern.proxy.core.policy import FORWARD, Drop, DropReason, EventPolicyEngine
-from skyvern.proxy.core.policy_pack import (
- CONSOLE_BURST_PER_SECOND,
- NOISY_EVENT_PACK_V1,
- UNCONSUMED_EVENTS,
-)
-from skyvern.proxy.core.session import Principal, ProxySession
-from tests.unit.proxy.test_event_policy import FakeClock
-
-FIXTURES_DIR = Path(__file__).parent / "fixtures"
-BROADBAND_WORKLOAD = "workload_page_load_broadband.jsonl"
-LOOPBACK_WORKLOAD = "workload_page_load_loopback.jsonl"
-
-# Every event the pinned Playwright driver or puppeteer-core registers a handler for,
-# enumerated from their sources rather than recalled. The pack must never drop one of
-# these: a client library that stops seeing them silently loses page state, which no
-# test of ours would notice.
-#
-# This is a MANUAL SNAPSHOT, and it is the safety net for future drop-set additions —
-# an omission here reads as "nothing consumes it" and would wave through a rule that
-# breaks a client. Adding to the drop set means re-enumerating BOTH pinned sources:
-#
-# playwright: grep -rhoE 'Network\.[a-zA-Z]+' <uv-cache>/playwright/driver/package/lib
-# puppeteer: grep -rhoE 'Network\.[a-zA-Z]+' <npm pack puppeteer-core>/package/src
-#
-# THE TRAP: a wrong path greps clean and every method looks unconsumed. Run a positive
-# control first — Network.responseReceived and Network.loadingFinished must come back
-# non-zero, or the path is wrong and the whole result is noise. Match unquoted: the
-# quoted form ('Network.x') only hits some driver builds.
-CONSUMED_BY_CLIENT_LIBRARIES = (
- "Network.loadingFailed",
- "Network.loadingFinished",
- "Network.requestServedFromCache",
- "Network.requestWillBeSent",
- "Network.requestWillBeSentExtraInfo",
- "Network.responseReceived",
- "Network.responseReceivedExtraInfo",
- "Network.webSocketClosed",
- "Network.webSocketCreated",
- "Network.webSocketFrameError",
- "Network.webSocketFrameReceived",
- "Network.webSocketFrameSent",
- "Network.webSocketHandshakeResponseReceived",
- "Network.webSocketWillSendHandshakeRequest",
- "Runtime.bindingCalled",
- "Runtime.exceptionThrown",
- "Runtime.executionContextCreated",
- "Runtime.executionContextDestroyed",
- "Runtime.executionContextsCleared",
- "Page.domContentEventFired",
- "Page.fileChooserOpened",
- "Page.frameAttached",
- "Page.frameDetached",
- "Page.frameNavigated",
- "Page.javascriptDialogOpening",
- "Page.lifecycleEvent",
- "Page.loadEventFired",
- "Page.screencastFrame",
- "Target.targetCreated",
- "Target.targetDestroyed",
- "Target.targetInfoChanged",
- "Log.entryAdded",
- "Fetch.authRequired",
- "Fetch.requestPaused",
-)
-
-SESSION_ID = "s_1"
-CHILD = "child-1"
-
-
-def _session() -> ProxySession:
- return ProxySession(
- session_id=SESSION_ID,
- upstream_ws_url="ws://upstream.internal:9222/x?token=secret-token",
- principal=Principal(principal_id="o_1", organization_id="o_1"),
- )
-
-
-def _engine(clock: FakeClock | None = None) -> EventPolicyEngine:
- return EventPolicyEngine(config=NOISY_EVENT_PACK_V1, clock=clock or FakeClock())
-
-
-def _events(fixture: str) -> list[CdpEvent]:
- frames = (decode_frame(line) for line in (FIXTURES_DIR / fixture).read_text().splitlines() if line.strip())
- return [frame for frame in frames if isinstance(frame, CdpEvent)]
-
-
-def _replay(events: list[CdpEvent]) -> tuple[int, int]:
- """Forwarded event count and wire bytes for one pass of the pack."""
- engine = _engine()
- session = _session()
- count = bytes_out = 0
- for event in events:
- if isinstance(engine.decide(event, session), Drop):
- continue
- count += 1
- bytes_out += len(encode_frame(event))
- return count, bytes_out
-
-
-# --- the curated drop set -------------------------------------------------------
-
-
[email protected]("method", UNCONSUMED_EVENTS)
-def test_pack_drops_every_unconsumed_noise_event(method: str) -> None:
- engine = _engine()
-
- decision = engine.decide(CdpEvent(method=method, params={}, session_id=CHILD), _session())
-
- assert decision == Drop(DropReason.POLICY)
-
-
[email protected]("method", UNCONSUMED_EVENTS)
-def test_an_unconsumed_drop_is_not_relaxed_by_declared_interest(method: str) -> None:
- """The drop set is unconditional: no client consumes these even when it has
- enabled the domain, so enabling Network must not resurrect the noise."""
- engine = _engine()
- session = _session()
- engine.observe_command(CdpCommand(id=1, method="Network.enable", session_id=CHILD), session)
-
- assert engine.decide(CdpEvent(method=method, params={}, session_id=CHILD), session) == Drop(DropReason.POLICY)
-
-
[email protected]("method", CONSUMED_BY_CLIENT_LIBRARIES)
-def test_pack_never_drops_an_event_a_client_library_consumes(method: str) -> None:
- engine = _engine()
-
- assert engine.decide(CdpEvent(method=method, params={}, session_id=CHILD), _session()) is FORWARD
-
-
[email protected]("method", sorted(LIFECYCLE_EVENTS))
-def test_pack_forwards_lifecycle_events(method: str) -> None:
- engine = _engine()
-
- assert engine.decide(CdpEvent(method=method, params={}, session_id=CHILD), _session()) is FORWARD
-
-
-def test_drop_set_and_consumed_set_are_disjoint() -> None:
- assert not set(UNCONSUMED_EVENTS) & set(CONSUMED_BY_CLIENT_LIBRARIES)
-
-
-# --- AC2: gating keys off actual Domain.enable ----------------------------------
-
-
-def test_console_throttle_relaxes_for_a_session_that_enabled_runtime() -> None:
- """AC2: observed interest, not an assumption, is what lifts the cap."""
- clock = FakeClock()
- engine = _engine(clock)
- session = _session()
- engine.observe_command(CdpCommand(id=1, method="Runtime.enable", session_id=CHILD), session)
-
- burst = [
- engine.decide(CdpEvent(method="Runtime.consoleAPICalled", params={}, session_id=CHILD), session)
- for _ in range(CONSOLE_BURST_PER_SECOND * 3)
- ]
-
- assert all(decision is FORWARD for decision in burst)
-
-
-def test_console_throttle_caps_a_burst_when_interest_was_never_observed() -> None:
- clock = FakeClock()
- engine = _engine(clock)
- session = _session()
-
- burst = [
- engine.decide(CdpEvent(method="Runtime.consoleAPICalled", params={}, session_id=CHILD), session)
- for _ in range(CONSOLE_BURST_PER_SECOND + 5)
- ]
-
- assert burst[:CONSOLE_BURST_PER_SECOND] == [FORWARD] * CONSOLE_BURST_PER_SECOND
- assert burst[CONSOLE_BURST_PER_SECOND:] == [Drop(DropReason.THROTTLED)] * 5
-
-
-def test_unknown_interest_still_delivers_console_events_rather_than_silencing_them() -> None:
- """The fail-safe rule: is_domain_enabled is False for a session the proxy never
- saw enable the domain (bounded tables evict), so False must never mean silence.
- An unobserved session keeps getting events, just rate-capped."""
- clock = FakeClock()
- engine = _engine(clock)
- session = _session()
-
- assert not engine.is_domain_enabled(session, "Runtime", CHILD)
- for second in range(5):
- clock.advance(1.0)
- decision = engine.decide(CdpEvent(method="Runtime.consoleAPICalled", params={}, session_id=CHILD), session)
- assert decision is FORWARD, f"silenced in window {second}"
-
-
-def test_interest_is_scoped_to_the_child_session_that_enabled_the_domain() -> None:
- clock = FakeClock()
- engine = _engine(clock)
- session = _session()
- engine.observe_command(CdpCommand(id=1, method="Runtime.enable", session_id=CHILD), session)
-
- relaxed = [
- engine.decide(CdpEvent(method="Runtime.consoleAPICalled", params={}, session_id=CHILD), session)
- for _ in range(CONSOLE_BURST_PER_SECOND + 5)
- ]
- other = [
- engine.decide(CdpEvent(method="Runtime.consoleAPICalled", params={}, session_id="child-2"), session)
- for _ in range(CONSOLE_BURST_PER_SECOND + 5)
- ]
-
- assert all(decision is FORWARD for decision in relaxed)
- assert Drop(DropReason.THROTTLED) in other
-
-
-# --- AC1: before/after on a recorded workload -----------------------------------
-
-
-def test_recorded_broadband_workload_sheds_most_events_and_half_its_bytes() -> None:
- """AC1, measured on a stream recorded from a real Chrome driven like Playwright
- drives it (flat auto-attach + Runtime/Network/Page enable) against a local page
- over emulated broadband. Thresholds sit below the measured 75.7% / 46.5% so this
- pins the win without pinning the fixture's exact byte count."""
- events = _events(BROADBAND_WORKLOAD)
- baseline_count = len(events)
- baseline_bytes = sum(len(encode_frame(event)) for event in events)
-
- count, bytes_out = _replay(events)
-
- assert 1 - count / baseline_count >= 0.50
- assert 1 - bytes_out / baseline_bytes >= 0.30
-
-
-def test_loopback_workload_reduction_is_the_floor_not_the_expectation() -> None:
- """The same page over loopback: Chrome takes each body in a few huge reads, so
- dataReceived barely fires and the pack has far less to shed. Recorded to keep the
- honest floor visible — the win is a function of how a real network chunks bodies,
- and a fast path is where this pack does least."""
- events = _events(LOOPBACK_WORKLOAD)
- baseline_count = len(events)
- baseline_bytes = sum(len(encode_frame(event)) for event in events)
-
- count, bytes_out = _replay(events)
-
- assert 0.10 <= 1 - count / baseline_count < 0.50
- assert 0 < 1 - bytes_out / baseline_bytes < 0.30
-
-
-def test_replaying_a_recorded_workload_forwards_every_consumed_event_untouched() -> None:
- """Compatibility: the pack sheds only its own drop set. Everything the recorded
- stream carried that a client library reads must survive, byte-for-byte."""
- engine = _engine()
- session = _session()
-
- for event in _events(BROADBAND_WORKLOAD):
- baseline = encode_frame(event)
- decision = engine.decide(event, session)
-
- if event.method in UNCONSUMED_EVENTS:
- assert isinstance(decision, Drop)
- continue
- assert decision is FORWARD
- assert encode_frame(event) == baseline
-
-
-# --- AC3: the reduction is visible per method -----------------------------------
-
-
[email protected]("rule", NOISY_EVENT_PACK_V1.rules, ids=lambda rule: rule.method)
-def test_every_gated_method_reports_its_own_metric_bucket(rule) -> None:
- """AC3: a dropped counter tagged cdp_method='other' cannot show which stream the
- reduction came from. Every method this pack names must survive the allowlist."""
- cdp_method, cdp_domain = _cdp_method_tags(rule.method)
-
- assert cdp_method == rule.method
- assert cdp_domain == rule.method.split(".", 1)[0]
-
-
-def test_pack_is_versioned() -> None:
- assert NOISY_EVENT_PACK_V1.version == 1
diff --git a/tests/unit/proxy/test_screencast.py b/tests/unit/proxy/test_screencast.py
deleted file mode 100644
index 76f5da1ed..000000000
--- a/tests/unit/proxy/test_screencast.py
+++ /dev/null
@@ -1,727 +0,0 @@
-"""Screencast protection at the proxy boundary (SKY-12502).
-
-An external client that drives Page.startScreencast through the proxy can pull an
-unbounded stream of JPEG frames, at whatever size and rate it asks for. The proxy caps
-the rate it delivers and bounds the capture params the client requests.
-
-The cap may only ever drop a frame the proxy has ACKED on the client's behalf. Chrome's
-screencast is ack-driven: it holds a small number of frames in flight and emits the next
-one only once an earlier one comes back acked. A dropped frame is a frame the client
-never sees and so never acks, which means a cap that merely drops does not throttle the
-stream — it stalls it to 0 FPS and never recovers. Every test here that withholds a frame
-therefore also asserts the ack that keeps the browser emitting.
-
-Scope is the EXTERNAL proxied path only. Cloud live-view is VNC + Page.captureScreenshot
-and never rides this code.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import contextlib
-
-import pytest
-
-from skyvern.proxy.adapters.memory import (
- AllowAllAuth,
- ForwardAllEventPolicy,
- InMemorySessionRegistry,
- NoOpMetrics,
-)
-from skyvern.proxy.adapters.websocket_server import CdpProxyServer, _cdp_method_tags
-from skyvern.proxy.core.frames import (
- CdpCommand,
- CdpEvent,
- CdpFrame,
- CdpResponse,
- RequestIdRemapper,
- decode_frame,
- encode_frame,
-)
-from skyvern.proxy.core.pipeline import Direction, MiddlewarePipeline
-from skyvern.proxy.core.policy import FORWARD, EventPolicyEngine, PolicyDecision, Rewrite
-from skyvern.proxy.core.screencast import (
- SCREENCAST_FRAME_ACK_METHOD,
- SCREENCAST_FRAME_EVENT,
- SCREENCAST_MAX_DIMENSION,
- SCREENCAST_MAX_FRAMES_PER_SECOND,
- SCREENCAST_MAX_QUALITY,
- SCREENCAST_MIN_EVERY_NTH_FRAME,
- SCREENCAST_PACK_V1,
- START_SCREENCAST_METHOD,
- bound_start_screencast,
- is_screencast_frame,
- screencast_frame_ack,
- screencast_pipeline,
-)
-from skyvern.proxy.core.session import ProxySession, ResolvedSession
-from tests.unit.proxy.test_event_policy import FakeClock
-from tests.unit.proxy.test_shared_connection import (
- UPSTREAM_URL,
- _command,
- _ControllableUpstream,
- _ScriptClient,
- _SharedBrowser,
- _until,
-)
-
-CDP_SESSION = "sess"
-
-
-def _screencast_server(browser: object, clock: FakeClock) -> CdpProxyServer:
- sessions = InMemorySessionRegistry()
- sessions.put(ResolvedSession(session_id="s1", upstream_adapter="memory", upstream_ws_url=UPSTREAM_URL))
- return CdpProxyServer(
- upstream=browser, # type: ignore[arg-type]
- sessions=sessions,
- auth=AllowAllAuth(),
- metrics=NoOpMetrics(),
- event_policy=EventPolicyEngine(config=SCREENCAST_PACK_V1, clock=clock),
- pipeline=screencast_pipeline(),
- )
-
-
-def _screencast_frame(number: int, session_id: str = CDP_SESSION, data_size: int = 16) -> CdpEvent:
- """A frame as Chrome really sends it: `sessionId` is the FRAME NUMBER, an integer,
- and the CDP session it belongs to is the frame's own session_id. `data_size` stands in
- for the base64 JPEG payload, which is essentially all of a real frame's weight."""
- return CdpEvent(
- method=SCREENCAST_FRAME_EVENT,
- params={
- "data": "/9j/4AAQSkZJRg==".ljust(data_size, "A"),
- "metadata": {"deviceWidth": 1920, "deviceHeight": 1080, "timestamp": 1.0},
- "sessionId": number,
- },
- session_id=session_id,
- )
-
-
-def _delivered_screencast_bytes(client: _ScriptClient) -> int:
- """Wire bytes of screencast frames the client was actually sent — the quantity the
- cap exists to bound, and the one a frame count alone does not show."""
- return sum(
- len(wire)
- for wire in client.sent
- if isinstance(decode_frame(wire), CdpEvent) and decode_frame(wire).method == SCREENCAST_FRAME_EVENT
- )
-
-
-def _acks(connection: _ControllableUpstream) -> list[CdpCommand]:
- frames = (decode_frame(wire) for wire in connection.sent)
- return [f for f in frames if isinstance(f, CdpCommand) and f.method == SCREENCAST_FRAME_ACK_METHOD]
-
-
-def _acked_numbers(connection: _ControllableUpstream) -> list[int]:
- return [(ack.params or {})["sessionId"] for ack in _acks(connection)]
-
-
-async def _attached_client(server: CdpProxyServer) -> tuple[_ScriptClient, asyncio.Task[None]]:
- """A client that owns CDP session `sess`, so frames scoped to it have a recipient
- and the drop is the policy's doing rather than a routing miss."""
- client = _ScriptClient([_command(1, "Target.attachToTarget", {"targetId": CDP_SESSION, "flatten": True})])
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: bool(client.sent))
- return client, task
-
-
-async def test_every_withheld_screencast_frame_is_still_acked_upstream() -> None:
- """THE constraint (AC3). The clock never advances, so the whole burst falls in one
- window and everything past the cap is withheld. The browser must see an ack for each
- withheld frame or it stops emitting and the client's stream is dead for good."""
- connection = _ControllableUpstream()
- server = _screencast_server(_SharedBrowser(connection), FakeClock())
- client, task = await _attached_client(server)
-
- burst = SCREENCAST_MAX_FRAMES_PER_SECOND * 4
- for number in range(1, burst + 1):
- connection.emit(_screencast_frame(number))
- withheld = burst - SCREENCAST_MAX_FRAMES_PER_SECOND
- # Acks are sent inline by the reader while delivered frames go out through a queue
- # and its writer, so the two settle independently — wait for both.
- assert await _until(
- lambda: (
- len(_acks(connection)) == withheld
- and len(client.events(SCREENCAST_FRAME_EVENT)) == SCREENCAST_MAX_FRAMES_PER_SECOND
- )
- )
-
- # The cap held on the delivered side...
- assert len(client.events(SCREENCAST_FRAME_EVENT)) == SCREENCAST_MAX_FRAMES_PER_SECOND
- # ...and every frame it withheld was acked upstream, in order, so the browser never
- # runs out of in-flight allowance and the stream keeps flowing.
- assert _acked_numbers(connection) == list(range(SCREENCAST_MAX_FRAMES_PER_SECOND + 1, burst + 1))
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_the_stream_still_flows_after_a_sustained_drop_run() -> None:
- """A withheld run must not be a cliff: once the window rolls the client is served
- again, which is what makes this a throttle rather than a stall."""
- connection = _ControllableUpstream()
- clock = FakeClock()
- server = _screencast_server(_SharedBrowser(connection), clock)
- client, task = await _attached_client(server)
-
- number = 0
- for window in range(1, 4):
- for _ in range(SCREENCAST_MAX_FRAMES_PER_SECOND * 3):
- number += 1
- connection.emit(_screencast_frame(number))
- delivered_by_now = SCREENCAST_MAX_FRAMES_PER_SECOND * window
- acked_by_now = number - delivered_by_now
- # Each window serves its budget and withholds the rest, and the run of withheld
- # frames in between never stops the next window being served.
- assert await _until(
- lambda: (
- len(client.events(SCREENCAST_FRAME_EVENT)) == delivered_by_now
- and len(_acks(connection)) == acked_by_now
- )
- )
- clock.advance(1.0)
-
- # No frame is ever both undelivered and unacked: the two sets partition the stream.
- delivered = {(event.params or {})["sessionId"] for event in client.events(SCREENCAST_FRAME_EVENT)}
- assert delivered.isdisjoint(_acked_numbers(connection))
- assert len(delivered) + len(_acked_numbers(connection)) == number
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def _burst(policy_kind: str, count: int, data_size: int) -> tuple[int, int]:
- """Push `count` screencast frames through a proxy running `policy_kind` and report
- what the client was delivered, as (frames, wire bytes)."""
- from skyvern.proxy.__main__ import build_event_policy, build_pipeline
-
- connection = _ControllableUpstream()
- sessions = InMemorySessionRegistry()
- sessions.put(ResolvedSession(session_id="s1", upstream_adapter="memory", upstream_ws_url=UPSTREAM_URL))
- server = CdpProxyServer(
- upstream=_SharedBrowser(connection), # type: ignore[arg-type]
- sessions=sessions,
- auth=AllowAllAuth(),
- metrics=NoOpMetrics(),
- event_policy=build_event_policy(policy_kind),
- pipeline=build_pipeline(policy_kind),
- )
- client, task = await _attached_client(server)
- for number in range(1, count + 1):
- connection.emit(_screencast_frame(number, data_size=data_size))
- # Every frame ends up either delivered or acked, under both policies — which is the
- # settling condition AND the invariant, so this cannot pass by measuring too early.
- assert await _until(lambda: len(client.events(SCREENCAST_FRAME_EVENT)) + len(_acks(connection)) == count)
- delivered = (len(client.events(SCREENCAST_FRAME_EVENT)), _delivered_screencast_bytes(client))
- client.release()
- await asyncio.wait_for(task, timeout=5)
- return delivered
-
-
-async def test_a_screencast_heavy_session_is_capped_in_frames_and_in_bytes() -> None:
- """AC1, against the pass-through baseline on the same stream. Bytes are the point:
- frames ARE the payload here, so an uncapped screencast is the most expensive thing an
- external client can ask this proxy to carry."""
- frames = SCREENCAST_MAX_FRAMES_PER_SECOND * 6
- data_size = 20_000 # a modest real JPEG frame
-
- baseline_frames, baseline_bytes = await _burst("forward-all", frames, data_size)
- capped_frames, capped_bytes = await _burst("screencast-v1", frames, data_size)
-
- # The baseline is a true pass-through: nothing shed, so the comparison is honest.
- assert baseline_frames == frames
- assert capped_frames == SCREENCAST_MAX_FRAMES_PER_SECOND
- # Both legs of AC1, and by a margin rather than a rounding error.
- assert capped_frames * 6 == baseline_frames
- assert capped_bytes < baseline_bytes // 5
- assert baseline_bytes > frames * data_size # the payload really was carried
-
-
-async def test_the_ack_carries_the_frame_number_and_the_frames_own_cdp_session() -> None:
- """The screencast domain reuses the name `sessionId` for a frame NUMBER. Acking the
- frame's own session_id instead (or reading params.sessionId as a session id, which
- yields None on every real frame) acks nothing the browser is waiting on."""
- connection = _ControllableUpstream()
- server = _screencast_server(_SharedBrowser(connection), FakeClock())
- client, task = await _attached_client(server)
-
- for number in range(1, SCREENCAST_MAX_FRAMES_PER_SECOND + 3):
- connection.emit(_screencast_frame(number))
- assert await _until(lambda: len(_acks(connection)) == 2)
-
- ack = _acks(connection)[0]
- assert ack.params == {"sessionId": SCREENCAST_MAX_FRAMES_PER_SECOND + 1}
- assert isinstance((ack.params or {})["sessionId"], int)
- assert ack.session_id == CDP_SESSION
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_the_synthesized_ack_response_never_reaches_the_client() -> None:
- """The ack rides the proxy's own __proxy__ lane, so its reply is consumed here. A
- client that never sent the ack must never see a response for it."""
- connection = _ControllableUpstream()
- server = _screencast_server(_SharedBrowser(connection), FakeClock())
- client, task = await _attached_client(server)
-
- for number in range(1, SCREENCAST_MAX_FRAMES_PER_SECOND * 3 + 1):
- connection.emit(_screencast_frame(number))
- assert await _until(lambda: len(_acks(connection)) == SCREENCAST_MAX_FRAMES_PER_SECOND * 2)
- await _until(lambda: False, timeout_ticks=50) # let every ack reply come back
-
- # The client's only response is the reply to the attach it actually sent.
- responses = [decode_frame(wire) for wire in client.sent if isinstance(decode_frame(wire), CdpResponse)]
- assert len(responses) == 1
- assert not any(r.result and r.result.get("echo") == SCREENCAST_FRAME_ACK_METHOD for r in responses)
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-# The three ways a screencast frame can fail to reach the client WITHOUT the policy
-# throttle being involved. None is reachable from the shipped config — the middleware
-# pipeline carries only the startScreencast bounder and no rule Rewrites a frame — but
-# the pipeline and the policy are both extension seams (SKY-12532 rules, SKY-12538's
-# Rewrite-to-error, future middleware), and the ack is the property the whole feature
-# rests on. Each of these leaves the browser waiting on a frame the client will never
-# ack, so each must be paid by the proxy or the stream stalls at 0 FPS.
-
-
-class _SuppressScreencast:
- """A middleware that drops screencast frames outright."""
-
- async def __call__(self, frame: CdpFrame, direction: Direction, session: ProxySession) -> CdpFrame | None:
- if direction is Direction.UPSTREAM_TO_CLIENT and is_screencast_frame(frame):
- return None
- return frame
-
-
-class _RenumberScreencast:
- """A middleware that rewrites a screencast frame's identity — it stays a screencast
- frame, but names a frame number the browser never sent."""
-
- async def __call__(self, frame: CdpFrame, direction: Direction, session: ProxySession) -> CdpFrame | None:
- if direction is Direction.UPSTREAM_TO_CLIENT and is_screencast_frame(frame):
- assert isinstance(frame, CdpEvent)
- return CdpEvent(
- method=SCREENCAST_FRAME_EVENT,
- params={**(frame.params or {}), "sessionId": 9999},
- session_id=frame.session_id,
- )
- return frame
-
-
-class _RedactScreencast:
- """A middleware standing in for a rewrite that PRESERVES the frame's identity (e.g.
- blanking the image data). The client can still ack it, so the proxy must not."""
-
- async def __call__(self, frame: CdpFrame, direction: Direction, session: ProxySession) -> CdpFrame | None:
- if direction is Direction.UPSTREAM_TO_CLIENT and is_screencast_frame(frame):
- assert isinstance(frame, CdpEvent)
- return CdpEvent(
- method=frame.method, params={**(frame.params or {}), "data": ""}, session_id=frame.session_id
- )
- return frame
-
-
-class _RewriteToNonScreencast:
- """A policy that replaces a screencast frame with some other event — the shape
- SKY-12538's synthesized-error rewrites will take."""
-
- def decide(self, event: CdpEvent, session: ProxySession) -> PolicyDecision:
- if event.method == SCREENCAST_FRAME_EVENT:
- return Rewrite(CdpEvent(method="Log.entryAdded", params={"entry": {}}, session_id=event.session_id))
- return FORWARD
-
- def observe_command(self, command: CdpCommand, session: ProxySession) -> None:
- return None
-
- def forget(self, session_id: str) -> None:
- return None
-
-
-def _server_with(
- connection: _ControllableUpstream,
- event_policy: object | None = None,
- middleware: object | None = None,
-) -> CdpProxyServer:
- sessions = InMemorySessionRegistry()
- sessions.put(ResolvedSession(session_id="s1", upstream_adapter="memory", upstream_ws_url=UPSTREAM_URL))
- return CdpProxyServer(
- upstream=_SharedBrowser(connection), # type: ignore[arg-type]
- sessions=sessions,
- auth=AllowAllAuth(),
- metrics=NoOpMetrics(),
- event_policy=event_policy or ForwardAllEventPolicy(), # type: ignore[arg-type]
- pipeline=MiddlewarePipeline([middleware] if middleware else []), # type: ignore[list-item]
- )
-
-
-async def test_a_frame_suppressed_by_a_middleware_is_still_acked() -> None:
- """The ack cannot key off the policy decision alone: a middleware that suppresses the
- frame never reaches the policy at all, and the browser is still waiting."""
- connection = _ControllableUpstream()
- server = _server_with(connection, middleware=_SuppressScreencast())
- client, task = await _attached_client(server)
-
- for number in range(1, 4):
- connection.emit(_screencast_frame(number))
- assert await _until(lambda: _acked_numbers(connection) == [1, 2, 3])
-
- assert client.events(SCREENCAST_FRAME_EVENT) == [] # the suppression still held
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_a_frame_renumbered_by_a_middleware_acks_the_original_not_the_rewrite() -> None:
- """The ack must name the frame the BROWSER sent. Derived from the post-pipeline frame
- it would name a frame number the browser never issued, leaving the real one unacked —
- the stall, one indirection further away."""
- connection = _ControllableUpstream()
- server = _server_with(connection, middleware=_RenumberScreencast())
- client, task = await _attached_client(server)
-
- connection.emit(_screencast_frame(7))
- assert await _until(lambda: _acked_numbers(connection) == [7] and len(client.events(SCREENCAST_FRAME_EVENT)) == 1)
-
- # The client got the rewrite (and will ack 9999, which settles nothing) — so the
- # original 7 was the proxy's to pay, and it paid exactly that.
- assert [(e.params or {})["sessionId"] for e in client.events(SCREENCAST_FRAME_EVENT)] == [9999]
- assert 9999 not in _acked_numbers(connection)
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_a_rewrite_that_keeps_the_frames_identity_is_left_for_the_client_to_ack() -> None:
- """The other side of the rule. The delivered frame still names frame 7 on the same
- session, so the client acks it — a second ack from the proxy would spend the browser's
- in-flight allowance twice for one frame."""
- connection = _ControllableUpstream()
- server = _server_with(connection, middleware=_RedactScreencast())
- client, task = await _attached_client(server)
-
- connection.emit(_screencast_frame(7))
- assert await _until(lambda: len(client.events(SCREENCAST_FRAME_EVENT)) == 1)
- await _until(lambda: False, timeout_ticks=50)
-
- assert _acks(connection) == []
- assert (client.events(SCREENCAST_FRAME_EVENT)[0].params or {})["sessionId"] == 7
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_a_frame_rewritten_to_another_event_is_still_acked() -> None:
- """A policy Rewrite delivers something, but not a screencast frame — so the client has
- nothing to ack and the browser is still waiting on the original."""
- connection = _ControllableUpstream()
- server = _server_with(connection, event_policy=_RewriteToNonScreencast())
- client, task = await _attached_client(server)
-
- connection.emit(_screencast_frame(5))
- assert await _until(lambda: _acked_numbers(connection) == [5] and len(client.events("Log.entryAdded")) == 1)
-
- assert len(client.events("Log.entryAdded")) == 1 # the rewrite was still delivered
- assert client.events(SCREENCAST_FRAME_EVENT) == []
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_a_delivered_frame_is_not_acked_by_the_proxy() -> None:
- """Only a withheld frame is the proxy's to ack. Acking a delivered one too would
- double-count against the browser's in-flight allowance and pre-empt the client's own
- ack — the upstream ack behaviour this must preserve."""
- connection = _ControllableUpstream()
- server = _screencast_server(_SharedBrowser(connection), FakeClock())
- client, task = await _attached_client(server)
-
- for number in range(1, SCREENCAST_MAX_FRAMES_PER_SECOND + 1):
- connection.emit(_screencast_frame(number))
- assert await _until(lambda: len(client.events(SCREENCAST_FRAME_EVENT)) == SCREENCAST_MAX_FRAMES_PER_SECOND)
- await _until(lambda: False, timeout_ticks=50)
-
- assert _acks(connection) == []
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_a_frame_with_no_ackable_number_is_forwarded_rather_than_withheld() -> None:
- """Fail-open. A frame the proxy cannot ack is a frame it must not withhold: the
- browser would wait forever on an ack that can never be sent. Forwarding hands the
- ack back to the client, which is the only party that can still send one."""
- connection = _ControllableUpstream()
- server = _screencast_server(_SharedBrowser(connection), FakeClock())
- client, task = await _attached_client(server)
-
- for number in range(1, SCREENCAST_MAX_FRAMES_PER_SECOND + 1):
- connection.emit(_screencast_frame(number)) # spend the budget
- unackable = CdpEvent(method=SCREENCAST_FRAME_EVENT, params={"data": "x"}, session_id=CDP_SESSION)
- connection.emit(unackable)
- assert await _until(lambda: len(client.events(SCREENCAST_FRAME_EVENT)) == SCREENCAST_MAX_FRAMES_PER_SECOND + 1)
-
- assert _acks(connection) == []
- assert client.events(SCREENCAST_FRAME_EVENT)[-1].params == {"data": "x"}
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_non_screencast_traffic_is_untouched_by_the_screencast_pack() -> None:
- """Screencast, Input, Network, Fetch and Runtime all multiplex ONE CDP connection.
- The cap is on Page.screencastFrame alone — anything coarser breaks input, downloads
- and recording on the same socket."""
- connection = _ControllableUpstream()
- server = _screencast_server(_SharedBrowser(connection), FakeClock())
- client, task = await _attached_client(server)
-
- noisy = [
- CdpEvent(method="Network.dataReceived", params={"requestId": "r1"}, session_id=CDP_SESSION),
- CdpEvent(method="Runtime.consoleAPICalled", params={"type": "log"}, session_id=CDP_SESSION),
- CdpEvent(method="Page.loadEventFired", params={"timestamp": 1.0}, session_id=CDP_SESSION),
- ]
- for _ in range(100):
- for event in noisy:
- connection.emit(event)
- assert await _until(lambda: len(client.events("Page.loadEventFired")) == 100)
-
- # Every non-screencast event got through, however noisy: this pack caps one method.
- assert len(client.events("Network.dataReceived")) == 100
- assert len(client.events("Runtime.consoleAPICalled")) == 100
- assert _acks(connection) == []
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-def test_start_screencast_params_are_bounded_into_the_range_the_proxy_serves() -> None:
- command = CdpCommand(
- id=7,
- method=START_SCREENCAST_METHOD,
- params={"format": "jpeg", "quality": 100, "maxWidth": 3840, "maxHeight": 2160, "everyNthFrame": 1},
- session_id=CDP_SESSION,
- )
-
- bounded = bound_start_screencast(command)
-
- assert (bounded.params or {})["maxWidth"] == SCREENCAST_MAX_DIMENSION
- assert (bounded.params or {})["maxHeight"] == SCREENCAST_MAX_DIMENSION
- assert (bounded.params or {})["everyNthFrame"] == SCREENCAST_MIN_EVERY_NTH_FRAME
- assert (bounded.params or {})["quality"] == SCREENCAST_MAX_QUALITY
- # The codec the client's decoder expects is never swapped, and the command still
- # addresses exactly what it addressed.
- assert (bounded.params or {})["format"] == "jpeg"
- assert bounded.id == 7
- assert bounded.session_id == CDP_SESSION
-
-
-def test_a_modest_screencast_request_is_left_alone() -> None:
- """A bound is a ceiling, not a setting: a client already asking for less keeps it."""
- command = CdpCommand(
- id=1,
- method=START_SCREENCAST_METHOD,
- params={"maxWidth": 640, "maxHeight": 480, "everyNthFrame": 10, "quality": 30},
- )
-
- bounded = bound_start_screencast(command)
-
- assert (bounded.params or {})["maxWidth"] == 640
- assert (bounded.params or {})["maxHeight"] == 480
- assert (bounded.params or {})["everyNthFrame"] == 10
- assert (bounded.params or {})["quality"] == 30
-
-
-def test_an_omitted_bound_is_imposed_rather_than_left_open() -> None:
- """Every param's absent default is its expensive one: Chrome reads a missing
- maxWidth/maxHeight as 'no bound' and encodes at full viewport size, a missing
- everyNthFrame as every frame, and a missing quality as its own high default. Absent
- is the common case, so it cannot be the unbounded one."""
- bounded = bound_start_screencast(CdpCommand(id=1, method=START_SCREENCAST_METHOD))
-
- assert (bounded.params or {})["maxWidth"] == SCREENCAST_MAX_DIMENSION
- assert (bounded.params or {})["maxHeight"] == SCREENCAST_MAX_DIMENSION
- assert (bounded.params or {})["everyNthFrame"] == SCREENCAST_MIN_EVERY_NTH_FRAME
- assert (bounded.params or {})["quality"] == SCREENCAST_MAX_QUALITY
-
-
[email protected]("value", [-1, 101, "60", 60.0, True, None])
-def test_a_nonsense_quality_is_replaced_by_the_bound(value: object) -> None:
- """Outside [0, 100] is not a request Chrome honours either, so it lands on the
- ceiling rather than being forwarded as-is."""
- bounded = bound_start_screencast(CdpCommand(id=1, method=START_SCREENCAST_METHOD, params={"quality": value}))
-
- assert (bounded.params or {})["quality"] == SCREENCAST_MAX_QUALITY
-
-
[email protected]("value", [0, -1, "1280", 1.5, True, None])
-def test_a_nonsense_dimension_is_replaced_by_the_bound(value: object) -> None:
- """A non-positive or non-integer bound is not a small request — Chrome ignores it
- and encodes unbounded, so it must land on the ceiling, not be honoured as-is."""
- bounded = bound_start_screencast(CdpCommand(id=1, method=START_SCREENCAST_METHOD, params={"maxWidth": value}))
-
- assert (bounded.params or {})["maxWidth"] == SCREENCAST_MAX_DIMENSION
-
-
-async def test_only_the_client_to_upstream_start_command_is_rewritten() -> None:
- """The bound belongs on the way in. A client's own command is what the proxy may
- clamp; nothing on the upstream leg is a startScreencast to rewrite."""
- connection = _ControllableUpstream()
- server = _screencast_server(_SharedBrowser(connection), FakeClock())
- client = _ScriptClient(
- [
- _command(1, "Target.attachToTarget", {"targetId": CDP_SESSION, "flatten": True}),
- _command(2, START_SCREENCAST_METHOD, {"maxWidth": 3840, "everyNthFrame": 1}),
- _command(3, "Page.enable"),
- ]
- )
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: len(connection.sent) == 3)
-
- sent = [decode_frame(wire) for wire in connection.sent]
- start = next(f for f in sent if isinstance(f, CdpCommand) and f.method == START_SCREENCAST_METHOD)
- assert (start.params or {})["maxWidth"] == SCREENCAST_MAX_DIMENSION
- assert (start.params or {})["everyNthFrame"] == SCREENCAST_MIN_EVERY_NTH_FRAME
- assert (start.params or {})["quality"] == SCREENCAST_MAX_QUALITY
- # A neighbouring command on the same socket is relayed byte-identically.
- enable = next(f for f in sent if isinstance(f, CdpCommand) and f.method == "Page.enable")
- assert enable.params is None
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-def test_the_ack_reads_the_frame_number_not_a_session_id() -> None:
- ack = screencast_frame_ack(_screencast_frame(42))
-
- assert ack is not None
- assert ack.method == SCREENCAST_FRAME_ACK_METHOD
- assert ack.params == {"sessionId": 42}
- assert ack.session_id == CDP_SESSION
-
-
[email protected]("params", [None, {}, {"sessionId": "sess"}, {"sessionId": None}, {"sessionId": True}])
-def test_an_unackable_frame_yields_no_ack(params: dict | None) -> None:
- """A frame number that is not an integer is not a frame number. `True` is an int to
- Python and would ack frame 1 — a frame the browser may genuinely be waiting on."""
- assert screencast_frame_ack(CdpEvent(method=SCREENCAST_FRAME_EVENT, params=params)) is None
-
-
-async def test_the_pinned_single_client_relay_consumes_the_proxy_acks_reply() -> None:
- """The pinned per-client relay hands frames straight to one client, so it has to
- consume the proxy lane's reply itself — the shared reader's routing is not there to do
- it. An unconsumed reply arrives as a response to whatever the client has in flight
- under that id, and the proxy lane allocates from the same id space precisely so that
- can never happen.
-
- No production caller today (the served path is the shared reader); this pins the
- contract so wiring it in later cannot quietly resurrect the trap.
- """
- from tests.unit.proxy.test_metrics_contract import _FakeClientWebSocket
-
- connection = _ControllableUpstream()
- server = _screencast_server(_SharedBrowser(connection), FakeClock())
- remapper = RequestIdRemapper()
- session = ProxySession(session_id="s1", upstream_ws_url=UPSTREAM_URL)
- ws = _FakeClientWebSocket("/s1")
- relay = asyncio.create_task(
- server._relay_upstream_to_client(ws, connection, session, remapper, {}) # type: ignore[arg-type]
- )
- # One frame over a spent budget, so the relay withholds it and pays the ack itself.
- for number in range(1, SCREENCAST_MAX_FRAMES_PER_SECOND + 2):
- connection.emit(_screencast_frame(number))
- # Settles once the ack has gone AND its reply has come back and been consumed (which
- # is what clears the pending mapping) — so this cannot pass by measuring too early.
- assert await _until(lambda: _acked_numbers(connection) == [11] and remapper.pending_count == 0)
-
- # The ack's reply came back on this same socket and stopped here: the client saw only
- # screencast frames, never a response to a command it never sent.
- sent = [decode_frame(wire) for wire in ws.sent]
- assert [f for f in sent if isinstance(f, CdpResponse)] == []
- assert len([f for f in sent if isinstance(f, CdpEvent)]) == SCREENCAST_MAX_FRAMES_PER_SECOND
-
- relay.cancel()
- with contextlib.suppress(asyncio.CancelledError):
- await relay
-
-
-def test_the_capped_event_is_a_named_metric_label_not_bucketed_to_other() -> None:
- """A drop counter tagged cdp_method="other" cannot say the reduction came from the
- screencast stream. The pack's own method is allowlisted; the set stays bounded."""
- assert _cdp_method_tags(SCREENCAST_FRAME_EVENT) == (SCREENCAST_FRAME_EVENT, "Page")
-
-
-def test_the_pack_caps_screencast_frames_and_nothing_else() -> None:
- """The blast radius, as data: one rule, on one event, never relaxed by interest.
-
- Interest-relaxation would void this cap outright — a screencast client enables Page
- like every other client, so `Page` being enabled says nothing about whether it asked
- for a screencast (Page.startScreencast does), and relaxing on it would exempt exactly
- the clients this bounds.
- """
- assert [rule.method for rule in SCREENCAST_PACK_V1.rules] == [SCREENCAST_FRAME_EVENT]
- rule = SCREENCAST_PACK_V1.rules[0]
- assert rule.max_per_window == SCREENCAST_MAX_FRAMES_PER_SECOND
- assert rule.window_seconds == 1.0
- assert rule.relax_when_enabled is False
- assert SCREENCAST_PACK_V1.version == 1
-
-
-async def test_the_default_proxy_neither_caps_nor_acks(monkeypatch: pytest.MonkeyPatch) -> None:
- """Filtering is opt-in. With no screencast policy the proxy is a pass-through and the
- client owns the ack loop exactly as it does without a proxy in the path."""
- from skyvern.proxy.__main__ import build_event_policy, build_pipeline
-
- connection = _ControllableUpstream()
- sessions = InMemorySessionRegistry()
- sessions.put(ResolvedSession(session_id="s1", upstream_adapter="memory", upstream_ws_url=UPSTREAM_URL))
- server = CdpProxyServer(
- upstream=_SharedBrowser(connection), # type: ignore[arg-type]
- sessions=sessions,
- auth=AllowAllAuth(),
- metrics=NoOpMetrics(),
- event_policy=build_event_policy("forward-all"),
- pipeline=build_pipeline("forward-all"),
- )
- client = _ScriptClient(
- [
- _command(1, "Target.attachToTarget", {"targetId": CDP_SESSION, "flatten": True}),
- _command(2, START_SCREENCAST_METHOD, {"maxWidth": 3840, "everyNthFrame": 1}),
- ]
- )
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: len(connection.sent) == 2)
-
- for number in range(1, 101):
- connection.emit(_screencast_frame(number))
- assert await _until(lambda: len(client.events(SCREENCAST_FRAME_EVENT)) == 100)
-
- # Every frame delivered, no frame acked by us, and the client's own params reached
- # the browser exactly as it wrote them.
- assert _acks(connection) == []
- start = decode_frame(connection.sent[1])
- assert isinstance(start, CdpCommand)
- assert start.params == {"maxWidth": 3840, "everyNthFrame": 1}
- assert encode_frame(start) == encode_frame(
- CdpCommand(id=start.id, method=START_SCREENCAST_METHOD, params={"maxWidth": 3840, "everyNthFrame": 1})
- )
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
diff --git a/tests/unit/proxy/test_session.py b/tests/unit/proxy/test_session.py
deleted file mode 100644
index 0d985f1b9..000000000
--- a/tests/unit/proxy/test_session.py
+++ /dev/null
@@ -1,63 +0,0 @@
-from __future__ import annotations
-
-import pytest
-
-from skyvern.proxy.core.session import Principal, ProxySession
-
-
- ("session_id", "upstream_ws_url"),
- [("", "ws://localhost:1"), ("session", ""), (" ", "ws://localhost:1")],
-)
-def test_proxy_session_rejects_empty_routing_fields(session_id: str, upstream_ws_url: str) -> None:
- with pytest.raises(ValueError):
- ProxySession(session_id=session_id, upstream_ws_url=upstream_ws_url)
-
-
-def test_proxy_session_tracks_flat_cdp_sessions() -> None:
- session = ProxySession(session_id="session", upstream_ws_url="ws://localhost:1")
-
- assert session.attach_cdp_session("page-session") is True
- assert session.attach_cdp_session("page-session") is False
- assert session.has_cdp_session("page-session") is True
- assert session.detach_cdp_session("page-session") is True
- assert session.detach_cdp_session("page-session") is False
- assert session.has_cdp_session("page-session") is False
-
-
-def test_proxy_session_rejects_empty_cdp_session_ids() -> None:
- session = ProxySession(session_id="session", upstream_ws_url="ws://localhost:1")
-
- with pytest.raises(ValueError):
- session.attach_cdp_session("")
- with pytest.raises(ValueError):
- session.has_cdp_session(" ")
- with pytest.raises(ValueError):
- session.detach_cdp_session("")
-
-
-def test_proxy_session_access_rule_is_principal_based() -> None:
- owner = Principal(principal_id="owner", organization_id="organization")
- session = ProxySession(session_id="session", upstream_ws_url="ws://localhost:1", principal=owner)
-
- assert session.allows_principal(owner)
- assert not session.allows_principal(Principal(principal_id="other", organization_id="organization"))
- assert ProxySession(session_id="open", upstream_ws_url="ws://localhost:1").allows_principal(
- Principal(principal_id="anyone")
- )
-
-
-def test_principal_rejects_empty_identity() -> None:
- with pytest.raises(ValueError):
- Principal(principal_id="")
-
-
-def test_proxy_session_repr_never_leaks_url_or_headers() -> None:
- session = ProxySession(
- session_id="s1",
- upstream_ws_url="ws://upstream.internal:9222/devtools/browser/abc?token=secret-token",
- connect_headers={"authorization": "Bearer secret-header"},
- )
- assert "secret-token" not in repr(session)
- assert session.upstream_ws_url not in repr(session)
- assert "secret-header" not in repr(session)
diff --git a/tests/unit/proxy/test_session_registry_contract.py b/tests/unit/proxy/test_session_registry_contract.py
deleted file mode 100644
index 9258751c8..000000000
--- a/tests/unit/proxy/test_session_registry_contract.py
+++ /dev/null
@@ -1,322 +0,0 @@
-"""Reusable contract suite for SessionRegistryPort adapters.
-
-Any adapter (generic here, cloud-specific under tests/cloud/) subclasses
-SessionRegistryContract and overrides make_registry(); every behavioral
-guarantee of the port is asserted once, here.
-"""
-
-from __future__ import annotations
-
-import asyncio
-
-import pytest
-
-from skyvern.proxy.adapters.caching import TtlCachingSessionRegistry
-from skyvern.proxy.adapters.memory import InMemorySessionRegistry
-from skyvern.proxy.core.session import ResolvedSession, SessionResolution, SessionResolutionStatus
-from skyvern.proxy.ports import SessionRegistryPort
-
-SECRET_UPSTREAM_URL = "ws://upstream.internal:9222/devtools/browser/abc?token=secret-token"
-
-
-def make_resolved_session(session_id: str = "pbs_active", organization_id: str | None = "o_1") -> ResolvedSession:
- return ResolvedSession(
- session_id=session_id,
- upstream_adapter="websocket",
- upstream_ws_url=SECRET_UPSTREAM_URL,
- organization_id=organization_id,
- )
-
-
-class SessionRegistryContract:
- def make_registry(
- self,
- *,
- active: tuple[ResolvedSession, ...] = (),
- pending: tuple[str, ...] = (),
- closed: tuple[str, ...] = (),
- expired: tuple[str, ...] = (),
- ) -> SessionRegistryPort:
- raise NotImplementedError
-
- @pytest.mark.asyncio
- async def test_unknown_session_is_an_explicit_negative(self) -> None:
- resolution = await self.make_registry().resolve("pbs_missing")
- assert resolution.status is SessionResolutionStatus.UNKNOWN
- assert resolution.session is None
-
- @pytest.mark.asyncio
- async def test_active_session_resolves_full_routing(self) -> None:
- seeded = make_resolved_session()
- registry = self.make_registry(active=(seeded,))
- resolution = await registry.resolve(seeded.session_id)
- assert resolution.status is SessionResolutionStatus.ACTIVE
- assert resolution.session == seeded
-
- @pytest.mark.asyncio
- async def test_pending_session_is_rejected_without_routing_data(self) -> None:
- resolution = await self.make_registry(pending=("pbs_pending",)).resolve("pbs_pending")
- assert resolution.status is SessionResolutionStatus.PENDING
- assert resolution.session is None
-
- @pytest.mark.asyncio
- async def test_closed_session_is_rejected_without_routing_data(self) -> None:
- resolution = await self.make_registry(closed=("pbs_closed",)).resolve("pbs_closed")
- assert resolution.status is SessionResolutionStatus.CLOSED
- assert resolution.session is None
-
- @pytest.mark.asyncio
- async def test_expired_session_is_rejected_without_routing_data(self) -> None:
- resolution = await self.make_registry(expired=("pbs_expired",)).resolve("pbs_expired")
- assert resolution.status is SessionResolutionStatus.EXPIRED
- assert resolution.session is None
-
- @pytest.mark.asyncio
- async def test_resolution_repr_never_leaks_the_upstream_url(self) -> None:
- seeded = make_resolved_session()
- resolution = await self.make_registry(active=(seeded,)).resolve(seeded.session_id)
- assert "secret-token" not in repr(resolution)
- assert seeded.upstream_ws_url not in repr(resolution)
-
- @pytest.mark.asyncio
- async def test_invalidate_is_safe_and_idempotent_for_unknown_sessions(self) -> None:
- registry = self.make_registry()
- await registry.invalidate("pbs_missing")
- await registry.invalidate("pbs_missing")
-
-
-class TestInMemorySessionRegistry(SessionRegistryContract):
- def make_registry(
- self,
- *,
- active: tuple[ResolvedSession, ...] = (),
- pending: tuple[str, ...] = (),
- closed: tuple[str, ...] = (),
- expired: tuple[str, ...] = (),
- ) -> InMemorySessionRegistry:
- registry = InMemorySessionRegistry()
- for session in active:
- registry.put(session)
- for session_id in pending:
- registry.mark_pending(session_id)
- for session_id in closed:
- registry.mark_closed(session_id)
- for session_id in expired:
- registry.mark_expired(session_id)
- return registry
-
- @pytest.mark.asyncio
- async def test_put_preserves_adapter_selector_and_connect_headers(self) -> None:
- seeded = ResolvedSession(
- session_id="pbs_vendor",
- upstream_adapter="hosted-pool-a",
- upstream_ws_url="wss://pool.internal/session/1",
- organization_id="o_2",
- connect_headers={"authorization": "Bearer operator-token"},
- )
- registry = self.make_registry(active=(seeded,))
- resolution = await registry.resolve("pbs_vendor")
- assert resolution.session == seeded
- assert resolution.session.connect_headers == {"authorization": "Bearer operator-token"}
-
- @pytest.mark.asyncio
- async def test_remove_returns_session_to_unknown(self) -> None:
- registry = self.make_registry(active=(make_resolved_session(),))
- registry.remove("pbs_active")
- resolution = await registry.resolve("pbs_active")
- assert resolution.status is SessionResolutionStatus.UNKNOWN
-
-
-class TestTtlCachingSessionRegistry(SessionRegistryContract):
- def make_registry(
- self,
- *,
- active: tuple[ResolvedSession, ...] = (),
- pending: tuple[str, ...] = (),
- closed: tuple[str, ...] = (),
- expired: tuple[str, ...] = (),
- ) -> SessionRegistryPort:
- inner = TestInMemorySessionRegistry().make_registry(
- active=active, pending=pending, closed=closed, expired=expired
- )
- return TtlCachingSessionRegistry(inner)
-
-
-class FakeClock:
- def __init__(self) -> None:
- self.now = 0.0
-
- def __call__(self) -> float:
- return self.now
-
- def advance(self, seconds: float) -> None:
- self.now += seconds
-
-
-class CountingRegistry:
- def __init__(self, inner: SessionRegistryPort) -> None:
- self.inner = inner
- self.resolve_calls = 0
- self.invalidated: list[str] = []
-
- async def resolve(self, session_id: str) -> SessionResolution:
- self.resolve_calls += 1
- return await self.inner.resolve(session_id)
-
- async def invalidate(self, session_id: str) -> None:
- self.invalidated.append(session_id)
- await self.inner.invalidate(session_id)
-
-
-def make_cached_counting_registry(
- inner: InMemorySessionRegistry, ttl_seconds: float = 15.0
-) -> tuple[TtlCachingSessionRegistry, CountingRegistry, FakeClock]:
- clock = FakeClock()
- counting = CountingRegistry(inner)
- return TtlCachingSessionRegistry(counting, ttl_seconds=ttl_seconds, clock=clock), counting, clock
-
-
-async def test_repeat_resolve_within_ttl_hits_backing_store_once() -> None:
- inner = InMemorySessionRegistry()
- inner.put(make_resolved_session())
- cached, counting, _ = make_cached_counting_registry(inner)
-
- first = await cached.resolve("pbs_active")
- second = await cached.resolve("pbs_active")
-
- assert counting.resolve_calls == 1
- assert first == second
-
-
-async def test_resolution_refreshes_after_ttl_elapses() -> None:
- inner = InMemorySessionRegistry()
- inner.put(make_resolved_session())
- cached, counting, clock = make_cached_counting_registry(inner, ttl_seconds=15.0)
-
- await cached.resolve("pbs_active")
- clock.advance(15.0)
- inner.mark_closed("pbs_active")
- resolution = await cached.resolve("pbs_active")
-
- assert counting.resolve_calls == 2
- assert resolution.status is SessionResolutionStatus.CLOSED
-
-
-async def test_invalidate_on_close_drops_cache_and_forwards() -> None:
- inner = InMemorySessionRegistry()
- inner.put(make_resolved_session())
- cached, counting, _ = make_cached_counting_registry(inner)
-
- assert (await cached.resolve("pbs_active")).status is SessionResolutionStatus.ACTIVE
- inner.mark_closed("pbs_active")
- await cached.invalidate("pbs_active")
- resolution = await cached.resolve("pbs_active")
-
- assert resolution.status is SessionResolutionStatus.CLOSED
- assert counting.invalidated == ["pbs_active"]
-
-
-async def test_unknown_and_pending_are_not_cached() -> None:
- inner = InMemorySessionRegistry()
- inner.mark_pending("pbs_pending")
- cached, counting, _ = make_cached_counting_registry(inner)
-
- assert (await cached.resolve("pbs_new")).status is SessionResolutionStatus.UNKNOWN
- assert (await cached.resolve("pbs_pending")).status is SessionResolutionStatus.PENDING
- inner.put(make_resolved_session(session_id="pbs_new"))
- inner.put(make_resolved_session(session_id="pbs_pending"))
-
- assert (await cached.resolve("pbs_new")).status is SessionResolutionStatus.ACTIVE
- assert (await cached.resolve("pbs_pending")).status is SessionResolutionStatus.ACTIVE
- assert counting.resolve_calls == 4
-
-
-async def test_terminal_negatives_are_cached() -> None:
- inner = InMemorySessionRegistry()
- inner.mark_closed("pbs_closed")
- inner.mark_expired("pbs_expired")
- cached, counting, _ = make_cached_counting_registry(inner)
-
- for _ in range(2):
- assert (await cached.resolve("pbs_closed")).status is SessionResolutionStatus.CLOSED
- assert (await cached.resolve("pbs_expired")).status is SessionResolutionStatus.EXPIRED
-
- assert counting.resolve_calls == 2
-
-
-async def test_active_cache_entry_is_capped_at_the_session_deadline() -> None:
- inner = InMemorySessionRegistry()
- inner.put(make_resolved_session(), expires_in_seconds=5.0)
- cached, counting, clock = make_cached_counting_registry(inner, ttl_seconds=15.0)
-
- await cached.resolve("pbs_active")
- clock.advance(3.0)
- await cached.resolve("pbs_active")
- assert counting.resolve_calls == 1
-
- clock.advance(2.0)
- inner.mark_expired("pbs_active")
- resolution = await cached.resolve("pbs_active")
-
- assert counting.resolve_calls == 2
- assert resolution.status is SessionResolutionStatus.EXPIRED
-
-
-class DelayedRegistry:
- """Inner registry whose first resolve blocks until released, to race invalidate()."""
-
- def __init__(self, resolution: SessionResolution) -> None:
- self.resolution = resolution
- self.entered = asyncio.Event()
- self.release = asyncio.Event()
- self.resolve_calls = 0
-
- async def resolve(self, session_id: str) -> SessionResolution:
- self.resolve_calls += 1
- if self.resolve_calls == 1:
- self.entered.set()
- await self.release.wait()
- return self.resolution
-
- async def invalidate(self, session_id: str) -> None:
- return None
-
-
-async def test_invalidate_during_inflight_resolve_never_caches_the_stale_result() -> None:
- inner = DelayedRegistry(SessionResolution.active(make_resolved_session()))
- cached = TtlCachingSessionRegistry(inner, ttl_seconds=60.0)
-
- inflight = asyncio.create_task(cached.resolve("pbs_active"))
- await inner.entered.wait()
- await cached.invalidate("pbs_active")
- inner.release.set()
- stale = await inflight
-
- assert stale.status is SessionResolutionStatus.ACTIVE
- inner.resolution = SessionResolution.closed()
- fresh = await cached.resolve("pbs_active")
- assert fresh.status is SessionResolutionStatus.CLOSED
- assert inner.resolve_calls == 2
-
-
-async def test_cache_size_is_bounded_with_oldest_entry_evicted() -> None:
- inner = InMemorySessionRegistry()
- for index in range(3):
- inner.put(make_resolved_session(session_id=f"pbs_{index}"))
- counting = CountingRegistry(inner)
- cached = TtlCachingSessionRegistry(counting, ttl_seconds=60.0, clock=FakeClock(), max_entries=2)
-
- for index in range(3):
- await cached.resolve(f"pbs_{index}")
- assert len(cached._cache) == 2
-
- await cached.resolve("pbs_0")
- assert counting.resolve_calls == 4
diff --git a/tests/unit/proxy/test_shared_connection.py b/tests/unit/proxy/test_shared_connection.py
deleted file mode 100644
index 994c207b8..000000000
--- a/tests/unit/proxy/test_shared_connection.py
+++ /dev/null
@@ -1,989 +0,0 @@
-"""Shared upstream-connection ownership for the CDP proxy (SKY-12499).
-
-The proxy owns ONE persistent upstream connection per browser (keyed by the
-resolved upstream target), ref-counted across clients: it opens on the first
-client and tears down when the last leaves. Two clients on one browser multiplex
-the single connection — responses route back by the shared, client-keyed request-id
-remapper (no cross-talk) and session-scoped events route by the flat-session
-sessionId learned at attach. An upstream failure closes every sharing client
-deterministically with no leaked tasks.
-"""
-
-from __future__ import annotations
-
-import asyncio
-from types import SimpleNamespace
-
-import pytest
-
-from skyvern.proxy.adapters.memory import AllowAllAuth, ForwardAllEventPolicy, InMemorySessionRegistry, NoOpMetrics
-from skyvern.proxy.adapters.websocket_server import (
- _MAX_PENDING_COMMANDS,
- _UPSTREAM_UNAVAILABLE_CLOSE_CODE,
- CdpProxyServer,
- _AttachIntent,
- _record_attach_intent,
- _SharedUpstream,
-)
-from skyvern.proxy.core.frames import CdpCommand, CdpEvent, CdpFrame, CdpResponse, decode_frame, encode_frame
-from skyvern.proxy.core.pipeline import Direction, MiddlewarePipeline
-from skyvern.proxy.core.session import ProxySession, ResolvedSession, UpstreamClosedError
-
-UPSTREAM_URL = "ws://browser.internal/devtools/browser/shared"
-
-
-class _ControllableUpstream:
- """One upstream connection: auto-answers commands (echoing the method, or a
- sessionId for an attach) and lets a test inject events or fail the socket."""
-
- def __init__(self) -> None:
- self._inbox: asyncio.Queue[str | None] = asyncio.Queue()
- self.sent: list[str] = []
- self.close_calls = 0
- self._closed = False
-
- async def send(self, raw: str) -> None:
- if self._closed:
- raise UpstreamClosedError("closed")
- self.sent.append(raw)
- frame = decode_frame(raw)
- if isinstance(frame, CdpCommand):
- if frame.method == "Target.attachToTarget":
- cdp_session_id = (frame.params or {}).get("targetId", "sess")
- self._inbox.put_nowait(encode_frame(CdpResponse(id=frame.id, result={"sessionId": cdp_session_id})))
- else:
- self._inbox.put_nowait(encode_frame(CdpResponse(id=frame.id, result={"echo": frame.method})))
-
- async def receive(self) -> str:
- if self._closed and self._inbox.empty():
- raise UpstreamClosedError("closed")
- item = await self._inbox.get()
- if item is None:
- raise UpstreamClosedError("closed")
- return item
-
- async def close(self) -> None:
- self.close_calls += 1
- self._closed = True
- self._inbox.put_nowait(None)
-
- def emit(self, frame: CdpFrame) -> None:
- self._inbox.put_nowait(encode_frame(frame))
-
- def fail(self) -> None:
- self._closed = True
- self._inbox.put_nowait(None)
-
-
-class _SilentUpstream(_ControllableUpstream):
- """Never answers a command, so every request stays in flight — the oracle for
- 'remap tables are empty after quiescence' (SKY-12500 AC5)."""
-
- async def send(self, raw: str) -> None:
- if self._closed:
- raise UpstreamClosedError("closed")
- self.sent.append(raw)
-
-
-class _AttachEventUpstream(_ControllableUpstream):
- """Answers an explicit attach the way Chrome really does: the browser-level
- Target.attachedToTarget EVENT lands BEFORE the command response, so at event time
- the attach has not yet been correlated to the client that asked for it."""
-
- async def send(self, raw: str) -> None:
- frame = decode_frame(raw)
- if isinstance(frame, CdpCommand) and frame.method == "Target.attachToTarget":
- target_id = (frame.params or {}).get("targetId", "sess")
- self.sent.append(raw)
- self.emit(_attached_to_target(target_id, target_id=target_id))
- self._inbox.put_nowait(encode_frame(CdpResponse(id=frame.id, result={"sessionId": target_id})))
- return
- await super().send(raw)
-
-
-class _SharedBrowser:
- """UpstreamBrowserPort that hands out one shared connection and counts dials."""
-
- def __init__(self, connection: _ControllableUpstream) -> None:
- self._connection = connection
- self.connect_calls = 0
- self.sessions: list[ProxySession] = []
-
- async def connect(self, session: ProxySession) -> _ControllableUpstream:
- self.connect_calls += 1
- self.sessions.append(session)
- return self._connection
-
-
-class _RedialBrowser:
- """UpstreamBrowserPort that dials a FRESH connection each time (a re-dial after a
- dead connection must not reuse the corpse)."""
-
- def __init__(self) -> None:
- self.connect_calls = 0
- self.connections: list[_ControllableUpstream] = []
-
- async def connect(self, session: ProxySession) -> _ControllableUpstream:
- self.connect_calls += 1
- connection = _ControllableUpstream()
- self.connections.append(connection)
- return connection
-
-
-class _GatedBrowser:
- """UpstreamBrowserPort whose dial blocks until the test opens the gate — used to
- freeze a client mid-acquire and drive the single-flight/attach-seam races."""
-
- def __init__(self, connection: _ControllableUpstream) -> None:
- self._connection = connection
- self.connect_calls = 0
- self.gate = asyncio.Event()
-
- async def connect(self, session: ProxySession) -> _ControllableUpstream:
- self.connect_calls += 1
- await self.gate.wait()
- return self._connection
-
-
-class _ScriptClient:
- """Client ws: yields its scripted frames, then stays live until released. A test
- can `feed` a further frame once the connection is live (e.g. after ownership of a
- session has been learned)."""
-
- def __init__(self, incoming: list[str]) -> None:
- self.request = SimpleNamespace(headers={}, path="/s1")
- self.sent: list[str] = []
- self.close_code: int | None = None
- self._incoming: asyncio.Queue[str | None] = asyncio.Queue()
- for raw in incoming:
- self._incoming.put_nowait(raw)
-
- def __aiter__(self) -> _ScriptClient:
- return self
-
- async def __anext__(self) -> str:
- raw = await self._incoming.get()
- if raw is None:
- raise StopAsyncIteration
- return raw
-
- def feed(self, raw: str) -> None:
- self._incoming.put_nowait(raw)
-
- async def send(self, raw: str) -> None:
- self.sent.append(raw)
-
- async def close(self, code: int = 1000, reason: str = "") -> None:
- self.close_code = code
- self.release()
-
- def release(self) -> None:
- self._incoming.put_nowait(None)
-
- def echoes(self) -> set[str]:
- methods = set()
- for wire in self.sent:
- frame = decode_frame(wire)
- if isinstance(frame, CdpResponse) and frame.result and "echo" in frame.result:
- methods.add(frame.result["echo"])
- return methods
-
- def event_sessions(self) -> set[str]:
- return {decode_frame(wire).session_id for wire in self.sent if isinstance(decode_frame(wire), CdpEvent)}
-
- def events(self, method: str) -> list[CdpEvent]:
- frames = (decode_frame(wire) for wire in self.sent)
- return [f for f in frames if isinstance(f, CdpEvent) and f.method == method]
-
- def errors(self) -> list[CdpResponse]:
- frames = (decode_frame(wire) for wire in self.sent)
- return [f for f in frames if isinstance(f, CdpResponse) and f.error is not None]
-
-
-class _SlowDrainClient(_ScriptClient):
- """A client whose sends are slow, so its teardown drain lingers — widening the
- window in which a racing client could observe stale shared state."""
-
- async def send(self, raw: str) -> None:
- await asyncio.sleep(0.1)
- self.sent.append(raw)
-
-
-def _server(browser: object, pipeline: MiddlewarePipeline | None = None) -> CdpProxyServer:
- sessions = InMemorySessionRegistry()
- sessions.put(ResolvedSession(session_id="s1", upstream_adapter="memory", upstream_ws_url=UPSTREAM_URL))
- return CdpProxyServer(
- upstream=browser, # type: ignore[arg-type]
- sessions=sessions,
- auth=AllowAllAuth(),
- metrics=NoOpMetrics(),
- event_policy=ForwardAllEventPolicy(),
- pipeline=pipeline,
- )
-
-
-async def _until(predicate, timeout_ticks: int = 2000) -> bool:
- for _ in range(timeout_ticks):
- if predicate():
- return True
- await asyncio.sleep(0)
- return predicate()
-
-
-def _channel_count(server: CdpProxyServer) -> int:
- entry = server._shared_upstreams.get(UPSTREAM_URL)
- return len(entry.channels) if entry is not None else 0
-
-
-def _command(client_id: int, method: str, params: dict | None = None) -> str:
- return encode_frame(CdpCommand(id=client_id, method=method, params=params))
-
-
-async def test_two_clients_share_a_single_upstream_connection() -> None:
- connection = _ControllableUpstream()
- browser = _SharedBrowser(connection)
- server = _server(browser)
- client_a = _ScriptClient([_command(1, "Page.enable")])
- client_b = _ScriptClient([_command(1, "Network.enable")])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: client_a.sent and client_b.sent)
-
- # One browser instance, dialed once, but both clients' commands reached it.
- assert browser.connect_calls == 1
- assert len(browser.sessions) == 1
- assert len(connection.sent) == 2
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
- # Torn down only when the last client left — closed exactly once.
- assert connection.close_calls == 1
-
-
-async def test_no_cross_talk_between_two_clients_on_the_shared_connection() -> None:
- connection = _ControllableUpstream()
- server = _server(_SharedBrowser(connection))
- # Both clients use client id 1; only the shared remapper keeps them apart.
- client_a = _ScriptClient([_command(1, "Page.enable")])
- client_b = _ScriptClient([_command(1, "Network.enable")])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: client_a.echoes() and client_b.echoes())
-
- # Each client gets its own response back under its own id 1, never the other's.
- assert client_a.echoes() == {"Page.enable"}
- assert client_b.echoes() == {"Network.enable"}
- assert all(decode_frame(wire).id == 1 for wire in client_a.sent)
- assert all(decode_frame(wire).id == 1 for wire in client_b.sent)
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
-
-async def test_flat_session_events_route_to_the_owning_client() -> None:
- connection = _ControllableUpstream()
- server = _server(_SharedBrowser(connection))
- client_a = _ScriptClient([_command(1, "Target.attachToTarget", {"targetId": "tA", "flatten": True})])
- client_b = _ScriptClient([_command(1, "Target.attachToTarget", {"targetId": "tB", "flatten": True})])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- # Wait until both attach replies land (so both sessionId owners are learned).
- assert await _until(lambda: client_a.sent and client_b.sent)
-
- connection.emit(CdpEvent(method="Page.loadEventFired", session_id="tA"))
- connection.emit(CdpEvent(method="Page.loadEventFired", session_id="tB"))
- assert await _until(lambda: "tA" in client_a.event_sessions() and "tB" in client_b.event_sessions())
-
- # A session-scoped event reaches only the client that owns that sessionId.
- assert client_a.event_sessions() == {"tA"}
- assert client_b.event_sessions() == {"tB"}
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
-
-async def test_upstream_failure_closes_every_sharing_client_without_leaks() -> None:
- connection = _ControllableUpstream()
- browser = _SharedBrowser(connection)
- server = _server(browser)
- client_a = _ScriptClient([_command(1, "Page.enable")])
- client_b = _ScriptClient([_command(1, "Network.enable")])
-
- before = set(asyncio.all_tasks())
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: client_a.sent and client_b.sent)
-
- # The single shared upstream drops: both clients must tear down deterministically.
- connection.fail()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
- assert connection.close_calls == 1 # one connection, closed once
- leaked = [t for t in set(asyncio.all_tasks()) - before if not t.done()]
- assert leaked == [] # no reader / relay / writer task leaked
-
-
-async def test_reader_death_evicts_the_entry_so_a_new_client_never_joins_a_dead_reader() -> None:
- # FIX 1: on reader death the pool entry must be evicted, so a client acquiring
- # during the (slow) teardown drain dials FRESH or is cleanly rejected — never
- # joins the dead reader and hangs. Without eviction the stale entry is reused and
- # the new client's send hits the corpse (no response) — the assertion below fails.
- browser = _RedialBrowser()
- server = _server(browser)
- client_a = _SlowDrainClient([_command(1, "Page.enable")])
-
- before = set(asyncio.all_tasks())
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- # A is attached and its reader is live (its slow drain would delay client_a.sent).
- assert await _until(lambda: _channel_count(server) >= 1)
- browser.connections[0].fail() # the shared upstream dies
-
- client_c = _ScriptClient([_command(1, "Runtime.enable")])
- task_c = asyncio.create_task(server._handle_client(client_c)) # type: ignore[arg-type]
- # C reaches a definitive outcome instead of hanging on the dead reader.
- assert await _until(
- lambda: client_c.echoes() or client_c.close_code == _UPSTREAM_UNAVAILABLE_CLOSE_CODE or task_c.done()
- )
- assert client_c.echoes() == {"Runtime.enable"} or client_c.close_code == _UPSTREAM_UNAVAILABLE_CLOSE_CODE
-
- client_c.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_c), timeout=5)
- # Fresh dial (a second connection) or a clean rejection — never a dead-reader join.
- assert browser.connect_calls == 2 or client_c.close_code == _UPSTREAM_UNAVAILABLE_CLOSE_CODE
- leaked = [t for t in set(asyncio.all_tasks()) - before if not t.done()]
- assert leaked == []
-
-
-async def test_cancel_at_the_attach_seam_releases_the_ref_and_evicts() -> None:
- # FIX 2: cancelling _handle_client after acquire (ref held) but at the attach seam
- # must still run the release path — no stuck refcount, no pinned entry, no leaked
- # socket/reader/channel. The pool lock is held here to freeze the client exactly at
- # _attach_client (which acquire has already passed), then it is cancelled.
- connection = _ControllableUpstream()
- browser = _GatedBrowser(connection)
- server = _server(browser)
- client = _ScriptClient([])
-
- before = set(asyncio.all_tasks())
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: browser.connect_calls == 1) # in acquire, ref held, dial in-flight
- await server._shared_upstreams_lock.acquire() # the next attach will block on this
- browser.gate.set() # dial completes; the client races to attach and blocks on the lock
- for _ in range(50): # let it reach the attach block (the only place it can be)
- await asyncio.sleep(0)
- assert not client.sent # never attached (would need the lock we hold)
-
- task.cancel()
- server._shared_upstreams_lock.release() # let the teardown's release_shared proceed
- with pytest.raises(asyncio.CancelledError):
- await task
-
- assert server._shared_upstreams == {} # entry evicted, ref returned to 0
- assert connection.close_calls == 1 # socket closed
- leaked = [t for t in set(asyncio.all_tasks()) - before if not t.done()]
- assert leaked == []
-
-
-async def test_cancelling_one_co_waiter_does_not_abort_the_shared_dial() -> None:
- # FIX 3: two first-clients wait on one single-flight dial; cancelling one must not
- # (via unshielded await) cancel the shared dial out from under the other.
- connection = _ControllableUpstream()
- browser = _GatedBrowser(connection)
- server = _server(browser)
- client_a = _ScriptClient([_command(1, "Page.enable")])
- client_b = _ScriptClient([_command(1, "Network.enable")])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: browser.connect_calls == 1) # one dial, both waiting on it
-
- task_a.cancel()
- with pytest.raises(asyncio.CancelledError):
- await task_a
- browser.gate.set() # the dial the cancelled waiter shared must still complete
-
- assert await _until(lambda: client_b.echoes()) # B acquired a live connection
- assert client_b.echoes() == {"Network.enable"}
- assert browser.connect_calls == 1 # the dial was neither aborted nor retried
-
- client_b.release()
- await asyncio.wait_for(task_b, timeout=5)
-
-
-def _auto_attach(client_id: int, session_id: str | None = None) -> str:
- return encode_frame(
- CdpCommand(
- id=client_id,
- method="Target.setAutoAttach",
- params={"autoAttach": True, "flatten": True, "waitForDebuggerOnStart": False},
- session_id=session_id,
- )
- )
-
-
-def _attached_to_target(cdp_session_id: str, parent: str | None = None, target_id: str | None = None) -> CdpEvent:
- return CdpEvent(
- method="Target.attachedToTarget",
- params={"sessionId": cdp_session_id, "targetInfo": {"targetId": target_id or f"target-{cdp_session_id}"}},
- session_id=parent,
- )
-
-
-async def test_explicit_attach_notice_reaches_the_attaching_client() -> None:
- # Chrome announces an explicit Target.attachToTarget with a browser-level
- # attachedToTarget EVENT that lands before the response — so the owner is not yet
- # known from the reply. Clients build their session object from that event (it is
- # how a CDP client learns the session exists), so it must reach the attacher and
- # nobody else. Routing it by the in-flight attach is what makes that possible.
- connection = _AttachEventUpstream()
- server = _server(_SharedBrowser(connection))
- client_a = _ScriptClient([_command(1, "Target.attachToTarget", {"targetId": "tA", "flatten": True})])
- client_b = _ScriptClient([_command(1, "Page.enable")])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: client_a.events("Target.attachedToTarget") and client_b.echoes())
-
- assert client_b.events("Target.attachedToTarget") == []
- # And the co-tenant still never sees that session's traffic.
- connection.emit(CdpEvent(method="Page.loadEventFired", session_id="tA"))
- assert await _until(lambda: "tA" in client_a.event_sessions())
- assert client_b.event_sessions() == set()
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
-
-async def test_autoattach_event_learns_the_owner_and_routes_only_to_it() -> None:
- # SKY-12500 (AC3): sessions arrive via autoAttach, not just explicit attach. The
- # client that enabled browser-level autoAttach owns what auto-attaches, so its
- # sessions' events must not reach the co-tenant on the shared connection.
- connection = _ControllableUpstream()
- server = _server(_SharedBrowser(connection))
- client_a = _ScriptClient([_auto_attach(1)])
- client_b = _ScriptClient([_command(1, "Page.enable")])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: client_a.sent and client_b.sent)
-
- connection.emit(_attached_to_target("S1"))
- connection.emit(CdpEvent(method="Page.loadEventFired", session_id="S1"))
- assert await _until(lambda: "S1" in client_a.event_sessions())
-
- # A owns S1: it alone sees the attach notice and the session's events.
- assert client_a.events("Target.attachedToTarget")
- assert client_b.events("Target.attachedToTarget") == []
- assert client_b.event_sessions() == set()
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
-
-async def test_concurrent_attaches_to_one_target_match_their_own_announcements() -> None:
- # Two clients may attach to the SAME target (an agent and a human debugger on one
- # page): Chrome opens a separate session per attach and announces them in the order
- # asked. Each announcement must go to the client whose attach it answers.
- connection = _SilentUpstream()
- server = _server(_SharedBrowser(connection))
- attach = _command(1, "Target.attachToTarget", {"targetId": "shared-target", "flatten": True})
- client_a = _ScriptClient([attach])
- client_b = _ScriptClient([])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- assert await _until(lambda: len(connection.sent) == 1) # A's attach is in flight first
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: _channel_count(server) == 2)
- client_b.feed(attach)
- assert await _until(lambda: len(connection.sent) == 2)
-
- connection.emit(_attached_to_target("S1", target_id="shared-target"))
- connection.emit(_attached_to_target("S2", target_id="shared-target"))
- assert await _until(
- lambda: client_a.events("Target.attachedToTarget") and client_b.events("Target.attachedToTarget")
- )
-
- # A asked first, so S1 is A's and S2 is B's — neither sees the other's session.
- assert [event.params["sessionId"] for event in client_a.events("Target.attachedToTarget")] == ["S1"]
- assert [event.params["sessionId"] for event in client_b.events("Target.attachedToTarget")] == ["S2"]
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
-
-async def _two_clients(server: CdpProxyServer, first: _ScriptClient, second: _ScriptClient, sent: list[str]) -> tuple:
- """Attach `first`, wait until its opening command is on the wire, then attach
- `second` — so a test can pin the order the two clients' frames reach the browser."""
- task_first = asyncio.create_task(server._handle_client(first)) # type: ignore[arg-type]
- assert await _until(lambda: len(sent) == 1)
- task_second = asyncio.create_task(server._handle_client(second)) # type: ignore[arg-type]
- assert await _until(lambda: _channel_count(server) == 2)
- return task_first, task_second
-
-
-async def test_autoattach_announcement_is_not_stolen_by_a_concurrent_explicit_attach() -> None:
- # A runs browser-level autoAttach; B explicitly attaches the SAME target while A's
- # auto-attach announcement is still queued. Nothing in an announcement says which
- # attach it answers — but crediting it to B is the unrecoverable way to be wrong:
- # B's explicit attach has a response that fixes B's own session, while A's
- # auto-attach has none, so a wrong guess hands A's session to B forever.
- connection = _SilentUpstream()
- server = _server(_SharedBrowser(connection))
- client_a = _ScriptClient([_auto_attach(1)])
- client_b = _ScriptClient([])
- task_a, task_b = await _two_clients(server, client_a, client_b, connection.sent)
- client_b.feed(_command(1, "Target.attachToTarget", {"targetId": "T", "flatten": True}))
- assert await _until(lambda: len(connection.sent) == 2)
- shared = server._shared_upstreams[UPSTREAM_URL]
- owner_a = next(iter(shared.channels)) # A attached first
-
- # Chrome auto-attaches T for A; the announcement lands while B's attach is in flight.
- connection.emit(_attached_to_target("S_A", target_id="T"))
- assert await _until(lambda: "S_A" in shared.session_owner)
-
- assert shared.session_owner["S_A"] == owner_a
- connection.emit(CdpEvent(method="Page.loadEventFired", session_id="S_A"))
- assert await _until(lambda: "S_A" in client_a.event_sessions())
- assert client_b.event_sessions() == set() # B never sees A's auto-attached session
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
-
-async def test_failed_attach_leaves_no_intent_for_a_later_attach_to_consume() -> None:
- # A's attach ERRORS, so it is never announced. Its intent must not linger: the next
- # client to attach that same target would otherwise be paired with it and have its
- # session handed to A.
- connection = _SilentUpstream()
- server = _server(_SharedBrowser(connection))
- client_a = _ScriptClient([_command(1, "Target.attachToTarget", {"targetId": "T", "flatten": True})])
- client_b = _ScriptClient([])
- task_a, task_b = await _two_clients(server, client_a, client_b, connection.sent)
-
- failed_id = decode_frame(connection.sent[0]).id
- connection.emit(CdpResponse(id=failed_id, error={"code": -32602, "message": "No target with given id found"}))
- assert await _until(lambda: client_a.errors())
-
- client_b.feed(_command(1, "Target.attachToTarget", {"targetId": "T", "flatten": True}))
- assert await _until(lambda: len(connection.sent) == 2)
- connection.emit(_attached_to_target("S_B", target_id="T"))
- assert await _until(lambda: client_b.events("Target.attachedToTarget"))
-
- shared = server._shared_upstreams[UPSTREAM_URL]
- assert shared.session_owner["S_B"] == list(shared.channels)[1] # B's own key
- assert client_a.events("Target.attachedToTarget") == [] # A's dead attach claims nothing
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
-
-async def test_failed_autoattach_enable_does_not_leave_the_client_credited() -> None:
- # A's setAutoAttach errors, so autoAttach is NOT on for it; the optimistic record
- # must be undone or A would be credited with sessions it never asked to own.
- connection = _SilentUpstream()
- server = _server(_SharedBrowser(connection))
- client_a = _ScriptClient([_auto_attach(1)])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- assert await _until(lambda: len(connection.sent) == 1)
- shared = server._shared_upstreams[UPSTREAM_URL]
- assert shared.autoattach_clients # recorded optimistically at send
-
- failed_id = decode_frame(connection.sent[0]).id
- connection.emit(CdpResponse(id=failed_id, error={"code": -32000, "message": "Not allowed"}))
- assert await _until(lambda: client_a.errors())
-
- assert shared.autoattach_clients == []
- assert shared.attach_intents == []
-
- client_a.release()
- await asyncio.wait_for(task_a, timeout=5)
-
-
-def _disable_auto_attach(client_id: int) -> str:
- return encode_frame(CdpCommand(id=client_id, method="Target.setAutoAttach", params={"autoAttach": False}))
-
-
-async def test_failed_autoattach_disable_restores_the_client_credit() -> None:
- # A enables autoAttach, then asks to disable it, but the browser REJECTS the disable —
- # so autoAttach is still live upstream. The optimistic removal must be undone or A's
- # later auto-attach announcements would be dropped/misrouted while the proxy believes
- # A opted out.
- connection = _SilentUpstream()
- server = _server(_SharedBrowser(connection))
- client_a = _ScriptClient([_auto_attach(1)])
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- assert await _until(lambda: len(connection.sent) == 1)
- shared = server._shared_upstreams[UPSTREAM_URL]
- owner_a = next(iter(shared.channels))
- assert shared.autoattach_clients == [owner_a] # enabled optimistically at send
-
- client_a.feed(_disable_auto_attach(2))
- assert await _until(lambda: len(connection.sent) == 2)
- assert shared.autoattach_clients == [] # removed optimistically at send
-
- disable_id = decode_frame(connection.sent[1]).id
- connection.emit(CdpResponse(id=disable_id, error={"code": -32000, "message": "Not allowed"}))
- assert await _until(lambda: client_a.errors())
-
- assert shared.autoattach_clients == [owner_a] # restored: the disable never took effect
-
- client_a.release()
- await asyncio.wait_for(task_a, timeout=5)
-
-
-async def _drop_autoattach_error(frame: CdpFrame, direction: Direction, session: ProxySession) -> CdpFrame | None:
- if direction is Direction.UPSTREAM_TO_CLIENT and isinstance(frame, CdpResponse) and frame.error is not None:
- return None
- return frame
-
-
-async def test_dropped_autoattach_enable_error_does_not_leave_the_client_credited() -> None:
- # An upstream middleware SUPPRESSES the rejected setAutoAttach response. The dropped
- # frame still carries the error, so the intent must settle as a failure — otherwise A
- # stays credited and the next browser-level attach is handed to it.
- connection = _SilentUpstream()
- server = _server(_SharedBrowser(connection), pipeline=MiddlewarePipeline([_drop_autoattach_error]))
- client_a = _ScriptClient([_auto_attach(1)])
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- assert await _until(lambda: len(connection.sent) == 1)
- shared = server._shared_upstreams[UPSTREAM_URL]
- assert shared.autoattach_clients # recorded optimistically at send
-
- enable_id = decode_frame(connection.sent[0]).id
- connection.emit(CdpResponse(id=enable_id, error={"code": -32000, "message": "Not allowed"}))
- # The error is suppressed, so A never receives it; wait for the intent to settle.
- assert await _until(lambda: shared.attach_intents == [])
- assert client_a.errors() == [] # the response was dropped, not delivered
-
- assert shared.autoattach_clients == [] # settled as a failure despite the suppressed response
-
- client_a.release()
- await asyncio.wait_for(task_a, timeout=5)
-
-
-async def test_ambiguous_attach_announcement_is_withheld_from_the_autoattach_client() -> None:
- # A runs browser-level autoAttach; B explicitly attaches the SAME target. The
- # announcement that arrives is indistinguishable — it could be A's auto-attach or B's
- # explicit attach — so it is withheld from BOTH (fail-closed) rather than delivered to
- # A across the tenant boundary. Durable ownership still goes to A (its auto-attach has
- # no response to self-correct); delivering it correctly needs the interception seam.
- connection = _SilentUpstream()
- server = _server(_SharedBrowser(connection))
- client_a = _ScriptClient([_auto_attach(1)])
- client_b = _ScriptClient([])
- task_a, task_b = await _two_clients(server, client_a, client_b, connection.sent)
- client_b.feed(_command(1, "Target.attachToTarget", {"targetId": "T", "flatten": True}))
- assert await _until(lambda: len(connection.sent) == 2)
- shared = server._shared_upstreams[UPSTREAM_URL]
- owner_a = next(iter(shared.channels)) # A attached first
-
- connection.emit(_attached_to_target("S_A", target_id="T"))
- # A owns S_A durably, so a following session-scoped event IS delivered to A; the queue
- # is FIFO, so once A sees it the announcement would already be visible had it been sent.
- connection.emit(CdpEvent(method="Page.loadEventFired", session_id="S_A"))
- assert await _until(lambda: "S_A" in client_a.event_sessions())
-
- assert shared.session_owner["S_A"] == owner_a # durable ownership still A's
- assert client_a.events("Target.attachedToTarget") == [] # announcement withheld — could be B's attach
- assert client_b.events("Target.attachedToTarget") == [] # and never crosses to the autoAttach client
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
-
-def _bare_shared() -> _SharedUpstream:
- return _SharedUpstream(session=ProxySession(session_id="s1", upstream_ws_url=UPSTREAM_URL))
-
-
-def test_attach_intent_eviction_drops_the_callers_own_oldest_not_a_co_tenants() -> None:
- # At the cap, a flooding client reclaims a slot at its OWN expense (its oldest intent),
- # never a co-tenant's unreconciled announcement — mirrors the remapper's owner-scoped
- # _evict_one_for. The GLOBAL oldest here belongs to a co-tenant, so a naive pop(0)
- # would drop the victim's intent.
- shared = _bare_shared()
- shared.attach_intents.append(_AttachIntent(0, "victim", "v-oldest", False, False)) # global oldest, a co-tenant's
- shared.attach_intents.append(_AttachIntent(1, "flooder", "own-old", False, False)) # flooder's oldest, not index 0
- for i in range(2, _MAX_PENDING_COMMANDS):
- shared.attach_intents.append(_AttachIntent(i, "victim", f"v{i}", False, False))
- assert len(shared.attach_intents) == _MAX_PENDING_COMMANDS
-
- _record_attach_intent(shared, _AttachIntent(9999, "flooder", "own-new", False, False))
-
- targets = [intent.target_id for intent in shared.attach_intents]
- assert [intent.client_key for intent in shared.attach_intents].count("victim") == _MAX_PENDING_COMMANDS - 1
- assert "v-oldest" in targets # the co-tenant's oldest was NOT evicted
- assert "own-old" not in targets # the flooder's OWN oldest was evicted instead
- assert "own-new" in targets # its new intent was recorded
-
-
-async def test_client_cannot_detach_a_co_tenants_session_addressed_via_params() -> None:
- # Target.detachFromTarget names its session in params, not the frame's sessionId —
- # so the top-level ownership gate alone would let B detach A's session.
- connection = _ControllableUpstream()
- server = _server(_SharedBrowser(connection))
- client_a = _ScriptClient([_command(1, "Target.attachToTarget", {"targetId": "tA", "flatten": True})])
- client_b = _ScriptClient([_command(1, "Page.enable")])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: client_a.sent and client_b.echoes())
- shared = server._shared_upstreams[UPSTREAM_URL]
- assert await _until(lambda: "tA" in shared.session_owner)
-
- client_b.feed(encode_frame(CdpCommand(id=2, method="Target.detachFromTarget", params={"sessionId": "tA"})))
- assert await _until(lambda: client_b.errors())
-
- assert client_b.errors()[0].id == 2
- upstream_methods = [f.method for f in map(decode_frame, connection.sent) if isinstance(f, CdpCommand)]
- assert "Target.detachFromTarget" not in upstream_methods # never reached the browser
- assert "tA" in shared.session_owner # and A still owns its session
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
-
-async def test_nested_autoattach_inherits_the_parent_session_owner() -> None:
- # An auto-attach nested under an owned session (an iframe/worker of A's page)
- # belongs to that session's owner.
- connection = _ControllableUpstream()
- server = _server(_SharedBrowser(connection))
- client_a = _ScriptClient([_command(1, "Target.attachToTarget", {"targetId": "tA", "flatten": True})])
- client_b = _ScriptClient([_command(1, "Page.enable")])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: client_a.sent and client_b.sent)
-
- connection.emit(_attached_to_target("child", parent="tA"))
- connection.emit(CdpEvent(method="Runtime.consoleAPICalled", session_id="child"))
- assert await _until(lambda: "child" in client_a.event_sessions())
-
- assert client_b.event_sessions() == set()
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
-
-async def test_session_scoped_event_for_an_unowned_session_is_never_broadcast() -> None:
- # SKY-12500 (AC1/AC3): before 12500 an event whose sessionId had no known owner
- # fanned out to EVERY client. A session-scoped event must never reach a non-owner;
- # with no owner to route to it is dropped, not broadcast.
- connection = _ControllableUpstream()
- server = _server(_SharedBrowser(connection))
- client_a = _ScriptClient([_command(1, "Page.enable")])
- client_b = _ScriptClient([_command(1, "Network.enable")])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: client_a.echoes() and client_b.echoes())
-
- connection.emit(CdpEvent(method="Page.loadEventFired", session_id="nobody"))
- # Let the reader route it, then prove it landed nowhere.
- connection.emit(CdpEvent(method="Target.targetCreated", params={"targetInfo": {}}))
- assert await _until(lambda: client_a.events("Target.targetCreated") and client_b.events("Target.targetCreated"))
-
- assert client_a.event_sessions() == {None} # the browser-level event only
- assert client_b.event_sessions() == {None}
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
-
-async def test_detach_event_releases_the_session_owner() -> None:
- # SKY-12500: ownership is GC'd on detach, so a long-lived client's attach/detach
- # churn cannot grow session_owner without bound.
- connection = _ControllableUpstream()
- server = _server(_SharedBrowser(connection))
- client_a = _ScriptClient([_auto_attach(1)])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- assert await _until(lambda: client_a.sent)
- shared = server._shared_upstreams[UPSTREAM_URL]
-
- connection.emit(_attached_to_target("S1"))
- assert await _until(lambda: "S1" in shared.session_owner)
-
- connection.emit(CdpEvent(method="Target.detachedFromTarget", params={"sessionId": "S1"}))
- # The owner still learns its session went away, and the entry is gone with it.
- assert await _until(lambda: client_a.events("Target.detachedFromTarget"))
- assert "S1" not in shared.session_owner
-
- client_a.release()
- await asyncio.wait_for(task_a, timeout=5)
-
-
-async def test_client_cannot_drive_another_clients_session() -> None:
- # SKY-12500 (AC2/AC3): the uplink is ownership-checked — B must not steer A's
- # session, and must fail deterministically rather than hang.
- connection = _ControllableUpstream()
- server = _server(_SharedBrowser(connection))
- client_a = _ScriptClient([_command(1, "Target.attachToTarget", {"targetId": "tA", "flatten": True})])
- client_b = _ScriptClient([_command(1, "Page.enable")])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: client_a.sent and client_b.echoes())
- shared = server._shared_upstreams[UPSTREAM_URL]
- assert await _until(lambda: "tA" in shared.session_owner)
-
- client_b.feed(encode_frame(CdpCommand(id=2, method="Page.navigate", session_id="tA")))
- assert await _until(lambda: client_b.errors())
-
- # B is told no; the hijack never reached the browser.
- assert client_b.errors()[0].id == 2
- upstream_sessions = [decode_frame(w).session_id for w in connection.sent]
- assert "tA" not in upstream_sessions
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
-
-async def test_remap_tables_are_empty_after_quiescence() -> None:
- # SKY-12500 (AC5): commands still in flight when a client leaves must not pin
- # remapper entries, and its learned session owners go with it.
- connection = _SilentUpstream()
- server = _server(_SharedBrowser(connection))
- client_a = _ScriptClient(
- [
- _command(1, "Page.enable"),
- _command(2, "Runtime.enable"),
- _command(3, "Target.attachToTarget", {"targetId": "never-answered"}),
- _auto_attach(4),
- ]
- )
- client_b = _ScriptClient([_command(1, "Network.enable")])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: len(connection.sent) == 5) # all in flight, none answered
- shared = server._shared_upstreams[UPSTREAM_URL]
- assert shared.remapper.pending_count == 5
- assert shared.attach_intents and shared.autoattach_clients
-
- client_a.release()
- client_b.release()
- await asyncio.wait_for(asyncio.gather(task_a, task_b), timeout=5)
-
- # Every table keyed by a client is empty once they are all gone.
- assert shared.remapper.pending_count == 0
- assert shared.session_owner == {}
- assert shared.attach_intents == []
- assert shared.autoattach_clients == []
- assert shared.channels == {}
- assert server._shared_upstreams == {}
-
-
-async def test_pending_mappings_stay_bounded_under_a_silent_upstream() -> None:
- # AC5's steady-state half: one long-lived client whose commands are never
- # answered must not grow the remapper without bound.
- connection = _SilentUpstream()
- server = _server(_SharedBrowser(connection))
- server._max_pending_requests = 8
- client = _ScriptClient([_command(client_id, "Page.enable") for client_id in range(1, 40)])
-
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- assert await _until(lambda: len(connection.sent) == 39)
- shared = server._shared_upstreams[UPSTREAM_URL]
-
- assert shared.remapper.pending_count == 8
-
- client.release()
- await asyncio.wait_for(task, timeout=5)
-
-
-async def test_many_clients_and_sessions_show_zero_cross_talk() -> None:
- # SKY-12500 (AC1): the concurrency oracle. Every client uses the SAME client-side
- # ids; only the shared remapper and the ownership map keep them apart.
- connection = _ControllableUpstream()
- server = _server(_SharedBrowser(connection))
- client_count, per_client = 8, 10
- clients = [
- _ScriptClient(
- [_command(1, "Target.attachToTarget", {"targetId": f"t{index}", "flatten": True})]
- + [_command(client_id, f"c{index}.m{client_id}") for client_id in range(2, per_client + 2)]
- )
- for index in range(client_count)
- ]
-
- tasks = [asyncio.create_task(server._handle_client(client)) for client in clients] # type: ignore[arg-type]
- assert await _until(lambda: all(len(client.echoes()) == per_client for client in clients))
-
- for index in range(client_count):
- connection.emit(CdpEvent(method="Page.loadEventFired", session_id=f"t{index}"))
- assert await _until(lambda: all(f"t{index}" in clients[index].event_sessions() for index in range(client_count)))
-
- for index, client in enumerate(clients):
- # Only this client's own responses, under its own ids, and only its session.
- assert client.echoes() == {f"c{index}.m{client_id}" for client_id in range(2, per_client + 2)}
- assert client.event_sessions() == {f"t{index}"}
- assert {decode_frame(wire).id for wire in client.sent if isinstance(decode_frame(wire), CdpResponse)} == set(
- range(1, per_client + 2)
- )
-
- for client in clients:
- client.release()
- await asyncio.wait_for(asyncio.gather(*tasks), timeout=5)
-
-
-async def test_departing_client_clears_its_session_owner_entries() -> None:
- # FIX 4: a client's learned sessionId -> client_key entries are dropped on its
- # teardown (the shared connection stays alive for the other client).
- connection = _ControllableUpstream()
- server = _server(_SharedBrowser(connection))
- client_a = _ScriptClient([_command(1, "Target.attachToTarget", {"targetId": "tA", "flatten": True})])
- client_b = _ScriptClient([_command(1, "Page.enable")])
-
- task_a = asyncio.create_task(server._handle_client(client_a)) # type: ignore[arg-type]
- task_b = asyncio.create_task(server._handle_client(client_b)) # type: ignore[arg-type]
- assert await _until(lambda: client_a.sent and client_b.sent)
- shared = server._shared_upstreams[UPSTREAM_URL]
- assert "tA" in shared.session_owner
-
- client_a.release()
- await asyncio.wait_for(task_a, timeout=5)
- assert "tA" not in shared.session_owner # cleared when A left; B keeps the connection alive
-
- client_b.release()
- await asyncio.wait_for(task_b, timeout=5)
diff --git a/tests/unit/proxy/test_slow_client.py b/tests/unit/proxy/test_slow_client.py
deleted file mode 100644
index 490ea4629..000000000
--- a/tests/unit/proxy/test_slow_client.py
+++ /dev/null
@@ -1,446 +0,0 @@
-"""Slow-client backpressure tests for the client-facing CDP proxy.
-
-The client-bound relay uses a bounded outbound queue drained by a dedicated
-writer, so a slow client never backs up the upstream browser (the runner-CPU
-goal). Droppable events are tail-dropped under overflow; a client that stays
-overflowed — or cannot even take a command response — is closed with WS 1013.
-"""
-
-from __future__ import annotations
-
-import asyncio
-from types import SimpleNamespace
-from typing import Mapping
-
-import pytest
-
-from skyvern.proxy.adapters.memory import AllowAllAuth, ForwardAllEventPolicy, InMemorySessionRegistry
-from skyvern.proxy.adapters.websocket_server import (
- _BACKPRESSURE_CLOSE_CODE,
- CdpProxyServer,
- _ClientChannel,
- _OutboundQueue,
- _SharedUpstream,
-)
-from skyvern.proxy.core.frames import CdpCommand, CdpEvent, CdpResponse, RequestIdRemapper, decode_frame, encode_frame
-from skyvern.proxy.core.session import Principal, ProxySession, ResolvedSession, UpstreamClosedError
-
-FRAMES_DROPPED = "skyvern.cdp_proxy.frames_dropped"
-BACKPRESSURE_CLOSED = "skyvern.cdp_proxy.client_backpressure_closed"
-
-
-class _RecordingMetrics:
- def __init__(self) -> None:
- self.calls: list[tuple[str, str, dict[str, str]]] = []
-
- def increment(self, name: str, amount: int = 1, tags: Mapping[str, str] | None = None) -> None:
- self.calls.append(("increment", name, dict(tags or {})))
-
- def observe(self, name: str, value: float, tags: Mapping[str, str] | None = None) -> None:
- self.calls.append(("observe", name, dict(tags or {})))
-
- def gauge(self, name: str, amount: int, tags: Mapping[str, str] | None = None) -> None:
- self.calls.append(("gauge", name, dict(tags or {})))
-
- def count(self, name: str, reason: str | None = None) -> int:
- return sum(1 for _op, n, tags in self.calls if n == name and (reason is None or tags.get("reason") == reason))
-
-
-class _BlockingClient:
- """A client whose first send() blocks until the connection is closed, so the
- writer stalls and the outbound queue fills — a stuck slow client. Any `incoming`
- frames are yielded first, for tests that need it to speak before it stalls."""
-
- def __init__(self, incoming: list[str] | None = None) -> None:
- self.request = SimpleNamespace(headers={}, path="/s1")
- self.sent: list[str] = []
- self.close_code: int | None = None
- self.close_reason: str | None = None
- self._incoming = list(incoming or [])
- self._closed = asyncio.Event()
-
- def __aiter__(self) -> _BlockingClient:
- return self
-
- async def __anext__(self) -> str:
- if self._incoming:
- return self._incoming.pop(0)
- await self._closed.wait()
- raise StopAsyncIteration
-
- async def send(self, raw: str) -> None:
- self.sent.append(raw)
- await self._closed.wait()
-
- async def close(self, code: int = 1000, reason: str = "") -> None:
- self.close_code = code
- self.close_reason = reason
- self._closed.set()
-
-
-class _FloodingUpstream:
- def __init__(self, count: int) -> None:
- self._remaining = count
-
- async def send(self, raw: str) -> None:
- return None
-
- async def receive(self) -> str:
- if self._remaining <= 0:
- raise UpstreamClosedError("drained")
- self._remaining -= 1
- return encode_frame(CdpEvent(method="Network.dataReceived", params={"n": self._remaining}))
-
- async def close(self) -> None:
- return None
-
-
-class _ScriptedUpstream:
- def __init__(self, frames: list[str]) -> None:
- self._frames = list(frames)
-
- async def send(self, raw: str) -> None:
- return None
-
- async def receive(self) -> str:
- if not self._frames:
- raise UpstreamClosedError("drained")
- return self._frames.pop(0)
-
- async def close(self) -> None:
- return None
-
-
-class _StaticUpstreamBrowser:
- def __init__(self, connection: object) -> None:
- self._connection = connection
-
- async def connect(self, session: ProxySession) -> object:
- return self._connection
-
-
-class _NoCloseWs:
- """Never sends; used to assert the relay itself initiates a clean close."""
-
- def __init__(self) -> None:
- self.close_code: int | None = None
- self.close_reason: str | None = None
-
- async def send(self, raw: str) -> None:
- return None
-
- async def close(self, code: int = 1000, reason: str = "") -> None:
- self.close_code = code
- self.close_reason = reason
-
-
-def _session() -> ProxySession:
- return ProxySession(
- session_id="s1", upstream_ws_url="ws://x/y", principal=Principal(principal_id="p", organization_id="o1")
- )
-
-
-def _server(
- metrics: object, connection: object, maxsize: int, close_after: int, maxbytes: int = 1_000_000
-) -> CdpProxyServer:
- sessions = InMemorySessionRegistry()
- sessions.put(ResolvedSession(session_id="s1", upstream_adapter="memory", upstream_ws_url="ws://x/y"))
- return CdpProxyServer(
- upstream=_StaticUpstreamBrowser(connection), # type: ignore[arg-type]
- sessions=sessions,
- auth=AllowAllAuth(),
- metrics=metrics, # type: ignore[arg-type]
- event_policy=ForwardAllEventPolicy(),
- client_queue_maxsize=maxsize,
- client_backpressure_close_after=close_after,
- client_queue_maxbytes=maxbytes,
- )
-
-
-class _SlowSendClient:
- """Sends with a small delay, so a queued upstream response is still in flight
- when the upstream completes — exercising the graceful drain."""
-
- def __init__(self, incoming: list[str]) -> None:
- self.request = SimpleNamespace(headers={}, path="/s1")
- self.sent: list[str] = []
- self.close_code: int | None = None
- self._incoming = list(incoming)
- self._closed = asyncio.Event()
-
- def __aiter__(self) -> _SlowSendClient:
- return self
-
- async def __anext__(self) -> str:
- if self._incoming:
- return self._incoming.pop(0)
- await self._closed.wait()
- raise StopAsyncIteration
-
- async def send(self, raw: str) -> None:
- await asyncio.sleep(0.02)
- self.sent.append(raw)
-
- async def close(self, code: int = 1000, reason: str = "") -> None:
- self.close_code = code
- self._closed.set()
-
-
-class _EchoThenCloseUpstream:
- def __init__(self) -> None:
- self._queue: asyncio.Queue[str | None] = asyncio.Queue()
- self._closed = False
-
- async def send(self, raw: str) -> None:
- frame = decode_frame(raw)
- self._queue.put_nowait(encode_frame(CdpResponse(id=frame.id, result={"ok": True})))
- self._queue.put_nowait(None)
-
- async def receive(self) -> str:
- if self._closed and self._queue.empty():
- raise UpstreamClosedError("closed")
- raw = await self._queue.get()
- if raw is None:
- raise UpstreamClosedError("closed")
- return raw
-
- async def close(self) -> None:
- self._closed = True
- self._queue.put_nowait(None)
-
-
-class _BlockingUpstream:
- def __init__(self) -> None:
- self._closed = asyncio.Event()
-
- async def send(self, raw: str) -> None:
- return None
-
- async def receive(self) -> str:
- await self._closed.wait()
- raise UpstreamClosedError("closed")
-
- async def close(self) -> None:
- self._closed.set()
-
-
-class _FloodingLargeUpstream:
- def __init__(self, count: int, frame_bytes: int) -> None:
- self._remaining = count
- self._payload = "x" * frame_bytes
-
- async def send(self, raw: str) -> None:
- return None
-
- async def receive(self) -> str:
- if self._remaining <= 0:
- raise UpstreamClosedError("drained")
- self._remaining -= 1
- return encode_frame(CdpEvent(method="Network.dataReceived", params={"body": self._payload}))
-
- async def close(self) -> None:
- return None
-
-
-async def test_final_upstream_response_is_delivered_before_teardown() -> None:
- # On clean upstream completion the queued command response must reach the client
- # before teardown, not be silently lost when the writer is cancelled.
- client = _SlowSendClient([encode_frame(CdpCommand(id=1, method="Page.enable"))])
- server = _server(_RecordingMetrics(), _EchoThenCloseUpstream(), maxsize=64, close_after=64)
-
- await asyncio.wait_for(server._handle_client(client), timeout=5) # type: ignore[arg-type]
-
- assert any('"id":1' in frame for frame in client.sent) # the response was delivered
- assert client.close_code != _BACKPRESSURE_CLOSE_CODE # drained in time, not force-closed
-
-
-async def test_cancelling_handle_client_reaps_all_children_cleanly() -> None:
- metrics = _RecordingMetrics()
- client = _BlockingClient()
- server = _server(metrics, _BlockingUpstream(), maxsize=8, close_after=8)
-
- before = set(asyncio.all_tasks())
- task = asyncio.create_task(server._handle_client(client)) # type: ignore[arg-type]
- for _ in range(500): # wait until the session is live (the +1 gauge fired)
- if any(name.endswith("active_sessions") for _op, name, _tags in metrics.calls):
- break
- await asyncio.sleep(0)
- task.cancel()
- with pytest.raises(asyncio.CancelledError):
- await task
-
- leaked = [t for t in set(asyncio.all_tasks()) - before if not t.done()]
- assert leaked == [] # no child relay/writer task leaked
- gauges = [1 for _op, name, _tags in metrics.calls if name.endswith("active_sessions")]
- assert len(gauges) == 2 # +1 then -1: teardown ran despite cancellation
-
-
-async def test_byte_budget_trips_backpressure_under_the_frame_count_limit() -> None:
- # The count bound is huge but the byte bound is small and each frame is large,
- # so memory pressure (not frame count) trips tail-drop and the 1013 close.
- metrics = _RecordingMetrics()
- client = _BlockingClient()
- server = _server(
- metrics, _FloodingLargeUpstream(count=500, frame_bytes=10_000), maxsize=100_000, close_after=3, maxbytes=50_000
- )
-
- await asyncio.wait_for(server._handle_client(client), timeout=5) # type: ignore[arg-type]
-
- assert client.close_code == _BACKPRESSURE_CLOSE_CODE
- assert metrics.count(FRAMES_DROPPED, reason="backpressure") >= 1
-
-
-async def test_sustained_event_overflow_drops_then_closes_the_slow_client() -> None:
- metrics = _RecordingMetrics()
- client = _BlockingClient()
- server = _server(metrics, _FloodingUpstream(count=200), maxsize=2, close_after=3)
-
- await asyncio.wait_for(server._handle_client(client), timeout=5) # type: ignore[arg-type]
-
- assert client.close_code == _BACKPRESSURE_CLOSE_CODE
- assert metrics.count(FRAMES_DROPPED, reason="backpressure") >= 1
- assert metrics.count(BACKPRESSURE_CLOSED) == 1
-
-
-async def test_non_positive_constructor_bound_is_clamped_and_backpressure_still_fires() -> None:
- # A <=0 maxsize would make asyncio.Queue unbounded and silently disable
- # backpressure; the constructor must clamp it like the env path does.
- metrics = _RecordingMetrics()
- client = _BlockingClient()
- server = _server(metrics, _FloodingUpstream(count=200), maxsize=0, close_after=-5)
- assert server._client_queue_maxsize == 1
- assert server._client_backpressure_close_after == 1
-
- await asyncio.wait_for(server._handle_client(client), timeout=5) # type: ignore[arg-type]
-
- assert client.close_code == _BACKPRESSURE_CLOSE_CODE # queue stayed bounded
- assert metrics.count(BACKPRESSURE_CLOSED) == 1
-
-
-async def test_backpressure_never_blocks_the_upstream_read_loop() -> None:
- # The upstream floods faster than the (stalled) client drains; the relay must
- # still finish (by dropping + closing) rather than deadlock on the client.
- metrics = _RecordingMetrics()
- client = _BlockingClient()
- server = _server(metrics, _FloodingUpstream(count=10_000), maxsize=4, close_after=8)
-
- await asyncio.wait_for(server._handle_client(client), timeout=5) # type: ignore[arg-type]
-
- assert client.close_code == _BACKPRESSURE_CLOSE_CODE
-
-
-async def test_a_command_response_is_never_dropped_slow_client_is_closed_instead() -> None:
- # A response is non-droppable: losing it desyncs the client, so a client too
- # slow to take even one is closed immediately (no drop, no wait for close_after).
- metrics = _RecordingMetrics()
- remapper = RequestIdRemapper()
- upstream_cmd = remapper.to_upstream("client", CdpCommand(id=1, method="Page.enable"))
- upstream = _ScriptedUpstream([encode_frame(CdpResponse(id=upstream_cmd.id, result={}))])
- outbound = _OutboundQueue(max_frames=1, max_bytes=1_000_000)
- outbound.put_nowait("prefill-so-the-queue-is-full")
- ws = _NoCloseWs()
- server = _server(metrics, upstream, maxsize=1, close_after=100)
- command_starts: dict[int, tuple[str, float]] = {upstream_cmd.id: ("Page.enable", 0.0)}
-
- await asyncio.wait_for(
- server._relay_upstream_to_client(ws, upstream, _session(), remapper, command_starts, outbound), # type: ignore[arg-type]
- timeout=5,
- )
-
- assert ws.close_code == _BACKPRESSURE_CLOSE_CODE
- assert metrics.count(FRAMES_DROPPED, reason="backpressure") == 0 # the response was not dropped
- assert metrics.count(BACKPRESSURE_CLOSED) == 1
-
-
-async def test_a_session_lifecycle_event_is_never_dropped_slow_client_is_closed_instead() -> None:
- # SKY-12500: attachedToTarget/detachedFromTarget are how a client builds and retires
- # its session objects. Tail-dropping one leaves it silently desynced from the browser
- # — holding a session it can never use, or believing a dead one is live — so like a
- # response they are non-droppable and a client too slow to take one is closed.
- metrics = _RecordingMetrics()
- lifecycle = CdpEvent(method="Target.attachedToTarget", params={"sessionId": "S"})
- upstream = _ScriptedUpstream([encode_frame(lifecycle)])
- outbound = _OutboundQueue(max_frames=1, max_bytes=1_000_000)
- outbound.put_nowait("prefill-so-the-queue-is-full")
- ws = _NoCloseWs()
- server = _server(metrics, upstream, maxsize=1, close_after=100)
-
- await asyncio.wait_for(
- server._relay_upstream_to_client(ws, upstream, _session(), RequestIdRemapper(), {}, outbound), # type: ignore[arg-type]
- timeout=5,
- )
-
- assert ws.close_code == _BACKPRESSURE_CLOSE_CODE
- assert metrics.count(FRAMES_DROPPED, reason="backpressure") == 0 # never silently dropped
- assert metrics.count(BACKPRESSURE_CLOSED) == 1
-
-
-async def test_shared_delivery_holds_the_same_lifecycle_rule() -> None:
- # The same guarantee at the shared multi-client choke point, not only the pinned
- # single-client relay.
- metrics = _RecordingMetrics()
- server = _server(metrics, _ScriptedUpstream([]), maxsize=1, close_after=100)
- shared = _SharedUpstream(session=_session())
- channel = _ClientChannel("c1", _OutboundQueue(max_frames=1, max_bytes=1_000_000))
- channel.outbound.put_nowait("prefill-so-the-queue-is-full")
- shared.channels["c1"] = channel
- lifecycle = CdpEvent(method="Target.detachedFromTarget", params={"sessionId": "S"})
-
- server._deliver_to_channel(shared, channel, lifecycle, encode_frame(lifecycle))
-
- assert channel.closed.is_set()
- assert channel.close_code == _BACKPRESSURE_CLOSE_CODE
- assert metrics.count(FRAMES_DROPPED, reason="backpressure") == 0
- assert metrics.count(BACKPRESSURE_CLOSED) == 1
-
-
-async def test_a_client_that_cannot_reclaim_a_slot_is_closed_not_served_at_a_co_tenants_cost() -> None:
- # SKY-12500: the shared pending table is full of a CO-TENANT's promised responses.
- # Admitting this client's command would mean evicting one of them, and a lost
- # response desyncs that victim invisibly — so the newcomer is closed instead.
- metrics = _RecordingMetrics()
- remapper = RequestIdRemapper(max_pending=1)
- victim = remapper.to_upstream("co-tenant", CdpCommand(id=1, method="Page.enable"))
- upstream = _ScriptedUpstream([])
- client = _BlockingClient([encode_frame(CdpCommand(id=1, method="Runtime.enable"))])
- server = _server(metrics, upstream, maxsize=8, close_after=8)
-
- await asyncio.wait_for(
- server._relay_client_to_upstream(client, upstream, _session(), remapper, {}), # type: ignore[arg-type]
- timeout=5,
- )
-
- assert client.close_code == _BACKPRESSURE_CLOSE_CODE
- assert metrics.count(BACKPRESSURE_CLOSED) == 1
- # The victim's mapping survived intact and still resolves to the victim.
- assert remapper.to_client(CdpResponse(id=victim.id, result={})) == ("co-tenant", CdpResponse(id=1, result={}))
-
-
-async def test_healthy_client_within_the_bound_is_never_dropped_or_closed() -> None:
- metrics = _RecordingMetrics()
- upstream = _ScriptedUpstream([encode_frame(CdpEvent(method="Page.loadEventFired", params={}))])
- outbound = _OutboundQueue(max_frames=64, max_bytes=1_000_000)
- ws = _NoCloseWs()
- server = _server(metrics, upstream, maxsize=64, close_after=8)
-
- with pytest.raises(UpstreamClosedError): # relay ends when the scripted upstream drains
- await asyncio.wait_for(
- server._relay_upstream_to_client(ws, upstream, _session(), RequestIdRemapper(), {}, outbound), # type: ignore[arg-type]
- timeout=5,
- )
-
- assert ws.close_code is None
- assert metrics.count(FRAMES_DROPPED) == 0
- assert metrics.count(BACKPRESSURE_CLOSED) == 0
- assert outbound.get_nowait() # the event was enqueued for delivery
diff --git a/tests/unit/proxy/test_upstream_browser_port_contract.py b/tests/unit/proxy/test_upstream_browser_port_contract.py
deleted file mode 100644
index 24051d596..000000000
--- a/tests/unit/proxy/test_upstream_browser_port_contract.py
+++ /dev/null
@@ -1,200 +0,0 @@
-"""Reusable contract suite for UpstreamBrowserPort adapters.
-
-Any adapter (generic here, cloud-specific under tests/cloud/) subclasses
-UpstreamBrowserPortContract and overrides make_port(); every behavioral
-guarantee of the port is asserted once, here.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import json
-
-import pytest
-import pytest_asyncio
-import websockets
-
-from skyvern.proxy.adapters.local_chrome import (
- LocalChromeUpstreamBrowser,
- _terminate_process,
- find_local_chrome_executable,
-)
-from skyvern.proxy.adapters.memory import InMemoryUpstreamBrowser
-from skyvern.proxy.adapters.websocket_upstream import WebSocketUpstreamBrowser
-from skyvern.proxy.core.errors import UpstreamConnectError
-from skyvern.proxy.core.session import ProxySession, UpstreamClosedError
-from skyvern.proxy.ports import UpstreamBrowserPort, UpstreamConnection
-from tests.unit.proxy._e2e_gate import require
-
-# Nothing listens on port 1, so a dial there is refused immediately rather than left to
-# hang until a timeout — a failure this contract can assert without waiting on one.
-UNREACHABLE_WS_URL = "ws://127.0.0.1:1/devtools/browser/test"
-
-
-class UpstreamBrowserPortContract:
- def make_port(self) -> UpstreamBrowserPort:
- raise NotImplementedError
-
- def make_session(self) -> ProxySession:
- return ProxySession(session_id="test-session", upstream_ws_url="ws://localhost:0/devtools/browser/test")
-
- @pytest.mark.asyncio
- async def test_connect_returns_open_connection(self) -> None:
- connection = await self.make_port().connect(self.make_session())
- assert isinstance(connection, UpstreamConnection)
- await connection.send('{"id": 1, "method": "Browser.getVersion"}')
- await connection.close()
-
- @pytest.mark.asyncio
- async def test_command_receives_frame_with_matching_id(self) -> None:
- connection = await self.make_port().connect(self.make_session())
- try:
- await connection.send('{"id": 42, "method": "Browser.getVersion"}')
-
- async def receive_until_response() -> None:
- while True:
- if json.loads(await connection.receive()).get("id") == 42:
- return
-
- await asyncio.wait_for(receive_until_response(), timeout=10)
- finally:
- await connection.close()
-
- @pytest.mark.asyncio
- async def test_send_after_close_raises_upstream_closed(self) -> None:
- connection = await self.make_port().connect(self.make_session())
- await connection.close()
- with pytest.raises(UpstreamClosedError):
- await connection.send('{"id": 1, "method": "Browser.getVersion"}')
-
- @pytest.mark.asyncio
- async def test_receive_after_close_raises_upstream_closed(self) -> None:
- connection = await self.make_port().connect(self.make_session())
- await connection.close()
- with pytest.raises(UpstreamClosedError):
- await connection.receive()
-
- @pytest.mark.asyncio
- async def test_close_is_idempotent(self) -> None:
- connection = await self.make_port().connect(self.make_session())
- await connection.close()
- await connection.close()
-
-
-class DialingUpstreamBrowserPortContract(UpstreamBrowserPortContract):
- """Extra clauses for an adapter that dials a real remote.
-
- The port's whole point is that callers never branch on transport exception types
- (skyvern.proxy.core.errors), so an adapter leaking a raw ConnectionRefusedError or a
- library's close exception breaks every caller's error handling while satisfying the
- happy-path clauses above. That guarantee only means something where there is a remote
- to fail: the in-memory loopback has none, so it does not inherit these rather than
- "passing" them vacuously — participation is declared by which contract an adapter
- subclasses, never by a skip.
- """
-
- def make_unreachable_port_and_session(self) -> tuple[UpstreamBrowserPort, ProxySession]:
- """A port + session whose connect() cannot succeed."""
- raise NotImplementedError
-
- async def break_remote(self, connection: UpstreamConnection) -> None:
- """Make the remote vanish WITHOUT closing our own end — the interruption a live
- session actually suffers, which is not the same as a local close()."""
- raise NotImplementedError
-
- @pytest.mark.asyncio
- async def test_failed_dial_raises_the_port_error_taxonomy(self, monkeypatch: pytest.MonkeyPatch) -> None:
- # One attempt: the adapter's default budget would spend ~15s of backoff proving
- # a retry behaviour this clause is not about, and a merge gate cannot afford it.
- monkeypatch.setenv("CDP_PROXY_CONNECT_RETRY_ATTEMPTS", "1")
- port, session = self.make_unreachable_port_and_session()
- with pytest.raises(UpstreamConnectError):
- await port.connect(session)
-
- @pytest.mark.asyncio
- async def test_remote_close_surfaces_as_upstream_closed_on_receive(self) -> None:
- connection = await self.make_port().connect(self.make_session())
- try:
- await self.break_remote(connection)
- with pytest.raises(UpstreamClosedError):
- # Drains whatever was already in flight; the close must arrive as the
- # taxonomy error rather than the transport's own exception.
- for _ in range(100):
- await asyncio.wait_for(connection.receive(), timeout=10)
- finally:
- await connection.close()
-
- @pytest.mark.asyncio
- async def test_remote_close_surfaces_as_upstream_closed_on_send(self) -> None:
- connection = await self.make_port().connect(self.make_session())
- try:
- await self.break_remote(connection)
- with pytest.raises(UpstreamClosedError):
- for _ in range(100):
- await asyncio.wait_for(connection.send('{"id": 1, "method": "Browser.getVersion"}'), timeout=10)
- finally:
- await connection.close()
-
-
-class TestInMemoryUpstreamBrowser(UpstreamBrowserPortContract):
- def make_port(self) -> UpstreamBrowserPort:
- return InMemoryUpstreamBrowser()
-
- @pytest.mark.asyncio
- async def test_loopback_echoes_sent_frames(self) -> None:
- connection = await self.make_port().connect(self.make_session())
- await connection.send('{"id": 7, "method": "Target.getTargets"}')
- assert await connection.receive() == '{"id": 7, "method": "Target.getTargets"}'
- await connection.close()
-
-
-class TestWebSocketUpstreamBrowser(DialingUpstreamBrowserPortContract):
- @pytest_asyncio.fixture(autouse=True)
- async def _echo_server(self):
- async def echo(ws: websockets.ServerConnection) -> None:
- async for message in ws:
- await ws.send(message)
-
- async with websockets.serve(echo, "127.0.0.1", 0) as server:
- self._server = server
- self._server_port = server.sockets[0].getsockname()[1]
- yield
-
- def make_port(self) -> UpstreamBrowserPort:
- return WebSocketUpstreamBrowser()
-
- def make_session(self) -> ProxySession:
- return ProxySession(
- session_id="test-session",
- upstream_ws_url=f"ws://127.0.0.1:{self._server_port}/devtools/browser/test",
- )
-
- def make_unreachable_port_and_session(self) -> tuple[UpstreamBrowserPort, ProxySession]:
- return WebSocketUpstreamBrowser(), ProxySession(session_id="test-session", upstream_ws_url=UNREACHABLE_WS_URL)
-
- async def break_remote(self, connection: UpstreamConnection) -> None:
- # Closing the server drops the peer's end of the socket without touching ours.
- self._server.close()
- await self._server.wait_closed()
-
-
-class TestLocalChromeUpstreamBrowser(DialingUpstreamBrowserPortContract):
- @pytest.fixture(autouse=True)
- def _require_chrome(self) -> None:
- # Skips on a machine without Chrome, but FAILS in CI (CDP_PROXY_E2E_REQUIRED=1):
- # this is the only subclass that exercises the port against a real browser, so
- # a silent skip leaves the contract proven against fakes alone.
- require(find_local_chrome_executable() is not None, "no local Chrome/Chromium executable available")
-
- def make_port(self) -> UpstreamBrowserPort:
- return LocalChromeUpstreamBrowser()
-
- def make_unreachable_port_and_session(self) -> tuple[UpstreamBrowserPort, ProxySession]:
- return LocalChromeUpstreamBrowser(executable_path="/nonexistent/no-such-browser"), self.make_session()
-
- async def break_remote(self, connection: UpstreamConnection) -> None:
- # The launched browser IS the remote here, so killing it is the only way to make
- # the peer vanish without closing our own end. Reaching for the process is why
- # this uses the adapter's own reaper: it is idempotent on an already-dead
- # process, so the clause's close() still tears the profile dir down.
- await _terminate_process(connection._process) # type: ignore[attr-defined]
diff --git a/tests/unit/proxy/test_websocket_server.py b/tests/unit/proxy/test_websocket_server.py
deleted file mode 100644
index 5e7cca20a..000000000
--- a/tests/unit/proxy/test_websocket_server.py
+++ /dev/null
@@ -1,503 +0,0 @@
-from __future__ import annotations
-
-import asyncio
-import contextlib
-import os
-import signal
-import socket
-from types import SimpleNamespace
-from typing import Mapping
-
-import pytest
-from websockets import exceptions as websockets_exceptions
-from websockets.asyncio import client as websockets_client
-from websockets.datastructures import Headers
-
-from skyvern.proxy.adapters.memory import (
- AllowAllAuth,
- ForwardAllEventPolicy,
- InMemorySessionRegistry,
- InMemoryUpstreamBrowser,
- NoOpMetrics,
-)
-from skyvern.proxy.adapters.websocket_server import CdpProxyServer, _parse_request
-from skyvern.proxy.core.session import (
- Principal,
- ProxySession,
- ResolvedSession,
- SessionResolution,
- principal_owns_resolution,
-)
-from skyvern.proxy.ports import AuthPort, SessionRegistryPort, UpstreamConnection
-
-
-class FakeClientWebSocket:
- def __init__(self, path: str, incoming: list[str] | None = None, headers: Headers | None = None) -> None:
- self.request = SimpleNamespace(headers=headers if headers is not None else Headers(), path=path)
- self.sent: list[str] = []
- self.close_code: int | None = None
- self.close_reason: str | None = None
- self._incoming = list(incoming or [])
-
- def __aiter__(self) -> FakeClientWebSocket:
- return self
-
- async def __anext__(self) -> str:
- if not self._incoming:
- raise StopAsyncIteration
- return self._incoming.pop(0)
-
- async def send(self, raw: str) -> None:
- self.sent.append(raw)
-
- async def close(self, code: int = 1000, reason: str = "") -> None:
- self.close_code = code
- self.close_reason = reason
-
-
-class RecordingUpstreamBrowser(InMemoryUpstreamBrowser):
- def __init__(self) -> None:
- self.connections: list[UpstreamConnection] = []
- self.sessions: list[ProxySession] = []
-
- async def connect(self, session: ProxySession) -> UpstreamConnection:
- connection = await super().connect(session)
- self.connections.append(connection)
- self.sessions.append(session)
- return connection
-
-
-class RecordingAuth:
- def __init__(self) -> None:
- self.credentials: Mapping[str, str] | None = None
-
- async def authenticate(self, credentials: Mapping[str, str]) -> Principal:
- self.credentials = credentials
- return Principal(principal_id="local-dev")
-
-
-class CountingRegistry:
- def __init__(self, inner: InMemorySessionRegistry) -> None:
- self._inner = inner
- self.resolve_calls = 0
-
- async def resolve(self, session_id: str): # type: ignore[no-untyped-def]
- self.resolve_calls += 1
- return await self._inner.resolve(session_id)
-
- async def invalidate(self, session_id: str) -> None:
- await self._inner.invalidate(session_id)
-
-
-class StaticAuth:
- def __init__(self, principal: Principal) -> None:
- self._principal = principal
-
- async def authenticate(self, credentials: Mapping[str, str]) -> Principal | None:
- return self._principal
-
- def authorize(self, principal: Principal, resolution: SessionResolution) -> bool:
- return principal_owns_resolution(principal, resolution)
-
-
-def make_server(
- upstream: RecordingUpstreamBrowser, sessions: SessionRegistryPort, auth: AuthPort | None = None
-) -> CdpProxyServer:
- return CdpProxyServer(
- upstream=upstream,
- sessions=sessions,
- auth=auth or AllowAllAuth(),
- metrics=NoOpMetrics(),
- event_policy=ForwardAllEventPolicy(),
- )
-
-
-def make_resolved_session(session_id: str = "s1", organization_id: str | None = None) -> ResolvedSession:
- return ResolvedSession(
- session_id=session_id,
- upstream_adapter="memory",
- upstream_ws_url="ws://localhost:0/x",
- organization_id=organization_id,
- connect_headers={"x-routing": "r1"},
- )
-
-
-async def test_client_disconnect_closes_upstream_connection() -> None:
- upstream = RecordingUpstreamBrowser()
- sessions = InMemorySessionRegistry()
- sessions.put(make_resolved_session())
- ws = FakeClientWebSocket(path="/s1", incoming=['{"id": 1, "method": "Page.enable"}'])
-
- await make_server(upstream, sessions)._handle_client(ws) # type: ignore[arg-type]
-
- assert len(upstream.connections) == 1
- assert upstream.connections[0]._closed # type: ignore[attr-defined]
-
-
-async def test_resolved_routing_reaches_the_upstream_connect() -> None:
- upstream = RecordingUpstreamBrowser()
- sessions = InMemorySessionRegistry()
- sessions.put(make_resolved_session())
- ws = FakeClientWebSocket(path="/s1")
-
- await make_server(upstream, sessions)._handle_client(ws) # type: ignore[arg-type]
-
- assert upstream.sessions[0].upstream_ws_url == "ws://localhost:0/x"
- assert upstream.sessions[0].connect_headers == {"x-routing": "r1"}
-
-
- ("seed", "expected_close_code"),
- [
- ("unknown", 4404),
- ("pending", 4409),
- ("closed", 4410),
- ("expired", 4408),
- ],
-)
-async def test_negative_resolutions_are_rejected_before_any_upstream_dial(seed: str, expected_close_code: int) -> None:
- upstream = RecordingUpstreamBrowser()
- sessions = InMemorySessionRegistry()
- if seed == "pending":
- sessions.mark_pending("s1")
- elif seed == "closed":
- sessions.mark_closed("s1")
- elif seed == "expired":
- sessions.mark_expired("s1")
- ws = FakeClientWebSocket(path="/s1")
-
- await make_server(upstream, sessions)._handle_client(ws) # type: ignore[arg-type]
-
- assert ws.close_code == expected_close_code
- assert not upstream.connections
-
-
[email protected]("foreign_state", ["active", "pending", "closed", "expired"])
-async def test_foreign_org_sessions_are_indistinguishable_from_unknown_ids(foreign_state: str) -> None:
- upstream = RecordingUpstreamBrowser()
- sessions = InMemorySessionRegistry()
- if foreign_state == "active":
- sessions.put(make_resolved_session(organization_id="o_owner"))
- elif foreign_state == "pending":
- sessions.mark_pending("s1", organization_id="o_owner")
- elif foreign_state == "closed":
- sessions.mark_closed("s1", organization_id="o_owner")
- elif foreign_state == "expired":
- sessions.mark_expired("s1", organization_id="o_owner")
- auth = StaticAuth(Principal(principal_id="caller", organization_id="o_other"))
- probe_ws = FakeClientWebSocket(path="/s1")
- unknown_ws = FakeClientWebSocket(path="/s_missing")
-
- await make_server(upstream, sessions, auth=auth)._handle_client(probe_ws) # type: ignore[arg-type]
- await make_server(upstream, sessions, auth=auth)._handle_client(unknown_ws) # type: ignore[arg-type]
-
- assert (
- (probe_ws.close_code, probe_ws.close_reason)
- == (unknown_ws.close_code, unknown_ws.close_reason)
- == (
- 4404,
- "unknown session",
- )
- )
- assert not upstream.connections
-
-
-async def test_org_owned_session_rejects_principal_without_organization() -> None:
- upstream = RecordingUpstreamBrowser()
- sessions = InMemorySessionRegistry()
- sessions.put(make_resolved_session(organization_id="o_owner"))
- ws = FakeClientWebSocket(path="/s1")
-
- await make_server(upstream, sessions)._handle_client(ws) # type: ignore[arg-type]
-
- assert ws.close_code == 4404
- assert not upstream.connections
-
-
- ("owned_state", "expected_close_code"),
- [
- ("pending", 4409),
- ("closed", 4410),
- ("expired", 4408),
- ],
-)
-async def test_owning_org_still_receives_lifecycle_close_codes(owned_state: str, expected_close_code: int) -> None:
- upstream = RecordingUpstreamBrowser()
- sessions = InMemorySessionRegistry()
- if owned_state == "pending":
- sessions.mark_pending("s1", organization_id="o_owner")
- elif owned_state == "closed":
- sessions.mark_closed("s1", organization_id="o_owner")
- elif owned_state == "expired":
- sessions.mark_expired("s1", organization_id="o_owner")
- ws = FakeClientWebSocket(path="/s1")
- auth = StaticAuth(Principal(principal_id="caller", organization_id="o_owner"))
-
- await make_server(upstream, sessions, auth=auth)._handle_client(ws) # type: ignore[arg-type]
-
- assert ws.close_code == expected_close_code
- assert not upstream.connections
-
-
-async def test_org_owned_session_allows_matching_organization() -> None:
- upstream = RecordingUpstreamBrowser()
- sessions = InMemorySessionRegistry()
- sessions.put(make_resolved_session(organization_id="o_owner"))
- ws = FakeClientWebSocket(path="/s1")
- auth = StaticAuth(Principal(principal_id="caller", organization_id="o_owner"))
-
- await make_server(upstream, sessions, auth=auth)._handle_client(ws) # type: ignore[arg-type]
-
- assert ws.close_code is None
- assert len(upstream.connections) == 1
-
-
-async def test_repeated_header_does_not_crash_auth() -> None:
- upstream = RecordingUpstreamBrowser()
- sessions = InMemorySessionRegistry()
- sessions.put(make_resolved_session())
- headers = Headers()
- headers["Cookie"] = "a=1"
- headers["Cookie"] = "b=2"
- headers["X-Api-Key"] = "secret"
- ws = FakeClientWebSocket(path="/s1", headers=headers)
- auth = RecordingAuth()
-
- await make_server(upstream, sessions, auth=auth)._handle_client(ws) # type: ignore[arg-type]
-
- # A repeated header must not raise an uncaught error; it is rejected as a
- # clean 4401 auth failure (dict(Headers) raises MultipleValuesError, caught).
- assert ws.close_code == 4401
- assert not upstream.connections
-
-
-async def test_query_string_in_path_resolves_session_id() -> None:
- upstream = RecordingUpstreamBrowser()
- sessions = InMemorySessionRegistry()
- sessions.put(make_resolved_session())
- ws = FakeClientWebSocket(path="/s1?token=abc")
-
- await make_server(upstream, sessions)._handle_client(ws) # type: ignore[arg-type]
-
- assert ws.close_code is None
- assert len(upstream.connections) == 1
-
-
-async def test_unowned_session_is_allowed() -> None:
- upstream = RecordingUpstreamBrowser()
- sessions = InMemorySessionRegistry()
- sessions.put(make_resolved_session())
- ws = FakeClientWebSocket(path="/s1")
-
- await make_server(upstream, sessions)._handle_client(ws) # type: ignore[arg-type]
-
- assert ws.close_code is None
- assert len(upstream.connections) == 1
-
-
-def test_path_credential_without_trailing_session_id_is_not_reused_as_session_id() -> None:
- session_id, credentials = _parse_request({}, "/key/secret-api-key")
- assert credentials["x-api-key"] == "secret-api-key"
- assert session_id == ""
-
-
-def test_path_credential_keeps_a_distinct_trailing_session_id() -> None:
- session_id, credentials = _parse_request({}, "/key/secret-api-key/s1")
- assert credentials["x-api-key"] == "secret-api-key"
- assert session_id == "s1"
-
-
-async def test_malformed_request_target_is_a_clean_close() -> None:
- upstream = RecordingUpstreamBrowser()
- sessions = InMemorySessionRegistry()
- ws = FakeClientWebSocket(path="//[") # urlsplit raises ValueError (invalid IPv6)
-
- await make_server(upstream, sessions)._handle_client(ws) # type: ignore[arg-type]
-
- assert ws.close_code == 4401
- assert not upstream.connections
-
-
-async def test_duplicate_credential_header_is_a_clean_close() -> None:
- upstream = RecordingUpstreamBrowser()
- sessions = InMemorySessionRegistry()
- headers = Headers()
- headers["x-api-key"] = "k1"
- headers["x-api-key"] = "k2"
- ws = FakeClientWebSocket(path="/s1")
- ws.request.headers = headers
-
- await make_server(upstream, sessions)._handle_client(ws) # type: ignore[arg-type]
-
- assert ws.close_code == 4401
- assert not upstream.connections
-
-
-async def test_credential_only_path_resolves_empty_session_and_rejects_4404() -> None:
- upstream = RecordingUpstreamBrowser()
- inner = InMemorySessionRegistry()
- counting = CountingRegistry(inner)
- ws = FakeClientWebSocket(path="/key/secret-api-key")
-
- await make_server(upstream, counting)._handle_client(ws) # type: ignore[arg-type]
-
- assert counting.resolve_calls == 1
- assert ws.close_code == 4404
- assert not upstream.connections
-
-
-async def test_session_is_resolved_once_per_connection_not_per_message() -> None:
- upstream = RecordingUpstreamBrowser()
- inner = InMemorySessionRegistry()
- inner.put(make_resolved_session())
- counting = CountingRegistry(inner)
- frames = [f'{{"id": {frame_id}, "method": "Page.enable"}}' for frame_id in range(1, 6)]
- ws = FakeClientWebSocket(path="/s1", incoming=frames)
-
- await make_server(upstream, counting)._handle_client(ws) # type: ignore[arg-type]
-
- assert counting.resolve_calls == 1
-
-
-# ---- serve_forever lifecycle: health endpoint + signal drain ----------------
-
-
-def _free_port() -> int:
- with socket.socket() as sock:
- sock.bind(("127.0.0.1", 0))
- return int(sock.getsockname()[1])
-
-
-async def _wait_for_listener(port: int, *, up: bool, attempts: int = 500) -> None:
- for _ in range(attempts):
- try:
- _, writer = await asyncio.open_connection("127.0.0.1", port)
- except OSError:
- if not up:
- return
- else:
- writer.close()
- if up:
- return
- await asyncio.sleep(0.01)
- raise AssertionError(f"listener on port {port} never became {'reachable' if up else 'refused'}")
-
-
-async def _http_get(port: int, path: str) -> tuple[int, bytes]:
- reader, writer = await asyncio.open_connection("127.0.0.1", port)
- writer.write(f"GET {path} HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n".encode())
- await writer.drain()
- raw = await reader.read(-1)
- writer.close()
- head, _, body = raw.partition(b"\r\n\r\n")
- return int(head.split(b" ", 2)[1]), body
-
-
-def _lifecycle_server(port: int) -> CdpProxyServer:
- sessions = InMemorySessionRegistry()
- sessions.put(make_resolved_session())
- return CdpProxyServer(
- upstream=RecordingUpstreamBrowser(),
- sessions=sessions,
- auth=AllowAllAuth(),
- metrics=NoOpMetrics(),
- event_policy=ForwardAllEventPolicy(),
- host="127.0.0.1",
- port=port,
- )
-
-
-async def test_serve_forever_healthz_and_sigterm_drain() -> None:
- port = _free_port()
- task = asyncio.create_task(_lifecycle_server(port).serve_forever())
- try:
- await _wait_for_listener(port, up=True)
- status, body = await _http_get(port, "/healthz")
- assert (status, body) == (200, b"ok")
-
- async with websockets_client.connect(f"ws://127.0.0.1:{port}/s1") as client:
- await client.ping()
- os.kill(os.getpid(), signal.SIGTERM)
- # Drain: the listener refuses new work while the live relay survives.
- await _wait_for_listener(port, up=False)
- await client.ping()
- await asyncio.wait_for(task, timeout=10)
- finally:
- if not task.done():
- task.cancel()
- with contextlib.suppress(asyncio.CancelledError):
- await task
-
-
-async def test_serve_forever_force_closes_stragglers_after_drain_timeout(
- monkeypatch: pytest.MonkeyPatch,
-) -> None:
- monkeypatch.setenv("CDP_PROXY_DRAIN_TIMEOUT_SECONDS", "1")
- port = _free_port()
- task = asyncio.create_task(_lifecycle_server(port).serve_forever())
- client = None
- try:
- await _wait_for_listener(port, up=True)
- client = await websockets_client.connect(f"ws://127.0.0.1:{port}/s1")
- os.kill(os.getpid(), signal.SIGTERM)
-
- # The straggler never closes; the drain budget expires and the server
- # exits anyway, closing the client with a clean 1001 instead of a RST.
- await asyncio.wait_for(task, timeout=10)
- with pytest.raises(websockets_exceptions.ConnectionClosed):
- await asyncio.wait_for(client.recv(), timeout=5)
- assert client.protocol.close_code == 1001
- finally:
- if client is not None:
- await client.close()
- if not task.done():
- task.cancel()
- with contextlib.suppress(asyncio.CancelledError):
- await task
-
-
-async def test_second_signal_during_drain_force_closes_immediately(
- monkeypatch: pytest.MonkeyPatch,
-) -> None:
- # Drain budget is huge on purpose: only the second signal can end the wait,
- # so a fast completion proves the fast-exit path rather than the timeout.
- monkeypatch.setenv("CDP_PROXY_DRAIN_TIMEOUT_SECONDS", "3600")
- port = _free_port()
- task = asyncio.create_task(_lifecycle_server(port).serve_forever())
- client = None
- try:
- await _wait_for_listener(port, up=True)
- client = await websockets_client.connect(f"ws://127.0.0.1:{port}/s1")
- os.kill(os.getpid(), signal.SIGTERM)
- await _wait_for_listener(port, up=False)
- os.kill(os.getpid(), signal.SIGTERM)
-
- await asyncio.wait_for(task, timeout=10)
- with pytest.raises(websockets_exceptions.ConnectionClosed):
- await asyncio.wait_for(client.recv(), timeout=5)
- assert client.protocol.close_code == 1001
- finally:
- if client is not None:
- await client.close()
- if not task.done():
- task.cancel()
- with contextlib.suppress(asyncio.CancelledError):
- await task
diff --git a/tests/unit/proxy/test_websocket_upstream.py b/tests/unit/proxy/test_websocket_upstream.py
deleted file mode 100644
index fa7e128ff..000000000
--- a/tests/unit/proxy/test_websocket_upstream.py
+++ /dev/null
@@ -1,376 +0,0 @@
-"""Failure-taxonomy and header-injection tests for the generic CDP-over-WebSocket adapter."""
-
-from __future__ import annotations
-
-import asyncio
-import logging
-import socket
-import ssl
-from http import HTTPStatus
-from unittest.mock import patch
-
-import pytest
-import websockets
-
-from skyvern.proxy.adapters.websocket_upstream import (
- WebSocketUpstreamBrowser,
- classify_connect_error,
- merge_connect_headers,
- resolve_retry_budget,
-)
-from skyvern.proxy.core.errors import (
- ProtocolConfigurationError,
- TransientConnectionError,
- VendorAuthError,
- VendorRateLimitError,
-)
-from skyvern.proxy.core.session import ProxySession
-
-
[email protected](autouse=True)
-def _tight_retry_budget(monkeypatch: pytest.MonkeyPatch) -> None:
- monkeypatch.setenv("CDP_PROXY_CONNECT_RETRY_ATTEMPTS", "1")
- monkeypatch.setenv("CDP_PROXY_CONNECT_BACKOFF_SECONDS", "0")
-
-
-def test_retry_budget_env_knobs_and_misconfig_fallback(monkeypatch: pytest.MonkeyPatch) -> None:
- monkeypatch.delenv("CDP_PROXY_CONNECT_RETRY_ATTEMPTS", raising=False)
- monkeypatch.delenv("CDP_PROXY_CONNECT_BACKOFF_SECONDS", raising=False)
- assert resolve_retry_budget() == (6, (1.0, 2.0, 3.0, 4.0, 5.0))
- monkeypatch.setenv("CDP_PROXY_CONNECT_RETRY_ATTEMPTS", "0")
- monkeypatch.setenv("CDP_PROXY_CONNECT_BACKOFF_SECONDS", "nope")
- assert resolve_retry_budget() == (6, (1.0, 2.0, 3.0, 4.0, 5.0))
- monkeypatch.setenv("CDP_PROXY_CONNECT_RETRY_ATTEMPTS", "3")
- monkeypatch.setenv("CDP_PROXY_CONNECT_BACKOFF_SECONDS", "-1")
- assert resolve_retry_budget() == (3, (1.0, 2.0, 3.0, 4.0, 5.0))
- monkeypatch.setenv("CDP_PROXY_CONNECT_BACKOFF_SECONDS", "0.5,1.5")
- assert resolve_retry_budget() == (3, (0.5, 1.5))
- monkeypatch.setenv("CDP_PROXY_CONNECT_BACKOFF_SECONDS", "nan")
- assert resolve_retry_budget() == (3, (1.0, 2.0, 3.0, 4.0, 5.0))
- monkeypatch.setenv("CDP_PROXY_CONNECT_BACKOFF_SECONDS", "inf")
- assert resolve_retry_budget() == (3, (1.0, 2.0, 3.0, 4.0, 5.0))
- monkeypatch.setenv("CDP_PROXY_CONNECT_BACKOFF_SECONDS", "301")
- assert resolve_retry_budget() == (3, (1.0, 2.0, 3.0, 4.0, 5.0))
- monkeypatch.setenv("CDP_PROXY_CONNECT_RETRY_ATTEMPTS", "101")
- monkeypatch.setenv("CDP_PROXY_CONNECT_BACKOFF_SECONDS", "1")
- assert resolve_retry_budget() == (6, (1.0,))
-
-
-def _session(url: str) -> ProxySession:
- return ProxySession(session_id="test-session", upstream_ws_url=url)
-
-
-# Bound but never listening: dialing it gets ECONNREFUSED while the bind holds the
-# port reserved, avoiding a TOCTOU race with other processes grabbing it.
-def refused_port():
- with socket.socket() as sock:
- sock.bind(("127.0.0.1", 0))
- yield sock.getsockname()[1]
-
-
-async def _echo(ws: websockets.ServerConnection) -> None:
- async for message in ws:
- await ws.send(message)
-
-
-def _reject_with(status: HTTPStatus):
- def handler(connection: websockets.ServerConnection, request): # type: ignore[no-untyped-def]
- return connection.respond(status, "rejected\n")
-
- return handler
-
-
-async def _connect_expecting(status: HTTPStatus, expected_error: type[Exception]) -> Exception:
- async with websockets.serve(_echo, "127.0.0.1", 0, process_request=_reject_with(status)) as server:
- port = server.sockets[0].getsockname()[1]
- with pytest.raises(expected_error) as excinfo:
- await WebSocketUpstreamBrowser().connect(_session(f"ws://127.0.0.1:{port}/devtools/browser/test"))
- return excinfo.value
-
-
-async def test_refused_connection_raises_transient(refused_port: int) -> None:
- with pytest.raises(TransientConnectionError):
- await WebSocketUpstreamBrowser().connect(_session(f"ws://127.0.0.1:{refused_port}/devtools/browser/test"))
-
-
-async def test_malformed_url_raises_protocol_configuration() -> None:
- with pytest.raises(ProtocolConfigurationError) as excinfo:
- await WebSocketUpstreamBrowser().connect(_session("ws://[invalid"))
- assert excinfo.value.__cause__ is None
- assert excinfo.value.__context__ is None
-
-
-async def test_auth_rejection_raises_vendor_auth() -> None:
- await _connect_expecting(HTTPStatus.UNAUTHORIZED, VendorAuthError)
- await _connect_expecting(HTTPStatus.FORBIDDEN, VendorAuthError)
-
-
-async def test_rate_limit_rejection_raises_vendor_rate_limit() -> None:
- await _connect_expecting(HTTPStatus.TOO_MANY_REQUESTS, VendorRateLimitError)
-
-
-async def test_non_websocket_endpoint_raises_protocol_configuration() -> None:
- await _connect_expecting(HTTPStatus.NOT_FOUND, ProtocolConfigurationError)
-
-
-async def test_server_error_rejection_is_transient() -> None:
- await _connect_expecting(HTTPStatus.SERVICE_UNAVAILABLE, TransientConnectionError)
-
-
-async def test_invalid_uri_raises_protocol_configuration() -> None:
- with pytest.raises(ProtocolConfigurationError):
- await WebSocketUpstreamBrowser().connect(_session("http://127.0.0.1:1/devtools/browser/test"))
-
-
-async def test_error_never_leaks_url_path_query_or_cause() -> None:
- async with websockets.serve(_echo, "127.0.0.1", 0, process_request=_reject_with(HTTPStatus.UNAUTHORIZED)) as server:
- port = server.sockets[0].getsockname()[1]
- with pytest.raises(VendorAuthError) as excinfo:
- await WebSocketUpstreamBrowser().connect(
- _session(f"ws://127.0.0.1:{port}/devtools/browser/SECRETPATH?token=SECRETTOKEN")
- )
- message = str(excinfo.value)
- assert "SECRETPATH" not in message
- assert "SECRETTOKEN" not in message
- assert f"ws://127.0.0.1:{port}" in message
- assert excinfo.value.__cause__ is None
- assert excinfo.value.__context__ is None
-
-
-async def test_transient_failure_retries_until_upstream_recovers(monkeypatch: pytest.MonkeyPatch) -> None:
- monkeypatch.setenv("CDP_PROXY_CONNECT_RETRY_ATTEMPTS", "3")
- handshakes = {"count": 0}
-
- def flaky(connection: websockets.ServerConnection, request): # type: ignore[no-untyped-def]
- handshakes["count"] += 1
- if handshakes["count"] == 1:
- return connection.respond(HTTPStatus.SERVICE_UNAVAILABLE, "warming up\n")
- return None
-
- async with websockets.serve(_echo, "127.0.0.1", 0, process_request=flaky) as server:
- port = server.sockets[0].getsockname()[1]
- connection = await WebSocketUpstreamBrowser().connect(_session(f"ws://127.0.0.1:{port}/devtools/browser/test"))
- await connection.send('{"id": 1, "method": "Browser.getVersion"}')
- assert await connection.receive() == '{"id": 1, "method": "Browser.getVersion"}'
- await connection.close()
- assert handshakes["count"] == 2
-
-
-async def test_mid_handshake_drop_is_transient_and_retries_to_exhaustion(monkeypatch: pytest.MonkeyPatch) -> None:
- monkeypatch.setenv("CDP_PROXY_CONNECT_RETRY_ATTEMPTS", "3")
- connections = {"count": 0}
-
- async def drop_after_request(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
- connections["count"] += 1
- await reader.read(1024)
- writer.close()
-
- server = await asyncio.start_server(drop_after_request, "127.0.0.1", 0)
- port = server.sockets[0].getsockname()[1]
- try:
- with pytest.raises(TransientConnectionError):
- await WebSocketUpstreamBrowser().connect(_session(f"ws://127.0.0.1:{port}/devtools/browser/test"))
- finally:
- server.close()
- await server.wait_closed()
- assert connections["count"] == 3
-
-
-async def test_persistent_transient_rejection_retries_to_exhaustion(monkeypatch: pytest.MonkeyPatch) -> None:
- monkeypatch.setenv("CDP_PROXY_CONNECT_RETRY_ATTEMPTS", "3")
- handshakes = {"count": 0}
-
- def reject(connection: websockets.ServerConnection, request): # type: ignore[no-untyped-def]
- handshakes["count"] += 1
- return connection.respond(HTTPStatus.SERVICE_UNAVAILABLE, "unavailable\n")
-
- async with websockets.serve(_echo, "127.0.0.1", 0, process_request=reject) as server:
- port = server.sockets[0].getsockname()[1]
- with pytest.raises(TransientConnectionError):
- await WebSocketUpstreamBrowser().connect(_session(f"ws://127.0.0.1:{port}/devtools/browser/test"))
- assert handshakes["count"] == 3
-
-
-async def test_non_transient_rejection_short_circuits_without_retry(monkeypatch: pytest.MonkeyPatch) -> None:
- monkeypatch.setenv("CDP_PROXY_CONNECT_RETRY_ATTEMPTS", "3")
- handshakes = {"count": 0}
-
- def reject(connection: websockets.ServerConnection, request): # type: ignore[no-untyped-def]
- handshakes["count"] += 1
- return connection.respond(HTTPStatus.UNAUTHORIZED, "rejected\n")
-
- async with websockets.serve(_echo, "127.0.0.1", 0, process_request=reject) as server:
- port = server.sockets[0].getsockname()[1]
- with pytest.raises(VendorAuthError):
- await WebSocketUpstreamBrowser().connect(_session(f"ws://127.0.0.1:{port}/devtools/browser/test"))
- assert handshakes["count"] == 1
-
-
-async def test_userinfo_credentials_never_leak_in_error_or_logs() -> None:
- async with websockets.serve(_echo, "127.0.0.1", 0, process_request=_reject_with(HTTPStatus.UNAUTHORIZED)) as server:
- port = server.sockets[0].getsockname()[1]
- with (
- patch("skyvern.proxy.adapters.websocket_upstream.LOG") as mock_log,
- pytest.raises(VendorAuthError) as excinfo,
- ):
- await WebSocketUpstreamBrowser().connect(
- _session(f"ws://user:[email protected]:{port}/devtools/browser/test")
- )
- message = str(excinfo.value)
- assert f"ws://127.0.0.1:{port}" in message
- assert "SECRET" not in message
- assert "user:" not in message
- # The chain must be fully scrubbed: the raw transport exception carries the URL.
- assert excinfo.value.__cause__ is None
- assert excinfo.value.__context__ is None
- logged = " ".join(
- str(value)
- for call in mock_log.warning.call_args_list + mock_log.info.call_args_list
- for value in call.kwargs.values()
- )
- assert "SECRET" not in logged
- assert "user:" not in logged
-
-
-async def test_open_timeout_is_transient() -> None:
- async def stall(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
- try:
- await reader.read()
- finally:
- writer.close()
-
- server = await asyncio.start_server(stall, "127.0.0.1", 0)
- port = server.sockets[0].getsockname()[1]
- try:
- with pytest.raises(TransientConnectionError):
- await WebSocketUpstreamBrowser(open_timeout_seconds=0.1).connect(
- _session(f"ws://127.0.0.1:{port}/devtools/browser/test")
- )
- finally:
- server.close()
- await server.wait_closed()
-
-
-async def test_injected_headers_reach_the_upstream_handshake() -> None:
- seen: dict[str, str | None] = {}
-
- def capture(connection: websockets.ServerConnection, request): # type: ignore[no-untyped-def]
- seen["authorization"] = request.headers.get("Authorization")
- return None
-
- async with websockets.serve(_echo, "127.0.0.1", 0, process_request=capture) as server:
- port = server.sockets[0].getsockname()[1]
- adapter = WebSocketUpstreamBrowser(connect_headers={"Authorization": "Bearer test-token"})
- connection = await adapter.connect(_session(f"ws://127.0.0.1:{port}/devtools/browser/test"))
- await connection.close()
- assert seen["authorization"] == "Bearer test-token"
-
-
-async def test_browser_address_discriminator_fragment_is_stripped_before_dialing() -> None:
- async with websockets.serve(_echo, "127.0.0.1", 0) as server:
- port = server.sockets[0].getsockname()[1]
- connection = await WebSocketUpstreamBrowser().connect(
- _session(f"ws://127.0.0.1:{port}/devtools/browser/test#pbs_123")
- )
- await connection.close()
-
-
-async def test_cross_origin_redirect_never_leaks_credentials_and_fails_closed() -> None:
- redirect_target: dict[str, str | None] = {}
-
- def capture_auth(connection: websockets.ServerConnection, request): # type: ignore[no-untyped-def]
- redirect_target["authorization"] = request.headers.get("Authorization")
- return None
-
- async with websockets.serve(_echo, "127.0.0.1", 0, process_request=capture_auth) as target:
- target_port = target.sockets[0].getsockname()[1]
-
- def redirect_to_target(connection: websockets.ServerConnection, request): # type: ignore[no-untyped-def]
- response = connection.respond(HTTPStatus.FOUND, "redirecting\n")
- response.headers["Location"] = f"ws://127.0.0.1:{target_port}/devtools/browser/test"
- return response
-
- async with websockets.serve(_echo, "127.0.0.1", 0, process_request=redirect_to_target) as redirector:
- redirect_port = redirector.sockets[0].getsockname()[1]
- adapter = WebSocketUpstreamBrowser(connect_headers={"Authorization": "Bearer redirect-secret"})
- with pytest.raises(ProtocolConfigurationError) as excinfo:
- await adapter.connect(_session(f"ws://127.0.0.1:{redirect_port}/devtools/browser/test"))
- assert redirect_target.get("authorization") is None
- assert "redirect-secret" not in str(excinfo.value)
-
-
-def test_tls_verification_failure_classified_as_configuration_not_transient() -> None:
- error = classify_connect_error(ssl.SSLCertVerificationError("certificate verify failed"), "wss://127.0.0.1")
- assert isinstance(error, ProtocolConfigurationError)
- assert not isinstance(error, TransientConnectionError)
-
-
-async def test_tls_failure_fails_closed_without_retry(monkeypatch: pytest.MonkeyPatch) -> None:
- monkeypatch.setenv("CDP_PROXY_CONNECT_RETRY_ATTEMPTS", "3")
- dials = {"count": 0}
-
- class _TlsFailingConnector:
- process_redirect = None
-
- def __await__(self): # type: ignore[no-untyped-def]
- dials["count"] += 1
- raise ssl.SSLCertVerificationError("certificate verify failed")
- yield # unreachable; makes __await__ a generator
-
- monkeypatch.setattr(
- "skyvern.proxy.adapters.websocket_upstream.websockets.connect",
- lambda *args, **kwargs: _TlsFailingConnector(),
- )
- with pytest.raises(ProtocolConfigurationError):
- await WebSocketUpstreamBrowser().connect(_session("wss://127.0.0.1:9/devtools/browser/test"))
- assert dials["count"] == 1
-
-
-async def test_websocket_debug_handshake_logs_are_scrubbed_of_credentials(caplog: pytest.LogCaptureFixture) -> None:
- async with websockets.serve(_echo, "127.0.0.1", 0) as server:
- port = server.sockets[0].getsockname()[1]
- caplog.set_level(logging.DEBUG, logger="websockets")
- adapter = WebSocketUpstreamBrowser(connect_headers={"Authorization": "Bearer log-secret"})
- connection = await adapter.connect(
- _session(f"ws://127.0.0.1:{port}/devtools/browser/SECRETPATH?token=SECRETTOKEN")
- )
- await connection.close()
- handshake = "\n".join(record.getMessage() for record in caplog.records if record.name == "websockets")
- assert "log-secret" not in handshake
- assert "Bearer" not in handshake
- assert "SECRETPATH" not in handshake
- assert "SECRETTOKEN" not in handshake
- # Guard against a vacuous pass: the DEBUG handshake must actually have been logged.
- assert any("HTTP/1.1" in record.getMessage() for record in caplog.records if record.name == "websockets")
-
-
-def test_connect_headers_merge_case_insensitively_with_operator_value_winning() -> None:
- merged = merge_connect_headers(
- {"Authorization": "Bearer operator"},
- {"authorization": "Bearer session", "x-routing": "r1"},
- )
- assert merged == {"Authorization": "Bearer operator", "x-routing": "r1"}
-
-
-def test_connect_headers_merge_keeps_session_only_headers() -> None:
- assert merge_connect_headers({}, {"x-routing": "r1"}) == {"x-routing": "r1"}
diff --git a/tests/unit/test_local_browser_profile.py b/tests/unit/test_local_browser_profile.py
new file mode 100644
index 000000000..ea3e98fde
--- /dev/null
+++ b/tests/unit/test_local_browser_profile.py
@@ -0,0 +1,862 @@
+from __future__ import annotations
+
+import os
+import stat
+import subprocess
+import sys
+import tempfile
+import threading
+import time
+from pathlib import Path
+from unittest.mock import MagicMock
+
+import psutil
+import pytest
+from structlog.testing import capture_logs
+
+from skyvern.library import local_browser_profile
+
+
[email protected](autouse=True)
+def _skip_unrequested_background_sweeps(monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(local_browser_profile, "_sweep_triggered", True)
+
+
+def _managed_root(tmp_path: Path) -> Path:
+ return tmp_path / f"skyvern-local-browsers-{os.getuid()}"
+
+
+def _lock_path(profile_path: Path) -> Path:
+ entries = list(profile_path.iterdir())
+ assert len(entries) == 1
+ return entries[0]
+
+
+def _stat_with_uid(result: os.stat_result, uid: int) -> os.stat_result:
+ values = list(result)
+ values[4] = uid
+ return os.stat_result(values)
+
+
+def _patch_wrong_uid(monkeypatch: pytest.MonkeyPatch, path: Path) -> None:
+ real_lstat = os.lstat
+
+ def wrong_uid_lstat(candidate: os.PathLike[str] | str, *args: object, **kwargs: object) -> os.stat_result:
+ result = real_lstat(candidate, *args, **kwargs)
+ return _stat_with_uid(result, os.getuid() + 1) if Path(candidate) == path else result
+
+ monkeypatch.setattr(os, "lstat", wrong_uid_lstat)
+
+
+def _matching_process(profile_path: Path) -> MagicMock:
+ process = MagicMock()
+ process.pid = 43210
+ process.info = {"pid": process.pid, "cmdline": ["chromium", f"--user-data-dir={profile_path}"]}
+ process.cmdline.return_value = process.info["cmdline"]
+ process.children.return_value = []
+ return process
+
+
+def _blank_cmdline_process(pid: int) -> MagicMock:
+ process = MagicMock()
+ process.pid = pid
+ process.info = {"pid": pid, "cmdline": None}
+ return process
+
+
+def _run_deletion_child(path: Path, identity: tuple[int, int]) -> subprocess.CompletedProcess[bytes]:
+ return subprocess.run(
+ [
+ sys.executable,
+ "-m",
+ "skyvern.library.local_browser_profile",
+ "--delete-profile-tree",
+ str(path),
+ str(identity[0]),
+ str(identity[1]),
+ ],
+ check=False,
+ stdout=subprocess.DEVNULL,
+ stderr=subprocess.DEVNULL,
+ timeout=5,
+ )
+
+
+def test_create_uses_secure_per_uid_root(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+
+ profile = local_browser_profile.create_local_browser_profile()
+ try:
+ root_stat = profile.path.parent.lstat()
+ profile_stat = profile.path.lstat()
+ assert profile.path.parent == _managed_root(tmp_path)
+ assert stat.S_IMODE(root_stat.st_mode) == 0o700
+ assert root_stat.st_uid == os.getuid()
+ assert stat.S_IMODE(profile_stat.st_mode) == 0o700
+ assert profile_stat.st_uid == os.getuid()
+ finally:
+ profile.release()
+
+
[email protected](os.name == "nt", reason="managed profile sweep is POSIX-only")
+def test_create_triggers_background_sweep_once(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ monkeypatch.setattr(local_browser_profile, "_sweep_triggered", False)
+ sweep = MagicMock()
+ monkeypatch.setattr(local_browser_profile, "sweep_local_browser_profiles_with_budget", sweep)
+ trigger = local_browser_profile.sweep_local_browser_profiles_once_in_background
+ threads: list[threading.Thread] = []
+
+ def capture_thread() -> threading.Thread | None:
+ thread = trigger()
+ if thread is not None:
+ threads.append(thread)
+ return thread
+
+ monkeypatch.setattr(local_browser_profile, "sweep_local_browser_profiles_once_in_background", capture_thread)
+
+ first = local_browser_profile.create_local_browser_profile()
+ second = local_browser_profile.create_local_browser_profile()
+ try:
+ assert len(threads) == 1
+ threads[0].join(timeout=5)
+ assert not threads[0].is_alive()
+ assert sweep.call_count == 1
+ finally:
+ first.release()
+ second.release()
+
+
[email protected](os.name == "nt", reason="managed profile sweep is POSIX-only")
+def test_budgeted_sweep_prevents_subsequent_background_sweep(monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(local_browser_profile, "_sweep_triggered", False)
+ monkeypatch.setattr(local_browser_profile.subprocess, "Popen", MagicMock(return_value=MagicMock()))
+ monkeypatch.setattr(local_browser_profile, "_wait_for_process_with_budget", MagicMock(return_value=0))
+
+ local_browser_profile.sweep_local_browser_profiles_with_budget()
+
+ assert local_browser_profile._sweep_triggered is True
+ assert local_browser_profile.sweep_local_browser_profiles_once_in_background() is None
+
+
[email protected](os.name == "nt", reason="managed profile sweep is POSIX-only")
+def test_create_background_sweep_reaps_orphan_and_preserves_new_profile(
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ monkeypatch.setenv("TMPDIR", str(tmp_path))
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ monkeypatch.setattr(local_browser_profile, "_sweep_triggered", False)
+ root = _managed_root(tmp_path)
+ root.mkdir(mode=local_browser_profile.PROFILE_ROOT_MODE)
+ root.chmod(local_browser_profile.PROFILE_ROOT_MODE)
+ orphan = root / f"{local_browser_profile.PROFILE_DIR_PREFIX}orphan"
+ orphan.mkdir(mode=local_browser_profile.PROFILE_DIR_MODE)
+ orphan.chmod(local_browser_profile.PROFILE_DIR_MODE)
+ lock_path = orphan / local_browser_profile.PROFILE_LOCK_FILENAME
+ lock_path.touch(mode=local_browser_profile.PROFILE_LOCK_MODE)
+ lock_path.chmod(local_browser_profile.PROFILE_LOCK_MODE)
+ monkeypatch.setattr(local_browser_profile, "terminate_local_browser_processes", lambda _path: True)
+ monkeypatch.setattr(
+ local_browser_profile,
+ "sweep_local_browser_profiles_with_budget",
+ local_browser_profile.sweep_orphaned_local_browser_profiles,
+ )
+ trigger = local_browser_profile.sweep_local_browser_profiles_once_in_background
+ threads: list[threading.Thread] = []
+
+ def capture_thread() -> threading.Thread | None:
+ thread = trigger()
+ if thread is not None:
+ threads.append(thread)
+ return thread
+
+ monkeypatch.setattr(local_browser_profile, "sweep_local_browser_profiles_once_in_background", capture_thread)
+
+ profile = local_browser_profile.create_local_browser_profile()
+ try:
+ assert len(threads) == 1
+ threads[0].join(timeout=5)
+ assert not threads[0].is_alive()
+ assert not orphan.exists()
+ assert profile.path.exists()
+ assert profile.revalidate()
+ finally:
+ profile.release()
+
+
+def test_windows_background_sweep_and_create_bypass_posix_paths(monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(local_browser_profile, "_sweep_triggered", False)
+ getuid = MagicMock(side_effect=AssertionError("os.getuid must not run on Windows"))
+ sweep = MagicMock(side_effect=AssertionError("profile sweep must not run on Windows"))
+ thread = MagicMock(side_effect=AssertionError("profile sweep thread must not start on Windows"))
+ monkeypatch.setattr(local_browser_profile.sys, "platform", "win32")
+ monkeypatch.setattr(local_browser_profile.os, "getuid", getuid)
+ monkeypatch.setattr(local_browser_profile, "sweep_local_browser_profiles_with_budget", sweep)
+ monkeypatch.setattr(local_browser_profile.threading, "Thread", thread)
+
+ assert local_browser_profile.sweep_local_browser_profiles_once_in_background() is None
+ assert local_browser_profile.create_local_browser_profile() is None
+ assert local_browser_profile._sweep_triggered is False
+ getuid.assert_not_called()
+ sweep.assert_not_called()
+ thread.assert_not_called()
+
+
[email protected]("unsafe_root", ["mode", "owner", "symlink"])
+def test_create_rejects_unsafe_root(
+ unsafe_root: str,
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ root = _managed_root(tmp_path)
+ if unsafe_root == "symlink":
+ target = tmp_path / "root-target"
+ target.mkdir(mode=0o700)
+ root.symlink_to(target, target_is_directory=True)
+ else:
+ root.mkdir(mode=0o700)
+ if unsafe_root == "mode":
+ root.chmod(0o755)
+ else:
+ _patch_wrong_uid(monkeypatch, root)
+
+ with pytest.raises((OSError, RuntimeError)):
+ local_browser_profile.create_local_browser_profile()
+
+
[email protected]("unsafe_root", ["mode", "owner", "symlink"])
+def test_sweep_rejects_unsafe_root(
+ unsafe_root: str,
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ profile = local_browser_profile.create_local_browser_profile()
+ profile.release()
+ root = profile.path.parent
+ if unsafe_root == "mode":
+ root.chmod(0o755)
+ elif unsafe_root == "owner":
+ _patch_wrong_uid(monkeypatch, root)
+ else:
+ target = tmp_path / "root-target"
+ root.rename(target)
+ root.symlink_to(target, target_is_directory=True)
+
+ cleanup = MagicMock(return_value=True)
+ monkeypatch.setattr(local_browser_profile, "cleanup_local_browser_profile", cleanup)
+
+ local_browser_profile.sweep_orphaned_local_browser_profiles()
+
+ assert profile.path.exists()
+ cleanup.assert_not_called()
+
+
[email protected]("lock_state", ["directory", "symlink", "wrong_mode"])
+def test_sweep_skips_malformed_lock_without_removing_the_candidate(
+ lock_state: str,
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ profile = local_browser_profile.create_local_browser_profile()
+ profile.release()
+ lock_path = _lock_path(profile.path)
+ lock_path.unlink()
+ if lock_state == "directory":
+ lock_path.mkdir()
+ elif lock_state == "symlink":
+ external_lock = tmp_path / "external-lock"
+ external_lock.touch()
+ lock_path.symlink_to(external_lock)
+ else:
+ lock_path.touch(mode=0o644)
+
+ terminate = MagicMock(return_value=True)
+ rmtree = MagicMock()
+ rmdir = MagicMock(side_effect=AssertionError("malformed locks must not trigger recovery"))
+ monkeypatch.setattr(local_browser_profile, "terminate_local_browser_processes", terminate)
+ monkeypatch.setattr(local_browser_profile.shutil, "rmtree", rmtree)
+ monkeypatch.setattr(local_browser_profile.os, "rmdir", rmdir)
+
+ local_browser_profile.sweep_orphaned_local_browser_profiles()
+
+ assert profile.path.exists()
+ terminate.assert_not_called()
+ rmtree.assert_not_called()
+ rmdir.assert_not_called()
+
+
+def test_sweep_removes_lockless_empty_candidate(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ profile = local_browser_profile.create_local_browser_profile()
+ profile.release()
+ _lock_path(profile.path).unlink()
+
+ assert local_browser_profile.sweep_orphaned_local_browser_profiles() == 1
+ assert not profile.path.exists()
+
+
+def test_sweep_leaves_lockless_nonempty_candidate(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ profile = local_browser_profile.create_local_browser_profile()
+ profile.release()
+ _lock_path(profile.path).unlink()
+ sentinel = profile.path / "sentinel"
+ sentinel.write_text("live data")
+
+ assert local_browser_profile.sweep_orphaned_local_browser_profiles() == 0
+ assert sentinel.read_text() == "live data"
+
+
+def test_sweep_rejects_candidate_owned_by_another_uid(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ profile = local_browser_profile.create_local_browser_profile()
+ profile.release()
+ _patch_wrong_uid(monkeypatch, profile.path)
+ cleanup = MagicMock(return_value=True)
+ monkeypatch.setattr(local_browser_profile, "cleanup_local_browser_profile", cleanup)
+
+ local_browser_profile.sweep_orphaned_local_browser_profiles()
+
+ assert profile.path.exists()
+ cleanup.assert_not_called()
+
+
[email protected]("replaced_identity", ["directory", "lock"])
+def test_creator_revalidates_directory_and_lock_identity(
+ replaced_identity: str,
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ profile = local_browser_profile.create_local_browser_profile()
+ assert profile.revalidate() is True
+
+ if replaced_identity == "directory":
+ original_path = profile.path.with_name(f"{profile.path.name}-original")
+ profile.path.rename(original_path)
+ profile.path.mkdir(mode=0o700)
+ else:
+ lock_path = _lock_path(profile.path)
+ lock_path.rename(lock_path.with_name(f"{lock_path.name}-original"))
+ lock_path.touch(mode=0o600)
+
+ try:
+ assert profile.revalidate() is False
+ finally:
+ profile.release()
+
+
[email protected](os.name == "nt", reason="flock is POSIX-only")
+def test_real_child_lock_preserves_live_profile_then_sweep_reaps_orphan(
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ profile = local_browser_profile.create_local_browser_profile()
+ lock_path = _lock_path(profile.path)
+ probe = subprocess.run(
+ [
+ sys.executable,
+ "-c",
+ (
+ "import fcntl, sys; f=open(sys.argv[1], 'rb'); "
+ "\ntry: fcntl.flock(f, fcntl.LOCK_EX | fcntl.LOCK_NB)"
+ "\nexcept BlockingIOError: raise SystemExit(0)"
+ "\nraise SystemExit(1)"
+ ),
+ str(lock_path),
+ ],
+ check=False,
+ timeout=5,
+ )
+ assert probe.returncode == 0
+ profile.release()
+
+ ready_path = tmp_path / "child-ready"
+ child = subprocess.Popen(
+ [
+ sys.executable,
+ "-c",
+ (
+ "import fcntl, pathlib, sys, time; f=open(sys.argv[1], 'rb'); "
+ "fcntl.flock(f, fcntl.LOCK_EX | fcntl.LOCK_NB); "
+ "pathlib.Path(sys.argv[2]).touch(); time.sleep(60)"
+ ),
+ str(lock_path),
+ str(ready_path),
+ ]
+ )
+ try:
+ deadline = time.monotonic() + 5
+ while not ready_path.exists() and child.poll() is None and time.monotonic() < deadline:
+ time.sleep(0.01)
+ assert ready_path.exists()
+ monkeypatch.setattr(local_browser_profile, "terminate_local_browser_processes", lambda _path: True)
+
+ local_browser_profile.sweep_orphaned_local_browser_profiles()
+
+ assert profile.path.exists()
+ finally:
+ child.terminate()
+ child.wait(timeout=5)
+
+ local_browser_profile.sweep_orphaned_local_browser_profiles()
+
+ assert not profile.path.exists()
+
+
[email protected]("failure", ["incomplete_scan", "denied_kill", "wait_error", "alive_survivor"])
+def test_kill_failure_prevents_profile_deletion(
+ failure: str,
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ profile_path = tmp_path / "profile"
+ profile_path.mkdir()
+ process = _matching_process(profile_path)
+ if failure == "incomplete_scan":
+ monkeypatch.setattr(
+ local_browser_profile.psutil,
+ "process_iter",
+ MagicMock(side_effect=psutil.AccessDenied(pid=process.pid)),
+ )
+ else:
+ monkeypatch.setattr(local_browser_profile.psutil, "process_iter", MagicMock(return_value=[process]))
+ if failure == "denied_kill":
+ process.kill.side_effect = psutil.AccessDenied(pid=process.pid)
+ elif failure == "wait_error":
+ monkeypatch.setattr(local_browser_profile.psutil, "wait_procs", MagicMock(side_effect=RuntimeError("wait")))
+ elif failure == "alive_survivor":
+ monkeypatch.setattr(local_browser_profile.psutil, "wait_procs", MagicMock(return_value=([], [process])))
+
+ rmtree = MagicMock()
+ monkeypatch.setattr(local_browser_profile.shutil, "rmtree", rmtree)
+
+ assert local_browser_profile.terminate_local_browser_processes(profile_path) is False
+ assert local_browser_profile.cleanup_local_browser_profile(profile_path) is False
+ rmtree.assert_not_called()
+
+
+def test_blank_cmdline_bystander_does_not_hide_matching_process(
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ profile_path = tmp_path / "profile"
+ profile_path.mkdir()
+ matching = _matching_process(profile_path)
+ blank = _blank_cmdline_process(43211)
+ monkeypatch.setattr(local_browser_profile.psutil, "process_iter", MagicMock(return_value=[blank, matching]))
+ monkeypatch.setattr(local_browser_profile.psutil, "wait_procs", MagicMock(return_value=([matching], [])))
+
+ assert local_browser_profile.cleanup_local_browser_profile(profile_path) is True
+ matching.kill.assert_called_once_with()
+ blank.kill.assert_not_called()
+ assert not profile_path.exists()
+
+
+def test_blank_cmdline_bystanders_do_not_block_deletion(
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ profile_path = tmp_path / "profile"
+ profile_path.mkdir()
+ blank = _blank_cmdline_process(43211)
+ monkeypatch.setattr(local_browser_profile.psutil, "process_iter", MagicMock(return_value=[blank]))
+
+ assert local_browser_profile.cleanup_local_browser_profile(profile_path) is True
+ blank.kill.assert_not_called()
+ assert not profile_path.exists()
+
+
+def test_unknown_cleanup_failure_skips_unlink(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ profile_path = tmp_path / "profile"
+ profile_path.mkdir()
+ monkeypatch.setattr(
+ local_browser_profile,
+ "terminate_local_browser_processes",
+ MagicMock(side_effect=RuntimeError("unknown")),
+ )
+ rmtree = MagicMock()
+ monkeypatch.setattr(local_browser_profile.shutil, "rmtree", rmtree)
+
+ assert local_browser_profile.cleanup_local_browser_profile(profile_path) is False
+ rmtree.assert_not_called()
+
+
+def test_cleanup_spawn_failure_returns_false_and_releases_lock(
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ profile = local_browser_profile.create_local_browser_profile()
+ lock_fd = profile._lock_fd
+ assert lock_fd is not None
+ monkeypatch.setattr(local_browser_profile, "terminate_local_browser_processes", lambda _path: True)
+ monkeypatch.setattr(
+ local_browser_profile.subprocess,
+ "Popen",
+ MagicMock(side_effect=OSError("cleanup spawn failed")),
+ )
+
+ with capture_logs() as logs:
+ assert local_browser_profile.cleanup_local_browser_profile(profile) is False
+
+ assert profile._lock_fd is None
+ with pytest.raises(OSError):
+ os.fstat(lock_fd)
+ assert any(log.get("event") == "local_browser_profile_cleanup_failed" for log in logs)
+
+
+def test_creation_rollback_failure_preserves_original_error(
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ monkeypatch.setattr(local_browser_profile.os, "open", MagicMock(side_effect=PermissionError("lock open failed")))
+ monkeypatch.setattr(
+ local_browser_profile,
+ "_remove_profile_directory_bounded",
+ MagicMock(side_effect=OSError("rollback cleanup failed")),
+ )
+
+ with pytest.raises(PermissionError, match="lock open failed"):
+ local_browser_profile.create_local_browser_profile()
+
+
[email protected](os.name == "nt", reason="managed profile sweep is POSIX-only")
+def test_sweep_cannot_reach_protected_sibling_directories(
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ protected_dirs = [tmp_path / "skyvern-browser", tmp_path / "skyvern-browser-profiles"]
+ for protected_dir in protected_dirs:
+ protected_dir.mkdir()
+ (protected_dir / "sentinel").write_text("live data")
+
+ profile = local_browser_profile.create_local_browser_profile()
+ profile.release()
+ monkeypatch.setattr(local_browser_profile, "terminate_local_browser_processes", lambda _path: True)
+
+ assert local_browser_profile.sweep_orphaned_local_browser_profiles() == 1
+ assert not profile.path.exists()
+ for protected_dir in protected_dirs:
+ assert protected_dir.is_dir()
+ assert (protected_dir / "sentinel").read_text() == "live data"
+
+
[email protected](os.name == "nt", reason="identity-checked deletion is POSIX-only")
+def test_deletion_child_rejects_symlink_replacement(tmp_path: Path) -> None:
+ protected = tmp_path / "protected"
+ protected.mkdir()
+ sentinel = protected / "sentinel"
+ sentinel.write_text("live data")
+ profile_path = tmp_path / "profile"
+ profile_path.mkdir()
+ profile_result = profile_path.lstat()
+ profile_path.rmdir()
+ profile_path.symlink_to(protected, target_is_directory=True)
+
+ result = _run_deletion_child(profile_path, (profile_result.st_dev, profile_result.st_ino))
+
+ assert result.returncode == 1
+ assert sentinel.read_text() == "live data"
+
+
[email protected](os.name == "nt", reason="identity-checked deletion is POSIX-only")
+def test_deletion_child_rejects_same_name_directory_replacement(tmp_path: Path) -> None:
+ profile_path = tmp_path / "profile"
+ profile_path.mkdir()
+ profile_result = profile_path.lstat()
+ profile_path.rename(tmp_path / "original-profile")
+ profile_path.mkdir()
+ sentinel = profile_path / "sentinel"
+ sentinel.write_text("replacement data")
+
+ result = _run_deletion_child(profile_path, (profile_result.st_dev, profile_result.st_ino))
+
+ assert result.returncode == 1
+ assert sentinel.read_text() == "replacement data"
+
+
+def test_timed_out_deletion_stops_before_cleanup_returns(
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ profile_path = tmp_path / "profile"
+ profile_path.mkdir()
+ late_mutation = tmp_path / "late-mutation"
+ children: list[subprocess.Popen[bytes]] = []
+ real_popen = subprocess.Popen
+
+ def start_blocking_deleter(*_args: object, **_kwargs: object) -> subprocess.Popen[bytes]:
+ child = real_popen(
+ [
+ sys.executable,
+ "-c",
+ (
+ "import pathlib, signal, sys, time; "
+ "signal.signal(signal.SIGTERM, signal.SIG_IGN); "
+ "time.sleep(0.5); pathlib.Path(sys.argv[1]).touch()"
+ ),
+ str(late_mutation),
+ ],
+ stdin=subprocess.DEVNULL,
+ stdout=subprocess.DEVNULL,
+ stderr=subprocess.DEVNULL,
+ )
+ children.append(child)
+ return child
+
+ monkeypatch.setattr(local_browser_profile, "PROFILE_DELETE_TIMEOUT_SECONDS", 0.1)
+ monkeypatch.setattr(local_browser_profile, "terminate_local_browser_processes", lambda _path: True)
+ monkeypatch.setattr(local_browser_profile.subprocess, "Popen", start_blocking_deleter)
+
+ started_at = time.monotonic()
+ try:
+ assert local_browser_profile.cleanup_local_browser_profile(profile_path) is False
+ assert time.monotonic() - started_at <= 0.2
+ assert children[0].poll() is not None
+ time.sleep(0.55)
+ assert not late_mutation.exists()
+ assert profile_path.exists()
+ finally:
+ for child in children:
+ if child.poll() is None:
+ child.kill()
+ child.wait()
+
+
[email protected](os.name == "nt", reason="managed profile sweep is POSIX-only")
+def test_interrupted_finalization_is_recovered_by_next_sweep(
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ profile = local_browser_profile.create_local_browser_profile()
+ lock_path = _lock_path(profile.path)
+ real_rmdir = os.rmdir
+ failed_once = False
+
+ def fail_first_profile_rmdir(path: os.PathLike[str] | str, *, dir_fd: int | None = None) -> None:
+ nonlocal failed_once
+ if str(path).startswith(".reap-") and dir_fd is not None and not failed_once:
+ failed_once = True
+ raise OSError("forced finalization failure")
+ real_rmdir(path, dir_fd=dir_fd)
+
+ monkeypatch.setattr(local_browser_profile.os, "rmdir", fail_first_profile_rmdir)
+ monkeypatch.setattr(local_browser_profile, "terminate_local_browser_processes", lambda _path: True)
+
+ assert local_browser_profile.cleanup_local_browser_profile(profile) is False
+ assert profile.path.is_dir()
+ assert not lock_path.exists()
+ assert not list(profile._root.glob(".reap-*"))
+
+ assert local_browser_profile.sweep_orphaned_local_browser_profiles() == 1
+ assert not profile.path.exists()
+
+
[email protected](os.name == "nt", reason="managed profile sweep is POSIX-only")
+def test_sweep_reclaims_leftover_empty_tombstone(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ root, _uid, _root_result = local_browser_profile._prepare_profile_root()
+ tombstone = root / ".reap-deadbeef"
+ tombstone.mkdir(mode=0o700)
+ hidden = root / ".hidden"
+ hidden.mkdir(mode=0o700)
+
+ assert local_browser_profile.sweep_orphaned_local_browser_profiles() == 1
+ assert not tombstone.exists()
+ assert hidden.is_dir()
+
+
[email protected](os.name == "nt", reason="managed profile sweep is POSIX-only")
+def test_finalization_removes_verified_profile(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ profile = local_browser_profile.create_local_browser_profile()
+ try:
+ assert local_browser_profile._finalize_managed_profile_removal(profile) is True
+ assert not profile.path.exists()
+ finally:
+ profile.release()
+
+
[email protected](os.name == "nt", reason="managed profile sweep is POSIX-only")
+def test_finalization_rejects_same_name_swap_at_tombstone_rename(
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ profile = local_browser_profile.create_local_browser_profile()
+ original_name = f"{profile.path.name}-original"
+ original_path = profile._root / original_name
+ real_rename = os.rename
+ swapped = False
+
+ def swap_before_tombstone_rename(
+ src: os.PathLike[str] | str,
+ dst: os.PathLike[str] | str,
+ *,
+ src_dir_fd: int | None = None,
+ dst_dir_fd: int | None = None,
+ ) -> None:
+ nonlocal swapped
+ if (
+ src == profile.path.name
+ and str(dst).startswith(".reap-")
+ and src_dir_fd is not None
+ and dst_dir_fd is not None
+ and not swapped
+ ):
+ swapped = True
+ real_rename(src, original_name, src_dir_fd=src_dir_fd, dst_dir_fd=dst_dir_fd)
+ os.mkdir(src, mode=0o700, dir_fd=src_dir_fd)
+ real_rename(src, dst, src_dir_fd=src_dir_fd, dst_dir_fd=dst_dir_fd)
+
+ monkeypatch.setattr(local_browser_profile.os, "rename", swap_before_tombstone_rename)
+ try:
+ assert local_browser_profile._finalize_managed_profile_removal(profile) is False
+ assert not profile.path.exists()
+ assert original_path.is_dir()
+ assert not list(profile._root.glob(".reap-*"))
+ finally:
+ profile.release()
+
+ assert local_browser_profile.sweep_orphaned_local_browser_profiles() == 1
+ assert not original_path.exists()
+
+
[email protected](os.name == "nt", reason="managed profile sweep is POSIX-only")
+def test_finalization_rejects_profile_symlink_swap(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
+ profile = local_browser_profile.create_local_browser_profile()
+ original_profile_path = profile.path.with_name(f"{profile.path.name}-original")
+ protected = tmp_path / "protected"
+ protected.mkdir()
+ sentinel = protected / "sentinel"
+ sentinel.write_text("live data")
+ real_open = os.open
+ swapped = False
+
+ def swap_before_finalize_open(
+ path: os.PathLike[str] | str,
+ flags: int,
+ mode: int = 0o777,
+ *,
+ dir_fd: int | None = None,
+ ) -> int:
+ nonlocal swapped
+ if Path(path) == profile._root and dir_fd is None and not swapped:
+ swapped = True
+ profile.path.rename(original_profile_path)
+ profile.path.symlink_to(protected, target_is_directory=True)
+ return real_open(path, flags, mode, dir_fd=dir_fd)
+
+ monkeypatch.setattr(local_browser_profile.os, "open", swap_before_finalize_open)
+ try:
+ assert local_browser_profile._finalize_managed_profile_removal(profile) is False
+ assert sentinel.read_text() == "live data"
+ finally:
+ profile.release()
+
+
[email protected](os.name == "nt", reason="managed profile sweep is POSIX-only")
+def test_sweep_surfaces_real_child_warning_output(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setenv("TMPDIR", str(tmp_path))
+ unsafe_root = _managed_root(tmp_path)
+ unsafe_root.mkdir(mode=0o700)
+ unsafe_root.chmod(0o755)
+
+ with capture_logs() as logs:
+ local_browser_profile.sweep_local_browser_profiles_with_budget()
+
+ assert any(
+ log.get("return_code") == 0 and "Unsafe local browser profile directory" in log.get("output_tail", "")
+ for log in logs
+ )
+
+
[email protected](os.name == "nt", reason="identity-checked deletion is POSIX-only")
+def test_tombstone_removal_detects_swap_at_terminal_rmdir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ root = tmp_path / "root"
+ root.mkdir(mode=0o700)
+ victim = root / "profile-victim"
+ victim.mkdir(mode=0o700)
+ expected = local_browser_profile._identity(os.lstat(victim))
+
+ real_rmdir = os.rmdir
+
+ def swapping_rmdir(name: str, *, dir_fd: int | None = None) -> None:
+ os.rename(name, ".swapped-aside", src_dir_fd=dir_fd, dst_dir_fd=dir_fd)
+ os.mkdir(name, mode=0o700, dir_fd=dir_fd)
+ real_rmdir(name, dir_fd=dir_fd)
+
+ monkeypatch.setattr(os, "rmdir", swapping_rmdir)
+ root_fd = os.open(root, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW | os.O_CLOEXEC)
+ try:
+ removed = local_browser_profile._remove_verified_directory_via_tombstone(
+ root_fd,
+ victim.name,
+ expected_identity=expected,
+ uid=os.getuid(),
+ )
+ finally:
+ os.close(root_fd)
+
+ assert removed is False
+ survivor = root / ".swapped-aside"
+ assert survivor.exists()
+ assert local_browser_profile._identity(os.lstat(survivor)) == expected
+
+
[email protected](os.name == "nt", reason="flock is POSIX-only")
+def test_create_retries_when_directory_reaped_mid_creation(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(tempfile, "gettempdir", lambda: str(tmp_path))
+ calls = {"count": 0}
+ real_mkdtemp = tempfile.mkdtemp
+
+ def reaped_first_mkdtemp(*args: object, **kwargs: object) -> str:
+ path = real_mkdtemp(*args, **kwargs)
+ calls["count"] += 1
+ if calls["count"] == 1:
+ os.rmdir(path)
+ return path
+
+ monkeypatch.setattr(tempfile, "mkdtemp", reaped_first_mkdtemp)
+ profile = local_browser_profile.create_local_browser_profile()
+ try:
+ assert profile is not None
+ assert calls["count"] == 2
+ assert profile.path.exists()
+ assert profile.revalidate()
+ finally:
+ if profile is not None:
+ profile.release()
+
+
[email protected](os.name == "nt", reason="flock is POSIX-only")
+def test_create_retries_when_sweep_wins_published_lock_race(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(tempfile, "gettempdir", lambda: str(tmp_path))
+ real_lock = local_browser_profile._lock_exclusive_nonblocking
+ calls = {"count": 0}
+
+ def sweep_wins_first(fd: int) -> None:
+ calls["count"] += 1
+ if calls["count"] == 1:
+ raise BlockingIOError
+ real_lock(fd)
+
+ monkeypatch.setattr(local_browser_profile, "_lock_exclusive_nonblocking", sweep_wins_first)
+ profile = local_browser_profile.create_local_browser_profile()
+ try:
+ assert profile is not None
+ assert profile.path.exists()
+ assert calls["count"] >= 2
+ finally:
+ if profile is not None:
+ profile.release()
diff --git a/tests/unit/test_proxy_url_custom.py b/tests/unit/test_proxy_url_custom.py
index 2355f53ed..87bd9fb3b 100644
--- a/tests/unit/test_proxy_url_custom.py
+++ b/tests/unit/test_proxy_url_custom.py
@@ -7,7 +7,7 @@ from skyvern.forge.sdk.db.utils import deserialize_proxy_location, serialize_pro
from skyvern.forge.sdk.schemas.browser_profiles import UpdateBrowserProfileRequest
from skyvern.forge.sdk.schemas.credentials import UpdateCredentialRequest
from skyvern.schemas.runs import GeoTarget, ProxyLocation
-from skyvern.webeye.browser_factory import BrowserContextFactory, _redact_proxy_url, _redact_url_query
+from skyvern.webeye.browser_factory import BrowserContextFactory, _redact_url_query
def test_redact_url_query_strips_presigned_signature() -> None:
@@ -26,33 +26,10 @@ def test_redact_url_query_plain_string_passes_through() -> None:
assert _redact_url_query("not-a-url") == "not-a-url"
-def test_redact_proxy_url_strips_password() -> None:
- assert _redact_proxy_url("http://user:[email protected]:8080") == "http://user:***@proxy.example.com:8080"
-
-
-def test_redact_proxy_url_keeps_username_only() -> None:
- assert _redact_proxy_url("http://[email protected]:8080") == "http://[email protected]:8080"
-
-
-def test_redact_proxy_url_no_creds() -> None:
- assert _redact_proxy_url("http://proxy.example.com:8080") == "http://proxy.example.com:8080"
-
-
-def test_redact_proxy_url_invalid() -> None:
- assert _redact_proxy_url("not-a-url") == "<redacted>"
-
-
-def test_build_browser_args_custom_proxy_url_takes_precedence() -> None:
+def test_build_browser_args_never_sets_proxy() -> None:
args = BrowserContextFactory.build_browser_args(
proxy_location={"url": "http://user:[email protected]:8080"},
)
- assert args["proxy"] == {"server": "http://user:[email protected]:8080"}
-
-
-def test_build_browser_args_invalid_custom_proxy_url_ignored() -> None:
- args = BrowserContextFactory.build_browser_args(
- proxy_location={"url": "not-a-valid-proxy-url"},
- )
assert "proxy" not in args
diff --git a/tests/unit/test_quickstart_command.py b/tests/unit/test_quickstart_command.py
index 113e43e13..f7bdecfed 100644
--- a/tests/unit/test_quickstart_command.py
+++ b/tests/unit/test_quickstart_command.py
@@ -2,12 +2,15 @@ from __future__ import annotations
import json
import os
+import signal
import subprocess
import sys
+import time
import types
from pathlib import Path
import pytest
+from structlog.testing import capture_logs
from typer.testing import CliRunner
import skyvern.cli.quickstart as quickstart_module
@@ -22,12 +25,67 @@ from skyvern.utils.env_paths import (
resolve_backend_env_path,
)
+_BLOCKING_SWEEP_WITH_DESCENDANT = """
+import subprocess
+import sys
+import time
+
+descendant_code = (
+ "import pathlib, signal, sys, time; "
+ "signal.signal(signal.SIGTERM, signal.SIG_IGN); "
+ "pathlib.Path(sys.argv[1]).touch(); "
+ "time.sleep(0.5); pathlib.Path(sys.argv[2]).touch()"
+)
+subprocess.Popen(
+ [sys.executable, "-c", descendant_code, sys.argv[1], sys.argv[2]],
+ stdin=subprocess.DEVNULL,
+ stdout=subprocess.DEVNULL,
+ stderr=subprocess.DEVNULL,
+)
+time.sleep(60)
+"""
+
def _set_home(monkeypatch, home: Path) -> None:
home.mkdir(parents=True, exist_ok=True)
monkeypatch.setenv("HOME", str(home))
+def _patch_minimal_run_mcp_dependencies(monkeypatch, events: list[str], run_mcp_server) -> types.ModuleType:
+ from skyvern.cli import run_commands
+
+ fake_auth = types.ModuleType("skyvern.cli.core.mcp_http_auth")
+ fake_auth.MCPAPIKeyMiddleware = object
+
+ fake_session_manager = types.ModuleType("skyvern.cli.core.session_manager")
+ fake_session_manager.set_stateless_http_mode = lambda _enabled: None
+
+ fake_telemetry = types.ModuleType("skyvern.cli.mcp_tools.telemetry")
+ fake_telemetry.configure_mcp_telemetry_runtime = lambda **_kwargs: None
+
+ fake_mcp_tools = types.ModuleType("skyvern.cli.mcp_tools")
+
+ class FakeMCP:
+ def run(self, *, transport: str, **_: object) -> None:
+ assert transport == "stdio"
+ run_mcp_server()
+
+ fake_mcp_tools.mcp = FakeMCP()
+
+ monkeypatch.setattr(run_commands, "prepare_cli_runtime", lambda **_kwargs: events.append("prepare"))
+ monkeypatch.setattr(run_commands.atexit, "register", lambda _: None)
+ monkeypatch.setattr(run_commands.signal, "signal", lambda *_args: None)
+ monkeypatch.setattr(run_commands, "set_concise_responses", lambda _: None)
+ eof_event = types.SimpleNamespace(set=lambda: None)
+ monkeypatch.setattr(run_commands, "_start_stdin_eof_watcher", lambda: (eof_event, eof_event))
+ monkeypatch.setattr(run_commands, "_cleanup_mcp_resources_blocking", lambda: events.append("cleanup"))
+ monkeypatch.setitem(sys.modules, "skyvern.cli.core.mcp_http_auth", fake_auth)
+ monkeypatch.setitem(sys.modules, "skyvern.cli.core.session_manager", fake_session_manager)
+ monkeypatch.setitem(sys.modules, "skyvern.cli.mcp_tools", fake_mcp_tools)
+ monkeypatch.setitem(sys.modules, "skyvern.cli.mcp_tools.telemetry", fake_telemetry)
+ return run_commands
+
+
def test_backend_env_read_uses_intent_specific_precedence(tmp_path, monkeypatch) -> None:
monkeypatch.chdir(tmp_path)
_set_home(monkeypatch, tmp_path / "home")
@@ -245,6 +303,9 @@ def test_run_mcp_prepares_cloud_env_before_starting_mcp(tmp_path, monkeypatch) -
fake_telemetry.configure_mcp_telemetry_runtime = fake_configure_mcp_telemetry_runtime
+ fake_local_browser_profile = types.ModuleType("skyvern.library.local_browser_profile")
+ fake_local_browser_profile.sweep_local_browser_profiles_with_budget = lambda: events.append("sweep")
+
fake_mcp_tools = types.ModuleType("skyvern.cli.mcp_tools")
class FakeMCP:
@@ -265,6 +326,7 @@ def test_run_mcp_prepares_cloud_env_before_starting_mcp(tmp_path, monkeypatch) -
monkeypatch.setattr(run_commands, "_start_stdin_eof_watcher", lambda: (eof_event, eof_event))
monkeypatch.setattr(run_commands, "_cleanup_mcp_resources_blocking", lambda: events.append("cleanup"))
monkeypatch.setitem(sys.modules, "skyvern.forge.sdk.forge_log", fake_forge_log)
+ monkeypatch.setitem(sys.modules, "skyvern.library.local_browser_profile", fake_local_browser_profile)
monkeypatch.setitem(sys.modules, "skyvern.cli.core.mcp_http_auth", fake_auth)
monkeypatch.setitem(sys.modules, "skyvern.cli.core.session_manager", fake_session_manager)
monkeypatch.setitem(sys.modules, "skyvern.cli.mcp_tools", fake_mcp_tools)
@@ -274,6 +336,7 @@ def test_run_mcp_prepares_cloud_env_before_starting_mcp(tmp_path, monkeypatch) -
assert events == [
"setup_logger",
+ "sweep",
"telemetry:local_cli:stdio",
"stateless:False",
"run:stdio",
@@ -282,6 +345,87 @@ def test_run_mcp_prepares_cloud_env_before_starting_mcp(tmp_path, monkeypatch) -
]
[email protected]("failure_stage", ["import", "call"])
+def test_run_mcp_logs_sweep_failure_and_continues(monkeypatch, failure_stage: str) -> None:
+ events: list[str] = []
+ run_commands = _patch_minimal_run_mcp_dependencies(monkeypatch, events, lambda: events.append("run"))
+
+ fake_local_browser_profile = types.ModuleType("skyvern.library.local_browser_profile")
+ if failure_stage == "import":
+
+ def missing_sweep(name: str) -> object:
+ if name == "sweep_local_browser_profiles_with_budget":
+ raise ImportError("sweep import failed")
+ raise AttributeError(name)
+
+ fake_local_browser_profile.__getattr__ = missing_sweep
+ else:
+
+ def failing_sweep() -> None:
+ raise RuntimeError("sweep call failed")
+
+ fake_local_browser_profile.sweep_local_browser_profiles_with_budget = failing_sweep
+ monkeypatch.setitem(sys.modules, "skyvern.library.local_browser_profile", fake_local_browser_profile)
+
+ with capture_logs() as logs:
+ run_commands.run_mcp()
+
+ assert events == ["prepare", "run", "cleanup"]
+ assert any(log.get("log_level") == "warning" for log in logs)
+
+
+def test_run_mcp_starts_within_sweep_budget_and_stops_blocking_child(tmp_path: Path, monkeypatch) -> None:
+ from skyvern.library import local_browser_profile
+
+ events: list[str] = []
+ children: list[subprocess.Popen[bytes]] = []
+ run_started_at: list[float] = []
+ real_popen = subprocess.Popen
+ budget_seconds = 0.3
+ descendant_ready = tmp_path / "descendant-ready"
+ late_mutation = tmp_path / "late-mutation"
+
+ def start_blocking_child(*_args: object, **kwargs: object) -> subprocess.Popen[bytes]:
+ child = real_popen(
+ [sys.executable, "-c", _BLOCKING_SWEEP_WITH_DESCENDANT, str(descendant_ready), str(late_mutation)],
+ stdin=subprocess.DEVNULL,
+ stdout=subprocess.DEVNULL,
+ stderr=kwargs["stderr"],
+ start_new_session=bool(kwargs.get("start_new_session", False)),
+ )
+ children.append(child)
+ deadline = time.monotonic() + 1
+ while not descendant_ready.exists() and child.poll() is None and time.monotonic() < deadline:
+ time.sleep(0.005)
+ assert descendant_ready.exists()
+ return child
+
+ def run_mcp_server() -> None:
+ run_started_at.append(time.monotonic())
+ assert children and children[0].poll() is not None
+ events.append("run")
+
+ run_commands = _patch_minimal_run_mcp_dependencies(monkeypatch, events, run_mcp_server)
+ monkeypatch.setattr(subprocess, "Popen", start_blocking_child)
+ monkeypatch.setattr(local_browser_profile, "PROFILE_SWEEP_STARTUP_BUDGET_SECONDS", budget_seconds)
+
+ started_at = time.monotonic()
+ try:
+ with capture_logs() as logs:
+ run_commands.run_mcp()
+ finally:
+ for child in children:
+ if child.poll() is None:
+ os.killpg(child.pid, signal.SIGKILL)
+ child.wait()
+
+ assert run_started_at[0] - started_at <= budget_seconds + 0.075
+ assert events == ["prepare", "run", "cleanup"]
+ assert any(log.get("event") == "local_browser_profile_sweep_timed_out" for log in logs)
+ time.sleep(0.55)
+ assert not late_mutation.exists()
+
+
@pytest.mark.parametrize(
"module_name",
[
diff --git a/tests/unit/test_run_commands_cleanup.py b/tests/unit/test_run_commands_cleanup.py
index db91d0838..ecac3abf7 100644
--- a/tests/unit/test_run_commands_cleanup.py
+++ b/tests/unit/test_run_commands_cleanup.py
@@ -1,12 +1,13 @@
from __future__ import annotations
+import shutil
import threading
-from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock, call
import pytest
from skyvern.cli import run_commands
+from skyvern.library import local_browser_profile
@pytest.fixture(autouse=True)
@@ -52,29 +53,35 @@ async def test_cleanup_mcp_resources_closes_auth_db_on_skyvern_close_error(monke
close_auth_db.assert_awaited_once()
-def test_cleanup_mcp_resources_sync_skips_threaded_cleanup_for_local_browser(
[email protected]("deleted", [True, False], ids=["deleted", "deferred"])
+def test_cleanup_mcp_resources_sync_routes_owned_profile_through_shared_cleanup(
monkeypatch: pytest.MonkeyPatch,
+ deleted: bool,
) -> None:
cleanup = AsyncMock()
- fallback = MagicMock()
- rmtree = MagicMock()
+ profile = MagicMock(name="profile")
+ profile_cleanup = MagicMock(return_value=deleted)
+ terminate = MagicMock()
+ rmtree = MagicMock(side_effect=AssertionError("run_commands must not delete profiles directly"))
thread = MagicMock()
monkeypatch.setattr(run_commands, "_cleanup_mcp_resources", cleanup)
monkeypatch.setattr(
run_commands,
"_current_local_browser_identity",
- lambda: (40101, "/tmp/skyvern-browser-owned", True),
+ lambda: ("/tmp/skyvern-browser-owned", True, profile),
)
- monkeypatch.setattr(run_commands, "_kill_local_browser_process_tree", fallback)
- monkeypatch.setattr(run_commands.shutil, "rmtree", rmtree)
+ monkeypatch.setattr(local_browser_profile, "cleanup_local_browser_profile", profile_cleanup)
+ monkeypatch.setattr(local_browser_profile, "terminate_local_browser_processes", terminate)
+ monkeypatch.setattr(shutil, "rmtree", rmtree)
monkeypatch.setattr(run_commands.threading, "Thread", thread)
run_commands._cleanup_mcp_resources_sync()
cleanup.assert_not_awaited()
thread.assert_not_called()
- fallback.assert_called_once_with(40101, "/tmp/skyvern-browser-owned")
- rmtree.assert_called_once_with("/tmp/skyvern-browser-owned", ignore_errors=True)
+ profile_cleanup.assert_called_once_with(profile)
+ terminate.assert_not_called()
+ rmtree.assert_not_called()
assert run_commands._mcp_cleanup_done is True
@@ -129,23 +136,30 @@ def test_cleanup_mcp_resources_blocking_ignores_reentrant_call(monkeypatch: pyte
cleanup.assert_awaited_once_with()
-def test_cleanup_mcp_resources_sync_preserves_explicit_user_data_dir(monkeypatch: pytest.MonkeyPatch) -> None:
[email protected]("terminated", [True, False], ids=["terminated", "termination_deferred"])
+def test_cleanup_mcp_resources_sync_preserves_explicit_user_data_dir(
+ monkeypatch: pytest.MonkeyPatch,
+ terminated: bool,
+) -> None:
cleanup = AsyncMock()
- fallback = MagicMock()
- rmtree = MagicMock()
+ profile_cleanup = MagicMock()
+ terminate = MagicMock(return_value=terminated)
+ rmtree = MagicMock(side_effect=AssertionError("explicit user data must not be deleted"))
monkeypatch.setattr(run_commands, "_cleanup_mcp_resources", cleanup)
monkeypatch.setattr(
run_commands,
"_current_local_browser_identity",
- lambda: (40101, "/tmp/skyvern-browser-explicit", False),
+ lambda: ("/tmp/skyvern-browser-explicit", False, None),
)
- monkeypatch.setattr(run_commands, "_kill_local_browser_process_tree", fallback)
- monkeypatch.setattr(run_commands.shutil, "rmtree", rmtree)
+ monkeypatch.setattr(local_browser_profile, "cleanup_local_browser_profile", profile_cleanup)
+ monkeypatch.setattr(local_browser_profile, "terminate_local_browser_processes", terminate)
+ monkeypatch.setattr(shutil, "rmtree", rmtree)
run_commands._cleanup_mcp_resources_sync()
cleanup.assert_not_awaited()
- fallback.assert_called_once_with(40101, "/tmp/skyvern-browser-explicit")
+ profile_cleanup.assert_not_called()
+ terminate.assert_called_once_with("/tmp/skyvern-browser-explicit")
rmtree.assert_not_called()
@@ -168,15 +182,28 @@ def test_stdin_eof_watcher_allows_native_clean_return(monkeypatch: pytest.Monkey
force_exit.assert_not_called()
-def test_stdin_eof_watcher_force_exits_after_bounded_local_fallback(monkeypatch: pytest.MonkeyPatch) -> None:
[email protected]("deleted", [True, False], ids=["deleted", "deferred"])
+def test_stdin_eof_watcher_force_exits_after_shared_profile_cleanup(
+ monkeypatch: pytest.MonkeyPatch,
+ deleted: bool,
+) -> None:
+ events: list[str] = []
poller = MagicMock()
poller.poll.return_value = [(123, run_commands.select.POLLHUP)]
+ profile = MagicMock(name="profile")
+ profile_cleanup = MagicMock(side_effect=lambda _profile: events.append("cleanup") or deleted)
+ terminate = MagicMock()
+ rmtree = MagicMock(side_effect=AssertionError("run_commands must not delete profiles directly"))
+ force_exit = MagicMock(side_effect=lambda _code: events.append("exit"))
monkeypatch.setattr(run_commands.select, "poll", lambda: poller)
- monkeypatch.setattr(run_commands, "_current_local_browser_identity", lambda: (40101, "/tmp/owned", True))
- fallback, force_exit = MagicMock(), MagicMock()
- rmtree = MagicMock()
- monkeypatch.setattr(run_commands, "_kill_local_browser_process_tree", fallback)
- monkeypatch.setattr(run_commands.shutil, "rmtree", rmtree)
+ monkeypatch.setattr(
+ run_commands,
+ "_current_local_browser_identity",
+ lambda: ("/tmp/owned", True, profile),
+ )
+ monkeypatch.setattr(local_browser_profile, "cleanup_local_browser_profile", profile_cleanup)
+ monkeypatch.setattr(local_browser_profile, "terminate_local_browser_processes", terminate)
+ monkeypatch.setattr(shutil, "rmtree", rmtree)
run_commands._watch_stdin_eof(
threading.Event(),
@@ -188,51 +215,23 @@ def test_stdin_eof_watcher_force_exits_after_bounded_local_fallback(monkeypatch:
shutdown_timeout=0,
)
- fallback.assert_called_once_with(40101, "/tmp/owned")
- rmtree.assert_called_once_with("/tmp/owned", ignore_errors=True)
+ profile_cleanup.assert_called_once_with(profile)
+ terminate.assert_not_called()
+ rmtree.assert_not_called()
force_exit.assert_called_once_with(0)
+ assert events == ["cleanup", "exit"]
assert run_commands._mcp_eof_shutdown_requested is True
-def test_local_browser_process_matching_is_scoped_to_instance(monkeypatch: pytest.MonkeyPatch) -> None:
- unreadable = SimpleNamespace(pid=10, info=None)
- matching_dir = SimpleNamespace(pid=11, info={"cmdline": ["chrome", "--user-data-dir=/tmp/owned"]})
- unrelated = SimpleNamespace(pid=12, info={"cmdline": ["chrome", "--user-data-dir=/tmp/other"]})
- get_pids_on_port = MagicMock(side_effect=AssertionError("port-only matching is unsafe"))
- monkeypatch.setattr(run_commands, "get_pids_on_port", get_pids_on_port)
- monkeypatch.setattr(run_commands.psutil, "process_iter", lambda _attrs: [unreadable, matching_dir, unrelated])
-
- assert {process.pid for process in run_commands._find_local_browser_processes(40101, "/tmp/owned")} == {11}
- assert run_commands._find_local_browser_processes(40101, None) == []
- get_pids_on_port.assert_not_called()
-
-
def test_mcp_eof_shutdown_ceiling_exceeds_worst_case_cleanup() -> None:
# The EOF watcher's os._exit(0) preempts cleanup unconditionally, so this must cover the cloud path.
assert run_commands._MCP_EOF_SHUTDOWN_TIMEOUT_SECONDS > (
- run_commands._MCP_GRACEFUL_CLEANUP_TIMEOUT_SECONDS + run_commands._MCP_PROCESS_KILL_TIMEOUT_SECONDS
+ run_commands._MCP_GRACEFUL_CLEANUP_TIMEOUT_SECONDS
+ + local_browser_profile.PROCESS_KILL_TIMEOUT_SECONDS
+ + local_browser_profile.PROFILE_DELETE_TIMEOUT_SECONDS
)
-def test_local_browser_fallback_kills_only_matching_process_tree(monkeypatch: pytest.MonkeyPatch) -> None:
- child = MagicMock(pid=20)
- root = MagicMock(pid=10)
- captured_before_graceful_close = MagicMock(pid=30)
- root.children.return_value = [child]
- monkeypatch.setattr(run_commands, "_find_local_browser_processes", lambda _port, _dir: [root])
- wait_procs = MagicMock(return_value=([], []))
- monkeypatch.setattr(run_commands.psutil, "wait_procs", wait_procs)
-
- run_commands._kill_local_browser_process_tree(
- 40101, "/tmp/skyvern-browser-owned", known_processes=[captured_before_graceful_close]
- )
-
- root.kill.assert_called_once_with()
- child.kill.assert_called_once_with()
- captured_before_graceful_close.kill.assert_called_once_with()
- wait_procs.assert_called_once()
-
-
@pytest.mark.parametrize(
("signum", "eof_shutdown", "expected_exit_code"),
[
diff --git a/tests/unit/test_sdk_run_metadata.py b/tests/unit/test_sdk_run_metadata.py
index 84c0e4f88..7ce57a0d7 100644
--- a/tests/unit/test_sdk_run_metadata.py
+++ b/tests/unit/test_sdk_run_metadata.py
@@ -1,14 +1,17 @@
+import asyncio
import json
from datetime import UTC, datetime
from pathlib import Path
from types import SimpleNamespace
-from unittest.mock import AsyncMock
+from unittest.mock import AsyncMock, Mock
import httpx
import pytest
+from structlog.testing import capture_logs
from skyvern.client import AsyncSkyvern, Skyvern
from skyvern.client.types.workflow_run_request_output import WorkflowRunRequestOutput as WorkflowRunRequest
+from skyvern.library import local_browser_profile
from skyvern.library.constants import DEFAULT_CDP_PORT
from skyvern.library.skyvern import Skyvern as LibrarySkyvern
@@ -17,6 +20,25 @@ def _make_browser(_client: object, _context: object, **kwargs: object) -> Simple
return SimpleNamespace(**kwargs)
+def _make_local_profile(path: Path, events: list[str], label: str = "") -> Mock:
+ path.mkdir(exist_ok=True)
+ prefix = f"{label}:" if label else ""
+ profile = Mock(spec=local_browser_profile.LocalBrowserProfile)
+ profile.path = path
+ profile.revalidate.side_effect = lambda: events.append(f"{prefix}revalidate") or True
+ profile.release.side_effect = lambda: events.append(f"{prefix}release")
+ return profile
+
+
+def _cleanup_mock(events: list[str]) -> Mock:
+ def cleanup(profile: Mock) -> bool:
+ events.append("cleanup")
+ profile.release()
+ return True
+
+ return Mock(side_effect=cleanup)
+
+
def test_workflow_run_request_accepts_run_metadata() -> None:
request = WorkflowRunRequest(workflow_id="wpid_123", run_metadata={"customer": "acme"})
@@ -58,17 +80,33 @@ async def test_skyvern_launch_local_browser_uses_unique_anonymous_instance_defau
tmp_path: Path,
) -> None:
client = LibrarySkyvern(base_url="https://api.example.test", api_key="test-key")
+ events: list[str] = []
assigned_ports = iter((40101, 40102))
+ profiles: list[Mock] = []
+ for name in ("one", "two"):
+ profiles.append(_make_local_profile(tmp_path / f"profile-{name}", events, name))
+ profile_iterator = iter(profiles)
+
+ def create_profile() -> Mock:
+ profile = next(profile_iterator)
+ events.append(f"{profile.path.name.removeprefix('profile-')}:create")
+ return profile
async def launch_persistent_context(**kwargs: object) -> object:
user_data_dir = Path(str(kwargs["user_data_dir"]))
- (user_data_dir / "DevToolsActivePort").write_text(f"{next(assigned_ports)}\n/devtools/browser/test\n")
+ events.append(f"{user_data_dir.name.removeprefix('profile-')}:launch")
return object()
+ async def read_port(user_data_dir: Path) -> int:
+ events.append(f"{user_data_dir.name.removeprefix('profile-')}:read-port")
+ return next(assigned_ports)
+
launch = AsyncMock(side_effect=launch_persistent_context)
playwright = SimpleNamespace(chromium=SimpleNamespace(launch_persistent_context=launch))
+ create = Mock(side_effect=create_profile)
monkeypatch.setattr(client, "_get_playwright", AsyncMock(return_value=playwright))
- monkeypatch.setattr("skyvern.library.skyvern.tempfile.gettempdir", lambda: str(tmp_path))
+ monkeypatch.setattr(local_browser_profile, "create_local_browser_profile", create)
+ monkeypatch.setattr("skyvern.library.skyvern._read_devtools_active_port", AsyncMock(side_effect=read_port))
instance_dirs = iter((tmp_path / "skyvern-browser-one", tmp_path / "skyvern-browser-two"))
def make_instance_dir(*, prefix: str) -> str:
@@ -86,7 +124,18 @@ async def test_skyvern_launch_local_browser_uses_unique_anonymous_instance_defau
first_call, second_call = launch.await_args_list
first_dir = Path(first_call.kwargs["user_data_dir"])
second_dir = Path(second_call.kwargs["user_data_dir"])
- assert first_dir != second_dir
+ assert [first_dir, second_dir] == [profile.path for profile in profiles]
+ assert events == [
+ "one:create",
+ "one:revalidate",
+ "one:launch",
+ "one:read-port",
+ "two:create",
+ "two:revalidate",
+ "two:launch",
+ "two:read-port",
+ ]
+ assert create.call_count == 2
assert [call.kwargs["args"] for call in (first_call, second_call)] == [["--remote-debugging-port=0"]] * 2
assert first.browser_address == "http://localhost:40101"
assert second.browser_address == "http://localhost:40102"
@@ -96,6 +145,46 @@ async def test_skyvern_launch_local_browser_uses_unique_anonymous_instance_defau
assert second.local_user_data_dir == str(second_dir)
assert first.local_user_data_dir_owned is True
assert second.local_user_data_dir_owned is True
+ assert first.local_browser_profile is profiles[0]
+ assert second.local_browser_profile is profiles[1]
+
+
[email protected]("failure_stage", ["lock_race", "identity_changed"])
+async def test_skyvern_launch_local_browser_aborts_before_chromium_when_profile_creation_is_unsafe(
+ monkeypatch: pytest.MonkeyPatch,
+ tmp_path: Path,
+ failure_stage: str,
+) -> None:
+ client = LibrarySkyvern(base_url="https://api.example.test", api_key="test-key")
+ launch = AsyncMock(side_effect=AssertionError("Chromium must not launch"))
+ playwright = SimpleNamespace(chromium=SimpleNamespace(launch_persistent_context=launch))
+ profile = SimpleNamespace(
+ path=tmp_path / "skyvern-browser-race",
+ revalidate=Mock(return_value=failure_stage != "identity_changed"),
+ release=Mock(),
+ )
+ create_profile = Mock(
+ side_effect=BlockingIOError("profile lock is held") if failure_stage == "lock_race" else None,
+ return_value=profile,
+ )
+ cleanup = Mock(side_effect=lambda candidate: candidate.release())
+
+ monkeypatch.setattr(client, "_get_playwright", AsyncMock(return_value=playwright))
+ monkeypatch.setattr(local_browser_profile, "create_local_browser_profile", create_profile)
+ monkeypatch.setattr(local_browser_profile, "cleanup_local_browser_profile", cleanup)
+
+ with pytest.raises(BlockingIOError if failure_stage == "lock_race" else RuntimeError):
+ await client.launch_local_browser()
+
+ create_profile.assert_called_once_with()
+ launch.assert_not_awaited()
+ if failure_stage == "identity_changed":
+ profile.revalidate.assert_called_once_with()
+ cleanup.assert_called_once_with(profile)
+ profile.release.assert_called_once_with()
+ else:
+ cleanup.assert_not_called()
@pytest.mark.asyncio
@@ -137,59 +226,403 @@ async def test_skyvern_launch_local_browser_preserves_explicit_config_defaults(
@pytest.mark.asyncio
[email protected]("failure_stage", ["launch", "port_read"])
-async def test_skyvern_launch_local_browser_removes_owned_dir_on_setup_failure(
[email protected]("failure_stage", ["launch", "port_read", "constructor"])
+async def test_skyvern_launch_local_browser_rolls_back_setup_failure(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
failure_stage: str,
) -> None:
client = LibrarySkyvern(base_url="https://api.example.test", api_key="test-key")
- browser_context = SimpleNamespace(close=AsyncMock())
- launch_fails = failure_stage == "launch"
- launch = AsyncMock(
- side_effect=RuntimeError("chromium failed to start") if launch_fails else None,
- return_value=None if launch_fails else browser_context,
- )
+ events: list[str] = []
+ profile = _make_local_profile(tmp_path / "profile-failed", events)
+
+ async def close_context() -> None:
+ events.append("context-close")
+
+ browser_context = SimpleNamespace(close=AsyncMock(side_effect=close_context))
+
+ async def launch_context(**_kwargs: object) -> object:
+ events.append("launch")
+ if failure_stage == "launch":
+ raise RuntimeError("chromium failed to start")
+ return browser_context
+
+ async def read_port(_user_data_dir: Path) -> int:
+ events.append("port-read")
+ if failure_stage == "port_read":
+ raise RuntimeError("missing port")
+ return 40101
+
+ def construct_browser(*args: object, **kwargs: object) -> SimpleNamespace:
+ events.append("construct")
+ if failure_stage == "constructor":
+ raise RuntimeError("wrapper failed")
+ return _make_browser(*args, **kwargs)
+
+ launch = AsyncMock(side_effect=launch_context)
playwright = SimpleNamespace(chromium=SimpleNamespace(launch_persistent_context=launch))
- user_data_dir = tmp_path / "skyvern-browser-failed"
- user_data_dir.mkdir()
+ create_profile = Mock(side_effect=lambda: events.append("create") or profile)
+ cleanup = _cleanup_mock(events)
+ constructor = Mock(side_effect=construct_browser)
+ fallback_dir = tmp_path / "skyvern-browser-failed"
monkeypatch.setattr(client, "_get_playwright", AsyncMock(return_value=playwright))
- monkeypatch.setattr("skyvern.library.skyvern.tempfile.gettempdir", lambda: str(tmp_path))
- monkeypatch.setattr("skyvern.library.skyvern.tempfile.mkdtemp", lambda *, prefix: str(user_data_dir))
+ monkeypatch.setattr(local_browser_profile, "create_local_browser_profile", create_profile)
+ monkeypatch.setattr(local_browser_profile, "cleanup_local_browser_profile", cleanup)
+ monkeypatch.setattr("skyvern.library.skyvern.tempfile.mkdtemp", lambda *, prefix: str(fallback_dir))
+ monkeypatch.setattr("skyvern.library.skyvern._read_devtools_active_port", AsyncMock(side_effect=read_port))
+ monkeypatch.setattr("skyvern.library.skyvern_browser.SkyvernBrowser", constructor)
+
+ expected_error = {
+ "launch": "chromium failed to start",
+ "port_read": "missing port",
+ "constructor": "wrapper failed",
+ }[failure_stage]
+ with pytest.raises(RuntimeError, match=expected_error):
+ await client.launch_local_browser()
+
+ if failure_stage == "launch":
+ browser_context.close.assert_not_awaited()
+ else:
+ browser_context.close.assert_awaited_once_with()
+ cleanup.assert_called_once_with(profile)
+ profile.release.assert_called_once_with()
+ expected_events = ["create", "revalidate", "launch"]
+ if failure_stage != "launch":
+ expected_events.append("port-read")
+ if failure_stage == "constructor":
+ expected_events.append("construct")
+ assert constructor.call_args.kwargs["local_browser_profile"] is profile
+ if failure_stage != "launch":
+ expected_events.append("context-close")
+ expected_events.extend(["cleanup", "release"])
+ assert events == expected_events
+
+
+async def test_skyvern_launch_local_browser_preserves_port_error_when_context_close_fails(
+ monkeypatch: pytest.MonkeyPatch,
+ tmp_path: Path,
+) -> None:
+ client = LibrarySkyvern(base_url="https://api.example.test", api_key="test-key")
+ events: list[str] = []
+ profile = _make_local_profile(tmp_path / "profile-failed", events)
+ browser_context = SimpleNamespace(close=AsyncMock(side_effect=RuntimeError("rollback close failed")))
+ port_error = RuntimeError("original port failure")
+ playwright = SimpleNamespace(
+ chromium=SimpleNamespace(launch_persistent_context=AsyncMock(return_value=browser_context))
+ )
+ cleanup = _cleanup_mock(events)
+
+ monkeypatch.setattr(client, "_get_playwright", AsyncMock(return_value=playwright))
+ monkeypatch.setattr(local_browser_profile, "create_local_browser_profile", Mock(return_value=profile))
+ monkeypatch.setattr(local_browser_profile, "cleanup_local_browser_profile", cleanup)
+ monkeypatch.setattr("skyvern.library.skyvern._read_devtools_active_port", AsyncMock(side_effect=port_error))
+
+ with capture_logs() as logs:
+ with pytest.raises(RuntimeError) as exc_info:
+ await client.launch_local_browser()
+
+ assert exc_info.value is port_error
+ browser_context.close.assert_awaited_once_with()
+ cleanup.assert_called_once_with(profile)
+ profile.release.assert_called_once_with()
+ assert any(log.get("event") == "local_browser_context_close_rollback_failed" for log in logs)
+
+
+async def test_skyvern_launch_local_browser_preserves_launch_error_when_cleanup_internals_fail(
+ monkeypatch: pytest.MonkeyPatch,
+ tmp_path: Path,
+) -> None:
+ client = LibrarySkyvern(base_url="https://api.example.test", api_key="test-key")
+ monkeypatch.setattr(local_browser_profile.tempfile, "tempdir", str(tmp_path))
+ profile = local_browser_profile.create_local_browser_profile()
+ launch = AsyncMock(side_effect=RuntimeError("original launch failure"))
+ playwright = SimpleNamespace(chromium=SimpleNamespace(launch_persistent_context=launch))
+ monkeypatch.setattr(client, "_get_playwright", AsyncMock(return_value=playwright))
+ monkeypatch.setattr(local_browser_profile, "create_local_browser_profile", Mock(return_value=profile))
+ monkeypatch.setattr(local_browser_profile, "terminate_local_browser_processes", Mock(return_value=True))
monkeypatch.setattr(
- "skyvern.library.skyvern._read_devtools_active_port",
- AsyncMock(side_effect=RuntimeError("missing port")),
+ local_browser_profile,
+ "_remove_profile_directory_bounded",
+ Mock(side_effect=OSError("cleanup spawn failure")),
)
- with pytest.raises(RuntimeError, match="chromium failed to start" if launch_fails else "missing port"):
+ with pytest.raises(RuntimeError, match="original launch failure"):
await client.launch_local_browser()
- if launch_fails:
- browser_context.close.assert_not_awaited()
+ assert profile._lock_fd is None
+
+
[email protected](("managed", "close_count"), [(True, 1), (True, 2), (False, 2)])
+async def test_skyvern_browser_close_uses_shared_cleanup_exactly_once(
+ monkeypatch: pytest.MonkeyPatch,
+ tmp_path: Path,
+ managed: bool,
+ close_count: int,
+) -> None:
+ from skyvern.library.skyvern_browser import SkyvernBrowser
+
+ events: list[str] = []
+ user_data_dir = tmp_path / ("managed" if managed else "explicit")
+ profile = _make_local_profile(user_data_dir, events)
+
+ async def close_context() -> None:
+ events.append("context-close")
+
+ browser_context = SimpleNamespace(close=AsyncMock(side_effect=close_context))
+ cleanup = _cleanup_mock(events)
+ monkeypatch.setattr("skyvern.library.skyvern_browser.BrowserContext.__init__", lambda *_args: None)
+ monkeypatch.setattr(local_browser_profile, "cleanup_local_browser_profile", cleanup)
+ browser = SkyvernBrowser(
+ Mock(),
+ browser_context,
+ local_user_data_dir=str(user_data_dir),
+ local_user_data_dir_owned=managed,
+ local_browser_profile=profile if managed else None,
+ )
+
+ for _ in range(close_count):
+ await browser.close()
+
+ browser_context.close.assert_awaited_once_with()
+ if managed:
+ cleanup.assert_called_once_with(profile)
+ profile.release.assert_called_once_with()
+ assert events == ["context-close", "cleanup", "release"]
else:
- browser_context.close.assert_awaited_once_with()
- assert not user_data_dir.exists()
+ cleanup.assert_not_called()
+ profile.release.assert_not_called()
+ assert events == ["context-close"]
@pytest.mark.asyncio
[email protected]("owned", [True, False])
-async def test_skyvern_browser_close_removes_only_owned_user_data_dir(tmp_path: Path, owned: bool) -> None:
+async def test_skyvern_browser_close_warns_when_profile_cleanup_is_deferred(
+ monkeypatch: pytest.MonkeyPatch,
+ tmp_path: Path,
+) -> None:
from skyvern.library.skyvern_browser import SkyvernBrowser
- user_data_dir = tmp_path / ("owned" if owned else "explicit")
- user_data_dir.mkdir()
+ user_data_dir = tmp_path / "managed"
+ profile = _make_local_profile(user_data_dir, [])
browser_context = SimpleNamespace(close=AsyncMock())
- browser = object.__new__(SkyvernBrowser)
- object.__setattr__(browser, "_browser_context", browser_context)
- object.__setattr__(browser, "_browser_session_id", None)
- object.__setattr__(browser, "_local_user_data_dir", str(user_data_dir))
- object.__setattr__(browser, "_local_user_data_dir_owned", owned)
+ cleanup = Mock(return_value=False)
+ monkeypatch.setattr("skyvern.library.skyvern_browser.BrowserContext.__init__", lambda *_args: None)
+ monkeypatch.setattr(local_browser_profile, "cleanup_local_browser_profile", cleanup)
+ browser = SkyvernBrowser(
+ Mock(),
+ browser_context,
+ local_user_data_dir=str(user_data_dir),
+ local_user_data_dir_owned=True,
+ local_browser_profile=profile,
+ )
+
+ with capture_logs() as logs:
+ await browser.close()
+
+ browser_context.close.assert_awaited_once_with()
+ cleanup.assert_called_once_with(profile)
+ assert any(log.get("event") == "local_browser_profile_cleanup_deferred" for log in logs)
+
+
+async def test_skyvern_browser_concurrent_close_shares_context_failure(monkeypatch: pytest.MonkeyPatch) -> None:
+ from skyvern.library.skyvern_browser import SkyvernBrowser
+
+ entered = asyncio.Event()
+ release = asyncio.Event()
+ close_error = RuntimeError("close failed")
+
+ async def close_context() -> None:
+ entered.set()
+ await release.wait()
+ raise close_error
+
+ skyvern = SimpleNamespace(close_browser_session=AsyncMock())
+ browser_context = SimpleNamespace(close=AsyncMock(side_effect=close_context))
+ monkeypatch.setattr("skyvern.library.skyvern_browser.BrowserContext.__init__", lambda *_args: None)
+ browser = SkyvernBrowser(skyvern, browser_context, browser_session_id="pbs_shared")
+
+ first = asyncio.create_task(browser.close())
+ await entered.wait()
+ second = asyncio.create_task(browser.close())
+ await asyncio.sleep(0)
+ assert not second.done()
+ release.set()
+ results = await asyncio.gather(first, second, return_exceptions=True)
+
+ assert all(result is close_error for result in results)
+ browser_context.close.assert_awaited_once_with()
+ skyvern.close_browser_session.assert_not_awaited()
+ assert browser._close_task is None
+ assert browser._closed is False
+
+async def test_skyvern_browser_concurrent_close_runs_sequence_once(monkeypatch: pytest.MonkeyPatch) -> None:
+ from skyvern.library.skyvern_browser import SkyvernBrowser
+
+ entered = asyncio.Event()
+ release = asyncio.Event()
+
+ async def close_context() -> None:
+ entered.set()
+ await release.wait()
+
+ skyvern = SimpleNamespace(close_browser_session=AsyncMock())
+ browser_context = SimpleNamespace(close=AsyncMock(side_effect=close_context))
+ monkeypatch.setattr("skyvern.library.skyvern_browser.BrowserContext.__init__", lambda *_args: None)
+ browser = SkyvernBrowser(skyvern, browser_context, browser_session_id="pbs_shared")
+
+ first = asyncio.create_task(browser.close())
+ await entered.wait()
+ second = asyncio.create_task(browser.close())
+ await asyncio.sleep(0)
+ assert not second.done()
+ release.set()
+ await asyncio.gather(first, second)
await browser.close()
browser_context.close.assert_awaited_once_with()
- assert user_data_dir.exists() is not owned
+ skyvern.close_browser_session.assert_awaited_once_with("pbs_shared")
+ assert browser._close_task is None
+ assert browser._closed is True
+
+
+async def test_skyvern_browser_cancelled_waiter_does_not_cancel_shared_close(monkeypatch: pytest.MonkeyPatch) -> None:
+ from skyvern.library.skyvern_browser import SkyvernBrowser
+
+ entered = asyncio.Event()
+ release = asyncio.Event()
+
+ async def close_context() -> None:
+ entered.set()
+ await release.wait()
+
+ skyvern = SimpleNamespace(close_browser_session=AsyncMock())
+ browser_context = SimpleNamespace(close=AsyncMock(side_effect=close_context))
+ monkeypatch.setattr("skyvern.library.skyvern_browser.BrowserContext.__init__", lambda *_args: None)
+ browser = SkyvernBrowser(skyvern, browser_context, browser_session_id="pbs_cancel_waiter")
+
+ first = asyncio.create_task(browser.close())
+ await entered.wait()
+ second = asyncio.create_task(browser.close())
+ await asyncio.sleep(0)
+ second.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await second
+ release.set()
+ await first
+
+ browser_context.close.assert_awaited_once_with()
+ skyvern.close_browser_session.assert_awaited_once_with("pbs_cancel_waiter")
+ assert browser._closed is True
+ assert browser._close_task is None
+
+
+async def test_skyvern_browser_cancelled_creator_leaves_shared_close_running(monkeypatch: pytest.MonkeyPatch) -> None:
+ from skyvern.library.skyvern_browser import SkyvernBrowser
+
+ entered = asyncio.Event()
+ release = asyncio.Event()
+
+ async def close_context() -> None:
+ entered.set()
+ await release.wait()
+
+ skyvern = SimpleNamespace(close_browser_session=AsyncMock())
+ browser_context = SimpleNamespace(close=AsyncMock(side_effect=close_context))
+ monkeypatch.setattr("skyvern.library.skyvern_browser.BrowserContext.__init__", lambda *_args: None)
+ browser = SkyvernBrowser(skyvern, browser_context, browser_session_id="pbs_cancel_creator")
+
+ first = asyncio.create_task(browser.close())
+ await entered.wait()
+ second = asyncio.create_task(browser.close())
+ await asyncio.sleep(0)
+ first.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await first
+ release.set()
+ await second
+
+ browser_context.close.assert_awaited_once_with()
+ skyvern.close_browser_session.assert_awaited_once_with("pbs_cancel_creator")
+ assert browser._closed is True
+ assert browser._close_task is None
+
+
+async def test_skyvern_browser_close_retries_after_context_failure(monkeypatch: pytest.MonkeyPatch) -> None:
+ from skyvern.library.skyvern_browser import SkyvernBrowser
+
+ skyvern = SimpleNamespace(close_browser_session=AsyncMock())
+ browser_context = SimpleNamespace(close=AsyncMock(side_effect=[RuntimeError("close failed"), None]))
+ monkeypatch.setattr("skyvern.library.skyvern_browser.BrowserContext.__init__", lambda *_args: None)
+ browser = SkyvernBrowser(
+ skyvern,
+ browser_context,
+ browser_session_id="pbs_retry",
+ )
+
+ with pytest.raises(RuntimeError, match="close failed"):
+ await browser.close()
+
+ assert browser._close_task is None
+ assert browser._closed is False
+ await browser.close()
+
+ assert browser_context.close.await_count == 2
+ skyvern.close_browser_session.assert_awaited_once_with("pbs_retry")
+ assert browser._close_task is None
+ assert browser._closed is True
+
+
+async def test_skyvern_launch_local_browser_windows_bypasses_managed_profiles_before_posix_calls(
+ monkeypatch: pytest.MonkeyPatch,
+ tmp_path: Path,
+) -> None:
+ client = LibrarySkyvern(base_url="https://api.example.test", api_key="test-key")
+ user_data_dir = tmp_path / "skyvern-browser-windows"
+ launch = AsyncMock(return_value=object())
+ playwright = SimpleNamespace(chromium=SimpleNamespace(launch_persistent_context=launch))
+ getuid = Mock(side_effect=AssertionError("os.getuid must not run on Windows"))
+ acquire_lock = Mock(side_effect=AssertionError("flock path must not run on Windows"))
+ create_profile = Mock(wraps=local_browser_profile.create_local_browser_profile)
+
+ def make_instance_dir(*, prefix: str) -> str:
+ assert prefix == "skyvern-browser-"
+ user_data_dir.mkdir()
+ return str(user_data_dir)
+
+ mkdtemp = Mock(side_effect=make_instance_dir)
+ monkeypatch.setattr(client, "_get_playwright", AsyncMock(return_value=playwright))
+ monkeypatch.setattr(local_browser_profile.sys, "platform", "win32")
+ monkeypatch.setattr(local_browser_profile.os, "getuid", getuid)
+ monkeypatch.setattr(local_browser_profile, "_lock_exclusive_nonblocking", acquire_lock)
+ monkeypatch.setattr(local_browser_profile, "create_local_browser_profile", create_profile)
+ monkeypatch.setattr("skyvern.library.skyvern.tempfile.mkdtemp", mkdtemp)
+ monkeypatch.setattr("skyvern.library.skyvern._read_devtools_active_port", AsyncMock(return_value=40123))
+ monkeypatch.setattr("skyvern.library.skyvern_browser.SkyvernBrowser", _make_browser)
+
+ browser = await client.launch_local_browser(headless=True)
+
+ create_profile.assert_called_once_with()
+ getuid.assert_not_called()
+ acquire_lock.assert_not_called()
+ mkdtemp.assert_called_once_with(prefix="skyvern-browser-")
+ assert launch.await_args.kwargs == {
+ "user_data_dir": str(user_data_dir),
+ "headless": True,
+ "args": ["--remote-debugging-port=0"],
+ }
+ assert browser.local_cdp_port == 40123
+ assert browser.local_user_data_dir == str(user_data_dir)
+ assert browser.local_user_data_dir_owned is True
def test_run_workflow_sends_run_metadata() -> None: