Skip to content

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

fix(mcp)!: Rename all browser based tools and move session validation inside session look up function

Karim shoairc8cc8977ff54
diff --git a/agent-skill/Scrapling-Skill.zip b/agent-skill/Scrapling-Skill.zip
index f125887..c8fbe3e 100644
Binary files a/agent-skill/Scrapling-Skill.zip and b/agent-skill/Scrapling-Skill.zip differ
diff --git a/agent-skill/Scrapling-Skill/SKILL.md b/agent-skill/Scrapling-Skill/SKILL.md
index 73b085d..5d2b9ca 100644
--- a/agent-skill/Scrapling-Skill/SKILL.md
+++ b/agent-skill/Scrapling-Skill/SKILL.md
@@ -255,7 +255,7 @@ data = page.css('.quote .text::text').getall()
### Shadow DOM
-Use `pierce_shadow=True` with dynamic or stealth Python fetchers and sessions to include open shadow roots. It defaults to `False` and can be overridden per request. For MCP, pass `pierce_shadow=true` to `fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`, or each `session_fetch` call.
+Use `pierce_shadow=True` with dynamic or stealth Python fetchers and sessions to include open shadow roots. It defaults to `False` and can be overridden per request. For MCP, pass `pierce_shadow=true` to `fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`, or each `browser_fetch` call.
Both browser CLI commands accept `--pierce-shadow` and `--no-pierce-shadow`:
diff --git a/agent-skill/Scrapling-Skill/references/mcp-server.md b/agent-skill/Scrapling-Skill/references/mcp-server.md
index a17840f..b96ecfa 100644
--- a/agent-skill/Scrapling-Skill/references/mcp-server.md
+++ b/agent-skill/Scrapling-Skill/references/mcp-server.md
@@ -1,12 +1,12 @@
# Scrapling MCP Server
-The Scrapling MCP server exposes thirteen tools over the MCP protocol. It supports CSS-selector-based content narrowing (reducing tokens by extracting only relevant elements before returning results), three levels of scraping capability (plain HTTP, browser-rendered, and stealth/anti-bot bypass), persistent browser session management, and page screenshots returned as real image content blocks. Fetch tools come in two modes: one-shot tools (`fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`) each launch and close their own browser, while `session_fetch` and `session_make_request` work through sessions opened with `open_session`/`open_request_session`.
+The Scrapling MCP server exposes fourteen tools over the MCP protocol. It supports CSS-selector-based content narrowing (reducing tokens by extracting only relevant elements before returning results), three levels of scraping capability (plain HTTP, browser-rendered, and stealth/anti-bot bypass), persistent browser session management, and page screenshots returned as real image content blocks. Fetch tools come in two modes: one-shot tools (`fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`) each launch and close their own browser, while `browser_fetch` and `session_make_request` work through sessions opened with `browser_open`/`open_request_session`.
-All scraping tools return a `ResponseModel` with fields: `status` (int), `content` (list of strings), `url` (str). The `screenshot` tool returns a list of MCP content blocks: an `ImageContent` (the screenshot bytes) followed by a `TextContent` (the post-redirect URL).
+Fetch and HTTP request tools return a `ResponseModel` with fields: `status` (int), `content` (list of strings), `url` (str). Bulk tools return a list of these responses. The `browser_screenshot` tool returns a list of MCP content blocks: an `ImageContent` (the screenshot bytes) followed by a `TextContent` (the post-redirect URL). `browser_snapshot` returns plain text.
## Shadow DOM
-Set `pierce_shadow=true` on `fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`, or `session_fetch` to include open Shadow DOM content. It defaults to `false`. For persistent sessions, pass it on each `session_fetch` call. See [Shadow DOM](fetching/dynamic.md#shadow-dom) for selector examples and limits.
+Set `pierce_shadow=true` on `fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`, or `browser_fetch` to include open Shadow DOM content. It defaults to `false`. For persistent sessions, pass it on each `browser_fetch` call. See [Shadow DOM](fetching/dynamic.md#shadow-dom) for selector examples and limits.
## One-shot tools
@@ -76,7 +76,7 @@ Opens a Chromium browser via Playwright to render JavaScript. Suitable for dynam
| `timezone_id` | str or null | null | Browser timezone, e.g. `"America/New_York"` |
| `locale` | str or null | null | Browser locale, e.g. `"en-GB"` |
-This is a one-shot tool: it always launches its own browser. To fetch through a persistent session, use `session_fetch`.
+This is a one-shot tool: it always launches its own browser. To fetch through a persistent session, use `browser_fetch`.
### `bulk_fetch` -- Browser fetch (multiple URLs)
@@ -96,7 +96,7 @@ Anti-bot bypass fetcher with fingerprint spoofing. Use this for sites with Cloud
| `allow_webgl` | bool | true | Keep WebGL enabled (disabling is detectable by WAFs) |
| `additional_args` | dict or null | null | Extra Playwright context args (overrides Scrapling defaults) |
-All parameters from `fetch` are also accepted. Like `fetch`, this is a one-shot tool that launches its own browser; use `session_fetch` for a stealthy session.
+All parameters from `fetch` are also accepted. Like `fetch`, this is a one-shot tool that launches its own browser; use `browser_fetch` for a stealthy session.
### `bulk_stealthy_fetch` -- Stealth browser fetch (multiple URLs)
@@ -104,9 +104,9 @@ Concurrent stealth version. Same parameters as `stealthy_fetch` except `url` is
## Session tools
-### `open_session` -- Create a persistent browser session
+### `browser_open` -- Create a persistent browser session
-Opens a browser session that stays alive across multiple `session_fetch` calls, avoiding the overhead of launching a new browser each time. It holds the browser-level configuration only; per-request options are passed to `session_fetch`. For plain HTTP requests without a browser, use `open_request_session` instead. Returns a `SessionCreatedModel` with `session_id`, `session_type`, `created_at`, `is_alive`, `settings` (the session's effective configuration for the AI agent; empty for CDP sessions), and `message`.
+Opens a browser session that stays alive across multiple `browser_fetch` calls, avoiding the overhead of launching a new browser each time. It holds the browser-level configuration only; per-request options are passed to `browser_fetch`. For plain HTTP requests without a browser, use `open_request_session` instead. Returns a `SessionCreatedModel` with `session_id`, `session_type`, `created_at`, `is_alive`, `settings` (the session's effective configuration for the AI agent; empty for CDP sessions), and `message`.
**Key parameters:**
@@ -119,9 +119,9 @@ Opens a browser session that stays alive across multiple `session_fetch` calls,
| `block_webrtc` | bool | false | (Stealthy only) Block WebRTC IP leak |
| `allow_webgl` | bool | true | (Stealthy only) Keep WebGL enabled |
-Plus the other browser-level session parameters (`proxy`, `real_chrome`, `cdp_url`, `locale`, `timezone_id`, `useragent`, `cookies`, `executable_path`, `additional_args`). Per-request options (`timeout`, `wait`, `google_search`, `network_idle`, `disable_resources`, `wait_selector`, `wait_selector_state`, `extra_headers`, `solve_cloudflare`) are not set here; pass them to `session_fetch`.
+Plus the other browser-level session parameters (`proxy`, `real_chrome`, `cdp_url`, `locale`, `timezone_id`, `useragent`, `cookies`, `executable_path`, `additional_args`). Per-request options (`timeout`, `wait`, `google_search`, `network_idle`, `disable_resources`, `wait_selector`, `wait_selector_state`, `extra_headers`, `solve_cloudflare`) are not set here; pass them to `browser_fetch`.
-One `session_fetch` works with either browser session type; `solve_cloudflare` only applies to a stealthy session.
+One `browser_fetch` works with either browser session type; `solve_cloudflare` only applies to a stealthy session.
### `open_request_session` -- Create a persistent HTTP requests session
@@ -133,15 +133,15 @@ Opens an HTTP session (no browser) that stays alive across multiple `session_mak
| `impersonate` | str | `"chrome"` | Browser fingerprint to impersonate on every request |
| `proxy` | str or null | null | Proxy URL used for every request, e.g. `"http://user:pass@host:port"` |
-### `session_fetch` -- Fetch through an open browser session (single URL)
+### `browser_fetch` -- Fetch through an open browser session (single URL)
-Fetches one URL through a browser session opened with `open_session` (dynamic or stealthy). The session holds the browser-level configuration; every parameter here applies to this request only. Raises on a requests session; use `session_make_request` there instead.
+Fetches one URL through a browser session opened with `browser_open` (dynamic or stealthy). The session holds the browser-level configuration; every parameter here applies to this request only. Raises on a requests session; use `session_make_request` there instead.
| Parameter | Type | Default | Description |
|-----------------------|--------------|--------------|----------------------------------------------------------------------------------------|
| `url` | str | required | URL to fetch |
-| `session_id` | str | required | ID of an open session created with `open_session` |
-| `extraction_type` | str | `"markdown"` | `"markdown"` / `"html"` / `"text"` |
+| `session_id` | str | required | ID of an open session created with `browser_open` |
+| `extraction_type` | str | `"markdown"` | `"markdown"` / `"html"` / `"text"` / `"snapshot"` |
| `css_selector` | str or null | null | Narrow content before extraction |
| `main_content_only` | bool | true | Restrict to `<body>` |
| `wait` | number | 0 | Extra wait (ms) after page load before extraction |
@@ -157,6 +157,12 @@ Fetches one URL through a browser session opened with `open_session` (dynamic or
| `blocked_domains` | list or null | null | Domain names to block for this request (subdomains matched too) |
| `solve_cloudflare` | bool | false | (Stealthy sessions only) Auto-solve Cloudflare challenges; errors on a dynamic session |
+With `extraction_type="snapshot"`, the response keeps `status` and `url` and returns one unchanged AI ARIA snapshot string in `content`. `css_selector` must match exactly one element; omit it for the whole page. `main_content_only` and `pierce_shadow` do not filter snapshots. Snapshot extraction is only available on `browser_fetch`.
+
+### `browser_snapshot` -- Read the current page without navigation
+
+Returns a plain-text AI ARIA snapshot of the current whole page, including element roles, names, and references. Takes `session_id` from `browser_open`, optional `depth` to limit the tree, and `boxes` (default `false`) to include element positions and sizes in viewport CSS pixels. Use it after `browser_fetch` finishes. Raises for an unknown or HTTP session, or a missing, closed, or busy page.
+
### `session_make_request` -- HTTP request through an open requests session
Makes an HTTP request (any method) through a session opened with `open_request_session`, reusing its cookies, connections, and browser fingerprint across calls. Same parameters as `make_request` plus a required `session_id`, minus the session-level `impersonate`, `proxy`, and `proxy_auth`. Raises on a browser session.
@@ -167,26 +173,26 @@ Closes a session (browser or requests) and frees its resources. Always close ses
| Parameter | Type | Default | Description |
|--------------|------|----------|----------------------------------|
-| `session_id` | str | required | Session ID from `open_session` |
+| `session_id` | str | required | Session ID from `browser_open` |
Returns a `SessionClosedModel` with `session_id` and `message`.
### `list_sessions` -- List active sessions
-Returns a list of `SessionInfo` objects, each with `session_id`, `session_type`, `created_at`, `is_alive`, and `settings` (same as `open_session` returns).
+Returns a list of `SessionInfo` objects, each with `session_id`, `session_type`, `created_at`, `is_alive`, and `settings` (same as `browser_open` returns).
No parameters.
-### `screenshot` -- Capture a page screenshot
+### `browser_screenshot` -- Capture a page screenshot
Navigates to a URL inside an existing browser session and returns the screenshot as an MCP `ImageContent` block (the bytes the model can see directly, not a base64 string in JSON) followed by a `TextContent` block carrying the post-redirect URL.
-Requires an open browser session. Call `open_session` first, then pass the `session_id` here. Both `dynamic` and `stealthy` sessions are accepted.
+Requires an open browser session. Call `browser_open` first, then pass the `session_id` here. Both `dynamic` and `stealthy` sessions are accepted.
| Parameter | Type | Default | Description |
|-----------------------|-----------------------|--------------|--------------------------------------------------------------------------------------|
| `url` | str | required | URL to navigate to and capture |
-| `session_id` | str | required | ID of an open browser session created with `open_session` |
+| `session_id` | str | required | ID of an open browser session created with `browser_open` |
| `image_type` | `"png"` / `"jpeg"` | `"png"` | Image format. Use `"jpeg"` for smaller payloads |
| `full_page` | bool | false | Capture the full scrollable page instead of just the viewport |
| `quality` | int or null | null | JPEG quality 0-100. Raises if passed with `image_type="png"` |
@@ -206,22 +212,23 @@ Requires an open browser session. Call `open_session` first, then pass the `sess
| Multiple JS-rendered pages | `bulk_fetch` |
| Cloudflare or strong anti-bot protection | `stealthy_fetch` (with `solve_cloudflare=true` for Turnstile) |
| Multiple protected pages | `bulk_stealthy_fetch` |
-| Multiple pages from the same site | `open_session` + `session_fetch` per page |
+| Multiple pages from the same site | `browser_open` + `browser_fetch` per page |
| Multiple plain HTTP requests to one site | `open_request_session` + `session_make_request` per request |
-| Need a screenshot of a page | `open_session` + `screenshot` with `session_id` |
+| Need a screenshot of a page | `browser_open` + `browser_screenshot` with `session_id` |
+| Read the current page's AI ARIA snapshot | `browser_snapshot` with `session_id` |
Start with `make_request` (fastest, lowest resource cost). Escalate to `fetch` if content requires JS rendering. Escalate to `stealthy_fetch` only if blocked. For multiple pages from the same site, use a persistent session to avoid browser launch overhead.
## Content extraction tips
- Use `css_selector` to narrow results before they reach the model -- this saves significant tokens.
-- `main_content_only=true` (default) strips nav/footer by restricting to `<body>`.
+- `main_content_only=true` (default) restricts HTML, Markdown, and text extraction to `<body>`.
- `extraction_type="markdown"` (default) is best for readability. Use `"text"` for minimal output, `"html"` when structure matters.
-- If a `css_selector` matches multiple elements, all are returned in the `content` list.
+- If a `css_selector` matches multiple elements, HTML, Markdown, and text extraction return all matches in the `content` list. Snapshot extraction requires exactly one match.
## Prompt injection protection
-When `main_content_only=true` (the default), the server automatically sanitizes scraped content to prevent prompt injection from malicious websites. It strips:
+For HTML, Markdown, and text extraction, `main_content_only=true` (the default) sanitizes scraped content to prevent prompt injection from malicious websites. It strips:
- CSS-hidden elements (`display:none`, `visibility:hidden`, `opacity:0`, `font-size:0`, `height:0`, `width:0`)
- `aria-hidden="true"` elements
@@ -233,7 +240,7 @@ Keep `main_content_only=true` for maximum protection.
## Ad blocking
-All browser-based tools (`fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`) and persistent sessions (`open_session`) automatically block requests to ~3,500 known ad and tracker domains. This is always enabled in the MCP server to save tokens and speed up page loads. No configuration needed.
+All browser-based tools (`fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`) and persistent sessions (`browser_open`) automatically block requests to ~3,500 known ad and tracker domains. This is always enabled in the MCP server to save tokens and speed up page loads. No configuration needed.
## Setup
@@ -269,7 +276,7 @@ docker run -p 8000:8000 -e SCRAPLING_MCP_AUTH_TOKEN="<your-token>" pyd4vinci/scr
## Custom browser executable
-Browser-based tools (`fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`, and `open_session`) can use a custom Chromium-compatible browser executable instead of the bundled Chromium. This is useful for custom browser builds or lightweight browser engines.
+Browser-based tools (`fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`, and `browser_open`) can use a custom Chromium-compatible browser executable instead of the bundled Chromium. This is useful for custom browser builds or lightweight browser engines.
To configure it once for the whole MCP server, pass the executable path when starting the server:
@@ -299,7 +306,7 @@ The MCP server name when registering with a client is `ScraplingServer`. The com
## Connecting to remote browsers
-`open_session` doesn't have to launch a browser locally. Pass a `cdp_url` and it connects to an already-running browser through the Chrome DevTools Protocol, whether that browser is on the same machine, another host, or a managed browser provider. Both session types (`dynamic` and `stealthy`) accept it, and the `session_id` you get back is used with `session_fetch` and `screenshot` as usual.
+`browser_open` doesn't have to launch a browser locally. Pass a `cdp_url` and it connects to an already-running browser through the Chrome DevTools Protocol, whether that browser is on the same machine, another host, or a managed browser provider. Both session types (`dynamic` and `stealthy`) accept it, and the `session_id` you get back is used with `browser_fetch`, `browser_snapshot`, and `browser_screenshot` as usual.
The URL can be a WebSocket endpoint (`ws://`/`wss://`), which is what managed browser providers hand out, or the HTTP endpoint of a browser started with `--remote-debugging-port=9222`, reached as `cdp_url="http://localhost:9222"`.
diff --git a/docs/ai/mcp-server.md b/docs/ai/mcp-server.md
index cb32b12..ee9748c 100644
--- a/docs/ai/mcp-server.md
+++ b/docs/ai/mcp-server.md
@@ -25,17 +25,17 @@ The Scrapling MCP Server provides fourteen powerful tools for web scraping, spli
### Session tools
#### 🔌 Session Management
-- **`open_session`**: Create a persistent browser session (dynamic or stealthy) that stays open across multiple `session_fetch` calls, avoiding the overhead of launching a new browser each time. It holds the browser-level configuration and returns the session's effective `settings` for the AI agent (empty for CDP sessions).
+- **`browser_open`**: Create a persistent browser session (dynamic or stealthy) that stays open across multiple `browser_fetch` calls, avoiding the overhead of launching a new browser each time. It holds the browser-level configuration and returns the session's effective `settings` for the AI agent (empty for CDP sessions).
- **`open_request_session`**: Create a persistent HTTP requests session (no browser) used with `session_make_request`, keeping cookies, connections, and the browser fingerprint (`impersonate`) between requests. It returns the same `settings` receipt and shows in `list_sessions` as a `static` session.
- **`close_session`**: Close a persistent session (browser or requests) and free its resources.
- **`list_sessions`**: List all active sessions with their details and `settings`.
#### 🎯 Fetching Through a Session
-- **`session_fetch`**: Fetch a single URL through an open browser session (dynamic or stealthy), carrying the per-request options for that call. This is the session counterpart of `fetch`/`stealthy_fetch`. Set `extraction_type="snapshot"` to return an AI ARIA snapshot in the usual response's `content` list.
+- **`browser_fetch`**: Fetch a single URL through an open browser session (dynamic or stealthy), carrying the per-request options for that call. This is the session counterpart of `fetch`/`stealthy_fetch`. Set `extraction_type="snapshot"` to return an AI ARIA snapshot in the usual response's `content` list.
- **`session_make_request`**: Make an HTTP request with any method through a session opened with `open_request_session`, reusing its cookies, connections, and browser fingerprint. This is the session counterpart of `make_request`.
#### 📸 Screenshots
-- **`screenshot`**: Capture a PNG or JPEG screenshot of a page using an open browser session, returned as an image content block the model can actually see (not a base64 string blob). Supports full-page captures, JPEG quality, and the usual readiness controls (`wait`, `wait_selector`, `network_idle`).
+- **`browser_screenshot`**: Capture a PNG or JPEG screenshot of a page using an open browser session, returned as an image content block the model can actually see (not a base64 string blob). Supports full-page captures, JPEG quality, and the usual readiness controls (`wait`, `wait_selector`, `network_idle`).
#### AI Snapshots
@@ -43,7 +43,7 @@ The Scrapling MCP Server provides fourteen powerful tools for web scraping, spli
### Shadow DOM
-Set `pierce_shadow=true` on `fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`, or `session_fetch` to include open Shadow DOM content. It defaults to `false`. For persistent sessions, pass it on each `session_fetch` call. See [Shadow DOM](../fetching/dynamic.md#shadow-dom) for selector examples and limits.
+Set `pierce_shadow=true` on `fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`, or `browser_fetch` to include open Shadow DOM content. It defaults to `false`. For persistent sessions, pass it on each `browser_fetch` call. See [Shadow DOM](../fetching/dynamic.md#shadow-dom) for selector examples and limits.
### Key Capabilities
- **Smart Content Extraction**: Convert web pages/elements to Markdown, HTML, or extract a clean version of the text content
@@ -69,8 +69,8 @@ If you don't know how to write/use CSS selectors, don't worry. You can tell the
Since version 0.4.15, the MCP server is reworked. If you are upgrading, note:
1. **The Streamable HTTP transport now requires authentication and binds to localhost.** `scrapling-mcp --http` on its own refuses to start; pass `--auth-token` (or the `SCRAPLING_MCP_AUTH_TOKEN` environment variable), or `--no-auth` to serve it unauthenticated on purpose. The default host is now `127.0.0.1` instead of `0.0.0.0`; pass `--host 0.0.0.0` to accept connections from the network.
-2. **The one-shot fetch tools no longer accept `session_id`.** `fetch`, `bulk_fetch`, `stealthy_fetch`, and `bulk_stealthy_fetch` always launch their own browser. To fetch through a session, use the new **`session_fetch`** tool (one URL per call, works with dynamic and stealthy sessions).
-3. **`open_session` takes browser-level parameters only.** The per-request options (`wait`, `timeout`, `google_search`, `network_idle`, `disable_resources`, `wait_selector`, `wait_selector_state`, `extra_headers`, `solve_cloudflare`) moved to `session_fetch` and are supplied on each call. `proxy` stays on `open_session` (it applies to the whole session, which runs a single tab). `max_pages` was removed too, since a session manages a single page per call now.
+2. **The one-shot fetch tools no longer accept `session_id`.** `fetch`, `bulk_fetch`, `stealthy_fetch`, and `bulk_stealthy_fetch` always launch their own browser. To fetch through a session, use the new **`browser_fetch`** tool (one URL per call, works with dynamic and stealthy sessions).
+3. **`browser_open` takes browser-level parameters only.** The per-request options (`wait`, `timeout`, `google_search`, `network_idle`, `disable_resources`, `wait_selector`, `wait_selector_state`, `extra_headers`, `solve_cloudflare`) moved to `browser_fetch` and are supplied on each call. `proxy` stays on `browser_open` (it applies to the whole session, which runs a single tab). `max_pages` was removed too, since a session manages a single page per call now.
4. **The `get` tool is renamed to `make_request`.** It now takes a `method` parameter and supports GET (default), POST, PUT, and DELETE, with `data`/`json` for request bodies. `bulk_get` is unchanged.
## Installation
@@ -176,7 +176,7 @@ Then, after you've added the server, you need to completely quit and restart the
### Custom Browser Executable
-Browser-based tools (`fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`, and `open_session`) can use a custom Chromium-compatible browser executable instead of the bundled Chromium. This is useful for custom browser builds or lightweight browser engines.
+Browser-based tools (`fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`, and `browser_open`) can use a custom Chromium-compatible browser executable instead of the bundled Chromium. This is useful for custom browser builds or lightweight browser engines.
To configure it once for the whole MCP server, pass the executable path when starting the server:
@@ -204,11 +204,11 @@ You can also set the `SCRAPLING_EXECUTABLE_PATH` environment variable before sta
### Connecting to Remote Browsers
-`open_session` doesn't have to launch a browser locally. Pass a CDP url and it will connect to an already-running browser through the [Chrome DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/), whether that browser is on the same machine, another host, or a managed browser provider:
+`browser_open` doesn't have to launch a browser locally. Pass a CDP url and it will connect to an already-running browser through the [Chrome DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/), whether that browser is on the same machine, another host, or a managed browser provider:
```
Open a stealthy browser session on wss://cdp.provider.example/session/abc123, then use it to scrape the product details from https://shop.example.com. Close the session when you're done.
```
-Both browser session types (`dynamic` and `stealthy`) accept it, and the `session_id` you get back is used with `session_fetch`, `browser_snapshot`, and `screenshot` as usual.
+Both browser session types (`dynamic` and `stealthy`) accept it, and the `session_id` you get back is used with `browser_fetch`, `browser_snapshot`, and `browser_screenshot` as usual.
The URL can be a WebSocket endpoint (`ws://`/`wss://`), which is what managed browser providers hand out, or the HTTP endpoint of a browser you started yourself with the remote debugging port enabled:
```commandline
@@ -385,7 +385,7 @@ We will gradually go from simple prompts to more complex ones. We will use Claud
```
Open a stealthy browser session, then use it to scrape the main details from the first 5 product pages on https://shop.example.com. Close the session when you're done.
```
- Claude will use `open_session` to create a persistent browser, call `session_fetch` for each product page through that session, and then call `close_session` at the end. This is significantly faster than launching a new browser for each page.
+ Claude will use `browser_open` to create a persistent browser, call `browser_fetch` for each product page through that session, and then call `close_session` at the end. This is significantly faster than launching a new browser for each page.
!!! danger
@@ -442,20 +442,20 @@ The MCP server automatically sanitizes scraped content when `main_content_only`
- **HTML comments**: `<!-- ... -->`
- **Zero-width characters**: Invisible unicode characters like zero-width spaces
-This protection runs automatically on all MCP tool responses. Keep `main_content_only=true` (the default) for maximum protection.
+This protection runs automatically for HTML, Markdown, and text extraction. Keep `main_content_only=true` (the default) for maximum protection.
### 6. Use Sessions for Multiple Requests
-- Use `open_session` to create a persistent browser session when scraping multiple pages, then call `session_fetch` for each page through that session
+- Use `browser_open` to create a persistent browser session when scraping multiple pages, then call `browser_fetch` for each page through that session
- For multiple plain HTTP requests, use `open_request_session` instead and call `session_make_request` per request; it keeps cookies, connections, and the browser fingerprint (`impersonate`) across calls without a browser
-- Sessions hold the session-level configuration set when opened (headless, locale, cookies, stealth toggles, etc. for browsers; `impersonate` and `proxy` for requests sessions); the per-request options (timeout, wait_selector, network_idle, solve_cloudflare, etc.) are passed to `session_fetch`/`session_make_request` on each call, with their defaults shown in the tool schemas
-- One `session_fetch` works with both browser session types; `solve_cloudflare` only applies to a stealthy session and raises a clear error on a dynamic one
+- Sessions hold the session-level configuration set when opened (headless, locale, cookies, stealth toggles, etc. for browsers; `impersonate` and `proxy` for requests sessions); the per-request options (timeout, wait_selector, network_idle, solve_cloudflare, etc.) are passed to `browser_fetch`/`session_make_request` on each call, with their defaults shown in the tool schemas
+- One `browser_fetch` works with both browser session types; `solve_cloudflare` only applies to a stealthy session and raises a clear error on a dynamic one
- The one-shot tools (`make_request`, `bulk_get`, `fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`) never take a session
- Always close sessions with `close_session` when done to free resources
- Use `list_sessions` to check which sessions are still active and see the `settings` each was created with (returned for the AI agent; empty for CDP sessions)
- Pass a custom `session_id` to the open tools to give sessions meaningful names (e.g. `"search"`, `"checkout"`) instead of the random hex default. They raise if the chosen ID is already in use, so you can detect collisions up front
### 7. Capturing Screenshots
-- `screenshot` only works through an existing browser session, so call `open_session` first (either `dynamic` or `stealthy` works)
+- `browser_screenshot` only works through an existing browser session, so call `browser_open` first (either `dynamic` or `stealthy` works)
- The image is returned as a real `ImageContent` block, not a base64 string in JSON, so the model sees the page directly
- Use `full_page=True` when you need everything below the fold; the default captures only the visible viewport
- Pick `image_type="jpeg"` with a `quality` value (0-100) for smaller payloads when pixel-perfect color isn't needed
diff --git a/docs/api-reference/mcp-server.md b/docs/api-reference/mcp-server.md
index 837faf6..82552ca 100644
--- a/docs/api-reference/mcp-server.md
+++ b/docs/api-reference/mcp-server.md
@@ -30,7 +30,7 @@ server = ScraplingMCPServer(executable_path="/path/to/chromium")
## Response Model
-The standardized response structure returned by the fetch and HTTP request tools. Bulk tools return a list of these responses. Session management tools use the models below, `screenshot` returns image and text content blocks, and `browser_snapshot` returns plain text.
+The standardized response structure returned by the fetch and HTTP request tools. Bulk tools return a list of these responses. Session management tools use the models below, `browser_screenshot` returns image and text content blocks, and `browser_snapshot` returns plain text.
## ::: scrapling.core.ai.ResponseModel
handler: python
@@ -56,9 +56,9 @@ Model classes for session management:
The main MCP server class that provides all web scraping tools:
-`browser_snapshot(session_id, depth=None, boxes=False)` returns a plain-text AI ARIA snapshot of the current whole page in a dynamic or stealthy session. Use it after `session_fetch` finishes. It reads the existing page without navigation and includes element references. `depth` limits the tree depth, and `boxes=True` adds element positions and sizes in viewport CSS pixels. An unknown session, an HTTP request session, or a missing, closed, or busy page returns an error.
+`browser_snapshot(session_id, depth=None, boxes=False)` returns a plain-text AI ARIA snapshot of the current whole page in a dynamic or stealthy session. Use it after `browser_fetch` finishes. It reads the existing page without navigation and includes element references. `depth` limits the tree depth, and `boxes=True` adds element positions and sizes in viewport CSS pixels. An unknown session, an HTTP request session, or a missing, closed, or busy page returns an error.
-`session_fetch` also accepts `extraction_type="snapshot"` to take an AI ARIA snapshot after loading the requested URL. It returns the usual `status`, `url`, and `content` fields, with one unchanged snapshot string in `content`. The existing `css_selector` can scope the snapshot to exactly one element; zero or multiple matches return an error. Without a selector, the snapshot covers the whole page. `main_content_only` and `pierce_shadow` do not filter snapshots. This extraction type is only available on the MCP `session_fetch` tool; browser fetchers still return the same `Response` object.
+`browser_fetch` also accepts `extraction_type="snapshot"` to take an AI ARIA snapshot after loading the requested URL. It returns the usual `status`, `url`, and `content` fields, with one unchanged snapshot string in `content`. The existing `css_selector` can scope the snapshot to exactly one element; zero or multiple matches return an error. Without a selector, the snapshot covers the whole page. `main_content_only` and `pierce_shadow` do not filter snapshots. This extraction type is only available on the MCP `browser_fetch` tool; browser fetchers still return the same `Response` object.
## ::: scrapling.core.ai.ScraplingMCPServer
handler: python
diff --git a/scrapling/core/ai.py b/scrapling/core/ai.py
index b9f7454..cbc9ac7 100644
--- a/scrapling/core/ai.py
+++ b/scrapling/core/ai.py
@@ -198,17 +198,17 @@ class ScraplingMCPServer:
"""Return a per-call executable path or the server-wide default."""
return executable_path or self._executable_path
- def _get_session(self, session_id: str, expected_type: Optional[SessionType]) -> _SessionEntry:
- """Look up a session by ID, optionally validating its type. Pass `None` to skip the type check."""
+ def _get_session(self, session_id: str, expected_type: List[SessionType]) -> _SessionEntry:
+ """Look up an active session by ID and validate its type against the allowed types."""
entry = self._sessions.get(session_id)
if entry is None:
raise ValueError(f"Session '{session_id}' not found. Use list_sessions to see active sessions.")
if not entry.session._is_alive:
raise ValueError(f"Session '{session_id}' is no longer alive. Open a new session.")
- if expected_type is not None and entry.session_type != expected_type:
+ if entry.session_type not in expected_type:
raise ValueError(
f"Session '{session_id}' is a '{entry.session_type}' session, but this tool requires a "
- f"'{expected_type}' session. Use the matching fetch tool for your session type."
+ f"{' or '.join(map(repr, expected_type))} session. Use the matching fetch tool for your session type."
)
return entry
@@ -234,7 +234,7 @@ class ScraplingMCPServer:
message=f"Session '{session_id}' ({session_type}) created successfully.",
)
- async def open_session(
+ async def browser_open(
self,
session_type: BrowserSessionType,
session_id: Optional[str] = None,
@@ -253,9 +253,9 @@ class ScraplingMCPServer:
allow_webgl: bool = True,
additional_args: Optional[Dict] = None,
) -> SessionCreatedModel:
- """Open a persistent browser session that can be reused across multiple `session_fetch` calls.
+ """Open a persistent browser session that can be reused across multiple `browser_fetch` calls.
This avoids the overhead of launching a new browser for each request. Sessions hold the browser-level
- configuration only; per-request options are passed to `session_fetch` on each call.
+ configuration only; per-request options are passed to `browser_fetch` on each call.
:param session_type: The type of session to open. Use "dynamic" for standard Playwright browser, or "stealthy" for anti-bot bypass with fingerprint spoofing.
:param session_id: Optional custom session ID. If not provided, a random 12-character hex ID will be generated. Useful for naming sessions for easier management.
@@ -363,9 +363,9 @@ class ScraplingMCPServer:
boxes: bool = False,
) -> str:
"""Return the AI ARIA snapshot of the current page in an open browser session as plain text.
- Fetch a page with `session_fetch` first. The snapshot includes element references and preserves the current page.
+ Fetch a page with `browser_fetch` first. The snapshot includes element references and preserves the current page.
- :param session_id: ID of an open browser session created with `open_session`.
+ :param session_id: ID of an open browser session created with `browser_open`.
:param depth: Limit the snapshot tree depth. Defaults to no limit.
:param boxes: Include element bounding boxes in viewport CSS pixels.
"""
@@ -379,23 +379,18 @@ class ScraplingMCPServer:
boxes: bool = False,
) -> str:
"""Reserve the current page and return its AI ARIA snapshot."""
- entry = self._get_session(session_id, expected_type=None)
- if entry.session_type == "static":
- raise ValueError(
- f"Session '{session_id}' is a 'static' session, so it can't take snapshots. "
- f"Open a 'dynamic' or 'stealthy' session for that."
- )
+ entry = self._get_session(session_id, expected_type=["dynamic", "stealthy"])
pool = entry.session.page_pool
if not pool.pages_count:
- raise ValueError(f"Session '{session_id}' has no page to snapshot. Use session_fetch first.")
+ raise ValueError(f"Session '{session_id}' has no page to snapshot. Use browser_fetch first.")
page_info = pool.get_ready_page()
if page_info is None:
raise RuntimeError(f"Session '{session_id}' has a busy page. Wait for the current request to finish.")
try:
if page_info.page.is_closed():
- raise RuntimeError(f"Session '{session_id}' has a closed page. Use session_fetch to open a new one.")
+ raise RuntimeError(f"Session '{session_id}' has a closed page. Use browser_fetch to open a new one.")
return await entry.session._snapshot(page_info.page, depth=depth, boxes=boxes, css_selector=css_selector)
finally:
if page_info.page.is_closed():
@@ -403,7 +398,7 @@ class ScraplingMCPServer:
else:
page_info.mark_ready()
- async def screenshot(
+ async def browser_screenshot(
self,
url: str,
session_id: str,
@@ -417,10 +412,10 @@ class ScraplingMCPServer:
timeout: int | float = 30000,
) -> List[ImageContent | TextContent]:
"""Capture a screenshot of a web page using an existing browser session and return it as an image.
- A browser session must be opened first with `open_session` (either `dynamic` or `stealthy`); the session ID is then passed here.
+ A browser session must be opened first with `browser_open` (either `dynamic` or `stealthy`); the session ID is then passed here.
:param url: The URL to navigate to and capture.
- :param session_id: ID of an open browser session created with `open_session`.
+ :param session_id: ID of an open browser session created with `browser_open`.
:param image_type: Image format. Defaults to "png". Use "jpeg" for smaller file sizes.
:param full_page: When True, captures the full scrollable page instead of just the viewport.
:param quality: Image quality (0-100) for JPEG only. Raises if passed with `image_type="png"`.
@@ -433,12 +428,7 @@ class ScraplingMCPServer:
if quality is not None and image_type != "jpeg":
raise ValueError("'quality' is only valid when 'image_type' is 'jpeg'.")
- entry = self._get_session(session_id, expected_type=None)
- if entry.session_type == "static":
- raise ValueError(
- f"Session '{session_id}' is a 'static' session, so it can't take screenshots. "
- f"Open a 'dynamic' or 'stealthy' session for that."
- )
+ entry = self._get_session(session_id, expected_type=["dynamic", "stealthy"])
screenshot_kwargs: Dict[str, Any] = {"type": image_type, "full_page": full_page}
if quality is not None:
@@ -974,7 +964,7 @@ class ScraplingMCPServer:
return [_translate_response(page, extraction_type, css_selector, main_content_only) for page in responses]
- async def session_fetch(
+ async def browser_fetch(
self,
url: str,
session_id: str,
@@ -994,11 +984,11 @@ class ScraplingMCPServer:
solve_cloudflare: bool = False,
pierce_shadow: bool = False,
) -> ResponseModel:
- """Fetch a URL through a browser session previously opened with `open_session` and return a structured output of the result.
+ """Fetch a URL through a browser session previously opened with `browser_open` and return a structured output of the result.
The session (dynamic or stealthy) holds the browser-level configuration; every option here applies to this request only, with the defaults shown.
:param url: The URL to request.
- :param session_id: ID of an open browser session created with `open_session`.
+ :param session_id: ID of an open browser session created with `browser_open`.
:param extraction_type: The type of content to extract from the page: "markdown", "html", "text", or "snapshot".
:param css_selector: CSS selector to extract the content from the page. If main_content_only is True, then it will be executed on the main content of the page.
:param main_content_only: Whether to extract only the main content of the page. The main content here is the data inside the `<body>` tag.
@@ -1016,11 +1006,7 @@ class ScraplingMCPServer:
:param blocked_domains: A list of domain names to block requests to for this request. Subdomains are also matched.
:param solve_cloudflare: (Stealthy sessions only) Solves all types of the Cloudflare's Turnstile/Interstitial challenges before returning the response.
"""
- entry = self._get_session(session_id, expected_type=None)
- if entry.session_type == "static":
- raise ValueError(
- f"Session '{session_id}' is a 'static' session. Use `session_make_request` with it instead."
- )
+ entry = self._get_session(session_id, expected_type=["dynamic", "stealthy"])
if solve_cloudflare and entry.session_type != "stealthy":
raise ValueError(
f"Session '{session_id}' is a '{entry.session_type}' session, so it can't solve Cloudflare "
@@ -1100,7 +1086,7 @@ class ScraplingMCPServer:
:param http3: Whether to use HTTP3. It might be problematic if used it with `impersonate`.
:param stealthy_headers: If enabled (default), it creates and adds real browser headers. It also sets a Google referer header.
"""
- entry = self._get_session(session_id, expected_type="static")
+ entry = self._get_session(session_id, expected_type=["static"])
request_kwargs: Dict[str, Any] = dict(
auth=_normalize_credentials(auth),
@@ -1147,20 +1133,20 @@ class ScraplingMCPServer:
],
"cache_hints": {"tools/list": CacheHint(ttl_ms=3_600_000, scope="public")},
"instructions": """Follow these instructions precisely:
-1. When the `open_session` or `open_request_session` tools are used, make sure to close the session with `close_session` after you finish, and use `list_sessions` if you lose track of the open sessions or their effective settings.
+1. When the `browser_open` or `open_request_session` tools are used, make sure to close the session with `close_session` after you finish, and use `list_sessions` if you lose track of the open sessions or their effective settings.
2. If the user didn't specify which tool to use, start with the `make_request` tool (a plain HTTP request, defaulting to GET; set `method` for POST/PUT/DELETE), then escalate. The `make_request` tool and `bulk_get` (its GET-only bulk version) are suitable only for low-to-mid protection levels.
For high-protection levels or websites that require JS loading, use the other tools directly.
3. For HTML, Markdown, and text extraction, if the `css_selector` resolves to more than one element, all the elements will be returned. Snapshot extraction requires a selector matching exactly one element, or no selector for the whole page.
4. For all fetch tools, the `extraction_type` parameter controls the format of the returned content: "markdown" (default) converts the page content to Markdown, "html" returns the raw HTML, and "text" returns the text content of the page.
5. For HTML, Markdown, and text extraction, `main_content_only` is enabled by default and returns only the content inside the page's `<body>` tag. Pass `main_content_only=False` when you need the full page instead.
6. If the task consists of multiple sequential requests to the same website, open a session once, then fetch through it to be more efficient:
- `open_session` + `session_fetch` per page for browsers, or `open_request_session` + `session_make_request` per request for plain HTTP.
-7. Sessions hold the session-level configuration set when opened, while `session_fetch`/`session_make_request` carry the per-request options and apply them on each call with the defaults shown in their schemas.
+ `browser_open` + `browser_fetch` per page for browsers, or `open_request_session` + `session_make_request` per request for plain HTTP.
+7. Sessions hold the session-level configuration set when opened, while `browser_fetch`/`session_make_request` carry the per-request options and apply them on each call with the defaults shown in their schemas.
The one-shot tools (`make_request`, `bulk_get`, `fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`) never touch sessions.
8. If you are making multiple parallel one-shot requests, use the bulk version of the tool to be more efficient.
9. If you are crawling/browsing a website, be more efficient by using the `css_selector` parameter to only access the parts you are interested in and save money/time. Example: use the `a` selector to extract the urls right away.
-10. The user can pass a CDP URL to connect to a remote browser session through the `open_session` tool, then use it with the session tools.
-11. Set `extraction_type="snapshot"` on `session_fetch` to get an AI ARIA snapshot with element references in its content field. Use `css_selector` to snapshot one element, or omit it for the whole page. `main_content_only` and `pierce_shadow` do not filter snapshots. Use `browser_snapshot` to read the current page again without navigating; it returns plain text. Its `depth` and `boxes` options limit the tree or include element bounding boxes.
+10. The user can pass a CDP URL to connect to a remote browser session through the `browser_open` tool, then use it with the session tools.
+11. Set `extraction_type="snapshot"` on `browser_fetch` to get an AI ARIA snapshot with element references in its content field. Use `css_selector` to snapshot one element, or omit it for the whole page. `main_content_only` and `pierce_shadow` do not filter snapshots. Use `browser_snapshot` to read the current page again without navigating; it returns plain text. Its `depth` and `boxes` options limit the tree or include element bounding boxes.
""",
}
if self._auth_token:
@@ -1171,7 +1157,7 @@ class ScraplingMCPServer:
server = MCPServer(name="Scrapling", **settings)
# Session management tools
server.add_tool(
- self.open_session, title="open_session", structured_output=True, annotations=_SESSION_TOOL_ANNOTATIONS
+ self.browser_open, title="browser_open", structured_output=True, annotations=_SESSION_TOOL_ANNOTATIONS
)
server.add_tool(
self.open_request_session,
@@ -1232,9 +1218,9 @@ class ScraplingMCPServer:
)
# Session-scoped fetch tools
server.add_tool(
- self.session_fetch,
- title="session_fetch",
- description=self.session_fetch.__doc__,
+ self.browser_fetch,
+ title="browser_fetch",
+ description=self.browser_fetch.__doc__,
structured_output=True,
annotations=_FETCH_TOOL_ANNOTATIONS,
)
@@ -1254,9 +1240,9 @@ class ScraplingMCPServer:
)
# Screenshot tool (returns image + url content blocks, not structured JSON)
server.add_tool(
- self.screenshot,
- title="screenshot",
- description=self.screenshot.__doc__,
+ self.browser_screenshot,
+ title="browser_screenshot",
+ description=self.browser_screenshot.__doc__,
structured_output=False,
annotations=_FETCH_TOOL_ANNOTATIONS,
)
diff --git a/tests/ai/test_ai_mcp.py b/tests/ai/test_ai_mcp.py
index a1f3e79..a47c60a 100644
--- a/tests/ai/test_ai_mcp.py
+++ b/tests/ai/test_ai_mcp.py
@@ -5,7 +5,7 @@ from typing import Any
from contextlib import contextmanager
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from threading import Thread
-from unittest.mock import patch
+from unittest.mock import Mock, patch
import pytest
import pytest_httpbin
@@ -22,6 +22,8 @@ from scrapling.core.ai import (
SessionInfo,
SessionCreatedModel,
SessionClosedModel,
+ SessionType,
+ _SessionEntry,
_normalize_credentials,
_page_pool_size,
_session_settings,
@@ -113,6 +115,61 @@ class _FakeStealthySession(_FakeAsyncBrowserSession):
_config_model = StealthConfig
+class TestSessionTypeChecks:
+ @pytest.mark.parametrize(
+ "session_type, allowed, error",
+ [
+ ("dynamic", ["dynamic", "stealthy"], None),
+ ("stealthy", ["dynamic", "stealthy"], None),
+ ("static", ["static"], None),
+ ("static", ["dynamic", "stealthy"], "requires a 'dynamic' or 'stealthy' session"),
+ ("dynamic", ["static"], "requires a 'static' session"),
+ ("stealthy", ["static"], "requires a 'static' session"),
+ ],
+ )
+ def test_get_session_checks_allowed_types(
+ self, session_type: SessionType, allowed: list[SessionType], error: str | None
+ ) -> None:
+ server = ScraplingMCPServer()
+ session = Mock(_is_alive=True)
+ entry = _SessionEntry(session, session_type)
+ server._sessions["test"] = entry
+ if error:
+ with pytest.raises(ValueError, match=error):
+ server._get_session("test", allowed)
+ else:
+ assert server._get_session("test", allowed) is entry
+ assert session.mock_calls == []
+
+ @pytest.mark.asyncio
+ @pytest.mark.parametrize(
+ "tool, session_type, required",
+ [
+ ("browser_fetch", "static", "'dynamic' or 'stealthy'"),
+ ("browser_screenshot", "static", "'dynamic' or 'stealthy'"),
+ ("browser_snapshot", "static", "'dynamic' or 'stealthy'"),
+ ("session_make_request", "dynamic", "'static'"),
+ ("session_make_request", "stealthy", "'static'"),
+ ],
+ )
+ async def test_tool_rejects_wrong_session_before_any_action(
+ self, tool: str, session_type: SessionType, required: str
+ ) -> None:
+ server = ScraplingMCPServer()
+ session = Mock(_is_alive=True)
+ server._sessions["test"] = _SessionEntry(session, session_type)
+ args = {"session_id": "test"}
+ if tool != "browser_snapshot":
+ args["url"] = "https://example.com"
+ async with Client(server._build_server("127.0.0.1", 8000)) as client:
+ result = await client.call_tool(tool, args)
+ assert result.is_error
+ assert result.content and isinstance(result.content[0], TextContent)
+ assert f"requires a {required} session" in result.content[0].text
+ assert session.mock_calls == []
+ assert server._sessions["test"].session is session
+
+
@pytest_httpbin.use_class_based_httpbin
class TestMCPServer:
"""Test MCP server functionality"""
@@ -192,7 +249,7 @@ class TestSessionManagement:
@pytest.mark.asyncio
async def test_open_and_close_session(self, server):
"""Test opening and closing a dynamic session"""
- result = await server.open_session(session_type="dynamic", headless=True)
+ result = await server.browser_open(session_type="dynamic", headless=True)
assert isinstance(result, SessionCreatedModel)
assert result.session_type == "dynamic"
assert result.is_alive is True
@@ -211,7 +268,7 @@ class TestSessionManagement:
assert sessions == []
# Open a session
- result = await server.open_session(session_type="dynamic", headless=True)
+ result = await server.browser_open(session_type="dynamic", headless=True)
session_id = result.session_id
# List should show it
@@ -226,29 +283,29 @@ class TestSessionManagement:
await server.close_session(session_id)
@pytest.mark.asyncio
- async def test_session_fetch_reuses_the_session(self, server, test_url):
+ async def test_browser_fetch_reuses_the_session(self, server, test_url):
"""Test fetching a page twice through a persistent dynamic session"""
- result = await server.open_session(session_type="dynamic", headless=True)
+ result = await server.browser_open(session_type="dynamic", headless=True)
session_id = result.session_id
- response = await server.session_fetch(url=test_url, session_id=session_id)
+ response = await server.browser_fetch(url=test_url, session_id=session_id)
assert isinstance(response, ResponseModel)
assert response.status == 200
# Fetch again with the same session (reuse)
- response2 = await server.session_fetch(url=test_url, session_id=session_id)
+ response2 = await server.browser_fetch(url=test_url, session_id=session_id)
assert isinstance(response2, ResponseModel)
assert response2.status == 200
await server.close_session(session_id)
@pytest.mark.asyncio
- async def test_session_fetch_accepts_per_request_overrides(self, server, test_url):
+ async def test_browser_fetch_accepts_per_request_overrides(self, server, test_url):
"""A per-request override is honored on a session fetch"""
- result = await server.open_session(session_type="dynamic", headless=True)
+ result = await server.browser_open(session_type="dynamic", headless=True)
session_id = result.session_id
- response = await server.session_fetch(url=test_url, session_id=session_id, network_idle=True, timeout=45000)
+ response = await server.browser_fetch(url=test_url, session_id=session_id, network_idle=True, timeout=45000)
assert isinstance(response, ResponseModel)
assert response.status == 200
@@ -261,37 +318,37 @@ class TestSessionManagement:
await server.close_session("nonexistent")
@pytest.mark.asyncio
- async def test_session_fetch_with_nonexistent_session(self, server, test_url):
+ async def test_browser_fetch_with_nonexistent_session(self, server, test_url):
"""Test fetching with a session ID that doesn't exist"""
with pytest.raises(ValueError, match="not found"):
- await server.session_fetch(url=test_url, session_id="nonexistent")
+ await server.browser_fetch(url=test_url, session_id="nonexistent")
@pytest.mark.asyncio
- async def test_session_fetch_with_closed_session(self, server, test_url):
+ async def test_browser_fetch_with_closed_session(self, server, test_url):
"""Test fetching with a session that has been closed"""
- result = await server.open_session(session_type="dynamic", headless=True)
+ result = await server.browser_open(session_type="dynamic", headless=True)
session_id = result.session_id
await server.close_session(session_id)
with pytest.raises(ValueError, match="not found"):
- await server.session_fetch(url=test_url, session_id=session_id)
+ await server.browser_fetch(url=test_url, session_id=session_id)
@pytest.mark.asyncio
- async def test_open_session_with_custom_id(self, server):
+ async def test_browser_open_with_custom_id(self, server):
"""Test opening a session with a custom session_id"""
- result = await server.open_session(session_type="dynamic", session_id="my-session", headless=True)
+ result = await server.browser_open(session_type="dynamic", session_id="my-session", headless=True)
assert isinstance(result, SessionCreatedModel)
assert result.session_id == "my-session"
await server.close_session("my-session")
@pytest.mark.asyncio
- async def test_open_session_duplicate_id_raises(self, server):
+ async def test_browser_open_duplicate_id_raises(self, server):
"""Test that opening a session with a duplicate session_id raises an error"""
- await server.open_session(session_type="dynamic", session_id="dupe", headless=True)
+ await server.browser_open(session_type="dynamic", session_id="dupe", headless=True)
with pytest.raises(ValueError, match="already exists"):
- await server.open_session(session_type="dynamic", session_id="dupe", headless=True)
+ await server.browser_open(session_type="dynamic", session_id="dupe", headless=True)
await server.close_session("dupe")
@@ -352,26 +409,26 @@ class TestStaticSessionManagement:
monkeypatch.setattr("scrapling.core.ai.AsyncDynamicSession", _FakeDynamicSession)
await server.open_request_session(session_id="shared")
with pytest.raises(ValueError, match="already exists"):
- await server.open_session(session_type="dynamic", session_id="shared")
+ await server.browser_open(session_type="dynamic", session_id="shared")
await server.close_session("shared")
@pytest.mark.asyncio
async def test_session_make_request_requires_a_static_session(self, server, monkeypatch):
"""session_make_request rejects browser sessions"""
monkeypatch.setattr("scrapling.core.ai.AsyncDynamicSession", _FakeDynamicSession)
- await server.open_session(session_type="dynamic", session_id="browser")
+ await server.browser_open(session_type="dynamic", session_id="browser")
with pytest.raises(ValueError, match="requires a 'static' session"):
await server.session_make_request(url="https://example.com", session_id="browser")
await server.close_session("browser")
@pytest.mark.asyncio
- async def test_session_fetch_and_screenshot_reject_static_sessions(self, server):
+ async def test_browser_fetch_and_screenshot_reject_static_sessions(self, server):
"""The browser session tools refuse a static session with a clear error"""
await server.open_request_session(session_id="st2")
- with pytest.raises(ValueError, match="session_make_request"):
- await server.session_fetch(url="https://example.com", session_id="st2")
- with pytest.raises(ValueError, match="can't take screenshots"):
- await server.screenshot(url="https://example.com", session_id="st2")
+ with pytest.raises(ValueError, match="requires a 'dynamic' or 'stealthy' session"):
+ await server.browser_fetch(url="https://example.com", session_id="st2")
+ with pytest.raises(ValueError, match="requires a 'dynamic' or 'stealthy' session"):
+ await server.browser_screenshot(url="https://example.com", session_id="st2")
await server.close_session("st2")
@@ -384,24 +441,24 @@ class TestExecutablePath:
_FakeStealthySession.instances = []
@pytest.mark.asyncio
- async def test_open_session_passes_executable_path(self, monkeypatch):
- """open_session forwards per-session executable_path to the dynamic session"""
+ async def test_browser_open_passes_executable_path(self, monkeypatch):
+ """browser_open forwards per-session executable_path to the dynamic session"""
monkeypatch.setattr("scrapling.core.ai.AsyncDynamicSession", _FakeDynamicSession)
server = ScraplingMCPServer()
- created = await server.open_session(session_type="dynamic", executable_path="/tmp/chrome")
+ created = await server.browser_open(session_type="dynamic", executable_path="/tmp/chrome")
assert _FakeDynamicSession.instances[0].kwargs["executable_path"] == "/tmp/chrome"
await server.close_session(created.session_id)
@pytest.mark.asyncio
- async def test_open_session_uses_environment_default(self, monkeypatch):
- """open_session uses SCRAPLING_EXECUTABLE_PATH when no per-call value is provided"""
+ async def test_browser_open_uses_environment_default(self, monkeypatch):
+ """browser_open uses SCRAPLING_EXECUTABLE_PATH when no per-call value is provided"""
monkeypatch.setenv("SCRAPLING_EXECUTABLE_PATH", "/opt/custom-chromium")
monkeypatch.setattr("scrapling.core.ai.AsyncStealthySession", _FakeStealthySession)
server = ScraplingMCPServer()
- created = await server.open_session(session_type="stealthy")
+ created = await server.browser_open(session_type="stealthy")
assert _FakeStealthySession.instances[0].kwargs["executable_path"] == "/opt/custom-chromium"
await server.close_session(created.session_id)
@@ -477,8 +534,8 @@ class TestBulkPagePool:
)
-class TestSessionFetchForwarding:
- """`session_fetch` forwards its per-request params by name to the session's fetch()."""
+class TestBrowserFetchForwarding:
+ """`browser_fetch` forwards its per-request params by name to the session's fetch()."""
@pytest.fixture(autouse=True)
def reset_fakes(self):
@@ -490,13 +547,13 @@ class TestSessionFetchForwarding:
return fake.instances[0].fetch_calls[0]
@pytest.mark.asyncio
- async def test_dynamic_session_fetch_forwards_the_per_request_params(self, monkeypatch):
+ async def test_dynamic_browser_fetch_forwards_the_per_request_params(self, monkeypatch):
"""A dynamic session receives every dynamic per-request param, including explicit None values"""
monkeypatch.setattr("scrapling.core.ai.AsyncDynamicSession", _FakeDynamicSession)
server = ScraplingMCPServer()
- opened = await server.open_session(session_type="dynamic")
+ opened = await server.browser_open(session_type="dynamic")
- await server.session_fetch(url="https://example.com/1", session_id=opened.session_id)
+ await server.browser_fetch(url="https://example.com/1", session_id=opened.session_id)
forwarded = self._fetch_call(_FakeDynamicSession)
assert forwarded == {
@@ -513,16 +570,16 @@ class TestSessionFetchForwarding:
"blocked_domains": None,
}, forwarded
assert "solve_cloudflare" not in forwarded, "solve_cloudflare must not reach a dynamic session"
- assert "proxy" not in forwarded, "proxy is session-level (open_session), never forwarded per request"
+ assert "proxy" not in forwarded, "proxy is session-level (browser_open), never forwarded per request"
@pytest.mark.asyncio
- async def test_stealthy_session_fetch_forwards_solve_cloudflare(self, monkeypatch):
+ async def test_stealthy_browser_fetch_forwards_solve_cloudflare(self, monkeypatch):
"""A stealthy session additionally receives solve_cloudflare"""
monkeypatch.setattr("scrapling.core.ai.AsyncStealthySession", _FakeStealthySession)
server = ScraplingMCPServer()
- opened = await server.open_session(session_type="stealthy")
+ opened = await server.browser_open(session_type="stealthy")
- await server.session_fetch(url="https://example.com/1", session_id=opened.session_id, solve_cloudflare=True)
+ await server.browser_fetch(url="https://example.com/1", session_id=opened.session_id, solve_cloudflare=True)
forwarded = self._fetch_call(_FakeStealthySession)
assert forwarded.get("solve_cloudflare") is True, forwarded
@@ -532,9 +589,9 @@ class TestSessionFetchForwarding:
"""Per-request overrides reach the session as given"""
monkeypatch.setattr("scrapling.core.ai.AsyncDynamicSession", _FakeDynamicSession)
server = ScraplingMCPServer()
- opened = await server.open_session(session_type="dynamic")
+ opened = await server.browser_open(session_type="dynamic")
- await server.session_fetch(
+ await server.browser_fetch(
url="https://example.com/1", session_id=opened.session_id, timeout=45000, wait_selector="#main"
)
@@ -547,24 +604,24 @@ class TestSessionFetchForwarding:
"""Asking a dynamic session to solve Cloudflare is a clear error, not a silent no-op"""
monkeypatch.setattr("scrapling.core.ai.AsyncDynamicSession", _FakeDynamicSession)
server = ScraplingMCPServer()
- opened = await server.open_session(session_type="dynamic")
+ opened = await server.browser_open(session_type="dynamic")
with pytest.raises(ValueError, match="can't solve Cloudflare"):
- await server.session_fetch(url="https://example.com/1", session_id=opened.session_id, solve_cloudflare=True)
+ await server.browser_fetch(url="https://example.com/1", session_id=opened.session_id, solve_cloudflare=True)
@pytest.mark.asyncio
async def test_unknown_session_raises(self):
server = ScraplingMCPServer()
with pytest.raises(ValueError, match="not found"):
- await server.session_fetch(url="https://example.com/1", session_id="nope")
+ await server.browser_fetch(url="https://example.com/1", session_id="nope")
class TestModeSplitContract:
"""The one-shot vs session split is derived from the library TypedDicts and must stay in sync."""
- def test_session_fetch_signature_matches_the_derived_fetch_keys(self):
- """session_fetch exposes exactly the stealth per-request keys (plus url/session_id/extraction trio)"""
- params = set(inspect.signature(ScraplingMCPServer.session_fetch).parameters) - {
+ def test_browser_fetch_signature_matches_the_derived_fetch_keys(self):
+ """browser_fetch exposes exactly the stealth per-request keys (plus url/session_id/extraction trio)"""
+ params = set(inspect.signature(ScraplingMCPServer.browser_fetch).parameters) - {
"self",
"url",
"session_id",
@@ -573,19 +630,19 @@ class TestModeSplitContract:
"main_content_only",
}
assert params == set(_STEALTH_FETCH_KEYS), (
- f"session_fetch params drifted from _STEALTH_FETCH_KEYS: {params ^ set(_STEALTH_FETCH_KEYS)}"
+ f"browser_fetch params drifted from _STEALTH_FETCH_KEYS: {params ^ set(_STEALTH_FETCH_KEYS)}"
)
- def test_session_fetch_defaults_match_the_library(self):
+ def test_browser_fetch_defaults_match_the_library(self):
"""Each per-request default equals the library config default so the AI sees the real value"""
defaults = {
name: p.default
- for name, p in inspect.signature(ScraplingMCPServer.session_fetch).parameters.items()
+ for name, p in inspect.signature(ScraplingMCPServer.browser_fetch).parameters.items()
if name in _STEALTH_FETCH_KEYS
}
library = models_default_values["StealthConfig"]
for name, value in defaults.items():
- assert value == library[name], f"session_fetch {name} default {value!r} != library {library[name]!r}"
+ assert value == library[name], f"browser_fetch {name} default {value!r} != library {library[name]!r}"
def test_one_shot_fetch_tools_have_no_session_id(self):
"""The one-shot tools no longer accept session_id"""
@@ -597,33 +654,33 @@ class TestModeSplitContract:
):
assert "session_id" not in inspect.signature(tool).parameters, f"{tool.__name__} still takes session_id"
- def test_open_session_holds_no_per_request_params(self):
- """open_session keeps browser-level params only, none of the per-request fetch keys"""
- params = set(inspect.signature(ScraplingMCPServer.open_session).parameters)
+ def test_browser_open_holds_no_per_request_params(self):
+ """browser_open keeps browser-level params only, none of the per-request fetch keys"""
+ params = set(inspect.signature(ScraplingMCPServer.browser_open).parameters)
assert params.isdisjoint(_STEALTH_FETCH_KEYS), (
- f"open_session still carries per-request params: {params & set(_STEALTH_FETCH_KEYS)}"
+ f"browser_open still carries per-request params: {params & set(_STEALTH_FETCH_KEYS)}"
)
def test_proxy_is_session_level_not_per_request(self):
- """A session runs one tab, so proxy is set once on open_session, never per request"""
- assert "proxy" in inspect.signature(ScraplingMCPServer.open_session).parameters
- assert "proxy" not in inspect.signature(ScraplingMCPServer.session_fetch).parameters
+ """A session runs one tab, so proxy is set once on browser_open, never per request"""
+ assert "proxy" in inspect.signature(ScraplingMCPServer.browser_open).parameters
+ assert "proxy" not in inspect.signature(ScraplingMCPServer.browser_fetch).parameters
assert "proxy" not in _STEALTH_FETCH_KEYS
@pytest.mark.asyncio
- async def test_open_session_forwards_proxy_to_the_session(self, monkeypatch):
+ async def test_browser_open_forwards_proxy_to_the_session(self, monkeypatch):
"""The session-level proxy reaches the underlying session so it applies to every fetch"""
monkeypatch.setattr("scrapling.core.ai.AsyncDynamicSession", _FakeDynamicSession)
_FakeDynamicSession.instances = []
server = ScraplingMCPServer()
- await server.open_session(session_type="dynamic", proxy="http://user:pass@host:8080")
+ await server.browser_open(session_type="dynamic", proxy="http://user:pass@host:8080")
assert _FakeDynamicSession.instances[0].kwargs["proxy"] == "http://user:pass@host:8080"
class TestSessionSettingsReceipt:
- """open_session and list_sessions return the session's effective settings."""
+ """browser_open and list_sessions return the session's effective settings."""
def test_session_settings_extracts_json_safe_fields(self):
"""The helper keeps JSON primitives and drops the rest (callables, structs, sequences)"""
@@ -648,12 +705,12 @@ class TestSessionSettingsReceipt:
assert all(isinstance(v, (str, int, float, bool)) or v is None for v in settings.values()), settings
@pytest.mark.asyncio
- async def test_open_session_and_list_report_the_receipt(self, monkeypatch):
- """open_session returns the receipt and list_sessions reports the same one"""
+ async def test_browser_open_and_list_report_the_receipt(self, monkeypatch):
+ """browser_open returns the receipt and list_sessions reports the same one"""
monkeypatch.setattr("scrapling.core.ai.AsyncDynamicSession", _FakeDynamicSession)
server = ScraplingMCPServer()
- created = await server.open_session(session_type="dynamic")
+ created = await server.browser_open(session_type="dynamic")
assert created.settings["headless"] is True
listed = await server.list_sessions()
@@ -705,9 +762,9 @@ class TestScreenshot:
@pytest.mark.asyncio
async def test_screenshot_png_with_dynamic_session(self, server, test_url):
"""PNG screenshot via a dynamic session returns image and url content blocks"""
- opened = await server.open_session(session_type="dynamic", headless=True)
+ opened = await server.browser_open(session_type="dynamic", headless=True)
try:
- result = await server.screenshot(url=test_url, session_id=opened.session_id)
+ result = await server.browser_screenshot(url=test_url, session_id=opened.session_id)
assert isinstance(result, list) and len(result) == 2
assert isinstance(result[0], ImageContent)
assert result[0].mime_type == "image/png"
@@ -719,9 +776,11 @@ class TestScreenshot:
@pytest.mark.asyncio
async def test_screenshot_jpeg_with_quality(self, server, test_url):
"""JPEG screenshot with quality parameter via a dynamic session"""
- opened = await server.open_session(session_type="dynamic", headless=True)
+ opened = await server.browser_open(session_type="dynamic", headless=True)
try:
- result = await server.screenshot(url=test_url, session_id=opened.session_id, image_type="jpeg", quality=80)
+ result = await server.browser_screenshot(
+ url=test_url, session_id=opened.session_id, image_type="jpeg", quality=80
+ )
assert isinstance(result[0], ImageContent)
assert result[0].mime_type == "image/jpeg"
finally:
@@ -730,9 +789,9 @@ class TestScreenshot:
@pytest.mark.asyncio
async def test_screenshot_with_stealthy_session(self, server, test_url):
"""PNG screenshot via a stealthy session"""
- opened = await server.open_session(session_type="stealthy", headless=True)
+ opened = await server.browser_open(session_type="stealthy", headless=True)
try:
- result = await server.screenshot(url=test_url, session_id=opened.session_id)
+ result = await server.browser_screenshot(url=test_url, session_id=opened.session_id)
assert isinstance(result[0], ImageContent)
assert result[0].mime_type == "image/png"
finally:
@@ -743,10 +802,14 @@ class TestScreenshot:
"""full_page=True produces an image taller than the viewport-only capture"""
tall_html = b"<html><body><div style='height:5000px;background:#abc'></div></body></html>"
with _serve_html(tall_html) as tall_url:
- opened = await server.open_session(session_type="dynamic", headless=True)
+ opened = await server.browser_open(session_type="dynamic", headless=True)
try:
- viewport_result = await server.screenshot(url=tall_url, session_id=opened.session_id, full_page=False)
- full_result = await server.screenshot(url=tall_url, session_id=opened.session_id, full_page=True)
+ viewport_result = await server.browser_screenshot(
+ url=tall_url, session_id=opened.session_id, full_page=False
+ )
+ full_result = await server.browser_screenshot(
+ url=tall_url, session_id=opened.session_id, full_page=True
+ )
viewport_png = base64.b64decode(viewport_result[0].data)
full_png = base64.b64decode(full_result[0].data)
@@ -759,15 +822,17 @@ class TestScreenshot:
async def test_screenshot_invalid_session_id_raises(self, server, test_url):
"""Unknown session_id raises ValueError"""
with pytest.raises(ValueError, match="not found"):
- await server.screenshot(url=test_url, session_id="does-not-exist")
+ await server.browser_screenshot(url=test_url, session_id="does-not-exist")
@pytest.mark.asyncio
async def test_screenshot_quality_with_png_raises(self, server, test_url):
"""quality is rejected when image_type is png"""
- opened = await server.open_session(session_type="dynamic", headless=True)
+ opened = await server.browser_open(session_type="dynamic", headless=True)
try:
with pytest.raises(ValueError, match="quality"):
- await server.screenshot(url=test_url, session_id=opened.session_id, image_type="png", quality=90)
+ await server.browser_screenshot(
+ url=test_url, session_id=opened.session_id, image_type="png", quality=90
+ )
finally:
await server.close_session(opened.session_id)
@@ -918,15 +983,18 @@ class TestServerToolRegistration:
async with Client(server) as client:
assert client.instructions
tools = {tool.name: tool for tool in (await client.list_tools()).tools}
+ for name in ("open_session", "session_fetch", "screenshot"):
+ result = await client.call_tool(name, {})
+ assert result.is_error
assert len(tools) == 14
- assert tools["screenshot"].output_schema is None
+ assert tools["browser_screenshot"].output_schema is None
assert tools["browser_snapshot"].output_schema is None
- assert "session_snapshot" not in tools
+ assert {"session_snapshot", "open_session", "session_fetch", "screenshot"}.isdisjoint(tools)
assert all(
tool.output_schema is not None
for name, tool in tools.items()
- if name not in ("screenshot", "browser_snapshot")
+ if name not in ("browser_screenshot", "browser_snapshot")
)
@pytest.mark.asyncio
@@ -956,7 +1024,7 @@ class TestServerToolRegistration:
assert set(tools["session_make_request"].input_schema["required"]) >= {"url", "session_id"}
assert set(tools["open_request_session"].input_schema["properties"]) == {"session_id", "impersonate", "proxy"}
- session_props = tools["session_fetch"].input_schema["properties"]
+ session_props = tools["browser_fetch"].input_schema["properties"]
assert session_props["timeout"]["default"] == 30000
assert session_props["pierce_shadow"]["type"] == "boolean"
assert session_props["pierce_shadow"]["default"] is False
@@ -964,11 +1032,11 @@ class TestServerToolRegistration:
assert session_props["extraction_type"]["default"] == "markdown"
assert set(session_props["extraction_type"]["enum"]) == {"markdown", "html", "text", "snapshot"}
assert {"depth", "boxes"}.isdisjoint(session_props)
- assert set(tools["session_fetch"].input_schema["required"]) >= {"url", "session_id"}
+ assert set(tools["browser_fetch"].input_schema["required"]) >= {"url", "session_id"}
- open_props = set(tools["open_session"].input_schema["properties"])
+ open_props = set(tools["browser_open"].input_schema["properties"])
assert open_props.isdisjoint(_STEALTH_FETCH_KEYS), (
- f"open_session still exposes per-request params: {open_props & set(_STEALTH_FETCH_KEYS)}"
+ f"browser_open still exposes per-request params: {open_props & set(_STEALTH_FETCH_KEYS)}"
)
@pytest.mark.asyncio
@@ -994,14 +1062,14 @@ class TestServerToolRegistration:
"bulk_fetch",
"stealthy_fetch",
"bulk_stealthy_fetch",
- "session_fetch",
+ "browser_fetch",
"session_make_request",
"browser_snapshot",
- "screenshot",
+ "browser_screenshot",
):
assert annotations[name].read_only_hint is True
assert annotations[name].open_world_hint is True
- for name in ("open_session", "open_request_session", "close_session"):
+ for name in ("browser_open", "open_request_session", "close_session"):
assert annotations[name].read_only_hint is False
assert annotations[name].destructive_hint is False
assert annotations[name].open_world_hint is True
@@ -1035,11 +1103,11 @@ async def test_shadow_option_changes_per_session_request(monkeypatch, session_ty
monkeypatch.setattr("scrapling.core.ai." + target, fake)
server = ScraplingMCPServer()._build_server("127.0.0.1", 8000)
async with Client(server) as client:
- await client.call_tool("open_session", {"session_type": session_type, "session_id": "shadow-test"})
+ await client.call_tool("browser_open", {"session_type": session_type, "session_id": "shadow-test"})
try:
for args, expected in [({"pierce_shadow": True}, True), ({"pierce_shadow": False}, False), ({}, False)]:
result = await client.call_tool(
- "session_fetch", {"url": "https://example.com", "session_id": "shadow-test", **args}
+ "browser_fetch", {"url": "https://example.com", "session_id": "shadow-test", **args}
)
assert not result.is_error
assert fake.instances[-1].fetch_calls[-1]["pierce_shadow"] is expected
diff --git a/tests/ai/test_browser_snapshot.py b/tests/ai/test_browser_snapshot.py
index 9136882..b848cd4 100644
--- a/tests/ai/test_browser_snapshot.py
+++ b/tests/ai/test_browser_snapshot.py
@@ -45,7 +45,7 @@ async def test_browser_snapshot_returns_plain_mcp_text(session_type: SessionType
assert tool.output_schema is None
assert set(tool.input_schema["properties"]) == {"session_id", "depth", "boxes"}
assert tool.input_schema["required"] == ["session_id"]
- fetch_props = tools["session_fetch"].input_schema["properties"]
+ fetch_props = tools["browser_fetch"].input_schema["properties"]
assert set(fetch_props["extraction_type"]["enum"]) == {"markdown", "html", "text", "snapshot"}
assert "depth" not in fetch_props and "boxes" not in fetch_props
assert "snapshot" not in tools["fetch"].input_schema["properties"]["extraction_type"]["enum"]
@@ -70,7 +70,7 @@ async def test_browser_snapshot_returns_plain_mcp_text(session_type: SessionType
("missing", "not found"),
("dead", "no longer alive"),
("static", "'static'"),
- ("empty", "session_fetch"),
+ ("empty", "browser_fetch"),
("busy", "busy"),
("closed", "closed"),
],
@@ -128,13 +128,13 @@ async def test_browser_snapshot_live_mcp_round_trip(session_type: SessionType, c
)
async with Client(server._build_server("127.0.0.1", 8000)) as client:
- opened = await client.call_tool("open_session", {"session_type": session_type, "session_id": "browser"})
+ opened = await client.call_tool("browser_open", {"session_type": session_type, "session_id": "browser"})
assert not opened.is_error
try:
session = server._sessions["browser"].session
await session.context.route("**/*", serve)
fetched = await client.call_tool(
- "session_fetch",
+ "browser_fetch",
{
"session_id": "browser",
"url": "https://snapshot.test/",
@@ -180,7 +180,7 @@ async def test_browser_snapshot_live_mcp_round_trip(session_type: SessionType, c
for selector in ("#missing", "button", "["):
previous_count = len(requests)
failed = await client.call_tool(
- "session_fetch",
+ "browser_fetch",
{
"session_id": "browser",
"url": "https://snapshot.test/",
@@ -228,7 +228,7 @@ async def test_browser_snapshot_reserves_and_releases_the_page_on_cancel() -> No
@pytest.mark.parametrize(
"extraction_type, expected", [("markdown", ["Hello", ""]), ("html", ["<p>Hello</p>", ""]), ("text", ["Hello", ""])]
)
-async def test_session_fetch_keeps_existing_formats(extraction_type: Any, expected: list[str]) -> None:
+async def test_browser_fetch_keeps_existing_formats(extraction_type: Any, expected: list[str]) -> None:
server, session, page = _server()
response = Response(
url="https://snapshot.test/final",
@@ -241,7 +241,7 @@ async def test_session_fetch_keeps_existing_formats(extraction_type: Any, expect
)
fetch = AsyncMock(return_value=response)
setattr(session, "fetch", fetch)
- result = await server.session_fetch("https://snapshot.test/", "browser", extraction_type, "p")
+ result = await server.browser_fetch("https://snapshot.test/", "browser", extraction_type, "p")
assert result.content == expected
assert result.status == 201
assert result.url == "https://snapshot.test/final"
@@ -250,7 +250,7 @@ async def test_session_fetch_keeps_existing_formats(extraction_type: Any, expect
@pytest.mark.asyncio
-async def test_session_fetch_snapshot_keeps_raw_content_and_response_metadata() -> None:
+async def test_browser_fetch_snapshot_keeps_raw_content_and_response_metadata() -> None:
server, session, page = _server()
snapshot = '\n- button "Save" [ref=e2]\n'
page.aria_snapshot.return_value = snapshot
@@ -265,7 +265,7 @@ async def test_session_fetch_snapshot_keeps_raw_content_and_response_metadata()
)
fetch = AsyncMock(return_value=response)
setattr(session, "fetch", fetch)
- result = await server.session_fetch("https://snapshot.test/start", "browser", extraction_type="snapshot")
+ result = await server.browser_fetch("https://snapshot.test/start", "browser", extraction_type="snapshot")
assert result.content == [snapshot]
assert result.status == 202
assert result.url == "https://snapshot.test/redirected"
@@ -274,7 +274,7 @@ async def test_session_fetch_snapshot_keeps_raw_content_and_response_metadata()
@pytest.mark.asyncio
-async def test_session_fetch_snapshot_error_does_not_refetch() -> None:
+async def test_browser_fetch_snapshot_error_does_not_refetch() -> None:
server, session, page = _server()
response = Response(
url="https://snapshot.test/final",
@@ -289,7 +289,7 @@ async def test_session_fetch_snapshot_error_does_not_refetch() -> None:
setattr(session, "fetch", fetch)
page.locator.return_value.aria_snapshot = AsyncMock(side_effect=TimeoutError("snapshot timeout"))
with pytest.raises(TimeoutError, match="snapshot timeout"):
- await server.session_fetch("https://snapshot.test/", "browser", extraction_type="snapshot", css_selector="main")
+ await server.browser_fetch("https://snapshot.test/", "browser", extraction_type="snapshot", css_selector="main")
fetch.assert_awaited_once()
assert {"extraction_type", "css_selector", "depth", "boxes"}.isdisjoint(fetch.call_args.kwargs)
page.aria_snapshot.assert_not_awaited()