Skip to content

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

docs: update docs and skill with the new changes

Karim shoairdc6a8b248057
diff --git a/agent-skill/Scrapling-Skill.zip b/agent-skill/Scrapling-Skill.zip
index c8fbe3e..bb3d2ff 100644
Binary files a/agent-skill/Scrapling-Skill.zip and b/agent-skill/Scrapling-Skill.zip differ
diff --git a/agent-skill/Scrapling-Skill/SKILL.md b/agent-skill/Scrapling-Skill/SKILL.md
index 5d2b9ca..9658a9d 100644
--- a/agent-skill/Scrapling-Skill/SKILL.md
+++ b/agent-skill/Scrapling-Skill/SKILL.md
@@ -255,7 +255,7 @@ data = page.css('.quote .text::text').getall()
### Shadow DOM
-Use `pierce_shadow=True` with dynamic or stealth Python fetchers and sessions to include open shadow roots. It defaults to `False` and can be overridden per request. For MCP, pass `pierce_shadow=true` to `fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`, or each `browser_fetch` call.
+Use `pierce_shadow=True` with dynamic or stealth Python fetchers and sessions to include open shadow roots. It defaults to `False` and can be overridden per request. For MCP, pass `pierce_shadow=true` to `browser_fetch_once`, `browser_fetch_many_once`, `browser_stealth_fetch_once`, `browser_stealth_fetch_many_once`, or each `browser_fetch` call.
Both browser CLI commands accept `--pierce-shadow` and `--no-pierce-shadow`:
@@ -427,4 +427,4 @@ This skill encapsulates almost all the published documentation in Markdown, so d
- Respect robots.txt and ToS. Use `robots_txt_obey = True` on spiders to enforce this automatically.
- Add delays (`download_delay`) for large crawls, or set `autothrottle_enabled = True` to let the spider pick the delay per domain and back off when the website starts blocking.
- Don't bypass paywalls or authentication without permission.
-- Never scrape personal/sensitive data.
\ No newline at end of file
+- Never scrape personal/sensitive data.
diff --git a/agent-skill/Scrapling-Skill/references/mcp-server.md b/agent-skill/Scrapling-Skill/references/mcp-server.md
index b96ecfa..e263c4e 100644
--- a/agent-skill/Scrapling-Skill/references/mcp-server.md
+++ b/agent-skill/Scrapling-Skill/references/mcp-server.md
@@ -1,12 +1,12 @@
# Scrapling MCP Server
-The Scrapling MCP server exposes fourteen tools over the MCP protocol. It supports CSS-selector-based content narrowing (reducing tokens by extracting only relevant elements before returning results), three levels of scraping capability (plain HTTP, browser-rendered, and stealth/anti-bot bypass), persistent browser session management, and page screenshots returned as real image content blocks. Fetch tools come in two modes: one-shot tools (`fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`) each launch and close their own browser, while `browser_fetch` and `session_make_request` work through sessions opened with `browser_open`/`open_request_session`.
+The Scrapling MCP server exposes sixteen tools over the MCP protocol. It supports CSS-selector-based content narrowing (reducing tokens by extracting only relevant elements before returning results), three levels of scraping capability (plain HTTP, browser-rendered, and stealth/anti-bot bypass), persistent browser session management, mouse actions, and page screenshots returned as real image content blocks. Fetch tools come in two modes: one-shot tools (`browser_fetch_once`, `browser_fetch_many_once`, `browser_stealth_fetch_once`, `browser_stealth_fetch_many_once`) each launch and close their own browser, while `browser_fetch` and `session_make_request` work through sessions opened with `browser_open`/`open_request_session`.
-Fetch and HTTP request tools return a `ResponseModel` with fields: `status` (int), `content` (list of strings), `url` (str). Bulk tools return a list of these responses. The `browser_screenshot` tool returns a list of MCP content blocks: an `ImageContent` (the screenshot bytes) followed by a `TextContent` (the post-redirect URL). `browser_snapshot` returns plain text.
+Fetch and HTTP request tools return a `ResponseModel` with fields: `status` (int), `content` (list of strings), `url` (str). Bulk tools return a list of these responses. The `browser_screenshot` tool returns a list of MCP content blocks: an `ImageContent` (the screenshot bytes) followed by a `TextContent` (the post-redirect URL). `browser_snapshot`, `browser_mouse_move`, and `browser_click` return plain text.
## Shadow DOM
-Set `pierce_shadow=true` on `fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`, or `browser_fetch` to include open Shadow DOM content. It defaults to `false`. For persistent sessions, pass it on each `browser_fetch` call. See [Shadow DOM](fetching/dynamic.md#shadow-dom) for selector examples and limits.
+Set `pierce_shadow=true` on `browser_fetch_once`, `browser_fetch_many_once`, `browser_stealth_fetch_once`, `browser_stealth_fetch_many_once`, or `browser_fetch` to include open Shadow DOM content. It defaults to `false`. For persistent sessions, pass it on each `browser_fetch` call. See [Shadow DOM](fetching/dynamic.md#shadow-dom) for selector examples and limits.
## One-shot tools
@@ -46,7 +46,7 @@ Fast HTTP request with browser fingerprint impersonation (TLS, headers). Support
Async concurrent GET-only version of `make_request`. Same parameters except `url` is replaced by `urls` (list of strings) and there are no `method`/`data`/`json` parameters. All URLs are fetched in parallel. Returns a list of `ResponseModel`.
-### `fetch` -- Browser fetch (single URL)
+### `browser_fetch_once` -- Browser fetch (single URL)
Opens a Chromium browser via Playwright to render JavaScript. Suitable for dynamic/SPA pages with no/low bot protection.
@@ -78,15 +78,15 @@ Opens a Chromium browser via Playwright to render JavaScript. Suitable for dynam
This is a one-shot tool: it always launches its own browser. To fetch through a persistent session, use `browser_fetch`.
-### `bulk_fetch` -- Browser fetch (multiple URLs)
+### `browser_fetch_many_once` -- Browser fetch (multiple URLs)
-Concurrent browser version of `fetch`. Same parameters except `url` is replaced by `urls` (list of strings). Each URL opens in a separate browser tab. Returns a list of `ResponseModel`.
+Concurrent browser version of `browser_fetch_once`. Same parameters except `url` is replaced by `urls` (list of strings). Each URL opens in a separate browser tab. Returns a list of `ResponseModel`.
-### `stealthy_fetch` -- Stealth browser fetch (single URL)
+### `browser_stealth_fetch_once` -- Stealth browser fetch (single URL)
Anti-bot bypass fetcher with fingerprint spoofing. Use this for sites with Cloudflare Turnstile/Interstitial or other strong protections.
-**Additional parameters (beyond those in `fetch`):**
+**Additional parameters (beyond those in `browser_fetch_once`):**
| Parameter | Type | Default | Description |
|--------------------|--------------|---------|------------------------------------------------------------------|
@@ -96,11 +96,11 @@ Anti-bot bypass fetcher with fingerprint spoofing. Use this for sites with Cloud
| `allow_webgl` | bool | true | Keep WebGL enabled (disabling is detectable by WAFs) |
| `additional_args` | dict or null | null | Extra Playwright context args (overrides Scrapling defaults) |
-All parameters from `fetch` are also accepted. Like `fetch`, this is a one-shot tool that launches its own browser; use `browser_fetch` for a stealthy session.
+All parameters from `browser_fetch_once` are also accepted. Like `browser_fetch_once`, this is a one-shot tool that launches its own browser; use `browser_fetch` for a stealthy session.
-### `bulk_stealthy_fetch` -- Stealth browser fetch (multiple URLs)
+### `browser_stealth_fetch_many_once` -- Stealth browser fetch (multiple URLs)
-Concurrent stealth version. Same parameters as `stealthy_fetch` except `url` is replaced by `urls` (list of strings). Returns a list of `ResponseModel`.
+Concurrent stealth version. Same parameters as `browser_stealth_fetch_once` except `url` is replaced by `urls` (list of strings). Returns a list of `ResponseModel`.
## Session tools
@@ -157,11 +157,44 @@ Fetches one URL through a browser session opened with `browser_open` (dynamic or
| `blocked_domains` | list or null | null | Domain names to block for this request (subdomains matched too) |
| `solve_cloudflare` | bool | false | (Stealthy sessions only) Auto-solve Cloudflare challenges; errors on a dynamic session |
-With `extraction_type="snapshot"`, the response keeps `status` and `url` and returns one unchanged AI ARIA snapshot string in `content`. `css_selector` must match exactly one element; omit it for the whole page. `main_content_only` and `pierce_shadow` do not filter snapshots. Snapshot extraction is only available on `browser_fetch`.
+With `extraction_type="snapshot"`, the response keeps `status` and `url` and returns one unchanged AI ARIA snapshot string in `content`. These snapshots always include element positions and sizes in viewport CSS pixels. `css_selector` must match exactly one element; omit it for the whole page. `main_content_only` and `pierce_shadow` do not filter snapshots. Snapshot extraction is only available on `browser_fetch`.
### `browser_snapshot` -- Read the current page without navigation
-Returns a plain-text AI ARIA snapshot of the current whole page, including element roles, names, and references. Takes `session_id` from `browser_open`, optional `depth` to limit the tree, and `boxes` (default `false`) to include element positions and sizes in viewport CSS pixels. Use it after `browser_fetch` finishes. Raises for an unknown or HTTP session, or a missing, closed, or busy page.
+Returns a plain-text AI ARIA snapshot of the current whole page, including element roles, names, and references. Takes `session_id` from `browser_open` and optional `depth` to limit the tree. Element positions and sizes in viewport CSS pixels are included by default; set `boxes=false` to omit them. Use it after `browser_fetch` finishes. Raises for an unknown or HTTP session, or a missing, closed, or busy page.
+
+### `browser_mouse_move` -- Move the mouse on the current page
+
+Moves the native mouse on the existing page in a dynamic or stealthy browser session. Call `browser_fetch` first, then use `browser_snapshot` to find coordinates.
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `session_id` | str | required | ID of an open browser session |
+| `x`, `y` | number | required | Finite CSS pixel coordinates from the main frame viewport's top-left |
+| `steps` | int | 1 | Number of mousemove events, at least 1 |
+
+Coordinates are viewport CSS pixels, not full-page screenshot coordinates. Returns plain text without an automatic snapshot. Invalid sessions and missing, closed, or busy pages return an error.
+
+### `browser_click` -- Click by selector, snapshot reference, or coordinates
+
+Clicks on the existing page in a dynamic or stealthy browser session. Call `browser_fetch` first. Supply exactly one target: `selector`, `ref`, or both `x` and `y`.
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `session_id` | str | required | ID of an open browser session |
+| `selector` | str or null | null | Nonempty Playwright selector, such as `button[type="submit"]` or `xpath=//button[@type="submit"]` |
+| `ref` | str or null | null | Nonempty reference from the current `browser_snapshot`, such as `e2` |
+| `x`, `y` | number or null | null | Finite CSS pixel coordinates from the main frame viewport's top-left; supply both |
+| `button` | `"left"` / `"right"` / `"middle"` | `"left"` | Button to click |
+| `click_count` | int | 1 | Number of clicks, at least 1; use 2 for a double-click |
+| `delay` | number | 0 | Finite nonnegative milliseconds between press and release |
+| `timeout` | number | 30000 | Finite nonnegative timeout in milliseconds for selector/reference clicks; 0 disables the limit |
+
+Selector and reference clicks use Playwright's locator click, which waits for the target to be ready and scrolls it into view. A selector must match exactly one element. Pass the exact snapshot reference value, such as `ref="e2"`. Take a new snapshot if the reference is stale. Coordinate clicks move the mouse before pressing and releasing the button, with no scroll or navigation wait. `timeout` does not apply to coordinate clicks.
+
+Invalid targets, unknown sessions, and missing, closed, or busy pages return an error.
+
+Returns plain text without an automatic snapshot. Use `browser_snapshot` to inspect the page afterward. A cancelled or timed-out click sends a button release before the page can be reused, even if it was still waiting for a target. This does not undo page actions. Do not repeat a failed click without checking the page state.
### `session_make_request` -- HTTP request through an open requests session
@@ -208,16 +241,18 @@ Requires an open browser session. Call `browser_open` first, then pass the `sess
|------------------------------------------|---------------------------------------------------------------|
| Static page, no bot protection | `make_request` |
| Multiple static pages | `bulk_get` |
-| JavaScript-rendered / SPA page | `fetch` |
-| Multiple JS-rendered pages | `bulk_fetch` |
-| Cloudflare or strong anti-bot protection | `stealthy_fetch` (with `solve_cloudflare=true` for Turnstile) |
-| Multiple protected pages | `bulk_stealthy_fetch` |
+| JavaScript-rendered / SPA page | `browser_fetch_once` |
+| Multiple JS-rendered pages | `browser_fetch_many_once` |
+| Cloudflare or strong anti-bot protection | `browser_stealth_fetch_once` (with `solve_cloudflare=true` for Turnstile) |
+| Multiple protected pages | `browser_stealth_fetch_many_once` |
| Multiple pages from the same site | `browser_open` + `browser_fetch` per page |
| Multiple plain HTTP requests to one site | `open_request_session` + `session_make_request` per request |
| Need a screenshot of a page | `browser_open` + `browser_screenshot` with `session_id` |
| Read the current page's AI ARIA snapshot | `browser_snapshot` with `session_id` |
+| Move the mouse on the current page | `browser_mouse_move` with `session_id` |
+| Click by selector, snapshot ref, or coordinates | `browser_click` with `session_id` |
-Start with `make_request` (fastest, lowest resource cost). Escalate to `fetch` if content requires JS rendering. Escalate to `stealthy_fetch` only if blocked. For multiple pages from the same site, use a persistent session to avoid browser launch overhead.
+Start with `make_request` (fastest, lowest resource cost). Escalate to `browser_fetch_once` if content requires JS rendering. Escalate to `browser_stealth_fetch_once` only if blocked. For multiple pages from the same site, use a persistent session to avoid browser launch overhead.
## Content extraction tips
@@ -240,7 +275,7 @@ Keep `main_content_only=true` for maximum protection.
## Ad blocking
-All browser-based tools (`fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`) and persistent sessions (`browser_open`) automatically block requests to ~3,500 known ad and tracker domains. This is always enabled in the MCP server to save tokens and speed up page loads. No configuration needed.
+All browser-based tools (`browser_fetch_once`, `browser_fetch_many_once`, `browser_stealth_fetch_once`, `browser_stealth_fetch_many_once`) and persistent sessions (`browser_open`) automatically block requests to ~3,500 known ad and tracker domains. This is always enabled in the MCP server to save tokens and speed up page loads. No configuration needed.
## Setup
@@ -276,7 +311,7 @@ docker run -p 8000:8000 -e SCRAPLING_MCP_AUTH_TOKEN="<your-token>" pyd4vinci/scr
## Custom browser executable
-Browser-based tools (`fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`, and `browser_open`) can use a custom Chromium-compatible browser executable instead of the bundled Chromium. This is useful for custom browser builds or lightweight browser engines.
+Browser-based tools (`browser_fetch_once`, `browser_fetch_many_once`, `browser_stealth_fetch_once`, `browser_stealth_fetch_many_once`, and `browser_open`) can use a custom Chromium-compatible browser executable instead of the bundled Chromium. This is useful for custom browser builds or lightweight browser engines.
To configure it once for the whole MCP server, pass the executable path when starting the server:
@@ -306,7 +341,7 @@ The MCP server name when registering with a client is `ScraplingServer`. The com
## Connecting to remote browsers
-`browser_open` doesn't have to launch a browser locally. Pass a `cdp_url` and it connects to an already-running browser through the Chrome DevTools Protocol, whether that browser is on the same machine, another host, or a managed browser provider. Both session types (`dynamic` and `stealthy`) accept it, and the `session_id` you get back is used with `browser_fetch`, `browser_snapshot`, and `browser_screenshot` as usual.
+`browser_open` doesn't have to launch a browser locally. Pass a `cdp_url` and it connects to an already-running browser through the Chrome DevTools Protocol, whether that browser is on the same machine, another host, or a managed browser provider. Both session types (`dynamic` and `stealthy`) accept it, and the `session_id` you get back is used with `browser_fetch`, `browser_snapshot`, `browser_mouse_move`, `browser_click`, and `browser_screenshot` as usual.
The URL can be a WebSocket endpoint (`ws://`/`wss://`), which is what managed browser providers hand out, or the HTTP endpoint of a browser started with `--remote-debugging-port=9222`, reached as `cdp_url="http://localhost:9222"`.
@@ -365,4 +400,4 @@ scrapling-mcp --http --allowed-host 'your-server.example.com:8000'
- Plain HTTP sends the token in cleartext, so put the server behind a reverse proxy that terminates TLS before exposing it to the internet.
- This is a single shared key, not per-client credentials, so every client uses the same token and rotating it means restarting the server.
- Starting with `--http --no-auth` still logs a warning that it's unauthenticated.
-- Passing both `--auth-token` and `--no-auth` keeps the token, so the server stays authenticated instead of quietly dropping it.
\ No newline at end of file
+- Passing both `--auth-token` and `--no-auth` keeps the token, so the server stays authenticated instead of quietly dropping it.
diff --git a/docs/ai/mcp-server.md b/docs/ai/mcp-server.md
index ee9748c..39b3055 100644
--- a/docs/ai/mcp-server.md
+++ b/docs/ai/mcp-server.md
@@ -6,7 +6,7 @@ The **Scrapling MCP Server** is a new feature that brings Scrapling's powerful W
## Features
-The Scrapling MCP Server provides fourteen powerful tools for web scraping, split into two modes: one-shot tools that each launch and close their own browser/client, and session tools that open a browser or an HTTP session once and then work through it.
+The Scrapling MCP Server provides sixteen tools for web scraping and browser interaction, split into two modes: one-shot tools that each launch and close their own browser/client, and session tools that open a browser or an HTTP session once and then work through it.
### One-shot tools
@@ -15,12 +15,12 @@ The Scrapling MCP Server provides fourteen powerful tools for web scraping, spli
- **`bulk_get`**: An async GET-only version of the above tool that allows scraping of multiple URLs at the same time!
#### 🌐 Dynamic Content Scraping
-- **`fetch`**: Rapidly fetch dynamic content with Chromium/Chrome browser with complete control over the request/browser, and more!
-- **`bulk_fetch`**: An async version of the above tool that allows scraping of multiple URLs in different browser tabs at the same time!
+- **`browser_fetch_once`**: Rapidly fetch dynamic content with Chromium/Chrome browser with complete control over the request/browser, and more!
+- **`browser_fetch_many_once`**: An async version of the above tool that allows scraping of multiple URLs in different browser tabs at the same time!
#### 🔒 Stealth Scraping
-- **`stealthy_fetch`**: Uses our Stealthy browser to bypass Cloudflare Turnstile/Interstitial and other anti-bot systems with complete control over the request/browser!
-- **`bulk_stealthy_fetch`**: An async version of the above tool that allows stealth scraping of multiple URLs in different browser tabs at the same time!
+- **`browser_stealth_fetch_once`**: Uses our Stealthy browser to bypass Cloudflare Turnstile/Interstitial and other anti-bot systems with complete control over the request/browser!
+- **`browser_stealth_fetch_many_once`**: An async version of the above tool that allows stealth scraping of multiple URLs in different browser tabs at the same time!
### Session tools
@@ -31,7 +31,7 @@ The Scrapling MCP Server provides fourteen powerful tools for web scraping, spli
- **`list_sessions`**: List all active sessions with their details and `settings`.
#### 🎯 Fetching Through a Session
-- **`browser_fetch`**: Fetch a single URL through an open browser session (dynamic or stealthy), carrying the per-request options for that call. This is the session counterpart of `fetch`/`stealthy_fetch`. Set `extraction_type="snapshot"` to return an AI ARIA snapshot in the usual response's `content` list.
+- **`browser_fetch`**: Fetch a single URL through an open browser session (dynamic or stealthy), carrying the per-request options for that call. This is the session counterpart of `browser_fetch_once`/`browser_stealth_fetch_once`. It can also capture a structured page snapshot with element references, positions, and sizes, helping the AI identify controls and plan mouse actions.
- **`session_make_request`**: Make an HTTP request with any method through a session opened with `open_request_session`, reusing its cookies, connections, and browser fingerprint. This is the session counterpart of `make_request`.
#### 📸 Screenshots
@@ -39,11 +39,16 @@ The Scrapling MCP Server provides fourteen powerful tools for web scraping, spli
#### AI Snapshots
-- **`browser_snapshot`**: Read the current whole page in an open browser session as an AI ARIA snapshot. Returns plain text with element roles, names, and references without loading the page again. Supports `depth` and `boxes`.
+- **`browser_snapshot`**: Give the AI a structured text view of the current page without reloading it. Includes element roles, names, references, positions, and sizes to help the AI understand the page and target clicks.
+
+#### Mouse Actions
+
+- **`browser_mouse_move`**: Move the browser's mouse to reveal hover menus, tooltips, and other content that appears on hover. Supports movement in multiple steps for controls that react as the pointer moves.
+- **`browser_click`**: Click buttons, follow links, and use other page controls through selectors, snapshot references, or page coordinates. Supports left, right, and middle clicks, plus double-clicks.
### Shadow DOM
-Set `pierce_shadow=true` on `fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`, or `browser_fetch` to include open Shadow DOM content. It defaults to `false`. For persistent sessions, pass it on each `browser_fetch` call. See [Shadow DOM](../fetching/dynamic.md#shadow-dom) for selector examples and limits.
+Set `pierce_shadow=true` on `browser_fetch_once`, `browser_fetch_many_once`, `browser_stealth_fetch_once`, `browser_stealth_fetch_many_once`, or `browser_fetch` to include open Shadow DOM content. It defaults to `false`. For persistent sessions, pass it on each `browser_fetch` call. See [Shadow DOM](../fetching/dynamic.md#shadow-dom) for selector examples and limits.
### Key Capabilities
- **Smart Content Extraction**: Convert web pages/elements to Markdown, HTML, or extract a clean version of the text content
@@ -64,14 +69,6 @@ The way other servers work is that they extract the content, then pass it all to
If you don't know how to write/use CSS selectors, don't worry. You can tell the AI in the prompt to write selectors to match possible fields for you and watch it try different combinations until it finds the right one, as we will show in the examples section.
-## Breaking changes
-
-Since version 0.4.15, the MCP server is reworked. If you are upgrading, note:
-
-1. **The Streamable HTTP transport now requires authentication and binds to localhost.** `scrapling-mcp --http` on its own refuses to start; pass `--auth-token` (or the `SCRAPLING_MCP_AUTH_TOKEN` environment variable), or `--no-auth` to serve it unauthenticated on purpose. The default host is now `127.0.0.1` instead of `0.0.0.0`; pass `--host 0.0.0.0` to accept connections from the network.
-2. **The one-shot fetch tools no longer accept `session_id`.** `fetch`, `bulk_fetch`, `stealthy_fetch`, and `bulk_stealthy_fetch` always launch their own browser. To fetch through a session, use the new **`browser_fetch`** tool (one URL per call, works with dynamic and stealthy sessions).
-3. **`browser_open` takes browser-level parameters only.** The per-request options (`wait`, `timeout`, `google_search`, `network_idle`, `disable_resources`, `wait_selector`, `wait_selector_state`, `extra_headers`, `solve_cloudflare`) moved to `browser_fetch` and are supplied on each call. `proxy` stays on `browser_open` (it applies to the whole session, which runs a single tab). `max_pages` was removed too, since a session manages a single page per call now.
-4. **The `get` tool is renamed to `make_request`.** It now takes a `method` parameter and supports GET (default), POST, PUT, and DELETE, with `data`/`json` for request bodies. `bulk_get` is unchanged.
## Installation
@@ -176,7 +173,7 @@ Then, after you've added the server, you need to completely quit and restart the
### Custom Browser Executable
-Browser-based tools (`fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`, and `browser_open`) can use a custom Chromium-compatible browser executable instead of the bundled Chromium. This is useful for custom browser builds or lightweight browser engines.
+Browser-based tools (`browser_fetch_once`, `browser_fetch_many_once`, `browser_stealth_fetch_once`, `browser_stealth_fetch_many_once`, and `browser_open`) can use a custom Chromium-compatible browser executable instead of the bundled Chromium. This is useful for custom browser builds or lightweight browser engines.
To configure it once for the whole MCP server, pass the executable path when starting the server:
@@ -208,7 +205,7 @@ You can also set the `SCRAPLING_EXECUTABLE_PATH` environment variable before sta
```
Open a stealthy browser session on wss://cdp.provider.example/session/abc123, then use it to scrape the product details from https://shop.example.com. Close the session when you're done.
```
-Both browser session types (`dynamic` and `stealthy`) accept it, and the `session_id` you get back is used with `browser_fetch`, `browser_snapshot`, and `browser_screenshot` as usual.
+Both browser session types (`dynamic` and `stealthy`) accept it, and the `session_id` you get back is used with `browser_fetch`, `browser_snapshot`, `browser_mouse_move`, `browser_click`, and `browser_screenshot` as usual.
The URL can be a WebSocket endpoint (`ws://`/`wss://`), which is what managed browser providers hand out, or the HTTP endpoint of a browser you started yourself with the remote debugging port enabled:
```commandline
@@ -330,7 +327,7 @@ We will gradually go from simple prompts to more complex ones. We will use Claud
Get the product names, prices, and descriptions from each page.
```
- Claude will use `bulk_fetch` to concurrently scrape all URLs, then analyze the extracted data.
+ Claude will use `browser_fetch_many_once` to concurrently scrape all URLs, then analyze the extracted data.
4. **More advanced workflow**
@@ -340,7 +337,7 @@ We will gradually go from simple prompts to more complex ones. We will use Claud
```
Note that I instructed it to use a bulk request for all the URLs collected. If I hadn't mentioned it, sometimes it works as intended, and other times it makes a separate request to each URL, which takes significantly longer. This prompt takes approximately one minute to complete.
- However, because I wasn't specific enough, it actually used the `stealthy_fetch` here and the `bulk_stealthy_fetch` in the second step, which unnecessarily consumed a large number of tokens. A better prompt would be:
+ However, because I wasn't specific enough, it actually used the `browser_stealth_fetch_once` here and the `browser_stealth_fetch_many_once` in the second step, which unnecessarily consumed a large number of tokens. A better prompt would be:
```
Use normal requests to extract the URLs of all games in this page, then do a bulk request to them and return a list of all action games: https://store.playstation.com/en-us/pages/browse
```
@@ -415,8 +412,8 @@ Here is some technical advice for you.
### 1. Choose the Right Tool
- **`make_request`**: Fast, simple websites
-- **`fetch`**: Sites with JavaScript/dynamic content
-- **`stealthy_fetch`**: Protected sites, Cloudflare, anti-bot systems
+- **`browser_fetch_once`**: Sites with JavaScript/dynamic content
+- **`browser_stealth_fetch_once`**: Protected sites, Cloudflare, anti-bot systems
### 2. Optimize Performance
- Use bulk tools for multiple URLs
@@ -449,7 +446,7 @@ This protection runs automatically for HTML, Markdown, and text extraction. Keep
- For multiple plain HTTP requests, use `open_request_session` instead and call `session_make_request` per request; it keeps cookies, connections, and the browser fingerprint (`impersonate`) across calls without a browser
- Sessions hold the session-level configuration set when opened (headless, locale, cookies, stealth toggles, etc. for browsers; `impersonate` and `proxy` for requests sessions); the per-request options (timeout, wait_selector, network_idle, solve_cloudflare, etc.) are passed to `browser_fetch`/`session_make_request` on each call, with their defaults shown in the tool schemas
- One `browser_fetch` works with both browser session types; `solve_cloudflare` only applies to a stealthy session and raises a clear error on a dynamic one
-- The one-shot tools (`make_request`, `bulk_get`, `fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`) never take a session
+- The one-shot tools (`make_request`, `bulk_get`, `browser_fetch_once`, `browser_fetch_many_once`, `browser_stealth_fetch_once`, `browser_stealth_fetch_many_once`) never take a session
- Always close sessions with `close_session` when done to free resources
- Use `list_sessions` to check which sessions are still active and see the `settings` each was created with (returned for the AI agent; empty for CDP sessions)
- Pass a custom `session_id` to the open tools to give sessions meaningful names (e.g. `"search"`, `"checkout"`) instead of the random hex default. They raise if the chosen ID is already in use, so you can detect collisions up front
@@ -459,7 +456,7 @@ This protection runs automatically for HTML, Markdown, and text extraction. Keep
- The image is returned as a real `ImageContent` block, not a base64 string in JSON, so the model sees the page directly
- Use `full_page=True` when you need everything below the fold; the default captures only the visible viewport
- Pick `image_type="jpeg"` with a `quality` value (0-100) for smaller payloads when pixel-perfect color isn't needed
-- The same `wait`, `wait_selector`, `network_idle`, and `timeout` controls used by `fetch` are available here too
+- The same `wait`, `wait_selector`, `network_idle`, and `timeout` controls used by `browser_fetch_once` are available here too
## Legal and Ethical Considerations
diff --git a/docs/api-reference/mcp-server.md b/docs/api-reference/mcp-server.md
index 82552ca..8efa07d 100644
--- a/docs/api-reference/mcp-server.md
+++ b/docs/api-reference/mcp-server.md
@@ -5,7 +5,7 @@ search:
# MCP Server API Reference
-The **Scrapling MCP Server** provides fourteen powerful tools for web scraping through the Model Context Protocol (MCP). This server integrates Scrapling's capabilities directly into AI chatbots and agents, allowing conversational web scraping with advanced anti-bot bypass features.
+The **Scrapling MCP Server** provides sixteen tools for web scraping and browser interaction through the Model Context Protocol (MCP). This server integrates Scrapling's capabilities directly into AI chatbots and agents, allowing conversational web scraping with advanced anti-bot bypass features.
You can start the MCP server by running:
@@ -30,7 +30,7 @@ server = ScraplingMCPServer(executable_path="/path/to/chromium")
## Response Model
-The standardized response structure returned by the fetch and HTTP request tools. Bulk tools return a list of these responses. Session management tools use the models below, `browser_screenshot` returns image and text content blocks, and `browser_snapshot` returns plain text.
+The standardized response structure returned by the fetch and HTTP request tools. Bulk tools return a list of these responses. Session management tools use the models below, `browser_screenshot` returns image and text content blocks, and `browser_snapshot`, `browser_mouse_move`, and `browser_click` return plain text.
## ::: scrapling.core.ai.ResponseModel
handler: python
@@ -56,9 +56,10 @@ Model classes for session management:
The main MCP server class that provides all web scraping tools:
-`browser_snapshot(session_id, depth=None, boxes=False)` returns a plain-text AI ARIA snapshot of the current whole page in a dynamic or stealthy session. Use it after `browser_fetch` finishes. It reads the existing page without navigation and includes element references. `depth` limits the tree depth, and `boxes=True` adds element positions and sizes in viewport CSS pixels. An unknown session, an HTTP request session, or a missing, closed, or busy page returns an error.
-
-`browser_fetch` also accepts `extraction_type="snapshot"` to take an AI ARIA snapshot after loading the requested URL. It returns the usual `status`, `url`, and `content` fields, with one unchanged snapshot string in `content`. The existing `css_selector` can scope the snapshot to exactly one element; zero or multiple matches return an error. Without a selector, the snapshot covers the whole page. `main_content_only` and `pierce_shadow` do not filter snapshots. This extraction type is only available on the MCP `browser_fetch` tool; browser fetchers still return the same `Response` object.
+- **`browser_snapshot`**: Inspect the current page without reloading it. Structured text includes element roles, names, references, positions, and sizes to help the AI target page controls.
+- **`browser_mouse_move`**: Reveal hover menus and tooltips with the browser's mouse, including movement in multiple steps.
+- **`browser_click`**: Click buttons, links, and other controls using selectors, snapshot references, or page coordinates. Supports left, right, and middle clicks, plus double-clicks.
+- **`browser_fetch`**: Fetch a page through an open browser session and extract HTML, Markdown, text, or a structured snapshot. Snapshots can cover the whole page or a chosen element and include element references, positions, and sizes for later interaction.
## ::: scrapling.core.ai.ScraplingMCPServer
handler: python