Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

6.8 KB
Raw Blame
---
id: remote-browser
title: "Remote browser services"
sidebar_label: "Remote browsers"
description: Connect Crawlee crawlers to remote browser services like Browser Use, Browserbase, Browserless, or Steel.
---

import ApiLink from '@site/src/components/ApiLink';
import CodeBlock from '@theme/CodeBlock';

import RemoteBrowserConfigSource from '!!raw-loader!./remote-browser-config.ts';
import RemoteBrowserProviderSource from '!!raw-loader!./remote-browser-provider.ts';
import RemoteBrowserPuppeteerSource from '!!raw-loader!./remote-browser-puppeteer.ts';

Instead of launching a local browser, Crawlee can connect to a remote browser service like [Browser Use](https://browser-use.com/), [Browserbase](https://browserbase.com/), [Browserless](https://browserless.io/), [Steel](https://steel.dev/), or any service that exposes a WebSocket/CDP endpoint. The crawler manages session rotation and the request lifecycle the same way it does locally — only the browser itself runs elsewhere.

Use this when you need IPs in specific regions, want to offload CPU/memory from your runner, or need stealth features the service provides.

## How it works

Set the crawler's `remoteBrowser` option with the connection details. The crawler builds its regular <ApiLink to="browser-pool/class/BrowserPool">`BrowserPool`</ApiLink> around a remote variant of its own browser plugin (<ApiLink to="browser-pool/class/RemotePlaywrightPlugin">`RemotePlaywrightPlugin`</ApiLink> or <ApiLink to="browser-pool/class/RemotePuppeteerPlugin">`RemotePuppeteerPlugin`</ApiLink>), so the connection is always for the matching browser — there's no plugin to construct and no way to mismatch the pool with the crawler. The plugin owns everything remote: resolving the endpoint per launch, connecting with the library's `connect()`, and releasing the session when the browser disconnects.

## Basic usage

The simplest form is a static connection URL. Use this when the service exposes a single endpoint and doesn't need per-session setup.

<CodeBlock language="ts" title="src/main.ts">{RemoteBrowserConfigSource}</CodeBlock>

`endpoint` can also be a function returning `{ url, context }`, called once per browser launch. Pair it with a `release` callback (it receives the `context`) to clean up sessions on the service side when the browser closes, crashes, or the pool is destroyed.

`maxOpenBrowsers` caps the number of concurrent remote browsers — set it to the service's concurrent-session limit to avoid 429 errors. The pool enforces it inside `newPage()`, which waits for a free slot rather than overshooting.

### Self-hosted

Some services ship a Docker image you can run locally or on your own infrastructure. For example, [Browserless](https://www.browserless.io/) has an open-source Chromium image:

```bash
docker run -p 3000:3000 -e CONCURRENT=4 ghcr.io/browserless/chromium
```

Point the pool at the local endpoint with `endpoint: 'ws://localhost:3000'`.

## Custom provider

For services with a session-create / session-release lifecycle, extend <ApiLink to="browser-pool/class/RemoteBrowserProvider">`RemoteBrowserProvider`</ApiLink> and pass the instance as the `endpoint`. `connect()` runs once per browser launch and returns the connection URL plus an optional `context` object passed back to `release()`. `maxOpenBrowsers` set on the provider is adopted by the crawler's pool.

<CodeBlock language="ts" title="src/main.ts">{RemoteBrowserProviderSource}</CodeBlock>

## Puppeteer

<ApiLink to="puppeteer-crawler/class/PuppeteerCrawler">`PuppeteerCrawler`</ApiLink> accepts the same `remoteBrowser` option. Puppeteer connects over CDP:

<CodeBlock language="ts" title="src/main.ts">{RemoteBrowserPuppeteerSource}</CodeBlock>

For Playwright you can choose the protocol via the `remoteBrowser.protocol` option: `'cdp'` (default, `connectOverCDP()`) or `'playwright'` (`connect()`, Playwright's own WebSocket protocol).

## Tuning or sharing the pool

`remoteBrowser` builds a pool the crawler owns and tears down. To tune the pool, or to share one across multiple crawlers, pass the same `remoteBrowser` to the pool factory for your crawler (<ApiLink to="playwright-crawler/function/playwrightBrowserPool">`playwrightBrowserPool()`</ApiLink>, <ApiLink to="puppeteer-crawler/function/puppeteerBrowserPool">`puppeteerBrowserPool()`</ApiLink>) and hand the result over as `browserPool` instead — a pool supplied that way is never destroyed by the crawler, so you control its lifecycle. Use `remoteBrowser` *or* `browserPool`, not both.

Declare it with `await using` and the browsers (and their remote sessions) are released once you are done with the pool:

```ts
import { PlaywrightCrawler, playwrightBrowserPool } from 'crawlee';

await using browserPool = playwrightBrowserPool({
    remoteBrowser: { endpoint: 'wss://production-sfo.browserless.io?token=xxx', maxOpenBrowsers: 2 },
    maxOpenPagesPerBrowser: 5,
});

await new PlaywrightCrawler({ browserPool, requestHandler: async () => { /* ... */ } }).run(['https://crawlee.dev']);
await new PlaywrightCrawler({ browserPool, requestHandler: async () => { /* ... */ } }).run(['https://apify.com']);
```

Building the pool from `@crawlee/browser-pool` directly works too, and lets you rotate between several services — the pool round-robins over its plugins:

```ts
import { BrowserPool, RemotePlaywrightPlugin } from '@crawlee/browser-pool';
import playwright from 'playwright';

const browserPool = new BrowserPool({
    browserPlugins: [
        new RemotePlaywrightPlugin(playwright.chromium, { endpoint: 'wss://production-sfo.browserless.io?token=xxx' }),
        new RemotePlaywrightPlugin(playwright.chromium, { endpoint: 'wss://connect.browserbase.com?apiKey=xxx' }),
        new RemotePlaywrightPlugin(playwright.chromium, { endpoint: 'wss://connect.browser-use.com?apiKey=xxx' }),
    ],
});
```

:::note Requires Node.js 24

The `await using` syntax needs Node.js 24 or later. On Node.js 22 call <ApiLink to="browser-pool/class/BrowserPool#destroy">`destroy()`</ApiLink> yourself instead — it is what the disposal hook calls anyway.

:::

## Limitations

- **`headless` and `launchOptions` don't apply.** The remote service controls headless mode and browser flags; configure them on the service side.
- **`useIncognitoPages` is forced to `true`** for Playwright remote connections — `connect()` / `connectOverCDP()` don't accept persistent contexts. For state shared across requests, use the `SessionPool`.
- **`userDataDir` has no effect** — there's no local profile when the browser runs remotely. Use the service's persistence API (e.g. Browserbase Contexts, Steel Profiles).

## Further reading

- <ApiLink to="browser-pool/class/RemotePlaywrightPlugin">`RemotePlaywrightPlugin`</ApiLink> / <ApiLink to="browser-pool/class/RemotePuppeteerPlugin">`RemotePuppeteerPlugin`</ApiLink> API reference
- <ApiLink to="browser-pool/class/RemoteBrowserProvider">`RemoteBrowserProvider` API reference</ApiLink>