Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

fix: improve browser launcher types (#1318)

Jindřich Bärf167db95ab75
diff --git a/src/browser_launchers/browser_launcher.js b/src/browser_launchers/browser_launcher.js
index 25fd7e5e5..b191b75cb 100644
--- a/src/browser_launchers/browser_launcher.js
+++ b/src/browser_launchers/browser_launcher.js
@@ -19,6 +19,13 @@ import { BrowserPlugin } from './browser_plugin'; // eslint-disable-line no-unus
* is taken from the `APIFY_CHROME_EXECUTABLE_PATH` environment variable if provided,
* or defaults to the typical Google Chrome executable location specific for the operating system.
* By default, this option is `false`.
+ * @property {boolean} [useIncognitoPages=false]
+ * With this option selected, all pages will be opened in a new incognito browser context.
+ * This means they will not share cookies nor cache and their resources will not be throttled by one another.
+ * @property {string} [userDataDir]
+ * Sets the [User Data Directory](https://chromium.googlesource.com/chromium/src/+/master/docs/user_data_dir.md) path.
+ * The user data directory contains profile data such as history, bookmarks, and cookies, as well as other per-installation local state.
+ * If not specified, a temporary directory is used instead.
* @property {*} [launcher]
* By default this function uses require("playwright").chromium`.
* If you want to use a different browser you can pass it by this property as `require("playwright").firefox
diff --git a/src/browser_launchers/playwright_launcher.js b/src/browser_launchers/playwright_launcher.js
index a9232df97..4e8e68c47 100644
--- a/src/browser_launchers/playwright_launcher.js
+++ b/src/browser_launchers/playwright_launcher.js
@@ -26,7 +26,7 @@ import BrowserLauncher from './browser_launcher';
*
* @typedef PlaywrightLaunchContext
* @property {LaunchOptions} [launchOptions]
- * `browserType.launch` [options](https://playwright.dev/docs/api/class-browsertype?_highlight=launch#browsertypelaunchoptions)
+ * `browserType.launch` [options](https://playwright.dev/docs/api/class-browsertype#browser-type-launch)
* @property {string} [proxyUrl]
* URL to a HTTP proxy server. It must define the port number,
* and it may also contain proxy username and password.
@@ -38,7 +38,13 @@ import BrowserLauncher from './browser_launcher';
* rather than the bundled Chromium. The path to Chrome executable
* is taken from the `APIFY_CHROME_EXECUTABLE_PATH` environment variable if provided,
* or defaults to the typical Google Chrome executable location specific for the operating system.
- * By default, this option is `false`.
+ * @property {boolean} [useIncognitoPages=false]
+ * With this option selected, all pages will be opened in a new incognito browser context.
+ * This means they will not share cookies nor cache and their resources will not be throttled by one another.
+ * @property {string} [userDataDir]
+ * Sets the [User Data Directory](https://chromium.googlesource.com/chromium/src/+/master/docs/user_data_dir.md) path.
+ * The user data directory contains profile data such as history, bookmarks, and cookies, as well as other per-installation local state.
+ * If not specified, a temporary directory is used instead.
* @property {*} [launcher]
* By default this function uses `require("playwright").chromium`.
* If you want to use a different browser you can pass it by this property as e.g. `require("playwright").firefox`
diff --git a/src/browser_launchers/puppeteer_launcher.js b/src/browser_launchers/puppeteer_launcher.js
index 6b3cf0757..f0c4201d6 100644
--- a/src/browser_launchers/puppeteer_launcher.js
+++ b/src/browser_launchers/puppeteer_launcher.js
@@ -1,5 +1,5 @@
import ow from 'ow';
-import { LaunchOptions } from 'puppeteer'; // eslint-disable-line no-unused-vars,import/named
+import { PuppeteerNode } from 'puppeteer'; // eslint-disable-line no-unused-vars,import/named
import { PuppeteerPlugin } from 'browser-pool';
import BrowserLauncher from './browser_launcher';
import { isAtHome } from '../utils';
@@ -33,8 +33,8 @@ const LAUNCH_PUPPETEER_DEFAULT_VIEWPORT = {
* ```
*
* @typedef PuppeteerLaunchContext
- * @property {LaunchOptions} [launchOptions]
- * `puppeteer.launch` [options](https://pptr.dev/#?product=Puppeteer&version=v5.5.0&show=api-puppeteerlaunchoptions)
+ * @property {Parameters<PuppeteerNode['launch']>[0]} [launchOptions]
+ * `puppeteer.launch` [options](https://pptr.dev/#?product=Puppeteer&version=v13.5.1&show=api-puppeteerlaunchoptions)
* @property {string} [proxyUrl]
* URL to a HTTP proxy server. It must define the port number,
* and it may also contain proxy username and password.
@@ -57,6 +57,9 @@ const LAUNCH_PUPPETEER_DEFAULT_VIEWPORT = {
*
* Take caution, because it can cause all kinds of unexpected errors and weird behavior.
* Apify SDK is not tested with any other library besides `puppeteer` itself.
+ * @property {boolean} [useIncognitoPages=false]
+ * With this option selected, all pages will be opened in a new incognito browser context.
+ * This means they will not share cookies nor cache and their resources will not be throttled by one another.
* @property {boolean} [stealth]
* This setting hides most of the known properties that identify headless Chrome and makes it nearly undetectable.
* It is recommended to use it together with the `useChrome` set to `true`.