Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

docs: mention `pageOptions` limitation (#1674)

Szymon Marczak0bc147e112e4
diff --git a/packages/browser-crawler/src/internals/browser-crawler.ts b/packages/browser-crawler/src/internals/browser-crawler.ts
index d00f0aa75..ebc25ec2a 100644
--- a/packages/browser-crawler/src/internals/browser-crawler.ts
+++ b/packages/browser-crawler/src/internals/browser-crawler.ts
@@ -205,6 +205,9 @@ export interface BrowserCrawlerOptions<
* },
* ]
* ```
+ *
+ * Modyfing `pageOptions` is supported only in Playwright incognito.
+ * See {@apilink PrePageCreateHook}
*/
preNavigationHooks?: BrowserHook<Context>[];
diff --git a/packages/browser-pool/src/browser-pool.ts b/packages/browser-pool/src/browser-pool.ts
index b2e714a20..61946b56e 100644
--- a/packages/browser-pool/src/browser-pool.ts
+++ b/packages/browser-pool/src/browser-pool.ts
@@ -108,7 +108,7 @@ export type PostLaunchHook<BC extends BrowserController> = (pageId: string, brow
* The hooks are called with three arguments:
* `pageId`: `string`, `browserController`: {@apilink BrowserController} and
* `pageOptions`: `object|undefined` - This only works if the underlying `BrowserController` supports new page options.
- * So far, new page options are only supported by `PlaywrightController`.
+ * So far, new page options are only supported by `PlaywrightController` in incognito contexts.
* If the page options are not supported by `BrowserController` the `pageOptions` argument is `undefined`.
*/
export type PrePageCreateHook<
@@ -176,7 +176,7 @@ export interface BrowserPoolHooks<
* The hooks are called with three arguments:
* `pageId`: `string`, `browserController`: {@apilink BrowserController} and
* `pageOptions`: `object|undefined` - This only works if the underlying `BrowserController` supports new page options.
- * So far, new page options are only supported by `PlaywrightController`.
+ * So far, new page options are only supported by `PlaywrightController` in incognito contexts.
* If the page options are not supported by `BrowserController` the `pageOptions` argument is `undefined`.
*/
prePageCreateHooks?: PrePageCreateHook<BC>[];
diff --git a/packages/http-crawler/src/internals/http-crawler.ts b/packages/http-crawler/src/internals/http-crawler.ts
index eb0af16c0..81d30b6a6 100644
--- a/packages/http-crawler/src/internals/http-crawler.ts
+++ b/packages/http-crawler/src/internals/http-crawler.ts
@@ -91,6 +91,9 @@ export interface HttpCrawlerOptions<Context extends InternalHttpCrawlingContext
* },
* ]
* ```
+ *
+ * Modyfing `pageOptions` is supported only in Playwright incognito.
+ * See {@apilink PrePageCreateHook}
*/
preNavigationHooks?: InternalHttpHook<Context>[];
diff --git a/packages/playwright-crawler/src/internals/playwright-crawler.ts b/packages/playwright-crawler/src/internals/playwright-crawler.ts
index aaa676993..50a6e22f4 100644
--- a/packages/playwright-crawler/src/internals/playwright-crawler.ts
+++ b/packages/playwright-crawler/src/internals/playwright-crawler.ts
@@ -92,6 +92,9 @@ export interface PlaywrightCrawlerOptions extends BrowserCrawlerOptions<
* },
* ]
* ```
+ *
+ * Modyfing `pageOptions` is supported only in Playwright incognito.
+ * See {@apilink PrePageCreateHook}
*/
preNavigationHooks?: PlaywrightHook[];
diff --git a/packages/puppeteer-crawler/src/internals/puppeteer-crawler.ts b/packages/puppeteer-crawler/src/internals/puppeteer-crawler.ts
index c861afdad..b15c8716b 100644
--- a/packages/puppeteer-crawler/src/internals/puppeteer-crawler.ts
+++ b/packages/puppeteer-crawler/src/internals/puppeteer-crawler.ts
@@ -42,6 +42,9 @@ export interface PuppeteerCrawlerOptions extends BrowserCrawlerOptions<
* },
* ]
* ```
+ *
+ * Modyfing `pageOptions` is supported only in Playwright incognito.
+ * See {@apilink PrePageCreateHook}
*/
preNavigationHooks?: PuppeteerHook[];