Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

docs: document removal of gotoOptions argument from browser navigation hooks (#3972)

Martin Adámekf2ef8993eacf
diff --git a/docs/upgrading/upgrading_v4.md b/docs/upgrading/upgrading_v4.md
index b9a39cda4..05c1b02a5 100644
--- a/docs/upgrading/upgrading_v4.md
+++ b/docs/upgrading/upgrading_v4.md
@@ -404,6 +404,26 @@ In v4, when `saveResponseCookies` is enabled (the default), browser cookies are
The `preNavigationHooks` option in `HttpCrawler` subclasses no longer accepts the `gotOptions` object as a second parameter. Modify the `crawlingContext` fields (e.g. `.request`) directly instead.
+### Browser navigation hooks no longer receive `gotoOptions` as a second argument
+
+The `preNavigationHooks` and `postNavigationHooks` of the browser crawlers (`PlaywrightCrawler`, `PuppeteerCrawler`) received the options object forwarded to `page.goto()` as a second parameter in v3. The hooks now receive only the crawling context, and the `page.goto()` options are available as its `gotoOptions` member, which can be mutated in place:
+
+```ts
+// v3
+preNavigationHooks: [
+ async (crawlingContext, gotoOptions) => {
+ gotoOptions.timeout = 60_000;
+ },
+],
+
+// v4
+preNavigationHooks: [
+ async ({ gotoOptions }) => {
+ gotoOptions.timeout = 60_000;
+ },
+],
+```
+
### Removed crawling context properties
#### Crawling context no longer includes Error for failed requests