Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
docs: document removal of gotoOptions argument from browser navigation hooks (#3972)
Martin Adámekf2ef8993eacf
diff --git a/docs/upgrading/upgrading_v4.md b/docs/upgrading/upgrading_v4.mdindex b9a39cda4..05c1b02a5 100644--- a/docs/upgrading/upgrading_v4.md+++ b/docs/upgrading/upgrading_v4.md@@ -404,6 +404,26 @@ In v4, when `saveResponseCookies` is enabled (the default), browser cookies areThe `preNavigationHooks` option in `HttpCrawler` subclasses no longer accepts the `gotOptions` object as a second parameter. Modify the `crawlingContext` fields (e.g. `.request`) directly instead.+### Browser navigation hooks no longer receive `gotoOptions` as a second argument++The `preNavigationHooks` and `postNavigationHooks` of the browser crawlers (`PlaywrightCrawler`, `PuppeteerCrawler`) received the options object forwarded to `page.goto()` as a second parameter in v3. The hooks now receive only the crawling context, and the `page.goto()` options are available as its `gotoOptions` member, which can be mutated in place:++```ts+// v3+preNavigationHooks: [+ async (crawlingContext, gotoOptions) => {+ gotoOptions.timeout = 60_000;+ },+],++// v4+preNavigationHooks: [+ async ({ gotoOptions }) => {+ gotoOptions.timeout = 60_000;+ },+],+```+### Removed crawling context properties#### Crawling context no longer includes Error for failed requests