Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
chore: improve comment
Szymon Marczakdade38ce9eca
diff --git a/src/crawlers/browser_crawler.js b/src/crawlers/browser_crawler.jsindex 93413c824..9fb876fc8 100644--- a/src/crawlers/browser_crawler.js+++ b/src/crawlers/browser_crawler.js@@ -391,7 +391,9 @@ export default class BrowserCrawler extends BasicCrawler {this._enhanceCrawlingContextWithPageInfo(crawlingContext, page, useIncognitoPages);// DO NOT MOVE THIS LINE ABOVE!- // `enhanceCrawlingContextWithPageInfo` can (and will!) override those properties.+ // `enhanceCrawlingContextWithPageInfo` gives us a valid session.+ // For example, `sessionPoolOptions.sessionOptions.maxUsageCount` can be `1`.+ // So we must not save the session prior to making sure it was used only once, otherwise we would use it twice.const { request, session } = crawlingContext;if (this.useSessionPool) {