Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Fix puppeteerPoolOptions not being used

Ondra Urbana02c9be451b0
diff --git a/src/puppeteer_crawler.js b/src/puppeteer_crawler.js
index 4271ebd52..01990d883 100644
--- a/src/puppeteer_crawler.js
+++ b/src/puppeteer_crawler.js
@@ -226,11 +226,11 @@ class PuppeteerCrawler {
if (value) log.deprecated(`PuppeteerCrawler: options.${key} is deprecated. Use options.puppeteerPoolOptions instead.`);
});
// puppeteerPoolOptions can be null or undefined or Object, so we merge it this way, because null is not replaced by defaults above.
- this.puppeteerPoolOptions = Object.assign(
+ this.puppeteerPoolOptions = _.defaults(
{},
- puppeteerPoolOptions,
{ launchPuppeteerFunction, launchPuppeteerOptions },
deprecatedPuppeteerPoolOptions,
+ puppeteerPoolOptions,
);
this.puppeteerPool = null; // Constructed when .run()