Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Added validation for persisting cookies per session used with no session pool (#810)

Jan Bárta2e017db9edd3
diff --git a/src/crawlers/puppeteer_crawler.js b/src/crawlers/puppeteer_crawler.js
index afde40776..1259634cb 100644
--- a/src/crawlers/puppeteer_crawler.js
+++ b/src/crawlers/puppeteer_crawler.js
@@ -279,6 +279,10 @@ class PuppeteerCrawler {
this.log = defaultLog.child({ prefix: 'PuppeteerCrawler' });
+ if (persistCookiesPerSession && !useSessionPool) {
+ throw new Error('Cannot use "options.persistCookiesPerSession" without "options.useSessionPool"');
+ }
+
if (options.gotoTimeoutSecs && options.gotoFunction) {
this.log.warning('You are using gotoTimeoutSecs with a custom gotoFunction. '
+ 'The timeout value will not be used. With a custom gotoFunction, you need to set the timeout in the function itself.');