Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Updated docs
Jan Curnb3e963441527
diff --git a/docs/api/Apify.md b/docs/api/Apify.mdindex 9ea69926d..5807e62d3 100644--- a/docs/api/Apify.md+++ b/docs/api/Apify.md@@ -359,7 +359,7 @@ The following table shows all currently emitted events:</tr><tr><td colspan="2">- Emitted in regular intervals to notify all components of Apify SDK that it is time to persist+ Emitted in regular intervals (by default 60 seconds) to notify all components of Apify SDK that it is time to persisttheir state, in order to avoid repeating all work when the actor restarts.This event is automatically emitted together with the <code>migrating</code> event,in which case the <code>isMigrating</code> flag is set to <code>true</code>. Otherwise the flag is <code>false</code>.diff --git a/src/events.js b/src/events.jsindex 063afa193..dd483dcd5 100644--- a/src/events.js+++ b/src/events.js@@ -4,6 +4,7 @@ import log from 'apify-shared/log';import { ENV_VARS, ACTOR_EVENT_NAMES } from 'apify-shared/consts';import { ACTOR_EVENT_NAMES_EX } from './constants';+// NOTE: This value is mentioned below in docs, if you update it here, update it there too.const PERSIST_STATE_INTERVAL_MILLIS = 60 * 1000;/**@@ -75,7 +76,7 @@ let persistStateInterval = null;* </tr>* <tr>* <td colspan="2">- * Emitted in regular intervals to notify all components of Apify SDK that it is time to persist+ * Emitted in regular intervals (by default 60 seconds) to notify all components of Apify SDK that it is time to persist* their state, in order to avoid repeating all work when the actor restarts.* This event is automatically emitted together with the <code>migrating</code> event,* in which case the <code>isMigrating</code> flag is set to <code>true</code>. Otherwise the flag is <code>false</code>.