Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Build docs
Ondra Urban6d48d714306f
diff --git a/docs/api/Apify.md b/docs/api/Apify.mdindex b9d1b3bbd..796fc366a 100644--- a/docs/api/Apify.md+++ b/docs/api/Apify.md@@ -316,7 +316,7 @@ The following table shows all currently emitted events:in which case the <code>isMigrating</code> flag is set to <code>true</code>. Otherwise the flag is <code>false</code>.<br><br>Note that the <code>persistState</code> event is provided merely for user convenience,- you can achieve the same effect using `setInterval()` and listening for the `migrating` event.+ you can achieve the same effect using <code>setInterval()</code> and listening for the <code>migrating</code> event.</td></tr></tbody>@@ -534,7 +534,7 @@ The `launchPuppeteer()` function alters the following Puppeteer options:<li>Sets <code>defaultViewport</code> Puppeteer option (if not already set)to a more reasonable default for screenshots and debugging.- You can set `options.defaultViewport` to `null` if you prefer to let Puppeteer+ You can set <code>options.defaultViewport</code> to <code>null</code> if you prefer to let Puppeteerchoose the default viewport size.</li></ul>diff --git a/docs/api/Dataset.md b/docs/api/Dataset.mdindex 00fea4df8..7461b3c8c 100644--- a/docs/api/Dataset.md+++ b/docs/api/Dataset.md@@ -204,14 +204,14 @@ API endpoint.**Example:**```{- "id": "WkzbQMuFYuamGv3YF",- "name": "my-dataset",- "userId": "wRsJZtadYvn4mBZmm",- "createdAt": new Date("2015-12-12T07:34:14.202Z"),- "modifiedAt": new Date("2015-12-13T08:36:13.202Z"),- "accessedAt": new Date("2015-12-14T08:36:13.202Z"),- "itemCount": 14,- "cleanItemCount": 10+ id: "WkzbQMuFYuamGv3YF",+ name: "my-dataset",+ userId: "wRsJZtadYvn4mBZmm",+ createdAt: new Date("2015-12-12T07:34:14.202Z"),+ modifiedAt: new Date("2015-12-13T08:36:13.202Z"),+ accessedAt: new Date("2015-12-14T08:36:13.202Z"),+ itemCount: 14,+ cleanItemCount: 10}```diff --git a/docs/api/RequestQueue.md b/docs/api/RequestQueue.mdindex 09ca151d4..d83e1b8d1 100644--- a/docs/api/RequestQueue.md+++ b/docs/api/RequestQueue.md@@ -227,12 +227,12 @@ API endpoint.**Example:**```{- "id": "WkzbQMuFYuamGv3YF",- "name": "my-queue",- "userId": "wRsJZtadYvn4mBZmm",- "createdAt": new Date("2015-12-12T07:34:14.202Z"),- "modifiedAt": new Date("2015-12-13T08:36:13.202Z"),- "accessedAt": new Date("2015-12-14T08:36:13.202Z"),+ id: "WkzbQMuFYuamGv3YF",+ name: "my-queue",+ userId: "wRsJZtadYvn4mBZmm",+ createdAt: new Date("2015-12-12T07:34:14.202Z"),+ modifiedAt: new Date("2015-12-13T08:36:13.202Z"),+ accessedAt: new Date("2015-12-14T08:36:13.202Z"),totalRequestCount: 0,handledRequestCount: 0,pendingRequestCount: 0,