Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Update docs

Ondra Urbanb2d152a4e6a4
diff --git a/docs/api/BasicCrawler.md b/docs/api/BasicCrawler.md
index dcd9850fa..59507545d 100644
--- a/docs/api/BasicCrawler.md
+++ b/docs/api/BasicCrawler.md
@@ -122,7 +122,7 @@ await crawler.run();
}</code></pre><p> where the <a href="request"><code>Request</code></a> instance corresponds to the failed request, and the <code>Error</code> instance
represents the last error thrown during processing of the request.</p>
<p> See
- <a href="https://github.com/apifytech/apify-js/blob/master/src/basic_crawler.js#L11" target="_blank">source code</a>
+ <a href="https://github.com/apifytech/apify-js/blob/master/src/crawlers/basic_crawler.js#L11" target="_blank">source code</a>
for the default implementation of this function.</p>
</td></tr><tr>
<td><code>[options.maxRequestRetries]</code></td><td><code>Number</code></td><td><code>3</code></td>
diff --git a/docs/api/CheerioCrawler.md b/docs/api/CheerioCrawler.md
index 93d4b1179..0d3b66684 100644
--- a/docs/api/CheerioCrawler.md
+++ b/docs/api/CheerioCrawler.md
@@ -217,7 +217,7 @@ await crawler.run();
error: Error,
}</code></pre><p> where the <a href="request"><code>Request</code></a> instance corresponds to the failed request, and the <code>Error</code> instance
represents the last error thrown during processing of the request.</p>
-<p> See <a href="https://github.com/apifytech/apify-js/blob/master/src/cheerio_crawler.js#L13">source code</a>
+<p> See <a href="https://github.com/apifytech/apify-js/blob/master/src/crawlers/cheerio_crawler.js#L13">source code</a>
for the default implementation of this function.</p>
</td></tr><tr>
<td><code>[options.maxRequestRetries]</code></td><td><code>Number</code></td><td><code>3</code></td>
diff --git a/docs/api/PuppeteerCrawler.md b/docs/api/PuppeteerCrawler.md
index f40bf12ab..6683e5a95 100644
--- a/docs/api/PuppeteerCrawler.md
+++ b/docs/api/PuppeteerCrawler.md
@@ -151,7 +151,7 @@ await crawler.run();
}</code></pre><p> Where the <a href="request"><code>Request</code></a> instance corresponds to the failed request, and the <code>Error</code> instance
represents the last error thrown during processing of the request.</p>
<p> See
- <a href="https://github.com/apifytech/apify-js/blob/master/src/puppeteer_crawler.js#L11" target="_blank">source code</a>
+ <a href="https://github.com/apifytech/apify-js/blob/master/src/crawlers/puppeteer_crawler.js#L11" target="_blank">source code</a>
for the default implementation of this function.</p>
</td></tr><tr>
<td><code>[options.maxRequestRetries]</code></td><td><code>Number</code></td><td><code>3</code></td>
diff --git a/src/crawlers/basic_crawler.js b/src/crawlers/basic_crawler.js
index 76887e5e4..435a8fba3 100644
--- a/src/crawlers/basic_crawler.js
+++ b/src/crawlers/basic_crawler.js
@@ -136,7 +136,7 @@ const DEFAULT_OPTIONS = {
* represents the last error thrown during processing of the request.
*
* See
- * <a href="https://github.com/apifytech/apify-js/blob/master/src/basic_crawler.js#L11" target="_blank">source code</a>
+ * <a href="https://github.com/apifytech/apify-js/blob/master/src/crawlers/basic_crawler.js#L11" target="_blank">source code</a>
* for the default implementation of this function.
* @param {Number} [options.maxRequestRetries=3]
* Indicates how many times the request is retried if [`handleRequestFunction()`](#new_BasicCrawler_new) fails.
diff --git a/src/crawlers/cheerio_crawler.js b/src/crawlers/cheerio_crawler.js
index 40d81b06e..4aafe3609 100644
--- a/src/crawlers/cheerio_crawler.js
+++ b/src/crawlers/cheerio_crawler.js
@@ -215,7 +215,7 @@ const DEFAULT_OPTIONS = {
* where the {@link Request} instance corresponds to the failed request, and the `Error` instance
* represents the last error thrown during processing of the request.
*
- * See <a href="https://github.com/apifytech/apify-js/blob/master/src/cheerio_crawler.js#L13">source code</a>
+ * See <a href="https://github.com/apifytech/apify-js/blob/master/src/crawlers/cheerio_crawler.js#L13">source code</a>
* for the default implementation of this function.
* @param {Number} [options.maxRequestRetries=3]
* Indicates how many times the request is retried if either `requestFunction` or `handlePageFunction` fails.
diff --git a/src/crawlers/puppeteer_crawler.js b/src/crawlers/puppeteer_crawler.js
index 1cf19f8bf..0561d77f9 100644
--- a/src/crawlers/puppeteer_crawler.js
+++ b/src/crawlers/puppeteer_crawler.js
@@ -146,7 +146,7 @@ const DEFAULT_OPTIONS = {
* represents the last error thrown during processing of the request.
*
* See
- * <a href="https://github.com/apifytech/apify-js/blob/master/src/puppeteer_crawler.js#L11" target="_blank">source code</a>
+ * <a href="https://github.com/apifytech/apify-js/blob/master/src/crawlers/puppeteer_crawler.js#L11" target="_blank">source code</a>
* for the default implementation of this function.
* @param {Number} [options.maxRequestRetries=3]
* Indicates how many times the request is retried if either `handlePageFunction()` or `gotoFunction()` fails.