Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Regenerated docs

Jan Curn8807c93e022a
diff --git a/docs/api/BasicCrawler.md b/docs/api/BasicCrawler.md
index 9b642a394..022050446 100644
--- a/docs/api/BasicCrawler.md
+++ b/docs/api/BasicCrawler.md
@@ -84,7 +84,8 @@ await crawler.run();
<pre><code>{
request: Request,
autoscaledPool: AutoscaledPool
-}</code></pre><p> where the <a href="request"><code>Request</code></a> instance represents the URL to crawl.</p>
+}
+</code></pre><p> where the <a href="request"><code>Request</code></a> instance represents the URL to crawl.</p>
<p> The function must return a promise, which is then awaited by the crawler.</p>
<p> If the function throws an exception, the crawler will try to re-crawl the
request later, up to <code>option.maxRequestRetries</code> times.
@@ -120,7 +121,8 @@ await crawler.run();
<pre><code>{
request: Request,
error: Error,
-}</code></pre><p> where the <a href="request"><code>Request</code></a> instance corresponds to the failed request, and the <code>Error</code> instance
+}
+</code></pre><p> where the <a href="request"><code>Request</code></a> instance corresponds to the failed request, and the <code>Error</code> instance
represents the last error thrown during processing of the request.</p>
<p> See
<a href="https://github.com/apifytech/apify-js/blob/master/src/crawlers/basic_crawler.js#L11" target="_blank">source code</a>