Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

builded docs.

Petr Patek35bbe328f65a
diff --git a/docs/api/PuppeteerCrawler.md b/docs/api/PuppeteerCrawler.md
index 184b9f957..f40bf12ab 100644
--- a/docs/api/PuppeteerCrawler.md
+++ b/docs/api/PuppeteerCrawler.md
@@ -136,9 +136,9 @@ await crawler.run();
<p> This is useful if you need to extend the page load timeout or select different criteria
to determine that the navigation succeeded.</p>
<p> Note that a single page object is only used to process a single request and it is closed afterwards.</p>
-<p> By default, the function invokes Puppeteer&#39;s <code>Page.goto()</code> with a timeout of 60 seconds.
+<p> By default, the function invokes <a href="puppeteer#puppeteer.gotoExtended"><code>Apify.utils.puppeteer.gotoExtended()</code></a> with a timeout of 60 seconds.
For details, see source code on
- <a href="https://github.com/apifytech/apify-js/blob/master/src/puppeteer_crawler.js#L9" target="_blank">GitHub</a>.</p>
+ <a href="https://github.com/apifytech/apify-js/blob/master/src/crawlers/puppeteer_crawler.js#L9" target="_blank">GitHub</a>.</p>
</td></tr><tr>
<td><code>[options.handleFailedRequestFunction]</code></td><td><code>function</code></td><td></td>
</tr>
diff --git a/docs/api/utils.md b/docs/api/utils.md
index 12ad0f83b..0847a43fa 100644
--- a/docs/api/utils.md
+++ b/docs/api/utils.md
@@ -130,23 +130,17 @@ objects.
## `utils.requestAsBrowser` ⇒ `http.IncomingMessage`
-Sends a HTTP request that looks like a request sent by a web browser,
-fully emulating browser's HTTP headers.
-
-This function is useful for web scraping of websites that send the full HTML in the first response.
-Thanks to this function, the target web server has no simple way to find out the request
-hasn't been sent by a full web browser. Using a headless browser for such requests
-is an order of magnitude more resource-intensive than this function.
-By default tt aborts all requests that returns 406 status codes or non-HTML content-types.
-You can override this behavior by passing custom `abortFunction`.
-
-Currently, the function sends requests the same way as Firefox web browser does.
-In the future, it might add support for other browsers too.
-
-Internally, the function uses httpRequest function from the [@apify/httpRequest](https://github.com/apifytech/http-request)
-NPM package to perform the request.
-All `options` not recognized by this function are passed to it,
-so see it for more details.
+Sends a HTTP request that looks like a request sent by a web browser, fully emulating browser's HTTP headers.
+
+This function is useful for web scraping of websites that send the full HTML in the first response. Thanks to this function, the target web server has
+no simple way to find out the request hasn't been sent by a full web browser. Using a headless browser for such requests is an order of magnitude more
+resource-intensive than this function. By default tt aborts all requests that returns 406 status codes or non-HTML content-types. You can override
+this behavior by passing custom `abortFunction`.
+
+Currently, the function sends requests the same way as Firefox web browser does. In the future, it might add support for other browsers too.
+
+Internally, the function uses httpRequest function from the [@apify/httpRequest](https://github.com/apifytech/http-request) NPM package to perform the
+request. All `options` not recognized by this function are passed to it, so see it for more details.
<table>
<thead>