Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Fix docs

Ondra Urbanfebb4dca2384
diff --git a/src/puppeteer_utils.js b/src/puppeteer_utils.js
index 2d652df86..4de3a0a89 100644
--- a/src/puppeteer_utils.js
+++ b/src/puppeteer_utils.js
@@ -382,14 +382,13 @@ const compileScript = (scriptString, context = Object.create(null)) => {
};
/**
- * *NOTE:* In recent versions of Puppeteer using this function entirely disables browser cache which resolves in sub-optimal
- * performance. Until this resolves, we suggest using this function only when needed and not to replace all `page.goto()`
- * calls in your actors with it.
- *
* Extended version of Puppeteer's `page.goto()` allowing to perform requests with HTTP method other than GET,
* with custom headers and POST payload. URL, method, headers and payload are taken from
* request parameter that must be an instance of Apify.Request class.
*
+ * *NOTE:* In recent versions of Puppeteer using requests other than GET, overriding headers and adding payloads disables
+ * browser cache which degrades performance.
+ *
* @param {Page} page
* Puppeteer <a href="https://pptr.dev/#?product=Puppeteer&show=api-class-page" target="_blank"><code>Page</code></a> object.
* @param {Request} request