Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Updated comments

Jan Curn00850e1bfa05
diff --git a/src/puppeteer.js b/src/puppeteer.js
index 6d8a09f4b..950e1b730 100644
--- a/src/puppeteer.js
+++ b/src/puppeteer.js
@@ -34,8 +34,9 @@ import { ENV_VARS, DEFAULT_USER_AGENT } from './constants';
* for more details.
* </p>
* @param {Object} [opts] Optional settings passed to `puppeteer.launch()`. Additionally the object can contain the following fields:
- * @param {String} [opts.proxyUrl] - URL to a proxy server. Currently only `http://` scheme is supported.
- * Port number must be specified. Proxy authentication is also supported. For example, `http://bob:[email protected]:1234`.
+ * @param {String} [opts.proxyUrl] - URL to a HTTP proxy server.
+ * Port number must be specified. Proxy username and password might also be provided.
+ * For example, `http://bob:[email protected]:1234`.
* @param {String} [opts.userAgent] - Default User-Agent for the browser.
* If not provided, the function sets it to a reasonable default.
* @returns {Promise} Promise object that resolves to Puppeteer's `Browser` instance.