Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Improved docs
Jan Curn58b0f20c4341
diff --git a/src/puppeteer.js b/src/puppeteer.jsindex a185bdb5d..3d613bfe1 100644--- a/src/puppeteer.js+++ b/src/puppeteer.js@@ -134,16 +134,18 @@ const getPuppeteerOrThrow = () => {** To use this function, you need to have the <a href="https://www.npmjs.com/package/puppeteer" target="_blank">puppeteer</a>* NPM package installed in your project.- * For example, you can use the `apify/actor-node-chrome` base Docker image for your act - see- * <a href="https://www.apify.com/docs/actor#base-images" target="_blank">documentation</a>- * for more details.+ * When running on the Apify cloud platform, you can achieve that simply+ * by using the `apify/actor-node-chrome` base Docker image for your act - see+ * <a href="https://www.apify.com/docs/actor#base-images" target="_blank">Apify Actor documentation</a>+ * for details.*- * For an example of usage, see the <a href="https://www.apify.com/apify/example-puppeteer">apify/example-puppeteer</a> act.- *- * @param {LaunchPuppeteerOptions} [opts] Optional settings passed to `puppeteer.launch()`. Additionally the object can- * contain the following fields:- * @returns {Promise} Promise object that resolves to Puppeteer's `Browser` instance.+ * For an example of usage, see the <a href="https://www.apify.com/apify/example-puppeteer">apify/example-puppeteer</a> actor.*+ * @param {LaunchPuppeteerOptions} [opts]+ * Optional settings passed to `puppeteer.launch()`. Additionally the object can+ * contain the following fields:+ * @returns {Promise}+ * Promise object that resolves to Puppeteer's `Browser` instance.* @memberof module:Apify* @name launchPuppeteer* @instancediff --git a/src/puppeteer_pool.js b/src/puppeteer_pool.jsindex 5920f48af..10c182b83 100644--- a/src/puppeteer_pool.js+++ b/src/puppeteer_pool.js@@ -46,7 +46,14 @@ class PuppeteerInstance {* Example usage:** ```javascript- * const puppeteerPool = new PuppeteerPool({ groups: 'some-proxy-group' });+ * const puppeteerPool = new PuppeteerPool({+ * launchPuppeteerFunction: () => {+ * // Use a new proxy with a new IP address for each new Chrome instance+ * return Apify.launchPuppeteer({+ * apifyProxySession: Math.random(),+ * });+ * },+ * });** const page1 = await puppeteerPool.newPage();* const page2 = await puppeteerPool.newPage();@@ -54,7 +61,7 @@ class PuppeteerInstance {** // ... do something with pages ...*- * // Close all the browsers.+ * // Close all browsers.* await puppeteerPool.destroy();* ```*