Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Updated Puppeteer crawler example
Jan Curn5806ead0c0dd
diff --git a/examples/puppeteer_crawler.js b/examples/puppeteer_crawler.jsindex a8c41b13a..ccd13afb9 100644--- a/examples/puppeteer_crawler.js+++ b/examples/puppeteer_crawler.js@@ -30,9 +30,9 @@ Apify.main(async () => {requestList,requestQueue,- // Run Puppeteer in headless mode. If you set headless to false, you'll see the scraping- // browsers showing up on your screen. This is great for debugging.- launchPuppeteerOptions: { headless: true },+ // Here you can set options that are passed to the Apify.launchPuppeteer() function.+ // For example, you can set "slowMo" to slow down Puppeteer operations to simplify debugging+ launchPuppeteerOptions: { slowMo: 2000 },// This function will be called for each URL to crawl.// Here you can write the Puppeteer scripts you are familiar with,