Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Updated PuppeteerCrawler docs
Jan Curn38cfd4612cd7
diff --git a/src/puppeteer_crawler.js b/src/puppeteer_crawler.jsindex df19390c8..ccecf8e33 100644--- a/src/puppeteer_crawler.js+++ b/src/puppeteer_crawler.js@@ -80,15 +80,18 @@ const PAGE_CLOSE_TIMEOUT_MILLIS = 30000;* ```* {* request: Request,+ * response: Response,* page: Page,* puppeteerPool: PuppeteerPool* }* ```** `request` is an instance of the {@link Request} object with details about the URL to open, HTTP method etc.- * `page` is an instance of the `Puppeteer`+ * `response` is an instance of the `Puppeteer`* <a href="https://pptr.dev/#?product=Puppeteer&show=api-class-page" target="_blank"><code>Page</code></a>- * class with `page.goto(request.url)` already called.+ * `page` is an instance of the `Puppeteer`+ * <a href="https://pptr.dev/#?product=Puppeteer&show=api-class-response" target="_blank"><code>Response</code></a>,+ * which is the main resource response as returned by `page.goto(request.url)`.* `puppeteerPool` is an instance of the {@link PuppeteerPool} used by this `PuppeteerCrawler`.* @param {RequestList} options.requestList* Static list of URLs to be processed.