Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Upgraded to Puppeteer version 1.12.2
Jan Curn873edce335cd
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex abf0d1465..a33319ee0 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,6 +4,7 @@ xxxin order to prevent stalling crawlers.- **BREAKING CHANGE:** `PseudoUrl` now performs case-insensitive matching, even for the query string part of the URLs.If you need case sensitive matching, use an appropriate `RegExp` in place of a Pseudo URL string+- **Upgraded to Puppeteer version 1.12.2**- Added `loadedUrl` property to `Request` that contains the final URL of the loaded page after all redirects.- Added memory overload warning log message.- Added `keyValueStore.getPublicUrl` function.diff --git a/package.json b/package.jsonindex c9b6e083c..42dc8f468 100644--- a/package.json+++ b/package.json@@ -93,7 +93,7 @@"why-is-node-running": "^2.0.3"},"optionalDependencies": {- "puppeteer": "1.11.0",+ "puppeteer": "1.12.2","selenium-webdriver": "^3.6.0"}}