Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Upgraded to [email protected]
Jan Curn242216f55e07
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex a33319ee0..f27630320 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,7 +4,7 @@ xxxin order to prevent stalling crawlers.- **BREAKING CHANGE:** `PseudoUrl` now performs case-insensitive matching, even for the query string part of the URLs.If you need case sensitive matching, use an appropriate `RegExp` in place of a Pseudo URL string-- **Upgraded to Puppeteer version 1.12.2**+- **Upgraded to [email protected]** and [email protected]- Added `loadedUrl` property to `Request` that contains the final URL of the loaded page after all redirects.- Added memory overload warning log message.- Added `keyValueStore.getPublicUrl` function.diff --git a/package.json b/package.jsonindex 42dc8f468..8d4d2f10a 100644--- a/package.json+++ b/package.json@@ -64,7 +64,7 @@"rimraf": "^2.6.2","underscore": "^1.9.1","ws": "^6.1.2",- "xregexp": "4.2.0"+ "xregexp": "^4.2.4"},"devDependencies": {"@babel/cli": "^7.0.0",