Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Fix merge conflicts

Ondra Urban2748ffd3516d
diff --cc CHANGELOG.md
index d9fe6fa57,f3e5f3d8c..bc8bc8b1d
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@@ -1,9 -1,13 +1,15 @@@
xxx
===================
+- Added `loadedUrl` property to `Request` that contains the `request.url` after redirects.
+- Reduced default `handlePageTimeoutSecs` for both `CheerioCrawler` and `PuppeteerCrawler` to `60` seconds.
+ - Added memory overload warning log message.
- 0.11.7 / 2018/01/30
+ 0.11.8 / 2019-02-05
+ ===================
+ - Fix the `keyValueStore.forEachKey()` method.
+ - Fix version of `puppeteer` to prevent errors with automatic updates.
+
+ 0.11.7 / 2019-01-30
===================
- Apify SDK now logs basic system info when `required`.
- Added `utils.createRequestDebugInfo()` function to create a standardized debug info from request and response.