Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Update changelog

Ondra Urban1d6bd403e11f
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8e91d3fdf..a3957865d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,13 @@
+xxx
+===================
+- **BREAKING CHANGE:** Added `puppeteerOperationTimeoutSecs` option to `PuppeteerPool`.
+ It defaults to 15 seconds and all Puppeteer operations such as `browser.newPage()`
+ or `puppeteer.launch()` will now time out. This is to prevent hanging requests.
+- **BREAKING CHANGE:** Added `handleRequestTimeoutSecs` option to `BasicCrawler` with a 60 second default.
+- `CheerioCrawler` and `PuppeteerCrawler` now automatically set `handleRequestTimeoutSecs` to double of
+ `handlePageTimeoutSecs`.
+- `puppeteerPool.newPage()` now retries twice before throwing an error.
+
0.12.2 / 2019-02-27
===================
- Added oldest active tab focusing to `PuppeteerPool` to combat resource throttling in Chromium.