Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Merging develop

Marek Trunkat290cf6018804
diff --cc CHANGELOG.md
index 3cff79242,13332509f..0198824f0
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@@ -1,15 -1,19 +1,21 @@@
- 0.5.14 / 2018-03-09
+ 0.5.14 / 2018-03-20
===================
- - Added back `--no-sandbox` to launch of Puppeteer to avoid issues on older kernels
+ - Removed again the --disable-dev-shm-usage flag when launching headless Chrome,
+ it might be causing issues with high IO overheads
+ - Upgraded Puppeteer to version 1.2.0
+- Added `finishWhenEmpty` and `maybeRunPromiseIntervalMillis` options to `AutoscaledPool` class.
+- Fixed false positive errors logged by `PuppeteerPool` class.
+ 0.5.11 / 2018-03-09
+ ===================
-- Added back --no-sandbox to launch of Puppeteer to avoid issues on older kernels
++- Added back `--no-sandbox` to launch of Puppeteer to avoid issues on older kernels
+
0.5.10 / 2018-03-09
===================
-- If the `APIFY_XVFB` env var is set to '1', then avoid headless mode and use Xvfb instead
+- If the `APIFY_XVFB` env var is set to `1`, then avoid headless mode and use Xvfb instead
- Updated DEFAULT_USER_AGENT to Linux Chrome
-- Consolidated startup options for Chrome - use --disable-dev-shm-usage, skip --no-sandbox,
- use --disable-gpu only on Windows
+- Consolidated startup options for Chrome - use `--disable-dev-shm-usage`, skip `--no-sandbox`,
+ use `--disable-gpu` only on Windows
- Updated docs and package description
0.5.8 / 2018-03-06
diff --cc package.json
index 7973959c9,d9cc5db9c..7909025f8
--- a/package.json
+++ b/package.json
@@@ -1,6 -1,6 +1,6 @@@
{
"name": "apify",
- "version": "0.5.15",
- "version": "0.5.14",
++ "version": "0.5.16",
"description": "Apify SDK for JavaScript / Node.js",
"main": "build/index.js",
"keywords": [