Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Merging develop
Marek Trunkat290cf6018804
diff --cc CHANGELOG.mdindex 3cff79242,13332509f..0198824f0--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -1,15 -1,19 +1,21 @@@- 0.5.14 / 2018-03-09+ 0.5.14 / 2018-03-20===================- - Added back `--no-sandbox` to launch of Puppeteer to avoid issues on older kernels+ - Removed again the --disable-dev-shm-usage flag when launching headless Chrome,+ it might be causing issues with high IO overheads+ - Upgraded Puppeteer to version 1.2.0+- Added `finishWhenEmpty` and `maybeRunPromiseIntervalMillis` options to `AutoscaledPool` class.+- Fixed false positive errors logged by `PuppeteerPool` class.+ 0.5.11 / 2018-03-09+ ===================-- Added back --no-sandbox to launch of Puppeteer to avoid issues on older kernels++- Added back `--no-sandbox` to launch of Puppeteer to avoid issues on older kernels+0.5.10 / 2018-03-09===================-- If the `APIFY_XVFB` env var is set to '1', then avoid headless mode and use Xvfb instead+- If the `APIFY_XVFB` env var is set to `1`, then avoid headless mode and use Xvfb instead- Updated DEFAULT_USER_AGENT to Linux Chrome-- Consolidated startup options for Chrome - use --disable-dev-shm-usage, skip --no-sandbox,- use --disable-gpu only on Windows+- Consolidated startup options for Chrome - use `--disable-dev-shm-usage`, skip `--no-sandbox`,+ use `--disable-gpu` only on Windows- Updated docs and package description0.5.8 / 2018-03-06diff --cc package.jsonindex 7973959c9,d9cc5db9c..7909025f8--- a/package.json+++ b/package.json@@@ -1,6 -1,6 +1,6 @@@{"name": "apify",- "version": "0.5.15",- "version": "0.5.14",++ "version": "0.5.16","description": "Apify SDK for JavaScript / Node.js","main": "build/index.js","keywords": [