Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Adding backwards compatibility for APIFY_LOCAL_EMULATION_DIR and updating CHANGELOG.md
Marek Trunkat3a0f44fbeac3
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex fcf5c7d73..d4deb2373 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -2,6 +2,9 @@ xxx==================- Updated DEFAULT_USER_AGENT and USER_AGENT_LIST- Added `recycleDiskCache` option to `PuppeteerPool` to enable reuse of disk cache and thus speedup browsing+- **WARNING**: `APIFY_LOCAL_EMULATION_DIR` environment variable was renamed to `APIFY_LOCAL_STORAGE_DIR`.+- Environment variables `APIFY_DEFAULT_KEY_VALUE_STORE_ID`, `APIFY_DEFAULT_REQUEST_QUEUE_ID` and `APIFY_DEFAULT_DATASET_ID`+ have now default value `default` so there is no need to define them when developing locally.0.6.4 / 2018-09-05==================diff --git a/src/index.js b/src/index.jsindex a8d71d9e2..2a68d8277 100644--- a/src/index.js+++ b/src/index.js@@ -27,6 +27,13 @@ if (!isProduction() || process.env[ENV_VARS.LOG_LEVEL] === 'DEBUG') log.isDebugM// Log as plain text not JSONlog.logJson = false;+// TODO: remove this when we release v1.0.0+const EMULATION_ENV_VAR = 'APIFY_LOCAL_EMULATION_DIR';+if (process.env[EMULATION_ENV_VAR]) {+ log.warning(`Environment variable "${EMULATION_ENV_VAR}" is deprecated!!! Use "${ENV_VARS.LOCAL_STORAGE_DIR}" instead!`);+ if (!process.env[ENV_VARS.LOCAL_STORAGE_DIR]) process.env[ENV_VARS.LOCAL_STORAGE_DIR] = process.env[EMULATION_ENV_VAR];+}+/***{include-readme-1}*{include-readme-2}