Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Bump version, update changelog, update and build docs
Ondra Urban9863916990bb
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 117b80237..2923f5288 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,7 +1,14 @@NEXT====================+++0.19.0 / 2020-01-20+====================- **BREAKING**: `APIFY_LOCAL_EMULATION_DIR` env var is no longer supported (deprecated on 2018-09-11).Use `APIFY_LOCAL_STORAGE_DIR` instead.+- `SessionPool` API updates and fixes. The API is no longer considered experimental.+- Logging of system info moved from `require` time to `Apify.main()` invocation.+- Use native `RegExp` instead of `xregexp` for unicode property escapes.0.18.1 / 2020-01-06====================diff --git a/docs/api/Session.md b/docs/api/Session.mdindex 15a4711c0..31609e764 100644--- a/docs/api/Session.md+++ b/docs/api/Session.md@@ -5,9 +5,9 @@ title: Session<a name="Session"></a>-Class aggregating data for session. Sessions are used to store information such as cookies and can be used for generating fingerprints and proxy-sessions. You can imagine each session as a specific user, with its own cookies, IP (via proxy) and potentially a unique browser fingerprint. Session-internal state can be enriched with custom user data for example some authorization tokens and specific headers in general.+Sessions are used to store information such as cookies and can be used for generating fingerprints and proxy sessions. You can imagine each session as+a specific user, with its own cookies, IP (via proxy) and potentially a unique browser fingerprint. Session internal state can be enriched with custom+user data for example some authorization tokens and specific headers in general.- [Session](session)- [`new exports.Session()`](#new_Session_new)diff --git a/docs/api/SessionPool.md b/docs/api/SessionPool.mdindex 2f5d108c0..d60ce8d47 100644--- a/docs/api/SessionPool.md+++ b/docs/api/SessionPool.md@@ -5,8 +5,8 @@ title: SessionPool<a name="SessionPool"></a>-Experimental feature and might change in the future releases. Handles the sessions rotation, creation and persistence. Creates a pool of-[`Session`](session) instances, that are randomly rotated. When some session is marked as blocked. It is removed and new one is created instead.+Handles the sessions rotation, creation and persistence. Creates a pool of [`Session`](session) instances, that are randomly rotated. When some+session is marked as blocked. It is removed and new one is created instead.Session pool is by default persisted in default [`KeyValueStore`](keyvaluestore). If you want to have one pool for all runs you have to specify`persistStateKeyValueStoreId`.diff --git a/package.json b/package.jsonindex 80a143288..6ad607b60 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "apify",- "version": "0.19.0",+ "version": "0.19.1","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=10.17.0"diff --git a/src/session_pool/session.js b/src/session_pool/session.jsindex 0ce019b77..92ab8a222 100644--- a/src/session_pool/session.js+++ b/src/session_pool/session.js@@ -10,7 +10,6 @@ const { Cookie } = tough;/**- * Class aggregating data for session.* Sessions are used to store information such as cookies and can be used for generating fingerprints and proxy sessions.* You can imagine each session as a specific user, with its own cookies, IP (via proxy) and potentially a unique browser fingerprint.* Session internal state can be enriched with custom user data for example some authorization tokens and specific headers in general.diff --git a/src/session_pool/session_pool.js b/src/session_pool/session_pool.jsindex e33f4428b..639862ead 100644--- a/src/session_pool/session_pool.js+++ b/src/session_pool/session_pool.js@@ -9,7 +9,6 @@ import { ACTOR_EVENT_NAMES_EX } from '../constants';/**- * Experimental feature and might change in the future releases.* Handles the sessions rotation, creation and persistence.* Creates a pool of {@link Session} instances, that are randomly rotated.* When some session is marked as blocked. It is removed and new one is created instead.