Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
chore(release): v3.3.2
Apify Release Bot4c929bd59cfd
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex edcc80039..cc9e85cc3 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -3,6 +3,27 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)+++### Bug Fixes++* **MemoryStorage:** cache requests in `RequestQueue` ([#1899](https://github.com/apify/crawlee/issues/1899)) ([063dcd1](https://github.com/apify/crawlee/commit/063dcd1c9e6652cd316cc0e8c4f4e4bbb70c246e))+* respect config object when creating `SessionPool` ([#1881](https://github.com/apify/crawlee/issues/1881)) ([db069df](https://github.com/apify/crawlee/commit/db069df80bc183c6b861c9ac82f1e278e57ea92b))+++### Features++* allow running single crawler instance multiple times ([#1844](https://github.com/apify/crawlee/issues/1844)) ([9e6eb1e](https://github.com/apify/crawlee/commit/9e6eb1e32f582a8837311aac12cc1d657432f3fa)), closes [#765](https://github.com/apify/crawlee/issues/765)+* **HttpCrawler:** add `parseWithCheerio` helper to `HttpCrawler` ([#1906](https://github.com/apify/crawlee/issues/1906)) ([ff5f76f](https://github.com/apify/crawlee/commit/ff5f76f9336c47c555c28038cdc72dc650bb5065))+* **router:** allow inline router definition ([#1877](https://github.com/apify/crawlee/issues/1877)) ([2d241c9](https://github.com/apify/crawlee/commit/2d241c9f88964ebd41a181069c378b6b7b5bf262))+* RQv2 memory storage support ([#1874](https://github.com/apify/crawlee/issues/1874)) ([049486b](https://github.com/apify/crawlee/commit/049486b772cc2accd2d2d226d8c8726e5ab933a9))+* support alternate storage clients when opening storages ([#1901](https://github.com/apify/crawlee/issues/1901)) ([661e550](https://github.com/apify/crawlee/commit/661e550dcf3609b75e2d7bc225c2f6914f45c93e))+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)diff --git a/lerna.json b/lerna.jsonindex e7b6fd772..b38805ab7 100644--- a/lerna.json+++ b/lerna.json@@ -2,7 +2,7 @@"packages": ["packages/*"],- "version": "3.3.1",+ "version": "3.3.2","command": {"version": {"conventionalCommits": true,diff --git a/packages/basic-crawler/CHANGELOG.md b/packages/basic-crawler/CHANGELOG.mdindex ca6d9ff03..a06b4ddd8 100644--- a/packages/basic-crawler/CHANGELOG.md+++ b/packages/basic-crawler/CHANGELOG.md@@ -3,6 +3,23 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)+++### Bug Fixes++* respect config object when creating `SessionPool` ([#1881](https://github.com/apify/crawlee/issues/1881)) ([db069df](https://github.com/apify/crawlee/commit/db069df80bc183c6b861c9ac82f1e278e57ea92b))+++### Features++* allow running single crawler instance multiple times ([#1844](https://github.com/apify/crawlee/issues/1844)) ([9e6eb1e](https://github.com/apify/crawlee/commit/9e6eb1e32f582a8837311aac12cc1d657432f3fa)), closes [#765](https://github.com/apify/crawlee/issues/765)+* **router:** allow inline router definition ([#1877](https://github.com/apify/crawlee/issues/1877)) ([2d241c9](https://github.com/apify/crawlee/commit/2d241c9f88964ebd41a181069c378b6b7b5bf262))+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)diff --git a/packages/basic-crawler/package.json b/packages/basic-crawler/package.jsonindex 03f39ecb0..f7902a7bf 100644--- a/packages/basic-crawler/package.json+++ b/packages/basic-crawler/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/basic",- "version": "3.3.1",+ "version": "3.3.2","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -48,9 +48,9 @@"@apify/log": "^2.0.0","@apify/timeout": "^0.3.0","@apify/utilities": "^2.0.0",- "@crawlee/core": "^3.3.1",- "@crawlee/types": "^3.3.1",- "@crawlee/utils": "^3.3.1",+ "@crawlee/core": "^3.3.2",+ "@crawlee/types": "^3.3.2",+ "@crawlee/utils": "^3.3.2","got-scraping": "^3.2.9","ow": "^0.28.1","tslib": "^2.4.0",diff --git a/packages/browser-crawler/CHANGELOG.md b/packages/browser-crawler/CHANGELOG.mdindex 9ba1a2e3b..a9d3b8c4b 100644--- a/packages/browser-crawler/CHANGELOG.md+++ b/packages/browser-crawler/CHANGELOG.md@@ -3,6 +3,14 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)++**Note:** Version bump only for package @crawlee/browser+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)**Note:** Version bump only for package @crawlee/browserdiff --git a/packages/browser-crawler/package.json b/packages/browser-crawler/package.jsonindex 037d1b426..ab69165b0 100644--- a/packages/browser-crawler/package.json+++ b/packages/browser-crawler/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/browser",- "version": "3.3.1",+ "version": "3.3.2","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -54,10 +54,10 @@},"dependencies": {"@apify/timeout": "^0.3.0",- "@crawlee/basic": "^3.3.1",- "@crawlee/browser-pool": "^3.3.1",- "@crawlee/types": "^3.3.1",- "@crawlee/utils": "^3.3.1",+ "@crawlee/basic": "^3.3.2",+ "@crawlee/browser-pool": "^3.3.2",+ "@crawlee/types": "^3.3.2",+ "@crawlee/utils": "^3.3.2","ow": "^0.28.1","tslib": "^2.4.0"}diff --git a/packages/browser-pool/CHANGELOG.md b/packages/browser-pool/CHANGELOG.mdindex a2acca03d..139f7ea36 100644--- a/packages/browser-pool/CHANGELOG.md+++ b/packages/browser-pool/CHANGELOG.md@@ -3,6 +3,14 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)++**Note:** Version bump only for package @crawlee/browser-pool+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)**Note:** Version bump only for package @crawlee/browser-pooldiff --git a/packages/browser-pool/package.json b/packages/browser-pool/package.jsonindex f3b2c5b93..35ce4fa63 100644--- a/packages/browser-pool/package.json+++ b/packages/browser-pool/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/browser-pool",- "version": "3.3.1",+ "version": "3.3.2","description": "Rotate multiple browsers using popular automation libraries such as Playwright or Puppeteer.","engines": {"node": ">=16.0.0"@@ -38,7 +38,7 @@"dependencies": {"@apify/log": "^2.0.0","@apify/timeout": "^0.3.0",- "@crawlee/types": "^3.3.1",+ "@crawlee/types": "^3.3.2","fingerprint-generator": "^2.0.6","fingerprint-injector": "^2.0.5","lodash.merge": "^4.6.2",diff --git a/packages/cheerio-crawler/CHANGELOG.md b/packages/cheerio-crawler/CHANGELOG.mdindex 3a145e4c5..3d123f7f0 100644--- a/packages/cheerio-crawler/CHANGELOG.md+++ b/packages/cheerio-crawler/CHANGELOG.md@@ -3,6 +3,17 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)+++### Features++* **router:** allow inline router definition ([#1877](https://github.com/apify/crawlee/issues/1877)) ([2d241c9](https://github.com/apify/crawlee/commit/2d241c9f88964ebd41a181069c378b6b7b5bf262))+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)diff --git a/packages/cheerio-crawler/package.json b/packages/cheerio-crawler/package.jsonindex be55a5f4c..a5707826d 100644--- a/packages/cheerio-crawler/package.json+++ b/packages/cheerio-crawler/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/cheerio",- "version": "3.3.1",+ "version": "3.3.2","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -53,8 +53,8 @@"access": "public"},"dependencies": {- "@crawlee/http": "^3.3.1",- "@crawlee/types": "^3.3.1",+ "@crawlee/http": "^3.3.2",+ "@crawlee/types": "^3.3.2","cheerio": "^1.0.0-rc.12","htmlparser2": "^9.0.0","tslib": "^2.4.0"diff --git a/packages/cli/CHANGELOG.md b/packages/cli/CHANGELOG.mdindex 2f09827a7..338a41377 100644--- a/packages/cli/CHANGELOG.md+++ b/packages/cli/CHANGELOG.md@@ -3,6 +3,14 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)++**Note:** Version bump only for package @crawlee/cli+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)**Note:** Version bump only for package @crawlee/clidiff --git a/packages/cli/package.json b/packages/cli/package.jsonindex 1098649b8..ef645703d 100644--- a/packages/cli/package.json+++ b/packages/cli/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/cli",- "version": "3.3.1",+ "version": "3.3.2","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -51,7 +51,7 @@"access": "public"},"dependencies": {- "@crawlee/templates": "^3.3.1",+ "@crawlee/templates": "^3.3.2","ansi-colors": "^4.1.3","fs-extra": "^11.0.0","inquirer": "^8.2.4",diff --git a/packages/core/CHANGELOG.md b/packages/core/CHANGELOG.mdindex 8381fc4ac..994bc959f 100644--- a/packages/core/CHANGELOG.md+++ b/packages/core/CHANGELOG.md@@ -3,6 +3,24 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)+++### Bug Fixes++* respect config object when creating `SessionPool` ([#1881](https://github.com/apify/crawlee/issues/1881)) ([db069df](https://github.com/apify/crawlee/commit/db069df80bc183c6b861c9ac82f1e278e57ea92b))+++### Features++* allow running single crawler instance multiple times ([#1844](https://github.com/apify/crawlee/issues/1844)) ([9e6eb1e](https://github.com/apify/crawlee/commit/9e6eb1e32f582a8837311aac12cc1d657432f3fa)), closes [#765](https://github.com/apify/crawlee/issues/765)+* **router:** allow inline router definition ([#1877](https://github.com/apify/crawlee/issues/1877)) ([2d241c9](https://github.com/apify/crawlee/commit/2d241c9f88964ebd41a181069c378b6b7b5bf262))+* support alternate storage clients when opening storages ([#1901](https://github.com/apify/crawlee/issues/1901)) ([661e550](https://github.com/apify/crawlee/commit/661e550dcf3609b75e2d7bc225c2f6914f45c93e))+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)diff --git a/packages/core/package.json b/packages/core/package.jsonindex 107766438..0df7a6be5 100644--- a/packages/core/package.json+++ b/packages/core/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/core",- "version": "3.3.1",+ "version": "3.3.2","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -59,9 +59,9 @@"@apify/pseudo_url": "^2.0.14","@apify/timeout": "^0.3.0","@apify/utilities": "^2.3.3",- "@crawlee/memory-storage": "^3.3.1",- "@crawlee/types": "^3.3.1",- "@crawlee/utils": "^3.3.1",+ "@crawlee/memory-storage": "^3.3.2",+ "@crawlee/types": "^3.3.2",+ "@crawlee/utils": "^3.3.2","@sapphire/async-queue": "^1.5.0","@types/tough-cookie": "^4.0.2","@vladfrangu/async_event_emitter": "^2.0.0",diff --git a/packages/crawlee/CHANGELOG.md b/packages/crawlee/CHANGELOG.mdindex cd380b41e..fb9a8e24d 100644--- a/packages/crawlee/CHANGELOG.md+++ b/packages/crawlee/CHANGELOG.md@@ -3,6 +3,14 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)++**Note:** Version bump only for package crawlee+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)**Note:** Version bump only for package crawleediff --git a/packages/crawlee/package.json b/packages/crawlee/package.jsonindex 78cfcd64c..fcdd1fbd3 100644--- a/packages/crawlee/package.json+++ b/packages/crawlee/package.json@@ -1,6 +1,6 @@{"name": "crawlee",- "version": "3.3.1",+ "version": "3.3.2","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -54,16 +54,16 @@"access": "public"},"dependencies": {- "@crawlee/basic": "^3.3.1",- "@crawlee/browser": "^3.3.1",- "@crawlee/cheerio": "^3.3.1",- "@crawlee/cli": "^3.3.1",- "@crawlee/core": "^3.3.1",- "@crawlee/http": "^3.3.1",- "@crawlee/jsdom": "^3.3.1",- "@crawlee/playwright": "^3.3.1",- "@crawlee/puppeteer": "^3.3.1",- "@crawlee/utils": "^3.3.1",+ "@crawlee/basic": "^3.3.2",+ "@crawlee/browser": "^3.3.2",+ "@crawlee/cheerio": "^3.3.2",+ "@crawlee/cli": "^3.3.2",+ "@crawlee/core": "^3.3.2",+ "@crawlee/http": "^3.3.2",+ "@crawlee/jsdom": "^3.3.2",+ "@crawlee/playwright": "^3.3.2",+ "@crawlee/puppeteer": "^3.3.2",+ "@crawlee/utils": "^3.3.2","import-local": "^3.1.0","tslib": "^2.4.0"},diff --git a/packages/http-crawler/CHANGELOG.md b/packages/http-crawler/CHANGELOG.mdindex 1d56285ca..1c94f588c 100644--- a/packages/http-crawler/CHANGELOG.md+++ b/packages/http-crawler/CHANGELOG.md@@ -3,6 +3,18 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)+++### Features++* **HttpCrawler:** add `parseWithCheerio` helper to `HttpCrawler` ([#1906](https://github.com/apify/crawlee/issues/1906)) ([ff5f76f](https://github.com/apify/crawlee/commit/ff5f76f9336c47c555c28038cdc72dc650bb5065))+* **router:** allow inline router definition ([#1877](https://github.com/apify/crawlee/issues/1877)) ([2d241c9](https://github.com/apify/crawlee/commit/2d241c9f88964ebd41a181069c378b6b7b5bf262))+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)diff --git a/packages/http-crawler/package.json b/packages/http-crawler/package.jsonindex 7005ece21..ffdfe9b07 100644--- a/packages/http-crawler/package.json+++ b/packages/http-crawler/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/http",- "version": "3.3.1",+ "version": "3.3.2","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -55,8 +55,8 @@"dependencies": {"@apify/timeout": "^0.3.0","@apify/utilities": "^2.0.0",- "@crawlee/basic": "^3.3.1",- "@crawlee/types": "^3.3.1",+ "@crawlee/basic": "^3.3.2",+ "@crawlee/types": "^3.3.2","@types/content-type": "^1.1.5","cheerio": "^1.0.0-rc.12","content-type": "^1.0.4",diff --git a/packages/jsdom-crawler/CHANGELOG.md b/packages/jsdom-crawler/CHANGELOG.mdindex fae21c6cc..f00e6ec38 100644--- a/packages/jsdom-crawler/CHANGELOG.md+++ b/packages/jsdom-crawler/CHANGELOG.md@@ -3,6 +3,18 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)+++### Features++* **HttpCrawler:** add `parseWithCheerio` helper to `HttpCrawler` ([#1906](https://github.com/apify/crawlee/issues/1906)) ([ff5f76f](https://github.com/apify/crawlee/commit/ff5f76f9336c47c555c28038cdc72dc650bb5065))+* **router:** allow inline router definition ([#1877](https://github.com/apify/crawlee/issues/1877)) ([2d241c9](https://github.com/apify/crawlee/commit/2d241c9f88964ebd41a181069c378b6b7b5bf262))+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)diff --git a/packages/jsdom-crawler/package.json b/packages/jsdom-crawler/package.jsonindex 87fe81d6f..4bc64a31f 100644--- a/packages/jsdom-crawler/package.json+++ b/packages/jsdom-crawler/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/jsdom",- "version": "3.3.1",+ "version": "3.3.2","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -55,8 +55,8 @@"dependencies": {"@apify/timeout": "^0.3.0","@apify/utilities": "^2.1.4",- "@crawlee/http": "^3.3.1",- "@crawlee/types": "^3.3.1",+ "@crawlee/http": "^3.3.2",+ "@crawlee/types": "^3.3.2","@types/jsdom": "^21.0.0","cheerio": "^1.0.0-rc.12","jsdom": "^22.0.0",diff --git a/packages/memory-storage/CHANGELOG.md b/packages/memory-storage/CHANGELOG.mdindex c23b3b11c..3b8d09856 100644--- a/packages/memory-storage/CHANGELOG.md+++ b/packages/memory-storage/CHANGELOG.md@@ -3,6 +3,22 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)+++### Bug Fixes++* **MemoryStorage:** cache requests in `RequestQueue` ([#1899](https://github.com/apify/crawlee/issues/1899)) ([063dcd1](https://github.com/apify/crawlee/commit/063dcd1c9e6652cd316cc0e8c4f4e4bbb70c246e))+++### Features++* RQv2 memory storage support ([#1874](https://github.com/apify/crawlee/issues/1874)) ([049486b](https://github.com/apify/crawlee/commit/049486b772cc2accd2d2d226d8c8726e5ab933a9))+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)diff --git a/packages/memory-storage/package.json b/packages/memory-storage/package.jsonindex 4f8901b62..c7290bccb 100644--- a/packages/memory-storage/package.json+++ b/packages/memory-storage/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/memory-storage",- "version": "3.3.1",+ "version": "3.3.2","description": "A simple in-memory storage implementation of the Apify API","engines": {"node": ">= 16"@@ -49,7 +49,7 @@},"dependencies": {"@apify/log": "^2.0.0",- "@crawlee/types": "^3.3.1",+ "@crawlee/types": "^3.3.2","@sapphire/async-queue": "^1.5.0","@sapphire/shapeshift": "^3.0.0","content-type": "^1.0.4",diff --git a/packages/playwright-crawler/CHANGELOG.md b/packages/playwright-crawler/CHANGELOG.mdindex 834122f14..c3a29c8d3 100644--- a/packages/playwright-crawler/CHANGELOG.md+++ b/packages/playwright-crawler/CHANGELOG.md@@ -3,6 +3,17 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)+++### Features++* **router:** allow inline router definition ([#1877](https://github.com/apify/crawlee/issues/1877)) ([2d241c9](https://github.com/apify/crawlee/commit/2d241c9f88964ebd41a181069c378b6b7b5bf262))+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)diff --git a/packages/playwright-crawler/package.json b/packages/playwright-crawler/package.jsonindex b2f4d6ab2..18e34fe08 100644--- a/packages/playwright-crawler/package.json+++ b/packages/playwright-crawler/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/playwright",- "version": "3.3.1",+ "version": "3.3.2","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -55,10 +55,10 @@"dependencies": {"@apify/datastructures": "^2.0.0","@apify/log": "^2.0.0",- "@crawlee/browser": "^3.3.1",- "@crawlee/browser-pool": "^3.3.1",- "@crawlee/types": "^3.3.1",- "@crawlee/utils": "^3.3.1",+ "@crawlee/browser": "^3.3.2",+ "@crawlee/browser-pool": "^3.3.2",+ "@crawlee/types": "^3.3.2",+ "@crawlee/utils": "^3.3.2","cheerio": "^1.0.0-rc.12","jquery": "^3.6.0","ow": "^0.28.1",diff --git a/packages/puppeteer-crawler/CHANGELOG.md b/packages/puppeteer-crawler/CHANGELOG.mdindex c6e4a0521..a62b946ea 100644--- a/packages/puppeteer-crawler/CHANGELOG.md+++ b/packages/puppeteer-crawler/CHANGELOG.md@@ -3,6 +3,17 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)+++### Features++* **router:** allow inline router definition ([#1877](https://github.com/apify/crawlee/issues/1877)) ([2d241c9](https://github.com/apify/crawlee/commit/2d241c9f88964ebd41a181069c378b6b7b5bf262))+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)diff --git a/packages/puppeteer-crawler/package.json b/packages/puppeteer-crawler/package.jsonindex bd07273ae..bf13dd9cf 100644--- a/packages/puppeteer-crawler/package.json+++ b/packages/puppeteer-crawler/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/puppeteer",- "version": "3.3.1",+ "version": "3.3.2","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -55,10 +55,10 @@"dependencies": {"@apify/datastructures": "^2.0.0","@apify/log": "^2.0.0",- "@crawlee/browser": "^3.3.1",- "@crawlee/browser-pool": "^3.3.1",- "@crawlee/types": "^3.3.1",- "@crawlee/utils": "^3.3.1",+ "@crawlee/browser": "^3.3.2",+ "@crawlee/browser-pool": "^3.3.2",+ "@crawlee/types": "^3.3.2",+ "@crawlee/utils": "^3.3.2","cheerio": "^1.0.0-rc.12","devtools-protocol": "*","jquery": "^3.6.0",diff --git a/packages/templates/CHANGELOG.md b/packages/templates/CHANGELOG.mdindex 58f355632..5896a6b4f 100644--- a/packages/templates/CHANGELOG.md+++ b/packages/templates/CHANGELOG.md@@ -3,6 +3,14 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)++**Note:** Version bump only for package @crawlee/templates+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)diff --git a/packages/templates/package.json b/packages/templates/package.jsonindex 1cbe87a4e..8d947b4ca 100644--- a/packages/templates/package.json+++ b/packages/templates/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/templates",- "version": "3.3.1",+ "version": "3.3.2","description": "Templates for the crawlee projects","engines": {"node": ">=16.0.0"diff --git a/packages/types/CHANGELOG.md b/packages/types/CHANGELOG.mdindex 938ea534d..bf2fb80a3 100644--- a/packages/types/CHANGELOG.md+++ b/packages/types/CHANGELOG.md@@ -3,6 +3,17 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)+++### Features++* RQv2 memory storage support ([#1874](https://github.com/apify/crawlee/issues/1874)) ([049486b](https://github.com/apify/crawlee/commit/049486b772cc2accd2d2d226d8c8726e5ab933a9))+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)**Note:** Version bump only for package @crawlee/typesdiff --git a/packages/types/package.json b/packages/types/package.jsonindex ae35e2450..ef6b54a3d 100644--- a/packages/types/package.json+++ b/packages/types/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/types",- "version": "3.3.1",+ "version": "3.3.2","description": "Shared types for the crawlee projects","engines": {"node": ">=16.0.0"diff --git a/packages/utils/CHANGELOG.md b/packages/utils/CHANGELOG.mdindex f4b8811c2..cde5c593c 100644--- a/packages/utils/CHANGELOG.md+++ b/packages/utils/CHANGELOG.md@@ -3,6 +3,14 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+## [3.3.2](https://github.com/apify/crawlee/compare/v3.3.1...v3.3.2) (2023-05-11)++**Note:** Version bump only for package @crawlee/utils+++++## [3.3.1](https://github.com/apify/crawlee/compare/v3.3.0...v3.3.1) (2023-04-11)diff --git a/packages/utils/package.json b/packages/utils/package.jsonindex 9dc07e55e..f037ed6b7 100644--- a/packages/utils/package.json+++ b/packages/utils/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/utils",- "version": "3.3.1",+ "version": "3.3.2","description": "A set of shared utilities that can be used by crawlers","engines": {"node": ">=16.0.0"@@ -49,7 +49,7 @@"dependencies": {"@apify/log": "^2.0.0","@apify/ps-tree": "^1.1.4",- "@crawlee/types": "^3.3.1",+ "@crawlee/types": "^3.3.2","cheerio": "^1.0.0-rc.12","got-scraping": "^3.2.9","ow": "^0.28.1",diff --git a/yarn.lock b/yarn.lockindex fa4cf7322..aca45bdad 100644--- a/yarn.lock+++ b/yarn.lock@@ -732,16 +732,16 @@ __metadata:languageName: nodelinkType: hard-"@crawlee/basic@^3.3.1, @crawlee/basic@workspace:packages/basic-crawler":+"@crawlee/basic@^3.3.2, @crawlee/basic@workspace:packages/basic-crawler":version: 0.0.0-use.localresolution: "@crawlee/basic@workspace:packages/basic-crawler"dependencies:"@apify/log": ^2.0.0"@apify/timeout": ^0.3.0"@apify/utilities": ^2.0.0- "@crawlee/core": ^3.3.1- "@crawlee/types": ^3.3.1- "@crawlee/utils": ^3.3.1+ "@crawlee/core": ^3.3.2+ "@crawlee/types": ^3.3.2+ "@crawlee/utils": ^3.3.2got-scraping: ^3.2.9ow: ^0.28.1tslib: ^2.4.0@@ -749,13 +749,13 @@ __metadata:languageName: unknownlinkType: soft-"@crawlee/browser-pool@^3.3.1, @crawlee/browser-pool@workspace:packages/browser-pool":+"@crawlee/browser-pool@^3.3.2, @crawlee/browser-pool@workspace:packages/browser-pool":version: 0.0.0-use.localresolution: "@crawlee/browser-pool@workspace:packages/browser-pool"dependencies:"@apify/log": ^2.0.0"@apify/timeout": ^0.3.0- "@crawlee/types": ^3.3.1+ "@crawlee/types": ^3.3.2fingerprint-generator: ^2.0.6fingerprint-injector: ^2.0.5lodash.merge: ^4.6.2@@ -777,37 +777,37 @@ __metadata:languageName: unknownlinkType: soft-"@crawlee/browser@^3.3.1, @crawlee/browser@workspace:packages/browser-crawler":+"@crawlee/browser@^3.3.2, @crawlee/browser@workspace:packages/browser-crawler":version: 0.0.0-use.localresolution: "@crawlee/browser@workspace:packages/browser-crawler"dependencies:"@apify/timeout": ^0.3.0- "@crawlee/basic": ^3.3.1- "@crawlee/browser-pool": ^3.3.1- "@crawlee/types": ^3.3.1- "@crawlee/utils": ^3.3.1+ "@crawlee/basic": ^3.3.2+ "@crawlee/browser-pool": ^3.3.2+ "@crawlee/types": ^3.3.2+ "@crawlee/utils": ^3.3.2ow: ^0.28.1tslib: ^2.4.0languageName: unknownlinkType: soft-"@crawlee/cheerio@^3.3.1, @crawlee/cheerio@workspace:packages/cheerio-crawler":+"@crawlee/cheerio@^3.3.2, @crawlee/cheerio@workspace:packages/cheerio-crawler":version: 0.0.0-use.localresolution: "@crawlee/cheerio@workspace:packages/cheerio-crawler"dependencies:- "@crawlee/http": ^3.3.1- "@crawlee/types": ^3.3.1+ "@crawlee/http": ^3.3.2+ "@crawlee/types": ^3.3.2cheerio: ^1.0.0-rc.12htmlparser2: ^9.0.0tslib: ^2.4.0languageName: unknownlinkType: soft-"@crawlee/cli@^3.3.1, @crawlee/cli@workspace:packages/cli":+"@crawlee/cli@^3.3.2, @crawlee/cli@workspace:packages/cli":version: 0.0.0-use.localresolution: "@crawlee/cli@workspace:packages/cli"dependencies:- "@crawlee/templates": ^3.3.1+ "@crawlee/templates": ^3.3.2ansi-colors: ^4.1.3fs-extra: ^11.0.0inquirer: ^8.2.4@@ -819,7 +819,7 @@ __metadata:languageName: unknownlinkType: soft-"@crawlee/core@^3.3.0, @crawlee/core@^3.3.1, @crawlee/core@workspace:packages/core":+"@crawlee/core@^3.3.0, @crawlee/core@^3.3.2, @crawlee/core@workspace:packages/core":version: 0.0.0-use.localresolution: "@crawlee/core@workspace:packages/core"dependencies:@@ -829,9 +829,9 @@ __metadata:"@apify/pseudo_url": ^2.0.14"@apify/timeout": ^0.3.0"@apify/utilities": ^2.3.3- "@crawlee/memory-storage": ^3.3.1- "@crawlee/types": ^3.3.1- "@crawlee/utils": ^3.3.1+ "@crawlee/memory-storage": ^3.3.2+ "@crawlee/types": ^3.3.2+ "@crawlee/utils": ^3.3.2"@sapphire/async-queue": ^1.5.0"@types/tough-cookie": ^4.0.2"@vladfrangu/async_event_emitter": ^2.0.0@@ -849,14 +849,14 @@ __metadata:languageName: unknownlinkType: soft-"@crawlee/http@^3.3.1, @crawlee/http@workspace:packages/http-crawler":+"@crawlee/http@^3.3.2, @crawlee/http@workspace:packages/http-crawler":version: 0.0.0-use.localresolution: "@crawlee/http@workspace:packages/http-crawler"dependencies:"@apify/timeout": ^0.3.0"@apify/utilities": ^2.0.0- "@crawlee/basic": ^3.3.1- "@crawlee/types": ^3.3.1+ "@crawlee/basic": ^3.3.2+ "@crawlee/types": ^3.3.2"@types/content-type": ^1.1.5cheerio: ^1.0.0-rc.12content-type: ^1.0.4@@ -869,14 +869,14 @@ __metadata:languageName: unknownlinkType: soft-"@crawlee/jsdom@^3.3.1, @crawlee/jsdom@workspace:packages/jsdom-crawler":+"@crawlee/jsdom@^3.3.2, @crawlee/jsdom@workspace:packages/jsdom-crawler":version: 0.0.0-use.localresolution: "@crawlee/jsdom@workspace:packages/jsdom-crawler"dependencies:"@apify/timeout": ^0.3.0"@apify/utilities": ^2.1.4- "@crawlee/http": ^3.3.1- "@crawlee/types": ^3.3.1+ "@crawlee/http": ^3.3.2+ "@crawlee/types": ^3.3.2"@types/jsdom": ^21.0.0cheerio: ^1.0.0-rc.12jsdom: ^22.0.0@@ -885,12 +885,12 @@ __metadata:languageName: unknownlinkType: soft-"@crawlee/memory-storage@^3.3.1, @crawlee/memory-storage@workspace:packages/memory-storage":+"@crawlee/memory-storage@^3.3.2, @crawlee/memory-storage@workspace:packages/memory-storage":version: 0.0.0-use.localresolution: "@crawlee/memory-storage@workspace:packages/memory-storage"dependencies:"@apify/log": ^2.0.0- "@crawlee/types": ^3.3.1+ "@crawlee/types": ^3.3.2"@sapphire/async-queue": ^1.5.0"@sapphire/shapeshift": ^3.0.0content-type: ^1.0.4@@ -902,16 +902,16 @@ __metadata:languageName: unknownlinkType: soft-"@crawlee/playwright@^3.3.1, @crawlee/playwright@workspace:packages/playwright-crawler":+"@crawlee/playwright@^3.3.2, @crawlee/playwright@workspace:packages/playwright-crawler":version: 0.0.0-use.localresolution: "@crawlee/playwright@workspace:packages/playwright-crawler"dependencies:"@apify/datastructures": ^2.0.0"@apify/log": ^2.0.0- "@crawlee/browser": ^3.3.1- "@crawlee/browser-pool": ^3.3.1- "@crawlee/types": ^3.3.1- "@crawlee/utils": ^3.3.1+ "@crawlee/browser": ^3.3.2+ "@crawlee/browser-pool": ^3.3.2+ "@crawlee/types": ^3.3.2+ "@crawlee/utils": ^3.3.2cheerio: ^1.0.0-rc.12jquery: ^3.6.0ow: ^0.28.1@@ -924,16 +924,16 @@ __metadata:languageName: unknownlinkType: soft-"@crawlee/puppeteer@^3.3.1, @crawlee/puppeteer@workspace:packages/puppeteer-crawler":+"@crawlee/puppeteer@^3.3.2, @crawlee/puppeteer@workspace:packages/puppeteer-crawler":version: 0.0.0-use.localresolution: "@crawlee/puppeteer@workspace:packages/puppeteer-crawler"dependencies:"@apify/datastructures": ^2.0.0"@apify/log": ^2.0.0- "@crawlee/browser": ^3.3.1- "@crawlee/browser-pool": ^3.3.1- "@crawlee/types": ^3.3.1- "@crawlee/utils": ^3.3.1+ "@crawlee/browser": ^3.3.2+ "@crawlee/browser-pool": ^3.3.2+ "@crawlee/types": ^3.3.2+ "@crawlee/utils": ^3.3.2cheerio: ^1.0.0-rc.12devtools-protocol: "*"jquery: ^3.6.0@@ -1003,7 +1003,7 @@ __metadata:languageName: unknownlinkType: soft-"@crawlee/templates@^3.3.1, @crawlee/templates@workspace:packages/templates":+"@crawlee/templates@^3.3.2, @crawlee/templates@workspace:packages/templates":version: 0.0.0-use.localresolution: "@crawlee/templates@workspace:packages/templates"dependencies:@@ -1015,7 +1015,7 @@ __metadata:languageName: unknownlinkType: soft-"@crawlee/types@^3.3.0, @crawlee/types@^3.3.1, @crawlee/types@workspace:packages/types":+"@crawlee/types@^3.3.0, @crawlee/types@^3.3.2, @crawlee/types@workspace:packages/types":version: 0.0.0-use.localresolution: "@crawlee/types@workspace:packages/types"dependencies:@@ -1023,13 +1023,13 @@ __metadata:languageName: unknownlinkType: soft-"@crawlee/utils@^3.3.0, @crawlee/utils@^3.3.1, @crawlee/utils@workspace:packages/utils":+"@crawlee/utils@^3.3.0, @crawlee/utils@^3.3.2, @crawlee/utils@workspace:packages/utils":version: 0.0.0-use.localresolution: "@crawlee/utils@workspace:packages/utils"dependencies:"@apify/log": ^2.0.0"@apify/ps-tree": ^1.1.4- "@crawlee/types": ^3.3.1+ "@crawlee/types": ^3.3.2cheerio: ^1.0.0-rc.12got-scraping: ^3.2.9ow: ^0.28.1@@ -4524,16 +4524,16 @@ __metadata:version: 0.0.0-use.localresolution: "crawlee@workspace:packages/crawlee"dependencies:- "@crawlee/basic": ^3.3.1- "@crawlee/browser": ^3.3.1- "@crawlee/cheerio": ^3.3.1- "@crawlee/cli": ^3.3.1- "@crawlee/core": ^3.3.1- "@crawlee/http": ^3.3.1- "@crawlee/jsdom": ^3.3.1- "@crawlee/playwright": ^3.3.1- "@crawlee/puppeteer": ^3.3.1- "@crawlee/utils": ^3.3.1+ "@crawlee/basic": ^3.3.2+ "@crawlee/browser": ^3.3.2+ "@crawlee/cheerio": ^3.3.2+ "@crawlee/cli": ^3.3.2+ "@crawlee/core": ^3.3.2+ "@crawlee/http": ^3.3.2+ "@crawlee/jsdom": ^3.3.2+ "@crawlee/playwright": ^3.3.2+ "@crawlee/puppeteer": ^3.3.2+ "@crawlee/utils": ^3.3.2import-local: ^3.1.0tslib: ^2.4.0peerDependencies: