Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
chore(release): v3.3.0
Apify Release Bot2e820aaa0f14
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex c4607692a..e6b160053 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -3,6 +3,29 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)+++### Bug Fixes++* add `proxyUrl` to `DownloadListOfUrlsOptions` ([779be1e](https://github.com/apify/crawlee/commit/779be1e4f29dff191d02e623eefb1bd5650c14ad)), closes [#1780](https://github.com/apify/crawlee/issues/1780)+* **CheerioCrawler:** pass ixXml down to response parser ([#1807](https://github.com/apify/crawlee/issues/1807)) ([af7a5c4](https://github.com/apify/crawlee/commit/af7a5c4efa94a53e5bdfeca340a9d7223d7dfda4)), closes [#1794](https://github.com/apify/crawlee/issues/1794)+* ignore invalid URLs in `enqueueLinks` in browser crawlers ([#1803](https://github.com/apify/crawlee/issues/1803)) ([5ac336c](https://github.com/apify/crawlee/commit/5ac336c5b83b212fd6281659b8ceee091e259ff1))+* **MemoryStorage:** request queues race conditions causing crashes ([#1806](https://github.com/apify/crawlee/issues/1806)) ([083a9db](https://github.com/apify/crawlee/commit/083a9db9ebcddd3fa886631234c790d4c5bcdf86)), closes [#1792](https://github.com/apify/crawlee/issues/1792)+* **MemoryStorage:** RequestQueue should respect `forefront` ([#1816](https://github.com/apify/crawlee/issues/1816)) ([b68e86a](https://github.com/apify/crawlee/commit/b68e86a97954bcbe30fde802fed5f263016fffe2)), closes [#1787](https://github.com/apify/crawlee/issues/1787)+* **MemoryStorage:** RequestQueue#handledRequestCount should update ([#1817](https://github.com/apify/crawlee/issues/1817)) ([a775e4a](https://github.com/apify/crawlee/commit/a775e4afea20d0b31492f44b90f61b6a903491b6)), closes [#1764](https://github.com/apify/crawlee/issues/1764)+++### Features++* add basic support for `setStatusMessage` ([#1790](https://github.com/apify/crawlee/issues/1790)) ([c318980](https://github.com/apify/crawlee/commit/c318980ec11d211b1a5c9e6bdbe76198c5d895be))+* **core:** add `exclude` option to `enqueueLinks` ([#1786](https://github.com/apify/crawlee/issues/1786)) ([2e833dc](https://github.com/apify/crawlee/commit/2e833dc4b0b82bb6741aa683f3fcba05244427df)), closes [#1785](https://github.com/apify/crawlee/issues/1785)+* move the status message implementation to Crawlee, noop in storage ([#1808](https://github.com/apify/crawlee/issues/1808)) ([99c3fdc](https://github.com/apify/crawlee/commit/99c3fdc18030b7898e6b6d149d6d94fab7881f09))+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)diff --git a/lerna.json b/lerna.jsonindex d5686ac36..ccc12437f 100644--- a/lerna.json+++ b/lerna.json@@ -1,20 +1,20 @@{- "packages": [- "packages/*"- ],- "version": "3.2.2",- "command": {- "version": {- "conventionalCommits": true,- "createRelease": "github",- "message": "chore(release): %s"- }- },- "npmClient": "yarn",- "useWorkspaces": true,- "useNx": false,- "ignoreChanges": [- "**/test/**",- "**/*.md"- ]+ "packages": [+ "packages/*"+ ],+ "version": "3.3.0",+ "command": {+ "version": {+ "conventionalCommits": true,+ "createRelease": "github",+ "message": "chore(release): %s"+ }+ },+ "npmClient": "yarn",+ "useWorkspaces": true,+ "useNx": false,+ "ignoreChanges": [+ "**/test/**",+ "**/*.md"+ ]}diff --git a/packages/basic-crawler/CHANGELOG.md b/packages/basic-crawler/CHANGELOG.mdindex 8b9309cee..a1ffcaf88 100644--- a/packages/basic-crawler/CHANGELOG.md+++ b/packages/basic-crawler/CHANGELOG.md@@ -3,6 +3,18 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)+++### Features++* add basic support for `setStatusMessage` ([#1790](https://github.com/apify/crawlee/issues/1790)) ([c318980](https://github.com/apify/crawlee/commit/c318980ec11d211b1a5c9e6bdbe76198c5d895be))+* move the status message implementation to Crawlee, noop in storage ([#1808](https://github.com/apify/crawlee/issues/1808)) ([99c3fdc](https://github.com/apify/crawlee/commit/99c3fdc18030b7898e6b6d149d6d94fab7881f09))+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)**Note:** Version bump only for package @crawlee/basicdiff --git a/packages/basic-crawler/package.json b/packages/basic-crawler/package.jsonindex a775bc227..d26f678f8 100644--- a/packages/basic-crawler/package.json+++ b/packages/basic-crawler/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/basic",- "version": "3.2.2",+ "version": "3.3.0","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -48,9 +48,9 @@"@apify/log": "^2.0.0","@apify/timeout": "^0.3.0","@apify/utilities": "^2.0.0",- "@crawlee/core": "^3.2.2",- "@crawlee/types": "^3.2.2",- "@crawlee/utils": "^3.2.2",+ "@crawlee/core": "^3.3.0",+ "@crawlee/types": "^3.3.0",+ "@crawlee/utils": "^3.3.0","got-scraping": "^3.2.9","ow": "^0.28.1","tslib": "^2.4.0",diff --git a/packages/browser-crawler/CHANGELOG.md b/packages/browser-crawler/CHANGELOG.mdindex 404a5c5dd..4af63ea6a 100644--- a/packages/browser-crawler/CHANGELOG.md+++ b/packages/browser-crawler/CHANGELOG.md@@ -3,6 +3,17 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)+++### Bug Fixes++* ignore invalid URLs in `enqueueLinks` in browser crawlers ([#1803](https://github.com/apify/crawlee/issues/1803)) ([5ac336c](https://github.com/apify/crawlee/commit/5ac336c5b83b212fd6281659b8ceee091e259ff1))+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)**Note:** Version bump only for package @crawlee/browserdiff --git a/packages/browser-crawler/package.json b/packages/browser-crawler/package.jsonindex 59f495544..7172c4784 100644--- a/packages/browser-crawler/package.json+++ b/packages/browser-crawler/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/browser",- "version": "3.2.2",+ "version": "3.3.0","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -54,10 +54,10 @@},"dependencies": {"@apify/timeout": "^0.3.0",- "@crawlee/basic": "^3.2.2",- "@crawlee/browser-pool": "^3.2.2",- "@crawlee/types": "^3.2.2",- "@crawlee/utils": "^3.2.2",+ "@crawlee/basic": "^3.3.0",+ "@crawlee/browser-pool": "^3.3.0",+ "@crawlee/types": "^3.3.0",+ "@crawlee/utils": "^3.3.0","ow": "^0.28.1","tslib": "^2.4.0"}diff --git a/packages/browser-pool/CHANGELOG.md b/packages/browser-pool/CHANGELOG.mdindex 6c902f19b..07103a8a4 100644--- a/packages/browser-pool/CHANGELOG.md+++ b/packages/browser-pool/CHANGELOG.md@@ -3,6 +3,14 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)++**Note:** Version bump only for package @crawlee/browser-pool+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)**Note:** Version bump only for package @crawlee/browser-pooldiff --git a/packages/browser-pool/package.json b/packages/browser-pool/package.jsonindex 0622045e7..f4d370841 100644--- a/packages/browser-pool/package.json+++ b/packages/browser-pool/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/browser-pool",- "version": "3.2.2",+ "version": "3.3.0","description": "Rotate multiple browsers using popular automation libraries such as Playwright or Puppeteer.","engines": {"node": ">=16.0.0"@@ -38,7 +38,7 @@"dependencies": {"@apify/log": "^2.0.0","@apify/timeout": "^0.3.0",- "@crawlee/types": "^3.2.2",+ "@crawlee/types": "^3.3.0","fingerprint-generator": "^2.0.6","fingerprint-injector": "^2.0.5","lodash.merge": "^4.6.2",diff --git a/packages/cheerio-crawler/CHANGELOG.md b/packages/cheerio-crawler/CHANGELOG.mdindex 836b95947..14b26e257 100644--- a/packages/cheerio-crawler/CHANGELOG.md+++ b/packages/cheerio-crawler/CHANGELOG.md@@ -3,6 +3,18 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)+++### Bug Fixes++* **CheerioCrawler:** pass ixXml down to response parser ([#1807](https://github.com/apify/crawlee/issues/1807)) ([af7a5c4](https://github.com/apify/crawlee/commit/af7a5c4efa94a53e5bdfeca340a9d7223d7dfda4)), closes [#1794](https://github.com/apify/crawlee/issues/1794)+* ignore invalid URLs in `enqueueLinks` in browser crawlers ([#1803](https://github.com/apify/crawlee/issues/1803)) ([5ac336c](https://github.com/apify/crawlee/commit/5ac336c5b83b212fd6281659b8ceee091e259ff1))+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)**Note:** Version bump only for package @crawlee/cheeriodiff --git a/packages/cheerio-crawler/package.json b/packages/cheerio-crawler/package.jsonindex 9dcf3da7d..a82beebb4 100644--- a/packages/cheerio-crawler/package.json+++ b/packages/cheerio-crawler/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/cheerio",- "version": "3.2.2",+ "version": "3.3.0","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -53,8 +53,8 @@"access": "public"},"dependencies": {- "@crawlee/http": "^3.2.2",- "@crawlee/types": "^3.2.2",+ "@crawlee/http": "^3.3.0",+ "@crawlee/types": "^3.3.0","cheerio": "1.0.0-rc.12","htmlparser2": "^8.0.1","tslib": "^2.4.0"diff --git a/packages/cli/CHANGELOG.md b/packages/cli/CHANGELOG.mdindex 60f426216..542e32758 100644--- a/packages/cli/CHANGELOG.md+++ b/packages/cli/CHANGELOG.md@@ -3,6 +3,14 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)++**Note:** Version bump only for package @crawlee/cli+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)**Note:** Version bump only for package @crawlee/clidiff --git a/packages/cli/package.json b/packages/cli/package.jsonindex 483823a70..b61173f0e 100644--- a/packages/cli/package.json+++ b/packages/cli/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/cli",- "version": "3.2.2",+ "version": "3.3.0","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -51,7 +51,7 @@"access": "public"},"dependencies": {- "@crawlee/templates": "^3.2.2",+ "@crawlee/templates": "^3.3.0","ansi-colors": "^4.1.3","fs-extra": "^11.0.0","inquirer": "^8.2.4",diff --git a/packages/core/CHANGELOG.md b/packages/core/CHANGELOG.mdindex b86addc04..2e1f8ef18 100644--- a/packages/core/CHANGELOG.md+++ b/packages/core/CHANGELOG.md@@ -3,6 +3,22 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)+++### Bug Fixes++* ignore invalid URLs in `enqueueLinks` in browser crawlers ([#1803](https://github.com/apify/crawlee/issues/1803)) ([5ac336c](https://github.com/apify/crawlee/commit/5ac336c5b83b212fd6281659b8ceee091e259ff1))+++### Features++* **core:** add `exclude` option to `enqueueLinks` ([#1786](https://github.com/apify/crawlee/issues/1786)) ([2e833dc](https://github.com/apify/crawlee/commit/2e833dc4b0b82bb6741aa683f3fcba05244427df)), closes [#1785](https://github.com/apify/crawlee/issues/1785)+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)**Note:** Version bump only for package @crawlee/corediff --git a/packages/core/package.json b/packages/core/package.jsonindex c1efc7350..2b70a956f 100644--- a/packages/core/package.json+++ b/packages/core/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/core",- "version": "3.2.2",+ "version": "3.3.0","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -59,9 +59,9 @@"@apify/pseudo_url": "^2.0.0","@apify/timeout": "^0.3.0","@apify/utilities": "^2.0.0",- "@crawlee/memory-storage": "^3.2.2",- "@crawlee/types": "^3.2.2",- "@crawlee/utils": "^3.2.2",+ "@crawlee/memory-storage": "^3.3.0",+ "@crawlee/types": "^3.3.0",+ "@crawlee/utils": "^3.3.0","@types/tough-cookie": "^4.0.2","@vladfrangu/async_event_emitter": "^2.0.0","csv-stringify": "^6.2.0",diff --git a/packages/crawlee/CHANGELOG.md b/packages/crawlee/CHANGELOG.mdindex 54fec3458..8246c5ad6 100644--- a/packages/crawlee/CHANGELOG.md+++ b/packages/crawlee/CHANGELOG.md@@ -3,6 +3,14 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)++**Note:** Version bump only for package crawlee+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)**Note:** Version bump only for package crawleediff --git a/packages/crawlee/package.json b/packages/crawlee/package.jsonindex 952157b9b..434bc25c4 100644--- a/packages/crawlee/package.json+++ b/packages/crawlee/package.json@@ -1,6 +1,6 @@{"name": "crawlee",- "version": "3.2.2",+ "version": "3.3.0","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -54,16 +54,16 @@"access": "public"},"dependencies": {- "@crawlee/basic": "^3.2.2",- "@crawlee/browser": "^3.2.2",- "@crawlee/cheerio": "^3.2.2",- "@crawlee/cli": "^3.2.2",- "@crawlee/core": "^3.2.2",- "@crawlee/http": "^3.2.2",- "@crawlee/jsdom": "^3.2.2",- "@crawlee/playwright": "^3.2.2",- "@crawlee/puppeteer": "^3.2.2",- "@crawlee/utils": "^3.2.2",+ "@crawlee/basic": "^3.3.0",+ "@crawlee/browser": "^3.3.0",+ "@crawlee/cheerio": "^3.3.0",+ "@crawlee/cli": "^3.3.0",+ "@crawlee/core": "^3.3.0",+ "@crawlee/http": "^3.3.0",+ "@crawlee/jsdom": "^3.3.0",+ "@crawlee/playwright": "^3.3.0",+ "@crawlee/puppeteer": "^3.3.0",+ "@crawlee/utils": "^3.3.0","import-local": "^3.1.0","tslib": "^2.4.0"},diff --git a/packages/http-crawler/CHANGELOG.md b/packages/http-crawler/CHANGELOG.mdindex 69e9b4a15..821357a10 100644--- a/packages/http-crawler/CHANGELOG.md+++ b/packages/http-crawler/CHANGELOG.md@@ -3,6 +3,14 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)++**Note:** Version bump only for package @crawlee/http+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)**Note:** Version bump only for package @crawlee/httpdiff --git a/packages/http-crawler/package.json b/packages/http-crawler/package.jsonindex a84174c7f..67fae88bf 100644--- a/packages/http-crawler/package.json+++ b/packages/http-crawler/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/http",- "version": "3.2.2",+ "version": "3.3.0","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -55,8 +55,8 @@"dependencies": {"@apify/timeout": "^0.3.0","@apify/utilities": "^2.0.0",- "@crawlee/basic": "^3.2.2",- "@crawlee/types": "^3.2.2",+ "@crawlee/basic": "^3.3.0",+ "@crawlee/types": "^3.3.0","@types/content-type": "^1.1.5","content-type": "^1.0.4","got-scraping": "^3.2.9",diff --git a/packages/jsdom-crawler/CHANGELOG.md b/packages/jsdom-crawler/CHANGELOG.mdindex 02a75a49d..8a10c8135 100644--- a/packages/jsdom-crawler/CHANGELOG.md+++ b/packages/jsdom-crawler/CHANGELOG.md@@ -3,6 +3,17 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)+++### Bug Fixes++* ignore invalid URLs in `enqueueLinks` in browser crawlers ([#1803](https://github.com/apify/crawlee/issues/1803)) ([5ac336c](https://github.com/apify/crawlee/commit/5ac336c5b83b212fd6281659b8ceee091e259ff1))+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)**Note:** Version bump only for package @crawlee/jsdomdiff --git a/packages/jsdom-crawler/package.json b/packages/jsdom-crawler/package.jsonindex 8c1ceb8bf..1e419769a 100644--- a/packages/jsdom-crawler/package.json+++ b/packages/jsdom-crawler/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/jsdom",- "version": "3.2.2",+ "version": "3.3.0","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -54,8 +54,8 @@},"dependencies": {"@apify/utilities": "^2.1.4",- "@crawlee/http": "^3.2.2",- "@crawlee/types": "^3.2.2",+ "@crawlee/http": "^3.3.0",+ "@crawlee/types": "^3.3.0","@types/jsdom": "^21.0.0","jsdom": "^21.0.0","ow": "^0.28.2",diff --git a/packages/memory-storage/CHANGELOG.md b/packages/memory-storage/CHANGELOG.mdindex fb0f71bb9..0f7efad4b 100644--- a/packages/memory-storage/CHANGELOG.md+++ b/packages/memory-storage/CHANGELOG.md@@ -3,6 +3,25 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)+++### Bug Fixes++* **MemoryStorage:** request queues race conditions causing crashes ([#1806](https://github.com/apify/crawlee/issues/1806)) ([083a9db](https://github.com/apify/crawlee/commit/083a9db9ebcddd3fa886631234c790d4c5bcdf86)), closes [#1792](https://github.com/apify/crawlee/issues/1792)+* **MemoryStorage:** RequestQueue should respect `forefront` ([#1816](https://github.com/apify/crawlee/issues/1816)) ([b68e86a](https://github.com/apify/crawlee/commit/b68e86a97954bcbe30fde802fed5f263016fffe2)), closes [#1787](https://github.com/apify/crawlee/issues/1787)+* **MemoryStorage:** RequestQueue#handledRequestCount should update ([#1817](https://github.com/apify/crawlee/issues/1817)) ([a775e4a](https://github.com/apify/crawlee/commit/a775e4afea20d0b31492f44b90f61b6a903491b6)), closes [#1764](https://github.com/apify/crawlee/issues/1764)+++### Features++* add basic support for `setStatusMessage` ([#1790](https://github.com/apify/crawlee/issues/1790)) ([c318980](https://github.com/apify/crawlee/commit/c318980ec11d211b1a5c9e6bdbe76198c5d895be))+* move the status message implementation to Crawlee, noop in storage ([#1808](https://github.com/apify/crawlee/issues/1808)) ([99c3fdc](https://github.com/apify/crawlee/commit/99c3fdc18030b7898e6b6d149d6d94fab7881f09))+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)diff --git a/packages/memory-storage/package.json b/packages/memory-storage/package.jsonindex 81c80a6dc..34c65e592 100644--- a/packages/memory-storage/package.json+++ b/packages/memory-storage/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/memory-storage",- "version": "3.2.2",+ "version": "3.3.0","description": "A simple in-memory storage implementation of the Apify API","engines": {"node": ">= 16"@@ -49,7 +49,7 @@},"dependencies": {"@apify/log": "^2.0.0",- "@crawlee/types": "^3.2.2",+ "@crawlee/types": "^3.3.0","@sapphire/async-queue": "^1.5.0","@sapphire/shapeshift": "^3.0.0","content-type": "^1.0.4",diff --git a/packages/playwright-crawler/CHANGELOG.md b/packages/playwright-crawler/CHANGELOG.mdindex b421fbd86..b7e8c3675 100644--- a/packages/playwright-crawler/CHANGELOG.md+++ b/packages/playwright-crawler/CHANGELOG.md@@ -3,6 +3,14 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)++**Note:** Version bump only for package @crawlee/playwright+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)**Note:** Version bump only for package @crawlee/playwrightdiff --git a/packages/playwright-crawler/package.json b/packages/playwright-crawler/package.jsonindex 082df9b79..8a4e8d132 100644--- a/packages/playwright-crawler/package.json+++ b/packages/playwright-crawler/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/playwright",- "version": "3.2.2",+ "version": "3.3.0","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -55,10 +55,10 @@"dependencies": {"@apify/datastructures": "^2.0.0","@apify/log": "^2.0.0",- "@crawlee/browser": "^3.2.2",- "@crawlee/browser-pool": "^3.2.2",- "@crawlee/types": "^3.2.2",- "@crawlee/utils": "^3.2.2",+ "@crawlee/browser": "^3.3.0",+ "@crawlee/browser-pool": "^3.3.0",+ "@crawlee/types": "^3.3.0",+ "@crawlee/utils": "^3.3.0","cheerio": "1.0.0-rc.12","jquery": "^3.6.0","ow": "^0.28.1",diff --git a/packages/puppeteer-crawler/CHANGELOG.md b/packages/puppeteer-crawler/CHANGELOG.mdindex 126c9216c..dbc9611e4 100644--- a/packages/puppeteer-crawler/CHANGELOG.md+++ b/packages/puppeteer-crawler/CHANGELOG.md@@ -3,6 +3,14 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)++**Note:** Version bump only for package @crawlee/puppeteer+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)**Note:** Version bump only for package @crawlee/puppeteerdiff --git a/packages/puppeteer-crawler/package.json b/packages/puppeteer-crawler/package.jsonindex cf9640014..d6b622887 100644--- a/packages/puppeteer-crawler/package.json+++ b/packages/puppeteer-crawler/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/puppeteer",- "version": "3.2.2",+ "version": "3.3.0","description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=16.0.0"@@ -55,10 +55,10 @@"dependencies": {"@apify/datastructures": "^2.0.0","@apify/log": "^2.0.0",- "@crawlee/browser": "^3.2.2",- "@crawlee/browser-pool": "^3.2.2",- "@crawlee/types": "^3.2.2",- "@crawlee/utils": "^3.2.2",+ "@crawlee/browser": "^3.3.0",+ "@crawlee/browser-pool": "^3.3.0",+ "@crawlee/types": "^3.3.0",+ "@crawlee/utils": "^3.3.0","cheerio": "1.0.0-rc.12","devtools-protocol": "*","jquery": "^3.6.0",diff --git a/packages/templates/CHANGELOG.md b/packages/templates/CHANGELOG.mdindex bea08a6ab..7056ee3cd 100644--- a/packages/templates/CHANGELOG.md+++ b/packages/templates/CHANGELOG.md@@ -3,6 +3,14 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)++**Note:** Version bump only for package @crawlee/templates+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)**Note:** Version bump only for package @crawlee/templatesdiff --git a/packages/templates/package.json b/packages/templates/package.jsonindex d89774786..45e9781d4 100644--- a/packages/templates/package.json+++ b/packages/templates/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/templates",- "version": "3.2.2",+ "version": "3.3.0","description": "Templates for the crawlee projects","engines": {"node": ">=16.0.0"diff --git a/packages/types/CHANGELOG.md b/packages/types/CHANGELOG.mdindex 8f0eedc57..ab2713bda 100644--- a/packages/types/CHANGELOG.md+++ b/packages/types/CHANGELOG.md@@ -3,6 +3,23 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)+++### Bug Fixes++* **MemoryStorage:** RequestQueue#handledRequestCount should update ([#1817](https://github.com/apify/crawlee/issues/1817)) ([a775e4a](https://github.com/apify/crawlee/commit/a775e4afea20d0b31492f44b90f61b6a903491b6)), closes [#1764](https://github.com/apify/crawlee/issues/1764)+++### Features++* add basic support for `setStatusMessage` ([#1790](https://github.com/apify/crawlee/issues/1790)) ([c318980](https://github.com/apify/crawlee/commit/c318980ec11d211b1a5c9e6bdbe76198c5d895be))+* move the status message implementation to Crawlee, noop in storage ([#1808](https://github.com/apify/crawlee/issues/1808)) ([99c3fdc](https://github.com/apify/crawlee/commit/99c3fdc18030b7898e6b6d149d6d94fab7881f09))+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)**Note:** Version bump only for package @crawlee/typesdiff --git a/packages/types/package.json b/packages/types/package.jsonindex bde491121..8e61df35a 100644--- a/packages/types/package.json+++ b/packages/types/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/types",- "version": "3.2.2",+ "version": "3.3.0","description": "Shared types for the crawlee projects","engines": {"node": ">=16.0.0"diff --git a/packages/utils/CHANGELOG.md b/packages/utils/CHANGELOG.mdindex a1dcbd973..e889cf530 100644--- a/packages/utils/CHANGELOG.md+++ b/packages/utils/CHANGELOG.md@@ -3,6 +3,17 @@All notable changes to this project will be documented in this file.See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.+# [3.3.0](https://github.com/apify/crawlee/compare/v3.2.2...v3.3.0) (2023-03-09)+++### Bug Fixes++* add `proxyUrl` to `DownloadListOfUrlsOptions` ([779be1e](https://github.com/apify/crawlee/commit/779be1e4f29dff191d02e623eefb1bd5650c14ad)), closes [#1780](https://github.com/apify/crawlee/issues/1780)+++++## [3.2.2](https://github.com/apify/crawlee/compare/v3.2.1...v3.2.2) (2023-02-08)**Note:** Version bump only for package @crawlee/utilsdiff --git a/packages/utils/package.json b/packages/utils/package.jsonindex a2953fbdd..651af4fb7 100644--- a/packages/utils/package.json+++ b/packages/utils/package.json@@ -1,6 +1,6 @@{"name": "@crawlee/utils",- "version": "3.2.2",+ "version": "3.3.0","description": "A set of shared utilities that can be used by crawlers","engines": {"node": ">=16.0.0"@@ -49,7 +49,7 @@"dependencies": {"@apify/log": "^2.0.0","@apify/ps-tree": "^1.1.4",- "@crawlee/types": "^3.2.2",+ "@crawlee/types": "^3.3.0","cheerio": "1.0.0-rc.12","got-scraping": "^3.2.9","ow": "^0.28.1",