Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
chore(ci): fix failing e2e and docs actions after rebase (#3874)
Jindřich Bärf745f873caed
diff --git a/.github/workflows/test-ci.yml b/.github/workflows/test-ci.ymlindex ce4e3bb61..05d8b26e4 100644--- a/.github/workflows/test-ci.yml+++ b/.github/workflows/test-ci.yml@@ -139,7 +139,7 @@ jobs:- name: Start Docusaurus serverrun: |cd website- nohup yarn docusaurus serve --port 3000 --no-open &+ nohup pnpm docusaurus serve --port 3000 --no-open &sleep 5curl -f http://localhost:3000 > /dev/nulldiff --git a/.github/workflows/test-e2e.yml b/.github/workflows/test-e2e.ymlindex 91b39c75a..796c87f2e 100644--- a/.github/workflows/test-e2e.yml+++ b/.github/workflows/test-e2e.yml@@ -10,10 +10,6 @@ concurrency:group: ${{ github.workflow }}-${{ github.ref }}cancel-in-progress: true-concurrency:- group: ${{ github.workflow }}-${{ github.ref }}- cancel-in-progress: true-jobs:build_and_test:name: Build & Test@@ -60,7 +56,7 @@ jobs:uses: actions/cache@v6with:path: ~/.cache/ms-playwright- key: playwright-${{ runner.os }}-${{ hashFiles('**/yarn.lock') }}+ key: playwright-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}restore-keys: |playwright-${{ runner.os }}-diff --git a/website/docusaurus.config.js b/website/docusaurus.config.jsindex 28462ef8e..e1fe1983d 100644--- a/website/docusaurus.config.js+++ b/website/docusaurus.config.js@@ -171,14 +171,6 @@ module.exports = {from: '/js/docs/guides/apify-platform',to: '/js/docs/deployment/apify-platform',},- {- from: '/js/docs/experiments/experiments-system-infomation-v2',- to: '/js/docs/experiments/experiments-system-information-v2',- },- {- from: '/js/docs/next/experiments/experiments-system-infomation-v2',- to: '/js/docs/next/experiments/experiments-system-information-v2',- },{from: '/js/docs/3.13/experiments/experiments-system-infomation-v2',to: '/js/docs/3.13/experiments/experiments-system-information-v2',@@ -193,7 +185,7 @@ module.exports = {},{from: '/js/docs/3.16/experiments/experiments-system-infomation-v2',- to: '/js/docs/experiments/experiments-system-information-v2',+ to: '/js/docs/3.16/experiments/experiments-system-information-v2',},],// createRedirects(existingPath) {diff --git a/website/versioned_docs/version-3.17/upgrading/upgrading_v3.md b/website/versioned_docs/version-3.17/upgrading/upgrading_v3.mdindex 39b6091c9..ab605f9bb 100644--- a/website/versioned_docs/version-3.17/upgrading/upgrading_v3.md+++ b/website/versioned_docs/version-3.17/upgrading/upgrading_v3.md@@ -31,7 +31,7 @@ The [`crawlee`](https://www.npmjs.com/package/crawlee) package consists of sever- [`@crawlee/memory-storage`](https://crawlee.dev/js/api/memory-storage): [`@apify/storage-local`](https://npmjs.com/package/@apify/storage-local) alternative- [`@crawlee/browser-pool`](https://crawlee.dev/js/api/browser-pool): previously [`browser-pool`](https://npmjs.com/package/browser-pool) package- [`@crawlee/utils`](https://crawlee.dev/js/api/utils): utility methods-- [`@crawlee/types`](https://crawlee.dev/js/api/types): holds TS interfaces mainly about the [`StorageClient`](https://crawlee.dev/js/api/core/interface/StorageClient)+- [`@crawlee/types`](https://crawlee.dev/js/api/types): holds TS interfaces mainly about the [`StorageClient`](https://crawlee.dev/js/api/3.17/core/interface/StorageClient)### Installing Crawlee