Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
chore: cache Playwright browsers and add install timeout in CI (#3691)
Martin Adámekb4ee1dec9128
diff --git a/.github/workflows/release.yml b/.github/workflows/release.ymlindex 8836f2b59..73e0e60c6 100644--- a/.github/workflows/release.yml+++ b/.github/workflows/release.yml@@ -71,9 +71,19 @@ jobs:turbo-${{ github.job }}-${{ github.ref_name }}-- name: Install Dependencies- run: |- yarn- yarn playwright install --with-deps+ run: yarn++ - name: Cache Playwright browsers+ uses: actions/cache@v5+ with:+ path: ~/.cache/ms-playwright+ key: playwright-${{ runner.os }}-${{ hashFiles('**/yarn.lock') }}+ restore-keys: |+ playwright-${{ runner.os }}-++ - name: Install Playwright browsers+ timeout-minutes: 10+ run: yarn playwright install --with-deps- name: Buildrun: yarn ci:builddiff --git a/.github/workflows/test-ci.yml b/.github/workflows/test-ci.ymlindex 9f1d25b53..f3cbdda4f 100644--- a/.github/workflows/test-ci.yml+++ b/.github/workflows/test-ci.yml@@ -59,9 +59,21 @@ jobs:turbo-${{ github.job }}-${{ matrix.node-version }}-${{ github.ref_name }}-- name: Install Dependencies- run: |- yarn- yarn playwright install --with-deps+ run: yarn+ env:+ YARN_IGNORE_NODE: 1++ - name: Cache Playwright browsers+ uses: actions/cache@v5+ with:+ path: ~/.cache/ms-playwright+ key: playwright-${{ runner.os }}-${{ hashFiles('**/yarn.lock') }}+ restore-keys: |+ playwright-${{ runner.os }}-++ - name: Install Playwright browsers+ timeout-minutes: 10+ run: yarn playwright install --with-depsenv:YARN_IGNORE_NODE: 1diff --git a/.github/workflows/test-e2e.yml b/.github/workflows/test-e2e.ymlindex 67f1ef82b..9eb97e18a 100644--- a/.github/workflows/test-e2e.yml+++ b/.github/workflows/test-e2e.yml@@ -66,8 +66,18 @@ jobs:- name: Install Dependenciesrun: yarn+ - name: Cache Playwright browsers+ if: (matrix.storage != 'PLATFORM')+ uses: actions/cache@v5+ with:+ path: ~/.cache/ms-playwright+ key: playwright-${{ runner.os }}-${{ hashFiles('**/yarn.lock') }}+ restore-keys: |+ playwright-${{ runner.os }}-+- name: Install Playwright Dependenciesif: (matrix.storage != 'PLATFORM')+ timeout-minutes: 10run: yarn playwright install --with-deps- name: Build