Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

18.3 KB
Raw Blame

Change Log

All notable changes to this project will be documented in this file. See Conventional Commits for commit guidelines.

3.18.1 (2026-08-12)

Note: Version bump only for package @crawlee/browser

3.18.0 (2026-08-04)

Bug Fixes

  • core: ignore explicitly undefined baseUrl in enqueueLinks helpers (#3924) (a58a9c1)
  • core: keep crawler enqueue limits when options carry explicit undefined (#3927) (5f1b0ab), closes #3924 #3924

Features

3.17.0 (2026-06-04)

Bug Fixes

  • BrowserCrawler closes ctx.page before errorHandler runs on navig… (#3655) (ab34886)

Features

  • add ignoreProxyCertificate option for the internal proxy-chain instance (#3418) (02eec66), closes #3369

3.16.0 (2026-02-06)

Performance Improvements

3.15.3 (2025-11-10)

Note: Version bump only for package @crawlee/browser

3.15.2 (2025-10-23)

Note: Version bump only for package @crawlee/browser

3.15.1 (2025-09-26)

Note: Version bump only for package @crawlee/browser

3.15.0 (2025-09-17)

Note: Version bump only for package @crawlee/browser

3.14.1 (2025-08-05)

Note: Version bump only for package @crawlee/browser

3.14.0 (2025-07-25)

Features

3.13.10 (2025-07-09)

Note: Version bump only for package @crawlee/browser

3.13.9 (2025-06-27)

Features

  • Report links skipped because of various filter conditions (#3026) (5a867bc), closes #3016

3.13.8 (2025-06-16)

Bug Fixes

  • Do not enqueue more links than what the crawler is capable of processing (#2990) (ea094c8), closes #2728

3.13.7 (2025-06-06)

Note: Version bump only for package @crawlee/browser

3.13.6 (2025-06-05)

Note: Version bump only for package @crawlee/browser

3.13.5 (2025-05-20)

Note: Version bump only for package @crawlee/browser

3.13.4 (2025-05-14)

Note: Version bump only for package @crawlee/browser

3.13.3 (2025-05-05)

Note: Version bump only for package @crawlee/browser

3.13.2 (2025-04-08)

Features

3.13.1 (2025-04-07)

Bug Fixes

Features

3.13.0 (2025-03-04)

Note: Version bump only for package @crawlee/browser

3.12.2 (2025-01-27)

Note: Version bump only for package @crawlee/browser

3.12.1 (2024-12-04)

Note: Version bump only for package @crawlee/browser

3.12.0 (2024-11-04)

Note: Version bump only for package @crawlee/browser

3.11.5 (2024-10-04)

Bug Fixes

  • puppeteer: rename ignoreHTTPSErrors to acceptInsecureCerts to support v23 (#2684) (f3927e6)

3.11.4 (2024-09-23)

Note: Version bump only for package @crawlee/browser

3.11.3 (2024-09-03)

Note: Version bump only for package @crawlee/browser

3.11.2 (2024-08-28)

Note: Version bump only for package @crawlee/browser

3.11.1 (2024-07-24)

Note: Version bump only for package @crawlee/browser

3.11.0 (2024-07-09)

Features

  • add iframe expansion to parseWithCheerio in browsers (#2542) (328d085), closes #2507
  • add ignoreIframes opt-out from the Cheerio iframe expansion (#2562) (474a8dc)

3.10.5 (2024-06-12)

Bug Fixes

  • declare missing peer dependencies in @crawlee/browser package (#2532) (3357c7f)
  • mark context.request.loadedUrl and id as required inside the request handler (#2531) (2b54660)

3.10.4 (2024-06-11)

Note: Version bump only for package @crawlee/browser

3.10.3 (2024-06-07)

Note: Version bump only for package @crawlee/browser

3.10.2 (2024-06-03)

Note: Version bump only for package @crawlee/browser

3.10.1 (2024-05-23)

Note: Version bump only for package @crawlee/browser

3.10.0 (2024-05-16)

Note: Version bump only for package @crawlee/browser

3.9.2 (2024-04-17)

Note: Version bump only for package @crawlee/browser

3.9.1 (2024-04-11)

Features

3.9.0 (2024-04-10)

Features

3.8.2 (2024-03-21)

Note: Version bump only for package @crawlee/browser

3.8.1 (2024-02-22)

Note: Version bump only for package @crawlee/browser

3.8.0 (2024-02-21)

Features

3.7.3 (2024-01-30)

Note: Version bump only for package @crawlee/browser

3.7.2 (2024-01-09)

Note: Version bump only for package @crawlee/browser

3.7.1 (2024-01-02)

Note: Version bump only for package @crawlee/browser

3.7.0 (2023-12-21)

Bug Fixes

  • retryOnBlocked doesn't override the blocked HTTP codes (#2243) (81672c3)

Features

3.6.2 (2023-11-26)

Note: Version bump only for package @crawlee/browser

3.6.1 (2023-11-15)

Features

3.6.0 (2023-11-15)

Note: Version bump only for package @crawlee/browser

3.5.8 (2023-10-17)

Note: Version bump only for package @crawlee/browser

3.5.7 (2023-10-05)

Note: Version bump only for package @crawlee/browser

3.5.6 (2023-10-04)

Note: Version bump only for package @crawlee/browser

3.5.5 (2023-10-02)

Features

3.5.4 (2023-09-11)

Note: Version bump only for package @crawlee/browser

3.5.3 (2023-08-31)

Bug Fixes

3.5.2 (2023-08-21)

Note: Version bump only for package @crawlee/browser

3.5.1 (2023-08-16)

Bug Fixes

  • log original error message on session rotation (#2022) (8a11ffb)

3.5.0 (2023-07-31)

Features

3.4.2 (2023-07-19)

Features

3.4.1 (2023-07-13)

Note: Version bump only for package @crawlee/browser

3.4.0 (2023-06-12)

Bug Fixes

3.3.3 (2023-05-31)

Note: Version bump only for package @crawlee/browser

3.3.2 (2023-05-11)

Note: Version bump only for package @crawlee/browser

3.3.1 (2023-04-11)

Note: Version bump only for package @crawlee/browser

3.3.0 (2023-03-09)

Bug Fixes

  • ignore invalid URLs in enqueueLinks in browser crawlers (#1803) (5ac336c)

3.2.2 (2023-02-08)

Note: Version bump only for package @crawlee/browser

3.2.1 (2023-02-07)

Note: Version bump only for package @crawlee/browser

3.2.0 (2023-02-07)

Bug Fixes

  • declare missing dependency on tslib (27e96c8), closes #1747

3.1.4 (2022-12-14)

Note: Version bump only for package @crawlee/browser

3.1.3 (2022-12-07)

Note: Version bump only for package @crawlee/browser

3.1.2 (2022-11-15)

Note: Version bump only for package @crawlee/browser

3.1.1 (2022-11-07)

Note: Version bump only for package @crawlee/browser

3.1.0 (2022-10-13)

Note: Version bump only for package @crawlee/browser

3.0.4 (2022-08-22)

Features