Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

17.4 KB
Raw Blame

Change Log

All notable changes to this project will be documented in this file. See Conventional Commits for commit guidelines.

3.18.1 (2026-08-12)

Note: Version bump only for package @crawlee/http

3.18.0 (2026-08-04)

Bug Fixes

Features

3.17.0 (2026-06-04)

Features

  • add abortDownload context helper to FileDownload (#3528) (3b3d8f6)

3.16.0 (2026-02-06)

Performance Improvements

3.15.3 (2025-11-10)

Note: Version bump only for package @crawlee/http

3.15.2 (2025-10-23)

Note: Version bump only for package @crawlee/http

3.15.1 (2025-09-26)

Note: Version bump only for package @crawlee/http

3.15.0 (2025-09-17)

Note: Version bump only for package @crawlee/http

3.14.1 (2025-08-05)

Note: Version bump only for package @crawlee/http

3.14.0 (2025-07-25)

Bug Fixes

3.13.10 (2025-07-09)

Note: Version bump only for package @crawlee/http

3.13.9 (2025-06-27)

Features

  • Report links skipped because of various filter conditions (#3026) (5a867bc), closes #3016

3.13.8 (2025-06-16)

Note: Version bump only for package @crawlee/http

3.13.7 (2025-06-06)

Note: Version bump only for package @crawlee/http

3.13.6 (2025-06-05)

Bug Fixes

  • enable full cookie support for ImpitHttpClient (#2991) (120f0a7)

3.13.5 (2025-05-20)

Features

  • add MinimumSpeedStream and ByteCounterStream helpers (#2970) (921c4ee)

3.13.4 (2025-05-14)

Note: Version bump only for package @crawlee/http

3.13.3 (2025-05-05)

Features

  • pass response to FileDownload.streamHandler (#2930) (008c4c7)

3.13.2 (2025-04-08)

Note: Version bump only for package @crawlee/http

3.13.1 (2025-04-07)

Bug Fixes

Features

3.13.0 (2025-03-04)

Note: Version bump only for package @crawlee/http

3.12.2 (2025-01-27)

Note: Version bump only for package @crawlee/http

3.12.1 (2024-12-04)

Note: Version bump only for package @crawlee/http

3.12.0 (2024-11-04)

Features

3.11.5 (2024-10-04)

Bug Fixes

  • http-crawler: avoid crashing when gotOptions.cache is on (#2686) (1106d3a)

3.11.4 (2024-09-23)

Note: Version bump only for package @crawlee/http

3.11.3 (2024-09-03)

Note: Version bump only for package @crawlee/http

3.11.2 (2024-08-28)

Note: Version bump only for package @crawlee/http

3.11.1 (2024-07-24)

Note: Version bump only for package @crawlee/http

3.11.0 (2024-07-09)

Note: Version bump only for package @crawlee/http

3.10.5 (2024-06-12)

Bug Fixes

  • mark context.request.loadedUrl and id as required inside the request handler (#2531) (2b54660)

3.10.4 (2024-06-11)

Bug Fixes

3.10.3 (2024-06-07)

Features

  • add waitForSelector context helper + parseWithCheerio in adaptive crawler (#2522) (6f88e73)

3.10.2 (2024-06-03)

Note: Version bump only for package @crawlee/http

3.10.1 (2024-05-23)

Note: Version bump only for package @crawlee/http

3.10.0 (2024-05-16)

Features

3.9.2 (2024-04-17)

Note: Version bump only for package @crawlee/http

3.9.1 (2024-04-11)

Note: Version bump only for package @crawlee/http

3.9.0 (2024-04-10)

Features

3.8.2 (2024-03-21)

Note: Version bump only for package @crawlee/http

3.8.1 (2024-02-22)

Note: Version bump only for package @crawlee/http

3.8.0 (2024-02-21)

Note: Version bump only for package @crawlee/http

3.7.3 (2024-01-30)

Note: Version bump only for package @crawlee/http

3.7.2 (2024-01-09)

Note: Version bump only for package @crawlee/http

3.7.1 (2024-01-02)

Note: Version bump only for package @crawlee/http

3.7.0 (2023-12-21)

Features

3.6.2 (2023-11-26)

Note: Version bump only for package @crawlee/http

3.6.1 (2023-11-15)

Note: Version bump only for package @crawlee/http

3.6.0 (2023-11-15)

Bug Fixes

  • retry incorrect Content-Type when response has blocked status code (#2176) (b54fb8b), closes #1994

Features

3.5.8 (2023-10-17)

Note: Version bump only for package @crawlee/http

3.5.7 (2023-10-05)

Note: Version bump only for package @crawlee/http

3.5.6 (2023-10-04)

Note: Version bump only for package @crawlee/http

3.5.5 (2023-10-02)

Note: Version bump only for package @crawlee/http

3.5.4 (2023-09-11)

Note: Version bump only for package @crawlee/http

3.5.3 (2023-08-31)

Bug Fixes

3.5.2 (2023-08-21)

Bug Fixes

Features

3.5.1 (2023-08-16)

Bug Fixes

  • log original error message on session rotation (#2022) (8a11ffb)

3.5.0 (2023-07-31)

Features

3.4.2 (2023-07-19)

Features

3.4.1 (2023-07-13)

Bug Fixes

  • http-crawler: replace IncomingMessage with PlainResponse for context's response (#1973) (2a1cc7f), closes #1964

3.4.0 (2023-06-12)

Note: Version bump only for package @crawlee/http

3.3.3 (2023-05-31)

Note: Version bump only for package @crawlee/http

3.3.2 (2023-05-11)

Features

  • HttpCrawler: add parseWithCheerio helper to HttpCrawler (#1906) (ff5f76f)
  • router: allow inline router definition (#1877) (2d241c9)

3.3.1 (2023-04-11)

Bug Fixes

  • jsdom: use no-op enqueueLinks in http crawlers when parsing fails (fd35270)

3.3.0 (2023-03-09)

Note: Version bump only for package @crawlee/http

3.2.2 (2023-02-08)

Note: Version bump only for package @crawlee/http

3.2.1 (2023-02-07)

Note: Version bump only for package @crawlee/http

3.2.0 (2023-02-07)

Bug Fixes

  • declare missing dependency on tslib (27e96c8), closes #1747

3.1.4 (2022-12-14)

Note: Version bump only for package @crawlee/http

3.1.3 (2022-12-07)

Note: Version bump only for package @crawlee/http

3.1.2 (2022-11-15)

Note: Version bump only for package @crawlee/http

3.1.1 (2022-11-07)

Note: Version bump only for package @crawlee/http

3.1.0 (2022-10-13)

Note: Version bump only for package @crawlee/http

3.0.4 (2022-08-22)

Note: Version bump only for package @crawlee/http