Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

ea9da4c
fix: Fix purgeOnStart on aliased storages (#4013)
Jan Buchar committed 3 months ago
4dbce0a
bb2f267
chore: convert const enums to regular enums (#4008)
Vlad Frangu committed 3 months ago
400fadf
perf(http-client): lazy-load tough-cookie (#4007)
Jindřich Bär committed 3 months ago
ea5b8c2
perf(utils): lazily import sax (#4001)
Jindřich Bär committed 3 months ago
e09444c
perf: lazy-load `file-type` and `csv-stringify` (#3989)
Jindřich Bär committed 3 months ago
60c4b55
dd7748d
c596d23
feat!: Transactional storage access (#3953)
Jan Buchar committed 3 months ago
bb6fc1d
28e24b4
a99f894
refactor: drop `fs-extra` dependency (#3969)
Jindřich Bär committed 3 months ago
7373844
docs: Reorganize the v4 upgrading guide (#3967)
Jan Buchar committed 3 months ago
1e482e4
chore: Add missing upgrading guide entries (#3965)
Jan Buchar committed 3 months ago
7c90ca0
feat: simplify `enqueueLinks` interface (#3533)
Richard Solar committed 3 months ago
3adf3ae
d53ab9b