Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

342dd44
feat: Implement storage aliases (#3636)
Jan Buchar committed 5 months ago
7edc793
49e417d
412ad09
chore: migrate v4 to pnpm 11 (#3645)
Martin Adámek committed 6 months ago
f673ca5
63ef023
1a8de68
1cf1060
9ffa53c
refactor!: remove `tieredProxyUrls` support (#3599)
Jindřich Bär committed 6 months ago
99b751f
chore(ci): restore Docker image dispatch, gated to master
Martin Adámek committed 6 months ago
ac41130
chore(ci): bake --yes into publish scripts
Martin Adámek committed 6 months ago
e4afb37
feat: redesign `Configuration` class for v4 (#3484)
Martin Adámek committed 6 months ago
7b60610
03e343a
b8cea5a
9047dc2
6cbd3ac