Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

fix: avoid duplicate final crawler persistence (#3866)
Akhil Trivedi committed 4 months ago
79dd6c4
9dc3160
3e45246
feat: support `puppeteer@25` (#3869)
Martin Adámek committed 4 months ago
1430062
test(e2e): use impit in the robots file test (#3857)
Martin Adámek committed 4 months ago
4d882e6
d6a2b39
ci: fix E2E tests (#3855)
Martin Adámek committed 4 months ago
3e3dc63
2c4eef2
chore(deps): update patch/minor dependencies to v3.10.2
renovate[bot] committed 4 months ago
07f53af
chore(ci): modernize Renovate config (#3829)
Vlada Dusek committed 4 months ago
f04f62f
chore(deps): update apify/actions action to v1.4.0
renovate[bot] committed 4 months ago
cd1d136
fix: declare missing dependencies (#3817)
gullmar committed 4 months ago
5a9a661
chore(deps): update apify/actions action to v1.3.2
renovate[bot] committed 4 months ago
266125d
b7b2db8