Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

chore(deps): update dependency puppeteer to v22.8.2
renovate[bot] committed 2 years ago
b9bc039
test: fix e2e tests for zero concurrency
Martin Adámek committed 2 years ago
663595d
perf: optimize `RequestList` memory footprint (#2466)
Martin Adámek committed 2 years ago
12210bd
docs: scrapy-vs-crawlee blog (#2431)
Saurav Jain committed 2 years ago
38c0942
fix: double tier decrement in tiered proxy (#2468)
Jindřich Bär committed 2 years ago
3a8204b
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
8071d34
chore(deps): update patch/minor dependencies (#2450)
renovate[bot] committed 2 years ago
41ee5ef
perf: improve scaling based on memory (#2459)
Martin Adámek committed 2 years ago
2d5d443
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
3a847f6
420fbc7
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
3ff1700
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
0a8a205
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
38a202f
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
8c24617
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
1d97de4
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
53f4a0f
1fa9a66
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
a41e2ac
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
88c863a
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
78eb4cc
63b9949
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
9caa6aa