Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
c1438b0
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
5f5e0de
docs: improve banner (#2528)
Saurav Jain committed 2 years ago
b5b1867
chore(deps): update yarn to v4.3.0
renovate[bot] committed 2 years ago
2acf9f5
docs: proxy tier logging (#2527)
Jindřich Bär committed 2 years ago
a588527
chore: add empty project templates
Martin Adámek committed 2 years ago
03d876e
fix: make `crawler.log` publicly accessible (#2526)
Martin Adámek committed 2 years ago
3e9e665
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
c1d6ac5
docs: use node 20 in docker examples
Martin Adámek committed 2 years ago
bf6d8ff
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
bfc1fa0
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
8d226ef
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
537624e
chore(release): update internal dependencies [skip ci]
Apify Release Bot committed 2 years ago
8a15f3b
chore(release): v3.10.3
Apify Release Bot committed 2 years ago
c7fc144
chore: fix some outdated defaults in the jsdoc comments
Martin Adámek committed 2 years ago
bf665be
fix: revert the scaling steps back to 5%
Martin Adámek committed 2 years ago
5bf32f8
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
c5521ed
9f0b796
195f176
chore: fix timeouts for `waitForSelector` in playwright
Martin Adámek committed 2 years ago
53ce1d3
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
f814ec5
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
c4d31f7
0976e18
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
f069892
chore: ignore lerna.json when formatting
Martin Adámek committed 2 years ago
5002c2f
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
3361377