Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Updated docs
Jan Curn committed 8 years ago
b68057f
Updated docs
Jan Curn committed 8 years ago
d970d8e
Updated texts in docs and website
Jan Curn committed 8 years ago
812287f
Updated docs
Jan Curn committed 8 years ago
1c83689
Updated changelog
Jan Curn committed 8 years ago
2022377
Fix docs
Ondra Urban committed 8 years ago
90116f0
Fix merge conflicts
Ondra Urban committed 8 years ago
7338a4e
Fix CheerioCrawler proxy bug
Ondra Urban committed 8 years ago
6510a8b
Fixed Puppeteer crawler example
Jan Curn committed 8 years ago
e53e5e2
Updated Puppeteer crawler example
Jan Curn committed 8 years ago
5806ead
5b703a0
Minor change in logging messages
Jan Curn committed 8 years ago
a0847d6
Bump version, update Changelog
Ondra Urban committed 8 years ago
a502014
Use error.stack
Ondra Urban committed 8 years ago
891670e
Implement PR comments
Ondra Urban committed 8 years ago
2dc02b8
Add omitStack option
Ondra Urban committed 8 years ago
144e524
Simplify type inspection and logging
Ondra Urban committed 8 years ago
b571c79
Fix docs
Ondra Urban committed 8 years ago
81d36b2
Add more typechecks
Ondra Urban committed 8 years ago
c93237d
Build docs
Ondra Urban committed 8 years ago
2811c90
Improve messages in request.pushErrorMessage
Ondra Urban committed 8 years ago
01667a8
Merge branch 'develop' into feature/actor-ui-inputs
Ondra Urban committed 8 years ago
0a74c31
Build docs
Ondra Urban committed 8 years ago
637da07
Add missing param to Docs
Ondra Urban committed 8 years ago
8948ea1
4876ed0
Merge branch 'develop' into feature/actor-ui-inputs
Ondra Urban committed 8 years ago
c5139ba
Remove unnecessary backup
Ondra Urban committed 8 years ago
5779f66