Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

73ef60d
Upgraded to [email protected]
Jan Curn committed 8 years ago
b42a8b2
3ac172f
0b75c5a
Change typecheck for userData
Ondra Urban committed 8 years ago
0fc6afb
Fix typo in default selector
Ondra Urban committed 8 years ago
fb54f1d
Add userData to enqueueLinks
Ondra Urban committed 8 years ago
18e136d
Build docs
Ondra Urban committed 8 years ago
7199b55
Update changelog, bump version
Ondra Urban committed 8 years ago
43ae80e
30a0619
73684d0
Merge branch 'develop' into fix/item-count
Ondra Urban committed 8 years ago
ea0dbe3
Updated USER_AGENT_LIST constant
Jan Curn committed 8 years ago
d3822c4
Merge branch 'master' into develop
Jan Curn committed 8 years ago
f05ff10
Bumped package.json version, updated CHANGELOG
Jan Curn committed 8 years ago
ae47f83
Merge tag 'v0.9.5'
Jan Curn committed 8 years ago
b159b2e
Improved publish.sh
Jan Curn committed 8 years ago
bbb279f
Updated docs
Jan Curn committed 8 years ago
3023693
Bumped package.json version, updated CHANGELOG
Jan Curn committed 8 years ago
6f05285
Apify.call() was causing permissions error
Jan Curn committed 8 years ago
f57abce
Updated docs
Jan Curn committed 8 years ago
20c2836
3cc2798
Fixed docs for utils.downloadListOfUrls()
Jan Curn committed 8 years ago
899efe6
Fixed docs for utils.downloadListOfUrls()
Jan Curn committed 8 years ago
7e36d02
Merge develop
Ondra Urban committed 8 years ago
2756ea9
Update changelog, bump version, build docs
Ondra Urban committed 8 years ago
47f8769