Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

added getPublicUrl function to key_value_store
Aaron Jackson committed 7 years ago
4443fc7
Build docs
Ondra Urban committed 7 years ago
8fd3636
Update changelog, bump version
Ondra Urban committed 7 years ago
db37353
eddf07d
30bdc03
Update tests to match new error message
Ondra Urban committed 7 years ago
ff22e84
Update setValue stringification error message
Ondra Urban committed 7 years ago
ffda713
added getPublicUrl function to key_value_store
Aaron Jackson committed 7 years ago
fa875f2
Updated docs
Jan Curn committed 7 years ago
863fcc2
Bumped package.json version, updated CHANGELOG
Jan Curn committed 7 years ago
5954d6f
Update docs
Ondra Urban committed 7 years ago
c821860
Updated publish_docs.sh
Jan Curn committed 7 years ago
cd8c800
Updated docs
Jan Curn committed 7 years ago
0d7e833
Bumped package.json version, updated CHANGELOG
Jan Curn committed 7 years ago
50ff126
b06ee5d
cfaf6fa
Updated docs
Jan Curn committed 7 years ago
d07c073
Bumped package.json version, updated CHANGELOG
Jan Curn committed 7 years ago
ef00e98
5af64c4
6a6d79b
b883e66
Bumped package.json version, updated CHANGELOG
Jan Curn committed 7 years ago
f708545
Updated docs
Jan Curn committed 7 years ago
2a3318b
f93f077
9561473
Updated NPM dependencies and CHANGELOG
Jan Curn committed 8 years ago
03ff46f
209316d