Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Bumped package.json version to 0.4.10
Jan Curn committed 9 years ago
ad2cf6f
Merge branch 'master' into develop
Jan Curn committed 9 years ago
66599bc
Bumped package.json version
Jan Curn committed 9 years ago
aed700f
Better logging in ProxyChain.shutdown()
Jan Curn committed 9 years ago
3c45fa3
Updating ./publish.sh
Marek Trunkat committed 9 years ago
cfe4d0d
Updated README.md
Jan Curn committed 9 years ago
99f24bb
Updated README.md
Jan Curn committed 9 years ago
d8c4e0a
Reverted to old version of publish.sh
Jan Curn committed 9 years ago
dce7e36
Updated README.md, bumped package.json version
Jan Curn committed 9 years ago
703250b
Updated LICENSE
Jan Curn committed 9 years ago
d823f5f
Renamed APIFY_ENV_VARS constant to ENV_VARS
Jan Curn committed 9 years ago
1d71e26
Bumped package.json version
Jan Curn committed 9 years ago
e346219
Bumped package.json version
Jan Curn committed 9 years ago
6b1b399
Updated README.md
Jan Curn committed 9 years ago
d218063
Better unit tests for Apifier.setValue()
Jan Curn committed 9 years ago
0706a5c
Attempt to avoid failed squid startups
Jan Curn committed 9 years ago
bcf48ea