Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Merge branch 'master' into feature/proxy-chain
Jan Curn committed 9 years ago
5abb77d
Development of better proxy chains
Jan Curn committed 9 years ago
b5a64c5
002b0b9
Updated unit tests
Jan Curn committed 9 years ago
83e1cbd
Fixed package.json
Jan Curn committed 9 years ago
2853a4e
Fixed invalid path
Jan Curn committed 9 years ago
62cc414
9b26ccc
Added Apify.launchWebDriver() function
Jan Curn committed 9 years ago
214255b
Bumped package.json version
Jan Curn committed 9 years ago
be1b9e0
Fixed links to docs
Jan Curn committed 9 years ago
06aa8f0
Updated docs
Jan Curn committed 9 years ago
cc43d70
Cosmetic fix in docs
Jan Curn committed 9 years ago
914e5b8
Merge branch 'develop'
Jan Curn committed 9 years ago
4189778
Bumped package.json version
Jan Curn committed 9 years ago
d98b6fa
Fixes in docs
Jan Curn committed 9 years ago
ca9c3c9
1910943
Bumped package.json version
Jan Curn committed 9 years ago
c1ed8a4
Updated docs and README
Jan Curn committed 9 years ago
8dd4561
Writing docs
Jan Curn committed 9 years ago
e746aa0
Bumped package.json version
Jan Curn committed 9 years ago
b44d927
Cosmetic changes in texts and docs
Jan Curn committed 9 years ago
ab88434
eb4a8f5
Updating launchPuppeter()
Marek Trunkat committed 9 years ago
99752be
Running puppetteer in tests in headless mode
Marek Trunkat committed 9 years ago
b8d90c0
Trying --disable-setuid-sandbox
Marek Trunkat committed 9 years ago
9083f51
Adding launchPuppeteer()
Marek Trunkat committed 9 years ago
b2bba5c