Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Update README examples
Ondra Urban committed 8 years ago
fc833c8
15c8fe7
012e7a2
Update changelog; bump version
Ondra Urban committed 8 years ago
802ec04
Add compileScript() function
Ondra Urban committed 8 years ago
f99b51c
Updating docs
Marek Trunkat committed 8 years ago
c0b5988
Updating docs
Marek Trunkat committed 8 years ago
e532761
Updating README.md
Marek Trunkat committed 8 years ago
993dd8d
Updating docs
Marek Trunkat committed 8 years ago
05feee1
Fighting with lint
Marek Trunkat committed 8 years ago
eb94c37
Merging develop
Marek Trunkat committed 8 years ago
702debc
cee63e3
Merge branch 'develop' into feature/copywriting
Marek Trunkát committed 8 years ago
9a43081
170fa56
Refactoring ENV variables
Marek Trunkat committed 8 years ago
db0977d
Change live view status icon on actor finish
Ondra Urban committed 8 years ago
d82ca40
Fix lint issue
Ondra Urban committed 8 years ago
3896b48
Fix deprecation warning, fix keepDuplicateUrls
Ondra Urban committed 8 years ago
1edd99a
Updated dataset documentation
Jan Curn committed 8 years ago
9c4ecb2
Writing new README file
Jan Curn committed 8 years ago
849f1cf
Writing new README file
Jan Curn committed 8 years ago
74cd639
Writing new README file
Jan Curn committed 8 years ago
1fedf05
Writing new README file
Jan Curn committed 8 years ago
b2731b2
Writing new README file
Jan Curn committed 8 years ago
2d8efe7
Writing new README file and updating docs
Jan Curn committed 8 years ago
5c86f3d
Merge branch 'develop' into feature/sdk-review
Jan Curn committed 8 years ago
e5fd9bc
Fix comment that prevents JSDoc build
Ondra Urban committed 8 years ago
76564d4
Fix missing test reports
Ondra Urban committed 8 years ago
cb41583
Update changelog; updage apify-shared
Ondra Urban committed 8 years ago
2c5a8ea