Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Avoid uncaught promise rejection
Ondra Urban committed 8 years ago
dcabece
Update docs
Ondra Urban committed 8 years ago
0c5dfd7
Merge branch 'develop' into feature/actor-ui-inputs
Ondra Urban committed 8 years ago
72eff00
Update readme source files
Ondra Urban committed 8 years ago
3f3efe1
Merge branch 'develop' into feature/actor-ui-inputs
Ondra Urban committed 8 years ago
3519df1
ccc8b5b
text edits
davidjohnbarton committed 8 years ago
32e88c2
Updated SDK main page
Jan Curn committed 8 years ago
bbf09a7
Updated README
Jan Curn committed 8 years ago
c0357a1
Updated README
Jan Curn committed 8 years ago
9285f4f
Updated README
Jan Curn committed 8 years ago
7096fb1
Resolve develop merge conflicts
Ondra Urban committed 8 years ago
470c317
ba8059e
15f1677
apify-shared version bump
Marek Trunkat committed 8 years ago
3e6290e
Fixing typo in changelog
Marek Trunkat committed 8 years ago
8a47fee
Bumping apify client to the latest version
Marek Trunkat committed 8 years ago
d9dd821
Footer links fix, syntax highlight style, png images optimized
Vratislav Bartonicek committed 8 years ago
0b5ae5c
Fix changelog typo
Ondra Urban committed 8 years ago
0c9280e
Implement proxyUrls in PuppeteerPool
Ondra Urban committed 8 years ago
168a03b
Implement and test PseudoUrl input
Ondra Urban committed 8 years ago
0162243
Remove ToC script
Ondra Urban committed 8 years ago
c5c4e63
Fix links in examples
Ondra Urban committed 8 years ago
414a930
Implement README build, fix publish
Ondra Urban committed 8 years ago
c3375f5
Bumped package.json version, updated CHANGELOG
Jan Curn committed 8 years ago
5aa58e9
Docusaurus (#205)
Ondra Urban committed 8 years ago
5d9fd09
Merge branch 'develop'
Marek Trunkat committed 8 years ago
94b7292
Update default user agent and their list
Ondra Urban committed 8 years ago
69568d1
Lintfix
Marek Trunkat committed 8 years ago
98f656f
Commenting one unstable Snapshotter test
Marek Trunkat committed 8 years ago
ade936c