Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Update Puppeteer to 1.18
Ondra Urban committed 7 years ago
8905a99
Build docs
Ondra Urban committed 7 years ago
d199fe7
705349a
Fix typo in PuppeteerCrawler docs [skip ci]
Ondra Urban committed 7 years ago
9a8ce7c
10778a0
Merge master, resolve conflicts
Ondra Urban committed 7 years ago
dc4ed7c
Merge master, resolve conflicts
Ondra Urban committed 7 years ago
b785fec
Update Travis to not deploy pull request builds
Ondra Urban committed 7 years ago
3207a85
Update dataset getData docs
Ondra Urban committed 7 years ago
0f7bbfd
Build docs
Ondra Urban committed 7 years ago
4a88a17
Add release docs
Ondra Urban committed 7 years ago
78d7eb6
Merge branch 'master' into feature/element_clicking
Ondra Urban committed 7 years ago
1a00a3d
Fix image release git repo
Ondra Urban committed 7 years ago
c39dccb
Fix image release git checkout
Ondra Urban committed 7 years ago
d0e0d22
Fix image relase git push command
Ondra Urban committed 7 years ago
fbfeaf7
Fix github token for image release
Ondra Urban committed 7 years ago
c63543a
Remove dry-run from release of beta images
Ondra Urban committed 7 years ago
4b75479
Bump version to update beta
Ondra Urban committed 7 years ago
3b54a59
Fix package.json path in before-deploy
Ondra Urban committed 7 years ago
0354308
Add logging from image release script
Ondra Urban committed 7 years ago
10a99b8
Re-encrypt NPM token for Travis
Ondra Urban committed 7 years ago
ed61d2b
Update publish script
Ondra Urban committed 7 years ago
e50f600
Rename updateRequest to transformRequest
Ondra Urban committed 7 years ago
2bd63d3
Update changelog, bump version
Ondra Urban committed 7 years ago
1f3c658
Update changelog, build docs
Ondra Urban committed 7 years ago
fd06491