Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Merge branch 'develop' into feature/create_webhook
Ondra Urban committed 7 years ago
c8d0609
Update examples
Ondra Urban committed 7 years ago
b384c5e
Throttle API in AutoscaledPool on 2nd retry
Ondra Urban committed 7 years ago
8a14a7d
Changed Apify.webhook() to Apify.addWebhook()
drobnikj committed 7 years ago
9eae0ac
Updated how we check local runs + test
drobnikj committed 7 years ago
af49070
Fixed comments, typos and other things from PR
drobnikj committed 7 years ago
e6e6bad
Bump version, changelog
drobnikj committed 7 years ago
d2b526d
37c0fc2
Merge branch 'master' into develop
drobnikj committed 7 years ago
ee7a359
bbfd5ac
Merge branch 'master' into develop
Ondra Urban committed 7 years ago
dcf8228
Remove Hotjar because it doesn't work
Ondra Urban committed 7 years ago
00fb530
Add Hotjar to Docusaurus
Ondra Urban committed 7 years ago
90b52f8
Progress commit
Ondra Urban committed 7 years ago
74d10c8
Initial commit
Ondra Urban committed 7 years ago
ae56569
Bump version, update Changelog, build docs
Ondra Urban committed 7 years ago
f8d63f8
757f7e1
Update Puppeteer to 1.13
Ondra Urban committed 7 years ago
5d30adb
28b0225
Implement PR comments
Ondra Urban committed 7 years ago
07755c9
Merge develop, resolve conflicts
Ondra Urban committed 7 years ago
bd59d73
Build docs
Ondra Urban committed 7 years ago
6d48d71
Update changelog
Ondra Urban committed 7 years ago
de35e05
fixed lint
Petr Patek committed 7 years ago
3a2be4b
Reword docs
Ondra Urban committed 7 years ago
ab07d11