Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Bump version, update Changelog
Ondra Urban committed 7 years ago
e258229
Fix request queue throttling at high concurrency
Ondra Urban committed 7 years ago
7cc0dfe
App: cosmetic change in warning message
Jan Curn committed 7 years ago
b6f9fe9
Build docs
Ondra Urban committed 7 years ago
50c64bd
72f3b64
Merge branch 'master' into develop
Ondra Urban committed 7 years ago
51fdad6
Update tutorial
Ondra Urban committed 7 years ago
4ec6fb9
Version bump, add fix to changelog
drobnikj committed 7 years ago
172fead
79990e9
Fixed addWebhook create webhook param
drobnikj committed 7 years ago
cabda0e
Merge branch 'master' into develop
Ondra Urban committed 7 years ago
aee8bdc
Build docs
Ondra Urban committed 7 years ago
b5352f6
Fix tutorial image links
Ondra Urban committed 7 years ago
24aa807
Update tutorial, add screenshots
Ondra Urban committed 7 years ago
cc75e6c
Update tutorial
Ondra Urban committed 7 years ago
4518cdf
Updated docs
Jan Curn committed 7 years ago
60cc23f
Updated docs
Jan Curn committed 7 years ago
3d16321
Updated error messages
Jan Curn committed 7 years ago
655b61b
Updated docs
Jan Curn committed 7 years ago
a279029
Added "recycleDiskCache" to docs
Jan Curn committed 7 years ago
96f91f1
Add no-undef rule to eslint
Ondra Urban committed 7 years ago
2d7a37a
Bump version, update Changelog
Ondra Urban committed 7 years ago
e0ab56b
Merge branch 'master' into develop
Ondra Urban committed 7 years ago
9c8d36b
Fix puppeteerPoolOptions not being used
Ondra Urban committed 7 years ago
a02c9be
Fix REQUEST_QUEUE_HEAD_MAX_LIMIT not defined error
Ondra Urban committed 7 years ago
3352d6b
Update docs
Ondra Urban committed 7 years ago
2b04f15
Bump version, update Changelog, build docs
Ondra Urban committed 7 years ago
56da020