Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Update changelog
Ondra Urban committed 7 years ago
f2a1b39
b284c9b
Merge branch 'develop' into fix/cheerio-leak
Ondra Urban committed 7 years ago
719c8b5
116feda
Merge branch 'develop' into fix/cheerio-leak
Ondra Urban committed 7 years ago
2de44e9
Update changelog
Ondra Urban committed 7 years ago
a3614e1
Update tests
Ondra Urban committed 7 years ago
de76c56
Update openRequestList
Ondra Urban committed 7 years ago
c799dc5
Update RequestList per PR comments
Ondra Urban committed 7 years ago
06cd451
Update openRequestList docs
Ondra Urban committed 7 years ago
bef032e
Pin version of xregexp, 4.2.2 is broken
Ondra Urban committed 7 years ago
ed8bd22
Update changelog
Ondra Urban committed 7 years ago
85918fe
fc90720
Update docs
Ondra Urban committed 7 years ago
46a90f3
Add draft of new getting started
Ondra Urban committed 7 years ago
6e7c243
5ebb934
Improved FACEBOOK_RESERVED_PATHS
Jan Curn committed 7 years ago
26c1207
4b96e6e
Version bump
Marek Trunkat committed 7 years ago
4ab228b
Client version bump
Marek Trunkat committed 7 years ago
1802438
Updated DEFAULT_USER_AGENT
Jan Curn committed 7 years ago
0b8d37e
Updated docs of Apify.utils.htmlToText()
Jan Curn committed 7 years ago
68e5925
Updated docs of Apify.utils.htmlToText()
Jan Curn committed 7 years ago
618933b
Updated FACEBOOK_RESERVED_PATHS
Jan Curn committed 7 years ago
b2644c6
23df324