Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Unrelated: Rename Library to Store
Ondra Urban committed 7 years ago
06a0bcd
Use systemInfo event in Snapshotter
Ondra Urban committed 7 years ago
c28642a
Improve docs
Ondra Urban committed 7 years ago
4dafa65
Improve docs
Ondra Urban committed 7 years ago
907efc7
Updated docs
Jan Curn committed 7 years ago
0b5fbd6
ff4dc2a
Updating apify client
Marek Trunkat committed 7 years ago
4be7261
Removing uneeded log file
Marek Trunkat committed 7 years ago
a394ad0
edefa97
Bump version, update Changelog, build docs
Ondra Urban committed 7 years ago
bbf7629
Fix lint issues
Ondra Urban committed 7 years ago
f40da63
Add payload template to .addWebhook()
Ondra Urban committed 7 years ago
7b59ba5
Updating docs
Marek Trunkat committed 7 years ago
311d513
Linting
Marek Trunkat committed 7 years ago
4ff73f0
Version bump
Marek Trunkat committed 7 years ago
d986bc4
Version bump
Marek Trunkat committed 7 years ago
3300b17
Updating shared
Marek Trunkat committed 7 years ago
251f697
8cb8a88
Merge branch 'develop' into feature/queue_consistency
Marek Trunkát committed 7 years ago
0e954d8
5656752
Build docs
Ondra Urban committed 7 years ago
d39b263
Merge branch 'develop'
Ondra Urban committed 7 years ago
7930679
Improved docs
Jan Curn committed 7 years ago
9e1fa36
Improved crawler docs
Jan Curn committed 7 years ago
0dc1e19
Using jsonStringifyExtended() for stringification
Marek Trunkat committed 7 years ago
51ad491
Update Puppeteer to 1.15
Ondra Urban committed 7 years ago
ca8c6ea
Bump version, update Changelog, build docs
Ondra Urban committed 7 years ago
4d4b721