Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

save
Petr Patek committed 7 years ago
bd7dcb3
builded docs once again.
Petr Patek committed 7 years ago
a41ce63
Updated docs
Jan Curn committed 7 years ago
d5a6e43
fix lint
Petr Patek committed 7 years ago
3788283
improved code and docs
Petr Patek committed 7 years ago
0b97439
Updated docs
Jan Curn committed 7 years ago
b3e9634
Updated docs
drobnikj committed 7 years ago
e32bd75
Updated comments
drobnikj committed 7 years ago
0389210
Improve docs
Ondra Urban committed 7 years ago
2f5ac46
fixed check user agent script
Petr Patek committed 7 years ago
1d5edf3
added openSessionPool.
Petr Patek committed 7 years ago
b0028f6
faf07fd
Improved tests.
Petr Patek committed 7 years ago
5f28f02
reworded.
Petr Patek committed 7 years ago
ae519ce
improved code added docs.
Petr Patek committed 7 years ago
42c2297
added tests
Petr Patek committed 7 years ago
65c7e4c
added comments and session tests.
Petr Patek committed 7 years ago
9ff861e
renamed session event
Petr Patek committed 7 years ago
93dca07
Improved code.
Petr Patek committed 7 years ago
35e7a67
Cheerio Scraper: Built docs
drobnikj committed 7 years ago
70f0364
Refactored and improved code.
Petr Patek committed 7 years ago
dde9f81
Build doc
drobnikj committed 7 years ago
4ab5fa1
fixed CheerioCrawler request timeouts.
Petr Patek committed 7 years ago
ca9f1d6