Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Updated to [email protected]
Jan Curn committed 8 years ago
68b03d1
Fix request.handledAt type; bump version
Ondra Urban committed 8 years ago
28ad5b4
Version bump
Marek Trunkat committed 8 years ago
4dca33e
e19f62d
Puting request at last the end of head
Marek Trunkat committed 8 years ago
2e44ad6
typo
Marek Trunkat committed 8 years ago
4342beb
Updated README
Jan Curn committed 8 years ago
218b82a
Version bump
Marek Trunkat committed 8 years ago
696b05c
d63dad9
Update puppeteer_pool.js
Marek Trunkát committed 8 years ago
fda8bb7
49085bb
e2f6ac0
Puppeteer pool - fix
Marek Trunkat committed 8 years ago
1325cb8
Version bump
Marek Trunkat committed 8 years ago
36c848e
0a21836
Fix RequestQueue not updating on Local
Ondra Urban committed 8 years ago
ce6f546
README - fixing typo
Marek Trunkat committed 8 years ago
1b00cf1
Added examples/puppeteer_sitemap.js
Jan Curn committed 8 years ago
659c767
README - fixing typo
Marek Trunkat committed 8 years ago
4561b52
Updated README and /examples
Jan Curn committed 8 years ago
4b6b6d0
8e0f0da
Updated README
Jan Curn committed 8 years ago
a3a99da