Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Update PR with comments
Ondra Urban committed 7 years ago
a89a866
Added unit test for Apify.Request
Jan Curn committed 7 years ago
ef3939b
Fix merge conflicts
Ondra Urban committed 7 years ago
2748ffd
Fix merge conflicts
Ondra Urban committed 7 years ago
bd187c2
Update _helper.js
Marek Trunkát committed 7 years ago
4fb208a
Linting
Marek Trunkat committed 7 years ago
72e1f9b
a88f4aa
Exiting tests when there unhandled promise rejection
Marek Trunkat committed 7 years ago
d5ef33d
901d6fa
Better error messages
Jan Curn committed 7 years ago
9662bc8
Added TODOs
Jan Curn committed 7 years ago
317be75
Added TODO
Jan Curn committed 7 years ago
43bd3ff
Updated docs
Jan Curn committed 7 years ago
aef2ed6
Fixed dates in CHANGELOG
Jan Curn committed 7 years ago
013199d
Added a TODO
Jan Curn committed 7 years ago
79b4e7b
Fixed formatting and typo.
Petr Patek committed 7 years ago
88d1ec9
added changelog
Petr Patek committed 7 years ago
bd07154
fixed formatting
Petr Patek committed 7 years ago
5c9a237
improved log.
Petr Patek committed 7 years ago
e7446bb
89fd0f1
Created critical memory overload log.
Petr Patek committed 7 years ago
3795ac0
Build docs
Ondra Urban committed 7 years ago
8169967
Fix lint issues
Ondra Urban committed 7 years ago
e2e1f60
Update changelog, bump version
Ondra Urban committed 7 years ago
309bfcd
Fix puppeteer version
Ondra Urban committed 7 years ago
b52eccf