Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Better error text for ApifyCallError (#74)
Jan Čurn committed 8 years ago
4deede1
Adding missing import
Marek Trunkat committed 8 years ago
fb3abe6
Updating readme
Marek Trunkat committed 8 years ago
d5691e2
Updating README.md
Marek Trunkat committed 8 years ago
114b097
Version bump
Marek Trunkat committed 8 years ago
025608f
Removing unneeded params from examples
Marek Trunkat committed 8 years ago
bfceb5f
Merge pull request #69 from juansgaitan/patch-3
Jan Čurn committed 8 years ago
7f31771
Add 9MB data max in pushData
Juan Sebastián Gaitán Villamizar committed 8 years ago
73eaf86
55ca0ce
Updating changelog
Marek Trunkat committed 8 years ago
05acc85
Feature/apify call throw (#66)
Marek Trunkát committed 8 years ago
69166cb
Fixing documentation
Marek Trunkat committed 8 years ago
9063696
Fixing documentation
Marek Trunkat committed 8 years ago
3e4ab8f
Fixing docs
Marek Trunkat committed 8 years ago
239237b
Version bump
Marek Trunkat committed 8 years ago
f245b97
Feature/docs (#62)
Marek Trunkát committed 8 years ago
614c951
Fix/apify shared (#63)
Jan Čurn committed 8 years ago
435f6bd
Version bump
Marek Trunkat committed 8 years ago
0636fd9
Feature/url list persist (#61)
Marek Trunkát committed 8 years ago
bd2476b
Feature/sigterm (#57)
Marek Trunkát committed 8 years ago
94680a9
f605715
Scaling pool down less aggresively (#56)
Marek Trunkát committed 8 years ago
94cf79c
Linting
Marek Trunkat committed 8 years ago
b01f4d2
Version bump
Marek Trunkat committed 8 years ago
b6a40d8