Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Updated README
Jan Curn committed 8 years ago
b508a3c
5c7832a
a6e33ed
Update README.md
Marek Trunkát committed 8 years ago
29a3f55
Update README.md
Marek Trunkát committed 8 years ago
977c458
Update README.md
Marek Trunkát committed 8 years ago
66cc0b7
Update README.md
Marek Trunkát committed 8 years ago
4f18408
Update README.md
Marek Trunkát committed 8 years ago
64e3fe0
Updated log text
Jan Curn committed 8 years ago
f6fa2d3
Edited README
davidjohnbarton committed 8 years ago
639186c
Updated README
Jan Curn committed 8 years ago
543e0b7
WIP - saving to add notes
davidjohnbarton committed 8 years ago
7d4b2ad
a617fc5
Updated README
Jan Curn committed 8 years ago
c2ca355
Updated docs and README
Jan Curn committed 8 years ago
c413f22
d79c749
Updated README and package.json description
Jan Curn committed 8 years ago
325a0fa
Still WIP - paused until later
davidjohnbarton committed 8 years ago
11dff4b
Update actor.js
davidjohnbarton committed 8 years ago
d140d6e
Update system_status.js
davidjohnbarton committed 8 years ago
e8588a7
Update snapshotter.js
davidjohnbarton committed 8 years ago
3f5959d
Version bump
Marek Trunkat committed 8 years ago
e1fcc1a
Removing unneeded code
Marek Trunkat committed 8 years ago
3a1cf05
Updated README
Jan Curn committed 8 years ago
6347478
Update autoscaled_pool.js
davidjohnbarton committed 8 years ago
9421ab5
Version bump
Marek Trunkat committed 8 years ago
7f26b55
69e90d0
Commenting
Marek Trunkat committed 8 years ago
bcb1d9d