Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Add auto-connect to live view client
Ondra Urban committed 7 years ago
6a1b4b5
Clean up client
Ondra Urban committed 7 years ago
f504511
Fix socket connection
Ondra Urban committed 7 years ago
15ca608
Always make one snapshot live view
Ondra Urban committed 7 years ago
1740635
f8a0323
Live view styles
Vratislav Bartonicek committed 7 years ago
d6c8707
Live view UI init
Vratislav Bartonicek committed 7 years ago
931d4d9
Fix prepare request
Ondra Urban committed 7 years ago
609c556
Bump version
Ondra Urban committed 7 years ago
2d932df
Fix and build docs
Ondra Urban committed 7 years ago
f963eb6
Improve docs, update Changelog
Ondra Urban committed 7 years ago
4db979b
Fix typos
Ondra Urban committed 7 years ago
a49d7b2
b9f5746
Fix lint issue
Ondra Urban committed 7 years ago
0579716
Apply PR comments
Ondra Urban committed 7 years ago
e3d2b06
Merge develop, resolve conflicts
Ondra Urban committed 7 years ago
8e9755a
Update changelog
Ondra Urban committed 7 years ago
7dbedba
70ecf59
PR - fixes
Marek Trunkat committed 7 years ago
8b727e3
Build docs
Ondra Urban committed 7 years ago
56b9b9f
93bcd95
Fixed tests.
Petr Patek committed 7 years ago
d63a278
More comments
Ondra Urban committed 7 years ago
9a0521e
Add comments
Ondra Urban committed 7 years ago
e903a70