Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

482ac55
Merge develop
Ondra Urban committed 8 years ago
d47d9f7
f3829fe
Add test
Ondra Urban committed 8 years ago
734bcca
Updated PuppeteerCrawler docs
Jan Curn committed 8 years ago
38cfd46
06992ab
Updating CHANGELOG.md
Marek Trunkat committed 8 years ago
1f6f731
Add error suppression listener
Ondra Urban committed 8 years ago
6bb1535
ab87854
Updating client version to the latest BETA
Marek Trunkat committed 8 years ago
23694b9
Merge branch 'develop' into feature/call_fixes
Marek Trunkát committed 8 years ago
b6a95fa
Renaming call params
Marek Trunkat committed 8 years ago
a202412
ea4ad3a
21b62a9
Fix docs
Ondra Urban committed 8 years ago
6b3bcb6
f9f84ea
Updated generated docs
Jan Curn committed 8 years ago
a7a4e70
Fixed lint error
Jan Curn committed 8 years ago
5b70607
Updated publish.sh, generated new docs
Jan Curn committed 8 years ago
268464f
Updated publish.sh
Jan Curn committed 8 years ago
a64c63e
Removed old code from publish.sh
Jan Curn committed 8 years ago
b4ab7e5
8a7bb32
Add throw error option to request.skip()
Ondra Urban committed 8 years ago
e984186
c63237b
Updating signature of openStorage function
Marek Trunkat committed 8 years ago
a3b5c1f
d371973
ff8c749
94a04f1