Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Feature/new docker image (#37)
Jan Čurn committed 8 years ago
fbc7062
Fixing links
Marek Trunkat committed 8 years ago
ea83476
Version bump
Marek Trunkat committed 8 years ago
ec017c7
Updating puppeteer and changelog
Marek Trunkat committed 8 years ago
d5c5c17
Version bump
Marek Trunkat committed 8 years ago
f83e509
Implementing actor events (#35)
Marek Trunkát committed 8 years ago
f59bad7
Fixing some imports
Marek Trunkat committed 8 years ago
371c859
Upgrading apify-shared
Marek Trunkat committed 8 years ago
8ed204b
Adding some examples and documentation
Marek Trunkat committed 8 years ago
13b6f47
Update comments maxConcurrency<->minConcurrency
JakubDrobnik committed 8 years ago
e3d371f
Adding example
Marek Trunkat committed 8 years ago
1a60268
Updating readme
Marek Trunkat committed 8 years ago
d4913ff
Version bump
Marek Trunkat committed 8 years ago
5a545d5
Documenting
Marek Trunkat committed 8 years ago
09404e7
Small imrovements of sdk docs
Marek Trunkat committed 8 years ago
aee715b
Feature/sdk improvements (#33)
Marek Trunkát committed 8 years ago
cc97e2d
Feature/call wait (#32)
Marek Trunkát committed 8 years ago
f5383b8
Default apify proxy hostname and apify proxy port (#30)
Jakub Drobník committed 8 years ago
a72c6ff
Typo in AutoscaledPool doc
JakubDrobnik committed 8 years ago
64f7a4e
Fixed forgotten change from seq. store to dataset in docs
Lukáš Křivka committed 8 years ago
24df9cd
Feature/call wait (#28)
Jan Čurn committed 8 years ago
15407c1
Feature/feedback2 (#26)
Jan Čurn committed 8 years ago
ecf76f1
Modify call opts params type (#27)
Juan Sebastián Gaitán Villamizar committed 8 years ago
31f3094
Version bump
Marek Trunkat committed 8 years ago
1d76040