Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

improved http2 support. (#846)
petrpatek committed 6 years ago
119598d
Bump @types/puppeteer from 3.0.5 to 5.4.0 (#833)
dependabot-preview[bot] committed 6 years ago
ed70c9e
681f221
0b45638
CheerioCrawler: postResponseFunction (#828)
Jan Bárta committed 6 years ago
4c4a74f
Fix stealth and SessionPool (#834)
Ondra Urban committed 6 years ago
b13a1ed
Add changelog
Ondra Urban committed 6 years ago
cf5f362
Bump version
Ondra Urban committed 6 years ago
e388f95
Hotfix stealth (#824)
petrpatek committed 6 years ago
a97dd8c
Fix deprecated waitFor (#827)
Lukáš Křivka committed 6 years ago
f297aa9
Feature/crawling context (#813)
Jan Bárta committed 6 years ago
de8b708
Create question.md
Ondra Urban committed 6 years ago
fb5e08f
Create feature_request.md
Ondra Urban committed 6 years ago
df3619a
Update bug_report.md
Ondra Urban committed 6 years ago
b9fea5d
Hotfix missing userData in transformRequestFunction
Ondra Urban committed 6 years ago
cd46d5a
Fix loadedUrl crashing validations
Ondra Urban committed 6 years ago
677903d
Fix test
Ondra Urban committed 6 years ago
a0152e0
4a2b12e
0c3d415
86761d3
Use beta version for client and storage-local
Ondra Urban committed 6 years ago
5fb9846
Changed the order of logged statistics (#795)
Jan Bárta committed 6 years ago
cf0120d
Apify Client v1 (#774)
Ondra Urban committed 6 years ago
c421c51
bbb90e7
Update changelog, bump default user agent
Ondra Urban committed 6 years ago
2db7813
Bump puppeteer from 5.2.1 to 5.3.1 (#785)
dependabot-preview[bot] committed 6 years ago
d0bbd9b