Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

feat: infiniteScroll has maxScrollHeight limit (#1945)
Jindřich Bär committed 3 years ago
44997bb
chore: update got to v13
Martin Adámek committed 3 years ago
ac0bd25
chore: update lerna to v7
Martin Adámek committed 3 years ago
99c4797
chore(deps): update dependency playwright to v1.35.0
renovate[bot] committed 3 years ago
7cce640
chore: update TS to v5+ in e2e tests
Martin Adámek committed 3 years ago
460c289
42ea9eb
chore: disable npm audit in e2e tests on platform 2
Martin Adámek committed 3 years ago
f6d5fa0
chore: disable npm audit in e2e tests on platform
Martin Adámek committed 3 years ago
60c41fa
chore: fix linkedom e2e test
Martin Adámek committed 3 years ago
47b7c7e
fix: respect `<base>` when enqueuing (#1936)
Martin Adámek committed 3 years ago
aeef572
feat: add LinkeDOMCrawler (#1907)
Pavel 'Strajk' Dolecek committed 3 years ago
1c69560
chore(deps): lock file maintenance
renovate[bot] committed 3 years ago
30f6be8
chore(deps): update patch/minor dependencies
renovate[bot] committed 3 years ago
de1b3ae
chore(deps): lock file maintenance
renovate[bot] committed 3 years ago
945b30b
chore(deps): update yarn to v3.6.0
renovate[bot] committed 3 years ago
9afdad5
chore(deps): lock file maintenance
renovate[bot] committed 3 years ago
da17c8f
chore(deps): update dependency turbo to v1.10.1
renovate[bot] committed 3 years ago
9486c16
chore(deps): update dependency puppeteer to v20.5.0
renovate[bot] committed 3 years ago
c23b1c4
chore(release): v3.3.3
Apify Release Bot committed 3 years ago
5366b91
chore(deps): update dependency turbo to v1.10.0
renovate[bot] committed 3 years ago
f8a29cb
d4a49e5
chore(e2e): bump images to nodejs 18 (#1928)
Vlad Frangu committed 3 years ago
ad56cde
4aabca4
6358af7
chore(deps): update dependency puppeteer to v20.4.0
renovate[bot] committed 3 years ago
0c9b331
chore: update more GitHub workflow secrets (#1923)
František Nesveda committed 3 years ago
f0a6348
aa473de