Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

fix: sitemaps support `application/xml` (#2408)
Jindřich Bär committed 2 years ago
cbcf47a
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
2228862
chore(deps): update dependency puppeteer to v22.6.3
renovate[bot] committed 2 years ago
5ac7b04
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
b081b68
3559436
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
07f80e5
chore(deps): update patch/minor dependencies
renovate[bot] committed 2 years ago
1d155ca
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
4127398
330598b
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
f35b805
chore(deps): update node.js to v20.12.1
renovate[bot] committed 2 years ago
e6eb9fd
a90177d
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
1491d12
chore(deps): update dependency turbo to v1.13.2
renovate[bot] committed 2 years ago
1890f43
chore(deps): update dependency puppeteer to v22.6.2
renovate[bot] committed 2 years ago
62efbaf
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
32eceb9
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
2fc48bd
chore(deps): update dependency turbo to v1.13.1
renovate[bot] committed 2 years ago
61a040f
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
d5cedd1
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
7f4c376
chore(deps): update patch/minor dependencies to v3.2.0
renovate[bot] committed 2 years ago
65c1d35
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
6832003
test: do not retry tests locally
Martin Adámek committed 2 years ago
25e8bcb
docs: add amazon blog (#2395)
Saurav Jain committed 2 years ago
87aab45
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
453dd3b
chore(deps): update patch/minor dependencies (#2369)
renovate[bot] committed 2 years ago
af38972
test: circumvent ORB in `blockRequests` tests (#2394)
Jindřich Bär committed 2 years ago
231ddcb