Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

chore(release): update internal dependencies [skip ci]
Apify Release Bot committed 2 years ago
1bb9570
chore(release): v3.9.2
Apify Release Bot committed 2 years ago
b80011e
fix: don't call `notify` in `addRequests()` (#2425)
Jindřich Bär committed 2 years ago
c4d5446
feat(sitemap): Support CDATA in sitemaps (#2424)
Pavel 'Strajk' Dolecek committed 2 years ago
635f046
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
915c6bf
6f2e6b0
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
eb13289
chore(deps): update dependency puppeteer to v22.6.5
renovate[bot] committed 2 years ago
e89993e
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
683652a
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
a1de9fb
eff9371
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
3497133
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
aefa8d5
chore(deps): update dependency puppeteer to v22.6.4
renovate[bot] committed 2 years ago
bf2b42e
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
d2807c6
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
4aabd16
chore(release): update internal dependencies [skip ci]
Apify Release Bot committed 2 years ago
c9bd1d9
chore(release): v3.9.1
Apify Release Bot committed 2 years ago
d74bb29
feat: `browserPerProxy` browser launch option (#2418)
Jindřich Bär committed 2 years ago
df57b29
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
0cd8017
chore(deps): update patch/minor dependencies
renovate[bot] committed 2 years ago
dfbf0e2
test: revert unwanted changes to fix tests
Martin Adámek committed 2 years ago
7955c88
docs: add v3.9 snapshot
Martin Adámek committed 2 years ago
28d83f3
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
6be87a4
chore(release): update internal dependencies [skip ci]
Apify Release Bot committed 2 years ago
a1bfb6e
chore(release): v3.9.0
Apify Release Bot committed 2 years ago
38d440c
cee4778
docs: cross-domain-linking (#2413)
Saurav Jain committed 2 years ago
e0a6218
9089bf1
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
ff5cfbf