Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

6bd8411
664bdc9
chore: bump playwright and puppeteer
Szymon Marczak committed 5 years ago
f9cbf8f
fix: incognito pages and user data dir (#1145)
Szymon Marczak committed 5 years ago
30523a3
Upgrade to GitHub-native Dependabot (#1007)
dependabot-preview[bot] committed 5 years ago
a63b30a
chore: build docs [skip ci]
Ondra Urban committed 5 years ago
15a98b8
chore: update changelog, bump version
Ondra Urban committed 5 years ago
81102d0
chore: upgrade `got-scraping` (#1128)
Szymon Marczak committed 5 years ago
94a71bf
chore: bump version
Martin Adámek committed 5 years ago
a4762e5
chore: unpin apify-client (#1125)
Martin Adámek committed 5 years ago
4e43351
chore: pin client to 1.3
Martin Adámek committed 5 years ago
d603700
chore: ship 2.0.5 (#1124)
Martin Adámek committed 5 years ago
bfa7daa
chore: update browser-pool (#1123)
Martin Adámek committed 5 years ago
5fd2907
feat: use session token with got-scraping (#1122)
Szymon Marczak committed 5 years ago
d3261ca
fix: bump `got-scraping` to `3.0.1` (#1121)
Szymon Marczak committed 5 years ago
b9e99b7
Disable SSL validation on MITM proxies (#1117)
Szymon Marczak committed 5 years ago
853c5cd
Use `got-scraping@3` (#1118)
Szymon Marczak committed 5 years ago
a31a2b1
Pin `got-scraping` to `2.1.3` (#1120)
Szymon Marczak committed 5 years ago
9a47b1e
chore: fix version number in changelog [skip ci]
Ondra Urban committed 5 years ago
b447522
chore: improve wording in changelog [skip ci]
Ondra Urban committed 5 years ago
3aae58d
fix: refactor `requestAsBrowser` to Got 12 (#1111)
Ondra Urban committed 5 years ago
ef9a4ad
chore: add aborting event to events docs [skip ci]
Ondra Urban committed 5 years ago
c89f532
chore: bump version, update changelog
Ondra Urban committed 5 years ago
ed41360
fix: cheerio issues with HTML serialization (#1113)
Ondra Urban committed 5 years ago
e0ae762