Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

b818033
fix: support reloading of sessions with lots of retries
Martin Adámek committed 4 years ago
ebc89d2
chore(deps): lock file maintenance
renovate[bot] committed 4 years ago
45bbd8d
chore(deps): lock file maintenance
renovate[bot] committed 4 years ago
8148912
chore(deps): lock file maintenance
renovate[bot] committed 4 years ago
a8529c6
chore(deps): update npm to v8.19.3
renovate[bot] committed 4 years ago
d70f3c0
docs: add launcher description (#1594)
Markéta Sauerová committed 4 years ago
97dcda2
74a1948
c3a4e1a
ci: merge test flow with release next flow (#1651)
Vlad Frangu committed 4 years ago
430199c
fix: concurrency option override order (#1649)
Jindřich Bär committed 4 years ago
7bbad03
docs: remove launch banner (#1650)
Jindřich Bär committed 4 years ago
d2d2e66
fix: mark session as bad on failed requests (#1647)
Jindřich Bär committed 4 years ago
445ae43
chore: try to test on node 19 (#1606)
Martin Adámek committed 4 years ago
4cbec98
test: fix migration test #2
Szymon Marczak committed 4 years ago
efe0ae5
test: fix migration test
Szymon Marczak committed 4 years ago
cf4176a
chore(deps): update patch/minor dependencies (#1643)
renovate[bot] committed 4 years ago
0163d76
4d96b09
docs: speed up start of gif (#1644)
Vlad Frangu committed 4 years ago
ee4816d
76549eb
docs: improve TS project guide (#1605)
Ndianabasi Udonkang committed 4 years ago
e7d1229
fix: upgrade to [email protected] (#1623)
Szymon Marczak committed 4 years ago
ce36d6b
de9db0c
chore(deps): update dependency turbo to v1.6.2
renovate[bot] committed 4 years ago
c46351c
chore(deps): lock file maintenance (#1613)
renovate[bot] committed 4 years ago
7a2217d
7310b06
docs: add new GIFs for quickstart (#1622)
Vlad Frangu committed 4 years ago
2f701e6