Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

feat: add multiple click (#1295)
audi Bookning committed 4 years ago
09eeb28
feat: bump puppeteer support to up to v13.x (#1301)
Vlad Frangu committed 4 years ago
0511bb6
91d152a
181ca2b
72a9855
chore(deps): update dependency typescript to v4.6.2
Renovate Bot committed 4 years ago
b775115
chore(deps): update dependency playwright to v1.19.2
Renovate Bot committed 4 years ago
986a674
docs: fix minimum node version in v2 snapshots
Martin Adámek committed 4 years ago
f09cfa8
docs: fix minimum node version
Martin Adámek committed 4 years ago
f91d0e1
test: try to remove timeouts from failing test
Martin Adámek committed 4 years ago
e644658
feat: accept more social media patterns (#1286)
Kristýna Lhoťanová committed 4 years ago
d1fdf65
test: fix flaky tests (#1298)
Andrey Bykov committed 4 years ago
25f3ae1
chore(deps): update dependency playwright to v1.19.1
Renovate Bot committed 4 years ago
67f9c9f
chore: bump to 2.2.3
Martin Adámek committed 4 years ago
bea36dd
chore: use stable versions of client and storage-local
Martin Adámek committed 4 years ago
f98a238
chore: try to downgrade playwright to 1.18
Martin Adámek committed 4 years ago
7dc1861
chore(deps): update patch/minor dependencies
Renovate Bot committed 4 years ago
bfd870a
chore: add over-ridden request queue timeout (#1288)
Andrey Bykov committed 4 years ago
f418f2f
docs: fix link in typedefs.js (#1287)
Jan Čurn committed 4 years ago
1d467ad
fix: cookies setting in preNavigationHooks (#1283)
Andrey Bykov committed 4 years ago
178d6e3
fix: ensure `request.headers` is set (#1281)
Martin Adámek committed 4 years ago
966292b
chore: update typescript and fix types in docs (#1241)
Szymon Marczak committed 4 years ago
f741dab
docs: adding playwright crawler (#1279)
Sapiens Malti committed 4 years ago
7281803
chore(deps): update patch/minor dependencies
Renovate Bot committed 4 years ago
30c2039
chore: bump version to 2.2.2
Martin Adámek committed 4 years ago
0df187a
chore: update changelog
Martin Adámek committed 4 years ago
51cdede
e622e6b