Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

fix: workaround for decodeURI (#1206)
Szymon Marczak committed 5 years ago
e248fb2
build: fix deploy action condition [skip ci]
Martin Adámek committed 5 years ago
e71869b
chore: use `apify-client` v2
Martin Adámek committed 5 years ago
d3555fc
cccea78
chore(deps): update dependency playwright to v1.15.2
Renovate Bot committed 5 years ago
cbbee5c
cd98da0
chore: update changelog
Martin Adámek committed 5 years ago
015a99a
chore(deps): update dependency playwright to v1.15.1
Renovate Bot committed 5 years ago
9418dde
chore: await the route.continue() before we return it
Martin Adámek committed 5 years ago
67b10c7
chore: update changelog and use minor version bump
Martin Adámek committed 5 years ago
97e1a8c
fix: catch errors inside request interceptors (#1192)
Martin Adámek committed 5 years ago
69a5a33
feat(utils): add `purgeLocalStorage` method (#1187)
Vlad Frangu committed 5 years ago
2ace074
chore(deps): update patch/minor dependencies
Renovate Bot committed 5 years ago
3ca040b
chore(deps): update dependency playwright to v1.15.0
Renovate Bot committed 5 years ago
f2376dc
add support for cgroups v2 (#1177)
Ondra Urban committed 5 years ago
f174531
2bc2e6b
fix: incorrect offset in `fixUrl` function (#1184)
Szymon Marczak committed 5 years ago
bc8c0a4
docs: enable google analytics
Martin Adámek committed 5 years ago
72f8de3
chore: update changelog
Martin Adámek committed 5 years ago
1af0e52
chore: run lockfile maintenance every day
Martin Adámek committed 5 years ago
8140645
ff4d4ba
feat: warn if apify proxy is used in proxyUrls (#1173)
Szymon Marczak committed 5 years ago
682f8b7
6c21c4a