Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
chore(deps): update dependency typescript to v6 (#3516)
1d48644 renovate[bot] committed 7 months ago
chore(deps): bump brace-expansion from 1.1.12 to 1.1.13 in /website/versioned_docs/version-3.16 (#3536)
c7c8d30 dependabot[bot] committed 7 months ago
chore(deps): bump path-to-regexp from 0.1.12 to 0.1.13 in /website (#3537)
6d30b47 dependabot[bot] committed 7 months ago
chore(deps): update dependency impit to ^0.13.0 (#3534)
516908a renovate[bot] committed 7 months ago
chore(deps): update actions/configure-pages action to v6 (#3535)
5637fd2 renovate[bot] committed 7 months ago
feat: add abortDownload context helper to FileDownload (#3528)
3b3d8f6 nikitachapovskii-dev committed 7 months ago
fix(e2e): resolve flaky Camoufox tests due to GitHub rate limits (#3527)
08c98cf Jindřich Bär committed 7 months ago
fix: Correctly track the number of requests handled by a crawler (#3410)
b2296ce Jan Buchar committed 7 months ago
chore(deps): bump yaml from 2.8.2 to 2.8.3 in /website (#3522)
abece11 dependabot[bot] committed 7 months ago
chore(deps): update actions/deploy-pages action to v5 (#3523)
364bf03 renovate[bot] committed 7 months ago
chore(deps): bump picomatch from 2.3.1 to 2.3.2 (#3520)
e5bb0c6 dependabot[bot] committed 7 months ago
chore(deps): bump picomatch from 2.3.1 to 2.3.2 in /website (#3521)
d8d0240 dependabot[bot] committed 7 months ago
chore(deps): update dependency impit to ^0.12.0 (#3519)
7831c28 renovate[bot] committed 7 months ago
chore(deps): bump flatted from 3.3.3 to 3.4.2 in /website (#3512)
21f58c4 dependabot[bot] committed 8 months ago
chore(deps): update endbug/add-and-commit action to v10 (#3514)
1cfd991 renovate[bot] committed 8 months ago
chore(deps): bump flatted from 3.3.3 to 3.4.2 (#3507)
484e7ac dependabot[bot] committed 8 months ago
chore(deps): update nick-fields/retry action to v4 (#3510)
d0dcb4c renovate[bot] committed 8 months ago
chore(deps): update dependency impit to ^0.11.0 (#3491)
3d19e67 renovate[bot] committed 8 months ago
chore(deps): update styfle/cancel-workflow-action action to v0.13.1 (#3490)
42679a5 renovate[bot] committed 8 months ago
chore(deps): bump dompurify from 3.3.1 to 3.3.3 in /website (#3488)
9b127bf dependabot[bot] committed 8 months ago
chore(deps): bump file-type from 21.3.1 to 21.3.2 (#3487)
3a1d903 dependabot[bot] committed 8 months ago
chore(deps): bump lodash from 4.17.21 to 4.17.23 (#3365)
0bf1452 dependabot[bot] committed 8 months ago
chore(deps): bump express-rate-limit from 8.2.1 to 8.3.0 (#3469)
78d4c46 dependabot[bot] committed 8 months ago
chore(deps): bump file-type from 20.5.0 to 21.3.1 (#3475)
37331c2 dependabot[bot] committed 8 months ago
chore(deps): bump tar from 7.5.9 to 7.5.11 in /website (#3477)
68f7a80 dependabot[bot] committed 8 months ago
chore(deps): bump undici from 7.20.0 to 7.24.0 in /website (#3485)
dd34603 dependabot[bot] committed 8 months ago
fix: correct failed requests delta count in periodic log message (#3482)
a098f42 Copilot committed 8 months ago