Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
chore(deps): update actions/upload-pages-artifact action to v4 (#3139)
872e052 renovate[bot] committed 1 year ago
chore(deps): update actions/checkout action to v5 (#3128)
948810e renovate[bot] committed 1 year ago
chore(deps): update dependency iconv-lite to ^0.7.0 (#3137)
f5ca450 renovate[bot] committed 1 year ago
chore(deps): update dependency camoufox-js to ^0.7.0 (#3134)
8b9ab6c renovate[bot] committed 1 year ago
chore(deps): update amannn/action-semantic-pull-request action to v6 (#3131)
9c08cc4 renovate[bot] committed 1 year ago
chore(deps): update dependency tough-cookie to v6 (#3133)
57a7051 renovate[bot] committed 1 year ago
feat: add `collectAllKeys` option for `BasicCrawler.exportData` (#3129)
2ddfc9c Jindřich Bär committed 1 year ago
fix: don't fail `exportData` calls on empty datasets (#3115)
298f170 MirzaSamadAhmedBaig committed 1 year ago
fix: `ImpitHttpClient` respects the internal `Request` timeout (#3103)
a35376d Jindřich Bär committed 1 year ago
chore(release): update internal dependencies [skip ci]
a59c014 Apify Release Bot committed 1 year ago
chore(release): update docs for minor version [skip ci]
9e594db Apify Release Bot committed 1 year ago
chore(release): update internal dependencies [skip ci]
c53f314 Apify Release Bot committed 1 year ago
chore: fix e2e tests after the documentation cleanup (#3069)
6a7248c Jindřich Bär committed 1 year ago
fix: validation of iterables when adding requests to the queue (#3091)
529a1dd Jan Buchar committed 1 year ago
chore(deps): bump form-data from 4.0.3 to 4.0.4 (#3067)
de8fada dependabot[bot] committed 1 year ago
chore(deps): bump axios from 1.10.0 to 1.11.0 in /website (#3093)
9195b2b dependabot[bot] committed 1 year ago