Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
chore(deps): update peter-evans/repository-dispatch action to v3 (#2304)
5cbce5a renovate[bot] committed 2 years ago
chore(deps): update styfle/cancel-workflow-action action to v0.12.1 (#2305)
70c7400 renovate[bot] committed 2 years ago
refactor(browser-pool): drop const modifier for BrowserName enum (#2289)
217a9ad Peïo Thibault committed 2 years ago
chore: more debug logs for checking enqueue links strategy (#2293)
ff67456 Vlad Frangu committed 2 years ago
ci: add in option to trigger full redeploy of current images (#2288)
61843c0 Vlad Frangu committed 2 years ago
docs: disable selection of API docs pills (e.g. for `optional` props)
d7a7c2f Martin Adámek committed 2 years ago