Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Fixed RequestQueue.addRequest() to gracefully handle invalid URLs
ec1ca22 Jan Curn committed 8 years ago
Merge pull request #210 from apifytech/feature/pusherror-improvement
28cf6cf Ondra Urban committed 8 years ago
Merge pull request #208 from apifytech/feature/actor-ui-inputs
c27699a Ondra Urban committed 8 years ago