Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Adding function to create standardised debug info from request and response.
a271d8d Marek Trunkat committed 7 years ago
Merge pull request #284 from apifytech/fix/launch-pup-error
4f4da8b Ondra Urban committed 7 years ago
Merge pull request #286 from apifytech/feature/cheerio-enqueuelinks
2f81e42 Ondra Urban committed 7 years ago
Better error for invalid launchPuppeteerFunction in PuppeteerPool
851e396 Jan Curn committed 7 years ago