Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Merge pull request #461 from apifytech/puppeteer-double-log-fix
363de05 Jan Čurn committed 7 years ago
Merge pull request #457 from apifytech/fix/http-request-version
e9a2cf7 petrpatek committed 7 years ago
increased http request version to fix decompress stream error handling.
53b03ae Petr Patek committed 7 years ago
Merge pull request #449 from apifytech/feature/incognito-pages
58fefc7 Ondra Urban committed 7 years ago
Merge pull request #435 from apifytech/feature/petr-cheerio-crawler-request-deprecation
bbf88db Ondra Urban committed 7 years ago