Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Added yarn.lock, added Apifier.heyIAmReady() function and unit tests
b85d216 Jan Curn committed 9 years ago
Added "prebulishOnly" event to package.json to ensure scripts are build and tested
7e0c3bc Jan Curn committed 9 years ago
Updated .gitignore, added new dev dependencies to package.json to fix NPM warning
e4e35e4 Jan Curn committed 9 years ago