Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Enforce puppeteer version for security and fix of deployment
df1b7cd Ondra Urban committed 7 years ago
added limit for iterators and error handling for too large input.
294db10 Petr Patek committed 7 years ago
Remove bluebird dependecies and fix autoscaled pool tests on Node 12
febcf24 Ondra Urban committed 7 years ago
Merge pull request #501 from apifytech/feature/fix_content_type_parsing
7125d0a Ondra Urban committed 7 years ago
Merge branch 'master' into feature/fix_content_type_parsing
f802368 Ondra Urban committed 7 years ago