Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Added more meaningful error message for Apify.setValue() for large objects
3c468a0 Jan Curn committed 8 years ago
Merge pull request #16 from Apifier/feature/AP-245-pushRecord
45fb615 Marek Trunkát committed 8 years ago
Upgraded to beta apify-client pacakage, fixed lint errors after upgrade of dev packages, minor improvements, bumped package.json version
c052121 Jan Curn committed 9 years ago
Updated publish.sh to allow beta deploy from other branches, enabled aws cloudfront invalidation
9b32347 Jan Curn committed 9 years ago