Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Merge pull request #221 from apifytech/feature/call_fixes
ab87854 Marek Trunkát committed 8 years ago
Merge pull request #225 from apifytech/fix/enqueue-links
ea4ad3a Marek Trunkát committed 8 years ago
Merge pull request #218 from apifytech/feature/enforce_cloud
21b62a9 Marek Trunkát committed 8 years ago
Fixed Apify.utils.puppeteer.enqueueLinks to allow null value for pseudoUrls param
396574d Jan Curn committed 8 years ago
Merge pull request #216 from apifytech/feature/puppeteer_error
d371973 Marek Trunkát committed 8 years ago
Merge pull request #215 from apifytech/feature/handledCount
ff8c749 Marek Trunkát committed 8 years ago
Merge pull request #217 from apifytech/feature/copywriting
94a04f1 Marek Trunkát committed 8 years ago
Merge pull request #219 from apifytech/feature/setMinMaxConcurrency
1fb1486 Marek Trunkát committed 8 years ago