Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Merge pull request #402 from apifytech/feature/block_requests_minor_changes
c6a3d76 Ondra Urban committed 7 years ago
Merge pull request #399 from apifytech/feature/better-errors
705349a Ondra Urban committed 7 years ago
Merge pull request #395 from apifytech/feature/undeprecated-handled-count
1c7342b Ondra Urban committed 7 years ago
Merge branch 'master' into feature/block_requests_minor_changes
0116e43 Ondra Urban committed 7 years ago
Merge pull request #374 from apifytech/feature/element_clicking
810a6ab Ondra Urban committed 7 years ago
Merge pull request #404 from apifytech/feature/continuous-deployment
bd05584 Ondra Urban committed 7 years ago