Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Fixing RequestQueue to better handle DynamoDB data inconsistency
3eb514b Jan Curn committed 7 years ago
Fixing RequestQueue to better handle DynamoDB data inconsistency
bc0194a Jan Curn committed 7 years ago
Merge branch 'develop' of github.com:apifytech/apify-js into develop
5739621 Ondra Urban committed 7 years ago
Merge pull request #388 from apifytech/feature/faster-live-view
c35344a Ondra Urban committed 7 years ago
Merge pull request #386 from apifytech/feature/missing_dataset_getData_options
518acff Ondra Urban committed 7 years ago
Merge pull request #384 from apifytech/feature/avoid-request-interception
2df4121 Ondra Urban committed 7 years ago
Merge pull request #380 from apifytech/feature/crawler-stats
e482739 Ondra Urban committed 7 years ago
Added missing simplified and skipFailedPages to dataset getData method
a9a8bb4 drobnikj committed 7 years ago