Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Development realted to Apify.utils.htmlToText(), adding tests
68cb0ab Jan Curn committed 7 years ago
Added new Apify.utils.htmlToText() function to improve Apify.utils.social.parseHandlesFromHtml()
4c0e5e3 Jan Curn committed 7 years ago
Merge branch 'feature/better-migration' into fix/cheerio-leak
7b2c3be Ondra Urban committed 7 years ago
Merge branch 'feature/request-list-persist' into fix/cheerio-leak
33004ad Ondra Urban committed 7 years ago
Merge pull request #264 from apifytech/fix/apify-call-error
06687f6 Ondra Urban committed 7 years ago
Merge pull request #262 from apifytech/feature/cheerio-compression
aad1730 Ondra Urban committed 7 years ago
Merge pull request #259 from apifytech/feature/log-crawler-finish
252b2cc Ondra Urban committed 7 years ago