Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Added back the recycleDiskCache option to PuppeteerPool now that the support is coming even in headless Chrome
842d0d2 Jan Curn committed 7 years ago
Merge pull request #315 from apifytech/feature/better-injectfile
cbc94b6 Ondra Urban committed 7 years ago
Updating docs - replacing getValue('INPUT') with getInput()
5320362 Marek Trunkat committed 7 years ago
Removing chaiAsPromised initialization from tests, moving to _helper.js
d1127ed Marek Trunkat committed 7 years ago
Merge pull request #308 from apifytech/feature/aspool-functions
57b672f Ondra Urban committed 7 years ago