Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Merge branch 'develop' into feature/avoid-request-interception
194efc5 Ondra Urban committed 7 years ago
Merge pull request #379 from apifytech/feauter/better-autoscale-logging
5a70776 Ondra Urban committed 7 years ago
Merge branch 'develop' into feauter/better-autoscale-logging
c5c3585 Ondra Urban committed 7 years ago
Merge pull request #378 from apifytech/feature/use-systeminfo-in-autoscaling
7573e91 Ondra Urban committed 7 years ago
Merge branch 'develop' into feature/use-systeminfo-in-autoscaling
bef53f5 Ondra Urban committed 7 years ago
Merge branch 'develop' into feature/use-systeminfo-in-autoscaling
22a9f1e Ondra Urban committed 7 years ago