Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Merge pull request #347 from apifytech/fix/snapshotter-rate-limit
9477f62 Ondra Urban committed 7 years ago
Merge branch 'develop' of github.com:apifytech/apify-js into develop
5e3d1af Ondra Urban committed 7 years ago
Merge pull request #327 from apifytech/feature/recycle-disk-cache
34644e8 Ondra Urban committed 7 years ago
Merge pull request #337 from apifytech/feature/petr_before_request
9930e7e Ondra Urban committed 7 years ago
Merge pull request #335 from apifytech/fix/add-puppeteer-timeouts
f98d15e Ondra Urban committed 7 years ago