Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Merge pull request #371 from apifytech/feature/streaming-cheerio
69c21f5 Ondra Urban committed 7 years ago
Merge pull request #370 from apifytech/chore/remove-bluebird
8ab5f8d Ondra Urban committed 7 years ago
Merge pull request #359 from apifytech/feature/hiding-tricks-puppeteer
8d4fbf6 Ondra Urban committed 7 years ago
Merge branch 'feature/hiding-tricks-puppeteer' of github.com:apifytech/apify-js into feature/hiding-tricks-puppeteer
5fc1038 Petr Patek committed 7 years ago
Request queue - better logs and comments, minor code improvements
fd75758 Jan Curn committed 7 years ago
Using client information returned by queue api to finish quickly in a case of a single client
1ae5c38 Marek Trunkat committed 7 years ago