Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
chore: define puppeteer and playwright as optional peer dependencies (#1056)
334e540 Martin Adámek committed 5 years ago
chore: add development support on windows and tests to CI (#868)
90b0f55 Ondra Urban committed 5 years ago
fix: proxyUrl without port throwing on browser launch (#1065)
ea10095 Ondra Urban committed 5 years ago
chore: bump puppeteer and playwright to get new browsers (#1044)
301819d Ondra Urban committed 5 years ago
feat: introduce new `Apify` and `Configuration` classes (#993)
d44f3b5 Martin Adámek committed 5 years ago
chore: update playwright, puppeteer and some other dependencies (#975)
99a6774 Martin Adámek committed 5 years ago