Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
chore(release): update internal dependencies [skip ci]
1292840 Apify Release Bot committed 2 years ago
docs: add note about webinar to the docs and readme (#2506)
384ee4a Saurav Jain committed 2 years ago
fix: improve fix for double extension in KVS with HTML files (#2505)
157927d Martin Adámek committed 2 years ago
docs: improve documentation around `addRequests` and `addRequestsBatched`
522c44b Martin Adámek committed 2 years ago
chore(release): update internal dependencies [skip ci]
5333e81 Apify Release Bot committed 2 years ago