Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
chore(release): update internal dependencies [skip ci]
1bb9570 Apify Release Bot committed 2 years ago
feat(sitemap): Support CDATA in sitemaps (#2424)
635f046 Pavel 'Strajk' Dolecek committed 2 years ago
fix: break up growing stack in `AutoscaledPool.notify` (#2422)
6f2e6b0 Jindřich Bär committed 2 years ago
chore(release): update internal dependencies [skip ci]
c9bd1d9 Apify Release Bot committed 2 years ago
chore(release): update internal dependencies [skip ci]
a1bfb6e Apify Release Bot committed 2 years ago
fix: include actual key in error message of KVS' `setValue` (#2411)
9089bf1 Jiří Moravčík committed 2 years ago