Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
chore(release): update internal dependencies [skip ci]
32d97a6 Apify Release Bot committed 2 years ago
fix: use namespace imports for cheerio to be compatible with v1 (#2641)
f48296f Martin Adámek committed 2 years ago
chore(deps): update dependency stream-chain to v3 (#2628)
fa6bb09 renovate[bot] committed 2 years ago
feat: `globs` & `regexps` for `SitemapRequestList` (#2631)
b5fd3a9 Jindřich Bär committed 2 years ago
docs: remove duplicated `the` in AutoscaledPool (#2624)
d9a3877 Ikko Eltociear Ashimine committed 2 years ago
fix: Use the correct mutex in memory storage RequestQueueClient (#2623)
2fa8a29 Jan Buchar committed 2 years ago
chore(deps): update dependency axios to v1.7.4 [security]
a5f7ce0 renovate[bot] committed 2 years ago