Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
docs: remove CNAME file to disable redirect from apify.github.io
0bfbc49 Martin Adámek committed 2 years ago
feat: add `ignoreIframes` opt-out from the Cheerio iframe expansion (#2562)
474a8dc Jindřich Bär committed 2 years ago
chore(deps): update oven-sh/setup-bun action to v2 (#2551)
ab7c413 renovate[bot] committed 2 years ago
feat: add `iframe` expansion to `parseWithCheerio` in browsers (#2542)
328d085 Jindřich Bär committed 2 years ago