Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
chore(deps): update dependency @docusaurus/faster to v3.6.2
9e29048 renovate[bot] committed 2 years ago
feat: `tieredProxyUrls` accept `null` for switching the proxy off (#2743)
82f4ea9 Jindřich Bär committed 2 years ago
chore(deps): update dependency @docusaurus/faster to v3.6.1
1e59f1b renovate[bot] committed 2 years ago