Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
docs: Fix timeouts and retry limit exceeded in interim introduction example (#2791)
1ea95f7 Angie Byron committed 1 year ago
chore(deps): update dependency @docusaurus/faster to v3.7.0
f912b8b renovate[bot] committed 1 year ago
fix(core): type definition of Dataset.reduce (#2774)
59bc6d1 Erik Funder Carstensen committed 1 year ago
chore: check TS in the tests and enable `strictNullChecks` in there (#2776)
fb84d69 Martin Adámek committed 1 year ago