Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
added getPublicUrl feature to key_value_store - fixed linting errors
fe02857 aarontjdev committed 7 years ago
Merge pull request #294 from novotnyj/fix/kv-store-forEachKey
08cab1e Jan Čurn committed 7 years ago
Merge pull request #285 from apifytech/feature/log_version
e518759 Ondra Urban committed 7 years ago
Merge pull request #290 from apifytech/feature/enqueueLinks-baseUrl
7de5dc5 Ondra Urban committed 7 years ago