Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
add documentation comments and do some cosmetic refactoring
f18417b Ondrej Urban committed 8 years ago
implement minimal working solution of screenshot capturing
c5473a2 Ondrej Urban committed 8 years ago
Merge branch 'develop' of github.com:Apifier/apify-runtime-js into develop
9891c57 Marek Trunkat committed 8 years ago
Attempt to increase performance of RequestQueue - prefetching queue head of concurrency*3
ff6df73 Marek Trunkat committed 8 years ago
Merge branch 'develop' of https://github.com/apifytech/apify-js into develop
d1993f8 Jan Curn committed 8 years ago
Merge branch 'develop' of github.com:Apifier/apify-runtime-js into develop
db6940c Marek Trunkat committed 8 years ago