Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Adding `--disable-background-timer-throttling` and `--enable-resource-load-scheduler=false` Chrome flags to launchPuppeteer()
7730652 Jan Curn committed 8 years ago
Bugfix: Dataset.getInfo() sometimes returned an object with `itemsCount` field instead of `itemCount`
0786cba Jan Curn committed 8 years ago
Merge branch 'develop' of github.com:Apifier/apify-runtime-js into develop
b4c6033 Marek Trunkat committed 8 years ago
Merge pull request #222 from apifytech/feature/html-only-cheerio
3cf8efb Ondra Urban committed 8 years ago
Merge pull request #226 from apifytech/feature/tunnel-agent-fix
d3c4c05 Ondra Urban committed 8 years ago
Merge branch 'develop' of github.com:Apifier/apify-runtime-js into develop
68fb470 Marek Trunkat committed 8 years ago