Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Disabling autoscaling when running out of Apify platform (#92)
3229016 Marek Trunkát committed 8 years ago
Merge branch 'develop' of github.com:Apifier/apify-runtime-js into develop
32cdcb0 Marek Trunkat committed 8 years ago
Merge branch 'develop' of github.com:Apifier/apify-runtime-js into develop
32cb76a Marek Trunkat committed 8 years ago
Not printing 'WebSocket was closed before the connection was established' error (#83)
7c8b61c Marek Trunkát committed 8 years ago
Fixed Apify.getApifyProxyUrl() to use proper regex for checks (#84)
237ce8d Jan Čurn committed 8 years ago
Improved comments for Apify proxy, updated keywords in package.json
810105a Jan Curn committed 8 years ago