Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
added process exit to end.
Petr Patek26b073215973
diff --git a/.travis/check-user-agent.js b/.travis/check-user-agent.jsindex b8885e4d0..2a158e7d4 100644--- a/.travis/check-user-agent.js+++ b/.travis/check-user-agent.js@@ -30,4 +30,5 @@ try {} catch (e) {console.log('Could not check User-Agent');console.error(e);+ process.exit(1)}