Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Update check-user-agent.js
Ondra Urban1b50159137c1
diff --git a/.travis/check-user-agent.js b/.travis/check-user-agent.jsindex 2a158e7d4..11d79a30b 100644--- a/.travis/check-user-agent.js+++ b/.travis/check-user-agent.js@@ -19,7 +19,7 @@ async function main() {process.exit(0);} else {console.warn('Default User-Agent has incorrect version of Chrome!');- console.warn('This can cause more blocking in head-full mode.');+ console.warn('This can cause more blocking in headful mode.');console.warn(`Default User-Agent: ${defaultUserAgentVersion} X Latest Chrome: ${latestStableVersion} `);process.exit(1);}