Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Added Squid install to .travis.yml
Jan Curn0ddfa11704ba
diff --git a/.travis.yml b/.travis.ymlindex 3d61d0014..f531ec153 100644--- a/.travis.yml+++ b/.travis.yml@@ -3,6 +3,10 @@ sudo: requiredaddons:chrome: betabefore_install:+before_install:+ - # Install Squid+ - sudo apt-get -qq update+ - sudo apt-get install -y squid- # Install ChromeDriver for Selenium- sudo curl -sS http://chromedriver.storage.googleapis.com/2.29/chromedriver_linux64.zip > /tmp/chromedriver.zip- sudo unzip /tmp/chromedriver.zip -d /tmp