Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Remove Node 8 from Travis

Ondra Urban3470d231f4a6
diff --git a/.travis.yml b/.travis.yml
index 2ee9656a1..ec267debd 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -5,7 +5,6 @@ if: (tag IS blank) OR tag =~ /^v\d+\.\d+\.\d+$/
node_js:
- '12'
- '10'
-- '8'
cache:
directories:
- "$HOME/.npm"
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 44d7d3951..eaef4ad25 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,7 +5,7 @@ NEXT
- Add default `SessionPool` implementation to `BasicCrawler`.
- Add the ability to create ad-hoc webhooks via `Apify.call()` and `Apify.callTask()`.
- Add an example of form filling with `Puppeteer`.
-- Add `country` option to `Apify.getProxyUrl()`.
+- Add `country` option to `Apify.getApifyProxyUrl()`.
- Add `Apify.utils.puppeteer.saveSnapshot()` helper to quickly save HTML and screenshot of a page.
- Add the ability to pass `got` supported options to `requestOptions` in `CheerioCrawler`
thus supporting things such as `cookieJar` again.
diff --git a/test/crawlers/cheerio_crawler.test.js b/test/crawlers/cheerio_crawler.test.js
index 927b5d59c..588b041eb 100644
--- a/test/crawlers/cheerio_crawler.test.js
+++ b/test/crawlers/cheerio_crawler.test.js
@@ -701,7 +701,7 @@ describe('CheerioCrawler', () => {
const opts = crawler._getRequestOptions(request);
proxies.push(opts.proxyUrl);
// it needs to return something valid
- return { dom: {}, responseStream: responseMock };
+ return { dom: {}, response: responseMock };
};
await crawler.run();
@@ -739,7 +739,7 @@ describe('CheerioCrawler', () => {
const opts = crawler._getRequestOptions(request);
proxies.push(opts.proxyUrl);
// it needs to return something valid
- return { dom: {}, responseStream: responseMock };
+ return { dom: {}, response: responseMock };
};
await crawler.run();