Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

chore: add logging to a flaky test

Ondra Urban9fbda1f15d8d
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index 1f07305f3..ad09da60c 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -77,13 +77,8 @@ jobs:
node-version: 14
registry-url: https://registry.npmjs.org/
-
- name: Load Cache
- uses: actions/cache@v2
- with:
- path: |
- node_modules
- build
- key: cache-${{ github.run_id }}-v14
+ name: Install Dependencies
+ run: npm install
-
# Determine if this is a beta or latest release
name: Set Release Tag
diff --git a/test/crawlers/cheerio_crawler.test.js b/test/crawlers/cheerio_crawler.test.js
index 59a00f907..b09321ffd 100644
--- a/test/crawlers/cheerio_crawler.test.js
+++ b/test/crawlers/cheerio_crawler.test.js
@@ -702,8 +702,15 @@ describe('CheerioCrawler', () => {
const { sessions } = cheerioCrawler.sessionPool;
expect(sessions.length).toBe(4);
sessions.forEach((session) => {
+ // TODO this test is flaky in CI and we need some more info to debug why.
+ if (session.errorScore !== 1) {
+ console.log('SESSIONS:');
+ console.dir(sessions);
+ }
+
expect(session.errorScore).toEqual(1);
});
+
log.setLevel(log.LEVELS.ERROR);
});