Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
chore: try to test on node 19 (#1606)
Martin Adámek4cbec981ba51
diff --git a/.github/workflows/test-ci.yml b/.github/workflows/test-ci.ymlindex 40abf10b0..bbafe00c9 100644--- a/.github/workflows/test-ci.yml+++ b/.github/workflows/test-ci.yml@@ -19,7 +19,7 @@ jobs:# tests on windows are extremely unstable# os: [ ubuntu-latest, windows-2019 ]os: [ ubuntu-latest ]- node-version: [ 16, 18 ]+ node-version: [ 16, 18, 19 ]steps:- name: Cancel Workflow Actiondiff --git a/test/core/crawlers/basic_crawler.test.ts b/test/core/crawlers/basic_crawler.test.tsindex bf475c333..b44b4e5fa 100644--- a/test/core/crawlers/basic_crawler.test.ts+++ b/test/core/crawlers/basic_crawler.test.ts@@ -646,7 +646,7 @@ describe('BasicCrawler', () => {setTimeout(() => queue.push(request0), 10);setTimeout(() => queue.push(request1), 100);- setTimeout(() => { void basicCrawler.teardown(); }, 150);+ setTimeout(() => { void basicCrawler.teardown(); }, 300);await basicCrawler.run();