Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

feat: log desired concurrency in the default status message

Martin Adámek9f0b79684d9e
diff --git a/packages/basic-crawler/src/internals/basic-crawler.ts b/packages/basic-crawler/src/internals/basic-crawler.ts
index a8acc75b1..1cd5cefa6 100644
--- a/packages/basic-crawler/src/internals/basic-crawler.ts
+++ b/packages/basic-crawler/src/internals/basic-crawler.ts
@@ -824,7 +824,7 @@ export class BasicCrawler<Context extends CrawlingContext = BasicCrawlingContext
const total = this.requestQueue?.getTotalCount() || this.requestList?.length();
message = `Crawled ${this.stats.state.requestsFinished}${total ? `/${total}` : ''} pages, ${
this.stats.state.requestsFailed
- } failed requests.`;
+ } failed requests, desired concurrency ${this.autoscaledPool?.desiredConcurrency ?? 0}.`;
}
if (this.statusMessageCallback) {