Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

fix(core): clear `AutoscaledPool.pause()` polling interval on timeout (#3847)

Anas Khan42b89146a372
diff --git a/packages/core/src/autoscaling/autoscaled_pool.ts b/packages/core/src/autoscaling/autoscaled_pool.ts
index 42f7e42e2..45e328a8e 100644
--- a/packages/core/src/autoscaling/autoscaled_pool.ts
+++ b/packages/core/src/autoscaling/autoscaled_pool.ts
@@ -430,8 +430,11 @@ export class AutoscaledPool {
this.isStopped = true;
await new Promise<void>((resolve, reject) => {
let timeout: NodeJS.Timeout;
+ let interval: NodeJS.Timeout;
if (timeoutSecs) {
timeout = setTimeout(() => {
+ // Clean up the polling interval to prevent it from leaking on timeout.
+ clearInterval(interval);
const err = new Error(
"The pool's running tasks did not finish" +
`in ${timeoutSecs} secs after pool.pause() invocation.`,
@@ -440,7 +443,7 @@ export class AutoscaledPool {
}, timeoutSecs);
}
- const interval = setInterval(() => {
+ interval = setInterval(() => {
if (this._currentConcurrency <= 0) {
// Clean up timeout and interval to prevent process hanging.
if (timeout) clearTimeout(timeout);