Skip to content

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

fix(queue-worker): proper concurrency

Gergő Móriczcf9010a1268e
diff --git a/apps/api/src/services/queue-worker.ts b/apps/api/src/services/queue-worker.ts
index 9524d37b5..20b42a524 100644
--- a/apps/api/src/services/queue-worker.ts
+++ b/apps/api/src/services/queue-worker.ts
@@ -338,7 +338,7 @@ const separateWorkerFun = (
lockDuration: 30 * 1000, // 30 seconds
stalledInterval: 30 * 1000, // 30 seconds
maxStalledCount: 10, // 10 times
- concurrency: 6, // from k8s setup
+ concurrency: Math.ceil(6 / scrapeQueueNames.length), // from k8s setup
useWorkerThreads: true,
});