Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
TypeScript aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
nerf base worker rate of job polling
Gergő Móricza9063909fd4e
diff --git a/apps/api/src/services/worker/nuq-worker.ts b/apps/api/src/services/worker/nuq-worker.tsindex 547c442b1..2bf4a3f59 100644--- a/apps/api/src/services/worker/nuq-worker.ts+++ b/apps/api/src/services/worker/nuq-worker.ts@@ -43,7 +43,7 @@ import { initializeBlocklist } from "../../scraper/WebScraper/utils/blocklist";process.on("SIGINT", shutdown);process.on("SIGTERM", shutdown);- let noJobTimeout = 500;+ let noJobTimeout = 1500;while (!isShuttingDown) {const job = await scrapeQueue.getJobToProcess();