Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
TypeScript aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
add scrape jobs on separate connections
Gergő Móriczb12c175b012a
diff --git a/apps/api/src/services/queue-jobs.ts b/apps/api/src/services/queue-jobs.tsindex ba05625fd..d2e00074e 100644--- a/apps/api/src/services/queue-jobs.ts+++ b/apps/api/src/services/queue-jobs.ts@@ -68,11 +68,13 @@ export async function _addScrapeJobToBullMQ(}}- await getScrapeQueue().add(jobId, webScraperOptions, {+ const conn = createRedisConnection();+ await getScrapeQueue(conn).add(jobId, webScraperOptions, {...options,priority: jobPriority,jobId,});+ conn.disconnect();}async function addScrapeJobRaw(