Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
TypeScript aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
fix(nuq-postgres): reap orphaned NULL times_out_at backlog rows (#3413)
firecrawl-spring[bot]436d2a55332e
diff --git a/apps/nuq-postgres/nuq.sql b/apps/nuq-postgres/nuq.sqlindex 1e8fb60e9..f1691c910 100644--- a/apps/nuq-postgres/nuq.sql+++ b/apps/nuq-postgres/nuq.sql@@ -120,8 +120,13 @@ SELECT cron.schedule('nuq_queue_scrape_lock_reaper', '15 seconds', $$$$);SELECT cron.schedule('nuq_queue_scrape_backlog_reaper', '* * * * *', $$+ -- Also sweep orphaned rows with NULL times_out_at older than the crawl+ -- Redis TTL window, so a missing timeout on a backlog row can't wedge+ -- nuq_group_crawl_finished forever via its NOT EXISTS check on backlog.DELETE FROM nuq.queue_scrape_backlog- WHERE nuq.queue_scrape_backlog.times_out_at < now();+ WHERE nuq.queue_scrape_backlog.times_out_at < now()+ OR (nuq.queue_scrape_backlog.times_out_at IS NULL+ AND nuq.queue_scrape_backlog.created_at < now() - interval '25 hours');$$);SELECT cron.schedule('nuq_queue_scrape_reindex', '0 9 * * *', $$