Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
TypeScript aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
debug(scrapeURL/ab): use a huge maxAge
Gergő Móricz8498a18c2c6d
diff --git a/apps/api/src/scraper/scrapeURL/index.ts b/apps/api/src/scraper/scrapeURL/index.tsindex a1a7c4e81..be4f0310e 100644--- a/apps/api/src/scraper/scrapeURL/index.ts+++ b/apps/api/src/scraper/scrapeURL/index.ts@@ -616,6 +616,7 @@ export async function scrapeURL(...meta.options,origin: (meta.options as any).origin ?? "api",timeout: meta.options.timeout ?? 30000,+ maxAge: 1000000000,},logger: abLogger,tryCount: 1,