Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
TypeScript aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
Update search.ts
Nicolasad49503f8aba
diff --git a/apps/api/src/controllers/v1/search.ts b/apps/api/src/controllers/v1/search.tsindex 7d7bf9d3a..1d8c59eb6 100644--- a/apps/api/src/controllers/v1/search.ts+++ b/apps/api/src/controllers/v1/search.ts@@ -73,13 +73,17 @@ async function scrapeSearchResult(teamId: options.teamId,});+ let statusCode = 0;+ if (error.message.includes("Could not scrape url")) {+ statusCode = 403;+ }// Return a minimal document with SERP results at top levelreturn {title: searchResult.title,description: searchResult.description,url: searchResult.url,metadata: {- statusCode: 0,+ statusCode,error: error.message,},};