Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
TypeScript aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
Update excludeTags.ts
Nicolasff906f7750cf
diff --git a/apps/api/src/scraper/WebScraper/utils/excludeTags.ts b/apps/api/src/scraper/WebScraper/utils/excludeTags.tsindex b813c813d..71e1637bb 100644--- a/apps/api/src/scraper/WebScraper/utils/excludeTags.ts+++ b/apps/api/src/scraper/WebScraper/utils/excludeTags.ts@@ -3,7 +3,6 @@ export const excludeNonMainTags = ["footer","nav","aside",- ".header",".top",".navbar",".footer",