Skip to content

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

Update excludeTags.ts

Nicolasff906f7750cf
diff --git a/apps/api/src/scraper/WebScraper/utils/excludeTags.ts b/apps/api/src/scraper/WebScraper/utils/excludeTags.ts
index b813c813d..71e1637bb 100644
--- a/apps/api/src/scraper/WebScraper/utils/excludeTags.ts
+++ b/apps/api/src/scraper/WebScraper/utils/excludeTags.ts
@@ -3,7 +3,6 @@ export const excludeNonMainTags = [
"footer",
"nav",
"aside",
- ".header",
".top",
".navbar",
".footer",