Skip to content

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

Update blocklist.ts

Nicolas064ce482c23c
diff --git a/apps/api/src/scraper/WebScraper/utils/blocklist.ts b/apps/api/src/scraper/WebScraper/utils/blocklist.ts
index 1b75a0715..3ed97d0b8 100644
--- a/apps/api/src/scraper/WebScraper/utils/blocklist.ts
+++ b/apps/api/src/scraper/WebScraper/utils/blocklist.ts
@@ -40,6 +40,7 @@ const allowedKeywords = [
'tos',
'://ads.tiktok.com',
'://tiktok.com/business',
+ '://developers.facebook.com'
];
export function isUrlBlocked(url: string): boolean {