Skip to content

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

fix(scraper): also reject application/octet-stream non-PDF binaries

firecrawl-spring[bot]d3c6f6f66403
diff --git a/apps/api/src/scraper/scrapeURL/engines/utils/specialtyHandler.ts b/apps/api/src/scraper/scrapeURL/engines/utils/specialtyHandler.ts
index 3e31e0a72..fa321551d 100644
--- a/apps/api/src/scraper/scrapeURL/engines/utils/specialtyHandler.ts
+++ b/apps/api/src/scraper/scrapeURL/engines/utils/specialtyHandler.ts
@@ -170,6 +170,7 @@ export async function specialtyScrapeCheck(
"application/x-executable",
"application/x-sharedlib",
"application/java-archive",
+ "application/octet-stream",
];
if (
unsupportedBinaryPrefixes.some(prefix => contentType.startsWith(prefix))