Skip to content

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

fix(scrapeURL/loop): re-add is long enough check with lt 0

Gergő Móricz97bf54214fa8
diff --git a/apps/api/src/scraper/scrapeURL/index.ts b/apps/api/src/scraper/scrapeURL/index.ts
index f29d4de04..cae049c4b 100644
--- a/apps/api/src/scraper/scrapeURL/index.ts
+++ b/apps/api/src/scraper/scrapeURL/index.ts
@@ -225,8 +225,7 @@ async function scrapeURLLoop(meta: Meta): Promise<ScrapeUrlResponse> {
};
// Success factors
- // const isLongEnough = engineResult.markdown.length >= 20;
- const isLongEnough = true; // just assume first engine is good
+ const isLongEnough = engineResult.markdown.length > 0;
const isGoodStatusCode =
(engineResult.statusCode >= 200 && engineResult.statusCode < 300) ||
engineResult.statusCode === 304;