Skip to content

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

feat(scrapeURL/index): add debug logging to document 404s

Gergő Móricz454a0716f201
diff --git a/apps/api/src/scraper/scrapeURL/engines/index/index.ts b/apps/api/src/scraper/scrapeURL/engines/index/index.ts
index c1e22fda7..b104a8d91 100644
--- a/apps/api/src/scraper/scrapeURL/engines/index/index.ts
+++ b/apps/api/src/scraper/scrapeURL/engines/index/index.ts
@@ -306,6 +306,9 @@ export async function scrapeURLWithIndex(
meta.logger.child({ module: "index", method: "getIndexFromGCS" }),
);
if (!doc) {
+ meta.logger.warn("Index document not found in GCS", {
+ indexDocumentId: id,
+ });
throw new EngineError("Document not found in GCS");
}