Skip to content

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

feat(scrapeURL/pdf): extend amount of time we're willing to wait for PDFs in crawl/batch scrape mode

Gergő Móriczecad76978d42
diff --git a/apps/api/src/scraper/scrapeURL/engines/pdf/index.ts b/apps/api/src/scraper/scrapeURL/engines/pdf/index.ts
index bdc916e08..19f0a7a87 100644
--- a/apps/api/src/scraper/scrapeURL/engines/pdf/index.ts
+++ b/apps/api/src/scraper/scrapeURL/engines/pdf/index.ts
@@ -62,7 +62,7 @@ async function scrapePDFWithLlamaParse(meta: Meta, tempFilePath: string): Promis
schema: z.object({
markdown: z.string(),
}),
- tryCount: 32,
+ tryCount: meta.options.timeout !== undefined ? 32 : 1200, // 5 minutes if timeout not specified
tryCooldown: 250,
});