Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

chore: fix timeouts for `waitForSelector` in playwright

Martin Adámek53ce1d37bb56
diff --git a/packages/jsdom-crawler/src/internals/jsdom-crawler.ts b/packages/jsdom-crawler/src/internals/jsdom-crawler.ts
index 46d7425ce..bd0404826 100644
--- a/packages/jsdom-crawler/src/internals/jsdom-crawler.ts
+++ b/packages/jsdom-crawler/src/internals/jsdom-crawler.ts
@@ -60,7 +60,7 @@ export interface JSDOMCrawlingContext<
document: Document;
/**
- * Wait for an element matching the selector to appear. Timeout is ignored.
+ * Wait for an element matching the selector to appear.
* Timeout defaults to 5s.
*
* **Example usage:**
diff --git a/packages/linkedom-crawler/src/internals/linkedom-crawler.ts b/packages/linkedom-crawler/src/internals/linkedom-crawler.ts
index ddb73bcad..b9048eaa7 100644
--- a/packages/linkedom-crawler/src/internals/linkedom-crawler.ts
+++ b/packages/linkedom-crawler/src/internals/linkedom-crawler.ts
@@ -55,7 +55,7 @@ export interface LinkeDOMCrawlingContext<
document: Document;
/**
- * Wait for an element matching the selector to appear. Timeout is ignored.
+ * Wait for an element matching the selector to appear.
* Timeout defaults to 5s.
*
* **Example usage:**
diff --git a/packages/playwright-crawler/src/internals/utils/playwright-utils.ts b/packages/playwright-crawler/src/internals/utils/playwright-utils.ts
index 399cbd9a0..ac70cfcd0 100644
--- a/packages/playwright-crawler/src/internals/utils/playwright-utils.ts
+++ b/packages/playwright-crawler/src/internals/utils/playwright-utils.ts
@@ -827,11 +827,11 @@ export function registerUtilsToContext(
await injectJQuery(context.page, { surviveNavigations: false });
};
context.blockRequests = async (options?: BlockRequestsOptions) => blockRequests(context.page, options);
- context.waitForSelector = async (selector: string, timeoutMs?: number) => {
+ context.waitForSelector = async (selector: string, timeoutMs = 5_000) => {
const locator = context.page.locator(selector).first();
await locator.waitFor({ timeout: timeoutMs, state: 'attached' });
};
- context.parseWithCheerio = async (selector?: string, timeoutMs?: number) => {
+ context.parseWithCheerio = async (selector?: string, timeoutMs = 5_000) => {
if (selector) {
await context.waitForSelector(selector, timeoutMs);
}