Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Update gotoFunction docs [skip ci]
Ondra Urban2826c7e08d2e
diff --git a/src/crawlers/puppeteer_crawler.js b/src/crawlers/puppeteer_crawler.jsindex 716ec8cbb..e08bf3098 100644--- a/src/crawlers/puppeteer_crawler.js+++ b/src/crawlers/puppeteer_crawler.js@@ -69,10 +69,10 @@ import { SessionPoolOptions } from '../session_pool/session_pool';* @property {PuppeteerGoto} [gotoFunction]* Overrides the function that opens the page in Puppeteer. The function should return the result of Puppeteer's* [page.goto()](https://pptr.dev/#?product=Puppeteer&show=api-pagegotourl-options) function,- * i.e. a `Promise` resolving to the [Response](https://pptr.dev/#?product=Puppeteer&show=api-class-response) object.+ * i.e. a `Promise` resolving to the [Response](https://pptr.dev/#?product=Puppeteer&show=api-class-httpresponse) object.*- * This is useful if you need to extend the page load timeout or select different criteria- * to determine that the navigation succeeded.+ * This is useful if you need to select different criteria to determine navigation success and also to do any+ * pre or post processing such as injecting cookies into the page.** Note that a single page object is only used to process a single request and it is closed afterwards.*