Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Improve clickElements docs [skip ci]

Ondra Urban87c0c00d02df
diff --git a/docs/api/puppeteer.md b/docs/api/puppeteer.md
index d1d4e5ba2..d9d2b6de3 100644
--- a/docs/api/puppeteer.md
+++ b/docs/api/puppeteer.md
@@ -114,7 +114,7 @@ await Apify.utils.enqueueLinksByClickingElements({
<td colspan="3"><p>A CSS selector matching elements to be clicked on. Unlike in <a href="utils#utils.enqueueLinks"><code>enqueueLinks()</code></a>, there is no default
value. This is to prevent suboptimal use of this function by using it too broadly.</p>
</td></tr><tr>
-<td><code>[options.pseudoUrls]</code></td><td><code>Array<Object&gt;</code> | <code>Array.&lt;String></code></td><td></td>
+<td><code>[options.pseudoUrls]</code></td><td><code>Array<(String|RegExp|Object)></code></td><td></td>
</tr>
<tr>
<td colspan="3"><p>An array of <a href="pseudourl"><code>PseudoUrl</code></a>s matching the URLs to be enqueued,
@@ -128,13 +128,20 @@ await Apify.utils.enqueueLinksByClickingElements({
<td><code>[options.transformRequestFunction]</code></td><td><code>function</code></td><td></td>
</tr>
<tr>
-<td colspan="3"><p>Just before a new <a href="request"><code>Request</code></a> is constructed and enqueued to the <a href="requestqueue"><code>RequestQueue</code></a>, this function can be used
+<td colspan="3"><p><strong>Signature:</strong> (<a href="request"><code>Request</code></a>): <a href="request"><code>Request</code></a></p>
+<p> Just before a new <a href="request"><code>Request</code></a> is constructed and enqueued to the <a href="requestqueue"><code>RequestQueue</code></a>, this function can be used
to remove it or modify its contents such as <code>userData</code>, <code>payload</code> or, most importantly <code>uniqueKey</code>. This is useful
when you need to enqueue multiple <code>Requests</code> to the queue that share the same URL, but differ in methods or payloads,
or to dynamically update or create <code>userData</code>.</p>
<p> For example: by adding <code>useExtendedUniqueKey: true</code> to the <code>request</code> object, <code>uniqueKey</code> will be computed from
a combination of <code>url</code>, <code>method</code> and <code>payload</code> which enables crawling of websites that navigate using form submits
(POST requests).</p>
+<p> <strong>Example:</strong></p>
+<pre><code class="language-javascript"> function transformRequestFunction(request) {
+ request.userData.foo = &#39;bar&#39;;
+ request.useExtendedUniqueKey = true;
+ return request;
+ }</code></pre>
</td></tr><tr>
<td><code>[options.waitForPageIdleSecs]</code></td><td><code>number</code></td><td><code>1</code></td>
</tr>
diff --git a/src/enqueue_links/click_elements.js b/src/enqueue_links/click_elements.js
index e505180f9..637e91920 100644
--- a/src/enqueue_links/click_elements.js
+++ b/src/enqueue_links/click_elements.js
@@ -63,7 +63,7 @@ const STARTING_Z_INDEX = 2147400000;
* @param {String} options.selector
* A CSS selector matching elements to be clicked on. Unlike in [`enqueueLinks()`](utils#utils.enqueueLinks), there is no default
* value. This is to prevent suboptimal use of this function by using it too broadly.
- * @param {Object[]|String[]} [options.pseudoUrls]
+ * @param {Array<String|RegExp|Object>} [options.pseudoUrls]
* An array of {@link PseudoUrl}s matching the URLs to be enqueued,
* or an array of strings or RegExps or plain Objects from which the {@link PseudoUrl}s can be constructed.
*
@@ -74,6 +74,8 @@ const STARTING_Z_INDEX = 2147400000;
* If `pseudoUrls` is an empty array, `null` or `undefined`, then the function
* enqueues all links found on the page.
* @param {Function} [options.transformRequestFunction]
+ * **Signature:** ({@link Request}): {@link Request}
+ *
* Just before a new {@link Request} is constructed and enqueued to the {@link RequestQueue}, this function can be used
* to remove it or modify its contents such as `userData`, `payload` or, most importantly `uniqueKey`. This is useful
* when you need to enqueue multiple `Requests` to the queue that share the same URL, but differ in methods or payloads,
@@ -82,6 +84,15 @@ const STARTING_Z_INDEX = 2147400000;
* For example: by adding `useExtendedUniqueKey: true` to the `request` object, `uniqueKey` will be computed from
* a combination of `url`, `method` and `payload` which enables crawling of websites that navigate using form submits
* (POST requests).
+ *
+ * **Example:**
+ * ```javascript
+ * function transformRequestFunction(request) {
+ * request.userData.foo = 'bar';
+ * request.useExtendedUniqueKey = true;
+ * return request;
+ * }
+ * ```
* @param {number} [options.waitForPageIdleSecs=1]
* Clicking in the page triggers various asynchronous operations that lead to new URLs being shown
* by the browser. It could be a simple JavaScript redirect or opening of a new tab in the browser.