Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Build docs

Ondra Urban1f48e1fd7034
diff --git a/docs/api/Dataset.md b/docs/api/Dataset.md
index c49aa55d6..4c0131a9b 100644
--- a/docs/api/Dataset.md
+++ b/docs/api/Dataset.md
@@ -53,7 +53,7 @@ await dataset.pushData([
* [Dataset](dataset)
* [`.pushData(data)`](#Dataset+pushData) ⇒ <code>Promise</code>
- * [`.getData([options])`](#Dataset+getData) ⇒ <code>Promise</code>
+ * [`.getData([options])`](#Dataset+getData) ⇒ <code>Promise&lt;(Array\|String\|Buffer)&gt;</code>
* [`.getInfo()`](#Dataset+getInfo) ⇒ <code>Promise&lt;Object&gt;</code>
* [`.forEach(iteratee, [options], [index])`](#Dataset+forEach) ⇒ <code>Promise</code>
* [`.map(iteratee, options)`](#Dataset+map) ⇒ <code>Promise&lt;Array&gt;</code>
@@ -101,11 +101,9 @@ The objects must be serializable to JSON and the JSON representation of each obj
</table>
<a name="Dataset+getData"></a>
-## `dataset.getData([options])` ⇒ <code>Promise</code>
+## `dataset.getData([options])` ⇒ <code>Promise&lt;(Array\|String\|Buffer)&gt;</code>
Returns items in the dataset based on the provided parameters.
-If format is `json` then the function doesn't return an array of records but [`PaginationList`](../typedefs/paginationlist) instead.
-
<table>
<thead>
<tr>
@@ -217,7 +215,7 @@ Iterates over dataset items, yielding each in turn to an `iteratee` function.
Each invocation of `iteratee` is called with two arguments: `(item, index)`.
If the `iteratee` function returns a Promise then it is awaited before the next call.
-If it throws an error, the iteration is aborted and the `forEach` functions throws the error.
+If it throws an error, the iteration is aborted and the `forEach` function throws the error.
**Example usage**
```javascript
diff --git a/docs/api/KeyValueStore.md b/docs/api/KeyValueStore.md
index 1bce0a512..804f7759f 100644
--- a/docs/api/KeyValueStore.md
+++ b/docs/api/KeyValueStore.md
@@ -67,6 +67,7 @@ await store.delete('some-key');
* [`.getValue(key)`](#KeyValueStore+getValue) ⇒ <code>Promise&lt;(Object\|String\|Buffer)&gt;</code>
* [`.setValue(key, value, [options])`](#KeyValueStore+setValue) ⇒ <code>Promise</code>
* [`.delete()`](#KeyValueStore+delete) ⇒ <code>Promise</code>
+ * [`.forEachKey(iteratee, [options])`](#KeyValueStore+forEachKey) ⇒ <code>Promise</code>
<a name="KeyValueStore+getValue"></a>
@@ -187,3 +188,45 @@ otherwise the actor process might finish before the value is stored!
Removes the key-value store either from the Apify cloud storage or from the local directory,
depending on the mode of operation.
+<a name="KeyValueStore+forEachKey"></a>
+
+## `keyValueStore.forEachKey(iteratee, [options])` ⇒ <code>Promise</code>
+Iterates over key value store keys, yielding each in turn to an `iteratee` function.
+Each invocation of `iteratee` is called with two arguments: `(item, index)`.
+
+If the `iteratee` function returns a Promise then it is awaited before the next call.
+If it throws an error, the iteration is aborted and the `forEachKey` function throws the error.
+
+**Example usage**
+```javascript
+const keyValueStore = await Apify.openKeyValueStore();
+keyValueStore.forEachKey(async (key, index) => {
+ console.log(`Key at ${index}: ${key}`);
+});
+```
+
+<table>
+<thead>
+<tr>
+<th>Param</th><th>Type</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td><code>iteratee</code></td><td><code>function</code></td>
+</tr>
+<tr>
+<td colspan="3"><p>A function that is called for every key in the key value store.</p>
+</td></tr><tr>
+<td><code>[options]</code></td><td><code>Object</code></td>
+</tr>
+<tr>
+<td colspan="3"><p>All <code>forEachKey()</code> parameters are passed
+ via an options object with the following keys:</p>
+</td></tr><tr>
+<td><code>[options.exclusiveStartKey]</code></td><td><code>string</code></td>
+</tr>
+<tr>
+<td colspan="3"><p>All keys up to this one (including) are skipped from the result.</p>
+</td></tr></tbody>
+</table>
diff --git a/docs/api/PuppeteerCrawler.md b/docs/api/PuppeteerCrawler.md
index 6c38cb290..a46437107 100644
--- a/docs/api/PuppeteerCrawler.md
+++ b/docs/api/PuppeteerCrawler.md
@@ -166,13 +166,6 @@ await crawler.run();
Always set this value in order to prevent infinite loops in misconfigured crawlers.
Note that in cases of parallel crawling, the actual number of pages visited might be slightly higher than this value.</p>
</td></tr><tr>
-<td><code>[options.reusePages]</code></td><td><code>boolean</code></td><td><code>false</code></td>
-</tr>
-<tr>
-<td colspan="3"><p>Individual browser tabs will be reused after their task is complete instead
- of closing them and spawning a new tab. This saves CPU resources.
- See <code>reusePages</code> parameter of <a href="puppeteerpool"><code>PuppeteerPool</code></a>.</p>
-</td></tr><tr>
<td><code>[options.maxOpenPagesPerInstance]</code></td><td><code>Number</code></td><td><code>50</code></td>
</tr>
<tr>
diff --git a/docs/api/PuppeteerPool.md b/docs/api/PuppeteerPool.md
index 68527de03..f0ce8d54f 100644
--- a/docs/api/PuppeteerPool.md
+++ b/docs/api/PuppeteerPool.md
@@ -64,13 +64,6 @@ await puppeteerPool.destroy();
<td colspan="3"><p>All <code>PuppeteerPool</code> parameters are passed
via an options object with the following keys:</p>
</td></tr><tr>
-<td><code>[options.reusePages]</code></td><td><code>boolean</code></td><td><code>false</code></td>
-</tr>
-<tr>
-<td colspan="3"><p>Individual browser tabs will be reused after their task is complete instead
- of closing them and spawning a new tab. This saves CPU resources.
- Try disabling this feature if you encounter any weird behavior in your crawlers.</p>
-</td></tr><tr>
<td><code>[options.maxOpenPagesPerInstance]</code></td><td><code>Number</code></td><td><code>50</code></td>
</tr>
<tr>