Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Update dataset getData docs

Ondra Urban0f7bbfd0802e
diff --git a/docs/api/Dataset.md b/docs/api/Dataset.md
index 187cfa4ac..3ce616678 100644
--- a/docs/api/Dataset.md
+++ b/docs/api/Dataset.md
@@ -53,7 +53,7 @@ await dataset.pushData([
* [Dataset](dataset)
* [`.pushData(data)`](#Dataset+pushData) ⇒ `Promise`
- * [`.getData([options])`](#Dataset+getData) ⇒ `Promise<(Array|String|Buffer)>`
+ * [`.getData([options])`](#Dataset+getData) ⇒ `Promise<Object>`
* [`.getInfo()`](#Dataset+getInfo) ⇒ `Promise<Object>`
* [`.forEach(iteratee, [options], [index])`](#Dataset+forEach) ⇒ `Promise`
* [`.map(iteratee, options)`](#Dataset+map) ⇒ `Promise<Array>`
@@ -101,8 +101,18 @@ The objects must be serializable to JSON and the JSON representation of each obj
</table>
<a name="Dataset+getData"></a>
-## `dataset.getData([options])` ⇒ `Promise<(Array|String|Buffer)>`
-Returns items in the dataset based on the provided parameters.
+## `dataset.getData([options])` ⇒ `Promise<Object>`
+Returns items in the dataset based on the provided parameters. The returned object
+has the following structure:
+
+```javascript
+{
+ items, // Array|String|Buffer based on chosen format parameter.
+ total, // Number
+ limit, // Number
+ offset, // Number
+}
+```
<table>
<thead>
@@ -121,7 +131,7 @@ Returns items in the dataset based on the provided parameters.
<td><code>[options.format]</code></td><td><code>String</code></td><td><code>&#x27;json&#x27;</code></td>
</tr>
<tr>
-<td colspan="3"><p>Format of the items, possible values are: <code>json</code>, <code>csv</code>, <code>xlsx</code>, <code>html</code>, <code>xml</code> and <code>rss</code>.</p>
+<td colspan="3"><p>Format of the <code>items</code> property, possible values are: <code>json</code>, <code>csv</code>, <code>xlsx</code>, <code>html</code>, <code>xml</code> and <code>rss</code>.</p>
</td></tr><tr>
<td><code>[options.offset]</code></td><td><code>Number</code></td><td><code>0</code></td>
</tr>
diff --git a/src/dataset.js b/src/dataset.js
index ec01c3dd2..b43b86445 100644
--- a/src/dataset.js
+++ b/src/dataset.js
@@ -199,12 +199,22 @@ export class Dataset {
}
/**
- * Returns items in the dataset based on the provided parameters.
+ * Returns items in the dataset based on the provided parameters. The returned object
+ * has the following structure:
+ *
+ * ```javascript
+ * {
+ * items, // Array|String|Buffer based on chosen format parameter.
+ * total, // Number
+ * limit, // Number
+ * offset, // Number
+ * }
+ * ```
*
* @param {Object} [options] All `getData()` parameters are passed
* via an options object with the following keys:
* @param {String} [options.format='json']
- * Format of the items, possible values are: `json`, `csv`, `xlsx`, `html`, `xml` and `rss`.
+ * Format of the `items` property, possible values are: `json`, `csv`, `xlsx`, `html`, `xml` and `rss`.
* @param {Number} [options.offset=0]
* Number of array elements that should be skipped at the start.
* @param {Number} [options.limit=250000]
@@ -239,7 +249,7 @@ export class Dataset {
* If set to `true` then function applies the `fields: ['url','pageFunctionResult','errorInfo']` and `unwind: 'pageFunctionResult'` options.
* @param {Boolean} [options.skipFailedPages]
* If set to `true` then all the items with errorInfo property will be skipped from the output.
- * @return {Promise<Array|String|Buffer>}
+ * @return {Promise<Object>}
*/
getData(options = {}) {
const { datasetId } = this;