Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Added missing simplified and skipFailedPages to dataset getData method

drobnikja9a8bb4c01e2
diff --git a/src/dataset.js b/src/dataset.js
index 22e151567..ddeff7f4c 100644
--- a/src/dataset.js
+++ b/src/dataset.js
@@ -235,6 +235,10 @@ export class Dataset {
* By default, the element name is `page` or `result`, depending on the value of the `simplified` option.
* @param {Boolean} [options.skipHeaderRow=false]
* If set to `true` then header row in CSV format is skipped.
+ * @param {Boolean} [options.simplified]
+ * If set to `true` then function applies the `fields: ['url','pageFunctionResult','errorInfo']` and `unwind: ['pageFunctionResult']` options.
+ * @param {Boolean} [options.skipFailedPages]
+ * If set to `true` then all the items with errorInfo property will be skipped from the output.
* @return {Promise<Array|String|Buffer>}
*/
getData(options = {}) {