Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

CheerioCrawler: Added doc for option json in handlePageFunction

drobnikj1e6f2e8bbbcb
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 075ec1e49..5f4c393c0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,6 +9,7 @@ NEXT
- Fix `Apify.utils.puppeteer.blockRequests()` always including default patterns.
- Fix parsing bad Content-Type headers for `CheerioCrawler`.
- Added regexps for Youtube videos (`YOUTUBE_REGEX`, `YOUTUBE_REGEX_GLOBAL`) to `utils.social`
+- Added documentation for option `json` in handlePageFunction of `CheerioCrawler`
0.16.1 / 2019-10-31
====================
diff --git a/docs/api/CheerioCrawler.md b/docs/api/CheerioCrawler.md
index 51e80ec57..5979f83e6 100644
--- a/docs/api/CheerioCrawler.md
+++ b/docs/api/CheerioCrawler.md
@@ -102,6 +102,7 @@ await crawler.run();
<pre><code>{
$: Cheerio, // the Cheerio object with parsed HTML
body: String|Buffer // the request body of the web page
+ json: Object, // the parsed object from JSON string if the response contains the content type application/json
request: Request,
contentType: Object, // Parsed `Content-Type` header
response: Object // An instance of Node&#39;s http.IncomingMessage object,
diff --git a/src/crawlers/cheerio_crawler.js b/src/crawlers/cheerio_crawler.js
index 4e6ca9cca..8e3f3e472 100644
--- a/src/crawlers/cheerio_crawler.js
+++ b/src/crawlers/cheerio_crawler.js
@@ -123,6 +123,7 @@ const DEFAULT_OPTIONS = {
* {
* $: Cheerio, // the Cheerio object with parsed HTML
* body: String|Buffer // the request body of the web page
+ * json: Object, // the parsed object from JSON string if the response contains the content type application/json
* request: Request,
* contentType: Object, // Parsed `Content-Type` header
* response: Object // An instance of Node's http.IncomingMessage object,