Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Add object serialization to Request constructor

Ondra Urban04f4a2f9a7ff
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 931c4203e..49313a41c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,8 @@
+NEXT
+====================
+- Add `SessionPool` implemenation to `CheerioCrawler`.
+- Fix `Request` constructor not making a copy of objects such as `userData` and `headers`.
+
0.17.0 / 2019-11-25
====================
- **BREAKING**: Node 8 and 9 are no longer supported. Please use Node 10.17.0 or higher.
diff --git a/docs/examples/BasicCrawler.md b/docs/examples/BasicCrawler.md
index 68a2455d3..ab644b4ad 100644
--- a/docs/examples/BasicCrawler.md
+++ b/docs/examples/BasicCrawler.md
@@ -8,7 +8,7 @@ and [`RequestList`](../api/requestlist) classes. The script just downloads sever
<a href="https://www.npmjs.com/package/request-promise" target="_blank">request-promise</a> library) and stores their raw HTML and URL to the default
dataset. In local configuration, the data will be stored as JSON files in `./apify_storage/datasets/default`.
-To run this example on the Apify Platform, select the `Node.js 8 on Alpine Linux (apify/actor-node-basic)` base image on the source tab of your actor
+To run this example on the Apify Platform, select the `Node.js 10 on Alpine Linux (apify/actor-node-basic)` base image on the source tab of your actor
configuration.
```javascript
diff --git a/docs/examples/CallActor.md b/docs/examples/CallActor.md
index f24a598b0..46f0dca3b 100644
--- a/docs/examples/CallActor.md
+++ b/docs/examples/CallActor.md
@@ -13,7 +13,7 @@ To make the example work, you'll need an <a href="https://my.apify.com/" target=
`APIFY_TOKEN` [environment variable](../guides/environmentvariables), or run the script using the CLI. If you deploy this actor to the Apify Cloud
then you can set up a scheduler for early morning.
-To run this example on the Apify Platform, select the `Node.js 8 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
+To run this example on the Apify Platform, select the `Node.js 10 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
actor configuration.
```javascript
diff --git a/docs/examples/CheerioCrawler.md b/docs/examples/CheerioCrawler.md
index 536959d0d..49a1f4333 100644
--- a/docs/examples/CheerioCrawler.md
+++ b/docs/examples/CheerioCrawler.md
@@ -7,7 +7,7 @@ This example demonstrates how to use [`CheerioCrawler`](../api/cheeriocrawler) t
plain HTTP request, parse the HTML using <a href="https://www.npmjs.com/package/cheerio" target="_blank">cheerio</a> and extract some data from it:
the page title and all H1 tags.
-To run this example on the Apify Platform, select the `Node.js 8 on Alpine Linux (apify/actor-node-basic)` base image on the source tab of your actor
+To run this example on the Apify Platform, select the `Node.js 10 on Alpine Linux (apify/actor-node-basic)` base image on the source tab of your actor
configuration.
```javascript
diff --git a/docs/examples/Forms.md b/docs/examples/Forms.md
index bd8722416..da96d0cd6 100644
--- a/docs/examples/Forms.md
+++ b/docs/examples/Forms.md
@@ -9,7 +9,7 @@ search term, repository owner, start date and language of the repository, then s
saved either on the Apify platform to the default <a href="https://sdk.apify.com/docs/api/dataset" target="_blank">dataset</a> or on the local machine
as JSON files in ./apify_storage/datasets/default.
-To run this example on the Apify Platform, select the `Node.js 8 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
+To run this example on the Apify Platform, select the `Node.js 10 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
actor configuration.
```javascript
diff --git a/docs/examples/HelloWorld.md b/docs/examples/HelloWorld.md
index 88cede7f3..f0a763222 100644
--- a/docs/examples/HelloWorld.md
+++ b/docs/examples/HelloWorld.md
@@ -5,7 +5,7 @@ title: Hello World
Run the following example to perform a recursive crawl of a website using Puppeteer.
-To run this example on the Apify Platform, select the `Node.js 8 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
+To run this example on the Apify Platform, select the `Node.js 10 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
actor configuration.
```javascript
diff --git a/docs/examples/PuppeteerCrawler.md b/docs/examples/PuppeteerCrawler.md
index 1a7d3d1ce..f8014e092 100644
--- a/docs/examples/PuppeteerCrawler.md
+++ b/docs/examples/PuppeteerCrawler.md
@@ -8,7 +8,7 @@ recursively scrape the <a href="https://news.ycombinator.com" target="_blank">Ha
starts with a single URL, finds links to next pages, enqueues them and continues until no more desired links are available. The results are stored to
the default dataset. In local configuration, the results are stored as JSON files in `./apify_storage/datasets/default`
-To run this example on the Apify Platform, select the `Node.js 8 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
+To run this example on the Apify Platform, select the `Node.js 10 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
actor configuration.
```javascript
diff --git a/docs/examples/PuppeteerSitemap.md b/docs/examples/PuppeteerSitemap.md
index 149e6462b..11acd5315 100644
--- a/docs/examples/PuppeteerSitemap.md
+++ b/docs/examples/PuppeteerSitemap.md
@@ -7,7 +7,7 @@ This example demonstrates how to use [`PuppeteerCrawler`](../api/puppeteercrawle
extracts page title and URL from each page and stores them as a record to the default dataset. In local configuration, the results are stored as JSON
files in `./apify_storage/datasets/default`
-To run this example on the Apify Platform, select the `Node.js 8 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
+To run this example on the Apify Platform, select the `Node.js 10 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
actor configuration.
```javascript
diff --git a/docs/examples/PuppeteerWithProxy.md b/docs/examples/PuppeteerWithProxy.md
index f4258b130..e7805ac6b 100644
--- a/docs/examples/PuppeteerWithProxy.md
+++ b/docs/examples/PuppeteerWithProxy.md
@@ -8,7 +8,7 @@ Proxy</a>. To make it work, you'll need an Apify Account that has access to the
<a href="https://my.apify.com/proxy" target="_blank">Proxy</a> page in the app. Just set it to the `APIFY_PROXY_PASSWORD`
[environment variable](../guides/environmentvariables) or run the script using the CLI.
-To run this example on the Apify Platform, select the `Node.js 8 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
+To run this example on the Apify Platform, select the `Node.js 10 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
actor configuration.
```javascript
diff --git a/docs/examples/Screenshots.md b/docs/examples/Screenshots.md
index 47e74a2f9..2cde1e444 100644
--- a/docs/examples/Screenshots.md
+++ b/docs/examples/Screenshots.md
@@ -18,7 +18,7 @@ On the Apify cloud, the input can be either set manually in the UI app or passed
<a href="https://apify.com/docs/api/v2#/reference/actors/run-collection/run-actor" target="_blank">Run actor API call</a>. For more details, see
<a href="https://apify.com/docs/actor#input-output" target="_blank">Input and output</a> in the Apify Actor documentation.
-To run this example on the Apify Platform, select the `Node.js 8 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
+To run this example on the Apify Platform, select the `Node.js 10 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
actor configuration.
```javascript
diff --git a/docs/examples/SynchronousRun.md b/docs/examples/SynchronousRun.md
index ffaf9ef64..083da1004 100644
--- a/docs/examples/SynchronousRun.md
+++ b/docs/examples/SynchronousRun.md
@@ -17,7 +17,7 @@ can test it directly there simply by sending a POST request to
https://api.apify.com/v2/acts/apify~example-golden-gate-webcam/run-sync?token=[YOUR_API_TOKEN]
```
-To run this example on the Apify Platform, select the `Node.js 8 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
+To run this example on the Apify Platform, select the `Node.js 10 + Chrome on Debian (apify/actor-node-chrome)` base image on the source tab of your
actor configuration.
```javascript
diff --git a/package.json b/package.json
index 40b0ffa32..f4db6e405 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
{
"name": "apify",
- "version": "0.17.0",
+ "version": "0.17.1",
"description": "The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.",
"engines": {
"node": ">=10.17.0"
diff --git a/src/request.js b/src/request.js
index 88e1c4b15..6a025c996 100644
--- a/src/request.js
+++ b/src/request.js
@@ -179,9 +179,9 @@ class Request {
this.payload = payload;
this.noRetry = noRetry;
this.retryCount = retryCount;
- this.errorMessages = errorMessages;
- this.headers = headers;
- this.userData = userData;
+ this.errorMessages = JSON.parse(JSON.stringify(errorMessages));
+ this.headers = JSON.parse(JSON.stringify(headers));
+ this.userData = JSON.parse(JSON.stringify(userData));
// Requests received from API will have ISOString dates,
// but we want to have a Date instance.