Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Fix doc generation and build docs

Ondra Urban71dbe9fad9ab
diff --git a/docs/api/Apify.md b/docs/api/Apify.md
index 0e06b5178..809dccad7 100644
--- a/docs/api/Apify.md
+++ b/docs/api/Apify.md
@@ -10,31 +10,31 @@ accessible from the left sidebar.
* [Apify](#module_Apify)
- * [`.addWebhook(options)`](#module_Apify.addWebhook) ⇒ <code>Promise&lt;Object&gt;</code>
- * [`.call(actId, [input], [options])`](#module_Apify.call) ⇒ [<code>Promise&lt;ActorRun&gt;</code>](../typedefs/actorrun)
- * [`.callTask(taskId, [input], [options])`](#module_Apify.callTask) ⇒ [<code>Promise&lt;ActorRun&gt;</code>](../typedefs/actorrun)
+ * [`.addWebhook(options)`](#module_Apify.addWebhook) ⇒ `Promise<Object>`
+ * [`.call(actId, [input], [options])`](#module_Apify.call) ⇒ [`Promise<ActorRun>`](../typedefs/actorrun)
+ * [`.callTask(taskId, [input], [options])`](#module_Apify.callTask) ⇒ [`Promise<ActorRun>`](../typedefs/actorrun)
* [`.client`](#module_Apify.client)
* [`.events`](#module_Apify.events)
- * [`.getApifyProxyUrl(options)`](#module_Apify.getApifyProxyUrl) ⇒ <code>String</code>
- * [`.getEnv()`](#module_Apify.getEnv) ⇒ <code>Object</code>
- * [`.getInput`](#module_Apify.getInput) ⇒ <code>Promise&lt;Object&gt;</code>
- * [`.getMemoryInfo()`](#module_Apify.getMemoryInfo) ⇒ <code>Promise&lt;Object&gt;</code>
- * [`.getValue(key)`](#module_Apify.getValue) ⇒ <code>Promise&lt;Object&gt;</code>
- * [`.isAtHome()`](#module_Apify.isAtHome) ⇒ <code>Boolean</code>
- * [`.launchPuppeteer([options])`](#module_Apify.launchPuppeteer) ⇒ <code>Promise&lt;Browser&gt;</code>
- * [`.launchWebDriver([options])`](#module_Apify.launchWebDriver) ⇒ <code>Promise</code>
+ * [`.getApifyProxyUrl(options)`](#module_Apify.getApifyProxyUrl) ⇒ `String`
+ * [`.getEnv()`](#module_Apify.getEnv) ⇒ `Object`
+ * [`.getInput`](#module_Apify.getInput) ⇒ `Promise<Object>`
+ * [`.getMemoryInfo()`](#module_Apify.getMemoryInfo) ⇒ `Promise<Object>`
+ * [`.getValue(key)`](#module_Apify.getValue) ⇒ `Promise<Object>`
+ * [`.isAtHome()`](#module_Apify.isAtHome) ⇒ `Boolean`
+ * [`.launchPuppeteer([options])`](#module_Apify.launchPuppeteer) ⇒ `Promise<Browser>`
+ * [`.launchWebDriver([options])`](#module_Apify.launchWebDriver) ⇒ `Promise`
* [`.main(userFunc)`](#module_Apify.main)
- * [`.metamorph(targetActorId, [input], [options])`](#module_Apify.metamorph) ⇒ <code>Promise&lt;undefined&gt;</code>
- * [`.openDataset([datasetIdOrName], [options])`](#module_Apify.openDataset) ⇒ [<code>Promise&lt;Dataset&gt;</code>](dataset)
- * [`.openKeyValueStore([storeIdOrName], [options])`](#module_Apify.openKeyValueStore) ⇒ [<code>Promise&lt;KeyValueStore&gt;</code>](keyvaluestore)
- * [`.openRequestList`](#module_Apify.openRequestList) ⇒ [<code>Promise&lt;RequestList&gt;</code>](requestlist)
- * [`.openRequestQueue`](#module_Apify.openRequestQueue) ⇒ [<code>Promise&lt;RequestQueue&gt;</code>](requestqueue)
- * [`.pushData(item)`](#module_Apify.pushData) ⇒ <code>Promise</code>
- * [`.setValue(key, value, [options])`](#module_Apify.setValue) ⇒ <code>Promise</code>
+ * [`.metamorph(targetActorId, [input], [options])`](#module_Apify.metamorph) ⇒ `Promise<undefined>`
+ * [`.openDataset([datasetIdOrName], [options])`](#module_Apify.openDataset) ⇒ [`Promise<Dataset>`](dataset)
+ * [`.openKeyValueStore([storeIdOrName], [options])`](#module_Apify.openKeyValueStore) ⇒ [`Promise<KeyValueStore>`](keyvaluestore)
+ * [`.openRequestList`](#module_Apify.openRequestList) ⇒ [`Promise<RequestList>`](requestlist)
+ * [`.openRequestQueue`](#module_Apify.openRequestQueue) ⇒ [`Promise<RequestQueue>`](requestqueue)
+ * [`.pushData(item)`](#module_Apify.pushData) ⇒ `Promise`
+ * [`.setValue(key, value, [options])`](#module_Apify.setValue) ⇒ `Promise`
<a name="module_Apify.addWebhook"></a>
-## `Apify.addWebhook(options)` ⇒ <code>Promise&lt;Object&gt;</code>
+## `Apify.addWebhook(options)` ⇒ `Promise<Object>`
Creates an ad-hoc webhook for the current actor run, which lets you receive a notification when the actor run finished or failed.
For more information about Apify actor webhooks, please see the <a href="https://apify.com/docs/webhook" target="_blank">documentation</a>.
@@ -53,7 +53,7 @@ In local environment, the function will print a warning and have no effect.
</tr>
<tr>
<td colspan="3"></td></tr><tr>
-<td><code>options.eventTypes</code></td><td><code>Array&lt;String&gt;</code></td>
+<td><code>options.eventTypes</code></td><td><code>Array<String></code></td>
</tr>
<tr>
<td colspan="3"><p>Array of event types, which you can set for actor run, see
@@ -77,7 +77,7 @@ In local environment, the function will print a warning and have no effect.
</table>
<a name="module_Apify.call"></a>
-## `Apify.call(actId, [input], [options])` ⇒ [<code>Promise&lt;ActorRun&gt;</code>](../typedefs/actorrun)
+## `Apify.call(actId, [input], [options])` ⇒ [`Promise<ActorRun>`](../typedefs/actorrun)
Runs an actor on the Apify platform using the current user account (determined by the `APIFY_TOKEN` environment variable),
waits for the actor to finish and fetches its output.
@@ -107,7 +107,7 @@ and several other API endpoints to obtain the output.
**Throws**:
-- [<code>ApifyCallError</code>](../typedefs/apifycallerror) If the run did not succeed, e.g. if it failed or timed out.
+- [`ApifyCallError`](../typedefs/apifycallerror) If the run did not succeed, e.g. if it failed or timed out.
<table>
<thead>
@@ -187,7 +187,7 @@ and several other API endpoints to obtain the output.
</table>
<a name="module_Apify.callTask"></a>
-## `Apify.callTask(taskId, [input], [options])` ⇒ [<code>Promise&lt;ActorRun&gt;</code>](../typedefs/actorrun)
+## `Apify.callTask(taskId, [input], [options])` ⇒ [`Promise<ActorRun>`](../typedefs/actorrun)
Runs an actor task on the Apify platform using the current user account (determined by the `APIFY_TOKEN` environment variable),
waits for the task to finish and fetches its output.
@@ -217,7 +217,7 @@ and several other API endpoints to obtain the output.
**Throws**:
-- [<code>ApifyCallError</code>](../typedefs/apifycallerror) If the run did not succeed, e.g. if it failed or timed out.
+- [`ApifyCallError`](../typedefs/apifycallerror) If the run did not succeed, e.g. if it failed or timed out.
<table>
<thead>
@@ -368,7 +368,7 @@ The following table shows all currently emitted events:
<a name="module_Apify.getApifyProxyUrl"></a>
-## `Apify.getApifyProxyUrl(options)` ⇒ <code>String</code>
+## `Apify.getApifyProxyUrl(options)` ⇒ `String`
Constructs an Apify Proxy URL using the specified settings.
The proxy URL can be used from Apify actors, web browsers or any other HTTP
proxy-enabled applications.
@@ -377,7 +377,7 @@ For more information, see
the <a href="https://my.apify.com/proxy" target="_blank">Apify Proxy</a> page in the app
or the <a href="https://apify.com/docs/proxy" target="_blank">documentation</a>.
-**Returns**: <code>String</code> - Returns the proxy URL, e.g. `http://auto:[email protected]:8000`.
+**Returns**: `String` - Returns the proxy URL, e.g. `http://auto:[email protected]:8000`.
<table>
<thead>
<tr>
@@ -400,7 +400,7 @@ or the <a href="https://apify.com/docs/proxy" target="_blank">documentation</a>.
or when using the <a href="https://github.com/apifytech/apify-cli" target="_blank">Apify CLI</a>
package and the user previously logged in (called <code>apify login</code>).</p>
</td></tr><tr>
-<td><code>[options.groups]</code></td><td><code>Array&lt;String&gt;</code></td>
+<td><code>[options.groups]</code></td><td><code>Array<String></code></td>
</tr>
<tr>
<td colspan="3"><p>Array of Apify Proxy groups to be used.
@@ -417,7 +417,7 @@ or the <a href="https://apify.com/docs/proxy" target="_blank">documentation</a>.
</table>
<a name="module_Apify.getEnv"></a>
-## `Apify.getEnv()` ⇒ <code>Object</code>
+## `Apify.getEnv()` ⇒ `Object`
Returns a new object which contains information parsed from all the `APIFY_XXX` environment variables.
It has properties such as the following:
@@ -456,7 +456,7 @@ If some of the variables are not defined or are invalid, the corresponding value
<a name="module_Apify.getInput"></a>
-## `Apify.getInput` ⇒ <code>Promise&lt;Object&gt;</code>
+## `Apify.getInput` ⇒ `Promise<Object>`
Gets the actor input value from the default [`KeyValueStore`](keyvaluestore) associated with the current actor run.
This is just a convenient shortcut for [`keyValueStore.getValue('INPUT')`](keyvaluestore#KeyValueStore+getValue).
@@ -474,10 +474,10 @@ await store.getValue('INPUT');
For more information, see [`Apify.openKeyValueStore()`](#module_Apify.openKeyValueStore)
and [`keyValueStore.getValue()`](keyvaluestore#KeyValueStore+getValue).
-**Returns**: <code>Promise&lt;Object&gt;</code> - Returns a promise that resolves once the record is stored.
+**Returns**: `Promise<Object>` - Returns a promise that resolves once the record is stored.
<a name="module_Apify.getMemoryInfo"></a>
-## `Apify.getMemoryInfo()` ⇒ <code>Promise&lt;Object&gt;</code>
+## `Apify.getMemoryInfo()` ⇒ `Promise<Object>`
Returns memory statistics of the process and the system, which is an object with the following properties:
```javascript
@@ -503,7 +503,7 @@ Therefore you shouldn't call it too often, like more than once per second.
<a name="module_Apify.getValue"></a>
-## `Apify.getValue(key)` ⇒ <code>Promise&lt;Object&gt;</code>
+## `Apify.getValue(key)` ⇒ `Promise<Object>`
Gets a value from the default [`KeyValueStore`](keyvaluestore) associated with the current actor run.
This is just a convenient shortcut for [`keyValueStore.getValue()`](keyvaluestore#KeyValueStore+getValue).
@@ -523,7 +523,7 @@ To store the value to the default-key value store, you can use the [`Apify.setVa
For more information, see [`Apify.openKeyValueStore()`](#module_Apify.openKeyValueStore)
and [`keyValueStore.getValue()`](keyvaluestore#KeyValueStore+getValue).
-**Returns**: <code>Promise&lt;Object&gt;</code> - Returns a promise that resolves once the record is stored.
+**Returns**: `Promise<Object>` - Returns a promise that resolves once the record is stored.
<table>
<thead>
<tr>
@@ -540,12 +540,12 @@ and [`keyValueStore.getValue()`](keyvaluestore#KeyValueStore+getValue).
</table>
<a name="module_Apify.isAtHome"></a>
-## `Apify.isAtHome()` ⇒ <code>Boolean</code>
+## `Apify.isAtHome()` ⇒ `Boolean`
Returns `true` when code is running on Apify platform and `false` otherwise (for example locally).
<a name="module_Apify.launchPuppeteer"></a>
-## `Apify.launchPuppeteer([options])` ⇒ <code>Promise&lt;Browser&gt;</code>
+## `Apify.launchPuppeteer([options])` ⇒ `Promise<Browser>`
Launches headless Chrome using Puppeteer pre-configured to work within the Apify platform.
The function has the same argument and the return value as `puppeteer.launch()`.
See <a href="https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#puppeteerlaunchoptions" target="_blank">
@@ -593,7 +593,7 @@ for details.
For an example of usage, see the [Synchronous run Example](../examples/synchronousrun)
or the [Puppeteer proxy Example](../examples/puppeteerwithproxy)
-**Returns**: <code>Promise&lt;Browser&gt;</code> - Promise that resolves to Puppeteer's `Browser` instance.
+**Returns**: `Promise<Browser>` - Promise that resolves to Puppeteer's `Browser` instance.
<table>
<thead>
<tr>
@@ -612,7 +612,7 @@ or the [Puppeteer proxy Example](../examples/puppeteerwithproxy)
</table>
<a name="module_Apify.launchWebDriver"></a>
-## `Apify.launchWebDriver([options])` ⇒ <code>Promise</code>
+## `Apify.launchWebDriver([options])` ⇒ `Promise`
Opens a new instance of Chrome web browser
controlled by <a href="http://www.seleniumhq.org/projects/webdriver/" target="_blank">Selenium WebDriver</a>.
The result of the function is the new instance of the
@@ -738,7 +738,7 @@ the promise will be awaited. The user function is called with no arguments.</p>
</table>
<a name="module_Apify.metamorph"></a>
-## `Apify.metamorph(targetActorId, [input], [options])` ⇒ <code>Promise&lt;undefined&gt;</code>
+## `Apify.metamorph(targetActorId, [input], [options])` ⇒ `Promise<undefined>`
Transforms this actor run to an actor run of a given actor. The system stops the current container and starts the new container
instead. All the default storages are preserved and the new input is stored under the `INPUT-METAMORPH-1` key in the same default key-value store.
@@ -784,7 +784,7 @@ instead. All the default storages are preserved and the new input is stored unde
</table>
<a name="module_Apify.openDataset"></a>
-## `Apify.openDataset([datasetIdOrName], [options])` ⇒ [<code>Promise&lt;Dataset&gt;</code>](dataset)
+## `Apify.openDataset([datasetIdOrName], [options])` ⇒ [`Promise<Dataset>`](dataset)
Opens a dataset and returns a promise resolving to an instance of the [`Dataset`](dataset) class.
Datasets are used to store structured data where each object stored has the same attributes,
@@ -820,7 +820,7 @@ For more details and code examples, see the [`Dataset`](dataset) class.
</table>
<a name="module_Apify.openKeyValueStore"></a>
-## `Apify.openKeyValueStore([storeIdOrName], [options])` ⇒ [<code>Promise&lt;KeyValueStore&gt;</code>](keyvaluestore)
+## `Apify.openKeyValueStore([storeIdOrName], [options])` ⇒ [`Promise<KeyValueStore>`](keyvaluestore)
Opens a key-value store and returns a promise resolving to an instance of the [`KeyValueStore`](keyvaluestore) class.
Key-value stores are used to store records or files, along with their MIME content type.
@@ -856,7 +856,7 @@ For more details and code examples, see the [`KeyValueStore`](keyvaluestore) cla
</table>
<a name="module_Apify.openRequestList"></a>
-## `Apify.openRequestList` ⇒ [<code>Promise&lt;RequestList&gt;</code>](requestlist)
+## `Apify.openRequestList` ⇒ [`Promise<RequestList>`](requestlist)
Opens a request list and returns a promise resolving to an instance
of the [`RequestList`](requestlist) class that is already initialized.
@@ -899,7 +899,7 @@ const requestList = await Apify.openRequestList('my-name', sources);
<p> If <code>null</code>, the list will not be persisted and will only be stored in memory. Process restart
will then cause the list to be crawled again from the beginning. We suggest always using a name.</p>
</td></tr><tr>
-<td><code>sources</code></td><td><code>Array&lt;Object&gt;</code> | <code>Array&lt;string&gt;</code></td>
+<td><code>sources</code></td><td><code>Array<Object&gt;</code> | <code>Array.&lt;string></code></td>
</tr>
<tr>
<td colspan="3"><p>An array of sources of URLs for the <code>RequestList</code>.
@@ -920,7 +920,7 @@ const requestList = await Apify.openRequestList('my-name', sources);
</table>
<a name="module_Apify.openRequestQueue"></a>
-## `Apify.openRequestQueue` ⇒ [<code>Promise&lt;RequestQueue&gt;</code>](requestqueue)
+## `Apify.openRequestQueue` ⇒ [`Promise<RequestQueue>`](requestqueue)
Opens a request queue and returns a promise resolving to an instance
of the [`RequestQueue`](requestqueue) class.
@@ -958,7 +958,7 @@ For more details and code examples, see the [`RequestQueue`](requestqueue) class
</table>
<a name="module_Apify.pushData"></a>
-## `Apify.pushData(item)` ⇒ <code>Promise</code>
+## `Apify.pushData(item)` ⇒ `Promise`
Stores an object or an array of objects to the default [`Dataset`](dataset) of the current actor run.
This is just a convenient shortcut for [`dataset.pushData()`](dataset#Dataset+pushData).
@@ -995,7 +995,7 @@ The objects must be serializable to JSON and the JSON representation of each obj
</table>
<a name="module_Apify.setValue"></a>
-## `Apify.setValue(key, value, [options])` ⇒ <code>Promise</code>
+## `Apify.setValue(key, value, [options])` ⇒ `Promise`
Stores or deletes a value in the default [`KeyValueStore`](keyvaluestore) associated with the current actor run.
This is just a convenient shortcut for [`keyValueStore.setValue()`](keyvaluestore#KeyValueStore+setValue).
diff --git a/docs/api/AutoscaledPool.md b/docs/api/AutoscaledPool.md
index 9dde7d47c..981d13c63 100644
--- a/docs/api/AutoscaledPool.md
+++ b/docs/api/AutoscaledPool.md
@@ -53,16 +53,16 @@ await pool.run();
* [AutoscaledPool](autoscaledpool)
* [`new AutoscaledPool(options)`](#new_AutoscaledPool_new)
- * [`.minConcurrency`](#AutoscaledPool+minConcurrency) ⇒ <code>number</code>
+ * [`.minConcurrency`](#AutoscaledPool+minConcurrency) ⇒ `number`
* [`.minConcurrency`](#AutoscaledPool+minConcurrency)
- * [`.maxConcurrency`](#AutoscaledPool+maxConcurrency) ⇒ <code>number</code>
+ * [`.maxConcurrency`](#AutoscaledPool+maxConcurrency) ⇒ `number`
* [`.maxConcurrency`](#AutoscaledPool+maxConcurrency)
- * [`.desiredConcurrency`](#AutoscaledPool+desiredConcurrency) ⇒ <code>number</code>
+ * [`.desiredConcurrency`](#AutoscaledPool+desiredConcurrency) ⇒ `number`
* [`.desiredConcurrency`](#AutoscaledPool+desiredConcurrency)
- * [`.currentConcurrency`](#AutoscaledPool+currentConcurrency) ⇒ <code>number</code>
- * [`.run()`](#AutoscaledPool+run) ⇒ <code>Promise</code>
- * [`.abort()`](#AutoscaledPool+abort) ⇒ <code>Promise</code>
- * [`.pause([timeoutSecs])`](#AutoscaledPool+pause) ⇒ <code>Promise</code>
+ * [`.currentConcurrency`](#AutoscaledPool+currentConcurrency) ⇒ `number`
+ * [`.run()`](#AutoscaledPool+run) ⇒ `Promise`
+ * [`.abort()`](#AutoscaledPool+abort) ⇒ `Promise`
+ * [`.pause([timeoutSecs])`](#AutoscaledPool+pause) ⇒ `Promise`
* [`.resume()`](#AutoscaledPool+resume)
<a name="new_AutoscaledPool_new"></a>
@@ -169,7 +169,7 @@ await pool.run();
</table>
<a name="AutoscaledPool+minConcurrency"></a>
-## `autoscaledPool.minConcurrency` ⇒ <code>number</code>
+## `autoscaledPool.minConcurrency` ⇒ `number`
Gets the minimum number of tasks running in parallel.
<a name="AutoscaledPool+minConcurrency"></a>
@@ -195,7 +195,7 @@ If you're not sure, just keep the default value and the concurrency will scale u
</table>
<a name="AutoscaledPool+maxConcurrency"></a>
-## `autoscaledPool.maxConcurrency` ⇒ <code>number</code>
+## `autoscaledPool.maxConcurrency` ⇒ `number`
Gets the maximum number of tasks running in parallel.
<a name="AutoscaledPool+maxConcurrency"></a>
@@ -218,7 +218,7 @@ Sets the maximum number of tasks running in parallel.
</table>
<a name="AutoscaledPool+desiredConcurrency"></a>
-## `autoscaledPool.desiredConcurrency` ⇒ <code>number</code>
+## `autoscaledPool.desiredConcurrency` ⇒ `number`
Gets the desired concurrency for the pool,
which is an estimated number of parallel tasks that the system can currently support.
@@ -243,18 +243,18 @@ in parallel if there's large enough supply of tasks.
</table>
<a name="AutoscaledPool+currentConcurrency"></a>
-## `autoscaledPool.currentConcurrency` ⇒ <code>number</code>
+## `autoscaledPool.currentConcurrency` ⇒ `number`
Gets the the number of parallel tasks currently running in the pool.
<a name="AutoscaledPool+run"></a>
-## `autoscaledPool.run()` ⇒ <code>Promise</code>
+## `autoscaledPool.run()` ⇒ `Promise`
Runs the auto-scaled pool. Returns a promise that gets resolved or rejected once
all the tasks are finished or one of them fails.
<a name="AutoscaledPool+abort"></a>
-## `autoscaledPool.abort()` ⇒ <code>Promise</code>
+## `autoscaledPool.abort()` ⇒ `Promise`
Aborts the run of the auto-scaled pool and destroys it. The promise returned from
the [`run()`](#AutoscaledPool+run) function will immediately resolve, no more new tasks
will be spawned and all running tasks will be left in their current state.
@@ -267,7 +267,7 @@ parts of their asynchronous chains of commands will not execute.
<a name="AutoscaledPool+pause"></a>
-## `autoscaledPool.pause([timeoutSecs])` ⇒ <code>Promise</code>
+## `autoscaledPool.pause([timeoutSecs])` ⇒ `Promise`
Prevents the auto-scaled pool from starting new tasks, but allows the running ones to finish
(unlike abort, which terminates them). Used together with [`resume()`](#AutoscaledPool+resume)
diff --git a/docs/api/BasicCrawler.md b/docs/api/BasicCrawler.md
index 04d771682..86c5ee368 100644
--- a/docs/api/BasicCrawler.md
+++ b/docs/api/BasicCrawler.md
@@ -64,7 +64,7 @@ await crawler.run();
* [BasicCrawler](basiccrawler)
* [`new BasicCrawler(options)`](#new_BasicCrawler_new)
- * [`.run()`](#BasicCrawler+run) ⇒ <code>Promise</code>
+ * [`.run()`](#BasicCrawler+run) ⇒ `Promise`
<a name="new_BasicCrawler_new"></a>
@@ -91,8 +91,7 @@ await crawler.run();
<pre><code>{
request: Request,
autoscaledPool: AutoscaledPool
-}
-</code></pre><p> where the <a href="request"><code>Request</code></a> instance represents the URL to crawl.</p>
+}</code></pre><p> where the <a href="request"><code>Request</code></a> instance represents the URL to crawl.</p>
<p> The function must return a promise, which is then awaited by the crawler.</p>
<p> If the function throws an exception, the crawler will try to re-crawl the
request later, up to <code>option.maxRequestRetries</code> times.
@@ -127,8 +126,7 @@ await crawler.run();
<pre><code>{
request: Request,
error: Error,
-}
-</code></pre><p> where the <a href="request"><code>Request</code></a> instance corresponds to the failed request, and the <code>Error</code> instance
+}</code></pre><p> where the <a href="request"><code>Request</code></a> instance corresponds to the failed request, and the <code>Error</code> instance
represents the last error thrown during processing of the request.</p>
<p> See
<a href="https://github.com/apifytech/apify-js/blob/master/src/basic_crawler.js#L11" target="_blank">source code</a>
@@ -168,6 +166,6 @@ await crawler.run();
</table>
<a name="BasicCrawler+run"></a>
-## `basicCrawler.run()` ⇒ <code>Promise</code>
+## `basicCrawler.run()` ⇒ `Promise`
Runs the crawler. Returns a promise that gets resolved once all the requests are processed.
diff --git a/docs/api/CheerioCrawler.md b/docs/api/CheerioCrawler.md
index 92ae578bc..0d4160221 100644
--- a/docs/api/CheerioCrawler.md
+++ b/docs/api/CheerioCrawler.md
@@ -77,7 +77,7 @@ await crawler.run();
* [CheerioCrawler](cheeriocrawler)
* [`new CheerioCrawler(options)`](#new_CheerioCrawler_new)
- * [`.run()`](#CheerioCrawler+run) ⇒ <code>Promise</code>
+ * [`.run()`](#CheerioCrawler+run) ⇒ `Promise`
<a name="new_CheerioCrawler_new"></a>
@@ -108,8 +108,7 @@ await crawler.run();
request: Request,
response: Object // An instance of Node&#39;s http.IncomingMessage object,
autoscaledPool: AutoscaledPool
-}
-</code></pre><p> With the <a href="request"><code>Request</code></a> object representing the URL to crawl.</p>
+}</code></pre><p> With the <a href="request"><code>Request</code></a> object representing the URL to crawl.</p>
<p> If the function returns a promise, it is awaited by the crawler.</p>
<p> If the function throws an exception, the crawler will try to re-crawl the
request later, up to <code>option.maxRequestRetries</code> times.
@@ -139,11 +138,15 @@ await crawler.run();
Provided <code>requestOptions</code> are added to internal defaults that cannot be overridden to ensure
the operation of <code>CheerioCrawler</code> and all its options. If you need more granular control over
your requests, use <a href="basiccrawler"><code>BasicCrawler</code></a>.</p>
-<p> The internal defaults include:</p>
-<pre><code> - `url`, `method`, `headers`: provided by `requestList` and/or `requestQueue`
- - `strictSSL`: use `options.ignoreSslErrors`
- - `proxy`: use `options.useApifyProxy` or `options.proxyUrls`
-</code></pre></td></tr><tr>
+<p> The mandatory internal defaults:</p>
+<pre><code> {
+ url,
+ method,
+ headers,
+ payload, // Are provided by RequestList and/or RequestQueue
+ strictSSL, // Use options.ignoreSslErrors
+ proxy, // Use options.useApifyProxy or options.proxyUrls
+ }</code></pre></td></tr><tr>
<td><code>[options.prepareRequestFunction]</code></td><td><code>function</code></td><td></td>
</tr>
<tr>
@@ -152,8 +155,7 @@ await crawler.run();
<p> The function receives the following object as an argument:</p>
<pre><code>{
request: Request
-}
-</code></pre><p> where the <a href="request"><code>Request</code></a> instance corresponds to the initialized request.</p>
+}</code></pre><p> where the <a href="request"><code>Request</code></a> instance corresponds to the initialized request.</p>
<p> The function should modify the properties of the passed <a href="request"><code>Request</code></a> instance
in place because there are already earlier references to it. Making a copy and returning it from
this function is therefore not supported, because it would create inconsistencies where
@@ -182,7 +184,7 @@ await crawler.run();
<a href="https://my.apify.com/proxy" target="_blank">Apify Proxy</a> for all connections.
For more information, see the <a href="https://apify.com/docs/proxy" target="_blank">documentation</a></p>
</td></tr><tr>
-<td><code>[options.apifyProxyGroups]</code></td><td><code>Array&lt;String&gt;</code></td><td></td>
+<td><code>[options.apifyProxyGroups]</code></td><td><code>Array<String></code></td><td></td>
</tr>
<tr>
<td colspan="3"><p>An array of proxy groups to be used
@@ -198,7 +200,7 @@ await crawler.run();
The identifier can only contain the following characters: <code>0-9</code>, <code>a-z</code>, <code>A-Z</code>, <code>&quot;.&quot;</code>, <code>&quot;_&quot;</code> and <code>&quot;~&quot;</code>.
Only applied if the <code>useApifyProxy</code> option is <code>true</code>.</p>
</td></tr><tr>
-<td><code>[options.proxyUrls]</code></td><td><code>Array&lt;String&gt;</code></td><td></td>
+<td><code>[options.proxyUrls]</code></td><td><code>Array<String></code></td><td></td>
</tr>
<tr>
<td colspan="3"><p>An array of custom proxy URLs to be used by the <code>CheerioCrawler</code> instance.
@@ -214,8 +216,7 @@ await crawler.run();
<pre><code>{
request: Request,
error: Error,
-}
-</code></pre><p> where the <a href="request"><code>Request</code></a> instance corresponds to the failed request, and the <code>Error</code> instance
+}</code></pre><p> where the <a href="request"><code>Request</code></a> instance corresponds to the failed request, and the <code>Error</code> instance
represents the last error thrown during processing of the request.</p>
<p> See <a href="https://github.com/apifytech/apify-js/blob/master/src/cheerio_crawler.js#L13">source code</a>
for the default implementation of this function.</p>
@@ -256,6 +257,6 @@ await crawler.run();
</table>
<a name="CheerioCrawler+run"></a>
-## `cheerioCrawler.run()` ⇒ <code>Promise</code>
+## `cheerioCrawler.run()` ⇒ `Promise`
Runs the crawler. Returns promise that gets resolved once all the requests got processed.
diff --git a/docs/api/Dataset.md b/docs/api/Dataset.md
index 7461b3c8c..8bd3b6127 100644
--- a/docs/api/Dataset.md
+++ b/docs/api/Dataset.md
@@ -52,17 +52,17 @@ await dataset.pushData([
* [Dataset](dataset)
- * [`.pushData(data)`](#Dataset+pushData) ⇒ <code>Promise</code>
- * [`.getData([options])`](#Dataset+getData) ⇒ <code>Promise&lt;(Array|String|Buffer)&gt;</code>
- * [`.getInfo()`](#Dataset+getInfo) ⇒ <code>Promise&lt;Object&gt;</code>
- * [`.forEach(iteratee, [options], [index])`](#Dataset+forEach) ⇒ <code>Promise</code>
- * [`.map(iteratee, options)`](#Dataset+map) ⇒ <code>Promise&lt;Array&gt;</code>
- * [`.reduce(iteratee, memo, options)`](#Dataset+reduce) ⇒ <code>Promise&lt;*&gt;</code>
- * [`.delete()`](#Dataset+delete) ⇒ <code>Promise</code>
+ * [`.pushData(data)`](#Dataset+pushData) ⇒ `Promise`
+ * [`.getData([options])`](#Dataset+getData) ⇒ `Promise<(Array|String|Buffer)>`
+ * [`.getInfo()`](#Dataset+getInfo) ⇒ `Promise<Object>`
+ * [`.forEach(iteratee, [options], [index])`](#Dataset+forEach) ⇒ `Promise`
+ * [`.map(iteratee, options)`](#Dataset+map) ⇒ `Promise<Array>`
+ * [`.reduce(iteratee, memo, options)`](#Dataset+reduce) ⇒ `Promise<*>`
+ * [`.delete()`](#Dataset+delete) ⇒ `Promise`
<a name="Dataset+pushData"></a>
-## `dataset.pushData(data)` ⇒ <code>Promise</code>
+## `dataset.pushData(data)` ⇒ `Promise`
Stores an object or an array of objects to the dataset.
The function returns a promise that resolves when the operation finishes.
It has no result, but throws on invalid args or other errors.
@@ -101,7 +101,7 @@ The objects must be serializable to JSON and the JSON representation of each obj
</table>
<a name="Dataset+getData"></a>
-## `dataset.getData([options])` ⇒ <code>Promise&lt;(Array|String|Buffer)&gt;</code>
+## `dataset.getData([options])` ⇒ `Promise<(Array|String|Buffer)>`
Returns items in the dataset based on the provided parameters.
<table>
@@ -192,7 +192,7 @@ Returns items in the dataset based on the provided parameters.
</table>
<a name="Dataset+getInfo"></a>
-## `dataset.getInfo()` ⇒ <code>Promise&lt;Object&gt;</code>
+## `dataset.getInfo()` ⇒ `Promise<Object>`
Returns an object containing general information about the dataset.
The function returns the same object as the Apify API Client's
@@ -217,7 +217,7 @@ API endpoint.
<a name="Dataset+forEach"></a>
-## `dataset.forEach(iteratee, [options], [index])` ⇒ <code>Promise</code>
+## `dataset.forEach(iteratee, [options], [index])` ⇒ `Promise`
Iterates over dataset items, yielding each in turn to an `iteratee` function.
Each invocation of `iteratee` is called with two arguments: `(item, index)`.
@@ -284,7 +284,7 @@ await dataset.forEach(async (item, index) => {
</table>
<a name="Dataset+map"></a>
-## `dataset.map(iteratee, options)` ⇒ <code>Promise&lt;Array&gt;</code>
+## `dataset.map(iteratee, options)` ⇒ `Promise<Array>`
Produces a new array of values by mapping each value in list through a transformation function `iteratee()`.
Each invocation of `iteratee()` is called with two arguments: `(element, index)`.
@@ -336,7 +336,7 @@ If `iteratee` returns a `Promise` then it's awaited before a next call.
</table>
<a name="Dataset+reduce"></a>
-## `dataset.reduce(iteratee, memo, options)` ⇒ <code>Promise&lt;*&gt;</code>
+## `dataset.reduce(iteratee, memo, options)` ⇒ `Promise<*>`
Reduces a list of values down to a single value.
Memo is the initial state of the reduction, and each successive step of it should be returned by `iteratee()`.
@@ -398,7 +398,7 @@ If `iteratee()` returns a `Promise` then it's awaited before a next call.
</table>
<a name="Dataset+delete"></a>
-## `dataset.delete()` ⇒ <code>Promise</code>
+## `dataset.delete()` ⇒ `Promise`
Removes the dataset either from the Apify cloud storage or from the local directory,
depending on the mode of operation.
diff --git a/docs/api/KeyValueStore.md b/docs/api/KeyValueStore.md
index b69991089..a04f774b1 100644
--- a/docs/api/KeyValueStore.md
+++ b/docs/api/KeyValueStore.md
@@ -67,15 +67,15 @@ await store.delete('some-key');
* [KeyValueStore](keyvaluestore)
- * [`.getValue(key)`](#KeyValueStore+getValue) ⇒ <code>Promise&lt;(Object|String|Buffer)&gt;</code>
- * [`.setValue(key, value, [options])`](#KeyValueStore+setValue) ⇒ <code>Promise</code>
- * [`.delete()`](#KeyValueStore+delete) ⇒ <code>Promise</code>
- * [`.getPublicUrl(key)`](#KeyValueStore+getPublicUrl) ⇒ <code>string</code>
- * [`.forEachKey(iteratee, [options])`](#KeyValueStore+forEachKey) ⇒ <code>Promise</code>
+ * [`.getValue(key)`](#KeyValueStore+getValue) ⇒ `Promise<(Object|String|Buffer)>`
+ * [`.setValue(key, value, [options])`](#KeyValueStore+setValue) ⇒ `Promise`
+ * [`.delete()`](#KeyValueStore+delete) ⇒ `Promise`
+ * [`.getPublicUrl(key)`](#KeyValueStore+getPublicUrl) ⇒ `string`
+ * [`.forEachKey(iteratee, [options])`](#KeyValueStore+forEachKey) ⇒ `Promise`
<a name="KeyValueStore+getValue"></a>
-## `keyValueStore.getValue(key)` ⇒ <code>Promise&lt;(Object|String|Buffer)&gt;</code>
+## `keyValueStore.getValue(key)` ⇒ `Promise<(Object|String|Buffer)>`
Gets a value from the key-value store.
The function returns a `Promise` that resolves to the record value,
@@ -98,7 +98,7 @@ const store = await Apify.openKeyValueStore('my-screenshots');
const buffer = await store.getValue('screenshot1.png');
```
-**Returns**: <code>Promise&lt;(Object|String|Buffer)&gt;</code> - Returns a promise that resolves to an object, string
+**Returns**: `Promise<(Object|String|Buffer)>` - Returns a promise that resolves to an object, string
or <a href="https://nodejs.org/api/buffer.html" target="_blank"><code>Buffer</code></a>, depending
on the MIME content type of the record.
<table>
@@ -118,7 +118,7 @@ const buffer = await store.getValue('screenshot1.png');
</table>
<a name="KeyValueStore+setValue"></a>
-## `keyValueStore.setValue(key, value, [options])` ⇒ <code>Promise</code>
+## `keyValueStore.setValue(key, value, [options])` ⇒ `Promise`
Saves or deletes a record in the key-value store.
The function returns a promise that resolves once the record has been saved or deleted.
@@ -188,13 +188,13 @@ otherwise the actor process might finish before the value is stored!
</table>
<a name="KeyValueStore+delete"></a>
-## `keyValueStore.delete()` ⇒ <code>Promise</code>
+## `keyValueStore.delete()` ⇒ `Promise`
Removes the key-value store either from the Apify cloud storage or from the local directory,
depending on the mode of operation.
<a name="KeyValueStore+getPublicUrl"></a>
-## `keyValueStore.getPublicUrl(key)` ⇒ <code>string</code>
+## `keyValueStore.getPublicUrl(key)` ⇒ `string`
Returns a URL for the given key that may be used to publicly
access the value in the remote key value store.
@@ -213,7 +213,7 @@ access the value in the remote key value store.
</table>
<a name="KeyValueStore+forEachKey"></a>
-## `keyValueStore.forEachKey(iteratee, [options])` ⇒ <code>Promise</code>
+## `keyValueStore.forEachKey(iteratee, [options])` ⇒ `Promise`
Iterates over key value store keys, yielding each in turn to an `iteratee` function.
Each invocation of `iteratee` is called with three arguments: `(key, index, info)`, where `key`
is the record key, `index` is a zero-based index of the key in the current iteration
diff --git a/docs/api/LiveViewServer.md b/docs/api/LiveViewServer.md
index 0e4c18772..9413e8a84 100644
--- a/docs/api/LiveViewServer.md
+++ b/docs/api/LiveViewServer.md
@@ -38,12 +38,12 @@ a better view into the browser, including DevTools, but `LiveViewServer` works t
* [LiveViewServer](liveviewserver)
* [`new LiveViewServer([options])`](#new_LiveViewServer_new)
- * [`.start()`](#LiveViewServer+start) ⇒ <code>Promise</code>
- * [`.stop()`](#LiveViewServer+stop) ⇒ <code>Promise</code>
- * [`.serve(page)`](#LiveViewServer+serve) ⇒ <code>Promise</code>
- * [`.getLastSnapshot()`](#LiveViewServer+getLastSnapshot) ⇒ <code>Object</code>
- * [`.isRunning()`](#LiveViewServer+isRunning) ⇒ <code>boolean</code>
- * [`.hasClients()`](#LiveViewServer+hasClients) ⇒ <code>boolean</code>
+ * [`.start()`](#LiveViewServer+start) ⇒ `Promise`
+ * [`.stop()`](#LiveViewServer+stop) ⇒ `Promise`
+ * [`.serve(page)`](#LiveViewServer+serve) ⇒ `Promise`
+ * [`.getLastSnapshot()`](#LiveViewServer+getLastSnapshot) ⇒ `Object`
+ * [`.isRunning()`](#LiveViewServer+isRunning) ⇒ `boolean`
+ * [`.hasClients()`](#LiveViewServer+hasClients) ⇒ `boolean`
<a name="new_LiveViewServer_new"></a>
@@ -92,19 +92,19 @@ a better view into the browser, including DevTools, but `LiveViewServer` works t
</table>
<a name="LiveViewServer+start"></a>
-## `liveViewServer.start()` ⇒ <code>Promise</code>
+## `liveViewServer.start()` ⇒ `Promise`
Starts the HTTP server with web socket connections enabled.
Snapshots will not be created until a client has connected.
<a name="LiveViewServer+stop"></a>
-## `liveViewServer.stop()` ⇒ <code>Promise</code>
+## `liveViewServer.stop()` ⇒ `Promise`
Prevents the server from receiving more connections. Existing connections
will not be terminated, but the server will not prevent a process exit.
<a name="LiveViewServer+serve"></a>
-## `liveViewServer.serve(page)` ⇒ <code>Promise</code>
+## `liveViewServer.serve(page)` ⇒ `Promise`
Serves a snapshot to all connected clients.
Screenshots are not served directly, only their index number
which is used by client to retrieve the screenshot.
@@ -126,10 +126,10 @@ Will time out and throw in `options.snapshotTimeoutSecs`.
</table>
<a name="LiveViewServer+getLastSnapshot"></a>
-## `liveViewServer.getLastSnapshot()` ⇒ <code>Object</code>
+## `liveViewServer.getLastSnapshot()` ⇒ `Object`
<a name="LiveViewServer+isRunning"></a>
-## `liveViewServer.isRunning()` ⇒ <code>boolean</code>
+## `liveViewServer.isRunning()` ⇒ `boolean`
<a name="LiveViewServer+hasClients"></a>
-## `liveViewServer.hasClients()` ⇒ <code>boolean</code>
+## `liveViewServer.hasClients()` ⇒ `boolean`
diff --git a/docs/api/PseudoUrl.md b/docs/api/PseudoUrl.md
index 245fa50a6..0d4dba8de 100644
--- a/docs/api/PseudoUrl.md
+++ b/docs/api/PseudoUrl.md
@@ -56,8 +56,8 @@ if (purl.matches('http://www.example.com/pages/my-awesome-page')) console.log('M
* [PseudoUrl](pseudourl)
* [`new PseudoUrl(purl, requestTemplate)`](#new_PseudoUrl_new)
- * [`.matches(url)`](#PseudoUrl+matches) ⇒ <code>Boolean</code>
- * [`.createRequest(url)`](#PseudoUrl+createRequest) ⇒ [<code>Request</code>](request)
+ * [`.matches(url)`](#PseudoUrl+matches) ⇒ `Boolean`
+ * [`.createRequest(url)`](#PseudoUrl+createRequest) ⇒ [`Request`](request)
<a name="new_PseudoUrl_new"></a>
@@ -86,10 +86,10 @@ if (purl.matches('http://www.example.com/pages/my-awesome-page')) console.log('M
</table>
<a name="PseudoUrl+matches"></a>
-## `pseudoUrl.matches(url)` ⇒ <code>Boolean</code>
+## `pseudoUrl.matches(url)` ⇒ `Boolean`
Determines whether a URL matches this pseudo-URL pattern.
-**Returns**: <code>Boolean</code> - Returns `true` if given URL matches pseudo-URL.
+**Returns**: `Boolean` - Returns `true` if given URL matches pseudo-URL.
<table>
<thead>
<tr>
@@ -106,7 +106,7 @@ Determines whether a URL matches this pseudo-URL pattern.
</table>
<a name="PseudoUrl+createRequest"></a>
-## `pseudoUrl.createRequest(url)` ⇒ [<code>Request</code>](request)
+## `pseudoUrl.createRequest(url)` ⇒ [`Request`](request)
Creates a Request object from a provided `requestTemplate` and given URL.
<table>
diff --git a/docs/api/PuppeteerCrawler.md b/docs/api/PuppeteerCrawler.md
index d7bd7fbd8..059741515 100644
--- a/docs/api/PuppeteerCrawler.md
+++ b/docs/api/PuppeteerCrawler.md
@@ -69,7 +69,7 @@ await crawler.run();
* [PuppeteerCrawler](puppeteercrawler)
* [`new PuppeteerCrawler(options)`](#new_PuppeteerCrawler_new)
- * [`.run()`](#PuppeteerCrawler+run) ⇒ <code>Promise</code>
+ * [`.run()`](#PuppeteerCrawler+run) ⇒ `Promise`
<a name="new_PuppeteerCrawler_new"></a>
@@ -99,8 +99,7 @@ await crawler.run();
page: Page,
puppeteerPool: PuppeteerPool,
autoscaledPool: AutoscaledPool
-}
-</code></pre><p> <code>request</code> is an instance of the <a href="request"><code>Request</code></a> object with details about the URL to open, HTTP method etc.
+}</code></pre><p> <code>request</code> is an instance of the <a href="request"><code>Request</code></a> object with details about the URL to open, HTTP method etc.
<code>response</code> is an instance of the <code>Puppeteer</code>
<a href="https://pptr.dev/#?product=Puppeteer&show=api-class-page" target="_blank"><code>Page</code></a>
<code>page</code> is an instance of the <code>Puppeteer</code>
@@ -154,8 +153,7 @@ await crawler.run();
<pre><code>{
request: Request,
error: Error,
-}
-</code></pre><p> Where the <a href="request"><code>Request</code></a> instance corresponds to the failed request, and the <code>Error</code> instance
+}</code></pre><p> Where the <a href="request"><code>Request</code></a> instance corresponds to the failed request, and the <code>Error</code> instance
represents the last error thrown during processing of the request.</p>
<p> See
<a href="https://github.com/apifytech/apify-js/blob/master/src/puppeteer_crawler.js#L11" target="_blank">source code</a>
@@ -216,6 +214,6 @@ await crawler.run();
</table>
<a name="PuppeteerCrawler+run"></a>
-## `puppeteerCrawler.run()` ⇒ <code>Promise</code>
+## `puppeteerCrawler.run()` ⇒ `Promise`
Runs the crawler. Returns promise that gets resolved once all the requests got processed.
diff --git a/docs/api/PuppeteerPool.md b/docs/api/PuppeteerPool.md
index 534a26737..1d8ad149f 100644
--- a/docs/api/PuppeteerPool.md
+++ b/docs/api/PuppeteerPool.md
@@ -42,10 +42,10 @@ await puppeteerPool.destroy();
* [PuppeteerPool](puppeteerpool)
* [`new PuppeteerPool([options])`](#new_PuppeteerPool_new)
- * [`.newPage()`](#PuppeteerPool+newPage) ⇒ <code>Promise&lt;Page&gt;</code>
- * [`.destroy()`](#PuppeteerPool+destroy) ⇒ <code>Promise</code>
- * [`.retire(browser)`](#PuppeteerPool+retire) ⇒ <code>Promise</code>
- * [`.recyclePage(page)`](#PuppeteerPool+recyclePage) ⇒ <code>Promise</code>
+ * [`.newPage()`](#PuppeteerPool+newPage) ⇒ `Promise<Page>`
+ * [`.destroy()`](#PuppeteerPool+destroy) ⇒ `Promise`
+ * [`.retire(browser)`](#PuppeteerPool+retire) ⇒ `Promise`
+ * [`.recyclePage(page)`](#PuppeteerPool+recyclePage) ⇒ `Promise`
<a name="new_PuppeteerPool_new"></a>
@@ -132,7 +132,7 @@ await puppeteerPool.destroy();
<p> Do not use the <code>options.recycleDiskCache</code> setting together with <code>--disk-cache-dir</code>
argument in <code>options.launchPuppeteerOptions.args</code>, the behavior is undefined.</p>
</td></tr><tr>
-<td><code>[options.proxyUrls]</code></td><td><code>Array&lt;String&gt;</code></td><td></td>
+<td><code>[options.proxyUrls]</code></td><td><code>Array<String></code></td><td></td>
</tr>
<tr>
<td colspan="3"><p>An array of custom proxy URLs to be used by the <code>PuppeteerPool</code> instance.
@@ -143,7 +143,7 @@ await puppeteerPool.destroy();
</table>
<a name="PuppeteerPool+newPage"></a>
-## `puppeteerPool.newPage()` ⇒ <code>Promise&lt;Page&gt;</code>
+## `puppeteerPool.newPage()` ⇒ `Promise<Page>`
Produces a new page instance either by reusing an idle page that currently isn't processing
any request or by spawning a new page (new browser tab) in one of the available
browsers when no idle pages are available.
@@ -152,12 +152,12 @@ To spawn a new browser tab for each page, set the `reusePages` constructor optio
<a name="PuppeteerPool+destroy"></a>
-## `puppeteerPool.destroy()` ⇒ <code>Promise</code>
+## `puppeteerPool.destroy()` ⇒ `Promise`
Closes all open browsers.
<a name="PuppeteerPool+retire"></a>
-## `puppeteerPool.retire(browser)` ⇒ <code>Promise</code>
+## `puppeteerPool.retire(browser)` ⇒ `Promise`
Manually retires a Puppeteer
<a href="https://pptr.dev/#?product=Puppeteer&show=api-class-browser" target="_blank"><code>Browser</code></a>
instance from the pool. The browser will continue to process open pages so that they may gracefully finish.
@@ -178,7 +178,7 @@ This is unlike `browser.close()` which will forcibly terminate the browser and a
</table>
<a name="PuppeteerPool+recyclePage"></a>
-## `puppeteerPool.recyclePage(page)` ⇒ <code>Promise</code>
+## `puppeteerPool.recyclePage(page)` ⇒ `Promise`
Closes the page, unless the `reuseTabs` option is set to true.
Then it would only flag the page for a future reuse, without actually closing it.
diff --git a/docs/api/Request.md b/docs/api/Request.md
index 867ad3da0..dd98f518a 100644
--- a/docs/api/Request.md
+++ b/docs/api/Request.md
@@ -84,7 +84,7 @@ const foo = request.userData.foo;
<tr>
<td colspan="3"><p>Indicates the number of times the crawling of the request has been retried on error.</p>
</td></tr><tr>
-<td><code>errorMessages</code></td><td><code>Array&lt;String&gt;</code></td>
+<td><code>errorMessages</code></td><td><code>Array<String></code></td>
</tr>
<tr>
<td colspan="3"><p>An array of error messages from request processing.</p>
@@ -165,8 +165,7 @@ to override the default behavior and specify which URLs shall be considered equa
<pre><code> {
Accept: &#39;text/html&#39;,
&#39;Content-Type&#39;: &#39;application/json&#39;
- }
-</code></pre></td></tr><tr>
+ }</code></pre></td></tr><tr>
<td><code>[options.userData]</code></td><td><code>Object</code></td><td><code>{}</code></td>
</tr>
<tr>
diff --git a/docs/api/RequestList.md b/docs/api/RequestList.md
index e3148655f..c69017d52 100644
--- a/docs/api/RequestList.md
+++ b/docs/api/RequestList.md
@@ -69,16 +69,16 @@ await requestList.reclaimRequest(request2);
* [RequestList](requestlist)
* [`new exports.RequestList(options)`](#new_RequestList_new)
- * [`.initialize()`](#RequestList+initialize) ⇒ <code>Promise</code>
- * [`.persistState()`](#RequestList+persistState) ⇒ <code>Promise</code>
- * [`.getState()`](#RequestList+getState) ⇒ <code>Object</code>
- * [`.isEmpty()`](#RequestList+isEmpty) ⇒ <code>Promise&lt;Boolean&gt;</code>
- * [`.isFinished()`](#RequestList+isFinished) ⇒ <code>Promise&lt;Boolean&gt;</code>
- * [`.fetchNextRequest()`](#RequestList+fetchNextRequest) ⇒ [<code>Promise&lt;Request&gt;</code>](request)
- * [`.markRequestHandled(request)`](#RequestList+markRequestHandled) ⇒ <code>Promise</code>
- * [`.reclaimRequest(request)`](#RequestList+reclaimRequest) ⇒ <code>Promise</code>
- * [`.length()`](#RequestList+length) ⇒ <code>Number</code>
- * [`.handledCount()`](#RequestList+handledCount) ⇒ <code>Number</code>
+ * [`.initialize()`](#RequestList+initialize) ⇒ `Promise`
+ * [`.persistState()`](#RequestList+persistState) ⇒ `Promise`
+ * [`.getState()`](#RequestList+getState) ⇒ `Object`
+ * [`.isEmpty()`](#RequestList+isEmpty) ⇒ `Promise<Boolean>`
+ * [`.isFinished()`](#RequestList+isFinished) ⇒ `Promise<Boolean>`
+ * [`.fetchNextRequest()`](#RequestList+fetchNextRequest) ⇒ [`Promise<Request>`](request)
+ * [`.markRequestHandled(request)`](#RequestList+markRequestHandled) ⇒ `Promise`
+ * [`.reclaimRequest(request)`](#RequestList+reclaimRequest) ⇒ `Promise`
+ * [`.length()`](#RequestList+length) ⇒ `Number`
+ * [`.handledCount()`](#RequestList+handledCount) ⇒ `Number`
<a name="new_RequestList_new"></a>
@@ -110,8 +110,7 @@ await requestList.reclaimRequest(request2);
{ method: &#39;GET&#39;, url: &#39;http://example.com/a/b&#39; },
// Batch import of URLs from a file hosted on the web
{ method: &#39;POST&#39;, requestsFromUrl: &#39;http://example.com/urls.txt&#39; },
-]
-</code></pre></td></tr><tr>
+]</code></pre></td></tr><tr>
<td><code>[options.persistStateKey]</code></td><td><code>String</code></td><td></td>
</tr>
<tr>
@@ -139,8 +138,7 @@ await requestList.reclaimRequest(request2);
&#39;unique-key-1&#39;: true,
&#39;unique-key-4&#39;: true,
},
-}
-</code></pre><p> Note that the preferred (and simpler) way to persist the state of crawling of the <code>RequestList</code>
+}</code></pre><p> Note that the preferred (and simpler) way to persist the state of crawling of the <code>RequestList</code>
is to use the <code>stateKeyPrefix</code> parameter instead.</p>
</td></tr><tr>
<td><code>[options.keepDuplicateUrls]</code></td><td><code>Boolean</code></td><td><code>false</code></td>
@@ -160,13 +158,13 @@ await requestList.reclaimRequest(request2);
</table>
<a name="RequestList+initialize"></a>
-## `requestList.initialize()` ⇒ <code>Promise</code>
+## `requestList.initialize()` ⇒ `Promise`
Loads all remote sources of URLs and potentially starts periodic state persistence.
This function must be called before you can start using the instance in a meaningful way.
<a name="RequestList+persistState"></a>
-## `requestList.persistState()` ⇒ <code>Promise</code>
+## `requestList.persistState()` ⇒ `Promise`
Persists the current state of the `RequestList` into the default [`KeyValueStore`](keyvaluestore).
The state is persisted automatically in regular intervals, but calling this method manually
is useful in cases where you want to have the most current state available after you pause
@@ -175,25 +173,25 @@ a server migration.
<a name="RequestList+getState"></a>
-## `requestList.getState()` ⇒ <code>Object</code>
+## `requestList.getState()` ⇒ `Object`
Returns an object representing the internal state of the `RequestList` instance.
Note that the object's fields can change in future releases.
<a name="RequestList+isEmpty"></a>
-## `requestList.isEmpty()` ⇒ <code>Promise&lt;Boolean&gt;</code>
+## `requestList.isEmpty()` ⇒ `Promise<Boolean>`
Resolves to `true` if the next call to [`fetchNextRequest`](#RequestList+fetchNextRequest) function
would return `null`, otherwise it resolves to `false`.
Note that even if the list is empty, there might be some pending requests currently being processed.
<a name="RequestList+isFinished"></a>
-## `requestList.isFinished()` ⇒ <code>Promise&lt;Boolean&gt;</code>
+## `requestList.isFinished()` ⇒ `Promise<Boolean>`
Returns `true` if all requests were already handled and there are no more left.
<a name="RequestList+fetchNextRequest"></a>
-## `requestList.fetchNextRequest()` ⇒ [<code>Promise&lt;Request&gt;</code>](request)
+## `requestList.fetchNextRequest()` ⇒ [`Promise<Request>`](request)
Gets the next [`Request`](request) to process. First, the function gets a request previously reclaimed
using the [`reclaimRequest`](#RequestList+reclaimRequest) function, if there is any.
Otherwise it gets the next request from sources.
@@ -203,7 +201,7 @@ requests to process.
<a name="RequestList+markRequestHandled"></a>
-## `requestList.markRequestHandled(request)` ⇒ <code>Promise</code>
+## `requestList.markRequestHandled(request)` ⇒ `Promise`
Marks request as handled after successful processing.
<table>
@@ -221,7 +219,7 @@ Marks request as handled after successful processing.
</table>
<a name="RequestList+reclaimRequest"></a>
-## `requestList.reclaimRequest(request)` ⇒ <code>Promise</code>
+## `requestList.reclaimRequest(request)` ⇒ `Promise`
Reclaims request to the list if its processing failed.
The request will become available in the next `this.fetchNextRequest()`.
@@ -240,11 +238,11 @@ The request will become available in the next `this.fetchNextRequest()`.
</table>
<a name="RequestList+length"></a>
-## `requestList.length()` ⇒ <code>Number</code>
+## `requestList.length()` ⇒ `Number`
Returns the total number of unique requests present in the `RequestList`.
<a name="RequestList+handledCount"></a>
-## `requestList.handledCount()` ⇒ <code>Number</code>
+## `requestList.handledCount()` ⇒ `Number`
Returns number of handled requests.
diff --git a/docs/api/RequestQueue.md b/docs/api/RequestQueue.md
index 72542eb06..47066d856 100644
--- a/docs/api/RequestQueue.md
+++ b/docs/api/RequestQueue.md
@@ -69,20 +69,20 @@ await queue.reclaimRequest(request2);
* [RequestQueue](requestqueue)
- * [`.addRequest(request, [options])`](#RequestQueue+addRequest) ⇒ [<code>QueueOperationInfo</code>](../typedefs/queueoperationinfo)
- * [`.getRequest(requestId)`](#RequestQueue+getRequest) ⇒ [<code>Promise&lt;Request&gt;</code>](request)
- * [`.fetchNextRequest()`](#RequestQueue+fetchNextRequest) ⇒ [<code>Promise&lt;Request&gt;</code>](request)
- * [`.markRequestHandled(request)`](#RequestQueue+markRequestHandled) ⇒ [<code>Promise&lt;QueueOperationInfo&gt;</code>](../typedefs/queueoperationinfo)
- * [`.reclaimRequest(request, [options])`](#RequestQueue+reclaimRequest) ⇒ [<code>Promise&lt;QueueOperationInfo&gt;</code>](../typedefs/queueoperationinfo)
- * [`.isEmpty()`](#RequestQueue+isEmpty) ⇒ <code>Promise&lt;Boolean&gt;</code>
- * [`.isFinished()`](#RequestQueue+isFinished) ⇒ <code>Promise&lt;Boolean&gt;</code>
- * [`.delete()`](#RequestQueue+delete) ⇒ <code>Promise</code>
- * [`.handledCount()`](#RequestQueue+handledCount) ⇒ <code>Promise&lt;number&gt;</code>
- * [`.getInfo()`](#RequestQueue+getInfo) ⇒ <code>Promise&lt;Object&gt;</code>
+ * [`.addRequest(request, [options])`](#RequestQueue+addRequest) ⇒ [`QueueOperationInfo`](../typedefs/queueoperationinfo)
+ * [`.getRequest(requestId)`](#RequestQueue+getRequest) ⇒ [`Promise<Request>`](request)
+ * [`.fetchNextRequest()`](#RequestQueue+fetchNextRequest) ⇒ [`Promise<Request>`](request)
+ * [`.markRequestHandled(request)`](#RequestQueue+markRequestHandled) ⇒ [`Promise<QueueOperationInfo>`](../typedefs/queueoperationinfo)
+ * [`.reclaimRequest(request, [options])`](#RequestQueue+reclaimRequest) ⇒ [`Promise<QueueOperationInfo>`](../typedefs/queueoperationinfo)
+ * [`.isEmpty()`](#RequestQueue+isEmpty) ⇒ `Promise<Boolean>`
+ * [`.isFinished()`](#RequestQueue+isFinished) ⇒ `Promise<Boolean>`
+ * [`.delete()`](#RequestQueue+delete) ⇒ `Promise`
+ * [`.handledCount()`](#RequestQueue+handledCount) ⇒ `Promise<number>`
+ * [`.getInfo()`](#RequestQueue+getInfo) ⇒ `Promise<Object>`
<a name="RequestQueue+addRequest"></a>
-## `requestQueue.addRequest(request, [options])` ⇒ [<code>QueueOperationInfo</code>](../typedefs/queueoperationinfo)
+## `requestQueue.addRequest(request, [options])` ⇒ [`QueueOperationInfo`](../typedefs/queueoperationinfo)
Adds a request to the queue.
If a request with the same `uniqueKey` property is already present in the queue,
@@ -114,10 +114,10 @@ it will not be updated. You can find out whether this happened from the resultin
</table>
<a name="RequestQueue+getRequest"></a>
-## `requestQueue.getRequest(requestId)` ⇒ [<code>Promise&lt;Request&gt;</code>](request)
+## `requestQueue.getRequest(requestId)` ⇒ [`Promise<Request>`](request)
Gets the request from the queue specified by ID.
-**Returns**: [<code>Promise&lt;Request&gt;</code>](request) - Returns the request object, or `null` if it was not found.
+**Returns**: [`Promise<Request>`](request) - Returns the request object, or `null` if it was not found.
<table>
<thead>
<tr>
@@ -134,13 +134,13 @@ Gets the request from the queue specified by ID.
</table>
<a name="RequestQueue+fetchNextRequest"></a>
-## `requestQueue.fetchNextRequest()` ⇒ [<code>Promise&lt;Request&gt;</code>](request)
+## `requestQueue.fetchNextRequest()` ⇒ [`Promise<Request>`](request)
Returns next request in the queue to be processed.
-**Returns**: [<code>Promise&lt;Request&gt;</code>](request) - Returns the request object, or `null` if there are no more pending requests.
+**Returns**: [`Promise<Request>`](request) - Returns the request object, or `null` if there are no more pending requests.
<a name="RequestQueue+markRequestHandled"></a>
-## `requestQueue.markRequestHandled(request)` ⇒ [<code>Promise&lt;QueueOperationInfo&gt;</code>](../typedefs/queueoperationinfo)
+## `requestQueue.markRequestHandled(request)` ⇒ [`Promise<QueueOperationInfo>`](../typedefs/queueoperationinfo)
Marks request handled after successful processing.
<table>
@@ -158,7 +158,7 @@ Marks request handled after successful processing.
</table>
<a name="RequestQueue+reclaimRequest"></a>
-## `requestQueue.reclaimRequest(request, [options])` ⇒ [<code>Promise&lt;QueueOperationInfo&gt;</code>](../typedefs/queueoperationinfo)
+## `requestQueue.reclaimRequest(request, [options])` ⇒ [`Promise<QueueOperationInfo>`](../typedefs/queueoperationinfo)
Reclaims failed request back to the queue, so that it can be processed later again.
The request record in the queue is updated using the provided `request` parameter.
For example, this lets you store the number of retries for the request.
@@ -188,32 +188,32 @@ For example, this lets you store the number of retries for the request.
</table>
<a name="RequestQueue+isEmpty"></a>
-## `requestQueue.isEmpty()` ⇒ <code>Promise&lt;Boolean&gt;</code>
+## `requestQueue.isEmpty()` ⇒ `Promise<Boolean>`
Resolves to `true` if the next call to [`fetchNextRequest`](#RequestQueue+fetchNextRequest) would return `null`, otherwise it resolves to `false`.
Note that even if the queue is empty, there might be some pending requests currently being processed.
If you need to ensure that there is no activity in the queue, use [`isFinished`](#RequestQueue+isFinished).
<a name="RequestQueue+isFinished"></a>
-## `requestQueue.isFinished()` ⇒ <code>Promise&lt;Boolean&gt;</code>
+## `requestQueue.isFinished()` ⇒ `Promise<Boolean>`
Resolves to `true` if all requests were already handled and there are no more left.
Due to the nature of distributed storage systems,
the function might occasionally return a false negative, but it will never return a false positive.
<a name="RequestQueue+delete"></a>
-## `requestQueue.delete()` ⇒ <code>Promise</code>
+## `requestQueue.delete()` ⇒ `Promise`
Removes the queue either from the Apify Cloud storage or from the local directory,
depending on the mode of operation.
<a name="RequestQueue+handledCount"></a>
-## `requestQueue.handledCount()` ⇒ <code>Promise&lt;number&gt;</code>
+## `requestQueue.handledCount()` ⇒ `Promise<number>`
Returns the number of handled requests.
<a name="RequestQueue+getInfo"></a>
-## `requestQueue.getInfo()` ⇒ <code>Promise&lt;Object&gt;</code>
+## `requestQueue.getInfo()` ⇒ `Promise<Object>`
Returns an object containing general information about the request queue.
The function returns the same object as the Apify API Client's
diff --git a/docs/api/Snapshotter.md b/docs/api/Snapshotter.md
index fbff90453..6418be907 100644
--- a/docs/api/Snapshotter.md
+++ b/docs/api/Snapshotter.md
@@ -30,12 +30,12 @@ typically received from the request queue, exceed the set limit within the set i
* [Snapshotter](snapshotter)
* [`new Snapshotter([options])`](#new_Snapshotter_new)
- * [`.start()`](#Snapshotter+start) ⇒ <code>Promise</code>
- * [`.stop()`](#Snapshotter+stop) ⇒ <code>Promise</code>
- * [`.getMemorySample([sampleDurationMillis])`](#Snapshotter+getMemorySample) ⇒ <code>Array</code>
- * [`.getEventLoopSample([sampleDurationMillis])`](#Snapshotter+getEventLoopSample) ⇒ <code>Array</code>
- * [`.getCpuSample([sampleDurationMillis])`](#Snapshotter+getCpuSample) ⇒ <code>Array</code>
- * [`.getClientSample(sampleDurationMillis)`](#Snapshotter+getClientSample) ⇒ <code>Array</code>
+ * [`.start()`](#Snapshotter+start) ⇒ `Promise`
+ * [`.stop()`](#Snapshotter+stop) ⇒ `Promise`
+ * [`.getMemorySample([sampleDurationMillis])`](#Snapshotter+getMemorySample) ⇒ `Array`
+ * [`.getEventLoopSample([sampleDurationMillis])`](#Snapshotter+getEventLoopSample) ⇒ `Array`
+ * [`.getCpuSample([sampleDurationMillis])`](#Snapshotter+getCpuSample) ⇒ `Array`
+ * [`.getClientSample(sampleDurationMillis)`](#Snapshotter+getClientSample) ⇒ `Array`
<a name="new_Snapshotter_new"></a>
@@ -114,17 +114,17 @@ typically received from the request queue, exceed the set limit within the set i
</table>
<a name="Snapshotter+start"></a>
-## `snapshotter.start()` ⇒ <code>Promise</code>
+## `snapshotter.start()` ⇒ `Promise`
Starts capturing snapshots at configured intervals.
<a name="Snapshotter+stop"></a>
-## `snapshotter.stop()` ⇒ <code>Promise</code>
+## `snapshotter.stop()` ⇒ `Promise`
Stops all resource capturing.
<a name="Snapshotter+getMemorySample"></a>
-## `snapshotter.getMemorySample([sampleDurationMillis])` ⇒ <code>Array</code>
+## `snapshotter.getMemorySample([sampleDurationMillis])` ⇒ `Array`
Returns a sample of latest memory snapshots, with the size of the sample defined
by the sampleDurationMillis parameter. If omitted, it returns a full snapshot history.
@@ -143,7 +143,7 @@ by the sampleDurationMillis parameter. If omitted, it returns a full snapshot hi
</table>
<a name="Snapshotter+getEventLoopSample"></a>
-## `snapshotter.getEventLoopSample([sampleDurationMillis])` ⇒ <code>Array</code>
+## `snapshotter.getEventLoopSample([sampleDurationMillis])` ⇒ `Array`
Returns a sample of latest event loop snapshots, with the size of the sample defined
by the sampleDurationMillis parameter. If omitted, it returns a full snapshot history.
@@ -162,7 +162,7 @@ by the sampleDurationMillis parameter. If omitted, it returns a full snapshot hi
</table>
<a name="Snapshotter+getCpuSample"></a>
-## `snapshotter.getCpuSample([sampleDurationMillis])` ⇒ <code>Array</code>
+## `snapshotter.getCpuSample([sampleDurationMillis])` ⇒ `Array`
Returns a sample of latest CPU snapshots, with the size of the sample defined
by the sampleDurationMillis parameter. If omitted, it returns a full snapshot history.
@@ -181,7 +181,7 @@ by the sampleDurationMillis parameter. If omitted, it returns a full snapshot hi
</table>
<a name="Snapshotter+getClientSample"></a>
-## `snapshotter.getClientSample(sampleDurationMillis)` ⇒ <code>Array</code>
+## `snapshotter.getClientSample(sampleDurationMillis)` ⇒ `Array`
Returns a sample of latest Client snapshots, with the size of the sample defined
by the sampleDurationMillis parameter. If omitted, it returns a full snapshot history.
diff --git a/docs/api/SystemStatus.md b/docs/api/SystemStatus.md
index c72658e66..e3077edc2 100644
--- a/docs/api/SystemStatus.md
+++ b/docs/api/SystemStatus.md
@@ -27,8 +27,8 @@ in the [`Snapshotter`](snapshotter) instance.
* [SystemStatus](systemstatus)
* [`new SystemStatus([options])`](#new_SystemStatus_new)
- * [`.getCurrentStatus()`](#SystemStatus+getCurrentStatus) ⇒ <code>Object</code>
- * [`.getHistoricalStatus()`](#SystemStatus+getHistoricalStatus) ⇒ <code>Object</code>
+ * [`.getCurrentStatus()`](#SystemStatus+getCurrentStatus) ⇒ `Object`
+ * [`.getHistoricalStatus()`](#SystemStatus+getHistoricalStatus) ⇒ `Object`
<a name="new_SystemStatus_new"></a>
@@ -80,7 +80,7 @@ in the [`Snapshotter`](snapshotter) instance.
</table>
<a name="SystemStatus+getCurrentStatus"></a>
-## `systemStatus.getCurrentStatus()` ⇒ <code>Object</code>
+## `systemStatus.getCurrentStatus()` ⇒ `Object`
Returns an object with the following structure:
```javascript
@@ -98,7 +98,7 @@ and `true` otherwise.
<a name="SystemStatus+getHistoricalStatus"></a>
-## `systemStatus.getHistoricalStatus()` ⇒ <code>Object</code>
+## `systemStatus.getHistoricalStatus()` ⇒ `Object`
Returns an object with the following structure:
```javascript
diff --git a/docs/api/log.md b/docs/api/log.md
index 50bd38338..1a2d4a348 100644
--- a/docs/api/log.md
+++ b/docs/api/log.md
@@ -50,8 +50,8 @@ environment variable, such as `APIFY_LOG_LEVEL=DEBUG`. This way, no code changes
are necessary to turn on your debug messages and start debugging right away.
-* [`log`](#log) : <code>object</code>
- * [`.LEVELS`](#log.LEVELS) : <code>Object</code>
+* [`log`](#log) : `object`
+ * [`.LEVELS`](#log.LEVELS) : `Object`
* [`.setLevel(level)`](#log.setLevel)
* [`.getLevel()`](#log.getLevel)
* [`.debug(message, [data])`](#log.debug)
@@ -62,7 +62,7 @@ are necessary to turn on your debug messages and start debugging right away.
<a name="log.LEVELS"></a>
-## `log.LEVELS` : <code>Object</code>
+## `log.LEVELS` : `Object`
Map of available log levels that's useful for easy setting of appropriate log levels.
Each log level is represented internally by a number. Eg. `log.LEVELS.DEBUG === 5`.
diff --git a/docs/api/puppeteer.md b/docs/api/puppeteer.md
index d9d59a602..a7989bc77 100644
--- a/docs/api/puppeteer.md
+++ b/docs/api/puppeteer.md
@@ -23,20 +23,20 @@ await puppeteer.injectJQuery(page);
```
-* [`puppeteer`](#puppeteer) : <code>object</code>
- * [`.addInterceptRequestHandler`](#puppeteer.addInterceptRequestHandler) ⇒ <code>Promise</code>
- * [`.removeInterceptRequestHandler`](#puppeteer.removeInterceptRequestHandler) ⇒ <code>Promise</code>
- * [`.gotoExtended`](#puppeteer.gotoExtended) ⇒ <code>Promise&lt;Response&gt;</code>
- * [`.injectFile(page, filePath, [options])`](#puppeteer.injectFile) ⇒ <code>Promise</code>
- * [`.injectJQuery(page)`](#puppeteer.injectJQuery) ⇒ <code>Promise</code>
- * [`.injectUnderscore(page)`](#puppeteer.injectUnderscore) ⇒ <code>Promise</code>
- * [`.blockResources(page, [resourceTypes])`](#puppeteer.blockResources) ⇒ <code>Promise</code>
- * [`.cacheResponses(page, cache, responseUrlRules)`](#puppeteer.cacheResponses) ⇒ <code>Promise</code>
- * [`.compileScript(scriptString, context)`](#puppeteer.compileScript) ⇒ <code>function</code>
+* [`puppeteer`](#puppeteer) : `object`
+ * [`.addInterceptRequestHandler`](#puppeteer.addInterceptRequestHandler) ⇒ `Promise`
+ * [`.removeInterceptRequestHandler`](#puppeteer.removeInterceptRequestHandler) ⇒ `Promise`
+ * [`.gotoExtended`](#puppeteer.gotoExtended) ⇒ `Promise<Response>`
+ * [`.injectFile(page, filePath, [options])`](#puppeteer.injectFile) ⇒ `Promise`
+ * [`.injectJQuery(page)`](#puppeteer.injectJQuery) ⇒ `Promise`
+ * [`.injectUnderscore(page)`](#puppeteer.injectUnderscore) ⇒ `Promise`
+ * [`.blockResources(page, [resourceTypes])`](#puppeteer.blockResources) ⇒ `Promise`
+ * [`.cacheResponses(page, cache, responseUrlRules)`](#puppeteer.cacheResponses) ⇒ `Promise`
+ * [`.compileScript(scriptString, context)`](#puppeteer.compileScript) ⇒ `function`
<a name="puppeteer.addInterceptRequestHandler"></a>
-## `puppeteer.addInterceptRequestHandler` ⇒ <code>Promise</code>
+## `puppeteer.addInterceptRequestHandler` ⇒ `Promise`
Adds request interception handler in similar to `page.on('request', handler);` but in addition to that
supports multiple parallel handlers.
@@ -103,7 +103,7 @@ await page.goto('http://example.com');
</table>
<a name="puppeteer.removeInterceptRequestHandler"></a>
-## `puppeteer.removeInterceptRequestHandler` ⇒ <code>Promise</code>
+## `puppeteer.removeInterceptRequestHandler` ⇒ `Promise`
Removes request interception handler for given page.
<table>
@@ -127,7 +127,7 @@ Removes request interception handler for given page.
</table>
<a name="puppeteer.gotoExtended"></a>
-## `puppeteer.gotoExtended` ⇒ <code>Promise&lt;Response&gt;</code>
+## `puppeteer.gotoExtended` ⇒ `Promise<Response>`
Extended version of Puppeteer's `page.goto()` allowing to perform requests with HTTP method other than GET,
with custom headers and POST payload. URL, method, headers and payload are taken from
request parameter that must be an instance of Apify.Request class.
@@ -157,7 +157,7 @@ request parameter that must be an instance of Apify.Request class.
</table>
<a name="puppeteer.injectFile"></a>
-## `puppeteer.injectFile(page, filePath, [options])` ⇒ <code>Promise</code>
+## `puppeteer.injectFile(page, filePath, [options])` ⇒ `Promise`
Injects a JavaScript file into a Puppeteer page.
Unlike Puppeteer's `addScriptTag` function, this function works on pages
with arbitrary Cross-Origin Resource Sharing (CORS) policies.
@@ -196,7 +196,7 @@ File contents are cached for up to 10 files to limit file system access.
</table>
<a name="puppeteer.injectJQuery"></a>
-## `puppeteer.injectJQuery(page)` ⇒ <code>Promise</code>
+## `puppeteer.injectJQuery(page)` ⇒ `Promise`
Injects the <a href="https://jquery.com/" target="_blank"><code>jQuery</code></a> library into a Puppeteer page.
jQuery is often useful for various web scraping and crawling tasks.
For example, it can help extract text from HTML elements using CSS selectors.
@@ -235,7 +235,7 @@ function in any way.
</table>
<a name="puppeteer.injectUnderscore"></a>
-## `puppeteer.injectUnderscore(page)` ⇒ <code>Promise</code>
+## `puppeteer.injectUnderscore(page)` ⇒ `Promise`
Injects the <a href="https://underscorejs.org/" target="_blank"><code>Underscore.js</code></a> library into a Puppeteer page.
Beware that the injected Underscore object will be set to the `window._` variable and thus it might cause conflicts with
@@ -268,7 +268,7 @@ const escapedHtml = await page.evaluate(() => {
</table>
<a name="puppeteer.blockResources"></a>
-## `puppeteer.blockResources(page, [resourceTypes])` ⇒ <code>Promise</code>
+## `puppeteer.blockResources(page, [resourceTypes])` ⇒ `Promise`
Forces the Puppeteer browser tab to block loading certain HTTP resources.
This is useful to speed up crawling of websites, since it reduces the amount
of data that need to be downloaded from the web.
@@ -317,7 +317,7 @@ await page.goto('https://www.example.com');
<tr>
<td colspan="3"><p>Puppeteer <a href="https://pptr.dev/#?product=Puppeteer&show=api-class-page" target="_blank"><code>Page</code></a> object.</p>
</td></tr><tr>
-<td><code>[resourceTypes]</code></td><td><code>Array&lt;String&gt;</code></td><td><code>[&#x27;stylesheet&#x27;, &#x27;font&#x27;, &#x27;image&#x27;, &#x27;media&#x27;]</code></td>
+<td><code>[resourceTypes]</code></td><td><code>Array<String></code></td><td><code>[&#x27;stylesheet&#x27;, &#x27;font&#x27;, &#x27;image&#x27;, &#x27;media&#x27;]</code></td>
</tr>
<tr>
<td colspan="3"><p>Array of resource types to block.</p>
@@ -325,7 +325,7 @@ await page.goto('https://www.example.com');
</table>
<a name="puppeteer.cacheResponses"></a>
-## `puppeteer.cacheResponses(page, cache, responseUrlRules)` ⇒ <code>Promise</code>
+## `puppeteer.cacheResponses(page, cache, responseUrlRules)` ⇒ `Promise`
Enables caching of intercepted responses into a provided object. Automatically enables request interception in Puppeteer.
*IMPORTANT*: Caching responses stores them to memory, so too loose rules could cause memory leaks for longer running crawlers.
This issue should be resolved or atleast mitigated in future iterations of this feature.
@@ -348,7 +348,7 @@ Enables caching of intercepted responses into a provided object. Automatically e
<tr>
<td colspan="3"><p>Object in which responses are stored</p>
</td></tr><tr>
-<td><code>responseUrlRules</code></td><td><code>Array&lt;(String|RegExp)&gt;</code></td>
+<td><code>responseUrlRules</code></td><td><code>Array<(String|RegExp)></code></td>
</tr>
<tr>
<td colspan="3"><p>List of rules that are used to check if the response should be cached.
@@ -357,7 +357,7 @@ Enables caching of intercepted responses into a provided object. Automatically e
</table>
<a name="puppeteer.compileScript"></a>
-## `puppeteer.compileScript(scriptString, context)` ⇒ <code>function</code>
+## `puppeteer.compileScript(scriptString, context)` ⇒ `function`
Compiles a Puppeteer script into an async function that may be executed at any time
by providing it with the following object:
```
@@ -383,7 +383,7 @@ Custom context may also be provided using the `context` parameter. To improve se
make sure to only pass the really necessary objects to the context. Preferably making
secured copies beforehand.
-**Returns**: <code>function</code> - `async ({ page, request }) => { scriptString }`
+**Returns**: `function` - `async ({ page, request }) => { scriptString }`
<table>
<thead>
<tr>
diff --git a/docs/api/social.md b/docs/api/social.md
index 800e907f2..9b3499ec3 100644
--- a/docs/api/social.md
+++ b/docs/api/social.md
@@ -16,26 +16,26 @@ const emails = Apify.utils.social.emailsFromText('[email protected] bob@example.
```
-* [`social`](#social) : <code>object</code>
- * [`.LINKEDIN_REGEX`](#social.LINKEDIN_REGEX) : <code>RegExp</code>
- * [`.LINKEDIN_REGEX_GLOBAL`](#social.LINKEDIN_REGEX_GLOBAL) : <code>RegExp</code>
- * [`.INSTAGRAM_REGEX`](#social.INSTAGRAM_REGEX) : <code>RegExp</code>
- * [`.INSTAGRAM_REGEX_GLOBAL`](#social.INSTAGRAM_REGEX_GLOBAL) : <code>RegExp</code>
- * [`.TWITTER_REGEX`](#social.TWITTER_REGEX) : <code>RegExp</code>
- * [`.TWITTER_REGEX_GLOBAL`](#social.TWITTER_REGEX_GLOBAL) : <code>RegExp</code>
- * [`.FACEBOOK_REGEX`](#social.FACEBOOK_REGEX) : <code>RegExp</code>
- * [`.FACEBOOK_REGEX_GLOBAL`](#social.FACEBOOK_REGEX_GLOBAL) : <code>RegExp</code>
- * [`.EMAIL_REGEX`](#social.EMAIL_REGEX) : <code>RegExp</code>
- * [`.EMAIL_REGEX_GLOBAL`](#social.EMAIL_REGEX_GLOBAL) : <code>RegExp</code>
- * [`.emailsFromText(text)`](#social.emailsFromText) ⇒ <code>Array&lt;String&gt;</code>
- * [`.emailsFromUrls(urls)`](#social.emailsFromUrls) ⇒ <code>Array&lt;String&gt;</code>
- * [`.phonesFromText(text)`](#social.phonesFromText) ⇒ <code>Array&lt;String&gt;</code>
- * [`.phonesFromUrls(urls)`](#social.phonesFromUrls) ⇒ <code>Array&lt;String&gt;</code>
- * [`.parseHandlesFromHtml(html, data)`](#social.parseHandlesFromHtml) ⇒ <code>*</code>
+* [`social`](#social) : `object`
+ * [`.LINKEDIN_REGEX`](#social.LINKEDIN_REGEX) : `RegExp`
+ * [`.LINKEDIN_REGEX_GLOBAL`](#social.LINKEDIN_REGEX_GLOBAL) : `RegExp`
+ * [`.INSTAGRAM_REGEX`](#social.INSTAGRAM_REGEX) : `RegExp`
+ * [`.INSTAGRAM_REGEX_GLOBAL`](#social.INSTAGRAM_REGEX_GLOBAL) : `RegExp`
+ * [`.TWITTER_REGEX`](#social.TWITTER_REGEX) : `RegExp`
+ * [`.TWITTER_REGEX_GLOBAL`](#social.TWITTER_REGEX_GLOBAL) : `RegExp`
+ * [`.FACEBOOK_REGEX`](#social.FACEBOOK_REGEX) : `RegExp`
+ * [`.FACEBOOK_REGEX_GLOBAL`](#social.FACEBOOK_REGEX_GLOBAL) : `RegExp`
+ * [`.EMAIL_REGEX`](#social.EMAIL_REGEX) : `RegExp`
+ * [`.EMAIL_REGEX_GLOBAL`](#social.EMAIL_REGEX_GLOBAL) : `RegExp`
+ * [`.emailsFromText(text)`](#social.emailsFromText) ⇒ `Array<String>`
+ * [`.emailsFromUrls(urls)`](#social.emailsFromUrls) ⇒ `Array<String>`
+ * [`.phonesFromText(text)`](#social.phonesFromText) ⇒ `Array<String>`
+ * [`.phonesFromUrls(urls)`](#social.phonesFromUrls) ⇒ `Array<String>`
+ * [`.parseHandlesFromHtml(html, data)`](#social.parseHandlesFromHtml) ⇒ `*`
<a name="social.LINKEDIN_REGEX"></a>
-## `social.LINKEDIN_REGEX` : <code>RegExp</code>
+## `social.LINKEDIN_REGEX` : `RegExp`
Regular expression to exactly match a single LinkedIn profile URL.
It has the following form: `/^...$/i` and matches URLs such as:
```
@@ -59,7 +59,7 @@ if (Apify.utils.social.LINKEDIN_REGEX.test('https://www.linkedin.com/in/alan-tur
<a name="social.LINKEDIN_REGEX_GLOBAL"></a>
-## `social.LINKEDIN_REGEX_GLOBAL` : <code>RegExp</code>
+## `social.LINKEDIN_REGEX_GLOBAL` : `RegExp`
Regular expression to find multiple LinkedIn profile URLs in a text or HTML.
It has the following form: `/.../ig` and matches URLs such as:
```
@@ -86,7 +86,7 @@ if (matches) console.log(`${matches.length} LinkedIn profiles found!`);
<a name="social.INSTAGRAM_REGEX"></a>
-## `social.INSTAGRAM_REGEX` : <code>RegExp</code>
+## `social.INSTAGRAM_REGEX` : `RegExp`
Regular expression to exactly match a single Instagram profile URL.
It has the following form: `/^...$/i` and matches URLs such as:
```
@@ -110,7 +110,7 @@ if (Apify.utils.social.INSTAGRAM_REGEX_STRING.test('https://www.instagram.com/ol
<a name="social.INSTAGRAM_REGEX_GLOBAL"></a>
-## `social.INSTAGRAM_REGEX_GLOBAL` : <code>RegExp</code>
+## `social.INSTAGRAM_REGEX_GLOBAL` : `RegExp`
Regular expression to find multiple Instagram profile URLs in a text or HTML.
It has the following form: `/.../ig` and matches URLs such as:
```
@@ -137,7 +137,7 @@ if (matches) console.log(`${matches.length} Instagram profiles found!`);
<a name="social.TWITTER_REGEX"></a>
-## `social.TWITTER_REGEX` : <code>RegExp</code>
+## `social.TWITTER_REGEX` : `RegExp`
Regular expression to exactly match a single Twitter profile URL.
It has the following form: `/^...$/i` and matches URLs such as:
```
@@ -160,7 +160,7 @@ if (Apify.utils.social.TWITTER_REGEX_STRING.test('https://www.twitter.com/apify'
<a name="social.TWITTER_REGEX_GLOBAL"></a>
-## `social.TWITTER_REGEX_GLOBAL` : <code>RegExp</code>
+## `social.TWITTER_REGEX_GLOBAL` : `RegExp`
Regular expression to find multiple Twitter profile URLs in a text or HTML.
It has the following form: `/.../ig` and matches URLs such as:
```
@@ -186,7 +186,7 @@ if (matches) console.log(`${matches.length} Twitter profiles found!`);
<a name="social.FACEBOOK_REGEX"></a>
-## `social.FACEBOOK_REGEX` : <code>RegExp</code>
+## `social.FACEBOOK_REGEX` : `RegExp`
Regular expression to exactly match a single Facebook profile URL.
It has the following form: `/^...$/i` and matches URLs such as:
```
@@ -211,7 +211,7 @@ if (Apify.utils.social.FACEBOOK_REGEX_STRING.test('https://www.facebook.com/apif
<a name="social.FACEBOOK_REGEX_GLOBAL"></a>
-## `social.FACEBOOK_REGEX_GLOBAL` : <code>RegExp</code>
+## `social.FACEBOOK_REGEX_GLOBAL` : `RegExp`
Regular expression to find multiple Facebook profile URLs in a text or HTML.
It has the following form: `/.../ig` and matches URLs such as:
```
@@ -238,23 +238,23 @@ if (matches) console.log(`${matches.length} Facebook profiles found!`);
<a name="social.EMAIL_REGEX"></a>
-## `social.EMAIL_REGEX` : <code>RegExp</code>
+## `social.EMAIL_REGEX` : `RegExp`
Regular expression to exactly match a single email address.
It has the following form: `/^...$/i`.
<a name="social.EMAIL_REGEX_GLOBAL"></a>
-## `social.EMAIL_REGEX_GLOBAL` : <code>RegExp</code>
+## `social.EMAIL_REGEX_GLOBAL` : `RegExp`
Regular expression to find multiple email addresses in a text.
It has the following form: `/.../ig`.
<a name="social.emailsFromText"></a>
-## `social.emailsFromText(text)` ⇒ <code>Array&lt;String&gt;</code>
+## `social.emailsFromText(text)` ⇒ `Array<String>`
The function extracts email addresses from a plain text.
Note that the function preserves the order of emails and keep duplicates.
-**Returns**: <code>Array&lt;String&gt;</code> - Array of emails addresses found.
+**Returns**: `Array<String>` - Array of emails addresses found.
If no emails are found, the function returns an empty array.
<table>
<thead>
@@ -272,12 +272,12 @@ If no emails are found, the function returns an empty array.
</table>
<a name="social.emailsFromUrls"></a>
-## `social.emailsFromUrls(urls)` ⇒ <code>Array&lt;String&gt;</code>
+## `social.emailsFromUrls(urls)` ⇒ `Array<String>`
The function extracts email addresses from a list of URLs.
Basically it looks for all `mailto:` URLs and returns valid email addresses from them.
Note that the function preserves the order of emails and keep duplicates.
-**Returns**: <code>Array&lt;String&gt;</code> - Array of emails addresses found.
+**Returns**: `Array<String>` - Array of emails addresses found.
If no emails are found, the function returns an empty array.
<table>
<thead>
@@ -287,7 +287,7 @@ If no emails are found, the function returns an empty array.
</thead>
<tbody>
<tr>
-<td><code>urls</code></td><td><code>Array&lt;String&gt;</code></td>
+<td><code>urls</code></td><td><code>Array<String></code></td>
</tr>
<tr>
<td colspan="3"><p>Array of URLs.</p>
@@ -295,12 +295,12 @@ If no emails are found, the function returns an empty array.
</table>
<a name="social.phonesFromText"></a>
-## `social.phonesFromText(text)` ⇒ <code>Array&lt;String&gt;</code>
+## `social.phonesFromText(text)` ⇒ `Array<String>`
The function attempts to extract phone numbers from a text. Please note that
the results might not be accurate, since phone numbers appear in a large variety of formats and conventions.
If you encounter some problems, please [file an issue](https://github.com/apifytech/apify-js/issues).
-**Returns**: <code>Array&lt;String&gt;</code> - Array of phone numbers found.
+**Returns**: `Array<String>` - Array of phone numbers found.
If no phone numbers are found, the function returns an empty array.
<table>
<thead>
@@ -318,11 +318,11 @@ If no phone numbers are found, the function returns an empty array.
</table>
<a name="social.phonesFromUrls"></a>
-## `social.phonesFromUrls(urls)` ⇒ <code>Array&lt;String&gt;</code>
+## `social.phonesFromUrls(urls)` ⇒ `Array<String>`
Finds phone number links in an array of URLs and extracts the phone numbers from them.
Note that the phone number links look like `tel://123456789`, `tel:/123456789` or `tel:123456789`.
-**Returns**: <code>Array&lt;String&gt;</code> - Array of phone numbers found.
+**Returns**: `Array<String>` - Array of phone numbers found.
If no phone numbers are found, the function returns an empty array.
<table>
<thead>
@@ -332,7 +332,7 @@ If no phone numbers are found, the function returns an empty array.
</thead>
<tbody>
<tr>
-<td><code>urls</code></td><td><code>Array&lt;String&gt;</code></td>
+<td><code>urls</code></td><td><code>Array<String></code></td>
</tr>
<tr>
<td colspan="3"><p>Array of URLs.</p>
@@ -340,7 +340,7 @@ If no phone numbers are found, the function returns an empty array.
</table>
<a name="social.parseHandlesFromHtml"></a>
-## `social.parseHandlesFromHtml(html, data)` ⇒ <code>*</code>
+## `social.parseHandlesFromHtml(html, data)` ⇒ `*`
The function attempts to extract emails, phone numbers and social profile URLs from a HTML document,
specifically LinkedIn, Twitter, Instagram and Facebook profile URLs.
The function removes duplicates from the resulting arrays and sorts the items alphabetically.
@@ -377,7 +377,7 @@ console.log('Social handles:');
console.dir(result);
```
-**Returns**: <code>*</code> - An object with the social handles.
+**Returns**: `*` - An object with the social handles.
<table>
<thead>
<tr>
diff --git a/docs/api/utils.md b/docs/api/utils.md
index 0906deda9..a8e2cc911 100644
--- a/docs/api/utils.md
+++ b/docs/api/utils.md
@@ -18,20 +18,20 @@ await Apify.utils.sleep(1500);
```
-* [`utils`](#utils) : <code>object</code>
- * [`.enqueueLinks`](#utils.enqueueLinks) ⇒ <code>Promise&lt;Array&lt;QueueOperationInfo&gt;&gt;</code>
- * [`.sleep`](#utils.sleep) ⇒ <code>Promise</code>
+* [`utils`](#utils) : `object`
+ * [`.enqueueLinks`](#utils.enqueueLinks) ⇒ `Promise<Array<QueueOperationInfo>>`
+ * [`.sleep`](#utils.sleep) ⇒ `Promise`
* [`.URL_NO_COMMAS_REGEX`](#utils.URL_NO_COMMAS_REGEX)
* [`.URL_WITH_COMMAS_REGEX`](#utils.URL_WITH_COMMAS_REGEX)
- * [`.isDocker()`](#utils.isDocker) ⇒ <code>Promise</code>
- * [`.downloadListOfUrls(options)`](#utils.downloadListOfUrls) ⇒ <code>Promise&lt;Array&lt;String&gt;&gt;</code>
- * [`.extractUrls(string, [urlRegExp])`](#utils.extractUrls) ⇒ <code>Array&lt;String&gt;</code>
- * [`.getRandomUserAgent()`](#utils.getRandomUserAgent) ⇒ <code>String</code>
- * [`.htmlToText(html)`](#utils.htmlToText) ⇒ <code>String</code>
+ * [`.isDocker()`](#utils.isDocker) ⇒ `Promise`
+ * [`.downloadListOfUrls(options)`](#utils.downloadListOfUrls) ⇒ `Promise<Array<String>>`
+ * [`.extractUrls(string, [urlRegExp])`](#utils.extractUrls) ⇒ `Array<String>`
+ * [`.getRandomUserAgent()`](#utils.getRandomUserAgent) ⇒ `String`
+ * [`.htmlToText(html)`](#utils.htmlToText) ⇒ `String`
<a name="utils.enqueueLinks"></a>
-## `utils.enqueueLinks` ⇒ <code>Promise&lt;Array&lt;QueueOperationInfo&gt;&gt;</code>
+## `utils.enqueueLinks` ⇒ `Promise<Array<QueueOperationInfo>>`
The function finds elements matching a specific CSS selector (HTML anchor (`<a>`) by default)
either in a Puppeteer page, or in a Cheerio object (parsed HTML),
and enqueues the corresponding links to the provided [`RequestQueue`](requestqueue).
@@ -62,7 +62,7 @@ await Apify.utils.enqueueLinks({
});
```
-**Returns**: <code>Promise&lt;Array&lt;QueueOperationInfo&gt;&gt;</code> - Promise that resolves to an array of [`QueueOperationInfo`](../typedefs/queueoperationinfo) objects.
+**Returns**: `Promise<Array<QueueOperationInfo>>` - Promise that resolves to an array of [`QueueOperationInfo`](../typedefs/queueoperationinfo) objects.
<table>
<thead>
<tr>
@@ -105,7 +105,7 @@ await Apify.utils.enqueueLinks({
<td colspan="3"><p>A base URL that will be used to resolve relative URLs when using Cheerio. Ignored when using Puppeteer,
since the relative URL resolution is done inside the browser automatically.</p>
</td></tr><tr>
-<td><code>[options.pseudoUrls]</code></td><td><code>Array&lt;Object&gt;</code> | <code>Array&lt;String&gt;</code></td><td></td>
+<td><code>[options.pseudoUrls]</code></td><td><code>Array<Object&gt;</code> | <code>Array.&lt;String></code></td><td></td>
</tr>
<tr>
<td colspan="3"><p>An array of <a href="pseudourl"><code>PseudoUrl</code></a>s matching the URLs to be enqueued,
@@ -127,23 +127,20 @@ await Apify.utils.enqueueLinks({
{
name: &#39;John&#39;,
surname: &#39;Doe&#39;,
-}
-</code></pre><pre><code>// userData
+}</code></pre><pre><code>// userData
{
name: &#39;Albert&#39;,
age: 31
-}
-</code></pre><pre><code>// Enqueued request.userData
+}</code></pre><pre><code>// Enqueued request.userData
{
name: &#39;Albert&#39;,
surname: &#39;Doe&#39;,
age: 31,
-}
-</code></pre></td></tr></tbody>
+}</code></pre></td></tr></tbody>
</table>
<a name="utils.sleep"></a>
-## `utils.sleep` ⇒ <code>Promise</code>
+## `utils.sleep` ⇒ `Promise`
Returns a `Promise` that resolves after a specific period of time. This is useful to implement waiting
in your code, e.g. to prevent overloading of target website or to avoid bot detection.
@@ -186,12 +183,12 @@ Note, however, that this may prevent parsing URLs from comma delimited lists, or
<a name="utils.isDocker"></a>
-## `utils.isDocker()` ⇒ <code>Promise</code>
+## `utils.isDocker()` ⇒ `Promise`
Returns a `Promise` that resolves to true if the code is running in a Docker container.
<a name="utils.downloadListOfUrls"></a>
-## `utils.downloadListOfUrls(options)` ⇒ <code>Promise&lt;Array&lt;String&gt;&gt;</code>
+## `utils.downloadListOfUrls(options)` ⇒ `Promise<Array<String>>`
Returns a promise that resolves to an array of urls parsed from the resource available at the provided url.
Optionally, custom regular expression and encoding may be provided.
@@ -226,7 +223,7 @@ Optionally, custom regular expression and encoding may be provided.
</table>
<a name="utils.extractUrls"></a>
-## `utils.extractUrls(string, [urlRegExp])` ⇒ <code>Array&lt;String&gt;</code>
+## `utils.extractUrls(string, [urlRegExp])` ⇒ `Array<String>`
Collects all URLs in an arbitrary string to an array, optionally using a custom regular expression.
<table>
@@ -248,12 +245,12 @@ Collects all URLs in an arbitrary string to an array, optionally using a custom
</table>
<a name="utils.getRandomUserAgent"></a>
-## `utils.getRandomUserAgent()` ⇒ <code>String</code>
+## `utils.getRandomUserAgent()` ⇒ `String`
Returns a randomly selected User-Agent header out of a list of the most common headers.
<a name="utils.htmlToText"></a>
-## `utils.htmlToText(html)` ⇒ <code>String</code>
+## `utils.htmlToText(html)` ⇒ `String`
The function converts a HTML document to a plain text.
The plain text generated by the function is similar to a text captured
@@ -279,7 +276,7 @@ const html = '<html><body>Some text</body></html>';
const text = htmlToText(cheerio.load(html, { decodeEntities: true }));
```
-**Returns**: <code>String</code> - Plain text
+**Returns**: `String` - Plain text
<table>
<thead>
<tr>
diff --git a/docs/typedefs/ActorRun.md b/docs/typedefs/ActorRun.md
index 9bf68d855..3e79ac0fa 100644
--- a/docs/typedefs/ActorRun.md
+++ b/docs/typedefs/ActorRun.md
@@ -57,8 +57,7 @@ For more details, see
&quot;origin&quot;: &quot;API&quot;,
&quot;clientIp&quot;: &quot;1.2.3.4&quot;,
&quot;userAgent&quot;: &quot;ApifyClient/0.2.13 (Linux; Node/v8.11.3)&quot;
- }
-</code></pre></td></tr><tr>
+ }</code></pre></td></tr><tr>
<td><code>stats</code></td><td><code>Object</code></td>
</tr>
<tr>
@@ -67,8 +66,7 @@ For more details, see
&quot;inputBodyLen&quot;: 22,
&quot;restartCount&quot;: 0,
&quot;workersUsed&quot;: 1,
- }
-</code></pre><p> Beware that object fields might change in future releases.</p>
+ }</code></pre><p> Beware that object fields might change in future releases.</p>
</td></tr><tr>
<td><code>options</code></td><td><code>Object</code></td>
</tr>
@@ -79,8 +77,7 @@ For more details, see
&quot;waitSecs&quot;: 0,
&quot;memoryMbytes&quot;: 256,
&quot;diskMbytes&quot;: 512
- }
-</code></pre></td></tr><tr>
+ }</code></pre></td></tr><tr>
<td><code>buildId</code></td><td><code>String</code></td>
</tr>
<tr>
@@ -132,6 +129,5 @@ For more details, see
&quot;body&quot;: {
&quot;message&quot;: &quot;Hello world!&quot;
}
- }
-</code></pre></td></tr></tbody>
+ }</code></pre></td></tr></tbody>
</table>
diff --git a/docs/typedefs/LaunchPuppeteerOptions.md b/docs/typedefs/LaunchPuppeteerOptions.md
index 2f9352c41..f8078d9d0 100644
--- a/docs/typedefs/LaunchPuppeteerOptions.md
+++ b/docs/typedefs/LaunchPuppeteerOptions.md
@@ -57,7 +57,7 @@ function and in addition, all the options available below.
<a href="https://my.apify.com/proxy" target="_blank">Apify Proxy</a> for all connections.
For more information, see the <a href="https://apify.com/docs/proxy" target="_blank">documentation</a></p>
</td></tr><tr>
-<td><code>[apifyProxyGroups]</code></td><td><code>Array&lt;String&gt;</code></td><td></td>
+<td><code>[apifyProxyGroups]</code></td><td><code>Array<String></code></td><td></td>
</tr>
<tr>
<td colspan="3"><p>An array of proxy groups to be used
diff --git a/src/cheerio_crawler.js b/src/cheerio_crawler.js
index e5d7555af..9f3a38071 100644
--- a/src/cheerio_crawler.js
+++ b/src/cheerio_crawler.js
@@ -146,11 +146,17 @@ const DEFAULT_OPTIONS = {
* the operation of `CheerioCrawler` and all its options. If you need more granular control over
* your requests, use {@link BasicCrawler}.
*
- * The internal defaults include:
- * - `url`, `method`, `headers`: provided by `requestList` and/or `requestQueue`
- * - `strictSSL`: use `options.ignoreSslErrors`
- * - `proxy`: use `options.useApifyProxy` or `options.proxyUrls`
- *
+ * The mandatory internal defaults:
+ * ```
+ * {
+ * url,
+ * method,
+ * headers,
+ * payload, // Are provided by RequestList and/or RequestQueue
+ * strictSSL, // Use options.ignoreSslErrors
+ * proxy, // Use options.useApifyProxy or options.proxyUrls
+ * }
+ * ```
* @param {Function} [options.prepareRequestFunction]
* A function that executes before the HTTP request is made to the target resource.
* This function is suitable for setting dynamic properties such as cookies to the {@link Request}.
diff --git a/website/package.json b/website/package.json
index 919da3609..f319a4016 100644
--- a/website/package.json
+++ b/website/package.json
@@ -10,7 +10,7 @@
"generate": "node ./tools/build_docs.js"
},
"devDependencies": {
- "docusaurus": "^1.5.1",
- "jsdoc-to-markdown": "^4.0.1"
+ "docusaurus": "^1.10.0",
+ "jsdoc-to-markdown": "^5.0.0"
}
}
diff --git a/website/tools/build_docs.js b/website/tools/build_docs.js
index ff791844b..62169487e 100644
--- a/website/tools/build_docs.js
+++ b/website/tools/build_docs.js
@@ -28,6 +28,7 @@ const getRenderOptions = (template, data) => ({
partial: [
path.join(__dirname, 'partials', 'params-table.hbs'),
path.join(__dirname, 'partials', 'properties-table.hbs'),
+ path.join(__dirname, 'partials', 'link.hbs'),
],
});
@@ -55,10 +56,10 @@ const generateFinalMarkdown = (title, text) => {
text = text.replace(rx, '');
// Remove 'Kind' annotations.
text = text.replace(/\*\*Kind\*\*.*\n/g, '');
- // Remove dots in type annotations
- const dotsRx = /([A-Z][a-z]+)\.(&lt;.+&gt;)/g;
+ // Remove dots in type annotations and replace entities
+ const dotsRx = /([A-Z][a-z]+)\.&lt;(.+)&gt;/g;
const replacer = (match, p1, p2) => {
- return p1 + p2.replace(dotsRx, replacer);
+ return `${p1}<${p2.replace(dotsRx, replacer)}>`;
};
text = text.replace(dotsRx, replacer);
// Fix class links
diff --git a/website/tools/helpers.js b/website/tools/helpers.js
index d6be11d21..e7d3643e1 100644
--- a/website/tools/helpers.js
+++ b/website/tools/helpers.js
@@ -1,11 +1,12 @@
-const { inlineLinks } = require('dmd/helpers/helpers'); // eslint-disable-line
const ddata = require('dmd/helpers/ddata'); // eslint-disable-line
+/* eslint-disable no-underscore-dangle */
+
exports.inlineLinks = (text, options) => {
if (text) {
const links = ddata.parseLink(text);
links.forEach((link) => {
- const linked = ddata._link(link.url, options); // eslint-disable-line no-underscore-dangle
+ const linked = ddata._link(link.url, options);
if (link.caption === link.url) link.caption = linked.name;
if (linked.url) link.url = linked.url;
const url = link.url.includes('+') ? link.url : link.url.toLowerCase();
@@ -19,3 +20,13 @@ exports.escape = function escape(input) {
if (typeof input !== 'string') return null;
return input.replace(/([*|])/g, '$1');
};
+
+exports.class = (options) => {
+ options.hash.kind = 'class';
+ const result = ddata._identifier(options);
+ if (result && result.kind === 'class') {
+ result.scope = 'global';
+ delete result.memberof;
+ }
+ return result ? options.fn(result) : 'ERROR, Cannot find class.';
+};
diff --git a/website/tools/partials/link.hbs b/website/tools/partials/link.hbs
new file mode 100644
index 000000000..612ad1f7f
--- /dev/null
+++ b/website/tools/partials/link.hbs
@@ -0,0 +1,25 @@
+{{! usage: link to="namepath" html=true/false caption="optional caption"~}}
+
+{{~#if html~}}
+ <code>
+
+ {{~#link to~}}
+ {{#if url~}}
+ <a href="{{{url}}}">{{#if ../../caption}}{{../../../caption}}{{else}}{{name}}{{/if}}</a>
+ {{~else~}}
+ {{#if ../../caption}}{{../../../caption}}{{else}}{{name}}{{/if~}}
+ {{/if~}}
+ {{/link~}}
+
+ </code>
+ {{~else~}}
+
+ {{#link to~}}
+ {{#if url~}}
+ [`{{#if ../../caption}}{{escape ../../../caption}}{{else}}{{escape name}}{{/if}}`]({{{url}}})
+ {{~else~}}
+ `{{#if ../../caption}}{{escape ../../../caption}}{{else}}{{escape name}}{{/if~}}`
+ {{~/if~}}
+ {{/link~}}
+
+{{/if~}}