Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Update changelog, bump version, build docs
Ondra Urbana59afd4252c9
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex c920505d6..065153b91 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,6 +1,7 @@xxx====================- `Snapshotter` will now log critical memory overload warnings at most once per 10 seconds.+_ Live view snapshots are now made right after navigation finishes, instead of right before page close.0.14.11 / 2019-05-28====================diff --git a/docs/api/LiveViewServer.md b/docs/api/LiveViewServer.mdindex 9413e8a84..3a9357003 100644--- a/docs/api/LiveViewServer.md+++ b/docs/api/LiveViewServer.md@@ -41,7 +41,6 @@ a better view into the browser, including DevTools, but `LiveViewServer` works t* [`.start()`](#LiveViewServer+start) ⇒ `Promise`* [`.stop()`](#LiveViewServer+stop) ⇒ `Promise`* [`.serve(page)`](#LiveViewServer+serve) ⇒ `Promise`- * [`.getLastSnapshot()`](#LiveViewServer+getLastSnapshot) ⇒ `Object`* [`.isRunning()`](#LiveViewServer+isRunning) ⇒ `boolean`* [`.hasClients()`](#LiveViewServer+hasClients) ⇒ `boolean`@@ -124,9 +123,6 @@ Will time out and throw in `options.snapshotTimeoutSecs`.<tr></tr></tbody></table>-<a name="LiveViewServer+getLastSnapshot"></a>--## `liveViewServer.getLastSnapshot()` ⇒ `Object`<a name="LiveViewServer+isRunning"></a>## `liveViewServer.isRunning()` ⇒ `boolean`diff --git a/docs/api/PuppeteerPool.md b/docs/api/PuppeteerPool.mdindex 1d8ad149f..ee6bb8d34 100644--- a/docs/api/PuppeteerPool.md+++ b/docs/api/PuppeteerPool.md@@ -46,6 +46,7 @@ await puppeteerPool.destroy();* [`.destroy()`](#PuppeteerPool+destroy) ⇒ `Promise`* [`.retire(browser)`](#PuppeteerPool+retire) ⇒ `Promise`* [`.recyclePage(page)`](#PuppeteerPool+recyclePage) ⇒ `Promise`+ * [`.serveLiveViewSnapshot(page)`](#PuppeteerPool+serveLiveViewSnapshot) ⇒ `Promise`<a name="new_PuppeteerPool_new"></a>@@ -199,3 +200,21 @@ or flagging it for reuse.<tr></tr></tbody></table>+<a name="PuppeteerPool+serveLiveViewSnapshot"></a>++## `puppeteerPool.serveLiveViewSnapshot(page)` ⇒ `Promise`+Tells the connected LiveViewServer to serve a snapshot when available.++<table>+<thead>+<tr>+<th>Param</th>+</tr>+</thead>+<tbody>+<tr>+<td><code>page</code></td>+</tr>+<tr>+</tr></tbody>+</table>diff --git a/docs/api/puppeteer.md b/docs/api/puppeteer.mdindex 691cf25ed..01eb88741 100644--- a/docs/api/puppeteer.md+++ b/docs/api/puppeteer.md@@ -128,14 +128,13 @@ Removes request interception handler for given page.<a name="puppeteer.gotoExtended"></a>## `puppeteer.gotoExtended` ⇒ `Promise<Response>`-*NOTE:* In recent versions of Puppeteer using this function entirely disables browser cache which resolves in sub-optimal-performance. Until this resolves, we suggest using this function only when needed and not to replace all `page.goto()`-calls in your actors with it.-Extended version of Puppeteer's `page.goto()` allowing to perform requests with HTTP method other than GET,with custom headers and POST payload. URL, method, headers and payload are taken fromrequest parameter that must be an instance of Apify.Request class.+*NOTE:* In recent versions of Puppeteer using requests other than GET, overriding headers and adding payloads disables+browser cache which degrades performance.+<table><thead><tr>diff --git a/package.json b/package.jsonindex 82c46a66e..cc84d7b85 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "apify",- "version": "0.14.11",+ "version": "0.14.12","description": "The scalable web crawling and scraping library for JavaScript. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.","engines": {"node": ">=8.14.0"