Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Merge branch 'master' into docs/session-managment-page
Petr Patek65991d1b0093
diff --cc docs/api/Session.mdindex da6117550,31609e764..d7b40fcc1--- a/docs/api/Session.md+++ b/docs/api/Session.md@@@ -19,13 -19,11 +19,11 @@@ user data for example some authorizatio- [`.getState()`](#Session+getState) ⇒ `Object`- [`.retire()`](#Session+retire)- [`.markBad()`](#Session+markBad)- - [`.checkStatus(statusCode)`](#Session+checkStatus) ⇒ `boolean`+ - [`.retireOnBlockedStatusCodes(statusCode, blockedStatusCodes)`](#Session+retireOnBlockedStatusCodes) ⇒ `boolean`- - [`.putResponse(response)`](#Session+putResponse)- - [`.putPuppeteerCookies(puppeteerCookies, url)`](#Session+putPuppeteerCookies)- - [`.setCookies(cookies, url)`](#Session+setCookies)- - [`.getCookies(url)`](#Session+getCookies) ⇒ `Array<Object>`+ - [`.setCookiesFromResponse(response)`](#Session+setCookiesFromResponse)+ - [`.setPuppeteerCookies(cookies, url)`](#Session+setPuppeteerCookies)+ - [`.getPuppeteerCookies(url)`](#Session+getPuppeteerCookies) ⇒ `Array<Object>`- [`.getCookieString(url)`](#Session+getCookieString) ⇒ `String`- - [`.getPuppeteerCookies(url)`](#Session+getPuppeteerCookies) ⇒ `*`<a name="new_Session_new"></a>@@@ -141,7 -139,7 +139,7 @@@ This method should be called after a suGets session state for persistence in KeyValueStore.--**Returns**: `Object` - represents session internal state.++**Returns**: `Object` - represents session internal state.<a name="Session+retire"></a>## `session.retire()`@@@ -176,16 -174,11 +174,16 @@@ Retires session based on status code</tr><tr><td colspan="3"><p>HTTP status code</p>+</td></tr><tr>+<td><code>blockedStatusCodes</code></td><td><code>Array<Number></code></td>+</tr>+<tr>+<td colspan="3"><p>Custom HTTP status codes that means blocking on particular website.</p></td></tr></tbody></table>- <a name="Session+putResponse"></a>+ <a name="Session+setCookiesFromResponse"></a>- ## `session.putResponse(response)`+ ## `session.setCookiesFromResponse(response)`Sets cookies from response to the cookieJar. Parses cookies from `set-cookie` header and sets them to `Session.cookieJar`.@@@ -290,22 -260,3 +265,5 @@@ Wrapper around `tough-cookie` Cookie ja<tr></tr></tbody></table>+<a name="Session+getPuppeteerCookies"></a>+- ## `session.getPuppeteerCookies(url)` ⇒ `*`-- Gets cookies in format ready for puppeteer.-- <table>- <thead>- <tr>- <th>Param</th><th>Type</th>- </tr>- </thead>- <tbody>- <tr>- <td><code>url</code></td><td><code>String</code></td>- </tr>- <tr>- </tr></tbody>- </table>