Skip to content

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

feat(fetchers): add functions to save browser session state and load it

Karim shoaird5b1995ce6ea
diff --git a/README.md b/README.md
index 65a973b..bd492f0 100644
--- a/README.md
+++ b/README.md
@@ -107,15 +107,6 @@ MySpider().start()
<a href="https://go.nodemaven.com/scraplingseptember" target="_blank">NodeMaven</a> - The most efficient proxy provider for Web Scraping and Automation with the Highest Quality IP on the market. Use code SCRAPLING35 for 35% discount.
</td>
</tr>
- <tr>
- <td width="200">
- <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank" title="Clean Proxies with No Nonsense.">
- <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxidize.png">
- </a>
- </td>
- <td> <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank"><b>Proxidize</b></a> provides mobile and residential proxies for scraping, browser automation, SEO monitoring, AI agents, and data collection. <i>Use code <b>scrapling20</b> for 20% off</i>.
- </td>
- </tr>
<tr>
<td width="200">
<a href="https://hypersolutions.co/?utm_source=github&utm_medium=readme&utm_campaign=scrapling" target="_blank" title="Bot Protection Bypass API for Akamai, DataDome, Incapsula & Kasada">
@@ -212,6 +203,16 @@ MySpider().start()
<a href="https://proxylane.dev/?utm_source=scrapling&utm_medium=partnership&utm_campaign=scrapling_platinum_202609&utm_content=github_readme" target="_blank">ProxyLane</a>: Reduce blocks and CAPTCHAs by 90%. 28M Premium Residential IPs for Heavy Web Scraping. 195 countries. City/ISP/ZIP targeting. Sticky/rotating sessions. Non-expiring traffic from $2/GB. Code <strong>SCRAPEANDTAKE</strong>: <em>25% off</em>.
</td>
</tr>
+ <tr>
+ <td width="200">
+ <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank" title="Buy Pay-as-you-go proxies from $1/GB">
+ <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/DataImpulse_Platinum.png">
+ </a>
+ </td>
+ <td>
+ $1/GB at <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank">DataImpulse</a>. Ethical first-party proxies. 90M+ IPs, 195+ countries. Residential, datacenter, mobile. Pay-as-you-go = no subscription. Traffic never expires.
+ </td>
+ </tr>
</table>
<i><sub>Do you want to show your ad here? Click [here](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
diff --git a/agent-skill/Scrapling-Skill.zip b/agent-skill/Scrapling-Skill.zip
index 7450e23..8f9b8ee 100644
Binary files a/agent-skill/Scrapling-Skill.zip and b/agent-skill/Scrapling-Skill.zip differ
diff --git a/agent-skill/Scrapling-Skill/references/fetching/dynamic.md b/agent-skill/Scrapling-Skill/references/fetching/dynamic.md
index 7385aa0..53c1ef3 100644
--- a/agent-skill/Scrapling-Skill/references/fetching/dynamic.md
+++ b/agent-skill/Scrapling-Skill/references/fetching/dynamic.md
@@ -419,6 +419,41 @@ Keeping the tabs open also means the page you fetched is still there for the nex
Versions 0.3.2 to 0.4.14 closed every tab after its request because reusing tabs used to leak settings between requests. Since 0.4.15, the settings are reset on every reuse, so the tabs stay open.
+### Save and Load Session State
+
+Use `save_state(path)` to save cookies, local storage and IndexedDB to a JSON file. Use `load_state(path)` to restore that file in another session, or replace the current session's stored state. Both methods accept a string or `pathlib.Path` and return `None`. The format is compatible with Playwright's storage state files.
+
+```python
+from scrapling.fetchers import DynamicSession
+
+with DynamicSession() as session:
+ session.fetch('https://example.com')
+ session.save_state('state.json')
+
+with DynamicSession() as session:
+ session.load_state('state.json')
+ page = session.fetch('https://example.com')
+```
+
+Save after your login or other browser actions finish. Call both methods inside a started session, after any pending fetches finish. Loading replaces existing cookies, local storage and IndexedDB. Fetch or reload the page to use the restored login; loading does not navigate existing tabs.
+
+The same methods work with `StealthySession`. Use `await` with `AsyncDynamicSession` and `AsyncStealthySession`:
+
+```python
+from scrapling.fetchers import AsyncDynamicSession
+
+async def restore_session():
+ async with AsyncDynamicSession() as session:
+ await session.load_state('state.json')
+ page = await session.fetch('https://example.com')
+ await session.save_state('state.json')
+ return page
+```
+
+These methods save and restore only the shared browser context. They do not affect temporary per-request proxy contexts. Sessions with `proxy_rotator` have no shared context, so these methods raise `RuntimeError`. A fixed session `proxy` works as usual.
+
+The file does not contain `sessionStorage`, open tabs or browser settings. Saving overwrites the file; its parent directory must exist. File and JSON errors are raised to the caller.
+
### Session Benefits
- **Browser reuse**: Much faster subsequent requests by reusing the same browser instance.
diff --git a/agent-skill/Scrapling-Skill/references/fetching/stealthy.md b/agent-skill/Scrapling-Skill/references/fetching/stealthy.md
index f26004e..70e9701 100644
--- a/agent-skill/Scrapling-Skill/references/fetching/stealthy.md
+++ b/agent-skill/Scrapling-Skill/references/fetching/stealthy.md
@@ -281,6 +281,10 @@ Saved response bytes come from Playwright and can differ from the server's origi
Recording is off by default. The default limit is 1,000 requests, with fixed limits of 1 MiB per saved response body and 20 MiB combined; old records are removed when retention limits are reached. These limits do not cover total browser memory, concurrent captures, temporary full-body reads, parsed HTML, or request bodies. Check `response.meta.get('body_note')` for non-text, oversized, or unreadable bodies. Saved text, including empty text, has no note. HEAD responses and status codes 204, 205, and 304 remain empty with no note. Clearing the history does not reuse IDs. See [Network History](dynamic.md#network-history) for filters and saved response access. See [Migrating in v5](dynamic.md#migrating-in-v5) for the API capture changes.
+### Save and Load Session State
+
+`StealthySession` supports `session.save_state('state.json')` and `session.load_state('state.json')`. Use `await` for both methods with `AsyncStealthySession`. The JSON file contains cookies, local storage and IndexedDB; loading replaces the current stored state. Call these methods inside a started session, after pending fetches finish. `sessionStorage` is not saved, and sessions with `proxy_rotator` are not supported. See [Save and Load Session State](dynamic.md#save-and-load-session-state) for examples and scope.
+
### Session Benefits
- **Browser reuse**: Much faster subsequent requests by reusing the same browser instance.
diff --git a/docs/README_AR.md b/docs/README_AR.md
index 31d8aff..bd78404 100644
--- a/docs/README_AR.md
+++ b/docs/README_AR.md
@@ -95,15 +95,6 @@ MySpider().start()
<a href="https://go.nodemaven.com/scraplingseptember" target="_blank">NodeMaven</a> - أكفأ مزود بروكسيات للـ Web Scraping والأتمتة، مع أعلى جودة IP في السوق. استخدم الكود SCRAPLING35 للحصول على خصم 35%.
</td>
</tr>
- <tr>
- <td width="200">
- <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank" title="Clean Proxies with No Nonsense.">
- <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxidize.png">
- </a>
- </td>
- <td> توفر <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank"><b>Proxidize</b></a> وكلاء جوّالين وسكنيين للاستخراج، وأتمتة المتصفح، ومراقبة تحسين محركات البحث (SEO)، ووكلاء الذكاء الاصطناعي، وجمع البيانات. <i>استخدم الرمز <b>scrapling20</b> للحصول على خصم 20%</i>.
- </td>
- </tr>
<tr>
<td width="200">
<a href="https://hypersolutions.co/?utm_source=github&utm_medium=readme&utm_campaign=scrapling" target="_blank" title="Bot Protection Bypass API for Akamai, DataDome, Incapsula & Kasada">
@@ -200,6 +191,16 @@ MySpider().start()
<a href="https://proxylane.dev/?utm_source=scrapling&utm_medium=partnership&utm_campaign=scrapling_platinum_202609&utm_content=github_readme" target="_blank">ProxyLane</a>: قلّل الحظر واختبارات CAPTCHA بنسبة 90%. 28 مليون عنوان IP سكني عالي الجودة لاستخراج بيانات الويب بكثافة. 195 دولة. استهداف حسب المدينة/ISP/الرمز البريدي (ZIP). جلسات ثابتة أو دوّارة. حركة بيانات لا تنتهي صلاحيتها تبدأ من $2/GB. الكود <strong>SCRAPEANDTAKE</strong>: <em>خصم 25%</em>.
</td>
</tr>
+ <tr>
+ <td width="200">
+ <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank" title="Buy Pay-as-you-go proxies from $1/GB">
+ <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/DataImpulse_Platinum.png">
+ </a>
+ </td>
+ <td>
+ $1/GB لدى <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank">DataImpulse</a>. بروكسيات من الطرف الأول بمصادر أخلاقية. أكثر من 90 مليون عنوان IP في أكثر من 195 دولة. بروكسيات سكنية ومراكز بيانات ومحمولة. الدفع حسب الاستخدام = بدون اشتراك. حركة البيانات لا تنتهي صلاحيتها.
+ </td>
+ </tr>
</table>
<i><sub>هل تريد عرض إعلانك هنا؟ انقر [هنا](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
diff --git a/docs/README_CN.md b/docs/README_CN.md
index d1ebb4e..d2f7447 100644
--- a/docs/README_CN.md
+++ b/docs/README_CN.md
@@ -95,15 +95,6 @@ MySpider().start()
<a href="https://go.nodemaven.com/scraplingseptember" target="_blank">NodeMaven</a> - 最高效的网页抓取与自动化代理提供商,拥有市场上质量最高的 IP。使用优惠码 SCRAPLING35 可享 35% 折扣。
</td>
</tr>
- <tr>
- <td width="200">
- <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank" title="Clean Proxies with No Nonsense.">
- <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxidize.png">
- </a>
- </td>
- <td> <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank"><b>Proxidize</b></a> 提供移动代理和住宅代理,适用于网页抓取、浏览器自动化、SEO 监控、AI 代理和数据收集。<i>使用优惠码 <b>scrapling20</b> 可享 20% 折扣</i>。
- </td>
- </tr>
<tr>
<td width="200">
<a href="https://hypersolutions.co/?utm_source=github&utm_medium=readme&utm_campaign=scrapling" target="_blank" title="Bot Protection Bypass API for Akamai, DataDome, Incapsula & Kasada">
@@ -200,6 +191,16 @@ MySpider().start()
<a href="https://proxylane.dev/?utm_source=scrapling&utm_medium=partnership&utm_campaign=scrapling_platinum_202609&utm_content=github_readme" target="_blank">ProxyLane</a>:将封禁和 CAPTCHA 验证码减少 90%。提供 2800 万个优质住宅 IP,适用于大规模网页抓取。覆盖 195 个国家。支持城市、ISP、邮政编码(ZIP)定向。支持固定和轮换会话。流量永不过期,低至 $2/GB。使用优惠码 <strong>SCRAPEANDTAKE</strong> 可享 <em>25% 折扣</em>。
</td>
</tr>
+ <tr>
+ <td width="200">
+ <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank" title="Buy Pay-as-you-go proxies from $1/GB">
+ <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/DataImpulse_Platinum.png">
+ </a>
+ </td>
+ <td>
+ <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank">DataImpulse</a>,价格为 $1/GB。符合道德规范的第一方代理。9000 万+ IP,覆盖 195+ 个国家。住宅、数据中心、移动代理。按量付费 = 无需订阅。流量永不过期。
+ </td>
+ </tr>
</table>
<i><sub>想在这里展示您的广告吗?点击 [这里](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
diff --git a/docs/README_DE.md b/docs/README_DE.md
index 2ed5647..5a56203 100644
--- a/docs/README_DE.md
+++ b/docs/README_DE.md
@@ -95,15 +95,6 @@ MySpider().start()
<a href="https://go.nodemaven.com/scraplingseptember" target="_blank">NodeMaven</a> - der effizienteste Proxy-Anbieter für Web Scraping und Automatisierung mit der höchsten IP-Qualität auf dem Markt. Nutze den Code SCRAPLING35 für 35% Rabatt.
</td>
</tr>
- <tr>
- <td width="200">
- <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank" title="Clean Proxies with No Nonsense.">
- <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxidize.png">
- </a>
- </td>
- <td> <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank"><b>Proxidize</b></a> bietet mobile und Residential-Proxies für Scraping, Browser-Automatisierung, SEO-Monitoring, KI-Agenten und Datenerfassung. <i>Mit dem Code <b>scrapling20</b> erhalten Sie 20% Rabatt</i>.
- </td>
- </tr>
<tr>
<td width="200">
<a href="https://hypersolutions.co/?utm_source=github&utm_medium=readme&utm_campaign=scrapling" target="_blank" title="Bot Protection Bypass API for Akamai, DataDome, Incapsula & Kasada">
@@ -200,6 +191,16 @@ MySpider().start()
<a href="https://proxylane.dev/?utm_source=scrapling&utm_medium=partnership&utm_campaign=scrapling_platinum_202609&utm_content=github_readme" target="_blank">ProxyLane</a>: Reduzieren Sie Sperren und CAPTCHAs um 90%. 28 Mio. Premium-Residential-IPs für umfangreiches Web Scraping. 195 Länder. Gezielte Auswahl nach Stadt/ISP/Postleitzahl. Sitzungen mit fester oder rotierender IP. Datenvolumen ohne Ablaufdatum ab $2/GB. Code <strong>SCRAPEANDTAKE</strong>: <em>25% Rabatt</em>.
</td>
</tr>
+ <tr>
+ <td width="200">
+ <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank" title="Buy Pay-as-you-go proxies from $1/GB">
+ <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/DataImpulse_Platinum.png">
+ </a>
+ </td>
+ <td>
+ $1/GB bei <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank">DataImpulse</a>. Ethisch bezogene Proxys aus erster Hand. Über 90 Mio. IPs, mehr als 195 Länder. Residential-, Datacenter- und Mobilproxies. Nutzungsabhängige Zahlung = kein Abo. Datenvolumen verfällt nie.
+ </td>
+ </tr>
</table>
<i><sub>Möchten Sie Ihre Anzeige hier zeigen? Klicken Sie [hier](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
diff --git a/docs/README_ES.md b/docs/README_ES.md
index 904714d..3ce5b02 100644
--- a/docs/README_ES.md
+++ b/docs/README_ES.md
@@ -95,15 +95,6 @@ MySpider().start()
<a href="https://go.nodemaven.com/scraplingseptember" target="_blank">NodeMaven</a> - el proveedor de proxies más eficiente para Web Scraping y automatización, con la mayor calidad de IP del mercado. Usa el código SCRAPLING35 para obtener un 35% de descuento.
</td>
</tr>
- <tr>
- <td width="200">
- <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank" title="Clean Proxies with No Nonsense.">
- <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxidize.png">
- </a>
- </td>
- <td> <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank"><b>Proxidize</b></a> proporciona proxies móviles y residenciales para scraping, automatización de navegadores, monitoreo de SEO, agentes de IA y recopilación de datos. <i>Usa el código <b>scrapling20</b> para obtener un 20% de descuento</i>.
- </td>
- </tr>
<tr>
<td width="200">
<a href="https://hypersolutions.co/?utm_source=github&utm_medium=readme&utm_campaign=scrapling" target="_blank" title="Bot Protection Bypass API for Akamai, DataDome, Incapsula & Kasada">
@@ -200,6 +191,16 @@ MySpider().start()
<a href="https://proxylane.dev/?utm_source=scrapling&utm_medium=partnership&utm_campaign=scrapling_platinum_202609&utm_content=github_readme" target="_blank">ProxyLane</a>: Reduce los bloqueos y los CAPTCHAs en un 90%. 28 millones de IPs residenciales premium para scraping web intensivo. 195 países. Segmentación por ciudad/ISP/código postal. Sesiones persistentes o rotativas. Tráfico sin caducidad desde $2/GB. Código <strong>SCRAPEANDTAKE</strong>: <em>25% de descuento</em>.
</td>
</tr>
+ <tr>
+ <td width="200">
+ <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank" title="Buy Pay-as-you-go proxies from $1/GB">
+ <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/DataImpulse_Platinum.png">
+ </a>
+ </td>
+ <td>
+ $1/GB en <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank">DataImpulse</a>. Proxies de primera mano de origen ético. Más de 90 millones de IPs, más de 195 países. Residenciales, de centros de datos y móviles. Pago por uso = sin suscripción. El tráfico nunca caduca.
+ </td>
+ </tr>
</table>
<i><sub>¿Quieres mostrar tu anuncio aquí? Haz clic [aquí](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
diff --git a/docs/README_FR.md b/docs/README_FR.md
index 269e841..95fe8d1 100644
--- a/docs/README_FR.md
+++ b/docs/README_FR.md
@@ -95,15 +95,6 @@ MySpider().start()
<a href="https://go.nodemaven.com/scraplingseptember" target="_blank">NodeMaven</a> - le fournisseur de proxys le plus efficace pour le Web Scraping et l'automatisation, avec la meilleure qualité d'IP du marché. Utilisez le code SCRAPLING35 pour obtenir 35% de réduction.
</td>
</tr>
- <tr>
- <td width="200">
- <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank" title="Clean Proxies with No Nonsense.">
- <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxidize.png">
- </a>
- </td>
- <td> <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank"><b>Proxidize</b></a> fournit des proxies mobiles et résidentiels pour le scraping, l'automatisation de navigateur, le suivi SEO, les agents IA et la collecte de données. <i>Utilisez le code <b>scrapling20</b> pour bénéficier de 20% de réduction</i>.
- </td>
- </tr>
<tr>
<td width="200">
<a href="https://hypersolutions.co/?utm_source=github&utm_medium=readme&utm_campaign=scrapling" target="_blank" title="Bot Protection Bypass API for Akamai, DataDome, Incapsula & Kasada">
@@ -200,6 +191,16 @@ MySpider().start()
<a href="https://proxylane.dev/?utm_source=scrapling&utm_medium=partnership&utm_campaign=scrapling_platinum_202609&utm_content=github_readme" target="_blank">ProxyLane</a> : Réduisez les blocages et les CAPTCHAs de 90 %. 28 millions d’adresses IP résidentielles premium pour le scraping web intensif. 195 pays. Ciblage par ville/ISP/code postal. Sessions fixes ou rotatives. Trafic sans expiration à partir de $2/GB. Code <strong>SCRAPEANDTAKE</strong> : <em>25 % de réduction</em>.
</td>
</tr>
+ <tr>
+ <td width="200">
+ <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank" title="Buy Pay-as-you-go proxies from $1/GB">
+ <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/DataImpulse_Platinum.png">
+ </a>
+ </td>
+ <td>
+ $1/GB chez <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank">DataImpulse</a>. Proxies de première main issus de sources éthiques. Plus de 90 millions d’IP, plus de 195 pays. Résidentiels, de datacenter et mobiles. Paiement à l’usage = sans abonnement. Le trafic n’expire jamais.
+ </td>
+ </tr>
</table>
<i><sub>Vous souhaitez afficher votre publicité ici ? Cliquez [ici](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
diff --git a/docs/README_JP.md b/docs/README_JP.md
index cf6b21d..a5df603 100644
--- a/docs/README_JP.md
+++ b/docs/README_JP.md
@@ -95,15 +95,6 @@ MySpider().start()
<a href="https://go.nodemaven.com/scraplingseptember" target="_blank">NodeMaven</a> - Webスクレイピングと自動化のための最も効率的なプロキシプロバイダーで、市場最高品質のIPを提供します。コード SCRAPLING35 で35%割引になります。
</td>
</tr>
- <tr>
- <td width="200">
- <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank" title="Clean Proxies with No Nonsense.">
- <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxidize.png">
- </a>
- </td>
- <td> <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank"><b>Proxidize</b></a> は、スクレイピング、ブラウザ自動化、SEO監視、AIエージェント、データ収集のためのモバイルおよびレジデンシャルプロキシを提供します。<i>コード <b>scrapling20</b> で20%オフ</i>。
- </td>
- </tr>
<tr>
<td width="200">
<a href="https://hypersolutions.co/?utm_source=github&utm_medium=readme&utm_campaign=scrapling" target="_blank" title="Bot Protection Bypass API for Akamai, DataDome, Incapsula & Kasada">
@@ -200,6 +191,16 @@ MySpider().start()
<a href="https://proxylane.dev/?utm_source=scrapling&utm_medium=partnership&utm_campaign=scrapling_platinum_202609&utm_content=github_readme" target="_blank">ProxyLane</a>:ブロックとCAPTCHAを90%削減。大規模なWebスクレイピング向けに、2,800万の高品質なレジデンシャルIPを提供。195か国に対応。都市・ISP・郵便番号(ZIP)の指定が可能。固定・ローテーションセッション。有効期限のない通信量を $2/GB から提供。コード <strong>SCRAPEANDTAKE</strong> で<em>25%オフ</em>。
</td>
</tr>
+ <tr>
+ <td width="200">
+ <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank" title="Buy Pay-as-you-go proxies from $1/GB">
+ <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/DataImpulse_Platinum.png">
+ </a>
+ </td>
+ <td>
+ <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank">DataImpulse</a> なら $1/GB。倫理的に調達された自社提供のプロキシ。9,000万以上のIP、195か国以上に対応。レジデンシャル、データセンター、モバイル。従量課金 = サブスクリプション不要。通信量に有効期限はありません。
+ </td>
+ </tr>
</table>
<i><sub>ここに広告を表示したいですか?[こちら](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)をクリック</sub></i>
diff --git a/docs/README_KR.md b/docs/README_KR.md
index 6e54f9c..43d3a92 100644
--- a/docs/README_KR.md
+++ b/docs/README_KR.md
@@ -95,15 +95,6 @@ MySpider().start()
<a href="https://go.nodemaven.com/scraplingseptember" target="_blank">NodeMaven</a> - 웹 스크래핑과 자동화를 위한 가장 효율적인 프록시 제공업체로, 시장에서 가장 높은 품질의 IP를 제공합니다. 코드 SCRAPLING35를 사용하면 35% 할인을 받을 수 있습니다.
</td>
</tr>
- <tr>
- <td width="200">
- <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank" title="Clean Proxies with No Nonsense.">
- <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxidize.png">
- </a>
- </td>
- <td> <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank"><b>Proxidize</b></a>는 스크래핑, 브라우저 자동화, SEO 모니터링, AI 에이전트, 데이터 수집을 위한 모바일 및 주거용 프록시를 제공합니다. <i>코드 <b>scrapling20</b>으로 20% 할인</i>.
- </td>
- </tr>
<tr>
<td width="200">
<a href="https://hypersolutions.co/?utm_source=github&utm_medium=readme&utm_campaign=scrapling" target="_blank" title="Bot Protection Bypass API for Akamai, DataDome, Incapsula & Kasada">
@@ -200,6 +191,16 @@ MySpider().start()
<a href="https://proxylane.dev/?utm_source=scrapling&utm_medium=partnership&utm_campaign=scrapling_platinum_202609&utm_content=github_readme" target="_blank">ProxyLane</a>: 차단과 CAPTCHA를 90% 줄입니다. 대규모 웹 스크래핑을 위한 2,800만 개의 프리미엄 주거용 IP. 195개 국가 지원. 도시, ISP, 우편번호(ZIP) 타겟팅. 고정 및 순환 세션. 유효 기간이 없는 트래픽을 $2/GB부터 제공합니다. 코드 <strong>SCRAPEANDTAKE</strong>를 사용하면 <em>25% 할인</em>.
</td>
</tr>
+ <tr>
+ <td width="200">
+ <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank" title="Buy Pay-as-you-go proxies from $1/GB">
+ <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/DataImpulse_Platinum.png">
+ </a>
+ </td>
+ <td>
+ <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank">DataImpulse</a>에서 $1/GB로 이용하세요. 윤리적으로 확보한 자체 프록시. 9,000만 개 이상의 IP, 195개 이상의 국가. 주거용, 데이터센터, 모바일 프록시. 사용량 기반 요금 = 구독 불필요. 트래픽에 유효 기간이 없습니다.
+ </td>
+ </tr>
</table>
<i><sub>여기에 광고를 게재하고 싶으신가요? [여기](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)를 클릭하세요</sub></i>
diff --git a/docs/README_PT_BR.md b/docs/README_PT_BR.md
index 53980c2..fcc27dd 100644
--- a/docs/README_PT_BR.md
+++ b/docs/README_PT_BR.md
@@ -97,15 +97,6 @@ MySpider().start()
<a href="https://go.nodemaven.com/scraplingseptember" target="_blank">NodeMaven</a> - o provedor de proxies mais eficiente para Web Scraping e automação, com a mais alta qualidade de IP do mercado. Use o código SCRAPLING35 para obter 35% de desconto.
</td>
</tr>
- <tr>
- <td width="200">
- <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank" title="Clean Proxies with No Nonsense.">
- <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxidize.png">
- </a>
- </td>
- <td> A <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank"><b>Proxidize</b></a> oferece proxies móveis e residenciais para scraping, automação de navegador, monitoramento de SEO, agentes de IA e coleta de dados. <i>Use o código <b>scrapling20</b> para 20% de desconto</i>.
- </td>
- </tr>
<tr>
<td width="200">
<a href="https://hypersolutions.co/?utm_source=github&utm_medium=readme&utm_campaign=scrapling" target="_blank" title="Bot Protection Bypass API for Akamai, DataDome, Incapsula & Kasada">
@@ -202,6 +193,16 @@ MySpider().start()
<a href="https://proxylane.dev/?utm_source=scrapling&utm_medium=partnership&utm_campaign=scrapling_platinum_202609&utm_content=github_readme" target="_blank">ProxyLane</a>: Reduza os bloqueios e os CAPTCHAs em 90%. 28 milhões de IPs residenciais premium para scraping intensivo da web. 195 países. Segmentação por cidade/ISP/CEP. Sessões fixas ou rotativas. Tráfego sem prazo de validade a partir de $2/GB. Código <strong>SCRAPEANDTAKE</strong>: <em>25% de desconto</em>.
</td>
</tr>
+ <tr>
+ <td width="200">
+ <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank" title="Buy Pay-as-you-go proxies from $1/GB">
+ <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/DataImpulse_Platinum.png">
+ </a>
+ </td>
+ <td>
+ $1/GB na <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank">DataImpulse</a>. Proxies de origem ética, direto da fonte. Mais de 90 milhões de IPs, mais de 195 países. Residenciais, de datacenter e móveis. Pagamento por uso = sem assinatura. O tráfego nunca expira.
+ </td>
+ </tr>
</table>
<i><sub>Quer mostrar seu anúncio aqui? Clique [aqui](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
diff --git a/docs/README_RU.md b/docs/README_RU.md
index b61cde5..c792b59 100644
--- a/docs/README_RU.md
+++ b/docs/README_RU.md
@@ -95,15 +95,6 @@ MySpider().start()
<a href="https://go.nodemaven.com/scraplingseptember" target="_blank">NodeMaven</a> - самый эффективный провайдер прокси для веб-скрапинга и автоматизации с самым высоким качеством IP на рынке. Используйте код SCRAPLING35 для получения скидки 35%.
</td>
</tr>
- <tr>
- <td width="200">
- <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank" title="Clean Proxies with No Nonsense.">
- <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxidize.png">
- </a>
- </td>
- <td> <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank"><b>Proxidize</b></a> предоставляет мобильные и резидентные прокси для скрейпинга, автоматизации браузера, SEO-мониторинга, ИИ-агентов и сбора данных. <i>Используйте код <b>scrapling20</b> для скидки 20%</i>.
- </td>
- </tr>
<tr>
<td width="200">
<a href="https://hypersolutions.co/?utm_source=github&utm_medium=readme&utm_campaign=scrapling" target="_blank" title="Bot Protection Bypass API for Akamai, DataDome, Incapsula & Kasada">
@@ -203,6 +194,16 @@ MySpider().start()
<a href="https://proxylane.dev/?utm_source=scrapling&utm_medium=partnership&utm_campaign=scrapling_platinum_202609&utm_content=github_readme" target="_blank">ProxyLane</a>: Сократите число блокировок и CAPTCHA на 90%. 28 млн премиальных резидентных IP-адресов для интенсивного веб-скрейпинга. 195 стран. Таргетинг по городу/ISP/почтовому индексу. Сессии с постоянным IP или ротацией. Трафик без срока действия от $2/GB. Код <strong>SCRAPEANDTAKE</strong>: <em>скидка 25%</em>.
</td>
</tr>
+ <tr>
+ <td width="200">
+ <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank" title="Buy Pay-as-you-go proxies from $1/GB">
+ <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/DataImpulse_Platinum.png">
+ </a>
+ </td>
+ <td>
+ $1/GB в <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank">DataImpulse</a>. Прокси из этичных источников, без посредников. Более 90 млн IP-адресов, более 195 стран. Резидентные, дата-центровые и мобильные. Оплата по мере использования = без подписки. Трафик без срока действия.
+ </td>
+ </tr>
</table>
<i><sub>Хотите показать здесь свою рекламу? Нажмите [здесь](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
diff --git a/docs/fetching/dynamic.md b/docs/fetching/dynamic.md
index 112d29a..0a0706d 100644
--- a/docs/fetching/dynamic.md
+++ b/docs/fetching/dynamic.md
@@ -167,6 +167,41 @@ Keeping the tabs open also means the page you fetched is still there for the nex
Versions 0.3.2 to 0.4.14 closed every tab after its request because reusing tabs used to leak settings between requests. Since 0.4.15, the settings are reset on every reuse, so the tabs stay open.
+### Save and Load Session State
+
+Use `save_state(path)` to save cookies, local storage and IndexedDB to a JSON file. Use `load_state(path)` to restore that file in another session, or replace the current session's stored state. Both methods accept a string or `pathlib.Path` and return `None`. The format is compatible with Playwright's storage state files.
+
+```python
+from scrapling.fetchers import DynamicSession
+
+with DynamicSession() as session:
+ session.fetch('https://example.com')
+ session.save_state('state.json')
+
+with DynamicSession() as session:
+ session.load_state('state.json')
+ page = session.fetch('https://example.com')
+```
+
+Save after your login or other browser actions finish. Call both methods inside a started session, after any pending fetches finish. Loading replaces existing cookies, local storage and IndexedDB. Fetch or reload the page to use the restored login; loading does not navigate existing tabs.
+
+The same methods work with `StealthySession`. Use `await` with `AsyncDynamicSession` and `AsyncStealthySession`:
+
+```python
+from scrapling.fetchers import AsyncDynamicSession
+
+async def restore_session():
+ async with AsyncDynamicSession() as session:
+ await session.load_state('state.json')
+ page = await session.fetch('https://example.com')
+ await session.save_state('state.json')
+ return page
+```
+
+These methods save and restore only the shared browser context. They do not affect temporary per-request proxy contexts. Sessions with `proxy_rotator` have no shared context, so these methods raise `RuntimeError`. A fixed session `proxy` works as usual.
+
+The file does not contain `sessionStorage`, open tabs or browser settings. Saving overwrites the file; its parent directory must exist. File and JSON errors are raised to the caller.
+
### Session Benefits
- **Browser reuse**: Much faster subsequent requests by reusing the same browser instance.
diff --git a/docs/fetching/stealthy.md b/docs/fetching/stealthy.md
index 2d57fbf..28ab956 100644
--- a/docs/fetching/stealthy.md
+++ b/docs/fetching/stealthy.md
@@ -292,6 +292,10 @@ Saved response bytes come from Playwright and can differ from the server's origi
Recording is off by default. The default limit is 1,000 requests, with fixed limits of 1 MiB per saved response body and 20 MiB combined; old records are removed when retention limits are reached. These limits do not cover total browser memory, concurrent captures, temporary full-body reads, parsed HTML, or request bodies. Check `response.meta.get('body_note')` for non-text, oversized, or unreadable bodies. Saved text, including empty text, has no note. HEAD responses and status codes 204, 205, and 304 remain empty with no note. Clearing the history does not reuse IDs. See [Network History](dynamic.md#network-history) for filters and saved response access. See [Migrating in v5](dynamic.md#migrating-in-v5) for the API capture changes.
+### Save and Load Session State
+
+`StealthySession` supports `session.save_state('state.json')` and `session.load_state('state.json')`. Use `await` for both methods with `AsyncStealthySession`. The JSON file contains cookies, local storage and IndexedDB; loading replaces the current stored state. Call these methods inside a started session, after pending fetches finish. `sessionStorage` is not saved, and sessions with `proxy_rotator` are not supported. See [Save and Load Session State](dynamic.md#save-and-load-session-state) for examples and scope.
+
### Session Benefits
- **Browser reuse**: Much faster subsequent requests by reusing the same browser instance.
diff --git a/docs/index.md b/docs/index.md
index dc13c98..d02e5f9 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -59,9 +59,6 @@ MySpider().start()
<a href="https://go.nodemaven.com/scraplingseptember" target="_blank" title="Proxies with the Highest IP Scores">
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/NodeMaven.jpg" class="ad">
</a>
- <a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank" title="Clean Proxies with No Nonsense.">
- <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxidize.png" class="ad">
- </a>
<a href="https://hypersolutions.co/?utm_source=github&utm_medium=readme&utm_campaign=scrapling" target="_blank" title="Bot Protection Bypass API for Akamai, DataDome, Incapsula & Kasada">
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/HyperSolutions.png" class="ad">
</a>
@@ -89,6 +86,9 @@ MySpider().start()
<a href="https://proxylane.dev/?utm_source=scrapling&utm_medium=partnership&utm_campaign=scrapling_platinum_202609&utm_content=github_readme" target="_blank" title="28M+ clean residential IPs for browser automation. No Blocks. No CAPTCHAs. Pay-as-you-Go.">
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxylane.png" class="ad">
</a>
+ <a href="https://dataimpulse.com/?utm_source=website&utm_medium=github_readme&utm_campaign=scrapling_platinum" target="_blank" title="Buy Pay-as-you-go proxies from $1/GB">
+ <img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/DataImpulse_Platinum.png" class="ad">
+ </a>
<br />
<br />
<a href="https://cloro.dev/?utm_source=referral&utm_medium=scrapling" target="_blank" title="The search API for the AI era"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/cloro.jpg"></a>
diff --git a/images/DataImpulse_Platinum.png b/images/DataImpulse_Platinum.png
new file mode 100644
index 0000000..7a235a3
Binary files /dev/null and b/images/DataImpulse_Platinum.png differ
diff --git a/images/proxidize.png b/images/proxidize.png
deleted file mode 100644
index d2e9b9b..0000000
Binary files a/images/proxidize.png and /dev/null differ
diff --git a/scrapling/engines/_browsers/_base.py b/scrapling/engines/_browsers/_base.py
index 42ab729..686472b 100644
--- a/scrapling/engines/_browsers/_base.py
+++ b/scrapling/engines/_browsers/_base.py
@@ -1,4 +1,5 @@
from time import time
+from pathlib import Path
from asyncio import sleep as asyncio_sleep, Lock
from contextlib import contextmanager, asynccontextmanager, suppress
@@ -64,6 +65,24 @@ class SyncSession:
def start(self) -> None:
pass
+ def save_state(self, path: str | Path) -> None:
+ """Save cookies, local storage and IndexedDB to a Playwright-compatible JSON file.
+
+ Requires a started session without proxy rotation. Does not save sessionStorage.
+ """
+ if not self._is_alive or self.context is None:
+ raise RuntimeError("Session state requires a started browser session without proxy rotation")
+ self.context.storage_state(path=Path(path), indexed_db=True)
+
+ def load_state(self, path: str | Path) -> None:
+ """Replace cookies, local storage and IndexedDB from a Playwright-compatible JSON file.
+
+ Requires a started session without proxy rotation. Fetch or reload pages to use the restored login.
+ """
+ if not self._is_alive or self.context is None:
+ raise RuntimeError("Session state requires a started browser session without proxy rotation")
+ self.context.set_storage_state(path)
+
def close_pages(self) -> None:
"""Close every open tab in the session's pool. The next request opens a fresh tab."""
for page_info in self.page_pool.clear():
@@ -252,6 +271,24 @@ class AsyncSession:
async def start(self) -> None:
pass
+ async def save_state(self, path: str | Path) -> None:
+ """Save cookies, local storage and IndexedDB to a Playwright-compatible JSON file.
+
+ Requires a started session without proxy rotation. Does not save sessionStorage.
+ """
+ if not self._is_alive or self.context is None:
+ raise RuntimeError("Session state requires a started browser session without proxy rotation")
+ await self.context.storage_state(path=Path(path), indexed_db=True)
+
+ async def load_state(self, path: str | Path) -> None:
+ """Replace cookies, local storage and IndexedDB from a Playwright-compatible JSON file.
+
+ Requires a started session without proxy rotation. Fetch or reload pages to use the restored login.
+ """
+ if not self._is_alive or self.context is None:
+ raise RuntimeError("Session state requires a started browser session without proxy rotation")
+ await self.context.set_storage_state(path)
+
async def close_pages(self) -> None:
"""Close every open tab in the session's pool. The next request opens a fresh tab."""
for page_info in self.page_pool.clear():
diff --git a/tests/fetchers/test_session_state.py b/tests/fetchers/test_session_state.py
new file mode 100644
index 0000000..3bb5fce
--- /dev/null
+++ b/tests/fetchers/test_session_state.py
@@ -0,0 +1,217 @@
+from json import JSONDecodeError, loads
+from os import getenv
+from pathlib import Path
+
+import pytest
+
+from scrapling.core._types import Any
+from scrapling.engines.toolbelt.proxy_rotation import ProxyRotator
+from scrapling.fetchers import AsyncDynamicSession, AsyncStealthySession, DynamicSession, StealthySession
+
+
+URL = "http://state.test/"
+HTML = '<!DOCTYPE html><html><head><link rel="icon" href="data:,"></head><body>ready</body></html>'
+STATE_ERROR = "Session state requires a started browser session without proxy rotation"
+WRITE_STATE = """async value => {
+ document.cookie = `login=${value}; Path=/; SameSite=Lax`;
+ localStorage.setItem('login', value);
+ sessionStorage.setItem('temporary', value);
+ await new Promise((resolve, reject) => {
+ const request = indexedDB.open('auth', 1);
+ request.onupgradeneeded = () => request.result.createObjectStore('tokens', {keyPath: 'id'});
+ request.onerror = () => reject(request.error);
+ request.onsuccess = () => {
+ const database = request.result;
+ const transaction = database.transaction('tokens', 'readwrite');
+ transaction.objectStore('tokens').put({id: 1, value});
+ transaction.oncomplete = () => { database.close(); resolve(); };
+ transaction.onerror = () => reject(transaction.error);
+ };
+ });
+ if (value !== 'saved') {
+ document.cookie = 'stale=true; Path=/; SameSite=Lax';
+ localStorage.setItem('stale', 'true');
+ await new Promise((resolve, reject) => {
+ const request = indexedDB.open('stale', 1);
+ request.onerror = () => reject(request.error);
+ request.onsuccess = () => { request.result.close(); resolve(); };
+ });
+ }
+}"""
+READ_STATE = """async () => {
+ const value = await new Promise((resolve, reject) => {
+ const request = indexedDB.open('auth', 1);
+ request.onerror = () => reject(request.error);
+ request.onsuccess = () => {
+ const database = request.result;
+ const transaction = database.transaction('tokens');
+ const record = transaction.objectStore('tokens').get(1);
+ record.onsuccess = () => resolve(record.result);
+ record.onerror = () => reject(record.error);
+ transaction.oncomplete = () => database.close();
+ };
+ });
+ return {
+ cookies: document.cookie,
+ localStorage: {...localStorage},
+ indexedDB: value,
+ databases: (await indexedDB.databases()).map(database => database.name).sort()
+ };
+}"""
+SAVED_STATE = {
+ "cookies": "login=saved",
+ "localStorage": {"login": "saved"},
+ "indexedDB": {"id": 1, "value": "saved"},
+ "databases": ["auth"],
+}
+
+
+def _options(**kwargs: Any) -> dict[str, Any]:
+ return {"executable_path": getenv("SCRAPLING_EXECUTABLE_PATH"), "google_search": False, "retries": 1, **kwargs}
+
+
+def _check_json(path: Path, native_state: dict[str, Any]) -> None:
+ state = loads(path.read_text())
+ assert state == native_state
+ assert [(cookie["name"], cookie["value"]) for cookie in state["cookies"]] == [("login", "saved")]
+ assert len(state["origins"]) == 1
+ origin = state["origins"][0]
+ assert origin["origin"] == URL.rstrip("/")
+ assert origin["localStorage"] == [{"name": "login", "value": "saved"}]
+ assert origin["indexedDB"][0]["name"] == "auth"
+ assert "sessionStorage" not in origin
+
+
[email protected]("session_type", [DynamicSession, StealthySession])
+def test_sync_state_round_trip(session_type: Any, tmp_path: Path) -> None:
+ path, native_path = tmp_path / "state.json", tmp_path / "native.json"
+ with session_type(**_options()) as source:
+ source.context.route("http://state.test/**", lambda route: route.fulfill(body=HTML, content_type="text/html"))
+ source.fetch(URL)
+ source.page_pool.pages[0].page.evaluate(WRITE_STATE, "saved")
+ assert source.save_state(path) is None
+ _check_json(path, source.context.storage_state(path=native_path, indexed_db=True))
+ for method in (source.save_state, source.load_state):
+ with pytest.raises(RuntimeError, match=STATE_ERROR):
+ method(path)
+ with session_type(**_options()) as restored:
+ restored.context.route("http://state.test/**", lambda route: route.fulfill(body=HTML, content_type="text/html"))
+ assert restored.load_state(str(path)) is None
+ restored.fetch(URL)
+ page = restored.page_pool.pages[0].page
+ assert page.evaluate(READ_STATE) == SAVED_STATE
+ assert page.evaluate("sessionStorage.length") == 0
+ page.evaluate(WRITE_STATE, "changed")
+ changed = page.evaluate(READ_STATE)
+ assert changed["indexedDB"]["value"] == "changed" and changed["databases"] == ["auth", "stale"]
+ assert "stale" in changed["cookies"] and "stale" in changed["localStorage"]
+ restored.load_state(native_path)
+ assert page.url == URL and not page.is_closed()
+ assert page.evaluate("sessionStorage.getItem('temporary')") == "changed"
+ restored.fetch(URL)
+ assert page.evaluate(READ_STATE) == SAVED_STATE
+ assert restored.save_state(str(path)) is None
+ _check_json(path, loads(native_path.read_text()))
+
+
[email protected]("session_type", [AsyncDynamicSession, AsyncStealthySession])
+async def test_async_state_round_trip(session_type: Any, tmp_path: Path) -> None:
+ async def route_page(route: Any) -> None:
+ await route.fulfill(body=HTML, content_type="text/html")
+
+ path, native_path = tmp_path / "state.json", tmp_path / "native.json"
+ async with session_type(**_options()) as source:
+ await source.context.route("http://state.test/**", route_page)
+ await source.fetch(URL)
+ await source.page_pool.pages[0].page.evaluate(WRITE_STATE, "saved")
+ assert await source.save_state(path) is None
+ _check_json(path, await source.context.storage_state(path=native_path, indexed_db=True))
+ for method in (source.save_state, source.load_state):
+ with pytest.raises(RuntimeError, match=STATE_ERROR):
+ await method(path)
+ async with session_type(**_options()) as restored:
+ await restored.context.route("http://state.test/**", route_page)
+ assert await restored.load_state(str(path)) is None
+ await restored.fetch(URL)
+ page = restored.page_pool.pages[0].page
+ assert await page.evaluate(READ_STATE) == SAVED_STATE
+ assert await page.evaluate("sessionStorage.length") == 0
+ await page.evaluate(WRITE_STATE, "changed")
+ changed = await page.evaluate(READ_STATE)
+ assert changed["indexedDB"]["value"] == "changed" and changed["databases"] == ["auth", "stale"]
+ assert "stale" in changed["cookies"] and "stale" in changed["localStorage"]
+ await restored.load_state(native_path)
+ assert page.url == URL and not page.is_closed()
+ assert await page.evaluate("sessionStorage.getItem('temporary')") == "changed"
+ await restored.fetch(URL)
+ assert await page.evaluate(READ_STATE) == SAVED_STATE
+ assert await restored.save_state(str(path)) is None
+ _check_json(path, loads(native_path.read_text()))
+
+
[email protected]("session_type", [DynamicSession, StealthySession])
+def test_sync_state_file_errors(session_type: Any, tmp_path: Path) -> None:
+ invalid = tmp_path / "invalid.json"
+ invalid.write_text("{")
+ with session_type(**_options()) as session:
+ session.context.add_cookies([{"name": "kept", "value": "true", "url": URL}])
+ for path, error in ((tmp_path / "missing.json", FileNotFoundError), (invalid, JSONDecodeError)):
+ with pytest.raises(error):
+ session.load_state(path)
+ assert [(cookie["name"], cookie["value"]) for cookie in session.context.cookies()] == [("kept", "true")]
+ with pytest.raises(FileNotFoundError):
+ session.save_state(tmp_path / "missing" / "state.json")
+ with pytest.raises(IsADirectoryError):
+ session.save_state("")
+
+
[email protected]("session_type", [AsyncDynamicSession, AsyncStealthySession])
+async def test_async_state_file_errors(session_type: Any, tmp_path: Path) -> None:
+ invalid = tmp_path / "invalid.json"
+ invalid.write_text("{")
+ async with session_type(**_options()) as session:
+ await session.context.add_cookies([{"name": "kept", "value": "true", "url": URL}])
+ for path, error in ((tmp_path / "missing.json", FileNotFoundError), (invalid, JSONDecodeError)):
+ with pytest.raises(error):
+ await session.load_state(path)
+ assert [(cookie["name"], cookie["value"]) for cookie in await session.context.cookies()] == [("kept", "true")]
+ with pytest.raises(FileNotFoundError):
+ await session.save_state(tmp_path / "missing" / "state.json")
+ with pytest.raises(IsADirectoryError):
+ await session.save_state("")
+
+
[email protected]("session_type", [DynamicSession, StealthySession])
+def test_sync_state_before_start(session_type: Any, tmp_path: Path) -> None:
+ session = session_type(**_options())
+ for method in (session.save_state, session.load_state):
+ with pytest.raises(RuntimeError, match=STATE_ERROR):
+ method(tmp_path / "state.json")
+
+
[email protected]("session_type", [AsyncDynamicSession, AsyncStealthySession])
+async def test_async_state_before_start(session_type: Any, tmp_path: Path) -> None:
+ session = session_type(**_options())
+ for method in (session.save_state, session.load_state):
+ with pytest.raises(RuntimeError, match=STATE_ERROR):
+ await method(tmp_path / "state.json")
+
+
[email protected]("session_type", [DynamicSession, StealthySession])
+def test_sync_state_with_proxy_rotation(session_type: Any, tmp_path: Path) -> None:
+ with session_type(**_options(proxy_rotator=ProxyRotator(["http://127.0.0.1:1"]))) as session:
+ for method in (session.save_state, session.load_state):
+ with pytest.raises(RuntimeError, match=STATE_ERROR):
+ method(tmp_path / "state.json")
+
+
[email protected]("session_type", [AsyncDynamicSession, AsyncStealthySession])
+async def test_async_state_with_proxy_rotation(session_type: Any, tmp_path: Path) -> None:
+ async with session_type(**_options(proxy_rotator=ProxyRotator(["http://127.0.0.1:1"]))) as session:
+ for method in (session.save_state, session.load_state):
+ with pytest.raises(RuntimeError, match=STATE_ERROR):
+ await method(tmp_path / "state.json")