Skip to content

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

fix: remove dead fetchall() after INSERT and correct retrieve() error message (#377)

Semianchuk Vitalii2f400555c3cd
diff --git a/README.md b/README.md
index 8d95c33..2f39ac9 100644
--- a/README.md
+++ b/README.md
@@ -98,11 +98,11 @@ MySpider().start()
</tr>
<tr>
<td width="200">
- <a href="https://coldproxy.com/" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
+ <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/coldproxy.png">
</a>
</td>
- <td> <a href="https://coldproxy.com/" target="_blank"><b>ColdProxy</b></a> provides residential and datacenter proxies for stable web scraping, public data collection, and geo-targeted testing across 195+ countries.
+ <td> <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank"><b>ColdProxy</b></a> provides residential and datacenter proxies for stable web scraping, public data collection, and geo-targeted testing across 195+ countries.
</td>
</tr>
<tr>
diff --git a/docs/README_AR.md b/docs/README_AR.md
index 972c562..4a43fea 100644
--- a/docs/README_AR.md
+++ b/docs/README_AR.md
@@ -94,11 +94,11 @@ MySpider().start()
</tr>
<tr>
<td width="200">
- <a href="https://coldproxy.com/" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
+ <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/coldproxy.png">
</a>
</td>
- <td> توفر <a href="https://coldproxy.com/" target="_blank"><b>ColdProxy</b></a> وكلاء سكنيين ووكلاء مراكز بيانات لاستخراج بيانات الويب بشكل مستقر، وجمع البيانات العامة، والاختبار الموجَّه جغرافياً في أكثر من 195 دولة.
+ <td> توفر <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank"><b>ColdProxy</b></a> وكلاء سكنيين ووكلاء مراكز بيانات لاستخراج بيانات الويب بشكل مستقر، وجمع البيانات العامة، والاختبار الموجَّه جغرافياً في أكثر من 195 دولة.
</td>
</tr>
<tr>
diff --git a/docs/README_CN.md b/docs/README_CN.md
index 9a144e2..c1ec984 100644
--- a/docs/README_CN.md
+++ b/docs/README_CN.md
@@ -94,11 +94,11 @@ MySpider().start()
</tr>
<tr>
<td width="200">
- <a href="https://coldproxy.com/" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
+ <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/coldproxy.png">
</a>
</td>
- <td> <a href="https://coldproxy.com/" target="_blank"><b>ColdProxy</b></a> 提供住宅代理和数据中心代理,用于稳定的网络抓取、公共数据收集,以及覆盖 195 多个国家/地区的地理定向测试。
+ <td> <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank"><b>ColdProxy</b></a> 提供住宅代理和数据中心代理,用于稳定的网络抓取、公共数据收集,以及覆盖 195 多个国家/地区的地理定向测试。
</td>
</tr>
<tr>
diff --git a/docs/README_DE.md b/docs/README_DE.md
index aad2ad2..c2d5079 100644
--- a/docs/README_DE.md
+++ b/docs/README_DE.md
@@ -94,11 +94,11 @@ MySpider().start()
</tr>
<tr>
<td width="200">
- <a href="https://coldproxy.com/" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
+ <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/coldproxy.png">
</a>
</td>
- <td> <a href="https://coldproxy.com/" target="_blank"><b>ColdProxy</b></a> bietet Residential- und Datacenter-Proxies für stabiles Web Scraping, öffentliche Datenerfassung und geografisch gezielte Tests in über 195 Ländern.
+ <td> <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank"><b>ColdProxy</b></a> bietet Residential- und Datacenter-Proxies für stabiles Web Scraping, öffentliche Datenerfassung und geografisch gezielte Tests in über 195 Ländern.
</td>
</tr>
<tr>
diff --git a/docs/README_ES.md b/docs/README_ES.md
index ce1513d..e5a7b22 100644
--- a/docs/README_ES.md
+++ b/docs/README_ES.md
@@ -94,11 +94,11 @@ MySpider().start()
</tr>
<tr>
<td width="200">
- <a href="https://coldproxy.com/" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
+ <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/coldproxy.png">
</a>
</td>
- <td> <a href="https://coldproxy.com/" target="_blank"><b>ColdProxy</b></a> proporciona proxies residenciales y de centros de datos para web scraping estable, recopilación de datos públicos y pruebas con segmentación geográfica en más de 195 países.
+ <td> <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank"><b>ColdProxy</b></a> proporciona proxies residenciales y de centros de datos para web scraping estable, recopilación de datos públicos y pruebas con segmentación geográfica en más de 195 países.
</td>
</tr>
<tr>
diff --git a/docs/README_FR.md b/docs/README_FR.md
index e1694c6..f27ff24 100644
--- a/docs/README_FR.md
+++ b/docs/README_FR.md
@@ -94,11 +94,11 @@ MySpider().start()
</tr>
<tr>
<td width="200">
- <a href="https://coldproxy.com/" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
+ <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/coldproxy.png">
</a>
</td>
- <td> <a href="https://coldproxy.com/" target="_blank"><b>ColdProxy</b></a> fournit des proxies résidentiels et de datacenter pour un web scraping stable, la collecte de données publiques et des tests géolocalisés dans plus de 195 pays.
+ <td> <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank"><b>ColdProxy</b></a> fournit des proxies résidentiels et de datacenter pour un web scraping stable, la collecte de données publiques et des tests géolocalisés dans plus de 195 pays.
</td>
</tr>
<tr>
diff --git a/docs/README_JP.md b/docs/README_JP.md
index f0e30dc..a9f0e10 100644
--- a/docs/README_JP.md
+++ b/docs/README_JP.md
@@ -94,11 +94,11 @@ MySpider().start()
</tr>
<tr>
<td width="200">
- <a href="https://coldproxy.com/" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
+ <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/coldproxy.png">
</a>
</td>
- <td> <a href="https://coldproxy.com/" target="_blank"><b>ColdProxy</b></a> は、安定したウェブスクレイピング、公開データ収集、195以上の国・地域でのジオターゲティングテストのために、レジデンシャルおよびデータセンタープロキシを提供します。
+ <td> <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank"><b>ColdProxy</b></a> は、安定したウェブスクレイピング、公開データ収集、195以上の国・地域でのジオターゲティングテストのために、レジデンシャルおよびデータセンタープロキシを提供します。
</td>
</tr>
<tr>
diff --git a/docs/README_KR.md b/docs/README_KR.md
index 6b013a7..193530a 100644
--- a/docs/README_KR.md
+++ b/docs/README_KR.md
@@ -94,11 +94,11 @@ MySpider().start()
</tr>
<tr>
<td width="200">
- <a href="https://coldproxy.com/" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
+ <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/coldproxy.png">
</a>
</td>
- <td> <a href="https://coldproxy.com/" target="_blank"><b>ColdProxy</b></a>는 안정적인 웹 스크래핑, 공개 데이터 수집, 195개 이상의 국가에서의 지역 타겟팅 테스트를 위한 주거용 및 데이터센터 프록시를 제공합니다.
+ <td> <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank"><b>ColdProxy</b></a>는 안정적인 웹 스크래핑, 공개 데이터 수집, 195개 이상의 국가에서의 지역 타겟팅 테스트를 위한 주거용 및 데이터센터 프록시를 제공합니다.
</td>
</tr>
<tr>
diff --git a/docs/README_PT_BR.md b/docs/README_PT_BR.md
index 65de3a2..e1defaa 100644
--- a/docs/README_PT_BR.md
+++ b/docs/README_PT_BR.md
@@ -96,11 +96,11 @@ MySpider().start()
</tr>
<tr>
<td width="200">
- <a href="https://coldproxy.com/" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
+ <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/coldproxy.png">
</a>
</td>
- <td> A <a href="https://coldproxy.com/" target="_blank"><b>ColdProxy</b></a> oferece proxies residenciais e de datacenter para web scraping estável, coleta de dados públicos e testes com segmentação geográfica em mais de 195 países.
+ <td> A <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank"><b>ColdProxy</b></a> oferece proxies residenciais e de datacenter para web scraping estável, coleta de dados públicos e testes com segmentação geográfica em mais de 195 países.
</td>
</tr>
<tr>
diff --git a/docs/README_RU.md b/docs/README_RU.md
index fdb708e..c153482 100644
--- a/docs/README_RU.md
+++ b/docs/README_RU.md
@@ -94,11 +94,11 @@ MySpider().start()
</tr>
<tr>
<td width="200">
- <a href="https://coldproxy.com/" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
+ <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/coldproxy.png">
</a>
</td>
- <td> <a href="https://coldproxy.com/" target="_blank"><b>ColdProxy</b></a> предоставляет резидентные и дата-центровые прокси для стабильного веб-скрейпинга, сбора публичных данных и гео-таргетированного тестирования в более чем 195 странах.
+ <td> <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank"><b>ColdProxy</b></a> предоставляет резидентные и дата-центровые прокси для стабильного веб-скрейпинга, сбора публичных данных и гео-таргетированного тестирования в более чем 195 странах.
</td>
</tr>
<tr>
diff --git a/docs/index.md b/docs/index.md
index fdf604f..4a886fa 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -59,7 +59,7 @@ MySpider().start()
<a href="https://proxidize.com/?utm_source=github&utm_medium=sponsorship&utm_campaign=scrapling&utm_content=d4vinci" target="_blank" title="Clean Proxies with No Nonsense.">
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxidize.png" class="ad">
</a>
- <a href="https://coldproxy.com/" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
+ <a href="https://coldproxy.com/?utm_source=scrapling&utm_medium=github&utm_campaign=coldproxy&utm_content=platinum_sponsor" target="_blank" title="Residential, IPv6 & Datacenter Proxies for Web Scraping">
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/coldproxy.png" class="ad">
</a>
<a href="https://hypersolutions.co/?utm_source=github&utm_medium=readme&utm_campaign=scrapling" target="_blank" title="Bot Protection Bypass API for Akamai, DataDome, Incapsula & Kasada">
diff --git a/scrapling/core/storage.py b/scrapling/core/storage.py
index d93a320..1fccabe 100644
--- a/scrapling/core/storage.py
+++ b/scrapling/core/storage.py
@@ -56,7 +56,7 @@ class StorageSystemMixin(ABC): # pragma: no cover
the docs for more info.
:return: A dictionary of the unique properties
"""
- raise NotImplementedError("Storage system must implement `save` method")
+ raise NotImplementedError("Storage system must implement `retrieve` method")
@staticmethod
@lru_cache(128, typed=True)
@@ -123,7 +123,6 @@ class SQLiteStorageSystem(StorageSystemMixin):
""",
(url, identifier, dumps(element_data)),
)
- self.cursor.fetchall()
self.connection.commit()
def retrieve(self, identifier: str) -> Optional[Dict[str, Any]]: