Skip to content

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

fix(parser): Better approach for web pages where the encoding is not always correctly declared

Karim shoair170205599d45
diff --git a/scrapling/parser.py b/scrapling/parser.py
index ac97b7d..95b88ee 100644
--- a/scrapling/parser.py
+++ b/scrapling/parser.py
@@ -121,7 +121,7 @@ class Selector(SelectorsGeneration):
self.__text = None
if root is None:
if isinstance(content, str):
- body = content.strip().replace("\x00", "").encode(encoding) or b"<html/>"
+ body = content.strip().replace("\x00", "") or "<html/>"
elif isinstance(content, bytes):
body = content.replace(b"\x00", b"")
else: