Skip to content

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

Stop using generated headers by browserforge

Karim shoair15b7e3aceeb9
diff --git a/scrapling/engines/pw.py b/scrapling/engines/pw.py
index 32c336a..6aa89ec 100644
--- a/scrapling/engines/pw.py
+++ b/scrapling/engines/pw.py
@@ -130,8 +130,8 @@ class PlaywrightEngine:
extra_headers = {}
useragent = self.useragent
else:
- extra_headers = generate_headers(browser_mode=True)
- useragent = extra_headers.get('User-Agent')
+ extra_headers = {}
+ useragent = generate_headers(browser_mode=True).get('User-Agent')
# Prepare the flags before diving
flags = DEFAULT_STEALTH_FLAGS