Skip to content

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

Update camo.py

Karim shoairee8f78feb1ac
diff --git a/scrapling/engines/camo.py b/scrapling/engines/camo.py
index 5335df5..f56ad69 100644
--- a/scrapling/engines/camo.py
+++ b/scrapling/engines/camo.py
@@ -87,11 +87,7 @@ class CamoufoxEngine:
if self.extra_headers:
page.set_extra_http_headers(self.extra_headers)
- if self.google_search:
- res = page.goto(url, referer=generate_convincing_referer(url))
- else:
- res = page.goto(url)
-
+ res = page.goto(url, referer=generate_convincing_referer(url) if self.google_search else None)
page.wait_for_load_state(state="load")
page.wait_for_load_state(state="domcontentloaded")
if self.network_idle: