🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Python aiai-scrapingautomationcrawlercrawlingcrawling-pythondatadata-extractionmcpmcp-serverplaywrightpythonscrapingselectorsstealthweb-scraperweb-scrapingweb-scraping-pythonwebscrapingxpath
style(spiders): Lower the default number of concurrent requests
Karim shoair2eea66dad157
diff --git a/scrapling/spiders/spider.py b/scrapling/spiders/spider.pyindex f14174f..c994d2b 100644--- a/scrapling/spiders/spider.py+++ b/scrapling/spiders/spider.py@@ -73,7 +73,7 @@ class Spider(ABC):allowed_domains: Set[str] = set()# Concurrency settings- concurrent_requests: int = 16+ concurrent_requests: int = 4concurrent_requests_per_domain: int = 0download_delay: float = 0.0max_blocked_retries: int = 3