Skip to content

Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.

fix: default max_pages_before_recycle in DEFAULT_CONFIG

ntohidi15f4b29b8ba2
diff --git a/deploy/docker/utils.py b/deploy/docker/utils.py
index 8f8ecb18..0fc2e69c 100644
--- a/deploy/docker/utils.py
+++ b/deploy/docker/utils.py
@@ -64,7 +64,7 @@ DEFAULT_CONFIG = {
"memory_threshold_percent": 95.0,
"rate_limiter": {"enabled": True, "base_delay": [1.0, 2.0]},
"timeouts": {"stream_init": 30.0, "batch_process": 300.0},
- "pool": {"max_pages": 40, "idle_ttl_sec": 300},
+ "pool": {"max_pages": 40, "idle_ttl_sec": 300, "max_pages_before_recycle": 200},
"browser": {
"kwargs": {"headless": True, "text_mode": True},
"extra_args": [