Stealth browser automation for AI agents over MCP and the Chrome DevTools Protocol: navigation, network hooks, DOM extraction, and pixel-accurate UI cloning.
Python ai-agent-toolsai-toolsanthropicanti-bot-bypassautomation-toolsbrowser-automationchrome-devtoolsclaude-mcpcloudflare-bypassdeveloper-toolsfastmcpmcp-servermodel-context-protocolnetwork-interceptionnodriverpythonstealth-browserundetectable-browserweb-automationweb-scraping
fix(cdp): enable page domain for page commands
Vibhek Sonieda1aebc5792
diff --git a/README.md b/README.mdindex 370d12e..d43a368 100644--- a/README.md+++ b/README.md@@ -53,8 +53,8 @@ Bypass Cloudflare, antibot systems, and social media blocks with real browser in## Features- **Antibot bypass** — Works on Cloudflare, Queue-It, and other protection systems that block traditional automation-- **96 tools across 11 sections** — From basic navigation to advanced CDP function execution-- **Modular loading** — Run the full 96-tool arsenal or a minimal 20-tool core; disable what you don't need+- **97 tools across 11 sections** — From basic navigation to advanced CDP function execution+- **Modular loading** — Run the full 97-tool arsenal or a minimal 20-tool core; disable what you don't need- **Pixel-accurate element cloning** — Extract complete elements with all CSS, DOM structure, events, and assets via CDP- **Network interception** — Inspect every request, response, header, and payload through your AI agent- **Dynamic hook system** — AI-generated Python functions that intercept and modify network traffic in real-time@@ -158,11 +158,11 @@ Restart your MCP client and ask your agent:## Modular Architecture-Choose exactly what functionality you need. Run the full 96-tool suite or strip it down to 20 core tools.+Choose exactly what functionality you need. Run the full 97-tool suite or strip it down to 20 core tools.| Mode | Tools | Use Case ||------|-------|----------|-| **Full** (default) | 96 | Complete browser automation and debugging |+| **Full** (default) | 97 | Complete browser automation and debugging || **Minimal** (`--minimal`) | 20 | Core browser automation only || **Custom** (`--disable-*`) | Your choice | Disable specific sections |@@ -293,7 +293,7 @@ Examples:| `element-extraction` | 9 | Element cloning and extraction || `file-extraction` | 9 | File-based extraction tools || `network-debugging` | 10 | Network monitoring and interception |-| `cdp-functions` | 13 | Chrome DevTools Protocol execution |+| `cdp-functions` | 14 | Chrome DevTools Protocol execution || `progressive-cloning` | 10 | Advanced element cloning || `cookies-storage` | 3 | Cookie and storage management || `tabs` | 5 | Tab management |@@ -402,6 +402,7 @@ Examples:| Tool | Description ||------|-------------|| `execute_cdp_command()` | Direct CDP commands (use snake_case) |+| `add_script_to_evaluate_on_new_document()` | Install pre-document scripts for API spoofing || `discover_global_functions()` | Find JavaScript functions || `discover_object_methods()` | Discover object methods (93+ methods) || `call_javascript_function()` | Execute any function |@@ -491,8 +492,8 @@ Examples:| Network debugging | Full request/response inspection via AI | Basic || API reverse engineering | Payload inspection through chat | Manual tools only || Dynamic hook system | AI-generated Python functions for real-time interception | Not available |-| Modular architecture | 11 sections, 20–96 tools | Fixed ~20 tools |-| Total tools | 96 (customizable) | ~20 |+| Modular architecture | 11 sections, 20–97 tools | Fixed ~20 tools |+| Total tools | 97 (customizable) | ~20 |Tested on: LinkedIn, Instagram, Twitter/X, Amazon, banking portals, government sites, Cloudflare-protected APIs, Nike SNKRS, Ticketmaster, Supreme.diff --git a/src/cdp_function_executor.py b/src/cdp_function_executor.pyindex 87b4609..bad476a 100644--- a/src/cdp_function_executor.py+++ b/src/cdp_function_executor.py@@ -100,19 +100,32 @@ class CDPFunctionExecutor:debug_logger.log_error("cdp_function_executor", "enable_runtime", e)return False+ async def enable_page(self, tab: Tab) -> bool:+ """+ Enables CDP Page domain for a tab.++ Args:+ tab (Tab): The browser tab.++ Returns:+ bool: True if enabled, False otherwise.+ """+ try:+ await tab.send(uc.cdp.page.enable())+ debug_logger.log_info("cdp_function_executor", "enable_page", "Page enabled for tab")+ return True+ except Exception as e:+ debug_logger.log_error("cdp_function_executor", "enable_page", e)+ return False+async def list_cdp_commands(self) -> List[str]:"""Lists CDP commands the generic executor will accept.- Includes core Runtime methods plus a curated set of Page-domain- methods (notably addScriptToEvaluateOnNewDocument) that are useful- for pre-page-script API spoofing (WebGL renderer, navigator props).-Returns:List[str]: List of command names."""commands = [- # Runtime domain"evaluate", "callFunctionOn", "addBinding", "removeBinding","compileScript", "runScript", "awaitPromise", "getProperties","getExceptionDetails", "globalLexicalScopeNames", "queryObjects",@@ -120,11 +133,14 @@ class CDPFunctionExecutor:"setAsyncCallStackDepth", "setCustomObjectFormatterEnabled","setMaxCallStackSizeToCapture", "runIfWaitingForDebugger","discardConsoleEntries", "getHeapUsage", "getIsolateId",- # Page domain (resolved from uc.cdp.page when not found on Runtime)"addScriptToEvaluateOnNewDocument","removeScriptToEvaluateOnNewDocument","reload", "navigate", "stopLoading", "getFrameTree","setBypassCSP", "setDocumentContent",+ "page.enable", "page.addScriptToEvaluateOnNewDocument",+ "page.removeScriptToEvaluateOnNewDocument",+ "page.reload", "page.navigate", "page.stopLoading",+ "page.getFrameTree", "page.setBypassCSP", "page.setDocumentContent",]return commands@@ -132,17 +148,32 @@ class CDPFunctionExecutor:"""Resolve a CDP command name across Runtime and Page domains.- nodriver exposes each CDP domain as a submodule under uc.cdp (e.g.- uc.cdp.runtime, uc.cdp.page). Method names are snake_cased on the- Python side but the wire uses camelCase. We accept either.- """- snake = "".join(["_" + c.lower() if c.isupper() else c for c in command]).lstrip("_")- for domain in (uc.cdp.runtime, uc.cdp.page):- for name in (command, snake):+ Args:+ command (str): CDP command name with optional domain prefix.++ Returns:+ tuple: Domain name and resolved method, or None values.+ """+ cdp_domains = {+ "runtime": uc.cdp.runtime,+ "page": uc.cdp.page,+ }+ raw_command = command.strip()+ explicit_domain = None+ if "." in raw_command:+ domain_part, raw_command = raw_command.split(".", 1)+ explicit_domain = domain_part.strip().lower()+ snake = "".join(["_" + c.lower() if c.isupper() else c for c in raw_command]).lstrip("_")+ domain_names = [explicit_domain] if explicit_domain else ["runtime", "page"]+ for domain_name in domain_names:+ domain = cdp_domains.get(domain_name)+ if domain is None:+ continue+ for name in (raw_command, snake):method = getattr(domain, name, None)if callable(method):- return method- return None+ return domain_name, method+ return None, Noneasync def execute_cdp_command(self, tab: Tab, command: str, params: Dict[str, Any]) -> Dict[str, Any]:"""@@ -157,10 +188,13 @@ class CDPFunctionExecutor:Dict[str, Any]: Result of the command execution."""try:- await self.enable_runtime(tab)- cdp_method = self._resolve_cdp_method(command)+ cdp_domain, cdp_method = self._resolve_cdp_method(command)if not cdp_method:raise ValueError(f"Unknown CDP command: {command}")+ if cdp_domain == "page":+ await self.enable_page(tab)+ else:+ await self.enable_runtime(tab)result = await tab.send(cdp_method(**params))debug_logger.log_info("cdp_function_executor", "execute_cdp_command", f"Executed {command} with params: {params}")return {@@ -865,4 +899,4 @@ class CDPFunctionExecutor:"dynamic_script_injection","python_js_bridge"]- }\ No newline at end of file+ }diff --git a/src/server.py b/src/server.pyindex 2c7a52d..dd51e90 100644--- a/src/server.py+++ b/src/server.py@@ -2327,45 +2327,37 @@ async def execute_cdp_command(async def add_script_to_evaluate_on_new_document(instance_id: str,source: str,+ world_name: Optional[str] = None,+ include_command_line_api: Optional[bool] = None,run_immediately: bool = True,) -> Dict[str, Any]:"""- Install a JavaScript snippet that runs on EVERY new document/frame BEFORE- any page script executes. Use this to spoof browser APIs (WebGL renderer,- navigator props, etc.) so values are already overridden when bot-detection- pages query them inline at parse time.-- Wraps CDP Page.addScriptToEvaluateOnNewDocument. Auto-enables Page domain- first (without that, the addScript silently no-ops on a fresh tab).+ Install JavaScript that runs before page scripts on each new document.Args:instance_id (str): Browser instance ID.- source (str): JavaScript source. Runs in the page's main world before- any other script on every navigation, including iframes.- run_immediately (bool): If True (default), the script also runs on the- current document if one is already loaded. Useful when- the tab is at about:blank and you want the spoof active- before any user navigation.+ source (str): JavaScript source to evaluate before page scripts.+ world_name (Optional[str]): Isolated world name, or None for the main world.+ include_command_line_api (Optional[bool]): Whether command line API is available.+ run_immediately (bool): Whether to run the script in existing contexts too.Returns:- Dict[str, Any]: { success, identifier } where identifier can be passed- to Page.removeScriptToEvaluateOnNewDocument later.+ Dict[str, Any]: Result with success state and script identifier."""- import nodriver as uctab = await browser_manager.get_tab(instance_id)if not tab:return {"success": False, "error": f"Instance not found: {instance_id}"}try:- # CRITICAL: Page domain must be enabled before addScript will take effect.- # nodriver does not auto-enable it for ad-hoc Page method calls.await tab.send(uc.cdp.page.enable())result = await tab.send(uc.cdp.page.add_script_to_evaluate_on_new_document(source=source,+ world_name=world_name,+ include_command_line_api=include_command_line_api,run_immediately=run_immediately,))- identifier = getattr(result, "identifier", None) or (result if isinstance(result, str) else None)- return {"success": True, "identifier": identifier}+ return {"success": True, "identifier": str(result)}except Exception as e:+ debug_logger.log_error("server", "add_script_to_evaluate_on_new_document", e)return {"success": False, "error": str(e)}