Skip to content

Open-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent API.

v1 actor

Gregor Žunič11d37dd68df4
diff --git a/browser_use/actor/README.md b/browser_use/actor/README.md
new file mode 100644
index 000000000..162f7fcc2
--- /dev/null
+++ b/browser_use/actor/README.md
@@ -0,0 +1,168 @@
+# Browser Actor
+
+Browser Actor is a web automation library built directly on CDP.
+
+## Usage
+
+```python
+from cdp_use import CDPClient
+from browser_use.actor import Browser, Target, Element, Mouse
+
+# Create client and browser
+client = CDPClient(ws_url)
+browser = Browser(client)
+```
+
+```python
+# Get targets (multiple ways)
+target = await browser.newTarget() # Create blank tab
+target = await browser.newTarget("https://example.com") # Create tab with URL
+targets = await browser.getTargets() # Get all existing tabs
+
+# Navigate target to URL
+await target.goto("https://example.com")
+
+await browser.closeTarget(target)
+```
+
+```python
+# Find elements by CSS selector
+elements = await target.getElementsByCSSSelector("input[type='text']")
+buttons = await target.getElementsByCSSSelector("button.submit")
+
+# Get element by backend node ID
+element = await target.getElement(backend_node_id=12345)
+```
+
+Unlike other libraries, the native implementation for `getElementsByCSSSelector` does not support waiting for the element to be visible.
+
+```python
+# Element actions
+await element.click(button='left', click_count=1, modifiers=['Control'])
+await element.fill("Hello World")
+await element.hover()
+await element.focus()
+await element.check()
+await element.selectOption(["option1", "option2"])
+
+# Element properties
+value = await element.getAttribute("value")
+box = await element.getBoundingBox()
+info = await element.getBasicInfo()
+```
+
+```python
+# Mouse operations
+mouse = await target.mouse
+await mouse.click(x=100, y=200, button='left')
+await mouse.move(x=300, y=400)
+```
+
+```python
+# Target operations
+mouse = await target.mouse
+await mouse.scroll(x=0, y=100, delta_y=-500) # x,y (coordinates to scroll on), delta_y (how much to scroll)
+await target.press("Control+A") # Key combinations supported
+await target.press("Escape")
+await target.setViewportSize(width=1920, height=1080)
+await target.reload()
+page_screenshot = await target.screenshot() # JPEG by default
+page_png = await target.screenshot(format="png")
+```
+Ø
+## Core Classes
+
+- **Browser**, **Target**, **Element**, **Mouse**: Core classes for browser operations
+
+## API Reference
+
+### Browser Methods
+- `newTarget(url=None)` → `Target` - Create blank tab or navigate to URL
+- `getTargets()` → `list[Target]` - Get all page/iframe targets
+- `closeTarget(target: Target | str)` - Close target by object or ID
+- `cookies(urls=None)` → `list[Cookie]` - Get cookies for specified URLs (or all if None)
+- `clearCookies()` - Clear all cookies
+
+### Target Methods
+- `getElementsByCSSSelector(selector: str)` → `list[Element]` - Find elements by CSS selector
+- `getElement(backend_node_id: int)` → `Element` - Get element by backend node ID
+- `goto(url: str)` - Navigate this target to URL
+- `goBack()`, `goForward()` - Navigate target history (with proper error handling)
+- `evaluate(page_function: str, *args)` → `str` - Execute JavaScript (MUST use (...args) => format) and return string (objects/arrays are JSON-stringified)
+- `press(key: str)` - Press key on page (supports "Control+A" format)
+- `setViewportSize(width: int, height: int)` - Set viewport dimensions
+- `reload()` - Reload the current page
+- `screenshot(format='jpeg', quality=None)` → `str` - Take page screenshot and return base64
+- `getUrl()` → `str`, `getTitle()` → `str` - Get page info
+
+### Element Methods (Supported Only)
+- `click(button='left', click_count=1, modifiers=None)` - Click element
+- `fill(text: str)` - Fill input with text (clears first)
+- `hover()` - Hover over element
+- `focus()` - Focus the element
+- `check()` - Toggle checkbox/radio button (clicks to change state)
+- `selectOption(values: str | list[str])` - Select dropdown options (string or array)
+- `dragTo(target: Element | Position, source_position=None, target_position=None)` - Drag to target
+- `getAttribute(name: str)` → `str | None` - Get attribute value
+- `getBoundingBox()` → `BoundingBox | None` - Get element position/size
+- `screenshot(format='jpeg', quality=None)` → `str` - Take element screenshot and return base64
+- `getBasicInfo()` → `ElementInfo` - Get comprehensive element information
+
+
+### Mouse Methods
+- `click(x: int, y: int, button='left', click_count=1)` - Click at coordinates
+- `move(x: int, y: int, steps=1)` - Move to coordinates
+- `down(button='left')`, `up(button='left')` - Press/release button
+- `scroll(x=0, y=0, delta_x=None, delta_y=None)` - Scroll page (x,y coordinates to scroll on, delta_x/delta_y how much to scroll)
+
+## Type Definitions
+
+### Position
+```python
+class Position(TypedDict):
+ x: float
+ y: float
+```
+
+### BoundingBox
+```python
+class BoundingBox(TypedDict):
+ x: float
+ y: float
+ width: float
+ height: float
+```
+
+### ElementInfo
+```python
+class ElementInfo(TypedDict):
+ backendNodeId: int
+ nodeId: int | None
+ nodeName: str
+ nodeType: int
+ nodeValue: str | None
+ attributes: dict[str, str]
+ boundingBox: BoundingBox | None
+```
+
+## Important LLM Usage Notes
+
+**This is NOT Playwright.**. You can NOT use other methods than the ones described here. Key constraints for code generation:
+
+**CRITICAL JAVASCRIPT EVALUATION RULES:**
+- `target.evaluate()` MUST use (...args) => format and always returns string (objects become JSON strings)
+- **STRING QUOTES**: Always use `target.evaluate('...')` (single quotes outside, double inside for CSS)
+- **CSS SELECTORS**: Use `"input[name=\\"email\\"]"` format inside evaluate calls
+- **ESCAPING**: Use `\\"` to escape double quotes inside selectors, never mix quote patterns
+
+**METHOD RESTRICTIONS:**
+- `getElementsByCSSSelector()` returns immediately, no waiting
+- For dropdowns: use `element.selectOption("value")` or `element.selectOption(["val1", "val2"])`, not `element.fill()`
+- No methods: `element.submit()`, `element.dispatchEvent()`, `element.getProperty()`, `target.querySelectorAll()`
+- Form submission: click submit button or use `target.press("Enter")`
+- Get properties: use `target.evaluate("() => element.value")` not `element.getProperty()`
+
+**ERROR PREVENTION:**
+- Loop prevention: verify page state changes with `target.getUrl()`, `target.getTitle()`, `element.getAttribute()`
+- Validate selectors before use: ensure no excessive escaping like `\\\\\\\\`
+- Test complex selectors: if a selector fails, simplify it step by step
diff --git a/browser_use/actor/__init__.py b/browser_use/actor/__init__.py
new file mode 100644
index 000000000..922ad703b
--- /dev/null
+++ b/browser_use/actor/__init__.py
@@ -0,0 +1,11 @@
+"""CDP-Use High-Level Library
+
+A Playwright-like library built on top of CDP (Chrome DevTools Protocol).
+"""
+
+from .browser import Browser
+from .element import Element
+from .mouse import Mouse
+from .target import Target
+
+__all__ = ['Browser', 'Target', 'Element', 'Mouse']
diff --git a/browser_use/actor/browser.py b/browser_use/actor/browser.py
new file mode 100644
index 000000000..390302c2b
--- /dev/null
+++ b/browser_use/actor/browser.py
@@ -0,0 +1,73 @@
+"""Browser class for high-level CDP operations."""
+
+from typing import TYPE_CHECKING, Union
+
+if TYPE_CHECKING:
+ from cdp_use.cdp.network.library import GetCookiesParameters
+ from cdp_use.cdp.network.types import Cookie
+ from cdp_use.cdp.target.commands import (
+ CloseTargetParameters,
+ CreateTargetParameters,
+ )
+ from cdp_use.client import CDPClient
+
+ from .target import Target
+
+
+class Browser:
+ """High-level browser interface built on CDP."""
+
+ def __init__(self, client: 'CDPClient'):
+ self._client = client
+
+ async def newTarget(self, url: str | None = None) -> 'Target':
+ """Create a new target (tab)."""
+ params: 'CreateTargetParameters' = {'url': url or 'about:blank'}
+ result = await self._client.send.Target.createTarget(params)
+
+ target_id = result['targetId']
+
+ # Import here to avoid circular import
+ from .target import Target
+
+ return Target(self._client, target_id)
+
+ async def getTargets(self) -> list['Target']:
+ """Get all available targets."""
+ result = await self._client.send.Target.getTargets()
+
+ targets = []
+ # Import here to avoid circular import
+ from .target import Target
+
+ for target_info in result['targetInfos']:
+ if target_info['type'] in ['page', 'iframe']:
+ targets.append(Target(self._client, target_info['targetId']))
+
+ return targets
+
+ async def closeTarget(self, target: Union['Target', str]) -> None:
+ """Close a target by Target object or target ID."""
+ # Import here to avoid circular import
+ from .target import Target
+
+ if isinstance(target, Target):
+ target_id = target._target_id
+ else:
+ target_id = str(target)
+
+ params: 'CloseTargetParameters' = {'targetId': target_id}
+ await self._client.send.Target.closeTarget(params)
+
+ async def cookies(self, urls: list[str] | None = None) -> list[Cookie]:
+ """Get cookies, optionally filtered by URLs."""
+ params: GetCookiesParameters = {}
+ if urls:
+ params['urls'] = urls
+
+ result = await self._client.send.Network.getCookies(params)
+ return result['cookies']
+
+ async def clearCookies(self) -> None:
+ """Clear all cookies."""
+ await self._client.send.Network.clearBrowserCookies()
diff --git a/browser_use/actor/element.py b/browser_use/actor/element.py
new file mode 100644
index 000000000..9e0de1a3d
--- /dev/null
+++ b/browser_use/actor/element.py
@@ -0,0 +1,610 @@
+"""Element class for element operations."""
+
+import asyncio
+from typing import TYPE_CHECKING, Literal, Union
+
+from cdp_use.client import logger
+from typing_extensions import TypedDict
+
+if TYPE_CHECKING:
+ from cdp_use.cdp.dom.commands import (
+ DescribeNodeParameters,
+ FocusParameters,
+ GetAttributesParameters,
+ GetBoxModelParameters,
+ PushNodesByBackendIdsToFrontendParameters,
+ RequestChildNodesParameters,
+ ResolveNodeParameters,
+ )
+ from cdp_use.cdp.input.commands import (
+ DispatchMouseEventParameters,
+ )
+ from cdp_use.cdp.input.types import MouseButton
+ from cdp_use.cdp.page.commands import CaptureScreenshotParameters
+ from cdp_use.cdp.page.types import Viewport
+ from cdp_use.client import CDPClient
+
+# Type definitions for element operations
+ModifierType = Literal['Alt', 'Control', 'Meta', 'Shift']
+
+
+class Position(TypedDict):
+ """2D position coordinates."""
+
+ x: float
+ y: float
+
+
+class BoundingBox(TypedDict):
+ """Element bounding box with position and dimensions."""
+
+ x: float
+ y: float
+ width: float
+ height: float
+
+
+class ElementInfo(TypedDict):
+ """Basic information about a DOM element."""
+
+ backendNodeId: int
+ nodeId: int | None
+ nodeName: str
+ nodeType: int
+ nodeValue: str | None
+ attributes: dict[str, str]
+ boundingBox: BoundingBox | None
+ error: str | None
+
+
+class Element:
+ """Element operations using BackendNodeId."""
+
+ def __init__(
+ self,
+ client: 'CDPClient',
+ backend_node_id: int,
+ session_id: str | None = None,
+ ):
+ self._client = client
+ self._backend_node_id = backend_node_id
+ self._session_id = session_id
+
+ async def _get_node_id(self) -> int:
+ """Get DOM node ID from backend node ID."""
+ params: 'PushNodesByBackendIdsToFrontendParameters' = {'backendNodeIds': [self._backend_node_id]}
+ result = await self._client.send.DOM.pushNodesByBackendIdsToFrontend(params, session_id=self._session_id)
+ return result['nodeIds'][0]
+
+ async def _get_remote_object_id(self) -> str | None:
+ """Get remote object ID for this element."""
+ node_id = await self._get_node_id()
+ params: 'ResolveNodeParameters' = {'nodeId': node_id}
+ result = await self._client.send.DOM.resolveNode(params, session_id=self._session_id)
+ object_id = result['object'].get('objectId', None)
+
+ if not object_id:
+ return None
+ return object_id
+
+ async def click(
+ self,
+ button: 'MouseButton' = 'left',
+ click_count: int = 1,
+ modifiers: list[ModifierType] | None = None,
+ ) -> None:
+ """Click the element using robust fallback strategies."""
+ # Try multiple methods to get element geometry
+ quads = []
+
+ # Method 1: Try DOM.getContentQuads first (best for inline elements)
+ try:
+ content_quads_result = await self._client.send.DOM.getContentQuads(
+ params={'backendNodeId': self._backend_node_id}, session_id=self._session_id
+ )
+ if 'quads' in content_quads_result and content_quads_result['quads']:
+ quads = content_quads_result['quads']
+ except Exception:
+ pass
+
+ # Method 2: Fall back to DOM.getBoxModel
+ if not quads:
+ try:
+ box_model = await self._client.send.DOM.getBoxModel(
+ params={'backendNodeId': self._backend_node_id}, session_id=self._session_id
+ )
+ if 'model' in box_model and 'content' in box_model['model']:
+ content_quad = box_model['model']['content']
+ if len(content_quad) >= 8:
+ quads = [content_quad]
+ except Exception:
+ pass
+
+ # Method 3: Fall back to JavaScript getBoundingClientRect
+ if not quads:
+ try:
+ result = await self._client.send.DOM.resolveNode(
+ params={'backendNodeId': self._backend_node_id}, session_id=self._session_id
+ )
+ if 'object' in result and 'objectId' in result['object']:
+ object_id = result['object']['objectId']
+
+ bounds_result = await self._client.send.Runtime.callFunctionOn(
+ params={
+ 'functionDeclaration': """
+ function() {
+ const rect = this.getBoundingClientRect();
+ return {
+ x: rect.left, y: rect.top,
+ width: rect.width, height: rect.height
+ };
+ }
+ """,
+ 'objectId': object_id,
+ 'returnByValue': True,
+ },
+ session_id=self._session_id,
+ )
+
+ if 'result' in bounds_result and 'value' in bounds_result['result']:
+ rect = bounds_result['result']['value']
+ x, y, w, h = rect['x'], rect['y'], rect['width'], rect['height']
+ quads = [[x, y, x + w, y, x + w, y + h, x, y + h]]
+ except Exception:
+ pass
+
+ # If we still don't have quads, fall back to JS click
+ if not quads:
+ try:
+ result = await self._client.send.DOM.resolveNode(
+ params={'backendNodeId': self._backend_node_id}, session_id=self._session_id
+ )
+ if 'object' in result and 'objectId' in result['object']:
+ object_id = result['object']['objectId']
+ await self._client.send.Runtime.callFunctionOn(
+ params={
+ 'functionDeclaration': 'function() { this.click(); }',
+ 'objectId': object_id,
+ },
+ session_id=self._session_id,
+ )
+ return
+ except Exception as e:
+ raise RuntimeError(f'Failed to click element: {e}')
+
+ # Calculate click coordinates from the best quad
+ best_quad = quads[0]
+ center_x = sum(best_quad[i] for i in range(0, 8, 2)) / 4
+ center_y = sum(best_quad[i] for i in range(1, 8, 2)) / 4
+
+ # Convert modifiers to CDP format
+ modifier_value = 0
+ if modifiers:
+ modifier_map = {'Alt': 1, 'Control': 2, 'Meta': 4, 'Shift': 8}
+ for mod in modifiers:
+ modifier_value |= modifier_map.get(mod, 0)
+
+ # Scroll element into view
+ try:
+ await self._client.send.DOM.scrollIntoViewIfNeeded(
+ params={'backendNodeId': self._backend_node_id}, session_id=self._session_id
+ )
+ await asyncio.sleep(0.05) # Wait for scroll to complete
+ except Exception:
+ pass # Continue even if scroll fails
+
+ # Move mouse to element first
+ try:
+ await self._client.send.Input.dispatchMouseEvent(
+ {
+ 'type': 'mouseMoved',
+ 'x': center_x,
+ 'y': center_y,
+ },
+ session_id=self._session_id,
+ )
+ await asyncio.sleep(0.05)
+ except Exception:
+ pass
+
+ # Perform the click with timeout handling
+ try:
+ # Mouse press
+ await asyncio.wait_for(
+ self._client.send.Input.dispatchMouseEvent(
+ {
+ 'type': 'mousePressed',
+ 'x': center_x,
+ 'y': center_y,
+ 'button': button,
+ 'clickCount': click_count,
+ 'modifiers': modifier_value,
+ },
+ session_id=self._session_id,
+ ),
+ timeout=1.0,
+ )
+ await asyncio.sleep(0.08)
+ except TimeoutError:
+ pass # Continue with mouse release
+
+ # Mouse release
+ try:
+ await asyncio.wait_for(
+ self._client.send.Input.dispatchMouseEvent(
+ {
+ 'type': 'mouseReleased',
+ 'x': center_x,
+ 'y': center_y,
+ 'button': button,
+ 'clickCount': click_count,
+ 'modifiers': modifier_value,
+ },
+ session_id=self._session_id,
+ ),
+ timeout=3.0,
+ )
+ except TimeoutError:
+ pass # Click may have succeeded even if release timed out
+
+ async def fill(self, value: str) -> None:
+ """Fill the input element with text using robust clearing and typing."""
+ # Focus the element first
+ try:
+ await self.focus()
+ except Exception:
+ logger.warning('Failed to focus element')
+
+ # Get object ID for advanced operations
+ result = await self._client.send.DOM.resolveNode(
+ params={'backendNodeId': self._backend_node_id}, session_id=self._session_id
+ )
+ if 'object' not in result or 'objectId' not in result['object']:
+ raise RuntimeError('Cannot resolve element for text input')
+
+ object_id = result['object']['objectId']
+
+ # Strategy 1: Direct JavaScript value setting (most reliable)
+ try:
+ await self._client.send.Runtime.callFunctionOn(
+ params={
+ 'functionDeclaration': """
+ function() {
+ this.value = "";
+ this.dispatchEvent(new Event("input", { bubbles: true }));
+ this.dispatchEvent(new Event("change", { bubbles: true }));
+ return this.value;
+ }
+ """,
+ 'objectId': object_id,
+ 'returnByValue': True,
+ },
+ session_id=self._session_id,
+ )
+ except Exception:
+ # Strategy 2: Triple-click + Delete fallback
+ try:
+ # Get element bounds for triple-click
+ bounds_result = await self._client.send.Runtime.callFunctionOn(
+ params={
+ 'functionDeclaration': 'function() { return this.getBoundingClientRect(); }',
+ 'objectId': object_id,
+ 'returnByValue': True,
+ },
+ session_id=self._session_id,
+ )
+
+ if bounds_result.get('result', {}).get('value'):
+ bounds = bounds_result['result'].get('value')
+
+ if not bounds:
+ raise RuntimeError('Element bounds not found')
+
+ center_x = bounds['x'] + bounds['width'] / 2
+ center_y = bounds['y'] + bounds['height'] / 2
+
+ # Triple-click to select all
+ await self._client.send.Input.dispatchMouseEvent(
+ {'type': 'mousePressed', 'x': center_x, 'y': center_y, 'button': 'left', 'clickCount': 3},
+ session_id=self._session_id,
+ )
+ await self._client.send.Input.dispatchMouseEvent(
+ {'type': 'mouseReleased', 'x': center_x, 'y': center_y, 'button': 'left', 'clickCount': 3},
+ session_id=self._session_id,
+ )
+
+ # Delete selected text
+ await self._client.send.Input.dispatchKeyEvent(
+ {'type': 'keyDown', 'key': 'Delete', 'code': 'Delete'},
+ session_id=self._session_id,
+ )
+ await self._client.send.Input.dispatchKeyEvent(
+ {'type': 'keyUp', 'key': 'Delete', 'code': 'Delete'},
+ session_id=self._session_id,
+ )
+ except Exception:
+ # Strategy 3: Ctrl/Cmd+A + Backspace
+ import platform
+
+ is_macos = platform.system() == 'Darwin'
+ modifier = 4 if is_macos else 2 # Meta=4 (Cmd), Ctrl=2
+
+ # Select all
+ await self._client.send.Input.dispatchKeyEvent(
+ {'type': 'keyDown', 'key': 'a', 'code': 'KeyA', 'modifiers': modifier},
+ session_id=self._session_id,
+ )
+ await self._client.send.Input.dispatchKeyEvent(
+ {'type': 'keyUp', 'key': 'a', 'code': 'KeyA', 'modifiers': modifier},
+ session_id=self._session_id,
+ )
+
+ # Delete with Backspace
+ await self._client.send.Input.dispatchKeyEvent(
+ {'type': 'keyDown', 'key': 'Backspace', 'code': 'Backspace'},
+ session_id=self._session_id,
+ )
+ await self._client.send.Input.dispatchKeyEvent(
+ {'type': 'keyUp', 'key': 'Backspace', 'code': 'Backspace'},
+ session_id=self._session_id,
+ )
+
+ # Type the new value character by character for better compatibility
+ for char in value:
+ # Send proper key events for each character
+ await self._client.send.Input.dispatchKeyEvent(
+ {'type': 'keyDown', 'key': char},
+ session_id=self._session_id,
+ )
+ await self._client.send.Input.dispatchKeyEvent(
+ {'type': 'char', 'text': char},
+ session_id=self._session_id,
+ )
+ await self._client.send.Input.dispatchKeyEvent(
+ {'type': 'keyUp', 'key': char},
+ session_id=self._session_id,
+ )
+ await asyncio.sleep(0.001) # Small delay for natural typing
+
+ async def hover(self) -> None:
+ """Hover over the element."""
+ box = await self.getBoundingBox()
+ if not box:
+ raise RuntimeError('Element is not visible or has no bounding box')
+
+ x = box['x'] + box['width'] / 2
+ y = box['y'] + box['height'] / 2
+
+ params: 'DispatchMouseEventParameters' = {'type': 'mouseMoved', 'x': x, 'y': y}
+ await self._client.send.Input.dispatchMouseEvent(params, session_id=self._session_id)
+
+ async def focus(self) -> None:
+ """Focus the element."""
+ node_id = await self._get_node_id()
+ params: 'FocusParameters' = {'nodeId': node_id}
+ await self._client.send.DOM.focus(params, session_id=self._session_id)
+
+ async def check(self) -> None:
+ """Check or uncheck a checkbox/radio button."""
+ await self.click()
+
+ async def selectOption(self, values: str | list[str]) -> None:
+ """Select option(s) in a select element."""
+ if isinstance(values, str):
+ values = [values]
+
+ # Focus the element first
+ try:
+ await self.focus()
+ except Exception:
+ logger.warning('Failed to focus element')
+
+ # For select elements, we need to find option elements and click them
+ # This is a simplified approach - in practice, you might need to handle
+ # different select types (single vs multi-select) differently
+ node_id = await self._get_node_id()
+
+ # Request child nodes to get the options
+ params: 'RequestChildNodesParameters' = {'nodeId': node_id, 'depth': 1}
+ await self._client.send.DOM.requestChildNodes(params, session_id=self._session_id)
+
+ # Get the updated node description with children
+ describe_params: 'DescribeNodeParameters' = {'nodeId': node_id, 'depth': 1}
+ describe_result = await self._client.send.DOM.describeNode(describe_params, session_id=self._session_id)
+
+ select_node = describe_result['node']
+
+ # Find and select matching options
+ for child in select_node.get('children', []):
+ if child.get('nodeName', '').lower() == 'option':
+ # Get option attributes
+ attrs = child.get('attributes', [])
+ option_attrs = {}
+ for i in range(0, len(attrs), 2):
+ if i + 1 < len(attrs):
+ option_attrs[attrs[i]] = attrs[i + 1]
+
+ option_value = option_attrs.get('value', '')
+ option_text = child.get('nodeValue', '')
+
+ # Check if this option should be selected
+ should_select = option_value in values or option_text in values
+
+ if should_select:
+ # Click the option to select it
+ option_node_id = child.get('nodeId')
+ if option_node_id:
+ # Get backend node ID for the option
+ option_describe_params: 'DescribeNodeParameters' = {'nodeId': option_node_id}
+ option_backend_result = await self._client.send.DOM.describeNode(
+ option_describe_params, session_id=self._session_id
+ )
+ option_backend_id = option_backend_result['node']['backendNodeId']
+
+ # Create an Element for the option and click it
+ option_element = Element(self._client, option_backend_id, self._session_id)
+ await option_element.click()
+
+ async def dragTo(
+ self,
+ target: Union['Element', Position],
+ source_position: Position | None = None,
+ target_position: Position | None = None,
+ ) -> None:
+ """Drag this element to another element or position."""
+ # Get source coordinates
+ if source_position:
+ source_x = source_position['x']
+ source_y = source_position['y']
+ else:
+ source_box = await self.getBoundingBox()
+ if not source_box:
+ raise RuntimeError('Source element is not visible')
+ source_x = source_box['x'] + source_box['width'] / 2
+ source_y = source_box['y'] + source_box['height'] / 2
+
+ # Get target coordinates
+ if isinstance(target, dict) and 'x' in target and 'y' in target:
+ target_x = target['x']
+ target_y = target['y']
+ else:
+ if target_position:
+ target_box = await target.getBoundingBox()
+ if not target_box:
+ raise RuntimeError('Target element is not visible')
+ target_x = target_box['x'] + target_position['x']
+ target_y = target_box['y'] + target_position['y']
+ else:
+ target_box = await target.getBoundingBox()
+ if not target_box:
+ raise RuntimeError('Target element is not visible')
+ target_x = target_box['x'] + target_box['width'] / 2
+ target_y = target_box['y'] + target_box['height'] / 2
+
+ # Perform drag operation
+ await self._client.send.Input.dispatchMouseEvent(
+ {'type': 'mousePressed', 'x': source_x, 'y': source_y, 'button': 'left'},
+ session_id=self._session_id,
+ )
+
+ await self._client.send.Input.dispatchMouseEvent(
+ {'type': 'mouseMoved', 'x': target_x, 'y': target_y},
+ session_id=self._session_id,
+ )
+
+ await self._client.send.Input.dispatchMouseEvent(
+ {'type': 'mouseReleased', 'x': target_x, 'y': target_y, 'button': 'left'},
+ session_id=self._session_id,
+ )
+
+ # Element properties and queries
+ async def getAttribute(self, name: str) -> str | None:
+ """Get an attribute value."""
+ node_id = await self._get_node_id()
+ params: 'GetAttributesParameters' = {'nodeId': node_id}
+ result = await self._client.send.DOM.getAttributes(params, session_id=self._session_id)
+
+ attributes = result['attributes']
+ for i in range(0, len(attributes), 2):
+ if attributes[i] == name:
+ return attributes[i + 1]
+ return None
+
+ async def getBoundingBox(self) -> BoundingBox | None:
+ """Get the bounding box of the element."""
+ try:
+ node_id = await self._get_node_id()
+ params: 'GetBoxModelParameters' = {'nodeId': node_id}
+ result = await self._client.send.DOM.getBoxModel(params, session_id=self._session_id)
+
+ if 'model' not in result:
+ return None
+
+ # Get content box (first 8 values are content quad: x1,y1,x2,y2,x3,y3,x4,y4)
+ content = result['model']['content']
+ if len(content) < 8:
+ return None
+
+ # Calculate bounding box from quad
+ x_coords = [content[i] for i in range(0, 8, 2)]
+ y_coords = [content[i] for i in range(1, 8, 2)]
+
+ x = min(x_coords)
+ y = min(y_coords)
+ width = max(x_coords) - x
+ height = max(y_coords) - y
+
+ return BoundingBox(x=x, y=y, width=width, height=height)
+
+ except Exception:
+ return None
+
+ async def screenshot(self, format: str = 'jpeg', quality: int | None = None) -> str:
+ """Take a screenshot of this element and return base64 encoded image.
+
+ Args:
+ format: Image format ('jpeg', 'png', 'webp')
+ quality: Quality 0-100 for JPEG format
+
+ Returns:
+ Base64-encoded image data
+ """
+ # Get element's bounding box
+ box = await self.getBoundingBox()
+ if not box:
+ raise RuntimeError('Element is not visible or has no bounding box')
+
+ # Create viewport clip for the element
+ viewport: 'Viewport' = {'x': box['x'], 'y': box['y'], 'width': box['width'], 'height': box['height'], 'scale': 1.0}
+
+ # Prepare screenshot parameters
+ params: 'CaptureScreenshotParameters' = {'format': format, 'clip': viewport}
+
+ if quality is not None and format.lower() == 'jpeg':
+ params['quality'] = quality
+
+ # Take screenshot
+ result = await self._client.send.Page.captureScreenshot(params, session_id=self._session_id)
+
+ return result['data']
+
+ async def getBasicInfo(self) -> ElementInfo:
+ """Get basic information about the element including coordinates and properties."""
+ try:
+ # Get basic node information
+ node_id = await self._get_node_id()
+ describe_result = await self._client.send.DOM.describeNode({'nodeId': node_id}, session_id=self._session_id)
+
+ node_info = describe_result['node']
+
+ # Get bounding box
+ bounding_box = await self.getBoundingBox()
+
+ # Get attributes as a proper dict
+ attributes_list = node_info.get('attributes', [])
+ attributes_dict: dict[str, str] = {}
+ for i in range(0, len(attributes_list), 2):
+ if i + 1 < len(attributes_list):
+ attributes_dict[attributes_list[i]] = attributes_list[i + 1]
+
+ return ElementInfo(
+ backendNodeId=self._backend_node_id,
+ nodeId=node_id,
+ nodeName=node_info.get('nodeName', ''),
+ nodeType=node_info.get('nodeType', 0),
+ nodeValue=node_info.get('nodeValue'),
+ attributes=attributes_dict,
+ boundingBox=bounding_box,
+ error=None,
+ )
+ except Exception as e:
+ return ElementInfo(
+ backendNodeId=self._backend_node_id,
+ nodeId=None,
+ nodeName='',
+ nodeType=0,
+ nodeValue=None,
+ attributes={},
+ boundingBox=None,
+ error=str(e),
+ )
diff --git a/browser_use/actor/mouse.py b/browser_use/actor/mouse.py
new file mode 100644
index 000000000..10ec3f3d1
--- /dev/null
+++ b/browser_use/actor/mouse.py
@@ -0,0 +1,139 @@
+"""Mouse class for mouse operations."""
+
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+ from cdp_use.cdp.input.commands import DispatchMouseEventParameters, SynthesizeScrollGestureParameters
+ from cdp_use.cdp.input.types import MouseButton
+ from cdp_use.client import CDPClient
+
+
+class Mouse:
+ """Mouse operations for a target."""
+
+ def __init__(self, client: 'CDPClient', session_id: str | None = None, target_id: str | None = None):
+ self._client = client
+ self._session_id = session_id
+ self._target_id = target_id
+
+ async def click(self, x: int, y: int, button: 'MouseButton' = 'left', click_count: int = 1) -> None:
+ """Click at the specified coordinates."""
+ # Mouse press
+ press_params: 'DispatchMouseEventParameters' = {
+ 'type': 'mousePressed',
+ 'x': x,
+ 'y': y,
+ 'button': button,
+ 'clickCount': click_count,
+ }
+ await self._client.send.Input.dispatchMouseEvent(
+ press_params,
+ session_id=self._session_id,
+ )
+
+ # Mouse release
+ release_params: 'DispatchMouseEventParameters' = {
+ 'type': 'mouseReleased',
+ 'x': x,
+ 'y': y,
+ 'button': button,
+ 'clickCount': click_count,
+ }
+ await self._client.send.Input.dispatchMouseEvent(
+ release_params,
+ session_id=self._session_id,
+ )
+
+ async def down(self, button: 'MouseButton' = 'left', click_count: int = 1) -> None:
+ """Press mouse button down."""
+ params: 'DispatchMouseEventParameters' = {
+ 'type': 'mousePressed',
+ 'x': 0, # Will use last mouse position
+ 'y': 0,
+ 'button': button,
+ 'clickCount': click_count,
+ }
+ await self._client.send.Input.dispatchMouseEvent(
+ params,
+ session_id=self._session_id,
+ )
+
+ async def up(self, button: 'MouseButton' = 'left', click_count: int = 1) -> None:
+ """Release mouse button."""
+ params: 'DispatchMouseEventParameters' = {
+ 'type': 'mouseReleased',
+ 'x': 0, # Will use last mouse position
+ 'y': 0,
+ 'button': button,
+ 'clickCount': click_count,
+ }
+ await self._client.send.Input.dispatchMouseEvent(
+ params,
+ session_id=self._session_id,
+ )
+
+ async def move(self, x: int, y: int, steps: int = 1) -> None:
+ """Move mouse to the specified coordinates."""
+ # TODO: Implement smooth movement with multiple steps if needed
+ _ = steps # Acknowledge parameter for future use
+
+ params: 'DispatchMouseEventParameters' = {'type': 'mouseMoved', 'x': x, 'y': y}
+ await self._client.send.Input.dispatchMouseEvent(params, session_id=self._session_id)
+
+ async def scroll(self, x: int = 0, y: int = 0, delta_x: int | None = None, delta_y: int | None = None) -> None:
+ """Scroll the page using robust CDP methods."""
+ if not self._session_id:
+ raise RuntimeError('Session ID is required for scroll operations')
+
+ # Activate the target first (critical for CDP calls to work)
+ if self._target_id:
+ try:
+ await self._client.send.Target.activateTarget(params={'targetId': self._target_id})
+ except Exception:
+ pass
+
+ # Method 1: Try mouse wheel event (most reliable)
+ try:
+ # Get viewport dimensions
+ layout_metrics = await self._client.send.Page.getLayoutMetrics(session_id=self._session_id)
+ viewport_width = layout_metrics['layoutViewport']['clientWidth']
+ viewport_height = layout_metrics['layoutViewport']['clientHeight']
+
+ # Use provided coordinates or center of viewport
+ scroll_x = x if x > 0 else viewport_width / 2
+ scroll_y = y if y > 0 else viewport_height / 2
+
+ # Calculate scroll deltas (positive = down/right)
+ scroll_delta_x = delta_x or 0
+ scroll_delta_y = delta_y or 0
+
+ # Dispatch mouse wheel event
+ await self._client.send.Input.dispatchMouseEvent(
+ params={
+ 'type': 'mouseWheel',
+ 'x': scroll_x,
+ 'y': scroll_y,
+ 'deltaX': scroll_delta_x,
+ 'deltaY': scroll_delta_y,
+ },
+ session_id=self._session_id,
+ )
+ return
+
+ except Exception:
+ pass
+
+ # Method 2: Fallback to synthesizeScrollGesture
+ try:
+ params: 'SynthesizeScrollGestureParameters' = {'x': x, 'y': y, 'xDistance': delta_x or 0, 'yDistance': delta_y or 0}
+ await self._client.send.Input.synthesizeScrollGesture(
+ params,
+ session_id=self._session_id,
+ )
+ except Exception:
+ # Method 3: JavaScript fallback
+ scroll_js = f'window.scrollBy({delta_x or 0}, {delta_y or 0})'
+ await self._client.send.Runtime.evaluate(
+ params={'expression': scroll_js, 'returnByValue': True},
+ session_id=self._session_id,
+ )
diff --git a/browser_use/actor/playground/playground.py b/browser_use/actor/playground/playground.py
new file mode 100755
index 000000000..2f126f3e0
--- /dev/null
+++ b/browser_use/actor/playground/playground.py
@@ -0,0 +1,243 @@
+#!/usr/bin/env python3
+"""
+Playground script to test the browser-use actor API.
+
+This script demonstrates:
+- Starting a browser session
+- Using the actor API to navigate and interact
+- Finding elements, clicking, scrolling, JavaScript evaluation
+- Testing most of the available methods
+"""
+
+import asyncio
+import json
+import logging
+
+from browser_use.actor import Browser
+from browser_use.browser.session import BrowserSession
+
+# Configure logging to see what's happening
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+
+async def main():
+ """Main playground function."""
+ logger.info('🚀 Starting browser actor playground')
+
+ # Create browser session
+ browser_session = BrowserSession()
+
+ try:
+ # Start the browser
+ await browser_session.start()
+ logger.info('✅ Browser session started')
+
+ # Get the CDP client from the session
+ cdp_client = browser_session.cdp_client
+
+ # Create the actor browser interface
+ browser = Browser(cdp_client)
+
+ # Navigate to Wikipedia
+ logger.info('📖 Navigating to Wikipedia...')
+ target = await browser.newTarget('https://en.wikipedia.org')
+
+ # Get basic page info
+ url = await target.getUrl()
+ title = await target.getTitle()
+ logger.info(f'📄 Page loaded: {title} ({url})')
+
+ # Take a screenshot
+ logger.info('📸 Taking initial screenshot...')
+ screenshot_b64 = await target.screenshot()
+ logger.info(f'📸 Screenshot captured: {len(screenshot_b64)} bytes')
+
+ # Set viewport size
+ logger.info('🖥️ Setting viewport to 1920x1080...')
+ await target.setViewportSize(1920, 1080)
+
+ # Execute some JavaScript to count links
+ logger.info('🔍 Counting article links using JavaScript...')
+ js_code = """() => {
+ // Find all article links on the page
+ const links = Array.from(document.querySelectorAll('a[href*="/wiki/"]:not([href*=":"])'))
+ .filter(link => !link.href.includes('Main_Page') && !link.href.includes('Special:'));
+
+ return {
+ total: links.length,
+ sample: links.slice(0, 3).map(link => ({
+ href: link.href,
+ text: link.textContent.trim()
+ }))
+ };
+ }"""
+
+ link_info = json.loads(await target.evaluate(js_code))
+ logger.info(f'🔗 Found {link_info["total"]} article links')
+ # Try to find and interact with links using CSS selector
+ try:
+ # Find article links on the page
+ links = await target.getElementsByCSSSelector('a[href*="/wiki/"]:not([href*=":"])')
+
+ if links:
+ logger.info(f'📋 Found {len(links)} wiki links via CSS selector')
+
+ # Pick the first link
+ link_element = links[0]
+
+ # Get link info using available methods
+ basic_info = await link_element.getBasicInfo()
+ link_href = await link_element.getAttribute('href')
+
+ logger.info(f'🎯 Selected element: <{basic_info["nodeName"]}>')
+ logger.info(f'🔗 Link href: {link_href}')
+
+ if basic_info['boundingBox']:
+ bbox = basic_info['boundingBox']
+ logger.info(f'📏 Position: ({bbox["x"]}, {bbox["y"]}) Size: {bbox["width"]}x{bbox["height"]}')
+
+ # Test element interactions with robust implementations
+ logger.info('👆 Hovering over the element...')
+ await link_element.hover()
+ await asyncio.sleep(1)
+
+ logger.info('🔍 Focusing the element...')
+ await link_element.focus()
+ await asyncio.sleep(0.5)
+
+ # Click the link using robust click method
+ logger.info('🖱️ Clicking the link with robust fallbacks...')
+ await link_element.click()
+
+ # Wait for navigation
+ await asyncio.sleep(3)
+
+ # Get new page info
+ new_url = await target.getUrl()
+ new_title = await target.getTitle()
+ logger.info(f'📄 Navigated to: {new_title}')
+ logger.info(f'🌐 New URL: {new_url}')
+ else:
+ logger.warning('❌ No links found to interact with')
+
+ except Exception as e:
+ logger.warning(f'⚠️ Link interaction failed: {e}')
+
+ # Scroll down the page
+ logger.info('📜 Scrolling down the page...')
+ mouse = await target.mouse
+ await mouse.scroll(x=0, y=100, delta_y=500)
+ await asyncio.sleep(1)
+
+ # Test mouse operations
+ logger.info('🖱️ Testing mouse operations...')
+ await mouse.move(x=100, y=200)
+ await mouse.click(x=150, y=250)
+
+ # Execute more JavaScript examples
+ logger.info('🧪 Testing JavaScript evaluation...')
+
+ # Simple expressions
+ page_height = await target.evaluate('() => document.body.scrollHeight')
+ current_scroll = await target.evaluate('() => window.pageYOffset')
+ logger.info(f'📏 Page height: {page_height}px, current scroll: {current_scroll}px')
+
+ # JavaScript with arguments
+ result = await target.evaluate('(x) => x * 2', 21)
+ logger.info(f'🧮 JavaScript with args: 21 * 2 = {result}')
+
+ # More complex JavaScript
+ page_stats = json.loads(
+ await target.evaluate("""() => {
+ return {
+ url: window.location.href,
+ title: document.title,
+ links: document.querySelectorAll('a').length,
+ images: document.querySelectorAll('img').length,
+ scrollTop: window.pageYOffset,
+ viewportHeight: window.innerHeight
+ };
+ }""")
+ )
+ logger.info(f'📊 Page stats: {page_stats}')
+
+ # Get page title using different methods
+ title_via_js = await target.evaluate('() => document.title')
+ title_via_api = await target.getTitle()
+ logger.info(f'📝 Title via JS: "{title_via_js}"')
+ logger.info(f'📝 Title via API: "{title_via_api}"')
+
+ # Take a final screenshot
+ logger.info('📸 Taking final screenshot...')
+ final_screenshot = await target.screenshot()
+ logger.info(f'📸 Final screenshot: {len(final_screenshot)} bytes')
+
+ # Test browser navigation with error handling
+ logger.info('⬅️ Testing browser back navigation...')
+ try:
+ await target.goBack()
+ await asyncio.sleep(2)
+
+ back_url = await target.getUrl()
+ back_title = await target.getTitle()
+ logger.info(f'📄 After going back: {back_title}')
+ logger.info(f'🌐 Back URL: {back_url}')
+ except RuntimeError as e:
+ logger.info(f'ℹ️ Navigation back failed as expected: {e}')
+
+ # Test creating new target
+ logger.info('🆕 Creating new blank target...')
+ new_target = await browser.newTarget()
+ new_target_url = await new_target.getUrl()
+ logger.info(f'🆕 New target created with URL: {new_target_url}')
+
+ # Get all targets
+ all_targets = await browser.getTargets()
+ logger.info(f'📑 Total targets: {len(all_targets)}')
+
+ # Test form interaction if we can find a form
+ try:
+ # Look for search input on the page
+ search_inputs = await target.getElementsByCSSSelector('input[type="search"], input[name*="search"]')
+
+ if search_inputs:
+ search_input = search_inputs[0]
+ logger.info('🔍 Found search input, testing form interaction...')
+
+ await search_input.focus()
+ await search_input.fill('test search query')
+ await target.press('Enter')
+
+ logger.info('✅ Form interaction test completed')
+ else:
+ logger.info('ℹ️ No search inputs found for form testing')
+
+ except Exception as e:
+ logger.info(f'ℹ️ Form interaction test skipped: {e}')
+
+ # wait 2 seconds before closing the new target
+ logger.info('🕒 Waiting 2 seconds before closing the new target...')
+ await asyncio.sleep(2)
+ logger.info('🗑️ Closing new target...')
+ await browser.closeTarget(new_target)
+
+ logger.info('✅ Playground completed successfully!')
+
+ input('Press Enter to continue...')
+
+ except Exception as e:
+ logger.error(f'❌ Error in playground: {e}', exc_info=True)
+
+ finally:
+ # Clean up
+ logger.info('🧹 Cleaning up...')
+ try:
+ await browser_session.stop()
+ logger.info('✅ Browser session stopped')
+ except Exception as e:
+ logger.error(f'❌ Error stopping browser: {e}')
+
+
+if __name__ == '__main__':
+ asyncio.run(main())
diff --git a/browser_use/actor/target.py b/browser_use/actor/target.py
new file mode 100644
index 000000000..d481556be
--- /dev/null
+++ b/browser_use/actor/target.py
@@ -0,0 +1,329 @@
+"""Target class for target-level operations."""
+
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+ from cdp_use.cdp.dom.commands import (
+ DescribeNodeParameters,
+ QuerySelectorAllParameters,
+ )
+ from cdp_use.cdp.emulation.commands import SetDeviceMetricsOverrideParameters
+ from cdp_use.cdp.input.commands import (
+ DispatchKeyEventParameters,
+ )
+ from cdp_use.cdp.page.commands import CaptureScreenshotParameters, NavigateParameters, NavigateToHistoryEntryParameters
+ from cdp_use.cdp.runtime.commands import EvaluateParameters
+ from cdp_use.cdp.target.commands import (
+ AttachToTargetParameters,
+ GetTargetInfoParameters,
+ )
+ from cdp_use.cdp.target.types import TargetInfo
+ from cdp_use.client import CDPClient
+
+ from .element import Element
+ from .mouse import Mouse
+
+
+class Target:
+ """Target operations (tab or iframe)."""
+
+ def __init__(self, client: 'CDPClient', target_id: str, session_id: str | None = None):
+ self._client = client
+ self._target_id = target_id
+ self._session_id: str | None = session_id
+ self._mouse: 'Mouse | None' = None
+
+ async def _ensure_session(self) -> str:
+ """Ensure we have a session ID for this target."""
+ if not self._session_id:
+ params: 'AttachToTargetParameters' = {'targetId': self._target_id, 'flatten': True}
+ result = await self._client.send.Target.attachToTarget(params)
+ self._session_id = result['sessionId']
+
+ # Enable necessary domains
+ import asyncio
+
+ await asyncio.gather(
+ self._client.send.Page.enable(session_id=self._session_id),
+ self._client.send.DOM.enable(session_id=self._session_id),
+ self._client.send.Runtime.enable(session_id=self._session_id),
+ self._client.send.Network.enable(session_id=self._session_id),
+ )
+
+ return self._session_id
+
+ @property
+ async def mouse(self) -> 'Mouse':
+ """Get the mouse interface for this target."""
+ if not self._mouse:
+ session_id = await self._ensure_session()
+ from .mouse import Mouse
+
+ self._mouse = Mouse(self._client, session_id, self._target_id)
+ return self._mouse
+
+ async def reload(self) -> None:
+ """Reload the target."""
+ session_id = await self._ensure_session()
+ await self._client.send.Page.reload(session_id=session_id)
+
+ async def getElement(self, backend_node_id: int) -> 'Element':
+ """Get an element by its backend node ID."""
+ session_id = await self._ensure_session()
+
+ from .element import Element
+
+ return Element(self._client, backend_node_id, session_id)
+
+ async def evaluate(self, page_function: str, *args) -> str:
+ """Execute JavaScript in the target.
+
+ Args:
+ page_function: JavaScript code that MUST start with (...args) => format
+ *args: Arguments to pass to the function
+
+ Returns:
+ String representation of the JavaScript execution result.
+ Objects and arrays are JSON-stringified.
+ """
+ session_id = await self._ensure_session()
+
+ # Clean and fix common JavaScript string parsing issues
+ page_function = self._fix_javascript_string(page_function)
+
+ # Enforce arrow function format
+ if not (page_function.startswith('(') and '=>' in page_function):
+ raise ValueError(f'JavaScript code must start with (...args) => format. Got: {page_function[:50]}...')
+
+ # Build the expression - call the arrow function with provided args
+ if args:
+ # Convert args to JSON representation for safe passing
+ import json
+
+ arg_strs = [json.dumps(arg) for arg in args]
+ expression = f'({page_function})({", ".join(arg_strs)})'
+ else:
+ expression = f'({page_function})()'
+
+ # Debug: print the actual expression being evaluated
+ print(f'DEBUG: Evaluating JavaScript: {repr(expression)}')
+
+ params: 'EvaluateParameters' = {'expression': expression, 'returnByValue': True, 'awaitPromise': True}
+ result = await self._client.send.Runtime.evaluate(
+ params,
+ session_id=session_id,
+ )
+
+ if 'exceptionDetails' in result:
+ raise RuntimeError(f'JavaScript evaluation failed: {result["exceptionDetails"]}')
+
+ value = result.get('result', {}).get('value')
+
+ # Always return string representation
+ if value is None:
+ return ''
+ elif isinstance(value, str):
+ return value
+ else:
+ # Convert objects, numbers, booleans to string
+ import json
+
+ try:
+ return json.dumps(value) if isinstance(value, (dict, list)) else str(value)
+ except (TypeError, ValueError):
+ return str(value)
+
+ def _fix_javascript_string(self, js_code: str) -> str:
+ """Fix common JavaScript string parsing issues when written as Python string."""
+
+ # Just do minimal, safe cleaning
+ js_code = js_code.strip()
+
+ # Only fix the most common and safe issues:
+
+ # 1. Remove obvious Python string wrapper quotes if they exist
+ if (js_code.startswith('"') and js_code.endswith('"')) or (js_code.startswith("'") and js_code.endswith("'")):
+ # Check if it's a wrapped string (not part of JS syntax)
+ inner = js_code[1:-1]
+ if inner.count('"') + inner.count("'") == 0 or '() =>' in inner:
+ js_code = inner
+
+ # 2. Only fix clearly escaped quotes that shouldn't be
+ # But be very conservative - only if we're sure it's a Python string artifact
+ if '\\"' in js_code and js_code.count('\\"') > js_code.count('"'):
+ js_code = js_code.replace('\\"', '"')
+ if "\\'" in js_code and js_code.count("\\'") > js_code.count("'"):
+ js_code = js_code.replace("\\'", "'")
+
+ # 3. Basic whitespace normalization only
+ js_code = js_code.strip()
+
+ # Final validation - ensure it's not empty
+ if not js_code:
+ raise ValueError('JavaScript code is empty after cleaning')
+
+ return js_code
+
+ async def screenshot(self, format: str = 'jpeg', quality: int | None = None) -> str:
+ """Take a screenshot and return base64 encoded image.
+
+ Args:
+ format: Image format ('jpeg', 'png', 'webp')
+ quality: Quality 0-100 for JPEG format
+
+ Returns:
+ Base64-encoded image data
+ """
+ session_id = await self._ensure_session()
+
+ params: 'CaptureScreenshotParameters' = {'format': format}
+
+ if quality is not None and format.lower() == 'jpeg':
+ params['quality'] = quality
+
+ result = await self._client.send.Page.captureScreenshot(params, session_id=session_id)
+
+ return result['data']
+
+ async def press(self, key: str) -> None:
+ """Press a key on the page (sends keyboard input to the focused element or page)."""
+ session_id = await self._ensure_session()
+
+ # Handle key combinations like "Control+A"
+ if '+' in key:
+ parts = key.split('+')
+ modifiers = parts[:-1]
+ main_key = parts[-1]
+
+ # Press modifier keys
+ for mod in modifiers:
+ params: 'DispatchKeyEventParameters' = {'type': 'keyDown', 'key': mod}
+ await self._client.send.Input.dispatchKeyEvent(params, session_id=session_id)
+
+ # Press main key
+ main_down_params: 'DispatchKeyEventParameters' = {'type': 'keyDown', 'key': main_key}
+ await self._client.send.Input.dispatchKeyEvent(main_down_params, session_id=session_id)
+
+ main_up_params: 'DispatchKeyEventParameters' = {'type': 'keyUp', 'key': main_key}
+ await self._client.send.Input.dispatchKeyEvent(main_up_params, session_id=session_id)
+
+ # Release modifier keys
+ for mod in reversed(modifiers):
+ release_params: 'DispatchKeyEventParameters' = {'type': 'keyUp', 'key': mod}
+ await self._client.send.Input.dispatchKeyEvent(release_params, session_id=session_id)
+ else:
+ # Simple key press
+ key_down_params: 'DispatchKeyEventParameters' = {'type': 'keyDown', 'key': key}
+ await self._client.send.Input.dispatchKeyEvent(key_down_params, session_id=session_id)
+
+ key_up_params: 'DispatchKeyEventParameters' = {'type': 'keyUp', 'key': key}
+ await self._client.send.Input.dispatchKeyEvent(key_up_params, session_id=session_id)
+
+ async def setViewportSize(self, width: int, height: int) -> None:
+ """Set the viewport size."""
+ session_id = await self._ensure_session()
+
+ params: 'SetDeviceMetricsOverrideParameters' = {
+ 'width': width,
+ 'height': height,
+ 'deviceScaleFactor': 1.0,
+ 'mobile': False,
+ }
+ await self._client.send.Emulation.setDeviceMetricsOverride(
+ params,
+ session_id=session_id,
+ )
+
+ # Target properties (from CDP getTargetInfo)
+ async def getTargetInfo(self) -> 'TargetInfo':
+ """Get target information."""
+ params: 'GetTargetInfoParameters' = {'targetId': self._target_id}
+ result = await self._client.send.Target.getTargetInfo(params)
+ return result['targetInfo']
+
+ async def getUrl(self) -> str:
+ """Get the current URL."""
+ info = await self.getTargetInfo()
+ return info.get('url', '')
+
+ async def getTitle(self) -> str:
+ """Get the current title."""
+ info = await self.getTargetInfo()
+ return info.get('title', '')
+
+ async def goto(self, url: str) -> None:
+ """Navigate this target to a URL."""
+ session_id = await self._ensure_session()
+
+ params: 'NavigateParameters' = {'url': url}
+ await self._client.send.Page.navigate(params, session_id=session_id)
+
+ async def goBack(self) -> None:
+ """Navigate back in history."""
+ session_id = await self._ensure_session()
+
+ try:
+ # Get navigation history
+ history = await self._client.send.Page.getNavigationHistory(session_id=session_id)
+ current_index = history['currentIndex']
+ entries = history['entries']
+
+ # Check if we can go back
+ if current_index <= 0:
+ raise RuntimeError('Cannot go back - no previous entry in history')
+
+ # Navigate to the previous entry
+ previous_entry_id = entries[current_index - 1]['id']
+ params: 'NavigateToHistoryEntryParameters' = {'entryId': previous_entry_id}
+ await self._client.send.Page.navigateToHistoryEntry(params, session_id=session_id)
+
+ except Exception as e:
+ raise RuntimeError(f'Failed to navigate back: {e}')
+
+ async def goForward(self) -> None:
+ """Navigate forward in history."""
+ session_id = await self._ensure_session()
+
+ try:
+ # Get navigation history
+ history = await self._client.send.Page.getNavigationHistory(session_id=session_id)
+ current_index = history['currentIndex']
+ entries = history['entries']
+
+ # Check if we can go forward
+ if current_index >= len(entries) - 1:
+ raise RuntimeError('Cannot go forward - no next entry in history')
+
+ # Navigate to the next entry
+ next_entry_id = entries[current_index + 1]['id']
+ params: 'NavigateToHistoryEntryParameters' = {'entryId': next_entry_id}
+ await self._client.send.Page.navigateToHistoryEntry(params, session_id=session_id)
+
+ except Exception as e:
+ raise RuntimeError(f'Failed to navigate forward: {e}')
+
+ # Element finding methods (these would need to be implemented based on DOM queries)
+ async def getElementsByCSSSelector(self, selector: str) -> list['Element']:
+ """Get elements by CSS selector."""
+ session_id = await self._ensure_session()
+
+ # Get document first
+ doc_result = await self._client.send.DOM.getDocument(session_id=session_id)
+ document_node_id = doc_result['root']['nodeId']
+
+ # Query selector all
+ query_params: 'QuerySelectorAllParameters' = {'nodeId': document_node_id, 'selector': selector}
+ result = await self._client.send.DOM.querySelectorAll(query_params, session_id=session_id)
+
+ elements = []
+ from .element import Element
+
+ # Convert node IDs to backend node IDs
+ for node_id in result['nodeIds']:
+ # Get backend node ID
+ describe_params: 'DescribeNodeParameters' = {'nodeId': node_id}
+ node_result = await self._client.send.DOM.describeNode(describe_params, session_id=session_id)
+ backend_node_id = node_result['node']['backendNodeId']
+ elements.append(Element(self._client, backend_node_id, session_id))
+
+ return elements
diff --git a/browser_use/dom/playground/tree.py b/browser_use/dom/playground/tree.py
deleted file mode 100644
index 2039f769a..000000000
--- a/browser_use/dom/playground/tree.py
+++ /dev/null
@@ -1,126 +0,0 @@
-import asyncio
-import json
-import time
-
-import aiofiles
-
-from browser_use.browser import BrowserProfile, BrowserSession
-from browser_use.browser.types import ViewportSize
-from browser_use.dom.service import DomService
-from browser_use.dom.views import EnhancedDOMTreeNode
-
-
-async def main():
- # async with async_playwright() as p:
- # playwright_browser = await p.chromium.launch(args=['--remote-debugging-port=9222'], headless=False)
- browser = BrowserSession(
- browser_profile=BrowserProfile(
- # executable_path='/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
- window_size=ViewportSize(width=1100, height=1000),
- disable_security=True,
- wait_for_network_idle_page_load_time=1,
- headless=False,
- args=['--incognito'],
- ),
- )
- # async with httpx.AsyncClient() as client:
- # version_info = await client.get('http://localhost:9222/json/version')
- # browser.cdp_url = version_info.json()['webSocketDebuggerUrl']
-
- # if not browser.cdp_url:
- # raise ValueError('CDP URL is not set') # can't happen in this case actually
-
- # await browser.create_new_tab('https://en.wikipedia.org/wiki/Apple_Inc.')
- # await browser.create_new_tab('https://semantic-ui.com/modules/dropdown.html#/definition')
- # await browser.navigate('https://v0-website-with-clickable-elements.vercel.app/iframe-buttons')
- from browser_use.browser.events import NavigateToUrlEvent
-
- nav_event = browser.event_bus.dispatch(
- NavigateToUrlEvent(url='https://v0-website-with-clickable-elements.vercel.app/nested-iframe')
- )
- await nav_event
-
- # Wait a moment for page to fully load
- await asyncio.sleep(2)
-
- while True:
- async with DomService(browser) as dom_service:
- start = time.time()
- # Get current target ID from browser session
- if browser.agent_focus and browser.agent_focus.target_id:
- target_id = browser.agent_focus.target_id
- else:
- # Get first available target
- targets = await browser._cdp_get_all_pages()
- if not targets:
- raise ValueError('No targets available')
- target_id = targets[0]['targetId']
-
- result = await dom_service.get_dom_tree(target_id)
- if isinstance(result, tuple):
- dom_tree = result[0]
- dom_timing = result[1] if len(result) > 1 else {}
- else:
- dom_tree = result
- dom_timing = {}
-
- end = time.time()
- print(f'Time taken: {end - start} seconds')
-
- async with aiofiles.open('tmp/enhanced_dom_tree.json', 'w') as f:
- await f.write(json.dumps(dom_tree.__json__(), indent=1))
-
- print('Saved enhanced dom tree to tmp/enhanced_dom_tree.json')
-
- # Print some sample information about visible/clickable elements
- visible_clickable_count = 0
- total_with_snapshot = 0
-
- def count_elements(node: EnhancedDOMTreeNode):
- nonlocal visible_clickable_count, total_with_snapshot
- if node.snapshot_node:
- total_with_snapshot += 1
- if node.is_visible and node.snapshot_node.is_clickable:
- visible_clickable_count += 1
- # print(f'Visible clickable element: {node.node_name} (cursor: {node.snapshot_node.cursor_style})')
-
- if node.children_nodes:
- for child in node.children_nodes:
- count_elements(child)
-
- count_elements(dom_tree)
- print(
- f'Found {visible_clickable_count} visible clickable elements out of {total_with_snapshot} elements with snapshot data'
- )
-
- serialized_dom_state, timing_info = await dom_service.get_serialized_dom_tree()
-
- async with aiofiles.open('tmp/serialized_dom_tree.txt', 'w') as f:
- await f.write(serialized_dom_state.llm_representation())
-
- # print(serialized)
- print('Saved serialized dom tree to tmp/serialized_dom_tree.txt')
-
- start = time.time()
- snapshot, dom_tree, ax_tree, _ = await dom_service._get_all_trees()
- end = time.time()
- print(f'Time taken: {end - start} seconds')
-
- async with aiofiles.open('tmp/snapshot.json', 'w') as f:
- await f.write(json.dumps(snapshot, indent=1))
-
- async with aiofiles.open('tmp/dom_tree.json', 'w') as f:
- await f.write(json.dumps(dom_tree, indent=1))
-
- async with aiofiles.open('tmp/ax_tree.json', 'w') as f:
- await f.write(json.dumps(ax_tree, indent=1))
-
- print('saved dom tree to tmp/dom_tree.json')
- print('saved snapshot to tmp/snapshot.json')
- print('saved ax tree to tmp/ax_tree.json')
-
- input('Done. Press Enter to continue...')
-
-
-if __name__ == '__main__':
- asyncio.run(main())