Skip to content

Open-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent API.

sync browser-use skill from harness main

Laith Weinberger3cd3816c575e
diff --git a/browser_use/skills/browser-use/SKILL.md b/browser_use/skills/browser-use/SKILL.md
index 26c4cc604..250f69151 100644
--- a/browser_use/skills/browser-use/SKILL.md
+++ b/browser_use/skills/browser-use/SKILL.md
@@ -84,8 +84,9 @@ Cloud profile cookie sync reference: https://github.com/browser-use/browser-harn
## Page Workflow
-- Screenshots first: use `capture_screenshot()` to understand visible state.
-- Clicking: screenshot -> read pixel -> `click_at_xy(x, y)` -> screenshot again.
+- Prefer to find elements with the accessibility tree, not screenshots: `cdp("Accessibility.getFullAXTree")["nodes"]` has every element's role, name, and `backendDOMNodeId` — filter in Python before printing (it is thousands of nodes). Coordinates: `q = cdp("DOM.getBoxModel", backendNodeId=n)["model"]["content"]; x, y = sum(q[0::2])/4, sum(q[1::2])/4` (viewport px, ready for `click_at_xy`; negative/oversized means scroll first).
+- Clicking: AX node -> box center -> `click_at_xy(x, y)` -> verify with a targeted `js(...)`/`page_info()` check.
+- Fall back to raw HTML via `js(...)` only when the AX tree lacks the element (canvas, exotic widgets); screenshot when layout or imagery matters.
- After navigation, call `wait_for_load()`.
- If the current tab is stale or internal, call `ensure_real_tab()`.
- Use `js(...)` for DOM inspection or extraction when coordinates are the wrong tool.
diff --git a/skills/browser-use/SKILL.md b/skills/browser-use/SKILL.md
index 26c4cc604..250f69151 100644
--- a/skills/browser-use/SKILL.md
+++ b/skills/browser-use/SKILL.md
@@ -84,8 +84,9 @@ Cloud profile cookie sync reference: https://github.com/browser-use/browser-harn
## Page Workflow
-- Screenshots first: use `capture_screenshot()` to understand visible state.
-- Clicking: screenshot -> read pixel -> `click_at_xy(x, y)` -> screenshot again.
+- Prefer to find elements with the accessibility tree, not screenshots: `cdp("Accessibility.getFullAXTree")["nodes"]` has every element's role, name, and `backendDOMNodeId` — filter in Python before printing (it is thousands of nodes). Coordinates: `q = cdp("DOM.getBoxModel", backendNodeId=n)["model"]["content"]; x, y = sum(q[0::2])/4, sum(q[1::2])/4` (viewport px, ready for `click_at_xy`; negative/oversized means scroll first).
+- Clicking: AX node -> box center -> `click_at_xy(x, y)` -> verify with a targeted `js(...)`/`page_info()` check.
+- Fall back to raw HTML via `js(...)` only when the AX tree lacks the element (canvas, exotic widgets); screenshot when layout or imagery matters.
- After navigation, call `wait_for_load()`.
- If the current tab is stale or internal, call `ensure_real_tab()`.
- Use `js(...)` for DOM inspection or extraction when coordinates are the wrong tool.