Open-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent API.
Python ai-agentsai-toolsbrowser-agentbrowser-automationbrowser-usecdpcloud-browsercomputer-useheadless-browserllmmcpplaywrightpythonremote-browserstealth-browserweb-automationweb-scraping
Update system prompts for clarity and adjust bounding box rendering in Python highlights
Magnus Müller8ecc46daf673
diff --git a/.gitignore b/.gitignoreindex 19c9d9c16..3072cda88 100644--- a/.gitignore+++ b/.gitignore@@ -57,4 +57,4 @@ token.jsontemp-profile-*-*.png\ No newline at end of file+screenshot.png\ No newline at end of filediff --git a/browser_use/agent/system_prompt.md b/browser_use/agent/system_prompt.mdindex 83640b832..55f90005a 100644--- a/browser_use/agent/system_prompt.md+++ b/browser_use/agent/system_prompt.md@@ -66,8 +66,8 @@ Note that:</browser_state><browser_vision>-You will be optionally provided with a screenshot of the browser with bounding boxes. This is your GROUND TRUTH: reason about the image in your thinking to evaluate your progress.-Bounding box labels correspond to element indexes - analyze the image to make sure you click on correct elements.+You will be provided with a screenshot of the current page with bounding boxes around interactive elements. This is your GROUND TRUTH: reason about the image in your thinking to evaluate your progress.+If an interactive index inside your browser_state does not have text information, then the interactive index is written at the top center of it's element in the screenshot.</browser_vision><browser_rules>diff --git a/browser_use/agent/system_prompt_flash.md b/browser_use/agent/system_prompt_flash.mdindex a2f9257fc..97d454868 100644--- a/browser_use/agent/system_prompt_flash.md+++ b/browser_use/agent/system_prompt_flash.md@@ -64,8 +64,8 @@ Note that:</browser_state><browser_vision>-You will be optionally provided with a screenshot of the browser with bounding boxes. This is your GROUND TRUTH: analyze the image to evaluate your progress.-Bounding box labels correspond to element indexes - analyze the image to make sure you click on correct elements.+You will be provided with a screenshot of the current page with bounding boxes around interactive elements. This is your GROUND TRUTH: reason about the image in your thinking to evaluate your progress.+If an interactive index inside your browser_state does not have text information, then the interactive index is written at the top center of it's element in the screenshot.</browser_vision><browser_rules>diff --git a/browser_use/agent/system_prompt_no_thinking.md b/browser_use/agent/system_prompt_no_thinking.mdindex a79cb569b..c155da1f3 100644--- a/browser_use/agent/system_prompt_no_thinking.md+++ b/browser_use/agent/system_prompt_no_thinking.md@@ -66,8 +66,8 @@ Note that:</browser_state><browser_vision>-You will be optionally provided with a screenshot of the browser with bounding boxes. This is your GROUND TRUTH: analyze the image to evaluate your progress.-Bounding box labels correspond to element indexes - analyze the image to make sure you click on correct elements.+You will be provided with a screenshot of the current page with bounding boxes around interactive elements. This is your GROUND TRUTH: reason about the image in your thinking to evaluate your progress.+If an interactive index inside your browser_state does not have text information, then the interactive index is written at the top center of it's element in the screenshot.</browser_vision><browser_rules>diff --git a/browser_use/browser/python_highlights.py b/browser_use/browser/python_highlights.pyindex a8e7f4d1d..dfd23976a 100644--- a/browser_use/browser/python_highlights.py+++ b/browser_use/browser/python_highlights.py@@ -131,15 +131,17 @@ def draw_enhanced_bounding_box_with_text(container_width = text_width + padding * 2container_height = text_height + padding * 2- # Position in top-left corner (inside if fits, outside if too small)- if element_width >= container_width and element_height >= container_height:- # Place inside top-left corner- bg_x1 = x1 + 2 # Small offset from edge- bg_y1 = y1 + 2+ # Position in top center - for small elements, place further up to avoid blocking content+ # Center horizontally within the element+ bg_x1 = x1 + (element_width - container_width) // 2++ # Simple rule: if element is small, place index further up to avoid blocking icons+ if element_width < 60 or element_height < 30:+ # Small element: place well above to avoid blocking content+ bg_y1 = max(0, y1 - container_height - 5)else:- # Place outside top-left corner- bg_x1 = x1- bg_y1 = max(0, y1 - container_height)+ # Regular element: place inside with small offset+ bg_y1 = y1 + 2bg_x2 = bg_x1 + container_widthbg_y2 = bg_y1 + container_height@@ -482,7 +484,7 @@ async def create_highlighted_screenshot_async(async def _write_screenshot():with open(filename, 'wb') as f:f.write(base64.b64decode(final_screenshot))- logger.debug('Saved screenshot to screenshot.png')+ logger.debug('Saved screenshot to ' + str(filename))await asyncio.to_thread(_write_screenshot)return final_screenshot