Skip to content

Open-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent API.

handle autocomplete fields

Saurav Panda6878bd1ca4fb
diff --git a/browser_use/agent/system_prompts/system_prompt.md b/browser_use/agent/system_prompts/system_prompt.md
index 86248e26a..cca04084f 100644
--- a/browser_use/agent/system_prompts/system_prompt.md
+++ b/browser_use/agent/system_prompts/system_prompt.md
@@ -75,6 +75,7 @@ Strictly follow these rules while using the browser and navigating the web:
- If the <user_request> includes specific page information such as product type, rating, price, location, etc., try to apply filters to be more efficient.
- The <user_request> is the ultimate goal. If the user specifies explicit steps, they have always the highest priority.
- If you input into a field, you might need to press enter, click the search button, or select from dropdown for completion.
+- For autocomplete/combobox fields (e.g. search boxes with suggestions, fields with role="combobox"): type your search text, then WAIT for the suggestions dropdown to appear in the next step. Review the new elements (marked with *[) and click the correct suggestion. Do NOT press Enter or submit before selecting from the suggestions -- autocomplete fields require clicking a suggestion to set the correct value.
- Don't login into a page if you don't have to. Don't login if you don't have the credentials.
- There are 2 types of tasks always first think which type of request you are dealing with:
1. Very specific step by step instructions:
diff --git a/browser_use/agent/system_prompts/system_prompt_anthropic.md b/browser_use/agent/system_prompts/system_prompt_anthropic.md
index 252eff9c1..75c998677 100644
--- a/browser_use/agent/system_prompts/system_prompt_anthropic.md
+++ b/browser_use/agent/system_prompts/system_prompt_anthropic.md
@@ -75,6 +75,7 @@ Strictly follow these rules while using the browser and navigating the web:
- If the <user_request> includes specific page information such as product type, rating, price, location, etc., ALWAYS look for filter/sort options FIRST before browsing results. Apply all relevant filters before scrolling through results.
- The <user_request> is the ultimate goal. If the user specifies explicit steps, they have always the highest priority.
- If you input into a field, you might need to press enter, click the search button, or select from dropdown for completion.
+- For autocomplete/combobox fields (e.g. search boxes with suggestions, fields with role="combobox"): type your search text, then WAIT for the suggestions dropdown to appear in the next step. Review the new elements (marked with *[) and click the correct suggestion. Do NOT press Enter or submit before selecting from the suggestions -- autocomplete fields require clicking a suggestion to set the correct value.
- Don't login into a page if you don't have to. Don't login if you don't have the credentials.
- There are 2 types of tasks always first think which type of request you are dealing with:
1. Very specific step by step instructions:
diff --git a/browser_use/agent/system_prompts/system_prompt_anthropic_flash.md b/browser_use/agent/system_prompts/system_prompt_anthropic_flash.md
index bf0d8d643..60baf4c9f 100644
--- a/browser_use/agent/system_prompts/system_prompt_anthropic_flash.md
+++ b/browser_use/agent/system_prompts/system_prompt_anthropic_flash.md
@@ -41,6 +41,7 @@ Strictly follow these rules while using the browser and navigating the web:
- If the <user_request> includes specific page information such as product type, rating, price, location, etc., ALWAYS look for filter/sort options FIRST before browsing results. Apply all relevant filters before scrolling through results. This is critical for efficiency.
- The <user_request> is the ultimate goal. If the user specifies explicit steps, they have always the highest priority.
- If you input into a field, you might need to press enter, click the search button, or select from dropdown for completion.
+- For autocomplete/combobox fields (e.g. search boxes with suggestions, fields with role="combobox"): type your search text, then WAIT for the suggestions dropdown to appear in the next step. Review the new elements (marked with *[) and click the correct suggestion. Do NOT press Enter or submit before selecting from the suggestions -- autocomplete fields require clicking a suggestion to set the correct value.
- Don't login into a page if you don't have to. Don't login if you don't have the credentials.
- There are 2 types of tasks:
1. Very specific step by step instructions: Follow them as very precise and don't skip steps. Try to complete everything as requested.
diff --git a/browser_use/agent/system_prompts/system_prompt_anthropic_no_thinking.md b/browser_use/agent/system_prompts/system_prompt_anthropic_no_thinking.md
index 6f279349d..52799c202 100644
--- a/browser_use/agent/system_prompts/system_prompt_anthropic_no_thinking.md
+++ b/browser_use/agent/system_prompts/system_prompt_anthropic_no_thinking.md
@@ -75,6 +75,7 @@ Strictly follow these rules while using the browser and navigating the web:
- If the <user_request> includes specific page information such as product type, rating, price, location, etc., ALWAYS look for filter/sort options FIRST before browsing results. Apply all relevant filters before scrolling through results.
- The <user_request> is the ultimate goal. If the user specifies explicit steps, they have always the highest priority.
- If you input into a field, you might need to press enter, click the search button, or select from dropdown for completion.
+- For autocomplete/combobox fields (e.g. search boxes with suggestions, fields with role="combobox"): type your search text, then WAIT for the suggestions dropdown to appear in the next step. Review the new elements (marked with *[) and click the correct suggestion. Do NOT press Enter or submit before selecting from the suggestions -- autocomplete fields require clicking a suggestion to set the correct value.
- Don't login into a page if you don't have to. Don't login if you don't have the credentials.
- There are 2 types of tasks always first think which type of request you are dealing with:
1. Very specific step by step instructions:
diff --git a/browser_use/agent/system_prompts/system_prompt_no_thinking.md b/browser_use/agent/system_prompts/system_prompt_no_thinking.md
index 086cf882e..02fe42673 100644
--- a/browser_use/agent/system_prompts/system_prompt_no_thinking.md
+++ b/browser_use/agent/system_prompts/system_prompt_no_thinking.md
@@ -75,6 +75,7 @@ Strictly follow these rules while using the browser and navigating the web:
- If the <user_request> includes specific page information such as product type, rating, price, location, etc., try to apply filters to be more efficient.
- The <user_request> is the ultimate goal. If the user specifies explicit steps, they have always the highest priority.
- If you input into a field, you might need to press enter, click the search button, or select from dropdown for completion.
+- For autocomplete/combobox fields (e.g. search boxes with suggestions, fields with role="combobox"): type your search text, then WAIT for the suggestions dropdown to appear in the next step. Review the new elements (marked with *[) and click the correct suggestion. Do NOT press Enter or submit before selecting from the suggestions -- autocomplete fields require clicking a suggestion to set the correct value.
- Don't login into a page if you don't have to. Don't login if you don't have the credentials.
- There are 2 types of tasks always first think which type of request you are dealing with:
1. Very specific step by step instructions:
diff --git a/browser_use/browser/watchdogs/default_action_watchdog.py b/browser_use/browser/watchdogs/default_action_watchdog.py
index 10c676f6c..ade35c32f 100644
--- a/browser_use/browser/watchdogs/default_action_watchdog.py
+++ b/browser_use/browser/watchdogs/default_action_watchdog.py
@@ -1656,6 +1656,26 @@ class DefaultActionWatchdog(BaseWatchdog):
# to update their internal state and trigger re-renders
await self._trigger_framework_events(object_id=object_id, cdp_session=cdp_session)
+ # Step 5: Read back actual value for verification (skip for sensitive data)
+ if not is_sensitive:
+ try:
+ await asyncio.sleep(0.05) # let autocomplete/formatter JS settle
+ readback_result = await cdp_session.cdp_client.send.Runtime.callFunctionOn(
+ params={
+ 'objectId': object_id,
+ 'functionDeclaration': 'function() { return this.value !== undefined ? this.value : this.textContent; }',
+ 'returnByValue': True,
+ },
+ session_id=cdp_session.session_id,
+ )
+ actual_value = readback_result.get('result', {}).get('value')
+ if actual_value is not None:
+ if input_coordinates is None:
+ input_coordinates = {}
+ input_coordinates['actual_value'] = actual_value
+ except Exception as e:
+ self.logger.debug(f'Value readback failed (non-critical): {e}')
+
# Return coordinates metadata if available
return input_coordinates
diff --git a/browser_use/dom/views.py b/browser_use/dom/views.py
index 14a618747..42aafb68b 100644
--- a/browser_use/dom/views.py
+++ b/browser_use/dom/views.py
@@ -47,6 +47,8 @@ DEFAULT_INCLUDE_ATTRIBUTES = [
'multiple', # Whether multiple files/selections are allowed
'inputmode', # Virtual keyboard hint (numeric, tel, email, url, etc.)
'autocomplete', # Autocomplete behavior hint
+ 'aria-autocomplete', # ARIA autocomplete type (list, inline, both)
+ 'list', # Associated datalist element ID
'data-mask', # Input mask format (e.g., phone numbers, credit cards)
'data-inputmask', # Alternative input mask attribute
'data-datepicker', # jQuery datepicker indicator
@@ -114,8 +116,10 @@ STATIC_ATTRIBUTES = {
'aria-disabled',
'aria-hidden',
'aria-pressed',
+ 'aria-autocomplete',
'aria-checked',
'aria-selected',
+ 'list',
'tabindex',
'alt',
'src',
diff --git a/browser_use/tools/service.py b/browser_use/tools/service.py
index bcd2d9fbc..3a8ccbadc 100644
--- a/browser_use/tools/service.py
+++ b/browser_use/tools/service.py
@@ -102,6 +102,21 @@ def handle_browser_error(e: BrowserError) -> ActionResult:
raise e
+def _is_autocomplete_field(node: EnhancedDOMTreeNode) -> bool:
+ """Detect if a node is an autocomplete/combobox field from its attributes."""
+ attrs = node.attributes or {}
+ if attrs.get('role') == 'combobox':
+ return True
+ aria_ac = attrs.get('aria-autocomplete', '')
+ if aria_ac and aria_ac != 'none':
+ return True
+ if attrs.get('list'):
+ return True
+ if attrs.get('aria-haspopup') and (attrs.get('aria-controls') or attrs.get('aria-owns')):
+ return True
+ return False
+
+
class Tools(Generic[Context]):
def __init__(
self,
@@ -417,6 +432,18 @@ class Tools(Generic[Context]):
logger.debug(log_msg)
+ # Check for value mismatch (non-sensitive only)
+ actual_value = None
+ if isinstance(input_metadata, dict):
+ actual_value = input_metadata.pop('actual_value', None)
+
+ if not has_sensitive_data and actual_value is not None and actual_value != params.text:
+ msg += f"\n⚠️ Note: the field's actual value '{actual_value}' differs from typed text '{params.text}'. The page may have reformatted or autocompleted your input."
+
+ # Check for autocomplete/combobox field
+ if _is_autocomplete_field(node):
+ msg += '\n💡 This is an autocomplete field. Wait for suggestions to appear, then click the correct suggestion instead of pressing Enter.'
+
# Include input coordinates in metadata if available
return ActionResult(
extracted_content=msg,
diff --git a/tests/ci/interactions/test_autocomplete_interaction.py b/tests/ci/interactions/test_autocomplete_interaction.py
new file mode 100644
index 000000000..17b59bfdc
--- /dev/null
+++ b/tests/ci/interactions/test_autocomplete_interaction.py
@@ -0,0 +1,228 @@
+"""Test autocomplete/combobox field detection and value readback.
+
+Tests cover:
+- Value mismatch detection when JS rewrites input value
+- Combobox field detection (role=combobox + aria-autocomplete)
+- Datalist field detection (input with list attribute)
+- No false positives on plain inputs
+- Sensitive data skips value verification
+"""
+
+import asyncio
+
+import pytest
+from pytest_httpserver import HTTPServer
+
+from browser_use.agent.views import ActionResult
+from browser_use.browser import BrowserSession
+from browser_use.browser.profile import BrowserProfile
+from browser_use.tools.service import Tools
+
+
[email protected](scope='session')
+def http_server():
+ """Create and provide a test HTTP server with autocomplete test pages."""
+ server = HTTPServer()
+ server.start()
+
+ # Page 1: Input with JS that rewrites value on change (simulates autocomplete replacing text)
+ server.expect_request('/autocomplete-rewrite').respond_with_data(
+ """
+ <!DOCTYPE html>
+ <html>
+ <head><title>Autocomplete Rewrite Test</title></head>
+ <body>
+ <input id="search" type="text" />
+ <script>
+ const input = document.getElementById('search');
+ input.addEventListener('change', function() {
+ // Simulate autocomplete rewriting the value
+ this.value = 'REWRITTEN_' + this.value;
+ });
+ </script>
+ </body>
+ </html>
+ """,
+ content_type='text/html',
+ )
+
+ # Page 2: Input with role=combobox + aria-autocomplete=list + aria-controls + listbox
+ server.expect_request('/combobox-field').respond_with_data(
+ """
+ <!DOCTYPE html>
+ <html>
+ <head><title>Combobox Field Test</title></head>
+ <body>
+ <div>
+ <input id="combo" type="text" role="combobox"
+ aria-autocomplete="list" aria-controls="suggestions-list"
+ aria-expanded="false" />
+ <ul id="suggestions-list" role="listbox" style="display:none;">
+ <li role="option">Option A</li>
+ <li role="option">Option B</li>
+ </ul>
+ </div>
+ </body>
+ </html>
+ """,
+ content_type='text/html',
+ )
+
+ # Page 3: Input with list attribute pointing to a datalist
+ server.expect_request('/datalist-field').respond_with_data(
+ """
+ <!DOCTYPE html>
+ <html>
+ <head><title>Datalist Field Test</title></head>
+ <body>
+ <input id="city" type="text" list="suggestions" />
+ <datalist id="suggestions">
+ <option value="New York">
+ <option value="Los Angeles">
+ <option value="Chicago">
+ </datalist>
+ </body>
+ </html>
+ """,
+ content_type='text/html',
+ )
+
+ # Page 4: Plain input with no autocomplete attributes
+ server.expect_request('/normal-input').respond_with_data(
+ """
+ <!DOCTYPE html>
+ <html>
+ <head><title>Normal Input Test</title></head>
+ <body>
+ <input id="plain" type="text" placeholder="Just a normal input" />
+ </body>
+ </html>
+ """,
+ content_type='text/html',
+ )
+
+ yield server
+ server.stop()
+
+
[email protected](scope='session')
+def base_url(http_server):
+ """Return the base URL for the test HTTP server."""
+ return f'http://{http_server.host}:{http_server.port}'
+
+
[email protected](scope='module')
+async def browser_session():
+ """Create and provide a Browser instance for testing."""
+ browser_session = BrowserSession(
+ browser_profile=BrowserProfile(
+ headless=True,
+ user_data_dir=None,
+ keep_alive=True,
+ chromium_sandbox=False,
+ )
+ )
+ await browser_session.start()
+ yield browser_session
+ await browser_session.kill()
+
+
[email protected](scope='function')
+def tools():
+ """Create and provide a Tools instance."""
+ return Tools()
+
+
+class TestAutocompleteInteraction:
+ """Test autocomplete/combobox detection and value readback."""
+
+ async def test_value_mismatch_detected(self, tools: Tools, browser_session: BrowserSession, base_url: str):
+ """Type into a field whose JS rewrites the value on change. Assert the ActionResult notes the mismatch."""
+ await tools.navigate(url=f'{base_url}/autocomplete-rewrite', new_tab=False, browser_session=browser_session)
+ await asyncio.sleep(0.3)
+ await browser_session.get_browser_state_summary()
+
+ input_index = await browser_session.get_index_by_id('search')
+ assert input_index is not None, 'Could not find search input'
+
+ result = await tools.input(index=input_index, text='hello', browser_session=browser_session)
+
+ assert isinstance(result, ActionResult)
+ assert result.extracted_content is not None
+ assert 'differs from typed text' in result.extracted_content, (
+ f'Expected mismatch note in extracted_content, got: {result.extracted_content}'
+ )
+
+ async def test_combobox_field_detected(self, tools: Tools, browser_session: BrowserSession, base_url: str):
+ """Type into a combobox field. Assert the ActionResult includes autocomplete guidance."""
+ await tools.navigate(url=f'{base_url}/combobox-field', new_tab=False, browser_session=browser_session)
+ await asyncio.sleep(0.3)
+ await browser_session.get_browser_state_summary()
+
+ combo_index = await browser_session.get_index_by_id('combo')
+ assert combo_index is not None, 'Could not find combobox input'
+
+ result = await tools.input(index=combo_index, text='test', browser_session=browser_session)
+
+ assert isinstance(result, ActionResult)
+ assert result.extracted_content is not None
+ assert 'autocomplete field' in result.extracted_content, (
+ f'Expected autocomplete guidance in extracted_content, got: {result.extracted_content}'
+ )
+
+ async def test_datalist_field_detected(self, tools: Tools, browser_session: BrowserSession, base_url: str):
+ """Type into a datalist-backed field. Assert the ActionResult includes autocomplete guidance."""
+ await tools.navigate(url=f'{base_url}/datalist-field', new_tab=False, browser_session=browser_session)
+ await asyncio.sleep(0.3)
+ await browser_session.get_browser_state_summary()
+
+ city_index = await browser_session.get_index_by_id('city')
+ assert city_index is not None, 'Could not find datalist input'
+
+ result = await tools.input(index=city_index, text='New', browser_session=browser_session)
+
+ assert isinstance(result, ActionResult)
+ assert result.extracted_content is not None
+ assert 'autocomplete field' in result.extracted_content, (
+ f'Expected autocomplete guidance in extracted_content, got: {result.extracted_content}'
+ )
+
+ async def test_normal_input_no_false_positive(self, tools: Tools, browser_session: BrowserSession, base_url: str):
+ """Type into a plain input. Assert the ActionResult does NOT contain autocomplete guidance."""
+ await tools.navigate(url=f'{base_url}/normal-input', new_tab=False, browser_session=browser_session)
+ await asyncio.sleep(0.3)
+ await browser_session.get_browser_state_summary()
+
+ plain_index = await browser_session.get_index_by_id('plain')
+ assert plain_index is not None, 'Could not find plain input'
+
+ result = await tools.input(index=plain_index, text='hello', browser_session=browser_session)
+
+ assert isinstance(result, ActionResult)
+ assert result.extracted_content is not None
+ assert 'autocomplete field' not in result.extracted_content, (
+ f'Got false positive autocomplete guidance on plain input: {result.extracted_content}'
+ )
+
+ async def test_sensitive_data_skips_value_verification(self, tools: Tools, browser_session: BrowserSession, base_url: str):
+ """Type sensitive data into the rewrite field. Assert no 'differs from typed text' note appears."""
+ await tools.navigate(url=f'{base_url}/autocomplete-rewrite', new_tab=False, browser_session=browser_session)
+ await asyncio.sleep(0.3)
+ await browser_session.get_browser_state_summary()
+
+ input_index = await browser_session.get_index_by_id('search')
+ assert input_index is not None, 'Could not find search input'
+
+ # Use tools.act() with sensitive_data to trigger the sensitive code path
+ result = await tools.input(
+ index=input_index,
+ text='secret123',
+ browser_session=browser_session,
+ sensitive_data={'password': 'secret123'},
+ )
+
+ assert isinstance(result, ActionResult)
+ assert result.extracted_content is not None
+ assert 'differs from typed text' not in result.extracted_content, (
+ f'Sensitive data should not show value mismatch: {result.extracted_content}'
+ )