Skip to content

Open-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent API.

Trace Rust Agent tool definitions in Laminar

Magnus Müllerbcf147a2e2aa
diff --git a/RUST_AGENT_INTEGRATION_FEATURES.md b/RUST_AGENT_INTEGRATION_FEATURES.md
index edf596c91..ef0a09ec2 100644
--- a/RUST_AGENT_INTEGRATION_FEATURES.md
+++ b/RUST_AGENT_INTEGRATION_FEATURES.md
@@ -1452,6 +1452,8 @@ Terminal core branch: `magnus/browser-use-rust-main-integration` at terminal mai
277. Rust Agent exact Laminar tool replay
- Python `browser_use.rust.Agent` Laminar replay now creates child `TOOL` spans for terminal tool calls, including tool arguments, status/error attributes, structured text results, and inline image parts when the terminal event carries them.
+ - Rust LLM spans now include the full model-visible tool catalog in the Laminar span input, Laminar's current `gen_ai.tool.definitions` field, and backwards-compatible `llm.request.functions.*` attributes, including browser_script descriptions and input schemas.
+ - Rust LLM spans now emit full `gen_ai.input.messages` / `gen_ai.output.messages` arrays in addition to indexed legacy prompt/completion attributes, so Laminar can render the same system text, user text, images, tool calls, and assistant outputs the model saw without our old wrapper nesting.
- Rust LLM spans now include cache-read/cache-write usage and per-bucket cost attributes derived from the same official usage fields used by Browser Use token accounting, so Laminar and the eval dashboard can be compared without hidden pricing heuristics.
- Timeout/error reconstruction now preserves the latest streamed assistant text as the visible final result when the terminal process fails after an LLM turn has already produced content, while keeping the original error on the history.
- Proof: browser-use `uv run pytest -q tests/ci/test_rust_agent.py -k 'laminar_run_summary or loads_started_session_events_after_terminal_timeout' --disable-warnings`, browser-use `uv run python -m py_compile browser_use/rust/service.py tests/ci/test_rust_agent.py`, and browser-use `git diff --check -- browser_use/rust/service.py tests/ci/test_rust_agent.py`.
@@ -1459,8 +1461,9 @@ Terminal core branch: `magnus/browser-use-rust-main-integration` at terminal mai
278. Rust terminal structured replay and cache-window parity
- Terminal session reconstruction now falls back to structured `summary`, `data`, and `outputs` fields when browser_script and other tools return successful structured payloads with empty `text`/`content`, preventing future LLM turns and Laminar replay from seeing a blank tool result while the agent actually received evidence.
- The Rust request builder keeps the latest model-visible message cacheable and adds one earlier non-system cache breakpoint for long histories, staying within Anthropic's four-breakpoint limit together with the system prompt and tool schema breakpoint.
+ - Terminal `model.turn.request` events now carry the full sanitized `llm_input.tools` array and repeated model round-trips from the same driver emit monotonic `turn_idx` values, so Laminar can display tool descriptions and order turns without relying only on span order.
- This is a general cache-window fix for long browser/research trajectories; it does not add task-specific instructions or domain-specific behavior.
- - Proof: terminal `cargo test -q -p browser-use-agent tool_output_event_uses_structured_browser_script_fallback_text -- --nocapture`, terminal `cargo test -q -p browser-use-agent open_stream_marks_an_earlier_cache_breakpoint_for_long_histories -- --nocapture`, terminal `cargo test -q -p browser-use-agent turn_request_event_carries_full_llm_input_messages -- --nocapture`, terminal `cargo test -q -p browser-use-agent turn_request_event_carries_all_observability_messages_without_text_budget -- --nocapture`, terminal `cargo fmt --check`, terminal `git diff --check -- crates/browser-use-agent/src/session/reconstruct.rs crates/browser-use-agent/src/session/reconstruct_tests.rs crates/browser-use-agent/src/turn/sampling.rs crates/browser-use-agent/src/turn/sampling_tests.rs`, and terminal `cargo build -q -p browser-use-cli --bin browser-use-terminal`.
+ - Proof: terminal `cargo test -q -p browser-use-agent repeated_sampling_requests_emit_monotonic_turn_indices -- --nocapture`, terminal `cargo test -q -p browser-use-agent fused_driver_advertises_dispatcher_tool_specs_on_request -- --nocapture`, terminal `cargo test -q -p browser-use-agent tool_output_event_uses_structured_browser_script_fallback_text -- --nocapture`, terminal `cargo test -q -p browser-use-agent open_stream_marks_an_earlier_cache_breakpoint_for_long_histories -- --nocapture`, terminal `cargo test -q -p browser-use-agent turn_request_event_carries_full_llm_input_messages -- --nocapture`, terminal `cargo test -q -p browser-use-agent turn_request_event_carries_all_observability_messages_without_text_budget -- --nocapture`, terminal `cargo fmt --check`, terminal `git diff --check -- crates/browser-use-agent/src/session/reconstruct.rs crates/browser-use-agent/src/session/reconstruct_tests.rs crates/browser-use-agent/src/turn/sampling.rs crates/browser-use-agent/src/turn/sampling_tests.rs`, and terminal `cargo build -q -p browser-use-cli --bin browser-use-terminal`.
## Current Verification
diff --git a/browser_use/rust/service.py b/browser_use/rust/service.py
index a7d3d8d76..6bbdb16e0 100644
--- a/browser_use/rust/service.py
+++ b/browser_use/rust/service.py
@@ -2581,13 +2581,16 @@ def _terminal_laminar_turn_input(
composition = {}
raw_tools = composition.get('tools')
tools = raw_tools if isinstance(raw_tools, list) else []
- tool_names = []
- for tool in tools[:30]:
- if isinstance(tool, dict) and isinstance(tool.get('name'), str):
- tool_names.append(tool['name'])
llm_input = request_payload.get('llm_input')
if not isinstance(llm_input, dict):
llm_input = {}
+ llm_tools = llm_input.get('tools')
+ if isinstance(llm_tools, list):
+ tools = [tool for tool in llm_tools if isinstance(tool, dict)]
+ tool_names = []
+ for tool in tools:
+ if isinstance(tool, dict) and isinstance(tool.get('name'), str):
+ tool_names.append(tool['name'])
messages = llm_input.get('messages')
if not isinstance(messages, list):
messages = []
@@ -2602,6 +2605,7 @@ def _terminal_laminar_turn_input(
'system_prompt_tokens': composition.get('system_prompt_tokens'),
'tools_count': len(tools),
'tool_names': tool_names,
+ 'tools': tools,
'system': system,
'messages': messages,
'message_count': _int_value(llm_input.get('message_count')),
@@ -2740,6 +2744,21 @@ def _terminal_laminar_span_input_messages(span_input: dict[str, Any]) -> list[di
return [*_terminal_laminar_system_messages(span_input), *normalized_messages]
+def _terminal_laminar_tools_for_span(span_input: dict[str, Any]) -> list[dict[str, Any]]:
+ tools = span_input.get('tools')
+ if not isinstance(tools, list):
+ return []
+ return [tool for tool in tools if isinstance(tool, dict)]
+
+
+def _terminal_laminar_span_input_payload(span_input: dict[str, Any]) -> Any:
+ messages = _terminal_laminar_span_input_messages(span_input)
+ tools = _terminal_laminar_tools_for_span(span_input)
+ if not tools:
+ return messages
+ return {'messages': messages, 'tools': tools}
+
+
def _terminal_laminar_span_output_messages(span_output: dict[str, Any]) -> list[dict[str, Any]]:
messages = span_output.get('messages')
return messages if isinstance(messages, list) else []
@@ -2766,6 +2785,90 @@ def _terminal_laminar_message_text(message: dict[str, Any]) -> str:
return '\n'.join(parts)
+def _terminal_laminar_semconv_content_part(part: Any) -> dict[str, Any]:
+ if not isinstance(part, dict):
+ return {'type': 'text', 'content': str(part)}
+ part_type = part.get('type')
+ if part_type in {'text', 'input_text', 'output_text'}:
+ text = part.get('text') or part.get('content')
+ if isinstance(text, str):
+ return {'type': 'text', 'content': text}
+ if part_type == 'image_url':
+ image_url = part.get('image_url')
+ url = image_url.get('url') if isinstance(image_url, dict) else None
+ if isinstance(url, str) and url:
+ if url.startswith('data:') and ';base64,' in url:
+ header, blob = url.split(';base64,', 1)
+ mime_type = header.removeprefix('data:') or 'application/octet-stream'
+ return {'type': 'blob', 'blob': blob, 'mimeType': mime_type}
+ return {'type': 'uri', 'uri': url}
+ if part_type in {'tool_use', 'tool_call'}:
+ tool_call: dict[str, Any] = {'type': 'tool_call'}
+ tool_id = part.get('id') or part.get('tool_call_id')
+ name = part.get('name')
+ arguments = part.get('arguments') if 'arguments' in part else part.get('input')
+ if isinstance(tool_id, str):
+ tool_call['id'] = tool_id
+ if isinstance(name, str):
+ tool_call['name'] = name
+ if arguments is not None:
+ tool_call['arguments'] = arguments
+ return tool_call
+ if part_type == 'tool_result':
+ response: dict[str, Any] = {'type': 'tool_call_response'}
+ tool_id = part.get('tool_use_id') or part.get('tool_call_id') or part.get('id')
+ if isinstance(tool_id, str):
+ response['id'] = tool_id
+ if 'content' in part:
+ response['response'] = part.get('content')
+ return response
+ content = part.get('content')
+ if isinstance(content, str):
+ return {'type': 'text', 'content': content}
+ return {'type': 'text', 'content': _terminal_laminar_json_attribute(part)}
+
+
+def _terminal_laminar_semconv_messages(messages: list[dict[str, Any]]) -> list[dict[str, Any]]:
+ semconv_messages: list[dict[str, Any]] = []
+ for message in messages:
+ if not isinstance(message, dict):
+ continue
+ role = message.get('role')
+ if not isinstance(role, str):
+ continue
+ content = message.get('content')
+ parts: list[dict[str, Any]]
+ if isinstance(content, list):
+ parts = [_terminal_laminar_semconv_content_part(part) for part in content]
+ elif isinstance(content, str):
+ parts = [{'type': 'text', 'content': content}]
+ else:
+ parts = []
+ tool_calls = message.get('tool_calls')
+ if isinstance(tool_calls, list):
+ for tool_call in tool_calls:
+ parts.append(_terminal_laminar_semconv_content_part({'type': 'tool_call', **tool_call}))
+ semconv_messages.append({'role': role, 'parts': parts})
+ return semconv_messages
+
+
+def _terminal_laminar_system_instructions(span_input: dict[str, Any]) -> str:
+ system_parts = span_input.get('system')
+ if not isinstance(system_parts, list):
+ return ''
+ texts: list[str] = []
+ for part in system_parts:
+ if isinstance(part, dict):
+ text = part.get('text') or part.get('content')
+ if isinstance(text, str):
+ texts.append(text)
+ elif part:
+ texts.append(_terminal_laminar_json_attribute(part))
+ elif isinstance(part, str):
+ texts.append(part)
+ return '\n\n'.join(texts)
+
+
def _terminal_laminar_indexed_message_attributes(
messages: list[dict[str, Any]], prefix: str, *, max_messages: int = 20
) -> dict[str, Any]:
@@ -2784,6 +2887,46 @@ def _terminal_laminar_indexed_message_attributes(
return attributes
+def _terminal_laminar_json_attribute(value: Any) -> str:
+ return json.dumps(value, ensure_ascii=False, default=str)
+
+
+def _terminal_laminar_tool_definition_attributes(tools: list[dict[str, Any]]) -> dict[str, Any]:
+ if not tools:
+ return {}
+ attributes: dict[str, Any] = {
+ 'gen_ai.tool.definitions': _terminal_laminar_json_attribute(tools),
+ 'gen_ai.request.tools': _terminal_laminar_json_attribute(tools),
+ 'llm.request.functions': _terminal_laminar_json_attribute(tools),
+ }
+ for index, tool in enumerate(tools):
+ name = tool.get('name')
+ description = tool.get('description')
+ input_schema = tool.get('input_schema') or tool.get('parameters')
+ output_schema = tool.get('output_schema')
+ namespace = tool.get('namespace')
+ namespace_description = tool.get('namespace_description')
+ prefix = f'llm.request.functions.{index}'
+ if isinstance(name, str):
+ attributes[f'{prefix}.name'] = name
+ attributes[f'gen_ai.request.tools.{index}.name'] = name
+ if isinstance(description, str):
+ attributes[f'{prefix}.description'] = description
+ attributes[f'gen_ai.request.tools.{index}.description'] = description
+ if input_schema is not None:
+ schema = _terminal_laminar_json_attribute(input_schema)
+ attributes[f'{prefix}.input_schema'] = schema
+ attributes[f'{prefix}.parameters'] = schema
+ attributes[f'gen_ai.request.tools.{index}.input_schema'] = schema
+ if output_schema is not None:
+ attributes[f'{prefix}.output_schema'] = _terminal_laminar_json_attribute(output_schema)
+ if isinstance(namespace, str):
+ attributes[f'{prefix}.namespace'] = namespace
+ if isinstance(namespace_description, str):
+ attributes[f'{prefix}.namespace_description'] = namespace_description
+ return attributes
+
+
def _terminal_laminar_gen_ai_attributes(
span_input: dict[str, Any],
span_output: dict[str, Any],
@@ -2795,10 +2938,13 @@ def _terminal_laminar_gen_ai_attributes(
'gen_ai.operation.name': 'chat',
'gen_ai.request.model': str(span_input.get('model') or ''),
'gen_ai.system': str(span_input.get('provider') or ''),
- 'gen_ai.system_instructions': _laminar_preview(span_input.get('system') or [], limit=30_000),
+ 'gen_ai.system_instructions': _terminal_laminar_system_instructions(span_input),
+ 'gen_ai.input.messages': _terminal_laminar_json_attribute(_terminal_laminar_semconv_messages(input_messages)),
+ 'gen_ai.output.messages': _terminal_laminar_json_attribute(_terminal_laminar_semconv_messages(output_messages)),
}
attributes.update(_terminal_laminar_indexed_message_attributes(input_messages, 'gen_ai.prompt'))
attributes.update(_terminal_laminar_indexed_message_attributes(output_messages, 'gen_ai.completion'))
+ attributes.update(_terminal_laminar_tool_definition_attributes(_terminal_laminar_tools_for_span(span_input)))
if usage:
attributes.update(
{
@@ -2923,8 +3069,8 @@ def _record_laminar_terminal_llm_spans(
usage.update(cost)
start_time = _event_time_seconds(turn_events[0], 0.0)
end_time = _event_time_seconds(turn_events[-1], start_time)
- span_input_messages = _terminal_laminar_span_input_messages(span_input)
- with _laminar_start_span('rust_core.llm', input=span_input_messages, span_type='LLM'):
+ span_input_payload = _terminal_laminar_span_input_payload(span_input)
+ with _laminar_start_span('rust_core.llm', input=span_input_payload, span_type='LLM'):
span_output = _terminal_laminar_turn_output(turn_events, raw_usage)
span_output_messages = _terminal_laminar_span_output_messages(span_output)
_laminar_set_span_attributes(
diff --git a/tests/ci/test_rust_agent.py b/tests/ci/test_rust_agent.py
index c771941e7..45863bc7c 100644
--- a/tests/ci/test_rust_agent.py
+++ b/tests/ci/test_rust_agent.py
@@ -7471,22 +7471,33 @@ def test_rust_agent_laminar_run_summary_populates_current_span(monkeypatch):
},
'llm_input': {
'system': [{'text': 'System prompt'}],
- 'messages': [
- {
- 'role': 'user',
- 'content': [
- {'type': 'text', 'text': 'Find the title on example.com'},
- {
- 'type': 'media',
- 'mime_type': 'image/png',
- 'data': 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAAB',
- 'detail': 'low',
- },
- ],
- }
- ],
- 'message_count': 1,
- 'omitted_earlier_messages': 0,
+ 'tools': [
+ {
+ 'name': 'browser_script',
+ 'description': 'Run browser Python. Use click_at_xy(x, y) to click visible page coordinates.',
+ 'input_schema': {
+ 'type': 'object',
+ 'properties': {'code': {'type': 'string'}},
+ 'required': ['code'],
+ },
+ }
+ ],
+ 'messages': [
+ {
+ 'role': 'user',
+ 'content': [
+ {'type': 'text', 'text': 'Find the title on example.com'},
+ {
+ 'type': 'media',
+ 'mime_type': 'image/png',
+ 'data': 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAAB',
+ 'detail': 'low',
+ },
+ ],
+ }
+ ],
+ 'message_count': 1,
+ 'omitted_earlier_messages': 0,
},
},
},
@@ -7559,21 +7570,23 @@ def test_rust_agent_laminar_run_summary_populates_current_span(monkeypatch):
assert FakeLaminar.events[-1][0] == 'agent.run.terminal_summary'
assert FakeLaminar.spans[0]['name'] == 'rust_core.llm'
assert FakeLaminar.spans[0]['span_type'] == 'LLM'
- assert FakeLaminar.spans[0]['input'][0]['role'] == 'system'
- assert FakeLaminar.spans[0]['input'][0]['content'][0]['text'] == 'System prompt'
- assert FakeLaminar.spans[0]['input'][1]['content'][0]['text'] == 'Find the title on example.com'
- assert FakeLaminar.spans[0]['input'][1]['content'][1] == {
+ assert FakeLaminar.spans[0]['input']['messages'][0]['role'] == 'system'
+ assert FakeLaminar.spans[0]['input']['messages'][0]['content'][0]['text'] == 'System prompt'
+ assert FakeLaminar.spans[0]['input']['messages'][1]['content'][0]['text'] == 'Find the title on example.com'
+ assert FakeLaminar.spans[0]['input']['messages'][1]['content'][1] == {
'type': 'image_url',
'image_url': {
'url': 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAAB',
'detail': 'low',
},
}
+ assert FakeLaminar.spans[0]['input']['tools'][0]['name'] == 'browser_script'
+ assert 'click_at_xy' in FakeLaminar.spans[0]['input']['tools'][0]['description']
span_attrs = {}
for attributes in FakeLaminar.spans[0]['attributes']:
span_attrs.update(attributes)
- assert span_attrs['tools_count'] == 2
- assert 'browser' in span_attrs['tool_names']
+ assert span_attrs['tools_count'] == 1
+ assert 'browser_script' in span_attrs['tool_names']
assert span_attrs['input_tokens'] == 100
assert span_attrs['cached_input_tokens'] == 5
assert span_attrs['cache_creation_input_tokens'] == 13
@@ -7594,6 +7607,15 @@ def test_rust_agent_laminar_run_summary_populates_current_span(monkeypatch):
assert span_attrs['gen_ai.usage.cache_creation_input_tokens'] == 13
assert span_attrs['gen_ai.usage.output_tokens'] == 20
assert span_attrs['gen_ai.usage.total_tokens'] == 133
+ assert span_attrs['llm.request.functions.0.name'] == 'browser_script'
+ assert 'click_at_xy' in span_attrs['llm.request.functions.0.description']
+ assert '"code"' in span_attrs['llm.request.functions.0.input_schema']
+ assert '"browser_script"' in span_attrs['gen_ai.tool.definitions']
+ assert 'click_at_xy' in span_attrs['gen_ai.tool.definitions']
+ assert '"browser_script"' in span_attrs['gen_ai.request.tools']
+ assert '"System prompt"' in span_attrs['gen_ai.input.messages']
+ assert 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAAB' in span_attrs['gen_ai.input.messages']
+ assert '"laminar answer"' in span_attrs['gen_ai.output.messages']
assert span_attrs['assistant_output_preview'] == 'laminar answer'
assert FakeLaminar.spans[0]['outputs'][-1][0]['content'][0]['text'] == 'laminar answer'
assert FakeLaminar.spans[1]['name'] == 'rust_core.tool.browser_script'