Automate browser based workflows with AI
Python aiapiautomationbrowserbrowser-automationcomputergptllmplaywrightpowerautomatepuppeteerpythonrpaseleniumvisionworkflow
SKY-9637: verify dynamic goto_url state before extraction (#5892)
Andrew Neilson104b8a4e249c
diff --git a/skyvern/forge/prompts/skyvern/workflow-copilot-agent.j2 b/skyvern/forge/prompts/skyvern/workflow-copilot-agent.j2index 2afbb613e..6590be319 100644--- a/skyvern/forge/prompts/skyvern/workflow-copilot-agent.j2+++ b/skyvern/forge/prompts/skyvern/workflow-copilot-agent.j2@@ -46,6 +46,19 @@ Prefer workflow-run evidence first. Use navigate_browser, get_browser_screenshot2. The latest workflow run evidence is missing, ambiguous, or contradictory.3. You need to validate a specific UI hypothesis before changing the workflow.+GOTO_URL STATE SHORTCUT POLICY:+Treat `goto_url` as a fast bootstrap, not as the source of truth for dynamic page state.+- URL guessing is allowed for the first pass when it gets you to a plausible results/search page quickly.+- URL refinement for dynamic page state is allowed ONLY when grounded in live site evidence: active chips, checked controls, state-changing links, form values, DOM attributes, or URL deltas observed after UI actions.+- Ungrounded URL parameter tweaking is not allowed. If the page does not verify the intended state, inspect it with a `code` or `validation` block, or apply missing state with a `navigation` block.+- For stateful search tasks with multiple stable inputs before results appear, do not spend one click-driven `navigation` block per field after the entry page loads. Before adding a second stable search-form/date/query-input block, use observed same-site evidence (current URL, form action, field names/values, result links, or URL deltas) to bootstrap stable search state with `goto_url`, or add a short `code` block that inspects the DOM and navigates/verifies the stable results page. Reserve click-driven `navigation` blocks for missing dynamic state or cases where the URL/form evidence is insufficient.+- Do not bypass the previous rule by putting the entry URL and stable search setup in the same `navigation` block. Split pure page loading into `goto_url` first, then inspect/bootstrap/apply state in later blocks.+- Before extraction on stateful search/result tasks, verify every requested constraint independently and simultaneously. Use live page evidence, not the presence of guessed query params.+- Represent broad constraints as intent, not as a single fragile URL shape. Verify the visible state that satisfies the intent instead of assuming one query parameter means the whole constraint is active.+- If a shortcut partially works, keep the useful state and apply/verify the missing constraints through UI/DOM evidence instead of looping on speculative `goto_url` variants.+- If a state-application block hits a per-tool budget, do not rerun the same navigation label unchanged. Split/replace it with a smaller one-action UI block, or use `code`/`validation` to inspect the visible DOM state before deciding what is missing.+- Put the verification evidence in the workflow itself (prefer `code` for DOM checks, `validation` for semantic/visual checks, and `navigation` only when state needs to be changed) and in `global_llm_context.decisions_made` as a per-constraint summary such as `constraint / expected / observed / source`.+CRITICAL RULES WHEN USING DIRECT BROWSER TOOLS:- If you call navigate_browser, you MUST call at least one observation tool (evaluate, get_browser_screenshot, click, type_text, scroll, select_option, press_key, or console_messages) before responding.- NEVER answer questions about page content, link counts, link targets, or page structure from memory or training data. Use evaluate or get_browser_screenshot.@@ -125,7 +138,7 @@ IMPORTANT WORKFLOW RULES:- "login" for authentication flows.- "wait" for explicit pauses between blocks.- "goto_url" for direct URL navigation and grounded URL-state shortcuts.-* Prefer a `goto_url` URL-parameter shortcut over a click-driven `navigation` block when requested state (dates, destination, filters, search terms) maps to a URL that is supplied by the user, observed on the site, present in a link/form/action URL, or follows a same-site pattern visible in context. Do NOT invent website-specific paths or query parameters from memory — when the URL shape is uncertain, use a narrow `navigation` block instead. Put any follow-up actions in subsequent blocks, and follow URL-state shortcuts with extraction or validation that checks the visible page state.+* Prefer a `goto_url` URL-parameter shortcut over a click-driven `navigation` block when requested state (dates, destination, search terms, stable search params) maps to a URL that is supplied by the user, observed on the site, present in a link/form/action URL, or follows a same-site pattern visible in context. Do NOT invent website-specific paths or query parameters from memory. For URL-encoded dynamic page state, `goto_url` may bootstrap the page but must be followed by a persisted verification block before extraction: use `code` for DOM checks, `validation` for semantic/visual checks, or `navigation` when missing state must be changed through the UI. Put any follow-up actions in subsequent blocks, and verify dynamic URL-state shortcuts from the live page before extraction.* Commit a draft workflow as soon as you have a plausible block sequence and a target URL/selector. Do NOT walk the search UI step-by-step or fetch page contents through `evaluate`/`fetch` before the first `update_and_run_blocks` call when a `goto_url` to a likely page can be tested directly. Page introspection through direct browser tools is a fallback after a block run fails or returns ambiguous evidence, not preparation for the first run.* Keep `navigation_goal` to ONE atomic action per block. You MUST split into separate blocks if your `navigation_goal` would cover more than one of: (a) a search-form input, (b) a date or filter selection, (c) a results-page click into a detail view, (d) a final extraction. Conjunctions like "and"/"then"/commas separating distinct UI actions inside a single `navigation_goal` are a strong signal you are bundling — decompose. The verifier checks every sub-condition after each step, so bundled goals return "continue" until the budget exhausts. When one request maps to several sub-tasks and no `goto_url` shortcut applies, decompose into one block per sub-task chained via `next_block_label`. The final stage of search → click → extract should be an `extraction` block, not another `navigation`.* Set `complete_criterion` on every decomposed `navigation` block to the observable post-action page state — without it, smaller blocks still return vague "continue" verdicts.diff --git a/skyvern/forge/sdk/copilot/context.py b/skyvern/forge/sdk/copilot/context.pyindex d50e30a1a..7f77a18c8 100644--- a/skyvern/forge/sdk/copilot/context.py+++ b/skyvern/forge/sdk/copilot/context.py@@ -210,6 +210,12 @@ class CopilotContext(AgentContext):probable_site_block_stop_nudge_count: int = 0per_tool_budget_nudge_count: int = 0effective_workflow_proxy_location: Any | None = None+ # Labels of navigation blocks that were canceled/failed inside a+ # PER_TOOL_BUDGET run. Armed after get_run_results inspects the budgeted+ # run, and cleared only when a workflow update removes or changes the+ # label away from navigation. Prevents rerunning the same oversized+ # navigation block unchanged.+ per_tool_budget_problem_block_labels: list[str] = field(default_factory=list)# Per-request frontier state. `verified_block_outputs` and# `verified_prefix_labels` are populated ONLY from fully-successful runs —diff --git a/skyvern/forge/sdk/copilot/enforcement.py b/skyvern/forge/sdk/copilot/enforcement.pyindex bb306ea57..f5e9b8db9 100644--- a/skyvern/forge/sdk/copilot/enforcement.py+++ b/skyvern/forge/sdk/copilot/enforcement.py@@ -251,15 +251,23 @@ POST_PER_TOOL_BUDGET_NUDGE = ("single tool call.\n""Do NOT retry the same chain. Do NOT change navigation_goal wording or ""selectors hoping it will run faster.\n"- "1. Shrink the requested block_labels list to the first 1-2 unverified "+ "1. Call get_run_results for the budgeted run. If it shows a navigation "+ "block was canceled or failed, do NOT run that same label again unchanged; "+ "split or replace it first.\n"+ "2. Shrink the requested block_labels list to the first 1-2 unverified ""blocks. The verified-prefix optimization will replay any earlier blocks ""from cached state without re-running the browser, so passing a smaller ""frontier is cheap.\n"- "2. Test that smaller frontier with update_and_run_blocks. If it succeeds, "- "extend by one block at a time on subsequent calls.\n"- "3. If your workflow only has 1-2 blocks and one block is still hitting "- "the budget, the single block is too ambitious — either narrow its "- "navigation_goal scope, or reply with a blocker explanation."+ "3. For page-state work, prefer a code or validation block to verify DOM "+ "state, active chips, checked controls, field values, or URL deltas. Use a "+ "navigation block only when missing state must be changed through the UI, "+ "and keep it to one atomic action.\n"+ "4. Test the smaller frontier. If it succeeds, extend by one block at a "+ "time on subsequent calls.\n"+ "5. If your workflow only has 1-2 blocks and one block is still hitting "+ "the budget, the single block is too ambitious — either narrow its scope, "+ "replace it with DOM/state verification plus smaller actions, or reply "+ "with a blocker explanation.")POST_NO_WORKFLOW_DELIVERY_NUDGE = (diff --git a/skyvern/forge/sdk/copilot/mcp_adapter.py b/skyvern/forge/sdk/copilot/mcp_adapter.pyindex a5ee9ca03..50a1daec0 100644--- a/skyvern/forge/sdk/copilot/mcp_adapter.py+++ b/skyvern/forge/sdk/copilot/mcp_adapter.py@@ -54,6 +54,7 @@ class SchemaOverlay:LOG = structlog.get_logger()+_INTERNAL_TOOL_ARG_KEYS = frozenset({"_summarized"})def _apply_schema_overlay(@@ -89,7 +90,7 @@ def _transform_args(arguments: dict[str, Any],overlay: SchemaOverlay,) -> dict[str, Any]:- mcp_args = {k: v for k, v in arguments.items() if k not in overlay.hide_params}+ mcp_args = {k: v for k, v in arguments.items() if k not in overlay.hide_params | _INTERNAL_TOOL_ARG_KEYS}for copilot_param, mcp_param in overlay.arg_transforms.items():if copilot_param in mcp_args:@@ -195,6 +196,7 @@ class SkyvernOverlayMCPServer(MCPServer):raise RuntimeError("Not connected — call connect() first")arguments = arguments or {}+ arguments = {k: v for k, v in arguments.items() if k not in _INTERNAL_TOOL_ARG_KEYS}copilot_ctx = self._context_provider()overlay = self._overlays.get(tool_name, SchemaOverlay())diff --git a/skyvern/forge/sdk/copilot/tools.py b/skyvern/forge/sdk/copilot/tools.pyindex 6aaf4fc60..340c3d8fe 100644--- a/skyvern/forge/sdk/copilot/tools.py+++ b/skyvern/forge/sdk/copilot/tools.py@@ -10,6 +10,7 @@ import timefrom collections import Counter, defaultdictfrom datetime import datetimefrom typing import Any, Literal+from urllib.parse import parse_qsl, urlparseimport structlogimport yaml@@ -376,6 +377,102 @@ def _maybe_clear_reconciliation_flag(copilot_ctx: Any, result: Any) -> None:copilot_ctx.pending_reconciliation_run_id = None+def _enum_or_string_name(value: Any) -> str:+ raw = getattr(value, "value", value)+ if not isinstance(raw, str):+ raw = getattr(value, "name", raw)+ return str(raw).strip().lower()+++def _per_tool_budget_problem_label_set(ctx: Any) -> set[str]:+ labels = getattr(ctx, "per_tool_budget_problem_block_labels", None)+ if not isinstance(labels, list):+ return set()+ return {label for label in labels if isinstance(label, str) and label}+++def _record_per_tool_budget_problem_blocks_from_results(copilot_ctx: Any, result: dict[str, Any]) -> None:+ if getattr(copilot_ctx, "last_failure_category_top", None) != PER_TOOL_BUDGET_FAILURE_CATEGORY:+ return+ data = result.get("data")+ if not isinstance(data, dict):+ return+ pending_run_id = getattr(copilot_ctx, "pending_reconciliation_run_id", None)+ resolved_run_id = data.get("workflow_run_id")+ if isinstance(pending_run_id, str) and pending_run_id and resolved_run_id != pending_run_id:+ return+ blocks = data.get("blocks")+ if not isinstance(blocks, list):+ return++ labels = _per_tool_budget_problem_label_set(copilot_ctx)+ for block in blocks:+ if not isinstance(block, dict):+ continue+ if _enum_or_string_name(block.get("block_type")) != BlockType.NAVIGATION.value:+ continue+ if _enum_or_string_name(block.get("status")) not in _FAILED_BLOCK_STATUSES:+ continue+ label = block.get("label")+ if isinstance(label, str) and label:+ labels.add(label)+ setattr(copilot_ctx, "per_tool_budget_problem_block_labels", sorted(labels))+++def _navigation_labels_in_workflow(workflow: Any) -> set[str]:+ definition = getattr(workflow, "workflow_definition", None)+ blocks = getattr(definition, "blocks", None)+ if not isinstance(blocks, list):+ return set()++ labels: set[str] = set()+ for block in blocks:+ if _enum_or_string_name(getattr(block, "block_type", None)) != BlockType.NAVIGATION.value:+ continue+ label = getattr(block, "label", None)+ if isinstance(label, str) and label:+ labels.add(label)+ return labels+++def _clear_resolved_per_tool_budget_problem_labels(copilot_ctx: Any, workflow: Any) -> None:+ problem_labels = _per_tool_budget_problem_label_set(copilot_ctx)+ if not problem_labels:+ return+ remaining = sorted(problem_labels & _navigation_labels_in_workflow(workflow))+ setattr(copilot_ctx, "per_tool_budget_problem_block_labels", remaining)+++def _requested_block_label_set(arguments: dict[str, Any] | None) -> set[str] | None:+ if not isinstance(arguments, dict):+ return None+ block_labels = arguments.get("block_labels")+ if not isinstance(block_labels, list):+ return None+ return {label for label in block_labels if isinstance(label, str) and label}+++def _per_tool_budget_problem_rerun_error(ctx: Any, arguments: dict[str, Any] | None) -> str | None:+ problem_labels = _per_tool_budget_problem_label_set(ctx)+ if not problem_labels:+ return None++ requested_labels = _requested_block_label_set(arguments)+ blocked_labels = problem_labels if not requested_labels else problem_labels & requested_labels+ if not blocked_labels:+ return None++ labels = ", ".join(sorted(blocked_labels))+ return (+ "The prior PER_TOOL_BUDGET run's get_run_results showed navigation "+ f"block(s) [{labels}] were canceled or failed while applying page state. "+ "Do NOT rerun those block label(s) unchanged with run_blocks_and_collect_debug. "+ "Update the workflow to split or replace the oversized navigation block first, "+ "prefer a code or validation block for DOM/page-state verification, or run only "+ "newly-created smaller block labels that apply one missing constraint at a time."+ )++# Streak threshold at which the copilot hard-aborts a tool call because the# same action sequence has repeated run-over-run with no intervening success.# The streak counter is incremented in ``update_repeated_failure_state`` AFTER@@ -605,6 +702,11 @@ def _tool_loop_error(ctx: AgentContext, tool_name: str, arguments: dict[str, Anyf"side effects on live sites.")+ if tool_name == "run_blocks_and_collect_debug":+ budget_rerun_error = _per_tool_budget_problem_rerun_error(ctx, arguments)+ if budget_rerun_error is not None:+ return budget_rerun_error+streak_raw = getattr(ctx, "repeated_action_fingerprint_streak_count", 0)streak = streak_raw if isinstance(streak_raw, int) else 0if streak >= REPEATED_ACTION_STREAK_ABORT_AT:@@ -750,6 +852,21 @@ async def _update_workflow(if banned_items:_record_banned_block_reject_span("_update_workflow", banned_items)return {"ok": False, "error": _banned_block_reject_message(banned_items)}++ goto_url_state_shortcuts = _detect_unverified_goto_url_state_shortcuts(workflow_yaml)+ if goto_url_state_shortcuts:+ _record_goto_url_state_shortcut_reject_span("_update_workflow", goto_url_state_shortcuts)+ return {"ok": False, "error": _goto_url_state_shortcut_message(goto_url_state_shortcuts)}++ navigation_url_search_setups = _detect_navigation_url_stable_search_setups(workflow_yaml)+ if navigation_url_search_setups:+ _record_navigation_url_stable_search_reject_span("_update_workflow", navigation_url_search_setups)+ return {"ok": False, "error": _navigation_url_stable_search_message(navigation_url_search_setups)}++ stable_search_drips = _detect_click_driven_stable_search_drips(workflow_yaml)+ if stable_search_drips:+ _record_goto_url_stable_search_drip_reject_span("_update_workflow", stable_search_drips)+ return {"ok": False, "error": _goto_url_stable_search_drip_message(stable_search_drips)}try:workflow = _process_workflow_yaml(workflow_id=ctx.workflow_id,@@ -2117,6 +2234,142 @@ _COPILOT_BANNED_BLOCK_ALTERNATIVES = ("`login` for authentication, or `goto_url` for pure URL navigation.")+_GOTO_URL_DYNAMIC_STATE_KEY_TOKENS = frozenset(+ {+ "attr",+ "attribute",+ "category",+ "constraint",+ "criteria",+ "criterion",+ "facet",+ "filter",+ "option",+ "refine",+ "refinement",+ "selected",+ "selection",+ "toggle",+ }+)+_GOTO_URL_DYNAMIC_STATE_VALUE_TOKENS = frozenset(+ {+ "facet",+ "filter",+ "refine",+ "refinement",+ "selected",+ "selection",+ }+)+_GOTO_URL_VERIFICATION_BLOCK_TYPES = frozenset({"code", "navigation", "validation"})+_GOTO_URL_DATA_PRODUCING_BLOCK_TYPES = frozenset({"extraction", "text_prompt"})+_GOTO_URL_STATE_ACTION_TOKENS = frozenset(+ {+ "apply",+ "check",+ "choose",+ "click",+ "confirm",+ "confirmed",+ "ensure",+ "inspect",+ "observe",+ "select",+ "set",+ "verify",+ "verified",+ "verifies",+ "verifying",+ }+)+_GOTO_URL_STATE_EVIDENCE_TOKENS = frozenset(+ {+ "active",+ "aria",+ "checked",+ "checkbox",+ "chip",+ "constraint",+ "control",+ "criteria",+ "criterion",+ "dom",+ "facet",+ "filter",+ "form",+ "input",+ "option",+ "selected",+ "state",+ "url",+ "value",+ }+)+_GOTO_URL_CODE_EVIDENCE_TOKENS = frozenset(+ {+ "all_text_contents",+ "attribute",+ "contents",+ "evaluate",+ "inner_text",+ "locator",+ "query",+ "query_selector",+ "selector",+ "text",+ "text_content",+ }+)+_GOTO_URL_DISMISSAL_TOKENS = frozenset({"banner", "cookie", "cookies", "dismiss", "modal", "popup", "popups"})+_GOTO_URL_SEARCH_SETUP_ACTION_TOKENS = frozenset(+ {+ "choose",+ "enter",+ "fill",+ "input",+ "pick",+ "select",+ "set",+ "type",+ }+)+_GOTO_URL_SEARCH_SETUP_CONTEXT_TOKENS = frozenset(+ {+ "calendar",+ "checkout",+ "checkin",+ "date",+ "dates",+ "find",+ "lookup",+ "query",+ "search",+ }+)+_GOTO_URL_SEARCH_SETUP_TERMINAL_TOKENS = frozenset(+ {+ "active",+ "checked",+ "checkbox",+ "chip",+ "constraint",+ "control",+ "dom",+ "extract",+ "extraction",+ "facet",+ "filter",+ "result",+ "results",+ "verified",+ "verify",+ }+)+_GOTO_URL_STRUCTURAL_STATE_SPLIT_RE = re.compile(r"[;|]")+_GOTO_URL_STRUCTURAL_ASSIGNMENT_LHS_RE = re.compile(r"[A-Za-z0-9_.:-]{1,64}")+_GOTO_URL_VALUE_SCHEME_RE = re.compile(r"^\s*[a-z][a-z0-9+.-]*:", re.IGNORECASE)+def _banned_block_reject_message(items: list[tuple[str, str]]) -> str:"""Uniform error text for the post-emission reject, sharing the@@ -2180,6 +2433,44 @@ def _record_workflow_proxy_location_span(workflow_yaml: str, workflow: Workflow)pass+def _record_goto_url_state_shortcut_reject_span(source_tool: str, items: list[dict[str, Any]]) -> None:+ with copilot_span(+ "goto_url_state_shortcut_reject",+ data={+ "source_tool": source_tool,+ "items": items,+ "labels": [item.get("label") for item in items],+ "dynamic_state_keys": sorted({key for item in items for key in item.get("dynamic_state_keys", [])}),+ },+ ):+ pass+++def _record_goto_url_stable_search_drip_reject_span(source_tool: str, items: list[dict[str, Any]]) -> None:+ with copilot_span(+ "goto_url_stable_search_drip_reject",+ data={+ "source_tool": source_tool,+ "items": items,+ "entry_labels": [item.get("entry_label") for item in items],+ "navigation_labels": [item.get("navigation_labels", []) for item in items],+ },+ ):+ pass+++def _record_navigation_url_stable_search_reject_span(source_tool: str, items: list[dict[str, Any]]) -> None:+ with copilot_span(+ "navigation_url_stable_search_reject",+ data={+ "source_tool": source_tool,+ "items": items,+ "labels": [item.get("label") for item in items],+ },+ ):+ pass++def _collect_banned_block_items(blocks: list[Any]) -> list[tuple[str, str]]:"""Recursively walk ``blocks`` (mirroring:func:`skyvern.forge.sdk.copilot.block_goal_wrapping._wrap_blocks_in_place`)@@ -2224,6 +2515,384 @@ def _parse_workflow_blocks(yaml_str: str | None) -> list[Any] | None:return blocks if isinstance(blocks, list) else None+def _normalized_query_token(value: str) -> str:+ return re.sub(r"[^a-z0-9]+", "_", value.lower()).strip("_")+++def _query_token_parts(value: str) -> set[str]:+ parts: set[str] = set()+ for part in value.split("_"):+ if not part:+ continue+ parts.add(part)+ if len(part) > 3 and part.endswith("s"):+ parts.add(part[:-1])+ return parts+++def _is_dynamic_state_query_key(key: str) -> bool:+ return bool(_query_token_parts(key) & _GOTO_URL_DYNAMIC_STATE_KEY_TOKENS)+++def _is_dynamic_state_query_value(value: str) -> bool:+ return bool(_query_token_parts(value) & _GOTO_URL_DYNAMIC_STATE_VALUE_TOKENS)+++def _is_url_like_query_value(value: str) -> bool:+ stripped = value.strip()+ if not stripped:+ return False+ if stripped.startswith("/") or _GOTO_URL_VALUE_SCHEME_RE.match(stripped):+ return True+ first_equals = stripped.find("=")+ first_query = stripped.find("?")+ return first_query != -1 and (first_equals == -1 or first_query < first_equals)+++def _assignment_lhs(clause: str) -> str | None:+ if "=" not in clause:+ return None+ lhs, rhs = clause.split("=", 1)+ lhs = lhs.strip()+ if not lhs or not rhs.strip():+ return None+ if not _GOTO_URL_STRUCTURAL_ASSIGNMENT_LHS_RE.fullmatch(lhs):+ return None+ return _normalized_query_token(lhs)+++def _is_structural_dynamic_state_query_value(value: str, outer_key: str) -> bool:+ if not value or _is_url_like_query_value(value):+ return False++ clauses = [clause.strip() for clause in _GOTO_URL_STRUCTURAL_STATE_SPLIT_RE.split(value) if clause.strip()]+ if not clauses:+ return False++ assignment_lhses = [lhs for lhs in (_assignment_lhs(clause) for clause in clauses) if lhs is not None]+ if len(assignment_lhses) >= 2:+ return True+ if len(assignment_lhses) == 1 and assignment_lhses[0] != outer_key:+ return True+ return False+++def _dynamic_state_query_keys(url: Any) -> list[str]:+ if not isinstance(url, str) or not url:+ return []+ try:+ pairs = parse_qsl(urlparse(url).query, keep_blank_values=True)+ except ValueError:+ return []+ if not pairs:+ return []++ normalized_pairs = [(_normalized_query_token(key), _normalized_query_token(value), value) for key, value in pairs]+ key_counts = Counter(key for key, _, _ in normalized_pairs if key)+ dynamic_keys: set[str] = set()+ for key, value, raw_value in normalized_pairs:+ if not key:+ continue+ if _is_structural_dynamic_state_query_value(raw_value, key):+ dynamic_keys.add(key)+ elif _is_dynamic_state_query_key(key):+ dynamic_keys.add(key)+ elif key_counts[key] > 1:+ dynamic_keys.add(key)+ elif _is_dynamic_state_query_value(value):+ dynamic_keys.add(key)+ return sorted(dynamic_keys)+++def _block_label_from_yaml(block: dict[str, Any]) -> str | None:+ label = block.get("label")+ return label if isinstance(label, str) and label else None+++def _next_top_level_block(+ *,+ blocks: list[dict[str, Any]],+ by_label: dict[str, dict[str, Any]],+ index_by_label: dict[str, int],+ current: dict[str, Any],+) -> dict[str, Any] | None:+ next_label = current.get("next_block_label")+ if isinstance(next_label, str) and next_label:+ return by_label.get(next_label)++ label = _block_label_from_yaml(current)+ if label is None:+ return None+ index = index_by_label.get(label)+ if index is None:+ return None+ if index + 1 >= len(blocks):+ return None+ return blocks[index + 1]+++def _next_data_block_without_verification(+ *,+ blocks: list[dict[str, Any]],+ by_label: dict[str, dict[str, Any]],+ index_by_label: dict[str, int],+ start: dict[str, Any],+) -> dict[str, Any] | None:+ seen: set[str] = set()+ current = start+ while True:+ next_block = _next_top_level_block(+ blocks=blocks,+ by_label=by_label,+ index_by_label=index_by_label,+ current=current,+ )+ if next_block is None:+ return None+ current = next_block++ label = _block_label_from_yaml(current)+ if label is not None:+ if label in seen:+ return None+ seen.add(label)++ block_type = str(current.get("block_type", "")).strip().lower()+ if _block_verifies_dynamic_goto_url_state(current):+ return None+ if block_type in _GOTO_URL_DATA_PRODUCING_BLOCK_TYPES:+ return current+++def _block_state_text(block: dict[str, Any]) -> str:+ values = []+ for key in (+ "navigation_goal",+ "complete_criterion",+ "terminate_criterion",+ "validation_goal",+ "code",+ "description",+ "title",+ ):+ value = block.get(key)+ if isinstance(value, str):+ values.append(value)+ return _normalized_query_token(" ".join(values))+++def _block_primary_goal_text(block: dict[str, Any]) -> str:+ for key in ("navigation_goal", "validation_goal", "code", "description", "title"):+ value = block.get(key)+ if not isinstance(value, str) or not value:+ continue+ first_fence = value.find("```")+ if first_fence == -1:+ return _normalized_query_token(value)+ second_fence = value.find("```", first_fence + 3)+ if second_fence != -1:+ fenced = value[first_fence + 3 : second_fence].strip()+ if fenced:+ return _normalized_query_token(fenced)+ return _normalized_query_token(value)+ return ""+++def _block_verifies_dynamic_goto_url_state(block: dict[str, Any]) -> bool:+ block_type = str(block.get("block_type", "")).strip().lower()+ if block_type not in _GOTO_URL_VERIFICATION_BLOCK_TYPES:+ return False++ parts = _query_token_parts(_block_state_text(block))+ if not parts:+ return False++ has_state_evidence = bool(parts & _GOTO_URL_STATE_EVIDENCE_TOKENS)+ if block_type == "code":+ return has_state_evidence and bool(parts & _GOTO_URL_CODE_EVIDENCE_TOKENS)+ if block_type == "validation":+ return has_state_evidence+ if parts & _GOTO_URL_DISMISSAL_TOKENS and not parts & {"active", "checked", "selected"}:+ return False+ return has_state_evidence and bool(parts & _GOTO_URL_STATE_ACTION_TOKENS)+++def _is_stable_search_setup_navigation(block: dict[str, Any]) -> bool:+ if str(block.get("block_type", "")).strip().lower() != "navigation":+ return False++ parts = _query_token_parts(_block_primary_goal_text(block))+ if not parts:+ return False+ if parts & _GOTO_URL_SEARCH_SETUP_TERMINAL_TOKENS:+ return False+ return bool(parts & _GOTO_URL_SEARCH_SETUP_ACTION_TOKENS) and bool(parts & _GOTO_URL_SEARCH_SETUP_CONTEXT_TOKENS)+++def _detect_navigation_url_stable_search_setups(workflow_yaml: str | None) -> list[dict[str, Any]]:+ blocks = _parse_workflow_blocks(workflow_yaml)+ if not blocks:+ return []++ findings: list[dict[str, Any]] = []+ for block in blocks:+ if not isinstance(block, dict):+ continue+ if not isinstance(block.get("url"), str) or not block["url"].strip():+ continue+ if not _is_stable_search_setup_navigation(block):+ continue+ findings.append(+ {+ "label": _block_label_from_yaml(block) or "<missing label>",+ "url": block["url"],+ }+ )+ return findings+++def _detect_click_driven_stable_search_drips(workflow_yaml: str | None) -> list[dict[str, Any]]:+ blocks = _parse_workflow_blocks(workflow_yaml)+ if not blocks:+ return []++ top_level_blocks = [block for block in blocks if isinstance(block, dict)]+ findings: list[dict[str, Any]] = []+ entry_label: str | None = None+ stable_nav_labels: list[str] = []++ for block in top_level_blocks:+ block_type = str(block.get("block_type", "")).strip().lower()+ label = _block_label_from_yaml(block) or "<missing label>"++ if block_type == "goto_url":+ entry_label = label+ stable_nav_labels = []+ continue++ if entry_label is None:+ continue++ if block_type in {"code", "validation"}:+ stable_nav_labels = []+ continue++ if block_type in _GOTO_URL_DATA_PRODUCING_BLOCK_TYPES:+ entry_label = None+ stable_nav_labels = []+ continue++ if _is_stable_search_setup_navigation(block):+ stable_nav_labels.append(label)+ if len(stable_nav_labels) >= 2:+ findings.append(+ {+ "entry_label": entry_label,+ "navigation_labels": stable_nav_labels.copy(),+ }+ )+ entry_label = None+ stable_nav_labels = []+ continue++ return findings+++def _detect_unverified_goto_url_state_shortcuts(workflow_yaml: str | None) -> list[dict[str, Any]]:+ blocks = _parse_workflow_blocks(workflow_yaml)+ if not blocks:+ return []++ top_level_blocks = [block for block in blocks if isinstance(block, dict)]+ by_label: dict[str, dict[str, Any]] = {}+ index_by_label: dict[str, int] = {}+ for index, block in enumerate(top_level_blocks):+ label = _block_label_from_yaml(block)+ if label is None:+ continue+ by_label[label] = block+ index_by_label[label] = index++ findings: list[dict[str, Any]] = []+ for block in top_level_blocks:+ if str(block.get("block_type", "")).strip().lower() != "goto_url":+ continue+ dynamic_state_keys = _dynamic_state_query_keys(block.get("url"))+ if not dynamic_state_keys:+ continue+ next_data_block = _next_data_block_without_verification(+ blocks=top_level_blocks,+ by_label=by_label,+ index_by_label=index_by_label,+ start=block,+ )+ if next_data_block is None:+ continue+ findings.append(+ {+ "label": _block_label_from_yaml(block) or "<missing label>",+ "dynamic_state_keys": dynamic_state_keys,+ "next_data_block_label": _block_label_from_yaml(next_data_block) or "<missing label>",+ }+ )+ return findings+++_GOTO_URL_STATE_SHORTCUT_MESSAGE_LIMIT = 5+++def _goto_url_state_shortcut_message(items: list[dict[str, Any]]) -> str:+ details = []+ for item in items[:_GOTO_URL_STATE_SHORTCUT_MESSAGE_LIMIT]:+ label = item.get("label", "?")+ keys = ", ".join(str(key) for key in item.get("dynamic_state_keys", []))+ next_label = item.get("next_data_block_label", "?")+ details.append(f"{label}: query keys [{keys}] feed directly into {next_label}")+ if len(items) > _GOTO_URL_STATE_SHORTCUT_MESSAGE_LIMIT:+ details.append(f"(and {len(items) - _GOTO_URL_STATE_SHORTCUT_MESSAGE_LIMIT} more)")+ return (+ "Workflow validation failed: goto_url may bootstrap stable page state, but URL-encoded dynamic page "+ "state must be verified from the live page before data extraction. Add a code, validation, or navigation block "+ "between the URL shortcut and extraction that inspects active chips, checked controls, search fields, "+ "or DOM/link/form evidence; applies any missing state through UI actions when needed; and verifies all "+ "requested constraints are active together. Offending shortcuts: "+ f"{'; '.join(details)}"+ )+++def _goto_url_stable_search_drip_message(items: list[dict[str, Any]]) -> str:+ details = []+ for item in items[:_GOTO_URL_STATE_SHORTCUT_MESSAGE_LIMIT]:+ entry_label = item.get("entry_label", "?")+ labels = ", ".join(str(label) for label in item.get("navigation_labels", []))+ details.append(f"{entry_label}: click-driven stable search setup blocks [{labels}]")+ if len(items) > _GOTO_URL_STATE_SHORTCUT_MESSAGE_LIMIT:+ details.append(f"(and {len(items) - _GOTO_URL_STATE_SHORTCUT_MESSAGE_LIMIT} more)")+ return (+ "Workflow validation failed: after a goto_url entry page, do not build stateful search setup as one "+ "click-driven navigation block per stable field. Use observed same-site evidence from the live page "+ "(current URL, form action, field names/values, result links, or URL deltas) to bootstrap stable search "+ "state with goto_url, or insert a short code/validation block that inspects the DOM and verifies/navigates "+ "the stable results page. Reserve click-driven navigation for missing dynamic state or when URL/form "+ f"evidence is insufficient. Offending sequence: {'; '.join(details)}"+ )+++def _navigation_url_stable_search_message(items: list[dict[str, Any]]) -> str:+ details = []+ for item in items[:_GOTO_URL_STATE_SHORTCUT_MESSAGE_LIMIT]:+ details.append(f"{item.get('label', '?')}: navigation url={item.get('url', '?')}")+ if len(items) > _GOTO_URL_STATE_SHORTCUT_MESSAGE_LIMIT:+ details.append(f"(and {len(items) - _GOTO_URL_STATE_SHORTCUT_MESSAGE_LIMIT} more)")+ return (+ "Workflow validation failed: a navigation block with a URL is combining entry-page navigation with "+ "stable search setup. Split pure page loading into a goto_url block first. Then use observed same-site "+ "evidence (current URL, form action, field names/values, result links, or URL deltas) to bootstrap "+ "stable search state with goto_url, or add a short code/validation block to inspect the DOM before "+ f"click-driven setup. Offending blocks: {'; '.join(details)}"+ )++def _detect_new_banned_blocks(submitted_yaml: str,prior_workflow_yaml: str | None,@@ -2653,6 +3322,7 @@ def _record_workflow_update_result(copilot_ctx: Any, result: dict[str, Any]) ->wf = result["_workflow"]copilot_ctx.last_workflow = wf+ _clear_resolved_per_tool_budget_problem_labels(copilot_ctx, wf)copilot_ctx.last_workflow_yaml = copilot_ctx.workflow_yaml or Nonecopilot_ctx.effective_workflow_proxy_location = getattr(wf, "proxy_location", None) or ProxyLocation.RESIDENTIALdata = result.get("data")@@ -2877,6 +3547,19 @@ async def update_workflow_tool("""Validate and update the workflow YAML definition.Provide the complete workflow YAML as a string.Returns the validated workflow or validation errors.++ If a `goto_url` URL encodes dynamic page state, do not wire it+ directly to extraction. Add a `code`, `validation`, or `navigation` block+ first that verifies the live page state from active chips, checked controls,+ fields, or DOM/link/form evidence.++ After a `goto_url` entry page for a stateful search, do not add one+ click-driven `navigation` block per stable field (query terms, dates,+ search-form inputs). Use observed current URL / form action / field+ names / result links / URL deltas to bootstrap stable search state with+ `goto_url`, or add a short `code`/`validation` block to inspect the DOM.+ Do not bypass this by combining a URL and stable search setup in a+ `navigation` block; split pure loading into `goto_url` first."""copilot_ctx = ctx.contextarguments = {"workflow_yaml": workflow_yaml}@@ -2972,6 +3655,12 @@ async def run_blocks_tool(the stored `credential_id`. If no stored credential matches, do NOT passthe inline value via `parameters`; stop and follow the CREDENTIALHANDLING refusal rule in the system prompt.++ When a prior `goto_url` used URL params for dynamic page state, the+ blocks you run must include live-page verification before extraction. If+ the visible state is uncertain or only partially applied, inspect it+ through a `code` or `validation` block, or apply missing state through a+ `navigation` block, instead of retrying guessed URL params."""copilot_ctx = ctx.contextarguments = {"block_labels": block_labels, "parameters": parameters or {}}@@ -3029,6 +3718,7 @@ async def get_run_results_tool(return json.dumps({"ok": False, "error": loop_error})result = await _get_run_results(params, copilot_ctx)+ _record_per_tool_budget_problem_blocks_from_results(copilot_ctx, result)_maybe_clear_reconciliation_flag(copilot_ctx, result)record_tool_step_result_for_ctx(copilot_ctx, "get_run_results", params, result)@@ -3061,6 +3751,21 @@ async def update_and_run_blocks_tool(the stored `credential_id`. If no stored credential matches, do NOT passthe inline value via `parameters`; stop and follow the CREDENTIALHANDLING refusal rule in the system prompt.++ Treat `goto_url` as a bootstrap for stateful search/result pages. If URL query+ params encode dynamic page state, include a `code`, `validation`, or+ `navigation` block before extraction that verifies every requested+ constraint is active simultaneously from live page state, and only refine+ URL params when they are grounded in observed DOM/link/form state or+ observed URL deltas.++ After a `goto_url` entry page for a stateful search, do not add one+ click-driven `navigation` block per stable field (query terms, dates,+ search-form inputs). Use observed current URL / form action / field+ names / result links / URL deltas to bootstrap stable search state with+ `goto_url`, or add a short `code`/`validation` block to inspect the DOM.+ Do not bypass this by combining a URL and stable search setup in a+ `navigation` block; split pure loading into `goto_url` first."""copilot_ctx = ctx.contextarguments = {"workflow_yaml": workflow_yaml, "block_labels": block_labels, "parameters": parameters or {}}diff --git a/tests/unit/test_copilot_goto_url_state_policy.py b/tests/unit/test_copilot_goto_url_state_policy.pynew file mode 100644index 000000000..fb6d196b1--- /dev/null+++ b/tests/unit/test_copilot_goto_url_state_policy.py@@ -0,0 +1,420 @@+"""Tests for Copilot's goto_url dynamic-state shortcut policy."""++from __future__ import annotations++from unittest.mock import AsyncMock, MagicMock, patch++import pytest+import yaml++from skyvern.forge.prompts import prompt_engine+from skyvern.forge.sdk.copilot.tools import (+ _detect_click_driven_stable_search_drips,+ _detect_navigation_url_stable_search_setups,+ _detect_unverified_goto_url_state_shortcuts,+ _update_workflow,+ run_blocks_tool,+ update_and_run_blocks_tool,+ update_workflow_tool,+)++_AGENT_TEMPLATE_DEFAULTS = dict(+ workflow_knowledge_base="test kb",+ current_datetime="2026-01-01T00:00:00Z",+ tool_usage_guide="",+ security_rules="",+)+++def _yaml(*blocks: dict) -> str:+ return yaml.safe_dump(+ {"title": "wf", "workflow_definition": {"blocks": list(blocks)}},+ sort_keys=False,+ )+++def _render_agent_prompt() -> str:+ return prompt_engine.load_prompt("workflow-copilot-agent", **_AGENT_TEMPLATE_DEFAULTS)+++class TestGotoUrlStateShortcutDetection:+ def test_plain_goto_url_to_extraction_is_allowed(self) -> None:+ submitted = _yaml(+ {"block_type": "goto_url", "label": "open_home", "url": "https://example.com/news"},+ {"block_type": "extraction", "label": "extract_items", "data_extraction_goal": "Extract items"},+ )++ assert _detect_unverified_goto_url_state_shortcuts(submitted) == []++ def test_stable_search_params_to_extraction_are_allowed(self) -> None:+ submitted = _yaml(+ {+ "block_type": "goto_url",+ "label": "open_results",+ "url": "https://example.com/search?q=budget+headphones&start=2026-08-20&end=2026-08-23",+ },+ {"block_type": "extraction", "label": "extract_results", "data_extraction_goal": "Extract results"},+ )++ assert _detect_unverified_goto_url_state_shortcuts(submitted) == []++ def test_short_state_tokens_do_not_match_unrelated_substrings(self) -> None:+ submitted = _yaml(+ {+ "block_type": "goto_url",+ "label": "open_embedded_report",+ "url": "https://example.com/report?embedded=true&classification=public",+ },+ {"block_type": "extraction", "label": "extract_report", "data_extraction_goal": "Extract report"},+ )++ assert _detect_unverified_goto_url_state_shortcuts(submitted) == []++ def test_dynamic_state_goto_url_directly_to_extraction_is_detected(self) -> None:+ submitted = _yaml(+ {+ "block_type": "goto_url",+ "label": "open_dynamic_results",+ "url": ("https://example.com/search?q=budget+headphones&facet=color&facet=size&filter=available"),+ },+ {"block_type": "extraction", "label": "extract_results", "data_extraction_goal": "Extract results"},+ )++ assert _detect_unverified_goto_url_state_shortcuts(submitted) == [+ {+ "label": "open_dynamic_results",+ "dynamic_state_keys": ["facet", "filter"],+ "next_data_block_label": "extract_results",+ }+ ]++ def test_repeated_opaque_query_key_directly_to_extraction_is_detected(self) -> None:+ submitted = _yaml(+ {+ "block_type": "goto_url",+ "label": "open_dynamic_results",+ "url": "https://example.com/search?q=budget+headphones&choice=wireless&choice=noise-canceling",+ },+ {"block_type": "extraction", "label": "extract_results", "data_extraction_goal": "Extract results"},+ )++ assert _detect_unverified_goto_url_state_shortcuts(submitted) == [+ {+ "label": "open_dynamic_results",+ "dynamic_state_keys": ["choice"],+ "next_data_block_label": "extract_results",+ }+ ]++ def test_dynamic_state_goto_url_with_validation_before_extraction_is_allowed(self) -> None:+ submitted = _yaml(+ {+ "block_type": "goto_url",+ "label": "open_dynamic_results",+ "url": ("https://example.com/search?q=budget+headphones&facet=color&facet=size&filter=available"),+ },+ {+ "block_type": "validation",+ "label": "verify_state",+ "complete_criterion": (+ "Verify query, dates, and each requested result constraint are all active from visible page state."+ ),+ },+ {"block_type": "extraction", "label": "extract_results", "data_extraction_goal": "Extract results"},+ )++ assert _detect_unverified_goto_url_state_shortcuts(submitted) == []++ def test_dynamic_state_goto_url_with_unrelated_navigation_before_extraction_is_detected(self) -> None:+ submitted = _yaml(+ {+ "block_type": "goto_url",+ "label": "open_dynamic_results",+ "url": (+ "https://example.com/search?q=budget+headphones&"+ "state=category%3Delectronics%3Bavailability%3Din_stock"+ ),+ },+ {+ "block_type": "navigation",+ "label": "dismiss_cookie_popup",+ "navigation_goal": "If a cookie popup is visible, close it so the main search form is usable.",+ },+ {"block_type": "extraction", "label": "extract_results", "data_extraction_goal": "Extract results"},+ )++ assert _detect_unverified_goto_url_state_shortcuts(submitted) == [+ {+ "label": "open_dynamic_results",+ "dynamic_state_keys": ["state"],+ "next_data_block_label": "extract_results",+ }+ ]++ def test_dynamic_state_goto_url_with_filter_navigation_before_extraction_is_allowed(self) -> None:+ submitted = _yaml(+ {+ "block_type": "goto_url",+ "label": "open_dynamic_results",+ "url": (+ "https://example.com/search?q=budget+headphones&"+ "state=category%3Delectronics%3Bavailability%3Din_stock"+ ),+ },+ {+ "block_type": "navigation",+ "label": "apply_missing_filters",+ "navigation_goal": (+ "Inspect the active constraint chips and checked controls, then select any missing "+ "requested options."+ ),+ },+ {"block_type": "extraction", "label": "extract_results", "data_extraction_goal": "Extract results"},+ )++ assert _detect_unverified_goto_url_state_shortcuts(submitted) == []++ def test_dynamic_state_goto_url_with_code_before_extraction_is_allowed(self) -> None:+ submitted = _yaml(+ {+ "block_type": "goto_url",+ "label": "open_dynamic_results",+ "url": ("https://example.com/search?q=budget+headphones&facet=color&facet=size&filter=available"),+ },+ {+ "block_type": "code",+ "label": "verify_state_from_dom",+ "code": (+ "checked_labels = await page.locator('[aria-checked=\"true\"], input:checked').all_text_contents()\n"+ "if not checked_labels:\n"+ " raise Exception('No active dynamic page state found')"+ ),+ },+ {"block_type": "extraction", "label": "extract_results", "data_extraction_goal": "Extract results"},+ )++ assert _detect_unverified_goto_url_state_shortcuts(submitted) == []++ def test_dynamic_state_goto_url_without_downstream_extraction_is_allowed(self) -> None:+ submitted = _yaml(+ {+ "block_type": "goto_url",+ "label": "open_dynamic_results",+ "url": "https://example.com/search?q=budget+headphones&facet=color&filter=available",+ }+ )++ assert _detect_unverified_goto_url_state_shortcuts(submitted) == []++ def test_structural_state_value_directly_to_extraction_is_detected(self) -> None:+ submitted = _yaml(+ {+ "block_type": "goto_url",+ "label": "open_dynamic_results",+ "url": (+ "https://example.com/search?q=budget+headphones&"+ "state=category%3Delectronics%3Bavailability%3Din_stock"+ ),+ },+ {"block_type": "extraction", "label": "extract_results", "data_extraction_goal": "Extract results"},+ )++ assert _detect_unverified_goto_url_state_shortcuts(submitted) == [+ {+ "label": "open_dynamic_results",+ "dynamic_state_keys": ["state"],+ "next_data_block_label": "extract_results",+ }+ ]++ def test_single_embedded_assignment_directly_to_extraction_is_detected(self) -> None:+ submitted = _yaml(+ {+ "block_type": "goto_url",+ "label": "open_dynamic_results",+ "url": "https://example.com/search?q=budget+headphones&f=shipping%3Dfree",+ },+ {"block_type": "extraction", "label": "extract_results", "data_extraction_goal": "Extract results"},+ )++ assert _detect_unverified_goto_url_state_shortcuts(submitted) == [+ {+ "label": "open_dynamic_results",+ "dynamic_state_keys": ["f"],+ "next_data_block_label": "extract_results",+ }+ ]++ def test_url_like_query_values_are_not_treated_as_embedded_state_assignments(self) -> None:+ submitted = _yaml(+ {+ "block_type": "goto_url",+ "label": "open_redirect",+ "url": "https://example.com/start?redirect=https%3A%2F%2Fexample.org%2Fsearch%3Fx%3Dy",+ },+ {"block_type": "extraction", "label": "extract_results", "data_extraction_goal": "Extract results"},+ )++ assert _detect_unverified_goto_url_state_shortcuts(submitted) == []+++class TestGotoUrlStableSearchDripDetection:+ def test_navigation_url_with_stable_search_setup_is_detected(self) -> None:+ submitted = _yaml(+ {+ "block_type": "navigation",+ "label": "enter_query",+ "url": "https://example.com/",+ "navigation_goal": (+ "```Dismiss any pop-up if it appears, then enter budget headphones in the product "+ "search field without submitting.```"+ ),+ },+ {"block_type": "extraction", "label": "extract_results", "data_extraction_goal": "Extract results"},+ )++ assert _detect_navigation_url_stable_search_setups(submitted) == [+ {"label": "enter_query", "url": "https://example.com/"}+ ]++ def test_navigation_url_without_stable_search_setup_is_allowed(self) -> None:+ submitted = _yaml(+ {+ "block_type": "navigation",+ "label": "open_article_and_accept_cookies",+ "url": "https://example.com/article",+ "navigation_goal": "```Accept the cookie banner if present.```",+ },+ {"block_type": "extraction", "label": "extract_article", "data_extraction_goal": "Extract article"},+ )++ assert _detect_navigation_url_stable_search_setups(submitted) == []++ def test_multiple_click_driven_stable_search_navigation_blocks_are_detected(self) -> None:+ submitted = _yaml(+ {"block_type": "goto_url", "label": "open_search_site", "url": "https://example.com/"},+ {+ "block_type": "navigation",+ "label": "enter_query",+ "navigation_goal": (+ "Achieve the following mini goal and once it's achieved, complete: "+ "```Enter budget headphones in the product search field. Do not submit the search yet.```"+ ),+ },+ {+ "block_type": "navigation",+ "label": "set_dates",+ "navigation_goal": (+ "Achieve the following mini goal and once it's achieved, complete: "+ "```Select July 10, 2026 as the start date and July 12, 2026 as the end date.```"+ ),+ },+ {"block_type": "extraction", "label": "extract_results", "data_extraction_goal": "Extract results"},+ )++ assert _detect_click_driven_stable_search_drips(submitted) == [+ {+ "entry_label": "open_search_site",+ "navigation_labels": ["enter_query", "set_dates"],+ }+ ]++ def test_stable_search_navigation_after_dom_inspection_is_allowed(self) -> None:+ submitted = _yaml(+ {"block_type": "goto_url", "label": "open_search_site", "url": "https://example.com/"},+ {+ "block_type": "navigation",+ "label": "enter_query",+ "navigation_goal": "```Enter budget headphones in the product search field.```",+ },+ {+ "block_type": "code",+ "label": "inspect_search_form",+ "code": "form_action = await page.locator('form').first.get_attribute('action')",+ },+ {+ "block_type": "navigation",+ "label": "set_dates",+ "navigation_goal": "```Select July 10, 2026 as the start date and July 12, 2026 as the end date.```",+ },+ {"block_type": "extraction", "label": "extract_results", "data_extraction_goal": "Extract results"},+ )++ assert _detect_click_driven_stable_search_drips(submitted) == []++ def test_single_stable_search_navigation_before_submit_is_allowed(self) -> None:+ submitted = _yaml(+ {"block_type": "goto_url", "label": "open_search_site", "url": "https://example.com/"},+ {+ "block_type": "navigation",+ "label": "enter_query",+ "navigation_goal": "```Enter budget headphones in the product search field.```",+ },+ {+ "block_type": "navigation",+ "label": "submit_search",+ "navigation_goal": "```Click the Search button to submit and show results.```",+ },+ {"block_type": "extraction", "label": "extract_results", "data_extraction_goal": "Extract results"},+ )++ assert _detect_click_driven_stable_search_drips(submitted) == []+++def _ctx(prior_yaml: str | None = None) -> MagicMock:+ ctx = MagicMock()+ ctx.workflow_yaml = prior_yaml+ ctx.workflow_id = "w_test"+ ctx.workflow_permanent_id = "wpid_test"+ ctx.organization_id = "o_test"+ return ctx+++async def test_update_workflow_rejects_unverified_goto_url_state_shortcut_and_emits_span() -> None:+ submitted = _yaml(+ {+ "block_type": "goto_url",+ "label": "open_dynamic_results",+ "url": "https://example.com/search?q=budget+headphones&facet=color&facet=size&filter=available",+ },+ {"block_type": "extraction", "label": "extract_results", "data_extraction_goal": "Extract results"},+ )+ ctx = _ctx()++ with (+ patch("skyvern.forge.sdk.copilot.tools._record_goto_url_state_shortcut_reject_span") as mock_span,+ patch("skyvern.forge.sdk.copilot.tools.app") as mock_app,+ ):+ mock_app.WORKFLOW_SERVICE.update_workflow_definition = AsyncMock()+ result = await _update_workflow({"workflow_yaml": submitted}, ctx)++ assert result["ok"] is False+ assert "goto_url may bootstrap stable page state" in result["error"]+ assert "URL-encoded dynamic page state must be verified" in result["error"]+ assert "open_dynamic_results" in result["error"]+ assert "extract_results" in result["error"]+ mock_app.WORKFLOW_SERVICE.update_workflow_definition.assert_not_called()+ mock_span.assert_called_once()+++class TestPromptAndToolDescriptions:+ def test_agent_prompt_contains_goto_url_state_policy(self) -> None:+ rendered = _render_agent_prompt()++ assert "GOTO_URL STATE SHORTCUT POLICY" in rendered+ assert "URL refinement for dynamic page state is allowed ONLY when grounded in live site evidence" in rendered+ assert "Before extraction on stateful search/result tasks" in rendered+ assert "do not spend one click-driven `navigation` block per field after the entry page loads" in rendered+ assert "current URL, form action, field names/values, result links, or URL deltas" in rendered+ assert "use `code` for DOM checks" in rendered+ assert "per-tool budget" in rendered+ assert "constraint / expected / observed / source" in rendered++ def test_tool_descriptions_warn_against_direct_state_shortcut_extraction(self) -> None:+ for tool in (update_workflow_tool, run_blocks_tool, update_and_run_blocks_tool):+ desc = tool.description # type: ignore[attr-defined]+ assert "goto_url" in desc+ assert "dynamic page state" in desc+ assert "code" in desc+ assert "validation" in descdiff --git a/tests/unit/test_copilot_hooks.py b/tests/unit/test_copilot_hooks.pyindex 5466b1209..e99c81d28 100644--- a/tests/unit/test_copilot_hooks.py+++ b/tests/unit/test_copilot_hooks.py@@ -219,7 +219,7 @@ class TestSchemaOverlay:arg_transforms={"clear_first": "clear"},forced_args={"inline": True},)- args = {"clear_first": True, "text": "hello"}+ args = {"clear_first": True, "text": "hello", "_summarized": "older tool call"}result = _transform_args(args, overlay)assert result == {"clear": True, "text": "hello", "inline": True}assert "clear_first" not in resultdiff --git a/tests/unit/test_copilot_per_tool_budget.py b/tests/unit/test_copilot_per_tool_budget.pyindex 1691389e4..16fcafa9f 100644--- a/tests/unit/test_copilot_per_tool_budget.py+++ b/tests/unit/test_copilot_per_tool_budget.py@@ -33,7 +33,13 @@ from skyvern.forge.sdk.copilot.failure_tracking import (PER_TOOL_BUDGET_FAILURE_CATEGORY,compute_failure_signature,)-from skyvern.forge.sdk.copilot.tools import _maybe_clear_reconciliation_flag, _record_run_blocks_result+from skyvern.forge.sdk.copilot.tools import (+ _maybe_clear_reconciliation_flag,+ _record_per_tool_budget_problem_blocks_from_results,+ _record_run_blocks_result,+ _record_workflow_update_result,+ _tool_loop_error,+)def _fresh_context() -> CopilotContext:@@ -283,6 +289,126 @@ def test_reconciliation_does_not_clear_when_run_id_mismatches() -> None:assert ctx.pending_reconciliation_run_id == "wr_1"+def test_get_run_results_arms_problem_navigation_label_for_budget_run() -> None:+ ctx = _fresh_context()+ ctx.pending_reconciliation_run_id = "wr_1"+ ctx.last_failure_category_top = PER_TOOL_BUDGET_FAILURE_CATEGORY++ _record_per_tool_budget_problem_blocks_from_results(+ ctx,+ {+ "ok": True,+ "data": {+ "workflow_run_id": "wr_1",+ "overall_status": "canceled",+ "blocks": [+ {"label": "open_results", "block_type": "GOTO_URL", "status": "completed"},+ {"label": "apply_filters", "block_type": "NAVIGATION", "status": "canceled"},+ {"label": "extract_results", "block_type": "EXTRACTION", "status": "created"},+ ],+ },+ },+ )++ assert ctx.per_tool_budget_problem_block_labels == ["apply_filters"]+++def test_get_run_results_does_not_arm_problem_label_for_different_run() -> None:+ ctx = _fresh_context()+ ctx.pending_reconciliation_run_id = "wr_1"+ ctx.last_failure_category_top = PER_TOOL_BUDGET_FAILURE_CATEGORY++ _record_per_tool_budget_problem_blocks_from_results(+ ctx,+ {+ "ok": True,+ "data": {+ "workflow_run_id": "wr_other",+ "overall_status": "canceled",+ "blocks": [{"label": "apply_filters", "block_type": "NAVIGATION", "status": "canceled"}],+ },+ },+ )++ assert ctx.per_tool_budget_problem_block_labels == []+++def test_tool_loop_error_blocks_rerun_of_problem_navigation_label() -> None:+ ctx = _fresh_context()+ ctx.per_tool_budget_problem_block_labels = ["apply_filters"]++ msg = _tool_loop_error(+ ctx,+ "run_blocks_and_collect_debug",+ {"block_labels": ["apply_filters"], "parameters": {}},+ )++ assert msg is not None+ assert "apply_filters" in msg+ assert "Do NOT rerun" in msg+ assert "code or validation" in msg+++def test_tool_loop_error_treats_missing_labels_as_rerun_all() -> None:+ ctx = _fresh_context()+ ctx.per_tool_budget_problem_block_labels = ["apply_filters"]++ msg = _tool_loop_error(ctx, "run_blocks_and_collect_debug", {"parameters": {}})++ assert msg is not None+ assert "apply_filters" in msg+++def test_tool_loop_error_allows_new_smaller_label_after_budget_problem() -> None:+ ctx = _fresh_context()+ ctx.per_tool_budget_problem_block_labels = ["apply_filters"]++ assert (+ _tool_loop_error(+ ctx,+ "run_blocks_and_collect_debug",+ {"block_labels": ["click_parking"], "parameters": {}},+ )+ is None+ )+++def test_successful_workflow_update_keeps_problem_label_while_still_navigation() -> None:+ ctx = _fresh_context()+ ctx.workflow_yaml = "updated yaml"+ ctx.per_tool_budget_problem_block_labels = ["apply_filters"]+ workflow = SimpleNamespace(+ workflow_definition=SimpleNamespace(+ blocks=[+ SimpleNamespace(block_type="navigation", label="apply_filters"),+ SimpleNamespace(block_type="extraction", label="extract_results"),+ ]+ )+ )++ _record_workflow_update_result(ctx, {"ok": True, "_workflow": workflow, "data": {"block_count": 2}})++ assert ctx.per_tool_budget_problem_block_labels == ["apply_filters"]+++def test_successful_workflow_update_clears_problem_label_changed_away_from_navigation() -> None:+ ctx = _fresh_context()+ ctx.workflow_yaml = "updated yaml"+ ctx.per_tool_budget_problem_block_labels = ["apply_filters"]+ workflow = SimpleNamespace(+ workflow_definition=SimpleNamespace(+ blocks=[+ SimpleNamespace(block_type="code", label="apply_filters"),+ SimpleNamespace(block_type="navigation", label="click_parking"),+ ]+ )+ )++ _record_workflow_update_result(ctx, {"ok": True, "_workflow": workflow, "data": {"block_count": 2}})++ assert ctx.per_tool_budget_problem_block_labels == []++def test_nudge_counter_resets_on_success() -> None:"""The counter must reset so a budget trip on a different chain in thesame turn gets the dedicated split nudge again."""