Skip to content

Automate browser based workflows with AI

fix: replace failed selectors in reviewer + add pipeline tracing to code-run-analyzer (#5381)

pedrohsdb5d23d0adc44b
diff --git a/skyvern/forge/prompts/skyvern/script-reviewer.j2 b/skyvern/forge/prompts/skyvern/script-reviewer.j2
index 6d8f0254f..dc743e2c4 100644
--- a/skyvern/forge/prompts/skyvern/script-reviewer.j2
+++ b/skyvern/forge/prompts/skyvern/script-reviewer.j2
@@ -8,7 +8,12 @@ You are a script reviewer for a browser automation system. Your job is to update
5. Add `text_patterns` derived from the page text in the episode data below. **NEVER include PII in text_patterns** — no email addresses, no personal names, no account numbers, no customer-specific identifiers. Use generic page structure indicators instead (form labels, button text, section headings, navigation links). Wrong: `"Welcome John Smith, [email protected], Billing & Payments"`. Right: `"Welcome, Billing & Payments, Sign in, Logout"`.
6. Keep classify option descriptions short (< 10 words) and mutually exclusive.
7. **SEMANTIC SELECTORS**: Use `ai='fallback'` with label-based selectors — NEVER copy xpaths from other branches. Build selectors using `:has-text()` (case-insensitive substring match). Examples: `selector='label:has-text("Full name") input'`, `selector='button:has-text("Submit")'`. If no semantic selector is possible (complex widgets), use `ai='fallback'` with only a `prompt=`.
-8. **ACCUMULATE SELECTORS**: NEVER remove a working selector. When the same field has different labels across variants (e.g. "Website" vs "URL"), use comma-separated CSS selectors: `selector='label:has-text("Website") input, label:has-text("URL") input'`. The first match wins.
+8. **SELECTOR MANAGEMENT**:
+ a. **Accumulate across variants**: When different page variants use different labels for the same field, use comma-separated CSS selectors: `selector='label:has-text("Website") input, label:has-text("URL") input'`. The first match wins.
+ b. **Replace failed selectors**: When an episode reports a `failed_selector` (the selector that was tried and did NOT match), REMOVE that specific selector from the list and replace it with a working alternative derived from the episode's `agent_actions` data (look for `css_suggestion`, `all_attributes`, `element_id`, `element_tag`). Do NOT keep selectors that are known to fail — they add timeout delays on every run.
+ c. **Broaden element types**: If a `button:has-text("X")` selector fails, the element may not be a `<button>`. Broaden to include other clickable elements: `button:has-text("X"), input[type="submit"], a:has-text("X"), [role="button"]:has-text("X")`.
+ d. When the episode has NO `agent_actions` data (only an error message like "AI click failed"), derive the selector from the **Page Text at failure** section — look for the button/link text mentioned in the error and build a broader selector.
+ e. **Same selector succeeds in agent but fails in script**: If the agent actions show the SAME selector succeeding that the script failed on, consider what might be different. Look at the agent actions BEFORE the successful one — did the agent perform additional steps first (dismissing a popup, waiting for a page transition, filling a prerequisite field)? If so, add those steps to the script before the failing action. If no additional steps are apparent, it may be a timing/page-load issue — try adding `await page.wait(seconds=3)` before the action to give the page time to render.
9. **PARAMETER NAMES — CRITICAL**:
a. The **Workflow Parameter Keys** section below lists EVERY valid parameter name. `context.parameters['key']` is ONLY allowed when `key` appears in that list.
b. **NEVER INVENT parameter names.** If a form field does not have a matching parameter key, do NOT create a `context.parameters['made_up_name']` reference. This will crash at runtime with a KeyError.
@@ -106,6 +111,17 @@ The cached code failed and the AI agent took over. Here is what happened:
{%- if a.all_attributes %} [attrs: {{ a.all_attributes | tojson }}]{% endif %}
{%- if not a.all_attributes and a.element_attributes %} [attrs: {{ a.element_attributes | tojson }}]{% endif %}
{% endfor %}
+{% elif episode.agent_actions is mapping and episode.agent_actions.failed_selector is defined %}
+ **Element-level fallback** — the selector failed but AI found the element:
+ - Failed selector: `{{ episode.agent_actions.failed_selector }}`
+ - Action: {{ episode.agent_actions.action_type or "click" }}: {{ episode.agent_actions.intention or "N/A" }}
+{%- if episode.agent_actions.element_tag %} [tag: {{ episode.agent_actions.element_tag }}]{% endif %}
+{%- if episode.agent_actions.element_text %} [text: "{{ episode.agent_actions.element_text }}"]{% endif %}
+{%- if episode.agent_actions.css_suggestion %} **[USE THIS SELECTOR: `{{ episode.agent_actions.css_suggestion }}`]**{% endif %}
+{%- if episode.agent_actions.all_attributes %} [attrs: {{ episode.agent_actions.all_attributes | tojson }}]{% endif %}
+{%- if episode.agent_actions.reasoning %}
+ - Reasoning: {{ episode.agent_actions.reasoning }}{% endif %}
+ **ACTION REQUIRED**: Remove `{{ episode.agent_actions.failed_selector }}` from the selector list and replace with a stable alternative using the attributes above.
{% elif episode.agent_actions is mapping and episode.agent_actions.navigation_goal is defined %}
Surgical fallback: "{{ episode.agent_actions.navigation_goal }}" — {{ episode.agent_actions.steps_taken }} steps, completed={{ episode.agent_actions.completed }}
{% elif episode.agent_actions %}
diff --git a/skyvern/forge/sdk/routes/scripts.py b/skyvern/forge/sdk/routes/scripts.py
index 77ad9454c..a6afed7e9 100644
--- a/skyvern/forge/sdk/routes/scripts.py
+++ b/skyvern/forge/sdk/routes/scripts.py
@@ -42,7 +42,10 @@ from skyvern.schemas.scripts import (
)
from skyvern.services import script_service, workflow_script_service
from skyvern.services.script_reviewer import ScriptReviewer, store_review_artifacts
-from skyvern.services.workflow_script_service import create_script_version_from_review
+from skyvern.services.workflow_script_service import (
+ create_script_version_from_review,
+ extract_cached_blocks_from_source,
+)
LOG = structlog.get_logger()
@@ -102,18 +105,33 @@ async def get_script_blocks_response(
return ScriptBlocksResponse(blocks={}, main_script=main_script, script_id=script_id, version=version)
result: dict[str, str] = {}
+ main_py_block_codes: dict[str, str] | None = None
# TODO(jdo): make concurrent to speed up
for script_block in script_blocks:
script_file_id = script_block.script_file_id
if not script_file_id:
+ # Reviewer-created blocks have no script_file_id — fall back to
+ # extracting the block code from main.py (lazy-loaded once).
+ block_label = script_block.script_block_label
+ if main_py_block_codes is None:
+ content = await _load_main_script_content(
+ organization_id=organization_id,
+ script_revision_id=script_revision_id,
+ )
+ main_py_block_codes = extract_cached_blocks_from_source(content) if content else {}
+
+ if block_label in main_py_block_codes:
+ result[block_label] = main_py_block_codes[block_label]
+ continue
+
LOG.info(
- "No script file ID found for script block",
+ "No script file ID found for script block and block not in main.py",
workflow_permanent_id=workflow_permanent_id,
organization_id=organization_id,
script_revision_id=script_revision_id,
- block_label=script_block.script_block_label,
+ block_label=block_label,
)
continue
diff --git a/skyvern/services/script_reviewer.py b/skyvern/services/script_reviewer.py
index 3caf933fe..4d1e2c656 100644
--- a/skyvern/services/script_reviewer.py
+++ b/skyvern/services/script_reviewer.py
@@ -1091,40 +1091,36 @@ class ScriptReviewer:
script_revision_id: str | None,
block_label: str,
) -> str | None:
- """Load the current cached code for a block from the database."""
+ """Load the current cached code for a block by extracting it from main.py.
+
+ main.py is the single source of truth for all cached block functions.
+ Each block is identified by its @skyvern.cached(cache_key='<label>') decorator.
+ """
if not script_revision_id:
return None
+ return await self._extract_block_from_main_py(
+ organization_id=organization_id,
+ script_revision_id=script_revision_id,
+ block_label=block_label,
+ )
+
+ async def _extract_block_from_main_py(
+ self,
+ organization_id: str,
+ script_revision_id: str,
+ block_label: str,
+ ) -> str | None:
+ """Extract a block function from main.py when no separate block file exists."""
+ from skyvern.services.workflow_script_service import extract_single_cached_block, load_main_py_content
+
try:
- script_blocks = await app.DATABASE.scripts.get_script_blocks_by_script_revision_id(
- script_revision_id=script_revision_id,
- organization_id=organization_id,
- )
- for sb in script_blocks:
- if sb.script_block_label == block_label and sb.script_file_id:
- # Load the code from the script file
- script_file = await app.DATABASE.scripts.get_script_file_by_id(
- script_revision_id=script_revision_id,
- file_id=sb.script_file_id,
- organization_id=organization_id,
- )
- if script_file and script_file.artifact_id:
- artifact = await app.DATABASE.artifacts.get_artifact_by_id(
- artifact_id=script_file.artifact_id,
- organization_id=organization_id,
- )
- if artifact:
- file_content = await app.ARTIFACT_MANAGER.retrieve_artifact(artifact)
- if isinstance(file_content, bytes):
- return file_content.decode("utf-8")
- elif isinstance(file_content, str):
- return file_content
+ content = await load_main_py_content(script_revision_id, organization_id)
+ if not content:
+ return None
+ return extract_single_cached_block(content, block_label)
except Exception:
- LOG.exception(
- "ScriptReviewer: failed to load block code",
- block_label=block_label,
- script_revision_id=script_revision_id,
- )
+ LOG.warning("Failed to extract block from main.py", block_label=block_label, exc_info=True)
return None
async def _load_all_block_codes(
@@ -1132,41 +1128,22 @@ class ScriptReviewer:
organization_id: str,
script_revision_id: str | None,
) -> dict[str, str]:
- """Load code for all blocks in a script revision.
+ """Load code for all blocks by extracting from main.py.
- Returns {block_label: code} for blocks that have code.
+ Returns {block_label: code} for all @skyvern.cached functions in main.py.
"""
+ from skyvern.services.workflow_script_service import extract_cached_blocks_from_source, load_main_py_content
+
if not script_revision_id:
return {}
try:
- script_blocks = await app.DATABASE.scripts.get_script_blocks_by_script_revision_id(
- script_revision_id=script_revision_id,
- organization_id=organization_id,
- )
- result: dict[str, str] = {}
- for sb in script_blocks:
- if not sb.script_file_id:
- continue
- script_file = await app.DATABASE.scripts.get_script_file_by_id(
- script_revision_id=script_revision_id,
- file_id=sb.script_file_id,
- organization_id=organization_id,
- )
- if script_file and script_file.artifact_id:
- artifact = await app.DATABASE.artifacts.get_artifact_by_id(
- artifact_id=script_file.artifact_id,
- organization_id=organization_id,
- )
- if artifact:
- file_content = await app.ARTIFACT_MANAGER.retrieve_artifact(artifact)
- if isinstance(file_content, bytes):
- result[sb.script_block_label] = file_content.decode("utf-8")
- elif isinstance(file_content, str):
- result[sb.script_block_label] = file_content
- return result
+ content = await load_main_py_content(script_revision_id, organization_id)
+ if not content:
+ return {}
+ return extract_cached_blocks_from_source(content)
except Exception:
LOG.exception(
- "ScriptReviewer: failed to load all block codes",
+ "ScriptReviewer: failed to load all block codes from main.py",
script_revision_id=script_revision_id,
)
return {}
@@ -2149,11 +2126,6 @@ class ScriptReviewer:
# Regex for email addresses in text_patterns (PII that should not be in cached scripts)
_EMAIL_RE: re.Pattern[str] = re.compile(r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}")
- # Regex to extract text_patterns dict values from page.classify() calls.
- # Matches: text_patterns={"key": "value with PII", ...}
- # Uses a simple approach: find text_patterns={, then extract all string values.
- _TEXT_PATTERN_VALUE_RE: re.Pattern[str] = re.compile(r"""["']([^"']{10,})["']""")
-
# Regex for :has-text("X") where X is very short (1-2 chars) — likely hardcoded run data
_SHORT_HAS_TEXT_RE: re.Pattern[str] = re.compile(r""":has-text\(\s*['"](.{1,2})['"]\s*\)""")
diff --git a/skyvern/services/workflow_script_service.py b/skyvern/services/workflow_script_service.py
index 3ccf76079..1ad33f6fd 100644
--- a/skyvern/services/workflow_script_service.py
+++ b/skyvern/services/workflow_script_service.py
@@ -26,6 +26,76 @@ from skyvern.services import script_service
LOG = structlog.get_logger()
jinja_sandbox_env = SandboxedEnvironment()
+# Shared regex for parsing @skyvern.cached decorator lines in main.py source.
+_CACHED_DECORATOR_RE = re.compile(
+ r"^@skyvern\.cached\(\s*cache_key\s*=\s*['\"]([^'\"]+)['\"]\s*\)",
+ re.MULTILINE,
+)
+
+
+def extract_cached_blocks_from_source(content: str) -> dict[str, str]:
+ """Parse all @skyvern.cached blocks from Python source.
+
+ Returns {block_label: code} for every block found. Each value includes
+ the decorator line through to the start of the next decorator (or EOF).
+ """
+ matches = list(_CACHED_DECORATOR_RE.finditer(content))
+ result: dict[str, str] = {}
+ for idx, match in enumerate(matches):
+ label = match.group(1)
+ start = match.start()
+ end = matches[idx + 1].start() if idx + 1 < len(matches) else len(content)
+ block_code = content[start:end].rstrip()
+ if block_code:
+ result[label] = block_code
+ return result
+
+
+def extract_single_cached_block(content: str, block_label: str) -> str | None:
+ """Extract a single @skyvern.cached block by label from Python source."""
+ pattern = re.compile(
+ rf"^@skyvern\.cached\(\s*cache_key\s*=\s*['\"]{re.escape(block_label)}['\"]\s*\)",
+ re.MULTILINE,
+ )
+ match = pattern.search(content)
+ if not match:
+ return None
+ start = match.start()
+ next_decorator = _CACHED_DECORATOR_RE.search(content[match.end() :])
+ end = match.end() + next_decorator.start() if next_decorator else len(content)
+ block_code = content[start:end].rstrip()
+ return block_code if block_code else None
+
+
+async def load_main_py_content(
+ script_revision_id: str,
+ organization_id: str,
+) -> str | None:
+ """Load and decode main.py content from S3 for a script revision."""
+ try:
+ script_files = await app.DATABASE.scripts.get_script_files(
+ script_revision_id=script_revision_id,
+ organization_id=organization_id,
+ )
+ for f in script_files:
+ if f.file_path == "main.py" and f.artifact_id:
+ artifact = await app.DATABASE.artifacts.get_artifact_by_id(f.artifact_id, organization_id)
+ if not artifact:
+ return None
+ raw_content = await app.ARTIFACT_MANAGER.retrieve_artifact(artifact)
+ if isinstance(raw_content, bytes):
+ return raw_content.decode("utf-8")
+ elif isinstance(raw_content, str):
+ return raw_content
+ return None
+ except Exception:
+ LOG.warning(
+ "Failed to load main.py content",
+ script_revision_id=script_revision_id,
+ exc_info=True,
+ )
+ return None
+
def _jinja_domain_filter(url: str) -> str:
"""Extract the domain (netloc) from a URL for use as a cache key grouping.
@@ -358,12 +428,31 @@ async def get_latest_published_script(
return best
+async def _extract_all_blocks_from_main_py(
+ script_revision_id: str,
+ organization_id: str,
+) -> dict[str, str]:
+ """Extract all @skyvern.cached block functions from main.py for a script revision.
+
+ Returns {block_label: code} for every block found. Used as a fallback when blocks
+ don't have individual block files (e.g. reviewer-created revisions).
+ """
+ content = await load_main_py_content(script_revision_id, organization_id)
+ if not content:
+ return {}
+ return extract_cached_blocks_from_source(content)
+
+
async def _load_cached_script_block_sources(
script: Script,
organization_id: str,
) -> dict[str, ScriptBlockSource]:
"""
Load existing script block sources (code + metadata) for a script revision so they can be reused.
+
+ Blocks may have code stored as individual block files (script_file_id) or only in main.py
+ (reviewer-created revisions store code exclusively in main.py). This function tries
+ the block file first, then falls back to extracting the block from main.py.
"""
cached_blocks: dict[str, ScriptBlockSource] = {}
@@ -372,6 +461,10 @@ async def _load_cached_script_block_sources(
organization_id=organization_id,
)
+ # Lazily loaded: block codes extracted from main.py, used as fallback when
+ # a block has no script_file_id (i.e. reviewer-created revisions).
+ main_py_block_codes: dict[str, str] | None = None
+
for script_block in script_blocks:
if not script_block.script_block_label:
continue
@@ -392,6 +485,16 @@ async def _load_cached_script_block_sources(
elif isinstance(file_content, str):
code_str = file_content
+ # Fallback: extract block code from main.py (reviewer-created revisions
+ # store all block code in main.py and don't create individual block files).
+ if not code_str:
+ if main_py_block_codes is None:
+ main_py_block_codes = await _extract_all_blocks_from_main_py(
+ script_revision_id=script.script_revision_id,
+ organization_id=organization_id,
+ )
+ code_str = main_py_block_codes.get(script_block.script_block_label)
+
if not code_str:
continue
@@ -988,26 +1091,22 @@ def _patch_main_py(main_py_content: str, updated_blocks: dict[str, str]) -> str:
async def _reconstruct_main_py_from_blocks(
base_main_py: str,
- new_revision_id: str,
- organization_id: str,
updated_blocks: dict[str, str],
) -> str | None:
- """Reconstruct main.py from the header + individual block files when _patch_main_py fails.
+ """Reconstruct main.py when _patch_main_py fails.
- Takes the header (imports, models, workflow function) from the base main.py
- and appends every block's code — both newly reviewed blocks and existing ones
- carried forward from the base. Each block file was individually compile-verified,
- so concatenation should produce valid Python.
+ Takes the header (imports, models, workflow function) from the base main.py,
+ then appends each block: updated blocks from the reviewer, plus existing
+ blocks extracted from the base main.py.
Returns the reconstructed source, or None if reconstruction fails.
"""
try:
# Extract the header: everything before the first @skyvern.cached decorator
lines = base_main_py.split("\n")
- decorator_pattern = re.compile(r"^@skyvern\.cached\(\s*cache_key\s*=\s*['\"]([^'\"]+)['\"]\s*\)")
header_end = len(lines)
for i, line in enumerate(lines):
- if decorator_pattern.match(line.strip()):
+ if _CACHED_DECORATOR_RE.match(line.strip()):
header_end = i
break
# Strip trailing blank lines from header
@@ -1015,36 +1114,12 @@ async def _reconstruct_main_py_from_blocks(
header_end -= 1
header = "\n".join(lines[:header_end])
- # Collect all block code: updated blocks from the reviewer, plus existing
- # blocks from the new revision's block files (which were either freshly
- # uploaded or copied from the base revision)
- block_codes: dict[str, str] = {}
+ # Extract all existing blocks from the base main.py
+ block_codes = extract_cached_blocks_from_source(base_main_py)
- # Load all block files for this revision from the DB
- block_files = await app.DATABASE.scripts.get_script_files(
- script_revision_id=new_revision_id,
- organization_id=organization_id,
- )
- for f in block_files:
- if not f.file_path.startswith("blocks/") or not f.artifact_id:
- continue
- # Extract label from path: "blocks/login.py" -> "login"
- label = f.file_path.removeprefix("blocks/").removesuffix(".py").removesuffix(".skyvern")
- if label in updated_blocks:
- # Prefer the reviewer's version (it's compile-verified)
- block_codes[label] = updated_blocks[label]
- else:
- # Load existing block content from S3
- artifact = await app.DATABASE.artifacts.get_artifact_by_id(f.artifact_id, organization_id)
- if artifact:
- content = await app.ARTIFACT_MANAGER.retrieve_artifact(artifact)
- if content:
- block_codes[label] = content.decode("utf-8") if isinstance(content, bytes) else content
-
- # Also include any updated blocks that don't have block files yet (new blocks)
+ # Override with updated blocks from the reviewer
for label, code in updated_blocks.items():
- if label not in block_codes:
- block_codes[label] = code
+ block_codes[label] = code
if not block_codes:
LOG.warning("No block code found for reconstruction")
@@ -1073,7 +1148,7 @@ async def _reconstruct_main_py_from_blocks(
# Final compile check
compile(reconstructed, "<reconstructed_main.py>", "exec")
LOG.info(
- "Successfully reconstructed main.py from block files",
+ "Successfully reconstructed main.py from base source",
block_count=len(block_codes),
block_labels=sorted(block_codes.keys()),
)
@@ -1288,70 +1363,29 @@ async def create_script_version_from_review(
conditional_blocks = conditional_blocks or {}
for sb in existing_blocks:
- if sb.script_block_label in updated_blocks:
- # This block has an updated version from the reviewer
- updated_code = updated_blocks[sb.script_block_label]
- content_bytes = updated_code.encode("utf-8")
- content_hash = hashlib.sha256(content_bytes).hexdigest()
- file_path = f"blocks/{sb.script_block_label}.py"
-
- # Upload code to S3 and create script file DB record
- artifact_id = await app.ARTIFACT_MANAGER.create_script_file_artifact(
- organization_id=organization_id,
- script_id=new_script.script_id,
- script_version=new_script.version,
- file_path=file_path,
- data=content_bytes,
- )
- new_file = await app.DATABASE.scripts.create_script_file(
- script_revision_id=new_script.script_revision_id,
- script_id=new_script.script_id,
- organization_id=organization_id,
- file_path=file_path,
- file_name=f"{sb.script_block_label}.py",
- file_type="file",
- content_hash=f"sha256:{content_hash}",
- file_size=len(content_bytes),
- mime_type="text/x-python",
- artifact_id=artifact_id,
- )
-
- # Determine if this is a conditional block being upgraded to code.
- # If so, flip requires_agent to False and set the run_signature.
- is_conditional_upgrade = sb.script_block_label in conditional_blocks
- block_requires_agent = False if is_conditional_upgrade else sb.requires_agent
- block_run_signature = (
- f'await skyvern.conditional(label="{sb.script_block_label}")'
- if is_conditional_upgrade
- else sb.run_signature
- )
+ # Determine if this is a conditional block being upgraded to code.
+ is_conditional_upgrade = sb.script_block_label in conditional_blocks
+ block_requires_agent = False if is_conditional_upgrade else sb.requires_agent
+ block_run_signature = (
+ f'await skyvern.conditional(label="{sb.script_block_label}")'
+ if is_conditional_upgrade
+ else sb.run_signature
+ )
- # Create script block entry pointing to the new file
- await app.DATABASE.scripts.create_script_block(
- organization_id=organization_id,
- script_id=new_script.script_id,
- script_revision_id=new_script.script_revision_id,
- script_block_label=sb.script_block_label,
- script_file_id=new_file.file_id,
- run_signature=block_run_signature,
- workflow_run_id=workflow_run.workflow_run_id if workflow_run else None,
- input_fields=sb.input_fields,
- requires_agent=block_requires_agent,
- )
- else:
- # Copy existing block as-is
- await app.DATABASE.scripts.create_script_block(
- organization_id=organization_id,
- script_id=new_script.script_id,
- script_revision_id=new_script.script_revision_id,
- script_block_label=sb.script_block_label,
- script_file_id=sb.script_file_id,
- run_signature=sb.run_signature,
- workflow_run_id=sb.workflow_run_id,
- workflow_run_block_id=sb.workflow_run_block_id,
- input_fields=sb.input_fields,
- requires_agent=sb.requires_agent,
- )
+ # Create script block entry (metadata only — code lives in main.py).
+ # All blocks in this version are attributed to the triggering run — even
+ # non-updated blocks — so the version has uniform provenance.
+ await app.DATABASE.scripts.create_script_block(
+ organization_id=organization_id,
+ script_id=new_script.script_id,
+ script_revision_id=new_script.script_revision_id,
+ script_block_label=sb.script_block_label,
+ run_signature=block_run_signature,
+ workflow_run_id=workflow_run.workflow_run_id if workflow_run else sb.workflow_run_id,
+ workflow_run_block_id=sb.workflow_run_block_id,
+ input_fields=sb.input_fields,
+ requires_agent=block_requires_agent,
+ )
# Patch main.py with updated block functions and copy non-block files.
# Prefers the base revision's main.py (cumulative patches), falls back to v1.
@@ -1365,13 +1399,13 @@ async def create_script_version_from_review(
patched_main = _fix_static_actions_in_for_loops(patched_main)
# Validate the patched main.py compiles. If the splice corrupted it,
- # reconstruct from individual block files (which are all compile-verified
- # by the reviewer) instead of persisting a broken script.
+ # reconstruct from the base main.py source instead of persisting
+ # a broken script.
try:
compile(patched_main, "<patched_main.py>", "exec")
except SyntaxError as exc:
LOG.warning(
- "Patched main.py has syntax error — reconstructing from block files",
+ "Patched main.py has syntax error — reconstructing from base source",
script_id=new_script.script_id,
error=str(exc),
lineno=exc.lineno,
@@ -1379,8 +1413,6 @@ async def create_script_version_from_review(
)
reconstructed = await _reconstruct_main_py_from_blocks(
base_main_py=main_py_content,
- new_revision_id=new_script.script_revision_id,
- organization_id=organization_id,
updated_blocks=updated_blocks,
)
if reconstructed:
@@ -1460,10 +1492,9 @@ async def create_script_version_from_review(
organization_id=organization_id,
)
- updated_block_file_paths = {f"blocks/{label}.py" for label in updated_blocks}
for f in source_files:
- # Skip main.py (already patched) and updated block files (already created)
- if f.file_path == "main.py" or f.file_path in updated_block_file_paths:
+ # Skip main.py (already patched) and any legacy block files
+ if f.file_path == "main.py" or f.file_path.startswith("blocks/"):
continue
await app.DATABASE.scripts.create_script_file(
script_revision_id=new_script.script_revision_id,
diff --git a/tests/unit/test_script_reviewer_quality.py b/tests/unit/test_script_reviewer_quality.py
index f638269ef..e57c6c4d1 100644
--- a/tests/unit/test_script_reviewer_quality.py
+++ b/tests/unit/test_script_reviewer_quality.py
@@ -441,3 +441,89 @@ async def block_fn(page, context):
await page.wait(ai='fallback', prompt='wait for page')
"""
assert self.reviewer._validate_missing_selectors(code) is None
+
+
+class TestExtractCachedBlocks:
+ """Tests for the shared block extraction utilities."""
+
+ def test_extract_all_blocks(self):
+ from skyvern.services.workflow_script_service import extract_cached_blocks_from_source
+
+ source = """import skyvern
+
[email protected](cache_key = 'login')
+async def login(page, context):
+ await page.goto('https://example.com')
+ await page.complete()
+
+
[email protected](cache_key = 'block_1')
+async def block_1(page, context):
+ await page.click(selector='button')
+ await page.complete()
+"""
+ result = extract_cached_blocks_from_source(source)
+ assert set(result.keys()) == {"login", "block_1"}
+ assert "page.goto" in result["login"]
+ assert "page.click" in result["block_1"]
+ # login block should NOT contain block_1 code
+ assert "page.click" not in result["login"]
+
+ def test_extract_single_block(self):
+ from skyvern.services.workflow_script_service import extract_single_cached_block
+
+ source = """import skyvern
+
[email protected](cache_key = 'login')
+async def login(page, context):
+ await page.goto('https://example.com')
+
+
[email protected](cache_key = 'block_1')
+async def block_1(page, context):
+ await page.click(selector='button')
+"""
+ result = extract_single_cached_block(source, "block_1")
+ assert result is not None
+ assert "page.click" in result
+ assert "page.goto" not in result
+
+ def test_extract_first_block(self):
+ from skyvern.services.workflow_script_service import extract_single_cached_block
+
+ source = """import skyvern
+
[email protected](cache_key = 'login')
+async def login(page, context):
+ await page.goto('https://example.com')
+
+
[email protected](cache_key = 'block_1')
+async def block_1(page, context):
+ await page.click(selector='button')
+"""
+ result = extract_single_cached_block(source, "login")
+ assert result is not None
+ assert "page.goto" in result
+ assert "page.click" not in result
+
+ def test_extract_single_block_not_found(self):
+ from skyvern.services.workflow_script_service import extract_single_cached_block
+
+ source = "@skyvern.cached(cache_key = 'login')\nasync def login(page, context): pass\n"
+ result = extract_single_cached_block(source, "nonexistent")
+ assert result is None
+
+ def test_extract_empty_source(self):
+ from skyvern.services.workflow_script_service import extract_cached_blocks_from_source
+
+ assert extract_cached_blocks_from_source("") == {}
+ assert extract_cached_blocks_from_source("import skyvern\n") == {}
+
+ def test_extract_last_block_goes_to_eof(self):
+ from skyvern.services.workflow_script_service import extract_cached_blocks_from_source
+
+ source = "@skyvern.cached(cache_key = 'only_block')\nasync def only(page, ctx):\n pass\n"
+ result = extract_cached_blocks_from_source(source)
+ assert "only_block" in result
+ assert "pass" in result["only_block"]