Automate browser based workflows with AI
Python aiapiautomationbrowserbrowser-automationcomputergptllmplaywrightpowerautomatepuppeteerpythonrpaseleniumvisionworkflow
Stop reporting DataDome and PerimeterX block pages as "no challenge" (SKY-17262) (#8695)
Andrew Neilson554d239bc08e
diff --git a/skyvern/forge/agent_functions.py b/skyvern/forge/agent_functions.pyindex dab68dcb2..e68609278 100644--- a/skyvern/forge/agent_functions.py+++ b/skyvern/forge/agent_functions.py@@ -87,6 +87,7 @@ from skyvern.webeye.actions.actions import Actionfrom skyvern.webeye.browser_engine import UNSET_SELECTION, BrowserEngineSelection, resolve_engine_selection_for_taskfrom skyvern.webeye.browser_state import BrowserStatefrom skyvern.webeye.scraper.scraped_page import ELEMENT_NODE_ATTRIBUTES, CleanupElementTreeFunc, json_to_html+from skyvern.webeye.utils.challenge_signature import ChallengeVendorfrom skyvern.webeye.utils.dom import SkyvernElementfrom skyvern.webeye.utils.page import SkyvernFrame, take_element_screenshot@@ -1959,6 +1960,16 @@ class AgentFunction:"""Solve and apply a reCAPTCHA token. OSS has no solver client."""return False+ async def detect_vendor_challenge(self, page: Page | RecordingPage) -> ChallengeVendor | None:+ """The vendor whose challenge page this is, detect-only; raises when the page cannot be read, so an+ unreadable page is never taken for a cleared one. OSS has no vendor probe, so it never reports a read."""+ raise NotImplementedError++ async def run_vendor_challenge_handler(self, page: Page | RecordingPage, vendor: ChallengeVendor) -> None:+ """Run the deployment's handler for ``vendor``'s challenge page, keeping its own failures; the caller+ re-detects afterward rather than trusting it."""+ return None+def supports_image_captcha_ocr(self) -> bool:"""Whether read_image_captcha_text has a solver behind it. OSS has none."""return Falsediff --git a/skyvern/forge/prompts/skyvern/workflow_knowledge_base.txt b/skyvern/forge/prompts/skyvern/workflow_knowledge_base.txtindex 84d9372d4..afdf787a0 100644--- a/skyvern/forge/prompts/skyvern/workflow_knowledge_base.txt+++ b/skyvern/forge/prompts/skyvern/workflow_knowledge_base.txt@@ -771,11 +771,11 @@ blocks:** CAPTCHA SOLVER (captcha_solver) **-Purpose: The managed captcha solver is a product capability, enabled by default for cloud organizations; it can be turned off per browser session under "Captcha Solver", and an organization-level kill switch exists on the platform side. It clears Cloudflare Turnstile and hCaptcha challenges through the solver extension and mints reCAPTCHA v2/v3 tokens. It cannot clear a widget the page never renders, an IP-reputation block ("unusual traffic", "access denied"), a DataDome or Akamai deny page, or an account security prompt ("verify it's you", authenticator codes).+Purpose: The managed captcha solver is a product capability, enabled by default for cloud organizations; it can be turned off per browser session under "Captcha Solver", and an organization-level kill switch exists on the platform side. It clears Cloudflare Turnstile and hCaptcha challenges through the solver extension and mints reCAPTCHA v2/v3 tokens, and it hands a DataDome or PerimeterX challenge page to that vendor's handler, reporting it solved only when a fresh look shows the page cleared. It cannot clear a widget the page never renders, an IP-reputation block ("unusual traffic", "access denied"), an Akamai deny page, or an account security prompt ("verify it's you", authenticator codes).How a run reaches it:- Task-shaped blocks (navigation, login, action) invoke the solver automatically when a challenge is detected.-- A code block invokes it only when its code calls the builtin `await solve_captcha(page)`; nothing runs the solver for a code block otherwise. A failed solve raises inside the code block and the run's action trace records a `solve_captcha` action with status `failed`.+- A code block invokes it only when its code calls the builtin `await solve_captcha(page)`; nothing runs the solver for a code block otherwise. A challenge that stays unsolved, or that the solver has no route for, raises inside the code block and the run's action trace records a `solve_captcha` action with status `failed`.- A distorted-text image CAPTCHA (characters drawn in an image, typed into a field) is read by the image form, `await solve_captcha(page, image=<selector>, input=<selector>)`. In Workflow Copilot, while scouting, `solve_page_challenge` with `image` and `input`, or the same call inside `run_browser_code`, runs the same OCR, so the selectors are proven on the live page before they are saved. `image` selects the observed <img>, <svg> or <canvas> holding the text and `input` the observed answer field; both come from the inspected page. It reads the text and types it, and does not submit. It requires image OCR enabled for the organization; otherwise it raises. It raises with nothing typed when the image yields no text, and the runtime enforces a per-block call limit, raising on every call past it. So the block runs a short bounded loop: solve; if the call raised, click the observed refresh control and retry; otherwise submit, and if the form is still shown, click the refresh control and retry. With no refresh control, raise instead of retrying; raise after the last try. An answer typed while scouting fits only that one image and never belongs in the saved code.Reading the run: the build-test packet's `challenge` record says whether the solver is available for this organization (`solver_available`), whether this run invoked it (`solver_attempted`, absent when unknown), and what happened (`solver_result`: failed, attempted, not_attempted, or unresolved). "not_attempted" on a code block means the code never called the builtin, not that the product declined. "attempted" means the solve step ran and returned without an error, which is not the same as the challenge clearing: when no solver handles the page the call still reports success. "unresolved" means the run's action history was unavailable or cut short, so whether the solver ran is unknown and should not be stated either way. `solver_available` is resolved per page, and is absent when it was only resolved for a different URL.diff --git a/skyvern/forge/sdk/copilot/output_utils.py b/skyvern/forge/sdk/copilot/output_utils.pyindex 7b255aa21..84f907b57 100644--- a/skyvern/forge/sdk/copilot/output_utils.py+++ b/skyvern/forge/sdk/copilot/output_utils.py@@ -1939,7 +1939,7 @@ _PAGE_CHALLENGE_OUTCOME_SUMMARIES = {"solved": "Challenge solver reported the challenge solved","typed": "Challenge solver typed the CAPTCHA image's text","none": "Challenge solver found no challenge on the page",- "unsupported": "Challenge solver found no challenge it can operate",+ "unsupported": "Challenge solver has no route for the challenge on screen","unsolved": "Challenge solver could not clear the challenge in this browser","unavailable": "Challenge solving is not available for this page",}diff --git a/skyvern/forge/sdk/copilot/tools/__init__.py b/skyvern/forge/sdk/copilot/tools/__init__.pyindex 397f83d99..43dee54c7 100644--- a/skyvern/forge/sdk/copilot/tools/__init__.py+++ b/skyvern/forge/sdk/copilot/tools/__init__.py@@ -1703,7 +1703,8 @@ async def solve_page_challenge_tool(ctx: RunContextWrapper, image: str | None =Use it when the page shows a human-verification or anti-bot challenge: a navigate result's`challenge_vendor`, or a challenge you see in a screenshot. With no arguments it detects reCAPTCHA,- hCaptcha and Cloudflare Turnstile widgets, including ones inside frames, and can take up to 120 seconds.+ hCaptcha and Cloudflare Turnstile widgets, including ones inside frames, and DataDome and PerimeterX+ challenge pages, and can take up to 120 seconds.For a distorted-text image CAPTCHA, pass `image`, the selector of its <img>, <svg> or <canvas> (or acontainer holding exactly one), and `input`, the selector of its answer field. The OCR a saved code@@ -1713,7 +1714,7 @@ async def solve_page_challenge_tool(ctx: RunContextWrapper, image: str | None =`outcome` is one of: `solved`; `typed` (image form: the text was typed, unconfirmed until the pageaccepts it); `none` (no challenge detected, nothing ran); `unsupported` (a challenge frame is on screen- but the solver found nothing it can operate); `unsolved` (with `timed_out`, `solver_failed` or, for the+ that the solver has no route for); `unsolved` (with `timed_out`, `solver_failed` or, for theimage form, `read_limit_reached` when that is why); or `unavailable` (solving or image OCR is off forthis organization or page). `solved` and `typed` are the solver's report, not proof the page moved on:look at the page again before continuing. Each attempt can bill an external solver.diff --git a/skyvern/forge/sdk/copilot/tools/page_challenge.py b/skyvern/forge/sdk/copilot/tools/page_challenge.pyindex ec3b46f35..92494607e 100644--- a/skyvern/forge/sdk/copilot/tools/page_challenge.py+++ b/skyvern/forge/sdk/copilot/tools/page_challenge.py@@ -17,14 +17,9 @@ from skyvern.forge.sdk.copilot.runtime import (sensitive_origin_page_has_active_run,)from skyvern.forge.sdk.workflow.models.block import CodeBlockCaptchaError, _code_block_solve_captcha_builtin-from skyvern.webeye.utils.captcha_solver import (- MAX_IMAGE_CAPTCHA_READS,- CaptchaChallengeUnsolvedError,- solve_challenge_ladder,-)+from skyvern.webeye.utils.captcha_solver import MAX_IMAGE_CAPTCHA_READS, ChallengeStatus, solve_challengefrom ._shared import browser_is_lost, on_working_page-from .scouting import rendered_challenge_vendorLOG = structlog.get_logger()@@ -44,9 +39,7 @@ _OUTCOME_TEXT = {"the page again before continuing."),"none": "No challenge was detected on the page or its visible frames, so nothing was solved.",- "unsupported": (- "A challenge frame is on screen, but the solver found no challenge it can operate, so nothing was attempted."- ),+ "unsupported": "A challenge frame is on screen, but the solver has no route for this challenge.","unsolved": "A challenge is present and the solver could not clear it in this browser session.","unavailable": "Challenge solving is not available for this organization or page, so nothing was attempted.",}@@ -112,30 +105,19 @@ async def _run_ladder(ctx: AgentContext, page: Page, session_id: str) -> dict[stunsolved: dict[str, Any] = {"ok": True, "outcome": "unsolved", "detail": _OUTCOME_TEXT["unsolved"]}try:async with asyncio.timeout(SOLVE_CEILING_SECONDS):- solved = await solve_challenge_ladder(+ challenge = await solve_challenge(page,organization_id=ctx.organization_id,browser_session_id=session_id,probe_child_frames=True,)- except CaptchaChallengeUnsolvedError:- return unsolvedexcept TimeoutError:return {**unsolved, "timed_out": True, "detail": f"The solver did not finish in {SOLVE_CEILING_SECONDS}s."}except Exception:LOG.warning("copilot solve_page_challenge solver failed", exc_info=True)return {**unsolved, "solver_failed": True, "detail": "The solver failed with an internal error."}- if solved:- return {"ok": True, "outcome": "solved", "detail": _OUTCOME_TEXT["solved"]}- vendor = await rendered_challenge_vendor([frame for frame in page.frames if frame.parent_frame is not None])- if vendor is not None:- return {- "ok": True,- "outcome": "unsupported",- "challenge_vendor": vendor,- "detail": _OUTCOME_TEXT["unsupported"],- }- return {"ok": True, "outcome": "none", "detail": _OUTCOME_TEXT["none"]}+ outcome = "none" if challenge.status is ChallengeStatus.ABSENT else challenge.status.value+ return {"ok": True, "outcome": outcome, "detail": _OUTCOME_TEXT[outcome], "challenge": challenge.receipt()}async def _read_image(ctx: AgentContext, page: Page, session_id: str, image: str, input: str) -> dict[str, Any]:diff --git a/skyvern/forge/sdk/copilot/tools/scouting.py b/skyvern/forge/sdk/copilot/tools/scouting.pyindex 2905f7386..3ad42f33b 100644--- a/skyvern/forge/sdk/copilot/tools/scouting.py+++ b/skyvern/forge/sdk/copilot/tools/scouting.py@@ -87,7 +87,7 @@ from skyvern.forge.sdk.copilot.secret_scrub import (scrub_secrets_from_text,)from skyvern.webeye.browser_state import BrowserState-from skyvern.webeye.utils.challenge_signature import CHALLENGE_VENDOR_SIGNATURE+from skyvern.webeye.utils import challenge_signaturefrom ._shared import (_DISCOVERY_PER_CALL_TIMEOUT_SECONDS,@@ -2010,9 +2010,6 @@ async def _arm_scout_popup_listener(ctx: AgentContext) -> None:LOG.warning("copilot_scout_popup_listener_failed", exc_info=True)-_CHALLENGE_VENDOR_FRAME_URL = re.compile(CHALLENGE_VENDOR_SIGNATURE, re.IGNORECASE)--def _release_scout_challenge_listeners(ctx: AgentContext) -> None:for detach in ctx.pending_scout_challenge_detachers:try:@@ -2022,96 +2019,6 @@ def _release_scout_challenge_listeners(ctx: AgentContext) -> None:ctx.pending_scout_challenge_detachers = []-# A managed widget preloads small and grows when it actually challenges, so "rendered" cannot mean-# "has a box": a 1x1 iframe measures 25 once default borders are counted. Anything at or under this-# much on-screen area is a placeholder rather than a challenge a person could answer.-# ponytail: one sentinel for every vendor — revisit if a vendor's real widget ships smaller than 16x16.-_CHALLENGE_FRAME_PLACEHOLDER_AREA = 256.0--# Layout area is not screen area: a widget preloaded at full size can sit off-viewport, inside a-# zero-size overflow:hidden ancestor, under a clip-path, behind visibility:hidden, or under an-# opacity:0 ancestor, and still report its whole box. checkVisibility answers the style half;-# IntersectionObserver answers the geometry half, ancestor clip rects included, so neither the-# viewport nor a clipping container has to be walked by hand.-_CHALLENGE_FRAME_ONSCREEN_AREA_JS = """-el => {- if (el.checkVisibility && !el.checkVisibility({- opacityProperty: true, visibilityProperty: true, contentVisibilityAuto: true,- })) return 0;- return new Promise(resolve => {- let observer = null;- let timer = null;- const done = area => {- if (observer) observer.disconnect();- if (timer) clearTimeout(timer);- resolve(area);- };- observer = new IntersectionObserver(entries => {- const rect = entries[entries.length - 1].intersectionRect;- done(rect.width * rect.height);- });- timer = setTimeout(() => done(null), 1000);- observer.observe(el);- });-}-"""---_ELEMENT_STYLE_VISIBLE_JS = """-el => !el.checkVisibility || el.checkVisibility({- opacityProperty: true, visibilityProperty: true, contentVisibilityAuto: true,-})-"""---async def _frame_element_style_visible(frame: Frame) -> bool:- try:- element = await frame.frame_element()- return bool(await element.evaluate(_ELEMENT_STYLE_VISIBLE_JS))- except Exception:- return False---async def _embedding_frames_style_visible(frame: Frame) -> bool:- """Whether every iframe embedding this one, up to the top page, is visible by style.-- The observer's implicit root already clips geometry through each ancestor frame, but style is judged- per document: an opacity:0 or visibility:hidden iframe further up still lets this frame report its- full box. An ancestor that cannot be read counts as hidden.- """- ancestors: list[Frame] = []- current = frame.parent_frame- while current is not None and current.parent_frame is not None:- ancestors.append(current)- current = current.parent_frame- return all(await asyncio.gather(*(_frame_element_style_visible(ancestor) for ancestor in ancestors)))---# The script's own timer cannot fire in a renderer that never yields, and the click pre-hook runs outside-# the MCP call timeout, so the deadline is held here.-_CHALLENGE_FRAME_PROBE_TIMEOUT_SECONDS = 2.0---async def _challenge_frame_rendered_area(frame: Frame) -> float | None:- """On-screen area of the frame's own element, or None when it cannot be measured in time."""- try:- return await asyncio.wait_for(- _measure_challenge_frame_area(frame), timeout=_CHALLENGE_FRAME_PROBE_TIMEOUT_SECONDS- )- except Exception:- return None---async def _measure_challenge_frame_area(frame: Frame) -> float | None:- element = await frame.frame_element()- area = await element.evaluate(_CHALLENGE_FRAME_ONSCREEN_AREA_JS)- if area is None:- return None- if not await _embedding_frames_style_visible(frame):- return 0.0- return float(area)--async def _arm_scout_challenge_listener(ctx: AgentContext) -> None:"""Arm a frame-navigation listener for the click about to dispatch.@@ -2133,7 +2040,7 @@ async def _arm_scout_challenge_listener(ctx: AgentContext) -> None:already_challenged = [framefor frame in page.frames- if frame.parent_frame is not None and _CHALLENGE_VENDOR_FRAME_URL.search(frame.url or "")+ if frame.parent_frame is not None and challenge_signature.CHALLENGE_VENDOR_FRAME_URL.search(frame.url or "")]def _capture(frame: Frame) -> None:@@ -2142,7 +2049,7 @@ async def _arm_scout_challenge_listener(ctx: AgentContext) -> None:baseline = (seen for seen, _area in ctx.pending_scout_challenge_prior_frames)if any(frame is seen for seen in (*already_challenged, *baseline, *ctx.pending_scout_challenge_frames)):return- if _CHALLENGE_VENDOR_FRAME_URL.search(frame.url or ""):+ if challenge_signature.CHALLENGE_VENDOR_FRAME_URL.search(frame.url or ""):ctx.pending_scout_challenge_frames.append(frame)# Listening before the measurement below yields, so a frame that mounts while it runs is still seen.@@ -2151,7 +2058,9 @@ async def _arm_scout_challenge_listener(ctx: AgentContext) -> None:ctx.pending_scout_challenge_armed_at = time.monotonic()# Measured together: each reading stops itself after its observer timeout, so a page with several# vendor frames costs one bound before the click dispatches rather than one per frame.- prior_areas = await asyncio.gather(*(_challenge_frame_rendered_area(frame) for frame in already_challenged))+ prior_areas = await asyncio.gather(+ *(challenge_signature.challenge_frame_rendered_area(frame) for frame in already_challenged)+ )ctx.pending_scout_challenge_prior_frames = list(zip(already_challenged, prior_areas, strict=True))except Exception:LOG.warning("copilot_scout_challenge_listener_failed", exc_info=True)@@ -2260,7 +2169,7 @@ async def _close_scout_challenge_baseline(ctx: AgentContext) -> None:returnarrivals = list(ctx.pending_scout_challenge_frames)ctx.pending_scout_challenge_frames.clear()- areas = await asyncio.gather(*(_challenge_frame_rendered_area(frame) for frame in arrivals))+ areas = await asyncio.gather(*(challenge_signature.challenge_frame_rendered_area(frame) for frame in arrivals))ctx.pending_scout_challenge_prior_frames.extend(zip(arrivals, areas, strict=True))# Frames that navigated during that measurement are baselined unmeasured, with no await before dispatch:# an unmeasured baseline frame is never credited, as new or as revealed.@@ -2291,25 +2200,9 @@ async def _on_screen_challenge_vendor(ctx: AgentContext) -> str | None:revealable = [framefor frame, prior_area in ctx.pending_scout_challenge_prior_frames- if prior_area is not None and prior_area <= _CHALLENGE_FRAME_PLACEHOLDER_AREA+ if prior_area is not None and prior_area <= challenge_signature.CHALLENGE_FRAME_PLACEHOLDER_AREA]- return await rendered_challenge_vendor([*ctx.pending_scout_challenge_frames, *revealable])---async def rendered_challenge_vendor(frames: list[Frame]) -> str | None:- """Vendor of the first of these frames that is a challenge frame rendered on screen above placeholder size."""- # A captured frame can navigate away before this runs, so each is matched on its current URL.- candidates = [- (frame, match) for frame in frames if (match := _CHALLENGE_VENDOR_FRAME_URL.search(frame.url or "")) is not None- ]- areas = await asyncio.gather(*(_challenge_frame_rendered_area(frame) for frame, _match in candidates))- for (_frame, match), area in zip(candidates, areas, strict=True):- if area is not None and area > _CHALLENGE_FRAME_PLACEHOLDER_AREA:- # The matched text is one of the signature's own literals, so the vendor is named from a closed- # vocabulary. A hostname would carry whatever the page put in it — a tenant slug or a secret, in- # any case or encoding — and no scrub can enumerate every spelling of that.- return match.group(0).casefold()- return None+ return await challenge_signature.rendered_challenge_vendor([*ctx.pending_scout_challenge_frames, *revealable])async def _maybe_attach_observed_challenge(ctx: AgentContext, result: dict[str, Any], *, url: str) -> None:@@ -2350,9 +2243,11 @@ async def attach_navigation_challenge_vendor(ctx: AgentContext, result: dict[strif browser_state is None:returnpage = await browser_state.get_or_create_page()- vendor = await rendered_challenge_vendor([frame for frame in page.frames if frame.parent_frame is not None])+ vendor = await challenge_signature.rendered_challenge_vendor(+ [frame for frame in page.frames if frame.parent_frame is not None]+ )vendor_frame_present = any(- frame.parent_frame is not None and _CHALLENGE_VENDOR_FRAME_URL.search(frame.url or "")+ frame.parent_frame is not None and challenge_signature.CHALLENGE_VENDOR_FRAME_URL.search(frame.url or "")for frame in page.frames)if vendor is None and (ctx.pending_scout_challenge_frames or vendor_frame_present):@@ -2362,7 +2257,9 @@ async def attach_navigation_challenge_vendor(ctx: AgentContext, result: dict[strowed = settings.COPILOT_SCOUT_ACT_OBSERVE_RECAPTURE_DELAY_SECONDS - (time.monotonic() - settle_started)if owed > 0:await asyncio.sleep(owed)- vendor = await rendered_challenge_vendor([frame for frame in page.frames if frame.parent_frame is not None])+ vendor = await challenge_signature.rendered_challenge_vendor(+ [frame for frame in page.frames if frame.parent_frame is not None]+ )except Exception:LOG.warning("copilot_navigation_challenge_vendor_failed", exc_info=True)returndiff --git a/skyvern/forge/sdk/workflow/models/block.py b/skyvern/forge/sdk/workflow/models/block.pyindex 1f33672a6..e980074ce 100644--- a/skyvern/forge/sdk/workflow/models/block.py+++ b/skyvern/forge/sdk/workflow/models/block.py@@ -346,9 +346,9 @@ from skyvern.webeye.playwright_input import playwright_input_defaults_for_pagefrom skyvern.webeye.real_browser_state import RealBrowserStatefrom skyvern.webeye.utils.captcha_solver import (MAX_IMAGE_CAPTCHA_READS,- CaptchaChallengeUnsolvedError,+ ChallengeStatus,resolve_captcha_image,- solve_challenge_ladder,+ solve_challenge,)from skyvern.webeye.utils.page import ScreenshotMode, SkyvernFrame@@ -5049,21 +5049,25 @@ async def _code_block_solve_captcha_builtin(_require_captcha_selector(image, "image")_require_captcha_selector(input, "input")+ recorded_output: dict[str, dict[str, str | None]] = {}+async def solve() -> bool:if image is not None and input is not None:return await _fill_image_captcha_text(page, image, input, organization_id=organization_id)- try:- return await solve_challenge_ladder(- page,- organization_id=organization_id,- workflow_run_id=workflow_run_id,- browser_session_id=browser_session_id,- )- except CaptchaChallengeUnsolvedError as exc:- raise CodeBlockCaptchaError("CAPTCHA could not be solved.") from exc+ outcome = await solve_challenge(+ page,+ organization_id=organization_id,+ workflow_run_id=workflow_run_id,+ browser_session_id=browser_session_id,+ probe_child_frames=True,+ )+ recorded_output["challenge"] = outcome.receipt()+ if outcome.status in (ChallengeStatus.UNSOLVED, ChallengeStatus.UNSUPPORTED):+ raise CodeBlockCaptchaError("CAPTCHA could not be solved.")+ return outcome.status is ChallengeStatus.SOLVEDif isinstance(page, RecordingPage):- return await page._record_solve_captcha(solve, workflow_run_id=workflow_run_id)+ return await page._record_solve_captcha(solve, workflow_run_id=workflow_run_id, extra_output=recorded_output)return await solve()diff --git a/skyvern/forge/sdk/workflow/models/code_block_recorder.py b/skyvern/forge/sdk/workflow/models/code_block_recorder.pyindex 4499a47e6..3bf140934 100644--- a/skyvern/forge/sdk/workflow/models/code_block_recorder.py+++ b/skyvern/forge/sdk/workflow/models/code_block_recorder.py@@ -486,7 +486,9 @@ class _Recorder:record_boolean_response: bool = False,workflow_run_id: str | None = None,record_failure_type_only: bool = False,+ extra_output: dict[str, Any] | None = None,) -> Any:+ """``extra_output`` is read after ``call`` settles, so the call can fill it on its failure path too."""generation = self.begin_failure_operation()started = time.monotonic()started_wall = naive_utc_now()@@ -561,6 +563,8 @@ class _Recorder:action.finished_at = naive_utc_now()if isinstance(action.output, dict):action.output["duration_ms"] = duration_ms+ if extra_output:+ action.output.update(extra_output)self.actions.append(action)if self._on_action is not None:try:@@ -1042,6 +1046,7 @@ class RecordingPage:call: Callable[[], Awaitable[bool]],*,workflow_run_id: str | None,+ extra_output: dict[str, Any] | None = None,) -> bool:return cast(bool,@@ -1055,6 +1060,7 @@ class RecordingPage:record_boolean_response=True,workflow_run_id=workflow_run_id,record_failure_type_only=True,+ extra_output=extra_output,),)diff --git a/skyvern/forge/taskv3/tools.py b/skyvern/forge/taskv3/tools.pyindex 4bc891786..e85d44d47 100644--- a/skyvern/forge/taskv3/tools.py+++ b/skyvern/forge/taskv3/tools.py@@ -89,7 +89,7 @@ from skyvern.webeye.actions.key_names import normalize_key_chordfrom skyvern.webeye.browser_driver_errors import is_driver_error, is_driver_timeout_errorfrom skyvern.webeye.browser_state import BLANK_PAGE_URLSfrom skyvern.webeye.navigation import driver_nav_error_code, record_task_nav_error_code, redact_url_secrets-from skyvern.webeye.utils.challenge_signature import CHALLENGE_VENDOR_SIGNATURE+from skyvern.webeye.utils.challenge_signature import CHALLENGE_VENDOR_FRAME_URL, CHALLENGE_VENDOR_SIGNATUREfrom skyvern.webeye.utils.page import OTP_INPUT_PRIVACY_JS, OTP_SAFE_FRAGMENT_HTML_JS, mask_otp_values_in_htmlif TYPE_CHECKING:@@ -214,8 +214,6 @@ _MARKER_ATTR_OPENS = ('data-tv3="', 'data-tv3-pick="')# is what separates ours from theirs._MINTED_MARKER_VALUE_RE = re.compile(r"\At\d+(?:-\d+)?\Z")-_CHALLENGE_VENDOR_FRAME_URL = re.compile(CHALLENGE_VENDOR_SIGNATURE, re.IGNORECASE)-# Containment is walked from the frame's host upward because the probe runs in an isolated world and# its shadow walk pierces only OPEN roots -- neither sees a widget iframe mounted inside a closed one._COVER_ANCESTOR_WALK_LIMIT = 40@@ -11969,7 +11967,7 @@ def build_browser_tools(parsed = urlparse(frame.url or "")if parsed.scheme not in ("http", "https") or not parsed.hostname:continue- if not _CHALLENGE_VENDOR_FRAME_URL.search(f"{parsed.scheme}://{parsed.hostname}{parsed.path}"):+ if not CHALLENGE_VENDOR_FRAME_URL.search(f"{parsed.scheme}://{parsed.hostname}{parsed.path}"):continuechain = [frame]while chain[-1].parent_frame is not None and chain[-1].parent_frame is not root:diff --git a/skyvern/webeye/utils/captcha_solver.py b/skyvern/webeye/utils/captcha_solver.pyindex aef14734a..66ff4f06b 100644--- a/skyvern/webeye/utils/captcha_solver.py+++ b/skyvern/webeye/utils/captcha_solver.py@@ -15,8 +15,11 @@ so this module stays OSS-clean: the OSS bases return False and the cloud overridfrom __future__ import annotationsimport asyncio+import contextlibimport time-from typing import TYPE_CHECKING, Any+from dataclasses import dataclass+from enum import StrEnum+from typing import TYPE_CHECKING, Any, Literalfrom urllib.parse import urlparseimport structlog@@ -25,6 +28,7 @@ from playwright.async_api import Frame, Pagefrom skyvern.config import settingsfrom skyvern.forge import app+from skyvern.webeye.utils.challenge_signature import ChallengeVendor, rendered_challenge_vendorif TYPE_CHECKING:from skyvern.forge.sdk.workflow.models.code_block_recorder import RecordingPage@@ -36,6 +40,31 @@ class CaptchaChallengeUnsolvedError(Exception):"""A captcha challenge was present on the page but no solver arm resolved it."""+class ChallengeStatus(StrEnum):+ ABSENT = "absent"+ SOLVED = "solved"+ UNSOLVED = "unsolved"+ UNSUPPORTED = "unsupported"+++ChallengeArm = Literal["vendor_handler", "dom_checkbox", "recaptcha_anchor_frame", "extension", "token"]+ChallengePageState = Literal["clear", "challenged", "not_rechecked"]+++@dataclass(frozen=True)+class ChallengeOutcome:+ """What a solve found and did. ``vendor`` is a ``ChallengeVendor`` value or a challenge-signature literal, never+ page text, and ``page_state`` is what a look at the page after the last arm showed."""++ status: ChallengeStatus+ vendor: str | None = None+ arm: ChallengeArm | None = None+ page_state: ChallengePageState = "not_rechecked"++ def receipt(self) -> dict[str, str | None]:+ return {"status": self.status.value, "vendor": self.vendor, "arm": self.arm, "page_state": self.page_state}++_CAPTCHA_CHECKBOX_SELECTOR = ", ".join(('input[type="checkbox"][id*="captcha" i]',@@ -91,6 +120,13 @@ _CHILD_FRAME_SCAN_BUDGET_SECONDS = 3# (resolve_captcha_solver_extension_timeout); a widened extension can never push the bounded arms past the# v3 tool's 120s ceiling — the token arm then gets less than a full solve needs, and the wider arm gates._LADDER_BUDGET_SECONDS = 110+# Above the few seconds a deployment's probe may wait for a vendor to draw a challenge it just served.+_VENDOR_PROBE_TIMEOUT_SECONDS = 5+_VENDOR_SETTLE_TIMEOUT_MS = 5_000+_VENDOR_CLEAR_WAIT_SECONDS = 10+_VENDOR_CLEAR_POLL_SECONDS = 0.5+# Rendered frames of these vendors have no solver arm, so a failed ladder over one is unsupported, not unsolved.+_UNSUPPORTED_FRAME_VENDORS = frozenset({"arkoselabs", "funcaptcha"})# Google's widget flips aria-checked after its own animation; a shorter wait reads as unsolved._RECAPTCHA_ANCHOR_SETTLE_MS = 2_000_CAPTCHA_CONTINUE_SELECTOR = ", ".join(@@ -278,40 +314,146 @@ async def solve_challenge_ladder(browser_session_id: str | None = None,probe_child_frames: bool = False,) -> bool:- """Solve a detected challenge through the bounded platform ladder; True when an arm passed.-- Thin public entry: it enters the ``AGENT_FUNCTION`` captcha-solver lifecycle scope exactly once- around the ladder, so a deployment can bind a page-scoped solver lifecycle for the whole solve- (its self-heal/teardown owned by that scope). ``probe_child_frames`` opts a caller into also probing- visible child frames' documents for markers; the default checks the main document plus a visible- ``challenges.cloudflare.com`` frame.- """+ """``solve_challenge`` as a bool: True when solved, False when no challenge is on the page, and+ CaptchaChallengeUnsolvedError when one is present but unsolved or unsupported."""+ outcome = await solve_challenge(+ page,+ organization_id=organization_id,+ workflow_run_id=workflow_run_id,+ browser_session_id=browser_session_id,+ probe_child_frames=probe_child_frames,+ )+ if outcome.status in (ChallengeStatus.UNSOLVED, ChallengeStatus.UNSUPPORTED):+ raise CaptchaChallengeUnsolvedError("CAPTCHA could not be solved.")+ return outcome.status is ChallengeStatus.SOLVED+++async def solve_challenge(+ page: Page | RecordingPage,+ *,+ organization_id: str | None = None,+ workflow_run_id: str | None = None,+ browser_session_id: str | None = None,+ probe_child_frames: bool = False,+) -> ChallengeOutcome:+ """Solve a detected challenge inside one captcha-solver lifecycle scope and report what happened. A vendor page+ goes only to its handler and counts as cleared only when a fresh probe finds neither the vendor nor a widget."""async with app.AGENT_FUNCTION.captcha_solver_lifecycle_scope(page):- return await _solve_challenge_ladder_impl(- page,- organization_id=organization_id,- workflow_run_id=workflow_run_id,- browser_session_id=browser_session_id,- probe_child_frames=probe_child_frames,- )+ start = time.monotonic()+ try:+ vendor = await _detect_vendor_challenge_retrying(page)+ vendor_read = True+ except NotImplementedError:+ vendor, vendor_read = None, False+ except Exception as exc:+ LOG.info("CAPTCHA vendor probe did not complete", arm="vendor_handler", error_type=type(exc).__name__)+ vendor, vendor_read = None, False+ routed = vendor.value if vendor is not None else None+ if vendor is not None:+ vendor_state: ChallengePageState+ try:+ async with asyncio.timeout(_LADDER_BUDGET_SECONDS - (time.monotonic() - start)):+ await app.AGENT_FUNCTION.run_vendor_challenge_handler(page, vendor)+ vendor_state = await _await_vendor_clear(page)+ except Exception as exc:+ LOG.info(+ "CAPTCHA vendor recheck did not complete",+ arm="vendor_handler",+ vendor=routed,+ error_type=type(exc).__name__,+ )+ vendor_state = "not_rechecked"+ LOG.info("CAPTCHA vendor handler finished", arm="vendor_handler", vendor=routed, page_state=vendor_state)+ if vendor_state != "clear":+ return ChallengeOutcome(ChallengeStatus.UNSOLVED, routed, "vendor_handler", vendor_state)++ try:+ # The arms are each bounded to fit the budget from ``start``; a vendor handler that spent most of it+ # would otherwise let the remaining unclamped probes and arms run past the caller's ceiling.+ async with asyncio.timeout(_LADDER_BUDGET_SECONDS - (time.monotonic() - start)):+ arm = await _solve_challenge_ladder_impl(+ page,+ start=start,+ organization_id=organization_id,+ workflow_run_id=workflow_run_id,+ browser_session_id=browser_session_id,+ probe_child_frames=probe_child_frames,+ )+ except TimeoutError:+ LOG.info("CAPTCHA ladder stopped at its budget", arm="ladder")+ return ChallengeOutcome(ChallengeStatus.UNSOLVED)+ except CaptchaChallengeUnsolvedError:+ rendered = await _rendered_frame_vendor(page)+ if rendered in _UNSUPPORTED_FRAME_VENDORS:+ return ChallengeOutcome(ChallengeStatus.UNSUPPORTED, rendered, page_state="challenged")+ return ChallengeOutcome(ChallengeStatus.UNSOLVED, rendered)+ if arm is not None:+ return ChallengeOutcome(ChallengeStatus.SOLVED, arm=arm)+ rendered = await _rendered_frame_vendor(page)+ if rendered is not None:+ status = ChallengeStatus.UNSUPPORTED if rendered in _UNSUPPORTED_FRAME_VENDORS else ChallengeStatus.UNSOLVED+ return ChallengeOutcome(status, rendered, page_state="challenged")+ if vendor is not None:+ return ChallengeOutcome(ChallengeStatus.SOLVED, routed, "vendor_handler", "clear")+ return ChallengeOutcome(ChallengeStatus.ABSENT, page_state="clear" if vendor_read else "not_rechecked")+++async def _detect_vendor_challenge(page: Page | RecordingPage) -> ChallengeVendor | None:+ async with asyncio.timeout(_VENDOR_PROBE_TIMEOUT_SECONDS):+ return await app.AGENT_FUNCTION.detect_vendor_challenge(page)+++async def _detect_vendor_challenge_retrying(page: Page | RecordingPage) -> ChallengeVendor | None:+ try:+ return await _detect_vendor_challenge(page)+ except NotImplementedError:+ raise+ except Exception as exc:+ LOG.info("CAPTCHA vendor probe failed; retrying once", arm="vendor_handler", error_type=type(exc).__name__)+ return await _detect_vendor_challenge(page)+++async def _await_vendor_clear(page: Page | RecordingPage) -> ChallengePageState:+ """Poll the vendor probe until two reads in a row are clear or the wait ends. A vendor may reload some time after+ a passed hold, and a fresh block page's challenge element is empty until its script fills it, so one clear read+ is not enough; a probe that fails mid-navigation reads as not yet."""+ state: ChallengePageState = "not_rechecked"+ clear_reads = 0+ deadline = time.monotonic() + _VENDOR_CLEAR_WAIT_SECONDS+ while True:+ with contextlib.suppress(PlaywrightError):+ await page.wait_for_load_state("load", timeout=_VENDOR_SETTLE_TIMEOUT_MS)+ try:+ if await _detect_vendor_challenge(page) is None:+ clear_reads += 1+ if clear_reads == 2:+ return "clear"+ else:+ clear_reads = 0+ state = "challenged"+ except Exception as exc:+ clear_reads = 0+ LOG.info("CAPTCHA vendor recheck read failed", arm="vendor_handler", error_type=type(exc).__name__)+ if time.monotonic() >= deadline and clear_reads == 0:+ return state+ await asyncio.sleep(_VENDOR_CLEAR_POLL_SECONDS)+++async def _rendered_frame_vendor(page: Page | RecordingPage) -> str | None:+ return await rendered_challenge_vendor([frame for frame in page.frames if frame.parent_frame is not None])async def _solve_challenge_ladder_impl(page: Page | RecordingPage,*,+ start: float,organization_id: str | None = None,workflow_run_id: str | None = None,browser_session_id: str | None = None,probe_child_frames: bool = False,-) -> bool:- """Solve a detected challenge through the bounded platform ladder; True when an arm passed.-- The initial structural probes are intentionally cheap. Solver routes are never called when neither- a challenge control nor vendor marker is present, and False distinguishes that no-op from a solve so- callers do not re-perceive a page nothing touched. Raises CaptchaChallengeUnsolvedError when a- challenge was present but no arm resolved it.- """- start = time.monotonic()+) -> ChallengeArm | None:+ """The arm that solved a detected widget, or None when the cheap probes find none and no solver is called.+ Raises CaptchaChallengeUnsolvedError when no arm, each clamped to the budget left since ``start``, clears it."""checkbox = page.locator(_CAPTCHA_CHECKBOX_SELECTOR)marker = page.locator(_CAPTCHA_MARKER_SELECTOR)# A challenge nested in a child frame only widens this presence check: the DOM-checkbox arm below@@ -326,7 +468,7 @@ async def _solve_challenge_ladder_impl(challenge_frames=True,)):- return False+ return Noneif await _bounded_locator_count(checkbox) == 1:candidate = checkbox.first@@ -342,11 +484,11 @@ async def _solve_challenge_ladder_impl(await continuation_candidate.click()await page.wait_for_timeout(100)if await _bounded_locator_count(checkbox) == 0:- return True+ return "dom_checkbox"else:# Checkbox challenges commonly complete on the checkbox# interaction itself and expose no associated continuation.- return True+ return "dom_checkbox"except Exception:LOG.info("CAPTCHA checkbox arm did not solve", arm="dom_checkbox")@@ -379,7 +521,7 @@ async def _solve_challenge_ladder_impl(await page.wait_for_timeout(_RECAPTCHA_ANCHOR_SETTLE_MS)if frame.is_detached() and urlparse(page.url)._replace(fragment="").geturl() != page_url_before_click:LOG.info("CAPTCHA anchor frame solved after navigation", arm="recaptcha_anchor_frame")- return True+ return "recaptcha_anchor_frame"token_is_populated = await _recaptcha_token_populated(token_scope)if (await candidate.get_attribute("aria-checked") == "true"@@ -388,7 +530,7 @@ async def _solve_challenge_ladder_impl(and await app.AGENT_FUNCTION.is_captcha_solver_completion_confirmed(page, default_result=True)):LOG.info("CAPTCHA anchor frame solved", arm="recaptcha_anchor_frame")- return True+ return "recaptcha_anchor_frame"# An inconclusive baseline fails the test above even when the click earned a token,# so read the widget rather than the verdict before deciding a reset is free.anchor_left_token = token_is_populated is True@@ -421,13 +563,19 @@ async def _solve_challenge_ladder_impl(default_extension_timeout = _HCAPTCHA_ARM_TIMEOUT_SECONDS if hcaptcha_present else _EXTENSION_ARM_TIMEOUT_SECONDS# Resolved inside the already-entered lifecycle scope so a deployment can widen this arm for a solver# it armed on scope entry, instead of cutting a slow legitimate solve at the generic bound.- extension_timeout = app.AGENT_FUNCTION.resolve_captcha_solver_extension_timeout(page, default_extension_timeout)- try:- async with asyncio.timeout(extension_timeout):- if await app.AGENT_FUNCTION.auto_solve_captchas(page):- return True- except Exception:- LOG.info("CAPTCHA extension arm did not solve", arm="extension")+ extension_timeout = min(+ app.AGENT_FUNCTION.resolve_captcha_solver_extension_timeout(page, default_extension_timeout),+ _LADDER_BUDGET_SECONDS - (time.monotonic() - start),+ )+ if extension_timeout <= 0:+ LOG.info("CAPTCHA extension arm skipped: ladder budget exhausted", arm="extension")+ else:+ try:+ async with asyncio.timeout(extension_timeout):+ if await app.AGENT_FUNCTION.auto_solve_captchas(page):+ return "extension"+ except Exception:+ LOG.info("CAPTCHA extension arm did not solve", arm="extension")recaptcha = page.locator(_RECAPTCHA_MARKER_SELECTOR)if await _bounded_locator_count(recaptcha) > 0:@@ -444,7 +592,7 @@ async def _solve_challenge_ladder_impl(workflow_run_id=workflow_run_id,browser_session_id=browser_session_id,):- return True+ return "token"except Exception:LOG.info("CAPTCHA token arm did not solve", arm="token")diff --git a/skyvern/webeye/utils/challenge_signature.py b/skyvern/webeye/utils/challenge_signature.pyindex 671b65fa2..43eee51dd 100644--- a/skyvern/webeye/utils/challenge_signature.py+++ b/skyvern/webeye/utils/challenge_signature.py@@ -1,12 +1,118 @@-"""Vendor signature shared by the Task V3 observe scan and the copilot scout click hooks.--Spliced verbatim into a JavaScript regex literal as well as compiled by ``re``, so it must stay-inside the syntax both engines read the same way: no named groups, no lookbehind, no ``/``.-"""+"""Challenge vendor signature and on-screen challenge-frame probe. The signature is also spliced into a JavaScript+regex literal, so it must stay in syntax both engines share: no named groups, no lookbehind, no ``/``."""from __future__ import annotations+import asyncio+import re+from enum import StrEnum++from playwright.async_api import Frame+CHALLENGE_VENDOR_SIGNATURE = (r"captcha|turnstile|challenges\.cloudflare|arkoselabs|funcaptcha|datadome|perimeterx"r"|verify you are human|security challenge")+CHALLENGE_VENDOR_FRAME_URL = re.compile(CHALLENGE_VENDOR_SIGNATURE, re.IGNORECASE)+++class ChallengeVendor(StrEnum):+ """A bot-defense vendor whose challenge page a deployment can hand to its own handler."""++ DATADOME = "datadome"+ PERIMETERX = "perimeterx"+++# A managed widget preloads small (a bordered 1x1 iframe measures 25) and grows when it challenges, so an area at or+# under this is a placeholder. ponytail: one sentinel for every vendor; revisit if a real widget ships under 16x16.+CHALLENGE_FRAME_PLACEHOLDER_AREA = 256.0++# A full-size preload can be off-viewport, clipped, or hidden by style and still report its whole box.+# checkVisibility answers style and IntersectionObserver answers geometry, ancestor clip rects included.+_CHALLENGE_FRAME_ONSCREEN_AREA_JS = """+el => {+ if (el.checkVisibility && !el.checkVisibility({+ opacityProperty: true, visibilityProperty: true, contentVisibilityAuto: true,+ })) return 0;+ return new Promise(resolve => {+ let observer = null;+ let timer = null;+ const done = area => {+ if (observer) observer.disconnect();+ if (timer) clearTimeout(timer);+ resolve(area);+ };+ observer = new IntersectionObserver(entries => {+ const rect = entries[entries.length - 1].intersectionRect;+ done(rect.width * rect.height);+ });+ timer = setTimeout(() => done(null), 1000);+ observer.observe(el);+ });+}+"""+++_ELEMENT_STYLE_VISIBLE_JS = """+el => !el.checkVisibility || el.checkVisibility({+ opacityProperty: true, visibilityProperty: true, contentVisibilityAuto: true,+})+"""+++async def _frame_element_style_visible(frame: Frame) -> bool:+ try:+ element = await frame.frame_element()+ return bool(await element.evaluate(_ELEMENT_STYLE_VISIBLE_JS))+ except Exception:+ return False+++async def _embedding_frames_style_visible(frame: Frame) -> bool:+ """Whether every iframe embedding this one is visible by style, since style is judged per document and a hidden+ ancestor iframe still lets this frame report its full box; an unreadable ancestor counts as hidden."""+ ancestors: list[Frame] = []+ current = frame.parent_frame+ while current is not None and current.parent_frame is not None:+ ancestors.append(current)+ current = current.parent_frame+ return all(await asyncio.gather(*(_frame_element_style_visible(ancestor) for ancestor in ancestors)))+++# The script's own timer cannot fire in a renderer that never yields, so the deadline is held here.+_CHALLENGE_FRAME_PROBE_TIMEOUT_SECONDS = 2.0+++async def challenge_frame_rendered_area(frame: Frame) -> float | None:+ """On-screen area of the frame's own element, or None when it cannot be measured in time."""+ try:+ return await asyncio.wait_for(+ _measure_challenge_frame_area(frame), timeout=_CHALLENGE_FRAME_PROBE_TIMEOUT_SECONDS+ )+ except Exception:+ return None+++async def _measure_challenge_frame_area(frame: Frame) -> float | None:+ element = await frame.frame_element()+ area = await element.evaluate(_CHALLENGE_FRAME_ONSCREEN_AREA_JS)+ if area is None:+ return None+ if not await _embedding_frames_style_visible(frame):+ return 0.0+ return float(area)+++async def rendered_challenge_vendor(frames: list[Frame]) -> str | None:+ """Vendor of the first of these frames that is a challenge frame rendered on screen above placeholder size."""+ # A captured frame can navigate away before this runs, so each is matched on its current URL.+ candidates = [+ (frame, match) for frame in frames if (match := CHALLENGE_VENDOR_FRAME_URL.search(frame.url or "")) is not None+ ]+ areas = await asyncio.gather(*(challenge_frame_rendered_area(frame) for frame, _match in candidates))+ for (_frame, match), area in zip(candidates, areas, strict=True):+ if area is not None and area > CHALLENGE_FRAME_PLACEHOLDER_AREA:+ # The vendor is named by the signature literal that matched, never by the hostname, which can carry+ # a tenant slug or a secret in a spelling no scrub can enumerate.+ return match.group(0).casefold()+ return Nonediff --git a/tests/unit/test_code_block_captcha.py b/tests/unit/test_code_block_captcha.pyindex a8ae4a965..251d9f665 100644--- a/tests/unit/test_code_block_captcha.py+++ b/tests/unit/test_code_block_captcha.py@@ -22,17 +22,25 @@ from skyvern.forge.sdk.workflow.models.code_block_recorder import RecordingPagefrom skyvern.webeye.actions.action_types import ActionTypefrom skyvern.webeye.actions.actions import ActionStatusfrom skyvern.webeye.utils import captcha_solver as captcha_solver_module+from skyvern.webeye.utils import challenge_signaturefrom skyvern.webeye.utils.captcha_solver import (MAX_IMAGE_CAPTCHA_READS,CaptchaChallengeUnsolvedError,+ ChallengeOutcome,+ ChallengeStatus,+ solve_challenge,solve_challenge_ladder,)+from skyvern.webeye.utils.challenge_signature import ChallengeVendorfrom tests.unit.conftest import OcrRecordingAgentFunction, ScopeRecordingAgentFunctionCHALLENGE_URL = ("https://challenges.cloudflare.com/cdn-cgi/challenge-platform/turnstile/if/ov2/av0/fake?sitekey=0xTESTKEY")CHALLENGE_SUBDOMAIN_URL = "https://edge.challenges.cloudflare.com/cdn-cgi/challenge-platform/fake"+SOLVED = ChallengeOutcome(ChallengeStatus.SOLVED, None, "extension")+ABSENT = ChallengeOutcome(ChallengeStatus.ABSENT, page_state="clear")+UNSOLVED = ChallengeOutcome(ChallengeStatus.UNSOLVED, "datadome", "vendor_handler", "challenged")class FakeLocator:@@ -223,6 +231,9 @@ class FakePage:async def wait_for_timeout(self, _milliseconds: int) -> None:await asyncio.sleep(0)+ async def wait_for_load_state(self, *_args: object, **_kwargs: object) -> None:+ return None+async def evaluate(self, expression: str, *_args: object) -> None:self.evaluated.append(expression)@@ -662,11 +673,13 @@ async def test_builtin_records_one_solver_action_with_nested_probe(monkeypatch: pytest.MonkeyPatch,result: bool,) -> None:- async def ladder(page: RecordingPage, **_kwargs: object) -> bool:+ outcome = SOLVED if result else ABSENT++ async def ladder(page: RecordingPage, **_kwargs: object) -> ChallengeOutcome:await page.locator("#challenge").click()- return result+ return outcome- monkeypatch.setattr(block_module, "solve_challenge_ladder", ladder)+ monkeypatch.setattr(block_module, "solve_challenge", ladder)page = RecordingPage(FakePage())assert await block_module._code_block_solve_captcha_builtin(page, workflow_run_id="wr_test") is result@@ -677,16 +690,17 @@ async def test_builtin_records_one_solver_action_with_nested_probe(assert actions[0].status == ActionStatus.completedassert actions[0].response == str(result).lower()assert actions[0].workflow_run_id == "wr_test"+ assert actions[0].output["challenge"] == outcome.receipt()@pytest.mark.asyncio-async def test_builtin_records_sanitized_unsolved_failure(monkeypatch: pytest.MonkeyPatch) -> None:- sensitive = "https://example.com/account?token=solver-secret#challenge"-- async def ladder(_page: RecordingPage, **_kwargs: object) -> bool:- raise CaptchaChallengeUnsolvedError(sensitive)-- monkeypatch.setattr(block_module, "solve_challenge_ladder", ladder)+ "outcome", [UNSOLVED, ChallengeOutcome(ChallengeStatus.UNSUPPORTED, "arkoselabs", None, "challenged")]+)+async def test_builtin_records_the_receipt_on_an_unsolved_failure(+ monkeypatch: pytest.MonkeyPatch, outcome: ChallengeOutcome+) -> None:+ monkeypatch.setattr(block_module, "solve_challenge", AsyncMock(return_value=outcome))page = RecordingPage(FakePage())with pytest.raises(CodeBlockCaptchaError, match="CAPTCHA could not be solved"):@@ -696,7 +710,7 @@ async def test_builtin_records_sanitized_unsolved_failure(monkeypatch: pytest.Moassert action.action_type == ActionType.SOLVE_CAPTCHAassert action.status == ActionStatus.failedassert action.response == "CodeBlockCaptchaError"- assert sensitive not in action.model_dump_json()+ assert action.output["challenge"] == outcome.receipt()@pytest.mark.asyncio@@ -704,10 +718,10 @@ async def test_builtin_records_only_the_unexpected_failure_type(monkeypatch: pytsensitive = "https://example.com/account?token=solver-secret#challenge"error = RuntimeError(sensitive)- async def ladder(_page: RecordingPage, **_kwargs: object) -> bool:+ async def ladder(_page: RecordingPage, **_kwargs: object) -> ChallengeOutcome:raise error- monkeypatch.setattr(block_module, "solve_challenge_ladder", ladder)+ monkeypatch.setattr(block_module, "solve_challenge", ladder)page = RecordingPage(FakePage())with pytest.raises(RuntimeError) as exc_info:@@ -724,12 +738,12 @@ async def test_builtin_records_only_the_unexpected_failure_type(monkeypatch: pytasync def test_authored_code_cannot_override_solver_workflow_run_id(monkeypatch: pytest.MonkeyPatch) -> None:ladder_calls = 0- async def ladder(_page: RecordingPage, **_kwargs: object) -> bool:+ async def ladder(_page: RecordingPage, **_kwargs: object) -> ChallengeOutcome:nonlocal ladder_callsladder_calls += 1- return False+ return ABSENT- monkeypatch.setattr(block_module, "solve_challenge_ladder", ladder)+ monkeypatch.setattr(block_module, "solve_challenge", ladder)page = RecordingPage(FakePage())block = CodeBlock.model_construct(code='await solve_captcha(page, workflow_run_id="wr_forged")',@@ -888,8 +902,8 @@ async def test_image_arm_stops_reading_at_the_per_block_limit(monkeypatch: pytesasync def test_solve_captcha_without_selectors_reaches_the_ladder_not_ocr(monkeypatch: pytest.MonkeyPatch) -> None:agent_function = OcrRecordingAgentFunction("XK7Q")monkeypatch.setattr(app, "AGENT_FUNCTION", agent_function)- ladder = AsyncMock(return_value=True)- monkeypatch.setattr(block_module, "solve_challenge_ladder", ladder)+ ladder = AsyncMock(return_value=SOLVED)+ monkeypatch.setattr(block_module, "solve_challenge", ladder)page, _image, text_box = _image_captcha_page()await _run_block_code("await solve_captcha(page)", page)@@ -929,8 +943,8 @@ async def test_solve_captcha_with_a_bad_selector_is_an_argument_error() -> None:agent_function = OcrRecordingAgentFunction("XK7Q")monkeypatch.setattr(app, "AGENT_FUNCTION", agent_function)- ladder = AsyncMock(return_value=True)- monkeypatch.setattr(block_module, "solve_challenge_ladder", ladder)+ ladder = AsyncMock(return_value=SOLVED)+ monkeypatch.setattr(block_module, "solve_challenge", ladder)page, image, text_box = _image_captcha_page()with pytest.raises(ValueError, match=message):@@ -1284,6 +1298,7 @@ async def test_nested_onscreen_match_after_an_offscreen_one_is_detected(monkeypaclass _WedgedFrame:url = "https://app.example/wedged"+ parent_frame = object()async def frame_element(self) -> None:await asyncio.Event().wait()@@ -1529,6 +1544,15 @@ async def test_ladder_honors_resolved_extension_timeout(await solve_challenge_ladder(FakePage(turnstile=True))+async def test_a_widened_extension_arm_is_still_cut_at_the_ladder_budget(monkeypatch: pytest.MonkeyPatch) -> None:+ monkeypatch.setattr(captcha_solver_module, "_LADDER_BUDGET_SECONDS", 0.02)+ monkeypatch.setattr(app, "AGENT_FUNCTION", _SlowExtensionSolverAgent(resolved=5))++ with pytest.raises(CaptchaChallengeUnsolvedError):+ await solve_challenge_ladder(FakePage(turnstile=True))++@pytest.mark.asyncio@pytest.mark.parametrize(("confirm", "expected_events"),@@ -1842,25 +1866,70 @@ class _WedgedChallengeFrame(_WedgedFrame):@pytest.mark.asyncioasync def test_wedged_challenge_frames_are_bounded_on_the_default_path(monkeypatch: pytest.MonkeyPatch) -> None:+ # An absent verdict costs one frame scan plus one rendered-frame probe, each spent once however many frames wedge.+ scan, probe = 0.3, 0.2monkeypatch.setattr(app, "AGENT_FUNCTION", AgentFunction())- monkeypatch.setattr(captcha_solver_module, "_CHILD_FRAME_SCAN_BUDGET_SECONDS", 0.3)+ monkeypatch.setattr(captcha_solver_module, "_CHILD_FRAME_SCAN_BUDGET_SECONDS", scan)+ monkeypatch.setattr(challenge_signature, "_CHALLENGE_FRAME_PROBE_TIMEOUT_SECONDS", probe)page = FakePage(frames=[_WedgedChallengeFrame() for _ in range(5)])started = time.monotonic()assert await solve_challenge_ladder(page) is False- assert time.monotonic() - started < 1.0+ assert time.monotonic() - started < scan + 2 * probe@pytest.mark.asyncioasync def test_the_opted_in_absent_path_runs_one_frame_scan(monkeypatch: pytest.MonkeyPatch) -> None:- # Two composed scans (challenge-host frames, then markers) would each spend the whole budget.+ # Two composed scans (challenge-host frames, then markers) would each spend the whole scan budget; the absent+ # verdict adds only the one rendered-frame probe.+ scan, probe = 0.3, 0.1monkeypatch.setattr(app, "AGENT_FUNCTION", AgentFunction())- monkeypatch.setattr(captcha_solver_module, "_CHILD_FRAME_SCAN_BUDGET_SECONDS", 0.3)+ monkeypatch.setattr(captcha_solver_module, "_CHILD_FRAME_SCAN_BUDGET_SECONDS", scan)+ monkeypatch.setattr(challenge_signature, "_CHALLENGE_FRAME_PROBE_TIMEOUT_SECONDS", probe)page = FakePage(frames=[_WedgedChallengeFrame()])started = time.monotonic()assert await solve_challenge_ladder(page, probe_child_frames=True) is False- assert time.monotonic() - started < 0.5+ assert time.monotonic() - started < scan + probe + 0.15+++async def test_a_deployment_without_a_vendor_probe_never_reports_the_page_clear(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ monkeypatch.setattr(app, "AGENT_FUNCTION", AgentFunction())++ outcome = await solve_challenge(FakePage())++ assert (outcome.status, outcome.page_state) == (ChallengeStatus.ABSENT, "not_rechecked")+++class _VendorClearsOnFirstRecheckAgent(AgentFunction):+ def __init__(self) -> None:+ super().__init__()+ self.reads = 0++ async def detect_vendor_challenge(self, page: Page | RecordingPage) -> ChallengeVendor | None:+ self.reads += 1+ return ChallengeVendor.DATADOME if self.reads == 1 else None+++async def test_a_vendor_path_that_spent_the_budget_cannot_report_the_page_solved(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ # The recheck leaves 0.1s of budget, and the wedged frame would hold the widget scan for a full second.+ monkeypatch.setattr(app, "AGENT_FUNCTION", _VendorClearsOnFirstRecheckAgent())+ monkeypatch.setattr(captcha_solver_module, "_LADDER_BUDGET_SECONDS", 0.3)+ monkeypatch.setattr(captcha_solver_module, "_VENDOR_CLEAR_POLL_SECONDS", 0.2)+ monkeypatch.setattr(captcha_solver_module, "_CHILD_FRAME_SCAN_BUDGET_SECONDS", 1.0)+ monkeypatch.setattr(challenge_signature, "_CHALLENGE_FRAME_PROBE_TIMEOUT_SECONDS", 0.05)++ started = time.monotonic()+ outcome = await solve_challenge(FakePage(frames=[_WedgedChallengeFrame()]), probe_child_frames=True)++ assert outcome.status is ChallengeStatus.UNSOLVED+ assert time.monotonic() - started < 0.6@pytest.mark.asynciodiff --git a/tests/unit/test_copilot_hooks.py b/tests/unit/test_copilot_hooks.pyindex 38ee3a01a..6b7862610 100644--- a/tests/unit/test_copilot_hooks.py+++ b/tests/unit/test_copilot_hooks.py@@ -61,6 +61,7 @@ from skyvern.forge.sdk.copilot.tools.scouting import ()from skyvern.forge.sdk.copilot.turn_halt import CopilotTurnHalt, TurnHaltKindfrom skyvern.webeye.persistent_sessions_manager import BrowserOperation, BrowserRetirement+from skyvern.webeye.utils import challenge_signature as challenge_signature_modulefrom tests.unit.copilot_test_helpers import (SENSITIVE_DISCLOSURE_WITHHOLDING_ARMS,FakeTabbedBrowserState,@@ -102,7 +103,7 @@ class _ListenerFrameElement:async def evaluate(self, script: str) -> float | bool | None:if self._frame.unresponsive:await asyncio.Event().wait()- if script == scouting_module._ELEMENT_STYLE_VISIBLE_JS:+ if script == challenge_signature_module._ELEMENT_STYLE_VISIBLE_JS:return self._frame.style_visiblereturn self._frame.area@@ -2588,14 +2589,14 @@ class TestScoutedInteractionCapture:) -> None:page = _ListenerPage(child_frame_urls=["https://existing.vendor.test/turnstile/v0/api.html"])existing = page.frames[1]- measure = scouting_module._challenge_frame_rendered_area+ measure = challenge_signature_module.challenge_frame_rendered_areaasync def _mount_while_measuring(frame: Any) -> float | None:if frame is existing and len(page.frames) == 2:page.navigate_frame("https://challenge.vendor.test/turnstile/v0/api.html", child_frame=True)return await measure(frame)- monkeypatch.setattr(scouting_module, "_challenge_frame_rendered_area", _mount_while_measuring)+ monkeypatch.setattr(challenge_signature_module, "challenge_frame_rendered_area", _mount_while_measuring)ctx, _clock, _sleeps = await self._timed_click(monkeypatch, page, elapsed=1.0, params=params)result = await self._post(ctx)@@ -2629,7 +2630,7 @@ class TestScoutedInteractionCapture:self, monkeypatch: pytest.MonkeyPatch) -> None:page = _ListenerPage()- measure = scouting_module._challenge_frame_rendered_area+ measure = challenge_signature_module.challenge_frame_rendered_areaasync def _mount_during_download_snapshot(_ctx: Any) -> frozenset[str]:page.navigate_frame("https://first.vendor.test/turnstile/v0/api.html", child_frame=True)@@ -2644,7 +2645,7 @@ class TestScoutedInteractionCapture:monkeypatch.setattr(mcp_hooks, "_scout_session_download_names", _mount_during_download_snapshot)monkeypatch.setattr(mcp_hooks, "_arm_scout_download_listener", AsyncMock())monkeypatch.setattr(mcp_hooks, "_arm_scout_popup_listener", AsyncMock())- monkeypatch.setattr(scouting_module, "_challenge_frame_rendered_area", _mount_while_measuring)+ monkeypatch.setattr(challenge_signature_module, "challenge_frame_rendered_area", _mount_while_measuring)ctx, _clock, _sleeps = await self._timed_click(monkeypatch, page, elapsed=1.0)result = await self._post(ctx)@@ -2659,13 +2660,13 @@ class TestScoutedInteractionCapture:) -> None:"""An intent click can resolve its target for seconds inside the tool before the browser clicks."""page = _ListenerPage(child_frame_urls=["https://existing.vendor.test/turnstile/v0/api.html"])- measure = scouting_module._challenge_frame_rendered_area+ measure = challenge_signature_module.challenge_frame_rendered_areactx, clock, sleeps = await self._timed_click(monkeypatch, page, elapsed=0.1, params={"intent": "search"})async def _record(seconds: float) -> None:sleeps.append(seconds)- monkeypatch.setattr(scouting_module, "_challenge_frame_rendered_area", measure)+ monkeypatch.setattr(challenge_signature_module, "challenge_frame_rendered_area", measure)monkeypatch.setattr(scouting_module, "asyncio", ScopedAsyncio(sleep=_record))clock.now = before_return@@ -2680,7 +2681,7 @@ class TestScoutedInteractionCapture:page = _ListenerPage(child_frame_urls=["https://existing.vendor.test/turnstile/v0/api.html"])existing = page.frames[1]mounted: list[_ListenerFrame] = []- measure = scouting_module._challenge_frame_rendered_area+ measure = challenge_signature_module.challenge_frame_rendered_areaasync def _mount_while_measuring(frame: Any) -> float | None:if frame is existing and not mounted:@@ -2688,7 +2689,7 @@ class TestScoutedInteractionCapture:mounted.append(page.frames[-1])return await measure(frame)- monkeypatch.setattr(scouting_module, "_challenge_frame_rendered_area", _mount_while_measuring)+ monkeypatch.setattr(challenge_signature_module, "challenge_frame_rendered_area", _mount_while_measuring)ctx, _clock, _sleeps = await self._timed_click(monkeypatch, page, elapsed=1.0)existing.area = 0.0mounted[0].area = _ON_SCREEN_AREA@@ -2907,7 +2908,7 @@ class TestScoutedInteractionCapture:in_flight -= 1return 1.0- monkeypatch.setattr(scouting_module, "_challenge_frame_rendered_area", _measure)+ monkeypatch.setattr(challenge_signature_module, "challenge_frame_rendered_area", _measure)page = _ListenerPage(child_frame_urls=["https://challenge.vendor.test/turnstile/v0/api.html"] * frames)ctx = await self._arm(monkeypatch, page)assert peak == frames, "before the click dispatches"@@ -2963,8 +2964,8 @@ class TestScoutedInteractionCapture:page = _ListenerPage(child_frame_urls=["https://challenge.vendor.test/turnstile/v0/api.html"])readings = [before]monkeypatch.setattr(- scouting_module,- "_challenge_frame_rendered_area",+ challenge_signature_module,+ "challenge_frame_rendered_area",AsyncMock(side_effect=lambda _frame: readings.pop(0) if readings else after),)ctx = await self._arm(monkeypatch, page)@@ -2979,7 +2980,7 @@ class TestScoutedInteractionCapture:page.frames.append(_ListenerFrame("https://challenge.vendor.test/turnstile/v0/api.html", page.frames[0], unresponsive=True))- monkeypatch.setattr(scouting_module, "_CHALLENGE_FRAME_PROBE_TIMEOUT_SECONDS", 0.05)+ monkeypatch.setattr(challenge_signature_module, "_CHALLENGE_FRAME_PROBE_TIMEOUT_SECONDS", 0.05)ctx = await asyncio.wait_for(self._arm(monkeypatch, page), timeout=5)diff --git a/tests/unit/test_copilot_page_challenge.py b/tests/unit/test_copilot_page_challenge.pyindex 800178123..98e615abb 100644--- a/tests/unit/test_copilot_page_challenge.py+++ b/tests/unit/test_copilot_page_challenge.py@@ -18,13 +18,16 @@ from skyvern.forge.sdk.copilot.runtime import (CopilotBrowserSessionUnavailable,)from skyvern.forge.sdk.copilot.tools import _shared, page_challenge-from skyvern.webeye.utils.captcha_solver import MAX_IMAGE_CAPTCHA_READS, CaptchaChallengeUnsolvedError+from skyvern.webeye.utils.captcha_solver import MAX_IMAGE_CAPTCHA_READS, ChallengeOutcome, ChallengeStatusfrom tests.unit.conftest import OcrRecordingAgentFunctionfrom tests.unit.test_code_block_captcha import _skip_no_browserfrom tests.unit.test_copilot_hooks import _ListenerPagefrom tests.unit.test_copilot_runtime import _FakeBrowserContext, _make_ctx-_VENDOR_FRAME = "https://challenge.vendor.test/turnstile/v0/api.html"+_SOLVED = ChallengeOutcome(ChallengeStatus.SOLVED, "perimeterx", "vendor_handler", "clear")+_ABSENT = ChallengeOutcome(ChallengeStatus.ABSENT, page_state="clear")+_UNSUPPORTED = ChallengeOutcome(ChallengeStatus.UNSUPPORTED, "arkoselabs", None, "challenged")+_UNSOLVED = ChallengeOutcome(ChallengeStatus.UNSOLVED, "datadome", "vendor_handler", "challenged")_UNTRIED_FRESH_BROWSER = "start_fresh_browser: a new browser session with no cookies, storage or challenge history"@@ -59,7 +62,7 @@ def _chat(monkeypatch.setattr(app, "AGENT_FUNCTION", agent_function)monkeypatch.setattr(_shared, "mcp_browser_context", _admitted)monkeypatch.setattr(_shared, "live_working_page", AsyncMock(return_value=page))- monkeypatch.setattr(page_challenge, "solve_challenge_ladder", ladder)+ monkeypatch.setattr(page_challenge, "solve_challenge", ladder)ctx = _make_ctx()ctx.browser_session_id = "pbs_chat"return ctx@@ -72,41 +75,43 @@ async def _hang(*_args: object, **_kwargs: object) -> bool:@pytest.mark.asyncio@pytest.mark.parametrize(- ("ladder", "frames", "available", "expected"),+ ("ladder", "available", "expected"),[- (AsyncMock(return_value=True), [], True, {"outcome": "solved"}),- (AsyncMock(return_value=False), [], True, {"outcome": "none"}),+ (AsyncMock(return_value=_SOLVED), True, {"outcome": "solved", "challenge": _SOLVED.receipt()}),+ (AsyncMock(return_value=_ABSENT), True, {"outcome": "none", "challenge": _ABSENT.receipt()}),(- AsyncMock(return_value=False),- [_VENDOR_FRAME],+ AsyncMock(return_value=_UNSUPPORTED),True,- {"outcome": "unsupported", "untried_in_this_request": [_UNTRIED_FRESH_BROWSER]},+ {+ "outcome": "unsupported",+ "challenge": _UNSUPPORTED.receipt(),+ "untried_in_this_request": [_UNTRIED_FRESH_BROWSER],+ },),(- AsyncMock(side_effect=CaptchaChallengeUnsolvedError()),- [],+ AsyncMock(return_value=_UNSOLVED),True,{"outcome": "unsolved",+ "challenge": _UNSOLVED.receipt(),"unsolved_challenges_in_this_browser_session": 1,"untried_in_this_request": [_UNTRIED_FRESH_BROWSER],},),- (AsyncMock(side_effect=_hang), [], True, {"outcome": "unsolved", "timed_out": True}),- (AsyncMock(side_effect=RuntimeError("boom")), [], True, {"outcome": "unsolved", "solver_failed": True}),- (AsyncMock(return_value=True), [], False, {"outcome": "unavailable"}),+ (AsyncMock(side_effect=_hang), True, {"outcome": "unsolved", "timed_out": True}),+ (AsyncMock(side_effect=RuntimeError("boom")), True, {"outcome": "unsolved", "solver_failed": True}),+ (AsyncMock(return_value=_SOLVED), False, {"outcome": "unavailable"}),],- ids=["solved", "none", "vendor_on_screen_but_undetected", "unsolved", "timed_out", "solver_failed", "unavailable"],+ ids=["solved", "none", "unsupported", "unsolved", "timed_out", "solver_failed", "unavailable"],)async def test_each_ladder_result_is_reported_as_its_own_outcome(monkeypatch: pytest.MonkeyPatch,ladder: AsyncMock,- frames: list[str],available: bool,expected: dict[str, Any],) -> None:monkeypatch.setattr(page_challenge, "SOLVE_CEILING_SECONDS", 0.05)- ctx = _chat(monkeypatch, _Page(frames), ladder, available=available)+ ctx = _chat(monkeypatch, _Page(), ladder, available=available)result = await page_challenge.solve_page_challenge(ctx)@@ -121,7 +126,7 @@ async def test_a_browser_lost_during_the_solve_is_reported_as_session_loss_not_u) -> None:page = _Page()page.closed = True- ctx = _chat(monkeypatch, page, AsyncMock(side_effect=CaptchaChallengeUnsolvedError()))+ ctx = _chat(monkeypatch, page, AsyncMock(return_value=_UNSOLVED))disposition = AsyncMock(return_value="reestablished")monkeypatch.setattr(_shared, "_browser_session_error_disposition", disposition)@@ -138,7 +143,7 @@ async def test_a_browser_lost_during_the_solve_is_reported_as_session_loss_not_uasync def test_unsolved_attempts_are_counted_per_browser_and_a_new_browser_starts_over(monkeypatch: pytest.MonkeyPatch,) -> None:- ladder = AsyncMock(side_effect=CaptchaChallengeUnsolvedError())+ ladder = AsyncMock(return_value=_UNSOLVED)ctx = _chat(monkeypatch, _Page(), ladder)for attempt in (1, 2):@@ -168,7 +173,7 @@ async def test_unsolved_attempts_are_counted_per_browser_and_a_new_browser_startasync def test_a_sensitive_page_is_never_sent_to_the_solver(monkeypatch: pytest.MonkeyPatch, taint_field: str, expected_error: str) -> None:- ladder = AsyncMock(return_value=True)+ ladder = AsyncMock(return_value=_SOLVED)ctx = _chat(monkeypatch, _Page(), ladder)setattr(ctx, taint_field, {"pbs_chat"})diff --git a/tests/unit/workflow/test_code_block_recorder.py b/tests/unit/workflow/test_code_block_recorder.pyindex badca4620..ddee6889b 100644--- a/tests/unit/workflow/test_code_block_recorder.py+++ b/tests/unit/workflow/test_code_block_recorder.py@@ -81,6 +81,7 @@ from skyvern.webeye.playwright_input import (playwright_input_defaults_for_page,register_playwright_input_context,)+from skyvern.webeye.utils.captcha_solver import ChallengeOutcome, ChallengeStatusclass FakeFrame:@@ -1798,15 +1799,15 @@ async def test_code_block_solver_lifecycle_streams_one_persisted_action(context = FakeWorkflowRunContext(secrets={"customer_path": "/account?token=solver-secret#challenge"})mocks = _patch_execute_environment(monkeypatch, page, context)- async def ladder(recording_page: RecordingPage, **_kwargs: object) -> bool:+ async def ladder(recording_page: RecordingPage, **_kwargs: object) -> ChallengeOutcome:await recording_page.locator("#challenge").click()if outcome == "caught_unsolved":- raise block_module.CaptchaChallengeUnsolvedError(- "https://example.com/account?token=solver-secret#challenge"- )- return outcome == "solved"+ return ChallengeOutcome(ChallengeStatus.UNSOLVED)+ if outcome == "solved":+ return ChallengeOutcome(ChallengeStatus.SOLVED, arm="extension")+ return ChallengeOutcome(ChallengeStatus.ABSENT, page_state="clear")- monkeypatch.setattr(block_module, "solve_challenge_ladder", ladder)+ monkeypatch.setattr(block_module, "solve_challenge", ladder)code = ("try:\n solver_result = await solve_captcha(page)\nexcept Exception:\n code_continued = True"if outcome == "caught_unsolved"@@ -1851,7 +1852,7 @@ async def test_code_block_solver_cancellation_streams_failed_action_and_propagatasync def cancel(_recording_page: RecordingPage, **_kwargs: object) -> bool:raise asyncio.CancelledError()- monkeypatch.setattr(block_module, "solve_challenge_ladder", cancel)+ monkeypatch.setattr(block_module, "solve_challenge", cancel)with pytest.raises(asyncio.CancelledError):await _make_code_block("await solve_captcha(page)", goal="handle challenge").execute(workflow_run_id="wr_test",