Automate browser based workflows with AI
Python aiapiautomationbrowserbrowser-automationcomputergptllmplaywrightpowerautomatepuppeteerpythonrpaseleniumvisionworkflow
Add any information that helps from the last message of CUA when there's no data extraction goal (#2149)
Shuchang Zheng1ed5f22200da
diff --git a/skyvern/forge/prompts/skyvern/cua-fallback-action.j2 b/skyvern/forge/prompts/skyvern/cua-fallback-action.j2index 78b9589e5..dda38c469 100644--- a/skyvern/forge/prompts/skyvern/cua-fallback-action.j2+++ b/skyvern/forge/prompts/skyvern/cua-fallback-action.j2@@ -12,6 +12,7 @@ Help the user decide what to do next based on the assistant's message. Here's thReturn the action to take next in the following JSON format:{"action": str // complete, terminate, solve_captcha, get_verification_code+ "useful_information": str // If there is any useful information the assistant has provided that contributes to the user goal, put it here.}User goal: {{ navigation_goal }}diff --git a/skyvern/webeye/actions/parse_actions.py b/skyvern/webeye/actions/parse_actions.pyindex 3ef46893e..3c4235509 100644--- a/skyvern/webeye/actions/parse_actions.py+++ b/skyvern/webeye/actions/parse_actions.py@@ -331,12 +331,19 @@ async def parse_cua_actions(prompt_name="cua-fallback-action",)skyvern_action_type = action_response.get("action")+ useful_information = action_response.get("useful_information")action = WaitAction(seconds=5,reasoning=reasoning,intention=reasoning,)if skyvern_action_type == "complete":+ if not task.data_extraction_goal and useful_information:+ await app.DATABASE.update_task(+ task.task_id,+ organization_id=task.organization_id,+ extracted_information=useful_information,+ )action = CompleteAction(reasoning=reasoning,intention=reasoning,