Automate browser based workflows with AI
Python aiapiautomationbrowserbrowser-automationcomputergptllmplaywrightpowerautomatepuppeteerpythonrpaseleniumvisionworkflow
use the main prompt to generate the observer task url and title (#1594)
Shuchang Zhengdc96595e7acd
diff --git a/skyvern/forge/prompts/skyvern/observer_generate_metadata.j2 b/skyvern/forge/prompts/skyvern/observer_generate_metadata.j2index b23f58fbf..33977f2d8 100644--- a/skyvern/forge/prompts/skyvern/observer_generate_metadata.j2+++ b/skyvern/forge/prompts/skyvern/observer_generate_metadata.j2@@ -4,8 +4,8 @@ MAKE SURE YOU OUTPUT VALID JSON. No text before or after JSON, no trailing commaReply in JSON format with the following keys:{- "thoughts": str, // Think step by step. If the user specifically mentioned which website it is to use, let's use that url. If the user doesn't specifify a specific website, to achieve what the user wants to do, what is most likely website url for achieving the goal?- "url": str, // The url to type into the browser. DO NOT change the url if user specified one.+ "thoughts": str, // Think step by step. Has the user specify a url to go? If yes, what is the complete url user specified? If not, to achieve what the user wants to do, what is most likely website url for achieving the goal?+ "url": str, // The initial url to type into the browser. If the user specified one, use exactly that url."title": str, // A descriptive and informative title for the goal. Use no more than 5 words}diff --git a/skyvern/forge/sdk/services/observer_service.py b/skyvern/forge/sdk/services/observer_service.pyindex 235ab55ed..19424e54a 100644--- a/skyvern/forge/sdk/services/observer_service.py+++ b/skyvern/forge/sdk/services/observer_service.py@@ -99,7 +99,7 @@ async def initialize_observer_cruise()metadata_prompt = prompt_engine.load_prompt("observer_generate_metadata", user_goal=user_prompt, user_url=user_url)- metadata_response = await app.SECONDARY_LLM_API_HANDLER(prompt=metadata_prompt, observer_thought=observer_thought)+ metadata_response = await app.LLM_API_HANDLER(prompt=metadata_prompt, observer_thought=observer_thought)# validateLOG.info(f"Initialized observer initial response: {metadata_response}")url: str = user_url or metadata_response.get("url", "")