Skip to content

Automate browser based workflows with AI

Change ai suggestion to take user input and context (#1620)

Shuchang Zheng6123b1d5f045
diff --git a/skyvern/forge/prompts/skyvern/suggest-data-schema.j2 b/skyvern/forge/prompts/skyvern/suggest-data-schema.j2
index b8ea050b0..edc0938ba 100644
--- a/skyvern/forge/prompts/skyvern/suggest-data-schema.j2
+++ b/skyvern/forge/prompts/skyvern/suggest-data-schema.j2
@@ -1,35 +1,58 @@
-You are given an input string from a user. This string is a data extraction goal for an AI agent. It tells the agent what to do on a web page.
+We are developing an interface for AI agent tasks that use JSON schemas to describe the shape of the data to that needs to extracted from a web page.
-A data extraction goal describes what data to extract from the page.
+You are given an input prompt from a user, and some additional context. Your goal is to generate a JSON schema given the user prompt and the context.
-Your goal when given an input data extraction goal is to provide a JSONC schema describing a shape for the data to be extracted.
+If additional context is given, try to use the it for further clues about the data that needs to be extracted. For example, the user might provide some detail about
+product information to be extracted in a "data_extraction_goal" inside the context, but maybe not necessarily pass it in the input prompt. In these cases, you should use the
+context.
-Good data schema examples:
+Here is an example:
+
+User prompt: Generate a data schema that extracts the title and link for the posts as a list
+Additional context:
+```json
+{
+ "url": "https://news.ycombinator.com",
+ "data_extraction_goal": "Extract the title and link of the top 5 posts",
+ "existing_schema": "null"
+}
+```
-Input data extraction goal: "Extract the title and link of the top post on Hacker News."
Suggested Data Schema:
```json
{
- "type": "object",
- "properties": {
- "title": {
- "type": "string",
- "description": "The title of the top post on Hacker News."
- },
- "link": {
- "type": "string",
- "format": "uri",
- "description": "The URL link to the top post on Hacker News."
+ "posts" : {
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "title": {
+ "type": "string",
+ "description": "Title of the post"
+ },
+ "link": {
+ "type": "string",
+ "description": "Link to the post"
+ }
+ },
+ "required": [
+ "title",
+ "link"
+ ]
}
- },
- "required": [
- "title",
- "link"
- ]
+ }
}
```
-Respond only with JSON output containing a single key "output" with the value of the suggested data schema given the following input data extraction goal:
-```
+{% if additional_context %}
+
+You are provided some additional context about the suggestion here:
+
+{{additional_context}}
+
+{% endif %}
+
+Respond only with JSON output containing a single key "output" with the value of the suggested data schema given the following input:
+
{{ input }}
-```
+
diff --git a/skyvern/forge/sdk/routes/agent_protocol.py b/skyvern/forge/sdk/routes/agent_protocol.py
index 8ea823800..352995273 100644
--- a/skyvern/forge/sdk/routes/agent_protocol.py
+++ b/skyvern/forge/sdk/routes/agent_protocol.py
@@ -956,7 +956,7 @@ async def make_ai_suggestion(
llm_prompt = ""
if ai_suggestion_type == AISuggestionType.DATA_SCHEMA:
- llm_prompt = prompt_engine.load_prompt("suggest-data-schema", input=data.input)
+ llm_prompt = prompt_engine.load_prompt("suggest-data-schema", input=data.input, additional_context=data.context)
try:
new_ai_suggestion = await app.DATABASE.create_ai_suggestion(
diff --git a/skyvern/forge/sdk/schemas/ai_suggestions.py b/skyvern/forge/sdk/schemas/ai_suggestions.py
index 00608a947..8e8d51e4d 100644
--- a/skyvern/forge/sdk/schemas/ai_suggestions.py
+++ b/skyvern/forge/sdk/schemas/ai_suggestions.py
@@ -20,3 +20,4 @@ class AISuggestion(AISuggestionBase):
class AISuggestionRequest(BaseModel):
input: str = Field(..., min_length=1)
+ context: dict[str, Any] | None = None