Skip to content

The Postgres development platform. Supabase gives you a dedicated Postgres database to build your web, mobile, and AI applications.

test(assistant): add eval case for `execute_sql` usage on default "Generate sample data" prompt (#42219)

Matt Rossmana127f2cbbc1c
diff --git a/apps/studio/evals/dataset.ts b/apps/studio/evals/dataset.ts
index 6e5d73bb273..a73e8638ba5 100644
--- a/apps/studio/evals/dataset.ts
+++ b/apps/studio/evals/dataset.ts
@@ -94,4 +94,16 @@ export const dataset: AssistantEvalCase[] = [
'Uses quotes around schema/table/columns with capital letters, special characters, and reserved keywords.',
},
},
+ {
+ input: {
+ prompt: 'Generate sample data for a blog with users, posts, and comments tables',
+ },
+ expected: {
+ requiredTools: ['execute_sql'],
+ },
+ metadata: {
+ category: ['sql_generation', 'schema_design'],
+ description: 'Invokes `execute_sql` from default "Generate sample data" prompt',
+ },
+ },
]