🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
fix(evals): use schema-compatible model preflight prompt (#18252)
Tobias Wochingerd29bd6acd644
diff --git a/packages/shared/src/server/llm/testModelCall.ts b/packages/shared/src/server/llm/testModelCall.tsindex adff7165bb..753dcc7be2 100644--- a/packages/shared/src/server/llm/testModelCall.ts+++ b/packages/shared/src/server/llm/testModelCall.ts@@ -43,7 +43,7 @@ export const testModelCall = async ({{role: ChatMessageRole.User,content:- 'Extract a score (1-5) and reasoning from this text: "This is a test. It worked perfectly because it matched all passing criteria."',+ "Generate one example response that conforms to the provided JSON schema. Respect the type, allowed values, and bounds of every field. Keep any reasoning brief.",type: ChatMessageType.User,},],