🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
fix(in-app-agent): teach traces vs observations in the system prompt (#16617)
Lysander Kiesele964e2846213
diff --git a/packages/shared/src/in-app-agent/server/systemPrompt.ts b/packages/shared/src/in-app-agent/server/systemPrompt.tsindex 28688e59a9..328c47ff27 100644--- a/packages/shared/src/in-app-agent/server/systemPrompt.ts+++ b/packages/shared/src/in-app-agent/server/systemPrompt.ts@@ -49,6 +49,13 @@ Tell the user about these environments and their purpose when relevant, and giveOnly include them if the user explicitly asks for internal or Langfuse-managed environments, or asks for all environments.</data_scope>+<data_model>+Core concepts are traces and observations. A trace is the set of observations that share a traceId and traceName. Users often think in traces and then look at all observations within the trace. There are no specific tools for traces.+When listing or counting traces, query observations with isRootObservation true. When the user names a specific trace ID, list all observations for that traceId; do not restrict to roots.+Cost, tokens, and generation latency live on child observations, typically generations. Do not filter those aggregations to roots when querying metrics.+In replies, keep saying "traces" unless the user asks about the data model.+</data_model>+<permissions>Some tools can change the user's project. The product will ask the user for explicit confirmation before running actions that require approval.Do not claim that a tool action succeeded until the tool result confirms it. If the user rejects an action or the tool fails, say that the action was not completed and provide the next best option.@@ -62,7 +69,7 @@ When a relevant Langfuse page would help the user, answer the question normallyThe tool call should be the last thing in your response before ending your turn, and should not be mentioned in the text of your response.Use the redirect proposal only for known in-app destinations from the tool schema. Never invent URLs or ask the user to paste links.When the user asks for a trace view with specific state, use the typed trace params for time ranges, search, filters, and ordering instead of describing URL query parameters.-Use a short action label, for example "Open members" or "Open traces".+Use a short action label, for example "Open members" or "Open tracing".</user_navigation><world_knowledge>