Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

fix(aio): say when backfill-estimate rejects a window (#129141)

Bernat Torres5bc18dfec8e2
diff --git a/products/ai_observability/mcp/tools.yaml b/products/ai_observability/mcp/tools.yaml
index dbeec22f4a3a..68db78f22e9f 100644
--- a/products/ai_observability/mcp/tools.yaml
+++ b/products/ai_observability/mcp/tools.yaml
@@ -479,8 +479,13 @@ tools:
anything. Always call this before llma-evaluation-backfill-create and confirm the count with the user.
total_units is what the backfill would evaluate. already_evaluated_units already have a result and are left
out unless rerun_existing is true. The window is clamped to the last 30 days and held back by the
- evaluation's settle time, so report the returned window_start and window_end, not the requested ones.
- Conditions default to the evaluation's own.
+ evaluation's settle time, so report the returned window_start and window_end, not the requested ones. A
+ trace or session evaluation only backfills units older than its settle time: the settle window under
+ fixed_window, or the max age under inactivity (up to 2 hours for traces, 7 days for sessions). A window that
+ lies entirely inside the settle time, entirely before the last 30 days, or beyond the project's
+ drop-events-older-than setting is rejected rather than clamped; the error says which limit applied.
+ Conditions default to the evaluation's own. A backfill needs at least one condition set, and every property
+ filter needs a value, so pass conditions when the evaluation has none or is still a draft.
param_overrides:
evaluation_id:
aliases:
diff --git a/services/mcp/schema/generated-tool-definitions.json b/services/mcp/schema/generated-tool-definitions.json
index ed86b7afd39d..dc81a9fecd75 100644
--- a/services/mcp/schema/generated-tool-definitions.json
+++ b/services/mcp/schema/generated-tool-definitions.json
@@ -9013,7 +9013,7 @@
}
},
"llma-evaluation-backfill-estimate": {
- "description": "Count how many units a backfill of an evaluation would evaluate over a time window, without starting anything. Always call this before llma-evaluation-backfill-create and confirm the count with the user. total_units is what the backfill would evaluate. already_evaluated_units already have a result and are left out unless rerun_existing is true. The window is clamped to the last 30 days and held back by the evaluation's settle time, so report the returned window_start and window_end, not the requested ones. Conditions default to the evaluation's own.",
+ "description": "Count how many units a backfill of an evaluation would evaluate over a time window, without starting anything. Always call this before llma-evaluation-backfill-create and confirm the count with the user. total_units is what the backfill would evaluate. already_evaluated_units already have a result and are left out unless rerun_existing is true. The window is clamped to the last 30 days and held back by the evaluation's settle time, so report the returned window_start and window_end, not the requested ones. A trace or session evaluation only backfills units older than its settle time: the settle window under fixed_window, or the max age under inactivity (up to 2 hours for traces, 7 days for sessions). A window that lies entirely inside the settle time, entirely before the last 30 days, or beyond the project's drop-events-older-than setting is rejected rather than clamped; the error says which limit applied. Conditions default to the evaluation's own. A backfill needs at least one condition set, and every property filter needs a value, so pass conditions when the evaluation has none or is still a draft.",
"category": "AI observability",
"feature": "llm_analytics",
"summary": "Estimate evaluation backfill",
diff --git a/services/mcp/schema/tool-definitions-all.json b/services/mcp/schema/tool-definitions-all.json
index 4ed3231c5241..ab7ce06cba89 100644
--- a/services/mcp/schema/tool-definitions-all.json
+++ b/services/mcp/schema/tool-definitions-all.json
@@ -9279,7 +9279,7 @@
}
},
"llma-evaluation-backfill-estimate": {
- "description": "Count how many units a backfill of an evaluation would evaluate over a time window, without starting anything. Always call this before llma-evaluation-backfill-create and confirm the count with the user. total_units is what the backfill would evaluate. already_evaluated_units already have a result and are left out unless rerun_existing is true. The window is clamped to the last 30 days and held back by the evaluation's settle time, so report the returned window_start and window_end, not the requested ones. Conditions default to the evaluation's own.",
+ "description": "Count how many units a backfill of an evaluation would evaluate over a time window, without starting anything. Always call this before llma-evaluation-backfill-create and confirm the count with the user. total_units is what the backfill would evaluate. already_evaluated_units already have a result and are left out unless rerun_existing is true. The window is clamped to the last 30 days and held back by the evaluation's settle time, so report the returned window_start and window_end, not the requested ones. A trace or session evaluation only backfills units older than its settle time: the settle window under fixed_window, or the max age under inactivity (up to 2 hours for traces, 7 days for sessions). A window that lies entirely inside the settle time, entirely before the last 30 days, or beyond the project's drop-events-older-than setting is rejected rather than clamped; the error says which limit applied. Conditions default to the evaluation's own. A backfill needs at least one condition set, and every property filter needs a value, so pass conditions when the evaluation has none or is still a draft.",
"category": "AI observability",
"feature": "llm_analytics",
"summary": "Estimate evaluation backfill",