Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(experiments): reject recalculations of experiments older than 180 days (#120585)

Rodrigo Ilorod0d95dcc41e7
diff --git a/frontend/src/scenes/experiments/MetricsView/new/Metrics.tsx b/frontend/src/scenes/experiments/MetricsView/new/Metrics.tsx
index b5e071e0e7f0..ff11c88d152c 100644
--- a/frontend/src/scenes/experiments/MetricsView/new/Metrics.tsx
+++ b/frontend/src/scenes/experiments/MetricsView/new/Metrics.tsx
@@ -1,7 +1,7 @@
import { useValues } from 'kea'
import { IconInfo } from '@posthog/icons'
-import { Tooltip } from '@posthog/lemon-ui'
+import { LemonBanner, Tooltip } from '@posthog/lemon-ui'
import { IconAreaChart } from 'lib/lemon-ui/icons'
@@ -38,6 +38,7 @@ function MetricsContent({ experiment, isSecondary }: { experiment: Experiment; i
primaryMetricsResultsErrors,
secondaryMetricsResults,
secondaryMetricsResultsErrors,
+ recalculationRejectionMessage,
} = useValues(experimentMetricsLogic({ experiment }))
const type = isSecondary ? 'secondary' : 'primary'
@@ -88,6 +89,11 @@ function MetricsContent({ experiment, isSecondary }: { experiment: Experiment; i
</div>
</div>
</div>
+ {!isSecondary && recalculationRejectionMessage && (
+ <LemonBanner type="warning" className="mb-2">
+ {recalculationRejectionMessage}
+ </LemonBanner>
+ )}
{metrics.length > 0 ? (
detailResult ? (
<div className="rounded-md border bg-surface-primary overflow-hidden divide-y divide-border">
diff --git a/frontend/src/scenes/experiments/experimentMetricsLogic.test.ts b/frontend/src/scenes/experiments/experimentMetricsLogic.test.ts
index bc52d42bcbc8..eb3936c686d2 100644
--- a/frontend/src/scenes/experiments/experimentMetricsLogic.test.ts
+++ b/frontend/src/scenes/experiments/experimentMetricsLogic.test.ts
@@ -110,6 +110,28 @@ const partialFailureRecalculation = {
],
}
+// A run the backend rejected because the experiment started too long ago. The create response has no
+// results; the latest read carries the earlier run's rows under the same window.
+const REJECTION_MESSAGE = 'This experiment started too long ago for its metrics to be recalculated.'
+const windowError = {
+ step: 'request',
+ message: REJECTION_MESSAGE,
+ error_type: 'recalculation_window_too_long',
+ retriable: false,
+}
+const rejectedRecalculation = {
+ ...baseRecalculation,
+ id: 'recalc-rejected',
+ status: 'failed',
+ completed_at: new Date().toISOString(),
+ metric_errors: { [PRIMARY_METRIC_UUID]: windowError, [SECONDARY_METRIC_UUID]: windowError },
+}
+const rejectedLatestRecalculation = {
+ ...rejectedRecalculation,
+ query_to: completedRecalculation.query_to,
+ results: completedRecalculation.results,
+}
+
// Timeseries cold-start placeholder: completed-status, primary filled from timeseries, secondary is a gap.
const timeseriesFallbackRecalculation = {
...baseRecalculation,
@@ -752,6 +774,81 @@ describe('experimentMetricsLogic', () => {
expect(createMock).not.toHaveBeenCalled()
})
+ it('keeps the earlier results and shows the message when the backend rejects the run', async () => {
+ let latest: object = completedRecalculation
+ useMocks({
+ get: { '/api/projects/:team_id/experiments/:id/metrics_recalculation/latest/': () => [200, latest] },
+ post: {
+ '/api/projects/:team_id/experiments/:id/metrics_recalculation/': () => [201, rejectedRecalculation],
+ },
+ })
+ mountLogic()
+ await expectLogic(logic).toDispatchActions(['setCurrentRecalculation', 'setPrimaryMetricsResults'])
+
+ latest = rejectedLatestRecalculation
+ await expectLogic(logic, () => {
+ logic.actions.triggerRecalculation('manual')
+ })
+ .toDispatchActions(['setCurrentRecalculation', 'loadLatestRecalculation', 'setCurrentRecalculation'])
+ .toNotHaveDispatchedActions(['pollRecalculation'])
+
+ expect(logic.values.recalculationRejectionMessage).toEqual(REJECTION_MESSAGE)
+ expect(logic.values.primaryMetricsResults[0]).toEqual(primaryResult)
+ expect(logic.values.primaryMetricsResultsErrors[0]).toBeNull()
+ expect(logic.values.secondaryMetricsResults[0]).toEqual(secondaryResult)
+ expect(logic.values.recalculatingMetricUuids).toEqual([])
+ })
+
+ it('does not heal a gap in a rejected run, because every heal gets the same rejection', async () => {
+ const createMock = jest.fn(() => [201, rejectedRecalculation])
+ useMocks({
+ get: {
+ '/api/projects/:team_id/experiments/:id/metrics_recalculation/latest/': () => [
+ 200,
+ {
+ ...rejectedLatestRecalculation,
+ metric_errors: { [PRIMARY_METRIC_UUID]: windowError },
+ results: completedRecalculation.results.slice(0, 1),
+ },
+ ],
+ },
+ post: { '/api/projects/:team_id/experiments/:id/metrics_recalculation/': createMock },
+ })
+ mountLogic()
+ await expectLogic(logic).toDispatchActions(['setCurrentRecalculation', 'setPrimaryMetricsResults'])
+
+ await expectLogic(logic, () => {
+ logic.actions.triggerRecalculation('manual')
+ }).toDispatchActions(['loadLatestRecalculation', 'setCurrentRecalculation'])
+ await new Promise((resolve) => setTimeout(resolve, 0))
+
+ expect(createMock).toHaveBeenCalledTimes(1)
+ })
+
+ it('clears the metric loading marks when the reload after a rejection fails', async () => {
+ let latestStatus = 200
+ useMocks({
+ get: {
+ '/api/projects/:team_id/experiments/:id/metrics_recalculation/latest/': () => [
+ latestStatus,
+ completedRecalculation,
+ ],
+ },
+ post: {
+ '/api/projects/:team_id/experiments/:id/metrics_recalculation/': () => [201, rejectedRecalculation],
+ },
+ })
+ mountLogic()
+ await expectLogic(logic).toDispatchActions(['setCurrentRecalculation', 'setPrimaryMetricsResults'])
+
+ latestStatus = 500
+ await expectLogic(logic, () => {
+ logic.actions.triggerRecalculation('manual')
+ }).toDispatchActions(['loadLatestRecalculation', 'setRecalculationLoading'])
+
+ expect(logic.values.recalculatingMetricUuids).toEqual([])
+ })
+
it('creates a recalculation for a stopped experiment to compute its final results', async () => {
const stopped = { ...EXPERIMENT, status: undefined, end_date: '2025-01-01T00:00:00Z' } as Experiment
const createMock = jest.fn(() => [201, pendingRecalculation])
diff --git a/frontend/src/scenes/experiments/experimentMetricsLogic.ts b/frontend/src/scenes/experiments/experimentMetricsLogic.ts
index f6b32ee6caa4..57ea35602b9f 100644
--- a/frontend/src/scenes/experiments/experimentMetricsLogic.ts
+++ b/frontend/src/scenes/experiments/experimentMetricsLogic.ts
@@ -121,6 +121,9 @@ export const RECALCULATION_STATUSES = {
failed: 'failed',
} as const
+/** pinned: the backend writes this error_type on a run it rejects because the experiment is too old. */
+const RECALCULATION_WINDOW_TOO_LONG_ERROR_TYPE = 'recalculation_window_too_long'
+
/** Transient per-metric retry state written by the calc activity between failed attempts. */
export interface MetricRetryInfo {
attempt: number
@@ -156,6 +159,11 @@ type MetricErrorEntry = { step?: string; message?: string; error_type?: string;
const metricErrorEntries = (recalculation: RecalculationPayload): Record<string, MetricErrorEntry> =>
(recalculation.metric_errors as Record<string, MetricErrorEntry> | null) ?? {}
+const rejectionMessage = (recalculation: RecalculationPayload): string | null =>
+ Object.values(metricErrorEntries(recalculation)).find(
+ (entry) => entry.error_type === RECALCULATION_WINDOW_TOO_LONG_ERROR_TYPE
+ )?.message ?? null
+
/**
* Metric uuids the run has an entry for: a result or an error. A failure, retriable or not, counts as
* covered, because only a manual retry or a manual recalculation runs a failed metric again.
@@ -234,18 +242,24 @@ const resolveResultByUuid = (
/**
* Resolver: a metric's failure as `{ detail, retriable }` (the MetricErrorState shape), or null. `metric_errors`
* wins (it covers FAILED rows AND discovery-step failures absent from `results`), falling back to a
- * failed row's error_message.
+ * failed row's error_message. A rejected run carries the earlier run's rows, so its error is dropped on a
+ * metric that has a row: the banner shows the message and the cell keeps the earlier result.
*/
const resolveErrorByUuid = (recalculation: RecalculationPayload): ResolveByUuid<MetricErrorState> => {
const metricErrors = metricErrorEntries(recalculation)
+ const uuidsWithRow = new Set((recalculation.results ?? []).map((r) => r.metric_uuid))
const failedResultMessageByUuid = new Map(
(recalculation.results ?? [])
.filter((r) => r.status === 'failed' && r.error_message)
.map((r) => [r.metric_uuid, r.error_message as string])
)
return (uuid) => {
- const message = metricErrors[uuid]?.message ?? failedResultMessageByUuid.get(uuid)
- return message ? { detail: message, retriable: metricErrors[uuid]?.retriable === true } : null
+ const entry =
+ metricErrors[uuid]?.error_type === RECALCULATION_WINDOW_TOO_LONG_ERROR_TYPE && uuidsWithRow.has(uuid)
+ ? undefined
+ : metricErrors[uuid]
+ const message = entry?.message ?? failedResultMessageByUuid.get(uuid)
+ return message ? { detail: message, retriable: entry?.retriable === true } : null
}
}
@@ -278,6 +292,7 @@ export interface experimentMetricsLogicValues {
completed: number
total: number
}
+ recalculationRejectionMessage: string | null
refreshEligibilityTick: number
secondaryMetricsResults: CachedNewExperimentQueryResponse[]
secondaryMetricsResultsErrors: (unknown | null)[]
@@ -348,6 +363,7 @@ export interface experimentMetricsLogicMeta {
total: number
}
totalMetricsCount: (arg: any) => number
+ recalculationRejectionMessage: (currentRecalculation: RecalculationPayload | null) => string | null
lastRefresh: (currentRecalculation: RecalculationPayload | null) => string | null
nextAllowedManualRefresh: (currentRecalculation: RecalculationPayload | null) => string | null
isManualRefreshBlocked: (
@@ -520,6 +536,10 @@ export const experimentMetricsLogic = kea<experimentMetricsLogicType>([
(experiment: Experiment): number =>
metricsInOrder(experiment, 'primary').length + metricsInOrder(experiment, 'secondary').length,
],
+ recalculationRejectionMessage: [
+ (s) => [s.currentRecalculation],
+ (recalc: RecalculationPayload | null): string | null => (recalc ? rejectionMessage(recalc) : null),
+ ],
lastRefresh: [
(s) => [s.currentRecalculation],
(recalc: RecalculationPayload | null): string | null => recalc?.query_to ?? null,
@@ -781,7 +801,8 @@ export const experimentMetricsLogic = kea<experimentMetricsLogicType>([
* otherwise the uncovered metric shows a perpetual loading state, since nothing else re-runs
* on page load. What is already shown stays visible and cells update in place as the run polls.
*/
- if (recalculationHasGap(props.experiment, recalculation)) {
+ /** A rejected run gets the same answer on every heal, so a heal would only loop. */
+ if (!rejectionMessage(recalculation) && recalculationHasGap(props.experiment, recalculation)) {
actions.triggerRecalculation(HEAL_TRIGGER_BY_RESULT_SOURCE[recalculation.result_source])
}
} catch (error: any) {
@@ -942,9 +963,20 @@ export const experimentMetricsLogic = kea<experimentMetricsLogicType>([
) {
/**
* Create can return an already-terminal run (e.g. a completed one finished between
- * request and response). Load its results directly rather than polling.
+ * request and response). Load its results directly rather than polling. A rejected
+ * run has no results on the create response, so latest serves the earlier ones.
*/
- applyResults(recalculation)
+ if (rejectionMessage(recalculation)) {
+ /** A failed reload never reaches applyResults, so the marks of this create clear here. */
+ actions.setRecalculatingMetricUuids(
+ values.recalculatingMetricUuids.filter(
+ (uuid) => !(uuid in metricErrorEntries(recalculation))
+ )
+ )
+ actions.loadLatestRecalculation()
+ } else {
+ applyResults(recalculation)
+ }
emitTerminalEvent(recalculation)
// Terminal-on-create arms no poll, so drain any rerun queued during this create here,
// the way the poll's terminal branch does. Otherwise a config change made mid-create
diff --git a/products/experiments/backend/admin/recalculation_panel.py b/products/experiments/backend/admin/recalculation_panel.py
index 0f7b9151cf83..578e6faf0456 100644
--- a/products/experiments/backend/admin/recalculation_panel.py
+++ b/products/experiments/backend/admin/recalculation_panel.py
@@ -28,7 +28,7 @@ from products.experiments.backend.recalculation import (
_derive_counters,
get_run_results,
metrics_recalculation_workflow_id,
- request_recalculation,
+ request_recalculation_without_age_limit,
start_metrics_recalculation_workflow,
)
@@ -152,7 +152,7 @@ def start_recalculation_for_experiment(
"""
try:
# admin_view gates these callers to staff, so request.user is always a real User, not AnonymousUser.
- result = request_recalculation(experiment, cast(User, request.user), trigger=trigger)
+ result = request_recalculation_without_age_limit(experiment, cast(User, request.user), trigger=trigger)
except Exception as e:
messages.error(request, f"Could not create recalculation: {e}")
return HttpResponseRedirect(fallback_url)
diff --git a/products/experiments/backend/presentation/views.py b/products/experiments/backend/presentation/views.py
index fd3c09cd2cce..865a2db80f5f 100644
--- a/products/experiments/backend/presentation/views.py
+++ b/products/experiments/backend/presentation/views.py
@@ -1497,8 +1497,10 @@ class EnterpriseExperimentsViewSet(
"""Trigger a batch recalculation of all metrics for this experiment.
Returns 201 with the new pending recalculation, or 200 with the active one if a recalculation is
- already pending or in progress for this experiment. A manual trigger within five minutes after the latest
- completed run finished returns 429 with a Retry-After header. The response payload intentionally does not
+ already pending or in progress for this experiment. An experiment that started too long ago gets 201 with
+ a run that is already failed: every metric carries the `recalculation_window_too_long` error in
+ `metric_errors`, no workflow starts, and a retry gets the same answer. A manual trigger within five
+ minutes after the latest completed run finished returns 429 with a Retry-After header. The response payload intentionally does not
include the `results` array — at POST time the workflow has just been queued and no per-metric
results exist yet. Clients should poll `GET metrics_recalculation/{id}/` for results as the workflow
progresses.
@@ -1520,7 +1522,8 @@ class EnterpriseExperimentsViewSet(
# the idempotent-reuse path without inspecting the HTTP status code.
is_existing = result.get("is_existing", False)
- if not is_existing:
+ # A rejected run is new but already FAILED, so it gets a 201 and no workflow.
+ if not is_existing and result["status"] == ExperimentMetricsRecalculation.Status.PENDING:
start_metrics_recalculation_workflow(
str(result["id"]),
team_id=experiment.team_id,
diff --git a/products/experiments/backend/recalculation.py b/products/experiments/backend/recalculation.py
index 58b9e7ac7a41..3d6fe0695627 100644
--- a/products/experiments/backend/recalculation.py
+++ b/products/experiments/backend/recalculation.py
@@ -17,12 +17,14 @@ from django.db import transaction
from django.db.models import Q, QuerySet
from django.utils import timezone
+import posthoganalytics
from prometheus_client import Counter
from rest_framework.exceptions import Throttled, ValidationError
from posthog.clickhouse.client import sync_execute
from posthog.clickhouse.client.connection import Workload
from posthog.clickhouse.query_tagging import Feature, Product, tags_context
+from posthog.event_usage import groups
from posthog.exceptions_capture import capture_exception
from posthog.models.scoping import get_current_team_id, team_scope
from posthog.models.user import User
@@ -41,6 +43,7 @@ from products.experiments.backend.models.experiment import (
from products.experiments.backend.result_serialization import strip_step_sessions
from products.experiments.backend.temporal.models import (
METRICS_RECALCULATION_WORKFLOW_NAME,
+ RECALCULATION_WINDOW_TOO_LONG_ERROR_TYPE,
ExperimentMetricsRecalculationWorkflowInputs,
)
@@ -65,6 +68,16 @@ _RATE_LIMITED_TRIGGERS = frozenset(
}
)
+# A run scans every event from the start date to now, so the oldest experiments make the most expensive
+# ClickHouse queries. request_recalculation records them as rejected runs and starts no workflow.
+RECALCULATION_MAX_EXPERIMENT_AGE = timedelta(days=180)
+RECALCULATION_AGE_LIMIT_FEATURE_FLAG = "experiment-recalculation-age-limit"
+RECALCULATION_REJECTED_EVENT = "experiment recalculation rejected"
+RECALCULATION_WINDOW_TOO_LONG_MESSAGE = (
+ "This experiment started too long ago for its metrics to be recalculated. "
+ "To measure new data, start a new experiment or move the start date closer to today."
+)
+
class RecalculationRateLimited(Throttled):
"""A user-driven trigger landed inside MIN_USER_RECALCULATION_INTERVAL. The API layer renders it as a 429
@@ -341,15 +354,99 @@ def _refresh_window_enforced(experiment: Experiment) -> bool:
return _organization_flag_enabled(RECALCULATION_RATE_LIMIT_FEATURE_FLAG, str(experiment.team.organization_id))
+def _recalculation_max_age(experiment: Experiment) -> timedelta | None:
+ # A flag error turns the limit off, so a flag outage never rejects a recalculation.
+ try:
+ enabled = _organization_flag_enabled(RECALCULATION_AGE_LIMIT_FEATURE_FLAG, str(experiment.team.organization_id))
+ except Exception as e:
+ capture_exception(e)
+ return None
+ return RECALCULATION_MAX_EXPERIMENT_AGE if enabled else None
+
+
+def _capture_recalculation_rejected(
+ recalc: ExperimentMetricsRecalculation, user: User | None, experiment_age: timedelta, max_age: timedelta
+) -> None:
+ # Telemetry must never fail the request, so every error is swallowed.
+ try:
+ team = recalc.experiment.team
+ posthoganalytics.capture(
+ distinct_id=user.distinct_id if user and user.distinct_id else f"team_{team.id}",
+ event=RECALCULATION_REJECTED_EVENT,
+ properties={
+ "experiment_id": recalc.experiment_id,
+ "team_id": team.id,
+ "organization_id": str(team.organization_id),
+ "trigger": recalc.trigger,
+ "recalculation_id": str(recalc.id),
+ "experiment_age_days": experiment_age.days,
+ "max_age_days": max_age.days,
+ },
+ groups=groups(organization=team.organization, team=team),
+ )
+ except Exception as e:
+ capture_exception(e)
+
+
+def _create_rejected_recalculation(
+ experiment: Experiment, user: User | None, trigger: str, experiment_age: timedelta, max_age: timedelta
+) -> ExperimentMetricsRecalculation:
+ """A terminal FAILED run with the window error on every metric. Callers hold the team scope and the lock.
+
+ completed_at makes the run the latest one, so a reload shows the error and no page-load heal runs again.
+ The run takes query_to from the latest terminal run, so the results lookup of this run returns the rows
+ of that run and the earlier results stay readable.
+ """
+ now = timezone.now()
+ previous = _terminal_recalculations(experiment).order_by("-created_at").first()
+ metrics = get_metrics_for_calculation(experiment)
+ error = {
+ "step": "request",
+ "message": RECALCULATION_WINDOW_TOO_LONG_MESSAGE,
+ "error_type": RECALCULATION_WINDOW_TOO_LONG_ERROR_TYPE,
+ "retriable": False,
+ "timestamp": now.isoformat(),
+ }
+ recalc = ExperimentMetricsRecalculation.objects.create(
+ team=experiment.team,
+ experiment=experiment,
+ trigger=trigger,
+ status=ExperimentMetricsRecalculation.Status.FAILED,
+ created_by=user,
+ total_metrics=len(metrics),
+ metric_uuids=[m.uuid for m in metrics],
+ metric_errors={m.uuid: dict(error) for m in metrics},
+ query_to=previous.query_to if previous is not None else None,
+ completed_at=now,
+ )
+ transaction.on_commit(lambda: _capture_recalculation_rejected(recalc, user, experiment_age, max_age))
+ return recalc
+
+
def request_recalculation(experiment: Experiment, user: User | None, trigger: str = "manual") -> dict:
"""Create an idempotent batch recalculation request for all experiment metrics.
If an active (pending or in_progress) run already exists for this experiment, returns the existing run's
serialized payload with ``is_existing=True`` — the caller should NOT start a new workflow in that case.
- A user-driven trigger inside ``MIN_USER_RECALCULATION_INTERVAL`` after the latest completed run finished
- raises ``RecalculationRateLimited`` when the organization has the rate-limit flag on. Otherwise creates a
- fresh pending row.
+ When the organization has the age-limit flag on, an experiment that started more than
+ ``RECALCULATION_MAX_EXPERIMENT_AGE`` ago gets a rejected run: status FAILED, ``is_existing=False``, and the
+ caller must not start a workflow for it either. A user-driven trigger inside
+ ``MIN_USER_RECALCULATION_INTERVAL`` after the latest completed run finished raises
+ ``RecalculationRateLimited`` when the organization has the rate-limit flag on. Otherwise creates a fresh
+ pending row.
"""
+ return _request_recalculation(experiment, user, trigger, max_age=_recalculation_max_age(experiment))
+
+
+def request_recalculation_without_age_limit(experiment: Experiment, user: User | None, trigger: str) -> dict:
+ """``request_recalculation`` without the experiment age limit, so staff can still recalculate an old
+ experiment from the admin panel."""
+ return _request_recalculation(experiment, user, trigger, max_age=None)
+
+
+def _request_recalculation(
+ experiment: Experiment, user: User | None, trigger: str, *, max_age: timedelta | None
+) -> dict:
if not experiment.is_launched:
raise ValidationError("Cannot recalculate metrics for experiment that hasn't started")
@@ -416,6 +513,12 @@ def request_recalculation(experiment: Experiment, user: User | None, trigger: st
_recalculation_stale_cleanup_counter.inc(len(stale_ids))
transaction.on_commit(lambda: _cancel_superseded_workflows([str(stale_id) for stale_id in stale_ids]))
+ # After the stale cleanup, so a stale run of an old experiment is failed and its workflow cancelled too.
+ experiment_age = timezone.now() - experiment.start_date if experiment.start_date is not None else None
+ if max_age is not None and experiment_age is not None and experiment_age > max_age:
+ rejected = _create_rejected_recalculation(experiment, user, trigger, experiment_age, max_age)
+ return build_job_payload(rejected, is_existing=False)
+
# Set total_metrics up front from the experiment definition so the client can show progress
# ("N of M") immediately, before the workflow's discovery activity confirms the same count.
metrics = get_metrics_for_calculation(experiment)
diff --git a/products/experiments/backend/temporal/ARCHITECTURE.md b/products/experiments/backend/temporal/ARCHITECTURE.md
index f0abe9eea739..a2c4d264ac0b 100644
--- a/products/experiments/backend/temporal/ARCHITECTURE.md
+++ b/products/experiments/backend/temporal/ARCHITECTURE.md
@@ -126,6 +126,7 @@ Two safeguards on the POST endpoint:
- **Idempotent reuse.** If an active recalc (pending or in-progress) exists for this experiment and is younger than 30 minutes, the POST returns it instead of creating a new one. Frontend can click Reload twice and get the same `recalculation_id`.
- **Per-experiment lock.** The service runs inside `transaction.atomic()` with `Experiment.objects.select_for_update()` to serialize concurrent POSTs. Without this, two simultaneous clicks would both see no active row, both reach `.create()`, and the second would hit the per-experiment unique constraint and return HTTP 500.
- **User refresh window.** A `manual` or `agent_mcp` POST within `MIN_USER_RECALCULATION_INTERVAL` (five minutes) after the latest completed run's `completed_at` gets a 429 with a `Retry-After` header (`RecalculationRateLimited`), so a client or an agent knows when the next run is allowed. The window measures from completion, not `query_to`, because a stopped experiment pins `query_to` to `end_date`. A failed latest run never anchors the window, and neither does a `timeseries_sync` run, which may lack metrics the daily workflow does not compute. System triggers (`cold_run`, `heal_latest_run`, `manual_retry`, config changes) are exempt. The `experiment-metric-recalculation-rate-limit` flag, targeted by organization, turns the window on; the backend evaluates it by organization id and the frontend reads the same flag to block the reload button. Local development (`DEBUG` on, outside tests) skips the window, and the frontend mirrors that through the `is_debug` preflight field.
+- **Experiment age limit.** When the `experiment-recalculation-age-limit` flag is on for the organization, a POST for an experiment that started more than `RECALCULATION_MAX_EXPERIMENT_AGE` (180 days) ago starts no workflow, because a run scans every event from the start date to now. `request_recalculation` creates a rejected run instead: FAILED, `completed_at` set, a `recalculation_window_too_long` entry in `metric_errors` for every metric (step `request`, not retriable), and the `query_to` of the latest terminal run, so the `latest` read returns the rejection message together with the earlier results. The API answers 201, and the backend sends the `experiment recalculation rejected` event. The check runs after the idempotent reuse, so an active run is still returned, and after the stale cleanup, so a stale run is still failed and its workflow cancelled. The frontend never heals a gap in a rejected run, because every heal gets the same rejection. The admin panel calls `request_recalculation_without_age_limit`, so staff can still recalculate an old experiment. Scheduled runs never reach the limit, because their discovery selects only experiments younger than 60 days.
### Activity-aware staleness
@@ -205,7 +206,7 @@ Red nodes advance the window; blue nodes reuse it.
A metric's terminal failure lands in `metric_errors` as `{step, message, error_type, retriable, timestamp}`.
`error_type` is the same taxonomy as the `experiment metric error` event (`classify_experiment_query_error`).
-`retriable` is the backend's own decision, not a frontend mapping of `error_type`: it is true when a transient error (`timeout`, `rate_limited`, `server_error`) used its final attempt, and false when the calc activity failed permanently (`validation_error`, `out_of_memory`, `byte_limit`, `insufficient_data`, a `ValueError`, or a discovery failure).
+`retriable` is the backend's own decision, not a frontend mapping of `error_type`: it is true when a transient error (`timeout`, `rate_limited`, `server_error`) used its final attempt, and false when the calc activity failed permanently (`validation_error`, `out_of_memory`, `byte_limit`, `insufficient_data`, a `ValueError`, or a discovery failure) or the request was rejected (`recalculation_window_too_long`).
A retriable failure reads as a gap, so the page load heals it; a non-retriable one waits for the user, because a new run with the same inputs fails the same way.
An entry without the flag predates it and counts as non-retriable, so an old run never heals in a loop.
@@ -249,7 +250,7 @@ The split is intentional: Grafana tells you about the worker process, PostHog te
Retries live in Temporal's `RetryPolicy` on the calc activity, not in the workflow. The policy is exponential: `initial_interval=5s`, `backoff_coefficient=2.0`, `maximum_interval=60s`, `maximum_attempts=MAX_METRIC_ATTEMPTS` (currently 8). So a metric backs off 5s, 10s, 20s, 40s, 60s, 60s, 60s across its retries, then the eighth attempt is final. `asyncio.gather(..., return_exceptions=True)` lets healthy metrics finish while a failing one retries; a retrying metric never blocks the others, because each activity retries on its own worker slot rather than holding a shared queue slot.
- A permanent failure fails fast rather than burning the retry budget. `NON_RETRYABLE_ERROR_TYPES` (`out_of_memory`, `byte_limit`, `validation_error`) plus `ValueError` are raised non-retryable, so the metric is terminal on the first trip. A different bucket, the cluster at-capacity error (`ClickHouseAtCapacity`, from `TOO_MANY_SIMULTANEOUS_QUERIES` or `CANNOT_SCHEDULE_TASK`) and `ConcurrencyLimitExceeded`, is retried after a random delay between 2 and 5 minutes (`CONCURRENCY_LIMIT_RETRY_DELAY_MIN_SECONDS` and `CONCURRENCY_LIMIT_RETRY_DELAY_MAX_SECONDS`) via `ApplicationError(next_retry_delay=...)`, out-of-band from the exponential schedule, because that error means "come back later", not "this query is wrong". The random delay spreads out metrics that bounced in the same burst, so their retries do not arrive together and hit the same saturated cluster again.
+ A permanent failure fails fast rather than burning the retry budget. `NON_RETRYABLE_ERROR_TYPES` (`out_of_memory`, `byte_limit`, `validation_error`, `recalculation_window_too_long`) plus `ValueError` are raised non-retryable, so the metric is terminal on the first trip. A different bucket, the cluster at-capacity error (`ClickHouseAtCapacity`, from `TOO_MANY_SIMULTANEOUS_QUERIES` or `CANNOT_SCHEDULE_TASK`) and `ConcurrencyLimitExceeded`, is retried after a random delay between 2 and 5 minutes (`CONCURRENCY_LIMIT_RETRY_DELAY_MIN_SECONDS` and `CONCURRENCY_LIMIT_RETRY_DELAY_MAX_SECONDS`) via `ApplicationError(next_retry_delay=...)`, out-of-band from the exponential schedule, because that error means "come back later", not "this query is wrong". The random delay spreads out metrics that bounced in the same burst, so their retries do not arrive together and hit the same saturated cluster again.
Two error classes have a lower attempt cap than `MAX_METRIC_ATTEMPTS`, because their retries rarely succeed and each one costs ClickHouse capacity: capacity errors stop at `CAPACITY_MAX_METRIC_ATTEMPTS` (3) and the `timeout` class at `TIMEOUT_MAX_METRIC_ATTEMPTS` (2). The activity compares the attempt number with the cap of the error it just got. At the cap it persists the failure with `retriable=True` and raises non-retryable, so Temporal stops early without a change to the workflow's `RetryPolicy`. A user retry can recover the metric later.
diff --git a/products/experiments/backend/temporal/models.py b/products/experiments/backend/temporal/models.py
index d622ce6e060f..f6a93520bc8b 100644
--- a/products/experiments/backend/temporal/models.py
+++ b/products/experiments/backend/temporal/models.py
@@ -54,7 +54,11 @@ RECALCULATION_RETRY_MAX_INTERVAL_SECONDS = 60
# burns a full ClickHouse query per attempt without changing the outcome, so the activity fails non-retryable
# and persists immediately. timeout is deliberately absent — the class conflates TIMEOUT_EXCEEDED with
# SOCKET_TIMEOUT (a transient network blip), and recalc timeouts are rare enough to keep the retry budget.
-NON_RETRYABLE_ERROR_TYPES = frozenset({"out_of_memory", "byte_limit", "validation_error"})
+# The window type comes from request_recalculation, not from a query: the experiment start date sets it.
+RECALCULATION_WINDOW_TOO_LONG_ERROR_TYPE = "recalculation_window_too_long"
+NON_RETRYABLE_ERROR_TYPES = frozenset(
+ {"out_of_memory", "byte_limit", "validation_error", RECALCULATION_WINDOW_TOO_LONG_ERROR_TYPE}
+)
# Per-attempt ceiling for the calc activity (its start_to_close_timeout). When it fires, Temporal ends the
# attempt from the outside. The next heartbeat delivers the cancellation, which kills the ClickHouse query, but
diff --git a/products/experiments/backend/temporal/scheduled_recalculation_activities.py b/products/experiments/backend/temporal/scheduled_recalculation_activities.py
index e45d7d6bc9b6..2d02418fe959 100644
--- a/products/experiments/backend/temporal/scheduled_recalculation_activities.py
+++ b/products/experiments/backend/temporal/scheduled_recalculation_activities.py
@@ -154,6 +154,9 @@ def _start_scheduled_recalculation_sync(experiment_id: int, hour: int) -> Schedu
return ScheduledRecalculationStartResult(
experiment_id=experiment_id, started=False, skip_reason=SKIP_ACTIVE_RUN
)
+ # Discovery selects only experiments younger than the age limit, so a rejected run is not expected here.
+ if payload["status"] != ExperimentMetricsRecalculation.Status.PENDING:
+ return ScheduledRecalculationStartResult(experiment_id=experiment_id, started=False)
recalculation_id = str(payload["id"])
try:
diff --git a/products/experiments/backend/test/test_metrics_recalculation_api.py b/products/experiments/backend/test/test_metrics_recalculation_api.py
index d1a3dfa87da6..d8ec49ea6212 100644
--- a/products/experiments/backend/test/test_metrics_recalculation_api.py
+++ b/products/experiments/backend/test/test_metrics_recalculation_api.py
@@ -17,7 +17,10 @@ from products.experiments.backend.models.experiment import (
ExperimentMetricResult,
ExperimentMetricsRecalculation,
)
-from products.experiments.backend.recalculation import RECALCULATION_ACTIVITY_QUEUE_FEATURE_FLAG
+from products.experiments.backend.recalculation import (
+ RECALCULATION_ACTIVITY_QUEUE_FEATURE_FLAG,
+ RECALCULATION_AGE_LIMIT_FEATURE_FLAG,
+)
from products.experiments.backend.temporal.models import ExperimentMetricsRecalculationWorkflowInputs
from products.feature_flags.backend.models.feature_flag import FeatureFlag
@@ -61,7 +64,7 @@ class TestMetricsRecalculationAPI(APIBaseTest):
created_by=self.user,
feature_flag=self._flag(flag_key),
name="exp",
- start_date=datetime(2026, 1, 1, tzinfo=UTC),
+ start_date=timezone.now() - timedelta(days=30),
)
exp.metrics = [_mean_metric("m1")]
exp.save()
@@ -204,6 +207,51 @@ class TestMetricsRecalculationAPI(APIBaseTest):
assert not mock_connect.called
assert ExperimentMetricsRecalculation.objects.filter(experiment=exp).count() == 1
+ @mock.patch(
+ "products.experiments.backend.recalculation.feature_enabled_or_false",
+ side_effect=lambda key, *args, **kwargs: key == RECALCULATION_AGE_LIMIT_FEATURE_FLAG,
+ )
+ @mock.patch("products.experiments.backend.recalculation.sync_connect")
+ @mock.patch("products.experiments.backend.recalculation.asyncio.run")
+ def test_post_for_old_experiment_rejects_without_a_workflow_and_keeps_earlier_results(
+ self, mock_run, mock_connect, _mock_flag
+ ):
+ exp = self._launched_experiment()
+ exp.start_date = timezone.now() - timedelta(days=200)
+ exp.save()
+ previous_window = timezone.now() - timedelta(days=1)
+ ExperimentMetricsRecalculation.objects.create(
+ team=self.team,
+ experiment=exp,
+ metric_uuids=["m1"],
+ status="completed",
+ query_to=previous_window,
+ completed_at=previous_window,
+ )
+ ExperimentMetricResult.objects.create(
+ experiment=exp,
+ metric_uuid="m1",
+ fingerprint=_recalc_fingerprint(_calculation_key(exp, "m1")),
+ query_from=exp.start_date,
+ query_to=previous_window,
+ status="completed",
+ result={"ok": True},
+ )
+
+ resp = self.client.post(self._post_url(exp.id), {"trigger": "manual"}, format="json")
+
+ assert resp.status_code == status.HTTP_201_CREATED, resp.content
+ body = resp.json()
+ assert body["status"] == "failed"
+ assert body["is_existing"] is False
+ assert body["metric_errors"]["m1"]["error_type"] == "recalculation_window_too_long"
+ assert not mock_connect.called
+
+ latest = self.client.get(self._latest_url(exp.id)).json()
+ assert latest["id"] == body["id"]
+ assert latest["metric_errors"]["m1"]["error_type"] == "recalculation_window_too_long"
+ assert [(r["metric_uuid"], r["result"]) for r in latest["results"]] == [("m1", {"ok": True})]
+
@mock.patch("products.experiments.backend.recalculation.sync_connect")
@mock.patch("products.experiments.backend.recalculation.asyncio.run")
def test_post_rejects_unlaunched_experiment(self, mock_run, mock_connect):
diff --git a/products/experiments/backend/test/test_recalculation_service.py b/products/experiments/backend/test/test_recalculation_service.py
index 53c05e659b98..9489dbb1d7c3 100644
--- a/products/experiments/backend/test/test_recalculation_service.py
+++ b/products/experiments/backend/test/test_recalculation_service.py
@@ -1,3 +1,4 @@
+from collections.abc import Callable
from datetime import UTC, datetime, timedelta
import pytest
@@ -14,6 +15,7 @@ from rest_framework.exceptions import ValidationError
from posthog.clickhouse.client import sync_execute
from posthog.clickhouse.query_tagging import Feature, Product, tags_context
from posthog.models.scoping import team_scope
+from posthog.models.user import User
from products.experiments.backend.metric_calculation.config import get_metric_calculation_config
from products.experiments.backend.metric_calculation.results import _recalc_fingerprint
@@ -25,6 +27,8 @@ from products.experiments.backend.models.experiment import (
ExperimentToSavedMetric,
)
from products.experiments.backend.recalculation import (
+ RECALCULATION_AGE_LIMIT_FEATURE_FLAG,
+ RECALCULATION_WINDOW_TOO_LONG_MESSAGE,
RecalculationRateLimited,
build_timeseries_cold_start_payload,
get_active_recalculation,
@@ -33,6 +37,7 @@ from products.experiments.backend.recalculation import (
get_recalculation_by_id,
get_run_results,
request_recalculation,
+ request_recalculation_without_age_limit,
)
from products.feature_flags.backend.models.feature_flag import FeatureFlag
@@ -77,7 +82,7 @@ class TestRecalculationService(BaseTest):
created_by=self.user,
feature_flag=self._flag(flag_key),
name="exp",
- start_date=datetime(2026, 1, 1, tzinfo=UTC),
+ start_date=timezone.now() - timedelta(days=30),
)
exp.metrics = [_mean_metric("m1")]
exp.save()
@@ -193,6 +198,106 @@ class TestRecalculationService(BaseTest):
assert result["is_existing"] is False
assert ExperimentMetricsRecalculation.objects.filter(experiment=exp).count() == 2
+ @parameterized.expand(
+ [
+ # (name, request function, age-limit flag outcome, experiment age in days, active run age in minutes, expected status)
+ ("flag_off_over_limit_starts_new", request_recalculation, False, 181, None, "pending"),
+ (
+ "flag_error_over_limit_starts_new",
+ request_recalculation,
+ RuntimeError("flags down"),
+ 181,
+ None,
+ "pending",
+ ),
+ ("under_limit_starts_new", request_recalculation, True, 179, None, "pending"),
+ ("at_limit_starts_new", request_recalculation, True, 180, None, "pending"),
+ ("over_limit_is_rejected", request_recalculation, True, 181, None, "failed"),
+ ("over_limit_reuses_active_run", request_recalculation, True, 181, 5, "pending"),
+ ("over_limit_fails_a_stale_run", request_recalculation, True, 181, 120, "failed"),
+ ("admin_over_limit_starts_new", request_recalculation_without_age_limit, True, 181, None, "pending"),
+ ]
+ )
+ @time_machine.travel("2026-09-01T12:00:00Z", tick=False)
+ def test_request_recalculation_age_limit(
+ self,
+ name: str,
+ request: Callable[[Experiment, User | None, str], dict],
+ flag_outcome: bool | Exception,
+ age_days: int,
+ active_run_minutes: int | None,
+ expected_status: str,
+ ):
+ def evaluate_flag(key: str, *args, **kwargs) -> bool:
+ if key != RECALCULATION_AGE_LIMIT_FEATURE_FLAG:
+ return False
+ if isinstance(flag_outcome, Exception):
+ raise flag_outcome
+ return flag_outcome
+
+ exp = self._launched_experiment(flag_key=f"age-{name}")
+ exp.start_date = timezone.now() - timedelta(days=age_days)
+ exp.save()
+ previous = ExperimentMetricsRecalculation.objects.create(
+ team=self.team,
+ experiment=exp,
+ status="completed",
+ query_to=timezone.now() - timedelta(days=1),
+ completed_at=timezone.now() - timedelta(days=1),
+ )
+ ExperimentMetricsRecalculation.objects.filter(id=previous.id).update(
+ created_at=timezone.now() - timedelta(days=1)
+ )
+ active = None
+ if active_run_minutes is not None:
+ active = ExperimentMetricsRecalculation.objects.create(team=self.team, experiment=exp, status="pending")
+ ExperimentMetricsRecalculation.objects.filter(id=active.id).update(
+ created_at=timezone.now() - timedelta(minutes=active_run_minutes)
+ )
+
+ with (
+ patch("products.experiments.backend.recalculation.feature_enabled_or_false", side_effect=evaluate_flag),
+ patch("products.experiments.backend.recalculation.posthoganalytics.capture") as mock_capture,
+ patch("products.experiments.backend.recalculation._cancel_superseded_workflows") as mock_cancel,
+ self.captureOnCommitCallbacks(execute=True),
+ ):
+ result = request(exp, self.user, "manual")
+
+ assert result["status"] == expected_status
+ assert result["is_existing"] is (active_run_minutes is not None and expected_status == "pending")
+ if expected_status == "pending":
+ mock_capture.assert_not_called()
+ return
+ if active is not None:
+ active.refresh_from_db()
+ assert active.status == "failed"
+ mock_cancel.assert_called_once_with([str(active.id)])
+
+ row = ExperimentMetricsRecalculation.objects.get(id=result["id"])
+ assert row.completed_at == timezone.now()
+ assert row.query_to == previous.query_to
+ assert row.metric_errors == {
+ "m1": {
+ "step": "request",
+ "message": RECALCULATION_WINDOW_TOO_LONG_MESSAGE,
+ "error_type": "recalculation_window_too_long",
+ "retriable": False,
+ "timestamp": timezone.now().isoformat(),
+ }
+ }
+ assert get_latest_recalculation(exp) == row
+ mock_capture.assert_called_once()
+ assert mock_capture.call_args.kwargs["event"] == "experiment recalculation rejected"
+ assert mock_capture.call_args.kwargs["properties"] == {
+ "experiment_id": exp.id,
+ "team_id": self.team.id,
+ "organization_id": str(self.organization.id),
+ "trigger": "manual",
+ "recalculation_id": result["id"],
+ "experiment_age_days": 181,
+ "max_age_days": 180,
+ }
+
def test_request_recalculation_is_idempotent(self):
exp = self._launched_experiment()
first = request_recalculation(exp, self.user, "manual")
diff --git a/products/experiments/frontend/generated/api.ts b/products/experiments/frontend/generated/api.ts
index ad9b77d47b1b..561b7fc9a74a 100644
--- a/products/experiments/frontend/generated/api.ts
+++ b/products/experiments/frontend/generated/api.ts
@@ -818,8 +818,10 @@ export const getExperimentsMetricsRecalculationCreateUrl = (projectId: string, i
* Trigger a batch recalculation of all metrics for this experiment.
*
* Returns 201 with the new pending recalculation, or 200 with the active one if a recalculation is
- * already pending or in progress for this experiment. A manual trigger within five minutes after the latest
- * completed run finished returns 429 with a Retry-After header. The response payload intentionally does not
+ * already pending or in progress for this experiment. An experiment that started too long ago gets 201 with
+ * a run that is already failed: every metric carries the `recalculation_window_too_long` error in
+ * `metric_errors`, no workflow starts, and a retry gets the same answer. A manual trigger within five
+ * minutes after the latest completed run finished returns 429 with a Retry-After header. The response payload intentionally does not
* include the `results` array — at POST time the workflow has just been queued and no per-metric
* results exist yet. Clients should poll `GET metrics_recalculation/{id}/` for results as the workflow
* progresses.
diff --git a/products/experiments/frontend/generated/api.zod.ts b/products/experiments/frontend/generated/api.zod.ts
index 7aa952051c21..e9a0d17ae4aa 100644
--- a/products/experiments/frontend/generated/api.zod.ts
+++ b/products/experiments/frontend/generated/api.zod.ts
@@ -1072,8 +1072,10 @@ export const ExperimentsEndCreateBody = /* @__PURE__ */ zod.object({
* Trigger a batch recalculation of all metrics for this experiment.
*
* Returns 201 with the new pending recalculation, or 200 with the active one if a recalculation is
- * already pending or in progress for this experiment. A manual trigger within five minutes after the latest
- * completed run finished returns 429 with a Retry-After header. The response payload intentionally does not
+ * already pending or in progress for this experiment. An experiment that started too long ago gets 201 with
+ * a run that is already failed: every metric carries the `recalculation_window_too_long` error in
+ * `metric_errors`, no workflow starts, and a retry gets the same answer. A manual trigger within five
+ * minutes after the latest completed run finished returns 429 with a Retry-After header. The response payload intentionally does not
* include the `results` array — at POST time the workflow has just been queued and no per-metric
* results exist yet. Clients should poll `GET metrics_recalculation/{id}/` for results as the workflow
* progresses.
diff --git a/products/experiments/mcp/tools.yaml b/products/experiments/mcp/tools.yaml
index 91db7d3732ab..012e4e214d1a 100644
--- a/products/experiments/mcp/tools.yaml
+++ b/products/experiments/mcp/tools.yaml
@@ -822,6 +822,12 @@ tools:
starting a second one.
+ An experiment that started too long ago cannot be recalculated. The response is then a run that is already
+ failed, and every metric in metric_errors has error_type recalculation_window_too_long. Do not retry or poll
+ it; tell the user the message from metric_errors. The earlier results stay available from
+ experiment-metrics-recalculation-latest-retrieve.
+
+
The response has no results array, the run has only just been queued. Take the returned id and poll
experiment-metrics-recalculation-retrieve with it, or call experiment-metrics-recalculation-latest-retrieve,
until status is completed or failed. Watch rows_read and completed_metrics for progress. Do not poll in a
diff --git a/services/mcp/schema/generated-tool-definitions.json b/services/mcp/schema/generated-tool-definitions.json
index be7739dd4f38..2cb99fa190f0 100644
--- a/services/mcp/schema/generated-tool-definitions.json
+++ b/services/mcp/schema/generated-tool-definitions.json
@@ -5369,7 +5369,7 @@
}
},
"experiment-metrics-recalculation-create": {
- "description": "Requires an experiment ID. If you don't have the ID, load the finding-experiments skill to resolve the user's reference first.\n\nStart a fresh calculation of every metric on this experiment against the latest data. Use this when the user asks to refresh, recalculate, or re-run results, or when results returned by experiment-metrics-recalculation-latest-retrieve look stale. Reading results does NOT trigger a new run, this tool is the only way to start one.\n\nThis queues background work that can take minutes on large experiments. Safe to call more than once: if a run is already pending or in progress, it returns that run (with is_existing true, HTTP 200) instead of starting a second one.\n\nThe response has no results array, the run has only just been queued. Take the returned id and poll experiment-metrics-recalculation-retrieve with it, or call experiment-metrics-recalculation-latest-retrieve, until status is completed or failed. Watch rows_read and completed_metrics for progress. Do not poll in a tight loop; wait a few seconds between checks.",
+ "description": "Requires an experiment ID. If you don't have the ID, load the finding-experiments skill to resolve the user's reference first.\n\nStart a fresh calculation of every metric on this experiment against the latest data. Use this when the user asks to refresh, recalculate, or re-run results, or when results returned by experiment-metrics-recalculation-latest-retrieve look stale. Reading results does NOT trigger a new run, this tool is the only way to start one.\n\nThis queues background work that can take minutes on large experiments. Safe to call more than once: if a run is already pending or in progress, it returns that run (with is_existing true, HTTP 200) instead of starting a second one.\n\nAn experiment that started too long ago cannot be recalculated. The response is then a run that is already failed, and every metric in metric_errors has error_type recalculation_window_too_long. Do not retry or poll it; tell the user the message from metric_errors. The earlier results stay available from experiment-metrics-recalculation-latest-retrieve.\n\nThe response has no results array, the run has only just been queued. Take the returned id and poll experiment-metrics-recalculation-retrieve with it, or call experiment-metrics-recalculation-latest-retrieve, until status is completed or failed. Watch rows_read and completed_metrics for progress. Do not poll in a tight loop; wait a few seconds between checks.",
"category": "Experiments",
"feature": "experiments",
"summary": "Recalculate experiment metrics",
diff --git a/services/mcp/schema/tool-definitions-all.json b/services/mcp/schema/tool-definitions-all.json
index cedd49bebc3c..dd7c19b0b216 100644
--- a/services/mcp/schema/tool-definitions-all.json
+++ b/services/mcp/schema/tool-definitions-all.json
@@ -5454,7 +5454,7 @@
}
},
"experiment-metrics-recalculation-create": {
- "description": "Requires an experiment ID. If you don't have the ID, load the finding-experiments skill to resolve the user's reference first.\n\nStart a fresh calculation of every metric on this experiment against the latest data. Use this when the user asks to refresh, recalculate, or re-run results, or when results returned by experiment-metrics-recalculation-latest-retrieve look stale. Reading results does NOT trigger a new run, this tool is the only way to start one.\n\nThis queues background work that can take minutes on large experiments. Safe to call more than once: if a run is already pending or in progress, it returns that run (with is_existing true, HTTP 200) instead of starting a second one.\n\nThe response has no results array, the run has only just been queued. Take the returned id and poll experiment-metrics-recalculation-retrieve with it, or call experiment-metrics-recalculation-latest-retrieve, until status is completed or failed. Watch rows_read and completed_metrics for progress. Do not poll in a tight loop; wait a few seconds between checks.",
+ "description": "Requires an experiment ID. If you don't have the ID, load the finding-experiments skill to resolve the user's reference first.\n\nStart a fresh calculation of every metric on this experiment against the latest data. Use this when the user asks to refresh, recalculate, or re-run results, or when results returned by experiment-metrics-recalculation-latest-retrieve look stale. Reading results does NOT trigger a new run, this tool is the only way to start one.\n\nThis queues background work that can take minutes on large experiments. Safe to call more than once: if a run is already pending or in progress, it returns that run (with is_existing true, HTTP 200) instead of starting a second one.\n\nAn experiment that started too long ago cannot be recalculated. The response is then a run that is already failed, and every metric in metric_errors has error_type recalculation_window_too_long. Do not retry or poll it; tell the user the message from metric_errors. The earlier results stay available from experiment-metrics-recalculation-latest-retrieve.\n\nThe response has no results array, the run has only just been queued. Take the returned id and poll experiment-metrics-recalculation-retrieve with it, or call experiment-metrics-recalculation-latest-retrieve, until status is completed or failed. Watch rows_read and completed_metrics for progress. Do not poll in a tight loop; wait a few seconds between checks.",
"category": "Experiments",
"feature": "experiments",
"summary": "Recalculate experiment metrics",
diff --git a/services/mcp/src/generated/experiments/api.ts b/services/mcp/src/generated/experiments/api.ts
index e6fdcbd71117..9b28d245aabc 100644
--- a/services/mcp/src/generated/experiments/api.ts
+++ b/services/mcp/src/generated/experiments/api.ts
@@ -19477,8 +19477,10 @@ export const ExperimentsLaunchCreateParams = () => zod.object({
* Trigger a batch recalculation of all metrics for this experiment.
*
* Returns 201 with the new pending recalculation, or 200 with the active one if a recalculation is
- * already pending or in progress for this experiment. A manual trigger within five minutes after the latest
- * completed run finished returns 429 with a Retry-After header. The response payload intentionally does not
+ * already pending or in progress for this experiment. An experiment that started too long ago gets 201 with
+ * a run that is already failed: every metric carries the `recalculation_window_too_long` error in
+ * `metric_errors`, no workflow starts, and a retry gets the same answer. A manual trigger within five
+ * minutes after the latest completed run finished returns 429 with a Retry-After header. The response payload intentionally does not
* include the `results` array — at POST time the workflow has just been queued and no per-metric
* results exist yet. Clients should poll `GET metrics_recalculation/{id}/` for results as the workflow
* progresses.