:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
fix(signals): stop checks after report resolution (#122734)
Mikayla Thompsonee1385d54907
diff --git a/docs/internal/signals-pr-lifecycle.md b/docs/internal/signals-pr-lifecycle.mdindex f6b4287f6c5b..333272b68789 100644--- a/docs/internal/signals-pr-lifecycle.md+++ b/docs/internal/signals-pr-lifecycle.md@@ -10,7 +10,7 @@ PR-driven dismissals emit `dismissal_cause=pr_closed_without_merge` when all linA person can resolve a report by recording a manual implementation or a useful finding that needs no action. Feedback from the state API, Slack, PR closures, report merges, refunds, and plan rollups saves its verdict so the activity log distinguishes resolutions from dismissals even after later state changes. Feedback predating recorded verdicts uses a neutral activity label.The `no_action_needed` reason requires a resolved state. It leaves the report resolved, cancels its open follow-up checks, and is excluded from fixed outcomes in inbox ranking. Manual resolution closes an open implementation PR; the resulting GitHub close event preserves the resolved report status.-Monitoring means implementation is complete and outcome verification is pending. Resolved means the report is finished. The organization-level `signals-report-monitoring` gate requires explicit enablement, including in local development, and treats missing definitions or evaluation failures as disabled.+Monitoring means implementation is complete and outcome verification is pending. Resolved means the report is finished and runs no follow-up checks. This includes stored Monitoring reports whose effective status becomes Resolved when the flag is disabled or unavailable. The organization-level `signals-report-monitoring` gate requires explicit enablement, including in local development, and treats missing definitions or evaluation failures as disabled.A Monitoring period begins with an explicit, timezone-aware implementation-complete timestamp supplied by the server. Merge-driven entry can use the linked PR's recorded `merged_at`, even when its webhook arrives later. The lifecycle never substitutes receipt time for a missing implementation timestamp. Repeated entry preserves the original period; archive and restore preserve it too. Reopening implementation clears the start so a later implementation creates a new period. Resolving after verification keeps the implementation timestamp for history.@@ -66,9 +66,9 @@ Report reads, research selection, and snapshot refresh use the same filter checkSnapshot refresh still requires access to both the live query and the shared measurement.A viewer with a property grant can read the live query while the team's default property restrictions keep the shared snapshot hidden.-Research writes measurable outcome goals as `metric_threshold` report checks and investigative goals as `agent` checks. A metric check stores a bounded live query, baseline, comparison, and soak window; the query and display format are copied from its report metric when it names one. The check waits for the report to resolve, then the coordinator runs it and records a verdict. Metric checks keep the configured soak separate from their query window and wait until that full window contains only post-resolution data. Each run pins absolute query bounds to avoid reusing a pre-resolution cached result. Reopening clears the measurement start; the next resolution starts it again. Existing active metric checks without a recorded start begin their window at the first coordinator tick. New metric checks validate count, rate, duration, baseline, and threshold values using the same rules as report metrics; existing configurations remain readable. A later research pass reviews every open check, preserves unchanged checks and their approvals, replaces changed checks, and retires omitted checks. A failed verification turn leaves existing checks alone. Verification can revise only Expected impact; other summary sections and their chart links must remain intact. The full replacement schedule, including its soak and measurement window, must finish before the 90-day horizon. The Follow-up checks sidebar shows schedules and results for both kinds. The Expected impact section uses the same metric checks to show goals and charts, behind the person-level `signals-expected-impact` display flag. A person can mark those measurements "Looks good" as a quality signal; approval does not control scheduling or execution. The "Suggest different metrics" action starts a discussion that can atomically replace relevant open metric checks while preserving unrelated checks. A failed replacement leaves the original running, and a successful replacement starts unapproved. Report observation metrics remain separate from these forward-looking checks.+Research writes measurable outcome goals as `metric_threshold` report checks and investigative goals as `agent` checks. A metric check stores a bounded live query, baseline, comparison, and soak window; the query and display format are copied from its report metric when it names one. Checks execute only during enabled Monitoring. The coordinator records interim metric observations within each check's soak window; final metric and authored agent verification remain separate work. Resolving a report stops execution, including recurrence checks. Historical results remain readable. New metric checks validate count, rate, duration, baseline, and threshold values using the same rules as report metrics; existing configurations remain readable. A later research pass reviews every open check, preserves unchanged checks and their approvals, replaces changed checks, and retires omitted checks. A failed verification turn leaves existing checks alone. Verification can revise only Expected impact; other summary sections and their chart links must remain intact. The full replacement schedule, including its soak and measurement window, must finish before the 90-day horizon. The Follow-up checks sidebar shows schedules and results for both kinds. The Expected impact section uses the same metric checks to show goals and charts, behind the person-level `signals-expected-impact` display flag. A person can mark those measurements "Looks good" as a quality signal; approval does not control scheduling or execution. The "Suggest different metrics" action starts a discussion that can atomically replace relevant open metric checks while preserving unrelated checks. A failed replacement leaves the original running, and a successful replacement starts unapproved. Report observation metrics remain separate from these forward-looking checks.-Reports awaiting human input also retain their generated checks, pending resolution. Pending sidebar rows show the minimum wait after resolution. Metric replacements require access to the query they schedule and preserve the remaining recurring runs and soak duration, including zero minutes. Replacement cannot override the soak. Creation and replacement share full schedule validation against the 90-day horizon. A replacement rejects a check moved by a concurrent report merge; reload the report and retry on the survivor. Units cannot contain null characters or unpaired Unicode surrogates. The Expected impact section shows finished verdicts and refreshes after agent tasks change checks.+Reports awaiting human input retain their generated checks for a later Monitoring period. Outside Monitoring, open checks display as paused in the Follow-up checks sidebar. Expected impact retains proposed or approved measurement labels before Monitoring, and shows paused checks on resolved or archived reports, including when the rollout is disabled. Stored schedules do not advertise future runs or active scouts. Historical verdicts and legacy schedule fields remain readable; remaining-run counts do not authorize post-resolution execution. Metric replacements require access to the query they schedule and preserve the remaining recurring runs and soak duration, including zero minutes. Replacement cannot override the soak. Creation and replacement share full schedule validation against the 90-day horizon. A replacement rejects a check moved by a concurrent report merge; reload the report and retry on the survivor. Units cannot contain null characters or unpaired Unicode surrogates. The Expected impact section shows finished verdicts and refreshes after agent tasks change checks.The `inbox-report-checks-replace` MCP tool requires `task:write` and `query:read`. Query-specific event, action, and cohort permissions still apply. The `signals-report-checks-replace` rollout flag hides the tool unless enabled. Keep it disabled until the replacement API is deployed in every region. This gate is separate from the Expected impact display flag.@@ -94,15 +94,15 @@ Research should make every effort to complete `config.monitoring` for each metriThe definition describes a measurement from Monitoring start through the observation time. A `linear` target scales by elapsed minutes divided by reference minutes and requires a single additive event count or property sum without a formula. Session-level sums are excluded because a session can contribute to more than one chart bucket. A `fixed` target keeps its bound as the observation period grows, including for percentages, averages, distinct counts, or explicitly fixed custom targets. Rolling weekly and monthly active-user queries are refused because their lookback overrides the Monitoring bounds; use a distinct user count instead. Display format alone never selects the policy. The baseline retains its own measured period and is not presented as an observation from Monitoring.-Expected impact keeps the activity chart visible and collapses the target period, scaling policy, baseline, and check timing under measurement details. Dynamic chart boundaries refresh at minute intervals so repeated views can reuse cached results. Bucket granularity follows the full displayed range and stays within the 1,000-point budget. The chart preserves date-range filters and spans one reference period on each side of Monitoring start, or now before Monitoring starts, extending through now if necessary. Start and now markers separate the surrounding activity from future dates, whose values remain empty. Compatible charts show target reference lines: additive linear targets scale to a full chart bucket, and fixed rates or property averages/quantiles retain their level. Fixed totals, distinct counts, unknown aggregations, linear calendar intervals longer than a day, and linear day buckets with unequal daylight-saving durations receive no reference line. These references do not impose per-bucket pass/fail rules. This read-time chart does not evaluate the Monitoring target or save a check result. The stored bounded query and whole-window comparison retain their existing Resolved execution meaning. Final Monitoring outcomes remain separate work.+Expected impact keeps the activity chart visible and collapses the target period, scaling policy, baseline, and check timing under measurement details. Dynamic chart boundaries refresh at minute intervals so repeated views can reuse cached results. Bucket granularity follows the full displayed range and stays within the 1,000-point budget. The chart preserves date-range filters and spans one reference period on each side of Monitoring start, or now before Monitoring starts, extending through now if necessary. Start and now markers separate the surrounding activity from future dates, whose values remain empty. Compatible charts show target reference lines: additive linear targets scale to a full chart bucket, and fixed rates or property averages/quantiles retain their level. Fixed totals, distinct counts, unknown aggregations, linear calendar intervals longer than a day, and linear day buckets with unequal daylight-saving durations receive no reference line. These references do not impose per-bucket pass/fail rules. This read-time chart does not evaluate the Monitoring target or save a check result. The stored bounded query and whole-window comparison remain available for later final verification. Final Monitoring outcomes remain separate work.During Monitoring, checks with an explicit Monitoring definition sample at their configured run interval, or every 24 hours when none is set. The first observation is due one interval after Monitoring starts. Interim observations run only before Monitoring start plus the check's `soak_minutes`. The full trailing query window does not delay them. Interim queries explicitly use the offline ClickHouse workload; interactive metric refreshes retain their default workload. The next due time is capped at that soak deadline, but this PR does not execute a final check or conclude a report there. A missing or zero soak, or a soak no longer than the sampling interval, produces no interim observations. Each query pins both bounds to Monitoring start and one captured observation cutoff, preserving the population filters. Pinned ranges include partial periods so their bounds match the scaled target. Linear comparisons scale both bounds by elapsed time divided by the reference period, without rounding; fixed comparisons stay unchanged.Each completed attempt appends a protected `check_observation` log entry with its exact bounds, soak deadline, display format, unit, effective comparison, value, and match result, or a sanitized query error. The log labels it as an interim measurement, shows whether the goal is on track, and displays time remaining when the measurement was taken. It does not assign a terminal outcome. Observations do not spend `runs_remaining`, change the report status, or emit recurrence signals. The next sample is due one interval after the cutoff; missed intervals do not create catch-up runs. Restoring Monitoring schedules from the latest attempt in that period, preserving a failed attempt's error streak and six-hour retry across suppression. Re-enabling the flag restores the measurement origin, the next attempt from the latest run, and the expiry from the original Monitoring start. This repair precedes the expiry sweep so a fallback expiry cannot shorten or extend the Monitoring horizon. A check that cannot fit its source queries into the remaining coordinator budget stays due without an observation or an error-budget change. Query failures retain the existing six-hour retry and consecutive-error limit, and the 90-day check horizon still applies. Writes reject a changed check, Monitoring period, report state, or flag, and duplicate workers cannot save the same attempt twice. Expiry sweeps and schedule repairs retain their check-row concurrency guards after database lock waits, so they cannot overwrite a concurrent cancellation, expiry extension, or saved observation.-With Monitoring disabled, observations disappear from the artefact API and counts. The existing Resolved executor uses the stored full-window query, comparison, and minimum soak. Observation values require the same query permissions as terminal check results. The inbox follows all artefact pages within the current project before deriving task runs, reviewers, and activity so observations cannot displace older report state. Polling waits for an in-flight paginated load to finish. With the Monitoring flag enabled, scouts see metric checks with an explicit definition as scheduled or due inside their Monitoring window; agent checks and metric checks without a definition remain paused until resolution. The soak deadline bounds interim execution; no final verdict, reopening, or follow-up report is added here.+With Monitoring disabled, observations disappear from the artefact API and counts. No Resolved executor runs. The coordinator also leaves disabled or unavailable Monitoring periods dormant. Observation values require the same query permissions as terminal check results. The inbox follows all artefact pages within the current project before deriving task runs, reviewers, and activity so observations cannot displace older report state. Polling waits for an in-flight paginated load to finish. With the Monitoring flag enabled, scouts see metric checks with an explicit definition as scheduled or due inside their Monitoring window; agent checks and metric checks without a definition remain paused pending final verification support. The soak deadline bounds interim execution; no final verdict, reopening, or follow-up report is added here.-Disabling Monitoring restores the legacy expiry for armed Monitoring metric checks from their existing measurement start and remaining schedule. The check saves this fallback deadline so further flag changes cannot shorten it after runs have been consumed or extend it after retries. Repair batches remember checks whose legacy expiry is also 90 days, allowing later checks to progress. Re-enabling restores the original horizon for active checks and leaves terminal checks unchanged. Archive and restore preserve the saved fallback deadline; reopening implementation clears it for the next period.+Disabling Monitoring stops execution without starting a post-resolution schedule or recalculating a recurring deadline. Re-enabling restores the original Monitoring horizon for active checks and leaves terminal checks unchanged. Pending checks arm only during enabled Monitoring, using the original implementation-complete timestamp. Queued legacy check-triggered scout activities skip before starting agent work, and already-running legacy scouts cannot record a result through the check-result tool. Scout check lists mark these checks paused and do not advertise them as waiting on a run.With `signals-report-metrics` enabled, research gathers short-window baselines before verification and prefers metric-based follow-up checks. It aims for practical 70-80% confidence in improvement, without claiming a calculated confidence score. Observed volume, recurrence, ingestion delay, and deployment timing justify the shortest useful query and wait. Frequent or high-volume checks can use 4-8 hours, allowing for several hours of deployment delay before the fix reaches users. Around 24 hours or several days require the corresponding activity or rollout evidence. Agent checks remain available when interpretation or unsupported measurements prevent a useful metric check. Missing activity is not success. These prompt preferences do not change execution, schema defaults, or final outcomes.@@ -112,7 +112,7 @@ Enabled research can author `run_interval_minutes` for a metric check with a Mon### Evidence qualification contract-`report_check_evidence.qualify_metric_evidence` is a pure contract for future final Monitoring evaluators. It has no production callers and changes neither Resolved execution nor interim observations. Qualification is independent of the target comparison: sufficient evidence can demonstrate a failed expectation, while a matching value with insufficient evidence cannot establish success.+`report_check_evidence.qualify_metric_evidence` is a pure contract for future final Monitoring evaluators. It has no production callers and changes neither scheduling nor interim observations. Qualification is independent of the target comparison: sufficient evidence can demonstrate a failed expectation, while a matching value with insufficient evidence cannot establish success.The input `MetricEvidence` carries explicit aggregation semantics, a measured value, and optional activity or denominator counts. Every measurement has an `EvidenceScope` naming the team, relevant population, and the check's existing fixed window. Supporting counts used for qualification must match all three. For property sums, relevant activity counts valid property contributions. A trusted source adapter establishes the population key and relevance; an arbitrary metric label does not. This module neither derives relevance from query text nor executes queries.@@ -152,9 +152,7 @@ against the linked GitHub PRs. The flag does not enable this backfill itself.## Follow-up measurement timing-Reopening or archiving a resolved report immediately parks its active checks and clears their measurement starts and dispatches. It resets both error and inconclusive retry streaks. Delayed resolution callbacks only arm checks while the report is still resolved. Results from an earlier measurement window or dispatch cannot update the current check.--Metric follow-up checks wait until their full trailing query window contains only post-resolution data. The configured soak is an independent minimum wait. Reopening a report clears the measurement anchor; resolving it again starts a new window. Legacy active metric checks without an anchor start their window at the next coordinator tick and recalculate expiry from the remaining schedule, capped at 90 days from that tick. Legacy rows do not distinguish supplied expiries from defaults, so both follow this re-arming policy. Checks with an existing anchor retain their expiry. A window that cannot finish before expiry records an inconclusive result instead of scheduling an unreachable run. Agent checks keep their soak-based schedule.+Reopening implementation clears the measurement anchor so a later Monitoring period starts fresh. Archiving parks active checks; restoring the same enabled Monitoring period preserves its original start and retry history. Delayed arming callbacks validate the current Monitoring period and never arm a Resolved report. Interim queries recheck the current report and flag before measuring, and saved observations reject stale report states or periods.New metric checks validate numeric goals and baselines against their metric kind, format, unit, and query. Existing check configurations remain readable.diff --git a/frontend/snapshots.yml b/frontend/snapshots.ymlindex 998ea1338d06..6e320679d6d6 100644--- a/frontend/snapshots.yml+++ b/frontend/snapshots.yml@@ -11797,49 +11797,49 @@ snapshots:scenes-app-inbox-detail-expected-impact--load-failure--light:hash: v1.k794b7964.85826d6ccb24433d86966725347d200f33d0708a733f4a02cec2ab4a9b239088.1BuwrA1Z8r6S6fT1Ui-MjT413KAf5IsWxKH9S6lXajYscenes-app-inbox-detail-expected-impact--metric-check--dark:- hash: v1.k794b7964.edd5299beb69a84a93bbf0f52027dbe6f13a5b9d9585a2c29baa188cb6ed7a42.kXjmkRCQpYM5AA02sRPqTPT0YaWs-WY_CmDsG0GdQLA+ hash: v1.k794b7964.da03dae120e49f2eadfd787294cd09753cec36420e4bc8269d177232caf89af8.Yb_sRouoITM7ucAEGywRJtD6ZZYNl4dnZmyiK8_16Bkscenes-app-inbox-detail-expected-impact--metric-check--light:- hash: v1.k794b7964.310dcf4af274f3011c148d9890a4a72507418b43a800535b192d91f4cb709791.HBW72C9W6XXe-cgONBY3vJ9VA4q_26sThTLfFgQv0X0+ hash: v1.k794b7964.7ab8e90eb19c85a88dc1bb00f25078f3ecae1532c11a13fa36be30863e7f4368.lg23x_vgJyHnX5NKQnStjyfu1wUfhRAetsusxBYUMU4scenes-app-inbox-detail-expected-impact--monitoring-definition--dark:- hash: v1.k794b7964.58e7bdde7ab8b92554a4b2ad7fb68424c27ebc7bb18e40e93d1671489fe2ed28.XMKOV6wUwzwsKWSVTiSbjj7y2abLYeFrhh1_vYlHRuc+ hash: v1.k794b7964.ae082b6c8b0a012d3202c17c9634b9ac315d056fc43bcce706ff6ea928d19138.71UgrCjD51Kcfhd3rtnjdqZujFR93GXF3J6boYjLty0scenes-app-inbox-detail-expected-impact--monitoring-definition--light:- hash: v1.k794b7964.ce4f3f94e83d15d3be80ef5fab7ca4148205d5389ea0ed06389e2d5b77a21c54.VrNcx2wCHyqnyK3qC2jiA6SLidLpUCR1y6-Fnjo0xn0+ hash: v1.k794b7964.c108b43638b8cce49fe8ed3c896e0ce3441a9ac04c4ee61ef5ba20791ddc4560.0ByZGqISFjPW0rQ0WfqHUBZMO9UYjyOaRRJ5vshQ0c8scenes-app-inbox-detail-expected-impact--monitoring-definition-flag-off--dark:- hash: v1.k794b7964.2159fc93ebc60413cf0baf9a6c23baf5f9002555a57d494983bcb253cc87b72a.Bnyv9obRcLTsAv-h4FIC7THgEui8LoOsj1H2qjW-lj0+ hash: v1.k794b7964.4497d91ad86cef9418def048c904a2131583ce05c1a15d2deb7e38b699df028d.3NQ1y1BDagcnvl_l2RVbVBg8Kyu9Lc0-pzlMepD7W5cscenes-app-inbox-detail-expected-impact--monitoring-definition-flag-off--light:- hash: v1.k794b7964.6a75f114a3b82e271240e13ebd0ba8d8d21f5a11da64fdf4e1c3fab06920adaa.NgM0xTjXuMDY9mDs2T1TozhdgCmL15Bp9TyklOe3htg+ hash: v1.k794b7964.7cab971b1c23034a13ac2b2a45a725eb7c410a3cda556e23f3303eb60a1dbc92.T05I-VgaKuqgym0kk7nZwj1wScADW4axDtfBds-gH1Escenes-app-inbox-detail-expected-impact--monitoring-definition-narrow--dark:- hash: v1.k794b7964.809623d87ec66a9f3c2438ac3b0d89838c738e5ceb8e2c866dc3cf315eee922f.77MBOHyuDbAABWmzn_GyRTQTycocf3ZViVv4V26yxC8+ hash: v1.k794b7964.76c4fdad05b00703fa8530424e61bb9789c4805793a70298164865ec630bae8d.Pk8LcjrJEdmO-8Gq3vEYYH2GcdLaqAyPMokg9XAQGdoscenes-app-inbox-detail-expected-impact--monitoring-definition-narrow--light:- hash: v1.k794b7964.cf0dc541d523067724d10755144c5fa65c2825d8bfe04928558454648a623c56.4h83sN5i9Pdzzxa65I-9wn7yqTAvyE4VZK2t9M8-f4g+ hash: v1.k794b7964.3c3b28a94b593ff7b39e902ec4ce8483036c932570eab787e898066e2636c811.IflAvfIywiiVs1CvlnkkwVoy-c380eAF_pS0xcpFzPUscenes-app-inbox-detail-expected-impact--monitoring-fixed-target--dark:- hash: v1.k794b7964.b60d812dc930d97a889442fd71319543843f6434f09d1923ea8f509942b8aa5b.DIfYuulD17nhvclTxpYZ5Y08x2zKXXW6DBGjJS_V880+ hash: v1.k794b7964.2dc49216d939b862ed4b4af4435a0ec07d4246f963890abeff85671fb6e9032b.gqy-VN-suwuM2gwwKbsIcYi7JwduPOekD-8wMvAx06Ascenes-app-inbox-detail-expected-impact--monitoring-fixed-target--light:- hash: v1.k794b7964.210fe320188acf7e5959ba35870e42bca8b4728fb29ed72031667b6ba6b74c63.WJruO37FR7o92GjIv8PE9VyGPEWq5X7nGhE1Rf4-Q7c+ hash: v1.k794b7964.4df11ac2d37c9aaccb50ab43f2f83afd910600f2887cd69d878f68313957e4e8.uF6EdinhHAWGdVlJ76JWlKz9HigzscmZh6OGiGuCk-8scenes-app-inbox-detail-expected-impact--monitoring-rate-target--dark:- hash: v1.k794b7964.b733c20ee84a5eaf01efcf7f7169a1058eab10cd17b6c9146d2f70693feaaa4a.MX4zc2myg-67cHuwFEsy3nRbg4AzqNPfMh4J73FOVic+ hash: v1.k794b7964.e9923773f00a6c0baf1e2c2891e0192f4fdc11acc897ab1f59cdb21fb2f03520.q_ISmmYWbCDEa19-uio1lzbld2iNG_uUROlkMR51NXAscenes-app-inbox-detail-expected-impact--monitoring-rate-target--light:- hash: v1.k794b7964.3823fa715a59a2bfe844c3e2045043c2dec6df075de3b617b4fb4b0a891ef381.DxKXtEZaT_xTocZLLQOjuJng-otgs_79wYHRvx34M9o+ hash: v1.k794b7964.1b8a7f47fd05086899755a3f919a46ff8ca3e72b6a881b5f3a648425ba7f5631.wUWr3my_HYpf3IHuJyH9NMG101IkjAgGExJIFsBGr6Escenes-app-inbox-detail-expected-impact--monitoring-window-ended--dark:- hash: v1.k794b7964.6ec50d57e6127d351c2261ee96bab3610d8ce692b263aee1c1159dba9d56770b.o41iNPIsGzixQa7phsk8RrRf-y0w1Z-uDUqEiS6PQqU+ hash: v1.k794b7964.ae082b6c8b0a012d3202c17c9634b9ac315d056fc43bcce706ff6ea928d19138.2xGELj0Bz1vbDuU5ZJyIm2h1Ck0xMkdPUdFkKOqWA-0scenes-app-inbox-detail-expected-impact--monitoring-window-ended--light:- hash: v1.k794b7964.01953dba2882e795858f3e65868b260fbb119c649f580d18622016c2620ed3d6.03C3bZztWuRWnPQdiH-kmpRPlSbwwoeMSy_Wu6Ne6PU+ hash: v1.k794b7964.c108b43638b8cce49fe8ed3c896e0ce3441a9ac04c4ee61ef5ba20791ddc4560.QVAeCVpM7VfD16qdypZ-MJg9eX4qopU-G0r2GYwhxA0scenes-app-inbox-detail-expected-impact--narrow--dark:- hash: v1.k794b7964.a2aea8677171f649ba45a21aa7103003773b36dcf0efe8093a2b4184897ac105.FESJsY6WQhEi2AtXx4RHoVDnx-pVcqRVizXrGQbSus0+ hash: v1.k794b7964.65edbc03c7425a2abaa6575c3c88458f57e386316f4869e74ca8a5be5e66abd6.xiIWioJYGe6e0pwn4phanMW1UnZgcdEgnUakgxq7b44scenes-app-inbox-detail-expected-impact--narrow--light:- hash: v1.k794b7964.fb0aa8004cfbe31b6b40b37ec5e39a35c8310307e28580e9ff08006229c8dbfd.MTcKNISbyB7gTLJI4uZUeRJhDbM_r500IBS_NHXnonI+ hash: v1.k794b7964.50f8492c10e8aaaf6fe29df14bcdda972c6a05226cc356699e581f6025d24bc3.RyPZaCDqK27kZ3NNqJHF1Y6FdWry8bvwvvE94vStrSMscenes-app-inbox-detail-expected-impact--range-goal--dark:- hash: v1.k794b7964.25ca512fe2e87c2c0e7b8d2e696e4ce6c51fd213931323f36d35f72f47325c01.deF-U-1oFJCQ19Yl-YcwVEgg_8te5wFly2OtkNJfMKk+ hash: v1.k794b7964.d775f73c0702b1409b7c11b9ccdd2c01e385d0371e2420f4ace53b60e175a185.GBoAu-DyZCKHHG-MoaqByqL4KQWIQWofxV0AaWWp2Doscenes-app-inbox-detail-expected-impact--range-goal--light:- hash: v1.k794b7964.b78440c96c51cd20fa18314e14555f52a0cee552b1ed3b437dc708189c633a25.SCoc4Z715pkdcCkH5lbjUeFUvcTt6KSYcgUgHbUg958+ hash: v1.k794b7964.ff9f962cfbc53d10b1f05c82f8314c83318f3c3aa8355a006b1405d03ef45249.4dDfLh9vzg3zUs8e7Kend-QoB8z8Qodx0FDxWgWgKkkscenes-app-inbox-detail-expected-impact--replacement-unavailable--dark:- hash: v1.k794b7964.11e024cae252d5f4a911741317d05326233cc0eae261a5c7d7347661dbe467c2.OSfzCDh8yBUfdfhDlQGNV8QbIvZitNT4evut8N46StE+ hash: v1.k794b7964.fcc3f4688b1ff08a3dd606db2337e1d621f0f85b1020a5a34fe32440a986ea4b.DJU5Tsk7kVflGl8SmM4jtY6CInUC9cHPNNYJwthjWfgscenes-app-inbox-detail-expected-impact--replacement-unavailable--light:- hash: v1.k794b7964.5e53f571b0a25a214aaa48bf8657bafa6fc5e5b7d0bf3121e9d325189a3854f7.Ldh-3_DNPkkNJwM66AlgwKjrLhd-4VRkkR9crqU9YRA+ hash: v1.k794b7964.255b55f7170b6523b6e21f9381eaea395e08283d289e4569a73a8f7ec752d662.wIq1CUQZYNqkWzgnP17AAaaLS-wOyEL9G7fyg9YQ1_Iscenes-app-inbox-detail-follow-up-checks--every-state--dark:- hash: v1.k794b7964.b2d0413b7187f874afae44518461bb0c5cd33192952321712580e25e3bddd9e1.G8AgFzVjqgR2YdWYjcDkw2Eaoqg97AWcCTYAGH7N-cg+ hash: v1.k794b7964.1fb2e55d040b2e54708b04edc022c95a9b223ed1a1f6911a5b97d5f29e615fd9.fDAJnYMmvwIB5sGK790KcV3QWzOvJcxFUQDmDhcAMRkscenes-app-inbox-detail-follow-up-checks--every-state--light:- hash: v1.k794b7964.4655bef6e24070355e1a13becdc153fa8e4abc3dcdbfd2d880db3789376ae8b8.uNlpCBaS1VvSDAmoBks4FPoKLMJIY1G63WlKQ7bd64k+ hash: v1.k794b7964.275081e63f24959c0017f4a3f295444edd91eb89f9cf9876ab4365dc29c8107c.FaarzsWlYt7_qiYo7KGl1dukS7877MEEhxlH1ZiiyzUscenes-app-inbox-detail-follow-up-checks--monitoring--dark:hash: v1.k794b7964.8203910c8643bee68c2895b77db7101544290e636a6cecaf6a7fd27d6e0fe87b.RurNhxFb2VgJ6Y5LHmClwOH4ihVHG6V9Rivg06oEqlEscenes-app-inbox-detail-follow-up-checks--monitoring--light:@@ -11849,13 +11849,13 @@ snapshots:scenes-app-inbox-detail-follow-up-checks--monitoring-narrow-rail--light:hash: v1.k794b7964.9baf07cf5b989f8f22c8fadba9d915297b026d3eee1128dbf28478cc88e0b98c.g2-gNbvDMXhe7Wkkt9JfjDNK3N3S3kyqfaLdwM96-98scenes-app-inbox-detail-follow-up-checks--narrow-rail--dark:- hash: v1.k794b7964.104726ab1adb7689ef0fa98cb6d95b32488a9d7a1384ff7b311dba39da93eaee.zynwpgfgVGiMUAfnykJ1qIB-DuF-fQ_VAvJWVyVhvIc+ hash: v1.k794b7964.be9f8ab7b553ea303e03e4b375a4156c486fad10ce4b66936fb4fbe923ad0bdd.m4ThsrMsAizyTDwlW6uvlP5yjhc0ZjL96P66-MNn85Mscenes-app-inbox-detail-follow-up-checks--narrow-rail--light:- hash: v1.k794b7964.4335078ee499a8b64b5a0c253a2128960c42afb6e657a9d8139350a9c775b89b.3Ze8wnGg4UHYbTtf0kAFagAWOSLrhDpuchVRUQfY01s+ hash: v1.k794b7964.bfa881131dd33abb1f00779499c5858ab7eab74738d5d564224c3550a102e260.64zPNw1gska6Ovqm9kGgeRutSl-TS_mG342pDW2MxuYscenes-app-inbox-detail-follow-up-checks--running--dark:- hash: v1.k794b7964.7c9c84d49f542f37fb20a337d4e097ceac512f7916476f8313f52403a0c8272e.Qbic0TtjJ_u8tITXga4NzK-pjvZVxEv_eDiiDbAZ6To+ hash: v1.k794b7964.57a616b7d836445e00ca0df54fed2537d723ec8349a3b60185c101f2b5b33642.iLshi-9beYAkMte1pr4v2FabFLItrIVd7k38Y6lT6NAscenes-app-inbox-detail-follow-up-checks--running--light:- hash: v1.k794b7964.a5e08bb11b3723765e5cc33b4afb36d6ad9217d9a8975497677cf4b2fc513071.Kvc9DMV6mta5XCyUg5mxeDQrjWC5zatz04E4OgsVWpc+ hash: v1.k794b7964.cc445c86aacf5799a1a4bda66e63264bda5dc4b5e52ac82008734620b57d3b2e.kh55Ac8x-rkJ6KcvB_Z2I8lVSTiw4DsztZayp-UOVBQscenes-app-inbox-detail-runs--narrow-rail--dark:hash: v1.k794b7964.b063aeeabbdbee852bc6a55c5e1e32b8075273ed3e211d8d0d490582a63d632f.ZNR24H-doELGCmC8Wwc5dFzukDi9dWeEuzsM7A5Fjfgscenes-app-inbox-detail-runs--narrow-rail--light:diff --git a/posthog/test/repo_invariants/setup_receivers_baseline.txt b/posthog/test/repo_invariants/setup_receivers_baseline.txtindex a04e4b1be97e..b64d886decd6 100644--- a/posthog/test/repo_invariants/setup_receivers_baseline.txt+++ b/posthog/test/repo_invariants/setup_receivers_baseline.txt@@ -236,7 +236,7 @@ post_save:products.metrics.backend.dashboard_import.receivers.schedule_import_fipost_save:products.posthog_ai.backend.receivers.mark_conversation_sandbox_on_first_runpost_save:products.review_hog.backend.ownership._invalidate_owned_repositories_on_savepost_save:products.review_hog.backend.receivers.handle_task_run_saved-post_save:products.signals.backend.receivers.arm_pending_checks_when_report_resolved+post_save:products.signals.backend.receivers.arm_pending_checks_when_report_monitoringpost_save:products.signals.backend.receivers.capture_status_change_analyticspost_save:products.signals.backend.receivers.close_pr_when_report_dismissedpost_save:products.signals.backend.receivers.emit_report_embedding_on_document_changediff --git a/products/signals/backend/models.py b/products/signals/backend/models.pyindex 11163f880019..aca7224158f2 100644--- a/products/signals/backend/models.py+++ b/products/signals/backend/models.py@@ -2463,10 +2463,8 @@ class SignalReportCheck(UUIDModel):days, and the check carries its own `next_run_at` rather than deriving one from a merged pullrequest — plenty of fixes land with no pull request to date the window from.- A check on a report that has not resolved has no date worth carrying, because the fix it tests is- not live yet. Such a check is stored PENDING with a `soak_minutes` window, and the report's- transition to RESOLVED sets its `next_run_at`. That makes the resolve the clock for every kind of- fix, including the ones that never had a pull request.+ Enabled Monitoring anchors pending checks to the implementation-complete timestamp. Resolved+ reports execute no checks, even when a legacy row still carries an active schedule.Terminal statuses are final. A check that passed, failed, errored out, ended inconclusive,expired, or was cancelled is never rescheduled; the author writes a new check instead, so a result artefact always refers@@ -2483,7 +2481,7 @@ class SignalReportCheck(UUIDModel):class Status(models.TextChoices):# Written while its report was still open, so it carries a soak rather than a date and- # waits for the report to resolve. Not due, not expired, and not terminal: the resolve+ # waits for Monitoring. Not due, not expired, and not terminal: Monitoring entry# transition arms it into ACTIVE (`report_check_authoring.arm_pending_checks`).PENDING = "pending"ACTIVE = "active"@@ -2540,7 +2538,7 @@ class SignalReportCheck(UUIDModel):next_run_at = models.DateTimeField()measurement_start_at = models.DateTimeField(null=True, blank=True)- # Minimum wait after resolution, separate from the query window a metric check must fill.+ # Each check's Monitoring window, anchored to implementation completion.soak_minutes = models.PositiveIntegerField(null=True, blank=True)# Null means one-shot. A recurring check re-arms at this interval until it runs out of runs or# reaches its expiry.diff --git a/products/signals/backend/receivers.py b/products/signals/backend/receivers.pyindex 92b05fe43e93..5e99020e785c 100644--- a/products/signals/backend/receivers.py+++ b/products/signals/backend/receivers.py@@ -476,7 +476,7 @@ def close_pr_when_report_dismissed(@receiver(post_save, sender=SignalReport)-def arm_pending_checks_when_report_resolved(+def arm_pending_checks_when_report_monitoring(sender: type[SignalReport],instance: SignalReport,created: bool,@@ -490,7 +490,7 @@ def arm_pending_checks_when_report_resolved():returneffective_status = instance.effective_status- if effective_status not in {SignalReport.Status.RESOLVED, SignalReport.Status.MONITORING}:+ if effective_status != SignalReport.Status.MONITORING:if prior_status in {SignalReport.Status.RESOLVED, SignalReport.Status.MONITORING}:from products.signals.backend.report_check_execution import park_report_checks_on_reopen@@ -498,19 +498,16 @@ def arm_pending_checks_when_report_resolved(returnteam_id = instance.team_idreport_id = str(instance.id)- monitoring_started_at = (- instance.monitoring_started_at if effective_status == SignalReport.Status.MONITORING else None- )- if effective_status == SignalReport.Status.MONITORING and monitoring_started_at is None:+ monitoring_started_at = instance.monitoring_started_at+ if monitoring_started_at is None:return- resolved_at = monitoring_started_at or timezone.now()# After commit so a rollback cannot start measurement; the callback also verifies the captured period.transaction.on_commit(partial(_arm_pending_checks_safely,team_id=team_id,report_id=report_id,- resolved_at=resolved_at,+ resolved_at=monitoring_started_at,expected_monitoring_started_at=monitoring_started_at,))diff --git a/products/signals/backend/report_check_authoring.py b/products/signals/backend/report_check_authoring.pyindex 3a982601f272..5d33d8b4f777 100644--- a/products/signals/backend/report_check_authoring.py+++ b/products/signals/backend/report_check_authoring.py@@ -5,11 +5,8 @@ they must agree on what a check row means, so the cap, the metric-query copy, anlive here rather than three times over. `report_checks.py` still owns the shapes and the bounds;this module owns the write.-The report's own state decides when a check first runs. A check on a resolved report names the date-to look on. A check on a report that is still open cannot: the fix it re-measures has not shipped,-and many fixes never get a merged pull request to date a soak window from. Such a check is stored-`pending` with a soak duration and armed by the report's transition to `resolved`, whatever caused-it. That makes the resolve the clock for every kind of fix.+Checks execute only during enabled Monitoring. Pending checks retain provisional dates until+Monitoring entry anchors their soak to the implementation-complete timestamp."""from __future__ import annotations@@ -86,18 +83,13 @@ def create_check() -> SignalReportCheck:"""Write one check on a report, armed or pending.- Pass `next_run_at` and `expires_at` for a check that names when to look. Pass `soak_minutes`- instead for one that names how long to wait after the report resolves.-- Which of the two the row uses is the report's call, not the caller's. A report that has not- resolved has no fix live yet, so a check on it is stored `pending` with a soak, and its dates- stay provisional until `arm_pending_checks` rewrites them at the resolve. A dated check keeps- the gap its author left as that soak. Only a resolved report takes a date as written.+ Legacy authoring accepts explicit dates or a soak duration. Pending dates remain provisional+ until Monitoring entry anchors the measurement window. Stored schedules on Resolved reports+ do not authorize execution.The report row is locked for the same reason the REST path locks it: the per-report cap is acount followed by an insert, which only holds if concurrent creates serialize. The status- decision is read under the same lock, so a resolve landing mid-write either precedes the row- or arms it.+ decision is read under the same lock, so Monitoring entry cannot snapshot an incomplete write."""if (next_run_at is None) == (soak_minutes is None):raise CheckCreationError("a check names either a next_run_at or a soak_minutes, not both and not neither")@@ -594,22 +586,17 @@ def arm_pending_checks(if report is None:return 0status = effective_report_status(report.status, team_id=team_id, monitoring_enabled=monitoring_enabled)- if expected_monitoring_started_at is not None:- if (- status != SignalReport.Status.MONITORING- or report.monitoring_started_at != expected_monitoring_started_at- ):- return 0- resolved_at = expected_monitoring_started_at- armed_at = timezone.now()- elif status == SignalReport.Status.RESOLVED:- armed_at = resolved_at- else:+ if status != SignalReport.Status.MONITORING or report.monitoring_started_at is None:+ return 0+ if (+ expected_monitoring_started_at is not None+ and report.monitoring_started_at != expected_monitoring_started_at+ ):return 0return _arm_pending_checks(team_id=team_id,report_id=report_id,- resolved_at=resolved_at,- armed_at=armed_at,- monitoring=status == SignalReport.Status.MONITORING,+ resolved_at=report.monitoring_started_at,+ armed_at=timezone.now(),+ monitoring=True,)diff --git a/products/signals/backend/report_check_execution.py b/products/signals/backend/report_check_execution.pyindex 47b8b6e04270..930add694199 100644--- a/products/signals/backend/report_check_execution.py+++ b/products/signals/backend/report_check_execution.py@@ -1,16 +1,7 @@-"""Execution of the checks attached to signal reports.+"""Coordinate interim check measurements during enabled Monitoring.-The coordinator calls ``run_due_report_checks`` on its tick, and each due check advances by one-step. A ``metric_threshold`` check is answered here and then: one bounded Trends query, one-comparison, one artefact, with no sandbox, no scout enrolment, and no LLM. An ``agent`` check needs-a run to decide it, so it is handed to ``report_check_agent`` and closed later by the tool that run-calls.--Terminal results go through ``record_check_verdict``. Monitoring metric observations use-``report_check_monitoring`` to save measurements without concluding the check or report.--The comparison is the alerts product's, not a second one written here, so a check and an alert word-a breach the same way and there is one place where "is this value out of bounds?" is decided.+Final verification and authored agent checks use separate execution contracts. The legacy verdict+utilities below do not run from the coordinator."""from __future__ import annotations@@ -842,16 +833,9 @@ def collect_due_checks(candidates = (eligible_checks.annotate(_monitoring_ends_at=_monitoring_window_end_expression()).filter(- report_status_filter(- [SignalReport.Status.RESOLVED],- field="report__status",- disabled_monitoring_organization_ids=disabled_monitoring_organization_ids,- )- | (- _monitoring_metric_filter(disabled_monitoring_organization_ids)- & Q(report__monitoring_started_at__gt=now - MAX_CHECK_HORIZON, _monitoring_ends_at__gt=now)- ),- # Agent and legacy metric checks must stay dormant until the report resolves.+ _monitoring_metric_filter(disabled_monitoring_organization_ids),+ report__monitoring_started_at__gt=now - MAX_CHECK_HORIZON,+ _monitoring_ends_at__gt=now,).select_related("report", "report__team", "team__organization")# Rank each team's rows against its own, then read those ranks in order, so every team's@@ -869,12 +853,6 @@ def collect_due_checks(per_team: dict[int, int] = {}due: list[SignalReportCheck] = []for check in candidates:- if (- check.report.status == SignalReport.Status.MONITORING- and check.team.organization_id in disabled_monitoring_organization_ids- and _resolved_monitoring_expiry(check) <= now- ):- continuetaken = per_team.get(check.team_id, 0)if taken >= MAX_CHECK_RUNS_PER_TEAM_PER_TICK:continue@@ -896,32 +874,26 @@ def _arm_monitoring_checks(now: datetime, *, disabled_monitoring_organization_id)reports = (SignalReport.objects.filter(- Q(monitoring_started_at__isnull=False) | Q(team__organization_id__in=disabled_monitoring_organization_ids),Exists(pending_checks),status=SignalReport.Status.MONITORING,+ monitoring_started_at__isnull=False,)+ .exclude(team__organization_id__in=disabled_monitoring_organization_ids).select_related("team").only("id", "status", "team_id", "monitoring_started_at", "team__organization_id").order_by("id")[:MAX_CHECK_PARKS_PER_TICK])for report in reports:- monitoring_enabled = report.team.organization_id not in disabled_monitoring_organization_idsarm_pending_checks(team_id=report.team_id,report_id=report.id,resolved_at=now,- monitoring_enabled=monitoring_enabled,- expected_monitoring_started_at=report.monitoring_started_at if monitoring_enabled else None,+ expected_monitoring_started_at=report.monitoring_started_at,)def run_due_report_checks(*, now: datetime | None = None, limit: int = MAX_CHECK_RUNS_PER_TICK) -> CheckRunSummary:- """Expire what timed out, park what lost its resolve, then advance every due check by one step.-- A `metric_threshold` check is measured and recorded here. An `agent` check is dispatched (or- its overdue dispatch is written off), and the run it started records the verdict later, so the- tick's time budget bounds the dispatch and not the investigation.- """+ """Expire stale checks and record due interim measurements during enabled Monitoring."""now = now or timezone.now()candidate_checks = SignalReportCheck.all_teams.filter(@@ -929,7 +901,6 @@ def run_due_report_checks(*, now: datetime | None = None, limit: int = MAX_CHECK)disabled_organizations = check_monitoring_disabled_organizations(candidate_checks)_prepare_monitoring_metrics(now, disabled_organizations)- _restore_resolved_monitoring_expiries(now, disabled_organizations)expired = expire_overdue_checks(now, disabled_monitoring_organization_ids=disabled_organizations)_arm_monitoring_checks(now, disabled_monitoring_organization_ids=disabled_organizations)park_checks_on_unresolved_reports(now, disabled_monitoring_organization_ids=disabled_organizations)@@ -947,87 +918,13 @@ def run_due_report_checks(*, now: datetime | None = None, limit: int = MAX_CHECKfor check in collect_due_checks(now, limit=limit, disabled_monitoring_organization_ids=disabled_organizations):if time.monotonic() >= deadline:break- if (- check.report.status == SignalReport.Status.MONITORING- and check.team.organization_id not in disabled_organizations- and check.kind == SignalReportCheck.Kind.METRIC_THRESHOLD- and check.config.get("monitoring")- and check.report.monitoring_started_at is not None- ):- try:- observation = measure_monitoring_check(check, now=now, deadline=deadline)- counts["observed"] += int(record_monitoring_observation(check, observation))- except (MetricQueryBudgetExhausted, MonitoringWindowClosed):- counts["deferred"] += 1- except Exception:- logger.exception(- "signals.report_check.monitoring_step_failed", check_id=str(check.id), team_id=check.team_id- )- continue- if (- check.report.status == SignalReport.Status.MONITORING- and check.team.organization_id in disabled_organizations- and _resolved_monitoring_expiry(check) != check.expires_at- ):- counts["deferred"] += 1- continue- if check.kind == SignalReportCheck.Kind.AGENT:- # Deferred: the agent lane reaches the scout harness and the Temporal client, and this- # module is imported by the REST route load, which never dispatches anything.- from products.signals.backend.report_check_agent import run_agent_check # noqa: PLC0415-- try:- counts[run_agent_check(check, now=now)] += 1- except Exception:- logger.exception(- "signals.report_check.agent_step_failed", check_id=str(check.id), team_id=check.team_id- )- continue- verdict: CheckVerdict | None = None- if check.kind == SignalReportCheck.Kind.METRIC_THRESHOLD:- start_at = check.measurement_start_at or now- try:- config = cast(MetricThresholdConfig, parse_check_config(check.kind, check.config))- query = resolve_check_query(config, check.report)- ready_at = metric_check_ready_at(query, check.report.team, start_at)- if config.monitoring is not None:- ready_at = max(ready_at, start_at + timedelta(minutes=check.soak_minutes or 0))- except (CheckConfigValidationError, ValueError):- ready_at = now- if check.measurement_start_at is None or now < ready_at:- next_run_at = max(check.next_run_at, ready_at)- expires_at = check.expires_at- if check.measurement_start_at is None:- expires_at = check_schedule_expires_at(- next_run_at=next_run_at,- run_interval_minutes=check.run_interval_minutes,- runs_remaining=check.runs_remaining,- start_at=start_at,- )- if next_run_at >= expires_at:- verdict = CheckVerdict(- outcome="inconclusive",- reason="unmeasurable",- explanation="The full measurement window cannot fit before this check expires.",- )- else:- SignalReportCheck.objects.for_team(check.team_id).filter(- id=check.id,- status=SignalReportCheck.Status.ACTIVE,- measurement_start_at=check.measurement_start_at,- ).update(- measurement_start_at=start_at,- next_run_at=next_run_at,- expires_at=expires_at,- updated_at=now,- )- continue- if verdict is None:- verdict = measure_check(check, deadline=deadline)try:- record_check_verdict(check, verdict)+ observation = measure_monitoring_check(check, now=now, deadline=deadline)+ counts["observed"] += int(record_monitoring_observation(check, observation))+ except (MetricQueryBudgetExhausted, MonitoringWindowClosed):+ counts["deferred"] += 1except Exception:- logger.exception("signals.report_check.persist_failed", check_id=str(check.id), team_id=check.team_id)- continue- counts[verdict.outcome] += 1+ logger.exception(+ "signals.report_check.monitoring_step_failed", check_id=str(check.id), team_id=check.team_id+ )return CheckRunSummary(expired=expired, **counts)diff --git a/products/signals/backend/report_check_monitoring.py b/products/signals/backend/report_check_monitoring.pyindex 6fdb6bc0228a..0612e6cc5a5e 100644--- a/products/signals/backend/report_check_monitoring.py+++ b/products/signals/backend/report_check_monitoring.py@@ -37,6 +37,13 @@ class MonitoringWindowClosed(Exception):def measure_monitoring_check(check: SignalReportCheck, *, now: datetime, deadline: float) -> CheckObservation:start = check.report.monitoring_started_atassert start is not None+ if not SignalReport.objects.filter(+ id=check.report_id,+ team_id=check.team_id,+ status=SignalReport.Status.MONITORING,+ monitoring_started_at=start,+ ).exists() or not team_report_monitoring_enabled(check.team_id):+ raise MonitoringWindowClosedend = monitoring_window_end(start, check.soak_minutes)if now >= end:raise MonitoringWindowCloseddiff --git a/products/signals/backend/scout_harness/tools/checks.py b/products/signals/backend/scout_harness/tools/checks.pyindex d2ed69e76c9f..0e5da07c1dd0 100644--- a/products/signals/backend/scout_harness/tools/checks.py+++ b/products/signals/backend/scout_harness/tools/checks.py@@ -86,7 +86,7 @@ def _pause_reason(check: SignalReportCheck, report_status: str, now: datetime) -return "its report is `monitoring`"if now >= min(monitoring_window_end(start, check.soak_minutes), start + MAX_CHECK_HORIZON):return "its Monitoring window ended"- elif report_status != SignalReport.Status.RESOLVED:+ else:return f"its report is `{report_status}`"if check.expires_at <= now:return "its horizon passed"@@ -113,6 +113,9 @@ def _resolve_dispatched_check(team: Team, run: SignalScoutRun, check_id: str) ->raise InvalidCheckResultError(f"check {check_id} not found")if check.kind != SignalReportCheck.Kind.AGENT:raise InvalidCheckResultError(f"check {check_id} is a `{check.kind}` check, which the coordinator measures")+ pause_reason = _pause_reason(check, check.report.effective_status, timezone.now())+ if pause_reason is not None:+ raise InvalidCheckResultError(f"check {check_id} is paused because {pause_reason}")if check.status == SignalReportCheck.Status.PENDING:raise InvalidCheckResultError(f"check {check_id} waits for its report to resolve, so there is no fix to measure yet"@@ -244,9 +247,9 @@ def _run_state(check: SignalReportCheck, report_status: str, dispatched_run_id:return "waiting_on_report"if check.status != SignalReportCheck.Status.ACTIVE:return check.status+ if _pause_reason(check, report_status, now) is not None:+ return "paused"if check.dispatched_at is None:- if _pause_reason(check, report_status, now) is not None:- return "paused"return "due" if check.next_run_at <= now else "scheduled"if check.dispatched_at + AGENT_CHECK_RESULT_WINDOW <= now:return "stale"@@ -256,6 +259,7 @@ def _run_state(check: SignalReportCheck, report_status: str, dispatched_run_id:def _summarize(check: SignalReportCheck, report_status: str, dispatched_run_id: str | None = None, now: datetime | None = None) -> ScoutCheckSummary:+ run_state = _run_state(check, report_status, dispatched_run_id, now or timezone.now())return ScoutCheckSummary(check_id=str(check.id),report_id=str(check.report_id),@@ -266,9 +270,8 @@ def _summarize(next_run_at=check.next_run_at,last_outcome=check.last_outcome,last_outcome_reason=check.last_outcome_reason,- run_state=_run_state(check, report_status, dispatched_run_id, now or timezone.now()),- # Cancel and expiry leave `dispatched_at` set, so only an open check can still owe a run.- waiting_on_run=check.status == SignalReportCheck.Status.ACTIVE and check.dispatched_at is not None,+ run_state=run_state,+ waiting_on_run=run_state in {"queued", "running", "stale"},dispatched_run_id=dispatched_run_id,dispatched_at=check.dispatched_at,)diff --git a/products/signals/backend/temporal/agentic/scout_scheduler.py b/products/signals/backend/temporal/agentic/scout_scheduler.pyindex 3390967cb2de..5dd58c796c02 100644--- a/products/signals/backend/temporal/agentic/scout_scheduler.py+++ b/products/signals/backend/temporal/agentic/scout_scheduler.py@@ -256,6 +256,19 @@ async def _run_signals_scout(input: RunSignalsScoutInput) -> RunSignalsScoutOutpwhich run outside the run-row try/except — so a transient drop is reported as a failedrun rather than escaping the activity and breaching the "never raises" contract."""+ # Queued check workflows can outlive their dispatch, so retiring the coordinator lane alone+ # cannot prevent them from starting paid scout work.+ if input.check_id is not None or input.triggered_by == TRIGGERED_BY_CHECK:+ return RunSignalsScoutOutput(+ run_id=None,+ task_run_id=None,+ status=None,+ runtime_s=0.0,+ skill_name=input.skill_name,+ skill_version=input.skill_version or 0,+ skip_reason="report_check_execution_disabled",+ )+# Skip the run when the team is over its self-driving credits quota or its daily report limit,# before any LLM work: a scout run exists to feed signals into the pipeline, and ingestion# drops them while either limit binds. The quota side reads `self_driving_quota_gate` ratherdiff --git a/products/signals/backend/test/test_report_check_concurrency.py b/products/signals/backend/test/test_report_check_concurrency.pyindex fd0076491dcf..baf23a6cbeaa 100644--- a/products/signals/backend/test/test_report_check_concurrency.py+++ b/products/signals/backend/test/test_report_check_concurrency.py@@ -224,8 +224,7 @@ class TestReportCheckConcurrency(NonAtomicBaseTest):else SignalReportArtefact.ArtefactType.CHECK_EXPIRED,).exists()- @parameterized.expand([("enabled", True), ("disabled", False)])- def test_schedule_repair_preserves_a_concurrent_observation(self, _name: str, flag: bool) -> None:+ def test_schedule_repair_preserves_a_concurrent_observation(self) -> None:now = timezone.now()start = now - timedelta(days=1)self.report.status = SignalReport.Status.MONITORING@@ -245,7 +244,7 @@ class TestReportCheckConcurrency(NonAtomicBaseTest):explanation="The metric query failed.",)with (- patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=flag),+ patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True),ThreadPoolExecutor(max_workers=1) as executor,):with transaction.atomic():diff --git a/products/signals/backend/test/test_report_checks.py b/products/signals/backend/test/test_report_checks.pyindex 5a7cc2e8ea4e..f155faa1b5dd 100644--- a/products/signals/backend/test/test_report_checks.py+++ b/products/signals/backend/test/test_report_checks.py@@ -16,7 +16,6 @@ from pydantic import ValidationError as PydanticValidationErrorfrom rest_framework import statusfrom rest_framework.request import Requestfrom rest_framework.test import APIRequestFactory, force_authenticate-from temporalio.exceptions import WorkflowAlreadyStartedErrorfrom posthog.clickhouse.client.connection import ClickHouseUserfrom posthog.clickhouse.workload import Workload@@ -40,7 +39,6 @@ from products.signals.backend.models import ()from products.signals.backend.report_check_agent import (AGENT_CHECK_RESULT_WINDOW,- CHECK_DISPATCH_DEFER_AFTER,FALLBACK_CHECK_SKILL_NAME,build_check_run_note,reactivate_checks_errored_by_scout_pause,@@ -66,7 +64,11 @@ from products.signals.backend.report_check_execution import (resolve_check_query,run_due_report_checks,)-from products.signals.backend.report_check_monitoring import measure_monitoring_check, record_monitoring_observation+from products.signals.backend.report_check_monitoring import (+ MonitoringWindowClosed,+ measure_monitoring_check,+ record_monitoring_observation,+)from products.signals.backend.report_check_research import check_versionsfrom products.signals.backend.report_check_timing import metric_check_ready_at, metric_check_window_startfrom products.signals.backend.report_checks import (@@ -83,7 +85,6 @@ from products.signals.backend.report_checks import (MAX_CHECK_SOAK_HOURS,MAX_CONSECUTIVE_CHECK_ERRORS,MIN_CHECK_INTERVAL_MINUTES,- MIN_CHECK_SOAK_HOURS,AgentCheckConfig,CheckComparison,CheckConfigValidationError,@@ -633,6 +634,53 @@ class TestReportCheckExecution(APIBaseTest):fields.update(overrides)return self._check(**fields)+ @parameterized.expand(+ [+ (f"{state}_{name}_{kind}", state, flag, kind)+ for state, name, flag in [+ (SignalReport.Status.RESOLVED, "enabled", True),+ (SignalReport.Status.MONITORING, "disabled", False),+ (SignalReport.Status.MONITORING, "unavailable", RuntimeError("flag unavailable")),+ ]+ for kind in [SignalReportCheck.Kind.METRIC_THRESHOLD, SignalReportCheck.Kind.AGENT]+ ]+ )+ def test_effectively_resolved_reports_do_not_execute_checks(+ self, _name: str, report_status: str, flag_result: bool | Exception, kind: str+ ) -> None:+ now = timezone.now()+ check = self._monitoring_check(+ start=now - timedelta(days=1),+ next_run_at=now - timedelta(minutes=1),+ kind=kind,+ run_interval_minutes=MIN_CHECK_INTERVAL_MINUTES,+ runs_remaining=3,+ dispatched_at=now - AGENT_CHECK_RESULT_WINDOW if kind == SignalReportCheck.Kind.AGENT else None,+ )+ SignalReport.objects.filter(team_id=self.team.id, id=self.report.id).update(status=report_status)+ with (+ patch(+ "products.signals.backend.report_content_gates.feature_enabled_or_false",+ return_value=flag_result if isinstance(flag_result, bool) else False,+ side_effect=flag_result if isinstance(flag_result, Exception) else None,+ ),+ patch(_MEASURE) as legacy_measure,+ patch("products.signals.backend.report_check_monitoring.measure_metric") as monitoring_measure,+ patch(_CONNECT),+ patch(_DISPATCH) as dispatch,+ ):+ assert collect_due_checks(now) == []+ summary = run_due_report_checks(now=now)+ legacy_measure.assert_not_called()+ monitoring_measure.assert_not_called()+ dispatch.assert_not_called()+ assert summary.passed == summary.failed == summary.errored == summary.dispatched == 0+ check.refresh_from_db()+ assert check.runs_remaining == 3+ assert check.last_run_at is None and check.last_outcome is None+ assert self._results() == []+ assert not self.report.artefacts.filter(type="check_observation").exists()+@parameterized.expand([("daily", None, 1440), ("authored", 360, 360)])def test_monitoring_samples_repeat_without_consuming_final_runs(self, _name: str, interval: int | None, minutes: int@@ -755,6 +803,29 @@ class TestReportCheckExecution(APIBaseTest):content = json.loads(self.report.artefacts.get(type="check_observation").content)assert content["observed_value"] == 2 and content["error"] is None+ @parameterized.expand([("resolved",), ("archived",), ("flag_off",), ("new_period",)])+ def test_invalidated_monitoring_snapshot_cannot_start_a_query(self, race: str) -> None:+ now = timezone.now()+ check = self._monitoring_check(start=now - timedelta(days=1), next_run_at=now - timedelta(minutes=1))+ assert check.report.monitoring_started_at == now - timedelta(days=1)+ if race in {"resolved", "archived"}:+ SignalReport.objects.filter(team_id=self.team.id, id=self.report.id).update(+ status=SignalReport.Status.RESOLVED if race == "resolved" else SignalReport.Status.SUPPRESSED+ )+ elif race == "new_period":+ SignalReport.objects.filter(team_id=self.team.id, id=self.report.id).update(monitoring_started_at=now)+ with (+ patch(+ "products.signals.backend.report_content_gates.feature_enabled_or_false",+ return_value=race != "flag_off",+ ),+ patch("products.signals.backend.report_check_monitoring.measure_metric") as measure,+ ):+ with self.assertRaises(MonitoringWindowClosed):+ measure_monitoring_check(check, now=now, deadline=time.monotonic() + 60)+ measure.assert_not_called()+ assert not self.report.artefacts.filter(type="check_observation").exists()+@parameterized.expand([("duplicate",),@@ -771,9 +842,12 @@ class TestReportCheckExecution(APIBaseTest):def test_stale_monitoring_observations_are_not_saved(self, race: str) -> None:start = timezone.now() - timedelta(days=1)check = self._monitoring_check(start=start, next_run_at=timezone.now() - timedelta(minutes=1))- with patch(- "products.signals.backend.report_check_monitoring.measure_metric",- return_value=MetricMeasurement(value=8, measured_at=timezone.now(), series=None),+ with (+ patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True),+ patch(+ "products.signals.backend.report_check_monitoring.measure_metric",+ return_value=MetricMeasurement(value=8, measured_at=timezone.now(), series=None),+ ),):observation = measure_monitoring_check(check, now=timezone.now(), deadline=time.monotonic() + 60)if race == "cancelled":@@ -811,7 +885,7 @@ class TestReportCheckExecution(APIBaseTest):("missing_start", True, "missing_start", -1, "paused"),("agent", True, "agent", -1, "paused"),("legacy_metric", True, "legacy_metric", -1, "paused"),- ("flag_off", False, "metric", -1, "due"),+ ("flag_off", False, "metric", -1, "paused"),])def test_scout_monitoring_run_state_matches_coordinator(@@ -840,319 +914,6 @@ class TestReportCheckExecution(APIBaseTest):assert listed.run_state == expectedassert {row.id for row in collect_due_checks(now)} == ({check.id} if expected == "due" else set())- @parameterized.expand(- [("disabled", False, 14, 14 * 24), ("unavailable", None, 14, 14 * 24), ("query_window", False, 2, 7 * 24 + 12)]- )- def test_flag_off_restores_the_legacy_soak_and_whole_window(- self, _name: str, flag: bool | None, soak_days: int, ready_hours: int- ) -> None:- start = datetime(2026, 10, 1, 12, tzinfo=UTC)- ready_at = start + timedelta(hours=ready_hours)- with time_machine.travel(start, tick=False):- check = self._monitoring_check(start=start, soak_minutes=soak_days * 1440)- with (- patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True),- patch(- "products.signals.backend.report_check_monitoring.measure_metric",- return_value=MetricMeasurement(value=8, measured_at=start, series=None),- ),- ):- run_due_report_checks(now=start + timedelta(days=1))- with (- patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=flag),- patch(_MEASURE, return_value=MetricMeasurement(value=2, measured_at=start, series=None)) as measure,- ):- run_due_report_checks(now=start + timedelta(days=2))- measure.assert_not_called()- check.refresh_from_db()- assert check.next_run_at == ready_at- assert check.expires_at == ready_at + DEFAULT_CHECK_EXPIRY_AFTER_LAST_RUN- with time_machine.travel(ready_at, tick=False):- run_due_report_checks()- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.PASSED- assert check.runs_remaining == 0-- @parameterized.expand([("repaired", 500, 45), ("overdue_backlog", 0, 45), ("future_backlog", 0, 20)])- @time_machine.travel("2026-10-01T12:00:00Z", tick=False)- def test_flag_off_defers_unrepaired_checks_and_expires_overdue_checks(- self, _name: str, repair_limit: int, age_days: int- ) -> None:- start = timezone.now() - timedelta(days=age_days)- check = self._monitoring_check(start=start)- with (- patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=False),- patch("products.signals.backend.report_check_execution.MAX_CHECK_PARKS_PER_TICK", repair_limit),- patch(_MEASURE) as measure,- ):- assert collect_due_checks(timezone.now()) == ([] if age_days > 44 else [check])- summary = run_due_report_checks()- measure.assert_not_called()- assert summary.deferred == (1 if age_days < 44 else 0)- check.refresh_from_db()- assert check.status == (SignalReportCheck.Status.EXPIRED if repair_limit else SignalReportCheck.Status.ACTIVE)- assert self._results() == []- if repair_limit:- with patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True):- run_due_report_checks(limit=0)- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.EXPIRED-- def test_expiry_restoration_only_changes_disabled_monitoring_checks(self) -> None:- now = timezone.now()- check = self._monitoring_check(start=now)- resolved = SignalReport.objects.create(team=self.team, status=SignalReport.Status.RESOLVED)- legacy = self._check(report=resolved)- other_team = Team.objects.create(organization=Organization.objects.create(name="Another organization"))- other_report = SignalReport.objects.create(- team=other_team, status=SignalReport.Status.MONITORING, monitoring_started_at=now- )- enabled = self._check(- team=other_team,- report=other_report,- config=check.config,- measurement_start_at=now,- expires_at=now + MAX_CHECK_HORIZON,- )- with patch(- "products.signals.backend.report_content_gates.feature_enabled_or_false",- side_effect=lambda _flag, organization_id, **kwargs: organization_id == str(other_team.organization_id),- ):- run_due_report_checks(now=now, limit=0)- for row, expected_expiry in (- (check, now + timedelta(days=44)),- (legacy, legacy.expires_at),- (enabled, enabled.expires_at),- ):- row.refresh_from_db()- assert row.expires_at == expected_expiry-- @parameterized.expand([("passed", False), ("errored", True)])- def test_flag_cycles_preserve_the_fallback_expiry_after_a_verdict(self, _name: str, query_error: bool) -> None:- start = datetime(2026, 10, 1, 12, tzinfo=UTC)- expected_expiry = start + timedelta(days=65)- with time_machine.travel(start + timedelta(days=13), tick=False):- check = self._monitoring_check(start=start, run_interval_minutes=7 * 1440, runs_remaining=4)- with patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=False):- run_due_report_checks(limit=0)- with (- time_machine.travel(start + timedelta(days=14), tick=False),- patch(- _MEASURE,- side_effect=RuntimeError("Query unavailable") if query_error else None,- return_value=MetricMeasurement(value=2, measured_at=timezone.now(), series=None),- ),- ):- run_due_report_checks()- check.refresh_from_db()- assert check.runs_remaining == (4 if query_error else 3)- assert check.expires_at == expected_expiry- with patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True):- run_due_report_checks(now=start + timedelta(days=15), limit=0)- check.refresh_from_db()- assert check.expires_at == start + MAX_CHECK_HORIZON- with patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=False):- run_due_report_checks(now=start + timedelta(days=16), limit=0)- check.refresh_from_db()- assert check.expires_at == expected_expiry-- with (- time_machine.travel(start + timedelta(days=16), tick=False),- patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True),- ):- self.report.refresh_from_db()- with self.captureOnCommitCallbacks(execute=True):- self.report.save(update_fields=self.report.transition_to(SignalReport.Status.SUPPRESSED))- self.report.save(update_fields=self.report.transition_to(self.report.restore_target_status()))- check.refresh_from_db()- assert check.monitoring_fallback_expires_at == expected_expiry- self.report.save(update_fields=self.report.transition_to(SignalReport.Status.READY))- check.refresh_from_db()- self.assertIsNone(check.monitoring_fallback_expires_at)-- def test_expiry_repair_advances_past_checks_whose_legacy_horizon_is_also_ninety_days(self) -> None:- now = timezone.now()- start = now - timedelta(days=2)- long_check = self._monitoring_check(start=start, run_interval_minutes=8 * 1440, runs_remaining=7)- later_start = now - timedelta(days=1)- later_report = SignalReport.objects.create(- team=self.team, status=SignalReport.Status.MONITORING, monitoring_started_at=later_start- )- later_check = self._check(- report=later_report,- config=long_check.config,- measurement_start_at=later_start,- soak_minutes=14 * 1440,- next_run_at=later_start + timedelta(days=14),- expires_at=later_start + MAX_CHECK_HORIZON,- )- with (- patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=False),- patch("products.signals.backend.report_check_execution.MAX_CHECK_PARKS_PER_TICK", 1),- ):- run_due_report_checks(now=now, limit=0)- run_due_report_checks(now=now, limit=0)- long_check.refresh_from_db()- later_check.refresh_from_db()- assert long_check.expires_at == start + MAX_CHECK_HORIZON- assert later_check.expires_at == later_start + timedelta(days=44)-- @parameterized.expand([("capped", 13, 5, 90), ("uncapped", 7, 3, 58)])- def test_legacy_flag_recovery_preserves_elapsed_recurring_runs(- self, _name: str, interval_days: int, runs: int, expiry_days: int- ) -> None:- start = datetime(2026, 10, 1, 12, tzinfo=UTC)- with time_machine.travel(start, tick=False):- check = self._monitoring_check(start=start, run_interval_minutes=interval_days * 1440, runs_remaining=runs)- for check_id, result_at in (- ("00000000-0000-0000-0000-000000000001", start),- (str(check.id), start - timedelta(days=1)),- ):- result = SignalReportArtefact.add_log(- team_id=self.team.id,- report_id=str(self.report.id),- content=CheckResult(- check_id=check_id,- kind=check.kind,- title="Earlier result",- outcome="passed",- explanation="The goal held.",- ),- attribution=ArtefactAttribution.system(),- )- SignalReportArtefact.objects.filter(id=result.id, team_id=self.team.id).update(created_at=result_at)- for run in range(runs - 1):- run_at = start + timedelta(days=14 + interval_days * run)- with (- time_machine.travel(run_at, tick=False),- patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=False),- patch(_MEASURE, return_value=MetricMeasurement(value=2, measured_at=run_at, series=None)),- ):- run_due_report_checks()- with time_machine.travel(run_at + timedelta(days=1), tick=False):- with patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True):- run_due_report_checks(limit=0)- SignalReportCheck.objects.for_team(self.team.id).filter(id=check.id).update(- monitoring_fallback_expires_at=None- )- with patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=False):- run_due_report_checks(limit=0)- check.refresh_from_db()- assert check.runs_remaining == 1- assert check.expires_at == start + timedelta(days=expiry_days)- final_run_at = start + timedelta(days=14 + interval_days * (runs - 1))- with (- time_machine.travel(final_run_at, tick=False),- patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=False),- patch(_MEASURE, return_value=MetricMeasurement(value=2, measured_at=final_run_at, series=None)),- ):- run_due_report_checks()- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.PASSED- assert check.runs_remaining == 0-- @parameterized.expand([("success", False), ("query_error", True)])- def test_flag_recovery_restores_the_latest_monitoring_attempt_schedule(self, _name: str, query_error: bool) -> None:- start = datetime(2026, 10, 1, 12, tzinfo=UTC)- sample_at = start + timedelta(days=1)- interval = CHECK_ERROR_RETRY_AFTER if query_error else timedelta(days=1)- with time_machine.travel(sample_at, tick=False):- check = self._monitoring_check(start=start)- with (- patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True),- patch(- "products.signals.backend.report_check_monitoring.measure_metric",- side_effect=RuntimeError("Query unavailable") if query_error else None,- return_value=MetricMeasurement(value=8, measured_at=sample_at, series=None),- ),- ):- run_due_report_checks()- with patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=False):- run_due_report_checks(now=sample_at + interval)- check.refresh_from_db()- assert check.next_run_at == start + timedelta(days=14)- assert check.expires_at == start + timedelta(days=44)- with (- patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True),- patch(- "products.signals.backend.report_check_monitoring.measure_metric",- return_value=MetricMeasurement(value=9, measured_at=sample_at, series=None),- ) as measure,- ):- run_due_report_checks(now=sample_at + interval - timedelta(minutes=1))- measure.assert_not_called()- check.refresh_from_db()- assert check.next_run_at == sample_at + interval- assert check.expires_at == start + MAX_CHECK_HORIZON- run_due_report_checks(now=sample_at + interval)- measure.assert_called_once()- assert self.report.artefacts.filter(type="check_observation").count() == 2-- @parameterized.expand([("early", 3, 41, False), ("late", 80, 91, False), ("repair_backlog", 80, 91, True)])- def test_flag_recovery_restores_the_original_monitoring_horizon(- self, _name: str, armed_day: int, recovery_day: int, backlog: bool- ) -> None:- start = datetime(2026, 10, 1, 12, tzinfo=UTC)- with time_machine.travel(start + timedelta(days=armed_day), tick=False):- check = self._monitoring_check(start=start, status=SignalReportCheck.Status.PENDING)- with patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=False):- run_due_report_checks()- recovery_at = start + timedelta(days=recovery_day)- with (- time_machine.travel(recovery_at, tick=False),- patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True),- patch(- "products.signals.backend.report_check_monitoring.measure_metric",- return_value=MetricMeasurement(value=8, measured_at=recovery_at, series=None),- ) as measure,- ):- with patch(- "products.signals.backend.report_check_execution.MAX_CHECK_PARKS_PER_TICK", 0 if backlog else 500- ):- run_due_report_checks()- check.refresh_from_db()- if not backlog:- assert check.expires_at == start + MAX_CHECK_HORIZON- if recovery_day >= 90:- measure.assert_not_called()- assert check.status == SignalReportCheck.Status.EXPIRED- assert not self.report.artefacts.filter(type="check_observation").exists()- else:- measure.assert_not_called()- assert check.status == SignalReportCheck.Status.ACTIVE- assert not self.report.artefacts.filter(type="check_observation").exists()-- def test_reenabling_monitoring_after_a_legacy_error_does_not_sample_past_the_soak(self) -> None:- now = datetime(2026, 10, 1, 12, tzinfo=UTC)- start = now - timedelta(days=3)- with time_machine.travel(now, tick=False):- check = self._monitoring_check(start=start, status=SignalReportCheck.Status.PENDING)- with patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=False):- run_due_report_checks()- check.refresh_from_db()- due_at = check.next_run_at- with (- time_machine.travel(due_at, tick=False),- patch(_MEASURE, side_effect=RuntimeError("Query unavailable")),- ):- run_due_report_checks()- sample_at = due_at + CHECK_ERROR_RETRY_AFTER- with (- time_machine.travel(sample_at, tick=False),- patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True),- patch(- "products.signals.backend.report_check_monitoring.measure_metric",- return_value=MetricMeasurement(value=8, measured_at=sample_at, series=None),- ) as measure,- ):- run_due_report_checks()- run_due_report_checks(now=sample_at + timedelta(minutes=1))- measure.assert_not_called()- check.refresh_from_db()- assert check.measurement_start_at == start- assert check.next_run_at == start + timedelta(days=14)- assert not self.report.artefacts.filter(type="check_observation").exists()-@parameterized.expand([("retry", 0, SignalReportCheck.Status.ACTIVE),@@ -1205,221 +966,6 @@ class TestReportCheckExecution(APIBaseTest):).order_by("created_at"))- @parameterized.expand(- [- ("legacy_without_anchor", None, None, 1),- ("legacy_recurring_near_expiry", None, 24 * 60, 3),- ("legacy_recurring_horizon", None, 30 * 24 * 60, 3),- ("prematurely_due", datetime(2026, 10, 1, 12, tzinfo=UTC), None, 1),- ]- )- def test_incomplete_window_defers_without_a_verdict(- self, _name: str, anchor: datetime | None, interval: int | None, runs: int- ) -> None:- now = datetime(2026, 10, 2, 12, tzinfo=UTC)- with time_machine.travel(now, tick=False):- check = self._check(measurement_start_at=anchor, run_interval_minutes=interval, runs_remaining=runs)- original_expiry = check.expires_at- with patch(_MEASURE) as measure:- summary = run_due_report_checks()- check.refresh_from_db()- assert not measure.called- assert summary.passed == summary.failed == summary.inconclusive == summary.errored == 0- assert self._results() == []- assert check.measurement_start_at == (anchor or now)- assert check.next_run_at == metric_check_ready_at(_PAGEVIEWS, self.team, anchor or now)- assert check.next_run_at < check.expires_at <= now + MAX_CHECK_HORIZON- if anchor is None:- last_run_at = check.next_run_at + timedelta(minutes=(interval or 0) * (runs - 1))- assert check.expires_at == min(- last_run_at + DEFAULT_CHECK_EXPIRY_AFTER_LAST_RUN, now + MAX_CHECK_HORIZON- )- assert check.expires_at > original_expiry- else:- assert check.expires_at == original_expiry- assert check.consecutive_errors == 0- assert check.runs_remaining == runs-- with time_machine.travel(check.next_run_at, tick=False):- with patch(_MEASURE, return_value=MetricMeasurement(value=3, measured_at=check.next_run_at, series=None)):- summary = run_due_report_checks()- check.refresh_from_db()- assert summary.expired == 0- assert summary.passed == 1- assert check.runs_remaining == runs - 1- assert len(self._results()) == 1-- @parameterized.expand(- [- ("legacy_window_exceeds_horizon", None, "-90d"),- ("anchored_deadline_before_window", datetime(2026, 10, 2, 12, tzinfo=UTC), "-30d"),- ]- )- def test_unreachable_window_records_inconclusive(self, _name: str, anchor: datetime | None, date_from: str) -> None:- now = datetime(2026, 10, 2, 12, tzinfo=UTC)- with time_machine.travel(now, tick=False):- check = self._check(- measurement_start_at=anchor,- config=_threshold_config(- query=trends_metric_query(- series=[{"kind": "EventsNode", "event": "$pageview"}], date_from=date_from- )- ),- )- original_expiry = check.expires_at- with patch(_MEASURE) as measure:- summary = run_due_report_checks()- check.refresh_from_db()- assert not measure.called- assert summary.inconclusive == 1- assert summary.expired == 0- assert check.status == SignalReportCheck.Status.INCONCLUSIVE- assert check.last_outcome_reason == "unmeasurable"- assert check.expires_at == original_expiry- assert check.next_run_at < now- assert len(self._results()) == 1-- def test_a_one_shot_check_that_holds_retires_as_passed_with_a_result(self) -> None:- check = self._check()- with patch(_MEASURE, return_value=MetricMeasurement(value=3.0, measured_at=timezone.now(), series=None)):- summary = run_due_report_checks()-- assert summary.passed == 1- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.PASSED- assert check.last_outcome == SignalReportCheck.Outcome.PASSED- assert check.last_run_at is not None- results = self._results()- assert len(results) == 1- assert '"outcome":"passed"' in results[0].content- assert '"observed_value":3.0' in results[0].content-- def test_a_recorded_verdict_is_reported_for_adoption(self) -> None:- self._check()- with (- patch(_CAPTURE) as capture,- patch(_ASYNC_CONNECT, return_value=AsyncMock()),- patch(_MEASURE, return_value=MetricMeasurement(value=42.0, measured_at=timezone.now(), series=None)),- ):- with self.captureOnCommitCallbacks(execute=True):- run_due_report_checks()-- # Filtered by event rather than counted: `_CAPTURE` patches an attribute on the shared- # `posthoganalytics` module, so every other capture in the commit — the breach on a resolved- # report emits a signal, which fires its own — lands on this same mock.- evaluated = [- call for call in capture.call_args_list if call.kwargs.get("event") == "signals_report_check_evaluated"- ]- assert len(evaluated) == 1- properties = evaluated[0].kwargs["properties"]- assert properties["outcome"] == "failed"- assert properties["check_status"] == SignalReportCheck.Status.FAILED- assert properties["kind"] == SignalReportCheck.Kind.METRIC_THRESHOLD- assert properties["metric_source"] == "query"- assert properties["run_id"] is None-- def test_a_check_that_breaches_retires_as_failed(self) -> None:- check = self._check()- with patch(_MEASURE, return_value=MetricMeasurement(value=42.0, measured_at=timezone.now(), series=None)):- run_due_report_checks()-- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.FAILED- assert '"outcome":"failed"' in self._results()[0].content-- def test_a_recurring_check_rearms_until_its_runs_are_spent(self) -> None:- check = self._check(run_interval_minutes=MIN_CHECK_INTERVAL_MINUTES, runs_remaining=2)- with patch(_MEASURE, return_value=MetricMeasurement(value=1.0, measured_at=timezone.now(), series=None)):- run_due_report_checks()-- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.ACTIVE- assert check.runs_remaining == 1- assert check.next_run_at > timezone.now()-- check.next_run_at = timezone.now() - timedelta(minutes=1)- check.save(update_fields=["next_run_at"])- with patch(_MEASURE, return_value=MetricMeasurement(value=1.0, measured_at=timezone.now(), series=None)):- run_due_report_checks()-- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.PASSED- assert len(self._results()) == 2-- def test_a_recurring_check_retires_rather_than_running_past_its_horizon(self) -> None:- check = self._check(- run_interval_minutes=MIN_CHECK_INTERVAL_MINUTES,- runs_remaining=2,- expires_at=timezone.now() + timedelta(hours=1),- )- with patch(_MEASURE, return_value=MetricMeasurement(value=1.0, measured_at=timezone.now(), series=None)):- run_due_report_checks()-- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.PASSED-- def test_a_check_that_cannot_be_measured_retries_then_retires(self) -> None:- check = self._check()- for _ in range(MAX_CONSECUTIVE_CHECK_ERRORS):- check.refresh_from_db()- check.next_run_at = timezone.now() - timedelta(minutes=1)- check.save(update_fields=["next_run_at"])- with patch(_MEASURE, side_effect=ValueError("query returned no series")):- run_due_report_checks()-- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.ERRORED- assert len(self._results()) == MAX_CONSECUTIVE_CHECK_ERRORS-- def test_a_single_failure_to_measure_only_delays_the_check(self) -> None:- check = self._check()- with patch(_MEASURE, side_effect=ValueError("boom")):- summary = run_due_report_checks()-- assert summary.errored == 1- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.ACTIVE- assert check.consecutive_errors == 1- assert check.next_run_at >= timezone.now() + CHECK_ERROR_RETRY_AFTER - timedelta(minutes=1)-- def test_a_check_whose_stored_config_stopped_parsing_records_an_errored_run(self) -> None:- check = self._check()- # A config that parsed when it was written and no longer does, as a tightened query rule leaves it.- SignalReportCheck.objects.for_team(self.team.id).filter(id=check.id).update(- config={"comparison": {"operator": "lte", "value": 10}}- )-- summary = run_due_report_checks()-- assert summary.errored == 1- check.refresh_from_db()- assert check.consecutive_errors == 1- assert check.next_run_at > timezone.now()- results = self._results()- assert len(results) == 1- assert '"outcome":"errored"' in results[0].content-- def test_an_errored_run_publishes_our_reason_but_not_a_raw_query_error(self) -> None:- check = self._check()- with patch(_MEASURE, side_effect=ValueError("metric query returned no series")):- run_due_report_checks()-- assert "metric query returned no series" in self._results()[0].content-- check.refresh_from_db()- check.next_run_at = timezone.now() - timedelta(minutes=1)- check.save(update_fields=["next_run_at"])- server_error = RuntimeError("DB::Exception: syntax error\nStack trace:\n0. secret internals")- with patch(_MEASURE, side_effect=server_error):- run_due_report_checks()-- latest = self._results()[-1].content- assert '"outcome":"errored"' in latest- assert "Stack trace" not in latest- assert "secret internals" not in latest- # Named like the passed and failed lines, so a report with several checks stays readable.- assert "Checkout errors stay low could not be measured" in latest-@parameterized.expand([("reopened", SignalReport.Status.READY),@@ -1427,10 +973,7 @@ class TestReportCheckExecution(APIBaseTest):("awaiting_input", SignalReport.Status.PENDING_INPUT),])- def test_an_active_check_on_an_unresolved_report_waits_for_the_next_resolve(self, _name, report_status) -> None:- # An active row on an open report: a report that left `resolved`, or a row written before- # the create path let the report decide. It must not run, and its old error streak must not- # count against the soak that follows the next resolve.+ def test_an_active_check_on_an_unresolved_report_stays_parked_after_resolution(self, _name, report_status) -> None:check = self._check(consecutive_errors=MAX_CONSECUTIVE_CHECK_ERRORS - 1,consecutive_inconclusive=len(AWAITING_DATA_RETRY_WAITS),@@ -1457,18 +1000,13 @@ class TestReportCheckExecution(APIBaseTest):assert check.consecutive_errors == 0assert check.measurement_start_at is None- before = timezone.now()with self.captureOnCommitCallbacks(execute=True):self.report.status = SignalReport.Status.RESOLVEDself.report.save(update_fields=["status"])-- armed = SignalReportCheck.objects.for_team(self.team.id).get(id=check.id)- assert armed.status == SignalReportCheck.Status.ACTIVE- assert armed.measurement_start_at is not None- assert before <= armed.measurement_start_at <= timezone.now()- assert armed.next_run_at == metric_check_ready_at(_PAGEVIEWS, self.team, armed.measurement_start_at)- assert armed.next_run_at >= before + timedelta(hours=MIN_CHECK_SOAK_HOURS)- assert collect_due_checks(timezone.now()) == []+ check.refresh_from_db()+ assert check.status == SignalReportCheck.Status.PENDING+ assert check.measurement_start_at is None+ assert collect_due_checks(timezone.now() + timedelta(days=7)) == []def test_an_active_check_on_an_unresolved_report_is_never_due(self) -> None:self._check()@@ -1559,7 +1097,7 @@ class TestReportCheckExecution(APIBaseTest):expected_status = {"cancelled": SignalReportCheck.Status.CANCELLED,"reopened": SignalReportCheck.Status.PENDING,- "resolved_again": SignalReportCheck.Status.ACTIVE,+ "resolved_again": SignalReportCheck.Status.PENDING,"redispatched": SignalReportCheck.Status.ACTIVE,}assert check.status == expected_status[reason]@@ -1647,23 +1185,24 @@ class TestReportCheckExecution(APIBaseTest):def test_one_team_with_a_backlog_does_not_starve_another_team(self) -> None:now = timezone.now()for minutes in (30, 20, 10):- self._check(next_run_at=now - timedelta(minutes=minutes))+ check = self._monitoring_check(start=now - timedelta(days=1), next_run_at=now - timedelta(minutes=minutes))other_team = self.organization.teams.create(name="Other")other_report = SignalReport.objects.create(- team=other_team, status=SignalReport.Status.RESOLVED, title="Other fix"+ team=other_team,+ status=SignalReport.Status.MONITORING,+ monitoring_started_at=now - timedelta(days=1),+ title="Other fix",)- other_check = SignalReportCheck.objects.for_team(other_team.id).create(+ other_check = self._check(team=other_team,report=other_report,- title="Their checkout errors stay low",- kind=SignalReportCheck.Kind.METRIC_THRESHOLD,- config=_threshold_config(),+ config=check.config,+ measurement_start_at=other_report.monitoring_started_at,+ soak_minutes=14 * 1440,next_run_at=now - timedelta(minutes=5),- expires_at=now + timedelta(days=30),)-- due = collect_due_checks(now, limit=2)-+ with patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True):+ due = collect_due_checks(now, limit=2)assert other_check.id in {check.id for check in due}def test_a_check_riding_a_report_metric_measures_that_metric(self) -> None:@@ -1681,41 +1220,6 @@ class TestReportCheckExecution(APIBaseTest):)assert resolve_check_query(config, self.report) == _PAGEVIEWS- def test_a_check_riding_a_metric_whose_query_stopped_conforming_errors(self) -> None:- # The runner reads one series out of the result, so a breakdown would turn an arbitrary- # slice into a recorded verdict.- self.report.metrics = [- {- "metric_id": "checkout-errors",- "title": "Checkout errors",- "kind": "occurrences",- "query": {- **_PAGEVIEWS,- "source": {**_PAGEVIEWS["source"], "breakdownFilter": {"breakdown": "$browser"}},- },- }- ]- self.report.save(update_fields=["metrics"])- check = self._check(config={"metric_id": "checkout-errors", "comparison": {"operator": "lte", "value": 1}})-- with patch(- _MEASURE, return_value=MetricMeasurement(value=0.0, measured_at=timezone.now(), series=None)- ) as measure:- run_due_report_checks()-- assert not measure.called- check.refresh_from_db()- assert check.last_outcome == SignalReportCheck.Outcome.ERRORED- assert '"outcome":"errored"' in self._results()[0].content-- def test_a_check_riding_a_metric_the_report_dropped_errors_rather_than_crashing(self) -> None:- check = self._check(config={"metric_id": "gone", "comparison": {"operator": "lte", "value": 1}})- run_due_report_checks()-- check.refresh_from_db()- assert check.last_outcome == SignalReportCheck.Outcome.ERRORED- assert '"outcome":"errored"' in self._results()[0].content-class TestReportCheckAPI(APIBaseTest):def setUp(self) -> None:@@ -2096,7 +1600,8 @@ class TestReportCheckAPI(APIBaseTest):with patch(_MEASURE, return_value=MetricMeasurement(value=0.0, measured_at=timezone.now(), series=None)) as measure:- run_due_report_checks()+ stored.refresh_from_db()+ measure_check(stored, deadline=time.monotonic() + 60)assert measure.call_args.kwargs["ch_user"] == ClickHouseUser.SIGNALSmeasured_query = measure.call_args.args[0]assert {key: value for key, value in measured_query["source"].items() if key != "dateRange"} == {@@ -2154,8 +1659,8 @@ class TestReportCheckAPI(APIBaseTest):SignalReportCheck.all_teams.filter(id=check_id).update(next_run_at=timezone.now() - timedelta(minutes=1), measurement_start_at=timezone.now() - timedelta(days=32))- with patch(_MEASURE, return_value=MetricMeasurement(value=0.0, measured_at=timezone.now(), series=None)):- run_due_report_checks()+ check = SignalReportCheck.objects.for_team(child.id).get(id=check_id)+ record_check_verdict(check, CheckVerdict(outcome="passed", explanation="Held.", observed_value=0.0))result = SignalReportArtefact.objects.get(report=report, type=SignalReportArtefact.ArtefactType.CHECK_RESULT)assert result.team_id == child.id@@ -2199,7 +1704,11 @@ class TestReportCheckAPI(APIBaseTest):else "products.signals.backend.report_check_monitoring.measure_metric",return_value=MetricMeasurement(value=4.0, measured_at=timezone.now(), series=None),):- run_due_report_checks()+ if artefact_type == "check_observation":+ run_due_report_checks()+ else:+ check = SignalReportCheck.objects.for_team(self.team.id).get(id=check_id)+ record_check_verdict(check, CheckVerdict(outcome="passed", explanation="Held.", observed_value=4.0))artefacts_url = f"/api/projects/{self.team.id}/signals/reports/{self.report.id}/artefacts/"def check_result() -> dict:@@ -2274,335 +1783,81 @@ class TestReportCheckAPI(APIBaseTest):class TestAgentCheckDispatch(APIBaseTest):- """The agent lane: what the coordinator does with a due `agent` check, and what it refuses."""-- def setUp(self) -> None:- super().setUp()- self.report = SignalReport.objects.create(- team=self.team, status=SignalReport.Status.RESOLVED, title="Checkout 500s"- )- self._enrol({"guaranteed_team_ids": [self.team.id]})- LLMSkill.objects.create(team=self.team, name=FALLBACK_CHECK_SKILL_NAME, is_latest=True, deleted=False)- self.scout_config = SignalScoutConfig.objects.create(- team=self.team,- skill_name=FALLBACK_CHECK_SKILL_NAME,- status=SignalScoutConfig.Status.ACTIVE,- enabled=True,- )-- def _enrol(self, payload: dict) -> None:- flag = patch(_FLAG_PAYLOAD, return_value=payload)- flag.start()- self.addCleanup(flag.stop)-- def _check(self, **overrides) -> SignalReportCheck:- now = timezone.now()- fields: dict = {- "team": self.team,- "report": self.report,- "title": "Checkout 500s stay gone",- "kind": SignalReportCheck.Kind.AGENT,- "config": {"instructions": "Re-read the issue and say whether it still fires."},- "next_run_at": now - timedelta(minutes=1),- "expires_at": now + timedelta(days=30),- }- fields.update(overrides)- return SignalReportCheck.objects.for_team(self.team.id).create(**fields)-- def _results(self) -> list[SignalReportArtefact]:- return list(- SignalReportArtefact.objects.filter(- report=self.report, type=SignalReportArtefact.ArtefactType.CHECK_RESULT- ).order_by("created_at")- )-- def test_a_due_check_starts_a_run_and_waits_for_it(self) -> None:- check = self._check()- with patch(_CONNECT), patch(_DISPATCH, return_value="wf-1") as dispatch:- summary = run_due_report_checks()-- assert summary.dispatched == 1- assert dispatch.call_args.kwargs["skill_name"] == FALLBACK_CHECK_SKILL_NAME- assert dispatch.call_args.kwargs["team_id"] == self.team.id- check.refresh_from_db()- # Still owed: a dispatch is not a verdict, so nothing is recorded and the row stays active- # with its next look pushed past the window the run has to answer in.- assert check.status == SignalReportCheck.Status.ACTIVE- assert check.dispatched_at is not None- assert check.next_run_at > timezone.now() + AGENT_CHECK_RESULT_WINDOW - timedelta(minutes=5)- assert self._results() == []-- def test_a_report_reopened_after_collection_does_not_dispatch_a_check(self) -> None:- check = self._check()- self.report.status = SignalReport.Status.READY- self.report.save(update_fields=["status"])-- with patch(_CONNECT), patch(_DISPATCH) as dispatch:- assert run_agent_check(check) == "deferred"-- dispatch.assert_not_called()- check.refresh_from_db()- assert check.dispatched_at is None- assert self._results() == []-- def test_the_run_note_carries_the_brief_and_the_resolution_note(self) -> None:- SignalReportArtefact.append_dismissal(- team_id=self.team.id,- report_id=str(self.report.id),- content=Dismissal(reason="resolved", note="Shipped the retry fix"),- attribution=ArtefactAttribution.system(),- )- check = self._check(- rationale="The fix was a retry, which can mask rather than remove the error.",- config={"instructions": "Re-read the issue.", "probe_hints": ["issue 4821"]},- )-- note = build_check_run_note(check, AgentCheckConfig.model_validate(check.config))-- assert str(check.id) in note- assert "Checkout 500s stay gone" in note- assert "which can mask rather than remove the error" in note- assert "issue 4821" in note- assert "Shipped the retry fix" in note- assert "scout-check-record-result" in note-- def test_a_dispatched_run_is_listed_on_its_check_and_records_its_verdict(self) -> None:- check = self._check()- with patch(_CONNECT), patch(_DISPATCH, return_value="wf-1") as dispatch:- run_due_report_checks()- assert dispatch.call_args.kwargs["check_id"] == str(check.id)-- (queued,) = list_report_checks(team=self.team, report_id=str(self.report.id))- assert (queued.run_state, queued.waiting_on_run, queued.dispatched_run_id) == ("queued", True, None)-- task = Task.objects.create(team=self.team, title="t", description="d")- run = SignalScoutRun.objects.create(- task_run=TaskRun.objects.create(task=task, team=self.team, status=TaskRun.Status.IN_PROGRESS),- team=self.team,- scout_config=self.scout_config,- skill_name=FALLBACK_CHECK_SKILL_NAME,- skill_version=1,- metadata={"check_id": dispatch.call_args.kwargs["check_id"]},- )- (running,) = list_report_checks(team=self.team, report_id=str(self.report.id))- assert (running.run_state, running.dispatched_run_id) == ("running", str(run.id))-- result = record_check_result(- team=self.team, run=run, check_id=str(check.id), outcome="passed", explanation="No events since the fix."- )-- assert result.check_status == SignalReportCheck.Status.PASSED- (closed,) = list_report_checks(team=self.team, report_id=str(self.report.id))- assert (closed.run_state, closed.waiting_on_run) == (SignalReportCheck.Status.PASSED, False)-- def test_a_check_naming_a_scout_runs_on_that_scout(self) -> None:- LLMSkill.objects.create(team=self.team, name=_OTHER_SKILL, is_latest=True, deleted=False)- SignalScoutConfig.objects.create(- team=self.team, skill_name=_OTHER_SKILL, status=SignalScoutConfig.Status.ACTIVE, enabled=True- )- self._check(config={"instructions": "Re-read the issue.", "skill_name": _OTHER_SKILL})-- with patch(_CONNECT), patch(_DISPATCH, return_value="wf-1") as dispatch:- run_due_report_checks()-- assert dispatch.call_args.kwargs["skill_name"] == _OTHER_SKILL-- @parameterized.expand(- [- (- "warned_lane_still_runs",- {- "enabled": True,- "status": SignalScoutConfig.Status.PENDING_PAUSE,- "pause_reason": SignalScoutConfig.PauseReason.NO_OUTPUT,- },- None,- ),- ("missing_skill_row", None, "has no"),- ]- )- def test_a_lane_that_cannot_run_records_a_visible_errored_result(self, _name, config_state, expected) -> None:- if config_state is None:- LLMSkill.objects.filter(team=self.team, name=FALLBACK_CHECK_SKILL_NAME).delete()- else:- for field, value in config_state.items():- setattr(self.scout_config, field, value)- self.scout_config.save()- check = self._check()-- with patch(_CONNECT), patch(_DISPATCH, return_value="wf-1") as dispatch:- summary = run_due_report_checks()-- # A scout the harness only warned is still scheduled, so a check on it still runs.- if expected is None:- assert summary.dispatched == 1- dispatch.assert_called_once()- assert self._results() == []- return-- assert summary.errored == 1- dispatch.assert_not_called()- results = self._results()- assert len(results) == 1- assert '"outcome":"errored"' in results[0].content- assert expected in results[0].content- check.refresh_from_db()- assert check.consecutive_errors == 1-- def test_a_check_on_a_project_with_no_scouts_is_cancelled_not_errored(self) -> None:- self.scout_config.delete()- check = self._check()-- with patch(_CONNECT), patch(_DISPATCH) as dispatch:- summary = run_due_report_checks()-- assert (summary.cancelled, summary.errored) == (1, 0)- dispatch.assert_not_called()- assert self._results() == []- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.CANCELLED- cancelled = SignalReportArtefact.objects.get(- report=self.report, type=SignalReportArtefact.ArtefactType.CHECK_CANCELLED- )- assert '"reason":"no_check_lane"' in cancelled.content-- def test_a_check_on_a_retired_scout_runs_on_the_follow_up_scout(self) -> None:- # Retiring a scout must not turn every open check bound to it into an errored result on a- # report. The fallback scout re-measures resolved reports for a living, so it answers them.- LLMSkill.objects.create(team=self.team, name="signals-scout-health-checks", is_latest=True, deleted=False)- SignalScoutConfig.objects.create(- team=self.team,- skill_name="signals-scout-health-checks",- status=SignalScoutConfig.Status.PAUSED_BY_SYSTEM,- pause_reason=SignalScoutConfig.PauseReason.RETIRED,- enabled=False,- )- self._check(- config={- "instructions": "Re-read the issue and say whether it still fires.",- "skill_name": "signals-scout-health-checks",- }- )-- with patch(_CONNECT), patch(_DISPATCH, return_value="wf-1") as dispatch:- summary = run_due_report_checks()-- assert summary.dispatched == 1- assert dispatch.call_args.kwargs["skill_name"] == FALLBACK_CHECK_SKILL_NAME- assert self._results() == []-- @parameterized.expand([("named_lane", "signals-scout-health-checks"), ("fallback_lane", None)])- def test_a_check_on_a_paused_scout_waits_for_the_resume_without_spending_errors(self, _name, skill_name) -> None:- # A pause is somebody's decision, so the check neither runs on another scout nor burns its- # error budget while the pause lasts.- if skill_name is None:- scout_config = self.scout_config- scout_config.enabled = False- scout_config.save()- config = {"instructions": "Re-read the issue and say whether it still fires."}- else:- LLMSkill.objects.create(team=self.team, name=skill_name, is_latest=True, deleted=False)- scout_config = SignalScoutConfig.objects.create(team=self.team, skill_name=skill_name, enabled=False)- config = {"instructions": "Re-read the issue and say whether it still fires.", "skill_name": skill_name}- check = self._check(config=config)-- for _ in range(MAX_CONSECUTIVE_CHECK_ERRORS):- SignalReportCheck.objects.for_team(self.team.id).filter(id=check.id).update(- next_run_at=timezone.now() - timedelta(minutes=1)- )- with patch(_CONNECT), patch(_DISPATCH) as dispatch:- summary = run_due_report_checks()- assert summary.deferred == 1- dispatch.assert_not_called()-- assert self._results() == []- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.ACTIVE- assert check.consecutive_errors == 0-- scout_config.enabled = True- scout_config.save()- check.refresh_from_db()- assert check.next_run_at <= timezone.now()- with patch(_CONNECT), patch(_DISPATCH, return_value="wf-1") as dispatch:- summary = run_due_report_checks()-- assert summary.dispatched == 1- assert dispatch.call_args.kwargs["skill_name"] == (skill_name or FALLBACK_CHECK_SKILL_NAME)-- def test_an_unenrolled_project_records_an_errored_result(self) -> None:- self._enrol({"guaranteed_team_ids": []})- self._check()-- with patch(_CONNECT), patch(_DISPATCH) as dispatch:- summary = run_due_report_checks()-- assert summary.errored == 1- dispatch.assert_not_called()- assert len(self._results()) == 1+ """The agent lane: what the coordinator does with a due `agent` check, and what it refuses."""- def test_a_lane_already_running_waits_instead_of_spending_an_error(self) -> None:- task = Task.objects.create(team=self.team, title="t", description="d")- task_run = TaskRun.objects.create(task=task, team=self.team, status=TaskRun.Status.IN_PROGRESS)- SignalScoutRun.objects.create(- task_run=task_run,+ def setUp(self) -> None:+ super().setUp()+ self.report = SignalReport.objects.create(+ team=self.team, status=SignalReport.Status.RESOLVED, title="Checkout 500s"+ )+ self._enrol({"guaranteed_team_ids": [self.team.id]})+ LLMSkill.objects.create(team=self.team, name=FALLBACK_CHECK_SKILL_NAME, is_latest=True, deleted=False)+ self.scout_config = SignalScoutConfig.objects.create(team=self.team,- scout_config=self.scout_config,skill_name=FALLBACK_CHECK_SKILL_NAME,- skill_version=1,+ status=SignalScoutConfig.Status.ACTIVE,+ enabled=True,)- check = self._check()-- with patch(_CONNECT), patch(_DISPATCH) as dispatch, patch(_CAPTURE) as capture:- summary = run_due_report_checks()- assert summary.deferred == 1- dispatch.assert_not_called()- assert capture.call_args.kwargs["event"] == "signals_report_check_dispatch"- assert capture.call_args.kwargs["properties"]["outcome"] == "deferred"- assert capture.call_args.kwargs["properties"]["reason"] == "run_in_flight"- assert self._results() == []- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.ACTIVE- assert check.consecutive_errors == 0- assert check.dispatched_at is None- assert check.next_run_at > timezone.now() + CHECK_DISPATCH_DEFER_AFTER - timedelta(minutes=5)+ def _enrol(self, payload: dict) -> None:+ flag = patch(_FLAG_PAYLOAD, return_value=payload)+ flag.start()+ self.addCleanup(flag.stop)- def test_a_dispatch_that_never_started_leaves_the_check_unclaimed(self) -> None:- check = self._check()- with patch(_CONNECT), patch(_DISPATCH, side_effect=WorkflowAlreadyStartedError("id", "type")):- summary = run_due_report_checks()+ def _check(self, **overrides) -> SignalReportCheck:+ now = timezone.now()+ fields: dict = {+ "team": self.team,+ "report": self.report,+ "title": "Checkout 500s stay gone",+ "kind": SignalReportCheck.Kind.AGENT,+ "config": {"instructions": "Re-read the issue and say whether it still fires."},+ "next_run_at": now - timedelta(minutes=1),+ "expires_at": now + timedelta(days=30),+ }+ fields.update(overrides)+ return SignalReportCheck.objects.for_team(self.team.id).create(**fields)- assert summary.deferred == 1- check.refresh_from_db()- assert check.dispatched_at is None- assert check.status == SignalReportCheck.Status.ACTIVE+ def _results(self) -> list[SignalReportArtefact]:+ return list(+ SignalReportArtefact.objects.filter(+ report=self.report, type=SignalReportArtefact.ArtefactType.CHECK_RESULT+ ).order_by("created_at")+ )- @parameterized.expand([("live_lane", True), ("paused_lane", False)])- def test_a_run_that_never_records_a_result_errors_the_check_unless_its_lane_is_paused(- self, _name, lane_enabled- ) -> None:- now = timezone.now()- check = self._check(dispatched_at=now - AGENT_CHECK_RESULT_WINDOW, next_run_at=now - timedelta(minutes=1))- self.scout_config.enabled = lane_enabled- self.scout_config.save()+ def test_a_report_reopened_after_collection_does_not_dispatch_a_check(self) -> None:+ check = self._check()+ self.report.status = SignalReport.Status.READY+ self.report.save(update_fields=["status"])with patch(_CONNECT), patch(_DISPATCH) as dispatch:- summary = run_due_report_checks()+ assert run_agent_check(check) == "deferred"dispatch.assert_not_called()check.refresh_from_db()assert check.dispatched_at is None- assert check.status == SignalReportCheck.Status.ACTIVE- if lane_enabled:- assert summary.errored == 1- assert check.consecutive_errors == 1- assert "ended without recording a result" in self._results()[0].content- else:- assert summary.deferred == 1- assert check.consecutive_errors == 0- assert self._results() == []- assert check.next_run_at > now + CHECK_DISPATCH_DEFER_AFTER - timedelta(minutes=5)+ assert self._results() == []++ def test_the_run_note_carries_the_brief_and_the_resolution_note(self) -> None:+ SignalReportArtefact.append_dismissal(+ team_id=self.team.id,+ report_id=str(self.report.id),+ content=Dismissal(reason="resolved", note="Shipped the retry fix"),+ attribution=ArtefactAttribution.system(),+ )+ check = self._check(+ rationale="The fix was a retry, which can mask rather than remove the error.",+ config={"instructions": "Re-read the issue.", "probe_hints": ["issue 4821"]},+ )++ note = build_check_run_note(check, AgentCheckConfig.model_validate(check.config))++ assert str(check.id) in note+ assert "Checkout 500s stay gone" in note+ assert "which can mask rather than remove the error" in note+ assert "issue 4821" in note+ assert "Shipped the retry fix" in note+ assert "scout-check-record-result" in note@parameterized.expand([@@ -2631,8 +1886,9 @@ class TestAgentCheckDispatch(APIBaseTest):if reactivated:assert check.status == SignalReportCheck.Status.ACTIVEassert check.consecutive_errors == 0- with patch(_CONNECT), patch(_DISPATCH, return_value="wf-1"):- assert run_due_report_checks().dispatched == 1+ with patch(_CONNECT), patch(_DISPATCH) as dispatch:+ assert run_due_report_checks().dispatched == 0+ dispatch.assert_not_called()else:assert check.status == SignalReportCheck.Status.ERRORED@@ -2684,69 +1940,6 @@ class TestCheckResultTool(APIBaseTest):payload.update(overrides)return record_check_result(team=self.team, run=self.scout_run, check_id=str(check.id), **payload)- def test_a_verdict_closes_the_check_and_lands_on_the_report(self) -> None:- check = self._check()-- result = self._record(check, observed_value=30.0)-- assert result.check_status == SignalReportCheck.Status.FAILED- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.FAILED- assert check.dispatched_at is None- assert check.last_outcome == SignalReportCheck.Outcome.FAILED- artefact = SignalReportArtefact.objects.get(- report=self.report, type=SignalReportArtefact.ArtefactType.CHECK_RESULT- )- assert '"outcome":"failed"' in artefact.content- assert "fired 30 times yesterday" in artefact.content- assert f'"run_id":"{self.scout_run.id}"' in artefact.content-- @parameterized.expand([("on_its_lane", False), ("bound_to_the_run", True)])- def test_a_pass_rearms_a_recurring_check_for_its_next_look(self, _name, bind_run) -> None:- check = self._check(run_interval_minutes=MIN_CHECK_INTERVAL_MINUTES, runs_remaining=2)- if bind_run:- self.scout_run.metadata = {"check_id": str(check.id)}- self.scout_run.save(update_fields=["metadata"])-- result = self._record(check, outcome="passed", explanation="No events since the fix merged.")-- assert result.check_status == SignalReportCheck.Status.ACTIVE- assert result.runs_remaining == 1- with self.assertRaises(InvalidCheckResultError):- self._record(check, outcome="passed", explanation="No events since the fix merged.")- check.refresh_from_db()- assert check.dispatched_at is None- assert check.runs_remaining == 1- assert check.next_run_at > timezone.now() + timedelta(minutes=MIN_CHECK_INTERVAL_MINUTES - 5)-- @parameterized.expand(- [- ("emit_only", "emit", 0),- ("emit_and_edit", "both", 0),- ("edit_only", "edit", 1),- ("no_report_tools", "none", 1),- ("unstamped", None, 0),- ]- )- def test_a_failure_resurfaces_only_when_the_lane_cannot_file_the_follow_up(- self, _name: str, report_channel: str | None, expected_signals: int- ) -> None:- check = self._check()- if report_channel is not None:- self.scout_run.metadata = {"report_channel": report_channel}- self.scout_run.save(update_fields=["metadata"])-- with patch(_EMIT_SIGNAL, new=AsyncMock(return_value=None)) as emit:- with self.captureOnCommitCallbacks(execute=True):- self._record(check)-- assert emit.call_count == expected_signals- if expected_signals:- sent = emit.call_args.kwargs- assert sent["source_type"] == SignalSourceType.CHECK_FAILED- assert sent["extra"]["report_id"] == str(self.report.id)- assert "fired 30 times yesterday" in sent["description"]-@parameterized.expand([("not_due_and_no_run_is_waiting", {"dispatched_at": None}),@@ -2768,66 +1961,38 @@ class TestCheckResultTool(APIBaseTest):@parameterized.expand([- ("due_and_not_dispatched_yet", {"dispatched_at": None, "next_run_at": timezone.now()}, False),- (- "dispatched_to_this_run_on_a_lane_that_resolves_elsewhere_now",- {"config": {"instructions": "x", "skill_name": _OTHER_SKILL}},- True,- ),- ("monitoring_disabled", {"dispatched_at": None, "next_run_at": timezone.now()}, False, False),- (- "monitoring_unavailable",- {"dispatched_at": None, "next_run_at": timezone.now()},- False,- RuntimeError("flag service unavailable"),- ),+ (f"{state}_{binding}", state, flag, bind_run)+ for state, flag in [+ (SignalReport.Status.RESOLVED, True),+ (SignalReport.Status.MONITORING, True),+ (SignalReport.Status.MONITORING, False),+ (SignalReport.Status.MONITORING, RuntimeError("flag unavailable")),+ ]+ for binding, bind_run in [("unbound", False), ("bound", True)]])- def test_a_check_this_run_may_answer_is_recorded(- self, _name, overrides, bind_run, flag_result: bool | Exception | None = None+ def test_reports_without_an_agent_executor_refuse_even_bound_callbacks(+ self, _name: str, report_status: str, flag_result: bool | Exception, bind_run: bool) -> None:- if flag_result is not None:- SignalReport.objects.filter(id=self.report.id).update(status=SignalReport.Status.MONITORING)- check = self._check(**overrides)+ SignalReport.objects.filter(id=self.report.id).update(status=report_status)+ check = self._check()if bind_run:self.scout_run.metadata = {"check_id": str(check.id)}self.scout_run.save(update_fields=["metadata"])-with patch("products.signals.backend.report_content_gates.feature_enabled_or_false",return_value=flag_result if isinstance(flag_result, bool) else False,side_effect=flag_result if isinstance(flag_result, Exception) else None,):- (listed,) = list_report_checks(team=self.team, report_id=str(self.report.id))- assert listed.run_state == ("due" if check.dispatched_at is None else "queued")- result = self._record(check)-- assert result.check_status == SignalReportCheck.Status.FAILED-- @parameterized.expand(- [- ("its_report_is_ready", SignalReport.Status.READY, timedelta(days=30)),- ("its_report_is_suppressed", SignalReport.Status.SUPPRESSED, timedelta(days=30)),- ("its_horizon_passed", SignalReport.Status.RESOLVED, -timedelta(minutes=1)),- ("its_report_is_monitoring", SignalReport.Status.MONITORING, timedelta(days=30)),- ]- )- def test_a_due_check_the_coordinator_would_not_dispatch_is_paused(self, _name, report_status, expires_in) -> None:- SignalReport.objects.filter(id=self.report.id).update(status=report_status)- now = timezone.now()- check = self._check(dispatched_at=None, next_run_at=now - timedelta(hours=1), expires_at=now + expires_in)-- with patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True):(listed,) = list_report_checks(team=self.team, report_id=str(self.report.id))assert listed.run_state == "paused"+ assert listed.waiting_on_run is Falsewith self.assertRaises(InvalidCheckResultError):self._record(check)-check.refresh_from_db()assert check.status == SignalReportCheck.Status.ACTIVE- assert not SignalReportArtefact.objects.filter(- report=self.report, type=SignalReportArtefact.ArtefactType.CHECK_RESULT- ).exists()+ assert check.last_outcome is None+ assert not self.report.artefacts.filter(type=SignalReportArtefact.ArtefactType.CHECK_RESULT).exists()def test_another_projects_check_is_not_reachable(self) -> None:other_team = self.organization.teams.create(name="Other")@@ -2846,33 +2011,6 @@ class TestCheckResultTool(APIBaseTest):with self.assertRaises(InvalidCheckResultError):self._record(other_check)- @parameterized.expand(- [- ("awaiting_data_looks_again", "awaiting_data", SignalReportCheck.Status.ACTIVE),- ("unmeasurable_ends_the_check", "unmeasurable", SignalReportCheck.Status.INCONCLUSIVE),- ]- )- def test_an_inconclusive_verdict_records_its_reason(self, _name: str, reason: str, expected_status: str) -> None:- check = self._check()-- result = self._record(check, outcome="inconclusive", reason=reason, explanation="No traffic since the fix.")-- assert result.check_status == expected_status- check.refresh_from_db()- assert check.last_outcome == SignalReportCheck.Outcome.INCONCLUSIVE- assert check.last_outcome_reason == reason- assert check.consecutive_errors == 0- (listed,) = list_report_checks(team=self.team, report_id=str(self.report.id))- assert listed.last_outcome_reason == reason- api_rows = self.client.get(f"/api/projects/{self.team.id}/signals/reports/{self.report.id}/checks/").json()- assert [(row["status"], row["last_outcome_reason"]) for row in api_rows["results"]] == [- (expected_status, reason)- ]- artefact = SignalReportArtefact.objects.get(- report=self.report, type=SignalReportArtefact.ArtefactType.CHECK_RESULT- )- assert f'"reason":"{reason}"' in artefact.content-@parameterized.expand([("blank_explanation", {"explanation": " "}),@@ -2890,8 +2028,6 @@ class TestCheckResultTool(APIBaseTest):class TestPendingChecks(APIBaseTest):- """A check written before the fix exists: it waits for the report to resolve, then soaks."""-def setUp(self) -> None:super().setUp()self.report = SignalReport.objects.create(team=self.team, status=SignalReport.Status.READY, title="Fix")@@ -2907,57 +2043,42 @@ class TestPendingChecks(APIBaseTest):**overrides,)- def _resolve(self) -> None:- with self.captureOnCommitCallbacks(execute=True):- self.report.status = SignalReport.Status.RESOLVED- self.report.save(update_fields=["status"])-@parameterized.expand([- (f"{kind}_{name}", kind, result, recovered)+ (f"{kind}_{name}", kind, result)for kind in [SignalReportCheck.Kind.METRIC_THRESHOLD, SignalReportCheck.Kind.AGENT]- for name, result, recovered in [- ("enabled", True, False),- ("disabled", False, False),- ("unavailable", RuntimeError("flag unavailable"), False),- ("recovered", RuntimeError("flag unavailable"), True),+ for name, result in [+ ("enabled", True),+ ("disabled", False),+ ("unavailable", RuntimeError("flag unavailable")),]])- def test_monitoring_entry_arms_checks_or_keeps_the_resolved_fallback(- self, _name: str, kind: str, flag_result: bool | Exception, recovered: bool+ def test_monitoring_entry_only_arms_checks_while_enabled(+ self, _name: str, kind: str, flag_result: bool | Exception) -> None:now = timezone.now()implemented_at = now - timedelta(days=3)config = _threshold_config()if kind == SignalReportCheck.Kind.AGENT:- LLMSkill.objects.create(team=self.team, name=FALLBACK_CHECK_SKILL_NAME, is_latest=True, deleted=False)- SignalScoutConfig.objects.create(team=self.team, skill_name=FALLBACK_CHECK_SKILL_NAME)+ _seed_check_lane(self.team)config = {"instructions": "Check whether checkout errors recur."}- reports = [self.report] + [- SignalReport.objects.create(team=self.team, status=SignalReport.Status.READY, title=f"Fix {index}")- for index in range(2)- ]- checks = [- create_check(- report=report,- title="Checkout errors stay low",- kind=kind,- config=config,- attribution=ArtefactAttribution.system(),- soak_minutes=DEFAULT_CHECK_SOAK_HOURS * 60,- )- for report in reports- ]+ check = create_check(+ report=self.report,+ title="Checkout errors stay low",+ kind=kind,+ config=config,+ attribution=ArtefactAttribution.system(),+ soak_minutes=DEFAULT_CHECK_SOAK_HOURS * 60,+ )with time_machine.travel(now, tick=False):with patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True):with self.captureOnCommitCallbacks(execute=False) as callbacks:- for report in reports:- report.save(- update_fields=report.transition_to(- SignalReport.Status.MONITORING, monitoring_started_at=implemented_at- )+ self.report.save(+ update_fields=self.report.transition_to(+ SignalReport.Status.MONITORING, monitoring_started_at=implemented_at)+ )with patch("products.signals.backend.report_content_gates.feature_enabled_or_false",return_value=flag_result if isinstance(flag_result, bool) else False,@@ -2965,46 +2086,38 @@ class TestPendingChecks(APIBaseTest):):for callback in callbacks:callback()- for check in checks:- check.refresh_from_db()- assert check.status == (- SignalReportCheck.Status.ACTIVE if flag_result is True else SignalReportCheck.Status.PENDING- )- coordinator_flag = True if recovered else flag_result- with patch(- "products.signals.backend.report_content_gates.feature_enabled_or_false",- return_value=coordinator_flag if isinstance(coordinator_flag, bool) else False,- side_effect=coordinator_flag if isinstance(coordinator_flag, Exception) else None,- ) as evaluate:- with patch(_MEASURE) as measure:- run_due_report_checks(now=now)- measure.assert_not_called()- assert evaluate.call_count == 1- anchor = implemented_at if coordinator_flag is True else now- expected_due = anchor + timedelta(hours=DEFAULT_CHECK_SOAK_HOURS)- if kind == SignalReportCheck.Kind.METRIC_THRESHOLD:- expected_due = max(expected_due, metric_check_ready_at(_PAGEVIEWS, self.team, anchor))- for check in checks:- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.ACTIVE- assert check.measurement_start_at == (- anchor if kind == SignalReportCheck.Kind.METRIC_THRESHOLD else None- )- assert check.next_run_at == expected_due- assert check.updated_at == now- assert {due.id for due in collect_due_checks(expected_due + timedelta(minutes=1))} == (- set() if coordinator_flag is True else {check.id for check in checks}+ run_due_report_checks(now=now)+ check.refresh_from_db()+ assert check.status == (+ SignalReportCheck.Status.ACTIVE if flag_result is True else SignalReportCheck.Status.PENDING)- if coordinator_flag is True:- with self.captureOnCommitCallbacks(execute=True):- self.report.save(update_fields=self.report.transition_to(SignalReport.Status.RESOLVED))- checks[0].refresh_from_db()- assert checks[0].next_run_at == expected_due- assert collect_due_checks(expected_due + timedelta(minutes=1)) == [checks[0]]-- @parameterized.expand([("successful", 0, False, 24), ("first_error", 0, True, 6), ("second_error", 1, True, 6)])+ assert check.measurement_start_at == (+ implemented_at if flag_result is True and kind == SignalReportCheck.Kind.METRIC_THRESHOLD else None+ )+ assert collect_due_checks(now) == []+ with patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True):+ run_due_report_checks(now=now)+ check.refresh_from_db()+ assert check.status == SignalReportCheck.Status.ACTIVE+ expected_due = implemented_at + timedelta(hours=DEFAULT_CHECK_SOAK_HOURS)+ if kind == SignalReportCheck.Kind.METRIC_THRESHOLD:+ expected_due = max(expected_due, metric_check_ready_at(_PAGEVIEWS, self.team, implemented_at))+ assert check.next_run_at == expected_due+ with self.captureOnCommitCallbacks(execute=True):+ self.report.save(update_fields=self.report.transition_to(SignalReport.Status.RESOLVED))+ assert collect_due_checks(expected_due + timedelta(minutes=1)) == []++ @parameterized.expand(+ [+ ("successful", 0, False, 24, False),+ ("first_error", 0, True, 6, False),+ ("second_error", 1, True, 6, False),+ ("flag_recovery", 0, False, 24, True),+ ("flag_recovery_after_error", 1, True, 6, True),+ ]+ )def test_monitoring_definition_arms_and_restores_its_observation_cadence(- self, _name: str, prior_errors: int, query_error: bool, delay_hours: int+ self, _name: str, prior_errors: int, query_error: bool, delay_hours: int, pause_for_flag: bool) -> None:now = timezone.now()with (@@ -3045,9 +2158,17 @@ class TestPendingChecks(APIBaseTest):):run_due_report_checks()with time_machine.travel(sample_at + timedelta(minutes=5), tick=False):- with self.captureOnCommitCallbacks(execute=True):- self.report.save(update_fields=self.report.transition_to(SignalReport.Status.SUPPRESSED))- self.report.save(update_fields=self.report.transition_to(self.report.restore_target_status()))+ if pause_for_flag:+ with patch(+ "products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=False+ ):+ run_due_report_checks()+ else:+ with self.captureOnCommitCallbacks(execute=True):+ self.report.save(update_fields=self.report.transition_to(SignalReport.Status.SUPPRESSED))+ self.report.save(+ update_fields=self.report.transition_to(self.report.restore_target_status())+ )run_due_report_checks()measure.assert_called_once()check.refresh_from_db()@@ -3179,7 +2300,7 @@ class TestPendingChecks(APIBaseTest):@parameterized.expand([- ("fixed_outside_posthog", SignalReportCheck.Status.ACTIVE),+ ("fixed_outside_posthog", SignalReportCheck.Status.PENDING),("no_action_needed", SignalReportCheck.Status.CANCELLED),])@@ -3233,83 +2354,24 @@ class TestPendingChecks(APIBaseTest):assert check.measurement_start_at is Noneassert collect_due_checks(timezone.now() + timedelta(days=7)) == []- def test_resolving_the_report_waits_for_a_full_query_window_after_the_soak(self) -> None:- check = self._pending()- before = timezone.now()-- self._resolve()-- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.ACTIVE- assert check.measurement_start_at is not None- assert before <= check.measurement_start_at <= timezone.now()- assert check.soak_minutes == DEFAULT_CHECK_SOAK_HOURS * 60- assert check.next_run_at == metric_check_ready_at(_PAGEVIEWS, self.team, check.measurement_start_at)- assert collect_due_checks(check.next_run_at + timedelta(minutes=1)) == [check]-@parameterized.expand([- # Every resolve path ends in the same save, which is why the arming hangs off the model- # rather off each caller: a merged pull request, a person in the inbox, and an MCP state- # write all have to start the same clock.("merged_pull_request", {"_status_from_pr_state": True}),("resolved_by_a_caller", {"_close_pr_on_resolve": True}),("resolved_with_no_marker", {}),])- def test_every_resolve_path_starts_the_clock(self, _name: str, markers: dict) -> None:+ def test_resolving_a_report_does_not_arm_pending_checks(self, _name: str, markers: dict) -> None:check = self._pending()-with self.captureOnCommitCallbacks(execute=True):for marker, value in markers.items():setattr(self.report, marker, value)self.report.status = SignalReport.Status.RESOLVEDself.report.save(update_fields=["status"])-- check.refresh_from_db()- assert check.status == SignalReportCheck.Status.ACTIVE-- def test_an_edit_after_the_resolve_does_not_re_arm_an_answered_check(self) -> None:- check = self._pending()- self._resolve()- check.refresh_from_db()- armed_at = check.next_run_at-- with self.captureOnCommitCallbacks(execute=True):- self.report.title = "Fix, retitled"- self.report.save(update_fields=["title"])-check.refresh_from_db()- assert check.next_run_at == armed_at-- def test_longer_soak_and_agent_timing_are_preserved(self) -> None:- _seed_check_lane(self.team)- metric = create_check(- report=self.report,- title="Short metric",- kind="metric_threshold",- config=_threshold_config(- query=trends_metric_query(series=[{"kind": "EventsNode", "event": "$pageview"}], date_from="-1d")- ),- attribution=ArtefactAttribution.system(),- soak_minutes=3 * 24 * 60,- )- agent = create_check(- report=self.report,- title="Investigate",- kind="agent",- config={"instructions": "Look for the exception."},- attribution=ArtefactAttribution.system(),- soak_minutes=24 * 60,- )- resolved_at = timezone.now()- with time_machine.travel(resolved_at, tick=False):- self._resolve()- metric.refresh_from_db()- agent.refresh_from_db()- assert metric.next_run_at == resolved_at + timedelta(days=3)- assert agent.next_run_at == resolved_at + timedelta(days=1)- assert agent.measurement_start_at is None+ assert check.status == SignalReportCheck.Status.PENDING+ assert check.measurement_start_at is None+ assert collect_due_checks(timezone.now() + timedelta(days=7)) == []@parameterized.expand([@@ -3343,7 +2405,13 @@ class TestPendingChecks(APIBaseTest):invalid = self._pending()valid = self._pending()SignalReportCheck.objects.for_team(self.team.id).filter(id=invalid.id).update(config={})- self._resolve()+ with patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True):+ with self.captureOnCommitCallbacks(execute=True):+ self.report.save(+ update_fields=self.report.transition_to(+ SignalReport.Status.MONITORING, monitoring_started_at=timezone.now()+ )+ )invalid.refresh_from_db()valid.refresh_from_db()assert invalid.status == valid.status == SignalReportCheck.Status.ACTIVE@@ -3545,20 +2613,16 @@ class TestScoutCheckTools(APIBaseTest):assert check.soak_minutes == 3 * 24 * 60assert collect_due_checks(timezone.now() + timedelta(days=7)) == []- def test_the_resolve_starts_the_clock_on_a_check_a_run_dated(self) -> None:+ def test_resolving_does_not_start_a_scout_authored_checks_clock(self) -> None:check = SignalReportCheck.objects.for_team(self.team.id).get(id=self._create().check_id)- before = timezone.now()-with self.captureOnCommitCallbacks(execute=True):self.report.status = SignalReport.Status.RESOLVEDself.report.save(update_fields=["status"])-check.refresh_from_db()- assert check.status == SignalReportCheck.Status.ACTIVE- assert check.measurement_start_at is not None- assert before <= check.measurement_start_at <= timezone.now()+ assert check.status == SignalReportCheck.Status.PENDING+ assert check.measurement_start_at is Noneassert check.soak_minutes == 3 * 24 * 60- assert check.next_run_at == metric_check_ready_at(_PAGEVIEWS, self.team, check.measurement_start_at)+ assert collect_due_checks(timezone.now() + timedelta(days=7)) == []def test_a_recurring_check_keeps_its_initial_soak_after_reopening(self) -> None:self.report.status = SignalReport.Status.RESOLVED@@ -4135,8 +3199,13 @@ class TestReportCheckLifecycleLog(APIBaseTest):attribution=ArtefactAttribution.system(),)- with self.captureOnCommitCallbacks(execute=True):- open_report.save(update_fields=open_report.transition_to(SignalReport.Status.RESOLVED))+ with patch("products.signals.backend.report_content_gates.feature_enabled_or_false", return_value=True):+ with self.captureOnCommitCallbacks(execute=True):+ open_report.save(+ update_fields=open_report.transition_to(+ SignalReport.Status.MONITORING, monitoring_started_at=timezone.now()+ )+ )entries = self._entries(SignalReportArtefact.ArtefactType.CHECK_SCHEDULED, open_report)check.refresh_from_db()diff --git a/products/signals/backend/test/test_scout_harness.py b/products/signals/backend/test/test_scout_harness.pyindex 1fdcb3922e53..1dc35f959c98 100644--- a/products/signals/backend/test/test_scout_harness.py+++ b/products/signals/backend/test/test_scout_harness.py@@ -3465,6 +3465,40 @@ async def test_activity_returns_skip_outcome_when_already_running(ateam):assert output.skip_reason is not None+ ("triggered_by", "check_id"),+ [("manual", "00000000-0000-0000-0000-000000000001"), (TRIGGERED_BY_CHECK, None)],+)+async def test_activity_skips_queued_report_checks_without_starting_scout_work(+ triggered_by: str, check_id: str | None+) -> None:+ with patch(+ "products.signals.backend.scout_harness.runner.arun_signals_scout",+ side_effect=AssertionError("Report checks must not start scout work"),+ ):+ output = await ActivityEnvironment().run(+ run_signals_scout_activity,+ RunSignalsScoutInput(+ team_id=7,+ skill_name="signals-scout-errors",+ skill_version=2,+ triggered_by=triggered_by,+ check_id=check_id,+ ),+ )++ assert output == RunSignalsScoutOutput(+ run_id=None,+ task_run_id=None,+ status=None,+ runtime_s=0.0,+ skill_name="signals-scout-errors",+ skill_version=2,+ skip_reason="report_check_execution_disabled",+ )++@pytest.mark.asyncio@pytest.mark.django_db@pytest.mark.parametrize(diff --git a/products/signals/frontend/inbox/components/detail/ReportCheckRow.stories.tsx b/products/signals/frontend/inbox/components/detail/ReportCheckRow.stories.tsxindex aa36d7ef8094..a0a7de45ac00 100644--- a/products/signals/frontend/inbox/components/detail/ReportCheckRow.stories.tsx+++ b/products/signals/frontend/inbox/components/detail/ReportCheckRow.stories.tsx@@ -2,7 +2,7 @@ import type { Meta, StoryObj } from '@storybook/react'import { IconTarget } from '@posthog/icons'-import type { SignalReportCheckApi } from 'products/signals/frontend/generated/api.schemas'+import type { SignalReportCheckApi, SignalReportStatusEnumApi } from 'products/signals/frontend/generated/api.schemas'import { DetailSection } from './DetailSection'import { buildReportCheckRows, reportChecksMeta } from './reportCheckPresentation'@@ -84,15 +84,21 @@ const explanations = new Map([['inconclusive', 'The crash only shows on a physical device, which no tool here can reach.'],])-function ChecksSection({ checks }: { checks: SignalReportCheckApi[] }): JSX.Element {+function ChecksSection({+ checks,+ reportStatus = 'resolved',+}: {+ checks: SignalReportCheckApi[]+ reportStatus?: SignalReportStatusEnumApi+}): JSX.Element {return (<DetailSectionicon={<IconTarget />}title="Follow-up checks"- meta={<span className="text-xs text-tertiary tabular-nums">{reportChecksMeta(checks)}</span>}+ meta={<span className="text-xs text-tertiary tabular-nums">{reportChecksMeta(checks, reportStatus)}</span>}><div className="flex flex-col gap-1.5">- {buildReportCheckRows(checks, explanations).map((row) => (+ {buildReportCheckRows(checks, explanations, reportStatus).map((row) => (<ReportCheckRow key={row.check.id} row={row} cancelling={false} onCancel={() => undefined} />))}</div>@@ -126,6 +132,7 @@ export const EveryState: Story = {}export const Running: Story = {+ name: 'Resolved with stale dispatch',args: { checks: [check({ id: 'running', dispatched_at: '2026-09-21T09:00:00Z' })] },}@@ -136,6 +143,7 @@ export const NarrowRail: Story = {export const Monitoring: Story = {args: {+ reportStatus: 'monitoring',checks: [check({id: 'monitoring-scheduled',diff --git a/products/signals/frontend/inbox/components/detail/ReportCheckRow.tsx b/products/signals/frontend/inbox/components/detail/ReportCheckRow.tsxindex da3a893e7ff8..63c1f9de633d 100644--- a/products/signals/frontend/inbox/components/detail/ReportCheckRow.tsx+++ b/products/signals/frontend/inbox/components/detail/ReportCheckRow.tsx@@ -20,7 +20,7 @@ export function ReportCheckRow({cancelling: booleanonCancel: (checkId: string) => void}): JSX.Element {- const { check, tag, detail, cancelled, cancellable } = row+ const { check, reportStatus, tag, detail, cancelled, cancellable } = rowreturn (<div@@ -47,7 +47,7 @@ export function ReportCheckRow({</div><div className="flex items-end gap-2 min-w-0 pl-[1.375rem]"><span className="min-w-0 flex-1 text-xs leading-snug text-tertiary">- <ReportCheckTiming check={check} detail={detail} />+ <ReportCheckTiming check={check} reportStatus={reportStatus} detail={detail} /></span>{cancellable && (<LemonButtondiff --git a/products/signals/frontend/inbox/components/detail/ReportCheckTiming.tsx b/products/signals/frontend/inbox/components/detail/ReportCheckTiming.tsxindex caab45ab2a1c..a00bee873bdc 100644--- a/products/signals/frontend/inbox/components/detail/ReportCheckTiming.tsx+++ b/products/signals/frontend/inbox/components/detail/ReportCheckTiming.tsx@@ -1,11 +1,19 @@import { TZLabel } from 'lib/components/TZLabel'-import type { SignalReportCheckApi } from 'products/signals/frontend/generated/api.schemas'+import type { SignalReportCheckApi, SignalReportStatusEnumApi } from 'products/signals/frontend/generated/api.schemas'import { monitoringPeriodLabel } from './reportCheckPresentation'-export function ReportCheckTiming({ check, detail }: { check: SignalReportCheckApi; detail: string }): JSX.Element {- const timing = check.monitoring_timing+export function ReportCheckTiming({+ check,+ reportStatus,+ detail,+}: {+ check: SignalReportCheckApi+ reportStatus: SignalReportStatusEnumApi+ detail: string+}): JSX.Element {+ const timing = reportStatus === 'monitoring' ? check.monitoring_timing : nullreturn (<span className="flex min-w-0 flex-col gap-1"><span>{detail}</span>diff --git a/products/signals/frontend/inbox/components/detail/ReportChecksSection.tsx b/products/signals/frontend/inbox/components/detail/ReportChecksSection.tsxindex eeb20064cc09..c87f8d5cf455 100644--- a/products/signals/frontend/inbox/components/detail/ReportChecksSection.tsx+++ b/products/signals/frontend/inbox/components/detail/ReportChecksSection.tsx@@ -43,7 +43,7 @@ export function ReportChecksSection({ report }: { report: SignalReport }): JSX.Ereturn null}- const rows = buildReportCheckRows(reportChecks, latestCheckExplanations(reportArtefacts ?? []))+ const rows = buildReportCheckRows(reportChecks, latestCheckExplanations(reportArtefacts ?? []), report.status)const { visible, hidden } = splitReportCheckRows(rows)return (@@ -51,7 +51,11 @@ export function ReportChecksSection({ report }: { report: SignalReport }): JSX.Eicon={<IconTarget />}title="Follow-up checks"collapsible- meta={<span className="text-xs text-tertiary tabular-nums">{reportChecksMeta(reportChecks)}</span>}+ meta={+ <span className="text-xs text-tertiary tabular-nums">+ {reportChecksMeta(reportChecks, report.status)}+ </span>+ }><div className="flex flex-col gap-1.5">{(showRetired ? rows : visible).map((row) => (diff --git a/products/signals/frontend/inbox/components/detail/ReportExpectedImpact.test.tsx b/products/signals/frontend/inbox/components/detail/ReportExpectedImpact.test.tsxindex 949da1c18578..77bab3e70f29 100644--- a/products/signals/frontend/inbox/components/detail/ReportExpectedImpact.test.tsx+++ b/products/signals/frontend/inbox/components/detail/ReportExpectedImpact.test.tsx@@ -113,11 +113,31 @@ describe('ReportExpectedImpact', () => {jest.restoreAllMocks()})- function renderMeasurements(checks: SignalReportCheckApi[]): void {+ function renderMeasurements(checks: SignalReportCheckApi[], reportStatus = report.status): void {logic.actions.loadReportChecksSuccess(checks)- render(<ReportExpectedImpact report={report} reportUrl="https://example.com/report-1" />)+ render(+ <ReportExpectedImpact+ report={{ ...report, status: reportStatus }}+ reportUrl="https://example.com/report-1"+ />+ )}+ it.each([+ [SignalReportStatus.READY, true],+ [SignalReportStatus.PENDING_INPUT, true],+ [SignalReportStatus.RESOLVED, false],+ [SignalReportStatus.SUPPRESSED, false],+ ] as const)('shows the measurement label for a %s report', (reportStatus, proposed) => {+ renderMeasurements([check, { ...check, id: 'approved', approved_at: '2026-09-30T00:00:00Z' }], reportStatus)++ expect(+ screen+ .getAllByText(/^(Proposed measurement|Approved measurement|Paused)$/)+ .map((label) => label.textContent)+ ).toEqual(proposed ? ['Proposed measurement', 'Approved measurement'] : ['Paused', 'Paused'])+ })+it.each(['linear', 'fixed'] as const)('keeps the chart and collapses the %s Monitoring definition',async (scaling) => {diff --git a/products/signals/frontend/inbox/components/detail/ReportExpectedImpact.tsx b/products/signals/frontend/inbox/components/detail/ReportExpectedImpact.tsxindex 58dced880dc5..0ab978bc59c6 100644--- a/products/signals/frontend/inbox/components/detail/ReportExpectedImpact.tsx+++ b/products/signals/frontend/inbox/components/detail/ReportExpectedImpact.tsx@@ -24,7 +24,11 @@ export function ReportExpectedImpact({ report, reportUrl }: { report: SignalRepouseValues(logic)const { approveReportCheck, loadReportChecks } = useActions(logic)const { currentProjectId, metricCheckReplacementDisabledReason } = useValues(inboxTaskKickoffLogic)- const measurements = buildReportCheckRows(reportChecks ?? [], latestCheckExplanations(reportArtefacts ?? []))+ const measurements = buildReportCheckRows(+ reportChecks ?? [],+ latestCheckExplanations(reportArtefacts ?? []),+ report.status+ ).filter(({ check }) => check.kind === 'metric_threshold' && check.status !== 'cancelled').flatMap((row) => {const { check } = row@@ -75,8 +79,12 @@ export function ReportExpectedImpact({ report, reportUrl }: { report: SignalRepo</div>)}{measurements.length ? (- measurements.map(({ check, config, metric, goal, detail, tag, cancellable }) => {+ measurements.map(({ check, reportStatus, config, metric, goal, detail, tag, cancellable }) => {const query = asReportMetricSeriesQuery(metric)+ const showApprovalLabel =+ cancellable &&+ !check.monitoring_timing &&+ !['resolved', 'suppressed', 'deleted'].includes(reportStatus)return (<div key={check.id} className="flex flex-col gap-2"><p className="m-0 font-semibold">@@ -86,7 +94,7 @@ export function ReportExpectedImpact({ report, reportUrl }: { report: SignalRepo)}</p><div className="m-0 flex flex-wrap items-center gap-x-1 text-secondary text-sm">- {cancellable && !check.monitoring_timing ? (+ {showApprovalLabel ? (<span>{check.approved_at ? 'Approved measurement' : 'Proposed measurement'}</span>) : (<LemonTag type={tag.type}>{tag.label}</LemonTag>@@ -128,12 +136,11 @@ export function ReportExpectedImpact({ report, reportUrl }: { report: SignalRepotimingDescription={<ReportCheckTimingcheck={check}+ reportStatus={reportStatus}detail={- check.monitoring_timing || !cancellable- ? detail- : check.soak_minutes != null && report.status !== 'resolved'- ? `Check observation window after implementation: ${monitoringPeriodLabel(check.soak_minutes)}.`- : detail+ showApprovalLabel && check.soak_minutes != null+ ? `Check observation window after implementation: ${monitoringPeriodLabel(check.soak_minutes)}.`+ : detail}/>}diff --git a/products/signals/frontend/inbox/components/detail/reportCheckPresentation.test.ts b/products/signals/frontend/inbox/components/detail/reportCheckPresentation.test.tsindex bd0bfb59dc15..a106472eed0d 100644--- a/products/signals/frontend/inbox/components/detail/reportCheckPresentation.test.ts+++ b/products/signals/frontend/inbox/components/detail/reportCheckPresentation.test.ts@@ -1,3 +1,6 @@+import { createElement } from 'react'+import { renderToStaticMarkup } from 'react-dom/server'+import { dayjs } from 'lib/dayjs'import type { SignalReportCheckApi } from 'products/signals/frontend/generated/api.schemas'@@ -13,6 +16,7 @@ import {reportChecksMeta,splitReportCheckRows,} from './reportCheckPresentation'+import { ReportCheckTiming } from './ReportCheckTiming'function makeCheck(overrides: Partial<SignalReportCheckApi> = {}): SignalReportCheckApi {return {@@ -45,7 +49,7 @@ function rowFor(tag: stringdetail: string} {- const [row] = buildReportCheckRows([makeCheck(overrides)], explanations)+ const [row] = buildReportCheckRows([makeCheck(overrides)], explanations, 'monitoring')return { tag: row.tag.label, detail: row.detail }}@@ -61,6 +65,55 @@ describe('reportCheckPresentation', () => {})describe('buildReportCheckRows', () => {+ it('hides cached measurement dates after the report leaves Monitoring', () => {+ const markup = renderToStaticMarkup(+ createElement(ReportCheckTiming, {+ check: makeCheck({+ monitoring_timing: {+ state: 'scheduled',+ started_at: '2026-09-20T09:00:00Z',+ ends_at: '2026-09-27T09:00:00Z',+ next_measurement_at: '2026-09-21T09:00:00Z',+ },+ }),+ reportStatus: 'resolved',+ detail: 'Checks run only while this report is in Monitoring.',+ })+ )+ expect(markup).toContain('Checks run only while this report is in Monitoring.')+ expect(markup).not.toContain('Next measurement:')+ expect(markup).not.toContain('Window end:')+ })++ it.each(['resolved', 'ready', 'suppressed', 'potential'] as const)(+ 'pauses open checks on a %s report without hiding historical verdicts',+ (reportStatus) => {+ const checks = [+ makeCheck({ id: 'pending', status: 'pending' }),+ makeCheck({ id: 'scheduled', kind: 'metric_threshold' }),+ makeCheck({ id: 'dispatched', dispatched_at: '2026-09-21T09:00:00Z' }),+ makeCheck({+ id: 'stale-timing',+ monitoring_timing: {+ state: 'scheduled',+ started_at: null,+ ends_at: null,+ next_measurement_at: '2026-09-27T09:00:00Z',+ },+ }),+ makeCheck({ id: 'passed', status: 'passed', last_run_at: '2026-09-27T09:00:00Z' }),+ ]+ const rows = buildReportCheckRows(checks, new Map([['passed', 'The expectation held.']]), reportStatus)+ for (const row of rows.filter(({ cancellable }) => cancellable)) {+ expect(row.tag).toEqual({ label: 'Paused', type: 'muted' })+ expect(row.detail).toBe('Checks run only while this report is in Monitoring.')+ expect(row.check).toBe(checks.find(({ id }) => id === row.check.id))+ }+ expect(rows.find(({ check }) => check.id === 'passed')?.detail).toBe('Sep 27 · The expectation held.')+ expect(reportChecksMeta(checks, reportStatus)).toBe('5 · paused')+ }+ )+// The section exists so a reader can tell a scheduled check from a waiting one from a check// that never ran. A wrong mapping here puts the wrong verdict word on the row.it.each<[string, Partial<SignalReportCheckApi>, string, string]>([@@ -201,7 +254,8 @@ describe('reportCheckPresentation', () => {makeCheck({ id: 'running', dispatched_at: '2026-09-21T09:00:00Z' }),makeCheck({ id: 'sooner', next_run_at: '2026-09-27T09:00:00Z' }),],- new Map()+ new Map(),+ 'monitoring')expect(rows.map((row) => row.check.id)).toEqual(['running',@@ -218,7 +272,7 @@ describe('reportCheckPresentation', () => {['a check waiting for the resolve', 'pending', true],['a check that already decided', 'passed', false],])('offers Stop on %s', (_name, status, cancellable) => {- const [row] = buildReportCheckRows([makeCheck({ status })], new Map())+ const [row] = buildReportCheckRows([makeCheck({ status })], new Map(), 'monitoring')expect(row.cancellable).toEqual(cancellable)})})@@ -233,7 +287,8 @@ describe('reportCheckPresentation', () => {makeCheck({ id: 'running', dispatched_at: '2026-09-21T09:00:00Z' }),makeCheck({ id: 'scheduled' }),],- new Map()+ new Map(),+ 'monitoring')const { visible, hidden } = splitReportCheckRows(rows)expect(visible.map((row) => row.check.id)).toEqual(['running', 'scheduled', 'cancelled-4', 'cancelled-3'])@@ -274,7 +329,12 @@ describe('reportCheckPresentation', () => {],['says nothing is left to run', [{ status: 'passed' }, { status: 'failed' }], '2 · all done'],])('%s', (_name, overrides, expected) => {- expect(reportChecksMeta(overrides.map((o) => makeCheck(o)))).toEqual(expected)+ expect(+ reportChecksMeta(+ overrides.map((o) => makeCheck(o)),+ 'monitoring'+ )+ ).toEqual(expected)})})diff --git a/products/signals/frontend/inbox/components/detail/reportCheckPresentation.ts b/products/signals/frontend/inbox/components/detail/reportCheckPresentation.tsindex 9b9c6d7a500d..40fa6e6f713e 100644--- a/products/signals/frontend/inbox/components/detail/reportCheckPresentation.ts+++ b/products/signals/frontend/inbox/components/detail/reportCheckPresentation.ts@@ -3,6 +3,7 @@ import type { LemonTagType } from '@posthog/lemon-ui'import { dayjs } from 'lib/dayjs'import type {+ SignalReportStatusEnumApi,SignalReportCheckApi,SignalReportCheckInconclusiveReasonEnumApi,} from 'products/signals/frontend/generated/api.schemas'@@ -25,6 +26,7 @@ const FALLBACK_LANE_LABEL = 'The follow-up scout'export interface ReportCheckRowData {check: SignalReportCheckApi+ reportStatus: SignalReportStatusEnumApitag: { label: string; type: LemonTagType }/** The line under the title: when it runs, who runs it, or what the verdict said. */detail: string@@ -162,7 +164,16 @@ const MONITORING_CHECK_ROWS: Record<},}-function openCheckRow(check: SignalReportCheckApi): Pick<ReportCheckRowData, 'tag' | 'detail'> {+function openCheckRow(+ check: SignalReportCheckApi,+ reportStatus: SignalReportStatusEnumApi+): Pick<ReportCheckRowData, 'tag' | 'detail'> {+ if (reportStatus !== 'monitoring') {+ return {+ tag: { label: 'Paused', type: 'muted' },+ detail: 'Checks run only while this report is in Monitoring.',+ }+ }if (check.monitoring_timing) {return MONITORING_CHECK_ROWS[check.monitoring_timing.state]}@@ -275,7 +286,8 @@ export function latestCheckExplanations(artefacts: SignalReportArtefact[]): Map<export function buildReportCheckRows(checks: SignalReportCheckApi[],- explanations: Map<string, string>+ explanations: Map<string, string>,+ reportStatus: SignalReportStatusEnumApi): ReportCheckRowData[] {return [...checks].sort((a, b) => {@@ -287,7 +299,8 @@ export function buildReportCheckRows(const open = OPEN_STATUSES.includes(check.status)return {check,- ...(open ? openCheckRow(check) : terminalCheckRow(check, explanations.get(check.id))),+ reportStatus,+ ...(open ? openCheckRow(check, reportStatus) : terminalCheckRow(check, explanations.get(check.id))),cancelled: check.status === 'cancelled',cancellable: open,}@@ -316,7 +329,10 @@ export function splitReportCheckRows(rows: ReportCheckRowData[]): {}/** The section header's right-hand summary: how many checks, and then the next thing that happens. */-export function reportChecksMeta(checks: SignalReportCheckApi[]): string {+export function reportChecksMeta(checks: SignalReportCheckApi[], reportStatus: SignalReportStatusEnumApi): string {+ if (reportStatus !== 'monitoring' && checks.some((check) => OPEN_STATUSES.includes(check.status))) {+ return `${checks.length} · paused`+ }if (checks.some((check) => check.monitoring_timing)) {return String(checks.length)}