Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(signals): launch first-report investigations after onboarding (#123464)

Georgiy Tarasovd8d4f101e277
diff --git a/products/signals/ARCHITECTURE.md b/products/signals/ARCHITECTURE.md
index 723885b7aa55..6933994bf51d 100644
--- a/products/signals/ARCHITECTURE.md
+++ b/products/signals/ARCHITECTURE.md
@@ -238,13 +238,15 @@ On re-promotion:
#### First report onboarding and email
-The `signals-scout-first-report` engine performs a bounded investigation of existing product or warehouse data. Its context includes configured products, observed data volumes, and accessible warehouse tables. The skill owns investigation policy and orientation; the shared prompt defers to its procedure. The skill searches across products and validates evidence before authoring at most one report, enforced by a project-scoped idempotency key. Early replay and persistence use that same key, so a saved report is returned before checking changed emission gates or repeating the safety judge.
+Guided and quick activation enqueue a one-time `signals-scout-first-report` investigation immediately after onboarding commits, only for projects in a test arm of `self-driving-banger-scout` and with evidence of product or warehouse data. Dispatch checks scout-source eligibility before preparing context or probing data. Data evidence can be recorded event ingestion in the project or an environment, a positive warehouse sync count, or a successful live product probe. Warehouse eligibility uses recorded sync metadata without querying warehouse tables; the scout validates freshness during its investigation. Empty projects stop before workflow or sandbox startup and show `no_data`. When no other data evidence exists, failed or inconclusive product probes retry and eventually show `incomplete`, without starting an investigation. Outside the rollout and in the control arm, guided activation starts no immediate scout and quick activation retains its general-scout dispatch. The scout receives a bounded project inventory, recent event volumes, native-product data counts, and accessible warehouse table metadata before investigating. It searches across products, including product-analytics anomalies, and authors at most one report through the direct report channel. A project-scoped report idempotency key enforces that limit independently of agent instructions. A report that was visible keeps the investigation successful after resolution or dismissal. The skill owns investigation policy and orientation; the shared prompt defers to its procedure. Early replay and persistence use the same project-scoped key, so a saved report is returned before checking changed emission gates or repeating the safety judge.
-This system scout is excluded from scheduled registration and suggestions and accepts only onboarding dispatches. Its template remains visible in the Skills store like other canonical scout skills. Its structured close-out distinguishes insufficient data, no actionable finding, and incomplete execution. The engine exposes persisted state and task IDs for onboarding consumers; activation and customer progress presentation are integrated separately.
+The first-report scout is excluded from scheduled registration and suggestions. Non-staff users do not see it in the normal scout roster or scout pickers; its configuration, runs, skill, and task remain available through the normal project-scoped APIs. Its template remains visible in the Skills store like other canonical scout skills. Its structured close-out distinguishes insufficient data, no actionable finding, and an incomplete investigation. Onboarding returns persisted state and the scout run, task, and task run IDs for progress consumers. An in-app notification tells the activating user when a run finishes, including non-report outcomes. No-data skips have no run. Missing or failed data probes remain unknown rather than being presented as an empty project.
-The project-group experiment `self-driving-banger-scout` decides whether the first-report scout runs and which model it uses: `control` runs no first-report scout, `test-sol` selects GPT-6.1 Sol on Codex with high reasoning and priority service, and `test-deepseek` selects DeepSeek Flash 1017 Fast on Pi. Both test arms use the same scout skill. The runner refuses a control assignment. Disabled, unknown, and unavailable flag values prevent production first-report runs; explicit eval and private-trial runtime overrides bypass enrollment. Assignment is evaluated once per run, recorded on the run, and attached to lifecycle and report events. Exposure starts before sandbox creation, after dispatch guards, so setup failures remain in the experiment denominator.
+The first-report email can also select a first-report-scout finding that needs a human decision. The onboarding API exposes the notification email only to the enrolled activating user when the email feature and sender configuration are enabled.
-`signals_scout_run_finished` records the structured first-report outcome, whether a report became visible, and its latency from task creation. A persisted visible report still counts when the agent subsequently fails. The experiment's exposure-to-report funnel includes the earlier setup interval; its five-minute delivery metric measures delivery, not report quality. Compare grounding and actionability separately, and join `task_run_id` to gateway usage for model cost; missing usage is unknown, never zero. Turning the experiment flag off prevents future first-report sandbox starts; already-running investigations may finish.
+The project-group experiment `self-driving-banger-scout` decides whether the first-report scout runs and which model it uses: `control` runs no first-report scout, `test-sol` selects GPT-6.1 Sol on Codex with high reasoning and priority service, and `test-deepseek` selects DeepSeek Flash 1017 Fast on Pi. Both test arms use the same scout skill. The runner refuses a control assignment. Disabled, unknown, and unavailable flag values prevent production first-report runs; explicit eval and private-trial runtime overrides bypass enrollment. Enrollment is recorded at onboarding completion and rechecked before dispatch and sandbox startup. Unavailable experiment evaluations during dispatch use the activation job’s bounded retries; an explicitly disabled or invalid assignment skips the investigation. The runtime assignment is recorded on the run and attached to lifecycle and report events. Onboarding job status events also include the first-report variant and pre-run outcome, so no-data skips are measurable without counting them as runs. The onboarding dispatch job captures exposure for every arm after the same guards: scout-source eligibility, fleet and spend limits, an active activating user, and data. The arms therefore share one denominator, and setup failures after dispatch remain in it.
+
+`signals_scout_run_finished` records the structured first-report outcome, whether a report became visible, and its latency from task creation. A persisted visible report still counts when the agent subsequently fails. The experiment's primary metric is whether a project resolves its first report within seven days of exposure, and the secondary metric is the time to that resolution. Both read `signal_report_status_changed` with `status = resolved` and `is_first_report = true`, where `is_first_report` marks the project's earliest visible report from any scout, so the control arm counts the first report its scheduled scouts produce. Compare grounding and actionability separately, and join `task_run_id` to gateway usage for model cost; missing usage is unknown, never zero. Turning the experiment flag off prevents future first-report sandbox starts; already-running investigations may finish.
Pi scouts require the Tasks worker's `SANDBOX_AI_GATEWAY_URL`, `SANDBOX_AI_GATEWAY_MINT_KEY`, and an entry in `SANDBOX_AI_GATEWAY_PRODUCTS` matching the scout, such as `signals_scout:first-report`. Only internally created scout tasks may mint these tokens. Pi provisioning fails when routing, minting, or recording the model pin fails; it never falls back to the legacy gateway. Before launching the experiment, verify the deployed Go gateway serves `posthog/deepseek-ai/deepseek-v4.1-flash-1017-fast` for `signals_scout`, with the Baseten host configured. The existing internal scout budget and paying-project configuration still apply.
diff --git a/products/signals/backend/facade/onboarding.py b/products/signals/backend/facade/onboarding.py
index 50bc43ffac94..0293d687099c 100644
--- a/products/signals/backend/facade/onboarding.py
+++ b/products/signals/backend/facade/onboarding.py
@@ -7,15 +7,25 @@ from django.db import transaction
from django.db.models import F
from django.utils import timezone
+from posthog.cloud_utils import is_cloud
from posthog.event_usage import report_user_action
from posthog.models.team.extensions import get_or_create_team_extension
from posthog.models.team.team import Team
from posthog.models.user import User
+from posthog.transactional_email import is_transactional_email_configured
from products.replay_vision.backend.facade.api import signal_emitting_scanner_ids
+from products.signals.backend.first_report_completion import first_report_completion_notifications_enabled
+from products.signals.backend.first_report_experiment import (
+ FIRST_REPORT_SCOUT_VARIANTS,
+ resolve_first_report_experiment,
+)
from products.signals.backend.first_report_notification import register_first_report_notification
+from products.signals.backend.first_report_scout import first_report_state
+from products.signals.backend.first_report_scout_types import FirstReportStatus
from products.signals.backend.models import (
AutonomyPriority,
+ SignalFirstReportNotification,
SignalScoutConfig,
SignalScoutSuggestionSet,
SignalSourceConfig,
@@ -25,6 +35,7 @@ from products.signals.backend.models import (
from products.signals.backend.onboarding_analytics import InboxOnboardingAnalytics
from products.signals.backend.onboarding_completion import InboxOnboardingReconciler
from products.signals.backend.onboarding_scouts import InboxOnboardingScouts
+from products.signals.backend.report_content_gates import team_first_report_email_enabled
from products.signals.backend.scout_harness.create_access import can_create_scout
from products.signals.backend.scout_harness.team_limits import withheld_skills_for_team
@@ -48,6 +59,10 @@ def onboarding_priorities() -> Sequence[tuple[str, str | Promise]]:
return AutonomyPriority.choices
+def onboarding_first_report_statuses() -> Sequence[tuple[str, str | Promise]]:
+ return FirstReportStatus.choices
+
+
def can_create_onboarding_scout(user: User, team: Team) -> bool:
return can_create_scout(user, team)
@@ -61,7 +76,12 @@ class InboxOnboarding:
def _state(self, config: SignalTeamConfig) -> dict[str, Any]:
state = config.onboarding or {}
- if any((state.get("draft") or {}).get(key) for key in ("suggestions", "activation_scouts")):
+ if (
+ self.team.parent_team_id
+ or state.get("first_report")
+ or state.get("jobs", {}).get("scout")
+ or any((state.get("draft") or {}).get(key) for key in ("suggestions", "activation_scouts"))
+ ):
self.authorize_scouts(False)
jobs = dict(state.get("jobs", {}))
if (
@@ -80,6 +100,27 @@ class InboxOnboarding:
"completed_at": state.get("completed_at"),
"variant": state.get("variant"),
**({"jobs": jobs} if "jobs" in state else {}),
+ "first_report": first_report_state(self.team.parent_team_id or self.team_id)
+ or (
+ {"status": "pending"}
+ if state.get("first_report_variant") in FIRST_REPORT_SCOUT_VARIANTS and jobs.get("scout") == "pending"
+ else None
+ ),
+ "first_report_completion_notification": bool(
+ state.get("first_report_variant") in FIRST_REPORT_SCOUT_VARIANTS
+ and state.get("activated_by") == self.user.pk
+ and first_report_completion_notifications_enabled(self.team.parent_team or self.team, self.user)
+ ),
+ "first_report_notification_email": (
+ self.user.email
+ if is_cloud()
+ and is_transactional_email_configured()
+ and team_first_report_email_enabled(self.team_id)
+ and SignalFirstReportNotification.objects.for_team(self.team_id)
+ .filter(activated_by=self.user, queued_at__isnull=True, cancelled_at__isnull=True)
+ .exists()
+ else None
+ ),
}
def state(self) -> dict[str, Any]:
@@ -125,8 +166,9 @@ class InboxOnboarding:
self._enqueue_activation_jobs(config)
return self._state(config)
- if (config.onboarding or {}).get("variant") == "test-github":
+ if (config.onboarding or {}).get("variant") in ("test-github", "test-ui"):
self.authorize_scouts(True)
+ if (config.onboarding or {}).get("variant") == "test-github":
if not any(
item["kind"] == "canonical" and item["skill_name"] == "signals-scout-general"
for item in draft["suggestions"]
@@ -186,6 +228,11 @@ class InboxOnboarding:
or any(source["source_product"] == "signals_scout" for source in draft["sources"])
)
withheld = withheld_skills_for_team(scout_team_id) if resume_operational_scouts else set()
+ experiment = (
+ resolve_first_report_experiment(self.team.parent_team or self.team)
+ if (config.onboarding or {}).get("variant") in ("test-github", "test-ui")
+ else None
+ )
user = self.user
with transaction.atomic():
config = SignalTeamConfig.objects.select_for_update().get(team_id=self.team_id, pk=config.pk)
@@ -199,8 +246,15 @@ class InboxOnboarding:
"completed_at": timezone.now().isoformat(),
"activated_by": user.pk,
}
+ if state.get("variant") in ("test-github", "test-ui"):
+ config.onboarding["jobs"] = {
+ "scout": "pending" if experiment or state.get("variant") == "test-github" else "skipped",
+ "recommendations": "skipped",
+ }
+ if experiment:
+ config.onboarding["first_report_variant"] = experiment.variant
if state.get("variant") == "test-github":
- config.onboarding["jobs"] = {"scout": "pending", "recommendations": "pending"}
+ config.onboarding["jobs"]["recommendations"] = "pending"
config.save(
update_fields=["onboarding", "autostart_enabled", "default_autostart_priority", "updated_at"]
)
@@ -213,7 +267,7 @@ class InboxOnboarding:
def _enqueue_activation_jobs(self, config: SignalTeamConfig) -> None:
state = config.onboarding or {}
- if state.get("variant") != "test-github":
+ if state.get("variant") not in ("test-github", "test-ui"):
return
self.authorize_scouts(True)
from products.signals.backend.tasks import (
diff --git a/products/signals/backend/first_report_experiment.py b/products/signals/backend/first_report_experiment.py
index 52aa4449478e..f247d1007813 100644
--- a/products/signals/backend/first_report_experiment.py
+++ b/products/signals/backend/first_report_experiment.py
@@ -7,6 +7,7 @@ import posthoganalytics
from posthog.dataclasses import frozen
from posthog.event_usage import groups
from posthog.exceptions_capture import capture_exception
+from posthog.ph_client import ph_scoped_capture
from products.signals.backend.agent_runtime import AgentRuntime
from products.signals.backend.models import SignalReport, SignalScoutRun
@@ -15,6 +16,7 @@ if TYPE_CHECKING:
from posthog.models.team.team import Team
FIRST_REPORT_EXPERIMENT = "self-driving-banger-scout"
+FIRST_REPORT_SCOUT_VARIANTS = frozenset({"test-sol", "test-deepseek"})
@frozen
@@ -38,7 +40,7 @@ class FirstReportExperiment:
return properties
-def resolve_first_report_experiment(team: Team) -> FirstReportExperiment | None:
+def resolve_first_report_experiment(team: Team, *, raise_on_error: bool = False) -> FirstReportExperiment | None:
try:
variant = posthoganalytics.get_feature_flag(
FIRST_REPORT_EXPERIMENT,
@@ -49,7 +51,11 @@ def resolve_first_report_experiment(team: Team) -> FirstReportExperiment | None:
)
except Exception as error:
capture_exception(error)
+ if raise_on_error:
+ raise
return None
+ if variant is None and raise_on_error:
+ raise RuntimeError("First-report experiment evaluation is unavailable")
if variant == "control":
return FirstReportExperiment(variant=variant, agent_runtime=None)
if variant == "test-sol":
@@ -67,19 +73,19 @@ def resolve_first_report_experiment(team: Team) -> FirstReportExperiment | None:
return None
-def capture_first_report_exposure(team: Team, experiment: FirstReportExperiment, run_id: str) -> None:
+def capture_first_report_exposure(team: Team, experiment: FirstReportExperiment) -> None:
try:
- posthoganalytics.capture(
- distinct_id=str(team.uuid),
- event="$experiment_exposure",
- groups=groups(team.organization, team),
- properties={
- **experiment.properties(),
- "$feature_flag": FIRST_REPORT_EXPERIMENT,
- "$feature_flag_response": experiment.variant,
- "run_id": run_id,
- },
- )
+ with ph_scoped_capture() as capture:
+ capture(
+ distinct_id=str(team.uuid),
+ event="$experiment_exposure",
+ groups=groups(team.organization, team),
+ properties={
+ **experiment.properties(),
+ "$feature_flag": FIRST_REPORT_EXPERIMENT,
+ "$feature_flag_response": experiment.variant,
+ },
+ )
except Exception as error:
capture_exception(error)
diff --git a/products/signals/backend/first_report_notification.py b/products/signals/backend/first_report_notification.py
index 3e7c80486cf1..ae798350cdb9 100644
--- a/products/signals/backend/first_report_notification.py
+++ b/products/signals/backend/first_report_notification.py
@@ -60,9 +60,13 @@ def _eligible_recipient(user: User | None, team: Team) -> bool:
def _visible_reports(team_id: int) -> QuerySet[SignalReport]:
return (
SignalReport.objects.filter(
+ Q(status=SignalReport.Status.READY, latest_actionability=ActionabilityChoice.IMMEDIATELY_ACTIONABLE.value)
+ | Q(
+ status=SignalReport.Status.PENDING_INPUT,
+ latest_actionability=ActionabilityChoice.REQUIRES_HUMAN_INPUT.value,
+ scout_idempotency_key=f"first-report:{team_id}",
+ ),
team_id=team_id,
- status=SignalReport.Status.READY,
- latest_actionability=ActionabilityChoice.IMMEDIATELY_ACTIONABLE.value,
latest_already_addressed=False,
)
.exclude(title="")
diff --git a/products/signals/backend/first_report_scout.py b/products/signals/backend/first_report_scout.py
index b48c31ec6b3f..2a1d082f838e 100644
--- a/products/signals/backend/first_report_scout.py
+++ b/products/signals/backend/first_report_scout.py
@@ -208,9 +208,10 @@ def first_report_state(team_id: int) -> dict[str, object]:
"started_at": run.task_run.created_at.isoformat(),
}
reports = SignalReport.objects.filter(
+ Q(first_visible_at__isnull=False)
+ | Q(status__in=[SignalReport.Status.READY, SignalReport.Status.PENDING_INPUT]),
team_id=team_id,
id__in=run.emitted_report_ids or [],
- status__in=[SignalReport.Status.READY, SignalReport.Status.PENDING_INPUT],
)
report = reports.order_by("first_visible_at").first()
if report is not None:
diff --git a/products/signals/backend/onboarding_activation.py b/products/signals/backend/onboarding_activation.py
index de060332636e..7952459e84c1 100644
--- a/products/signals/backend/onboarding_activation.py
+++ b/products/signals/backend/onboarding_activation.py
@@ -1,5 +1,7 @@
from __future__ import annotations
+from typing import Any
+
from django.db import transaction
from django.utils import timezone
@@ -9,8 +11,21 @@ from posthog.models.user import User
from posthog.ph_client import ph_scoped_capture
from posthog.temporal.common.client import sync_connect
+from products.signals.backend.constants import FIRST_REPORT_SCOUT
+from products.signals.backend.first_report_experiment import (
+ FIRST_REPORT_SCOUT_VARIANTS,
+ capture_first_report_exposure,
+ resolve_first_report_experiment,
+)
+from products.signals.backend.first_report_scout import (
+ build_first_report_context,
+ first_report_state,
+ set_first_report_state,
+)
+from products.signals.backend.first_report_scout_types import FirstReportStatus
from products.signals.backend.models import SignalScoutConfig, SignalTeamConfig
from products.signals.backend.onboarding_analytics import InboxOnboardingAnalytics
+from products.signals.backend.onboarding_completion import InboxOnboardingReconciler
from products.signals.backend.scout_harness.run_gates import check_fleet_gates, check_run_in_flight, check_spend_gates
from products.signals.backend.scout_harness.suggestions import read_suggestion_settings, suggestions_allowed_for_team
from products.signals.backend.temporal.agentic.scout_scheduler import start_onboarding_signals_scout_run
@@ -33,6 +48,11 @@ class InboxOnboardingActivation:
"job": job,
"status": status,
"reason": reason,
+ "experiment_variant": (config.onboarding or {}).get("first_report_variant"),
+ "first_report_outcome": first_report_state(config.team.parent_team_id or self.team_id).get("status")
+ if job == "scout"
+ and (config.onboarding or {}).get("first_report_variant") in FIRST_REPORT_SCOUT_VARIANTS
+ else None,
},
)
@@ -46,13 +66,70 @@ class InboxOnboardingActivation:
config.save(update_fields=["onboarding", "updated_at"])
transaction.on_commit(lambda: self._capture_status(config, job, status, reason), robust=True)
- def _start_scout(self, team: Team) -> bool:
+ def _start_first_report(self, team: Team, state: dict[str, Any], *, final_attempt: bool) -> bool:
+ experiment = resolve_first_report_experiment(team, raise_on_error=True)
+ # Every arm passes the same gates before exposure, so the arms share one denominator.
+ # A control project then gets the activation it would have had without the experiment.
+ runs_scout = experiment is not None and experiment.runs_scout
+
+ def stop(status: FirstReportStatus) -> bool:
+ if state.get("first_report_variant") in FIRST_REPORT_SCOUT_VARIANTS:
+ set_first_report_state(team.id, status)
+ return False if runs_scout else self._start_default_scout(team, state)
+
+ if experiment is None:
+ return stop(FirstReportStatus.INCOMPLETE)
+ if (
+ not InboxOnboardingReconciler.scouts_can_emit(team.id)
+ or check_fleet_gates(team.id)
+ or check_spend_gates(team)
+ ):
+ return stop(FirstReportStatus.INCOMPLETE)
+ if runs_scout and check_run_in_flight(team.id, FIRST_REPORT_SCOUT):
+ return True
+ acting_user_id = state.get("activated_by")
+ user = (
+ team.all_users_with_access().filter(pk=acting_user_id, is_active=True).first() if acting_user_id else None
+ )
+ if user is None:
+ return stop(FirstReportStatus.INCOMPLETE)
+ context = build_first_report_context(team, user)
+ if not context.has_data():
+ if not context.data_is_unknown():
+ return stop(FirstReportStatus.NO_DATA)
+ if not final_attempt:
+ raise RuntimeError("Project data availability is unknown")
+ return stop(FirstReportStatus.INCOMPLETE)
+ capture_first_report_exposure(team, experiment)
+ if not runs_scout:
+ return self._start_default_scout(team, state)
+ scout, _ = SignalScoutConfig.objects.for_team(team.id).get_or_create(
+ team_id=team.id,
+ skill_name=FIRST_REPORT_SCOUT,
+ defaults={
+ "enabled": False,
+ "emit": True,
+ "created_by": user,
+ "enabled_by": user,
+ "write_scopes": [],
+ },
+ )
+ set_first_report_state(team.id, FirstReportStatus.PENDING, activation_team_id=self.team_id)
+ start_onboarding_signals_scout_run(
+ sync_connect(), team_id=team.id, skill_name=scout.skill_name, run_note=context.prompt_context()
+ )
+ return True
+
+ def _start_default_scout(self, team: Team, state: dict[str, Any]) -> bool:
+ return state.get("variant") == "test-github" and self._start_general_scout(team)
+
+ def _start_general_scout(self, team: Team) -> bool:
scout = (
SignalScoutConfig.objects.for_team(team.id)
.filter(skill_name="signals-scout-general", enabled=True, emit=True)
.first()
)
- if scout is None or check_fleet_gates(team.id) or check_spend_gates(team):
+ if not scout or check_fleet_gates(team.id) or check_spend_gates(team):
return False
if check_run_in_flight(team.id, scout.skill_name):
return True
@@ -77,21 +154,28 @@ class InboxOnboardingActivation:
team_id=self.team_id
)
state = config.onboarding or {}
- if state.get("variant") != "test-github" or not state.get("completed_at"):
+ if state.get("variant") not in ("test-github", "test-ui") or not state.get("completed_at"):
return
if state.get("jobs", {}).get(job) in ("started", "skipped"):
return
team = config.team.parent_team or config.team
+ enrolled = job == "scout" and bool(state.get("first_report_variant"))
+ first_report = state.get("first_report_variant") in FIRST_REPORT_SCOUT_VARIANTS
if team.organization.is_ai_data_processing_approved is not True:
+ if enrolled and first_report:
+ set_first_report_state(team.id, FirstReportStatus.INCOMPLETE)
self._record_status(job, "skipped", "ai_consent_required")
return
try:
- started = (
- self._start_scout(team)
- if job == "scout"
- else self._start_recommendations(team, state.get("activated_by"))
- )
+ if enrolled:
+ started = self._start_first_report(team, state, final_attempt=final_attempt)
+ elif job == "scout":
+ started = self._start_default_scout(team, state)
+ else:
+ started = self._start_recommendations(team, state.get("activated_by"))
except Exception:
+ if enrolled and first_report and final_attempt:
+ set_first_report_state(team.id, FirstReportStatus.INCOMPLETE)
self._record_status(job, "failed" if final_attempt else "pending", "dispatch_error")
raise
self._record_status(job, "started" if started else "skipped", None if started else "not_eligible")
diff --git a/products/signals/backend/presentation/onboarding.py b/products/signals/backend/presentation/onboarding.py
index dc4feb2b87a6..c4dd188ce01e 100644
--- a/products/signals/backend/presentation/onboarding.py
+++ b/products/signals/backend/presentation/onboarding.py
@@ -24,6 +24,7 @@ from products.signals.backend.facade.onboarding import (
InboxOnboarding,
InboxOnboardingValidationError,
can_create_onboarding_scout,
+ onboarding_first_report_statuses,
onboarding_priorities,
onboarding_source_products,
onboarding_source_types,
@@ -99,15 +100,51 @@ class InboxOnboardingJobStatus(models.TextChoices):
class InboxOnboardingJobsSerializer(serializers.Serializer):
scout = serializers.ChoiceField(
- choices=InboxOnboardingJobStatus.choices, help_text="Dispatch status of the first general scout run."
+ choices=InboxOnboardingJobStatus.choices, help_text="Dispatch status of the first-report scout run."
)
recommendations = serializers.ChoiceField(
choices=InboxOnboardingJobStatus.choices, help_text="Dispatch status of personalized scout recommendations."
)
+@extend_schema_serializer(many=False)
+class FirstReportStateSerializer(serializers.Serializer):
+ scout_run_id = serializers.UUIDField(
+ required=False, help_text="Scout run ID for inspecting the investigation through the scout API."
+ )
+ task_id = serializers.UUIDField(
+ required=False, help_text="Task ID for following the investigation's live activity."
+ )
+ task_run_id = serializers.UUIDField(
+ required=False, help_text="Task run ID for following the investigation's live activity."
+ )
+ started_at = serializers.DateTimeField(required=False, help_text="When the investigation's task run started.")
+ status = serializers.ChoiceField(
+ choices=onboarding_first_report_statuses(), help_text="Progress of the first investigation."
+ )
+ report_id = serializers.UUIDField(
+ required=False, help_text="The first useful report produced by the investigation."
+ )
+ checked_products = serializers.ListField(
+ child=serializers.CharField(), required=False, help_text="Products checked during the investigation."
+ )
+ missing_data = serializers.ListField(
+ child=serializers.CharField(), required=False, help_text="Data or history needed for a reliable finding."
+ )
+
+
@extend_schema_serializer(many=False)
class InboxOnboardingStateSerializer(serializers.Serializer):
+ first_report_completion_notification = serializers.BooleanField(
+ required=False,
+ help_text="Whether the current user can receive an in-app notification when this investigation finishes.",
+ )
+ first_report_notification_email = serializers.EmailField(
+ required=False,
+ allow_null=True,
+ help_text="The current user's email when enrolled for a first-report notification.",
+ )
+ first_report = FirstReportStateSerializer(required=False, allow_null=True, help_text="First investigation outcome.")
jobs = InboxOnboardingJobsSerializer(required=False, help_text="Background work started by quick activation.")
draft = InboxOnboardingDraftSerializer(allow_null=True, help_text="Saved setup choices, without activating them.")
completed_at = serializers.DateTimeField(allow_null=True, help_text="When verified setup completed.")
diff --git a/products/signals/backend/receivers.py b/products/signals/backend/receivers.py
index 5e99020e785c..e7db86aa24f5 100644
--- a/products/signals/backend/receivers.py
+++ b/products/signals/backend/receivers.py
@@ -921,6 +921,17 @@ def _status_change_properties(
}
+def _is_first_visible_report(team_id: int, report_id: str) -> bool:
+ # The id breaks a tie between reports that became visible in the same instant.
+ first = (
+ SignalReport.objects.filter(team_id=team_id, first_visible_at__isnull=False)
+ .order_by("first_visible_at", "id")
+ .values_list("id", flat=True)
+ .first()
+ )
+ return first is not None and str(first) == report_id
+
+
@receiver(post_save, sender=SignalReport)
def capture_status_change_analytics(
sender: type[SignalReport],
@@ -994,6 +1005,7 @@ def capture_status_change_analytics(
properties={
**properties,
"previous_status": previous_status,
+ "is_first_report": _is_first_visible_report(instance.team_id, report_id),
**_actor_properties(actor, actor_user),
**_classification_snapshot(
report_id,
diff --git a/products/signals/backend/scout_harness/runner.py b/products/signals/backend/scout_harness/runner.py
index e0df3ef6eff5..326b60e04e0a 100644
--- a/products/signals/backend/scout_harness/runner.py
+++ b/products/signals/backend/scout_harness/runner.py
@@ -35,11 +35,7 @@ from products.mcp_store.backend.facade.api import get_sandbox_mcp_server_names
from products.signals.backend.agent_runtime import STEP_SCOUT, AgentRuntime, resolve_agent_runtime
from products.signals.backend.constants import FIRST_REPORT_SCOUT, FIRST_REPORT_SCOUT_TIMEOUT_SECONDS
from products.signals.backend.first_report_completion import complete_first_report_investigation
-from products.signals.backend.first_report_experiment import (
- FirstReportExperiment,
- capture_first_report_exposure,
- resolve_first_report_experiment,
-)
+from products.signals.backend.first_report_experiment import FirstReportExperiment, resolve_first_report_experiment
from products.signals.backend.first_report_scout import set_first_report_state
from products.signals.backend.first_report_scout_types import FirstReportScoutResult
from products.signals.backend.models import SignalScoutConfig, SignalScoutRun
@@ -487,10 +483,6 @@ async def _arun_signals_scout(
)
agent_runtime = experiment.agent_runtime
runtime = agent_runtime.runtime if agent_runtime is not None else "acp"
- if experiment is not None:
- await database_sync_to_async(capture_first_report_exposure, thread_sensitive=False)(
- team, experiment, str(run_id)
- )
if agent_runtime is not None:
runtime_adapter: str | None = agent_runtime.runtime_adapter
model: str | None = agent_runtime.model
diff --git a/products/signals/backend/tasks.py b/products/signals/backend/tasks.py
index ded251bc71b9..240174d8b03a 100644
--- a/products/signals/backend/tasks.py
+++ b/products/signals/backend/tasks.py
@@ -12,6 +12,7 @@ from celery import Task, shared_task
from celery.exceptions import SoftTimeLimitExceeded
from slack_sdk.errors import SlackApiError
+from posthog.celery_queues import CeleryQueue
from posthog.clickhouse.client.connection import ClickHouseUser
from posthog.cloud_utils import get_cached_instance_license
from posthog.egress.github.transport import GitHubEgressBudgetExhausted, GitHubRateLimitError
@@ -86,7 +87,14 @@ from products.tasks.backend.facade.repo_activity import RepositoryCommitActivity
logger = structlog.get_logger(__name__)
-@shared_task(bind=True, ignore_result=True, autoretry_for=(Exception,), retry_backoff=True, max_retries=5)
+@shared_task(
+ bind=True,
+ ignore_result=True,
+ queue=CeleryQueue.LONG_RUNNING.value,
+ autoretry_for=(Exception,),
+ retry_backoff=True,
+ max_retries=5,
+)
@with_team_scope()
def launch_inbox_onboarding_job(self: Task, team_id: int, job: str) -> None:
from products.signals.backend.onboarding_activation import (
diff --git a/products/signals/backend/test/test_first_report_notification.py b/products/signals/backend/test/test_first_report_notification.py
index 28c67a78758d..61bf6ff4cd37 100644
--- a/products/signals/backend/test/test_first_report_notification.py
+++ b/products/signals/backend/test/test_first_report_notification.py
@@ -86,6 +86,16 @@ class TestFirstReportNotification(BaseTest):
def notification(self) -> SignalFirstReportNotification:
return SignalFirstReportNotification.objects.for_team(self.team.id).get()
+ def test_first_investigation_can_notify_about_a_report_needing_a_decision(self) -> None:
+ report = self.report(
+ status=SignalReport.Status.PENDING_INPUT,
+ latest_actionability="requires_human_input",
+ scout_idempotency_key=f"first-report:{self.team.id}",
+ )
+ deliver_first_report_email(self.team.id)
+ assert self.notification().report_id == report.id
+ self.send.assert_called_once()
+
@parameterized.expand(
[
(
diff --git a/products/signals/backend/test/test_first_report_scout.py b/products/signals/backend/test/test_first_report_scout.py
index 02643e5d5086..d7b14da58d1d 100644
--- a/products/signals/backend/test/test_first_report_scout.py
+++ b/products/signals/backend/test/test_first_report_scout.py
@@ -228,10 +228,17 @@ class TestFirstReportScout(BaseTest):
("report_created", "ready", "report_created"),
("report_created", "pending_input", "report_created"),
("report_created", "ready", "report_created", True),
+ ("report_created", "resolved", "report_created", False, True),
+ ("report_created", "suppressed", "report_created", False, True),
]
)
def test_onboarding_success_requires_a_visible_persisted_report(
- self, outcome: str, report_status: str | None, expected: str, child_environment: bool = False
+ self,
+ outcome: str,
+ report_status: str | None,
+ expected: str,
+ child_environment: bool = False,
+ previously_visible: bool = False,
) -> None:
activator = User.objects.create_and_join(self.organization, "[email protected]", None)
activation_team = (
@@ -245,7 +252,15 @@ class TestFirstReportScout(BaseTest):
set_first_report_state(self.team.id, "running")
task = Task.objects.create(team=self.team, title="First investigation", created_by=self.user)
task_run = TaskRun.objects.create(team=self.team, task=task, status="completed")
- report = SignalReport.objects.create(team=self.team, status=report_status) if report_status else None
+ report = (
+ SignalReport.objects.create(
+ team=self.team,
+ status=report_status,
+ first_visible_at=timezone.now() if previously_visible else None,
+ )
+ if report_status
+ else None
+ )
run = SignalScoutRun.objects.create(
team=self.team,
task_run=task_run,
diff --git a/products/signals/backend/test/test_inbox_onboarding.py b/products/signals/backend/test/test_inbox_onboarding.py
index 762b914f91b9..d10ea654bec9 100644
--- a/products/signals/backend/test/test_inbox_onboarding.py
+++ b/products/signals/backend/test/test_inbox_onboarding.py
@@ -203,9 +203,19 @@ class TestInboxOnboarding(APIBaseTest):
assert not (SignalTeamConfig.objects.get(team=self.team).onboarding or {}).get("completed_at")
@parameterized.expand(
- [(None, False), ("control", False), ("test-ui", False), ("test-github", False), ("test-github", True)]
+ [
+ (None, False, None),
+ ("control", False, None),
+ ("test-ui", False, None),
+ ("test-ui", False, "test-sol"),
+ ("test-ui", False, "control"),
+ ("test-github", False, None),
+ ("test-github", True, "test-sol"),
+ ]
)
- def test_completion_is_durable_and_retry_safe(self, variant: str | None, automatic_implementation: bool) -> None:
+ def test_completion_is_durable_and_retry_safe(
+ self, variant: str | None, automatic_implementation: bool, arm: str | None
+ ) -> None:
self.draft["automatic_implementation"] = automatic_implementation
if variant == "test-github":
self.draft["suggestions"] = [self.suggestion]
@@ -224,6 +234,10 @@ class TestInboxOnboarding(APIBaseTest):
return_value=not automatic_implementation,
),
patch("products.signals.backend.tasks.launch_inbox_onboarding_job.delay") as launch,
+ patch(
+ "products.signals.backend.first_report_experiment.posthoganalytics.get_feature_flag",
+ return_value=arm or False,
+ ),
self.captureOnCommitCallbacks(execute=True),
):
first = self.client.post(f"{self.url}complete/", self.draft, format="json")
@@ -238,8 +252,14 @@ class TestInboxOnboarding(APIBaseTest):
(self.team.id, "scout"),
(self.team.id, "recommendations"),
}
+ elif variant == "test-ui" and arm:
+ assert config.onboarding["jobs"] == {"scout": "pending", "recommendations": "skipped"}
+ assert {call.args for call in launch.call_args_list} == {(self.team.id, "scout")}
else:
launch.assert_not_called()
+ if variant in ("test-ui", "test-github"):
+ assert config.onboarding.get("first_report_variant") == arm
+ assert bool(first.json().get("first_report")) is (arm == "test-sol")
assert config.autostart_enabled is automatic_implementation
assert config.default_autostart_priority == "P1"
assert config.onboarding["activated_by"] == self.user.id
@@ -325,13 +345,15 @@ class TestInboxOnboarding(APIBaseTest):
(True, "scout", 200),
(False, "source", 403),
(True, "source", 200),
- (False, "none", 200),
+ (False, "none", 403),
+ (True, "none", 200),
]
)
def test_child_completion_checks_parent_scout_access(
self, parent_access: bool, selection: str, expected_status: int
) -> None:
child = Team.objects.create(organization=self.organization, parent_team=self.team, name="Child")
+ SignalTeamConfig.objects.create(team=child, onboarding={"variant": "test-ui"})
SignalSourceConfig.objects.create(
team=child, source_product="error_tracking", source_type="issue_created", enabled=True
)
@@ -448,7 +470,7 @@ class TestInboxOnboarding(APIBaseTest):
scout.refresh_from_db()
assert scout.enabled
- @parameterized.expand([("expose",), ("reconcile",)])
+ @parameterized.expand([("expose",), ("reconcile",), ("read",)])
def test_child_scoped_credential_cannot_start_parent_reconciliation(self, action: str) -> None:
child = Team.objects.create(organization=self.organization, parent_team=self.team, name="Child")
raw = generate_random_token_personal()
@@ -456,15 +478,19 @@ class TestInboxOnboarding(APIBaseTest):
label="onboarding",
user=self.user,
secure_value=hash_key_value(raw),
- scopes=["task:write"],
+ scopes=["task:write", "signal_scout:read"],
scoped_teams=[child.id],
)
self.client.logout()
- response = self.client.post(
- f"/api/projects/{child.id}/signals/onboarding/{action}/",
- {"variant": "control"},
- format="json",
- HTTP_AUTHORIZATION=f"Bearer {raw}",
+ response = (
+ self.client.get(f"/api/projects/{child.id}/signals/onboarding/", HTTP_AUTHORIZATION=f"Bearer {raw}")
+ if action == "read"
+ else self.client.post(
+ f"/api/projects/{child.id}/signals/onboarding/{action}/",
+ {"variant": "control"},
+ format="json",
+ HTTP_AUTHORIZATION=f"Bearer {raw}",
+ )
)
assert response.status_code == 403, response.content
@@ -624,14 +650,100 @@ class TestInboxOnboarding(APIBaseTest):
class TestInboxOnboardingActivation(BaseTest):
- @parameterized.expand([("scout", 0), ("scout", 5), ("recommendations", 0), ("recommendations", 5)])
- def test_dispatch_retries_use_one_workflow_and_preserve_completed_jobs(self, job: str, retry_count: int) -> None:
+ @parameterized.expand([(False, True), (True, True), (False, False), (False, True, False)])
+ def test_scout_dispatch_requires_data_and_eligibility(
+ self, first_probe_fails: bool, enrolled: bool, source_enabled: bool = True
+ ) -> None:
+ self.organization.is_ai_data_processing_approved = True
+ self.organization.save()
+ if not source_enabled:
+ SignalSourceConfig.objects.create(
+ team=self.team, source_product="signals_scout", source_type="cross_source_issue", enabled=False
+ )
+ config = SignalTeamConfig.objects.create(
+ team=self.team,
+ onboarding={
+ "variant": "test-ui",
+ "first_report_variant": "test-sol",
+ "completed_at": timezone.now().isoformat(),
+ "activated_by": self.user.pk,
+ "jobs": {"scout": "pending"},
+ },
+ )
+ with (
+ patch("products.signals.backend.onboarding_activation.ph_scoped_capture") as capture,
+ self.captureOnCommitCallbacks(execute=True),
+ patch(
+ "products.signals.backend.scout_harness.profile.builders.execute_hogql_query",
+ return_value=MagicMock(results=[["$pageview", 100, 10, 20, 5, timezone.now(), timezone.now()]]),
+ ),
+ patch(
+ "products.signals.backend.first_report_scout.execute_hogql_query",
+ side_effect=lambda **kwargs: MagicMock(results=[[0, None]] if len(kwargs["query"].select) == 2 else []),
+ ) as probe,
+ patch("products.signals.backend.onboarding_activation.sync_connect") as connect,
+ patch(
+ "products.signals.backend.first_report_experiment.posthoganalytics.get_feature_flag",
+ return_value="test-sol" if enrolled else False,
+ ),
+ patch(
+ "posthoganalytics.get_feature_flag_payload",
+ return_value={"enabled": True, "guaranteed_team_ids": [self.team.id]},
+ ),
+ ):
+ if first_probe_fails:
+ healthy_probe = probe.side_effect
+ probe.side_effect = RuntimeError("Query unavailable")
+ with self.assertRaisesRegex(RuntimeError, "data availability is unknown"):
+ InboxOnboardingActivation(self.team.id).run("scout", final_attempt=False)
+ config.refresh_from_db()
+ assert config.onboarding is not None
+ assert config.onboarding["jobs"]["scout"] == "pending"
+ probe.side_effect = healthy_probe
+ InboxOnboardingActivation(self.team.id).run("scout")
+ connect.assert_not_called()
+ config.refresh_from_db()
+ assert config.onboarding is not None
+ assert config.onboarding["first_report"]["status"] == (
+ "no_data" if enrolled and source_enabled else "incomplete"
+ )
+ assert (
+ capture.return_value.__enter__.return_value.call_args.kwargs["properties"]["first_report_outcome"]
+ == config.onboarding["first_report"]["status"]
+ )
+ if not enrolled or not source_enabled:
+ probe.assert_not_called()
+ assert (
+ not SignalScoutConfig.objects.for_team(self.team.id)
+ .filter(skill_name="signals-scout-first-report")
+ .exists()
+ )
+
+ @parameterized.expand(
+ [
+ ("scout", 0, None),
+ ("scout", 5, None),
+ ("scout", 0, "test-sol"),
+ ("scout", 5, "test-sol"),
+ ("scout", 0, "control"),
+ ("scout", 0, "test-sol", "flag_error"),
+ ("scout", 5, "test-sol", "flag_error"),
+ ("scout", 0, "test-sol", "flag_missing"),
+ ("scout", 5, "test-sol", "flag_missing"),
+ ("recommendations", 0, "test-sol"),
+ ("recommendations", 5, "test-sol"),
+ ]
+ )
+ def test_dispatch_retries_use_one_workflow_and_preserve_completed_jobs(
+ self, job: str, retry_count: int, arm: str | None, failure_source: str = "workflow"
+ ) -> None:
self.organization.is_ai_data_processing_approved = True
self.organization.save()
config = SignalTeamConfig.objects.create(
team=self.team,
onboarding={
"variant": "test-github",
+ "first_report_variant": arm,
"completed_at": timezone.now().isoformat(),
"activated_by": self.user.pk,
"jobs": {"scout": "pending", "recommendations": "pending"},
@@ -642,6 +754,19 @@ class TestInboxOnboardingActivation(BaseTest):
client.start_workflow = AsyncMock()
with (
patch("products.signals.backend.onboarding_activation.sync_connect", return_value=client),
+ patch(
+ "products.signals.backend.first_report_experiment.posthoganalytics.get_feature_flag",
+ return_value=arm or False,
+ ) as flag,
+ patch("products.signals.backend.first_report_experiment.ph_scoped_capture") as exposure,
+ patch(
+ "products.signals.backend.scout_harness.profile.builders.execute_hogql_query",
+ return_value=MagicMock(results=[["$pageview", 100, 10, 20, 5, timezone.now(), timezone.now()]]),
+ ),
+ patch(
+ "products.signals.backend.first_report_scout.execute_hogql_query",
+ return_value=MagicMock(results=[[100, timezone.now()]]),
+ ),
patch("products.signals.backend.onboarding_activation.ph_scoped_capture") as capture,
patch("products.signals.backend.onboarding_analytics.posthoganalytics.feature_enabled", return_value=True),
self.captureOnCommitCallbacks(execute=True),
@@ -651,7 +776,12 @@ class TestInboxOnboardingActivation(BaseTest):
return_value={"enabled": True, "guaranteed_team_ids": [self.team.id]},
),
):
- client.start_workflow.side_effect = RuntimeError("Temporarily unavailable")
+ if failure_source == "flag_error":
+ flag.side_effect = RuntimeError("Flag evaluation unavailable")
+ elif failure_source == "flag_missing":
+ flag.return_value = None
+ else:
+ client.start_workflow.side_effect = RuntimeError("Temporarily unavailable")
launch_inbox_onboarding_job.push_request(retries=retry_count)
try:
with patch.object(launch_inbox_onboarding_job, "retry", side_effect=Retry()), self.assertRaises(Retry):
@@ -661,6 +791,14 @@ class TestInboxOnboardingActivation(BaseTest):
config.refresh_from_db()
assert config.onboarding is not None
assert config.onboarding["jobs"][job] == ("failed" if retry_count == 5 else "pending")
+ if job == "scout" and arm == "test-sol" and failure_source == "workflow":
+ assert config.onboarding["first_report_activation_team_id"] == self.team.id
+ if failure_source != "workflow":
+ client.start_workflow.assert_not_called()
+ if retry_count == 5:
+ assert config.onboarding["first_report"]["status"] == "incomplete"
+ flag.side_effect = None
+ flag.return_value = arm or False
client.start_workflow.reset_mock(side_effect=True)
activation = InboxOnboardingActivation(self.team.id)
activation.run(job)
@@ -679,6 +817,16 @@ class TestInboxOnboardingActivation(BaseTest):
assert config.onboarding is not None
assert config.onboarding["jobs"][job] == "started"
assert client.start_workflow.call_args.kwargs["id"] == dispatch["id"]
+ if job == "scout":
+ started_skill = client.start_workflow.call_args.args[1].skill_name
+ assert started_skill == ("signals-scout-first-report" if arm == "test-sol" else "signals-scout-general")
+ exposures = exposure.return_value.__enter__.return_value.call_args_list
+ assert {event.kwargs["properties"]["$feature_flag_response"] for event in exposures} == (
+ {arm} if arm else set()
+ )
+ assert SignalScoutConfig.objects.for_team(self.team.id).filter(
+ skill_name="signals-scout-first-report"
+ ).exists() is (job == "scout" and arm == "test-sol")
events = capture.return_value.__enter__.return_value.call_args_list
assert [event.kwargs["properties"]["status"] for event in events] == (
["failed", "started", "started"] if retry_count == 5 else ["started", "started"]
diff --git a/products/signals/backend/test/test_scout_harness.py b/products/signals/backend/test/test_scout_harness.py
index df7cbcc7e693..02381ac90e86 100644
--- a/products/signals/backend/test/test_scout_harness.py
+++ b/products/signals/backend/test/test_scout_harness.py
@@ -2888,7 +2888,6 @@ async def test_first_report_experiment_routes_and_records_assignment(ateam, vari
assert run.metadata["experiment_variant"] == variant
assert run.metadata["runtime"] == runtime
events = {call.kwargs["event"]: call.kwargs["properties"] for call in capture.call_args_list}
- assert events["$experiment_exposure"]["$feature_flag_response"] == variant
assert events["signals_scout_run_started"]["experiment_variant"] == variant
assert events["signals_scout_run_finished"]["experiment_variant"] == variant
assert events["signals_scout_run_finished"]["first_report_outcome"] == "no_actionable_finding"
diff --git a/products/signals/backend/test/test_status_change_telemetry.py b/products/signals/backend/test/test_status_change_telemetry.py
index 637f1117dc4a..4553357dca03 100644
--- a/products/signals/backend/test/test_status_change_telemetry.py
+++ b/products/signals/backend/test/test_status_change_telemetry.py
@@ -55,6 +55,19 @@ class TestCaptureStatusChangeAnalytics(BaseTest):
assert kwargs["properties"].get("resolution_reason") is None
assert kwargs["properties"].get("dismissal_cause") is None
+ @parameterized.expand([("earliest_visible", 0, True), ("later_visible", 1, False), ("never_visible", 2, False)])
+ def test_label_marks_the_projects_first_visible_report(self, _name, index, expected):
+ now = timezone.now()
+ reports = [self._create_report() for _ in range(3)]
+ SignalReport.objects.filter(id=reports[0].id).update(first_visible_at=now - timedelta(hours=2))
+ SignalReport.objects.filter(id=reports[1].id).update(first_visible_at=now - timedelta(hours=1))
+ SignalReport.objects.filter(id=reports[2].id).update(first_visible_at=None)
+ report = SignalReport.objects.get(id=reports[index].id)
+ with patch("products.signals.backend.receivers.posthoganalytics.capture") as mock_capture:
+ with self.captureOnCommitCallbacks(execute=True):
+ report.save(update_fields=report.transition_to(SignalReport.Status.RESOLVED))
+ assert mock_capture.call_args.kwargs["properties"]["is_first_report"] is expected
+
@parameterized.expand(
[
("pipeline_without_actor", None, "system", False, None, None),
diff --git a/products/signals/frontend/generated/api.schemas.ts b/products/signals/frontend/generated/api.schemas.ts
index 95a341e50e2f..d719bcf05439 100644
--- a/products/signals/frontend/generated/api.schemas.ts
+++ b/products/signals/frontend/generated/api.schemas.ts
@@ -774,6 +774,54 @@ export interface InboxSummaryApi {
people: InboxSummaryPersonApi[] | null
}
+/**
+ * * `pending` - Pending
+ * * `running` - Running
+ * * `no_data` - No data
+ * * `report_created` - Report created
+ * * `insufficient_data` - Insufficient data
+ * * `no_actionable_finding` - No actionable finding
+ * * `incomplete` - Incomplete
+ */
+export type FirstReportStatusEnumApi = (typeof FirstReportStatusEnumApi)[keyof typeof FirstReportStatusEnumApi]
+
+export const FirstReportStatusEnumApi = {
+ Pending: 'pending',
+ Running: 'running',
+ NoData: 'no_data',
+ ReportCreated: 'report_created',
+ InsufficientData: 'insufficient_data',
+ NoActionableFinding: 'no_actionable_finding',
+ Incomplete: 'incomplete',
+} as const
+
+export interface FirstReportStateApi {
+ /** Scout run ID for inspecting the investigation through the scout API. */
+ scout_run_id?: string
+ /** Task ID for following the investigation's live activity. */
+ task_id?: string
+ /** Task run ID for following the investigation's live activity. */
+ task_run_id?: string
+ /** When the investigation's task run started. */
+ started_at?: string
+ /** Progress of the first investigation.
+ *
+ * * `pending` - Pending
+ * * `running` - Running
+ * * `no_data` - No data
+ * * `report_created` - Report created
+ * * `insufficient_data` - Insufficient data
+ * * `no_actionable_finding` - No actionable finding
+ * * `incomplete` - Incomplete */
+ status: FirstReportStatusEnumApi
+ /** The first useful report produced by the investigation. */
+ report_id?: string
+ /** Products checked during the investigation. */
+ checked_products?: string[]
+ /** Data or history needed for a reliable finding. */
+ missing_data?: string[]
+}
+
/**
* * `pending` - Pending
* * `started` - Started
@@ -791,7 +839,7 @@ export const InboxOnboardingJobStatusEnumApi = {
} as const
export interface InboxOnboardingJobsApi {
- /** Dispatch status of the first general scout run.
+ /** Dispatch status of the first-report scout run.
*
* * `pending` - Pending
* * `started` - Started
@@ -1363,6 +1411,15 @@ export const InboxOnboardingVariantEnumApi = {
} as const
export interface InboxOnboardingStateApi {
+ /** Whether the current user can receive an in-app notification when this investigation finishes. */
+ first_report_completion_notification?: boolean
+ /**
+ * The current user's email when enrolled for a first-report notification.
+ * @nullable
+ */
+ first_report_notification_email?: string | null
+ /** First investigation outcome. */
+ first_report?: FirstReportStateApi | null
/** Background work started by quick activation. */
jobs?: InboxOnboardingJobsApi
/** Saved setup choices, without activating them. */
diff --git a/services/mcp/src/api/generated.ts b/services/mcp/src/api/generated.ts
index 83c588c14018..507fa1732ce9 100644
--- a/services/mcp/src/api/generated.ts
+++ b/services/mcp/src/api/generated.ts
@@ -55951,6 +55951,55 @@ export namespace Schemas {
created_at: string | null;
}
+ /**
+ * * `pending` - Pending
+ * * `running` - Running
+ * * `no_data` - No data
+ * * `report_created` - Report created
+ * * `insufficient_data` - Insufficient data
+ * * `no_actionable_finding` - No actionable finding
+ * * `incomplete` - Incomplete
+ */
+ export type FirstReportStatusEnum = typeof FirstReportStatusEnum[keyof typeof FirstReportStatusEnum];
+
+
+ export const FirstReportStatusEnum = {
+ Pending: 'pending',
+ Running: 'running',
+ NoData: 'no_data',
+ ReportCreated: 'report_created',
+ InsufficientData: 'insufficient_data',
+ NoActionableFinding: 'no_actionable_finding',
+ Incomplete: 'incomplete',
+ } as const;
+
+ export interface FirstReportState {
+ /** Scout run ID for inspecting the investigation through the scout API. */
+ scout_run_id?: string;
+ /** Task ID for following the investigation's live activity. */
+ task_id?: string;
+ /** Task run ID for following the investigation's live activity. */
+ task_run_id?: string;
+ /** When the investigation's task run started. */
+ started_at?: string;
+ /** Progress of the first investigation.
+ *
+ * * `pending` - Pending
+ * * `running` - Running
+ * * `no_data` - No data
+ * * `report_created` - Report created
+ * * `insufficient_data` - Insufficient data
+ * * `no_actionable_finding` - No actionable finding
+ * * `incomplete` - Incomplete */
+ status: FirstReportStatusEnum;
+ /** The first useful report produced by the investigation. */
+ report_id?: string;
+ /** Products checked during the investigation. */
+ checked_products?: string[];
+ /** Data or history needed for a reliable finding. */
+ missing_data?: string[];
+ }
+
export type FlagEvaluationsModeEnum = typeof FlagEvaluationsModeEnum[keyof typeof FlagEvaluationsModeEnum];
@@ -63875,7 +63924,7 @@ export namespace Schemas {
} as const;
export interface InboxOnboardingJobs {
- /** Dispatch status of the first general scout run.
+ /** Dispatch status of the first-report scout run.
*
* * `pending` - Pending
* * `started` - Started
@@ -63892,6 +63941,15 @@ export namespace Schemas {
}
export interface InboxOnboardingState {
+ /** Whether the current user can receive an in-app notification when this investigation finishes. */
+ first_report_completion_notification?: boolean;
+ /**
+ * The current user's email when enrolled for a first-report notification.
+ * @nullable
+ */
+ first_report_notification_email?: string | null;
+ /** First investigation outcome. */
+ first_report?: FirstReportState | null;
/** Background work started by quick activation. */
jobs?: InboxOnboardingJobs;
/** Saved setup choices, without activating them. */