:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Signals System Architecture
Overview
The Signals product is a signal grouping and report-generation pipeline. Signals from multiple products and integrations — including session replay, AI observability, error tracking, GitHub, Linear, Zendesk, and the headless Signals agent itself (a scheduled scout that emits cross-source findings into the same pipeline) — are emitted into a shared ClickHouse embeddings table, grouped into SignalReports via embedding similarity + LLM matching, and then optionally promoted into an agentic report-research flow.
Today the active ingestion path is emitter → buffer → grouping v2. The summary path is no longer a simple "summarize signals" LLM step: it runs a report-level safety judge, selects a repository, then performs sandbox-backed multi-turn research that produces findings, actionability, priority, title, summary, and suggested reviewers. Reports that are immediately actionable can automatically start a Tasks coding run via the autonomy system.
A report is a living document: every piece of work done on it — judgments, repo selection, research findings, pushed commits, task runs, free-form notes — is recorded as an attributed, schema-validated, append-only SignalReportArtefact. The status-bearing types resolve latest-wins; the rest accumulate as a work log. This is the surface autonomous agents (the pipeline, custom scouts, and other inbox-driven products) read and write as they act on a report.
Temporal Workflows
Signals ingestion uses a three-stage pipeline: emitter → buffer → grouping v2. The emitter and buffer workflows are defined in backend/temporal/emitter.py and backend/temporal/buffer.py. The grouping v2 workflow is in backend/temporal/grouping_v2.py and delegates to the shared _process_signal_batch() implementation in backend/temporal/grouping.py. The report summary workflow is defined in backend/temporal/summary.py.
The original TeamSignalGroupingWorkflow (v1) in backend/temporal/grouping.py is still registered but is no longer started by emit_signal(). Its shared activities and _process_signal_batch() implementation are still actively used by v2.
Signals workflows and activities are registered in backend/temporal/__init__.py and wired into the VIDEO_EXPORT_TASK_QUEUE worker (the temporal-worker-video-export fleet) by posthog/management/commands/start_temporal_worker.py.
The inbox ranking scoring sweep is the one exception. SELF_DRIVING_WORKFLOWS and SELF_DRIVING_ACTIVITIES register it only on SELF_DRIVING_TASK_QUEUE, which the temporal-worker-self-driving fleet polls. See the self-driving Temporal worker runbook for the fleet, its schedule, and its checks.
If you add or remove a Signals workflow/activity from backend/temporal/__init__.py, you also need to update posthog/temporal/tests/ai/test_module_integrity.py (TestSignalsProductModuleIntegrity). That test intentionally snapshots the registered workflow/activity lists and will fail until its expected names are updated.
Several additional Signals workflows also exist but are not part of the main report pipeline:
backfill-error-tracking(backend/temporal/backfill_error_tracking.py) — backfills recent error tracking issues as signalsemit-eval-signal(backend/temporal/emit_eval_signal.py) - compatibility-only workflow retained for existing Temporal histories; its activity no longer emits signalsrun-signals-scout-coordinator(backend/temporal/agentic/scout_coordinator.py) — periodic tick (everyCOORDINATOR_INTERVAL_MINUTES = 30) that fans out scheduled runs for every enabledSignalScoutConfig, per (team, skill). Spec'd separately below.RunSignalsScoutWorkflow(backend/temporal/agentic/scout_scheduler.py) — child workflow per planned run; thin wrapper around the harness activity. Spec'd separately below.
Activity decoration
Every async Signals Temporal activity is decorated with @scoped_temporal() from posthog/temporal/common/scoped.py (not upstream @posthoganalytics.scoped()). It scopes posthoganalytics.tag() calls to the activity invocation and auto-captures uncaught exceptions into PostHog error tracking with the workflow's tags attached. The upstream decorator wraps async def in a sync wrapper, breaking Temporal's iscoroutinefunction dispatch — the worker returns the unawaited coroutine and crashes on JSON encoding. scoped_temporal() is the async-aware equivalent (sync helpers can keep using upstream @posthoganalytics.scoped()).
from posthog.temporal.common.scoped import scoped_temporal
@temporalio.activity.defn
@scoped_temporal()
async def my_activity(input: ...) -> ...:
...
Signal Ingestion Pipeline (v2)
The v1 TeamSignalGroupingWorkflow buffered raw EmitSignalInputs in memory and carried them over on continue_as_new. Under higher signal volume the continue_as_new payload could grow too large. The v2 pipeline fixes that by flushing buffered signals to object storage and passing only lightweight object keys between workflows.
emit_signal() SignalEmitterWorkflow BufferSignalsWorkflow TeamSignalGroupingV2Workflow
│ │ │ │
├─ start (idempotent) ───────────────────────────────────────────────────►│ │
│ │ │ │
├─ start (fire-and-forget) ──────────►│ │ │
│ │ │ │
│ activity: query buffer size │ │
│ ├─ get_buffer_size ───────────────►│ │
│ │◄─ size ─────────────────────────┤│ │
│ │ (poll+sleep if full) │ │
│ activity: signal submit_signal │ │
│ ├─────────────────────────────────►│ │
│ │ │ │
│ │ (buffer fills / timeout) │
│ │ │ │
│ │ activity: safety filter + flush │
│ │ ├──► object storage: signals/signal_batches/<uuid>
│ │ │ │
│ │ activity: signal-with-start │
│ │ ├─ submit_batch(object_key) ──────►│
│ │ │ │
│ │ continue_as_new activity: read from object storage
│ │ │ ├──► object storage
│ │ │ │
│ │ │ _process_signal_batch()
│ │ │ │
│ │ │ continue_as_new
SignalEmitterWorkflow (signal-emitter)
Ephemeral per-signal workflow that provides backpressure between emit_signal() and the buffer. One instance per signal, with workflow ID signal-emitter-{team_id}-{uuid}.
Defined in backend/temporal/emitter.py.
Flow:
- Run
submit_signal_to_buffer_activity, which: a. Queries the buffer workflow’sget_buffer_sizequery b. If the buffer is full (>= BUFFER_MAX_SIZE), polls with jittered sleep until space is available, heartbeating while it waits c. Signals the buffer workflow’ssubmit_signalhandler with the signal
This keeps emit_signal() fire-and-forget while the emitter workflow absorbs backpressure. The activity currently uses a 3-hour start_to_close_timeout and 10-minute heartbeat_timeout.
BufferSignalsWorkflow (buffer-signals)
Buffers incoming signals in memory and periodically flushes them to object storage. One instance per team, with workflow ID buffer-signals-{team_id}.
Defined in backend/temporal/buffer.py.
Architecture:
- New signals arrive via
@workflow.signal(submit_signal), sent bySignalEmitterWorkflowinstances. - Exposes
@workflow.query(get_buffer_size) so emitters can implement backpressure by polling buffer occupancy before sending. - The main loop waits for signals, then waits until either the buffer reaches
BUFFER_MAX_SIZE(20) orBUFFER_FLUSH_TIMEOUT_SECONDS(5s) elapses since the first signal arrived. - On flush: drains the buffer, runs the safety filter on all signals in parallel via
safety_filter_activity, writes the safe signals to object storage atsignals/signal_batches/<uuid>viaflush_signals_to_s3_activity, then sends the object key to the grouping v2 workflow viasignal_with_start_grouping_v2_activity(which starts the workflow if needed). If the entire batch is unsafe, the flush and grouping steps are skipped. - If the buffer is already full again after flushing (signals arrived during the flush activities), it loops immediately and flushes again rather than paying a
continue_as_newrestart cost. - Otherwise it calls
continue_as_new, carrying over any signals that arrived between drain and now viaBufferSignalsInput.pending_signals. - Stored batch objects are cleaned up by object-storage lifecycle policies, not by the workflows.
TeamSignalGroupingV2Workflow (team-signal-grouping-v2)
Long-running entity workflow that processes batches of signals from object storage. One instance per team, with workflow ID team-signal-grouping-v2-{team_id}.
Defined in backend/temporal/grouping_v2.py.
Architecture:
- Receives object keys via
@workflow.signal(submit_batch), sent byBufferSignalsWorkflow. - Buffers pending object keys in memory as
list[str], which is much lighter than buffering fullEmitSignalInputs. - The main loop waits for a batch key, downloads the signals via
read_signals_from_s3_activity, then processes the full batch via_process_signal_batch()fromgrouping.py. - Caches signal type examples across batches with a TTL (
TYPE_EXAMPLES_CACHE_TTL, 5 minutes). - Calls
continue_as_newafter each batch, carrying over any pending keys that arrived during processing. - Supports pause/unpause via
set_paused_until,clear_paused, andget_paused_state. The API exposes this throughSignalProcessingViewSet. - Errors processing a batch are caught and logged — the workflow continues to the next batch.
TeamSignalGroupingWorkflow (team-signal-grouping) — v1, legacy
A long-running entity workflow that serializes all signal grouping for a single team. Exactly one instance per team, with workflow ID team-signal-grouping-{team_id}. It is still registered, but emit_signal() no longer starts it.
What is still active:
- The workflow class itself is legacy.
- The shared
_process_signal_batch()implementation in this file is still the active grouping implementation used by v2.
Signal processing flow (per batch, in _process_signal_batch()):
- Embed all signal descriptions + fetch signal type examples from ClickHouse in parallel
- Generate 1–3 search queries per signal via LLM, using type examples for cross-source context
- Embed each query
- Semantic search the ClickHouse
document_embeddingsHogQL alias for nearest neighbors viacosineDistance() - LLM match — decide whether the signal belongs to an existing report or needs a new one
- Assign the signal to a
SignalReportin Postgres, increment counts/weights, check promotion threshold, and emit the signal into the embeddings pipeline in one atomic operation - Wait for ClickHouse — poll until the just-emitted signals are query-visible so subsequent grouping decisions can find them
- If promoted (weight ≥ threshold), spawn child
SignalReportSummaryWorkflowwithParentClosePolicy.ABANDON;WorkflowAlreadyStartedErroris ignored
Steps 1–4 run in parallel across the batch. Steps 5–7 run sequentially per signal, and earlier-in-batch matches are injected into later signals’ candidate sets before the LLM match call.
SignalReportSummaryWorkflow (signal-report-summary)
Runs when a report is promoted to candidate. The current flow is safety judge → repository selection → agentic research → state transition.
Defined in backend/temporal/summary.py.
Flow:
- Quota gate →
check_report_quota_gate_activity(stage=summary_entry; see Billing limit enforcement below). When it blocks, the workflow exits and the report stayscandidate. The gate re-runs before repository selection (pre_repo_selection) and before agentic research (pre_research) — a block there reverts the reportin_progress → candidateviarevert_report_to_candidate_activitybefore exiting, so a promotion rule can pick it up again. All three sites are gated withworkflow.patched("self-driving-quota-gates"). - Fetch signals for the report from ClickHouse →
fetch_signals_for_report_activityAn empty result is retried up toEMPTY_FETCH_RETRY_ATTEMPTStimes atEMPTY_FETCH_RETRY_INTERVAL, since freshly emitted signals can trail the run into ClickHouse. If it stays empty,report_has_assigned_signals_activitychecks Postgressignal_count: signals assigned but not yet visible → the workflow exits and the report stayscandidatefor re-promotion; none assigned →failedwithfailure_reason=no_signals_found. Gated withworkflow.patched("signals-empty-fetch-retry"). - Mark in-progress in Postgres and advance
signals_at_runby 3 →mark_report_in_progress_activity - Safety judge →
report_safety_judge_activity- Evaluates the underlying signals for prompt injection / manipulation attempts
- Persists a
safety_judgmentartefact
- Select repository →
select_repository_activity(temporal/agentic/select_repository.py) — see Repository Selection below- If no repository is selected, the workflow synthesizes a “repository selection required” title/summary and transitions to
pending_input
- If no repository is selected, the workflow synthesizes a “repository selection required” title/summary and transitions to
- Agentic research →
run_agentic_report_activity(temporal/agentic/report.py)- Sandbox-backed multi-turn research over the selected repository plus MCP data
- Appends
repo_selection,signal_finding,actionability_judgment,priority_judgment, andsuggested_reviewersartefacts atomically on success (append-only — on re-research only entries that actually changed are appended)
- Conditional coding-task auto-start (inside
run_agentic_report_activity) — see Autonomy & Auto-Start below - Apply the decision:
- If not actionable →
reset_report_to_potential_activity(weight resets to 0, status becomespotential) - If requires human input →
mark_report_pending_input_activity - If immediately actionable →
mark_report_ready_activity
- If not actionable →
- If new signals arrived during the run,
mark_report_ready_activityre-promotes the report tocandidateand the workflow loops internally rather than spawning a new workflow - When a run completes without immediately looping, the workflow publishes a Kafka
signals report completedmessage viapublish_report_completed_activity
On any unhandled exception, the workflow marks the report failed and re-raises.
Timeout / retry details:
- The grouping workflow starts the summary child with a 1-hour
execution_timeout, not 5 hours - The heavy step is the agentic research activity, which uses a 4-hour
start_to_close_timeoutand 5-minute heartbeat timeout - Most activities use 3-attempt retry policies
- Repo selection and agentic research use single-attempt retries to avoid duplicate sandbox work
Repository Selection
Before running the agentic research, the workflow calls select_repository_activity (temporal/agentic/select_repository.py) — a separate activity that selects the most relevant repository from the team’s GitHub integrations via select_repository_for_report() (report_generation/select_repo.py).
Keeping repository selection in its own activity gives it independent retry / timeout behavior and makes it reusable across re-promotions.
- Re-promoted report: loads and reuses the latest
repo_selectionartefact if it contains a repository - 0 repos connected: returns
None→ workflow transitions topending_input - 1 repo connected: returns it directly
- N repos connected: runs a sandbox repo-discovery agent using
PostHog/.githubas a lightweight dummy clone; the agent usesghCLI to inspect candidate repositories and choose the best match - The activity runs in a sandbox environment restricted to GitHub-related domains
Repository heavy cache
IntegrationRepositoryCacheEntry (Postgres, defined in posthog/models/integration_repository_cache.py) stores per-repo README + recursive blob paths + descriptive metadata, populated lazily by GitHubRepositoryFullCache.sync_full_cache_entry(). It's exposed to the selection agent as the HogQL system table system.integration_repository_cache so the agent can grep paths server-side via ARRAY JOIN splitByString('\n', tree_paths) instead of hitting GitHub's /search/code endpoint (30 req/min hard ceiling). The lightweight (id, name, full_name) list stays on Integration.repository_cache (JSONField) so the IDE repo-dropdown read path is unchanged.
sync_full_cache_entry() uses a two-tier freshness check, both keyed off default_branch_sha as the hydration sentinel (so repos legitimately without a README still hit the fast paths):
- TTL gate — fresh row → return immediately, zero API calls.
- SHA gate — past TTL, two cheap calls to compare the live default-branch SHA against the cached one. Match → refresh only mutable metadata, skip README and tree refetch.
- Heavy refetch — SHA changed → fetch README and the recursive file tree pinned to the same commit, then upsert.
Bulk sync (sync_full_cache) fans the per-repo sync out via run_parallel_with_backoff (concurrency 10) over the JSONField list as source of truth — orphan rows are evicted, per-repo errors are returned in-place rather than raised. Secondary rate limits propagate with retry hints so the helper backs off cooperatively. The bulk sync is single-flighted per integration via a Redis lock: concurrent reports for the same team queue behind the leader (poll every 1s, hard cap 20m wait) and then read the warm cache; the leader heartbeats every 60s to extend its 15m lease, and a lost lease cancels the in-flight body to prevent duplicate syncs.
Eligibility filter: Before invoking the agent, select_repository_for_report drops candidates that are archived or missing from the heavy cache (e.g., a row whose sync errored during a cold start). The prompt treats SQL hits as primary evidence, so a missing row would read as a false negative. If filtering leaves zero or one candidates, the activity short-circuits without running the agent.
Past corrections: select_repository_for_team — the chokepoint every signals-side selection funnels through (report pipeline, custom agents, scout emit_report) — builds a "past selection corrections" block from the team's recent wrong_repo dismissals (repo_corrections.wrong_repo_corrections_block: newest first, one entry per report and per distinct selected→corrected lesson, capped at 20 within 180 days) and passes it to the tasks-side select_repository, which renders it into the prompt between the candidate list and the cache instructions. Caller-side injection rather than agent self-lookup is deliberate: the agent's SQL surface stays limited to the cache table, and an injected block is guaranteed in front of the agent on every run. Building the block is best-effort — on failure selection proceeds without it.
Truncation caveat: GitHub's recursive tree endpoint truncates at ~50k entries / 7MB. The tree_truncated flag marks affected rows; tree_paths is incomplete on those rows and will silently miss files in HogQL grep. Consumers must filter on tree_truncated=False and explicitly degrade for truncated repos. Affects <2% of repos; paginated subtree fetch is future work.
Re-promotion
Reports are re-promoted when new evidence arrives. A READY report re-promotes when a new matching signal carries it to its next research bucket, and a report reset to potential re-promotes once it clears the signals_at_run snooze gate again.
Research buckets. The research activity reads every non-deleted signal, so re-research cost scales with report size, and a report that re-researched on every signal spent most of its runs re-reading a report that had barely moved. Instead, a report researches at the cumulative signal counts in RESEARCH_SIGNAL_BUCKETS (SIGNAL_RESEARCH_SIGNAL_BUCKETS, default 1,2,4,10), which by their number also cap it at four research passes. Between buckets, and past the last one, new signals are still assigned, weighted, and emitted, but no new summary run spawns.
next_research_bucket(signals_researched) is the single predicate, enforced in two places: the grouping promotion gate (assign_and_emit_signal_activity), which fires signal_report_reresearch_skipped per withheld signal with a skip_reason of below_next_bucket or buckets_exhausted so the saved volume is trackable, and the summary self-loop (mark_report_ready_activity), which decides whether signals that landed mid-run are worth another pass.
SignalReport.signals_researched is the count the report's last completed pass covered, written when a run reaches READY. It is null until then, and researched_signal_count reconstructs it from the signals_at_run stamp for reports researched before the column existed, so the backlog carries no backfill. Reading it makes the predicate skip buckets the report is already past, so a report whose first pass ran at 5 signals waits for 10 rather than firing passes 2 and 3 back to back on the same evidence. Two properties follow from measuring completed work rather than attempts. A run that pauses on a quota gate before researching leaves the value untouched, so it costs the report nothing. And promotion tests the count the report has reached against its next bucket rather than the crossing itself, so a bucket reached while the quota or daily-report gate suppressed promotion is still claimable on the next signal instead of being skipped for the report's life.
Buckets gate only the READY re-research path. Re-promotions through the potential gate stay uncapped — first research, candidate self-heal, snooze return, and a not-actionable reset re-accumulating weight — because they are weight / signals_at_run-gated rather than per-signal, so strong new evidence can still resurface a large report.
On re-promotion:
- Repo selection reuses the previous
repo_selectionartefact when possible - Agentic research reconstructs previous findings / actionability / priority from artefacts and reuses prior work signal-by-signal when still valid
- Agentic artefacts are append-only — the previous run's rows are kept; the new run appends only the entries that actually changed (the agent confirms a still-correct finding/judgment instead of regenerating it), and status types resolve latest-wins
task_runartefacts are never removed on re-promotion; they are the historical record of research and auto-started coding runs- Auto-start is deduplicated per report by a legacy
SignalReportTaskimplementation link (alongside the protectedtask_runlog), checked inside the report-rowselect_for_update - Workflow ID includes
run_counton reruns to avoid Temporal ID collisions with earlier executions
First report onboarding and email
Guided and quick activation enqueue a one-time signals-scout-first-report investigation immediately after onboarding commits, only for projects in a test arm of self-driving-banger-scout and with evidence of product or warehouse data. Dispatch checks scout-source eligibility before preparing context or probing data. Data evidence can be recorded event ingestion in the project or an environment, a positive warehouse sync count, or a successful live product probe. Warehouse eligibility uses recorded sync metadata without querying warehouse tables; the scout validates freshness during its investigation. Empty projects stop before workflow or sandbox startup and show no_data. When no other data evidence exists, failed or inconclusive product probes retry and eventually show incomplete, without starting an investigation. Outside the first-report rollout and in its control arm, guided activation starts no immediate scout and quick activation dispatches the general scout. The scout receives the products with completed onboarding (products_in_use, not proof of data usage) and a compact list of PostHog sources in investigation priority order: error tracking; logs, traces, and metrics; product analytics and dashboards; AI observability; then session recordings. The list distinguishes observed data from configured products and historical ingestion, and mentions accessible warehouse tables only when present, without enumerating them. Missing sources remain unverified rather than empty. The scout uses activity logs to find relevant analyses and combines sources to validate promising leads and establish impact, inspecting recordings only when a targeted sample can answer a concrete question. It authors at most one report through the direct report channel. A project-scoped report idempotency key enforces that limit independently of agent instructions. A report that was visible keeps the investigation successful after resolution or dismissal. The skill owns investigation policy and orientation; the shared prompt defers to its procedure. Early replay and persistence use the same project-scoped key, so a saved report is returned before checking changed emission gates or repeating the safety judge.
The first-report scout is excluded from scheduled registration and suggestions. Non-staff users do not see it in the normal scout roster or scout pickers; its configuration, runs, skill, and task remain available through the normal project-scoped APIs. Its template remains visible in the Skills store like other canonical scout skills. Onboarding returns the scout run, task, and task run IDs, and embeds the existing live run viewer so customers can follow the investigation in the inbox. The inbox folds the setup card to its header while a first-report state exists and shows one first-report card beneath it only after the selected report lists finish loading and the current filters return no reports: status and next steps in the left pane, the actual investigation in the right pane. A created report removes the card and refreshes the report list, so the first report appears as a normal row. A failed refresh shows a retry action even when the previous list was empty; retry reloads the failed sections and their counts. Empty projects see the existing onboarding example report instead of a run, without a connect-data callout. The panes stack in narrow containers. Running investigations show a rough 2½-minute estimate, and the terminal outcomes without a report retain their activity in the same place. The viewer follows the task until it finishes, including when a report appears before the final task output. Its structured close-out distinguishes insufficient data, no actionable finding, and an incomplete investigation. An in-app notification tells the activating user when a run finishes, including non-report outcomes. The page promises “We’ll notify you when your first report is ready” in every pre-report state only when the activating user is enrolled for the configured first-report email; completing a run without a report does not fulfill that promise. No-data skips have no run. Missing or failed data probes remain unknown rather than being presented as an empty project.
The first-report email can also select a first-report-scout finding that needs a human decision. The onboarding UI promises an email only to the enrolled activating user when the email feature and sender configuration are enabled.
The project-group experiment self-driving-banger-scout decides whether the first-report scout runs and which model it uses: control runs no first-report scout, test-sol selects GPT-6.1 Sol on Codex with high reasoning and priority service, and test-deepseek selects DeepSeek Flash 1017 Fast on Pi. Both test arms use the same scout skill. The runner refuses a control assignment. Disabled, unknown, and unavailable flag values prevent production first-report runs; explicit eval and private-trial runtime overrides bypass enrollment. Enrollment is recorded at onboarding completion and rechecked before dispatch and sandbox startup. Unavailable experiment evaluations during dispatch use the activation job’s bounded retries; an explicitly disabled or invalid assignment skips the investigation. The runtime assignment is recorded on the run and attached to lifecycle and report events. Onboarding job status events also include the first-report variant and pre-run outcome, so no-data skips are measurable without counting them as runs. The onboarding dispatch job captures exposure for every arm after the same guards: scout-source eligibility, fleet and spend limits, an active activating user, and data. The arms therefore share one denominator, and setup failures after dispatch remain in it.
signals_scout_run_finished records the structured first-report outcome, whether a report became visible, and its latency from task creation. A persisted visible report still counts when the agent subsequently fails. The experiment's primary metric is whether a project resolves its first report within seven days of exposure, and the secondary metric is the time to that resolution. Both read signal_report_status_changed with status = resolved and is_first_report = true, where is_first_report marks the project's earliest visible report from any scout, so the control arm counts the first report its scheduled scouts produce. Compare grounding and actionability separately, and join task_run_id to gateway usage for model cost; missing usage is unknown, never zero. Turning the experiment flag off prevents future first-report sandbox starts; already-running investigations may finish.
Pi scouts require the Tasks worker's SANDBOX_AI_GATEWAY_URL, SANDBOX_AI_GATEWAY_MINT_KEY, and an entry in SANDBOX_AI_GATEWAY_PRODUCTS matching the scout, such as signals_scout:first-report. Only internally created scout tasks may mint these tokens. Pi provisioning fails when routing, minting, or recording the model pin fails; it never falls back to the legacy gateway. Before launching the experiment, verify the deployed Go gateway serves posthog/deepseek-ai/deepseek-v4.1-flash-1017-fast for signals_scout, with the Baseten host configured. The existing internal scout budget and paying-project configuration still apply.
Guided and quick onboarding create a SignalFirstReportNotification in the activation transaction, preserving the activating user for that environment. The first report that becomes ready, immediately actionable, and not already addressed after activation triggers a transactional email to that user's email address, tracked against their distinct ID. When the parent project's first investigation records this environment as its activation origin, email delivery selects the parent's reports and links while retaining the environment's recipient and notification preferences. The recipient must retain access to both projects. Other environment notifications keep their own report scope.
Sending requires the organization-scoped signals-first-report-email feature flag and is cloud-only. A disabled, missing, or unavailable flag leaves notifications pending without claiming or cancelling them, including in DEBUG. Sending also remains disabled until the dedicated POSTHOG_TRANSACTIONAL_EMAIL_API_KEY is configured. SIGNALS_FIRST_REPORT_EMAIL_TEMPLATE_ID in backend/constants.py selects the first-report transactional email template. The shared POSTHOG_PROJECT_ID and POSTHOG_HOST settings default to 2 and https://us.posthog.com, independently of the recipient's region. Both support optional environment overrides to select another sending project or host. The API key must be a project secret API key with hog_flow:write for that sending project, and the template must be an active transactional email template with a verified sender. Sending uses the shared transactional email client synchronously inside the notification task so the notification row can record the API's acceptance. Template creation and deployment configuration are separate from this code.
The template can use {{ variables.project_id }}, {{ variables.report_id }}, {{ variables.report_title }}, {{ variables.report_summary }}, {{ variables.report_priority }}, {{ variables.report_url }}, and {{ variables.activation_id }}. report_summary is the summary's first line, capped at 320 characters, matching the report's Inbox preview. report_priority is the latest Inbox priority judgment (P0–P4), or an empty string when unavailable. Both values are captured with the title and URL when the report is first claimed and remain unchanged on retry. Templates must omit the description and priority badge when those variables are missing, including for payloads captured before they were introduced. All template data is supplied explicitly; the recipient need not have a person profile in the sending project.
Report and actionability writes enqueue delivery on the Celery email queue after commit. A sweep on the default Celery queue runs every five minutes to recover missed enqueues and retry confirmed rejections. The sweep skips database queries and task fan-out when cloud delivery or the shared sender configuration is disabled. A Retry-After response sets next_attempt_at; both delivery claims and the sweep respect that deadline, and the Celery retry uses the same delay. Delivery rechecks report visibility, organization membership, project access, and report access, and never substitutes another recipient. The selected report and template data persist across retries. A selected report undergoing research stays pending until it becomes visible again. queued_at and message_id record acceptance by the transactional pipeline, not delivery to the recipient's mailbox. Sending does not wait for implementation or Slack notification delivery.
The transactional send API has no idempotency key. send_started_at is committed before the send, so concurrent workers, uncertain responses, and worker loss cannot automatically queue a duplicate. Confirmed pre-send failures, quota rejections, and connection timeouts clear that marker and can retry. A row with send_started_at set and queued_at unset requires reconciliation against the sending project's workflow history before retrying: record the accepted message if found, or clear the marker only after confirming nothing was queued. A crash before the request can also leave such a row. Uncertain outcomes are held rather than retried automatically.
Report links carry link_source=first_report_email, which the existing Inbox report opened event records. Workflows already captures email sends, delivery, opens, and link clicks. Join report engagement to the existing Inbox report action completed event (action_type=create_pr, outcome=success) by report_id to measure successful implementation kickoff, and use pr_created.signal_report_id for tracked PR outcomes. Inbox report action with action_type=copy_implementation_prompt records handoff intent; agent=clipboard distinguishes copying from agent deep links. Successful kickoff means the API accepted the implementation request, and handoff intent does not prove an external agent started work.
SignalReportReingestionWorkflow (signal-report-reingestion)
Deletes a report and re-ingests its signals through the current Signals ingestion pipeline. Useful when grouping decisions need to be re-evaluated after prompt or matching changes.
Defined in backend/temporal/reingestion.py. Workflow ID: signal-report-reingestion-{team_id}-{report_id}.
Flow:
- Fetch signals for the report from ClickHouse →
fetch_signals_for_report_activity. If no signals are found, the workflow deletes the report only. - Soft-delete signals in ClickHouse →
soft_delete_report_signals_activity2b. Wait for ClickHouse →wait_for_signal_in_clickhouse_activity, so re-emitted signals do not race stale rows during semantic search - Delete report in Postgres →
delete_report_activity - Re-ingest signals →
reingest_signals_activity, which converts eachSignalDataback into anemit_signal()call
emit_signal() now starts the buffer + emitter + grouping v2 pipeline, so reingestion flows through the same active path as fresh Signals traffic.
TeamSignalReingestionWorkflow (team-signal-reingestion)
Soft-deletes every non-deleted signal for a team and queues all of them for re-ingestion through the active pipeline.
Defined in backend/temporal/reingestion.py. Workflow ID: team-signal-reingestion-{team_id}.
This workflow is intended for full-team regrouping after changes to matching / prompting / grouping behavior.
It also supports a delete-only mode via TeamSignalReingestionWorkflowInputs(delete_only=True) / reingest_team_signals --delete, which soft-deletes all team signals and clears ORM report state without re-emitting anything.
Flow:
- Capture existing grouping pause state →
get_grouping_paused_state_activity - Pause grouping v2 for the team, extending the pause to at least
now + 10 minutes→pause_grouping_until_activity - Process one batch of non-deleted signals →
process_team_signals_batch_activity- Fetches the current first
50non-deleted signals from ClickHouse usingORDER BY timestamp DESC, document_id DESC - Soft-deletes each signal by emitting a replacement row with the original
document_id, timestamp, and metadata plusdeleted=true - In normal mode, calls
emit_signal()for each signal while grouping is paused, so the new signals are buffered but not grouped yet - In delete-only mode, skips the
emit_signal()step entirely - Waits for the deleted rows from that batch to land in ClickHouse before returning
- Fetches the current first
- Refresh the pause window if needed so grouping stays paused across long runs while keeping workflow history small
- Repeat until a batch processes
0remaining non-deleted signals - Delete all team reports + artefacts in Postgres →
delete_team_reports_activity- Deletes all
SignalReportArtefactrows for the team - Deletes all
SignalReportrows for the team - Runs after all signal batches have been processed, but while grouping is still paused
- In normal mode, this gives the re-emitted signals a clean ORM state to regroup into once processing resumes
- In delete-only mode, it leaves both ClickHouse and ORM report state cleared
- Deletes all
- Restore the prior grouping pause state →
restore_grouping_pause_activity
Important detail: the workflow intentionally does not paginate across iterations with offsets. Each batch mutates the underlying non-deleted result set by emitting delete rows, so once those rows land in ClickHouse the result set shrinks. Re-fetching the current first batch each time avoids skipping signals.
This workflow is currently started via the Django management command reingest_team_signals, not via a REST endpoint. Pass --delete to run the delete-only variant.
SignalReportDeletionWorkflow (signal-report-deletion)
Soft-deletes a report and all its signals. Triggered by DELETE /signal_reports/{id}/.
Defined in backend/temporal/deletion.py. Workflow ID: signal-report-deletion-{team_id}-{report_id}.
Flow:
- Fetch signals for the report from ClickHouse. If none are found, skip to report deletion.
- Soft-delete signals in ClickHouse 2b. Wait for ClickHouse until the deleted rows land
- Delete report in Postgres
This shares the same activities as reingestion; the only difference is that it stops after deletion.
SignalsScoutCoordinatorWorkflow (run-signals-scout-coordinator)
Polling coordinator for the headless Signals agent. Driven by a Temporal Schedule
defined in backend/temporal/agentic/schedule.py with every=COORDINATOR_INTERVAL_MINUTES
(30min) and ScheduleOverlapPolicy.SKIP to drop ticks rather than queue them. The tick
is just polling granularity. Each scout's own run_interval_minutes schedule decides
when it actually runs, unless an optional project-local cron run_cron_schedule overrides it.
Defined in backend/temporal/agentic/scout_coordinator.py.
Flow:
- Activity
fetch_enabled_signals_scout_runs_activitybounds candidates to the teams enrolled via thesignals-scoutfeature flag's JSON payload allowlist —guaranteed_team_idsminusskip_team_ids, with a hardcoded fail-safe default (_participating_teams→_enrolled_team_ids, modeled onposthog/temporal/ai_observability/team_discovery.py). Enrollment is flag-driven: editing the payload in the flag UI enrolls or drains a team next tick with no manual seed. For each enrolled team it callssync_canonical_skills(team, prune=True)to mirror the on-disksignals-scout-*skills onto the team'sLLMSkillrows, then callsscout_harness/config_registry.register_missing_configs. That auto-registers aSignalScoutConfigfor anysignals-scout-*skill lacking one, and returns the union of those skills with every live skill that already holds a config — so a bare-named scout is dispatched like any other config, while a config whose skill is gone is skipped. Bare-named scouts get their config through the API: thescout-createendpoint atomically creates a custom skill and config, andscout-config-createis the lower-level explicit upsert for an existing skill. The create modal is not one of those routes — it callsscout-create, but its name field always applies thesignals-scout-prefix. Failures here are logged and the tick continues — a stale skill is preferable to a dead tick. - For each enabled config, the coordinator computes how overdue the scout is. Rolling schedules are due when
last_run_at is None, ornow - last_run_at >= run_interval_minutes. A config with arun_cron_scheduleis due at the earliest unfulfilled cron slot, evaluated as wall-clock time in the project's timezone (so slots follow daylight-saving changes). It runs on the first coordinator tick at or after that slot. Saving the config after a slot has passed waits for the next slot. There is no sampling — every due scout is planned. - Due runs are sorted most-overdue-first and truncated at
MAX_RUNS_PER_TICK(50 per tick; the cost bound — overflow catches up next tick).last_run_atis advanced via.update()for everything dispatched (bypassessave(), so the per-tick write never hits the activity log). Planned runs are re-sorted by(team_id, skill_name)for stable child IDs. - Each
PlannedRunbecomes a childRunSignalsScoutWorkflowstarted withParentClosePolicy.ABANDONand a deterministic workflow ID per(team_id, skill_name, tick_id)so retried coordinators can't double-launch within a tick.
The coordinator's lifetime is seconds regardless of fan-out width; throttling happens at the Temporal task queue + worker concurrency layer. Pausing a scout is enabled=False on its config; slowing it is a larger run_interval_minutes. A cron run_cron_schedule (e.g. 30 9 * * *, 0 9 * * 1-5) anchors runs to wall-clock slots instead; null preserves the rolling interval behavior. All are tunable via the scout-config-update MCP tool.
Per-scout holdback (withheld_skills). The same flag payload carries a hard denylist for keeping an unreleased scout off the fleet while dogfooding it on a single project. A withheld_skills list (a default_team_config fleet-wide default, overridable per team via team_configs[<id>].withheld_skills with replace-not-merge semantics — set [] to release the full fleet to one team) names scouts that, for a held-back team, are never seeded into its LLMSkill rows (sync_canonical_skills skips them), never get a SignalScoutConfig (register_missing_configs drops them from its return), and are never dispatched. Resolved by _resolve_withheld_skills, most-specific-layer-first like the run caps. Unlike the soft enabled_skills seed allowlist (a default a user can still toggle on), this is a hard gate at the seed + dispatch layer — e.g. default_team_config.withheld_skills = ["signals-scout-error-tracking"] with team_configs["2"].withheld_skills = [] dogfoods error tracking on project 2 only.
RunSignalsScoutWorkflow
Child workflow per planned run. Defined in backend/temporal/agentic/scout_scheduler.py.
Thin wrapper around run_signals_scout_activity, which delegates to
scout_harness.runner.arun_signals_scout. The activity inserts the SignalScoutRun
bridge row at the start of the run; status, timing, and chat log live on the linked
tasks.TaskRun via MultiTurnSession, not on the bridge row. The workflow's only
job is to spawn the activity with start_to_close_timeout=WORKFLOW_HARD_CEILING_S,
a 2-minute heartbeat, and RetryPolicy(maximum_attempts=1) — the spec calls for "fail
safe and silent": a bad run does not retry blindly; the next scheduled tick will try
again. Single-flight is a best-effort app-layer guard: _has_running_run skips
dispatch when a prior run for the same (team, skill_name) has
task_run.status = IN_PROGRESS. An earlier partial unique constraint on
(team, skill_name) WHERE status='running' was dropped together with the bridge
model's own status column; active recovery of stranded IN_PROGRESS task runs
(_self_heal_stale_runs is a no-op today) is a tracked follow-up.
Three dispatch paths, one workflow. The coordinator's per-tick children are the schedule
(triggered_by="schedule"). Two off-schedule paths reuse the same workflow under their own stable id
namespaces, so they single-flight independently at the Temporal server: the run endpoint
("manual") and a workflow's "Run scout" step ("workflow", backend/scout_harness/workflow_runs.py,
called from the workflows product's workflow_scout_runs endpoint). Neither stamps last_run_at,
and neither feeds the failure-streak breaker whose threshold is sized on the schedule. The gates they
share (enrolment kill switch, daily run budget, Signals quota, daily report limit, in-flight
single-flight) live in backend/scout_harness/run_gates.py; the workflow path adds a 30-minute
per-(team, skill) cooldown, a backstop that does not have to trust the workflow's masking config.
Findings emitted during the run go through the harness's emit_signal_* tools,
which call emit_signal() with source_product="signals_scout" and
source_type="cross_source_issue" — from there the signal flows through the same
emitter → buffer → grouping v2 path as any other source.
See backend/scout_harness/AGENTS.md for the harness internals (runner, prompt
assembly, scratchpad + profile + run-history reads, lazy seed) and
skills/AGENTS.md for the scout fleet convention.
Django Models (Postgres)
Defined in backend/models.py.
SignalReport
The core model. Status machine:
potential → candidate → in_progress → ready
→ pending_input
→ failed
→ potential (reset by actionability judge)
→ candidate (mid-run quota pause; re-promotes on the next matching signal)
# Re-promotion: a READY report is re-promoted to candidate by the signal that carries it to its
# next bucket in RESEARCH_SIGNAL_BUCKETS, triggering a new summary run that reuses the previous
# repo selection and findings for already-seen signals. Suppressed between buckets and once every
# bucket is used (see Research buckets).
# RESOLVED is terminal and never re-promotes: a recurrence spawns a fresh report, linked back to the
# resolved one via a recurrence_of report link (assign_and_emit_signal_activity). A report dismissed with a
# reason that claims the issue is fixed (already_fixed, fixed_outside_posthog, pr_merged) makes the
# same claim, so it forks the same way. A wrong_repo dismissal also forks so repository selection
# can run again; every other dismissal reason absorbs later signals silently
# (recurrence.py). The pipeline records each successor with a typed recurrence_of link. Generic related_to
# links do not control signal assignment. Matching follows this successor chain, including
# preference-dismissed reports that must keep absorbing signals. Repeated feedback does not break
# the chain. The parent lock protects both live assignment and backfill creation.
ready → candidate
# Resolve: a report is marked resolved when the requested work is done. Two paths:
# - automatic: the tasks GitHub webhook resolves a report when its linked implementation PR merges
# - user-driven: the state API accepts state='resolved' (single + bulk) so a user or MCP agent
# can resolve directly — useful for PR-less fixes (skill-body changes, NO_REPO reports) that the
# webhook could never reach. Restricted to the same statuses the machine allows below.
ready | pending_input | failed → resolved (a run that died in processing still describes real work,
so whoever fixed it can say so instead of being pushed onto the dismissal path)
suppressed → resolved (resolve an archived report straight out of the archive; the state action
refuses this unless it was researched before being archived, so an unresearched
report can't be laundered candidate → suppressed → resolved)
# Refund interaction: a refund is final (later PR runs are never billed). The refund path suppresses
# the report so it leaves the inbox and the dismissal receiver closes the implementation PR that was
# paid for and then refunded — including a manually-resolved one, whose PR may still be open. Only a
# report resolved by a merged PR stays resolved (nothing to close). The state action then refuses to
# move a refunded report back to potential — the live status the pipeline re-promotes from — whether
# it sits in suppressed or resolved, and refuses to resolve it back out of the archive.
# Transitions enforced by SignalReport.transition_to():
# - deleted is terminal (no transitions out; excluded from API via queryset)
# - suppressed transitions back to potential, or straight to the researched status it held before
# being archived (ready | pending_input | resolved | failed) — see restore_target_status()
# - any non-deleted status can transition to deleted or suppressed
# - reset/snooze = transition in_progress | pending_input | ready | resolved | failed → potential;
# snooze_for=N sets signals_at_run = signal_count + N
suppressed → potential
any (except deleted) → deleted
any (except deleted) → suppressed
| Field | Type | Description |
|---|---|---|
team |
FK → Team | Owning team |
status |
CharField | One of: potential, candidate, in_progress, pending_input, ready, failed, deleted, suppressed |
total_weight |
Float | Sum of all assigned signal weights (reset to 0 if deemed not actionable) |
signal_count |
Int | Number of signals assigned |
title |
Text (nullable) | LLM-generated title (set during matching or summarization) |
summary |
Text (nullable) | LLM-generated summary |
metrics |
JSON list | Up to six typed report measurements. Every metric stores a bounded live Trends query plus an optional latest cached snapshot, but not longitudinal query results. |
error |
Text (nullable) | Error message if failed, or reason if pending input / reset to potential |
signals_at_run |
Int | Forward-looking promotion threshold. A potential or ready report will not be (re-)promoted to candidate until signal_count >= signals_at_run. Defaults to 0, so fresh reports always pass immediately. Advanced by 3 each time a summary run starts, preventing the report from immediately re-promoting. Snoozing sets this to signal_count + N, pushing the threshold forward by an additional N signals. |
run_count |
Int | How many times the summary workflow has run for this report. Incremented on each candidate → in_progress transition. Used in the Temporal workflow ID to give re-promoted reports a unique execution ID. |
promoted_at |
DateTime (nullable) | When report was promoted to candidate (cleared on reset to potential) |
last_run_at |
DateTime (nullable) | When summary workflow last ran |
first_visible_at |
DateTime (nullable) | Set-once: when the report first became user-visible (entered ready, pending_input, or failed, the statuses the inbox lists). Stamped by transition_to() for pipeline reports and by create_scout_report for scout reports born visible; never cleared, so re-research and suppress/restore cycles don't recount it against SignalTeamConfig.max_reports_per_day. Null for historical reports or reports that never surfaced. |
conversation |
DEPRECATED | Was: FK → Conversation. Wrapped in deprecate_field() |
relevant_user_count |
DEPRECATED | Was: Int for relevant user count. Wrapped in deprecate_field() |
cluster_centroid |
DEPRECATED | Was: ArrayField(Float) for video segment clustering. Wrapped in deprecate_field() |
cluster_centroid_updated_at |
DEPRECATED | Was: DateTime for centroid freshness. Wrapped in deprecate_field() |
Indexes: (team, status, promoted_at), (team, created_at)
SignalReportArtefact
An append-only, attributed, schema-validated log of the work done on a report. A report reads as a living document: the evidence the research agent gathered, the commits it pushed, the task runs that executed, and free-form notes, accumulating over time. Producers never mutate in place — re-assessing appends a new row.
| Field | Type | Description |
|---|---|---|
team |
FK → Team | Owning team |
report |
FK → SignalReport | Parent report (related_name="artefacts") |
type |
CharField | Artefact type (see ArtefactType enum) |
content |
TextField | JSON content stored as text |
created_at |
DateTime | Auto-set; rows are ordered newest-first |
updated_at |
DateTime (nullable, auto_now) |
Log entries are editable in place |
created_by |
FK → User (SET_NULL, nullable) |
Attribution — the user who produced the row, if any |
task |
FK → Task (SET_NULL, nullable) |
Attribution — the task that produced the row, if any |
created_by / task are nullable: legacy rows and explicit system writes carry NULLs.
The activity log shows PR references as title links with repository, number, and latest known state. The title comes from the PR's explicitly attached implementation task, with a PR-number fallback when that task is unavailable. Replacement entries group replacement and previous PRs separately; closure outcomes remain distinct from current PR state.
Status vs log. Everything is append-only; the STATUS_ARTEFACT_TYPES / LOG_ARTEFACT_TYPES sets classify what an entry means:
- status — the report's current state (
safety_judgment,actionability_judgment,priority_judgment,repo_selection,suggested_reviewers). Each (re)assessment appends a row; the current status is the latest row of that type (serializers derive priority/actionability/reviewers viaorder_by("-created_at")[:1]). - log — discrete work (
code_reference,commit,task_run,note); these accumulate and are addressable by UUID (PATCH/DELETE). signal_findingis in neither set: its identity is(report, content.signal_id), latest per signal wins.dismissalentries stack.
Artefact types (SignalReportArtefact.ArtefactType enum):
| Type | Content |
|---|---|
video_segment |
Video segment data from session clustering |
safety_judgment |
{"choice": bool, "explanation": "..."} — true = safe |
actionability_judgment |
{"actionability": "immediately_actionable" | "requires_human_input" | "not_actionable", "explanation": "...", "already_addressed": bool} |
priority_judgment |
{"priority": "P0"|"P1"|"P2"|"P3"|"P4", "explanation": "..."} |
signal_finding |
{"signal_id": "...", "relevant_code_paths": [...], "relevant_commit_hashes": {"abc1234": "reason"}, "data_queried": "...", "verified": bool} |
repo_selection |
{"repository": "owner/repo" | null, "reason": "...", "task_id"?: "..."} |
suggested_reviewers |
[{"github_login": "..." | null, "user_uuid": "..." | null, "github_name": "...", "relevant_commits": [...], "reason"?: "...", "source_skill"?: "...", "is_skill_owner"?: bool}] — stored reviewers can use either identity and gain current PostHog user data and display metadata at serializer read time |
dismissal |
{"reason"?, "note"?, "selected_repository"?, "corrected_repository"?, "user_id"?, "user_uuid"?, "slack_user_id"?} — stacking dismissal entries; the repository fields are set on wrong_repo dismissals |
code_reference |
{"file_path": "...", "start_line": int, "end_line": int, "contents": "...", "relevance_note": "..."} — a span of source lines (single line = equal start/end) |
commit |
{"repository": "owner/repo", "branch": "...", "commit_sha": "...", "message": "...", "note"?: "..."} — one pushed commit |
task_run |
{"task_id": "...", "run_id"?: "...", "product": "...", "type": "..."} — a task run associated with the report (see below) |
note |
{"note": "...", "author"?: "..."} — free-form note (markdown allowed) |
check_result |
{"check_id": "...", "kind": "...", "title": "...", "outcome": "passed"|"failed"|"errored"|"inconclusive", "explanation": "...", "reason"?, "observed_value"?, "baseline_value"?, "threshold"?, "run_id"?} — one run of a report check |
report_link |
{"kind": "depends_on"|"part_of"|"follow_up_of"|"duplicate_of"|"recurrence_of", "report_id": "...", "reason"?: "..."} — a typed, directed link to another report in the same project (see Typed report links) |
autostart_skip |
{"skip_reason": "duplicate_of"|"blocked_by_dependency"|"plan_parent", "linked_report_id"?: "...", "detail": "..."} — a link gate held automatic implementation back (see Typed report links) |
Content schemas. artefact_schemas.py is the canonical, pydantic-only home of every content shape, collected in ARTEFACT_CONTENT_SCHEMAS (one model per type; a test asserts exact coverage). Raw payloads become typed models once, at the boundaries (parse_artefact_content); the model helpers derive a row's type from the content model's class (artefact_type_for), so a type can never mismatch its content. repo_selection reuses the tasks product's RepoSelectionResult DTO directly (kept in the dependency-light leaf module repo_selection/types.py so importing the schema registry doesn't pull in the sandbox runtime). Reads of legacy rows stay tolerant — parse failures are skipped or degraded, never raised.
Charts. SignalReport.charts holds the queries the inbox renders on the report, so a finding about a metric move can be seen rather than taken on trust. They sit on the report rather than in its artefact log because a chart is part of what the report says, not a record of something that happened to it — it illustrates the summary and is replaced with it, where a log entry would accumulate versions the reader never asked for. Their schema lives in report_charts.py (ReportChart), kept as dependency-light as artefact_schemas.py for the same reason: it loads with the signals models.
query is an InsightVizNode / DataVisualizationNode / SavedInsightNode node, kept as an unparsed dict — validating it against the real node models would mean importing posthog.schema, so only the kind allowlist, a size bound, and the executable-payload refusals (bytecode, a nested HogQuery, sendRawQuery — a chart renders data, it does not run code — plus a nested SuggestedQuestionsQuery, refused for cost, since its runner calls an LLM once per reader who opens the report) are enforced server-side, and the renderer degrades in place on a bad query. That is the same contract a notebook's ph-query node has.
chart_id is the author's own slug because a report's summary and its charts are written in one call: the summary places a chart by referencing it as a markdown link with a chart: target ([Daily signups](chart:signups-drop)), and an unreferenced chart still renders below the summary. Ids are unique within the set, so a reference resolves to exactly one chart. Placement is resolved from the same markdown parse the renderer runs (products/signals/frontend/inbox/utils/chartPlacement.ts) rather than by matching the raw summary: a reference in a code span is literal text, one in a table cell or heading has no room to draw, and a repeated reference points back at a chart rather than asking for a second copy — each of those resolves to "not placed here", so the chart falls to the trailing block instead of being drawn twice or lost. A paragraph holding nothing but references lays its charts out as a row. Height comes from the node (a BoldNumber gets a short box, a retention grid a tall scrolling one) unless the author sets size.
Scouts write them via charts on emit_report / edit_report. On an edit charts is the report's whole set — it replaces what was there, the way summary replaces the summary. Omitting the field (or sending null) leaves the existing charts alone; an explicit empty list clears them, so a scout can retract a chart its finding has outgrown rather than only ever swap one for another. On the emit path the safety judge sees a report's chart titles, captions, and queries alongside its prose, so an injected chart is suppressed with the report. The edit path judges nothing today — not title, summary, notes, reviewers, or charts — so a chart set by a later edit reaches the reader unscreened, exactly as an edited summary does.
The main agentic research pipeline authors charts too, so every signal source that funnels through it (not just scouts) can produce a visual report. The research agent's presentation step (report_generation/research.py) returns charts alongside the title and summary; the caller activity (temporal/agentic/report.py) replaces SignalReport.charts with them, and a re-research is shown the report's current charts to keep, refresh, or drop. Unlike the scout emit path, these charts are not safety-judged: the pipeline judge screens the input signals before research runs, so it never sees research-authored charts — the same as it never sees the research-authored title/summary, which are agent output from already-screened signals.
Report checks. SignalReportCheck is the report's only forward-looking row: a title, a rationale, a kind-specific config, and the time to evaluate it. Everything else on a report describes what was already observed, so a claim that stopped holding was only ever found by a person going back to look. A check turns that into a scheduled measurement whose verdict lands in the artefact log as a check_result.
The schedule is the check's own clock (next_run_at, optional run_interval_minutes, runs_remaining, expires_at), not something derived from a merged pull request: many fixes land in the skills store with no pull request to date a soak window from. Bounds live in report_checks.py — at most 5 active checks per report, a 90-day horizon, a 6-hour interval floor, at most 10 runs, and a 30-day default expiry after the last scheduled run. Terminal statuses (passed, failed, errored, expired, cancelled) are final, so a stored result always refers to a row whose state explains it; a new expectation is a new check.
metric_threshold needs no agent: it measures one bounded live Trends query — either its own or the query behind a metric_id the report already shows — and compares the result. The query rides report_metrics.validate_live_metric_query, so a check inherits the metric contract's node allowlist, relative window, and single-output-series rule. The comparison is the alerts product's evaluate_threshold, which is why the operators are lte, gte, and between: those map exactly onto the alerts bounds, and strict operators would mean a second comparison engine for a distinction a soak does not make.
agent is the kind for a claim no single number settles, which is most of the inbox: "did the exception stop?" needs the issue looked up and its recent events read rather than a threshold compared. Its config is {instructions, skill_name?, probe_hints}, all of it untrusted prompt material. report_check_agent.py owns the lane. A due check is dispatched as a scout run with triggered_by="check" — a fourth dispatch source next to schedule / manual / workflow, so it shares the fleet, spend, and single-flight gates in run_gates.py but stamps no last_run_at and never feeds the failure-streak breaker, whose threshold is sized on the scout's own cadence. The run note carries the check id, title, rationale, instructions, probe hints, the report, and the note whoever resolved or dismissed it typed; the prompt renders it as the run's assignment (prompt._CHECK_NOTE_TEMPLATE) rather than as the nudge a manual run's note is.
Most reports are pipeline-authored and have no scout behind them, so skill_name is optional: a check that names none runs on signals-scout-inbox-validation, the fleet's follow-up scout, which is an operational scout and so is seeded enabled on every enrolled team. Authoring applies the same lane rule (agent_check_lane_available): an agent check is written only when its lane has a config and a live skill row, and a paused lane still counts. A project with no scout configs at all cannot run the check, so the coordinator cancels such a check with a no_check_lane reason instead of recording an error. When the lane otherwise cannot run — the project is not enrolled, the skill is withheld or its row is gone — the check records an errored result naming the reason, so a reader finds out on the report rather than waiting out a silent horizon. A refusal that can clear (a run of the same scout in flight, a daily run budget, a spend gate, a paused scout) defers the check by an hour instead, spending no error. A paused scout is in that group because three errored runs during a pause would retire the check before the scout resumes. The check does not move to another scout while it waits. A dispatched run that records no verdict counts as an errored run, unless its scout is paused when the result window ends: then the check waits too. When a scout is enabled again, the checks deferred on it become due at once, so the next tick dispatches them. reactivate_pause_errored_report_checks repairs the checks an older dispatch retired as errored for a paused scout.
A dispatch is not a verdict. The row stays active with dispatched_at stamped, and only scout-check-record-result (scout_harness/tools/checks.py) closes it. The dispatch passes check_id through the workflow input, and the runner stamps it on the run row's metadata. That run may close its check once, also when the lane now resolves to another scout. Its verdict clears dispatched_at, so a retry after a recurring check re-arms is refused. A run on the check's own lane may also close an active check that is due now or waits on a run, so a scout that measures a due check in passing can record the result. A pending check is refused, because its report has not resolved. A due check the coordinator would not dispatch is refused too, because its report is suppressed or its horizon passed. A run that ends without calling the tool leaves dispatched_at set, and the first tick past AGENT_CHECK_RESULT_WINDOW records an errored result saying so.
scout-report-check-list returns a run_state for each check (waiting_on_report, paused, scheduled, due, queued, running, stale, or the terminal status), plus waiting_on_run, dispatched_run_id and dispatched_at. The run id is resolved from the run rows stamped with the check, so it stays null until the dispatched run starts.
Execution rides the scout coordinator tick (run_due_signal_report_checks_activity, gated behind a workflow.patched marker). Each tick expires what timed out, then advances the due rows by one step, capped per tick and per team. A pass on a recurring check re-arms it anchored on now so an outage cannot owe a burst of catch-up runs; a breach retires the check as failed; a run that cannot be measured retries and only retires after three consecutive errors. A check runs only while its report is resolved. Each tick moves an active check whose report is not resolved back to pending (park_checks_on_unresolved_reports), which covers a report that was reopened or archived and any row written active on an open report. The move keeps the check's soak (or the gap its author left before the first run), clears its error streak and any open dispatch, and resets its horizon, so the next resolve arms it with a fresh soak and retry accounting starts again after that resolve. A check that names a report metric stores a copy of that metric's query at creation, so a later edit to the metric does not change what the check measures. Both kinds are written through one funnel (record_check_verdict) that re-reads the row under a lock, so a check cancelled while its run was working records nothing.
A check written before the fix exists has no date to carry. The research pipeline's verification turn is that case: it authors the check in the same pass that writes the report. Such a check is stored pending with a soak_minutes window, and the report's transition to resolved is what sets next_run_at — a merged pull request's webhook, a manual resolve in the inbox, and an MCP state write all end in the same save, so the resolve is the clock for every kind of fix, including the ones that never had a pull request. receivers.arm_pending_checks_when_report_resolved is the choke point. A pending check counts against the per-report cap, is never collected as due, and expires at the 90-day horizon if its report never resolves.
A failed verdict on a resolved report is the one outcome nobody would otherwise read, because the inbox no longer lists the report. An agent check has a scout in the loop: the check brief (_check_relapse_section in scout_harness/prompt.py) tells a lane that holds emit_report to author a fresh report linked follow_up_of the original before it records failed. Every other failure takes the deterministic path: a metric_threshold breach, or a failed agent verdict from a run whose stamped report_channel lacks emit_report, on a resolved report emits a source_product="signals_check" signal carrying the verdict and the origin report id. The grouping stage then applies the rule it already applies to any signal landing on a resolved report — a fresh report, linked back by related_to and by a typed follow_up_of link carrying the verdict as its reason. Nothing reopens the terminal one. The two rows say different things: related_to (and recurrence_of) says the issue came back, follow_up_of says a measurement of the fix breached, and the research agent reads the typed one as context.
signals_check / check_failed is self-authorizing in SignalSourceConfig.is_source_enabled, alongside the Replay Vision scanner. It is the inbox emitting to itself, so there is no team to configure: check_failed is deliberately outside SignalSourceConfig.SourceType, the subset that carries a per-team row, so a row-backed gate would refuse the pair forever. The check the team already authored is the opt-in. CheckFailedSignalInput is registered in SIGNAL_INPUT_VARIANTS for the same reason — without it validate_signal_input rejects the pair as an unknown signal type, and the producer logs and swallows the error.
Five best-effort events in report_check_telemetry.py follow a check: signals_report_check_skipped for a research spec dropped before it was stored; signals_report_check_created from the shared write, so every author is counted; signals_report_check_dispatch for each attempt to hand an agent check to a run, with the refusing gate as reason on a deferral; signals_report_check_evaluated for each verdict, where a check_status of active means the check looks again; and signals_report_checks_expired, one per project per sweep, whose never_ran_count is the checks that were written and never looked at.
Checks are written in one place, report_check_authoring.create_check, which owns the per-report cap, the metric-query copy, and the attribution for every caller. Only two authors write a check: a scout run, and the research pipeline. SignalReportCheckViewSet (/signals/reports/:id/checks/) is read and cancel only, and the public MCP surface is inbox-report-checks-list / -retrieve. There is deliberately no caller-facing create: an agent check puts its author's prose in front of a privileged scout run, which inbox:write does not authorize, and every other surface that does so demands llm_skill:write plus skill-editor access. Scouts create and cancel through the scout tools scout-report-check-create / -list / -cancel (gated on the report-channel allowed_tools opt-in and the signal_scout_report:write scope, the two layers scout-emit-report already sits behind, because a check is a write on a report). Research writes its specs in mark_report_ready_activity. The scout tools also bind to the run the caller's sandbox token was minted for, so a run cannot write, list or cancel under a sibling's task, and a write goes through the same emit preflight as a signal or a report, so a dry-run scout writes no check. There is no update: editing a threshold after a result would make the recorded verdict unreadable.
Research authors its checks in the verification turn (FixVerificationOutput.checks, a list of CheckSpec). The turn is offered the metric_threshold kind only while the report-metrics rollout is on, because such a check references one of the report's own metrics by metric_id, and the agent kind only while the team is enrolled in scouts and has a lane that can run it, because nothing would ever run a check with no scout behind it. When only the metric kind is on, the prompt says so. create_checks_from_specs drops an agent spec that still has no lane, keeps the others, and fires signals_report_check_skipped. The daily run budget is not part of that gate: a research check stays pending until its report resolves, and the dispatcher defers a throttled check by itself. With neither, the checks field never reaches the schema. The specs are persisted by mark_report_ready_activity, in the same transaction that writes the title, summary, charts and metrics, so a spec naming a metric that transition did not store is dropped there rather than kept pointing at nothing. Only a pass that settles writes its specs: a pass that loops back to candidate is about to be replaced, and its checks would be armed later against prose they were not written for. For the same reason a settled pass that returns specs cancels the report's earlier pending checks first, which only research writes. A pass that returns none leaves them alone, because the verification turn is best-effort. The plan and the check read as one thing in the timeline: the prose is the plan, and to_note() names the check it scheduled.
Report metrics. SignalReport.metrics is typed report content, not an artefact log or a materialized time series. Its dependency-light schema lives in report_metrics.py (ReportMetric). Each entry has a stable id, title, semantic kind, role, display format, a required bounded live query, and an optional cached snapshot. Legacy comparison data stays compatible in storage but is not exposed as a live comparison. A report accepts at most six metrics, at most one primary metric, and at most one affected_users metric.
Every metric uses an InsightVizNode around one TrendsQuery over event or action sources. Consumers derive a BoldNumber for the whole-window value and an ActionsBar for longitudinal buckets. An affected_users metric uses exactly one series with math: dau. An affected_sessions metric uses exactly one series with math: unique_session. Neither kind uses formulas or group math. The default daily window starts at -13d, which gives 14 inclusive buckets including today. A longer window keeps only the trailing 14 buckets in the strip, while the whole-window value uses the full window. New authoring omits comparisons until the server can keep equal adjacent windows live. Non-currency units are lowercase. Revenue uses an uppercase ISO currency code such as USD.
Report list serialization returns snapshot-only metric metadata. It omits live query definitions and legacy comparisons, and it never runs a query. When a person opens the inbox list or a report, the frontend posts the visible ids to refresh_metrics. A snapshot measured in the last 15 minutes is served as is through a query-cache override. One call runs at most 40 source series and stops on a best-effort 20-second deadline. Refresh writes value, value_at, and series, and clears comparison. The endpoint allows 10 requests per minute for each team. There is no background job, so a report nobody opens costs no queries. Full metric-state compare-and-swap persistence prevents an old query result from overwriting a concurrent report edit. Snapshot refreshes leave updated_at unchanged so the inbox does not reorder reports.
Snapshots are materialized without a requesting user under the team's default property rules. List and detail serialization re-check access before exposing data-bearing fields. Both response shapes omit legacy comparisons. An unsafe query definition is null on detail. Snapshot-only or queryless rows are legacy or malformed and are always redacted.
Opening a report detail mounts every metric through the normal frontend Query path, so execution and caching follow the same query service used by insights. Consumers derive separate BoldNumber aggregate and ActionsBar longitudinal requests from each stored definition. If a query fails, the detail can show the latest cached snapshot instead. Longitudinal points remain in the analytics engine rather than being copied into a second time-series table.
Agentic research writes metrics atomically with the matching title, summary, and charts. Its independent organization-level rollout uses signals-report-metrics (team_report_metrics_enabled in report_content_gates.py, on in DEBUG), rather than inheriting chart eligibility. While report metrics are enabled, the research output is the report's whole metric set: a nonempty list replaces the old set and an empty list clears it. Re-research receives the current set so it can repeat the metrics that remain valid. None is reserved for the feature-disabled or older-workflow replay path and preserves the stored set. Scouts set the initial whole set through emit_report. On edit_report, omitting metrics or sending null preserves the set, an empty list clears it, and a supplied list replaces it. Both scout write paths read the same rollout flag. While it is off, a supplied set is dropped before the safety judge, so nothing is stored and an edit preserves what the report holds. An empty list still clears. A metrics-only edit changes Inbox content but does not redeliver the report to Slack.
Attribution. Every write helper (append_status / add_log / append_finding / append_dismissal) requires an ArtefactAttribution — exactly one of from_user(user_id) / from_task(task_id) / system() — so no write site can silently skip it. Agent writes are attributed deterministically: sandbox provisioning bakes the agent's task id into an X-PostHog-Task-Id header on its MCP config, forwarded by the MCP server on every API call; the LLM never handles its own task id. The header is attribution metadata, not an authorization boundary (the token is team-scoped and the named task must belong to the same team).
Write surface. SignalReportArtefactViewSet exposes POST / PATCH / DELETE for any type (a status write appends a new latest-wins row), the bespoke suggested_reviewers PUT, and a diff action that renders a commit artefact's branch against the repository default branch via GitHubIntegration.get_diff (GitHub compare API, validated repo/ref/sha). All gated by scope_object = "inbox" (inbox:write). Custom agents queue artefacts during a run via CustomSignalAgent.register_artefact, persisted in the report's transaction and attributed to the agent's task — except commit (written automatically by the signed-commit harness) and task_run (written by report persistence), which never need registering there.
Claim content stores a server-generated display_name: a phase label for internal agents, or the user name plus external client (for example, "Alex's Codex").
Unknown clients use "Alex's agent". The label is fixed at claim time and does not participate in ownership checks; older claims can omit it.
Claims and task association. Agents use the claim endpoint to start or resume work, attach pull_requests, and release ownership.
The returned claim_id identifies the work attempt; stale or foreign claims are rejected, and taking another actor's claim requires explicit takeover=true.
The shared ownership helper derives the active owner from the latest work_claim artefact and its work_release entries.
Internal task agents automatically receive a task_run association on claim; clients do not write task associations separately.
When a person holds the claim and starts an implementation task themselves, the task works under that claim instead of replacing it, so the person stays the owner while the agent writes the PR.
A task started by anyone else is refused while a person holds the claim.
That task can resume the person's claim and add pull requests and notes to it, but only the person can release it.
A takeover can name the claim it expects to replace in takeover_claim_id, and the server refuses it when a different claim is active.
In the Inbox, a suggested reviewer claims a report from their own row, and the report shows who works on it under the title. Agent and task claims are not shown there, because a person still has to review the agent's work.
Notes, commits, and PR links can reference the claim without mixing that context into actor attribution.
Attribution identifies the authenticated user plus coding-agent name, or the internal task, and describes who attached the PR rather than its GitHub author.
Pull requests. SignalReportPullRequest stores GitHub identity and state per team; pull_request artefacts link it to reports and optional claims.
Reads combine these links with legacy assignments and task outputs, while all new writes use artefacts and shared PR state.
The shared PR reader requires a team ID and excludes reports and links outside that team.
Only legacy assignments are backfilled; task-output PRs remain readable without a full backfill.
SignalReportAssignment is a read-only fallback until backfilled; once claim history exists it cannot restore a released owner.
GitHub-verified state takes precedence over imported snapshots, and merged is terminal.
A report completes when every linked PR is closed or merged: resolved if any merged, otherwise suppressed.
Legacy single-PR fields select an active PR first, then merged before closed, with deterministic ordering.
Auto-start retains its SignalReportTask implementation gate alongside task-run artefacts.
Notes:
- The serializer still supports legacy
actionability_judgmentpayloads that usedchoiceinstead ofactionability.
Indexes: (report) (signals_sig_report__idx) plus a latest-wins (report, type, -created_at) index backing status derivation.
SignalTeamConfig
Per-team singleton config for Signals settings, including the default autonomy priority threshold.
| Field | Type | Description |
|---|---|---|
id |
UUID (PK) | Primary key (UUIDModel) |
team |
OneToOne → Team | Owning team (related_name="signal_team_config") |
default_autostart_priority |
CharField | Default severity threshold for auto-start (P0–P4, where P0 is highest) |
github_issue_writeback_enabled |
Boolean | Opt-in public comments on source GitHub issues. Defaults to false. |
created_at |
DateTime | Auto-set on creation |
updated_at |
DateTime | Auto-set on save |
Notes:
- Created on first access through
get_or_create_team_extension. Readers treat a missing row as the field defaults. default_autostart_prioritydefaults toP4(every report priority auto-starts). The inbox UI exposes it as the "Project threshold" control on the PR generation card.SignalUserAutonomyConfig.autostart_priorityholds a per-user override (null= use the team default). The inbox UI exposes it as the "My threshold" control on the same card, where a "Default" segment maps tonulland inherits the project threshold.github_issue_writeback_enabledadds a report link to each source GitHub issue after the report notification completes. The comment contains no report title or research. The report requires project access.- GitHub comments run in a separate Temporal activity on every settle, including settles after the first notification. GitHub failures cannot prevent the report notification.
SignalReportGithubCommentholds one claim per report and issue, with a lowercase repository name. Completed claims prevent repeat comments. Pending claims expire after ten minutes.- A later settle checks existing comments for a stable marker before it retries an expired claim. An incomplete comment read leaves the claim pending. GitHub provides no atomic comment idempotency key, so this is best-effort recovery.
- The worker checks the current issue state immediately before posting. Closed issues, locked issues, and pull requests receive no comment.
- A report's
source_issueartefacts count as source issues too. Scout and user-authored reports skip research, so their writeback runs from a Celery task queued after the write commits (source_issues.py). A source issue gets a comment only after the team's GitHub source ingests that issue; grouping queues the writeback again when the issue's signal joins the report. - Grouping attaches a GitHub issue signal to the oldest open report whose
source_issuenames the same issue, before similarity matching and without the specificity check. Resolved, suppressed, failed, closed, and deleted reports do not take it. - See GitHub issue comments for recovery limits and diagnostic logs.
SignalUserAutonomyConfig
Per-user opt-in config for Signals autonomy. Existence of a row means the user is opted in.
| Field | Type | Description |
|---|---|---|
id |
UUID (PK) | Primary key (UUIDModel) |
user |
OneToOne → User | The opted-in user (related_name="signal_autonomy_config") |
autostart_priority |
CharField (nullable) | Per-user priority override (P0–P4); null = use team's default_autostart_priority |
created_at |
DateTime | Auto-set on creation |
updated_at |
DateTime | Auto-set on save |
Notes:
- One row per user (enforced by
OneToOneField) - User is not scoped to a team — the autostart logic resolves team membership at runtime
- Managed via
PUT /api/users/@me/signal_autonomy/(opt in / update) andDELETE(opt out)
SignalReportTask (legacy — implementation gate and replacement discovery)
The legacy report↔task link table.
General task↔report association has moved to task_run artefacts (a task_run artefact's task FK is the association; purpose comes from its (product, type)).
Two readers keep this table alive.
The first is the auto-start idempotency gate: record_implementation_task dual-writes a relationship="implementation" row here and the task_run artefact, and auto-start checks this table (alongside protected task-run artefacts) when deciding whether an implementation has already started.
The second is replacement discovery: automated_targets reads the same rows to list a report's implementation tasks before it decides which automated PRs a new research pass may replace.
backfill_task_run_artefacts converting every legacy row into a task_run artefact is a precondition for dropping this table, not the whole job: both readers must move to the artefact log first.
A drop that leaves replacement discovery pointed at this table finds no candidates, so the feature reports "nothing to replace" instead of failing.
SignalSourceConfig
Per-team configuration for which signal sources are enabled.
| Field | Type | Description |
|---|---|---|
team |
FK → Team | Owning team (reverse accessor sealed with related_name="+") |
source_product |
CharField | One of: session_replay, llm_analytics, github, linear, zendesk, conversations, error_tracking, signals_scout (SourceProduct enum) |
source_type |
CharField | One of: session_analysis_cluster, evaluation_report, issue, ticket, issue_created, issue_reopened, issue_spiking, cross_source_issue (SourceType enum) |
enabled |
Boolean | Whether this source is active (default True) |
config |
JSONField | Source-specific configuration (for the Linear issue source, linear_team_ids), plus the shared steering keys (steering, default_not_actionable) read by the emission actionability gate |
created_by |
FK → User | User who created the config (nullable) |
Behavioral notes:
llm_analyticssignals go through the standard enabled-row check like every other source.evaluation_reportis the only type AI observability emits, gated by its own(llm_analytics, evaluation_report)row (the inbox "AI observability" toggle). The per-result workflow and activity remain registered only so existing Temporal histories can finish replaying.- For session replay configs, serializer validation enforces that
config.recording_filtersis a JSON object when present. - Two
configkeys steer the emission actionability gate per source:config.steering(free text, the team's preferences about the source's records — injected into the canonical actionability prompt, never replacing it) andconfig.default_not_actionable(boolean — flips the gate's "when in doubt" posture from keep to filter). Serializer validation enforcessteeringis a string capped atSTEERING_MAX_LENGTH(2000 chars) anddefault_not_actionableis a boolean; injection escapes braces so the text can never break the gate's one-word output contract (seebackend/emission/steering.py). The keys apply to sources that runrun_signal_pipeline(data warehouse imports and Conversations), and to the direct-emitting sources listed inDIRECT_STEERABLE_SOURCES(error tracking and health checks), whichemit_signalruns through the gate inbackend/emission/direct_gate.pybefore queueing the signal. A direct source is judged only when the team has written steering, and only against that text: the canonical prompt there states no criteria of its own, so a first rule filters what it describes and nothing else. Sources outside both sets persist the keys unread, and future consumers (report research, the autostart gate) are expected to read the same text. - The Linear issue source reads
config.linear_team_ids(list of Linear team id strings, capped atSCOPE_IDS_MAX_COUNT): the warehouse still syncs the whole workspace, but the emission fetcher adds anINclause on the issue's team id, so only issues from those teams become signals. An absent, empty, or malformed list means every team, which keeps rows written before the key existed working and stops a bad row from silently narrowing the scope. Applies from the next sync; reports already in the inbox are not touched. The fetcher-side mechanism is generic (scope_field/scope_config_keyonSignalSourceTableConfig), Linear is the only source that declares it. The key itself is declared incontracts.SCOPE_CONFIG_KEYS, so the API validates the same key the fetcher filters on. - The product analytics source (
analytics/anomaly_investigation) acceptsconfig.alert_scope:investigated_only(the default when the key is absent) orall_firing_alerts. The serializer rejects any other value, and rejects the key on every other source (contracts.ALERT_SCOPE_SOURCE). Other products read the scope throughfacade.source_configs.get_analytics_alert_scope(team_id=...), never the row. The read returnsinvestigated_onlyunless the stored value isall_firing_alertsand thesignals-analytics-all-alertsflag is on for the team's organization. The flag check fails closed. The read does not checkenabled: the emitter still gates onis_source_enabled. Withall_firing_alerts, insight alert evaluation (posthog/temporal/alerts/alert_firing_signal.py) emitsanalytics/alert_firingwhen an alert that the investigation agent does not investigate starts a firing episode. It emits once per episode (source id and idempotency key<alert id>:<first check id>), and a failure never changes the check. The evaluation thread runs the gates after the check is committed. The activity awaits the emission on its event loop once that thread returns, becauseemit_signalneeds the default executor that the thread runs on. A per-team token bucket caps it, and an unavailable bucket skips the signal. Each skip after a read logsalert_firing.signal_skippedwith the team and a reason. Alerts the agent investigates keep the verdict gate in both scopes. Theposthog_insight_alerts_firing_signals_totalcounter records oneoutcomeper episode that passes the episode gates (emitted,source_disabled,scope,filtered,filter_invalid,cap,cap_unavailable,not_dispatched). It has no team label. A drop by the cap (caporcap_unavailable) also capturesalert firing signal skippedwithteam_id,alert_idandreason, which gives the per-team view for sizing the cap. A Redis key on the source id lets only the first attempt of an episode record, so an activity retry never counts it twice. A cancelled attempt records nothing. A failure to record never changes the check or the signal.alert_firinghas no config row of its own:is_source_enabledchecks theanomaly_investigationrow for it, so the one product analytics toggle turns both off. Guidance written on that row filters bothanomaly_investigationandalert_firing, through the direct steering gate. The same row acceptsconfig.alert_allow_patternsandconfig.alert_block_patterns: lists of RE2 patterns (at mostALERT_PATTERNS_MAX_COUNT, each at mostALERT_PATTERN_MAX_LENGTHcharacters) matched with search semantics against the alert name and the insight name. When allow patterns are set an alert must match one, and a block match drops the alert even when it also matches an allow pattern. The serializer rejects a pattern that does not compile, and rejects both keys on every other source. Both signal paths read them throughfacade.source_configs.get_analytics_alert_name_patterns(team_id=...)and decide with the pureevaluate_alert_filter. The read ignores the patterns while thesignals-analytics-all-alertsflag is off. It fails closed: a stored list that no longer validates, or a flag check that raises, drops every alert.alert_firingapplies the filter after the scope and episode gates and before it charges the cap, so a filtered alert spends no token and makes no guidance call, and logsfilteredorfilter_invalid. The investigation workflow applies it after the verdict gate and beforeemit_signal, with the same two reasons onanomaly_investigation.signal_skipped. The filter decides only what reaches the inbox, not alert evaluation, notifications, or whether the agent runs. - The serializer exposes a computed
statusfield:- data-import-backed sources (
github,linear,zendesk) derive status fromExternalDataSchema
- data-import-backed sources (
- The
signals_scoutsource variant pairs withsource_type=cross_source_issueand is the emission channel used by the headless Signals agent'semit_signal_*tools. It is the only(source_product, source_type)pair the agent emits today.
Constraints: Unique on (team, source_product, source_type)
SignalScoutConfig
Per-scout binding for the headless Signals agent: one row per (team, skill_name); the config is what makes the skill a scout, so a scout may carry any valid scout name. That set is narrower than the valid skill names. skill_loader.reserved_scout_name_error gates both registration paths, refusing the inbox's reserved sub-page names and any name prefix another product owns. The coordinator auto-registers a row when it discovers any live signals-scout-* skill on a participating team, canonical or hand-authored; a scout whose name lacks the prefix is registered directly instead, through scout-create or scout-config-create (the create modal always applies the prefix). Changes are activity-logged (they drive spend); team-level participation is gated by the signals-scout flag at the coordinator, not here. See backend/scout_harness/AGENTS.md for the harness internals.
| Field | Type | Description |
|---|---|---|
team |
FK → Team | Owning team (related_name="signal_scout_configs"). unique_together(team, skill_name). |
skill_name |
CharField | The scout skill this row controls, under any valid scout name. Auto-registered by the coordinator for any live signals-scout-* skill on a participating team; a scout without the prefix is registered directly instead. |
enabled |
Boolean | Per-scout switch; defaults True. Derived from status (enabled = status in (active, pending_pause)) but kept as a real column for SQL filtering and the warehouse mirror; save() reconciles the pair for writers that set only one side. |
status |
Char (choices) | Lifecycle source of truth: active / pending_pause / paused_by_system / paused_by_user. Who paused is the state itself: the system may resume its own pauses but must never touch a human's. pending_pause still runs (a warning, not a pause) and is cleared by any human config edit. |
pause_reason |
Char (nullable) | Why the system paused or warned: no_output (warn-only: silence flags a scout but never pauses it) / ignored (nobody acts on its reports: warns, then pauses) / repeated_failures. Also names the writer that owns the pause: a system writer may only move a pause carrying a reason it owns, and the inactivity sweep owns both of its reasons (transition_status_by_system()). Null outside pending_pause / paused_by_system. |
status_changed_at |
DateTime (nullable) | When status last changed. Anchors the cold-start grace window (in_cold_start_grace()): any move back to active re-anchors it, deliberately independent of the actor so the window survives account deletion. |
status_changed_by |
FK → User (nullable) | The human behind the last status change; null on system transitions, so a human re-enable and a system resume are distinguishable and a system pause never reads as a person's action. |
auto_pause_exempt |
Boolean | Opt-out from the daily inactivity sweep (scout_harness/inactivity.py, the no_output / ignored writer), for watchdog scouts whose whole job is staying quiet. Defaults False and only ever set explicitly: a human re-enable of a swept scout relies on the in_cold_start_grace re-anchor for its fresh window instead of minting permanent immunity. |
emit |
Boolean | Dry-run vs emit. Defaults True: a freshly authored scout is live from its first tick. Flip to False for dry-run — the scout runs and logs but emit_finding writes nothing — to validate it on a team before its findings reach the inbox. |
run_interval_minutes |
PositiveInteger | Minutes between runs. The coordinator dispatches rolling schedules when last_run_at is None or now - last_run_at >= run_interval_minutes. Default 1440 (daily). Validated 30 <= N <= 43200. |
run_cron_schedule |
Char (nullable) | Optional five-field cron expression anchoring runs to wall-clock slots; takes precedence over run_interval_minutes when set. Interpreted in the project's timezone so daylight-saving changes are applied automatically. Occurrences must be ≥ 30 minutes apart. Null keeps the rolling interval behavior. |
schedule_changed_at |
DateTime (nullable) | Stamped by the config serializers only when a schedule field actually changes. Anchors the cron due-check (with last_run_at / created_at) instead of updated_at, so an unrelated emit/enabled save can never defer an overdue scheduled run. |
last_run_at |
DateTime (nullable) | Stamped after each dispatch; drives the due-check. Excluded from activity logging (written every run). |
created_by |
FK → User (nullable) | Audit pointer |
enabled_by |
FK → User (nullable) | Who last flipped enabled — tracked because enablement drives spend. |
Scout rubrics
The nullable rubrics JSON field stores a scout's evaluation criteria and revision, plus its latest suggestion request and result.
Until the first save, the editor supplies enabled shared defaults for evidence, clarity, actionability, priority, instructions, and memory use.
Each criterion contains an identifier, description, passing condition, applicability, and enabled state.
These are definitions for later evaluations; saving them does not score runs or change scout execution.
The rubric editor and /api/projects/{team_id}/signals/scout/rubrics/{config_id}/ endpoints require a staff user in project 2.
The API lives in backend/presentation/scout_rubrics.py and calls backend/facade/rubrics.py; rubric persistence and generation dispatch stay in backend/scout_harness/rubrics.py.
PUT replaces the criteria only when the supplied revision matches, returning 409 for stale edits.
Its optional adopt_generation_id adopts that completed generation's governing reference context for the whole saved rubric. The generation identifier must still match, even when the rubric revision has not changed.
Ordinary criterion edits preserve the saved reference. Skill edits and later generations never update it automatically.
Every save must include all shared defaults. Owners can edit or disable them, but cannot remove them.
POST .../generate/ queues the generate-scout-rubrics Temporal workflow and returns the active request when one already exists.
It accepts optional context (up to 2,000 characters), saved on that generation request. Concurrent requests keep the first request's context; later requests do not inherit it.
The editor exposes this as a single optional paragraph. It adds priorities without replacing the scout's responsibilities, shared defaults or saved choices.
The backend captures the complete governing description, instructions, report rules and all reference files from the exact skill version before the session starts. This context is immutable within a generation and is reused if that attempt resumes.
The complete reference is attached as a session file before the sandbox starts. The initial prompt contains a short index and recent run summaries; the generator can read the full attachment in sections without truncating instructions or reference files. Historical summaries and saved criteria remain generation inputs rather than governing instructions.
The session uses the existing read-only PostHog MCP posture to inspect ordinary scout runs, transcripts, reports, notes and relevant live project data. Research cannot change the scout's responsibilities or replace its captured instructions with a newer live skill. Missing history or unavailable optional tools do not prevent instruction-based suggestions; the summary identifies material evidence limits. External MCP connections and GitHub access stay disabled, and normal internal Tasks lifecycle permissions remain unchanged.
The first request drafts complete criteria using effective defaults and disabled choices. A second request supplies the complete saved rubric and selects whole draft items by index, without rewriting them.
One conditional format correction is shared across both steps. Selection and format correction do no additional research.
Criteria explain the required result in plain language, with specific source references where needed to preserve complex conditions and exceptions. Later evaluations must include those reference instructions alongside the rubric.
The generation prompt ends with writing guidance and a short example for the scout's owner. The selection step also writes an owner-facing summary; it cannot change the selected criteria.
Its task identifiers, status, and validated result persist on the config so the user can leave the page and return later.
Completion preserves saved criteria, rejects results from replaced requests, and requires explicit user selection and saving to adopt suggestions.
The saved reference context and its generation identifier survive replacement of the latest suggestions. Legacy rubrics without a complete captured reference remain readable; reading or editing them does not invent provenance. Owners must regenerate suggestions and explicitly adopt the complete reference before starting trials, but can keep their existing criteria.
Expired or terminal requests reject late worker updates. An update to an expired request records the failure and completion time.
A request that cannot start, because of the daily limit or a dispatch failure, restores the last completed suggestions.
SignalScoutRun
Thin bridge from a Tasks TaskRun to the scout skill that ran inside it: one scout-domain row per scheduled agent run that links its TaskRun to the skill it executed. Status, timing, error context, and the full chat log live on the TaskRun; emitted findings are Signal / SignalReport rows written by emit_signal(). This row carries only the scout-specific fields that need to be queryable as real columns.
| Field | Type | Description |
|---|---|---|
team |
FK → Team | Owning team (related_name="signal_scout_runs"). Denormalised tenant boundary; canonical value is task_run.task.team. |
task_run |
OneToOne → tasks.TaskRun | The TaskRun the scout span ran inside (related_name="signal_scout_run", CASCADE — bridge row goes when the TaskRun is purged). |
scout_config |
FK → SignalScoutConfig (SET_NULL) | Audit pointer; SET_NULL so deleting and recreating a config doesn't destroy run history. |
skill_name |
CharField(200) | The scout skill the run executed. |
skill_version |
Int | The LLMSkill.version snapshot at run start. |
summary |
TextField | One-paragraph close-out the agent writes at end-of-run. Searchable via ILIKE on the list endpoint so future runs can dedupe even when no Signal row was emitted. |
metadata |
JSONField | Server-written run context in two regions: runner-stamped top-level keys (always harness_prompt_version / report_channel / skill_origin; plus model / runtime_adapter / reasoning_effort when routing overrode the default, plus triggered_by when the run was not dispatched by the schedule) and the nested derived map of harness-computed booleans written at finalize. See below. |
created_at |
DateTime | Auto-set on creation. |
Status, timing, and chat log live on the linked TaskRun. The bridge row carries no status / started_at / completed_at / findings / run_metrics of its own — those moved to tasks.TaskRun so the LLM-analytics token / cost roll-up, the Tasks UI, and the harness all see one canonical record.
Run provenance is stamped, not re-derived. harness_prompt_version (a hash of scout_harness/prompt.py plus the imported constants its templates render, identifying the prompt build), report_channel (none/emit/edit/both, since the prompt branches on the two report capabilities separately), skill_origin, and github_guidance are written at run creation because none can be reconstructed later: the harness prompt carries no version history, allowed_tools is editable, and a seeded canonical skill row becomes custom as soon as a team edits it. Together they let a model or prompt A/B compare only runs that were given the same prompt shape. All four ride the signals_scout_run_started / _finished events too, which is where that readout happens. triggered_by is stamped the same way but only when it is not schedule, so absence reads as the default path; the workflow trigger's cooldown reads it back to tell its own prior fires apart from scheduled patrols and human "Run now" presses.
Run metadata is derived, never self-reported. metadata.derived answers "what kind of run was this?" (has_emit_report, has_edit_report, has_self_improvement, has_chart, has_self_validation, has_structured_output, has_not_in_use_closeout) and is computed server-side at finalize by scout_harness/derived_metadata.py from the run's own settled output. Asking the scout to declare these instead would make each flag only as reliable as the model remembering to write it, so the column stays queryable precisely because nothing in it is agent-authored. An absent derived region is unknown rather than all-false: the run predates the field, never reached finalize, or its stamp failed (best-effort, logged). MultiTurnSession owns the TaskRun lifecycle; the on_task_run_created hook attaches the TaskRun to the bridge row before the agent's first turn.
Tasks UI cross-link. Run serializers expose a computed task_url field on scout-runs-list and scout-runs-retrieve MCP responses — /project/{team_id}/tasks/{task_run.task_id}?runId={task_run_id}. task_url is null for rows whose task_run link is missing (rows aborted before MultiTurnSession.start() returned).
Indexes: (team, skill_name).
Constraints: None at the DB level today. Single-flight is a best-effort app-layer guard (_has_running_run against task_run.status = IN_PROGRESS). An earlier partial unique constraint on (team, skill_name) WHERE status='running' was dropped together with the bridge model's own status column; a task_run.status-based constraint plus active recovery of stranded IN_PROGRESS task runs (_self_heal_stale_runs is a no-op today) is a tracked follow-up.
SignalScratchpad
Narrow per-team scratchpad surface the scout fleet writes during runs and reads back on future runs (known issues, false positives, dedupe fingerprints, learned team quirks). Distinct from SignalProjectProfile: profile is deterministic ground truth, scratchpad is the scout's inferred learnings (possibly wrong). MCP-readable across agents so PostHog AI and other scouts can see what the fleet has learned about a team.
Both report-pipeline stages write here too, under the pipeline:implementation and pipeline:report-research identities: the implementation run keys on the repository it changed, the research run on the entities a report names. See "Fleet memory the implementation run writes back" and "Fleet memory the research run writes back" below for the protocols and the scope that gates them. One keyspace, many writers: any agent on the team can overwrite any key, which is why every prompt that writes says to search a key and condense it rather than overwrite it blind, and why every reader treats entries as untrusted input.
| Field | Type | Description |
|---|---|---|
team |
FK → Team | Owning team (related_name="signal_scratchpads") |
key |
CharField(300) | Semantic key, agent-chosen; unique per team |
content |
TextField | Prose for prompt injection — the agent reads this verbatim |
created_by_run |
FK → SignalScoutRun (SET_NULL) | The run that wrote this entry; SET_NULL so deleting a run row doesn't destroy the entry |
expires_at |
DateTime (nullable) | Optional TTL — null (the default) is durable; past expiry drops the row from search_scratchpad |
created_at |
DateTime | Auto-set on creation |
updated_at |
DateTime | Auto-set on save |
Constraints: Unique on (team, key).
authority and tags (with their backing GIN index) were dropped in the PR2 review simplification — retrieval is now plain ILIKE over key + content.
expires_at was dropped in that same pass and later restored: a scout that wrote a time-boxed memory almost never came back to forget it, so stale entries kept loading into run prompts. Entries stay durable by default. Expiry first hides a row from search_scratchpad (include_expired=True is the human audit path); within its post-expiry grace the key stays taken and both forget and the remember upsert still find it. The daily prune_expired_scratchpad_entries janitor then hard-deletes rows whose expiry is more than SCRATCHPAD_EXPIRY_GRACE_DAYS in the past, so lapsed entries cannot pile up forever; durable (expires_at-null) rows are never swept. remember writes the whole entry, so a write that omits expires_at clears an expiry an earlier write set. A followup: key may not carry one — remember rejects that, since an expired follow-up would vanish from the self-validation queue before the scout could work it.
SignalProjectProfile
Deterministic snapshot of "what's true about this project" — the agent's orientation surface. Time-series so future phases can diff a new profile against the previous row to populate payload.deltas. Computed by scout_harness/profile/builders.py from authoritative tables; v1 writes 10 inventory sections (events, properties, cohorts, feature flags, experiments, surveys, dashboards, insights, data warehouse sources, integrations).
| Field | Type | Description |
|---|---|---|
team |
FK → Team | Owning team (related_name="signal_project_profiles") |
computed_at |
DateTime | Auto-set on creation |
expires_at |
DateTime | Soft TTL — get_project_profile treats rows past expiry as cache misses and recomputes (~36h gives a safety margin against the daily refresh). |
source_version |
CharField(40) | Bumps when the inventory schema changes meaningfully so get_project_profile can invalidate stale rows without a manual backfill |
payload |
JSONField | {inventory: {...}} in v1; deltas, activity_notes, narrative slots reserved for later phases. Inline jsonb is fine — even a rich profile is small. |
Indexes: (team, -computed_at) — supports the ORDER BY computed_at DESC LIMIT 1 lookup used by get_project_profile.
SignalEmissionRecord
Tracks which source records have been emitted as signals. Owned by the signals app so source models (e.g. Ticket) stay decoupled. One row per source record, upserted on emission.
| Field | Type | Description |
|---|---|---|
team |
FK → Team | Owning team |
source_product |
CharField(100) | Mirror of SignalSourceConfig.SourceProduct |
source_type |
CharField(100) | Mirror of SignalSourceConfig.SourceType |
source_id |
CharField(200) | Source-side primary key |
emitted_at |
DateTime | When this record was last emitted |
Constraints: Unique on (team, source_product, source_type, source_id)
Indexes: (team, source_product, source_type)
ClickHouse Storage
Signals are stored in the posthog_document_embeddings table, which is shared across products (error tracking, session replay, AI observability, etc.).
Table Schema
Defined in products/error_tracking/backend/embedding.py:
| Column | Type | Description |
|---|---|---|
team_id |
Int64 | Team identifier |
product |
LowCardinality(String) | Product bucket — signals uses 'signals' |
document_type |
LowCardinality(String) | Document type — 'signal' for signals, 'report' for report documents (see below) |
model_name |
LowCardinality(String) | Embedding model name (e.g., text-embedding-3-small-1536) |
rendering |
LowCardinality(String) | How content was rendered — signals use 'plain', report documents use 'title_summary_v1' or 'title_v1' |
document_id |
String | Unique signal ID (UUID) |
timestamp |
DateTime64(3, 'UTC') | Document creation time |
inserted_at |
DateTime64(3, 'UTC') | When the embedding was inserted (used for dedup) |
content |
String | The signal description text |
metadata |
String | JSON string containing report_id, source_product, source_type, source_id, weight, extra, match_metadata, deleted |
embedding |
Array(Float64) | The embedding vector |
Engine: ReplacingMergeTree (sharded), partitioned by toMonday(timestamp), 3-month TTL.
Ordering: (team_id, toDate(timestamp), product, document_type, model_name, rendering, cityHash64(document_id))
Data Flow
emit_embedding_request() → Kafka (document_embeddings_input topic)
→ Kafka table → Materialized View → Writable Distributed table → Sharded ReplacingMergeTree
Report Documents
Alongside the per-signal rows, each SignalReport is embedded under document_type = 'report' with document_id = the report UUID.
A report holds one row per rendering of its text, all on that same document_id:
| Rendering | Content | Emitted when | Snapshotted by the dataset dag |
|---|---|---|---|
title_summary_v1 |
Title and summary, joined by a blank line | Either field changes | Yes |
title_v1 |
Title alone | The title changes | No |
EMBEDDING_RENDERINGS in backend/report_embeddings.py is the full list. Whoever adds a rendering adds it there, because the retraction path walks it.
The title-only rendering exists because the title is the only text the inbox shows before someone opens a report,
so the ranking model can measure what the summary adds to a prediction of an open instead of reading one vector that mixes the two.
A report with a summary and no title yet holds only the composed document, since there is no title to embed.
These vectors are the feature-side building block for the inbox ranking model,
whose label stream is the signal_report_status_changed event emitted by backend/receivers.py.
The dataset dag selects title_summary_v1 alone (dags/inbox_ranking/dataset/dag.py); title_v1 gets its own snapshot when the model is ready to compare the two.
Emission lives in backend/report_embeddings.py,
driven by a post_save receiver that fires whenever a report's title or summary actually changes.
That covers the matcher writing text at creation, the summary workflow on IN_PROGRESS -> READY, re-research runs, and the scout channel's update_authored_content.
The unchanged-text shortcut is applied per rendering, so a summary-only edit re-embeds the composed document and skips the title vector it would rewrite identically.
Coverage of a newly added rendering is forward-only: there is no backfill, so an existing report gains its title_v1 row when its title next changes.
Two properties are load-bearing:
- The row's
timestampis pinned to the report'screated_at, never the emission time. The table partitions bytoMonday(timestamp)and orders bytoDate(timestamp), so a re-emission stamped "now" would land in a different partition and sit alongside the earlier row rather than superseding it. The trade-off is that the 3-month TTL runs from report creation, so a report open longer than that loses its vector while still live. renderingis versioned rather than'plain', because a report document is a composition of fields we expect to extend. It is part of the table'sORDER BY, so several compositions sit in the table at once and can be compared instead of one silently replacing another.title_v1andtitle_summary_v1already rely on that, and a later v2 of either works the same way.
Metadata is deliberately limited to report_id plus the deleted tombstone flag.
It only refreshes when the report's text changes or the report is deleted, so mutable state (status, priority, signal_count) would go stale there.
Join that from Postgres or the status-change event stream instead.
Safety gating and retraction
A report's title and summary are derived from its signals, so they inherit the same prompt-injection exposure.
Emission is therefore gated on the durable safety_judgment artefact: when the latest verdict is choice: false, no vector is written.
That read is pinned to the writer (using("default")), because it runs immediately after the transaction that wrote the verdict and ReplicaRouter documents replication lag on exactly that pattern.
Withholding new emissions is not sufficient on its own, because a report can be embedded while safe and only later be judged unsafe.
Three paths therefore retract a report's vectors by re-emitting the rows with metadata.deleted = true, preserving created_at so each one replaces the live row in the same partition:
- Deletion: the report-level counterpart to
soft_delete_report_signals. Both the soft path, wheredelete_report_activityflips status toDELETED, and a harddelete()of the row, which is whatdelete_team_reports_activityand thecleanup_signalscommand issue. The hard path matters most: once the row is gone, no later write can retract the vector. - A later unsafe verdict: the summary workflow re-judges safety on every run, and a READY report re-researches whenever new signals join it.
- An unreviewed edit: the
PATCHendpoint and the scoutedit_reportchannel supply text the judge has never seen, so the report is retracted and left unindexed until the pipeline writes judged text again.
emit_report_tombstone writes one tombstone per rendering in EMBEDDING_RENDERINGS, not one per report.
rendering is part of the ReplacingMergeTree key, so a tombstone retracts only its own rendering and leaves every other rendering's content live until the 3-month TTL expires it.
A rendering that is emitted but missing from EMBEDDING_RENDERINGS therefore keeps unsafe text indexed.
These retraction paths use the full list by default, so callers do not need to select each rendering.
When a text edit removes a rendering, the receiver passes only the removed rendering to emit_report_tombstone.
Clearing both fields retracts both renderings.
Tombstones carry fixed placeholder content (TOMBSTONE_CONTENT) rather than the report's own text.
Content is not part of the ReplacingMergeTree key, so a placeholder supersedes a live row just as well, and it means a tombstone can be emitted without first knowing whether a live row exists, which is the question none of these paths can answer cheaply.
It also guarantees a retraction can never introduce the very text the safety judge withheld.
This is a deliberate divergence from soft_delete_report_signals, which re-emits signal text verbatim and so makes rows filterable rather than erased.
Readers must filter with NOT JSONExtractBool(metadata, 'deleted'), as every existing signals query already does.
A caller removing reports in bulk should delete the reports and let their artefacts cascade, rather than deleting artefacts first.
Django reports the cascade as the artefact's deletion origin, which is what lets the verdict receiver skip reconciliation it does not need:
the report is going away in the same operation and retracts its own embedding.
Deleting artefacts directly costs two extra queries each, which delete_team_reports_activity cannot afford inside its five minute budget.
Soft Deletion
Signals are soft-deleted by re-emitting the embedding row with metadata.deleted = true, preserving the original timestamp so it lands in the same partition and replaces the original via ReplacingMergeTree. Most read queries filter with NOT JSONExtractBool(metadata, 'deleted'); the exception is wait_for_signal_in_clickhouse_activity, which deliberately includes deleted rows (it only cares that the row landed, not whether it's visible).
HogQL Queries
The canonical ClickHouse / HogQL helpers live in backend/temporal/signal_queries.py and use the HogQL alias document_embeddings. Most read queries share a dedup subquery that uses argMax(..., inserted_at) grouped by document_id, which makes reads stable despite ReplacingMergeTree merge timing.
All Signals queries filter to product = 'signals' and document_type = 'signal'. Key queries:
- Fetch signal type examples (
fetch_signal_type_examples_activity): fetches one example signal per unique(source_product, source_type)pair from the last month, selecting the most recent example per type. Used to give the search-query generation LLM context about heterogeneous signal types. - Semantic search (
run_signal_semantic_search_activity): usescosineDistance(embedding, {embedding})to find nearest neighbors with a non-emptyreport_id, limited to the last month. - Fetch for report (
fetch_signals_for_report_activity): fetches all non-deleted signals for a report, ordered by timestamp ascending. - Wait for ClickHouse (
wait_for_signal_in_clickhouse_activity): a two-tier poll tuned byWaitForClickHouseMode. Every attempt first checks the embedding worker's recently-seen store (a cheap key-value lookup overposthog.api.embedding_worker.async_get_recently_seen_documents). When a document already exists in ClickHouse, its cachedemitted_atmust be later than that document's latestinserted_at; newly created documents only require a cache hit. Once the store confirms every signal, the wait either ends outright (optimistic— grouping batches that promoted no report, and report deletion) or runs the ClickHouse confirmation query (ch_confirmed— reprocessing, and grouping batches that promoted a report, since the summary workflow spawned right after reads those rows from ClickHouse). For the first 5 minutes the wait is otherwise store-exclusive; after that ClickHouse also polls on every 3rd attempt (a third of the store's rate) as a fallback, since the store is best-effort. The ClickHouse query counts emitteddocument_ids within a widened timestamp range (min(timestamp)-2mtomax(timestamp)+2m) andinserted_at >= now() - 30 minutes, which avoids matching stale earlier emissions of the same IDs while tolerating precision loss and queueing delay. This query intentionally does not filter ondeleted. - Filter reports by source product (
fetch_report_ids_for_source_products): used by the list API’ssource_productfilter. Note that it currently has a hardLIMIT 300.
execute_hogql_query_with_retry() in backend/temporal/clickhouse.py wraps transient ClickHouse failures with heartbeat-safe retry behavior for activities.
Soft Deletion Caveat
soft_delete_report_signals() lives in backend/temporal/signal_queries.py (with a compatibility re-export in backend/utils.py), not in backend/api.py.
It soft-deletes report signals by re-emitting the same document_id and original timestamp with metadata.deleted = true, across all embedding models, so the new row replaces the original in the same ReplacingMergeTree partition.
One implementation caveat: the helper currently fetches report signals with limit=5000. So while the intent is “delete all signals for the report”, very large reports can currently leave undeleted rows beyond that cap.
API Layer
Entry Point: emit_signal() (backend/api.py)
The primary programmatic entry point. Called by other PostHog products and Signals-related workflows to emit signals.
Current behavior:
- Verifies org-level AI data processing approval
- Checks
SignalSourceConfig.is_source_enabled() - Validates the payload against
posthog.schema.SignalInput - Idempotently starts
BufferSignalsWorkflow - Fire-and-forget starts
SignalEmitterWorkflow
So emit_signal() now feeds the buffer + emitter + grouping v2 path rather than signaling the legacy v1 grouping workflow directly.
Guards / validation:
- The team’s organization must have
is_ai_data_processing_approved - The source must be enabled by
SignalSourceConfig.is_source_enabled() - Description length is capped at
MAX_SIGNAL_DESCRIPTION_TOKENS(currently 8000 tokens) - The input must validate against the shared
SignalInputschema
Utility: soft_delete_report_signals()
The canonical helper lives in backend/temporal/signal_queries.py (with a compatibility re-export in backend/utils.py), not in backend/api.py.
It soft-deletes report signals by re-emitting them with metadata.deleted = true while preserving original timestamps so rows replace originals via ReplacingMergeTree. It intentionally includes already-deleted rows for idempotency, but currently fetches at most 5000 signals per report.
REST Endpoints
All signals endpoints live under the projects/:team_id/signals/ path, registered on the projects_router in posthog/api/__init__.py.
SignalViewSet (DEBUG only)
| Method | Path | Description |
|---|---|---|
| POST | signals/emit/ |
Manually emit a signal (debug/testing only) |
InternalSignalViewSet
Internal-only service-to-service endpoint authenticated via X-Internal-Api-Secret.
| Method | Path | Description |
|---|---|---|
| POST | /api/projects/{team_id}/internal/signals/emit |
Emit a signal for a specific team |
SignalSourceConfigViewSet
Full CRUD for per-team signal source configurations. Uses IsAuthenticated + APIScopePermission with scope object task.
| Method | Path | Description |
|---|---|---|
| GET | signals/source_configs/ |
List configs for the team |
| POST | signals/source_configs/ |
Create a new config |
| GET | signals/source_configs/{id}/ |
Retrieve a config |
| PATCH | signals/source_configs/{id}/ |
Update a config |
| DELETE | signals/source_configs/{id}/ |
Delete a config |
Important side effects:
- Creating an enabled
error_tracking / issue_createdconfig starts the error-tracking backfill workflow - Enabling data-import-backed sources can trigger external data syncs
SignalTeamConfigViewSet
Team-scoped singleton config for the default autonomy priority threshold. Uses IsAuthenticated + APIScopePermission (scope: task). Returns 404 if no config exists for the team.
| Method | Path | Description |
|---|---|---|
| GET | signals/config/ |
Retrieve the team's SignalTeamConfig |
| POST | signals/config/ |
Update default_autostart_priority |
User Autonomy Config (action on UserViewSet)
Per-user autonomy opt-in, registered as an action on the root UserViewSet at api/users/:uuid/signal_autonomy/.
| Method | Path | Description |
|---|---|---|
| GET | users/@me/signal_autonomy/ |
Get own autonomy config (404 if not opted in) |
| PUT | users/@me/signal_autonomy/ |
Opt in / update autostart_priority |
| DELETE | users/@me/signal_autonomy/ |
Opt out (deletes the SignalUserAutonomyConfig row) |
Notes:
- Non-staff users can only access
@me; staff can access any user by UUID PUTbody:{"autostart_priority": "P2"}or{"autostart_priority": null}(use team default)- Inherits auth/permissions from
UserViewSet(scope:user)
SignalReportViewSet
Read + delete + state transitions. Uses IsAuthenticated + APIScopePermission (scope: task). Composed from RetrieveModelMixin, ListModelMixin, DestroyModelMixin, and GenericViewSet. Deleted reports are excluded from all endpoints via safely_get_queryset.
| Method | Path | Description |
|---|---|---|
| GET | signals/reports/ |
List reports. Excludes deleted always and excludes suppressed by default. Supports ?status=, ?search=, ?source_product=, ?suggested_reviewers=, ?task_id= (resolved through task_run artefacts), and ?ordering=. Use ?count_only=true for the filtered total without report rows or metadata enrichment. |
| GET | signals/reports/{id}/ |
Retrieve a single report |
| DELETE | signals/reports/{id}/ |
Soft-delete a report and its signals. Starts SignalReportDeletionWorkflow. On success returns 202. If the workflow is already running, returns 200 {"status": "already_running"}. The API immediately transitions the Postgres report to deleted to hide it from list results while ClickHouse cleanup runs asynchronously. |
| POST | signals/reports/{id}/state/ |
Transition report state. Body: { "state": "suppressed" | "potential" | "resolved", ...transition_to kwargs }. suppressed, potential, and resolved are exposed via API (resolved for user-driven "work is done"; the model still gates which statuses may resolve). Returns 409 on invalid transitions and 400 on invalid arguments. |
| POST | signals/reports/{id}/merge/ |
Merge duplicate reports into this one. Body: { "source_report_ids": [...], "reason": "..." }. One transaction moves the sources' signals, log artefacts, pull requests, task runs and checks onto the survivor, then archives each source with reason merged and a duplicate_of link back. See report_merge.py. |
| POST | signals/reports/{id}/reingest/ |
Delete a report and re-ingest its signals. Starts SignalReportReingestionWorkflow. On success returns 202. If already running, returns 200 {"status": "already_running"}. Same team access as other report endpoints; personal API keys need inbox:write. |
| GET | signals/reports/{id}/artefacts/ |
List all artefacts for a report, ordered by -created_at |
| GET | signals/reports/{id}/signals/ |
Fetch all signals for a report from ClickHouse, including full metadata |
| GET | signals/reports/available_reviewers/ |
List available suggested reviewers for the team |
Ordering: Configurable via ?ordering= with comma-separated fields. Supported fields: status, is_suggested_reviewer, signal_count, total_weight, priority, created_at, updated_at, id.
The status clause sorts by annotated pipeline_status_rank (not lexicographic status text). Ascending rank lists earlier pipeline stages first. Ranks:
0—readyand actionable (includes reports with no actionability judgment yet)1—readyand latest actionability judgment isnot_actionable2—pending_input3—in_progress4—candidate5—potential6—failed7—suppressed8—deleted(deleted rows are not returned by the API; rank exists for queryset consistency)
So with ordering=status, failed sorts after actionable ready. With ordering=-status, failed sorts before actionable ready.
Default ordering is -is_suggested_reviewer,status,-updated_at,id.
Dismissal feedback. dismissal_reason / dismissal_note on the state and bulk-state bodies persist as a stacking dismissal artefact on the report, which stays the record of truth.
wrong_assignee records a routing error, while wontfix_irrelevant records that the report is not useful to the project; neither says the report analysis is incorrect.
When the caller typed a note, that feedback is also forwarded to a SignalScoutNote (dismissal_notes.forward_dismissal_note).
The artefact alone only reaches a scout if some later run happens to search the inbox and land on that report; notes are read by name at the start of every run, which is where a "this was noise, and here is why" verdict belongs.
"Forward" rather than "promote" because promotion in this product means the pipeline moving a report up the status ladder to candidate (promoted_at, signals_at_run, re-promotion), and nothing here changes a report's standing.
Forwarding targets the scout that authored the report (resolved in Postgres from SignalScoutRun.emitted_report_ids, then edited_report_ids) and addresses the whole fleet when no run claims it, and it writes one note per targeted scout per request, so a bulk dismissal of 40 reports under one note does not write 40 notes.
Each note describes the status the report actually landed in, not the requested target, so a restore out of the archive is never reported to a scout as a snooze.
A report that ends up resolved is never forwarded: resolving says the report did its job, so the note on it records how the work landed rather than whether filing it was worth it, and this channel is for the latter.
Derived notes carry origin=report_dismissal and a 30-day TTL so they cannot permanently crowd out deliberate human steering, and a failure to write one never fails the transition the caller asked for.
Because the notes table is otherwise gated to skill-authoring authorization (scouts read note content verbatim while holding privileged sandbox tools), forwarding re-checks that the dismisser could have left the note by hand: canonical-project access, a token whose scoped_teams cover that project, and the llm_skill editor level, mirroring SignalScoutNoteViewSet (dismissal_notes._may_steer_scouts).
In practice the llm_skill leg only bites in orgs with the access-control entitlement where an admin has explicitly restricted skill editing, since default_access_level grants editor otherwise; the token scoped_teams and child-environment legs are what constrain the common case.
A dismisser who can't clear that bar still gets their feedback recorded on the report; it just doesn't enter the steering channel.
Two audience rules follow from a note living on the canonical project while the report it quotes may not.
Reports on a child environment are never forwarded, since the note would be readable by people with no access to that environment.
And on the read side, scout-notes-list withholds report_dismissal notes from callers who can't read reports (task scope plus task RBAC), so the notes surface can't be used to read report ids, titles, and dismissal text around the reports API (scout_harness/views._may_read_reports).
Reviewer corrections. A human adding or removing a suggested reviewer is also forwarded to a SignalScoutNote (reviewer_correction_notes.forward_reviewer_correction_note), under the same guard as the suggested_reviewers_changed activity row: a user-attributed edit that changed the set, so agent writes and order-only rewrites forward nothing.
Scouts treat a correction as authoritative routing precedent and cache it in reviewer: scratchpad keys, and until this channel existed nothing told them one had happened — the project profile's recent_reviewer_corrections covers about two days of edits on a busy project, so a login removed by hand kept being suggested by every scout holding it.
The note is the trigger, not the record: it asks the scout to condense its own memory, and nothing here writes to the scratchpad, because a mechanical entry would overwrite ownership evidence a scout verified for itself.
It is the only derived origin addressed to more than one scout. Every live scout that emitted or edited the report is a target, because a scout that set the reviewers through an edit did the routing the human just corrected. A removed login also goes to every scout whose reviewer: memory names it, resolved by one exact-term search under the reviewer: key prefix and keeping the live scout skills behind those entries. The targets are ordered emitting scout, editing scouts, then holders, and capped at MAX_NOTE_TARGETS per correction.
A report no live scout touched falls back to the fleet-wide target only when no holder resolved, because a run reads the fleet-wide notes alongside its own and would otherwise hear the same edit twice.
An impersonated edit forwards nothing: the reviewer-corrections project profile already excludes impersonated activity rows, since a support-staff edit is not the team's ownership evidence.
Logins are shape-checked against GitHub's login rule before rendering, and an edit contributes at most MAX_CORRECTION_LOGINS of them: both write paths into the reviewers artefact accept any string of any length, and the value lands inside a backtick span in a prompt every scout reads.
The note names the editor, because who corrected the routing is what tells a scout whether a domain owner spoke or somebody trimmed a list in passing; the name is normalized to one line, capped, and kept out of the backtick spans the suppression parser reads back, and an account with no name falls back to a neutral word.
Repeated edits are coalesced rather than queued: a login already named in a note to a target, in the same direction, inside SUPPRESSION_WINDOW is left out of the next one, so one person trimming the same login off ten reports in a morning tells each scout once.
Suppression is per direction, so an edit that reverses an earlier one inside the window — a login added and then removed — still forwards, since that is exactly the stale-routing correction the channel exists to carry.
Authorization, the 30-day TTL, the child-environment rule, and the read-side withholding are the dismissal ones (dismissal_notes.principal_may_steer_scouts); the key scopes aren't demanded on top, because the logins already reach scouts through the report's reviewers artefact and the profile.
Forwarding runs after commit and is best-effort, and signals_suggested_reviewers_resolved carries correction_notes_written / correction_note_targets so what it achieved is measurable next to the edit itself.
Removal reasons. A person who removes a reviewer through the reviewers PUT remove action can say why with removal_reason (wrong_area, already_covered, unavailable, other) and an optional removal_note.
The reason rides on the suggested_reviewers_changed activity row as extra changes, with removal_reason_reviewers naming whom it covers, so recent_reviewer_corrections hands both to research and the scouts. Only the people the editor chose to remove carry the reason: a reviewer that member validation drops in the same write gets neither the reason nor a correction note.
It also rides on the forwarded correction note, where it replaces the generic caution with guidance for that reason.
Each reason maps to its own guidance in reviewer_correction_notes._REASON_GUIDANCE. wrong_area steers similar work away from the person and marks their older history in that area as stale. already_covered and unavailable are never ownership evidence. The list stays short on purpose: wrong_area covers "moved off it" and "someone else owns it", and unavailable covers being away and having too many reports.
The note is quoted line by line, so it cannot pose as a section that the suppression parser reads back.
The note goes only to the forwarded scout note, and only when its author passes principal_may_steer_scouts, the gate a hand-written scout note passes. It stays out of the activity row and the profile, because activity-log readers need no report access and a note can quote dismissal text. The reason code is a fixed choice and needs no gate.
The web dismiss dialog no longer offers wrong_assignee: a removal with a reason reaches research, and a dismissal closes the report for everyone. The state API still accepts the code from older clients.
A wrong_assignee dismissal from an older client or an agent also takes the caller off the reviewers with wrong_area and the dismissal note, because research reads the reviewer corrections and never the dismissal note.
A bulk dismissal processes only the reports that name the caller, and it skips the per-report correction notes, because the dismissal already sends each scout one note for the batch.
Wrong-repo dismissals. The wrong_repo reason code carries an optional corrected_repository (owner/repo, refused with any other reason).
The next matching signal starts a fresh report linked back by recurrence_of, while the original stays dismissed.
Further matching signals follow that link to the same successor, which uses the normal weight threshold for research promotion.
Repository selection runs afresh with the project's correction feedback; the successor does not reuse the dismissed report's selection or treat its findings as evidence of a previous fix.
If selection returns the repository rejected by the parent's latest dismissal, the successor requests repository input before research, including when that repository is the only eligible candidate. This check applies only to the linked successor, so unrelated reports can still use that repository.
The successor does not copy corrected_repository into its selection; the correction informs normal selection, which checks current eligibility and source visibility.
The dismissal artefact denormalizes selected_repository (the report's latest repo_selection at dismissal time) next to the correction, so a selection mistake is one self-contained record.
When the corrected repository is currently connected to the team, the state action also appends it as a user-attributed repo_selection artefact, latest-wins.
Restoring the report does not itself re-research; the appended selection means the report's next research run, once new signals re-promote it, targets the corrected repository instead of repeating the rejected pick.
The appended selection carries autostart_eligible=False: a correction names the repo a person would target, never PR intent, so it must not outrank an earlier selection's False stamp on the autostart path.
A wrong-repo dismissal without a usable correction (none named, or the named repository is not connected through the team's own GitHub integration, which research clones through) instead clears the selection: a user-attributed repo_selection with repository=None and autostart_eligible=False, so research cannot reuse the rejected pick and the report's next run re-selects with the corrections block in its prompt.
The clear is skipped when the report has no reusable selection to begin with.
These records feed back into selection: repo_corrections.wrong_repo_corrections_block renders the team's recent wrong-repo dismissals (newest first, one per report and one per distinct selected→corrected lesson, so a bulk dismissal cannot flood the block; capped) and select_repository_for_team passes the block into the selection agent's prompt on every signals-side selection — see Repository Selection.
SignalReportArtefactViewSet
The artefact log read/write surface, nested under the reports router (environment_signal_report_artefacts). Team-scoped via safely_get_queryset (report id from the URL + self.team); gated by scope_object = "task".
| Method | Path | Description |
|---|---|---|
| GET | signals/reports/{id}/artefacts/ |
List all artefacts for a report, ordered by -created_at |
| GET | signals/reports/{id}/artefacts/{aid}/ |
Retrieve one artefact |
| POST | signals/reports/{id}/artefacts/ |
Append an artefact of any type (status types append a new latest-wins row); per-type schema validation |
| PATCH | signals/reports/{id}/artefacts/{aid}/ |
Edit a log/status row's content in place |
| DELETE | signals/reports/{id}/artefacts/{aid}/ |
Delete an artefact row |
| GET | signals/reports/{id}/artefacts/{aid}/diff/ |
Render a commit artefact's branch against the repo default branch via GitHubIntegration.get_diff |
POSTing a task_run artefact is the "associate me" operation: content.task_id defaults from the X-PostHog-Task-Id header, product/type default to tasks/agent_run, the named task must belong to the team, and re-association is idempotent. The bespoke suggested_reviewers PUT (reviewer enrichment) also lives here. Writes are attributed to the header's task when present, else the requesting user.
SignalProcessingViewSet
View + control API for the v2 grouping pipeline. Uses scope object INTERNAL.
| Method | Path | Description |
|---|---|---|
| GET | signals/processing/ |
Return current pause state |
| PUT | signals/processing/pause/ |
Pause grouping until a given timestamp |
| DELETE | signals/processing/pause/ |
Clear the paused state |
Signals Agent endpoints (backend/scout_harness/views.py)
The harness exposes three viewsets routed under environment_signals_scout_* basenames in posthog/api/__init__.py. They are surfaced to MCP callers as scout-* tools via products/signals/mcp/tools.yaml. Reads are scoped to the team; writes (scratchpad remember / forget, signal emit) require the matching MCP scope.
SignalScoutRunViewSet— list / retrieve scout run rows; nested actionruns/{id}/emit-signal/for the harness to push findings during a run.SignalScratchpadViewSet— search / remember / forgetSignalScratchpadrows for the team. Thescout-scratchpad-searchtool is the agent's primary "what do I already know" read at prompt-assembly time.SignalProjectProfileViewSet—GET .../current/returns the freshest non-expiredSignalProjectProfilerow for the team (recomputes if the cache is stale).
Generated MCP tool names:
| Tool | Purpose |
|---|---|
scout-runs-list |
List scout runs (filterable by skill / status / time) |
scout-runs-retrieve |
Fetch a single run row including the full findings payload |
scout-emit-signal |
Push a finding from inside a run (used by the harness's emit_signal_* tools) |
scout-scratchpad-search |
Search durable scratchpad entries for the team |
scout-scratchpad-remember |
Create or update a scratchpad entry |
scout-scratchpad-forget |
Remove a scratchpad entry |
scout-project-profile-get |
Read the current SignalProjectProfile snapshot |
Serializers (backend/serializers.py)
SignalSourceConfigSerializer- Exposes
id,source_product,source_type,enabled,config,created_at,updated_at,status - Validates that
recording_filtersin config is a JSON object forsession_replay - Computes
statusfrom external data import state for data-import-backed sources
- Exposes
SignalTeamConfigSerializer- ModelSerializer for
SignalTeamConfig - Exposes
id,default_autostart_priority,created_at,updated_at
- ModelSerializer for
SignalUserAutonomyConfigSerializer- ModelSerializer for
SignalUserAutonomyConfig - Exposes
id,user(nested withid,uuid,first_name,last_name,email),autostart_priority,created_at,updated_at
- ModelSerializer for
SignalUserAutonomyConfigCreateSerializer- Plain Serializer for PUT requests on the user autonomy action
- Accepts
autostart_priority(optional, nullable)
SignalReportSerializer- Exposes
id,title,summary,status,total_weight,signal_count,signals_at_run,created_at,updated_at,artefact_count,priority,actionability,already_addressed,is_suggested_reviewer prioritycomes from the latestPRIORITY_JUDGMENTartefactactionabilitycomes from the latestACTIONABILITY_JUDGMENTartefact and supports both current (actionability) and legacy (choice) payloadsalready_addressedalso comes from the latest actionability artefactis_suggested_reviewer: list/detail annotate from the requesting user’s linked GitHub login against thesuggested_reviewersartefact. Alwaysfalsewhen there is nothing to review:failedreports, orreadywith latest actionabilitynot_actionable(even if the artefact names the user)
- Exposes
SignalReportArtefactSerializer- Exposes
id,type,content,created_at - Parses JSON text into structured content
- Exposes
- For
suggested_reviewers, enriches the stored payload with fresh PostHog org-member data, the source skill, a source label, and a concise explanation. Scout labels use the current team config display name when set. Reasons are limited to 500 characters on write, and older oversized reasons are omitted from the explanation on read. Inbox and scout reviewer edits discard oversized legacy reason text while keeping the reviewer and valid commit evidence. The inbox marks reviewers without a linked PostHog member.- The inbox groups reviewers only when their displayed explanations and source categories match, then shows the explanation once below the grouped people. Scouts with different display names share one category and one badge; code history and other sources remain separate.
Analytics Events
All events use distinct_id = team.uuid and groups(organization, team). Per-signal events carry source_product, source_type, source_id — pivot on source_id to trace one signal.
Lifecycle (in order):
signal_data_source_entered/signal_data_source_summarized/signal_data_source_filtered— data-source pipeline only (pipeline.py)alert firing signal skipped— the per-team cap stopped ananalytics/alert_firingsignal beforeemit_signal()(+team_id,alert_id,alert_check_id,reason:cap|cap_unavailable)signal_emission_started—emit_signal()past validationsignal_emitted—emit_signal()after Temporal dispatch succeedssignal_assigned_to_report— grouping assigned the signal (+report_id,is_new_report,promoted)signal_report_reresearch_skipped— signal hit an already-researched report past the re-research cap, so no new run spawned (+report_id,signal_count,status,threshold). Fires per suppressed signalsignal_report_quota_paused— a quota gate observed the team's org over its self-driving credits limit (+stage:promotion|summary_entry|pre_repo_selection|pre_research|autostart|manual_create|task_create|implementation_run,enforced,report_id). See Billing limit enforcementsignal_report_daily_limit_paused— a gate paused work because the team hit itsmax_reports_per_day(+stage:ingestion|scout_run|promotion|summary_entry|pre_repo_selection|pre_research,limit,reports_today,report_idnullable). See Per-team daily report limitsignal_report_free_trial_paused— a gate held back a pull request because the team's org is on a Self-driving free trial (+stage:autostart|manual_create|task_create|task_run,report_idnullable). See Self-driving free trialsignal_report_started— report run began (+report_id,signal_count,run_count,source_products).mark_report_in_progress_activityfires it when a summary run moves the report fromcandidatetoin_progress. A promoted report (promoted_atset) does not always get it: thesummary_entryquota and daily-limit gate, or a deferred empty signal fetch, returns before that step and leaves the reportcandidate. A born-visible report does not fire it at creation. A scout report can fire it later, when grouping matches a new signal to the report through its backing signals and starts re-research. Custom-agent and stack-layer reports have no backing signals, so they never fire itsignals_repo_research_started/signals_repo_research_completed— repo selection stage (+report_id,result:reused|selected|no_repo|failed, optionalfailure_reason:no_github_integration|agentic_activity_error)signal_report_completed— terminal per run (+result:ready|failed|pending_input|not_actionable, optionalfailure_reason)signals_autostart_steering_attached— an implementation task was created (+report_id,task_id,notes_attached,scratchpad_available,memory_protocol,nudge_rendered). Fires on every auto-start, so the share carrying fleet steering is readable. See Auto-Start Flowsignals_autostart_skipped— an auto-start evaluation started nothing (+report_id,skip_reason:not_actionable|requires_human_input|already_addressed|no_priority|autostart_disabled|quota_exhausted|no_runner|free_trial|duplicate_of|blocked_by_dependency|plan_parent,linked_report_idnullable). The idempotent "a task already exists" skip is deliberately not counted, because it fires on every re-evaluation of a report whose run already started. See Auto-Start Flowsignals_report_linked— a typedreport_linkwas written (+report_id,linked_report_id,kind,has_reason,actor_kind,actor_agent). Fired from the one write path every producer shares, so links get a denominator they do not otherwise have. See Typed report links
Tracing one signal: filter on properties.source_id = <id> to follow it through the funnel, then pivot to properties.report_id from signal_assigned_to_report to see the report's lifecycle.
Telemetry is best-effort; failures are logged, not raised.
LLM Integration
Most direct LLM calls use Anthropic via the shared call_llm() helper in backend/temporal/llm.py, with model selection driven by SIGNAL_MATCHING_LLM_MODEL (default: claude-sonnet-5). The emission stage (summarization, actionability) uses its own SIGNAL_EMISSION_LLM_MODEL, and the two safety stages use SIGNAL_SAFETY_LLM_MODEL (both default to claude-sonnet-5). call_llm() takes a model argument so a stage can pin its own model; the safety stages pass SAFETY_MODEL so a matching-model swap can never silently retune the security gate. Each model's request shape (assistant prefill, per-request temperature, extended thinking) is resolved from MODEL_CAPABILITIES in backend/temporal/llm.py, so swapping either default is a config change. Adaptive-thinking models run every call at ADAPTIVE_MODEL_EFFORT (medium), set through effort_kwargs() in the same module. call_llm() also takes cache_system_prompt, which marks the system prompt as a prompt-cache prefix. The per-signal safety filter opts in because its prompt is above the model's cache minimum and runs continuously. No other stage opts in, for two different reasons: the report judge, matching and specificity prompts are static but sit below the minimum, so there is nothing to cache; query generation builds its prompt per team from the current signal type examples (_build_query_generation_system_prompt()), so the prefix changes between batches and a cache write would never be read.
That said, not all “LLM-ish” behavior in Signals goes through call_llm() anymore:
- Grouping-time query generation / matching / specificity checks use
call_llm() - The buffer safety filter uses
call_llm() - The report safety judge uses
call_llm() - Eval-signal summarization uses
call_llm() - Repository selection runs via the sandbox agent flow, not
call_llm() - Agentic report research runs via
MultiTurnSessioninreport_generation/research.py, notcall_llm()
The sandbox-backed paths (repo selection, report research, scout runs, and the coding runs autonomy auto-starts) all
land in the posthog-sandbox-self-driving Modal app rather than the default one, because their tasks carry
origin_product=signal_report / signals_scout (see SELF_DRIVING_ORIGIN_PRODUCTS in
products/tasks/backend/logic/services/sandbox.py). The app is resolved inside the Tasks provisioning activity, so
nothing in Signals selects it. Same image, resources, and egress policy as any other run — the split only separates
the fleet's Modal cost from user-driven runs. Egress is still governed by the SandboxEnvironment each caller
upserts (temporal/agentic/__init__.py).
call_llm() (backend/temporal/llm.py)
A generic helper that abstracts the retry / validate / append-errors pattern used by direct LLM calls. It takes a system prompt, user prompt, validation function, and options like thinking, temperature, and retries.
Key behaviors:
- Retry with conversational self-correction: on validation failure, the full response content is appended as an assistant message, followed by the validation error as a user message
- JSON enforcement: for non-thinking calls, the helper pre-fills the assistant response with
{to discourage markdown fences; for thinking calls it strips common fenced-JSON wrappers - Extended thinking: when
thinking=Trueand the selected model supports it, enables Anthropic extended thinking with increased token budgets - Debug logging: in
DEBUG, logs raw failed responses to help tune prompts / validators
Grouping-time LLM calls
The architecture doc previously attributed these functions to backend/temporal/llm.py, but that file is only the helper. The grouping-specific call sites live in backend/temporal/grouping.py.
Search-query generation
Generates 1–3 cross-source search queries for a signal from different angles (feature/component, behavior/issue, impact), using recent signal type examples as context. Queries are truncated before embedding.
Match-to-report decision
Chooses between:
- matching the signal to an existing report candidate
- creating a new report candidate with title + summary
The validator ensures the returned signal_id and query_index are valid for the candidate set given to the model.
Match-specificity verification
A second grouping-time LLM check used before broadening an existing report too aggressively.
After a research pass completes on a report, the check also receives the report's summary as the researched cause. A new signal joins only when the fix that the research describes also resolves it. A shared page, HTTP status, or product area is not enough. The check can rename a report only before its first research pass. After that pass, the research owns the title, so the title and the summary continue to describe the same cause.
Safety filter (backend/temporal/safety_filter.py)
Per-signal safety classifier that runs in the buffer workflow before signals are flushed to object storage.
It blocks a signal only when the content tries to manipulate the coding agent: instruction override, hidden instructions, encoded payloads, secret exfiltration, or remote code execution. It does not block a signal for its topic. Security-sensitive tickets, the team's own risky changes, first-party monitoring reports, scanner traffic logged as errors, and vulnerability reports pass, because a human reviews every resulting pull request and the report judge and the agent's own rules sit downstream. The user prompt carries the signal's source and the current date, so the classifier applies the right trust context and reads an unfamiliar date or version as real rather than fabricated. One prompt serves every source; the source line, not a separate prompt, supplies the trust context.
Returns:
{"safe": true, "threat_type": "", "explanation": ""}{"safe": false, "threat_type": "...", "explanation": "..."}
If the provider returns an empty response, the signal is treated as unsafe with threat type provider_safety_filter.
A block fires a signal_blocked_by_safety_filter event in the internal project, carrying the threat type and the explanation, but not the full signal text.
The explanation quotes the fragment that caused the block, so the event holds a short excerpt of the signal.
Product analytics keeps that excerpt indefinitely.
For this reason the prompt tells the classifier to describe a secret value instead of quoting it.
The full text of a blocked signal goes to the LLM analytics store for the safety-filter judge scout: the activity captures an $ai_span named safety_filter_block (BLOCKED_SIGNAL_SPAN_NAME) through the SDK's AI capture lane, with the description and source in $ai_input_state and the verdict in $ai_output_state.
That lands in posthog.ai_events, which drops content after its retention period (30 days by default), and the block event links to it through $ai_trace_id.
The pipeline keeps this copy itself because the LLM gateway that served the call is not a reliable source: the Go gateway's mirrored generation events carry no input or output.
This is the first line of defense; it prevents adversarial signals from consuming embedding / search / matching work.
Report safety judge (backend/temporal/report_safety_judge.py)
Report-level safety review that runs before repository selection and agentic research. It evaluates the underlying grouped signals for manipulation attempts that could steer a downstream coding agent, under the same five-category definition the per-signal safety filter uses (instruction override, hidden instructions, encoded payload, secret exfiltration, remote code execution) and the same do-not-block list, so the two stages cannot disagree on a signal's topic. The rendered signals sit inside a <signal_data> block whose closing tag is neutralized in content, and the prompt treats everything inside the block as untrusted data.
Returns {"choice": bool, "explanation": "..."} and stores the result as a safety_judgment artefact. Extended thinking is enabled.
Importantly, this judge currently assesses the signals, not a generated title/summary — those do not exist yet when the judge runs.
Agentic research outputs (backend/report_generation/research.py)
The old standalone summary / actionability judge files referenced in earlier docs no longer exist. Their responsibilities now live inside the multi-turn research flow and its persisted output models.
The research flow produces:
SignalFindingsignal_idrelevant_code_pathsrelevant_commit_hashesdata_queriedverified
ActionabilityAssessmentexplanationactionability— judged againstbackend/report_actionability.py, the criteria the scout harness prompt renders too, so an authored report and a researched one are held to one standardalready_addressed
PriorityAssessmentexplanationpriority
ReportPresentationOutputtitlesummary
These are assembled into ReportResearchOutput, then persisted by run_agentic_report_activity.
A task_run artefact (type=research) is appended immediately after the MultiTurnSession starts (before any research turns), associating the sandbox Task with the report and attributing it to that task.
Autonomy & Auto-Start
The autonomy system allows Signals to automatically start a Tasks coding run when a report is immediately actionable.
Configuration
Autonomy is configured at two levels:
Team level (
SignalTeamConfig): Sets thedefault_autostart_prioritythreshold (P0–P4). Created on first access throughget_or_create_team_extension. Managed viaGET/POST /api/projects/:team_id/signals/config/.User level (
SignalUserAutonomyConfig): Per-user opt-in. A row existing means the user is opted in. Each user can optionally override the team priority threshold withautostart_priority. Managed viaGET/PUT/DELETE /api/users/@me/signal_autonomy/.
The management command enable_signals_autonomy can set both in one shot:
python manage.py enable_signals_autonomy <team_id> <priority> <emails>
Auto-Start Flow
Runs inside maybe_autostart_implementation_task() in backend/auto_start.py, called after artefact persistence in both run_agentic_report_activity (the signals pipeline) and run_custom_signal_agent_activity (custom agents).
Guard clause — all must pass:
- Report actionability is
immediately_actionable - Report is not
already_addressed— which covers both a fix that has landed and one already in flight (an open PR, a recently active branch, an assigned / in-progress issue or agent task). Research is instructed to check for in-flight work and set the flag, so this gate is what keeps us off work a human or another agent already has going; the implementation prompt repeats the check as a pre-flight - Report has a
priority_judgment - Report has suggested reviewers
- No legacy
SignalReportTaskimplementation row exists for the report (checked inside aselect_for_updateon the report row, so concurrent evaluations can't double-start) - The team's org is not on a Self-driving free trial, and not over its self-driving credits quota with enforcement on (see Billing limit enforcement)
- No typed
report_linkholds the report back (see Typed report links)
User selection via _resolve_autostart_assignee() in backend/auto_start.py:
- Map reviewer GitHub logins to PostHog user IDs via social auth (preserving reviewer relevance order)
- Single query: fetch
Userobjects whose ID is in that list and who have aSignalUserAutonomyConfigrow (joined viaselect_related) - Walk candidates in reviewer order. For each user:
a. Verify team membership via
user.teams.filter(id=team_id).exists()b. Resolve their effective priority threshold: personalautostart_priorityif set, otherwise the team'sdefault_autostart_priorityc. Ifreport_priority_rank <= threshold_rank→ return that user - If no user matches → skip
Task creation:
Task.create_and_run(origin_product=SIGNAL_REPORT, ...)record_implementation_taskwrites the legacySignalReportTaskimplementation gate row (in the same transaction) and appends animplementationtask_runartefact- Errors are caught and logged but do not fail the report workflow
Origin section per pull request (backend/pr_origin.py, always on).
A reviewer needs to know where a self-driving pull request came from, not only the problem it fixes. The backend writes an ## Origin section below the Problem section, so the agent cannot paraphrase it. The section lists the source types with PostHog entity links, the scout when it ships with PostHog, the first signal date, the inbox report, the likely cause commit from the newest signal_finding (linked only when the suggested reviewers carry it under the pull request's repository), and whether auto-start or a person started the run.
Pull requests are often public, so the section renders an allowlist only. Signal content, ticket text, counts, sequential ticket numbers, custom scout names, and issue URLs outside the pull request's own repository never reach it. fetch_origin_sources_for_report aggregates in ClickHouse, so a large report returns a bounded result.
The write edits only a pull request that the GitHub webhook confirmed in the run's verified_pr_urls, because a run's output.pr_url is writable through the API. An unconfirmed URL retries. HTML-comment markers per report let a retry replace the section in place, and an unmarked ## Origin from a repository template stays untouched.
Tracker issue per pull request (backend/tracker_issues.py, off by default).
Some teams cannot merge a pull request unless a tracked work item points at it. SignalTeamConfig.issue_tracking_integration names the tracker (GitHub, GitLab, Linear, or Jira) and issue_tracking_config names the target inside it; a null integration means the team wants no tracker issues, so one field is both the switch and the target.
_create_implementation_task_if_absent opens the issue after it creates the implementation task, outside the report lock because the call is network I/O. This order prevents an issue from outliving a task transaction that fails. A SignalReportTrackerIssue row per report keeps two evaluations from opening duplicate issues. Linear receives a direct attachment after the pull request opens, and the pull request body also links to the issue.
The create never raises. A provider failure is stored on the row as status=failed with a short reason, which the report surfaces next to the pull request, and the run opens its pull request either way.
Once the pull request exists, link_report_tracker_issues (scheduled from the task-run PR sync receiver) writes the Origin section and then appends the reference to the pull request body, behind an HTML-comment marker so the append happens once. Both edits go through edit_pull_request_body in backend/pull_request_body.py, one after the other, so they never race on the same body. GitHub rejects If-Match on a pull request update, so each edit reads the body again before it writes and retries when the body changed. GitHub gets Closes #n; the other providers get the issue link. A Linear issue also gets the pull request as an attachment, best effort, because the scope for it may not be granted.
An irreversible end closes the tracker issue: a resolve asked for through the state API, a merged pull request (closed as done), or a deleted report. A suppressed or snoozed report keeps its issue open, because both come back, and so does a failed run, because its report stays in the inbox and the work item is still real.
One DRI per pull request (backend/reviewer_pr_assignment.py).
Every self-driving pull request opens with no assignee, so it never reaches a GitHub "Assigned to me" view. schedule_reviewer_pr_assignment runs when a pull request first reaches a report, and assigns on a worker.
At assignment time, select exactly one human responsible for the pull request. Explicit human intent wins; otherwise derive a DRI. Machine-generated claims never participate in that decision.
This governs what the rule adds. Nobody is ever unassigned, so a person can still assign three people by hand.
"Assignee" here means GitHub's assignee, the person responsible. A single assignee receives a review request even when another person or team request exists. Without a single owner, the worker requests the highest-ranked eligible report reviewer only when no request exists. Each attempt requests at most one person and preserves other requests. The worker skips the author, pending recipients, completed reviewers, and people whose review request was removed. Incomplete review or removal history prevents a new request. A separate app can still request the owning team for review.
Under the signals-pr-dri-assignee flag, the candidates are, most responsible first:
- The person who chose the work. A claim a person made, or one their agent made for them, since an agent claim records the person it ran as. A task claim does not count: an auto-started report is claimed by its own implementation task, and that task runs as the report's top suggested reviewer, so it names a commit-history guess rather than a decision.
- Somebody who asked to be assigned through
SignalUserAutonomyConfig.github_assign_on_pull_request. Where several did, the one who owns the changed code wins, else the highest-ranked suggested reviewer among them. - A member of the team that owns the changed files, when the repository declares ownership in
owners.yaml(backend/pr_owning_team.py). The members follow in random order. A member the report suggests gets no priority, because commit history would otherwise give one prolific committer every pull request. The core ownership module (posthog.ownership.paths.resolve_path_owners) resolves the team. GitHub teams are the canonical team identity, so the members come from GitHub's team members API, which needs the organization members read permission on the GitHub app. - The suggested reviewers in rank order, when no ownership source names a team.
A candidate from rungs 1, 2 and 4 must be an organization member with a connected GitHub account. A member of the owning team needs only GitHub team membership, because that membership is the ownership, and does not need a PostHog account. Each candidate is checked with GitHub's read-only assignee check first, because the add-assignees call drops a login without push access instead of failing. A candidate GitHub cannot assign moves the walk to the next one, for up to four checks, and a failed check stops it. A report with no candidate at all gets no DRI.
A pull request that already has an assignee is left alone, and so is one that somebody ever unassigned by hand, because a later report event must not undo that decision.
Without the flag the older rule stands: every suggested reviewer who opted in is added as an assignee, and there is no DRI.
[!WARNING] Keep the
signals-pr-dri-assigneeflag on the PostHog organization only until a team-level off switch exists in the inbox settings. Assigning people on a customer's GitHub is that team's decision, and today a team cannot turn the rule off: removingowners.yamlonly changes who gets picked.
The ownership lookup reads the files through posthog.ownership, with the GitHub integration the team already connected, so a private repository resolves wherever that installation covers it. A repository no installation covers is still read anonymously, which answers for a public repository only. The lookup reads owners.yaml and nothing else, so a repository declaring ownership in CODEOWNERS alone falls back to the suggested reviewers. CODEOWNERS as a second format behind the same resolver interface is still open.
Fleet steering in the task description (load_report_steering in backend/report_steering.py).
Scouts read the team's steering notes at the start of every run, and the implementation run that acts on their report has to be able to reach the same guidance, such as "this area is frozen". The description does not paste the notes. It carries a short nudge instead: the team leaves notes and the fleet keeps a scratchpad, either can hold a freeze, a fix in flight, an earlier decision, report feedback, or a repository gotcha, and the run reads them through scout-notes-list (a small content_max_chars first, then full text only for notes that touch the change) and scout-scratchpad-search (keys_only=true, once per entity it will change and once for pattern:impl:<repository>). The run decides what applies.
Pasting cost the run a large share of its prompt before it read any code, and most fleet notes are about how to write reports or route reviewers, not how to change code. The author of the report read the notes already: a scout at cold start, the research stage through load_research_steering. The tradeoff is that the prompt no longer guarantees the run sees a "do not change this area" note. The upstream author has seen it, and the nudge names freezes and fixes in flight as the main thing to look for.
In the pull model the run can read every note origin, the derived ones included, because its token can read reports. That is accepted: the run already reads the report, its source issues, and PR comments. What the old HUMAN-only filter protected against was pasting that text in unasked, and the description still never carries note text. Nobody has to address notes to the implementation stage, so pipeline:implementation stays out of PIPELINE_AUDIENCES.
A report on a child environment gets no steering at all. Notes and fleet memory live on the canonical project, so the nudge would send the run to search a team that holds none of them. The read is best effort: a failure costs steering, never the run.
Fleet memory the implementation run writes back (_IMPLEMENTATION_MEMORY in backend/report_steering.py).
The nudge above is read-only, so every run re-derived the same repository knowledge: where a fix actually lives, which dead end not to walk again, the step the tests need first. An autostarted run gets the write half too: at the end of the run it decides what the next run would want to know and writes it, keyed pattern:impl:<repository>:<area>. The repository sits in the key because one project can connect several, and an area name like billing means something different in each.
The nudge's sweep runs keys_only=true and reads bodies only for the keys that matter: search_scratchpad returns 20 entries by default and content is capped at 50,000 characters, so an unbounded sweep can cost more context than the report the run is acting on. The write half opens with a skip clause for a run that does not hold scout-scratchpad-remember, because a task description is written once at creation while a rerun of that task is minted full by the tasks API (full_mcp_run_sources in products/tasks/backend/facade/api.py) and carries no scratchpad write scope. The posture gate below decides what a description says; the skip clause is what keeps a later run off an instruction the description could not have known about.
One property is inherited rather than solved: remember upserts a key under a short write lock, so two runs condensing the same key concurrently lose the earlier one's addition. That is true of every scratchpad writer, scouts included, and closing it needs a compare-and-merge operation on the tool rather than a prompt rule. The repository in the key narrows the collision surface; it does not remove it.
Three rules ride with it. Describe, never quote: nothing copied out of an issue body, a PR comment, a code comment, a log line, or an error message goes into an entry, because repository text is controlled by anyone who can open an issue and the scratchpad is read by every scout and every later run. Search the key first, then condense, since remember replaces a key in place and the keyspace is shared. Always set expires_at, 30 days by default, which inverts the scout default (durable unless time-boxed) because a claim about a moving repository ages differently from a claim about a team's data shape.
The section renders only when the run's token actually carries signal_scratchpad_internal:write. load_report_steering takes memory_writable, and auto-start derives it from oauth.grants_scratchpad_write(IMPLEMENTATION_MCP_SCOPES), the same constant it mints the token with, so the instruction cannot outlive the scope behind it. A person pressing Create PR on a report goes through the tasks API, which stores the submitted description unchanged and picks full from run_source for the MCP scopes only. That run gets no steering section from this module. Entries a run writes are attributed pipeline:implementation through backend/pipeline_identity.py. signals_autostart_steering_attached carries nudge_rendered and memory_protocol, so the two postures stay separable in the data. notes_attached is always 0 there, and scratchpad_available is always false, because the implementation run reads neither.
Fleet steering in the research prompt (load_research_steering in backend/report_steering.py).
The research stage decides a report's findings, actionability, priority, and title, and it read no steering at all. So a reviewer who dismissed an earlier report with "this is expected, it's the approval flow" shaped every scheduled scout run and nothing else, while the stage that judges whether the same topic is worth surfacing again never saw it. The initial research prompt now carries a ## Steering from this team section with the same child-environment rule and the same best-effort posture.
The section does not paste the newest notes. It tells the run to search them with scout-notes-list and text, a case-insensitive match on note content, once per entity the report names. A newest-first page was the wrong unit: on a busy team it filled with dismissals of unrelated reports, and a dismissal of a similar report was gone before the next similar report arrived. The nudge renders only when the team holds at least one live note. Every origin is reachable, the derived ones especially. Those carry what a person said when they dismissed, discussed, or rated a report, which is the only feedback this stage has ever had. The run is read-only, its prompt already carries the report's own raw signals, and it writes back only to the report on the same team, so the report content a derived note quotes reaches nobody it could not already reach. The read goes through the notes API, so _may_read_reports still applies to the run's token and user.
The one audience still pasted in is pipeline:report-research (scout_harness/note_targets.PIPELINE_AUDIENCES), capped at 10 notes and 1,000 characters each, because every note there is addressed to this stage. The section states the untrusted-input rule the scout harness carries, and asks the run to name the note in the assessment explanation it changed, so a reviewer can see their feedback land. It also says that a note counts only when it speaks to the same behavior, entity, or area the signals describe. Without that, a stage whose notes can only push toward "do not file" would be invited to stretch a loosely related dismissal over a distinct finding.
signals_research_steering_attached fires for every research run with notes_attached, dismissal_notes_attached, pipeline_notes_attached, scratchpad_available, memory_protocol, and nudge_rendered. Only the research audience is pasted now, so notes_attached equals pipeline_notes_attached and dismissal_notes_attached stays near 0; nudge_rendered, read against the run's scout-notes-list calls, is what shows whether older feedback could reach the run. Joining it to signal_report_completed on report_id is how "did steering change what the pipeline decided?" gets answered from outcomes rather than from the agent's own claim, which is why the run reports no self-assessed "I applied this" field.
Fleet memory the research run writes back (_research_memory in backend/report_steering.py).
The research stage read the fleet's memory and wrote none of its own, so every judgment it formed died with the run and the next report about the same entity re-derived it. It now gets the write half on the same gate as the implementation run: load_research_steering takes memory_writable, and the caller activity derives it from oauth.grants_scratchpad_write(RESEARCH_MCP_SCOPES), the constant it mints the sandbox token with. Entries are attributed pipeline:report-research through backend/pipeline_identity.py, resolved from the token's task rather than from anything the agent says about itself. Attribution names the writer that opened a key, not the last one to touch it, because _upsert_entry keeps the original created_by_run / created_by_identity on update. The protocol therefore tells a run condensing someone else's entry to say so in the content, since the label will not.
The two stages share the three write rules and differ in what they key on. An implementation run keys on a repository it is changing; a research run keys on the entities a report names, so its entries are noise:<entity>, already_addressed:<entity>, and pattern:<domain>:<entity> for judgments, plus pattern:research:<topic> for what it worked out along the way, and they sit in the same keyspace the scouts already read under those prefixes. That is also why load_research_steering renders the search pointer whenever the research protocol renders, even on an empty scratchpad: the per-entity sweep the research entries depend on is the pointer's own instruction. The implementation nudge carries its own search step.
Two smaller points. The report id is interpolated into the section, because the research prompt renders one only on a re-research (previous_report_id), so a first run asked to cite "this report" would omit or invent it and leave an entry whose provenance nobody can check. And the concurrent-condensation limitation recorded for the implementation side is inherited here unchanged: remember upserts under a short write lock, so two runs condensing the same key concurrently lose the earlier addition. Entity-keyed entries collide more readily than repository-keyed ones, since a scout and a research run watching the same error id write the same key.
Per-run counts of entries read and written are deliberately absent from the steering event. Both calls happen inside the sandbox over MCP, so the activity never holds them; they are readable only as tool-call telemetry.
Priority Rank
P0 is the highest severity (rank 0), P4 is the lowest (rank 4). A report auto-starts only if report_priority_rank <= user_threshold_rank. So a team with default_autostart_priority=P4 will auto-start on any priority, while P0 will only auto-start for the most critical reports.
Task Tracking
Report ↔ task relationships are recorded as task_run artefacts (see SignalReportArtefact), one per associated run, with the run's purpose carried in the artefact's (product, type). The built-in pipeline writes product="signals" with type in {research, implementation, repo_selection}; custom agents supply their own identifier() pair.
Auto-start dedup is separate from this freeform log: maybe_autostart_implementation_task() (backend/auto_start.py) gates on a legacy SignalReportTask implementation row, checked inside the report-row select_for_update, so concurrent evaluations can't double-start. Both the auto-start and the manual tasks-API path go through record_implementation_task, which dual-writes that gate row and the implementation task_run artefact — the transitional arrangement until the backfill lets the gate move to artefacts (see SignalReportTask).
Legacy eval-signal compatibility (backend/temporal/emit_eval_signal.py)
New evaluation workflows record a removal patch and skip the per-result signal path. The old workflow and activity names remain registered as no-op compatibility definitions so histories that recorded those commands can replay. Evaluation reports are the only evaluation-based AI observability signal source.
Resetting self-driving state for local re-testing
The self-driving wizard (npx @posthog/wizard … self-driving) enables signal sources, materializes and enables the scout fleet, and creates custom scout skills for the project. To re-test a run from a clean slate without manually undoing each change, use the dev-only reset_signals_self_driving command (the practical inverse of enable_signals_autonomy; DEBUG-gated like cleanup_signals):
python manage.py reset_signals_self_driving --team-id 1 --yes \
--install-dir /path/to/your/test/project
For a fresh team the signals config tables are empty, so the reset deletes rather than disables — the next wizard run's sync call re-creates the canonical fleet (enabled) exactly as it would for a brand-new team. (Disabling instead would leave the fleet off on the next run, since the wizard's scout step only disables misfits and relies on the fresh default being enabled.)
Cleared for the team:
| What | Tables / artifacts |
|---|---|
| Signal sources | SignalSourceConfig (all rows) |
| Scout fleet config | SignalScoutConfig (all rows) |
| Custom scouts | LLMSkill rows that hold a SignalScoutConfig row or whose name starts signals-scout-, and are not seeded — i.e. not in the set of names carrying metadata.seeded_by == "signals_scout_harness" (covers the common case where the key is absent entirely) (cascades LLMSkillFile) |
| Scout run-state | SignalScratchpad, SignalProjectProfile, SignalScoutRun, SignalEmissionRecord |
| Emitted findings | SignalReport + artefacts + ClickHouse rows + Temporal workflows (delegates to cleanup_signals; skip with --keep-findings) |
| Wizard report | <install-dir>/posthog-self-driving-report.md (only if --install-dir is given) |
| Wizard log | /tmp/posthog-wizard.log → backed up to /tmp/posthog-wizard-previous-<timestamp>.log then removed (override --wizard-log, skip --keep-log) |
Preserved: canonical scouts and the authoring-scouts companion, identified by metadata.seeded_by == "signals_scout_harness". That tag is the practical marker this DEBUG reset uses; it is not a perfect canonical test on its own — _scout_origin also requires the name to ship on disk, since duplicate_skill copies the tag verbatim — but the wizard authors custom scouts via llma-skill-create with no tag, so tag-only suffices here. The command does not touch SignalTeamConfig or SignalUserAutonomyConfig (autostart / per-user opt-in are set by enable_signals_autonomy, not the wizard); llm_analytics sources are gated by their SignalSourceConfig rows like any other source.
Typed report links
A report_link artefact is one directed row on the report the sentence starts from: "this report kind that report". Five kinds: depends_on, part_of, follow_up_of, duplicate_of, recurrence_of. Nothing is mirrored onto the target, because the direction is the payload, and writing the reverse row would assert the opposite relationship. SignalReportArtefact owns the write invariants (no self-link, one team, no cycle within a kind, checked under a per-team advisory lock), and they run on the common add_log path so every surface gets them. The type is unwritable through the artefact API: the pipeline and the scout tools are its only authors.
backend/typed_report_links.py is the read side, and every reader goes through it. Link reads use the writer database to avoid replica delay.
outgoing_linksis a seek on(report, type): what this report says about others.incoming_linksseeks onlink_target_id, the target report id that every write and edit copies out ofcontent. A backfill migration filled the rows written before the column existed. No row is mirrored onto the target, because the direction is the payload. Candidates are then confirmed by parsing. A deleted source report is dropped by default; the recurrence chain is the one reader that asks for them, because it walks through deleted intermediates to find the live successor.duplicate_chainfollowsduplicate_ofand returns every report on the path, nearest first. A reader deciding whether the cluster is already being worked on needs all of them, because a pull request stays on the report whose run opened it: in A -> B -> C the fix can be in flight on B while C carries none.duplicate_rootreturns the last member of that chain, which is the id that names the cluster.
Rows that no longer parse name no edge, the same tolerance every other read of the artefact log has, and both readers are bounded by the graph budgets on SignalReportArtefact.
What reads them
Research context (_load_linked_report_context in temporal/agentic/report.py). Outgoing follow_up_of, depends_on and part_of targets are loaded with their title, summary, the code paths their findings named, and their pull requests with state, and rendered into the research prompt grouped by kind with the link's reason. Each kind carries what the agent is expected to do with it: a follow_up_of report starts from the linked report's findings and cites its pull request only when one is known, a depends_on report scopes itself to what the dependency does not cover, and a part_of child stays inside its step. Without this every pass re-derives an investigation the pipeline already paid for and usually misses its pull request. The latest safety judgment must explicitly approve a linked report before its content enters the prompt. Research uses the latest finding per signal, removes repeated relationships, and loads at most ten relationships it can use, so a run of deleted or unjudged targets does not spend that budget. Escaped data tags mark linked fields as untrusted evidence, with a 12,000-character limit on the rendered section. duplicate_of is absent because a duplicate never reaches research, and recurrence_of has its own richer read (_load_resolved_report_context).
Auto-start gates (_evaluate_link_gates in backend/auto_start.py), evaluated after the team's master switch and before the quota and runner gates. A team that opted out of auto-start entirely gets no gate reason at all, and a duplicate or a plan parent never counts as quota-held work, because the work was never this report's to do:
skip_reason |
Rule |
|---|---|
duplicate_of |
The report has an outgoing duplicate_of edge, and some report on that chain is resolved or carries a pull request that is not known to be closed |
blocked_by_dependency |
Some outgoing depends_on target carries no pull request, or only ones known to be closed |
plan_parent |
The report has incoming part_of edges, so it is the plan and the steps do the work |
The implementation path checks these gates again under the report lock before task creation. A gate returns AutostartOutcome(status="blocked"), writes an autostart_skip artefact naming the deciding report, and fires signals_autostart_skipped. The artefact exists only for the link gates: every other skip is a property of the report a reader can already see, while a link gate's reason lives on a different report. The gates hold the automatic path only, so pressing Implement in the inbox still starts a run. When a report's pull request syncs from its run, start_dependent_stack_layers re-evaluates the reports that depend on it (see Stacked plans).
Plan roll-up (backend/plan_rollup.py). A part_of child carries the work and closes on its own pull request; the plan usually carries nothing of its own, so without the roll-up it sits in the inbox forever after the last step lands. When every live step of a plan is closed, the plan takes their verdict: resolved when at least one resolved, suppressed when they all were, untouched while any step is open. A deleted step is not a verdict and is dropped. Deleting a step also triggers the check. A plan that becomes ready checks its own steps, so completion can follow a failed attempt during research. Each plan is locked before the check reads and updates its state. The walk continues through plans that are already closed. Archiving a resolved plan is refused, for the same reason _apply_pr_report_state refuses it. The walk continues up a plan of plans, bounded by MAX_PLAN_ROLLUP_LEVELS, which tracks the MAX_REPORT_LINK_GRAPH_LEVELS depth validated_report_link_write accepts: a hierarchy the write path allows but the walk stops short of would leave its top plan open, with every step that could retrigger the walk already closed. A plan with a pending replacement is skipped like any other report, and that read, like the pull request read beside it, is pinned to the writer because a replacement written moments earlier must not read as absent. A plan a person pressed Implement on does carry its own pull request, and while that pull request is open, draft, or unknown the steps decide nothing: resolving on their verdict would hide a plan whose own work is still in flight, and archiving on it would send the dismissal receiver to close that pull request. It is hooked on SignalReport's status change (receivers.roll_up_plan_parents_when_report_closes) rather than on the pull request webhook, so a merged pull request, a manual resolve, a bulk state change and an MCP state write all reach it. Plenty of steps close with no pull request at all. A part_of row written on a step that already closed runs the check too, from the link write path, because that write announces no status change of its own.
Stacked plans (backend/stack_plan.py). Research can return an optional layers plan on ReportPresentationOutput: two to six layers, each with a title, a scope, and the index of an earlier layer it depends on. A forward reference falls back to the previous layer, so the order is always acyclic. When the pass settles ready, mark_report_ready_activity creates one ready child report per layer in the same transaction. Each child gets the scope as its summary, the plan's latest safety, actionability, priority, repository and reviewer judgments, a part_of link to the plan, and a depends_on link to its dependency. A layer is born ready without promoted_at, like a scout report: it runs no research, so it fires no signal_report_started, and the inbox ranking spine takes it in through its born-visible branch. A plan that already has children keeps them on a re-research. The plan itself is held by the plan_parent gate, and its settle-point auto-start then starts the layers whose dependencies already have a pull request. After that, each pull request sync on a report queues start_dependent_stack_layers, which starts the waiting layers. A layer whose dependency has an open or draft pull request starts on that dependency's automation branch, and the run state carries stack_base_branch, so the task launch protects that branch as the PR base instead of the base of the dependency's PR. A merged dependency leaves the layer on the default base. Replacing a whole stack after a lower layer is superseded, a server-side GitHub stack link, and a collapsed inbox list are not built yet.
Report detail (backend/report_plan.py). The report detail response carries a plan field: the plan the report leads or belongs to, with each step's title, status, suppression_source, primary pull request, and the steps of the same plan it depends_on. A report with incoming part_of edges is a plan and lists its own steps, even when it is also a step of a larger plan. Otherwise its oldest outgoing part_of names the plan. Deleted steps, and dependencies on reports outside the plan, are left out. The field is null in list responses, because it costs several queries per report. The inbox renders it as a Plan block in the summary, after the Solution section, and hides the ## Plan sentence that _layer_summary writes on each layer. The frontend finds that sentence by its text (isGeneratedPlanSection), so the two must change together.
The recurrence chain (backend/recurrence.py) reads incoming recurrence_of edges to follow a forked report to its current live successor.
The inbox renders both types in the work log: a report_link row shows its kind, reason, and a link to the other report, and an autostart_skip row says in plain words why nothing started and links the deciding report.
Billing limit enforcement
Self-driving bills a flat credit charge per report that ships an implementation PR (see backend/billing.py),
so a team whose org is over its signals_credits quota must stop generating new research and PRs — including flows already mid-pipeline.
The authority for "over quota" is the Redis quota-limiter flag (ee.billing.quota_limiting, resource signals_credits),
read through quota.is_team_signals_quota_limited() and bundled with the rollout flag by quota.self_driving_quota_gate().
The quota is org-level (the org's billing cap): usage from every team in the org counts against one shared limit.
When the org crosses it, the quota cron fans the limited flag out to every team token in the org, so the per-team check reads an org-wide verdict and all of an org's teams pause together.
Enforcement is feature-flag-gated for rollout (self-driving-quota-enforcement, org-keyed).
Gates always run and emit telemetry when the team is limited;
they only block when the flag is on, so the would-block volume is measurable before enablement (enforced=false events are dark-launch rows).
Every check fails open — a Redis or flag-read error lets work through rather than stalling the fleet — and the bypass is alertable via signals_quota_check_failed_open_total.
Gates, in pipeline order:
| Stage | Where | Behavior when limited (enforced) |
|---|---|---|
| ingestion | check_signals_quota_limited_activity (buffer flush) |
Batch dropped before safety filter / flush (pre-existing gate) |
scout_run |
run_signals_scout_activity + manual-trigger API |
Run skipped / trigger 429s. The manual path captures the pause itself, since a rejected trigger never reaches the activity |
promotion |
assign_and_emit_signal_activity |
Signal still assigned, weighted, and emitted, but the report is not promoted — no summary run spawns; status left untouched |
summary_entry |
SignalReportSummaryWorkflow, before any work |
Workflow exits; report stays candidate |
pre_repo_selection / pre_research |
SignalReportSummaryWorkflow, between the heavy steps |
Workflow reverts the report in_progress → candidate and exits; an in-flight research activity is never interrupted |
autostart |
maybe_autostart_implementation_task (all callers) |
No implementation task is created — the step that would lead to the billable PR |
manual_create / task_create |
tasks facade create_task / create_and_run_task |
Creating a self-driving implementation task (e.g. "start work" from a report) returns 402 with a raise-the-limit message; create_pr=False sessions (research, repo selection) are exempt. task_create is the facade backstop behind auto-start, kept as its own stage so the manual path stays attributable |
implementation_run |
enforce_self_driving_run_quota (process-task workflow) |
A PR-opening (create_pr) run that hasn't recorded its PR yet re-checks every 5 minutes and cancels itself through the standard cancellation path |
The billable event re-evaluates the quota immediately.
When a self-driving-origin run records its first PR URL (agent report, PATCH, or GitHub webhook backstop), the tasks facade queues refresh_org_self_driving_quota (Celery), which recomputes the org's live signals_credits usage and re-runs the Redis limiter — so the PR that crosses the limit flips the flag within seconds instead of at the next 15-minute quota cron tick.
The cron remains the backstop.
The cron writes its verdict by reconciling the Redis set against the snapshot it took at its start, so a limit the push refresh writes while a cron run is in flight survives that run instead of being wiped until the next tick.
One timing edge: the live count is keyed to the implementation run's creation day (UTC), so a PR recorded just after midnight by a run created before midnight falls in the previous day's window and does not move the live counters.
It reaches enforcement hours later, via that day's usage report; the charge itself still lands in the correct day.
A quota-cancelled run leaves only the report.
release_quota_cancelled_implementation removes the run's SignalReportTask gate row and implementation task_run artefact (which would otherwise permanently block re-implementation) and appends a system note artefact explaining the stop.
Reports that already shipped a billable PR (through a sibling run of the same task) are skipped entirely: that gate row is billing's evidence for the charge, and deleting it would re-bill the report on its next implementation and break refund eligibility.
Runs that already recorded a PR URL are never cancelled: the report is already billed, so finishing the run and its CI follow-ups costs nothing more.
The release path is deliberately not hardened against a PR landing concurrently with the cancel (or an earlier run's PR on the same task): a PR that slips through ships un-billed.
Enforcement exists to stop runaway generation, not to guarantee billing capture, so under-billing errs in the customer's favor.
A cancel that errors part-way is logged loudly (self_driving_quota_cancel_failed) rather than counted as a quota-check fail-open, and nothing is released:
a still-alive run gets the cancel retried on the next recheck, while a run the completion signal did reach keeps its implementation records until they are released manually.
Scout suggestion scans are deliberately not gated.
A suggestion scan (arun_scout_suggestions, and the suggestions/refresh/ endpoint) opens no pull request, so it can never move the meter, and its output is a stored batch the scouts tab reads rather than signals the pipeline would drop.
Skipping one also costs more than it saves: the coordinator stamps last_requested_at at dispatch, so a skipped project waits the full refresh_days window even after its billing period resets.
The pause is blanket: billing-exempt reports pause too.
Every gate reads only the org-level quota verdict; none consults a report's billing_exempt_reason, so PostHog-system reports (health checks, engineering analytics, exempt scout skills) — which never consume the quota — pause alongside billable work while the org is over its cap.
This is deliberate: a single org-level contract ("over the cap → self-driving pauses") stays simple to reason about and support, the paused work is PostHog-generated and resumes organically once the limit lifts, and an org at its spend ceiling also stops consuming agent compute for non-billable reports.
Resume is organic, never automatic.
Nothing re-spawns paused work when the limit is raised or the billing period resets:
a candidate report re-promotes on its next matching signal (ingestion unmutes once the team is no longer limited),
and a ready report's implementation starts only when a later research cycle re-evaluates auto-start (or a user starts it manually).
Telemetry: every gate observation emits signal_report_quota_paused with stage, enforced, report_id (null at stages without one), team_id, and organization_id.
Promotion-stage events are emitted only when the signal would actually have promoted the report, so the volume measures withheld summary runs.
Per-team daily report limit
A second, independent pause with the same shape as the billing gates: SignalTeamConfig.max_reports_per_day caps how many reports may surface to a team's inbox per day.
Users who get more reports than they can read stop reading them, so once the cap is reached the whole generation pipeline stops — as early as ingestion, so no money is spent on work nobody will see — until the day resets.
Implemented in backend/daily_limit.py, a sibling of quota.py that may import the signals models (it is not on the billing.py ← usage_report ← quota_limiting early-import path that keeps quota.py model-free).
- Count basis: a report counts when it first becomes user-visible — its first transition into
ready,pending_input, orfailed— recorded by the set-onceSignalReport.first_visible_attimestamp. Re-research of an already-visible report never recounts. Historical failed reports without a stamp remain backlog and do not consume the rollout day's slots.transition_to()stamps pipeline reports;create_scout_reportstamps scout report-channel reports born directly in a visible status. - Day boundary: midnight in the project timezone (
team.timezone_info, viateam_day_start()), unlike the billing quota's UTCget_current_day(). This is a user-experience cap, so the day follows how the team perceives "today's reports". - Scope and default: per team,
null= unlimited. There is no enforcement flag: setting the field (viasignals/config/or the inbox usage rail) is the opt-in, soDailyReportLimitGatehas noenforcedsplit —limitedalone blocks.
Gate table (the same sites as the billing gates; each activity body resolves daily_report_limit_gate(team) next to the billing check, so no workflow command sequences changed and no new workflow.patched markers exist):
| Stage | Where | Behavior when the limit is reached |
|---|---|---|
| ingestion | check_signals_quota_limited_activity (buffer flush) |
Batch dropped before safety filter / flush; drop metric reason daily_report_limit |
| scout runs | run_signals_scout_activity + manual-trigger API |
Run skipped (skip_reason="daily_report_limit") / trigger returns 429 |
promotion |
assign_and_emit_signal_activity |
Signal still assigned, weighted, and emitted, but no summary run spawns; status untouched |
summary_entry / pre_repo_selection / pre_research |
check_report_quota_gate_activity |
Same pause + revert semantics as the billing gates above |
The implementation-side gates (autostart, manual_create, task_create, implementation_run) deliberately stay billing-only: implementing an already-generated report adds nothing new to read.
Research runs already in flight when the cap is hit finish and surface — money already spent should show results — so a day can overshoot the cap by the handful of concurrent runs; the gate is a pre-check, never a serialized reservation.
Billing-exempt reports count toward and are paused by the limit, matching the billing gates' blanket posture.
Resume is organic, exactly like the billing pause: after local midnight (or when the field is raised/cleared) the gates simply start passing again — dropped signals are gone, suppressed candidates re-promote on their next matching signal, and scouts run on their next coordinator tick.
Every check fails open (metric signals_daily_limit_check_failed_open_total).
Telemetry: each gate emits signal_report_daily_limit_paused with stage (ingestion | scout_run | promotion | summary_entry | pre_repo_selection | pre_research), limit, reports_today, report_id (null at pre-report stages), team_id, and organization_id.
No enforced property: every event is a real block.
Unlike the billing gates, the ingestion and scout stages do emit (with report_id=null) — the limit is user-set and the volume is bounded by the team's own signal flow, and the events answer "why did nothing arrive today".
Promotion-stage events keep the billing discipline of firing only when the signal would have promoted; both events fire independently when both limits bind.
Self-driving free trial
A third pause with the same shape, on the implementation side only: while the self-driving-free-trial flag is on for an org (org-keyed, toggled by sales per trial), Self-driving still researches and writes reports but opens no pull requests.
Billing happens only when a PR opens, so nothing is billed, and the exemption and refund machinery in billing.py is not involved.
Implemented in backend/free_trial.py, a sibling of quota.py with the same fail-open posture: a flag-read error counts as "not on trial", because a flag outage must not stop every org's PRs.
| Stage | Where | Behavior on a trial |
|---|---|---|
autostart |
maybe_autostart_implementation_task (all callers) |
No implementation task is created; the report stays ready |
manual_create / task_create |
tasks facade create_task / create_and_run_task |
An implementation task from a report is refused with 402, code self_driving_free_trial (FreeTrialPullRequestRefused); a discussion task from a report is allowed |
task_run |
tasks facade run_task |
Starting or retrying an implementation task that already existed is refused the same way; a discussion task keeps running |
The research-side gates are not involved: a trial org is meant to get every report.
The block is blanket, so billing-exempt reports (health checks) are held back too.
Discuss stays open: a Discuss run can still open a PR, and billing never counts it, because billing only counts implementation records. That is an accepted risk of a sales trial.
The inbox reads the same flag and keeps Create PR visible but disabled (detail pane, triage view, card context menu) with the sentence the refusal carries (FREE_TRIAL_PR_MESSAGE); the desktop app gets the server refusal only.
A run that is already in flight when the flag goes on is not stopped, because there is no mid-run check like the quota gate's; it can still open its PR.
Resume is organic, like the other pauses: nothing restarts when the flag goes off; a held-back report gets its PR on the next auto-start re-evaluation, or by hand.
Telemetry: each gate emits signal_report_free_trial_paused with stage (autostart | manual_create | task_create | task_run), report_id, team_id, and organization_id. Every event is a real block.
Data Types (backend/temporal/types.py)
| Type | Description |
|---|---|
EmitSignalInputs |
Workflow input: team_id, source_product, source_type, source_id, description, weight, extra |
BufferSignalsInput |
Buffer workflow input: team_id, pending_signals: list[EmitSignalInputs] (carried over on continue_as_new) |
TeamSignalGroupingV2Input |
Grouping v2 workflow input: team_id, pending_batch_keys: list[str] (carried over on continue_as_new) |
TeamSignalGroupingInput |
Legacy v1 entity workflow input: team_id, pending_signals: list[EmitSignalInputs] (carried over on continue_as_new) |
ReadSignalsFromS3Input |
Activity input: object_key |
ReadSignalsFromS3Output |
Activity output: signals: list[EmitSignalInputs] |
SignalCandidate |
Search result: signal_id, report_id, content, source_product, source_type, distance |
MatchedMetadata |
Metadata when matched to existing report: parent_signal_id, match_query, reason |
NoMatchMetadata |
Metadata when no match found: reason, rejected_signal_ids |
MatchMetadata |
Union type: MatchedMetadata | NoMatchMetadata |
ExistingReportMatch |
LLM decided signal matches existing report: report_id, match_metadata: MatchedMetadata |
NewReportMatch |
LLM decided signal needs new group: title, summary, match_metadata: NoMatchMetadata |
MatchResult |
Union: ExistingReportMatch | NewReportMatch |
SignalReportSummaryWorkflowInputs |
Summary workflow input: team_id, report_id |
SignalReportDeletionWorkflowInputs |
Deletion workflow input: team_id, report_id |
SignalReportReingestionWorkflowInputs |
Reingestion workflow input: team_id, report_id |
SignalTypeExample |
One example per (source_product, source_type) pair: source_product, source_type, content, timestamp, extra |
SignalData |
Signal fetched from ClickHouse: signal_id, content, source_product, source_type, source_id, weight, timestamp, extra |
Rendering Helpers
render_signal_to_text(signal, index=None) and render_signals_to_text(signals) in types.py provide a canonical text representation of signals for LLM prompts. All signal rendering in prompts goes through these helpers to ensure consistency. Each signal renders as:
Signal {index}:
- Source: {source_product} / {source_type}
- Weight: {weight}
- Timestamp: {timestamp}
- Description: {content}
Key Configuration
| Setting | Default | Description |
|---|---|---|
SIGNAL_WEIGHT_THRESHOLD |
1.0 |
Total weight needed to promote a report to candidate |
SIGNAL_MATCHING_LLM_MODEL |
claude-sonnet-5 |
LLM model for matching, grouping, and specificity signal operations |
SIGNAL_SAFETY_LLM_MODEL |
claude-sonnet-5 |
LLM model for both safety stages (per-signal filter, report judge). Not tied to the matching model, so a matching-model swap cannot silently retune the security gate |
SIGNAL_EMISSION_LLM_MODEL |
claude-sonnet-5 |
LLM model for emission-stage summarization and actionability checks |
MAX_RESPONSE_TOKENS |
4096 |
Base max tokens for LLM responses (thinking uses 3× for max_tokens, 2× for budget) |
| Embedding model | text-embedding-3-small-1536 |
OpenAI embedding model used for signal content |
| Task queue | VIDEO_EXPORT_TASK_QUEUE |
Temporal task queue for all workflows except the ranking sweep |
| Ranking sweep task queue | SELF_DRIVING_TASK_QUEUE |
Temporal task queue for the inbox ranking scoring sweep, polled by the temporal-worker-self-driving fleet |
BUFFER_MAX_SIZE |
20 |
Max signals buffered in memory before flush to S3 |
BUFFER_FLUSH_TIMEOUT_SECONDS |
5 |
Max seconds to wait for buffer to fill before flushing |
| S3 prefix | signals/signal_batches/ |
Object storage path for signal batch files (cleaned up by S3 lifecycle policies) |
COORDINATOR_INTERVAL_MINUTES |
30 |
Signals agent coordinator poll cadence (Temporal schedule, SKIP overlap policy) |
MAX_RUNS_PER_TICK |
50 |
Hard cap on planned runs per coordinator tick (most-overdue-first, truncated after sort) |
SignalScoutConfig.run_interval_minutes |
1440 |
Per-scout default schedule in minutes (daily); due-check, no sampling (30–43200) |
SignalScoutConfig.run_cron_schedule |
None |
Optional project-local cron schedule (overrides the interval); null keeps the rolling interval |
SignalScoutConfig.emit |
True |
Per-scout emit gate — defaults emit-on; flip to False for dry-run (scout runs and logs, but emit_finding writes nothing) |
Testing refunds locally
How to exercise the PR refund flow (backend/billing.py, the refund action, the credited-path billing sync) against a fully local two-service setup:
- Billing service: follow "Developing locally" in the billing repo's README (boots on
http://localhost:8100). The dispute endpoint (POST /api/signals/dispute-pr) must exist on the billing checkout. Dev mode auto-creates theLicenseandCustomerrows on first authenticated call — nothing to seed there. - Posthog: start the stack with
BILLING_SERVICE_URL=http://localhost:8100visible to both the web backend and the celery worker (put it in.env.local, which wins over.env.development). The dev license auto-creates as…::license-so-secret, matching billing's defaultLICENSE_SECRET_KEY; a staleee_licenserow with a different secret is the classic JWT-auth failure — checkSELECT key FROM ee_license;. - Feature flag:
signals-pr-refundsmust exist, active at 100%, in the local self-capture project (create via UI or shell;sync_feature_flagsalso works but flips every flag). Without it the refund endpoints return 404. Backend picks it up within the ~90s SDK poll (or restart); the frontend needs a page reload. - Data:
python manage.py seed_refund_test_data --team-id 1seeds the full matrix — excluded-path (PR today), credited-path (PR yesterday), billing-exempt, and a no-PR exemption-command target. - Expectations: with an unseeded billing side the credited path legitimately returns
credit_amount_usd = "0.00"— that IS the success path (free plan → nothing to credit). Non-$0 outcomes ("15.00"with paid-tier usage) require seeding a paid inbox state on the billing side; the recipe, a posthog-free curl smoke test, and how to verify the Stripe balance transaction live in the billing repo (internal):notes/testing-signals-disputes-locally.md. Inspecting the paid state via the posthog billing page (/organization/billing) additionally requires organization owner — local dev users are typically onlyadministrator, so bumpOrganizationMembership.leveltoOWNER(15) first. Verifybilling_synced_at/credit_amount_usdonsignals_signalreportrefund(posthog) and, for non-$0 outcomes, theCreditrow keyedsignals_pr_dispute:{billing_customer_id}:{refund_id}(billing). $0 outcomes persist nothing billing-side (credit_id: nullin the response) — the refund row is the whole record. Celery-side analytics events (signals_pr_refund_credit_issued/_failed) are dropped locally (ph_scoped_captureis cloud-gated); the sync itself is unaffected.
File Map
products/signals/
├── ARCHITECTURE.md # This file
├── backend/
│ ├── admin.py # Django admin for SignalReport + SignalReportArtefact
│ ├── api.py # emit_signal() entry point + source/org guards
│ ├── daily_limit.py # Per-team daily report limit gate (max_reports_per_day)
│ ├── apps.py # Django app config
│ ├── models.py # SignalReport, SignalReportArtefact, SignalTeamConfig, SignalUserAutonomyConfig, SignalReportTask, SignalSourceConfig, SignalScoutConfig, SignalScoutRun, SignalScratchpad, SignalProjectProfile, SignalEmissionRecord
│ ├── serializers.py # DRF serializers for source configs, reports, artefacts, team config, user autonomy config
│ ├── utils.py # Compatibility re-exports for signal query helpers
│ ├── views.py # SignalViewSet, InternalSignalViewSet, SignalSourceConfigViewSet, SignalTeamConfigViewSet, SignalReportViewSet, SignalReportArtefactViewSet, SignalProcessingViewSet
│ ├── scout_harness/ # Headless Signals agent — see scout_harness/AGENTS.md
│ │ ├── AGENTS.md
│ │ ├── __init__.py # Public re-exports (LoadedSkill, sync helpers, …)
│ │ ├── runner.py # Per-run entrypoint; owns SignalScoutRun lifecycle + sandbox loop
│ │ ├── prompt.py # System prompt assembly (skill + scratchpad + profile + run history)
│ │ ├── skill_loader.py # Resolves a scout's LLMSkill row for a run
│ │ ├── lazy_seed.py # Canonical SKILL.md → LLMSkill sync (sync_canonical_skills)
│ │ ├── limits.py # DEFAULT_MAX_RUNTIME_S + ACTIVITY_SLACK_S + WORKFLOW_HARD_CEILING_S + trigger-source vocabulary
│ │ ├── run_gates.py # Pre-dispatch gates shared by the off-schedule run paths
│ │ ├── workflow_runs.py # Dispatch for a workflow step that runs a scout
│ │ ├── serializers.py # DRF serializers for runs / scratchpad / project profile
│ │ ├── views.py # SignalScoutRunViewSet, SignalScratchpadViewSet, SignalProjectProfileViewSet
│ │ ├── profile/
│ │ │ ├── builders.py # Deterministic builders for SignalProjectProfile inventory
│ │ │ └── schema.py # Dataclasses for the inventory payload shape
│ │ └── tools/ # Harness-internal tools the agent calls inside a run
│ │ ├── emit.py # emit_signal_* — pushes findings as cross_source_issue signals
│ │ ├── scratchpad.py # remember / forget / search_scratchpad — read/write/delete SignalScratchpad
│ │ ├── profile.py # project_profile_* — read SignalProjectProfile snapshot
│ │ └── runs.py # runs_* — read past SignalScoutRun rows for dedupe
│ ├── github_issues/ # Placeholder directory
│ ├── management/
│ │ ├── AGENTS.md
│ │ └── commands/
│ │ ├── analyze_report.py
│ │ ├── cleanup_signals.py
│ │ ├── delete_all_signal_reports_for_team.py
│ │ ├── enable_signals_autonomy.py # Sets team default priority + opts in users by email
│ │ ├── export_session_video.py
│ │ ├── ingest_signals_json.py
│ │ ├── ingest_video_segments.py
│ │ ├── list_signal_reports.py
│ │ ├── parse_sandbox_log.py
│ │ ├── reingest_team_signals.py # Starts TeamSignalReingestionWorkflow for a team
│ │ ├── reset_signals_self_driving.py # Dev-only: undo a self-driving wizard run (configs, custom scouts, run-state)
│ │ ├── run_signals_scout.py # One-shot scout run; bypasses the coordinator
│ │ ├── select_repo.py
│ │ ├── signal_pipeline_status.py
│ │ └── sync_signals_scout_skills.py # Force a canonical SKILL.md sync to LLMSkill rows
│ ├── report_generation/
│ │ ├── AGENTS.md # Documentation for the agentic report generation flow
│ │ ├── research.py # Multi-turn sandbox research orchestration + output schemas + research task_run artefact creation
│ │ ├── resolve_reviewers.py # Suggested-reviewer resolution and enrichment helpers
│ │ └── select_repo.py # Repository selection sandbox flow
│ ├── test/
│ │ ├── test_agentic_report_activity.py
│ │ ├── test_api.py
│ │ ├── test_resolve_user_id.py
│ │ ├── test_signal_report_api.py
│ │ └── test_signal_source_config_api.py
│ ├── migrations/
│ │ ├── 0001_initial.py
│ │ ├── 0002_signalreport_clustering_fields.py
│ │ ├── 0003_alter_signalreport_status_and_more.py
│ │ ├── 0004_alter_content_type.py
│ │ ├── 0005_signalreportartefact_report_idx.py
│ │ ├── 0006_signal_source_config.py
│ │ ├── 0007_backfill_signal_source_config.py
│ │ ├── 0008_alter_signalsourceconfig_source_product_and_more.py
│ │ ├── 0009_add_new_signal_report_statuses.py
│ │ ├── 0010_add_data_import_signal_source_choices.py
│ │ ├── 0011_add_error_tracking_signal_types.py
│ │ ├── 0012_signalreport_run_count_and_more.py
│ │ ├── 0013_signalreport_suggested_reviewers.py
│ │ ├── 0014_signalreportartefact_report_type_idx.py
│ │ ├── 0015_alter_signalsourceconfig_source_product_and_more.py
│ │ ├── 0016_signalautonomyconfig_alter_signalreportartefact_type.py # SignalTeamConfig, SignalUserAutonomyConfig, SignalReportTask
│ │ ├── 0017_add_resolved_signal_report_status.py
│ │ ├── 0018_alter_signalreportartefact_type.py
│ │ ├── 0019_alter_signalsourceconfig_source_product_and_more.py
│ │ ├── 0020_signaluserautonomyconfig_slack_notification_fields.py
│ │ ├── 0021_add_signals_scout_source.py # cross_source_issue source variant
│ │ ├── 0022_add_signal_scout_models.py # SignalScoutConfig, SignalScoutRun, SignalScratchpad
│ │ ├── 0023_signalscoutrun_summary.py # SignalScoutRun.summary
│ │ ├── 0024_signalprojectprofile.py # SignalProjectProfile
│ │ └── 0027_reshape_scout_config_per_scout.py # per-(team, skill) config + per-scout schedules (drops runs_per_tick/enabled_skill_names)
│ └── temporal/
│ ├── __init__.py # Registers Signals workflows and activities
│ ├── agentic/
│ │ ├── __init__.py # Sandbox env / user-resolution helpers
│ │ ├── scout_coordinator.py # SignalsScoutCoordinatorWorkflow + per-tick due-check activity
│ │ ├── scout_scheduler.py # RunSignalsScoutWorkflow + run_signals_scout_activity
│ │ ├── schedule.py # Temporal Schedule definition (cadence + SKIP overlap policy)
│ │ ├── report.py # Agentic report activity + artefact persistence
│ │ └── select_repository.py # Repository selection activity
│ ├── backfill_error_tracking.py # Backfill recent error tracking issues as signals
│ ├── buffer.py # BufferSignalsWorkflow + object-storage flush/backpressure activities
│ ├── clickhouse.py # Retry wrapper for HogQL / ClickHouse activity queries
│ ├── deletion.py # SignalReportDeletionWorkflow
│ ├── emit_eval_signal.py # Compatibility definitions for old Temporal histories
│ ├── emitter.py # SignalEmitterWorkflow — per-signal backpressure bridge
│ ├── grouping.py # Legacy v1 workflow + active shared grouping implementation
│ ├── grouping_v2.py # Active grouping v2 workflow + pause/unpause support
│ ├── llm.py # Shared Anthropic helper + token limits / thinking config
│ ├── reingestion.py # SignalReportReingestionWorkflow + TeamSignalReingestionWorkflow
│ ├── report_safety_judge.py # Report-level safety judge activity
│ ├── safety_filter.py # Per-signal safety classifier activity
│ ├── signal_queries.py # Canonical HogQL helpers for fetch/search/soft-delete/wait
│ ├── summary.py # SignalReportSummaryWorkflow + report state transition activities
│ └── types.py # Shared dataclasses + signal rendering helpers
├── dags/
│ └── inbox_ranking/ # Dagster dags for the report-ranking model (daily modeling dataset) — see its README.md + AGENTS.md
├── skills/ # Signals skills — see skills/AGENTS.md
│ ├── AGENTS.md
│ ├── signals/ # Official PostHog skill (published via posthog_ai/dist): querying signals data
│ ├── inbox-exploration/ # Official PostHog skill (published via posthog_ai/dist): browsing the inbox
│ ├── signals-scout-general/ # Scout fleet: cross-product generalist (SKILL.md + emit.md + conventions.md)
│ ├── signals-scout-ai-observability/ # Scout fleet: AI observability anomaly watcher
│ ├── signals-scout-logs/ # Scout fleet: logs anomaly watcher
│ ├── signals-scout-error-tracking/ # Scout fleet: error tracking anomaly watcher
│ ├── signals-scout-revenue-analytics/ # Scout fleet: revenue anomaly watcher
│ ├── signals-scout-surveys/ # Scout fleet: surveys anomaly + theme-aggregation watcher
│ ├── signals-scout-observability-gaps/ # Scout fleet: structural-gap watcher (P3 recommendations)
│ └── signals-scout-csp-violations/ # Scout fleet: CSP violation watcher
└── frontend/ # Frontend components (not covered here)