Skip to content

armgit / Pulse

Public Imported from GitHub
0

Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.

Record Patrol investigation conclusions as a typed tool call

rcourtmanb3b5ad6e1fc4
diff --git a/docs/release-control/v6/internal/subsystems/agent-lifecycle.md b/docs/release-control/v6/internal/subsystems/agent-lifecycle.md
index 2140384fe0..267021b175 100644
--- a/docs/release-control/v6/internal/subsystems/agent-lifecycle.md
+++ b/docs/release-control/v6/internal/subsystems/agent-lifecycle.md
@@ -10165,3 +10165,13 @@ the credential reveal render as shared `InlineNotice`s; agent and security
facts as a `DetailSectionTable`; the connection id and identity evidence sit
in a closed Details disclosure. No diagnosis, repair or credential flow
changes: this is presentation only.
+
+The Patrol investigation wiring in `internal/api/ai_handlers.go` now passes the
+typed conclusion an investigation records with `patrol_conclude_investigation`
+to the private orchestrator. It also seeds the orchestrator's concurrency and
+timeout from the Patrol provider: one investigation at a time and a 25-minute
+default on a subscription agent. The digest email names the new `unclassified`
+and `interrupted` outcomes. None of this enrols, commands, updates or removes an
+agent, or touches agent identity or credentials. Investigations still reach
+hosts only through the existing read-only evidence tools and the governed
+action lifecycle.
diff --git a/docs/release-control/v6/internal/subsystems/ai-runtime.md b/docs/release-control/v6/internal/subsystems/ai-runtime.md
index 34d4912b1e..05931e26bd 100644
--- a/docs/release-control/v6/internal/subsystems/ai-runtime.md
+++ b/docs/release-control/v6/internal/subsystems/ai-runtime.md
@@ -9058,3 +9058,109 @@ diagnostics (`journalctl -u ... --since`, `dmesg -T`, `ss -tlnp`, `ip addr`,
read-only. Proof: `TestExecutionIntent_ArgumentAwareReadCommands` and
`TestExecutionIntent_RouteOperationBoundary` in
`internal/ai/tools/tools_query_readonly_test.go`.
+
+### Typed investigation conclusions and subscription-aware investigation limits (22 October 2026)
+
+Patrol investigations record their zero-action conclusion through
+`patrol_conclude_investigation`, an investigation-only, side-effect-free,
+read-kind, mutation-none tool on the request-local `ProposalCapture`. The model
+supplies one of `resolved`, `cannot_fix` or `needs_attention` and a reason.
+The capture keeps the first conclusion: an identical replay is accepted and a
+different outcome is refused. `InvestigationRunResult.Conclusion` carries it to
+the orchestrator beside any proposal. A typed proposal still owns remediation
+and outranks a conclusion, and the conclusion consumes no evidence budget.
+Once a conclusion is recorded, the next turn is text-only: the summary.
+The final turn, whether reached by the turn limit or by the deadline reserve,
+offers only the tools that end a run: the conclusion, plus the typed proposal
+once the evidence budget is spent. Evidence exhaustion offers the proposal and
+the conclusion.
+
+A run reserves time to conclude before its context deadline. The reserve is
+one more evidence turn plus the conclusion and summary turns, each estimated as
+the slowest turn so far (20 s minimum), plus 15 s, capped at half the run's
+budget. When the time left reaches the reserve, the current turn becomes the
+final turn. The final summary call may use the time left before that deadline
+instead of the 20 s interactive bound, which interactive chat keeps.
+
+On subscription agents, a positive `StreamIdleTimeout` remains an earlier
+complete-turn deadline, because the transport buffers the whole turn.
+Investigation turns there get a four-minute caller-owned stall budget, while
+Watch keeps 60 s. Without an explicit setting, an investigation on a
+subscription agent defaults to 25 minutes and to one investigation at a time,
+because that transport serves one call at a time.
+
+`LoadConfig` and autonomy saves republish the timeout and, through the optional
+`aicontracts.InvestigationConcurrencyLimitUpdater`, the concurrency limit, so a
+provider change applies without rebuilding the orchestrator. Both read the
+orchestrator before taking the service lock, because `PatrolService` takes its
+own lock before the service lock. Patrol's outer deadline for an automatic
+investigation follows the effective timeout plus a 5-minute grace instead of a
+fixed 10 minutes.
+
+The stuck-run sweep looks up the session of the run each running finding names
+among every session the orchestrator's store holds for the finding, so a newer
+session cannot hide a live run. Without the store it uses the finding's latest
+session, and leaves the finding alone when a different run is the latest. A
+live run is reset as
+`timed_out` only after its own `InvestigationSession.Deadline` plus that
+grace, or its start plus the current timeout and grace when no deadline is
+recorded. So a timeout lowered mid-run does not cut it short, and the previous
+run's `LastInvestigatedAt` no longer resets a live re-investigation. A run the
+orchestrator holds no session for, or closed as `interrupted` when it reloaded
+after a restart, is recorded as `interrupted`. Such a run keeps its attempt, so
+a run that crashes Pulse cannot retry forever. A run the orchestrator closed
+without updating the finding takes the result its session recorded, once the
+orchestrator's own update has had the grace to land, or `timed_out` when the
+session recorded none. Without an orchestrator the sweep keeps the
+`LastInvestigatedAt` rule. A first investigation cut short by a hard restart no
+longer stays running for good. `FindingsStore.RecoverStuckInvestigation`
+applies the recovery only while the finding still shows that run as running,
+so a result or a newer run that landed after the sweep looked is kept. A run
+that started before the finding's current occurrence is recorded only as
+`interrupted`, never with its session's result. An outcome the action
+lifecycle recorded on the finding during the run, such as
+`fix_queued`, is kept and the run completes, so a queued fix still blocks
+another investigation. The close stamps `LastInvestigatedAt` like any other
+closed run.
+
+Two outcomes join the vocabulary. `unclassified` records a completed run that
+left neither a typed conclusion nor a recognisable marker. The private
+orchestrator no longer defaults that case to `needs_attention`, which ends
+investigation of the finding. `unclassified` keeps the normal cooldown and
+attempt limit, presents as needs-attention in the loop state, and lands in the
+telemetry `other` bucket with no schema change. `interrupted` records a run
+cut short by a restart, shutdown or caller cancellation. It is not
+`timed_out`, has no cooldown, and reports as `in_progress`. A run cancelled by
+shutdown spends no attempt.
+
+A typed conclusion recorded before a later provider failure or summary timeout
+stands when the run holds no proposal or action. A failure without one remains
+failed, and fallback prose is still never classified as a conclusion. The
+prose-marker reader remains a fallback. It accepts only line-start markers
+inside the conclusion section. It ignores fenced and indented-code lines,
+reading any fence-like line as a fence and ending it only where every
+CommonMark reading would, and it treats a quote as running to the next blank
+line. Where that differs from CommonMark it hides more, which can only leave
+a run unclassified. It now tolerates markdown emphasis, list markers,
+numbering, case and heading punctuation.
+
+Proofs:
+- `agentic_investigation_conclusion_test.go`: final-turn tools, the summary
+ after a conclusion, the deadline reserve, and the reserve, summary-timeout
+ and subscription turn-budget helpers.
+- `tools_conclude_test.go`: the closed vocabulary, first conclusion wins,
+ profile and capture gating, and refusals.
+- `findings_test.go`: `interrupted` and `unclassified` eligibility.
+- `patrol_investigation_limits_test.go`: the sweep (live runs held to their
+ own deadline, a live or pending run behind a newer session, restart
+ reconciliation, the settle window, session results, kept action outcomes,
+ a result from before a regression, the guarded recovery write and the
+ no-orchestrator rule) and the run ceiling.
+- `service_patrol_limits_test.go`: provider-aware limits on reload.
+- `ai_config_test.go`, `contracts_test.go` (including the session `deadline`
+ wire shape), `patrol_cost_test.go` and the orchestrator-setup case in
+ `ai_handlers_investigation_additional_test.go`.
+
+This changes outcome recording and run limits only. It widens no mutation,
+approval or execution boundary, and it is not measured improvement on
+installed systems.
diff --git a/docs/release-control/v6/internal/subsystems/api-contracts.md b/docs/release-control/v6/internal/subsystems/api-contracts.md
index f2ea283cae..6fd8b32499 100644
--- a/docs/release-control/v6/internal/subsystems/api-contracts.md
+++ b/docs/release-control/v6/internal/subsystems/api-contracts.md
@@ -12586,3 +12586,25 @@ Routes, caller scope/token filtering, cache policy and probe count are unchanged
`TestContract_DiagnosticsAPIRejectionGuidanceSurvivesSerialization` cover real
HTTP statuses, private-body controls, same-credential fixture recovery and wire
attribution. They are not installed permission or guest-recovery acceptance.
+
+### Typed investigation conclusion tool (22 October 2026)
+
+`patrol_conclude_investigation` joins `patrol_propose_action` and
+`patrol_action_capabilities` as an investigation-only tool whose canonical name
+and descriptor live in `internal/agentcapabilities/`. It is statically
+read-kind and mutation-none, projected into the investigation scope surface,
+rejected by registry policy outside the investigation profile, and unavailable
+without the request-local capture. The model supplies only `outcome`
+(`resolved`, `cannot_fix` or `needs_attention`) and `reason`. The tool records
+the run's conclusion for the private orchestrator, which persists the outcome;
+it creates no proposal and grants no plan, approval or execution authority.
+The Pulse–enterprise boundary stays additive:
+`aicontracts.OrchestratorInvestigationResult.Conclusion`,
+`aicontracts.InvestigationConclusion`, the `unclassified` and `interrupted`
+outcome constants and the optional `InvestigationConcurrencyLimitUpdater` are
+new. `aicontracts.InvestigationSession.Deadline` (`deadline`, omitted when
+unrecorded) carries when a run's own timeout expires, so investigation session
+payloads gain one optional field. An older orchestrator ignores the conclusion
+and keeps reading prose markers, and records no deadline, so core measures its
+runs from their start. Proofs: `patrol_scope_tools_test.go`, `names_test.go`,
+`tools_conclude_test.go` and `contracts_test.go`.
diff --git a/docs/release-control/v6/internal/subsystems/storage-recovery.md b/docs/release-control/v6/internal/subsystems/storage-recovery.md
index 61df861998..1c75958192 100644
--- a/docs/release-control/v6/internal/subsystems/storage-recovery.md
+++ b/docs/release-control/v6/internal/subsystems/storage-recovery.md
@@ -7158,3 +7158,10 @@ The Proxmox page surface's Overview guest heading ("Guests on <node>", Show all
nodes) now renders through the shared `PlatformHostScopedTableTitle` that the
Docker and vSphere overviews also use; its Storage, Replication, Backups, Ceph
and Mail Gateway tabs, and every storage and recovery rule, are unchanged.
+
+The `internal/api` changes for typed Patrol investigation conclusions open no
+storage, backup or recovery path. They hand the run's typed conclusion to the
+private orchestrator, make investigation concurrency and timeout follow the
+Patrol provider, and label the `unclassified` and `interrupted` outcomes in the
+digest email. Recovery evidence, restore rules and storage capabilities are
+unchanged, and Patrol gains no storage remediation capability.
diff --git a/internal/agentcapabilities/invocation.go b/internal/agentcapabilities/invocation.go
index d9f121ef6c..9c89aac6dc 100644
--- a/internal/agentcapabilities/invocation.go
+++ b/internal/agentcapabilities/invocation.go
@@ -223,6 +223,9 @@ var registryInvocationDescriptors = map[string]InvocationDescriptor{
PatrolReportFindingToolName: staticClass(ToolCallKindWrite, MutationPulseState),
PatrolResolveFindingToolName: staticClass(ToolCallKindWrite, MutationPulseState),
PatrolProposeObserverToolName: staticClass(ToolCallKindWrite, MutationPulseState),
+ // patrol_conclude_investigation only records the run's typed conclusion
+ // in the request-local capture; the orchestrator persists the outcome.
+ PatrolConcludeInvestigationToolName: staticClass(ToolCallKindRead, MutationNone),
}
// Clone returns a deep copy of the descriptor so callers can never
diff --git a/internal/agentcapabilities/patrol_scope_tools.go b/internal/agentcapabilities/patrol_scope_tools.go
index 2479a3ffc4..f707677faf 100644
--- a/internal/agentcapabilities/patrol_scope_tools.go
+++ b/internal/agentcapabilities/patrol_scope_tools.go
@@ -111,8 +111,9 @@ func PatrolDetectionToolNamesForResourceTypes(resourceTypes []string) ([]string,
}
// PatrolInvestigationToolNamesForResourceTypes composes scoped evidence with
-// the side-effect-free capability lookup and typed proposal sink. Planning,
-// approval, execution, and verification remain outside this surface.
+// the side-effect-free capability lookup, typed proposal sink and typed
+// conclusion capture. Planning, approval, execution, and verification remain
+// outside this surface.
func PatrolInvestigationToolNamesForResourceTypes(resourceTypes []string) ([]string, bool) {
names, ok := PatrolEvidenceToolNamesForResourceTypes(resourceTypes)
if !ok {
@@ -121,5 +122,6 @@ func PatrolInvestigationToolNamesForResourceTypes(resourceTypes []string) ([]str
return append(names,
PatrolActionCapabilitiesToolName,
PatrolProposeActionToolName,
+ PatrolConcludeInvestigationToolName,
), true
}
diff --git a/internal/agentcapabilities/patrol_scope_tools_test.go b/internal/agentcapabilities/patrol_scope_tools_test.go
index 35b782172c..cbd3839984 100644
--- a/internal/agentcapabilities/patrol_scope_tools_test.go
+++ b/internal/agentcapabilities/patrol_scope_tools_test.go
@@ -23,7 +23,7 @@ func TestPatrolScopedToolProjectionUsesTypedLeastManifest(t *testing.T) {
if !ok {
t.Fatal("physical-disk scope was not recognized")
}
- for _, required := range []string{PulseQueryToolName, PulseReadToolName, PulseMetricsToolName, PulseStorageToolName, PatrolActionCapabilitiesToolName, PatrolProposeActionToolName} {
+ for _, required := range []string{PulseQueryToolName, PulseReadToolName, PulseMetricsToolName, PulseStorageToolName, PatrolActionCapabilitiesToolName, PatrolProposeActionToolName, PatrolConcludeInvestigationToolName} {
if !slices.Contains(storageInvestigation, required) {
t.Fatalf("storage investigation tools missing %q: %v", required, storageInvestigation)
}
@@ -61,6 +61,9 @@ func TestPatrolScopedToolProjectionComposesProfileOwnedTools(t *testing.T) {
if slices.Contains(detection, PatrolProposeActionToolName) {
t.Fatalf("detection surface gained investigation proposal authority: %v", detection)
}
+ if slices.Contains(detection, PatrolConcludeInvestigationToolName) {
+ t.Fatalf("detection surface gained the investigation conclusion capture: %v", detection)
+ }
}
func TestGuestInvestigationsCanReadCollectedBackupEvidence(t *testing.T) {
@@ -71,7 +74,7 @@ func TestGuestInvestigationsCanReadCollectedBackupEvidence(t *testing.T) {
t.Fatalf("guest investigation lacks backup evidence: %v", names)
}
for _, name := range names {
- if !IsPatrolInfrastructureEvidenceToolName(name) && name != PatrolActionCapabilitiesToolName && name != PatrolProposeActionToolName {
+ if !IsPatrolInfrastructureEvidenceToolName(name) && name != PatrolActionCapabilitiesToolName && name != PatrolProposeActionToolName && name != PatrolConcludeInvestigationToolName {
t.Fatalf("read projection gained action authority: %s", name)
}
}
diff --git a/internal/agentcapabilities/tool_names.go b/internal/agentcapabilities/tool_names.go
index 864925603d..5ca83af966 100644
--- a/internal/agentcapabilities/tool_names.go
+++ b/internal/agentcapabilities/tool_names.go
@@ -49,4 +49,10 @@ const (
// model choose an action target from evidence instead of pinning
// remediation to the finding's initial resource.
PatrolActionCapabilitiesToolName = "patrol_action_capabilities"
+ // PatrolConcludeInvestigationToolName is the investigation-only,
+ // side-effect-free conclusion capture. It records one typed zero-action
+ // conclusion (resolved, cannot_fix or needs_attention) in the
+ // request-local capture sink so the orchestrator never has to infer the
+ // outcome from prose. It grants no planning, approval or execution.
+ PatrolConcludeInvestigationToolName = "patrol_conclude_investigation"
)
diff --git a/internal/ai/chat/agentic.go b/internal/ai/chat/agentic.go
index 792ed4976d..d759e2c418 100644
--- a/internal/ai/chat/agentic.go
+++ b/internal/ai/chat/agentic.go
@@ -812,6 +812,18 @@ func (a *AgenticLoop) executeWithTools(ctx context.Context, sessionID string, me
// limit check below owns compaction when the request actually needs it.
currentTurnStartIndex := len(providerMessages)
+ // A Patrol investigation reserves time to conclude before its deadline.
+ // Once another evidence turn would leave too little for the conclusion
+ // and its summary, the current turn becomes the final turn, so the run
+ // ends with a recorded conclusion instead of a timeout.
+ investigationDeadline, investigationHasDeadline := ctx.Deadline()
+ investigationHasDeadline = investigationHasDeadline && isPatrolInvestigationExecution(a.currentExecutionProfile())
+ var investigationBudget, slowestInvestigationTurn time.Duration
+ var investigationTurnStartedAt time.Time
+ if investigationHasDeadline {
+ investigationBudget = time.Until(investigationDeadline)
+ }
+
for turn < maxTurns ||
(patrolOutputLimitRecoveryPending && !patrolOutputLimitRecoveryAttempted) ||
(investigationOutputLimitRecoveryPending && !investigationOutputLimitRecoveryAttempted) {
@@ -863,6 +875,24 @@ func (a *AgenticLoop) executeWithTools(ctx context.Context, sessionID string, me
default:
}
+ if investigationHasDeadline {
+ now := time.Now()
+ if !investigationTurnStartedAt.IsZero() {
+ slowestInvestigationTurn = max(slowestInvestigationTurn, now.Sub(investigationTurnStartedAt))
+ }
+ investigationTurnStartedAt = now
+ remaining := investigationDeadline.Sub(now)
+ if turn < maxTurns-1 && remaining <= investigationConclusionReserve(slowestInvestigationTurn, investigationBudget) {
+ log.Warn().
+ Int("turn", turn).
+ Dur("remaining", remaining).
+ Dur("slowest_turn", slowestInvestigationTurn).
+ Str("session_id", sessionID).
+ Msg("[AgenticLoop] Investigation deadline approaching — making this the final turn")
+ maxTurns = turn + 1
+ }
+ }
+
log.Info().
Int("turn", turn).
Int("messages", len(providerMessages)).
@@ -884,7 +914,7 @@ func (a *AgenticLoop) executeWithTools(ctx context.Context, sessionID string, me
SystemCacheablePrefix: stablePrompt,
Tools: tools,
ExecutionID: a.executionID,
- StreamIdleTimeout: streamIdleTimeout,
+ StreamIdleTimeout: investigationStreamIdleTimeout(a.currentExecutionProfile(), providerName, streamIdleTimeout),
}
if isPatrolDetectionExecution(a.currentExecutionProfile()) && patrolFindingsReadCompleted {
req.Tools = withoutProviderTool(req.Tools, agentcapabilities.PatrolGetFindingsToolName)
@@ -941,6 +971,9 @@ func (a *AgenticLoop) executeWithTools(ctx context.Context, sessionID string, me
Int("max_turns", maxTurns).
Str("session_id", sessionID).
Msg("[AgenticLoop] Approaching max turns — restricting Watch to finding decisions")
+ } else if isPatrolInvestigationExecution(a.currentExecutionProfile()) && !investigationOutputLimitRecoveryTurn {
+ // The investigation block below restricts the final turn to the
+ // tools that conclude the run.
} else {
req.Tools = nil
textOnlySafetyBrake = true
@@ -964,8 +997,30 @@ func (a *AgenticLoop) executeWithTools(ctx context.Context, sessionID string, me
if a.maxEvidenceCalls > 0 {
req.System += fmt.Sprintf("\nEvidence-tool calls remaining within this run's configured limit: %d.", max(0, a.maxEvidenceCalls-a.totalEvidenceCalls))
}
- if a.maxEvidenceCalls > 0 && a.totalEvidenceCalls >= a.maxEvidenceCalls {
- req.Tools = investigationTerminalTools(tools)
+ evidenceBudgetSpent := a.maxEvidenceCalls > 0 && a.totalEvidenceCalls >= a.maxEvidenceCalls
+ switch {
+ case a.executor.InvestigationConcluded():
+ // The conclusion is recorded; the next response is its summary.
+ req.Tools = nil
+ textOnlySafetyBrake = true
+ case turn >= maxTurns-1:
+ // Final turn (turn limit or deadline reserve): record a typed
+ // conclusion rather than leave the outcome to be inferred from
+ // prose. A proposal stays available only once evidence is spent,
+ // as before.
+ req.Tools = investigationTerminalTools(tools, evidenceBudgetSpent)
+ textOnlySafetyBrake = len(req.Tools) == 0
+ if !textOnlySafetyBrake {
+ req.System += investigationFinalConclusionSystemNote
+ }
+ log.Warn().
+ Int("turn", turn).
+ Int("max_turns", maxTurns).
+ Int("tools", len(req.Tools)).
+ Str("session_id", sessionID).
+ Msg("[AgenticLoop] Final investigation turn — restricting tools to the conclusion")
+ case evidenceBudgetSpent:
+ req.Tools = investigationTerminalTools(tools, true)
textOnlySafetyBrake = len(req.Tools) == 0
}
}
diff --git a/internal/ai/chat/agentic_additional_test.go b/internal/ai/chat/agentic_additional_test.go
index ce8f48ddcd..a8641c8d05 100644
--- a/internal/ai/chat/agentic_additional_test.go
+++ b/internal/ai/chat/agentic_additional_test.go
@@ -137,10 +137,22 @@ func TestInvestigationEvidenceBudgetHelpers(t *testing.T) {
{Name: agentcapabilities.PatrolProposeActionToolName},
{Name: agentcapabilities.PatrolActionCapabilitiesToolName},
}
- terminal := investigationTerminalTools(available)
+ terminal := investigationTerminalTools(available, true)
if len(terminal) != 1 || terminal[0].Name != agentcapabilities.PatrolProposeActionToolName {
t.Fatalf("terminal tools = %+v, want only patrol_propose_action", terminal)
}
+ withConclusion := append(available, providers.Tool{Name: agentcapabilities.PatrolConcludeInvestigationToolName})
+ terminal = investigationTerminalTools(withConclusion, true)
+ if len(terminal) != 2 || terminal[0].Name != agentcapabilities.PatrolProposeActionToolName || terminal[1].Name != agentcapabilities.PatrolConcludeInvestigationToolName {
+ t.Fatalf("terminal tools = %+v, want patrol_propose_action then patrol_conclude_investigation", terminal)
+ }
+ terminal = investigationTerminalTools(withConclusion, false)
+ if len(terminal) != 1 || terminal[0].Name != agentcapabilities.PatrolConcludeInvestigationToolName {
+ t.Fatalf("final-turn terminal tools = %+v, want only patrol_conclude_investigation", terminal)
+ }
+ if isInvestigationEvidenceTool(agentcapabilities.PatrolConcludeInvestigationToolName) {
+ t.Fatal("recording a conclusion must not consume evidence budget")
+ }
if !isInvestigationEvidenceTool(agentcapabilities.PulseQueryToolName) {
t.Fatal("read-only query must consume evidence budget")
}
diff --git a/internal/ai/chat/agentic_final.go b/internal/ai/chat/agentic_final.go
index c162f673b5..aadc6d7ce0 100644
--- a/internal/ai/chat/agentic_final.go
+++ b/internal/ai/chat/agentic_final.go
@@ -94,8 +94,9 @@ func (a *AgenticLoop) ensureFinalTextResponseWithSystemPrompt(
var suppressLeakedToolContent bool
var pendingVisibleContent string
- // Keep this bounded so a stuck provider stream doesn't drag the whole request.
- summaryCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
+ // Keep this bounded so a stuck provider stream doesn't drag the whole
+ // request. An investigation may use the time left before its deadline.
+ summaryCtx, cancel := context.WithTimeout(ctx, investigationSummaryTimeout(ctx, a.currentExecutionProfile(), 20*time.Second))
defer cancel()
summaryErr := a.provider.ChatStream(summaryCtx, summaryReq, func(event providers.StreamEvent) {
diff --git a/internal/ai/chat/agentic_investigation_budget.go b/internal/ai/chat/agentic_investigation_budget.go
index e2dc990c14..29f3f06bb4 100644
--- a/internal/ai/chat/agentic_investigation_budget.go
+++ b/internal/ai/chat/agentic_investigation_budget.go
@@ -1,6 +1,9 @@
package chat
import (
+ "context"
+ "time"
+
"github.com/rcourtman/pulse-go-rewrite/internal/agentcapabilities"
"github.com/rcourtman/pulse-go-rewrite/internal/ai/providers"
aitools "github.com/rcourtman/pulse-go-rewrite/internal/ai/tools"
@@ -30,11 +33,88 @@ func isInvestigationEvidenceTool(name string) bool {
return agentcapabilities.IsPatrolInfrastructureEvidenceToolName(name)
}
-func investigationTerminalTools(available []providers.Tool) []providers.Tool {
+// investigationTerminalTools are the tools that end an investigation: the
+// typed action proposal and the typed conclusion. includePropose is false on
+// a final turn that historically offered no proposal.
+func investigationTerminalTools(available []providers.Tool, includePropose bool) []providers.Tool {
+ var terminal []providers.Tool
+ seen := make(map[string]bool, 2)
for _, tool := range available {
- if tool.Name == agentcapabilities.PatrolProposeActionToolName {
- return []providers.Tool{tool}
+ if seen[tool.Name] {
+ continue
+ }
+ switch tool.Name {
+ case agentcapabilities.PatrolProposeActionToolName:
+ if !includePropose {
+ continue
+ }
+ case agentcapabilities.PatrolConcludeInvestigationToolName:
+ default:
+ continue
}
+ seen[tool.Name] = true
+ terminal = append(terminal, tool)
+ }
+ return terminal
+}
+
+const investigationFinalConclusionSystemNote = "\nThis is the investigation's final turn. Do not gather more evidence. Record your conclusion now with patrol_conclude_investigation, based only on the evidence already collected; you will then be asked for the final summary."
+
+// Turn-time estimates for reserving a conclusion before the investigation
+// deadline. Concluding takes up to two provider calls: the conclusion turn
+// and the final summary.
+const (
+ investigationMinTurnEstimate = 20 * time.Second
+ investigationConclusionTimeMargin = 15 * time.Second
+)
+
+// investigationConclusionReserve is how much of the deadline must remain to
+// start another evidence turn: that turn plus the conclusion and summary
+// turns, each estimated as the slowest turn seen so far, plus a margin. It is
+// capped at half the run's budget so short timeouts still gather evidence.
+func investigationConclusionReserve(slowestTurn, budget time.Duration) time.Duration {
+ reserve := 3*max(slowestTurn, investigationMinTurnEstimate) + investigationConclusionTimeMargin
+ if budget > 0 && reserve > budget/2 {
+ reserve = budget / 2
+ }
+ return reserve
+}
+
+// patrolSubscriptionInvestigationTurnTimeout is the complete-turn budget for an
+// investigation on a subscription agent. That transport buffers the whole turn,
+// so Patrol's stream idle timeout is the deadline for the entire turn. Watch
+// keeps its short stall budget for latency; an investigation turn starts a CLI
+// process, resends the conversation and reasons at high effort.
+const patrolSubscriptionInvestigationTurnTimeout = 4 * time.Minute
+
+// investigationStreamIdleTimeout returns the caller-owned provider stall
+// budget for one turn. Only an investigation on a buffered subscription
+// transport is widened; every other profile and provider keeps base.
+func investigationStreamIdleTimeout(profile aitools.ExecutionProfile, providerName string, base time.Duration) time.Duration {
+ if !isPatrolInvestigationExecution(profile) || base <= 0 || base >= patrolSubscriptionInvestigationTurnTimeout {
+ return base
+ }
+ switch providers.SubscriptionAgent(providerName) {
+ case providers.SubscriptionAgentCodex, providers.SubscriptionAgentClaude:
+ return patrolSubscriptionInvestigationTurnTimeout
+ default:
+ return base
+ }
+}
+
+// investigationSummaryTimeout bounds the final summary call. Interactive chat
+// keeps the historical 20s bound; an investigation may use the time left
+// before its deadline, because a slow provider's summary is its result.
+func investigationSummaryTimeout(ctx context.Context, profile aitools.ExecutionProfile, base time.Duration) time.Duration {
+ if !isPatrolInvestigationExecution(profile) {
+ return base
+ }
+ deadline, ok := ctx.Deadline()
+ if !ok {
+ return base
+ }
+ if remaining := time.Until(deadline); remaining > base {
+ return remaining
}
- return nil
+ return base
}
diff --git a/internal/ai/chat/agentic_investigation_budget_w0716_coverage_test.go b/internal/ai/chat/agentic_investigation_budget_w0716_coverage_test.go
index c396f64ffa..4694f449b0 100644
--- a/internal/ai/chat/agentic_investigation_budget_w0716_coverage_test.go
+++ b/internal/ai/chat/agentic_investigation_budget_w0716_coverage_test.go
@@ -71,7 +71,7 @@ func Test_w0716_budget_InvestigationTerminalTools(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
- got := investigationTerminalTools(tt.available)
+ got := investigationTerminalTools(tt.available, true)
if !tt.wantFound {
if got != nil {
t.Fatalf("investigationTerminalTools() = %+v, want nil", got)
diff --git a/internal/ai/chat/agentic_investigation_conclusion_test.go b/internal/ai/chat/agentic_investigation_conclusion_test.go
new file mode 100644
index 0000000000..3200c60621
--- /dev/null
+++ b/internal/ai/chat/agentic_investigation_conclusion_test.go
@@ -0,0 +1,211 @@
+package chat
+
+import (
+ "context"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/rcourtman/pulse-go-rewrite/internal/agentcapabilities"
+ "github.com/rcourtman/pulse-go-rewrite/internal/ai/providers"
+ "github.com/rcourtman/pulse-go-rewrite/internal/ai/tools"
+ "github.com/rcourtman/pulse-go-rewrite/pkg/aicontracts"
+)
+
+func investigationConclusionTestTools() []providers.Tool {
+ return []providers.Tool{
+ {Name: agentcapabilities.PulseQueryToolName},
+ {Name: agentcapabilities.PatrolProposeActionToolName},
+ {Name: agentcapabilities.PatrolConcludeInvestigationToolName},
+ }
+}
+
+func newConclusionTestExecutor(capture *tools.ProposalCapture) *tools.PulseToolExecutor {
+ executor := tools.NewPulseToolExecutor(tools.ExecutorConfig{StateProvider: &mockStateProvider{}})
+ executor.ApplyExecutionProfile(tools.ProfilePatrolInvestigation)
+ if capture != nil {
+ executor.SetProposalCapture(capture)
+ }
+ return executor
+}
+
+func toolNames(list []providers.Tool) []string {
+ names := make([]string, 0, len(list))
+ for _, tool := range list {
+ names = append(names, tool.Name)
+ }
+ return names
+}
+
+func TestAgenticLoopPatrolInvestigationFinalTurnOffersOnlyTheConclusion(t *testing.T) {
+ provider := &stubStreamingProvider{}
+ var requests []providers.ChatRequest
+ provider.chatStream = func(_ context.Context, req providers.ChatRequest, callback providers.StreamCallback) error {
+ requests = append(requests, req)
+ if len(requests) == 1 {
+ callback(providers.StreamEvent{Type: "done", Data: providers.DoneEvent{ToolCalls: []providers.ToolCall{{ID: "evidence-1", Name: agentcapabilities.PulseQueryToolName, Input: map[string]interface{}{"action": "search", "query": "disks"}}}}})
+ return nil
+ }
+ callback(providers.StreamEvent{Type: "content", Data: providers.ContentEvent{Text: "final investigation summary"}})
+ callback(providers.StreamEvent{Type: "done", Data: providers.DoneEvent{}})
+ return nil
+ }
+
+ loop := NewAgenticLoop(provider, newConclusionTestExecutor(nil), "system")
+ loop.SetExecutionProfile(tools.ProfilePatrolInvestigation)
+ loop.SetMaxTurns(2)
+
+ if _, err := loop.ExecuteWithTools(context.Background(), "final-turn", []Message{{Role: "user", Content: "investigate"}}, investigationConclusionTestTools(), func(StreamEvent) {}); err != nil {
+ t.Fatalf("ExecuteWithTools: %v", err)
+ }
+ if len(requests) != 2 {
+ t.Fatalf("provider requests = %d, want 2", len(requests))
+ }
+ if names := toolNames(requests[1].Tools); len(names) != 1 || names[0] != agentcapabilities.PatrolConcludeInvestigationToolName {
+ t.Fatalf("final-turn tools = %v, want only patrol_conclude_investigation (no proposal before the evidence budget is spent)", names)
+ }
+ if !strings.Contains(requests[1].System, "final turn") {
+ t.Fatalf("final-turn system prompt missing the conclusion instruction: %q", requests[1].System)
+ }
+}
+
+func TestAgenticLoopPatrolInvestigationAsksForSummaryAfterConclusion(t *testing.T) {
+ provider := &stubStreamingProvider{}
+ var requests []providers.ChatRequest
+ provider.chatStream = func(_ context.Context, req providers.ChatRequest, callback providers.StreamCallback) error {
+ requests = append(requests, req)
+ if len(requests) == 1 {
+ callback(providers.StreamEvent{Type: "done", Data: providers.DoneEvent{ToolCalls: []providers.ToolCall{{
+ ID: "conclude-1",
+ Name: agentcapabilities.PatrolConcludeInvestigationToolName,
+ Input: map[string]interface{}{"outcome": "needs_attention", "reason": "the pool needs a replacement disk"},
+ }}}})
+ return nil
+ }
+ callback(providers.StreamEvent{Type: "content", Data: providers.ContentEvent{Text: "### Investigation Summary\nThe pool is degraded."}})
+ callback(providers.StreamEvent{Type: "done", Data: providers.DoneEvent{}})
+ return nil
+ }
+
+ capture := tools.NewProposalCapture(tools.ProposalIdentity{ProposalID: "p1", FindingID: "f1", InvestigationID: "i1"}, nil)
+ loop := NewAgenticLoop(provider, newConclusionTestExecutor(capture), "system")
+ loop.SetExecutionProfile(tools.ProfilePatrolInvestigation)
+ loop.SetMaxTurns(6)
+
+ result, err := loop.ExecuteWithTools(context.Background(), "conclude-then-summarize", []Message{{Role: "user", Content: "investigate"}}, investigationConclusionTestTools(), func(StreamEvent) {})
+ if err != nil {
+ t.Fatalf("ExecuteWithTools: %v", err)
+ }
+ conclusion := capture.Conclusion()
+ if conclusion == nil || conclusion.Outcome != aicontracts.OutcomeNeedsAttention {
+ t.Fatalf("recorded conclusion = %+v, want needs_attention", conclusion)
+ }
+ if len(requests) != 2 {
+ t.Fatalf("provider requests = %d, want 2 (conclusion, then summary)", len(requests))
+ }
+ if len(requests[1].Tools) != 0 {
+ t.Fatalf("post-conclusion tools = %v, want none: the next response is the summary", toolNames(requests[1].Tools))
+ }
+ if len(result) == 0 || !strings.Contains(result[len(result)-1].Content, "The pool is degraded.") {
+ t.Fatalf("final result = %+v, want the summary", result)
+ }
+}
+
+func TestAgenticLoopPatrolInvestigationConcludesBeforeItsDeadline(t *testing.T) {
+ provider := &stubStreamingProvider{}
+ var requests []providers.ChatRequest
+ provider.chatStream = func(_ context.Context, req providers.ChatRequest, callback providers.StreamCallback) error {
+ requests = append(requests, req)
+ if len(requests) == 1 {
+ // A slow evidence turn leaves less than the conclusion reserve.
+ time.Sleep(1100 * time.Millisecond)
+ callback(providers.StreamEvent{Type: "done", Data: providers.DoneEvent{ToolCalls: []providers.ToolCall{{ID: "evidence-1", Name: agentcapabilities.PulseQueryToolName, Input: map[string]interface{}{"action": "search", "query": "disks"}}}}})
+ return nil
+ }
+ callback(providers.StreamEvent{Type: "content", Data: providers.ContentEvent{Text: "final investigation summary"}})
+ callback(providers.StreamEvent{Type: "done", Data: providers.DoneEvent{}})
+ return nil
+ }
+
+ loop := NewAgenticLoop(provider, newConclusionTestExecutor(nil), "system")
+ loop.SetExecutionProfile(tools.ProfilePatrolInvestigation)
+ loop.SetMaxTurns(12)
+
+ // A 2s budget reserves 1s (half the budget) to conclude.
+ ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
+ defer cancel()
+ if _, err := loop.ExecuteWithTools(ctx, "deadline-reserve", []Message{{Role: "user", Content: "investigate"}}, investigationConclusionTestTools(), func(StreamEvent) {}); err != nil {
+ t.Fatalf("ExecuteWithTools: %v", err)
+ }
+ if len(requests) != 2 {
+ t.Fatalf("provider requests = %d, want 2", len(requests))
+ }
+ if names := toolNames(requests[1].Tools); len(names) != 1 || names[0] != agentcapabilities.PatrolConcludeInvestigationToolName {
+ t.Fatalf("tools after the reserve was reached = %v, want only patrol_conclude_investigation", names)
+ }
+}
+
+func TestInvestigationConclusionReserve(t *testing.T) {
+ cases := []struct {
+ name string
+ slowest time.Duration
+ budget time.Duration
+ want time.Duration
+ }{
+ {name: "no turn seen yet uses the minimum estimate", slowest: 0, budget: 10 * time.Minute, want: 75 * time.Second},
+ // Another evidence turn plus the conclusion and summary turns.
+ {name: "slow subscription turns reserve three of them", slowest: 90 * time.Second, budget: 25 * time.Minute, want: 285 * time.Second},
+ {name: "near-limit subscription turns still fit", slowest: 225 * time.Second, budget: 25 * time.Minute, want: 690 * time.Second},
+ {name: "short budgets keep half for evidence", slowest: 0, budget: time.Minute, want: 30 * time.Second},
+ }
+ for _, tc := range cases {
+ if got := investigationConclusionReserve(tc.slowest, tc.budget); got != tc.want {
+ t.Fatalf("%s: reserve = %s, want %s", tc.name, got, tc.want)
+ }
+ }
+}
+
+func TestInvestigationStreamIdleTimeoutWidensOnlySubscriptionInvestigationTurns(t *testing.T) {
+ base := PatrolProviderStreamIdleTimeout
+ cases := []struct {
+ name string
+ profile tools.ExecutionProfile
+ provider string
+ want time.Duration
+ }{
+ {name: "codex subscription investigation", profile: tools.ProfilePatrolInvestigation, provider: string(providers.SubscriptionAgentCodex), want: patrolSubscriptionInvestigationTurnTimeout},
+ {name: "claude subscription investigation", profile: tools.ProfilePatrolInvestigation, provider: string(providers.SubscriptionAgentClaude), want: patrolSubscriptionInvestigationTurnTimeout},
+ {name: "subscription Watch keeps its stall budget", profile: tools.ProfilePatrolDetection, provider: string(providers.SubscriptionAgentCodex), want: base},
+ {name: "streaming provider investigation keeps its idle budget", profile: tools.ProfilePatrolInvestigation, provider: "anthropic", want: base},
+ }
+ for _, tc := range cases {
+ if got := investigationStreamIdleTimeout(tc.profile, tc.provider, base); got != tc.want {
+ t.Fatalf("%s: stream idle timeout = %s, want %s", tc.name, got, tc.want)
+ }
+ }
+ if got := investigationStreamIdleTimeout(tools.ProfileInteractiveAssistant, string(providers.SubscriptionAgentCodex), 0); got != 0 {
+ t.Fatalf("interactive runs without a stall budget must stay unbounded, got %s", got)
+ }
+}
+
+func TestInvestigationSummaryTimeoutUsesTheRemainingDeadline(t *testing.T) {
+ base := 20 * time.Second
+ if got := investigationSummaryTimeout(context.Background(), tools.ProfilePatrolInvestigation, base); got != base {
+ t.Fatalf("no deadline: timeout = %s, want %s", got, base)
+ }
+
+ long, cancelLong := context.WithTimeout(context.Background(), 5*time.Minute)
+ defer cancelLong()
+ if got := investigationSummaryTimeout(long, tools.ProfilePatrolInvestigation, base); got <= 4*time.Minute {
+ t.Fatalf("investigation summary timeout = %s, want the remaining ~5m", got)
+ }
+ if got := investigationSummaryTimeout(long, tools.ProfileInteractiveAssistant, base); got != base {
+ t.Fatalf("interactive summary timeout = %s, want %s", got, base)
+ }
+
+ short, cancelShort := context.WithTimeout(context.Background(), 5*time.Second)
+ defer cancelShort()
+ if got := investigationSummaryTimeout(short, tools.ProfilePatrolInvestigation, base); got != base {
+ t.Fatalf("near-deadline summary timeout = %s, want %s (the deadline still bounds it)", got, base)
+ }
+}
diff --git a/internal/ai/chat/service_investigation.go b/internal/ai/chat/service_investigation.go
index aa85733fd5..70997806ae 100644
--- a/internal/ai/chat/service_investigation.go
+++ b/internal/ai/chat/service_investigation.go
@@ -53,6 +53,10 @@ type InvestigationRunResult struct {
ModelTurns int
EvidenceCalls int
ToolCalls int
+ // Conclusion is the typed conclusion the model recorded with
+ // patrol_conclude_investigation, nil when it recorded none. It survives
+ // a later provider failure, like a persisted proposal.
+ Conclusion *tools.CapturedConclusion
}
// ExecuteInvestigationStream runs one Patrol investigation under the
@@ -193,6 +197,7 @@ func (s *Service) ExecuteInvestigationStream(ctx context.Context, req Investigat
result := &InvestigationRunResult{
Content: content,
Proposal: proposal,
+ Conclusion: capture.Conclusion(),
InputTokens: loop.GetTotalInputTokens(),
OutputTokens: loop.GetTotalOutputTokens(),
ModelTurns: loop.GetTotalModelTurns(),
diff --git a/internal/ai/findings.go b/internal/ai/findings.go
index 1c0fefb1cb..4126defa10 100644
--- a/internal/ai/findings.go
+++ b/internal/ai/findings.go
@@ -137,6 +137,8 @@ const (
InvestigationOutcomeNeedsAttention InvestigationOutcome = "needs_attention" // Requires user intervention
InvestigationOutcomeCannotFix InvestigationOutcome = "cannot_fix" // AI determined it cannot fix this
InvestigationOutcomeTimedOut InvestigationOutcome = "timed_out" // Transient timeout, will retry sooner
+ InvestigationOutcomeUnclassified InvestigationOutcome = "unclassified" // Finished, but its conclusion could not be read
+ InvestigationOutcomeInterrupted InvestigationOutcome = "interrupted" // Cancelled by a restart or shutdown, will retry
)
// FindingLifecycleEvent represents an append-only log entry of finding lifecycle transitions.
@@ -457,6 +459,9 @@ func investigationCooldownForOutcome(outcome string) time.Duration {
return 0
case InvestigationOutcomeTimedOut:
return investigationTimeoutCooldown
+ case InvestigationOutcomeInterrupted:
+ // A restart cut the run short; nothing about the finding changed.
+ return 0
case InvestigationOutcomeFixFailed, InvestigationOutcomeFixVerificationFailed:
return investigationFixRetryCooldown
default:
@@ -466,7 +471,9 @@ func investigationCooldownForOutcome(outcome string) time.Duration {
func (f *Finding) investigationCooldown() time.Duration {
if InvestigationStatus(f.InvestigationStatus) == InvestigationStatusFailed &&
- (f.InvestigationOutcome == "" || InvestigationOutcome(f.InvestigationOutcome) == InvestigationOutcomeTimedOut) {
+ (f.InvestigationOutcome == "" ||
+ InvestigationOutcome(f.InvestigationOutcome) == InvestigationOutcomeTimedOut ||
+ InvestigationOutcome(f.InvestigationOutcome) == InvestigationOutcomeInterrupted) {
// The run failed before producing a conclusion. The scheduled Watch
// cadence owns the next assessment; there is no additional cooldown.
// Admission, attempt limits and action authority still apply.
@@ -505,7 +512,9 @@ func deriveLoopState(f *Finding) FindingLoopState {
return FindingLoopStateRemediating
case InvestigationOutcomeFixFailed, InvestigationOutcomeFixVerificationFailed:
return FindingLoopStateRemediationFailed
- case InvestigationOutcomeFixRejected, InvestigationOutcomeFixVerificationUnknown, InvestigationOutcomeNeedsAttention, InvestigationOutcomeCannotFix:
+ case InvestigationOutcomeFixRejected, InvestigationOutcomeFixVerificationUnknown, InvestigationOutcomeNeedsAttention, InvestigationOutcomeCannotFix, InvestigationOutcomeUnclassified:
+ // An unclassified run still left a summary for the operator to read;
+ // unlike needs_attention it stays eligible for another run.
return FindingLoopStateNeedsAttention
case InvestigationOutcomeTimedOut:
return FindingLoopStateTimedOut
@@ -2679,6 +2688,50 @@ func (s *FindingsStore) UpdateInvestigation(id, sessionID, status, outcome strin
return true
}
+// RecoverStuckInvestigation closes a run that is no longer live with the given
+// status and outcome, but only while the finding still shows that run as
+// running, so a result or a newer run that landed since the caller looked is
+// never overwritten. runStartedAt is the start of the run's session, or zero
+// when the orchestrator holds none. A run that started before the finding's
+// current occurrence is only recorded as interrupted: its result is not this
+// occurrence's. A run starts with no outcome, so an outcome already present
+// came from the action lifecycle during this run; it is kept, and the run
+// completes, so a queued fix still blocks another investigation. The close
+// becomes the finding's latest investigation time.
+func (s *FindingsStore) RecoverStuckInvestigation(id, sessionID string, runStartedAt time.Time, status InvestigationStatus, outcome InvestigationOutcome, at time.Time) bool {
+ s.mu.Lock()
+ f, exists := s.findings[id]
+ if !exists || f.InvestigationStatus != string(InvestigationStatusRunning) || f.InvestigationSessionID != sessionID {
+ s.mu.Unlock()
+ return false
+ }
+ if !runStartedAt.IsZero() && !f.acceptsInvestigationResult(sessionID, runStartedAt) {
+ status, outcome = InvestigationStatusFailed, InvestigationOutcomeInterrupted
+ }
+ prevStatus := f.InvestigationStatus
+ prevOutcome := f.InvestigationOutcome
+ if prevOutcome != "" {
+ status = InvestigationStatusCompleted
+ } else {
+ f.InvestigationOutcome = string(outcome)
+ if outcome == InvestigationOutcomeFixVerified {
+ s.resolveVerifiedFindingLocked(f, prevOutcome)
+ }
+ }
+ f.InvestigationStatus = string(status)
+ f.LastInvestigatedAt = &at
+ s.appendLifecycleLocked(f, "investigation_updated", "Investigation state updated", f.LoopState, f.LoopState, map[string]string{
+ "prev_status": prevStatus,
+ "next_status": string(status),
+ "prev_outcome": prevOutcome,
+ "next_outcome": f.InvestigationOutcome,
+ })
+ s.syncLoopStateLocked(f)
+ s.mu.Unlock()
+ s.scheduleSave()
+ return true
+}
+
// resolveVerifiedFindingLocked projects the first verified resolution without
// moving its timestamp or appending another verification when history is replayed.
// A genuine regression clears ResolvedAt and can therefore resolve again.
diff --git a/internal/ai/findings_test.go b/internal/ai/findings_test.go
index 5edd24d25e..0210625557 100644
--- a/internal/ai/findings_test.go
+++ b/internal/ai/findings_test.go
@@ -2443,3 +2443,43 @@ func TestFindingsStore_OperatorStateProjectionFor_RejectsEmptyResourceID(t *test
t.Error("expected false on empty resource id")
}
}
+
+func newEligibleInvestigationFinding(status InvestigationStatus, outcome InvestigationOutcome, lastInvestigated time.Time) *Finding {
+ return &Finding{
+ ID: "eligible-" + string(outcome),
+ Severity: FindingSeverityWarning,
+ InvestigationStatus: string(status),
+ InvestigationOutcome: string(outcome),
+ InvestigationAttempts: 1,
+ LastInvestigatedAt: &lastInvestigated,
+ }
+}
+
+func TestInterruptedAndUnclassifiedInvestigationsStayEligible(t *testing.T) {
+ now := time.Now()
+
+ interrupted := newEligibleInvestigationFinding(InvestigationStatusFailed, InvestigationOutcomeInterrupted, now)
+ if !interrupted.ShouldInvestigate("approval") || !interrupted.CanRetryInvestigation() {
+ t.Fatal("a run cut short by a restart must be retried at the next scheduled check")
+ }
+ if got := deriveLoopState(interrupted); got != FindingLoopStateDetected {
+ t.Fatalf("interrupted loop state = %q, want %q", got, FindingLoopStateDetected)
+ }
+
+ unclassified := newEligibleInvestigationFinding(InvestigationStatusCompleted, InvestigationOutcomeUnclassified, now)
+ if got := deriveLoopState(unclassified); got != FindingLoopStateNeedsAttention {
+ t.Fatalf("unclassified loop state = %q, want %q", got, FindingLoopStateNeedsAttention)
+ }
+ if unclassified.ShouldInvestigate("approval") {
+ t.Fatal("an unclassified run must still respect the normal cooldown")
+ }
+ unclassified = newEligibleInvestigationFinding(InvestigationStatusCompleted, InvestigationOutcomeUnclassified, now.Add(-2*time.Hour))
+ if !unclassified.ShouldInvestigate("approval") || !unclassified.CanRetryInvestigation() {
+ t.Fatal("an unreadable conclusion must not become terminal the way needs_attention is")
+ }
+
+ needsAttention := newEligibleInvestigationFinding(InvestigationStatusCompleted, InvestigationOutcomeNeedsAttention, now.Add(-2*time.Hour))
+ if needsAttention.ShouldInvestigate("approval") {
+ t.Fatal("a real needs_attention conclusion stays terminal")
+ }
+}
diff --git a/internal/ai/investigation_records.go b/internal/ai/investigation_records.go
index f4ee5ab0a2..bb20509e4c 100644
--- a/internal/ai/investigation_records.go
+++ b/internal/ai/investigation_records.go
@@ -203,7 +203,8 @@ func deriveInvestigationRecordConfidence(record *aicontracts.InvestigationRecord
return aicontracts.InvestigationRecordConfidenceHigh
case aicontracts.OutcomeFixQueued, aicontracts.OutcomeFixExecuted, aicontracts.OutcomeCannotFix, aicontracts.OutcomeNeedsAttention:
return aicontracts.InvestigationRecordConfidenceMedium
- case aicontracts.OutcomeFixRejected, aicontracts.OutcomeFixFailed, aicontracts.OutcomeFixVerificationFailed, aicontracts.OutcomeFixVerificationUnknown, aicontracts.OutcomeTimedOut:
+ case aicontracts.OutcomeFixRejected, aicontracts.OutcomeFixFailed, aicontracts.OutcomeFixVerificationFailed, aicontracts.OutcomeFixVerificationUnknown, aicontracts.OutcomeTimedOut,
+ aicontracts.OutcomeUnclassified, aicontracts.OutcomeInterrupted:
return aicontracts.InvestigationRecordConfidenceLow
}
if strings.TrimSpace(record.Conclusion) != "" || len(record.Evidence) > 0 {
diff --git a/internal/ai/patrol_digest.go b/internal/ai/patrol_digest.go
index 1a97e19772..ff9cdaf554 100644
--- a/internal/ai/patrol_digest.go
+++ b/internal/ai/patrol_digest.go
@@ -430,12 +430,14 @@ func PatrolDigestOutcomeOrder(byOutcome map[string]int) []string {
"fix_verification_failed": 2,
"cannot_fix": 3,
"timed_out": 4,
- "fix_queued": 5,
- "fix_executed": 6,
- "fix_verification_unknown": 7,
- "fix_verified": 8,
- "resolved": 9,
- "fix_rejected": 10,
+ "unclassified": 5,
+ "fix_queued": 6,
+ "fix_executed": 7,
+ "fix_verification_unknown": 8,
+ "fix_verified": 9,
+ "resolved": 10,
+ "fix_rejected": 11,
+ "interrupted": 12,
}
keys := make([]string, 0, len(byOutcome))
for key := range byOutcome {
diff --git a/internal/ai/patrol_findings.go b/internal/ai/patrol_findings.go
index 86f47d42d7..adad51a674 100644
--- a/internal/ai/patrol_findings.go
+++ b/internal/ai/patrol_findings.go
@@ -1551,42 +1551,49 @@ func normalizeFindingKey(key string) string {
return normalized
}
-// recoverStuckInvestigations detects findings stuck in "running" state for longer than
-// the investigation timeout and resets them to "failed/timed_out" so they can be retried.
-// This handles the case where an investigation goroutine panics or is killed without
-// properly updating the finding status.
+// investigationRunGrace covers work after the orchestrator's own deadline,
+// such as recording the failure, before Patrol treats a run as abandoned.
+const investigationRunGrace = 5 * time.Minute
+
+// investigationRunCeiling is the longest one investigation may legitimately
+// run: the effective configured timeout plus grace.
+func (p *PatrolService) investigationRunCeiling() time.Duration {
+ timeout := aicontracts.DefaultInvestigationConfig().Timeout
+ if p.aiService != nil {
+ if cfg := p.aiService.GetConfig(); cfg != nil {
+ timeout = cfg.GetPatrolInvestigationTimeout()
+ }
+ }
+ return timeout + investigationRunGrace
+}
+
+// recoverStuckInvestigations releases findings left in "running" by a run that
+// is no longer live, so they can be investigated again. This handles an
+// investigation goroutine that panicked or hung, and a run cut short by a
+// restart before it could update the finding.
func (p *PatrolService) recoverStuckInvestigations() {
if p.findings == nil {
return
}
- const stuckThreshold = 15 * time.Minute // investigation timeout is 10min; allow 5min grace
+ ceiling := p.investigationRunCeiling()
+ orchestrator := p.GetInvestigationOrchestrator()
+ now := time.Now()
active := p.findings.GetActive(FindingSeverityWarning)
recovered := 0
for _, f := range active {
if f.InvestigationStatus != string(InvestigationStatusRunning) {
continue
}
- if f.LastInvestigatedAt == nil {
+ result, stuck := stuckInvestigationResult(f, orchestrator, ceiling, now)
+ if !stuck || !p.findings.RecoverStuckInvestigation(f.ID, f.InvestigationSessionID, result.runStartedAt, result.status, result.outcome, now) {
continue
}
- if time.Since(*f.LastInvestigatedAt) < stuckThreshold {
- continue
- }
- // This finding has been "running" for too long — reset it
- p.findings.UpdateInvestigation(
- f.ID,
- f.InvestigationSessionID,
- string(InvestigationStatusFailed),
- string(InvestigationOutcomeTimedOut),
- f.LastInvestigatedAt,
- f.InvestigationAttempts,
- )
recovered++
log.Warn().
Str("finding_id", f.ID).
Str("resource", f.ResourceName).
- Time("last_investigated", *f.LastInvestigatedAt).
- Msg("AI Patrol: Recovered stuck investigation (exceeded timeout)")
+ Str("outcome", string(result.outcome)).
+ Msg("AI Patrol: Recovered stuck investigation")
}
if recovered > 0 {
log.Info().Int("recovered", recovered).
@@ -1594,6 +1601,107 @@ func (p *PatrolService) recoverStuckInvestigations() {
}
}
+// stuckInvestigation is how to close a run that is no longer live.
+// runStartedAt is the start of the run's session, zero when the orchestrator
+// holds none, so the store can check the run belongs to the finding's current
+// occurrence before it applies the session's result.
+type stuckInvestigation struct {
+ status InvestigationStatus
+ outcome InvestigationOutcome
+ runStartedAt time.Time
+}
+
+// stuckInvestigationResult reports whether a finding marked as running has a
+// run that is no longer live, and how to close it. The orchestrator's session
+// for that run decides liveness and, once the run is closed, its result.
+// Restarts are recorded as interrupted rather than timed out, but the run
+// still spends its attempt: a run that crashes Pulse must not retry forever.
+func stuckInvestigationResult(f *Finding, orchestrator InvestigationOrchestrator, ceiling time.Duration, now time.Time) (stuckInvestigation, bool) {
+ if orchestrator == nil {
+ // No session to consult: measure from the last recorded investigation.
+ if f.LastInvestigatedAt == nil || now.Sub(*f.LastInvestigatedAt) < ceiling {
+ return stuckInvestigation{}, false
+ }
+ return stuckInvestigation{status: InvestigationStatusFailed, outcome: InvestigationOutcomeTimedOut}, true
+ }
+ session, known := recordedInvestigationSession(orchestrator, f)
+ if !known {
+ return stuckInvestigation{}, false
+ }
+ if session == nil {
+ // The orchestrator records its session before it marks the finding as
+ // running, so a run it holds no record of ended with an earlier process.
+ return stuckInvestigation{status: InvestigationStatusFailed, outcome: InvestigationOutcomeInterrupted}, true
+ }
+ closed := stuckInvestigation{status: InvestigationStatusFailed, outcome: InvestigationOutcomeTimedOut, runStartedAt: session.StartedAt}
+ switch session.Status {
+ case aicontracts.InvestigationStatusPending, aicontracts.InvestigationStatusRunning:
+ // A live run keeps the timeout it started with, so a timeout lowered
+ // since then does not cut it short.
+ limit := session.StartedAt.Add(ceiling)
+ if session.Deadline != nil {
+ limit = session.Deadline.Add(investigationRunGrace)
+ }
+ if now.Before(limit) {
+ return stuckInvestigation{}, false
+ }
+ return closed, true
+ }
+ if session.Outcome == aicontracts.OutcomeInterrupted {
+ // Closed when the orchestrator restarted, or cancelled by shutdown.
+ closed.outcome = InvestigationOutcomeInterrupted
+ return closed, true
+ }
+ // The orchestrator closed the run without updating the finding. Its own
+ // update follows the close at once, so give it time to land first.
+ closedAt := session.StartedAt
+ if session.CompletedAt != nil {
+ closedAt = *session.CompletedAt
+ }
+ if now.Sub(closedAt) < investigationRunGrace {
+ return stuckInvestigation{}, false
+ }
+ if session.Outcome != "" {
+ // The session holds the run's result; the finding never received it.
+ // Without one the run closed with no result, such as past its limit.
+ closed.status = InvestigationStatus(session.Status)
+ closed.outcome = InvestigationOutcome(session.Outcome)
+ }
+ return closed, true
+}
+
+// investigationStoreProvider is implemented by orchestrators that expose their
+// session store, which holds every recorded run of a finding.
+type investigationStoreProvider interface {
+ GetStore() aicontracts.InvestigationStore
+}
+
+// recordedInvestigationSession returns the orchestrator's session for the run
+// a finding names, or nil when the orchestrator holds no record of it. known
+// is false when the orchestrator cannot tell: it does not expose its store,
+// and a different run is the finding's latest.
+func recordedInvestigationSession(orchestrator InvestigationOrchestrator, f *Finding) (session *InvestigationSession, known bool) {
+ if provider, ok := orchestrator.(investigationStoreProvider); ok {
+ if store := provider.GetStore(); store != nil {
+ for _, recorded := range store.GetByFinding(f.ID) {
+ if recorded != nil && recorded.SessionID == f.InvestigationSessionID {
+ return recorded, true
+ }
+ }
+ return nil, true
+ }
+ }
+ latest := orchestrator.GetInvestigationByFinding(f.ID)
+ switch {
+ case latest == nil:
+ return nil, true
+ case latest.SessionID == f.InvestigationSessionID:
+ return latest, true
+ default:
+ return nil, false
+ }
+}
+
// retryTimedOutInvestigations re-triggers investigation for findings that failed due to timeout.
// Called at the end of each patrol run to give timed-out investigations another chance
// without waiting for the full 1-hour cooldown.
@@ -1767,7 +1875,10 @@ func (p *PatrolService) maybeInvestigateFinding(f *Finding) bool {
}
currentAutonomy := aiService.GetEffectivePatrolAutonomyLevel()
- ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
+ // The orchestrator enforces the configured investigation timeout; this
+ // outer deadline is a safety net, so it follows that timeout rather than
+ // cutting longer configured or subscription-agent runs short.
+ ctx, cancel := context.WithTimeout(context.Background(), currentCfg.GetPatrolInvestigationTimeout()+investigationRunGrace)
defer cancel()
if err := orchestrator.InvestigateFinding(ctx, invFinding, currentAutonomy); err != nil {
log.Error().
diff --git a/internal/ai/patrol_investigation_limits_test.go b/internal/ai/patrol_investigation_limits_test.go
new file mode 100644
index 0000000000..205e20b95e
--- /dev/null
+++ b/internal/ai/patrol_investigation_limits_test.go
@@ -0,0 +1,295 @@
+package ai
+
+import (
+ "testing"
+ "time"
+
+ "github.com/rcourtman/pulse-go-rewrite/internal/config"
+ "github.com/rcourtman/pulse-go-rewrite/pkg/aicontracts"
+)
+
+type sessionReportingOrchestrator struct {
+ stubInvestigationOrchestrator
+ sessions map[string]*InvestigationSession
+ store aicontracts.InvestigationStore
+}
+
+func (s *sessionReportingOrchestrator) GetInvestigationByFinding(findingID string) *InvestigationSession {
+ return s.sessions[findingID]
+}
+
+func (s *sessionReportingOrchestrator) GetStore() aicontracts.InvestigationStore {
+ return s.store
+}
+
+// findingSessionsStore is a session store that only answers which sessions a
+// finding has recorded, the one query the stuck-run sweep makes.
+type findingSessionsStore struct {
+ aicontracts.InvestigationStore
+ byFinding map[string][]*InvestigationSession
+}
+
+func (s findingSessionsStore) GetByFinding(findingID string) []*InvestigationSession {
+ return s.byFinding[findingID]
+}
+
+func runningInvestigationFinding(id, sessionID string, lastInvestigated *time.Time) *Finding {
+ return &Finding{
+ ID: id,
+ ResourceID: "vm:" + id,
+ ResourceType: "vm",
+ Title: "Investigation " + id,
+ Severity: FindingSeverityWarning,
+ Category: FindingCategoryReliability,
+ InvestigationSessionID: sessionID,
+ InvestigationStatus: string(InvestigationStatusRunning),
+ LastInvestigatedAt: lastInvestigated,
+ InvestigationAttempts: 1,
+ }
+}
+
+func assertInvestigationState(t *testing.T, findings *FindingsStore, id string, status InvestigationStatus, outcome InvestigationOutcome) {
+ t.Helper()
+ got := findings.Get(id)
+ if got.InvestigationStatus != string(status) || got.InvestigationOutcome != string(outcome) {
+ t.Fatalf("%s = %s/%s, want %s/%s", id, got.InvestigationStatus, got.InvestigationOutcome, status, outcome)
+ }
+}
+
+func TestRecoverStuckInvestigationsMeasuresLiveRunsFromTheirOwnStart(t *testing.T) {
+ now := time.Now()
+ previousRun := now.Add(-2 * time.Hour)
+ findings := NewFindingsStore()
+ // A re-investigation still running: LastInvestigatedAt is the previous run.
+ findings.Add(runningInvestigationFinding("live", "chat-live", &previousRun))
+ // A first investigation whose run outlived its ceiling.
+ findings.Add(runningInvestigationFinding("stuck", "chat-stuck", nil))
+
+ orchestrator := &sessionReportingOrchestrator{sessions: map[string]*InvestigationSession{
+ "live": {ID: "inv-live", FindingID: "live", SessionID: "chat-live", Status: aicontracts.InvestigationStatusRunning, StartedAt: now.Add(-time.Minute)},
+ "stuck": {ID: "inv-stuck", FindingID: "stuck", SessionID: "chat-stuck", Status: aicontracts.InvestigationStatusRunning, StartedAt: now.Add(-time.Hour)},
+ }}
+ patrol := &PatrolService{findings: findings, investigationOrchestrator: orchestrator}
+ patrol.recoverStuckInvestigations()
+
+ assertInvestigationState(t, findings, "live", InvestigationStatusRunning, "")
+ assertInvestigationState(t, findings, "stuck", InvestigationStatusFailed, InvestigationOutcomeTimedOut)
+}
+
+func TestRecoverStuckInvestigationsKeepsALiveRunToItsOwnDeadline(t *testing.T) {
+ now := time.Now()
+ findings := NewFindingsStore()
+ findings.Add(runningInvestigationFinding("long", "chat-long", nil))
+ findings.Add(runningInvestigationFinding("overdue", "chat-overdue", nil))
+
+ // Both runs started 16 minutes ago under a 25-minute timeout that has
+ // since been lowered to the 10-minute default, a 15-minute ceiling.
+ started := now.Add(-16 * time.Minute)
+ longDeadline := started.Add(25 * time.Minute)
+ overdueDeadline := now.Add(-investigationRunGrace - time.Minute)
+ orchestrator := &sessionReportingOrchestrator{sessions: map[string]*InvestigationSession{
+ "long": {ID: "inv-long", FindingID: "long", SessionID: "chat-long", Status: aicontracts.InvestigationStatusRunning, StartedAt: started, Deadline: &longDeadline},
+ "overdue": {ID: "inv-overdue", FindingID: "overdue", SessionID: "chat-overdue", Status: aicontracts.InvestigationStatusRunning, StartedAt: started, Deadline: &overdueDeadline},
+ }}
+ patrol := &PatrolService{findings: findings, investigationOrchestrator: orchestrator}
+ patrol.recoverStuckInvestigations()
+
+ assertInvestigationState(t, findings, "long", InvestigationStatusRunning, "")
+ assertInvestigationState(t, findings, "overdue", InvestigationStatusFailed, InvestigationOutcomeTimedOut)
+}
+
+func TestRecoverStuckInvestigationsRecordsRestartsAsInterrupted(t *testing.T) {
+ now := time.Now()
+ previousRun := now.Add(-2 * time.Hour)
+ justClosed := now.Add(-time.Second)
+ closedAWhileAgo := now.Add(-investigationRunGrace - time.Minute)
+ findings := NewFindingsStore()
+ // Closed as interrupted when the orchestrator reloaded after a restart.
+ findings.Add(runningInvestigationFinding("restarted", "chat-restarted", &previousRun))
+ // A first run the orchestrator holds no record of. Before, nothing ever
+ // released it, which also blocked a manual re-run.
+ findings.Add(runningInvestigationFinding("orphan", "chat-gone", nil))
+ // Closed with a result moments ago; the orchestrator's own update is due.
+ findings.Add(runningInvestigationFinding("settling", "chat-settling", nil))
+ // Closed without a result long enough ago that no update is coming.
+ findings.Add(runningInvestigationFinding("abandoned", "chat-abandoned", nil))
+
+ orchestrator := &sessionReportingOrchestrator{sessions: map[string]*InvestigationSession{
+ "restarted": {ID: "inv-restarted", FindingID: "restarted", SessionID: "chat-restarted", Status: aicontracts.InvestigationStatusFailed, Outcome: aicontracts.OutcomeInterrupted, StartedAt: now.Add(-3 * time.Minute), CompletedAt: &justClosed},
+ "settling": {ID: "inv-settling", FindingID: "settling", SessionID: "chat-settling", Status: aicontracts.InvestigationStatusCompleted, Outcome: aicontracts.OutcomeResolved, StartedAt: now.Add(-3 * time.Minute), CompletedAt: &justClosed},
+ "abandoned": {ID: "inv-abandoned", FindingID: "abandoned", SessionID: "chat-abandoned", Status: aicontracts.InvestigationStatusFailed, StartedAt: now.Add(-time.Hour), CompletedAt: &closedAWhileAgo},
+ }}
+ patrol := &PatrolService{findings: findings, investigationOrchestrator: orchestrator}
+ patrol.recoverStuckInvestigations()
+
+ assertInvestigationState(t, findings, "restarted", InvestigationStatusFailed, InvestigationOutcomeInterrupted)
+ assertInvestigationState(t, findings, "orphan", InvestigationStatusFailed, InvestigationOutcomeInterrupted)
+ assertInvestigationState(t, findings, "settling", InvestigationStatusRunning, "")
+ assertInvestigationState(t, findings, "abandoned", InvestigationStatusFailed, InvestigationOutcomeTimedOut)
+ // The interrupted run still spends its attempt, so a run that crashes
+ // Pulse cannot retry forever.
+ if got := findings.Get("orphan").InvestigationAttempts; got != 1 {
+ t.Fatalf("orphan attempts = %d, want 1", got)
+ }
+ if !findings.Get("orphan").ShouldInvestigate("approval") {
+ t.Fatal("an interrupted first run must be eligible for another investigation")
+ }
+ // The close counts as the latest investigation, so the digest sees it.
+ if got := findings.Get("orphan").LastInvestigatedAt; got == nil || got.Before(now) {
+ t.Fatalf("orphan LastInvestigatedAt = %v, want the recovery time", got)
+ }
+}
+
+func TestRecoverStuckInvestigationsKeepsAnActionOutcome(t *testing.T) {
+ now := time.Now()
+ closedAWhileAgo := now.Add(-investigationRunGrace - time.Minute)
+ findings := NewFindingsStore()
+ for _, id := range []string{"queued", "queued-interrupted"} {
+ f := runningInvestigationFinding(id, "chat-"+id, nil)
+ // The action lifecycle recorded the queued fix while the run was
+ // still running, and a restart came before the run finished.
+ f.InvestigationOutcome = string(InvestigationOutcomeFixQueued)
+ findings.Add(f)
+ }
+ orchestrator := &sessionReportingOrchestrator{sessions: map[string]*InvestigationSession{
+ "queued": {ID: "inv-queued", FindingID: "queued", SessionID: "chat-queued", Status: aicontracts.InvestigationStatusFailed, Outcome: aicontracts.OutcomeFixQueued, StartedAt: now.Add(-time.Hour), CompletedAt: &closedAWhileAgo},
+ "queued-interrupted": {ID: "inv-queued-interrupted", FindingID: "queued-interrupted", SessionID: "chat-queued-interrupted", Status: aicontracts.InvestigationStatusFailed, Outcome: aicontracts.OutcomeInterrupted, StartedAt: now.Add(-time.Hour), CompletedAt: &closedAWhileAgo},
+ }}
+ patrol := &PatrolService{findings: findings, investigationOrchestrator: orchestrator}
+ patrol.recoverStuckInvestigations()
+
+ for _, id := range []string{"queued", "queued-interrupted"} {
+ assertInvestigationState(t, findings, id, InvestigationStatusCompleted, InvestigationOutcomeFixQueued)
+ if findings.Get(id).ShouldInvestigate("approval") {
+ t.Fatalf("%s: a queued fix must keep blocking another investigation", id)
+ }
+ }
+}
+
+func TestRecoverStuckInvestigationOnlyClosesTheRunItWasGiven(t *testing.T) {
+ now := time.Now()
+ findings := NewFindingsStore()
+ // A newer run started after the sweep read the finding.
+ findings.Add(runningInvestigationFinding("moved-on", "chat-newer", nil))
+ // The run finished on its own after the sweep read the finding.
+ finished := runningInvestigationFinding("finished", "chat-finished", nil)
+ finished.InvestigationStatus = string(InvestigationStatusCompleted)
+ finished.InvestigationOutcome = string(InvestigationOutcomeResolved)
+ findings.Add(finished)
+
+ if findings.RecoverStuckInvestigation("moved-on", "chat-older", time.Time{}, InvestigationStatusFailed, InvestigationOutcomeInterrupted, now) {
+ t.Fatal("recovering an older run overwrote the newer one")
+ }
+ assertInvestigationState(t, findings, "moved-on", InvestigationStatusRunning, "")
+ if findings.RecoverStuckInvestigation("finished", "chat-finished", time.Time{}, InvestigationStatusFailed, InvestigationOutcomeTimedOut, now) {
+ t.Fatal("recovery overwrote a finished run")
+ }
+ assertInvestigationState(t, findings, "finished", InvestigationStatusCompleted, InvestigationOutcomeResolved)
+ if findings.RecoverStuckInvestigation("missing", "chat-missing", time.Time{}, InvestigationStatusFailed, InvestigationOutcomeTimedOut, now) {
+ t.Fatal("recovery reported success for an unknown finding")
+ }
+}
+
+func TestRecoverStuckInvestigationsKeepsALiveRunBehindANewerSession(t *testing.T) {
+ now := time.Now()
+ findings := NewFindingsStore()
+ findings.Add(runningInvestigationFinding("running", "chat-running", nil))
+ findings.Add(runningInvestigationFinding("pending", "chat-pending", nil))
+ // A second admission for each finding created its session but has not
+ // marked the finding yet, so it is the finding's latest session.
+ newer := func(id string) *InvestigationSession {
+ return &InvestigationSession{ID: "inv-newer-" + id, FindingID: id, SessionID: "chat-newer-" + id, Status: aicontracts.InvestigationStatusPending, StartedAt: now}
+ }
+ store := findingSessionsStore{byFinding: map[string][]*InvestigationSession{
+ "running": {{ID: "inv-running", FindingID: "running", SessionID: "chat-running", Status: aicontracts.InvestigationStatusRunning, StartedAt: now.Add(-time.Minute)}, newer("running")},
+ // Marked on the finding, not yet running in the store.
+ "pending": {{ID: "inv-pending", FindingID: "pending", SessionID: "chat-pending", Status: aicontracts.InvestigationStatusPending, StartedAt: now.Add(-time.Minute)}, newer("pending")},
+ }}
+ orchestrator := &sessionReportingOrchestrator{
+ sessions: map[string]*InvestigationSession{"running": newer("running"), "pending": newer("pending")},
+ store: store,
+ }
+ patrol := &PatrolService{findings: findings, investigationOrchestrator: orchestrator}
+ patrol.recoverStuckInvestigations()
+
+ assertInvestigationState(t, findings, "running", InvestigationStatusRunning, "")
+ assertInvestigationState(t, findings, "pending", InvestigationStatusRunning, "")
+
+ // Without the store, a newer latest session leaves the named run unknown,
+ // and the sweep leaves it alone rather than guess.
+ patrol.investigationOrchestrator = &sessionReportingOrchestrator{sessions: orchestrator.sessions}
+ patrol.recoverStuckInvestigations()
+ assertInvestigationState(t, findings, "running", InvestigationStatusRunning, "")
+}
+
+func TestRecoverStuckInvestigationsKeepsAResultFromBeforeARegression(t *testing.T) {
+ now := time.Now()
+ closedAWhileAgo := now.Add(-investigationRunGrace - time.Minute)
+ regressedAt := now.Add(-10 * time.Minute)
+ findings := NewFindingsStore()
+ f := runningInvestigationFinding("regressed", "chat-old", nil)
+ f.LastRegressionAt = &regressedAt
+ findings.Add(f)
+ // The run started before the finding regressed, and its session recorded
+ // a verified fix for the earlier occurrence.
+ store := findingSessionsStore{byFinding: map[string][]*InvestigationSession{
+ "regressed": {{ID: "inv-old", FindingID: "regressed", SessionID: "chat-old", Status: aicontracts.InvestigationStatusCompleted, Outcome: aicontracts.OutcomeFixVerified, StartedAt: regressedAt.Add(-10 * time.Minute), CompletedAt: &closedAWhileAgo}},
+ }}
+ patrol := &PatrolService{findings: findings, investigationOrchestrator: &sessionReportingOrchestrator{store: store}}
+ patrol.recoverStuckInvestigations()
+
+ assertInvestigationState(t, findings, "regressed", InvestigationStatusFailed, InvestigationOutcomeInterrupted)
+ if got := findings.Get("regressed"); got.ResolvedAt != nil {
+ t.Fatal("an earlier occurrence's verified fix resolved the current one")
+ }
+}
+
+func TestRecoverStuckInvestigationsAppliesTheSessionResult(t *testing.T) {
+ now := time.Now()
+ closedAWhileAgo := now.Add(-investigationRunGrace - time.Minute)
+ findings := NewFindingsStore()
+ // The session store saved the run's result; the finding store did not
+ // before a crash.
+ findings.Add(runningInvestigationFinding("resolved", "chat-resolved", nil))
+ findings.Add(runningInvestigationFinding("queued", "chat-queued", nil))
+ orchestrator := &sessionReportingOrchestrator{sessions: map[string]*InvestigationSession{
+ "resolved": {ID: "inv-resolved", FindingID: "resolved", SessionID: "chat-resolved", Status: aicontracts.InvestigationStatusCompleted, Outcome: aicontracts.OutcomeResolved, StartedAt: now.Add(-time.Hour), CompletedAt: &closedAWhileAgo},
+ // Closed at load after a restart while its proposal was being submitted.
+ "queued": {ID: "inv-queued", FindingID: "queued", SessionID: "chat-queued", Status: aicontracts.InvestigationStatusFailed, Outcome: aicontracts.OutcomeFixQueued, StartedAt: now.Add(-time.Hour), CompletedAt: &closedAWhileAgo},
+ }}
+ patrol := &PatrolService{findings: findings, investigationOrchestrator: orchestrator}
+ patrol.recoverStuckInvestigations()
+
+ assertInvestigationState(t, findings, "resolved", InvestigationStatusCompleted, InvestigationOutcomeResolved)
+ assertInvestigationState(t, findings, "queued", InvestigationStatusFailed, InvestigationOutcomeFixQueued)
+ for _, id := range []string{"resolved", "queued"} {
+ if findings.Get(id).ShouldInvestigate("approval") {
+ t.Fatalf("%s: the run's recorded result must not be retried as a timeout", id)
+ }
+ }
+}
+
+func TestRecoverStuckInvestigationsWithoutOrchestratorUsesLastInvestigation(t *testing.T) {
+ now := time.Now()
+ previousRun := now.Add(-2 * time.Hour)
+ findings := NewFindingsStore()
+ findings.Add(runningInvestigationFinding("old", "chat-old", &previousRun))
+ findings.Add(runningInvestigationFinding("unknown", "chat-unknown", nil))
+ patrol := &PatrolService{findings: findings}
+ patrol.recoverStuckInvestigations()
+
+ assertInvestigationState(t, findings, "old", InvestigationStatusFailed, InvestigationOutcomeTimedOut)
+ assertInvestigationState(t, findings, "unknown", InvestigationStatusRunning, "")
+}
+
+func TestInvestigationRunCeilingFollowsTheConfiguredTimeout(t *testing.T) {
+ if got, want := (&PatrolService{}).investigationRunCeiling(), 10*time.Minute+investigationRunGrace; got != want {
+ t.Fatalf("default ceiling = %s, want %s", got, want)
+ }
+ cfg := config.NewDefaultAIConfig()
+ cfg.PatrolModel = config.AIProviderCodexSubscription + ":gpt-5.6-luna"
+ patrol := &PatrolService{aiService: &Service{cfg: cfg}}
+ if got, want := patrol.investigationRunCeiling(), time.Duration(config.DefaultPatrolSubscriptionInvestigationTimeoutSec)*time.Second+investigationRunGrace; got != want {
+ t.Fatalf("subscription ceiling = %s, want %s", got, want)
+ }
+}
diff --git a/internal/ai/patrol_lifecycle_integration_test.go b/internal/ai/patrol_lifecycle_integration_test.go
index c2feed1871..0f11975f7f 100644
--- a/internal/ai/patrol_lifecycle_integration_test.go
+++ b/internal/ai/patrol_lifecycle_integration_test.go
@@ -9,6 +9,7 @@ import (
"github.com/rcourtman/pulse-go-rewrite/internal/ai/tools"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
"github.com/rcourtman/pulse-go-rewrite/internal/models"
+ "github.com/rcourtman/pulse-go-rewrite/pkg/aicontracts"
)
// TestLifecycle_FindingToInvestigationToApproval verifies the end-to-end lifecycle:
@@ -151,12 +152,20 @@ func TestLifecycle_StuckInvestigationRecovery(t *testing.T) {
AnalyzeGuests: true,
})
- // Set up orchestrator (required for MaybeInvestigateFinding path)
- orch := newIntegrationOrchestrator(true)
- ps.SetInvestigationOrchestrator(orch)
-
// Create a finding and manually set it to "running" with LastInvestigatedAt 20 minutes ago
twentyMinAgo := time.Now().Add(-20 * time.Minute)
+
+ // The orchestrator still holds the run's session, started 20 minutes ago,
+ // so recovery treats it as a hung run rather than one lost to a restart.
+ ps.SetInvestigationOrchestrator(&sessionReportingOrchestrator{sessions: map[string]*InvestigationSession{
+ "stuck-finding-1": {
+ ID: "inv-stuck-1",
+ FindingID: "stuck-finding-1",
+ SessionID: "session-stuck-1",
+ Status: aicontracts.InvestigationStatusRunning,
+ StartedAt: twentyMinAgo,
+ },
+ }})
stuckFinding := &Finding{
ID: "stuck-finding-1",
Key: "high-mem",
diff --git a/internal/ai/service.go b/internal/ai/service.go
index 3cb2d32b6d..55cef9860a 100644
--- a/internal/ai/service.go
+++ b/internal/ai/service.go
@@ -903,6 +903,7 @@ func (s *Service) ApplyPatrolAutonomyConfig(cfg *config.AIConfig) error {
if s == nil || cfg == nil {
return fmt.Errorf("Pulse Patrol config is unavailable")
}
+ orchestrator := s.liveInvestigationOrchestrator()
s.mu.Lock()
defer s.mu.Unlock()
if s.cfg == nil {
@@ -920,18 +921,42 @@ func (s *Service) ApplyPatrolAutonomyConfig(cfg *config.AIConfig) error {
activation := *cfg.PatrolAutopilotActivation
s.cfg.PatrolAutopilotActivation = &activation
}
- if s.patrolService != nil {
- orchestrator := s.patrolService.GetInvestigationOrchestrator()
- if updater, ok := orchestrator.(aicontracts.InvestigationExecutionLimitUpdater); ok {
- updater.SetInvestigationExecutionLimits(
- s.cfg.GetPatrolInvestigationBudget(),
- s.cfg.GetPatrolInvestigationTimeout(),
- )
- }
- }
+ publishInvestigationExecutionLimits(orchestrator, s.cfg)
return nil
}
+// liveInvestigationOrchestrator returns the Patrol investigation orchestrator
+// without holding s.mu while it takes the Patrol lock: PatrolService takes its
+// own lock before s.mu (GetIntelligence), so the reverse order could deadlock.
+func (s *Service) liveInvestigationOrchestrator() InvestigationOrchestrator {
+ s.mu.RLock()
+ patrol := s.patrolService
+ s.mu.RUnlock()
+ if patrol == nil {
+ return nil
+ }
+ return patrol.GetInvestigationOrchestrator()
+}
+
+// publishInvestigationExecutionLimits publishes the configured evidence
+// budget, timeout and concurrency to a live investigation orchestrator. The
+// timeout default and the concurrency follow the Patrol provider, so a
+// provider change applies without rebuilding the orchestrator.
+func publishInvestigationExecutionLimits(orchestrator InvestigationOrchestrator, cfg *config.AIConfig) {
+ if orchestrator == nil || cfg == nil {
+ return
+ }
+ if updater, ok := orchestrator.(aicontracts.InvestigationExecutionLimitUpdater); ok {
+ updater.SetInvestigationExecutionLimits(
+ cfg.GetPatrolInvestigationBudget(),
+ cfg.GetPatrolInvestigationTimeout(),
+ )
+ }
+ if updater, ok := orchestrator.(aicontracts.InvestigationConcurrencyLimitUpdater); ok {
+ updater.SetInvestigationConcurrencyLimit(cfg.GetPatrolInvestigationConcurrency())
+ }
+}
+
func clonePatrolAutopilotAcknowledgements(records []unifiedresources.PatrolAutopilotAcknowledgement) []unifiedresources.PatrolAutopilotAcknowledgement {
cloned := append([]unifiedresources.PatrolAutopilotAcknowledgement(nil), records...)
for index := range cloned {
@@ -1564,10 +1589,12 @@ func (s *Service) LoadConfig() error {
providerClient, providerInitErr = buildConfiguredProvider(context.Background(), cfg, orgID, true)
}
+ orchestrator := s.liveInvestigationOrchestrator()
s.mu.Lock()
s.cfg = cfg
s.provider = providerClient
s.providerInitErr = providerInitErr
+ publishInvestigationExecutionLimits(orchestrator, cfg)
s.mu.Unlock()
return nil
diff --git a/internal/ai/service_patrol_limits_test.go b/internal/ai/service_patrol_limits_test.go
new file mode 100644
index 0000000000..6ffed4b9d5
--- /dev/null
+++ b/internal/ai/service_patrol_limits_test.go
@@ -0,0 +1,57 @@
+package ai
+
+import (
+ "testing"
+ "time"
+
+ "github.com/rcourtman/pulse-go-rewrite/internal/config"
+)
+
+type concurrencyRecordingOrchestrator struct {
+ stubInvestigationOrchestrator
+ maxConcurrent int
+}
+
+func (s *concurrencyRecordingOrchestrator) SetInvestigationConcurrencyLimit(maxConcurrent int) {
+ s.maxConcurrent = maxConcurrent
+}
+
+func TestLoadConfigPublishesProviderAwareInvestigationLimits(t *testing.T) {
+ persistence := config.NewConfigPersistence(t.TempDir())
+ saved := config.NewDefaultAIConfig()
+ saved.PatrolModel = config.AIProviderCodexSubscription + ":gpt-5.6-luna"
+ if err := persistence.SaveAIConfig(*saved); err != nil {
+ t.Fatalf("SaveAIConfig: %v", err)
+ }
+
+ orchestrator := &concurrencyRecordingOrchestrator{}
+ service := &Service{
+ persistence: persistence,
+ patrolService: &PatrolService{investigationOrchestrator: orchestrator},
+ }
+ if err := service.LoadConfig(); err != nil {
+ t.Fatalf("LoadConfig: %v", err)
+ }
+ if orchestrator.maxConcurrent != 1 {
+ t.Fatalf("concurrency = %d, want 1 for a subscription agent that serves one call at a time", orchestrator.maxConcurrent)
+ }
+ if want := time.Duration(config.DefaultPatrolSubscriptionInvestigationTimeoutSec) * time.Second; orchestrator.timeout != want {
+ t.Fatalf("timeout = %s, want %s", orchestrator.timeout, want)
+ }
+
+ // Switching Patrol to an API-key provider restores the normal limits on
+ // the next config load, without rebuilding the orchestrator.
+ saved.PatrolModel = config.AIProviderAnthropic + ":claude-sonnet-4-5"
+ if err := persistence.SaveAIConfig(*saved); err != nil {
+ t.Fatalf("SaveAIConfig: %v", err)
+ }
+ if err := service.LoadConfig(); err != nil {
+ t.Fatalf("LoadConfig: %v", err)
+ }
+ if orchestrator.maxConcurrent != config.MaxConcurrentInvestigations {
+ t.Fatalf("concurrency = %d, want %d", orchestrator.maxConcurrent, config.MaxConcurrentInvestigations)
+ }
+ if want := time.Duration(config.DefaultPatrolInvestigationTimeoutSec) * time.Second; orchestrator.timeout != want {
+ t.Fatalf("timeout = %s, want %s", orchestrator.timeout, want)
+ }
+}
diff --git a/internal/ai/tools/executor.go b/internal/ai/tools/executor.go
index 874846cf6e..7b4fb6d7eb 100644
--- a/internal/ai/tools/executor.go
+++ b/internal/ai/tools/executor.go
@@ -1161,10 +1161,11 @@ func (e *PulseToolExecutor) isToolAvailable(name string) bool {
return e.GetPatrolFindingCreator() != nil
case agentcapabilities.PatrolProposeObserverToolName:
return e.GetPatrolObserverProposer() != nil
- case agentcapabilities.PatrolProposeActionToolName, agentcapabilities.PatrolActionCapabilitiesToolName:
+ case agentcapabilities.PatrolProposeActionToolName, agentcapabilities.PatrolActionCapabilitiesToolName, agentcapabilities.PatrolConcludeInvestigationToolName:
// These investigation-only tools share the request-local proposal
- // capture: it supplies both trusted correlation and the tenant-bound
- // capability catalog. They must never appear without that boundary.
+ // capture: it supplies trusted correlation, the tenant-bound
+ // capability catalog and the run's conclusion sink. They must never
+ // appear without that boundary.
return e.proposalCapture != nil
default:
return e.hasReadState()
@@ -1230,6 +1231,9 @@ func (e *PulseToolExecutor) registerTools() {
e.registerProposeTools()
+ // patrol_conclude_investigation - typed zero-action investigation conclusion
+ e.registerConcludeTool()
+
// pulse_summarize - retrospective synthesis for one resource or a fleet
e.registerSummarizeTools()
}
diff --git a/internal/ai/tools/names_test.go b/internal/ai/tools/names_test.go
index 57e35fec4d..56ae238b61 100644
--- a/internal/ai/tools/names_test.go
+++ b/internal/ai/tools/names_test.go
@@ -30,6 +30,7 @@ func TestKnownToolNamesIncludesRegisteredTools(t *testing.T) {
agentcapabilities.PatrolAssessFindingToolName,
agentcapabilities.PatrolActionCapabilitiesToolName,
agentcapabilities.PatrolProposeActionToolName,
+ agentcapabilities.PatrolConcludeInvestigationToolName,
}
for _, name := range expected {
if !IsKnownToolName(name) {
diff --git a/internal/ai/tools/proposal_capture.go b/internal/ai/tools/proposal_capture.go
index 58897e6a89..1c5732e6d7 100644
--- a/internal/ai/tools/proposal_capture.go
+++ b/internal/ai/tools/proposal_capture.go
@@ -85,6 +85,9 @@ type ProposalCapture struct {
catalog ProposalCatalog
planner ProposalPlanner
proposal *CapturedProposal
+ // conclusion is the run's one typed zero-action conclusion, recorded
+ // by patrol_conclude_investigation (see tools_conclude.go).
+ conclusion *CapturedConclusion
}
func (i ProposalIdentity) clone() ProposalIdentity {
diff --git a/internal/ai/tools/registry.go b/internal/ai/tools/registry.go
index 674c1e9324..b190e1d57e 100644
--- a/internal/ai/tools/registry.go
+++ b/internal/ai/tools/registry.go
@@ -161,7 +161,7 @@ func (p InvocationPolicy) Allows(toolName string, class agentcapabilities.Invoca
func isPatrolInvestigationOnlyTool(toolName string) bool {
switch toolName {
- case agentcapabilities.PatrolProposeActionToolName, agentcapabilities.PatrolActionCapabilitiesToolName:
+ case agentcapabilities.PatrolProposeActionToolName, agentcapabilities.PatrolActionCapabilitiesToolName, agentcapabilities.PatrolConcludeInvestigationToolName:
return true
default:
return false
diff --git a/internal/ai/tools/tools_conclude.go b/internal/ai/tools/tools_conclude.go
new file mode 100644
index 0000000000..f290ee85d6
--- /dev/null
+++ b/internal/ai/tools/tools_conclude.go
@@ -0,0 +1,149 @@
+package tools
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "strings"
+
+ "github.com/rcourtman/pulse-go-rewrite/internal/agentcapabilities"
+ "github.com/rcourtman/pulse-go-rewrite/pkg/aicontracts"
+)
+
+// maxConclusionReasonRunes bounds the recorded reason. The run's summary
+// carries the full narrative; the conclusion only needs its justification.
+const maxConclusionReasonRunes = 2000
+
+// CapturedConclusion is the one typed zero-action conclusion an investigation
+// recorded. Outcome is aicontracts.OutcomeResolved, OutcomeCannotFix or
+// OutcomeNeedsAttention.
+type CapturedConclusion struct {
+ Outcome aicontracts.InvestigationOutcome
+ Reason string
+ InvocationID string
+}
+
+// ParseConclusionOutcome maps a model-authored outcome onto the closed
+// conclusion vocabulary. Anything else is refused, never defaulted.
+func ParseConclusionOutcome(value string) (aicontracts.InvestigationOutcome, bool) {
+ switch strings.ToLower(strings.TrimSpace(value)) {
+ case string(aicontracts.OutcomeResolved):
+ return aicontracts.OutcomeResolved, true
+ case string(aicontracts.OutcomeCannotFix):
+ return aicontracts.OutcomeCannotFix, true
+ case string(aicontracts.OutcomeNeedsAttention):
+ return aicontracts.OutcomeNeedsAttention, true
+ default:
+ return "", false
+ }
+}
+
+// RecordConclusion keeps the first conclusion for the run. Replaying the same
+// outcome is accepted; a different outcome is refused so the recorded
+// conclusion cannot flip after the model has acted on it.
+func (c *ProposalCapture) RecordConclusion(invocationID string, outcome aicontracts.InvestigationOutcome, reason string) (*CapturedConclusion, error) {
+ if c == nil {
+ return nil, errors.New("investigation conclusions are not available in this run")
+ }
+ reason = strings.TrimSpace(reason)
+ if runes := []rune(reason); len(runes) > maxConclusionReasonRunes {
+ reason = string(runes[:maxConclusionReasonRunes])
+ }
+ c.mu.Lock()
+ defer c.mu.Unlock()
+ if c.conclusion != nil {
+ existing := *c.conclusion
+ if existing.Outcome == outcome {
+ return &existing, nil
+ }
+ return &existing, fmt.Errorf("this investigation already concluded %s; a different conclusion needs a new investigation", existing.Outcome)
+ }
+ c.conclusion = &CapturedConclusion{Outcome: outcome, Reason: reason, InvocationID: invocationID}
+ recorded := *c.conclusion
+ return &recorded, nil
+}
+
+// Conclusion returns a copy of the recorded conclusion, nil when none exists.
+func (c *ProposalCapture) Conclusion() *CapturedConclusion {
+ if c == nil {
+ return nil
+ }
+ c.mu.Lock()
+ defer c.mu.Unlock()
+ if c.conclusion == nil {
+ return nil
+ }
+ recorded := *c.conclusion
+ return &recorded
+}
+
+// InvestigationConcluded reports whether this run has recorded its
+// conclusion. The agentic loop then asks only for the final summary.
+func (e *PulseToolExecutor) InvestigationConcluded() bool {
+ return e != nil && e.proposalCapture.Conclusion() != nil
+}
+
+// registerConcludeTool registers the investigation-only conclusion capture.
+// The registry policy rejects it outside the Patrol investigation profile,
+// and it is offered only when the request-local capture exists.
+func (e *PulseToolExecutor) registerConcludeTool() {
+ e.registry.registerBuiltin(RegisteredTool{
+ Definition: Tool{
+ Name: agentcapabilities.PatrolConcludeInvestigationToolName,
+ Description: `Record this investigation's conclusion. Call it once, when the evidence supports a conclusion, then write the final summary. Choose exactly one outcome:
+- resolved: the evidence shows the finding's condition no longer holds.
+- cannot_fix: the problem is real, but no advertised capability can remediate it.
+- needs_attention: a person must act or decide; the reason says what they should do.
+
+If you proposed an action, its planning result is the investigation outcome and this call is optional. Side-effect-free: it records the conclusion for this run only and plans, approves or executes nothing.`,
+ InputSchema: InputSchema{
+ Type: "object",
+ Properties: map[string]PropertySchema{
+ "outcome": {
+ Type: "string",
+ Description: "The conclusion this investigation reached",
+ Enum: []string{string(aicontracts.OutcomeResolved), string(aicontracts.OutcomeCannotFix), string(aicontracts.OutcomeNeedsAttention)},
+ },
+ "reason": {
+ Type: "string",
+ Description: "The evidence for this conclusion and, for needs_attention, what the person should do",
+ },
+ },
+ Required: []string{"outcome", "reason"},
+ },
+ },
+ Handler: func(ctx context.Context, exec *PulseToolExecutor, args map[string]interface{}) (CallToolResult, error) {
+ return exec.executeConcludeInvestigation(ctx, args)
+ },
+ Governance: ToolGovernance{
+ ActionMode: ToolActionRead,
+ ApprovalPolicy: ToolApprovalScopeOnly,
+ ApprovalSummary: "records the investigation's typed conclusion without changing infrastructure or Pulse state",
+ Summary: "Records one typed zero-action conclusion during a Patrol investigation; the orchestrator persists the outcome.",
+ },
+ })
+}
+
+// executeConcludeInvestigation validates and records the run's conclusion.
+func (e *PulseToolExecutor) executeConcludeInvestigation(ctx context.Context, args map[string]interface{}) (CallToolResult, error) {
+ capture := e.proposalCapture
+ if capture == nil {
+ return NewErrorResult(fmt.Errorf("investigation conclusions are not available in this run")), nil
+ }
+ outcome, ok := ParseConclusionOutcome(stringArg(args, "outcome"))
+ if !ok {
+ return NewErrorResult(fmt.Errorf("outcome must be one of resolved, cannot_fix or needs_attention")), nil
+ }
+ reason := strings.TrimSpace(stringArg(args, "reason"))
+ if reason == "" {
+ return NewErrorResult(fmt.Errorf("reason is required")), nil
+ }
+ recorded, err := capture.RecordConclusion(InvocationIDFromContext(ctx), outcome, reason)
+ if err != nil {
+ return NewErrorResult(err), nil
+ }
+ return NewJSONResult(map[string]interface{}{
+ "recorded_outcome": string(recorded.Outcome),
+ "next_step": "Write the final summary sections now: Investigation Summary, Root Cause, Affected Resources and Recommendation. Do not call further tools.",
+ }), nil
+}
diff --git a/internal/ai/tools/tools_conclude_test.go b/internal/ai/tools/tools_conclude_test.go
new file mode 100644
index 0000000000..3960ae9074
--- /dev/null
+++ b/internal/ai/tools/tools_conclude_test.go
@@ -0,0 +1,126 @@
+package tools
+
+import (
+ "context"
+ "strings"
+ "testing"
+
+ "github.com/rcourtman/pulse-go-rewrite/internal/agentcapabilities"
+ "github.com/rcourtman/pulse-go-rewrite/pkg/aicontracts"
+ "github.com/stretchr/testify/assert"
+ "github.com/stretchr/testify/require"
+)
+
+func executeConclude(t *testing.T, exec *PulseToolExecutor, id string, args map[string]interface{}) CallToolResult {
+ t.Helper()
+ result, err := exec.ExecuteInvocation(context.Background(), ToolInvocation{
+ ID: id,
+ Name: agentcapabilities.PatrolConcludeInvestigationToolName,
+ Arguments: args,
+ })
+ require.NoError(t, err)
+ require.NotEmpty(t, result.Content)
+ return result
+}
+
+func TestParseConclusionOutcomeAcceptsOnlyTheClosedVocabulary(t *testing.T) {
+ for input, want := range map[string]aicontracts.InvestigationOutcome{
+ "resolved": aicontracts.OutcomeResolved,
+ " CANNOT_FIX ": aicontracts.OutcomeCannotFix,
+ "Needs_Attention": aicontracts.OutcomeNeedsAttention,
+ } {
+ got, ok := ParseConclusionOutcome(input)
+ assert.True(t, ok, input)
+ assert.Equal(t, want, got, input)
+ }
+ for _, input := range []string{"", "fix_verified", "timed_out", "unclassified", "resolved: yes"} {
+ _, ok := ParseConclusionOutcome(input)
+ assert.False(t, ok, "%q must not parse as a conclusion", input)
+ }
+}
+
+func TestRecordConclusionKeepsTheFirstConclusion(t *testing.T) {
+ capture := NewProposalCapture(ProposalIdentity{ProposalID: "p1", FindingID: "f1", InvestigationID: "i1"}, nil)
+ assert.Nil(t, capture.Conclusion())
+
+ recorded, err := capture.RecordConclusion("call-1", aicontracts.OutcomeResolved, " usage is back under threshold ")
+ require.NoError(t, err)
+ assert.Equal(t, aicontracts.OutcomeResolved, recorded.Outcome)
+ assert.Equal(t, "usage is back under threshold", recorded.Reason)
+
+ _, err = capture.RecordConclusion("call-2", aicontracts.OutcomeResolved, "replayed")
+ require.NoError(t, err, "replaying the same outcome is accepted")
+
+ _, err = capture.RecordConclusion("call-3", aicontracts.OutcomeNeedsAttention, "changed my mind")
+ require.Error(t, err, "a different outcome must not replace the recorded conclusion")
+
+ got := capture.Conclusion()
+ require.NotNil(t, got)
+ assert.Equal(t, aicontracts.OutcomeResolved, got.Outcome)
+ assert.Equal(t, "usage is back under threshold", got.Reason)
+ assert.Equal(t, "call-1", got.InvocationID)
+
+ long := NewProposalCapture(ProposalIdentity{}, nil)
+ recorded, err = long.RecordConclusion("call-1", aicontracts.OutcomeCannotFix, strings.Repeat("x", maxConclusionReasonRunes+50))
+ require.NoError(t, err)
+ assert.Len(t, []rune(recorded.Reason), maxConclusionReasonRunes)
+}
+
+func TestConcludeInvestigationIsInvestigationProfileOnly(t *testing.T) {
+ args := map[string]interface{}{"outcome": "needs_attention", "reason": "replace the failing disk"}
+ for _, profile := range []ExecutionProfile{ProfileInteractiveAssistant, ProfilePatrolDetection} {
+ exec := NewPulseToolExecutor(ExecutorConfig{})
+ exec.SetControlLevel(ControlLevelAutonomous)
+ exec.ApplyExecutionProfile(profile)
+ exec.SetProposalCapture(NewProposalCapture(ProposalIdentity{}, nil))
+
+ result := executeConclude(t, exec, "call-x", args)
+ assert.Contains(t, result.Content[0].Text, "Invocation blocked",
+ "profile %d must reject the conclusion capture at the registry boundary", profile)
+ for _, tool := range exec.registry.ListTools(exec.invocationPolicy()) {
+ assert.NotEqual(t, agentcapabilities.PatrolConcludeInvestigationToolName, tool.Name,
+ "profile %d must not offer the conclusion capture", profile)
+ }
+ }
+
+ // Without the request-local capture the tool is not offered at all.
+ bare := NewPulseToolExecutor(ExecutorConfig{})
+ bare.ApplyExecutionProfile(ProfilePatrolInvestigation)
+ for _, tool := range bare.ListTools() {
+ assert.NotEqual(t, agentcapabilities.PatrolConcludeInvestigationToolName, tool.Name,
+ "an investigation without a capture must not offer the conclusion tool")
+ }
+
+ capture := NewProposalCapture(ProposalIdentity{ProposalID: "p1", FindingID: "f1", InvestigationID: "i1"}, nil)
+ exec := newInvestigationExecutor(t, capture)
+ offered := false
+ for _, tool := range exec.ListTools() {
+ if tool.Name == agentcapabilities.PatrolConcludeInvestigationToolName {
+ offered = true
+ }
+ }
+ assert.True(t, offered, "investigation profile must offer patrol_conclude_investigation")
+ assert.False(t, exec.InvestigationConcluded())
+
+ result := executeConclude(t, exec, "call-a", args)
+ assert.False(t, result.IsError, result.Content[0].Text)
+ assert.Contains(t, result.Content[0].Text, "needs_attention")
+ assert.True(t, exec.InvestigationConcluded())
+ require.NotNil(t, capture.Conclusion())
+ assert.Equal(t, aicontracts.OutcomeNeedsAttention, capture.Conclusion().Outcome)
+ assert.Equal(t, "call-a", capture.Conclusion().InvocationID)
+}
+
+func TestConcludeInvestigationRefusesInvalidArguments(t *testing.T) {
+ exec := newInvestigationExecutor(t, NewProposalCapture(ProposalIdentity{ProposalID: "p1", FindingID: "f1", InvestigationID: "i1"}, nil))
+
+ for name, args := range map[string]map[string]interface{}{
+ "unknown outcome": {"outcome": "fix_verified", "reason": "done"},
+ "missing reason": {"outcome": "resolved"},
+ "blank reason": {"outcome": "resolved", "reason": " "},
+ } {
+ result := executeConclude(t, exec, "call-"+strings.ReplaceAll(name, " ", "-"), args)
+ assert.True(t, result.IsError, "%s must be refused: %s", name, result.Content[0].Text)
+ }
+ assert.False(t, exec.InvestigationConcluded(), "refused calls must not record a conclusion")
+}
diff --git a/internal/api/ai_handlers.go b/internal/api/ai_handlers.go
index a345e6f71e..02713e706b 100644
--- a/internal/api/ai_handlers.go
+++ b/internal/api/ai_handlers.go
@@ -1941,6 +1941,7 @@ func (h *AISettingsHandler) setupInvestigationOrchestrator(orgID string, svc *ai
invConfig.MaxEvidenceCalls = cfg.GetPatrolInvestigationBudget()
invConfig.MaxTurns = aicontracts.InvestigationModelTurnLimit(invConfig.MaxEvidenceCalls)
invConfig.Timeout = cfg.GetPatrolInvestigationTimeout()
+ invConfig.MaxConcurrent = cfg.GetPatrolInvestigationConcurrency()
}
// The knowledge store supplies the operator's infrastructure notes.
@@ -2058,6 +2059,12 @@ func (a *orchestratorChatAdapter) ExecuteInvestigationStream(ctx context.Context
EvidenceIDs: captured.Identity.EvidenceIDs,
}
}
+ if runResult.Conclusion != nil {
+ result.Conclusion = &aicontracts.InvestigationConclusion{
+ Outcome: runResult.Conclusion.Outcome,
+ Reason: runResult.Conclusion.Reason,
+ }
+ }
return result, err
}
diff --git a/internal/api/ai_handlers_investigation_additional_test.go b/internal/api/ai_handlers_investigation_additional_test.go
index b00f92d771..c51e310375 100644
--- a/internal/api/ai_handlers_investigation_additional_test.go
+++ b/internal/api/ai_handlers_investigation_additional_test.go
@@ -885,3 +885,51 @@ func TestPatrolExpiredActionHydrationRemovesQueuedOutcomeWithoutRewritingEvidenc
t.Fatal("duplicate hydration republished unchanged record")
}
}
+
+func TestSetupInvestigationOrchestratorUsesSubscriptionAgentLimits(t *testing.T) {
+ prevStoreFactory := getCreateInvestigationStore()
+ SetCreateInvestigationStore(func(dataDir string) aicontracts.InvestigationStore {
+ return newTestInvestigationStore()
+ })
+ t.Cleanup(func() { SetCreateInvestigationStore(prevStoreFactory) })
+
+ var created []aicontracts.InvestigationConfig
+ prevOrchestratorFactory := getCreateInvestigationOrchestrator()
+ SetCreateInvestigationOrchestrator(func(deps aicontracts.OrchestratorDeps) aicontracts.InvestigationOrchestrator {
+ created = append(created, deps.Config)
+ return &stubInvestigationOrchestrator{}
+ })
+ t.Cleanup(func() { SetCreateInvestigationOrchestrator(prevOrchestratorFactory) })
+
+ persistence := config.NewConfigPersistence(t.TempDir())
+ saved := config.NewDefaultAIConfig()
+ saved.PatrolModel = config.AIProviderCodexSubscription + ":gpt-5.6-luna"
+ if err := persistence.SaveAIConfig(*saved); err != nil {
+ t.Fatalf("SaveAIConfig: %v", err)
+ }
+ tenantSvc := ai.NewService(persistence, nil)
+ if err := tenantSvc.LoadConfig(); err != nil {
+ t.Fatalf("LoadConfig: %v", err)
+ }
+ tenantSvc.SetStateProvider(&MockStateProvider{})
+
+ handler := &AISettingsHandler{
+ investigationStores: make(map[string]aicontracts.InvestigationStore),
+ }
+ handler.chatHandler = &AIHandler{
+ services: map[string]AIService{
+ "tenant-1": chat.NewService(chat.Config{AIConfig: config.NewDefaultAIConfig()}),
+ },
+ }
+ handler.setupInvestigationOrchestrator("tenant-1", tenantSvc)
+
+ if len(created) != 1 {
+ t.Fatalf("orchestrator factory calls = %d, want 1", len(created))
+ }
+ if created[0].MaxConcurrent != 1 {
+ t.Fatalf("MaxConcurrent = %d, want 1 for a subscription agent that serves one call at a time", created[0].MaxConcurrent)
+ }
+ if want := time.Duration(config.DefaultPatrolSubscriptionInvestigationTimeoutSec) * time.Second; created[0].Timeout != want {
+ t.Fatalf("Timeout = %s, want %s", created[0].Timeout, want)
+ }
+}
diff --git a/internal/api/patrol_digest_email.go b/internal/api/patrol_digest_email.go
index 3479acc372..ba9202b8b6 100644
--- a/internal/api/patrol_digest_email.go
+++ b/internal/api/patrol_digest_email.go
@@ -144,6 +144,7 @@ func patrolDigestEmailLines(digest ai.PatrolDigest) []patrolDigestEmailLine {
"cannot_fix": "could not fix", "timed_out": "timed out", "fix_queued": "fix waiting for approval",
"fix_executed": "fix run", "fix_verification_unknown": "fix run, result unknown", "fix_verified": "fixed and verified",
"resolved": "resolved", "fix_rejected": "fix declined",
+ "unclassified": "finished without a clear conclusion", "interrupted": "interrupted by a restart",
}
outcomeDetails := []string{}
for _, key := range ai.PatrolDigestOutcomeOrder(digest.Investigations.ByOutcome) {
diff --git a/internal/config/ai.go b/internal/config/ai.go
index aafdd672a8..e57ab85dbc 100644
--- a/internal/config/ai.go
+++ b/internal/config/ai.go
@@ -179,6 +179,11 @@ const (
InvestigationCooldownHours = 1 // Hours before re-investigating same finding
)
+// DefaultPatrolSubscriptionInvestigationTimeoutSec is the default when Patrol
+// runs on a subscription agent CLI, which starts a new process and resends
+// the conversation on every turn (25 minutes).
+const DefaultPatrolSubscriptionInvestigationTimeoutSec = 1500
+
const (
// DefaultAIModelQuickstart is retained only to recognize and retire legacy
// Pulse-hosted model aliases from pre-GA config.
@@ -1093,9 +1098,32 @@ func (c *AIConfig) GetPatrolInvestigationBudget() int {
return c.PatrolInvestigationBudget
}
-// GetPatrolInvestigationTimeout returns the investigation timeout as a duration
+// PatrolUsesSubscriptionAgent reports whether Patrol runs on a local
+// subscription agent CLI (Codex or Claude). Such an agent serves one call at
+// a time for all of Patrol and starts a new process for every turn.
+func (c *AIConfig) PatrolUsesSubscriptionAgent() bool {
+ provider, _ := ParseModelString(c.GetPatrolModel())
+ return provider == AIProviderCodexSubscription || provider == AIProviderClaudeSubscription
+}
+
+// GetPatrolInvestigationConcurrency returns how many investigations may run
+// at once. A subscription agent serves one call at a time, so parallel
+// investigations would only queue against each other's deadlines.
+func (c *AIConfig) GetPatrolInvestigationConcurrency() int {
+ if c.PatrolUsesSubscriptionAgent() {
+ return 1
+ }
+ return MaxConcurrentInvestigations
+}
+
+// GetPatrolInvestigationTimeout returns the investigation timeout as a
+// duration. Without an explicit setting, a subscription agent gets the longer
+// default because each of its turns takes minutes rather than seconds.
func (c *AIConfig) GetPatrolInvestigationTimeout() time.Duration {
if c.PatrolInvestigationTimeoutSec <= 0 {
+ if c.PatrolUsesSubscriptionAgent() {
+ return time.Duration(DefaultPatrolSubscriptionInvestigationTimeoutSec) * time.Second
+ }
return time.Duration(DefaultPatrolInvestigationTimeoutSec) * time.Second
}
// Clamp to reasonable range (60-1800 seconds / 30 minutes)
diff --git a/internal/config/ai_config_test.go b/internal/config/ai_config_test.go
index 685503d24a..6832eb1c04 100644
--- a/internal/config/ai_config_test.go
+++ b/internal/config/ai_config_test.go
@@ -1410,3 +1410,54 @@ func TestResolvedPatrolAutonomyLevelDefaultsOnlyNeverChosenLicensedInstalls(t *t
})
}
}
+
+func TestPatrolSubscriptionAgentGetsLongerTimeoutAndOneInvestigationAtATime(t *testing.T) {
+ cases := []struct {
+ name string
+ cfg AIConfig
+ subscription bool
+ wantTimeout time.Duration
+ wantConcurrency int
+ }{
+ {
+ name: "codex subscription patrol model",
+ cfg: AIConfig{PatrolModel: AIProviderCodexSubscription + ":gpt-5.6-luna"},
+ subscription: true,
+ wantTimeout: DefaultPatrolSubscriptionInvestigationTimeoutSec * time.Second,
+ wantConcurrency: 1,
+ },
+ {
+ name: "claude subscription inherited from the main model",
+ cfg: AIConfig{Model: AIProviderClaudeSubscription + ":sonnet"},
+ subscription: true,
+ wantTimeout: DefaultPatrolSubscriptionInvestigationTimeoutSec * time.Second,
+ wantConcurrency: 1,
+ },
+ {
+ name: "api key provider",
+ cfg: AIConfig{PatrolModel: AIProviderAnthropic + ":claude-sonnet-4-5"},
+ wantTimeout: DefaultPatrolInvestigationTimeoutSec * time.Second,
+ wantConcurrency: MaxConcurrentInvestigations,
+ },
+ {
+ name: "explicit timeout wins over the subscription default",
+ cfg: AIConfig{PatrolModel: AIProviderCodexSubscription + ":gpt-5.6-luna", PatrolInvestigationTimeoutSec: 900},
+ subscription: true,
+ wantTimeout: 900 * time.Second,
+ wantConcurrency: 1,
+ },
+ }
+ for _, tc := range cases {
+ t.Run(tc.name, func(t *testing.T) {
+ if got := tc.cfg.PatrolUsesSubscriptionAgent(); got != tc.subscription {
+ t.Fatalf("PatrolUsesSubscriptionAgent() = %v, want %v", got, tc.subscription)
+ }
+ if got := tc.cfg.GetPatrolInvestigationTimeout(); got != tc.wantTimeout {
+ t.Fatalf("GetPatrolInvestigationTimeout() = %s, want %s", got, tc.wantTimeout)
+ }
+ if got := tc.cfg.GetPatrolInvestigationConcurrency(); got != tc.wantConcurrency {
+ t.Fatalf("GetPatrolInvestigationConcurrency() = %d, want %d", got, tc.wantConcurrency)
+ }
+ })
+ }
+}
diff --git a/internal/telemetry/patrol_cost.go b/internal/telemetry/patrol_cost.go
index 8a3009e222..e1e8e5cc65 100644
--- a/internal/telemetry/patrol_cost.go
+++ b/internal/telemetry/patrol_cost.go
@@ -253,6 +253,15 @@ func PatrolInvestigationOutcomeBucket(outcome, status string) string {
return PatrolInvestigationOutcomeCannotFix
case "timed_out":
return PatrolInvestigationOutcomeTimedOut
+ case "unclassified":
+ // A finished run whose conclusion could not be read. Every other
+ // outcome the orchestrator writes has its own bucket, so "other"
+ // effectively isolates these runs.
+ return PatrolInvestigationOutcomeOther
+ case "interrupted":
+ // A restart cut the run short and the finding awaits another run,
+ // so it is not yet concluded.
+ return PatrolInvestigationOutcomeInProgress
case "":
switch strings.ToLower(strings.TrimSpace(status)) {
case "pending", "running":
diff --git a/internal/telemetry/patrol_cost_test.go b/internal/telemetry/patrol_cost_test.go
index 3975fc6aa2..575d150919 100644
--- a/internal/telemetry/patrol_cost_test.go
+++ b/internal/telemetry/patrol_cost_test.go
@@ -234,6 +234,8 @@ func TestPatrolInvestigationOutcomeBucketPartitionsEveryState(t *testing.T) {
{"needs_attention", "needs_attention", PatrolInvestigationOutcomeNeedsAttention},
{"cannot_fix", "completed", PatrolInvestigationOutcomeCannotFix},
{"timed_out", "failed", PatrolInvestigationOutcomeTimedOut},
+ {"unclassified", "completed", PatrolInvestigationOutcomeOther},
+ {"interrupted", "failed", PatrolInvestigationOutcomeInProgress},
{"", "pending", PatrolInvestigationOutcomeInProgress},
{"", "running", PatrolInvestigationOutcomeInProgress},
{"", "failed", PatrolInvestigationOutcomeFailed},
@@ -252,7 +254,7 @@ func TestPatrolInvestigationOutcomeBucketPartitionsEveryState(t *testing.T) {
if counts.Total() != len(cases) {
t.Fatalf("outcome counts total = %d, want %d (every investigated finding must land in exactly one bucket)", counts.Total(), len(cases))
}
- if counts.FixFailed != 2 || counts.InProgress != 2 || counts.Other != 3 || counts.NeedsAttention != 2 {
+ if counts.FixFailed != 2 || counts.InProgress != 3 || counts.Other != 4 || counts.NeedsAttention != 2 || counts.TimedOut != 1 {
t.Fatalf("unexpected partition: %+v", counts)
}
}
diff --git a/pkg/aicontracts/contracts_test.go b/pkg/aicontracts/contracts_test.go
index be93a154a6..061cc77832 100644
--- a/pkg/aicontracts/contracts_test.go
+++ b/pkg/aicontracts/contracts_test.go
@@ -420,3 +420,63 @@ func TestInvestigationBudgetSeparatesEvidenceCallsFromModelTurns(t *testing.T) {
}
}
}
+
+func TestInvestigationSessionCarriesItsRunDeadline(t *testing.T) {
+ payload, err := json.Marshal(EmptyInvestigationSession())
+ if err != nil {
+ t.Fatalf("marshal session: %v", err)
+ }
+ if strings.Contains(string(payload), `"deadline"`) {
+ t.Fatalf("a session with no recorded deadline must omit it: %s", payload)
+ }
+ deadline := time.Date(2026, 10, 23, 1, 30, 0, 0, time.UTC)
+ session := EmptyInvestigationSession()
+ session.Deadline = &deadline
+ payload, err = json.Marshal(session)
+ if err != nil {
+ t.Fatalf("marshal session: %v", err)
+ }
+ if !strings.Contains(string(payload), `"deadline":"2026-10-23T01:30:00Z"`) {
+ t.Fatalf("session must carry its run deadline: %s", payload)
+ }
+}
+
+func TestInvestigationResultCarriesTypedConclusion(t *testing.T) {
+ resultType := reflect.TypeOf(OrchestratorInvestigationResult{})
+ field, ok := resultType.FieldByName("Conclusion")
+ if !ok {
+ t.Fatal("investigation result must carry a typed Conclusion")
+ }
+ if field.Type != reflect.TypeOf((*InvestigationConclusion)(nil)) {
+ t.Fatalf("Conclusion field type = %v, want *InvestigationConclusion", field.Type)
+ }
+
+ empty, err := json.Marshal(OrchestratorInvestigationResult{})
+ if err != nil {
+ t.Fatalf("marshal empty result: %v", err)
+ }
+ if strings.Contains(string(empty), "conclusion") {
+ t.Fatalf("a run without a conclusion must omit the field: %s", empty)
+ }
+ payload, err := json.Marshal(OrchestratorInvestigationResult{Conclusion: &InvestigationConclusion{Outcome: OutcomeCannotFix, Reason: "no capability covers pool expansion"}})
+ if err != nil {
+ t.Fatalf("marshal concluded result: %v", err)
+ }
+ if !strings.Contains(string(payload), `"conclusion":{"outcome":"cannot_fix","reason":"no capability covers pool expansion"}`) {
+ t.Fatalf("conclusion wire shape changed: %s", payload)
+ }
+}
+
+func TestUnclassifiedAndInterruptedAreCanonicalWireValues(t *testing.T) {
+ for outcome, want := range map[InvestigationOutcome]string{
+ OutcomeUnclassified: "unclassified",
+ OutcomeInterrupted: "interrupted",
+ } {
+ if string(outcome) != want {
+ t.Fatalf("outcome %q must keep wire value %q", outcome, want)
+ }
+ }
+ if _, ok := reflect.TypeOf((*InvestigationConcurrencyLimitUpdater)(nil)).Elem().MethodByName("SetInvestigationConcurrencyLimit"); !ok {
+ t.Fatal("the optional concurrency updater must keep SetInvestigationConcurrencyLimit")
+ }
+}
diff --git a/pkg/aicontracts/interfaces.go b/pkg/aicontracts/interfaces.go
index 8f6ecc8161..4ed094003c 100644
--- a/pkg/aicontracts/interfaces.go
+++ b/pkg/aicontracts/interfaces.go
@@ -35,6 +35,14 @@ type InvestigationExecutionLimitUpdater interface {
SetInvestigationExecutionLimits(maxTurns int, timeout time.Duration)
}
+// InvestigationConcurrencyLimitUpdater is the optional live-configuration
+// contract for orchestrators whose concurrent-investigation limit can change
+// without rebuilding the orchestrator. A Patrol provider that serves one call
+// at a time needs one investigation at a time.
+type InvestigationConcurrencyLimitUpdater interface {
+ SetInvestigationConcurrencyLimit(maxConcurrent int)
+}
+
// InvestigationStoreMaintainer is an optional interface for orchestrators that
// expose their investigation store for periodic maintenance.
type InvestigationStoreMaintainer interface {
diff --git a/pkg/aicontracts/investigation.go b/pkg/aicontracts/investigation.go
index 5693f0e497..a1bd4158d1 100644
--- a/pkg/aicontracts/investigation.go
+++ b/pkg/aicontracts/investigation.go
@@ -46,8 +46,26 @@ const (
OutcomeNeedsAttention InvestigationOutcome = "needs_attention"
OutcomeCannotFix InvestigationOutcome = "cannot_fix"
OutcomeTimedOut InvestigationOutcome = "timed_out"
+ // OutcomeUnclassified records a completed run whose conclusion could
+ // not be read: it recorded no structured conclusion and its summary has
+ // no recognisable conclusion marker. It is not a request for human
+ // intervention, so the finding stays eligible for another run.
+ OutcomeUnclassified InvestigationOutcome = "unclassified"
+ // OutcomeInterrupted records a run cancelled before its deadline, for
+ // example by a Pulse restart or shutdown. It is neither a timeout nor a
+ // conclusion, so the finding stays eligible for another run.
+ OutcomeInterrupted InvestigationOutcome = "interrupted"
)
+// InvestigationConclusion is the structured zero-action conclusion a Patrol
+// investigation records with patrol_conclude_investigation. Outcome is
+// OutcomeResolved, OutcomeCannotFix or OutcomeNeedsAttention. Remediation
+// still travels only as a typed action proposal.
+type InvestigationConclusion struct {
+ Outcome InvestigationOutcome `json:"outcome"`
+ Reason string `json:"reason"`
+}
+
// ErrVerificationUnknown indicates the verifier could not conclusively determine
// whether a fix resolved the underlying issue. Callers may treat this as a
// distinct outcome from "verification failed" (issue persists).
@@ -187,6 +205,10 @@ type InvestigationSession struct {
EvidenceIDs []string `json:"evidence_ids"`
Summary string `json:"summary,omitempty"`
Error string `json:"error,omitempty"`
+ // Deadline is when this run's own execution timeout expires. A run keeps
+ // the timeout it started with, so recovery measures a live run against
+ // this deadline rather than the timeout configured now.
+ Deadline *time.Time `json:"deadline,omitempty"`
}
func EmptyInvestigationSession() InvestigationSession {
diff --git a/pkg/aicontracts/orchestrator_deps.go b/pkg/aicontracts/orchestrator_deps.go
index 45fc75aa66..fcf3e1d875 100644
--- a/pkg/aicontracts/orchestrator_deps.go
+++ b/pkg/aicontracts/orchestrator_deps.go
@@ -69,6 +69,9 @@ type OrchestratorInvestigationResult struct {
ModelTurns int `json:"model_turns"`
EvidenceCalls int `json:"evidence_calls"`
ToolCalls int `json:"tool_calls"`
+ // Conclusion is the model's structured zero-action conclusion, nil when
+ // the run recorded none. It carries no remediation authority.
+ Conclusion *InvestigationConclusion `json:"conclusion,omitempty"`
}
// OrchestratorFindingsStore provides access to patrol findings for the orchestrator.