Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

Executors that worked and could not be reached, and a writeback that now checks itself (#800)

Beenu Aroracb2fc445ac0b
diff --git a/CHANGELOG.md b/CHANGELOG.md
index cb166ce1d..48fe0290a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -175,6 +175,78 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`labeled` is in the trigger types. A unit-tested function with no caller is
indistinguishable from a working feature until something asserts the call.
+- **Two response verbs worked and could not be reached.** `ack_alert` and
+ `suppress_alert` have had Splunk, Elastic and Defender arms since Phase 3.3
+ and were wired into `EXECUTOR_REGISTRY` — and appeared in none of the three
+ registries that make a verb dispatchable: the live-action adapters, the
+ capability contracts, or the capability vocabulary. Governed dispatch
+ answered `executor_not_found` for code that ran, which reads as a
+ misconfigured integration rather than a capability nobody connected. This is
+ the mirror image of a defect already fixed here in the other direction,
+ where eleven capabilities had a contract and no executor at all — including
+ `unisolate_host`, so the rollback for the most disruptive action in the
+ product resolved to nothing.
+
+ Both verbs now have six vendor adapters (one per vendor arm), a capability
+ contract, an entry in the vocabulary on both sides of the mirror, and a
+ verification probe. Each adapter pins `alert_vendor` so a tenant with two
+ SIEMs configured does not have the target chosen by credential ordering; the
+ pin is still checked against the credentials, so pinning a vendor the tenant
+ has not configured simulates rather than claiming an arm that could not have
+ run.
+
+ `ack_alert` is LOW impact and automatic: it marks a finding in-progress and
+ owned by AiSOC, removing nothing from anyone's view, and two analysts
+ working the same notable is the cost of not doing it. `suppress_alert` is
+ LOW impact and **analyst-gated**, because what makes the disposition
+ writeback safe to automate is the mapping that refuses to close a confirmed
+ true positive, and this verb has no such bound — it closes whatever it is
+ pointed at, on the caller's say-so.
+
+- **A one-directional gate would have missed all of it.** `check_action_contract.py`
+ now compares the four registries a verb needs in **both** directions, and
+ `test_capability_reachability.py` injects drift in each direction and asserts
+ the gate names it. The dominant failure shape in this repository is a check
+ that compares A against B and never B against A, so drift in the direction
+ things actually change passes while the check prints OK — the graph-schema
+ check reported OK with 17 node labels declared and 28 implemented.
+
+ Running it found three more mismatches beyond the two above. `create_ticket`
+ and `notify` were registered, contracted and dispatchable across four
+ vendors while absent from `KNOWN_CAPABILITIES` and the connectors
+ `Capability` enum, so the registry logged `capability_unknown` for them at
+ every startup; both are now in the vocabulary. `chatops_verify` has an
+ executor no adapter reaches, and three `ActionType` members
+ (`capture_forensics`, `add_ioc_to_blocklist`, `run_playbook`) have no
+ executor at all — the first of those is proposed by name by the
+ investigation agent on the C2/exfiltration path, so the product recommends
+ evidence acquisition it cannot perform. Those four are recorded in the
+ gate's exemption lists with the reason each is open. Both lists are
+ ratchets: an entry that gains an implementation and is not removed fails the
+ build.
+
+- **The disposition writeback now verifies itself against the vendor.** It
+ shipped declaring no verification probe, which was honest — a declared probe
+ that does not run is the defect the contract gate exists to catch — but the
+ standing rule is that an unverifiable action is not an autonomous one, and
+ this action is automatic. The probe re-reads the finding and compares its
+ state against the plan re-derived from the same verdict through the same
+ `plan_writeback` the executor used, rather than being told separately what
+ to expect. Splunk ES reads back the `incident_review` collection (a new
+ `SplunkClient.get_notable_event_state`) and confirms status `5` on a close,
+ or status `1` *and* the owner AiSOC set on an escalation, since status alone
+ cannot distinguish a notable an analyst had already picked up. QRadar
+ confirms `CLOSED` through `get_offense`, whose docstring had claimed to be
+ the writeback probe since it was written and had no caller.
+
+ A QRadar **escalation** deliberately reports `unverified`: escalating leaves
+ the offense `OPEN`, which is also its prior state, so confirming "still
+ OPEN" would certify a write that never happened — the same shape as the
+ isolation probe that returned `bool(device_id)` and would have certified an
+ uncontained host. Elastic, Sentinel and Defender expose no read of a
+ finding's state and report `unverified` too. `ack_alert` and
+ `suppress_alert` are verified against the same read-back.
+
- **A dry run called the customer's production SIEM.** The live-action dry-run
path works by stripping credentials so the executor falls through to
simulation, and the strip list did not match what the client factory reads:
diff --git a/apps/docs/docs/compliance/evidence-pack.md b/apps/docs/docs/compliance/evidence-pack.md
index ae0d80499..402241637 100644
--- a/apps/docs/docs/compliance/evidence-pack.md
+++ b/apps/docs/docs/compliance/evidence-pack.md
@@ -33,7 +33,7 @@ and says so.
| Prompt-injection resistance | 32-payload adversarial corpus with a recall floor and a false-positive ceiling | `services/agents/tests/adversarial/` |
| Claim-to-gate traceability | Every product claim mapped to the CI job that fails when it stops being true | `docs/audit/CLAIM_TO_GATE_MATRIX.md` |
-The last row is the one worth reading first. It is the honest index: 105 rows
+The last row is the one worth reading first. It is the honest index: 106 rows
`GATED`, 9 `PARTIAL` with the specific gap named, and a ratchet that refuses
to let a row sit at `NO GATE`.
diff --git a/apps/docs/docs/concepts/live-actions.md b/apps/docs/docs/concepts/live-actions.md
index a9aaf3e4f..b2de83681 100644
--- a/apps/docs/docs/concepts/live-actions.md
+++ b/apps/docs/docs/concepts/live-actions.md
@@ -216,7 +216,55 @@ included in the agent's planning catalogue. No core changes required.
these fields.
- **Soft capability validation.** Capabilities outside the canonical
enum are accepted (with a warning) so plugins can ship novel verbs
- without waiting for a core release.
+ without waiting for a core release. That leniency is for *plugins*.
+ In-tree executors are held to the vocabulary by the reachability gate
+ below, because a warning in a startup log is not a control.
+
+---
+
+## Reachability
+
+Four registries have to agree before a verb can be dispatched under
+governance:
+
+| Registry | What it supplies |
+| --- | --- |
+| `EXECUTOR_REGISTRY` | The vendor call itself, keyed by `ActionType` |
+| `_BUILTIN_ADAPTERS` | The `(vendor_id, capability)` pair the dispatcher looks up |
+| `CAPABILITY_CONTRACTS` | Impact, reversibility, verification, approval tier |
+| `KNOWN_CAPABILITIES` | The vocabulary everything validates against |
+
+A verb can be complete in three of the four and unreachable, and nothing in
+the failure message says which one is missing. `ack_alert` and
+`suppress_alert` were exactly that for several releases: real executors with
+Splunk, Elastic and Defender arms, wired into `EXECUTOR_REGISTRY`, and absent
+from the other three. Governed dispatch answered `executor_not_found` for code
+that worked, which reads as a misconfigured integration rather than a
+capability nobody connected.
+
+`scripts/check_action_contract.py` now compares all four, **in both
+directions**, on every pull request. Both directions is the part that matters.
+The characteristic failure of a drift check is that it compares A against B
+and never B against A, so drift in the direction things actually change passes
+while the check prints OK — this repository's graph-schema check did precisely
+that, reporting OK with 17 node labels declared in YAML and 28 implemented in
+Go, because it only ever looked for labels the YAML had and the code lacked.
+
+The gate reports:
+
+- an executor whose capability is absent from the vocabulary, or from the
+ contracts, or which never registers;
+- a capability with a contract and no executor behind it;
+- a legacy `ActionType` executor that no adapter reaches, so it bypasses the
+ contract, the approval matrix and the autonomy policy;
+- an adapter pointing at an `ActionType` with no executor, which would fail at
+ execution rather than at registration;
+- an `ActionType` with no executor at all.
+
+Two exemption lists carry the cases that are known and not yet closed, each
+with the reason it is open. They are ratchets: an entry that gains an
+implementation and is not removed fails the build, because a baseline nobody
+prunes is a gate that quietly stops checking.
---
diff --git a/apps/docs/docs/integrations/siem-writeback.md b/apps/docs/docs/integrations/siem-writeback.md
index 1e429d97a..959c685d8 100644
--- a/apps/docs/docs/integrations/siem-writeback.md
+++ b/apps/docs/docs/integrations/siem-writeback.md
@@ -177,3 +177,38 @@ What bounds it is not the approval tier but the disposition mapping above,
plus the execute flag. Re-opening a finding an analyst may since have actioned
would race them in their own console, so there is no automatic rollback: the
contract says `MANUAL_ONLY` and means it.
+
+### Verification
+
+The writeback shipped declaring no verification probe. That was honest —
+claiming a probe that does not run is worse than claiming nothing — but the
+standing rule is that an unverifiable action is not an autonomous one, and
+this action is automatic. It now has a real read-back.
+
+After a writeback, AiSOC re-reads the finding and compares its state against
+the plan it derived from the same verdict, through the same code path the
+executor used. The expectation is re-derived rather than passed alongside the
+disposition: a probe told separately what to expect can be told wrong.
+
+| Vendor | Closing verdict | Escalating verdict |
+| --- | --- | --- |
+| Splunk ES | Reads the `incident_review` collection; confirms status `5` (closed) | Confirms status `1` (in progress) **and** that the owner is the one AiSOC set |
+| QRadar | Reads the offense; confirms status `CLOSED` | Not verifiable — see below |
+| Elastic, Sentinel, Defender | Not verifiable | Not verifiable |
+
+Three results are possible and they mean different things. `verified` is a
+real query that found the effect. `failed` is a real query that found the
+effect **absent** — the write did not land, and the finding is still sitting
+in an analyst's queue while AiSOC believes it is handled. `unverified` means
+no read-back was available, said plainly rather than reported as success.
+
+**A QRadar escalation is honestly unverifiable.** Escalating annotates the
+offense and leaves it `OPEN`, which is also the state it was in beforehand, so
+"still OPEN" would confirm a write that never happened. That is the same shape
+as an earlier isolation probe which returned "does this hostname resolve to a
+device" — true of every host in the fleet, contained or not — and would have
+certified an uncontained host. Where a vendor cannot tell you, AiSOC says so.
+
+The three vendors with no read-back are a limitation of what their APIs
+expose, not a decision. If you run only Elastic, a writeback reports
+`unverified` and that is the accurate answer.
diff --git a/docs/audit/CLAIM_TO_GATE_MATRIX.md b/docs/audit/CLAIM_TO_GATE_MATRIX.md
index 9b098d014..904aca774 100644
--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md
+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md
@@ -106,7 +106,8 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`
| The model never writes SQL and never supplies its own tenant | architecture (investigation tools) | `ci.yml` api job (`test_investigation_tools.py`) — every tool composes parameterised SQL with the tenant bound from the session, an injected `tenant_id` argument is discarded, entity names from untrusted alert payloads are bound not interpolated, and row and lookback limits are clamped | GATED | - |
| A tool that cannot answer reports why, rather than returning nothing | architecture (investigation tools) | `ci.yml` api job — the four tools whose data class nothing ingests return `available: false` naming the connector that would populate it, never an empty result set (which reads to a model as 'I checked and found nothing'), and the test asserts the reason is an ingestion gap rather than an argument error, since both set the same flag | GATED | - |
| Identity, organisational and business context can be imported and is re-runnable | architecture (context graph) | `ci.yml` api job (`test_context_import.py`) — an import never deletes, omitted fields do not blank stored values, `is_active` is derived from `end_date` rather than trusted (an export claiming a departed employee is active would erase the finding this data exists to enable), ids are namespaced per tenant, an unrecognised criticality is rejected rather than coerced because the autonomy policy reads it, and a CMDB host or account the platform has never ingested is reported as a coverage gap rather than created as a node that looks monitored; plus `graph` package tests for CloudAccount derivation from ARNs | GATED | Vulnerability and Secret nodes still need a scanner and CSPM feed; the threat chain needs the threatintel writer |
-| Every response action declares its impact, reversal, permission and verification | README (governed response) | `ci.yml` actions job: `scripts/check_action_contract.py` asserts all 30 executors declare a coherent contract. Defaults are unsafe on purpose (irreversible + prohibited), so an undeclared action fails closed and is named rather than quietly becoming dispatchable. A HIGH-impact action with no verification probe fails here rather than in an incident, and a declared reverse capability absent from the vocabulary fails too - the rollback path would otherwise resolve to nothing while believing it has a route back | GATED | - |
+| Every response action declares its impact, reversal, permission and verification | README (governed response) | `ci.yml` actions job: `scripts/check_action_contract.py` asserts every registered executor declares a coherent contract. Defaults are unsafe on purpose (irreversible + prohibited), so an undeclared action fails closed and is named rather than quietly becoming dispatchable. A HIGH-impact action with no verification probe fails here rather than in an incident, and a declared reverse capability absent from the vocabulary fails too - the rollback path would otherwise resolve to nothing while believing it has a route back | GATED | - |
+| An executor that exists is reachable, and a capability that is declared is implemented | concepts/live-actions.md (reachability) | `ci.yml` actions job: `scripts/check_action_contract.py` compares the four registries a verb needs — the legacy `ActionType` executor, the live-action adapter, the capability contract and the vocabulary — **in both directions**, plus `test_capability_reachability.py`, which injects drift in each direction and asserts the gate names it. `ack_alert` and `suppress_alert` had working Splunk, Elastic and Defender arms and appeared in none of the other three, so governed dispatch answered `executor_not_found` for code that ran; `create_ticket` and `notify` were registered and dispatchable while absent from the vocabulary they validate against. Both exemption lists are ratchets that fail when an entry gains an implementation and is not removed | GATED | three `ActionType` members still have no executor (`capture_forensics`, `add_ioc_to_blocklist`, `run_playbook`) and `chatops_verify` has no adapter — each is recorded in the gate with its reason rather than closed |
| Approval is a function of confidence and impact, and only ever stricter | README (graduated autonomy) | `ci.yml` actions job (`test_approval_matrix.py`, 45 cases) - severe and irreversible impact never auto-execute at any tier or confidence, a contract demanding a human survives L4 at 100%, an unknown tier collapses to the most restrictive, and a missing confidence score is the lowest band rather than a free pass (otherwise a scoring bug becomes an autonomous action) | GATED | - |
| The actions capability mirror matches the connectors enum | capabilities.py (mirror contract) | `ci.yml` actions job - folded into the contract gate. capabilities.py has documented this check at `tests/test_capability_mirror.py` for months; no such file exists, which is how four reverse verbs the contracts name went missing from both sides | GATED | - |
| The benchmark is runnable against a third-party agent | README (public benchmark) | `ci.yml` agents job - `packages/aisoc-benchmark` defines a one-method adapter protocol with no import from this repository, plus an HTTP adapter, and the suite includes a live run against an external stub. A benchmark a vendor cannot run against their own product is a self-report | GATED | the published scoreboard still carries only our own runs; third-party submissions need a review process, not more code |
@@ -125,7 +126,7 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`
## Summary
-- GATED: 105
+- GATED: 106
- PARTIAL: 9
- NO GATE: 0 (**every claim is backed by a failing test.** The last NO GATE — the weekly benchmark scoreboard running live against `main` — closed in Phase E1: `scripts/check_scoreboard.py` ties the published scoreboard to a deterministic per-PR live-agent MITRE-accuracy run, while the funded weekly `wet-eval.yml` appends the LLM-tier rows. The remaining PARTIAL rows are honest, named deferrals — each states the specific gap and what closes it — not unproven claims.)
diff --git a/scripts/check_action_contract.py b/scripts/check_action_contract.py
index cc7012e91..f5debafb7 100644
--- a/scripts/check_action_contract.py
+++ b/scripts/check_action_contract.py
@@ -24,9 +24,12 @@ Run: python3 scripts/check_action_contract.py
from __future__ import annotations
import argparse
+import logging
import sys
from pathlib import Path
+import structlog
+
REPO_ROOT = Path(__file__).resolve().parent.parent
ACTIONS = REPO_ROOT / "services" / "actions"
sys.path.insert(0, str(ACTIONS))
@@ -311,6 +314,190 @@ def check_verification_probes() -> list[str]:
return errors
+#: Legacy ``ActionType`` executors that are deliberately not reachable through
+#: governed live-action dispatch, with the reason each one is not.
+#:
+#: A baseline, not a pass: the gate fails on anything new, and fails again when
+#: one of these gains an adapter and is not removed.
+KNOWN_UNGOVERNED_EXECUTORS: dict[str, str] = {
+ "chatops_verify": (
+ "returns ActionStatus.RUNNING — the prompt was delivered and a human "
+ "has not answered yet. LiveActionStatus has no state for that, and "
+ "_to_live_status collapses everything that is not FAILED or a "
+ "simulation into SUCCEEDED, so registering it as-is would report an "
+ "unanswered question as a completed action. Needs a status for "
+ "'delivered, awaiting a reply' before it can be dispatched honestly."
+ ),
+}
+
+#: ``ActionType`` members with no executor behind them at all. The legacy REST
+#: route answers "No executor found for action type", which reads as a
+#: misconfiguration rather than a verb nobody built.
+KNOWN_ACTION_TYPES_WITHOUT_EXECUTOR: dict[str, str] = {
+ "capture_forensics": (
+ "No executor anywhere. Worse than unused: "
+ "services/agents/app/agents/investigation_agent.py proposes it by name "
+ "on the C2/exfiltration path, so the product recommends evidence "
+ "acquisition it cannot perform, at the point where preserving evidence "
+ "matters most. Needs a real acquisition arm, not a stub."
+ ),
+ "add_ioc_to_blocklist": (
+ "Superseded by block_ioc, which has a Defender arm, a contract and a "
+ "registered adapter. Nothing dispatches this one."
+ ),
+ "run_playbook": (
+ "Playbook execution lives in services/agents. This member is a leftover "
+ "from before that split; the actions service has never run one."
+ ),
+}
+
+
+def check_capability_reachability() -> list[str]:
+ """Every direction between an executor and the things that make it reachable.
+
+ Four registries have to agree before a verb can be dispatched under
+ governance: the legacy ``ActionType`` executor, the live-action adapter,
+ the capability contract, and the capability vocabulary. Nothing compared
+ them, so a verb could be complete in three of the four and unreachable.
+
+ ``ack_alert`` and ``suppress_alert`` were exactly that — real executors
+ with Splunk, Elastic and Defender arms, wired into ``EXECUTOR_REGISTRY``,
+ and absent from the adapters, the contracts and the vocabulary. Governed
+ dispatch answered ``executor_not_found`` for code that worked.
+
+ Every comparison below runs **both ways**, which is the part that matters.
+ The dominant failure in this repository is a check that compares A against
+ B and never B against A: the graph-schema drift check reported OK while the
+ YAML declared 17 labels and the Go code had 28, because it only ever looked
+ for labels the YAML had and the code lacked. A one-directional version of
+ this check would have passed on ``ack_alert`` too — it is missing in the
+ direction nothing was looking.
+ """
+ from app.live_actions import builtins, registry
+ from app.live_actions.capabilities import KNOWN_CAPABILITIES
+ from app.live_actions.capability_contracts import CAPABILITY_CONTRACTS
+ from app.models.action import ActionType
+ from app.services.executor_registry import EXECUTOR_REGISTRY
+
+ errors: list[str] = []
+ adapters = builtins._BUILTIN_ADAPTERS # noqa: SLF001
+ adapter_capabilities = {cls.capability for cls in adapters}
+ adapter_action_types = {getattr(cls, "_legacy_action_type", None) for cls in adapters}
+ adapter_action_types.discard(None)
+
+ # ── Direction 1: adapter → vocabulary ──────────────────────────────────
+ # register_executor() only *warns* on an unknown capability, so this drift
+ # is invisible outside a startup log. check_capability_mirror compares the
+ # connectors enum against the mirror in both directions but neither
+ # against the executors, which is how create_ticket and notify came to be
+ # registered, contracted and dispatchable while absent from the vocabulary
+ # they are validated against.
+ for capability in sorted(adapter_capabilities - KNOWN_CAPABILITIES):
+ errors.append(
+ f"{capability}: a registered executor implements it, but it is absent "
+ f"from KNOWN_CAPABILITIES and therefore from the connectors "
+ f"Capability enum. The registry logs live_action.capability_unknown "
+ f"for it at every startup and anything validating against the "
+ f"vocabulary rejects it. Add it to both."
+ )
+
+ # ── Direction 2: adapter → contract ────────────────────────────────────
+ # apply_contract() leaves the unsafe defaults when there is no entry, which
+ # the per-executor grading catches indirectly. Name it directly so the
+ # failure says what to do rather than "declares no required_permission".
+ for capability in sorted(adapter_capabilities - set(CAPABILITY_CONTRACTS)):
+ errors.append(
+ f"{capability}: a registered executor implements it and no capability "
+ f"contract declares what it does to an estate. Nothing decided its "
+ f"impact, reversibility, verification or approval tier."
+ )
+
+ # ── Direction 3: adapter → live-action registry ────────────────────────
+ # An adapter that never registers is as unreachable as one that does not
+ # exist, and the only difference is that this one looks finished.
+ #
+ # Registering ~60 executors emits a line each; the gate's own output is
+ # what a failing build needs to be readable, so drop anything below ERROR
+ # for the duration.
+ structlog.configure(wrapper_class=structlog.make_filtering_bound_logger(logging.ERROR))
+ registry.reset_for_tests()
+ builtin_count = builtins.register_builtin_executors(overwrite=True)
+ for cls in adapters:
+ if registry.get_executor(cls.vendor_id, cls.capability) is None:
+ errors.append(f"{cls.__module__}.{cls.__name__}: declares " f"{cls.vendor_id}/{cls.capability} but did not register")
+ if builtin_count != len(adapters):
+ errors.append(f"register_builtin_executors registered {builtin_count} of " f"{len(adapters)} adapters")
+
+ # ── Direction 4: legacy executor → adapter ─────────────────────────────
+ # The one that was missing. A legacy executor with no adapter is reachable
+ # only through the ActionType REST route, which has no contract, no
+ # approval matrix and no autonomy policy in front of it.
+ for action_type in sorted(EXECUTOR_REGISTRY, key=lambda a: a.value):
+ if action_type in adapter_action_types:
+ continue
+ if action_type.value in KNOWN_UNGOVERNED_EXECUTORS:
+ continue
+ executor = EXECUTOR_REGISTRY[action_type]
+ errors.append(
+ f"{action_type.value}: {type(executor).__name__} is registered in "
+ f"EXECUTOR_REGISTRY and no live-action adapter reaches it. Governed "
+ f"dispatch answers executor_not_found for a verb that works, so the "
+ f"code is unreachable from the agent loop, the planner and the "
+ f"approval matrix. Add an adapter per vendor arm, a capability "
+ f"contract, and the verb to the vocabulary."
+ )
+
+ # ── Direction 5: adapter → legacy executor ─────────────────────────────
+ # The mirror of 4. An adapter pointing at an ActionType with no legacy
+ # executor raises at execute() rather than at registration.
+ for action_type in sorted(adapter_action_types, key=lambda a: a.value):
+ if action_type not in EXECUTOR_REGISTRY:
+ errors.append(
+ f"{action_type.value}: a live-action adapter delegates to this "
+ f"ActionType and EXECUTOR_REGISTRY has no executor for it. The "
+ f"failure would surface at execute(), not at registration."
+ )
+
+ # ── Direction 6: ActionType → legacy executor ──────────────────────────
+ for action_type in sorted(ActionType, key=lambda a: a.value):
+ if action_type in EXECUTOR_REGISTRY:
+ continue
+ if action_type.value in KNOWN_ACTION_TYPES_WITHOUT_EXECUTOR:
+ continue
+ errors.append(
+ f"{action_type.value}: an ActionType with no executor in "
+ f"EXECUTOR_REGISTRY. The API accepts it and then answers 'No "
+ f"executor found for action type', which reads as a broken "
+ f"deployment rather than a verb that was never built."
+ )
+
+ errors.extend(_ratchet("KNOWN_UNGOVERNED_EXECUTORS", KNOWN_UNGOVERNED_EXECUTORS, {a.value for a in adapter_action_types}))
+ errors.extend(
+ _ratchet(
+ "KNOWN_ACTION_TYPES_WITHOUT_EXECUTOR",
+ KNOWN_ACTION_TYPES_WITHOUT_EXECUTOR,
+ {a.value for a in EXECUTOR_REGISTRY},
+ )
+ )
+ return errors
+
+
+def _ratchet(name: str, baseline: dict[str, str], resolved_set: set[str]) -> list[str]:
+ """A baseline may only ever shrink.
+
+ Without this the exemption list is where the next one hides: an entry that
+ quietly gained an implementation leaves the gate checking one fewer thing
+ every release, and nothing says so.
+ """
+ resolved = sorted(set(baseline) & resolved_set)
+ if not resolved:
+ return []
+ return [
+ f"{name} lists {', '.join(resolved)}, which is no longer missing. "
+ f"Remove the entry — a stale exemption is a gate that stopped checking."
+ ]
+
+
def check_capability_mirror() -> list[str]:
"""The actions mirror must match the connectors Capability enum.
@@ -369,6 +556,7 @@ def main(argv: list[str] | None = None) -> int:
+ check_capability_mirror()
+ check_verification_probes()
+ check_no_orphan_capabilities()
+ + check_capability_reachability()
)
except ImportError as exc:
print(f"action-contract: cannot import the actions package: {exc}", file=sys.stderr)
diff --git a/services/actions/app/clients/splunk_client.py b/services/actions/app/clients/splunk_client.py
index 9e65d0d74..9f6d8d020 100644
--- a/services/actions/app/clients/splunk_client.py
+++ b/services/actions/app/clients/splunk_client.py
@@ -12,6 +12,7 @@ Credentials expected in ActionRequest.parameters:
from __future__ import annotations
+import json
from typing import Any
import httpx
@@ -245,6 +246,53 @@ class SplunkClient:
"response": resp.json() if resp.content else {},
}
+ async def get_notable_event_state(self, event_id: str) -> dict[str, Any] | None:
+ """Read a notable event's current status and owner back from Splunk ES.
+
+ The read-back half of acknowledge / suppress / disposition writeback.
+ Every lifecycle change made through ``/services/notable_update`` lands
+ in the ``incident_review`` KV store collection, keyed by ``rule_id`` —
+ the same value sent there as ``ruleUIDs`` — so this reads the effect of
+ the write rather than the acceptance of the request.
+
+ Returns ``{"status": "5", "owner": "aisoc"}`` for the most recent
+ review entry, or ``None`` when the collection cannot be read (Splunk ES
+ not installed, the token lacks access) or has no entry for this
+ notable. ``None`` is deliberate and load-bearing: "we cannot tell" and
+ "the write did not land" are different facts, and only the caller knows
+ which state it was expecting.
+ """
+ async with httpx.AsyncClient(timeout=30.0, verify=self._verify_ssl) as client:
+ await self._authenticate(client)
+ resp = await client.get(
+ f"{self._host}/servicesNS/nobody/SplunkEnterpriseSecuritySuite/storage/collections/data/incident_review",
+ headers=self._headers(),
+ params={
+ "query": json.dumps({"rule_id": event_id}),
+ "sort": "-time",
+ "limit": 1,
+ "output_mode": "json",
+ },
+ )
+ if resp.status_code == 404:
+ # The Enterprise Security app is not installed, so there is no
+ # incident review store to read. Not an error — this deployment
+ # simply has no read-back, and saying so beats raising.
+ logger.info("splunk.notable_event.no_incident_review", event_id=event_id)
+ return None
+ resp.raise_for_status()
+ entries = resp.json() if resp.content else []
+ if not isinstance(entries, list) or not entries:
+ return None
+ entry = entries[0]
+ if not isinstance(entry, dict):
+ return None
+ status = entry.get("status")
+ return {
+ "status": None if status is None else str(status),
+ "owner": entry.get("owner"),
+ }
+
async def suppress_notable_event(
self,
event_id: str,
diff --git a/services/actions/app/live_actions/builtins.py b/services/actions/app/live_actions/builtins.py
index 8b6dcb953..b58d445c3 100644
--- a/services/actions/app/live_actions/builtins.py
+++ b/services/actions/app/live_actions/builtins.py
@@ -64,9 +64,11 @@ from app.executors.notification import (
NotifySlackExecutor,
)
from app.executors.siem import (
+ AckAlertExecutor,
BlockIOCExecutor,
CreateNotableEventExecutor,
SearchSIEMExecutor,
+ SuppressAlertExecutor,
SyncDetectionRuleExecutor,
UpdateAlertDispositionExecutor,
UpdateWatcherExecutor,
@@ -561,6 +563,106 @@ class DefenderUpdateAlertDisposition(_DispositionWriteback):
_credential_keys = _DEFENDER_IOC_KEYS
+# ---------------------------------------------------------------------------
+# Alert lifecycle — acknowledge and suppress
+# ---------------------------------------------------------------------------
+#
+# Both executors have had Splunk, Elastic and Defender arms since Phase 3.3,
+# sat in EXECUTOR_REGISTRY the whole time, and had no adapter here — so
+# governed dispatch answered executor_not_found for working code, and the only
+# route to it was the ActionType REST endpoint, which has no capability
+# contract, no approval matrix and no autonomy policy in front of it.
+#
+# `alert_vendor` is pinned per arm for the same reason the writeback pins it:
+# a tenant with two SIEMs configured would otherwise have the target chosen by
+# whichever credential block `_ack_vendor` happens to check first. The pin is
+# still verified against the credentials inside `siem._ack_vendor`, so pinning
+# a vendor the tenant has not configured simulates rather than claiming an arm
+# that could not have run.
+
+
+class _AlertLifecycleAdapter(_LegacyExecutorAdapter):
+ """Shared body for the ack / suppress vendor arms.
+
+ Declares no ``capability`` and no ``vendor_id``: an intermediate class
+ naming one without the other is graded by the action-contract gate as a
+ half-declared executor. Each concrete arm below declares both.
+ """
+
+ requires_credentials = True
+
+ async def execute(self, request: LiveActionRequest) -> LiveActionResult:
+ pinned = {**request.params, "alert_vendor": self.vendor_id}
+ return await super().execute(request.model_copy(update={"params": pinned}))
+
+ def _summarise(self, output: dict[str, Any], status: LiveActionStatus) -> str:
+ alert_id = output.get("alert_id") or ""
+ verb = self.capability.replace("_", " ")
+ if status == LiveActionStatus.FAILED:
+ return f"Failed to {verb} {self.vendor_id} finding {alert_id}".strip()
+ if status == LiveActionStatus.SIMULATED:
+ return f"Simulated {verb} on {self.vendor_id} finding {alert_id}".strip()
+ return f"{verb.capitalize()} on {self.vendor_id} finding {alert_id}".strip()
+
+
+class _AckAlert(_AlertLifecycleAdapter):
+ _legacy_executor = AckAlertExecutor()
+ _legacy_action_type = ActionType.ACK_ALERT
+
+
+class _SuppressAlert(_AlertLifecycleAdapter):
+ _legacy_executor = SuppressAlertExecutor()
+ _legacy_action_type = ActionType.SUPPRESS_ALERT
+
+
+@apply_contract
+class SplunkAckAlert(_AckAlert):
+ capability = "ack_alert"
+ vendor_id = "splunk"
+ description = "Acknowledge a Splunk ES notable and assign it to AiSOC."
+ _credential_keys = _SPLUNK_KEYS
+
+
+@apply_contract
+class ElasticAckAlert(_AckAlert):
+ capability = "ack_alert"
+ vendor_id = "elastic"
+ description = "Acknowledge an Elastic Security signal."
+ _credential_keys = _ELASTIC_KEYS
+
+
+@apply_contract
+class DefenderAckAlert(_AckAlert):
+ capability = "ack_alert"
+ vendor_id = "defender"
+ description = "Acknowledge a Microsoft Defender alert and assign it."
+ _credential_keys = _DEFENDER_IOC_KEYS
+
+
+@apply_contract
+class SplunkSuppressAlert(_SuppressAlert):
+ capability = "suppress_alert"
+ vendor_id = "splunk"
+ description = "Close a Splunk ES notable event."
+ _credential_keys = _SPLUNK_KEYS
+
+
+@apply_contract
+class ElasticSuppressAlert(_SuppressAlert):
+ capability = "suppress_alert"
+ vendor_id = "elastic"
+ description = "Close an Elastic Security signal."
+ _credential_keys = _ELASTIC_KEYS
+
+
+@apply_contract
+class DefenderSuppressAlert(_SuppressAlert):
+ capability = "suppress_alert"
+ vendor_id = "defender"
+ description = "Resolve a Microsoft Defender alert with a classification."
+ _credential_keys = _DEFENDER_IOC_KEYS
+
+
# ---------------------------------------------------------------------------
# Phase B2 — previously-unregistered vendor adapters.
#
@@ -733,6 +835,14 @@ _BUILTIN_ADAPTERS: tuple[type[LiveActionExecutor], ...] = (
SentinelUpdateAlertDisposition,
QRadarUpdateAlertDisposition,
DefenderUpdateAlertDisposition,
+ # Alert lifecycle: executors that existed with three vendor arms each and
+ # were unreachable through governed dispatch.
+ SplunkAckAlert,
+ ElasticAckAlert,
+ DefenderAckAlert,
+ SplunkSuppressAlert,
+ ElasticSuppressAlert,
+ DefenderSuppressAlert,
# Phase B2 — previously-unregistered vendors
SentinelOneIsolateHost,
EntraDisableUser,
diff --git a/services/actions/app/live_actions/capabilities.py b/services/actions/app/live_actions/capabilities.py
index 7ad8b819e..0a62a7211 100644
--- a/services/actions/app/live_actions/capabilities.py
+++ b/services/actions/app/live_actions/capabilities.py
@@ -90,9 +90,18 @@ KNOWN_CAPABILITIES: frozenset[str] = frozenset(
# written back onto the vendor finding that produced the alert.
"update_alert_disposition",
"update_watcher",
+ # Alert lifecycle. Executors with three vendor arms each existed for
+ # both of these while neither verb appeared here, in the contracts or
+ # in the live-action registry, so governed dispatch answered
+ # executor_not_found for code that worked.
+ "ack_alert",
+ "suppress_alert",
# TICKET
"push_case",
"push_status",
+ "create_ticket",
+ # NOTIFY
+ "notify",
# AUDIT
"read_audit_trail",
}
diff --git a/services/actions/app/live_actions/capability_contracts.py b/services/actions/app/live_actions/capability_contracts.py
index c2fc3d10d..18c53ad79 100644
--- a/services/actions/app/live_actions/capability_contracts.py
+++ b/services/actions/app/live_actions/capability_contracts.py
@@ -381,6 +381,7 @@ CAPABILITY_CONTRACTS: dict[str, CapabilityContract] = {
approval=ApprovalRequirement.AUTOMATIC,
reversal=Reversal.MANUAL_ONLY,
required_permission=_TICKET,
+ has_verification_probe=True,
note=(
"Writes a verdict onto a finding in the customer's SIEM. Classified "
"the same as create_notable_event and push_status, which it is the "
@@ -391,7 +392,51 @@ CAPABILITY_CONTRACTS: dict[str, CapabilityContract] = {
"closed, and a verdict outside the taxonomy is refused rather than "
"guessed (app/services/disposition_writeback.py). On top of that "
"the feature ships dry-run by default: AISOC_SIEM_WRITEBACK_EXECUTE "
- "is off, so an operator opts in to the vendor call, not out of it."
+ "is off, so an operator opts in to the vendor call, not out of it. "
+ "The probe re-reads the finding and compares its state against the "
+ "plan the executor derived from the same disposition: Splunk's "
+ "notable status and QRadar's offense status both change observably "
+ "on a close. A QRadar escalation leaves the offense OPEN, which is "
+ "also its prior state, so that combination answers indeterminate "
+ "rather than confirming a write that may not have happened."
+ ),
+ ),
+ # ── Alert lifecycle ────────────────────────────────────────────────────
+ #
+ # Impact is the same as update_alert_disposition, and for the same reason:
+ # these move a queue item, not an estate. The approval tiers differ,
+ # because what bounds the writeback is its disposition mapping and these
+ # two verbs have none — they do what the caller says.
+ "ack_alert": CapabilityContract(
+ impact=ActionImpact.LOW,
+ approval=ApprovalRequirement.AUTOMATIC,
+ reversal=Reversal.MANUAL_ONLY,
+ required_permission=_TICKET,
+ has_verification_probe=True,
+ note=(
+ "Marks a vendor finding in-progress and owned by AiSOC. It removes "
+ "nothing from anyone's view — it says who is already looking, which "
+ "is the point of automating it: two analysts working the same "
+ "notable is the cost of not doing so. The probe reads the notable's "
+ "status back, so the claim is checked rather than inferred from a "
+ "202."
+ ),
+ ),
+ "suppress_alert": CapabilityContract(
+ impact=ActionImpact.LOW,
+ approval=ApprovalRequirement.ANALYST,
+ reversal=Reversal.MANUAL_ONLY,
+ required_permission=_TICKET,
+ has_verification_probe=True,
+ note=(
+ "Closes a vendor finding outright. The impact is a queue item, the "
+ "same as update_alert_disposition — but analyst-gated where that "
+ "one is automatic, because the thing that makes the writeback safe "
+ "to automate is the disposition mapping that refuses to close a "
+ "confirmed true positive, and this verb has no such bound. It "
+ "closes whatever it is pointed at, on the caller's say-so. Prefer "
+ "update_alert_disposition for anything driven by a verdict; this "
+ "exists for an analyst acting deliberately."
),
),
# ── Ticketing and notification ─────────────────────────────────────────
diff --git a/services/actions/app/services/verification.py b/services/actions/app/services/verification.py
index 34a835be1..e32871bf7 100644
--- a/services/actions/app/services/verification.py
+++ b/services/actions/app/services/verification.py
@@ -32,7 +32,9 @@ import structlog
from app.clients.factories import _entra_client, _okta_client
from app.executors.endpoint import _cs_client
+from app.executors.siem import _ack_vendor, _qradar_client, _splunk_client
from app.models.action import ActionType
+from app.services.disposition_writeback import WritebackAction, plan_writeback
logger = structlog.get_logger()
@@ -160,11 +162,112 @@ async def _probe_allow_ip(target: str, params: dict[str, Any]) -> bool | None:
return None if still_blocked is None else not still_blocked
+#: Splunk ES notable status codes. A new notable is 0 ("unassigned"), so
+#: reading 1 back is evidence an acknowledgement landed; 5 is closed.
+_SPLUNK_STATUS_IN_PROGRESS = "1"
+_SPLUNK_STATUS_CLOSED = "5"
+
+
+async def _read_back_finding_state(target: str, params: dict[str, Any], *, expect_closed: bool) -> bool | None:
+ """Re-read a vendor finding and say whether it reached the expected state.
+
+ Shared by the three verbs that move a finding through its lifecycle:
+ acknowledge, suppress, and the disposition writeback that does one or the
+ other depending on the verdict.
+
+ Two of the five vendor arms expose a read-back and three do not, so this
+ answers ``None`` for the rest rather than inventing a confirmation. That
+ is the same shape as ``_probe_block_ip``, where only AWS security groups
+ are readable.
+ """
+ vendor = _ack_vendor(params)
+
+ if vendor == "splunk":
+ splunk = _splunk_client(params)
+ if splunk is None:
+ return None
+ state = await splunk.get_notable_event_state(target)
+ if state is None or state.get("status") is None:
+ return None
+ status = str(state["status"])
+ if expect_closed:
+ # Unambiguous: a notable nobody closed is not status 5.
+ return status == _SPLUNK_STATUS_CLOSED
+ # An acknowledgement sets status to "in progress" *and* assigns an
+ # owner. Status alone would be weak — a notable an analyst had already
+ # picked up reads the same — so the owner has to match too. An analyst
+ # who takes the notable over afterwards makes this report FAILED,
+ # which is the safe direction to be wrong in.
+ expected_owner = str(params.get("owner") or "aisoc")
+ return status == _SPLUNK_STATUS_IN_PROGRESS and str(state.get("owner") or "") == expected_owner
+
+ if vendor == "qradar":
+ qradar = _qradar_client(params)
+ if qradar is None:
+ return None
+ offense = await qradar.get_offense(target)
+ status = str(offense.get("status") or "").upper()
+ if not status:
+ return None
+ if expect_closed:
+ return status == "CLOSED"
+ # An escalation annotates the offense and leaves it OPEN — which is
+ # also the state it was in beforehand. Returning True for "still OPEN"
+ # would certify a write that never happened, which is exactly what the
+ # isolation probe did when it returned bool(device_id). Indeterminate
+ # is the honest answer, and QRadar exposes nothing better: the note
+ # that carries the verdict is not on the offense record.
+ return None
+
+ # Elastic, Sentinel and Defender: their clients expose no read of a
+ # finding's current state, so there is nothing to compare against.
+ return None
+
+
+async def _probe_alert_disposition(target: str, params: dict[str, Any]) -> bool | None:
+ """Confirm an AiSOC verdict actually reached the finding that produced it.
+
+ The writeback shipped claiming no probe, which was honest — a declared
+ probe that does not run is the defect the contract gate exists to catch —
+ but the standing rule is that an unverifiable action is not an autonomous
+ one, and this one is automatic. So it gets a real read-back.
+
+ The expected state is re-derived from the same ``disposition`` through the
+ same :func:`plan_writeback` the executor used, rather than passed in
+ alongside it. A probe told separately what to expect can be told wrong.
+ """
+ plan = plan_writeback(params.get("disposition"), confidence=params.get("confidence"))
+ if plan.action is WritebackAction.REFUSE:
+ # The executor deliberately wrote nothing, so there is no effect to
+ # confirm. Reporting VERIFIED here would mean "the refusal worked",
+ # which is not what a verification outcome is read as.
+ return None
+ return await _read_back_finding_state(target, params, expect_closed=plan.action is WritebackAction.CLOSE)
+
+
+async def _probe_ack_alert(target: str, params: dict[str, Any]) -> bool | None:
+ """Confirm an acknowledgement moved the finding to in-progress and owned."""
+ return await _read_back_finding_state(target, params, expect_closed=False)
+
+
+async def _probe_suppress_alert(target: str, params: dict[str, Any]) -> bool | None:
+ """Confirm a suppression actually closed the finding.
+
+ Worth having in its own right: a close that silently fails leaves the
+ alert in the analyst queue while AiSOC reports it handled, and the two
+ views of the same finding then disagree with nobody watching.
+ """
+ return await _read_back_finding_state(target, params, expect_closed=True)
+
+
_DEFAULT_PROBES: dict[ActionType, Probe] = {
ActionType.ISOLATE_HOST: _probe_isolate,
ActionType.BLOCK_IP: _probe_block_ip,
ActionType.DISABLE_USER: _probe_disable_user,
ActionType.ALLOW_IP: _probe_allow_ip,
+ ActionType.UPDATE_ALERT_DISPOSITION: _probe_alert_disposition,
+ ActionType.ACK_ALERT: _probe_ack_alert,
+ ActionType.SUPPRESS_ALERT: _probe_suppress_alert,
}
diff --git a/services/actions/tests/test_capability_reachability.py b/services/actions/tests/test_capability_reachability.py
new file mode 100644
index 000000000..4800a2cf6
--- /dev/null
+++ b/services/actions/tests/test_capability_reachability.py
@@ -0,0 +1,195 @@
+"""An executor and the registries that make it reachable must agree, both ways.
+
+Four things have to line up before a verb can be dispatched under governance:
+the legacy ``ActionType`` executor, the live-action adapter, the capability
+contract, and the capability vocabulary. Nothing compared them, so a verb could
+be complete in three of the four and unreachable.
+
+``ack_alert`` and ``suppress_alert`` were exactly that — real executors with
+Splunk, Elastic and Defender arms, wired into ``EXECUTOR_REGISTRY``, and absent
+from the adapters, the contracts and the vocabulary. Governed dispatch answered
+``executor_not_found`` for code that worked, which reads as a misconfigured
+integration rather than a capability nobody connected.
+
+The tests below assert both directions of every comparison. A one-directional
+version would have passed on all of this: the drift is in the direction nothing
+was looking, which is what the graph-schema check demonstrated when it reported
+OK with 17 labels declared and 28 implemented.
+"""
+
+from __future__ import annotations
+
+import subprocess
+import sys
+from pathlib import Path
+
+import pytest
+from app.live_actions import builtins, registry
+from app.live_actions.capabilities import KNOWN_CAPABILITIES
+from app.live_actions.capability_contracts import CAPABILITY_CONTRACTS
+from app.models.action import ActionType
+from app.services.executor_registry import EXECUTOR_REGISTRY
+
+REPO_ROOT = Path(__file__).resolve().parents[3]
+GATE = REPO_ROOT / "scripts" / "check_action_contract.py"
+
+# The gate owns the exemption baselines; the tests read them from there rather
+# than restating them, so a second copy cannot drift from the enforced one.
+sys.path.insert(0, str(REPO_ROOT / "scripts"))
+
+ADAPTERS = builtins._BUILTIN_ADAPTERS # noqa: SLF001
+
+#: Adapters that bridge to a legacy ``ActionType`` executor. The read-only
+#: investigation verbs and the vendor-breadth executors are native
+#: ``LiveActionExecutor`` implementations with no legacy counterpart, so the
+#: ActionType comparisons below do not apply to them.
+BRIDGING_ADAPTERS = [cls for cls in ADAPTERS if getattr(cls, "_legacy_action_type", None) is not None]
+
+
[email protected](autouse=True)
+def _registered() -> None:
+ registry.reset_for_tests()
+ builtins.register_builtin_executors(overwrite=True)
+
+
[email protected]("capability", ["ack_alert", "suppress_alert"])
[email protected]("vendor", ["splunk", "elastic", "defender"])
+def test_alert_lifecycle_verbs_resolve_through_governed_dispatch(vendor: str, capability: str) -> None:
+ """The regression. Every one of these six returned None before this change."""
+ assert registry.get_executor(vendor, capability) is not None, (
+ f"{vendor}/{capability} does not resolve, so dispatcher.dispatch answers "
+ f"executor_not_found for an executor that has a working {vendor} arm."
+ )
+
+
[email protected]("capability", sorted({cls.capability for cls in ADAPTERS}))
+def test_every_implemented_capability_is_in_the_vocabulary(capability: str) -> None:
+ """Direction the mirror check never ran.
+
+ ``check_capability_mirror`` compares the connectors enum against the
+ actions mirror in both directions and neither against the executors, which
+ is how ``create_ticket`` and ``notify`` came to be registered, contracted
+ and dispatchable while absent from the vocabulary they are validated
+ against — four executors logging ``capability_unknown`` at every startup.
+ """
+ assert capability in KNOWN_CAPABILITIES
+
+
[email protected]("capability", sorted({cls.capability for cls in ADAPTERS}))
+def test_every_implemented_capability_has_a_contract(capability: str) -> None:
+ detail = f"{capability} is dispatchable and nothing declares its impact, reversibility, verification or approval tier."
+ assert capability in CAPABILITY_CONTRACTS, detail
+
+
+def test_every_legacy_executor_is_reachable_or_explicitly_exempt() -> None:
+ """Direction 4: a legacy executor with no adapter is ungoverned.
+
+ The ActionType REST route has no capability contract, no approval matrix
+ and no autonomy policy in front of it, so an executor reachable only that
+ way bypasses every control the contract system exists to apply.
+ """
+ from check_action_contract import KNOWN_UNGOVERNED_EXECUTORS
+
+ reachable = {cls._legacy_action_type for cls in BRIDGING_ADAPTERS}
+ unreachable = {at.value for at in EXECUTOR_REGISTRY if at not in reachable}
+ assert unreachable <= set(KNOWN_UNGOVERNED_EXECUTORS), (
+ f"{sorted(unreachable - set(KNOWN_UNGOVERNED_EXECUTORS))} have executors that "
+ f"governed dispatch cannot reach and no recorded reason."
+ )
+
+
+def test_every_adapter_delegates_to_an_executor_that_exists() -> None:
+ """Direction 5, the mirror of the above: the failure would be at execute()."""
+ for cls in BRIDGING_ADAPTERS:
+ action_type = cls._legacy_action_type
+ assert action_type in EXECUTOR_REGISTRY, f"{cls.__name__} delegates to {action_type} and EXECUTOR_REGISTRY has no executor for it"
+
+
+def test_exemption_lists_only_ever_shrink() -> None:
+ """A baseline is where the next one hides if nothing prunes it."""
+ from check_action_contract import (
+ KNOWN_ACTION_TYPES_WITHOUT_EXECUTOR,
+ KNOWN_UNGOVERNED_EXECUTORS,
+ )
+
+ reachable = {cls._legacy_action_type for cls in BRIDGING_ADAPTERS}
+ assert not (set(KNOWN_UNGOVERNED_EXECUTORS) & {at.value for at in reachable})
+ assert not (set(KNOWN_ACTION_TYPES_WITHOUT_EXECUTOR) & {at.value for at in EXECUTOR_REGISTRY})
+
+ # Every exemption carries a reason, because "why is this here" is the
+ # question a stale baseline cannot answer.
+ for reason in (*KNOWN_UNGOVERNED_EXECUTORS.values(), *KNOWN_ACTION_TYPES_WITHOUT_EXECUTOR.values()):
+ assert len(reason.strip()) > 40
+
+
+def test_every_action_type_has_an_executor_or_a_recorded_reason() -> None:
+ from check_action_contract import KNOWN_ACTION_TYPES_WITHOUT_EXECUTOR
+
+ missing = {at.value for at in ActionType if at not in EXECUTOR_REGISTRY}
+ assert missing <= set(KNOWN_ACTION_TYPES_WITHOUT_EXECUTOR)
+
+
+def _run_gate() -> subprocess.CompletedProcess:
+ return subprocess.run( # noqa: S603
+ [sys.executable, str(GATE)],
+ capture_output=True,
+ text=True,
+ cwd=str(REPO_ROOT),
+ )
+
+
+def test_the_gate_passes_on_the_tree_as_committed() -> None:
+ result = _run_gate()
+ assert result.returncode == 0, result.stderr
+
+
+ ("drift", "expected"),
+ [
+ # Remove a verb from the vocabulary while its executors stay: the
+ # direction check_capability_mirror covers.
+ ("vocabulary", "absent from KNOWN_CAPABILITIES"),
+ # Remove an adapter while the legacy executor stays: the direction
+ # nothing covered, and the one ack_alert drifted in.
+ ("adapter", "no live-action adapter reaches it"),
+ # Remove the contract while the adapter stays.
+ ("contract", "no capability contract declares"),
+ ],
+)
+def test_the_gate_fails_when_drift_is_introduced_in_each_direction(tmp_path: Path, drift: str, expected: str) -> None:
+ """The gate has to fail for the right reason, not merely fail.
+
+ A check that only ever runs green is indistinguishable from one that does
+ nothing, so each direction is driven by removing exactly one registration
+ and asserting the message names it.
+ """
+ script = {
+ "vocabulary": (
+ "import app.live_actions.capabilities as caps\n"
+ "caps.KNOWN_CAPABILITIES = frozenset(c for c in caps.KNOWN_CAPABILITIES if c != 'ack_alert')\n"
+ ),
+ "adapter": (
+ "import app.live_actions.builtins as b\n"
+ "b._BUILTIN_ADAPTERS = tuple(c for c in b._BUILTIN_ADAPTERS if c.capability != 'ack_alert')\n"
+ ),
+ "contract": "import app.live_actions.capability_contracts as cc\ncc.CAPABILITY_CONTRACTS.pop('ack_alert', None)\n",
+ }[drift]
+
+ driver = tmp_path / "drive.py"
+ driver.write_text(
+ "import sys\n"
+ f"sys.path.insert(0, {str(REPO_ROOT / 'services' / 'actions')!r})\n"
+ f"sys.path.insert(0, {str(REPO_ROOT / 'scripts')!r})\n"
+ f"{script}"
+ "import check_action_contract as gate\n"
+ "raise SystemExit(gate.main([]))\n",
+ encoding="utf-8",
+ )
+ result = subprocess.run( # noqa: S603
+ [sys.executable, str(driver)],
+ capture_output=True,
+ text=True,
+ cwd=str(REPO_ROOT),
+ )
+ assert result.returncode == 1, f"the gate passed with {drift} drift injected:\n{result.stdout}"
+ assert expected in result.stderr, result.stderr
diff --git a/services/actions/tests/test_dry_run_credential_strip.py b/services/actions/tests/test_dry_run_credential_strip.py
index 57685c8fc..e97f91eb9 100644
--- a/services/actions/tests/test_dry_run_credential_strip.py
+++ b/services/actions/tests/test_dry_run_credential_strip.py
@@ -72,11 +72,15 @@ FACTORIES = {
}
#: vendor_id -> the declared key tuple every adapter for that vendor must use.
+#: Defender is here too: it has no factory (the arms build the client inline),
+#: but its adapters strip a key tuple like everyone else, and the ack /
+#: suppress arms added it as a third capability.
VENDOR_KEYS = {
"splunk": siem.SPLUNK_CLIENT_PARAM_KEYS,
"elastic": siem.ELASTIC_CLIENT_PARAM_KEYS,
"sentinel": siem.SENTINEL_CLIENT_PARAM_KEYS,
"qradar": siem.QRADAR_CLIENT_PARAM_KEYS,
+ "defender": siem.DEFENDER_CLIENT_PARAM_KEYS,
}
@@ -132,6 +136,11 @@ def test_every_registered_siem_adapter_strips_its_vendors_full_key_set() -> None
if expected is None:
continue
executor = registry.get_executor(descriptor.vendor_id, descriptor.capability)
+ # The strip list is how ``_LegacyExecutorAdapter`` implements dry_run.
+ # The native read-only executors honour it with an early return before
+ # they build a client, so they have no strip list to grade.
+ if not isinstance(executor, builtins._LegacyExecutorAdapter): # noqa: SLF001
+ continue
strip = getattr(executor, "_credential_keys", ())
detail = f"{descriptor.vendor_id}/{descriptor.capability}: strips {sorted(strip)} vs factory {sorted(expected)}"
assert set(strip) == set(expected), detail
@@ -177,3 +186,46 @@ async def test_dry_run_writeback_never_reaches_a_vendor(monkeypatch) -> None:
)
assert result.status is LiveActionStatus.SIMULATED
assert result.details.get("written") is False
+
+
[email protected]("capability", ["ack_alert", "suppress_alert"])
[email protected]("vendor", ["splunk", "elastic"])
+async def test_dry_run_alert_lifecycle_never_reaches_a_vendor(monkeypatch, vendor: str, capability: str) -> None:
+ """The same guarantee for the two verbs that just became reachable.
+
+ They pin ``alert_vendor`` per arm so a tenant with two SIEMs configured
+ does not have the target chosen by credential ordering. A pin is a routing
+ hint and never a licence to skip the credential check, so a dry run — which
+ works by removing the credentials — must still simulate rather than
+ resolving to the pinned vendor and calling it.
+ """
+ builtins.register_builtin_executors(overwrite=True)
+
+ def _tripwire(name, real):
+ def _wrapped(params):
+ client = real(params)
+ if client is not None:
+ raise AssertionError(f"a dry run built a live client via {name}")
+ return None
+
+ return _wrapped
+
+ for name in ("_splunk_client", "_elastic_client", "_sentinel_client", "_qradar_client"):
+ monkeypatch.setattr(siem, name, _tripwire(name, getattr(siem, name)))
+
+ executor = registry.get_executor(vendor, capability)
+ assert executor is not None, f"{vendor}/{capability} is not registered"
+ result = await executor.execute(
+ LiveActionRequest(
+ capability=capability,
+ vendor_id=vendor,
+ target="FINDING-1",
+ params=dict(FULL_CREDENTIALS),
+ dry_run=True,
+ tenant_id=uuid4(),
+ )
+ )
+
+ assert result.status is LiveActionStatus.SIMULATED
+ assert "Simulation mode" in str(result.details.get("note", ""))
diff --git a/services/actions/tests/test_live_action_builtins.py b/services/actions/tests/test_live_action_builtins.py
index 8ba5fbb3c..a2a106375 100644
--- a/services/actions/tests/test_live_action_builtins.py
+++ b/services/actions/tests/test_live_action_builtins.py
@@ -59,7 +59,7 @@ def test_register_builtin_executors_returns_full_count() -> None:
# reachable, Entra six with one. The writeback arms are the return leg
# of a two-way SIEM integration: AiSOC's verdict back onto the notable,
# signal, incident or offense that produced the alert.
- assert count == 57
+ assert count == 63
def test_builtin_executors_cover_canonical_vendor_capability_pairs() -> None:
@@ -92,13 +92,17 @@ def test_builtin_executors_cover_canonical_vendor_capability_pairs() -> None:
("crowdstrike", "quarantine_file"),
("crowdstrike", "run_script"),
("crowdstrike", "unisolate_host"),
+ ("defender", "ack_alert"),
("defender", "block_ioc"),
("defender", "update_alert_disposition"),
("defender", "get_host"),
("defender", "isolate_host"),
("defender", "run_av_scan"),
+ ("defender", "suppress_alert"),
("defender", "unisolate_host"),
+ ("elastic", "ack_alert"),
("elastic", "search_siem"),
+ ("elastic", "suppress_alert"),
("elastic", "update_alert_disposition"),
("elastic", "update_watcher"),
("fortigate", "allow_ip"),
@@ -127,7 +131,9 @@ def test_builtin_executors_cover_canonical_vendor_capability_pairs() -> None:
("sentinelone", "unisolate_host"),
("servicenow", "create_ticket"),
("slack", "notify"),
+ ("splunk", "ack_alert"),
("splunk", "create_notable_event"),
+ ("splunk", "suppress_alert"),
("splunk", "search_siem"),
("splunk", "sync_detection_rule"),
("splunk", "update_alert_disposition"),
@@ -161,7 +167,7 @@ def test_register_builtin_executors_is_idempotent_with_overwrite() -> None:
register_builtin_executors()
# Second call without overwrite would raise — confirm overwrite works.
count = register_builtin_executors(overwrite=True)
- assert count == 57
+ assert count == 63
def test_register_builtin_twice_without_overwrite_raises() -> None:
diff --git a/services/actions/tests/test_siem_client_signatures.py b/services/actions/tests/test_siem_client_signatures.py
index 7a07d8484..3c197fe46 100644
--- a/services/actions/tests/test_siem_client_signatures.py
+++ b/services/actions/tests/test_siem_client_signatures.py
@@ -21,14 +21,17 @@ from unittest.mock import create_autospec
from uuid import uuid4
import pytest
+from app.clients.defender_client import DefenderClient
from app.clients.elastic_client import ElasticClient
from app.clients.qradar_client import QRadarClient
from app.clients.sentinel_client import SentinelClient
from app.clients.splunk_client import SplunkClient
from app.executors import siem
from app.executors.siem import (
+ AckAlertExecutor,
CreateNotableEventExecutor,
SearchSIEMExecutor,
+ SuppressAlertExecutor,
UpdateAlertDispositionExecutor,
)
from app.models.action import ActionRequest, ActionStatus, ActionType
@@ -134,3 +137,77 @@ async def test_disposition_writeback_matches_every_client_signature(
assert result.status is ActionStatus.COMPLETED, result.error
assert result.output["written"] is True
getattr(client, method).assert_called_once()
+
+
+ ("executor_cls", "action_type", "vendor", "factory_attr", "client_cls", "method"),
+ [
+ (AckAlertExecutor, ActionType.ACK_ALERT, "splunk", "_splunk_client", SplunkClient, "acknowledge_notable_event"),
+ (AckAlertExecutor, ActionType.ACK_ALERT, "elastic", "_elastic_client", ElasticClient, "acknowledge_alert"),
+ (SuppressAlertExecutor, ActionType.SUPPRESS_ALERT, "splunk", "_splunk_client", SplunkClient, "suppress_notable_event"),
+ (SuppressAlertExecutor, ActionType.SUPPRESS_ALERT, "elastic", "_elastic_client", ElasticClient, "close_alert"),
+ ],
+)
+async def test_alert_lifecycle_arms_match_their_client_signatures(
+ monkeypatch,
+ executor_cls: type,
+ action_type: ActionType,
+ vendor: str,
+ factory_attr: str,
+ client_cls: type,
+ method: str,
+) -> None:
+ """The ack / suppress arms, autospec'd, now that they are reachable.
+
+ These executors were unreachable through governed dispatch for as long as
+ they have existed, so nothing exercised their client calls under a spec
+ that could reject a wrong keyword. Registering them without this would
+ move the ``TypeError`` from unreachable code into a live SOC.
+ """
+ client = _autospec(client_cls)
+ getattr(client, method).return_value = {"success": True}
+ for attr in ("_splunk_client", "_elastic_client", "_sentinel_client", "_qradar_client"):
+ monkeypatch.setattr(siem, attr, (lambda params: client) if attr == factory_attr else (lambda params: None))
+
+ result = await executor_cls().execute(
+ _request(
+ action_type,
+ target="FINDING-88",
+ alert_vendor=vendor,
+ owner="aisoc",
+ comment="Handled by AiSOC.",
+ )
+ )
+
+ assert result.status is ActionStatus.COMPLETED, result.error
+ getattr(client, method).assert_called_once()
+
+
+async def test_defender_alert_lifecycle_arms_match_the_client_signature(monkeypatch) -> None:
+ """Defender builds its client inline rather than through a factory, so it
+ is patched at the class rather than at a factory function."""
+ from app.clients import defender_client as defender_module
+
+ client = _autospec(DefenderClient)
+ client.acknowledge_alert.return_value = {"success": True}
+ client.suppress_alert.return_value = {"success": True}
+ monkeypatch.setattr(defender_module, "DefenderClient", lambda **kwargs: client)
+ for attr in ("_splunk_client", "_elastic_client", "_sentinel_client", "_qradar_client"):
+ monkeypatch.setattr(siem, attr, lambda params: None)
+
+ creds = {
+ "mde_tenant_id": "00000000-0000-0000-0000-000000000001",
+ "mde_client_id": "00000000-0000-0000-0000-000000000002",
+ "mde_client_secret": "unit-test-placeholder",
+ "alert_vendor": "defender",
+ }
+
+ ack = await AckAlertExecutor().execute(_request(ActionType.ACK_ALERT, target="MDE-1", **creds))
+ suppress = await SuppressAlertExecutor().execute(_request(ActionType.SUPPRESS_ALERT, target="MDE-1", **creds))
+
+ assert ack.status is ActionStatus.COMPLETED, ack.error
+ assert suppress.status is ActionStatus.COMPLETED, suppress.error
+ client.acknowledge_alert.assert_called_once()
+ client.suppress_alert.assert_called_once()
diff --git a/services/actions/tests/test_writeback_verification.py b/services/actions/tests/test_writeback_verification.py
new file mode 100644
index 000000000..a16ddb94f
--- /dev/null
+++ b/services/actions/tests/test_writeback_verification.py
@@ -0,0 +1,359 @@
+"""The disposition writeback is checked against the vendor, not against a 202.
+
+The writeback shipped declaring no verification probe, which was the honest
+choice at the time — a declared probe that does not run is the defect the
+contract gate exists to catch. But the standing rule is that an unverifiable
+action is not an autonomous one, and this one is automatic, so it needed a real
+read-back rather than a lower tier.
+
+What makes a probe real is that it can return False. An earlier isolation probe
+returned ``bool(device_id)`` — true of every host in the fleet, contained or
+not — and would have certified an uncontained host. Each vendor arm below is
+therefore driven through both the state it should reach and a state it should
+not, and the arms that cannot tell the two apart answer UNVERIFIED rather than
+inventing a confirmation.
+"""
+
+from __future__ import annotations
+
+import json
+from typing import Any
+
+import httpx
+import pytest
+import respx
+from app.clients.qradar_client import QRadarClient
+from app.clients.splunk_client import SplunkClient
+from app.models.action import ActionType
+from app.services import verification
+from app.services.verification import PostActionVerifier, VerificationOutcome
+
+SPLUNK_CREDS = {
+ "splunk_url": "https://splunk.example.invalid:8089",
+ "splunk_token": "unit-test-placeholder",
+}
+QRADAR_CREDS = {
+ "qradar_url": "https://qradar.example.invalid",
+ "qradar_token": "unit-test-placeholder",
+}
+ELASTIC_CREDS = {
+ "elastic_url": "https://es.example.invalid:9243",
+ "kibana_url": "https://kibana.example.invalid:5601",
+ "elastic_api_key": "unit-test-placeholder",
+}
+
+
+class _FakeSplunk:
+ """Returns whatever ``incident_review`` state the test wants read back."""
+
+ def __init__(self, state: dict[str, Any] | None) -> None:
+ self._state = state
+ self.calls: list[str] = []
+
+ async def get_notable_event_state(self, event_id: str) -> dict[str, Any] | None:
+ self.calls.append(event_id)
+ return self._state
+
+
+class _FakeQRadar:
+ def __init__(self, offense: dict[str, Any]) -> None:
+ self._offense = offense
+ self.calls: list[str] = []
+
+ async def get_offense(self, offense_id: str) -> dict[str, Any]:
+ self.calls.append(offense_id)
+ return self._offense
+
+
+def _use_splunk(monkeypatch: pytest.MonkeyPatch, state: dict[str, Any] | None) -> _FakeSplunk:
+ client = _FakeSplunk(state)
+ monkeypatch.setattr(verification, "_splunk_client", lambda params: client)
+ return client
+
+
+def _use_qradar(monkeypatch: pytest.MonkeyPatch, offense: dict[str, Any]) -> _FakeQRadar:
+ client = _FakeQRadar(offense)
+ monkeypatch.setattr(verification, "_qradar_client", lambda params: client)
+ return client
+
+
+async def _verify(action: ActionType, target: str, params: dict[str, Any]) -> VerificationOutcome:
+ result = await PostActionVerifier().verify(action, target, params)
+ return result.outcome
+
+
+# ── Splunk ────────────────────────────────────────────────────────────────
+
+
+async def test_splunk_close_is_verified_when_the_notable_is_actually_closed(monkeypatch) -> None:
+ client = _use_splunk(monkeypatch, {"status": "5", "owner": "aisoc"})
+
+ outcome = await _verify(
+ ActionType.UPDATE_ALERT_DISPOSITION,
+ "NOTABLE-7f3a",
+ {**SPLUNK_CREDS, "disposition": "false_positive"},
+ )
+
+ assert outcome is VerificationOutcome.VERIFIED
+ assert client.calls == ["NOTABLE-7f3a"], "the probe must actually re-query the vendor"
+
+
+async def test_splunk_close_fails_when_the_notable_is_still_open(monkeypatch) -> None:
+ """The property that makes this a probe: it can say the effect is absent.
+
+ A writeback the vendor rejected, or silently dropped, leaves the notable in
+ the analyst queue while AiSOC reports it handled. That disagreement is what
+ a responder needs told.
+ """
+ _use_splunk(monkeypatch, {"status": "1", "owner": "aisoc"})
+
+ outcome = await _verify(
+ ActionType.UPDATE_ALERT_DISPOSITION,
+ "NOTABLE-7f3a",
+ {**SPLUNK_CREDS, "disposition": "benign"},
+ )
+
+ assert outcome is VerificationOutcome.FAILED
+
+
+async def test_splunk_escalation_checks_the_owner_as_well_as_the_status(monkeypatch) -> None:
+ """Status alone is weak here: a notable an analyst already picked up reads
+ as in-progress whether or not AiSOC's write landed. The owner is what makes
+ the two distinguishable."""
+ _use_splunk(monkeypatch, {"status": "1", "owner": "aisoc"})
+ assert (
+ await _verify(
+ ActionType.UPDATE_ALERT_DISPOSITION,
+ "NOTABLE-1",
+ {**SPLUNK_CREDS, "disposition": "true_positive"},
+ )
+ is VerificationOutcome.VERIFIED
+ )
+
+ _use_splunk(monkeypatch, {"status": "1", "owner": "someone-else"})
+ assert (
+ await _verify(
+ ActionType.UPDATE_ALERT_DISPOSITION,
+ "NOTABLE-1",
+ {**SPLUNK_CREDS, "disposition": "true_positive"},
+ )
+ is VerificationOutcome.FAILED
+ )
+
+
+async def test_no_incident_review_entry_is_unverified_not_failed(monkeypatch) -> None:
+ """Splunk ES absent, or no review row yet, is "cannot tell" — not "did not
+ happen". Conflating them turns a deployment fact into a false alarm."""
+ _use_splunk(monkeypatch, None)
+
+ outcome = await _verify(
+ ActionType.UPDATE_ALERT_DISPOSITION,
+ "NOTABLE-1",
+ {**SPLUNK_CREDS, "disposition": "false_positive"},
+ )
+
+ assert outcome is VerificationOutcome.UNVERIFIED
+
+
+# ── QRadar ────────────────────────────────────────────────────────────────
+
+
+ ("status", "expected"),
+ [("CLOSED", VerificationOutcome.VERIFIED), ("OPEN", VerificationOutcome.FAILED)],
+)
+async def test_qradar_close_reads_the_offense_status_back(monkeypatch, status: str, expected) -> None:
+ _use_qradar(monkeypatch, {"id": 42, "status": status})
+ monkeypatch.setattr(verification, "_splunk_client", lambda params: None)
+
+ outcome = await _verify(
+ ActionType.UPDATE_ALERT_DISPOSITION,
+ "42",
+ {**QRADAR_CREDS, "disposition": "false_positive", "qradar_closing_reason_id": 3},
+ )
+
+ assert outcome is expected
+
+
+async def test_qradar_escalation_is_unverified_rather_than_trivially_true(monkeypatch) -> None:
+ """The ``bool(device_id)`` trap, avoided explicitly.
+
+ A QRadar escalation annotates the offense and leaves it OPEN — which is
+ also the state it was in beforehand. Reporting VERIFIED because the offense
+ is still OPEN would certify a write that never happened, so this answers
+ indeterminate instead.
+ """
+ _use_qradar(monkeypatch, {"id": 42, "status": "OPEN"})
+ monkeypatch.setattr(verification, "_splunk_client", lambda params: None)
+
+ outcome = await _verify(
+ ActionType.UPDATE_ALERT_DISPOSITION,
+ "42",
+ {**QRADAR_CREDS, "disposition": "true_positive"},
+ )
+
+ assert outcome is VerificationOutcome.UNVERIFIED
+
+
+# ── Arms with no read-back, and the refusal path ──────────────────────────
+
+
+async def test_vendors_without_a_read_back_report_unverified() -> None:
+ """Elastic, Sentinel and Defender expose no read of a finding's state.
+
+ Saying so is the point. The alternative — treating "no probe for this
+ vendor" as success — is what makes a verification field worthless.
+ """
+ outcome = await _verify(
+ ActionType.UPDATE_ALERT_DISPOSITION,
+ "SIGNAL-1",
+ {**ELASTIC_CREDS, "disposition": "false_positive"},
+ )
+
+ assert outcome is VerificationOutcome.UNVERIFIED
+
+
+async def test_a_refused_writeback_is_unverified_not_verified(monkeypatch) -> None:
+ """``needs_review`` writes nothing, so there is no effect to confirm.
+
+ VERIFIED here would read as "the verdict reached the finding", which is the
+ opposite of what happened.
+ """
+ client = _use_splunk(monkeypatch, {"status": "5", "owner": "aisoc"})
+
+ outcome = await _verify(
+ ActionType.UPDATE_ALERT_DISPOSITION,
+ "NOTABLE-1",
+ {**SPLUNK_CREDS, "disposition": "needs_review"},
+ )
+
+ assert outcome is VerificationOutcome.UNVERIFIED
+ assert client.calls == [], "a refusal wrote nothing, so the probe must not claim to confirm it"
+
+
+async def test_the_probe_derives_the_expected_state_from_the_disposition(monkeypatch) -> None:
+ """One notable, one state read back, two verdicts, opposite answers.
+
+ The expectation is re-derived through the same ``plan_writeback`` the
+ executor used rather than passed in beside the disposition: a probe told
+ separately what to expect can be told wrong.
+ """
+ _use_splunk(monkeypatch, {"status": "5", "owner": "aisoc"})
+
+ closed = await _verify(
+ ActionType.UPDATE_ALERT_DISPOSITION,
+ "NOTABLE-1",
+ {**SPLUNK_CREDS, "disposition": "false_positive"},
+ )
+ escalated = await _verify(
+ ActionType.UPDATE_ALERT_DISPOSITION,
+ "NOTABLE-1",
+ {**SPLUNK_CREDS, "disposition": "true_positive"},
+ )
+
+ assert closed is VerificationOutcome.VERIFIED
+ assert escalated is VerificationOutcome.FAILED
+
+
+# ── The two newly-reachable verbs get the same read-back ──────────────────
+
+
+ ("action", "status", "expected"),
+ [
+ (ActionType.ACK_ALERT, "1", VerificationOutcome.VERIFIED),
+ (ActionType.ACK_ALERT, "0", VerificationOutcome.FAILED),
+ (ActionType.SUPPRESS_ALERT, "5", VerificationOutcome.VERIFIED),
+ (ActionType.SUPPRESS_ALERT, "1", VerificationOutcome.FAILED),
+ ],
+)
+async def test_ack_and_suppress_are_verified_against_the_same_store(monkeypatch, action: ActionType, status: str, expected) -> None:
+ _use_splunk(monkeypatch, {"status": status, "owner": "aisoc"})
+
+ assert await _verify(action, "NOTABLE-1", dict(SPLUNK_CREDS)) is expected
+
+
+def test_every_probe_the_contract_claims_is_registered() -> None:
+ """Mirrors the gate, at unit-test speed: a declared probe that is not
+ registered is a claim rather than a fact, and this is the specific claim
+ the contract exists to make trustworthy."""
+ from app.live_actions.capability_contracts import CAPABILITY_CONTRACTS
+
+ registered = {a.value for a in verification._DEFAULT_PROBES} # noqa: SLF001
+ for capability in ("update_alert_disposition", "ack_alert", "suppress_alert"):
+ assert CAPABILITY_CONTRACTS[capability].has_verification_probe is True
+ assert capability in registered
+
+
+async def test_the_probe_clients_expose_the_methods_the_probe_calls() -> None:
+ """Autospec'd signature check for the read-back half.
+
+ Simulation never constructs a client, so a renamed read-back would raise
+ only on a live call — the same defect that shipped twice on the forward
+ path (``max_results`` vs ``max_count``, ``title`` vs ``event_data``).
+ """
+ from unittest.mock import create_autospec
+
+ splunk = create_autospec(SplunkClient, spec_set=True, instance=True)
+ qradar = create_autospec(QRadarClient, spec_set=True, instance=True)
+
+ # spec_set raises AttributeError for a method the real class lacks, and
+ # autospec rejects a keyword the real signature does not take.
+ splunk.get_notable_event_state.return_value = {"status": "5", "owner": "aisoc"}
+ qradar.get_offense.return_value = {"id": 42, "status": "CLOSED"}
+ assert await splunk.get_notable_event_state(event_id="NOTABLE-1") == {"status": "5", "owner": "aisoc"}
+ assert await qradar.get_offense(offense_id="42") == {"id": 42, "status": "CLOSED"}
+
+
+async def test_splunk_read_back_parses_the_incident_review_collection() -> None:
+ """The real client against the real request shape, not a fake.
+
+ ``/services/notable_update`` records every lifecycle change in the
+ ``incident_review`` KV store collection keyed by ``rule_id`` — the same
+ value it is sent as ``ruleUIDs`` — so this reads the effect of the write
+ rather than the acceptance of the request.
+ """
+ route = respx.get(
+ "https://splunk.example.invalid:8089/servicesNS/nobody/SplunkEnterpriseSecuritySuite/storage/collections/data/incident_review"
+ ).mock(return_value=httpx.Response(200, json=[{"rule_id": "NOTABLE-1", "status": "5", "owner": "aisoc"}]))
+
+ client = SplunkClient(host="https://splunk.example.invalid:8089", token="unit-test-placeholder")
+ state = await client.get_notable_event_state("NOTABLE-1")
+
+ assert state == {"status": "5", "owner": "aisoc"}
+ assert json.loads(route.calls.last.request.url.params["query"]) == {"rule_id": "NOTABLE-1"}
+
+
+async def test_splunk_read_back_is_none_without_enterprise_security() -> None:
+ """A deployment without ES has no incident review store. That is a fact
+ about the deployment, not an error, and certainly not a confirmation."""
+ respx.get(url__regex=r".*incident_review.*").mock(return_value=httpx.Response(404))
+
+ client = SplunkClient(host="https://splunk.example.invalid:8089", token="unit-test-placeholder")
+
+ assert await client.get_notable_event_state("NOTABLE-1") is None
+
+
+async def test_splunk_read_back_is_none_when_the_notable_has_no_review_row() -> None:
+ respx.get(url__regex=r".*incident_review.*").mock(return_value=httpx.Response(200, json=[]))
+
+ client = SplunkClient(host="https://splunk.example.invalid:8089", token="unit-test-placeholder")
+
+ assert await client.get_notable_event_state("NOTABLE-1") is None
diff --git a/services/connectors/app/connectors/base.py b/services/connectors/app/connectors/base.py
index 0d94aeed5..8c9abb6a0 100644
--- a/services/connectors/app/connectors/base.py
+++ b/services/connectors/app/connectors/base.py
@@ -140,10 +140,21 @@ class Capability(str, Enum):
# this verb carries AiSOC's verdict back out to the same finding, so a
# notable AiSOC dismissed is not re-triaged by a human in Splunk.
UPDATE_ALERT_DISPOSITION = "update_alert_disposition"
+ # Alert lifecycle. Both have had Splunk, Elastic and Defender arms since
+ # Phase 3.3 and appeared in neither vocabulary, so the only way to reach
+ # working code was the ungoverned ActionType REST route.
+ ACK_ALERT = "ack_alert"
+ SUPPRESS_ALERT = "suppress_alert"
# TICKET — bidirectional ITSM (Jira / ServiceNow / etc.).
PUSH_CASE = "push_case"
PUSH_STATUS = "push_status"
+ # Registered against Jira, ServiceNow, PagerDuty and Slack for as long as
+ # those adapters have existed, and absent here — so the registry logged
+ # `capability_unknown` for four executors at every startup. Distinct from
+ # push_case, which syncs an existing case rather than opening a record.
+ CREATE_TICKET = "create_ticket"
+ NOTIFY = "notify"
# AUDIT — read-only configuration / posture queries.
READ_AUDIT_TRAIL = "read_audit_trail"