Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
fix(fusion): baseline subjects must be fields a connector actually emits
Beenu Arora8622b921d426
diff --git a/DEPTH_PROGRESS.md b/DEPTH_PROGRESS.mdindex 22d438e75..da40ea80d 100644--- a/DEPTH_PROGRESS.md+++ b/DEPTH_PROGRESS.md@@ -471,7 +471,12 @@ phase's own new deviations take D12 and D13.locations; peer-group deviation; volume per principal and per OAuth app —each a typed `AnomalySignal`, never an alert on its own. Wired into`DetectionEngine.evaluate_async` beside the 3.3 first-seen store, so a- CORE install computes them with no `full` service running. 21 tests. The+ CORE install computes them with no `full` service running. 22 tests,+ one of which caught this work's own worst defect: the first draft+ resolved client-family rarity from three field names **no connector+ emits**, so that signal could never have been computed anywhere. Every+ subject chain is now held against the gate's own emitted-field+ namespace, proved by reinstating the broken chain. Thereachability gate's "behavioural baseline" family was **narrowed**: itmatched `^active_`, which swept in a systemd unit state and an ITSMlookup and reported both as blocked on a baseline that would never havediff --git a/docs/decisions/0011-behavioural-baselines-in-core.md b/docs/decisions/0011-behavioural-baselines-in-core.mdindex 101b966b6..74f5e2228 100644--- a/docs/decisions/0011-behavioural-baselines-in-core.md+++ b/docs/decisions/0011-behavioural-baselines-in-core.md@@ -138,6 +138,15 @@ members while the total still advances, so the baseline degrades into"stopped learning new values for this principal" rather than into a wrongdenominator or an outage.+**Every subject chain is held against the emitted-field namespace.** A+baseline over a field nothing emits is a signal that cannot be computed, and+the field name reads as working, so it is worse than a missing feature. The+first draft of this module got that wrong: `baseline_client_family_rarity`+resolved its subject from `client_family`, `user_agent_family` or+`client_app`, and **no connector emits any of the three**. A test now asserts+each chain intersects the namespace `scripts/check_detection_fields.py`+builds, and was proved by reinstating the broken chain and watching it fail.+**Peer groups are only as good as the group key.** The plan names the Phase2.5 posture snapshot as the source. Until that lands, the group is taken fromwhatever the identity provider already puts on the event — `department`,diff --git a/services/fusion/app/main.py b/services/fusion/app/main.pyindex 8f1415315..32fb4ee59 100644--- a/services/fusion/app/main.py+++ b/services/fusion/app/main.py@@ -15,13 +15,13 @@ from app.services import ioc_matchfrom app.services.alert_enricher import AlertEnricherfrom app.services.alert_sink import AlertSinkfrom app.services.attack_chain_grouper import AttackChainGrouper+from app.services.baselines import BaselineStorefrom app.services.confidence import ConfidenceScorerfrom app.services.correlator import Correlatorfrom app.services.deduplicator import Deduplicatorfrom app.services.detection_engine import DetectionEnginefrom app.services.dlq_sink import PostgresDLQfrom app.services.entity_risk import EntityRiskEngine-from app.services.baselines import BaselineStorefrom app.services.first_seen import FirstSeenStorefrom app.services.fusion_engine import FusionEnginefrom app.services.ioc_match import TenantIocMatcherdiff --git a/services/fusion/app/services/baselines.py b/services/fusion/app/services/baselines.pyindex a3a14fd50..123554b93 100644--- a/services/fusion/app/services/baselines.py+++ b/services/fusion/app/services/baselines.py@@ -98,11 +98,21 @@ _TOTAL_FIELD = "__total__"#:#: Rarity is always *of a value, for a principal*, so each entry needs both.#: The principal is resolved once per event by :func:`resolve_principal`.+#:+#: **Every chain must contain at least one name a connector actually emits**,+#: and a test asserts it against the same namespace+#: `scripts/check_detection_fields.py` builds. The first draft of this table+#: failed that for `baseline_client_family_rarity`: it listed+#: `client_family`, `user_agent_family` and `client_app`, and **no connector+#: emits any of the three**, so that rarity could never have been computed on+#: any deployment. It would have been a published signal that was structurally+#: incapable of firing — the exact shape this repository keeps finding, and+#: worse here than a missing feature because the field name reads as working.RARITY_FIELDS: dict[str, tuple[str, ...]] = {- "baseline_action_rarity": ("event_name", "action", "operation", "event_type"),- "baseline_resource_rarity": ("resource", "resource_name", "object_name", "target"),- "baseline_client_family_rarity": ("client_family", "user_agent_family", "client_app"),- "baseline_location_rarity": ("src_country", "country", "geo_country"),+ "baseline_action_rarity": ("event_name", "eventName", "action", "event_type"),+ "baseline_resource_rarity": ("resource", "Resource", "resource_name", "target"),+ "baseline_client_family_rarity": ("user_agent", "userAgent", "clientAppUsed", "client_family"),+ "baseline_location_rarity": ("country", "ClientCountry", "countryOrRegion", "src_country"),}#: Event fields naming the principal whose baseline is being read, in trust@@ -116,7 +126,7 @@ PRINCIPAL_FIELDS: tuple[str, ...] = ("actor","principal","user_arn",- "actor_uid",+ "userPrincipalName",)#: Event fields naming the OAuth application acting, for the per-application@@ -124,11 +134,11 @@ PRINCIPAL_FIELDS: tuple[str, ...] = (#: OAuth grant acts *as* a user, so the user's own volume looks normal while#: the application's does not.OAUTH_APP_FIELDS: tuple[str, ...] = (- "oauth_app",- "oauth_client_id","application",- "app_id",+ "applicationName",+ "app_name","client_id",+ "oauth_app",)#: Event fields naming the peer group a principal belongs to. Depth plan 3.4@@ -137,12 +147,11 @@ OAUTH_APP_FIELDS: tuple[str, ...] = (#: event, which is what Workspace, Entra and Okta all emit. Resolving to#: nothing yields no peer signal rather than a group of one.PEER_GROUP_FIELDS: tuple[str, ...] = (- "department",- "user_department","group",- "user_group",- "job_title",+ "groups","role",+ "ROLE_NAME",+ "department",)#: Every derived field this module can produce. Asserted against the gate'sdiff --git a/services/fusion/tests/test_baselines.py b/services/fusion/tests/test_baselines.pyindex 1d90176df..825062c1a 100644--- a/services/fusion/tests/test_baselines.py+++ b/services/fusion/tests/test_baselines.py@@ -30,6 +30,9 @@ from app.services.baselines import (DERIVED_FIELDS,MIN_PEER_MEMBERS,MIN_VOLUME_SAMPLES,+ OAUTH_APP_FIELDS,+ PEER_GROUP_FIELDS,+ PRINCIPAL_FIELDS,RARITY_FIELDS,BaselineStore,resolve_principal,@@ -299,6 +302,37 @@ def test_every_rarity_field_is_declared_as_derived() -> None:assert set(RARITY_FIELDS) <= DERIVED_FIELDS+def test_every_subject_chain_names_a_field_some_connector_emits() -> None:+ """A baseline over a field nothing emits is a signal that cannot fire.++ This is the test that earns its place. The first draft of+ `RARITY_FIELDS` gave `baseline_client_family_rarity` the chain+ `client_family` / `user_agent_family` / `client_app`, and **no connector+ emits any of the three** — so that rarity would have been published,+ counted, and structurally incapable of being computed on any+ deployment. Worse than a missing feature, because the field name reads+ as working.++ The namespace comes from the same static analysis+ `scripts/check_detection_fields.py` performs, so the two cannot+ disagree about what a connector emits.+ """+ spec = importlib.util.spec_from_file_location("cdf_emit", _REPO / "scripts" / "check_detection_fields.py")+ assert spec and spec.loader+ gate = importlib.util.module_from_spec(spec)+ spec.loader.exec_module(gate)+ emitted = gate._connector_namespace()++ chains: dict[str, tuple[str, ...]] = {+ **RARITY_FIELDS,+ "__principal__": PRINCIPAL_FIELDS,+ "__oauth_app__": OAUTH_APP_FIELDS,+ "__peer_group__": PEER_GROUP_FIELDS,+ }+ unreachable = {name: chain for name, chain in chains.items() if not (set(chain) & emitted)}+ assert not unreachable, f"no connector emits any candidate for: {sorted(unreachable)}"++def test_the_narrowed_family_no_longer_claims_unrelated_fields() -> None:"""`^active_` swept in a systemd unit state and an ITSM lookup.