Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

Resolve identity on the generic normalizer path, and make connector names resolve both ways (#808)

Beenu Arora8cafcd97e184
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index cb579fc29..c5adf2ff6 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -280,6 +280,20 @@ jobs:
# commit the result.
- name: Verify connector count is in sync (Phase 1.1)
run: python scripts/generate_connector_count.py --check
+ # The count says how many connectors exist; this says whether the name
+ # each one is registered under still reaches a normalization profile.
+ # The connectors service declares `crowdstrike` and `okta` while the
+ # ingest profiles were keyed `crowdstrike_falcon` and `okta_system_log`,
+ # so events arriving under the advertised names — including the README's
+ # own push example — missed the lookup and fell to the generic fallback.
+ #
+ # The self-test runs first and on the same tree: it injects drift in each
+ # direction and requires the gate to catch each one, so a refactor that
+ # quietly stops checking a direction fails here rather than printing OK.
+ - name: Connector profile drift gate — self-test
+ run: python3 scripts/check_connector_profiles.py --self-test
+ - name: Connector profile drift gate
+ run: python3 scripts/check_connector_profiles.py
# The count claim above says how many connectors exist; this says
# whether a reader can find out how to use them. 35 of 84 had no page
# at all and the sidebar listed 34 of 96, so most connector
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 9536dac52..e43933a4e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -77,6 +77,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
+- **`scripts/check_connector_profiles.py` — a connector-type drift gate that
+ reads in both directions.** Nothing compared the profile keys in
+ `services/ingest/internal/normalizer/normalizer.go` against the identifiers
+ the connectors service declares, and they had drifted apart in both
+ directions at once. The gate enumerates four name spaces — the Go profiles
+ and type aliases, the 84 declared `connector_id`s, the `ConnectorType`
+ union, and every `connector_type` appearing in a documented example — and
+ fails when a name in any of them resolves to nothing.
+
+ It also checks that a documented example reaches a *profile*, not merely a
+ declared connector: a reader pastes a flat payload, which never carries the
+ canonical envelope, so only a profile or alias can give the event a vendor
+ identity. That is the specific check that catches the README defect.
+
+ `--self-test` injects twelve defects, one per direction and per failure
+ code, and requires the gate to catch each; it runs in CI on the same tree
+ immediately before the gate itself. The resolved repo root, every file read
+ and every count are printed before the verdict, and an input that is missing
+ or parses empty is a hard error rather than a quiet pass — a gate that
+ reports OK about a tree it never opened is worse than no gate.
+
- **Competitor product names removed from the docs portal, the benchmark page
and the archived plan subtree, and a CI gate added to keep them out.** AiSOC
names no competitor product, but two published comparison tables and the
@@ -292,6 +313,61 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
+- **An event ingested under a connector type with no profile became an alert
+ with no host, no user and no source IP.** `_canonicalAliases` in
+ `services/ingest/internal/normalizer/normalizer.go` already resolved
+ `actor.user.name`, `device.name` and `src_endpoint.ip` from the spellings
+ connectors actually use, but the pass that applied it was gated behind the
+ canonical-envelope branch. Anything reaching the generic fallback — every
+ push through `/v1/ingest/batch`, which sends a flat payload and so never
+ matches that branch — resolved `title` and `external_id` and nothing else.
+
+ Identity is what the rest of the platform is built on: entity extraction,
+ the Investigation Rail's pivots, the `{tenant}:{entity}:{tactic}`
+ correlation key, the entity graph and UEBA all key off those three fields.
+ An alert still appeared, which is what made it look like it had worked, but
+ it carried no entity chips and correlated into the `unknown` bucket. The
+ alias pass now runs for every profile and fills only destinations the field
+ map left empty, so a vendor profile's own mapping still wins.
+
+ Three defects in the same family, found while fixing it:
+
+ - **A nested vendor object was written whole into a scalar identity slot.**
+ Fourteen of the 84 registered connectors emit `actor`, and four emit it as
+ the vendor's object rather than a name, so `actor.user.name` became a map
+ — an entity that renders as a map and correlates as garbage. Identity
+ resolution now takes strings only and digs one level (`actor.name`,
+ `actor.displayName`, `user.name`) for the scalar underneath.
+ - **A vendor profile's severity ladder is spelled the way its vendor spells
+ it.** `crowdstrike_falcon`'s is capitalised, and a pushed payload is
+ whatever the caller wrote, so a lowercase `high` scored 0 and rendered as
+ Unknown. The shared five-tier ladder is consulted when the profile's own
+ has no entry for the value; `critical` stays its own tier.
+ - **A vendor profile left `message` empty for a pushed payload**, because it
+ maps its vendor's field name, so the promoter generated
+ "Security Finding from <product>" over the title the caller actually sent.
+
+- **`connector_type` values the product advertises did not reach their
+ profile.** The connectors service declares `crowdstrike` and `okta`; the
+ ingest profiles were keyed `crowdstrike_falcon` and `okta_system_log`. The
+ README's own push example used `crowdstrike`, so the example a new user
+ copies missed the lookup, fell to the generic fallback and lost its vendor
+ attribution — 80 of the 84 declared connectors had no profile entry under
+ the name they are registered with.
+
+ A `connectorTypeAliases` map resolves both spellings rather than renaming
+ either. The longer names are load-bearing elsewhere — `packages/types`'
+ `ConnectorType` union, the CLI's default, the graph extractor, the actions
+ credential resolver — so renaming one side would have broken the other.
+
+- **Two registered connectors reached no normalization path at all.**
+ `auditd` emitted `source` without `raw_event`, so it failed the canonical
+ envelope check and lost the host it carries; it now emits both.
+ `email_inbox` returns the message envelope shaped for
+ `email-forwarded.yaml`, so it matched neither branch and strict mode
+ rejected it outright; it now has a profile mirroring that template, the way
+ `ai_runtime` mirrors `ai-runtime.yaml`.
+
- **The agent recommended evidence acquisition the platform could not
perform.** `capture_forensics` was an `ActionType` with no executor
anywhere, and `services/agents/app/agents/investigation_agent.py` proposes
diff --git a/README.md b/README.md
index 7aaf9a460..cced87168 100644
--- a/README.md
+++ b/README.md
@@ -94,8 +94,9 @@ Sentinel, Elastic, CrowdStrike, Okta, AWS (GuardDuty / CloudTrail / Security
Hub), Wiz, and Kubernetes audit logs. The full list is in the
[connector docs](https://beenuar.github.io/AiSOC/docs/connectors/api-coverage).
-A connector without a vendor profile still ingests through a generic mapping —
-events flow, but entity extraction is weaker.
+A connector without a vendor profile still ingests through a generic mapping,
+which resolves host, user and source IP from the usual spellings so the alert
+is still pivotable. A vendor profile adds that vendor's own field names on top.
## How it works
diff --git a/docs/architecture/README.md b/docs/architecture/README.md
index 4ffb7fbfc..6c2fa756b 100644
--- a/docs/architecture/README.md
+++ b/docs/architecture/README.md
@@ -25,8 +25,18 @@ curl -X POST http://localhost:8081/v1/ingest/batch \
**2. Ingest normalizes it.** [`services/ingest`](../../services/ingest) maps
the vendor payload onto a common OCSF-shaped envelope using a per-connector
-profile. A connector with no profile falls through to a vendor-neutral
-generic profile — the event still flows, but entity extraction is weaker.
+profile. A connector with no profile falls through to a vendor-neutral generic
+profile, which resolves `actor.user.name`, `device.name` and `src_endpoint.ip`
+from the spellings connectors actually use (`user` / `username` / `actor`,
+`host` / `hostname`, `src_ip` / `source_ip` / `client_ip`) in a fixed order.
+The event keeps the entities the rest of the platform pivots on; what a vendor
+profile adds is that vendor's own field names and its OCSF class.
+
+`connector_type` is matched against the profile keys, and the identifiers the
+connectors service declares are the ones to use — `crowdstrike`, not
+`crowdstrike_falcon`. Both resolve, and
+[`scripts/check_connector_profiles.py`](../../scripts/check_connector_profiles.py)
+fails CI if any name in either direction stops resolving.
**3. It enters the event spine.** Ingest publishes to the Kafka topic
`aisoc.raw_events`. This is the boundary that makes everything downstream
diff --git a/scripts/check_connector_profiles.py b/scripts/check_connector_profiles.py
new file mode 100644
index 000000000..11bc82e1d
--- /dev/null
+++ b/scripts/check_connector_profiles.py
@@ -0,0 +1,590 @@
+#!/usr/bin/env python3
+"""Connector-type drift gate: names must resolve, in both directions.
+
+Why this exists
+---------------
+``connectorProfiles`` in ``services/ingest/internal/normalizer/normalizer.go``
+decides how a connector's events become OCSF, and it is keyed by
+``connector_type``. The connectors service declares its own identifiers in
+``services/connectors/app/connectors/``. Nothing compared the two, and they
+drifted: the service declares ``crowdstrike`` and ``okta`` while the profiles
+were keyed ``crowdstrike_falcon`` and ``okta_system_log``, so events arriving
+under the names the product advertises missed the lookup entirely. The README's
+own push example was one of them.
+
+A missing profile is not cosmetic. In strict mode the normalizer rejects an
+unknown connector type outright. In lenient mode it falls to the generic
+profile, and before the identity aliases were wired into that path the
+resulting alert carried no host, no user and no source IP — so entity
+extraction, the Investigation Rail's pivots, the ``{tenant}:{entity}:{tactic}``
+correlation key, the entity graph and UEBA all had nothing to work with. The
+alert still appeared, which is what made it look like it had worked.
+
+What it checks, in both directions
+----------------------------------
+The repository's dominant bug shape is a one-directional check: it compares A
+against B, never B against A, so drift in the direction things actually change
+slips through while the check prints OK. Every rule below is therefore stated
+as a resolution requirement on a specific name, and both name spaces are
+enumerated.
+
+ GO -> PY every ``connectorProfiles`` key and every ``connectorTypeAliases``
+ entry must name something that exists.
+ PY -> GO every declared ``connector_id`` must reach a normalization path
+ that survives strict mode: its own profile, an alias to one, or
+ the canonical ``raw_event`` + ``source`` envelope.
+ DOCS -> * every ``connector_type`` in a documented example must resolve.
+ MECHANISM the identity aliases the generic path depends on must still be
+ declared, and the Go tests that prove the behaviour must exist.
+ Coverage without the behaviour behind it is the vacuous case.
+
+Usage
+-----
+ python3 scripts/check_connector_profiles.py # gate
+ python3 scripts/check_connector_profiles.py --json
+ python3 scripts/check_connector_profiles.py --self-test # prove it bites
+
+``--repo-root`` overrides the tree under inspection. The resolved root, every
+file read and every count are printed before the verdict, and a missing or
+empty input is a hard error rather than a quiet zero: a gate that resolves its
+own location and reports OK about a tree it never opened is worse than no gate.
+"""
+
+from __future__ import annotations
+
+import argparse
+import ast
+import json
+import re
+import sys
+from pathlib import Path
+
+NORMALIZER_REL = Path("services/ingest/internal/normalizer/normalizer.go")
+NORMALIZER_TEST_REL = Path("services/ingest/internal/normalizer/normalizer_test.go")
+CONNECTORS_REL = Path("services/connectors/app/connectors")
+TEMPLATES_REL = Path("services/ingest/internal/normalizer/templates")
+
+# Profile keys that are push-path template names rather than connector ids.
+# Each must name a webhook template that exists, so this cannot become a
+# dumping ground for keys nobody can account for.
+PUSH_ONLY_PROFILES: dict[str, str] = {
+ "ai_runtime": "ai-runtime.yaml",
+ "ai_guardrail": "ai-finding.yaml",
+}
+
+# Legacy profile keys that the wider platform uses as a connector_type even
+# though the connectors service spells the id differently. They stay because
+# packages/types' ConnectorType union, the CLI default, the graph extractor and
+# the actions credential resolver all key off them; the connectorTypeAliases
+# map is what makes the declared id resolve to the same profile. Each entry
+# must be reachable via an alias or still declared in the union, which is
+# checked against those files rather than asserted here.
+LEGACY_PROFILE_KEYS = {"crowdstrike_falcon", "okta_system_log", "splunk_enterprise"}
+
+# packages/types' ConnectorType union is a third name space, and it predates
+# the connectors registry. These members name no connector the service
+# declares and no profile: they are the console's older vocabulary
+# (``google_chronicle`` for ``chronicle``, ``ibm_qradar`` for ``qradar``) plus
+# four generic transports that were never connectors at all. They are recorded
+# rather than fixed here because the union is shared with the console.
+#
+# This is a ratchet: it may shrink, never grow. A new unresolvable union member
+# fails the gate.
+KNOWN_UNION_ONLY_TYPES = {
+ "custom_webhook",
+ "google_chronicle",
+ "http_pull",
+ "ibm_qradar",
+ "kafka",
+ "palo_alto_cortex",
+ "slack",
+ "syslog",
+ "teams",
+ "vectra_ai",
+}
+
+TS_TYPES_REL = Path("packages/types/src/connector.ts")
+
+# Identity destinations the generic path depends on. Losing any one of these
+# is the defect this gate exists for.
+REQUIRED_IDENTITY_DESTINATIONS = ("actor.user.name", "device.name", "src_endpoint.ip")
+
+# Behavioural proof. Coverage arithmetic alone would still pass if someone
+# deleted the alias pass, so the gate requires the tests that exercise it.
+REQUIRED_GO_TESTS = (
+ "TestGenericProfileResolvesIdentityFields",
+ "TestDeclaredConnectorIdResolvesToVendorProfile",
+ "TestIdentityAliasResolutionIsDeterministic",
+)
+
+_GO_BLOCK = "var {name} = map[string]{vtype}{{"
+
+
+class GateError(RuntimeError):
+ """An input could not be read. Never downgraded to a passing result."""
+
+
+# --------------------------------------------------------------------------
+# Go side
+# --------------------------------------------------------------------------
+def _go_block(src: str, header: str, open_after: str | None = None) -> str:
+ """The brace-balanced literal following `header`.
+
+ `open_after` exists for a slice of anonymous structs, where the first brace
+ after the header opens the type definition rather than the literal: the
+ value starts at the `}{` that closes the struct and opens the composite.
+ """
+ start = src.find(header)
+ if start == -1:
+ raise GateError(f"could not find `{header}` in the normalizer")
+ if open_after is not None:
+ marker = src.find(open_after, start)
+ if marker == -1:
+ raise GateError(f"could not find `{open_after}` after `{header}`")
+ start = marker
+ depth, i = 0, src.index("{", start)
+ for j in range(i, len(src)):
+ if src[j] == "{":
+ depth += 1
+ elif src[j] == "}":
+ depth -= 1
+ if depth == 0:
+ return src[i : j + 1]
+ raise GateError(f"unbalanced braces after `{header}`")
+
+
+def parse_profile_keys(src: str) -> set[str]:
+ block = _go_block(src, "var connectorProfiles = map[string]connectorProfile{")
+ # Top-level keys only: nested fieldMap entries are indented deeper.
+ return set(re.findall(r'^\t"([^"]+)":\s*\{', block, re.MULTILINE))
+
+
+def parse_type_aliases(src: str) -> dict[str, str]:
+ # Absent is a valid state to report on, not a parse error: a tree with no
+ # alias map is exactly the tree whose declared ids reach no profile, and
+ # the gate should say so rather than abort before it gets there.
+ if "var connectorTypeAliases = map[string]string{" not in src:
+ return {}
+ block = _go_block(src, "var connectorTypeAliases = map[string]string{")
+ return dict(re.findall(r'"([^"]+)":\s*"([^"]+)"', block))
+
+
+def parse_identity_destinations(src: str) -> dict[str, list[str]]:
+ block = _go_block(src, "var _canonicalAliases = []struct {", open_after="}{")
+ out: dict[str, list[str]] = {}
+ for dst, sources in re.findall(r'\{"([^"]+)",\s*\[\]string\{([^}]*)\}\}', block):
+ out[dst] = re.findall(r'"([^"]+)"', sources)
+ return out
+
+
+# --------------------------------------------------------------------------
+# Python side
+# --------------------------------------------------------------------------
+def _class_connector_id(tree: ast.Module) -> str | None:
+ for cls in (n for n in ast.walk(tree) if isinstance(n, ast.ClassDef)):
+ for stmt in cls.body:
+ if (
+ isinstance(stmt, ast.Assign)
+ and any(getattr(t, "id", None) == "connector_id" for t in stmt.targets)
+ and isinstance(stmt.value, ast.Constant)
+ and isinstance(stmt.value.value, str)
+ and stmt.value.value
+ ):
+ return stmt.value.value
+ if (
+ isinstance(stmt, ast.AnnAssign)
+ and getattr(stmt.target, "id", None) == "connector_id"
+ and isinstance(stmt.value, ast.Constant)
+ and isinstance(stmt.value.value, str)
+ and stmt.value.value
+ ):
+ return stmt.value.value
+ return None
+
+
+def _emits_canonical_envelope(tree: ast.Module) -> bool:
+ """Whether the connector's normalized event carries `raw_event` + `source`.
+
+ Every string key contributed by a dict literal, a subscript assignment or
+ an envelope helper inside the normalize path counts. Restricting this to
+ keys of a directly-returned dict literal reported 13 connectors as
+ non-canonical when a runtime probe of all 84 found 2 — eleven build the
+ envelope through a helper or a spread, and a gate that fails eleven
+ correct connectors gets switched off.
+ """
+
+ def in_normalize_path(name: str) -> bool:
+ return name == "normalize" or name.startswith(("_envelope", "_normalize", "_to_", "_build"))
+
+ keys: set[str] = set()
+ functions = (ast.FunctionDef, ast.AsyncFunctionDef)
+ for fn in (n for n in ast.walk(tree) if isinstance(n, functions) and in_normalize_path(n.name)):
+ for node in ast.walk(fn):
+ if isinstance(node, ast.Dict):
+ keys.update(k.value for k in node.keys if isinstance(k, ast.Constant) and isinstance(k.value, str))
+ elif isinstance(node, ast.Subscript) and isinstance(node.slice, ast.Constant) and isinstance(node.slice.value, str):
+ keys.add(node.slice.value)
+ return {"raw_event", "source"} <= keys
+
+
+def parse_connectors(directory: Path) -> dict[str, bool]:
+ """connector_id -> emits a canonical envelope."""
+ out: dict[str, bool] = {}
+ for path in sorted(directory.glob("*.py")):
+ if path.name in {"__init__.py", "base.py"}:
+ continue
+ tree = ast.parse(path.read_text(encoding="utf-8"))
+ connector_id = _class_connector_id(tree)
+ if connector_id:
+ out[connector_id] = _emits_canonical_envelope(tree)
+ return out
+
+
+# --------------------------------------------------------------------------
+# Docs side
+# --------------------------------------------------------------------------
+DOC_GLOBS = ("README.md", "docs/**/*.md", "apps/docs/docs/**/*.md", "apps/docs/docs/**/*.mdx")
+DOC_PATTERN = re.compile(r"""["']?connector_type["']?\s*[:=]\s*["']([a-z0-9_]+)["']""")
+
+
+def parse_doc_examples(root: Path) -> dict[str, list[str]]:
+ """connector_type -> the documents that show it."""
+ found: dict[str, list[str]] = {}
+ for pattern in DOC_GLOBS:
+ for path in sorted(root.glob(pattern)):
+ if "node_modules" in path.parts or "plans" in path.parts:
+ continue
+ for name in DOC_PATTERN.findall(path.read_text(encoding="utf-8", errors="replace")):
+ found.setdefault(name, []).append(str(path.relative_to(root)))
+ return found
+
+
+# --------------------------------------------------------------------------
+# Gate
+# --------------------------------------------------------------------------
+def parse_union_types(path: Path) -> set[str]:
+ """Members of packages/types' ConnectorType union."""
+ src = path.read_text(encoding="utf-8")
+ match = re.search(r"export type ConnectorType\s*=(.*?);", src, re.DOTALL)
+ if not match:
+ raise GateError(f"could not find the ConnectorType union in {path}")
+ return set(re.findall(r'"([^"]+)"', match.group(1)))
+
+
+def evaluate(
+ profile_keys: set[str],
+ type_aliases: dict[str, str],
+ identity: dict[str, list[str]],
+ connectors: dict[str, bool],
+ doc_examples: dict[str, list[str]],
+ go_tests: set[str],
+ template_names: set[str],
+ union_types: set[str],
+) -> list[tuple[str, str]]:
+ failures: list[tuple[str, str]] = []
+ declared = set(connectors)
+ # A name is resolvable if it is a declared connector, a profile key, or an
+ # alias that leads to one.
+ resolvable = declared | profile_keys | set(type_aliases)
+
+ # GO -> PY. Every profile key must name something real.
+ for key in sorted(profile_keys):
+ if key in declared:
+ continue
+ if key in PUSH_ONLY_PROFILES:
+ template = PUSH_ONLY_PROFILES[key]
+ if template not in template_names:
+ failures.append(
+ ("profile-template-missing", f"profile {key!r} is allowed as a push-only template key, but {template} does not exist")
+ )
+ continue
+ if key in LEGACY_PROFILE_KEYS:
+ if key not in set(type_aliases.values()) and key not in union_types:
+ failures.append(
+ (
+ "legacy-profile-unreachable",
+ f"profile {key!r} is a legacy name: no connector declares it, no alias points at it, "
+ "and the ConnectorType union no longer carries it — nothing can reach this profile",
+ )
+ )
+ continue
+ failures.append(("profile-names-nothing", f"profile {key!r} names a connector type that does not exist"))
+
+ # GO -> PY. Aliases must start somewhere real and land somewhere real.
+ for source, target in sorted(type_aliases.items()):
+ if source not in declared:
+ failures.append(("alias-source-undeclared", f"alias {source!r} -> {target!r}: no connector declares {source!r}"))
+ if target not in profile_keys:
+ failures.append(("alias-target-missing", f"alias {source!r} -> {target!r}: {target!r} is not a profile key"))
+ if source in profile_keys:
+ failures.append(("alias-shadowed", f"alias {source!r} is shadowed by its own profile entry and never fires"))
+
+ # PY -> GO. Every declared connector must survive strict mode.
+ for connector_id, canonical in sorted(connectors.items()):
+ if connector_id in profile_keys or connector_id in type_aliases or canonical:
+ continue
+ failures.append(
+ (
+ "strict-mode-unreachable",
+ f"connector {connector_id!r} has no profile, no alias and emits no canonical envelope: "
+ "strict mode rejects it and lenient mode resolves only a title",
+ )
+ )
+
+ # TS -> *. The console's union is the third name space, and drift here is
+ # how `splunk_enterprise` came to exist on one side only. The ratchet may
+ # shrink; a new unresolvable member fails.
+ for name in sorted(union_types - resolvable - KNOWN_UNION_ONLY_TYPES):
+ failures.append(("union-type-unresolvable", f"ConnectorType union member {name!r} names no connector, profile or alias"))
+ for name in sorted(KNOWN_UNION_ONLY_TYPES & resolvable):
+ failures.append(
+ (
+ "union-ratchet-stale",
+ f"{name!r} now resolves; remove it from KNOWN_UNION_ONLY_TYPES so the ratchet keeps shrinking",
+ )
+ )
+
+ # DOCS. A copied example that resolves to nothing is the defect that taught
+ # us to write this gate.
+ for name, where in sorted(doc_examples.items()):
+ locations = ", ".join(sorted(set(where)))
+ if name not in resolvable:
+ failures.append(("doc-example-unresolvable", f"documented connector_type {name!r} resolves to nothing (in {locations})"))
+ continue
+ # Being a declared connector id is not enough for a documented example.
+ # A reader pastes a flat payload, which carries no `raw_event`, so the
+ # canonical branch never runs and only a profile or an alias can give
+ # the event a vendor identity. This is the exact shape of the original
+ # defect: the README said `crowdstrike`, the profile was keyed
+ # `crowdstrike_falcon`, and the example a new user copied fell through
+ # to the generic fallback while still producing an alert.
+ if name not in profile_keys and name not in type_aliases:
+ failures.append(
+ (
+ "doc-example-no-profile",
+ f"documented connector_type {name!r} has no profile and no alias, so the flat payload "
+ f"shown in {locations} falls through to the generic fallback",
+ )
+ )
+
+ # MECHANISM. The aliases the generic path leans on, and the tests that
+ # prove it actually leans on them.
+ for destination in REQUIRED_IDENTITY_DESTINATIONS:
+ if not identity.get(destination):
+ failures.append(
+ ("identity-alias-missing", f"_canonicalAliases declares no sources for {destination!r}; the generic path cannot resolve it")
+ )
+ for test in REQUIRED_GO_TESTS:
+ if test not in go_tests:
+ failures.append(("behaviour-test-missing", f"{test} is gone; coverage arithmetic would still pass with the alias pass deleted"))
+ return failures
+
+
+def load(root: Path) -> dict:
+ normalizer = root / NORMALIZER_REL
+ normalizer_test = root / NORMALIZER_TEST_REL
+ connectors_dir = root / CONNECTORS_REL
+ templates_dir = root / TEMPLATES_REL
+ ts_types = root / TS_TYPES_REL
+ for path in (normalizer, normalizer_test, connectors_dir, templates_dir, ts_types):
+ if not path.exists():
+ raise GateError(f"expected input does not exist: {path}")
+
+ src = normalizer.read_text(encoding="utf-8")
+ data = {
+ "profile_keys": parse_profile_keys(src),
+ "type_aliases": parse_type_aliases(src),
+ "identity": parse_identity_destinations(src),
+ "connectors": parse_connectors(connectors_dir),
+ "doc_examples": parse_doc_examples(root),
+ "go_tests": set(re.findall(r"^func (Test\w+)\(", normalizer_test.read_text(encoding="utf-8"), re.MULTILINE)),
+ "template_names": {p.name for p in templates_dir.glob("*.yaml")},
+ "union_types": parse_union_types(ts_types),
+ }
+ # An empty parse means the format moved, not that the tree is clean.
+ for name in ("profile_keys", "connectors", "identity", "go_tests", "template_names", "union_types"):
+ if not data[name]:
+ raise GateError(f"parsed zero {name} — refusing to report a clean tree from an empty read")
+ return data
+
+
+def main(argv: list[str] | None = None) -> int:
+ parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+ parser.add_argument("--repo-root", type=Path, default=Path(__file__).resolve().parent.parent)
+ parser.add_argument("--json", action="store_true")
+ parser.add_argument("--self-test", action="store_true", help="prove the gate detects injected drift in each direction")
+ args = parser.parse_args(argv)
+
+ if args.self_test:
+ return self_test(args.repo_root.resolve())
+
+ root = args.repo_root.resolve()
+ try:
+ data = load(root)
+ except GateError as exc:
+ print(f"check_connector_profiles: FAILED to read the tree: {exc}", file=sys.stderr)
+ return 2
+
+ failures = evaluate(**data)
+ declared = data["connectors"]
+ with_profile = sorted(c for c in declared if c in data["profile_keys"] or c in data["type_aliases"])
+ canonical_only = sorted(c for c in declared if c not in with_profile and declared[c])
+ generic_only = sorted(c for c in declared if c not in with_profile and not declared[c])
+
+ if args.json:
+ print(
+ json.dumps(
+ {
+ "repo_root": str(root),
+ "declared_connectors": len(declared),
+ "profile_keys": sorted(data["profile_keys"]),
+ "type_aliases": data["type_aliases"],
+ "with_profile": with_profile,
+ "canonical_envelope_only": canonical_only,
+ "generic_fallback_only": generic_only,
+ "doc_examples": data["doc_examples"],
+ "failures": [{"code": c, "detail": d} for c, d in failures],
+ },
+ indent=2,
+ sort_keys=True,
+ )
+ )
+ return 1 if failures else 0
+
+ # Name the tree and the inputs before the verdict.
+ print(f"repo root {root}")
+ print(f"normalizer {NORMALIZER_REL} ({len(data['profile_keys'])} profiles, {len(data['type_aliases'])} type aliases)")
+ print(f"connectors {CONNECTORS_REL} ({len(declared)} declared connector ids)")
+ print(f"templates {TEMPLATES_REL} ({len(data['template_names'])} webhook templates)")
+ print(f"console types {TS_TYPES_REL} ({len(data['union_types'])} ConnectorType union members)")
+ occurrences = sum(len(v) for v in data["doc_examples"].values())
+ print(f"doc examples {len(data['doc_examples'])} distinct connector_type values across {occurrences} occurrences")
+ print()
+ print(
+ f"resolution path {len(with_profile)} via profile/alias, "
+ f"{len(canonical_only)} via canonical envelope, {len(generic_only)} generic-only"
+ )
+ if generic_only:
+ print(f" generic-only: {', '.join(generic_only)}")
+ print()
+ if failures:
+ print(f"FAIL — {len(failures)} connector-type name(s) do not resolve:")
+ for code, detail in failures:
+ print(f" [{code}] {detail}")
+ return 1
+ print("OK — every profile key names a real connector type, every declared connector")
+ print(" reaches a normalization path, and every documented example resolves.")
+ return 0
+
+
+# --------------------------------------------------------------------------
+# Self-test
+# --------------------------------------------------------------------------
+def self_test(root: Path) -> int:
+ """Inject drift in each direction and require the gate to catch each one.
+
+ A gate is only worth its runtime if it fails when it should. Each case
+ mutates the real parsed inputs and asserts the specific code fires, so a
+ refactor that quietly stops checking one direction fails here rather than
+ printing OK forever.
+ """
+ try:
+ base = load(root)
+ except GateError as exc:
+ print(f"self-test: cannot read the tree: {exc}", file=sys.stderr)
+ return 2
+
+ if evaluate(**base):
+ print("self-test: the unmodified tree already fails; fix that first", file=sys.stderr)
+ for code, detail in evaluate(**base):
+ print(f" [{code}] {detail}", file=sys.stderr)
+ return 1
+
+ def mutate(**overrides) -> dict:
+ data = {k: (v.copy() if hasattr(v, "copy") else v) for k, v in base.items()}
+ for key, fn in overrides.items():
+ fn(data[key])
+ return data
+
+ cases: list[tuple[str, str, dict]] = [
+ (
+ "GO -> PY: a profile key naming a connector that does not exist",
+ "profile-names-nothing",
+ mutate(profile_keys=lambda s: s.add("acme_xdr_9000")),
+ ),
+ (
+ "PY -> GO: a declared connector with no profile and no canonical envelope",
+ "strict-mode-unreachable",
+ mutate(connectors=lambda d: d.update({"brand_new_connector": False})),
+ ),
+ (
+ "DOCS: a documented example whose connector_type resolves to nothing",
+ "doc-example-unresolvable",
+ mutate(doc_examples=lambda d: d.update({"crowdstrik": ["README.md"]})),
+ ),
+ (
+ "DOCS: a documented example naming a real connector with no profile behind it",
+ "doc-example-no-profile",
+ mutate(doc_examples=lambda d: d.update({"confluence_audit": ["README.md"]})),
+ ),
+ (
+ "GO -> PY: an alias pointing at a profile that does not exist",
+ "alias-target-missing",
+ mutate(type_aliases=lambda d: d.update({"okta": "okta_nonexistent"})),
+ ),
+ (
+ "GO -> PY: an alias starting from a connector nobody declares",
+ "alias-source-undeclared",
+ mutate(type_aliases=lambda d: d.update({"not_a_connector": "splunk"})),
+ ),
+ (
+ "MECHANISM: the identity aliases the generic path depends on are removed",
+ "identity-alias-missing",
+ mutate(identity=lambda d: d.pop("actor.user.name", None)),
+ ),
+ (
+ "MECHANISM: the Go test proving the behaviour is deleted",
+ "behaviour-test-missing",
+ mutate(go_tests=lambda s: s.discard("TestGenericProfileResolvesIdentityFields")),
+ ),
+ (
+ "GO -> PY: a push-only profile key whose template is gone",
+ "profile-template-missing",
+ mutate(template_names=lambda s: s.discard("ai-runtime.yaml")),
+ ),
+ (
+ "TS -> *: a union member naming no connector, profile or alias",
+ "union-type-unresolvable",
+ mutate(union_types=lambda s: s.add("acme_console_only")),
+ ),
+ (
+ "TS -> *: the union ratchet left stale after a name starts resolving",
+ "union-ratchet-stale",
+ mutate(connectors=lambda d: d.update({"vectra_ai": True})),
+ ),
+ (
+ "GO -> PY: the last route to a legacy profile key disappears",
+ "legacy-profile-unreachable",
+ mutate(union_types=lambda s: s.discard("splunk_enterprise")),
+ ),
+ ]
+
+ print(f"self-test against {root}")
+ print("clean tree: 0 failures (the baseline every case below perturbs)\n")
+ ok = True
+ for description, expected_code, data in cases:
+ codes = {code for code, _ in evaluate(**data)}
+ caught = expected_code in codes
+ ok &= caught
+ print(f" {'PASS' if caught else 'FAIL'} {description}")
+ print(f" expected [{expected_code}] got {sorted(codes) or 'nothing'}")
+ print()
+ if not ok:
+ print("self-test FAILED: the gate did not catch drift it claims to catch")
+ return 1
+ print(f"self-test OK: {len(cases)} injected defects, each caught by its own code")
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/services/connectors/app/connectors/auditd.py b/services/connectors/app/connectors/auditd.py
index 74dc0948d..5cf1ca9ac 100644
--- a/services/connectors/app/connectors/auditd.py
+++ b/services/connectors/app/connectors/auditd.py
@@ -678,6 +678,11 @@ class AuditdConnector(BaseConnector):
return {
"source": self.connector_id,
+ # The ingest normalizer treats `raw_event` + `source` as the
+ # canonical connector envelope. Without it this event missed that
+ # check and fell to the generic fallback, which took the host below
+ # with it.
+ "raw_event": raw,
"category": "endpoint",
"external_id": raw.get("event_id"),
"title": title,
diff --git a/services/ingest/internal/normalizer/normalizer.go b/services/ingest/internal/normalizer/normalizer.go
index b8b4ab2d6..62c2f664f 100644
--- a/services/ingest/internal/normalizer/normalizer.go
+++ b/services/ingest/internal/normalizer/normalizer.go
@@ -254,6 +254,33 @@ var connectorProfiles = map[string]connectorProfile{
// as a lowercase string; the map below mirrors KubernetesAuditConnector
// so on-prem file_tail and webhook events end up with the same
// severity_id.
+ // email_inbox — the pull side of the forwarded-email path.
+ //
+ // EmailInboxConnector.normalize() deliberately returns the message
+ // envelope unchanged, because it is shaped for the email-forwarded.yaml
+ // template rather than for the canonical connector envelope. That left it
+ // as the one registered connector whose events reach neither a profile nor
+ // the canonical branch, so strict mode rejected them outright and lenient
+ // mode resolved a title and nothing else.
+ //
+ // The field map mirrors email-forwarded.yaml so the pull and push paths
+ // produce the same OCSF shape, the same way ai_runtime mirrors
+ // ai-runtime.yaml. Keep the two aligned.
+ "email_inbox": {
+ product: OcsfProduct{Name: "Forwarded Email", VendorName: "Email"},
+ classUID: 2001,
+ className: "Security Finding",
+ fieldMap: map[string]string{
+ "subject": "finding.title",
+ "body": "finding.desc",
+ "message_id": "finding.uid",
+ "from": "actor.user.email_addr",
+ "to": "target_user.email_addr",
+ },
+ severityMap: map[string]int{
+ "critical": 5, "high": 4, "medium": 3, "normal": 3, "low": 2, "info": 1, "informational": 1,
+ },
+ },
"kubernetes_audit": {
product: OcsfProduct{Name: "Kubernetes Audit", VendorName: "Kubernetes"},
classUID: 6003,
@@ -353,13 +380,53 @@ var _canonicalFieldMap = map[string]string{
// so a missing actor does not merely blank a column — it collapses the
// entity segment to "unknown" and every one of that connector's alerts
// correlates into the same bucket.
+//
+// The dotted sources dig one level for a scalar. Replaying the registry's 84
+// connectors through normalize() found 14 emitting `actor`, and four of those
+// emit it as the vendor's nested object rather than a name. A bare key alone
+// would hand that object to setNestedField and write a map into a slot the
+// entity extractor reads as a name, so each bare key is followed by the
+// scalar-bearing paths underneath it.
var _canonicalAliases = []struct {
dst string
sources []string
}{
- {"actor.user.name", []string{"actor", "username", "user", "user_name"}},
- {"device.name", []string{"hostname", "host", "device_name"}},
- {"src_endpoint.ip", []string{"src_ip", "source_ip", "client_ip"}},
+ {"actor.user.name", []string{"actor", "actor.name", "actor.displayName", "actor.username", "username", "user", "user.name", "user.username", "user_name"}},
+ {"device.name", []string{"hostname", "host", "host.name", "device_name", "device", "device.name", "device.hostname"}},
+ {"src_endpoint.ip", []string{"src_ip", "source_ip", "client_ip", "src_endpoint.ip", "client.ipAddress"}},
+ // Not identity, but the same fill-the-blank rule and the same reason to
+ // run everywhere. A vendor profile names the field its vendor's rows use
+ // — crowdstrike_falcon reads event_simpleName, not title — so a pushed
+ // payload that reaches a vendor profile left `message` empty and the
+ // promoter fell back to generating "Security Finding from <product>",
+ // discarding the title the caller actually sent. These two are the same
+ // pair _canonicalFieldMap declares; listing them here extends their reach
+ // to the vendor profiles without giving them a second mechanism.
+ {"message", []string{"title"}},
+ {"finding.uid", []string{"external_id"}},
+}
+
+// connectorTypeAliases maps a connector identifier the connectors service
+// actually declares onto the profile key that carries its vendor field map.
+//
+// Two vocabularies grew up either side of the wire. services/connectors
+// declares `crowdstrike` and `okta`; this file keyed their profiles
+// `crowdstrike_falcon` and `okta_system_log`, and so does the rest of the
+// platform — packages/types' ConnectorType union, the CLI's default, the
+// graph extractor and the actions credential resolver all use the longer
+// names. Renaming either side breaks the other, so both resolve instead.
+//
+// Without this an event whose connector_type is the name the product itself
+// advertises — including the one the README tells a new user to paste — missed
+// the profile lookup and fell to the generic fallback, taking generic vendor
+// attribution with it.
+//
+// `splunk` is deliberately absent: it has its own profile for the canonical
+// envelope SplunkConnector emits, which is a different shape from the raw
+// rows `splunk_enterprise` maps.
+var connectorTypeAliases = map[string]string{
+ "crowdstrike": "crowdstrike_falcon",
+ "okta": "okta_system_log",
}
// canonicalClassByConnector overrides the default Security Finding class for
@@ -453,6 +520,11 @@ func (n *Normalizer) Normalize(raw *RawEvent) (*NormalizedEvent, error) {
} else {
var ok bool
profile, ok = connectorProfiles[raw.ConnectorType]
+ if !ok {
+ if aliased, isAlias := connectorTypeAliases[raw.ConnectorType]; isAlias {
+ profile, ok = connectorProfiles[aliased]
+ }
+ }
if !ok {
if n.cfg.NormalizerMode == "strict" {
return nil, fmt.Errorf("unknown connector type: %s", raw.ConnectorType)
@@ -495,27 +567,45 @@ func (n *Normalizer) Normalize(raw *RawEvent) (*NormalizedEvent, error) {
}
}
- // Canonical envelopes additionally resolve identity aliases in declared
- // order, so a connector that spells the actor `username` is not silently
- // anonymised. Only applies where the profile uses the canonical map;
- // hand-written vendor profiles already name their own fields.
- if isCanonical {
- for _, alias := range _canonicalAliases {
- if getNestedField(ocsf, alias.dst) != nil {
- continue
- }
- for _, src := range alias.sources {
- if val := getNestedField(raw.Payload, src); val != nil {
- setNestedField(ocsf, alias.dst, val)
- break
- }
- }
+ // Resolve identity aliases in declared order for every profile, filling
+ // only destinations the field map left empty.
+ //
+ // This used to run for canonical envelopes alone, which left the generic
+ // fallback resolving nothing but title and external_id. Everything
+ // downstream is keyed on identity — entity extraction, the Investigation
+ // Rail's pivots, the {tenant}:{entity}:{tactic} correlation key, the
+ // entity graph, UEBA — so an event that arrives through the fallback
+ // became an alert with no host, no user and no IP, and nobody could pivot
+ // from it. The alert appears, which is what makes it look like it worked.
+ //
+ // Filling only empty destinations is what keeps this safe to run over the
+ // hand-written vendor profiles too: a profile that names its own field for
+ // a slot always wins, and the aliases reach only the slots it left blank.
+ for _, alias := range _canonicalAliases {
+ if getNestedField(ocsf, alias.dst) != nil {
+ continue
+ }
+ if val, found := firstIdentityString(raw.Payload, alias.sources); found {
+ setNestedField(ocsf, alias.dst, val)
}
}
- // Map severity
+ // Map severity. A vendor profile's own ladder wins; the shared five-tier
+ // ladder is consulted only when that ladder has no entry for the value.
+ //
+ // The fallback matters because a profile's ladder is spelled the way its
+ // vendor spells it — crowdstrike_falcon's is capitalised — while a pushed
+ // payload is written by whoever is pushing, and the README's is lowercase.
+ // Without the fallback a "high" that the ladder spells "High" scores 0 and
+ // renders as Unknown. Case folding alone would not do: the shared ladder
+ // keeps `critical` a distinct fifth tier, so a vendor-native critical maps
+ // to critical rather than collapsing into high.
if sevField, ok := raw.Payload["severity"].(string); ok {
- if sevID, found := profile.severityMap[sevField]; found {
+ sevID, found := profile.severityMap[sevField]
+ if !found {
+ sevID, found = _canonicalSeverityMap[strings.ToLower(strings.TrimSpace(sevField))]
+ }
+ if found {
ocsf["severity_id"] = sevID
ocsf["severity"] = sevField
} else {
@@ -711,6 +801,27 @@ func setNestedField(m map[string]interface{}, path string, val interface{}) {
setNestedField(nested, parts[1], val)
}
+// firstIdentityString returns the first non-empty string among the given dotted
+// payload paths, in the order given, and whether one was found.
+//
+// The string requirement is the point. An identity destination — actor.user.name,
+// device.name, src_endpoint.ip — is a scalar the entity extractor turns into a
+// pivotable chip and the correlator folds into {tenant}:{entity}:{tactic}.
+// Several connectors pass the vendor's nested actor object through under the
+// same key a scalar would use, and writing that object into the slot produces
+// an entity that renders as a map and correlates as garbage. Skipping it lets
+// the next path in the list — the scalar one underneath — resolve instead.
+func firstIdentityString(payload map[string]interface{}, paths []string) (string, bool) {
+ for _, p := range paths {
+ if s, ok := getNestedField(payload, p).(string); ok {
+ if trimmed := strings.TrimSpace(s); trimmed != "" {
+ return trimmed, true
+ }
+ }
+ }
+ return "", false
+}
+
// firstString returns the first non-empty string value among the given payload
// keys, or "" if none are present.
func firstString(payload map[string]interface{}, keys ...string) string {
diff --git a/services/ingest/internal/normalizer/normalizer_test.go b/services/ingest/internal/normalizer/normalizer_test.go
index a8e4be18c..7cc534b8c 100644
--- a/services/ingest/internal/normalizer/normalizer_test.go
+++ b/services/ingest/internal/normalizer/normalizer_test.go
@@ -402,8 +402,10 @@ func TestCanonicalEnvelopeResolvesHostAndIPAliases(t *testing.T) {
}
}
-// Hand-written vendor profiles name their own fields and must not have the
-// canonical aliases applied on top of them.
+// A hand-written vendor profile's own field mapping always wins. The aliases
+// run over every profile now, so this pins the precedence rather than the
+// absence: they may fill a destination the vendor map left empty, they may
+// never overwrite one it filled.
func TestAliasesDoNotApplyToRawVendorProfiles(t *testing.T) {
n := newTestNormalizer()
ev, err := n.Normalize(&RawEvent{
@@ -425,3 +427,231 @@ func TestAliasesDoNotApplyToRawVendorProfiles(t *testing.T) {
t.Errorf("actor.user.name = %v, want the vendor profile's own mapping", got)
}
}
+
+// The generic fallback must resolve identity, not just a title.
+//
+// This is the README's own push example: a flat payload, a connector type with
+// no profile of its own. Before, the fallback's field map held `title` and
+// `external_id` and nothing else, and the alias pass was gated behind the
+// canonical-envelope branch — so the event became an alert with no host, no
+// user and no IP. Entity extraction, the Investigation Rail's pivots, the
+// {tenant}:{entity}:{tactic} correlation key, the entity graph and UEBA all
+// read those three fields, so the alert arrived with nothing to pivot from.
+// The alert appearing at all is what made it look like it had worked.
+func TestGenericProfileResolvesIdentityFields(t *testing.T) {
+ n := newLenientNormalizer()
+ ev, err := n.Normalize(&RawEvent{
+ ConnectorID: "edr-1",
+ ConnectorType: "acme_xdr", // deliberately absent from connectorProfiles
+ TenantID: "11111111-1111-1111-1111-111111111111",
+ ReceivedAt: "2026-09-23T00:00:00Z",
+ Payload: map[string]interface{}{
+ "severity": "high",
+ "title": "Encoded PowerShell from Office",
+ "host": "WIN-FIN-01",
+ "user": "alice",
+ "src_ip": "10.20.30.40",
+ },
+ })
+ if err != nil {
+ t.Fatalf("Normalize returned error: %v", err)
+ }
+ for _, tc := range []struct{ path, want string }{
+ {"device.name", "WIN-FIN-01"},
+ {"actor.user.name", "alice"},
+ {"src_endpoint.ip", "10.20.30.40"},
+ } {
+ if got := nested(t, ev.OcsfEvent, tc.path); got != tc.want {
+ t.Errorf("%s = %v, want %q — the generic fallback must resolve identity", tc.path, got, tc.want)
+ }
+ }
+ // Still promotable and still vendor-neutral, which the fallback already got right.
+ if ev.OcsfEvent["class_uid"] != 2001 {
+ t.Errorf("class_uid = %v, want 2001", ev.OcsfEvent["class_uid"])
+ }
+ if ev.OcsfEvent["severity_id"] != 4 {
+ t.Errorf("severity_id = %v, want 4", ev.OcsfEvent["severity_id"])
+ }
+}
+
+// The connector identifier the product advertises must reach the profile that
+// carries its vendor field map.
+//
+// services/connectors declares `crowdstrike`; this file keyed the profile
+// `crowdstrike_falcon`. The README tells a new user to push the former, so the
+// example fell through to the generic fallback and was attributed to nothing
+// in particular. Both names resolve now, because the longer one is load-bearing
+// elsewhere — the ConnectorType union, the CLI default, the graph extractor.
+func TestDeclaredConnectorIdResolvesToVendorProfile(t *testing.T) {
+ for _, tc := range []struct {
+ connectorType string
+ wantVendor string
+ wantClass int
+ }{
+ {"crowdstrike", "CrowdStrike", 2001},
+ {"crowdstrike_falcon", "CrowdStrike", 2001},
+ {"okta", "Okta", 3002},
+ {"okta_system_log", "Okta", 3002},
+ } {
+ t.Run(tc.connectorType, func(t *testing.T) {
+ n := newLenientNormalizer()
+ ev, err := n.Normalize(&RawEvent{
+ ConnectorID: "c-1",
+ ConnectorType: tc.connectorType,
+ TenantID: "11111111-1111-1111-1111-111111111111",
+ ReceivedAt: "2026-09-23T00:00:00Z",
+ // Flat, human-authored, lowercase severity — a pushed payload,
+ // not a vendor row.
+ Payload: map[string]interface{}{
+ "severity": "high",
+ "title": "Encoded PowerShell from Office",
+ "host": "WIN-FIN-01",
+ "user": "alice",
+ "src_ip": "10.20.30.40",
+ },
+ })
+ if err != nil {
+ t.Fatalf("Normalize returned error: %v", err)
+ }
+ meta := ev.OcsfEvent["metadata"].(map[string]interface{})
+ product := meta["product"].(OcsfProduct)
+ if product.VendorName != tc.wantVendor {
+ t.Errorf("VendorName = %q, want %q — the declared id must reach the vendor profile", product.VendorName, tc.wantVendor)
+ }
+ if ev.OcsfEvent["class_uid"] != tc.wantClass {
+ t.Errorf("class_uid = %v, want %d", ev.OcsfEvent["class_uid"], tc.wantClass)
+ }
+ // Reaching a vendor profile must not cost the identity fields the
+ // generic fallback would have resolved.
+ if got := nested(t, ev.OcsfEvent, "device.name"); got != "WIN-FIN-01" {
+ t.Errorf("device.name = %v, want WIN-FIN-01", got)
+ }
+ if got := nested(t, ev.OcsfEvent, "actor.user.name"); got != "alice" {
+ t.Errorf("actor.user.name = %v, want alice", got)
+ }
+ if got := nested(t, ev.OcsfEvent, "src_endpoint.ip"); got != "10.20.30.40" {
+ t.Errorf("src_endpoint.ip = %v, want 10.20.30.40", got)
+ }
+ // ...nor the severity. crowdstrike_falcon's ladder is capitalised
+ // and a pushed payload is lowercase; without the shared-ladder
+ // fallback this scores 0 and renders as Unknown.
+ if ev.OcsfEvent["severity_id"] != 4 {
+ t.Errorf("severity_id = %v, want 4 for %q", ev.OcsfEvent["severity_id"], "high")
+ }
+ // ...nor the caller's own title. A vendor profile maps its
+ // vendor's field name, so `message` was left empty and the
+ // promoter generated "Security Finding from <product>" over the
+ // top of the title the caller sent.
+ if got := nested(t, ev.OcsfEvent, "message"); got != "Encoded PowerShell from Office" {
+ t.Errorf("message = %v, want the caller's own title", got)
+ }
+ })
+ }
+}
+
+// critical is the fifth tier, not a louder high. A vendor-native critical that
+// collapses into high silently downgrades the most urgent thing in the queue.
+func TestCriticalSeverityDoesNotCollapseIntoHigh(t *testing.T) {
+ for _, connectorType := range []string{"acme_xdr", "crowdstrike"} {
+ t.Run(connectorType, func(t *testing.T) {
+ n := newLenientNormalizer()
+ ev, err := n.Normalize(&RawEvent{
+ ConnectorID: "c-1",
+ ConnectorType: connectorType,
+ TenantID: "11111111-1111-1111-1111-111111111111",
+ ReceivedAt: "2026-09-23T00:00:00Z",
+ Payload: map[string]interface{}{"title": "t", "severity": "critical"},
+ })
+ if err != nil {
+ t.Fatalf("Normalize returned error: %v", err)
+ }
+ if ev.OcsfEvent["severity_id"] != 5 {
+ t.Errorf("severity_id = %v, want 5 (critical is its own tier)", ev.OcsfEvent["severity_id"])
+ }
+ })
+ }
+}
+
+// An identity destination is a scalar the entity extractor turns into a chip
+// and the correlator folds into its key. Several connectors pass the vendor's
+// nested actor object through under the same key a scalar would use; writing
+// that object into the slot yields an entity that renders as a map.
+func TestIdentityAliasesSkipNonScalarsAndDigForTheName(t *testing.T) {
+ n := newLenientNormalizer()
+ ev, err := n.Normalize(&RawEvent{
+ ConnectorID: "c-1",
+ ConnectorType: "acme_xdr",
+ TenantID: "11111111-1111-1111-1111-111111111111",
+ ReceivedAt: "2026-09-23T00:00:00Z",
+ Payload: map[string]interface{}{
+ "title": "t",
+ // The vendor's object under the key a scalar would use.
+ "actor": map[string]interface{}{"id": "u-1", "name": "alice"},
+ "device": map[string]interface{}{"name": "WIN-01"},
+ },
+ })
+ if err != nil {
+ t.Fatalf("Normalize returned error: %v", err)
+ }
+ got := nested(t, ev.OcsfEvent, "actor.user.name")
+ if _, isStr := got.(string); !isStr {
+ t.Fatalf("actor.user.name = %#v (%T), want a string — a map here is an unpivotable entity", got, got)
+ }
+ if got != "alice" {
+ t.Errorf("actor.user.name = %v, want alice dug out of the nested actor", got)
+ }
+ if got := nested(t, ev.OcsfEvent, "device.name"); got != "WIN-01" {
+ t.Errorf("device.name = %v, want WIN-01", got)
+ }
+}
+
+// Go randomises map iteration, so alias precedence has to come from a slice.
+// Resolving the same payload repeatedly must yield the same answer every time;
+// a map-ordered implementation passes this roughly one run in two.
+func TestIdentityAliasResolutionIsDeterministic(t *testing.T) {
+ payload := func() map[string]interface{} {
+ return map[string]interface{}{
+ "title": "t",
+ // Every alias source for actor.user.name carries a distinct value,
+ // so any ordering instability shows up as a different winner.
+ "actor": "by-actor", "username": "by-username",
+ "user": "by-user", "user_name": "by-user-name",
+ "hostname": "by-hostname", "host": "by-host", "device_name": "by-device-name",
+ "src_ip": "1.1.1.1", "source_ip": "2.2.2.2", "client_ip": "3.3.3.3",
+ }
+ }
+ const runs = 200
+ for i := 0; i < runs; i++ {
+ n := newLenientNormalizer()
+ ev, err := n.Normalize(&RawEvent{
+ ConnectorID: "c-1", ConnectorType: "acme_xdr",
+ TenantID: "11111111-1111-1111-1111-111111111111",
+ ReceivedAt: "2026-09-23T00:00:00Z", Payload: payload(),
+ })
+ if err != nil {
+ t.Fatalf("Normalize returned error: %v", err)
+ }
+ if got := nested(t, ev.OcsfEvent, "actor.user.name"); got != "by-actor" {
+ t.Fatalf("run %d: actor.user.name = %v, want by-actor (first declared source)", i, got)
+ }
+ if got := nested(t, ev.OcsfEvent, "device.name"); got != "by-hostname" {
+ t.Fatalf("run %d: device.name = %v, want by-hostname", i, got)
+ }
+ if got := nested(t, ev.OcsfEvent, "src_endpoint.ip"); got != "1.1.1.1" {
+ t.Fatalf("run %d: src_endpoint.ip = %v, want 1.1.1.1", i, got)
+ }
+ }
+}
+
+// Every connectorTypeAliases entry must point at a profile that exists, and
+// must not shadow a connector type that already has its own profile.
+func TestConnectorTypeAliasesPointAtRealProfiles(t *testing.T) {
+ for alias, target := range connectorTypeAliases {
+ if _, ok := connectorProfiles[target]; !ok {
+ t.Errorf("alias %q -> %q, but %q is not in connectorProfiles", alias, target, target)
+ }
+ if _, ok := connectorProfiles[alias]; ok {
+ t.Errorf("alias %q shadows its own profile entry; the alias is dead code", alias)
+ }
+ }
+}