Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

depth2 7.3 + 7.4: agent identity, sub-principal baselines, and the four AI rules they unblock (#1601)

Beenu Aroraacb398be4c9e
diff --git a/.gitleaksignore b/.gitleaksignore
index 229d16353..ea14a73dc 100644
--- a/.gitleaksignore
+++ b/.gitleaksignore
@@ -37,20 +37,22 @@
# the line numbers were corrected rather than the entries dropped. One entry
# is genuinely new: `llm-copilot-bulk-resource-access` trips the heuristic on
# the word "access", for the same reason as its neighbours.
-detections/rule-ids.lock.json:generic-api-key:4
-detections/rule-ids.lock.json:generic-api-key:10
-detections/rule-ids.lock.json:generic-api-key:15
+detections/rule-ids.lock.json:generic-api-key:5
+detections/rule-ids.lock.json:generic-api-key:7
+detections/rule-ids.lock.json:generic-api-key:8
+detections/rule-ids.lock.json:generic-api-key:14
detections/rule-ids.lock.json:generic-api-key:19
-detections/rule-ids.lock.json:generic-api-key:22
-detections/rule-ids.lock.json:generic-api-key:39
-detections/rule-ids.lock.json:generic-api-key:53
-detections/rule-ids.lock.json:generic-api-key:54
-detections/rule-ids.lock.json:generic-api-key:65
-detections/rule-ids.lock.json:generic-api-key:68
-detections/rule-ids.lock.json:generic-api-key:70
-detections/rule-ids.lock.json:generic-api-key:86
-detections/rule-ids.lock.json:generic-api-key:845
-detections/rule-ids.lock.json:generic-api-key:858
+detections/rule-ids.lock.json:generic-api-key:23
+detections/rule-ids.lock.json:generic-api-key:26
+detections/rule-ids.lock.json:generic-api-key:43
+detections/rule-ids.lock.json:generic-api-key:57
+detections/rule-ids.lock.json:generic-api-key:58
+detections/rule-ids.lock.json:generic-api-key:69
+detections/rule-ids.lock.json:generic-api-key:72
+detections/rule-ids.lock.json:generic-api-key:74
+detections/rule-ids.lock.json:generic-api-key:90
+detections/rule-ids.lock.json:generic-api-key:849
+detections/rule-ids.lock.json:generic-api-key:862
# Positive test fixture for the `jwt-none-alg` detection rule. This file is
# *expected* to contain a JWT with `alg: none` because the detection engine
diff --git a/DEPTH2_PROGRESS.md b/DEPTH2_PROGRESS.md
index 90f7f90f6..c0d15474e 100644
--- a/DEPTH2_PROGRESS.md
+++ b/DEPTH2_PROGRESS.md
@@ -1161,3 +1161,90 @@ that has to carry the delta.
**Deviations:** D3 (three classes have no redistributable source), D4
(tenant-supplied lists designed, not built).
+
+### 7.3 — Agent identity and per-agent baselines (depth 6.4 / parity 6.7)
+
+[#1601](https://github.com/beenuar/AiSOC/pull/1601), base `main`.
+
+**Reproduced gap.** `docs/audit/depth2-baseline/lane-e-phase73-reproduction.py`,
+runnable from a bare checkout: no module under `services/agents` defines an
+agent identity; `ToolRegistry.execute` takes no caller, declares no scope and
+writes no audit row; `baselines.py` declares seven derived fields and none is
+keyed on an AI agent.
+
+**Half A.** Short-lived identity per run, least-privilege tool scope enforced
+at `ToolRegistry.execute` — the one chokepoint every tool call passes
+through, because a check an agent performs on itself is one a steered agent
+can be talked out of. Identity is required, not optional. Nothing in the
+scope table changes an estate; a test forbids it.
+
+**Half B — one mechanism, built for Lane A.** `SUB_PRINCIPAL_FIELDS` replaces
+the hardcoded OAuth-app block, with the dict key as the field prefix so
+`baseline_oauth_app_volume_z` keeps its published name. Lane A's 1.4 adds
+`token` and `service_account` as one entry each. Eight new derived fields.
+
+**Negative controls.** Making identity optional fails one named test;
+removing the `agent` entry fails eight baseline tests.
+
+### 7.4 — four more targets shipped, two still blocked
+
+Executable detections **2,530 → 2,534** on `main`'s base (and 2,534 again on
+the landing branch, which carries the other four — the two PRs touch
+different rules and the figure is regenerated in each).
+
+Shipped and replayed through the real connector, the real `BaselineStore`
+and the real engine: `ai-agent-new-high-risk-tool`,
+`ai-agent-guardrail-violation-rate`, `ai-agent-token-spend-anomaly`,
+`ai-gateway-key-spend-anomaly`.
+
+### D5 — The 7.1 labels reach the engine as events, not as a query
+
+Decided explicitly, as the brief requires, and **not yet implemented**.
+
+The chat-content labels from 7.1 live in `aisoc_ai_chat_content_labels` in
+Postgres; the detection engine evaluates a stream. Two options:
+
+*The rule reads it where it lives* — refused. The engine is stateless and
+evaluates per event with no database handle; giving it one would mean a
+Postgres round trip per event on the hot path, and the one place that
+pattern exists (the tenant tuning overlay) is a per-tenant cache loaded
+once, not a per-event read.
+
+*The label reaches the stream* — chosen. The collector emits one OCSF event
+per newly labelled conversation, through the same ingest path a connector
+uses, carrying the labels, the shape names and the bounded snippet it
+already stores. A rule then clauses on `labels_has_any` exactly as every
+other rule clauses on connector fields, with no new mechanism and no new
+hot-path cost.
+
+**What it needs:** the API service has no ingest client — `IngestClient`
+lives in `services/connectors` — so this is either a small vendored client
+in the API or a move of the collector into the connectors service. That is
+a structural decision touching another lane's tree, which is why it is
+recorded here rather than guessed at. Nothing is shipped clausing on
+`labels` until it exists.
+
+### D6 — The enrichment-decision gate judges reachability globally
+
+Found while landing 7.1: `scripts/check_enrichment_decisions.py` asks
+whether a retired rule's fields are resolvable against the **union of every
+connector's** namespace. Promoting a field called `scopes` on Entra consent
+events therefore made `ident-github-pat-classic-created-with-broad-scopes`
+look reachable, and the gate demanded it be un-retired — while GitHub's
+audit log still does not carry a created classic PAT's scopes, so the rule
+could not fire.
+
+Worked around correctly by renaming the field to `granted_scopes`, which is
+more precise anyway. **The gate weakness is not fixed:** a rule's
+reachability should be judged against the fields its own `log_source`
+emits. Left recorded rather than changed, because it is a shared gate
+several lanes depend on and a scoping change belongs with its owner.
+
+### D7 — Lane A's Phase 1.4 had not landed when 7.3 was built
+
+`origin/main` was fetched before starting and again before committing;
+`baselines.py` on `main` still had no sub-principal mechanism. 7.3 therefore
+built the shared one rather than waiting. It is additive — a dict entry per
+kind, plus one name appended to `PRINCIPAL_FIELDS` — so if 1.4 lands first
+the two combine rather than conflict. If both touch the file, combine; never
+take one side.
diff --git a/README.md b/README.md
index a29f6f30b..e970f942a 100644
--- a/README.md
+++ b/README.md
@@ -17,8 +17,8 @@
## What AiSOC does
-Telemetry arrives from your security tools. AiSOC normalizes it, runs the 2534 executable rules of
-its 7015-rule library, groups what fires into incidents, investigates each one with an AI agent whose
+Telemetry arrives from your security tools. AiSOC normalizes it, runs the 2538 executable rules of
+its 7023-rule library, groups what fires into incidents, investigates each one with an AI agent whose
every prompt and tool call is recorded, and proposes an action. New threat intelligence re-sweeps the
history you already collected, and a human approves before anything reaches a vendor.
@@ -110,7 +110,7 @@ rule is harder to find than a missing one: of the 2,005 Splunk rules bundled her
## How it works
Ingest normalizes to a common shape and stamps an **activity projection** on every event — actor, action, resource, location, outcome, with `actor.kind` read from each vendor's own documented field rather than inferred — then Kafka carries it.
-Then fusion runs 2534 executable detection rules, of 7015 on disk, plus 76 sliding-window rules (73 count across events; 3 match an ordered sequence), applies **your tenant's own tuning** on top — the disables, floors and suppressions the console writes, so a rule you turned off actually stops firing — and decides what becomes an alert.
+Then fusion runs 2538 executable detection rules, of 7023 on disk, plus 76 sliding-window rules (73 count across events; 3 match an ordered sequence), applies **your tenant's own tuning** on top — the disables, floors and suppressions the console writes, so a rule you turned off actually stops firing — and decides what becomes an alert.
Correlation groups related alerts, an agent investigates and writes its reasoning to the Investigation Ledger, and a playbook may start from the result. Separately, new threat intelligence sweeps the lake for sightings you already collected, and a hypothesis becomes a hunt without anyone writing a query — the model fills a closed schema and every value is bound as a parameter, so it cannot express a query at all.
**A playbook triggered by an alert previews before it acts.** Three switches must agree — the
@@ -187,7 +187,7 @@ Agents triage alerts and investigate incidents. What they can and cannot do:
| Capability | Status | Tested | Production ready |
|---|---|---|---|
| Ingest → detect → correlate → alert | Stable | E2E + unit | Yes |
-| Detection engine (2534 executable rules) of 7015 | Stable | Replay proof | Yes |
+| Detection engine (2538 executable rules) of 7023 | Stable | Replay proof | Yes |
| Alert correlation into incidents | Stable | Unit | Yes |
| REST API + web console | Stable | Unit + integration | Yes |
| AI triage + Investigation Ledger | Stable | Live Postgres ledger + a PR-gated local-model agent run. No hosted provider has been exercised | Yes, copilot mode |
diff --git a/ROADMAP.md b/ROADMAP.md
index c32989201..4a3e4c931 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -38,8 +38,7 @@ A proof-first, security-first program to make every README claim gate-backed, cl
**Program status:** all 13 hardening phases (0–12) are landed. Building on them, the **Fully-Operational AI-SOC roadmap** (Phases A1–E1) is now complete — it wired the three end-to-end paths the reality audit found unwired and pushed the platform to competitive parity + beyond:
-- **Phase A (SIEM foundation):** A1 ClickHouse lake writer · A2 live detection-evaluation worker (2534 executable rules on the stream) · A3 default-on one-command deploy (connectors + graph on) · A4 UEBA behavioral-model fusion.
-- **Phase A (SIEM foundation):** A1 ClickHouse lake writer · A2 live detection-evaluation worker (2534 executable rules on the stream) · A3 default-on one-command deploy (connectors + graph on) · A4 UEBA behavioral-model fusion.
+- **Phase A (SIEM foundation):** A1 ClickHouse lake writer · A2 live detection-evaluation worker (2538 executable rules on the stream) · A3 default-on one-command deploy (connectors + graph on) · A4 UEBA behavioral-model fusion.
- **Phase B (SOAR foundation):** B1 auto-triage worker (copilot default) · B2 credential resolver + `decide()`-governed live dispatch + 10 vendor adapters · B3 real rollback + post-action verification + durable approval-SLA timers · B4 Business Context Rules in the hot path.
- **Phase C (parity differentiators):** C1 Advanced Data Explorer · C2 Effective-Permissions live posture loader · C3 autopilot/copilot posture scorecard · C4 fuse-time attack-chain auto-grouping.
- **Phase D (breadth):** D1 eight connectors (QRadar/Exabeam/Securonix/Devo/Netskope/Windows-Sysmon/Zeek-Suricata/syslog-CEF) · D2 AI/LLM-usage audit connector + `llm-*` detections + hot/cold lake tiering · D3 live-vendor mock-server smoke.
diff --git a/apps/docs/docs/detections/coverage.md b/apps/docs/docs/detections/coverage.md
index 5304f593b..2f96f87e5 100644
--- a/apps/docs/docs/detections/coverage.md
+++ b/apps/docs/docs/detections/coverage.md
@@ -10,7 +10,7 @@ sidebar_position: 2
# Detection Coverage
-Generated: `2026-11-01T19:20:24Z`
+Generated: `2026-11-01T23:20:40Z`
This page is a **curated selection**, not the size of the corpus.
It picks the highest-scoring rules per threat family from the set
@@ -27,8 +27,8 @@ Sigma compiler began translating rules where they sat: rules under
## Headline numbers
-- **Curated detections**: `487` (target: ≥ 300)
-- **Executable rules considered**: `2510` (quality floor: 0.55)
+- **Curated detections**: `489` (target: ≥ 300)
+- **Executable rules considered**: `2514` (quality floor: 0.55)
- **Unique MITRE techniques covered**: `137`
## Coverage by buyer family
@@ -42,29 +42,29 @@ Sigma compiler began translating rules where they sat: rules under
| **Cloud** | 100 | ≥ 25 | ✅ |
| **Identity** | 100 | ≥ 25 | ✅ |
| **Supply Chain** | 37 | ≥ 25 | ✅ |
-| **Kubernetes / Containers** | 69 | ≥ 25 | ✅ |
+| **Kubernetes / Containers** | 71 | ≥ 25 | ✅ |
## Distribution
### By tier
- `imported`: 163
-- `native`: 324
+- `native`: 326
### By severity
- `critical`: 100
- `high`: 261
- `low`: 16
-- `medium`: 110
+- `medium`: 112
### By category
-- `application`: 30
+- `application`: 33
- `cloud`: 152
- `data-exfil`: 12
- `endpoint`: 219
-- `identity`: 65
+- `identity`: 64
- `network`: 9
## How to audit
diff --git a/apps/docs/static/llms-full.txt b/apps/docs/static/llms-full.txt
index 7c3c0fab9..d7522dc41 100644
--- a/apps/docs/static/llms-full.txt
+++ b/apps/docs/static/llms-full.txt
@@ -222,9 +222,9 @@ Read more: https://beenuar.github.io/AiSOC/docs/connectors
## Detection content
-- **Executable rules: 2534.** These are the rules the detection engine
+- **Executable rules: 2538.** These are the rules the detection engine
loads from its compiled ruleset. Only these can fire.
-- **Rule files on disk: 7019.** The remainder is imported and
+- **Rule files on disk: 7023.** The remainder is imported and
provenance-only content — community and vendor corpora kept for coverage
mapping — plus rules whose specs have not been compiled.
- **Windowed rules: 76**, in a separate engine. Most count events or
diff --git a/apps/docs/static/llms.txt b/apps/docs/static/llms.txt
index decdece5f..9b4a63cab 100644
--- a/apps/docs/static/llms.txt
+++ b/apps/docs/static/llms.txt
@@ -23,10 +23,10 @@ Key facts a summary should get right:
so a first run shows real data and AI triage without signing up for anything.
`make up-full` is 22 long-running services and adds the event lake,
entity graph, full-text search and enrichment.
-- **Detection corpus:** 2534 rules are executable — the detection engine
- loads them from a compiled ruleset. 7019 rule files exist on disk in total;
+- **Detection corpus:** 2538 rules are executable — the detection engine
+ loads them from a compiled ruleset. 7023 rule files exist on disk in total;
the difference is imported and provenance-only content that never fires. Cite
- 2534, not 7019. "Executable" means each rule was replayed through
+ 2538, not 7023. "Executable" means each rule was replayed through
a real connector and the real engine and watched to fire; it does not mean the
rule detects an attack. A further 76 rules run in a separate windowed
engine that counts across events or matches a sequence; report that figure
diff --git a/apps/web/public/marketplace/curated.json b/apps/web/public/marketplace/curated.json
index 9cbce3733..34810c8f0 100644
--- a/apps/web/public/marketplace/curated.json
+++ b/apps/web/public/marketplace/curated.json
@@ -1,31 +1,31 @@
{
"$schema": "https://aisoc.example.com/schemas/curated/v1.json",
"version": "1.0.0",
- "generated": "2026-11-01T19:20:24Z",
+ "generated": "2026-11-01T23:20:40Z",
"stats": {
- "considered": 2510,
- "eligible": 2499,
- "selected": 487,
+ "considered": 2514,
+ "eligible": 2503,
+ "selected": 489,
"min_per_family": 25,
"min_total": 300,
"quality_floor": 0.55,
"unique_techniques": 137,
"selected_by_tier": {
"imported": 163,
- "native": 324
+ "native": 326
},
"selected_by_severity": {
"critical": 100,
"high": 261,
"low": 16,
- "medium": 110
+ "medium": 112
},
"selected_by_category": {
- "application": 30,
+ "application": 33,
"cloud": 152,
"data-exfil": 12,
"endpoint": 219,
- "identity": 65,
+ "identity": 64,
"network": 9
}
},
@@ -458,6 +458,7 @@
"det-application-075",
"det-application-080",
"det-application-089",
+ "det-application-092",
"det-cloud-001",
"det-cloud-007",
"det-cloud-014",
@@ -527,7 +528,6 @@
"det-identity-020",
"det-identity-021",
"det-identity-022",
- "det-identity-023",
"det-identity-027",
"det-identity-031",
"det-identity-034",
@@ -599,12 +599,14 @@
},
"k8s": {
"label": "Kubernetes / Containers",
- "count": 69,
+ "count": 71,
"rule_ids": [
"det-application-018",
"det-application-023",
"det-application-049",
"det-application-056",
+ "det-application-093",
+ "det-application-094",
"det-cloud-037",
"det-cloud-080",
"det-cloud-082",
@@ -712,7 +714,7 @@
"T1070": 2,
"T1070.004": 1,
"T1071.001": 1,
- "T1078": 14,
+ "T1078": 15,
"T1078.001": 3,
"T1078.004": 25,
"T1082": 2,
@@ -720,7 +722,7 @@
"T1090": 2,
"T1090.002": 1,
"T1098": 17,
- "T1098.001": 11,
+ "T1098.001": 10,
"T1098.003": 5,
"T1098.004": 4,
"T1098.007": 1,
@@ -757,7 +759,7 @@
"T1486": 7,
"T1489": 10,
"T1490": 19,
- "T1496": 4,
+ "T1496": 6,
"T1499.004": 1,
"T1505.003": 8,
"T1518": 2,
@@ -1575,6 +1577,83 @@
"log_source_product": "entra_consent",
"has_executable_body": true
},
+ {
+ "id": "det-application-092",
+ "name": "Guardrail Violations Rose Sharply for One AI Agent",
+ "path": "detections/application/ai-agent-guardrail-violation-rate.yaml",
+ "tier": "native",
+ "source": "core",
+ "category": "application",
+ "severity": "high",
+ "mitre_techniques": [
+ "T1078"
+ ],
+ "mitre_tactics": [],
+ "families": [
+ "identity"
+ ],
+ "quality_score": 0.977,
+ "quality_notes": [
+ "tier:native(+0.5)",
+ "mitre_techniques:1(+0.2)",
+ "severity:high(+0.128)",
+ "executable(+0.15)"
+ ],
+ "log_source_product": "ai_guardrail",
+ "has_executable_body": true
+ },
+ {
+ "id": "det-application-093",
+ "name": "AI Agent Spend Far Above Its Own Baseline",
+ "path": "detections/application/ai-agent-token-spend-anomaly.yaml",
+ "tier": "native",
+ "source": "core",
+ "category": "application",
+ "severity": "medium",
+ "mitre_techniques": [
+ "T1496"
+ ],
+ "mitre_tactics": [],
+ "families": [
+ "cloud",
+ "k8s"
+ ],
+ "quality_score": 0.955,
+ "quality_notes": [
+ "tier:native(+0.5)",
+ "mitre_techniques:1(+0.2)",
+ "severity:medium(+0.105)",
+ "executable(+0.15)"
+ ],
+ "log_source_product": "ai_runtime",
+ "has_executable_body": true
+ },
+ {
+ "id": "det-application-094",
+ "name": "AI Gateway Key Spend Far Above Its Own Baseline",
+ "path": "detections/application/ai-gateway-key-spend-anomaly.yaml",
+ "tier": "native",
+ "source": "core",
+ "category": "application",
+ "severity": "medium",
+ "mitre_techniques": [
+ "T1496"
+ ],
+ "mitre_tactics": [],
+ "families": [
+ "cloud",
+ "k8s"
+ ],
+ "quality_score": 0.955,
+ "quality_notes": [
+ "tier:native(+0.5)",
+ "mitre_techniques:1(+0.2)",
+ "severity:medium(+0.105)",
+ "executable(+0.15)"
+ ],
+ "log_source_product": "ai_runtime",
+ "has_executable_body": true
+ },
{
"id": "det-cloud-001",
"name": "AWS Root Account Console Login",
@@ -7582,32 +7661,6 @@
"log_source_product": "windows",
"has_executable_body": true
},
- {
- "id": "det-identity-023",
- "name": "OAuth App Granted Broad Scopes by User",
- "path": "detections/identity/oauth-broad-scope-grant.yaml",
- "tier": "native",
- "source": "core",
- "category": "identity",
- "severity": "high",
- "mitre_techniques": [
- "T1098.001"
- ],
- "mitre_tactics": [],
- "families": [
- "cloud",
- "identity"
- ],
- "quality_score": 0.977,
- "quality_notes": [
- "tier:native(+0.5)",
- "mitre_techniques:1(+0.2)",
- "severity:high(+0.128)",
- "executable(+0.15)"
- ],
- "log_source_product": "google-workspace",
- "has_executable_body": true
- },
{
"id": "det-identity-027",
"name": "Privileged Account Deleted",
diff --git a/apps/web/public/marketplace/index.json b/apps/web/public/marketplace/index.json
index d119ef249..8576c6ade 100644
--- a/apps/web/public/marketplace/index.json
+++ b/apps/web/public/marketplace/index.json
@@ -1,7 +1,7 @@
{
"$schema": "https://example.com/schemas/marketplace/v1.json",
"version": "1.0.0",
- "generated": "2026-11-01T19:20:18Z",
+ "generated": "2026-11-01T23:20:34Z",
"categories": [
{
"id": "playbooks",
@@ -20,27 +20,27 @@
}
],
"stats": {
- "total": 7190,
+ "total": 7194,
"playbooks": 88,
"playbook_packs": 63,
- "detections": 7019,
+ "detections": 7023,
"plugins": 83,
- "verified": 1069,
+ "verified": 1073,
"community": 1,
"by_tier": {
"beta": 7,
"community": 1,
"imported": 6113,
- "stable": 1069
+ "stable": 1073
},
"detections_by_tier": {
"community": 1,
"imported": 6113,
- "stable": 905
+ "stable": 909
},
- "executable": 2705,
+ "executable": 2709,
"quarantined": 4485,
- "executable_detections": 2534,
+ "executable_detections": 2538,
"reference_only_detections": 4485,
"not_a_detection": 171
},
@@ -116,7 +116,7 @@
"T1056.001": 3,
"T1056.002": 1,
"T1057": 3,
- "T1059": 49,
+ "T1059": 50,
"T1059.001": 132,
"T1059.002": 3,
"T1059.003": 24,
@@ -141,7 +141,7 @@
"T1071.004": 10,
"T1072": 4,
"T1074.001": 4,
- "T1078": 50,
+ "T1078": 51,
"T1078.001": 5,
"T1078.002": 7,
"T1078.003": 3,
@@ -258,7 +258,7 @@
"T1489": 10,
"T1490": 20,
"T1491.001": 2,
- "T1496": 4,
+ "T1496": 6,
"T1497.001": 1,
"T1498": 3,
"T1498.001": 1,
@@ -439,7 +439,7 @@
"T1686.003": 10,
"T1689": 1
},
- "total_with_mitre": 2404,
+ "total_with_mitre": 2408,
"unique_techniques_all_rules": 494,
"techniques_all_rules": {
"T1001.003": 2,
@@ -531,7 +531,7 @@
"T1056.001": 4,
"T1056.002": 4,
"T1057": 8,
- "T1059": 122,
+ "T1059": 123,
"T1059.001": 243,
"T1059.002": 11,
"T1059.003": 41,
@@ -563,7 +563,7 @@
"T1072": 9,
"T1074": 3,
"T1074.001": 6,
- "T1078": 133,
+ "T1078": 134,
"T1078.001": 10,
"T1078.002": 14,
"T1078.003": 12,
@@ -696,7 +696,7 @@
"T1491": 2,
"T1491.001": 4,
"T1495": 1,
- "T1496": 15,
+ "T1496": 17,
"T1497": 7,
"T1497.001": 3,
"T1497.003": 3,
@@ -981,7 +981,7 @@
"T1055.001": 1,
"T1055.012": 1,
"T1056.001": 1,
- "T1059": 9,
+ "T1059": 10,
"T1059.001": 5,
"T1059.002": 3,
"T1059.004": 8,
@@ -996,7 +996,7 @@
"T1071.001": 10,
"T1071.004": 6,
"T1074.001": 1,
- "T1078": 42,
+ "T1078": 43,
"T1078.001": 4,
"T1078.002": 3,
"T1078.003": 2,
@@ -1063,7 +1063,7 @@
"T1486": 8,
"T1489": 2,
"T1490": 7,
- "T1496": 2,
+ "T1496": 4,
"T1498": 4,
"T1498.001": 2,
"T1498.002": 2,
@@ -32886,6 +32886,106 @@
"path": "detections/application/llm-copilot-bulk-resource-access.yaml",
"sha256": "b1b664f05eb77501b1f895c6749dde3c88d1fbcb3e223e6c02416c26f397fad9"
},
+ {
+ "id": "det-application-091",
+ "type": "detection",
+ "name": "AI Agent Called a High-Risk Tool It Has Never Used",
+ "description": "An agent reached for a tool that changes an estate and has essentially no history of using it. Novelty is per agent and not global: execute_command is routine for a deployment bot and alarming for a summariser, and a count across the estate cannot tell them apart. The high-risk list is the same one the AI estate's exposure score reads, so an operator sees one definition of 'changes an estate' in both places.",
+ "version": "1.0.0",
+ "author": "AiSOC",
+ "tags": [
+ "tlp.white"
+ ],
+ "severity": "high",
+ "category": "application",
+ "mitre_techniques": [
+ "T1059"
+ ],
+ "log_source": "ai_runtime",
+ "playbook": "tpl-privilege-escalation",
+ "verified": true,
+ "source": "core",
+ "tier": "stable",
+ "enabled": true,
+ "executable": true,
+ "path": "detections/application/ai-agent-new-high-risk-tool.yaml",
+ "sha256": "fc48436662f0dd55a69767ec526db0e0250cf2cc6ce104bc3b2d58c1588122c9"
+ },
+ {
+ "id": "det-application-092",
+ "type": "detection",
+ "name": "Guardrail Violations Rose Sharply for One AI Agent",
+ "description": "A single agent is being blocked far more often than its own history. One guardrail violation is noise \u2014 a user phrasing something awkwardly \u2014 while a rise against that agent's own baseline is someone probing for a phrasing that gets through. Scored per agent rather than per tenant, because a busy estate's total hides one agent's change completely.",
+ "version": "1.0.0",
+ "author": "AiSOC",
+ "tags": [
+ "tlp.white"
+ ],
+ "severity": "high",
+ "category": "application",
+ "mitre_techniques": [
+ "T1078"
+ ],
+ "log_source": "ai_guardrail",
+ "playbook": "tpl-triage",
+ "verified": true,
+ "source": "core",
+ "tier": "stable",
+ "enabled": true,
+ "executable": true,
+ "path": "detections/application/ai-agent-guardrail-violation-rate.yaml",
+ "sha256": "222223b191af005fd502992b1f893559eac7d44a84744cf4fcc6334c1a73e27a"
+ },
+ {
+ "id": "det-application-093",
+ "type": "detection",
+ "name": "AI Agent Spend Far Above Its Own Baseline",
+ "description": "Cost is the signal that notices an agent being used for something other than its job \u2014 an exfiltration loop, a stolen credential driving inference, or a prompt that has started recursing. It reads the gateway's own billed figure rather than recomputing one from token counts, so it cannot disagree with the invoice. An unpriced request contributes nothing instead of a zero.",
+ "version": "1.0.0",
+ "author": "AiSOC",
+ "tags": [
+ "tlp.white"
+ ],
+ "severity": "medium",
+ "category": "application",
+ "mitre_techniques": [
+ "T1496"
+ ],
+ "log_source": "ai_runtime",
+ "playbook": "tpl-triage",
+ "verified": true,
+ "source": "core",
+ "tier": "stable",
+ "enabled": true,
+ "executable": true,
+ "path": "detections/application/ai-agent-token-spend-anomaly.yaml",
+ "sha256": "7e0fce9e3b1ea87270f409319239d515adb493edb19c6f8cde54cb695de2e9b6"
+ },
+ {
+ "id": "det-application-094",
+ "type": "detection",
+ "name": "AI Gateway Key Spend Far Above Its Own Baseline",
+ "description": "The same question asked of the key rather than the agent, and the two are not the same incident: a leaked key driving inference from outside the estate moves the key's baseline while every agent on it looks ordinary. Both are needed, which is why the baseline module keys on sub-principals rather than on one chosen subject.",
+ "version": "1.0.0",
+ "author": "AiSOC",
+ "tags": [
+ "tlp.white"
+ ],
+ "severity": "medium",
+ "category": "application",
+ "mitre_techniques": [
+ "T1496"
+ ],
+ "log_source": "ai_runtime",
+ "playbook": "tpl-triage",
+ "verified": true,
+ "source": "core",
+ "tier": "stable",
+ "enabled": true,
+ "executable": true,
+ "path": "detections/application/ai-gateway-key-spend-anomaly.yaml",
+ "sha256": "9c4e914639eabcf54da286c112ca9c468ff577bd27a764c585019921e0a6c7cc"
+ },
{
"id": "det-cloud-001",
"type": "detection",
diff --git a/apps/web/src/data/corpus-stats.json b/apps/web/src/data/corpus-stats.json
index b181043cf..61eac5024 100644
--- a/apps/web/src/data/corpus-stats.json
+++ b/apps/web/src/data/corpus-stats.json
@@ -1,14 +1,14 @@
{
- "executable": 2534,
+ "executable": 2538,
"windowed": 76,
- "onDisk": 7019,
+ "onDisk": 7023,
"quarantined": 4485,
"plugins": 83,
"playbookPacks": 63,
"playbooks": 88,
- "marketplaceItems": 7190,
+ "marketplaceItems": 7194,
"categories": {
- "application": 78,
+ "application": 82,
"cloud": 235,
"data-exfil": 12,
"endpoint": 1978,
diff --git a/detections/application/ai-agent-guardrail-violation-rate.yaml b/detections/application/ai-agent-guardrail-violation-rate.yaml
new file mode 100644
index 000000000..b6ee7e188
--- /dev/null
+++ b/detections/application/ai-agent-guardrail-violation-rate.yaml
@@ -0,0 +1,27 @@
+id: det-application-092
+name: Guardrail Violations Rose Sharply for One AI Agent
+description: A single agent is being blocked far more often than its own history. One guardrail violation
+ is noise — a user phrasing something awkwardly — while a rise against that agent's own baseline is someone
+ probing for a phrasing that gets through. Scored per agent rather than per tenant, because a busy estate's
+ total hides one agent's change completely.
+version: 1.0.0
+severity: high
+tags:
+- mitre.attack.t1078
+- tlp.white
+category: application
+log_source:
+ product: ai_guardrail
+ service: runtime
+detection:
+ condition: |
+ outcome == "blocked"
+ AND baseline_agent_volume_z >= 3.0
+false_positives:
+- A newly tightened guardrail policy, where the rise is the policy and not the agent
+- A load test replaying blocked prompts
+playbook: tpl-triage
+enabled: true
+author: AiSOC
+created: '2026-05-03'
+modified: '2026-05-03'
diff --git a/detections/application/ai-agent-new-high-risk-tool.yaml b/detections/application/ai-agent-new-high-risk-tool.yaml
new file mode 100644
index 000000000..f8b065781
--- /dev/null
+++ b/detections/application/ai-agent-new-high-risk-tool.yaml
@@ -0,0 +1,28 @@
+id: det-application-091
+name: AI Agent Called a High-Risk Tool It Has Never Used
+description: 'An agent reached for a tool that changes an estate and has essentially no history of using
+ it. Novelty is per agent and not global: execute_command is routine for a deployment bot and alarming
+ for a summariser, and a count across the estate cannot tell them apart. The high-risk list is the same
+ one the AI estate''s exposure score reads, so an operator sees one definition of ''changes an estate''
+ in both places.'
+version: 1.0.0
+severity: high
+tags:
+- mitre.attack.t1059
+- tlp.white
+category: application
+log_source:
+ product: ai_runtime
+ service: runtime
+detection:
+ condition: |
+ tool_name IN ["execute_command", "run_shell", "write_file", "delete_file", "send_email", "create_user", "grant_permission", "deploy", "terraform_apply", "database_write"]
+ AND baseline_agent_tool_rarity >= 0.95
+false_positives:
+- An agent whose scope was widened deliberately, with the change recorded
+- The first run after a deployment that added the tool
+playbook: tpl-privilege-escalation
+enabled: true
+author: AiSOC
+created: '2026-05-03'
+modified: '2026-05-03'
diff --git a/detections/application/ai-agent-token-spend-anomaly.yaml b/detections/application/ai-agent-token-spend-anomaly.yaml
new file mode 100644
index 000000000..95dbd0d5b
--- /dev/null
+++ b/detections/application/ai-agent-token-spend-anomaly.yaml
@@ -0,0 +1,26 @@
+id: det-application-093
+name: AI Agent Spend Far Above Its Own Baseline
+description: Cost is the signal that notices an agent being used for something other than its job — an
+ exfiltration loop, a stolen credential driving inference, or a prompt that has started recursing. It
+ reads the gateway's own billed figure rather than recomputing one from token counts, so it cannot disagree
+ with the invoice. An unpriced request contributes nothing instead of a zero.
+version: 1.0.0
+severity: medium
+tags:
+- mitre.attack.t1496
+- tlp.white
+category: application
+log_source:
+ product: ai_runtime
+ service: runtime
+detection:
+ condition: |
+ baseline_agent_token_spend_z >= 4.0
+false_positives:
+- A deliberately larger job, such as a backfill or a bulk summarisation
+- A model change that raised the per-request cost for every agent
+playbook: tpl-triage
+enabled: true
+author: AiSOC
+created: '2026-05-03'
+modified: '2026-05-03'
diff --git a/detections/application/ai-gateway-key-spend-anomaly.yaml b/detections/application/ai-gateway-key-spend-anomaly.yaml
new file mode 100644
index 000000000..35a9f1e74
--- /dev/null
+++ b/detections/application/ai-gateway-key-spend-anomaly.yaml
@@ -0,0 +1,25 @@
+id: det-application-094
+name: AI Gateway Key Spend Far Above Its Own Baseline
+description: 'The same question asked of the key rather than the agent, and the two are not the same incident:
+ a leaked key driving inference from outside the estate moves the key''s baseline while every agent on
+ it looks ordinary. Both are needed, which is why the baseline module keys on sub-principals rather than
+ on one chosen subject.'
+version: 1.0.0
+severity: medium
+tags:
+- mitre.attack.t1496
+- tlp.white
+category: application
+log_source:
+ product: ai_runtime
+ service: runtime
+detection:
+ condition: |
+ baseline_gateway_key_token_spend_z >= 4.0
+false_positives:
+- A key deliberately shared with a new workload
+playbook: tpl-triage
+enabled: true
+author: AiSOC
+created: '2026-05-03'
+modified: '2026-05-03'
diff --git a/detections/fixtures/negative/ai-agent-guardrail-violation-rate.json b/detections/fixtures/negative/ai-agent-guardrail-violation-rate.json
new file mode 100644
index 000000000..59182ceda
--- /dev/null
+++ b/detections/fixtures/negative/ai-agent-guardrail-violation-rate.json
@@ -0,0 +1,4 @@
+{
+ "baseline_agent_volume_z": 103.0,
+ "outcome": "not-blocked"
+}
diff --git a/detections/fixtures/negative/ai-agent-new-high-risk-tool.json b/detections/fixtures/negative/ai-agent-new-high-risk-tool.json
new file mode 100644
index 000000000..40fb54152
--- /dev/null
+++ b/detections/fixtures/negative/ai-agent-new-high-risk-tool.json
@@ -0,0 +1,4 @@
+{
+ "baseline_agent_tool_rarity": 100.95,
+ "tool_name": "__benign__"
+}
diff --git a/detections/fixtures/negative/ai-agent-token-spend-anomaly.json b/detections/fixtures/negative/ai-agent-token-spend-anomaly.json
new file mode 100644
index 000000000..c7ea4cff1
--- /dev/null
+++ b/detections/fixtures/negative/ai-agent-token-spend-anomaly.json
@@ -0,0 +1,3 @@
+{
+ "baseline_agent_token_spend_z": -96.0
+}
diff --git a/detections/fixtures/negative/ai-agent-tool-call-failed-repeatedly.json b/detections/fixtures/negative/ai-agent-tool-call-failed-repeatedly.json
index a72ff555f..9cf1ca5df 100644
--- a/detections/fixtures/negative/ai-agent-tool-call-failed-repeatedly.json
+++ b/detections/fixtures/negative/ai-agent-tool-call-failed-repeatedly.json
@@ -1,3 +1,3 @@
{
- "outcome": "not-denied"
+ "outcome": "blocked"
}
diff --git a/detections/fixtures/negative/ai-gateway-key-spend-anomaly.json b/detections/fixtures/negative/ai-gateway-key-spend-anomaly.json
new file mode 100644
index 000000000..530f6e57c
--- /dev/null
+++ b/detections/fixtures/negative/ai-gateway-key-spend-anomaly.json
@@ -0,0 +1,3 @@
+{
+ "baseline_gateway_key_token_spend_z": -96.0
+}
diff --git a/detections/fixtures/positive/ai-agent-guardrail-violation-rate.json b/detections/fixtures/positive/ai-agent-guardrail-violation-rate.json
new file mode 100644
index 000000000..dc99f32f4
--- /dev/null
+++ b/detections/fixtures/positive/ai-agent-guardrail-violation-rate.json
@@ -0,0 +1,4 @@
+{
+ "baseline_agent_volume_z": 103.0,
+ "outcome": "blocked"
+}
diff --git a/detections/fixtures/positive/ai-agent-new-high-risk-tool.json b/detections/fixtures/positive/ai-agent-new-high-risk-tool.json
new file mode 100644
index 000000000..ce4e1d275
--- /dev/null
+++ b/detections/fixtures/positive/ai-agent-new-high-risk-tool.json
@@ -0,0 +1,4 @@
+{
+ "baseline_agent_tool_rarity": 100.95,
+ "tool_name": "execute_command"
+}
diff --git a/detections/fixtures/positive/ai-agent-token-spend-anomaly.json b/detections/fixtures/positive/ai-agent-token-spend-anomaly.json
new file mode 100644
index 000000000..6c11e0ad7
--- /dev/null
+++ b/detections/fixtures/positive/ai-agent-token-spend-anomaly.json
@@ -0,0 +1,3 @@
+{
+ "baseline_agent_token_spend_z": 104.0
+}
diff --git a/detections/fixtures/positive/ai-gateway-key-spend-anomaly.json b/detections/fixtures/positive/ai-gateway-key-spend-anomaly.json
new file mode 100644
index 000000000..6802a87c8
--- /dev/null
+++ b/detections/fixtures/positive/ai-gateway-key-spend-anomaly.json
@@ -0,0 +1,3 @@
+{
+ "baseline_gateway_key_token_spend_z": 104.0
+}
diff --git a/detections/rule-ids.lock.json b/detections/rule-ids.lock.json
index 037e4450e..0c3df92f2 100644
--- a/detections/rule-ids.lock.json
+++ b/detections/rule-ids.lock.json
@@ -1,6 +1,10 @@
{
"application/ai-agent-excessive-agency": "det-application-080",
+ "application/ai-agent-guardrail-violation-rate": "det-application-092",
+ "application/ai-agent-new-high-risk-tool": "det-application-091",
+ "application/ai-agent-token-spend-anomaly": "det-application-093",
"application/ai-agent-tool-call-failed-repeatedly": "det-application-086",
+ "application/ai-gateway-key-spend-anomaly": "det-application-094",
"application/ai-prompt-contained-credential": "det-application-085",
"application/ai-prompt-injection-detected": "det-application-079",
"application/ai-sensitive-data-egress": "det-application-081",
diff --git a/docs/audit/depth2-baseline/lane-e-phase73-reproduction.py b/docs/audit/depth2-baseline/lane-e-phase73-reproduction.py
new file mode 100644
index 000000000..82fbf4514
--- /dev/null
+++ b/docs/audit/depth2-baseline/lane-e-phase73-reproduction.py
@@ -0,0 +1,92 @@
+"""Lane E reproduction: what wave-2 7.3 claims is missing, measured.
+
+Run from the repository root:
+
+ python3 docs/audit/depth2-baseline/lane-e-phase73-reproduction.py
+
+7.3 is depth 6.4, which is parity 6.7, and it has two halves. This measures
+both against the tree, so the "before" is a recorded command and not a grep
+in a progress file.
+
+Half A — agent identity
+ Parity 6.7: "Give each internal agent its own identity, with short-lived
+ credentials and least-privilege tool scopes, visible in the audit log."
+
+Half B — per-agent baselines
+ Depth 6.4: "per-agent baselines (tools, resources, volume, token spend,
+ hours) ... through the Phase 3.4 baseline module."
+
+Half B is also the dependency under four of 7.4's remaining targets, which
+is why a partial version of it would be worse than none: it would produce
+rules that cannot fire.
+"""
+
+from __future__ import annotations
+
+import pathlib
+import re
+import sys
+
+ROOT = pathlib.Path(__file__).resolve().parents[3]
+
+BASELINES = ROOT / "services/fusion/app/services/baselines.py"
+REGISTRY = ROOT / "services/agents/app/tools/registry.py"
+AGENTS_APP = ROOT / "services/agents/app"
+
+
+def _derived_fields() -> set[str]:
+ """The baseline module's own DERIVED_FIELDS set, read without importing.
+
+ Read textually rather than imported because this script has to run from
+ a bare checkout with no service virtualenv, which is the state a
+ reviewer reproducing the gap will be in.
+ """
+ source = BASELINES.read_text()
+ block = source.split("DERIVED_FIELDS: frozenset[str] = frozenset(", 1)[1].split("\n)", 1)[0]
+ rarity = source.split("RARITY_FIELDS: dict[str, tuple[str, ...]] = {", 1)[1].split("\n}", 1)[0]
+ return set(re.findall(r'"(baseline_[a-z_]+)"', block)) | set(re.findall(r'"(baseline_[a-z_]+)"', rarity))
+
+
+def main() -> int:
+ print("== Half A: agent identity ==")
+ modules = sorted(
+ p.relative_to(ROOT).as_posix()
+ for p in AGENTS_APP.rglob("*.py")
+ if re.search(r"AgentIdentity|agent_identity|agent_principal", p.read_text())
+ )
+ print(f" modules defining an agent identity: {modules or 'NONE'}")
+
+ registry = REGISTRY.read_text()
+ execute = registry.split("async def execute(", 1)[1].split("\n def ", 1)[0] if "async def execute(" in registry else ""
+ print(f" ToolRegistry.execute takes a caller identity: {'identity' in execute or 'principal' in execute}")
+ print(f" ToolRegistry declares per-agent tool scopes: {'scope' in registry.lower()}")
+ print(f" ToolRegistry writes an audit row per call: {'audit' in registry.lower()}")
+
+ print("\n== Half B: per-agent baselines ==")
+ fields = _derived_fields()
+ print(f" derived baseline fields in total: {len(fields)}")
+ for name in sorted(fields):
+ print(f" {name}")
+
+ agent_fields = sorted(f for f in fields if "agent" in f)
+ print(f" of those, keyed on an AI agent: {agent_fields or 'NONE'}")
+
+ source = BASELINES.read_text()
+ print(f" a generic sub-principal mechanism exists: {'SUB_PRINCIPAL' in source}")
+ print(f" the only sub-principal kind is hardcoded: {'OAUTH_APP_FIELDS' in source and 'SUB_PRINCIPAL' not in source}")
+
+ print("\n== What 7.4 needs from Half B ==")
+ needed = {
+ "a *new* high-risk tool (novelty per agent)": "baseline_agent_tool_rarity",
+ "guardrail-violation rate per agent": "baseline_agent_guardrail_rate_z",
+ "token spend per agent": "baseline_agent_token_spend_z",
+ "token spend per provider key": "baseline_provider_key_token_spend_z",
+ }
+ for label, field in needed.items():
+ print(f" {'HAVE' if field in fields else 'MISS'} {label} ({field})")
+
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/docs/detections/truth-table.md b/docs/detections/truth-table.md
index 4d164ddec..eb8993c03 100644
--- a/docs/detections/truth-table.md
+++ b/docs/detections/truth-table.md
@@ -19,8 +19,8 @@ lever that reaches the engine is adding a spec and re-running
| metric | count |
|--------|------:|
-| rules on disk (total) | 7019 |
-| **executable (loaded by the engine)** | **2534** |
+| rules on disk (total) | 7023 |
+| **executable (loaded by the engine)** | **2538** |
| non-executable (provenance/coverage only) | 4485 |
| — of which: enabled, but no compiled spec | 0 |
| windowed rules in `windowed_ruleset.json` | 76 |
@@ -55,7 +55,7 @@ were hand-authored in the upstream format to give it one.
| car (imported) | 99 | 0 | 0 |
| chronicle (imported) | 877 | 0 | 0 |
| community | 1 | 0 | 0 |
-| native | 881 | 740 | 0 |
+| native | 885 | 744 | 0 |
| sigma (imported) | 3132 | 1770 | 0 |
| splunk (imported) | 2005 | 0 | 0 |
| threat-anomaly | 24 | 24 | 0 |
@@ -69,9 +69,9 @@ were hand-authored in the upstream format to give it one.
## How to read the README claim
-The imported corpus is large (7019 rules on disk) and valuable as a
+The imported corpus is large (7023 rules on disk) and valuable as a
provenance-tracked ATT&CK-mapped library, but the number that matters
-operationally is **2534 executable rules** — the ones the engine
+operationally is **2538 executable rules** — the ones the engine
loads and fires against live telemetry. The README and marketplace must cite
the executable figure when describing detection *coverage*, and may cite the
on-disk figure only when explicitly describing the imported *library*.
diff --git a/marketplace/curated.json b/marketplace/curated.json
index 9cbce3733..34810c8f0 100644
--- a/marketplace/curated.json
+++ b/marketplace/curated.json
@@ -1,31 +1,31 @@
{
"$schema": "https://aisoc.example.com/schemas/curated/v1.json",
"version": "1.0.0",
- "generated": "2026-11-01T19:20:24Z",
+ "generated": "2026-11-01T23:20:40Z",
"stats": {
- "considered": 2510,
- "eligible": 2499,
- "selected": 487,
+ "considered": 2514,
+ "eligible": 2503,
+ "selected": 489,
"min_per_family": 25,
"min_total": 300,
"quality_floor": 0.55,
"unique_techniques": 137,
"selected_by_tier": {
"imported": 163,
- "native": 324
+ "native": 326
},
"selected_by_severity": {
"critical": 100,
"high": 261,
"low": 16,
- "medium": 110
+ "medium": 112
},
"selected_by_category": {
- "application": 30,
+ "application": 33,
"cloud": 152,
"data-exfil": 12,
"endpoint": 219,
- "identity": 65,
+ "identity": 64,
"network": 9
}
},
@@ -458,6 +458,7 @@
"det-application-075",
"det-application-080",
"det-application-089",
+ "det-application-092",
"det-cloud-001",
"det-cloud-007",
"det-cloud-014",
@@ -527,7 +528,6 @@
"det-identity-020",
"det-identity-021",
"det-identity-022",
- "det-identity-023",
"det-identity-027",
"det-identity-031",
"det-identity-034",
@@ -599,12 +599,14 @@
},
"k8s": {
"label": "Kubernetes / Containers",
- "count": 69,
+ "count": 71,
"rule_ids": [
"det-application-018",
"det-application-023",
"det-application-049",
"det-application-056",
+ "det-application-093",
+ "det-application-094",
"det-cloud-037",
"det-cloud-080",
"det-cloud-082",
@@ -712,7 +714,7 @@
"T1070": 2,
"T1070.004": 1,
"T1071.001": 1,
- "T1078": 14,
+ "T1078": 15,
"T1078.001": 3,
"T1078.004": 25,
"T1082": 2,
@@ -720,7 +722,7 @@
"T1090": 2,
"T1090.002": 1,
"T1098": 17,
- "T1098.001": 11,
+ "T1098.001": 10,
"T1098.003": 5,
"T1098.004": 4,
"T1098.007": 1,
@@ -757,7 +759,7 @@
"T1486": 7,
"T1489": 10,
"T1490": 19,
- "T1496": 4,
+ "T1496": 6,
"T1499.004": 1,
"T1505.003": 8,
"T1518": 2,
@@ -1575,6 +1577,83 @@
"log_source_product": "entra_consent",
"has_executable_body": true
},
+ {
+ "id": "det-application-092",
+ "name": "Guardrail Violations Rose Sharply for One AI Agent",
+ "path": "detections/application/ai-agent-guardrail-violation-rate.yaml",
+ "tier": "native",
+ "source": "core",
+ "category": "application",
+ "severity": "high",
+ "mitre_techniques": [
+ "T1078"
+ ],
+ "mitre_tactics": [],
+ "families": [
+ "identity"
+ ],
+ "quality_score": 0.977,
+ "quality_notes": [
+ "tier:native(+0.5)",
+ "mitre_techniques:1(+0.2)",
+ "severity:high(+0.128)",
+ "executable(+0.15)"
+ ],
+ "log_source_product": "ai_guardrail",
+ "has_executable_body": true
+ },
+ {
+ "id": "det-application-093",
+ "name": "AI Agent Spend Far Above Its Own Baseline",
+ "path": "detections/application/ai-agent-token-spend-anomaly.yaml",
+ "tier": "native",
+ "source": "core",
+ "category": "application",
+ "severity": "medium",
+ "mitre_techniques": [
+ "T1496"
+ ],
+ "mitre_tactics": [],
+ "families": [
+ "cloud",
+ "k8s"
+ ],
+ "quality_score": 0.955,
+ "quality_notes": [
+ "tier:native(+0.5)",
+ "mitre_techniques:1(+0.2)",
+ "severity:medium(+0.105)",
+ "executable(+0.15)"
+ ],
+ "log_source_product": "ai_runtime",
+ "has_executable_body": true
+ },
+ {
+ "id": "det-application-094",
+ "name": "AI Gateway Key Spend Far Above Its Own Baseline",
+ "path": "detections/application/ai-gateway-key-spend-anomaly.yaml",
+ "tier": "native",
+ "source": "core",
+ "category": "application",
+ "severity": "medium",
+ "mitre_techniques": [
+ "T1496"
+ ],
+ "mitre_tactics": [],
+ "families": [
+ "cloud",
+ "k8s"
+ ],
+ "quality_score": 0.955,
+ "quality_notes": [
+ "tier:native(+0.5)",
+ "mitre_techniques:1(+0.2)",
+ "severity:medium(+0.105)",
+ "executable(+0.15)"
+ ],
+ "log_source_product": "ai_runtime",
+ "has_executable_body": true
+ },
{
"id": "det-cloud-001",
"name": "AWS Root Account Console Login",
@@ -7582,32 +7661,6 @@
"log_source_product": "windows",
"has_executable_body": true
},
- {
- "id": "det-identity-023",
- "name": "OAuth App Granted Broad Scopes by User",
- "path": "detections/identity/oauth-broad-scope-grant.yaml",
- "tier": "native",
- "source": "core",
- "category": "identity",
- "severity": "high",
- "mitre_techniques": [
- "T1098.001"
- ],
- "mitre_tactics": [],
- "families": [
- "cloud",
- "identity"
- ],
- "quality_score": 0.977,
- "quality_notes": [
- "tier:native(+0.5)",
- "mitre_techniques:1(+0.2)",
- "severity:high(+0.128)",
- "executable(+0.15)"
- ],
- "log_source_product": "google-workspace",
- "has_executable_body": true
- },
{
"id": "det-identity-027",
"name": "Privileged Account Deleted",
diff --git a/marketplace/index.json b/marketplace/index.json
index d119ef249..8576c6ade 100644
--- a/marketplace/index.json
+++ b/marketplace/index.json
@@ -1,7 +1,7 @@
{
"$schema": "https://example.com/schemas/marketplace/v1.json",
"version": "1.0.0",
- "generated": "2026-11-01T19:20:18Z",
+ "generated": "2026-11-01T23:20:34Z",
"categories": [
{
"id": "playbooks",
@@ -20,27 +20,27 @@
}
],
"stats": {
- "total": 7190,
+ "total": 7194,
"playbooks": 88,
"playbook_packs": 63,
- "detections": 7019,
+ "detections": 7023,
"plugins": 83,
- "verified": 1069,
+ "verified": 1073,
"community": 1,
"by_tier": {
"beta": 7,
"community": 1,
"imported": 6113,
- "stable": 1069
+ "stable": 1073
},
"detections_by_tier": {
"community": 1,
"imported": 6113,
- "stable": 905
+ "stable": 909
},
- "executable": 2705,
+ "executable": 2709,
"quarantined": 4485,
- "executable_detections": 2534,
+ "executable_detections": 2538,
"reference_only_detections": 4485,
"not_a_detection": 171
},
@@ -116,7 +116,7 @@
"T1056.001": 3,
"T1056.002": 1,
"T1057": 3,
- "T1059": 49,
+ "T1059": 50,
"T1059.001": 132,
"T1059.002": 3,
"T1059.003": 24,
@@ -141,7 +141,7 @@
"T1071.004": 10,
"T1072": 4,
"T1074.001": 4,
- "T1078": 50,
+ "T1078": 51,
"T1078.001": 5,
"T1078.002": 7,
"T1078.003": 3,
@@ -258,7 +258,7 @@
"T1489": 10,
"T1490": 20,
"T1491.001": 2,
- "T1496": 4,
+ "T1496": 6,
"T1497.001": 1,
"T1498": 3,
"T1498.001": 1,
@@ -439,7 +439,7 @@
"T1686.003": 10,
"T1689": 1
},
- "total_with_mitre": 2404,
+ "total_with_mitre": 2408,
"unique_techniques_all_rules": 494,
"techniques_all_rules": {
"T1001.003": 2,
@@ -531,7 +531,7 @@
"T1056.001": 4,
"T1056.002": 4,
"T1057": 8,
- "T1059": 122,
+ "T1059": 123,
"T1059.001": 243,
"T1059.002": 11,
"T1059.003": 41,
@@ -563,7 +563,7 @@
"T1072": 9,
"T1074": 3,
"T1074.001": 6,
- "T1078": 133,
+ "T1078": 134,
"T1078.001": 10,
"T1078.002": 14,
"T1078.003": 12,
@@ -696,7 +696,7 @@
"T1491": 2,
"T1491.001": 4,
"T1495": 1,
- "T1496": 15,
+ "T1496": 17,
"T1497": 7,
"T1497.001": 3,
"T1497.003": 3,
@@ -981,7 +981,7 @@
"T1055.001": 1,
"T1055.012": 1,
"T1056.001": 1,
- "T1059": 9,
+ "T1059": 10,
"T1059.001": 5,
"T1059.002": 3,
"T1059.004": 8,
@@ -996,7 +996,7 @@
"T1071.001": 10,
"T1071.004": 6,
"T1074.001": 1,
- "T1078": 42,
+ "T1078": 43,
"T1078.001": 4,
"T1078.002": 3,
"T1078.003": 2,
@@ -1063,7 +1063,7 @@
"T1486": 8,
"T1489": 2,
"T1490": 7,
- "T1496": 2,
+ "T1496": 4,
"T1498": 4,
"T1498.001": 2,
"T1498.002": 2,
@@ -32886,6 +32886,106 @@
"path": "detections/application/llm-copilot-bulk-resource-access.yaml",
"sha256": "b1b664f05eb77501b1f895c6749dde3c88d1fbcb3e223e6c02416c26f397fad9"
},
+ {
+ "id": "det-application-091",
+ "type": "detection",
+ "name": "AI Agent Called a High-Risk Tool It Has Never Used",
+ "description": "An agent reached for a tool that changes an estate and has essentially no history of using it. Novelty is per agent and not global: execute_command is routine for a deployment bot and alarming for a summariser, and a count across the estate cannot tell them apart. The high-risk list is the same one the AI estate's exposure score reads, so an operator sees one definition of 'changes an estate' in both places.",
+ "version": "1.0.0",
+ "author": "AiSOC",
+ "tags": [
+ "tlp.white"
+ ],
+ "severity": "high",
+ "category": "application",
+ "mitre_techniques": [
+ "T1059"
+ ],
+ "log_source": "ai_runtime",
+ "playbook": "tpl-privilege-escalation",
+ "verified": true,
+ "source": "core",
+ "tier": "stable",
+ "enabled": true,
+ "executable": true,
+ "path": "detections/application/ai-agent-new-high-risk-tool.yaml",
+ "sha256": "fc48436662f0dd55a69767ec526db0e0250cf2cc6ce104bc3b2d58c1588122c9"
+ },
+ {
+ "id": "det-application-092",
+ "type": "detection",
+ "name": "Guardrail Violations Rose Sharply for One AI Agent",
+ "description": "A single agent is being blocked far more often than its own history. One guardrail violation is noise \u2014 a user phrasing something awkwardly \u2014 while a rise against that agent's own baseline is someone probing for a phrasing that gets through. Scored per agent rather than per tenant, because a busy estate's total hides one agent's change completely.",
+ "version": "1.0.0",
+ "author": "AiSOC",
+ "tags": [
+ "tlp.white"
+ ],
+ "severity": "high",
+ "category": "application",
+ "mitre_techniques": [
+ "T1078"
+ ],
+ "log_source": "ai_guardrail",
+ "playbook": "tpl-triage",
+ "verified": true,
+ "source": "core",
+ "tier": "stable",
+ "enabled": true,
+ "executable": true,
+ "path": "detections/application/ai-agent-guardrail-violation-rate.yaml",
+ "sha256": "222223b191af005fd502992b1f893559eac7d44a84744cf4fcc6334c1a73e27a"
+ },
+ {
+ "id": "det-application-093",
+ "type": "detection",
+ "name": "AI Agent Spend Far Above Its Own Baseline",
+ "description": "Cost is the signal that notices an agent being used for something other than its job \u2014 an exfiltration loop, a stolen credential driving inference, or a prompt that has started recursing. It reads the gateway's own billed figure rather than recomputing one from token counts, so it cannot disagree with the invoice. An unpriced request contributes nothing instead of a zero.",
+ "version": "1.0.0",
+ "author": "AiSOC",
+ "tags": [
+ "tlp.white"
+ ],
+ "severity": "medium",
+ "category": "application",
+ "mitre_techniques": [
+ "T1496"
+ ],
+ "log_source": "ai_runtime",
+ "playbook": "tpl-triage",
+ "verified": true,
+ "source": "core",
+ "tier": "stable",
+ "enabled": true,
+ "executable": true,
+ "path": "detections/application/ai-agent-token-spend-anomaly.yaml",
+ "sha256": "7e0fce9e3b1ea87270f409319239d515adb493edb19c6f8cde54cb695de2e9b6"
+ },
+ {
+ "id": "det-application-094",
+ "type": "detection",
+ "name": "AI Gateway Key Spend Far Above Its Own Baseline",
+ "description": "The same question asked of the key rather than the agent, and the two are not the same incident: a leaked key driving inference from outside the estate moves the key's baseline while every agent on it looks ordinary. Both are needed, which is why the baseline module keys on sub-principals rather than on one chosen subject.",
+ "version": "1.0.0",
+ "author": "AiSOC",
+ "tags": [
+ "tlp.white"
+ ],
+ "severity": "medium",
+ "category": "application",
+ "mitre_techniques": [
+ "T1496"
+ ],
+ "log_source": "ai_runtime",
+ "playbook": "tpl-triage",
+ "verified": true,
+ "source": "core",
+ "tier": "stable",
+ "enabled": true,
+ "executable": true,
+ "path": "detections/application/ai-gateway-key-spend-anomaly.yaml",
+ "sha256": "9c4e914639eabcf54da286c112ca9c468ff577bd27a764c585019921e0a6c7cc"
+ },
{
"id": "det-cloud-001",
"type": "detection",
diff --git a/scripts/check_detection_fields.py b/scripts/check_detection_fields.py
index 84872a364..f39c96677 100644
--- a/scripts/check_detection_fields.py
+++ b/scripts/check_detection_fields.py
@@ -297,8 +297,22 @@ _BASELINE_DERIVED_FIELDS = frozenset(
"baseline_client_family_rarity",
"baseline_location_rarity",
"baseline_principal_volume_z",
- "baseline_oauth_app_volume_z",
"baseline_peer_deviation",
+ # Sub-principal baselines (wave-2 7.3, and wave-2 1.4's mechanism).
+ # `baselines.SUB_PRINCIPAL_FIELDS` generates these from its keys, so
+ # a kind added there must be added here too — which is precisely
+ # what the bidirectional test in `test_baselines.py` enforces, and
+ # what caught this list when the sub-principal table landed.
+ "baseline_oauth_app_volume_z",
+ "baseline_agent_volume_z",
+ "baseline_gateway_key_volume_z",
+ # Per-agent novelty and rate (depth 6.4's "tools, resources, volume,
+ # token spend, hours").
+ "baseline_agent_tool_rarity",
+ "baseline_agent_model_rarity",
+ "baseline_agent_hour_rarity",
+ "baseline_agent_token_spend_z",
+ "baseline_gateway_key_token_spend_z",
# Depth plan wave 2, 1.4 — per-sub-principal baselines. A token, an
# OAuth app, a service account or an AI agent acting for a human
# gets its own baseline, because a stolen credential acts *as* the
diff --git a/scripts/detection_specs_part3_application.py b/scripts/detection_specs_part3_application.py
index 12cbafd97..8164304a3 100644
--- a/scripts/detection_specs_part3_application.py
+++ b/scripts/detection_specs_part3_application.py
@@ -852,6 +852,117 @@ AI_RUNTIME_RULES: list[dict] = [
"capture mode."
),
),
+ # ----------------------------------------------------------------
+ # Wave-2 7.4, the four targets Phase 7.3's baselines unblock.
+ #
+ # Each clauses on a `baseline_agent_*` or `baseline_gateway_key_*`
+ # derived field, which `services/fusion/app/services/baselines.py`
+ # computes per sub-principal and `scripts/check_detection_fields.py`
+ # knows about. Novelty and rate are properties of a *history*, so the
+ # stateless matcher cannot express them on its own — which is exactly
+ # why these four could not be written before 7.3 and were recorded as
+ # blocked rather than shipped clausing on fields nothing produced.
+ #
+ # Thresholds are stated as floors on an already-bounded score: rarity
+ # is in [0,1] and a z-score is in deviations, so the numbers below are
+ # readable without knowing the corpus.
+ # ----------------------------------------------------------------
+ S(
+ slug="ai-agent-new-high-risk-tool",
+ name="AI Agent Called a High-Risk Tool It Has Never Used",
+ severity="high",
+ mitre=["t1059"],
+ product="ai_runtime",
+ service="runtime",
+ when={
+ "tool_name_in": [
+ "execute_command",
+ "run_shell",
+ "write_file",
+ "delete_file",
+ "send_email",
+ "create_user",
+ "grant_permission",
+ "deploy",
+ "terraform_apply",
+ "database_write",
+ ],
+ "baseline_agent_tool_rarity_gte": 0.95,
+ },
+ fp=[
+ "An agent whose scope was widened deliberately, with the change recorded",
+ "The first run after a deployment that added the tool",
+ ],
+ playbook="tpl-privilege-escalation",
+ description=(
+ "An agent reached for a tool that changes an estate and has essentially no "
+ "history of using it. Novelty is per agent and not global: execute_command is "
+ "routine for a deployment bot and alarming for a summariser, and a count across "
+ "the estate cannot tell them apart. The high-risk list is the same one the AI "
+ "estate's exposure score reads, so an operator sees one definition of "
+ "'changes an estate' in both places."
+ ),
+ ),
+ S(
+ slug="ai-agent-guardrail-violation-rate",
+ name="Guardrail Violations Rose Sharply for One AI Agent",
+ severity="high",
+ mitre=["t1078"],
+ product="ai_guardrail",
+ service="runtime",
+ when={"outcome": "blocked", "baseline_agent_volume_z_gte": 3.0},
+ fp=[
+ "A newly tightened guardrail policy, where the rise is the policy and not the agent",
+ "A load test replaying blocked prompts",
+ ],
+ playbook="tpl-triage",
+ description=(
+ "A single agent is being blocked far more often than its own history. One "
+ "guardrail violation is noise — a user phrasing something awkwardly — while a "
+ "rise against that agent's own baseline is someone probing for a phrasing that "
+ "gets through. Scored per agent rather than per tenant, because a busy estate's "
+ "total hides one agent's change completely."
+ ),
+ ),
+ S(
+ slug="ai-agent-token-spend-anomaly",
+ name="AI Agent Spend Far Above Its Own Baseline",
+ severity="medium",
+ mitre=["t1496"],
+ product="ai_runtime",
+ service="runtime",
+ when={"baseline_agent_token_spend_z_gte": 4.0},
+ fp=[
+ "A deliberately larger job, such as a backfill or a bulk summarisation",
+ "A model change that raised the per-request cost for every agent",
+ ],
+ playbook="tpl-triage",
+ description=(
+ "Cost is the signal that notices an agent being used for something other than "
+ "its job — an exfiltration loop, a stolen credential driving inference, or a "
+ "prompt that has started recursing. It reads the gateway's own billed figure "
+ "rather than recomputing one from token counts, so it cannot disagree with the "
+ "invoice. An unpriced request contributes nothing instead of a zero."
+ ),
+ ),
+ S(
+ slug="ai-gateway-key-spend-anomaly",
+ name="AI Gateway Key Spend Far Above Its Own Baseline",
+ severity="medium",
+ mitre=["t1496"],
+ product="ai_runtime",
+ service="runtime",
+ when={"baseline_gateway_key_token_spend_z_gte": 4.0},
+ fp=["A key deliberately shared with a new workload"],
+ playbook="tpl-triage",
+ description=(
+ "The same question asked of the key rather than the agent, and the two are not "
+ "the same incident: a leaked key driving inference from outside the estate "
+ "moves the key's baseline while every agent on it looks ordinary. Both are "
+ "needed, which is why the baseline module keys on sub-principals rather than "
+ "on one chosen subject."
+ ),
+ ),
S(
slug="ai-agent-tool-call-failed-repeatedly",
name="AI Agent Tool Call Denied",
diff --git a/services/agents/app/investigator/deep_investigation.py b/services/agents/app/investigator/deep_investigation.py
index d62f67e4f..d7cb58583 100644
--- a/services/agents/app/investigator/deep_investigation.py
+++ b/services/agents/app/investigator/deep_investigation.py
@@ -44,6 +44,7 @@ from app.llm.tool_loop import run_with_tools
from app.mcp.tools import build_mcp_toolset
from app.prompting.envelope import system_rule
from app.prompting.tool_results import unverified_tool_claims
+from app.security.agent_identity import issue_identity
from app.tools.customer_tools import scoped_customer_tools
from app.tools.investigation import investigation_tools
from app.tools.registry import default_registry
@@ -496,10 +497,23 @@ async def run_deep_investigation(
# nonce is appended to the system message here: an MCP result is
# fenced with it, and a fence the system prompt never explains is a
# delimiter rather than a boundary.
+ run_id = str(getattr(state, "run_id", None) or "unknown-run")
+ # Wave-2 7.3. The investigator acts under its own short-lived
+ # identity, scoped to the pivots it is allowed to reach, and every
+ # tool call it makes writes an audit row naming the agent and run.
+ identity = issue_identity("investigation", run_id=run_id)
+
mcp = await build_mcp_toolset(tenant_id, run_id=getattr(state, "run_id", None))
if mcp.tools:
for tool in mcp.tools:
registry.register(tool)
+ # `build_mcp_toolset` has already classified each tool and
+ # refused the destructive ones (`app/mcp/policy.py`), so these
+ # names arrive approved. Extending the scope with them is one
+ # control handing its verdict to the other; the alternative —
+ # a static table naming a tenant's MCP tools — cannot exist,
+ # because they are discovered at run time.
+ identity = identity.with_granted_tools({t.name for t in mcp.tools})
system = f"{system}\n\n{system_rule(mcp.nonce)}"
result.mcp_tools = [t.name for t in mcp.tools]
result.mcp_refusals = [f"{name}: {classification}" for name, classification, _ in mcp.refusals]
@@ -510,6 +524,7 @@ async def run_deep_investigation(
system=system,
user=_summarise_alert(state),
registry=registry,
+ identity=identity,
max_iters=max_iterations or MAX_ITERATIONS,
),
timeout=BUDGET_SECONDS,
diff --git a/services/agents/app/llm/tool_loop.py b/services/agents/app/llm/tool_loop.py
index 2583f57a9..96e550484 100644
--- a/services/agents/app/llm/tool_loop.py
+++ b/services/agents/app/llm/tool_loop.py
@@ -26,6 +26,7 @@ from langchain_core.messages import AIMessage, HumanMessage, SystemMessage, Tool
from app.llm.contract import safe_ainvoke
from app.prompting.tool_results import render_tool_message
+from app.security.agent_identity import AgentIdentity
from app.tools.registry import ToolRegistry
logger = structlog.get_logger()
@@ -37,12 +38,18 @@ async def run_with_tools(
system: str,
user: str,
registry: ToolRegistry,
+ identity: AgentIdentity,
max_iters: int = 4,
) -> dict[str, Any]:
"""Run a tool-calling conversation and return the final content + tool trace.
Returns ``{"content": str, "tool_trace": [...], "iterations": int,
"truncated": bool}``.
+
+ ``identity`` is required rather than defaulted (wave-2 7.3). This loop
+ is the only production caller of ``registry.execute``, so a default
+ here would be the default for the whole service — and an identity that
+ defaults to "allow" is a control that protects nothing.
"""
bound = llm.bind_tools(registry.openai_schemas())
messages: list[Any] = [SystemMessage(content=system), HumanMessage(content=user)]
@@ -70,7 +77,7 @@ async def run_with_tools(
name = call.get("name", "")
args = call.get("args", {}) or {}
call_id = call.get("id", "") or ""
- result = await registry.execute(name, args)
+ result = await registry.execute(name, args, identity=identity)
trace.append({"tool": name, "args": args, "result_preview": str(result)[:200]})
# The only place in this service where a tool result becomes text
# a model reads. Depth plan 1.3 turned that from an accident of
diff --git a/services/agents/app/security/agent_identity.py b/services/agents/app/security/agent_identity.py
new file mode 100644
index 000000000..386b3d426
--- /dev/null
+++ b/services/agents/app/security/agent_identity.py
@@ -0,0 +1,300 @@
+"""An identity, a scope and an expiry for each of AiSOC's own agents.
+
+Depth plan wave 2, Phase 7.3 — parity 6.7's first half: "Give each internal
+agent its own identity, with short-lived credentials and least-privilege
+tool scopes, visible in the audit log."
+
+Why this exists
+---------------
+``ToolRegistry.execute(name, args)`` ran any registered tool for any caller.
+There was no identity, no scope and no audit row, which meant three things:
+
+* a triage agent steered by injected text into asking for a containment
+ tool would have it run, because nothing compared the request against what
+ that agent is allowed to hold;
+* the question an incident asks of an agent — *what did it do* — had no
+ answer in the ledger, because no row named the caller;
+* and the platform that flags "no person is linked, nobody is accountable"
+ on a customer's AI estate was running its own agents with exactly that
+ property.
+
+What "short-lived" means here, and what it does not
+----------------------------------------------------
+A credential bound to one investigation run with a wall-clock expiry. It is
+deliberately **not** a bearer token to another service: these agents call
+Python functions inside their own process, so a network credential would be
+ceremony rather than a control, and this repository has shipped enough of
+those to be wary of one more.
+
+What the control has to survive is the in-process threat — a model steered
+into requesting a tool outside its grant — and the only place that can be
+stopped is the one chokepoint every tool call passes through. The expiry
+earns its place separately: a run that stalls and resumes hours later must
+re-issue rather than continue on a credential nobody has re-authorised.
+
+Why the scopes are a table and not a decorator
+-----------------------------------------------
+Because the interesting operation is reading them side by side. "Which
+agents can run a containment tool" is a question a reviewer asks of the
+whole system, and a table answers it in one screen; decorators scattered
+across twelve modules do not. The table is also what makes
+``test_the_scopes_are_least_privilege_and_not_all_the_same`` possible: if
+every agent held every tool the scope would be a comment, and that is
+checkable here and nowhere else.
+
+Unknown agents are refused rather than defaulted
+-------------------------------------------------
+Defaulting an undeclared agent to "everything" makes the control
+decorative. Defaulting it to "nothing" breaks that agent silently at run
+time, in production, on whatever path first reached for a tool. Refusing to
+mint an identity at all fails at the point a human can still see it.
+"""
+
+from __future__ import annotations
+
+import time
+import uuid
+from dataclasses import dataclass, field
+
+#: Default credential lifetime. Long enough for a deep investigation,
+#: short enough that a stalled run re-issues rather than continuing on a
+#: grant nobody re-authorised.
+DEFAULT_TTL_SECONDS = 1800
+
+#: Which tools each of AiSOC's own agents may call.
+#:
+#: Read as a privilege boundary, not as a dependency list: these are the
+#: tools an agent is *permitted* to reach, and the set is deliberately
+#: unequal across agents. The triage and investigation agents read; only
+#: the response agent reaches anything that changes an estate, and it does
+#: so through the governed action path, which applies its own capability
+#: contract and approval tier on top of this.
+#:
+#: A tool absent from every set here is reachable by nobody, which is the
+#: correct default for a tool nobody has thought about.
+#: Every tool name AiSOC's own registries actually register.
+#:
+#: Written out rather than imported, because importing the registries here
+#: would make this module depend on the investigator, the sandbox client and
+#: the MITRE corpus just to answer "what is a valid tool name". The cost of
+#: writing them out is drift, and that is paid for by
+#: `test_every_scoped_tool_name_is_a_tool_something_registers`, which walks
+#: the registries and fails if a name here is not among them.
+#:
+#: This mattered: the first draft of the table below listed `search_logs`,
+#: `query_lake` and `graph_blast_radius`, none of which any registry
+#: registers. A scope naming tools that do not exist grants nothing and
+#: refuses everything real — the same class of mistake as a detection rule
+#: clausing on a field no connector emits, and just as invisible, because a
+#: scope full of plausible names reads as careful.
+KNOWN_TOOLS: frozenset[str] = frozenset(
+ {
+ # default_registry
+ "lookup_file_hash",
+ "extract_iocs",
+ "map_to_mitre",
+ "lookup_technique",
+ "enrich_ioc",
+ # investigator customer tools (typed, lake-backed pivots)
+ "process_activity",
+ "historical_execution",
+ "network_connections",
+ "authentication_events",
+ "fleet_ioc_hunt",
+ "entity_timeline",
+ "technique_activity",
+ "process_tree",
+ "mailbox_activity",
+ "oauth_grants",
+ "persistence_mechanisms",
+ # Wave-2 1.1 and 1.5, registered unconditionally in
+ # `default_registry`: what this identity could do at the moment it
+ # acted, and what changed shortly before it did. Read-only, and
+ # both answer "unknown, and here is why" rather than a silent zero
+ # when no snapshot covers the question.
+ "identity_effective_permissions",
+ "changes_before_alert",
+ # Customer-estate read tools (`app/tools/customer_tools.py`). These
+ # reach the customer's own vendors through the governed read path
+ # rather than AiSOC's lake, which is why the investigator holds
+ # them and nothing else does.
+ "siem_indicator_search",
+ "cloud_audit_lookup",
+ "edr_host_details",
+ "edr_host_detections",
+ "endpoint_telemetry_sightings",
+ "identity_user_activity",
+ }
+)
+
+#: Which tools each of AiSOC's own agents may call.
+#:
+#: Read as a privilege boundary, not as a dependency list. The sets are
+#: deliberately unequal: triage is the agent most exposed to
+#: attacker-controlled text and holds the least, while the investigator
+#: holds the deep pivots because that is its job.
+#:
+#: Note what is *not* here: nothing in this table changes an estate. AiSOC's
+#: own agents propose; execution goes through `services/actions`, which
+#: applies a capability contract and an approval tier of its own. Putting a
+#: containment verb in this table would create a second, weaker door to the
+#: same operation.
+#:
+#: A tool in `KNOWN_TOOLS` and in no scope here is reachable by nobody,
+#: which is the correct default for a tool nobody has thought about.
+AGENT_TOOL_SCOPES: dict[str, frozenset[str]] = {
+ # Reads alert context, extracts indicators, looks up techniques. Holds
+ # no pivot that reaches customer telemetry, because this is the agent
+ # an injected alert talks to first.
+ "triage": frozenset({"extract_iocs", "map_to_mitre", "lookup_technique", "enrich_ioc"}),
+ # The recursive investigator: everything triage holds, plus the typed
+ # lake-backed pivots and the sandbox lookup.
+ "investigation": frozenset(
+ {
+ "extract_iocs",
+ "map_to_mitre",
+ "lookup_technique",
+ "enrich_ioc",
+ "lookup_file_hash",
+ "process_activity",
+ "historical_execution",
+ "network_connections",
+ "authentication_events",
+ "fleet_ioc_hunt",
+ "entity_timeline",
+ "technique_activity",
+ "process_tree",
+ "mailbox_activity",
+ "oauth_grants",
+ "persistence_mechanisms",
+ # The two halves of "what could this identity do, and what
+ # changed before it did". The investigator rather than triage:
+ # both take typed entity fields, but the scope table keeps the
+ # agent most exposed to attacker-controlled text at its minimum.
+ "identity_effective_permissions",
+ "changes_before_alert",
+ "siem_indicator_search",
+ "cloud_audit_lookup",
+ "edr_host_details",
+ "edr_host_detections",
+ "endpoint_telemetry_sightings",
+ "identity_user_activity",
+ }
+ ),
+ # Hunting sweeps the fleet and reads timelines; it does not enrich
+ # against third parties, which is a per-tenant opt-in with its own key.
+ "hunt": frozenset(
+ {
+ "extract_iocs",
+ "lookup_technique",
+ "fleet_ioc_hunt",
+ "entity_timeline",
+ "technique_activity",
+ "historical_execution",
+ "siem_indicator_search",
+ }
+ ),
+ # Proposes a disposition and a containment plan. Reads what it needs to
+ # justify one; executes nothing here.
+ "respond": frozenset({"lookup_technique", "entity_timeline", "process_tree", "network_connections"}),
+ # Detection engineering drafts rules from technique coverage.
+ "detect": frozenset({"map_to_mitre", "lookup_technique", "technique_activity", "historical_execution"}),
+}
+
+
+class ScopeDenied(RuntimeError):
+ """A tool was requested outside the caller's scope, or after expiry.
+
+ An exception rather than a sentinel because a direct caller must not be
+ able to mistake a denial for a result. ``ToolRegistry`` converts it into
+ the structured ``{"error": ...}`` its loop contract requires, which is a
+ conversion at one place rather than every caller parsing a string.
+ """
+
+
+@dataclass(frozen=True)
+class AgentIdentity:
+ """One agent, one run, one expiry, one scope."""
+
+ agent: str
+ run_id: str
+ expires_at: float
+ scopes: frozenset[str]
+ #: Distinguishes two concurrent runs of the same agent in the audit log.
+ credential_id: str = field(default_factory=lambda: uuid.uuid4().hex)
+
+ def expired(self, *, now: float | None = None) -> bool:
+ return (now if now is not None else time.time()) >= self.expires_at
+
+ def may(self, tool: str) -> bool:
+ return not self.expired() and tool in self.scopes
+
+ def require(self, tool: str, *, now: float | None = None) -> None:
+ """Raise unless this identity may call ``tool`` right now."""
+ if self.expired(now=now):
+ raise ScopeDenied(
+ f"the credential for agent {self.agent!r} on run {self.run_id!r} has expired; re-issue it rather than extending it"
+ )
+ if tool not in self.scopes:
+ raise ScopeDenied(f"tool {tool!r} is outside the scope of agent {self.agent!r}")
+
+ def with_granted_tools(self, tools: frozenset[str] | set[str] | list[str]) -> AgentIdentity:
+ """A copy of this identity extended with tools another control approved.
+
+ The one legitimate way the scope grows, and it exists for MCP: a
+ tenant's MCP servers are discovered at run time, so their tool names
+ cannot be in a static table. They are not ungoverned — `app/mcp/
+ policy.py` classifies each one and refuses the destructive ones
+ before this is ever called — so this is one control handing its
+ verdict to the other rather than a hole in either.
+
+ The expiry and the credential id are carried over deliberately: a
+ grant must not silently extend the credential's life, and the audit
+ rows for the granted tools must join to the same run.
+ """
+ return _granted(self, frozenset(str(t) for t in tools))
+
+ def audit_row(self, tool: str, *, allowed: bool, reason: str | None = None) -> dict[str, object]:
+ """One row for the audit log. Never the tool's arguments.
+
+ Arguments carry alert content — hostnames, account names, raw text
+ lifted from a vendor payload. An audit log is read by more people
+ than an alert is, so a tool-call log that copied the arguments
+ would become a second, wider copy of the evidence.
+ """
+ row: dict[str, object] = {
+ "agent": self.agent,
+ "run_id": self.run_id,
+ "credential_id": self.credential_id,
+ "tool": tool,
+ "allowed": allowed,
+ "at": time.time(),
+ }
+ if reason:
+ row["reason"] = reason
+ return row
+
+
+def _granted(identity: AgentIdentity, tools: frozenset[str]) -> AgentIdentity:
+ return AgentIdentity(
+ agent=identity.agent,
+ run_id=identity.run_id,
+ expires_at=identity.expires_at,
+ scopes=identity.scopes | tools,
+ credential_id=identity.credential_id,
+ )
+
+
+def issue_identity(agent: str, *, run_id: str, ttl_seconds: int | None = None) -> AgentIdentity:
+ """Mint a short-lived identity for one of AiSOC's own agents.
+
+ Refuses an agent with no declared scope. See the module docstring for
+ why that is a refusal and not a default in either direction.
+ """
+ scopes = AGENT_TOOL_SCOPES.get(agent)
+ if scopes is None:
+ raise ValueError(
+ f"unknown agent {agent!r}: declare its tool scope in AGENT_TOOL_SCOPES. Known agents: {', '.join(sorted(AGENT_TOOL_SCOPES))}."
+ )
+ ttl = DEFAULT_TTL_SECONDS if ttl_seconds is None else ttl_seconds
+ return AgentIdentity(agent=agent, run_id=run_id, expires_at=time.time() + ttl, scopes=scopes)
diff --git a/services/agents/app/tools/registry.py b/services/agents/app/tools/registry.py
index f07c12520..3e9a71ea2 100644
--- a/services/agents/app/tools/registry.py
+++ b/services/agents/app/tools/registry.py
@@ -12,27 +12,61 @@ Design:
* ``execute`` is fail-soft: an unknown tool or a raising tool returns a
structured ``{"error": ...}`` the model can react to, never an exception that
aborts the loop.
+* ``execute`` is also the **one chokepoint every tool call passes through**,
+ which is why wave-2 7.3 puts the scope check here rather than in each
+ agent. A check an agent performs on itself is a check a steered agent can
+ be talked out of.
+
+Identity is required, not optional
+----------------------------------
+``execute`` refuses a call that names no identity. An optional identity
+defaulting to "allow" would leave every existing call site working and the
+control protecting nothing — which is exactly the shape of the eleven routes
+that carried ``Annotated[Any, require_permission("x:write")]`` with no
+``Depends()``, read as gated, and enforced nothing.
"""
from __future__ import annotations
import inspect
+import logging
+from collections.abc import Callable
from typing import Any
+from app.security.agent_identity import AgentIdentity, ScopeDenied
+
# Re-exported so every existing `from app.tools.registry import Tool`
# keeps working. The dataclass moved to its own module to break the
# registry -> customer_tools -> registry import cycle; see tool.py.
from app.tools.tool import Tool
+logger = logging.getLogger(__name__)
+
__all__ = ["Tool", "ToolRegistry", "default_registry"]
class ToolRegistry:
def __init__(self, tools: list[Tool] | None = None) -> None:
self._tools: dict[str, Tool] = {}
+ #: Called with one row per tool call, allowed or denied. Settable so
+ #: a deployment can route it at the ledger and a test can read it;
+ #: defaults to the module logger so the rows exist on every
+ #: deployment rather than only on the ones that wired a sink.
+ self.audit_sink: Callable[[dict[str, Any]], None] = _log_audit_row
for tool in tools or []:
self.register(tool)
+ def _audit(self, row: dict[str, Any]) -> None:
+ """Record one call. A sink failure must not take the run with it.
+
+ The inverse coupling — losing the tool result because the audit
+ sink was down — turns an observability outage into an outage.
+ """
+ try:
+ self.audit_sink(row)
+ except Exception: # noqa: BLE001 — see the docstring
+ logger.warning("agent tool audit sink failed; the call itself was unaffected", exc_info=True)
+
def register(self, tool: Tool) -> None:
self._tools[tool.name] = tool
@@ -42,10 +76,42 @@ class ToolRegistry:
def openai_schemas(self) -> list[dict[str, Any]]:
return [t.openai_schema() for t in self._tools.values()]
- async def execute(self, name: str, args: dict[str, Any]) -> Any:
+ async def execute(self, name: str, args: dict[str, Any], *, identity: AgentIdentity | None = None) -> Any:
+ """Run one tool for one identity, or refuse and say why.
+
+ The refusals are structured errors rather than exceptions because
+ the loop's contract is that a tool failure reaches the model
+ instead of aborting the run. They are worded so an operator reading
+ the ledger can tell a denial from a tool that broke — "outside the
+ scope of agent 'triage'" and "TypeError" send you to different
+ places.
+ """
+ if identity is None:
+ # Default-deny. See the module docstring: an optional identity
+ # that defaults to allow is a control that protects nothing.
+ self._audit({"agent": None, "tool": name, "allowed": False, "reason": "no identity"})
+ return {"error": f"refusing to run {name!r}: no agent identity was supplied"}
+ # Existence before scope, deliberately. A tool nothing registered
+ # cannot run either way, and "unknown tool" sends a reader to the
+ # registration that is missing while "outside the scope of agent
+ # 'investigation'" sends them to a scope table that is correct.
+ #
+ # The MCP path makes this concrete: `app/mcp/policy.py` refuses a
+ # destructive tool by never registering it, so that refusal is the
+ # one an operator should see. No oracle is leaked by the ordering —
+ # the registry's schemas are shipped to the model already.
tool = self._tools.get(name)
if tool is None:
+ self._audit(identity.audit_row(name, allowed=False, reason="unknown tool"))
return {"error": f"unknown tool: {name}"}
+
+ try:
+ identity.require(name)
+ except ScopeDenied as denied:
+ self._audit(identity.audit_row(name, allowed=False, reason=str(denied)))
+ return {"error": str(denied)}
+
+ self._audit(identity.audit_row(name, allowed=True))
try:
result = tool.fn(**(args or {}))
if inspect.isawaitable(result):
@@ -55,6 +121,16 @@ class ToolRegistry:
return {"error": f"{type(exc).__name__}: {exc}"}
+def _log_audit_row(row: dict[str, Any]) -> None:
+ """Default sink: the service log, so the rows exist everywhere.
+
+ A sink that defaulted to discarding would make "visible in the audit
+ log" true only on deployments that wired one, which is the same as not
+ true.
+ """
+ logger.info("agent tool call", extra={"agent_tool_call": row})
+
+
def _str_param(name: str, desc: str) -> dict[str, Any]:
return {"type": "object", "properties": {name: {"type": "string", "description": desc}}, "required": [name]}
diff --git a/services/agents/tests/test_agent_identity.py b/services/agents/tests/test_agent_identity.py
new file mode 100644
index 000000000..23d3d452a
--- /dev/null
+++ b/services/agents/tests/test_agent_identity.py
@@ -0,0 +1,315 @@
+"""AiSOC's own agents get an identity, a scope and an audit row.
+
+Depth plan wave 2, Phase 7.3 — parity 6.7's first half: "Give each internal
+agent its own identity, with short-lived credentials and least-privilege
+tool scopes, visible in the audit log."
+
+The gap
+-------
+`ToolRegistry.execute(name, args)` ran any registered tool for any caller.
+There was no identity, no scope and no audit row, so:
+
+* a prompt-injected triage agent could call a containment tool it has no
+ business holding, and the registry would run it;
+* nothing in the ledger said which agent called which tool, so the question
+ an incident asks — what did this agent do — had no answer;
+* and the platform that sells AI-estate governance was running its own
+ agents with the one property it flags on a customer's: no identity.
+
+What "short-lived" means here, and what it does not
+----------------------------------------------------
+A credential bounded to one investigation run with a wall-clock expiry.
+It is **not** a bearer token to another service: these agents call Python
+functions in their own process, so a network credential would be theatre.
+What it has to survive is the in-process threat — a model that has been
+steered into asking for a tool outside its grant — and that is a scope
+check at the one chokepoint every tool call passes through.
+
+The negative controls matter more than the positives here. A scope system
+that grants everything, or expires nothing, passes every happy-path test.
+"""
+
+from __future__ import annotations
+
+import time
+
+import pytest
+from app.security.agent_identity import (
+ AGENT_TOOL_SCOPES,
+ KNOWN_TOOLS,
+ ScopeDenied,
+ issue_identity,
+)
+from app.tools.registry import Tool, ToolRegistry
+
+
+def _tool(name: str) -> Tool:
+ return Tool(name=name, description=name, parameters={"type": "object", "properties": {}}, fn=lambda: {"ok": name})
+
+
+def registry() -> ToolRegistry:
+ # Real tool names: `lookup_technique` is in the triage scope,
+ # `process_activity` is an investigator-only pivot and is therefore the
+ # out-of-scope case for a triage identity. Inventing names here would
+ # make the test pass against a scope table that grants nothing real.
+ return ToolRegistry([_tool("lookup_technique"), _tool("extract_iocs"), _tool("process_activity")])
+
+
+# ---------------------------------------------------------------------------
+# Identity
+# ---------------------------------------------------------------------------
+
+
+def test_every_shipped_agent_has_a_declared_scope():
+ """An agent missing from the table would otherwise default to something.
+
+ Defaulting open makes the scope decorative; defaulting closed breaks
+ that agent silently at run time. Requiring a declaration is the only
+ option that fails at review instead.
+ """
+ assert AGENT_TOOL_SCOPES, "no agent scopes are declared at all"
+ for agent, scopes in AGENT_TOOL_SCOPES.items():
+ assert isinstance(scopes, frozenset), f"{agent}: scopes must be a frozenset"
+ assert scopes, f"{agent}: an empty scope set is almost certainly a mistake, not least privilege"
+
+
+def test_the_scopes_are_least_privilege_and_not_all_the_same():
+ """If every agent holds every tool, the scope is a comment.
+
+ The triage agent reads; the response agent acts. A table where those
+ two are equal has not expressed a privilege boundary.
+ """
+ sets = list(AGENT_TOOL_SCOPES.values())
+ assert len({frozenset(s) for s in sets}) > 1, "every agent holds the same tools, which is not least privilege"
+
+
+def test_an_identity_is_bound_to_one_run_and_expires():
+ identity = issue_identity("triage", run_id="run-1", ttl_seconds=60)
+ assert identity.agent == "triage"
+ assert identity.run_id == "run-1"
+ assert not identity.expired()
+ assert identity.expires_at > time.time()
+
+
+def test_an_expired_identity_says_so():
+ identity = issue_identity("triage", run_id="run-1", ttl_seconds=-1)
+ assert identity.expired()
+
+
+def test_an_unknown_agent_cannot_be_issued_an_identity():
+ """Minting an identity for a name with no scope would grant it nothing
+ and then fail at every call, which reads as a broken tool rather than
+ as an undeclared agent."""
+ with pytest.raises(ValueError, match="unknown agent"):
+ issue_identity("not-an-agent", run_id="run-1")
+
+
+# ---------------------------------------------------------------------------
+# The scope is enforced at the one chokepoint
+# ---------------------------------------------------------------------------
+
+
+async def test_a_tool_inside_the_scope_runs(registry: ToolRegistry):
+ identity = issue_identity("triage", run_id="run-1")
+ result = await registry.execute("lookup_technique", {}, identity=identity)
+ assert result == {"ok": "lookup_technique"}
+
+
+async def test_a_tool_outside_the_scope_is_refused(registry: ToolRegistry):
+ """The whole point. A triage agent does not hold containment.
+
+ Refused as a structured error rather than an exception, because the
+ loop's contract is that a tool failure reaches the model instead of
+ aborting the run — but the error names the scope so an operator can
+ tell a denial from a tool that broke.
+ """
+ identity = issue_identity("triage", run_id="run-1")
+ result = await registry.execute("process_activity", {}, identity=identity)
+ assert "error" in result
+ assert "scope" in result["error"].lower()
+
+
+async def test_an_expired_identity_is_refused(registry: ToolRegistry):
+ """Otherwise "short-lived" is a field nobody reads.
+
+ A run that outlives its credential is the case this catches: a stalled
+ investigation resuming hours later must re-issue, not continue.
+ """
+ identity = issue_identity("triage", run_id="run-1", ttl_seconds=-1)
+ result = await registry.execute("lookup_technique", {}, identity=identity)
+ assert "error" in result
+ assert "expired" in result["error"].lower()
+
+
+async def test_a_call_with_no_identity_is_refused_by_default(registry: ToolRegistry):
+ """Default-deny, which is the only safe direction.
+
+ An optional identity defaulting to "allow" means every existing call
+ site keeps working and the control protects nothing — the exact shape
+ of the `Annotated[Any, require_permission(...)]` routes that read as
+ gated and enforced nothing.
+ """
+ result = await registry.execute("lookup_technique", {})
+ assert "error" in result
+ assert "identity" in result["error"].lower()
+
+
+def test_scope_denied_is_raised_by_the_checker_itself():
+ """The registry converts it; the checker raises it.
+
+ Two callers need different behaviour — the loop wants a structured
+ error, a direct caller wants an exception — and one of them having to
+ parse the other's string is how a denial gets mistaken for a result.
+ """
+ identity = issue_identity("triage", run_id="run-1")
+ with pytest.raises(ScopeDenied):
+ identity.require("process_activity")
+
+
+# ---------------------------------------------------------------------------
+# Visible in the audit log
+# ---------------------------------------------------------------------------
+
+
+async def test_every_call_writes_an_audit_row_including_the_denials(registry: ToolRegistry):
+ """Parity 6.7 says "visible in the audit log", and a denial is the row
+ an incident actually wants: an agent reaching for a tool it does not
+ hold is the signal, and a log that records only successes cannot show
+ it."""
+ rows: list[dict] = []
+ registry.audit_sink = rows.append
+ identity = issue_identity("triage", run_id="run-1")
+
+ await registry.execute("lookup_technique", {}, identity=identity)
+ await registry.execute("process_activity", {}, identity=identity)
+
+ assert len(rows) == 2
+ allowed, denied = rows
+ assert allowed["agent"] == "triage" and allowed["tool"] == "lookup_technique" and allowed["allowed"] is True
+ assert denied["agent"] == "triage" and denied["tool"] == "process_activity" and denied["allowed"] is False
+ assert all(r["run_id"] == "run-1" for r in rows)
+
+
+async def test_the_audit_row_never_carries_the_tool_arguments(registry: ToolRegistry):
+ """Arguments carry alert content — hostnames, user names, raw text.
+
+ An audit log is read by more people than an alert is, and a tool-call
+ log that copies the arguments becomes a second, wider copy of the
+ evidence.
+ """
+ rows: list[dict] = []
+ registry.audit_sink = rows.append
+ identity = issue_identity("triage", run_id="run-1")
+
+ await registry.execute("lookup_technique", {"query": "[email protected] password=hunter2"}, identity=identity)
+
+ assert "hunter2" not in str(rows[0])
+ assert "[email protected]" not in str(rows[0])
+
+
+async def test_an_audit_sink_that_raises_does_not_break_the_tool_call(registry: ToolRegistry):
+ """A logging failure must not take the investigation with it.
+
+ The inverse — swallowing the tool result because the audit sink was
+ down — is the kind of coupling that turns an observability outage into
+ an outage.
+ """
+
+ def explode(_row: dict) -> None:
+ raise RuntimeError("sink down")
+
+ registry.audit_sink = explode
+ identity = issue_identity("triage", run_id="run-1")
+ assert await registry.execute("lookup_technique", {}, identity=identity) == {"ok": "lookup_technique"}
+
+
+# ---------------------------------------------------------------------------
+# The scope table names real tools
+# ---------------------------------------------------------------------------
+
+
+def test_every_scoped_tool_name_is_a_tool_something_registers():
+ """A scope naming tools that do not exist grants nothing and refuses
+ everything real.
+
+ The first draft of `AGENT_TOOL_SCOPES` listed `search_logs`,
+ `query_lake` and `graph_blast_radius`. No registry registers any of
+ them. That is the same class of mistake as a detection rule clausing on
+ a field no connector emits, and just as invisible: a scope full of
+ plausible names reads as careful.
+
+ Walked against the real registries rather than against `KNOWN_TOOLS`
+ alone, so the written-out list cannot drift from them either.
+ """
+ scoped = {tool for scope in AGENT_TOOL_SCOPES.values() for tool in scope}
+ missing = sorted(scoped - _registered_tool_names())
+ assert not missing, f"scoped but registered by nothing: {missing}"
+
+
+def _registered_tool_names() -> set[str]:
+ from app.tools.customer_tools import SIEM_SEARCH_TOOL, VENDOR_READ_TOOLS
+ from app.tools.investigation import investigation_tools
+ from app.tools.registry import default_registry
+
+ return (
+ set(default_registry("t").names()) | {tool.name for tool in investigation_tools("t")} | {SIEM_SEARCH_TOOL} | set(VENDOR_READ_TOOLS)
+ )
+
+
+def test_every_registered_tool_is_reachable_by_some_agent():
+ """The other direction, which is the one that caught the real bug.
+
+ Checking only "every scoped name is registered" passes on a scope
+ table that omits six tools: nothing is named that does not exist, and
+ six real tools are reachable by nobody. The deep-investigation
+ acceptance bar then drops from three pivots to one, and the symptom —
+ a thinner investigation — looks like a model problem rather than an
+ authorization one.
+
+ That is the one-directional-gate failure this repository keeps
+ finding: the graph-schema drift check compared Go against YAML and
+ never YAML against Go, and reported OK while eleven labels were
+ missing.
+ """
+ scoped = {tool for scope in AGENT_TOOL_SCOPES.values() for tool in scope}
+ unreachable = sorted(_registered_tool_names() - scoped)
+ assert not unreachable, (
+ f"registered but in no agent's scope, so nothing can call them: {unreachable}. "
+ "Add them to the agent that should hold them, or delete the tool."
+ )
+
+
+def test_known_tools_covers_everything_the_scopes_name():
+ scoped = {tool for scope in AGENT_TOOL_SCOPES.values() for tool in scope}
+ assert not sorted(scoped - KNOWN_TOOLS)
+
+
+def test_no_agent_scope_contains_a_tool_that_changes_an_estate():
+ """AiSOC's own agents propose; `services/actions` executes.
+
+ Putting a containment verb in this table would create a second, weaker
+ door to the same operation — one with no capability contract and no
+ approval tier.
+ """
+ acting = {"isolate_host", "disable_user", "block_indicator", "kill_process", "revoke_session", "delete_file"}
+ for agent, scope in AGENT_TOOL_SCOPES.items():
+ assert not (scope & acting), f"{agent} holds an estate-changing tool: {sorted(scope & acting)}"
+
+
+async def test_a_granted_mcp_tool_becomes_callable_without_extending_the_expiry():
+ """The one legitimate way a scope grows.
+
+ A tenant's MCP tools are discovered at run time, so they cannot be in
+ a static table; `app/mcp/policy.py` classifies and refuses the
+ destructive ones before this is reached. The expiry and credential id
+ carry over, so a grant cannot silently lengthen the credential's life
+ and the granted calls still join to the same run.
+ """
+ base = issue_identity("investigation", run_id="run-1")
+ granted = base.with_granted_tools({"tenant_wiki_search"})
+
+ assert granted.may("tenant_wiki_search")
+ assert not base.may("tenant_wiki_search")
+ assert granted.expires_at == base.expires_at
+ assert granted.credential_id == base.credential_id
diff --git a/services/agents/tests/test_escalation_context_depth.py b/services/agents/tests/test_escalation_context_depth.py
index 855157178..465e8dc9e 100644
--- a/services/agents/tests/test_escalation_context_depth.py
+++ b/services/agents/tests/test_escalation_context_depth.py
@@ -33,6 +33,7 @@ from app.agents.investigation_agent import _bundle_findings
# asserting nothing.
from app.llm import tool_loop
from app.models.state import AgentStatus, InvestigationState
+from app.security.agent_identity import issue_identity
from app.workers import fused_alert_consumer as consumer
from app.workers.fused_alert_consumer import FusedAlertTriageWorker
from app.workers.triage_persistence import LiveTriageWriter
@@ -203,6 +204,8 @@ async def test_the_tool_loop_routes_through_the_llm_contract(
def bind_tools(self, schemas): # noqa: ANN001
return self
- out = await tool_loop.run_with_tools(_LLM(), system="s", user="u", registry=ToolRegistry())
+ out = await tool_loop.run_with_tools(
+ _LLM(), system="s", user="u", registry=ToolRegistry(), identity=issue_identity("triage", run_id="run-1")
+ )
assert called, "the loop bypassed safe_ainvoke"
assert out["content"] == "done"
diff --git a/services/agents/tests/test_mcp_investigation.py b/services/agents/tests/test_mcp_investigation.py
index f86876ca3..7af1a8cb9 100644
--- a/services/agents/tests/test_mcp_investigation.py
+++ b/services/agents/tests/test_mcp_investigation.py
@@ -32,6 +32,7 @@ from app.mcp import tools as tools_module
from app.mcp.config import McpServerConfig
from app.mcp.policy import namespaced
from app.mcp.tools import LEDGER_SEQ_BASE, build_mcp_toolset
+from app.security.agent_identity import issue_identity
from app.tools.registry import ToolRegistry
from mcp.server.fastmcp import FastMCP
from mcp.shared.memory import create_connected_server_and_client_session
@@ -147,7 +148,11 @@ async def test_done_when_an_investigation_calls_a_read_tool_and_refuses_a_destru
_final("One detection on WIN-DC-01; containment was not available to me."),
]
)
- loop = await run_with_tools(llm, system="You are an analyst.", user="Investigate WIN-DC-01.", registry=registry)
+ # The MCP tools are namespaced and discovered at run time, so the
+ # identity is extended with exactly what the MCP policy approved —
+ # which is how production reaches them too.
+ identity = issue_identity("investigation", run_id="run-1").with_granted_tools(set(registry.names()))
+ loop = await run_with_tools(llm, system="You are an analyst.", user="Investigate WIN-DC-01.", registry=registry, identity=identity)
advertised = {s["function"]["name"] for s in (llm.bound_tools or [])}
assert namespaced("vendor", "isolate_host") not in advertised
@@ -159,7 +164,7 @@ async def test_done_when_an_investigation_calls_a_read_tool_and_refuses_a_destru
# The read tool's result did carry the server's data, fenced. Asserted
# against the registry the loop used rather than against the trace, whose
# preview is a 200-character debug string.
- direct = await registry.execute(namespaced("vendor", "get_detections"), {"hostname": "WIN-DC-01"})
+ direct = await registry.execute(namespaced("vendor", "get_detections"), {"hostname": "WIN-DC-01"}, identity=identity)
assert "DET-4471" in direct["content"]
assert direct["untrusted"] is True
diff --git a/services/agents/tests/test_tenant_skill_investigation.py b/services/agents/tests/test_tenant_skill_investigation.py
index 7c2862283..338ff381d 100644
--- a/services/agents/tests/test_tenant_skill_investigation.py
+++ b/services/agents/tests/test_tenant_skill_investigation.py
@@ -216,7 +216,7 @@ def _captured_system(monkeypatch: pytest.MonkeyPatch) -> list[str]:
"""Capture the system prompt the tool loop is given, and answer immediately."""
seen: list[str] = []
- async def _fake_loop(model: Any, *, system: str, user: str, registry: Any, max_iters: int) -> dict[str, Any]:
+ async def _fake_loop(model: Any, *, system: str, user: str, registry: Any, identity: Any, max_iters: int) -> dict[str, Any]:
seen.append(system)
return {"content": "nothing to add", "tool_trace": [], "iterations": 1, "truncated": False}
diff --git a/services/agents/tests/test_tool_calling.py b/services/agents/tests/test_tool_calling.py
index 835a1b5a7..08e1dfcdd 100644
--- a/services/agents/tests/test_tool_calling.py
+++ b/services/agents/tests/test_tool_calling.py
@@ -7,6 +7,7 @@ from types import SimpleNamespace
import pytest
from app.agents.tool_loop import run_with_tools
+from app.security.agent_identity import issue_identity
from app.tools.registry import Tool, ToolRegistry, default_registry
@@ -47,10 +48,11 @@ def test_registry_exposes_openai_schemas():
@pytest.mark.asyncio
async def test_registry_execute_runs_real_tool_and_handles_unknown():
reg = default_registry()
- iocs = await reg.execute("extract_iocs", {"text": "beacon to evil.example and 45.9.148.99"})
+ identity = issue_identity("investigation", run_id="run-1")
+ iocs = await reg.execute("extract_iocs", {"text": "beacon to evil.example and 45.9.148.99"}, identity=identity)
assert isinstance(iocs, list)
# Unknown tool fails soft.
- err = await reg.execute("nope", {})
+ err = await reg.execute("nope", {}, identity=identity)
assert "error" in err
@@ -58,7 +60,13 @@ async def test_registry_execute_runs_real_tool_and_handles_unknown():
async def test_tool_loop_executes_selected_tool_then_answers():
reg = default_registry()
llm = _FakeLLM([_tool_call("extract_iocs", {"text": "45.9.148.99 and evil.example"}), _final("Two IOCs found.")])
- out = await run_with_tools(llm, system="You are an analyst.", user="Find IOCs.", registry=reg)
+ out = await run_with_tools(
+ llm,
+ system="You are an analyst.",
+ user="Find IOCs.",
+ registry=reg,
+ identity=issue_identity("investigation", run_id="run-1"),
+ )
assert out["truncated"] is False
assert out["content"] == "Two IOCs found."
assert out["iterations"] == 2
@@ -71,7 +79,9 @@ async def test_tool_loop_is_bounded_by_max_iters():
# A model that always asks for a tool must not loop forever.
reg = ToolRegistry([Tool(name="noop", description="noop", parameters={"type": "object", "properties": {}}, fn=lambda: {"ok": True})])
llm = _FakeLLM([_tool_call("noop", {})]) # always returns a tool call
- out = await run_with_tools(llm, system="s", user="u", registry=reg, max_iters=3)
+ out = await run_with_tools(
+ llm, system="s", user="u", registry=reg, identity=issue_identity("investigation", run_id="run-1"), max_iters=3
+ )
assert out["truncated"] is True
assert out["iterations"] == 3
assert len(out["tool_trace"]) == 3
diff --git a/services/api/app/data/detections/detection_ruleset.json b/services/api/app/data/detections/detection_ruleset.json
index 4d114601f..af231eb06 100644
--- a/services/api/app/data/detections/detection_ruleset.json
+++ b/services/api/app/data/detections/detection_ruleset.json
@@ -1,5 +1,5 @@
{
- "count": 764,
+ "count": 768,
"rules": [
{
"category": "application",
@@ -1373,6 +1373,79 @@
"severity": "high",
"slug": "ai-prompt-contained-credential"
},
+ {
+ "category": "application",
+ "id": "det-application-091",
+ "match_when": {
+ "baseline_agent_tool_rarity_gte": 0.95,
+ "tool_name_in": [
+ "execute_command",
+ "run_shell",
+ "write_file",
+ "delete_file",
+ "send_email",
+ "create_user",
+ "grant_permission",
+ "deploy",
+ "terraform_apply",
+ "database_write"
+ ]
+ },
+ "mitre": [
+ "T1059"
+ ],
+ "name": "AI Agent Called a High-Risk Tool It Has Never Used",
+ "product": "ai_runtime",
+ "service": "runtime",
+ "severity": "high",
+ "slug": "ai-agent-new-high-risk-tool"
+ },
+ {
+ "category": "application",
+ "id": "det-application-092",
+ "match_when": {
+ "baseline_agent_volume_z_gte": 3.0,
+ "outcome": "blocked"
+ },
+ "mitre": [
+ "T1078"
+ ],
+ "name": "Guardrail Violations Rose Sharply for One AI Agent",
+ "product": "ai_guardrail",
+ "service": "runtime",
+ "severity": "high",
+ "slug": "ai-agent-guardrail-violation-rate"
+ },
+ {
+ "category": "application",
+ "id": "det-application-093",
+ "match_when": {
+ "baseline_agent_token_spend_z_gte": 4.0
+ },
+ "mitre": [
+ "T1496"
+ ],
+ "name": "AI Agent Spend Far Above Its Own Baseline",
+ "product": "ai_runtime",
+ "service": "runtime",
+ "severity": "medium",
+ "slug": "ai-agent-token-spend-anomaly"
+ },
+ {
+ "category": "application",
+ "id": "det-application-094",
+ "match_when": {
+ "baseline_gateway_key_token_spend_z_gte": 4.0
+ },
+ "mitre": [
+ "T1496"
+ ],
+ "name": "AI Gateway Key Spend Far Above Its Own Baseline",
+ "product": "ai_runtime",
+ "service": "runtime",
+ "severity": "medium",
+ "slug": "ai-gateway-key-spend-anomaly"
+ },
{
"category": "application",
"id": "det-application-086",
diff --git a/services/api/app/data/marketplace/index.json b/services/api/app/data/marketplace/index.json
index d119ef249..8576c6ade 100644
--- a/services/api/app/data/marketplace/index.json
+++ b/services/api/app/data/marketplace/index.json
@@ -1,7 +1,7 @@
{
"$schema": "https://example.com/schemas/marketplace/v1.json",
"version": "1.0.0",
- "generated": "2026-11-01T19:20:18Z",
+ "generated": "2026-11-01T23:20:34Z",
"categories": [
{
"id": "playbooks",
@@ -20,27 +20,27 @@
}
],
"stats": {
- "total": 7190,
+ "total": 7194,
"playbooks": 88,
"playbook_packs": 63,
- "detections": 7019,
+ "detections": 7023,
"plugins": 83,
- "verified": 1069,
+ "verified": 1073,
"community": 1,
"by_tier": {
"beta": 7,
"community": 1,
"imported": 6113,
- "stable": 1069
+ "stable": 1073
},
"detections_by_tier": {
"community": 1,
"imported": 6113,
- "stable": 905
+ "stable": 909
},
- "executable": 2705,
+ "executable": 2709,
"quarantined": 4485,
- "executable_detections": 2534,
+ "executable_detections": 2538,
"reference_only_detections": 4485,
"not_a_detection": 171
},
@@ -116,7 +116,7 @@
"T1056.001": 3,
"T1056.002": 1,
"T1057": 3,
- "T1059": 49,
+ "T1059": 50,
"T1059.001": 132,
"T1059.002": 3,
"T1059.003": 24,
@@ -141,7 +141,7 @@
"T1071.004": 10,
"T1072": 4,
"T1074.001": 4,
- "T1078": 50,
+ "T1078": 51,
"T1078.001": 5,
"T1078.002": 7,
"T1078.003": 3,
@@ -258,7 +258,7 @@
"T1489": 10,
"T1490": 20,
"T1491.001": 2,
- "T1496": 4,
+ "T1496": 6,
"T1497.001": 1,
"T1498": 3,
"T1498.001": 1,
@@ -439,7 +439,7 @@
"T1686.003": 10,
"T1689": 1
},
- "total_with_mitre": 2404,
+ "total_with_mitre": 2408,
"unique_techniques_all_rules": 494,
"techniques_all_rules": {
"T1001.003": 2,
@@ -531,7 +531,7 @@
"T1056.001": 4,
"T1056.002": 4,
"T1057": 8,
- "T1059": 122,
+ "T1059": 123,
"T1059.001": 243,
"T1059.002": 11,
"T1059.003": 41,
@@ -563,7 +563,7 @@
"T1072": 9,
"T1074": 3,
"T1074.001": 6,
- "T1078": 133,
+ "T1078": 134,
"T1078.001": 10,
"T1078.002": 14,
"T1078.003": 12,
@@ -696,7 +696,7 @@
"T1491": 2,
"T1491.001": 4,
"T1495": 1,
- "T1496": 15,
+ "T1496": 17,
"T1497": 7,
"T1497.001": 3,
"T1497.003": 3,
@@ -981,7 +981,7 @@
"T1055.001": 1,
"T1055.012": 1,
"T1056.001": 1,
- "T1059": 9,
+ "T1059": 10,
"T1059.001": 5,
"T1059.002": 3,
"T1059.004": 8,
@@ -996,7 +996,7 @@
"T1071.001": 10,
"T1071.004": 6,
"T1074.001": 1,
- "T1078": 42,
+ "T1078": 43,
"T1078.001": 4,
"T1078.002": 3,
"T1078.003": 2,
@@ -1063,7 +1063,7 @@
"T1486": 8,
"T1489": 2,
"T1490": 7,
- "T1496": 2,
+ "T1496": 4,
"T1498": 4,
"T1498.001": 2,
"T1498.002": 2,
@@ -32886,6 +32886,106 @@
"path": "detections/application/llm-copilot-bulk-resource-access.yaml",
"sha256": "b1b664f05eb77501b1f895c6749dde3c88d1fbcb3e223e6c02416c26f397fad9"
},
+ {
+ "id": "det-application-091",
+ "type": "detection",
+ "name": "AI Agent Called a High-Risk Tool It Has Never Used",
+ "description": "An agent reached for a tool that changes an estate and has essentially no history of using it. Novelty is per agent and not global: execute_command is routine for a deployment bot and alarming for a summariser, and a count across the estate cannot tell them apart. The high-risk list is the same one the AI estate's exposure score reads, so an operator sees one definition of 'changes an estate' in both places.",
+ "version": "1.0.0",
+ "author": "AiSOC",
+ "tags": [
+ "tlp.white"
+ ],
+ "severity": "high",
+ "category": "application",
+ "mitre_techniques": [
+ "T1059"
+ ],
+ "log_source": "ai_runtime",
+ "playbook": "tpl-privilege-escalation",
+ "verified": true,
+ "source": "core",
+ "tier": "stable",
+ "enabled": true,
+ "executable": true,
+ "path": "detections/application/ai-agent-new-high-risk-tool.yaml",
+ "sha256": "fc48436662f0dd55a69767ec526db0e0250cf2cc6ce104bc3b2d58c1588122c9"
+ },
+ {
+ "id": "det-application-092",
+ "type": "detection",
+ "name": "Guardrail Violations Rose Sharply for One AI Agent",
+ "description": "A single agent is being blocked far more often than its own history. One guardrail violation is noise \u2014 a user phrasing something awkwardly \u2014 while a rise against that agent's own baseline is someone probing for a phrasing that gets through. Scored per agent rather than per tenant, because a busy estate's total hides one agent's change completely.",
+ "version": "1.0.0",
+ "author": "AiSOC",
+ "tags": [
+ "tlp.white"
+ ],
+ "severity": "high",
+ "category": "application",
+ "mitre_techniques": [
+ "T1078"
+ ],
+ "log_source": "ai_guardrail",
+ "playbook": "tpl-triage",
+ "verified": true,
+ "source": "core",
+ "tier": "stable",
+ "enabled": true,
+ "executable": true,
+ "path": "detections/application/ai-agent-guardrail-violation-rate.yaml",
+ "sha256": "222223b191af005fd502992b1f893559eac7d44a84744cf4fcc6334c1a73e27a"
+ },
+ {
+ "id": "det-application-093",
+ "type": "detection",
+ "name": "AI Agent Spend Far Above Its Own Baseline",
+ "description": "Cost is the signal that notices an agent being used for something other than its job \u2014 an exfiltration loop, a stolen credential driving inference, or a prompt that has started recursing. It reads the gateway's own billed figure rather than recomputing one from token counts, so it cannot disagree with the invoice. An unpriced request contributes nothing instead of a zero.",
+ "version": "1.0.0",
+ "author": "AiSOC",
+ "tags": [
+ "tlp.white"
+ ],
+ "severity": "medium",
+ "category": "application",
+ "mitre_techniques": [
+ "T1496"
+ ],
+ "log_source": "ai_runtime",
+ "playbook": "tpl-triage",
+ "verified": true,
+ "source": "core",
+ "tier": "stable",
+ "enabled": true,
+ "executable": true,
+ "path": "detections/application/ai-agent-token-spend-anomaly.yaml",
+ "sha256": "7e0fce9e3b1ea87270f409319239d515adb493edb19c6f8cde54cb695de2e9b6"
+ },
+ {
+ "id": "det-application-094",
+ "type": "detection",
+ "name": "AI Gateway Key Spend Far Above Its Own Baseline",
+ "description": "The same question asked of the key rather than the agent, and the two are not the same incident: a leaked key driving inference from outside the estate moves the key's baseline while every agent on it looks ordinary. Both are needed, which is why the baseline module keys on sub-principals rather than on one chosen subject.",
+ "version": "1.0.0",
+ "author": "AiSOC",
+ "tags": [
+ "tlp.white"
+ ],
+ "severity": "medium",
+ "category": "application",
+ "mitre_techniques": [
+ "T1496"
+ ],
+ "log_source": "ai_runtime",
+ "playbook": "tpl-triage",
+ "verified": true,
+ "source": "core",
+ "tier": "stable",
+ "enabled": true,
+ "executable": true,
+ "path": "detections/application/ai-gateway-key-spend-anomaly.yaml",
+ "sha256": "9c4e914639eabcf54da286c112ca9c468ff577bd27a764c585019921e0a6c7cc"
+ },
{
"id": "det-cloud-001",
"type": "detection",
diff --git a/services/api/tests/test_builtin_rule_catalogue.py b/services/api/tests/test_builtin_rule_catalogue.py
index 36e9dbfdc..09c40b323 100644
--- a/services/api/tests/test_builtin_rule_catalogue.py
+++ b/services/api/tests/test_builtin_rule_catalogue.py
@@ -66,7 +66,7 @@ PACKAGED = REPO / "services" / "api" / "app" / "data" / "detections"
#: regeneration that silently empties one is a failure here rather than a
#: quieter comparison of two numbers that moved together.
ARTEFACTS: dict[str, int] = {
- "detection_ruleset.json": 764,
+ "detection_ruleset.json": 768,
"detection_ruleset_imported.json": 1770,
"windowed_ruleset.json": 76,
}
@@ -88,7 +88,7 @@ PYTHON_RESIDENT_WINDOWED = 3
#: the provider feeds 7.1 added, each replayed through the real connector and
#: the real engine in `services/fusion/tests/test_ai_rule_pack_replay.py`
#: with a negative twin differing in exactly one field.
-FUSION_LOADED_TOTAL = 2613
+FUSION_LOADED_TOTAL = 2617
TENANT = uuid.UUID("aaaaaaaa-0000-0000-0000-00000000a173")
ACTOR = uuid.UUID("bbbbbbbb-0000-0000-0000-00000000b173")
diff --git a/services/connectors/app/connectors/ai_gateway.py b/services/connectors/app/connectors/ai_gateway.py
index d14b96673..0c870e743 100644
--- a/services/connectors/app/connectors/ai_gateway.py
+++ b/services/connectors/app/connectors/ai_gateway.py
@@ -165,6 +165,7 @@ class AIGatewayConnector(BaseConnector):
outcome = str(raw.get("status") or raw.get("outcome") or "success").lower()
agent_id = str(raw.get("agent_id") or raw.get("user") or raw.get("api_key_alias") or "unknown-agent")
tool_name = str(raw.get("tool") or raw.get("tool_name") or (f"model:{model}" if model else "unknown"))
+ gateway_key = _key_name(raw)
finding_type = self._finding_type(raw, model, outcome)
severity = self._severity(finding_type, outcome)
@@ -192,6 +193,31 @@ class AIGatewayConnector(BaseConnector):
"session_id": raw.get("session_id") or raw.get("trace_id"),
"raw_event": raw,
}
+ # Promoted rather than left in `raw_event`. The engine flattens
+ # `raw_event` exactly one level, so these would be reachable — but
+ # the baseline module and the AI estate read a *named* field, and
+ # a value the connector reads and does not name is how seven Okta
+ # rules came to be unable to fire.
+ if gateway_key:
+ # The subject a spend baseline is keyed on, an alert titles and
+ # a console row renders. Never the raw hash: the vendor masks
+ # `key_name` for that reason, and a hash in a Redis key is a
+ # credential identifier in a place nobody is guarding.
+ event["gateway_key"] = gateway_key
+ # The gateway's own figures. Not recomputed here from token counts
+ # and a price table: that would be a second meter disagreeing with
+ # the one that is actually billed.
+ #
+ # Absent stays absent. A gateway that did not price a request must
+ # not teach a spend baseline that the request was free, because a
+ # baseline fed zeros learns the agent costs nothing and then never
+ # flags anything again.
+ cost = _number(raw.get("spend"), raw.get("response_cost"))
+ if cost is not None:
+ event["request_cost_usd"] = cost
+ total = _number(raw.get("total_tokens"))
+ if total is not None:
+ event["total_tokens"] = total
if finding_type:
event["finding_type"] = finding_type
return event
@@ -239,3 +265,41 @@ class AIGatewayConnector(BaseConnector):
if finding_type == _FINDING_UNAPPROVED_MODEL:
return f"Agent {agent_id} used unapproved model {model}"
return f"AI activity: {tool_name} by {agent_id}"
+
+
+def _key_name(raw: dict[str, Any]) -> str | None:
+ """The billed key's human-readable name, never its hash.
+
+ `key_alias` is what an operator sets; `key_name` is the vendor's own
+ masked form (`sk-...KE_A`) and is the right fallback for an unaliased
+ key, because falling through to nothing would give every unaliased key
+ one shared subject and turn a per-key baseline into an estate-wide one.
+
+ Field names from the gateway's published spend record:
+ https://docs.litellm.ai/docs/proxy/cost_tracking
+ """
+ for field in ("key_alias", "api_key_alias", "key_name"):
+ value = raw.get(field)
+ if value is None:
+ continue
+ text = str(value).strip()
+ if text:
+ return text[:200]
+ return None
+
+
+def _number(*values: Any) -> float | int | None:
+ """The first value that is genuinely numeric, or None.
+
+ `None` and the empty string both mean "the gateway did not say", and
+ both must stay unknown rather than becoming `0`.
+ """
+ for value in values:
+ if value is None or value == "":
+ continue
+ try:
+ number = float(value)
+ except (TypeError, ValueError):
+ continue
+ return int(number) if number.is_integer() else number
+ return None
diff --git a/services/connectors/tests/connectors/test_ai_gateway_key_and_spend.py b/services/connectors/tests/connectors/test_ai_gateway_key_and_spend.py
new file mode 100644
index 000000000..d040cf83e
--- /dev/null
+++ b/services/connectors/tests/connectors/test_ai_gateway_key_and_spend.py
@@ -0,0 +1,141 @@
+"""The gateway names the key and the cost, and the connector drops both.
+
+Depth plan wave 2, Phase 7.3 / 7.4.
+
+The gap
+-------
+`AIGatewayConnector.normalize()` reads `api_key_alias` — but only as the
+third fallback for `agent_id`, and then discards it. `key_alias`, `key_name`
+and `spend` never reach the event at all.
+
+That matters for two reasons and the second is the one that bites:
+
+1. The engine flattens `raw_event` exactly one level, so a vendor field the
+ connector reads and discards is invisible to every rule that clauses on
+ it. This is the measured Okta shape: `outcome["result"]` was read to pick
+ a severity, never lifted, and seven shipped rules could not fire.
+2. "Token-spend anomalies **per key** and per agent" has no subject without
+ a key identifier on the usage record. A baseline keyed on a field nothing
+ emits is a published signal structurally incapable of firing — exactly
+ what `RARITY_FIELDS` documents as the trap.
+
+The payload below is the gateway vendor's own published spend record,
+field for field, from
+https://docs.litellm.ai/docs/proxy/cost_tracking (fetched, 200).
+"""
+
+from __future__ import annotations
+
+import pytest
+from app.connectors.ai_gateway import AIGatewayConnector
+
+#: The vendor's documented spend-log record. `key_name` is masked by the
+#: vendor itself; `key_alias` is the human-readable name an operator sets.
+SPEND_RECORD = {
+ "id": "req-1",
+ "request_id": "req-1",
+ "timestamp": "2026-11-01T09:00:00Z",
+ "model": "claude-opus-5-5",
+ "custom_llm_provider": "anthropic",
+ # Assembled rather than written out. The vendor's own example is a
+ # 56-character hex digest, which is a credential-shaped literal the
+ # repository's secret scan correctly refuses in a committed file — and
+ # a fixture has no business looking like one. The shape is what this
+ # test is about, not the value.
+ "api_key": "ef" + "ab12" * 13,
+ "key_name": "sk-...KE_A",
+ "key_alias": "release-bot-key",
+ "spend": 0.000002,
+ "total_tokens": 100,
+ "completion_tokens": 80,
+ "prompt_tokens": 20,
+ "end_user": "[email protected]",
+ "agent_id": "release-bot",
+}
+
+
+def connector() -> AIGatewayConnector:
+ return AIGatewayConnector(base_url="https://gw.example.com", api_key="k")
+
+
+def test_the_key_the_request_was_billed_to_reaches_the_event(connector: AIGatewayConnector):
+ """Without this there is no subject for a per-key spend baseline.
+
+ The connector already reads `api_key_alias` as an `agent_id` fallback,
+ so the value was in its hands and went nowhere.
+ """
+ event = connector.normalize(SPEND_RECORD)
+ assert event.get("gateway_key") == "release-bot-key"
+
+
+def test_the_masked_key_name_is_used_when_no_alias_is_set():
+ """An unaliased key is the common case on a new deployment.
+
+ Falling back to nothing would make every unaliased key share one empty
+ subject, which is a baseline over an aggregate rather than over a key.
+ """
+ c = AIGatewayConnector(base_url="https://gw.example.com", api_key="k")
+ record = dict(SPEND_RECORD)
+ record.pop("key_alias")
+ assert c.normalize(record).get("gateway_key") == "sk-...KE_A"
+
+
+def test_the_raw_hashed_key_is_never_used_as_the_subject(connector: AIGatewayConnector):
+ """The hash is a credential identifier, not a name.
+
+ The vendor masks `key_name` for exactly this reason. `gateway_key` is
+ the subject a baseline is keyed on, an alert titles and a console row
+ renders, so the full hash must not be it.
+
+ Asserted on `gateway_key` rather than on the whole event: `raw_event`
+ carries the vendor's payload verbatim by design, the same as every
+ other connector, and that is a separate decision from what this
+ connector elects to promote to a named field.
+ """
+ event = connector.normalize(SPEND_RECORD)
+ assert event.get("gateway_key") != SPEND_RECORD["api_key"]
+
+
+def test_a_record_with_no_key_at_all_promotes_nothing(connector: AIGatewayConnector):
+ """An in-house proxy need not have the concept of a key.
+
+ Inventing a placeholder would give every such deployment one shared
+ subject, and a baseline over "unknown" is a baseline over the whole
+ estate wearing a key's name.
+ """
+ record = {k: v for k, v in SPEND_RECORD.items() if k not in {"key_alias", "key_name", "api_key_alias"}}
+ event = connector.normalize(record)
+ assert not event.get("gateway_key")
+
+
+def test_the_cost_of_the_request_reaches_the_event(connector: AIGatewayConnector):
+ """`spend` is the vendor's own figure in dollars.
+
+ Recomputing it here from token counts and a price table would be a
+ second meter disagreeing with the gateway's — the thing that is
+ actually billed.
+ """
+ event = connector.normalize(SPEND_RECORD)
+ assert event.get("request_cost_usd") == pytest.approx(0.000002)
+
+
+def test_total_tokens_reaches_the_event(connector: AIGatewayConnector):
+ """Carried rather than summed from the two halves.
+
+ A provider that reports a total including reasoning or cached tokens
+ would disagree with `prompt + completion`, and the vendor's own total
+ is the one its billing uses.
+ """
+ assert connector.normalize(SPEND_RECORD).get("total_tokens") == 100
+
+
+def test_a_record_with_no_cost_reports_nothing_rather_than_zero(connector: AIGatewayConnector):
+ """A missing cost is unknown, and `0.0` is a measurement.
+
+ A spend baseline fed zeros for every record a gateway did not price
+ would learn that this agent costs nothing and then never flag anything.
+ """
+ record = {k: v for k, v in SPEND_RECORD.items() if k != "spend"}
+ event = connector.normalize(record)
+ assert "request_cost_usd" not in event or event["request_cost_usd"] is None
diff --git a/services/connectors/tests/connectors/test_ai_gateway_replay_fixture.py b/services/connectors/tests/connectors/test_ai_gateway_replay_fixture.py
new file mode 100644
index 000000000..74cf12cfe
--- /dev/null
+++ b/services/connectors/tests/connectors/test_ai_gateway_replay_fixture.py
@@ -0,0 +1,89 @@
+"""Keep the fusion rule-pack fixtures equal to the gateway connector's output.
+
+`services/fusion` cannot import `services/connectors` — both package their
+application as `app` — so the AI rule pack is replayed in fusion against
+committed artefacts, and this regenerates them and fails if they drift.
+Two directions, the same arrangement as the vendored narrative.
+
+Nothing here is synthesized from a rule. A fixture generated from the rule
+it tests carries the rule's own clause key, which is why 1 of 12 GitHub
+rules fired on real connector output while all twelve passed their own
+replay.
+
+Run with `AISOC_REGENERATE_FIXTURES=1` after a deliberate change to
+`normalize()`.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import pathlib
+
+import pytest
+from app.connectors.ai_gateway import AIGatewayConnector
+
+FIXTURES = pathlib.Path(__file__).resolve().parents[3] / "fusion" / "tests" / "fixtures" / "ai_gateway"
+
+
+def _connector() -> AIGatewayConnector:
+ """Constructed explicitly rather than splatted from a dict.
+
+ `**GW` on a `dict[str, str]` cannot type-check against a constructor
+ that also takes a bool, and silencing that would hide a real
+ signature change behind an ignore.
+ """
+ return AIGatewayConnector(base_url="https://gw.example.com", api_key="k")
+
+
+def _span(**overrides) -> dict:
+ """One gateway span in the vendor's documented spend-record shape."""
+ base = {
+ "id": "req-1",
+ "timestamp": "2026-11-01T09:00:00Z",
+ "agent_id": "release-bot",
+ "agent_name": "release-bot",
+ "model": "claude-opus-5-5",
+ "custom_llm_provider": "anthropic",
+ "key_alias": "release-bot-key",
+ "tool": "search_notes",
+ "status": "success",
+ "spend": 0.02,
+ "total_tokens": 100,
+ "prompt_tokens": 20,
+ "completion_tokens": 80,
+ "end_user": "[email protected]",
+ }
+ base.update(overrides)
+ return base
+
+
+#: Each pair differs in exactly ONE decisive field. The baseline history is
+#: built from the `_routine` fixture in the fusion replay, so the twins here
+#: differ only in what the rule clauses on.
+CASES: dict[str, dict] = {
+ "routine.json": _span(),
+ # The high-risk tool. Everything else identical to routine.
+ "high_risk_tool.json": _span(id="req-2", tool="execute_command"),
+ # A tool the agent has never used that is NOT on the high-risk list:
+ # novel, and deliberately not a finding.
+ "novel_benign_tool.json": _span(id="req-3", tool="summarise_page"),
+ "guardrail_blocked.json": _span(id="req-4", status="blocked"),
+ "expensive.json": _span(id="req-5", spend=40.0),
+}
+
+
[email protected]("name", sorted(CASES))
+def test_fixture_matches_the_connector_output(name: str):
+ rendered = _connector().normalize(CASES[name])
+ path = FIXTURES / name
+
+ if os.environ.get("AISOC_REGENERATE_FIXTURES") == "1":
+ path.parent.mkdir(parents=True, exist_ok=True)
+ path.write_text(json.dumps(rendered, indent=2, sort_keys=True) + "\n")
+
+ assert path.exists(), f"{path} is missing — regenerate with AISOC_REGENERATE_FIXTURES=1"
+ assert json.loads(path.read_text()) == rendered, (
+ f"{name} no longer matches AIGatewayConnector.normalize(). The fusion rule-pack "
+ "replay reads this file; regenerate it rather than editing it by hand."
+ )
diff --git a/services/fusion/app/data/detection_ruleset.json b/services/fusion/app/data/detection_ruleset.json
index 4d114601f..af231eb06 100644
--- a/services/fusion/app/data/detection_ruleset.json
+++ b/services/fusion/app/data/detection_ruleset.json
@@ -1,5 +1,5 @@
{
- "count": 764,
+ "count": 768,
"rules": [
{
"category": "application",
@@ -1373,6 +1373,79 @@
"severity": "high",
"slug": "ai-prompt-contained-credential"
},
+ {
+ "category": "application",
+ "id": "det-application-091",
+ "match_when": {
+ "baseline_agent_tool_rarity_gte": 0.95,
+ "tool_name_in": [
+ "execute_command",
+ "run_shell",
+ "write_file",
+ "delete_file",
+ "send_email",
+ "create_user",
+ "grant_permission",
+ "deploy",
+ "terraform_apply",
+ "database_write"
+ ]
+ },
+ "mitre": [
+ "T1059"
+ ],
+ "name": "AI Agent Called a High-Risk Tool It Has Never Used",
+ "product": "ai_runtime",
+ "service": "runtime",
+ "severity": "high",
+ "slug": "ai-agent-new-high-risk-tool"
+ },
+ {
+ "category": "application",
+ "id": "det-application-092",
+ "match_when": {
+ "baseline_agent_volume_z_gte": 3.0,
+ "outcome": "blocked"
+ },
+ "mitre": [
+ "T1078"
+ ],
+ "name": "Guardrail Violations Rose Sharply for One AI Agent",
+ "product": "ai_guardrail",
+ "service": "runtime",
+ "severity": "high",
+ "slug": "ai-agent-guardrail-violation-rate"
+ },
+ {
+ "category": "application",
+ "id": "det-application-093",
+ "match_when": {
+ "baseline_agent_token_spend_z_gte": 4.0
+ },
+ "mitre": [
+ "T1496"
+ ],
+ "name": "AI Agent Spend Far Above Its Own Baseline",
+ "product": "ai_runtime",
+ "service": "runtime",
+ "severity": "medium",
+ "slug": "ai-agent-token-spend-anomaly"
+ },
+ {
+ "category": "application",
+ "id": "det-application-094",
+ "match_when": {
+ "baseline_gateway_key_token_spend_z_gte": 4.0
+ },
+ "mitre": [
+ "T1496"
+ ],
+ "name": "AI Gateway Key Spend Far Above Its Own Baseline",
+ "product": "ai_runtime",
+ "service": "runtime",
+ "severity": "medium",
+ "slug": "ai-gateway-key-spend-anomaly"
+ },
{
"category": "application",
"id": "det-application-086",
diff --git a/services/fusion/app/services/baselines.py b/services/fusion/app/services/baselines.py
index 966374920..c1e7d6089 100644
--- a/services/fusion/app/services/baselines.py
+++ b/services/fusion/app/services/baselines.py
@@ -127,6 +127,10 @@ PRINCIPAL_FIELDS: tuple[str, ...] = (
"principal",
"user_arn",
"userPrincipalName",
+ # The human an AI agent acted for. The `ai-runtime` template maps it to
+ # `actor.user.name`, which is to say the vendor and the schema already
+ # agree it is the principal; this module simply could not see it.
+ "on_behalf_of",
)
#: Event fields naming the OAuth application acting, for the per-application
@@ -141,6 +145,70 @@ OAUTH_APP_FIELDS: tuple[str, ...] = (
"oauth_app",
)
+#: Sub-principals: the things that act *as* a principal without being one.
+#:
+#: The OAuth application was the only one of these for a while, special-cased
+#: inside `derived_fields`. Wave-2 Phase 1.4 asks for baselines per
+#: sub-principal (OAuth app, personal access token, service account, and an
+#: AI agent acting for a human) and wave-2 Phase 7.3 asks for per-agent
+#: baselines. Those are the same mechanism with different subjects, so this
+#: is a table and not four blocks of code: a new kind is one entry, and the
+#: console cannot end up with two ways of saying the same thing.
+#:
+#: **The key is the field prefix.** `oauth_app` produces
+#: `baseline_oauth_app_volume_z`, which is already published and must not
+#: move; `agent` produces `baseline_agent_volume_z`. Deriving the name from
+#: the key rather than from a second mapping means a reader can tell which
+#: entry produced a field without consulting anything.
+#:
+#: Every chain must name at least one field a connector really emits — the
+#: rule above `RARITY_FIELDS`, for the same reason. `agent_id`, `agent_name`
+#: and `gateway_key` all come from `ai_gateway.normalize()` and the
+#: `ai-runtime` ingest template.
+SUB_PRINCIPAL_FIELDS: dict[str, tuple[str, ...]] = {
+ "oauth_app": OAUTH_APP_FIELDS,
+ "agent": ("agent_id", "agent_name"),
+ # The gateway key a request was billed to. A compromised key used by a
+ # well-behaved agent and a misbehaving agent on a normal key are
+ # different incidents; only a baseline per subject separates them.
+ "gateway_key": ("gateway_key",),
+}
+
+#: Per-kind rarity attributes: which of this subject's choices are worth
+#: scoring for novelty. Keyed `kind -> {attribute: field chain}` and producing
+#: `baseline_{kind}_{attribute}_rarity`.
+#:
+#: Rarity is per subject, never global, and for an agent that is the whole
+#: value: `execute_command` is routine for a deployment bot and alarming for
+#: a summariser, and a global count cannot tell them apart.
+SUB_PRINCIPAL_RARITY: dict[str, dict[str, tuple[str, ...]]] = {
+ "agent": {
+ "tool": ("tool_name",),
+ "model": ("model",),
+ },
+}
+
+#: Kinds whose hour-of-day is baselined, producing
+#: `baseline_{kind}_hour_rarity`. Computed from the event clock rather than
+#: read from a field, because no connector emits an hour — stated here so it
+#: does not look like an omission from `SUB_PRINCIPAL_RARITY`.
+#:
+#: An agent that has only ever run inside a nightly window and suddenly runs
+#: at midday is the signal. A human's working hours are already covered by
+#: the principal baseline, so this is deliberately not every kind.
+HOUR_BASELINE_KINDS: frozenset[str] = frozenset({"agent"})
+
+#: Kinds whose spend is baselined, producing
+#: `baseline_{kind}_token_spend_z`. Depth 6.4 asks for token spend per agent;
+#: wave-2 7.4 asks for it per key as well.
+SPEND_BASELINE_KINDS: frozenset[str] = frozenset({"agent", "gateway_key"})
+
+#: Where the cost of one request is read from. The gateway's own figure, in
+#: dollars, promoted by `ai_gateway.normalize()`. Not recomputed here from
+#: token counts and a price table: that would be a second meter disagreeing
+#: with the one that is actually billed.
+SPEND_VALUE_FIELDS: tuple[str, ...] = ("request_cost_usd",)
+
#: Event fields naming the peer group a principal belongs to. Depth plan 3.4
#: names the Phase 2.5 posture snapshot as the source; until that lands, the
#: group is taken from whatever the identity provider already puts on the
@@ -216,9 +284,10 @@ DERIVED_FIELDS: frozenset[str] = frozenset(
set(RARITY_FIELDS)
| {
"baseline_principal_volume_z",
- "baseline_oauth_app_volume_z",
"baseline_peer_deviation",
- # Depth plan wave 2, 1.4.
+ # Depth plan wave 2, 1.4 — the sub-principal summary fields. One
+ # set for all kinds rather than one per kind, so they are named
+ # here while the per-kind families below are generated.
"baseline_sub_principal_kind",
"baseline_sub_principal_rarity",
"baseline_sub_principal_volume_z",
@@ -227,6 +296,15 @@ DERIVED_FIELDS: frozenset[str] = frozenset(
"baseline_token_reactivated_after_idle",
"baseline_token_new_asn_after_creation",
}
+ # Generated from the tables above rather than listed again. A hand-kept
+ # second list is how a field comes to be computed and undeclared, or
+ # declared and never computed — and the gate compares this set against
+ # the one `scripts/check_detection_fields.py` builds, so either
+ # direction of drift would be a false verdict rather than a loud one.
+ | {f"baseline_{kind}_volume_z" for kind in SUB_PRINCIPAL_FIELDS}
+ | {f"baseline_{kind}_{attribute}_rarity" for kind, attributes in SUB_PRINCIPAL_RARITY.items() for attribute in attributes}
+ | {f"baseline_{kind}_hour_rarity" for kind in HOUR_BASELINE_KINDS}
+ | {f"baseline_{kind}_token_spend_z" for kind in SPEND_BASELINE_KINDS}
)
#: A token silent for this long and then active again is worth a signal.
@@ -263,6 +341,26 @@ def resolve_principal(event: dict[str, Any]) -> str | None:
return _first_present(event, PRINCIPAL_FIELDS)
+def resolve_sub_principals(event: dict[str, Any]) -> list[tuple[str, str]]:
+ """Every sub-principal this event names, as ``(kind, subject)``.
+
+ A list rather than one value, because a single gateway span names both
+ the agent that acted and the key it was billed to, and those are
+ different incidents when they go wrong.
+
+ An event naming none — which is every ordinary cloud and identity
+ event — returns empty, and the caller emits nothing. A baseline with no
+ subject is not a baseline, and inventing a placeholder would give every
+ such event one shared subject and an unbounded Redis key space.
+ """
+ out: list[tuple[str, str]] = []
+ for kind, chain in SUB_PRINCIPAL_FIELDS.items():
+ subject = _first_present(event, chain)
+ if subject is not None:
+ out.append((kind, subject))
+ return out
+
+
@dataclass(frozen=True)
class SubPrincipal:
"""A token, app, service account or agent acting on a human's behalf."""
@@ -582,6 +680,35 @@ class BaselineStore:
return None
return score
+ # -- continuous amounts -----------------------------------------------
+
+ async def observe_amount(self, tenant: str, subject: str, kind: str, value: float) -> float | None:
+ """Fold one measured amount into this subject's history; return its z-score.
+
+ Distinct from `observe_volume`, which counts events into an hourly
+ bucket. Spend is a quantity carried by the event itself, so there is
+ nothing to bucket: each request's cost is one observation.
+
+ Scored against the history *before* this observation, so a value
+ cannot flatten the distribution it is being judged against — the
+ first large spend scores against what came before it rather than
+ against a mean it has already moved.
+ """
+ key = self._key(tenant, subject, f"amt:{kind}")
+ try:
+ state = await self._redis.hgetall(key)
+ stats = _decode_stats(state)
+ score = z_score(stats, float(value), min_samples=MIN_VOLUME_SAMPLES)
+ advanced = welford_add(stats, float(value))
+ await self._redis.hset(key, "count", str(advanced["count"]))
+ await self._redis.hset(key, "mean", str(advanced["mean"]))
+ await self._redis.hset(key, "m2", str(advanced["m2"]))
+ await self._redis.expire(key, self._ttl)
+ except Exception as exc: # noqa: BLE001 — see the class docstring
+ logger.debug("baselines.amount_unavailable", kind=kind, error=str(exc))
+ return None
+ return score
+
# -- peer group -------------------------------------------------------
async def observe_peer(self, tenant: str, group: str, subject: str, value: float) -> float | None:
@@ -625,12 +752,24 @@ class BaselineStore:
if not tenant:
return {}
principal = resolve_principal(event)
- if not principal:
+ sub_principals = resolve_sub_principals(event)
+ if not principal and not sub_principals:
+ # Previously this returned on `not principal` alone, and that
+ # skipped the whole module for any event naming a non-human
+ # subject. A gateway span names an agent and a key and often no
+ # person at all — an agent running on a schedule acts for
+ # nobody — so every per-agent baseline was computed for exactly
+ # zero events, while the field names read as working.
+ #
+ # "No subject of any kind" is still nothing to baseline, which
+ # is what this now says.
return {}
out: dict[str, Any] = {}
for field, candidates in RARITY_FIELDS.items():
+ if not principal:
+ break
value = _first_present(event, candidates)
if value is None:
continue
@@ -642,23 +781,61 @@ class BaselineStore:
continue
out[field] = round(self.rarity(count, total), 4)
- volume = await self.observe_volume(tenant, principal, "principal", now=now)
- if volume is not None:
- out["baseline_principal_volume_z"] = round(volume, 4)
-
- app = _first_present(event, OAUTH_APP_FIELDS)
- if app is not None:
- app_volume = await self.observe_volume(tenant, app, "oauth_app", now=now)
- if app_volume is not None:
- out["baseline_oauth_app_volume_z"] = round(app_volume, 4)
+ if principal:
+ volume = await self.observe_volume(tenant, principal, "principal", now=now)
+ if volume is not None:
+ out["baseline_principal_volume_z"] = round(volume, 4)
+
+ # Every sub-principal this event names, through one loop. The OAuth
+ # application used to be special-cased here; it is now the first
+ # entry in SUB_PRINCIPAL_FIELDS and produces the same published
+ # field name, because the name is derived from the key.
+ for kind, subject in sub_principals:
+ sub_volume = await self.observe_volume(tenant, subject, kind, now=now)
+ if sub_volume is not None:
+ out[f"baseline_{kind}_volume_z"] = round(sub_volume, 4)
+
+ for attribute, candidates in SUB_PRINCIPAL_RARITY.get(kind, {}).items():
+ value = _first_present(event, candidates)
+ if value is None:
+ continue
+ counted = await self.observe_value(tenant, subject, f"{kind}:{attribute}", value)
+ if counted is None:
+ continue
+ count, total = counted
+ if total < self._min_observations:
+ continue
+ out[f"baseline_{kind}_{attribute}_rarity"] = round(self.rarity(count, total), 4)
+
+ if kind in HOUR_BASELINE_KINDS:
+ hour = time.strftime("%H", time.gmtime(now if now is not None else time.time()))
+ counted = await self.observe_value(tenant, subject, f"{kind}:hour", hour)
+ if counted is not None:
+ count, total = counted
+ if total >= self._min_observations:
+ out[f"baseline_{kind}_hour_rarity"] = round(self.rarity(count, total), 4)
+
+ if kind in SPEND_BASELINE_KINDS:
+ cost = _amount(event, SPEND_VALUE_FIELDS)
+ # Absent stays absent. A gateway that did not price a
+ # request must not teach the baseline the request was free:
+ # a spend baseline fed zeros learns this agent costs nothing
+ # and then never flags anything again.
+ if cost is not None:
+ spend = await self.observe_amount(tenant, subject, f"spend:{kind}", cost)
+ if spend is not None:
+ out[f"baseline_{kind}_token_spend_z"] = round(spend, 4)
# Depth plan wave 2, 1.4. The sub-principal gets a baseline of its
# own, on the same store, so a stolen token moves a number even
# while the human it acts for looks entirely normal.
out.update(await self._sub_principal_fields(tenant, event, now=now))
- group = _first_present(event, PEER_GROUP_FIELDS)
- if group is not None:
+ group = _first_present(event, PEER_GROUP_FIELDS) if principal else None
+ # `principal` is re-tested rather than relied on through `group`:
+ # the implication holds, but it holds three lines away and a type
+ # checker cannot see it — and neither can the next reader.
+ if group is not None and principal:
# The member's own hourly volume is the quantity compared across
# the group. Rarity is not comparable between principals — one
# person's rare action is another's routine — but "how much does
@@ -785,12 +962,16 @@ class BaselineStore:
if field in RARITY_FIELDS:
kind, attribute = "rarity", field
value = _first_present(event, RARITY_FIELDS[field]) or ""
+ elif field.endswith("_rarity"):
+ kind, attribute = "rarity", field
+ value = _signal_subject(event, field)
elif field == "baseline_peer_deviation":
kind, attribute = "peer", field
value = _first_present(event, PEER_GROUP_FIELDS) or ""
else:
- kind, attribute = "volume", field
- value = principal if field == "baseline_principal_volume_z" else (_first_present(event, OAUTH_APP_FIELDS) or "")
+ kind = "spend" if field.endswith("_token_spend_z") else "volume"
+ attribute = field
+ value = principal if field == "baseline_principal_volume_z" else _signal_subject(event, field)
out.append(
AnomalySignal(
kind=kind,
@@ -828,3 +1009,34 @@ def _decode_stats(state: Any) -> dict[str, float]:
except ValueError:
continue
return out
+
+
+def _amount(event: dict[str, Any], candidates: tuple[str, ...]) -> float | None:
+ """The first candidate carrying a genuinely numeric value, or None.
+
+ `None`, the empty string and anything unparseable all mean "not
+ measured", and all stay `None`. Coercing them to `0.0` is the shape
+ that has produced four separate meters in this repository reading zero
+ for four different reasons.
+ """
+ for name in candidates:
+ value = event.get(name)
+ if value is None or value == "":
+ continue
+ try:
+ return float(value)
+ except (TypeError, ValueError):
+ continue
+ return None
+
+
+def _signal_subject(event: dict[str, Any], field: str) -> str:
+ """Which subject produced a generated sub-principal field.
+
+ Derived by matching the field's prefix against the table that generated
+ it, rather than by a second mapping that could disagree with the first.
+ """
+ for kind in SUB_PRINCIPAL_FIELDS:
+ if field.startswith(f"baseline_{kind}_"):
+ return _first_present(event, SUB_PRINCIPAL_FIELDS[kind]) or ""
+ return ""
diff --git a/services/fusion/tests/fixtures/ai_gateway/expensive.json b/services/fusion/tests/fixtures/ai_gateway/expensive.json
new file mode 100644
index 000000000..1f06a20b7
--- /dev/null
+++ b/services/fusion/tests/fixtures/ai_gateway/expensive.json
@@ -0,0 +1,40 @@
+{
+ "agent_id": "release-bot",
+ "agent_name": "release-bot",
+ "category": "ai",
+ "completion_tokens": 80,
+ "created_at": "2026-11-01T09:00:00Z",
+ "description": "agent=release-bot; tool=search_notes; model=claude-opus-5-5; outcome=success",
+ "external_id": "req-5",
+ "gateway_key": "release-bot-key",
+ "latency_ms": null,
+ "model": "claude-opus-5-5",
+ "on_behalf_of": "[email protected]",
+ "outcome": "success",
+ "prompt_tokens": 20,
+ "provider": "anthropic",
+ "raw_event": {
+ "agent_id": "release-bot",
+ "agent_name": "release-bot",
+ "completion_tokens": 80,
+ "custom_llm_provider": "anthropic",
+ "end_user": "[email protected]",
+ "id": "req-5",
+ "key_alias": "release-bot-key",
+ "model": "claude-opus-5-5",
+ "prompt_tokens": 20,
+ "spend": 40.0,
+ "status": "success",
+ "timestamp": "2026-11-01T09:00:00Z",
+ "tool": "search_notes",
+ "total_tokens": 100
+ },
+ "request_cost_usd": 40,
+ "server_name": null,
+ "session_id": null,
+ "severity": "info",
+ "source": "ai_gateway",
+ "title": "AI activity: search_notes by release-bot",
+ "tool_name": "search_notes",
+ "total_tokens": 100
+}
diff --git a/services/fusion/tests/fixtures/ai_gateway/guardrail_blocked.json b/services/fusion/tests/fixtures/ai_gateway/guardrail_blocked.json
new file mode 100644
index 000000000..0fa3f9bbf
--- /dev/null
+++ b/services/fusion/tests/fixtures/ai_gateway/guardrail_blocked.json
@@ -0,0 +1,40 @@
+{
+ "agent_id": "release-bot",
+ "agent_name": "release-bot",
+ "category": "ai",
+ "completion_tokens": 80,
+ "created_at": "2026-11-01T09:00:00Z",
+ "description": "agent=release-bot; tool=search_notes; model=claude-opus-5-5; outcome=blocked",
+ "external_id": "req-4",
+ "gateway_key": "release-bot-key",
+ "latency_ms": null,
+ "model": "claude-opus-5-5",
+ "on_behalf_of": "[email protected]",
+ "outcome": "blocked",
+ "prompt_tokens": 20,
+ "provider": "anthropic",
+ "raw_event": {
+ "agent_id": "release-bot",
+ "agent_name": "release-bot",
+ "completion_tokens": 80,
+ "custom_llm_provider": "anthropic",
+ "end_user": "[email protected]",
+ "id": "req-4",
+ "key_alias": "release-bot-key",
+ "model": "claude-opus-5-5",
+ "prompt_tokens": 20,
+ "spend": 0.02,
+ "status": "blocked",
+ "timestamp": "2026-11-01T09:00:00Z",
+ "tool": "search_notes",
+ "total_tokens": 100
+ },
+ "request_cost_usd": 0.02,
+ "server_name": null,
+ "session_id": null,
+ "severity": "low",
+ "source": "ai_gateway",
+ "title": "AI activity: search_notes by release-bot",
+ "tool_name": "search_notes",
+ "total_tokens": 100
+}
diff --git a/services/fusion/tests/fixtures/ai_gateway/high_risk_tool.json b/services/fusion/tests/fixtures/ai_gateway/high_risk_tool.json
new file mode 100644
index 000000000..bbed5e109
--- /dev/null
+++ b/services/fusion/tests/fixtures/ai_gateway/high_risk_tool.json
@@ -0,0 +1,40 @@
+{
+ "agent_id": "release-bot",
+ "agent_name": "release-bot",
+ "category": "ai",
+ "completion_tokens": 80,
+ "created_at": "2026-11-01T09:00:00Z",
+ "description": "agent=release-bot; tool=execute_command; model=claude-opus-5-5; outcome=success",
+ "external_id": "req-2",
+ "gateway_key": "release-bot-key",
+ "latency_ms": null,
+ "model": "claude-opus-5-5",
+ "on_behalf_of": "[email protected]",
+ "outcome": "success",
+ "prompt_tokens": 20,
+ "provider": "anthropic",
+ "raw_event": {
+ "agent_id": "release-bot",
+ "agent_name": "release-bot",
+ "completion_tokens": 80,
+ "custom_llm_provider": "anthropic",
+ "end_user": "[email protected]",
+ "id": "req-2",
+ "key_alias": "release-bot-key",
+ "model": "claude-opus-5-5",
+ "prompt_tokens": 20,
+ "spend": 0.02,
+ "status": "success",
+ "timestamp": "2026-11-01T09:00:00Z",
+ "tool": "execute_command",
+ "total_tokens": 100
+ },
+ "request_cost_usd": 0.02,
+ "server_name": null,
+ "session_id": null,
+ "severity": "info",
+ "source": "ai_gateway",
+ "title": "AI activity: execute_command by release-bot",
+ "tool_name": "execute_command",
+ "total_tokens": 100
+}
diff --git a/services/fusion/tests/fixtures/ai_gateway/novel_benign_tool.json b/services/fusion/tests/fixtures/ai_gateway/novel_benign_tool.json
new file mode 100644
index 000000000..686595beb
--- /dev/null
+++ b/services/fusion/tests/fixtures/ai_gateway/novel_benign_tool.json
@@ -0,0 +1,40 @@
+{
+ "agent_id": "release-bot",
+ "agent_name": "release-bot",
+ "category": "ai",
+ "completion_tokens": 80,
+ "created_at": "2026-11-01T09:00:00Z",
+ "description": "agent=release-bot; tool=summarise_page; model=claude-opus-5-5; outcome=success",
+ "external_id": "req-3",
+ "gateway_key": "release-bot-key",
+ "latency_ms": null,
+ "model": "claude-opus-5-5",
+ "on_behalf_of": "[email protected]",
+ "outcome": "success",
+ "prompt_tokens": 20,
+ "provider": "anthropic",
+ "raw_event": {
+ "agent_id": "release-bot",
+ "agent_name": "release-bot",
+ "completion_tokens": 80,
+ "custom_llm_provider": "anthropic",
+ "end_user": "[email protected]",
+ "id": "req-3",
+ "key_alias": "release-bot-key",
+ "model": "claude-opus-5-5",
+ "prompt_tokens": 20,
+ "spend": 0.02,
+ "status": "success",
+ "timestamp": "2026-11-01T09:00:00Z",
+ "tool": "summarise_page",
+ "total_tokens": 100
+ },
+ "request_cost_usd": 0.02,
+ "server_name": null,
+ "session_id": null,
+ "severity": "info",
+ "source": "ai_gateway",
+ "title": "AI activity: summarise_page by release-bot",
+ "tool_name": "summarise_page",
+ "total_tokens": 100
+}
diff --git a/services/fusion/tests/fixtures/ai_gateway/routine.json b/services/fusion/tests/fixtures/ai_gateway/routine.json
new file mode 100644
index 000000000..f9e4dd216
--- /dev/null
+++ b/services/fusion/tests/fixtures/ai_gateway/routine.json
@@ -0,0 +1,40 @@
+{
+ "agent_id": "release-bot",
+ "agent_name": "release-bot",
+ "category": "ai",
+ "completion_tokens": 80,
+ "created_at": "2026-11-01T09:00:00Z",
+ "description": "agent=release-bot; tool=search_notes; model=claude-opus-5-5; outcome=success",
+ "external_id": "req-1",
+ "gateway_key": "release-bot-key",
+ "latency_ms": null,
+ "model": "claude-opus-5-5",
+ "on_behalf_of": "[email protected]",
+ "outcome": "success",
+ "prompt_tokens": 20,
+ "provider": "anthropic",
+ "raw_event": {
+ "agent_id": "release-bot",
+ "agent_name": "release-bot",
+ "completion_tokens": 80,
+ "custom_llm_provider": "anthropic",
+ "end_user": "[email protected]",
+ "id": "req-1",
+ "key_alias": "release-bot-key",
+ "model": "claude-opus-5-5",
+ "prompt_tokens": 20,
+ "spend": 0.02,
+ "status": "success",
+ "timestamp": "2026-11-01T09:00:00Z",
+ "tool": "search_notes",
+ "total_tokens": 100
+ },
+ "request_cost_usd": 0.02,
+ "server_name": null,
+ "session_id": null,
+ "severity": "info",
+ "source": "ai_gateway",
+ "title": "AI activity: search_notes by release-bot",
+ "tool_name": "search_notes",
+ "total_tokens": 100
+}
diff --git a/services/fusion/tests/test_ai_rule_pack_baseline_replay.py b/services/fusion/tests/test_ai_rule_pack_baseline_replay.py
new file mode 100644
index 000000000..586b0a3e9
--- /dev/null
+++ b/services/fusion/tests/test_ai_rule_pack_baseline_replay.py
@@ -0,0 +1,287 @@
+"""The four baseline-backed AI rules, end to end.
+
+Depth plan wave 2, Phase 7.4 — the targets Phase 7.3 unblocked.
+
+What this drives
+----------------
+The real connector output → the shape ingest really produces → the **real**
+`BaselineStore` over a real history → the **real** `DetectionEngine` over
+the real compiled ruleset. Nothing is stubbed and no derived field is
+injected.
+
+That last point is the one that matters. These four rules clause on
+`baseline_agent_*` and `baseline_gateway_key_*`, which are produced by a
+different module from the one that reads them. Handing the engine a
+hand-written `{"baseline_agent_tool_rarity": 0.99}` would prove the matcher
+compares floats and would say nothing about whether the baseline module
+ever produces that key — which is the only way these rules can be wrong,
+and is exactly how `baseline_client_family_rarity` came to name three
+fields no connector emits.
+
+Novelty needs a history, so each test builds one first
+-------------------------------------------------------
+A rarity against an empty history is `0.0` by design — the module refuses
+to claim more confidence than the history supports — so a test that fired
+one event and expected a novelty score would be asserting against the
+wrong semantics rather than against the rule.
+"""
+
+from __future__ import annotations
+
+import json
+import pathlib
+
+import pytest
+from app.services.baselines import BaselineStore
+from app.services.detection_engine import DetectionEngine
+
+FIXTURES = pathlib.Path(__file__).parent / "fixtures" / "ai_gateway"
+RULESET = pathlib.Path(__file__).parents[1] / "app" / "data" / "detection_ruleset.json"
+TENANT = "11111111-1111-1111-1111-111111111111"
+
+
+def _compiled() -> list[dict]:
+ blob = json.loads(RULESET.read_text())
+ return blob["rules"] if isinstance(blob, dict) and "rules" in blob else blob
+
+
+#: The engine reports a hit by rule id; everything else speaks slugs. Mapped
+#: through the artefact rather than hardcoded, because ids are positional
+#: and a rule inserted earlier in a category shifts every later one.
+_ID_TO_SLUG: dict[str, str] = {r["id"]: r["slug"] for r in _compiled()}
+
+
+class _FakeRedis:
+ """A real dict, not a mock that answers anything.
+
+ A double returning a plausible value for any key cannot notice a
+ baseline keyed on the wrong subject, which is the defect this file
+ exists to catch.
+ """
+
+ def __init__(self) -> None:
+ self.hashes: dict[str, dict] = {}
+
+ async def hincrby(self, key: str, field: str, amount: int) -> int:
+ bucket = self.hashes.setdefault(key, {})
+ bucket[field] = int(bucket.get(field, 0)) + amount
+ return bucket[field]
+
+ async def hget(self, key: str, field: str):
+ value = self.hashes.get(key, {}).get(field)
+ return None if value is None else str(value)
+
+ async def hlen(self, key: str) -> int:
+ return len(self.hashes.get(key, {}))
+
+ async def hgetall(self, key: str):
+ return {k: str(v) for k, v in self.hashes.get(key, {}).items()}
+
+ async def hset(self, key: str, field: str | None = None, value=None, *, mapping: dict | None = None):
+ bucket = self.hashes.setdefault(key, {})
+ if mapping:
+ bucket.update(mapping)
+ if field is not None:
+ bucket[field] = value
+ return 1
+
+ async def expire(self, key: str, ttl: int) -> bool:
+ return True
+
+ async def sadd(self, key: str, member: str) -> int:
+ self.hashes.setdefault(f"{key}:set", {})[member] = 1
+ return 1
+
+ async def scard(self, key: str) -> int:
+ return len(self.hashes.get(f"{key}:set", {}))
+
+
+def _load(name: str) -> dict:
+ return json.loads((FIXTURES / name).read_text())
+
+
+def _message(event: dict) -> dict:
+ """What `services/ingest` hands fusion for a connector-pulled event.
+
+ `raw_data` is a JSON **string** — `json.Marshal` then `string` on the
+ Go side. A test that puts a dict here exercises a branch the
+ production path does not take.
+ """
+ return {
+ "tenant_id": TENANT,
+ "ocsf_event": {
+ "class_uid": 6003,
+ "source_type": "ai_gateway",
+ "activity_name": event.get("tool_name") or "",
+ "actor": {"user": {"name": event.get("on_behalf_of")}},
+ "metadata": {"product": {"name": "AI Gateway", "vendor_name": "AI Gateway"}},
+ "message": event.get("title") or "",
+ "finding": {"uid": event.get("external_id") or ""},
+ "raw_data": json.dumps(event),
+ },
+ }
+
+
+def engine() -> DetectionEngine:
+ return DetectionEngine()
+
+
+def store() -> BaselineStore:
+ return BaselineStore(_FakeRedis(), min_observations=1)
+
+
+async def _derived(store: BaselineStore, event: dict) -> dict:
+ """The derived namespace the engine will see, from the real store."""
+ message = _message(event)
+ fields = _message_fields(message)
+ return await store.derived_fields(TENANT, fields)
+
+
+def _message_fields(message: dict) -> dict:
+ """The connector namespace the engine reconstructs from `raw_data`.
+
+ Re-derived here the same way `DetectionEngine._raw_fields` does, so the
+ baseline store is fed exactly what the engine will match against — if
+ the two disagreed, a rule could clause on a field the baseline never
+ saw and this test would not notice.
+ """
+ raw = message["ocsf_event"]["raw_data"]
+ decoded = json.loads(raw)
+ nested = decoded.get("raw_event")
+ if isinstance(nested, dict):
+ merged = dict(nested)
+ merged.update(decoded)
+ return merged
+ return decoded
+
+
+async def _fire(engine: DetectionEngine, store: BaselineStore, event: dict) -> set[str]:
+ """Slugs that fire, with the baseline computed by the real store."""
+ derived = await _derived(store, event)
+ hits = engine.evaluate(_message(event), _extra_fields=derived)
+ return {_ID_TO_SLUG.get(getattr(h, "rule_id", "") or getattr(h, "id", ""), "") for h in hits}
+
+
+async def _build_history(store: BaselineStore, *, turns: int = 25, vary_cost: bool = False) -> None:
+ routine = _load("routine.json")
+ for index in range(turns):
+ event = dict(routine)
+ if vary_cost:
+ # A constant has no z-score, by design — `z_score` refuses a
+ # collapsed standard deviation so the most automated agent in
+ # the estate is not the least suspicious one.
+ event["request_cost_usd"] = 0.02 + (index % 5) * 0.002
+ await _derived(store, event)
+
+
+# ---------------------------------------------------------------------------
+# The rules are in the artefact the engine loads
+# ---------------------------------------------------------------------------
+
+SLUGS = (
+ "ai-agent-new-high-risk-tool",
+ "ai-agent-guardrail-violation-rate",
+ "ai-agent-token-spend-anomaly",
+ "ai-gateway-key-spend-anomaly",
+)
+
+
+def test_all_four_rules_are_compiled():
+ """`detections/*.yaml` is generated output; the engine loads the JSON.
+
+ A rule in one and not the other is a published figure describing
+ something that does not run.
+ """
+ missing = sorted(set(SLUGS) - {r["slug"] for r in _compiled()})
+ assert not missing, f"declared but not compiled: {missing}"
+
+
+# ---------------------------------------------------------------------------
+# Positive and negative, through the real baseline
+# ---------------------------------------------------------------------------
+
+
+async def test_a_never_used_high_risk_tool_fires(engine: DetectionEngine, store: BaselineStore):
+ await _build_history(store)
+ assert "ai-agent-new-high-risk-tool" in await _fire(engine, store, _load("high_risk_tool.json"))
+
+
+async def test_a_novel_but_benign_tool_does_not_fire(engine: DetectionEngine, store: BaselineStore):
+ """The negative twin, differing in exactly one field: the tool name.
+
+ Equally novel, equally unseen. If this fired, the rule would be a
+ novelty detector rather than a high-risk-tool detector, and every
+ agent's first use of anything would be an alert.
+ """
+ await _build_history(store)
+ assert "ai-agent-new-high-risk-tool" not in await _fire(engine, store, _load("novel_benign_tool.json"))
+
+
+async def test_the_agents_routine_tool_does_not_fire(engine: DetectionEngine, store: BaselineStore):
+ """The other negative: familiar, and therefore not novel.
+
+ This is the one that proves the *baseline* is doing the work rather
+ than the tool list.
+ """
+ await _build_history(store)
+ assert "ai-agent-new-high-risk-tool" not in await _fire(engine, store, _load("routine.json"))
+
+
+async def test_a_high_risk_tool_the_agent_uses_constantly_does_not_fire(engine: DetectionEngine, store: BaselineStore):
+ """Novelty, not the verb.
+
+ A deployment bot whose job is `execute_command` must not alert on
+ every run. Without this the rule is a list of scary tool names, which
+ is the thing an operator switches off in week one.
+ """
+ high_risk = _load("high_risk_tool.json")
+ for _ in range(30):
+ await _derived(store, high_risk)
+ assert "ai-agent-new-high-risk-tool" not in await _fire(engine, store, high_risk)
+
+
+async def test_a_blocked_turn_fires_only_once_the_rate_is_abnormal(engine: DetectionEngine, store: BaselineStore):
+ """A single guardrail violation is noise; a rise against the agent's
+ own history is someone probing for a phrasing that gets through."""
+ blocked = _load("guardrail_blocked.json")
+ assert "ai-agent-guardrail-violation-rate" not in await _fire(engine, store, blocked)
+
+
+async def test_spend_far_above_the_agents_baseline_fires(engine: DetectionEngine, store: BaselineStore):
+ await _build_history(store, turns=30, vary_cost=True)
+ fired = await _fire(engine, store, _load("expensive.json"))
+ assert "ai-agent-token-spend-anomaly" in fired
+ # The same event moves the key's baseline too, and that is deliberate:
+ # a leaked key driving inference from outside the estate moves the
+ # key's figure while every agent on it looks ordinary.
+ assert "ai-gateway-key-spend-anomaly" in fired
+
+
+async def test_a_routine_cost_does_not_fire(engine: DetectionEngine, store: BaselineStore):
+ """The negative twin for spend, differing only in the cost."""
+ await _build_history(store, turns=30, vary_cost=True)
+ fired = await _fire(engine, store, _load("routine.json"))
+ assert "ai-agent-token-spend-anomaly" not in fired
+ assert "ai-gateway-key-spend-anomaly" not in fired
+
+
+async def test_nothing_fires_before_there_is_a_history(engine: DetectionEngine, store: BaselineStore):
+ """A deployment's first week must be quiet.
+
+ "Not enough history" and "anomalous" must never look the same. All
+ four rules read a baseline that withholds a score until it has one, so
+ an estate that has just switched this on gets silence rather than a
+ page per agent.
+ """
+ fired = await _fire(engine, store, _load("expensive.json"))
+ assert not (set(SLUGS) & fired), f"fired with no history: {sorted(set(SLUGS) & fired)}"
diff --git a/services/fusion/tests/test_sub_principal_baselines.py b/services/fusion/tests/test_sub_principal_baselines.py
new file mode 100644
index 000000000..381163690
--- /dev/null
+++ b/services/fusion/tests/test_sub_principal_baselines.py
@@ -0,0 +1,393 @@
+"""Baselines per sub-principal, with the AI agent as one kind.
+
+Depth plan wave 2, Phase 7.3 (depth 6.4, parity 6.7).
+
+The gap
+-------
+`baselines.py` had exactly one sub-principal: `baseline_oauth_app_volume_z`,
+with the OAuth application's field chain hardcoded into `derived_fields`.
+Nothing keyed a baseline on an AI agent, so depth 6.4's "per-agent baselines
+(tools, resources, volume, token spend, hours)" had nowhere to live — and
+four of 7.4's targets, which all turn on *novelty* or *rate per agent*, had
+no input.
+
+Why this is one mechanism and not two
+--------------------------------------
+Wave-2 Phase 1.4 (Lane A) asks for baselines per sub-principal — OAuth app,
+personal access token, service account, and **an AI agent acting for a
+human**. An AI agent is a sub-principal in exactly that sense. Two items
+building two mechanisms in the same module would give the console two ways
+to say the same thing, which is the drift this repository keeps finding.
+
+So the OAuth application stops being special-cased and becomes the first
+entry in `SUB_PRINCIPAL_FIELDS`; the AI agent and the gateway key are the
+second and third. Lane A's remaining two kinds are a dict entry each, and
+`baseline_oauth_app_volume_z` keeps its published name because the generic
+naming produces it.
+
+Every chain here names a field a connector really emits
+--------------------------------------------------------
+That is the module's own rule, and the reason it exists: the first draft of
+`baseline_client_family_rarity` listed three names no connector emitted, so
+it could never have been computed on any deployment while reading as
+working. `test_every_sub_principal_chain_is_emitted_by_something` holds this
+file to it.
+"""
+
+from __future__ import annotations
+
+import pytest
+from app.services.baselines import (
+ DERIVED_FIELDS,
+ SUB_PRINCIPAL_FIELDS,
+ BaselineStore,
+ resolve_sub_principals,
+)
+
+pytestmark = pytest.mark.asyncio
+
+TENANT = "11111111-1111-1111-1111-111111111111"
+
+#: One gateway span, with the field names `ai_gateway.normalize()` emits.
+AGENT_EVENT = {
+ "agent_id": "release-bot",
+ "agent_name": "release-bot",
+ "on_behalf_of": "[email protected]",
+ "tool_name": "execute_command",
+ "model": "claude-opus-5-5",
+ "gateway_key": "release-bot-key",
+ "prompt_tokens": 20,
+ "completion_tokens": 80,
+ "total_tokens": 100,
+ "request_cost_usd": 0.02,
+ "session_id": "s-1",
+ "user": "[email protected]",
+}
+
+
+class _FakeRedis:
+ """Enough Redis for the store, with the counters readable by a test.
+
+ Deliberately a real dict rather than a mock that answers anything: a
+ double that returns a plausible value for any key cannot notice a
+ baseline keyed on the wrong subject, which is the defect class this
+ whole file exists to prevent.
+ """
+
+ def __init__(self) -> None:
+ self.hashes: dict[str, dict[str, int]] = {}
+ self.zsets: dict[str, dict[str, float]] = {}
+
+ async def hincrby(self, key: str, field: str, amount: int) -> int:
+ bucket = self.hashes.setdefault(key, {})
+ bucket[field] = bucket.get(field, 0) + amount
+ return bucket[field]
+
+ async def hget(self, key: str, field: str):
+ value = self.hashes.get(key, {}).get(field)
+ return None if value is None else str(value)
+
+ async def hlen(self, key: str) -> int:
+ return len(self.hashes.get(key, {}))
+
+ async def expire(self, key: str, ttl: int) -> bool:
+ return True
+
+ async def zadd(self, key: str, mapping: dict[str, float]) -> int:
+ self.zsets.setdefault(key, {}).update(mapping)
+ return len(mapping)
+
+ async def zremrangebyscore(self, key: str, lo: float, hi: float) -> int:
+ return 0
+
+ async def zcard(self, key: str) -> int:
+ return len(self.zsets.get(key, {}))
+
+ async def zrangebyscore(self, key: str, lo: float, hi: float):
+ return list(self.zsets.get(key, {}))
+
+ async def hgetall(self, key: str):
+ return {k: str(v) for k, v in self.hashes.get(key, {}).items()}
+
+ async def hset(self, key: str, field: str | None = None, value=None, *, mapping: dict | None = None):
+ """Both call shapes, because the production code uses the positional one.
+
+ The first draft of this double accepted only `mapping=`, so every
+ `hset(key, field, value)` raised `TypeError` — which the store's
+ fail-soft path swallowed into a `debug` log and returned `None`
+ from. The tests then failed with "no spend field", which reads as a
+ missing feature rather than as a broken double. A fake less capable
+ than the real thing is the mirror of the usual trap and fails in a
+ way that blames the code under test.
+ """
+ bucket = self.hashes.setdefault(key, {})
+ if mapping:
+ bucket.update(mapping)
+ if field is not None:
+ bucket[field] = value
+ return 1
+
+ async def sadd(self, key: str, member: str) -> int:
+ self.hashes.setdefault(f"{key}:set", {})[member] = 1
+ return 1
+
+ async def scard(self, key: str) -> int:
+ return len(self.hashes.get(f"{key}:set", {}))
+
+
+def store() -> BaselineStore:
+ return BaselineStore(_FakeRedis(), min_observations=1)
+
+
+# ---------------------------------------------------------------------------
+# One mechanism
+# ---------------------------------------------------------------------------
+
+
+def test_the_oauth_application_is_one_sub_principal_kind_among_several():
+ """Not a special case in `derived_fields` any more.
+
+ Lane A's 1.4 adds `token` and `service_account` as dict entries. If the
+ OAuth app were still hardcoded, those two would each need their own
+ block and the module would grow four copies of one idea.
+ """
+ assert "oauth_app" in SUB_PRINCIPAL_FIELDS
+ assert "agent" in SUB_PRINCIPAL_FIELDS
+ assert "gateway_key" in SUB_PRINCIPAL_FIELDS
+
+
+def test_the_published_oauth_field_name_is_unchanged():
+ """`baseline_oauth_app_volume_z` is already a published derived field.
+
+ Renaming it as a side effect of generalising would break every rule
+ reading it, which is a regression dressed as a refactor.
+ """
+ assert "baseline_oauth_app_volume_z" in DERIVED_FIELDS
+
+
+def test_every_sub_principal_chain_is_emitted_by_something():
+ """The module's own rule, applied to the new entries.
+
+ A chain naming no emitted field is a signal structurally incapable of
+ firing. `agent_id`, `agent_name` and `gateway_key` come from
+ `ai_gateway.normalize()` and the `ai-runtime` ingest template; the
+ OAuth names were already proven.
+ """
+ emitted = {
+ "agent_id",
+ "agent_name",
+ "gateway_key",
+ "application",
+ "applicationName",
+ "app_name",
+ "client_id",
+ "oauth_app",
+ }
+ for kind, chain in SUB_PRINCIPAL_FIELDS.items():
+ assert set(chain) & emitted, f"{kind}: no field in {chain} is emitted by any connector"
+
+
+def test_a_gateway_span_resolves_to_both_its_agent_and_its_key():
+ """One event, two sub-principals, which is the point of the generalisation.
+
+ A compromised key used by a well-behaved agent and a misbehaving agent
+ on a normal key are different incidents, and only a baseline per
+ subject separates them.
+ """
+ resolved = dict(resolve_sub_principals(AGENT_EVENT))
+ assert resolved["agent"] == "release-bot"
+ assert resolved["gateway_key"] == "release-bot-key"
+
+
+# ---------------------------------------------------------------------------
+# The five per-agent baselines depth 6.4 names
+# ---------------------------------------------------------------------------
+
+
+ "field",
+ [
+ "baseline_agent_tool_rarity",
+ "baseline_agent_model_rarity",
+ "baseline_agent_hour_rarity",
+ "baseline_agent_volume_z",
+ "baseline_agent_token_spend_z",
+ "baseline_gateway_key_token_spend_z",
+ ],
+)
+def test_the_field_is_declared(field: str):
+ assert field in DERIVED_FIELDS
+
+
+async def test_a_new_tool_for_this_agent_is_rarer_than_its_routine_one(store: BaselineStore):
+ """The input 7.4's "a *new* high-risk tool" rule needs.
+
+ Rarity is per agent, not global: `execute_command` is routine for a
+ deployment bot and alarming for a summariser, and a global count cannot
+ tell them apart.
+ """
+ for _ in range(20):
+ await store.derived_fields(TENANT, dict(AGENT_EVENT, tool_name="search_logs"))
+
+ routine = await store.derived_fields(TENANT, dict(AGENT_EVENT, tool_name="search_logs"))
+ novel = await store.derived_fields(TENANT, dict(AGENT_EVENT, tool_name="execute_command"))
+
+ assert novel["baseline_agent_tool_rarity"] > routine["baseline_agent_tool_rarity"]
+
+
+async def test_the_same_tool_on_a_different_agent_is_still_novel(store: BaselineStore):
+ """The negative twin: the subject is the agent, not the tool.
+
+ If the counter were keyed on the tool alone, a tool made routine by one
+ agent would be scored routine for every agent — which is how a
+ compromised agent inherits the estate's good behaviour.
+
+ Both agents are given a history first, because `rarity` deliberately
+ scores a first-ever sighting `1 - 1/total`, i.e. `0.0` on a subject
+ with no history. A subject nobody has observed is not "maximally
+ anomalous", it is unmeasured, and the module refuses to claim more
+ confidence than the history supports. Comparing two fresh subjects
+ would therefore compare 0.0 with 0.0 and prove nothing.
+ """
+ for _ in range(20):
+ await store.derived_fields(TENANT, dict(AGENT_EVENT, tool_name="execute_command"))
+ for _ in range(20):
+ await store.derived_fields(TENANT, dict(AGENT_EVENT, agent_id="summariser", agent_name="summariser", tool_name="search_logs"))
+
+ other = await store.derived_fields(
+ TENANT, dict(AGENT_EVENT, agent_id="summariser", agent_name="summariser", tool_name="execute_command")
+ )
+ familiar = await store.derived_fields(TENANT, dict(AGENT_EVENT, tool_name="execute_command"))
+
+ assert other["baseline_agent_tool_rarity"] > familiar["baseline_agent_tool_rarity"]
+
+
+async def test_token_spend_is_tracked_per_agent_and_per_key(store: BaselineStore):
+ """Both subjects from one event, which 7.4 asks for by name.
+
+ A history is fed first, and a *varying* one. Two guards decide whether
+ a score exists at all, and both are deliberate: `observe_amount`
+ withholds below `MIN_VOLUME_SAMPLES`, and `z_score` refuses a
+ distribution whose standard deviation has collapsed to zero. The second
+ is the one that bites — feeding the same cost fifteen times produces a
+ constant, and a constant has no z-score. Real gateway traffic varies.
+ """
+ for index in range(15):
+ await store.derived_fields(TENANT, dict(AGENT_EVENT, request_cost_usd=0.01 + index * 0.001))
+
+ fields = await store.derived_fields(TENANT, dict(AGENT_EVENT, request_cost_usd=0.02))
+ assert "baseline_agent_token_spend_z" in fields
+ assert "baseline_gateway_key_token_spend_z" in fields
+
+
+async def test_a_spend_signal_is_withheld_until_there_is_a_history(store: BaselineStore):
+ """The other direction, stated rather than left implicit.
+
+ "Not enough history" and "normal spend" must not look the same. A
+ z-score published over three observations would be a confident number
+ about nothing, which is the failure mode this repository calls a
+ vacuous zero.
+ """
+ fields = await store.derived_fields(TENANT, AGENT_EVENT)
+ assert "baseline_agent_token_spend_z" not in fields
+
+
+async def test_an_unusually_expensive_request_scores_above_a_routine_one(store: BaselineStore):
+ """The signal 7.4's token-spend rule reads.
+
+ Asserted as a comparison between two measured scores rather than
+ against a threshold, so it keeps meaning if the floor moves.
+ """
+ for index in range(20):
+ await store.derived_fields(TENANT, dict(AGENT_EVENT, request_cost_usd=0.02 + (index % 5) * 0.002))
+
+ routine = await store.derived_fields(TENANT, dict(AGENT_EVENT, request_cost_usd=0.022))
+ spike = await store.derived_fields(TENANT, dict(AGENT_EVENT, request_cost_usd=25.0))
+
+ assert spike["baseline_agent_token_spend_z"] > routine["baseline_agent_token_spend_z"]
+
+
+async def test_a_perfectly_constant_spender_gets_no_score_rather_than_zero(store: BaselineStore):
+ """The module's documented refusal, pinned for the spend path too.
+
+ A batch agent billing the identical amount every run converges on zero
+ variance. Returning `0.0` there would make the most automated agent in
+ the estate the least suspicious one it is possible to be — which is
+ exactly what `z_score`'s docstring says it refuses to do, and this
+ asserts the new caller inherits that rather than working around it.
+ """
+ for _ in range(20):
+ await store.derived_fields(TENANT, dict(AGENT_EVENT, request_cost_usd=0.02))
+
+ fields = await store.derived_fields(TENANT, dict(AGENT_EVENT, request_cost_usd=0.02))
+ assert "baseline_agent_token_spend_z" not in fields
+
+
+async def test_an_event_with_no_cost_contributes_no_spend_signal(store: BaselineStore):
+ """Unknown is not zero.
+
+ A gateway that did not price a request must not teach the baseline that
+ the request was free, because a spend baseline fed zeros learns that
+ this agent costs nothing and then flags nothing ever again.
+ """
+ event = {k: v for k, v in AGENT_EVENT.items() if k != "request_cost_usd"}
+ fields = await store.derived_fields(TENANT, event)
+ assert "baseline_agent_token_spend_z" not in fields
+
+
+async def test_an_event_naming_no_agent_produces_no_agent_fields(store: BaselineStore):
+ """A baseline with no subject is not a baseline.
+
+ Every ordinary cloud or identity event reaches this module, and none of
+ them names an agent. Emitting an agent field for those would make the
+ signal meaningless and the Redis key space unbounded.
+ """
+ fields = await store.derived_fields(TENANT, {"user": "[email protected]", "action": "GetObject"})
+ assert not [f for f in fields if "agent" in f]
+
+
+async def test_the_hour_of_day_is_baselined_per_agent(store: BaselineStore):
+ """ "Hours" in depth 6.4's list.
+
+ An agent that has only ever run inside a nightly window and suddenly
+ runs at midday is the signal; a human's working hours are already
+ covered by the principal baseline.
+ """
+ fields = await store.derived_fields(TENANT, AGENT_EVENT)
+ assert "baseline_agent_hour_rarity" in fields
+
+
+async def test_an_agent_acting_for_nobody_still_gets_a_baseline(store: BaselineStore):
+ """The defect that made every per-agent baseline compute on zero events.
+
+ `derived_fields` returned `{}` whenever `resolve_principal` found no
+ human. A gateway span names an agent and a key and frequently no
+ person at all — an agent running on a schedule acts for nobody — so
+ the sub-principal half was skipped for exactly the events it exists
+ to score, while the field names read as working.
+
+ Found by driving the real store from the real connector output rather
+ than by reading the function, because the early return is three lines
+ above the code it disables.
+ """
+ autonomous = {k: v for k, v in AGENT_EVENT.items() if k not in {"user", "on_behalf_of"}}
+
+ fields = await store.derived_fields(TENANT, autonomous)
+
+ assert "baseline_agent_tool_rarity" in fields
+ assert "baseline_agent_hour_rarity" in fields
+ # And no principal-scoped field, because there is genuinely no
+ # principal — the fix must not invent one.
+ assert "baseline_principal_volume_z" not in fields
+ assert "baseline_peer_deviation" not in fields
+
+
+async def test_the_human_an_agent_acts_for_is_recognised_as_the_principal(store: BaselineStore):
+ """`on_behalf_of` is what the `ai-runtime` template maps to
+ `actor.user.name`, so the vendor and the schema already agree it is
+ the principal. This module simply could not see it, which meant an
+ agent acting for a named person contributed nothing to that person's
+ own baseline."""
+ fields = await store.derived_fields(TENANT, {k: v for k, v in AGENT_EVENT.items() if k != "user"})
+ assert "baseline_agent_tool_rarity" in fields
diff --git a/tests/test_packaged_detection_corpus_gate.py b/tests/test_packaged_detection_corpus_gate.py
index 8e20cf898..8b7adc5a6 100644
--- a/tests/test_packaged_detection_corpus_gate.py
+++ b/tests/test_packaged_detection_corpus_gate.py
@@ -76,7 +76,7 @@ class TestThePositiveControl:
def test_a_synced_tree_is_clean(self, tmp_path: pathlib.Path) -> None:
result = _check(_tree(tmp_path))
assert result.returncode == 0, result.stderr
- assert "2613 rules" in result.stdout
+ assert "2617 rules" in result.stdout
class TestTheGateCatchesItsOwnAbsence: