Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

depth 3.3: identity privilege, a first-seen store, and a reason for every rule that still cannot fire (#1226)

Beenu Arora2e552430d89c
diff --git a/.github/workflows/validate-detections.yml b/.github/workflows/validate-detections.yml
index 993bf9717..5fae184ab 100644
--- a/.github/workflows/validate-detections.yml
+++ b/.github/workflows/validate-detections.yml
@@ -124,6 +124,17 @@ jobs:
- name: Prove the Sigma-correlation gate is not vacuous
run: python3 scripts/check_sigma_correlations.py --self-test
+ # Depth plan 3.3 built two enrichment inputs and retired what still
+ # could not fire. Deleting a rule and fixing a rule move the
+ # unreachable ratchet by the same amount, so this makes them
+ # different: every retired rule names a real spec and a reason, and a
+ # rule whose fields the platform *can* now resolve may not be retired.
+ - name: Verify every rule that cannot fire carries a reason
+ run: python3 scripts/check_enrichment_decisions.py --check
+
+ - name: Prove the enrichment-decision gate is not vacuous
+ run: python3 scripts/check_enrichment_decisions.py --self-test
+
# Recompiles every imported Sigma rule AND re-proves it: each shipped
# rule is replayed, as a vendor-shaped event, through its connector's
# real normalize() and the real engine. A rule that stops firing — because
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 5a178b881..2e13fec36 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -103,6 +103,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
The published executable figure falls 2,603 → 2,529 because 74 rules that
could not fire stopped being counted as executable; the windowed corpus is
now published as its own figure beside it rather than going uncounted.
+- **Per-tenant identity privilege and a first-seen store, so the rules that
+ read them can fire.** 18 rules matched on a `*_priv` or `*_is_admin`
+ boolean and nothing computed any of them, so each read `None` on that
+ clause and could never fire. They now resolve from
+ `identity_nodes.privilege_tier`, per tenant and hot-reloaded alongside the
+ existing tuning overlay — one tenant's administrators are not another's,
+ and an unknown subject contributes **no key** rather than `False`, because
+ `False` asserts the account is ordinary and would be wrong on every
+ deployment that has imported no directory. A per-tenant first-seen store
+ (`services/fusion/app/services/first_seen.py`) answers
+ `<attr>_seen_before` and `session_age_hours` from Redis and closes two
+ more. 18 rules that still cannot fire are retired with an individual
+ reason in `scripts/enrichment_decisions.py`, and
+ `scripts/check_enrichment_decisions.py` refuses to let a rule whose fields
+ *are* resolvable be retired — deleting a rule and fixing one move the
+ ratchet by the same amount, and that gate is what makes them different.
+ `MAX_UNREACHABLE` falls 119 → 2; the two that remain need the behavioural
+ baseline of a later item.
+- **Two matcher operators three shipped rules were already using.**
+ `not_startswith_any` and `not_endswith` had never existed, so
+ `path_not_startswith_any` was read as a field name and the three rules
+ using it could not fire — while the reachability gate, whose operator list
+ had drifted from the matcher's in *both* directions, stripped a suffix
+ nothing implemented and reported them reachable. The same drift in the
+ other direction had five rules counted as unreachable for a `neq` the
+ matcher gained and the gate did not. Both lists are now compared in both
+ directions by a test.
- **Ordered and unordered sequences in the windowed engine, and a Sigma
correlation importer that feeds it.** A threshold cannot express an
ordering, and the ordering is usually the detection: "fifty failed logons"
diff --git a/DEPTH_PROGRESS.md b/DEPTH_PROGRESS.md
index 6ca5c7853..f7531a588 100644
--- a/DEPTH_PROGRESS.md
+++ b/DEPTH_PROGRESS.md
@@ -278,6 +278,67 @@ Instead, every selector field, `group-by` and `distinct_by` is checked
against the emitted-field namespace with **no ceiling**, which caught three
invented fields on the first run of the gate.
+### D9 -- Five of the 45 "unreachable" rules were never unreachable
+
+`scripts/check_detection_fields.py` carries its own copy of the matcher's
+operator suffixes, under a comment saying it mirrors
+`detection_matcher.OPERATORS`. It did not. `neq` had been added to the
+matcher and not to the gate, so `approver_role_neq: "codeowner"` — which the
+engine reads correctly as `approver_role` with `!=` — was reported as a rule
+naming a field called `approver_role_neq`. Five rules were counted against
+the ratchet for a defect that had already been fixed.
+
+The same comparison in the other direction found a worse one: the gate
+listed `not_startswith_any` and `not_endswith`, which the **matcher** did not
+implement. Three shipped rules use `path_not_startswith_any`, so the matcher
+read the whole string as a field name and all three could never fire — while
+the gate, stripping a suffix nothing implemented, reported them reachable.
+Both operators are now implemented in the vendored matcher and the canonical
+one, the fixture synthesiser knows them, and
+`test_the_gate_and_the_matcher_agree_on_every_operator` compares the two
+lists in both directions.
+
+### D10 -- 18 rules are retired rather than fixed, and why each one is
+
+The plan says of one family: "build them where cheap, otherwise quarantine
+with a reason". Applied to all three families, that leaves 18 rules that need
+work belonging to another item, recorded individually in
+`scripts/enrichment_decisions.py`:
+
+| kind | n | needs |
+|---|---|---|
+| `needs-connector-field` | 9 | a field the vendor has and no connector surfaces — depth plan 2.2 |
+| `needs-external-source` | 6 | a registry, a credential report or a factor record — 2.5 and beyond |
+| `needs-inventory` | 3 | an inventory this platform does not hold (DC machine accounts, privileged OAuth scopes) |
+
+`scripts/check_enrichment_decisions.py` is what stops this being a way to
+move the ratchet: it fails if a retired rule's fields are all resolvable.
+Proved by retiring a working rule and watching it fail.
+
+The line that mattered most was refusing a plausible substitute.
+`domain_age_days` wants a domain's registration age from a registry; the
+first-seen store could have answered "when this deployment first saw it" and
+the number would have looked right in a diff. It would also have fired the
+rule on most of the internet. Five such age-of-a-thing fields are named in a
+test that forbids the first-seen store from ever growing them.
+
+### D11 -- Identity privilege reads a table a tenant must populate
+
+The `*_priv` booleans resolve from `identity_nodes.privilege_tier >= 2`, the
+column migration 018 created for the question. The table's only writer is
+`POST /api/v1/identity-graph/nodes`, so a tenant that has imported no
+directory gets **no key at all** and those 18 rules stay silent — they are
+reachable "when the tenant has identity data", the same standard this gate
+already applies to a vendor-payload field. That is weaker than "reachable on
+any deployment" and is stated rather than glossed. Depth plan 2.5's posture
+collectors are what will populate it automatically.
+
+Four booleans in the same family are deliberately **not** computed —
+`scope_priv`, `act_as_user_priv`, `gpo_link_priv` and `account_is_dc` — and
+are among the 18 above. Each has no subject field any source emits, and
+guessing a subject would produce a boolean that is confidently wrong rather
+than absent.
+
## Phase 1: Verdict quality you can publish
- [x] **1.1** Balanced, labelled verdict corpus
@@ -306,6 +367,9 @@ invented fields on the first run of the gate.
- [x] **3.2** Ordered sequences and Sigma correlations — the windowed engine
stages ordered and unordered sequences; all four translatable Sigma
correlation types compile. See D7 and D8.
+- [x] **3.3** Enrichment inputs (parity 5.5) — identity privilege and a
+ per-tenant first-seen store built; 18 rules retired with a reason.
+ `MAX_UNREACHABLE` 45 → 2. See D9–D11.
- [ ] **3.2** Ordered sequences and Sigma correlations
- [ ] **3.3** Enrichment inputs (parity 5.5)
- [ ] **3.4** Behavioural baselines in CORE
@@ -383,5 +447,7 @@ invented fields on the first run of the gate.
| 2026-10-07 | 0.1 | This file created at base commit `1b8bc2d4`. |
| 2026-10-07 | 0.2 | Every figure re-derived. Two matched exactly (detections, unreachable families); executor arms measured 74 against a captured 73; the cloud/identity/SaaS/code figure measured 395 against a captured 461 on a grouping the plan does not pin, recorded above. Two measurement caveats found: executable and quarantined overlap by 1,724 rules, and the "69% Windows" figure does not reproduce from the index. |
| 2026-10-07 | 0.3 | `make up` and `make smoke` (10/10) pass. Injection suite and load-harness baselines committed. `make up-full` deferred (D2) and hosted model rows blocked (D3). Fixing D1 was a precondition for the load-harness baseline. |
+| 2026-10-09 | 3.3 | Identity privilege (18 rules) and a per-tenant first-seen store (2) built; 5 rules were never unreachable (gate operator drift, D9); 18 retired with a reason (D10). `MAX_UNREACHABLE` 45 → 2, the two remaining both needing the 3.4 baseline. Two matcher operators three shipped rules already used were implemented. |
+| 2026-10-09 | 3.2 | Ordered and unordered sequences in the windowed engine; all four translatable Sigma correlation types compile, seven refusal reasons recorded. No upstream correlation rule exists in this tree to import (D8), so the corpus is first-party and labelled as such. |
| 2026-10-09 | 4.1 | Reproduced: `check_cloud_native_collection.py` on the unmodified tree reported all three collection paths absent. Implemented `aws_cloudtrail_s3` (SQS-notified S3 objects; management events, data events and VPC flow logs; delete-after-emit), `gcp_pubsub` (REST pull on a log-sink subscription; ack-after-read) and `azure_event_hubs` (Capture blobs over the Data Lake Gen2 JSON API, with a focused Avro OCF reader). Negative controls recorded in the PR: deleting the SQS message before the object is read, acknowledging the Pub/Sub batch before it is built, ignoring the Avro union branch index, and ignoring the flow-log header line each fail a named test, and all were restored. Connectors suite 1014 -> 1098 passed. |
| 2026-10-09 | 3.1 | All 74 decided. 50 translated into `wd-*` rules derived from each original's own clauses and replayed through the real engine (162 assertions); 24 refused with a reason across five kinds. `MAX_UNREACHABLE` 119 → 45, published executable 2,603 → 2,529, windowed 18 → 68. Three findings recorded as D4–D6 below. |
diff --git a/README.md b/README.md
index 114202e51..881bb7029 100644
--- a/README.md
+++ b/README.md
@@ -17,7 +17,7 @@
## What AiSOC does
-Telemetry arrives from your security tools. AiSOC normalizes it, runs the 2529 executable rules of
+Telemetry arrives from your security tools. AiSOC normalizes it, runs the 2511 executable rules of
its 6991-rule library, groups what fires into incidents, investigates each one with an AI agent whose
every prompt and tool call is recorded, and proposes an action. New threat intelligence re-sweeps the
history you already collected, and a human approves before anything reaches a vendor.
@@ -111,7 +111,7 @@ of those are partial: field matches carried, thresholds did not
## How it works
Ingest normalizes to a common shape and Kafka carries it, then
-fusion runs 2529 executable detection rules, of 6991 on disk, plus 72 sliding-window rules that count across events, applies **your tenant's own tuning**
+fusion runs 2511 executable detection rules, of 6991 on disk, plus 72 sliding-window rules that count across events, applies **your tenant's own tuning**
on top — the disables, floors and suppressions the console writes, so a rule you turned off actually stops firing — and decides what
becomes an alert. Correlation groups related alerts, an agent investigates and writes its reasoning
to the Investigation Ledger, and a playbook may start from the result. Separately, new threat
@@ -195,7 +195,7 @@ Agents triage alerts and investigate incidents. What they can and cannot do:
| Capability | Status | Tested | Production ready |
|---|---|---|---|
| Ingest → detect → correlate → alert | Stable | E2E + unit | Yes |
-| Detection engine (2529 executable rules) of 6991 | Stable | Replay proof | Yes |
+| Detection engine (2511 executable rules) of 6991 | Stable | Replay proof | Yes |
| Alert correlation into incidents | Stable | Unit | Yes |
| REST API + web console | Stable | Unit + integration | Yes |
| AI triage + Investigation Ledger | Stable | Live Postgres ledger + a PR-gated local-model agent run. No hosted provider has been exercised | Yes, copilot mode |
diff --git a/ROADMAP.md b/ROADMAP.md
index 50f97a2ae..9058c65bb 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -38,7 +38,8 @@ A proof-first, security-first program to make every README claim gate-backed, cl
**Program status:** all 13 hardening phases (0–12) are landed. Building on them, the **Fully-Operational AI-SOC roadmap** (Phases A1–E1) is now complete — it wired the three end-to-end paths the reality audit found unwired and pushed the platform to competitive parity + beyond:
-- **Phase A (SIEM foundation):** A1 ClickHouse lake writer · A2 live detection-evaluation worker (2529 executable rules on the stream) · A3 default-on one-command deploy (connectors + graph on) · A4 UEBA behavioral-model fusion.
+- **Phase A (SIEM foundation):** A1 ClickHouse lake writer · A2 live detection-evaluation worker (2511 executable rules on the stream) · A3 default-on one-command deploy (connectors + graph on) · A4 UEBA behavioral-model fusion.
+- **Phase A (SIEM foundation):** A1 ClickHouse lake writer · A2 live detection-evaluation worker (2511 executable rules on the stream) · A3 default-on one-command deploy (connectors + graph on) · A4 UEBA behavioral-model fusion.
- **Phase B (SOAR foundation):** B1 auto-triage worker (copilot default) · B2 credential resolver + `decide()`-governed live dispatch + 10 vendor adapters · B3 real rollback + post-action verification + durable approval-SLA timers · B4 Business Context Rules in the hot path.
- **Phase C (parity differentiators):** C1 Advanced Data Explorer · C2 Effective-Permissions live posture loader · C3 autopilot/copilot posture scorecard · C4 fuse-time attack-chain auto-grouping.
- **Phase D (breadth):** D1 eight connectors (QRadar/Exabeam/Securonix/Devo/Netskope/Windows-Sysmon/Zeek-Suricata/syslog-CEF) · D2 AI/LLM-usage audit connector + `llm-*` detections + hot/cold lake tiering · D3 live-vendor mock-server smoke.
diff --git a/apps/docs/docs/detections/coverage.md b/apps/docs/docs/detections/coverage.md
index 8b2103e22..89ad501c5 100644
--- a/apps/docs/docs/detections/coverage.md
+++ b/apps/docs/docs/detections/coverage.md
@@ -10,7 +10,7 @@ sidebar_position: 2
# Detection Coverage
-Generated: `2026-10-09T03:35:19Z`
+Generated: `2026-10-09T04:33:12Z`
This page is a **curated selection**, not the size of the corpus.
It picks the highest-scoring rules per threat family from the set
@@ -27,9 +27,9 @@ Sigma compiler began translating rules where they sat: rules under
## Headline numbers
-- **Curated detections**: `490` (target: ≥ 300)
-- **Executable rules considered**: `2529` (quality floor: 0.55)
-- **Unique MITRE techniques covered**: `136`
+- **Curated detections**: `489` (target: ≥ 300)
+- **Executable rules considered**: `2511` (quality floor: 0.55)
+- **Unique MITRE techniques covered**: `137`
## Coverage by buyer family
@@ -41,30 +41,30 @@ Sigma compiler began translating rules where they sat: rules under
| **Data Exfiltration** | 44 | ≥ 25 | ✅ |
| **Cloud** | 100 | ≥ 25 | ✅ |
| **Identity** | 100 | ≥ 25 | ✅ |
-| **Supply Chain** | 41 | ≥ 25 | ✅ |
+| **Supply Chain** | 40 | ≥ 25 | ✅ |
| **Kubernetes / Containers** | 69 | ≥ 25 | ✅ |
## Distribution
### By tier
-- `imported`: 160
-- `native`: 330
+- `imported`: 163
+- `native`: 326
### By severity
-- `critical`: 100
-- `high`: 266
+- `critical`: 99
+- `high`: 264
- `low`: 16
-- `medium`: 108
+- `medium`: 110
### By category
-- `application`: 31
-- `cloud`: 150
+- `application`: 29
+- `cloud`: 152
- `data-exfil`: 13
-- `endpoint`: 218
-- `identity`: 69
+- `endpoint`: 219
+- `identity`: 67
- `network`: 9
## How to audit
diff --git a/apps/docs/static/llms-full.txt b/apps/docs/static/llms-full.txt
index 9ff7f2b6e..73cea9825 100644
--- a/apps/docs/static/llms-full.txt
+++ b/apps/docs/static/llms-full.txt
@@ -77,7 +77,7 @@ Read more: https://beenuar.github.io/AiSOC/docs/connectors
## Detection content
-- **Executable rules: 2529.** These are the rules the detection engine
+- **Executable rules: 2511.** These are the rules the detection engine
loads from its compiled ruleset. Only these can fire.
- **Rule files on disk: 6991.** The remainder is imported and
provenance-only content — community and vendor corpora kept for coverage
diff --git a/apps/docs/static/llms.txt b/apps/docs/static/llms.txt
index 67dec28b8..6de84139b 100644
--- a/apps/docs/static/llms.txt
+++ b/apps/docs/static/llms.txt
@@ -21,10 +21,10 @@ Key facts a summary should get right:
generates its own secrets, creates an administrator, ships a real CISA Known
Exploited Vulnerabilities feed, and runs a local model, so a first run shows
real data and AI triage without signing up for anything.
-- **Detection corpus:** 2529 rules are executable — the detection engine
+- **Detection corpus:** 2511 rules are executable — the detection engine
loads them from a compiled ruleset. 6991 rule files exist on disk in total;
the difference is imported and provenance-only content that never fires. Cite
- 2529, not 6991. "Executable" means each rule was replayed through
+ 2511, not 6991. "Executable" means each rule was replayed through
a real connector and the real engine and watched to fire; it does not mean the
rule detects an attack.
- **Agent evaluation:** the published benchmark distinguishes metrics that
diff --git a/apps/web/public/marketplace/curated.json b/apps/web/public/marketplace/curated.json
index f57d39ce6..fc74a534a 100644
--- a/apps/web/public/marketplace/curated.json
+++ b/apps/web/public/marketplace/curated.json
@@ -1,31 +1,31 @@
{
"$schema": "https://aisoc.example.com/schemas/curated/v1.json",
"version": "1.0.0",
- "generated": "2026-10-09T03:35:19Z",
+ "generated": "2026-10-09T04:33:12Z",
"stats": {
- "considered": 2529,
- "eligible": 2518,
- "selected": 490,
+ "considered": 2511,
+ "eligible": 2500,
+ "selected": 489,
"min_per_family": 25,
"min_total": 300,
"quality_floor": 0.55,
- "unique_techniques": 136,
+ "unique_techniques": 137,
"selected_by_tier": {
- "imported": 160,
- "native": 330
+ "imported": 163,
+ "native": 326
},
"selected_by_severity": {
- "critical": 100,
- "high": 266,
+ "critical": 99,
+ "high": 264,
"low": 16,
- "medium": 108
+ "medium": 110
},
"selected_by_category": {
- "application": 31,
- "cloud": 150,
+ "application": 29,
+ "cloud": 152,
"data-exfil": 13,
- "endpoint": 218,
- "identity": 69,
+ "endpoint": 219,
+ "identity": 67,
"network": 9
}
},
@@ -107,7 +107,6 @@
"count": 100,
"rule_ids": [
"det-application-004",
- "det-application-020",
"det-application-075",
"det-application-085",
"det-cloud-047",
@@ -137,9 +136,7 @@
"det-identity-056",
"det-identity-057",
"det-identity-058",
- "det-identity-059",
"det-identity-061",
- "det-identity-075",
"det-identity-088",
"det-identity-101",
"det-identity-105",
@@ -184,9 +181,12 @@
"sigmahq-sigma-48d91a3a-2363-43ba-a456-ca71ac3da5c2",
"sigmahq-sigma-4d78a000-ab52-4564-88a5-7ab5242b20c7",
"sigmahq-sigma-53bb4f7f-48a8-4475-ac30-5a82ddfdf6fc",
+ "sigmahq-sigma-5496ff55-42ec-4369-81cb-00f417029e25",
"sigmahq-sigma-58f4ea09-0fc2-4520-ba18-b85c540b0eaf",
"sigmahq-sigma-5a44727c-3b85-4713-8c44-4401d5499629",
+ "sigmahq-sigma-603c6630-5225-49c1-8047-26c964553e0e",
"sigmahq-sigma-60f6535a-760f-42a9-be3f-c9a0a025906e",
+ "sigmahq-sigma-6393e346-1977-46ef-8987-ad414a145fad",
"sigmahq-sigma-7679d464-4f74-45e2-9e01-ac66c5eb041a",
"sigmahq-sigma-7892ec59-c5bb-496d-8968-e5d210ca3ac4",
"sigmahq-sigma-7ec2c172-dceb-4c10-92c9-87c1881b7e18",
@@ -555,7 +555,7 @@
},
"supply-chain": {
"label": "Supply Chain",
- "count": 41,
+ "count": 40,
"rule_ids": [
"det-application-006",
"det-application-014",
@@ -567,7 +567,6 @@
"det-application-048",
"det-application-049",
"det-application-052",
- "det-application-053",
"det-application-054",
"det-application-057",
"det-application-059",
@@ -719,7 +718,7 @@
"T1071.001": 1,
"T1078": 14,
"T1078.001": 3,
- "T1078.004": 24,
+ "T1078.004": 25,
"T1082": 2,
"T1087": 1,
"T1090": 2,
@@ -732,7 +731,7 @@
"T1102": 2,
"T1105": 1,
"T1106": 2,
- "T1110": 8,
+ "T1110": 10,
"T1110.001": 2,
"T1110.002": 1,
"T1110.004": 1,
@@ -746,14 +745,14 @@
"T1190": 11,
"T1195": 1,
"T1195.001": 1,
- "T1195.002": 7,
+ "T1195.002": 6,
"T1199": 11,
"T1201": 1,
"T1203": 1,
"T1204.002": 1,
"T1210": 4,
"T1211": 1,
- "T1212": 5,
+ "T1212": 4,
"T1218": 4,
"T1219": 2,
"T1484.001": 1,
@@ -767,7 +766,7 @@
"T1505.003": 8,
"T1518": 2,
"T1525": 8,
- "T1528": 6,
+ "T1528": 5,
"T1530": 17,
"T1531": 1,
"T1537": 7,
@@ -785,9 +784,9 @@
"T1552.007": 1,
"T1553.002": 1,
"T1554": 3,
- "T1555": 5,
+ "T1555": 6,
"T1555.003": 3,
- "T1556": 41,
+ "T1556": 40,
"T1556.001": 1,
"T1556.006": 5,
"T1556.007": 1,
@@ -816,6 +815,7 @@
"T1610": 11,
"T1611": 24,
"T1612": 1,
+ "T1621": 1,
"T1685": 2
},
"items": [
@@ -996,31 +996,6 @@
"log_source_product": "kubernetes",
"has_executable_body": true
},
- {
- "id": "det-application-020",
- "name": "API Returned Object Outside Caller's Tenant",
- "path": "detections/application/api-permission-bypass-idor.yaml",
- "tier": "native",
- "source": "core",
- "category": "application",
- "severity": "critical",
- "mitre_techniques": [
- "T1212"
- ],
- "mitre_tactics": [],
- "families": [
- "credential-access"
- ],
- "quality_score": 1.0,
- "quality_notes": [
- "tier:native(+0.5)",
- "mitre_techniques:1(+0.2)",
- "severity:critical(+0.150)",
- "executable(+0.15)"
- ],
- "log_source_product": "app",
- "has_executable_body": true
- },
{
"id": "det-application-023",
"name": "GitLab Runner Registered as Shared Without Approval",
@@ -1224,31 +1199,6 @@
"log_source_product": "github",
"has_executable_body": true
},
- {
- "id": "det-application-053",
- "name": "NPM Package Published From New Account",
- "path": "detections/application/app-npm-package-publish-new-account.yaml",
- "tier": "native",
- "source": "core",
- "category": "application",
- "severity": "high",
- "mitre_techniques": [
- "T1195.002"
- ],
- "mitre_tactics": [],
- "families": [
- "supply-chain"
- ],
- "quality_score": 0.977,
- "quality_notes": [
- "tier:native(+0.5)",
- "mitre_techniques:1(+0.2)",
- "severity:high(+0.128)",
- "executable(+0.15)"
- ],
- "log_source_product": "npm",
- "has_executable_body": true
- },
{
"id": "det-application-054",
"name": "PyPI Package Published From New IP",
@@ -8025,32 +7975,6 @@
"log_source_product": "identity",
"has_executable_body": true
},
- {
- "id": "det-identity-059",
- "name": "Session Timeout Extended For Privileged Group",
- "path": "detections/identity/ident-session-timeout-extended-priv.yaml",
- "tier": "native",
- "source": "core",
- "category": "identity",
- "severity": "medium",
- "mitre_techniques": [
- "T1556"
- ],
- "mitre_tactics": [],
- "families": [
- "credential-access",
- "identity"
- ],
- "quality_score": 0.955,
- "quality_notes": [
- "tier:native(+0.5)",
- "mitre_techniques:1(+0.2)",
- "severity:medium(+0.105)",
- "executable(+0.15)"
- ],
- "log_source_product": "identity",
- "has_executable_body": true
- },
{
"id": "det-identity-061",
"name": "Login From ASN Recently Seen In Credential-Stuffing Activity",
@@ -8102,32 +8026,6 @@
"log_source_product": "identity",
"has_executable_body": true
},
- {
- "id": "det-identity-075",
- "name": "OAuth App Acting As Privileged User",
- "path": "detections/identity/ident-app-impersonation-priv.yaml",
- "tier": "native",
- "source": "core",
- "category": "identity",
- "severity": "high",
- "mitre_techniques": [
- "T1528"
- ],
- "mitre_tactics": [],
- "families": [
- "credential-access",
- "identity"
- ],
- "quality_score": 0.977,
- "quality_notes": [
- "tier:native(+0.5)",
- "mitre_techniques:1(+0.2)",
- "severity:high(+0.128)",
- "executable(+0.15)"
- ],
- "log_source_product": "identity",
- "has_executable_body": true
- },
{
"id": "det-identity-079",
"name": "Domain Admin Group Membership Added (AD)",
@@ -10700,6 +10598,35 @@
"log_source_product": "azure",
"has_executable_body": true
},
+ {
+ "id": "sigmahq-sigma-5496ff55-42ec-4369-81cb-00f417029e25",
+ "name": "Multifactor Authentication Interrupted",
+ "path": "detections/sigma-imports/_quarantine/multifactor-authentication-interrupted.yaml",
+ "tier": "imported",
+ "source": "sigmahq",
+ "category": "cloud",
+ "severity": "medium",
+ "mitre_techniques": [
+ "T1078.004",
+ "T1110",
+ "T1621"
+ ],
+ "mitre_tactics": [],
+ "families": [
+ "credential-access",
+ "cloud",
+ "identity"
+ ],
+ "quality_score": 0.755,
+ "quality_notes": [
+ "tier:imported(+0.3)",
+ "mitre_techniques:3(+0.2)",
+ "severity:medium(+0.105)",
+ "executable(+0.15)"
+ ],
+ "log_source_product": "azure",
+ "has_executable_body": true
+ },
{
"id": "sigmahq-sigma-551d9c1f-816c-445b-a7a6-7a3864720d60",
"name": "Potential Excel.EXE DCOM Lateral Movement Via ActivateMicrosoftApp",
@@ -10903,6 +10830,31 @@
"log_source_product": "windows",
"has_executable_body": true
},
+ {
+ "id": "sigmahq-sigma-603c6630-5225-49c1-8047-26c964553e0e",
+ "name": "Enumerate Credentials from Windows Credential Manager With PowerShell",
+ "path": "detections/sigma-imports/_quarantine/enumerate-credentials-from-windows-credential-manager-with-powershell.yaml",
+ "tier": "imported",
+ "source": "sigmahq",
+ "category": "endpoint",
+ "severity": "medium",
+ "mitre_techniques": [
+ "T1555"
+ ],
+ "mitre_tactics": [],
+ "families": [
+ "credential-access"
+ ],
+ "quality_score": 0.755,
+ "quality_notes": [
+ "tier:imported(+0.3)",
+ "mitre_techniques:1(+0.2)",
+ "severity:medium(+0.105)",
+ "executable(+0.15)"
+ ],
+ "log_source_product": "windows",
+ "has_executable_body": true
+ },
{
"id": "sigmahq-sigma-60bfeac3-0d35-4302-8efb-1dd16f715bc6",
"name": "Suspicious SysAidServer Child",
@@ -11009,6 +10961,32 @@
"log_source_product": "windows",
"has_executable_body": true
},
+ {
+ "id": "sigmahq-sigma-6393e346-1977-46ef-8987-ad414a145fad",
+ "name": "AWS ConsoleLogin Failed Authentication",
+ "path": "detections/sigma-imports/_quarantine/aws-consolelogin-failed-authentication.yaml",
+ "tier": "imported",
+ "source": "sigmahq",
+ "category": "cloud",
+ "severity": "medium",
+ "mitre_techniques": [
+ "T1110"
+ ],
+ "mitre_tactics": [],
+ "families": [
+ "credential-access",
+ "cloud"
+ ],
+ "quality_score": 0.755,
+ "quality_notes": [
+ "tier:imported(+0.3)",
+ "mitre_techniques:1(+0.2)",
+ "severity:medium(+0.105)",
+ "executable(+0.15)"
+ ],
+ "log_source_product": "aws",
+ "has_executable_body": true
+ },
{
"id": "sigmahq-sigma-639c9081-f482-47d3-a0bd-ddee3d4ecd76",
"name": "All Backups Deleted Via Wbadmin.EXE",
diff --git a/apps/web/public/marketplace/index.json b/apps/web/public/marketplace/index.json
index ec51bfa25..d85ef41a6 100644
--- a/apps/web/public/marketplace/index.json
+++ b/apps/web/public/marketplace/index.json
@@ -1,7 +1,7 @@
{
"$schema": "https://example.com/schemas/marketplace/v1.json",
"version": "1.0.0",
- "generated": "2026-10-09T03:35:13Z",
+ "generated": "2026-10-09T04:33:06Z",
"categories": [
{
"id": "playbooks",
@@ -38,14 +38,14 @@
"imported": 6113,
"stable": 877
},
- "executable": 2696,
- "quarantined": 4462,
- "executable_detections": 2529,
- "reference_only_detections": 4462,
+ "executable": 2678,
+ "quarantined": 4480,
+ "executable_detections": 2511,
+ "reference_only_detections": 4480,
"not_a_detection": 167
},
"mitre_coverage": {
- "unique_techniques": 389,
+ "unique_techniques": 387,
"techniques": {
"T1001.003": 1,
"T1003": 21,
@@ -54,7 +54,7 @@
"T1003.003": 21,
"T1003.004": 10,
"T1003.005": 8,
- "T1003.006": 7,
+ "T1003.006": 6,
"T1003.008": 1,
"T1005": 7,
"T1007": 2,
@@ -137,15 +137,15 @@
"T1070.005": 4,
"T1070.006": 1,
"T1071": 7,
- "T1071.001": 14,
- "T1071.004": 11,
+ "T1071.001": 13,
+ "T1071.004": 10,
"T1072": 4,
"T1074.001": 4,
- "T1078": 50,
+ "T1078": 49,
"T1078.001": 5,
"T1078.002": 7,
"T1078.003": 2,
- "T1078.004": 47,
+ "T1078.004": 45,
"T1082": 13,
"T1083": 9,
"T1087": 10,
@@ -210,7 +210,7 @@
"T1190": 39,
"T1195": 3,
"T1195.001": 1,
- "T1195.002": 8,
+ "T1195.002": 7,
"T1197": 13,
"T1199": 12,
"T1200": 3,
@@ -224,7 +224,7 @@
"T1207": 2,
"T1210": 4,
"T1211": 4,
- "T1212": 5,
+ "T1212": 4,
"T1213": 2,
"T1216": 8,
"T1216.001": 2,
@@ -249,7 +249,7 @@
"T1404": 2,
"T1482": 14,
"T1484": 1,
- "T1484.001": 8,
+ "T1484.001": 7,
"T1484.002": 2,
"T1485": 29,
"T1486": 11,
@@ -271,7 +271,7 @@
"T1518.001": 2,
"T1525": 8,
"T1526": 2,
- "T1528": 9,
+ "T1528": 8,
"T1529": 3,
"T1530": 27,
"T1531": 7,
@@ -289,7 +289,6 @@
"T1546.001": 4,
"T1546.002": 1,
"T1546.003": 11,
- "T1546.004": 2,
"T1546.007": 1,
"T1546.008": 5,
"T1546.009": 1,
@@ -315,7 +314,7 @@
"T1550": 5,
"T1550.001": 2,
"T1550.002": 6,
- "T1550.003": 4,
+ "T1550.003": 3,
"T1550.004": 1,
"T1552": 3,
"T1552.001": 17,
@@ -331,18 +330,18 @@
"T1553.005": 5,
"T1554": 3,
"T1555": 8,
- "T1555.001": 5,
+ "T1555.001": 4,
"T1555.003": 9,
"T1555.004": 3,
"T1555.005": 1,
- "T1556": 43,
+ "T1556": 42,
"T1556.001": 1,
"T1556.003": 1,
"T1556.004": 1,
- "T1556.006": 16,
+ "T1556.006": 15,
"T1556.007": 1,
"T1557": 2,
- "T1557.001": 13,
+ "T1557.001": 12,
"T1557.002": 3,
"T1557.003": 1,
"T1558": 3,
@@ -371,7 +370,6 @@
"T1564.008": 1,
"T1565": 2,
"T1565.001": 3,
- "T1565.002": 1,
"T1566": 5,
"T1566.001": 15,
"T1566.002": 2,
@@ -437,7 +435,7 @@
"T1686.003": 10,
"T1689": 1
},
- "total_with_mitre": 2398,
+ "total_with_mitre": 2380,
"unique_techniques_all_rules": 493,
"techniques_all_rules": {
"T1001.003": 2,
@@ -31112,10 +31110,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/application/api-permission-bypass-idor.yaml",
- "sha256": "907beebb04b60d1689cfbab9e63429968a56e4e59b580a9837feb62e6ddebc7a"
+ "quarantine_reason": "not executable (needs-connector-field): compares the object's owning tenant with the caller's, and no application source in this tree surfaces either. The engine can compare two fields of one event; it cannot compare two fields that are not there.",
+ "sha256": "3f757cd10acefaf0b218f68cd8f303aef0f8f9570eb30887494e602a8d28a41d"
},
{
"id": "det-application-021",
@@ -31939,10 +31938,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/application/app-npm-package-publish-new-account.yaml",
- "sha256": "54cefdefd2ce54c8dc2a82f3f8ab0ce19663f18acec4e1c515b85fdfde352a60"
+ "quarantine_reason": "not executable (needs-external-source): wants the publisher account's age at the registry. The first-seen store knows when this deployment first saw the account, which is a different number: substituting it would fire on every publisher a quiet tenant has not seen before.",
+ "sha256": "636bbcd6478dc775efa4bca041e41f09de50c63bfaa801e7d5f66fd1383de7b6"
},
{
"id": "det-application-054",
@@ -33831,10 +33831,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/cloud/aws-iam-key-rotation-skipped.yaml",
- "sha256": "8d0a8e51f512c731c5e600f7f31e9ba210793a0241ab6d27bd55cc4fad68c9c6"
+ "quarantine_reason": "not executable (needs-external-source): wants an access key's creation date. CloudTrail reports key *use* and not key age; the date is in the IAM credential report, which arrives with the AWS posture snapshot (depth plan 2.5).",
+ "sha256": "a18d457a4dae77733cdbb4b1a24795938a42482111a9db3fa866f11316e9fdf9"
},
{
"id": "det-cloud-043",
@@ -35213,10 +35214,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/cloud/aws-cloudfront-origin-changed.yaml",
- "sha256": "8e47eb25a342cc5028d93e531d35b98576fd146e849e9528bcfb384a243f47e2"
+ "quarantine_reason": "not executable (needs-external-source): wants a distribution's creation date, which is configuration state rather than an event, and arrives with the AWS posture snapshot (depth plan 2.5).",
+ "sha256": "e3e1416f71dd1cadea373cb7396fbba0fbef8862b167fb86495a5e10a94e1d44"
},
{
"id": "det-cloud-098",
@@ -41549,10 +41551,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/endpoint/win-dcsync-via-replication.yaml",
- "sha256": "d24ef85ca4947366004a5d44c0237cd781d577b98843f807702ef15f21d784ba"
+ "quarantine_reason": "not executable (needs-inventory): asks whether the replicating account is a domain controller. That needs a directory inventory of DC machine accounts; `identity_nodes` carries a privilege tier per principal and no such flag.",
+ "sha256": "6e91e68fc557d499900940313406cf94950ae9e4b8aeb6e8d2d91342b8e44123"
},
{
"id": "det-endpoint-102",
@@ -42474,10 +42477,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/endpoint/linux-bashrc-modified-by-other-user.yaml",
- "sha256": "c1b4c05c210199bc384ae90694227b5e84f43bddb91d79473caf5c58ace568bd"
+ "quarantine_reason": "not executable (needs-connector-field): compares the acting uid with the file owner's uid. The acting uid is emitted and the owner's is not, so the comparison resolves to nothing. An EDR file-write event carries the owner; surfacing it is connector work.",
+ "sha256": "cc5992ffa893502cd43175a97ec74d6a2f044e4a28e20b9adca0b99327a6313a"
},
{
"id": "det-endpoint-139",
@@ -42499,10 +42503,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/endpoint/linux-zshrc-modified-by-other-user.yaml",
- "sha256": "e9afa054a50fb70992783ad32ea39f15cf98ae337bc84c5ca81cd9df63719203"
+ "quarantine_reason": "not executable (needs-connector-field): compares the acting uid with the file owner's uid, and no source emits the owner. Same gap as the bashrc rule beside it.",
+ "sha256": "3cd1e6e8fb88bcb344eb9ceb83e58a39709663fd4a8df9c817bd9e051f2ebc9a"
},
{
"id": "det-endpoint-140",
@@ -42977,7 +42982,7 @@
"enabled": true,
"executable": true,
"path": "detections/endpoint/linux-suid-binary-create.yaml",
- "sha256": "ddea068f5f1a32a4fd23c79b464192229a6c00e3095a44760677dfec1b9617cd"
+ "sha256": "3d3ca5b7f2181109fee0a0b5955bdaf540989956fff011bfaecd4ecc967e6570"
},
{
"id": "det-endpoint-159",
@@ -43002,7 +43007,7 @@
"enabled": true,
"executable": true,
"path": "detections/endpoint/linux-guid-binary-create.yaml",
- "sha256": "e72010b019d39c76506e7e699487041c004008e4d122c4664d19643e133e5f2c"
+ "sha256": "02ae1b8931fa60b6bd38d23f5646258168151d0d16f57dcc9354360f1fc120de"
},
{
"id": "det-endpoint-160",
@@ -44399,10 +44404,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/endpoint/macos-keychain-db-read-by-other-uid.yaml",
- "sha256": "7bb0f15de4393bb1494efd9b97919bf49b98857195bff4c3b5dfd98342070b48"
+ "quarantine_reason": "not executable (needs-connector-field): compares the reading uid with the keychain owner's uid, and no source emits the owner.",
+ "sha256": "4dedf02467737bad5de7bd0f4dab2012df27751364714a0c942d36c6c6c5c713"
},
{
"id": "det-endpoint-216",
@@ -44703,7 +44709,7 @@
"enabled": true,
"executable": true,
"path": "detections/endpoint/macos-hyperkit-launch-untrusted.yaml",
- "sha256": "90613cf5832155b845a99380e7a52c2e08773fb5c3a61603338b80017d3f2743"
+ "sha256": "60416ddeb3d1c53c5f8f25f105052145e7d1480f4a20b308a6e4517332196762"
},
{
"id": "det-endpoint-228",
@@ -48040,10 +48046,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-fido2-key-added-then-removed-old.yaml",
- "sha256": "f4b13b89ad3565bb9643193e55c5c21cf8cbc8a465938242fb42a8a285cafada"
+ "quarantine_reason": "not executable (needs-external-source): wants the enrolment age of the factor being removed. The IdP holds it on the factor record; no audit event carries it.",
+ "sha256": "ee688d0398ead4dc67662644061a6d07b5f8485ad01062b4d98bd59f560ea07b"
},
{
"id": "det-identity-056",
@@ -48140,10 +48147,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-session-timeout-extended-priv.yaml",
- "sha256": "9a373aa9a6a175b68302072000507e9a6e40a369fbfc6121058417579a8f2940"
+ "quarantine_reason": "not executable (needs-inventory): asks whether a policy's scope is privileged. The scope is emitted; what is missing is a list of which scopes are privileged, which no source provides and no tenant configures today.",
+ "sha256": "4545d4c8552f22c40a58d15da4a9ef527a4aaf9284849583f4f4499783ea104d"
},
{
"id": "det-identity-060",
@@ -48391,10 +48399,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-pim-activation-self-approval.yaml",
- "sha256": "62d06042576f1213050be6e25f138581b6e7b4a3a994c8fe186ce6a8e4d88573"
+ "quarantine_reason": "not executable (needs-connector-field): compares the approver with the requester. The requester is emitted as `actor`; no identity source in this tree emits the approver, so a self-approval cannot be distinguished from an approval.",
+ "sha256": "e447491a06018da66c246a6d81d3f857e617e322c0f891fecb779fb29447784a"
},
{
"id": "det-identity-070",
@@ -48467,10 +48476,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-svc-account-interactive-login.yaml",
- "sha256": "59f246821383293c8a4e4a5fb84648022d9e32756a8145cd32c803e6e2dce423"
+ "quarantine_reason": "not executable (needs-connector-field): asks whether a logon was interactive. Windows 4624 answers it in `LogonType` (2, 10 and 11 are interactive) and no source lifts that field, so the boolean has nothing to derive from.",
+ "sha256": "36104ddc9687ef897c0fdf63c78e91c6ac90202ff64b21e036943129167aa58a"
},
{
"id": "det-identity-073",
@@ -48542,10 +48552,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-app-impersonation-priv.yaml",
- "sha256": "f992053a0951136374eb33d81408673952bfe564259ca5ddb1451af9ae38a75e"
+ "quarantine_reason": "not executable (needs-connector-field): asks whether the impersonated user is privileged. The privilege lookup now exists; the impersonated user is not emitted by any source, so there is no subject to look up.",
+ "sha256": "d4d3ff1d175a6491ef9a7a39f351e0a4ee2e0f369ea9a0194d3bc3afcdb242c9"
},
{
"id": "det-identity-076",
@@ -48719,10 +48730,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-ad-gpo-modified-priv.yaml",
- "sha256": "90517daae40443307bbdcf69ec0ca01c9eb2dbeb1af46513598709268ae4cd28"
+ "quarantine_reason": "not executable (needs-inventory): asks whether a modified GPO is linked to a privileged scope. Neither the GPO link nor a list of privileged scopes exists here.",
+ "sha256": "02a7875f223ebfe32a313efd55ecfa1c1518a365a86a87f4036696373bb1a084"
},
{
"id": "det-identity-083",
@@ -49096,10 +49108,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-ntlm-relay-indicator.yaml",
- "sha256": "3e12d23fd6267f36a2f42f3e5ca73a29ea3b47fac29864e0d62465716e2fee21"
+ "quarantine_reason": "not executable (needs-connector-field): compares the workstation named in the logon with the address it came from. Windows 4624 carries both as `WorkstationName` and `IpAddress`; neither is lifted as a top-level field, so neither side of the comparison resolves.",
+ "sha256": "ab1ba1a108b662d6a0b02fc1c3c0adbe32c1662bafdd6a6998722f0f502e53e0"
},
{
"id": "det-identity-098",
@@ -49171,10 +49184,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-pass-the-ticket-anomalous-tgt-source.yaml",
- "sha256": "79920bed0bc5489e7955b08475d27da9be4b2bafe3cab3090ab899527a5ee339"
+ "quarantine_reason": "not executable (needs-connector-field): compares the address a Kerberos ticket was issued to with the address using it. That needs two events joined on the ticket, which is a correlation, and neither address is emitted today.",
+ "sha256": "96b243fbd17c39386292761f18de494a66ff10ccd91d2d7564a8a25a99385761"
},
{
"id": "det-identity-101",
@@ -50882,10 +50896,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/network/newly-registered-domain-traffic.yaml",
- "sha256": "057aec7cd6a3ee71569f7bed994faacb67605c916337cb6dbf6e10621e74eac4"
+ "quarantine_reason": "not executable (needs-external-source): wants a domain's registration age, which comes from RDAP or WHOIS. Nothing in this tree queries either, and answering it from when this deployment first saw the domain would fire on most of the internet.",
+ "sha256": "4359781d0f7bbce674a247135b2a7300420761b7064884cc362fda7b32ac6a26"
},
{
"id": "det-network-009",
@@ -51464,10 +51479,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/network/net-dns-newly-registered-domain.yaml",
- "sha256": "9ee0d5678db29c8113018cc0b8311a624287f21827183910d386e3b2fd36646b"
+ "quarantine_reason": "not executable (needs-external-source): wants a domain's registration age from RDAP or WHOIS, which nothing here queries. Same gap as the proxy-side rule beside it.",
+ "sha256": "5bf1bd34f9d1a37e0e49e2cf8af21f6155c6b9d5eff24b0864598a01e767f373"
},
{
"id": "det-network-032",
diff --git a/apps/web/src/data/corpus-stats.json b/apps/web/src/data/corpus-stats.json
index 9bbc8e5de..e61dd390d 100644
--- a/apps/web/src/data/corpus-stats.json
+++ b/apps/web/src/data/corpus-stats.json
@@ -1,19 +1,19 @@
{
- "executable": 2529,
+ "executable": 2511,
"windowed": 72,
"onDisk": 6991,
- "quarantined": 4462,
+ "quarantined": 4480,
"plugins": 80,
"playbookPacks": 62,
"playbooks": 87,
"marketplaceItems": 7158,
"categories": {
- "application": 78,
- "cloud": 237,
+ "application": 76,
+ "cloud": 235,
"data-exfil": 13,
- "endpoint": 1982,
- "identity": 154,
- "network": 65
+ "endpoint": 1978,
+ "identity": 146,
+ "network": 63
},
"generatedFrom": [
"services/fusion/app/data/detection_ruleset.json",
diff --git a/detections/application/api-permission-bypass-idor.yaml b/detections/application/api-permission-bypass-idor.yaml
index 977c88853..2fcb35a67 100644
--- a/detections/application/api-permission-bypass-idor.yaml
+++ b/detections/application/api-permission-bypass-idor.yaml
@@ -18,7 +18,10 @@ detection:
false_positives:
- Admin-impersonation operations (must originate from internal tooling)
playbook: tpl-data-exposure
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-connector-field): compares the object''s owning tenant with
+ the caller''s, and no application source in this tree surfaces either. The engine can compare two fields
+ of one event; it cannot compare two fields that are not there.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/application/app-npm-package-publish-new-account.yaml b/detections/application/app-npm-package-publish-new-account.yaml
index 6877029da..6a2534c38 100644
--- a/detections/application/app-npm-package-publish-new-account.yaml
+++ b/detections/application/app-npm-package-publish-new-account.yaml
@@ -19,7 +19,10 @@ detection:
false_positives:
- New maintainer onboarding
playbook: tpl-supply-chain
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-external-source): wants the publisher account''s age at the
+ registry. The first-seen store knows when this deployment first saw the account, which is a different
+ number: substituting it would fire on every publisher a quiet tenant has not seen before.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/cloud/aws-cloudfront-origin-changed.yaml b/detections/cloud/aws-cloudfront-origin-changed.yaml
index 834f45571..c2c8f0c32 100644
--- a/detections/cloud/aws-cloudfront-origin-changed.yaml
+++ b/detections/cloud/aws-cloudfront-origin-changed.yaml
@@ -19,7 +19,9 @@ detection:
false_positives:
- Approved migration
playbook: tpl-impact
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-external-source): wants a distribution''s creation date, which
+ is configuration state rather than an event, and arrives with the AWS posture snapshot (depth plan 2.5).'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/cloud/aws-iam-key-rotation-skipped.yaml b/detections/cloud/aws-iam-key-rotation-skipped.yaml
index 69bee66a4..d1017d448 100644
--- a/detections/cloud/aws-iam-key-rotation-skipped.yaml
+++ b/detections/cloud/aws-iam-key-rotation-skipped.yaml
@@ -19,7 +19,10 @@ false_positives:
- Sanctioned automation/IaC pipeline
- Service-account key intentionally pinned
playbook: tpl-credential-hygiene
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-external-source): wants an access key''s creation date. CloudTrail
+ reports key *use* and not key age; the date is in the IAM credential report, which arrives with the
+ AWS posture snapshot (depth plan 2.5).'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/endpoint/linux-bashrc-modified-by-other-user.yaml b/detections/endpoint/linux-bashrc-modified-by-other-user.yaml
index 1d99ab369..3fc5aa502 100644
--- a/detections/endpoint/linux-bashrc-modified-by-other-user.yaml
+++ b/detections/endpoint/linux-bashrc-modified-by-other-user.yaml
@@ -20,7 +20,10 @@ false_positives:
- Sanctioned automation/IaC pipeline
- Configuration mgmt managed dotfiles
playbook: tpl-persistence
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-connector-field): compares the acting uid with the file owner''s
+ uid. The acting uid is emitted and the owner''s is not, so the comparison resolves to nothing. An EDR
+ file-write event carries the owner; surfacing it is connector work.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/endpoint/linux-guid-binary-create.yaml b/detections/endpoint/linux-guid-binary-create.yaml
index 97404aaa8..70c3c4481 100644
--- a/detections/endpoint/linux-guid-binary-create.yaml
+++ b/detections/endpoint/linux-guid-binary-create.yaml
@@ -15,7 +15,7 @@ detection:
condition: |
syscall == "fchmod"
AND mode_octal CONTAINS "2"
- AND path_not STARTSWITH_ANY ["/usr/bin", "/usr/sbin", "/bin", "/sbin"]
+ AND path NOT STARTSWITH_ANY ["/usr/bin", "/usr/sbin", "/bin", "/sbin"]
false_positives:
- Approved deploy
playbook: tpl-privilege-escalation
diff --git a/detections/endpoint/linux-suid-binary-create.yaml b/detections/endpoint/linux-suid-binary-create.yaml
index c65077358..155bb58ba 100644
--- a/detections/endpoint/linux-suid-binary-create.yaml
+++ b/detections/endpoint/linux-suid-binary-create.yaml
@@ -15,7 +15,7 @@ detection:
condition: |
syscall == "fchmod"
AND mode_octal CONTAINS "4"
- AND path_not STARTSWITH_ANY ["/usr/bin", "/usr/sbin", "/bin", "/sbin"]
+ AND path NOT STARTSWITH_ANY ["/usr/bin", "/usr/sbin", "/bin", "/sbin"]
false_positives:
- Approved deploy of internal tool
playbook: tpl-privilege-escalation
diff --git a/detections/endpoint/linux-zshrc-modified-by-other-user.yaml b/detections/endpoint/linux-zshrc-modified-by-other-user.yaml
index eb2cf159b..9a766f6d9 100644
--- a/detections/endpoint/linux-zshrc-modified-by-other-user.yaml
+++ b/detections/endpoint/linux-zshrc-modified-by-other-user.yaml
@@ -19,7 +19,9 @@ detection:
false_positives:
- Sanctioned automation/IaC pipeline
playbook: tpl-persistence
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-connector-field): compares the acting uid with the file owner''s
+ uid, and no source emits the owner. Same gap as the bashrc rule beside it.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/endpoint/macos-hyperkit-launch-untrusted.yaml b/detections/endpoint/macos-hyperkit-launch-untrusted.yaml
index 2cd644a15..136a8ce4b 100644
--- a/detections/endpoint/macos-hyperkit-launch-untrusted.yaml
+++ b/detections/endpoint/macos-hyperkit-launch-untrusted.yaml
@@ -15,7 +15,7 @@ detection:
condition: |
event_type == "exec"
AND process_path CONTAINS "/hyperkit"
- AND process_path_not STARTSWITH_ANY ["/Applications/", "/usr/local/bin/"]
+ AND process_path NOT STARTSWITH_ANY ["/Applications/", "/usr/local/bin/"]
false_positives:
- Custom virtualisation tooling
playbook: tpl-execution
diff --git a/detections/endpoint/macos-keychain-db-read-by-other-uid.yaml b/detections/endpoint/macos-keychain-db-read-by-other-uid.yaml
index f3acc1469..030285b73 100644
--- a/detections/endpoint/macos-keychain-db-read-by-other-uid.yaml
+++ b/detections/endpoint/macos-keychain-db-read-by-other-uid.yaml
@@ -19,7 +19,9 @@ detection:
false_positives:
- DFIR responder running a documented playbook
playbook: tpl-credential-access
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-connector-field): compares the reading uid with the keychain
+ owner''s uid, and no source emits the owner.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/endpoint/win-dcsync-via-replication.yaml b/detections/endpoint/win-dcsync-via-replication.yaml
index 7a1dfa759..f552c8c72 100644
--- a/detections/endpoint/win-dcsync-via-replication.yaml
+++ b/detections/endpoint/win-dcsync-via-replication.yaml
@@ -19,7 +19,10 @@ detection:
false_positives:
- AD migration tool with replication granted
playbook: tpl-credential-access
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-inventory): asks whether the replicating account is a domain
+ controller. That needs a directory inventory of DC machine accounts; `identity_nodes` carries a privilege
+ tier per principal and no such flag.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/fixtures/negative/linux-guid-binary-create.json b/detections/fixtures/negative/linux-guid-binary-create.json
index 0082527b0..1e2349016 100644
--- a/detections/fixtures/negative/linux-guid-binary-create.json
+++ b/detections/fixtures/negative/linux-guid-binary-create.json
@@ -1,5 +1,5 @@
{
"mode_octal": "prefix-2-suffix",
- "path_not": "/usr/bin-tail",
+ "path": "__benign__",
"syscall": "openat"
}
diff --git a/detections/fixtures/negative/linux-suid-binary-create.json b/detections/fixtures/negative/linux-suid-binary-create.json
index d92a019a9..b8c7e3487 100644
--- a/detections/fixtures/negative/linux-suid-binary-create.json
+++ b/detections/fixtures/negative/linux-suid-binary-create.json
@@ -1,5 +1,5 @@
{
"mode_octal": "prefix-4-suffix",
- "path_not": "/usr/bin-tail",
+ "path": "__benign__",
"syscall": "openat"
}
diff --git a/detections/fixtures/negative/macos-hyperkit-launch-untrusted.json b/detections/fixtures/negative/macos-hyperkit-launch-untrusted.json
index 7da5070c4..5230c8199 100644
--- a/detections/fixtures/negative/macos-hyperkit-launch-untrusted.json
+++ b/detections/fixtures/negative/macos-hyperkit-launch-untrusted.json
@@ -1,5 +1,4 @@
{
"event_type": "http_request",
- "process_path": "prefix-/hyperkit-suffix",
- "process_path_not": "/Applications/-tail"
+ "process_path": "/hyperkit"
}
diff --git a/detections/fixtures/positive/linux-guid-binary-create.json b/detections/fixtures/positive/linux-guid-binary-create.json
index e8033fa1b..039b3b418 100644
--- a/detections/fixtures/positive/linux-guid-binary-create.json
+++ b/detections/fixtures/positive/linux-guid-binary-create.json
@@ -1,5 +1,5 @@
{
"mode_octal": "prefix-2-suffix",
- "path_not": "/usr/bin-tail",
+ "path": "__benign__",
"syscall": "fchmod"
}
diff --git a/detections/fixtures/positive/linux-suid-binary-create.json b/detections/fixtures/positive/linux-suid-binary-create.json
index 1fd7cdd0b..826715107 100644
--- a/detections/fixtures/positive/linux-suid-binary-create.json
+++ b/detections/fixtures/positive/linux-suid-binary-create.json
@@ -1,5 +1,5 @@
{
"mode_octal": "prefix-4-suffix",
- "path_not": "/usr/bin-tail",
+ "path": "__benign__",
"syscall": "fchmod"
}
diff --git a/detections/fixtures/positive/macos-hyperkit-launch-untrusted.json b/detections/fixtures/positive/macos-hyperkit-launch-untrusted.json
index ab6053f20..9de6fb00f 100644
--- a/detections/fixtures/positive/macos-hyperkit-launch-untrusted.json
+++ b/detections/fixtures/positive/macos-hyperkit-launch-untrusted.json
@@ -1,5 +1,4 @@
{
"event_type": "exec",
- "process_path": "prefix-/hyperkit-suffix",
- "process_path_not": "/Applications/-tail"
+ "process_path": "/hyperkit"
}
diff --git a/detections/identity/ident-ad-gpo-modified-priv.yaml b/detections/identity/ident-ad-gpo-modified-priv.yaml
index bd756170b..c97989089 100644
--- a/detections/identity/ident-ad-gpo-modified-priv.yaml
+++ b/detections/identity/ident-ad-gpo-modified-priv.yaml
@@ -18,7 +18,9 @@ detection:
false_positives:
- Approved GPO refactor
playbook: tpl-defense-evasion
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-inventory): asks whether a modified GPO is linked to a privileged
+ scope. Neither the GPO link nor a list of privileged scopes exists here.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/identity/ident-app-impersonation-priv.yaml b/detections/identity/ident-app-impersonation-priv.yaml
index 176371707..0b27b063b 100644
--- a/detections/identity/ident-app-impersonation-priv.yaml
+++ b/detections/identity/ident-app-impersonation-priv.yaml
@@ -19,7 +19,10 @@ detection:
false_positives:
- Approved IaaS automation
playbook: tpl-credential-access
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-connector-field): asks whether the impersonated user is privileged.
+ The privilege lookup now exists; the impersonated user is not emitted by any source, so there is no
+ subject to look up.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/identity/ident-fido2-key-added-then-removed-old.yaml b/detections/identity/ident-fido2-key-added-then-removed-old.yaml
index f54d037c2..39156e9c9 100644
--- a/detections/identity/ident-fido2-key-added-then-removed-old.yaml
+++ b/detections/identity/ident-fido2-key-added-then-removed-old.yaml
@@ -19,7 +19,9 @@ detection:
false_positives:
- Documented break-glass procedure
playbook: tpl-persistence
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-external-source): wants the enrolment age of the factor being
+ removed. The IdP holds it on the factor record; no audit event carries it.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/identity/ident-ntlm-relay-indicator.yaml b/detections/identity/ident-ntlm-relay-indicator.yaml
index 3b3935335..a79890c29 100644
--- a/detections/identity/ident-ntlm-relay-indicator.yaml
+++ b/detections/identity/ident-ntlm-relay-indicator.yaml
@@ -19,7 +19,10 @@ detection:
false_positives:
- Authorised red-team or penetration test
playbook: tpl-credential-access
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-connector-field): compares the workstation named in the logon
+ with the address it came from. Windows 4624 carries both as `WorkstationName` and `IpAddress`; neither
+ is lifted as a top-level field, so neither side of the comparison resolves.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/identity/ident-pass-the-ticket-anomalous-tgt-source.yaml b/detections/identity/ident-pass-the-ticket-anomalous-tgt-source.yaml
index 657471f62..cd5392036 100644
--- a/detections/identity/ident-pass-the-ticket-anomalous-tgt-source.yaml
+++ b/detections/identity/ident-pass-the-ticket-anomalous-tgt-source.yaml
@@ -19,7 +19,10 @@ detection:
false_positives:
- NAT / proxy environment
playbook: tpl-lateral-movement
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-connector-field): compares the address a Kerberos ticket was
+ issued to with the address using it. That needs two events joined on the ticket, which is a correlation,
+ and neither address is emitted today.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/identity/ident-pim-activation-self-approval.yaml b/detections/identity/ident-pim-activation-self-approval.yaml
index b39f2bb0f..3dd804838 100644
--- a/detections/identity/ident-pim-activation-self-approval.yaml
+++ b/detections/identity/ident-pim-activation-self-approval.yaml
@@ -18,7 +18,10 @@ detection:
false_positives:
- Documented break-glass procedure
playbook: tpl-privilege-escalation
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-connector-field): compares the approver with the requester.
+ The requester is emitted as `actor`; no identity source in this tree emits the approver, so a self-approval
+ cannot be distinguished from an approval.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/identity/ident-session-timeout-extended-priv.yaml b/detections/identity/ident-session-timeout-extended-priv.yaml
index 09b098d1e..2c041d6b1 100644
--- a/detections/identity/ident-session-timeout-extended-priv.yaml
+++ b/detections/identity/ident-session-timeout-extended-priv.yaml
@@ -19,7 +19,10 @@ detection:
false_positives:
- Confirmed test/tuning activity
playbook: tpl-defense-evasion
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-inventory): asks whether a policy''s scope is privileged. The
+ scope is emitted; what is missing is a list of which scopes are privileged, which no source provides
+ and no tenant configures today.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/identity/ident-svc-account-interactive-login.yaml b/detections/identity/ident-svc-account-interactive-login.yaml
index 79dd67ea9..3a30127a1 100644
--- a/detections/identity/ident-svc-account-interactive-login.yaml
+++ b/detections/identity/ident-svc-account-interactive-login.yaml
@@ -19,7 +19,10 @@ detection:
false_positives:
- Approved automation runner using human UI
playbook: tpl-credential-access
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-connector-field): asks whether a logon was interactive. Windows
+ 4624 answers it in `LogonType` (2, 10 and 11 are interactive) and no source lifts that field, so the
+ boolean has nothing to derive from.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/network/net-dns-newly-registered-domain.yaml b/detections/network/net-dns-newly-registered-domain.yaml
index 7d544028d..f78dc57a0 100644
--- a/detections/network/net-dns-newly-registered-domain.yaml
+++ b/detections/network/net-dns-newly-registered-domain.yaml
@@ -18,7 +18,9 @@ detection:
false_positives:
- Brand-new SaaS launch (rare; verify business need)
playbook: tpl-command-and-control
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-external-source): wants a domain''s registration age from RDAP
+ or WHOIS, which nothing here queries. Same gap as the proxy-side rule beside it.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/detections/network/newly-registered-domain-traffic.yaml b/detections/network/newly-registered-domain-traffic.yaml
index 6e4ee2199..11cd5c16c 100644
--- a/detections/network/newly-registered-domain-traffic.yaml
+++ b/detections/network/newly-registered-domain-traffic.yaml
@@ -18,7 +18,10 @@ detection:
false_positives:
- Marketing campaigns and event-specific landing pages
playbook: tpl-c2
-enabled: true
+enabled: false
+quarantine_reason: 'not executable (needs-external-source): wants a domain''s registration age, which
+ comes from RDAP or WHOIS. Nothing in this tree queries either, and answering it from when this deployment
+ first saw the domain would fire on most of the internet.'
author: AiSOC
created: '2026-05-03'
modified: '2026-05-03'
diff --git a/docs/detections/truth-table.md b/docs/detections/truth-table.md
index 24cffc82e..33b3f5e2c 100644
--- a/docs/detections/truth-table.md
+++ b/docs/detections/truth-table.md
@@ -20,8 +20,8 @@ lever that reaches the engine is adding a spec and re-running
| metric | count |
|--------|------:|
| rules on disk (total) | 6991 |
-| **executable (loaded by the engine)** | **2529** |
-| non-executable (provenance/coverage only) | 4462 |
+| **executable (loaded by the engine)** | **2511** |
+| non-executable (provenance/coverage only) | 4480 |
| — of which: enabled, but no compiled spec | 0 |
| windowed rules in `windowed_ruleset.json` | 72 |
@@ -55,7 +55,7 @@ were hand-authored in the upstream format to give it one.
| car (imported) | 99 | 0 | 0 |
| chronicle (imported) | 877 | 0 | 0 |
| community | 1 | 0 | 0 |
-| native | 877 | 759 | 0 |
+| native | 877 | 741 | 0 |
| sigma (imported) | 3132 | 1770 | 0 |
| splunk (imported) | 2005 | 0 | 0 |
@@ -63,14 +63,14 @@ were hand-authored in the upstream format to give it one.
| reason | count |
|--------|------:|
-| `enabled: false` | 249 |
+| `enabled: false` | 267 |
| under `_quarantine/` (untranslated on import) | 4213 |
## How to read the README claim
The imported corpus is large (6991 rules on disk) and valuable as a
provenance-tracked ATT&CK-mapped library, but the number that matters
-operationally is **2529 executable rules** — the ones the engine
+operationally is **2511 executable rules** — the ones the engine
loads and fires against live telemetry. The README and marketplace must cite
the executable figure when describing detection *coverage*, and may cite the
on-disk figure only when explicitly describing the imported *library*.
diff --git a/marketplace/curated.json b/marketplace/curated.json
index f57d39ce6..fc74a534a 100644
--- a/marketplace/curated.json
+++ b/marketplace/curated.json
@@ -1,31 +1,31 @@
{
"$schema": "https://aisoc.example.com/schemas/curated/v1.json",
"version": "1.0.0",
- "generated": "2026-10-09T03:35:19Z",
+ "generated": "2026-10-09T04:33:12Z",
"stats": {
- "considered": 2529,
- "eligible": 2518,
- "selected": 490,
+ "considered": 2511,
+ "eligible": 2500,
+ "selected": 489,
"min_per_family": 25,
"min_total": 300,
"quality_floor": 0.55,
- "unique_techniques": 136,
+ "unique_techniques": 137,
"selected_by_tier": {
- "imported": 160,
- "native": 330
+ "imported": 163,
+ "native": 326
},
"selected_by_severity": {
- "critical": 100,
- "high": 266,
+ "critical": 99,
+ "high": 264,
"low": 16,
- "medium": 108
+ "medium": 110
},
"selected_by_category": {
- "application": 31,
- "cloud": 150,
+ "application": 29,
+ "cloud": 152,
"data-exfil": 13,
- "endpoint": 218,
- "identity": 69,
+ "endpoint": 219,
+ "identity": 67,
"network": 9
}
},
@@ -107,7 +107,6 @@
"count": 100,
"rule_ids": [
"det-application-004",
- "det-application-020",
"det-application-075",
"det-application-085",
"det-cloud-047",
@@ -137,9 +136,7 @@
"det-identity-056",
"det-identity-057",
"det-identity-058",
- "det-identity-059",
"det-identity-061",
- "det-identity-075",
"det-identity-088",
"det-identity-101",
"det-identity-105",
@@ -184,9 +181,12 @@
"sigmahq-sigma-48d91a3a-2363-43ba-a456-ca71ac3da5c2",
"sigmahq-sigma-4d78a000-ab52-4564-88a5-7ab5242b20c7",
"sigmahq-sigma-53bb4f7f-48a8-4475-ac30-5a82ddfdf6fc",
+ "sigmahq-sigma-5496ff55-42ec-4369-81cb-00f417029e25",
"sigmahq-sigma-58f4ea09-0fc2-4520-ba18-b85c540b0eaf",
"sigmahq-sigma-5a44727c-3b85-4713-8c44-4401d5499629",
+ "sigmahq-sigma-603c6630-5225-49c1-8047-26c964553e0e",
"sigmahq-sigma-60f6535a-760f-42a9-be3f-c9a0a025906e",
+ "sigmahq-sigma-6393e346-1977-46ef-8987-ad414a145fad",
"sigmahq-sigma-7679d464-4f74-45e2-9e01-ac66c5eb041a",
"sigmahq-sigma-7892ec59-c5bb-496d-8968-e5d210ca3ac4",
"sigmahq-sigma-7ec2c172-dceb-4c10-92c9-87c1881b7e18",
@@ -555,7 +555,7 @@
},
"supply-chain": {
"label": "Supply Chain",
- "count": 41,
+ "count": 40,
"rule_ids": [
"det-application-006",
"det-application-014",
@@ -567,7 +567,6 @@
"det-application-048",
"det-application-049",
"det-application-052",
- "det-application-053",
"det-application-054",
"det-application-057",
"det-application-059",
@@ -719,7 +718,7 @@
"T1071.001": 1,
"T1078": 14,
"T1078.001": 3,
- "T1078.004": 24,
+ "T1078.004": 25,
"T1082": 2,
"T1087": 1,
"T1090": 2,
@@ -732,7 +731,7 @@
"T1102": 2,
"T1105": 1,
"T1106": 2,
- "T1110": 8,
+ "T1110": 10,
"T1110.001": 2,
"T1110.002": 1,
"T1110.004": 1,
@@ -746,14 +745,14 @@
"T1190": 11,
"T1195": 1,
"T1195.001": 1,
- "T1195.002": 7,
+ "T1195.002": 6,
"T1199": 11,
"T1201": 1,
"T1203": 1,
"T1204.002": 1,
"T1210": 4,
"T1211": 1,
- "T1212": 5,
+ "T1212": 4,
"T1218": 4,
"T1219": 2,
"T1484.001": 1,
@@ -767,7 +766,7 @@
"T1505.003": 8,
"T1518": 2,
"T1525": 8,
- "T1528": 6,
+ "T1528": 5,
"T1530": 17,
"T1531": 1,
"T1537": 7,
@@ -785,9 +784,9 @@
"T1552.007": 1,
"T1553.002": 1,
"T1554": 3,
- "T1555": 5,
+ "T1555": 6,
"T1555.003": 3,
- "T1556": 41,
+ "T1556": 40,
"T1556.001": 1,
"T1556.006": 5,
"T1556.007": 1,
@@ -816,6 +815,7 @@
"T1610": 11,
"T1611": 24,
"T1612": 1,
+ "T1621": 1,
"T1685": 2
},
"items": [
@@ -996,31 +996,6 @@
"log_source_product": "kubernetes",
"has_executable_body": true
},
- {
- "id": "det-application-020",
- "name": "API Returned Object Outside Caller's Tenant",
- "path": "detections/application/api-permission-bypass-idor.yaml",
- "tier": "native",
- "source": "core",
- "category": "application",
- "severity": "critical",
- "mitre_techniques": [
- "T1212"
- ],
- "mitre_tactics": [],
- "families": [
- "credential-access"
- ],
- "quality_score": 1.0,
- "quality_notes": [
- "tier:native(+0.5)",
- "mitre_techniques:1(+0.2)",
- "severity:critical(+0.150)",
- "executable(+0.15)"
- ],
- "log_source_product": "app",
- "has_executable_body": true
- },
{
"id": "det-application-023",
"name": "GitLab Runner Registered as Shared Without Approval",
@@ -1224,31 +1199,6 @@
"log_source_product": "github",
"has_executable_body": true
},
- {
- "id": "det-application-053",
- "name": "NPM Package Published From New Account",
- "path": "detections/application/app-npm-package-publish-new-account.yaml",
- "tier": "native",
- "source": "core",
- "category": "application",
- "severity": "high",
- "mitre_techniques": [
- "T1195.002"
- ],
- "mitre_tactics": [],
- "families": [
- "supply-chain"
- ],
- "quality_score": 0.977,
- "quality_notes": [
- "tier:native(+0.5)",
- "mitre_techniques:1(+0.2)",
- "severity:high(+0.128)",
- "executable(+0.15)"
- ],
- "log_source_product": "npm",
- "has_executable_body": true
- },
{
"id": "det-application-054",
"name": "PyPI Package Published From New IP",
@@ -8025,32 +7975,6 @@
"log_source_product": "identity",
"has_executable_body": true
},
- {
- "id": "det-identity-059",
- "name": "Session Timeout Extended For Privileged Group",
- "path": "detections/identity/ident-session-timeout-extended-priv.yaml",
- "tier": "native",
- "source": "core",
- "category": "identity",
- "severity": "medium",
- "mitre_techniques": [
- "T1556"
- ],
- "mitre_tactics": [],
- "families": [
- "credential-access",
- "identity"
- ],
- "quality_score": 0.955,
- "quality_notes": [
- "tier:native(+0.5)",
- "mitre_techniques:1(+0.2)",
- "severity:medium(+0.105)",
- "executable(+0.15)"
- ],
- "log_source_product": "identity",
- "has_executable_body": true
- },
{
"id": "det-identity-061",
"name": "Login From ASN Recently Seen In Credential-Stuffing Activity",
@@ -8102,32 +8026,6 @@
"log_source_product": "identity",
"has_executable_body": true
},
- {
- "id": "det-identity-075",
- "name": "OAuth App Acting As Privileged User",
- "path": "detections/identity/ident-app-impersonation-priv.yaml",
- "tier": "native",
- "source": "core",
- "category": "identity",
- "severity": "high",
- "mitre_techniques": [
- "T1528"
- ],
- "mitre_tactics": [],
- "families": [
- "credential-access",
- "identity"
- ],
- "quality_score": 0.977,
- "quality_notes": [
- "tier:native(+0.5)",
- "mitre_techniques:1(+0.2)",
- "severity:high(+0.128)",
- "executable(+0.15)"
- ],
- "log_source_product": "identity",
- "has_executable_body": true
- },
{
"id": "det-identity-079",
"name": "Domain Admin Group Membership Added (AD)",
@@ -10700,6 +10598,35 @@
"log_source_product": "azure",
"has_executable_body": true
},
+ {
+ "id": "sigmahq-sigma-5496ff55-42ec-4369-81cb-00f417029e25",
+ "name": "Multifactor Authentication Interrupted",
+ "path": "detections/sigma-imports/_quarantine/multifactor-authentication-interrupted.yaml",
+ "tier": "imported",
+ "source": "sigmahq",
+ "category": "cloud",
+ "severity": "medium",
+ "mitre_techniques": [
+ "T1078.004",
+ "T1110",
+ "T1621"
+ ],
+ "mitre_tactics": [],
+ "families": [
+ "credential-access",
+ "cloud",
+ "identity"
+ ],
+ "quality_score": 0.755,
+ "quality_notes": [
+ "tier:imported(+0.3)",
+ "mitre_techniques:3(+0.2)",
+ "severity:medium(+0.105)",
+ "executable(+0.15)"
+ ],
+ "log_source_product": "azure",
+ "has_executable_body": true
+ },
{
"id": "sigmahq-sigma-551d9c1f-816c-445b-a7a6-7a3864720d60",
"name": "Potential Excel.EXE DCOM Lateral Movement Via ActivateMicrosoftApp",
@@ -10903,6 +10830,31 @@
"log_source_product": "windows",
"has_executable_body": true
},
+ {
+ "id": "sigmahq-sigma-603c6630-5225-49c1-8047-26c964553e0e",
+ "name": "Enumerate Credentials from Windows Credential Manager With PowerShell",
+ "path": "detections/sigma-imports/_quarantine/enumerate-credentials-from-windows-credential-manager-with-powershell.yaml",
+ "tier": "imported",
+ "source": "sigmahq",
+ "category": "endpoint",
+ "severity": "medium",
+ "mitre_techniques": [
+ "T1555"
+ ],
+ "mitre_tactics": [],
+ "families": [
+ "credential-access"
+ ],
+ "quality_score": 0.755,
+ "quality_notes": [
+ "tier:imported(+0.3)",
+ "mitre_techniques:1(+0.2)",
+ "severity:medium(+0.105)",
+ "executable(+0.15)"
+ ],
+ "log_source_product": "windows",
+ "has_executable_body": true
+ },
{
"id": "sigmahq-sigma-60bfeac3-0d35-4302-8efb-1dd16f715bc6",
"name": "Suspicious SysAidServer Child",
@@ -11009,6 +10961,32 @@
"log_source_product": "windows",
"has_executable_body": true
},
+ {
+ "id": "sigmahq-sigma-6393e346-1977-46ef-8987-ad414a145fad",
+ "name": "AWS ConsoleLogin Failed Authentication",
+ "path": "detections/sigma-imports/_quarantine/aws-consolelogin-failed-authentication.yaml",
+ "tier": "imported",
+ "source": "sigmahq",
+ "category": "cloud",
+ "severity": "medium",
+ "mitre_techniques": [
+ "T1110"
+ ],
+ "mitre_tactics": [],
+ "families": [
+ "credential-access",
+ "cloud"
+ ],
+ "quality_score": 0.755,
+ "quality_notes": [
+ "tier:imported(+0.3)",
+ "mitre_techniques:1(+0.2)",
+ "severity:medium(+0.105)",
+ "executable(+0.15)"
+ ],
+ "log_source_product": "aws",
+ "has_executable_body": true
+ },
{
"id": "sigmahq-sigma-639c9081-f482-47d3-a0bd-ddee3d4ecd76",
"name": "All Backups Deleted Via Wbadmin.EXE",
diff --git a/marketplace/index.json b/marketplace/index.json
index ec51bfa25..d85ef41a6 100644
--- a/marketplace/index.json
+++ b/marketplace/index.json
@@ -1,7 +1,7 @@
{
"$schema": "https://example.com/schemas/marketplace/v1.json",
"version": "1.0.0",
- "generated": "2026-10-09T03:35:13Z",
+ "generated": "2026-10-09T04:33:06Z",
"categories": [
{
"id": "playbooks",
@@ -38,14 +38,14 @@
"imported": 6113,
"stable": 877
},
- "executable": 2696,
- "quarantined": 4462,
- "executable_detections": 2529,
- "reference_only_detections": 4462,
+ "executable": 2678,
+ "quarantined": 4480,
+ "executable_detections": 2511,
+ "reference_only_detections": 4480,
"not_a_detection": 167
},
"mitre_coverage": {
- "unique_techniques": 389,
+ "unique_techniques": 387,
"techniques": {
"T1001.003": 1,
"T1003": 21,
@@ -54,7 +54,7 @@
"T1003.003": 21,
"T1003.004": 10,
"T1003.005": 8,
- "T1003.006": 7,
+ "T1003.006": 6,
"T1003.008": 1,
"T1005": 7,
"T1007": 2,
@@ -137,15 +137,15 @@
"T1070.005": 4,
"T1070.006": 1,
"T1071": 7,
- "T1071.001": 14,
- "T1071.004": 11,
+ "T1071.001": 13,
+ "T1071.004": 10,
"T1072": 4,
"T1074.001": 4,
- "T1078": 50,
+ "T1078": 49,
"T1078.001": 5,
"T1078.002": 7,
"T1078.003": 2,
- "T1078.004": 47,
+ "T1078.004": 45,
"T1082": 13,
"T1083": 9,
"T1087": 10,
@@ -210,7 +210,7 @@
"T1190": 39,
"T1195": 3,
"T1195.001": 1,
- "T1195.002": 8,
+ "T1195.002": 7,
"T1197": 13,
"T1199": 12,
"T1200": 3,
@@ -224,7 +224,7 @@
"T1207": 2,
"T1210": 4,
"T1211": 4,
- "T1212": 5,
+ "T1212": 4,
"T1213": 2,
"T1216": 8,
"T1216.001": 2,
@@ -249,7 +249,7 @@
"T1404": 2,
"T1482": 14,
"T1484": 1,
- "T1484.001": 8,
+ "T1484.001": 7,
"T1484.002": 2,
"T1485": 29,
"T1486": 11,
@@ -271,7 +271,7 @@
"T1518.001": 2,
"T1525": 8,
"T1526": 2,
- "T1528": 9,
+ "T1528": 8,
"T1529": 3,
"T1530": 27,
"T1531": 7,
@@ -289,7 +289,6 @@
"T1546.001": 4,
"T1546.002": 1,
"T1546.003": 11,
- "T1546.004": 2,
"T1546.007": 1,
"T1546.008": 5,
"T1546.009": 1,
@@ -315,7 +314,7 @@
"T1550": 5,
"T1550.001": 2,
"T1550.002": 6,
- "T1550.003": 4,
+ "T1550.003": 3,
"T1550.004": 1,
"T1552": 3,
"T1552.001": 17,
@@ -331,18 +330,18 @@
"T1553.005": 5,
"T1554": 3,
"T1555": 8,
- "T1555.001": 5,
+ "T1555.001": 4,
"T1555.003": 9,
"T1555.004": 3,
"T1555.005": 1,
- "T1556": 43,
+ "T1556": 42,
"T1556.001": 1,
"T1556.003": 1,
"T1556.004": 1,
- "T1556.006": 16,
+ "T1556.006": 15,
"T1556.007": 1,
"T1557": 2,
- "T1557.001": 13,
+ "T1557.001": 12,
"T1557.002": 3,
"T1557.003": 1,
"T1558": 3,
@@ -371,7 +370,6 @@
"T1564.008": 1,
"T1565": 2,
"T1565.001": 3,
- "T1565.002": 1,
"T1566": 5,
"T1566.001": 15,
"T1566.002": 2,
@@ -437,7 +435,7 @@
"T1686.003": 10,
"T1689": 1
},
- "total_with_mitre": 2398,
+ "total_with_mitre": 2380,
"unique_techniques_all_rules": 493,
"techniques_all_rules": {
"T1001.003": 2,
@@ -31112,10 +31110,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/application/api-permission-bypass-idor.yaml",
- "sha256": "907beebb04b60d1689cfbab9e63429968a56e4e59b580a9837feb62e6ddebc7a"
+ "quarantine_reason": "not executable (needs-connector-field): compares the object's owning tenant with the caller's, and no application source in this tree surfaces either. The engine can compare two fields of one event; it cannot compare two fields that are not there.",
+ "sha256": "3f757cd10acefaf0b218f68cd8f303aef0f8f9570eb30887494e602a8d28a41d"
},
{
"id": "det-application-021",
@@ -31939,10 +31938,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/application/app-npm-package-publish-new-account.yaml",
- "sha256": "54cefdefd2ce54c8dc2a82f3f8ab0ce19663f18acec4e1c515b85fdfde352a60"
+ "quarantine_reason": "not executable (needs-external-source): wants the publisher account's age at the registry. The first-seen store knows when this deployment first saw the account, which is a different number: substituting it would fire on every publisher a quiet tenant has not seen before.",
+ "sha256": "636bbcd6478dc775efa4bca041e41f09de50c63bfaa801e7d5f66fd1383de7b6"
},
{
"id": "det-application-054",
@@ -33831,10 +33831,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/cloud/aws-iam-key-rotation-skipped.yaml",
- "sha256": "8d0a8e51f512c731c5e600f7f31e9ba210793a0241ab6d27bd55cc4fad68c9c6"
+ "quarantine_reason": "not executable (needs-external-source): wants an access key's creation date. CloudTrail reports key *use* and not key age; the date is in the IAM credential report, which arrives with the AWS posture snapshot (depth plan 2.5).",
+ "sha256": "a18d457a4dae77733cdbb4b1a24795938a42482111a9db3fa866f11316e9fdf9"
},
{
"id": "det-cloud-043",
@@ -35213,10 +35214,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/cloud/aws-cloudfront-origin-changed.yaml",
- "sha256": "8e47eb25a342cc5028d93e531d35b98576fd146e849e9528bcfb384a243f47e2"
+ "quarantine_reason": "not executable (needs-external-source): wants a distribution's creation date, which is configuration state rather than an event, and arrives with the AWS posture snapshot (depth plan 2.5).",
+ "sha256": "e3e1416f71dd1cadea373cb7396fbba0fbef8862b167fb86495a5e10a94e1d44"
},
{
"id": "det-cloud-098",
@@ -41549,10 +41551,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/endpoint/win-dcsync-via-replication.yaml",
- "sha256": "d24ef85ca4947366004a5d44c0237cd781d577b98843f807702ef15f21d784ba"
+ "quarantine_reason": "not executable (needs-inventory): asks whether the replicating account is a domain controller. That needs a directory inventory of DC machine accounts; `identity_nodes` carries a privilege tier per principal and no such flag.",
+ "sha256": "6e91e68fc557d499900940313406cf94950ae9e4b8aeb6e8d2d91342b8e44123"
},
{
"id": "det-endpoint-102",
@@ -42474,10 +42477,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/endpoint/linux-bashrc-modified-by-other-user.yaml",
- "sha256": "c1b4c05c210199bc384ae90694227b5e84f43bddb91d79473caf5c58ace568bd"
+ "quarantine_reason": "not executable (needs-connector-field): compares the acting uid with the file owner's uid. The acting uid is emitted and the owner's is not, so the comparison resolves to nothing. An EDR file-write event carries the owner; surfacing it is connector work.",
+ "sha256": "cc5992ffa893502cd43175a97ec74d6a2f044e4a28e20b9adca0b99327a6313a"
},
{
"id": "det-endpoint-139",
@@ -42499,10 +42503,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/endpoint/linux-zshrc-modified-by-other-user.yaml",
- "sha256": "e9afa054a50fb70992783ad32ea39f15cf98ae337bc84c5ca81cd9df63719203"
+ "quarantine_reason": "not executable (needs-connector-field): compares the acting uid with the file owner's uid, and no source emits the owner. Same gap as the bashrc rule beside it.",
+ "sha256": "3cd1e6e8fb88bcb344eb9ceb83e58a39709663fd4a8df9c817bd9e051f2ebc9a"
},
{
"id": "det-endpoint-140",
@@ -42977,7 +42982,7 @@
"enabled": true,
"executable": true,
"path": "detections/endpoint/linux-suid-binary-create.yaml",
- "sha256": "ddea068f5f1a32a4fd23c79b464192229a6c00e3095a44760677dfec1b9617cd"
+ "sha256": "3d3ca5b7f2181109fee0a0b5955bdaf540989956fff011bfaecd4ecc967e6570"
},
{
"id": "det-endpoint-159",
@@ -43002,7 +43007,7 @@
"enabled": true,
"executable": true,
"path": "detections/endpoint/linux-guid-binary-create.yaml",
- "sha256": "e72010b019d39c76506e7e699487041c004008e4d122c4664d19643e133e5f2c"
+ "sha256": "02ae1b8931fa60b6bd38d23f5646258168151d0d16f57dcc9354360f1fc120de"
},
{
"id": "det-endpoint-160",
@@ -44399,10 +44404,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/endpoint/macos-keychain-db-read-by-other-uid.yaml",
- "sha256": "7bb0f15de4393bb1494efd9b97919bf49b98857195bff4c3b5dfd98342070b48"
+ "quarantine_reason": "not executable (needs-connector-field): compares the reading uid with the keychain owner's uid, and no source emits the owner.",
+ "sha256": "4dedf02467737bad5de7bd0f4dab2012df27751364714a0c942d36c6c6c5c713"
},
{
"id": "det-endpoint-216",
@@ -44703,7 +44709,7 @@
"enabled": true,
"executable": true,
"path": "detections/endpoint/macos-hyperkit-launch-untrusted.yaml",
- "sha256": "90613cf5832155b845a99380e7a52c2e08773fb5c3a61603338b80017d3f2743"
+ "sha256": "60416ddeb3d1c53c5f8f25f105052145e7d1480f4a20b308a6e4517332196762"
},
{
"id": "det-endpoint-228",
@@ -48040,10 +48046,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-fido2-key-added-then-removed-old.yaml",
- "sha256": "f4b13b89ad3565bb9643193e55c5c21cf8cbc8a465938242fb42a8a285cafada"
+ "quarantine_reason": "not executable (needs-external-source): wants the enrolment age of the factor being removed. The IdP holds it on the factor record; no audit event carries it.",
+ "sha256": "ee688d0398ead4dc67662644061a6d07b5f8485ad01062b4d98bd59f560ea07b"
},
{
"id": "det-identity-056",
@@ -48140,10 +48147,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-session-timeout-extended-priv.yaml",
- "sha256": "9a373aa9a6a175b68302072000507e9a6e40a369fbfc6121058417579a8f2940"
+ "quarantine_reason": "not executable (needs-inventory): asks whether a policy's scope is privileged. The scope is emitted; what is missing is a list of which scopes are privileged, which no source provides and no tenant configures today.",
+ "sha256": "4545d4c8552f22c40a58d15da4a9ef527a4aaf9284849583f4f4499783ea104d"
},
{
"id": "det-identity-060",
@@ -48391,10 +48399,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-pim-activation-self-approval.yaml",
- "sha256": "62d06042576f1213050be6e25f138581b6e7b4a3a994c8fe186ce6a8e4d88573"
+ "quarantine_reason": "not executable (needs-connector-field): compares the approver with the requester. The requester is emitted as `actor`; no identity source in this tree emits the approver, so a self-approval cannot be distinguished from an approval.",
+ "sha256": "e447491a06018da66c246a6d81d3f857e617e322c0f891fecb779fb29447784a"
},
{
"id": "det-identity-070",
@@ -48467,10 +48476,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-svc-account-interactive-login.yaml",
- "sha256": "59f246821383293c8a4e4a5fb84648022d9e32756a8145cd32c803e6e2dce423"
+ "quarantine_reason": "not executable (needs-connector-field): asks whether a logon was interactive. Windows 4624 answers it in `LogonType` (2, 10 and 11 are interactive) and no source lifts that field, so the boolean has nothing to derive from.",
+ "sha256": "36104ddc9687ef897c0fdf63c78e91c6ac90202ff64b21e036943129167aa58a"
},
{
"id": "det-identity-073",
@@ -48542,10 +48552,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-app-impersonation-priv.yaml",
- "sha256": "f992053a0951136374eb33d81408673952bfe564259ca5ddb1451af9ae38a75e"
+ "quarantine_reason": "not executable (needs-connector-field): asks whether the impersonated user is privileged. The privilege lookup now exists; the impersonated user is not emitted by any source, so there is no subject to look up.",
+ "sha256": "d4d3ff1d175a6491ef9a7a39f351e0a4ee2e0f369ea9a0194d3bc3afcdb242c9"
},
{
"id": "det-identity-076",
@@ -48719,10 +48730,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-ad-gpo-modified-priv.yaml",
- "sha256": "90517daae40443307bbdcf69ec0ca01c9eb2dbeb1af46513598709268ae4cd28"
+ "quarantine_reason": "not executable (needs-inventory): asks whether a modified GPO is linked to a privileged scope. Neither the GPO link nor a list of privileged scopes exists here.",
+ "sha256": "02a7875f223ebfe32a313efd55ecfa1c1518a365a86a87f4036696373bb1a084"
},
{
"id": "det-identity-083",
@@ -49096,10 +49108,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-ntlm-relay-indicator.yaml",
- "sha256": "3e12d23fd6267f36a2f42f3e5ca73a29ea3b47fac29864e0d62465716e2fee21"
+ "quarantine_reason": "not executable (needs-connector-field): compares the workstation named in the logon with the address it came from. Windows 4624 carries both as `WorkstationName` and `IpAddress`; neither is lifted as a top-level field, so neither side of the comparison resolves.",
+ "sha256": "ab1ba1a108b662d6a0b02fc1c3c0adbe32c1662bafdd6a6998722f0f502e53e0"
},
{
"id": "det-identity-098",
@@ -49171,10 +49184,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-pass-the-ticket-anomalous-tgt-source.yaml",
- "sha256": "79920bed0bc5489e7955b08475d27da9be4b2bafe3cab3090ab899527a5ee339"
+ "quarantine_reason": "not executable (needs-connector-field): compares the address a Kerberos ticket was issued to with the address using it. That needs two events joined on the ticket, which is a correlation, and neither address is emitted today.",
+ "sha256": "96b243fbd17c39386292761f18de494a66ff10ccd91d2d7564a8a25a99385761"
},
{
"id": "det-identity-101",
@@ -50882,10 +50896,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/network/newly-registered-domain-traffic.yaml",
- "sha256": "057aec7cd6a3ee71569f7bed994faacb67605c916337cb6dbf6e10621e74eac4"
+ "quarantine_reason": "not executable (needs-external-source): wants a domain's registration age, which comes from RDAP or WHOIS. Nothing in this tree queries either, and answering it from when this deployment first saw the domain would fire on most of the internet.",
+ "sha256": "4359781d0f7bbce674a247135b2a7300420761b7064884cc362fda7b32ac6a26"
},
{
"id": "det-network-009",
@@ -51464,10 +51479,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/network/net-dns-newly-registered-domain.yaml",
- "sha256": "9ee0d5678db29c8113018cc0b8311a624287f21827183910d386e3b2fd36646b"
+ "quarantine_reason": "not executable (needs-external-source): wants a domain's registration age from RDAP or WHOIS, which nothing here queries. Same gap as the proxy-side rule beside it.",
+ "sha256": "5bf1bd34f9d1a37e0e49e2cf8af21f6155c6b9d5eff24b0864598a01e767f373"
},
{
"id": "det-network-032",
diff --git a/scripts/check_detection_fields.py b/scripts/check_detection_fields.py
index 1c03a10ff..d2b57b09f 100644
--- a/scripts/check_detection_fields.py
+++ b/scripts/check_detection_fields.py
@@ -84,7 +84,20 @@ NORMALIZER = ROOT / "services" / "ingest" / "internal" / "normalizer" / "normali
# `scripts/check_windowed_translation.py`). Two of the 74 carried a second
# clause as well, which is why the identity-enrichment family moved from 24
# to 22 and the comparison family from 8 to 7.
-MAX_UNREACHABLE = 45
+#
+# 45 before depth plan 3.3, which closed the rest in four steps and left two:
+# * five rules were never unreachable — this gate's operator list had not
+# been given `neq` when the matcher was, so `approver_role_neq` was read
+# as a field name rather than as `approver_role` with `!=`;
+# * 18 rules now resolve their `*_priv` booleans from the tenant's own
+# directory import (`tenant_overlay.IDENTITY_FIELDS`);
+# * 2 resolve from the per-tenant first-seen store (`first_seen.py`);
+# * 18 are retired with a reason in `scripts/enrichment_decisions.py`,
+# gated by `scripts/check_enrichment_decisions.py`, which refuses to let
+# a rule whose fields *are* resolvable be retired to move this number.
+#
+# The two that remain both need the behavioural baseline of depth plan 3.4.
+MAX_UNREACHABLE = 2
#: Fields that no telemetry carries because they are computed, not observed:
#: sliding-window counters, allowlist membership, privilege flags, and
@@ -127,7 +140,15 @@ _DERIVED_FIELD_PATTERN = re.compile(
#: Operator suffixes the matcher strips to recover the field name. Order
#: matters: longer suffixes must be tried first so `not_contains_any` is not
#: shortened to `contains_any`. Mirrors `OPERATORS` in
-#: `services/fusion/app/services/detection_matcher.py`.
+#: `services/fusion/app/services/detection_matcher.py`, and
+#: `services/fusion/tests/test_detection_matcher_parity.py` asserts the two
+#: lists agree in **both** directions.
+#:
+#: They did not. `neq` was added to the matcher and not here, so five rules
+#: whose clauses the engine reads correctly as `<field>` + `!=` were reported
+#: by this gate as naming a field called `<field>_neq` — unreachable,
+#: ratcheted, and fine all along. A list that mirrors another and is checked
+#: in one direction only drifts in exactly the direction things change.
_OPERATOR_SUFFIXES = (
"not_contains_any",
"not_startswith_any",
@@ -147,6 +168,7 @@ _OPERATOR_SUFFIXES = (
"not_in",
"contains",
"match",
+ "neq",
"gte",
"lte",
"in",
@@ -211,6 +233,45 @@ def _rule_fields(match_when: dict) -> set[str]:
#: breaks.
_DERIVED_TIME_FIELDS = frozenset({"is_business_hours", "is_after_hours", "is_weekend"})
+#: Booleans the **tenant overlay** computes, from that tenant's own directory
+#: import (`identity_nodes.privilege_tier`). Depth plan 3.3.
+#:
+#: Named here rather than pattern-matched on `_priv$`, because four booleans
+#: in the same family are deliberately *not* computed — `scope_priv`,
+#: `act_as_user_priv`, `gpo_link_priv` and `account_is_dc` have no subject
+#: field any source emits. A pattern would have swept those in and reported
+#: four rules reachable that are not.
+#:
+#: `services/fusion/tests/test_tenant_tuning_overlay.py` asserts this set and
+#: `tenant_overlay.IDENTITY_FIELDS` agree in both directions, so the list
+#: cannot drift the way the operator list below did.
+_TENANT_DERIVED_FIELDS = frozenset(
+ {
+ "user_priv",
+ "target_user_priv",
+ "role_priv",
+ "actor_role_priv",
+ "actor_is_admin",
+ }
+)
+
+#: Booleans and ages the **first-seen store** computes
+#: (`services/fusion/app/services/first_seen.py`). Depth plan 3.3.
+#:
+#: Only observation-shaped subjects are here. `domain_age_days`,
+#: `key_age_days` and the other ages-of-a-thing are not, and will not be:
+#: answering "how old is this domain" with "when did we first see it" would
+#: fire a newly-registered-domain rule on most of the internet.
+#: `services/fusion/tests/test_first_seen.py` asserts this set and the
+#: store's own tables agree in both directions.
+_FIRST_SEEN_DERIVED_FIELDS = frozenset(
+ {
+ "publisher_ip_seen_before",
+ "device_fingerprint_seen_before",
+ "session_age_hours",
+ }
+)
+
#: `<left>_eq_<right>` / `<left>_neq_<right>`, resolved by comparing two
#: fields of the same event. Reachable only when *both* sides are in the
#: namespace — a comparison against a field nothing emits is still dead, and
@@ -221,7 +282,7 @@ _DERIVED_COMPARISON_RE = re.compile(r"^(?P<left>.+?)_(?:eq|neq)_(?P<right>.+)$")
def _is_derivable(field: str, namespace: set[str]) -> bool:
"""True when the engine can compute this field from what it already has."""
- if field in _DERIVED_TIME_FIELDS:
+ if field in _DERIVED_TIME_FIELDS or field in _TENANT_DERIVED_FIELDS or field in _FIRST_SEEN_DERIVED_FIELDS:
return True
match = _DERIVED_COMPARISON_RE.match(field)
if not match:
diff --git a/scripts/check_enrichment_decisions.py b/scripts/check_enrichment_decisions.py
new file mode 100644
index 000000000..8f92ef530
--- /dev/null
+++ b/scripts/check_enrichment_decisions.py
@@ -0,0 +1,216 @@
+#!/usr/bin/env python3
+"""CI gate: a rule that still cannot fire carries a reason, not just an absence.
+
+Depth plan 3.3 built two enrichment inputs — per-tenant identity privilege
+and a per-tenant first-seen store — and said of what remained: *"build them
+where cheap, otherwise quarantine with a reason."*
+
+Quarantining is the dangerous half, because deleting a rule and fixing a rule
+move `MAX_UNREACHABLE` by the same amount. This gate makes them different:
+
+* every rule in `enrichment_decisions.DECISIONS` names a real spec, carries a
+ known kind and a reason long enough to act on;
+* every retired rule is in fact gone from the stateless corpus, so a decision
+ that was recorded and never applied fails here;
+* **no rule is retired whose fields the platform now computes** — the check
+ that stops a working rule being quarantined to make a number move. It is
+ the one that would catch the lazy version of this whole item.
+
+Usage:
+ python3 scripts/check_enrichment_decisions.py # enforce
+ python3 scripts/check_enrichment_decisions.py --check # same, for CI
+ python3 scripts/check_enrichment_decisions.py --list # every decision
+ python3 scripts/check_enrichment_decisions.py --self-test # prove it works
+"""
+
+from __future__ import annotations
+
+import argparse
+import importlib.util
+import json
+import sys
+from pathlib import Path
+from typing import Any
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+
+from gate_toolkit import SKELETON, refuses_an_empty_tree, repo_root, self_test_main
+
+ROOT = repo_root()
+RULESET = ROOT / "services" / "fusion" / "app" / "data" / "detection_ruleset.json"
+
+#: A reason shorter than this is not a reason. It is the only record a reader
+#: of the catalogue gets of why a shipped rule does not run.
+_MIN_REASON_CHARS = 60
+
+
+def _load(name: str) -> Any:
+ spec = importlib.util.spec_from_file_location(name, ROOT / "scripts" / f"{name}.py")
+ if spec is None or spec.loader is None: # pragma: no cover
+ raise SystemExit(f"{Path(sys.argv[0]).name}: cannot load scripts/{name}.py")
+ module = importlib.util.module_from_spec(spec)
+ sys.modules[name] = module
+ spec.loader.exec_module(module)
+ return module
+
+
+def check(
+ decisions: dict[str, Any],
+ kinds: frozenset[str],
+ specs: dict[str, dict[str, Any]],
+ engine_ids: set[str],
+ lock: dict[str, str],
+ fields_gate: Any,
+) -> list[str]:
+ problems: list[str] = []
+ if not specs:
+ problems.append("no detection specs were read — refusing to report a tree with no rules as clean")
+ return problems
+
+ namespace = fields_gate._connector_namespace() | fields_gate._template_namespace() | fields_gate._go_normalizer_namespace()
+
+ for key, decision in decisions.items():
+ if key not in specs:
+ problems.append(f"{key} is decided but no spec declares it; a slug was renamed or removed")
+ continue
+ if decision.kind not in kinds:
+ problems.append(f"{key} is refused with an unknown kind {decision.kind!r}")
+ if len(decision.reason.strip()) < _MIN_REASON_CHARS:
+ problems.append(f"{key} is refused with a reason too short to act on ({len(decision.reason.strip())} chars)")
+
+ rule_id = lock.get(key)
+ if rule_id and rule_id in engine_ids:
+ problems.append(f"{key} ({rule_id}) is retired but still in {RULESET.name} — scripts/export_detection_ruleset.py must drop it")
+
+ # The check that matters. A rule every one of whose fields the
+ # platform can now resolve is a rule that works, and retiring it
+ # lowers the ratchet by removing a detection rather than by closing
+ # a gap.
+ missing = [
+ f
+ for f in fields_gate._rule_fields(specs[key].get("match_when") or {})
+ if f not in namespace and not fields_gate._is_derivable(f, namespace)
+ ]
+ if not missing:
+ problems.append(
+ f"{key} is retired, but every field it reads is now resolvable. "
+ "Retiring a rule that works lowers the ratchet without closing anything — un-retire it."
+ )
+ return problems
+
+
+def _self_test_injections(
+ decisions: dict[str, Any],
+ kinds: frozenset[str],
+ specs: dict[str, dict[str, Any]],
+ engine_ids: set[str],
+ lock: dict[str, str],
+ fields_gate: Any,
+) -> list[tuple[str, bool]]:
+ module = _load("enrichment_decisions")
+ results = [("the tree as committed has no findings to begin with", not check(decisions, kinds, specs, engine_ids, lock, fields_gate))]
+
+ first = next(iter(decisions))
+
+ blunt = dict(decisions)
+ blunt[first] = module.Refusal(module.NEEDS_INVENTORY, "too hard")
+ results.append(("catches a refusal with no usable reason", bool(check(blunt, kinds, specs, engine_ids, lock, fields_gate))))
+
+ typo = dict(decisions)
+ typo[first] = module.Refusal("needs-something", decisions[first].reason)
+ results.append(
+ ("catches a refusal kind that is not one of the declared families", bool(check(typo, kinds, specs, engine_ids, lock, fields_gate)))
+ )
+
+ invented = dict(decisions)
+ invented["identity/a-slug-that-does-not-exist"] = decisions[first]
+ results.append(("catches a decision for a rule no spec declares", bool(check(invented, kinds, specs, engine_ids, lock, fields_gate))))
+
+ still_shipped = set(engine_ids) | {lock.get(first, "")}
+ results.append(
+ ("catches a retired rule the exporter still ships", bool(check(decisions, kinds, specs, still_shipped, lock, fields_gate)))
+ )
+
+ # The important one: retire a rule that works, and require it to be
+ # caught. Picked from the corpus rather than invented, so the injection
+ # is a rule the engine really does load and really can fire.
+ working = next(
+ key
+ for key, spec in specs.items()
+ if key not in decisions
+ and spec.get("match_when")
+ and not [
+ f
+ for f in fields_gate._rule_fields(spec["match_when"])
+ if f not in (fields_gate._connector_namespace() | fields_gate._template_namespace() | fields_gate._go_normalizer_namespace())
+ and not fields_gate._is_derivable(f, set())
+ ]
+ )
+ over_retired = dict(decisions)
+ over_retired[working] = decisions[first]
+ results.append(
+ (
+ f"catches a working rule retired to move the ratchet ({working})",
+ any("every field it reads is now resolvable" in p for p in check(over_retired, kinds, specs, engine_ids, lock, fields_gate)),
+ )
+ )
+
+ results.append(("refuses a corpus it read no specs from", bool(check(decisions, kinds, {}, engine_ids, lock, fields_gate))))
+
+ skeleton_refused, detail = refuses_an_empty_tree(Path(__file__).name, ["--check"], shape=SKELETON)
+ results.append((f"refuses a tree whose directories exist and hold no rules ({detail.splitlines()[0]})", skeleton_refused))
+ return results
+
+
+def main() -> int:
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument("--check", action="store_true", help="enforce (the default; accepted so CI can be explicit)")
+ parser.add_argument("--list", action="store_true", help="print every decision")
+ parser.add_argument("--self-test", action="store_true", help="prove the gate catches what it claims to")
+ args = parser.parse_args()
+
+ if not RULESET.is_file():
+ print(f"{Path(sys.argv[0]).name}: no detection ruleset at {RULESET} — nothing to check", file=sys.stderr)
+ return 2
+
+ module = _load("enrichment_decisions")
+ fields_gate = _load("check_detection_fields")
+ sys.path.insert(0, str(ROOT / "scripts"))
+ from detection_specs_index import all_specs # noqa: PLC0415
+ from generate_detections import load_id_lock # noqa: PLC0415
+
+ specs = {f"{category}/{spec['slug']}": spec for category, spec in all_specs()}
+ lock = load_id_lock()
+ engine_ids = {str(r.get("id")) for r in json.loads(RULESET.read_text(encoding="utf-8")).get("rules") or []}
+
+ if args.self_test:
+ return self_test_main(
+ Path(__file__).name,
+ ["--check"],
+ extra=_self_test_injections(module.DECISIONS, module.KINDS, specs, engine_ids, lock, fields_gate),
+ )
+
+ if args.list:
+ for key, decision in module.DECISIONS.items():
+ print(f"RETIRED {key} [{decision.kind}] {decision.reason.splitlines()[0]}")
+ return 0
+
+ problems = check(module.DECISIONS, module.KINDS, specs, engine_ids, lock, fields_gate)
+ kinds: dict[str, int] = {}
+ for decision in module.DECISIONS.values():
+ kinds[decision.kind] = kinds.get(decision.kind, 0) + 1
+ print(f"enrichment decisions: {len(module.DECISIONS)} rules retired with a reason")
+ for kind, count in sorted(kinds.items(), key=lambda kv: -kv[1]):
+ print(f" {count:4d} {kind}")
+
+ if problems:
+ print(f"\nERROR: {len(problems)} problem(s):", file=sys.stderr)
+ for problem in problems:
+ print(f" - {problem}", file=sys.stderr)
+ return 1
+ print("\nOK: every retired rule names a real spec, a reason, and a gap the platform genuinely has")
+ return 0
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/scripts/check_windowed_translation.py b/scripts/check_windowed_translation.py
index 2d303ec9a..e0ab340c9 100644
--- a/scripts/check_windowed_translation.py
+++ b/scripts/check_windowed_translation.py
@@ -73,10 +73,11 @@ _MIN_REASON_CHARS = 60
#: reads the stateless ruleset only, so until now a windowed rule could name
#: anything.
#:
-#: The two standing entries both read `role_priv`, which arrives with the
-#: identity enrichment of depth plan 3.3. Lower this in the PR that builds it;
-#: never raise it to make a red build green.
-MAX_UNREACHABLE_WINDOWED = 2
+#: Two standing entries read `role_priv` when this was written. Depth plan
+#: 3.3 built that enrichment, so the ceiling came down with it. Never raise
+#: it to make a red build green: a windowed rule naming a computed field
+#: counts nothing forever.
+MAX_UNREACHABLE_WINDOWED = 0
def _load_sibling(name: str) -> Any:
diff --git a/scripts/detection_specs_part3_helpers.py b/scripts/detection_specs_part3_helpers.py
index f23eb3c1e..e72c76e3b 100644
--- a/scripts/detection_specs_part3_helpers.py
+++ b/scripts/detection_specs_part3_helpers.py
@@ -37,10 +37,12 @@ _OP_SUFFIXES: tuple[tuple[str, str], ...] = tuple(
# that field, so the rule matched itself and nothing else.
("_neq", "neq"),
("_pattern_match_any", "pattern_match_any"),
+ ("_not_startswith_any", "not_startswith_any"),
("_not_endswith_any", "not_endswith_any"),
("_not_contains_any", "not_contains_any"),
("_pattern_match", "pattern_match"),
("_not_startswith", "not_startswith"),
+ ("_not_endswith", "not_endswith"),
("_startswith_any", "startswith_any"),
("_endswith_any", "endswith_any"),
("_contains_any", "contains_any"),
@@ -126,6 +128,10 @@ def _pos_for(op: str, expected: Any) -> Any:
return _SENTINEL_STR
if op == "not_startswith":
return _SENTINEL_STR
+ if op == "not_startswith_any":
+ return _SENTINEL_STR
+ if op == "not_endswith":
+ return _SENTINEL_STR
if op == "has_any":
if isinstance(expected, list) and expected:
return list(expected[:1])
@@ -244,6 +250,12 @@ def _neg_for(op: str, expected: Any, field: str | None = None) -> Any:
return _SENTINEL_STR
if op == "not_startswith":
return f"{expected}-tail"
+ if op == "not_startswith_any":
+ if isinstance(expected, list) and expected:
+ return f"{expected[0]}-tail"
+ return _SENTINEL_STR
+ if op == "not_endswith":
+ return f"head{expected}"
if op == "has_any":
return [_SENTINEL_STR]
if op == "match" or op == "pattern_match":
diff --git a/scripts/enrichment_decisions.py b/scripts/enrichment_decisions.py
new file mode 100644
index 000000000..c687d6b30
--- /dev/null
+++ b/scripts/enrichment_decisions.py
@@ -0,0 +1,169 @@
+#!/usr/bin/env python3
+"""Rules that still cannot fire after depth plan 3.3, and exactly why.
+
+3.3 asked for three enrichment inputs and said of one of them: *"build them
+where cheap, otherwise quarantine with a reason."* That instruction is the
+shape of this file, applied to all three families.
+
+What was built
+--------------
+* **Identity privilege** (`services/fusion/app/services/tenant_overlay.py`),
+ from `identity_nodes.privilege_tier`. Closed 18 rules.
+* **First-seen** (`services/fusion/app/services/first_seen.py`), per tenant in
+ Redis. Closed 2 rules.
+* **A gate-drift fix.** Five rules were never unreachable: the reachability
+ gate's operator list had not been given `neq` when the matcher was, so
+ `approver_role_neq` was read as a field name instead of as `approver_role`
+ with `!=`.
+
+What is left, and why each one is a refusal rather than a fix
+-------------------------------------------------------------
+Eighteen rules. Every one needs something that belongs to a different piece
+of work, named per entry. They keep their id and their YAML and stop being
+counted as executable, which they never were.
+
+The line this file holds, and the reason it matters more than the count: a
+plausible substitute is worse than an absence. `domain_age_days` wants a
+domain's registration age from a registry. The first-seen store could answer
+"the first time this deployment saw it" and the number would look right in a
+diff — and the rule would then fire on every domain a quiet tenant had not
+happened to see, which is most of the internet. Refusing is the correct
+engineering answer, not the lazy one.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+# `KINDS` belongs here: `check_enrichment_decisions.py` reads it as
+# `module.KINDS` after loading this file by path, which no static
+# analysis can follow — CodeQL reported it as an unused global. Naming
+# it as an export is both true and the thing that makes the use visible.
+__all__ = ["DECISIONS", "KINDS", "NEEDS_CONNECTOR_FIELD", "NEEDS_EXTERNAL_SOURCE", "NEEDS_INVENTORY", "Refusal"]
+
+#: The field exists at the vendor and no connector surfaces it. Closing these
+#: is the activity projection of depth plan 2.2.
+NEEDS_CONNECTOR_FIELD = "needs-connector-field"
+
+#: The answer lives outside any telemetry: a registry, a directory snapshot,
+#: a vendor configuration API. Depth plan 2.5 for posture, elsewhere for the
+#: rest.
+NEEDS_EXTERNAL_SOURCE = "needs-external-source"
+
+#: The answer needs an inventory this platform does not hold — which machine
+#: accounts are domain controllers, which OAuth scopes are privileged.
+NEEDS_INVENTORY = "needs-inventory"
+
+KINDS = frozenset({NEEDS_CONNECTOR_FIELD, NEEDS_EXTERNAL_SOURCE, NEEDS_INVENTORY})
+
+
+@dataclass(frozen=True)
+class Refusal:
+ kind: str
+ reason: str
+
+
+DECISIONS: dict[str, Refusal] = {
+ # ── comparisons whose operands nothing emits ────────────────────────
+ "application/api-permission-bypass-idor": Refusal(
+ NEEDS_CONNECTOR_FIELD,
+ "compares the object's owning tenant with the caller's, and no application "
+ "source in this tree surfaces either. The engine can compare two fields of "
+ "one event; it cannot compare two fields that are not there.",
+ ),
+ "endpoint/linux-bashrc-modified-by-other-user": Refusal(
+ NEEDS_CONNECTOR_FIELD,
+ "compares the acting uid with the file owner's uid. The acting uid is "
+ "emitted and the owner's is not, so the comparison resolves to nothing. "
+ "An EDR file-write event carries the owner; surfacing it is connector work.",
+ ),
+ "endpoint/linux-zshrc-modified-by-other-user": Refusal(
+ NEEDS_CONNECTOR_FIELD,
+ "compares the acting uid with the file owner's uid, and no source emits the owner. Same gap as the bashrc rule beside it.",
+ ),
+ "endpoint/macos-keychain-db-read-by-other-uid": Refusal(
+ NEEDS_CONNECTOR_FIELD,
+ "compares the reading uid with the keychain owner's uid, and no source emits the owner.",
+ ),
+ "identity/ident-pim-activation-self-approval": Refusal(
+ NEEDS_CONNECTOR_FIELD,
+ "compares the approver with the requester. The requester is emitted as "
+ "`actor`; no identity source in this tree emits the approver, so a "
+ "self-approval cannot be distinguished from an approval.",
+ ),
+ "identity/ident-ntlm-relay-indicator": Refusal(
+ NEEDS_CONNECTOR_FIELD,
+ "compares the workstation named in the logon with the address it came from. "
+ "Windows 4624 carries both as `WorkstationName` and `IpAddress`; neither is "
+ "lifted as a top-level field, so neither side of the comparison resolves.",
+ ),
+ "identity/ident-pass-the-ticket-anomalous-tgt-source": Refusal(
+ NEEDS_CONNECTOR_FIELD,
+ "compares the address a Kerberos ticket was issued to with the address using "
+ "it. That needs two events joined on the ticket, which is a correlation, and "
+ "neither address is emitted today.",
+ ),
+ # ── ages of a thing, which a first-seen store must not guess ────────
+ "application/app-npm-package-publish-new-account": Refusal(
+ NEEDS_EXTERNAL_SOURCE,
+ "wants the publisher account's age at the registry. The first-seen store "
+ "knows when this deployment first saw the account, which is a different "
+ "number: substituting it would fire on every publisher a quiet tenant has "
+ "not seen before.",
+ ),
+ "cloud/aws-iam-key-rotation-skipped": Refusal(
+ NEEDS_EXTERNAL_SOURCE,
+ "wants an access key's creation date. CloudTrail reports key *use* and not "
+ "key age; the date is in the IAM credential report, which arrives with the "
+ "AWS posture snapshot (depth plan 2.5).",
+ ),
+ "cloud/aws-cloudfront-origin-changed": Refusal(
+ NEEDS_EXTERNAL_SOURCE,
+ "wants a distribution's creation date, which is configuration state rather "
+ "than an event, and arrives with the AWS posture snapshot (depth plan 2.5).",
+ ),
+ "identity/ident-fido2-key-added-then-removed-old": Refusal(
+ NEEDS_EXTERNAL_SOURCE,
+ "wants the enrolment age of the factor being removed. The IdP holds it on the factor record; no audit event carries it.",
+ ),
+ "network/newly-registered-domain-traffic": Refusal(
+ NEEDS_EXTERNAL_SOURCE,
+ "wants a domain's registration age, which comes from RDAP or WHOIS. Nothing "
+ "in this tree queries either, and answering it from when this deployment "
+ "first saw the domain would fire on most of the internet.",
+ ),
+ "network/net-dns-newly-registered-domain": Refusal(
+ NEEDS_EXTERNAL_SOURCE,
+ "wants a domain's registration age from RDAP or WHOIS, which nothing here queries. Same gap as the proxy-side rule beside it.",
+ ),
+ # ── privilege questions with no subject on the event ────────────────
+ "endpoint/win-dcsync-via-replication": Refusal(
+ NEEDS_INVENTORY,
+ "asks whether the replicating account is a domain controller. That needs a "
+ "directory inventory of DC machine accounts; `identity_nodes` carries a "
+ "privilege tier per principal and no such flag.",
+ ),
+ "identity/ident-session-timeout-extended-priv": Refusal(
+ NEEDS_INVENTORY,
+ "asks whether a policy's scope is privileged. The scope is emitted; what is "
+ "missing is a list of which scopes are privileged, which no source provides "
+ "and no tenant configures today.",
+ ),
+ "identity/ident-app-impersonation-priv": Refusal(
+ NEEDS_CONNECTOR_FIELD,
+ "asks whether the impersonated user is privileged. The privilege lookup now "
+ "exists; the impersonated user is not emitted by any source, so there is no "
+ "subject to look up.",
+ ),
+ "identity/ident-ad-gpo-modified-priv": Refusal(
+ NEEDS_INVENTORY,
+ "asks whether a modified GPO is linked to a privileged scope. Neither the GPO link nor a list of privileged scopes exists here.",
+ ),
+ # ── other ───────────────────────────────────────────────────────────
+ "identity/ident-svc-account-interactive-login": Refusal(
+ NEEDS_CONNECTOR_FIELD,
+ "asks whether a logon was interactive. Windows 4624 answers it in "
+ "`LogonType` (2, 10 and 11 are interactive) and no source lifts that field, "
+ "so the boolean has nothing to derive from.",
+ ),
+}
diff --git a/scripts/export_detection_ruleset.py b/scripts/export_detection_ruleset.py
index 2455717aa..3501449b6 100644
--- a/scripts/export_detection_ruleset.py
+++ b/scripts/export_detection_ruleset.py
@@ -41,7 +41,7 @@ OUT = ROOT / "services" / "fusion" / "app" / "data" / "detection_ruleset.json"
def _build() -> list[dict]:
from detection_specs_index import all_specs # noqa: PLC0415
from generate_detections import load_id_lock # noqa: PLC0415
- from windowed_translation import DECISIONS # noqa: PLC0415
+ from rule_retirement import retired # noqa: PLC0415
# The id comes from detections/rule-ids.lock.json, the same lookup
# generate_detections.py uses for the YAML projection. This used to be
@@ -56,21 +56,22 @@ def _build() -> list[dict]:
# anywhere but the end shifts every id after it, so an alert's rule id
# would not survive a release.
id_lock = load_id_lock()
+ retirements = retired()
rules: list[dict] = []
for category, spec in all_specs():
match_when = spec.get("match_when")
if not match_when:
continue
slug = spec["slug"]
- # A rule whose clauses name a sliding-window counter cannot fire here:
- # the stateless matcher sees one event, so `fail_count` and
- # `events_per_minute` read None on the first clause. Shipping it in
- # this artefact counted it as executable and it never was.
- # `scripts/windowed_translation.py` records, per rule, whether it
- # became a `wd-*` rule or was refused with a reason;
- # `check_windowed_translation.py` enforces that nothing is simply
+ # A rule that cannot fire must not ship in this artefact, because
+ # membership of it is what "executable" means and what the published
+ # total counts. Two tables record why a rule was removed —
+ # `windowed_translation.py` for the sliding-window family and
+ # `enrichment_decisions.py` for the enrichment ones — and
+ # `rule_retirement.py` is the union, so neither caller can read one
+ # and miss the other. Their gates enforce that nothing is simply
# dropped.
- if f"{category}/{slug}" in DECISIONS:
+ if f"{category}/{slug}" in retirements:
continue
rule_id = id_lock.get(f"{category}/{slug}")
if rule_id is None:
diff --git a/scripts/generate_detections.py b/scripts/generate_detections.py
index 53b284005..88e4b1e04 100644
--- a/scripts/generate_detections.py
+++ b/scripts/generate_detections.py
@@ -84,10 +84,12 @@ from detection_specs_index import CATEGORIES # noqa: E402 (after sys.path twea
OPERATORS: list[tuple[str, str, str]] = sorted(
[
("_pattern_match_any", "pattern_match_any", "PATTERN_MATCH_ANY"),
+ ("_not_startswith_any", "not_startswith_any", "NOT STARTSWITH_ANY"),
("_not_endswith_any", "not_endswith_any", "NOT ENDSWITH_ANY"),
("_not_contains_any", "not_contains_any", "NOT CONTAINS_ANY"),
("_pattern_match", "pattern_match", "PATTERN_MATCH"),
("_not_startswith", "not_startswith", "NOT STARTSWITH"),
+ ("_not_endswith", "not_endswith", "NOT ENDSWITH"),
("_startswith_any", "startswith_any", "STARTSWITH_ANY"),
("_endswith_any", "endswith_any", "ENDSWITH_ANY"),
("_contains_any", "contains_any", "CONTAINS_ANY"),
@@ -334,6 +336,24 @@ def _check(field: str, op: str, expected: Any, event: dict[str, Any]) -> bool:
return False
return not any(actual.endswith(str(s)) for s in expected)
+ # Three shipped rules used `not_startswith_any` and it had never been an
+ # operator, so `path_not_startswith_any` was read as a field name. The
+ # reachability gate hid that by stripping a suffix the matcher did not
+ # know, so all three were reported reachable and could not fire.
+ #
+ # A missing field returns False, exactly as the negations beside it do: a
+ # negation that flips to True on an absent field fires on every event
+ # that happens not to carry it.
+ if op == "not_startswith_any":
+ if not isinstance(expected, list) or not isinstance(actual, str):
+ return False
+ return not any(actual.startswith(str(s)) for s in expected)
+
+ if op == "not_endswith":
+ if not isinstance(actual, str):
+ return False
+ return not actual.endswith(str(expected))
+
if op == "not_contains_any":
if not isinstance(expected, list):
return False
@@ -411,7 +431,7 @@ yaml.add_representer(_LiteralStr, _literal_str_representer) # type: ignore[arg-
def _retirement_reason(category: str, slug: str) -> str:
"""Why this rule is not loaded by the stateless engine, or '' if it is."""
- from windowed_translation import retirement_reason # noqa: PLC0415
+ from rule_retirement import retirement_reason # noqa: PLC0415
return retirement_reason(f"{category}/{slug}")
diff --git a/scripts/rule_retirement.py b/scripts/rule_retirement.py
new file mode 100644
index 000000000..c4d7001ec
--- /dev/null
+++ b/scripts/rule_retirement.py
@@ -0,0 +1,41 @@
+#!/usr/bin/env python3
+"""One answer to "why is this rule not loaded", from every decision table.
+
+Two tables record rules that leave the stateless corpus:
+`windowed_translation.py` (depth plan 3.1) and `enrichment_decisions.py`
+(3.3). `export_detection_ruleset.py` and `generate_detections.py` both need
+the union, and both asking each table in turn is how one of them comes to
+ask only one.
+"""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+
+__all__ = ["retired", "retirement_reason"]
+
+
+def retired() -> dict[str, str]:
+ """`<category>/<slug>` → the sentence the catalogue publishes."""
+ from enrichment_decisions import DECISIONS as ENRICHMENT # noqa: PLC0415
+ from windowed_translation import DECISIONS as WINDOWED # noqa: PLC0415
+ from windowed_translation import retirement_reason as windowed_reason # noqa: PLC0415
+
+ out = {key: windowed_reason(key) for key in WINDOWED}
+ overlap = sorted(set(out) & set(ENRICHMENT))
+ if overlap:
+ raise SystemExit(
+ f"rule_retirement: {len(overlap)} rule(s) are retired by two tables: {', '.join(overlap)}. "
+ "One rule, one reason — a reader of the catalogue cannot be given two."
+ )
+ for key, decision in ENRICHMENT.items():
+ out[key] = f"not executable ({decision.kind}): {decision.reason}"
+ return out
+
+
+def retirement_reason(key: str) -> str:
+ """The sentence for one rule, or '' when it still ships."""
+ return retired().get(key, "")
diff --git a/services/api/app/data/marketplace/index.json b/services/api/app/data/marketplace/index.json
index ec51bfa25..d85ef41a6 100644
--- a/services/api/app/data/marketplace/index.json
+++ b/services/api/app/data/marketplace/index.json
@@ -1,7 +1,7 @@
{
"$schema": "https://example.com/schemas/marketplace/v1.json",
"version": "1.0.0",
- "generated": "2026-10-09T03:35:13Z",
+ "generated": "2026-10-09T04:33:06Z",
"categories": [
{
"id": "playbooks",
@@ -38,14 +38,14 @@
"imported": 6113,
"stable": 877
},
- "executable": 2696,
- "quarantined": 4462,
- "executable_detections": 2529,
- "reference_only_detections": 4462,
+ "executable": 2678,
+ "quarantined": 4480,
+ "executable_detections": 2511,
+ "reference_only_detections": 4480,
"not_a_detection": 167
},
"mitre_coverage": {
- "unique_techniques": 389,
+ "unique_techniques": 387,
"techniques": {
"T1001.003": 1,
"T1003": 21,
@@ -54,7 +54,7 @@
"T1003.003": 21,
"T1003.004": 10,
"T1003.005": 8,
- "T1003.006": 7,
+ "T1003.006": 6,
"T1003.008": 1,
"T1005": 7,
"T1007": 2,
@@ -137,15 +137,15 @@
"T1070.005": 4,
"T1070.006": 1,
"T1071": 7,
- "T1071.001": 14,
- "T1071.004": 11,
+ "T1071.001": 13,
+ "T1071.004": 10,
"T1072": 4,
"T1074.001": 4,
- "T1078": 50,
+ "T1078": 49,
"T1078.001": 5,
"T1078.002": 7,
"T1078.003": 2,
- "T1078.004": 47,
+ "T1078.004": 45,
"T1082": 13,
"T1083": 9,
"T1087": 10,
@@ -210,7 +210,7 @@
"T1190": 39,
"T1195": 3,
"T1195.001": 1,
- "T1195.002": 8,
+ "T1195.002": 7,
"T1197": 13,
"T1199": 12,
"T1200": 3,
@@ -224,7 +224,7 @@
"T1207": 2,
"T1210": 4,
"T1211": 4,
- "T1212": 5,
+ "T1212": 4,
"T1213": 2,
"T1216": 8,
"T1216.001": 2,
@@ -249,7 +249,7 @@
"T1404": 2,
"T1482": 14,
"T1484": 1,
- "T1484.001": 8,
+ "T1484.001": 7,
"T1484.002": 2,
"T1485": 29,
"T1486": 11,
@@ -271,7 +271,7 @@
"T1518.001": 2,
"T1525": 8,
"T1526": 2,
- "T1528": 9,
+ "T1528": 8,
"T1529": 3,
"T1530": 27,
"T1531": 7,
@@ -289,7 +289,6 @@
"T1546.001": 4,
"T1546.002": 1,
"T1546.003": 11,
- "T1546.004": 2,
"T1546.007": 1,
"T1546.008": 5,
"T1546.009": 1,
@@ -315,7 +314,7 @@
"T1550": 5,
"T1550.001": 2,
"T1550.002": 6,
- "T1550.003": 4,
+ "T1550.003": 3,
"T1550.004": 1,
"T1552": 3,
"T1552.001": 17,
@@ -331,18 +330,18 @@
"T1553.005": 5,
"T1554": 3,
"T1555": 8,
- "T1555.001": 5,
+ "T1555.001": 4,
"T1555.003": 9,
"T1555.004": 3,
"T1555.005": 1,
- "T1556": 43,
+ "T1556": 42,
"T1556.001": 1,
"T1556.003": 1,
"T1556.004": 1,
- "T1556.006": 16,
+ "T1556.006": 15,
"T1556.007": 1,
"T1557": 2,
- "T1557.001": 13,
+ "T1557.001": 12,
"T1557.002": 3,
"T1557.003": 1,
"T1558": 3,
@@ -371,7 +370,6 @@
"T1564.008": 1,
"T1565": 2,
"T1565.001": 3,
- "T1565.002": 1,
"T1566": 5,
"T1566.001": 15,
"T1566.002": 2,
@@ -437,7 +435,7 @@
"T1686.003": 10,
"T1689": 1
},
- "total_with_mitre": 2398,
+ "total_with_mitre": 2380,
"unique_techniques_all_rules": 493,
"techniques_all_rules": {
"T1001.003": 2,
@@ -31112,10 +31110,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/application/api-permission-bypass-idor.yaml",
- "sha256": "907beebb04b60d1689cfbab9e63429968a56e4e59b580a9837feb62e6ddebc7a"
+ "quarantine_reason": "not executable (needs-connector-field): compares the object's owning tenant with the caller's, and no application source in this tree surfaces either. The engine can compare two fields of one event; it cannot compare two fields that are not there.",
+ "sha256": "3f757cd10acefaf0b218f68cd8f303aef0f8f9570eb30887494e602a8d28a41d"
},
{
"id": "det-application-021",
@@ -31939,10 +31938,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/application/app-npm-package-publish-new-account.yaml",
- "sha256": "54cefdefd2ce54c8dc2a82f3f8ab0ce19663f18acec4e1c515b85fdfde352a60"
+ "quarantine_reason": "not executable (needs-external-source): wants the publisher account's age at the registry. The first-seen store knows when this deployment first saw the account, which is a different number: substituting it would fire on every publisher a quiet tenant has not seen before.",
+ "sha256": "636bbcd6478dc775efa4bca041e41f09de50c63bfaa801e7d5f66fd1383de7b6"
},
{
"id": "det-application-054",
@@ -33831,10 +33831,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/cloud/aws-iam-key-rotation-skipped.yaml",
- "sha256": "8d0a8e51f512c731c5e600f7f31e9ba210793a0241ab6d27bd55cc4fad68c9c6"
+ "quarantine_reason": "not executable (needs-external-source): wants an access key's creation date. CloudTrail reports key *use* and not key age; the date is in the IAM credential report, which arrives with the AWS posture snapshot (depth plan 2.5).",
+ "sha256": "a18d457a4dae77733cdbb4b1a24795938a42482111a9db3fa866f11316e9fdf9"
},
{
"id": "det-cloud-043",
@@ -35213,10 +35214,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/cloud/aws-cloudfront-origin-changed.yaml",
- "sha256": "8e47eb25a342cc5028d93e531d35b98576fd146e849e9528bcfb384a243f47e2"
+ "quarantine_reason": "not executable (needs-external-source): wants a distribution's creation date, which is configuration state rather than an event, and arrives with the AWS posture snapshot (depth plan 2.5).",
+ "sha256": "e3e1416f71dd1cadea373cb7396fbba0fbef8862b167fb86495a5e10a94e1d44"
},
{
"id": "det-cloud-098",
@@ -41549,10 +41551,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/endpoint/win-dcsync-via-replication.yaml",
- "sha256": "d24ef85ca4947366004a5d44c0237cd781d577b98843f807702ef15f21d784ba"
+ "quarantine_reason": "not executable (needs-inventory): asks whether the replicating account is a domain controller. That needs a directory inventory of DC machine accounts; `identity_nodes` carries a privilege tier per principal and no such flag.",
+ "sha256": "6e91e68fc557d499900940313406cf94950ae9e4b8aeb6e8d2d91342b8e44123"
},
{
"id": "det-endpoint-102",
@@ -42474,10 +42477,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/endpoint/linux-bashrc-modified-by-other-user.yaml",
- "sha256": "c1b4c05c210199bc384ae90694227b5e84f43bddb91d79473caf5c58ace568bd"
+ "quarantine_reason": "not executable (needs-connector-field): compares the acting uid with the file owner's uid. The acting uid is emitted and the owner's is not, so the comparison resolves to nothing. An EDR file-write event carries the owner; surfacing it is connector work.",
+ "sha256": "cc5992ffa893502cd43175a97ec74d6a2f044e4a28e20b9adca0b99327a6313a"
},
{
"id": "det-endpoint-139",
@@ -42499,10 +42503,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/endpoint/linux-zshrc-modified-by-other-user.yaml",
- "sha256": "e9afa054a50fb70992783ad32ea39f15cf98ae337bc84c5ca81cd9df63719203"
+ "quarantine_reason": "not executable (needs-connector-field): compares the acting uid with the file owner's uid, and no source emits the owner. Same gap as the bashrc rule beside it.",
+ "sha256": "3cd1e6e8fb88bcb344eb9ceb83e58a39709663fd4a8df9c817bd9e051f2ebc9a"
},
{
"id": "det-endpoint-140",
@@ -42977,7 +42982,7 @@
"enabled": true,
"executable": true,
"path": "detections/endpoint/linux-suid-binary-create.yaml",
- "sha256": "ddea068f5f1a32a4fd23c79b464192229a6c00e3095a44760677dfec1b9617cd"
+ "sha256": "3d3ca5b7f2181109fee0a0b5955bdaf540989956fff011bfaecd4ecc967e6570"
},
{
"id": "det-endpoint-159",
@@ -43002,7 +43007,7 @@
"enabled": true,
"executable": true,
"path": "detections/endpoint/linux-guid-binary-create.yaml",
- "sha256": "e72010b019d39c76506e7e699487041c004008e4d122c4664d19643e133e5f2c"
+ "sha256": "02ae1b8931fa60b6bd38d23f5646258168151d0d16f57dcc9354360f1fc120de"
},
{
"id": "det-endpoint-160",
@@ -44399,10 +44404,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/endpoint/macos-keychain-db-read-by-other-uid.yaml",
- "sha256": "7bb0f15de4393bb1494efd9b97919bf49b98857195bff4c3b5dfd98342070b48"
+ "quarantine_reason": "not executable (needs-connector-field): compares the reading uid with the keychain owner's uid, and no source emits the owner.",
+ "sha256": "4dedf02467737bad5de7bd0f4dab2012df27751364714a0c942d36c6c6c5c713"
},
{
"id": "det-endpoint-216",
@@ -44703,7 +44709,7 @@
"enabled": true,
"executable": true,
"path": "detections/endpoint/macos-hyperkit-launch-untrusted.yaml",
- "sha256": "90613cf5832155b845a99380e7a52c2e08773fb5c3a61603338b80017d3f2743"
+ "sha256": "60416ddeb3d1c53c5f8f25f105052145e7d1480f4a20b308a6e4517332196762"
},
{
"id": "det-endpoint-228",
@@ -48040,10 +48046,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-fido2-key-added-then-removed-old.yaml",
- "sha256": "f4b13b89ad3565bb9643193e55c5c21cf8cbc8a465938242fb42a8a285cafada"
+ "quarantine_reason": "not executable (needs-external-source): wants the enrolment age of the factor being removed. The IdP holds it on the factor record; no audit event carries it.",
+ "sha256": "ee688d0398ead4dc67662644061a6d07b5f8485ad01062b4d98bd59f560ea07b"
},
{
"id": "det-identity-056",
@@ -48140,10 +48147,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-session-timeout-extended-priv.yaml",
- "sha256": "9a373aa9a6a175b68302072000507e9a6e40a369fbfc6121058417579a8f2940"
+ "quarantine_reason": "not executable (needs-inventory): asks whether a policy's scope is privileged. The scope is emitted; what is missing is a list of which scopes are privileged, which no source provides and no tenant configures today.",
+ "sha256": "4545d4c8552f22c40a58d15da4a9ef527a4aaf9284849583f4f4499783ea104d"
},
{
"id": "det-identity-060",
@@ -48391,10 +48399,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-pim-activation-self-approval.yaml",
- "sha256": "62d06042576f1213050be6e25f138581b6e7b4a3a994c8fe186ce6a8e4d88573"
+ "quarantine_reason": "not executable (needs-connector-field): compares the approver with the requester. The requester is emitted as `actor`; no identity source in this tree emits the approver, so a self-approval cannot be distinguished from an approval.",
+ "sha256": "e447491a06018da66c246a6d81d3f857e617e322c0f891fecb779fb29447784a"
},
{
"id": "det-identity-070",
@@ -48467,10 +48476,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-svc-account-interactive-login.yaml",
- "sha256": "59f246821383293c8a4e4a5fb84648022d9e32756a8145cd32c803e6e2dce423"
+ "quarantine_reason": "not executable (needs-connector-field): asks whether a logon was interactive. Windows 4624 answers it in `LogonType` (2, 10 and 11 are interactive) and no source lifts that field, so the boolean has nothing to derive from.",
+ "sha256": "36104ddc9687ef897c0fdf63c78e91c6ac90202ff64b21e036943129167aa58a"
},
{
"id": "det-identity-073",
@@ -48542,10 +48552,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-app-impersonation-priv.yaml",
- "sha256": "f992053a0951136374eb33d81408673952bfe564259ca5ddb1451af9ae38a75e"
+ "quarantine_reason": "not executable (needs-connector-field): asks whether the impersonated user is privileged. The privilege lookup now exists; the impersonated user is not emitted by any source, so there is no subject to look up.",
+ "sha256": "d4d3ff1d175a6491ef9a7a39f351e0a4ee2e0f369ea9a0194d3bc3afcdb242c9"
},
{
"id": "det-identity-076",
@@ -48719,10 +48730,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-ad-gpo-modified-priv.yaml",
- "sha256": "90517daae40443307bbdcf69ec0ca01c9eb2dbeb1af46513598709268ae4cd28"
+ "quarantine_reason": "not executable (needs-inventory): asks whether a modified GPO is linked to a privileged scope. Neither the GPO link nor a list of privileged scopes exists here.",
+ "sha256": "02a7875f223ebfe32a313efd55ecfa1c1518a365a86a87f4036696373bb1a084"
},
{
"id": "det-identity-083",
@@ -49096,10 +49108,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-ntlm-relay-indicator.yaml",
- "sha256": "3e12d23fd6267f36a2f42f3e5ca73a29ea3b47fac29864e0d62465716e2fee21"
+ "quarantine_reason": "not executable (needs-connector-field): compares the workstation named in the logon with the address it came from. Windows 4624 carries both as `WorkstationName` and `IpAddress`; neither is lifted as a top-level field, so neither side of the comparison resolves.",
+ "sha256": "ab1ba1a108b662d6a0b02fc1c3c0adbe32c1662bafdd6a6998722f0f502e53e0"
},
{
"id": "det-identity-098",
@@ -49171,10 +49184,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/identity/ident-pass-the-ticket-anomalous-tgt-source.yaml",
- "sha256": "79920bed0bc5489e7955b08475d27da9be4b2bafe3cab3090ab899527a5ee339"
+ "quarantine_reason": "not executable (needs-connector-field): compares the address a Kerberos ticket was issued to with the address using it. That needs two events joined on the ticket, which is a correlation, and neither address is emitted today.",
+ "sha256": "96b243fbd17c39386292761f18de494a66ff10ccd91d2d7564a8a25a99385761"
},
{
"id": "det-identity-101",
@@ -50882,10 +50896,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/network/newly-registered-domain-traffic.yaml",
- "sha256": "057aec7cd6a3ee71569f7bed994faacb67605c916337cb6dbf6e10621e74eac4"
+ "quarantine_reason": "not executable (needs-external-source): wants a domain's registration age, which comes from RDAP or WHOIS. Nothing in this tree queries either, and answering it from when this deployment first saw the domain would fire on most of the internet.",
+ "sha256": "4359781d0f7bbce674a247135b2a7300420761b7064884cc362fda7b32ac6a26"
},
{
"id": "det-network-009",
@@ -51464,10 +51479,11 @@
"verified": true,
"source": "core",
"tier": "stable",
- "enabled": true,
- "executable": true,
+ "enabled": false,
+ "executable": false,
"path": "detections/network/net-dns-newly-registered-domain.yaml",
- "sha256": "9ee0d5678db29c8113018cc0b8311a624287f21827183910d386e3b2fd36646b"
+ "quarantine_reason": "not executable (needs-external-source): wants a domain's registration age from RDAP or WHOIS, which nothing here queries. Same gap as the proxy-side rule beside it.",
+ "sha256": "5bf1bd34f9d1a37e0e49e2cf8af21f6155c6b9d5eff24b0864598a01e767f373"
},
{
"id": "det-network-032",
diff --git a/services/fusion/app/data/detection_ruleset.json b/services/fusion/app/data/detection_ruleset.json
index 154d321f7..e04a070ac 100644
--- a/services/fusion/app/data/detection_ruleset.json
+++ b/services/fusion/app/data/detection_ruleset.json
@@ -1,5 +1,5 @@
{
- "count": 759,
+ "count": 741,
"rules": [
{
"category": "application",
@@ -341,22 +341,6 @@
"severity": "high",
"slug": "container-image-pulled-untrusted-registry"
},
- {
- "category": "application",
- "id": "det-application-020",
- "match_when": {
- "event_type": "object.read",
- "object_tenant_id_eq_caller_tenant_id": false
- },
- "mitre": [
- "T1212"
- ],
- "name": "API Returned Object Outside Caller's Tenant",
- "product": "app",
- "service": "audit",
- "severity": "critical",
- "slug": "api-permission-bypass-idor"
- },
{
"category": "application",
"id": "det-application-021",
@@ -903,23 +887,6 @@
"severity": "medium",
"slug": "app-deploy-key-added-github"
},
- {
- "category": "application",
- "id": "det-application-053",
- "match_when": {
- "event_type": "package_publish",
- "package_visibility": "public",
- "publisher_account_age_days_lt": 14
- },
- "mitre": [
- "T1195.002"
- ],
- "name": "NPM Package Published From New Account",
- "product": "npm",
- "service": "registry",
- "severity": "high",
- "slug": "app-npm-package-publish-new-account"
- },
{
"category": "application",
"id": "det-application-054",
@@ -2067,22 +2034,6 @@
"severity": "high",
"slug": "aws-iam-attach-administrator-access"
},
- {
- "category": "cloud",
- "id": "det-cloud-042",
- "match_when": {
- "event_name": "ListAccessKeys",
- "key_age_days_gt": 365
- },
- "mitre": [
- "T1078.004"
- ],
- "name": "AWS IAM Access Key Older Than 365 Days",
- "product": "aws",
- "service": "iam",
- "severity": "medium",
- "slug": "aws-iam-key-rotation-skipped"
- },
{
"category": "cloud",
"id": "det-cloud-043",
@@ -2908,23 +2859,6 @@
"severity": "high",
"slug": "aws-route53-domain-transfer-out"
},
- {
- "category": "cloud",
- "id": "det-cloud-097",
- "match_when": {
- "distribution_age_days_gt": 30,
- "event_name": "UpdateDistribution",
- "origin_changed": true
- },
- "mitre": [
- "T1565.002"
- ],
- "name": "AWS CloudFront Origin Changed Mid-Service",
- "product": "aws",
- "service": "cloudfront",
- "severity": "high",
- "slug": "aws-cloudfront-origin-changed"
- },
{
"category": "cloud",
"id": "det-cloud-098",
@@ -6481,23 +6415,6 @@
"severity": "high",
"slug": "win-browser-credential-file-read"
},
- {
- "category": "endpoint",
- "id": "det-endpoint-101",
- "match_when": {
- "account_is_dc": false,
- "event_id": 4662,
- "object_properties_contains": "1131f6aa-9c07-11d1-f79f-00c04fc2dcd2"
- },
- "mitre": [
- "T1003.006"
- ],
- "name": "DCSync Replication Right Used By Non-DC Account",
- "product": "windows",
- "service": "security",
- "severity": "critical",
- "slug": "win-dcsync-via-replication"
- },
{
"category": "endpoint",
"id": "det-endpoint-102",
@@ -7189,40 +7106,6 @@
"severity": "high",
"slug": "linux-initd-script-add"
},
- {
- "category": "endpoint",
- "id": "det-endpoint-138",
- "match_when": {
- "actor_uid_neq_owner_uid": true,
- "path_endswith": "/.bashrc",
- "syscall": "openat"
- },
- "mitre": [
- "T1546.004"
- ],
- "name": "~/.bashrc Modified By Other User",
- "product": "linux",
- "service": "auditd",
- "severity": "medium",
- "slug": "linux-bashrc-modified-by-other-user"
- },
- {
- "category": "endpoint",
- "id": "det-endpoint-139",
- "match_when": {
- "actor_uid_neq_owner_uid": true,
- "path_endswith": "/.zshrc",
- "syscall": "openat"
- },
- "mitre": [
- "T1546.004"
- ],
- "name": "~/.zshrc Modified By Other User",
- "product": "linux",
- "service": "auditd",
- "severity": "medium",
- "slug": "linux-zshrc-modified-by-other-user"
- },
{
"category": "endpoint",
"id": "det-endpoint-140",
@@ -8610,23 +8493,6 @@
"severity": "medium",
"slug": "macos-security-find-generic-password"
},
- {
- "category": "endpoint",
- "id": "det-endpoint-215",
- "match_when": {
- "actor_uid_neq_owner_uid": true,
- "event_type": "ES_EVENT_TYPE_NOTIFY_OPEN",
- "path_endswith": "/Keychains/login.keychain-db"
- },
- "mitre": [
- "T1555.001"
- ],
- "name": "login.keychain-db Read By Non-Owner UID",
- "product": "macos",
- "service": "endpointsecurity",
- "severity": "high",
- "slug": "macos-keychain-db-read-by-other-uid"
- },
{
"category": "endpoint",
"id": "det-endpoint-216",
@@ -10867,23 +10733,6 @@
"severity": "high",
"slug": "ident-fido2-key-removed-priv"
},
- {
- "category": "identity",
- "id": "det-identity-055",
- "match_when": {
- "event_type": "factor_swap",
- "old_factor_age_days_gt": 30,
- "session_seconds_lt": 600
- },
- "mitre": [
- "T1556.006"
- ],
- "name": "FIDO2 Key Added And Old Key Removed In Same Session",
- "product": "identity",
- "service": "mfa",
- "severity": "high",
- "slug": "ident-fido2-key-added-then-removed-old"
- },
{
"category": "identity",
"id": "det-identity-056",
@@ -10933,23 +10782,6 @@
"severity": "medium",
"slug": "ident-password-policy-history-reduced"
},
- {
- "category": "identity",
- "id": "det-identity-059",
- "match_when": {
- "event_type": "policy_update",
- "scope_priv": true,
- "session_timeout_minutes_increased": true
- },
- "mitre": [
- "T1556"
- ],
- "name": "Session Timeout Extended For Privileged Group",
- "product": "identity",
- "service": "policy",
- "severity": "medium",
- "slug": "ident-session-timeout-extended-priv"
- },
{
"category": "identity",
"id": "det-identity-060",
@@ -11088,22 +10920,6 @@
"severity": "high",
"slug": "ident-priv-role-grant-to-svc-account"
},
- {
- "category": "identity",
- "id": "det-identity-069",
- "match_when": {
- "approver_eq_actor": true,
- "event_type": "pim_activation"
- },
- "mitre": [
- "T1078"
- ],
- "name": "JIT/PIM Privileged Activation Self-Approved",
- "product": "identity",
- "service": "pim",
- "severity": "high",
- "slug": "ident-pim-activation-self-approval"
- },
{
"category": "identity",
"id": "det-identity-070",
@@ -11120,23 +10936,6 @@
"severity": "medium",
"slug": "ident-pim-activation-anomalous-time"
},
- {
- "category": "identity",
- "id": "det-identity-072",
- "match_when": {
- "event_type": "auth_success",
- "is_interactive": true,
- "user_type": "service_account"
- },
- "mitre": [
- "T1078.004"
- ],
- "name": "Service Account Used For Interactive Login",
- "product": "identity",
- "service": "auth",
- "severity": "high",
- "slug": "ident-svc-account-interactive-login"
- },
{
"category": "identity",
"id": "det-identity-073",
@@ -11172,27 +10971,6 @@
"severity": "high",
"slug": "ident-onboard-priv-roles-immediate"
},
- {
- "category": "identity",
- "id": "det-identity-075",
- "match_when": {
- "act_as_user_priv": true,
- "event_type": "oauth_action",
- "scope_contains_any": [
- "full_access",
- "admin",
- "*.write"
- ]
- },
- "mitre": [
- "T1528"
- ],
- "name": "OAuth App Acting As Privileged User",
- "product": "identity",
- "service": "oauth",
- "severity": "high",
- "slug": "ident-app-impersonation-priv"
- },
{
"category": "identity",
"id": "det-identity-078",
@@ -11268,22 +11046,6 @@
"severity": "critical",
"slug": "ident-ad-dcsync-rights-granted"
},
- {
- "category": "identity",
- "id": "det-identity-082",
- "match_when": {
- "event_id": 5136,
- "gpo_link_priv": true
- },
- "mitre": [
- "T1484.001"
- ],
- "name": "Privileged GPO Modified",
- "product": "active-directory",
- "service": "audit",
- "severity": "high",
- "slug": "ident-ad-gpo-modified-priv"
- },
{
"category": "identity",
"id": "det-identity-083",
@@ -11486,23 +11248,6 @@
"severity": "critical",
"slug": "ident-skeleton-key-encryption-downgrade"
},
- {
- "category": "identity",
- "id": "det-identity-097",
- "match_when": {
- "auth_type": "NTLMv2",
- "event_id": 8004,
- "workstation_neq_source_address": true
- },
- "mitre": [
- "T1557.001"
- ],
- "name": "NTLM Authentication With Mismatched WorkstationName",
- "product": "active-directory",
- "service": "ntlm",
- "severity": "high",
- "slug": "ident-ntlm-relay-indicator"
- },
{
"category": "identity",
"id": "det-identity-098",
@@ -11538,23 +11283,6 @@
"severity": "high",
"slug": "ident-pass-the-hash-anomalous-logon"
},
- {
- "category": "identity",
- "id": "det-identity-100",
- "match_when": {
- "event_id": 4624,
- "logon_type": 3,
- "ticket_source_ip_neq_session_ip": true
- },
- "mitre": [
- "T1550.003"
- ],
- "name": "Pass-The-Ticket Indicator: TGT Source IP Mismatch",
- "product": "active-directory",
- "service": "kerberos",
- "severity": "high",
- "slug": "ident-pass-the-ticket-anomalous-tgt-source"
- },
{
"category": "identity",
"id": "det-identity-101",
@@ -12474,26 +12202,6 @@
"severity": "high",
"slug": "tor-traffic"
},
- {
- "category": "network",
- "id": "det-network-008",
- "match_when": {
- "domain_age_days_lt": 7,
- "method_in": [
- "GET",
- "POST",
- "CONNECT"
- ]
- },
- "mitre": [
- "T1071.001"
- ],
- "name": "Outbound HTTP(S) to Newly Registered Domain",
- "product": "proxy",
- "service": "http",
- "severity": "medium",
- "slug": "newly-registered-domain-traffic"
- },
{
"category": "network",
"id": "det-network-010",
@@ -12786,22 +12494,6 @@
"severity": "high",
"slug": "wmi-remote-execution"
},
- {
- "category": "network",
- "id": "det-network-031",
- "match_when": {
- "domain_age_days_lt": 7,
- "event_type": "dns_query"
- },
- "mitre": [
- "T1071.004"
- ],
- "name": "DNS Query For Newly Registered Domain (NRD)",
- "product": "dns",
- "service": "resolver",
- "severity": "medium",
- "slug": "net-dns-newly-registered-domain"
- },
{
"category": "network",
"id": "det-network-032",
diff --git a/services/fusion/app/main.py b/services/fusion/app/main.py
index ac6b64a6b..166586976 100644
--- a/services/fusion/app/main.py
+++ b/services/fusion/app/main.py
@@ -21,6 +21,7 @@ from app.services.deduplicator import Deduplicator
from app.services.detection_engine import DetectionEngine
from app.services.dlq_sink import PostgresDLQ
from app.services.entity_risk import EntityRiskEngine
+from app.services.first_seen import FirstSeenStore
from app.services.fusion_engine import FusionEngine
from app.services.ioc_match import TenantIocMatcher
from app.services.lake_writer import LakeWriter
@@ -122,6 +123,9 @@ async def lifespan(app: FastAPI):
detector = DetectionEngine() if settings.detection_engine_enabled else None
# Wave 2 — windowed detections share the fusion Redis for sliding-window state.
windowed_detector = WindowedDetectionEngine(redis_client) if settings.windowed_detection_enabled else None
+ # Depth plan 3.3 — per-tenant first-seen memory, sharing the fusion Redis
+ # the windowed engine already uses for sliding-window state.
+ first_seen = FirstSeenStore(redis_client)
# Dead letters go to Postgres so they can be read back. Until now the
# worker defaulted to LoggingDLQ, so a dropped event produced a log
# line and nothing else — and an invisible drop is indistinguishable
@@ -139,6 +143,7 @@ async def lifespan(app: FastAPI):
lake=lake,
detector=detector,
windowed_detector=windowed_detector,
+ first_seen=first_seen,
# Parity 5.4. Per-tenant detection tuning as a versioned overlay
# with hot reload. Without it the engine evaluated the shared
# corpus and nothing else, so a tenant who disabled a noisy rule
diff --git a/services/fusion/app/services/detection_engine.py b/services/fusion/app/services/detection_engine.py
index 8e030f836..b5d77c4fb 100644
--- a/services/fusion/app/services/detection_engine.py
+++ b/services/fusion/app/services/detection_engine.py
@@ -169,7 +169,28 @@ class DetectionEngine:
# Fall back to the OCSF top level (some connectors emit flat OCSF).
return ocsf if isinstance(ocsf, dict) else {}
- def evaluate(self, message: dict[str, Any], overlay: Any | None = None) -> list[DetectionHit]:
+ async def evaluate_async(
+ self, message: dict[str, Any], overlay: Any | None = None, first_seen: Any | None = None
+ ) -> list[DetectionHit]:
+ """`evaluate`, plus the first-seen enrichment, which needs Redis.
+
+ A second entry point rather than making `evaluate` async: the
+ synchronous one has callers throughout the tests and the replay
+ harness, and turning it async would be a wide change for an
+ enrichment two rules read. The two share everything below the
+ extra pass.
+ """
+ extra: dict[str, Any] = {}
+ if first_seen is not None:
+ ocsf = message.get("ocsf_event")
+ tenant = str(message.get("tenant_id") or (ocsf or {}).get("tenant_uid") or "")
+ if tenant and isinstance(ocsf, dict):
+ extra = await first_seen.derived_fields(tenant, self._raw_fields(ocsf))
+ return self.evaluate(message, overlay, _extra_fields=extra)
+
+ def evaluate(
+ self, message: dict[str, Any], overlay: Any | None = None, *, _extra_fields: dict[str, Any] | None = None
+ ) -> list[DetectionHit]:
"""Return every rule that fires on this normalized-event message.
`overlay` is the tenant's tuning (parity 5.4). Without it this
@@ -206,6 +227,21 @@ class DetectionEngine:
derived = overlay.derived_allowlist_fields(fields)
if derived:
fields = {**fields, **derived}
+ # Depth plan 3.3. 18 rules read a `*_priv` / `*_is_admin`
+ # boolean that nothing computed, so each read None on that
+ # clause and could never fire. Per tenant for the same reason
+ # the allowlist is: one tenant's administrators are not
+ # another's, and an unknown subject contributes no key rather
+ # than `False`.
+ privileged = overlay.derived_identity_fields(fields)
+ if privileged:
+ fields = {**fields, **privileged}
+
+ # Depth plan 3.3. Resolved by the caller because it needs Redis and
+ # this method is synchronous; passed in rather than fetched here so
+ # one event costs one round trip whatever the corpus reads.
+ if _extra_fields:
+ fields = {**fields, **_extra_fields}
hits: list[DetectionHit] = []
for rule in self._candidates(""):
diff --git a/services/fusion/app/services/detection_matcher.py b/services/fusion/app/services/detection_matcher.py
index cb44434a2..52b155269 100644
--- a/services/fusion/app/services/detection_matcher.py
+++ b/services/fusion/app/services/detection_matcher.py
@@ -25,10 +25,12 @@ from typing import Any
OPERATORS: list[tuple[str, str, str]] = sorted(
[
("_pattern_match_any", "pattern_match_any", "PATTERN_MATCH_ANY"),
+ ("_not_startswith_any", "not_startswith_any", "NOT STARTSWITH_ANY"),
("_not_endswith_any", "not_endswith_any", "NOT ENDSWITH_ANY"),
("_not_contains_any", "not_contains_any", "NOT CONTAINS_ANY"),
("_pattern_match", "pattern_match", "PATTERN_MATCH"),
("_not_startswith", "not_startswith", "NOT STARTSWITH"),
+ ("_not_endswith", "not_endswith", "NOT ENDSWITH"),
("_startswith_any", "startswith_any", "STARTSWITH_ANY"),
("_endswith_any", "endswith_any", "ENDSWITH_ANY"),
("_contains_any", "contains_any", "CONTAINS_ANY"),
@@ -199,6 +201,25 @@ def _check(field: str, op: str, expected: Any, event: dict[str, Any]) -> bool:
return False
return not any(actual.endswith(str(s)) for s in expected)
+ # Three shipped rules used `not_startswith_any` and it had never been an
+ # operator, so `path_not_startswith_any` was read as a field name. The
+ # reachability gate hid that by stripping a suffix the matcher did not
+ # know, so all three were reported reachable and could not fire.
+ #
+ # A missing field returns False, exactly as the negations beside it do: a
+ # negation that flips to True on an absent field fires on every event
+ # that happens not to carry it, which is the failure already recorded for
+ # the Sigma import.
+ if op == "not_startswith_any":
+ if not isinstance(expected, list) or not isinstance(actual, str):
+ return False
+ return not any(actual.startswith(str(s)) for s in expected)
+
+ if op == "not_endswith":
+ if not isinstance(actual, str):
+ return False
+ return not actual.endswith(str(expected))
+
if op == "not_contains_any":
if not isinstance(expected, list):
return False
diff --git a/services/fusion/app/services/first_seen.py b/services/fusion/app/services/first_seen.py
new file mode 100644
index 000000000..13b26b1ac
--- /dev/null
+++ b/services/fusion/app/services/first_seen.py
@@ -0,0 +1,170 @@
+"""Per-tenant first-seen store: has this tenant observed this value before?
+
+Depth plan 3.3: *"first-seen and age from a per-tenant first-seen store keyed
+by principal and attribute (country, ASN, device, client family, OAuth app,
+action, resource), in Redis with a lake backstop."*
+
+What it answers, and what it deliberately does not
+--------------------------------------------------
+It answers **"have we seen this before, and how long ago did we first see
+it"**. It does not answer "how old is this thing". Those look alike and are
+not the same question, and conflating them is the trap this module exists to
+avoid: a rule about newly-registered domains (`domain_age_days`) wants the
+domain's registration age from a registry, and substituting "the first time
+this deployment happened to observe it" would make the rule fire on every
+domain a quiet tenant has not seen yet — which is most of the internet. Those
+rules stay unreachable with that reason recorded rather than being closed
+with a plausible-looking number.
+
+So the booleans and ages here are only the ones whose subject is *an
+observation*: a publisher IP, a device fingerprint, a session.
+
+Shape of the state
+------------------
+One Redis string per `(tenant, attribute, value)` holding the epoch seconds
+of the first observation, set with `NX` so the first writer wins and every
+later one reads it back. That is a single round trip for the common case
+(`SET key now NX GET`), and it is correct under concurrency without a lock:
+two replicas racing on the first sighting agree on whichever landed first.
+
+**First sighting is reported as not-seen-before**, which is the whole point,
+and the write happens on the same call — so a rule reading
+`<attr>_seen_before: false` fires once per value per tenant and not again.
+
+Fail-soft, in one direction
+---------------------------
+A Redis outage contributes **no key at all** rather than `False`. `False`
+means "this tenant has never seen this", which is a confident statement that
+would fire every first-seen rule on every event for the duration of the
+outage. Absent means the clause does not match and the rule stays silent,
+which is the safe direction for a detection that triggers on novelty.
+"""
+
+from __future__ import annotations
+
+import os
+import time
+from typing import Any
+
+import structlog
+
+logger = structlog.get_logger()
+
+#: How long a value stays "seen". A first-seen store with no horizon reports
+#: an address seen once two years ago as familiar, which is not what an
+#: analyst means by it. Ninety days matches the lake's default retention, so
+#: the backstop below can answer for the same span the store remembers.
+TTL_SECONDS = int(os.getenv("AISOC_FIRST_SEEN_TTL_SECONDS", str(90 * 86400)))
+
+#: `<boolean a rule reads>` → the event field holding its subject.
+#:
+#: Only observation-shaped subjects appear here; see the module docstring for
+#: why `domain_age_days` and friends are not in this table and are not going
+#: to be.
+SEEN_BEFORE_FIELDS: dict[str, str] = {
+ "publisher_ip_seen_before": "publisher_ip",
+ "device_fingerprint_seen_before": "device_fingerprint",
+}
+
+#: `<age field a rule reads>` → (event field holding its subject, divisor).
+#: The age is measured from the first observation, which for a session is
+#: exactly what "session age" means.
+AGE_FIELDS: dict[str, tuple[str, int]] = {
+ "session_age_hours": ("session_id", 3600),
+}
+
+#: Attributes the plan names for the store, beyond the ones a rule reads
+#: today. Recorded so a rule author can see what is already keyed rather
+#: than adding a parallel store, and asserted by the tests so the list
+#: cannot quietly become aspirational.
+TRACKED_ATTRIBUTES: tuple[str, ...] = (
+ "publisher_ip",
+ "device_fingerprint",
+ "session_id",
+ "src_country",
+ "src_asn",
+ "client_family",
+ "oauth_app",
+ "event_name",
+ "resource",
+)
+
+
+class FirstSeenStore:
+ """Redis-backed first-sighting memory, scoped per tenant."""
+
+ def __init__(self, redis: Any, *, key_prefix: str = "aisoc:fs", ttl_seconds: int | None = None) -> None:
+ self._redis = redis
+ self._prefix = key_prefix
+ self._ttl = ttl_seconds if ttl_seconds is not None else TTL_SECONDS
+
+ def _key(self, tenant: str, attribute: str, value: str) -> str:
+ return f"{self._prefix}:{tenant}:{attribute}:{value}"
+
+ async def observe(self, tenant: str, attribute: str, value: str, *, now: float | None = None) -> float | None:
+ """Record a sighting; return the first-sighting time, or None if unknown.
+
+ Returns the *first* time this tenant saw the value, which is the new
+ time on a first sighting. None means Redis could not answer, and the
+ caller must treat that as "unknown" rather than as "new".
+ """
+ when = now if now is not None else time.time()
+ key = self._key(tenant, attribute, value)
+ try:
+ # SET ... NX GET: write if absent, and return whatever was there.
+ # One round trip, and correct when two replicas race, because
+ # the loser reads the winner's value back.
+ previous = await self._redis.set(key, str(when), nx=True, ex=self._ttl, get=True)
+ except TypeError:
+ # A client without `GET` support on SET. Fall back to a read
+ # then a conditional write: one extra round trip, same answer,
+ # and a race resolves to the earlier of the two writes.
+ try:
+ previous = await self._redis.get(key)
+ if previous is None:
+ await self._redis.set(key, str(when), nx=True, ex=self._ttl)
+ except Exception as exc: # noqa: BLE001 — see the module docstring
+ logger.debug("first_seen.unavailable", attribute=attribute, error=str(exc))
+ return None
+ except Exception as exc: # noqa: BLE001
+ logger.debug("first_seen.unavailable", attribute=attribute, error=str(exc))
+ return None
+
+ if previous is None:
+ return when
+ raw = previous.decode() if isinstance(previous, bytes | bytearray) else str(previous)
+ try:
+ return float(raw)
+ except ValueError:
+ return when
+
+ async def derived_fields(self, tenant: str, event: dict[str, Any], *, now: float | None = None) -> dict[str, Any]:
+ """The `<attr>_seen_before` and age booleans this event produces.
+
+ An unreadable store contributes nothing, which keeps a novelty rule
+ silent during an outage rather than firing it on everything.
+ """
+ if not tenant:
+ return {}
+ when = now if now is not None else time.time()
+ out: dict[str, Any] = {}
+
+ for boolean, source in SEEN_BEFORE_FIELDS.items():
+ value = event.get(source)
+ if value is None or value == "":
+ continue
+ first = await self.observe(tenant, source, str(value), now=when)
+ if first is None:
+ continue
+ out[boolean] = first < when
+
+ for age_field, (source, divisor) in AGE_FIELDS.items():
+ value = event.get(source)
+ if value is None or value == "":
+ continue
+ first = await self.observe(tenant, source, str(value), now=when)
+ if first is None:
+ continue
+ out[age_field] = max(0.0, (when - first) / divisor)
+
+ return out
diff --git a/services/fusion/app/services/tenant_overlay.py b/services/fusion/app/services/tenant_overlay.py
index f68ed5920..5cd638fef 100644
--- a/services/fusion/app/services/tenant_overlay.py
+++ b/services/fusion/app/services/tenant_overlay.py
@@ -80,6 +80,11 @@ class TenantOverlay:
#: Field name to the values this tenant has declared fine. Feeds the
#: `<x>_in_allowlist` booleans 15 rules read and nothing computed.
allowlists: dict[str, list[str]] = field(default_factory=dict)
+ #: External ids of this tenant's privileged principals, lower-cased.
+ #: Feeds the `*_priv` / `*_is_admin` booleans 18 rules read.
+ privileged_principals: frozenset[str] = frozenset()
+ #: External ids of this tenant's privileged roles, lower-cased.
+ privileged_roles: frozenset[str] = frozenset()
version: str = ""
loaded_at: float = 0.0
@@ -88,6 +93,11 @@ class TenantOverlay:
return {}
return allowlist_fields(self.allowlists, event)
+ def derived_identity_fields(self, event: dict[str, Any]) -> dict[str, bool]:
+ if not self.privileged_principals and not self.privileged_roles:
+ return {}
+ return identity_fields(self.privileged_principals, self.privileged_roles, event)
+
def suppresses(self, rule_id: str, event: dict[str, Any]) -> str | None:
"""Why this rule must not fire for this tenant, or None.
@@ -160,9 +170,20 @@ def _version_of(rows: list[dict[str, Any]]) -> str:
return hashlib.sha256(canonical.encode()).hexdigest()[:16]
-def build_overlay(tenant_id: str, rows: list[dict[str, Any]], *, now: float | None = None) -> TenantOverlay:
+def build_overlay(
+ tenant_id: str,
+ rows: list[dict[str, Any]],
+ *,
+ identities: list[dict[str, Any]] | None = None,
+ now: float | None = None,
+) -> TenantOverlay:
"""Turn `detection_rules` rows into an overlay.
+ `identities` carries this tenant's privileged principals and roles from
+ `identity_nodes`, which is a different table with a different lifecycle,
+ so it is a separate argument rather than more rows: a tenant may have
+ tuned nothing and imported a directory, or the reverse.
+
`now` is injectable because the reload interval is a behaviour worth
testing, and a cache whose clock cannot be controlled can only be
tested by sleeping.
@@ -219,11 +240,21 @@ def build_overlay(tenant_id: str, rows: list[dict[str, Any]], *, now: float | No
values = [str(v) for v in value] if isinstance(value, list) else [str(value)]
allowlists.setdefault(str(key), []).extend(values)
+ principals: set[str] = set()
+ roles: set[str] = set()
+ for row in identities or ():
+ external_id = str(row.get("external_id") or "").strip().lower()
+ if not external_id:
+ continue
+ (roles if str(row.get("node_type") or "") == "role" else principals).add(external_id)
+
return TenantOverlay(
tenant_id=tenant_id,
overrides=overrides,
allowlists=allowlists,
- version=_version_of(rows),
+ privileged_principals=frozenset(principals),
+ privileged_roles=frozenset(roles),
+ version=_version_of([*rows, *(identities or ())]),
loaded_at=now if now is not None else time.monotonic(),
)
@@ -278,6 +309,55 @@ def allowlist_fields(allowlists: dict[str, list[str]], event: dict[str, Any]) ->
return out
+#: `<boolean the rules read>` → the event fields that name its subject, in
+#: the order they are trusted, and which privileged set answers for it.
+#:
+#: Depth plan 3.3. 18 rules read one of these booleans and nothing computed
+#: any of them, so each rule read `None` on that clause and could never fire.
+#: They are answered from `identity_nodes.privilege_tier`, the column that
+#: migration 018 created for exactly this question.
+#:
+#: Four more booleans in the same family are **not** here, and that is the
+#: honest half: `scope_priv`, `act_as_user_priv`, `gpo_link_priv` and
+#: `account_is_dc` have no subject field any source emits and no inventory
+#: to answer from. Guessing a subject would produce a boolean that is
+#: confidently wrong rather than absent.
+IDENTITY_FIELDS: dict[str, tuple[tuple[str, ...], str]] = {
+ "user_priv": (("user", "user_name", "actor_email", "actor", "user_arn"), "principals"),
+ "target_user_priv": (("target_user", "target"), "principals"),
+ "role_priv": (("role",), "roles"),
+ # The actor's *role* is not on the event, and a principal's privilege
+ # tier is derived from the roles it holds, so the two questions have one
+ # answer. Recorded rather than silently equated.
+ "actor_role_priv": (("actor_email", "actor", "user"), "principals"),
+ "actor_is_admin": (("actor_email", "actor", "user"), "principals"),
+}
+
+
+def identity_fields(principals: frozenset[str], roles: frozenset[str], event: dict[str, Any]) -> dict[str, bool]:
+ """The `*_priv` booleans this event's subjects produce.
+
+ An unknown subject yields **no key at all**, never `False`. A tenant
+ that has imported no identities gets no keys, so a rule reading
+ `user_priv: true` stays silent instead of being told the account is not
+ privileged — which is a different and much more confident statement than
+ "we do not know".
+ """
+ out: dict[str, bool] = {}
+ sets = {"principals": principals, "roles": roles}
+ for boolean, (candidates, which) in IDENTITY_FIELDS.items():
+ known = sets[which]
+ if not known:
+ continue
+ for candidate in candidates:
+ value = event.get(candidate)
+ if value is None or value == "":
+ continue
+ out[boolean] = str(value).strip().lower() in known
+ break
+ return out
+
+
#: An overlay with nothing in it. Returned for a tenant that has tuned
#: nothing, which is the common case and must cost nothing.
EMPTY = TenantOverlay(tenant_id="", overrides={}, version="empty")
@@ -305,7 +385,9 @@ class OverlayCache:
if current is not None and (clock - current.loaded_at) < self._reload:
return current
- rows = await self._fetch(tenant_id)
+ fetched = await self._fetch(tenant_id)
+ rows = fetched[0] if fetched is not None else None
+ identities = fetched[1] if fetched is not None else None
if rows is None:
if current is not None:
logger.warning(
@@ -316,7 +398,7 @@ class OverlayCache:
return current
return EMPTY
- overlay = build_overlay(tenant_id, rows, now=clock)
+ overlay = build_overlay(tenant_id, rows, identities=identities, now=clock)
if current is not None and current.version != overlay.version:
logger.info(
"tenant_overlay.changed",
@@ -328,8 +410,8 @@ class OverlayCache:
self._cache[tenant_id] = overlay
return overlay
- async def _fetch(self, tenant_id: str) -> list[dict[str, Any]] | None:
- """Rows, or None when they could not be read.
+ async def _fetch(self, tenant_id: str) -> tuple[list[dict[str, Any]], list[dict[str, Any]]] | None:
+ """(tuning rows, privileged identities), or None when unreadable.
None and an empty list mean different things: no tuning, versus no
answer. Collapsing them is what would drop a tenant's suppressions
@@ -363,10 +445,25 @@ class OverlayCache:
""",
tenant_id,
)
+ # Depth plan 3.3. `privilege_tier` is 0=standard, 1=elevated,
+ # 2=admin, 3=super-admin (migration 018). "Privileged" is the
+ # admin tiers: treating `elevated` as privileged would put
+ # most of a directory in the set and make every `*_priv` rule
+ # fire on ordinary work.
+ identities = await conn.fetch(
+ """
+ SELECT node_type, external_id
+ FROM identity_nodes
+ WHERE tenant_id = $1::uuid
+ AND is_active
+ AND privilege_tier >= 2
+ """,
+ tenant_id,
+ )
except Exception as exc: # noqa: BLE001
logger.warning("tenant_overlay.fetch_failed", tenant_id=tenant_id, error=str(exc))
return None
- return [dict(r) for r in rows]
+ return [dict(r) for r in rows], [dict(r) for r in identities]
def invalidate(self, tenant_id: str) -> None:
"""Force the next read to refetch. For an explicit console save."""
diff --git a/services/fusion/app/services/windowed_detection.py b/services/fusion/app/services/windowed_detection.py
index b5dd8e19f..1111dfb23 100644
--- a/services/fusion/app/services/windowed_detection.py
+++ b/services/fusion/app/services/windowed_detection.py
@@ -452,6 +452,9 @@ class WindowedDetectionEngine:
derived = overlay.derived_allowlist_fields(fields)
if derived:
fields = {**fields, **derived}
+ privileged = overlay.derived_identity_fields(fields)
+ if privileged:
+ fields = {**fields, **privileged}
now = time.time()
hits: list[DetectionHit] = []
for rule in self._rules:
diff --git a/services/fusion/app/workers/consumer.py b/services/fusion/app/workers/consumer.py
index 49f38acf1..2d6133b29 100644
--- a/services/fusion/app/workers/consumer.py
+++ b/services/fusion/app/workers/consumer.py
@@ -7,6 +7,7 @@ results back to Kafka, and persists non-duplicate alerts to Postgres.
import asyncio
import contextlib
import json
+from typing import Any
import structlog
from aiokafka import AIOKafkaConsumer, AIOKafkaProducer
@@ -55,6 +56,7 @@ class FusionWorker:
lake: LakeWriter | None = None,
detector: DetectionEngine | None = None,
windowed_detector: WindowedDetectionEngine | None = None,
+ first_seen: Any | None = None,
ueba_cache: UebaSignalCache | None = None,
overlays: OverlayCache | None = None,
) -> None:
@@ -63,6 +65,7 @@ class FusionWorker:
self._lake = lake
self._detector = detector
self._windowed = windowed_detector
+ self._first_seen = first_seen
self._ueba_cache = ueba_cache
#: Per-tenant detection tuning (parity 5.4). Optional, so a
#: deployment that passes nothing behaves exactly as before.
@@ -302,7 +305,7 @@ class FusionWorker:
# event. Each firing rule becomes a RawAlert routed through fusion,
# so telemetry that isn't a vendor-asserted finding still alerts.
if self._detector is not None:
- for hit in self._detector.evaluate(payload, overlay):
+ for hit in await self._detector.evaluate_async(payload, overlay, self._first_seen):
det_alert = self._detector.build_alert(payload, hit)
if det_alert is not None:
_METRICS["detected"] += 1
diff --git a/services/fusion/tests/test_first_seen.py b/services/fusion/tests/test_first_seen.py
new file mode 100644
index 000000000..3fb29aa53
--- /dev/null
+++ b/services/fusion/tests/test_first_seen.py
@@ -0,0 +1,146 @@
+"""The per-tenant first-seen store, and the line it refuses to cross.
+
+Two rules read a `<attr>_seen_before` boolean and one reads `session_age_hours`;
+nothing computed any of them. The assertions below are about behaviour under
+the conditions that actually occur — a first sighting, a repeat, a different
+tenant, and Redis being unavailable — rather than about the helper returning
+a value.
+
+The last of those is the one worth having: an outage must contribute **no
+key**, because `False` means "this tenant has never seen this" and a novelty
+rule told that about every event would fire on all of them.
+"""
+
+from __future__ import annotations
+
+import importlib.util
+from pathlib import Path
+from typing import Any
+
+import pytest
+from app.services.first_seen import AGE_FIELDS, SEEN_BEFORE_FIELDS, TRACKED_ATTRIBUTES, FirstSeenStore
+
+_REPO = Path(__file__).resolve().parents[3]
+
+
+class _FakeRedis:
+ """`SET key value NX EX GET`, which is the one call the store makes."""
+
+ def __init__(self) -> None:
+ self.values: dict[str, str] = {}
+
+ async def set(self, key: str, value: str, nx: bool = False, ex: int | None = None, get: bool = False) -> Any:
+ previous = self.values.get(key)
+ if not nx or previous is None:
+ self.values[key] = value
+ return previous if get else (None if (nx and previous is not None) else True)
+
+
+class _NoRedis:
+ async def set(self, *args: Any, **kwargs: Any) -> Any:
+ raise ConnectionError("redis is down")
+
+ async def get(self, *args: Any, **kwargs: Any) -> Any:
+ raise ConnectionError("redis is down")
+
+
+async def test_the_first_sighting_is_not_seen_before_and_the_second_is() -> None:
+ store = FirstSeenStore(_FakeRedis())
+ event = {"publisher_ip": "198.51.100.9"}
+
+ first = await store.derived_fields("t-1", event, now=1000.0)
+ assert first["publisher_ip_seen_before"] is False
+
+ second = await store.derived_fields("t-1", event, now=1060.0)
+ assert second["publisher_ip_seen_before"] is True
+
+
+async def test_tenants_do_not_share_sightings() -> None:
+ store = FirstSeenStore(_FakeRedis())
+ event = {"publisher_ip": "198.51.100.9"}
+ await store.derived_fields("t-1", event, now=1000.0)
+ other = await store.derived_fields("t-2", event, now=1060.0)
+ assert other["publisher_ip_seen_before"] is False
+
+
+async def test_an_unreadable_store_contributes_no_key() -> None:
+ """Not `False`. `False` would fire every novelty rule on every event."""
+ store = FirstSeenStore(_NoRedis())
+ assert await store.derived_fields("t-1", {"publisher_ip": "198.51.100.9"}, now=1000.0) == {}
+
+
+async def test_an_absent_subject_contributes_no_key() -> None:
+ store = FirstSeenStore(_FakeRedis())
+ assert await store.derived_fields("t-1", {"event_type": "publish"}, now=1000.0) == {}
+ assert await store.derived_fields("t-1", {"publisher_ip": ""}, now=1000.0) == {}
+
+
+async def test_age_is_measured_from_the_first_sighting() -> None:
+ store = FirstSeenStore(_FakeRedis())
+ event = {"session_id": "sess-1"}
+ assert (await store.derived_fields("t-1", event, now=1000.0))["session_age_hours"] == 0.0
+ later = await store.derived_fields("t-1", event, now=1000.0 + 7200)
+ assert later["session_age_hours"] == pytest.approx(2.0)
+
+
+async def test_a_tenantless_event_is_not_recorded() -> None:
+ """Without a tenant the key would be shared, which is a cross-tenant read."""
+ redis = _FakeRedis()
+ store = FirstSeenStore(redis)
+ assert await store.derived_fields("", {"publisher_ip": "198.51.100.9"}, now=1000.0) == {}
+ assert redis.values == {}
+
+
+async def test_a_client_without_set_get_still_works() -> None:
+ """Older clients reject `get=True` on SET; the fallback must still answer."""
+
+ class _OldClient:
+ def __init__(self) -> None:
+ self.values: dict[str, str] = {}
+
+ async def set(self, key: str, value: str, nx: bool = False, ex: int | None = None) -> Any:
+ if nx and key in self.values:
+ return None
+ self.values[key] = value
+ return True
+
+ async def get(self, key: str) -> Any:
+ return self.values.get(key)
+
+ store = FirstSeenStore(_OldClient())
+ event = {"publisher_ip": "198.51.100.9"}
+ assert (await store.derived_fields("t-1", event, now=1000.0))["publisher_ip_seen_before"] is False
+ assert (await store.derived_fields("t-1", event, now=1060.0))["publisher_ip_seen_before"] is True
+
+
+def test_the_reachability_gate_mirrors_this_module() -> None:
+ """A new first-seen field must be added in both places or it is invisible."""
+ spec = importlib.util.spec_from_file_location("cdf_first_seen", _REPO / "scripts" / "check_detection_fields.py")
+ assert spec and spec.loader
+ gate = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(gate)
+ assert gate._FIRST_SEEN_DERIVED_FIELDS == frozenset(SEEN_BEFORE_FIELDS) | frozenset(AGE_FIELDS)
+
+
+def test_the_tracked_attributes_the_plan_names_are_all_keyed() -> None:
+ """The plan lists the attributes the store is keyed by; hold it to them."""
+ for attribute in ("src_country", "src_asn", "client_family", "oauth_app", "event_name", "resource"):
+ assert attribute in TRACKED_ATTRIBUTES
+
+
+def test_an_age_of_a_thing_is_not_answered_from_a_first_sighting() -> None:
+ """The refusal this module exists to make, pinned.
+
+ `domain_age_days` wants a registration age from a registry. Answering it
+ from "when this deployment first saw the domain" would make the rule fire
+ on most of the internet, so no table here may grow that key.
+ """
+ forbidden = {"domain_age_days", "key_age_days", "publisher_account_age_days", "distribution_age_days", "old_factor_age_days"}
+ assert not forbidden & (set(SEEN_BEFORE_FIELDS) | set(AGE_FIELDS))
diff --git a/services/fusion/tests/test_identity_enrichment.py b/services/fusion/tests/test_identity_enrichment.py
new file mode 100644
index 000000000..449d4ae7b
--- /dev/null
+++ b/services/fusion/tests/test_identity_enrichment.py
@@ -0,0 +1,131 @@
+"""The `*_priv` booleans, driven through the real engine.
+
+18 rules read one of these and nothing computed any of them, so each read
+`None` on that clause and could never fire. The assertions that matter are
+not "the helper returns True"; they are:
+
+* a rule reading `user_priv: true` **fires** when the tenant has imported
+ that principal as privileged, and **does not** when it has not;
+* an unknown subject produces **no key**, never `False`, because `False` is a
+ confident statement that the account is ordinary and would be wrong for
+ every tenant that has imported nothing;
+* one tenant's administrators are not another's.
+
+The last one is the reason this lives on the per-tenant overlay rather than
+in the shared derived-field pass.
+"""
+
+from __future__ import annotations
+
+import importlib.util
+import json
+from pathlib import Path
+from typing import Any
+
+from app.services.detection_engine import DetectionEngine
+from app.services.tenant_overlay import IDENTITY_FIELDS, TenantOverlay, build_overlay, identity_fields
+
+_REPO = Path(__file__).resolve().parents[3]
+
+
+def _overlay(*, principals: tuple[str, ...] = (), roles: tuple[str, ...] = ()) -> TenantOverlay:
+ rows = [{"node_type": "human_user", "external_id": p} for p in principals]
+ rows += [{"node_type": "role", "external_id": r} for r in roles]
+ return build_overlay("t-1", [], identities=rows, now=0.0)
+
+
+def _event(**fields: Any) -> dict[str, Any]:
+ return {"tenant_id": "t-1", "ocsf_event": {"tenant_uid": "t-1", "raw_data": json.dumps(fields)}}
+
+
+RULE = {
+ "id": "det-test-priv",
+ "slug": "test-priv",
+ "name": "Privileged user signed in from an anonymising VPN",
+ "severity": "high",
+ "category": "identity",
+ "mitre": [],
+ "match_when": {"event_type": "auth_success", "user_priv": True},
+}
+
+
+def test_the_rule_fires_only_for_a_privileged_principal() -> None:
+ engine = DetectionEngine([RULE])
+ privileged = _overlay(principals=("[email protected]",))
+
+ hits = engine.evaluate(_event(event_type="auth_success", user="[email protected]"), privileged)
+ assert [h.rule_id for h in hits] == ["det-test-priv"]
+
+ assert not engine.evaluate(_event(event_type="auth_success", user="[email protected]"), privileged)
+
+
+def test_without_an_overlay_the_rule_stays_silent() -> None:
+ """A tenant that has imported no directory gets no keys, not `False`.
+
+ This is the difference between "we do not know" and "this account is
+ ordinary", and the second would be asserted about every account on every
+ deployment that never imported anything.
+ """
+ engine = DetectionEngine([RULE])
+ assert not engine.evaluate(_event(event_type="auth_success", user="[email protected]"))
+ assert not engine.evaluate(_event(event_type="auth_success", user="[email protected]"), _overlay())
+
+
+def test_an_unknown_subject_produces_no_key() -> None:
+ derived = identity_fields(frozenset({"[email protected]"}), frozenset(), {"user": "[email protected]"})
+ assert derived == {"user_priv": False, "actor_role_priv": False, "actor_is_admin": False}
+
+ assert identity_fields(frozenset({"[email protected]"}), frozenset(), {"event_type": "auth_success"}) == {}
+
+
+def test_one_tenants_administrators_are_not_anothers() -> None:
+ engine = DetectionEngine([RULE])
+ event = _event(event_type="auth_success", user="[email protected]")
+ assert engine.evaluate(event, _overlay(principals=("[email protected]",)))
+ assert not engine.evaluate(event, _overlay(principals=("[email protected]",)))
+
+
+def test_the_subject_is_matched_case_insensitively() -> None:
+ """Directories and audit logs disagree about case on the same address."""
+ derived = identity_fields(frozenset({"[email protected]"}), frozenset(), {"user": "[email protected]"})
+ assert derived["user_priv"] is True
+
+
+def test_roles_and_principals_are_separate_sets() -> None:
+ """A role named like a user must not make that user privileged."""
+ derived = identity_fields(frozenset(), frozenset({"global-admin"}), {"user": "global-admin", "role": "global-admin"})
+ assert derived["role_priv"] is True
+ assert "user_priv" not in derived
+
+
+def test_the_reachability_gate_mirrors_this_module() -> None:
+ """The gate names these booleans; a new one must be added in both places.
+
+ The operator list in the same gate had drifted exactly this way — `neq`
+ was added to the matcher and not to the gate, so five rules the engine
+ reads correctly were reported unreachable for four months. Asserting in
+ both directions is the cheap half of not repeating it.
+ """
+ spec = importlib.util.spec_from_file_location("cdf_identity", _REPO / "scripts" / "check_detection_fields.py")
+ assert spec and spec.loader
+ gate = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(gate)
+ assert gate._TENANT_DERIVED_FIELDS == frozenset(IDENTITY_FIELDS), (
+ "scripts/check_detection_fields.py and tenant_overlay.IDENTITY_FIELDS disagree about which privilege booleans the platform computes"
+ )
+
+
+def test_the_gate_and_the_matcher_agree_on_every_operator() -> None:
+ """Both directions. One direction is how `neq` went missing for months."""
+ spec = importlib.util.spec_from_file_location("cdf_ops", _REPO / "scripts" / "check_detection_fields.py")
+ assert spec and spec.loader
+ gate = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(gate)
+
+ from app.services.detection_matcher import OPERATORS # noqa: PLC0415
+
+ matcher_suffixes = {suffix.lstrip("_") for suffix, _name, _sql in OPERATORS}
+ gate_suffixes = set(gate._OPERATOR_SUFFIXES)
+ assert gate_suffixes == matcher_suffixes, (
+ f"only in the gate: {sorted(gate_suffixes - matcher_suffixes)}; only in the matcher: {sorted(matcher_suffixes - gate_suffixes)}"
+ )
diff --git a/services/fusion/tests/test_tenant_tuning_overlay.py b/services/fusion/tests/test_tenant_tuning_overlay.py
index d1a76e3a6..244788983 100644
--- a/services/fusion/tests/test_tenant_tuning_overlay.py
+++ b/services/fusion/tests/test_tenant_tuning_overlay.py
@@ -140,15 +140,29 @@ class TestTheSeverityFloor:
class _Conn:
- def __init__(self, rows, fail=False) -> None: # noqa: ANN001
+ """A connection that answers each query with that query's own rows.
+
+ It used to return the same rows whatever was asked, which was fine while
+ one reload issued one query and became a double more capable than a real
+ connection as soon as a second table was read: the detection-rule rows
+ would have been handed back as identity rows too. Routing on the table
+ name keeps `fetches` meaning "reloads" and keeps the double honest.
+ """
+
+ def __init__(self, rows, fail=False, identities=None) -> None: # noqa: ANN001
self.rows = rows
+ self.identities = identities or []
self.fail = fail
self.fetches = 0
+ self.identity_fetches = 0
- async def fetch(self, sql, *args): # noqa: ANN001, ARG002
- self.fetches += 1
+ async def fetch(self, sql, *args): # noqa: ANN001
if self.fail:
raise RuntimeError("database down")
+ if "identity_nodes" in sql:
+ self.identity_fetches += 1
+ return list(self.identities)
+ self.fetches += 1
return [r for r in self.rows if str(r.get("_tenant", args[0])) == str(args[0])]
diff --git a/services/fusion/tests/test_windowed_translation_replay.py b/services/fusion/tests/test_windowed_translation_replay.py
index 7c0fdc0a9..82521cc26 100644
--- a/services/fusion/tests/test_windowed_translation_replay.py
+++ b/services/fusion/tests/test_windowed_translation_replay.py
@@ -236,6 +236,9 @@ async def test_a_translated_rule_reads_a_tenant_allowlist_boolean() -> None:
def derived_allowlist_fields(self, fields: dict[str, Any]) -> dict[str, Any]:
return {clause: False}
+ def derived_identity_fields(self, fields: dict[str, Any]) -> dict[str, Any]:
+ return {}
+
loaded = {r.id: r for r in load_window_rules()}
engine = WindowedDetectionEngine(_FakeRedis(), rules=(loaded[rule["id"]],))
tenant = str(uuid4())
diff --git a/tests/isolation/test_tenant_tuning_live.py b/tests/isolation/test_tenant_tuning_live.py
index ca6219168..385ec19b5 100644
--- a/tests/isolation/test_tenant_tuning_live.py
+++ b/tests/isolation/test_tenant_tuning_live.py
@@ -160,14 +160,21 @@ class TestTheQueryRunsAtAll:
from app.services.tenant_overlay import OverlayCache
await _write_tuning(pool, tenant, status="disabled")
- rows = await OverlayCache(pool, reload_seconds=0)._fetch(tenant)
+ fetched = await OverlayCache(pool, reload_seconds=0)._fetch(tenant)
- assert rows is not None, (
- "_fetch returned None, which means the query raised and the fail-soft path "
+ assert fetched is not None, (
+ "_fetch returned None, which means a query raised and the fail-soft path "
"swallowed it. A tenant with tuning configured would see none applied, and "
"nothing would say so."
)
+ # Two result sets since depth plan 3.3: the tuning rows, and this
+ # tenant's privileged identities from `identity_nodes`. Both run
+ # against the real schema here, which is the point — the second
+ # query is as able to name a column that does not exist as the
+ # first was, and that defect looked exactly like "no tuning".
+ rows, identities = fetched
assert len(rows) == 1, f"expected the one tuned rule, got {rows!r}"
+ assert identities == [], f"this tenant imported no directory, so the privilege query must return nothing, got {identities!r}"
class TestSuppression: