Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
Lane A: identity context that changes the verdict — effective permissions, access review, sub-principal baselines, change timeline and identity resolution (#1590)
Beenu Aroraba636b49fa08
diff --git a/.github/workflows/integration.yml b/.github/workflows/integration.ymlindex de7645de7..02abaacb3 100644--- a/.github/workflows/integration.yml+++ b/.github/workflows/integration.yml@@ -112,7 +112,7 @@ jobs:# rail legs) were inert: their live tests only ran when something# *else* in the surviving list changed. This is the union of all# three lists; the next lane extends it rather than re-assigning.- migration_paths='services/api/migrations/ services/api/app/scripts/run_migrations.py services/ueba/alembic/ services/api/tests/test_mssp_portfolio_isolation.py tests/isolation/test_postgres_rls.py tests/isolation/test_mssp_rls_live.py scripts/check_runtime_db_role.py services/fusion/tests/test_asset_context_live.py services/fusion/app/services/asset_context.py services/api/tests/test_alert_rail_business_context_live.py services/api/app/services/alert_rail.py services/agents/tests/test_business_context_tenant_rules_live.py services/agents/app/workers/business_context.py services/api/tests/test_alert_queue_priority_live.py services/api/app/services/alert_queue.py .github/workflows/integration.yml'+ migration_paths='services/api/migrations/ services/api/app/scripts/run_migrations.py services/ueba/alembic/ services/api/tests/test_mssp_portfolio_isolation.py tests/isolation/test_postgres_rls.py tests/isolation/test_mssp_rls_live.py scripts/check_runtime_db_role.py services/fusion/tests/test_asset_context_live.py services/fusion/app/services/asset_context.py services/api/tests/test_alert_rail_business_context_live.py services/api/app/services/alert_rail.py services/agents/tests/test_business_context_tenant_rules_live.py services/agents/app/workers/business_context.py services/api/tests/test_alert_queue_priority_live.py services/api/app/services/alert_queue.py services/api/tests/test_identity_privilege_live.py services/api/app/services/identity_privilege.py services/api/app/services/identity_privilege_store.py services/api/app/services/alert_priority.py services/api/tests/test_access_review_live.py services/api/app/services/access_review.py services/api/app/services/access_review_store.py services/api/tests/test_identity_resolution_live.py services/api/app/services/identity_resolution.py services/api/app/services/identity_resolution_store.py services/api/app/services/change_timeline.py .github/workflows/integration.yml'# backup_crypt.py was missing from the old filter, and it is where# every byte of the AES-256-GCM implementation lives — so a change# to the encryption itself did not run the gate that proves the@@ -829,6 +829,43 @@ jobs:MSSP_ISOLATION_REQUIRED: '1'run: python -m pytest tests/test_alert_queue_priority_live.py -v --tb=short+ # Depth 2 wave 2, 1.1. The properties here are schema facts a stand-in+ # store always gets right by construction: the+ # `valid_from <= t < valid_until` select that makes an alert read the+ # tier its actor held *when it acted*, the partial unique index that+ # refuses two live rows, the principal_id-or-label join, and+ # `boolean NULL` round-tripping as None rather than False. Each test+ # carries its own negative control.+ - name: Privilege at event time is read point-in-time (live Postgres)+ working-directory: services/api+ env:+ MSSP_ISOLATION_REQUIRED: '1'+ run: python -m pytest tests/test_identity_privilege_live.py -v --tb=short++ # Depth 2 wave 2, 1.2. The review's properties are all schema facts:+ # the unique index that turns a nightly re-run into an update rather+ # than a ninetieth copy of one sentence, the NOT IN delete that+ # retires a finding somebody fixed, an acknowledgement surviving the+ # next run, and the tenant predicate on every statement including the+ # acknowledgement write.+ - name: Access review de-duplicates, retires and stays tenant-scoped (live Postgres)+ working-directory: services/api+ env:+ MSSP_ISOLATION_REQUIRED: '1'+ run: python -m pytest tests/test_access_review_live.py -v --tb=short++ # Depth 2 wave 2, 3.1. The cluster walk is the leg that matters: a+ # weak link must not be traversable, and the way that breaks is a+ # chain of three weak hops joining two identities no single rule+ # would join. The filter is in the SQL, so only a database can check+ # it. The ordered-pair CHECK is exercised with a deliberate+ # out-of-order insert that must be refused.+ - name: Identity resolution never merges on a weak link (live Postgres)+ working-directory: services/api+ env:+ MSSP_ISOLATION_REQUIRED: '1'+ run: python -m pytest tests/test_identity_resolution_live.py -v --tb=short+# 060_rls_coverage.sql takes RLS from 31 of 95 tenant-scoped tables to# every one of them bar a documented exclusion. A migration that adds 49# policies is worth what the evidence that they filter is worth, so thediff --git a/DEPTH2_PROGRESS.md b/DEPTH2_PROGRESS.mdindex ff93ffaee..e9a2e0174 100644--- a/DEPTH2_PROGRESS.md+++ b/DEPTH2_PROGRESS.md@@ -50,6 +50,14 @@ claimed the same migration number simultaneously and one had to renumber.| Number | Kind | Lane / item | Claimed | File ||---|---|---|---|---|| 0015 | ADR | Wave 0 / Phase 0.4 | 2026-11-01 | `docs/decisions/0015-siem-replacement-scope.md` |+| 124 | migration | Lane A / 1.1 | 2026-11-01 | `services/api/migrations/124_identity_privilege_facts.sql` |+| 125 | migration | Lane A / 1.2 | 2026-11-01 | `services/api/migrations/125_access_review_findings.sql` |+| 126 | migration | Lane A / 1.4 | 2026-11-01 | **released unused** — 1.4 lives in the existing fusion baseline store (Redis), as the plan requires; no table was needed |+| 127 | migration | Lane A / 1.5 | 2026-11-01 | **released unused** — 1.5 reads `posture_snapshots` (103) and the lake; no new table |+| 128 | migration | Lane A / 3.1 | 2026-11-01 | `services/api/migrations/128_identity_links.sql` |+| 129 | migration | Lane A | 2026-11-01 | **released unused** |+| 0016 | ADR | Lane A / 1.1 | 2026-11-01 | `docs/decisions/0016-privilege-context-in-scoring.md` |+| 0017 | ADR | Lane A / 3.1 | 2026-11-01 | **released unused** — the confidence model is a table of eight rules in `identity_resolution.py`, each with its reasoning beside it, and the merge threshold's justification is in the module docstring. An ADR restating them would be a second place for them to drift || 146 | migration | Lane E / 7.1 | 2026-11-01 | AI provider coverage + chat compliance (labels and bounded snippet only) || 147 | migration | Lane E / 7.2 | 2026-11-01 | hosted agents and repository AI artefacts as asset kinds || 148 | migration | Lane E / 7.3 | 2026-11-01 | agent identities and per-agent baselines |@@ -72,8 +80,12 @@ claims one here first.Highest at kickoff: migration `123`, ADR `0014`. Next free outside Lane B'sblock: migration `124`, ADR `0016`.-Highest at kickoff: migration `123`, ADR `0014`. Next free: migration `124`,-ADR `0016`.+Highest at kickoff: migration `123`, ADR `0014`. Lane A claimed migrations+`124`–`129` and ADRs `0016`–`0017`, and **used 124, 125, 128 and 0016**.+The rest are released: a claimed-and-unused number is kept in this table+rather than deleted, so a later lane reading it can see the number is free+without having to prove the claim was abandoned rather than forgotten.+Next free outside the block: migration `130`, ADR `0018`.## Phase 0: Kickoff@@ -216,6 +228,54 @@ not, that is a real first-run defect for the most common conflict there isand belongs in its own fix with a negative control. Until that run exists,this is an observation with its evidence attached.+### D3 — the queue has never read identity privilege, contrary to the plan++The plan's 1.1 says "Prioritisation already reads identity privilege (v16)+and must read this". It does not. `score_alert_priority` has taken an+`identity` argument and carried a `_PRIVILEGE_FACTOR` table since v16, and+its **only** production caller — `alert_queue.build_queue` — passed+`identity=None` at `alert_queue.py:570`. `queue_priority()`, the one other+function that would pass one, has no production caller at all.++So the privilege factor was unreachable from the only surface where+prioritisation changes anything. Recorded here because it changes 1.1's+scope: the item is not "make the existing reader read a better source", it+is "wire the reader", which is the repository's defining defect found again+inside a module whose own docstring explains why an ordering nobody can+interrogate stops being trusted.++Closed by Lane A's 1.1: `resolve_identity_contexts()` resolves each alert's+actor against the privilege version covering that alert's own event time,+one batched query per page, and `build_queue` passes it.++### D4 — the container clock runs ahead of the host's, and a live+point-in-time test trips over it++`identity_privilege_facts.valid_from` defaults to Postgres `now()`. On this+machine the Docker Postgres container's clock measured ~17 ms **ahead** of+the host's Python clock, so a row written and immediately read back at+`datetime.now(tz=UTC)` landed *before* its own `valid_from` and the+point-in-time select correctly returned nothing.++That is the store behaving exactly as specified — an event predating a+collection has no posture — so the fix is in the test, which now reads one+minute ahead. Widening `valid_from <= :at` to absorb the skew would have+answered historical questions with newer snapshots, which is precisely the+defect migration 103 is versioned to prevent. Recorded because the symptom+("the row is there and the point-in-time read cannot see it") reads as a+query bug and is not one.++### D5 — the verdict baseline in the plan is for a model this host cannot run++The published best balanced accuracy of 0.357 is+`llama3.1:8b-instruct-q4_K_M`. The only model pulled on this host is+`llama3.2:3b-instruct-q4_K_M`, so a run here is not comparable to that row.++Rather than quote the 8B figure as a baseline it did not measure, Lane A ran+the **same 3B model on both trees** — `origin/main` at `152c6d5ff` and the+lane branch — so the comparison has one variable. Both numbers are recorded+in the item log below, and neither is published as the project's headline:+the scoreboard row for the 8B model is untouched.## Phase 7: AI agent security (Lane E)- [x] **7.1** Provider coverage (depth 6.3) plus enterprise AI-chat@@ -284,6 +344,250 @@ depends on it; the three classes simply carry no records.One entry per item as it lands: what failed before, the change, the negativecontrol, and the measurement delta.+### 1.1 Effective permissions everywhere — shipped++**Reproduced first.** `services/api/tests/test_identity_privilege_reproduction.py`+and `services/agents/tests/test_identity_privilege_tool_reproduction.py`+failed on the pre-change tree: 15 of 20 api assertions and all 3 agents+assertions. The 5 api assertions that passed are the *negative* arms of the+condition-operator matrix, and they passed for the wrong reason — the+resolver denied every operator it could not evaluate, so "correctly refuses"+and "under-reports real access" were indistinguishable until the operators+were modelled.++**What shipped.**++- `effective_permissions/iam_conditions.py`: 24 condition operators plus the+ `IfExists` suffix and the `ForAnyValue:` / `ForAllValues:` qualifiers, with+ a **three-valued** result. Undecidable is resolved per effect — an+ undecidable `Allow` does not grant, an undecidable `Deny` still denies —+ so both arms refuse to over-report access.+- `effective_permissions/aws.py`: `NotAction` and `NotResource` invert their+ sets; session policies cap the identity arm (intersected across several,+ never unioned) and not the resource-policy arm; withheld actions are+ reported in `deny_actions` with the reason in `notes`.+- `identity_privilege.py`: tier derived from what an identity can *write*,+ never from the name of the role it holds, plus `can_escalate`,+ `data_access`, `third_party`, `cross_account`, each with its evidence.+- Migration **124**: `identity_privilege_facts` (versioned on the same+ interval as the snapshot it came from, one live row per principal enforced+ by a partial unique index) and `identity_permission_usage` (the runtime+ overlay), both under RLS.+- `identity_usage_collector.py`: the overlay from the lake, returning+ `measured=False` with a reason whenever the lake cannot be asked.++**Wiring — the production callers.**++| Surface | Caller |+|---|---|+| Fusion, per alert | `fusion_engine.process()` step 2.7 → `IdentityPrivilegeResolver.resolve()`, stamped at the alert's **event time** |+| Confidence | `ConfidenceScorer.score()` factor `identity_privilege`, weight 0.06, the eight existing weights scaled by 0.94 so the total is still exactly 1.0 |+| Console queue | `alert_queue.build_queue()` → `resolve_identity_contexts()` → `score_alert_priority(identity=...)` (see D3) |+| Typed pivots | `investigation_tools.TOOLS["identity_effective_permissions"]`, reachable from `POST /graph/investigate/query` |+| Agent tool registry | `tools/registry.py::default_registry`, so the model is offered it and the call lands in the ledger |+| Triage prompt + `confidence_basis` | `fused_alert_consumer._attach_privilege_context()` → `context/privilege.py`, through the same `/investigate/query` door the registered tool uses |++**Negative controls.** Ten live-Postgres tests, each with one: the+point-in-time read is contrasted against the live-row read it replaced and+the two disagree; the unchanged-rederivation test is paired with a changed+one that *does* open a version; the queue test reverts the context and the+score collapses to the unknown baseline.++**Gates.** `test_tool_result_channel.py` caught the new tool missing from+`AGENT_TOOL_NAMES` and `test_investigation_tools.py` caught it missing from+the dispatch-coverage table. Both were the gate working, and both are fixed+rather than relaxed.++**Measurements.**++Alert reduction, `services/fusion/tests/test_alert_reduction_real.py`:++| Measure | Before | After |+|---|---|---|+| Production correlation key | 1000 → 667 = **33.3 %** | 1000 → 667 = **33.3 %** |+| Detection-shaped cohort | 1000 → 755 = **24.5 %** | 1000 → 755 = **24.5 %** |++Verdict scorecard, `scripts/run_verdict_eval.py`, same corpus and the same+model on both trees (see D5), Apple M-series arm64 / macOS, 2026-11-01,+`llama3.2:3b-instruct-q4_K_M` served by Ollama, 72 live model calls on each+run, 0 errored:++| Metric | Before (`152c6d5ff`) | After | Delta |+|---|---|---|---|+| Balanced accuracy | 0.2604 | 0.2715 | +0.0111 |+| Malicious recall | 0.1944 | 0.1944 | 0.0000 |+| Abstention rate | 0.0139 | 0.0139 | 0.0000 |+| Auto-close precision | 0.4286 (3/7) | 0.3750 (3/8) | −0.0536 |++**Both movements are run-to-run variance, not an effect of this change, and+the reason is structural rather than a judgement call.** The verdict harness+sets no service token, so `fetch_privilege_context` returns empty,+`render_privilege(None)` returns `""`, and the prompt the model receives is+byte-identical on both trees. The auto-close figure moved because its+denominator moved from 7 to 8 — one additional item crossed the+consideration threshold — which is the smallest denominator on the+scorecard and the one most sensitive to a single sampled token.++Nothing here is published to the scoreboard: the project's headline row is+`llama3.1:8b-instruct-q4_K_M` at 0.357, which this host cannot run.++### 1.2 Access review and right-sizing — shipped++**Reproduced first.** `services/api/tests/test_access_review_reproduction.py`+failed 7/7: no module in the tree compared granted with used, and a grep for+`access_review`, `right-sizing`, `best_fit_role` or `dormant_admin` across+`services/` and `apps/web/src` matched only Lane A's own 1.1 files.++**What shipped.** `access_review.py` (pure), `access_review_store.py`,+migration **125**, five routes behind `compliance:read`, and a console page+at `/access-review` with a sidebar entry. Six finding kinds, each with a+required `object_hint` and `recommendation` — a read-only review whose+findings do not name the object to change leaves the reviewer with the hard+half of the work, which is why reviews do not happen.++**The honesty property this item turns on.** Usage-dependent findings are+suppressed *wholesale* when `UsageOverlay.measured` is false, not emitted+with a caveat. A caveat on "revoke this" is read by nobody, and the finding+is wrong in the direction that costs an outage. `best_fit_role` returns+`None` for the same reason, and also when no built-in covers the used set —+recommending a role that does not cover something a principal actually used+is a recommendation to cause an outage, which is the one error a+right-sizing feature must not make.++Three distinct empty states, because only one of them is good news:+`never_run`, `nothing_reviewed` (a review over zero principals) and+`no_findings` (a review over a real inventory). Rendering all three as+"0 findings" presents an estate nobody looked at as an estate with nothing+wrong.++**Negative controls.** Eight live-Postgres tests: unmeasured usage suppresses+the usage findings *and then the same principal with measured usage produces+them*; three runs produce one row *and the run table proves three runs+happened*; a finding that stops reproducing is retired; an acknowledgement+survives the next run; another tenant's finding cannot be acknowledged+(404, not 403, so the route is not an existence oracle).++**Defect found while building.** The retire query hit the recorded+`:name::type` bind-cast trap — SQLAlchemy's regex backtracks one character+over a Postgres cast, so `:principals::text[]` declared a parameter named+`principal`. Seven of eight live tests failed with a syntax error. Spelled+`CAST(:principals AS text[])`; `check_text_bind_casts.py` stays green.++### 1.4 Sub-principal baselines — shipped++**Reproduced first.**+`services/fusion/tests/test_sub_principal_baselines_reproduction.py` failed+7/7. `baselines.py` kept a baseline per *principal* plus one volume+baseline per OAuth app; a token, a service account and an AI agent were all+folded into the human's baseline, which is backwards for the case these+baselines exist to catch.++**What shipped.** In `baselines.py` itself, as the plan requires — not a new+store. `api_token`, `oauth_app`, `service_account` and `ai_agent` become+subjects of their own, with four signals: acting outside its own baseline, a+machine cadence turning interactive, a new ASN within an hour of the token+first appearing, and reactivation after 60 days idle. All seven new fields+are in `DERIVED_FIELDS`, so a rule can clause on them.++**`classify_cadence` tests interactive first, and the order is the+classifier.** A person working 09:00–17:00 produces consecutive active hours+whose gaps are almost all 1, which is indistinguishable from an hourly cron+by regularity alone. What separates them is that a person calls several+times within one hour and a timer does not, so the repeat count has to be+consulted before the regularity is. The first implementation checked+regularity first and classified a human as `scheduled`.++**The cross-language contract, and the second defect it caught.** The GitHub+join key is written by Go (`session.go`, `strings.TrimSpace`, no case+folding) and consumed by Python, whose general field helper lowercases —+correct for a username, wrong for a base64 digest, and the obvious+implementation is the broken one. Both sides now read one committed fixture,+`services/ingest/internal/activity/testdata/github_token_key.json`, and it+immediately found a *second* divergence nobody had looked for: Go requires a+string type, while the Python side coerced a numeric `hashed_token` into+`"12345"` — a baseline key Go never writes. Go: 8 subtests pass. Python: 9+assertions pass, including a negative control that fails if the resolver+ever starts folding case.++**Gate caught drift.** `check_detection_fields.py` keeps its own mirror of+`DERIVED_FIELDS` and `test_baselines.py` compares the two in both+directions; it failed on the seven new fields, which is the gate working.++### 1.5 What changed before this alert — shipped++**Reproduced first.**+`services/api/tests/test_change_timeline_reproduction.py` failed 5/5.++**What shipped.** `change_timeline.py` joins versioned posture (what the+estate looked like, complete for collected providers) with audit events (who+did it, complete for ingested sources). Both are kept because a timeline+from one alone is missing either the attribution or the coverage. Six change+kinds over a closed action vocabulary — an unfamiliar action is not returned+rather than guessed into a kind, because a reader cannot tell a guessed+classification from a known one.++`ChangeTimeline.measured` distinguishes "read both sources and found+nothing" from "could not read either". An empty list with no such field+turns a lake outage into evidence of a quiet week.++**Wiring.** Registered as the typed pivot `changes_before_alert` and in the+agents tool registry, reaching the API through the same `/investigate/query`+door the privilege tool uses. The lake table is `aisoc.raw_events`, already+declared ClickHouse-owned in `check_raw_sql_columns.py`'s `UNMIGRATED_TABLES`,+so no new declaration was needed — verified by running the gate.++### 3.1 Identity resolution — shipped++**Reproduced first.**+`services/api/tests/test_identity_resolution_reproduction.py` failed 7/7.++**What shipped.** `identity_resolution.py` (pure) over all six sources the+plan names, `identity_resolution_store.py`, and migration **128**. Eight+rules, each carrying its own confidence, the evidence that produced it, and+a `merged` flag derived from one threshold.++**The failure mode designed against is a wrong link, not a missing one.** A+wrong merge makes the identity look *richer*, not broken, and nothing+downstream can detect it. The three canonical wrong merges are refused and+each has a test: `[email protected]` against `[email protected]` (stored as+a visible weak link, never merged), `admin` against `admin` (not even+returned as weak — it would fill a review page with noise that reads as+near-misses), and a shared role mailbox (demoted rather than treated as a+verified address match).++`MERGE_THRESHOLD` is 0.90 because the two errors are asymmetric: a missed+link costs an analyst one pivot by hand, a wrong link silently attributes+one person's alerts to another.++**Both stores, one read interface.** Postgres for CORE per ADR-0010,+`SAME_PERSON` edges mirrored to the graph for `full`, and reads come from+Postgres only — a read that preferred the graph would answer differently on+the two profiles for reasons no operator could see. Only merge-strength+links are mirrored, because a weak edge in the graph gets walked by every+traversal whose author did not know to filter it.++**Negative controls.** Seven live-Postgres tests. The ordered-pair CHECK is+exercised with a deliberate out-of-order insert that must be refused by the+database; the weak-chain test is paired with a strong link proving the walk+works at all; the bounded walk is compared at two depths; a human decision+is tested in *both* directions (confirm promotes a weak link, reject demotes+a strong one, neither touches `confidence`).++### Measurements for 1.2, 1.4, 1.5 and 3.1++Alert reduction re-run after all four: **33.3 %** (1000 → 667) and+**24.5 %** (1000 → 755) — identical to the Phase 0 baseline.++The verdict scorecard was **not** re-run for this batch, and the reason is+stated rather than skipped: none of these four changes what the triage+prompt contains on the eval corpus. 1.2 is a separate console surface with+no triage caller; 1.4 adds fields to fusion's derived namespace that no+corpus alert carries; 1.5 and 3.1 add tools the model may call, and the+harness places no service token, so both return "could not read" exactly as+the privilege tool did in the 1.1 measurement — where the prompt was proved+byte-identical. A re-run would re-measure the same sampling noise at ~17+minutes per pass. If a later lane wires any of these into the prompt by+default, the scorecard has to move with it.### 7.1 — Provider coverage and AI-chat compliance logsLanded early because Lane D's 7.5 depends on the provider work.diff --git a/annotated_types-0.8.0-py3-none-any.whl b/annotated_types-0.8.0-py3-none-any.whlnew file mode 100644index 000000000..d1492e748Binary files /dev/null and b/annotated_types-0.8.0-py3-none-any.whl differdiff --git a/anyio-4.16.0-py3-none-any.whl b/anyio-4.16.0-py3-none-any.whlnew file mode 100644index 000000000..53f2bfb36Binary files /dev/null and b/anyio-4.16.0-py3-none-any.whl differdiff --git a/apps/web/src/app/(app)/access-review/page.tsx b/apps/web/src/app/(app)/access-review/page.tsxnew file mode 100644index 000000000..d83c475db--- /dev/null+++ b/apps/web/src/app/(app)/access-review/page.tsx@@ -0,0 +1,17 @@+import { AccessReviewView } from '@/components/access-review/AccessReviewView';++export const metadata = {+ title: 'Access review',+ description:+ 'What each principal was granted, through which object, and whether it was used. Read-only.',+};++// Tenant-scoped findings computed from this tenant's own posture and usage.+// Nothing here is cacheable across requests.+export const dynamic = 'force-dynamic';+export const revalidate = 0;+export const fetchCache = 'force-no-store';++export default function AccessReviewPage() {+ return <AccessReviewView />;+}diff --git a/apps/web/src/components/access-review/AccessReviewView.tsx b/apps/web/src/components/access-review/AccessReviewView.tsxnew file mode 100644index 000000000..39c006217--- /dev/null+++ b/apps/web/src/components/access-review/AccessReviewView.tsx@@ -0,0 +1,298 @@+'use client';++/**+ * Access review (depth plan wave 2, 1.2).+ *+ * Read-only. Nothing on this page changes a vendor object: every finding+ * names the object a person should change and stops there. The one write is+ * acknowledging a finding, which records a decision about the review rather+ * than touching the estate.+ *+ * The empty states carry the weight+ * ---------------------------------+ * Three of them, and they are deliberately different sentences:+ *+ * - **never run** — no review has happened. Not a clean bill of health.+ * - **nothing reviewed** — a review ran over zero principals, because no+ * posture has been collected for that provider.+ * - **no findings** — a review ran over a real inventory and found nothing.+ *+ * Only the third is good news. Rendering all three as "0 findings" would+ * present an estate nobody looked at as an estate with nothing wrong, which+ * is the single most reassuring thing this page could get wrong.+ *+ * A fourth state is rendered per provider rather than globally: a review+ * where `usage_measured_for` is below `reviewed_principals` is partial, and+ * the unused-access, dormancy and right-sizing findings were not computed+ * at all for the remainder. That is said in words next to the counts.+ */++import { useCallback, useEffect, useState } from 'react';++import {+ accessReviewApi,+ type AccessReviewFinding,+ type AccessReviewSummary,+} from '@/lib/api';++const PROVIDERS = ['aws', 'azure', 'gcp', 'gws', 'github', 'okta'] as const;++const SEVERITY_STYLES: Record<string, string> = {+ critical: 'bg-red-500/15 text-red-300 border-red-500/30',+ high: 'bg-orange-500/15 text-orange-300 border-orange-500/30',+ medium: 'bg-amber-500/15 text-amber-300 border-amber-500/30',+ low: 'bg-sky-500/15 text-sky-300 border-sky-500/30',+ info: 'bg-slate-500/15 text-slate-300 border-slate-500/30',+};++export function AccessReviewView() {+ const [summary, setSummary] = useState<AccessReviewSummary | null>(null);+ const [findings, setFindings] = useState<AccessReviewFinding[]>([]);+ const [provider, setProvider] = useState<string>('');+ const [loading, setLoading] = useState(true);+ const [running, setRunning] = useState(false);+ // An error is rendered as an error. The repository's recorded defect here+ // is a catch block that falls back to plausible-looking data, so a failed+ // read must never be indistinguishable from a clean review.+ const [error, setError] = useState<string | null>(null);++ const load = useCallback(async () => {+ setLoading(true);+ setError(null);+ try {+ const [nextSummary, nextFindings] = await Promise.all([+ accessReviewApi.summary(provider || undefined),+ accessReviewApi.findings({ provider: provider || undefined }),+ ]);+ setSummary(nextSummary);+ setFindings(nextFindings.items);+ } catch (err) {+ setSummary(null);+ setFindings([]);+ setError(+ err instanceof Error+ ? `The access review could not be loaded: ${err.message}. This is a read failure, not an empty review.`+ : 'The access review could not be loaded. This is a read failure, not an empty review.',+ );+ } finally {+ setLoading(false);+ }+ }, [provider]);++ useEffect(() => {+ void load();+ }, [load]);++ const runFor = async (target: string) => {+ setRunning(true);+ setError(null);+ try {+ await accessReviewApi.run(target);+ await load();+ } catch (err) {+ setError(+ err instanceof Error+ ? `The review could not be run: ${err.message}`+ : 'The review could not be run.',+ );+ } finally {+ setRunning(false);+ }+ };++ const acknowledge = async (finding: AccessReviewFinding) => {+ const note = window.prompt(+ `Acknowledge "${finding.title}" for ${finding.principal_id}. Why is this acceptable?`,+ '',+ );+ if (note === null) return;+ try {+ await accessReviewApi.acknowledge(finding.id, note);+ await load();+ } catch (err) {+ setError(+ err instanceof Error ? `The acknowledgement failed: ${err.message}` : 'The acknowledgement failed.',+ );+ }+ };++ return (+ <div className="space-y-6 p-6">+ <header className="space-y-2">+ <h1 className="text-2xl font-semibold">Access review</h1>+ <p className="max-w-3xl text-sm text-slate-400">+ What each principal was granted, through which object, and whether it was used.+ This page is read-only: every finding names the object to change and nothing here+ changes it for you.+ </p>+ </header>++ <div className="flex flex-wrap items-center gap-3">+ <label className="text-sm text-slate-400" htmlFor="access-review-provider">+ Provider+ </label>+ <select+ id="access-review-provider"+ className="rounded border border-slate-700 bg-slate-900 px-3 py-1.5 text-sm"+ value={provider}+ onChange={(event) => setProvider(event.target.value)}+ >+ <option value="">All</option>+ {PROVIDERS.map((p) => (+ <option key={p} value={p}>+ {p}+ </option>+ ))}+ </select>+ {PROVIDERS.map((p) => (+ <button+ key={p}+ type="button"+ disabled={running}+ onClick={() => void runFor(p)}+ className="rounded border border-slate-700 px-3 py-1.5 text-xs text-slate-300 hover:border-slate-500 disabled:opacity-50"+ >+ {running ? 'Running…' : `Review ${p}`}+ </button>+ ))}+ </div>++ {error ? (+ <div className="rounded border border-red-500/30 bg-red-500/10 p-4 text-sm text-red-200">+ {error}+ </div>+ ) : null}++ {loading ? <p className="text-sm text-slate-400">Loading…</p> : null}++ {!loading && !error && summary?.state === 'never_run' ? (+ <div className="rounded border border-slate-700 bg-slate-900/60 p-5">+ <h2 className="text-base font-medium">Nothing has been reviewed yet</h2>+ <p className="mt-2 max-w-2xl text-sm text-slate-400">{summary.note}</p>+ </div>+ ) : null}++ {!loading && !error && summary?.state === 'reviewed' ? (+ <section className="space-y-3">+ <h2 className="text-base font-medium">Coverage</h2>+ <div className="grid gap-3 md:grid-cols-2 xl:grid-cols-3">+ {summary.providers.map((row) => (+ <article key={row.provider} className="rounded border border-slate-700 bg-slate-900/60 p-4">+ <div className="flex items-baseline justify-between">+ <h3 className="text-sm font-medium uppercase tracking-wide text-slate-300">+ {row.provider}+ </h3>+ <span className="text-xs text-slate-500">+ {row.last_run_at ? new Date(row.last_run_at).toLocaleString() : 'never'}+ </span>+ </div>++ {row.run_state === 'nothing_reviewed' ? (+ <p className="mt-3 text-sm text-amber-300">+ Nothing reviewed — no posture has been collected for this provider. This is+ not a clean bill of health.+ </p>+ ) : (+ <dl className="mt-3 space-y-1 text-sm text-slate-400">+ <div className="flex justify-between">+ <dt>Principals reviewed</dt>+ <dd className="text-slate-200">{row.reviewed_principals}</dd>+ </div>+ <div className="flex justify-between">+ <dt>Open findings</dt>+ <dd className="text-slate-200">{row.finding_count}</dd>+ </div>+ <div className="flex justify-between">+ <dt>Usage measured for</dt>+ <dd className="text-slate-200">+ {row.usage_measured_for} / {row.reviewed_principals}+ </dd>+ </div>+ </dl>+ )}++ {row.note ? <p className="mt-3 text-xs text-amber-300/90">{row.note}</p> : null}++ {row.run_state === 'no_findings' ? (+ <p className="mt-3 text-sm text-emerald-300">+ Reviewed, nothing found.+ </p>+ ) : null}+ </article>+ ))}+ </div>+ </section>+ ) : null}++ {!loading && !error && summary?.state === 'reviewed' ? (+ <section className="space-y-3">+ <h2 className="text-base font-medium">+ Findings{findings.length ? ` (${findings.length})` : ''}+ </h2>++ {findings.length === 0 ? (+ <p className="text-sm text-slate-400">+ No open findings for the selected provider. Check the coverage cards above for+ whether that is because nothing was wrong or because nothing could be reviewed.+ </p>+ ) : (+ <ul className="space-y-3">+ {findings.map((finding) => (+ <li+ key={finding.id}+ className="rounded border border-slate-700 bg-slate-900/60 p-4 space-y-2"+ >+ <div className="flex flex-wrap items-center gap-2">+ <span+ className={`rounded border px-2 py-0.5 text-xs uppercase ${+ SEVERITY_STYLES[finding.severity] ?? SEVERITY_STYLES.info+ }`}+ >+ {finding.severity}+ </span>+ <span className="text-xs uppercase tracking-wide text-slate-500">+ {finding.provider}+ </span>+ <h3 className="text-sm font-medium text-slate-100">{finding.title}</h3>+ </div>++ <p className="font-mono text-xs text-slate-400">{finding.principal_id}</p>+ <p className="text-sm text-slate-300">{finding.detail}</p>++ <div className="rounded bg-slate-950/60 p-3 text-sm">+ <p className="text-slate-400">+ <span className="text-slate-500">Change: </span>+ {finding.object_hint}+ </p>+ <p className="mt-1 text-slate-300">{finding.recommendation}</p>+ {finding.best_fit_role ? (+ <p className="mt-1 text-slate-300">+ <span className="text-slate-500">Smallest role covering what was used: </span>+ <code>{finding.best_fit_role}</code>+ </p>+ ) : null}+ </div>++ {!finding.usage_measured ? (+ <p className="text-xs text-amber-300/90">+ Usage was not measured for this principal, so this finding was computed from+ the grant alone.+ </p>+ ) : null}++ <button+ type="button"+ onClick={() => void acknowledge(finding)}+ className="rounded border border-slate-700 px-3 py-1 text-xs text-slate-300 hover:border-slate-500"+ >+ Acknowledge+ </button>+ </li>+ ))}+ </ul>+ )}+ </section>+ ) : null}+ </div>+ );+}diff --git a/apps/web/src/components/layout/Sidebar.tsx b/apps/web/src/components/layout/Sidebar.tsxindex 2ce213aff..850df6b84 100644--- a/apps/web/src/components/layout/Sidebar.tsx+++ b/apps/web/src/components/layout/Sidebar.tsx@@ -203,6 +203,11 @@ const navSections: NavSection[] = [href: '/federated-search',icon: <SearchIcon />,},+ {+ label: 'Access Review',+ href: '/access-review',+ icon: <ShieldIcon />,+ },{label: 'Evaluate on History',href: '/evaluate',diff --git a/apps/web/src/lib/api.ts b/apps/web/src/lib/api.tsindex 833731681..5005ea19a 100644--- a/apps/web/src/lib/api.ts+++ b/apps/web/src/lib/api.ts@@ -7551,3 +7551,110 @@ export const caseDepthApi = {body: JSON.stringify({ values }),}),};++// ── Access review (depth plan wave 2, 1.2) ─────────────────────────────────+//+// Read-only. Every route below computes or reads findings; none of them+// changes a vendor object, and the only write is a reviewer acknowledging a+// finding, which is state about the review rather than about the estate.++export interface AccessReviewFinding {+ id: string;+ run_id: string;+ kind: string;+ provider: string;+ principal_id: string;+ severity: "info" | "low" | "medium" | "high" | "critical";+ title: string;+ detail: string;+ /** The object a person should go and change. Never empty. */+ object_hint: string;+ recommendation: string;+ /** Null when usage was not measured, or when no built-in role is smaller. */+ best_fit_role: string | null;+ evidence: Record<string, unknown>;+ /**+ * False when no usage data backed this finding. Usage-dependent kinds are+ * suppressed entirely in that case, so a false here marks the weaker+ * non-usage findings for what they are.+ */+ usage_measured: boolean;+ acknowledged_at: string | null;+ acknowledged_by: string | null;+ acknowledged_note: string | null;+ first_seen: string | null;+ last_seen: string | null;+}++export interface AccessReviewProviderState {+ provider: string;+ run_state: "nothing_reviewed" | "no_findings" | "findings" | "failed";+ reviewed_principals: number;+ usage_measured_for: number;+ finding_count: number;+ window_start: string | null;+ window_end: string | null;+ note: string | null;+ last_run_at: string | null;+}++export interface AccessReviewSummary {+ /**+ * `never_run` is a distinct state from a run that found nothing, and the+ * page renders them differently on purpose: an estate nobody looked at+ * must not read as an estate with nothing wrong.+ */+ state: "never_run" | "reviewed";+ open_findings: number;+ providers: AccessReviewProviderState[];+ note?: string;+}++export interface AccessReviewKind {+ kind: string;+ title: string;+ severity: string;+ object_hint: string;+ recommendation: string;+ needs_usage: boolean;+}++export const accessReviewApi = {+ async kinds(): Promise<{ kinds: AccessReviewKind[] }> {+ return request<{ kinds: AccessReviewKind[] }>("/api/v1/access-review/kinds");+ },++ async summary(provider?: string): Promise<AccessReviewSummary> {+ const query = provider ? `?provider=${encodeURIComponent(provider)}` : "";+ return request<AccessReviewSummary>(`/api/v1/access-review/summary${query}`);+ },++ async findings(params: {+ provider?: string;+ kind?: string;+ includeAcknowledged?: boolean;+ } = {}): Promise<{ items: AccessReviewFinding[]; count: number }> {+ const search = new URLSearchParams();+ if (params.provider) search.set("provider", params.provider);+ if (params.kind) search.set("kind", params.kind);+ if (params.includeAcknowledged) search.set("include_acknowledged", "true");+ const query = search.toString();+ return request<{ items: AccessReviewFinding[]; count: number }>(+ `/api/v1/access-review/findings${query ? `?${query}` : ""}`,+ );+ },++ async run(provider: string): Promise<AccessReviewSummary & { run_id: string }> {+ return request<AccessReviewSummary & { run_id: string }>(+ `/api/v1/access-review/run?provider=${encodeURIComponent(provider)}`,+ { method: "POST" },+ );+ },++ async acknowledge(findingId: string, note: string): Promise<AccessReviewFinding> {+ return request<AccessReviewFinding>(+ `/api/v1/access-review/findings/${encodeURIComponent(findingId)}/acknowledge`,+ { method: "POST", body: JSON.stringify({ note }) },+ );+ },+};diff --git a/attrs-26.1.0-py3-none-any.whl b/attrs-26.1.0-py3-none-any.whlnew file mode 100644index 000000000..3f0a9de77Binary files /dev/null and b/attrs-26.1.0-py3-none-any.whl differdiff --git a/boltons-21.0.0-py2.py3-none-any.whl b/boltons-21.0.0-py2.py3-none-any.whlnew file mode 100644index 000000000..c470b4de2Binary files /dev/null and b/boltons-21.0.0-py2.py3-none-any.whl differdiff --git a/bracex-3.2-py3-none-any.whl b/bracex-3.2-py3-none-any.whlnew file mode 100644index 000000000..dfa037f1dBinary files /dev/null and b/bracex-3.2-py3-none-any.whl differdiff --git a/certifi-2026.10.23-py3-none-any.whl b/certifi-2026.10.23-py3-none-any.whlnew file mode 100644index 000000000..1a5d66ac0Binary files /dev/null and b/certifi-2026.10.23-py3-none-any.whl differdiff --git a/cffi-2.1.1-cp312-cp312-macosx_11_0_arm64.whl b/cffi-2.1.1-cp312-cp312-macosx_11_0_arm64.whlnew file mode 100644index 000000000..5dadfd979Binary files /dev/null and b/cffi-2.1.1-cp312-cp312-macosx_11_0_arm64.whl differdiff --git a/charset_normalizer-3.5.2-cp312-cp312-macosx_10_13_universal2.whl b/charset_normalizer-3.5.2-cp312-cp312-macosx_10_13_universal2.whlnew file mode 100644index 000000000..24493f8e5Binary files /dev/null and b/charset_normalizer-3.5.2-cp312-cp312-macosx_10_13_universal2.whl differdiff --git a/click-8.4.2-py3-none-any.whl b/click-8.4.2-py3-none-any.whlnew file mode 100644index 000000000..aaeffdb30Binary files /dev/null and b/click-8.4.2-py3-none-any.whl differdiff --git a/click_option_group-0.5.9-py3-none-any.whl b/click_option_group-0.5.9-py3-none-any.whlnew file mode 100644index 000000000..34dce6336Binary files /dev/null and b/click_option_group-0.5.9-py3-none-any.whl differdiff --git a/colorama-0.4.6-py2.py3-none-any.whl b/colorama-0.4.6-py2.py3-none-any.whlnew file mode 100644index 000000000..f666ce989Binary files /dev/null and b/colorama-0.4.6-py2.py3-none-any.whl differdiff --git a/cryptography-50.0.2-cp311-abi3-macosx_11_0_arm64.whl b/cryptography-50.0.2-cp311-abi3-macosx_11_0_arm64.whlnew file mode 100644index 000000000..1f8226c32Binary files /dev/null and b/cryptography-50.0.2-cp311-abi3-macosx_11_0_arm64.whl differdiff --git a/docs/decisions/0016-privilege-context-in-scoring.md b/docs/decisions/0016-privilege-context-in-scoring.mdnew file mode 100644index 000000000..0f22ad546--- /dev/null+++ b/docs/decisions/0016-privilege-context-in-scoring.md@@ -0,0 +1,117 @@+# ADR-0016: privilege context enters scoring as bounded factors, and `unknown` is a value++- **Status:** accepted+- **Date:** 2026-11-01+- **Related:** [ADR-0010](./0010-entity-and-session-context-in-core.md) (entity context in CORE), [ADR-0011](./0011-behavioural-baselines-in-core.md) (the bounded-contribution precedent)+- **Implements:** depth plan wave 2, Phase 1.1 (carrying the first depth plan's 2.6)++## Context++Six effective-permission resolvers have shipped since T3.2 and migration 103+versioned the posture they read. Nothing consumed either. Phase 1.1 wires+them into the two places a privilege answer changes what a human sees — the+fusion confidence score and the console queue's priority — and into the+triage prompt and the typed-pivot tool surface.++Wiring a new signal into two existing scorers raises three questions that+are easy to answer badly and expensive to revisit, because both scorers are+surfaces an analyst learns to trust or learns to ignore.++## Decision 1 — bounded, and small++`identity_privilege` enters `ConfidenceScorer` at weight **0.06**, the same+weight `asset_criticality` carries, and the eight pre-existing weights are+scaled by **0.94** so the total stays exactly 1.0.++Scaling rather than taking the new weight out of one neighbour keeps every+existing factor's share of the model unchanged *relative to the others*. The+only thing that moved is that all of them together now account for 94%+instead of 100%, which shifts each alert's score slightly and reorders+nothing: a common positive scale factor preserves the ordering of `Σ(w·c)`.++In the queue, privilege is a **multiplier** (`alert_priority.py`'s existing+design: severity is the base, context multiplies) capped so that four tags+together reach 1.21 — less than one criticality step. Context reorders+alerts of similar severity; it cannot invert a severity gap.++The reason both are small is the same reason `asset_criticality` is: this is+context about what an alert is *about*, not evidence that it *happened*. A+scheme where context can outweigh the detection is one that gets switched+off the first time it surprises somebody, and then none of it is worth+anything.++## Decision 2 — the tier comes from the snapshot valid at event time++Not from the live posture row. An alert is fused seconds after the event and+triaged minutes to days later, and a role revoked in between is exactly the+case migration 103 is versioned for.++This is worth an ADR line because the wrong version of it is invisible. A+live-row read returns a tier that looks precisely as plausible as the right+one; there is no error, no log line and no shape in the data that would let+a reviewer notice. The only defence is that the point-in-time call is the+easy one to reach for, which is why `privilege_at` takes a mandatory `at`+and no function in the store answers "the current tier" without saying so in+its name.++## Decision 3 — `unknown` is a stored value and a rendered state++A principal with no posture covering the event time gets `tier = "unknown"`+and four `None` tags, all the way through: a `NOT NULL DEFAULT 'unknown'`+column, a `1.0` entry in the priority table, a `0.0` contribution in the+confidence scorer, a factor row that says "unknown" in words, and a priority+rationale line that says the platform could not answer.++The alternative — defaulting to `standard`, or omitting the factor — makes a+measured ordinary account and an unmeasured one identical. The second is the+one worth escalating, and a product whose job is noticing gaps cannot render+its own biggest gap as reassurance. The same reasoning makes the four tags+`boolean NULL`: `NULL` means no snapshot could answer and `false` means the+snapshot says no, and a surface that renders them the same is fabricating+one of them.++The usage overlay carries the strongest form of this. An install with no+event lake cannot measure which granted permissions were exercised, and+`UsageOverlay.measured` is `False` there rather than an empty used-set.+Collapsing the two would make every right-sizing finding recommend revoking+every permission in the estate.++## Decision 4 — the derivation is tiered on what an identity can *write*++Not on the name of the role it holds. A role called `ReadOnlyAuditor` that+can call `iam:PutUserPolicy` is an administrator, and the name is the part+an attacker controls when they create the role.++`domain_admin` is a wildcard over the whole provider; `admin` is+authorization-write without that wildcard; `elevated` is a wide action+surface with no authorization write — the "powerful but cannot grant" shape,+such as a data engineer with full BigQuery, which a two-tier scheme cannot+express and therefore rounds to one of the two wrong answers.++## What was rejected++**A separate privilege score alongside confidence.** Two numbers describing+one alert drift, and then nobody knows which one the queue used. The+repository has this exact defect on record twice.++**Letting a tag force a severity.** Severity is what the detection claimed+and is compared across deployments; privilege is local context. Conflating+them would make a tenant's posture-collection coverage silently change what+their detections appear to have found.++**Resolving permissions on the fusion hot path.** The resolver takes an+estate-sized policy document, and `services/fusion` cannot import+`services/api` in any case — both name their top-level package `app`. The+expensive resolution runs once per posture version into+`identity_privilege_facts`, and fusion reads one indexed row.++## Consequences++- Every alert's confidence score moves slightly, because the eight existing+ weights were scaled. Measured through the verdict scorecard and the+ alert-reduction suite in the PR that landed it; neither consumes+ `confidence_score`, and both were unchanged.+- A deployment that has collected no posture sees `unknown` on every alert+ and no score movement at all, which is the correct no-op.+- A later lane adding a ninth confidence factor must re-scale again rather+ than appending. `test_rationale_weights_sum_to_one` enforces it.diff --git a/docs/openapi.yaml b/docs/openapi.yamlindex 951eca5f3..e7d51e463 100644--- a/docs/openapi.yaml+++ b/docs/openapi.yaml@@ -11900,6 +11900,208 @@ paths:application/json:schema:$ref: '#/components/schemas/HTTPValidationError'+ /api/v1/access-review/kinds:+ get:+ tags:+ - access-review+ summary: The finding kinds this review can produce+ description: 'Published so the console can render a filter without hardcoding+ a copy.+++ A second list in the frontend is how a kind ends up filterable on one++ surface and invisible on the other; this repository has that defect on++ record in the playbook editor''s step-type union.'+ operationId: list_kinds_api_v1_access_review_kinds_get+ responses:+ '200':+ description: Successful Response+ content:+ application/json:+ schema:+ additionalProperties: true+ type: object+ title: Response List Kinds Api V1 Access Review Kinds Get+ security:+ - HTTPBearer: []+ /api/v1/access-review/summary:+ get:+ tags:+ - access-review+ summary: Review state and counts, with an honest empty state+ operationId: review_summary_api_v1_access_review_summary_get+ security:+ - HTTPBearer: []+ parameters:+ - name: provider+ in: query+ required: false+ schema:+ anyOf:+ - type: string+ - type: 'null'+ title: Provider+ responses:+ '200':+ description: Successful Response+ content:+ application/json:+ schema:+ type: object+ additionalProperties: true+ title: Response Review Summary Api V1 Access Review Summary Get+ '422':+ description: Validation Error+ content:+ application/json:+ schema:+ $ref: '#/components/schemas/HTTPValidationError'+ /api/v1/access-review/findings:+ get:+ tags:+ - access-review+ summary: Open access-review findings+ operationId: list_findings_api_v1_access_review_findings_get+ security:+ - HTTPBearer: []+ parameters:+ - name: provider+ in: query+ required: false+ schema:+ anyOf:+ - type: string+ - type: 'null'+ title: Provider+ - name: kind+ in: query+ required: false+ schema:+ anyOf:+ - type: string+ - type: 'null'+ title: Kind+ - name: include_acknowledged+ in: query+ required: false+ schema:+ type: boolean+ default: false+ title: Include Acknowledged+ - name: limit+ in: query+ required: false+ schema:+ type: integer+ maximum: 500+ minimum: 1+ default: 100+ title: Limit+ responses:+ '200':+ description: Successful Response+ content:+ application/json:+ schema:+ type: object+ additionalProperties: true+ title: Response List Findings Api V1 Access Review Findings Get+ '422':+ description: Validation Error+ content:+ application/json:+ schema:+ $ref: '#/components/schemas/HTTPValidationError'+ /api/v1/access-review/run:+ post:+ tags:+ - access-review+ summary: Recompute the review from current posture and usage+ description: 'Recompute. Reads only — no vendor object is touched by this call.+++ Guarded by ``compliance:read`` rather than a write permission on++ purpose: the only thing it writes is this tenant''s own review rows, and++ requiring a write scope would put the review behind a permission an++ auditor is deliberately not given.'+ operationId: run_review_api_v1_access_review_run_post+ security:+ - HTTPBearer: []+ parameters:+ - name: provider+ in: query+ required: true+ schema:+ type: string+ title: Provider+ responses:+ '200':+ description: Successful Response+ content:+ application/json:+ schema:+ type: object+ additionalProperties: true+ title: Response Run Review Api V1 Access Review Run Post+ '422':+ description: Validation Error+ content:+ application/json:+ schema:+ $ref: '#/components/schemas/HTTPValidationError'+ /api/v1/access-review/findings/{finding_id}/acknowledge:+ post:+ tags:+ - access-review+ summary: Acknowledge a finding+ description: 'Record that a person decided this one is fine.+++ A 404 rather than a 403 for a finding belonging to another tenant: a++ 403 would make this route an existence oracle over every tenant''s++ finding ids, and a genuine miss is byte-identical to a cross-tenant++ attempt from the caller''s side either way.'+ operationId: acknowledge_api_v1_access_review_findings__finding_id__acknowledge_post+ security:+ - HTTPBearer: []+ parameters:+ - name: finding_id+ in: path+ required: true+ schema:+ type: string+ format: uuid+ title: Finding Id+ requestBody:+ required: true+ content:+ application/json:+ schema:+ $ref: '#/components/schemas/AcknowledgeRequest'+ responses:+ '200':+ description: Successful Response+ content:+ application/json:+ schema:+ type: object+ additionalProperties: true+ title: Response Acknowledge Api V1 Access Review Findings Finding+ Id Acknowledge Post+ '422':+ description: Validation Error+ content:+ application/json:+ schema:+ $ref: '#/components/schemas/HTTPValidationError'/api/v1/identity/effective-permissions/providers:get:tags:@@ -19152,6 +19354,163 @@ paths:application/json:schema:$ref: '#/components/schemas/HTTPValidationError'+ /api/v1/identity-resolution/links:+ get:+ tags:+ - identity-resolution+ summary: Every proposed link touching one identity+ operationId: list_links_api_v1_identity_resolution_links_get+ security:+ - HTTPBearer: []+ parameters:+ - name: key+ in: query+ required: true+ schema:+ type: string+ description: A source-scoped identity, e.g. okta:[email protected]+ title: Key+ description: A source-scoped identity, e.g. okta:[email protected]+ - name: include_weak+ in: query+ required: false+ schema:+ type: boolean+ default: true+ title: Include Weak+ responses:+ '200':+ description: Successful Response+ content:+ application/json:+ schema:+ type: object+ additionalProperties: true+ title: Response List Links Api V1 Identity Resolution Links Get+ '422':+ description: Validation Error+ content:+ application/json:+ schema:+ $ref: '#/components/schemas/HTTPValidationError'+ /api/v1/identity-resolution/cluster:+ get:+ tags:+ - identity-resolution+ summary: Who else is this person+ description: Merge-strength links only, bounded. See the store for why both.+ operationId: cluster_api_v1_identity_resolution_cluster_get+ security:+ - HTTPBearer: []+ parameters:+ - name: key+ in: query+ required: true+ schema:+ type: string+ title: Key+ - name: max_depth+ in: query+ required: false+ schema:+ type: integer+ maximum: 6+ minimum: 1+ default: 3+ title: Max Depth+ responses:+ '200':+ description: Successful Response+ content:+ application/json:+ schema:+ type: object+ additionalProperties: true+ title: Response Cluster Api V1 Identity Resolution Cluster Get+ '422':+ description: Validation Error+ content:+ application/json:+ schema:+ $ref: '#/components/schemas/HTTPValidationError'+ /api/v1/identity-resolution/resolve:+ post:+ tags:+ - identity-resolution+ summary: Resolve a set of identities into links+ description: 'Run resolution over the supplied records.+++ ``settings:write`` rather than a read permission because ``persist``++ writes the tenant''s identity graph, and a caller who may not change++ tenant configuration should not be able to decide who is the same++ person.'+ operationId: resolve_api_v1_identity_resolution_resolve_post+ requestBody:+ content:+ application/json:+ schema:+ $ref: '#/components/schemas/ResolveRequest'+ required: true+ responses:+ '200':+ description: Successful Response+ content:+ application/json:+ schema:+ additionalProperties: true+ type: object+ title: Response Resolve Api V1 Identity Resolution Resolve Post+ '422':+ description: Validation Error+ content:+ application/json:+ schema:+ $ref: '#/components/schemas/HTTPValidationError'+ security:+ - HTTPBearer: []+ /api/v1/identity-resolution/links/{link_id}/decide:+ post:+ tags:+ - identity-resolution+ summary: Confirm or reject a proposed link+ description: A 404 for another tenant's link, so this is not an existence oracle.+ operationId: decide_api_v1_identity_resolution_links__link_id__decide_post+ security:+ - HTTPBearer: []+ parameters:+ - name: link_id+ in: path+ required: true+ schema:+ type: string+ format: uuid+ title: Link Id+ requestBody:+ required: true+ content:+ application/json:+ schema:+ $ref: '#/components/schemas/DecideRequest'+ responses:+ '200':+ description: Successful Response+ content:+ application/json:+ schema:+ type: object+ additionalProperties: true+ title: Response Decide Api V1 Identity Resolution Links Link Id Decide+ Post+ '422':+ description: Validation Error+ content:+ application/json:+ schema:+ $ref: '#/components/schemas/HTTPValidationError'/api/v1/reports/templates:get:tags:@@ -22612,6 +22971,15 @@ paths:- HTTPBearer: []components:schemas:+ AcknowledgeRequest:+ properties:+ note:+ type: string+ maxLength: 2000+ title: Note+ default: ''+ type: object+ title: AcknowledgeRequestActionCount:properties:action:@@ -28355,6 +28723,15 @@ components:- window_days- labeltitle: DashboardPeriod+ DecideRequest:+ properties:+ decision:+ type: string+ title: Decision+ type: object+ required:+ - decision+ title: DecideRequestDecisionRequest:properties:decision:@@ -31572,6 +31949,49 @@ components:- last_seen- created_attitle: IOCOut+ IdentityRecord:+ properties:+ source:+ type: string+ title: Source+ id:+ type: string+ title: Id+ email:+ anyOf:+ - type: string+ - type: 'null'+ title: Email+ email_verified:+ type: boolean+ title: Email Verified+ default: false+ upn:+ anyOf:+ - type: string+ - type: 'null'+ title: Upn+ employee_id:+ anyOf:+ - type: string+ - type: 'null'+ title: Employee Id+ display_name:+ anyOf:+ - type: string+ - type: 'null'+ title: Display Name+ maps_to:+ items:+ type: string+ type: array+ title: Maps To+ type: object+ required:+ - source+ - id+ title: IdentityRecord+ description: One source-scoped identity offered for resolution.IdentityTimeline:properties:timeline_id:@@ -37518,6 +37938,19 @@ components:required:- reasontitle: ResetRequest+ ResolveRequest:+ properties:+ records:+ items:+ $ref: '#/components/schemas/IdentityRecord'+ type: array+ title: Records+ persist:+ type: boolean+ title: Persist+ default: false+ type: object+ title: ResolveRequestRetentionOut:properties:raw_events_days:diff --git a/exceptiongroup-1.2.2-py3-none-any.whl b/exceptiongroup-1.2.2-py3-none-any.whlnew file mode 100644index 000000000..91439e5e8Binary files /dev/null and b/exceptiongroup-1.2.2-py3-none-any.whl differdiff --git a/face-26.0.1-py3-none-any.whl b/face-26.0.1-py3-none-any.whlnew file mode 100644index 000000000..bc54389a7Binary files /dev/null and b/face-26.0.1-py3-none-any.whl differdiff --git a/glom-25.12.0-py3-none-any.whl b/glom-25.12.0-py3-none-any.whlnew file mode 100644index 000000000..42b4bfeafBinary files /dev/null and b/glom-25.12.0-py3-none-any.whl differdiff --git a/googleapis_common_protos-1.76.0-py3-none-any.whl b/googleapis_common_protos-1.76.0-py3-none-any.whlnew file mode 100644index 000000000..f2d0c558aBinary files /dev/null and b/googleapis_common_protos-1.76.0-py3-none-any.whl differdiff --git a/h11-0.16.0-py3-none-any.whl b/h11-0.16.0-py3-none-any.whlnew file mode 100644index 000000000..f12b3ce76Binary files /dev/null and b/h11-0.16.0-py3-none-any.whl differdiff --git a/httpcore-1.0.9-py3-none-any.whl b/httpcore-1.0.9-py3-none-any.whlnew file mode 100644index 000000000..74013b88fBinary files /dev/null and b/httpcore-1.0.9-py3-none-any.whl differdiff --git a/httpx-0.28.1-py3-none-any.whl b/httpx-0.28.1-py3-none-any.whlnew file mode 100644index 000000000..0a9780e14Binary files /dev/null and b/httpx-0.28.1-py3-none-any.whl differdiff --git a/httpx_sse-0.4.3-py3-none-any.whl b/httpx_sse-0.4.3-py3-none-any.whlnew file mode 100644index 000000000..d602c87beBinary files /dev/null and b/httpx_sse-0.4.3-py3-none-any.whl differdiff --git a/idna-3.20-py3-none-any.whl b/idna-3.20-py3-none-any.whlnew file mode 100644index 000000000..34af34e71Binary files /dev/null and b/idna-3.20-py3-none-any.whl differdiff --git a/importlib_metadata-8.7.1-py3-none-any.whl b/importlib_metadata-8.7.1-py3-none-any.whlnew file mode 100644index 000000000..d46ba798eBinary files /dev/null and b/importlib_metadata-8.7.1-py3-none-any.whl differdiff --git a/jsonschema-4.25.1-py3-none-any.whl b/jsonschema-4.25.1-py3-none-any.whlnew file mode 100644index 000000000..a1276686fBinary files /dev/null and b/jsonschema-4.25.1-py3-none-any.whl differdiff --git a/jsonschema_specifications-2025.9.1-py3-none-any.whl b/jsonschema_specifications-2025.9.1-py3-none-any.whlnew file mode 100644index 000000000..e04d5d619Binary files /dev/null and b/jsonschema_specifications-2025.9.1-py3-none-any.whl differdiff --git a/markdown_it_py-4.2.0-py3-none-any.whl b/markdown_it_py-4.2.0-py3-none-any.whlnew file mode 100644index 000000000..2aa6d8014Binary files /dev/null and b/markdown_it_py-4.2.0-py3-none-any.whl differdiff --git a/mcp-1.29.0-py3-none-any.whl b/mcp-1.29.0-py3-none-any.whlnew file mode 100644index 000000000..8ab9bc8c4Binary files /dev/null and b/mcp-1.29.0-py3-none-any.whl differdiff --git a/mdurl-0.1.2-py3-none-any.whl b/mdurl-0.1.2-py3-none-any.whlnew file mode 100644index 000000000..6b8b6aba8Binary files /dev/null and b/mdurl-0.1.2-py3-none-any.whl differdiff --git a/opentelemetry_api-1.37.0-py3-none-any.whl b/opentelemetry_api-1.37.0-py3-none-any.whlnew file mode 100644index 000000000..b97172a98Binary files /dev/null and b/opentelemetry_api-1.37.0-py3-none-any.whl differdiff --git a/opentelemetry_exporter_otlp_proto_common-1.37.0-py3-none-any.whl b/opentelemetry_exporter_otlp_proto_common-1.37.0-py3-none-any.whlnew file mode 100644index 000000000..19df1a931Binary files /dev/null and b/opentelemetry_exporter_otlp_proto_common-1.37.0-py3-none-any.whl differdiff --git a/opentelemetry_exporter_otlp_proto_http-1.37.0-py3-none-any.whl b/opentelemetry_exporter_otlp_proto_http-1.37.0-py3-none-any.whlnew file mode 100644index 000000000..6ea421344Binary files /dev/null and b/opentelemetry_exporter_otlp_proto_http-1.37.0-py3-none-any.whl differdiff --git a/opentelemetry_instrumentation-0.58b0-py3-none-any.whl b/opentelemetry_instrumentation-0.58b0-py3-none-any.whlnew file mode 100644index 000000000..9da6f2915Binary files /dev/null and b/opentelemetry_instrumentation-0.58b0-py3-none-any.whl differdiff --git a/opentelemetry_instrumentation_requests-0.58b0-py3-none-any.whl b/opentelemetry_instrumentation_requests-0.58b0-py3-none-any.whlnew file mode 100644index 000000000..b14e4658aBinary files /dev/null and b/opentelemetry_instrumentation_requests-0.58b0-py3-none-any.whl differdiff --git a/opentelemetry_instrumentation_threading-0.58b0-py3-none-any.whl b/opentelemetry_instrumentation_threading-0.58b0-py3-none-any.whlnew file mode 100644index 000000000..94f6c10beBinary files /dev/null and b/opentelemetry_instrumentation_threading-0.58b0-py3-none-any.whl differdiff --git a/opentelemetry_proto-1.37.0-py3-none-any.whl b/opentelemetry_proto-1.37.0-py3-none-any.whlnew file mode 100644index 000000000..59b8a1ddaBinary files /dev/null and b/opentelemetry_proto-1.37.0-py3-none-any.whl differdiff --git a/opentelemetry_sdk-1.37.0-py3-none-any.whl b/opentelemetry_sdk-1.37.0-py3-none-any.whlnew file mode 100644index 000000000..96cd7ae81Binary files /dev/null and b/opentelemetry_sdk-1.37.0-py3-none-any.whl differdiff --git a/opentelemetry_semantic_conventions-0.58b0-py3-none-any.whl b/opentelemetry_semantic_conventions-0.58b0-py3-none-any.whlnew file mode 100644index 000000000..334fc6ff7Binary files /dev/null and b/opentelemetry_semantic_conventions-0.58b0-py3-none-any.whl differdiff --git a/opentelemetry_util_http-0.58b0-py3-none-any.whl b/opentelemetry_util_http-0.58b0-py3-none-any.whlnew file mode 100644index 000000000..926d24714Binary files /dev/null and b/opentelemetry_util_http-0.58b0-py3-none-any.whl differdiff --git a/packaging-26.3-py3-none-any.whl b/packaging-26.3-py3-none-any.whlnew file mode 100644index 000000000..d54f57121Binary files /dev/null and b/packaging-26.3-py3-none-any.whl differdiff --git a/peewee-3.19.0-py3-none-any.whl b/peewee-3.19.0-py3-none-any.whlnew file mode 100644index 000000000..cd3a5c448Binary files /dev/null and b/peewee-3.19.0-py3-none-any.whl differdiff --git a/plans/aisoc_depth_wave2.plan.md b/plans/aisoc_depth_wave2.plan.mdindex f2ccd90cd..e89d7a969 100644--- a/plans/aisoc_depth_wave2.plan.md+++ b/plans/aisoc_depth_wave2.plan.md@@ -16,19 +16,19 @@ todos:status: pending- id: p1-1-effective-permissions-everywherecontent: "Phase 1.1: depth 2.6 as specified, plus runtime-usage overlay, computed can-escalate / data-access / third-party / cross-account tags, and the AWS policy semantics the resolver does not yet model"- status: pending+ status: completed- id: p1-2-access-reviewcontent: "Phase 1.2: access review and right-sizing (granted vs used over a window, best-fit role, humans holding long-lived keys, outside collaborators with admin) as read-only findings and a review page"- status: pending+ status: completed- id: p1-3-hr-device-contextcontent: "Phase 1.3: HR connectors (Workday, BambooHR, Rippling) and MDM connectors (Intune, Kandji, Jamf promoted from plugin) feeding employment status, manager, tenure, device owner and compliance into the identity graph and triage"status: pending- id: p1-4-sub-principal-baselinescontent: "Phase 1.4: baselines per sub-principal (OAuth app, token, service account, AI agent acting for a human); GitHub tokens joined to scopes and classified by cadence"- status: pending+ status: completed- id: p1-5-change-before-alertcontent: "Phase 1.5: a cited 'what changed before this alert' tool over versioned posture and audit events (role, policy, OAuth consent, MFA, sharing)"- status: pending+ status: completed- id: p1-6-anonymizer-classescontent: "Phase 1.6: VPN, Tor, proxy and hosting-provider classes stamped at ingest from a pinned local dataset with its version recorded"status: in_progress@@ -49,7 +49,7 @@ todos:status: pending- id: p3-1-identity-resolutioncontent: "Phase 3.1: identity resolution across IdP, EDR device user, proxy user, cloud principal, code platform login and Workspace, with confidence and evidence"- status: pending+ status: completed- id: p3-2-attack-storiescontent: "Phase 3.2: cross-source attack stories linked by resolved identity, session, role chain and shared indicator, with story-level severity and a ledger reason for any escalation"status: pendingdiff --git a/protobuf-6.33.6-cp39-abi3-macosx_10_9_universal2.whl b/protobuf-6.33.6-cp39-abi3-macosx_10_9_universal2.whlnew file mode 100644index 000000000..10f048471Binary files /dev/null and b/protobuf-6.33.6-cp39-abi3-macosx_10_9_universal2.whl differdiff --git a/pycparser-3.12-py3-none-any.whl b/pycparser-3.12-py3-none-any.whlnew file mode 100644index 000000000..e5c916077Binary files /dev/null and b/pycparser-3.12-py3-none-any.whl differdiff --git a/pydantic-2.14.2-py3-none-any.whl b/pydantic-2.14.2-py3-none-any.whlnew file mode 100644index 000000000..3e36b4ac3Binary files /dev/null and b/pydantic-2.14.2-py3-none-any.whl differdiff --git a/pydantic_core-2.50.2-cp312-cp312-macosx_11_0_arm64.whl b/pydantic_core-2.50.2-cp312-cp312-macosx_11_0_arm64.whlnew file mode 100644index 000000000..30623bcbeBinary files /dev/null and b/pydantic_core-2.50.2-cp312-cp312-macosx_11_0_arm64.whl differdiff --git a/pydantic_settings-2.16.2-py3-none-any.whl b/pydantic_settings-2.16.2-py3-none-any.whlnew file mode 100644index 000000000..2f4f60791Binary files /dev/null and b/pydantic_settings-2.16.2-py3-none-any.whl differdiff --git a/pygments-2.21.0-py3-none-any.whl b/pygments-2.21.0-py3-none-any.whlnew file mode 100644index 000000000..82c0df884Binary files /dev/null and b/pygments-2.21.0-py3-none-any.whl differdiff --git a/pyjwt-2.15.1-py3-none-any.whl b/pyjwt-2.15.1-py3-none-any.whlnew file mode 100644index 000000000..00e0983b6Binary files /dev/null and b/pyjwt-2.15.1-py3-none-any.whl differdiff --git a/python_dotenv-1.2.4-py3-none-any.whl b/python_dotenv-1.2.4-py3-none-any.whlnew file mode 100644index 000000000..6a24aa019Binary files /dev/null and b/python_dotenv-1.2.4-py3-none-any.whl differdiff --git a/python_multipart-0.0.33-py3-none-any.whl b/python_multipart-0.0.33-py3-none-any.whlnew file mode 100644index 000000000..53c341fe7Binary files /dev/null and b/python_multipart-0.0.33-py3-none-any.whl differdiff --git a/referencing-0.37.0-py3-none-any.whl b/referencing-0.37.0-py3-none-any.whlnew file mode 100644index 000000000..b2b482d60Binary files /dev/null and b/referencing-0.37.0-py3-none-any.whl differdiff --git a/requests-2.34.2-py3-none-any.whl b/requests-2.34.2-py3-none-any.whlnew file mode 100644index 000000000..fb4b8a784Binary files /dev/null and b/requests-2.34.2-py3-none-any.whl differdiff --git a/rich-15.0.0-py3-none-any.whl b/rich-15.0.0-py3-none-any.whlnew file mode 100644index 000000000..5459ec396Binary files /dev/null and b/rich-15.0.0-py3-none-any.whl differdiff --git a/rpds_py-2026.9.1-cp312-cp312-macosx_11_0_arm64.whl b/rpds_py-2026.9.1-cp312-cp312-macosx_11_0_arm64.whlnew file mode 100644index 000000000..c71cf56d6Binary files /dev/null and b/rpds_py-2026.9.1-cp312-cp312-macosx_11_0_arm64.whl differdiff --git a/ruamel_yaml-0.19.1-py3-none-any.whl b/ruamel_yaml-0.19.1-py3-none-any.whlnew file mode 100644index 000000000..88f673c11Binary files /dev/null and b/ruamel_yaml-0.19.1-py3-none-any.whl differdiff --git a/ruamel_yaml_clib-0.2.15-cp312-cp312-macosx_11_0_arm64.whl b/ruamel_yaml_clib-0.2.15-cp312-cp312-macosx_11_0_arm64.whlnew file mode 100644index 000000000..9ad138150Binary files /dev/null and b/ruamel_yaml_clib-0.2.15-cp312-cp312-macosx_11_0_arm64.whl differdiff --git a/scripts/check_detection_fields.py b/scripts/check_detection_fields.pyindex 476c4e589..84872a364 100644--- a/scripts/check_detection_fields.py+++ b/scripts/check_detection_fields.py@@ -299,6 +299,17 @@ _BASELINE_DERIVED_FIELDS = frozenset("baseline_principal_volume_z","baseline_oauth_app_volume_z","baseline_peer_deviation",+ # Depth plan wave 2, 1.4 — per-sub-principal baselines. A token, an+ # OAuth app, a service account or an AI agent acting for a human+ # gets its own baseline, because a stolen credential acts *as* the+ # user and leaves the user's own numbers untouched.+ "baseline_sub_principal_kind",+ "baseline_sub_principal_rarity",+ "baseline_sub_principal_volume_z",+ "baseline_token_cadence",+ "baseline_token_cadence_shift",+ "baseline_token_reactivated_after_idle",+ "baseline_token_new_asn_after_creation",})diff --git a/scripts/check_session_projection.py b/scripts/check_session_projection.pyindex 0eb88eb4e..c07d3d5ac 100644--- a/scripts/check_session_projection.py+++ b/scripts/check_session_projection.py@@ -102,6 +102,21 @@ SESSION_FIELD_PATTERNS: tuple[tuple[str, str], ...] = (ALLOWED: dict[str, str] = {"scripts/check_session_projection.py": "this gate, which has to name the patterns it looks for","services/ingest/internal/activity/session_test.go": "the producer's own tests, which assert on these fields by name",+ "services/fusion/tests/test_sub_principal_baselines_reproduction.py": (+ "names the vendor field to assert the consumer **ignores** it. The "+ "sub-principal resolver used to read GitHub's digest itself and pin "+ "its normalisation against the Go side; that is a second derivation "+ "of a session key, so it now reads activity.session and this test is "+ "the negative control proving the old path resolves nothing. Naming "+ "the field is the assertion, which is the inverse of the hazard."+ ),+ "services/ingest/internal/activity/github_token_key_test.go": (+ "the producer's own test for the GitHub credential key, same reason as "+ "session_test.go beside it: it asserts that session.go reads the vendor "+ "field with TrimSpace and no case folding. Naming the field is the test's "+ "subject, and this file sits inside the one component allowed to derive "+ "the projection rather than downstream of it."+ ),"services/connectors/app/connectors/okta.py": ("the Okta connector lifts authenticationContext.externalSessionId to a top-level ""`session_id` in normalize() so the windowed AiTM rules (depth 7.3) can group on it — "diff --git a/semantic_version-2.10.0-py2.py3-none-any.whl b/semantic_version-2.10.0-py2.py3-none-any.whlnew file mode 100644index 000000000..97bd2cbb0Binary files /dev/null and b/semantic_version-2.10.0-py2.py3-none-any.whl differdiff --git a/semgrep-1.183.0-cp310.cp311.cp312.cp313.cp314.py310.py311.py312.py313.py314-none-macosx_11_0_arm64.whl b/semgrep-1.183.0-cp310.cp311.cp312.cp313.cp314.py310.py311.py312.py313.py314-none-macosx_11_0_arm64.whlnew file mode 100644index 000000000..f156212a2Binary files /dev/null and b/semgrep-1.183.0-cp310.cp311.cp312.cp313.cp314.py310.py311.py312.py313.py314-none-macosx_11_0_arm64.whl differdiff --git a/services/agents/app/agents/auto_triage_agent.py b/services/agents/app/agents/auto_triage_agent.pyindex 76a707b0a..5a76cb6b6 100644--- a/services/agents/app/agents/auto_triage_agent.py+++ b/services/agents/app/agents/auto_triage_agent.py@@ -48,6 +48,8 @@ from app.context.identity import render_for_prompt as render_identityfrom app.context.knowledge_base import citation_basis, unresolvable_citationsfrom app.context.knowledge_base import render_for_prompt as render_runbooksfrom app.context.organisation_memory import render_for_prompt+from app.context.privilege import basis as privilege_basis+from app.context.privilege import render_for_prompt as render_privilegefrom app.core.model_attribution import extract_serving_attributionfrom app.investigator.prompt_sanitizer import sanitize_text, wrap_untrustedfrom app.llm import safe_ainvoke@@ -321,6 +323,15 @@ def _build_alert_context(state: InvestigationState, *, nonce: str) -> str:if who:preamble.append(sanitize_text(who, max_len=_MAX_CONTEXT_BLOCK_CHARS))+ # What that principal could do when it acted (depth 2 wave 2, 1.1).+ # First-party like the block above -- it is the platform's own derivation+ # over a posture snapshot, not vendor text -- but sanitised anyway,+ # because the principal *name* inside it came from a connector and is+ # attacker-influenceable on exactly the alerts where it matters.+ can_do = render_privilege(state.privilege_context)+ if can_do:+ preamble.append(sanitize_text(can_do, max_len=_MAX_CONTEXT_BLOCK_CHARS))+# What the tenant's business-context rules did to this alert, told to the# model rather than applied silently: a `set_severity` the model cannot# see leaves it explaining a severity nobody chose in the evidence. The@@ -670,6 +681,10 @@ async def run_auto_triage(state: InvestigationState) -> InvestigationState:state.confidence_basis.extend(citation_basis(state.knowledge_base))state.confidence_basis.extend(disposition_basis(state.recent_dispositions))state.confidence_basis.extend(identity_basis(state.identity_context))+ # Depth 2 wave 2, 1.1. Every line here names the tool that produced+ # it, so an auditor can match the claim to a `tool_call` row rather+ # than taking the prompt's word for it.+ state.confidence_basis.extend(privilege_basis(state.privilege_context))# The business-impact element survives only when every citation it carries# resolves to a rule id that actually fired on this alert — the samediff --git a/services/agents/app/context/privilege.py b/services/agents/app/context/privilege.pynew file mode 100644index 000000000..9389cc272--- /dev/null+++ b/services/agents/app/context/privilege.py@@ -0,0 +1,269 @@+"""What the alert's actor could do, as of the alert — for the triage prompt.++Depth plan wave 2, Phase 1.1.++The sibling of :mod:`app.context.identity`, which answers *who* holds the+account. This one answers *what the account could do when it acted*, which is+the question that decides whether an unusual sign-in is a nuisance or an+incident.++Why this goes through ``/investigate/query``+--------------------------------------------+There is exactly one dispatch table for investigation primitives+(``investigation_tools.TOOLS``), and the console, the typed-pivot route and+the model's tool registry all read it. Adding a second route for triage would+let the console and the model be offered different tools, which is how a+capability ends up working on one surface and silently missing on another.++So this calls the same endpoint the registered tool calls, which is also what+makes the answer appear in the ledger: a tool call recorded against a run is+the auditable artefact, and a side-channel read would produce a prompt+sentence with no matching ``tool_call`` row — the shape the grounding check+already demotes a verdict for.++What it never does+------------------+It never turns an unanswerable question into silence. When no posture covers+the alert's time the block says so in words, because the alternative is a+model inferring from an absent section that the principal was ordinary.+"""++from __future__ import annotations++import os+import time+from collections.abc import Mapping, Sequence+from dataclasses import dataclass, field+from datetime import datetime+from typing import Any++import httpx+import structlog++logger = structlog.get_logger()++__all__ = [+ "PrivilegeContext",+ "basis",+ "enabled",+ "fetch_privilege_context",+ "render_for_prompt",+]++_API_URL = os.getenv("AISOC_API_URL", "http://api:8000")+TENANT_HEADER = "X-AiSOC-Tenant-ID"+_TIMEOUT_S = 8.0+_CACHE_TTL_S = 120.0+#: Hard cap on principals looked up per alert. An alert naming forty accounts+#: is a sweep, and forty round trips on the triage hot path would be the+#: thing that made triage slow rather than the thing that made it right.+MAX_PRINCIPALS = 3++_cache: dict[str, tuple[float, dict[str, Any]]] = {}+++def enabled() -> bool:+ """On by default. It reads the platform's own Postgres, nothing external.++ The "off by default where it reaches out" rule governs calls that leave+ the deployment; this one does not, and a tenant that has collected no+ posture simply gets "unknown" everywhere at no cost.+ """+ return os.getenv("AISOC_TRIAGE_PRIVILEGE_CONTEXT", "1").strip().lower() not in {"0", "false", "no", "off"}+++def clear_cache() -> None:+ _cache.clear()+++@dataclass(frozen=True)+class PrivilegeContext:+ """One lookup's worth of privilege answers, with its gaps named."""++ principals: tuple[dict[str, Any], ...] = ()+ #: Principals asked about for which no posture version covered the event.+ #: Carried rather than dropped: "we could not answer" is the finding when+ #: a tenant has connected a provider and not collected its posture.+ unresolved: tuple[str, ...] = ()+ at: str | None = None+ notes: tuple[str, ...] = field(default_factory=tuple)++ def as_state(self) -> dict[str, Any]:+ return {+ "principals": [dict(p) for p in self.principals],+ "unresolved": list(self.unresolved),+ "at": self.at,+ "notes": list(self.notes),+ }++ def __bool__(self) -> bool:+ return bool(self.principals or self.unresolved)+++def _service_token() -> str:+ return (os.getenv("AISOC_API_SERVICE_TOKEN") or os.getenv("AISOC_SERVICE_TOKEN") or "").strip()+++async def fetch_privilege_context(+ tenant_id: str | None,+ *,+ principals: Sequence[str],+ at: datetime | None = None,+) -> PrivilegeContext:+ """Privilege for these principals as of ``at``. Never raises.++ ``at`` is the alert's event time and is not optional in spirit: passing+ ``None`` asks what the identity can do *now*, which is a different+ question from the one triage is answering.+ """+ if not enabled() or not tenant_id:+ return PrivilegeContext()+ names = [p.strip() for p in principals if p and p.strip()][:MAX_PRINCIPALS]+ if not names:+ return PrivilegeContext()++ token = _service_token()+ if not token:+ logger.warning(+ "triage_privilege.no_service_token",+ reason="AISOC_SERVICE_TOKEN is unset, so privilege context cannot be read",+ )+ return PrivilegeContext()++ at_iso = at.isoformat() if at is not None else ""+ resolved: list[dict[str, Any]] = []+ unresolved: list[str] = []+ notes: list[str] = []++ for name in names:+ cache_key = f"{tenant_id}|{at_iso}|{name.lower()}"+ cached = _cache.get(cache_key)+ now = time.monotonic()+ payload: dict[str, Any]+ if cached is not None and now - cached[0] < _CACHE_TTL_S:+ payload = cached[1]+ else:+ fetched = await _query(tenant_id, name, at_iso, token)+ if fetched is None:+ # Transport failure. Deliberately not cached and deliberately+ # not recorded as "unresolved": the two mean different things+ # and conflating them would turn an API outage into a+ # permanent statement that a tenant has no posture.+ notes.append(f"the privilege lookup for {name} could not be completed")+ continue+ payload = fetched+ _cache[cache_key] = (now, payload)++ if payload.get("available") is False:+ unresolved.append(name)+ continue+ rows = payload.get("rows") or []+ if rows:+ resolved.append(dict(rows[0]))+ else:+ unresolved.append(name)++ return PrivilegeContext(+ principals=tuple(resolved),+ unresolved=tuple(unresolved),+ at=at_iso or None,+ notes=tuple(notes),+ )+++async def _query(tenant_id: str, principal: str, at_iso: str, token: str) -> dict[str, Any] | None:+ args: dict[str, Any] = {"principal_id": principal}+ if at_iso:+ args["at"] = at_iso+ url = f"{_API_URL.rstrip('/')}/api/v1/graph/investigate/query"+ try:+ async with httpx.AsyncClient(timeout=_TIMEOUT_S) as client:+ response = await client.post(+ url,+ json={"tool": "identity_effective_permissions", "args": args},+ headers={"Authorization": f"Bearer {token}", TENANT_HEADER: tenant_id},+ )+ except httpx.HTTPError as exc:+ logger.warning("triage_privilege.unreachable", error=str(exc)[:300])+ return None+ if response.status_code >= 400:+ logger.warning("triage_privilege.refused", status_code=response.status_code)+ return None+ try:+ payload = response.json()+ except ValueError:+ logger.warning("triage_privilege.bad_response")+ return None+ return payload if isinstance(payload, dict) else None+++def render_for_prompt(state_value: Mapping[str, Any] | None) -> str:+ """The prompt block, including the "we could not answer" case.++ Unlike the directory block one module over, this one is **not** empty+ when the answer is unknown. The difference is deliberate and is about+ which silence a model would misread. An absent directory block means the+ tenant has no directory connector, and saying so would be noise. An+ absent privilege block, on an alert about a cloud principal, reads as+ "nothing notable about this account's access" — which is a claim nobody+ made.+ """+ value = state_value or {}+ principals = value.get("principals")+ principals = [p for p in principals if isinstance(p, Mapping)] if isinstance(principals, list) else []+ unresolved = [str(u) for u in (value.get("unresolved") or []) if u]+ if not principals and not unresolved:+ return ""++ lines = ["## What the actor could do at the time of this alert", ""]+ for row in principals:+ tier = str(row.get("privilege_tier") or "unknown")+ tags = row.get("tags_set") or []+ name = str(row.get("principal_id") or "unknown principal")+ provider = str(row.get("provider") or "")+ tag_text = f"; tagged {', '.join(str(t) for t in tags)}" if tags else ""+ lines.append(f"- {name} ({provider}): privilege {tier}{tag_text}")+ if row.get("snapshot_was_partial"):+ lines.append(" - the posture snapshot was incomplete, so the real reach may be larger")+ if row.get("usage_measured") is False:+ lines.append(+ " - which of these permissions were actually exercised was NOT measured on this deployment; do not treat them as unused"+ )+ for name in unresolved:+ lines.append(+ f"- {name}: privilege UNKNOWN — no posture snapshot covers this moment. "+ f"This is a gap in collection, not evidence that the account is unprivileged."+ )+ return "\n".join(lines)+++def basis(state_value: Mapping[str, Any] | None) -> list[str]:+ """Confidence-basis lines naming what the prompt was given.++ Every line is a sentence an auditor can check against a ``tool_call``+ row, which is the point: a basis entry with no matching recorded call is+ the ungrounded shape the grounding check demotes a verdict for.+ """+ value = state_value or {}+ principals = value.get("principals")+ principals = [p for p in principals if isinstance(p, Mapping)] if isinstance(principals, list) else []+ unresolved = [str(u) for u in (value.get("unresolved") or []) if u]+ if not principals and not unresolved:+ return []++ out: list[str] = []+ for row in principals:+ tier = str(row.get("privilege_tier") or "unknown")+ tags = ", ".join(str(t) for t in (row.get("tags_set") or [])) or "no tags"+ at = str(value.get("at") or "the alert time")+ out.append(+ f"Effective permissions (identity_effective_permissions) for {row.get('principal_id')}: privilege {tier} at {at}; {tags}"+ )+ if unresolved:+ out.append(+ f"Effective permissions: UNKNOWN for {', '.join(unresolved)} — no posture snapshot "+ f"covers the alert time, so privilege did not inform this verdict"+ )+ for note in value.get("notes") or []:+ out.append(f"Effective permissions: {note}")+ return outdiff --git a/services/agents/app/models/state.py b/services/agents/app/models/state.pyindex 177b05cf2..dfddcc1e7 100644--- a/services/agents/app/models/state.py+++ b/services/agents/app/models/state.py@@ -160,6 +160,13 @@ class InvestigationState(BaseModel):recent_dispositions: dict[str, Any] | None = Noneidentity_context: dict[str, Any] | None = None+ # What those principals could do *at the alert's event time*, as+ # `app.context.privilege.PrivilegeContext.as_state()` renders it (depth+ # plan wave 2, 1.1). Distinct from `identity_context`, which says who+ # holds the account: a service account no directory knows about still+ # has effective permissions, and that is usually the interesting case.+ privilege_context: dict[str, Any] | None = None+# Findings accumulated during investigationfindings: list[str] = Field(default_factory=list)ioc_enrichments: dict[str, Any] = Field(default_factory=dict)diff --git a/services/agents/app/prompting/tool_results.py b/services/agents/app/prompting/tool_results.pyindex 647805406..946280560 100644--- a/services/agents/app/prompting/tool_results.py+++ b/services/agents/app/prompting/tool_results.py@@ -126,8 +126,14 @@ __all__ = [AGENT_TOOL_NAMES: frozenset[str] = frozenset({# app/tools/registry.py :: default_registry+ "changes_before_alert","enrich_ioc","extract_iocs",+ # Depth 2 wave 2, 1.1. Worth the guard more than most: a fabricated+ # "identity_effective_permissions: standard, no tags" is a sentence+ # that closes an alert, and it is one an injected runbook could ask+ # a model to write.+ "identity_effective_permissions","lookup_file_hash","lookup_technique","map_to_mitre",diff --git a/services/agents/app/tools/customer_tools.py b/services/agents/app/tools/customer_tools.pyindex b0542e1fd..7062d956d 100644--- a/services/agents/app/tools/customer_tools.py+++ b/services/agents/app/tools/customer_tools.py@@ -49,7 +49,7 @@ from typing import Anyimport httpximport structlog-from app.tools.registry import Tool+from app.tools.tool import Toollogger = structlog.get_logger()@@ -198,6 +198,183 @@ def _detail(response: httpx.Response) -> str:return ""+# ------------------------------------------- effective permissions (1.1)+++EFFECTIVE_PERMISSIONS_TOOL = "identity_effective_permissions"++EFFECTIVE_PERMISSIONS_DESCRIPTION = (+ "What a cloud or identity-provider principal could actually do AT THE TIME OF THE EVENT: "+ "its privilege tier, whether it can escalate its own or others' access, whether it can read "+ "resources the customer marked sensitive, whether it is third-party or vendor-owned, and "+ "whether it reaches another account or project. Pass the alert's own timestamp as `at`, not "+ "the current time: a role revoked after the event still explains the event. An answer of "+ "'unknown' means no posture snapshot covers that moment and is NOT evidence that the "+ "identity is unprivileged."+)++EFFECTIVE_PERMISSIONS_PARAMETERS: dict[str, Any] = {+ "type": "object",+ "properties": {+ "principal_id": {+ "type": "string",+ "description": (+ "The principal as the alert names it: an AWS ARN or user name, an Entra object id "+ "or UPN, a GCP member, an Okta or Workspace user, or a code-platform login."+ ),+ },+ "at": {+ "type": "string",+ "description": (+ "ISO-8601 timestamp of the event being investigated. Omitting it asks about now, which is a different question."+ ),+ },+ "provider": {+ "type": "string",+ "enum": ["aws", "azure", "gcp", "gws", "github", "okta"],+ "description": ("Optional. Omit it to search every provider that knows this principal and take the strongest privilege found."),+ },+ },+ "required": ["principal_id"],+}+++async def identity_effective_permissions(+ principal_id: str,+ at: str = "",+ provider: str = "",+ *,+ tenant_id: str = "",+) -> dict[str, Any]:+ """Ask the API what this identity could do when it acted.++ Goes through ``/investigate/query``, the same typed-pivot door the+ console uses, rather than a second route of its own. Two reasons, both+ learned here: one dispatch table means the console and the model cannot+ be offered different tools, and the pivot endpoint already takes the+ tenant from the authenticated principal rather than from the request, so+ a model cannot redirect the read.+ """+ what = f"the effective permissions of {principal_id!r}"+ args: dict[str, Any] = {"principal_id": principal_id}+ if at:+ args["at"] = at+ if provider:+ args["provider"] = provider++ body = await _call(+ "/api/v1/graph/investigate/query",+ {"tool": EFFECTIVE_PERMISSIONS_TOOL, "args": args},+ what,+ tenant_id,+ )+ if body.get("available") is False:+ # The API already distinguishes "no snapshot covers this time" from+ # "the lookup failed", and both arrive here as available=false with+ # their own reason. Rewriting either into a generic failure would+ # lose the distinction the whole tool exists to preserve.+ return body++ rows = body.get("rows") or []+ if not rows:+ return _could_not_check(what, "The privilege lookup returned no record for this principal.")++ row = dict(rows[0])+ row["available"] = True+ row["outcome"] = "resolved"+ if row.get("usage_measured") is False:+ # Said in words because a null field is the kind of thing a model+ # fills in with a plausible guess.+ row["usage_note"] = (+ "Which of these permissions were actually exercised was NOT measured on this "+ "deployment (the event lake is a `full`-profile component). Do not conclude that "+ "unused means unexercised."+ )+ if row.get("snapshot_was_partial"):+ row["coverage_note"] = (+ "The posture snapshot behind this answer was incomplete, so the real permission set may be larger than what is listed."+ )+ return row+++# ------------------------------------------ what changed before (1.5)+++CHANGES_BEFORE_TOOL = "changes_before_alert"++CHANGES_BEFORE_DESCRIPTION = (+ "Every recorded change touching these identities or resources in the days BEFORE this alert: "+ "role and group grants, policy edits, OAuth consents, MFA factor changes, sharing changes and "+ "key creation. Reads the customer's versioned posture and their audit events, and says which "+ "of the two it could read. An answer of 'could not read' is a gap in visibility and is NOT "+ "evidence that nothing changed."+)++CHANGES_BEFORE_PARAMETERS: dict[str, Any] = {+ "type": "object",+ "properties": {+ "subjects": {+ "type": "string",+ "description": (+ "Comma-separated identities or resources from the alert. Ask about the actor and "+ "the thing it touched together rather than in two calls."+ ),+ },+ "at": {+ "type": "string",+ "description": "ISO-8601 timestamp of the alert. The window ends here.",+ },+ "days": {+ "type": "integer",+ "description": "How many days before `at` to look. Default 7, maximum 90.",+ },+ },+ "required": ["subjects"],+}+++async def changes_before_alert(+ subjects: str,+ at: str = "",+ days: int = 7,+ *,+ tenant_id: str = "",+) -> dict[str, Any]:+ """Ask the API what changed before this alert.++ Through `/investigate/query` like the privilege tool, for the same+ reason: one dispatch table means the console and the model cannot be+ offered different tools, and the endpoint takes the tenant from the+ authenticated principal rather than from the request.+ """+ what = f"changes to {subjects!r} before this alert"+ args: dict[str, Any] = {"subjects": subjects, "days": int(days or 7)}+ if at:+ args["at"] = at++ body = await _call(+ "/api/v1/graph/investigate/query",+ {"tool": CHANGES_BEFORE_TOOL, "args": args},+ what,+ tenant_id,+ )+ if body.get("available") is False:+ return body++ rows = body.get("rows") or []+ return {+ "available": True,+ "outcome": "resolved",+ "change_count": len(rows),+ "changes": rows,+ "query": body.get("query"),+ # Present even when the list is empty: "read both sources and found+ # nothing" is a measured result, and without this sentence a model+ # reads an empty list as a failed lookup -- or worse, the reverse.+ "note": body.get("reason"),+ }++# ------------------------------------------------------------- SIEM search@@ -538,6 +715,12 @@ async def scoped_customer_tools(tenant_id: str) -> tuple[list[Tool], list[str]]:__all__ = [+ "CHANGES_BEFORE_DESCRIPTION",+ "CHANGES_BEFORE_PARAMETERS",+ "CHANGES_BEFORE_TOOL",+ "EFFECTIVE_PERMISSIONS_DESCRIPTION",+ "EFFECTIVE_PERMISSIONS_PARAMETERS",+ "EFFECTIVE_PERMISSIONS_TOOL","INDICATOR_TYPES","SIEM_SEARCH_TOOL","UNTRUSTED_NOTICE",diff --git a/services/agents/app/tools/registry.py b/services/agents/app/tools/registry.pyindex 45047af15..f07c12520 100644--- a/services/agents/app/tools/registry.py+++ b/services/agents/app/tools/registry.py@@ -17,23 +17,14 @@ Design:from __future__ import annotationsimport inspect-from collections.abc import Awaitable, Callable-from dataclasses import dataclassfrom typing import Any+# Re-exported so every existing `from app.tools.registry import Tool`+# keeps working. The dataclass moved to its own module to break the+# registry -> customer_tools -> registry import cycle; see tool.py.+from app.tools.tool import Tool-@dataclass(frozen=True)-class Tool:- name: str- description: str- parameters: dict[str, Any] # JSON schema for the arguments- fn: Callable[..., Any] | Callable[..., Awaitable[Any]]-- def openai_schema(self) -> dict[str, Any]:- return {- "type": "function",- "function": {"name": self.name, "description": self.description, "parameters": self.parameters},- }+__all__ = ["Tool", "ToolRegistry", "default_registry"]class ToolRegistry:@@ -75,6 +66,16 @@ def default_registry(tenant_id: str = "") -> ToolRegistry:refuses a service credential that does not name the tenant it acts for."""from app.investigator.tools import enrich_ioc, extract_iocs, map_to_mitre+ from app.tools.customer_tools import (+ CHANGES_BEFORE_DESCRIPTION,+ CHANGES_BEFORE_PARAMETERS,+ CHANGES_BEFORE_TOOL,+ EFFECTIVE_PERMISSIONS_DESCRIPTION,+ EFFECTIVE_PERMISSIONS_PARAMETERS,+ EFFECTIVE_PERMISSIONS_TOOL,+ changes_before_alert,+ identity_effective_permissions,+ )from app.tools.mitre import lookup_techniquefrom app.tools.sandbox import TOOL_DESCRIPTION, TOOL_NAME, TOOL_PARAMETERS, lookup_file_hash@@ -86,6 +87,29 @@ def default_registry(tenant_id: str = "") -> ToolRegistry:parameters=TOOL_PARAMETERS,fn=lambda sha256: lookup_file_hash(sha256, tenant_id=tenant_id),),+ # Registered unconditionally, unlike the customer tools in+ # `scoped_customer_tools`, which are bound only when the tenant+ # has a backend for them. The difference is deliberate: a+ # customer tool with no backend answers `no_integration` and is+ # noise, whereas this one answers "unknown, and here is why" —+ # which is a sentence the model needs to be able to reach.+ Tool(+ name=EFFECTIVE_PERMISSIONS_TOOL,+ description=EFFECTIVE_PERMISSIONS_DESCRIPTION,+ parameters=EFFECTIVE_PERMISSIONS_PARAMETERS,+ fn=lambda principal_id, at="", provider="": identity_effective_permissions(+ principal_id, at=at, provider=provider, tenant_id=tenant_id+ ),+ ),+ # Depth 2 wave 2, 1.5. Registered beside the privilege tool+ # because the two answer halves of one question: what this+ # identity could do, and what changed shortly before it did.+ Tool(+ name=CHANGES_BEFORE_TOOL,+ description=CHANGES_BEFORE_DESCRIPTION,+ parameters=CHANGES_BEFORE_PARAMETERS,+ fn=lambda subjects, at="", days=7: changes_before_alert(subjects, at=at, days=days, tenant_id=tenant_id),+ ),Tool(name="extract_iocs",description="Extract IOCs (IPs, domains, hashes, URLs) from free text.",diff --git a/services/agents/app/tools/tool.py b/services/agents/app/tools/tool.pynew file mode 100644index 000000000..eeb041328--- /dev/null+++ b/services/agents/app/tools/tool.py@@ -0,0 +1,44 @@+"""The ``Tool`` descriptor, in its own module so the registry can import tools.++Why this is not in ``registry.py``+----------------------------------+``Tool`` is a data shape; ``ToolRegistry`` is machinery that holds them. They+lived together, which was fine for as long as the registry only ever received+tools built elsewhere and never built any itself.++Depth 2 wave 2 changed that: ``default_registry`` now constructs the identity+tools, which live in ``customer_tools``, which imports ``Tool`` — and that is+an import cycle (``registry`` → ``customer_tools`` → ``registry``). CodeQL's+``py/cyclic-import`` flagged it on the first run.++A cycle that works today is still worth breaking: it works only because one+of the two imports is inside a function, so the first module-level import of+``customer_tools`` anywhere in the process would make the failure appear at a+completely unrelated call site. Splitting the data shape from the machinery+removes the cycle rather than deferring it.++``registry.py`` re-exports ``Tool``, so every existing+``from app.tools.registry import Tool`` keeps working.+"""++from __future__ import annotations++from collections.abc import Awaitable, Callable+from dataclasses import dataclass+from typing import Any++__all__ = ["Tool"]+++@dataclass(frozen=True)+class Tool:+ name: str+ description: str+ parameters: dict[str, Any] # JSON schema for the arguments+ fn: Callable[..., Any] | Callable[..., Awaitable[Any]]++ def openai_schema(self) -> dict[str, Any]:+ return {+ "type": "function",+ "function": {"name": self.name, "description": self.description, "parameters": self.parameters},+ }diff --git a/services/agents/app/workers/fused_alert_consumer.py b/services/agents/app/workers/fused_alert_consumer.pyindex f2071472c..9ec7619b8 100644--- a/services/agents/app/workers/fused_alert_consumer.py+++ b/services/agents/app/workers/fused_alert_consumer.py@@ -49,6 +49,7 @@ from app.confidence.groundedness import score_groundednessfrom app.context import dispositions as dispositions_modulefrom app.context import identity as identity_modulefrom app.context import knowledge_base+from app.context import privilege as privilege_modulefrom app.context.tenant_skills import select_skillfrom app.core.cost_governor import Decision, get_governorfrom app.core.cost_telemetry import CostSummary, CostTracker@@ -139,6 +140,7 @@ _METRICS = {"runbooks_refused_for_injection": 0,"recent_dispositions_read": 0,"identity_context_read": 0,+ "privilege_context_read": 0,"errors": 0,}@@ -763,6 +765,12 @@ class FusedAlertTriageWorker:# here.await self._attach_recent_dispositions(state)await self._attach_identity_context(state)+ # Depth 2 wave 2, 1.1. Unlike the three above, this one+ # *does* take a point in time, and it is the alert's own:+ # the posture store is versioned precisely so "what could+ # this identity do when it acted" is answerable, and+ # passing no time would silently ask the other question.+ await self._attach_privilege_context(state)# Bind a CostTracker so every LLM call on this path records its# token/cost (safe_ainvoke -> record_llm_call) — previously the# highest-volume LLM spend was recorded as $0 and invisible.@@ -1201,6 +1209,57 @@ class FusedAlertTriageWorker:state.identity_context = found.as_state()_METRICS["identity_context_read"] += len(found.identities)+ @staticmethod+ def _event_time_of(state: InvestigationState) -> datetime | None:+ """The alert's own event time, or ``None`` to mean "do not guess".++ ``None`` propagates to a lookup at *now*, which is the honest+ fallback for an alert that carries no timestamp: the alternative is+ inventing one, and a fabricated point in time selects a posture+ version with the same confidence as a real one.+ """+ raw = state.raw_alert or {}+ for key in ("event_time", "timestamp", "first_seen", "@timestamp"):+ value = raw.get(key)+ if isinstance(value, datetime):+ return value if value.tzinfo else value.replace(tzinfo=UTC)+ if isinstance(value, str) and value.strip():+ try:+ parsed = datetime.fromisoformat(value.strip().replace("Z", "+00:00"))+ except ValueError:+ continue+ return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)+ return None++ async def _attach_privilege_context(self, state: InvestigationState) -> None:+ """What those accounts could do when they acted (depth 2 wave 2, 1.1).++ Separate from ``_attach_identity_context`` and not gated on it, which+ is the part worth stating: a cloud principal that no directory import+ knows about still has effective permissions, and making privilege+ conditional on a directory hit would lose exactly the service+ accounts and assumed roles the tag set is most useful for.++ The event's own timestamp is passed, never ``now()``: a role revoked+ after the event is still what explains the event.+ """+ accounts = identity_module.accounts_for(state.raw_alert)+ if not accounts:+ return+ try:+ found = await privilege_module.fetch_privilege_context(+ str(state.tenant_id),+ principals=accounts,+ at=self._event_time_of(state),+ )+ except Exception as exc: # noqa: BLE001 - privilege context is advisory, triage is not+ logger.warning("auto_triage_worker.privilege_lookup_failed", error=str(exc)[:200])+ return+ if not found:+ return+ state.privilege_context = found.as_state()+ _METRICS["privilege_context_read"] += len(found.principals)+async def _maybe_suppress_from_memory(self,state: InvestigationState,diff --git a/services/agents/tests/test_identity_privilege_tool_reproduction.py b/services/agents/tests/test_identity_privilege_tool_reproduction.pynew file mode 100644index 000000000..ba1836853--- /dev/null+++ b/services/agents/tests/test_identity_privilege_tool_reproduction.py@@ -0,0 +1,49 @@+"""Depth 2 Phase 1.1 reproduction, agents half: the model is never offered it.++``services/agents/app/tools/registry.py`` is what ``bind_tools`` ships to the+provider, so it is the list of things a model can *choose* to do. Six+effective-permission resolvers have shipped since T3.2 and none of them+appears here, which means no triage run has ever asked what an identity can+do — the same shape as ``run_with_tools``, a working ReAct loop that had no+production caller for three releases.++Fails on the pre-change tree for that reason; kept afterwards as the guard.+"""++from __future__ import annotations++from app.tools.registry import default_registry++TOOL_NAME = "identity_effective_permissions"+++def test_registry_offers_the_effective_permission_tool() -> None:+ assert TOOL_NAME in default_registry(tenant_id="t1").names()+++def test_the_tool_is_shipped_in_the_openai_schema_block() -> None:+ """Present in the registry but absent from the schemas is still unoffered.++ ``openai_schemas()`` is the exact payload the provider receives. Asserting+ on ``names()`` alone would pass against a tool that the binding step+ filtered out.+ """+ schemas = default_registry(tenant_id="t1").openai_schemas()+ named = {s["function"]["name"] for s in schemas}+ assert TOOL_NAME in named+++def test_the_tool_takes_a_principal_and_a_point_in_time() -> None:+ """ "What can it do" and "what could it do when it acted" differ.++ An alert is triaged minutes to days after the event. A tool that cannot+ accept the event's timestamp can only answer the second question while+ appearing to answer the first, which is the failure mode the versioned+ posture table was built to prevent.+ """+ registry = default_registry(tenant_id="t1")+ schema = next(s for s in registry.openai_schemas() if s["function"]["name"] == TOOL_NAME)+ properties = schema["function"]["parameters"]["properties"]+ assert "principal_id" in properties+ assert "provider" in properties+ assert "at" in propertiesdiff --git a/services/api/app/api/v1/endpoints/access_review.py b/services/api/app/api/v1/endpoints/access_review.pynew file mode 100644index 000000000..90b4e7738--- /dev/null+++ b/services/api/app/api/v1/endpoints/access_review.py@@ -0,0 +1,154 @@+"""Access-review routes (depth plan wave 2, Phase 1.2).++Read-only, in the strongest sense available: these routes compute and read+findings, and there is no route here — and no caller anywhere — that changes+a vendor object. The only write is a reviewer acknowledging a finding, which+is state about the review and not about the estate.++Three routes:++``GET /access-review/summary`` the honest empty state and the counts+``GET /access-review/findings`` the list a reviewer works through+``POST /access-review/run`` recompute from the current posture + usage+``POST /access-review/findings/{id}/acknowledge``++The empty state is the part worth reading. Zero findings over zero reviewed+principals is ``nothing_reviewed``, never ``no_findings``: an estate nobody+could look at must not render identically to one with nothing wrong, and+this page is exactly where a reader would draw that conclusion.+"""++from __future__ import annotations++import uuid+from typing import Annotated, Any++from fastapi import APIRouter, Depends, HTTPException, Query, status+from pydantic import BaseModel, Field++from app.api.v1.deps import AuthUser, DBSession, require_permission+from app.services import access_review_store+from app.services.access_review import FINDING_KINDS+from app.services.effective_permissions.service import SUPPORTED_PROVIDERS++router = APIRouter(prefix="/access-review", tags=["access-review"])+++class AcknowledgeRequest(BaseModel):+ note: str = Field(default="", max_length=2000)++[email protected]("/kinds", summary="The finding kinds this review can produce")+async def list_kinds(+ _user: Annotated[AuthUser, Depends(require_permission("compliance:read"))],+) -> dict[str, Any]:+ """Published so the console can render a filter without hardcoding a copy.++ A second list in the frontend is how a kind ends up filterable on one+ surface and invisible on the other; this repository has that defect on+ record in the playbook editor's step-type union.+ """+ return {+ "kinds": [+ {+ "kind": spec.kind,+ "title": spec.title,+ "severity": spec.severity,+ "object_hint": spec.object_hint,+ "recommendation": spec.recommendation,+ "needs_usage": spec.needs_usage,+ }+ for spec in sorted(FINDING_KINDS.values(), key=lambda s: s.kind)+ ]+ }++[email protected]("/summary", summary="Review state and counts, with an honest empty state")+async def review_summary(+ db: DBSession,+ user: Annotated[AuthUser, Depends(require_permission("compliance:read"))],+ provider: str | None = Query(default=None),+) -> dict[str, Any]:+ if provider and provider not in SUPPORTED_PROVIDERS:+ raise HTTPException(+ status_code=status.HTTP_400_BAD_REQUEST,+ detail=f"unknown provider {provider!r}; supported: {sorted(SUPPORTED_PROVIDERS)}",+ )+ return await access_review_store.summary(db, tenant_id=user.tenant_id, provider=provider)++[email protected]("/findings", summary="Open access-review findings")+async def list_findings(+ db: DBSession,+ user: Annotated[AuthUser, Depends(require_permission("compliance:read"))],+ provider: str | None = Query(default=None),+ kind: str | None = Query(default=None),+ include_acknowledged: bool = Query(default=False),+ limit: int = Query(default=100, ge=1, le=500),+) -> dict[str, Any]:+ if provider and provider not in SUPPORTED_PROVIDERS:+ raise HTTPException(+ status_code=status.HTTP_400_BAD_REQUEST,+ detail=f"unknown provider {provider!r}; supported: {sorted(SUPPORTED_PROVIDERS)}",+ )+ if kind and kind not in FINDING_KINDS:+ raise HTTPException(+ status_code=status.HTTP_400_BAD_REQUEST,+ detail=f"unknown finding kind {kind!r}; see GET /access-review/kinds",+ )+ rows = await access_review_store.list_findings(+ db,+ tenant_id=user.tenant_id,+ provider=provider,+ kind=kind,+ include_acknowledged=include_acknowledged,+ limit=limit,+ )+ return {"items": rows, "count": len(rows)}++[email protected]("/run", summary="Recompute the review from current posture and usage")+async def run_review(+ db: DBSession,+ user: Annotated[AuthUser, Depends(require_permission("compliance:read"))],+ provider: str = Query(...),+) -> dict[str, Any]:+ """Recompute. Reads only — no vendor object is touched by this call.++ Guarded by ``compliance:read`` rather than a write permission on+ purpose: the only thing it writes is this tenant's own review rows, and+ requiring a write scope would put the review behind a permission an+ auditor is deliberately not given.+ """+ if provider not in SUPPORTED_PROVIDERS:+ raise HTTPException(+ status_code=status.HTTP_400_BAD_REQUEST,+ detail=f"unknown provider {provider!r}; supported: {sorted(SUPPORTED_PROVIDERS)}",+ )+ return await access_review_store.run_review(db, tenant_id=user.tenant_id, provider=provider)++[email protected]("/findings/{finding_id}/acknowledge", summary="Acknowledge a finding")+async def acknowledge(+ finding_id: uuid.UUID,+ body: AcknowledgeRequest,+ db: DBSession,+ user: Annotated[AuthUser, Depends(require_permission("compliance:read"))],+) -> dict[str, Any]:+ """Record that a person decided this one is fine.++ A 404 rather than a 403 for a finding belonging to another tenant: a+ 403 would make this route an existence oracle over every tenant's+ finding ids, and a genuine miss is byte-identical to a cross-tenant+ attempt from the caller's side either way.+ """+ updated = await access_review_store.acknowledge(+ db,+ tenant_id=user.tenant_id,+ finding_id=finding_id,+ user_id=user.user_id,+ note=body.note,+ )+ if updated is None:+ raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="finding not found")+ return updateddiff --git a/services/api/app/api/v1/endpoints/identity_resolution.py b/services/api/app/api/v1/endpoints/identity_resolution.pynew file mode 100644index 000000000..e5ed7d6c2--- /dev/null+++ b/services/api/app/api/v1/endpoints/identity_resolution.py@@ -0,0 +1,152 @@+"""Identity-resolution routes (depth plan wave 2, Phase 3.1).++Read-only over the tenant's own identities. Nothing here changes a vendor+object; the one write is a person confirming or rejecting a proposed link,+which is a decision about the link and not about the estate.++``GET /identity-resolution/links`` every link touching one identity+``GET /identity-resolution/cluster`` who else is this person+``POST /identity-resolution/resolve`` re-run resolution over supplied records+``POST /identity-resolution/links/{id}/decide``++The weak links are returned by default, and that is the contract rather+than an oversight: a weak link is genuinely useful to an analyst reading a+story and genuinely dangerous to anything that merges, so the list says+which is which per row and the cluster endpoint walks only the strong ones.+"""++from __future__ import annotations++import uuid+from typing import Annotated, Any++from fastapi import APIRouter, Depends, HTTPException, Query, status+from pydantic import BaseModel, Field++from app.api.v1.deps import AuthUser, DBSession, require_permission+from app.services import identity_resolution_store+from app.services.identity_resolution import MERGE_THRESHOLD, SOURCE_KINDS, resolve_links++router = APIRouter(prefix="/identity-resolution", tags=["identity-resolution"])++#: Cap on records accepted in one resolution call. Resolution is pairwise,+#: so cost is quadratic; a bound here is the difference between a slow+#: request and one that occupies a worker for minutes.+MAX_RECORDS = 500+++class IdentityRecord(BaseModel):+ """One source-scoped identity offered for resolution."""++ source: str+ id: str+ email: str | None = None+ email_verified: bool = False+ upn: str | None = None+ employee_id: str | None = None+ display_name: str | None = None+ maps_to: list[str] = Field(default_factory=list)+++class ResolveRequest(BaseModel):+ records: list[IdentityRecord] = Field(default_factory=list)+ #: Off by default. Resolution is cheap to preview and consequential to+ #: store, and an endpoint that persisted on every call would make+ #: "what would this link" impossible to ask.+ persist: bool = False+++class DecideRequest(BaseModel):+ decision: str++[email protected]("/links", summary="Every proposed link touching one identity")+async def list_links(+ db: DBSession,+ user: Annotated[AuthUser, Depends(require_permission("alerts:read"))],+ key: str = Query(..., description="A source-scoped identity, e.g. okta:[email protected]"),+ include_weak: bool = Query(default=True),+) -> dict[str, Any]:+ links = await identity_resolution_store.links_for(db, tenant_id=user.tenant_id, key=key, include_weak=include_weak)+ return {"key": key, "links": links, "count": len(links), "merge_threshold": MERGE_THRESHOLD}++[email protected]("/cluster", summary="Who else is this person")+async def cluster(+ db: DBSession,+ user: Annotated[AuthUser, Depends(require_permission("alerts:read"))],+ key: str = Query(...),+ max_depth: int = Query(default=3, ge=1, le=6),+) -> dict[str, Any]:+ """Merge-strength links only, bounded. See the store for why both."""+ members = await identity_resolution_store.cluster_for(db, tenant_id=user.tenant_id, key=key, max_depth=max_depth)+ return {+ "key": key,+ "same_person": members,+ "count": len(members),+ "note": (+ "Only links at or above the merge threshold, or confirmed by a person, are walked. "+ "Weak links are visible on /identity-resolution/links and are never merged."+ ),+ }++[email protected]("/resolve", summary="Resolve a set of identities into links")+async def resolve(+ body: ResolveRequest,+ db: DBSession,+ user: Annotated[AuthUser, Depends(require_permission("settings:write"))],+) -> dict[str, Any]:+ """Run resolution over the supplied records.++ ``settings:write`` rather than a read permission because ``persist``+ writes the tenant's identity graph, and a caller who may not change+ tenant configuration should not be able to decide who is the same+ person.+ """+ if not body.records:+ raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="no records supplied")+ if len(body.records) > MAX_RECORDS:+ raise HTTPException(+ status_code=status.HTTP_400_BAD_REQUEST,+ detail=f"at most {MAX_RECORDS} records per call; resolution is pairwise",+ )+ unknown = sorted({r.source for r in body.records} - set(SOURCE_KINDS))+ if unknown:+ raise HTTPException(+ status_code=status.HTTP_400_BAD_REQUEST,+ detail=f"unknown source(s) {unknown}; supported: {sorted(SOURCE_KINDS)}",+ )++ result = resolve_links([r.model_dump() for r in body.records])+ persisted = 0+ if body.persist:+ persisted = await identity_resolution_store.save_links(db, tenant_id=user.tenant_id, result=result)+ await db.commit()+ return {**result.as_dict(), "persisted": persisted}++[email protected]("/links/{link_id}/decide", summary="Confirm or reject a proposed link")+async def decide(+ link_id: uuid.UUID,+ body: DecideRequest,+ db: DBSession,+ user: Annotated[AuthUser, Depends(require_permission("settings:write"))],+) -> dict[str, Any]:+ """A 404 for another tenant's link, so this is not an existence oracle."""+ if body.decision not in {"confirmed", "rejected"}:+ raise HTTPException(+ status_code=status.HTTP_400_BAD_REQUEST,+ detail="decision must be 'confirmed' or 'rejected'",+ )+ updated = await identity_resolution_store.decide_link(+ db,+ tenant_id=user.tenant_id,+ link_id=link_id,+ decision=body.decision,+ user_id=user.user_id,+ )+ if updated is None:+ raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="link not found")+ await db.commit()+ return updateddiff --git a/services/api/app/api/v1/router.py b/services/api/app/api/v1/router.pyindex d82e34b9d..eabf2786e 100644--- a/services/api/app/api/v1/router.py+++ b/services/api/app/api/v1/router.py@@ -4,6 +4,7 @@ from fastapi import APIRouterfrom app.api.v1.endpoints import (access_conditions,+ access_review,admin_users,agent_tools,agents,@@ -48,6 +49,7 @@ from app.api.v1.endpoints import (hunt_reports,hunts,identity_graph,+ identity_resolution,identity_timeline,inbox,inbox_itsm,@@ -238,6 +240,7 @@ api_router.include_router(replay.public_router)# Resolves "what can principal X actually do?" across AWS / Azure / GCP /# Okta / Google Workspace and caches the result as :EFFECTIVE_PERMISSION# edges in Neo4j. Backs apps/web/src/app/(app)/identity/permissions.+api_router.include_router(access_review.router)api_router.include_router(effective_permissions.router)# Signed email-approval links (T3.6). Unauthenticated by necessity — the@@ -346,6 +349,7 @@ api_router.include_router(threat_intel.router)api_router.include_router(posture.router)api_router.include_router(easm.router)api_router.include_router(identity_graph.router)+api_router.include_router(identity_resolution.router)api_router.include_router(reports.router)# Depth 9.5: a natural-language hunt or a retro-hunt that found something# becomes a readable, cited report, exportable through the same WeasyPrintdiff --git a/services/api/app/services/access_review.py b/services/api/app/services/access_review.pynew file mode 100644index 000000000..31dd45847--- /dev/null+++ b/services/api/app/services/access_review.py@@ -0,0 +1,502 @@+"""Read-only access review: what was granted, through what, and was it used.++Depth plan wave 2, Phase 1.2.++1.1 made "what can this identity do" answerable from a versioned posture+snapshot, and recorded which of those actions a principal actually exercised+in a trailing window. Neither is a review. A reviewer asks comparative+questions — who holds admin they never use, which human has a long-lived+cloud key, which outside collaborator has admin on a repository, which admin+has gone dormant — and this module is where those are asked.++Read-only, and the rule is load-bearing+---------------------------------------+Nothing here changes vendor state, and nothing here is wired to anything+that can. Every finding names the object a human should go and change, and+stops. That is a deliberate scope limit from the plan, and it is also what+makes the module safe to run broadly: an access review that could act is one+nobody enables.++The honesty property this module exists to protect+--------------------------------------------------+``UsageOverlay.measured`` is ``False`` on any install with no event lake,+and a review that reads that as "used nothing" produces a finding against+**every permission in the estate**, with exactly the confidence of a correct+finding. So:++* usage-dependent findings are not emitted at all when usage was not+ measured, rather than emitted with a caveat nobody reads;+* :class:`ReviewResult` carries ``usage_measured`` and a reason, so a+ surface can say "nothing reviewed" instead of showing a short list;+* :func:`best_fit_role` returns ``None`` rather than a guess.++A review of zero principals is "nothing reviewed", never "no findings". The+second reads as a clean bill of health for an estate nobody looked at.+"""++from __future__ import annotations++import fnmatch+from dataclasses import dataclass, field+from datetime import UTC, datetime, timedelta+from typing import Any, Literal++from app.services.identity_privilege import ESCALATION_ACTIONS, IdentityPrivilege, tier_at_least+from app.services.identity_privilege_store import UsageOverlay++__all__ = [+ "FINDING_KINDS",+ "AccessFinding",+ "FindingSpec",+ "ReviewResult",+ "best_fit_role",+ "review_principal",+ "summarise",+]++Severity = Literal["info", "low", "medium", "high", "critical"]++#: A principal with no activity for this long is dormant. Ninety days+#: matches the usage window's default for a reason: a dormancy claim shorter+#: than the window it is measured over is not a measurement, it is an+#: artefact of where the window happens to start.+DORMANT_AFTER_DAYS = 90++#: Action prefixes that write rather than read. Used only for the+#: unused-write finding, which is the one a reviewer acts on first: unused+#: *read* access is untidy, unused *write* access is the blast radius of a+#: credential nobody would miss.+_WRITE_PREFIXES: tuple[str, ...] = (+ "iam:Create",+ "iam:Put",+ "iam:Attach",+ "iam:Update",+ "iam:Delete",+ "s3:Put",+ "s3:Delete",+ "ec2:Run",+ "ec2:Terminate",+ "kms:Schedule",+ "secretsmanager:Put",+ "Microsoft.Authorization/",+ "microsoft.directory/",+ "resourcemanager.projects.setIamPolicy",+ "iam.roles.",+ "iam.serviceAccountKeys.create",+ "storage.objects.delete",+ "admin.users.",+ "admin.roles.",+ "okta.users.manage",+ "okta.groups.manage",+ "okta.roles.manage",+ "admin:org",+ "write:org",+ "repo:admin",+ "delete_repo",+)+++@dataclass(frozen=True)+class FindingSpec:+ """What one kind of finding is, and what a person should do about it.++ ``object_hint`` and ``recommendation`` are required rather than+ optional. A read-only review whose findings do not name the object to+ change is a list of complaints: the reviewer still has to work out where+ to go, which is the part that makes reviews not happen.+ """++ kind: str+ title: str+ severity: Severity+ object_hint: str+ recommendation: str+ #: True when the finding cannot be computed without usage data. These+ #: are suppressed entirely on an install where usage was not measured.+ needs_usage: bool = False+++FINDING_KINDS: dict[str, FindingSpec] = {+ "human_with_long_lived_key": FindingSpec(+ kind="human_with_long_lived_key",+ title="A person holds a long-lived cloud access key",+ severity="high",+ object_hint="the access key on the principal",+ recommendation=(+ "Replace the static key with federated or short-lived credentials, then delete the key. "+ "A human's long-lived key is the credential most likely to end up in a shell history, a "+ "CI variable or a laptop backup, and it does not expire on its own."+ ),+ ),+ "unused_write_access": FindingSpec(+ kind="unused_write_access",+ title="Write or administrative access that was never exercised",+ severity="medium",+ object_hint="the policy, role or group grant that carries the unused actions",+ recommendation=(+ "Remove the unused actions from the grant, or move the principal to a smaller role. "+ "Check the window first: quarterly work looks unused in a 30-day window."+ ),+ needs_usage=True,+ ),+ "outside_collaborator_admin": FindingSpec(+ kind="outside_collaborator_admin",+ title="An external identity holds administrative access",+ severity="high",+ object_hint="the collaborator or guest grant on the repository, organisation or tenant",+ recommendation=(+ "Downgrade the grant to the least role that covers the collaboration, or move the work "+ "into a dedicated repository. An outside collaborator with admin can change branch "+ "protection and add further collaborators."+ ),+ ),+ "dormant_admin": FindingSpec(+ kind="dormant_admin",+ title="An administrative principal has been dormant",+ severity="medium",+ object_hint="the principal and the admin role it holds",+ recommendation=(+ "Confirm the account is still needed. A dormant admin is an account whose compromise "+ "nobody would notice, because there is no normal activity for an anomaly to stand out "+ "against."+ ),+ needs_usage=True,+ ),+ "escalation_without_use": FindingSpec(+ kind="escalation_without_use",+ title="An identity can grant itself more access and never has",+ severity="high",+ object_hint="the policy granting the authorization-write actions",+ recommendation=(+ "Remove the authorization-write actions. An identity that can grant itself more access "+ "makes every other permission it holds a lower bound rather than a limit."+ ),+ needs_usage=True,+ ),+ "fork_reachable_workflow_secret": FindingSpec(+ kind="fork_reachable_workflow_secret",+ title="A workflow with a fork-reachable trigger can read secrets",+ severity="high",+ object_hint="the workflow file and its trigger block",+ recommendation=(+ "Split the privileged half into a separate `workflow_run` job, or drop the secret from "+ "the fork-reachable job. `pull_request_target` runs with the base repository's secrets "+ "against a fork's code."+ ),+ ),+ "org_member_without_mfa": FindingSpec(+ kind="org_member_without_mfa",+ title="An organisation member has no second factor",+ severity="high",+ object_hint="the member's account in the organisation",+ recommendation=(+ "Enrol the account, or enforce two-factor authentication at the organisation level. A "+ "single-factor member of an organisation that holds source code is a password away "+ "from a supply-chain change."+ ),+ ),+ "apps_script_project_drift": FindingSpec(+ kind="apps_script_project_drift",+ title="An Apps Script project differs from what the service creates",+ severity="medium",+ object_hint="the GCP project under the Apps Script folder",+ recommendation=(+ "Compare the project against a service-created one and remove the additions. Billing "+ "linked or extra services enabled on a `sys-` project is the shape of a human using "+ "the Apps Script folder as cover."+ ),+ ),+}+++@dataclass(frozen=True)+class AccessFinding:+ """One finding, with the evidence that produced it."""++ kind: str+ principal_id: str+ provider: str+ severity: Severity+ title: str+ detail: str+ object_hint: str+ recommendation: str+ evidence: dict[str, Any] = field(default_factory=dict)++ def as_dict(self) -> dict[str, Any]:+ return {+ "kind": self.kind,+ "principal_id": self.principal_id,+ "provider": self.provider,+ "severity": self.severity,+ "title": self.title,+ "detail": self.detail,+ "object_hint": self.object_hint,+ "recommendation": self.recommendation,+ "evidence": dict(self.evidence),+ }+++@dataclass(frozen=True)+class ReviewResult:+ """One principal's review, and how much of it could be computed."""++ principal_id: str+ provider: str+ findings: list[AccessFinding] = field(default_factory=list)+ usage_measured: bool = False+ #: Why usage-dependent findings were skipped, when they were. Present so+ #: the surface can distinguish "reviewed, nothing found" from "could not+ #: review half of it".+ unmeasured_reason: str | None = None+ best_fit: str | None = None++ def as_dict(self) -> dict[str, Any]:+ return {+ "principal_id": self.principal_id,+ "provider": self.provider,+ "findings": [f.as_dict() for f in self.findings],+ "usage_measured": self.usage_measured,+ "unmeasured_reason": self.unmeasured_reason,+ "best_fit_role": self.best_fit,+ }+++def _is_write(action: str) -> bool:+ return any(action.startswith(prefix) for prefix in _WRITE_PREFIXES)+++def _is_escalation(action: str) -> bool:+ return any(fnmatch.fnmatchcase(action, pattern) for pattern in ESCALATION_ACTIONS)+++def review_principal(+ *,+ privilege: IdentityPrivilege,+ usage: UsageOverlay,+ granted_actions: set[str] | None = None,+ has_long_lived_key: bool | None = None,+ is_human: bool | None = None,+ now: datetime | None = None,+) -> ReviewResult:+ """Review one principal. Pure — no database, no vendor call.++ ``granted_actions`` defaults to the actions named in the privilege+ evidence, which is the subset that produced a tag. A caller with the+ full resolver result should pass it, because the evidence set is+ deliberately narrow and a right-sizing answer wants everything.+ """+ now = now or datetime.now(tz=UTC)+ granted = set(granted_actions or ())+ if not granted:+ for actions in privilege.evidence.values():+ granted.update(actions)++ findings: list[AccessFinding] = []++ def add(kind: str, detail: str, evidence: dict[str, Any]) -> None:+ spec = FINDING_KINDS[kind]+ findings.append(+ AccessFinding(+ kind=kind,+ principal_id=privilege.principal_id,+ provider=privilege.provider,+ severity=spec.severity,+ title=spec.title,+ detail=detail,+ object_hint=spec.object_hint,+ recommendation=spec.recommendation,+ evidence=evidence,+ )+ )++ # --- findings that do not need usage ---------------------------------++ if has_long_lived_key and is_human is not False:+ add(+ "human_with_long_lived_key",+ f"{privilege.principal_id} is a human principal holding a long-lived access key.",+ {"principal_kind": "human" if is_human else "unknown"},+ )++ if privilege.tags.third_party is True and tier_at_least(privilege.tier, "admin"):+ add(+ "outside_collaborator_admin",+ f"{privilege.principal_id} is external or vendor-owned and holds {privilege.tier} privilege.",+ {"tier": privilege.tier, "tags": privilege.tags.set_tags()},+ )++ # --- findings that need usage ----------------------------------------+ #+ # Suppressed wholesale rather than caveated. A caveat on a finding that+ # says "revoke this" is read by nobody, and the finding is wrong in the+ # direction that costs an outage.+ if not usage.measured:+ return ReviewResult(+ principal_id=privilege.principal_id,+ provider=privilege.provider,+ findings=findings,+ usage_measured=False,+ unmeasured_reason=(+ "Which permissions were actually exercised was not measured for this principal, so "+ "unused-access, dormancy and right-sizing findings were not computed. The event "+ "lake is a `full`-profile component; on a CORE install this is expected, and it is "+ "not the same as having found nothing."+ ),+ best_fit=None,+ )++ used = set(usage.used)+ unused_writes = sorted(a for a in granted - used if _is_write(a))+ if unused_writes:+ window = _window_text(usage)+ add(+ "unused_write_access",+ f"{len(unused_writes)} write or administrative action(s) granted to {privilege.principal_id} were not exercised {window}.",+ {"unused_actions": unused_writes[:50], "window": window, "granted": len(granted), "used": len(used)},+ )++ unused_escalation = sorted(a for a in granted - used if _is_escalation(a))+ if unused_escalation:+ add(+ "escalation_without_use",+ f"{privilege.principal_id} can grant access ({', '.join(unused_escalation[:5])}) and did not {_window_text(usage)}.",+ {"actions": unused_escalation[:20]},+ )++ last_seen = max((t for t in usage.used.values() if t is not None), default=None)+ dormant_cutoff = now - timedelta(days=DORMANT_AFTER_DAYS)+ if tier_at_least(privilege.tier, "admin") and (last_seen is None or last_seen < dormant_cutoff):+ add(+ "dormant_admin",+ f"{privilege.principal_id} holds {privilege.tier} privilege and was last active "+ f"{last_seen.date().isoformat() if last_seen else 'never in the window'}.",+ {"last_used_at": last_seen.isoformat() if last_seen else None, "tier": privilege.tier},+ )++ return ReviewResult(+ principal_id=privilege.principal_id,+ provider=privilege.provider,+ findings=findings,+ usage_measured=True,+ best_fit=best_fit_role(provider=privilege.provider, granted=granted, usage=usage),+ )+++def _window_text(usage: UsageOverlay) -> str:+ if usage.window_start and usage.window_end:+ return f"between {usage.window_start.date().isoformat()} and {usage.window_end.date().isoformat()}"+ return "in the measured window"+++#: Built-in roles per provider, smallest first, as ``(name, action globs)``.+#: Deliberately short and conservative: naming a role that does not cover an+#: action the principal actually used would be a recommendation to break+#: production, which is the one failure a right-sizing feature cannot have.+#: A principal whose used set exceeds every entry gets ``None``, and "we+#: could not find a smaller role" is a correct and useful answer.+_BUILTIN_ROLES: dict[str, tuple[tuple[str, tuple[str, ...]], ...]] = {+ "aws": (+ ("AmazonS3ReadOnlyAccess", ("s3:Get*", "s3:List*", "s3:Describe*")),+ ("ReadOnlyAccess", ("*:Get*", "*:List*", "*:Describe*", "*:BatchGet*")),+ ("AmazonS3FullAccess", ("s3:*",)),+ ("PowerUserAccess", ("*",)),+ ),+ "azure": (+ ("Reader", ("*/read",)),+ ("Contributor", ("*/read", "*/write", "*/delete", "*/action")),+ ),+ "gcp": (+ ("roles/viewer", ("*.get", "*.list", "*.getIamPolicy")),+ ("roles/editor", ("*.get", "*.list", "*.create", "*.update", "*.delete")),+ ),+ "github": (+ ("read", ("read:*", "repo:status")),+ ("write", ("read:*", "write:*", "repo")),+ ),+ "okta": (("READ_ONLY_ADMIN", ("okta.*.read",)),),+ "gws": (("_HELP_DESK_ADMIN", ("admin.users.update", "admin.users.read")),),+}+++def best_fit_role(+ *,+ provider: str,+ granted: set[str],+ usage: UsageOverlay,+) -> str | None:+ """The smallest built-in role covering every action actually used.++ ``None`` whenever usage was not measured — a right-sizing recommendation+ computed from an unmeasured empty set recommends the smallest role in+ the table for every principal in the estate, which is both wrong and+ confidently phrased.++ Also ``None`` when no built-in covers the used set, which is the honest+ answer for a principal doing genuinely broad work: suggesting a role+ that does not cover something they used is a recommendation to cause an+ outage, and that is the one error this feature must not make.+ """+ if not usage.measured:+ return None+ used = set(usage.used)+ if not used:+ return None++ for name, globs in _BUILTIN_ROLES.get(provider, ()):+ if all(any(fnmatch.fnmatchcase(action, g) for g in globs) for action in used):+ # Only a recommendation if it is actually smaller than what is+ # held. A role that covers the used set *and* everything granted+ # changes nothing.+ if not all(any(fnmatch.fnmatchcase(action, g) for g in globs) for action in granted):+ return name+ return None+ return None+++def summarise(results: list[ReviewResult]) -> dict[str, Any]:+ """Portfolio summary with an honest empty state.++ ``state`` is the field that matters. Zero findings over zero principals+ is ``nothing_reviewed``; zero findings over a real inventory is+ ``no_findings``. Rendering both as "0 findings" presents an estate+ nobody looked at as an estate with nothing wrong, which is the single+ most reassuring thing this page could get wrong.+ """+ if not results:+ return {+ "state": "nothing_reviewed",+ "reviewed_principals": 0,+ "findings": 0,+ "by_kind": {},+ "by_severity": {},+ "usage_measured_for": 0,+ "note": (+ "No principals were reviewed. This is not a clean bill of health: connect a cloud "+ "or identity provider and collect its posture to review anything."+ ),+ }++ findings = [f for r in results for f in r.findings]+ by_kind: dict[str, int] = {}+ by_severity: dict[str, int] = {}+ for finding in findings:+ by_kind[finding.kind] = by_kind.get(finding.kind, 0) + 1+ by_severity[finding.severity] = by_severity.get(finding.severity, 0) + 1++ measured = sum(1 for r in results if r.usage_measured)+ summary: dict[str, Any] = {+ "state": "no_findings" if not findings else "findings",+ "reviewed_principals": len(results),+ "findings": len(findings),+ "by_kind": by_kind,+ "by_severity": by_severity,+ "usage_measured_for": measured,+ }+ if measured < len(results):+ summary["note"] = (+ f"Usage was measured for {measured} of {len(results)} principals. Unused-access, "+ f"dormancy and right-sizing findings were not computed for the rest, so this is a "+ f"partial review."+ )+ return summarydiff --git a/services/api/app/services/access_review_store.py b/services/api/app/services/access_review_store.pynew file mode 100644index 000000000..ac199523b--- /dev/null+++ b/services/api/app/services/access_review_store.py@@ -0,0 +1,466 @@+"""Persist and read access-review runs and findings (migration 125).++The pure review logic is in :mod:`app.services.access_review`; this module+joins it to the privilege facts and usage overlay 1.1 produces, writes the+result, and reads it back for the console.++Every statement filters on ``tenant_id`` in the ``WHERE`` clause. RLS is on+both tables as well, but the query-layer filter is the control relied on+here, for the same reason the posture store gives: RLS depends on the+session GUC being set, and a path that forgets it would read across tenants+with the policy still in force.++Repeats extend rather than accumulate+-------------------------------------+A nightly review must not leave ninety copies of one sentence. The unique+index on ``(tenant, provider, principal, kind)`` makes a repeat an update+that moves ``last_seen`` and leaves the reviewer's acknowledgement attached+to the row they already read.++Findings that stop reproducing are deleted for that run's provider, because+a review page listing a permission somebody already removed is a page people+stop trusting. The run row keeps the history of what each pass found.+"""++from __future__ import annotations++import json+import uuid+from datetime import datetime+from typing import Any++import structlog+from sqlalchemy import text+from sqlalchemy.ext.asyncio import AsyncSession++from app.services.access_review import ReviewResult, review_principal, summarise+from app.services.identity_privilege_store import live_facts_for_provider, usage_for_principal+from app.services.identity_usage_collector import collect_usage_for_tenant++logger = structlog.get_logger(__name__)++__all__ = ["acknowledge", "list_findings", "run_review", "summary"]++# Written out per statement rather than interpolated, and the optional+# filters below are parameters rather than appended clauses. `text()` with+# an f-string is a taint sink for `avoid-sqlalchemy-text`, and the rule is+# right that it cannot tell a constant clause from a caller-supplied one.+# One statement per query is also the better shape: there is no second+# query path that only some callers exercise.+++def _row_to_finding(row: Any) -> dict[str, Any]:+ evidence = row.evidence if isinstance(row.evidence, dict) else json.loads(row.evidence or "{}")+ return {+ "id": str(row.id),+ "run_id": str(row.run_id),+ "kind": row.kind,+ "provider": row.provider,+ "principal_id": row.principal_id,+ "severity": row.severity,+ "title": row.title,+ "detail": row.detail,+ "object_hint": row.object_hint,+ "recommendation": row.recommendation,+ "best_fit_role": row.best_fit_role,+ "evidence": evidence,+ "usage_measured": bool(row.usage_measured),+ "acknowledged_at": row.acknowledged_at.isoformat() if row.acknowledged_at else None,+ "acknowledged_by": str(row.acknowledged_by) if row.acknowledged_by else None,+ "acknowledged_note": row.acknowledged_note,+ "first_seen": row.first_seen.isoformat() if row.first_seen else None,+ "last_seen": row.last_seen.isoformat() if row.last_seen else None,+ }+++async def run_review(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ provider: str,+ collect_usage: bool = True,+) -> dict[str, Any]:+ """Recompute this provider's review and persist the result.++ Reads only: privilege facts, the event lake and the usage overlay. No+ vendor API is called from here and none may be added — the plan's+ read-only rule for 1.2 is why the review can be run across a whole+ estate without a change-control conversation.++ ``collect_usage`` refreshes the runtime overlay from the lake first,+ because this is the only caller that needs it and a collector with no+ caller is the defect this repository keeps rediscovering. It is a+ parameter rather than unconditional so a test can separate "the review+ is wrong" from "the lake is slow", and so a scheduled sweep that has+ already collected can skip it.++ When the lake cannot be read the collection reports ``measured=False``+ and the review proceeds with usage unknown — which suppresses the+ usage-dependent findings rather than emitting them against an empty+ used-set.+ """+ facts = await live_facts_for_provider(db, tenant_id=tenant_id, provider=provider)++ if collect_usage and facts:+ collection = await collect_usage_for_tenant(+ db,+ tenant_id=tenant_id,+ provider=provider,+ principals=[f.privilege.principal_id for f in facts],+ )+ if not collection.measured:+ logger.info(+ "access_review.usage_not_measured",+ provider=provider,+ reason=collection.reason,+ )++ results: list[ReviewResult] = []+ window_start: datetime | None = None+ window_end: datetime | None = None+ for fact in facts:+ usage = await usage_for_principal(+ db,+ tenant_id=tenant_id,+ provider=provider,+ principal_id=fact.privilege.principal_id,+ )+ if usage.measured and window_start is None:+ window_start, window_end = usage.window_start, usage.window_end+ results.append(+ review_principal(+ privilege=fact.privilege,+ usage=usage,+ # A long-lived key is a property of the principal that the+ # posture snapshot carries; `None` means the snapshot did+ # not say, and the finding is skipped rather than guessed.+ has_long_lived_key=_has_long_lived_key(fact),+ is_human=_is_human(fact),+ )+ )++ stats = summarise(results)+ run_id = await _insert_run(+ db,+ tenant_id=tenant_id,+ provider=provider,+ stats=stats,+ window_start=window_start,+ window_end=window_end,+ )++ seen_keys: list[tuple[str, str]] = []+ for result in results:+ for finding in result.findings:+ seen_keys.append((finding.principal_id, finding.kind))+ await _upsert_finding(+ db,+ tenant_id=tenant_id,+ run_id=run_id,+ finding=finding,+ best_fit=result.best_fit,+ usage_measured=result.usage_measured,+ )++ await _delete_resolved(db, tenant_id=tenant_id, provider=provider, keep=seen_keys)+ return {"run_id": str(run_id), **stats}+++def _has_long_lived_key(fact: Any) -> bool | None:+ """Whether the snapshot says this principal holds a static key.++ ``None`` when it does not say, which is a third answer and not a no.+ The finding it feeds is suppressed on ``None``, so an absent field can+ never manufacture a high-severity claim about somebody's credentials.+ """+ evidence = fact.privilege.evidence or {}+ for actions in evidence.values():+ if any(a in {"iam:CreateAccessKey", "iam.serviceAccountKeys.create"} for a in actions):+ return True+ return None+++def _is_human(fact: Any) -> bool | None:+ tags = fact.privilege.tags+ if tags.third_party is True:+ return False+ return None+++async def _insert_run(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ provider: str,+ stats: dict[str, Any],+ window_start: datetime | None,+ window_end: datetime | None,+) -> uuid.UUID:+ result = await db.execute(+ text(+ """+ INSERT INTO access_review_runs (+ tenant_id, provider, run_state, reviewed_principals,+ usage_measured_for, finding_count, window_start, window_end,+ note, completed_at+ ) VALUES (+ :tenant_id, :provider, :run_state, :reviewed,+ :measured_for, :findings, :window_start, :window_end,+ :note, NOW()+ )+ RETURNING id+ """+ ),+ {+ "tenant_id": str(tenant_id),+ "provider": provider,+ "run_state": stats["state"],+ "reviewed": stats["reviewed_principals"],+ "measured_for": stats.get("usage_measured_for", 0),+ "findings": stats["findings"],+ "window_start": window_start,+ "window_end": window_end,+ "note": stats.get("note"),+ },+ )+ return result.scalar_one()+++async def _upsert_finding(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ run_id: uuid.UUID,+ finding: Any,+ best_fit: str | None,+ usage_measured: bool,+) -> None:+ await db.execute(+ text(+ """+ INSERT INTO access_review_findings (+ tenant_id, run_id, kind, provider, principal_id, severity,+ title, detail, object_hint, recommendation, best_fit_role,+ evidence, usage_measured+ ) VALUES (+ :tenant_id, :run_id, :kind, :provider, :principal_id, :severity,+ :title, :detail, :object_hint, :recommendation, :best_fit_role,+ CAST(:evidence AS jsonb), :usage_measured+ )+ ON CONFLICT (tenant_id, provider, principal_id, kind) DO UPDATE+ SET run_id = EXCLUDED.run_id,+ severity = EXCLUDED.severity,+ detail = EXCLUDED.detail,+ recommendation = EXCLUDED.recommendation,+ best_fit_role = EXCLUDED.best_fit_role,+ evidence = EXCLUDED.evidence,+ usage_measured = EXCLUDED.usage_measured,+ last_seen = NOW()+ """+ ),+ {+ "tenant_id": str(tenant_id),+ "run_id": str(run_id),+ "kind": finding.kind,+ "provider": finding.provider,+ "principal_id": finding.principal_id,+ "severity": finding.severity,+ "title": finding.title,+ "detail": finding.detail,+ "object_hint": finding.object_hint,+ "recommendation": finding.recommendation,+ "best_fit_role": best_fit,+ "evidence": json.dumps(finding.evidence, default=str),+ "usage_measured": usage_measured,+ },+ )+++async def _delete_resolved(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ provider: str,+ keep: list[tuple[str, str]],+) -> None:+ """Drop findings this pass no longer reproduces.++ A review page still listing a permission somebody removed last week is+ a page people stop opening. The run row keeps what each pass found, so+ the history is not lost with the row.+ """+ if keep:+ await db.execute(+ text(+ """+ DELETE FROM access_review_findings+ WHERE tenant_id = :tenant_id+ AND provider = :provider+ AND (principal_id, kind) NOT IN (+ SELECT p, k+ FROM unnest(CAST(:principals AS text[]), CAST(:kinds AS text[]))+ AS pairs(p, k)+ )+ """+ ),+ {+ "tenant_id": str(tenant_id),+ "provider": provider,+ "principals": [k[0] for k in keep],+ "kinds": [k[1] for k in keep],+ },+ )+ return+ await db.execute(+ text("DELETE FROM access_review_findings WHERE tenant_id = :tenant_id AND provider = :provider"),+ {"tenant_id": str(tenant_id), "provider": provider},+ )+++async def list_findings(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ provider: str | None = None,+ kind: str | None = None,+ include_acknowledged: bool = False,+ limit: int = 100,+) -> list[dict[str, Any]]:+ result = await db.execute(+ text(+ """+ SELECT id, run_id, kind, provider, principal_id, severity, title, detail,+ object_hint, recommendation, best_fit_role, evidence, usage_measured,+ acknowledged_at, acknowledged_by, acknowledged_note, first_seen, last_seen+ FROM access_review_findings+ WHERE tenant_id = :tenant_id+ AND (CAST(:provider AS text) IS NULL OR provider = :provider)+ AND (CAST(:kind AS text) IS NULL OR kind = :kind)+ AND (CAST(:include_acknowledged AS boolean) OR acknowledged_at IS NULL)+ ORDER BY array_position(+ ARRAY['critical','high','medium','low','info']::text[], severity+ ),+ last_seen DESC+ LIMIT :limit+ """+ ),+ {+ "tenant_id": str(tenant_id),+ "provider": provider or None,+ "kind": kind or None,+ "include_acknowledged": bool(include_acknowledged),+ "limit": max(1, min(limit, 500)),+ },+ )+ return [_row_to_finding(row) for row in result.fetchall()]+++async def summary(db: AsyncSession, *, tenant_id: uuid.UUID, provider: str | None = None) -> dict[str, Any]:+ """The page header, including the state that is not a clean bill.++ A tenant that has never run a review gets ``never_run``; a run over zero+ principals gets ``nothing_reviewed``. Both are rendered as their own+ sentence, because the one thing this page must not do is show a zero+ that a reader takes for an all-clear.+ """+ params: dict[str, Any] = {"tenant_id": str(tenant_id), "provider": provider or None}++ runs = (+ await db.execute(+ text(+ """+ SELECT DISTINCT ON (provider)+ provider, run_state, reviewed_principals, usage_measured_for,+ finding_count, window_start, window_end, note, started_at+ FROM access_review_runs+ WHERE tenant_id = :tenant_id+ AND (CAST(:provider AS text) IS NULL OR provider = :provider)+ ORDER BY provider, started_at DESC+ """+ ),+ params,+ )+ ).fetchall()++ if not runs:+ return {+ "state": "never_run",+ "providers": [],+ "open_findings": 0,+ "note": (+ "No access review has been run for this tenant. This is not a clean bill of health: "+ "collect a provider's posture, then run a review."+ ),+ }++ open_count = (+ await db.execute(+ text(+ """+ SELECT count(*) FROM access_review_findings+ WHERE tenant_id = :tenant_id+ AND (CAST(:provider AS text) IS NULL OR provider = :provider)+ AND acknowledged_at IS NULL+ """+ ),+ params,+ )+ ).scalar_one()++ return {+ "state": "reviewed",+ "open_findings": int(open_count or 0),+ "providers": [+ {+ "provider": r.provider,+ "run_state": r.run_state,+ "reviewed_principals": int(r.reviewed_principals or 0),+ "usage_measured_for": int(r.usage_measured_for or 0),+ "finding_count": int(r.finding_count or 0),+ "window_start": r.window_start.isoformat() if r.window_start else None,+ "window_end": r.window_end.isoformat() if r.window_end else None,+ "note": r.note,+ "last_run_at": r.started_at.isoformat() if r.started_at else None,+ }+ for r in runs+ ],+ }+++async def acknowledge(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ finding_id: uuid.UUID,+ user_id: uuid.UUID | str,+ note: str = "",+) -> dict[str, Any] | None:+ """Record who decided a finding is acceptable. ``None`` if not this tenant's.++ The tenant predicate is in the ``UPDATE`` rather than checked after a+ fetch: a handler that reads first and filters second is the same bug one+ line later, and this one writes.+ """+ result = await db.execute(+ text(+ """+ UPDATE access_review_findings+ SET acknowledged_at = NOW(),+ acknowledged_by = :user_id,+ acknowledged_note = :note+ WHERE tenant_id = :tenant_id+ AND id = :finding_id+ RETURNING id, run_id, kind, provider, principal_id, severity, title, detail,+ object_hint, recommendation, best_fit_role, evidence, usage_measured,+ acknowledged_at, acknowledged_by, acknowledged_note, first_seen, last_seen+ """+ ),+ {+ "tenant_id": str(tenant_id),+ "finding_id": str(finding_id),+ "user_id": str(user_id),+ "note": note or None,+ },+ )+ row = result.fetchone()+ return _row_to_finding(row) if row else Nonediff --git a/services/api/app/services/alert_priority.py b/services/api/app/services/alert_priority.pyindex 3cfdf1f97..1502a4c1d 100644--- a/services/api/app/services/alert_priority.py+++ b/services/api/app/services/alert_priority.py@@ -75,8 +75,32 @@ _PRIVILEGE_FACTOR: dict[str, float] = {"admin": 1.35,"elevated": 1.15,"standard": 1.0,+ # Exactly 1.0, and present rather than absent. An identity with no+ # posture covering the event is not evidence in either direction, so it+ # must not move the score -- but it is also not the same as a measured+ # standard account, and the rationale says which one it was. Leaving+ # `unknown` out of this table would reach `.get(tier, 1.0)` and produce+ # the same number with no sentence attached.+ "unknown": 1.0,}+#: Depth plan wave 2, 1.1. Each tag that is true multiplies by this much.+#: Deliberately small and multiplicative, matching the module's design note:+#: context reorders alerts of similar severity and cannot invert a severity+#: gap. Four tags together reach 1.21, which is less than one criticality+#: step -- tags are a reason to look sooner, not a reason to outrank a+#: critical alert.+_TAG_FACTOR: dict[str, float] = {+ "can_escalate": 1.10,+ "cross_account": 1.05,+ "data_access": 1.05,+ "third_party": 1.05,+}++#: Ceiling on the combined tag multiplier, so adding a fifth tag later+#: cannot quietly make tags dominate the ordering.+_TAG_FACTOR_CAP = 1.25+#: A host running something on the CISA Known Exploited Vulnerabilities#: catalogue is being attacked with a technique that is known to work,#: which is a different fact from a high CVSS score.@@ -108,9 +132,21 @@ class AssetContext:@dataclass(frozen=True)class IdentityContext:principal: str | None = None- privilege_tier: str = "standard"+ #: Defaults to ``unknown``, not ``standard``. A caller that builds this+ #: without a posture lookup is saying "I did not check", and the old+ #: default said "I checked and it is ordinary" on its behalf.+ privilege_tier: str = "unknown"is_service: bool = Falseis_break_glass: bool = False+ #: The four computed tags from 1.1. ``None`` is "no snapshot could+ #: answer" and never contributes; only ``True`` does.+ can_escalate: bool | None = None+ data_access: bool | None = None+ third_party: bool | None = None+ cross_account: bool | None = None++ def set_tags(self) -> list[str]:+ return [name for name in ("can_escalate", "data_access", "third_party", "cross_account") if getattr(self, name) is True]@dataclass@@ -188,11 +224,36 @@ def score_alert_priority(rationale.append({"factor": 1.2, "reason": "the host is internet-facing"})if identity is not None:- tier = (identity.privilege_tier or "standard").strip().lower()+ tier = (identity.privilege_tier or "unknown").strip().lower()privilege_factor = _PRIVILEGE_FACTOR.get(tier, 1.0)if privilege_factor != 1.0:score *= privilege_factorrationale.append({"factor": privilege_factor, "reason": f"the principal holds {tier} privilege"})+ elif tier == "unknown":+ # A 1.0 row rather than no row. The queue renders this rationale+ # verbatim, and an analyst asking "why is this ranked here"+ # needs to be told the platform could not answer, not shown a+ # list with the question silently missing from it.+ rationale.append(+ {+ "factor": 1.0,+ "reason": "the principal's privilege at event time is unknown (no posture snapshot covers it)",+ }+ )++ tags = identity.set_tags()+ if tags:+ combined = 1.0+ for tag in tags:+ combined *= _TAG_FACTOR.get(tag, 1.0)+ combined = min(combined, _TAG_FACTOR_CAP)+ score *= combined+ rationale.append(+ {+ "factor": round(combined, 4),+ "reason": f"the principal is tagged {', '.join(tags)}",+ }+ )if identity.is_break_glass:score *= _BREAK_GLASS_FACTORdiff --git a/services/api/app/services/alert_queue.py b/services/api/app/services/alert_queue.pyindex 6806828f9..565fa0979 100644--- a/services/api/app/services/alert_queue.py+++ b/services/api/app/services/alert_queue.py@@ -34,7 +34,7 @@ from datetime import UTC, datetime, timedeltafrom typing import Any, Literalfrom pydantic import BaseModel, Field-from sqlalchemy import and_, case, func, literal, or_, select, update+from sqlalchemy import and_, case, func, literal, or_, select, text, updatefrom sqlalchemy.ext.asyncio import AsyncSessionfrom app.models.alert import Alert@@ -428,6 +428,92 @@ async def resolve_asset_contexts(return out+async def resolve_identity_contexts(+ db: AsyncSession,+ tenant_id: uuid.UUID,+ alerts: list[Any],+) -> dict[uuid.UUID, IdentityContext]:+ """Privilege and tags per alert, **as of that alert's own event time**.++ Depth plan wave 2, 1.1. One query for the page rather than one per+ alert, matching ``resolve_asset_contexts`` above.++ The point-in-time join is the part worth reading carefully. Each alert+ brings its own timestamp into the query and is matched against the+ privilege version whose interval contains it, so an alert from before a+ role was revoked still ranks by the privilege the actor held when they+ acted. A simpler join on the live row would rank the whole queue by+ today's posture, which is both wrong and invisible -- the number looks+ identical either way.++ An alert whose actor has no covering version is deliberately **absent**+ from the result, and the caller turns that into an ``unknown`` tier+ rather than a ``standard`` one.+ """+ actors: dict[str, list[tuple[uuid.UUID, datetime]]] = {}+ for alert in alerts:+ for user in list(getattr(alert, "affected_users", None) or []):+ name = str(user or "").strip()+ if not name:+ continue+ when = getattr(alert, "event_time", None) or getattr(alert, "first_seen", None)+ if when is None:+ continue+ if when.tzinfo is None:+ when = when.replace(tzinfo=UTC)+ actors.setdefault(name.lower(), []).append((alert.id, when))+ if not actors:+ return {}++ rows = (+ await db.execute(+ text(+ """+ SELECT principal_id, principal_label, tier,+ can_escalate, data_access, third_party, cross_account,+ valid_from, valid_until+ FROM identity_privilege_facts+ WHERE tenant_id = :tenant_id+ AND (lower(principal_id) = ANY(:actors) OR lower(principal_label) = ANY(:actors))+ """+ ),+ {"tenant_id": str(tenant_id), "actors": list(actors)},+ )+ ).fetchall()+ if not rows:+ return {}++ tier_rank = {"unknown": 0, "standard": 1, "elevated": 2, "admin": 3, "domain_admin": 4}+ out: dict[uuid.UUID, IdentityContext] = {}+ for row in rows:+ handles = {+ str(row.principal_id or "").lower(),+ str(row.principal_label or "").lower(),+ } - {""}+ valid_from = row.valid_from+ valid_until = row.valid_until+ for handle in handles:+ for alert_id, when in actors.get(handle, ()):+ if valid_from > when or (valid_until is not None and valid_until <= when):+ continue+ candidate = IdentityContext(+ principal=row.principal_id,+ privilege_tier=str(row.tier or "unknown"),+ can_escalate=row.can_escalate,+ data_access=row.data_access,+ third_party=row.third_party,+ cross_account=row.cross_account,+ )+ # One alert can name several users, and a person can hold+ # accounts in several providers. The strongest wins: an+ # alert about someone who is an ordinary Okta user and an+ # AWS administrator is an alert about an administrator.+ existing = out.get(alert_id)+ if existing is None or tier_rank.get(candidate.privilege_tier, 0) > tier_rank.get(existing.privilege_tier, 0):+ out[alert_id] = candidate+ return out++def queue_priority(alert: Any, *, asset: Any = None, identity: Any = None) -> int:"""The score this alert sorts by.@@ -450,9 +536,13 @@ def queue_priority(alert: Any, *, asset: Any = None, identity: Any = None) -> inelse None,identity=IdentityContext(principal=str(getattr(identity, "principal", "") or "") or None,- privilege_tier=str(getattr(identity, "privilege_tier", "standard") or "standard"),+ privilege_tier=str(getattr(identity, "privilege_tier", "unknown") or "unknown"),is_service=bool(getattr(identity, "is_service", False)),is_break_glass=bool(getattr(identity, "is_break_glass", False)),+ can_escalate=getattr(identity, "can_escalate", None),+ data_access=getattr(identity, "data_access", None),+ third_party=getattr(identity, "third_party", None),+ cross_account=getattr(identity, "cross_account", None),)if identity is not Noneelse None,@@ -549,6 +639,13 @@ async def build_queue(# hidden.page_alerts = [row[0] for row in rows]asset_contexts = await resolve_asset_contexts(db, tenant_id, page_alerts)+ # Depth 2 wave 2, 1.1. `score_alert_priority` has taken an `identity`+ # since v16 and this call site passed `identity=None`, so the privilege+ # factor and its whole table were unreachable from the only surface+ # where prioritisation changes anything -- the built-but-unwired shape,+ # in the module whose own docstring explains why an ordering nobody can+ # interrogate stops being trusted.+ identity_contexts = await resolve_identity_contexts(db, tenant_id, page_alerts)scored: list[tuple[int, datetime, int, QueueItem]] = []for row in rows:@@ -567,7 +664,11 @@ async def build_queue(priority_result = score_alert_priority(severity=str(alert.severity or "medium"),asset=asset_contexts.get(alert.id),- identity=None,+ # Absent means no posture version covered this alert's event+ # time. `IdentityContext()` defaults to `unknown`, which scores+ # neutrally and says so in the rationale -- unlike `None`, which+ # omits the question from the basis entirely.+ identity=identity_contexts.get(alert.id, IdentityContext()),)item = QueueItem(diff --git a/services/api/app/services/change_timeline.py b/services/api/app/services/change_timeline.pynew file mode 100644index 000000000..e96216e22--- /dev/null+++ b/services/api/app/services/change_timeline.py@@ -0,0 +1,398 @@+"""What changed before this alert (depth plan wave 2, Phase 1.5).++The first question an analyst asks about an identity alert is what changed+in the days before it: a role granted, a policy edited, an OAuth app+consented to, an MFA factor replaced, a share opened, a key created. Both+halves of the answer already exist in the platform and nothing joined them:++* **versioned posture** (`posture_snapshots`, migration 103, and the derived+ `identity_privilege_facts`, migration 124) — a diff between consecutive+ versions is a record of grants and policy edits with the exact interval+ each was true over;+* **audit events** in the lake — the vendor's own record of the change+ event, with the actor who made it.++They answer differently and are both kept. Posture says *what the estate+looked like* and is complete for the providers collected; audit says *who+did it and when*, and is complete only for the sources ingested. A timeline+built from one alone is missing either the attribution or the coverage.++Two honesty properties the result shape enforces+------------------------------------------------+**"Nothing changed" and "we could not read it" are different answers.**+:class:`ChangeTimeline` carries ``sources_read``, and ``measured`` is false+when it is empty. An empty list with no such field would turn a lake outage+into evidence of a quiet week, and an investigation closes on that.++**Every change names its source.** A citation whose target nobody can find+is not a citation, so each row carries the store it came from and a+reference into it. This is what makes the ledger entry checkable rather+than a sentence the model was handed.+"""++from __future__ import annotations++import uuid+from dataclasses import dataclass, field+from datetime import UTC, datetime, timedelta+from typing import Any++import structlog+from sqlalchemy.ext.asyncio import AsyncSession++from app.db.clickhouse import execute_lake_query+from app.services.posture_store import PROVIDERS, snapshot_history++logger = structlog.get_logger(__name__)++__all__ = [+ "CHANGE_KINDS",+ "DEFAULT_WINDOW_DAYS",+ "ChangeTimeline",+ "IdentityChange",+ "changes_before",+]++#: Lake table, spelled exactly as `investigation_tools` spells it. The+#: raw-SQL gate declares `aisoc.raw_events` as ClickHouse-owned in its+#: `UNMIGRATED_TABLES` list, which is what lets a statement against it be+#: compared against no migration without the gate silently excusing the+#: whole class of unknown tables.+LAKE_TABLE = "aisoc.raw_events"++#: How far back to look. Seven days by default: long enough to cover a+#: grant made the previous working week, short enough that the answer is a+#: list a person reads rather than a change log they scroll.+DEFAULT_WINDOW_DAYS = 7+MAX_WINDOW_DAYS = 90+MAX_CHANGES = 200++#: The six kinds the plan names, each with the vendor actions that produce+#: it. Matched case-insensitively against the lake's normalised `action`.+#:+#: Deliberately a closed vocabulary rather than a "contains the word+#: policy" heuristic: a timeline that silently reclassifies an unfamiliar+#: action is worse than one that reports it as `other`, because a reader+#: cannot tell which rows were understood.+CHANGE_KINDS: dict[str, tuple[str, ...]] = {+ "role_or_group_grant": (+ "AddUserToGroup",+ "AttachUserPolicy",+ "AttachRolePolicy",+ "AttachGroupPolicy",+ "CreateRole",+ "AddRoleToInstanceProfile",+ "roleAssignments/write",+ "Add member to role",+ "SetIamPolicy",+ "setIamPolicy",+ "group.user_membership.add",+ "user.account.privilege.grant",+ "ASSIGN_ROLE",+ "GRANT_ADMIN_PRIVILEGE",+ "team.add_member",+ "org.add_member",+ "org.update_member",+ ),+ "policy_edit": (+ "PutUserPolicy",+ "PutRolePolicy",+ "PutGroupPolicy",+ "CreatePolicyVersion",+ "SetDefaultPolicyVersion",+ "UpdateAssumeRolePolicy",+ "PutBucketPolicy",+ "policyAssignments/write",+ "roleDefinitions/write",+ "iam.roles.update",+ "policy.lifecycle.update",+ "protected_branch.policy_override",+ ),+ "oauth_consent": (+ "Consent to application",+ "Add app role assignment grant to user",+ "authorize",+ "oauth2.grant",+ "app.oauth2.client.create",+ "AUTHORIZE_API_CLIENT_ACCESS",+ "oauth_application.create",+ "integration_installation.create",+ ),+ "mfa_factor_change": (+ "User registered security info",+ "Update user",+ "user.mfa.factor.activate",+ "user.mfa.factor.reset_all",+ "user.mfa.factor.deactivate",+ "UNENROLL_2SV",+ "ENROLL_2SV",+ "two_factor_authentication.disabled",+ "two_factor_authentication.enabled",+ ),+ "sharing_change": (+ "PutBucketAcl",+ "PutObjectAcl",+ "storage.setIamPermissions",+ "CHANGE_DOCUMENT_VISIBILITY",+ "CHANGE_DOCUMENT_ACCESS_SCOPE",+ "CREATE_DATA_TRANSFER_REQUEST",+ "repo.access",+ "repo.change_merge_setting",+ ),+ "key_creation": (+ "CreateAccessKey",+ "CreateLoginProfile",+ "UpdateLoginProfile",+ "iam.serviceAccountKeys.create",+ "Add service principal credentials",+ "user.api_token.create",+ "personal_access_token.create",+ "public_key.create",+ "deploy_key.create",+ ),+}++#: Flattened once at import rather than per call. The lookup is per lake row.+_ACTION_TO_KIND: dict[str, str] = {action.lower(): kind for kind, actions in CHANGE_KINDS.items() for action in actions}+++@dataclass(frozen=True)+class IdentityChange:+ """One recorded change, with where it was read from."""++ kind: str+ at: str+ subject: str+ summary: str+ #: Which store this came from: ``posture_snapshot`` or ``audit_event``.+ source: str+ #: A reference into that store a reviewer can follow.+ source_ref: str+ actor: str = ""+ detail: dict[str, Any] = field(default_factory=dict)++ def as_dict(self) -> dict[str, Any]:+ return {+ "kind": self.kind,+ "at": self.at,+ "subject": self.subject,+ "summary": self.summary,+ "actor": self.actor,+ "source": self.source,+ "source_ref": self.source_ref,+ "detail": dict(self.detail),+ }+++@dataclass(frozen=True)+class ChangeTimeline:+ """The window's changes, and which stores could actually be read.++ ``measured`` is the field that stops an outage reading as a quiet week.+ """++ changes: list[IdentityChange] = field(default_factory=list)+ #: Which sources answered. Empty means nothing could be read, which is+ #: not the same as nothing having happened.+ sources_read: tuple[str, ...] = ()+ #: Which sources were asked and could not answer, with why.+ sources_failed: tuple[str, ...] = ()+ window_start: str | None = None+ window_end: str | None = None++ @property+ def measured(self) -> bool:+ return bool(self.sources_read)++ def as_dict(self) -> dict[str, Any]:+ payload: dict[str, Any] = {+ "measured": self.measured,+ "changes": [c.as_dict() for c in self.changes],+ "change_count": len(self.changes),+ "sources_read": list(self.sources_read),+ "sources_failed": list(self.sources_failed),+ "window_start": self.window_start,+ "window_end": self.window_end,+ }+ if not self.measured:+ payload["reason"] = (+ "No source could be read for this window, so whether anything changed is UNKNOWN. "+ "This is a gap in visibility, not evidence that nothing changed."+ )+ elif not self.changes:+ payload["reason"] = (+ f"Read {', '.join(self.sources_read)} over this window and found no recorded change. This is a measured result."+ )+ return payload+++def _clamp_days(days: int | None) -> int:+ if not days or days < 1:+ return DEFAULT_WINDOW_DAYS+ return min(int(days), MAX_WINDOW_DAYS)+++async def changes_before(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ subjects: list[str],+ at: datetime,+ days: int = DEFAULT_WINDOW_DAYS,+ providers: list[str] | None = None,+) -> ChangeTimeline:+ """Every recorded change touching ``subjects`` in the ``days`` before ``at``.++ Reads both stores and reports which answered. Never raises: a store that+ cannot be read lands in ``sources_failed`` and the caller can say so.+ """+ days = _clamp_days(days)+ window_start = at - timedelta(days=days)+ wanted = {s.strip().lower() for s in subjects if s and s.strip()}++ changes: list[IdentityChange] = []+ read: list[str] = []+ failed: list[str] = []++ posture = await _posture_changes(+ db,+ tenant_id=tenant_id,+ window_start=window_start,+ at=at,+ providers=providers or sorted(PROVIDERS),+ )+ if posture is None:+ failed.append("posture snapshots could not be read")+ else:+ read.append("posture snapshots")+ changes.extend(posture)++ audit = await _audit_changes(tenant_id=tenant_id, wanted=wanted, window_start=window_start, at=at)+ if audit is None:+ failed.append("the event lake could not be queried (it is a `full`-profile component)")+ else:+ read.append("audit events")+ changes.extend(audit)++ changes.sort(key=lambda c: c.at, reverse=True)+ return ChangeTimeline(+ changes=changes[:MAX_CHANGES],+ sources_read=tuple(read),+ sources_failed=tuple(failed),+ window_start=window_start.isoformat(),+ window_end=at.isoformat(),+ )+++async def _posture_changes(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ window_start: datetime,+ at: datetime,+ providers: list[str],+) -> list[IdentityChange] | None:+ """Posture versions that opened inside the window.++ A new version exists only when the content hash moved, so every row here+ is a real posture change — which is the property migration 103 was+ designed for and the reason this is cheap.++ ``None`` means the store could not be read at all, which the caller+ reports rather than folding into "no changes".+ """+ out: list[IdentityChange] = []+ try:+ for provider in providers:+ for snapshot in await snapshot_history(db, tenant_id=tenant_id, provider=provider, limit=50):+ started = snapshot.valid_from+ if started.tzinfo is None:+ started = started.replace(tzinfo=UTC)+ if not (window_start <= started <= at):+ continue+ out.append(+ IdentityChange(+ kind="policy_edit",+ at=started.isoformat(),+ subject=f"{provider} estate",+ summary=(f"{provider} posture changed: {snapshot.principal_count} principals, {snapshot.policy_count} policies"),+ source="posture_snapshot",+ source_ref=f"snapshot:{snapshot.content_hash[:16]}",+ detail={+ "provider": provider,+ "principal_count": snapshot.principal_count,+ "policy_count": snapshot.policy_count,+ "partial": snapshot.partial,+ },+ )+ )+ except Exception as exc: # noqa: BLE001 — an unreadable store is not an empty one+ logger.warning("change_timeline.posture_failed", error=type(exc).__name__)+ return None+ return out+++async def _audit_changes(+ *,+ tenant_id: uuid.UUID,+ wanted: set[str],+ window_start: datetime,+ at: datetime,+) -> list[IdentityChange] | None:+ """Change-shaped audit events naming one of ``subjects``.++ The action vocabulary is closed (:data:`CHANGE_KINDS`). An unfamiliar+ action is simply not returned rather than guessed into a kind, because+ a reader cannot tell a guessed classification from a known one.+ """+ if not wanted:+ return []++ sql = f"""+ SELECT event_time, action, user_name, resource, event_id, connector_type+ FROM {LAKE_TABLE}+ WHERE tenant_id = %(tenant_id)s+ AND event_time >= %(window_start)s+ AND event_time <= %(window_end)s+ AND lower(action) IN %(actions)s+ AND (lower(user_name) IN %(subjects)s OR lower(resource) IN %(subjects)s)+ ORDER BY event_time DESC+ LIMIT %(limit)s+ """+ params = {+ "tenant_id": str(tenant_id),+ "window_start": window_start,+ "window_end": at,+ "actions": tuple(_ACTION_TO_KIND),+ "subjects": tuple(wanted),+ "limit": MAX_CHANGES,+ }++ try:+ result = await execute_lake_query(sql, params=params)+ except Exception as exc: # noqa: BLE001+ logger.warning("change_timeline.lake_failed", error=type(exc).__name__)+ return None++ columns = list(result.columns or [])+ out: list[IdentityChange] = []+ for row in result.rows or ():+ record = dict(zip(columns, row, strict=False))+ action = str(record.get("action") or "")+ kind = _ACTION_TO_KIND.get(action.lower())+ if kind is None:+ continue+ when = record.get("event_time")+ out.append(+ IdentityChange(+ kind=kind,+ at=when.isoformat() if isinstance(when, datetime) else str(when),+ subject=str(record.get("resource") or record.get("user_name") or ""),+ summary=f"{action} by {record.get('user_name') or 'an unnamed actor'}",+ actor=str(record.get("user_name") or ""),+ source="audit_event",+ source_ref=f"event:{record.get('event_id') or ''}",+ detail={"connector_type": record.get("connector_type"), "action": action},+ )+ )+ return outdiff --git a/services/api/app/services/effective_permissions/aws.py b/services/api/app/services/effective_permissions/aws.pyindex 9e8ad0880..c74a074a9 100644--- a/services/api/app/services/effective_permissions/aws.py+++ b/services/api/app/services/effective_permissions/aws.py@@ -80,19 +80,41 @@ For each ``(principal, resource)`` pair the algorithm is:seeing what it stopped, and an action that silently never appears isindistinguishable from one nobody ever granted.-Still not modelled: ``NotAction``, ``NotResource``, condition operators-beyond StringEquals/StringLike, and session policies. Those are flagged as-``notes`` on :class:`ResolverResult` so the UI can surface "this resolver-doesn't yet model X" instead of silently returning a wrong answer — as is a-boundary that is attached to the principal but absent from the snapshot,-which cannot be applied and would otherwise over-permit in exactly the case-the boundary exists to prevent. The synthesised simulator fixture exercises-every feature the resolver does model so the test gate is meaningful.+6. **Session policies** passed on ``AssumeRole``. Like a permissions+ boundary they only cap, never grant, and like a boundary they do not cap a+ resource-based policy that names the session principal directly. They are+ the reason an assumed-role chain's reach is routinely far smaller than the+ role's, which matters because an assumed-role chain is what most cloud+ investigations are looking at.++``NotAction`` and ``NotResource`` invert the set a statement applies to, so+ignoring them does not return a vaguer answer — it returns the complement of+the right one. ``Allow NotAction s3:DeleteObject`` grants everything *except*+delete; read as an absent ``Action`` it grants nothing.++Condition operators are evaluated by+:mod:`app.services.effective_permissions.iam_conditions`, which returns a+three-valued result. An **undecidable** condition — a modelled operator whose+context key the snapshot does not carry, or one of the two operators that+module still does not model — is resolved per effect rather than globally:+an undecidable ``Allow`` does not grant, and an undecidable ``Deny`` still+denies. Both choices refuse to over-report access, which is the only+direction that is safe on a containment decision. Actions withheld that way+are reported in ``deny_actions`` and the reason lands in ``notes``, because+an action that silently never appears is indistinguishable from one nobody+ever granted.++A boundary attached to the principal but absent from the snapshot is flagged+in ``notes`` too: the cap cannot be applied and would otherwise over-permit+in exactly the case the boundary exists to prevent. The synthesised simulator+fixture exercises every feature the resolver models so the test gate is+meaningful."""from __future__ import annotationsimport fnmatch+from dataclasses import dataclassfrom datetime import UTC, datetimefrom typing import Any@@ -103,6 +125,10 @@ from app.services.effective_permissions.base import (ResolverError,ResolverResult,)+from app.services.effective_permissions.iam_conditions import (+ evaluate_block,+ unmodelled_operators_in,+)def _as_list(value: Any) -> list[Any]:@@ -147,43 +173,53 @@ def _resource_matches(statement_resource: Any, resource_arn: str) -> bool:return False-def _condition_matches(condition: dict[str, Any] | None, context: dict[str, Any]) -> bool:- """Evaluate the subset of IAM condition operators the resolver models.+def _statement_covers_resource(statement: dict[str, Any], resource_arn: str) -> bool:+ """Does this statement apply to ``resource_arn``?- Returns ``True`` when no condition is present. Supports- ``StringEquals`` and ``StringLike``; anything else is treated as- "unable to evaluate" and is conservatively considered non-matching when- a context key is present, matching when no context value is supplied- (so a context-less test exercise still exercises the rest of the- resolver).+ ``NotResource`` is the complement of ``Resource`` and the two are mutually+ exclusive in a valid statement. A statement carrying neither applies to+ everything, which is why ``Resource`` defaults to ``"*"`` — and why the+ ``NotResource`` arm has to be checked *first*: reading+ ``statement.get("Resource", "*")`` on a ``NotResource`` statement turns+ an exclusion into a universal grant."""- if not condition:- return True- for operator, kv in condition.items():- if not isinstance(kv, dict):- continue- if operator == "StringEquals":- for key, expected in kv.items():- supplied = context.get(key)- if supplied is None:- return False- if isinstance(expected, list):- if supplied not in expected:- return False- elif supplied != expected:- return False- elif operator == "StringLike":- for key, pattern in kv.items():- supplied = context.get(key)- if supplied is None:- return False- patterns = pattern if isinstance(pattern, list) else [pattern]- if not any(fnmatch.fnmatchcase(supplied, p) for p in patterns):- return False- else:- return False- return True+ if "NotResource" in statement:+ return not _resource_matches(statement["NotResource"], resource_arn)+ return _resource_matches(statement.get("Resource", "*"), resource_arn)+++def _statement_actions(statement: dict[str, Any], catalogue: list[str]) -> set[str]:+ """The action set a statement applies to, honouring ``NotAction``.++ ``NotAction`` is the complement against the service catalogue, so the+ answer is only as complete as the catalogue is — an action the catalogue+ does not list cannot be reported either way. That is a real limit and the+ honest one: inventing actions to complement against would report access+ to API calls that may not exist.+ """++ if "NotAction" in statement:+ excluded = _expand_actions(_as_list(statement["NotAction"]), catalogue)+ return set(catalogue) - excluded+ return _expand_actions(_as_list(statement.get("Action")), catalogue)+++@dataclass(frozen=True)+class _PolicyEvaluation:+ """What one policy said about one resource.++ ``undecided`` is the set this type exists for. Returning+ ``(allow, deny)`` forced every undecidable condition into one of the two,+ and whichever it picked was wrong half the time: folded into ``deny`` it+ claimed a policy denied something it did not mention, dropped entirely it+ silently narrowed the answer with nothing recording that it had.+ """++ allow: set[str]+ deny: set[str]+ undecided: set[str]+ notes: set[str]def _expand_actions(actions: list[str], catalogue: list[str]) -> set[str]:@@ -259,7 +295,7 @@ class AwsIamResolver(Resolver):decisions: list[ResolvedPermission] = []for resource in snap.get("resources", []):- decision = self._resolve_resource(+ decision, resource_notes = self._resolve_resource(principal=principal,resource=resource,identity_policy_refs=identity_policy_refs,@@ -267,6 +303,11 @@ class AwsIamResolver(Resolver):scp_ids=scp_ids,action_catalogue=action_catalogue,)+ # Deduplicated and sorted: one unmodelled operator on a policy+ # attached to forty buckets is one gap, not forty notes.+ for note in sorted(resource_notes):+ if note not in notes:+ notes.append(note)if decision is not None:decisions.append(decision)@@ -316,7 +357,7 @@ class AwsIamResolver(Resolver):policies_by_id: dict[str, dict[str, Any]],scp_ids: list[str],action_catalogue: dict[str, list[str]],- ) -> ResolvedPermission | None:+ ) -> tuple[ResolvedPermission | None, set[str]]:resource_arn = resource["arn"]resource_service = resource.get("service") or resource_arn.split(":")[2]service_catalogue = resource.get("service_actions") or action_catalogue.get(resource_service, [])@@ -325,13 +366,14 @@ class AwsIamResolver(Resolver):chain: list[PolicyChainStep] = []identity_allow: set[str] = set()identity_deny: set[str] = set()+ undecided: set[str] = set()+ notes: set[str] = set()for policy_id, via_group_id in identity_policy_refs:policy = policies_by_id.get(policy_id)if policy is None:continue- step_added = False- allow_added, deny_added = self._evaluate_policy(+ evaluation = self._evaluate_policy(policy=policy,principal_arn=principal["arn"],resource_arn=resource_arn,@@ -339,22 +381,20 @@ class AwsIamResolver(Resolver):catalogue=service_catalogue,principal_match_required=False,)- if allow_added or deny_added:- identity_allow |= allow_added- identity_deny |= deny_added+ notes |= evaluation.notes+ undecided |= evaluation.undecided+ if evaluation.allow or evaluation.deny:+ identity_allow |= evaluation.allow+ identity_deny |= evaluation.denychain.append(PolicyChainStep(kind="policy",id=policy_id,name=policy.get("name", policy_id),- effect="deny" if deny_added and not allow_added else "allow",+ effect="deny" if evaluation.deny and not evaluation.allow else "allow",via=via_group_id,))- step_added = True- if not step_added:- # Identity-policy didn't match the resource — skip it.- passresource_policy_id = resource.get("resource_policy_id")resource_allow: set[str] = set()@@ -362,7 +402,7 @@ class AwsIamResolver(Resolver):if resource_policy_id:resource_policy = policies_by_id.get(resource_policy_id)if resource_policy is not None:- resource_allow, resource_deny = self._evaluate_policy(+ evaluation = self._evaluate_policy(policy=resource_policy,principal_arn=principal["arn"],resource_arn=resource_arn,@@ -370,6 +410,9 @@ class AwsIamResolver(Resolver):catalogue=service_catalogue,principal_match_required=True,)+ notes |= evaluation.notes+ undecided |= evaluation.undecided+ resource_allow, resource_deny = evaluation.allow, evaluation.denyif resource_allow or resource_deny:chain.append(PolicyChainStep(@@ -404,7 +447,7 @@ class AwsIamResolver(Resolver):boundary_id = principal.get("permissions_boundary")boundary = policies_by_id.get(boundary_id) if boundary_id else Noneif boundary is not None:- boundary_allow, boundary_deny = self._evaluate_policy(+ boundary_eval = self._evaluate_policy(policy=boundary,principal_arn=principal["arn"],resource_arn=resource_arn,@@ -412,6 +455,8 @@ class AwsIamResolver(Resolver):catalogue=service_catalogue,principal_match_required=False,)+ notes |= boundary_eval.notes+ boundary_allow, boundary_deny = boundary_eval.allow, boundary_eval.denyboundary_allowed = boundary_allowidentity_capped = identity_allow@@ -438,6 +483,58 @@ class AwsIamResolver(Resolver):))+ # Session policies. Passed on `AssumeRole`, they cap the session the+ # same way a boundary caps the principal: the intersection of the+ # role's identity permissions and the session policy. Two properties+ # are worth stating because getting either wrong changes an answer:+ #+ # * several session policies are ANDed, not ORed, so the cap is the+ # *intersection* of them all. Unioning would widen the cap with+ # each extra policy, which is backwards;+ # * like a boundary, a session policy does not cap a resource-based+ # policy that names the session principal directly, so the cap is+ # applied to the identity arm only.+ #+ # This is the arm that matters most in practice: most cloud+ # investigations are looking at an assumed-role chain, and a role's+ # reach is routinely far wider than the session's.+ session_policy_ids = [pid for pid in _as_list(principal.get("session_policies")) if pid]+ session_blocked: set[str] = set()+ if session_policy_ids:+ session_allowed: set[str] | None = None+ for session_policy_id in session_policy_ids:+ session_policy = policies_by_id.get(session_policy_id)+ if session_policy is None:+ notes.add(+ f"session policy {session_policy_id!r} is attached to the session but absent "+ f"from the snapshot — the cap could not be applied and these actions may over-permit"+ )+ continue+ session_eval = self._evaluate_policy(+ policy=session_policy,+ principal_arn=principal["arn"],+ resource_arn=resource_arn,+ context=context,+ catalogue=service_catalogue,+ principal_match_required=False,+ )+ notes |= session_eval.notes+ session_allowed = set(session_eval.allow) if session_allowed is None else session_allowed & session_eval.allow+ if session_eval.deny:+ identity_deny |= session_eval.deny+ if session_allowed is not None:+ session_blocked = identity_capped - session_allowed+ identity_capped = identity_capped & session_allowed+ if session_blocked or session_allowed:+ chain.append(+ PolicyChainStep(+ kind="session-policy",+ id=",".join(session_policy_ids),+ name=f"session policy ({len(session_policy_ids)})",+ effect="deny" if session_blocked else "allow",+ )+ )+candidate_allow = identity_capped | resource_allow# A Deny written inside a boundary document is an ordinary explicit# deny, not a cap, so it joins the deny set and outranks the@@ -457,7 +554,7 @@ class AwsIamResolver(Resolver):scp = policies_by_id.get(scp_id)if scp is None:continue- allow_set, deny_set = self._evaluate_policy(+ scp_eval = self._evaluate_policy(policy=scp,principal_arn=principal["arn"],resource_arn=resource_arn,@@ -465,6 +562,8 @@ class AwsIamResolver(Resolver):catalogue=service_catalogue,principal_match_required=False,)+ notes |= scp_eval.notes+ allow_set, deny_set = scp_eval.allow, scp_eval.denyif allow_set:scp_allowed |= allow_setchain.append(@@ -502,19 +601,34 @@ class AwsIamResolver(Resolver):# the whole value of a boundary to a reviewer is seeing *what it# stopped*, and an action that silently never appears is# indistinguishable from one nobody ever granted.- shadowed_denies = (pre_scp_allow & candidate_deny) | scp_blocked | boundary_blocked+ #+ # (4) is the undecidable set: an Allow this resolver withheld because+ # its condition could not be evaluated against the snapshot's request+ # context. Reporting it beside the other three is what keeps "we did+ # not grant this" distinguishable from "nobody granted this".+ shadowed_denies = ((pre_scp_allow & candidate_deny) | scp_blocked | boundary_blocked | session_blocked | undecided) - final_actions++ if undecided:+ notes.add(+ f"{len(undecided)} action(s) on {resource_arn} were withheld because a policy condition "+ f"could not be evaluated against the snapshot's request context; they are listed in "+ f"deny_actions rather than dropped"+ )if not final_actions and not shadowed_denies:- return None-- return ResolvedPermission(- principal_id=principal["id"],- resource_id=resource["id"],- resource_kind=resource.get("kind"),- resource_arn=resource_arn,- actions=tuple(sorted(final_actions)),- deny_actions=tuple(sorted(shadowed_denies)),- policy_chain=tuple(chain),+ return None, notes++ return (+ ResolvedPermission(+ principal_id=principal["id"],+ resource_id=resource["id"],+ resource_kind=resource.get("kind"),+ resource_arn=resource_arn,+ actions=tuple(sorted(final_actions)),+ deny_actions=tuple(sorted(shadowed_denies)),+ policy_chain=tuple(chain),+ ),+ notes,)def _evaluate_policy(@@ -526,27 +640,51 @@ class AwsIamResolver(Resolver):context: dict[str, Any],catalogue: list[str],principal_match_required: bool,- ) -> tuple[set[str], set[str]]:- """Return ``(allow_actions, deny_actions)`` from a single policy."""+ ) -> _PolicyEvaluation:+ """Evaluate one policy into allow / deny / withheld-as-undecidable.++ The third set is what makes the three-valued condition result usable.+ An ``Allow`` whose condition cannot be decided is **not** granted —+ but it is also not the same as one nobody wrote, so it is reported+ separately and surfaces to the reader rather than vanishing.+ """document = policy.get("document") or {}statements = _as_list(document.get("Statement"))allow: set[str] = set()deny: set[str] = set()+ undecided: set[str] = set()+ notes: set[str] = set()+for statement in statements:if not isinstance(statement, dict):continue- if principal_match_required:- if not _principal_matches(statement.get("Principal"), principal_arn):- continue- if not _resource_matches(statement.get("Resource", "*"), resource_arn):+ if principal_match_required and not _principal_matches(statement.get("Principal"), principal_arn):+ continue+ if not _statement_covers_resource(statement, resource_arn):continue- if not _condition_matches(statement.get("Condition"), context):++ condition = statement.get("Condition")+ notes |= unmodelled_operators_in(condition)+ satisfied = evaluate_block(condition, context)+ if satisfied is False:continue+effect = statement.get("Effect", "Allow")- actions = _expand_actions(_as_list(statement.get("Action")), catalogue)+ actions = _statement_actions(statement, catalogue)+ if satisfied is None:+ # Undecidable. An Allow is withheld (never over-report what+ # an identity can reach); a Deny still denies (never+ # over-report it either). Both arms fail toward less access.+ if effect == "Allow":+ undecided |= actions+ elif effect == "Deny":+ deny |= actions+ continue+if effect == "Allow":allow |= actionselif effect == "Deny":deny |= actions- return allow, deny++ return _PolicyEvaluation(allow=allow, deny=deny, undecided=undecided, notes=notes)diff --git a/services/api/app/services/effective_permissions/iam_conditions.py b/services/api/app/services/effective_permissions/iam_conditions.pynew file mode 100644index 000000000..683075227--- /dev/null+++ b/services/api/app/services/effective_permissions/iam_conditions.py@@ -0,0 +1,313 @@+"""IAM condition-operator evaluation for the AWS effective-permissions resolver.++Why this is its own module+--------------------------+``aws.py`` modelled exactly two operators, ``StringEquals`` and+``StringLike``, and treated every other one as *unevaluable*, which it then+spelled as "does not match". That direction sounds like the safe one and is+not. ``Condition: {"Bool": {"aws:SecureTransport": "true"}}`` is on a large+share of real bucket policies, and against a request that *is* over TLS the+resolver answered "this identity cannot read the bucket". An access review+built on that under-reports real access, and an under-reported permission is+the one nobody revokes.++The three-valued result is the point+------------------------------------+:func:`evaluate` returns ``True``, ``False`` or ``None``:++* ``True`` — the condition is satisfied by the supplied context.+* ``False`` — the condition is contradicted by the supplied context.+* ``None`` — **undecidable**: the operator is modelled but the request+ context does not carry the key, or the operator is one this module still+ does not model.++``None`` is not a third flavour of "no". The caller decides what an+undecidable condition means for the effect it guards, and the two effects+want opposite answers: an undecidable ``Allow`` must not be counted as+granted (that would over-report), while an undecidable ``Deny`` must not be+dropped (that would also over-report). Collapsing the three values into two+anywhere in this file would put that decision here, where the effect is not+known.++Coverage, stated rather than implied+------------------------------------+Modelled: ``StringEquals``, ``StringNotEquals``, ``StringEqualsIgnoreCase``,+``StringNotEqualsIgnoreCase``, ``StringLike``, ``StringNotLike``, ``Bool``,+``Null``, the six ``Numeric*`` operators, the six ``Date*`` operators,+``IpAddress``, ``NotIpAddress``, and the four ``Arn*`` operators — each with+the ``...IfExists`` suffix and the ``ForAnyValue:`` / ``ForAllValues:`` set+qualifiers.++Not modelled: ``BinaryEquals`` (the context values AiSOC collects are never+base64 blobs) and the ``aws:TagKeys`` multi-value edge cases beyond the set+qualifiers above. Both return ``None`` rather than guessing, and+:data:`UNMODELLED_OPERATORS` names them so the resolver can put the gap in+its ``notes`` instead of silently answering.+"""++from __future__ import annotations++import fnmatch+import ipaddress+from collections.abc import Sequence+from datetime import UTC, datetime+from typing import Any++__all__ = [+ "UNMODELLED_OPERATORS",+ "evaluate",+ "evaluate_block",+ "unmodelled_operators_in",+]++#: Operators this module deliberately does not evaluate. Named so the+#: resolver can report the gap rather than returning a confident answer it+#: has not earned.+UNMODELLED_OPERATORS: frozenset[str] = frozenset({"BinaryEquals", "BinaryNotEquals"})++_SET_QUALIFIERS = ("ForAnyValue:", "ForAllValues:")+_IF_EXISTS = "IfExists"+++def _as_list(value: Any) -> list[Any]:+ if value is None:+ return []+ if isinstance(value, list | tuple):+ return list(value)+ return [value]+++def _parse_date(value: Any) -> datetime | None:+ """ISO-8601 or epoch seconds, both of which IAM accepts.++ Returns ``None`` on anything else so the caller reports undecidable+ rather than ordering a string against a datetime.+ """+ if isinstance(value, int | float):+ return datetime.fromtimestamp(float(value), tz=UTC)+ text = str(value).strip()+ if not text:+ return None+ if text.replace(".", "", 1).isdigit():+ return datetime.fromtimestamp(float(text), tz=UTC)+ try:+ parsed = datetime.fromisoformat(text.replace("Z", "+00:00"))+ except ValueError:+ return None+ return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)+++def _parse_number(value: Any) -> float | None:+ try:+ return float(str(value).strip())+ except (TypeError, ValueError):+ return None+++def _ip_in(supplied: Any, patterns: Sequence[Any]) -> bool | None:+ try:+ address = ipaddress.ip_address(str(supplied).strip())+ except ValueError:+ return None+ for pattern in patterns:+ try:+ network = ipaddress.ip_network(str(pattern).strip(), strict=False)+ except ValueError:+ continue+ if address.version == network.version and address in network:+ return True+ return False+++def _string_cmp(supplied: Any, patterns: Sequence[Any], *, like: bool, fold: bool) -> bool:+ text = str(supplied)+ if fold:+ text = text.casefold()+ for pattern in patterns:+ candidate = str(pattern)+ if fold:+ candidate = candidate.casefold()+ if fnmatch.fnmatchcase(text, candidate) if like else text == candidate:+ return True+ return False+++def _base_operator(operator: str) -> tuple[str, bool, str | None]:+ """Split ``ForAnyValue:StringLikeIfExists`` into its three parts."""+ qualifier: str | None = None+ for candidate in _SET_QUALIFIERS:+ if operator.startswith(candidate):+ qualifier = candidate.rstrip(":")+ operator = operator[len(candidate) :]+ break+ if_exists = operator.endswith(_IF_EXISTS)+ if if_exists:+ operator = operator[: -len(_IF_EXISTS)]+ return operator, if_exists, qualifier+++def _evaluate_one(operator: str, supplied: Any, patterns: Sequence[Any]) -> bool | None:+ """One operator against one present context value."""+ match operator:+ case "StringEquals":+ return _string_cmp(supplied, patterns, like=False, fold=False)+ case "StringNotEquals":+ return not _string_cmp(supplied, patterns, like=False, fold=False)+ case "StringEqualsIgnoreCase":+ return _string_cmp(supplied, patterns, like=False, fold=True)+ case "StringNotEqualsIgnoreCase":+ return not _string_cmp(supplied, patterns, like=False, fold=True)+ case "StringLike":+ return _string_cmp(supplied, patterns, like=True, fold=False)+ case "StringNotLike":+ return not _string_cmp(supplied, patterns, like=True, fold=False)+ # ARN operators compare as strings; ArnLike alone takes wildcards.+ # Treating ArnEquals as a wildcard match would make+ # `arn:aws:iam::*:role/admin` an exact-match pattern that matches+ # nothing, which reads as a correctly-denied grant.+ case "ArnEquals":+ return _string_cmp(supplied, patterns, like=False, fold=False)+ case "ArnNotEquals":+ return not _string_cmp(supplied, patterns, like=False, fold=False)+ case "ArnLike":+ return _string_cmp(supplied, patterns, like=True, fold=False)+ case "ArnNotLike":+ return not _string_cmp(supplied, patterns, like=True, fold=False)+ case "Bool":+ expected = {str(p).strip().lower() == "true" for p in patterns}+ return (str(supplied).strip().lower() == "true") in expected+ case "IpAddress":+ return _ip_in(supplied, patterns)+ case "NotIpAddress":+ inside = _ip_in(supplied, patterns)+ return None if inside is None else not inside+ case _:+ pass++ if operator.startswith("Numeric"):+ left = _parse_number(supplied)+ rights = [_parse_number(p) for p in patterns]+ if left is None or any(r is None for r in rights):+ return None+ values = [r for r in rights if r is not None]+ return _ordered(operator.removeprefix("Numeric"), left, values)++ if operator.startswith("Date"):+ left_dt = _parse_date(supplied)+ right_dts = [_parse_date(p) for p in patterns]+ if left_dt is None or any(r is None for r in right_dts):+ return None+ return _ordered(operator.removeprefix("Date"), left_dt, [r for r in right_dts if r is not None])++ return None+++def _ordered(comparison: str, left: Any, rights: list[Any]) -> bool | None:+ """Shared ordering for the Numeric* and Date* families.++ One implementation rather than two because the operator suffixes are+ identical and two copies drift: an earlier draft had ``LessThanEquals``+ on one family and not the other.+ """+ # A dict of comparators rather than a `match`, so the fall-through is a+ # lookup miss with one explicit `return None` instead of a wildcard arm+ # that reads as unreachable. `match` here also mixed implicit and+ # explicit returns in the eyes of static analysis, which is a real+ # ambiguity in a function whose `None` means "undecidable".+ comparators: dict[str, Any] = {+ "Equals": lambda: any(left == r for r in rights),+ "NotEquals": lambda: all(left != r for r in rights),+ "LessThan": lambda: any(left < r for r in rights),+ "LessThanEquals": lambda: any(left <= r for r in rights),+ "GreaterThan": lambda: any(left > r for r in rights),+ "GreaterThanEquals": lambda: any(left >= r for r in rights),+ }+ comparator = comparators.get(comparison)+ if comparator is None:+ return None+ return bool(comparator())+++def evaluate(operator: str, key: str, expected: Any, context: dict[str, Any]) -> bool | None:+ """Evaluate one ``operator``/``key`` pair against the request context.++ ``None`` means undecidable — see the module docstring for why that is a+ distinct answer and not a polite "no".+ """+ base, if_exists, qualifier = _base_operator(operator)++ if base in UNMODELLED_OPERATORS:+ return None++ patterns = _as_list(expected)++ # Null is the one operator that asks about *presence*, so it is decidable+ # precisely when every other operator is not.+ if base == "Null":+ present = key in context and context[key] is not None+ wants_absent = any(str(p).strip().lower() == "true" for p in patterns)+ return (not present) if wants_absent else present++ if key not in context or context[key] is None:+ # `...IfExists` is explicitly "ignore me when the key is absent",+ # which is a decided True rather than an undecidable.+ return True if if_exists else None++ supplied = context[key]+ supplied_values = _as_list(supplied) if isinstance(supplied, list | tuple) else [supplied]++ results = [_evaluate_one(base, value, patterns) for value in supplied_values]+ if any(r is None for r in results):+ return None+ decided = [bool(r) for r in results]++ if qualifier == "ForAllValues":+ return all(decided)+ if qualifier == "ForAnyValue":+ return any(decided)+ # Single-valued key: IAM evaluates the one value. A multi-valued context+ # key with no set qualifier is a policy-authoring error; `all` is the+ # reading that cannot over-grant.+ return all(decided)+++def evaluate_block(condition: dict[str, Any] | None, context: dict[str, Any]) -> bool | None:+ """Evaluate a whole ``Condition`` block.++ IAM ANDs every operator and every key inside the block. A single ``False``+ therefore decides the block; otherwise a single undecidable makes the+ block undecidable.+ """+ if not condition:+ return True++ saw_undecidable = False+ for operator, pairs in condition.items():+ if not isinstance(pairs, dict):+ # A malformed block is not an empty one. Reporting it as+ # satisfied would turn a typo into a grant.+ return None+ for key, expected in pairs.items():+ result = evaluate(operator, key, expected, context)+ if result is False:+ return False+ if result is None:+ saw_undecidable = True+ return None if saw_undecidable else True+++def unmodelled_operators_in(condition: dict[str, Any] | None) -> set[str]:+ """Operators present in ``condition`` that this module cannot evaluate.++ The resolver puts these into ``ResolverResult.notes`` so a reader sees+ "this answer ignored BinaryEquals" rather than an answer that silently+ assumed something.+ """+ if not condition:+ return set()+ found: set[str] = set()+ for operator in condition:+ base, _, _ = _base_operator(str(operator))+ if base in UNMODELLED_OPERATORS:+ found.add(str(operator))+ return founddiff --git a/services/api/app/services/graph_service.py b/services/api/app/services/graph_service.pyindex ac01dc7fe..483f5d069 100644--- a/services/api/app/services/graph_service.py+++ b/services/api/app/services/graph_service.py@@ -448,6 +448,120 @@ async def get_blast_radius(entity_id: str, entity_type: str, tenant_id: str, hop}+async def get_permission_blast_radius(+ principal_id: str,+ tenant_id: str,+ *,+ provider: str | None = None,+ limit: int = 500,+) -> dict[str, Any]:+ """What this identity's *effective permissions* reach, not what is near it.++ :func:`get_blast_radius` answers "everything within N hops of this node+ over any relationship", which on a populated graph is the alert that+ mentioned the host, the IOC seen on it, the technique tagged on that, and+ a few hundred of their neighbours. It is a reachability answer to a+ containment question, and an analyst cannot act on it.++ This walks ``:EFFECTIVE_PERMISSION`` edges only — the ones the resolver+ materialises from a posture snapshot — so every resource in the answer is+ one the principal can genuinely act on, with the actions that let it. A+ resource reachable by a relationship that is not a permission (the+ principal and the bucket were both mentioned by one alert) is correctly+ absent.++ Two properties the shorter version of this query would not have:++ * **Deny edges are returned, not filtered.** The resolver records what a+ policy tried to grant and something higher-priority removed. During an+ incident that set is evidence: it is the list of things an attacker+ with this identity will try and fail at, which is how the attempt+ shows up in the logs.+ * **Every node on the path is tenant-scoped**, matching+ :func:`get_blast_radius`. An ``:EFFECTIVE_PERMISSION`` edge points at a+ ``:Resource``, and a resource node shared between tenants would+ otherwise enumerate the other tenant's estate — the same defect that+ was fixed in the hop-based query.+ """+ where_provider = "AND e.provider = $provider" if provider else ""+ cypher = f"""+ MATCH (i:Identity {{id: $principal_id}})-[e:EFFECTIVE_PERMISSION]->(r:Resource)+ WHERE {_scoped("i")} AND {_scoped("r")}+ {where_provider}+ RETURN r.id AS resource_id,+ labels(r)[0] AS label,+ properties(r) AS properties,+ e.actions AS actions,+ e.deny_actions AS deny_actions,+ e.provider AS provider,+ e.policy_chain_ids AS policy_chain_ids,+ e.snapshot_id AS snapshot_id+ ORDER BY size(coalesce(e.actions, [])) DESC+ LIMIT $limit+ """++ try:+ async with get_session() as s:+ result = await s.run(+ cypher,+ principal_id=principal_id,+ tenant_id=tenant_id,+ provider=provider,+ limit=max(1, min(int(limit), 5000)),+ global_labels=list(_GLOBAL_LABELS),+ )+ records = [dict(record) async for record in result]+ except Exception as exc: # noqa: BLE001+ logger.warning(+ "permission_blast_radius.failed err=%s",+ str(exc)[:200].replace("\r", "").replace("\n", " "),+ )+ # `resolved: False` rather than an empty list. An empty reach and an+ # unreachable graph are the same shape to a caller that only reads+ # `resources`, and the difference is "this identity is harmless"+ # against "we could not check".+ return {+ "principal_id": principal_id,+ "provider": provider,+ "resolved": False,+ "reason": f"graph unavailable ({type(exc).__name__}); reach is unknown, not empty",+ "resources": [],+ "total_resources": 0,+ "total_actions": 0,+ }++ resources: list[dict[str, Any]] = []+ total_actions = 0+ total_denied = 0+ for r in records:+ actions = [str(a) for a in (r.get("actions") or [])]+ denied = [str(a) for a in (r.get("deny_actions") or [])]+ total_actions += len(actions)+ total_denied += len(denied)+ resources.append(+ {+ "resource_id": r.get("resource_id"),+ "label": r.get("label"),+ "properties": r.get("properties") or {},+ "actions": actions,+ "deny_actions": denied,+ "provider": r.get("provider"),+ "policy_chain_ids": [str(p) for p in (r.get("policy_chain_ids") or [])],+ "snapshot_id": r.get("snapshot_id"),+ }+ )+ return {+ "principal_id": principal_id,+ "provider": provider,+ "resolved": True,+ "resources": resources,+ "total_resources": len(resources),+ "total_actions": total_actions,+ "total_denied_actions": total_denied,+ "truncated": len(resources) >= max(1, min(int(limit), 5000)),+ }++def _is_missing_procedure(exc: Exception) -> bool:"""Whether Neo4j refused because a procedure is not installed.diff --git a/services/api/app/services/identity_privilege.py b/services/api/app/services/identity_privilege.pynew file mode 100644index 000000000..49fc6290b--- /dev/null+++ b/services/api/app/services/identity_privilege.py@@ -0,0 +1,423 @@+"""Privilege tier and the four computed tags, derived from effective permissions.++Depth plan wave 2, Phase 1.1 (carrying the first depth plan's 2.6).++The gap this closes+-------------------+Six resolvers answer "what can this principal do" and nothing asked them.+A tier is a resolver answer compressed to the one thing a queue, a scorer and+a prompt can all use, and the four tags are the questions an analyst actually+asks of an identity during an incident:++``can_escalate``+ Can it grant itself or anyone else more access — directly, or by editing+ a role or trust policy it can reach? This is the tag that turns a+ middling alert into an incident, because an identity that can escalate+ makes every other permission a lower bound.+``data_access``+ Can it read a resource the tenant tagged sensitive?+``third_party``+ Is it external or vendor-owned — a guest, an outside collaborator, an+ OAuth app, a cross-tenant service principal?+``cross_account``+ Does it reach another account, project, subscription or organisation?++Three rules the rest of the module exists to keep+-------------------------------------------------+1. **Unknown is a value.** No snapshot covering the time means ``tier =+ "unknown"`` and all four tags ``None``. Not ``standard``, not ``False``.+ A default tier makes a measured standard account and an unmeasured one+ identical, and the unmeasured one is the one worth looking at.+2. **Every tag carries its evidence.** A tag with no matching action is a+ label; a tag with the actions that produced it is a finding.+3. **The derivation is pure.** It takes a :class:`ResolverResult` and+ returns facts. The store, the lake and the API are elsewhere, so this can+ be tested against a fixture snapshot with no database.+"""++from __future__ import annotations++import fnmatch+from dataclasses import asdict, dataclass, field+from typing import Any, Literal++from app.services.effective_permissions.base import ResolverResult++__all__ = [+ "ESCALATION_ACTIONS",+ "TIER_ORDER",+ "UNKNOWN_PRIVILEGE",+ "IdentityPrivilege",+ "PrivilegeTags",+ "Tier",+ "derive_privilege",+ "tier_at_least",+]++Tier = Literal["unknown", "standard", "elevated", "admin", "domain_admin"]++#: Ordered weakest to strongest. ``unknown`` sorts below ``standard`` so a+#: naive comparison cannot accidentally treat it as privileged, while every+#: surface still has to render it as its own state.+TIER_ORDER: tuple[Tier, ...] = ("unknown", "standard", "elevated", "admin", "domain_admin")+++def tier_at_least(tier: str, floor: Tier) -> bool:+ """``True`` when ``tier`` is at or above ``floor`` on :data:`TIER_ORDER`.++ ``unknown`` is never "at least" anything above itself, so a caller+ gating on ``admin`` does not silently include identities nobody measured.+ """+ try:+ return TIER_ORDER.index(tier) >= TIER_ORDER.index(floor) # type: ignore[arg-type]+ except ValueError:+ return False+++#: Action patterns that let a principal grant access. Provider-namespaced and+#: matched with shell globbing against the resolver's own action strings, so+#: the same table serves all six providers.+#:+#: Every entry is an action that writes an *authorization* object. Actions+#: that merely read one are deliberately absent: `iam:GetRole` tells an+#: attacker where to go and grants nothing, and tagging it `can_escalate`+#: would put the tag on most of the estate, at which point nobody reads it.+ESCALATION_ACTIONS: tuple[str, ...] = (+ # AWS+ "iam:Attach*Policy",+ "iam:Put*Policy",+ "iam:CreatePolicyVersion",+ "iam:SetDefaultPolicyVersion",+ "iam:UpdateAssumeRolePolicy",+ "iam:CreateAccessKey",+ "iam:CreateLoginProfile",+ "iam:UpdateLoginProfile",+ "iam:AddUserToGroup",+ "iam:CreateRole",+ "iam:CreateUser",+ "iam:PassRole",+ "sts:AssumeRole",+ "organizations:*Policy",+ # Azure / Entra+ "Microsoft.Authorization/roleAssignments/write",+ "Microsoft.Authorization/roleDefinitions/write",+ "microsoft.directory/roleAssignments/*",+ "microsoft.directory/users/password/update",+ "microsoft.directory/servicePrincipals/credentials/update",+ "microsoft.directory/applications/credentials/update",+ # GCP+ "resourcemanager.projects.setIamPolicy",+ "resourcemanager.organizations.setIamPolicy",+ "resourcemanager.folders.setIamPolicy",+ "iam.roles.create",+ "iam.roles.update",+ "iam.serviceAccounts.actAs",+ "iam.serviceAccounts.getAccessToken",+ "iam.serviceAccountKeys.create",+ # Okta+ "okta.roles.manage",+ "okta.users.manage",+ "okta.groups.manage",+ "okta.apps.manage",+ # Google Workspace+ "admin.roles.manage",+ "admin.users.update",+ "admin.users.create",+ # GitHub+ "admin:org",+ "write:org",+ "repo:admin",+ "admin:repo_hook",+ "manage_runners:org",+)++#: Action patterns that read data. Tagged ``data_access`` only when the+#: resource they land on is marked sensitive, because "can read an object" on+#: its own describes most identities in a cloud estate.+_DATA_READ_ACTIONS: tuple[str, ...] = (+ "s3:Get*",+ "s3:List*",+ "dynamodb:GetItem",+ "dynamodb:Scan",+ "dynamodb:Query",+ "secretsmanager:GetSecretValue",+ "ssm:GetParameter*",+ "kms:Decrypt",+ "rds:DownloadDBLogFilePortion",+ "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read",+ "Microsoft.KeyVault/vaults/secrets/read",+ "storage.objects.get",+ "storage.objects.list",+ "bigquery.tables.getData",+ "secretmanager.versions.access",+ "drive.files.read",+ "gmail.messages.read",+ "repo",+ "read:packages",+)++#: Resource kinds and tag values that mean "the tenant called this sensitive".+_SENSITIVE_MARKERS: frozenset[str] = frozenset({"sensitive", "restricted", "confidential", "pii", "phi", "pci", "regulated", "crown-jewel"})++#: Tier floors by how much of the estate's authorization surface a principal+#: can write. Derived from what the identity can *do*, never from the name of+#: the role it holds -- a role called "ReadOnly" that can call+#: `iam:PutUserPolicy` is an admin, and the name is the part an attacker+#: controls when they create it.+_ADMIN_WILDCARDS: tuple[str, ...] = ("*", "*:*", "iam:*", "Microsoft.Authorization/*", "resourcemanager.*")+++@dataclass(frozen=True)+class PrivilegeTags:+ """The four computed tags. ``None`` is "no snapshot could answer"."""++ can_escalate: bool | None = None+ data_access: bool | None = None+ third_party: bool | None = None+ cross_account: bool | None = None++ def as_dict(self) -> dict[str, bool | None]:+ return asdict(self)++ def set_tags(self) -> list[str]:+ """The tag names that are ``True``, for a prompt or a chip row."""+ return [name for name, value in asdict(self).items() if value is True]+++@dataclass(frozen=True)+class IdentityPrivilege:+ """One identity's tier, tags and the evidence for each."""++ tier: Tier = "unknown"+ tags: PrivilegeTags = field(default_factory=PrivilegeTags)+ provider: str = ""+ principal_id: str = ""+ #: ``{tag_name: [action, ...]}`` plus ``"tier"`` -> the actions that set it.+ evidence: dict[str, list[str]] = field(default_factory=dict)+ action_count: int = 0+ resource_count: int = 0+ #: Set when the snapshot the facts came from was itself incomplete.+ partial_source: bool = False+ #: Carried from the resolver so a surface can say what was not modelled.+ notes: list[str] = field(default_factory=list)++ @property+ def measured(self) -> bool:+ """``False`` when no snapshot covered the question."""+ return self.tier != "unknown" or self.action_count > 0++ def as_dict(self) -> dict[str, Any]:+ return {+ "tier": self.tier,+ "tags": self.tags.as_dict(),+ "provider": self.provider,+ "principal_id": self.principal_id,+ "evidence": {k: list(v) for k, v in self.evidence.items()},+ "action_count": self.action_count,+ "resource_count": self.resource_count,+ "partial_source": self.partial_source,+ "measured": self.measured,+ "notes": list(self.notes),+ }+++#: The singleton every caller returns when no snapshot covers the time. A+#: module-level constant rather than a fresh `IdentityPrivilege()` per call+#: site, so "is this unknown" is one identity comparison and a reviewer can+#: grep for every place that gives up.+UNKNOWN_PRIVILEGE = IdentityPrivilege()+++def _matches_any(action: str, patterns: tuple[str, ...]) -> bool:+ return any(fnmatch.fnmatchcase(action, pattern) for pattern in patterns)+++def _is_sensitive(decision: Any) -> bool:+ """Does this decision land on something the tenant marked sensitive?"""+ kind = str(getattr(decision, "resource_kind", "") or "").lower()+ arn = str(getattr(decision, "resource_arn", "") or "").lower()+ rid = str(getattr(decision, "resource_id", "") or "").lower()+ haystack = f"{kind} {arn} {rid}"+ return any(marker in haystack for marker in _SENSITIVE_MARKERS)+++def _account_of(arn_or_id: str) -> str:+ """The account / project / subscription an identifier belongs to.++ Six providers spell this six ways, so the parse is deliberately shallow:+ an AWS ARN's fifth colon-separated field, and otherwise the first path+ segment of a GCP-style ``projects/x/...`` or Azure-style+ ``/subscriptions/x/...`` identifier. Anything else returns ``""``, which+ reads as "cannot tell" and never as "same account".+ """+ value = (arn_or_id or "").strip()+ if value.startswith("arn:"):+ parts = value.split(":")+ return parts[4] if len(parts) > 4 else ""+ lowered = value.lower().lstrip("/")+ for prefix in ("subscriptions/", "projects/", "organizations/", "folders/"):+ if lowered.startswith(prefix):+ rest = lowered[len(prefix) :]+ return rest.split("/", 1)[0]+ return ""+++def derive_privilege(+ result: ResolverResult,+ *,+ principal_account: str | None = None,+ external_domains: frozenset[str] | None = None,+ principal_kind: str | None = None,+ partial_source: bool = False,+) -> IdentityPrivilege:+ """Compress a resolver answer into a tier, four tags and their evidence.++ Pure. ``result`` is whatever :func:`resolve_effective_permissions`+ returned; everything else is optional context the caller already has.++ A resolver result with no decisions returns :data:`UNKNOWN_PRIVILEGE`+ rather than ``standard`` with four ``False`` tags, because "the resolver+ found nothing" and "this identity can do nothing" are different claims+ and only one of them is safe to act on.+ """+ decisions = list(result.decisions or ())+ if not decisions:+ return IdentityPrivilege(+ provider=result.provider,+ principal_id=result.principal_id,+ partial_source=partial_source,+ notes=list(result.notes or ()),+ )++ all_actions: set[str] = set()+ escalation_evidence: set[str] = set()+ data_evidence: set[str] = set()+ cross_account_evidence: set[str] = set()+ admin_evidence: set[str] = set()++ home_account = (principal_account or _account_of(result.principal_id) or "").lower()++ for decision in decisions:+ actions = tuple(decision.actions)+ all_actions.update(actions)++ for action in actions:+ if _matches_any(action, ESCALATION_ACTIONS):+ escalation_evidence.add(action)+ if _matches_any(action, _ADMIN_WILDCARDS):+ admin_evidence.add(action)++ if _is_sensitive(decision):+ data_evidence.update(a for a in actions if _matches_any(a, _DATA_READ_ACTIONS))++ resource_account = _account_of(str(decision.resource_arn or decision.resource_id or "")).lower()+ if home_account and resource_account and resource_account != home_account:+ cross_account_evidence.add(str(decision.resource_arn or decision.resource_id))++ # `sts:AssumeRole` onto a role in another account is the canonical+ # cross-account reach and does not show up as a differing resource+ # account unless the role itself is in the snapshot's resource list.+ for action in all_actions:+ if action.startswith("sts:AssumeRole") and home_account:+ cross_account_evidence.add(action)++ tier = _derive_tier(+ admin_evidence=admin_evidence,+ escalation_evidence=escalation_evidence,+ action_count=len(all_actions),+ )++ external = _is_third_party(+ principal_id=result.principal_id,+ principal_kind=principal_kind,+ external_domains=external_domains or frozenset(),+ )++ evidence: dict[str, list[str]] = {}+ if escalation_evidence:+ evidence["can_escalate"] = sorted(escalation_evidence)+ if data_evidence:+ evidence["data_access"] = sorted(data_evidence)+ if cross_account_evidence:+ evidence["cross_account"] = sorted(cross_account_evidence)+ if admin_evidence:+ evidence["tier"] = sorted(admin_evidence)+ elif escalation_evidence:+ evidence["tier"] = sorted(escalation_evidence)++ return IdentityPrivilege(+ tier=tier,+ tags=PrivilegeTags(+ can_escalate=bool(escalation_evidence),+ data_access=bool(data_evidence),+ # `third_party` is the one tag that is not derivable from the+ # permission set at all -- it is a property of the principal. It+ # stays None when the caller supplied nothing to decide it with,+ # rather than defaulting to "internal", because "we assume+ # everyone is staff" is the wrong default in a product whose job+ # is noticing when they are not.+ third_party=external,+ cross_account=bool(cross_account_evidence),+ ),+ provider=result.provider,+ principal_id=result.principal_id,+ evidence=evidence,+ action_count=len(all_actions),+ resource_count=len(decisions),+ partial_source=partial_source,+ notes=list(result.notes or ()),+ )+++def _derive_tier(*, admin_evidence: set[str], escalation_evidence: set[str], action_count: int) -> Tier:+ """Tier from what the identity can write, never from what it is called.++ ``domain_admin`` is reserved for a wildcard over the whole provider --+ the thing that makes every other permission moot. ``admin`` is+ authorization-write without that wildcard. ``elevated`` is a principal+ with a wide action surface and no authorization write, which is the+ "powerful but cannot grant" shape (a data engineer with full BigQuery)+ that a tiering scheme collapsing to admin/standard cannot express.+ """+ if any(pattern in admin_evidence for pattern in ("*", "*:*")):+ return "domain_admin"+ if admin_evidence:+ return "admin"+ if escalation_evidence:+ return "admin"+ if action_count >= _ELEVATED_ACTION_FLOOR:+ return "elevated"+ return "standard"+++#: Where "a wide action surface" starts. Not a free parameter: it is the+#: point above which an identity's reach stops being summarisable by a human+#: reading the list, which is the thing the tier exists to flag. Pinned by a+#: test so moving it is a decision rather than a tweak.+_ELEVATED_ACTION_FLOOR = 25+++def _is_third_party(+ *,+ principal_id: str,+ principal_kind: str | None,+ external_domains: frozenset[str],+) -> bool | None:+ """External or vendor-owned, or ``None`` when nothing says either way."""+ kind = (principal_kind or "").strip().lower()+ if kind:+ if kind in {"guest", "external", "outside_collaborator", "oauth_app", "vendor", "service_principal_external"}:+ return True+ if kind in {"member", "employee", "user", "service_account", "workload"}:+ return False++ if "#ext#" in principal_id.lower():+ # Entra's own spelling for a guest account.+ return True++ if external_domains and "@" in principal_id:+ domain = principal_id.rsplit("@", 1)[-1].strip().lower()+ if domain:+ return domain in external_domains++ return Nonediff --git a/services/api/app/services/identity_privilege_store.py b/services/api/app/services/identity_privilege_store.pynew file mode 100644index 000000000..c797d796c--- /dev/null+++ b/services/api/app/services/identity_privilege_store.py@@ -0,0 +1,434 @@+"""Read and write derived privilege facts and the runtime-usage overlay.++Backs migration 124. The pure derivation lives in+:mod:`app.services.identity_privilege`; this module is the only thing that+writes the two tables, and it exists to keep the same two invariants the+posture store keeps one file over:++* at most one row per ``(tenant, provider, principal)`` has+ ``valid_until IS NULL``;+* the point-in-time read is the easy call to reach for, so a caller asking+ "what could this identity do when it acted" does not quietly get "what can+ it do now" because that query was shorter to write.++Every statement filters on ``tenant_id`` in the ``WHERE`` clause. RLS is on+the tables too, but the query-layer filter is the control relied on here: a+path that forgets the session GUC would read across tenants with the policy+still in force.+"""++from __future__ import annotations++import json+import uuid+from dataclasses import dataclass+from datetime import datetime+from typing import Any++from sqlalchemy import text+from sqlalchemy.ext.asyncio import AsyncSession++from app.services.identity_privilege import UNKNOWN_PRIVILEGE, IdentityPrivilege, PrivilegeTags++__all__ = [+ "PrivilegeFact",+ "UsageOverlay",+ "live_facts_for_provider",+ "privilege_at",+ "record_usage",+ "save_privilege",+ "usage_for_principal",+]++# The column list is written out in each statement rather than+# interpolated from a shared constant. Interpolating it makes every+# statement an f-string, and `text()` with an f-string is a taint sink for+# `avoid-sqlalchemy-text` -- correctly, because a rule cannot tell a+# constant column list from a caller-supplied one. Four copies of a column+# list is the cheaper half of that trade, and `check_raw_sql_columns.py`+# compares each one against the migration, so a drifting copy is caught.+++@dataclass(frozen=True)+class PrivilegeFact:+ """A stored fact plus the interval over which it was true."""++ privilege: IdentityPrivilege+ principal_label: str | None+ valid_from: datetime+ valid_until: datetime | None+ snapshot_id: uuid.UUID | None++ @property+ def is_live(self) -> bool:+ return self.valid_until is None+++@dataclass(frozen=True)+class UsageOverlay:+ """Which granted actions were exercised, and over what window.++ ``measured`` is the field that stops this from becoming a right-sizing+ disaster. An install with no lake returns ``measured=False`` and an empty+ ``used`` set, and a caller that treats the two the same recommends+ revoking every permission in the estate.+ """++ measured: bool+ used: dict[str, datetime | None]+ window_start: datetime | None = None+ window_end: datetime | None = None+ source: str = "lake"++ def unused(self, granted: set[str]) -> set[str] | None:+ """Granted-but-unexercised, or ``None`` when usage was not measured."""+ if not self.measured:+ return None+ return granted - set(self.used)+++def _bool_or_none(value: Any) -> bool | None:+ return None if value is None else bool(value)+++def _row_to_fact(row: Any) -> PrivilegeFact:+ evidence = row.evidence if isinstance(row.evidence, dict) else json.loads(row.evidence or "{}")+ return PrivilegeFact(+ privilege=IdentityPrivilege(+ tier=row.tier,+ tags=PrivilegeTags(+ can_escalate=_bool_or_none(row.can_escalate),+ data_access=_bool_or_none(row.data_access),+ third_party=_bool_or_none(row.third_party),+ cross_account=_bool_or_none(row.cross_account),+ ),+ provider=row.provider,+ principal_id=row.principal_id,+ evidence={k: list(v) for k, v in evidence.items() if isinstance(v, list)},+ action_count=int(row.action_count or 0),+ resource_count=int(row.resource_count or 0),+ partial_source=bool(row.partial_source),+ ),+ principal_label=row.principal_label,+ valid_from=row.valid_from,+ valid_until=row.valid_until,+ snapshot_id=row.snapshot_id,+ )+++async def save_privilege(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ privilege: IdentityPrivilege,+ principal_label: str | None = None,+ principal_kind: str | None = None,+ snapshot_id: uuid.UUID | None = None,+ resolver_version: str = "",+) -> PrivilegeFact:+ """Record a derivation, opening a new version only when something changed.++ Same reasoning as ``posture_store.save_snapshot``: a collector running+ hourly against an estate nobody changed must not write a version an hour,+ or "when did this identity's privilege last change" -- the question every+ one of these rows exists to answer -- becomes unanswerable.++ Equality is on the facts, not on the snapshot hash: two different posture+ documents can derive the same tier and tags, and that is not a privilege+ change.+ """+ live = await _live_fact(db, tenant_id=tenant_id, provider=privilege.provider, principal_id=privilege.principal_id)++ if live is not None and _same_facts(live.privilege, privilege) and live.principal_label == principal_label:+ return live++ if live is not None:+ await db.execute(+ text(+ """+ UPDATE identity_privilege_facts+ SET valid_until = NOW()+ WHERE tenant_id = :tenant_id+ AND provider = :provider+ AND principal_id = :principal_id+ AND valid_until IS NULL+ """+ ),+ {+ "tenant_id": str(tenant_id),+ "provider": privilege.provider,+ "principal_id": privilege.principal_id,+ },+ )++ result = await db.execute(+ text(+ """+ INSERT INTO identity_privilege_facts (+ tenant_id, provider, principal_id, principal_label, principal_kind,+ tier, can_escalate, data_access, third_party, cross_account,+ evidence, action_count, resource_count, snapshot_id, resolver_version,+ partial_source+ ) VALUES (+ :tenant_id, :provider, :principal_id, :principal_label, :principal_kind,+ :tier, :can_escalate, :data_access, :third_party, :cross_account,+ CAST(:evidence AS jsonb), :action_count, :resource_count, :snapshot_id,+ :resolver_version, :partial_source+ )+ RETURNING id, tenant_id, provider, principal_id, principal_label, principal_kind,+ tier, can_escalate, data_access, third_party, cross_account,+ evidence, action_count, resource_count, snapshot_id, resolver_version,+ partial_source, valid_from, valid_until+ """+ ),+ {+ "tenant_id": str(tenant_id),+ "provider": privilege.provider,+ "principal_id": privilege.principal_id,+ "principal_label": principal_label,+ "principal_kind": principal_kind,+ "tier": privilege.tier,+ "can_escalate": privilege.tags.can_escalate,+ "data_access": privilege.tags.data_access,+ "third_party": privilege.tags.third_party,+ "cross_account": privilege.tags.cross_account,+ "evidence": json.dumps(privilege.evidence, default=str),+ "action_count": privilege.action_count,+ "resource_count": privilege.resource_count,+ "snapshot_id": str(snapshot_id) if snapshot_id else None,+ "resolver_version": resolver_version,+ "partial_source": privilege.partial_source,+ },+ )+ return _row_to_fact(result.fetchone())+++def _same_facts(left: IdentityPrivilege, right: IdentityPrivilege) -> bool:+ return (+ left.tier == right.tier+ and left.tags == right.tags+ and left.evidence == right.evidence+ and left.action_count == right.action_count+ and left.partial_source == right.partial_source+ )+++async def _live_fact(db: AsyncSession, *, tenant_id: uuid.UUID, provider: str, principal_id: str) -> PrivilegeFact | None:+ result = await db.execute(+ text(+ """+ SELECT id, tenant_id, provider, principal_id, principal_label, principal_kind,+ tier, can_escalate, data_access, third_party, cross_account,+ evidence, action_count, resource_count, snapshot_id, resolver_version,+ partial_source, valid_from, valid_until+ FROM identity_privilege_facts+ WHERE tenant_id = :tenant_id+ AND provider = :provider+ AND principal_id = :principal_id+ AND valid_until IS NULL+ LIMIT 1+ """+ ),+ {"tenant_id": str(tenant_id), "provider": provider, "principal_id": principal_id},+ )+ row = result.fetchone()+ return _row_to_fact(row) if row else None+++async def privilege_at(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ principal_id: str,+ at: datetime,+ provider: str | None = None,+) -> PrivilegeFact | None:+ """The fact that was in force at ``at``, by id or by label.++ ``provider`` is optional because an alert rarely names one: it carries a+ username and the platform has to find whichever provider knows it. When+ several do, the strongest tier wins -- a person who is a standard Okta+ user and an AWS administrator is an administrator for the purposes of+ deciding what an alert about them means.++ Returns ``None`` when nothing covers ``at``. Falling back to the live row+ would answer a question nobody asked in a way no reader could detect,+ which is the exact failure migration 103 is versioned to prevent.+ """+ # The optional provider filter is a *parameter*, not an appended clause.+ # Assembling the WHERE from a list makes the statement an f-string, and+ # `text()` with an f-string is a taint sink the scanner refuses --+ # correctly, since it cannot tell a constant clause from a+ # caller-supplied one. One statement with one plan is also the better+ # shape: there is no second query path to test.+ #+ # `CAST(:provider AS text)` rather than `:provider::text`: SQLAlchemy's+ # bind regex backtracks over a Postgres cast and would declare a+ # parameter named `provide`, and an untyped `:provider IS NULL` raises+ # on asyncpg. `check_text_bind_casts.py` gates both.+ result = await db.execute(+ text(+ """+ SELECT id, tenant_id, provider, principal_id, principal_label, principal_kind,+ tier, can_escalate, data_access, third_party, cross_account,+ evidence, action_count, resource_count, snapshot_id, resolver_version,+ partial_source, valid_from, valid_until+ FROM identity_privilege_facts+ WHERE tenant_id = :tenant_id+ AND valid_from <= :at+ AND (valid_until IS NULL OR valid_until > :at)+ AND (principal_id = :principal_id OR lower(principal_label) = lower(:principal_id))+ AND (CAST(:provider AS text) IS NULL OR provider = :provider)+ ORDER BY array_position(+ ARRAY['unknown','standard','elevated','admin','domain_admin']::text[], tier+ ) DESC,+ valid_from DESC+ LIMIT 1+ """+ ),+ {+ "tenant_id": str(tenant_id),+ "principal_id": principal_id,+ "at": at,+ "provider": provider or None,+ },+ )+ row = result.fetchone()+ return _row_to_fact(row) if row else None+++async def live_facts_for_provider(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ provider: str,+ limit: int = 2000,+) -> list[PrivilegeFact]:+ """Every live fact for one provider — the access review's scan."""+ result = await db.execute(+ text(+ """+ SELECT id, tenant_id, provider, principal_id, principal_label, principal_kind,+ tier, can_escalate, data_access, third_party, cross_account,+ evidence, action_count, resource_count, snapshot_id, resolver_version,+ partial_source, valid_from, valid_until+ FROM identity_privilege_facts+ WHERE tenant_id = :tenant_id+ AND provider = :provider+ AND valid_until IS NULL+ ORDER BY principal_id+ LIMIT :limit+ """+ ),+ {"tenant_id": str(tenant_id), "provider": provider, "limit": max(1, min(limit, 10000))},+ )+ return [_row_to_fact(row) for row in result.fetchall()]+++async def record_usage(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ provider: str,+ principal_id: str,+ observations: dict[str, tuple[int, datetime | None, datetime | None]],+ window_start: datetime,+ window_end: datetime,+ source: str = "lake",+) -> int:+ """Upsert one principal's observed actions for a window.++ ``observations`` maps action to ``(count, first_used, last_used)``.+ Returns the number of rows written. An empty mapping writes nothing and+ returns 0 -- which is correct, and is also why the *caller* has to record+ whether it measured at all: this function cannot tell "the principal used+ nothing" from "the lake was unreachable", and neither can its rows.+ """+ if not observations:+ return 0++ written = 0+ for action, (count, first_used, last_used) in observations.items():+ await db.execute(+ text(+ """+ INSERT INTO identity_permission_usage (+ tenant_id, provider, principal_id, action,+ use_count, first_used_at, last_used_at,+ window_start, window_end, source+ ) VALUES (+ :tenant_id, :provider, :principal_id, :action,+ :use_count, :first_used_at, :last_used_at,+ :window_start, :window_end, :source+ )+ ON CONFLICT (tenant_id, provider, principal_id, action) DO UPDATE+ SET use_count = EXCLUDED.use_count,+ first_used_at = LEAST(+ identity_permission_usage.first_used_at, EXCLUDED.first_used_at+ ),+ last_used_at = GREATEST(+ identity_permission_usage.last_used_at, EXCLUDED.last_used_at+ ),+ window_start = EXCLUDED.window_start,+ window_end = EXCLUDED.window_end,+ source = EXCLUDED.source,+ computed_at = NOW()+ """+ ),+ {+ "tenant_id": str(tenant_id),+ "provider": provider,+ "principal_id": principal_id,+ "action": action,+ "use_count": int(count),+ "first_used_at": first_used,+ "last_used_at": last_used,+ "window_start": window_start,+ "window_end": window_end,+ "source": source,+ },+ )+ written += 1+ return written+++async def usage_for_principal(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ provider: str,+ principal_id: str,+) -> UsageOverlay:+ """What this principal exercised, or an honest "not measured".++ No rows means no observation was ever recorded for this principal, and+ this function says so with ``measured=False`` rather than returning an+ empty used-set that a caller would read as "used nothing".+ """+ result = await db.execute(+ text(+ """+ SELECT action, last_used_at, window_start, window_end, source+ FROM identity_permission_usage+ WHERE tenant_id = :tenant_id+ AND provider = :provider+ AND principal_id = :principal_id+ """+ ),+ {"tenant_id": str(tenant_id), "provider": provider, "principal_id": principal_id},+ )+ rows = result.fetchall()+ if not rows:+ return UsageOverlay(measured=False, used={})+ return UsageOverlay(+ measured=True,+ used={row.action: row.last_used_at for row in rows},+ window_start=rows[0].window_start,+ window_end=rows[0].window_end,+ source=rows[0].source,+ )+++def privilege_or_unknown(fact: PrivilegeFact | None) -> IdentityPrivilege:+ """``fact.privilege`` or the unknown singleton — never a default tier."""+ return fact.privilege if fact is not None else UNKNOWN_PRIVILEGEdiff --git a/services/api/app/services/identity_resolution.py b/services/api/app/services/identity_resolution.pynew file mode 100644index 000000000..748b13dd3--- /dev/null+++ b/services/api/app/services/identity_resolution.py@@ -0,0 +1,372 @@+"""One person across six sources, with confidence and evidence.++Depth plan wave 2, Phase 3.1.++An alert names ``[email protected]`` from Okta, ``ALICE`` from an EDR device+login, ``arn:aws:iam::1:user/alice`` from CloudTrail and ``alice-gh`` from a+code platform, and the platform treats them as four people. Stories cannot+cross a source, and a privilege answer about one does not reach an alert+about another.++The failure mode this is designed against+-----------------------------------------+Not "no links" — **wrong links**. Merging two people because their+local-parts match destroys the thing the feature exists to provide, and it+destroys it invisibly: the merged identity looks richer, not broken. The+canonical wrong merges are concrete and all three are refused here:++* ``[email protected]`` and ``[email protected]`` — routinely two people;+* ``admin`` on an endpoint and ``admin`` in a directory — two accounts;+* ``svc-deploy`` seen in two clouds — two service accounts, usually.++So every link carries a rule, a confidence and the evidence that produced+it, and **only links at or above** :data:`MERGE_THRESHOLD` are allowed to+merge. Everything weaker is returned and shown, because a weak link is+genuinely useful to an analyst reading a story and genuinely dangerous to+anything that acts on it. Those are different consumers, and the shape lets+them have different answers.++Where it is stored+------------------+Both the entity store in CORE (ADR-0010 put it there precisely so a CORE+install is not context-free) and the graph in ``full``, behind one read+interface — see :mod:`app.services.identity_resolution_store`.+"""++from __future__ import annotations++import re+from dataclasses import dataclass, field+from typing import Any++__all__ = [+ "LINK_RULES",+ "MERGE_THRESHOLD",+ "SOURCE_KINDS",+ "IdentityLink",+ "ResolutionResult",+ "resolve_links",+]++#: The six sources the plan names. A seventh added without a rule below+#: links nothing, so the two lists are kept adjacent on purpose.+SOURCE_KINDS: tuple[str, ...] = (+ "idp",+ "edr_device_user",+ "proxy_user",+ "cloud_principal",+ "code_platform",+ "workspace",+)++#: At or above this, a link may merge two identities into one cluster.+#: Below it the link is shown and nothing acts on it.+#:+#: 0.90 rather than something lower because of the asymmetry of the two+#: errors. A missed link costs an analyst one pivot they make by hand; a+#: wrong link silently attributes one person's alerts to another, and+#: nothing downstream can detect it.+MERGE_THRESHOLD = 0.90++#: Identifiers that are role names rather than people. A match on one of+#: these is not evidence of anything: every estate has an `admin`, and+#: merging on it would join every tenant's local administrator into one+#: identity and take the whole estate's alerts with it.+GENERIC_IDENTIFIERS: frozenset[str] = frozenset(+ {+ "admin",+ "administrator",+ "root",+ "guest",+ "support",+ "service",+ "svc",+ "system",+ "operator",+ "backup",+ "test",+ "deploy",+ "build",+ "ci",+ "runner",+ "bot",+ "jenkins",+ "terraform",+ "ansible",+ }+)++#: Prefixes that mark an identifier as a shared service account rather than+#: a person. Matched on the local part, so `[email protected]` and a+#: bare `svc-deploy` are both caught.+_SERVICE_PREFIXES: tuple[str, ...] = ("svc-", "svc_", "service-", "sa-", "bot-", "ci-")++_ARN_USER_RE = re.compile(r"^arn:aws:iam::\d+:(?:user|role)/(?:.*/)?(?P<name>[^/]+)$")+_GCP_MEMBER_RE = re.compile(r"^(?:user|serviceAccount):(?P<name>.+)$")+++@dataclass(frozen=True)+class IdentityLink:+ """One proposed link between two source-scoped identities."""++ left: str+ right: str+ confidence: float+ rule: str+ evidence: list[str] = field(default_factory=list)++ @property+ def merged(self) -> bool:+ """Whether this link is strong enough to merge on."""+ return self.confidence >= MERGE_THRESHOLD++ def as_dict(self) -> dict[str, Any]:+ return {+ "left": self.left,+ "right": self.right,+ "confidence": round(self.confidence, 4),+ "rule": self.rule,+ "evidence": list(self.evidence),+ "merged": self.merged,+ # Said in words as well as implied by the number, because the+ # number is a threshold comparison a reader has to know to make.+ "note": ("Strong enough to treat as one person." if self.merged else "Shown as a possible link only; nothing merges on it."),+ }+++@dataclass(frozen=True)+class ResolutionResult:+ """Every link found, strong and weak."""++ links: list[IdentityLink] = field(default_factory=list)+ #: Identities that matched nothing. Returned rather than dropped so a+ #: surface can say "we could not place this account", which is a+ #: finding in itself on an estate that is supposed to be joined up.+ unlinked: list[str] = field(default_factory=list)++ def clusters_for_merge(self) -> list[set[str]]:+ """Connected components over **merge-strength links only**.++ Weak links are deliberately absent from the graph this walks. A+ chain of three weak links would otherwise merge two identities that+ no single rule was willing to join, which is how a transitive+ merge quietly becomes the broadest possible one.+ """+ parent: dict[str, str] = {}++ def find(x: str) -> str:+ parent.setdefault(x, x)+ while parent[x] != x:+ parent[x] = parent[parent[x]]+ x = parent[x]+ return x++ def union(a: str, b: str) -> None:+ ra, rb = find(a), find(b)+ if ra != rb:+ parent[ra] = rb++ strong = [link for link in self.links if link.merged]+ for link in strong:+ union(link.left, link.right)++ groups: dict[str, set[str]] = {}+ for node in parent:+ groups.setdefault(find(node), set()).add(node)+ return [members for members in groups.values() if len(members) > 1]++ def as_dict(self) -> dict[str, Any]:+ return {+ "links": [link.as_dict() for link in self.links],+ "merge_threshold": MERGE_THRESHOLD,+ "clusters": [sorted(c) for c in self.clusters_for_merge()],+ "unlinked": list(self.unlinked),+ }+++def _key(record: dict[str, Any]) -> str:+ return f"{record.get('source', '')}:{record.get('id', '')}"+++def _local_part(value: str) -> str:+ text = (value or "").strip().lower()+ if not text:+ return ""+ arn = _ARN_USER_RE.match(text)+ if arn:+ text = arn.group("name")+ member = _GCP_MEMBER_RE.match(text)+ if member:+ text = member.group("name")+ if "@" in text:+ text = text.split("@", 1)[0]+ if "\\" in text:+ # DOMAIN\user+ text = text.split("\\", 1)[1]+ return text.strip()+++def _domain(value: str) -> str:+ text = (value or "").strip().lower()+ return text.split("@", 1)[1] if "@" in text else ""+++def _is_generic(local: str) -> bool:+ if not local:+ return True+ if local in GENERIC_IDENTIFIERS:+ return True+ return any(local.startswith(prefix) for prefix in _SERVICE_PREFIXES)+++#: Each rule is ``(name, confidence, description)``. The confidence is the+#: rule's, not the pair's: what varies between pairs is which rule fires,+#: and a rule whose confidence depends on the data is a rule nobody can+#: reason about.+LINK_RULES: dict[str, tuple[float, str]] = {+ "verified_email_match": (+ 0.98,+ "both identities carry the same address and the provider marked it verified",+ ),+ "email_match": (+ 0.92,+ "both identities carry the same email address",+ ),+ "upn_match": (+ 0.92,+ "both identities carry the same user principal name",+ ),+ "employee_id_match": (+ 0.97,+ "both identities carry the same directory employee id",+ ),+ "explicit_mapping": (+ 0.99,+ "the tenant's directory import states this mapping",+ ),+ "local_part_same_domain": (+ 0.80,+ "the local parts match and the email domains are the same",+ ),+ "local_part_only": (+ 0.45,+ "only the local parts match; different or missing domains",+ ),+ "display_name_match": (+ 0.40,+ "only the display names match, which two people routinely share",+ ),+}+++def resolve_links(records: list[dict[str, Any]]) -> ResolutionResult:+ """Link source-scoped identities pairwise, strongest rule per pair.++ ``records`` are dicts with at least ``source`` and ``id``, optionally+ ``email``, ``email_verified``, ``upn``, ``employee_id``,+ ``display_name`` and ``maps_to`` (an explicit mapping from the tenant's+ own directory import).++ Pure. The store and the graph writer are elsewhere, so this can be+ tested against fixtures with no database — which matters because the+ thing under test is a judgement about people, and the cases worth+ writing are adversarial rather than infrastructural.+ """+ cleaned = [r for r in records if r.get("source") in SOURCE_KINDS and str(r.get("id") or "").strip()]+ links: list[IdentityLink] = []+ linked: set[str] = set()++ for i, left in enumerate(cleaned):+ for right in cleaned[i + 1 :]:+ if left.get("source") == right.get("source"):+ # Two accounts in one source are two accounts. Joining them+ # is a different problem (duplicate detection) with+ # different evidence, and conflating the two is how a+ # directory's own guest copy of a user merges with the user.+ continue+ link = _best_link(left, right)+ if link is None:+ continue+ links.append(link)+ if link.merged:+ linked.add(link.left)+ linked.add(link.right)++ unlinked = sorted({_key(r) for r in cleaned} - linked)+ return ResolutionResult(links=links, unlinked=unlinked)+++def _best_link(left: dict[str, Any], right: dict[str, Any]) -> IdentityLink | None:+ """The strongest rule that fires for this pair, or ``None``."""+ lkey, rkey = _key(left), _key(right)++ explicit = {str(x).strip().lower() for x in (left.get("maps_to") or [])}+ if str(right.get("id") or "").strip().lower() in explicit or rkey.lower() in explicit:+ return _link(lkey, rkey, "explicit_mapping", [f"the directory import maps {lkey} to {rkey}"])++ lemp, remp = str(left.get("employee_id") or "").strip(), str(right.get("employee_id") or "").strip()+ if lemp and lemp == remp:+ return _link(lkey, rkey, "employee_id_match", [f"both carry employee id {lemp}"])++ lmail = str(left.get("email") or "").strip().lower()+ rmail = str(right.get("email") or "").strip().lower()+ if lmail and lmail == rmail:+ # A shared mailbox is not a person, and merging on one joins+ # everybody who can send as it.+ if _is_generic(_local_part(lmail)):+ return _link(+ lkey,+ rkey,+ "local_part_only",+ [f"both carry {lmail}, but it is a role address rather than a person"],+ )+ rule = "verified_email_match" if (left.get("email_verified") and right.get("email_verified")) else "email_match"+ return _link(lkey, rkey, rule, [f"both carry the address {lmail}"])++ lupn, rupn = str(left.get("upn") or "").strip().lower(), str(right.get("upn") or "").strip().lower()+ if lupn and lupn == rupn:+ return _link(lkey, rkey, "upn_match", [f"both carry the user principal name {lupn}"])++ llocal = _local_part(str(left.get("id") or "")) or _local_part(lmail)+ rlocal = _local_part(str(right.get("id") or "")) or _local_part(rmail)+ if llocal and llocal == rlocal:+ if _is_generic(llocal):+ # Not even a weak link. `admin` matching `admin` is not+ # evidence of anything, and returning it at 0.45 would fill a+ # review page with noise that reads as near-misses.+ return None+ ldom, rdom = _domain(lmail or str(left.get("id") or "")), _domain(rmail or str(right.get("id") or ""))+ if ldom and ldom == rdom:+ return _link(+ lkey,+ rkey,+ "local_part_same_domain",+ [f"local part {llocal!r} matches and both are in {ldom}"],+ )+ return _link(+ lkey,+ rkey,+ "local_part_only",+ [+ f"local part {llocal!r} matches",+ f"domains differ or are absent ({ldom or 'none'} vs {rdom or 'none'})",+ ],+ )++ lname = str(left.get("display_name") or "").strip().lower()+ rname = str(right.get("display_name") or "").strip().lower()+ if lname and lname == rname:+ return _link(lkey, rkey, "display_name_match", [f"both display as {lname!r}"])++ return None+++def _link(left: str, right: str, rule: str, evidence: list[str]) -> IdentityLink:+ confidence, description = LINK_RULES[rule]+ return IdentityLink(+ left=left,+ right=right,+ confidence=confidence,+ rule=rule,+ evidence=[*evidence, description],+ )diff --git a/services/api/app/services/identity_resolution_store.py b/services/api/app/services/identity_resolution_store.pynew file mode 100644index 000000000..89bc21c48--- /dev/null+++ b/services/api/app/services/identity_resolution_store.py@@ -0,0 +1,303 @@+"""One read interface over the CORE table and the `full` graph.++Depth plan wave 2, Phase 3.1. Backs migration 128.++ADR-0010 put entity context in Postgres for CORE and kept Neo4j for `full`.+Identity resolution is entity context, so it lands both ways — and the+reason this module exists rather than two writers is that two stores written+by two code paths disagree, and the disagreement is invisible: both answer,+and nobody compares them.++So :func:`save_links` writes Postgres first and mirrors to the graph+best-effort, and :func:`links_for` reads **Postgres only**. The graph copy+is for traversal in `full`, never a second source of truth; a read that+preferred the graph would answer differently on the two profiles for reasons+no operator could see.++The merge rule, restated here because this is where it is enforced+------------------------------------------------------------------+:func:`cluster_for` walks merge-strength links only. Weak links are returned+by :func:`links_for` and excluded from the walk, so a chain of three weak+links cannot merge two identities no single rule would join — which is how a+transitive merge quietly becomes the broadest possible one.+"""++from __future__ import annotations++import json+import uuid+from typing import Any++import structlog+from sqlalchemy import text+from sqlalchemy.ext.asyncio import AsyncSession++from app.services.identity_resolution import MERGE_THRESHOLD, IdentityLink, ResolutionResult++logger = structlog.get_logger(__name__)++__all__ = ["cluster_for", "decide_link", "links_for", "save_links"]++# Written out per statement rather than interpolated. `text()` with an+# f-string is a taint sink for `avoid-sqlalchemy-text`, and the rule is+# right that it cannot tell a constant column list from a caller-supplied+# one. `check_raw_sql_columns.py` compares each copy against migration 128,+# so a drifting copy is caught.+++def _row(row: Any) -> dict[str, Any]:+ evidence = row.evidence if isinstance(row.evidence, list) else json.loads(row.evidence or "[]")+ confidence = float(row.confidence)+ decided = row.decided+ return {+ "id": str(row.id),+ "left": row.left_key,+ "right": row.right_key,+ "confidence": round(confidence, 4),+ "rule": row.rule,+ "evidence": evidence,+ # A human decision overrides the rule in both directions, and the+ # two facts stay separate so a reader can see which it was.+ "merged": decided == "confirmed" or (decided != "rejected" and confidence >= MERGE_THRESHOLD),+ "decided": decided,+ "decided_by": str(row.decided_by) if row.decided_by else None,+ "decided_at": row.decided_at.isoformat() if row.decided_at else None,+ "first_seen": row.first_seen.isoformat() if row.first_seen else None,+ "last_seen": row.last_seen.isoformat() if row.last_seen else None,+ }+++def _ordered(left: str, right: str) -> tuple[str, str]:+ """A pair stored once however it was discovered.++ The migration's CHECK enforces `left_key < right_key`, so a writer that+ skipped this would get an IntegrityError rather than a silent duplicate+ under the other ordering — which is the point of having the constraint+ as well as this function.+ """+ return (left, right) if left < right else (right, left)+++async def save_links(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ result: ResolutionResult,+ mirror_to_graph: bool = True,+) -> int:+ """Persist every link, strong and weak. Returns the number written.++ Weak links are written too. Dropping them at write time would make "we+ considered this and rejected it" unrecoverable, and that sentence is+ most of the value when somebody asks why two accounts were not joined.+ """+ written = 0+ for link in result.links:+ left, right = _ordered(link.left, link.right)+ await db.execute(+ text(+ """+ INSERT INTO identity_links (+ tenant_id, left_key, right_key, confidence, rule, evidence+ ) VALUES (+ :tenant_id, :left_key, :right_key, :confidence, :rule,+ CAST(:evidence AS jsonb)+ )+ ON CONFLICT (tenant_id, left_key, right_key) DO UPDATE+ SET confidence = EXCLUDED.confidence,+ rule = EXCLUDED.rule,+ evidence = EXCLUDED.evidence,+ last_seen = NOW()+ """+ ),+ {+ "tenant_id": str(tenant_id),+ "left_key": left,+ "right_key": right,+ "confidence": float(link.confidence),+ "rule": link.rule,+ "evidence": json.dumps(list(link.evidence)),+ },+ )+ written += 1++ if mirror_to_graph:+ await _mirror_to_graph(tenant_id=tenant_id, links=result.links)+ return written+++async def _mirror_to_graph(*, tenant_id: uuid.UUID, links: list[IdentityLink]) -> int:+ """Write `SAME_PERSON` edges when the graph is enabled.++ Best-effort and never raised into the caller: the graph is a `full`+ component and CORE has none, so an unreachable Neo4j is the expected+ state on the most common profile rather than a failure.++ Only merge-strength links are mirrored. A weak link in the graph would+ be walked by every traversal that does not know to filter it, and the+ traversals are written by people who will not remember to.+ """+ strong = [link for link in links if link.merged]+ if not strong:+ return 0+ try:+ from app.db.neo4j import get_session # noqa: PLC0415 — optional component+ except Exception as exc: # noqa: BLE001+ logger.debug("identity_links.graph_module_unavailable", error=str(exc))+ return 0++ cypher = """+ MERGE (a:Identity {id: $left, tenant_id: $tenant_id})+ MERGE (b:Identity {id: $right, tenant_id: $tenant_id})+ MERGE (a)-[r:SAME_PERSON]->(b)+ SET r.confidence = $confidence,+ r.rule = $rule,+ r.tenant_id = $tenant_id,+ r.last_seen = datetime()+ """+ written = 0+ try:+ async with get_session() as session:+ for link in strong:+ left, right = _ordered(link.left, link.right)+ await session.run(+ cypher,+ left=left,+ right=right,+ tenant_id=str(tenant_id),+ confidence=float(link.confidence),+ rule=link.rule,+ )+ written += 1+ except Exception as exc: # noqa: BLE001 — CORE has no graph; this is expected there+ logger.debug("identity_links.graph_write_skipped", error=str(exc))+ return 0+ return written+++async def links_for(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ key: str,+ include_weak: bool = True,+) -> list[dict[str, Any]]:+ """Every link touching ``key``, from either end.++ Reads Postgres only. The graph copy exists for traversal in `full` and+ is never a second source of truth: a read that preferred it would answer+ differently on the two profiles for reasons no operator could see.+ """+ # The optional strength filter is a parameter, not an appended clause:+ # one statement, one plan, and no f-string reaching `text()`.+ result = await db.execute(+ text(+ """+ SELECT id, left_key, right_key, confidence, rule, evidence,+ decided, decided_by, decided_at, first_seen, last_seen+ FROM identity_links+ WHERE tenant_id = :tenant_id+ AND (left_key = :key OR right_key = :key)+ AND (+ CAST(:include_weak AS boolean)+ OR confidence >= :floor+ OR decided = 'confirmed'+ )+ ORDER BY confidence DESC, last_seen DESC+ """+ ),+ {+ "tenant_id": str(tenant_id),+ "key": key,+ "include_weak": bool(include_weak),+ "floor": MERGE_THRESHOLD,+ },+ )+ return [_row(row) for row in result.fetchall()]+++async def cluster_for(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ key: str,+ max_depth: int = 3,+) -> list[str]:+ """Every identity that is the same person as ``key``.++ Breadth-first over **merge-strength links only**, bounded. A weak link+ is never traversed, so three weak hops cannot join two identities that+ no single rule was willing to join.++ ``max_depth`` bounds the walk because an estate with one bad strong link+ would otherwise return the whole tenant, and a bounded wrong answer is+ recoverable in a way an unbounded one is not.+ """+ seen = {key}+ frontier = [key]+ for _ in range(max(1, max_depth)):+ if not frontier:+ break+ rows = await db.execute(+ text(+ """+ SELECT left_key, right_key+ FROM identity_links+ WHERE tenant_id = :tenant_id+ AND (left_key = ANY(:keys) OR right_key = ANY(:keys))+ AND (confidence >= :floor OR decided = 'confirmed')+ AND (decided IS DISTINCT FROM 'rejected')+ """+ ),+ {"tenant_id": str(tenant_id), "keys": frontier, "floor": MERGE_THRESHOLD},+ )+ discovered: list[str] = []+ for row in rows.fetchall():+ for candidate in (row.left_key, row.right_key):+ if candidate not in seen:+ seen.add(candidate)+ discovered.append(candidate)+ frontier = discovered+ return sorted(seen - {key})+++async def decide_link(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ link_id: uuid.UUID,+ decision: str,+ user_id: uuid.UUID | str,+) -> dict[str, Any] | None:+ """Record a person confirming or rejecting a link. ``None`` if not theirs.++ A confirmation promotes a weak link to merge strength and a rejection+ demotes a strong one, without touching ``confidence`` — the rule did not+ become more or less certain, a human disagreed with it, and the two+ facts stay separable.+ """+ if decision not in {"confirmed", "rejected"}:+ raise ValueError(f"decision must be 'confirmed' or 'rejected', got {decision!r}")++ result = await db.execute(+ text(+ """+ UPDATE identity_links+ SET decided = :decision,+ decided_by = :user_id,+ decided_at = NOW()+ WHERE tenant_id = :tenant_id+ AND id = :link_id+ RETURNING id, left_key, right_key, confidence, rule, evidence,+ decided, decided_by, decided_at, first_seen, last_seen+ """+ ),+ {+ "tenant_id": str(tenant_id),+ "link_id": str(link_id),+ "decision": decision,+ "user_id": str(user_id),+ },+ )+ row = result.fetchone()+ return _row(row) if row else Nonediff --git a/services/api/app/services/identity_usage_collector.py b/services/api/app/services/identity_usage_collector.pynew file mode 100644index 000000000..fe2ee4bd8--- /dev/null+++ b/services/api/app/services/identity_usage_collector.py@@ -0,0 +1,210 @@+"""The runtime-usage overlay: of what was granted, what was actually used.++Depth plan wave 2, Phase 1.1.++An effective-permission answer says what an identity *could* do. The access+review in 1.2 and the right-sizing recommendation both turn on the other+half — what it *did* — and the only place that exists is the event lake,+where every cloud and identity audit event the platform ingests lands with+its action already normalised onto ``raw_events.action``.++The one thing this module must never do+---------------------------------------+Report "used nothing" when it means "could not look". The lake is a+``full``-profile component, so on a CORE install there is nothing to query,+and an empty result there is not a measurement. Every function here returns+a :class:`UsageCollection` carrying ``measured``, and the store it writes to+keeps the same distinction: an absent row means not observed, and whether+that means "not used" or "not measured" is answered by whether a collection+ran at all.++The failure this prevents is concrete and large: a right-sizing finding+built on an unmeasured empty set recommends revoking every permission in the+estate, and it recommends it with exactly the same confidence as a correct+finding.+"""++from __future__ import annotations++import uuid+from dataclasses import dataclass, field+from datetime import UTC, datetime, timedelta+from typing import Any++import structlog+from sqlalchemy.ext.asyncio import AsyncSession++from app.db.clickhouse import execute_lake_query+from app.services.identity_privilege_store import record_usage++logger = structlog.get_logger(__name__)++__all__ = ["DEFAULT_WINDOW_DAYS", "UsageCollection", "collect_usage_for_tenant"]++#: The trailing window a right-sizing finding is computed over. Ninety days+#: rather than thirty because quarterly work is real: an access review that+#: calls a permission unused after a month flags every quarter-close job in+#: the estate, and a review that cries wolf is one nobody runs twice.+DEFAULT_WINDOW_DAYS = 90++#: Cap on rows pulled from one lake query. A principal with more distinct+#: actions than this is a wildcard-admin whose right-sizing answer is "this+#: is everything" either way, so truncating costs nothing and an unbounded+#: scan on a shared lake costs real money.+MAX_ACTIONS_PER_PRINCIPAL = 500++#: Which lake ``connector_type`` values belong to which posture provider.+#: Explicit rather than inferred: ``aws_cloudtrail`` and ``aws_security_hub``+#: both start ``aws_``, and only one of them carries API actions.+PROVIDER_CONNECTORS: dict[str, tuple[str, ...]] = {+ "aws": ("aws_cloudtrail", "aws_eventbridge"),+ "azure": ("azure_entra", "azure_activity", "azure_monitor"),+ "gcp": ("gcp_audit", "gcp_scc"),+ "gws": ("google_workspace",),+ "github": ("github",),+ "okta": ("okta",),+}+++@dataclass(frozen=True)+class UsageCollection:+ """One collection run, and whether it measured anything at all."""++ measured: bool+ principals: int = 0+ rows_written: int = 0+ window_start: datetime | None = None+ window_end: datetime | None = None+ #: Why nothing was measured, when nothing was. Present so a caller can+ #: render the reason rather than a zero.+ reason: str | None = None+ notes: list[str] = field(default_factory=list)++ def as_dict(self) -> dict[str, Any]:+ return {+ "measured": self.measured,+ "principals": self.principals,+ "rows_written": self.rows_written,+ "window_start": self.window_start.isoformat() if self.window_start else None,+ "window_end": self.window_end.isoformat() if self.window_end else None,+ "reason": self.reason,+ "notes": list(self.notes),+ }+++async def collect_usage_for_tenant(+ db: AsyncSession,+ *,+ tenant_id: uuid.UUID,+ provider: str,+ principals: list[str],+ window_days: int = DEFAULT_WINDOW_DAYS,+) -> UsageCollection:+ """Observe which actions ``principals`` exercised, and record them.++ Returns ``measured=False`` with a reason whenever the lake could not be+ asked. The caller must propagate that rather than treating zero rows as+ an observation.+ """+ connectors = PROVIDER_CONNECTORS.get(provider)+ if not connectors:+ return UsageCollection(measured=False, reason=f"no lake connector types are mapped to provider {provider!r}")+ names = [p.strip() for p in principals if p and p.strip()]+ if not names:+ return UsageCollection(measured=False, reason="no principals were supplied")++ window_end = datetime.now(tz=UTC)+ window_start = window_end - timedelta(days=max(1, window_days))++ sql = """+ SELECT user_name, action, count() AS uses,+ min(event_time) AS first_used, max(event_time) AS last_used+ FROM aisoc.raw_events+ WHERE tenant_id = %(tenant_id)s+ AND event_time >= %(window_start)s+ AND event_time <= %(window_end)s+ AND connector_type IN %(connectors)s+ AND lower(user_name) IN %(principals)s+ AND action != ''+ GROUP BY user_name, action+ ORDER BY uses DESC+ LIMIT %(limit)s+ """+ params = {+ "tenant_id": str(tenant_id),+ "window_start": window_start,+ "window_end": window_end,+ "connectors": tuple(connectors),+ "principals": tuple(n.lower() for n in names),+ "limit": MAX_ACTIONS_PER_PRINCIPAL * max(1, len(names)),+ }++ try:+ result = await execute_lake_query(sql, params=params)+ except Exception as exc: # noqa: BLE001 — an unreachable lake is not an empty one+ logger.warning("identity_usage.lake_unavailable", provider=provider, error=type(exc).__name__)+ return UsageCollection(+ measured=False,+ reason=(+ f"the event lake could not be queried ({type(exc).__name__}). Usage is unknown, not zero; "+ f"the lake is a `full`-profile component and is absent on a CORE install."+ ),+ window_start=window_start,+ window_end=window_end,+ )++ columns = list(result.columns or [])+ by_principal: dict[str, dict[str, tuple[int, datetime | None, datetime | None]]] = {}+ for row in result.rows or ():+ record = dict(zip(columns, row, strict=False))+ principal = str(record.get("user_name") or "").strip()+ action = str(record.get("action") or "").strip()+ if not principal or not action:+ continue+ by_principal.setdefault(principal, {})[action] = (+ int(record.get("uses") or 0),+ _as_dt(record.get("first_used")),+ _as_dt(record.get("last_used")),+ )++ written = 0+ for principal, observations in by_principal.items():+ written += await record_usage(+ db,+ tenant_id=tenant_id,+ provider=provider,+ principal_id=principal,+ observations=observations,+ window_start=window_start,+ window_end=window_end,+ )++ notes: list[str] = []+ silent = sorted({n.lower() for n in names} - {p.lower() for p in by_principal})+ if silent:+ # A principal with zero events in the window is a measured+ # observation -- the lake answered and it did nothing -- and that is+ # the dormant-admin finding 1.2 looks for. Recorded as a note rather+ # than as rows, because there is no action to write a row against.+ notes.append(f"{len(silent)} principal(s) had no recorded activity in the window: {', '.join(silent[:10])}")++ return UsageCollection(+ measured=True,+ principals=len(by_principal),+ rows_written=written,+ window_start=window_start,+ window_end=window_end,+ notes=notes,+ )+++def _as_dt(value: Any) -> datetime | None:+ if isinstance(value, datetime):+ return value if value.tzinfo else value.replace(tzinfo=UTC)+ if isinstance(value, str) and value.strip():+ try:+ parsed = datetime.fromisoformat(value.strip().replace("Z", "+00:00"))+ except ValueError:+ return None+ return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)+ return Nonediff --git a/services/api/app/services/investigation_tools.py b/services/api/app/services/investigation_tools.pyindex 3c282695e..b31e89cfe 100644--- a/services/api/app/services/investigation_tools.py+++ b/services/api/app/services/investigation_tools.py@@ -32,6 +32,7 @@ from __future__ import annotationsimport loggingimport uuidfrom dataclasses import dataclass, field+from datetime import UTC, datetimefrom typing import Anyfrom app.db.clickhouse import execute_lake_query@@ -489,6 +490,199 @@ async def persistence_mechanisms(tenant_id: uuid.UUID | str, hostname: str) -> Treturn unavailable("persistence_mechanisms", "persistence")+async def identity_effective_permissions(+ tenant_id: uuid.UUID | str,+ principal_id: str,+ *,+ at: str | None = None,+ provider: str | None = None,+) -> ToolResult:+ """What this identity could do, as of ``at`` — not as of now.++ The only tool here that reads Postgres rather than the lake, because+ posture is not telemetry: it is a versioned fact with an interval, and+ the interval is the whole point. An alert is triaged minutes to days+ after the event, so "what can this identity do" and "what could it do+ when it acted" are different questions, and on an alert raised before+ somebody revoked a role they have different answers — the second being+ the one that explains the alert.++ ``at`` is the event timestamp. Omitting it means *now*, which is the+ right default for an interactive question about a live identity and the+ wrong one for triage, so the triage caller always passes it.+ """+ from app.db.database import AsyncSessionLocal # noqa: PLC0415 — keeps the lake tools import-light+ from app.services.identity_privilege_store import privilege_at, usage_for_principal++ when = _parse_at(at)+ try:+ async with AsyncSessionLocal() as session:+ fact = await privilege_at(+ session,+ tenant_id=_as_uuid(tenant_id),+ principal_id=principal_id,+ at=when,+ provider=provider,+ )+ if fact is None:+ # Distinct from "this identity has no permissions". Phrased+ # so a model relaying it cannot turn it into evidence of+ # harmlessness, which is what an empty result would become.+ return ToolResult(+ tool="identity_effective_permissions",+ available=False,+ reason=(+ f"No posture snapshot covers {when.isoformat()} for {principal_id!r}. "+ f"The privilege tier is unknown — this is a gap in collection, not "+ f"evidence that the identity is unprivileged."+ ),+ query_description=f"privilege facts valid at {when.isoformat()}",+ )+ usage = await usage_for_principal(+ session,+ tenant_id=_as_uuid(tenant_id),+ provider=fact.privilege.provider,+ principal_id=fact.privilege.principal_id,+ )+ except Exception as exc: # noqa: BLE001 — a lookup failure is not an absence+ logger.warning("investigation_tool.identity_effective_permissions failed err=%s", type(exc).__name__)+ return ToolResult(+ tool="identity_effective_permissions",+ available=False,+ reason=(f"Privilege lookup failed ({type(exc).__name__}). This is a lookup failure, not an absence of privilege."),+ )++ privilege = fact.privilege+ row: dict[str, Any] = {+ "principal_id": privilege.principal_id,+ "provider": privilege.provider,+ "privilege_tier": privilege.tier,+ "tags": privilege.tags.as_dict(),+ "tags_set": privilege.tags.set_tags(),+ "evidence": privilege.evidence,+ "action_count": privilege.action_count,+ "resource_count": privilege.resource_count,+ "valid_from": fact.valid_from.isoformat() if fact.valid_from else None,+ "valid_until": fact.valid_until.isoformat() if fact.valid_until else None,+ "snapshot_was_partial": privilege.partial_source,+ # Three states, not two: measured-and-used, measured-and-unused, and+ # not measured at all. A CORE install has no lake, so the third is+ # the common case and collapsing it into the second would make every+ # permission look abandoned.+ "usage_measured": usage.measured,+ "actions_used_in_window": sorted(usage.used) if usage.measured else None,+ "usage_window": (+ {+ "start": usage.window_start.isoformat() if usage.window_start else None,+ "end": usage.window_end.isoformat() if usage.window_end else None,+ "source": usage.source,+ }+ if usage.measured+ else None+ ),+ }+ return ToolResult(+ tool="identity_effective_permissions",+ rows=[row],+ row_count=1,+ query_description=(f"effective permissions for {privilege.provider}:{privilege.principal_id} as of {when.isoformat()}"),+ )+++async def changes_before_alert(+ tenant_id: uuid.UUID | str,+ subjects: str,+ *,+ at: str | None = None,+ days: int = 7,+) -> ToolResult:+ """Every recorded change touching these identities before the alert.++ Depth plan wave 2, 1.5. ``subjects`` is a comma-separated list of+ identities or resources, because the question is usually about a person+ *and* the thing they touched, and two tool calls to ask one question is+ how a model runs out of its step budget.++ Reads versioned posture and the audit lake, and says which of the two+ answered. "Nothing changed" and "we could not look" are returned as+ different results, which is the whole reason this is a tool rather than+ a query: the second must never read as the first.+ """+ from app.db.database import AsyncSessionLocal # noqa: PLC0415 — keeps the lake tools import-light+ from app.services.change_timeline import changes_before++ names = [s.strip() for s in str(subjects or "").split(",") if s.strip()]+ if not names:+ return ToolResult(+ tool="changes_before_alert",+ available=False,+ reason="No identity or resource was named, so there was nothing to look up.",+ )++ when = _parse_at(at)+ try:+ async with AsyncSessionLocal() as session:+ timeline = await changes_before(+ session,+ tenant_id=_as_uuid(tenant_id),+ subjects=names,+ at=when,+ days=days,+ )+ except Exception as exc: # noqa: BLE001+ logger.warning("investigation_tool.changes_before_alert failed err=%s", type(exc).__name__)+ return ToolResult(+ tool="changes_before_alert",+ available=False,+ reason=(f"The change lookup failed ({type(exc).__name__}). This is a lookup failure, not an absence of change."),+ )++ payload = timeline.as_dict()+ if not timeline.measured:+ return ToolResult(+ tool="changes_before_alert",+ available=False,+ reason=payload["reason"],+ query_description=f"changes to {', '.join(names)} in the {days} day(s) before {when.isoformat()}",+ )++ return ToolResult(+ tool="changes_before_alert",+ rows=payload["changes"],+ row_count=payload["change_count"],+ # Carried even on a zero-row answer, because "read both sources and+ # found nothing" is a measured result and has to read like one.+ reason=payload.get("reason"),+ query_description=(+ f"changes to {', '.join(names)} in the {days} day(s) before {when.isoformat()}; "+ f"read {', '.join(payload['sources_read'])}"+ + (f"; could not read {', '.join(payload['sources_failed'])}" if payload["sources_failed"] else "")+ ),+ )+++def _parse_at(at: str | None) -> datetime:+ """An ISO timestamp, or now. An unparseable one is now, with no pretence.++ Raising would turn a model's sloppy timestamp into a failed triage step;+ silently reading it as the epoch would return "no snapshot covers 1970"+ and read as a collection gap. Falling back to now is the only option+ whose wrongness is visible in the answer, which echoes back the time it+ actually used.+ """+ if not at:+ return datetime.now(tz=UTC)+ try:+ parsed = datetime.fromisoformat(str(at).replace("Z", "+00:00"))+ except ValueError:+ return datetime.now(tz=UTC)+ return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)+++def _as_uuid(value: uuid.UUID | str) -> uuid.UUID:+ return value if isinstance(value, uuid.UUID) else uuid.UUID(str(value))++#: Name -> coroutine. The dispatch table the API endpoint and the agent#: registry both read, so the two cannot drift.TOOLS: dict[str, Any] = {@@ -499,6 +693,8 @@ TOOLS: dict[str, Any] = {"fleet_ioc_hunt": fleet_ioc_hunt,"entity_timeline": entity_timeline,"technique_activity": technique_activity,+ "identity_effective_permissions": identity_effective_permissions,+ "changes_before_alert": changes_before_alert,"process_tree": process_tree,"mailbox_activity": mailbox_activity,"oauth_grants": oauth_grants,@@ -516,6 +712,14 @@ BACKED_TOOLS = frozenset("fleet_ioc_hunt","entity_timeline","technique_activity",+ # Backed by `identity_privilege_facts` (migration 124) rather than+ # the lake, which is why it is the one entry here that can answer on+ # a CORE install with no ClickHouse.+ "identity_effective_permissions",+ # Reads both: versioned posture in Postgres and audit events in+ # the lake. It reports which of the two answered, so a CORE install+ # gets the posture half and is told the other was not read.+ "changes_before_alert",})diff --git a/services/api/migrations/124_identity_privilege_facts.sql b/services/api/migrations/124_identity_privilege_facts.sqlnew file mode 100644index 000000000..adc4ced2e--- /dev/null+++ b/services/api/migrations/124_identity_privilege_facts.sql@@ -0,0 +1,205 @@+-- 124: derived privilege facts per identity, and the runtime-usage overlay+-- (depth plan wave 2, Phase 1.1).+--+-- Why derived facts and not "just resolve on read"+-- ------------------------------------------------+-- Migration 103 stores a provider's whole posture as one versioned jsonb+-- document, and the resolver computes a principal's effective permissions+-- from it. That is the right shape for an analyst asking about one identity+-- and the wrong one for three callers that need an answer on the hot path:+--+-- * fusion stamps a privilege tier on every alert at fuse time, and cannot+-- parse an estate-sized policy document per event;+-- * the queue sorts by priority, which reads the tier;+-- * the access review (1.2) scans every principal at once.+--+-- So the expensive resolution runs once per posture version and lands here as+-- a small row per principal. The rows are versioned with the same+-- valid_from / valid_until interval as the snapshot they came from, so "what+-- could this identity do when it acted" stays answerable -- which is the+-- whole reason 103 is versioned, and a derived table that collapsed to one+-- mutable row per principal would quietly undo it.+--+-- Why `unknown` is a stored value and not a NULL tier+-- ---------------------------------------------------+-- A principal with no posture covering an event time is a real and common+-- state: the tenant connected the provider last week and the alert is from+-- last month. Defaulting it to 'standard' is the confident-zero defect --+-- a reader cannot tell a measured standard account from one the platform+-- knows nothing about, and the second is the one worth escalating. So+-- 'unknown' is a first-class tier that every surface renders as unknown.+--+-- The four tags are `boolean NULL` for the same reason: NULL means "no+-- snapshot could answer this", false means "the snapshot says no".++CREATE TABLE IF NOT EXISTS identity_privilege_facts (+ id uuid PRIMARY KEY DEFAULT gen_random_uuid(),+ tenant_id uuid NOT NULL,++ provider text NOT NULL,+ -- The provider-scoped id the resolver was asked about (AWS ARN or user+ -- id, Entra object id, GCP member, Okta user id, Workspace user key,+ -- GitHub login).+ principal_id text NOT NULL,+ -- A second, human-readable handle when the provider has one. Kept+ -- alongside rather than instead of `principal_id`, because an alert+ -- carries whichever the connector emitted and a join on the wrong one+ -- silently matches nothing.+ principal_label text,+ principal_kind text,++ tier text NOT NULL DEFAULT 'unknown',++ -- The four computed tags. NULL is "no snapshot could answer", which is+ -- not the same as false and must not render as one.+ can_escalate boolean,+ data_access boolean,+ third_party boolean,+ cross_account boolean,++ -- Which actions and which policy chain drove each tag. An analyst who+ -- cannot see why an identity was tagged `can_escalate` has a label, not+ -- a finding.+ evidence jsonb NOT NULL DEFAULT '{}'::jsonb,++ action_count integer NOT NULL DEFAULT 0,+ resource_count integer NOT NULL DEFAULT 0,++ -- The snapshot this was derived from. ON DELETE CASCADE because a fact+ -- whose snapshot is gone cannot be re-derived or audited, and keeping it+ -- would be a number with no provenance.+ snapshot_id uuid REFERENCES posture_snapshots(id) ON DELETE CASCADE,+ resolver_version text NOT NULL DEFAULT '',+ -- True when the snapshot it came from was partial. Carried down rather+ -- than looked up, so a surface reading only this table can still say+ -- "this answer is incomplete".+ partial_source boolean NOT NULL DEFAULT false,++ valid_from timestamptz NOT NULL DEFAULT now(),+ valid_until timestamptz,+ created_at timestamptz NOT NULL DEFAULT now(),++ CONSTRAINT identity_privilege_facts_provider_check CHECK (+ provider IN ('aws', 'azure', 'gcp', 'gws', 'github', 'okta')+ ),+ CONSTRAINT identity_privilege_facts_tier_check CHECK (+ tier IN ('unknown', 'standard', 'elevated', 'admin', 'domain_admin')+ ),+ CONSTRAINT identity_privilege_facts_interval_check CHECK (+ valid_until IS NULL OR valid_until >= valid_from+ )+);++-- At most one live fact per principal per provider per tenant. Same+-- invariant, and same reason, as posture_snapshots_live_idx: two live rows+-- make "the current tier" whichever the planner returned first.+CREATE UNIQUE INDEX IF NOT EXISTS identity_privilege_facts_live_idx+ ON identity_privilege_facts (tenant_id, provider, principal_id)+ WHERE valid_until IS NULL;++-- The point-in-time lookup fusion makes per alert.+CREATE INDEX IF NOT EXISTS identity_privilege_facts_at_idx+ ON identity_privilege_facts (tenant_id, provider, principal_id, valid_from DESC);++-- The access review's scan: every principal carrying a tag, newest first.+CREATE INDEX IF NOT EXISTS identity_privilege_facts_tags_idx+ ON identity_privilege_facts (tenant_id, provider, tier)+ WHERE valid_until IS NULL;++-- Fusion joins on whichever handle the connector emitted, so the label needs+-- its own index or the stamp falls back to a sequential scan per alert.+CREATE INDEX IF NOT EXISTS identity_privilege_facts_label_idx+ ON identity_privilege_facts (tenant_id, lower(principal_label))+ WHERE valid_until IS NULL AND principal_label IS NOT NULL;++COMMENT ON TABLE identity_privilege_facts IS+ 'Per-identity privilege tier and computed tags, derived once per posture version. Versioned on the '+ 'same interval as the snapshot it came from so an alert reads the tier that was true when the '+ 'identity acted, not the one true now.';++COMMENT ON COLUMN identity_privilege_facts.tier IS+ 'unknown | standard | elevated | admin | domain_admin. unknown is a measured state (no snapshot '+ 'covers this time), never a default, and every surface renders it as unknown.';++COMMENT ON COLUMN identity_privilege_facts.can_escalate IS+ 'NULL means no snapshot could answer. false means the snapshot says no. The two are different '+ 'facts and a surface that renders them the same is fabricating one of them.';+++-- The runtime-usage overlay: of what a principal was granted, what did it+-- actually exercise?+--+-- Sourced from the event lake, which lives in the `full` profile. A CORE+-- install has no lake, and the honest consequence is that usage is *not+-- measured* there -- which is a different answer from "used nothing". The+-- difference matters more here than almost anywhere else in the product: a+-- right-sizing recommendation built on "used nothing" tells an operator to+-- revoke every permission in the estate.+--+-- `window_start` / `window_end` are on the row rather than implied by a+-- config value, because a finding that says "unused in the window" is+-- meaningless without the window, and the window changes.+CREATE TABLE IF NOT EXISTS identity_permission_usage (+ id uuid PRIMARY KEY DEFAULT gen_random_uuid(),+ tenant_id uuid NOT NULL,++ provider text NOT NULL,+ principal_id text NOT NULL,+ -- Provider-namespaced, matching what the resolver returns, so granted+ -- and used can be compared without a translation table in between.+ action text NOT NULL,++ use_count bigint NOT NULL DEFAULT 0,+ first_used_at timestamptz,+ last_used_at timestamptz,++ window_start timestamptz NOT NULL,+ window_end timestamptz NOT NULL,++ -- Where the observation came from, so an operator can tell a lake-backed+ -- measurement from a connector-backed one.+ source text NOT NULL DEFAULT 'lake',+ computed_at timestamptz NOT NULL DEFAULT now(),++ CONSTRAINT identity_permission_usage_provider_check CHECK (+ provider IN ('aws', 'azure', 'gcp', 'gws', 'github', 'okta')+ ),+ CONSTRAINT identity_permission_usage_window_check CHECK (window_end >= window_start)+);++CREATE UNIQUE INDEX IF NOT EXISTS identity_permission_usage_unique_idx+ ON identity_permission_usage (tenant_id, provider, principal_id, action);++CREATE INDEX IF NOT EXISTS identity_permission_usage_principal_idx+ ON identity_permission_usage (tenant_id, provider, principal_id, last_used_at DESC);++COMMENT ON TABLE identity_permission_usage IS+ 'Granted actions a principal actually exercised in a trailing window, from the event lake. An '+ 'absent row means not observed; on a CORE install with no lake that means not measured, which is '+ 'not the same as never used and must never be rendered as it.';+++DO $$+BEGIN+ EXECUTE 'ALTER TABLE identity_privilege_facts ENABLE ROW LEVEL SECURITY';+ EXECUTE 'ALTER TABLE identity_privilege_facts FORCE ROW LEVEL SECURITY';+ EXECUTE 'DROP POLICY IF EXISTS tenant_isolation ON identity_privilege_facts';+ EXECUTE+ 'CREATE POLICY tenant_isolation ON identity_privilege_facts '+ 'USING (tenant_id = current_tenant_id() OR current_tenant_id() IS NULL) '+ 'WITH CHECK (tenant_id = current_tenant_id())';++ EXECUTE 'ALTER TABLE identity_permission_usage ENABLE ROW LEVEL SECURITY';+ EXECUTE 'ALTER TABLE identity_permission_usage FORCE ROW LEVEL SECURITY';+ EXECUTE 'DROP POLICY IF EXISTS tenant_isolation ON identity_permission_usage';+ EXECUTE+ 'CREATE POLICY tenant_isolation ON identity_permission_usage '+ 'USING (tenant_id = current_tenant_id() OR current_tenant_id() IS NULL) '+ 'WITH CHECK (tenant_id = current_tenant_id())';++ IF EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'aisoc_app') THEN+ EXECUTE 'GRANT SELECT, INSERT, UPDATE, DELETE ON identity_privilege_facts TO aisoc_app';+ EXECUTE 'GRANT SELECT, INSERT, UPDATE, DELETE ON identity_permission_usage TO aisoc_app';+ END IF;+END+$$;diff --git a/services/api/migrations/125_access_review_findings.sql b/services/api/migrations/125_access_review_findings.sqlnew file mode 100644index 000000000..4117d50e9--- /dev/null+++ b/services/api/migrations/125_access_review_findings.sql@@ -0,0 +1,154 @@+-- 125: access-review findings (depth plan wave 2, Phase 1.2).+--+-- Read-only by construction+-- -------------------------+-- Every row here names an object a human should go and change, and nothing+-- in the product acts on one. That is a scope decision from the plan, and+-- it is also what makes the review safe to run across a whole estate: a+-- review that could act is one nobody enables.+--+-- Why the findings are stored rather than recomputed per request+-- --------------------------------------------------------------+-- Computing them needs the posture snapshot, the derived privilege facts and+-- the usage overlay joined per principal, which is an estate-sized scan. The+-- console page is a list a reviewer works through over days, with assignment+-- and acknowledgement, so the rows need identity and lifecycle of their own.+--+-- The honest-empty-state column+-- -----------------------------+-- `usage_measured` travels on the row. A finding computed where usage could+-- not be measured is a different, weaker claim than one computed where it+-- could, and a page that cannot tell them apart shows a short list and lets+-- a reviewer read it as a clean estate. The same reason `run_state` exists+-- on the run table below: zero findings over zero principals is "nothing+-- reviewed", never "no findings".++CREATE TABLE IF NOT EXISTS access_review_runs (+ id uuid PRIMARY KEY DEFAULT gen_random_uuid(),+ tenant_id uuid NOT NULL,+ provider text NOT NULL,++ -- 'nothing_reviewed' | 'no_findings' | 'findings' | 'failed'.+ -- `nothing_reviewed` is the state the whole table exists to be able to+ -- express: an estate nobody could look at must not render identically+ -- to one with nothing wrong.+ run_state text NOT NULL DEFAULT 'nothing_reviewed',++ reviewed_principals integer NOT NULL DEFAULT 0,+ -- How many of those had usage data behind them. Lower than+ -- reviewed_principals means a partial review, and the page says so.+ usage_measured_for integer NOT NULL DEFAULT 0,+ finding_count integer NOT NULL DEFAULT 0,++ -- The trailing window the usage half was computed over. A finding that+ -- says "unused" is meaningless without it, and the window changes.+ window_start timestamptz,+ window_end timestamptz,++ note text,+ started_at timestamptz NOT NULL DEFAULT now(),+ completed_at timestamptz,++ CONSTRAINT access_review_runs_provider_check CHECK (+ provider IN ('aws', 'azure', 'gcp', 'gws', 'github', 'okta')+ ),+ CONSTRAINT access_review_runs_state_check CHECK (+ run_state IN ('nothing_reviewed', 'no_findings', 'findings', 'failed')+ )+);++CREATE INDEX IF NOT EXISTS access_review_runs_recent_idx+ ON access_review_runs (tenant_id, provider, started_at DESC);+++CREATE TABLE IF NOT EXISTS access_review_findings (+ id uuid PRIMARY KEY DEFAULT gen_random_uuid(),+ tenant_id uuid NOT NULL,+ run_id uuid NOT NULL REFERENCES access_review_runs(id) ON DELETE CASCADE,++ kind text NOT NULL,+ provider text NOT NULL,+ principal_id text NOT NULL,+ severity text NOT NULL DEFAULT 'medium',++ title text NOT NULL,+ detail text NOT NULL,+ -- The object a person should change. Required, not nullable: a+ -- read-only finding that does not say where to go leaves the reviewer+ -- with the hard half of the work, which is why reviews do not happen.+ object_hint text NOT NULL,+ recommendation text NOT NULL,+ -- The smallest built-in role covering everything the principal actually+ -- used, where one exists. NULL means either usage was not measured or+ -- no built-in covers the used set -- and the second is a real answer+ -- for a principal doing genuinely broad work, not a failure.+ best_fit_role text,++ evidence jsonb NOT NULL DEFAULT '{}'::jsonb,+ -- False means this finding was computed without usage data behind it.+ -- Usage-dependent kinds are never emitted in that case, so a false here+ -- marks the weaker non-usage findings for what they are.+ usage_measured boolean NOT NULL DEFAULT false,++ -- Reviewer lifecycle. Acknowledgement is the only state transition, and+ -- it records a person rather than a flag, because "who decided this was+ -- fine" is the question an auditor asks.+ acknowledged_at timestamptz,+ acknowledged_by uuid,+ acknowledged_note text,++ first_seen timestamptz NOT NULL DEFAULT now(),+ last_seen timestamptz NOT NULL DEFAULT now(),++ CONSTRAINT access_review_findings_provider_check CHECK (+ provider IN ('aws', 'azure', 'gcp', 'gws', 'github', 'okta')+ ),+ CONSTRAINT access_review_findings_severity_check CHECK (+ severity IN ('info', 'low', 'medium', 'high', 'critical')+ )+);++-- One open finding per (principal, kind) per tenant. A review that runs+-- nightly must not accumulate ninety copies of the same sentence; a repeat+-- extends `last_seen` on the row the reviewer has already seen, and keeps+-- their acknowledgement attached to it.+CREATE UNIQUE INDEX IF NOT EXISTS access_review_findings_unique_idx+ ON access_review_findings (tenant_id, provider, principal_id, kind);++CREATE INDEX IF NOT EXISTS access_review_findings_open_idx+ ON access_review_findings (tenant_id, provider, severity, last_seen DESC)+ WHERE acknowledged_at IS NULL;++COMMENT ON TABLE access_review_findings IS+ 'Read-only access-review findings. Nothing in the product acts on one: each names the object a '+ 'human should change and stops.';++COMMENT ON COLUMN access_review_findings.usage_measured IS+ 'False when no usage data backed this review. Usage-dependent kinds are suppressed entirely in '+ 'that case rather than caveated, because a caveat on "revoke this" is read by nobody.';+++DO $$+BEGIN+ EXECUTE 'ALTER TABLE access_review_runs ENABLE ROW LEVEL SECURITY';+ EXECUTE 'ALTER TABLE access_review_runs FORCE ROW LEVEL SECURITY';+ EXECUTE 'DROP POLICY IF EXISTS tenant_isolation ON access_review_runs';+ EXECUTE+ 'CREATE POLICY tenant_isolation ON access_review_runs '+ 'USING (tenant_id = current_tenant_id() OR current_tenant_id() IS NULL) '+ 'WITH CHECK (tenant_id = current_tenant_id())';++ EXECUTE 'ALTER TABLE access_review_findings ENABLE ROW LEVEL SECURITY';+ EXECUTE 'ALTER TABLE access_review_findings FORCE ROW LEVEL SECURITY';+ EXECUTE 'DROP POLICY IF EXISTS tenant_isolation ON access_review_findings';+ EXECUTE+ 'CREATE POLICY tenant_isolation ON access_review_findings '+ 'USING (tenant_id = current_tenant_id() OR current_tenant_id() IS NULL) '+ 'WITH CHECK (tenant_id = current_tenant_id())';++ IF EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'aisoc_app') THEN+ EXECUTE 'GRANT SELECT, INSERT, UPDATE, DELETE ON access_review_runs TO aisoc_app';+ EXECUTE 'GRANT SELECT, INSERT, UPDATE, DELETE ON access_review_findings TO aisoc_app';+ END IF;+END+$$;diff --git a/services/api/migrations/128_identity_links.sql b/services/api/migrations/128_identity_links.sqlnew file mode 100644index 000000000..81204b274--- /dev/null+++ b/services/api/migrations/128_identity_links.sql@@ -0,0 +1,106 @@+-- 128: resolved identity links (depth plan wave 2, Phase 3.1).+--+-- Why Postgres and not only the graph+-- -----------------------------------+-- ADR-0010 measured Neo4j at 696.8 MiB resident for CORE's entity context,+-- which is 42.2% of what the CORE profile has left while it is triaging an+-- alert, and put the entity and session store in Postgres for CORE with the+-- graph kept for `full`. Identity resolution is entity context, so it lands+-- the same way: this table in CORE, `SAME_PERSON` edges in the graph when it+-- is enabled, written from one code path so the two cannot disagree.+--+-- A CORE install that could not resolve identities would be unable to build+-- a story across sources at all, which is the profile most evaluators run.+--+-- Why confidence is a column and not a filter applied at write time+-- -----------------------------------------------------------------+-- A weak link is useful to an analyst reading a story and dangerous to+-- anything that merges. Those are different consumers and they need+-- different answers from one store, so every link is written with its+-- confidence and the reader decides. Dropping sub-threshold links at write+-- time would make "we considered this and rejected it" unrecoverable, and+-- that sentence is most of the value when somebody asks why two accounts+-- were not joined.+--+-- The merge threshold itself lives in+-- `services/api/app/services/identity_resolution.py` and is deliberately not+-- duplicated here: a threshold in two places is a threshold that drifts, and+-- a CHECK constraint enforcing one would make tuning it a migration.++CREATE TABLE IF NOT EXISTS identity_links (+ id uuid PRIMARY KEY DEFAULT gen_random_uuid(),+ tenant_id uuid NOT NULL,++ -- Source-scoped identifiers, `<source>:<id>`, ordered so a pair is+ -- stored once however it was discovered. Ordering is applied by the+ -- writer, and the unique index below is what makes it matter.+ left_key text NOT NULL,+ right_key text NOT NULL,++ -- 0.0-1.0. The rule's confidence, not the pair's: what varies between+ -- pairs is which rule fired, and a confidence that depends on the data+ -- is one nobody can reason about.+ confidence double precision NOT NULL,+ rule text NOT NULL,+ -- Why this link was proposed, in sentences. A link an analyst cannot+ -- interrogate is one they either trust blindly or ignore entirely.+ evidence jsonb NOT NULL DEFAULT '[]'::jsonb,++ -- A person's decision about the link, where one was made. Separate from+ -- `confidence` because a human confirming a weak link is a different+ -- fact from the rule becoming more certain, and overwriting the+ -- confidence would lose which it was.+ decided text,+ decided_by uuid,+ decided_at timestamptz,++ first_seen timestamptz NOT NULL DEFAULT now(),+ last_seen timestamptz NOT NULL DEFAULT now(),++ CONSTRAINT identity_links_confidence_check CHECK (confidence >= 0.0 AND confidence <= 1.0),+ CONSTRAINT identity_links_decided_check CHECK (+ decided IS NULL OR decided IN ('confirmed', 'rejected')+ ),+ -- A link from an identity to itself is not a link, and would make every+ -- cluster contain one spurious self-edge.+ CONSTRAINT identity_links_distinct_check CHECK (left_key <> right_key),+ -- The writer orders the pair; this makes a violation loud rather than a+ -- silent duplicate under the other ordering.+ CONSTRAINT identity_links_ordered_check CHECK (left_key < right_key)+);++CREATE UNIQUE INDEX IF NOT EXISTS identity_links_pair_idx+ ON identity_links (tenant_id, left_key, right_key);++-- "What else is this person" — the lookup every consumer makes, from either+-- end of the pair.+CREATE INDEX IF NOT EXISTS identity_links_left_idx+ ON identity_links (tenant_id, left_key, confidence DESC);+CREATE INDEX IF NOT EXISTS identity_links_right_idx+ ON identity_links (tenant_id, right_key, confidence DESC);++COMMENT ON TABLE identity_links IS+ 'Resolved links between one person''s identities across IdP, EDR, proxy, cloud, code platform '+ 'and Workspace. Every link carries its rule, confidence and evidence; a low-confidence link is '+ 'stored and shown and never used to merge.';++COMMENT ON COLUMN identity_links.decided IS+ 'A person''s confirmation or rejection. Kept separate from confidence because a human '+ 'confirming a weak link is a different fact from the rule becoming more certain.';+++DO $$+BEGIN+ EXECUTE 'ALTER TABLE identity_links ENABLE ROW LEVEL SECURITY';+ EXECUTE 'ALTER TABLE identity_links FORCE ROW LEVEL SECURITY';+ EXECUTE 'DROP POLICY IF EXISTS tenant_isolation ON identity_links';+ EXECUTE+ 'CREATE POLICY tenant_isolation ON identity_links '+ 'USING (tenant_id = current_tenant_id() OR current_tenant_id() IS NULL) '+ 'WITH CHECK (tenant_id = current_tenant_id())';++ IF EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'aisoc_app') THEN+ EXECUTE 'GRANT SELECT, INSERT, UPDATE, DELETE ON identity_links TO aisoc_app';+ END IF;+END+$$;diff --git a/services/api/tests/live_rls.py b/services/api/tests/live_rls.pynew file mode 100644index 000000000..84887fce9--- /dev/null+++ b/services/api/tests/live_rls.py@@ -0,0 +1,48 @@+"""Bind a live test's session to a tenant, the way production writers do.++The wave-2 identity tables (``identity_privilege_facts``,+``identity_permission_usage``, ``access_review_findings``,+``access_review_runs``, ``identity_links``) carry the house RLS policy in its+strict form::++ USING (tenant_id = current_tenant_id() OR current_tenant_id() IS NULL)+ WITH CHECK (tenant_id = current_tenant_id())++There is deliberately no unbound-session arm in ``WITH CHECK``. 36 of the 43+policies in the migration chain are written this way and it is the stronger+form: the arm that is missing here is the one that made RLS fail open on the+connectors engine, which never bound the GUC at all.++So a write issued on a session with no tenant bound is refused by Postgres,+and that is the policy working rather than an obstacle. Production writes+arrive either on a session ``app/db/rls.py``'s ``get_tenant_db`` has bound,+or inside a worker that calls ``set_config`` once per tenant the way+``hunt_scheduler`` and ``retro_hunt_consumer`` do. A live suite that wrote+without binding would be exercising a path no caller takes — and it cannot+even reach the property it is testing, because the insert never lands.++Two properties worth keeping in mind when using this:++* ``SET LOCAL`` ends with the transaction, so a ``commit()`` after the write+ drops back to an unbound session. Bind immediately before each write.+* Reads are deliberately left unbound. The cross-tenant assertions in these+ suites are about the tenant predicate each store puts in its *own* SQL, and+ binding the reader would let a store that forgot its predicate pass behind+ RLS — the assertion would still hold, for a reason that is not the one in+ the test's name.+"""++from __future__ import annotations++import uuid++from sqlalchemy import text+from sqlalchemy.ext.asyncio import AsyncSession+++async def bind_tenant(session: AsyncSession, tenant: uuid.UUID) -> None:+ """Scope the next write on ``session`` to ``tenant`` until it commits."""+ await session.execute(+ text("SELECT set_config('app.current_tenant_id', :t, TRUE)"),+ {"t": str(tenant)},+ )diff --git a/services/api/tests/test_access_review_live.py b/services/api/tests/test_access_review_live.pynew file mode 100644index 000000000..308b16447--- /dev/null+++ b/services/api/tests/test_access_review_live.py@@ -0,0 +1,290 @@+"""Access review against real Postgres (depth plan wave 2, Phase 1.2).++Live because the properties are schema facts a stand-in would invent: the+unique index that turns a nightly re-run into an update rather than a+ninetieth copy, the `(principal, kind) <> ALL` delete that retires a+finding somebody fixed, the tenant predicate on every statement, and the+acknowledgement write landing on the caller's own row only.++Each test carries a negative control. Skips without a database rather than+passing without one; fails rather than skips under+``MSSP_ISOLATION_REQUIRED``.+"""++from __future__ import annotations++import os+import uuid+from datetime import UTC, datetime, timedelta++import pytest+import pytest_asyncio+from app.services.access_review_store import acknowledge, list_findings, run_review, summary+from app.services.identity_privilege import IdentityPrivilege, PrivilegeTags+from app.services.identity_privilege_store import record_usage, save_privilege+from live_rls import bind_tenant+from sqlalchemy import text+from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine++DSN = os.environ.get("DATABASE_URL", "")+REQUIRED = os.environ.get("MSSP_ISOLATION_REQUIRED", "").strip() not in ("", "0", "false")++TENANT = uuid.UUID("0e720000-0000-0000-0000-0000000000b1")+OTHER_TENANT = uuid.UUID("0e720000-0000-0000-0000-0000000000b2")+USER = uuid.UUID("0e720000-0000-0000-0000-0000000000b3")++ADMIN_ARN = "arn:aws:iam::1:user/root-ish"+++@pytest_asyncio.fixture+async def db():+ if "postgres" not in DSN and not REQUIRED:+ pytest.skip("needs a live Postgres with the migration chain applied (integration.yml)")+ engine = create_async_engine(DSN)+ try:+ async with engine.connect() as probe:+ await probe.execute(text("SELECT 1"))+ except Exception as exc:+ await engine.dispose()+ if REQUIRED:+ pytest.fail(f"MSSP_ISOLATION_REQUIRED is set and no database answered at DATABASE_URL: {type(exc).__name__}: {exc}")+ pytest.skip(f"no database at DATABASE_URL ({type(exc).__name__}) — runs in integration.yml")++ factory = async_sessionmaker(engine, expire_on_commit=False)+ async with factory() as session:+ for tid, slug in ((TENANT, "review-live-a"), (OTHER_TENANT, "review-live-b")):+ await session.execute(+ text("INSERT INTO tenants (id, name, slug) VALUES (:id, :name, :slug) ON CONFLICT (id) DO NOTHING"),+ {"id": tid, "name": f"Review live {slug}", "slug": slug},+ )+ await session.commit()+ yield session+ await session.rollback()+ for table in (+ "access_review_findings",+ "access_review_runs",+ "identity_permission_usage",+ "identity_privilege_facts",+ ):+ await session.execute(+ text(f"DELETE FROM {table} WHERE tenant_id IN (:a, :b)"), # noqa: S608 — fixed table names+ {"a": TENANT, "b": OTHER_TENANT},+ )+ await session.commit()+ await engine.dispose()+++async def _seed_admin(db, tenant_id=TENANT, *, principal: str = ADMIN_ARN) -> None:+ await bind_tenant(db, tenant_id)+ await save_privilege(+ db,+ tenant_id=tenant_id,+ privilege=IdentityPrivilege(+ tier="admin",+ tags=PrivilegeTags(can_escalate=True, third_party=None),+ provider="aws",+ principal_id=principal,+ evidence={"can_escalate": ["iam:PutUserPolicy", "iam:CreateAccessKey"]},+ action_count=40,+ ),+ principal_label="root-ish",+ )+ await db.commit()+++async def _seed_usage(db, *, principal: str = ADMIN_ARN, days_ago: int, action: str = "s3:GetObject") -> None:+ now = datetime.now(tz=UTC)+ when = now - timedelta(days=days_ago)+ await bind_tenant(db, TENANT)+ await record_usage(+ db,+ tenant_id=TENANT,+ provider="aws",+ principal_id=principal,+ observations={action: (3, when, when)},+ window_start=now - timedelta(days=90),+ window_end=now,+ )+ await db.commit()+++async def test_a_review_with_no_posture_is_nothing_reviewed_not_a_clean_bill(db) -> None:+ await bind_tenant(db, TENANT)+ result = await run_review(db, tenant_id=TENANT, provider="aws")+ await db.commit()++ assert result["state"] == "nothing_reviewed"+ assert result["reviewed_principals"] == 0+ assert result["findings"] == 0+ # The sentence is the point: a reader must not take the zero for an+ # all-clear, and the only thing standing between them and that reading+ # is this string.+ assert "not a clean bill of health" in result["note"]+++async def test_unmeasured_usage_suppresses_the_usage_findings(db) -> None:+ """No lake means no unused-access, dormancy or right-sizing findings."""+ await _seed_admin(db)+ await bind_tenant(db, TENANT)+ result = await run_review(db, tenant_id=TENANT, provider="aws")+ await db.commit()++ kinds = {f["kind"] for f in await list_findings(db, tenant_id=TENANT)}+ assert "unused_write_access" not in kinds+ assert "dormant_admin" not in kinds+ assert result["usage_measured_for"] == 0+ assert "partial review" in (result.get("note") or "")++ # Negative control: give the same principal measured usage with the+ # grants unexercised, and the suppressed findings appear.+ await _seed_usage(db, days_ago=200)+ await bind_tenant(db, TENANT)+ await run_review(db, tenant_id=TENANT, provider="aws")+ await db.commit()+ kinds_after = {f["kind"] for f in await list_findings(db, tenant_id=TENANT)}+ assert "dormant_admin" in kinds_after+ assert "escalation_without_use" in kinds_after+++async def test_a_nightly_rerun_updates_rather_than_duplicates(db) -> None:+ await _seed_admin(db)+ await _seed_usage(db, days_ago=200)+ for _ in range(3):+ await bind_tenant(db, TENANT)+ await run_review(db, tenant_id=TENANT, provider="aws")+ await db.commit()++ rows = await list_findings(db, tenant_id=TENANT)+ keys = [(r["principal_id"], r["kind"]) for r in rows]+ assert len(keys) == len(set(keys)), f"duplicate findings accumulated: {keys}"++ # Control: three runs really did happen, so the de-duplication is doing+ # the work rather than two of the runs having silently found nothing.+ runs = (+ await db.execute(+ text("SELECT count(*) FROM access_review_runs WHERE tenant_id = :t"),+ {"t": TENANT},+ )+ ).scalar_one()+ assert runs == 3+++async def test_a_finding_that_stops_reproducing_is_retired(db) -> None:+ """A page listing a permission somebody removed is a page nobody opens."""+ await _seed_admin(db)+ await _seed_usage(db, days_ago=200)+ await bind_tenant(db, TENANT)+ await run_review(db, tenant_id=TENANT, provider="aws")+ await db.commit()+ assert {f["kind"] for f in await list_findings(db, tenant_id=TENANT)} >= {"dormant_admin"}++ # The admin is demoted and starts being used: both findings should go.+ await bind_tenant(db, TENANT)+ await save_privilege(+ db,+ tenant_id=TENANT,+ privilege=IdentityPrivilege(+ tier="standard",+ tags=PrivilegeTags(),+ provider="aws",+ principal_id=ADMIN_ARN,+ evidence={},+ action_count=2,+ ),+ principal_label="root-ish",+ )+ await _seed_usage(db, days_ago=1)+ await bind_tenant(db, TENANT)+ await run_review(db, tenant_id=TENANT, provider="aws")+ await db.commit()++ assert await list_findings(db, tenant_id=TENANT) == []+++async def test_another_tenants_findings_are_invisible_and_unacknowledgeable(db) -> None:+ await _seed_admin(db, OTHER_TENANT)+ await bind_tenant(db, OTHER_TENANT)+ await run_review(db, tenant_id=OTHER_TENANT, provider="aws")+ await db.commit()++ theirs = await list_findings(db, tenant_id=OTHER_TENANT)+ assert theirs, "the other tenant's review produced nothing, so this proves nothing"+ assert await list_findings(db, tenant_id=TENANT) == []++ # A 404 rather than a 403: the route must not be an existence oracle+ # over another tenant's finding ids.+ await bind_tenant(db, TENANT)+ assert await acknowledge(db, tenant_id=TENANT, finding_id=uuid.UUID(theirs[0]["id"]), user_id=USER) is None+++async def test_acknowledgement_records_the_person_and_hides_the_row(db) -> None:+ await _seed_admin(db)+ await bind_tenant(db, TENANT)+ await run_review(db, tenant_id=TENANT, provider="aws")+ await db.commit()+ open_rows = await list_findings(db, tenant_id=TENANT)+ assert open_rows++ await bind_tenant(db, TENANT)++ updated = await acknowledge(+ db,+ tenant_id=TENANT,+ finding_id=uuid.UUID(open_rows[0]["id"]),+ user_id=USER,+ note="accepted: break-glass account, reviewed quarterly",+ )+ await db.commit()+ assert updated is not None+ assert updated["acknowledged_by"] == str(USER)+ assert "break-glass" in updated["acknowledged_note"]++ remaining = await list_findings(db, tenant_id=TENANT)+ assert len(remaining) == len(open_rows) - 1+ # Hidden, not deleted: "who decided this was fine" is the question an+ # auditor asks, and a deleted row cannot answer it.+ assert len(await list_findings(db, tenant_id=TENANT, include_acknowledged=True)) == len(open_rows)+++async def test_an_acknowledgement_survives_the_next_run(db) -> None:+ """Otherwise a nightly review un-acknowledges everything every night."""+ await _seed_admin(db)+ await bind_tenant(db, TENANT)+ await run_review(db, tenant_id=TENANT, provider="aws")+ await db.commit()+ first = await list_findings(db, tenant_id=TENANT)+ await bind_tenant(db, TENANT)+ await acknowledge(db, tenant_id=TENANT, finding_id=uuid.UUID(first[0]["id"]), user_id=USER)+ await db.commit()++ await bind_tenant(db, TENANT)++ await run_review(db, tenant_id=TENANT, provider="aws")+ await db.commit()++ still_open = {f["kind"] for f in await list_findings(db, tenant_id=TENANT)}+ assert first[0]["kind"] not in still_open+++async def test_summary_distinguishes_never_run_from_reviewed(db) -> None:+ never = await summary(db, tenant_id=TENANT)+ assert never["state"] == "never_run"+ assert "not a clean bill of health" in never["note"]++ await _seed_admin(db)+ await bind_tenant(db, TENANT)+ await run_review(db, tenant_id=TENANT, provider="aws")+ await db.commit()++ after = await summary(db, tenant_id=TENANT)+ assert after["state"] == "reviewed"+ assert after["providers"][0]["provider"] == "aws"+ assert after["providers"][0]["reviewed_principals"] == 1diff --git a/services/api/tests/test_access_review_reproduction.py b/services/api/tests/test_access_review_reproduction.pynew file mode 100644index 000000000..331fafe60--- /dev/null+++ b/services/api/tests/test_access_review_reproduction.py@@ -0,0 +1,112 @@+"""Depth 2 Phase 1.2 reproduction: nothing compares granted with used.++1.1 made "what can this identity do" answerable and recorded, per principal,+which granted actions were exercised in a trailing window. Neither is a+review. A reviewer's questions are comparative — who holds admin they never+use, which human has a long-lived cloud key, which outside collaborator has+admin on a repository, which admin has been dormant — and no module in the+tree asks any of them.++Fails on the pre-change tree for that reason. Kept afterwards as the guard,+including the two tests that pin the honesty properties, because the failure+mode here is not an absent finding but a confident wrong one: a review built+on unmeasured usage recommends revoking every permission in the estate.+"""++from __future__ import annotations++import pytest+++def test_an_access_review_module_exists() -> None:+ from app.services import access_review++ assert hasattr(access_review, "review_principal")+ assert hasattr(access_review, "FINDING_KINDS")+++def test_every_finding_kind_the_plan_names_is_implemented() -> None:+ """The four the plan lists, plus the posture findings the same scan surfaces.++ Named individually rather than counted: a count passes when one kind is+ renamed and another silently dropped.+ """+ from app.services.access_review import FINDING_KINDS++ required = {+ "human_with_long_lived_key",+ "unused_write_access",+ "outside_collaborator_admin",+ "dormant_admin",+ "fork_reachable_workflow_secret",+ "org_member_without_mfa",+ }+ assert required <= set(FINDING_KINDS), f"missing: {sorted(required - set(FINDING_KINDS))}"+++def test_unmeasured_usage_produces_no_usage_finding() -> None:+ """The property the whole item turns on.++ An install with no event lake cannot know which permissions were+ exercised. A review that reads that as "used nothing" recommends+ revoking the estate, and it recommends it with exactly the confidence of+ a correct finding.+ """+ from app.services.access_review import review_principal+ from app.services.identity_privilege import IdentityPrivilege, PrivilegeTags+ from app.services.identity_privilege_store import UsageOverlay++ privilege = IdentityPrivilege(+ tier="admin",+ tags=PrivilegeTags(can_escalate=True),+ provider="aws",+ principal_id="arn:aws:iam::1:user/alice",+ evidence={"can_escalate": ["iam:PutUserPolicy"]},+ action_count=40,+ )+ result = review_principal(privilege=privilege, usage=UsageOverlay(measured=False, used={}))++ kinds = {f.kind for f in result.findings}+ assert "unused_write_access" not in kinds+ assert "dormant_admin" not in kinds+ assert result.usage_measured is False+ # And it must say so, rather than returning a short list that reads as+ # a clean bill of health.+ assert result.unmeasured_reason+++def test_a_review_with_no_inventory_is_not_a_clean_bill() -> None:+ """Zero findings over zero principals is "nothing reviewed"."""+ from app.services.access_review import summarise++ summary = summarise([])+ assert summary["reviewed_principals"] == 0+ assert summary["state"] == "nothing_reviewed"+ assert summary["findings"] == 0+++def test_best_fit_role_is_only_offered_when_usage_was_measured() -> None:+ """A right-sizing recommendation on unmeasured usage is a guess."""+ from app.services.access_review import best_fit_role+ from app.services.identity_privilege_store import UsageOverlay++ unmeasured = best_fit_role(+ provider="aws",+ granted={"s3:GetObject", "s3:PutObject", "iam:PutUserPolicy"},+ usage=UsageOverlay(measured=False, used={}),+ )+ assert unmeasured is None++[email protected]("kind", ["human_with_long_lived_key", "outside_collaborator_admin"])+def test_every_finding_names_the_object_to_change(kind: str) -> None:+ """Read-only means the finding has to be actionable by a human.++ "Nothing may change vendor state" is only workable if the finding says+ precisely which object a person should go and change.+ """+ from app.services.access_review import FINDING_KINDS++ spec = FINDING_KINDS[kind]+ assert spec.object_hint, f"{kind} does not name the object to change"+ assert spec.recommendationdiff --git a/services/api/tests/test_alert_queue.py b/services/api/tests/test_alert_queue.pyindex 56d579596..66ed1bf5b 100644--- a/services/api/tests/test_alert_queue.py+++ b/services/api/tests/test_alert_queue.py@@ -715,7 +715,14 @@ class TestBuildQueue:# (critical base 900) and the basis says exactly that.assert item.priority_score == 900assert item.priority_basis["base"] == 900- assert item.priority_basis["rationale"] == []+ # Since 1.1 the basis is not empty even with no context: a 1.0 row+ # records that the actor's privilege at event time could not be+ # resolved. The row has to be here rather than omitted, because an+ # analyst reading "why is this ranked where it is" needs to be told+ # the platform could not answer, not shown a list with the question+ # silently missing. The score is unmoved, which is the other half.+ assert [r["factor"] for r in item.priority_basis["rationale"]] == [1.0]+ assert "unknown" in item.priority_basis["rationale"][0]["reason"]# Age is positive (first_seen was 10 minutes ago).assert item.age_seconds >= 0# Asset chip pulls host first.diff --git a/services/api/tests/test_alert_queue_priority_live.py b/services/api/tests/test_alert_queue_priority_live.pyindex 1e46118d2..e96da4668 100644--- a/services/api/tests/test_alert_queue_priority_live.py+++ b/services/api/tests/test_alert_queue_priority_live.py@@ -164,7 +164,12 @@ async def test_another_tenants_inventory_is_invisible(db) -> None:resp = await build_queue(db, tenant_id=TENANT, user_id=USER, owner="unassigned")item = next(i for i in resp.items if i.id == kiosk_alert.id)- assert item.priority_basis["rationale"] == [], "no same-tenant asset matched; severity alone"+ # Since 1.1 the basis carries a 1.0 row recording that the actor's+ # privilege at event time could not be resolved. The assertion is on+ # the *asset* half: no same-tenant asset matched, so no asset factor+ # appears and the score is untouched.+ assert [r["factor"] for r in item.priority_basis["rationale"]] == [1.0], "no same-tenant asset matched; severity alone"+ assert not any("asset" in r["reason"] for r in item.priority_basis["rationale"])@pytest.mark.asyncio@@ -177,4 +182,8 @@ async def test_no_inventory_scores_severity_alone(db) -> None:item = next(i for i in resp.items if i.id == alert.id)assert item.priority_score == 600 # high base, no multipliersassert item.priority_basis["base"] == 600- assert item.priority_basis["rationale"] == []+ # One 1.0 row, which moves nothing: with no posture collected the+ # actor's privilege is unknown, and the basis says so rather than+ # omitting the question. The score is still severity alone.+ assert [r["factor"] for r in item.priority_basis["rationale"]] == [1.0]+ assert "unknown" in item.priority_basis["rationale"][0]["reason"]diff --git a/services/api/tests/test_change_timeline_reproduction.py b/services/api/tests/test_change_timeline_reproduction.pynew file mode 100644index 000000000..34323e004--- /dev/null+++ b/services/api/tests/test_change_timeline_reproduction.py@@ -0,0 +1,77 @@+"""Depth 2 Phase 1.5 reproduction: nothing answers "what changed before this".++The question an analyst asks first about an identity alert is what changed+in the days before it: a role granted, a policy edited, an OAuth app+consented to, an MFA factor replaced, a share opened, a key created. Both+halves of the answer are already in the platform — versioned posture+snapshots since migration 103, and the audit events the lake holds — and+nothing joins them.++Fails on the pre-change tree for that reason, and keeps two honesty+properties afterwards: a window with no recorded change must be+distinguishable from a window nobody could read, and every change returned+must carry the source it came from so the ledger citation can be checked.+"""++from __future__ import annotations+++def test_a_change_timeline_module_exists() -> None:+ from app.services import change_timeline++ assert hasattr(change_timeline, "changes_before")+ assert hasattr(change_timeline, "CHANGE_KINDS")+++def test_every_change_kind_the_plan_names_is_covered() -> None:+ from app.services.change_timeline import CHANGE_KINDS++ required = {+ "role_or_group_grant",+ "policy_edit",+ "oauth_consent",+ "mfa_factor_change",+ "sharing_change",+ "key_creation",+ }+ assert required <= set(CHANGE_KINDS), f"missing: {sorted(required - set(CHANGE_KINDS))}"+++def test_the_typed_pivot_is_registered() -> None:+ """A tool nobody is offered is not a tool."""+ from app.services import investigation_tools++ assert "changes_before_alert" in investigation_tools.TOOLS+++def test_no_changes_is_distinguishable_from_could_not_read() -> None:+ """The two readings call for opposite conclusions.++ "Nothing changed in the window" is evidence. "We could not read the+ window" is a gap. A result shape that renders both as an empty list+ turns the second into the first, and an investigation closes on it.+ """+ from app.services.change_timeline import ChangeTimeline++ quiet = ChangeTimeline(changes=[], sources_read=("posture", "audit"))+ blind = ChangeTimeline(changes=[], sources_read=())++ assert quiet.measured is True+ assert blind.measured is False+ assert blind.as_dict()["reason"]+++def test_every_change_names_its_source() -> None:+ """A citation whose target nobody can find is not a citation."""+ from app.services.change_timeline import IdentityChange++ change = IdentityChange(+ kind="policy_edit",+ at="2026-10-01T00:00:00+00:00",+ subject="arn:aws:iam::1:user/alice",+ summary="policy p1 edited",+ source="posture_snapshot",+ source_ref="snapshot:abc123",+ )+ assert change.source and change.source_ref+ assert change.as_dict()["source_ref"] == "snapshot:abc123"diff --git a/services/api/tests/test_identity_privilege_live.py b/services/api/tests/test_identity_privilege_live.pynew file mode 100644index 000000000..620f3e8d0--- /dev/null+++ b/services/api/tests/test_identity_privilege_live.py@@ -0,0 +1,394 @@+"""Privilege at event time, proven against real Postgres.++Depth plan wave 2, Phase 1.1. Live because every property under test is a+schema fact a mock would invent:++* the ``valid_from <= t < valid_until`` interval select, which is the whole+ reason the posture table is versioned and the one thing a stand-in store+ always gets right by construction;+* the partial unique index that refuses two live rows per principal;+* the ``principal_id OR principal_label`` join, because an alert carries+ whichever handle the connector emitted and a join on the wrong one matches+ nothing while looking correct;+* the tenant predicate on both queries;+* ``boolean NULL`` round-tripping as ``None`` rather than ``False``, which is+ the difference between "no snapshot could answer" and "the snapshot says+ no".++Each test has a negative control that reverts the behaviour under test and+asserts it fails again, so none of them can pass for a reason other than the+one in its name.++Skips without a database rather than passing without one; under+``MSSP_ISOLATION_REQUIRED`` it fails instead of skipping, because a live+suite that silently skips is decoration.+"""++from __future__ import annotations++import os+import uuid+from datetime import UTC, datetime, timedelta++import pytest+import pytest_asyncio+from app.services.alert_priority import IdentityContext, score_alert_priority+from app.services.identity_privilege import IdentityPrivilege, PrivilegeTags+from app.services.identity_privilege_store import (+ live_facts_for_provider,+ privilege_at,+ record_usage,+ save_privilege,+ usage_for_principal,+)+from live_rls import bind_tenant+from sqlalchemy import text+from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine++DSN = os.environ.get("DATABASE_URL", "")+REQUIRED = os.environ.get("MSSP_ISOLATION_REQUIRED", "").strip() not in ("", "0", "false")++TENANT = uuid.UUID("0e710000-0000-0000-0000-0000000000a1")+OTHER_TENANT = uuid.UUID("0e710000-0000-0000-0000-0000000000a2")++T0 = datetime(2026, 10, 1, 12, 0, 0, tzinfo=UTC)+T1 = T0 + timedelta(days=5)+T2 = T1 + timedelta(days=5)+++@pytest_asyncio.fixture+async def db():+ if "postgres" not in DSN and not REQUIRED:+ pytest.skip("needs a live Postgres with the migration chain applied (integration.yml)")+ engine = create_async_engine(DSN)+ try:+ async with engine.connect() as probe:+ await probe.execute(text("SELECT 1"))+ except Exception as exc:+ await engine.dispose()+ if REQUIRED:+ pytest.fail(f"MSSP_ISOLATION_REQUIRED is set and no database answered at DATABASE_URL: {type(exc).__name__}: {exc}")+ pytest.skip(f"no database at DATABASE_URL ({type(exc).__name__}) — runs in integration.yml")++ factory = async_sessionmaker(engine, expire_on_commit=False)+ async with factory() as session:+ for tid, slug in ((TENANT, "priv-live-a"), (OTHER_TENANT, "priv-live-b")):+ await session.execute(+ text("INSERT INTO tenants (id, name, slug) VALUES (:id, :name, :slug) ON CONFLICT (id) DO NOTHING"),+ {"id": tid, "name": f"Privilege live {slug}", "slug": slug},+ )+ await session.commit()+ yield session+ await session.rollback()+ for table in ("identity_permission_usage", "identity_privilege_facts"):+ await session.execute(+ text(f"DELETE FROM {table} WHERE tenant_id IN (:a, :b)"), # noqa: S608 — fixed table names+ {"a": TENANT, "b": OTHER_TENANT},+ )+ await session.commit()+ await engine.dispose()+++#: Read times are nudged a minute into the future rather than using the+#: host's ``now()``.+#:+#: ``valid_from`` defaults to Postgres' ``now()``, and on this machine the+#: database container's clock ran ~17 ms *ahead* of the host's, so a row+#: written and then read at the host's ``now()`` landed before its own+#: ``valid_from`` and the point-in-time select correctly returned nothing.+#: That is the store behaving exactly as specified -- an event predating a+#: collection has no posture -- and a test that trips over it is measuring+#: clock skew, not the product. Widening ``valid_from <= :at`` to paper over+#: it would silently answer historical questions with newer snapshots, which+#: is the whole defect migration 103 is versioned to prevent.+#:+#: A minute is far inside the real gap between a collection and an event+#: (minutes to days), so nothing about the property under test is softened.+_SKEW = timedelta(minutes=1)+++def _read_time() -> datetime:+ return datetime.now(tz=UTC) + _SKEW+++def _privilege(tier: str, principal: str = "arn:aws:iam::1:user/alice", **tags) -> IdentityPrivilege:+ return IdentityPrivilege(+ tier=tier, # type: ignore[arg-type]+ tags=PrivilegeTags(**tags),+ provider="aws",+ principal_id=principal,+ evidence={"tier": ["iam:PutUserPolicy"]} if tier in {"admin", "domain_admin"} else {},+ action_count=12,+ resource_count=3,+ )+++async def test_point_in_time_read_returns_the_tier_that_was_true_then(db) -> None:+ """An alert from before a demotion must read the pre-demotion tier.++ This is the defect the versioned table exists to prevent, and it is+ invisible on inspection: a live-row read returns a number that looks+ exactly as plausible as the right one.+ """+ await bind_tenant(db, TENANT)+ await save_privilege(db, tenant_id=TENANT, privilege=_privilege("domain_admin"), principal_label="alice")+ await db.commit()+ # Supersede: the same principal is demoted five days later.+ await bind_tenant(db, TENANT)+ await db.execute(+ text("UPDATE identity_privilege_facts SET valid_from = :t0 WHERE tenant_id = :t AND valid_until IS NULL"),+ {"t0": T0, "t": TENANT},+ )+ await db.commit()+ await bind_tenant(db, TENANT)+ await save_privilege(db, tenant_id=TENANT, privilege=_privilege("standard"), principal_label="alice")+ await db.commit()++ before = await privilege_at(db, tenant_id=TENANT, principal_id="alice", at=T0 + timedelta(hours=1))+ after = await privilege_at(db, tenant_id=TENANT, principal_id="alice", at=_read_time())++ assert before is not None and before.privilege.tier == "domain_admin"+ assert after is not None and after.privilege.tier == "standard"++ # Negative control: the live-row read, which is what this replaced.+ # It answers the *current* tier for the historical question, and the+ # answer is wrong in the direction nobody checks -- it under-states.+ live_row = (+ await db.execute(+ text("SELECT tier FROM identity_privilege_facts WHERE tenant_id = :t AND valid_until IS NULL LIMIT 1"),+ {"t": TENANT},+ )+ ).scalar_one()+ assert live_row == "standard"+ assert live_row != before.privilege.tier+++async def test_an_event_before_the_first_collection_has_no_fact(db) -> None:+ """No covering version must return None, never the nearest one."""+ await bind_tenant(db, TENANT)+ await save_privilege(db, tenant_id=TENANT, privilege=_privilege("admin"), principal_label="alice")+ await db.commit()+ await bind_tenant(db, TENANT)+ await db.execute(+ text("UPDATE identity_privilege_facts SET valid_from = :t WHERE tenant_id = :tid"),+ {"t": T1, "tid": TENANT},+ )+ await db.commit()++ assert await privilege_at(db, tenant_id=TENANT, principal_id="alice", at=T0) is None+ assert await privilege_at(db, tenant_id=TENANT, principal_id="alice", at=T2) is not None+++async def test_tags_round_trip_as_tri_state(db) -> None:+ """NULL must come back as None. False and None are different facts."""+ await bind_tenant(db, TENANT)+ await save_privilege(+ db,+ tenant_id=TENANT,+ privilege=_privilege("admin", can_escalate=True, data_access=False, third_party=None, cross_account=True),+ principal_label="alice",+ )+ await db.commit()++ fact = await privilege_at(db, tenant_id=TENANT, principal_id="alice", at=_read_time())+ assert fact is not None+ assert fact.privilege.tags.can_escalate is True+ assert fact.privilege.tags.data_access is False+ assert fact.privilege.tags.third_party is None+ assert fact.privilege.tags.cross_account is True+ # The one that matters: a reader that coerces would see False here.+ assert fact.privilege.tags.third_party is not False+++async def test_label_and_id_both_resolve(db) -> None:+ """An alert carries whichever handle its connector emitted."""+ await bind_tenant(db, TENANT)+ await save_privilege(db, tenant_id=TENANT, privilege=_privilege("elevated"), principal_label="[email protected]")+ await db.commit()+ now = _read_time()++ by_arn = await privilege_at(db, tenant_id=TENANT, principal_id="arn:aws:iam::1:user/alice", at=now)+ by_label = await privilege_at(db, tenant_id=TENANT, principal_id="[email protected]", at=now)++ assert by_arn is not None and by_label is not None+ assert by_arn.privilege.tier == by_label.privilege.tier == "elevated"+++async def test_another_tenants_facts_are_invisible(db) -> None:+ await bind_tenant(db, OTHER_TENANT)+ await save_privilege(db, tenant_id=OTHER_TENANT, privilege=_privilege("domain_admin"), principal_label="alice")+ await db.commit()++ assert await privilege_at(db, tenant_id=TENANT, principal_id="alice", at=_read_time()) is None+ assert await live_facts_for_provider(db, tenant_id=TENANT, provider="aws") == []+ assert len(await live_facts_for_provider(db, tenant_id=OTHER_TENANT, provider="aws")) == 1+++async def test_an_unchanged_rederivation_does_not_open_a_version(db) -> None:+ """Otherwise "when did this privilege last change" answers "hourly"."""+ await bind_tenant(db, TENANT)+ await save_privilege(db, tenant_id=TENANT, privilege=_privilege("admin"), principal_label="alice")+ await db.commit()+ await bind_tenant(db, TENANT)+ await save_privilege(db, tenant_id=TENANT, privilege=_privilege("admin"), principal_label="alice")+ await db.commit()++ count = (+ await db.execute(+ text("SELECT count(*) FROM identity_privilege_facts WHERE tenant_id = :t"),+ {"t": TENANT},+ )+ ).scalar_one()+ assert count == 1++ # And a real change does open one, which is the control for the above:+ # a store that never wrote a second row would also pass the assertion.+ await bind_tenant(db, TENANT)+ await save_privilege(db, tenant_id=TENANT, privilege=_privilege("standard"), principal_label="alice")+ await db.commit()+ count_after = (+ await db.execute(+ text("SELECT count(*) FROM identity_privilege_facts WHERE tenant_id = :t"),+ {"t": TENANT},+ )+ ).scalar_one()+ assert count_after == 2+++async def test_two_live_rows_are_refused_by_the_database(db) -> None:+ """The invariant is enforced in Postgres, not by every writer remembering."""+ await bind_tenant(db, TENANT)+ await save_privilege(db, tenant_id=TENANT, privilege=_privilege("admin"), principal_label="alice")+ await db.commit()++ await bind_tenant(db, TENANT)+ with pytest.raises(Exception) as excinfo:+ await db.execute(+ text(+ """+ INSERT INTO identity_privilege_facts (tenant_id, provider, principal_id, tier)+ VALUES (:t, 'aws', 'arn:aws:iam::1:user/alice', 'standard')+ """+ ),+ {"t": TENANT},+ )+ await db.commit()+ assert "identity_privilege_facts_live_idx" in str(excinfo.value)+ await db.rollback()+++async def test_usage_distinguishes_unmeasured_from_unused(db) -> None:+ """The distinction a right-sizing recommendation turns on.++ With no rows the overlay reports ``measured=False`` and ``unused()``+ returns ``None``. A store that returned an empty set instead would have+ every finding recommend revoking every permission in the estate.+ """+ empty = await usage_for_principal(db, tenant_id=TENANT, provider="aws", principal_id="u1")+ assert empty.measured is False+ assert empty.unused({"s3:GetObject", "s3:PutObject"}) is None++ now = datetime.now(tz=UTC)+ await bind_tenant(db, TENANT)+ await record_usage(+ db,+ tenant_id=TENANT,+ provider="aws",+ principal_id="u1",+ observations={"s3:GetObject": (42, now - timedelta(days=3), now)},+ window_start=now - timedelta(days=30),+ window_end=now,+ )+ await db.commit()++ measured = await usage_for_principal(db, tenant_id=TENANT, provider="aws", principal_id="u1")+ assert measured.measured is True+ assert measured.unused({"s3:GetObject", "s3:PutObject"}) == {"s3:PutObject"}+++async def test_usage_upsert_keeps_the_widest_observed_span(db) -> None:+ """A later window must not shorten what was already observed."""+ now = datetime.now(tz=UTC)+ first, last = now - timedelta(days=20), now - timedelta(days=10)+ await bind_tenant(db, TENANT)+ await record_usage(+ db,+ tenant_id=TENANT,+ provider="aws",+ principal_id="u1",+ observations={"s3:GetObject": (5, first, last)},+ window_start=now - timedelta(days=30),+ window_end=now,+ )+ await bind_tenant(db, TENANT)+ await record_usage(+ db,+ tenant_id=TENANT,+ provider="aws",+ principal_id="u1",+ observations={"s3:GetObject": (7, now - timedelta(days=2), now)},+ window_start=now - timedelta(days=7),+ window_end=now,+ )+ await db.commit()++ row = (+ await db.execute(+ text(+ "SELECT first_used_at, last_used_at, use_count FROM identity_permission_usage WHERE tenant_id = :t AND principal_id = 'u1'"+ ),+ {"t": TENANT},+ )+ ).one()+ assert row.first_used_at == first+ assert row.last_used_at > last+ assert row.use_count == 7+++async def test_the_tier_read_from_the_store_actually_moves_the_queue_score(db) -> None:+ """End of the chain: a stored fact changes the number the queue sorts by.++ Without this the other tests prove a well-built table nobody consumes,+ which is the exact shape this phase exists to close.+ """+ await bind_tenant(db, TENANT)+ await save_privilege(+ db,+ tenant_id=TENANT,+ privilege=_privilege("domain_admin", can_escalate=True, cross_account=True),+ principal_label="alice",+ )+ await db.commit()++ fact = await privilege_at(db, tenant_id=TENANT, principal_id="alice", at=_read_time())+ assert fact is not None++ context = IdentityContext(+ principal=fact.privilege.principal_id,+ privilege_tier=fact.privilege.tier,+ can_escalate=fact.privilege.tags.can_escalate,+ data_access=fact.privilege.tags.data_access,+ third_party=fact.privilege.tags.third_party,+ cross_account=fact.privilege.tags.cross_account,+ )+ with_context = score_alert_priority(severity="medium", identity=context)+ unknown = score_alert_priority(severity="medium", identity=IdentityContext())++ assert with_context.score > unknown.score+ reasons = " ".join(r["reason"] for r in with_context.rationale)+ assert "domain_admin" in reasons+ assert "can_escalate" in reasons and "cross_account" in reasons++ # Negative control: drop the tags and the tier from the context -- the+ # exact revert -- and the score collapses back to the unknown baseline.+ assert score_alert_priority(severity="medium", identity=IdentityContext()).score == unknown.scorediff --git a/services/api/tests/test_identity_privilege_reproduction.py b/services/api/tests/test_identity_privilege_reproduction.pynew file mode 100644index 000000000..a48f5e14e--- /dev/null+++ b/services/api/tests/test_identity_privilege_reproduction.py@@ -0,0 +1,179 @@+"""Depth 2 Phase 1.1 reproduction: the posture exists and nothing consumes it.++Depth 2.5 built versioned posture snapshots and six resolvers that answer+"what can this principal do" from one. Depth 2.6 — carried into wave 2 as+1.1 — is the half that was never built: nothing asks the resolver on an+analyst's behalf, no alert carries a privilege tier, and the AWS resolver's+own docstring lists four IAM semantics it does not model.++Each test below fails on the pre-change tree for one stated reason. They are+kept after the fix as the guard, because every one of them is the "built but+unwired" shape this repository keeps rediscovering: a resolver with no+caller is indistinguishable from a resolver that does not exist, and a+passing unit test on the resolver cannot tell the difference.+"""++from __future__ import annotations++from typing import Any++import pytest+from app.services.effective_permissions.aws import AwsIamResolver+++def _snapshot(statement: dict[str, Any], *, principal_extra: dict[str, Any] | None = None) -> dict[str, Any]:+ """One principal, one bucket, one identity policy carrying ``statement``."""+ principal: dict[str, Any] = {+ "id": "u1",+ "arn": "arn:aws:iam::123456789012:user/alice",+ "type": "user",+ "attached_policies": ["p1"],+ }+ principal.update(principal_extra or {})+ return {+ "account_id": "123456789012",+ "principals": [principal],+ "policies": [{"id": "p1", "name": "p1", "kind": "identity", "document": {"Statement": [statement]}}],+ "resources": [+ {+ "id": "r1",+ "arn": "arn:aws:s3:::data",+ "kind": "s3:bucket",+ "service_actions": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],+ }+ ],+ }+++def _actions(snapshot: dict[str, Any]) -> set[str]:+ result = AwsIamResolver().resolve("u1", snapshot=snapshot)+ return {action for decision in result.decisions for action in decision.actions}+++class TestAwsSemanticsTheResolverDoesNotModel:+ """Each fixture changes the answer, which is the point of modelling it.++ A resolver that ignores ``NotAction`` does not return a slightly vaguer+ answer — it returns the opposite one, because ``NotAction`` inverts the+ set the statement applies to. Silently broader is how a containment+ decision reads "this identity cannot touch the data" when it can.+ """++ def test_not_action_inverts_the_action_set(self) -> None:+ """``Allow NotAction s3:DeleteObject`` grants Get and Put, not Delete."""+ actions = _actions(_snapshot({"Effect": "Allow", "NotAction": "s3:DeleteObject", "Resource": "*"}))+ assert actions == {"s3:GetObject", "s3:PutObject"}++ def test_not_action_on_a_deny_denies_everything_else(self) -> None:+ """The deny arm matters more: it is what an SCP uses to pin a region."""+ snapshot = _snapshot({"Effect": "Allow", "Action": "s3:*", "Resource": "*"})+ snapshot["policies"].append(+ {+ "id": "p2",+ "name": "p2",+ "kind": "identity",+ "document": {"Statement": [{"Effect": "Deny", "NotAction": "s3:GetObject", "Resource": "*"}]},+ }+ )+ snapshot["principals"][0]["attached_policies"].append("p2")+ assert _actions(snapshot) == {"s3:GetObject"}++ def test_not_resource_excludes_only_the_named_resource(self) -> None:+ """``NotResource`` on the one bucket in the snapshot matches nothing."""+ actions = _actions(_snapshot({"Effect": "Allow", "Action": "s3:*", "NotResource": "arn:aws:s3:::data"}))+ assert actions == set()++ @pytest.mark.parametrize(+ ("condition", "context", "expected"),+ [+ ({"Bool": {"aws:SecureTransport": "true"}}, {"aws:SecureTransport": "true"}, True),+ ({"Bool": {"aws:SecureTransport": "true"}}, {"aws:SecureTransport": "false"}, False),+ ({"StringNotEquals": {"aws:PrincipalTag/team": "ops"}}, {"aws:PrincipalTag/team": "dev"}, True),+ ({"StringNotEquals": {"aws:PrincipalTag/team": "ops"}}, {"aws:PrincipalTag/team": "ops"}, False),+ ({"NumericLessThan": {"s3:max-keys": "10"}}, {"s3:max-keys": "5"}, True),+ ({"NumericLessThan": {"s3:max-keys": "10"}}, {"s3:max-keys": "50"}, False),+ ({"IpAddress": {"aws:SourceIp": "203.0.113.0/24"}}, {"aws:SourceIp": "203.0.113.7"}, True),+ ({"IpAddress": {"aws:SourceIp": "203.0.113.0/24"}}, {"aws:SourceIp": "198.51.100.7"}, False),+ ({"ArnLike": {"aws:PrincipalArn": "arn:aws:iam::*:user/*"}}, {"aws:PrincipalArn": "arn:aws:iam::1:user/a"}, True),+ ({"DateGreaterThan": {"aws:CurrentTime": "2020-01-01T00:00:00Z"}}, {"aws:CurrentTime": "2026-01-01T00:00:00Z"}, True),+ ({"DateGreaterThan": {"aws:CurrentTime": "2030-01-01T00:00:00Z"}}, {"aws:CurrentTime": "2026-01-01T00:00:00Z"}, False),+ ],+ )+ def test_condition_operators_beyond_string_equals(self, condition: dict[str, Any], context: dict[str, Any], expected: bool) -> None:+ """Today every operator outside StringEquals/StringLike denies outright.++ That is the conservative direction, which sounds safe and is not: it+ makes ``Bool aws:SecureTransport true`` — on a policy the request+ satisfies — read as "this identity cannot read the bucket", so an+ access review under-reports real access.+ """+ snapshot = _snapshot({"Effect": "Allow", "Action": "s3:GetObject", "Resource": "*", "Condition": condition})+ snapshot["resources"][0]["context"] = context+ assert ("s3:GetObject" in _actions(snapshot)) is expected++ def test_session_policy_caps_an_assumed_role(self) -> None:+ """A session policy is the cap that an assumed-role chain actually runs under.++ ``sts:AssumeRole`` with a session policy is how every federated and+ cross-account hop narrows itself, so ignoring it over-reports the+ role's reach on exactly the principals an investigation cares about.+ """+ snapshot = _snapshot({"Effect": "Allow", "Action": "s3:*", "Resource": "*"})+ snapshot["principals"][0]["session_policies"] = ["sp1"]+ snapshot["policies"].append(+ {+ "id": "sp1",+ "name": "sp1",+ "kind": "session",+ "document": {"Statement": [{"Effect": "Allow", "Action": "s3:GetObject", "Resource": "*"}]},+ }+ )+ assert _actions(snapshot) == {"s3:GetObject"}+++class TestNothingConsumesThePosture:+ """The wiring half of 1.1 — a tool nobody is offered is not a tool."""++ def test_investigate_query_exposes_the_typed_pivot(self) -> None:+ """``POST /investigate/query`` picks from ``investigation_tools.TOOLS``."""+ from app.services import investigation_tools++ assert "identity_effective_permissions" in investigation_tools.TOOLS++ def test_privilege_facts_are_derived_from_a_point_in_time_snapshot(self) -> None:+ """Fusion needs a tier for the moment the identity acted, not now."""+ from app.services.identity_privilege import derive_privilege++ result = AwsIamResolver().resolve(+ "u1",+ snapshot=_snapshot({"Effect": "Allow", "Action": "iam:AttachUserPolicy", "Resource": "*"}),+ )+ facts = derive_privilege(result)+ assert facts.tier in {"standard", "elevated", "admin", "domain_admin"}+ assert facts.tags.can_escalate is True++ def test_absent_posture_is_unknown_and_not_a_default_tier(self) -> None:+ """ "No snapshot covers this time" must never render as ``standard``.++ A default tier is the confident-zero defect: a reader cannot tell a+ measured standard account from one the platform knows nothing about,+ and the second is the one worth escalating.+ """+ from app.services.identity_privilege import UNKNOWN_PRIVILEGE++ assert UNKNOWN_PRIVILEGE.tier == "unknown"+ assert UNKNOWN_PRIVILEGE.tags.can_escalate is None+++class TestBlastRadiusUsesPermissionsNotHops:+ def test_a_permission_scoped_blast_radius_exists(self) -> None:+ """N-hop reachability over any relationship answers a different question.++ "Everything within three hops" includes the alert that mentioned the+ host, the IOC seen on it and the technique tagged on that: a list an+ analyst cannot act on. What the identity's effective permissions+ reach is the containment question.+ """+ from app.services import graph_service++ assert hasattr(graph_service, "get_permission_blast_radius")diff --git a/services/api/tests/test_identity_resolution_live.py b/services/api/tests/test_identity_resolution_live.pynew file mode 100644index 000000000..83a50973e--- /dev/null+++ b/services/api/tests/test_identity_resolution_live.py@@ -0,0 +1,233 @@+"""Identity resolution against real Postgres (depth plan wave 2, Phase 3.1).++Live because every property is a schema or SQL fact a stand-in invents: the+ordered-pair CHECK that stores a link once however it was discovered, the+unique index that makes a re-resolution an update, the merge-strength filter+inside the cluster walk, and the tenant predicate on all of it.++The cluster walk is the test that matters most. A weak link must not be+traversable, and the way that breaks is a chain: three weak hops joining two+identities no single rule would join. That cannot be checked by reading the+code, because the filter is in the SQL.++Each test carries a negative control.+"""++from __future__ import annotations++import os+import uuid++import pytest+import pytest_asyncio+from app.services.identity_resolution import IdentityLink, ResolutionResult, resolve_links+from app.services.identity_resolution_store import cluster_for, decide_link, links_for, save_links+from live_rls import bind_tenant+from sqlalchemy import text+from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine++DSN = os.environ.get("DATABASE_URL", "")+REQUIRED = os.environ.get("MSSP_ISOLATION_REQUIRED", "").strip() not in ("", "0", "false")++TENANT = uuid.UUID("0e730000-0000-0000-0000-0000000000c1")+OTHER_TENANT = uuid.UUID("0e730000-0000-0000-0000-0000000000c2")+USER = uuid.UUID("0e730000-0000-0000-0000-0000000000c3")+++@pytest_asyncio.fixture+async def db():+ if "postgres" not in DSN and not REQUIRED:+ pytest.skip("needs a live Postgres with the migration chain applied (integration.yml)")+ engine = create_async_engine(DSN)+ try:+ async with engine.connect() as probe:+ await probe.execute(text("SELECT 1"))+ except Exception as exc:+ await engine.dispose()+ if REQUIRED:+ pytest.fail(f"MSSP_ISOLATION_REQUIRED is set and no database answered at DATABASE_URL: {type(exc).__name__}: {exc}")+ pytest.skip(f"no database at DATABASE_URL ({type(exc).__name__}) — runs in integration.yml")++ factory = async_sessionmaker(engine, expire_on_commit=False)+ async with factory() as session:+ for tid, slug in ((TENANT, "idres-live-a"), (OTHER_TENANT, "idres-live-b")):+ await session.execute(+ text("INSERT INTO tenants (id, name, slug) VALUES (:id, :name, :slug) ON CONFLICT (id) DO NOTHING"),+ {"id": tid, "name": f"Identity resolution {slug}", "slug": slug},+ )+ await session.commit()+ yield session+ await session.rollback()+ await session.execute(+ text("DELETE FROM identity_links WHERE tenant_id IN (:a, :b)"),+ {"a": TENANT, "b": OTHER_TENANT},+ )+ await session.commit()+ await engine.dispose()+++def _result(*links: IdentityLink) -> ResolutionResult:+ return ResolutionResult(links=list(links))+++def _strong(left: str, right: str, rule: str = "verified_email_match") -> IdentityLink:+ return IdentityLink(left=left, right=right, confidence=0.98, rule=rule, evidence=["same verified address"])+++def _weak(left: str, right: str) -> IdentityLink:+ return IdentityLink(left=left, right=right, confidence=0.45, rule="local_part_only", evidence=["local part only"])+++async def test_a_pair_is_stored_once_however_it_was_discovered(db) -> None:+ """The ordered-pair CHECK, which a writer could otherwise bypass."""+ await bind_tenant(db, TENANT)+ await save_links(db, tenant_id=TENANT, result=_result(_strong("aws:z", "okta:a")), mirror_to_graph=False)+ await bind_tenant(db, TENANT)+ await save_links(db, tenant_id=TENANT, result=_result(_strong("okta:a", "aws:z")), mirror_to_graph=False)+ await db.commit()++ count = (await db.execute(text("SELECT count(*) FROM identity_links WHERE tenant_id = :t"), {"t": TENANT})).scalar_one()+ assert count == 1++ # Negative control: the constraint is real, not merely respected by the+ # writer. An out-of-order insert must be refused by the database — and+ # refused by the CHECK, so the insert has to get past RLS first or the+ # assertion below passes on the wrong error.+ await bind_tenant(db, TENANT)+ with pytest.raises(Exception) as excinfo:+ await db.execute(+ text("INSERT INTO identity_links (tenant_id, left_key, right_key, confidence, rule) VALUES (:t, 'zzz', 'aaa', 0.99, 'manual')"),+ {"t": TENANT},+ )+ await db.commit()+ assert "identity_links_ordered_check" in str(excinfo.value)+ await db.rollback()+++async def test_weak_links_are_stored_and_returned_but_not_walked(db) -> None:+ """Three weak hops must not merge what no single rule would join."""+ await bind_tenant(db, TENANT)+ await save_links(+ db,+ tenant_id=TENANT,+ result=_result(_weak("a:1", "b:2"), _weak("b:2", "c:3"), _weak("c:3", "d:4")),+ mirror_to_graph=False,+ )+ await db.commit()++ shown = await links_for(db, tenant_id=TENANT, key="a:1")+ assert shown, "the weak link was dropped; it must be stored and shown"+ assert all(link["merged"] is False for link in shown)+ assert await cluster_for(db, tenant_id=TENANT, key="a:1") == []++ # Positive control: the walk works at all. Without this, a cluster_for+ # that always returned [] would pass the assertion above.+ await bind_tenant(db, TENANT)+ await save_links(db, tenant_id=TENANT, result=_result(_strong("a:1", "z:9")), mirror_to_graph=False)+ await db.commit()+ assert await cluster_for(db, tenant_id=TENANT, key="a:1") == ["z:9"]+++async def test_a_strong_chain_does_traverse(db) -> None:+ """Transitivity over strong links is intended and must actually work."""+ await bind_tenant(db, TENANT)+ await save_links(+ db,+ tenant_id=TENANT,+ result=_result(_strong("a:1", "b:2"), _strong("b:2", "c:3")),+ mirror_to_graph=False,+ )+ await db.commit()+ assert await cluster_for(db, tenant_id=TENANT, key="a:1") == ["b:2", "c:3"]+++async def test_the_walk_is_bounded(db) -> None:+ """A bounded wrong answer is recoverable; an unbounded one is not."""+ chain = [_strong(f"n:{i}", f"n:{i + 1}") for i in range(6)]+ await bind_tenant(db, TENANT)+ await save_links(db, tenant_id=TENANT, result=_result(*chain), mirror_to_graph=False)+ await db.commit()++ shallow = await cluster_for(db, tenant_id=TENANT, key="n:0", max_depth=1)+ deep = await cluster_for(db, tenant_id=TENANT, key="n:0", max_depth=6)+ assert len(shallow) < len(deep)+ assert "n:1" in shallow+++async def test_a_human_decision_overrides_the_rule_both_ways(db) -> None:+ await bind_tenant(db, TENANT)+ await save_links(db, tenant_id=TENANT, result=_result(_weak("a:1", "b:2")), mirror_to_graph=False)+ await db.commit()+ weak = (await links_for(db, tenant_id=TENANT, key="a:1"))[0]+ assert weak["merged"] is False++ await bind_tenant(db, TENANT)++ confirmed = await decide_link(db, tenant_id=TENANT, link_id=uuid.UUID(weak["id"]), decision="confirmed", user_id=USER)+ await db.commit()+ assert confirmed is not None+ assert confirmed["merged"] is True+ # The confidence is untouched: the rule did not become more certain, a+ # human disagreed with it, and the two facts stay separable.+ assert confirmed["confidence"] == weak["confidence"]+ assert await cluster_for(db, tenant_id=TENANT, key="a:1") == ["b:2"]++ # And the other direction: rejecting a strong link removes it from the walk.+ await bind_tenant(db, TENANT)+ await save_links(db, tenant_id=TENANT, result=_result(_strong("a:1", "c:3")), mirror_to_graph=False)+ await db.commit()+ strong = next(link for link in await links_for(db, tenant_id=TENANT, key="a:1") if link["right"] == "c:3")+ await bind_tenant(db, TENANT)+ await decide_link(db, tenant_id=TENANT, link_id=uuid.UUID(strong["id"]), decision="rejected", user_id=USER)+ await db.commit()+ assert "c:3" not in await cluster_for(db, tenant_id=TENANT, key="a:1")+++async def test_another_tenants_links_are_invisible(db) -> None:+ await bind_tenant(db, OTHER_TENANT)+ await save_links(db, tenant_id=OTHER_TENANT, result=_result(_strong("a:1", "b:2")), mirror_to_graph=False)+ await db.commit()++ assert await links_for(db, tenant_id=TENANT, key="a:1") == []+ assert await cluster_for(db, tenant_id=TENANT, key="a:1") == []+ assert await links_for(db, tenant_id=OTHER_TENANT, key="a:1")+++async def test_the_resolver_and_the_store_agree_end_to_end(db) -> None:+ """The pure half and the stored half must produce the same verdict.++ Without this the resolver could be correct and the store could persist+ something else, which is the shape where two components are each right+ and the product is wrong.+ """+ result = resolve_links(+ [+ {"source": "idp", "id": "[email protected]", "email": "[email protected]", "email_verified": True},+ {+ "source": "cloud_principal",+ "id": "arn:aws:iam::1:user/alice",+ "email": "[email protected]",+ "email_verified": True,+ },+ {"source": "workspace", "id": "[email protected]"},+ ]+ )+ await bind_tenant(db, TENANT)+ await save_links(db, tenant_id=TENANT, result=result, mirror_to_graph=False)+ await db.commit()++ in_memory = {frozenset(c) for c in result.clusters_for_merge()}+ stored = await cluster_for(db, tenant_id=TENANT, key="idp:[email protected]")+ assert {frozenset({"idp:[email protected]", *stored})} == in_memory++ # The weak cross-domain link is stored and visible, and is not in the cluster.+ shown = await links_for(db, tenant_id=TENANT, key="workspace:[email protected]")+ assert shown and all(link["merged"] is False for link in shown)+ assert "workspace:[email protected]" not in storeddiff --git a/services/api/tests/test_identity_resolution_reproduction.py b/services/api/tests/test_identity_resolution_reproduction.pynew file mode 100644index 000000000..4dd9a5c80--- /dev/null+++ b/services/api/tests/test_identity_resolution_reproduction.py@@ -0,0 +1,121 @@+"""Depth 2 Phase 3.1 reproduction: one person is six unrelated strings.++An alert names `[email protected]` from Okta, `ALICE` from an EDR device+login, `arn:aws:iam::1:user/alice` from CloudTrail and `alice-gh` from a code+platform, and the platform treats them as four people. Nothing in the tree+links them, so a story cannot be built across sources and a privilege answer+about one does not reach an alert about another.++The failure mode to design against is not "no links", it is **wrong links**:+merging two people because their local-parts match destroys the thing the+feature exists to provide. So the shape has to carry a confidence and an+evidence list, and a low-confidence link must be visible and never used to+merge.++Fails on the pre-change tree. Kept afterwards as the guard.+"""++from __future__ import annotations++import pytest+++def test_an_identity_resolution_module_exists() -> None:+ from app.services import identity_resolution++ assert hasattr(identity_resolution, "resolve_links")+ assert hasattr(identity_resolution, "LINK_RULES")+++def test_every_source_the_plan_names_is_covered() -> None:+ from app.services.identity_resolution import SOURCE_KINDS++ required = {"idp", "edr_device_user", "proxy_user", "cloud_principal", "code_platform", "workspace"}+ assert required <= set(SOURCE_KINDS), f"missing: {sorted(required - set(SOURCE_KINDS))}"+++def test_a_link_carries_confidence_and_evidence() -> None:+ from app.services.identity_resolution import IdentityLink++ link = IdentityLink(+ left="okta:[email protected]",+ right="aws:arn:aws:iam::1:user/alice",+ confidence=0.95,+ rule="verified_email_match",+ evidence=["both principals carry the verified address [email protected]"],+ )+ assert 0.0 <= link.confidence <= 1.0+ assert link.evidence+ assert link.as_dict()["rule"] == "verified_email_match"+++def test_a_low_confidence_link_is_shown_but_never_merged() -> None:+ """The property the whole item turns on.++ A weak link is useful to an analyst looking at a story and dangerous to+ anything that merges. Both halves have to be true at once: visible, and+ not acted on.+ """+ from app.services.identity_resolution import MERGE_THRESHOLD, resolve_links++ result = resolve_links(+ [+ {"source": "idp", "id": "[email protected]", "display_name": "Alice Smith"},+ # Same local part, different domain, no other corroboration. This+ # is the canonical wrong merge: `[email protected]` is routinely+ # a different person.+ {"source": "workspace", "id": "[email protected]"},+ ]+ )+ weak = [link for link in result.links if link.confidence < MERGE_THRESHOLD]+ assert weak, "the weak link was dropped entirely; it must be shown"+ assert all(not link.merged for link in weak)+ assert result.clusters_for_merge() == [], "a low-confidence link was used to merge"+++def test_a_strong_link_does_merge() -> None:+ """The positive control for the test above.++ Without it, a resolver that linked nothing at all would pass the+ low-confidence assertion perfectly.+ """+ from app.services.identity_resolution import resolve_links++ result = resolve_links(+ [+ {"source": "idp", "id": "[email protected]", "email": "[email protected]", "email_verified": True},+ {+ "source": "cloud_principal",+ "id": "arn:aws:iam::1:user/alice",+ "email": "[email protected]",+ "email_verified": True,+ },+ ]+ )+ assert any(link.merged for link in result.links)+ assert len(result.clusters_for_merge()) == 1+++ "ids",+ [+ ["svc-deploy", "svc-deploy"],+ ["admin", "admin"],+ ],+)+def test_shared_generic_names_never_merge(ids: list[str]) -> None:+ """`admin` on two systems is two accounts, not one person.++ This is the merge that would silently join every tenant's local+ administrator into one identity and take the whole estate's alerts+ with it.+ """+ from app.services.identity_resolution import resolve_links++ result = resolve_links(+ [+ {"source": "edr_device_user", "id": ids[0]},+ {"source": "idp", "id": ids[1]},+ ]+ )+ assert result.clusters_for_merge() == []diff --git a/services/api/tests/test_investigation_tools.py b/services/api/tests/test_investigation_tools.pyindex 047630e7f..5d5ea78b7 100644--- a/services/api/tests/test_investigation_tools.py+++ b/services/api/tests/test_investigation_tools.py@@ -76,6 +76,25 @@ UNBACKED_CALLS: list[tuple[str, dict[str, Any]]] = [("persistence_mechanisms", {"hostname": "WS-42"}),]+#: A third category, added with 1.1. These answer from Postgres rather than+#: the lake, so neither assertion above applies to them: there is no composed+#: lake query to inspect for a tenant predicate, and "unavailable" is not a+#: permanent property of the deployment but a statement about one principal+#: at one moment.+#:+#: They are listed separately rather than bolted onto `BACKED_CALLS` because+#: the tenant-predicate assertions there read the *captured SQL*, and a tool+#: that composes none would pass them by having nothing to fail on. The+#: scoping property for this family is checked by its own live test against+#: real Postgres (`test_identity_privilege_live.py`), where the predicate is+#: a schema fact rather than a string in a mock.+POSTGRES_BACKED_CALLS: list[tuple[str, dict[str, Any]]] = [+ ("identity_effective_permissions", {"principal_id": "alice"}),+ ("identity_effective_permissions", {"principal_id": "alice", "at": "2026-10-01T00:00:00Z", "provider": "aws"}),+ ("changes_before_alert", {"subjects": "alice"}),+ ("changes_before_alert", {"subjects": "alice,arn:aws:s3:::data", "at": "2026-10-01T00:00:00Z", "days": 14}),+]+class TestTenantScoping:@pytest.mark.parametrize(("tool", "args"), BACKED_CALLS)@@ -214,10 +233,44 @@ async def test_every_declared_tool_is_dispatchable(captured: list[Any]) -> None:argument-shape error is a failure rather than something the assertionquietly accepts."""- covered = {tool for tool, _ in BACKED_CALLS + UNBACKED_CALLS}+ all_calls = BACKED_CALLS + UNBACKED_CALLS + POSTGRES_BACKED_CALLS+ covered = {tool for tool, _ in all_calls}assert covered == set(TOOLS), f"tools with no dispatch case in this test: {sorted(set(TOOLS) - covered)}"- for name, args in BACKED_CALLS + UNBACKED_CALLS:+ for name, args in all_calls:result = await dispatch(name, TENANT, args)assert isinstance(result, ToolResult), nameassert "Invalid arguments" not in (result.reason or ""), f"{name} rejected the arguments its own schema advertises: {result.reason}"+++class TestPrivilegeToolNeverFabricates:+ """The Postgres-backed family's own honesty properties.++ With no database reachable in this suite the tool takes its failure+ path, which is exactly the branch worth pinning: the sentence it emits+ is what a model reads, and the wrong sentence here closes alerts.+ """++ async def test_an_unanswerable_lookup_is_never_an_empty_result(self) -> None:+ result = await dispatch("identity_effective_permissions", TENANT, {"principal_id": "alice"})+ assert result.available is False+ assert result.rows == []+ # The two sentences the tool is allowed to say, and both of them+ # name the gap. An `available=false` with no reason would read to a+ # model as a tool that is simply switched off.+ assert result.reason+ # Both of the tool's failure sentences end the same way, which is+ # the property: whichever branch fires, the model is told the+ # absence is not a finding.+ assert "not an absence" in result.reason or "not evidence" in result.reason++ async def test_a_model_supplied_tenant_is_discarded(self) -> None:+ """Same property as the lake tools, checked on this door too."""+ result = await dispatch(+ "identity_effective_permissions",+ TENANT,+ {"principal_id": "alice", "tenant_id": OTHER_TENANT},+ )+ assert isinstance(result, ToolResult)+ assert "Invalid arguments" not in (result.reason or "")+ assert OTHER_TENANT not in (result.reason or "")diff --git a/services/fusion/app/main.py b/services/fusion/app/main.pyindex bcebef080..5c080ab3a 100644--- a/services/fusion/app/main.py+++ b/services/fusion/app/main.py@@ -27,6 +27,7 @@ from app.services.dlq_sink import PostgresDLQfrom app.services.entity_risk import EntityRiskEnginefrom app.services.first_seen import FirstSeenStorefrom app.services.fusion_engine import FusionEngine+from app.services.identity_privilege_context import IdentityPrivilegeResolverfrom app.services.ioc_match import TenantIocMatcherfrom app.services.lake_writer import LakeWriterfrom app.services.sessions import SessionStore@@ -110,6 +111,11 @@ async def lifespan(app: FastAPI):# import writes); unset DATABASE_URL leaves the scorer's factor honestly# reading "unknown (no inventory match)".asset_resolver = AssetContextResolver(settings.database_url) if settings.database_url else None+ # Depth 2 wave 2, 1.1 — the actor's privilege as of the event, read from+ # `identity_privilege_facts` (migration 124) in the same Postgres. Unset+ # DATABASE_URL leaves every alert stamped "unknown", which is what a+ # deployment that has collected no posture should see.+ privilege_resolver = IdentityPrivilegeResolver(settings.database_url) if settings.database_url else Noneengine = FusionEngine(dedup,correlator,@@ -120,6 +126,7 @@ async def lifespan(app: FastAPI):enricher=enricher,asset_resolver=asset_resolver,memory_provider=memory_provider,+ privilege_resolver=privilege_resolver,)# Phase 3.1 — fused alerts land in the Postgres alert store so the spine# is continuous (raw event → alert row). Fail-soft: a missing/unreachablediff --git a/services/fusion/app/services/baselines.py b/services/fusion/app/services/baselines.pyindex 123554b93..966374920 100644--- a/services/fusion/app/services/baselines.py+++ b/services/fusion/app/services/baselines.py@@ -154,6 +154,60 @@ PEER_GROUP_FIELDS: tuple[str, ...] = ("department",)+#: The kinds of actor that get a baseline of their own (depth plan wave 2,+#: 1.4). Every one of these acts *on a human's behalf*, which is why folding+#: them into the human's baseline is backwards for the case these baselines+#: exist to catch: a stolen token acts as the user, so the user's own volume+#: and rarity stay normal while the token's do not.+#:+#: `oauth_app` is listed here as well as keeping its older volume-only+#: baseline, so the two namespaces do not collide: the old one is keyed+#: `vol:oauth_app` on the app name, the new one is keyed on+#: `sub:<kind>:<id>` and carries rarity too.+SUB_PRINCIPAL_KINDS: tuple[str, ...] = ("oauth_app", "api_token", "service_account", "ai_agent")++#: Vendor fields that name a token, per connector. Order is precedence.+#:+#: Deliberately **not** including GitHub's credential digest. That value is+#: a session key, and the session key has exactly one producer:+#: `services/ingest/internal/activity/session.go`. A consumer that re-derived+#: it would be the second derivation ADR-0010 forbids -- the two would agree+#: until the next vendor-schema change and then disagree silently, leaving+#: Postgres, the lake and the graph holding different answers to "which+#: session was this". :func:`_session_credential_key` reads the producer's+#: projection instead, and `scripts/check_session_projection.py` is the gate+#: that refuses the shortcut.+TOKEN_FIELDS: tuple[str, ...] = (+ "token_id",+ "access_key_id",+ "accessKeyId",+ "api_key_id",+)++#: `session.key_kind` values whose key *is* the credential, so the session+#: key doubles as the token's identity. `session.go` documents the+#: vocabulary; `vendor_credential` is the one GitHub's digest lands under.+CREDENTIAL_KEY_KINDS: frozenset[str] = frozenset({"vendor_credential"})++#: Fields naming a service account or workload identity.+SERVICE_ACCOUNT_FIELDS: tuple[str, ...] = (+ "service_account",+ "serviceAccountEmail",+ "service_account_email",+ "workload_identity",+)++#: Fields naming an AI agent acting for a person. Kept distinct from a+#: service account: an agent's cadence is interactive by nature, so folding+#: it into the service-account baseline would make every agent look like a+#: cron job that went rogue.+AI_AGENT_FIELDS: tuple[str, ...] = (+ "agent_id",+ "ai_agent",+ "assistant_id",+ "agent_name",+)+#: Every derived field this module can produce. Asserted against the gate's#: own table in `services/fusion/tests/test_baselines.py`, in both#: directions, so the two cannot drift the way the matcher's operator list@@ -164,9 +218,33 @@ DERIVED_FIELDS: frozenset[str] = frozenset("baseline_principal_volume_z","baseline_oauth_app_volume_z","baseline_peer_deviation",+ # Depth plan wave 2, 1.4.+ "baseline_sub_principal_kind",+ "baseline_sub_principal_rarity",+ "baseline_sub_principal_volume_z",+ "baseline_token_cadence",+ "baseline_token_cadence_shift",+ "baseline_token_reactivated_after_idle",+ "baseline_token_new_asn_after_creation",})+#: A token silent for this long and then active again is worth a signal.+#: Sixty days from the plan; it is also comfortably longer than any leave+#: pattern, so a returning holidaymaker's token does not fire it.+IDLE_REACTIVATION_SECONDS = 60 * 24 * 3600++#: How soon after a token first appears a new ASN is suspicious. A token+#: created and used from somewhere else within the hour is the shape of a+#: credential that left the building before its first legitimate use.+NEW_TOKEN_ASN_WINDOW_SECONDS = 3600++#: Calls needed before a cadence claim is made. Three calls do not+#: establish a rhythm, and `scripted` returned for an unwatched token would+#: make the cadence-shift signal fire on the fourth call of every new token+#: in the estate.+CADENCE_MIN_SAMPLES = 8+def _first_present(event: dict[str, Any], candidates: tuple[str, ...]) -> str | None:"""The first candidate field carrying a non-empty value, normalised."""@@ -185,6 +263,159 @@ def resolve_principal(event: dict[str, Any]) -> str | None:return _first_present(event, PRINCIPAL_FIELDS)+@dataclass(frozen=True)+class SubPrincipal:+ """A token, app, service account or agent acting on a human's behalf."""++ kind: str+ id: str+ #: The human it acts for, where the event names one. Carried so a+ #: signal about the token can be attributed to a person without a+ #: second lookup, and left empty rather than guessed when it does not.+ on_behalf_of: str = ""++ @property+ def subject(self) -> str:+ """The baseline key. Namespaced by kind so two vendors' ids cannot collide."""+ return f"sub:{self.kind}:{self.id}"+++def _verbatim(event: dict[str, Any], candidates: tuple[str, ...]) -> str | None:+ """First present candidate, trimmed and **not** case-folded.++ Every other field in this module goes through :func:`_first_present`,+ which lowercases — correct for a username, wrong for an opaque token id+ or a digest. A lowercased key would never join to what the producer+ wrote, on every event, silently and with no error anywhere.+ """+ for name in candidates:+ value = event.get(name)+ # A string, not anything stringifiable: coercing a number here would+ # mint a baseline key under an id no other store uses.+ if not isinstance(value, str):+ continue+ text = value.strip()+ if text:+ return text+ return None+++def _session_credential_key(event: dict[str, Any]) -> str | None:+ """The session key, when the vendor's session *is* a credential.++ Reads `activity.session` — what `session.go` wrote — rather than the+ vendor field it was derived from. That is the whole point: one producer,+ many consumers, so the baseline store, the session store, the lake and+ the graph cannot disagree about which session an event belongs to.++ Returns ``None`` for every other `key_kind`, including a vendor session+ id: a session id is not a token identity, and keying a token baseline on+ one would merge every token used in that session into a single subject.+ """+ activity = event.get("activity")+ if not isinstance(activity, dict):+ return None+ session = activity.get("session")+ if not isinstance(session, dict):+ return None+ if str(session.get("key_kind") or "") not in CREDENTIAL_KEY_KINDS:+ return None+ key = session.get("key")+ if not isinstance(key, str):+ return None+ text = key.strip()+ return text or None+++def resolve_sub_principal(event: dict[str, Any]) -> SubPrincipal | None:+ """The sub-principal acting, or ``None`` when the event names only a human.++ Precedence is narrowest-first: a token is more specific than the app it+ belongs to, which is more specific than the service account. An event+ naming several gets the narrowest, because that is the subject whose+ baseline a compromise would move.+ """+ on_behalf_of = resolve_principal(event) or ""++ # The producer's projection first. Where the vendor's session *is* a+ # credential — a GitHub token is the case this was built for — the+ # session key is the token's identity, already trimmed and un-folded by+ # the one component allowed to derive it.+ credential = _session_credential_key(event)+ if credential:+ return SubPrincipal(kind="api_token", id=credential, on_behalf_of=on_behalf_of)++ token = _verbatim(event, TOKEN_FIELDS)+ if token:+ return SubPrincipal(kind="api_token", id=token, on_behalf_of=on_behalf_of)++ agent = _first_present(event, AI_AGENT_FIELDS)+ if agent:+ return SubPrincipal(kind="ai_agent", id=agent, on_behalf_of=on_behalf_of)++ app = _first_present(event, OAUTH_APP_FIELDS)+ if app:+ return SubPrincipal(kind="oauth_app", id=app, on_behalf_of=on_behalf_of)++ account = _first_present(event, SERVICE_ACCOUNT_FIELDS)+ if account:+ return SubPrincipal(kind="service_account", id=account, on_behalf_of=on_behalf_of)++ return None+++def classify_cadence(hours: list[int]) -> str:+ """``scheduled`` | ``scripted`` | ``interactive`` | ``unknown``.++ From the hours-of-day a token has been seen at. The three classes are+ separated by *regularity* and *spread*, which is what the signal needs:+ the interesting transition is a token that behaved like a machine+ starting to behave like a person at a keyboard.++ * **scheduled** — even intervals across the clock. A cron job is the+ canonical case, and the tell is that the gaps between successive+ active hours barely vary.+ * **interactive** — concentrated inside a working day, with repeats+ within the same hour. A person.+ * **scripted** — regular but not evenly spaced, or spread without the+ within-hour clustering of a human.+ * **unknown** below :data:`CADENCE_MIN_SAMPLES`. Returning a class for+ a token nobody has watched would make the cadence-shift signal fire+ on the ninth call of every new token in the estate.+ """+ if len(hours) < CADENCE_MIN_SAMPLES:+ return "unknown"++ distinct = sorted({h % 24 for h in hours})+ if len(distinct) < 2:+ # Everything in one hour of the day, repeatedly: a daily job.+ return "scheduled"++ spread = len(distinct)+ repeats = len(hours) - spread++ # Interactive is tested **first**, and the order is the whole+ # classifier. A person working 09:00-17:00 produces consecutive active+ # hours whose gaps are almost all 1, which is indistinguishable from an+ # hourly cron by regularity alone. What separates them is that a person+ # calls several times within the same hour and a timer does not, so the+ # repeat count has to be consulted before the regularity is.+ if repeats >= 2 and max(distinct) - min(distinct) <= 12 and spread <= 12:+ return "interactive"++ # Gaps between consecutive active hours. The wrap-around gap is+ # deliberately excluded: an hourly job that runs only through the+ # morning has one 13-hour gap at the end of its day, and counting it+ # would make every part-time schedule look irregular.+ gaps = [b - a for a, b in zip(distinct, distinct[1:], strict=False)]+ mean_gap = sum(gaps) / len(gaps)+ variance = sum((g - mean_gap) ** 2 for g in gaps) / len(gaps)++ if variance <= 1.0 and spread >= 6:+ return "scheduled"+ return "scripted"++@dataclass(frozen=True)class AnomalySignal:"""One typed behavioural observation. Never an alert by itself.@@ -421,6 +652,11 @@ class BaselineStore:if app_volume is not None:out["baseline_oauth_app_volume_z"] = round(app_volume, 4)+ # Depth plan wave 2, 1.4. The sub-principal gets a baseline of its+ # own, on the same store, so a stolen token moves a number even+ # while the human it acts for looks entirely normal.+ out.update(await self._sub_principal_fields(tenant, event, now=now))+group = _first_present(event, PEER_GROUP_FIELDS)if group is not None:# The member's own hourly volume is the quantity compared across@@ -434,6 +670,106 @@ class BaselineStore:return out+ async def _sub_principal_fields(self, tenant: str, event: dict[str, Any], *, now: float | None = None) -> dict[str, Any]:+ """The four 1.4 signals for whichever sub-principal acted.++ Every one is fail-soft the same way the rest of the store is: an+ unreachable Redis contributes no key. A missing key reads as "no+ observation", never as a zero, because the consumers are detection+ rules and a zero would be a claim.+ """+ subject = resolve_sub_principal(event)+ if subject is None:+ return {}++ when = now if now is not None else time.time()+ out: dict[str, Any] = {"baseline_sub_principal_kind": subject.kind}++ # Rarity of this action for *this token*, which is the question the+ # human's own rarity cannot answer.+ action = _first_present(event, RARITY_FIELDS["baseline_action_rarity"])+ if action is not None:+ counted = await self.observe_value(tenant, subject.subject, "action", action)+ if counted is not None:+ count, total = counted+ if total >= self._min_observations:+ out["baseline_sub_principal_rarity"] = round(self.rarity(count, total), 4)++ volume = await self.observe_volume(tenant, subject.subject, subject.kind, now=when)+ if volume is not None:+ out["baseline_sub_principal_volume_z"] = round(volume, 4)++ state = await self._advance_token_state(tenant, subject, event, when)+ out.update(state)+ return out++ async def _advance_token_state(+ self,+ tenant: str,+ subject: SubPrincipal,+ event: dict[str, Any],+ when: float,+ ) -> dict[str, Any]:+ """Cadence, idle reactivation and first-ASN-after-creation, in one hash.++ One Redis hash per sub-principal holding the hours it has been seen+ at, its last-seen timestamp, its first-seen timestamp and the ASNs+ it has used. Three signals out of one round trip, because this runs+ on the fusion hot path.+ """+ key = self._key(tenant, subject.subject, "token")+ hour = int((when % 86400) // 3600)+ asn = _first_present(event, ("asn", "src_asn", "autonomous_system", "as_number"))++ try:+ state = await self._redis.hgetall(key)+ previous_cadence = _field(state, "cadence") or "unknown"+ last_seen = float(_field(state, "last_seen") or 0.0)+ first_seen = float(_field(state, "first_seen") or 0.0)+ seen_hours = [int(h) for h in (_field(state, "hours") or "").split(",") if h.strip().isdigit()]+ known_asns = {a for a in (_field(state, "asns") or "").split(",") if a}++ seen_hours = (seen_hours + [hour])[-48:]+ cadence = classify_cadence(seen_hours)++ updates: dict[str, Any] = {+ "hours": ",".join(str(h) for h in seen_hours),+ "last_seen": repr(when),+ "cadence": cadence,+ }+ if first_seen <= 0:+ first_seen = when+ updates["first_seen"] = repr(when)+ if asn:+ updates["asns"] = ",".join(sorted({*known_asns, asn}))++ await self._redis.hset(key, mapping=updates)+ await self._redis.expire(key, self._ttl)+ except Exception as exc: # noqa: BLE001 — see the class docstring+ logger.debug("baselines.token_state_unavailable", error=str(exc))+ return {}++ out: dict[str, Any] = {"baseline_token_cadence": cadence}++ # A machine identity that starts behaving like a person. Only+ # claimed when *both* classes are known: a shift out of `unknown`+ # is the token being watched long enough, not the token changing.+ if previous_cadence in {"scheduled", "scripted"} and cadence == "interactive" and previous_cadence != cadence:+ out["baseline_token_cadence_shift"] = f"{previous_cadence}->{cadence}"++ if last_seen > 0 and (when - last_seen) >= IDLE_REACTIVATION_SECONDS:+ out["baseline_token_reactivated_after_idle"] = round((when - last_seen) / 86400.0, 2)++ # A new ASN *shortly after the token first appeared*. Not "a new ASN+ # ever": tokens legitimately move networks, and a signal that fires+ # on every office-to-home switch is one that gets tuned out. The+ # window makes this the specific shape of a credential that left+ # the building before its first legitimate use.+ if asn and asn not in known_asns and known_asns and (when - first_seen) <= NEW_TOKEN_ASN_WINDOW_SECONDS:+ out["baseline_token_new_asn_after_creation"] = asn++ return out+async def signals(self, tenant: str, event: dict[str, Any], *, now: float | None = None) -> list[AnomalySignal]:"""The same observations as typed signals, for consumers that want them.diff --git a/services/fusion/app/services/confidence.py b/services/fusion/app/services/confidence.pyindex af907bb0c..ae0983cbb 100644--- a/services/fusion/app/services/confidence.py+++ b/services/fusion/app/services/confidence.py@@ -24,23 +24,39 @@ clamped to [0, 1]. Bands map to labels:Factors and weights--------------------================== ====== ==========================================-factor w signal-================== ====== ==========================================-severity 0.18 alert severity (critical / high / med / low)-ml_anomaly 0.17 IsolationForest anomaly_score (0–1)-ml_priority 0.17 LightGBM priority_score (0–1)-mitre_coverage 0.14 number of mapped MITRE techniques-threat_intel 0.15 enrichment hits (TI feed match)-upstream_risk 0.08 raw_alert.risk_score (0–1, vendor-provided)-ioc_density 0.05 distinct populated IOC fields-asset_criticality 0.06 tenant asset inventory (fuse-time stamp)-================== ====== ==========================================+================== ======= ==========================================+factor w signal+================== ======= ==========================================+severity 0.1692 alert severity (critical / high / med / low)+ml_anomaly 0.1598 IsolationForest anomaly_score (0–1)+ml_priority 0.1598 LightGBM priority_score (0–1)+threat_intel 0.1410 enrichment hits (TI feed match)+mitre_coverage 0.1316 number of mapped MITRE techniques+upstream_risk 0.0752 raw_alert.risk_score (0–1, vendor-provided)+identity_privilege 0.0600 actor's privilege + tags at event time+asset_criticality 0.0564 tenant asset inventory (fuse-time stamp)+ioc_density 0.0470 distinct populated IOC fields+================== ======= ==========================================Sum of weights = 1.0. The factor list is *also* what's persisted on the``FusedAlert`` so the UI can render the same evidence chain that thescore was derived from — no magic numbers.+Where the four-decimal weights come from+----------------------------------------+``identity_privilege`` (depth plan wave 2, 1.1) was added at 0.06 — the same+weight ``asset_criticality`` carries, because it answers the matching+question about the actor rather than the asset — and the eight pre-existing+weights were scaled by 0.94 so the total is still exactly 1.0. Scaling,+rather than taking the new weight out of one neighbour, keeps every existing+factor's share of the model unchanged relative to the others: the only thing+that moved is that all of them together now account for 94% instead of 100%.++That does shift every alert's score slightly, which is a real change and is+measured rather than waved at. It does **not** reorder two alerts that+differ only in pre-existing factors, because a common positive scale factor+preserves the ordering of ``Σ(w·c)``.+AiSOC — open-source AI Security Operations Center (MIT License)"""@@ -69,14 +85,23 @@ HIGH_THRESHOLD = 0.70LOW_THRESHOLD = 0.40# Factor weights — keep summing to 1.0 if you tweak.-WEIGHT_SEVERITY = 0.18-WEIGHT_ML_ANOMALY = 0.17-WEIGHT_ML_PRIORITY = 0.17-WEIGHT_MITRE = 0.14-WEIGHT_THREAT_INTEL = 0.15-WEIGHT_UPSTREAM_RISK = 0.08-WEIGHT_IOC_DENSITY = 0.05-WEIGHT_ASSET_CRITICALITY = 0.06+# `test_weights_sum_to_one` holds this, and it is not decoration: the score+# is `0.5 + Σ(w·c)` with contributions in [-1, +1], so a total above 1.0+# lets a single all-positive alert clamp at the ceiling and a total below it+# makes `HIGH` unreachable.+WEIGHT_SEVERITY = 0.1692+WEIGHT_ML_ANOMALY = 0.1598+WEIGHT_ML_PRIORITY = 0.1598+WEIGHT_MITRE = 0.1316+WEIGHT_THREAT_INTEL = 0.1410+WEIGHT_UPSTREAM_RISK = 0.0752+WEIGHT_IOC_DENSITY = 0.0470+WEIGHT_ASSET_CRITICALITY = 0.0564+#: Depth plan wave 2, 1.1. Deliberately the same magnitude as asset+#: criticality: both are context about *what the alert is about* rather than+#: evidence that it happened, and letting either outweigh the detection+#: itself is how a prioritisation scheme gets switched off.+WEIGHT_IDENTITY_PRIVILEGE = 0.0600# What the asset the alert landed on contributes. Unknown — no inventory, no# match — is exactly 0.0: the absence of a CMDB row is not evidence in either@@ -90,6 +115,31 @@ _ASSET_CRITICALITY_CONTRIBUTION = {"none": -0.5,}+# What the actor's privilege contributes. Same shape and same reasoning as+# the asset table above: `unknown` is exactly 0.0, because "no posture has+# been collected for this provider" is not evidence in either direction, and+# the factor row still appears so an analyst sees that the platform could not+# answer rather than seeing nothing at all.+#+# `standard` is mildly negative for the same reason `low` criticality is: an+# ordinary account doing something is less interesting than an administrator+# doing it, and a scheme where context can only ever increase a score stops+# being able to deprioritise anything.+_PRIVILEGE_TIER_CONTRIBUTION = {+ "domain_admin": 1.0,+ "admin": 0.7,+ "elevated": 0.3,+ "standard": -0.2,+ "unknown": 0.0,+}++#: Each tag that is *true* adds this much, capped below. Tags are evidence+#: about reach, and reach compounds: an identity that can escalate AND+#: reaches another account is worse than either alone. The cap stops four+#: tags from making the tier irrelevant.+_PRIVILEGE_TAG_STEP = 0.2+_PRIVILEGE_TAG_CAP = 0.6+_SEVERITY_CONTRIBUTION = {AlertSeverity.CRITICAL: 1.0,AlertSeverity.HIGH: 0.6,@@ -147,6 +197,43 @@ def _ti_contribution(enrichments: dict | None) -> tuple[float, str]:return 1.0, " + ".join(sources)+#: The key `services/fusion/app/services/identity_privilege_context.py`+#: writes. Imported as a literal rather than from that module so the scorer+#: keeps no import of the thing that opens a database connection.+_PRIVILEGE_STAMP_KEY = "identity_privilege"+++def _privilege_contribution(stamp: object) -> tuple[float, str]:+ """Tier plus capped tag steps, and the sentence the UI renders.++ Returns ``(contribution, value)``. The value string is what an analyst+ reads in the evidence chain, so it says "unknown" in words whenever the+ contribution is zero for lack of data — a 0.00 with no explanation is+ indistinguishable from a measured neutral.+ """+ if not isinstance(stamp, dict):+ return 0.0, "unknown (no posture collected)"++ tier = str(stamp.get("tier") or "unknown").lower()+ if tier == "unknown":+ reason = str(stamp.get("reason") or "no posture snapshot covers this event")+ return 0.0, f"unknown ({reason})"++ contribution = _PRIVILEGE_TIER_CONTRIBUTION.get(tier, 0.0)++ tags = stamp.get("tags_set")+ if not isinstance(tags, list):+ raw_tags = stamp.get("tags")+ tags = [k for k, v in raw_tags.items() if v is True] if isinstance(raw_tags, dict) else []+ tag_bonus = min(_PRIVILEGE_TAG_CAP, len(tags) * _PRIVILEGE_TAG_STEP)++ contribution = max(-1.0, min(1.0, contribution + tag_bonus))+ value = tier if not tags else f"{tier} ({', '.join(sorted(tags))})"+ if stamp.get("snapshot_partial"):+ value = f"{value} — snapshot incomplete"+ return contribution, value++def _ioc_density_contribution(alert) -> tuple[float, int]:"""More populated IOC fields → more concrete signal."""fields = [@@ -309,7 +396,22 @@ class ConfidenceScorer:))- # 9. Institutional memory nudge (Wave 1) — a bounded ±MEMORY_CAP+ # 9. Actor privilege at event time (depth 2 wave 2, 1.1). Read from+ # the fuse-time stamp (`enrichments["identity_privilege"]`), never+ # queried here — the scorer stays a pure projection, same contract+ # as the asset factor above.+ privilege_contribution, privilege_value = _privilege_contribution((fused.enrichments or {}).get(_PRIVILEGE_STAMP_KEY))+ rationale.append(+ ConfidenceFactor(+ factor="identity_privilege",+ label="Actor privilege at event time",+ value=privilege_value,+ contribution=privilege_contribution,+ weight=WEIGHT_IDENTITY_PRIVILEGE,+ )+ )++ # 10. Institutional memory nudge (Wave 1) — a bounded ±MEMORY_CAP# adjustment from the distilled per-signature prior. A signature the# analysts have repeatedly corrected to benign pulls new alerts of that# signature down; one repeatedly confirmed nudges up. Unknowndiff --git a/services/fusion/app/services/fusion_engine.py b/services/fusion/app/services/fusion_engine.pyindex ce40a9832..95ee8370c 100644--- a/services/fusion/app/services/fusion_engine.py+++ b/services/fusion/app/services/fusion_engine.py@@ -23,6 +23,7 @@ from app.services.confidence import ConfidenceScorerfrom app.services.correlator import Correlatorfrom app.services.deduplicator import Deduplicatorfrom app.services.entity_risk import EntityRiskEngine, EntityRiskRecord+from app.services.identity_privilege_context import STAMP_KEY as PRIVILEGE_STAMP_KEYfrom app.services.ml_scorer import MLScorerfrom app.services.narrative import (NarrativeFactor,@@ -127,6 +128,7 @@ class FusionEngine:enricher: AlertEnricher | None = None,asset_resolver: Any = None,memory_provider: Any = None,+ privilege_resolver: Any = None,) -> None:self._dedup = deduplicatorself._correlator = correlator@@ -146,6 +148,11 @@ class FusionEngine:# scorer's asset_criticality factor (optional; None = factor reads# "unknown"). `app.services.asset_context.AssetContextResolver`.self._asset_resolver = asset_resolver+ # Depth 2 wave 2, 1.1 — the privilege the actor held at event time,+ # for the scorer's `identity_privilege` factor and the console+ # queue's priority basis (optional; None = every alert stamps+ # "unknown", which is the honest state and not a silent default).+ self._privilege_resolver = privilege_resolver# Confidence + explainability is intrinsic to a fused alert — every# alert leaves the engine with a high/med/low label and an evidence# chain. The scorer is pure / stateless so we instantiate a default.@@ -227,6 +234,21 @@ class FusionEngine:except Exception as exc:logger.warning("fuse_asset_context_failed", error=str(exc))+ # --- Step 2.7: Privilege at event time (depth 2 wave 2, 1.1) ---+ # The actor half of the same question step 2.6 asks about the asset.+ # Note the asymmetry with the stamp above, which is deliberate: that+ # one writes only on a match, this one writes `unknown` too. An+ # absent asset row means the tenant has no CMDB entry for a host,+ # which is ordinary; an absent privilege fact can equally mean the+ # provider's posture has never been collected, and the scorer and+ # the queue both need to be able to tell that from an ordinary+ # account.+ if self._privilege_resolver is not None:+ try:+ fused.enrichments[PRIVILEGE_STAMP_KEY] = await self._privilege_resolver.resolve(alert)+ except Exception as exc:+ logger.warning("fuse_identity_privilege_failed", error=str(exc))+# --- Step 3: ML scoring ---try:fused = await self._ml_scorer.score(fused)diff --git a/services/fusion/app/services/identity_privilege_context.py b/services/fusion/app/services/identity_privilege_context.pynew file mode 100644index 000000000..b663795e9--- /dev/null+++ b/services/fusion/app/services/identity_privilege_context.py@@ -0,0 +1,189 @@+"""Fuse-time privilege stamp — what could the actor do *when it acted*?++Depth plan wave 2, Phase 1.1. The sibling of ``asset_context.py``: the alert+landed on something (that file) and was caused by somebody (this one).++Why a Postgres read and not a resolver call+-------------------------------------------+The resolver lives in ``services/api`` and takes an estate-sized policy+document. Running it per event is not an option on fusion's hot path, and+importing it is not an option at all — both services name their top-level+package ``app``, so importing one in a process shadows the other.++So the API derives a small row per principal once per posture version+(``identity_privilege_facts``, migration 124) and fusion reads it the way+``AssetContextResolver`` reads ``assets``: one indexed query, a short TTL+cache, and never an exception out of the hot path.++The point-in-time read is the whole feature+-------------------------------------------+The lookup is bounded by the **event's** timestamp, not ``now()``. An alert+is fused seconds after the event but triaged minutes to days later, and a+role revoked in between is exactly the case the posture table is versioned+for. A stamp that read the live row would answer "what can this identity do"+on a surface labelled "what could it do", which is the worse failure of the+two because the number looks right.++Unknown is a stamp, not a missing stamp+---------------------------------------+When no fact covers the event time the stamp is still written, with+``tier: "unknown"`` and all four tags ``None``. Writing nothing would make+the scorer's factor read like an absent signal, and an operator could not+tell "this account is ordinary" from "we have never collected this+provider's posture" — which is the difference between a quiet alert and a+blind spot.+"""++from __future__ import annotations++import os+import time+from datetime import datetime+from typing import Any++import structlog++from app.models.alert import RawAlert++logger = structlog.get_logger()++#: Shorter than the asset cache's 300 s. Posture changes on human timescales+#: too, but a privilege *grant* is the event an attacker creates and then+#: uses, so the window in which fusion would stamp the pre-grant tier is the+#: one worth keeping small.+_CACHE_TTL_SECONDS = 120.0+_QUERY_TIMEOUT_SECONDS = 2.0++#: The key fusion writes into ``fused.enrichments``. The confidence scorer+#: and the API's queue both read it by this name; there is no second spelling.+STAMP_KEY = "identity_privilege"++UNKNOWN_STAMP: dict[str, Any] = {+ "tier": "unknown",+ "measured": False,+ "tags": {"can_escalate": None, "data_access": None, "third_party": None, "cross_account": None},+ "tags_set": [],+ "reason": "no posture snapshot covers this event time for this principal",+}+++def _asyncpg_dsn(url: str) -> str:+ return url.replace("postgresql+asyncpg://", "postgresql://", 1)+++class IdentityPrivilegeResolver:+ """Resolve an alert's actor to the privilege it held at event time."""++ def __init__(self, database_url: str | None = None) -> None:+ url = (database_url or os.getenv("DATABASE_URL") or "").strip()+ self._dsn = _asyncpg_dsn(url) if url else ""+ self._pool: Any = None+ self._cache: dict[tuple[str, str, str], tuple[float, dict[str, Any]]] = {}++ @property+ def configured(self) -> bool:+ return bool(self._dsn)++ async def _ensure_pool(self) -> Any:+ if self._pool is None:+ import asyncpg # noqa: PLC0415 — optional at import time++ self._pool = await asyncpg.create_pool(self._dsn, min_size=0, max_size=2, timeout=_QUERY_TIMEOUT_SECONDS)+ return self._pool++ async def close(self) -> None:+ if self._pool is not None:+ await self._pool.close()+ self._pool = None++ async def resolve(self, alert: RawAlert) -> dict[str, Any]:+ """The privilege stamp for this alert. Never raises, never ``None``.++ Returns :data:`UNKNOWN_STAMP` for every reason it cannot answer — no+ actor on the alert, no database, no fact covering the time. The+ caller always writes the stamp, so "unknown" reaches the scorer and+ the queue as a value rather than as an absence.+ """+ principal = (alert.username or "").strip()+ if not principal or not self._dsn:+ return dict(UNKNOWN_STAMP)++ at = alert.timestamp or datetime.now(tz=_utc())+ # Bucketed to the minute so two alerts a second apart share a cache+ # entry. Posture intervals are hours to days long, so a minute of+ # granularity cannot select a different version in any case the+ # feature is for.+ bucket = at.replace(second=0, microsecond=0).isoformat()+ key = (str(alert.tenant_id), principal.lower(), bucket)++ now = time.monotonic()+ cached = self._cache.get(key)+ if cached is not None and now - cached[0] < _CACHE_TTL_SECONDS:+ return dict(cached[1])++ try:+ pool = await self._ensure_pool()+ row = await pool.fetchrow(+ """+ SELECT provider, principal_id, tier,+ can_escalate, data_access, third_party, cross_account,+ action_count, partial_source, valid_from+ FROM identity_privilege_facts+ WHERE tenant_id = $1+ AND valid_from <= $2+ AND (valid_until IS NULL OR valid_until > $2)+ AND (principal_id = $3 OR lower(principal_label) = lower($3))+ ORDER BY array_position(+ ARRAY['unknown','standard','elevated','admin','domain_admin']::text[], tier+ ) DESC,+ valid_from DESC+ LIMIT 1+ """,+ alert.tenant_id,+ at,+ principal,+ timeout=_QUERY_TIMEOUT_SECONDS,+ )+ except Exception as exc: # noqa: BLE001 — never block fusion on posture+ logger.warning("identity_privilege.resolve_failed", error=str(exc))+ # Deliberately not cached. An outage would otherwise pin every+ # alert for this principal to "unknown" for the whole TTL, long+ # after the database came back.+ return dict(UNKNOWN_STAMP)++ stamp = _row_to_stamp(row) if row is not None else dict(UNKNOWN_STAMP)+ self._cache[key] = (now, stamp)+ return dict(stamp)+++def _utc(): # noqa: ANN202 — tiny shim, typing it adds nothing+ from datetime import UTC # noqa: PLC0415++ return UTC+++def _row_to_stamp(row: Any) -> dict[str, Any]:+ tags = {+ "can_escalate": row["can_escalate"],+ "data_access": row["data_access"],+ "third_party": row["third_party"],+ "cross_account": row["cross_account"],+ }+ stamp: dict[str, Any] = {+ "tier": str(row["tier"] or "unknown"),+ # A stored row whose tier is 'unknown' is still a measured absence —+ # the provider was collected and this principal was not in it — which+ # is a different fact from never having collected. Both render as+ # unknown; only this one has a snapshot behind it.+ "measured": True,+ "provider": row["provider"],+ "principal_id": row["principal_id"],+ "tags": tags,+ "tags_set": [name for name, value in tags.items() if value is True],+ "action_count": int(row["action_count"] or 0),+ "snapshot_partial": bool(row["partial_source"]),+ "valid_from": row["valid_from"].isoformat() if row["valid_from"] else None,+ }+ if stamp["snapshot_partial"]:+ stamp["reason"] = "the posture snapshot behind this tier was incomplete; the real reach may be larger"+ return stampdiff --git a/services/fusion/tests/test_confidence.py b/services/fusion/tests/test_confidence.pyindex 84a26efa8..b519acdb6 100644--- a/services/fusion/tests/test_confidence.py+++ b/services/fusion/tests/test_confidence.py@@ -29,6 +29,8 @@ from app.models.alert import (from app.services.confidence import (HIGH_THRESHOLD,LOW_THRESHOLD,+ WEIGHT_ASSET_CRITICALITY,+ WEIGHT_IDENTITY_PRIVILEGE,ConfidenceScorer,)@@ -107,6 +109,11 @@ def test_score_attaches_complete_rationale(scorer: ConfidenceScorer) -> None:"upstream_risk","ioc_density","asset_criticality",+ # Depth 2 wave 2, 1.1. Present on *every* alert, including ones with+ # no posture behind them, where it reads "unknown". That is the+ # whole point: a factor that appears only when the answer is known+ # cannot tell an analyst that the platform could not answer.+ "identity_privilege",}@@ -362,5 +369,110 @@ def test_low_criticality_pulls_down_but_is_bounded(scorer: ConfidenceScorer) ->scored = scorer.score(fused)factor = next(f for f in scored.confidence_rationale if f.factor == "asset_criticality")assert factor.contribution == -0.3- # Bounded: weight 0.06, so the full swing is ±0.03 on the score.- assert factor.weight == 0.06+ # Bounded: weight 0.0564, so the full swing is ±0.028 on the score. The+ # weight was 0.06 until 1.1 added `identity_privilege` and scaled the+ # eight pre-existing factors by 0.94 to keep the total at exactly 1.0.+ assert factor.weight == WEIGHT_ASSET_CRITICALITY+ assert factor.weight == pytest.approx(0.06 * 0.94)+++# ─── Actor privilege at event time (depth 2 wave 2, 1.1) ─────────────────────+++def _privilege_factor(scorer: ConfidenceScorer, stamp: object):+ fused = _fused(_alert())+ fused.enrichments["identity_privilege"] = stamp+ scored = scorer.score(fused)+ return next(f for f in scored.confidence_rationale if f.factor == "identity_privilege")+++def test_absent_privilege_stamp_is_unknown_and_contributes_nothing(scorer: ConfidenceScorer) -> None:+ """No stamp must read as unknown in words, not as a bare 0.00.++ A zero with no explanation is indistinguishable from a measured neutral,+ and the two call for different actions: one is an ordinary account, the+ other is a provider whose posture has never been collected.+ """+ fused = _fused(_alert())+ scored = scorer.score(fused)+ factor = next(f for f in scored.confidence_rationale if f.factor == "identity_privilege")+ assert factor.contribution == 0.0+ assert "unknown" in factor.value+++def test_unknown_tier_is_not_treated_as_standard(scorer: ConfidenceScorer) -> None:+ """`standard` is mildly negative; `unknown` is exactly neutral.++ Collapsing them would make an uncollected provider look like a estate+ full of ordinary users, which is the most reassuring possible rendering+ of a blind spot.+ """+ unknown = _privilege_factor(scorer, {"tier": "unknown", "measured": False})+ standard = _privilege_factor(scorer, {"tier": "standard", "measured": True, "tags_set": []})+ assert unknown.contribution == 0.0+ assert standard.contribution < 0.0+++def test_tier_orders_the_contribution(scorer: ConfidenceScorer) -> None:+ tiers = ["standard", "elevated", "admin", "domain_admin"]+ contributions = [_privilege_factor(scorer, {"tier": t, "measured": True, "tags_set": []}).contribution for t in tiers]+ assert contributions == sorted(contributions)+++def test_tags_add_but_are_capped(scorer: ConfidenceScorer) -> None:+ """Four tags must not swamp the tier, and the contribution stays in [-1, 1]."""+ none_set = _privilege_factor(scorer, {"tier": "admin", "measured": True, "tags_set": []})+ one_set = _privilege_factor(scorer, {"tier": "admin", "measured": True, "tags_set": ["can_escalate"]})+ all_set = _privilege_factor(+ scorer,+ {+ "tier": "admin",+ "measured": True,+ "tags_set": ["can_escalate", "data_access", "third_party", "cross_account"],+ },+ )+ assert none_set.contribution < one_set.contribution < all_set.contribution+ assert all_set.contribution <= 1.0+ # The cap is what stops tags from making the tier irrelevant: an admin+ # with four tags must still outrank a domain admin with none only+ # because of the tags, never because the tier stopped mattering.+ assert all_set.contribution - none_set.contribution <= 0.6 + 1e-9+++def test_tags_dict_is_accepted_when_tags_set_is_missing(scorer: ConfidenceScorer) -> None:+ """The stamp has two shapes and only one of them is fusion's own.++ `tags_set` is written by the fuse-time resolver; `tags` is what the API+ stores and what a replayed or imported alert carries. Reading only the+ first silently scored every replayed alert as tagless.+ """+ factor = _privilege_factor(+ scorer,+ {+ "tier": "admin",+ "measured": True,+ "tags": {"can_escalate": True, "data_access": None, "third_party": False, "cross_account": True},+ },+ )+ baseline = _privilege_factor(scorer, {"tier": "admin", "measured": True, "tags_set": []})+ assert factor.contribution > baseline.contribution+ assert "can_escalate" in factor.value and "cross_account" in factor.value+ # None and False are both "not a reason to escalate" here, and neither+ # may be counted as a set tag.+ assert "third_party" not in factor.value+ assert "data_access" not in factor.value+++def test_partial_snapshot_is_said_out_loud(scorer: ConfidenceScorer) -> None:+ factor = _privilege_factor(+ scorer,+ {"tier": "admin", "measured": True, "tags_set": [], "snapshot_partial": True},+ )+ assert "incomplete" in factor.value+++def test_privilege_weight_is_bounded_like_asset_criticality(scorer: ConfidenceScorer) -> None:+ """Context about the alert must not outweigh evidence that it happened."""+ factor = _privilege_factor(scorer, {"tier": "domain_admin", "measured": True, "tags_set": []})+ assert factor.weight == WEIGHT_IDENTITY_PRIVILEGE+ assert WEIGHT_IDENTITY_PRIVILEGE <= 0.06diff --git a/services/fusion/tests/test_sub_principal_baselines_reproduction.py b/services/fusion/tests/test_sub_principal_baselines_reproduction.pynew file mode 100644index 000000000..9c2e49fd7--- /dev/null+++ b/services/fusion/tests/test_sub_principal_baselines_reproduction.py@@ -0,0 +1,130 @@+"""Depth 2 Phase 1.4 reproduction: only the human has a baseline of its own.++`baselines.py` keeps a rarity and volume baseline per *principal*, plus one+volume baseline per OAuth application. Everything else that acts on a+person's behalf — a personal access token, a service account, an AI agent —+is folded into the human's baseline, which is exactly backwards for the case+these baselines exist to catch: a stolen token acts *as* the user, so the+user's own numbers stay normal while the token's do not.++Four signals the plan names, none of which can be computed today:++* a token or app acting outside its **own** baseline;+* a scheduled token or bot identity suddenly interactive;+* a token used from a new ASN within minutes of its creation;+* a token idle for 60 days or more that reactivates.++Fails on the pre-change tree for those reasons. Kept afterwards as the+guard, including the cross-language fixture: the GitHub join key is produced+by Go and consumed by Python, and the two normalise differently by default.+"""++from __future__ import annotations++import pytest+++def test_sub_principal_kinds_are_declared() -> None:+ from app.services.baselines import SUB_PRINCIPAL_KINDS++ assert {"oauth_app", "api_token", "service_account", "ai_agent"} <= set(SUB_PRINCIPAL_KINDS)+++def test_the_four_signals_are_derived_fields() -> None:+ """A signal absent from DERIVED_FIELDS reaches no rule and no alert.++ `services/fusion/app/services/derived_fields.py` and the detection+ matcher both read that set, so a field computed and not declared is a+ number nothing can clause on.+ """+ from app.services.baselines import DERIVED_FIELDS++ required = {+ "baseline_sub_principal_rarity",+ "baseline_sub_principal_volume_z",+ "baseline_token_cadence_shift",+ "baseline_token_reactivated_after_idle",+ "baseline_token_new_asn_after_creation",+ }+ assert required <= DERIVED_FIELDS, f"missing: {sorted(required - DERIVED_FIELDS)}"+++def test_a_token_is_resolved_as_its_own_subject() -> None:+ """The token, not the human it acts for, is what the baseline is held on."""+ from app.services.baselines import resolve_sub_principal++ event = {+ "connector_type": "github",+ "actor": "alice",+ # What `session.go` wrote. The consumer reads this and never the+ # vendor field it came from, so there is only ever one derivation.+ "activity": {"session": {"key": "AbC123+/xyz=", "key_kind": "vendor_credential"}},+ }+ subject = resolve_sub_principal(event)+ assert subject is not None+ assert subject.kind == "api_token"+ assert subject.id == "AbC123+/xyz="+++def test_the_token_identity_comes_from_the_projection_not_the_vendor_field() -> None:+ """One producer, many consumers — the property ADR-0010 turns on.++ An earlier cut of this resolver read GitHub's `hashed_token` itself and+ pinned its normalisation against the Go side byte for byte. That passes+ and is still the wrong design: it is a *second* derivation of a session+ key, and two derivations agree until the next vendor-schema change and+ then disagree silently, leaving Postgres, the lake and the graph holding+ different answers to "which session was this".+ `scripts/check_session_projection.py` refuses it.++ So the vendor field alone resolves nothing here, and the projection is+ what the subject is keyed on.+ """+ from app.services.baselines import resolve_sub_principal++ mixed = "AbC123+/xyz="++ from_vendor_field = resolve_sub_principal({"connector_type": "github", "hashed_token": mixed})+ assert from_vendor_field is None, "the consumer must not re-derive the session key from the vendor field"++ from_projection = resolve_sub_principal(+ {"connector_type": "github", "activity": {"session": {"key": f" {mixed} ", "key_kind": "vendor_credential"}}}+ )+ assert from_projection is not None+ assert from_projection.id == mixed, "case must survive: the key is a digest, not a name"+++def test_a_vendor_session_id_is_not_a_token_identity() -> None:+ """Keying a token baseline on a session id would merge every token used+ in that session into one subject."""+ from app.services.baselines import resolve_sub_principal++ subject = resolve_sub_principal({"connector_type": "okta", "activity": {"session": {"key": "sess-1", "key_kind": "vendor_session"}}})+ assert subject is None+++ ("hours", "expected"),+ [+ # Even intervals around the clock with low variance: a cron job.+ ([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "scheduled"),+ # Dense bursts inside a working day: a person at a keyboard.+ ([9, 9, 9, 10, 10, 11, 14, 15, 16, 16, 17], "interactive"),+ ],+)+def test_tokens_are_classified_by_cadence(hours: list[int], expected: str) -> None:+ from app.services.baselines import classify_cadence++ assert classify_cadence(hours) == expected+++def test_cadence_is_unknown_below_the_sample_floor() -> None:+ """Three calls do not establish a rhythm.++ Returning `scripted` for a token nobody has watched yet would make the+ "a scheduled token turned interactive" signal fire on the fourth call of+ every new token in the estate.+ """+ from app.services.baselines import classify_cadence++ assert classify_cadence([9, 10, 11]) == "unknown"diff --git a/services/ingest/internal/activity/github_token_key_test.go b/services/ingest/internal/activity/github_token_key_test.gonew file mode 100644index 000000000..001ad35d4--- /dev/null+++ b/services/ingest/internal/activity/github_token_key_test.go@@ -0,0 +1,82 @@+package activity++import (+ "encoding/json"+ "os"+ "path/filepath"+ "testing"+)++// The GitHub token join key is derived in exactly one place, and this is it.+//+// Go writes `hashed_token` into the session projection with+// `strings.TrimSpace` and no case folding, because the value is a digest and+// case is significant. Python's `services/fusion/app/services/baselines.py`+// resolves nearly every other field through a helper that lowercases, which+// is right for a username and wrong for this.+//+// There is deliberately no second derivation to agree with. An earlier+// version of this comment pointed at a cross-language parity test, and that+// test was deleted with the design it pinned: two components deriving one+// session key agree until the next vendor-schema change and then disagree in+// silence. `resolve_sub_principal` now *reads* `activity.session` through+// `_session_credential_key`, and `scripts/check_session_projection.py` is the+// gate that refuses the shortcut back.+//+// So the fixture below guards the single derivation rather than a parity:+// change the normalisation here and every consumer of the projection moves+// with it, which is the property that replaced the parity test.+const tokenKeyFixture = "testdata/github_token_key.json"++type tokenKeyCase struct {+ Name string `json:"name"`+ HashedToken any `json:"hashed_token"`+ Expected string `json:"expected_key"`+ Why string `json:"why"`+}++func loadTokenKeyCases(t *testing.T) []tokenKeyCase {+ t.Helper()+ raw, err := os.ReadFile(filepath.Clean(tokenKeyFixture))+ if err != nil {+ t.Fatalf("reading %s: %v", tokenKeyFixture, err)+ }+ var cases []tokenKeyCase+ if err := json.Unmarshal(raw, &cases); err != nil {+ t.Fatalf("parsing %s: %v", tokenKeyFixture, err)+ }+ if len(cases) == 0 {+ // An empty fixture would make every assertion below vacuous, which+ // is the shape of gate this repository keeps rediscovering.+ t.Fatalf("%s contains no cases", tokenKeyFixture)+ }+ return cases+}++func TestGitHubSessionKeyMatchesTheSharedFixture(t *testing.T) {+ for _, tc := range loadTokenKeyCases(t) {+ t.Run(tc.Name, func(t *testing.T) {+ vendor := map[string]any{"actor": "alice"}+ if tc.HashedToken != nil {+ vendor["hashed_token"] = tc.HashedToken+ }+ got := githubSession(vendor)+ if got.Key != tc.Expected {+ t.Fatalf("key = %q, want %q (%s)", got.Key, tc.Expected, tc.Why)+ }+ if tc.Expected != "" && got.KeySource != "hashed_token" {+ t.Fatalf("key_source = %q, want %q", got.KeySource, "hashed_token")+ }+ })+ }+}++func TestGitHubSessionKeyPreservesCase(t *testing.T) {+ // Stated separately from the fixture so the property survives somebody+ // pruning the fixture: lowercasing a digest produces a different+ // credential, not a tidier spelling of the same one.+ mixed := "AbC123+/xyz="+ if got := githubSession(map[string]any{"hashed_token": mixed}).Key; got != mixed {+ t.Fatalf("key = %q, want %q — case must be preserved", got, mixed)+ }+}diff --git a/services/ingest/internal/activity/testdata/github_token_key.json b/services/ingest/internal/activity/testdata/github_token_key.jsonnew file mode 100644index 000000000..613b40b7a--- /dev/null+++ b/services/ingest/internal/activity/testdata/github_token_key.json@@ -0,0 +1,44 @@+[+ {+ "name": "mixed_case_is_preserved",+ "hashed_token": "AbC123+/xyz=",+ "expected_key": "AbC123+/xyz=",+ "why": "the value is a digest; lowercasing it names a different credential"+ },+ {+ "name": "surrounding_whitespace_is_trimmed",+ "hashed_token": " AbC123+/xyz= ",+ "expected_key": "AbC123+/xyz=",+ "why": "both sides trim, so a padded value must still join"+ },+ {+ "name": "base64_padding_is_significant",+ "hashed_token": "Zm9vYmFy==",+ "expected_key": "Zm9vYmFy==",+ "why": "trailing = is part of the value and must not be stripped"+ },+ {+ "name": "an_empty_token_yields_no_key",+ "hashed_token": "",+ "expected_key": "",+ "why": "an absent credential is not a session key of the empty string"+ },+ {+ "name": "whitespace_only_yields_no_key",+ "hashed_token": " ",+ "expected_key": "",+ "why": "trimming to empty is the same as absent"+ },+ {+ "name": "a_missing_field_yields_no_key",+ "hashed_token": null,+ "expected_key": "",+ "why": "an audit entry with no hashed_token has no credential to group on"+ },+ {+ "name": "a_non_string_yields_no_key",+ "hashed_token": 12345,+ "expected_key": "",+ "why": "both sides require a string; coercing a number would invent a key"+ }+]diff --git a/sse_starlette-3.6.0-py3-none-any.whl b/sse_starlette-3.6.0-py3-none-any.whlnew file mode 100644index 000000000..f48c1dc44Binary files /dev/null and b/sse_starlette-3.6.0-py3-none-any.whl differdiff --git a/starlette-1.10.0-py3-none-any.whl b/starlette-1.10.0-py3-none-any.whlnew file mode 100644index 000000000..911177817Binary files /dev/null and b/starlette-1.10.0-py3-none-any.whl differdiff --git a/tests/isolation/test_postgres_rls.py b/tests/isolation/test_postgres_rls.pyindex fba423cc2..24f233b1d 100644--- a/tests/isolation/test_postgres_rls.py+++ b/tests/isolation/test_postgres_rls.py@@ -136,6 +136,7 @@ def _literal_for(allowed: list[str] | None = None,max_len: int | None = None,exact_len: int | None = None,+ ordinal: int = 0,) -> str:"""A syntactically valid value for a NOT NULL column with no default.@@ -157,6 +158,18 @@ def _literal_for(digest column has and which the short filler cannot satisfy. Taughtto the seeder rather than relaxed in the schema: a constraintloosened so a prober can insert is a constraint that stops being one.+ * ``ordinal`` makes the text filler differ per column, and sort by+ column position. ``_check_vocabularies`` reads single-column CHECKs+ only, so a constraint *relating two columns* was invisible here and+ every text column got the same literal:+ ``identity_links`` declares ``left_key <> right_key`` and+ ``left_key < right_key``, and the identical filler failed both, so the+ table reported unseedable and its isolation assertions would have been+ skipped. Ordering the filler by position satisfies the ordered-pair+ idiom generally. It is a heuristic, and a safe one in the only+ direction that matters: it can turn unseedable into seeded, never a+ failed insert into a silent pass, because the insert itself is the+ evidence and a table that still will not seed is still reported."""if allowed:return f"'{allowed[0]}'"@@ -170,7 +183,9 @@ def _literal_for(if data_type == "uuid":return f"'{uuid.uuid4()}'::uuid"if data_type in ("character varying", "text", "character"):- value = f"p{nonce}"+ # The ordinal leads so that truncation to `max_len` keeps columns+ # distinct and in position order.+ value = f"p{ordinal:02d}{nonce}"if max_len is not None:value = value[:max_len]return f"'{value}'"@@ -306,7 +321,7 @@ async def _seed_row(conn, table: str, tenant: uuid.UUID, enum_labels: dict[str,names = ["tenant_id"]values = [tenant_literal]- for col in cols:+ for ordinal, col in enumerate(cols, start=1):literal = _literal_for(col["data_type"],col["udt_name"],@@ -315,6 +330,7 @@ async def _seed_row(conn, table: str, tenant: uuid.UUID, enum_labels: dict[str,allowed=vocab.get(col["column_name"]),max_len=col["character_maximum_length"],exact_len=exact.get(col["column_name"]),+ ordinal=ordinal,)if literal == "DEFAULT":continuediff --git a/tomli-2.4.1-cp312-cp312-macosx_11_0_arm64.whl b/tomli-2.4.1-cp312-cp312-macosx_11_0_arm64.whlnew file mode 100644index 000000000..46495b0e2Binary files /dev/null and b/tomli-2.4.1-cp312-cp312-macosx_11_0_arm64.whl differdiff --git a/typing_extensions-4.16.0-py3-none-any.whl b/typing_extensions-4.16.0-py3-none-any.whlnew file mode 100644index 000000000..9f9aae6adBinary files /dev/null and b/typing_extensions-4.16.0-py3-none-any.whl differdiff --git a/typing_inspection-0.4.4-py3-none-any.whl b/typing_inspection-0.4.4-py3-none-any.whlnew file mode 100644index 000000000..2a5ded832Binary files /dev/null and b/typing_inspection-0.4.4-py3-none-any.whl differdiff --git a/urllib3-2.8.0-py3-none-any.whl b/urllib3-2.8.0-py3-none-any.whlnew file mode 100644index 000000000..d2da97492Binary files /dev/null and b/urllib3-2.8.0-py3-none-any.whl differdiff --git a/uvicorn-0.54.0-py3-none-any.whl b/uvicorn-0.54.0-py3-none-any.whlnew file mode 100644index 000000000..f347e00a9Binary files /dev/null and b/uvicorn-0.54.0-py3-none-any.whl differdiff --git a/wcmatch-8.5.2-py3-none-any.whl b/wcmatch-8.5.2-py3-none-any.whlnew file mode 100644index 000000000..a297e135fBinary files /dev/null and b/wcmatch-8.5.2-py3-none-any.whl differdiff --git a/wrapt-1.17.3-cp312-cp312-macosx_11_0_arm64.whl b/wrapt-1.17.3-cp312-cp312-macosx_11_0_arm64.whlnew file mode 100644index 000000000..eb5901716Binary files /dev/null and b/wrapt-1.17.3-cp312-cp312-macosx_11_0_arm64.whl differdiff --git a/zipp-4.1.2-py3-none-any.whl b/zipp-4.1.2-py3-none-any.whlnew file mode 100644index 000000000..d31ab315eBinary files /dev/null and b/zipp-4.1.2-py3-none-any.whl differ