Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
Merge PR #1313: depth 7.3 — AitM detected on the session
beenuar4539245fa19f
diff --cc .train_resolve.pyindex a5b90640e,000000000..e52d3aa37mode 100644,000000..100644--- a/.train_resolve.py+++ b/.train_resolve.py@@@ -1,54 -1,0 +1,90 @@@+#!/usr/bin/env python3+"""Train conflict resolver for the recurring tracker shapes (session tool, not shipped).++Usage: python3 .train_resolve.py <file> <mode>+ modes:+ union -> ours + theirs for every hunk+ theirs -> theirs for every hunk+ roadmap -> theirs, deduping repeated 'claim-to-gate matrix stands at' paragraphs+Run from the worktree root. Refuses to leave markers behind.+"""+import re+import sys+import pathlib++CONFLICT = re.compile(r"<<<<<<< [^\n]*\n(.*?)(?:\|\|\|\|\|\|\| [^\n]*\n.*?)?=======\n(.*?)>>>>>>> [^\n]*\n", re.DOTALL)++++def merge_checkboxes(ours: str, theirs: str) -> str:++ """Same line set modulo [x]/[ ]: checked wins per line. Falls back to union."""++ ol, tl = ours.splitlines(True), theirs.splitlines(True)++ norm = lambda s: s.replace("- [x]", "- [ ]", 1)++ if len(ol) == len(tl) and all(norm(a) == norm(b) for a, b in zip(ol, tl)):++ return "".join(a if "- [x]" in a else b for a, b in zip(ol, tl))++ return ours + theirs++++++def merge_matrix(ours: str, theirs: str) -> str:++ """Union of table rows from both sides, then a single trailing non-row block++ (theirs'), for hunks spanning appended matrix rows + the Summary."""++ def split(block: str):++ rows, rest, in_rows = [], [], True++ for line in block.splitlines(True):++ if in_rows and line.startswith("|"):++ rows.append(line)++ else:++ in_rows = False++ rest.append(line)++ return rows, "".join(rest)++++ orows, orest = split(ours)++ trows, trest = split(theirs)++ seen, rows = set(), []++ for r in orows + trows:++ if r not in seen:++ seen.add(r)++ rows.append(r)++ return "".join(rows) + (trest if trest.strip() else orest)+++++def dedupe_matrix_paragraph(text: str) -> str:+ paras = [p for p in text.split("\n\n") if p.strip()]+ firsts = [p for p in paras if p.lstrip().startswith("The claim-to-gate matrix stands at")]+ if len(firsts) > 1:+ return firsts[0] + "\n\n"+ return text+++def main() -> None:+ path, mode = sys.argv[1], sys.argv[2]+ p = pathlib.Path(path)+ t = p.read_text()+ out, idx, n = [], 0, 0+ for m in CONFLICT.finditer(t):+ ours, theirs = m.group(1), m.group(2)+ if mode == "union":+ r = ours + theirs+ elif mode == "theirs":+ r = theirs+ elif mode == "roadmap":+ r = dedupe_matrix_paragraph(theirs)++ elif mode == "checkboxes":++ r = merge_checkboxes(ours, theirs)++ elif mode == "matrix":++ r = merge_matrix(ours, theirs)+ else:+ raise SystemExit(f"unknown mode {mode}")+ out.append(t[idx : m.start()])+ out.append(r)+ idx = m.end()+ n += 1+ out.append(t[idx:])+ res = "".join(out)+ if "<<<<<<<" in res or ">>>>>>>" in res:+ raise SystemExit(f"{path}: markers left behind")+ p.write_text(res)+ print(f"{path}: {n} hunks -> {mode}")+++if __name__ == "__main__":+ main()diff --cc CHANGELOG.mdindex 5c54eea96,5e6f80d11..eaf36de2a--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -231,141 -231,39 +231,174 @@@ and this project adheres to [Semantic V### Added+- **Endpoint discovery of the AI estate, with no new agent** (depth plan 6.2).+ `packs/aisoc-ai-estate.yaml` finds coding agents and MCP server processes,+ the configuration files the popular MCP clients read, AI IDE extensions and+ model-provider credential files, through the osquery agent a fleet already+ runs. `apps/docs/docs/concepts/ai-estate-discovery.md` records what each+ source can **and cannot** see, which is the half that makes an empty column+ readable: a device missing from a provider-audit row is not a finding,+ because no provider audit log has ever carried one.++ The finding that made this a fix as well as a feature: the endpoint branch+ of the recorder read `process.name` off the OCSF event, and the canonical+ OCSF profile carries `title`, `external_id` and the identity aliases and+ nothing else. `fleetdm`, `osctrl`, `auditd` and `windows_event` all put+ their process fields in the connector payload under `raw_data`, so that+ branch found `None` on every endpoint source this product has and could+ never have produced one row — while its own tests passed, because they+ built events in the shape the module expected. It now reconstructs the same+ flat namespace `DetectionEngine._raw_fields` does, and+ `scripts/check_ai_endpoint_discovery.py` makes reading it a rule.++ Recorded rather than implied: `crowdstrike`, `sentinelone` and+ `azure_defender` poll the vendor's own **detections**, not raw process+ telemetry, so they cannot supply a process start. An operator running an+ EDR and nothing else has a coverage gap here, not a quiet estate, and the+ documentation says so in a callout.++- **An AI-estate inventory built from observation rather than registration**+ (depth plan 6.1). `aisoc_ai_assets` (migration `110`, tenant-scoped with+ forced row-level security) holds every AI app, coding agent, MCP server,+ IDE extension, model-provider key and OAuth grant AiSOC has seen, with the+ person and device behind each. The gap it closes is that `aisoc_mcp_servers`+ records what an operator *declared*, and the row that matters in an AI+ governance review is the one present in observation and absent from the+ declaration.++ Measured the way the recurring defect here demands: the writer is in the+ fusion consumer's raw-events branch, the one path every normalized event+ already travels on the default profile, because a recorder behind a flag or+ in a service CORE does not start is a table nothing writes and a page that+ renders "no AI estate" to a tenant that has one.+ `scripts/check_ai_asset_inventory.py` makes that a rule rather than a+ comment — it requires the consumer to call the recorder *and* the service+ to construct it, since an unconstructed recorder is a `None` every call+ site skips in silence. `ai-estate-live.yml` writes a fixture estate of+ three kinds into a real PostgreSQL as the DML-only `aisoc_app` role, and+ two negative controls re-inject real defects: reverting `NULLS NOT+ DISTINCT` on the identity constraint turns one provider-key row into+ three, and a tenant policy of `USING (true)` lets one tenant read+ another's estate.++ Also: graph schema `v1.2 → v1.3` adds the `AIAsset` label with+ `(:User)-[:OPERATES]->(:AIAsset)` and `(:Endpoint)-[:HOSTS]->(:AIAsset)`,+ emitted by `services/ingest` as a post-pass rather than a connector case —+ AI telemetry arrives on three connector types and as plain process+ activity, so keying on the connector would have covered one of them and+ looked complete. A new `ai_estate:read` / `ai_estate:write` permission+ pair, held by real roles and mintable into an API key, because the+ inventory names which agent on which laptop holds which provider key and+ `hunts:read` already demonstrated what a permission no role holds is worth.++- **A reported message's headers, URLs and attachments are analysed once and+ cited where an auditor looks (depth 7.2).** Nothing in the tree read an+ `Authentication-Results` header before this — `grep -rl+ "authentication-results" services/` returned no match — so a message whose+ SPF and DMARC had both failed at the receiving boundary produced a verdict+ that had never looked. The analysis is three components in one pass, and+ the rule it turns on is that **an unanalysed artefact must never read as a+ clean one**, which is the same rule the submit path already applied to+ attachments and which `services/api/app/services/phishing/headers.py` now+ applies to authentication. SPF, DKIM and DMARC each carry three values, and+ the third is the point: `not_evaluated` is published rather than folded+ into either of the others, and `neutral`, `none`, `permerror` and+ `temperror` are **not** passes — treating them as passes is how a domain+ with a broken SPF record reads as authenticated. A URL on a deployment with+ no provider configured says so per URL rather than vanishing from the+ output, and that specific defect was found by a negative control that at+ first failed to fail, because the tests built their own result list instead+ of driving the producer. `analysed` and `unanalysed` are counted separately+ and never summed — in the dataclass, the response model and migration+ `111`'s column comment — because the sum is the number this product must+ not publish: five artefacts nothing examined, rendered as "5 artefacts, 0+ findings", says the opposite of what happened. The verdict is only ever+ raised, and SPF **and** DMARC must both fail to raise it, because a+ forwarded message routinely breaks SPF and flagging that would alert on+ every mailing list in the estate. Citations go into the investigation+ ledger as `evidence_cited` rows, one per component, written even when a+ component found nothing — a ledger with no row for the URL check cannot be+ told from one where the check passed. The ledger suite has no teardown by+ design: deleting a run cascades to its events, migration `008`'s trigger+ refuses, and defeating an append-only control to tidy a test fixture would+ be the wrong fix.+- **Reported phish now goes from intake to purge (depth 7.1).** The gap was+ not that phishing was unhandled — `POST /phishing/submit` has triaged+ artefacts since migration 015 — but that nothing in the product could+ *remove a message*. The live-action registry held 79 builtin arms and not+ one could touch a mailbox, so governed dispatch answered+ `executor_not_found` for `purge_email`: a response that could block the+ sender's domain at the firewall and could not take back the mail. Five+ verbs enter both halves of the capability vocabulary (`search_mailboxes`,+ `purge_email`, `restore_email`, `block_sender`, `unblock_sender`) with ten+ vendor arms over Microsoft Graph and the Gmail API. Only what the vendors+ document: purge is a move to `recoverableitemsdeletions` or to Trash, never+ a hard delete, and `restore_email` is a real reverse rather than a declared+ one. There is deliberately **no tenant-wide blocked-sender verb**, because+ neither vendor exposes its organisation-level block list through these+ APIs; the sender block is per mailbox and says so in the summary an analyst+ reads, not only in a docstring. Reported-message intake normalises three+ sources into one record — an M365 report mailbox over Graph, Workspace user+ reports, and the existing `email_inbox` connector — and the hard case is+ the common one: a user who reports by forwarding becomes the *sender*, so+ the adapter reads the original sender and recipients out of the preserved+ headers and leaves the sender empty rather than naming the victim as the+ attacker. Campaign grouping is a five-step ladder and `grouping_basis`+ records which step matched, because four are facts about the message and+ subject similarity is a judgement. The purge is **one proposal and one+ dispatch per mailbox**: handing a single executor twenty-five addresses+ would be one approval covering a list the approver never read.+ `scaled_impact` turns the recipient count into the tier — MODERATE at one,+ HIGH at five, SEVERE at twenty-five, where `NEVER_AUTONOMOUS` makes it+ mandatory-human at any confidence and any autonomy tier — and the proposal+ and every dispatch under it read the same function, so the approver and the+ executor cannot see different actions. Verification re-queries the mailbox,+ and that probe is the one in the service whose confirming answer is an+ *absence*. Everything ships **off**: `AISOC_FEATURE_MAILBOX_REMEDIATION`+ and `PHISHING_INTAKE_WORKER_ENABLED` both default false, and a disabled+ deployment answers FAILED naming the setting rather than simulating a purge+ — `NotifySlackExecutor` reporting `message_sent: true` for a message it+ never sent is the precedent. Migration `110`. Gates:+ `check_action_contract.py` (86 → 97 executors), `check_module_reachability.py`+ (981 → 986 reachable), `check_vendor_catalog_ids.py`,+ `test_mailbox_remediation.py` (60 cases through real `respx` HTTP) and+ `test_phishing_campaign_purge_live.py` (17 cases against a real Postgres+ with the full chain). **No Microsoft 365 or Google Workspace tenant was+ exercised**: every vendor claim rests on the documented request shape and a+ mock.+ - **An adversary-in-the-middle detection family, on sessions (depth 7.3).**+ The attack is invisible to every rule the corpus had, and the reason is+ structural rather than a coverage gap: an AiTM proxy relays the victim's+ credentials *and* their MFA response, so the sign-in is legitimate by every+ check the identity provider makes. There is no failed authentication to+ count and often no impossible travel to measure, because the proxy sits in+ the victim's own country. What gives it away is the stolen session being+ used from somewhere it was never established — so the entity these rules+ group by is the **session**, not the user. That is why the connector work+ had to land first: `OktaConnector.normalize()` returned `None` for+ `session_id`, `src_asn`, `user_agent`, `client_browser` and+ `mfa_satisfied` on a System Log row carrying all five, because the engine+ flattens `raw_event` exactly one level and `authenticationContext` arrives+ as a dict — the same shape as the `outcome` hoist in #1279. Four rules:+ three distinct-value rules at threshold two (a second network, a second+ client family, a second country for one session) and one ordered sequence+ at `critical` — an MFA-satisfied sign-in, then the same session from+ another network within half an hour, which a threshold cannot express.+ `mfa_satisfied` is three-valued, because treating "this event says nothing+ about MFA" as `False` would fire on every directory read, and a denied push+ is an MFA event and emphatically not a satisfied one. On Entra,+ `correlationId` is deliberately **not** a fallback for `sessionId`: one+ identifies a request and the other a session, and substituting would make+ every sign-in its own session so the reuse detection could never fire while+ looking like it worked. The clause "this event carries a session" is spelled+ `session_id_neq: null`, using an operator that exists — inventing `_exists`+ would have been read as a field literally named `session_id_exists`, which+ is the `approver_role_neq` defect one release later. Fixtures are+ hand-authored from an Okta System Log row and a Microsoft Graph `signIn`+ resource rather than synthesised from the rules, and the negative twin+ inverts only the clause each rule is about — same session, same user, same+ event type, same outcome, same network. Windowed rules 72 → 76.+- `services/api/app/models/detection_rule.py` maps the `author` column, whichhas existed since `001_init.sql` and was mapped by nothing — while fusion'stenant overlay reads it on every reload to say *who* silenced a rule when itdiff --cc DEPTH_PROGRESS.mdindex e1a5b812a,cfe6ac465..982e0e4e6--- a/DEPTH_PROGRESS.md+++ b/DEPTH_PROGRESS.md@@@ -862,9 -622,9 +862,9 @@@ endpoint produces once triage falls thr## Phase 7: Phishing operations-- [ ] **7.1** Intake to purge (gap-closure Phase 10 / parity 6.4)-- [ ] **7.2** Header, URL and attachment analysis+- [x] **7.1** Intake to purge (gap-closure Phase 10 / parity 6.4)+- [x] **7.2** Header, URL and attachment analysis- - [ ] **7.3** Adversary-in-the-middle session reuse+ - [x] **7.3** Adversary-in-the-middle session reuse## Phase 8: Enterprise gates@@@ -1006,29 -732,51 +1006,77 @@@| 2026-10-11 | 8.1 | axe extended to the case list and case workspace, the two of parity 4.7's five views that were left out. Four real defects found and fixed: two `<select>` controls with no accessible name (one of them the control that writes a case's status), two indistinguishable `<aside>` landmarks, and panel headings at `h3` directly under the page's `h1`. |+| 2026-10-11 | 7.1 | Reproduced: against the unmodified tree, `registry.list_vendors_for_capability` returned `[]` for all five mailbox verbs and `dispatcher.dispatch` answered `executor_not_found` for `purge_email` over 79 registered builtin arms -- a phishing response that could block the sender's domain at the firewall and could not remove the message it arrived in. Added the five verbs to both halves of the vocabulary, two mail clients (Graph, Gmail with per-mailbox domain-wide delegation), ten vendor arms, migration `104`, three intake adapters, campaign grouping, the governed fan-out and a report-mailbox worker. Only documented vendor APIs: purge is a soft delete on both and `restore_email` is a real reverse; there is no tenant-wide sender block because neither vendor exposes one through these APIs, and the per-mailbox scope is in the summary an analyst reads rather than only in a docstring. Everything ships off. Negative controls recorded in the PR: defaulting the feature flag on reds four tests, inverting the verification probe's polarity reds two, reading the envelope sender instead of the preserved headers reds four adapter tests including the one asserting the report mailbox is not proposed for purge, incrementing `recipient_count` instead of recomputing it reds the grouping test, and dropping `recipient_count` from the per-mailbox dispatch reds the fan-out test -- which is the quiet way a SEVERE action becomes a MODERATE one. Gates: `check_action_contract` 86 -> 97 executors, `check_module_reachability` 981 -> 986 of 1014, `check_route_authz` 262 -> 265 authorizing with both ceilings unmoved, matrix 302 -> 303 rows. Actions suite 1158 -> 1228; api 44 new cases, 17 of them against a real Postgres with the full chain. Recorded as **PARTIAL**, not GATED: no Microsoft 365 or Google Workspace tenant was exercised (D-7.1). |+| 2026-10-11 | 7.2 | Reproduced: `grep -rl "authentication-results" services/` returned no match on the unmodified tree, so nothing in the product read a sender-authentication header and a message whose SPF and DMARC had both failed produced a verdict that had never looked. Built header analysis with a three-valued result, URL submission through the existing sandbox contract, the attachment hash lookup the submit path already had, and `evidence_cited` citations into the investigation ledger. Migration `111` stores the headers on the submission so a retriage replays them rather than re-fetching a message the purge has usually removed. `analysed` and `unanalysed` are counted separately in three places and summed in none. Negative controls: treating `none`/`neutral` as passes reds two tests; silently skipping an unanalysable URL reds three. **That second control is the finding worth keeping** -- it failed to fail on the first attempt, because every test built its own `ArtefactResult` list and never ran the producer, so five cases were added that drive `analyse_message` with the registry patched empty. A second finding came from the ledger suite's teardown: deleting a run cascades to its events and migration 008's trigger refuses, so the fixture was dropped rather than the append-only control defeated. Gates: matrix 303 -> 304 rows (300 GATED), `check_text_bind_casts` and `check_raw_sql_columns` clean over 6 new columns, `check_route_authz` unmoved. 45 new api cases, 6 of them against a real Postgres with the full chain. |++### D-7.1 -- No Microsoft 365 or Google Workspace tenant was exercised++Every mailbox verb in 7.1 is built against the vendors' documented request+shapes and asserted with a recorded `respx` mock: the Graph `$filter` on a+bracketed `internetMessageId`, the `move` to `recoverableitemsdeletions`, the+inbox `messageRules` body, Gmail's `rfc822msgid:` operator, `trash`/`untrash`+and the settings filter. **Nobody has watched either vendor accept any of+them.** The maintainer-only list already carries "provide cloud and SaaS test+tenants"; this is the same block, named here so the claim row's PARTIAL has a+reason a reader can find.++Two specific consequences, stated rather than implied. A decoder and a fixture+that share an author share a misreading, so a field Graph spells differently+from the reference would pass every test here. And the Gmail filter body and+the Graph inbox-rule body were written from the API reference rather than+taken from a working call, which is the half of `block_sender` least likely+to be right first time.++The claim row returns to GATED when a live tenant is available. Nothing was+weakened to accommodate this: the row says PARTIAL, the operations page says+"treat the first run against your own tenant as the first run", and the+CHANGELOG entry says it too.+| 2026-10-11 | 2.5 | Reproduced: `test_posture_snapshot_coverage.py` on the unmodified tree asserted the *gap* — every resolver in `effective_permissions/` was pure and the only production snapshot source returned `{}`, so every live call 412'd and the subsystem had never answered a question about a real estate. Five collectors now exist under `services/connectors/app/connectors/posture/`, reached by a `__posture_snapshot__` sentinel through each connector's `get_resource_config`, and the coverage test is inverted: it now fails when a registered resolver has no declared collector. Building it found three things worth recording. (a) `BaseConnector.get_resource_config` declares `at_ts: str | None = None` while AWS, Okta and GitHub each required a positional `ts`, so a caller written against the base class raised `TypeError` on three of the connectors — standardised, with `test_every_implementation_matches_the_base_signature` holding it. (b) Google Workspace's directory read needs scopes the audit-log credential is not consented for, so the JWT is now built per scope: asking for both in one assertion would have broken alert polling on every tenant that had not pre-consented the new ones. (c) A snapshot is only an answer if you know *when* it was true — an alert is triaged minutes to days after the event, and a single mutable "current posture" row answers "what can this identity do now" while appearing to answer "what could it do when it acted", which is the worse failure because the number looks right. Migration 103 therefore versions snapshots on `valid_from`/`valid_until` (`valid_to` in the plan; `valid_from`/`valid_until` is the existing spelling in migration 018 and two spellings of one concept in one schema is worse than a recorded deviation — D11). Negative controls, both run against live Postgres and both restored: dropping `posture_snapshots_live_idx` leaves `test_a_second_live_row_is_refused_by_the_database` reporting `DID NOT RAISE IntegrityError`, and defeating the content-hash comparison in `save_snapshot` fails `test_unchanged_recollection_does_not_open_a_version` with two distinct row ids — without that comparison an unchanged estate re-collected hourly writes a version an hour and "when did this posture last change" becomes unanswerable because every row is a change. Evidence: 12 posture-store tests against live Postgres with the migration chain applied, 65 across the posture and effective-permission suites, 1,145 connector tests, 4,774 API tests. Eleven API failures are pre-existing on `main` and unrelated — recorded as D12. |+ | 2026-10-11 | 7.3 | Reproduced: `OktaConnector.normalize()` returned `None` for `session_id`, `src_asn`, `user_agent`, `client_browser` and `mfa_satisfied` on a System Log row carrying all five, so the family had nothing to group by and could not have been written. Same shape as the `outcome` hoist in #1279 -- the engine flattens `raw_event` one level and `authenticationContext` arrives as a dict. Hoisted the session, network and client fields on Okta and Entra, then wrote four rules grouping by **session** rather than by user: three distinct-value rules at threshold 2 and one ordered sequence at critical. Fixtures hand-authored from an Okta System Log row and a Graph `signIn` resource; the negative twin inverts only the clause each rule is about. Windowed 72 -> 76; executable detections unchanged at 2,511, because a windowed rule is not in that corpus. Negative controls: removing the `session_id` hoist reds 4 connector tests; switching one rule's `group_by` to `actor_email` reds 2 engine tests. Two findings recorded as D-7.3a (the threshold engine windows by arrival and only sequences window by event time) and D-7.3b (the reported-message link is deferred with a reason). |++ ### D-7.3a -- The threshold engine windows by arrival, not by event time++ Found by a test that failed. An earlier version of+ `test_a_replay_outside_the_sequence_window_does_not_fire` fed two events whose+ `event_time` fields were two hours apart and expected the threshold rules to+ stay quiet. They fired, because `_evaluate_thresholds` scores each Redis+ member with `time.time()` while `_evaluate_sequences` reads+ `parse_event_time`. Both events arrived in the same second, so the window+ never moved.++ This is recorded rather than patched. Arrival-time scoring is what makes a+ threshold cheap, and a connector polling a vendor every five minutes delivers+ a burst whose arrival order is the only clock fusion has for it. The+ consequence is specific and worth knowing: **the one-hour figure on the three+ `wd-aitm-*` threshold rules is a budget on ingestion, not on the vendor's+ timestamps.** A replay that happened an hour ago and is polled now lands in+ the same window as the sign-in. For this family that direction is the safe+ one -- it widens rather than narrows -- but a future rule whose correctness+ depends on the vendor's clock needs a sequence, not a threshold.++ ### D-7.3b -- The reported-message half of 7.3 is deferred, with the reason++ The plan's 7.3 asks for the session-reuse family "plus the link from a+ reported message whose URL precedes that sign-in". The session half is+ implemented and gated. The link is not, and inventing it would have been+ worse than deferring it.++ The obstacle is not effort, it is that nothing can feed it today. The+ reported-message URLs live in `aisoc_phishing_submissions` in `services/api`+ (depth 7.1), and the windowed engine consumes Kafka events in+ `services/fusion`; fusion has no read of the API's tables and should not grow+ one for a detection. The alternative -- staging a *gateway's* phishing+ verdict instead of AiSOC's own reported-message corpus -- was examined and+ refused: the one secure-email-gateway connector in the tree, `proofpoint`,+ fetches `/v2/siem/messages/blocked` only, so it never sees a click, and its+ `normalize()` emits neither the recipient nor the URL. Hoisting fields for an+ event the connector does not fetch is a rule that cannot fire wearing a rule+ that can.++ What would close it, in the order it has to happen: `proofpoint` (or a+ `netskope` / `zscaler` web arm) fetching click events and emitting+ `recipient`, `url` and the gateway's classification; then a sequence rule+ staging that click and the MFA-satisfied sign-in by the same principal. The+ AiSOC-authored half -- joining a tenant's *own* reported-message URLs to a+ sign-in -- belongs in the API beside the campaign store rather than in+ fusion, as an enrichment on the alert the session rules raise.diff --cc ROADMAP.mdindex 16634b2af,9144165fc..559eb1829--- a/ROADMAP.md+++ b/ROADMAP.md@@@ -45,8 -45,9 +45,8 @@@ A proof-first, security-first program t- **Phase D (breadth):** D1 eight connectors (QRadar/Exabeam/Securonix/Devo/Netskope/Windows-Sysmon/Zeek-Suricata/syslog-CEF) · D2 AI/LLM-usage audit connector + `llm-*` detections + hot/cold lake tiering · D3 live-vendor mock-server smoke.- **Phase E (prove it):** E1 CI-gated benchmark scoreboard tied to a deterministic live-agent MITRE-accuracy run.- The claim-to-gate matrix stands at **304 rows — 301 GATED / 3 PARTIAL / 0 NO GATE**. Four rows are PARTIAL. "A reported phishing message can be grouped into a campaign and purged across its recipients" is PARTIAL because no Microsoft 365 or Google Workspace tenant is reachable from CI or from this maintainer, so every vendor claim in depth 7.1 rests on the documented request shape and a recorded mock; it returns to GATED when a live tenant is available. "The retention window a tenant chooses is the one the lake honours" stays PARTIAL until a live ClickHouse test watches a row survive past the old 90-day boundary. "Cloud-native collection" stays PARTIAL until a capture recorded from a live Event Hub replaces the fixture synthesised from the Avro specification. And "SSO completes a sign-in": fix pass 4.1 drove both handlers end to end against real Postgres and real signed assertions, and it stays PARTIAL until the containerised-IdP leg of `sso-live.yml` has been observed to pass, because the identity providers in that suite are trust material this repository authors. Every other product claim is backed by a failing test, none is a named deferral either, and the ratchet (`MAX_NO_GATE=0`) forbids any regression. Every PARTIAL row was closed by building the gate it named, never by relabelling. The last two rested on the live-agent eval workflow, which this file said had simply never run: the real reason was that its live path imported a class that exists nowhere in `services/agents`, so its first dispatch failed in 92 seconds and every later one would have. With that fixed the workflow measures, and the groundedness floor is **0.40 over a deterministic 10-incident slice**, derived from ten runs across two environments rather than chosen — seven local runs returning 0.5561 to four decimal places, three GitHub-runner runs returning 0.5821, 0.5329 and 0.5933. Every one is a locally-served `qwen2.5:0.5b`; no hosted provider has been exercised and the floor describes none. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).+ The claim-to-gate matrix stands at **303 rows — 301 GATED / 2 PARTIAL / 0 NO GATE**. Two rows are PARTIAL. "The retention window a tenant chooses is the one the lake honours" stays PARTIAL until a live ClickHouse test watches a row survive past the old 90-day boundary. "Cloud-native collection" stays PARTIAL until a capture recorded from a live Event Hub replaces the fixture synthesised from the Avro specification. "SSO completes a sign-in" was the third, and depth 8.1 closed it by building the gate it named rather than by relabelling: `sso-live.yml` now stands up a containerised Keycloak and both legs have been observed to pass — `keycloak-oidc` takes a real `id_token` through `_verify_id_token` against the provider's published JWKS, and `keycloak-saml` walks the whole SP-initiated flow to the real ACS. Every other product claim is backed by a failing test, none is a named deferral either, and the ratchet (`MAX_NO_GATE=0`) forbids any regression. Every PARTIAL row was closed by building the gate it named, never by relabelling. The last two rested on the live-agent eval workflow, which this file said had simply never run: the real reason was that its live path imported a class that exists nowhere in `services/agents`, so its first dispatch failed in 92 seconds and every later one would have. With that fixed the workflow measures, and the groundedness floor is **0.40 over a deterministic 10-incident slice**, derived from ten runs across two environments rather than chosen — seven local runs returning 0.5561 to four decimal places, three GitHub-runner runs returning 0.5821, 0.5329 and 0.5933. Every one is a locally-served `qwen2.5:0.5b`; no hosted provider has been exercised and the floor describes none. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).-The claim-to-gate matrix stands at **303 rows — 301 GATED / 2 PARTIAL / 0 NO GATE**. Three rows are PARTIAL. "The retention window a tenant chooses is the one the lake honours" stays PARTIAL until a live ClickHouse test watches a row survive past the old 90-day boundary. "Cloud-native collection" stays PARTIAL until a capture recorded from a live Event Hub replaces the fixture synthesised from the Avro specification. And "SSO completes a sign-in": fix pass 4.1 drove both handlers end to end against real Postgres and real signed assertions, and it stays PARTIAL until the containerised-IdP leg of `sso-live.yml` has been observed to pass, because the identity providers in that suite are trust material this repository authors. Every other product claim is backed by a failing test, none is a named deferral either, and the ratchet (`MAX_NO_GATE=0`) forbids any regression. Every PARTIAL row was closed by building the gate it named, never by relabelling. The last two rested on the live-agent eval workflow, which this file said had simply never run: the real reason was that its live path imported a class that exists nowhere in `services/agents`, so its first dispatch failed in 92 seconds and every later one would have. With that fixed the workflow measures, and the groundedness floor is **0.40 over a deterministic 10-incident slice**, derived from ten runs across two environments rather than chosen — seven local runs returning 0.5561 to four decimal places, three GitHub-runner runs returning 0.5821, 0.5329 and 0.5933. Every one is a locally-served `qwen2.5:0.5b`; no hosted provider has been exercised and the floor describes none. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).## v4.0 — Shippeddiff --cc docs/audit/CLAIM_TO_GATE_MATRIX.mdindex 81855fa28,62ed6667b..1dc6b6ce1--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md@@@ -313,8 -310,7 +313,9 @@@ Statuses: `GATED` (a CI job fails when| A stored report schedule produces a report, and says whether it was delivered | `apps/docs/docs/operations/notifications.md` (scheduled reports) | `ci.yml :: Python — Tests` (api, `test_delivery_5_2.py::TestCronMatching` — nine match cases, six unparseable expressions that raise rather than matching everything, a schedule falling between two five-minute polls, an already-run schedule not firing twice, and a week-long outage collapsing into one report) | core | GATED | `report_templates.cron_schedule` was storable, editable and **read by nothing**, so a tenant could set a schedule, see it saved and never receive anything. Asking "does the current minute match" would have checked 08:57 and 09:02 and never 09:00; the matcher asks over the interval since the last run. Migration 098 adds `delivery_status`, because `delivered_at` alone cannot tell "no relay configured" from "the relay refused" from "not run yet" || An outbound event webhook is signed, retried, and its failures are visible | `apps/docs/docs/operations/notifications.md` (outbound webhooks) | `ci.yml :: Python — Tests` (api, `test_delivery_5_2.py::TestWebhookSignature` and `::TestWebhookDelivery` — signature round-trip through the real client, a modified body and a re-stamped timestamp both refused, five malformed headers refused without raising, the 4xx/5xx split, the ladder ending in the dead-letter view, byte-identical bodies across attempts, and an undecryptable secret killing the delivery rather than sending it unsigned) | core | GATED | AiSOC could be told things and could tell nobody anything: `grep -rln outbound_webhook` over `services/` and `apps/` returned nothing. The signature is the shape Stripe, GitHub and Slack use, because a scheme nobody recognises is a scheme nobody verifies. The retry queue and the dead-letter list are one table at different points on one ladder, so there is no window where a delivery is in neither || The fix for `fake_tool_output` is measured on payloads authored after it was frozen, and that measurement carries no floor | `services/agents/tests/adversarial/behavioural_holdout.py`; `apps/docs/docs/benchmark.md` | `ci.yml :: Python — Tests` (`tests/adversarial/test_behavioural_holdout.py` pins the held-out flip, catch and unsafe-action rates **exactly**, in both directions, so a change moving any of them either way has to say so in the same commit; every payload is asserted to carry a steering phrase or an action trigger so the denominator cannot be padded with strings that were never going to work, asserted disjoint from the tuned corpus, and every declared seam asserted to have a payload) + `test_tool_result_channel.py` (the channel invariant the guard rule rests on, walked out of the service's syntax tree, plus a 20-entry good corpus and a negative control that lifts the rule from the table and watches the family reopen) | core | GATED | **Flip 85% and catch 10% on 20 held-out payloads, against 0% and 100% on the 3 tuned ones.** Published rather than closed, because tuning until they agree is the move this repository has measured the cost of. A worst-case probe rather than a neutral sample: the tuned payloads predate the fix and the held-out ones were aimed at named properties of it, so part of the gap is adversarial-versus-naive. The sharper finding is that the guard and the ledger-provenance demotion **share a detector**, so they are one control counted twice rather than two layers, which a test asserts. No floor and no ratchet, deliberately: a target on a held-out set is an instruction to tune against it, and a test fails if the two rates ever agree so that agreement cannot pass silently |+| A reported phishing message can be grouped into a campaign and purged across its recipients, at the tier the recipient count earns | `apps/docs/docs/operations/mailbox-remediation.md`; depth plan 7.1 | `ci.yml :: Python — Lint & Type-check` (`check_action_contract.py` — the five mailbox verbs are in both halves of the vocabulary, every one has an executor, `purge_email` declares `restore_email` as a platform reverse that exists, and the probe it declares is reachable through an `ActionType`; `check_module_reachability.py` — the intake adapters have a production caller, which is the report-mailbox worker rather than only the route) + `ci.yml :: Python — Tests` (actions, `test_mailbox_remediation.py` — 60 cases driving the real arms over `respx`: the Graph filter is asserted to carry the **bracketed** message id and to request `parentFolderId`, because a mock that answers whatever is asked cannot notice a field the query never requested; a hard `DELETE` route is wired to raise, so a purge that stopped being a soft delete fails rather than passes; a dry run is asserted to make **zero** HTTP calls; a mailbox with no copy answers FAILED with `match_count` 0 rather than counting toward a purge total; the rollback is asserted by call count and destination folder rather than by a boolean; and the verification probe's polarity has its own inverted-control test, because for a purge the confirming answer is an absence) + `ci.yml :: Python — Tests` (api, `test_phishing_intake_adapters.py` — 27 cases over the documented Graph, Gmail and `email_inbox` shapes, including the forwarded report where the reporter is the envelope sender) + `integration.yml` (api, `test_phishing_campaign_purge_live.py` — 17 cases against a real Postgres with the migration chain applied: two reports of one message join one campaign, `recipient_count` is recomputed rather than incremented, a 25-recipient campaign is `severe`/`mandatory_human` at every tier and every confidence, each mailbox gets its own dispatch carrying the campaign's count, and twenty of twenty-five purged is `partially_purged`) | core | PARTIAL (no Microsoft 365 or Google Workspace tenant is reachable from CI or from this maintainer, so every vendor claim rests on the documented request shape and a recorded mock. Two consequences stated rather than implied: nobody has observed Graph accept the `move` to `recoverableitemsdeletions` or Gmail accept the `trash`, and the Gmail filter and Graph inbox-rule bodies were written from the API reference rather than taken from a working call. Returns to GATED when a live tenant is available) | a maintainer-supplied M365 and Workspace test tenant |+| A reported message's sender authentication, URLs and attachments are analysed, and an unanalysed artefact never reads as a clean one | `apps/docs/docs/operations/mailbox-remediation.md`; depth plan 7.2 | `ci.yml :: Python — Tests` (api, `test_phishing_message_analysis.py` — 39 cases: every RFC 8601 non-verdict token (`none`, `neutral`, `policy`, `permerror`, `temperror`) and an absent header all resolve to `not_evaluated` rather than to a pass, each producing its own indicator; every method gets an entry even when no clause mentions it, so a console cannot render `SPF fail` and silently omit DMARC; the ARC-sealed header is read when the plain one is absent, which is the forwarded-report case; the verdict merge raises on SPF **and** DMARC failing and not on either alone; and a socket-patched test asserts header analysis reaches no network. Five of the cases drive `analyse_message` itself with the provider registry patched empty, because the hand-built-fixture version of this suite stayed green when the 'no provider configured' branch was deleted — a test that asserts on a fixture it wrote cannot notice a producer that stopped producing) + `integration.yml` (api, `test_phishing_analysis_ledger_live.py` — 6 cases against a real Postgres: one run and one `evidence_cited` row per component with `seq` dense from one, the unevaluated list carried in the payload, an unanalysed URL cited as unanalysed with 'not a clean verdict' in the summary, the rows immutable against migration 008's real trigger, and an unwritable ledger returning `None` rather than raising) | core | GATED | — |+ | An adversary-in-the-middle session replay is detected on the session, not on the user | `services/fusion/app/data/windowed_ruleset.json` (`wd-aitm-*`); depth plan 7.3 | `ci.yml :: python-test` (fusion, `test_aitm_session_rules.py` — 20 cases driving the **real** `WindowedDetectionEngine`, the real matcher and the **shipped** ruleset loaded off disk rather than a hand-made rule list: the replay fires all four rules, the negative twin fires none of them, a single event fires nothing, two sessions for one user do not complete each other, an event with no session id is not grouped at all, the sequence does not fire without its MFA stage or in the wrong order, and one tenant's sign-in cannot complete another's replay) + `ci.yml` connectors matrix (`test_aitm_session_fields_reach_detection.py` — 18 cases driving the real `OktaConnector.normalize()` and `AzureEntraConnector.normalize()` over hand-authored vendor payloads, including that `mfa_satisfied` is three-valued and that a Graph `correlationId` does not stand in for a missing `sessionId`) + `ci.yml :: python-lint` (`check_windowed_translation.py` — the ceiling of 0 windowed rules selecting on a computed field, which is what forced the connector hoist rather than a rule naming a field nothing emits; `export_windowed_ruleset.py --check` for drift between the authored list and the artefact the engine loads) | core | GATED | The two halves live in different interpreters because `services/connectors` and `services/fusion` both name their package `app`; the connector test pins the payload the engine test transcribes, so a connector change that drops a field reds the connector test rather than silently passing the engine one |## Summary