Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
Merge PR #1329: depth 8.1 — enforced audit retention with verifiable chain
beenuar00355b84fbb7
diff --cc ROADMAP.mdindex 62129aee3,9144165fc..2b82407b2--- a/ROADMAP.md+++ b/ROADMAP.md@@@ -45,7 -45,9 +45,8 @@@ A proof-first, security-first program t- **Phase D (breadth):** D1 eight connectors (QRadar/Exabeam/Securonix/Devo/Netskope/Windows-Sysmon/Zeek-Suricata/syslog-CEF) · D2 AI/LLM-usage audit connector + `llm-*` detections + hot/cold lake tiering · D3 live-vendor mock-server smoke.- **Phase E (prove it):** E1 CI-gated benchmark scoreboard tied to a deterministic live-agent MITRE-accuracy run.- The claim-to-gate matrix stands at **311 rows — 309 GATED / 2 PARTIAL / 0 NO GATE**. Two rows are PARTIAL. "The retention window a tenant chooses is the one the lake honours" stays PARTIAL until a live ClickHouse test watches a row survive past the old 90-day boundary. "Cloud-native collection" stays PARTIAL until a capture recorded from a live Event Hub replaces the fixture synthesised from the Avro specification. "SSO completes a sign-in" was the third, and depth 8.1 closed it by building the gate it named rather than by relabelling: `sso-live.yml` now stands up a containerised Keycloak and both legs have been observed to pass — `keycloak-oidc` takes a real `id_token` through `_verify_id_token` against the provider's published JWKS, and `keycloak-saml` walks the whole SP-initiated flow to the real ACS. Every other product claim is backed by a failing test, none is a named deferral either, and the ratchet (`MAX_NO_GATE=0`) forbids any regression. Every PARTIAL row was closed by building the gate it named, never by relabelling. The last two rested on the live-agent eval workflow, which this file said had simply never run: the real reason was that its live path imported a class that exists nowhere in `services/agents`, so its first dispatch failed in 92 seconds and every later one would have. With that fixed the workflow measures, and the groundedness floor is **0.40 over a deterministic 10-incident slice**, derived from ten runs across two environments rather than chosen — seven local runs returning 0.5561 to four decimal places, three GitHub-runner runs returning 0.5821, 0.5329 and 0.5933. Every one is a locally-served `qwen2.5:0.5b`; no hosted provider has been exercised and the floor describes none. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).+ The claim-to-gate matrix stands at **303 rows — 301 GATED / 2 PARTIAL / 0 NO GATE**. Two rows are PARTIAL. "The retention window a tenant chooses is the one the lake honours" stays PARTIAL until a live ClickHouse test watches a row survive past the old 90-day boundary. "Cloud-native collection" stays PARTIAL until a capture recorded from a live Event Hub replaces the fixture synthesised from the Avro specification. "SSO completes a sign-in" was the third, and depth 8.1 closed it by building the gate it named rather than by relabelling: `sso-live.yml` now stands up a containerised Keycloak and both legs have been observed to pass — `keycloak-oidc` takes a real `id_token` through `_verify_id_token` against the provider's published JWKS, and `keycloak-saml` walks the whole SP-initiated flow to the real ACS. Every other product claim is backed by a failing test, none is a named deferral either, and the ratchet (`MAX_NO_GATE=0`) forbids any regression. Every PARTIAL row was closed by building the gate it named, never by relabelling. The last two rested on the live-agent eval workflow, which this file said had simply never run: the real reason was that its live path imported a class that exists nowhere in `services/agents`, so its first dispatch failed in 92 seconds and every later one would have. With that fixed the workflow measures, and the groundedness floor is **0.40 over a deterministic 10-incident slice**, derived from ten runs across two environments rather than chosen — seven local runs returning 0.5561 to four decimal places, three GitHub-runner runs returning 0.5821, 0.5329 and 0.5933. Every one is a locally-served `qwen2.5:0.5b`; no hosted provider has been exercised and the floor describes none. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).+-The claim-to-gate matrix stands at **303 rows — 301 GATED / 2 PARTIAL / 0 NO GATE**. Three rows are PARTIAL. "The retention window a tenant chooses is the one the lake honours" stays PARTIAL until a live ClickHouse test watches a row survive past the old 90-day boundary. "Cloud-native collection" stays PARTIAL until a capture recorded from a live Event Hub replaces the fixture synthesised from the Avro specification. And "SSO completes a sign-in": fix pass 4.1 drove both handlers end to end against real Postgres and real signed assertions, and it stays PARTIAL until the containerised-IdP leg of `sso-live.yml` has been observed to pass, because the identity providers in that suite are trust material this repository authors. Every other product claim is backed by a failing test, none is a named deferral either, and the ratchet (`MAX_NO_GATE=0`) forbids any regression. Every PARTIAL row was closed by building the gate it named, never by relabelling. The last two rested on the live-agent eval workflow, which this file said had simply never run: the real reason was that its live path imported a class that exists nowhere in `services/agents`, so its first dispatch failed in 92 seconds and every later one would have. With that fixed the workflow measures, and the groundedness floor is **0.40 over a deterministic 10-incident slice**, derived from ten runs across two environments rather than chosen — seven local runs returning 0.5561 to four decimal places, three GitHub-runner runs returning 0.5821, 0.5329 and 0.5933. Every one is a locally-served `qwen2.5:0.5b`; no hosted provider has been exercised and the floor describes none. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).## v4.0 — Shippeddiff --cc docs/audit/CLAIM_TO_GATE_MATRIX.mdindex 254c75c26,3898f927c..cd7fb9e2a--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md@@@ -80,8 -69,8 +80,9 @@@ Statuses: `GATED` (a CI job fails when| Analyst disposition history drives human-approval-gated tuning proposals | README (self-improving detections) | `ci.yml` api job (`test_wave1_loop_edges.py` — the previously-orphaned `suggest_tuning` engine is wired to real per-rule disposition history via `/detection/tuning/auto-suggest`, opening DRAFT proposals; `test_detection_tuner.py` fixture-gates the engine) + `ci.yml :: Python — Lint & Type-check` (`check_raw_sql_columns.py` — every table and column a raw `text()` INSERT/UPDATE names must be one some migration creates) | core | GATED (the gap this row used to name was wrong in the repository's favour: no job ran `services/api/tests/` against a real database at all, and the only assertion on the insert was a substring test that would still have passed on the wrong table name, because the wrong name contains the right one. The structural gate caught what no mock could — `detection_rule_proposals` has never had a `source` column, so all three automated proposal paths raised on their first row on every deployment; migration 073 adds it) | — || The rule-management surface lists the rules the detection engine actually runs, and disabling one reaches the engine | `apps/docs/docs/console/rule-library.md`; [#1273](https://github.com/beenuar/AiSOC/issues/1273) | `ci.yml :: python-lint` (`sync_packaged_detection_rulesets.py --self-test` then `--check`: the three compiled artefacts and the three Python-resident windowed rules must exist byte-identical inside the API's Docker build context, checked in both directions so a file only on the packaged side fails too) + `validate-detections.yml` (the same check, on the workflow a corpus change triggers) + `ci.yml :: Python — Tests` (api, `test_builtin_rule_catalogue.py` — the catalogue's rule-id set is asserted **equal** to the union of ids fusion's own `_load_ruleset()`, `load_window_rules()` and `load_sequence_rules()` return, run in a subprocess because both services name their top-level package `app`; the console route is driven through a real ASGI app and must report 2,586; and disabling a built-in is followed by handing the stored row to fusion's real `build_overlay`, loaded by path, which must report it suppressed) + web type-check and `DetectionsView.honesty.test.tsx` | core | GATED | Both API routes and the MCP tool read the Postgres `detection_rules` table and nothing in production writes a built-in row — `tenant_id` is `NOT NULL` in every migration, so the platform-wide row they look for cannot exist. The constraint that shaped the fix is packaging: the API image is built from `services/api`, so `services/fusion/app/data/` was never reachable and a handler change alone would pass in a checkout and fail in every container. Built-ins are not seeded into Postgres; the artefact stays the single source of truth for what runs and a tenant row materialises on first write, keyed on `provenance->>'source_id'`, which is what the overlay already joins on || Detections can be backtested over historical lake data before promotion | README (detection engineering) | `ci.yml` api job (`test_backtest.py` — bounded/allow-listed SQL construction with injection-sanitised source filter, lake-row→event mapping with JSON payload expansion, exact would-fire count + hit-rate + samples; `test_backtest_gate.py` — opt-in `AISOC_BACKTEST_MAX_HIT_RATE` /decide gate) + web type-check (rule-editor backtest panel) + `isolation-live.yml` (`tests/isolation/test_backtest_lake_live.py` — the real `fetch_lake_events` against ClickHouse 23.8, table built from `clickhouse/001_init.sql` and rows seeded through the production `lake_writer.event_to_row`) | full | GATED (the gap this row used to name was wrong: nothing ran the live fetch, and the tenant-scoping path existed twice — `detection_rules.py` carried its own near-identical copy, now deleted, with a test asserting exactly one definition survives. The live path exposed a defect no unit could: the writer stores OCSF `raw_data` into a column named `raw_payload` and `rows_to_events` expanded neither, so the payload expansion its docstring promises had never run on a real lake row. `services/api/app/services/backtest.py` was also absent from the workflow's path filter, so a change to the function under test did not trigger the job) | — |+| Asking the affected person, then their manager, is a workflow a playbook branches on — not a message that is sent and forgotten | `services/agents/app/playbook/verification.py`; `playbooks/packs/v1/suspicious-signin/user-verification-containment.playbook.json`; depth plan 5.4 | `ci.yml :: python-lint` (`check_stateful_verification.py --self-test` then the gate: the answer is HMAC-signed over token, answer **and** stage — so a link minted for the subject stops verifying once the question escalates — compared with `compare_digest`; an unanswered prompt escalates to the manager and then resumes with `timed_out`, a value a playbook can read; the confirmed-true-positive refusal is checked **before** the opt-in, because that rule has no opt-out and the other order would let a tenant opt into messaging a compromised account; and `aisoc_verification_policies` has a writer, since a control that fails closed with no way to switch it on is a feature that reads as broken. Eleven injected violations, one per rule) + `ci.yml` agents job (`test_playbook_verification.py`, 20 cases driving the real engine end to end: ask, pause, signed answer, branch, approval — and the stale-link case, where the subject's own `denied` link is refused after the escalation and a manager-stage one is accepted) + `verification-live.yml` (`test_verification_pause_live.py`, 12 cases against real Postgres with every migration applied, because only a real transaction can show that the conditional UPDATEs claim a row once and that migration 102's CHECK admits all five outcomes the code writes) | core | GATED | — || The retention window a tenant chooses is the one the lake honours, and a legal hold outranks it | `services/api/app/services/retention.py`; `services/api/clickhouse/001_init.sql`; depth plan 4.3 | `ci.yml :: python-lint` (`check_retention_window.py --self-test` then the gate: the table TTL, the tiering DELETE and `MAX_LAKE_DAYS` are held to one number at or above 400 days, a `MODIFY TTL` migration exists so an existing volume converges — `001_init.sql` runs only on a fresh one — and the purge worker reads the holds in force, decides through `may_purge`, and reports what it withheld. Eight injected violations, one per rule) + `ci.yml` api job (`test_retention_legal_hold.py`, 13 cases driving the worker against recording doubles. **Half of them are the positive control**, because a purge stuck closed withholds everything on every deployment and passes every "did the hold work" assertion perfectly: a tenant with no hold is asserted to still lose its aged rows, and a hold on one tenant is asserted not to freeze another. Proven against the pre-fix worker, where five fail) | core | PARTIAL (the lake half is static: ClickHouse enforces a table TTL during merges and no test in CI watches a row survive past day 90 on a real server, so what is gated is that the three numbers agree and that the migration exists, not that ClickHouse honoured it. The purge half is gated against doubles rather than live Postgres and ClickHouse. And a hold currently stops the whole tenant's sweep rather than only its own subjects, because both purges are bulk statements — recorded as follow-up rather than approximated) | a live-store retention test |+ | The audit retention window a tenant configures is enforced, and the surviving chain still verifies | `docs/decisions/0011-truncating-a-hash-chained-audit-log.md`; `services/api/app/workers/retention_purge.py`; depth plan 8.1 | `audit-retention-live.yml :: audit-retention-live` (`tests/isolation/test_audit_retention_live.py`, 12 cases against real Postgres as the DML-only `aisoc_app` role: the window removes the rows past it, the checkpoint agrees with the rows it describes from both sides, the remainder replays clean from the anchor, a direct DELETE and a hand-set purge flag are both refused by privilege, and the checkpoint is itself immutable. The job reads the DELETE privilege out of the catalogue, refuses a run that skipped a case, and carries a negative control that ignores the anchor and requires the suite to go red) | core | GATED | `retention_policies.audit_days` was storable since the table existed and purged by nothing, because `audit_log` is an append-only hash chain: delete the oldest rows and `verify_chain_breaks`, which replays from `prev_hash = NULL`, reports a mismatch at index 0 — an intact chain reads as tampered with, which is the worst outcome for a control whose only job is to tell those apart. Migration `111` adds `audit_retention_truncate`, a `SECURITY DEFINER` function that writes a checkpoint (the cutoff, the window, the count, the hash of the last row removed and the predecessor the oldest survivor names) and only then deletes, in one transaction; the verifier re-anchors on it. The application role holds **no DELETE on `audit_log`**, so the function is the only path rather than the intended one — the trigger's transaction-local setting is a second control, but a custom GUC can be set by anybody. **Does not unblock tenant deletion**: `audit_log`'s trigger from migration `101` refuses the cascade from `tenants` too, so `DELETE FROM tenants` raises for any tenant that has written a row. Pre-existing, tracked separately, and the checkpoint table deliberately does not repeat it || Detections can be authored in Python (`def rule(event)`) with a fixture gate | README (detection engineering) | `python-detections.yml` (`aisoc_detections.runner` fails a blind rule that misses a positive or a noisy one that fires on a negative; every bundled detection must ship ≥1 positive + ≥1 negative case) + `test_framework.py` (fail-closed evaluate, metadata validation) | core | GATED | - || AI Detection Builder: NL → rule + AUTO-generated fixtures → non-circular eval-gate → governed proposal | README (detection engineering) | `ci.yml` api job (`test_ai_detection_builder.py` — fixtures are DERIVED from the generated rule's own selection, not invented, and genuinely pass the non-circular gate: fires on the positive, quiet on the negative) | core | GATED | - || No-code / simple detection builder compiles to a valid rule | README (detection engineering) | `ci.yml` web type-check + lint (`SimpleRuleBuilder.tsx` `buildSigma` compiles field/operator/value rows to Sigma, dropped into the governed editor) | core | GATED | - |diff --cc docs/decisions/README.mdindex c649d8232,d1116df91..3295b38be--- a/docs/decisions/README.md+++ b/docs/decisions/README.md@@@ -24,7 -24,7 +24,8 @@@ ADRs are immutable once accepted. If a| [ADR-0008](./0008-scim-trust-boundary.md) | SCIM is a third-party write surface into identity, and is scoped by its credential | accepted | 2026-09-27 || [ADR-0009](./0009-auto-close-without-an-earned-grant.md) | Whether a tenant with no closure policy may auto-close | proposed | 2026-10-04 || [ADR-0010](./0010-entity-and-session-context-in-core.md) | Entity and session context in CORE is a Postgres store; Neo4j stays in `full` | accepted | 2026-10-11 |+| [ADR-0011](./0011-behavioural-baselines-in-core.md) | Behavioural baselines in CORE are a fusion module; the UEBA service stays in `full` | accepted | 2026-10-11 |+ | [ADR-0011](./0011-truncating-a-hash-chained-audit-log.md) | Truncating a hash-chained audit log without making an intact chain look tampered with | accepted | 2026-10-11 |## Authoring conventions