Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

Merge remote-tracking branch 'origin/main' into depth/6.1-ai-asset-inventory

Beenu Arorad2ef4c10dea6
diff --cc .github/workflows/ci.yml
index 0dbdeba0b,8de41fb49..1681a6170
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@@ -701,29 -701,18 +701,42 @@@ jobs
run: python3 scripts/check_cloud_native_collection.py --self-test
- name: Cloud-native collection gate
run: python3 scripts/check_cloud_native_collection.py
+ # Depth 9.2. The probe set behind the recall measurement has to stay
+ # usable: every probe must name an item that is in the corpus, the
+ # probes must stay low-overlap with their answers (a probe written in
+ # the document's own words hands the lexical baseline a score it could
+ # not earn on a real question), and the baseline must still be a real
+ # retriever rather than a stub. The scorer itself runs too: with no
+ # embedding backend on a runner it prints "not measured" and exits 0,
+ # which is the honest row and also proves the refusal path works.
+ - name: Memory recall scorer — self-test
+ run: python3 scripts/score_memory_recall.py --self-test
+ - name: Memory recall scorer
+ run: python3 scripts/score_memory_recall.py
++
+ # Depth 6.1. The AI estate is a stored value a console page reads, and
+ # the failure that costs most here is not an absent feature but a
+ # present one nothing writes: the page renders "no AI estate" to a
+ # tenant that has one, which is the single most dangerous false
+ # statement it could make. The WRITER rule is therefore the
+ # load-bearing one — it requires the fusion consumer to call the
+ # recorder on the raw-events branch *and* the service to construct it
+ # at startup, because an unconstructed recorder is a None that every
+ # call site skips in silence.
+ - name: AI asset inventory gate — self-test
+ run: python3 scripts/check_ai_asset_inventory.py --self-test
+ - name: AI asset inventory gate
+ run: python3 scripts/check_ai_asset_inventory.py
+ # Depth 6.2. The endpoint half, and the rule that carries it is
+ # NAMESPACE: the four endpoint connectors put their process fields in
+ # the connector payload under `raw_data`, and the canonical OCSF
+ # profile maps none of them — so a classifier reading `process.name`
+ # finds None on every real source while its unit tests pass against
+ # events built in the shape it expects.
+ - name: AI endpoint discovery gate — self-test
+ run: python3 scripts/check_ai_endpoint_discovery.py --self-test
+ - name: AI endpoint discovery gate
+ run: python3 scripts/check_ai_endpoint_discovery.py
# One layer in from the name: the profile gate asks whether a connector
# reaches a normalization path, this asks which OCSF class it reaches
# when it gets there. The class uid divided by a thousand is the
diff --cc DEPTH_PROGRESS.md
index c5eb6b0bc,5567c305c..ca371fc17
--- a/DEPTH_PROGRESS.md
+++ b/DEPTH_PROGRESS.md
@@@ -64,11 -64,12 +64,17 @@@ is written. Next free at capture: migra
| migration `107` | 9.4 — `107_case_war_rooms.sql` | claimed |
| migration `108` | 9.5 — `108_hunt_reports.sql` | claimed |
| migration `109` | 9.6 — `109_push_devices.sql` | claimed |
+ | migration `110` | 8.1 — `110_console_webauthn.sql` | claimed |
+ | migration `111` | 8.1 — `111_audit_retention.sql` | claimed |
+ | migration `112` | 8.1 — released, taken by another lane (D12) | **struck** |
+ | migration `113` | 8.1 — `113_entitlement_enforcement.sql` | claimed |
+ | ADR `0011` | 8.1 — truncating a hash-chained audit log | claimed |
+
+| migration `110` | 6.1 — `110_ai_assets.sql` | claimed |
+| migration `111` | 6.4 — `111_ai_agent_identities.sql` | claimed |
+| ADR `0011` | 6.6 — AI containment arms off by default, and the rollback a kill-switch needs | claimed |
+| — | 5.4 — `104_playbook_verification.sql` on `depth/5.4-5.6-lane-c`, and 7.1 — `104_phishing_campaigns.sql` on `depth/7.1-7.3-phishing` | **collision, unresolved here.** Both lanes wrote a `104` on a branch without a row in this table. Phase 9's `104` is the one with a claim, so these two need renumbering by their own lanes; recorded rather than renamed from outside, because renaming another lane's migration from here is how a branch comes back to a file that no longer exists |
+
Highest migration on `main` at claim time was `101_audit_immutable_nullify.sql`,
so `095`--`101` were taken by sibling lanes between the capture above and this
claim; re-read `services/api/migrations/` rather than trusting the capture row.
@@@ -84,15 -80,13 +90,22 @@@ and `103` were already spoken for in th
through `109`, one per item. A number left unused at the end of the phase is
released by striking its row rather than by silently leaving a gap.
+ Item 8.1 re-read `services/api/migrations/` on 2026-10-11 before claiming:
+ `105_semantic_memory.sql` was the highest on `main`, and `102`--`109` were
+ already spoken for above, so 8.1 takes `110`--`112` and ADR `0011`. The three
+ sub-items that need no schema — the containerised-IdP CI leg,
+ internationalisation and accessibility — claim no number. The same release
+ rule applies: an unused number is struck, not silently skipped.
+
+**Phase 6 moved `105`/`106` to `110`/`111` on 2026-10-11**, because Phase 9's
+claim for `104`--`109` reached this table on `main` first and a claim table
+nobody defers to is not a coordination mechanism. The lesson is the one this
+section already states and that three lanes still fell to: the table is
+authoritative only for lanes that write to it **before** the file, and four
+branches now carry a `104` or a `105` that was never claimed here. Reading
+`main` is not enough; read `git ls-tree` over every `origin/*` branch as
+well, which is what found this.
+
## Phase 0: Kickoff
- [x] **0.1** `DEPTH_PROGRESS.md` exists (this file), modelled on
@@@ -547,6 -724,10 +831,12 @@@ endpoint produces once triage falls thr
| 2026-10-09 | 3.3 | Identity privilege (18 rules) and a per-tenant first-seen store (2) built; 5 rules were never unreachable (gate operator drift, D9); 18 retired with a reason (D10). `MAX_UNREACHABLE` 45 → 2, the two remaining both needing the 3.4 baseline. Two matcher operators three shipped rules already used were implemented. |
| 2026-10-09 | 3.2 | Ordered and unordered sequences in the windowed engine; all four translatable Sigma correlation types compile, seven refusal reasons recorded. No upstream correlation rule exists in this tree to import (D8), so the corpus is first-party and labelled as such. |
| 2026-10-09 | 4.1 | Reproduced: `check_cloud_native_collection.py` on the unmodified tree reported all three collection paths absent. Implemented `aws_cloudtrail_s3` (SQS-notified S3 objects; management events, data events and VPC flow logs; delete-after-emit), `gcp_pubsub` (REST pull on a log-sink subscription; ack-after-read) and `azure_event_hubs` (Capture blobs over the Data Lake Gen2 JSON API, with a focused Avro OCF reader). Negative controls recorded in the PR: deleting the SQS message before the object is read, acknowledging the Pub/Sub batch before it is built, ignoring the Avro union branch index, and ignoring the flow-log header line each fail a named test, and all were restored. Connectors suite 1014 -> 1098 passed. |
+| 2026-10-11 | 6.2 | Reproduced: 13 of 16 new cases failed, driving the shapes the four endpoint connectors actually emit. Root cause was 6.1's endpoint branch reading `process.name` off OCSF, which the canonical profile never populates — it found `None` on every real source while its own tests passed. Fixed by reconstructing the connector namespace. Shipped the osquery pack (5 queries, packs 5 -> 6), the source-capability table, the operator doc and a 6-rule gate. Fusion suite 683 -> 700. Claim row is PARTIAL: no osquery agent runs in CI, and the EDR connectors return vendor detections rather than process telemetry, so the plan's "CrowdStrike, Defender, SentinelOne" wording is not satisfiable with these connectors. |
+| 2026-10-11 | 6.1 | Reproduced: the new gate reported nine problems across five of eight rules on the unmodified tree. Built `aisoc_ai_assets` (migration 110), its routes behind a new `ai_estate:*` permission pair, the fusion recorder that writes it on the default profile, the `AIAsset` graph label with `OPERATES`/`HOSTS` edges, and the `/ai-estate` page. Two negative controls against a real PostgreSQL: reverting `NULLS NOT DISTINCT` turns one provider-key row into three, and a `USING (true)` policy leaks across tenants. Claim matrix 302 -> 303 rows, 300 GATED. Writing the gate found two blind spots in the gate itself. |
| 2026-10-09 | 3.1 | All 74 decided. 50 translated into `wd-*` rules derived from each original's own clauses and replayed through the real engine (162 assertions); 24 refused with a reason across five kinds. `MAX_UNREACHABLE` 119 → 45, published executable 2,603 → 2,529, windowed 18 → 68. Three findings recorded as D4–D6 below. |
+ | 2026-10-11 | 8.1 | parity 4.1's containerised-IdP leg **observed to pass**, both halves: `keycloak-oidc` verifies a real `id_token` against Keycloak's published JWKS with an audience-mismatch negative control, and `keycloak-saml` walks the whole SP-initiated flow to the real ACS and signs `[email protected]` in as `infosec`, with an untrusted-key control answering 400. Run `38124780886`, as `ENVIRONMENT=test` against the DML-only `aisoc_app` role. The SSO claim row moves PARTIAL → GATED on that evidence. Building it found two shipped defects no self-authored provider could expose: `resolve_saml_connection` bound `:i` untyped so `:i IS NULL` raised on asyncpg, and Keycloak's default role-list mapper emits one `<Attribute Name="Role">` per role, which python3-saml refuses. |
+ | 2026-10-11 | 8.1 | WebAuthn as a console second factor and MFA enforced per role. Fixing the branch's own gates found three `passkey_credentials` reads scoped by user id alone on a tenant-scoped table, and a permission-model violation where the MFA policy route imported `ROLE_PERMISSIONS` directly; the role *vocabulary* now has one owner in `role_grants.known_roles()`. 23 of 23 live cases pass against real Postgres, and deleting the login integration fails 14 of them — measured, where the workflow had said eight. |
+ | 2026-10-11 | 8.1 | An enforced audit retention window with a re-anchored checkpoint, ADR-0011, and 12 live cases. Writing them found that migration `111` declared `ON DELETE CASCADE` on the checkpoint table and then made the cascade impossible; and, separately, that `audit_log`'s own trigger from migration `101` blocks `DELETE FROM tenants` for every tenant that has written a row, so tenant deletion is impossible on `main`. Reproduced and filed as #1330 rather than fixed here. |
+ | 2026-10-11 | 8.1 | Opt-in per-tenant limit enforcement (migration `113`, D12), wired to `POST /connectors` and `POST /tenants/me/users` through one helper, with `GET`/`PUT /usage/enforcement` as the operator surface. Nine live cases, half of them positive controls: a guard that never refuses fails 1, a guard stuck closed fails 4. |
+ | 2026-10-11 | 8.1 | axe extended to the case list and case workspace, the two of parity 4.7's five views that were left out. Four real defects found and fixed: two `<select>` controls with no accessible name (one of them the control that writes a case's status), two indistinguishable `<aside>` landmarks, and panel headings at `h3` directly under the page's `h1`. |
+
diff --cc ROADMAP.md
index aeb5272c3,d029c22f9..f8189c1d7
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@@ -45,7 -45,7 +45,9 @@@ A proof-first, security-first program t
- **Phase D (breadth):** D1 eight connectors (QRadar/Exabeam/Securonix/Devo/Netskope/Windows-Sysmon/Zeek-Suricata/syslog-CEF) · D2 AI/LLM-usage audit connector + `llm-*` detections + hot/cold lake tiering · D3 live-vendor mock-server smoke.
- **Phase E (prove it):** E1 CI-gated benchmark scoreboard tied to a deterministic live-agent MITRE-accuracy run.
+ The claim-to-gate matrix stands at **302 rows — 300 GATED / 2 PARTIAL / 0 NO GATE**. Two rows are PARTIAL. "The retention window a tenant chooses is the one the lake honours" stays PARTIAL until a live ClickHouse test watches a row survive past the old 90-day boundary. "Cloud-native collection" stays PARTIAL until a capture recorded from a live Event Hub replaces the fixture synthesised from the Avro specification. "SSO completes a sign-in" was the third, and depth 8.1 closed it by building the gate it named rather than by relabelling: `sso-live.yml` now stands up a containerised Keycloak and both legs have been observed to pass — `keycloak-oidc` takes a real `id_token` through `_verify_id_token` against the provider's published JWKS, and `keycloak-saml` walks the whole SP-initiated flow to the real ACS. Every other product claim is backed by a failing test, none is a named deferral either, and the ratchet (`MAX_NO_GATE=0`) forbids any regression. Every PARTIAL row was closed by building the gate it named, never by relabelling. The last two rested on the live-agent eval workflow, which this file said had simply never run: the real reason was that its live path imported a class that exists nowhere in `services/agents`, so its first dispatch failed in 92 seconds and every later one would have. With that fixed the workflow measures, and the groundedness floor is **0.40 over a deterministic 10-incident slice**, derived from ten runs across two environments rather than chosen — seven local runs returning 0.5561 to four decimal places, three GitHub-runner runs returning 0.5821, 0.5329 and 0.5933. Every one is a locally-served `qwen2.5:0.5b`; no hosted provider has been exercised and the floor describes none. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).
++
+The claim-to-gate matrix stands at **304 rows — 300 GATED / 4 PARTIAL / 0 NO GATE**. Four rows are PARTIAL. "The retention window a tenant chooses is the one the lake honours" stays PARTIAL until a live ClickHouse test watches a row survive past the old 90-day boundary. "Cloud-native collection" stays PARTIAL until a capture recorded from a live Event Hub replaces the fixture synthesised from the Avro specification. And "SSO completes a sign-in": fix pass 4.1 drove both handlers end to end against real Postgres and real signed assertions, and it stays PARTIAL until the containerised-IdP leg of `sso-live.yml` has been observed to pass, because the identity providers in that suite are trust material this repository authors. "Endpoint discovery of the AI estate" stays PARTIAL until a containerised osquery runs `packs/aisoc-ai-estate.yaml` end to end: the pack is asserted to load and to select the columns the recorder classifies on, but no test has watched a real `osqueryd` execute one of its queries against a real filesystem. Every other product claim is backed by a failing test, none is a named deferral either, and the ratchet (`MAX_NO_GATE=0`) forbids any regression. Every PARTIAL row was closed by building the gate it named, never by relabelling. The last two rested on the live-agent eval workflow, which this file said had simply never run: the real reason was that its live path imported a class that exists nowhere in `services/agents`, so its first dispatch failed in 92 seconds and every later one would have. With that fixed the workflow measures, and the groundedness floor is **0.40 over a deterministic 10-incident slice**, derived from ten runs across two environments rather than chosen — seven local runs returning 0.5561 to four decimal places, three GitHub-runner runs returning 0.5821, 0.5329 and 0.5933. Every one is a locally-served `qwen2.5:0.5b`; no hosted provider has been exercised and the floor describes none. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).
## v4.0 — Shipped
diff --cc apps/docs/docs/compliance/evidence-pack.md
index ef13f1440,7890b5387..16884fac9
--- a/apps/docs/docs/compliance/evidence-pack.md
+++ b/apps/docs/docs/compliance/evidence-pack.md
@@@ -34,7 -34,7 +34,9 @@@ and says so
| Claim-to-gate traceability | Every product claim mapped to the CI job that fails when it stops being true | `docs/audit/CLAIM_TO_GATE_MATRIX.md` |
The last row is the one worth reading first. It is the honest index: 300 rows
+ `GATED`, 2 `PARTIAL`, and a ratchet that refuses to let a row sit at
++
+`GATED`, 4 `PARTIAL`, and a ratchet that refuses to let a row sit at
`NO GATE`. Re-read the figure from the table rather than from here — a count
copied into prose goes stale in silence, which is why
`scripts/readme_gates.py` cross-checks this sentence against the matrix.
diff --cc docs/audit/CLAIM_TO_GATE_MATRIX.md
index 6081b7810,712149e18..307617d59
--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md
+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md
@@@ -315,9 -313,9 +315,12 @@@ Statuses: `GATED` (a CI job fails when
## Summary
- - Total: 304
+ - Total: 302
- GATED: 300
+ - PARTIAL: 2
++
++- Total: 304
+- PARTIAL: 4
- NO GATE: 0
Counts come from `scripts/check_claim_gate_matrix.py`, which is also the gate. Recompute
diff --cc docs/openapi.yaml
index eeeb2caa2,977d4b391..941b804b1
--- a/docs/openapi.yaml
+++ b/docs/openapi.yaml
@@@ -7035,13 -7372,38 +7372,36 @@@ paths
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
- /api/v1/ai-assets:
- get:
+ /api/v1/memory/items:
+ post:
tags:
+ - semantic-memory
+ summary: Ingest Item
+ operationId: ingest_item_api_v1_memory_items_post
+ security:
+ - HTTPBearer: []
+ requestBody:
+ required: true
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/MemoryItemIn'
+ responses:
+ '201':
+ description: Successful Response
- content:
- application/json:
- schema:
+ $ref: '#/components/schemas/MemoryIngestResult'
+ '422':
+ description: Validation Error
- content:
- application/json:
- schema:
+ $ref: '#/components/schemas/HTTPValidationError'
+ get:
- tags:
- - semantic-memory
+ summary: List Items
+ operationId: list_items_api_v1_memory_items_get
++
++ /api/v1/ai-assets:
+ - ai-estate
+ summary: List Ai Assets
+ description: The estate, newest sighting first.
+ operationId: list_ai_assets_api_v1_ai_assets_get
security:
- HTTPBearer: []
parameters:
@@@ -7059,11 -7419,10 +7417,17 @@@
in: query
required: false
schema:
+ anyOf:
+ - type: string
+ - type: 'null'
+ title: State
++
++ description: Filter to one asset kind.
++ - name: unlinked
+ type: boolean
+ description: Only assets with no person linked.
+ default: false
+ title: Unlinked
- description: Only assets with no person linked.
- name: limit
in: query
required: false
@@@ -7087,81 -7438,149 +7443,160 @@@
content:
application/json:
schema:
- $ref: '#/components/schemas/AiAssetListResponse'
+ type: array
+ items:
+ $ref: '#/components/schemas/MemoryItemOut'
+ title: Response List Items Api V1 Memory Items Get
'422':
description: Validation Error
-- content:
-- application/json:
-- schema:
$ref: '#/components/schemas/HTTPValidationError'
- /api/v1/ai-assets/{asset_id}:
- get:
+ /api/v1/memory/items/{item_id}:
+ delete:
tags:
- - ai-estate
- summary: Get Ai Asset
- operationId: get_ai_asset_api_v1_ai_assets__asset_id__get
+ - semantic-memory
+ summary: Retire Item
+ description: Retire, never hard-delete. 'What did the agent know' must stay
+ answerable.
+ operationId: retire_item_api_v1_memory_items__item_id__delete
security:
- HTTPBearer: []
parameters:
- - name: asset_id
+ - name: item_id
in: path
required: true
-- schema:
type: string
format: uuid
+ title: Item Id
+ - name: reason
+ in: query
+ required: false
- schema:
- type: string
+ default: ''
+ title: Reason
++
++ maximum: 200
++ default: 50
++ - name: offset
++ type: integer
++ minimum: 0
++ default: 0
++ title: Offset
++ $ref: '#/components/schemas/AiAssetListResponse'
++ /api/v1/ai-assets/{asset_id}:
++ get:
++ - ai-estate
++ summary: Get Ai Asset
++ operationId: get_ai_asset_api_v1_ai_assets__asset_id__get
++ - name: asset_id
+ title: Asset Id
responses:
'200':
description: Successful Response
content:
application/json:
schema:
+ $ref: '#/components/schemas/MemoryItemOut'
++
+ $ref: '#/components/schemas/AiAssetModel'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
- /api/v1/ai-assets/{asset_id}/sanction:
+ /api/v1/memory/recall:
post:
tags:
+ - semantic-memory
+ summary: Recall
+ description: 'Vector recall over this tenant''s memory. Every hit carries its
+ provenance.
+
+
+ ``knowledge_base:read`` is the right permission and it already exists:
+
+ ``POST /kb/query`` uses it for the full-text half of the same corpus,
+
+ and every role that triages holds it. Reusing it rather than minting a
+
+ ``memory:read`` keeps one answer to "may this principal search what the
+
+ organisation knows" — two permissions over one corpus is two answers
+
+ that can disagree, and the disagreement surfaces on the day somebody
+
+ narrows a role.'
+ operationId: recall_api_v1_memory_recall_post
+ requestBody:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/MemoryRecallIn'
+ required: true
+ responses:
+ '200':
+ description: Successful Response
- content:
- application/json:
- schema:
+ items:
+ $ref: '#/components/schemas/MemoryRecallHit'
+ type: array
+ title: Response Recall Api V1 Memory Recall Post
+ '422':
+ description: Validation Error
- content:
- application/json:
- schema:
+ $ref: '#/components/schemas/HTTPValidationError'
+ security:
+ - HTTPBearer: []
+ /api/v1/memory/conflicts:
+ get:
- tags:
- - semantic-memory
+ summary: List Conflicts
+ operationId: list_conflicts_api_v1_memory_conflicts_get
- responses:
- '200':
- description: Successful Response
- content:
- application/json:
- schema:
- items:
+ $ref: '#/components/schemas/MemoryConflictOut'
- type: array
+ title: Response List Conflicts Api V1 Memory Conflicts Get
- security:
- - HTTPBearer: []
+ /api/v1/memory/conflicts/{item_id}/resolve:
- post:
- tags:
- - semantic-memory
+ summary: Resolve Conflict
+ description: Settle a conflict by naming which side survives. The other is retired.
+ operationId: resolve_conflict_api_v1_memory_conflicts__item_id__resolve_post
- security:
- - HTTPBearer: []
+ parameters:
+ - name: item_id
++
++ /api/v1/ai-assets/{asset_id}/sanction:
+ - ai-estate
+ summary: Sanction Ai Asset
+ description: 'Approve an asset, refuse it, or put it back to undecided.
+
+
+ The risk score is recomputed here rather than left stale, through the
+
+ same `score_risk` the recorder calls, because the sanction decision is
+
+ one of its inputs: an asset that has just been refused and is still
+
+ running is a different exposure from one nobody had looked at.'
+ operationId: sanction_ai_asset_api_v1_ai_assets__asset_id__sanction_post
- security:
- - HTTPBearer: []
- parameters:
+ - name: asset_id
in: path
required: true
schema:
type: string
format: uuid
+ title: Item Id
++
+ title: Asset Id
requestBody:
required: true
content:
application/json:
schema:
+ $ref: '#/components/schemas/MemoryConflictResolution'
++
+ $ref: '#/components/schemas/SanctionRequest'
responses:
'200':
description: Successful Response
content:
application/json:
schema:
+ $ref: '#/components/schemas/MemoryConflictOut'
++
+ $ref: '#/components/schemas/AiAssetModel'
'422':
description: Validation Error
content: