Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

Merge PR #1318: depth 9.6 — native responder

beenuar64e24130d30e
diff --cc .github/workflows/ci.yml
index 47ce4d84e,707c3612f..2a1412089
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@@ -716,19 -701,16 +716,29 @@@ jobs
run: python3 scripts/check_cloud_native_collection.py --self-test
- name: Cloud-native collection gate
run: python3 scripts/check_cloud_native_collection.py
+ # Depth 9.1. Moving the MCP server off stdio and onto HTTP changes who
+ # is allowed to decide what: the credential arrives per request and
+ # several tenants share one process. This asserts the properties that
+ # makes safe — a transport exists, it never falls back to the
+ # process-wide API key, every call is authorized by the API before
+ # dispatch and reported after, a non-read-only tool cannot run on a
+ # bearer token alone, the API's tool catalogue matches the TypeScript
+ # registry name for name and flag for flag, and something deploys it.
+ # Self-test first, on the same tree: one injected violation per rule.
+ - name: MCP over HTTP gate — self-test
+ run: python3 scripts/check_mcp_over_http.py --self-test
+ - name: MCP over HTTP gate
+ run: python3 scripts/check_mcp_over_http.py
+ # Depth 9.6. Voice input transcribes on the device and sends text, and
+ # "no audio leaves the device" is a claim a comment cannot carry. This
+ # refuses any capture API or binary request body under
+ # apps/mobile/src. Also wired into mobile.yml, which only runs when
+ # that app changes — here it is unconditional, because a claim this
+ # strong should not be graded by a path filter.
+ - name: No audio egress gate — self-test
+ run: python3 scripts/check_no_audio_egress.py --self-test
+ - name: No audio egress gate
+ run: python3 scripts/check_no_audio_egress.py
# Depth 9.2. The probe set behind the recall measurement has to stay
# usable: every probe must name an item that is in the corpus, the
# probes must stay low-overlap with their answers (a probe written in
diff --cc .train_resolve.py
index e52d3aa37,000000000..8e4d91dda
mode 100644,000000..100644
--- a/.train_resolve.py
+++ b/.train_resolve.py
@@@ -1,90 -1,0 +1,93 @@@
+#!/usr/bin/env python3
+"""Train conflict resolver for the recurring tracker shapes (session tool, not shipped).
+
+Usage: python3 .train_resolve.py <file> <mode>
+ modes:
+ union -> ours + theirs for every hunk
+ theirs -> theirs for every hunk
+ roadmap -> theirs, deduping repeated 'claim-to-gate matrix stands at' paragraphs
+Run from the worktree root. Refuses to leave markers behind.
+"""
+import re
+import sys
+import pathlib
+
- CONFLICT = re.compile(r"<<<<<<< [^\n]*\n(.*?)(?:\|\|\|\|\|\|\| [^\n]*\n.*?)?=======\n(.*?)>>>>>>> [^\n]*\n", re.DOTALL)
++CONFLICT = re.compile(
++ r"^<<<<<<< [^\n]*\n(.*?)(?:^\|\|\|\|\|\|\| [^\n]*\n.*?)?^=======\n(.*?)^>>>>>>> [^\n]*\n",
++ re.DOTALL | re.MULTILINE,
++)
+
+
+def merge_checkboxes(ours: str, theirs: str) -> str:
+ """Same line set modulo [x]/[ ]: checked wins per line. Falls back to union."""
+ ol, tl = ours.splitlines(True), theirs.splitlines(True)
+ norm = lambda s: s.replace("- [x]", "- [ ]", 1)
+ if len(ol) == len(tl) and all(norm(a) == norm(b) for a, b in zip(ol, tl)):
+ return "".join(a if "- [x]" in a else b for a, b in zip(ol, tl))
+ return ours + theirs
+
+
+def merge_matrix(ours: str, theirs: str) -> str:
+ """Union of table rows from both sides, then a single trailing non-row block
+ (theirs'), for hunks spanning appended matrix rows + the Summary."""
+ def split(block: str):
+ rows, rest, in_rows = [], [], True
+ for line in block.splitlines(True):
+ if in_rows and line.startswith("|"):
+ rows.append(line)
+ else:
+ in_rows = False
+ rest.append(line)
+ return rows, "".join(rest)
+
+ orows, orest = split(ours)
+ trows, trest = split(theirs)
+ seen, rows = set(), []
+ for r in orows + trows:
+ if r not in seen:
+ seen.add(r)
+ rows.append(r)
+ return "".join(rows) + (trest if trest.strip() else orest)
+
+
+def dedupe_matrix_paragraph(text: str) -> str:
+ paras = [p for p in text.split("\n\n") if p.strip()]
+ firsts = [p for p in paras if p.lstrip().startswith("The claim-to-gate matrix stands at")]
+ if len(firsts) > 1:
+ return firsts[0] + "\n\n"
+ return text
+
+
+def main() -> None:
+ path, mode = sys.argv[1], sys.argv[2]
+ p = pathlib.Path(path)
+ t = p.read_text()
+ out, idx, n = [], 0, 0
+ for m in CONFLICT.finditer(t):
+ ours, theirs = m.group(1), m.group(2)
+ if mode == "union":
+ r = ours + theirs
+ elif mode == "theirs":
+ r = theirs
+ elif mode == "roadmap":
+ r = dedupe_matrix_paragraph(theirs)
+ elif mode == "checkboxes":
+ r = merge_checkboxes(ours, theirs)
+ elif mode == "matrix":
+ r = merge_matrix(ours, theirs)
+ else:
+ raise SystemExit(f"unknown mode {mode}")
+ out.append(t[idx : m.start()])
+ out.append(r)
+ idx = m.end()
+ n += 1
+ out.append(t[idx:])
+ res = "".join(out)
+ if "<<<<<<<" in res or ">>>>>>>" in res:
+ raise SystemExit(f"{path}: markers left behind")
+ p.write_text(res)
+ print(f"{path}: {n} hunks -> {mode}")
+
+
+if __name__ == "__main__":
+ main()
diff --cc DEPTH_PROGRESS.md
index 6cc04f333,1219ecc1a..395295860
--- a/DEPTH_PROGRESS.md
+++ b/DEPTH_PROGRESS.md
@@@ -969,61 -702,48 +969,102 @@@ endpoint produces once triage falls thr
`aisoc_cases.alert_ids` is `uuid[]` while the two columns beside it are
JSONB, which a stand-in would have accepted.
- [ ] **9.4** War rooms
+
+- [ ] **9.3** Case depth (parity 5.6 remainder)
+- [~] **9.4** War rooms. **Slack is complete; Teams is built and
+ maintainer-blocked, and the block is stated in the product rather than
+ here.** One action (`POST /war-rooms/cases/{id}`, and a console panel on
+ the case) creates the channel, invites the assignees, posts the case
+ header and records the room; a sweep worker posts the timeline and
+ verdicts as they change; a decision taken in the channel comes back onto
+ the case through a service-token-only route. Idempotency is the design
+ rather than a refinement: every post is keyed by a fingerprint of the
+ *event value*, unique per room, so a retry is a no-op, two replicas cannot
+ double-post, and a case that goes `open → closed → open` posts three times
+ because a re-opened case is news. The sweep derives its events from the
+ case row rather than a cursor, so a missed tick cannot lose one. Opening
+ fails **loudly** (502 carrying the vendor's own sentence) because an
+ analyst pressed a button; syncing fails **soft** and records `last_error`,
+ which the console renders — a room that has silently stopped mirroring is
+ worse than one that was never opened, and "never synced" renders
+ differently from "synced with nothing to say". A chat button reaches four
+ verbs and **`close` is not one of them**. 37 new tests (17 slack-bot, 9
+ teams-bot, 11 API). **Open and stated, not inferred:** creating a Teams
+ channel needs Microsoft Graph with an Entra application registration and
+ admin consent, which no maintainer has provisioned, so that route answers
+ **501 naming all four missing settings** and deliberately does not fall
+ back to the incoming webhook — one shared channel per deployment would put
+ the next incident's traffic in the last one's thread. The Graph calls
+ themselves have therefore never run against a real tenant. Neither has the
+ Slack path against a real workspace: it is covered by 17 tests against a
+ fake client, and no Slack token exists here.
- [ ] **9.5** Hunt to report
+- [x] **9.5** Hunt to report. A natural-language hunt or a retro-hunt
+ sighting becomes a stored report answering what, how, where and when, with
+ ledger citations, readable as JSON or HTML and exportable as PDF through
+ the **same WeasyPrint path and the same unavailability error** the weekly
+ digest uses — one renderer, because two drift and the PDF stops matching
+ the page somebody approved. Three refusals carry the item: a hunt that has
+ never run is refused (a report about an unexecuted query is a statement
+ about the query, not the estate), an **uncited report is refused at
+ persist time** (an uncited report is an assertion), and a section that
+ cannot be derived stays **empty** and renders as "not recorded" rather
+ than as a confident sentence about nothing. The retro-hunt builder
+ computes the gap between when intelligence arrived and when the estate
+ first saw the indicator, which is the number a retro-hunt exists to
+ produce and which a reader would otherwise get by subtracting two
+ timestamps. Graded by `.github/workflows/hunt-report-live.yml` against a
+ real Postgres with three negative controls. 14 new tests. The live suite
+ earned its keep three times on its first run: `alerts` has no singular
+ `affected_host` (it is `affected_hosts`, a JSONB array, and `entities` is
+ what fusion actually writes), `investigation_events` has **no `case_id` at
+ all** so the ledger is only reachable by joining `investigation_runs`, and
+ that table's `case_id` is TEXT rather than UUID so binding a UUID raises
+ in asyncpg instead of coercing. Each would have shipped under a SQLite
+ stand-in, and the middle one would have cited an empty set on every
+ report.
+- [ ] **9.6** Native responder
+ - [~] **9.6** Native responder. **Two of three bullets closed; the third is
+ maintainer-blocked and the block is in the product, not in this note.**
+ *Device build:* `mobile.yml` gains an Android job that runs
+ `expo prebuild` and Gradle `assembleDebug`, asserts the APK really
+ contains `classes.dex` (a green build that produced nothing is a shape
+ this repository has shipped) and uploads it — an APK that installs and
+ runs on a handset. Not store-signable (a Google Play account) and not a
+ *release* build: `:app:createBundleReleaseJsAndAssets` dies inside
+ Metro's asset hashing on the runner, Node 20 fixes it, and
+ `scripts/check_toolchain_pins.py` correctly refuses a second Node version
+ because 20 left security support in April 2026 and no image ships it.
+ Lowering that gate to make this job pass would have been the wrong trade,
+ so the limit is stated instead. iOS
+ compiles **and links** for the simulator on a macOS runner — the Expo
+ config generates a valid Xcode project, the pods resolve and every native
+ module builds. It deliberately does not run Metro's bundle phase, which
+ dies on that runner inside one of expo-router's PNGs for a Metro/Node
+ reason unrelated to this app; the Android job bundles, so that half is
+ still graded. A device build needs an Apple developer account. *Voice:* `apps/mobile/src/lib/voice.ts` takes a recogniser that
+ returns a **string**, so no audio type appears anywhere in the module and
+ there is nothing for a later edit to upload; a device with no offline
+ model is **refused** rather than silently using the platform's cloud
+ recogniser; the transcript is shown before it is sent, because on-device
+ recognition is worse and a security question misheard is a question
+ answered about the wrong host. Gated by `scripts/check_no_audio_egress.py`
+ (self-test: six checks, including that a *comment* saying the app does not
+ record is not mistaken for the app recording), wired into both `ci.yml`
+ unconditionally and `mobile.yml`. *Push:* `services/realtime` sent every
+ subscription through `web-push`, which signs a VAPID JWT and POSTs to the
+ endpoint **URL** — a bare APNs device token is not a URL, so a
+ subscription saved by the mobile app was accepted, counted and failed on
+ every send, which is the worst of the three states because the count
+ looked healthy. `native-push.ts` speaks APNs and FCM HTTP v1 and
+ distinguishes delivered, device-gone and **not-configured**, the last
+ naming each missing variable and never reporting a delivery. 19 new tests
+ (11 realtime, 8 mobile); mobile suite 34 → 42. **Open and stated:** no
+ APNs `.p8` and no FCM service account exist here, so nothing has been
+ delivered to a real handset, and no microphone has exercised the voice
+ path. The README's claim table was inverted rather than deleted — it now
+ has to keep saying that an iPhone *device* build and a real-handset push
+ have **not** happened, and `mobile.yml` fails if it stops.
## Phase 10: Close the books
@@@ -1036,9 -756,7 +1077,10 @@@
|---|---|---|
| 2026-10-11 | 9.3 | Reproduced: the alert merge columns had zero writers outside the ORM model; `grep -rn custom_field services/api` and `grep -rn workload` (excluding workload *identities*) both counted zero; and `grep -rn bulk apps/web/src/components/alerts` found only unrelated comments, so the only bulk verb in the product was `POST /alerts/bulk-close` with no console surface at all. The two-case-table trap named in the brief was already closed by migration 083 and `check_one_case_table.py` reports OK, so `aisoc_cases` is the table this item deepens. Three negative controls recorded in the PR and wired into the live workflow. The live suite found a real defect on its first run: `aisoc_cases.alert_ids` is `uuid[]` while `ioc_ids` and `artifact_ids` beside it are JSONB, and the merge statement had assumed JSONB. |
+| 2026-10-11 | 9.1 | Reproduced: on `origin/main` the greps for `StreamableHTTP` in `services/mcp/src/`, for `aisoc-mcp` in `docker-compose.yml` and `infra/helm/`, for a per-tenant tool grant in `services/api/app/`, and for an `mcp:` audit action all counted **zero**, and `check_mcp_over_http.py` reported seven problems. Built the HTTP transport, the authorize/complete pair, the tenant grant table, the approval queue bound to a call's arguments, and the generated tool catalogue held in parity with the TypeScript registry. Four negative controls recorded in the PR; one of them found a hole in the new gate itself, which reported `http.ts` for reading the process-wide API key when the only mention was a comment saying it deliberately does not — the detector now strips comments before matching, and refuses a `{...cfg}` spread as well, because a spread is correct today and a credential fallback the day somebody reorders the literal. |
+| 2026-10-11 | 9.4 | Reproduced: `grep -rn 'conversations.create|channels.create|war_room'` across every service and the console returned four hits, all prose — a playbook template describing a "Notify war-room" step and an eval fixture quoting the phrase. Both bots could only answer: slack-bot had one outbound route (an approval card) and teams-bot posts to a hand-configured webhook. Built the channel creation, the keyed sweep, the decision route and the console panel. Teams is 501 with its four missing settings named, which is the honest state and the one a stock install is in. |
+| 2026-10-11 | 9.5 | Reproduced: `grep -n hunt` across `app/services/report_builder.py` and `app/api/v1/endpoints/reports.py` returned nothing — the report surface and the hunt surface did not know about each other, so a hunt produced rows and a retro-hunt produced a sightings row and neither produced anything a person could read or forward. Built the four-section report, the citation rule, the two builders and the JSON/HTML/PDF export. Three negative controls in the live workflow: allowing an uncited report, filling an empty section with a sentence, and reporting a hunt that never ran each have to turn the suite red. The live run found three schema facts a stand-in would have hidden, recorded in the item above. |
+ | 2026-10-11 | 9.6 | Reproduced: `grep -c 'eas build|gradlew|xcodebuild|expo prebuild' .github/workflows/mobile.yml` returned **0** and the workflow actively *required* the README to say no device build had happened; `grep -rni 'speech|voice' apps/mobile/src apps/mobile/app` returned **0**; and `services/realtime/src/push.ts` sent every record through `webpush.sendNotification`, so a native token was accepted and could never be delivered to. Built the Android APK job, the simulator job, the on-device voice module and the APNs/FCM sender. Negative control: removing the on-device refusal so the cloud recogniser could be used fails two of the eight voice tests. The README guard was inverted rather than deleted — "we added a build so the honesty check is no longer needed" is how an ungated claim comes back. |
| 2026-10-11 | 9.2 | Reproduced: `knowledge_base.py` searched with `to_tsvector`/`plainto_tsquery` and nothing else; `grep -rn qdrant services/api/app/` found only the migration runner and the tenant-deletion sweep, no recall path; and no memory row anywhere carried an author, a source, a scope, an expiry or a state. Built the store of record, the Qdrant index over it, the conflict state machine, the case-closure writer and the recall scorer. Two negative controls recorded in the PR: restoring last-write-wins on a contradiction fails three tests, and restoring vector truncation fails one. The measurement is negative and is published as such — see D11, which also records that it caught the truncation defect. |
| 2026-10-07 | 0.1 | This file created at base commit `1b8bc2d4`. |
| 2026-10-07 | 0.2 | Every figure re-derived. Two matched exactly (detections, unreachable families); executor arms measured 74 against a captured 73; the cloud/identity/SaaS/code figure measured 395 against a captured 461 on a grouping the plan does not pin, recorded above. Two measurement caveats found: executable and quarantined overlap by 1,724 rules, and the "69% Windows" figure does not reproduce from the index. |