Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
Merge remote-tracking branch 'origin/main' into fix/ingest-auth-and-blocking-scanners
Beenu Arorada6ce438e980
diff --cc CHANGELOG.mdindex 343ee55f1,57a1b82bc..585f90987--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -7,94 -7,84 +7,170 @@@ and this project adheres to [Semantic V## [Unreleased]+### BREAKING++- **`POST /v1/ingest` and `POST /v1/ingest/batch` now require a credential,+ and the tenant comes from the credential rather than from the+ `X-Tenant-ID` header.** The endpoint the README tells users to push+ telemetry to, and the one `make smoke` exercises, authenticated nothing:+ it read `X-Tenant-ID`, believed it, and wrote events for whatever tenant+ the caller named. Anyone who could reach the port could write alerts into+ any tenant by typing that tenant's UUID. Compose binds the port to+ `127.0.0.1`, which contains it locally, but ingesting real telemetry means+ exposing it and nothing said so at the moment the operator did. The+ comment at `services/ingest/internal/server/server.go` asserting that+ "`/v1/ingest` is token-authenticated per request" had been false since it+ was written; it is true now.++ Rather than invent a second mechanism this extends the one the service+ already had. `/v1/inbox/*` resolves a minted token from+ `tenant_inbox_tokens` to a tenant, with optional HMAC-SHA256 over+ `X-Signature`; `/v1/ingest` now resolves the same kind of token, pinned to+ a new `connector-push` template exactly as `/v1/inbox/cef` is pinned to+ `cef-syslog` — an inbox token is pasted into a third party's webhook+ config, so minting one for PagerDuty must not also hand PagerDuty a+ general ingest credential. A second shape covers AiSOC's own services:+ `services/connectors` polls on behalf of many tenants and cannot hold a+ per-tenant token, so it presents `AISOC_SERVICE_TOKEN` and declares the+ tenant on `X-Tenant-ID`, which is then checked against the tenants table+ before it becomes a scope.++ `X-Tenant-ID` is still read and is never authority. It is **intersected**+ with what the credential authorises, so naming an outside tenant narrows+ to nothing and is refused rather than reaching out — the same+ intersection-only rule `app/security/tenant_scope.py` already implements+ for the Python services, ported to Go in+ `services/ingest/internal/ingestauth`. There is no dev-mode bypass:+ `AISOC_DEV_MODE` does not reach this path, and an ingest service that+ cannot verify a credential answers 503 rather than accepting the write.++ **Every deployment must act.** Mint a push token per external pusher with+ `make ingest-token`, and set `AISOC_SERVICE_TOKEN` on both `ingest` and+ `connectors` if you use pull connectors — without it, connector polling is+ refused, loudly, at startup on one side and per request on the other. Full+ procedure: `apps/docs/docs/operations/ingest-authentication.md`.++### Added++- `make ingest-token` / `python -m app.scripts.mint_ingest_token` mints the+ push credential for a fresh deployment, the role `bootstrap_admin` plays+ for the console login. Idempotent — a second run returns the same token+ rather than leaving a trail of live credentials nobody revokes.+- `connector-push`, `ai-runtime`, `ai-finding` and `k8s-audit` are now+ mintable inbox templates. The last three already shipped as YAML and were+ named in the AI SDK's own setup instructions and in the Kubernetes+ connector docs, but none was in `ALLOWED_TEMPLATE_IDS`, so the documented+ setup for the AI-estate feature returned 400 and the templates were+ unreachable.+- `scripts/check_inbox_templates.py` compares shipped templates against+ mintable ids **in both directions**, plus the console catalog, and carries+ a `--self-test` that injects drift in each direction and requires the gate+ to catch it. A one-directional check would have printed OK throughout the+ defect above.+- The golden pipeline asserts that an unauthenticated push is refused, not+ only that an authenticated one succeeds — the happy path would go on+ passing if authentication were removed.+- `/v1/ingest` now bounds the request body (`INGEST_MAX_BODY_BYTES`, 10 MiB+ default). `MAX_BATCH_SIZE` only counts events after the payload is+ decoded, so nothing bounded the decode itself.+++++- **`scripts/generate_corpus_stats.py`** — generates++ `apps/web/src/data/corpus-stats.json` + `corpusStats.ts` from the compiled++ engine ruleset, the generated detection truth table, and the marketplace++ index, reconciling all three against each other and refusing to publish if++ they disagree. Every landing surface imports the constants, so none can++ carry its own literal. `--check` is wired into `ci.yml :: python-lint`;++ `--self-test` hand-edits the artefact and requires the drift to be caught.++ The artefact keeps `executable` and `onDisk`/`quarantined` as separate++ fields, and the UI leads with the executable count.++- **`scripts/check_alert_reduction_claims.py`** — prose cannot be generated++ the way a count can, so the retraction is gated instead. No published++ surface may assert the legacy harness runs the production grouping; any++ surface quoting 75.3 % must carry the retraction; the `alert_reduction`++ suite card may not be declared `kind: 'measurement'`; and every surface++ publishing the real figure must quote `PUBLISHED_REDUCTION_PCT`, a new++ constant in `services/fusion/tests/test_alert_reduction_real.py` that the++ test asserts against its own measurement — so the chain from measurement++ to published prose has no hand-copied link. A paragraph that dates or++ negates the claim is exempt, so the retraction can quote the wording it++ retracts.+++### Changed++- **gitleaks and Trivy are blocking gates.** All five scanners in+ `security.yml` carried `continue-on-error`, so a new finding never redded+ a pull request. Gitleaks found 35 findings on its first real run; every+ one was triaged and recorded in `.gitleaksignore` with a written reason —+ detection rule-id maps, osquery and redaction test fixtures, placeholder+ PEMs, a published AWS example key, a FIPS-197 test vector, and a+ `YOUR_TOKEN` doc placeholder. Four of the six entries the allow-list+ already held had rotted through line drift and were suppressing nothing.+ Trivy reports zero fixable HIGH/CRITICAL and blocks at that.++ Semgrep (92 findings, 35 at ERROR), checkov (91 failed checks) and tfsec+ (32, of which 7 CRITICAL) stay in observe mode. The counts are now written+ into the workflow so the remaining work is visible rather than implied.++ The secret-scan job also fails when the scanner cannot install, instead of+ reporting clean, and proves it is reading the tree before trusting a clean+ result: with the allow-list moved aside the triaged findings must+ reappear. "Clean" and "did not run" previously looked identical.+++++- **`readme_gates.py` covers the governance documents.** Its `FIGURE_DOCS`++ list named one compliance page, which is why `ROADMAP.md` drifted with CI++ green. `ROADMAP.md` and `RELEASES.md` are now on the list, the matrix++ **row total** is compared as well as the GATED/PARTIAL split, and a figure++ the prose explicitly dates ("the count at that time") is exempt so history++ need not be rewritten.++- **`check_scoreboard.py --check` verifies `agent_version` against++ `VERSION`.** `--refresh` already stamped it; nothing compared it, and the++ freshness gate reads only the date — which a refresh keeps current — so a++ row could be two days old and still labelled two majors behind.+++ ### Fixed++ - **A retracted benchmark figure was still badged "Real measurement".**+ `apps/docs/docs/benchmark.md` withdrew the 75.3 % alert-reduction claim —+ the harness that produced it groups on four tiers of `(rule_id, host,+ user)` while the shipping `RawAlert.correlation_key()` groups on+ `{tenant}:{entity}:{tactic}`, so it does not merely re-implement fusion's+ grouping, it implements *different* grouping. The retraction reached one+ surface of five. `BenchmarkResults.tsx` rendered a green **"Real+ measurement"** badge on `0.753` with a blurb claiming the harness used the+ production rules, "same logic"; `benchmarks/alert-reduction.md`+ (`sidebar_position: 1`) called it a "faithful in-harness re-implementation"+ and headlined 75.3 %; `ComparisonTable.tsx` qualified it as "measured on+ fixed noisy stream". Two more were found while gating the invariant:+ `benchmark-methodology.md`, and `benchmark.md` itself, which re-asserted+ the claim in its own intro blockquote. All five now carry the wording+ `benchmark.md` already uses. The comparison table quotes **33.3 %**, the+ figure measured against the key the product actually runs.+ - **The landing page published three different wrong corpus counts.**+ "6,998 detections" in four places (the tree indexes 7,016, of which 5,937+ are quarantined and the engine loads **833**), "57 plugins" (77), "7,117+ community items" (7,155), and `218 rules across 5 categories` on the+ contributor leaderboard (833 across 6). The 6,998 figure also quoted the+ imported corpus as the detection capability, presenting quarantined rules+ as executable.+ - **Governance figures had gone stale.** `ROADMAP.md` published "136 rows —+ 128 GATED / 8 PARTIAL" against a matrix holding 139 / 131, in the same+ sentence that tells the reader to recount with the script "rather than+ trusting a figure quoted in prose — this line has gone stale before".+ `CLAIM_TO_GATE_MATRIX.md` carried a stale executable-rule figure (939) in+ a row note. The scoreboard's newest substrate row was labelled `v8.1.1`+ while `VERSION` read `10.0.0`; it is refreshed from a fresh deterministic+ run (0.97, unchanged) and now carries the tree's version. The scoreboard+ keeps its three rows, all `substrate: true`, and no live-LLM row was+ invented.+ - **The "Design partners" block on the landing page was removed** rather+ than updated. Four dashed "Partner A–D" chips under the caption+ "Reference partners onboarding through Q2 2026": placeholders rather than+ fabricated logos, but four of them assert a partner count nothing in the+ repository supports, and the window closed in June 2026 while still being+ advertised as upcoming.+-### Added--- **`scripts/generate_corpus_stats.py`** — generates- `apps/web/src/data/corpus-stats.json` + `corpusStats.ts` from the compiled- engine ruleset, the generated detection truth table, and the marketplace- index, reconciling all three against each other and refusing to publish if- they disagree. Every landing surface imports the constants, so none can- carry its own literal. `--check` is wired into `ci.yml :: python-lint`;- `--self-test` hand-edits the artefact and requires the drift to be caught.- The artefact keeps `executable` and `onDisk`/`quarantined` as separate- fields, and the UI leads with the executable count.-- **`scripts/check_alert_reduction_claims.py`** — prose cannot be generated- the way a count can, so the retraction is gated instead. No published- surface may assert the legacy harness runs the production grouping; any- surface quoting 75.3 % must carry the retraction; the `alert_reduction`- suite card may not be declared `kind: 'measurement'`; and every surface- publishing the real figure must quote `PUBLISHED_REDUCTION_PCT`, a new- constant in `services/fusion/tests/test_alert_reduction_real.py` that the- test asserts against its own measurement — so the chain from measurement- to published prose has no hand-copied link. A paragraph that dates or- negates the claim is exempt, so the retraction can quote the wording it- retracts.--### Changed--- **`readme_gates.py` covers the governance documents.** Its `FIGURE_DOCS`- list named one compliance page, which is why `ROADMAP.md` drifted with CI- green. `ROADMAP.md` and `RELEASES.md` are now on the list, the matrix- **row total** is compared as well as the GATED/PARTIAL split, and a figure- the prose explicitly dates ("the count at that time") is exempt so history- need not be rewritten.-- **`check_scoreboard.py --check` verifies `agent_version` against- `VERSION`.** `--refresh` already stamped it; nothing compared it, and the- freshness gate reads only the date — which a refresh keeps current — so a- row could be two days old and still labelled two majors behind.-## [10.0.0] — 2026-09-25**Two ways for a control to be absent: not written, or written and notdiff --cc RELEASES.mdindex c3b7d29fd,fb9af3fe7..f6c1d49c6--- a/RELEASES.md+++ b/RELEASES.md@@@ -2,7 -2,7 +2,7 @@@This file mirrors what used to live in the "What's new" section of [`README.md`](README.md). The complete, machine-readable inventory (with file paths, env-var diffs, and per-release test counts) lives in [`CHANGELOG.md`](CHANGELOG.md).- > **TL;DR for first-time visitors:** AiSOC is on `v10.0.0`, released 2026-09-25 — a release about controls that were written and not reachable. Row-level security covered 92 tables and filtered nothing, because every service connected to Postgres as a superuser; fifty-eight routes carried no authentication at all; UEBA reported healthy and could not write an anomaly; AI triage was wired to a gateway both of its resolvers ignored. **It is a major because upgrading requires action** — services now connect as a DML-only `aisoc_app` role. Every product claim is backed by a failing CI test (claim-to-gate matrix: 139 rows — 131 GATED / 8 PARTIAL / 0 NO GATE). The latest GitHub release with notes and downloads: <https://github.com/beenuar/AiSOC/releases/latest>.-> **TL;DR for first-time visitors:** AiSOC is on `v10.0.0`, released 2026-09-25 — a release about controls that were written and not reachable. Row-level security covered 92 tables and filtered nothing, because every service connected to Postgres as a superuser; fifty-eight routes carried no authentication at all; UEBA reported healthy and could not write an anomaly; AI triage was wired to a gateway both of its resolvers ignored. **It is a major because upgrading requires action** — services now connect as a DML-only `aisoc_app` role. Every product claim is backed by a failing CI test (claim-to-gate matrix: 142 rows — 134 GATED / 8 PARTIAL / 0 NO GATE). The latest GitHub release with notes and downloads: <https://github.com/beenuar/AiSOC/releases/latest>.++> **TL;DR for first-time visitors:** AiSOC is on `v10.0.0`, released 2026-09-25 — a release about controls that were written and not reachable. Row-level security covered 92 tables and filtered nothing, because every service connected to Postgres as a superuser; fifty-eight routes carried no authentication at all; UEBA reported healthy and could not write an anomaly; AI triage was wired to a gateway both of its resolvers ignored. **It is a major because upgrading requires action** — services now connect as a DML-only `aisoc_app` role. Every product claim is backed by a failing CI test (claim-to-gate matrix: 143 rows — 135 GATED / 8 PARTIAL / 0 NO GATE). The latest GitHub release with notes and downloads: <https://github.com/beenuar/AiSOC/releases/latest>.---diff --cc ROADMAP.mdindex 1ace56766,65cffc439..1ecb73498--- a/ROADMAP.md+++ b/ROADMAP.md@@@ -44,7 -44,7 +44,7 @@@ A proof-first, security-first program t- **Phase D (breadth):** D1 eight connectors (QRadar/Exabeam/Securonix/Devo/Netskope/Windows-Sysmon/Zeek-Suricata/syslog-CEF) · D2 AI/LLM-usage audit connector + `llm-*` detections + hot/cold lake tiering · D3 live-vendor mock-server smoke.- **Phase E (prove it):** E1 CI-gated benchmark scoreboard tied to a deterministic live-agent MITRE-accuracy run.- The claim-to-gate matrix stands at **136 rows — 128 GATED / 8 PARTIAL / 0 NO GATE** — **every product claim is backed by a failing test** and the ratchet (`MAX_NO_GATE=0`) forbids any regression. The 8 remaining PARTIAL rows are honest, named deferrals; none was relabelled without building the gate it names. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).-The claim-to-gate matrix stands at **142 rows — 134 GATED / 8 PARTIAL / 0 NO GATE** — **every product claim is backed by a failing test** and the ratchet (`MAX_NO_GATE=0`) forbids any regression. The 8 remaining PARTIAL rows are honest, named deferrals; none was relabelled without building the gate it names. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).++The claim-to-gate matrix stands at **143 rows — 135 GATED / 8 PARTIAL / 0 NO GATE** — **every product claim is backed by a failing test** and the ratchet (`MAX_NO_GATE=0`) forbids any regression. The 8 remaining PARTIAL rows are honest, named deferrals; none was relabelled without building the gate it names. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).## v4.0 — Shippeddiff --cc apps/docs/docs/compliance/evidence-pack.mdindex 612fa9993,0a3d63370..47a5fde06--- a/apps/docs/docs/compliance/evidence-pack.md+++ b/apps/docs/docs/compliance/evidence-pack.md@@@ -33,7 -33,7 +33,7 @@@ and says so| Prompt-injection resistance | 32-payload adversarial corpus with a recall floor and a false-positive ceiling | `services/agents/tests/adversarial/` || Claim-to-gate traceability | Every product claim mapped to the CI job that fails when it stops being true | `docs/audit/CLAIM_TO_GATE_MATRIX.md` |- The last row is the one worth reading first. It is the honest index: 132 rows-The last row is the one worth reading first. It is the honest index: 134 rows++The last row is the one worth reading first. It is the honest index: 135 rows`GATED`, 8 `PARTIAL` with the specific gap named, and a ratchet that refusesto let a row sit at `NO GATE`.diff --cc docs/audit/CLAIM_TO_GATE_MATRIX.mdindex 46da48e5a,a3c3a0df0..c140bb4df--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md@@@ -151,7 -153,7 +154,7 @@@ Statuses: `GATED` (a CI job fails when## Summary- - GATED: 132-- GATED: 134++- GATED: 135- PARTIAL: 8- NO GATE: 0 (**every claim is backed by a failing test.** The last NO GATE — the weekly benchmark scoreboard running live against `main` — closed in Phase E1: `scripts/check_scoreboard.py` ties the published scoreboard to a deterministic per-PR live-agent MITRE-accuracy run, while the funded weekly `wet-eval.yml` appends the LLM-tier rows. The remaining PARTIAL rows are honest, named deferrals — each states the specific gap and what closes it — not unproven claims.)