Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

Merge remote-tracking branch 'origin/main' into fix/ingest-auth-and-blocking-scanners

Beenu Arorada6ce438e980
diff --cc CHANGELOG.md
index 343ee55f1,57a1b82bc..585f90987
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@@ -7,94 -7,84 +7,170 @@@ and this project adheres to [Semantic V
## [Unreleased]
+### BREAKING
+
+- **`POST /v1/ingest` and `POST /v1/ingest/batch` now require a credential,
+ and the tenant comes from the credential rather than from the
+ `X-Tenant-ID` header.** The endpoint the README tells users to push
+ telemetry to, and the one `make smoke` exercises, authenticated nothing:
+ it read `X-Tenant-ID`, believed it, and wrote events for whatever tenant
+ the caller named. Anyone who could reach the port could write alerts into
+ any tenant by typing that tenant's UUID. Compose binds the port to
+ `127.0.0.1`, which contains it locally, but ingesting real telemetry means
+ exposing it and nothing said so at the moment the operator did. The
+ comment at `services/ingest/internal/server/server.go` asserting that
+ "`/v1/ingest` is token-authenticated per request" had been false since it
+ was written; it is true now.
+
+ Rather than invent a second mechanism this extends the one the service
+ already had. `/v1/inbox/*` resolves a minted token from
+ `tenant_inbox_tokens` to a tenant, with optional HMAC-SHA256 over
+ `X-Signature`; `/v1/ingest` now resolves the same kind of token, pinned to
+ a new `connector-push` template exactly as `/v1/inbox/cef` is pinned to
+ `cef-syslog` — an inbox token is pasted into a third party's webhook
+ config, so minting one for PagerDuty must not also hand PagerDuty a
+ general ingest credential. A second shape covers AiSOC's own services:
+ `services/connectors` polls on behalf of many tenants and cannot hold a
+ per-tenant token, so it presents `AISOC_SERVICE_TOKEN` and declares the
+ tenant on `X-Tenant-ID`, which is then checked against the tenants table
+ before it becomes a scope.
+
+ `X-Tenant-ID` is still read and is never authority. It is **intersected**
+ with what the credential authorises, so naming an outside tenant narrows
+ to nothing and is refused rather than reaching out — the same
+ intersection-only rule `app/security/tenant_scope.py` already implements
+ for the Python services, ported to Go in
+ `services/ingest/internal/ingestauth`. There is no dev-mode bypass:
+ `AISOC_DEV_MODE` does not reach this path, and an ingest service that
+ cannot verify a credential answers 503 rather than accepting the write.
+
+ **Every deployment must act.** Mint a push token per external pusher with
+ `make ingest-token`, and set `AISOC_SERVICE_TOKEN` on both `ingest` and
+ `connectors` if you use pull connectors — without it, connector polling is
+ refused, loudly, at startup on one side and per request on the other. Full
+ procedure: `apps/docs/docs/operations/ingest-authentication.md`.
+
+### Added
+
+- `make ingest-token` / `python -m app.scripts.mint_ingest_token` mints the
+ push credential for a fresh deployment, the role `bootstrap_admin` plays
+ for the console login. Idempotent — a second run returns the same token
+ rather than leaving a trail of live credentials nobody revokes.
+- `connector-push`, `ai-runtime`, `ai-finding` and `k8s-audit` are now
+ mintable inbox templates. The last three already shipped as YAML and were
+ named in the AI SDK's own setup instructions and in the Kubernetes
+ connector docs, but none was in `ALLOWED_TEMPLATE_IDS`, so the documented
+ setup for the AI-estate feature returned 400 and the templates were
+ unreachable.
+- `scripts/check_inbox_templates.py` compares shipped templates against
+ mintable ids **in both directions**, plus the console catalog, and carries
+ a `--self-test` that injects drift in each direction and requires the gate
+ to catch it. A one-directional check would have printed OK throughout the
+ defect above.
+- The golden pipeline asserts that an unauthenticated push is refused, not
+ only that an authenticated one succeeds — the happy path would go on
+ passing if authentication were removed.
+- `/v1/ingest` now bounds the request body (`INGEST_MAX_BODY_BYTES`, 10 MiB
+ default). `MAX_BATCH_SIZE` only counts events after the payload is
+ decoded, so nothing bounded the decode itself.
+
++
++- **`scripts/generate_corpus_stats.py`** — generates
++ `apps/web/src/data/corpus-stats.json` + `corpusStats.ts` from the compiled
++ engine ruleset, the generated detection truth table, and the marketplace
++ index, reconciling all three against each other and refusing to publish if
++ they disagree. Every landing surface imports the constants, so none can
++ carry its own literal. `--check` is wired into `ci.yml :: python-lint`;
++ `--self-test` hand-edits the artefact and requires the drift to be caught.
++ The artefact keeps `executable` and `onDisk`/`quarantined` as separate
++ fields, and the UI leads with the executable count.
++- **`scripts/check_alert_reduction_claims.py`** — prose cannot be generated
++ the way a count can, so the retraction is gated instead. No published
++ surface may assert the legacy harness runs the production grouping; any
++ surface quoting 75.3 % must carry the retraction; the `alert_reduction`
++ suite card may not be declared `kind: 'measurement'`; and every surface
++ publishing the real figure must quote `PUBLISHED_REDUCTION_PCT`, a new
++ constant in `services/fusion/tests/test_alert_reduction_real.py` that the
++ test asserts against its own measurement — so the chain from measurement
++ to published prose has no hand-copied link. A paragraph that dates or
++ negates the claim is exempt, so the retraction can quote the wording it
++ retracts.
++
+### Changed
+
+- **gitleaks and Trivy are blocking gates.** All five scanners in
+ `security.yml` carried `continue-on-error`, so a new finding never redded
+ a pull request. Gitleaks found 35 findings on its first real run; every
+ one was triaged and recorded in `.gitleaksignore` with a written reason —
+ detection rule-id maps, osquery and redaction test fixtures, placeholder
+ PEMs, a published AWS example key, a FIPS-197 test vector, and a
+ `YOUR_TOKEN` doc placeholder. Four of the six entries the allow-list
+ already held had rotted through line drift and were suppressing nothing.
+ Trivy reports zero fixable HIGH/CRITICAL and blocks at that.
+
+ Semgrep (92 findings, 35 at ERROR), checkov (91 failed checks) and tfsec
+ (32, of which 7 CRITICAL) stay in observe mode. The counts are now written
+ into the workflow so the remaining work is visible rather than implied.
+
+ The secret-scan job also fails when the scanner cannot install, instead of
+ reporting clean, and proves it is reading the tree before trusting a clean
+ result: with the allow-list moved aside the triaged findings must
+ reappear. "Clean" and "did not run" previously looked identical.
+
++
++- **`readme_gates.py` covers the governance documents.** Its `FIGURE_DOCS`
++ list named one compliance page, which is why `ROADMAP.md` drifted with CI
++ green. `ROADMAP.md` and `RELEASES.md` are now on the list, the matrix
++ **row total** is compared as well as the GATED/PARTIAL split, and a figure
++ the prose explicitly dates ("the count at that time") is exempt so history
++ need not be rewritten.
++- **`check_scoreboard.py --check` verifies `agent_version` against
++ `VERSION`.** `--refresh` already stamped it; nothing compared it, and the
++ freshness gate reads only the date — which a refresh keeps current — so a
++ row could be two days old and still labelled two majors behind.
++
+ ### Fixed
+
+ - **A retracted benchmark figure was still badged "Real measurement".**
+ `apps/docs/docs/benchmark.md` withdrew the 75.3 % alert-reduction claim —
+ the harness that produced it groups on four tiers of `(rule_id, host,
+ user)` while the shipping `RawAlert.correlation_key()` groups on
+ `{tenant}:{entity}:{tactic}`, so it does not merely re-implement fusion's
+ grouping, it implements *different* grouping. The retraction reached one
+ surface of five. `BenchmarkResults.tsx` rendered a green **"Real
+ measurement"** badge on `0.753` with a blurb claiming the harness used the
+ production rules, "same logic"; `benchmarks/alert-reduction.md`
+ (`sidebar_position: 1`) called it a "faithful in-harness re-implementation"
+ and headlined 75.3 %; `ComparisonTable.tsx` qualified it as "measured on
+ fixed noisy stream". Two more were found while gating the invariant:
+ `benchmark-methodology.md`, and `benchmark.md` itself, which re-asserted
+ the claim in its own intro blockquote. All five now carry the wording
+ `benchmark.md` already uses. The comparison table quotes **33.3 %**, the
+ figure measured against the key the product actually runs.
+ - **The landing page published three different wrong corpus counts.**
+ "6,998 detections" in four places (the tree indexes 7,016, of which 5,937
+ are quarantined and the engine loads **833**), "57 plugins" (77), "7,117
+ community items" (7,155), and `218 rules across 5 categories` on the
+ contributor leaderboard (833 across 6). The 6,998 figure also quoted the
+ imported corpus as the detection capability, presenting quarantined rules
+ as executable.
+ - **Governance figures had gone stale.** `ROADMAP.md` published "136 rows —
+ 128 GATED / 8 PARTIAL" against a matrix holding 139 / 131, in the same
+ sentence that tells the reader to recount with the script "rather than
+ trusting a figure quoted in prose — this line has gone stale before".
+ `CLAIM_TO_GATE_MATRIX.md` carried a stale executable-rule figure (939) in
+ a row note. The scoreboard's newest substrate row was labelled `v8.1.1`
+ while `VERSION` read `10.0.0`; it is refreshed from a fresh deterministic
+ run (0.97, unchanged) and now carries the tree's version. The scoreboard
+ keeps its three rows, all `substrate: true`, and no live-LLM row was
+ invented.
+ - **The "Design partners" block on the landing page was removed** rather
+ than updated. Four dashed "Partner A–D" chips under the caption
+ "Reference partners onboarding through Q2 2026": placeholders rather than
+ fabricated logos, but four of them assert a partner count nothing in the
+ repository supports, and the window closed in June 2026 while still being
+ advertised as upcoming.
+
-### Added
-
-- **`scripts/generate_corpus_stats.py`** — generates
- `apps/web/src/data/corpus-stats.json` + `corpusStats.ts` from the compiled
- engine ruleset, the generated detection truth table, and the marketplace
- index, reconciling all three against each other and refusing to publish if
- they disagree. Every landing surface imports the constants, so none can
- carry its own literal. `--check` is wired into `ci.yml :: python-lint`;
- `--self-test` hand-edits the artefact and requires the drift to be caught.
- The artefact keeps `executable` and `onDisk`/`quarantined` as separate
- fields, and the UI leads with the executable count.
-- **`scripts/check_alert_reduction_claims.py`** — prose cannot be generated
- the way a count can, so the retraction is gated instead. No published
- surface may assert the legacy harness runs the production grouping; any
- surface quoting 75.3 % must carry the retraction; the `alert_reduction`
- suite card may not be declared `kind: 'measurement'`; and every surface
- publishing the real figure must quote `PUBLISHED_REDUCTION_PCT`, a new
- constant in `services/fusion/tests/test_alert_reduction_real.py` that the
- test asserts against its own measurement — so the chain from measurement
- to published prose has no hand-copied link. A paragraph that dates or
- negates the claim is exempt, so the retraction can quote the wording it
- retracts.
-
-### Changed
-
-- **`readme_gates.py` covers the governance documents.** Its `FIGURE_DOCS`
- list named one compliance page, which is why `ROADMAP.md` drifted with CI
- green. `ROADMAP.md` and `RELEASES.md` are now on the list, the matrix
- **row total** is compared as well as the GATED/PARTIAL split, and a figure
- the prose explicitly dates ("the count at that time") is exempt so history
- need not be rewritten.
-- **`check_scoreboard.py --check` verifies `agent_version` against
- `VERSION`.** `--refresh` already stamped it; nothing compared it, and the
- freshness gate reads only the date — which a refresh keeps current — so a
- row could be two days old and still labelled two majors behind.
-
## [10.0.0] — 2026-09-25
**Two ways for a control to be absent: not written, or written and not
diff --cc RELEASES.md
index c3b7d29fd,fb9af3fe7..f6c1d49c6
--- a/RELEASES.md
+++ b/RELEASES.md
@@@ -2,7 -2,7 +2,7 @@@
This file mirrors what used to live in the "What's new" section of [`README.md`](README.md). The complete, machine-readable inventory (with file paths, env-var diffs, and per-release test counts) lives in [`CHANGELOG.md`](CHANGELOG.md).
- > **TL;DR for first-time visitors:** AiSOC is on `v10.0.0`, released 2026-09-25 — a release about controls that were written and not reachable. Row-level security covered 92 tables and filtered nothing, because every service connected to Postgres as a superuser; fifty-eight routes carried no authentication at all; UEBA reported healthy and could not write an anomaly; AI triage was wired to a gateway both of its resolvers ignored. **It is a major because upgrading requires action** — services now connect as a DML-only `aisoc_app` role. Every product claim is backed by a failing CI test (claim-to-gate matrix: 139 rows — 131 GATED / 8 PARTIAL / 0 NO GATE). The latest GitHub release with notes and downloads: <https://github.com/beenuar/AiSOC/releases/latest>.
-> **TL;DR for first-time visitors:** AiSOC is on `v10.0.0`, released 2026-09-25 — a release about controls that were written and not reachable. Row-level security covered 92 tables and filtered nothing, because every service connected to Postgres as a superuser; fifty-eight routes carried no authentication at all; UEBA reported healthy and could not write an anomaly; AI triage was wired to a gateway both of its resolvers ignored. **It is a major because upgrading requires action** — services now connect as a DML-only `aisoc_app` role. Every product claim is backed by a failing CI test (claim-to-gate matrix: 142 rows — 134 GATED / 8 PARTIAL / 0 NO GATE). The latest GitHub release with notes and downloads: <https://github.com/beenuar/AiSOC/releases/latest>.
++> **TL;DR for first-time visitors:** AiSOC is on `v10.0.0`, released 2026-09-25 — a release about controls that were written and not reachable. Row-level security covered 92 tables and filtered nothing, because every service connected to Postgres as a superuser; fifty-eight routes carried no authentication at all; UEBA reported healthy and could not write an anomaly; AI triage was wired to a gateway both of its resolvers ignored. **It is a major because upgrading requires action** — services now connect as a DML-only `aisoc_app` role. Every product claim is backed by a failing CI test (claim-to-gate matrix: 143 rows — 135 GATED / 8 PARTIAL / 0 NO GATE). The latest GitHub release with notes and downloads: <https://github.com/beenuar/AiSOC/releases/latest>.
---
diff --cc ROADMAP.md
index 1ace56766,65cffc439..1ecb73498
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@@ -44,7 -44,7 +44,7 @@@ A proof-first, security-first program t
- **Phase D (breadth):** D1 eight connectors (QRadar/Exabeam/Securonix/Devo/Netskope/Windows-Sysmon/Zeek-Suricata/syslog-CEF) · D2 AI/LLM-usage audit connector + `llm-*` detections + hot/cold lake tiering · D3 live-vendor mock-server smoke.
- **Phase E (prove it):** E1 CI-gated benchmark scoreboard tied to a deterministic live-agent MITRE-accuracy run.
- The claim-to-gate matrix stands at **136 rows — 128 GATED / 8 PARTIAL / 0 NO GATE** — **every product claim is backed by a failing test** and the ratchet (`MAX_NO_GATE=0`) forbids any regression. The 8 remaining PARTIAL rows are honest, named deferrals; none was relabelled without building the gate it names. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).
-The claim-to-gate matrix stands at **142 rows — 134 GATED / 8 PARTIAL / 0 NO GATE** — **every product claim is backed by a failing test** and the ratchet (`MAX_NO_GATE=0`) forbids any regression. The 8 remaining PARTIAL rows are honest, named deferrals; none was relabelled without building the gate it names. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).
++The claim-to-gate matrix stands at **143 rows — 135 GATED / 8 PARTIAL / 0 NO GATE** — **every product claim is backed by a failing test** and the ratchet (`MAX_NO_GATE=0`) forbids any regression. The 8 remaining PARTIAL rows are honest, named deferrals; none was relabelled without building the gate it names. Count the table rows with `python3 scripts/check_claim_gate_matrix.py` rather than trusting a figure quoted in prose — this line has gone stale before. The six lettered deferrals are scoped in [`docs/audit/DEFERRED_SUBPHASES.md`](docs/audit/DEFERRED_SUBPHASES.md).
## v4.0 — Shipped
diff --cc apps/docs/docs/compliance/evidence-pack.md
index 612fa9993,0a3d63370..47a5fde06
--- a/apps/docs/docs/compliance/evidence-pack.md
+++ b/apps/docs/docs/compliance/evidence-pack.md
@@@ -33,7 -33,7 +33,7 @@@ and says so
| Prompt-injection resistance | 32-payload adversarial corpus with a recall floor and a false-positive ceiling | `services/agents/tests/adversarial/` |
| Claim-to-gate traceability | Every product claim mapped to the CI job that fails when it stops being true | `docs/audit/CLAIM_TO_GATE_MATRIX.md` |
- The last row is the one worth reading first. It is the honest index: 132 rows
-The last row is the one worth reading first. It is the honest index: 134 rows
++The last row is the one worth reading first. It is the honest index: 135 rows
`GATED`, 8 `PARTIAL` with the specific gap named, and a ratchet that refuses
to let a row sit at `NO GATE`.
diff --cc docs/audit/CLAIM_TO_GATE_MATRIX.md
index 46da48e5a,a3c3a0df0..c140bb4df
--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md
+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md
@@@ -151,7 -153,7 +154,7 @@@ Statuses: `GATED` (a CI job fails when
## Summary
- - GATED: 132
-- GATED: 134
++- GATED: 135
- PARTIAL: 8
- NO GATE: 0 (**every claim is backed by a failing test.** The last NO GATE — the weekly benchmark scoreboard running live against `main` — closed in Phase E1: `scripts/check_scoreboard.py` ties the published scoreboard to a deterministic per-PR live-agent MITRE-accuracy run, while the funded weekly `wet-eval.yml` appends the LLM-tier rows. The remaining PARTIAL rows are honest, named deferrals — each states the specific gap and what closes it — not unproven claims.)