Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
docs: catch the portal, changelog and counts up with what merged since v20.0.0 (#1474)
Beenu Arora40d97bee7085
diff --git a/AGENTS.md b/AGENTS.mdindex 821e7ec94..0ad08eec9 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -94,7 +94,7 @@ of these regenerates:| Figure | Current | Command || --- | --- | --- |-| Executable detection rules | 2,511 of 6,991 on disk | `python3 scripts/generate_corpus_stats.py --check` |+| Executable detection rules | 2,535 of 7,015 on disk | `python3 scripts/generate_corpus_stats.py --check` || Rules that cannot fire | 2 | `python3 scripts/check_detection_fields.py` || Connectors | 87 | `python3 scripts/generate_connector_count.py --check` || Services per profile | CORE 16, full 21 | `python3 scripts/check_profile_service_counts.py` |diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 176d71601..40f245bc3 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -9,6 +9,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Security+- `@vitest/mocker` moves to 4.1.12 in all three npm lockfiles (root+ workspaces, `services/realtime`, `services/mcp/cursor-extension`), closing+ CVE-2026-112949 and CVE-2026-112950 (#1468). The bump rode the Oct-22+ Dependabot wave rather than waiting for it to be noticed as a security+ item.- The complete 2026-10 Pillow advisory set — published to OSV in waves acrossthe day, 28 CVEs against 12.3.0 at last count, every fix 13.0.0-only(verified per-CVE via OSV) — is recorded against `services/threatintel`'s@@ -18,6 +23,109 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0release by 2027-01-15. `agents` and `api` already carry the patched 13.0.0and are unaffected.+### Added++- The API answers unauthenticated `GET /api/v1/health` (liveness: the process+ serves HTTP, nothing else claimed) and `GET /api/v1/health/ready`+ (readiness: can it do its job, named per dependency) (#1471, part of+ #1446). `database` is a **block** dependency — without Postgres the status+ goes 503 and a load balancer should stop sending traffic — while `cache` is+ **degrade**: Redis loss is reported (`cache: false`) at status 200, because+ failing readiness over a cache would pull the API out of the balancer to+ fix a cache. Every probe is hard-capped well inside a prober's budget, so+ "down and we say so" cannot decay into "down and we say nothing".+ `services/threatintel`'s `/health` now names its Qdrant leg the same way+ (`vector_store`, degrade).+- The chart `stable` channel is guarded by **two version lines, and a+ dispatchable repair tool exists** (#1469, closing #1440). v20.0.0 — chart+ 7.9.0, not a chart major — carried chart `stable` across an *application*+ major while every image `stable` tag correctly stayed on v19, because the+ chart guard read only the chart's own line; the decision now also compares+ the channel's `appVersion` major, read from the registry, and fails closed+ when the incoming chart's appVersion cannot be read.+ `.github/workflows/chart-channel.yml` is the operator decision the guard+ points at: it re-points `stable` at a named, already-published chart+ version, byte-identical, in either direction — it was dispatched to roll+ the channel back to 7.8.0 (appVersion v19.0.0), where it stays until a+ deliberate promotion. `tests/test_release_channel_tags.py` replays whole+ release ladders over the lifted decision blocks for both lines.++### Fixed++- **A connector checkpoint no longer advances past events ingest rejected**+ (#1441, closing #1438; contributed by+ [meshreyy](https://github.com/meshreyy)). Ingest deliberately reports+ per-event normalization failures as HTTP 200 with `accepted`/`rejected`+ counts, so HTTP success alone never proved advancing was safe: a poll whose+ batch was partially rejected moved the cursor anyway, and the rejected+ events were lost permanently — a data-loss class, invisible because the+ poll reported success. The scheduler now persists the staged cursor only+ when every submitted event was accepted, withholds it otherwise with a+ warning naming the counts, and re-fetches the page on the next overlapping+ poll, where the sink deduplicates what was already accepted.+- The live-agent eval dispatches every incident on **one event loop**, and an+ incident whose investigation fails outright scores **0 with the reason+ recorded** (#1472, closing #1466). Per-incident `asyncio.run` left+ langchain-openai's cached async HTTP client bound to a dead loop, so+ recon's model call failed on every incident after the first and the agent+ silently ran its deterministic heuristics — while failed incidents were+ dropped from the sample, inflating the mean and shrinking the sample below+ the gate's own size requirement. Failures now participate as 0 in the mean+ with reasons in the report's `groundedness.zero_scored` block, so the+ sample always equals the dispatched prefix. The floor was re-derived under+ the fixed harness and **stays 0.40**, below the lowest fixed-harness run+ (0.44) with a deliberately thinner margin: one failed investigation costs+ the mean about 0.05 under zero-fill, so a model failing three or more of+ ten investigations dips below the floor and reds the job — which is the+ signal that outcome should produce. The pre-fix runs stay recorded in+ `live_agent_floor.json` for provenance, labelled as pre-fix.+- The chaos store-outage legs grade **real behaviour instead of artifacts of+ their own probe contract** (#1471, part 1 of #1446). The previous shape+ probed one API URL for all six stores: `/api/v1/health/ready` did not exist+ (404 → every key "reads None"), the ingest URL pointed at the enrichment+ service's port, the recovery poll re-read the same 404 forever, and the+ ClickHouse/Neo4j legs graded containers the `core` profile never started.+ Each leg now names the readiness surface that owns the store and the+ dependents the expectation declares; the redis and qdrant legs pass+ end-to-end. The postgres leg stays red **by design** — fusion and ingest do+ not yet export the store keys the contract reads — with the finding naming+ the service; the follow-ups live on #1446, which stays open.+- The perf harness records the payload bytes the producer actually sent,+ closing 17 days of scheduled `perf.yml` failures in which+ `throughput_claims.py --check` required `payload_bytes` as context for+ `ingest_eps` and nothing had ever written it (#1444).+- `react-native-screens` is held at the version Expo SDK 54's compatibility+ table names, and Dependabot is taught the hold so it stops re-proposing the+ incompatible major (#1443).+- The monthly adoption snapshot lands on the unprotected `adoption-data`+ branch, needing no owner credential (#1470, closing #1445): `main` is+ branch-protected with no bypass actors, so the workflow's direct push+ failed (GH006) and a `GITHUB_TOKEN` PR can never merge because events that+ token creates do not trigger the required `pull_request` checks.+ `docs/adoption/README.md` on `main` is now a pointer; the ledger and its+ full history live in `snapshot.json` on the data branch.+- Three CI legs stopped failing for reasons that were never about the code+ under test: the eval matrix's non-floor leg passes the checker's own+ deliberate-comparison flag instead of tripping its different-model refusal+ (#1465); jobs that start PostgreSQL fall back between the two official+ registries when one throttles (#1434, contributed by+ [meshreyy](https://github.com/meshreyy)); and the nightly/live suites seed+ the state the product actually requires — `scripts/ensure_env.py` writes+ the `.env` the stack reads, and the scheduler live suite admits its stub+ vendor through the documented `AISOC_SSRF_ALLOW_HOSTS` path rather than a+ test-only bypass (#1432).++### Changed++- `fastapi` moves to 0.146.0 repo-wide — every one of the 32 declaration+ sites and all 13 poetry locks in one move, so the version CI tests is the+ version every image ships (#1473).+- The Oct-22 Dependabot waves are folded: python (`sqlalchemy` 2.1.6,+ `strawberry-graphql` 0.336.2, `pydantic-settings` 2.16.1, `langchain-core`+ 1.6.13, `langgraph-sdk` 0.4.8, `tenacity` 9.2.1 across api, actions and+ agents, #1467) and npm (root workspaces, realtime and the MCP cursor+ extension, #1468).+## [20.0.0] - 2026-10-21### BREAKINGdiff --git a/README.md b/README.mdindex 2f3ccb1b6..3b684173c 100644--- a/README.md+++ b/README.md@@ -18,7 +18,7 @@## What AiSOC doesTelemetry arrives from your security tools. AiSOC normalizes it, runs the 2535 executable rules of-its 6991-rule library, groups what fires into incidents, investigates each one with an AI agent whose+its 7015-rule library, groups what fires into incidents, investigates each one with an AI agent whoseevery prompt and tool call is recorded, and proposes an action. New threat intelligence re-sweeps thehistory you already collected, and a human approves before anything reaches a vendor.@@ -112,7 +112,7 @@ rule is harder to find than a missing one: of the 2,005 Splunk rules bundled her## How it worksIngest normalizes to a common shape and stamps an **activity projection** on every event — actor, action, resource, location, outcome, with `actor.kind` read from each vendor's own documented field rather than inferred — then Kafka carries it.-Then fusion runs 2535 executable detection rules, of 6991 on disk, plus 76 sliding-window rules (73 count across events; 3 match an ordered sequence), applies **your tenant's own tuning** on top — the disables, floors and suppressions the console writes, so a rule you turned off actually stops firing — and decides what becomes an alert.+Then fusion runs 2535 executable detection rules, of 7015 on disk, plus 76 sliding-window rules (73 count across events; 3 match an ordered sequence), applies **your tenant's own tuning** on top — the disables, floors and suppressions the console writes, so a rule you turned off actually stops firing — and decides what becomes an alert.Correlation groups related alerts, an agent investigates and writes its reasoning to the Investigation Ledger, and a playbook may start from the result. Separately, new threat intelligence sweeps the lake for sightings you already collected, and a hypothesis becomes a hunt without anyone writing a query — the model fills a closed schema and every value is bound as a parameter, so it cannot express a query at all.**A playbook triggered by an alert previews before it acts.** Three switches must agree — the@@ -191,7 +191,7 @@ Agents triage alerts and investigate incidents. What they can and cannot do:| Capability | Status | Tested | Production ready ||---|---|---|---|| Ingest → detect → correlate → alert | Stable | E2E + unit | Yes |-| Detection engine (2535 executable rules) of 6991 | Stable | Replay proof | Yes |+| Detection engine (2535 executable rules) of 7015 | Stable | Replay proof | Yes || Alert correlation into incidents | Stable | Unit | Yes || REST API + web console | Stable | Unit + integration | Yes || AI triage + Investigation Ledger | Stable | Live Postgres ledger + a PR-gated local-model agent run. No hosted provider has been exercised | Yes, copilot mode |diff --git a/apps/docs/docs/architecture.md b/apps/docs/docs/architecture.mdindex 3caa82179..3a621a6a6 100644--- a/apps/docs/docs/architecture.md+++ b/apps/docs/docs/architecture.md@@ -124,14 +124,14 @@ block the front door, and the reason an event is not lost when fusion restarts.## 4 · Fusion evaluates detections[`services/fusion`](https://github.com/beenuar/AiSOC/tree/main/services/fusion)-(container port 8003) consumes the spine and runs **2,511 executable detection+(container port 8003) consumes the spine and runs **2,535 executable detectionrules** against each event.The rules the engine runs are compiled into[`app/data/detection_ruleset.json`](https://github.com/beenuar/AiSOC/blob/main/services/fusion/app/data/detection_ruleset.json)and its imported counterpart beside it. The YAML under `detections/` is a-*projection* of those, not the engine's input — 6,991 rules are on disk and-2,511 load, which is why the corpus figure and the executable figure are+*projection* of those, not the engine's input — 7,015 rules are on disk and+2,535 load, which is why the corpus figure and the executable figure arepublished separately. See[the detection truth table](https://github.com/beenuar/AiSOC/blob/main/docs/detections/truth-table.md).diff --git a/apps/docs/docs/benchmark-methodology.md b/apps/docs/docs/benchmark-methodology.mdindex 7af5ee3e9..bc84fa3e5 100644--- a/apps/docs/docs/benchmark-methodology.md+++ b/apps/docs/docs/benchmark-methodology.md@@ -139,10 +139,35 @@ network round trips.The weekly job pushes its results into the same `eval-results` branch as thesubstrate run, so historical trend lines stay in one place.-Cross-link: [`wet-eval.yml`](https://github.com/beenuar/AiSOC/blob/main/.github/workflows/wet-eval.yml)-is added by **T5.5** in the v8.0 plan. Until that lands, the per-template-latency / token / USD cells on the benchmark page are placeholders rather-than imputed numbers.+Two semantics of the live dispatch are load-bearing for reading any report it+produces:++- **One event loop per run.** Every incident is dispatched on the same asyncio+ loop. The harness previously ran `asyncio.run` per incident, which left+ langchain-openai's cached async HTTP client bound to a dead loop — so the+ model call failed on every incident after the first and the agent silently+ ran its deterministic heuristics while the run still reported itself live.+ A report's `llm_calls_placed` is the check: it is published beside every+ mean, and a run that placed no call at all is refused.+- **Failures score 0, with reasons.** An incident whose investigation fails+ outright (raises, or returns `status=failed`) participates in the mean as+ **0**, with the reason recorded in the report's `groundedness.zero_scored`+ block — so the sample size always equals the dispatched prefix. Dropping+ failures is survivor bias: it inflates the mean and shrinks the sample+ below the gate's own size requirement, which is exactly how the pre-fix+ harness overstated itself.++The groundedness floor asserted by+[`live-agent-eval.yml`](https://github.com/beenuar/AiSOC/blob/main/.github/workflows/live-agent-eval.yml)+is **0.40** over a deterministic 10-incident slice, enforced by+`scripts/check_live_agent_floor.py` against the runs recorded in+[`live_agent_floor.json`](https://github.com/beenuar/AiSOC/blob/main/services/agents/tests/eval_data/live_agent_floor.json)+— derivation history included, pre-fix runs kept and labelled. The floor sits+below the lowest fixed-harness run observed (0.44), and the margin is+deliberately thin: one failed investigation costs the mean about 0.05 under+zero-fill, so a model failing three or more of ten investigations reds the+job, which is the signal that outcome should produce. The loader refuses a+floor above its own recorded evidence.## 4. Suites and what each one actually measuresdiff --git a/apps/docs/docs/concepts/detections.md b/apps/docs/docs/concepts/detections.mdindex a3896ffc5..0398684d5 100644--- a/apps/docs/docs/concepts/detections.md+++ b/apps/docs/docs/concepts/detections.md@@ -27,7 +27,7 @@ coverage page all do now, and it is why they agree.:::-The library holds **6,991 ATT&CK-mapped rules on disk, of which 2,511 are+The library holds **7,015 ATT&CK-mapped rules on disk, of which 2,535 areexecutable** — the count the detection engine actually loads. Those two numberstravel together everywhere, because a library figure presented on its own readsas coverage and is not.diff --git a/apps/docs/docs/glossary.md b/apps/docs/docs/glossary.mdindex a932d98c2..0a49ac933 100644--- a/apps/docs/docs/glossary.md+++ b/apps/docs/docs/glossary.md@@ -27,8 +27,8 @@ API (EDR, SIEM, cloud, IAM, SaaS, VCS, network) and pushes normalized OCSFrecords into the ingest spine. See the [connector catalog](./connectors).**Detection rule** — A query (Sigma / YARA / KQL / EQL) that fires an-alert when matching events are seen. AiSOC's library holds 6,991 rules on-disk, of which 2,511 are **executable** (see below) — 833 native and 1,770+alert when matching events are seen. AiSOC's library holds 7,015 rules on+disk, of which 2,535 are **executable** (see below) — 765 native and 1,770translated from SigmaHQ. Every imported rule is tagged with provenance.**Investigation Ledger** — Append-only, replayable record of everydiff --git a/apps/docs/docs/intro.md b/apps/docs/docs/intro.mdindex e4cb1216a..339b28065 100644--- a/apps/docs/docs/intro.md+++ b/apps/docs/docs/intro.md@@ -68,7 +68,7 @@ for the full list.- **UEBA** — per-user Welford online baseline, Z-score anomaly scoring, and Kafka-integrated anomaly publishing.- **Honeytokens** — HMAC-SHA256 signed deceptive credentials (URL, file, AWS key, email) with first-touch webhook alerting.- **Purple Team** — Atomic Red Team YAML parser + Caldera executor, ATT&CK coverage heatmap, tabletop sessions.-- **Detection engineering** — **2,511 executable rules of a 6,991-rule library** (833 native plus 1,770 translated from SigmaHQ; the remainder is imported from Splunk Security Content, Chronicle and MITRE CAR and kept for provenance and coverage mapping, since this repository has no evaluator for those query languages). The two figures always travel together, because a library figure on its own reads as coverage. Executable means the rule was replayed through its real connector and the real engine and [watched to fire](detections/coverage.md) — not that it detects an attack. Rules run over OpenSearch + ClickHouse, with YARA, KQL / EQL and a community catalog offering one-click install.+- **Detection engineering** — **2,535 executable rules of a 7,015-rule library** (765 native plus 1,770 translated from SigmaHQ; the remainder is imported from Splunk Security Content, Chronicle and MITRE CAR and kept for provenance and coverage mapping, since this repository has no evaluator for those query languages). The two figures always travel together, because a library figure on its own reads as coverage. Executable means the rule was replayed through its real connector and the real engine and [watched to fire](detections/coverage.md) — not that it detects an attack. Rules run over OpenSearch + ClickHouse, with YARA, KQL / EQL and a community catalog offering one-click install.- **Detection-as-Code (DAC)** — propose, review, eval-gate, and promote detection rules via `/api/v1/detection-proposals`. Every proposal carries an eval result; candidates that regress MITRE accuracy are blocked from promotion.- **Detection confidence** — each fused alert carries a `high / medium / low` confidence label and an ordered evidence chain. The label is derived from weighted factors, not manually assigned.- **Detection drift monitoring** — scheduled ATT&CK coverage snapshots enable "delta vs. last week" tracking on the MITRE heatmap.diff --git a/apps/docs/docs/operations/ha-deployment.md b/apps/docs/docs/operations/ha-deployment.mdindex 034c14247..020aaea0d 100644--- a/apps/docs/docs/operations/ha-deployment.md+++ b/apps/docs/docs/operations/ha-deployment.md@@ -159,6 +159,38 @@ election) cannot leave a pod running with nothing consuming.If `/readyz` is 503 for a fusion replica, look at the broker before you look atthe pod.+### The API's own probes: liveness and dependency readiness++The API additionally answers two unauthenticated endpoints — a load balancer+or probe has no credential, and the bodies carry only booleans about the+service's own dependencies:++- `GET /api/v1/health` — **liveness**: the process serves HTTP, and nothing+ else is claimed. It deliberately touches no store, because a liveness probe+ that fails when a dependency is down invites the orchestrator to restart a+ healthy process, converting an outage into an outage plus a restart storm.+- `GET /api/v1/health/ready` — **readiness**: can this service do its job+ right now, named per dependency. `database` is a *block* dependency: the+ API is the system of record's front door, so without PostgreSQL the status+ goes **503** and a balancer should stop sending traffic here — the body+ still names every dependency, because failing *with the reason* is the+ point. `cache` is a *degrade* dependency: Redis loss is reported+ (`cache: false`) while the status stays **200**, because failing readiness+ over a cache would pull the API out of the balancer to fix a cache.++Each probe is hard-capped (1.5 s) well inside a prober's read budget, so a+store that is down produces "down and we say so" rather than a probe that+waits out a connect timeout and says nothing. `services/threatintel`'s+`/health` names its Qdrant leg the same way: `vector_store` is degrade — IOC+ingestion continues without it and enrichment responses say the similarity+leg was skipped.++These are distinct from `/readyz` above: `/readyz` answers for *consumer+subscriptions* (is anything consuming), the dependency probes answer for+*stores* (can a request be served). The weekly chaos store-outage grader+(`scripts/chaos/store_outage.py`) reads exactly these keys while a store is+down, so their contract is exercised, not just published.+## MigrationsThe chart does not run them. Apply the chain as the **owner** role, withdiff --git a/apps/docs/docs/operations/release-policy.md b/apps/docs/docs/operations/release-policy.mdindex 5fdbfdebc..2447059ee 100644--- a/apps/docs/docs/operations/release-policy.md+++ b/apps/docs/docs/operations/release-policy.md@@ -125,12 +125,31 @@ from major 8 until you choose it. Quote it, or your shell will glob.:::-The chart follows the **chart's** major rather than the application's — the-break it protects an operator from is a values-schema break, and the chart-versions independently of `appVersion`. A chart `stable` tag is published for-OCI tooling that can address tags directly (`docker buildx imagetools-inspect`, `oras`), and a chart major stops its promotion until a maintainer-dispatches with `promote_stable: true`, exactly as for images.+The chart `stable` tag is guarded by **two version lines, not one**. The+chart's own major announces a values-schema break and stops the channel, as+for images. But the chart's values default every unpinned image tag to+`Chart.AppVersion`, so the channel also carries operators across+*application* majors — which is how a chart minor (7.9.0, `appVersion`+v20.0.0) once moved chart `stable` onto a breaking application release while+every image `stable` tag correctly stayed on v19. The decision now also+compares the channel's `appVersion` major, read from the registry like+everything else about the channel, and **fails closed** when the incoming+chart's `appVersion` cannot be read: refusing is recoverable, a wrong+crossing is not. A chart `stable` tag is published for OCI tooling that can+address tags directly (`docker buildx imagetools inspect`, `oras`), and+either major boundary stops promotion until a maintainer dispatches with+`promote_stable: true`, exactly as for images.++### Repairing a channel that is pointing at the wrong version++`chart-channel.yml` (workflow dispatch) is the deliberate human action the+guard points at. It re-points the chart `stable` tag at a named,+**already-published** chart version — byte-identical manifest, either+direction, with before/after evidence in the log. It never republishes a+chart *version*: `helm push` overwrites silently and an overwritten version+tag is unrecoverable, so the repair tool only moves the mutable channel tag.+Until it existed, the only way `stable` could move was another release tag+push, which is no tool at all when the channel is already wrong.Pin an explicit version in production; a constraint is for the case where youwould otherwise have pinned nothing at all.diff --git a/docs/architecture/README.md b/docs/architecture/README.mdindex 83362320c..a9179e688 100644--- a/docs/architecture/README.md+++ b/docs/architecture/README.md@@ -93,8 +93,8 @@ cannot drift from what the Go binary actually loads.independent: anything can consume the spine without ingest knowing about it.**4. Fusion consumes it and evaluates detections.**-[`services/fusion`](../../services/fusion) runs the 2,511 executable rules-against the event — 741 native and 1,770 imported Sigma rules, each of which+[`services/fusion`](../../services/fusion) runs the 2,535 executable rules+against the event — 765 native and 1,770 imported Sigma rules, each of whichwas replayed through its real connector and this engine and watched to firebefore it was allowed into the compiled ruleset. Separately it decides whether the event is *promotable* —a vendor finding (OCSF category 2) or anything at severity ≥ high becomes an@@ -133,7 +133,7 @@ alerts while the console showed it disabled — the worst shape a defect canhave, because it tells the operator the problem is solved.It is an overlay rather than a per-tenant ruleset: the *difference* applied-over one shared corpus, not N copies of 2,511 rules rebuilt whenever anyone+over one shared corpus, not N copies of 2,535 rules rebuilt whenever anyoneedits anything. Suppression is applied **after** the match so the dropped hitis logged with the tuning, its author and its reason — "no alert" with noexplanation is indistinguishable from a rule that simply did not match. A@@ -434,7 +434,7 @@ flowchart TDC --> E["OCSF-shaped envelope"]D --> EE --> F[("Kafka aisoc.raw_events")]- F --> G["Detection engine<br/>2,511 executable rules<br/>+ 72 windowed"]+ F --> G["Detection engine<br/>2,535 executable rules<br/>+ 76 windowed"]F --> H{"Promotable?<br/>category 2, or severity >= high"}G -->|rule fires| I["Alert"]H -->|yes| Idiff --git a/docs/audit/CLAIM_TO_GATE_MATRIX.md b/docs/audit/CLAIM_TO_GATE_MATRIX.mdindex b45e29085..0aeac33e0 100644--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md@@ -31,7 +31,7 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`| Runs entirely on your infrastructure, and evidence sent to a hosted model is pseudonymized first | README L63 | `ci.yml :: python-test` (agents) runs `test_privacy_redactor.py` (zero raw PII survives) | full | GATED (redaction gated; the CLI now has no default telemetry endpoint and `telemetry.test.ts` pins that an enabled-but-unconfigured run makes zero network calls; **v9.0 adds the Helm egress default-deny** — `infra/helm/aisoc/templates/networkpolicy.yaml`, gated by `helm.yml` which asserts it denies by default, excludes `169.254.169.254/32` from every allowed CIDR, and stays opt-in because a CNI that does not enforce NetworkPolicy ignores it silently; **the platform-wide half landed with `check_default_egress.py`** — every service's declared settings defaults are read and a public host that no air-gap guard covers fails, with each exemption naming the guard that justifies it so deleting the guard withdraws the exemption — plus `tests/test_no_default_egress.py`, which patches the socket layer and drives each service's import and startup with a blank environment, covering 7 of 13 services in CI and naming the missing package for each of the other 6 rather than skipping silently. `services/purple-team` declared an ATT&CK CDN URL nothing in the service read, and it was deleted rather than exempted. **the container half landed with `check_container_egress.py`** — images built from the commit under test start on a docker network created with `--internal`, with a DNS sinkhole as their only resolver, and every name they ask for is classified with the same predicate the services enforce air-gap policy with. Observation, not only prevention, because a block alone cannot tell "made no call" from "made a call that failed". Two controls, because zero observations is what a correct run and a blind probe both look like and the API's unconfigured startup legitimately records zero — it dials Postgres on loopback, which needs no DNS: a canary wired with the identical network and `--dns` flags must be seen by the sinkhole, and a container that emitted no log line and asked for no name is a finding rather than a pass. `container-egress.yml` also builds an image that deliberately resolves a public name and requires the gate to fail on it. One limit stated rather than implied: a dial straight to a public IP literal asks no DNS, so it is prevented by the `--internal` network and statically checked by `check_default_egress.py`, but not observed. **Narrowed 2026-10-01**: the claim used to read "no data exfiltration", which a reader takes to cover a *configured* hosted model as well. It does not. When an operator configures a hosted provider, evidence reaches it unredacted: `services/agents/app/privacy/redactor.py` exists and is unit-tested, and **no LLM call site invokes it**, so `test_privacy_redactor.py` gates a function rather than a path. **Restored 2026-10-01 by parity 2.4.** Hosted calls now go through the pseudonymizer at the contract layer, which is the one place all sixteen agent call sites already pass through, and the row rests on `test_egress_pseudonymization.py`, which drives `safe_ainvoke` and inspects what a fake provider actually received rather than calling the redactor directly. Testing the path rather than the function immediately found a real gap the unit test could not: a bare username in prose (`running as priya.raghavan`) was sent in the clear, because usernames were only redacted in the `DOMAIN\user` form or under a user-ish key. The file also pins the documented trade-off in the other direction, that public IOCs are deliberately **not** redacted, and carries a test that disables the control and requires the leak assertion to fail, so it cannot pass vacuously) | — || Detection-as-Code rejects candidates that regress MITRE accuracy | README L170 | `ci.yml :: python-test` (`test_detection_eval.py` — candidate `rule_body` run through the real engine vs its own positive/negative fixtures; approval requires it) + `p1-eval` w2-dac baseline | core | GATED | - || 800+ native detection rules | README L78, L170 | `validate-detections.yml` (strict fixture replay) | core | GATED | - |-| 6000+ imported detection rules | README L78 | `validate-detections.yml` (parse/provenance + `detection_truth_table.py --check`); README cites the executable figure (2511) not the on-disk figure (6991) for coverage | core | GATED | - |+| 6000+ imported detection rules | README L78 | `validate-detections.yml` (parse/provenance + `detection_truth_table.py --check`); README cites the executable figure (2535) not the on-disk figure (7015) for coverage | core | GATED | - || L0-L4 automation maturity gates every action | README L171 | `ci.yml` autonomy-policy drift test + `test_autonomy_safety.py` (dry-run-default, rollback-capability contract, verification-mandate, scorecard) + `test_governed_dispatch.py` (Phase B2 — `decide()` wired into the live-actions `/dispatch` path: above-tier→dry-run, dry-run-off→queued, L0→blocked, executor never invoked when queued/blocked) | core | GATED | - || Hunt-as-Code + `/hunt` | README L172 | `ci.yml :: p1-eval` (`hunt_corpus`) | core | GATED | - || Weekly benchmark scoreboard runs live against `main` | README L173 | `ci.yml` agents job (Phase E1 — `scripts/check_scoreboard.py` runs the deterministic live-agent MITRE-accuracy eval over the 200-incident corpus and fails if the published `scoreboard.json` substrate row drifts > 0.02, the schema is invalid, or a substrate row is mislabelled) + `wet-eval.yml` (weekly funded LLM rows) | core | GATED | - |@@ -100,7 +100,7 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`| Documentation links on the front door resolve | README / SUPPORT / CONTRIBUTING | `link-check.yml :: front-door` (hard gate over the five files a new visitor reads; `:: docs-tree` observes the wider documentation tree so pre-existing rot is burned down deliberately rather than blocking unrelated PRs) | core | GATED | - || The CLI makes no network call unless an endpoint is configured | `packages/aisoc-lite/TELEMETRY.md` | `ci.yml` aisoc-lite vitest (`telemetry.test.ts` — there is no default endpoint, and an enabled-but-unconfigured run is asserted to make zero `fetch` calls) | core | GATED | - || The visual assets the README embeds exist | README (console tiles + walkthrough) | `readme-gates.yml :: static` (`gate_demo_asset_references` covers `apps/web/public/screenshots/` as well as `demo/`, so a renamed or deleted tile fails instead of rendering as a broken image) | core | GATED | - |-| Groundedness: the agent asserts no indicator absent from the evidence it was given | README (groundedness axis) | `live-agent-eval.yml` (dispatches the real four-agent LangGraph pipeline against a locally-served model, scores every incident with `app.confidence.groundedness`, and `scripts/check_live_agent_floor.py` fails the run below the floor recorded in `services/agents/tests/eval_data/live_agent_floor.json`) + `ci.yml :: Python — Tests` (`test_groundedness_eval.py`, `test_groundedness_gate.py`) | core | GATED (three sub-claims, all now covered. The scorer is proven by `test_groundedness_eval.py`; `test_groundedness_gate.py` drives the real triage worker and proves an ungrounded `false_positive` at 0.97 confidence is demoted to `needs_review` rather than auto-closed; and the third — that *the live agent* asserts no indicator absent from its evidence — is now measured, because the workflow runs. It could not before: its live path imported `InvestigatorAgent`, a class that exists nowhere in `services/agents`, so the first dispatch failed in 92 seconds and every later one would have. **The floor is 0.40 on a deterministic 10-incident slice, derived from runs rather than chosen**: seven local runs returned 0.5561 to four decimal places (the agents decode greedily at temperature 0, so it is a point mass on one machine), while three GitHub-runner runs returned 0.5821, 0.5329 and 0.5933 — a spread of 0.0604, because a quantised model decoding on a different kernel diverges at the margins. The floor sits 0.1329 below the lowest run observed anywhere, about 2.2 times that spread, and still fails anything a quarter worse. The declaration carries every run behind it and the loader refuses a floor above its own evidence. **Scope, stated because it is narrow**: this grades a locally-served `qwen2.5:0.5b`, no hosted provider has ever been exercised, and the job is scheduled plus on-demand rather than per-PR because a threshold tight enough to be useful on a PR would be tight enough to flake on a runner nobody controls) | — |+| Groundedness: the agent asserts no indicator absent from the evidence it was given | README (groundedness axis) | `live-agent-eval.yml` (dispatches the real four-agent LangGraph pipeline against a locally-served model, scores every incident with `app.confidence.groundedness`, and `scripts/check_live_agent_floor.py` fails the run below the floor recorded in `services/agents/tests/eval_data/live_agent_floor.json`) + `ci.yml :: Python — Tests` (`test_groundedness_eval.py`, `test_groundedness_gate.py`) | core | GATED (three sub-claims, all now covered. The scorer is proven by `test_groundedness_eval.py`; `test_groundedness_gate.py` drives the real triage worker and proves an ungrounded `false_positive` at 0.97 confidence is demoted to `needs_review` rather than auto-closed; and the third — that *the live agent* asserts no indicator absent from its evidence — is now measured, because the workflow runs. It could not before: its live path imported `InvestigatorAgent`, a class that exists nowhere in `services/agents`, so the first dispatch failed in 92 seconds and every later one would have. **The floor is 0.40 on a deterministic 10-incident slice, derived from runs rather than chosen — and re-derived after the harness itself was found to flatter (#1466/#1472)**: the original ten runs (seven local at 0.5561, three GitHub runners spread 0.5329–0.5933) were taken on a harness whose per-incident `asyncio.run` left langchain-openai's cached async client on a dead loop, so recon fell back to heuristics after the first incident, and whose scorer dropped failed incidents from the sample — survivor bias in the flattering direction. Under the fixed harness (one loop per run; a failed investigation scores 0 with the reason recorded in `groundedness.zero_scored`, so the sample always equals the dispatched prefix) the observed means are 0.51, 0.445 and 0.44 locally plus 0.5234 re-expressed from the two identical CI runs that motivated the fix. The floor stays 0.40, below the lowest fixed-harness run (0.44); the thinner margin is deliberate — one outright failure costs the mean about 0.05 under zero-fill, so three or more failures in ten dip below the floor and red the job, which is the signal that outcome should produce. Pre-fix runs stay recorded in the declaration, labelled pre-fix, for provenance; the loader refuses a floor above its own recorded evidence. **Scope, stated because it is narrow**: this grades a locally-served `qwen2.5:0.5b`, no hosted provider has ever been exercised, and the job is scheduled plus on-demand rather than per-PR because a threshold tight enough to be useful on a PR would be tight enough to flake on a runner nobody controls) | — || A live-agent eval can run with no funded provider key | (implied by the public eval harness) | `live-agent-eval.yml` (Ollama-hosted local model reached through `OPENAI_BASE_URL`; `--wet-limit` keeps it inside a CI budget, `--wet-require-live` stops a degraded run publishing substrate numbers as live-agent performance, and `check_live_agent_floor.py` asserts the result) | core | GATED (**observed**, not designed-for. [Run 36517390566](https://github.com/beenuar/AiSOC/actions/runs/36517390566) on `main` dispatched the real agent over ten incidents, placed 62 LLM calls, measured a groundedness of 0.5933 and cleared the floor — with no secret of any kind in the repository. This row previously said 0 runs, and the reason was not scheduling: the live path named a class that does not exist, and two further defects would each have published a number nobody measured — MITRE accuracy read a corpus key that is not there, so it would have reported `0.0000`, and a self-hosted model was priced against the gpt-4o rate card. A third shape slipped past `--wet-require-live` entirely: with the model pins unset every agent caught its provider error and used its deterministic path, and the run reported 0.8050 at 0.11s per investigation — so the harness now refuses a run that placed no LLM call, and the report carries `llm_calls_placed` beside the mean. Runtime is a real constraint and is handled rather than hidden: an unbounded completion ran to 40,960 tokens and twenty minutes on a runner, which is why `app/investigator/limits.py` bounds it — and a timeout reports as `cancelled`, not `failed`, so a run that measured nothing would not have looked like a failure) | — || Response actions are verified against the vendor, not assumed from a 200 | README (verified response) | `ci.yml` actions matrix (`test_dispatch_verification.py` — asserts the dispatcher reaches the verifier on the live path, that an action which reported success with the effect absent is downgraded to FAILED, and that `unverified` is never upgraded to a confirmation) | core | GATED | - || Host isolation is confirmed by reading real containment state | (implied by verified response) | `ci.yml` actions matrix (`test_verification.py` — a resolvable-but-uncontained host must score FAILED; `containment_pending` must not count as verified) | core | GATED | - |diff --git a/docs/audit/REPOSITORY_REALITY.md b/docs/audit/REPOSITORY_REALITY.mdindex 937c18d14..f93b5302f 100644--- a/docs/audit/REPOSITORY_REALITY.md+++ b/docs/audit/REPOSITORY_REALITY.md@@ -123,9 +123,9 @@ SSRF allow-list entry, and is commented as such.| Item | Count | Status ||---|---|---|-| Rules the engine loads | **2511** | **WORKING** — each was replayed through its real connector and the real engine and watched to fire |-| Detection rules on disk | **6991** | the rest are imports in languages with **no evaluator** here (Splunk SPL, Chronicle YARA-L, CAR pseudocode); the engine never reads them |-| YAML files under `detections/` | **7022** | 25 are response playbooks (`detections/playbooks/`) and 6 are Sigma correlation files (`detections/sigma-correlations/`); neither is a stateless rule |+| Rules the engine loads | **2535** | **WORKING** — each was replayed through its real connector and the real engine and watched to fire |+| Detection rules on disk | **7015** | the rest are imports in languages with **no evaluator** here (Splunk SPL, Chronicle YARA-L, CAR pseudocode); the engine never reads them |+| YAML files under `detections/` | **7046** | 25 are response playbooks (`detections/playbooks/`) and 6 are Sigma correlation files (`detections/sigma-correlations/`); neither is a stateless rule || Hunts under `hunts/` | **68** | **WORKING** — each replayed against a synthetic scenario and observed to fire, and each graded in the other direction too. Not a claim a hunt fires on a given deployment's telemetry, which depends on whether its connectors emit the fields the hunt names. The negative half of that grading was satisfiable without testing anything until Phase 8.4: a negative drawn from a different log source never fires on any hunt, so `check_hunt_scenarios.py` now requires it to differ from its positive in exactly one indicator field. It found 8 violations on first run, 3 in the 5 hunts that predate the phase |The gap matters and has been published both ways in the past. `make stats`diff --git a/docs/contributing/detection-translation-drive.md b/docs/contributing/detection-translation-drive.mdindex 8f6b180aa..e6ac41dce 100644--- a/docs/contributing/detection-translation-drive.md+++ b/docs/contributing/detection-translation-drive.md@@ -18,10 +18,10 @@ which is generated by `scripts/detection_truth_table.py` and gated by| car (imported) | 99 | 0 || community | 1 | 0 |-**2,511 of 6,991 rules fire today.** The other 4,462 are on disk for-provenance and ATT&CK coverage mapping — 4,213 of them under `_quarantine/`+**2,535 of 7,015 rules fire today.** The other 4,480 are on disk for+provenance and ATT&CK coverage mapping — 4,213 of them quarantined on importbecause their body is an upstream query language the engine does not execute,-and 175 carrying `enabled: false` with a reason.+and 267 carrying `enabled: false` with a reason.This table is copied from the generated file rather than counted by hand, andit has still gone stale once: it read `native 869/869` and `sigma 3132/77`diff --git a/docs/contributing/strategies-and-actions.md b/docs/contributing/strategies-and-actions.mdindex b3f7bf83e..885245ce6 100644--- a/docs/contributing/strategies-and-actions.md+++ b/docs/contributing/strategies-and-actions.md@@ -14,7 +14,7 @@ defend had been published for months. The connector catalogue went the sameway — 35 of 84 had no documentation, and the sidebar listed 34 pages against96 files, so most of what existed could not be found.-The detection figure is 2,511 now, and how it moved is the part worth keeping.+The detection figure is 2,535 now, and how it moved is the part worth keeping.It was not closed by writing rules. Someone asked why the existing ones couldnot fire and found a single cause — Windows events nest their payload onelevel below the namespace the matcher reads, so the two most-used fields indiff --git a/docs/design/landing-page-component-recipes.md b/docs/design/landing-page-component-recipes.mdindex 8a2aeb65c..a8bb869ec 100644--- a/docs/design/landing-page-component-recipes.md+++ b/docs/design/landing-page-component-recipes.md@@ -97,7 +97,7 @@- **Primitives:**- Aceternity **Glowing Effect** (`/components/glowing-effect`) — wraps each card border. `glow={true}`, `proximity={64}`, `inactiveZone={0.4}`.- Aceternity **Bento Grid** — used loosely (flat 2 × 2, equal heights, no skew).- - MagicUI **Number Ticker** — stat lines. P1 `value={EXECUTABLE_DETECTION_COUNT}` (from `apps/web/src/data/corpusStats.ts`, currently 2511 — do not hard-code it; the literal `6998` sat here while the corpus moved, and `2603` sat here after that), P2 `value={17}`, P3 `value={4}`, P4 `value={6}`.+ - MagicUI **Number Ticker** — stat lines. P1 `value={EXECUTABLE_DETECTION_COUNT}` (from `apps/web/src/data/corpusStats.ts`, currently 2535 — do not hard-code it; the literal `6998` sat here while the corpus moved, and `2603` sat here after that), P2 `value={17}`, P3 `value={4}`, P4 `value={6}`.- **Custom:** 24 × 24 brand glyphs (`open-source`, `graph`, `agentic`, `deploy-anywhere`). Active borders: P2 → `landing.gradient.pillars`, P3 → `landing.accent.ember`, P1 / P4 → `brand-500/30`.- **Anatomy:** `Pillars` → `PillarsHeading`, `PillarsGrid` (4 × `PillarCard`), each → `PillarIcon`, `PillarTitle`, `PillarBody`, `PillarStat`, `PillarLink`.- **States:** Default `landing.shadow.1`, `surface-border`. Hover → Glowing Effect on; lift `-2 px`. Focus → existing `:focus-visible` 2 px `brand-500` ring. <768 stacks. Reduced-motion → Glowing `disabled={true}`; lift instant.