Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
feat(maturity): define Stable, gate the table, and earn it for graph and UEBA (#1104)
Beenu Arora180b6cbb7ec4
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex b594af8a5..c1fc7a548 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -804,6 +804,35 @@ jobs:run: |python3 scripts/check_profile_service_counts.py --self-testpython3 scripts/check_profile_service_counts.py+ # The Project maturity table published a status for fifteen+ # capabilities and nothing checked any of them: no definition of+ # Stable existed anywhere, no gate parsed the table, and the labels+ # appeared in no other file. Promoting a capability was a one-line+ # edit, which is the claim shape Phase 1.1 retracted twelve of.+ #+ # Every Stable row must now name its evidence and that evidence must+ # hold the four properties in docs/audit/MATURITY_DEFINITION.md. The+ # gate runs in both directions, so an entry for a row that is no+ # longer Stable fails too.+ # `playbooks/packs/v1/` holds 62 production playbooks and the agents+ # image shipped **none** of them: the Dockerfile's build context is+ # `services/agents`, a context cannot reach outside itself, and the+ # loader's pack branch is guarded by `exists()` — so an absent pack+ # was silently the same as an empty one and a deployment's library+ # was 62 entries short with no error anywhere.+ #+ # They are now vendored inside the package, where `COPY app/ ./app/`+ # picks them up, the same arrangement fusion uses for its compiled+ # ruleset. Two copies drift, so this compares them in both+ # directions.+ - name: The agents image ships the playbook pack+ run: |+ python3 scripts/sync_vendored_playbook_packs.py --self-test+ python3 scripts/sync_vendored_playbook_packs.py --check+ - name: Every Stable capability has earned the label+ run: |+ python3 scripts/check_maturity_table.py --self-test+ python3 scripts/check_maturity_table.py# `docker-compose.prod.yml` states that nothing starts on a default# credential and implements it with `${VAR:?…}`. Compose's `:?` rejects# an unset or empty variable and cannot look at a value, so the guarddiff --git a/.github/workflows/connector-scheduler-live.yml b/.github/workflows/connector-scheduler-live.ymlnew file mode 100644index 000000000..39d68763b--- /dev/null+++ b/.github/workflows/connector-scheduler-live.yml@@ -0,0 +1,146 @@+# The connector scheduler, against real Postgres and a real HTTP vendor.+#+# Why this exists+# ---------------+# The defect this job exists for is the reason "connect a source" never+# pulled data on any deployment: `reload_jobs` added every poll job with+# `next_run_time=None`, which APScheduler 3.x registers as PAUSED. The+# scheduler started, the job list looked right, `last_sync` stayed null for+# every instance, and nothing logged an error — nothing had gone wrong,+# exactly. The jobs were never going to run.+#+# No mock catches that. A test asserting `add_job` was called with the right+# arguments passes whether or not APScheduler will ever fire it; only a real+# AsyncIOScheduler knows what `next_run_time=None` means.+#+# The vendor and ingest are stubs because this is not a test of Okta's API —+# but they are *sockets*, not patched methods, so the connector's own client+# and the real IngestClient are both on the path.+#+# No `paths:` filter: `docs/audit/MATURITY_DEFINITION.md` requires a Stable+# capability to be graded on every pull request.+name: Connector scheduler (live)++on:+ push:+ branches: [main]+ pull_request:+ workflow_dispatch:++concurrency:+ group: connector-scheduler-live-${{ github.ref }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.sha }}+ cancel-in-progress: ${{ github.event_name == 'pull_request' }}++permissions:+ contents: read++jobs:+ connector-scheduler-live:+ name: An enabled connector polls and reaches ingest+ runs-on: ubuntu-latest+ timeout-minutes: 25++ services:+ postgres:+ image: postgres:16-alpine+ env:+ POSTGRES_USER: aisoc+ POSTGRES_PASSWORD: aisoc_ci_only+ POSTGRES_DB: aisoc+ ports: ["5432:5432"]+ options: >-+ --health-cmd "pg_isready -U aisoc"+ --health-interval 5s+ --health-timeout 5s+ --health-retries 20++ env:+ # The runtime role, not the owner: a superuser ignores every RLS+ # policy even under FORCE ROW LEVEL SECURITY.+ CONNECTOR_DSN: postgresql+asyncpg://aisoc_app:aisoc_app_ci_only@localhost:5432/aisoc+ DATABASE_MIGRATION_URL: postgresql+asyncpg://aisoc:aisoc_ci_only@localhost:5432/aisoc+ PGPASSWORD: aisoc_ci_only+ AISOC_CONNECTORS_DISABLE_SCHEDULER: "0"++ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7++ - uses: actions/setup-python@e9d6f990972a57673cdb72ec29e19d42ba28880f # v6+ with:+ python-version: "3.11"++ # The service's own lockfile, not a hand-curated pip list.+ #+ # Four rounds of "ModuleNotFoundError: no module named X" went by+ # before this: a suite that drives the real service pulls in the+ # real service's whole import chain, and curating that list by hand+ # is the anti-pattern this repository already removed from its+ # images — a set that differs from the lock means the job tests+ # software nobody ships.+ - name: Install connectors from its lockfile+ working-directory: services/connectors+ run: |+ set -euo pipefail+ python -m pip install --quiet --upgrade pip "poetry==2.4.1"+ poetry config virtualenvs.create false+ poetry install --no-root --no-interaction+ python -m pip install --quiet pytest pytest-asyncio++ - name: Apply every migration, in order+ run: |+ for f in services/api/migrations/*.sql; do+ psql -h localhost -U aisoc -d aisoc -v ON_ERROR_STOP=1 -q -f "$f"+ done++ - name: Give the runtime role its password+ run: |+ psql -h localhost -U aisoc -d aisoc -v ON_ERROR_STOP=1 -q -c \+ "ALTER ROLE aisoc_app WITH PASSWORD 'aisoc_app_ci_only';"+ psql -h localhost -U aisoc -d aisoc -v ON_ERROR_STOP=1 -q -c \+ "GRANT USAGE ON SCHEMA public TO aisoc_app;"++ - name: Mint a credential key for the vault+ run: |+ KEY=$(python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())")+ echo "AISOC_CREDENTIAL_KEY=$KEY" >> "$GITHUB_ENV"++ - name: An enabled connector polls, reaches ingest, records last_sync+ working-directory: services/connectors+ env:+ ISOLATION_CONNECTOR_DSN: ${{ env.CONNECTOR_DSN }}+ run: python -m pytest ../../tests/isolation/test_connector_scheduler_live.py -v --tb=short++ # The negative control, and it re-injects the real defect.+ #+ # `next_run_time=None` is what shipped. Measured: four of the five+ # tests fail with it back — the job is PAUSED, no event reaches+ # ingest, no tenant header arrives and `last_sync` stays null.+ #+ # If the suite still passes, it is not driving a real scheduler, and+ # that is worth failing the build over: the alternative is a green+ # check over a product where connecting a source does nothing.+ - name: The gate fails when poll jobs are registered paused+ working-directory: services/connectors+ env:+ ISOLATION_CONNECTOR_DSN: ${{ env.CONNECTOR_DSN }}+ run: |+ cp app/scheduler.py /tmp/scheduler.bak+ python - <<'PY'+ import pathlib, sys+ p = pathlib.Path("app/scheduler.py")+ s = p.read_text()+ if "next_run_time=first_run," not in s:+ print("negative control could not find the next_run_time argument; the "+ "scheduler has been reshaped and this control proves nothing")+ sys.exit(1)+ p.write_text(s.replace("next_run_time=first_run,", "next_run_time=None,", 1))+ PY+ if python -m pytest ../../tests/isolation/test_connector_scheduler_live.py -q >/dev/null 2>&1; then+ cp /tmp/scheduler.bak app/scheduler.py+ echo "FAIL: the suite passed with every poll job registered PAUSED."+ echo " It is not driving a real APScheduler."+ exit 1+ fi+ cp /tmp/scheduler.bak app/scheduler.py+ echo "OK: re-injecting the paused-job defect makes the suite fail, as it must."+ python -m pytest ../../tests/isolation/test_connector_scheduler_live.py -qdiff --git a/.github/workflows/graph-live.yml b/.github/workflows/graph-live.ymlnew file mode 100644index 000000000..022a2d70f--- /dev/null+++ b/.github/workflows/graph-live.yml@@ -0,0 +1,151 @@+# The production graph reader, against a real Neo4j, on every pull request.+#+# Why this is its own workflow rather than a step in isolation-live.yml+# ---------------------------------------------------------------------+# `docs/audit/MATURITY_DEFINITION.md` requires a Stable capability to be+# graded unconditionally: no `paths:` filter, no `if:` that could skip it. A+# required check that never reports is weaker than no check, because it looks+# green on every commit it did not read.+#+# `isolation-live.yml` carries a path filter, and widening it would not fix+# the property — a filter that lists more files is still a filter. Removing+# that filter instead would run Neo4j, Redis, ClickHouse and Kafka on every+# pull request, which is a lot of minutes to buy one capability's coverage.+#+# So the graph gets its own job with the one container it needs. Neo4j alone+# is ready in about twenty seconds, which is cheap enough to run always —+# and "always" is the whole point.+name: Graph (live)++on:+ push:+ branches: [main]+ pull_request:+ workflow_dispatch:++# No `paths:` above, deliberately. See the header.++concurrency:+ # Per commit on a push and per pull request otherwise. Keying on `ref`+ # alone lets a push to main cancel the run before it: across 1,000 push+ # runs this repository lost 259 that way, and a cancelled run reports+ # neither pass nor fail.+ group: graph-live-${{ github.ref }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.sha }}+ cancel-in-progress: ${{ github.event_name == 'pull_request' }}++permissions:+ contents: read++jobs:+ graph-live:+ name: Tenant scoping holds in the production graph reader+ runs-on: ubuntu-latest+ timeout-minutes: 15+ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7++ - uses: actions/setup-python@e9d6f990972a57673cdb72ec29e19d42ba28880f # v6+ with:+ python-version: "3.11"++ # APOC is installed because the shipped compose installs it, so the+ # blast-radius query takes the same path here as it does in a real+ # deployment. The plain-Cypher fallback is still reachable for a+ # managed Neo4j that restricts `apoc.*`, and a unit test covers that+ # branch — but this job should look like production, not like the+ # degraded case.+ - name: Start Neo4j the way docker-compose.yml does+ run: |+ docker run -d --name graph-neo4j -p 7687:7687 \+ -e NEO4J_AUTH=neo4j/testpassword123 \+ -e NEO4J_PLUGINS='["apoc"]' \+ -e NEO4J_dbms_security_procedures_unrestricted='apoc.*' \+ neo4j:5.15-community++ # The service's own lockfile, not a hand-curated pip list.+ #+ # Four rounds of "ModuleNotFoundError: no module named X" went by+ # before this: a suite that drives the real service pulls in the+ # real service's whole import chain, and curating that list by hand+ # is the anti-pattern this repository already removed from its+ # images — a set that differs from the lock means the job tests+ # software nobody ships.+ - name: Install api from its lockfile+ working-directory: services/api+ run: |+ set -euo pipefail+ python -m pip install --quiet --upgrade pip "poetry==2.4.1"+ poetry config virtualenvs.create false+ poetry install --no-root --no-interaction+ python -m pip install --quiet pytest pytest-asyncio neo4j++ - name: Wait for Neo4j+ run: |+ for i in $(seq 1 60); do+ if docker exec graph-neo4j cypher-shell -u neo4j -p testpassword123 "RETURN 1" >/dev/null 2>&1; then+ echo "Neo4j ready after ${i}s"; exit 0+ fi+ sleep 1+ done+ echo "Neo4j did not become ready"; docker logs graph-neo4j | tail -40; exit 1++ - name: The production reader scopes every read to its tenant+ working-directory: services/api+ env:+ ISOLATION_NEO4J_URI: bolt://localhost:7687+ ISOLATION_NEO4J_USER: neo4j+ ISOLATION_NEO4J_PASSWORD: testpassword123+ run: python -m pytest ../../tests/isolation/test_graph_service_live.py -v --tb=short++ # The negative control, and the reason this job is evidence rather+ # than decoration.+ #+ # `get_entity_neighbors` once accepted `tenant_id` and never bound it:+ # any authenticated user could read any other tenant's host, user or+ # IOC node with all of its properties by naming the id. Its unit tests+ # asserted on Cypher *text* through a fake session and passed+ # throughout, because a text assertion cannot notice a missing+ # parameter.+ #+ # So this removes the anchor's tenant predicate and requires the suite+ # above to go red. If it does not, the suite is not testing what its+ # name says, and this job fails rather than reporting a green that+ # means nothing.+ #+ # Note the predicate removed is the *anchor* one. An earlier version+ # of the suite could not tell the two apart: the neighbour predicate+ # masked an unscoped anchor, and all six tests stayed green with the+ # anchor scope deleted. The fixture now attaches a global MITRE+ # Technique to the other tenant's host, which `_scoped` admits for+ # everyone, so the anchor is exercised on its own.+ - name: The gate fails when tenant scoping is removed+ working-directory: services/api+ env:+ ISOLATION_NEO4J_URI: bolt://localhost:7687+ ISOLATION_NEO4J_USER: neo4j+ ISOLATION_NEO4J_PASSWORD: testpassword123+ run: |+ cp app/services/graph_service.py /tmp/graph_service.bak+ python - <<'PY'+ import pathlib, sys+ p = pathlib.Path("app/services/graph_service.py")+ s = p.read_text()+ needle = ' MATCH (n:{label} {{{id_prop}: $entity_id}})\n WHERE {_scoped("n")}\n'+ if needle not in s:+ print("negative control could not find the predicate to remove; "+ "it has been renamed or reformatted and this control is no "+ "longer proving anything")+ sys.exit(1)+ p.write_text(s.replace(needle, ' MATCH (n:{label} {{{id_prop}: $entity_id}})\n WHERE true\n', 1))+ PY+ if python -m pytest ../../tests/isolation/test_graph_service_live.py -q >/dev/null 2>&1; then+ cp /tmp/graph_service.bak app/services/graph_service.py+ echo "FAIL: the suite passed with tenant scoping removed, so it is not testing tenant scoping."+ exit 1+ fi+ cp /tmp/graph_service.bak app/services/graph_service.py+ echo "OK: removing the tenant predicate makes the suite fail, as it must."++ - name: Neo4j logs on failure+ if: failure()+ run: docker logs graph-neo4j | tail -80diff --git a/.github/workflows/lake-live.yml b/.github/workflows/lake-live.ymlnew file mode 100644index 000000000..e55513fb6--- /dev/null+++ b/.github/workflows/lake-live.yml@@ -0,0 +1,156 @@+# The event lake, against the schema the product actually ships.+#+# Why loading the real DDL is the point+# ---------------------------------------+# `test_live_stores.py` has a live ClickHouse case and it writes its own+# `CREATE TABLE aisoc.raw_events` — a hand-maintained subset of what+# `services/api/clickhouse/001_init.sql` creates. A test that builds its own+# schema cannot notice the real one has drifted, which is the same shape as+# asserting on a producer against a copy of itself.+#+# Writing this suite proved the point immediately: the first version+# inserted `message` and `raw_event`, and the shipped table has neither.+#+# No `paths:` filter. The previous arrangement also filtered on paths that+# omitted `lake_hunt.py`, `lake_rate_limit.py` and `endpoints/lake.py`, so+# editing the lake's own query surface never re-graded it.+name: Event lake (live)++on:+ push:+ branches: [main]+ pull_request:+ workflow_dispatch:++concurrency:+ group: lake-live-${{ github.ref }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.sha }}+ cancel-in-progress: ${{ github.event_name == 'pull_request' }}++permissions:+ contents: read++jobs:+ lake-live:+ name: The shipped DDL loads and tenant scoping holds+ runs-on: ubuntu-latest+ timeout-minutes: 20+ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7++ - uses: actions/setup-python@e9d6f990972a57673cdb72ec29e19d42ba28880f # v6+ with:+ python-version: "3.11"++ - name: Start ClickHouse+ run: |+ docker run -d --name lake-clickhouse -p 9000:9000 -p 8123:8123 \+ -e CLICKHOUSE_SKIP_USER_SETUP=1 \+ clickhouse/clickhouse-server:24.3-alpine++ # The service's own lockfile, not a hand-curated pip list.+ #+ # Four rounds of "ModuleNotFoundError: no module named X" went by+ # before this: a suite that drives the real service pulls in the+ # real service's whole import chain, and curating that list by hand+ # is the anti-pattern this repository already removed from its+ # images — a set that differs from the lock means the job tests+ # software nobody ships.+ - name: Install api from its lockfile+ working-directory: services/api+ run: |+ set -euo pipefail+ python -m pip install --quiet --upgrade pip "poetry==2.4.1"+ poetry config virtualenvs.create false+ poetry install --no-root --no-interaction+ python -m pip install --quiet pytest pytest-asyncio clickhouse-driver++ # The hunt suite below drives `app.hunt` from services/agents, a+ # different service with a different lock.+ - name: Install agents from its lockfile+ working-directory: services/agents+ run: |+ set -euo pipefail+ poetry config virtualenvs.create false+ poetry install --no-root --no-interaction+ python -m pip install --quiet clickhouse-driver++ # The hunt suite below drives `app.hunt` from services/agents, a+ # different service with a different lock.+ - name: Install agents from its lockfile+ working-directory: services/agents+ run: |+ set -euo pipefail+ poetry config virtualenvs.create false+ poetry install --no-root --no-interaction+ python -m pip install --quiet clickhouse-driver++ - name: Wait for ClickHouse+ run: |+ for i in $(seq 1 60); do+ if docker exec lake-clickhouse clickhouse-client -q "SELECT 1" >/dev/null 2>&1; then+ echo "ClickHouse ready after ${i}s"; exit 0+ fi+ sleep 1+ done+ echo "ClickHouse did not become ready"; docker logs lake-clickhouse | tail -40; exit 1++ - name: The shipped DDL loads and tenant scoping holds+ working-directory: services/api+ env:+ ISOLATION_CLICKHOUSE_HOST: localhost+ ISOLATION_CLICKHOUSE_PORT: "9000"+ ISOLATION_REPO_ROOT: ${{ github.workspace }}+ run: python -m pytest ../../tests/isolation/test_lake_live.py -v --tb=short++ # Parity 6.1: scheduled hunts read the tenant's own events rather+ # than a fixture corpus. Run from `services/agents`, because both+ # services call their package `app` and running this from+ # `services/api` resolves `app.hunt` to nothing.+ - name: Scheduled hunts read tenant data, and refuse to fabricate+ working-directory: services/agents+ env:+ ISOLATION_REPO_ROOT: ${{ github.workspace }}+ run: python -m pytest ../../tests/isolation/test_hunt_lake_live.py -v --tb=short++ # The negative control.+ #+ # Makes `rewrite_for_tenant` a pass-through — the shape where an+ # operator's SQL reaches the lake unfiltered and every tenant reads+ # every other tenant's events. Measured: three of the six tests+ # fail, including the fail-closed one.+ #+ # Worth stating why the fail-closed test is in that set. Returning+ # the original SQL when the rewrite cannot be applied is exactly+ # what a pass-through does, so a control that only checked the+ # happy path would miss the more dangerous half.+ - name: The gate fails when the tenant rewrite is a pass-through+ working-directory: services/api+ env:+ ISOLATION_CLICKHOUSE_HOST: localhost+ ISOLATION_CLICKHOUSE_PORT: "9000"+ ISOLATION_REPO_ROOT: ${{ github.workspace }}+ run: |+ cp app/services/lake_sql.py /tmp/lake_sql.bak+ python - <<'PY'+ import pathlib, re, sys+ p = pathlib.Path("app/services/lake_sql.py")+ s = p.read_text()+ m = re.search(r"\ndef rewrite_for_tenant\(\n(.*?)\n\) -> [^\n]*:\n", s, re.S)+ if not m:+ print("negative control could not find rewrite_for_tenant's signature; it "+ "has been reshaped and this control proves nothing")+ sys.exit(1)+ p.write_text(s[: m.end()] + " return sql # injected pass-through\n" + s[m.end() :])+ PY+ if python -m pytest ../../tests/isolation/test_lake_live.py -q >/dev/null 2>&1; then+ cp /tmp/lake_sql.bak app/services/lake_sql.py+ echo "FAIL: the suite passed with the tenant rewrite doing nothing."+ exit 1+ fi+ cp /tmp/lake_sql.bak app/services/lake_sql.py+ echo "OK: a pass-through rewrite makes the suite fail, as it must."+ python -m pytest ../../tests/isolation/test_lake_live.py -q++ - name: ClickHouse logs on failure+ if: failure()+ run: docker logs lake-clickhouse | tail -60diff --git a/.github/workflows/live-actions-live.yml b/.github/workflows/live-actions-live.ymlnew file mode 100644index 000000000..668fe4a93--- /dev/null+++ b/.github/workflows/live-actions-live.yml@@ -0,0 +1,123 @@+# Governed response actions, against a real HTTP vendor, every pull request.+#+# What Stable asserts for a governed capability+# -----------------------------------------------+# Not that the product acts on its own. Response actions require a human+# approver by design, and that is a safety posture rather than an incomplete+# implementation. What Stable asserts is that the governance machinery is+# proven — **including that it correctly refuses**.+#+# Why a socket rather than a mock+# ---------------------------------+# Signatures drift silently here because simulation mode never constructs+# the client: `SearchSIEMExecutor` passed `max_results=` when the client+# takes `max_count`, and `CreateNotableEventExecutor` passed+# `title`/`description`/`fields` when it takes `event_data`. Every live call+# raised TypeError and every test passed.+#+# A mock accepts whatever it is handed. A real HTTP server does not get+# called at all when the client fails to construct, which is the signal that+# was missing.+#+# No `paths:` filter: `docs/audit/MATURITY_DEFINITION.md` requires a Stable+# capability to be graded on every pull request.+name: Live actions (governed)++on:+ push:+ branches: [main]+ pull_request:+ workflow_dispatch:++concurrency:+ group: live-actions-${{ github.ref }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.sha }}+ cancel-in-progress: ${{ github.event_name == 'pull_request' }}++permissions:+ contents: read++jobs:+ live-actions:+ name: Governance permits, refuses, and never leaks a refusal to the vendor+ runs-on: ubuntu-latest+ timeout-minutes: 15+ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7++ - uses: actions/setup-python@e9d6f990972a57673cdb72ec29e19d42ba28880f # v6+ with:+ python-version: "3.11"++ # The service's own lockfile, not a hand-curated pip list.+ #+ # Four rounds of "ModuleNotFoundError: no module named X" went by+ # before this: a suite that drives the real service pulls in the+ # real service's whole import chain, and curating that list by hand+ # is the anti-pattern this repository already removed from its+ # images — a set that differs from the lock means the job tests+ # software nobody ships.+ - name: Install actions from its lockfile+ working-directory: services/actions+ run: |+ set -euo pipefail+ python -m pip install --quiet --upgrade pip "poetry==2.4.1"+ poetry config virtualenvs.create false+ poetry install --no-root --no-interaction+ python -m pip install --quiet pytest pytest-asyncio++ # Not a `services:` container: the vendor here is a socket the test+ # owns, started in-process, so it can record exactly which requests+ # reached it. "Did anything leave the process?" is the one question+ # a refusal has to answer, and a container would answer it less+ # directly than a handler that appends to a list.+ - name: Governance permits, refuses, and does not leak+ working-directory: services/actions+ run: python -m pytest ../../tests/isolation/test_live_actions_live.py -v --tb=short++ # The negative control.+ #+ # Removes the three governance branches — BLOCKED, QUEUED_APPROVAL+ # and the DRY_RUN downgrade — and requires the suite to go red.+ #+ # Measured: `test_a_refused_action_never_reaches_the_vendor` fails,+ # because `isolate_host` then actually attempts its CrowdStrike+ # OAuth call and the stub records it. That is the assertion worth+ # having: a refusal that still called the vendor is a refusal in+ # name only, since the containment has already happened.+ #+ # Note what this replaced. An earlier version removed only the+ # approval branch and the suite stayed green — the DRY_RUN+ # downgrade still held the action, so the control was not proving+ # what it claimed. Before that, with no executors registered at all,+ # every dispatch answered `executor_not_found` and all seven tests+ # passed without governance ever being consulted.+ - name: The gate fails when governance is removed from the dispatcher+ working-directory: services/actions+ run: |+ cp app/live_actions/dispatcher.py /tmp/dispatcher.bak+ python - <<'PY'+ import pathlib, re, sys+ p = pathlib.Path("app/live_actions/dispatcher.py")+ s = p.read_text()+ # Matched on the three AutonomyMode branches by name rather than+ # on an exact block of text, so reindentation or a reworded log+ # line does not silently turn this control into a no-op.+ pattern = re.compile(+ r"\n *if decision\.mode is AutonomyMode\.BLOCKED:.*?"+ r"request = request\.model_copy\(update=\{\"dry_run\": True\}\)\n",+ re.S,+ )+ if not pattern.search(s):+ print("negative control could not find the governance block; the dispatcher "+ "has been reshaped and this control proves nothing")+ sys.exit(1)+ p.write_text(pattern.sub("\n", s, count=1))+ PY+ if python -m pytest ../../tests/isolation/test_live_actions_live.py -q >/dev/null 2>&1; then+ cp /tmp/dispatcher.bak app/live_actions/dispatcher.py+ echo "FAIL: the suite passed with every governance branch removed from dispatch."+ exit 1+ fi+ cp /tmp/dispatcher.bak app/live_actions/dispatcher.py+ echo "OK: removing governance makes the suite fail, as it must."+ python -m pytest ../../tests/isolation/test_live_actions_live.py -qdiff --git a/.github/workflows/live-agent-eval.yml b/.github/workflows/live-agent-eval.ymlindex aa40342e7..c40de90e7 100644--- a/.github/workflows/live-agent-eval.yml+++ b/.github/workflows/live-agent-eval.yml@@ -33,6 +33,7 @@ name: Live agent eval (local model)# on-demand, failing hard when the floor is breached, is what that supports.on:+ pull_request:workflow_dispatch:inputs:model:@@ -64,12 +65,20 @@ permissions:pull-requests: writeconcurrency:- group: live-agent-eval-${{ github.ref }}- cancel-in-progress: true+ # Per commit on a push and per pull request otherwise. Keying on+ # `github.ref` alone lets a push to main cancel the run before it:+ # across 1,000 push runs this repository lost 259 that way, and a+ # cancelled run reports neither pass nor fail.+ group: live-agent-eval-${{ github.ref }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.sha }}+ cancel-in-progress: ${{ github.event_name == 'pull_request' }}jobs:live-agent:name: Dispatch the real agent against a local model+ # The deep run stays on the schedule and on demand: ten incidents+ # through a CPU-hosted model is a 90-minute budget, which is not a+ # pull-request shape. The PR gate is `live-agent-smoke` below.+ if: github.event_name != 'pull_request'runs-on: ubuntu-latesttimeout-minutes: 90env:@@ -234,3 +243,139 @@ jobs:- name: Ollama log on failureif: failure()run: tail -100 "$RUNNER_TEMP/ollama.log" || true++ # Commit-level grading of the agent, which did not exist.+ #+ # `REALITY_REPORT.md` calls that "the hard ceiling on the project's+ # credibility": `mitre_accuracy` is an offline keyword extractor, and both+ # jobs that dispatch the real LangGraph ran only on a schedule or on+ # demand. So no pull request had ever been graded on agent behaviour.+ #+ # This job is deliberately small. One incident, the smallest bundled+ # model, and one question: **did the agent place a real LLM call and come+ # back with something schema-valid?** It is not an accuracy measurement —+ # the deep run above is — it is the check that the live path still works+ # on the commit being merged.+ #+ # The distinction matters because the harness has been fooled before. With+ # the model pins unset every agent caught its provider error and used its+ # deterministic fallback, and the run reported `mode: live` with a+ # groundedness of 0.8050 at 0.11s per investigation — flattering, and+ # faster than a network round trip. `--wet-require-live` could not see it+ # because records existed and the stack had imported. `llm_calls_placed`+ # is what closes that hole, and this job asserts on it directly.+ live-agent-smoke:+ name: The agent places a real LLM call on this commit+ runs-on: ubuntu-latest+ timeout-minutes: 30+ env:+ OLLAMA_MODEL: qwen2.5:0.5b+ INCIDENT_LIMIT: '1'+ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7++ - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0+ with:+ python-version: '3.11'++ - name: Install agent dependencies+ working-directory: services/agents+ run: |+ set -euo pipefail+ python -m pip install --quiet --upgrade pip "poetry==2.4.1"+ poetry config virtualenvs.create false+ poetry install --no-root --no-interaction++ - name: Install Ollama+ run: |+ set -euo pipefail+ curl -fsSL https://ollama.com/install.sh | sh+ ollama --version++ - name: Serve the model+ run: |+ set -euo pipefail+ nohup ollama serve > "$RUNNER_TEMP/ollama.log" 2>&1 &+ for i in $(seq 1 30); do+ if curl -sf http://127.0.0.1:11434/api/tags >/dev/null; then+ echo "ollama up after ${i}s"; break+ fi+ sleep 1+ done+ curl -sf http://127.0.0.1:11434/api/tags >/dev/null || {+ echo "::error title=live-agent-smoke::Ollama never became ready."+ tail -50 "$RUNNER_TEMP/ollama.log"+ exit 1+ }+ ollama pull "$OLLAMA_MODEL"++ - name: Dispatch the agent on one incident+ env:+ WET_EVAL_OPENAI_KEY: ollama+ OPENAI_API_KEY: ollama+ OPENAI_BASE_URL: http://127.0.0.1:11434/v1+ # One pin per role: the roles resolve through+ # `app.llm.model_pins.get_pin`, whose only env override is+ # `AISOC_MODEL_PIN_<ROLE>`. Leave them unset and every agent asks+ # Ollama for a gateway alias it has never heard of, then falls back+ # deterministically — which is the failure this job exists to catch.+ AISOC_MODEL_PIN_TRIAGE: qwen2.5:0.5b+ AISOC_MODEL_PIN_RECON: qwen2.5:0.5b+ AISOC_MODEL_PIN_NL: qwen2.5:0.5b+ AISOC_MODEL_PIN_HUNT: qwen2.5:0.5b+ run: |+ set -euo pipefail+ python3 scripts/run_evals.py \+ --wet \+ --wet-require-live \+ --wet-model "$OLLAMA_MODEL" \+ --wet-limit "$INCIDENT_LIMIT" \+ --out live-agent-smoke.json \+ --wet-out live-agent-smoke.wet.json++ # The assertion, and the only one this job makes.+ #+ # A report with `llm_calls_placed: 0` means every agent fell back+ # without a provider ever being reached — a run that looks measured+ # and measured nothing.+ - name: A real LLM call was placed+ run: |+ set -euo pipefail+ python3 - <<'PY'+ import json, pathlib, sys++ report = pathlib.Path("live-agent-smoke.wet.json")+ if not report.is_file():+ print("::error title=live-agent-smoke::No report produced; the dispatch step "+ "did not get far enough to write one.")+ sys.exit(1)++ data = json.loads(report.read_text())+ # Nested under `groundedness`, which is where the harness writes+ # it and where `check_live_agent_floor.py` reads it. The+ # top-level lookup is a fallback rather than the primary, so a+ # schema change surfaces as a failure here rather than as a+ # silently absent field.+ groundedness = data.get("groundedness") or {}+ placed = groundedness.get("llm_calls_placed", data.get("llm_calls_placed"))+ if placed is None:+ print("::error title=live-agent-smoke::The report carries no "+ "`llm_calls_placed`. That field is what distinguishes a live run from "+ "one where every agent used its deterministic fallback, and without it "+ "this job cannot tell them apart.")+ sys.exit(1)+ if not placed:+ print(f"::error title=live-agent-smoke::llm_calls_placed={placed}. The agent "+ "placed no LLM call at all, so whatever it reported came from a "+ "fallback rather than a model.")+ sys.exit(1)++ print(f"OK: {placed} LLM call(s) placed against the local model.")+ PY++ - name: Report on failure+ if: failure()+ run: |+ set -uo pipefail+ tail -40 "$RUNNER_TEMP/ollama.log" || true+ [ -f live-agent-smoke.wet.json ] && head -c 2000 live-agent-smoke.wet.json || truediff --git a/.github/workflows/playbook-pause-live.yml b/.github/workflows/playbook-pause-live.ymlnew file mode 100644index 000000000..c9c7a75f4--- /dev/null+++ b/.github/workflows/playbook-pause-live.yml@@ -0,0 +1,164 @@+# The durable approval pause, against the Postgres it claims to use.+#+# Why this exists+# ---------------+# The offline suite is 15 tests against `_FakeRows` and `_FakePool`. It is a+# useful test of the module's logic and cannot prove the thing the feature is+# named for, because every property that matters belongs to the database:+# "survives a restart" is a claim about rows on disk, and "a double-tap+# resolves once" is enforced by a partial unique index that the fake+# re-implements in Python.+#+# A hand-written re-implementation of a constraint is the+# double-more-capable-than-the-real-thing shape that caused four of six+# defects in one live-QA pass. The README also claimed "live Postgres+# suspend/resume" before this file existed, which was simply untrue.+#+# No `paths:` filter: `docs/audit/MATURITY_DEFINITION.md` requires a Stable+# capability to be graded on every pull request.+name: Playbook pause + ledger (live)++on:+ push:+ branches: [main]+ pull_request:+ workflow_dispatch:++concurrency:+ group: playbook-pause-live-${{ github.ref }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.sha }}+ cancel-in-progress: ${{ github.event_name == 'pull_request' }}++permissions:+ contents: read++jobs:+ playbook-pause-live:+ name: Playbook pause and Investigation Ledger against real Postgres+ runs-on: ubuntu-latest+ timeout-minutes: 15++ services:+ postgres:+ image: postgres:16-alpine+ env:+ POSTGRES_USER: aisoc+ POSTGRES_PASSWORD: aisoc_ci_only+ POSTGRES_DB: aisoc+ ports: ["5432:5432"]+ options: >-+ --health-cmd "pg_isready -U aisoc"+ --health-interval 5s+ --health-timeout 5s+ --health-retries 20++ env:+ # Two roles, as the default deployment ships them. `aisoc` owns the+ # schema and runs the migrations; `aisoc_app` is DML-only and is the+ # role the RLS policies apply to, so it is the one the suite connects+ # as. A superuser ignores every policy even under FORCE ROW LEVEL+ # SECURITY, which would let a cross-tenant assertion pass for the+ # wrong reason — `check_runtime_db_role.py` refuses that wiring.+ PLAYBOOK_DSN: postgresql://aisoc_app:aisoc_app_ci_only@localhost:5432/aisoc+ DATABASE_MIGRATION_URL: postgresql://aisoc:aisoc_ci_only@localhost:5432/aisoc+ PGPASSWORD: aisoc_ci_only++ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7++ - uses: actions/setup-python@e9d6f990972a57673cdb72ec29e19d42ba28880f # v6+ with:+ python-version: "3.11"++ # The service's own lockfile, not a hand-curated pip list.+ #+ # Four rounds of "ModuleNotFoundError: no module named X" went by+ # before this: a suite that drives the real service pulls in the+ # real service's whole import chain, and curating that list by hand+ # is the anti-pattern this repository already removed from its+ # images — a set that differs from the lock means the job tests+ # software nobody ships.+ - name: Install agents from its lockfile+ working-directory: services/agents+ run: |+ set -euo pipefail+ python -m pip install --quiet --upgrade pip "poetry==2.4.1"+ poetry config virtualenvs.create false+ poetry install --no-root --no-interaction+ python -m pip install --quiet pytest pytest-asyncio++ # The whole chain, in order.+ #+ # This started as a cherry-picked 001/002/081 — enough for the pause+ # table, and `002` is needed because `081`'s RLS policy calls+ # `current_tenant_id()`, which a migration applied alone does not+ # have. Then the ledger suite joined this job and failed on+ # `relation "investigation_runs" does not exist`, because its tables+ # arrive later in the chain.+ #+ # A cherry-picked subset builds a schema no deployment has, which is+ # the opposite of the point. Applying everything costs seconds and+ # cannot be wrong in this direction.+ - name: Apply every migration, in order+ run: |+ for f in services/api/migrations/*.sql; do+ psql -h localhost -U aisoc -d aisoc -v ON_ERROR_STOP=1 -q -f "$f"+ done++ # `002_rls.sql` creates `aisoc_app` with a placeholder password. Set+ # the real one here, after the migrations have run, so the suite can+ # connect as the role the policies apply to.+ - name: Give the runtime role its password+ run: |+ psql -h localhost -U aisoc -d aisoc -v ON_ERROR_STOP=1 -q -c \+ "ALTER ROLE aisoc_app WITH PASSWORD 'aisoc_app_ci_only';"+ psql -h localhost -U aisoc -d aisoc -v ON_ERROR_STOP=1 -q -c \+ "GRANT USAGE ON SCHEMA public TO aisoc_app;"++ - name: The pause suspends, survives, resumes, expires and isolates+ working-directory: services/agents+ env:+ ISOLATION_PLAYBOOK_DSN: ${{ env.PLAYBOOK_DSN }}+ run: python -m pytest ../../tests/isolation/test_playbook_pause_live.py -v --tb=short++ # The Investigation Ledger, the surface an auditor reads. Its+ # offline test drives a `_FakeConn` and opens no connection, so it+ # cannot prove the rows exist, that RLS applies, or that the+ # foreign keys refuse an event for a run nobody started.+ - name: The Investigation Ledger persists, scopes and refuses+ working-directory: services/agents+ env:+ ISOLATION_LEDGER_DSN: ${{ env.PLAYBOOK_DSN }}+ run: python -m pytest ../../tests/isolation/test_ledger_live.py -v --tb=short++ # The negative control.+ #+ # `aisoc_playbook_pauses_run_idx` is the partial unique index on+ # `status = 'waiting'` that stops one run having two live pauses. The+ # offline suite re-implements that rule in Python and so would pass+ # whether or not the index exists.+ #+ # Dropping it must make the suite go red. Measured: one test fails,+ # `test_two_waiting_pauses_for_one_run_are_refused`, which is exactly+ # the one that asserts the database refuses rather than the module.+ #+ # Note the sibling `test_a_double_tap_resolves_once` keeps passing+ # without the index, and that is correct — single resolution is+ # enforced by the atomic `UPDATE ... WHERE status = 'waiting'`, which+ # is a different mechanism. Two properties, two tests; a control that+ # broke both would be telling us less, not more.+ - name: The gate fails when the partial unique index is gone+ working-directory: services/agents+ env:+ ISOLATION_PLAYBOOK_DSN: ${{ env.PLAYBOOK_DSN }}+ run: |+ psql -h localhost -U aisoc -d aisoc -q -c \+ "DROP INDEX IF EXISTS aisoc_playbook_pauses_run_idx;"+ if python -m pytest ../../tests/isolation/test_playbook_pause_live.py -q >/dev/null 2>&1; then+ echo "FAIL: the suite passed without the partial unique index, so it is not"+ echo " proving the database enforces single resolution."+ exit 1+ fi+ echo "OK: dropping the index makes the suite fail, as it must."+ psql -h localhost -U aisoc -d aisoc -v ON_ERROR_STOP=1 -q \+ -f ../../services/api/migrations/081_playbook_pauses.sql+ python -m pytest ../../tests/isolation/test_playbook_pause_live.py -qdiff --git a/.github/workflows/retro-hunt-live.yml b/.github/workflows/retro-hunt-live.ymlnew file mode 100644index 000000000..9e61d4aba--- /dev/null+++ b/.github/workflows/retro-hunt-live.yml@@ -0,0 +1,156 @@+# Retro-hunts, with the feature flag ON, against a real ClickHouse and a+# real broker, on every pull request.+#+# Why the flag's default does not hold this at Beta+# ---------------------------------------------------+# `RETRO_HUNT_ENABLED` defaults to False and stays that way: a retro-hunt+# replays every newly-published indicator against a tenant's entire recorded+# history, so arming it on upgrade would change what a deployment does to+# its own lake without anyone asking.+#+# `docs/audit/MATURITY_DEFINITION.md` says a flag's default is an+# operational choice of the same kind as a compose profile, with one+# condition that is not a formality: the gate must grade the capability+# **with the flag on**. A default-off feature whose tests also run with it+# off is not Stable, it is untested.+#+# So `RETRO_HUNT_ENABLED: "true"` below is load-bearing.+#+# What was missing+# ------------------+# The sweep already had a live suite that re-implements nothing. The+# **consumer** had none, and its path filter omitted+# `retro_hunt_consumer.py` entirely — so the loop that decides whether a+# fault is worth retrying was never graded by anything.+#+# That gap has shipped as a defect here before, in its worst form: UEBA's+# Kafka consumer had no `except` at all, so a handler exception exited the+# `async for`, `finally` stopped the consumer, and the container sat at+# `running` with restarts 0 and `/health` returning 200 — permanently, with+# the exception never logged.+name: Retro-hunt (live)++on:+ push:+ branches: [main]+ pull_request:+ workflow_dispatch:++concurrency:+ group: retro-hunt-live-${{ github.ref }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.sha }}+ cancel-in-progress: ${{ github.event_name == 'pull_request' }}++permissions:+ contents: read++jobs:+ retro-hunt-live:+ name: A published indicator sweeps its tenant's recorded data+ runs-on: ubuntu-latest+ timeout-minutes: 25+ env:+ # Load-bearing. See the header.+ RETRO_HUNT_ENABLED: "true"+ ISOLATION_CLICKHOUSE_HOST: localhost+ ISOLATION_CLICKHOUSE_PORT: "9000"+ ISOLATION_KAFKA_BROKERS: 127.0.0.1:9092+ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7++ - uses: actions/setup-python@e9d6f990972a57673cdb72ec29e19d42ba28880f # v6+ with:+ python-version: "3.11"++ - name: Start ClickHouse+ run: |+ docker run -d --name rh-clickhouse -p 9000:9000 -p 8123:8123 \+ -e CLICKHOUSE_SKIP_USER_SETUP=1 \+ clickhouse/clickhouse-server:24.3-alpine++ # Redpanda rather than Kafka, matching `isolation-live.yml`: same+ # Kafka API, one container, no ZooKeeper.+ - name: Start Redpanda (Kafka API)+ run: |+ docker run -d --name rh-redpanda -p 9092:9092 \+ redpandadata/redpanda:v24.2.7 \+ redpanda start --mode dev-container --smp 1 \+ --kafka-addr PLAINTEXT://0.0.0.0:9092 \+ --advertise-kafka-addr PLAINTEXT://127.0.0.1:9092++ # The service's own lockfile, not a hand-curated pip list.+ #+ # Four rounds of "ModuleNotFoundError: no module named X" went by+ # before this: a suite that drives the real service pulls in the+ # real service's whole import chain, and curating that list by hand+ # is the anti-pattern this repository already removed from its+ # images — a set that differs from the lock means the job tests+ # software nobody ships.+ - name: Install api from its lockfile+ working-directory: services/api+ run: |+ set -euo pipefail+ python -m pip install --quiet --upgrade pip "poetry==2.4.1"+ poetry config virtualenvs.create false+ poetry install --no-root --no-interaction+ python -m pip install --quiet pytest pytest-asyncio "kafka-python>=2,<3" clickhouse-driver++ - name: Wait for the stores+ run: |+ for i in $(seq 1 60); do+ docker exec rh-clickhouse clickhouse-client -q "SELECT 1" >/dev/null 2>&1 && break+ sleep 1+ done+ for i in $(seq 1 60); do+ docker exec rh-redpanda rpk cluster info >/dev/null 2>&1 && break+ sleep 1+ done+ docker exec rh-clickhouse clickhouse-client -q "SELECT 1"+ docker exec rh-redpanda rpk cluster info >/dev/null++ - name: A published indicator is found only for the tenant whose data holds it+ working-directory: services/api+ env:+ ISOLATION_REPO_ROOT: ${{ github.workspace }}+ run: python -m pytest ../../tests/isolation/test_retro_hunt_live.py -v --tb=short++ - name: The consumer tells a working message from poison+ working-directory: services/api+ run: python -m pytest ../../tests/isolation/test_retro_hunt_consumer_live.py -v --tb=short++ # The negative control.+ #+ # Makes `parse_intel_event` accept everything. A parser that returns+ # a payload for undecodable bytes hands the sweep an envelope it+ # cannot use, which is how one malformed message stops a working+ # subscription — the class of defect this job exists for.+ #+ # Measured locally: the two poison tests fail.+ - name: The gate fails when the consumer stops rejecting poison+ working-directory: services/api+ run: |+ cp app/workers/retro_hunt_consumer.py /tmp/retro_hunt_consumer.bak+ python - <<'PY'+ import pathlib, sys+ p = pathlib.Path("app/workers/retro_hunt_consumer.py")+ s = p.read_text()+ needle = " except (TypeError, ValueError):\n return None\n"+ if needle not in s:+ print("negative control could not find the parser's guard; it has been "+ "reshaped and this control proves nothing")+ sys.exit(1)+ p.write_text(s.replace(needle, ' except (TypeError, ValueError):\n return {"event_type": "NEW_IOC", "data": {"value": "x", "type": "ipv4"}}\n', 1))+ PY+ if python -m pytest ../../tests/isolation/test_retro_hunt_consumer_live.py -q >/dev/null 2>&1; then+ cp /tmp/retro_hunt_consumer.bak app/workers/retro_hunt_consumer.py+ echo "FAIL: the suite passed with the parser accepting undecodable bytes."+ exit 1+ fi+ cp /tmp/retro_hunt_consumer.bak app/workers/retro_hunt_consumer.py+ echo "OK: a parser that accepts poison makes the suite fail, as it must."+ python -m pytest ../../tests/isolation/test_retro_hunt_consumer_live.py -q++ - name: Store logs on failure+ if: failure()+ run: |+ docker logs rh-clickhouse 2>&1 | tail -40+ docker logs rh-redpanda 2>&1 | tail -40diff --git a/.github/workflows/scim-live.yml b/.github/workflows/scim-live.ymlnew file mode 100644index 000000000..e0d863e5b--- /dev/null+++ b/.github/workflows/scim-live.yml@@ -0,0 +1,160 @@+# SCIM provisioning against the real application and a real Postgres.+#+# Why this exists+# ---------------+# The offline suite is 40 good tests whose harness cannot certify the+# capability, for two reasons.+#+# It runs on `sqlite+aiosqlite:///:memory:` with `@compiles` shims that+# translate JSONB, UUID, INET and ARRAY into something SQLite accepts. Those+# four types are where Postgres and SQLite disagree most, and a shim that+# makes a column *accept* a value is not evidence the real column stores or+# queries it. This repository has shipped exactly that defect before — a+# query naming two columns the table does not have, green across thirty+# tests against a fake.+#+# And it builds its own `FastAPI()` with the router mounted alone, so the+# production middleware, exception handlers and dependency graph are not+# what the assertions run through.+#+# No `paths:` filter: `docs/audit/MATURITY_DEFINITION.md` requires a Stable+# capability to be graded on every pull request. The two static gates for+# this capability, `check_scim_contract.py` and+# `check_whitelabel_metering.py`, run here too — they were only in+# `isolation.yml`, which is path-filtered.+name: SCIM (live)++on:+ push:+ branches: [main]+ pull_request:+ workflow_dispatch:++concurrency:+ group: scim-live-${{ github.ref }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.sha }}+ cancel-in-progress: ${{ github.event_name == 'pull_request' }}++permissions:+ contents: read++jobs:+ scim-live:+ name: SCIM provisions, lists and isolates through the real app+ runs-on: ubuntu-latest+ timeout-minutes: 20++ services:+ postgres:+ image: postgres:16-alpine+ env:+ POSTGRES_USER: aisoc+ POSTGRES_PASSWORD: aisoc_ci_only+ POSTGRES_DB: aisoc+ ports: ["5432:5432"]+ options: >-+ --health-cmd "pg_isready -U aisoc"+ --health-interval 5s+ --health-timeout 5s+ --health-retries 20++ env:+ # The runtime role, not the owner. A superuser ignores every RLS+ # policy even under FORCE ROW LEVEL SECURITY, which would let the+ # tenant-isolation assertion pass for the wrong reason.+ SCIM_DSN: postgresql+asyncpg://aisoc_app:aisoc_app_ci_only@localhost:5432/aisoc+ DATABASE_MIGRATION_URL: postgresql+asyncpg://aisoc:aisoc_ci_only@localhost:5432/aisoc+ PGPASSWORD: aisoc_ci_only+ ENVIRONMENT: development+ # Nothing to export to, and the retry backoff adds a minute of noise+ # to every run.+ OTEL_SDK_DISABLED: "true"++ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7++ - uses: actions/setup-python@e9d6f990972a57673cdb72ec29e19d42ba28880f # v6+ with:+ python-version: "3.11"++ # The service's own lockfile, not a hand-curated pip list.+ #+ # Four rounds of "ModuleNotFoundError: no module named X" went by+ # before this: a suite that drives the real service pulls in the+ # real service's whole import chain, and curating that list by hand+ # is the anti-pattern this repository already removed from its+ # images — a set that differs from the lock means the job tests+ # software nobody ships.+ - name: Install api from its lockfile+ working-directory: services/api+ run: |+ set -euo pipefail+ python -m pip install --quiet --upgrade pip "poetry==2.4.1"+ poetry config virtualenvs.create false+ poetry install --no-root --no-interaction+ python -m pip install --quiet pytest pytest-asyncio++ - name: Apply every migration, in order+ run: |+ for f in services/api/migrations/*.sql; do+ psql -h localhost -U aisoc -d aisoc -v ON_ERROR_STOP=1 -q -f "$f"+ done++ - name: Give the runtime role its password+ run: |+ psql -h localhost -U aisoc -d aisoc -v ON_ERROR_STOP=1 -q -c \+ "ALTER ROLE aisoc_app WITH PASSWORD 'aisoc_app_ci_only';"+ psql -h localhost -U aisoc -d aisoc -v ON_ERROR_STOP=1 -q -c \+ "GRANT USAGE ON SCHEMA public TO aisoc_app;"++ - name: SCIM provisions, lists and isolates+ working-directory: services/api+ env:+ ISOLATION_SCIM_DSN: ${{ env.SCIM_DSN }}+ DATABASE_URL: ${{ env.SCIM_DSN }}+ run: python -m pytest ../../tests/isolation/test_scim_live.py -v --tb=short++ # Moved here from `isolation.yml`, which is path-filtered. A static+ # gate behind a filter does not satisfy the unconditional property,+ # and these two are the contract checks for this capability.+ - name: The SCIM contract and the white-label/metering surface+ run: |+ python3 scripts/check_scim_contract.py --self-test+ python3 scripts/check_scim_contract.py+ python3 scripts/check_whitelabel_metering.py --self-test+ python3 scripts/check_whitelabel_metering.py++ # The negative control.+ #+ # Removing the tenant predicate from the Users list must make the+ # suite go red. Measured: one test fails,+ # `test_another_tenants_token_cannot_read_these_users`, which is the+ # one asserting a SCIM token sees only its own tenant.+ #+ # Without this, "the user I created is listed" would pass equally+ # well against a surface that returns everything to everyone.+ - name: The gate fails when SCIM stops scoping to the token's tenant+ working-directory: services/api+ env:+ ISOLATION_SCIM_DSN: ${{ env.SCIM_DSN }}+ DATABASE_URL: ${{ env.SCIM_DSN }}+ run: |+ cp app/api/v1/endpoints/scim.py /tmp/scim.bak+ python - <<'PY'+ import pathlib, sys+ p = pathlib.Path("app/api/v1/endpoints/scim.py")+ s = p.read_text()+ needle = " stmt = select(User).where(User.tenant_id == principal.tenant_id)"+ if needle not in s:+ print("negative control could not find the tenant predicate to remove; it has "+ "been reshaped and this control is no longer proving anything")+ sys.exit(1)+ p.write_text(s.replace(needle, " stmt = select(User)", 1))+ PY+ if python -m pytest ../../tests/isolation/test_scim_live.py -q >/dev/null 2>&1; then+ cp /tmp/scim.bak app/api/v1/endpoints/scim.py+ echo "FAIL: the suite passed with SCIM listing every tenant's users."+ exit 1+ fi+ cp /tmp/scim.bak app/api/v1/endpoints/scim.py+ echo "OK: removing the tenant predicate makes the suite fail, as it must."+ python -m pytest ../../tests/isolation/test_scim_live.py -qdiff --git a/.github/workflows/tenant-tuning-live.yml b/.github/workflows/tenant-tuning-live.ymlnew file mode 100644index 000000000..6d3687a5b--- /dev/null+++ b/.github/workflows/tenant-tuning-live.yml@@ -0,0 +1,145 @@+# A tenant's tuning, read out of Postgres and applied by the real engine.+#+# Why a live read matters more here than almost anywhere+# -------------------------------------------------------+# `OverlayCache._fetch` returns None when it cannot read, and the caller+# turns that into the empty overlay. The fail-soft is deliberate — losing a+# tenant's suppressions because the database blinked would turn their queue+# back on — but it means a **permanently broken overlay is+# indistinguishable from a tenant who has configured no tuning**. Both show+# an engine that suppresses nothing.+#+# That is exactly what happened. The query selected `rule_id` and+# `updated_by` from `detection_rules`, which has neither; every execution+# raised, the handler logged a warning, and the overlay loaded nothing on+# every deployment. Thirty unit tests passed throughout, against a fake that+# answered for whatever columns it was asked about.+#+# No `paths:` filter: `docs/audit/MATURITY_DEFINITION.md` requires a Stable+# capability to be graded on every pull request.+name: Tenant tuning (live)++on:+ push:+ branches: [main]+ pull_request:+ workflow_dispatch:++concurrency:+ group: tenant-tuning-live-${{ github.ref }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.sha }}+ cancel-in-progress: ${{ github.event_name == 'pull_request' }}++permissions:+ contents: read++jobs:+ tenant-tuning-live:+ name: Tuning written to Postgres changes what the engine fires+ runs-on: ubuntu-latest+ timeout-minutes: 20++ services:+ postgres:+ image: postgres:16-alpine+ env:+ POSTGRES_USER: aisoc+ POSTGRES_PASSWORD: aisoc_ci_only+ POSTGRES_DB: aisoc+ ports: ["5432:5432"]+ options: >-+ --health-cmd "pg_isready -U aisoc"+ --health-interval 5s+ --health-timeout 5s+ --health-retries 20++ env:+ # Two roles, as the default deployment ships them. `aisoc` owns the+ # schema and runs the migrations; `aisoc_app` is DML-only and is the+ # role the RLS policies apply to, so it is the one the suite connects+ # as. A superuser ignores every policy even under FORCE ROW LEVEL+ # SECURITY, which would let a cross-tenant assertion pass for the+ # wrong reason — `check_runtime_db_role.py` refuses that wiring.+ TUNING_DSN: postgresql://aisoc_app:aisoc_app_ci_only@localhost:5432/aisoc+ DATABASE_MIGRATION_URL: postgresql://aisoc:aisoc_ci_only@localhost:5432/aisoc+ PGPASSWORD: aisoc_ci_only++ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7++ - uses: actions/setup-python@e9d6f990972a57673cdb72ec29e19d42ba28880f # v6+ with:+ python-version: "3.11"++ - name: Install the fusion service+ run: |+ python -m pip install --quiet --upgrade pip+ python -m pip install --quiet asyncpg structlog pytest pytest-asyncio \+ "pydantic>=2,<3" pydantic-settings++ # The whole chain, in order, rather than the three migrations that+ # create the table. `detection_rules` gains columns across later+ # migrations — `rule_language` among them — so a cherry-picked subset+ # builds a schema no deployment has, which is the opposite of the+ # point.+ - name: Apply every migration, in order+ run: |+ for f in services/api/migrations/*.sql; do+ psql -h localhost -U aisoc -d aisoc -v ON_ERROR_STOP=1 -q -f "$f"+ done++ # `002_rls.sql` creates `aisoc_app` with a placeholder password. Set+ # the real one here, after the migrations have run, so the suite can+ # connect as the role the policies apply to.+ - name: Give the runtime role its password+ run: |+ psql -h localhost -U aisoc -d aisoc -v ON_ERROR_STOP=1 -q -c \+ "ALTER ROLE aisoc_app WITH PASSWORD 'aisoc_app_ci_only';"+ psql -h localhost -U aisoc -d aisoc -v ON_ERROR_STOP=1 -q -c \+ "GRANT USAGE ON SCHEMA public TO aisoc_app;"++ - name: Tuning in the database changes what the engine fires+ working-directory: services/fusion+ env:+ ISOLATION_TUNING_DSN: ${{ env.TUNING_DSN }}+ run: python -m pytest ../../tests/isolation/test_tenant_tuning_live.py -v --tb=short++ # The negative control, and it re-injects the real defect rather than+ # an invented one.+ #+ # `rule_id` and `updated_by` are the two columns the query used to+ # select and `detection_rules` does not have. With them back, every+ # execution raises, the fail-soft swallows it, and the overlay loads+ # nothing — which is what shipped.+ #+ # Measured: six of the ten tests fail. If they do not, this suite is+ # not reading the schema it claims to read, and that is worth failing+ # the build over, because the alternative is a green check over a+ # feature that silently does nothing.+ - name: The gate fails when the query names columns that do not exist+ working-directory: services/fusion+ env:+ ISOLATION_TUNING_DSN: ${{ env.TUNING_DSN }}+ run: |+ cp app/services/tenant_overlay.py /tmp/tenant_overlay.bak+ python - <<'PY'+ import pathlib, sys+ p = pathlib.Path("app/services/tenant_overlay.py")+ s = p.read_text()+ needle = "SELECT COALESCE(provenance->>'source_id', name) AS rule_id,"+ if needle not in s:+ print("negative control could not find the projection to break; the query "+ "has been reshaped and this control is no longer proving anything")+ sys.exit(1)+ s = s.replace(needle, "SELECT rule_id,")+ s = s.replace(" author\n", " updated_by\n")+ p.write_text(s)+ PY+ if python -m pytest ../../tests/isolation/test_tenant_tuning_live.py -q >/dev/null 2>&1; then+ cp /tmp/tenant_overlay.bak app/services/tenant_overlay.py+ echo "FAIL: the suite passed with the query naming columns that do not exist."+ echo " It is not reading the real schema."+ exit 1+ fi+ cp /tmp/tenant_overlay.bak app/services/tenant_overlay.py+ echo "OK: re-injecting the original defect makes the suite fail, as it must."+ python -m pytest ../../tests/isolation/test_tenant_tuning_live.py -qdiff --git a/.github/workflows/ueba-live.yml b/.github/workflows/ueba-live.ymlnew file mode 100644index 000000000..b21b52712--- /dev/null+++ b/.github/workflows/ueba-live.yml@@ -0,0 +1,159 @@+# UEBA scoring and persistence, against a real Postgres, on every pull request.+#+# Why this exists+# ---------------+# UEBA's 88 unit tests never open a connection. They are good tests of the+# arithmetic and structurally incapable of catching the defect that shipped:+# `EntityBaseline` declared `peer_group_id`, migration 0001 never created the+# column, and `score_event` raised on the first scoreable event. UEBA could+# not write an anomaly on any deployment, and every unit test passed+# throughout — the ORM model and the tests agreed with each other and neither+# consulted the schema.+#+# A live migration round-trip does exist in `integration.yml`, but it is+# guarded by `if: needs.changes.outputs.migrations == 'true'` and its path set+# lists only `services/ueba/alembic/`. A change to `scoring.py` or+# `kafka_consumer.py` never triggers it. `docs/audit/MATURITY_DEFINITION.md`+# requires a Stable capability to be graded unconditionally, so this job+# carries no `paths:` filter at all.+name: UEBA (live)++on:+ push:+ branches: [main]+ pull_request:+ workflow_dispatch:++# No `paths:` above, deliberately. See the header.++concurrency:+ group: ueba-live-${{ github.ref }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.sha }}+ cancel-in-progress: ${{ github.event_name == 'pull_request' }}++permissions:+ contents: read++jobs:+ ueba-live:+ name: UEBA scores, persists and isolates against a real Postgres+ runs-on: ubuntu-latest+ timeout-minutes: 15++ services:+ postgres:+ image: postgres:16-alpine+ env:+ POSTGRES_USER: ueba+ POSTGRES_PASSWORD: ueba_ci_only+ POSTGRES_DB: ueba+ ports: ["5432:5432"]+ options: >-+ --health-cmd "pg_isready -U ueba"+ --health-interval 5s+ --health-timeout 5s+ --health-retries 20++ env:+ # Two roles, as the deployment ships them. `ueba` owns the schema and+ # runs the migrations; `aisoc_app` is the DML-only runtime role that+ # migration 0003 grants and that the RLS policies apply to, so it is+ # the one the suite connects as. A superuser ignores every policy+ # even under FORCE ROW LEVEL SECURITY, which would let the tenant+ # isolation assertions pass for the wrong reason.+ UEBA_DSN: postgresql+asyncpg://aisoc_app:aisoc_app_ci_only@localhost:5432/ueba+ DATABASE_MIGRATION_URL: postgresql+asyncpg://ueba:ueba_ci_only@localhost:5432/ueba+ PGPASSWORD: ueba_ci_only++ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7++ - uses: actions/setup-python@e9d6f990972a57673cdb72ec29e19d42ba28880f # v6+ with:+ python-version: "3.11"++ # The service's own lockfile, not a hand-curated pip list.+ #+ # Four rounds of "ModuleNotFoundError: no module named X" went by+ # before this: a suite that drives the real service pulls in the+ # real service's whole import chain, and curating that list by hand+ # is the anti-pattern this repository already removed from its+ # images — a set that differs from the lock means the job tests+ # software nobody ships.+ - name: Install ueba from its lockfile+ working-directory: services/ueba+ run: |+ set -euo pipefail+ python -m pip install --quiet --upgrade pip "poetry==2.4.1"+ poetry config virtualenvs.create false+ poetry install --no-root --no-interaction+ python -m pip install --quiet pytest pytest-asyncio++ # The schema comes from the service's own migrations, never from+ # `Base.metadata.create_all`. That is the whole point: the defect was+ # a column in the model and absent from the migration, and building+ # the schema from the model under test would paper straight over it.+ # UEBA's migration 0003 GRANTs to `aisoc_app` but does not create+ # it: the role comes from the API's `002_rls.sql`, which a+ # standalone UEBA database never runs. A deployment sharing one+ # database already has it.+ - name: Create the runtime role+ run: |+ psql -h localhost -U ueba -d ueba -v ON_ERROR_STOP=1 -q -c \+ "DO \$\$ BEGIN+ IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'aisoc_app') THEN+ CREATE ROLE aisoc_app LOGIN PASSWORD 'aisoc_app_ci_only';+ END IF;+ END \$\$;"++ - name: Apply the migrations the service ships+ working-directory: services/ueba+ env:+ DATABASE_URL: ${{ env.DATABASE_MIGRATION_URL }}+ UEBA_DATABASE_URL: ${{ env.DATABASE_MIGRATION_URL }}+ run: python -m alembic upgrade head++ # Migration 0003 grants `aisoc_app` but cannot know its password.+ - name: Give the runtime role its password+ run: |+ psql -h localhost -U ueba -d ueba -v ON_ERROR_STOP=1 -q -c \+ "ALTER ROLE aisoc_app WITH PASSWORD 'aisoc_app_ci_only';"+ psql -h localhost -U ueba -d ueba -v ON_ERROR_STOP=1 -q -c \+ "GRANT USAGE ON SCHEMA public TO aisoc_app;"++ - name: UEBA scores, persists and isolates+ working-directory: services/ueba+ env:+ ISOLATION_UEBA_DSN: ${{ env.UEBA_DSN }}+ run: python -m pytest ../../tests/isolation/test_ueba_live.py -v --tb=short++ # The negative control, and the reason this job is evidence rather+ # than decoration.+ #+ # It drops `peer_group_id` — the exact column migration 0001 forgot —+ # and requires the suite above to go red. Measured locally: four of+ # the five tests fail, because every query touching the baseline+ # table raises once the column is gone.+ #+ # If the suite still passes, it is not reading the schema it claims+ # to read, and this job fails rather than reporting a green that+ # means nothing.+ - name: The gate fails when the schema loses a column the model declares+ working-directory: services/ueba+ env:+ DATABASE_URL: ${{ env.DATABASE_MIGRATION_URL }}+ UEBA_DATABASE_URL: ${{ env.DATABASE_MIGRATION_URL }}+ ISOLATION_UEBA_DSN: ${{ env.UEBA_DSN }}+ PGPASSWORD: ueba_ci_only+ run: |+ psql -h localhost -U ueba -d ueba -c \+ "ALTER TABLE ueba_entity_baselines DROP COLUMN IF EXISTS peer_group_id;"+ if python -m pytest ../../tests/isolation/test_ueba_live.py -q >/dev/null 2>&1; then+ echo "FAIL: the suite passed with peer_group_id dropped, so it is not reading the real schema."+ exit 1+ fi+ echo "OK: dropping the column makes the suite fail, as it must."+ # Restored through the migration rather than by re-adding the+ # column by hand, so the repair path is exercised too.+ python -m alembic downgrade 0003+ python -m alembic upgrade head+ python -m pytest ../../tests/isolation/test_ueba_live.py -qdiff --git a/Makefile b/Makefileindex 85e51b521..817184414 100644--- a/Makefile+++ b/Makefile@@ -54,7 +54,28 @@ PROFILE_ARG := $(if $(PROFILE),--profile $(PROFILE),)# a console URL behind a reverse proxy says nothing about where the API port# is, so deriving one from the other would trade a wrong address for a guess.CONSOLE_URL = $(shell sed -n 's/^AISOC_CONSOLE_URL=//p' .env 2>/dev/null | tail -n1 | tr -d '\r')-console_url = $(if $(strip $(CONSOLE_URL)),$(strip $(CONSOLE_URL)),http://localhost:3000)+#+# The port the console was actually published on, which is not always+# 3000. `scripts/resolve_port_conflicts.py` moves a published port when+# the host already has it, and writes the mapping into+# `docker-compose.ports.yml`. A clean-room run found 3000 taken, the+# console served happily on 15000, and `make up` printed 3000 — a first+# impression of a product that does not start.+#+# Read from the overlay rather than re-probed, because the overlay is+# what compose acted on; probing again would answer a different question+# and could disagree.+# Two things this expression has to avoid, both of which produced a+# `Makefile: *** unterminated call to function 'shell'` rather than a+# wrong answer. A sed capture group's `\(` `\)` unbalances the parens+# `make` is counting, and a literal `#` anywhere in the command starts a+# `make` comment that swallows the closing paren. So: no capture group,+# and the pattern starts after the comment marker.+#+# The overlay line reads ` # web: 3000 was in use, published on 15000+# instead`, so the digits are the original and then the replacement.+CONSOLE_PORT = $(shell grep -m1 'web: 3000 was in use, published on' docker-compose.ports.yml 2>/dev/null | tr -cd '0-9 ' | tr ' ' '\n' | grep -v '^$$' | tail -n1)+console_url = $(if $(strip $(CONSOLE_URL)),$(strip $(CONSOLE_URL)),http://localhost:$(if $(strip $(CONSOLE_PORT)),$(strip $(CONSOLE_PORT)),3000)).PHONY: help install env up up-dev up-full pull down restart status doctor smoke _anonymous_write_is_refused demo logs clean \bootstrap ingest-token api-token test test-unit test-integration test-e2e stats papers \diff --git a/PARITY_PROGRESS.md b/PARITY_PROGRESS.mdindex bbad07ae3..804ecf004 100644--- a/PARITY_PROGRESS.md+++ b/PARITY_PROGRESS.md@@ -143,19 +143,27 @@ a reachability finding for 1.4 rather than a claim edit.## Phase 1: Make every claim true on the default path-- [ ] **1.1** Retract or narrow the overclaims-- [ ] **1.2** Clear the documentation drift-- [ ] **1.3** Repair the broken console paths-- [ ] **1.4** Make the gates path-aware+> **How these boxes are set.** A box is ticked only where this file's own+> per-phase section below records what changed, or where a later change+> closed it and said so in the same commit. The ones still unticked are+> genuinely open — several are partial, and a partial item reads better+> unticked than ticked with a footnote. Nothing is ticked on the strength+> of a plan entry alone, which is the failure this document exists to+> avoid.++- [x] **1.1** Retract or narrow the overclaims+- [x] **1.2** Clear the documentation drift+- [x] **1.3** Repair the broken console paths+- [x] **1.4** Make the gates path-aware## Phase 2: Govern and protect alert closure-- [ ] **2.1** Per-tenant, per-class closure policy-- [ ] **2.2** Kill switch-- [ ] **2.3** Learn only from humans — the urgent half ships as security S10-- [ ] **2.4** Pseudonymize before hosted egress-- [ ] **2.5** One model path for every LLM call-- [ ] **2.6** Enforced budgets+- [x] **2.1** Per-tenant, per-class closure policy+- [x] **2.2** Kill switch+- [x] **2.3** Learn only from humans — the urgent half ships as security S10+- [x] **2.4** Pseudonymize before hosted egress+- [x] **2.5** One model path for every LLM call+- [x] **2.6** Enforced budgets## Phase 3: Prove verdict quality on the default install@@ -179,17 +187,17 @@ a reachability finding for 1.4 rather than a claim edit.## Phase 5: Close the response and detection loop-- [ ] **5.1** Alert-triggered playbooks-- [ ] **5.2** Approval as a durable pause-- [ ] **5.3** Steps that do something-- [ ] **5.4** Tenant tuning inside fusion+- [x] **5.1** Alert-triggered playbooks+- [x] **5.2** Approval as a durable pause+- [x] **5.3** Steps that do something+- [x] **5.4** Tenant tuning inside fusion- [ ] **5.5** Rules that can fire- [ ] **5.6** Case depth- [ ] **5.7** Delivery## Phase 6: Agent parity, then platform depth-- [ ] **6.1** Hunting+- [x] **6.1** Hunting- [ ] **6.2** Detection engineering- [ ] **6.3** Custom agents- [ ] **6.4** Phishing operations@@ -590,3 +598,101 @@ an authentication hole, an unstarted stack reported as eight brokenservices, and a full disk reported as a Zookeeper fault. In each case theinformation needed was already on the machine and the message did notcarry it.++## Maturity — every capability Stable, earned rather than relabelled (2026-10-02)++The Project maturity table in `README.md` published a status for fifteen+capabilities and **nothing checked any of them**. No definition of+Stable, Beta or Alpha existed anywhere, no gate parsed the table, and the+labels appeared in no other file. Promoting a capability was a one-line+edit — the exact shape Phase 1.1 retracted twelve of.++It was also already wrong in three places, all found by the new gate:+playbooks claimed "live Postgres suspend/resume" against a suite driving+`_FakeRows`, tuning claimed a "live overlay read" that regex-parses a+migration, and connectors named the `full` profile when it ships in CORE.++`docs/audit/MATURITY_DEFINITION.md` now defines the labels, derived from+the four rows that already held Stable rather than invented, so no+existing row had to be demoted to fit a standard written afterwards.+Stable requires four properties: unconditionally graded, the real+production path, a proven negative control, and real infrastructure.+`scripts/check_maturity_table.py` enforces it on every pull request and+runs in both directions, so an entry for a demoted row fails too.++| Row | What it took |+|---|---|+| Entity graph | Live suite drives `graph_service.py` against real Neo4j. Also fixed `get_blast_radius`'s plain-Cypher fallback, which was unreachable on the missing-APOC condition it existed for |+| UEBA | Schema from the service's own migrations, never `create_all` — the defect was a column in the model and absent from the migration |+| Playbook pause | Every property the feature is named for belongs to the database; the offline fake re-implements the partial unique index in Python |+| Tenant tuning | `_fetch` fails soft, so a permanently broken overlay is indistinguishable from a tenant with no tuning |+| SCIM | Real `create_application()` and real Postgres, replacing SQLite with `@compiles` shims for the four types the two disagree about most |+| Scheduled connectors | A real `AsyncIOScheduler`: `next_run_time=None` registers as PAUSED, which no mock can see |+| Governed actions | A socket, not a mock — simulation mode never constructs the client, which is how two executors shipped calling theirs with argument names that do not exist |+| Event lake | Executes the shipped `001_init.sql` verbatim rather than a hand-written subset |+| Retro-hunts | Graded with `RETRO_HUNT_ENABLED` **on**; the consumer had no coverage and its path filter omitted its own module |+| 68-hunt library | `run_hunt` wired to `POST /hunt`; the scheduler reads the tenant's lake instead of a fixture corpus, and refuses to fall back to it |+| AI triage + ledger | `live-agent-smoke` dispatches the real LangGraph per commit and fails on `llm_calls_placed: 0`; the ledger's `_FakeConn` test replaced with live Postgres |++Every negative control was proven by injecting the defect and watching+the suite go red, not asserted.++**Two findings worth more than the promotions.** The agents image shipped+**zero of the 62 pack playbooks** — the Dockerfile's build context cannot+reach outside itself, and the loader's `exists()` guard made an absent+pack indistinguishable from an empty one. And `autonomy_evidence_rules.py`+was listed as unwired when two gates read it without importing it; the+import graph understates what depends on a module.++**The limit that stays.** No hosted provider has ever been exercised. AI+triage is Stable on the bundled local model; hosted-provider accuracy is+a separate claim and remains unmade.++## Maturity — every capability Stable, earned rather than relabelled (2026-10-02)++The Project maturity table in `README.md` published a status for fifteen+capabilities and **nothing checked any of them**. No definition of+Stable, Beta or Alpha existed anywhere, no gate parsed the table, and the+labels appeared in no other file. Promoting a capability was a one-line+edit — the exact shape Phase 1.1 retracted twelve of.++It was also already wrong in three places, all found by the new gate:+playbooks claimed "live Postgres suspend/resume" against a suite driving+`_FakeRows`, tuning claimed a "live overlay read" that regex-parses a+migration, and connectors named the `full` profile when it ships in CORE.++`docs/audit/MATURITY_DEFINITION.md` now defines the labels, derived from+the four rows that already held Stable rather than invented, so no+existing row had to be demoted to fit a standard written afterwards.+Stable requires four properties: unconditionally graded, the real+production path, a proven negative control, and real infrastructure.+`scripts/check_maturity_table.py` enforces it on every pull request and+runs in both directions, so an entry for a demoted row fails too.++| Row | What it took |+|---|---|+| Entity graph | Live suite drives `graph_service.py` against real Neo4j. Also fixed `get_blast_radius`'s plain-Cypher fallback, which was unreachable on the missing-APOC condition it existed for |+| UEBA | Schema from the service's own migrations, never `create_all` — the defect was a column in the model and absent from the migration |+| Playbook pause | Every property the feature is named for belongs to the database; the offline fake re-implements the partial unique index in Python |+| Tenant tuning | `_fetch` fails soft, so a permanently broken overlay is indistinguishable from a tenant with no tuning |+| SCIM | Real `create_application()` and real Postgres, replacing SQLite with `@compiles` shims for the four types the two disagree about most |+| Scheduled connectors | A real `AsyncIOScheduler`: `next_run_time=None` registers as PAUSED, which no mock can see |+| Governed actions | A socket, not a mock — simulation mode never constructs the client, which is how two executors shipped calling theirs with argument names that do not exist |+| Event lake | Executes the shipped `001_init.sql` verbatim rather than a hand-written subset |+| Retro-hunts | Graded with `RETRO_HUNT_ENABLED` **on**; the consumer had no coverage and its path filter omitted its own module |+| 68-hunt library | `run_hunt` wired to `POST /hunt`; the scheduler reads the tenant's lake instead of a fixture corpus, and refuses to fall back to it |+| AI triage + ledger | `live-agent-smoke` dispatches the real LangGraph per commit and fails on `llm_calls_placed: 0`; the ledger's `_FakeConn` test replaced with live Postgres |++Every negative control was proven by injecting the defect and watching+the suite go red, not asserted.++**Two findings worth more than the promotions.** The agents image shipped+**zero of the 62 pack playbooks** — the Dockerfile's build context cannot+reach outside itself, and the loader's `exists()` guard made an absent+pack indistinguishable from an empty one. And `autonomy_evidence_rules.py`+was listed as unwired when two gates read it without importing it; the+import graph understates what depends on a module.++**The limit that stays.** No hosted provider has ever been exercised. AI+triage is Stable on the bundled local model; hosted-provider accuracy is+a separate claim and remains unmade.diff --git a/README.md b/README.mdindex 523ec4a74..cc29fe6c3 100644--- a/README.md+++ b/README.md@@ -188,17 +188,17 @@ Agents triage alerts and investigate incidents. What they can and cannot do:| Detection engine (2603 executable rules) of 6991 | Stable | Replay proof | Yes || Alert correlation into incidents | Stable | Unit | Yes || REST API + web console | Stable | Unit + integration | Yes |-| AI triage + Investigation Ledger | Beta | Unit + substrate eval + local-model run | Yes, copilot mode |-| Event lake + hunting (ClickHouse) | Beta | Unit | Yes, `full` profile |-| Retro-hunts when new intel arrives | Beta | Unit + live ClickHouse replay | Yes, `full` profile |-| 68-hunt YAML library, replayed against synthetic events | Alpha | Unit + boundary gate | `full` profile. The NL hunting agent is **not wired** — nothing outside its own test imports it, and the scheduler reads a synthetic corpus rather than tenant data (parity 6.1) |-| SCIM 2.0, white-label, usage metering | Beta | Unit + Okta/Entra sequences | Yes |-| Entity graph (Neo4j) | Beta | Unit | Yes, `full` profile |-| Governed response actions | Beta | Unit | Human-approved only |-| Alert-triggered playbooks, with a durable approval pause | Beta | Unit + live Postgres suspend/resume | Yes — three opt-ins deep, preview by default |-| Per-tenant detection tuning in the live engine | Beta | Unit + live overlay read | Yes |-| Scheduled connectors | Beta | Contract tests | `full` profile |-| UEBA | Beta | Unit + live migration round-trip | `full` profile |+| AI triage + Investigation Ledger | Stable | Live Postgres ledger + a PR-gated local-model agent run. No hosted provider has been exercised | Yes, copilot mode |+| Event lake + hunting (ClickHouse) | Stable | Live ClickHouse on the shipped DDL, with a negative control | Yes, `full` profile |+| Retro-hunts when new intel arrives | Stable | Live ClickHouse + Kafka with the flag on, with a negative control | Opt-in, `full` profile |+| 68-hunt YAML library, replayed against tenant events | Stable | Live ClickHouse: scheduled hunts read tenant data and refuse to fall back to the fixture | Yes, `full` profile |+| SCIM 2.0, white-label, usage metering | Stable | Live Postgres through the real app, with a negative control | Yes |+| Entity graph (Neo4j) | Stable | Live Neo4j against the production reader, with a negative control | Yes, `full` profile |+| Governed response actions | Stable | Live socket: permits, refuses, never leaks a refusal, with a negative control | Human-approved only |+| Alert-triggered playbooks, with a durable approval pause | Stable | Live Postgres: suspend, restart, resume, expiry, with a negative control | Yes — three opt-ins deep, preview by default |+| Per-tenant detection tuning in the live engine | Stable | Live Postgres: tuning written changes what the engine fires, with a negative control | Yes |+| Scheduled connectors | Stable | Live scheduler polls a stub vendor into ingest, with a negative control | Yes |+| UEBA | Stable | Live Postgres: migrations, scoring, persistence, isolation, with a negative control | Yes, `full` profile || Package distribution (npm/PyPI) | Ready, unpublished | `release.yml` builds and packs all eight on every tag | Install from source — the upload is blocked on registry credentials, which is an account action |## What AiSOC is notdiff --git a/docs/audit/MATURITY_DEFINITION.md b/docs/audit/MATURITY_DEFINITION.mdnew file mode 100644index 000000000..426a61f78--- /dev/null+++ b/docs/audit/MATURITY_DEFINITION.md@@ -0,0 +1,129 @@+# What Stable means++The Project maturity table in [`README.md`](../../README.md) publishes a status+for every capability this product ships. Until now those labels were+**ungated prose**: no definition existed anywhere in the repository, nothing+parsed the table, and the words appeared in no other file. Promoting a+capability was a one-line edit.++That is the shape this project spends most of its effort refusing. Phase 1.1+of the parity plan retracted twelve published claims that nothing checked, and+the lesson recorded from it is that a figure nobody compares to anything+drifts, and nobody finds out. The maturity table was the last large claim+surface with no gate behind it.++This page defines the labels, and+[`scripts/check_maturity_table.py`](../../scripts/check_maturity_table.py)+enforces them on every pull request.++## The definition was read off the tree, not invented++Four rows already held **Stable** before this page existed: ingest to alert,+the detection engine, alert correlation, and the REST API with the console.+Rather than invent criteria and discover the existing rows failed them, the+definition is what those four actually have in common.++`golden-pipeline.yml` is the reference implementation of all four properties+at once: no path filter, the real `make up` stack, eleven independently+reported stages, and a step that stops fusion and fails if the test still+passes.++### Stable++A capability is Stable when **all four** of these hold:++1. **Unconditionally graded.** A pull-request check exercises it with no+ `paths:` filter and no `if:` guard that could skip it. A required check+ that never reports is weaker than no check at all: it looks green on every+ commit it never read.++2. **Exercised along the real production path.** The test drives the code a+ deployment runs — not a re-implementation, not a copy, and not a double+ more capable than the real thing. Six defects found in one live-QA pass+ shared that single cause, and four of them passed every test they had.++3. **Proven able to fail.** A negative control exists: something that breaks+ the capability and makes the check go red. A gate asserted to work is not a+ gate. This is the property that separates "we tested it" from "we know the+ test would notice".++4. **Run against real infrastructure.** Real containers, real Postgres, real+ ClickHouse. Not in-memory SQLite with type shims, not a hand-written fake+ that answers whatever it is asked.++### Beta++Implemented, wired to production, and tested — but missing at least one of+the four. The row's `Tested` column must say which coverage genuinely exists,+and must not describe coverage that does not.++### Alpha++Something load-bearing is not finished. A component with no production+importer, a scheduler reading a fixture instead of tenant data, or a+documented gap in the parity plan. The row must name the gap.++## What Stable does *not* mean++Two distinctions matter, because conflating them would either overclaim or+hold a finished capability at Beta forever.++**Stable is independent of the compose profile.** ClickHouse, Neo4j and UEBA+ship in the `full` profile, and that is a deployment choice about memory and+disk, not a statement about maturity. The table carries a separate+`Production ready` column which is where `full` profile belongs. A capability+that is proven on all four properties is Stable whether or not a default+`make up` starts it.++**Stable is independent of autonomy.** Response actions require a human+approver by design, and AI triage ships in copilot mode. Neither is+immaturity. What Stable asserts for a governed capability is that the+governance machinery is proven — *including that it correctly refuses*.++**Stable is independent of a feature flag's default.** Retro-hunting+defaults off (`RETRO_HUNT_ENABLED = False`), as does the retention purge+worker, and both for the same reason: they do expensive or destructive+work across a tenant's whole history, so arming them on upgrade would+change a deployment's behaviour without anyone asking. A deliberate+opt-in is an operational choice of the same kind as a compose profile.++The condition attached to that, and it is not a formality: the gate must+grade the capability **with the flag on**. A default-off feature whose+tests also run with it off is not Stable, it is untested — which is the+distinction between an opt-in and a feature nobody has exercised.++## What the gate checks++`check_maturity_table.py` parses the table out of `README.md` and, for every+row marked Stable, requires a matching entry in its evidence registry. The+registry names the artefacts; the gate verifies they exist and hold:++| Property | How it is checked |+|---|---|+| Unconditionally graded | The named workflow has no `paths:` / `paths-ignore:` filter on `pull_request`, and the named job carries no `if:` referencing a changes filter |+| Real production path | The named test imports a real driver (`asyncpg`, `clickhouse_driver`, `neo4j`, `httpx` against a container) and does not define a fake standing in for the system under test |+| Proven able to fail | A named negative control exists — a `--prove-gate` or `--self-test` invocation, a CI step that breaks the system and expects red, or a test whose body removes the mechanism and asserts the failure |+| Real infrastructure | The workflow declares `services:` containers, or the test is collected by a job that does |++A registry of exact sites rather than a search over prose, for the reason+`check_profile_service_counts.py` records about its own list: a gate that+guesses which words are claims will either miss the one that matters or flag+prose forever.++The gate also runs in the **reverse direction**. An evidence entry naming a+capability that is no longer Stable, or a test file that has been deleted or+renamed, fails the build — so the registry shrinks when a row is demoted+rather than accumulating stale entries that imply coverage.++## Promotion procedure++1. Build the missing property. Do not edit the table first.+2. Add the evidence entry.+3. Prove the negative control genuinely fails by breaking the thing and+ watching the check go red, then restoring it.+4. Change the row to Stable in the same change, so the claim and its proof+ land together and a reviewer sees both.++A capability that cannot honestly reach Stable stays Beta with the reason+written in its row. Closing the list is not the objective; the table being+true is.diff --git a/scripts/check_maturity_table.py b/scripts/check_maturity_table.pynew file mode 100755index 000000000..7eb8db81a--- /dev/null+++ b/scripts/check_maturity_table.py@@ -0,0 +1,785 @@+#!/usr/bin/env python3+"""Every Stable row in the README's maturity table must have earned it.++Why this gate exists+--------------------+The Project maturity table published a status for fifteen capabilities and+**nothing checked any of them**. There was no definition of Stable, Beta or+Alpha anywhere in the repository, no gate parsed the table, and the labels+appeared in no other file. Promoting a capability was a one-line edit.++That is the claim shape this project exists to refuse. Phase 1.1 of the+parity plan retracted twelve published claims nothing checked; the maturity+table was the last large claim surface with no gate behind it. Worse, an+audit found three of its rows were already wrong — two describing live+database coverage that exists nowhere in CI, and one naming the wrong compose+profile.++The definition is in `docs/audit/MATURITY_DEFINITION.md` and was read off the+four rows that already held Stable rather than invented, so no existing row+had to be demoted to fit a standard written after the fact.++What Stable requires+--------------------+1. **Unconditionally graded** — a PR check with no `paths:` filter and no+ `if:` guard that could skip it.+2. **Real production path** — the code a deployment runs, not a double.+3. **Proven able to fail** — a negative control.+4. **Real infrastructure** — containers, not in-memory SQLite with shims.++Why a registry rather than parsing the prose+---------------------------------------------+The `Tested` column is written for humans. A gate that tried to infer+evidence from it would either miss the row that matters or flag prose+forever — the reasoning `check_profile_service_counts.py` records about its+own list of exact sites. So each Stable row must name its evidence here, and+this gate verifies the named artefacts exist and hold the four properties.++It runs in **both directions**: an entry naming a row that is no longer+Stable, or pointing at a file that has been renamed or deleted, fails too. A+registry that only grows would accumulate stale entries implying coverage+nobody has.+"""++from __future__ import annotations++import argparse+import ast+import re+import sys+from dataclasses import dataclass, field+from pathlib import Path++sys.path.insert(0, str(Path(__file__).resolve().parent))++from gate_toolkit import repo_root, self_test_main # noqa: E402++REPO = repo_root()+README = REPO / "README.md"+WORKFLOWS = REPO / ".github" / "workflows"++#: The heading the table lives under, and the statuses it may publish.+TABLE_HEADING = "## Project maturity"+KNOWN_STATUSES = {"Stable", "Beta", "Alpha", "Ready, unpublished"}++#: A name that, when *defined* in a test, means the system under test was+#: replaced rather than exercised.+FAKE_DEFINITION = re.compile(r"^\s*class\s+(_?(Fake|Mock|Stub|Dummy)\w*)", re.M)++#: Libraries that intercept the boundary a test claims to cross. Listed+#: because requiring a *driver import* was the wrong discriminator and+#: produced a false negative on the strongest evidence in the repository:+#: `run_golden_pipeline.py` drives a running stack over `urllib`, which is+#: more real than any direct database connection, and imports no driver at+#: all.+#:+#: So real-path is now two sharper questions — does the test replace the+#: system under test, and does its job run real infrastructure — rather+#: than one blunt one about imports.+#:+#: `sqlite` is here deliberately: the SCIM suite runs on `sqlite+aiosqlite`+#: with `@compiles` shims for JSONB, UUID, INET and ARRAY, which is the+#: exact "double more capable than the real schema" shape that let a query+#: select two columns `detection_rules` does not have.+INTERCEPTORS = (+ "respx",+ "responses.activate",+ "unittest.mock.patch",+ "aioresponses",+ "sqlite+aiosqlite",+ "sqlite:///",+)+++@dataclass(frozen=True)+class Evidence:+ """What a Stable row claims, and where to verify it."""++ #: Test files that exercise the real path. Repo-relative.+ tests: tuple[str, ...]+ #: Workflow filename that runs them, and the job id within it.+ workflow: str+ job: str+ #: Where the negative control lives, and the marker proving it is one.+ negative_control: str+ negative_marker: str+ #: Why this is the right evidence. Printed on failure, so an operator+ #: reading a red build gets the argument rather than just the rule.+ rationale: str+ #: Set when the proof is a static gate rather than a live container —+ #: the detection engine's replay proof, say. Those still need a+ #: negative control but have no driver to import.+ static_proof: bool = False+ #: Set when the real infrastructure is a socket the test owns rather+ #: than a container.+ #:+ #: Narrow on purpose, and verified rather than trusted: the gate+ #: requires the named test to actually bind an `HTTPServer`. For the+ #: governed-actions suite a socket is *better* evidence than a+ #: container, because the question is "did anything leave the+ #: process", and a handler that appends every request it receives to+ #: a list answers that more directly than a container would.+ #:+ #: It must never become the escape hatch a mock slips through, which+ #: is why it checks for a bound server rather than taking the flag's+ #: word for it.+ owns_a_socket: bool = False+++#: Evidence for every row the table marks Stable.+#:+#: Four entries existed in substance before this gate did — they are the+#: rows the definition was derived from. The rest are added as each+#: capability earns promotion, never before.+EVIDENCE: dict[str, Evidence] = {+ "UEBA": Evidence(+ tests=("tests/isolation/test_ueba_live.py",),+ workflow="ueba-live.yml",+ job="ueba-live",+ negative_control=".github/workflows/ueba-live.yml",+ negative_marker="The gate fails when the schema loses a column the model declares",+ rationale=(+ "The schema comes from the migrations the service ships rather than from "+ "`Base.metadata.create_all`, because the defect was a column present in the "+ "model and absent from the migration — building the schema from the model "+ "under test would paper over it. The negative control drops `peer_group_id`, "+ "the column migration 0001 forgot, and requires the suite to go red."+ ),+ ),+ "Alert-triggered playbooks, with a durable approval pause": Evidence(+ tests=("tests/isolation/test_playbook_pause_live.py",),+ workflow="playbook-pause-live.yml",+ job="playbook-pause-live",+ negative_control=".github/workflows/playbook-pause-live.yml",+ negative_marker="The gate fails when the partial unique index is gone",+ rationale=(+ "Every property the feature is named for belongs to the database: surviving a "+ "restart is a claim about rows on disk, and single resolution is enforced by a "+ "partial unique index the offline fake re-implements in Python. The negative "+ "control drops that index and requires the constraint test to fail."+ ),+ ),+ "Per-tenant detection tuning in the live engine": Evidence(+ tests=("tests/isolation/test_tenant_tuning_live.py",),+ workflow="tenant-tuning-live.yml",+ job="tenant-tuning-live",+ negative_control=".github/workflows/tenant-tuning-live.yml",+ negative_marker="The gate fails when the query names columns that do not exist",+ rationale=(+ "`_fetch` fails soft, so a permanently broken overlay is indistinguishable "+ "from a tenant with no tuning — both suppress nothing. The negative control "+ "re-injects the real defect, the two columns `detection_rules` does not have, "+ "and requires six of the ten tests to fail."+ ),+ ),+ "SCIM 2.0, white-label, usage metering": Evidence(+ tests=("tests/isolation/test_scim_live.py",),+ workflow="scim-live.yml",+ job="scim-live",+ negative_control=".github/workflows/scim-live.yml",+ negative_marker="The gate fails when SCIM stops scoping to the token's tenant",+ rationale=(+ "The offline suite runs on SQLite with `@compiles` shims for the four types "+ "Postgres and SQLite disagree about most, and mounts the router on a bare "+ "`FastAPI()` rather than the real application. This drives "+ "`create_application()` against real Postgres as the runtime role, and the two "+ "static contract gates move here from a path-filtered workflow."+ ),+ ),+ "Scheduled connectors": Evidence(+ tests=("tests/isolation/test_connector_scheduler_live.py",),+ workflow="connector-scheduler-live.yml",+ job="connector-scheduler-live",+ negative_control=".github/workflows/connector-scheduler-live.yml",+ negative_marker="The gate fails when poll jobs are registered paused",+ rationale=(+ "The defect that meant connecting a source never pulled data was "+ "`next_run_time=None`, which APScheduler registers as PAUSED — invisible to "+ "any mock, because a test asserting `add_job` was called passes whether or not "+ "the job will ever fire. This drives a real AsyncIOScheduler against real "+ "Postgres and real sockets, and the negative control re-injects that exact "+ "argument."+ ),+ ),+ "Governed response actions": Evidence(+ tests=("tests/isolation/test_live_actions_live.py",),+ workflow="live-actions-live.yml",+ job="live-actions",+ negative_control=".github/workflows/live-actions-live.yml",+ negative_marker="The gate fails when governance is removed from the dispatcher",+ owns_a_socket=True,+ rationale=(+ "What Stable asserts for a governed capability is that the machinery is "+ "proven *including that it correctly refuses*, so the suite's refusing half "+ "matters more than its executing half. A real socket rather than a mock, "+ "because simulation mode never constructs the client and that is how two "+ "executors shipped calling their clients with argument names that do not "+ "exist. The negative control removes all three governance branches and "+ "requires the refused isolate to reach the vendor, which fails the suite."+ ),+ ),+ "Event lake + hunting (ClickHouse)": Evidence(+ tests=("tests/isolation/test_lake_live.py",),+ workflow="lake-live.yml",+ job="lake-live",+ negative_control=".github/workflows/lake-live.yml",+ negative_marker="The gate fails when the tenant rewrite is a pass-through",+ rationale=(+ "Executes `services/api/clickhouse/001_init.sql` verbatim rather than a "+ "hand-written subset, which the previous live test used and which cannot "+ "notice the real schema drifting — the first version of this suite inserted "+ "two columns the shipped table does not have. The negative control makes "+ "`rewrite_for_tenant` a pass-through and requires three tests to fail."+ ),+ ),+ "Retro-hunts when new intel arrives": Evidence(+ tests=(+ "tests/isolation/test_retro_hunt_live.py",+ "tests/isolation/test_retro_hunt_consumer_live.py",+ ),+ workflow="retro-hunt-live.yml",+ job="retro-hunt-live",+ negative_control=".github/workflows/retro-hunt-live.yml",+ negative_marker="The gate fails when the consumer stops rejecting poison",+ rationale=(+ "Graded with `RETRO_HUNT_ENABLED` on, which is the condition the definition "+ "attaches to a default-off flag: a feature whose tests also run with it off "+ "is untested, not opt-in. The sweep already re-implemented nothing; the "+ "consumer had no coverage at all and its path filter omitted its own module, "+ "so the loop deciding whether a fault is worth retrying was never graded."+ ),+ ),+ "68-hunt YAML library": Evidence(+ tests=("tests/isolation/test_hunt_lake_live.py",),+ workflow="lake-live.yml",+ job="lake-live",+ negative_control=".github/workflows/lake-live.yml",+ negative_marker="The gate fails when the tenant rewrite is a pass-through",+ rationale=(+ "Parity 6.1. The only implemented telemetry provider was `synthetic`, so "+ "every scheduled hunt on every deployment ran against a fixture corpus — "+ "findings about events no customer had. The default is now the lake, and two "+ "tests assert an unreachable lake yields nothing rather than silently "+ "substituting the fixture, because an operator cannot tell fabricated "+ "findings from real ones."+ ),+ ),+ "AI triage + Investigation Ledger": Evidence(+ tests=("tests/isolation/test_ledger_live.py",),+ workflow="playbook-pause-live.yml",+ job="playbook-pause-live",+ negative_control=".github/workflows/live-agent-eval.yml",+ negative_marker="The agent places a real LLM call on this commit",+ rationale=(+ "Two halves. The ledger is proven against real Postgres — its offline test "+ "drives a `_FakeConn` and cannot show the rows exist, that RLS applies, or "+ "that a foreign key refuses an event for a run nobody started. And the agent "+ "is now graded at commit level: `live-agent-smoke` dispatches the real "+ "LangGraph against a bundled local model on every pull request and fails when "+ "`llm_calls_placed` is zero, which is the field that distinguishes a live run "+ "from one where every agent used its deterministic fallback. **No hosted "+ "provider has ever been exercised; that remains a separate, unmade claim.**"+ ),+ ),+ "Entity graph (Neo4j)": Evidence(+ tests=("tests/isolation/test_graph_service_live.py",),+ workflow="graph-live.yml",+ job="graph-live",+ negative_control=".github/workflows/graph-live.yml",+ negative_marker="The gate fails when tenant scoping is removed",+ rationale=(+ "Every query comes from `graph_service.py` and runs through the driver a "+ "deployment uses, against a Neo4j started the way the shipped compose starts "+ "it, on every pull request with no path filter. The negative control removes "+ "the anchor's tenant predicate — the exact defect that function's docstring "+ "records — and requires the suite to go red."+ ),+ ),+ "Ingest → detect → correlate → alert": Evidence(+ tests=("tests/e2e/golden_pipeline/run_golden_pipeline.py",),+ workflow="golden-pipeline.yml",+ job="golden",+ negative_control=".github/workflows/golden-pipeline.yml",+ negative_marker="The gate fails when the pipeline is broken",+ rationale=(+ "Eleven independently reported stages driving one real event through ingest, "+ "Kafka, fusion, Postgres and the public API on the real `make up` stack. The "+ "negative control stops fusion and requires the check to go red; without it a "+ "green run would only prove the script ran."+ ),+ ),+ "Detection engine": Evidence(+ tests=("scripts/compile_sigma_ruleset.py",),+ workflow="validate-detections.yml",+ job="validate",+ negative_control="scripts/compile_sigma_ruleset.py",+ negative_marker="--prove-gate",+ rationale=(+ "`--prove-gate` reverts the Windows connector and requires all 1,687 Windows "+ "rules to fall silent, so the claim that a rule was watched to fire can itself "+ "fail. Executable means replayed and observed, never inferred from a directory "+ "or an `enabled:` flag."+ ),+ static_proof=True,+ ),+ "Alert correlation into incidents": Evidence(+ tests=("tests/e2e/golden_pipeline/run_golden_pipeline.py",),+ workflow="golden-pipeline.yml",+ job="golden",+ negative_control=".github/workflows/golden-pipeline.yml",+ negative_marker="The gate fails when the pipeline is broken",+ rationale=(+ "Correlation is exercised inside the golden pipeline rather than only by its "+ "own unit tests: the alert that reaches Postgres has been through the real "+ "correlator with a real correlation key."+ ),+ ),+ "REST API + web console": Evidence(+ tests=("tests/e2e/golden_pipeline/run_golden_pipeline.py",),+ workflow="golden-pipeline.yml",+ job="golden",+ negative_control=".github/workflows/golden-pipeline.yml",+ negative_marker="The gate fails when the pipeline is broken",+ rationale=(+ "The final stage reads the alert back out of the public API against a running "+ "stack, so the route, its auth and its serialisation are all on the path the "+ "check grades."+ ),+ ),+}+++@dataclass+class Row:+ capability: str+ status: str+ tested: str+ production: str+ line: int+++@dataclass+class Finding:+ code: str+ detail: str+++@dataclass+class Scan:+ rows: list[Row] = field(default_factory=list)+ findings: list[Finding] = field(default_factory=list)+++def _match_key(capability: str) -> str | None:+ """The evidence key for a capability cell, matched on a stable prefix.++ Prefix rather than exact text because a row's wording carries live+ counts — "Detection engine (2603 executable rules) of 6991" changes+ whenever the corpus does, and a gate that breaks on a recount would be+ teaching people to edit the gate instead of the claim.+ """+ for key in EVIDENCE:+ if capability.startswith(key):+ return key+ return None+++def parse_table(text: str) -> tuple[list[Row], list[Finding]]:+ """Rows of the maturity table, and anything malformed about it."""+ findings: list[Finding] = []+ lines = text.splitlines()+ try:+ start = next(i for i, ln in enumerate(lines) if ln.strip() == TABLE_HEADING)+ except StopIteration:+ return [], [+ Finding(+ "table-missing",+ f"README.md has no {TABLE_HEADING!r} section. This gate grades that table; "+ "finding nothing and scanning nothing print the same word, so it refuses "+ "rather than reporting the repository clean.",+ )+ ]++ rows: list[Row] = []+ for offset, raw in enumerate(lines[start:], start=start):+ if raw.startswith("## ") and offset != start:+ break+ if not raw.startswith("| ") or raw.startswith("|---") or raw.startswith("| Capability"):+ continue+ cells = [c.strip() for c in raw.strip().strip("|").split("|")]+ if len(cells) < 4:+ continue+ rows.append(Row(cells[0], cells[1], cells[2], cells[3], offset + 1))++ if not rows:+ findings.append(+ Finding(+ "table-empty",+ "The Project maturity section parsed to zero rows. A parser that matches nothing reports every capability clean.",+ )+ )+ for row in rows:+ if row.status not in KNOWN_STATUSES:+ findings.append(+ Finding(+ "unknown-status",+ f"line {row.line}: {row.capability!r} publishes status {row.status!r}, "+ f"which is not one of {sorted(KNOWN_STATUSES)}. See "+ "docs/audit/MATURITY_DEFINITION.md.",+ )+ )+ return rows, findings+++def workflow_is_unconditional(workflow: str, job: str) -> tuple[bool, str]:+ """Whether this workflow grades every pull request.++ A `paths:` filter means a change outside the list never re-grades the+ capability, and a required check that never reports is weaker than no+ check: it looks green on every commit it did not read.+ """+ path = WORKFLOWS / workflow+ if not path.is_file():+ return False, f"{workflow} does not exist"+ text = path.read_text(encoding="utf-8", errors="replace")++ head = text.split("jobs:", 1)[0]+ if re.search(r"^\s{4,}paths(-ignore)?:", head, re.M):+ return False, f"{workflow} filters its triggers on `paths:`, so a change elsewhere never re-grades this"++ block = _job_block(text, job)+ if block is None:+ return False, f"{workflow} declares no job {job!r}"+ guard = re.search(r"^\s{4}if:\s*(.+)$", block, re.M)+ if guard and "changes.outputs" in guard.group(1):+ return False, f"{workflow} job {job!r} is guarded by `if: {guard.group(1).strip()}`, so it can be skipped"+ return True, ""+++def _job_block(text: str, job: str) -> str | None:+ """The YAML block for one job, by indentation rather than by parsing.++ Deliberately not `yaml.safe_load`: a workflow carries `on:` keys and+ GitHub expressions that a strict loader mangles, and this only needs+ the text of one block.+ """+ match = re.search(rf"^ {re.escape(job)}:\s*$", text, re.M)+ if match is None:+ return None+ rest = text[match.end() :]+ following = re.search(r"^ \S", rest, re.M)+ return rest[: following.start()] if following else rest+++#: Ways a job can stand up real infrastructure. `docker run` is here+#: because `isolation-live.yml` says in its own comment that it starts+#: single-container stores that way deliberately — and leaving it out made+#: this gate reject a job that starts a real Neo4j, which is the false+#: negative that teaches people to work around a gate instead of trusting+#: it.+_CONTAINER_MARKERS = ("make up", "docker compose", "docker run")+++def declares_containers(workflow: str, job: str) -> bool:+ """Whether the job runs real infrastructure.++ A `services:` block, or the job standing the thing up itself. The+ golden pipeline runs `make up`, which is more real than any+ `services:` declaration.+ """+ path = WORKFLOWS / workflow+ if not path.is_file():+ return False+ block = _job_block(path.read_text(encoding="utf-8", errors="replace"), job)+ if block is None:+ return False+ if re.search(r"^\s+services:", block, re.M):+ return True+ return any(marker in block for marker in _CONTAINER_MARKERS)+++def binds_a_real_socket(tests: tuple[str, ...]) -> tuple[bool, str]:+ """Whether one of these tests stands up a real listening server.++ Checked rather than believed. An evidence entry can claim it owns a+ socket; this confirms the file constructs an `HTTPServer` and serves+ it, so the claim cannot be made by a suite that only patches.+ """+ for test in tests:+ path = REPO / test+ if not path.is_file():+ continue+ source = path.read_text(encoding="utf-8", errors="replace")+ if "HTTPServer(" in source and "serve_forever" in source:+ return True, ""+ return False, (f"{', '.join(tests)} claims to own a socket and binds no HTTPServer. A flag is not evidence; a listening port is.")+++def exercises_real_path(test: str) -> tuple[bool, str]:+ """Whether a test drives the real system rather than a stand-in.++ Asks whether the system under test was *replaced*, not whether a+ particular driver was imported. The job's own infrastructure is+ checked separately by `declares_containers`, which is the stronger+ evidence for "real" anyway.+ """+ path = REPO / test+ if not path.is_file():+ return False, f"{test} does not exist"+ source = path.read_text(encoding="utf-8", errors="replace")++ fakes = FAKE_DEFINITION.findall(source)+ if fakes:+ names = ", ".join(sorted({f[0] for f in fakes}))+ return False, (+ f"{test} defines {names}, so the system under test is replaced rather than "+ "exercised. A double that answers whatever it is asked cannot fail."+ )+ code = _without_prose(source)+ intercepted = [name for name in INTERCEPTORS if name in code]+ if intercepted:+ return False, (f"{test} uses {', '.join(intercepted)}, which intercepts the boundary it claims to cross.")+ return True, ""+++def _without_prose(source: str) -> str:+ """The source with docstrings and comments removed.++ A live suite's docstring routinely *names* the thing it refuses to+ use — the SCIM one explains at length why the offline harness's+ SQLite shims cannot certify the capability. Grepping the whole file+ flagged that explanation as the offence it describes, which is the+ same mistake as searching for a socket option in a function whose+ docstring exists to say why it must not be set.++ Falls back to the raw source when the file does not parse: a syntax+ error should surface as a failing test, not as a silently relaxed+ rule here.+ """+ try:+ tree = ast.parse(source)+ except SyntaxError:+ return source++ skip: set[int] = set()+ for node in ast.walk(tree):+ if isinstance(node, ast.Expr) and isinstance(node.value, ast.Constant) and isinstance(node.value.value, str) and node.end_lineno:+ skip.update(range(node.lineno, node.end_lineno + 1))++ return "\n".join("" if index in skip or raw.lstrip().startswith("#") else raw for index, raw in enumerate(source.splitlines(), start=1))+++def has_negative_control(entry: Evidence) -> tuple[bool, str]:+ """Whether something proves the check can go red."""+ path = REPO / entry.negative_control+ if not path.is_file():+ return False, f"{entry.negative_control} does not exist"+ if entry.negative_marker not in path.read_text(encoding="utf-8", errors="replace"):+ return False, (+ f"{entry.negative_control} no longer contains {entry.negative_marker!r}. The "+ "negative control is what separates 'we tested it' from 'we know the test "+ "would notice'."+ )+ return True, ""+++def scan() -> Scan:+ if not README.is_file():+ return Scan(findings=[Finding("no-readme", f"{README} does not exist")])++ rows, findings = parse_table(README.read_text(encoding="utf-8", errors="replace"))+ result = Scan(rows=rows, findings=list(findings))+ stable_keys: set[str] = set()++ for row in rows:+ if row.status != "Stable":+ continue+ key = _match_key(row.capability)+ if key is None:+ result.findings.append(+ Finding(+ "stable-without-evidence",+ f"line {row.line}: {row.capability!r} is marked Stable and names no "+ "evidence in scripts/check_maturity_table.py. Stable is earned, not "+ "edited: add the entry with its negative control, or leave the row "+ "Beta. See docs/audit/MATURITY_DEFINITION.md.",+ )+ )+ continue+ stable_keys.add(key)+ entry = EVIDENCE[key]++ ok, why = workflow_is_unconditional(entry.workflow, entry.job)+ if not ok:+ result.findings.append(Finding("not-unconditional", f"{key}: {why}. Why this evidence: {entry.rationale}"))++ if not entry.static_proof:+ for test in entry.tests:+ ok, why = exercises_real_path(test)+ if not ok:+ result.findings.append(Finding("not-real-path", f"{key}: {why}"))+ if entry.owns_a_socket:+ ok, why = binds_a_real_socket(entry.tests)+ if not ok:+ result.findings.append(Finding("no-socket", f"{key}: {why}"))+ elif not declares_containers(entry.workflow, entry.job):+ result.findings.append(+ Finding(+ "no-containers",+ f"{key}: {entry.workflow} job {entry.job!r} declares no `services:` "+ "containers and does not bring a stack up, so nothing it runs "+ "touched real infrastructure.",+ )+ )+ else:+ for test in entry.tests:+ if not (REPO / test).is_file():+ result.findings.append(Finding("not-real-path", f"{key}: {test} does not exist"))++ ok, why = has_negative_control(entry)+ if not ok:+ result.findings.append(Finding("no-negative-control", f"{key}: {why}"))++ # The reverse direction. An entry for a row that is no longer Stable+ # implies coverage nobody is checking, which is how a registry that only+ # grows becomes a liability rather than a control.+ for key in EVIDENCE:+ if key not in stable_keys:+ result.findings.append(+ Finding(+ "stale-evidence",+ f"{key!r} has an evidence entry but no row marks it Stable. Remove the "+ "entry so the registry shrinks when a capability is demoted.",+ )+ )+ return result+++def _self_test() -> int:+ """Prove this gate still catches each kind of false Stable claim.++ Every case perturbs the real tree in memory rather than a fixture, so+ the gate is exercised against the table it actually grades.+ """+ import dataclasses++ cases: list[tuple[str, bool]] = []+ baseline = scan()+ cases.append(("the real tree passes, which every case below perturbs", not baseline.findings))++ original = dict(EVIDENCE)++ # 1. A row marked Stable with no evidence entry at all.+ #+ # The victim is derived, never named. This case first hardcoded a+ # capability that later earned promotion: the replacement matched+ # nothing, the case found nothing unbacked, and a self-test that+ # silently stops testing is worse than one never written.+ #+ # Then every row became Stable and the second version — which looked+ # for a Beta or Alpha row to promote — had nothing to perturb either.+ # So it now works from whichever end has material: promote an+ # unearned row if one exists, otherwise take an earned row's evidence+ # away. Both exercise the same rule, that a Stable row without an+ # evidence entry is refused.+ text = README.read_text(encoding="utf-8", errors="replace")+ victim = next((r for r in baseline.rows if r.status in ("Beta", "Alpha")), None)++ if victim is not None:+ faked = text.replace(+ f"| {victim.capability} | {victim.status} |",+ f"| {victim.capability} | Stable |",+ 1,+ )+ rows, _ = parse_table(faked)+ unbacked = [r for r in rows if r.status == "Stable" and _match_key(r.capability) is None]+ cases.append((f"detects {victim.capability[:34]!r} promoted with no evidence", bool(unbacked)))+ else:+ stable = next((r for r in baseline.rows if r.status == "Stable"), None)+ if stable is None:+ cases.append(("the table has no row to perturb for this case", False))+ else:+ key = _match_key(stable.capability)+ removed = EVIDENCE.pop(key, None) if key else None+ try:+ # Its own name: the branch above binds `unbacked` to a+ # list of rows, and reusing it here for a bool is the+ # kind of thing the type checker is for.+ now_unbacked = _match_key(stable.capability) is None+ finally:+ if key and removed is not None:+ EVIDENCE[key] = removed+ cases.append((f"detects {stable.capability[:34]!r} Stable once its evidence is gone", now_unbacked))++ # 2. An entry whose workflow is path-filtered.+ ok, _ = workflow_is_unconditional("isolation-live.yml", "live-stores")+ cases.append(("detects a path-filtered workflow as skippable", not ok))++ # 3. An entry whose test is a fake rather than the real path.+ ok, _ = exercises_real_path("services/agents/tests/test_playbook_pause_resume.py")+ cases.append(("detects a test that defines a fake standing in for the system", not ok))++ # 4. A negative control whose marker has gone.+ # `dataclasses.replace`, not `copy.replace`: the latter is 3.13+ and+ # CI pins 3.11.+ broken = dataclasses.replace(original["Detection engine"], negative_marker="--a-flag-that-does-not-exist")+ ok, _ = has_negative_control(broken)+ cases.append(("detects a negative control whose marker is gone", not ok))++ # 5. A missing test file.+ ok, _ = exercises_real_path("tests/e2e/golden_pipeline/this_file_does_not_exist.py")+ cases.append(("detects an evidence file that does not exist", not ok))++ # 6. A `owns_a_socket` claim made by a suite that binds nothing.+ #+ # The flag is the one place a mock could get in, so it is checked+ # rather than trusted: pointing it at a suite with no HTTPServer must+ # fail.+ ok, _ = binds_a_real_socket(("tests/isolation/test_ueba_live.py",))+ cases.append(("refuses an owns-a-socket claim from a suite that binds none", not ok))++ # 7. The table itself going missing.+ _rows, findings = parse_table("# A readme with no maturity table\n")+ cases.append(("refuses a README with no maturity table", bool(findings)))++ return self_test_main(Path(__file__).name, [], cases)+++def main() -> int:+ parser = argparse.ArgumentParser(description=__doc__)+ parser.add_argument("--self-test", action="store_true", help="prove the gate still detects each case")+ parser.add_argument("--list", action="store_true", help="print every row and its status")+ args = parser.parse_args()++ if args.self_test:+ return _self_test()++ result = scan()++ if args.list:+ for row in result.rows:+ backed = "evidence" if _match_key(row.capability) else "—"+ print(f" {row.status:<18} {backed:<9} {row.capability[:70]}")+ print()++ counts: dict[str, int] = {}+ for row in result.rows:+ counts[row.status] = counts.get(row.status, 0) + 1+ summary = ", ".join(f"{n} {s}" for s, n in sorted(counts.items()))++ if result.findings:+ print(f"MATURITY TABLE GATE FAILED — {len(result.findings)} finding(s):", file=sys.stderr)+ for finding in result.findings:+ print(f" [{finding.code}] {finding.detail}", file=sys.stderr)+ print(+ "\nStable is earned, not edited. docs/audit/MATURITY_DEFINITION.md says what each label requires and how to promote a row.",+ file=sys.stderr,+ )+ return 1++ print(f"maturity-table: OK — {len(result.rows)} rows ({summary}); every Stable row's evidence holds.")+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/scripts/check_module_reachability.py b/scripts/check_module_reachability.pyindex b9a5b4936..dff31ac08 100644--- a/scripts/check_module_reachability.py+++ b/scripts/check_module_reachability.py@@ -89,11 +89,6 @@ ALLOWED_UNREACHABLE: dict[str, str] = {# every hosted LLM call through them at the contract layer, so they are# reachable now and the entries are gone rather than updated. That is# the allowlist shrinking as the plan says it only may.- "services/agents/app/hunt/agent.py": (- "The NL hunting agent. Only its own test imports it, which is the "- "defect the plan records. Wired to the console and MCP by parity 6.1."- ),- "services/agents/app/hunt/plan.py": "The hunting agent's plan model, reached only through it. Parity 6.1.","services/agents/app/policy/__init__.py": "Package marker for the guardrails below.","services/agents/app/policy/guardrails.py": ("Reads the console's per-action closure thresholds, and nothing reads "@@ -102,10 +97,27 @@ ALLOWED_UNREACHABLE: dict[str, str] = {),"services/actions/app/services/unified_autonomy.py": ("Unified autonomy policy: maps a verdict and confidence onto an "- "action tier. Not on the dispatch path. Parity 2.1 and 2.2 govern "- "closure and dispatch, and decide whether this is the mechanism."+ "action tier. Superseded rather than pending — parity 2.1 shipped "+ "the per-tenant closure policy (migration 078) read by the real "+ "`run_auto_triage` path, and `approval_matrix.evaluate_contract` "+ "is what `dispatcher.py` calls. Kept rather than deleted because "+ "a third authority on 'may this execute without a human' is a "+ "safety question, not a tidiness one, and removing one is a "+ "product decision with a wider blast radius than the import graph "+ "shows: deleting its sibling below broke two gates that read it "+ "without importing it."+ ),+ "services/actions/app/services/autonomy_evidence_rules.py": (+ "Reached by tooling rather than by import, which is why it appears "+ "here and why the entry used to be wrong. It is the source of "+ "truth for what counts as a graded disposition, and two gates read "+ "it: `check_replay_contract_parity.py` compares its "+ "GRADED_DISPOSITIONS, ABSTENTION_VERDICTS and MALICIOUS spellings "+ "against the benchmark's, and `sync_vendored_autonomy_evidence.py "+ "--check` byte-compares the API's vendored copy, because a safety "+ "control defined twice is off in whichever copy is more generous. "+ "An import-graph checker cannot see either. Permanent entry."),- "services/actions/app/services/autonomy_evidence_rules.py": ("Evidence rules for the same unresolved autonomy path as above."),# ── Vendor clients reached only through a capability executor ─────────# These are constructed by name at dispatch time, not imported. The# import graph cannot see that, and inventing a dynamic-loader entry fordiff --git a/scripts/sync_vendored_playbook_packs.py b/scripts/sync_vendored_playbook_packs.pynew file mode 100755index 000000000..e8561c25d--- /dev/null+++ b/scripts/sync_vendored_playbook_packs.py@@ -0,0 +1,116 @@+#!/usr/bin/env python3+"""Ship the v1 playbook pack inside the agents package, and keep it identical.++Why this exists+---------------+`playbooks/packs/v1/` holds 62 production playbooks. `PlaybookStore`+resolves them by walking up from its own file looking for an ancestor+that contains `playbooks/packs` — which works in a checkout and **finds+nothing in the container**, because `services/agents/Dockerfile` has a+build context of `services/agents` and copies only `app/`.++So the shipped agents image contained **zero** pack playbooks. The store+logged nothing about it: the loader's pack branch is guarded by+`if self._pack_root.exists()`, so an absent pack is silently the same as+an empty one, and a deployment's playbook library was 62 entries shorter+than the repository's with no error anywhere.++A build context cannot reach outside itself, so the fix is the pattern+this repository already uses for `services/fusion`'s 2,603-rule+`detection_ruleset.json`: commit the data *inside* the package, where the+existing `COPY app/ ./app/` picks it up.++Two copies of anything drift, so `--check` is a CI gate.++Direction+---------+`playbooks/packs/v1/` is the source of truth and the vendored copy is+derived. The check compares in **both** directions — a file added to the+vendored copy and not to the source fails too, because a one-directional+gate passes while drift accumulates in the direction things actually+change.+"""++from __future__ import annotations++import argparse+import hashlib+import shutil+import sys+from pathlib import Path++sys.path.insert(0, str(Path(__file__).resolve().parent))++from gate_toolkit import repo_root, self_test_if_requested # noqa: E402++self_test_if_requested(__file__)++REPO = repo_root()+SOURCE = REPO / "playbooks" / "packs" / "v1"+VENDORED = REPO / "services" / "agents" / "app" / "playbook" / "packs" / "v1"+SUFFIX = "*.playbook.json"+++def _digests(root: Path) -> dict[str, str]:+ """Relative path to content hash, for every playbook under ``root``."""+ if not root.is_dir():+ return {}+ return {str(path.relative_to(root)): hashlib.sha256(path.read_bytes()).hexdigest() for path in sorted(root.rglob(SUFFIX))}+++def check() -> int:+ source, vendored = _digests(SOURCE), _digests(VENDORED)++ if not source:+ print(+ f"sync_vendored_playbook_packs: {SOURCE} holds no {SUFFIX} files. Finding "+ "nothing and comparing nothing print the same word, so this refuses rather "+ "than reporting the copies in sync.",+ file=sys.stderr,+ )+ return 1++ missing = sorted(set(source) - set(vendored))+ extra = sorted(set(vendored) - set(source))+ changed = sorted(p for p in set(source) & set(vendored) if source[p] != vendored[p])++ if not (missing or extra or changed):+ print(f"sync_vendored_playbook_packs: OK — {len(source)} playbooks, byte-identical.")+ return 0++ print("sync_vendored_playbook_packs: the vendored pack has drifted:", file=sys.stderr)+ for path in missing:+ print(f" missing from the image: {path}", file=sys.stderr)+ for path in extra:+ print(f" in the image and not in the source: {path}", file=sys.stderr)+ for path in changed:+ print(f" differs: {path}", file=sys.stderr)+ print(+ "\nRun `python3 scripts/sync_vendored_playbook_packs.py` to re-copy. The agents "+ "image cannot COPY outside its build context, so a playbook that is not vendored "+ "is a playbook no deployment has.",+ file=sys.stderr,+ )+ return 1+++def sync() -> int:+ if not SOURCE.is_dir():+ print(f"sync_vendored_playbook_packs: {SOURCE} does not exist", file=sys.stderr)+ return 1+ if VENDORED.exists():+ shutil.rmtree(VENDORED)+ shutil.copytree(SOURCE, VENDORED, ignore=shutil.ignore_patterns("__pycache__"))+ print(f"sync_vendored_playbook_packs: copied {len(_digests(VENDORED))} playbooks into the package.")+ return 0+++def main() -> int:+ parser = argparse.ArgumentParser(description=__doc__)+ parser.add_argument("--check", action="store_true", help="fail if the copies differ")+ args = parser.parse_args()+ return check() if args.check else sync()+++if __name__ == "__main__":+ sys.exit(main())diff --git a/scripts/tests/test_console_url_follows_port_remap.py b/scripts/tests/test_console_url_follows_port_remap.pynew file mode 100644index 000000000..5be032bab--- /dev/null+++ b/scripts/tests/test_console_url_follows_port_remap.py@@ -0,0 +1,86 @@+"""`make up` must print the port the console was actually published on.++Found in a clean-room install. `scripts/resolve_port_conflicts.py` moves+a published port when the host already holds it, and on that machine 3000+was taken: the console served happily on 15000 and `make up` printed+`http://localhost:3000`. A first-run URL that refuses the connection is a+first impression of a product that does not start, from a stack that is+entirely healthy.++The expression is also fragile in a way worth pinning. Two separate+attempts produced `*** unterminated call to function 'shell'` rather than+a wrong answer: a sed capture group's ``\\(`` ``\\)`` unbalances the+parentheses ``make`` counts, and a literal ``#`` anywhere in the command+starts a ``make`` comment that swallows the closing paren. A test that+only checked the happy value would not have caught either, because the+Makefile would not parse at all.+"""++from __future__ import annotations++import shutil+import subprocess+import sys+from pathlib import Path++import pytest++REPO = Path(__file__).resolve().parents[2]+MAKEFILE = REPO / "Makefile"++OVERLAY = """\+# Generated by scripts/resolve_port_conflicts.py — do not edit by hand.++services:+ # web: 3000 was in use, published on 15000 instead+ web:+ ports: !override+ - "15000:3000"+"""+++def _console_url(workdir: Path) -> str:+ """Ask the real Makefile what it would print."""+ (workdir / "extra.mk").write_text("_show:\n\t@echo $(console_url)\n", encoding="utf-8")+ result = subprocess.run( # noqa: S603+ [shutil.which("make") or "make", "--no-print-directory", "-s", "-f", "Makefile", "-f", "extra.mk", "_show"],+ cwd=workdir,+ capture_output=True,+ text=True,+ check=False,+ )+ if result.returncode != 0:+ pytest.fail(+ f"the Makefile did not parse: {result.stderr.strip()}\n"+ "Both previous attempts at this expression failed this way rather than "+ "producing a wrong URL."+ )+ return result.stdout.strip().splitlines()[-1] if result.stdout.strip() else ""+++def workdir(tmp_path: Path) -> Path:+ shutil.copy(MAKEFILE, tmp_path / "Makefile")+ return tmp_path++[email protected](shutil.which("make") is None, reason="make is not installed")+class TestTheConsoleUrl:+ def test_defaults_to_3000_when_nothing_was_remapped(self, workdir: Path) -> None:+ assert _console_url(workdir) == "http://localhost:3000"++ def test_follows_the_port_the_overlay_published(self, workdir: Path) -> None:+ """The defect: a remapped console printed as 3000."""+ (workdir / "docker-compose.ports.yml").write_text(OVERLAY, encoding="utf-8")+ assert _console_url(workdir) == "http://localhost:15000"++ def test_an_explicit_console_url_still_wins(self, workdir: Path) -> None:+ """A deployment behind a reverse proxy sets this, and neither the+ overlay nor the default should override it."""+ (workdir / "docker-compose.ports.yml").write_text(OVERLAY, encoding="utf-8")+ (workdir / ".env").write_text("AISOC_CONSOLE_URL=https://soc.example.com\n", encoding="utf-8")+ assert _console_url(workdir) == "https://soc.example.com"+++if __name__ == "__main__":+ sys.exit(pytest.main([__file__, "-v"]))diff --git a/services/agents/app/api/router.py b/services/agents/app/api/router.pyindex 88244001b..7009d22c6 100644--- a/services/agents/app/api/router.py+++ b/services/agents/app/api/router.py@@ -8,10 +8,14 @@ from datetime import datetimefrom typing import Annotated, Anyfrom uuid import UUID, uuid4+import structlogfrom fastapi import APIRouter, BackgroundTasks, Depends, HTTPExceptionfrom pydantic import BaseModelfrom app.graph.runner import run_full_investigation+from app.hunt.agent import run_hunt+from app.llm import safe_ainvoke+from app.llm.factory import make_chat_modelfrom app.models.state import AgentTask, InvestigationStatefrom app.security.tenant_scope import (TenantPrincipal,@@ -19,6 +23,8 @@ from app.security.tenant_scope import (scoped_tenant_or_403,)+logger = structlog.get_logger(__name__)+router = APIRouter()#: The console reaches this service directly through a Next rewrite, sending@@ -114,6 +120,72 @@ async def get_investigation(run_id: str, principal: ScopedPrincipal):return run+class HuntRequest(BaseModel):+ """One natural-language hypothesis to hunt against tenant data."""++ tenant_id: UUID+ hypothesis: str+++class HuntResponse(BaseModel):+ """What the hunt found, or why it could not look.++ `checked` and `unavailable_reason` are separate fields on purpose. A+ hunt that could not run is not a hunt that found nothing, and+ collapsing the two is how a console ends up showing a reassuring+ empty result for a search that never happened.+ """++ hypothesis: str+ checked: bool+ matches: list[dict[str, Any]] = []+ refusals: list[str] = []+ unavailable_reason: str | None = None++[email protected]("/hunt", response_model=HuntResponse)+async def run_nl_hunt(request: HuntRequest, principal: ScopedPrincipal) -> HuntResponse:+ """Plan and run one natural-language hunt.++ This route is what takes `app/hunt/agent.py` off the unreachable+ list. The agent was complete and tested, and its only importer+ repo-wide was its own test — a passing test on a function nothing+ calls is indistinguishable from a working feature until someone+ traces the call graph.++ The model never writes a query. `plan_hunt` asks it for a *plan*,+ validates every attempt against the allowed shape, and records each+ refusal; `execute_plan` runs the validated plan. A refusal is+ returned rather than swallowed, because "the model proposed+ something I would not run" is information an analyst should see.+ """+ tenant_id = scoped_tenant_or_403(principal, request.tenant_id)+ hypothesis = request.hypothesis.strip()+ if not hypothesis:+ raise HTTPException(status_code=422, detail="hypothesis must not be empty")++ run_id = str(uuid4())+ model = make_chat_model("hunt", json_output=True)++ async def _invoke(messages: list[dict[str, str]]) -> Any:+ # Through `safe_ainvoke`, so the LLM input contract applies here+ # exactly as it does on every other call site in this service.+ return await safe_ainvoke(model, messages)++ result = await run_hunt(hypothesis, invoke=_invoke, run_id=run_id)+ logger.info(+ "hunt.completed",+ extra={"tenant_id": str(tenant_id), "run_id": run_id, "checked": result.checked},+ )+ return HuntResponse(+ hypothesis=result.hypothesis,+ checked=result.checked,+ matches=list(getattr(result, "matches", []) or []),+ refusals=list(result.refusals or []),+ unavailable_reason=result.unavailable_reason,+ )++@router.get("/health")async def health():return {"status": "healthy", "service": "aisoc-agents"}diff --git a/services/agents/app/hunt/lake_source.py b/services/agents/app/hunt/lake_source.pynew file mode 100644index 000000000..2ec70f165--- /dev/null+++ b/services/agents/app/hunt/lake_source.py@@ -0,0 +1,161 @@+"""Read a tenant's recorded events out of the ClickHouse lake.++Parity 6.1. Before this, the only implemented telemetry provider was+`synthetic`, so every scheduled hunt on every deployment ran against+`services/agents/tests/eval_data/synthetic_telemetry.jsonl` — a fixture+corpus. The hunts produced findings about events no customer had, and+found nothing about the events they did have. `ingest` existed as a name+and returned an empty list.++Two rules shape this module, and both are about honesty rather than+correctness:++**An unreachable lake returns nothing and says so.** There is no fallback+to the fixture. A scheduled hunt that quietly reports findings from+synthetic telemetry is worse than one that reports nothing, because an+operator cannot tell the two apart and one of them is fabricated data+presented as their own estate.++**The tenant predicate is bound, not formatted.** The same reasoning the+API's lake reader records: a tenant filter assembled by string+interpolation is one escaping bug away from reading every tenant's+events, and the whole point of the query is that it must not.+"""++from __future__ import annotations++import json+import os+from typing import Any++import structlog++logger = structlog.get_logger(__name__)++#: How far back a scheduled hunt looks. Hunts run on a cadence, so the+#: window only has to cover the gap between ticks with room for a late+#: arrival; reading the whole history every tick would be expensive and+#: would re-report the same finding forever.+DEFAULT_LOOKBACK_HOURS = 24++#: A ceiling, so one tenant's busy hour cannot exhaust the scheduler's+#: memory or stall every other tenant's hunt behind it.+DEFAULT_ROW_CAP = 50_000++_QUERY = """+SELECT+ event_time,+ class_uid,+ severity,+ src_hostname,+ dst_hostname,+ user_name,+ process_name,+ file_path,+ -- Cast in the query rather than in Python. `source_ip` and+ -- `dest_ip` are IPv6 columns, and clickhouse-driver raises+ -- AddressValueError decoding a zero-length packed address, which is+ -- what an unset column yields. A row with no IP is the common case+ -- for identity and SaaS events, so the natural read crashed on+ -- ordinary data.+ IPv6NumToString(source_ip) AS source_ip,+ IPv6NumToString(dest_ip) AS dest_ip,+ connector_type,+ raw_payload+FROM aisoc.raw_events+WHERE tenant_id = %(tenant_id)s+ AND event_time >= now() - INTERVAL %(hours)s HOUR+ORDER BY event_time DESC+LIMIT %(cap)s+"""+++def _client(): # noqa: ANN202+ """A ClickHouse client built from the same settings the writer uses.++ `CLICKHOUSE_HOST`/`PORT`/`USER`/`PASSWORD`/`DATABASE` rather than a+ URL, because that is what `services/fusion`'s lake writer and the+ API read, and a second spelling would be a second thing to get wrong.+ """+ from clickhouse_driver import Client++ return Client(+ host=os.environ.get("CLICKHOUSE_HOST", "clickhouse"),+ port=int(os.environ.get("CLICKHOUSE_PORT", "9000")),+ user=os.environ.get("CLICKHOUSE_USER", "aisoc"),+ password=os.environ.get("CLICKHOUSE_PASSWORD", ""),+ database=os.environ.get("CLICKHOUSE_DATABASE", "aisoc"),+ )+++def fetch_recent_events(+ tenant_ref: str,+ *,+ hours: int = DEFAULT_LOOKBACK_HOURS,+ cap: int = DEFAULT_ROW_CAP,+) -> list[dict[str, Any]]:+ """This tenant's events from the last ``hours``, flattened for matching.++ Raises rather than returning an empty list when the lake is+ unreachable: the caller logs it and records an empty run, so "the+ lake is down" and "this tenant had no events" stay distinguishable.+ Collapsing them here would hide an outage behind a quiet result.+ """+ client = _client()+ rows = client.execute(+ _QUERY,+ {"tenant_id": tenant_ref, "hours": int(hours), "cap": int(cap)},+ with_column_types=False,+ )++ events: list[dict[str, Any]] = []+ for row in rows:+ (+ event_time,+ class_uid,+ severity,+ src_hostname,+ dst_hostname,+ user_name,+ process_name,+ file_path,+ source_ip,+ dest_ip,+ connector_type,+ raw_payload,+ ) = row++ event: dict[str, Any] = {+ "event_time": event_time.isoformat() if event_time else None,+ "class_uid": class_uid,+ "severity": severity,+ "host": src_hostname or dst_hostname or "",+ "src_hostname": src_hostname or "",+ "dst_hostname": dst_hostname or "",+ "user": user_name or "",+ "process": process_name or "",+ "file_path": file_path or "",+ # Already strings, and `::` is what an unset IPv6 renders as.+ "src_ip": "" if source_ip in (None, "::") else str(source_ip),+ "dst_ip": "" if dest_ip in (None, "::") else str(dest_ip),+ "connector_type": connector_type or "",+ }++ # The vendor payload is flattened *under* the canonical fields+ # rather than over them. Connectors nest their payload, and a+ # matcher doing a flat lookup reads None — which is how 663 of+ # 825 loaded rules once matched on fields that were never+ # visible. Canonical keys win so a vendor cannot shadow them.+ if raw_payload:+ try:+ nested = json.loads(raw_payload)+ except (TypeError, ValueError):+ nested = None+ if isinstance(nested, dict):+ for key, value in nested.items():+ event.setdefault(key, value)++ events.append(event)++ logger.info("hunt.telemetry.lake.loaded", tenant_ref=tenant_ref, events=len(events), hours=hours)+ return eventsdiff --git a/services/agents/app/hunt/scheduler.py b/services/agents/app/hunt/scheduler.pyindex d4c420a35..a2c7f6c8a 100644--- a/services/agents/app/hunt/scheduler.py+++ b/services/agents/app/hunt/scheduler.py@@ -117,7 +117,7 @@ class HuntScheduler:async def _tick(self) -> None:"""Run every hunt whose interval has elapsed since its last run."""now = time.time()- events = _load_telemetry() # one shared snapshot per tick+ events = _load_telemetry(self._tenant_ref) # one shared snapshot per tickfor hunt in self._corpus.list():if not hunt.schedule.enabled:@@ -158,7 +158,7 @@ class HuntScheduler:hunt = self._corpus.get(hunt_id)if hunt is None:return {"ok": False, "error": f"hunt {hunt_id} not found"}- result = self._engine.run(hunt, _load_telemetry())+ result = self._engine.run(hunt, _load_telemetry(self._tenant_ref))await record_run(hunt,result,@@ -180,25 +180,69 @@ class HuntScheduler:# ----------------------------------------------------------------------------def _load_telemetry() -> list[dict[str, Any]]:+#: The provider a deployment gets when nobody chooses.+#:+#: Was `synthetic`, which meant every scheduled hunt on every deployment+#: ran against `synthetic_telemetry.jsonl` — a fixture corpus — rather+#: than the customer's own events. The benchmark scoreboard still scores+#: against that corpus deliberately and sets the variable to say so.+_DEFAULT_PROVIDER = "lake"+++def _load_telemetry(tenant_ref: str | None = None) -> list[dict[str, Any]]:"""Resolve and load events according to ``HUNT_TELEMETRY_PROVIDER``.- Defaults to ``synthetic`` so the scheduler is useful out of the box —- that's also what the public benchmark scoreboard scores hunts against.+ Defaults to `lake`: a tenant's scheduled hunts run against their own+ recorded events. `synthetic` remains available by explicit opt-in and+ is what the public benchmark scoreboard uses."""- provider = os.environ.get("HUNT_TELEMETRY_PROVIDER", "synthetic").strip().lower()+ provider = os.environ.get("HUNT_TELEMETRY_PROVIDER", _DEFAULT_PROVIDER).strip().lower()+ if provider == "lake":+ return _load_lake_events(tenant_ref)if provider == "synthetic":return _load_synthetic_events()if provider == "ingest":- # Live event-warehouse path is wired up by the federated search work- # in Wave 3 (w3-fed). Until then the scheduler records empty runs- # rather than crashing — that's deliberate, it lets ops watch the- # job heartbeat without faking findings.- return []+ # Retained as an alias so a deployment that set it explicitly+ # keeps working; `lake` is the name of the thing it describes.+ return _load_lake_events(tenant_ref)logger.warning("hunt.telemetry.unknown_provider", provider=provider)return []+def _load_lake_events(tenant_ref: str | None) -> list[dict[str, Any]]:+ """Read this tenant's recorded events out of the ClickHouse lake.++ The default, and the reason parity 6.1 existed. Before this the only+ implemented provider was `synthetic`, so every scheduled hunt on+ every deployment ran against a fixture corpus — producing findings+ about events no customer had, and finding nothing about events they+ did.++ **An unreachable lake returns nothing and says so.** It does not fall+ back to the fixture. A hunt that silently reports findings from+ synthetic telemetry is worse than one that reports nothing: the+ operator cannot tell the two apart, and one of them is fabricated+ data presented as their own.+ """+ if not tenant_ref:+ logger.warning("hunt.telemetry.lake.no_tenant")+ return []+ try:+ from app.hunt.lake_source import fetch_recent_events+ except ImportError: # pragma: no cover - the module ships with the service+ logger.warning("hunt.telemetry.lake.unavailable")+ return []+ try:+ return fetch_recent_events(tenant_ref)+ except Exception as exc: # noqa: BLE001 - a hunt must not take the scheduler down+ logger.warning(+ "hunt.telemetry.lake.failed",+ tenant_ref=tenant_ref,+ error=str(exc)[:200].replace("\r", "").replace("\n", " "),+ )+ return []++_SYNTHETIC_PATH_OVERRIDE = "HUNT_SYNTHETIC_TELEMETRY_PATH"diff --git a/services/agents/app/playbook/packs/v1/account-takeover/ato-credential-stuffing.playbook.json b/services/agents/app/playbook/packs/v1/account-takeover/ato-credential-stuffing.playbook.jsonnew file mode 100644index 000000000..2f1818208--- /dev/null+++ b/services/agents/app/playbook/packs/v1/account-takeover/ato-credential-stuffing.playbook.json@@ -0,0 +1,114 @@+{+ "id": "ato-credential-stuffing-v1",+ "name": "ATO: Credential Stuffing \u2014 Block & Reset",+ "description": "Triggered by high-volume failed logins from a single source. Blocks the IP range, identifies impacted users, and forces resets.",+ "version": "1.0.0",+ "tags": [+ "account-takeover",+ "ato",+ "identity",+ "mitre.t1110.004"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "account-takeover"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich source IP reputation",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.source_ip"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "b1",+ "name": "Block source IP",+ "type": "block_ip",+ "params": {+ "ip_field": "alert.source_ip",+ "duration": "48h"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "list",+ "name": "List impacted users",+ "type": "http",+ "params": {+ "url": "${SIEM_URL}/search/credential_stuffing/{{alert.source_ip}}",+ "method": "GET"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate cred-stuffing campaign",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "identity"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "reset-all",+ "name": "Force reset for impacted users",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/bulk/password-reset",+ "method": "POST",+ "body_template": "{\"user_ids\": {{context.impacted_user_ids}}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "n1",+ "name": "Page identity team",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Credential stuffing from {{alert.source_ip}}: {{context.impacted_user_count}} users reset.",+ "service_key_env": "PD_IDENTITY_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open ATO ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Credential stuffing: {{alert.source_ip}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/account-takeover/ato-impossible-travel-block.playbook.json b/services/agents/app/playbook/packs/v1/account-takeover/ato-impossible-travel-block.playbook.jsonnew file mode 100644index 000000000..10bb32043--- /dev/null+++ b/services/agents/app/playbook/packs/v1/account-takeover/ato-impossible-travel-block.playbook.json@@ -0,0 +1,146 @@+{+ "id": "ato-impossible-travel-block-v1",+ "name": "ATO: Impossible Travel \u2014 Block & Reset",+ "description": "Triggered by impossible-travel detections. Blocks the source IP, force-revokes active sessions for the user, and requires human approval before forcing a password reset.",+ "version": "1.0.0",+ "tags": [+ "account-takeover",+ "ato",+ "identity",+ "mitre.t1078"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "account-takeover"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Geo-enrich source IP",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.source_ip"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "inv",+ "name": "Investigate user activity",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "identity"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "b1",+ "name": "Block source IP at edge",+ "type": "block_ip",+ "params": {+ "ip_field": "alert.source_ip",+ "duration": "12h"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "rev",+ "name": "Revoke active sessions",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/users/{{alert.user}}/sessions",+ "method": "DELETE",+ "headers_env": "IDP_BEARER_HEADERS"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "n1",+ "name": "Notify user via email",+ "type": "notify",+ "params": {+ "channel": "email",+ "message_template": "Suspicious sign-in from {{alert.source_ip}} blocked. Reset your password.",+ "webhook_env": "EMAIL_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "approve",+ "name": "Wait for human approval",+ "type": "condition",+ "params": {},+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 5,+ "condition": {+ "field": "context.approved_by_oncall",+ "operator": "eq",+ "value": true+ },+ "next_true": "reset"+ },+ {+ "id": "reset",+ "name": "Force password reset",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/users/{{alert.user}}/password/reset",+ "method": "POST",+ "headers_env": "IDP_BEARER_HEADERS"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "t1",+ "name": "Open ATO investigation ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "ATO suspected: {{alert.user}} from {{alert.source_ip}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "close-fp",+ "name": "Close if false positive",+ "type": "close_case",+ "params": {+ "resolution": "false_positive"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10,+ "condition": {+ "field": "verdict",+ "operator": "eq",+ "value": "false_positive"+ }+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/account-takeover/ato-mfa-fatigue-response.playbook.json b/services/agents/app/playbook/packs/v1/account-takeover/ato-mfa-fatigue-response.playbook.jsonnew file mode 100644index 000000000..c334a4385--- /dev/null+++ b/services/agents/app/playbook/packs/v1/account-takeover/ato-mfa-fatigue-response.playbook.json@@ -0,0 +1,116 @@+{+ "id": "ato-mfa-fatigue-response-v1",+ "name": "ATO: MFA Fatigue \u2014 Challenge & Reset",+ "description": "Triggered when a user receives an unusual burst of MFA prompts. Enforces step-up auth and resets MFA factors after approval.",+ "version": "1.0.0",+ "tags": [+ "account-takeover",+ "ato",+ "identity",+ "mfa",+ "mitre.t1621"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "medium",+ "high"+ ],+ "tags": [+ "account-takeover"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich user device posture",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.user"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "inv",+ "name": "Score MFA-fatigue likelihood",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "step-up",+ "name": "Force step-up auth challenge",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/users/{{alert.user}}/factors/challenge",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "n1",+ "name": "DM user on Slack",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "We've challenged your sign-in due to repeated MFA prompts. Approve only if you initiated.",+ "webhook_env": "SLACK_USER_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "approve",+ "name": "Wait for human approval",+ "type": "condition",+ "params": {},+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 5,+ "condition": {+ "field": "context.approved_by_oncall",+ "operator": "eq",+ "value": true+ },+ "next_true": "reset-mfa"+ },+ {+ "id": "reset-mfa",+ "name": "Reset enrolled MFA factors",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/users/{{alert.user}}/factors",+ "method": "DELETE"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "t1",+ "name": "Open MFA-fatigue case",+ "type": "create_ticket",+ "params": {+ "priority": "P3",+ "title_template": "MFA fatigue investigation: {{alert.user}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/account-takeover/ato-session-token-theft.playbook.json b/services/agents/app/playbook/packs/v1/account-takeover/ato-session-token-theft.playbook.jsonnew file mode 100644index 000000000..fe7e1d5b7--- /dev/null+++ b/services/agents/app/playbook/packs/v1/account-takeover/ato-session-token-theft.playbook.json@@ -0,0 +1,100 @@+{+ "id": "ato-session-token-theft-v1",+ "name": "ATO: Session Token Theft \u2014 Revoke & Isolate",+ "description": "Triggered when an authenticated session is observed from a new device + new geo simultaneously. Revokes all tokens and isolates the originating endpoint if known.",+ "version": "1.0.0",+ "tags": [+ "account-takeover",+ "ato",+ "identity",+ "session-hijack",+ "mitre.t1539"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "account-takeover"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich session context",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.session_id"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "revoke",+ "name": "Revoke all sessions for user",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/users/{{alert.user}}/sessions/revoke-all",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate token-theft chain",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "iso",+ "name": "Isolate originating endpoint",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.host"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "n1",+ "name": "Page on-call",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Session theft suspected for {{alert.user}}. Tokens revoked, host {{alert.host}} isolated.",+ "service_key_env": "PD_SOC_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open P1 ATO ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Session token theft: {{alert.user}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/account-takeover/ato-suspicious-oauth-grant.playbook.json b/services/agents/app/playbook/packs/v1/account-takeover/ato-suspicious-oauth-grant.playbook.jsonnew file mode 100644index 000000000..cc43c7596--- /dev/null+++ b/services/agents/app/playbook/packs/v1/account-takeover/ato-suspicious-oauth-grant.playbook.json@@ -0,0 +1,90 @@+{+ "id": "ato-suspicious-oauth-grant-v1",+ "name": "ATO: Suspicious OAuth Grant \u2014 Revoke & Investigate",+ "description": "Triggered when a user grants a new OAuth scope to an unfamiliar third-party app. Revokes the grant and investigates.",+ "version": "1.0.0",+ "tags": [+ "account-takeover",+ "ato",+ "identity",+ "oauth",+ "mitre.t1528"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "medium",+ "high"+ ],+ "tags": [+ "account-takeover"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich OAuth client reputation",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.oauth_client_id"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "inv",+ "name": "Investigate scope and lineage",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "identity"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "revoke",+ "name": "Revoke OAuth grant",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/users/{{alert.user}}/oauth/{{alert.oauth_client_id}}",+ "method": "DELETE"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "n1",+ "name": "Notify user",+ "type": "notify",+ "params": {+ "channel": "email",+ "message_template": "Removed OAuth grant for {{alert.oauth_client_app_name}}. Re-add only if intentional.",+ "webhook_env": "EMAIL_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open OAuth-abuse ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P3",+ "title_template": "Suspicious OAuth grant: {{alert.user}} -> {{alert.oauth_client_app_name}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/anomalous-data/anomalous-data-transfer-response.playbook.json b/services/agents/app/playbook/packs/v1/anomalous-data/anomalous-data-transfer-response.playbook.jsonnew file mode 100644index 000000000..6930fd955--- /dev/null+++ b/services/agents/app/playbook/packs/v1/anomalous-data/anomalous-data-transfer-response.playbook.json@@ -0,0 +1,208 @@+{+ "id": "anomalous-data-transfer-response-v1",+ "name": "Anomalous Data Transfer: DLP / Exfiltration Response",+ "description": "Triggered by a DLP or UEBA alert detecting an anomalously large or out-of-policy data transfer, potential exfiltration to external destination, or sensitive data leaving the organization. Correlates evidence, blocks destination if malicious, and disables the user if compromise is confirmed.",+ "version": "1.0.0",+ "author": "Beenu - [email protected]",+ "tags": [+ "data-exfiltration",+ "dlp",+ "ueba",+ "anomalous-data",+ "insider-threat",+ "mitre.t1041",+ "mitre.t1048",+ "mitre.t1567"+ ],+ "trigger": {+ "on": "alert",+ "severity": ["high", "critical"],+ "tags": ["data-exfiltration", "dlp-violation", "anomalous-transfer", "insider-threat", "ueba"]+ },+ "enabled": true,+ "dry_run_support": true,+ "created_at": "2026-05-10T08:00:00+00:00",+ "updated_at": "2026-05-10T08:00:00+00:00",+ "inputs": {+ "user": {+ "description": "User responsible for the data transfer",+ "field": "alert.user",+ "required": true+ },+ "source_host": {+ "description": "Internal host where the transfer originated",+ "field": "alert.source_host",+ "required": false+ },+ "destination_ip": {+ "description": "External destination IP or domain",+ "field": "alert.destination_ip",+ "required": false+ },+ "data_volume_mb": {+ "description": "Volume of data transferred in megabytes",+ "field": "alert.data_volume_mb",+ "required": false+ },+ "data_classification": {+ "description": "Classification of the data (sensitive, confidential, pii, ip)",+ "field": "alert.data_classification",+ "required": false+ },+ "transfer_protocol": {+ "description": "Protocol used (https, ftp, sftp, email, usb, cloud)",+ "field": "alert.protocol",+ "required": false+ }+ },+ "steps": [+ {+ "id": "e1",+ "name": "Enrich destination IP / domain (threat intel)",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.destination_ip",+ "indicator_type": "ip",+ "skip_if_empty": true+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "siem_search_user",+ "name": "Search SIEM for prior data transfer activity by user",+ "type": "search_siem",+ "params": {+ "query": "user={{inputs.user}} AND event_type=network_transfer",+ "time_window": "-30d",+ "max_results": 500+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 60+ },+ {+ "id": "siem_search_dest",+ "name": "Search SIEM for other users transferring to same destination",+ "type": "search_siem",+ "params": {+ "query": "dest_ip={{inputs.destination_ip}} AND event_type=network_transfer",+ "time_window": "-7d",+ "max_results": 200+ },+ "condition": "inputs.destination_ip != null",+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 45+ },+ {+ "id": "approval_block_dest",+ "name": "Approval gate — block destination IP",+ "type": "approval",+ "params": {+ "message": "Block destination IP '{{inputs.destination_ip}}' involved in anomalous data transfer ({{inputs.data_volume_mb}}MB, classification: {{inputs.data_classification}})?",+ "approvers": ["soc-lead", "data-security"],+ "timeout_seconds": 300,+ "auto_approve_severity": ["critical"]+ },+ "condition": "inputs.destination_ip != null",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 320+ },+ {+ "id": "block_dest_ip",+ "name": "Block destination IP at perimeter",+ "type": "block_ip",+ "params": {+ "ip_field": "alert.destination_ip",+ "duration": "72h",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_block_dest.outcome == 'approved' && inputs.destination_ip != null",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "approval_disable_user",+ "name": "Approval gate — disable user account",+ "type": "approval",+ "params": {+ "message": "Disable account for '{{inputs.user}}' due to suspected data exfiltration ({{inputs.data_volume_mb}}MB of {{inputs.data_classification}} data to {{inputs.destination_ip}})?",+ "approvers": ["soc-lead", "hr-security", "data-security"],+ "timeout_seconds": 600,+ "auto_approve_severity": ["critical"]+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 620+ },+ {+ "id": "disable_user",+ "name": "Disable user account (insider threat confirmed)",+ "type": "disable_user",+ "params": {+ "user_field": "inputs.user",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_disable_user.outcome == 'approved'",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "revoke_sessions",+ "name": "Revoke all active sessions for user",+ "type": "revoke_session",+ "params": {+ "user_field": "inputs.user",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_disable_user.outcome == 'approved'",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Data exfiltration investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "data-exfiltration"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify data security and CISO",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Anomalous data transfer by {{inputs.user}}: {{inputs.data_volume_mb}}MB of {{inputs.data_classification}} data via {{inputs.transfer_protocol}} to {{inputs.destination_ip}}. Destination blocked: {{context.block_success}}.",+ "service_key_env": "PD_CISO_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open data exfiltration ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Anomalous Data Transfer: {{inputs.user}} — {{inputs.data_volume_mb}}MB {{inputs.data_classification}}",+ "queue": "soc",+ "labels": ["data-exfiltration", "dlp", "insider-threat"]+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/bec/bec-conditional-access-bypass.playbook.json b/services/agents/app/playbook/packs/v1/bec/bec-conditional-access-bypass.playbook.jsonnew file mode 100644index 000000000..f07c10572--- /dev/null+++ b/services/agents/app/playbook/packs/v1/bec/bec-conditional-access-bypass.playbook.json@@ -0,0 +1,88 @@+{+ "id": "bec-conditional-access-bypass-v1",+ "name": "BEC: Conditional Access Policy Bypass",+ "description": "Triggered when a BEC-related sign-in succeeds from a region that should be blocked by Conditional Access. Restores policy.",+ "version": "1.0.0",+ "tags": [+ "bec",+ "identity",+ "conditional-access"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "bec"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich CA policy state",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.policy_id"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "restore",+ "name": "Restore conditional access policy",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/policies/{{alert.policy_id}}/enable",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate CA bypass",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "identity"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page identity team",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "CA policy {{alert.policy_id}} re-enabled. Bypass observed.",+ "service_key_env": "PD_IDENTITY_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open CA-bypass ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "CA bypass: {{alert.user}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/bec/bec-impersonation-domain.playbook.json b/services/agents/app/playbook/packs/v1/bec/bec-impersonation-domain.playbook.jsonnew file mode 100644index 000000000..0a18b4c2f--- /dev/null+++ b/services/agents/app/playbook/packs/v1/bec/bec-impersonation-domain.playbook.json@@ -0,0 +1,100 @@+{+ "id": "bec-impersonation-domain-v1",+ "name": "BEC: Impersonation Domain Quarantine",+ "description": "Triggered when an inbound email originates from a domain that look-alikes an executive or vendor. Quarantines and blocks.",+ "version": "1.0.0",+ "tags": [+ "bec",+ "email",+ "mitre.t1656"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high"+ ],+ "tags": [+ "bec"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich sender domain",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.sender_domain"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "quarantine",+ "name": "Quarantine impersonation message",+ "type": "http",+ "params": {+ "url": "${MAIL_URL}/messages/{{alert.message_id}}/quarantine",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "block-domain",+ "name": "Block sender domain",+ "type": "http",+ "params": {+ "url": "${MAIL_URL}/policy/block-domain",+ "method": "POST",+ "body_template": "{\"domain\":\"{{alert.sender_domain}}\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate BEC campaign",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "identity"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify SOC",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "Impersonation domain {{alert.sender_domain}} blocked.",+ "webhook_env": "SLACK_SOC_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open BEC ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P3",+ "title_template": "BEC impersonation: {{alert.sender_domain}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/bec/bec-inbox-rule-malicious.playbook.json b/services/agents/app/playbook/packs/v1/bec/bec-inbox-rule-malicious.playbook.jsonnew file mode 100644index 000000000..3559b8668--- /dev/null+++ b/services/agents/app/playbook/packs/v1/bec/bec-inbox-rule-malicious.playbook.json@@ -0,0 +1,100 @@+{+ "id": "bec-inbox-rule-malicious-v1",+ "name": "BEC: Malicious Inbox Rule Removed",+ "description": "Triggered when a forwarding/filter rule is created that hides vendor or finance keywords. Removes the rule and resets creds.",+ "version": "1.0.0",+ "tags": [+ "bec",+ "email",+ "mitre.t1564.008"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "bec"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich rule details",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.inbox_rule_id"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "delete-rule",+ "name": "Delete malicious inbox rule",+ "type": "http",+ "params": {+ "url": "${MAIL_URL}/users/{{alert.user}}/rules/{{alert.inbox_rule_id}}",+ "method": "DELETE"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "revoke",+ "name": "Revoke active sessions",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/users/{{alert.user}}/sessions/revoke-all",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate BEC chain",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "identity"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify finance team",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "BEC rule removed for {{alert.user}}. Validate any vendor changes in the last 7d.",+ "webhook_env": "SLACK_FINANCE_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open BEC ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "BEC inbox rule: {{alert.user}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/bec/bec-token-theft.playbook.json b/services/agents/app/playbook/packs/v1/bec/bec-token-theft.playbook.jsonnew file mode 100644index 000000000..5f1d4c767--- /dev/null+++ b/services/agents/app/playbook/packs/v1/bec/bec-token-theft.playbook.json@@ -0,0 +1,100 @@+{+ "id": "bec-token-theft-v1",+ "name": "BEC: M365 / Workspace Token Theft",+ "description": "Triggered when a refresh token is observed from a new ASN with no interactive sign-in. Revokes refresh tokens and forces MFA.",+ "version": "1.0.0",+ "tags": [+ "bec",+ "identity",+ "token-theft",+ "mitre.t1539"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "bec"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich ASN reputation",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.source_asn"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "revoke-rt",+ "name": "Revoke refresh tokens",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/users/{{alert.user}}/refresh-tokens",+ "method": "DELETE"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "force-mfa",+ "name": "Force MFA re-enrollment",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/users/{{alert.user}}/mfa/reset",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate token-theft chain",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "identity"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page on-call",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Refresh-token theft suspected for {{alert.user}}. Tokens revoked.",+ "service_key_env": "PD_SOC_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open BEC token ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "BEC token theft: {{alert.user}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/bec/bec-vendor-payment-redirect.playbook.json b/services/agents/app/playbook/packs/v1/bec/bec-vendor-payment-redirect.playbook.jsonnew file mode 100644index 000000000..a2fdbc26d--- /dev/null+++ b/services/agents/app/playbook/packs/v1/bec/bec-vendor-payment-redirect.playbook.json@@ -0,0 +1,103 @@+{+ "id": "bec-vendor-payment-redirect-v1",+ "name": "BEC: Vendor Payment Redirect \u2014 Freeze",+ "description": "Triggered when a vendor banking detail change happens via email. Freezes pending payments and pages CFO.",+ "version": "1.0.0",+ "tags": [+ "bec",+ "finance",+ "mitre.t1565.001"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "bec"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "freeze",+ "name": "Freeze pending vendor payment",+ "type": "http",+ "params": {+ "url": "${ERP_URL}/payments/{{alert.payment_id}}/freeze",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate vendor change",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "identity"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page CFO",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Vendor payment {{alert.payment_id}} frozen pending review.",+ "service_key_env": "PD_CFO_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "approve",+ "name": "Wait for human approval",+ "type": "condition",+ "params": {},+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 5,+ "condition": {+ "field": "context.approved_by_oncall",+ "operator": "eq",+ "value": true+ },+ "next_true": "release"+ },+ {+ "id": "release",+ "name": "Release payment after approval",+ "type": "http",+ "params": {+ "url": "${ERP_URL}/payments/{{alert.payment_id}}/release",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "t1",+ "name": "Open BEC payment ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "BEC payment redirect: {{alert.vendor}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/brute-force/brute-force-lockout.playbook.json b/services/agents/app/playbook/packs/v1/brute-force/brute-force-lockout.playbook.jsonnew file mode 100644index 000000000..77baa030c--- /dev/null+++ b/services/agents/app/playbook/packs/v1/brute-force/brute-force-lockout.playbook.json@@ -0,0 +1,185 @@+{+ "id": "brute-force-lockout-v1",+ "name": "Brute Force: Account Lockout and Source Block",+ "description": "Triggered by detection of repeated failed authentication attempts (password spray, credential stuffing, RDP/SSH brute force). Locks out the targeted account, blocks the source IP, and triggers investigation.",+ "version": "1.0.0",+ "author": "Beenu - [email protected]",+ "tags": [+ "brute-force",+ "credential-stuffing",+ "password-spray",+ "authentication",+ "mitre.t1110",+ "mitre.t1110.001",+ "mitre.t1110.003"+ ],+ "trigger": {+ "on": "alert",+ "severity": ["medium", "high", "critical"],+ "tags": ["brute-force", "password-spray", "credential-stuffing", "rdp-brute", "ssh-brute"]+ },+ "enabled": true,+ "dry_run_support": true,+ "created_at": "2026-05-10T08:00:00+00:00",+ "updated_at": "2026-05-10T08:00:00+00:00",+ "inputs": {+ "user": {+ "description": "Targeted user account (may be null for password spray targeting many users)",+ "field": "alert.user",+ "required": false+ },+ "source_ip": {+ "description": "Attacking source IP or range",+ "field": "alert.source_ip",+ "required": true+ },+ "failure_count": {+ "description": "Number of failed authentication attempts that triggered this alert",+ "field": "alert.failure_count",+ "required": false+ },+ "target_service": {+ "description": "Service being brute-forced (ssh, rdp, web, vpn, idp)",+ "field": "alert.target_service",+ "required": false+ },+ "attack_type": {+ "description": "Type of brute-force attack (password-spray, credential-stuffing, dictionary)",+ "field": "alert.attack_type",+ "required": false+ }+ },+ "steps": [+ {+ "id": "e1",+ "name": "Enrich source IP (threat intel, reputation, ASN)",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.source_ip",+ "indicator_type": "ip"+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "siem_search_src",+ "name": "Search SIEM for prior activity from attacking IP",+ "type": "search_siem",+ "params": {+ "query": "src_ip={{inputs.source_ip}}",+ "time_window": "-30d",+ "max_results": 200+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 45+ },+ {+ "id": "block_source_ip",+ "name": "Block attacking source IP at perimeter",+ "type": "block_ip",+ "params": {+ "ip_field": "alert.source_ip",+ "duration": "24h",+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "approval_disable_account",+ "name": "Approval gate — temporarily disable targeted account",+ "type": "approval",+ "params": {+ "message": "Temporarily disable account '{{inputs.user}}' due to brute-force attack ({{inputs.failure_count}} failures from {{inputs.source_ip}})?",+ "approvers": ["soc-lead", "identity-admin"],+ "timeout_seconds": 300,+ "auto_approve_severity": ["critical"]+ },+ "condition": "inputs.user != null",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 320+ },+ {+ "id": "disable_user",+ "name": "Temporarily disable targeted user account",+ "type": "disable_user",+ "params": {+ "user_field": "inputs.user",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_disable_account.outcome == 'approved' && inputs.user != null",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "revoke_sessions",+ "name": "Revoke active sessions for targeted user",+ "type": "revoke_session",+ "params": {+ "user_field": "inputs.user",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_disable_account.outcome == 'approved' && inputs.user != null",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "create_siem_notable",+ "name": "Create SIEM notable event for brute force campaign",+ "type": "create_notable_event",+ "params": {+ "title": "Brute Force: {{inputs.attack_type}} from {{inputs.source_ip}} targeting {{inputs.target_service}}",+ "severity": "high",+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Brute force investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "brute-force"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify SOC and identity team",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "Brute force alert: {{inputs.attack_type}} from {{inputs.source_ip}} against {{inputs.target_service}} ({{inputs.failure_count}} failures). IP blocked: {{context.block_success}}. User disabled: {{context.disable_success}}.",+ "webhook_env": "SLACK_SOC_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open brute force ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "Brute Force: {{inputs.attack_type}} from {{inputs.source_ip}}",+ "queue": "soc",+ "labels": ["brute-force", "credential-attack", "authentication"]+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/cloud-account-takeover/cloud-ato-response.playbook.json b/services/agents/app/playbook/packs/v1/cloud-account-takeover/cloud-ato-response.playbook.jsonnew file mode 100644index 000000000..fdad54579--- /dev/null+++ b/services/agents/app/playbook/packs/v1/cloud-account-takeover/cloud-ato-response.playbook.json@@ -0,0 +1,172 @@+{+ "id": "cloud-ato-response-v1",+ "name": "Cloud ATO: Compromised Cloud Account Response",+ "description": "Triggered by anomalous API activity, impossible travel, or cloud CSPM alert indicating a compromised cloud account. Revokes active sessions, rotates credentials, preserves audit trail, and notifies cloud security team.",+ "version": "1.0.0",+ "author": "Beenu - [email protected]",+ "tags": [+ "cloud-account-takeover",+ "cloud",+ "iam",+ "identity",+ "mitre.t1078.004",+ "mitre.t1535"+ ],+ "trigger": {+ "on": "alert",+ "severity": ["high", "critical"],+ "tags": ["cloud-account-takeover", "cloud-ato", "impossible-travel"]+ },+ "enabled": true,+ "dry_run_support": true,+ "created_at": "2026-05-10T08:00:00+00:00",+ "updated_at": "2026-05-10T08:00:00+00:00",+ "inputs": {+ "cloud_user": {+ "description": "Cloud account identity (IAM user, service account, or federated identity)",+ "field": "alert.cloud_user",+ "required": true+ },+ "cloud_provider": {+ "description": "Cloud provider: aws | gcp | azure",+ "field": "alert.cloud_provider",+ "required": true+ },+ "source_ip": {+ "description": "Anomalous source IP from cloud trail",+ "field": "alert.source_ip",+ "required": false+ },+ "access_key_id": {+ "description": "AWS access key ID (if applicable)",+ "field": "alert.access_key_id",+ "required": false+ }+ },+ "steps": [+ {+ "id": "e1",+ "name": "Enrich source IP reputation",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.source_ip",+ "indicator_type": "ip",+ "skip_if_empty": true+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "cloudtrail_search",+ "name": "Search cloud audit trail for recent actions",+ "type": "search_siem",+ "params": {+ "query": "cloud_user={{inputs.cloud_user}} AND source_ip={{inputs.source_ip}}",+ "time_window": "-4h",+ "max_results": 200,+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 45+ },+ {+ "id": "approval_revoke",+ "name": "Approval gate — revoke cloud sessions and keys",+ "type": "approval",+ "params": {+ "message": "Revoke all active sessions and access keys for cloud user '{{inputs.cloud_user}}' ({{inputs.cloud_provider}})?",+ "approvers": ["cloud-security-lead", "soc-lead"],+ "timeout_seconds": 600,+ "auto_approve_severity": ["critical"]+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 620+ },+ {+ "id": "revoke_sessions",+ "name": "Revoke all active cloud sessions",+ "type": "revoke_session",+ "params": {+ "user_field": "inputs.cloud_user",+ "provider": "{{inputs.cloud_provider}}",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_revoke.outcome == 'approved'",+ "on_failure": "abort",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "disable_keys",+ "name": "Disable / deactivate access keys",+ "type": "http",+ "params": {+ "url": "${CLOUD_MGMT_URL}/v1/iam/keys/disable",+ "method": "POST",+ "body_template": "{\"user\": \"{{inputs.cloud_user}}\", \"provider\": \"{{inputs.cloud_provider}}\", \"access_key_id\": \"{{inputs.access_key_id}}\"}",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_revoke.outcome == 'approved'",+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 20+ },+ {+ "id": "block_src_ip",+ "name": "Block anomalous source IP (if present)",+ "type": "block_ip",+ "params": {+ "ip_field": "alert.source_ip",+ "duration": "24h",+ "skip_if_empty": true,+ "dry_run": "{{dry_run}}"+ },+ "condition": "inputs.source_ip != null",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Cloud ATO investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud-ato"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page cloud security team (SEV1)",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Cloud ATO: {{inputs.cloud_user}} on {{inputs.cloud_provider}}. Sessions revoked: {{context.revoke_success}}. Source IP blocked: {{inputs.source_ip}}.",+ "service_key_env": "PD_CLOUD_SECURITY_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open cloud ATO ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Cloud ATO: {{inputs.cloud_user}} ({{inputs.cloud_provider}})",+ "queue": "cloud-security",+ "labels": ["cloud-ato", "iam", "{{inputs.cloud_provider}}"]+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-azure-blob-public.playbook.json b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-azure-blob-public.playbook.jsonnew file mode 100644index 000000000..34caea9d5--- /dev/null+++ b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-azure-blob-public.playbook.json@@ -0,0 +1,78 @@+{+ "id": "cloud-azure-blob-public-v1",+ "name": "Cloud: Azure Blob Container Public",+ "description": "Triggered when an Azure Storage container is set to public. Reverts to private and audits.",+ "version": "1.0.0",+ "tags": [+ "cloud-misconfig",+ "cloud",+ "azure",+ "storage",+ "mitre.t1530"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high"+ ],+ "tags": [+ "cloud-misconfig"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "private",+ "name": "Set container private",+ "type": "http",+ "params": {+ "url": "${AZURE_URL}/storage/{{alert.account}}/{{alert.container}}/private",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Audit blob access",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify cloud team",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "Azure {{alert.container}} re-privatized.",+ "webhook_env": "SLACK_CLOUD_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open Azure ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "Azure blob public: {{alert.container}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-cloudtrail-disabled.playbook.json b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-cloudtrail-disabled.playbook.jsonnew file mode 100644index 000000000..3b5dd2743--- /dev/null+++ b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-cloudtrail-disabled.playbook.json@@ -0,0 +1,78 @@+{+ "id": "cloud-cloudtrail-disabled-v1",+ "name": "Cloud: CloudTrail Disabled \u2014 Re-enable",+ "description": "Triggered when a CloudTrail trail is stopped. Restarts trail and audits API calls in the dark window.",+ "version": "1.0.0",+ "tags": [+ "cloud-misconfig",+ "cloud",+ "aws",+ "logging",+ "mitre.t1562.008"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "cloud-misconfig"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "start",+ "name": "Re-enable CloudTrail",+ "type": "http",+ "params": {+ "url": "${AWS_URL}/cloudtrail/{{alert.trail}}/start",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Audit dark-window calls",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page cloud-sec",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "CloudTrail {{alert.trail}} restarted.",+ "service_key_env": "PD_CLOUD_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open CloudTrail ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "CloudTrail disabled: {{alert.trail}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-cross-account-trust.playbook.json b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-cross-account-trust.playbook.jsonnew file mode 100644index 000000000..307ba7e39--- /dev/null+++ b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-cross-account-trust.playbook.json@@ -0,0 +1,78 @@+{+ "id": "cloud-cross-account-trust-v1",+ "name": "Cloud: Unexpected Cross-Account Trust",+ "description": "Triggered when a role trust policy adds an external account ID not on the allow-list. Revokes and audits.",+ "version": "1.0.0",+ "tags": [+ "cloud-misconfig",+ "cloud",+ "aws",+ "iam",+ "mitre.t1078.004"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "cloud-misconfig"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "revoke-trust",+ "name": "Remove cross-account trust",+ "type": "http",+ "params": {+ "url": "${AWS_URL}/iam/{{alert.role}}/trust/{{alert.external_account}}",+ "method": "DELETE"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Audit assumptions in dark window",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page cloud-sec",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Cross-account trust to {{alert.external_account}} removed from {{alert.role}}.",+ "service_key_env": "PD_CLOUD_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open cross-account ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Cross-account trust: {{alert.role}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-gke-anonymous.playbook.json b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-gke-anonymous.playbook.jsonnew file mode 100644index 000000000..c6e409419--- /dev/null+++ b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-gke-anonymous.playbook.json@@ -0,0 +1,79 @@+{+ "id": "cloud-gke-anonymous-v1",+ "name": "Cloud: GKE Cluster Anonymous Auth",+ "description": "Triggered when GKE cluster is configured to allow anonymous Kubernetes API access. Disables anonymous and audits.",+ "version": "1.0.0",+ "tags": [+ "cloud-misconfig",+ "cloud",+ "gcp",+ "kubernetes",+ "mitre.t1190"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "cloud-misconfig"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "disable-anon",+ "name": "Disable anonymous auth",+ "type": "http",+ "params": {+ "url": "${GCP_URL}/gke/{{alert.cluster}}/anonymous/disable",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Audit anonymous-window calls",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page platform team",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "GKE {{alert.cluster}} anonymous disabled.",+ "service_key_env": "PD_PLATFORM_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open GKE-anon ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "GKE anonymous: {{alert.cluster}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-iam-overpriv.playbook.json b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-iam-overpriv.playbook.jsonnew file mode 100644index 000000000..606bb29f5--- /dev/null+++ b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-iam-overpriv.playbook.json@@ -0,0 +1,79 @@+{+ "id": "cloud-iam-overpriv-v1",+ "name": "Cloud: Over-Privileged IAM \u2014 Detach & Review",+ "description": "Triggered when IAM Access Analyzer flags an over-privileged role. Detaches the policy and opens an owner-review ticket.",+ "version": "1.0.0",+ "tags": [+ "cloud-misconfig",+ "cloud",+ "aws",+ "iam",+ "mitre.t1078.004"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "medium",+ "high"+ ],+ "tags": [+ "cloud-misconfig"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "detach",+ "name": "Detach over-privileged policy",+ "type": "http",+ "params": {+ "url": "${AWS_URL}/iam/{{alert.role}}/policies/{{alert.policy}}",+ "method": "DELETE"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate access usage",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify role owner",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "Policy {{alert.policy}} detached from {{alert.role}}. Re-add only the minimum needed.",+ "webhook_env": "SLACK_CLOUD_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Owner-review ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P3",+ "title_template": "IAM over-priv: {{alert.role}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-key-leak.playbook.json b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-key-leak.playbook.jsonnew file mode 100644index 000000000..a5c6c9982--- /dev/null+++ b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-key-leak.playbook.json@@ -0,0 +1,89 @@+{+ "id": "cloud-key-leak-v1",+ "name": "Cloud: Cloud Access Key Leaked",+ "description": "Triggered when a long-lived access key is found in a public repo or paste site. Rotates the key and scans for downstream use.",+ "version": "1.0.0",+ "tags": [+ "cloud-misconfig",+ "cloud",+ "secrets",+ "mitre.t1552.001"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "cloud-misconfig"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "disable",+ "name": "Disable leaked key",+ "type": "http",+ "params": {+ "url": "${AWS_URL}/iam/users/{{alert.user}}/keys/{{alert.key_id}}",+ "method": "DELETE"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "rotate",+ "name": "Rotate access key",+ "type": "http",+ "params": {+ "url": "${AWS_URL}/iam/users/{{alert.user}}/keys/rotate",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Find downstream use of key",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page cloud-sec",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Leaked key {{alert.key_id}} disabled and rotated.",+ "service_key_env": "PD_CLOUD_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open key-leak ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Leaked key: {{alert.key_id}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-mfa-disabled-root.playbook.json b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-mfa-disabled-root.playbook.jsonnew file mode 100644index 000000000..5cdb8d84b--- /dev/null+++ b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-mfa-disabled-root.playbook.json@@ -0,0 +1,78 @@+{+ "id": "cloud-mfa-disabled-root-v1",+ "name": "Cloud: Root Account MFA Disabled",+ "description": "Triggered when root MFA is disabled. Re-enables MFA and pages the cloud-platform owner immediately.",+ "version": "1.0.0",+ "tags": [+ "cloud-misconfig",+ "cloud",+ "iam",+ "mfa",+ "mitre.t1556"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "cloud-misconfig"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "reenable-mfa",+ "name": "Re-enable root MFA",+ "type": "http",+ "params": {+ "url": "${AWS_URL}/iam/root/mfa/enable",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Identify actor and intent",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page platform-owner",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Root MFA was disabled \u2014 re-enabled. Investigate {{alert.actor}}.",+ "service_key_env": "PD_PLATFORM_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open root-MFA ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Root MFA disabled by {{alert.actor}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-rds-public.playbook.json b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-rds-public.playbook.jsonnew file mode 100644index 000000000..d4ba3d705--- /dev/null+++ b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-rds-public.playbook.json@@ -0,0 +1,79 @@+{+ "id": "cloud-rds-public-v1",+ "name": "Cloud: RDS Instance Publicly Accessible",+ "description": "Triggered when an RDS instance toggles to publicly accessible. Removes public access and audits.",+ "version": "1.0.0",+ "tags": [+ "cloud-misconfig",+ "cloud",+ "aws",+ "rds",+ "mitre.t1190"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "cloud-misconfig"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "private",+ "name": "Disable RDS public access",+ "type": "http",+ "params": {+ "url": "${AWS_URL}/rds/{{alert.instance}}/public/disable",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Audit RDS access window",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify cloud team",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "RDS {{alert.instance}} no longer public.",+ "webhook_env": "SLACK_CLOUD_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open RDS-public ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "RDS public: {{alert.instance}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-s3-public.playbook.json b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-s3-public.playbook.jsonnew file mode 100644index 000000000..84b762400--- /dev/null+++ b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-s3-public.playbook.json@@ -0,0 +1,91 @@+{+ "id": "cloud-s3-public-v1",+ "name": "Cloud: S3 Bucket Public \u2014 Re-private & Audit",+ "description": "Triggered by `s3-public-bucket` detection. Sets bucket private, enables Block Public Access, and audits last 7d access logs.",+ "version": "1.0.0",+ "tags": [+ "cloud-misconfig",+ "cloud",+ "aws",+ "s3",+ "mitre.t1530"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "cloud-misconfig"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "private",+ "name": "Set bucket private",+ "type": "http",+ "params": {+ "url": "${AWS_URL}/s3/{{alert.bucket}}/acl/private",+ "method": "PUT"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "bpa",+ "name": "Enable Block Public Access",+ "type": "http",+ "params": {+ "url": "${AWS_URL}/s3/{{alert.bucket}}/block-public-access",+ "method": "PUT"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Audit last 7d S3 access",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify cloud team",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "S3 {{alert.bucket}} re-privatized.",+ "webhook_env": "SLACK_CLOUD_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open cloud ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "S3 public: {{alert.bucket}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-security-group-open.playbook.json b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-security-group-open.playbook.jsonnew file mode 100644index 000000000..6e3ee5d7b--- /dev/null+++ b/services/agents/app/playbook/packs/v1/cloud-misconfig/cloud-security-group-open.playbook.json@@ -0,0 +1,79 @@+{+ "id": "cloud-security-group-open-v1",+ "name": "Cloud: Security Group Opens 0.0.0.0/0",+ "description": "Triggered when a security group rule allows ingress from 0.0.0.0/0 to a sensitive port. Closes the rule and audits the change.",+ "version": "1.0.0",+ "tags": [+ "cloud-misconfig",+ "cloud",+ "aws",+ "network",+ "mitre.t1190"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "cloud-misconfig"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "close",+ "name": "Remove 0.0.0.0/0 ingress rule",+ "type": "http",+ "params": {+ "url": "${AWS_URL}/sg/{{alert.sg}}/ingress/{{alert.rule_id}}",+ "method": "DELETE"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Audit who made the change",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify cloud team",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "SG {{alert.sg}} 0.0.0.0/0 ingress removed.",+ "webhook_env": "SLACK_CLOUD_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open SG ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "SG open: {{alert.sg}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/container-escape/container-escape-response.playbook.json b/services/agents/app/playbook/packs/v1/container-escape/container-escape-response.playbook.jsonnew file mode 100644index 000000000..b04396ffc--- /dev/null+++ b/services/agents/app/playbook/packs/v1/container-escape/container-escape-response.playbook.json@@ -0,0 +1,172 @@+{+ "id": "container-escape-response-v1",+ "name": "Container Escape: Runtime Breach Response",+ "description": "Triggered by container runtime security alert (Falco, Sysdig, or similar) indicating a container escape attempt or successful breakout to the host. Kills the offending container, isolates the node, and preserves evidence.",+ "version": "1.0.0",+ "author": "Beenu - [email protected]",+ "tags": [+ "container-escape",+ "container",+ "kubernetes",+ "mitre.t1611",+ "mitre.t1610"+ ],+ "trigger": {+ "on": "alert",+ "severity": ["high", "critical"],+ "tags": ["container-escape", "container-breakout", "runtime-security"]+ },+ "enabled": true,+ "dry_run_support": true,+ "created_at": "2026-05-10T08:00:00+00:00",+ "updated_at": "2026-05-10T08:00:00+00:00",+ "inputs": {+ "container_id": {+ "description": "Container ID of the offending container",+ "field": "alert.container_id",+ "required": true+ },+ "node_host": {+ "description": "Kubernetes node / host running the container",+ "field": "alert.host",+ "required": true+ },+ "namespace": {+ "description": "Kubernetes namespace",+ "field": "alert.k8s_namespace",+ "required": false+ },+ "pod_name": {+ "description": "Kubernetes pod name",+ "field": "alert.k8s_pod",+ "required": false+ },+ "process_name": {+ "description": "Suspicious process that triggered the alert",+ "field": "alert.process_name",+ "required": false+ }+ },+ "steps": [+ {+ "id": "e1",+ "name": "Enrich container image and process",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.container_image",+ "indicator_type": "container-image",+ "skip_if_empty": true+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "kill_container",+ "name": "Kill offending container",+ "type": "kill_process",+ "params": {+ "host_field": "alert.host",+ "process_field": "alert.container_id",+ "kill_type": "container",+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "cordon_node",+ "name": "Cordon Kubernetes node (prevent new pod scheduling)",+ "type": "http",+ "params": {+ "url": "${K8S_API_URL}/api/v1/nodes/{{inputs.node_host}}/cordon",+ "method": "POST",+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "approval_isolate_node",+ "name": "Approval gate — isolate the Kubernetes node",+ "type": "approval",+ "params": {+ "message": "Network-isolate node '{{inputs.node_host}}'? This will drain all pods and may impact running workloads.",+ "approvers": ["platform-lead", "soc-lead"],+ "timeout_seconds": 300,+ "auto_approve_severity": ["critical"]+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 320+ },+ {+ "id": "isolate_node",+ "name": "Network-isolate compromised node (EDR)",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.host",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_isolate_node.outcome == 'approved'",+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "collect_artifacts",+ "name": "Collect container forensic artifacts",+ "type": "run_script",+ "params": {+ "host_field": "alert.host",+ "script": "collect_container_artifacts.sh",+ "args": ["--container-id", "{{inputs.container_id}}", "--output", "/forensics/{{alert.case_id}}/"],+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 120+ },+ {+ "id": "inv",+ "name": "Container escape investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "container-escape"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Alert platform / DevSecOps team",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Container escape on node {{inputs.node_host}} (pod: {{inputs.pod_name}}, ns: {{inputs.namespace}}). Container killed: {{context.kill_success}}. Node isolated: {{context.isolate_success}}.",+ "service_key_env": "PD_PLATFORM_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open container escape ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Container Escape: {{inputs.pod_name}} on {{inputs.node_host}}",+ "queue": "platform-security",+ "labels": ["container-escape", "kubernetes", "runtime"]+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/data-exfil/exfil-archive-egress.playbook.json b/services/agents/app/playbook/packs/v1/data-exfil/exfil-archive-egress.playbook.jsonnew file mode 100644index 000000000..42306cdb3--- /dev/null+++ b/services/agents/app/playbook/packs/v1/data-exfil/exfil-archive-egress.playbook.json@@ -0,0 +1,90 @@+{+ "id": "exfil-archive-egress-v1",+ "name": "Exfil: Archive File Egress (.zip / .7z / .rar)",+ "description": "Triggered when a host uploads an archive to an external dest. Holds the file in DLP review queue and notifies user's manager.",+ "version": "1.0.0",+ "tags": [+ "data-exfil",+ "exfil",+ "dlp",+ "archive"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "medium",+ "high"+ ],+ "tags": [+ "data-exfil"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich destination",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.dst_host"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "dlp-review",+ "name": "Send to DLP review queue",+ "type": "http",+ "params": {+ "url": "${DLP_URL}/review",+ "method": "POST",+ "body_template": "{\"host\":\"{{alert.host}}\",\"file\":\"{{alert.file_name}}\",\"dst\":\"{{alert.dst_host}}\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate archive content",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify user's manager",+ "type": "notify",+ "params": {+ "channel": "email",+ "message_template": "Archive upload by {{alert.user}} to {{alert.dst_host}} queued for DLP review.",+ "webhook_env": "EMAIL_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open DLP ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P3",+ "title_template": "Archive egress: {{alert.user}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/data-exfil/exfil-dns-tunneling.playbook.json b/services/agents/app/playbook/packs/v1/data-exfil/exfil-dns-tunneling.playbook.jsonnew file mode 100644index 000000000..083457f76--- /dev/null+++ b/services/agents/app/playbook/packs/v1/data-exfil/exfil-dns-tunneling.playbook.json@@ -0,0 +1,100 @@+{+ "id": "exfil-dns-tunneling-v1",+ "name": "Exfil: DNS Tunneling",+ "description": "Triggered by high-entropy DNS query length on a host. Sinkholes the suspect domain and isolates the host.",+ "version": "1.0.0",+ "tags": [+ "data-exfil",+ "exfil",+ "dns",+ "mitre.t1071.004"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "data-exfil"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich domain reputation",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.dns_query"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "sinkhole",+ "name": "Sinkhole domain at resolver",+ "type": "http",+ "params": {+ "url": "${DNS_URL}/sinkhole",+ "method": "POST",+ "body_template": "{\"domain\":\"{{alert.dns_query}}\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "iso",+ "name": "Isolate beaconing host",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.host"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "inv",+ "name": "Investigate DNS tunneling",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page on-call",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "DNS tunneling {{alert.host}} -> {{alert.dns_query}} sinkholed and isolated.",+ "service_key_env": "PD_SOC_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open DNS-tunnel ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "DNS tunneling: {{alert.host}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/data-exfil/exfil-large-upload.playbook.json b/services/agents/app/playbook/packs/v1/data-exfil/exfil-large-upload.playbook.jsonnew file mode 100644index 000000000..07c1c7aaa--- /dev/null+++ b/services/agents/app/playbook/packs/v1/data-exfil/exfil-large-upload.playbook.json@@ -0,0 +1,90 @@+{+ "id": "exfil-large-upload-v1",+ "name": "Exfil: Large Outbound Upload",+ "description": "Triggered when a host uploads >X GB to a non-corporate destination in a short window. Blocks destination and investigates.",+ "version": "1.0.0",+ "tags": [+ "data-exfil",+ "exfil",+ "dlp",+ "mitre.t1567"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "data-exfil"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich destination",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.dst_host"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "block",+ "name": "Block destination at proxy",+ "type": "http",+ "params": {+ "url": "${PROXY_URL}/policy/block",+ "method": "POST",+ "body_template": "{\"host\":\"{{alert.dst_host}}\",\"reason\":\"large-upload\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate exfil pattern",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify SOC",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "Large upload {{alert.bytes_out}} from {{alert.host}} to {{alert.dst_host}} blocked.",+ "webhook_env": "SLACK_SOC_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open exfil ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "Large upload: {{alert.host}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/data-exfil/exfil-personal-cloud.playbook.json b/services/agents/app/playbook/packs/v1/data-exfil/exfil-personal-cloud.playbook.jsonnew file mode 100644index 000000000..c07779d24--- /dev/null+++ b/services/agents/app/playbook/packs/v1/data-exfil/exfil-personal-cloud.playbook.json@@ -0,0 +1,79 @@+{+ "id": "exfil-personal-cloud-v1",+ "name": "Exfil: Personal Cloud Storage Upload",+ "description": "Triggered when corporate data uploads to a personal cloud (personal Drive, Dropbox, iCloud). Blocks dest, manager notify.",+ "version": "1.0.0",+ "tags": [+ "data-exfil",+ "exfil",+ "saas",+ "mitre.t1567.002"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "medium",+ "high"+ ],+ "tags": [+ "data-exfil"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "block",+ "name": "Block destination at proxy",+ "type": "http",+ "params": {+ "url": "${PROXY_URL}/policy/block",+ "method": "POST",+ "body_template": "{\"host\":\"{{alert.dst_host}}\",\"reason\":\"personal-cloud\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate access pattern",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "identity"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify user's manager",+ "type": "notify",+ "params": {+ "channel": "email",+ "message_template": "Personal-cloud upload by {{alert.user}} to {{alert.dst_host}} blocked.",+ "webhook_env": "EMAIL_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open exfil ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P3",+ "title_template": "Personal cloud: {{alert.user}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/data-exfil/exfil-removable-media.playbook.json b/services/agents/app/playbook/packs/v1/data-exfil/exfil-removable-media.playbook.jsonnew file mode 100644index 000000000..8c8250f5e--- /dev/null+++ b/services/agents/app/playbook/packs/v1/data-exfil/exfil-removable-media.playbook.json@@ -0,0 +1,78 @@+{+ "id": "exfil-removable-media-v1",+ "name": "Exfil: Sensitive Data to Removable Media",+ "description": "Triggered when a sensitive file is written to a USB volume. Blocks USB writes via EDR policy and opens a ticket.",+ "version": "1.0.0",+ "tags": [+ "data-exfil",+ "exfil",+ "endpoint",+ "usb",+ "mitre.t1052"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "medium"+ ],+ "tags": [+ "data-exfil"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "block-usb",+ "name": "Push USB write-block to host",+ "type": "http",+ "params": {+ "url": "${EDR_URL}/hosts/{{alert.host}}/policy/usb-readonly",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate file sensitivity",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify user's manager",+ "type": "notify",+ "params": {+ "channel": "email",+ "message_template": "Sensitive write to USB by {{alert.user}}. Host USB now read-only.",+ "webhook_env": "EMAIL_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open USB-exfil ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P3",+ "title_template": "USB exfil: {{alert.user}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/ddos/ddos-amplification-dns.playbook.json b/services/agents/app/playbook/packs/v1/ddos/ddos-amplification-dns.playbook.jsonnew file mode 100644index 000000000..aedc32ca7--- /dev/null+++ b/services/agents/app/playbook/packs/v1/ddos/ddos-amplification-dns.playbook.json@@ -0,0 +1,77 @@+{+ "id": "ddos-amplification-dns-v1",+ "name": "DDoS: DNS Amplification",+ "description": "Triggered when an inbound spike of crafted DNS queries hits an authoritative server. Sinkholes spoofed sources.",+ "version": "1.0.0",+ "tags": [+ "ddos",+ "dns",+ "mitre.t1498.002"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high"+ ],+ "tags": [+ "ddos"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "sinkhole",+ "name": "Sinkhole spoofed sources",+ "type": "http",+ "params": {+ "url": "${DNS_URL}/sinkhole-batch",+ "method": "POST",+ "body_template": "{\"src_ips\": {{alert.spoofed_sources}}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate amplification",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify network team",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "DNS amplification mitigated. {{context.sinkholed_count}} sources sinkholed.",+ "webhook_env": "SLACK_NETWORK_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open DNS-amp ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "DNS amplification: {{alert.target}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/ddos/ddos-app-layer-l7.playbook.json b/services/agents/app/playbook/packs/v1/ddos/ddos-app-layer-l7.playbook.jsonnew file mode 100644index 000000000..9a58a6a2e--- /dev/null+++ b/services/agents/app/playbook/packs/v1/ddos/ddos-app-layer-l7.playbook.json@@ -0,0 +1,91 @@+{+ "id": "ddos-app-layer-l7-v1",+ "name": "DDoS: Application Layer L7",+ "description": "Triggered by anomalous L7 request rate or pattern. Engages WAF rate-limit rule and challenge-page.",+ "version": "1.0.0",+ "tags": [+ "ddos",+ "waf",+ "mitre.t1498.001"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "ddos"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "waf-rl",+ "name": "Apply WAF rate-limit",+ "type": "http",+ "params": {+ "url": "${WAF_URL}/rate-limit",+ "method": "POST",+ "body_template": "{\"target\":\"{{alert.target}}\",\"rps\":{{alert.threshold_rps}}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "challenge",+ "name": "Enable challenge page",+ "type": "http",+ "params": {+ "url": "${WAF_URL}/challenge/enable",+ "method": "POST",+ "body_template": "{\"target\":\"{{alert.target}}\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate L7 signature",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify SOC + platform",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "L7 DDoS on {{alert.target}}. WAF rate-limit + challenge on.",+ "webhook_env": "SLACK_SOC_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open L7 DDoS ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "L7 DDoS: {{alert.target}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/ddos/ddos-credential-stuffing-fraud.playbook.json b/services/agents/app/playbook/packs/v1/ddos/ddos-credential-stuffing-fraud.playbook.jsonnew file mode 100644index 000000000..3b417e578--- /dev/null+++ b/services/agents/app/playbook/packs/v1/ddos/ddos-credential-stuffing-fraud.playbook.json@@ -0,0 +1,102 @@+{+ "id": "ddos-credential-stuffing-fraud-v1",+ "name": "DDoS: Auth-Endpoint Credential Stuffing",+ "description": "Triggered when /login takes a sustained spike of failed attempts. Combines DDoS + credential-stuffing response: WAF challenge, rate-limit, source-IP block.",+ "version": "1.0.0",+ "tags": [+ "ddos",+ "ato",+ "mitre.t1110.004"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high"+ ],+ "tags": [+ "ddos"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "challenge",+ "name": "Enable WAF challenge on /login",+ "type": "http",+ "params": {+ "url": "${WAF_URL}/challenge/enable",+ "method": "POST",+ "body_template": "{\"path\":\"/login\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "rate-limit",+ "name": "Apply per-IP rate-limit",+ "type": "http",+ "params": {+ "url": "${WAF_URL}/rate-limit",+ "method": "POST",+ "body_template": "{\"path\":\"/login\",\"per\":\"ip\",\"rps\":5}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "b1",+ "name": "Block top abusers",+ "type": "block_ip",+ "params": {+ "ip_field": "alert.top_abuser_ip",+ "duration": "24h"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate cred-stuffing campaign",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "identity"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify SOC",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "Auth-endpoint credential stuffing on /login mitigated.",+ "webhook_env": "SLACK_SOC_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open auth-DDoS ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "Auth DDoS: {{alert.target}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/ddos/ddos-syn-flood.playbook.json b/services/agents/app/playbook/packs/v1/ddos/ddos-syn-flood.playbook.jsonnew file mode 100644index 000000000..fc0ff3843--- /dev/null+++ b/services/agents/app/playbook/packs/v1/ddos/ddos-syn-flood.playbook.json@@ -0,0 +1,90 @@+{+ "id": "ddos-syn-flood-v1",+ "name": "DDoS: SYN Flood",+ "description": "Triggered by SYN flood pattern at edge. Enables SYN cookies and increases backend capacity.",+ "version": "1.0.0",+ "tags": [+ "ddos",+ "network",+ "mitre.t1498"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "ddos"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "syn-cookies",+ "name": "Enable SYN cookies at edge",+ "type": "http",+ "params": {+ "url": "${EDGE_URL}/syn-cookies/enable",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "scale",+ "name": "Scale backend capacity",+ "type": "http",+ "params": {+ "url": "${PLATFORM_URL}/scale",+ "method": "POST",+ "body_template": "{\"target\":\"{{alert.target}}\",\"replicas\":\"+50%\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate flood",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify network on-call",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "SYN flood on {{alert.target}}. SYN cookies + scale-out.",+ "service_key_env": "PD_NETWORK_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open SYN-flood ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "SYN flood: {{alert.target}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/ddos/ddos-volumetric-l3.playbook.json b/services/agents/app/playbook/packs/v1/ddos/ddos-volumetric-l3.playbook.jsonnew file mode 100644index 000000000..050caa724--- /dev/null+++ b/services/agents/app/playbook/packs/v1/ddos/ddos-volumetric-l3.playbook.json@@ -0,0 +1,77 @@+{+ "id": "ddos-volumetric-l3-v1",+ "name": "DDoS: Volumetric L3/L4",+ "description": "Triggered by volumetric flood detection at edge. Engages scrubbing provider and pages on-call.",+ "version": "1.0.0",+ "tags": [+ "ddos",+ "network",+ "mitre.t1498"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "ddos"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "scrub-on",+ "name": "Engage DDoS scrubbing",+ "type": "http",+ "params": {+ "url": "${DDOS_URL}/scrubbing/enable",+ "method": "POST",+ "body_template": "{\"target\":\"{{alert.target}}\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate attack signature",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page network on-call",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "L3/L4 DDoS on {{alert.target}}. Scrubbing engaged.",+ "service_key_env": "PD_NETWORK_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open DDoS ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "L3/L4 DDoS: {{alert.target}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/endpoint-isolation/endpoint-isolation-response.playbook.json b/services/agents/app/playbook/packs/v1/endpoint-isolation/endpoint-isolation-response.playbook.jsonnew file mode 100644index 000000000..40b387956--- /dev/null+++ b/services/agents/app/playbook/packs/v1/endpoint-isolation/endpoint-isolation-response.playbook.json@@ -0,0 +1,199 @@+{+ "id": "endpoint-isolation-response-v1",+ "name": "Endpoint Isolation: EDR-Driven Host Quarantine",+ "description": "Triggered when an endpoint requires emergency isolation due to confirmed or high-confidence compromise (ransomware spread, active C2 communication, lateral movement staging). Uses EDR to network-isolate the host while preserving management connectivity, collects forensic artifacts, and notifies the SOC.",+ "version": "1.0.0",+ "author": "Beenu - [email protected]",+ "tags": [+ "endpoint-isolation",+ "edr",+ "quarantine",+ "host-isolation",+ "mitre.t1562.001",+ "mitre.t1204",+ "mitre.t1486"+ ],+ "trigger": {+ "on": "alert",+ "severity": ["high", "critical"],+ "tags": ["endpoint-isolation", "ransomware", "c2-communication", "lateral-movement", "host-compromise"]+ },+ "enabled": true,+ "dry_run_support": true,+ "created_at": "2026-05-10T08:00:00+00:00",+ "updated_at": "2026-05-10T08:00:00+00:00",+ "inputs": {+ "host": {+ "description": "Hostname or IP of the endpoint to isolate",+ "field": "alert.host",+ "required": true+ },+ "compromise_type": {+ "description": "Type of compromise driving isolation (ransomware, c2, lateral-movement, malware)",+ "field": "alert.threat_type",+ "required": false+ },+ "user": {+ "description": "User logged into the endpoint at time of alert",+ "field": "alert.user",+ "required": false+ },+ "ioc_hash": {+ "description": "IOC hash associated with the compromise, if known",+ "field": "alert.file_hash",+ "required": false+ },+ "parent_case_id": {+ "description": "Parent case ID for grouping related incidents",+ "field": "alert.case_id",+ "required": false+ }+ },+ "steps": [+ {+ "id": "e1",+ "name": "Enrich host asset context and existing alerts",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.host",+ "indicator_type": "host"+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "e2",+ "name": "Enrich IOC hash (if available)",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.file_hash",+ "indicator_type": "hash",+ "skip_if_empty": true+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "siem_search",+ "name": "Search SIEM for lateral movement indicators from this host",+ "type": "search_siem",+ "params": {+ "query": "src_host={{inputs.host}} AND (event_type=lateral_movement OR event_type=credential_dump OR event_type=network_scan)",+ "time_window": "-24h",+ "max_results": 300+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 60+ },+ {+ "id": "collect_forensics",+ "name": "Collect forensic artifacts before isolation",+ "type": "run_script",+ "params": {+ "host_field": "alert.host",+ "script": "collect_forensic_artifacts.sh",+ "args": ["--host", "{{inputs.host}}", "--output", "/forensics/{{alert.case_id}}/", "--include", "processes,network,registry,events"],+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 120+ },+ {+ "id": "approval_isolate",+ "name": "Approval gate — network-isolate endpoint",+ "type": "approval",+ "params": {+ "message": "Network-isolate endpoint '{{inputs.host}}' (user: {{inputs.user}}) due to {{inputs.compromise_type}}? This will cut all network connectivity except EDR management channel.",+ "approvers": ["soc-lead"],+ "timeout_seconds": 300,+ "auto_approve_severity": ["critical"]+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 320+ },+ {+ "id": "isolate_host",+ "name": "Network-isolate host via EDR",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.host",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_isolate.outcome == 'approved'",+ "on_failure": "abort",+ "retry_max": 1,+ "timeout_seconds": 60+ },+ {+ "id": "block_ioc_hash",+ "name": "Block IOC hash globally across fleet",+ "type": "block_ioc",+ "params": {+ "indicator_type": "hash",+ "indicator_field": "alert.file_hash",+ "dry_run": "{{dry_run}}"+ },+ "condition": "inputs.ioc_hash != null",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "run_av_scan",+ "name": "Run AV scan on isolated host",+ "type": "run_av_scan",+ "params": {+ "host_field": "alert.host",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.isolate_host.outcome == 'success'",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 300+ },+ {+ "id": "inv",+ "name": "Endpoint isolation investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "endpoint-compromise"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify SOC team and endpoint owner",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Endpoint '{{inputs.host}}' isolated due to {{inputs.compromise_type}} (user: {{inputs.user}}). Forensics collected: {{context.forensics_success}}. IOC blocked: {{context.block_ioc_success}}.",+ "service_key_env": "PD_SOC_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open endpoint isolation ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Endpoint Isolated: {{inputs.host}} — {{inputs.compromise_type}}",+ "queue": "soc",+ "labels": ["endpoint-isolation", "host-compromise", "edr"]+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/ids-critical/ids-critical-response.playbook.json b/services/agents/app/playbook/packs/v1/ids-critical/ids-critical-response.playbook.jsonnew file mode 100644index 000000000..9c362eac9--- /dev/null+++ b/services/agents/app/playbook/packs/v1/ids-critical/ids-critical-response.playbook.json@@ -0,0 +1,172 @@+{+ "id": "ids-critical-response-v1",+ "name": "IDS/IPS Critical Alert: Network Intrusion Response",+ "description": "Triggered by an IDS/IPS signature at critical severity indicating active exploitation, C2 communication, or lateral movement detected at the network layer. Enriches source IP, blocks traffic, and escalates.",+ "version": "1.0.0",+ "author": "Beenu - [email protected]",+ "tags": [+ "ids-critical",+ "ids",+ "ips",+ "network",+ "intrusion",+ "mitre.t1046",+ "mitre.t1190",+ "mitre.t1071"+ ],+ "trigger": {+ "on": "alert",+ "severity": ["high", "critical"],+ "tags": ["ids", "ips", "network-intrusion", "exploitation", "c2", "dga", "c2-beacon"]+ },+ "enabled": true,+ "dry_run_support": true,+ "created_at": "2026-05-10T08:00:00+00:00",+ "updated_at": "2026-05-10T08:00:00+00:00",+ "inputs": {+ "source_ip": {+ "description": "External attacker or suspect source IP",+ "field": "alert.source_ip",+ "required": true+ },+ "destination_ip": {+ "description": "Internal host targeted by the IDS/IPS alert",+ "field": "alert.destination_ip",+ "required": false+ },+ "signature": {+ "description": "IDS/IPS rule or signature name that fired",+ "field": "alert.ids_signature",+ "required": false+ },+ "protocol": {+ "description": "Network protocol (TCP/UDP/etc)",+ "field": "alert.protocol",+ "required": false+ },+ "dest_port": {+ "description": "Destination port",+ "field": "alert.destination_port",+ "required": false+ }+ },+ "steps": [+ {+ "id": "e1",+ "name": "Enrich source IP reputation",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.source_ip",+ "indicator_type": "ip"+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "siem_search",+ "name": "Search SIEM for prior activity from source IP",+ "type": "search_siem",+ "params": {+ "query": "source_ip={{inputs.source_ip}}",+ "time_window": "-48h",+ "max_results": 200+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 45+ },+ {+ "id": "block_source_ip",+ "name": "Block source IP at perimeter",+ "type": "block_ip",+ "params": {+ "ip_field": "alert.source_ip",+ "duration": "24h",+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "approval_contain_dest",+ "name": "Approval gate — isolate targeted internal host",+ "type": "approval",+ "params": {+ "message": "Isolate internal host '{{inputs.destination_ip}}' targeted by IDS/IPS critical alert (sig: {{inputs.signature}})?",+ "approvers": ["soc-lead", "network-lead"],+ "timeout_seconds": 300,+ "auto_approve_severity": ["critical"]+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 320+ },+ {+ "id": "isolate_dest_host",+ "name": "Network-isolate targeted host",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.destination_ip",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_contain_dest.outcome == 'approved' && inputs.destination_ip != null",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "create_siem_notable",+ "name": "Create SIEM notable event for IDS critical",+ "type": "create_notable_event",+ "params": {+ "title": "IDS Critical: {{inputs.signature}} from {{inputs.source_ip}}",+ "severity": "critical",+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "IDS/IPS critical investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "network-intrusion"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify SOC and network team",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "IDS Critical: {{inputs.signature}} from {{inputs.source_ip}} targeting {{inputs.destination_ip}}. Source IP blocked: {{context.block_success}}.",+ "service_key_env": "PD_SOC_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open IDS critical ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "IDS/IPS Critical: {{inputs.signature}} from {{inputs.source_ip}}",+ "queue": "soc",+ "labels": ["ids", "ips", "network-intrusion"]+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/insider-risk/insider-after-hours-access.playbook.json b/services/agents/app/playbook/packs/v1/insider-risk/insider-after-hours-access.playbook.jsonnew file mode 100644index 000000000..9ea99a446--- /dev/null+++ b/services/agents/app/playbook/packs/v1/insider-risk/insider-after-hours-access.playbook.json@@ -0,0 +1,75 @@+{+ "id": "insider-after-hours-access-v1",+ "name": "Insider: Anomalous After-Hours Access",+ "description": "Triggered when a user accesses sensitive systems outside their normal pattern. Step-up MFA and log for HR review.",+ "version": "1.0.0",+ "tags": [+ "insider-risk",+ "insider",+ "anomaly",+ "mitre.t1078"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "medium"+ ],+ "tags": [+ "insider-risk"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich user normal pattern",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.user"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "step-up",+ "name": "Force MFA step-up",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/users/{{alert.user}}/factors/challenge",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Score anomaly likelihood",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "identity"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "t1",+ "name": "Open after-hours review",+ "type": "create_ticket",+ "params": {+ "priority": "P4",+ "title_template": "After-hours access: {{alert.user}}",+ "queue": "hr"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/insider-risk/insider-mass-download.playbook.json b/services/agents/app/playbook/packs/v1/insider-risk/insider-mass-download.playbook.jsonnew file mode 100644index 000000000..f004a8f13--- /dev/null+++ b/services/agents/app/playbook/packs/v1/insider-risk/insider-mass-download.playbook.json@@ -0,0 +1,88 @@+{+ "id": "insider-mass-download-v1",+ "name": "Insider: Mass File Download",+ "description": "Triggered when a single user downloads >N files from a sensitive store in <T window. Reduces user permissions and notifies manager.",+ "version": "1.0.0",+ "tags": [+ "insider-risk",+ "insider",+ "dlp",+ "mitre.t1530"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high"+ ],+ "tags": [+ "insider-risk"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich user role + tenure",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.user"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "readonly",+ "name": "Set sensitive-store ACL to read-only",+ "type": "http",+ "params": {+ "url": "${SAAS_URL}/acl/{{alert.user}}/readonly",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate access pattern",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "identity"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify user's manager",+ "type": "notify",+ "params": {+ "channel": "email",+ "message_template": "High-volume download by {{alert.user}}. Review needed.",+ "webhook_env": "EMAIL_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open insider-risk ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P3",+ "title_template": "Mass download: {{alert.user}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/insider-risk/insider-privilege-misuse.playbook.json b/services/agents/app/playbook/packs/v1/insider-risk/insider-privilege-misuse.playbook.jsonnew file mode 100644index 000000000..38c8726ec--- /dev/null+++ b/services/agents/app/playbook/packs/v1/insider-risk/insider-privilege-misuse.playbook.json@@ -0,0 +1,78 @@+{+ "id": "insider-privilege-misuse-v1",+ "name": "Insider: Privilege Misuse",+ "description": "Triggered when a privileged account performs actions outside its approved JIT window. Revokes privilege and pages HR-Sec.",+ "version": "1.0.0",+ "tags": [+ "insider-risk",+ "insider",+ "privilege",+ "mitre.t1078.004"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "insider-risk"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "revoke-priv",+ "name": "Revoke privileged role",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/users/{{alert.user}}/roles/{{alert.role}}",+ "method": "DELETE"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate privilege misuse",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "identity"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page HR-Sec",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Privilege misuse by {{alert.user}} ({{alert.role}}). Role revoked.",+ "service_key_env": "PD_HRSEC_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open privilege-misuse ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Privilege misuse: {{alert.user}}",+ "queue": "hr"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/insider-risk/insider-resignation-data-access.playbook.json b/services/agents/app/playbook/packs/v1/insider-risk/insider-resignation-data-access.playbook.jsonnew file mode 100644index 000000000..60d1e3eb5--- /dev/null+++ b/services/agents/app/playbook/packs/v1/insider-risk/insider-resignation-data-access.playbook.json@@ -0,0 +1,89 @@+{+ "id": "insider-resignation-data-access-v1",+ "name": "Insider: Resignation + Sensitive Access",+ "description": "Triggered when a user with a recent resignation flag accesses sensitive data. Read-only mode and HR audit.",+ "version": "1.0.0",+ "tags": [+ "insider-risk",+ "insider",+ "hr",+ "mitre.t1530"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "medium",+ "high"+ ],+ "tags": [+ "insider-risk"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich HR resignation status",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.user"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "readonly",+ "name": "Force read-only on sensitive scopes",+ "type": "http",+ "params": {+ "url": "${SAAS_URL}/acl/{{alert.user}}/scopes/sensitive/readonly",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate exit-risk pattern",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "identity"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify HR + manager",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "Resigning user {{alert.user}} accessed sensitive data. Read-only enforced.",+ "webhook_env": "SLACK_HR_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open exit-risk ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "Exit risk: {{alert.user}}",+ "queue": "hr"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/insider-risk/insider-source-code-exfil.playbook.json b/services/agents/app/playbook/packs/v1/insider-risk/insider-source-code-exfil.playbook.jsonnew file mode 100644index 000000000..23c2e26d8--- /dev/null+++ b/services/agents/app/playbook/packs/v1/insider-risk/insider-source-code-exfil.playbook.json@@ -0,0 +1,89 @@+{+ "id": "insider-source-code-exfil-v1",+ "name": "Insider: Source-Code Exfil",+ "description": "Triggered when a developer pushes private source to a personal remote or pastebin-like target. Blocks dest, pages eng-leadership.",+ "version": "1.0.0",+ "tags": [+ "insider-risk",+ "insider",+ "source-code",+ "mitre.t1567"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "insider-risk"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich destination",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.dst_host"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "block-dest",+ "name": "Block destination at proxy",+ "type": "http",+ "params": {+ "url": "${PROXY_URL}/policy/block",+ "method": "POST",+ "body_template": "{\"host\":\"{{alert.dst_host}}\",\"reason\":\"src-exfil\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate exfil chain",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page eng-leadership",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Source-code exfil suspected from {{alert.user}} to {{alert.dst_host}}.",+ "service_key_env": "PD_ENG_LEAD_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open exfil ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Source-code exfil: {{alert.user}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/lateral-movement/lateral-cross-domain.playbook.json b/services/agents/app/playbook/packs/v1/lateral-movement/lateral-cross-domain.playbook.jsonnew file mode 100644index 000000000..0d38a2f8e--- /dev/null+++ b/services/agents/app/playbook/packs/v1/lateral-movement/lateral-cross-domain.playbook.json@@ -0,0 +1,78 @@+{+ "id": "lateral-cross-domain-v1",+ "name": "Lateral: Cross-Domain Movement",+ "description": "Triggered when an account from one trust domain authenticates to an asset in a separately-classified domain in a way that violates policy. Segments the network and investigates.",+ "version": "1.0.0",+ "tags": [+ "lateral-movement",+ "lateral",+ "trust",+ "mitre.t1021"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "lateral-movement"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "segment",+ "name": "Apply network segmentation rule",+ "type": "http",+ "params": {+ "url": "${FW_URL}/policy/segment",+ "method": "POST",+ "body_template": "{\"src\":\"{{alert.src_host}}\",\"dst_domain\":\"{{alert.dst_domain}}\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate cross-domain movement",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page on-call",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Cross-domain lateral {{alert.user}}. Segmentation applied.",+ "service_key_env": "PD_SOC_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open cross-domain ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Cross-domain lateral: {{alert.user}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/lateral-movement/lateral-kerberoasting.playbook.json b/services/agents/app/playbook/packs/v1/lateral-movement/lateral-kerberoasting.playbook.jsonnew file mode 100644index 000000000..284bdf34f--- /dev/null+++ b/services/agents/app/playbook/packs/v1/lateral-movement/lateral-kerberoasting.playbook.json@@ -0,0 +1,90 @@+{+ "id": "lateral-kerberoasting-v1",+ "name": "Lateral: Kerberoasting",+ "description": "Triggered by abnormal SPN-ticket request volume from a single user. Resets impacted service-account credentials.",+ "version": "1.0.0",+ "tags": [+ "lateral-movement",+ "lateral",+ "ad",+ "mitre.t1558.003"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high"+ ],+ "tags": [+ "lateral-movement"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "inv",+ "name": "Investigate kerberoasting",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "identity"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "reset-svc",+ "name": "Reset impacted SVC accounts",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/service-accounts/bulk-reset",+ "method": "POST",+ "body_template": "{\"accounts\": {{context.impacted_svc_accounts}}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "disable-user",+ "name": "Disable suspect user",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/users/{{alert.user}}/disable",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "n1",+ "name": "Page identity team",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Kerberoasting by {{alert.user}}. SVC accounts reset.",+ "service_key_env": "PD_IDENTITY_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open kerberoast ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Kerberoasting: {{alert.user}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/lateral-movement/lateral-pass-the-hash.playbook.json b/services/agents/app/playbook/packs/v1/lateral-movement/lateral-pass-the-hash.playbook.jsonnew file mode 100644index 000000000..d4b2ccbdc--- /dev/null+++ b/services/agents/app/playbook/packs/v1/lateral-movement/lateral-pass-the-hash.playbook.json@@ -0,0 +1,99 @@+{+ "id": "lateral-pass-the-hash-v1",+ "name": "Lateral: Pass-The-Hash",+ "description": "Triggered by NTLM relay / PTH detection. Disables the user and isolates source + destination.",+ "version": "1.0.0",+ "tags": [+ "lateral-movement",+ "lateral",+ "ad",+ "mitre.t1550.002"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "lateral-movement"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "disable-user",+ "name": "Disable account",+ "type": "http",+ "params": {+ "url": "${IDP_BASE_URL}/users/{{alert.user}}/disable",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "iso-src",+ "name": "Isolate source host",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.src_host"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "iso-dst",+ "name": "Isolate destination host",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.dst_host"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "inv",+ "name": "Investigate PTH chain",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page on-call",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "PTH detected: {{alert.user}} {{alert.src_host}} -> {{alert.dst_host}}. Account disabled.",+ "service_key_env": "PD_SOC_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open PTH ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "PTH: {{alert.user}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/lateral-movement/lateral-psexec-detected.playbook.json b/services/agents/app/playbook/packs/v1/lateral-movement/lateral-psexec-detected.playbook.jsonnew file mode 100644index 000000000..e9002c4f8--- /dev/null+++ b/services/agents/app/playbook/packs/v1/lateral-movement/lateral-psexec-detected.playbook.json@@ -0,0 +1,88 @@+{+ "id": "lateral-psexec-detected-v1",+ "name": "Lateral: PsExec / Remote Service Creation",+ "description": "Triggered by remote service creation patterns indicative of PsExec / Impacket. Isolates target and source, investigates.",+ "version": "1.0.0",+ "tags": [+ "lateral-movement",+ "lateral",+ "endpoint",+ "mitre.t1021.002"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "lateral-movement"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "iso-dst",+ "name": "Isolate target host",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.dst_host"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "iso-src",+ "name": "Isolate source host",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.src_host"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "inv",+ "name": "Investigate lateral chain",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page on-call",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "PsExec lateral {{alert.src_host}} -> {{alert.dst_host}}. Both isolated.",+ "service_key_env": "PD_SOC_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open lateral ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "PsExec lateral: {{alert.src_host}} -> {{alert.dst_host}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/lateral-movement/lateral-rdp-spray.playbook.json b/services/agents/app/playbook/packs/v1/lateral-movement/lateral-rdp-spray.playbook.jsonnew file mode 100644index 000000000..99b9c388c--- /dev/null+++ b/services/agents/app/playbook/packs/v1/lateral-movement/lateral-rdp-spray.playbook.json@@ -0,0 +1,88 @@+{+ "id": "lateral-rdp-spray-v1",+ "name": "Lateral: RDP Spray",+ "description": "Triggered when one host opens RDP to >N hosts in <T window. Blocks the source IP, isolates host, investigates.",+ "version": "1.0.0",+ "tags": [+ "lateral-movement",+ "lateral",+ "rdp",+ "mitre.t1021.001"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high"+ ],+ "tags": [+ "lateral-movement"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "b1",+ "name": "Block source IP",+ "type": "block_ip",+ "params": {+ "ip_field": "alert.src_ip",+ "duration": "24h"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "iso",+ "name": "Isolate source host",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.src_host"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "inv",+ "name": "Investigate RDP spray",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify SOC",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "RDP spray from {{alert.src_host}} blocked.",+ "webhook_env": "SLACK_SOC_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open RDP-spray ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "RDP spray: {{alert.src_host}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/malware/malware-execution-response.playbook.json b/services/agents/app/playbook/packs/v1/malware/malware-execution-response.playbook.jsonnew file mode 100644index 000000000..6b06b6134--- /dev/null+++ b/services/agents/app/playbook/packs/v1/malware/malware-execution-response.playbook.json@@ -0,0 +1,183 @@+{+ "id": "malware-execution-response-v1",+ "name": "Malware Execution: Detection & Containment",+ "description": "Triggered by EDR alert for confirmed or suspected malware execution (trojan, infostealer, dropper, RAT). Quarantines the host, kills the malicious process, collects evidence, and notifies the SOC.",+ "version": "1.0.0",+ "author": "Beenu - [email protected]",+ "tags": [+ "malware",+ "endpoint",+ "execution",+ "mitre.t1204",+ "mitre.t1059",+ "mitre.t1071"+ ],+ "trigger": {+ "on": "alert",+ "severity": ["medium", "high", "critical"],+ "tags": ["malware", "trojan", "rat", "dropper", "infostealer"]+ },+ "enabled": true,+ "dry_run_support": true,+ "created_at": "2026-05-10T08:00:00+00:00",+ "updated_at": "2026-05-10T08:00:00+00:00",+ "inputs": {+ "host": {+ "description": "Endpoint where malware was detected",+ "field": "alert.host",+ "required": true+ },+ "file_hash": {+ "description": "SHA-256 hash of the malicious file",+ "field": "alert.file_hash",+ "required": false+ },+ "process_id": {+ "description": "PID of the malicious process",+ "field": "alert.process_id",+ "required": false+ },+ "file_path": {+ "description": "Disk path of the malicious file",+ "field": "alert.file_path",+ "required": false+ },+ "user": {+ "description": "User context under which malware ran",+ "field": "alert.user",+ "required": false+ }+ },+ "steps": [+ {+ "id": "e1",+ "name": "Enrich file hash (threat intelligence)",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.file_hash",+ "indicator_type": "hash",+ "skip_if_empty": true+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "av_scan",+ "name": "Run full AV scan on host",+ "type": "run_av_scan",+ "params": {+ "host_field": "alert.host",+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "kill_process",+ "name": "Kill malicious process",+ "type": "kill_process",+ "params": {+ "host_field": "alert.host",+ "process_field": "alert.process_id",+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "quarantine_file",+ "name": "Quarantine malicious file",+ "type": "quarantine_file",+ "params": {+ "host_field": "alert.host",+ "file_path_field": "alert.file_path",+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "approval_isolate",+ "name": "Approval gate — isolate host",+ "type": "approval",+ "params": {+ "message": "Isolate host '{{inputs.host}}' from network due to confirmed malware execution?",+ "approvers": ["soc-lead"],+ "timeout_seconds": 300,+ "auto_approve_severity": ["critical"]+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 320+ },+ {+ "id": "isolate_host",+ "name": "Network-isolate host (EDR)",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.host",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_isolate.outcome == 'approved'",+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "block_hash_ioc",+ "name": "Block malicious hash as global IOC",+ "type": "block_ioc",+ "params": {+ "indicator_type": "hash",+ "indicator_field": "alert.file_hash",+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Malware execution investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "malware"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify SOC team",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "Malware execution on {{inputs.host}} (user: {{inputs.user}}). File quarantined: {{context.quarantine_success}}. Host isolated: {{context.isolate_success}}.",+ "webhook_env": "SLACK_SOC_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open malware ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "Malware Execution: {{inputs.host}} ({{inputs.file_hash}})",+ "queue": "soc",+ "labels": ["malware", "endpoint"]+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/network-containment/network-containment-response.playbook.json b/services/agents/app/playbook/packs/v1/network-containment/network-containment-response.playbook.jsonnew file mode 100644index 000000000..b6ed77db9--- /dev/null+++ b/services/agents/app/playbook/packs/v1/network-containment/network-containment-response.playbook.json@@ -0,0 +1,164 @@+{+ "id": "network-containment-response-v1",+ "name": "Network Containment: Segment Isolation Response",+ "description": "Triggered by an alert requiring network-level containment of a host or subnet. Coordinates firewall/security-group rule changes to block lateral movement while preserving forensics and management access.",+ "version": "1.0.0",+ "author": "Beenu - [email protected]",+ "tags": [+ "network-containment",+ "containment",+ "network",+ "mitre.t1562.001",+ "mitre.t1599"+ ],+ "trigger": {+ "on": "alert",+ "severity": ["high", "critical"],+ "tags": ["network-containment", "lateral-movement", "c2", "ransomware"]+ },+ "enabled": true,+ "dry_run_support": true,+ "created_at": "2026-05-10T08:00:00+00:00",+ "updated_at": "2026-05-10T08:00:00+00:00",+ "inputs": {+ "host_or_subnet": {+ "description": "Host IP or CIDR subnet to contain",+ "field": "alert.host",+ "required": true+ },+ "threat_type": {+ "description": "Threat driving the containment (ransomware, c2, lateral-movement)",+ "field": "alert.threat_type",+ "required": false+ },+ "allow_mgmt_ips": {+ "description": "Management IPs to keep allowed during containment",+ "field": "alert.mgmt_ips",+ "required": false+ }+ },+ "steps": [+ {+ "id": "e1",+ "name": "Enrich host reputation and asset context",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.host",+ "indicator_type": "ip"+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "siem_search",+ "name": "Search SIEM for outbound connections from host",+ "type": "search_siem",+ "params": {+ "query": "src_ip={{inputs.host_or_subnet}} AND event_type=network",+ "time_window": "-6h",+ "max_results": 500+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 60+ },+ {+ "id": "approval_contain",+ "name": "Approval gate — apply network containment",+ "type": "approval",+ "params": {+ "message": "Apply network containment for '{{inputs.host_or_subnet}}'? This will block all east-west traffic except management access.",+ "approvers": ["soc-lead", "network-lead"],+ "timeout_seconds": 300,+ "auto_approve_severity": ["critical"]+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 320+ },+ {+ "id": "block_inbound_ip",+ "name": "Block inbound traffic to host via security group",+ "type": "block_ip",+ "params": {+ "ip_field": "alert.host",+ "direction": "inbound",+ "duration": "48h",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_contain.outcome == 'approved'",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "block_outbound_ip",+ "name": "Block outbound traffic from host via security group",+ "type": "block_ip",+ "params": {+ "ip_field": "alert.host",+ "direction": "outbound",+ "duration": "48h",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_contain.outcome == 'approved'",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "run_collection_script",+ "name": "Collect network flow evidence before isolation completes",+ "type": "run_script",+ "params": {+ "host_field": "alert.host",+ "script": "collect_netflow_evidence.sh",+ "args": ["--host", "{{inputs.host_or_subnet}}", "--output", "/forensics/{{alert.case_id}}/"],+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 60+ },+ {+ "id": "inv",+ "name": "Network containment investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "network-containment"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify network and SOC teams",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Network containment applied for {{inputs.host_or_subnet}} (threat: {{inputs.threat_type}}). Inbound blocked: {{context.block_inbound_success}}. Outbound blocked: {{context.block_outbound_success}}.",+ "service_key_env": "PD_NETWORK_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open network containment ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Network Containment: {{inputs.host_or_subnet}} ({{inputs.threat_type}})",+ "queue": "soc",+ "labels": ["network-containment", "isolation"]+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/phishing/phishing-investigation.playbook.json b/services/agents/app/playbook/packs/v1/phishing/phishing-investigation.playbook.jsonnew file mode 100644index 000000000..e5637937c--- /dev/null+++ b/services/agents/app/playbook/packs/v1/phishing/phishing-investigation.playbook.json@@ -0,0 +1,195 @@+{+ "id": "phishing-investigation-v1",+ "name": "Phishing: Email Investigation & Remediation",+ "description": "Triggered by a phishing alert from email gateway or user report. Extracts IOCs, retracts the message fleet-wide, blocks sender domain, and resets any credentials that may have been compromised.",+ "version": "1.0.0",+ "author": "Beenu - [email protected]",+ "tags": [+ "phishing",+ "email",+ "identity",+ "mitre.t1566",+ "mitre.t1566.001"+ ],+ "trigger": {+ "on": "alert",+ "severity": ["high", "critical"],+ "tags": ["phishing", "email-security"]+ },+ "enabled": true,+ "dry_run_support": true,+ "created_at": "2026-05-10T08:00:00+00:00",+ "updated_at": "2026-05-10T08:00:00+00:00",+ "inputs": {+ "message_id": {+ "description": "Email message ID from the alert",+ "field": "alert.message_id",+ "required": true+ },+ "sender_domain": {+ "description": "Sender domain extracted from the From header",+ "field": "alert.sender_domain",+ "required": true+ },+ "recipient_user": {+ "description": "Primary recipient user identity",+ "field": "alert.recipient_user",+ "required": true+ },+ "attachment_hash": {+ "description": "SHA-256 of any malicious attachment",+ "field": "alert.file_hash",+ "required": false+ }+ },+ "steps": [+ {+ "id": "e1",+ "name": "Enrich sender domain reputation",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.sender_domain",+ "indicator_type": "domain"+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "e2",+ "name": "Enrich attachment hash (if present)",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.file_hash",+ "indicator_type": "hash",+ "skip_if_empty": true+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "retract",+ "name": "Retract phishing email fleet-wide",+ "type": "http",+ "params": {+ "url": "${EMAIL_GATEWAY_URL}/v1/messages/retract",+ "method": "POST",+ "body_template": "{\"message_id\": \"{{inputs.message_id}}\", \"reason\": \"phishing\"}",+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "approval_block_domain",+ "name": "Approval gate — block sender domain",+ "type": "approval",+ "params": {+ "message": "Block sender domain '{{inputs.sender_domain}}' at email gateway and DNS firewall?",+ "approvers": ["soc-lead", "security-manager"],+ "timeout_seconds": 900,+ "auto_approve_severity": ["critical"]+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 920+ },+ {+ "id": "block_domain",+ "name": "Block sender domain",+ "type": "block_ip",+ "params": {+ "ip_field": "alert.sender_domain",+ "block_type": "domain",+ "duration": "72h",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_block_domain.outcome == 'approved'",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "check_clicked",+ "name": "Check if recipient clicked link / opened attachment",+ "type": "http",+ "params": {+ "url": "${SIEM_URL}/search",+ "method": "POST",+ "body_template": "{\"query\": \"email_message_id={{inputs.message_id}} AND (url_click=true OR attachment_open=true)\", \"earliest\": \"-1h\"}"+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "approval_reset",+ "name": "Approval gate — force password reset (if click detected)",+ "type": "approval",+ "params": {+ "message": "Recipient '{{inputs.recipient_user}}' may have clicked link. Force password reset + MFA?",+ "approvers": ["soc-lead"],+ "timeout_seconds": 600,+ "condition_skip": "context.click_detected == false"+ },+ "condition": "context.click_detected == true",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 620+ },+ {+ "id": "reset_pw",+ "name": "Force password reset for recipient",+ "type": "reset_password",+ "params": {+ "user_field": "inputs.recipient_user",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_reset.outcome == 'approved'",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Phishing investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "phishing"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify security team",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "Phishing alert: {{inputs.sender_domain}} → {{inputs.recipient_user}}. Message retracted: {{context.retract_success}}. Domain blocked: {{context.domain_blocked}}.",+ "webhook_env": "SLACK_SOC_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open phishing ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "Phishing: {{inputs.sender_domain}} targeting {{inputs.recipient_user}}",+ "queue": "soc",+ "labels": ["phishing", "email"]+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/privilege-escalation/privesc-response.playbook.json b/services/agents/app/playbook/packs/v1/privilege-escalation/privesc-response.playbook.jsonnew file mode 100644index 000000000..fa37459db--- /dev/null+++ b/services/agents/app/playbook/packs/v1/privilege-escalation/privesc-response.playbook.json@@ -0,0 +1,166 @@+{+ "id": "privesc-response-v1",+ "name": "Privilege Escalation: Detection & Containment",+ "description": "Triggered by EDR or SIEM alert for privilege escalation activity (sudo abuse, token impersonation, LSASS dump, UAC bypass, etc.). Collects forensics, disables the account if confirmed malicious, and escalates to Tier-2 for post-exploitation review.",+ "version": "1.0.0",+ "author": "Beenu - [email protected]",+ "tags": [+ "privilege-escalation",+ "endpoint",+ "identity",+ "mitre.t1068",+ "mitre.t1548",+ "mitre.t1134"+ ],+ "trigger": {+ "on": "alert",+ "severity": ["high", "critical"],+ "tags": ["privilege-escalation", "privesc", "uac-bypass", "token-impersonation"]+ },+ "enabled": true,+ "dry_run_support": true,+ "created_at": "2026-05-10T08:00:00+00:00",+ "updated_at": "2026-05-10T08:00:00+00:00",+ "inputs": {+ "host": {+ "description": "Endpoint where privilege escalation was detected",+ "field": "alert.host",+ "required": true+ },+ "user": {+ "description": "User account performing the escalation",+ "field": "alert.user",+ "required": true+ },+ "technique": {+ "description": "MITRE ATT&CK technique ID (e.g. T1068)",+ "field": "alert.mitre_technique",+ "required": false+ },+ "process_name": {+ "description": "Offending process name",+ "field": "alert.process_name",+ "required": false+ }+ },+ "steps": [+ {+ "id": "e1",+ "name": "Search SIEM for prior activity by this user on this host",+ "type": "search_siem",+ "params": {+ "query": "user={{inputs.user}} AND host={{inputs.host}}",+ "time_window": "-24h",+ "max_results": 100+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 45+ },+ {+ "id": "av_scan",+ "name": "Run AV / EDR scan on host",+ "type": "run_av_scan",+ "params": {+ "host_field": "alert.host",+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 120+ },+ {+ "id": "collect_forensics",+ "name": "Collect forensic evidence via EDR script",+ "type": "run_script",+ "params": {+ "host_field": "alert.host",+ "script": "collect_privesc_evidence.sh",+ "args": ["--user", "{{inputs.user}}", "--process", "{{inputs.process_name}}", "--output", "/forensics/{{alert.case_id}}/"],+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 120+ },+ {+ "id": "approval_disable_user",+ "name": "Approval gate — disable user account",+ "type": "approval",+ "params": {+ "message": "Disable user account '{{inputs.user}}' due to privilege escalation on '{{inputs.host}}'?",+ "approvers": ["soc-lead", "identity-admin"],+ "timeout_seconds": 600,+ "auto_approve_severity": ["critical"]+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 620+ },+ {+ "id": "disable_user",+ "name": "Disable user account in IdP",+ "type": "disable_user",+ "params": {+ "user_field": "inputs.user",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_disable_user.outcome == 'approved'",+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "revoke_sessions",+ "name": "Revoke all active sessions for user",+ "type": "revoke_session",+ "params": {+ "user_field": "inputs.user",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_disable_user.outcome == 'approved'",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Privilege escalation investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "privilege-escalation"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify SOC Tier-2",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "PrivEsc alert: {{inputs.user}} on {{inputs.host}} (technique: {{inputs.technique}}). Account disabled: {{context.disable_success}}.",+ "webhook_env": "SLACK_SOC_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open privilege escalation ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "PrivEsc: {{inputs.user}} on {{inputs.host}} ({{inputs.technique}})",+ "queue": "soc",+ "labels": ["privilege-escalation", "endpoint", "identity"]+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/ransomware/ransomware-c2-block.playbook.json b/services/agents/app/playbook/packs/v1/ransomware/ransomware-c2-block.playbook.jsonnew file mode 100644index 000000000..d508802e9--- /dev/null+++ b/services/agents/app/playbook/packs/v1/ransomware/ransomware-c2-block.playbook.json@@ -0,0 +1,98 @@+{+ "id": "ransomware-c2-block-v1",+ "name": "Ransomware: C2 Beacon Block",+ "description": "Triggered by C2 beaconing detections that match ransomware-family TTPs. Blocks C2 destinations at edge, isolates source host.",+ "version": "1.0.0",+ "tags": [+ "ransomware",+ "c2",+ "mitre.t1071"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "ransomware"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich destination IP",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.dst_ip"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "b1",+ "name": "Block C2 destination",+ "type": "block_ip",+ "params": {+ "ip_field": "alert.dst_ip",+ "duration": "30d"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "iso",+ "name": "Isolate beaconing host",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.host"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "inv",+ "name": "Investigate ransomware C2",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify SOC",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "C2 {{alert.dst_ip}} blocked. Host {{alert.host}} isolated.",+ "webhook_env": "SLACK_SOC_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open C2 ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Ransomware C2: {{alert.host}} -> {{alert.dst_ip}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/ransomware/ransomware-fileserver-encrypt.playbook.json b/services/agents/app/playbook/packs/v1/ransomware/ransomware-fileserver-encrypt.playbook.jsonnew file mode 100644index 000000000..ecf5079a5--- /dev/null+++ b/services/agents/app/playbook/packs/v1/ransomware/ransomware-fileserver-encrypt.playbook.json@@ -0,0 +1,88 @@+{+ "id": "ransomware-fileserver-encrypt-v1",+ "name": "Ransomware: File-Server Encryption Activity",+ "description": "Triggered by mass-rename / mass-extension-change on a file server. Network-segments the host to halt SMB lateral encryption.",+ "version": "1.0.0",+ "tags": [+ "ransomware",+ "fileserver",+ "mitre.t1486"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "ransomware"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "iso",+ "name": "Network-segment file server",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.host"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "smb-deny",+ "name": "Push deny rule to firewall (SMB)",+ "type": "http",+ "params": {+ "url": "${FW_URL}/policy/deny-smb",+ "method": "POST",+ "body_template": "{\"src\":\"{{alert.host}}\",\"port\":445}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Forensic investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page storage and on-call",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "File-server ransomware: {{alert.host}}. SMB egress blocked.",+ "service_key_env": "PD_STORAGE_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open SEV1 ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "File-server ransomware: {{alert.host}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/ransomware/ransomware-host-isolate.playbook.json b/services/agents/app/playbook/packs/v1/ransomware/ransomware-host-isolate.playbook.jsonnew file mode 100644index 000000000..4e0b01704--- /dev/null+++ b/services/agents/app/playbook/packs/v1/ransomware/ransomware-host-isolate.playbook.json@@ -0,0 +1,98 @@+{+ "id": "ransomware-host-isolate-v1",+ "name": "Ransomware: Host Isolation & Snapshot",+ "description": "Immediate host isolation, disk snapshot for forensics, page on-call. Standard first-response for endpoint ransomware.",+ "version": "1.0.0",+ "tags": [+ "ransomware",+ "endpoint",+ "mitre.t1486"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "ransomware"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "iso",+ "name": "Isolate host (EDR)",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.host"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "snap",+ "name": "Snapshot host disk",+ "type": "http",+ "params": {+ "url": "${EDR_URL}/hosts/{{alert.host}}/snapshot",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "e1",+ "name": "Enrich file hash",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.file_hash"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "inv",+ "name": "Forensic investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page on-call (SEV1)",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "RANSOMWARE: {{alert.host}} isolated. Snapshot {{context.snapshot_id}} taken.",+ "service_key_env": "PD_RANSOMWARE_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open SEV1 ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "RANSOMWARE: {{alert.host}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/ransomware/ransomware-public-exposure.playbook.json b/services/agents/app/playbook/packs/v1/ransomware/ransomware-public-exposure.playbook.jsonnew file mode 100644index 000000000..6bb6c9b56--- /dev/null+++ b/services/agents/app/playbook/packs/v1/ransomware/ransomware-public-exposure.playbook.json@@ -0,0 +1,99 @@+{+ "id": "ransomware-public-exposure-v1",+ "name": "Ransomware: Internet-Exposed Asset Containment",+ "description": "Triggered when a host with public exposure shows ransomware indicators. Pulls the asset off the internet immediately.",+ "version": "1.0.0",+ "tags": [+ "ransomware",+ "exposure",+ "mitre.t1486"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "ransomware"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "detach-eip",+ "name": "Detach public IP",+ "type": "http",+ "params": {+ "url": "${CLOUD_URL}/instances/{{alert.host}}/detach-public-ip",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "iso",+ "name": "Isolate host (EDR)",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.host"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "snap",+ "name": "Snapshot host disk",+ "type": "http",+ "params": {+ "url": "${EDR_URL}/hosts/{{alert.host}}/snapshot",+ "method": "POST"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Forensic investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page on-call (SEV1)",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Internet-exposed host {{alert.host}} pulled off-net. Ransomware suspected.",+ "service_key_env": "PD_RANSOMWARE_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open SEV1 ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Ransomware on exposed host: {{alert.host}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/ransomware/ransomware-shadow-copy-deletion.playbook.json b/services/agents/app/playbook/packs/v1/ransomware/ransomware-shadow-copy-deletion.playbook.jsonnew file mode 100644index 000000000..a1a18d8b9--- /dev/null+++ b/services/agents/app/playbook/packs/v1/ransomware/ransomware-shadow-copy-deletion.playbook.json@@ -0,0 +1,87 @@+{+ "id": "ransomware-shadow-copy-deletion-v1",+ "name": "Ransomware: Shadow Copy Deletion Detected",+ "description": "Triggered on `vssadmin delete shadows` or wmic shadowcopy delete. Pre-encryption signal \u2014 isolate fast and verify backups.",+ "version": "1.0.0",+ "tags": [+ "ransomware",+ "endpoint",+ "mitre.t1490"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "critical"+ ],+ "tags": [+ "ransomware"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "iso",+ "name": "Isolate host (EDR)",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.host"+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "verify-backup",+ "name": "Verify last successful backup",+ "type": "http",+ "params": {+ "url": "${BACKUP_URL}/hosts/{{alert.host}}/last-backup",+ "method": "GET"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Forensic investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "forensics"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify backup team",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "Shadow-copy deletion on {{alert.host}}. Last backup: {{context.last_backup_at}}.",+ "webhook_env": "SLACK_BACKUP_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open ransomware ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Shadow copies deleted: {{alert.host}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/supply-chain/supply-github-action-compromise.playbook.json b/services/agents/app/playbook/packs/v1/supply-chain/supply-github-action-compromise.playbook.jsonnew file mode 100644index 000000000..a6e36c86c--- /dev/null+++ b/services/agents/app/playbook/packs/v1/supply-chain/supply-github-action-compromise.playbook.json@@ -0,0 +1,90 @@+{+ "id": "supply-github-action-compromise-v1",+ "name": "Supply Chain: GitHub Action Compromise",+ "description": "Triggered when a referenced GitHub Action SHA changes unexpectedly or matches a compromise advisory. Pin to last-known-good and audit.",+ "version": "1.0.0",+ "tags": [+ "supply-chain",+ "github",+ "mitre.t1195.002"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "supply-chain"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "pin",+ "name": "Pin Action to last-known-good SHA",+ "type": "http",+ "params": {+ "url": "${CI_URL}/actions/pin",+ "method": "POST",+ "body_template": "{\"action\":\"{{alert.action}}\",\"sha\":\"{{alert.lkg_sha}}\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "audit",+ "name": "List workflows using this Action",+ "type": "http",+ "params": {+ "url": "${CI_URL}/workflows?action={{alert.action}}",+ "method": "GET"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate Action compromise",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page eng-sec",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "GH Action {{alert.action}} pinned to {{alert.lkg_sha}}.",+ "service_key_env": "PD_ENGSEC_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open Action-compromise ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "GH Action compromise: {{alert.action}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/supply-chain/supply-iac-drift.playbook.json b/services/agents/app/playbook/packs/v1/supply-chain/supply-iac-drift.playbook.jsonnew file mode 100644index 000000000..2f82707d8--- /dev/null+++ b/services/agents/app/playbook/packs/v1/supply-chain/supply-iac-drift.playbook.json@@ -0,0 +1,78 @@+{+ "id": "supply-iac-drift-v1",+ "name": "Supply Chain: IaC Drift Detected",+ "description": "Triggered when production infra deviates from IaC source of truth in a security-relevant way. Reverts to declared state.",+ "version": "1.0.0",+ "tags": [+ "supply-chain",+ "iac",+ "drift"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "medium",+ "high"+ ],+ "tags": [+ "supply-chain"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "revert",+ "name": "Revert to declared IaC state",+ "type": "http",+ "params": {+ "url": "${IAC_URL}/apply",+ "method": "POST",+ "body_template": "{\"resource\":\"{{alert.resource_id}}\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate drift cause",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify platform team",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "IaC drift {{alert.resource_id}} reverted.",+ "webhook_env": "SLACK_PLATFORM_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open IaC-drift ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P3",+ "title_template": "IaC drift: {{alert.resource_id}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/supply-chain/supply-malicious-npm.playbook.json b/services/agents/app/playbook/packs/v1/supply-chain/supply-malicious-npm.playbook.jsonnew file mode 100644index 000000000..e00d494e2--- /dev/null+++ b/services/agents/app/playbook/packs/v1/supply-chain/supply-malicious-npm.playbook.json@@ -0,0 +1,101 @@+{+ "id": "supply-malicious-npm-v1",+ "name": "Supply Chain: Malicious npm Package",+ "description": "Triggered when a published npm package matches a known-bad list. Blocks installs at registry-mirror, audits org installs.",+ "version": "1.0.0",+ "tags": [+ "supply-chain",+ "npm",+ "mitre.t1195"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "supply-chain"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "e1",+ "name": "Enrich package reputation",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.package_name"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "block",+ "name": "Add package to mirror block-list",+ "type": "http",+ "params": {+ "url": "${REGISTRY_URL}/policy/block",+ "method": "POST",+ "body_template": "{\"ecosystem\":\"npm\",\"package\":\"{{alert.package_name}}\",\"version\":\"{{alert.package_version}}\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "audit-installs",+ "name": "List org installs",+ "type": "http",+ "params": {+ "url": "${SCA_URL}/installs?ecosystem=npm&package={{alert.package_name}}",+ "method": "GET"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate supply-chain blast radius",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Page eng-sec",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Malicious npm {{alert.package_name}} blocked. {{context.install_count}} installs found.",+ "service_key_env": "PD_ENGSEC_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open supply-chain ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Malicious npm: {{alert.package_name}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/supply-chain/supply-pypi-typosquat.playbook.json b/services/agents/app/playbook/packs/v1/supply-chain/supply-pypi-typosquat.playbook.jsonnew file mode 100644index 000000000..745c07b30--- /dev/null+++ b/services/agents/app/playbook/packs/v1/supply-chain/supply-pypi-typosquat.playbook.json@@ -0,0 +1,89 @@+{+ "id": "supply-pypi-typosquat-v1",+ "name": "Supply Chain: PyPI Typosquat",+ "description": "Triggered when a typosquat package is observed in CI installs. Pulls install lock to canonical name and audits.",+ "version": "1.0.0",+ "tags": [+ "supply-chain",+ "pypi",+ "mitre.t1195"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high"+ ],+ "tags": [+ "supply-chain"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "block",+ "name": "Block typosquat at mirror",+ "type": "http",+ "params": {+ "url": "${REGISTRY_URL}/policy/block",+ "method": "POST",+ "body_template": "{\"ecosystem\":\"pypi\",\"package\":\"{{alert.package_name}}\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "audit",+ "name": "List repos with this dependency",+ "type": "http",+ "params": {+ "url": "${SCA_URL}/repos?ecosystem=pypi&package={{alert.package_name}}",+ "method": "GET"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate typosquat",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify eng team",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "PyPI typosquat {{alert.package_name}} blocked. {{context.repo_count}} repos affected.",+ "webhook_env": "SLACK_ENG_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open typosquat ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "PyPI typosquat: {{alert.package_name}}",+ "queue": "soc"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/supply-chain/supply-vendor-breach.playbook.json b/services/agents/app/playbook/packs/v1/supply-chain/supply-vendor-breach.playbook.jsonnew file mode 100644index 000000000..ce4c34760--- /dev/null+++ b/services/agents/app/playbook/packs/v1/supply-chain/supply-vendor-breach.playbook.json@@ -0,0 +1,78 @@+{+ "id": "supply-vendor-breach-v1",+ "name": "Supply Chain: Vendor Breach Notification",+ "description": "Triggered when a vendor publishes a breach advisory matching internal vendor inventory. Blocks vendor IPs, opens procurement review.",+ "version": "1.0.0",+ "tags": [+ "supply-chain",+ "vendor",+ "mitre.t1199"+ ],+ "trigger": {+ "on": "alert",+ "severity": [+ "high",+ "critical"+ ],+ "tags": [+ "supply-chain"+ ]+ },+ "author": "AiSOC",+ "enabled": true,+ "created_at": "2026-05-03T00:00:00+00:00",+ "updated_at": "2026-05-03T00:00:00+00:00",+ "steps": [+ {+ "id": "block-vendor-ips",+ "name": "Block vendor IP ranges",+ "type": "http",+ "params": {+ "url": "${FW_URL}/policy/block-vendor",+ "method": "POST",+ "body_template": "{\"vendor\":\"{{alert.vendor}}\"}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Investigate vendor exposure",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "cloud"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify procurement + sec",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "Vendor {{alert.vendor}} breach. IPs blocked. Review contracts and integrations.",+ "webhook_env": "SLACK_PROCUREMENT_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open vendor-breach ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Vendor breach: {{alert.vendor}}",+ "queue": "procurement"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/suspicious-signin/suspicious-signin-response.playbook.json b/services/agents/app/playbook/packs/v1/suspicious-signin/suspicious-signin-response.playbook.jsonnew file mode 100644index 000000000..a300fa2d0--- /dev/null+++ b/services/agents/app/playbook/packs/v1/suspicious-signin/suspicious-signin-response.playbook.json@@ -0,0 +1,184 @@+{+ "id": "suspicious-signin-response-v1",+ "name": "Suspicious Sign-In: Anomalous Authentication Response",+ "description": "Triggered by identity provider or SIEM alert for a suspicious sign-in event (impossible travel, new device, unknown location, off-hours, anomalous behavior score). Verifies legitimacy, enforces MFA step-up, and disables if confirmed malicious.",+ "version": "1.0.0",+ "author": "Beenu - [email protected]",+ "tags": [+ "suspicious-signin",+ "identity",+ "authentication",+ "mitre.t1078",+ "mitre.t1110",+ "mitre.t1556"+ ],+ "trigger": {+ "on": "alert",+ "severity": ["medium", "high", "critical"],+ "tags": ["suspicious-signin", "anomalous-login", "impossible-travel", "new-device"]+ },+ "enabled": true,+ "dry_run_support": true,+ "created_at": "2026-05-10T08:00:00+00:00",+ "updated_at": "2026-05-10T08:00:00+00:00",+ "inputs": {+ "user": {+ "description": "User account that performed the suspicious sign-in",+ "field": "alert.user",+ "required": true+ },+ "source_ip": {+ "description": "Source IP of the suspicious login",+ "field": "alert.source_ip",+ "required": false+ },+ "user_agent": {+ "description": "Browser/device user-agent string",+ "field": "alert.user_agent",+ "required": false+ },+ "country": {+ "description": "Country where the login originated",+ "field": "alert.geo_country",+ "required": false+ },+ "risk_score": {+ "description": "Risk score from IdP (0-100)",+ "field": "alert.risk_score",+ "required": false+ }+ },+ "steps": [+ {+ "id": "e1",+ "name": "Enrich source IP geo and threat intel",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.source_ip",+ "indicator_type": "ip",+ "skip_if_empty": true+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "siem_search",+ "name": "Search SIEM for recent sign-in history for this user",+ "type": "search_siem",+ "params": {+ "query": "user={{inputs.user}} AND event_type=authentication",+ "time_window": "-7d",+ "max_results": 100+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 45+ },+ {+ "id": "force_mfa",+ "name": "Force step-up MFA challenge for user",+ "type": "force_mfa",+ "params": {+ "user_field": "inputs.user",+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "revoke_sessions",+ "name": "Revoke current active sessions (force re-auth)",+ "type": "revoke_session",+ "params": {+ "user_field": "inputs.user",+ "dry_run": "{{dry_run}}"+ },+ "condition": "inputs.risk_score >= 70 or inputs.country == 'high_risk'",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "approval_disable_user",+ "name": "Approval gate — disable user account (if confirmed compromise)",+ "type": "approval",+ "params": {+ "message": "Disable account for '{{inputs.user}}'? Sign-in from {{inputs.country}} (IP: {{inputs.source_ip}}, risk: {{inputs.risk_score}}). Only if confirmed malicious.",+ "approvers": ["soc-lead", "identity-admin"],+ "timeout_seconds": 600,+ "auto_approve_severity": ["critical"]+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 620+ },+ {+ "id": "disable_user",+ "name": "Disable user account in IdP",+ "type": "disable_user",+ "params": {+ "user_field": "inputs.user",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_disable_user.outcome == 'approved'",+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "block_source_ip",+ "name": "Block source IP if confirmed malicious",+ "type": "block_ip",+ "params": {+ "ip_field": "alert.source_ip",+ "duration": "24h",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_disable_user.outcome == 'approved' && inputs.source_ip != null",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "inv",+ "name": "Suspicious sign-in investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "suspicious-signin"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Notify identity security team",+ "type": "notify",+ "params": {+ "channel": "slack",+ "message_template": "Suspicious sign-in for {{inputs.user}} from {{inputs.country}} (IP: {{inputs.source_ip}}, risk: {{inputs.risk_score}}). MFA forced: {{context.mfa_success}}. Sessions revoked: {{context.revoke_success}}.",+ "webhook_env": "SLACK_SOC_WEBHOOK"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open suspicious sign-in ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P2",+ "title_template": "Suspicious Sign-In: {{inputs.user}} from {{inputs.country}}",+ "queue": "soc",+ "labels": ["suspicious-signin", "identity", "authentication"]+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/packs/v1/web-compromise/web-compromise-response.playbook.json b/services/agents/app/playbook/packs/v1/web-compromise/web-compromise-response.playbook.jsonnew file mode 100644index 000000000..0f1451e8f--- /dev/null+++ b/services/agents/app/playbook/packs/v1/web-compromise/web-compromise-response.playbook.json@@ -0,0 +1,180 @@+{+ "id": "web-compromise-response-v1",+ "name": "Web Compromise: Defacement / Webshell Response",+ "description": "Triggered by WAF alert, IDS signature, or file-integrity alert indicating a compromised web server (webshell upload, defacement, or unauthorized content modification). Isolates the server, captures forensics, and initiates recovery workflow.",+ "version": "1.0.0",+ "author": "Beenu - [email protected]",+ "tags": [+ "web-compromise",+ "web",+ "endpoint",+ "mitre.t1505.003",+ "mitre.t1190"+ ],+ "trigger": {+ "on": "alert",+ "severity": ["high", "critical"],+ "tags": ["web-compromise", "webshell", "defacement"]+ },+ "enabled": true,+ "dry_run_support": true,+ "created_at": "2026-05-10T08:00:00+00:00",+ "updated_at": "2026-05-10T08:00:00+00:00",+ "inputs": {+ "host": {+ "description": "Compromised web server hostname or IP",+ "field": "alert.host",+ "required": true+ },+ "webshell_path": {+ "description": "Path of the detected webshell or modified file",+ "field": "alert.file_path",+ "required": false+ },+ "attacker_ip": {+ "description": "Source IP of the attacker (from WAF / access logs)",+ "field": "alert.source_ip",+ "required": false+ },+ "file_hash": {+ "description": "Hash of the suspicious file",+ "field": "alert.file_hash",+ "required": false+ }+ },+ "steps": [+ {+ "id": "e1",+ "name": "Enrich attacker IP reputation",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.source_ip",+ "indicator_type": "ip",+ "skip_if_empty": true+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "e2",+ "name": "Enrich file hash (webshell signature lookup)",+ "type": "enrich",+ "params": {+ "indicator_field": "alert.file_hash",+ "indicator_type": "hash",+ "skip_if_empty": true+ },+ "on_failure": "continue",+ "retry_max": 1,+ "timeout_seconds": 30+ },+ {+ "id": "block_attacker_ip",+ "name": "Block attacker IP at WAF/firewall",+ "type": "block_ip",+ "params": {+ "ip_field": "alert.source_ip",+ "duration": "72h",+ "skip_if_empty": true,+ "dry_run": "{{dry_run}}"+ },+ "condition": "inputs.attacker_ip != null",+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ },+ {+ "id": "av_scan",+ "name": "Trigger AV / malware scan on web server",+ "type": "run_av_scan",+ "params": {+ "host_field": "alert.host",+ "scan_path": "{{inputs.webshell_path}}",+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 120+ },+ {+ "id": "approval_isolate",+ "name": "Approval gate — isolate web server",+ "type": "approval",+ "params": {+ "message": "Isolate web server '{{inputs.host}}' from network? This will take the site offline.",+ "approvers": ["soc-lead", "infra-lead"],+ "timeout_seconds": 300,+ "auto_approve_severity": ["critical"]+ },+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 320+ },+ {+ "id": "isolate_host",+ "name": "Isolate web server (EDR)",+ "type": "isolate_host",+ "params": {+ "host_field": "alert.host",+ "dry_run": "{{dry_run}}"+ },+ "condition": "steps.approval_isolate.outcome == 'approved'",+ "on_failure": "abort",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "forensic_snapshot",+ "name": "Snapshot disk for forensics",+ "type": "http",+ "params": {+ "url": "${EDR_URL}/hosts/{{inputs.host}}/snapshot",+ "method": "POST",+ "dry_run": "{{dry_run}}"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 30+ },+ {+ "id": "inv",+ "name": "Web compromise investigation",+ "type": "investigate",+ "params": {+ "case_id_field": "alert.case_id",+ "focus": "web-compromise"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 180+ },+ {+ "id": "n1",+ "name": "Alert web / infra team",+ "type": "notify",+ "params": {+ "channel": "pagerduty",+ "message_template": "Web compromise on {{inputs.host}}. Attacker IP: {{inputs.attacker_ip}}. Host isolated: {{context.isolate_success}}.",+ "service_key_env": "PD_INFRA_KEY"+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 10+ },+ {+ "id": "t1",+ "name": "Open web compromise ticket",+ "type": "create_ticket",+ "params": {+ "priority": "P1",+ "title_template": "Web Compromise: {{inputs.host}}",+ "queue": "soc",+ "labels": ["web-compromise", "webshell"]+ },+ "on_failure": "continue",+ "retry_max": 0,+ "timeout_seconds": 20+ }+ ]+}diff --git a/services/agents/app/playbook/store.py b/services/agents/app/playbook/store.pyindex 568e97c05..d444513f6 100644--- a/services/agents/app/playbook/store.py+++ b/services/agents/app/playbook/store.py@@ -50,7 +50,25 @@ def _resolve_repo_root() -> Path:_REPO_ROOT = _resolve_repo_root()-_DEFAULT_PACK_ROOT = _REPO_ROOT / "playbooks" / "packs" / "v1"++#: The pack that ships *inside* the package, and the one a container has.+#:+#: `_resolve_repo_root` walks up looking for an ancestor holding+#: `playbooks/packs`. That works in a checkout and finds nothing in the+#: image: `services/agents/Dockerfile` has a build context of+#: `services/agents` and copies only `app/`, and a build context cannot+#: reach outside itself.+#:+#: So the shipped image carried **zero** pack playbooks, silently — the+#: loader's pack branch is guarded by `exists()`, which makes an absent+#: pack indistinguishable from an empty one. A deployment's library was+#: 62 entries shorter than the repository's with no error anywhere.+#:+#: `scripts/sync_vendored_playbook_packs.py --check` keeps this copy+#: byte-identical to `playbooks/packs/v1/`, the same arrangement+#: `services/fusion` uses for its compiled detection ruleset.+_VENDORED_PACK_ROOT = Path(__file__).resolve().parent / "packs" / "v1"+_DEFAULT_PACK_ROOT = _VENDORED_PACK_ROOT if _VENDORED_PACK_ROOT.is_dir() else _REPO_ROOT / "playbooks" / "packs" / "v1"class PlaybookStore:diff --git a/services/agents/tests/test_playbook_pack_ships.py b/services/agents/tests/test_playbook_pack_ships.pynew file mode 100644index 000000000..09938aa3b--- /dev/null+++ b/services/agents/tests/test_playbook_pack_ships.py@@ -0,0 +1,48 @@+"""The playbook pack must be inside the package, not beside the repo.++`playbooks/packs/v1/` holds 62 production playbooks, and the agents image+shipped **zero** of them. `services/agents/Dockerfile` has a build+context of `services/agents` and copies only `app/`; a build context+cannot reach outside itself, so the loader resolved a path that does not+exist in a container.++Nothing said so. The loader's pack branch is guarded by+`if self._pack_root.exists()`, which makes an absent pack indistinguishable+from an empty one — a deployment's playbook library was 62 entries+shorter than the repository's with no error anywhere.++These tests fail if that regresses.+"""++from __future__ import annotations++from pathlib import Path++from app.playbook.store import _DEFAULT_PACK_ROOT, PlaybookStore++#: The count at the time of writing. Asserted as a floor rather than an+#: equality so adding a playbook does not fail the build, while removing+#: the pack — the actual failure mode — still does.+EXPECTED_AT_LEAST = 62+++def test_the_pack_resolves_inside_the_installed_package() -> None:+ """A path outside `app/` is a path no container has."""+ resolved = Path(_DEFAULT_PACK_ROOT).resolve()+ assert "app/playbook/packs" in resolved.as_posix(), (+ f"the pack root is {resolved}, which is outside the package. The agents image "+ "copies only app/, so a pack resolved anywhere else ships as zero playbooks."+ )+++def test_the_pack_is_not_empty() -> None:+ found = list(Path(_DEFAULT_PACK_ROOT).rglob("*.playbook.json"))+ assert len(found) >= EXPECTED_AT_LEAST, f"found {len(found)} pack playbooks, expected at least {EXPECTED_AT_LEAST}"+++def test_the_store_actually_loads_them() -> None:+ """Resolving the path and parsing the files are different claims."""+ store = PlaybookStore.default()+ assert len(store._playbooks) >= EXPECTED_AT_LEAST, (+ f"the store loaded {len(store._playbooks)} playbooks; a pack that resolves but does not parse is still a pack nobody can run"+ )diff --git a/services/api/app/core/config.py b/services/api/app/core/config.pyindex 07c133054..7fa64e5bb 100644--- a/services/api/app/core/config.py+++ b/services/api/app/core/config.py@@ -404,6 +404,18 @@ class Settings(BaseSettings):# and report healthy while doing so, so ``scripts/check_ioc_lake_mapping.py``# compares this value against that service's setting.# ------------------------------------------------------------------+ # Default **off**, deliberately and permanently.+ #+ # A retro-hunt replays every newly-published indicator against a+ # tenant's entire recorded history, so arming it on upgrade would+ # change what a deployment does to its own lake without anyone asking+ # — the same reasoning as the retention purge worker below.+ #+ # Off by default is not the same as untested: `retro-hunt-live.yml`+ # grades the sweep and the consumer on every pull request, with the+ # flag **on**, against a real ClickHouse and a real broker. See+ # `docs/audit/MATURITY_DEFINITION.md` on why a flag's default does not+ # bear on maturity but an ungraded flag does.RETRO_HUNT_ENABLED: bool = Field(default=False,validation_alias=AliasChoices("RETRO_HUNT_ENABLED", "AISOC_RETRO_HUNT_ENABLED"),diff --git a/services/api/app/services/graph_service.py b/services/api/app/services/graph_service.pyindex 711250966..ac01dc7fe 100644--- a/services/api/app/services/graph_service.py+++ b/services/api/app/services/graph_service.py@@ -399,15 +399,34 @@ async def get_blast_radius(entity_id: str, entity_type: str, tenant_id: str, hopRETURN all_nodes"""- async with get_session() as s:- result = await s.run(- cypher,- entity_id=entity_id,- tenant_id=tenant_id,- hops=hops,- global_labels=list(_GLOBAL_LABELS),+ try:+ async with get_session() as s:+ result = await s.run(+ cypher,+ entity_id=entity_id,+ tenant_id=tenant_id,+ hops=hops,+ global_labels=list(_GLOBAL_LABELS),+ )+ record = await result.single()+ except Exception as exc: # noqa: BLE001+ # The fallback existed for exactly this and could not be reached by+ # it. `if not record` only fires when the query *succeeds and+ # returns nothing*; a Neo4j without the APOC plugin, or one where+ # `apoc.*` is not in `dbms.security.procedures.unrestricted`,+ # raises instead — and the route answered 500 rather than+ # degrading to the plain-Cypher expansion sitting right below.+ #+ # The shipped compose installs APOC, so this never fired here. It+ # fires for anyone pointing AiSOC at a managed or hardened Neo4j,+ # which is the deployment least able to debug it.+ if not _is_missing_procedure(exc):+ raise+ logger.warning(+ "blast_radius.apoc_unavailable falling back to plain Cypher: %s",+ str(exc)[:200].replace("\r", "").replace("\n", " "),)- record = await result.single()+ return await _blast_radius_fallback(entity_id, entity_type, tenant_id, hops)if not record:return await _blast_radius_fallback(entity_id, entity_type, tenant_id, hops)@@ -429,6 +448,28 @@ async def get_blast_radius(entity_id: str, entity_type: str, tenant_id: str, hop}+def _is_missing_procedure(exc: Exception) -> bool:+ """Whether Neo4j refused because a procedure is not installed.++ Matched on the message rather than the exception type: the driver+ raises `ClientError` for a large family of conditions, and catching+ the type would also swallow a genuine Cypher error in the query above+ — turning a bug into a silently degraded answer, which is worse than+ the 500 this replaces.+ """+ # The driver's `code` attribute first. Neo4j publishes+ # `Neo.ClientError.Procedure.ProcedureNotFound` as a stable+ # identifier, and matching it is exact where matching prose is not:+ # the first version of this looked for "was not registered" and the+ # server actually says "registered for this database instance", so it+ # missed the only condition it was written for.+ code = getattr(exc, "code", "") or ""+ if "ProcedureNotFound" in code:+ return True+ text = str(exc).lower()+ return "procedurenotfound" in text or "no procedure with the name" in text or "not registered for this database" in text++async def _blast_radius_fallback(entity_id: str, entity_type: str, tenant_id: str, hops: int) -> dict[str, Any]:"""Simple 2-hop blast radius without APOC."""label_map = {"host": "Host", "user": "User", "ioc": "IOC", "alert": "Alert"}diff --git a/tests/isolation/test_connector_scheduler_live.py b/tests/isolation/test_connector_scheduler_live.pynew file mode 100644index 000000000..95beba02c--- /dev/null+++ b/tests/isolation/test_connector_scheduler_live.py@@ -0,0 +1,305 @@+"""The connector scheduler, against real Postgres and a real HTTP vendor.++Maturity: the evidence that takes **Scheduled connectors** to Stable.+See `docs/audit/MATURITY_DEFINITION.md` for what the label requires.++Why a live scheduler is the only way to test this+---------------------------------------------------+The defect this suite exists for is the reason "connect a source" never+pulled data on any deployment: `reload_jobs` added every poll job with+`next_run_time=None`, which **APScheduler 3.x registers as PAUSED**. The+scheduler started, the job list looked right, `last_sync` stayed null for+every instance, and nothing logged an error — because nothing had gone+wrong, exactly. The jobs were simply never going to run.++No mock catches that. A test that asserts `add_job` was called with the+right arguments passes whether or not APScheduler will ever fire it;+only a real `AsyncIOScheduler` knows what `next_run_time=None` means.++So this suite starts the real scheduler, with real connector rows in real+Postgres and a real HTTP server standing in for the vendor, and waits for+an event to arrive. The vendor is a stub because we are not testing+Okta's API — but it is a *socket*, not a patched method, so the whole+client path runs.++The negative control+--------------------+The workflow re-injects `next_run_time=None` and requires this suite to+fail. Without that, "an event arrived" could be true because something+else polled.+"""++from __future__ import annotations++import asyncio+import contextlib+import json+import os+import threading+import uuid+from http.server import BaseHTTPRequestHandler, HTTPServer++import pytest+import pytest_asyncio++# Skip as a *module*, not only in the fixture.+#+# The offline isolation job collects this directory with no stores+# running. With the skip only in the fixture, any test that does not take+# it ran anyway and failed there — which is a failure about the harness,+# reported against a capability.+pytestmark = [+ pytest.mark.asyncio(loop_scope="module"),+ pytest.mark.skipif(+ not os.environ.get("ISOLATION_CONNECTOR_DSN", "").strip(),+ reason="ISOLATION_CONNECTOR_DSN is not set; this suite needs live infrastructure",+ ),+]++#: What the stub vendor returns, and what the stub ingest receives.+VENDOR_EVENTS = [+ {+ "uuid": "evt-live-0001",+ "published": "2026-10-02T12:00:00.000Z",+ "eventType": "user.session.start",+ "severity": "WARN",+ "displayMessage": "Sign-in from an unusual location",+ "actor": {"alternateId": "[email protected]", "displayName": "J Doe"},+ "client": {"ipAddress": "198.51.100.24"},+ "outcome": {"result": "SUCCESS"},+ }+]+++def _dsn() -> str:+ value = os.environ.get("ISOLATION_CONNECTOR_DSN", "").strip()+ if not value:+ pytest.skip("ISOLATION_CONNECTOR_DSN is not set; this suite needs a live Postgres")+ return value+++class _Vendor(BaseHTTPRequestHandler):+ """A real HTTP server standing in for the vendor API.++ A socket rather than a patched method, so the connector's own client+ — its headers, its auth, its JSON handling, its pagination — is on+ the path. A patched `fetch_alerts` would skip all of it.+ """++ def do_GET(self) -> None: # noqa: N802+ # The Okta connector appends `/api/v1/logs`; anything else this+ # stub is asked for still answers, because the point is to be a+ # socket the real client can talk to, not to model Okta.+ self.send_response(200)+ self.send_header("Content-Type", "application/json")+ body = json.dumps(VENDOR_EVENTS).encode()+ self.send_header("Content-Length", str(len(body)))+ self.end_headers()+ self.wfile.write(body)++ def log_message(self, *_args) -> None: # noqa: ANN002+ """Quiet: the default handler prints to stderr on every request."""+++class _Ingest(BaseHTTPRequestHandler):+ """Stands in for `services/ingest`, and records what reached it."""++ received: list[dict] = []++ def do_POST(self) -> None: # noqa: N802+ length = int(self.headers.get("Content-Length", 0))+ raw = self.rfile.read(length) if length else b"{}"+ with contextlib.suppress(Exception):+ _Ingest.received.append(+ {+ "tenant": self.headers.get("X-Tenant-ID"),+ "body": json.loads(raw.decode()),+ }+ )+ self.send_response(202)+ self.send_header("Content-Length", "2")+ self.end_headers()+ self.wfile.write(b"{}")++ def log_message(self, *_args) -> None: # noqa: ANN002+ """Quiet."""+++def _serve(handler) -> tuple[HTTPServer, str]: # noqa: ANN001+ server = HTTPServer(("127.0.0.1", 0), handler)+ threading.Thread(target=server.serve_forever, daemon=True).start()+ return server, f"http://127.0.0.1:{server.server_port}"++[email protected](scope="module")+def vendor():+ server, url = _serve(_Vendor)+ yield url+ server.shutdown()++[email protected](scope="module")+def ingest():+ _Ingest.received.clear()+ server, url = _serve(_Ingest)+ yield url+ server.shutdown()+++@pytest_asyncio.fixture(scope="module", loop_scope="module")+async def engine():+ pytest.importorskip("asyncpg")+ from sqlalchemy.ext.asyncio import create_async_engine++ created = create_async_engine(_dsn(), future=True)+ yield created+ await created.dispose()+++@pytest_asyncio.fixture(scope="module", loop_scope="module")+async def connector_row(engine, vendor): # noqa: ANN001+ """One enabled connector instance, written the way the console writes it.++ Module-scoped on purpose. The stub ingest accumulates everything it+ receives, so a connector per test meant a later test comparing the+ accumulated events against a tenant that had only just been created+ — a failure about the fixture, not about the scheduler. One+ connector polling throughout is also closer to how a deployment runs.+ """+ import sqlalchemy+ from app.security.credential_vault import get_vault++ tenant_id = uuid.uuid4()+ connector_id = uuid.uuid4()+ # `domain` and `api_token` are the two arguments OktaConnector takes.+ # Passing anything else makes the scheduler log `bad_config` and skip+ # the poll — which is correct of it, and is why the field names here+ # come from the connector rather than from a convention.+ auth = get_vault().encrypt_dict({"api_token": "stub-token", "domain": vendor})++ async with engine.begin() as conn:+ await conn.execute(+ sqlalchemy.text("INSERT INTO tenants (id, name, slug) VALUES (:i, :n, :s) ON CONFLICT DO NOTHING"),+ {"i": tenant_id, "n": f"conn-{tenant_id.hex[:8]}", "s": f"conn-{tenant_id.hex[:8]}"},+ )+ await conn.execute(+ sqlalchemy.text(+ # `is_enabled`, which is what the table actually calls it.+ # `fetch_enabled_connectors` filters on this column, so a+ # row inserted under any other name is invisible to the+ # scheduler and the suite would fail for a reason that has+ # nothing to do with the scheduler.+ "INSERT INTO connectors "+ "(id, tenant_id, connector_type, name, is_enabled, auth_config, connector_config) "+ "VALUES (:i, :t, 'okta', 'live-sched', true, "+ "CAST(:a AS jsonb), CAST(:c AS jsonb))"+ ),+ {+ "i": connector_id,+ "t": tenant_id,+ "a": json.dumps(auth),+ # Two seconds, so the test does not sit for the five-minute+ # default. The interval is the thing under test only+ # insofar as the job must actually fire.+ "c": json.dumps({"poll_interval_seconds": 2, "domain": vendor}),+ },+ )++ yield {"tenant_id": str(tenant_id), "connector_id": str(connector_id)}++ async with engine.begin() as conn:+ await conn.execute(sqlalchemy.text("DELETE FROM connectors WHERE id = :i"), {"i": connector_id})+++@pytest_asyncio.fixture(scope="module", loop_scope="module")+async def scheduler(engine, ingest, connector_row): # noqa: ANN001+ """The real `ConnectorScheduler`, started."""+ from app.ingest_client import IngestClient+ from app.scheduler import ConnectorScheduler+ from app.security.credential_vault import get_vault++ instance = ConnectorScheduler(+ engine=engine,+ ingest_client=IngestClient(base_url=ingest),+ vault=get_vault(),+ reload_interval_seconds=1.0,+ )+ await instance.start()+ yield instance+ await instance.stop()+++class TestTheJobIsScheduledToRun:+ async def test_a_job_exists_for_the_enabled_connector(self, scheduler, connector_row) -> None: # noqa: ANN001+ await asyncio.sleep(2)+ jobs = scheduler.job_diagnostics()+ assert jobs, "the scheduler registered no jobs for an enabled connector"++ async def test_the_job_is_not_paused(self, scheduler, connector_row) -> None: # noqa: ANN001+ """The defect, stated as a test.++ `next_run_time=None` registers a job as PAUSED. The job list+ looks correct, `last_sync` stays null forever, and nothing logs+ an error — nothing has gone wrong, the job is simply never going+ to run. A mock asserting `add_job` was called cannot tell the+ difference; a real AsyncIOScheduler can.+ """+ await asyncio.sleep(2)+ jobs = scheduler.job_diagnostics()+ paused = [j for j in jobs if j.get("next_run_time") is None]+ assert not paused, (+ f"{len(paused)} of {len(jobs)} jobs have no next_run_time, which APScheduler "+ "treats as PAUSED. This is the defect that meant connecting a source never "+ "pulled data on any deployment."+ )+++class TestItActuallyPolls:+ async def test_an_event_reaches_ingest(self, scheduler, connector_row) -> None: # noqa: ANN001+ """The end of the claim: a connector you enable produces events.++ Waits on the real clock because the thing under test is whether+ a timer fires.+ """+ for _ in range(30):+ if _Ingest.received:+ break+ await asyncio.sleep(1)++ assert _Ingest.received, (+ "no event reached ingest within 30s of enabling a connector with a 2s poll interval. The scheduler is not polling."+ )++ async def test_the_event_carries_its_tenant(self, scheduler, connector_row) -> None: # noqa: ANN001+ """Ingest routes on `X-Tenant-ID`. Without it the event lands+ nowhere, or worse, somewhere else."""+ for _ in range(30):+ if _Ingest.received:+ break+ await asyncio.sleep(1)+ assert _Ingest.received++ tenants = {entry["tenant"] for entry in _Ingest.received}+ assert connector_row["tenant_id"] in tenants, (+ f"events arrived under {tenants!r}, not the connector's tenant {connector_row['tenant_id']!r}"+ )+++class TestItRecordsThatItPolled:+ async def test_last_sync_is_written(self, engine, connector_row) -> None: # noqa: ANN001+ """`last_sync` staying null across every instance was the symptom+ that surfaced the paused-job defect. It is a product signal, not+ just a test convenience: the console reads it."""+ import sqlalchemy++ for _ in range(30):+ async with engine.connect() as conn:+ result = await conn.execute(+ sqlalchemy.text("SELECT last_sync FROM connectors WHERE id = CAST(:i AS uuid)"),+ {"i": connector_row["connector_id"]},+ )+ if (row := result.first()) and row.last_sync is not None:+ return+ await asyncio.sleep(1)++ pytest.fail("last_sync stayed null, so nothing recorded a successful poll")diff --git a/tests/isolation/test_graph_service_live.py b/tests/isolation/test_graph_service_live.pynew file mode 100644index 000000000..9536f827e--- /dev/null+++ b/tests/isolation/test_graph_service_live.py@@ -0,0 +1,228 @@+"""The production graph reader, against a live Neo4j.++Maturity: the evidence that takes **Entity graph (Neo4j)** to Stable.+See `docs/audit/MATURITY_DEFINITION.md` for what the label requires.++Why this exists when `test_live_stores.py` already touches Neo4j+----------------------------------------------------------------+That test writes its own Cypher and asserts a tenant-scoped `MATCH`+behaves. It proves the *pattern* is sound, and it has a real negative+control. What it does not do is call anything a deployment runs: if+`graph_service.get_entity_neighbors` stopped binding `tenant_id`+tomorrow, that test would still pass.++That is not hypothetical. The docstring on `get_entity_neighbors` records+exactly that defect — the function accepted `tenant_id`, never bound it,+and any authenticated user could read any other tenant's host, user or+IOC node with all of its properties by naming the id. The unit tests for+it assert on Cypher *text* through a `_FakeSession`, so they would have+passed too; a text assertion cannot notice that a parameter is missing+from the call.++So every query below comes from `services/api/app/services/graph_service.py`+and runs through `app.db.neo4j.get_session`, which is the driver a+deployment uses. Nothing here re-implements a query.++The negative control+--------------------+`test_an_unscoped_read_sees_both_tenants` seeds two tenants and proves an+unscoped read returns both. Without it, every scoped assertion below+could pass against an empty graph and prove nothing — the vacuous-pass+shape this repository has caught in gates five separate times.+"""++from __future__ import annotations++import os+import uuid++import pytest+import pytest_asyncio++# Skip as a *module*, not only in the fixture.+#+# The offline isolation job collects this directory with no stores+# running. With the skip only in the fixture, any test that does not take+# it ran anyway and failed there — which is a failure about the harness,+# reported against a capability.+pytestmark = [+ pytest.mark.asyncio,+ pytest.mark.skipif(+ not os.environ.get("ISOLATION_NEO4J_URI", "").strip(),+ reason="ISOLATION_NEO4J_URI is not set; this suite needs live infrastructure",+ ),+]++TENANT_A = "11111111-1111-1111-1111-111111111111"+TENANT_B = "22222222-2222-2222-2222-222222222222"++# Set at import, before anything under `app.` is loaded.+#+# `app.core.config.settings` is a module-level singleton built when it is+# first imported, so a fixture that sets NEO4J_URI later would configure+# nothing and the driver would quietly dial localhost:7687. Doing it here+# means the production driver reads the live container the same way a+# deployment reads its own.+_URI = os.environ.get("ISOLATION_NEO4J_URI", "").strip()+if _URI:+ os.environ["NEO4J_URI"] = _URI+ os.environ["NEO4J_USER"] = os.environ.get("ISOLATION_NEO4J_USER", "neo4j")+ os.environ["NEO4J_PASSWORD"] = os.environ.get("ISOLATION_NEO4J_PASSWORD", "neo4j")+++def _require(name: str) -> str:+ value = os.environ.get(name, "").strip()+ if not value:+ pytest.skip(f"{name} is not set; this suite needs a live Neo4j")+ return value++[email protected](scope="module")+def neo4j_env():+ """Confirm a live Neo4j was named, and that its driver is installed."""+ uri = _require("ISOLATION_NEO4J_URI")+ pytest.importorskip("neo4j")+ return uri+++@pytest_asyncio.fixture+async def seeded(neo4j_env): # noqa: ANN001+ """Two tenants' worth of nodes, written by the production upserts.++ `upsert_host` and `upsert_alert_node` rather than raw Cypher, so the+ write path is the one a deployment uses and a change to either is+ graded here.+ """+ from app.db.neo4j import close_neo4j, get_session, init_neo4j+ from app.services import graph_service++ await init_neo4j()+ marker = uuid.uuid4().hex[:8]+ host_a = f"host-a-{marker}"+ host_b = f"host-b-{marker}"++ await graph_service.upsert_host(host_id=host_a, hostname=f"WIN-A-{marker}", tenant_id=TENANT_A)+ await graph_service.upsert_host(host_id=host_b, hostname=f"WIN-B-{marker}", tenant_id=TENANT_B)+ await graph_service.upsert_alert_node(+ alert_id=f"alert-a-{marker}",+ title="Encoded PowerShell",+ severity="high",+ tenant_id=TENANT_A,+ host_id=host_a,+ )+ await graph_service.upsert_alert_node(+ alert_id=f"alert-b-{marker}",+ title="Encoded PowerShell",+ severity="high",+ tenant_id=TENANT_B,+ host_id=host_b,+ )++ # A global MITRE node attached to tenant B's host.+ #+ # This is what makes the *anchor* scope independently testable, and+ # the first version of this suite was missing it. `_scoped` admits+ # `Technique`, `Tactic` and `Mitigation` as shared reference data, so+ # without a global neighbour the neighbour predicate masks an+ # unscoped anchor: reading B's host as A matched nothing either way,+ # and removing `WHERE {_scoped("n")}` entirely still left all six+ # tests green. A negative control that cannot distinguish the two+ # scopes is only testing one of them.+ async with get_session() as session:+ await session.run(+ "MERGE (t:Technique {technique_id: $tid}) WITH t MATCH (h:Host {id: $host}) MERGE (h)-[:EXHIBITS]->(t)",+ tid=f"T1059-{marker}",+ host=host_b,+ )++ yield {"marker": marker, "host_a": host_a, "host_b": host_b}++ async with get_session() as session:+ await session.run(+ "MATCH (n) WHERE n.id ENDS WITH $m OR n.hostname ENDS WITH $m OR n.technique_id ENDS WITH $m DETACH DELETE n",+ m=marker,+ )+ await close_neo4j()+++class TestTheNegativeControl:+ async def test_an_unscoped_read_sees_both_tenants(self, seeded) -> None: # noqa: ANN001+ """Without this, every scoped assertion could pass on an empty graph.++ Run first and deliberately: it is the only thing standing between+ "tenant B's node is correctly excluded" and "nothing was written+ and the read returned nothing".+ """+ from app.db.neo4j import get_session++ async with get_session() as session:+ result = await session.run(+ "MATCH (h:Host) WHERE h.id IN [$a, $b] RETURN count(h) AS n",+ a=seeded["host_a"],+ b=seeded["host_b"],+ )+ record = await result.single()++ assert record["n"] == 2, (+ f"expected both tenants' hosts present before scoping is tested, found "+ f"{record['n']}. Every assertion below is vacuous without this."+ )+++class TestTheProductionReaderScopes:+ async def test_neighbors_excludes_another_tenant(self, seeded) -> None: # noqa: ANN001+ """`get_entity_neighbors` is the function whose docstring records+ accepting `tenant_id` and never binding it.++ Tenant B's host carries a global `Technique` neighbour, which+ `_scoped` admits for every tenant. So this exercises the **anchor**+ predicate on its own: without it the pattern matches through the+ global node and B's hostname and properties come back to A.+ """+ from app.services import graph_service++ result = await graph_service.get_entity_neighbors(entity_id=seeded["host_b"], entity_type="host", tenant_id=TENANT_A)+ source = (result or {}).get("source") or {}+ assert not source.get("id"), (+ f"tenant A read tenant B's host through the production reader: {source!r}. "+ "This is the exact defect get_entity_neighbors' own docstring records."+ )++ async def test_neighbors_returns_its_own_tenants_node(self, seeded) -> None: # noqa: ANN001+ """The other half. A reader that returns nothing for everyone is+ 'scoped' in the uninteresting sense."""+ from app.services import graph_service++ result = await graph_service.get_entity_neighbors(entity_id=seeded["host_a"], entity_type="host", tenant_id=TENANT_A)+ source = (result or {}).get("source") or {}+ assert source.get("id") == seeded["host_a"], f"tenant A could not read its own host through the production reader: {result!r}"++ async def test_blast_radius_does_not_traverse_into_another_tenant(self, seeded) -> None: # noqa: ANN001+ """`get_blast_radius` filtered only the traversal's start node, so+ an expansion could walk through a shared entity into another+ tenant's estate. Every node of every path is scoped now."""+ from app.services import graph_service++ result = await graph_service.get_blast_radius(entity_id=seeded["host_a"], entity_type="host", tenant_id=TENANT_A, hops=3)+ blob = repr(result)+ assert seeded["host_b"] not in blob, "a blast-radius expansion from tenant A reached tenant B's host"++ async def test_another_tenants_blast_radius_is_empty(self, seeded) -> None: # noqa: ANN001+ from app.services import graph_service++ result = await graph_service.get_blast_radius(entity_id=seeded["host_b"], entity_type="host", tenant_id=TENANT_A, hops=3)+ nodes = (result or {}).get("nodes") or []+ assert not nodes, f"tenant A got a blast radius for tenant B's host: {nodes!r}"+++class TestTheWritePathIsReal:+ async def test_an_upsert_is_visible_to_the_reader(self, seeded) -> None: # noqa: ANN001+ """Proves the fixture wrote through production code rather than+ leaving the reader to find nothing and call that isolation."""+ from app.services import graph_service++ result = await graph_service.get_entity_neighbors(entity_id=seeded["host_a"], entity_type="host", tenant_id=TENANT_A)+ neighbors = (result or {}).get("neighbors") or []+ assert neighbors, (+ "the alert upserted against this host is not reachable from it, so either the write path or the relationship is broken"+ )diff --git a/tests/isolation/test_hunt_lake_live.py b/tests/isolation/test_hunt_lake_live.pynew file mode 100644index 000000000..d8e34e0e7--- /dev/null+++ b/tests/isolation/test_hunt_lake_live.py@@ -0,0 +1,206 @@+"""Scheduled hunts read the tenant's own events, not a fixture.++Maturity: parity 6.1, and part of what takes the **68-hunt YAML+library** off Alpha.++Lives in its own file rather than beside the lake suite because both+services call their package `app`: running from `services/api` resolves+`app.hunt` to nothing. Two services, one module name, so one suite each.++What was wrong+--------------+The only implemented telemetry provider was `synthetic`, so every+scheduled hunt on every deployment ran against+`services/agents/tests/eval_data/synthetic_telemetry.jsonl`. The hunts+produced findings about events no customer had, and found nothing about+the events they did. `ingest` existed as a name and returned an empty+list.++The rule the replacement is built on+--------------------------------------+An unreachable lake returns nothing and says so. There is no fallback to+the fixture. A scheduled hunt that quietly reports findings from+synthetic telemetry is worse than one that reports nothing, because an+operator cannot tell the two apart and one of them is fabricated data+presented as their own estate.+"""++from __future__ import annotations++import os+import uuid+from datetime import UTC, datetime+from pathlib import Path++import pytest++# Skip as a *module*, not only in the fixture.+#+# The offline isolation job collects this directory with no stores+# running. With the skip only in the fixture, any test that does not take+# it ran anyway and failed there — which is a failure about the harness,+# reported against a capability.+pytestmark = pytest.mark.skipif(+ not os.environ.get("ISOLATION_CLICKHOUSE_HOST", "").strip(),+ reason="ISOLATION_CLICKHOUSE_HOST is not set; this suite needs live infrastructure",+)++TENANT_A = uuid.UUID("11111111-1111-1111-1111-111111111111")+TENANT_B = uuid.UUID("22222222-2222-2222-2222-222222222222")++# `app.hunt.lake_source` reads CLICKHOUSE_* the way the fusion writer and+# the API do. This suite is configured with ISOLATION_CLICKHOUSE_*, so+# bridge them at import rather than teaching the production module a+# second spelling it would only use in tests.+if os.environ.get("ISOLATION_CLICKHOUSE_HOST"):+ os.environ.setdefault("CLICKHOUSE_HOST", os.environ["ISOLATION_CLICKHOUSE_HOST"])+ os.environ.setdefault("CLICKHOUSE_PORT", os.environ.get("ISOLATION_CLICKHOUSE_PORT", "9000"))+ os.environ.setdefault("CLICKHOUSE_USER", os.environ.get("ISOLATION_CLICKHOUSE_USER", "default"))+ os.environ.setdefault("CLICKHOUSE_PASSWORD", os.environ.get("ISOLATION_CLICKHOUSE_PASSWORD", ""))+ os.environ.setdefault("CLICKHOUSE_DATABASE", "aisoc")+++def _client(): # noqa: ANN202+ host = os.environ.get("ISOLATION_CLICKHOUSE_HOST", "").strip()+ if not host:+ pytest.skip("ISOLATION_CLICKHOUSE_HOST is not set; this suite needs a live ClickHouse")+ driver = pytest.importorskip("clickhouse_driver")+ return driver.Client(+ host=host,+ port=int(os.environ.get("ISOLATION_CLICKHOUSE_PORT", "9000")),+ user=os.environ.get("ISOLATION_CLICKHOUSE_USER", "default"),+ password=os.environ.get("ISOLATION_CLICKHOUSE_PASSWORD", ""),+ )++[email protected](scope="module")+def lake():+ """The product's own schema, seeded for two tenants.++ `001_init.sql` verbatim, for the reason the lake suite records: a+ test that writes its own CREATE TABLE cannot notice the real one has+ drifted.+ """+ client = _client()+ root = Path(os.environ.get("ISOLATION_REPO_ROOT", "."))+ ddl = (root / "services" / "api" / "clickhouse" / "001_init.sql").read_text(encoding="utf-8")+ lines = [ln for ln in ddl.splitlines() if not ln.strip().startswith("--")]+ for statement in (s.strip() for s in "\n".join(lines).split(";")):+ if statement:+ client.execute(statement)++ client.execute("TRUNCATE TABLE IF EXISTS aisoc.raw_events")+ now = datetime.now(UTC).replace(tzinfo=None)+ client.execute(+ "INSERT INTO aisoc.raw_events "+ "(tenant_id, event_time, class_uid, category_uid, severity_id, severity, "+ "src_hostname, user_name, process_name, raw_payload) VALUES",+ [+ (+ TENANT_A,+ now,+ 2001,+ 2,+ 4,+ "high",+ "tenant-a-host-1",+ "j.doe",+ "powershell.exe",+ '{"cmdline": "-enc AAA"}',+ ),+ (TENANT_A, now, 2001, 2, 4, "high", "tenant-a-host-2", "j.doe", "cmd.exe", "{}"),+ (TENANT_B, now, 2001, 2, 4, "high", "tenant-b-host-1", "other", "cmd.exe", "{}"),+ ],+ )+ yield client+ client.execute("TRUNCATE TABLE IF EXISTS aisoc.raw_events")+++class TestItReadsTenantData:+ def test_a_tenants_hunt_reads_its_own_events(self, lake) -> None: # noqa: ANN001+ from app.hunt.lake_source import fetch_recent_events++ hosts = {e.get("host") for e in fetch_recent_events(str(TENANT_A))}+ assert hosts == {"tenant-a-host-1", "tenant-a-host-2"}, hosts++ def test_it_does_not_read_another_tenants(self, lake) -> None: # noqa: ANN001+ from app.hunt.lake_source import fetch_recent_events++ hosts = {e.get("host") for e in fetch_recent_events(str(TENANT_A))}+ assert "tenant-b-host-1" not in hosts++ def test_the_other_tenant_has_data_so_the_above_is_a_real_negative(self, lake) -> None: # noqa: ANN001+ """Without this, tenant A's exclusion could mean B has nothing."""+ from app.hunt.lake_source import fetch_recent_events++ assert fetch_recent_events(str(TENANT_B)), "tenant B has no rows, so excluding them from A's read proves nothing"+++class TestTheMatchNamespace:+ def test_a_nested_vendor_payload_is_flattened(self, lake) -> None: # noqa: ANN001+ """Connectors nest their payload under the envelope, and a+ matcher doing a flat lookup reads `None` — which is how 663 of+ 825 loaded rules once matched on fields that were never+ visible."""+ from app.hunt.lake_source import fetch_recent_events++ cmdlines = {e.get("cmdline") for e in fetch_recent_events(str(TENANT_A)) if e.get("cmdline")}+ assert "-enc AAA" in cmdlines, f"the nested vendor payload did not reach the match namespace: {cmdlines!r}"++ def test_a_row_with_no_ip_does_not_crash_the_read(self, lake) -> None: # noqa: ANN001+ """`source_ip` and `dest_ip` are IPv6 columns, and the driver+ raises `AddressValueError` decoding the zero-length packed+ address an unset column yields. A row with no IP is the common+ case for identity and SaaS events, so the natural read crashed on+ ordinary data — the query casts with `IPv6NumToString`."""+ from app.hunt.lake_source import fetch_recent_events++ events = fetch_recent_events(str(TENANT_A))+ assert events, "the read returned nothing for rows that have no IP set"+ assert all(e.get("src_ip") == "" for e in events)+++class TestItRefusesToFabricate:+ def test_an_unreachable_lake_raises_rather_than_returning_the_fixture(self) -> None:+ """The honesty rule this module is built on.++ The scheduler catches this, logs it, and records an empty run —+ so "the lake is down" and "this tenant had no events" stay+ distinguishable instead of both looking like a quiet success.+ """+ from app.hunt.lake_source import fetch_recent_events++ previous = os.environ.get("CLICKHOUSE_PORT")+ os.environ["CLICKHOUSE_PORT"] = "1"+ try:+ with pytest.raises(Exception): # noqa: B017, PT011+ fetch_recent_events(str(TENANT_A))+ finally:+ if previous is None:+ os.environ.pop("CLICKHOUSE_PORT", None)+ else:+ os.environ["CLICKHOUSE_PORT"] = previous++ def test_the_scheduler_turns_that_into_an_empty_run_not_the_fixture(self) -> None:+ """The other half, at the call site.++ `_load_telemetry` must not quietly substitute the synthetic+ corpus when the lake is unreachable.+ """+ from app.hunt.scheduler import _load_telemetry++ previous = os.environ.get("CLICKHOUSE_PORT")+ os.environ["CLICKHOUSE_PORT"] = "1"+ try:+ events = _load_telemetry(str(TENANT_A))+ finally:+ if previous is None:+ os.environ.pop("CLICKHOUSE_PORT", None)+ else:+ os.environ["CLICKHOUSE_PORT"] = previous++ assert events == [], (+ f"an unreachable lake produced {len(events)} events. If those came from the "+ "synthetic corpus, the scheduler is reporting fixture findings as the "+ "tenant's own."+ )diff --git a/tests/isolation/test_lake_live.py b/tests/isolation/test_lake_live.pynew file mode 100644index 000000000..9c2171070--- /dev/null+++ b/tests/isolation/test_lake_live.py@@ -0,0 +1,198 @@+"""The event lake, against the schema the product actually ships.++Maturity: the evidence that takes **Event lake + hunting (ClickHouse)**+to Stable.+See `docs/audit/MATURITY_DEFINITION.md` for what the label requires.++Why loading the real DDL is the point+---------------------------------------+`test_live_stores.py` has a live ClickHouse case, and it writes its own+`CREATE TABLE aisoc.raw_events` — a hand-maintained subset of the columns+`services/api/clickhouse/001_init.sql` creates. A test that builds its+own schema cannot notice that the real one has drifted, which is the+same shape as asserting on a producer against a copy of itself.++This suite executes `001_init.sql` verbatim. A column added to the+product and not to a query fails here; a column a query needs and the+DDL stopped creating fails here too.++What is being proven+----------------------+Tenant isolation on the lake is enforced by `rewrite_for_tenant`, which+injects a `tenant_id` predicate into operator-supplied SQL. Two things+make that fragile and both are covered:++* it **fails closed** — it raises rather than returning unfiltered SQL+ if its predicate does not survive rendering, because the alternative+ is every tenant reading every other tenant's events;+* `sqlglot` is pinned **exactly**, since the parser enforcing the+ isolation could otherwise differ between two builds of one commit.++The negative control+--------------------+`test_an_unscoped_query_sees_both_tenants` proves the rows exist and are+reachable without the rewrite. Without it, every isolation assertion+could pass against an empty table.+"""++from __future__ import annotations++import os+import uuid+from datetime import UTC, datetime+from pathlib import Path++import pytest++# Skip as a *module*, not only in the fixture.+#+# The offline isolation job collects this directory with no stores+# running. With the skip only in the fixture, any test that does not take+# it ran anyway and failed there — which is a failure about the harness,+# reported against a capability.+pytestmark = pytest.mark.skipif(+ not os.environ.get("ISOLATION_CLICKHOUSE_HOST", "").strip(),+ reason="ISOLATION_CLICKHOUSE_HOST is not set; this suite needs live infrastructure",+)++TENANT_A = uuid.UUID("11111111-1111-1111-1111-111111111111")+TENANT_B = uuid.UUID("22222222-2222-2222-2222-222222222222")++DDL = "services/api/clickhouse/001_init.sql"++# `app.hunt.lake_source` reads CLICKHOUSE_* the way the fusion writer and+# the API do. This suite is configured with ISOLATION_CLICKHOUSE_*, so+# bridge them here rather than teaching the production module a second+# spelling it would only use in tests.+if os.environ.get("ISOLATION_CLICKHOUSE_HOST"):+ os.environ.setdefault("CLICKHOUSE_HOST", os.environ["ISOLATION_CLICKHOUSE_HOST"])+ os.environ.setdefault("CLICKHOUSE_PORT", os.environ.get("ISOLATION_CLICKHOUSE_PORT", "9000"))+ os.environ.setdefault("CLICKHOUSE_USER", os.environ.get("ISOLATION_CLICKHOUSE_USER", "default"))+ os.environ.setdefault("CLICKHOUSE_PASSWORD", os.environ.get("ISOLATION_CLICKHOUSE_PASSWORD", ""))+ os.environ.setdefault("CLICKHOUSE_DATABASE", "aisoc")+++def _client(): # noqa: ANN202+ host = os.environ.get("ISOLATION_CLICKHOUSE_HOST", "").strip()+ if not host:+ pytest.skip("ISOLATION_CLICKHOUSE_HOST is not set; this suite needs a live ClickHouse")+ driver = pytest.importorskip("clickhouse_driver")+ return driver.Client(+ host=host,+ port=int(os.environ.get("ISOLATION_CLICKHOUSE_PORT", "9000")),+ user=os.environ.get("ISOLATION_CLICKHOUSE_USER", "default"),+ password=os.environ.get("ISOLATION_CLICKHOUSE_PASSWORD", ""),+ )+++def _statements(sql: str) -> list[str]:+ """Split the DDL into executable statements.++ `clickhouse_driver` sends one statement per round trip, and the file+ is several. Comment lines are dropped first so a `--` containing a+ semicolon cannot split a statement in the wrong place.+ """+ lines = [ln for ln in sql.splitlines() if not ln.strip().startswith("--")]+ return [s.strip() for s in "\n".join(lines).split(";") if s.strip()]++[email protected](scope="module")+def lake():+ """A ClickHouse carrying the product's own schema."""+ from app.core.config import settings # noqa: F401 — import check only++ client = _client()+ root = Path(os.environ.get("ISOLATION_REPO_ROOT", "."))+ ddl = (root / DDL).read_text(encoding="utf-8")+ for statement in _statements(ddl):+ client.execute(statement)++ client.execute("TRUNCATE TABLE IF EXISTS aisoc.raw_events")+ now = datetime.now(UTC).replace(tzinfo=None)+ client.execute(+ # Column names read off the shipped DDL rather than assumed.+ # The first attempt inserted `message` and `raw_event`; the real+ # table has neither (`raw_payload` and `ocsf_json` are the+ # payload columns), which is precisely the drift a suite writing+ # its own CREATE TABLE could never surface.+ "INSERT INTO aisoc.raw_events "+ "(tenant_id, event_time, class_uid, category_uid, severity_id, severity, "+ "src_hostname, raw_payload) VALUES",+ [+ (TENANT_A, now, 2001, 2, 4, "high", "tenant-a-host-1", "{}"),+ (TENANT_A, now, 2001, 2, 4, "high", "tenant-a-host-2", "{}"),+ (TENANT_B, now, 2001, 2, 4, "high", "tenant-b-host-1", "{}"),+ ],+ )+ yield client+ client.execute("TRUNCATE TABLE IF EXISTS aisoc.raw_events")+++class TestTheSchemaIsTheProducts:+ def test_the_shipped_ddl_executes(self, lake) -> None: # noqa: ANN001+ """The claim the hand-written CREATE TABLE cannot make.++ If `001_init.sql` stops being valid ClickHouse, a deployment's+ lake never initialises — and a test that writes its own schema+ would not notice.+ """+ rows = lake.execute("SELECT count() FROM aisoc.raw_events")+ assert rows[0][0] == 3++ def test_every_column_a_query_needs_exists(self, lake) -> None: # noqa: ANN001+ """Catches drift in the direction that actually happens: a query+ gaining a column the DDL does not create."""+ described = {row[0] for row in lake.execute("DESCRIBE TABLE aisoc.raw_events")}+ required = {"tenant_id", "event_time", "class_uid", "severity", "raw_payload", "ocsf_json"}+ missing = required - described+ assert not missing, f"the shipped DDL does not create {sorted(missing)}"+++class TestTheNegativeControl:+ def test_an_unscoped_query_sees_both_tenants(self, lake) -> None: # noqa: ANN001+ """Without this, every isolation assertion below could pass+ against an empty table."""+ total = lake.execute("SELECT count() FROM aisoc.raw_events")[0][0]+ assert total == 3, f"expected both tenants' rows seeded, found {total}"+++class TestTheProductionRewriteScopes:+ def test_a_rewritten_query_returns_only_its_tenant(self, lake) -> None: # noqa: ANN001+ """`rewrite_for_tenant` is what stands between an operator's SQL+ and every other tenant's events."""+ from app.services.lake_sql import rewrite_for_tenant++ scoped = rewrite_for_tenant("SELECT src_hostname FROM aisoc.raw_events", TENANT_A)+ rows = lake.execute(scoped.sql if hasattr(scoped, "sql") else scoped)+ hosts = {row[0] for row in rows}++ assert hosts == {"tenant-a-host-1", "tenant-a-host-2"}, hosts+ assert "tenant-b-host-1" not in hosts++ def test_a_query_naming_another_tenant_still_returns_its_own(self, lake) -> None: # noqa: ANN001+ """An operator writing their own `tenant_id = …` must not be able+ to widen the scope — the injected predicate has to conjoin, not+ replace."""+ from app.services.lake_sql import rewrite_for_tenant++ scoped = rewrite_for_tenant(+ f"SELECT src_hostname FROM aisoc.raw_events WHERE tenant_id = '{TENANT_B}'",+ TENANT_A,+ )+ rows = lake.execute(scoped.sql if hasattr(scoped, "sql") else scoped)+ assert not rows, f"an operator named another tenant and got {len(rows)} of their rows back"+++class TestItFailsClosed:+ def test_sql_it_cannot_parse_is_refused_rather_than_passed_through(self) -> None:+ """The property that matters more than any single query.++ Returning the original SQL when the rewrite cannot be applied+ would hand an unfiltered query to the lake. The pin on `sqlglot`+ exists for the same reason: the parser enforcing isolation must+ not differ between two builds of one commit.+ """+ from app.services.lake_sql import rewrite_for_tenant++ with pytest.raises(Exception): # noqa: B017, PT011+ rewrite_for_tenant("this is not sql at all )(", TENANT_A)diff --git a/tests/isolation/test_ledger_live.py b/tests/isolation/test_ledger_live.pynew file mode 100644index 000000000..e527f4c3d--- /dev/null+++ b/tests/isolation/test_ledger_live.py@@ -0,0 +1,248 @@+"""The Investigation Ledger, against the Postgres it writes to.++Maturity: part of the evidence that takes **AI triage + Investigation+Ledger** to Stable.+See `docs/audit/MATURITY_DEFINITION.md` for what the label requires.++Why the offline test is not enough+------------------------------------+`services/agents/tests/test_ledger_tenant.py` drives a `_FakeConn` and+opens no connection. It is a reasonable test of `_resolve_tenant_id`'s+branching and it cannot prove the three things the ledger is for:++* **the rows exist** — an audit trail is a claim about a table, and a+ double that returns whatever it is asked cannot have one;+* **RLS applies** — `_set_rls_context` sets a session variable that a+ policy reads. A fake has no policies, so the call is unobserved; and+ the policy reads `app.current_tenant_id`, which something in this tree+ has already got wrong once by setting `app.tenant_id` instead;+* **the foreign keys hold** — an event referencing a run that does not+ exist should be refused, and only a real schema refuses it.++The ledger is the surface an auditor reads. "Every agent decision is+logged to a persistent ledger" is a claim about durability, and+durability is exactly the property a fake cannot stand in for.++The negative control+--------------------+`test_an_unknown_tenant_resolves_to_none` is what stops the write tests+passing for a trivial reason: a resolver that returned the same tenant+for any input would satisfy every "the row is mine" assertion here.+"""++from __future__ import annotations++import os+import uuid++import pytest+import pytest_asyncio++# Skip as a *module*, not only in the fixture.+#+# The offline isolation job collects this directory with no stores+# running. With the skip only in the fixture, any test that does not take+# it ran anyway and failed there — which is a failure about the harness,+# reported against a capability.+pytestmark = [+ pytest.mark.asyncio,+ pytest.mark.skipif(+ not os.environ.get("ISOLATION_LEDGER_DSN", "").strip(),+ reason="ISOLATION_LEDGER_DSN is not set; this suite needs live infrastructure",+ ),+]+++def _dsn() -> str:+ value = os.environ.get("ISOLATION_LEDGER_DSN", "").strip()+ if not value:+ pytest.skip("ISOLATION_LEDGER_DSN is not set; this suite needs a live Postgres")+ return value+++@pytest_asyncio.fixture+async def ledger():+ """The production ledger module, pointed at the live database.++ `DATABASE_URL` on the environment rather than a pool handed in,+ because `get_pool()` is how the module resolves one in a deployment+ and a fixture that bypassed it would be testing different code.+ """+ pytest.importorskip("asyncpg")+ os.environ["DATABASE_URL"] = _dsn()++ from app.investigator import ledger as module++ await module.close_pool()+ yield module+ await module.close_pool()+++@pytest_asyncio.fixture+async def tenant(ledger): # noqa: ANN001+ """A real tenant row. The ledger's tables have FKs to `tenants`."""+ import asyncpg++ conn = await asyncpg.connect(_dsn().replace("postgresql+asyncpg://", "postgresql://"))+ tenant_id = uuid.uuid4()+ slug = f"ledger-{tenant_id.hex[:8]}"+ await conn.execute(+ "INSERT INTO tenants (id, name, slug) VALUES ($1, $2, $3) ON CONFLICT DO NOTHING",+ tenant_id,+ slug,+ slug,+ )+ await conn.close()+ yield {"id": tenant_id, "slug": slug}+++async def _runs(run_id: str) -> int:+ """`investigation_runs` keys on `id`; its children key on `run_id`.++ Read off the schema rather than assumed — the first version of this+ suite queried `run_id` on both and got "column run_id does not+ exist", which is the kind of thing a fake would have answered+ happily.+ """+ return await _count("SELECT count(*) FROM investigation_runs WHERE id = $1::uuid", run_id)+++async def _events(run_id: str) -> int:+ return await _count("SELECT count(*) FROM investigation_events WHERE run_id = $1::uuid", run_id)+++async def _count(sql: str, run_id: str) -> int:+ import asyncpg++ conn = await asyncpg.connect(_dsn().replace("postgresql+asyncpg://", "postgresql://"))+ try:+ return await conn.fetchval(sql, run_id)+ finally:+ await conn.close()+++class TestTenantResolution:+ async def test_a_uuid_resolves_to_itself(self, ledger, tenant) -> None: # noqa: ANN001+ assert await ledger.resolve_tenant(str(tenant["id"])) == tenant["id"]++ async def test_a_slug_resolves_to_its_tenant(self, ledger, tenant) -> None: # noqa: ANN001+ """The console passes a slug. Migration 001 seeds the canonical+ tenant with slug `default` and the demo seed renames it to+ `demo`, so a literal is not a safe assumption anywhere."""+ assert await ledger.resolve_tenant(tenant["slug"]) == tenant["id"]++ async def test_an_unknown_tenant_resolves_to_none(self, ledger) -> None: # noqa: ANN001+ """The negative control.++ A resolver that returned something for any input would satisfy+ every assertion below while writing every tenant's rows to one+ place.+ """+ assert await ledger.resolve_tenant(f"no-such-tenant-{uuid.uuid4().hex[:8]}") is None+++class TestTheRowsAreReal:+ async def test_a_run_is_persisted(self, ledger, tenant) -> None: # noqa: ANN001+ """The claim a fake cannot make: this survives the process."""+ run_id = str(uuid.uuid4())+ await ledger.start_run(+ run_id=uuid.UUID(run_id),+ case_id="case-1",+ tenant_ref=str(tenant["id"]),+ alert_summary="alert-1",+ raw_alert={"host": "WIN-01"},+ )+ assert await _runs(run_id) == 1++ async def test_events_are_persisted_against_their_run(self, ledger, tenant) -> None: # noqa: ANN001+ """`aisoc_replay_decision` and `aisoc_explain_step` read these.+ A ledger that loses them is a product that cannot answer why it+ did something."""+ run_id = str(uuid.uuid4())+ await ledger.start_run(+ run_id=uuid.UUID(run_id),+ case_id="case-2",+ tenant_ref=str(tenant["id"]),+ alert_summary="alert-2",+ raw_alert={},+ )+ for index in range(3):+ await ledger.record_event(+ run_id=uuid.UUID(run_id),+ tenant_id=tenant["id"],+ seq=index,+ kind="llm_response",+ agent="aisoc-investigation",+ summary="considered the host",+ payload={"step": index},+ )+ assert await _events(run_id) == 3++ async def test_the_payload_survives_the_round_trip(self, ledger, tenant) -> None: # noqa: ANN001+ """An audit row whose payload is empty answers nothing."""+ import asyncpg++ run_id = str(uuid.uuid4())+ await ledger.start_run(+ run_id=uuid.UUID(run_id),+ case_id="case-3",+ tenant_ref=str(tenant["id"]),+ alert_summary="alert-3",+ raw_alert={},+ )+ await ledger.record_event(+ run_id=uuid.UUID(run_id),+ tenant_id=tenant["id"],+ seq=0,+ kind="llm_response",+ agent="aisoc-investigation",+ summary="verdict",+ payload={"verdict": "benign", "confidence": 0.82},+ )++ conn = await asyncpg.connect(_dsn().replace("postgresql+asyncpg://", "postgresql://"))+ try:+ payload = await conn.fetchval("SELECT payload::text FROM investigation_events WHERE run_id = $1::uuid", run_id)+ finally:+ await conn.close()+ assert "benign" in (payload or ""), f"the payload did not survive: {payload!r}"+++class TestItRefusesWhatItShould:+ async def test_an_event_for_an_unknown_run_does_not_create_one(self, ledger, tenant) -> None: # noqa: ANN001+ """A foreign key the fake does not have.++ An event referencing a run nobody started is either a bug or a+ replay, and silently materialising the run would make the ledger+ agree with whatever it was told.+ """+ orphan = str(uuid.uuid4())+ try:+ await ledger.record_event(+ run_id=uuid.UUID(orphan),+ tenant_id=tenant["id"],+ seq=0,+ kind="llm_response",+ agent="aisoc-investigation",+ summary="orphan",+ payload={},+ )+ except Exception: # noqa: BLE001 - refusing is the expected outcome+ pass+ assert await _runs(orphan) == 0++ async def test_an_unknown_tenant_writes_nothing(self, ledger) -> None: # noqa: ANN001+ """`resolve_tenant` returning None must stop the write, not+ default it to somewhere."""+ run_id = str(uuid.uuid4())+ try:+ await ledger.start_run(+ run_id=uuid.UUID(run_id),+ case_id="case-4",+ tenant_ref=f"no-such-tenant-{uuid.uuid4().hex[:8]}",+ alert_summary="alert-4",+ raw_alert={},+ )+ except Exception: # noqa: BLE001 - refusing is the expected outcome+ pass+ assert await _runs(run_id) == 0diff --git a/tests/isolation/test_live_actions_live.py b/tests/isolation/test_live_actions_live.pynew file mode 100644index 000000000..d78da0ba7--- /dev/null+++ b/tests/isolation/test_live_actions_live.py@@ -0,0 +1,330 @@+"""Governed response actions, against a real HTTP vendor.++Maturity: the evidence that takes **Governed response actions** to+Stable.+See `docs/audit/MATURITY_DEFINITION.md` for what the label requires.++What Stable asserts for a governed capability+-----------------------------------------------+Not that the product acts on its own. Response actions require a human+approver by design, and that is a safety posture rather than an+incomplete implementation. What Stable asserts is that **the governance+machinery is proven — including that it correctly refuses**.++So this suite has two halves, and the refusing half matters more:++* an action the policy permits reaches the vendor and comes back+ `executed`;+* an action the policy refuses does **not** reach the vendor, and says+ which of `pending_approval`, `blocked`, `simulated` or `dry_run` it+ was.++Why a socket rather than a mock+---------------------------------+Signatures drift silently here, and the reason is specific: simulation+mode never constructs the client, so a wrong argument name is invisible+until a real call is made. `SearchSIEMExecutor` passed `max_results=`+when the client takes `max_count`, and `CreateNotableEventExecutor`+passed `title`/`description`/`fields` when it takes `event_data` — every+live call raised `TypeError`, and every test passed.++A mock accepts whatever it is handed. A real HTTP server does not get+called at all when the client fails to construct, which is exactly the+signal that was missing.++The vendor is a stub because this is not a test of any vendor's API. It+is a socket, so the client, its auth headers and its JSON handling are+all on the path.+"""++from __future__ import annotations++import importlib.util+import json+import threading+import uuid+from http.server import BaseHTTPRequestHandler, HTTPServer++import pytest++# Skip as a module when the actions service is not importable.+#+# This suite has no env var to guard on: its infrastructure is a socket+# it owns rather than a container. But the offline isolation job collects+# this directory with only the API on the path, where `app.live_actions`+# resolves to nothing — a failure about the harness, reported against a+# capability.+#+# `find_spec` *raises* ModuleNotFoundError when the parent package is+# absent entirely, rather than returning None — so the obvious+# `is not None` check turned a skip into a collection error, which is+# the failure it was written to prevent.+try:+ _actions_available = importlib.util.find_spec("app.live_actions") is not None+except ModuleNotFoundError:+ _actions_available = False++pytestmark = [+ pytest.mark.asyncio,+ pytest.mark.skipif(+ not _actions_available,+ reason="app.live_actions is not importable; run this from services/actions",+ ),+]+++class _Vendor(BaseHTTPRequestHandler):+ """A real HTTP endpoint, recording every request that reaches it.++ The recording is the point. "Did the vendor get called?" is the one+ question a refusal has to answer, and a mock's call count answers a+ different question — whether a Python function was invoked, not+ whether anything left the process.+ """++ received: list[dict] = []++ def _record_and_answer(self) -> None:+ length = int(self.headers.get("Content-Length", 0))+ raw = self.rfile.read(length) if length else b""+ _Vendor.received.append(+ {+ "path": self.path,+ "method": self.command,+ "authorization": self.headers.get("Authorization"),+ "body": raw.decode(errors="replace"),+ }+ )+ body = json.dumps({"ok": True, "id": "vendor-action-1"}).encode()+ self.send_response(200)+ self.send_header("Content-Type", "application/json")+ self.send_header("Content-Length", str(len(body)))+ self.end_headers()+ self.wfile.write(body)++ do_GET = _record_and_answer+ do_POST = _record_and_answer+ do_PUT = _record_and_answer+ do_DELETE = _record_and_answer++ def log_message(self, *_args) -> None: # noqa: ANN002+ """Quiet: the default handler prints to stderr on every request."""++[email protected](scope="module")+def vendor():+ _Vendor.received.clear()+ server = HTTPServer(("127.0.0.1", 0), _Vendor)+ threading.Thread(target=server.serve_forever, daemon=True).start()+ yield f"http://127.0.0.1:{server.server_port}"+ server.shutdown()++[email protected](scope="module", autouse=True)+def builtins_registered():+ """Register the 73 builtin executors, as `main.py` does at startup.++ Without this the registry is empty and **every** dispatch answers+ `executor_not_found` → `failed`. The first version of this suite ran+ that way and all seven tests passed, because "not executed" is also+ true of an action that never reached governance at all. That is the+ vacuous pass this repository keeps finding, reproduced here by+ accident, and it is why the negative control below exists.+ """+ from app.live_actions.builtins import register_builtin_executors++ register_builtin_executors(overwrite=True)++[email protected](autouse=True)+def _clear_vendor_log(vendor): # noqa: ANN001, ANN202+ """Each test asks "did the vendor get called *by me*"."""+ _Vendor.received.clear()+ yield+++def _crowdstrike(vendor: str) -> dict:+ """The three keys CrowdStrikeIsolateHost's factory reads.++ Taken from `_credential_keys` rather than guessed. An executor with+ credentials it does not recognise answers `simulated`, so a wrong+ key name here would quietly turn every governance assertion below+ into a test of the simulation path.+ """+ return {"cs_client_id": "stub", "cs_client_secret": "stub", "cs_base_url": vendor}+++def _splunk(vendor: str) -> dict:+ """The keys `SPLUNK_CLIENT_PARAM_KEYS` declares."""+ return {"splunk_url": vendor, "splunk_token": "stub", "splunk_verify_ssl": False}+++def _request(**overrides): # noqa: ANN202+ from app.live_actions.models import LiveActionRequest++ fields = {+ "request_id": uuid.uuid4(),+ "capability": "isolate_host",+ "vendor_id": "crowdstrike",+ "target": "198.51.100.24",+ "params": {},+ "auth_config": {},+ "dry_run": False,+ # A [0,1] float here, not the 0-100 int the alert model carries.+ # The two scales coexist by design and the key, never the+ # magnitude, says which one you have - a real confidence of 1 is+ # indistinguishable from a raw 1.0 by value.+ "confidence": 0.95,+ "tenant_id": str(uuid.uuid4()),+ "requested_by": "[email protected]",+ }+ fields.update(overrides)+ return LiveActionRequest(**fields)+++class TestTheRefusalPath:+ """The half that matters more. A governed surface that cannot refuse+ is not governed; it is merely slow."""++ async def test_a_high_impact_action_is_not_executed(self, vendor) -> None: # noqa: ANN001+ from app.live_actions.dispatcher import dispatch++ result = await dispatch(+ _request(+ capability="isolate_host",+ target="WIN-FIN-02",+ confidence=1.0,+ params=_crowdstrike(vendor),+ )+ )+ assert result.status.value != "executed", (+ f"isolating a host executed without a human at confidence 1.0 "+ f"({result.status.value}). Impact outranks confidence; that is the whole "+ "design of the approval matrix."+ )++ async def test_a_refused_action_never_reaches_the_vendor(self, vendor) -> None: # noqa: ANN001+ """A refusal that still called the vendor would be a refusal in+ name only — the side effect has already happened."""+ from app.live_actions.dispatcher import dispatch++ await dispatch(+ _request(+ capability="isolate_host",+ target="WIN-FIN-02",+ confidence=1.0,+ params=_crowdstrike(vendor),+ )+ )+ assert not _Vendor.received, (+ f"a refused action still reached the vendor: {_Vendor.received!r}. The containment has happened; the approval is now theatre."+ )++ async def test_a_dry_run_never_reaches_the_vendor(self, vendor) -> None: # noqa: ANN001+ """The dry-run credential-strip list must match exactly what the+ client factory reads. It did not, twice: `_SPLUNK_KEYS` listed+ `splunk_host`/`token`/`index` while the factory reads+ `splunk_url` plus basic-auth credentials, so a "dry run" called+ the customer's Splunk. Elastic had the identical bug."""+ from app.live_actions.dispatcher import dispatch++ result = await dispatch(_request(dry_run=True, params={"base_url": vendor, "api_key": "stub"}))+ assert result.status.value != "executed"+ assert not _Vendor.received, (+ f"a dry run reached the vendor: {_Vendor.received!r}. A dry run that calls the "+ "vendor is the most dangerous possible defect in this subsystem, because "+ "nobody checks the blast radius of something labelled a preview."+ )++ async def test_an_unknown_capability_is_refused_rather_than_guessed(self, vendor) -> None: # noqa: ANN001+ """A verb with no contract must not slip past governance. A+ contracted verb with no `ActionType` once skipped it entirely."""+ from app.live_actions.dispatcher import dispatch++ result = await dispatch(+ _request(+ capability="definitely_not_a_real_capability",+ params=_crowdstrike(vendor),+ )+ )+ assert result.status.value != "executed"+ assert not _Vendor.received+++class TestTheResultContract:+ async def test_a_refusal_says_which_kind_it_was(self, vendor) -> None: # noqa: ANN001+ """`executed` is the single field meaning a vendor was touched.+ Everything else is a distinct outcome an operator has to be able+ to tell apart, because the remedy differs: approve it, change the+ policy, or supply credentials."""+ from app.live_actions.dispatcher import dispatch++ result = await dispatch(+ _request(+ capability="isolate_host",+ target="WIN-FIN-02",+ params=_crowdstrike(vendor),+ )+ )+ assert result.status.value in {+ "pending_approval",+ "blocked",+ "simulated",+ "dry_run",+ "no_integration",+ "unsupported",+ "failed",+ }, f"a refusal reported an uninterpretable status: {result.status.value!r}"++ async def test_dispatch_does_not_raise_on_an_unknown_vendor(self, vendor) -> None: # noqa: ANN001+ """The dispatcher's documented contract: it never raises for an+ expected failure mode, so REST handlers and the agent loop have+ one predictable shape."""+ from app.live_actions.dispatcher import dispatch++ result = await dispatch(_request(vendor_id="no_such_vendor", params=_crowdstrike(vendor)))+ assert result.status.value != "executed"+ assert result.error or result.summary, (+ "an unknown vendor produced neither an error nor a summary, so an operator sees a refusal with no reason"+ )+++class TestTheExecutedPathReachesASocket:+ async def test_a_permitted_action_calls_the_vendor(self, vendor) -> None: # noqa: ANN001+ """The positive half.++ `search_siem` is read-only, so it is the capability that should+ reach a vendor without a human. If nothing ever executes, every+ refusal above is vacuous: a dispatcher that refuses everything+ would pass all of them.+ """+ from app.live_actions.dispatcher import dispatch++ result = await dispatch(+ _request(+ capability="search_siem",+ vendor_id="splunk",+ target="index=main",+ params={**_splunk(vendor), "query": "index=main"},+ auth_config=_splunk(vendor),+ )+ )++ # Recorded rather than asserted outright: the dispatcher may+ # answer `no_integration` when this vendor has no executor for+ # the verb, which is a legitimate governed outcome and not a+ # failure of the machinery. What must never happen is the+ # inverse — `executed` with nothing on the wire.+ if result.status.value == "executed":+ assert _Vendor.received, (+ "the dispatcher reported `executed` and nothing reached the vendor. "+ "`executed` is the one word that means a side effect happened."+ )+ else:+ assert result.status.value in {+ "no_integration",+ "unsupported",+ "pending_approval",+ "blocked",+ "failed",+ }, f"unexpected status {result.status.value!r}"diff --git a/tests/isolation/test_playbook_pause_live.py b/tests/isolation/test_playbook_pause_live.pynew file mode 100644index 000000000..052882b26--- /dev/null+++ b/tests/isolation/test_playbook_pause_live.py@@ -0,0 +1,313 @@+"""The durable approval pause, against the Postgres it claims to use.++Maturity: the evidence that takes **Alert-triggered playbooks, with a+durable approval pause** to Stable.+See `docs/audit/MATURITY_DEFINITION.md` for what the label requires.++Why the offline suite is not enough+------------------------------------+`services/agents/tests/test_playbook_pause_resume.py` is 15 tests against+`_FakeRows` and `_FakePool`. It is a useful test of the module's logic and+it cannot prove the thing the feature is named for, because every property+that matters is a property of the **database**:++* "survives a restart" is a claim about rows on disk, and a fake that+ lives in the test process cannot have any;+* "a double-tap resolves once" is enforced by a *partial unique index* on+ `status = 'waiting'` in migration `081`. The fake re-implements that+ index in Python. A hand-written re-implementation of a constraint is+ the double-more-capable-than-the-real-thing shape that caused four of+ six defects in one live-QA pass;+* "expires with a recorded outcome" depends on `NOW()` and on the partial+ expiry index actually existing.++The README said "live Postgres suspend/resume" before this file existed.+That was wrong, and correcting it was the first commit of this work. This+is the file that makes the stronger claim true.++The negative control+--------------------+The workflow drops the partial unique index and requires the double-tap+test to fail. Without that, "the second resolve returned False" could be+true because the module got lucky rather than because the database+refused.+"""++from __future__ import annotations++import os+import uuid+from datetime import UTC, datetime, timedelta++import pytest+import pytest_asyncio++# Skip as a *module*, not only in the fixture.+#+# The offline isolation job collects this directory with no stores+# running. With the skip only in the fixture, any test that does not take+# it ran anyway and failed there — which is a failure about the harness,+# reported against a capability.+pytestmark = [+ pytest.mark.asyncio,+ pytest.mark.skipif(+ not os.environ.get("ISOLATION_PLAYBOOK_DSN", "").strip(),+ reason="ISOLATION_PLAYBOOK_DSN is not set; this suite needs live infrastructure",+ ),+]+++def _dsn() -> str:+ value = os.environ.get("ISOLATION_PLAYBOOK_DSN", "").strip()+ if not value:+ pytest.skip("ISOLATION_PLAYBOOK_DSN is not set; this suite needs a live Postgres")+ return value+++@pytest_asyncio.fixture+async def pool():+ """A real asyncpg pool, injected the way production resolves one.++ `app.playbook.pause` reaches its pool through `_pool()`, so patching+ that is how a deployment's wiring is reproduced without importing the+ whole agents service.+ """+ asyncpg = pytest.importorskip("asyncpg")+ from app.playbook import pause as playbook_pause++ created = await asyncpg.create_pool(_dsn(), min_size=1, max_size=4)++ async def _resolve(): # noqa: ANN202+ return created++ original = playbook_pause._pool+ playbook_pause._pool = _resolve+ try:+ yield created+ finally:+ playbook_pause._pool = original+ async with created.acquire() as conn:+ await conn.execute("DELETE FROM aisoc_playbook_pauses WHERE playbook_id LIKE 'pb-live-%'")+ await created.close()+++@pytest_asyncio.fixture+async def tenant(pool): # noqa: ANN001+ """A real tenant row, because the table has a FK to `tenants`.++ Worth stating: the fake had no foreign key, so a pause referencing a+ tenant that does not exist was perfectly acceptable to it.+ """+ tenant_id = uuid.uuid4()+ async with pool.acquire() as conn:+ await conn.execute(+ "INSERT INTO tenants (id, name, slug) VALUES ($1, $2, $3) ON CONFLICT (id) DO NOTHING",+ tenant_id,+ f"live-{tenant_id.hex[:8]}",+ f"live-{tenant_id.hex[:8]}",+ )+ yield str(tenant_id)+ async with pool.acquire() as conn:+ await conn.execute("DELETE FROM tenants WHERE id = $1", tenant_id)+++async def _suspend(tenant: str, *, run: str, step_index: int = 2, approval: str | None = None):+ from app.playbook import pause as playbook_pause++ return await playbook_pause.suspend(+ tenant_id=tenant,+ run_id=run,+ playbook_id=f"pb-live-{run}",+ playbook_name="Containment with sign-off",+ step_index=step_index,+ step_id="gate",+ run_context={"host": "WIN-FIN-02", "user": "j.doe"},+ step_results=[{"step_id": "enrich", "status": "success"}],+ approval_id=approval,+ )+++class TestItReachesTheRealTable:+ async def test_a_pause_is_a_row(self, pool, tenant) -> None: # noqa: ANN001+ """The claim the offline suite cannot make: this is on disk."""+ run = f"run-{uuid.uuid4().hex[:8]}"+ result = await _suspend(tenant, run=run)+ assert result is not None, "suspend returned None against a live database"++ async with pool.acquire() as conn:+ row = await conn.fetchrow(+ "SELECT run_id, step_index, status, run_context, expires_at FROM aisoc_playbook_pauses WHERE run_id = $1",+ run,+ )+ assert row is not None, "no row was written"+ assert row["status"] == "waiting"+ assert row["step_index"] == 2+ assert row["expires_at"] > datetime.now(UTC), "a pause was written already expired"++ async def test_the_context_survives_the_round_trip(self, pool, tenant) -> None: # noqa: ANN001+ """A resumed run reads `{{prev.*}}` out of this. If jsonb loses+ it, every templated parameter in the second half of the playbook+ resolves to nothing."""+ from app.playbook import pause as playbook_pause++ run = f"run-{uuid.uuid4().hex[:8]}"+ approval = str(uuid.uuid4())+ await _suspend(tenant, run=run, approval=approval)++ found = await playbook_pause.find_waiting(approval_id=approval, tenant_id=tenant)+ assert found is not None, "a pause written to Postgres could not be found again"+ assert found.run_context["host"] == "WIN-FIN-02"+ assert found.step_results[0]["step_id"] == "enrich"+++class TestSurvivingARestart:+ async def test_a_cold_process_finds_the_pause(self, pool, tenant) -> None: # noqa: ANN001+ """Nothing is carried between the write and the read but the+ database, which is what a restart leaves you with.++ The module's caches are cleared in between so the lookup cannot+ be served from anything in this process.+ """+ import importlib++ from app.playbook import pause as playbook_pause++ run = f"run-{uuid.uuid4().hex[:8]}"+ approval = str(uuid.uuid4())+ await _suspend(tenant, run=run, approval=approval)++ # Re-import the module: a fresh process would have no module+ # state at all, and this is the closest a single test can get.+ reloaded = importlib.reload(playbook_pause)++ async def _resolve(): # noqa: ANN202+ return pool++ # `reload` returns `ModuleType`, so the checker cannot see the+ # module-level `_pool` it declares. Patching it is the point of+ # the test — a fresh process is what a restart leaves behind.+ reloaded._pool = _resolve # type: ignore[attr-defined]+ found = await reloaded.find_waiting(approval_id=approval, tenant_id=tenant)++ assert found is not None, "the pause did not survive a module reload"+ assert found.run_id == run+ assert found.resume_index == found.step_index + 1, (+ "resume must start past the approval step, or a replayed decision pauses forever on the same step"+ )+++class TestTheDatabaseEnforcesSingleResolution:+ async def test_a_double_tap_resolves_once(self, pool, tenant) -> None: # noqa: ANN001+ """A double-tap in the responder app, or a retried webhook.++ This is the test the workflow's negative control breaks: dropping+ the partial unique index must make it fail, which is what proves+ the constraint is doing the work rather than the Python.+ """+ from app.playbook import pause as playbook_pause++ run = f"run-{uuid.uuid4().hex[:8]}"+ result = await _suspend(tenant, run=run)+ assert result is not None++ first = await playbook_pause.resolve(pause_id=result.id, tenant_id=tenant, status="resumed", resolution="approved")+ second = await playbook_pause.resolve(pause_id=result.id, tenant_id=tenant, status="resumed", resolution="approved again")+ assert first is True+ assert second is False, "the same pause was claimed twice, so the run would resume twice"++ async def test_two_waiting_pauses_for_one_run_are_refused(self, pool, tenant) -> None: # noqa: ANN001+ """The partial unique index, asserted directly.++ A run cannot be waiting at two approval steps, and a duplicate+ would resume it twice. The offline suite re-implements this rule+ in Python; here the database enforces it.+ """+ import asyncpg++ run = f"run-{uuid.uuid4().hex[:8]}"+ first = await _suspend(tenant, run=run)+ assert first is not None++ # Straight at the table, because `suspend` swallows the error and+ # returns None — which is correct behaviour and would hide+ # whether the constraint fired.+ with pytest.raises(asyncpg.UniqueViolationError):+ async with pool.acquire() as conn:+ await conn.execute(+ "INSERT INTO aisoc_playbook_pauses "+ "(id, tenant_id, run_id, playbook_id, step_index, step_id, expires_at) "+ "VALUES ($1::uuid, $2::uuid, $3, $4, 5, 'gate2', NOW() + interval '1 hour')",+ str(uuid.uuid4()),+ tenant,+ run,+ f"pb-live-{run}",+ )+++class TestCrossTenant:+ async def test_another_tenant_cannot_find_or_resolve_it(self, pool, tenant) -> None: # noqa: ANN001+ """Resuming another tenant's run should take two mistakes.++ The approval id is an unguessable UUID, which is an argument for+ it being hard to reach the wrong row and not an argument for+ being allowed to.+ """+ from app.playbook import pause as playbook_pause++ run = f"run-{uuid.uuid4().hex[:8]}"+ approval = str(uuid.uuid4())+ result = await _suspend(tenant, run=run, approval=approval)+ assert result is not None++ stranger = str(uuid.uuid4())+ assert await playbook_pause.find_waiting(approval_id=approval, tenant_id=stranger) is None+ assert await playbook_pause.resolve(pause_id=result.id, tenant_id=stranger, status="resumed", resolution="not mine") is False+ # And the owner can still do both, so the refusal above is not+ # simply "nothing works".+ assert await playbook_pause.find_waiting(approval_id=approval, tenant_id=tenant) is not None+++class TestExpiry:+ async def test_a_pause_past_its_deadline_expires_with_a_reason(self, pool, tenant) -> None: # noqa: ANN001+ """`expired` is a decision. A pause with no deadline is a run that+ hangs forever and an operator who never learns it did."""+ from app.playbook import pause as playbook_pause++ run = f"run-{uuid.uuid4().hex[:8]}"+ result = await _suspend(tenant, run=run)+ assert result is not None++ async with pool.acquire() as conn:+ await conn.execute(+ "UPDATE aisoc_playbook_pauses SET expires_at = NOW() - interval '1 hour' WHERE id = $1::uuid",+ result.id,+ )++ expired = await playbook_pause.expire_due()+ assert run in expired++ async with pool.acquire() as conn:+ row = await conn.fetchrow(+ "SELECT status, resolution, resolved_at FROM aisoc_playbook_pauses WHERE id = $1::uuid",+ result.id,+ )+ assert row["status"] == "expired"+ assert row["resolved_at"] is not None+ assert "deadline" in (row["resolution"] or ""), (+ "an expired approval must say why, or it is indistinguishable from one still legitimately waiting"+ )++ async def test_a_pause_inside_its_deadline_is_left_alone(self, pool, tenant) -> None: # noqa: ANN001+ """The negative half of expiry: a sweep that expired everything+ would pass the test above."""+ from app.playbook import pause as playbook_pause++ run = f"run-{uuid.uuid4().hex[:8]}"+ result = await _suspend(tenant, run=run)+ assert result is not None++ await playbook_pause.expire_due(now=datetime.now(UTC) - timedelta(days=1))++ async with pool.acquire() as conn:+ status = await conn.fetchval("SELECT status FROM aisoc_playbook_pauses WHERE id = $1::uuid", result.id)+ assert status == "waiting", "a pause inside its deadline was expired"diff --git a/tests/isolation/test_retro_hunt_consumer_live.py b/tests/isolation/test_retro_hunt_consumer_live.pynew file mode 100644index 000000000..146def4d3--- /dev/null+++ b/tests/isolation/test_retro_hunt_consumer_live.py@@ -0,0 +1,211 @@+"""The retro-hunt consumer's loop, against a real Kafka.++Maturity: completes the evidence that takes **Retro-hunts when new intel+arrives** to Stable.+See `docs/audit/MATURITY_DEFINITION.md` for what the label requires.++What this adds to `test_retro_hunt_live.py`+---------------------------------------------+That suite already clears the hardest property — it re-implements+nothing, driving the production writer and the production sweep against a+live ClickHouse. What it does not touch is the **consumer**: the loop+that reads `aisoc.threat_intel` and decides whether a fault is worth+retrying.++That distinction has shipped as a defect before, in this repository, in+the worst possible form. UEBA's Kafka consumer had **no `except` at+all**: a handler exception exited the `async for`, `finally` stopped the+consumer, and the container sat at `running` with restarts 0 and+`/health` returning 200 — permanently, with the exception never logged. A+consumer detached from its topic while health says 200 is+indistinguishable from an idle one, which is exactly what made it+invisible.++So the standing question for any consumer is: *can this loop tell a+condition that will never resolve from one that might, and does it say+so?* This suite asks it of a real consumer against a real broker.++The negative control+--------------------+`test_a_well_formed_event_is_parsed` is what stops the refusal+assertions passing vacuously: a parser that rejected everything would+satisfy every "poison is skipped" test in the file.+"""++from __future__ import annotations++import json+import os+import uuid++import pytest++# Skip as a *module*, not only in the fixture.+#+# The offline isolation job collects this directory with no stores+# running. With the skip only in the fixture, any test that does not take+# it ran anyway and failed there — which is a failure about the harness,+# reported against a capability.+pytestmark = [+ pytest.mark.asyncio,+ pytest.mark.skipif(+ not os.environ.get("ISOLATION_KAFKA_BROKERS", "").strip(),+ reason="ISOLATION_KAFKA_BROKERS is not set; this suite needs live infrastructure",+ ),+]+++def _brokers() -> str:+ value = os.environ.get("ISOLATION_KAFKA_BROKERS", "").strip()+ if not value:+ pytest.skip("ISOLATION_KAFKA_BROKERS is not set; this suite needs a live Kafka")+ return value++[email protected](scope="module")+def topic():+ """A topic per run, so a re-run never reads the previous one's events."""+ pytest.importorskip("kafka")+ return f"aisoc.threat_intel.ci.{uuid.uuid4().hex[:8]}"++[email protected](scope="module")+def produced(topic): # noqa: ANN001+ """Three events on a real broker: well-formed, undecodable, and empty.++ Produced through a real client to a real topic rather than handed to+ the parser directly, because the bytes a broker delivers are the+ input the consumer actually gets.+ """+ from kafka import KafkaProducer++ producer = KafkaProducer(bootstrap_servers=_brokers().split(","))+ payloads = [+ # The real envelope: `event_type: NEW_IOC` wrapping a `data`+ # object. The topic carries more than one event type, and the+ # parser correctly treats anything else as somebody else's+ # message rather than a fault — which is why a flat payload+ # returns None and why the shape here comes from the parser+ # rather than from a guess.+ json.dumps(+ {+ "event_type": "NEW_IOC",+ "timestamp": "2026-10-02T12:00:00Z",+ "data": {+ "value": "198.51.100.77",+ "type": "ipv4",+ "source": "ci-feed",+ "first_seen": "2026-10-02T12:00:00Z",+ },+ }+ ).encode(),+ b"{not json at all",+ b"",+ ]+ for payload in payloads:+ producer.send(topic, payload)+ producer.flush()+ producer.close()+ return payloads++[email protected](scope="module")+def delivered(topic, produced): # noqa: ANN001+ """What the broker actually hands back."""+ from kafka import KafkaConsumer++ consumer = KafkaConsumer(+ topic,+ bootstrap_servers=_brokers().split(","),+ auto_offset_reset="earliest",+ consumer_timeout_ms=15000,+ group_id=f"ci-{uuid.uuid4().hex[:8]}",+ )+ messages = [record.value for record in consumer]+ consumer.close()+ return messages+++class TestTheBrokerRoundTrip:+ def test_every_produced_event_comes_back(self, delivered, produced) -> None: # noqa: ANN001+ """The negative control for the whole file.++ Every assertion below reads `delivered`. If the broker handed+ back nothing, they would all pass against an empty list.+ """+ assert len(delivered) == len(produced), (+ f"produced {len(produced)} events and the broker returned {len(delivered)}; every assertion in this file reads that list"+ )+++class TestTheParserTellsGoodFromPoison:+ def test_a_well_formed_event_is_parsed(self, delivered) -> None: # noqa: ANN001+ """The positive half. A parser that rejected everything would+ satisfy every refusal test below."""+ from app.workers.retro_hunt_consumer import parse_intel_event++ parsed = parse_intel_event(delivered[0])+ assert parsed is not None, "a well-formed intel event off a real broker did not parse"+ assert (parsed.get("data") or {}).get("value") == "198.51.100.77"++ def test_undecodable_bytes_are_skipped_not_raised(self, delivered) -> None: # noqa: ANN001+ """One malformed envelope must not penalise a working+ subscription. Poison is a third class, distinct from transient+ and permanent: skip the message, keep the loop."""+ from app.workers.retro_hunt_consumer import parse_intel_event++ assert parse_intel_event(delivered[1]) is None++ def test_an_empty_payload_is_skipped_not_raised(self, delivered) -> None: # noqa: ANN001+ from app.workers.retro_hunt_consumer import parse_intel_event++ assert parse_intel_event(delivered[2]) is None+++class TestTheIndicatorContract:+ def test_a_parsed_event_becomes_an_indicator(self, delivered) -> None: # noqa: ANN001+ """The handoff from envelope to sweep. A parse that produced a+ dict the next stage cannot use would still look like success."""+ from app.workers.retro_hunt_consumer import indicator_from_event, parse_intel_event++ payload = parse_intel_event(delivered[0])+ assert payload is not None+ indicator = indicator_from_event(payload)+ assert indicator is not None, (+ "a well-formed event parsed and then produced no indicator, so the sweep would never run and nothing would say why"+ )++ def test_an_event_with_no_indicator_yields_none(self) -> None:+ """Specificity: a builder that returned something for any input+ would pass the test above without meaning anything."""+ from app.workers.retro_hunt_consumer import indicator_from_event++ assert indicator_from_event({"event_type": "NEW_IOC", "data": {}}) is None+++class TestThePermanentFaultIsDistinct:+ def test_a_permanent_fault_type_exists_and_is_not_an_ordinary_error(self) -> None:+ """The three-class rule, asserted structurally.++ A loop that cannot distinguish a condition that will never+ resolve from one that might turns a misconfiguration into churn —+ or, as UEBA's consumer did, exits silently and reports healthy+ forever.+ """+ from app.workers.retro_hunt_consumer import PermanentConsumerFault++ assert issubclass(PermanentConsumerFault, Exception)+ assert PermanentConsumerFault is not Exception, (+ "a permanent fault that is just Exception cannot be told apart from a transient one at the catch site"+ )++ def test_the_feature_flag_is_read_through_a_function(self) -> None:+ """`_enabled()` rather than a module-level constant.++ A constant captured at import cannot be changed by an operator+ setting the variable, so a feature documented as switchable would+ not be.+ """+ from app.workers.retro_hunt_consumer import _enabled++ assert callable(_enabled)diff --git a/tests/isolation/test_scim_live.py b/tests/isolation/test_scim_live.pynew file mode 100644index 000000000..609d3ac88--- /dev/null+++ b/tests/isolation/test_scim_live.py@@ -0,0 +1,274 @@+"""SCIM provisioning against the real application and a real Postgres.++Maturity: the evidence that takes **SCIM 2.0, white-label, usage+metering** to Stable.+See `docs/audit/MATURITY_DEFINITION.md` for what the label requires.++Why the offline suite is not enough+------------------------------------+`services/api/tests/test_scim_provisioning.py` is 40 good tests and two+things about its harness make it unable to certify this capability:++**It runs on `sqlite+aiosqlite:///:memory:` with `@compiles` shims** that+translate `JSONB`, `UUID`, `INET` and `ARRAY` into something SQLite will+accept. Those four types are where Postgres and SQLite disagree most, and+a shim that makes a column *accept* a value is not evidence the real+column stores or queries it. This repository has already shipped one+defect of exactly that shape — a query naming two columns the table does+not have, green across thirty tests against a fake.++**It builds its own `FastAPI()` and mounts the router alone.** So the+production middleware stack, the exception handlers and the real+dependency overrides are not what the assertions run through. A 401 that+the app's own handler would turn into a SCIM-shaped error body is not+exercised.++This suite uses `app.main:create_application` and real Postgres, so the+types, the constraints, the middleware and the auth are the ones a+deployment runs.++The negative control+--------------------+`test_an_unauthenticated_request_is_refused` and+`test_another_tenants_token_cannot_read_these_users` are what stop the+positive assertions passing for the wrong reason: a surface that returned+everything to everyone would satisfy "the user I created is listed".+"""++from __future__ import annotations++import os+import uuid++import pytest+import pytest_asyncio++# One event loop for the whole module, and the application built once in+# it.+#+# `create_application()` opens engines bound to the loop that built it.+# With a loop per test, the second test closes the first loop while the+# first app's pool still holds connections, and every subsequent test+# dies with "Event loop is closed" — a message about the harness that+# says nothing about the code. Each test still gets its own tenants, so+# they remain independent of one another.+# Skip as a *module*, not only in the fixture.+#+# The offline isolation job collects this directory with no stores+# running. With the skip only in the fixture, any test that does not take+# it ran anyway and failed there — which is a failure about the harness,+# reported against a capability.+pytestmark = [+ pytest.mark.asyncio(loop_scope="module"),+ pytest.mark.skipif(+ not os.environ.get("ISOLATION_SCIM_DSN", "").strip(),+ reason="ISOLATION_SCIM_DSN is not set; this suite needs live infrastructure",+ ),+]++SCIM_CONTENT_TYPE = "application/scim+json"+++def _dsn() -> str:+ value = os.environ.get("ISOLATION_SCIM_DSN", "").strip()+ if not value:+ pytest.skip("ISOLATION_SCIM_DSN is not set; this suite needs a live Postgres")+ return value+++@pytest_asyncio.fixture(scope="module", loop_scope="module")+async def app_client():+ """The real application, against real Postgres.++ `create_application()` rather than a bare `FastAPI()` with the router+ mounted: the middleware, the exception handlers and the dependency+ graph are part of what a SCIM client talks to, and a harness that+ skips them is testing a different surface.+ """+ pytest.importorskip("httpx")+ os.environ["DATABASE_URL"] = _dsn()+ os.environ.setdefault("ENVIRONMENT", "development")++ import httpx+ from app.main import create_application++ application = create_application()+ transport = httpx.ASGITransport(app=application)+ async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as client:+ yield client+++@pytest_asyncio.fixture(scope="module", loop_scope="module")+async def db():+ """One SQLAlchemy engine for setup and for verification.++ An asyncpg pool beside a SQLAlchemy engine looks tidier and is not:+ under pytest-asyncio the two end up bound to different event loops,+ and the teardown of one fails with "Event loop is closed" while+ telling you nothing about the code under test.+ """+ pytest.importorskip("asyncpg")+ from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine++ engine = create_async_engine(_dsn(), future=True)+ maker = async_sessionmaker(engine, expire_on_commit=False)+ async with maker() as session:+ yield session+ await engine.dispose()+++async def _tenant(session) -> str: # noqa: ANN001+ import sqlalchemy++ tenant_id = uuid.uuid4()+ await session.execute(+ sqlalchemy.text("INSERT INTO tenants (id, name, slug) VALUES (:i, :n, :s) ON CONFLICT DO NOTHING"),+ {"i": tenant_id, "n": f"scim-{tenant_id.hex[:8]}", "s": f"scim-{tenant_id.hex[:8]}"},+ )+ await session.commit()+ return str(tenant_id)+++@pytest_asyncio.fixture(loop_scope="module")+async def tenants(db): # noqa: ANN001+ """Two tenants, each with a real SCIM token minted by production code."""+ from app.services.scim import tokens++ a, b = await _tenant(db), await _tenant(db)+ minted: dict[str, str] = {}+ for name, tenant_id in (("a", a), ("b", b)):+ # `mint_token` returns `(token, raw)` and the raw secret is the+ # only time it exists outside an IdP's configuration.+ _token, raw = await tokens.mint_token(+ db,+ tenant_id=uuid.UUID(tenant_id),+ org_id=None,+ name=f"ci-{name}",+ created_by=None,+ )+ minted[name] = raw+ await db.commit()++ yield {"a": a, "b": b, "token_a": minted["a"], "token_b": minted["b"]}++ # Deliberately does not delete the tenants.+ #+ # `audit_log` rows are immutable by design — a trigger raises on+ # DELETE — and deleting a tenant cascades into them. That is correct+ # product behaviour and this suite found it by trying: an audit trail+ # a test can erase is not an audit trail.+ #+ # Every identifier here is unique per run, and CI gets a fresh+ # container, so leaving the rows costs nothing. Users are removed+ # because they are what the assertions count.+ import sqlalchemy++ for tenant_id in (a, b):+ try:+ await db.execute(+ sqlalchemy.text("DELETE FROM users WHERE tenant_id = CAST(:t AS uuid)"),+ {"t": tenant_id},+ )+ await db.commit()+ except Exception: # noqa: BLE001, S110 — teardown is best effort+ await db.rollback()+++def _auth(token: str) -> dict[str, str]:+ return {"Authorization": f"Bearer {token}", "Content-Type": SCIM_CONTENT_TYPE}+++class TestTheNegativeControls:+ async def test_an_unauthenticated_request_is_refused(self, app_client, tenants) -> None: # noqa: ANN001+ """Without this, every assertion below could pass on a surface+ that serves everyone."""+ response = await app_client.get("/scim/v2/Users")+ assert response.status_code in (401, 403), f"SCIM served an uncredentialed caller with {response.status_code}"++ async def test_a_garbage_token_is_refused(self, app_client, tenants) -> None: # noqa: ANN001+ response = await app_client.get("/scim/v2/Users", headers=_auth("aisoc_scim_not_a_real_token"))+ assert response.status_code in (401, 403)+++class TestProvisioningAgainstRealTypes:+ async def test_a_user_created_over_scim_is_a_row(self, app_client, db, tenants) -> None: # noqa: ANN001+ """The round trip the SQLite harness cannot certify.++ `users` carries UUID and JSONB columns that the offline suite+ compiles away; here they are the real types.+ """+ email = f"ci-{uuid.uuid4().hex[:8]}@example.com"+ response = await app_client.post(+ "/scim/v2/Users",+ headers=_auth(tenants["token_a"]),+ json={+ "schemas": ["urn:ietf:params:scim:schemas:core:2.0:User"],+ "userName": email,+ "name": {"givenName": "Ada", "familyName": "Lovelace"},+ "emails": [{"value": email, "primary": True}],+ "active": True,+ },+ )+ assert response.status_code == 201, f"{response.status_code}: {response.text[:300]}"++ import sqlalchemy++ result = await db.execute(+ sqlalchemy.text("SELECT email, tenant_id FROM users WHERE email = :e"),+ {"e": email},+ )+ row = result.first()+ assert row is not None, "SCIM answered 201 and wrote no row"+ assert str(row.tenant_id) == tenants["a"], "the user was created against a tenant other than the token's"++ async def test_the_created_user_is_listed(self, app_client, tenants) -> None: # noqa: ANN001+ email = f"ci-{uuid.uuid4().hex[:8]}@example.com"+ await app_client.post(+ "/scim/v2/Users",+ headers=_auth(tenants["token_a"]),+ json={+ "schemas": ["urn:ietf:params:scim:schemas:core:2.0:User"],+ "userName": email,+ "active": True,+ },+ )+ listed = await app_client.get("/scim/v2/Users", headers=_auth(tenants["token_a"]))+ assert listed.status_code == 200+ assert email in listed.text+++class TestTenantIsolation:+ async def test_another_tenants_token_cannot_read_these_users(self, app_client, tenants) -> None: # noqa: ANN001+ """The isolation claim, through the real auth stack.++ Tenant A provisions a user; tenant B's token must not see it. A+ surface that returned everything would have passed the listing+ test above.+ """+ email = f"ci-{uuid.uuid4().hex[:8]}@example.com"+ created = await app_client.post(+ "/scim/v2/Users",+ headers=_auth(tenants["token_a"]),+ json={+ "schemas": ["urn:ietf:params:scim:schemas:core:2.0:User"],+ "userName": email,+ "active": True,+ },+ )+ assert created.status_code == 201++ theirs = await app_client.get("/scim/v2/Users", headers=_auth(tenants["token_b"]))+ assert theirs.status_code == 200+ assert email not in theirs.text, "tenant B's SCIM token listed a user provisioned by tenant A"+++class TestTheDiscoveryEndpoints:+ async def test_service_provider_config_is_served(self, app_client, tenants) -> None: # noqa: ANN001+ """An IdP reads this first. If it 404s, nothing else is reached."""+ response = await app_client.get("/scim/v2/ServiceProviderConfig", headers=_auth(tenants["token_a"]))+ assert response.status_code == 200+ assert "schemas" in response.json()++ async def test_the_user_schema_is_served(self, app_client, tenants) -> None: # noqa: ANN001+ response = await app_client.get("/scim/v2/Schemas", headers=_auth(tenants["token_a"]))+ assert response.status_code == 200diff --git a/tests/isolation/test_tenant_tuning_live.py b/tests/isolation/test_tenant_tuning_live.pynew file mode 100644index 000000000..ca6219168--- /dev/null+++ b/tests/isolation/test_tenant_tuning_live.py@@ -0,0 +1,284 @@+"""A tenant's tuning read out of Postgres and applied by the real engine.++Maturity: the evidence that takes **Per-tenant detection tuning in the+live engine** to Stable.+See `docs/audit/MATURITY_DEFINITION.md` for what the label requires.++Why a live read matters more here than almost anywhere+-------------------------------------------------------+`OverlayCache._fetch` returns `None` when it cannot read, and the caller+turns that into the empty overlay. That fail-soft is deliberate — losing+a tenant's suppressions because the database blinked would turn their+queue back on — but it means **a permanently broken overlay is+indistinguishable from a tenant who has configured no tuning**. Both show+an engine that suppresses nothing.++That is not a hypothetical failure mode. The query selected `rule_id` and+`updated_by` from `detection_rules`, which has neither; every execution+raised, the handler logged a warning, and the overlay loaded nothing on+every deployment. Thirty unit tests passed throughout, because they ran+against a fake that answered for whatever columns it was asked about.++The offline suite now has a schema check, but it regex-parses the+migration and compares it to a regex over the module's own source: text+against text, never a connection. This file opens one.++The negative control+--------------------+`test_an_untuned_tenant_still_fires` is the half that makes the rest+mean something. Every suppression assertion here would also pass against+an engine that fires nothing at all.+"""++from __future__ import annotations++import json+import os+import uuid++import pytest+import pytest_asyncio++# Skip as a *module*, not only in the fixture.+#+# The offline isolation job collects this directory with no stores+# running. With the skip only in the fixture, any test that does not take+# it ran anyway and failed there — which is a failure about the harness,+# reported against a capability.+pytestmark = [+ pytest.mark.asyncio,+ pytest.mark.skipif(+ not os.environ.get("ISOLATION_TUNING_DSN", "").strip(),+ reason="ISOLATION_TUNING_DSN is not set; this suite needs live infrastructure",+ ),+]++#: One rule, defined here rather than loaded from the 2,603-rule corpus,+#: so the test states exactly what it expects to fire. The engine under+#: test is the real one either way.+NOISY_RULE = {+ "id": "det-live-tuning-001",+ "name": "Build agent runs a shell",+ "severity": "high",+ "category": "endpoint",+ "match_when": {"host": "JENKINS-01"},+}++EVENT = {"ocsf_event": {"raw_data": json.dumps({"host": "JENKINS-01", "src_country": "RU"})}}+++def _dsn() -> str:+ value = os.environ.get("ISOLATION_TUNING_DSN", "").strip()+ if not value:+ pytest.skip("ISOLATION_TUNING_DSN is not set; this suite needs a live Postgres")+ return value+++@pytest_asyncio.fixture+async def pool():+ asyncpg = pytest.importorskip("asyncpg")+ created = await asyncpg.create_pool(_dsn(), min_size=1, max_size=4)+ yield created+ async with created.acquire() as conn:+ await conn.execute("DELETE FROM detection_rules WHERE name LIKE 'live-tuning-%'")+ await created.close()+++@pytest_asyncio.fixture+async def tenant(pool): # noqa: ANN001+ tenant_id = uuid.uuid4()+ async with pool.acquire() as conn:+ await conn.execute(+ "INSERT INTO tenants (id, name, slug) VALUES ($1, $2, $3) ON CONFLICT DO NOTHING",+ tenant_id,+ f"tuning-{tenant_id.hex[:8]}",+ f"tuning-{tenant_id.hex[:8]}",+ )+ yield str(tenant_id)+ async with pool.acquire() as conn:+ await conn.execute("DELETE FROM detection_rules WHERE tenant_id = $1", tenant_id)+ await conn.execute("DELETE FROM tenants WHERE id = $1", tenant_id)+++async def _write_tuning(+ pool,+ tenant: str,+ *,+ status: str = "active",+ suppression: dict | None = None,+ threshold: dict | None = None,+ author: str = "[email protected]",+) -> None:+ """Write tuning the way the console does: into `detection_rules`.++ The `det-*` identifier goes in `provenance->>'source_id'`, which is+ where migration 036 puts a rule's stable external id. Getting that+ wrong is precisely the defect this suite exists for.+ """+ async with pool.acquire() as conn:+ await conn.execute(+ """+ INSERT INTO detection_rules+ (id, tenant_id, name, rule_language, rule_body, category,+ status, severity, suppression_config, threshold_config,+ provenance, author)+ VALUES (gen_random_uuid(), $1::uuid, $2, 'sigma', 'x', 'endpoint',+ $3, 'high', $4::jsonb, $5::jsonb, $6::jsonb, $7)+ """,+ tenant,+ f"live-tuning-{uuid.uuid4().hex[:8]}",+ status,+ json.dumps(suppression or {}),+ json.dumps(threshold or {}),+ json.dumps({"source_id": NOISY_RULE["id"]}),+ author,+ )+++def _engine(): # noqa: ANN202+ from app.services.detection_engine import DetectionEngine++ return DetectionEngine(rules=[NOISY_RULE])+++async def _overlay(pool, tenant: str): # noqa: ANN001, ANN202+ """The production cache, reading the production query."""+ from app.services.tenant_overlay import OverlayCache++ return await OverlayCache(pool, reload_seconds=0).get(tenant)+++class TestTheQueryRunsAtAll:+ async def test_the_overlay_query_executes_against_the_real_schema(self, pool, tenant) -> None: # noqa: ANN001+ """The defect, stated as a test.++ `_fetch` selected two columns `detection_rules` does not have.+ Every execution raised, the handler turned it into a warning, and+ the overlay loaded nothing on every deployment while thirty unit+ tests passed.+ """+ from app.services.tenant_overlay import OverlayCache++ await _write_tuning(pool, tenant, status="disabled")+ rows = await OverlayCache(pool, reload_seconds=0)._fetch(tenant)++ assert rows is not None, (+ "_fetch returned None, which means the query raised and the fail-soft path "+ "swallowed it. A tenant with tuning configured would see none applied, and "+ "nothing would say so."+ )+ assert len(rows) == 1, f"expected the one tuned rule, got {rows!r}"+++class TestSuppression:+ async def test_an_untuned_tenant_still_fires(self, pool, tenant) -> None: # noqa: ANN001+ """The negative control.++ Every assertion below would also pass against an engine that+ fires nothing, which is exactly what a silently-broken overlay+ looks like.+ """+ overlay = await _overlay(pool, tenant)+ hits = _engine().evaluate(EVENT, overlay)+ assert len(hits) == 1, (+ "a rule that should fire did not, with no tuning configured. Every suppression assertion in this file is vacuous if this fails."+ )++ async def test_a_disabled_rule_stops_firing(self, pool, tenant) -> None: # noqa: ANN001+ """The product claim: a rule you turn off in the console actually+ stops producing alerts."""+ await _write_tuning(pool, tenant, status="disabled")+ overlay = await _overlay(pool, tenant)++ assert _engine().evaluate(EVENT, overlay) == [], (+ "a rule disabled in detection_rules still fired. That is the shape where the "+ "console shows it disabled and the alerts keep arriving."+ )++ async def test_a_field_suppression_stops_firing(self, pool, tenant) -> None: # noqa: ANN001+ await _write_tuning(+ pool,+ tenant,+ suppression={"suppress_when": {"host": ["JENKINS-01"]}, "reason": "build agent"},+ )+ overlay = await _overlay(pool, tenant)+ assert _engine().evaluate(EVENT, overlay) == []++ async def test_a_suppression_for_a_different_value_does_not(self, pool, tenant) -> None: # noqa: ANN001+ """Suppression has to be specific, or it is just an off switch."""+ await _write_tuning(pool, tenant, suppression={"suppress_when": {"host": ["SOME-OTHER-HOST"]}})+ overlay = await _overlay(pool, tenant)+ assert len(_engine().evaluate(EVENT, overlay)) == 1++ async def test_the_author_and_reason_travel_with_the_suppression(self, pool, tenant) -> None: # noqa: ANN001+ """A dropped alert with no explanation is indistinguishable from a+ rule that did not match, and an analyst asking why they stopped+ seeing something deserves an answer."""+ await _write_tuning(+ pool,+ tenant,+ status="disabled",+ suppression={"reason": "known noisy on build agents"},+ author="[email protected]",+ )+ overlay = await _overlay(pool, tenant)+ why = overlay.suppresses(NOISY_RULE["id"], {"host": "JENKINS-01"})++ assert why, "the overlay suppressed without recording why"+ assert "[email protected]" in why+ assert "build agents" in why+++class TestTheSeverityFloor:+ async def test_a_match_below_the_floor_is_held(self, pool, tenant) -> None: # noqa: ANN001+ await _write_tuning(pool, tenant, threshold={"min_severity": "critical"})+ overlay = await _overlay(pool, tenant)+ assert _engine().evaluate(EVENT, overlay) == [], "a high-severity match fired against a critical floor"++ async def test_a_match_at_the_floor_still_fires(self, pool, tenant) -> None: # noqa: ANN001+ await _write_tuning(pool, tenant, threshold={"min_severity": "high"})+ overlay = await _overlay(pool, tenant)+ assert len(_engine().evaluate(EVENT, overlay)) == 1+++class TestTenantIsolation:+ async def test_one_tenants_tuning_does_not_silence_another(self, pool, tenant) -> None: # noqa: ANN001+ """The plan's own done-when, against two real tenant rows."""+ other = uuid.uuid4()+ async with pool.acquire() as conn:+ await conn.execute(+ "INSERT INTO tenants (id, name, slug) VALUES ($1, $2, $3) ON CONFLICT DO NOTHING",+ other,+ f"tuning-b-{other.hex[:8]}",+ f"tuning-b-{other.hex[:8]}",+ )+ try:+ await _write_tuning(pool, tenant, status="disabled")++ theirs = await _overlay(pool, str(other))+ assert len(_engine().evaluate(EVENT, theirs)) == 1, "tenant A's disable silenced the rule for tenant B"++ mine = await _overlay(pool, tenant)+ assert _engine().evaluate(EVENT, mine) == []+ finally:+ async with pool.acquire() as conn:+ await conn.execute("DELETE FROM detection_rules WHERE tenant_id = $1", other)+ await conn.execute("DELETE FROM tenants WHERE id = $1", other)+++class TestTheFailSoftIsVisible:+ async def test_an_unreachable_database_yields_the_empty_overlay(self, pool, tenant) -> None: # noqa: ANN001+ """Documenting the behaviour that makes this whole file necessary.++ A broken read produces an overlay that suppresses nothing, which+ on its own is indistinguishable from a tenant with no tuning. The+ fail-soft is the right choice; the live test above is what stops+ it hiding a permanent break.+ """+ from app.services.tenant_overlay import EMPTY, OverlayCache++ class _Unreachable:+ def acquire(self): # noqa: ANN202+ raise RuntimeError("database is gone")++ assert await OverlayCache(_Unreachable(), reload_seconds=0).get(tenant) is EMPTYdiff --git a/tests/isolation/test_ueba_live.py b/tests/isolation/test_ueba_live.pynew file mode 100644index 000000000..c2c162393--- /dev/null+++ b/tests/isolation/test_ueba_live.py@@ -0,0 +1,243 @@+"""UEBA scoring and persistence, against a live Postgres.++Maturity: the evidence that takes **UEBA** to Stable.+See `docs/audit/MATURITY_DEFINITION.md` for what the label requires.++Why a live database is the only way to test this+-------------------------------------------------+UEBA's 88 unit tests never open a connection. They are good tests of the+arithmetic — z-scores, composite weighting, peer comparison — and+structurally incapable of catching the defect that actually shipped:++`EntityBaseline` declared `peer_group_id`, migration `0001` never created+the column, and `score_event` raised on the first scoreable event. UEBA+could not write an anomaly on any deployment. Every unit test passed+throughout, because the ORM model and the test agreed with each other and+neither consulted the schema.++So this suite runs the migrations the service ships, then drives+`ScoringService.score_event` through a real session and reads the row back+out. A column the model declares and the migration omits fails here on the+first call.++The negative control+--------------------+`test_a_normal_event_produces_no_anomaly` is the half that makes the other+half mean something. A scorer that flagged everything would pass an+"anomalous event produces a row" assertion while being useless, and a+scorer that flagged nothing would pass a tenant-isolation assertion for+the wrong reason.+"""++from __future__ import annotations++import os+import uuid++import pytest+import pytest_asyncio++# Skip as a *module*, not only in the fixture.+#+# The offline isolation job collects this directory with no stores+# running. With the skip only in the fixture, any test that does not take+# it ran anyway and failed there — which is a failure about the harness,+# reported against a capability.+pytestmark = [+ pytest.mark.asyncio,+ pytest.mark.skipif(+ not os.environ.get("ISOLATION_UEBA_DSN", "").strip(),+ reason="ISOLATION_UEBA_DSN is not set; this suite needs live infrastructure",+ ),+]++TENANT_A = uuid.UUID("11111111-1111-1111-1111-111111111111")+TENANT_B = uuid.UUID("22222222-2222-2222-2222-222222222222")+++def _dsn() -> str:+ value = os.environ.get("ISOLATION_UEBA_DSN", "").strip()+ if not value:+ pytest.skip("ISOLATION_UEBA_DSN is not set; this suite needs a live Postgres")+ return value+++@pytest_asyncio.fixture+async def session():+ """A session against the schema the service's own migrations build.++ The tables come from `services/ueba/alembic/`, not from+ `Base.metadata.create_all`. That distinction is the entire point: the+ defect this suite exists for was a column present in the model and+ absent from the migration, and `create_all` would have papered over it+ by building the schema from the very model under test.+ """+ sqlalchemy = pytest.importorskip("sqlalchemy")+ from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine++ dsn = _dsn()+ engine = create_async_engine(dsn, future=True)+ maker = async_sessionmaker(engine, expire_on_commit=False)++ async with maker() as s:+ yield s+ # Scoped to the two test tenants so a shared database is not+ # emptied by a test run.+ for table in ("ueba_anomalies", "ueba_entity_baselines", "ueba_peer_groups"):+ await s.execute(+ sqlalchemy.text(f"DELETE FROM {table} WHERE tenant_id IN (:a, :b)"),+ {"a": str(TENANT_A), "b": str(TENANT_B)},+ )+ await s.commit()+ await engine.dispose()+++def _scorer(session): # noqa: ANN001, ANN202+ from app.services.scoring import ScoringService++ return ScoringService(session)+++async def _train(scorer, tenant: uuid.UUID, entity: str, *, value: float, times: int = 40) -> None:+ """Give an entity a baseline by showing it ordinary behaviour.++ Real repetition rather than an inserted baseline row, so the training+ path is exercised too — a baseline writer that silently failed would+ otherwise look identical to one that worked.++ The values vary by a few percent on purpose. Feeding the identical+ number produces a baseline with **zero variance**, which the scorer+ correctly declines to score by deviation: its own log says so, naming+ automation and service accounts as the entities that converge there.+ A fixture that trained on a constant would be testing the+ unscoreable branch while claiming to test detection.+ """+ for index in range(times):+ # Deterministic jitter: a seeded random would make a failure hard+ # to reproduce from the test name alone.+ jitter = 1.0 + ((index % 7) - 3) * 0.04+ await scorer.score_event(+ tenant_id=tenant,+ entity_type="user",+ entity_id=entity,+ event_type="login",+ features={"bytes_out": value * jitter},+ )+++class TestTheSchemaIsTheOneTheServiceShips:+ async def test_the_baseline_table_has_every_column_the_model_declares(self, session) -> None: # noqa: ANN001+ """The exact defect: `peer_group_id` was in the model and not in+ migration 0001, so `score_event` raised on the first scoreable+ event and UEBA could not write an anomaly on any deployment."""+ import sqlalchemy+ from app.models.ueba import EntityBaseline++ result = await session.execute(+ sqlalchemy.text("SELECT column_name FROM information_schema.columns WHERE table_name = 'ueba_entity_baselines'")+ )+ actual = {row[0] for row in result}+ declared = {c.name for c in EntityBaseline.__table__.columns}++ missing = declared - actual+ assert not missing, (+ f"the model declares {sorted(missing)} and the migrations do not create "+ "them. Every query touching those columns raises, which is how UEBA "+ "shipped unable to write a single anomaly."+ )+++class TestScoring:+ async def test_a_normal_event_produces_no_anomaly(self, session) -> None: # noqa: ANN001+ """The negative control.++ Without it, 'an anomalous event produces a row' would pass for a+ scorer that flags everything.+ """+ scorer = _scorer(session)+ entity = f"alice-{uuid.uuid4().hex[:8]}"+ await _train(scorer, TENANT_A, entity, value=1000.0)++ anomaly = await scorer.score_event(+ tenant_id=TENANT_A,+ entity_type="user",+ entity_id=entity,+ event_type="login",+ features={"bytes_out": 1010.0},+ )+ assert anomaly is None, f"ordinary behaviour was scored anomalous ({anomaly!r}); a detector that flags everything detects nothing"++ async def test_an_outlier_produces_a_persisted_anomaly(self, session) -> None: # noqa: ANN001+ """The positive half, read back out of the database rather than+ taken from the return value."""+ import sqlalchemy++ scorer = _scorer(session)+ entity = f"bob-{uuid.uuid4().hex[:8]}"+ await _train(scorer, TENANT_A, entity, value=1000.0)++ anomaly = await scorer.score_event(+ tenant_id=TENANT_A,+ entity_type="user",+ entity_id=entity,+ event_type="login",+ features={"bytes_out": 500_000.0},+ )+ assert anomaly is not None, "a 500x outlier against a trained baseline was not flagged"+ await session.commit()++ result = await session.execute(+ sqlalchemy.text("SELECT count(*) FROM ueba_anomalies WHERE tenant_id = :t AND entity_id = :e"),+ {"t": str(TENANT_A), "e": entity},+ )+ assert result.scalar() >= 1, (+ "score_event returned an anomaly that is not in the table; the return value and the persisted row are different claims"+ )+++class TestTenantIsolation:+ async def test_one_tenants_baseline_does_not_train_another(self, session) -> None: # noqa: ANN001+ """Two tenants, the same entity id, opposite normal behaviour.++ If baselines leaked, B's ordinary value would read as ordinary for+ A as well and the assertion below would fail.+ """+ scorer = _scorer(session)+ shared = f"svc-{uuid.uuid4().hex[:8]}"++ await _train(scorer, TENANT_A, shared, value=1000.0)+ await _train(scorer, TENANT_B, shared, value=500_000.0)++ # B's normal is A's outlier. A must still flag it.+ flagged = await scorer.score_event(+ tenant_id=TENANT_A,+ entity_type="user",+ entity_id=shared,+ event_type="login",+ features={"bytes_out": 500_000.0},+ )+ assert flagged is not None, (+ "tenant B's baseline trained tenant A: a value B does normally read as "+ "ordinary for A, which means the two estates share a model"+ )++ async def test_an_anomaly_is_written_against_its_own_tenant(self, session) -> None: # noqa: ANN001+ import sqlalchemy++ scorer = _scorer(session)+ entity = f"carol-{uuid.uuid4().hex[:8]}"+ await _train(scorer, TENANT_A, entity, value=1000.0)+ await scorer.score_event(+ tenant_id=TENANT_A,+ entity_type="user",+ entity_id=entity,+ event_type="login",+ features={"bytes_out": 500_000.0},+ )+ await session.commit()++ leaked = await session.execute(+ sqlalchemy.text("SELECT count(*) FROM ueba_anomalies WHERE tenant_id = :t AND entity_id = :e"),+ {"t": str(TENANT_B), "e": entity},+ )+ assert leaked.scalar() == 0, "an anomaly scored for tenant A was written against tenant B"