Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

fix(deploy): generate and deliver the credential eight modules already read (#974)

Beenu Arorad8eca2bbad83
diff --git a/.env.example b/.env.example
index 54bfec002..807454972 100644
--- a/.env.example
+++ b/.env.example
@@ -398,13 +398,43 @@ AISOC_SERVICE_TOKEN=
# service answers 503 on every mutating route, which is deliberate: it used to
# answer 200 to anyone who could reach the port.
AISOC_ACTIONS_SERVICE_TOKEN=
+#
+# The agents service gets its own too, and it is the one that could not fall
+# back: eight modules in `services/agents` present it as
+# `X-AiSOC-Service-Token` on the API's internal context routes, and
+# `service_token_valid` on the API verifies THIS variable and no other — it
+# does not accept AISOC_SERVICE_TOKEN. Nothing generated it and no manifest
+# passed it, so organisation memory, tenant skills, recent dispositions,
+# identity context, knowledge-base runbooks, the MCP toolset, the playbook
+# action bridge and SIEM writeback were all off on every default install, each
+# logging a `no_service_token` warning and returning an empty result that
+# reads exactly like "this tenant has none".
+#
+# The same value is verified in the other direction: the API posts a replay to
+# the agents service as a bearer, and `app.security.tenant_scope` resolves
+# `AISOC_AGENTS_SERVICE_TOKEN` ahead of the shared one. `make up` generates it
+# and compose passes it to both.
+AISOC_AGENTS_SERVICE_TOKEN=
# Bearer for realtime's /internal/* fan-out routes (API/agents -> realtime),
# sent as the x-internal-token header. `make up` generates it. The guard used
# to treat an unset value as authorized, and nothing set it.
REALTIME_INTERNAL_TOKEN=
-# Optional per-service overrides, if you'd rather not share one secret:
+# Optional per-service overrides, if you'd rather not share one secret. Each
+# is read ahead of AISOC_SERVICE_TOKEN by the service it names, on both the
+# presenting and the verifying side, so setting one narrows a single hop.
+# `scripts/check_service_token_wiring.py` asserts every one of these that
+# production code reads is listed here — four were not, which is how an
+# operator would go looking for a variable the code uses and find nothing.
# AISOC_PURPLE_TEAM_SERVICE_TOKEN=
# AISOC_HONEYTOKENS_SERVICE_TOKEN=
# AISOC_UEBA_SERVICE_TOKEN=
# AISOC_INGEST_SERVICE_TOKEN=
+# AISOC_CONNECTORS_SERVICE_TOKEN=
+# AISOC_FUSION_SERVICE_TOKEN=
+# AISOC_THREATINTEL_SERVICE_TOKEN=
+#
+# Bearer the agents service presents to the ChatOps fan-out (slack-bot).
+# Unset means no ChatOps notification is sent; the approval is still durable
+# and visible in the console.
+# AISOC_INTERNAL_TOKEN=
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index ea030fbb0..8711aec74 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -715,6 +715,18 @@ jobs:
run: |
python3 scripts/check_failure_is_not_a_verdict.py --self-test
python3 scripts/check_failure_is_not_a_verdict.py
+ # `AISOC_AGENTS_SERVICE_TOKEN` was read by eight modules in
+ # services/agents and one in services/api, verified by a function that
+ # reads it and nothing else, and produced by nothing — not the
+ # generator, not `.env.example`, not one compose manifest. Organisation
+ # memory, tenant skills, dispositions, identity context, runbooks, the
+ # MCP toolset, the action bridge and SIEM writeback were all off on
+ # every default install, each logging its own warning and returning an
+ # empty result that reads like "this tenant has none".
+ - name: Service credentials production code reads are produced by first run
+ run: |
+ python3 scripts/check_service_token_wiring.py --self-test
+ python3 scripts/check_service_token_wiring.py
# The marketplace index has to exist in three build contexts, none of
# which can see the other two. The API's copy was missing, so its image
# shipped no catalogue and the marketplace answered 503 on every
diff --git a/.github/workflows/compose-smoke.yml b/.github/workflows/compose-smoke.yml
index 562166fc0..c122c035e 100644
--- a/.github/workflows/compose-smoke.yml
+++ b/.github/workflows/compose-smoke.yml
@@ -202,16 +202,17 @@ jobs:
COMPOSE_PROJECT_NAME: aisoc-prodguard
run: |
set -euo pipefail
- real() { openssl rand -hex 24; }
+ # Derived from the compose file, not listed here. A hand-written
+ # list is a second copy of the guards, and the first `${VAR:?}`
+ # added after it was written broke this step with
+ # `required variable AISOC_AGENTS_SERVICE_TOKEN is missing a value`.
{
- for name in POSTGRES_PASSWORD AISOC_APP_DB_PASSWORD REDIS_PASSWORD SECRET_KEY \
- AISOC_CREDENTIAL_KEY AISOC_SERVICE_TOKEN AISOC_ACTIONS_SERVICE_TOKEN \
- AISOC_REALTIME_JWT_SECRET REALTIME_INTERNAL_TOKEN METRICS_TOKEN \
- JWT_SECRET NEO4J_PASSWORD CLICKHOUSE_PASSWORD GRAFANA_ADMIN_PASSWORD; do
- printf '%s=%s\n' "$name" "$(real)"
- done
+ python3 scripts/check_published_secrets.py --list-guarded \
+ | while read -r name; do printf '%s=%s\n' "$name" "$(openssl rand -hex 24)"; done
printf 'AISOC_VERSION=%s\n' "${AISOC_VERSION}"
} > /tmp/prod-real.env
+ test "$(wc -l < /tmp/prod-real.env)" -gt 10 \
+ || { echo "::error::no guarded variables were derived"; exit 1; }
sed 's|^POSTGRES_PASSWORD=.*|POSTGRES_PASSWORD=aisoc_dev_secret|' /tmp/prod-real.env > /tmp/prod-published.env
echo "::group::published credential — must refuse"
diff --git a/README.md b/README.md
index 6f97af9a0..ed8096954 100644
--- a/README.md
+++ b/README.md
@@ -57,8 +57,8 @@ Needs Docker Compose v2 with **8 GB memory and 20 GB free disk in the Docker VM*
was measured against. The first run downloads a ~2 GB language model into a named volume; only
`make clean` fetches it again.
-`make up` also creates `.env` and generates the **ten** secrets in it — the credential vault, the
-session signing key, the four service-to-service credentials and the four datastore passwords — then
+`make up` also creates `.env` and generates the **eleven** secrets in it — the credential vault,
+the session signing key, the five service-to-service credentials and the four datastore passwords — then
creates an administrator and prints its password. That password is generated on your machine, shown once, and stored nowhere:
copy it, or mint a new one with `make bootstrap ARGS=--reset-password`.
diff --git a/apps/docs/docs/deployment/docker.md b/apps/docs/docs/deployment/docker.md
index a67873799..81500d969 100644
--- a/apps/docs/docs/deployment/docker.md
+++ b/apps/docs/docs/deployment/docker.md
@@ -117,7 +117,7 @@ COMPOSE_PROFILES=connectors,osquery,slack docker compose up -d
## Production
```bash
-make env # generates the ten secrets the stack needs
+make env # generates the eleven secrets the stack needs
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
```
diff --git a/docker-compose.prod.yml b/docker-compose.prod.yml
index 76c7501ab..4c4de2ac8 100644
--- a/docker-compose.prod.yml
+++ b/docker-compose.prod.yml
@@ -93,6 +93,7 @@ services:
AISOC_CREDENTIAL_KEY: ${AISOC_CREDENTIAL_KEY:?set AISOC_CREDENTIAL_KEY — run `make env` to generate it}
AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:?set AISOC_SERVICE_TOKEN — run `make env` to generate it}
AISOC_ACTIONS_SERVICE_TOKEN: ${AISOC_ACTIONS_SERVICE_TOKEN:?set AISOC_ACTIONS_SERVICE_TOKEN — run `make env` to generate it}
+ AISOC_AGENTS_SERVICE_TOKEN: ${AISOC_AGENTS_SERVICE_TOKEN:?set AISOC_AGENTS_SERVICE_TOKEN — run `make env` to generate it}
AISOC_REALTIME_JWT_SECRET: ${AISOC_REALTIME_JWT_SECRET:?set AISOC_REALTIME_JWT_SECRET — run `make env` to generate it}
REALTIME_INTERNAL_TOKEN: ${REALTIME_INTERNAL_TOKEN:?set REALTIME_INTERNAL_TOKEN — run `make env` to generate it}
METRICS_TOKEN: ${METRICS_TOKEN:?set METRICS_TOKEN — guards /metrics outside development; `openssl rand -hex 32`}
@@ -176,6 +177,11 @@ services:
AISOC_CREDENTIAL_KEY: ${AISOC_CREDENTIAL_KEY:?set AISOC_CREDENTIAL_KEY — run `make env` to generate it}
AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:?set AISOC_SERVICE_TOKEN — run `make env` to generate it}
AISOC_ACTIONS_SERVICE_TOKEN: ${AISOC_ACTIONS_SERVICE_TOKEN:?set AISOC_ACTIONS_SERVICE_TOKEN — run `make env` to generate it}
+ # Verified by `alert_writeback.service_token_valid`, which reads this
+ # and only this. Unset, every internal context route the agents service
+ # calls refuses it, and triage silently runs without organisation
+ # memory, tenant skills, dispositions, identity or runbooks.
+ AISOC_AGENTS_SERVICE_TOKEN: ${AISOC_AGENTS_SERVICE_TOKEN:?set AISOC_AGENTS_SERVICE_TOKEN — run `make env` to generate it}
AISOC_REALTIME_JWT_SECRET: ${AISOC_REALTIME_JWT_SECRET:?set AISOC_REALTIME_JWT_SECRET — run `make env` to generate it}
REALTIME_INTERNAL_TOKEN: ${REALTIME_INTERNAL_TOKEN:?set REALTIME_INTERNAL_TOKEN — run `make env` to generate it}
# The remaining two secrets `enforce_secure_defaults` hard-fails on, and
@@ -244,6 +250,9 @@ services:
AISOC_DEV_MODE: 0
AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:?set AISOC_SERVICE_TOKEN}
AISOC_ACTIONS_SERVICE_TOKEN: ${AISOC_ACTIONS_SERVICE_TOKEN:?set AISOC_ACTIONS_SERVICE_TOKEN}
+ # The same value the api service declares above. Eight modules here
+ # present it; the API verifies it without falling back.
+ AISOC_AGENTS_SERVICE_TOKEN: ${AISOC_AGENTS_SERVICE_TOKEN:?set AISOC_AGENTS_SERVICE_TOKEN}
NEO4J_PASSWORD: ${NEO4J_PASSWORD:?set NEO4J_PASSWORD}
actions:
diff --git a/docker-compose.yml b/docker-compose.yml
index 63d154574..809e8effb 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -417,6 +417,15 @@ services:
# variable shipped against a variable nothing ever set.
AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:-}
AISOC_CONNECTORS_SERVICE_TOKEN: ${AISOC_CONNECTORS_SERVICE_TOKEN:-}
+ # The agents service's credential, on the verifying side. `alert_writeback
+ # .service_token_valid` reads this and only this — it does NOT fall back
+ # to AISOC_SERVICE_TOKEN — so the internal context routes the agents
+ # service calls (`/feedback/context-statements`,
+ # `/tenant-skills/resolved/active`, `/kb/runbooks/for-triage`,
+ # `/feedback/recent-dispositions`, `/graph/identity-context`, the MCP
+ # registry and `/alerts/{id}/source-writeback`) refuse every caller until
+ # it is set. Must match the value on `agents`.
+ AISOC_AGENTS_SERVICE_TOKEN: ${AISOC_AGENTS_SERVICE_TOKEN:-}
CONNECTORS_SERVICE_URL: ${CONNECTORS_SERVICE_URL:-http://connectors:8003}
# Where GET /api/v1/threat-intel/indicators is served from. The console
# has called that path since the IOC inbox shipped and nothing answered
@@ -576,6 +585,13 @@ services:
environment:
KAFKA_BOOTSTRAP_SERVERS: kafka:29092
REDIS_URL: redis://:${REDIS_PASSWORD:-redis_dev_secret}@redis:6379/3
+ # `tenant_scope` refuses outright when neither a console secret nor a
+ # service token is configured and dev mode is off, and this service
+ # received none of the three — the same trio `connectors` and
+ # `threatintel` already carry.
+ AISOC_DEV_MODE: ${AISOC_DEV_MODE:-1}
+ SECRET_KEY: ${SECRET_KEY:-}
+ AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:-}
DATABASE_URL: postgresql+asyncpg://aisoc_app:${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}@postgres:5432/aisoc
# The ClickHouse event lake is a `full`-profile capability: it backs
# /lake/sql and the hunt surface, not alerting. CORE runs without it and
@@ -648,6 +664,31 @@ services:
# service, so this only resolves when that profile is up.
AISOC_SLACK_BOT_URL: ${AISOC_SLACK_BOT_URL:-}
AISOC_INTERNAL_TOKEN: ${AISOC_INTERNAL_TOKEN:-}
+ # Bearer for realtime's /internal/* fan-out. `app/playbook/engine.py`
+ # reads it to publish step progress; unset, realtime refuses the call
+ # and a running playbook is invisible in the console.
+ REALTIME_INTERNAL_TOKEN: ${REALTIME_INTERNAL_TOKEN:-}
+ # The three below are the same trio `connectors` and `threatintel`
+ # already carry, and this service had none of them. `tenant_scope`
+ # refuses outright when neither a console secret nor a service token is
+ # configured *and* dev mode is off — which is what every tenant-scoped
+ # route here answered: 503, with no credential material to verify
+ # against and nothing in the image supplying any.
+ AISOC_DEV_MODE: ${AISOC_DEV_MODE:-1}
+ SECRET_KEY: ${SECRET_KEY:-}
+ AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:-}
+ # The agents service's own service credential, used in both directions:
+ # presented as `X-AiSOC-Service-Token` on the API's internal context
+ # routes, and verified by `app.security.tenant_scope` when the API calls
+ # back (it resolves `AISOC_AGENTS_SERVICE_TOKEN` before the shared one).
+ #
+ # Eight modules in this service read it and nothing produced it, so on a
+ # default install organisation memory, tenant skills, recent
+ # dispositions, identity context, knowledge-base runbooks, the MCP
+ # toolset, the playbook action bridge and SIEM writeback were all off —
+ # each logging its own `no_service_token` warning and returning empty.
+ # `make env` generates it; it must match the value on `api` below.
+ AISOC_AGENTS_SERVICE_TOKEN: ${AISOC_AGENTS_SERVICE_TOKEN:-}
networks:
- aisoc
restart: unless-stopped
@@ -816,6 +857,13 @@ services:
# DATABASE_URL and the role's actual credential come from one value.
AISOC_APP_DB_PASSWORD: ${AISOC_APP_DB_PASSWORD:-aisoc_app_dev_secret}
AISOC_OSQUERY_TLS_ENROLL_SECRET: ${AISOC_OSQUERY_TLS_ENROLL_SECRET:-change-me-in-production}
+ # `tenant_scope` refuses outright when neither a console secret nor a
+ # service token is configured and dev mode is off, and this service
+ # received none of the three — the same trio `connectors` and
+ # `threatintel` already carry.
+ AISOC_DEV_MODE: ${AISOC_DEV_MODE:-1}
+ SECRET_KEY: ${SECRET_KEY:-}
+ AISOC_SERVICE_TOKEN: ${AISOC_SERVICE_TOKEN:-}
# The ingest service is registered as `ingest-worker` in this compose
# file; the previous `http://ingest:8080` would not resolve via DNS.
AISOC_INGEST_BASE_URL: http://ingest-worker:8080
diff --git a/scripts/check_published_secrets.py b/scripts/check_published_secrets.py
index f024beb48..b4e0d9972 100644
--- a/scripts/check_published_secrets.py
+++ b/scripts/check_published_secrets.py
@@ -369,12 +369,25 @@ def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--check-env", action="store_true", help="inspect this process's environment")
parser.add_argument("--check", action="store_true", help="verdict over the repository")
+ parser.add_argument(
+ "--list-guarded",
+ action="store_true",
+ help="print every `${VAR:?}` variable in the production compose file, one per line",
+ )
args = parser.parse_args(argv)
if args.check_env:
return _check_env()
if args.check:
return _check_repo()
- parser.error("choose --check-env or --check")
+ if args.list_guarded:
+ # For the compose-smoke step that has to supply a value for every
+ # guarded variable. Hand-listing them meant the next `${VAR:?}` added
+ # to that file broke the smoke run — which is what happened the first
+ # time one was, and is a list maintained in two places by definition.
+ for name in sorted(guarded_variables(repo_root() / "docker-compose.prod.yml")):
+ print(name)
+ return 0
+ parser.error("choose --check-env, --check or --list-guarded")
return 2
diff --git a/scripts/check_service_token_wiring.py b/scripts/check_service_token_wiring.py
new file mode 100644
index 000000000..764a040ff
--- /dev/null
+++ b/scripts/check_service_token_wiring.py
@@ -0,0 +1,273 @@
+#!/usr/bin/env python3
+"""A service credential read by production code must be produced by first run.
+
+The hole this closes
+--------------------
+``AISOC_AGENTS_SERVICE_TOKEN`` was read by eight modules in ``services/agents``
+and one in ``services/api``, verified by ``alert_writeback.service_token_valid``
+— which reads that variable and no other, with no fallback to the shared
+``AISOC_SERVICE_TOKEN`` — and produced by **nothing**. It was absent from
+``scripts/ensure_env.py``, from ``.env.example`` and from every compose
+manifest. So on every default install:
+
+* organisation memory (``/feedback/context-statements``)
+* tenant skills (``/tenant-skills/resolved/active``)
+* recent analyst dispositions (``/feedback/recent-dispositions``)
+* identity context (``/graph/identity-context``)
+* knowledge-base runbooks (``/kb/runbooks/for-triage``)
+* the MCP toolset, the playbook action bridge and SIEM writeback
+
+were off. Each reader logs its own ``no_service_token`` warning and returns an
+empty result, and an empty result is indistinguishable from "this tenant has
+none" at every surface above it.
+
+Nothing failed. That is the shape worth gating: a credential whose absence
+disables a capability quietly is exactly the credential nobody notices is
+missing.
+
+What is asserted
+----------------
+For every ``AISOC_*_SERVICE_TOKEN`` / ``*_INTERNAL_TOKEN`` variable that
+production code under ``services/*/app/`` reads:
+
+1. ``scripts/ensure_env.py`` generates it, or it is declared below as
+ deliberately operator-supplied with a reason.
+2. ``.env.example`` lists it, so an operator reading the template knows it
+ exists.
+3. Every compose service built from a source tree that reads it receives it
+ in ``docker-compose.yml``. The source-tree-to-service mapping is derived
+ from each service's ``build.context``, not hand-written, because a
+ hand-written map is the thing that stops matching.
+
+The direction that drifts is code gaining a reader, so that is the direction
+this runs in: a new ``os.getenv("AISOC_FOO_SERVICE_TOKEN")`` fails here until
+first run produces it and compose delivers it.
+"""
+
+from __future__ import annotations
+
+import argparse
+import ast
+import re
+import sys
+from pathlib import Path
+
+# `scripts/` is on sys.path when this file is run as a program, but not when a
+# test loads it by path with importlib. gate_toolkit sits beside it either way.
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+
+from gate_toolkit import repo_root, self_test_if_requested
+
+self_test_if_requested(__file__)
+
+#: The credential family. A shared secret two AiSOC processes present to each
+#: other — as opposed to a vendor API key, which an operator supplies and no
+#: amount of first-run generation can invent.
+TOKEN_NAME_RE = re.compile(r"^[A-Z0-9_]*(?:SERVICE_TOKEN|INTERNAL_TOKEN)$")
+
+#: Variables in the family that first run deliberately does not generate.
+#: Each is a *narrowing override* of a token that is generated: setting one
+#: replaces the shared secret for a single hop, which is an operator's choice
+#: and meaningless to invent a value for.
+#: Only variables a literal ``os.getenv("NAME")`` reads are listed. The
+#: per-service overrides resolved through an f-string —
+#: ``os.getenv(f"AISOC_{SERVICE_NAME}_SERVICE_TOKEN")`` in each service's
+#: ``tenant_scope`` — are deliberately out of the literal scan's reach and
+#: are not listed here: a row nothing reads is checked below and removed.
+OPERATOR_SUPPLIED: dict[str, str] = {
+ "AISOC_CONNECTORS_SERVICE_TOKEN": "per-service override for AISOC_SERVICE_TOKEN",
+ "AISOC_THREATINTEL_SERVICE_TOKEN": "per-service override for AISOC_SERVICE_TOKEN",
+ "AISOC_FUSION_SERVICE_TOKEN": "per-service override for AISOC_SERVICE_TOKEN",
+ "AISOC_INGEST_SERVICE_TOKEN": "per-service override for AISOC_SERVICE_TOKEN",
+ "AISOC_INTERNAL_TOKEN": "bearer for the opt-in chatops fan-out; unset means no ChatOps notification",
+}
+
+#: Service source trees whose compose delivery is not asserted, with the
+#: reason. Kept as a named list so an exemption is a visible edit.
+UNMAPPED_SOURCE_DIRS: dict[str, str] = {}
+
+#: ``variable -> the per-service override that satisfies it``.
+#:
+#: Each service's ``tenant_scope.resolve_service_token`` prefers
+#: ``AISOC_<SERVICE>_SERVICE_TOKEN`` and falls back to the shared
+#: ``AISOC_SERVICE_TOKEN``, so delivering the override is delivering the
+#: credential. Without this the gate would demand the shared secret on a
+#: service that deliberately holds its own — `actions`, which can isolate a
+#: host, is not supposed to share one with the read-mostly services.
+SATISFIED_BY_OVERRIDE: dict[str, str] = {
+ "AISOC_SERVICE_TOKEN": "AISOC_{SERVICE}_SERVICE_TOKEN",
+}
+
+
+def _readers(root: Path) -> dict[str, set[str]]:
+ """``variable -> {service source directory}``, from the AST.
+
+ AST rather than a regex over the text so a variable named in a docstring
+ explaining the defect is not counted as a reader — the mistake this
+ repository has made twice with text scans over compose files.
+ """
+ found: dict[str, set[str]] = {}
+ services = root / "services"
+ if not services.is_dir():
+ return found
+ for path in sorted(services.glob("*/app/**/*.py")):
+ if "test" in path.parts:
+ continue
+ try:
+ tree = ast.parse(path.read_text(encoding="utf-8"))
+ except SyntaxError:
+ continue
+ service_dir = path.relative_to(services).parts[0]
+ for node in ast.walk(tree):
+ if not isinstance(node, ast.Call):
+ continue
+ func = node.func
+ is_getenv = (isinstance(func, ast.Attribute) and func.attr == "getenv") or (isinstance(func, ast.Name) and func.id == "getenv")
+ if not is_getenv or not node.args:
+ continue
+ first = node.args[0]
+ if isinstance(first, ast.Constant) and isinstance(first.value, str) and TOKEN_NAME_RE.match(first.value):
+ found.setdefault(first.value, set()).add(service_dir)
+ return found
+
+
+def _generated(root: Path) -> set[str]:
+ """Keys of ``ensure_env.GENERATED``, read by AST.
+
+ Importing would run the module, which is cheap but would also mean this
+ gate could not report on a tree whose ``ensure_env.py`` does not import.
+ """
+ path = root / "scripts" / "ensure_env.py"
+ if not path.exists():
+ return set()
+ tree = ast.parse(path.read_text(encoding="utf-8"))
+ for node in ast.walk(tree):
+ # Carried alongside the name rather than read back off `node`, which
+ # `ast.walk` types as the base `AST`: the isinstance checks above narrow
+ # inside their own branch and not after the if/elif closes.
+ target: str | None = None
+ assigned: ast.expr | None = None
+ if isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name):
+ target = node.target.id
+ assigned = node.value
+ elif isinstance(node, ast.Assign) and len(node.targets) == 1 and isinstance(node.targets[0], ast.Name):
+ target = node.targets[0].id
+ assigned = node.value
+ if target == "GENERATED" and isinstance(assigned, ast.Dict):
+ return {k.value for k in assigned.keys if isinstance(k, ast.Constant) and isinstance(k.value, str)}
+ return set()
+
+
+def _env_example_names(root: Path) -> set[str]:
+ path = root / ".env.example"
+ if not path.exists():
+ return set()
+ names = set()
+ for raw in path.read_text(encoding="utf-8").splitlines():
+ line = raw.strip().lstrip("#").strip()
+ name, sep, _ = line.partition("=")
+ if sep and re.fullmatch(r"[A-Z][A-Z0-9_]*", name.strip()):
+ names.add(name.strip())
+ return names
+
+
+def _compose_delivery(root: Path) -> tuple[dict[str, set[str]], dict[str, set[str]]]:
+ """``(source dir -> compose services, compose service -> variables it gets)``.
+
+ The first map comes from each service's ``build.context``, so adding a
+ second compose service for one source tree is covered without an edit
+ here.
+ """
+ import yaml # noqa: PLC0415 — the gate needs a YAML parser; the readers scan do not
+
+ services = (yaml.safe_load((root / "docker-compose.yml").read_text(encoding="utf-8")) or {}).get("services") or {}
+ by_source: dict[str, set[str]] = {}
+ delivered: dict[str, set[str]] = {}
+ for name, service in services.items():
+ build = (service or {}).get("build")
+ context = build.get("context") if isinstance(build, dict) else build
+ if isinstance(context, str) and context.startswith("./services/"):
+ by_source.setdefault(context.removeprefix("./services/").strip("/"), set()).add(name)
+ environment = (service or {}).get("environment") or {}
+ keys = environment.keys() if isinstance(environment, dict) else [str(e).partition("=")[0] for e in environment]
+ delivered[name] = set(keys)
+ return by_source, delivered
+
+
+def check(root: Path | None = None) -> list[str]:
+ root = root or repo_root()
+ readers = _readers(root)
+ if not readers:
+ return [
+ "scanned services/*/app and found no service-credential reader at all. "
+ "A clean result over an empty scan is the failure this gate exists to prevent."
+ ]
+
+ generated = _generated(root)
+ declared = _env_example_names(root)
+ by_source, delivered = _compose_delivery(root)
+ findings: list[str] = []
+
+ for variable, source_dirs in sorted(readers.items()):
+ where = ", ".join(f"services/{d}" for d in sorted(source_dirs))
+ if variable not in generated and variable not in OPERATOR_SUPPLIED:
+ findings.append(
+ f"{variable} is read by {where} and produced by nothing. "
+ "Add it to scripts/ensure_env.py::GENERATED, or to OPERATOR_SUPPLIED here with a reason."
+ )
+ continue
+ if variable not in declared:
+ findings.append(f"{variable} is read by {where} and is not listed in .env.example.")
+ if variable in OPERATOR_SUPPLIED:
+ continue
+ for source_dir in sorted(source_dirs):
+ if source_dir in UNMAPPED_SOURCE_DIRS:
+ continue
+ targets = by_source.get(source_dir)
+ if not targets:
+ findings.append(
+ f"services/{source_dir} reads {variable} and no compose service builds from it, "
+ "so nothing could deliver the value. Add it to UNMAPPED_SOURCE_DIRS with a reason if that is right."
+ )
+ continue
+ override = SATISFIED_BY_OVERRIDE.get(variable)
+ for target in sorted(targets):
+ names = delivered.get(target, set())
+ if variable in names:
+ continue
+ if override and override.format(SERVICE=source_dir.replace("-", "_").upper()) in names:
+ continue
+ findings.append(
+ f"compose service `{target}` builds services/{source_dir}, which reads {variable}, "
+ "and its `environment:` does not pass it — so the value cannot reach the process."
+ )
+
+ for variable, reason in sorted(OPERATOR_SUPPLIED.items()):
+ if variable not in readers:
+ findings.append(f"{variable} is declared operator-supplied ({reason}) and nothing reads it. Remove the row.")
+ for source_dir, reason in sorted(UNMAPPED_SOURCE_DIRS.items()):
+ if not (root / "services" / source_dir).is_dir():
+ findings.append(f"services/{source_dir} is exempted ({reason}) and does not exist. Remove the row.")
+
+ return findings
+
+
+def main(argv: list[str] | None = None) -> int:
+ parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+ parser.add_argument("--check", action="store_true", help="render the verdict (default)")
+ parser.parse_args(argv)
+
+ root = repo_root()
+ findings = check(root)
+ readers = _readers(root)
+ if findings:
+ print(f"FAIL — {len(findings)} finding(s):", file=sys.stderr)
+ for finding in findings:
+ print(f" - {finding}", file=sys.stderr)
+ return 1
+ print(f"OK: {len(readers)} service credentials read by production code are all produced by first run and delivered by compose.")
+ return 0
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/scripts/ensure_env.py b/scripts/ensure_env.py
index 8760c5858..d291d6c69 100644
--- a/scripts/ensure_env.py
+++ b/scripts/ensure_env.py
@@ -105,6 +105,20 @@ GENERATED: dict[str, tuple[Callable[[], str], str]] = {
lambda: secrets.token_urlsafe(32),
"Shared bearer for the actions service (API -> actions). Gates every response action.",
),
+ # The agents service's own credential, and the one case where the two
+ # sides could not meet in the middle: eight modules in `services/agents`
+ # read it, and `alert_writeback.service_token_valid` on the API verifies
+ # it *without* falling back to AISOC_SERVICE_TOKEN. Nothing generated it
+ # and no manifest passed it, so on every default install organisation
+ # memory, tenant skills, recent dispositions, identity context,
+ # knowledge-base runbooks, the MCP toolset, the playbook action bridge
+ # and SIEM writeback were all silently off — each logging its own
+ # `no_service_token` warning into a stream nobody reads and returning an
+ # empty result that looks exactly like "this tenant has none".
+ "AISOC_AGENTS_SERVICE_TOKEN": (
+ lambda: secrets.token_urlsafe(32),
+ "Shared bearer between the API and the agents service. Gates triage context, MCP tools and SIEM writeback.",
+ ),
# The HS256 key the API signs realtime tickets with and the realtime edge
# verifies them against. Both sides previously fell back to a constant
# committed to this repository whenever this was unset — which no manifest