Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

feat(gates): prove no service reaches the public internet by default (#972)

Beenu Aroraa6e42a6f45fb
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index fb226b5b1..0d5d7fc0b 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -584,6 +584,23 @@ jobs:
run: python3 scripts/check_sdk_surface.py --self-test
- name: SDK surface gate
run: python3 scripts/check_sdk_surface.py
+ # "Runs entirely on your infrastructure" was gated in three places that
+ # each covered a corner — the CLI's telemetry, the Helm egress policy,
+ # the agents' redactor — and nowhere platform-wide. This reads every
+ # service's declared settings defaults and fails on a public host that
+ # no air-gap guard covers. `services/purple-team` declared an ATT&CK CDN
+ # URL that nothing in the service ever read: dead config still publishes
+ # an external default, so it was deleted rather than exempted.
+ #
+ # The host classification is lifted out of the service's own
+ # `airgap.py` rather than reimplemented, so the gate cannot drift from
+ # the enforcement it reports on, and each exemption names the guard that
+ # justifies it — delete the guard and the exemption stops applying.
+ # The dynamic half is `tests/test_no_default_egress.py`.
+ - name: Default egress gate — self-test
+ run: python3 scripts/check_default_egress.py --self-test
+ - name: Default egress gate
+ run: python3 scripts/check_default_egress.py
# The count claim above says how many connectors exist; this says
# whether a reader can find out how to use them. 35 of 84 had no page
# at all and the sidebar listed 34 of 96, so most connector
diff --git a/apps/docs/docs/deployment/env-vars.md b/apps/docs/docs/deployment/env-vars.md
index 23155355b..d96933cbd 100644
--- a/apps/docs/docs/deployment/env-vars.md
+++ b/apps/docs/docs/deployment/env-vars.md
@@ -402,7 +402,6 @@ All variables use the `PURPLE_TEAM_` prefix.
| `PURPLE_TEAM_CALDERA_API_KEY` | `ADMIN123` | **Rotate before going live.** Caldera REST API key |
| `PURPLE_TEAM_ART_REPO_PATH` | `/opt/atomic-red-team` | Filesystem path to the Atomic Red Team checkout |
| `PURPLE_TEAM_ART_ATOMICS_PATH` | `/opt/atomic-red-team/atomics` | Path to the `atomics/` directory inside ART |
-| `PURPLE_TEAM_ATTACK_STIX_URL` | `https://raw.githubusercontent.com/mitre/cti/master/enterprise-attack/enterprise-attack.json` | URL to the MITRE ATT&CK STIX bundle |
| `PURPLE_TEAM_OTEL_ENDPOINT` | `http://localhost:4317` | OTLP collector endpoint |
| `PURPLE_TEAM_SERVICE_NAME` | `aisoc-purple-team` | OTel service name |
| `PURPLE_TEAM_HOST` | `0.0.0.0` | HTTP listener interface |
diff --git a/scripts/check_default_egress.py b/scripts/check_default_egress.py
new file mode 100644
index 000000000..112e6b386
--- /dev/null
+++ b/scripts/check_default_egress.py
@@ -0,0 +1,818 @@
+#!/usr/bin/env python3
+"""No service may name a public host in its default configuration.
+
+Why this exists
+---------------
+The front page says AiSOC "runs entirely on your infrastructure" with no data
+exfiltration. Three things already hold parts of that line: the CLI makes zero
+network calls when telemetry is unconfigured, the Helm chart denies egress by
+default, and the agents' redactor keeps raw PII out of anything that leaves a
+process. What none of them answer is the platform-wide question — **boot every
+service with an empty environment and is anything pointed at the internet?**
+
+That question is settled in two places at once, because either half alone is
+satisfied by the other's blind spot:
+
+* this gate, which reads the *declared defaults* and is exhaustive over them
+ but cannot see a URL built at runtime;
+* ``tests/test_no_default_egress.py``, which imports each service under a
+ socket guard and is blind to code paths startup does not reach, but sees
+ every host actually dialled on the paths it does.
+
+The defect that motivated it
+----------------------------
+``services/purple-team`` declared::
+
+ attack_stix_url: str = "https://raw.githubusercontent.com/mitre/cti/..."
+
+and nothing in the service read it. A dead setting still publishes an external
+default: it is documented, it is settable, and the next person to need an ATT&CK
+bundle wires the existing field rather than asking whether the service should be
+reaching a CDN at all. It was deleted rather than exempted.
+
+What counts as a settings module
+--------------------------------
+Structurally, never by filename alone — ``services/threatintel`` keeps its
+settings in ``app/config.py`` while ``services/api`` uses ``app/core/config.py``,
+and a gate keyed on one path would silently stop covering a service that moved
+its file. A module qualifies when it declares a ``BaseSettings`` subclass, or
+when it is named ``config.py`` / ``settings.py`` under ``services/<svc>/app/``.
+The second clause is deliberately wider than the first: it means a config module
+that stops using pydantic keeps its coverage.
+
+Vendor API clients are out of scope, and that is a real limit rather than an
+oversight. ``services/actions/app/clients/crowdstrike_rtr.py`` names
+``api.crowdstrike.com`` because that *is* CrowdStrike; the host is reached only
+once an operator has configured that integration with credentials, so it is a
+configured destination, not a default. The claim under test is about what an
+unconfigured install does.
+
+Reusing the shipped predicate instead of writing a third copy
+-------------------------------------------------------------
+"Is this host private?" is already answered twice in this tree, by
+``services/api/app/core/airgap.py`` and its mirror in
+``services/threatintel/app/airgap.py``. A third copy here would be the fastest
+possible way to make the gate disagree with the enforcement it reports on.
+
+Neither module imports cleanly from ``scripts/``: both bind a pydantic
+``settings`` object and one pulls in ``structlog``, and this gate runs on the
+bare interpreter in the lint job. Stubbing those imports would work until the
+module grew another one. So the predicate is read *structurally*: its function
+and the suffix tuple it closes over are lifted out of the shipped source by
+``ast``, checked to reference nothing but ``ipaddress``, and compiled on their
+own. The gate therefore executes the same code the service executes, and a
+rename or deletion in either module fails the gate loudly rather than silently
+forking its behaviour.
+
+The two copies are then compared *by behaviour* over a probe set rather than
+byte-for-byte, because they are legitimately worded differently. If they ever
+disagree about a host, this gate's verdict would be correct for one service and
+wrong for the other, so a disagreement is a finding.
+
+Bidirectional by construction
+-----------------------------
+The failure shape this repository keeps rediscovering is the one-directional
+check: it compares A against B, never B against A, and drift in the direction
+things actually move slips past while the check prints OK. An allow-list keyed
+only on "may this setting name a public host?" has exactly that shape — delete
+the guard that made the exemption defensible and the exemption outlives it.
+
+So every entry names the enforcement that justifies it, and all three
+directions are errors:
+
+ * tree -> allow-list : a public default with no entry.
+ * allow-list -> tree : an entry whose setting, module or host is gone.
+ * entry -> enforcement : a named guard that is not defined where the entry
+ says, or that is defined and called by nothing. Deleting the guard must
+ revoke the exemption, or the guard was never what made it safe.
+
+Non-vacuity
+-----------
+A gate that passes while inspecting nothing launders the claim it stands for.
+The repository root comes from git rather than ``__file__``; the scan must reach
+a plausible number of settings modules and URL defaults; and it must classify at
+least one host as private, because a classifier stuck on "public" and one stuck
+on "private" both report a clean tree when the corpus is empty. ``--self-test``
+proves the detector separates known-bad from known-good, that all three
+directions fire, that the happy path is still reachable, and that the gate
+refuses a tree with no content.
+
+Usage:
+ python3 scripts/check_default_egress.py --self-test
+ python3 scripts/check_default_egress.py
+ python3 scripts/check_default_egress.py --list
+
+Exit codes: 0 clean, 1 findings, 2 the scan itself could not run.
+"""
+
+from __future__ import annotations
+
+import argparse
+import ast
+import builtins
+import ipaddress
+import subprocess
+import sys
+from dataclasses import dataclass
+from pathlib import Path
+from urllib.parse import urlparse
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+
+from gate_toolkit import repo_root, self_test_main # noqa: E402
+
+# --------------------------------------------------------------------------
+# The shipped predicate, and its mirror.
+# --------------------------------------------------------------------------
+
+#: The module whose private-address logic this gate executes verbatim.
+PREDICATE_MODULE = "services/api/app/core/airgap.py"
+
+#: The mirror that must keep agreeing with it. Two copies of a security
+#: predicate are a fork waiting to happen; this is what notices.
+PREDICATE_MIRROR = "services/threatintel/app/airgap.py"
+
+#: The names lifted out of the shipped module, in the order they must be bound.
+PREDICATE_SUFFIXES = "_PRIVATE_SUFFIXES"
+PREDICATE_FUNCTION = "_is_private_address"
+
+#: Everything the lifted function is allowed to reach for beyond its own
+#: arguments, locals and the builtins. If it grows a dependency on the service's
+#: settings object the lift stops being safe, and this is what refuses it.
+PREDICATE_FREE_NAMES = frozenset({"ipaddress", PREDICATE_SUFFIXES})
+
+#: Hosts the two copies are compared over. Covers each branch either one takes:
+#: literal IPs across the private ranges, loopback, link-local, unspecified, the
+#: internal suffixes, a single-label container name, and public hosts of the
+#: exact shapes that appear in this tree's defaults.
+PARITY_PROBES = (
+ "127.0.0.1",
+ "10.0.0.5",
+ "192.168.1.10",
+ "172.16.0.1",
+ "169.254.169.254",
+ "0.0.0.0",
+ "::1",
+ "fd00::1",
+ "localhost",
+ "ollama.local",
+ "vllm.internal",
+ "gateway.lan",
+ "host.intranet",
+ "db.corp",
+ "nas.home",
+ "box.localdomain",
+ "connectors",
+ "aisoc-api",
+ "raw.githubusercontent.com",
+ "otx.alienvault.com",
+ "www.cisa.gov",
+ "api.openai.com",
+ "8.8.8.8",
+)
+
+# --------------------------------------------------------------------------
+# Scope.
+# --------------------------------------------------------------------------
+
+#: Filenames that hold settings regardless of whether they use pydantic.
+SETTINGS_FILENAMES = frozenset({"config.py", "settings.py"})
+
+#: URL schemes worth classifying. A scheme that never leaves the host (``file``)
+#: or that carries no host at all says nothing about egress.
+NETWORK_SCHEMES = frozenset(
+ {
+ "http",
+ "https",
+ "ws",
+ "wss",
+ "grpc",
+ "grpcs",
+ "redis",
+ "rediss",
+ "amqp",
+ "amqps",
+ "mongodb",
+ "kafka",
+ "postgres",
+ "postgresql",
+ "mysql",
+ "clickhouse",
+ "bolt",
+ "neo4j",
+ }
+)
+
+#: Calls whose fallback argument is a default. ``os.getenv("X", "https://…")``
+#: declares an egress default exactly as surely as a class attribute does.
+ENV_LOOKUPS = (("os", "getenv"), ("os", "environ", "get"), ("environ", "get"), ("getenv",))
+
+#: Floors for the scan itself. Set well under today's counts: they exist to
+#: catch a walk that broke, not to pin a number that moves with the tree.
+MIN_SETTINGS_MODULES = 8
+MIN_URL_DEFAULTS = 15
+
+#: Files that must exist for a directory to be this repository.
+SENTINELS = (PREDICATE_MODULE, PREDICATE_MIRROR, "services/purple-team/app/core/config.py")
+
+
+# --------------------------------------------------------------------------
+# The allow-list. Shrink-only, and every entry names what enforces it.
+# --------------------------------------------------------------------------
+
+
+@dataclass(frozen=True)
+class Exemption:
+ """A public default somebody signed off on, and the guard that justifies it."""
+
+ module: str
+ setting: str
+ host: str
+ reason: str
+ #: (file, symbol) pairs. Each symbol must be defined in that file *and*
+ #: called from somewhere in the owning service. An exemption whose guard is
+ #: deleted, renamed or orphaned stops applying.
+ enforced_by: tuple[tuple[str, str], ...]
+
+
+_THREATINTEL_GUARD = (
+ (PREDICATE_MIRROR, "is_host_allowed_for_airgap"),
+ ("services/threatintel/app/main.py", "_airgap_check_feed_url"),
+)
+
+ALLOWED: tuple[Exemption, ...] = (
+ Exemption(
+ module="services/threatintel/app/config.py",
+ setting="OTX_BASE_URL",
+ host="otx.alienvault.com",
+ reason=(
+ "AlienVault OTX is a public feed with no internal mirror to point at, so the default has to name it. "
+ "Under AISOC_AIRGAPPED the feed is refused at registration time rather than at request time, because a "
+ "failed poll every 30 minutes is itself a signal that an instance exists."
+ ),
+ enforced_by=_THREATINTEL_GUARD,
+ ),
+ Exemption(
+ module="services/threatintel/app/config.py",
+ setting="CISA_KEV_URL",
+ host="www.cisa.gov",
+ reason=(
+ "The CISA Known Exploited Vulnerabilities catalog is the authoritative public source and needs no API key, "
+ "which is why it is what CORE ships so a first run shows real data. An air-gapped deployment mirrors the "
+ "JSON internally and allow-lists the mirror; the feed is otherwise refused at registration time."
+ ),
+ enforced_by=_THREATINTEL_GUARD,
+ ),
+)
+
+
+# --------------------------------------------------------------------------
+# Findings.
+# --------------------------------------------------------------------------
+
+
+@dataclass(frozen=True)
+class UrlDefault:
+ """A URL-valued default found in a settings module."""
+
+ module: str
+ setting: str
+ lineno: int
+ url: str
+ host: str
+
+ @property
+ def service(self) -> str:
+ parts = Path(self.module).parts
+ return parts[1] if len(parts) > 1 else "?"
+
+
+class GateError(RuntimeError):
+ """Raised when the gate cannot trust its own inputs."""
+
+
+# --------------------------------------------------------------------------
+# Lifting the shipped predicate.
+# --------------------------------------------------------------------------
+
+
+def _free_names(fn: ast.FunctionDef) -> set[str]:
+ """Names ``fn`` loads that it neither binds nor inherits from the builtins."""
+ bound = {a.arg for a in (*fn.args.posonlyargs, *fn.args.args, *fn.args.kwonlyargs)}
+ if fn.args.vararg:
+ bound.add(fn.args.vararg.arg)
+ if fn.args.kwarg:
+ bound.add(fn.args.kwarg.arg)
+ loaded: set[str] = set()
+ for node in ast.walk(fn):
+ if isinstance(node, ast.Name):
+ (loaded if isinstance(node.ctx, ast.Load) else bound).add(node.id)
+ elif isinstance(node, (ast.comprehension,)):
+ for target in ast.walk(node.target):
+ if isinstance(target, ast.Name):
+ bound.add(target.id)
+ return loaded - bound - set(dir(builtins))
+
+
+def load_shipped_predicate(root: Path, module: str):
+ """Compile ``_is_private_address`` out of ``module`` and return it.
+
+ Read rather than imported, and read rather than copied: the gate runs the
+ service's own classification without needing the service's dependencies,
+ and cannot drift from it.
+ """
+ path = root / module
+ try:
+ source = path.read_text(encoding="utf-8")
+ except OSError as exc:
+ raise GateError(f"cannot read the shipped air-gap predicate at {module}: {exc}") from exc
+
+ tree = ast.parse(source, filename=module)
+ fn = next((n for n in tree.body if isinstance(n, ast.FunctionDef) and n.name == PREDICATE_FUNCTION), None)
+ suffixes = next(
+ (n for n in tree.body if isinstance(n, (ast.Assign, ast.AnnAssign)) and PREDICATE_SUFFIXES in ast.dump(n)),
+ None,
+ )
+ if fn is None:
+ raise GateError(
+ f"{module} no longer defines {PREDICATE_FUNCTION}() — this gate classifies hosts with that function, so it cannot run"
+ )
+ if suffixes is None:
+ raise GateError(
+ f"{module} no longer defines {PREDICATE_SUFFIXES} — {PREDICATE_FUNCTION}() closes over it, so the lift is incomplete"
+ )
+
+ stray = _free_names(fn) - PREDICATE_FREE_NAMES
+ if stray:
+ raise GateError(
+ f"{module}::{PREDICATE_FUNCTION}() now reads {', '.join(sorted(stray))}, which this gate cannot supply. "
+ "Lifting it would run different code than the service does; re-read the function and widen "
+ "PREDICATE_FREE_NAMES deliberately, or give the gate a supported way to import it."
+ )
+
+ namespace: dict[str, object] = {"ipaddress": ipaddress}
+ lifted = ast.Module(body=[suffixes, fn], type_ignores=[])
+ exec(compile(lifted, filename=f"{module} (lifted)", mode="exec"), namespace) # noqa: S102 - repo's own tracked source, vetted above
+ return namespace[PREDICATE_FUNCTION]
+
+
+def predicate_parity(root: Path) -> list[str]:
+ """Both shipped copies must answer every probe host the same way."""
+ primary = load_shipped_predicate(root, PREDICATE_MODULE)
+ mirror = load_shipped_predicate(root, PREDICATE_MIRROR)
+ disagreements = [host for host in PARITY_PROBES if bool(primary(host)) != bool(mirror(host))]
+ if not disagreements:
+ return []
+ return [
+ f"PREDICATE FORK {PREDICATE_MODULE} and {PREDICATE_MIRROR} disagree about: {', '.join(disagreements)}",
+ " Two copies of the private-address rule means this gate's verdict is right for one service",
+ " and wrong for the other. Reconcile them before trusting either.",
+ ]
+
+
+# --------------------------------------------------------------------------
+# Finding the settings modules and their URL defaults.
+# --------------------------------------------------------------------------
+
+
+def _is_settings_class(node: ast.ClassDef) -> bool:
+ return any("BaseSettings" in ast.unparse(base) for base in node.bases)
+
+
+def is_settings_module(rel: str, tree: ast.Module) -> bool:
+ """Whether ``rel`` declares service settings, by structure then by name."""
+ parts = Path(rel).parts
+ if len(parts) < 3 or parts[0] != "services" or parts[2] != "app":
+ return False
+ if any(isinstance(n, ast.ClassDef) and _is_settings_class(n) for n in ast.walk(tree)):
+ return True
+ return Path(rel).name in SETTINGS_FILENAMES
+
+
+def _call_path(node: ast.expr) -> tuple[str, ...]:
+ """Dotted name of a call target, e.g. ``os.environ.get`` -> ('os','environ','get')."""
+ parts: list[str] = []
+ cur: ast.expr | None = node
+ while isinstance(cur, ast.Attribute):
+ parts.append(cur.attr)
+ cur = cur.value
+ if isinstance(cur, ast.Name):
+ parts.append(cur.id)
+ return tuple(reversed(parts))
+
+
+def _string_defaults(value: ast.expr) -> list[tuple[str, int]]:
+ """Every ``(string, line)`` that ``value`` supplies as a default.
+
+ Returns the unwrapped string rather than the ``ast.Constant`` so the
+ ``isinstance`` narrowing done here survives into the caller; handing back
+ the node leaves every reader to re-prove that ``.value`` is a ``str``.
+
+ Covers the four shapes settings take in this tree: a bare literal, the
+ first positional or ``default=`` of a ``Field(...)``, and the fallback
+ argument of an environment lookup.
+ """
+ if isinstance(value, ast.Constant) and isinstance(value.value, str):
+ return [(value.value, value.lineno)]
+ if not isinstance(value, ast.Call):
+ return []
+
+ path = _call_path(value.func)
+ candidates: list[ast.expr] = []
+
+ if path and path[-1] == "Field":
+ candidates.extend(value.args[:1])
+ candidates.extend(kw.value for kw in value.keywords if kw.arg == "default")
+ elif path in ENV_LOOKUPS and len(value.args) >= 2:
+ candidates.append(value.args[1])
+
+ return [(node.value, node.lineno) for node in candidates if isinstance(node, ast.Constant) and isinstance(node.value, str)]
+
+
+def _target_name(node: ast.stmt) -> str | None:
+ if isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name):
+ return node.target.id
+ if isinstance(node, ast.Assign) and len(node.targets) == 1 and isinstance(node.targets[0], ast.Name):
+ return node.targets[0].id
+ return None
+
+
+def url_defaults(rel: str, tree: ast.Module) -> list[UrlDefault]:
+ """Every URL-valued default declared at module scope or in a settings class."""
+ statements: list[ast.stmt] = []
+ for node in tree.body:
+ statements.append(node)
+ if isinstance(node, ast.ClassDef):
+ statements.extend(node.body)
+
+ found: list[UrlDefault] = []
+ for stmt in statements:
+ name = _target_name(stmt)
+ value = getattr(stmt, "value", None)
+ if name is None or value is None:
+ continue
+ for raw, lineno in _string_defaults(value):
+ parsed = _parse_url(raw)
+ if parsed is None:
+ continue
+ found.append(UrlDefault(module=rel, setting=name, lineno=lineno, url=raw, host=parsed))
+ return found
+
+
+def _parse_url(raw: str) -> str | None:
+ """The hostname of ``raw`` if it is a network URL, else None."""
+ if "://" not in raw:
+ return None
+ try:
+ parsed = urlparse(raw.strip())
+ except ValueError:
+ return None
+ if parsed.scheme.lower() not in NETWORK_SCHEMES:
+ return None
+ try:
+ host = parsed.hostname
+ except ValueError:
+ return None
+ return host.lower() if host else None
+
+
+def scan(root: Path, paths: list[str]) -> tuple[list[UrlDefault], list[str]]:
+ """Every URL default across every settings module. Returns (defaults, modules)."""
+ defaults: list[UrlDefault] = []
+ modules: list[str] = []
+ for rel in sorted(paths):
+ if not rel.endswith(".py") or not rel.startswith("services/"):
+ continue
+ try:
+ tree = ast.parse((root / rel).read_text(encoding="utf-8", errors="replace"), filename=rel)
+ except (OSError, SyntaxError, ValueError):
+ continue
+ if not is_settings_module(rel, tree):
+ continue
+ modules.append(rel)
+ defaults.extend(url_defaults(rel, tree))
+ return defaults, modules
+
+
+# --------------------------------------------------------------------------
+# Enforcement: the direction that makes an exemption revocable.
+# --------------------------------------------------------------------------
+
+
+def _defines(tree: ast.Module, symbol: str) -> bool:
+ for node in ast.walk(tree):
+ if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)) and node.name == symbol:
+ return True
+ if isinstance(node, (ast.Assign, ast.AnnAssign)) and _target_name(node) == symbol:
+ return True
+ return False
+
+
+def _calls(tree: ast.Module, symbol: str) -> bool:
+ return any(isinstance(n, ast.Call) and _call_path(n.func)[-1:] == (symbol,) for n in ast.walk(tree))
+
+
+def enforcement_problems(root: Path, paths: list[str], exemption: Exemption) -> list[str]:
+ """Whether the guard an exemption names is defined and actually called."""
+ problems: list[str] = []
+ service = Path(exemption.module).parts[1]
+ label = f"{exemption.module}::{exemption.setting}"
+
+ for guard_module, symbol in exemption.enforced_by:
+ path = root / guard_module
+ if not path.exists():
+ problems.append(f"EXEMPTION UNENFORCED {label}: names {guard_module}::{symbol}, and that file is gone")
+ continue
+ try:
+ tree = ast.parse(path.read_text(encoding="utf-8", errors="replace"), filename=guard_module)
+ except (OSError, SyntaxError, ValueError) as exc:
+ problems.append(f"EXEMPTION UNENFORCED {label}: cannot parse {guard_module} ({exc})")
+ continue
+ if not _defines(tree, symbol):
+ problems.append(f"EXEMPTION UNENFORCED {label}: {guard_module} no longer defines {symbol}")
+ continue
+
+ callers = [
+ rel
+ for rel in paths
+ if rel.startswith(f"services/{service}/app/")
+ and rel.endswith(".py")
+ and _calls_in(root, rel, symbol, skip_definition_of=symbol if rel == guard_module else None)
+ ]
+ if not callers:
+ problems.append(
+ f"EXEMPTION UNENFORCED {label}: {guard_module}::{symbol} is defined but called from nowhere in "
+ f"services/{service}/app/ — a guard nothing invokes did not make this default safe"
+ )
+ return problems
+
+
+def _calls_in(root: Path, rel: str, symbol: str, skip_definition_of: str | None = None) -> bool:
+ """Whether ``rel`` calls ``symbol`` outside that symbol's own definition."""
+ try:
+ tree = ast.parse((root / rel).read_text(encoding="utf-8", errors="replace"), filename=rel)
+ except (OSError, SyntaxError, ValueError):
+ return False
+ if skip_definition_of:
+ tree.body = [n for n in tree.body if not (isinstance(n, (ast.FunctionDef, ast.AsyncFunctionDef)) and n.name == skip_definition_of)]
+ return _calls(tree, symbol)
+
+
+# --------------------------------------------------------------------------
+# The comparison, in all three directions.
+# --------------------------------------------------------------------------
+
+
+def compare(
+ defaults: list[UrlDefault],
+ allowed: tuple[Exemption, ...],
+ is_private,
+ *,
+ root: Path | None = None,
+ paths: list[str] | None = None,
+) -> list[str]:
+ """Returns a list of failure lines; empty means clean.
+
+ ``root``/``paths`` are optional so the self-test can exercise the two
+ list-comparison directions without a tree. When they are supplied the third
+ direction — does the named guard still exist and run — is checked too.
+ """
+ problems: list[str] = []
+ index = {(e.module, e.setting): e for e in allowed}
+
+ # Direction 1: tree -> allow-list. A public default nobody signed off on.
+ matched: set[tuple[str, str]] = set()
+ for default in sorted(defaults, key=lambda d: (d.module, d.lineno)):
+ if is_private(default.host):
+ continue
+ entry = index.get((default.module, default.setting))
+ if entry is None:
+ problems.append(
+ f"EGRESS DEFAULT {default.module}:{default.lineno} {default.setting} = {default.url!r} "
+ f"-> public host {default.host!r}, no allow-list entry"
+ )
+ continue
+ if entry.host != default.host:
+ problems.append(
+ f"HOST CHANGED {default.module}:{default.lineno} {default.setting} now names {default.host!r}, "
+ f"allow-list records {entry.host!r} — re-read the default before moving the entry"
+ )
+ continue
+ matched.add((default.module, default.setting))
+
+ # Direction 2: allow-list -> tree. An exemption outliving what it excused.
+ for entry in allowed:
+ if (entry.module, entry.setting) in matched:
+ continue
+ problems.append(
+ f"STALE EXEMPTION {entry.module}::{entry.setting} is allow-listed for {entry.host!r}, and no such "
+ "public default is there any more — drop the entry"
+ )
+
+ # Direction 3: exemption -> enforcement. Deleting the guard revokes the pass.
+ if root is not None and paths is not None:
+ for entry in allowed:
+ if (entry.module, entry.setting) in matched:
+ problems.extend(enforcement_problems(root, paths, entry))
+
+ return problems
+
+
+# --------------------------------------------------------------------------
+# Self-test.
+# --------------------------------------------------------------------------
+
+
+def _fake_private(host: str) -> bool:
+ """A stand-in classifier for the list-direction checks, which are about the
+ allow-list bookkeeping rather than about host classification."""
+ return host.endswith(".internal") or host == "localhost"
+
+
+def self_test() -> int:
+ checks: list[tuple[str, bool]] = []
+
+ root = repo_root()
+
+ # The lifted predicate must be the shipped one, and must separate the two
+ # classes of host this gate turns on.
+ try:
+ is_private = load_shipped_predicate(root, PREDICATE_MODULE)
+ lifted_ok = True
+ detail = ""
+ except GateError as exc:
+ is_private = _fake_private
+ lifted_ok = False
+ detail = f" ({exc})"
+ checks.append((f"lifts {PREDICATE_FUNCTION}() out of {PREDICATE_MODULE}{detail}", lifted_ok))
+
+ if lifted_ok:
+ private_hosts = ("127.0.0.1", "10.0.0.5", "localhost", "ollama.local", "connectors", "::1")
+ public_hosts = ("raw.githubusercontent.com", "otx.alienvault.com", "www.cisa.gov", "api.openai.com", "8.8.8.8")
+ checks.append(("shipped predicate calls every known-private host private", all(is_private(h) for h in private_hosts)))
+ checks.append(("shipped predicate calls every known-public host public", not any(is_private(h) for h in public_hosts)))
+ checks.append((f"{PREDICATE_MODULE} and {PREDICATE_MIRROR} agree on every probe host", not predicate_parity(root)))
+
+ # The URL extractor must see each of the four default shapes, and must not
+ # invent findings from strings that are not network URLs.
+ sample = (
+ "import os\n"
+ "from pydantic import Field\n"
+ "from pydantic_settings import BaseSettings\n"
+ "class S(BaseSettings):\n"
+ " plain: str = 'https://plain.example.com'\n"
+ " positional: str = Field('https://positional.example.com')\n"
+ " keyword: str = Field(default='https://keyword.example.com')\n"
+ " private: str = 'http://localhost:8000'\n"
+ " not_a_url: str = 'plain text, no scheme'\n"
+ " local_file: str = 'file:///etc/aisoc.yaml'\n"
+ "ENV = os.getenv('X', 'https://env.example.com')\n"
+ )
+ tree = ast.parse(sample)
+ rel = "services/sample/app/core/config.py"
+ checks.append(("a BaseSettings module is in scope whatever it is called", is_settings_module(rel, tree)))
+ checks.append(("a module outside services/<svc>/app/ is not", not is_settings_module("scripts/thing/config.py", ast.parse(sample))))
+
+ found = {d.setting: d.host for d in url_defaults(rel, tree)}
+ for shape, setting, host in (
+ ("a bare literal default", "plain", "plain.example.com"),
+ ("Field(...) positional", "positional", "positional.example.com"),
+ ("Field(default=...)", "keyword", "keyword.example.com"),
+ ("os.getenv fallback", "ENV", "env.example.com"),
+ ("a private default", "private", "localhost"),
+ ):
+ checks.append((f"extractor reads {shape}", found.get(setting) == host))
+ checks.append(("extractor ignores a string that is not a URL", "not_a_url" not in found))
+ checks.append(("extractor ignores a non-network scheme", "local_file" not in found))
+
+ # Direction 1.
+ leak = UrlDefault(rel, "leak", 1, "https://evil.example.com", "evil.example.com")
+ checks.append(("a public default with no entry is a finding", bool(compare([leak], (), _fake_private))))
+
+ # Direction 2 — the one a one-directional gate misses.
+ ghost = Exemption(rel, "gone", "gone.example.com", "r", ())
+ checks.append(("an allow-list entry with no matching default is a finding", bool(compare([], (ghost,), _fake_private))))
+
+ # A moved host must not inherit the old entry's sign-off.
+ entry = Exemption(rel, "leak", "known.example.com", "r", ())
+ checks.append(("an entry whose default changed host is a finding", bool(compare([leak], (entry,), _fake_private))))
+
+ # The happy path has to stay reachable, or every check above is satisfied
+ # by a comparison that always fails.
+ ok_entry = Exemption(rel, "leak", "evil.example.com", "r", ())
+ checks.append(("an exactly-matching entry passes", not compare([leak], (ok_entry,), _fake_private)))
+ checks.append(
+ ("a private default needs no entry", not compare([UrlDefault(rel, "p", 1, "http://localhost", "localhost")], (), _fake_private))
+ )
+
+ # Direction 3, proven against the real tree: an entry naming a guard that
+ # is not there must stop exempting, which is what makes deleting the guard
+ # unable to keep the pass.
+ paths = tracked_files(root)
+ real = ALLOWED[0]
+ missing_guard = Exemption(real.module, real.setting, real.host, real.reason, (("services/threatintel/app/airgap.py", "no_such_guard"),))
+ orphan_guard = Exemption(real.module, real.setting, real.host, real.reason, (("services/threatintel/app/airgap.py", "airgap_status"),))
+ real_defaults, _ = scan(root, paths)
+ checks.append(
+ (
+ "an exemption naming a guard that is not defined stops applying",
+ bool(compare(real_defaults, (missing_guard,), is_private, root=root, paths=paths)),
+ )
+ )
+ checks.append(
+ (
+ "an exemption naming a guard nothing calls stops applying",
+ bool(compare(real_defaults, (orphan_guard,), is_private, root=root, paths=paths)),
+ )
+ )
+
+ return self_test_main(Path(__file__).name, extra=checks)
+
+
+# --------------------------------------------------------------------------
+# Entry point.
+# --------------------------------------------------------------------------
+
+
+def tracked_files(root: Path) -> list[str]:
+ out = subprocess.run( # noqa: S603 - fixed argv, no shell
+ ["git", "-C", str(root), "ls-files", "-z"],
+ capture_output=True,
+ text=True,
+ check=False,
+ )
+ return [p for p in out.stdout.split("\0") if p]
+
+
+def main() -> int:
+ parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+ parser.add_argument("--self-test", action="store_true", help="prove the gate still detects what it claims, then exit")
+ parser.add_argument("--list", action="store_true", help="print every URL default found, with its classification")
+ args = parser.parse_args()
+
+ if args.self_test:
+ return self_test()
+
+ root = repo_root()
+ missing = [s for s in SENTINELS if not (root / s).exists()]
+ if missing:
+ print(f"FAIL: {root} does not look like the AiSOC repository (missing: {', '.join(missing)})")
+ return 2
+
+ paths = tracked_files(root)
+ if not paths:
+ print(f"FAIL: git listed no tracked files under {root}; the scan is broken, not the tree clean")
+ return 2
+
+ try:
+ is_private = load_shipped_predicate(root, PREDICATE_MODULE)
+ except GateError as exc:
+ print(f"FAIL: {exc}")
+ return 2
+
+ defaults, modules = scan(root, paths)
+ print(f"scanning {len(paths)} tracked files under {root}")
+ print(f"settings modules {len(modules)} across {len({Path(m).parts[1] for m in modules})} service(s)")
+ print(f"url defaults {len(defaults)}")
+
+ if len(modules) < MIN_SETTINGS_MODULES:
+ print(f"FAIL: found {len(modules)} settings module(s), expected >= {MIN_SETTINGS_MODULES}; the walk is broken, not the tree clean")
+ return 2
+ if len(defaults) < MIN_URL_DEFAULTS:
+ print(f"FAIL: found {len(defaults)} URL default(s), expected >= {MIN_URL_DEFAULTS}; the extractor is broken, not the tree clean")
+ return 2
+
+ private = [d for d in defaults if is_private(d.host)]
+ if not private:
+ print("FAIL: not one default classified as private. Every service points somewhere internal by default, so a")
+ print(" result with none means the classifier is stuck, not that the tree changed.")
+ return 2
+ print(f"classification {len(private)} private, {len(defaults) - len(private)} public")
+
+ if args.list:
+ print()
+ for default in sorted(defaults, key=lambda d: (d.module, d.lineno)):
+ verdict = "private" if is_private(default.host) else "PUBLIC "
+ print(f" {verdict} {default.module}:{default.lineno} {default.setting} -> {default.host}")
+ print()
+
+ problems = predicate_parity(root) + compare(defaults, ALLOWED, is_private, root=root, paths=paths)
+
+ if problems:
+ print(f"\nFAIL: {len(problems)} problem(s).\n")
+ for line in problems:
+ print(f" {line}")
+ print(
+ "\nA service must not reach the public internet on a default nobody chose. Either give the\n"
+ "setting an internal default, delete it if nothing reads it, or add an Exemption naming the\n"
+ "module that enforces air-gap policy on it. Never allow-list a default you could remove."
+ )
+ return 1
+
+ print(f"OK — {len(defaults)} URL default(s) across {len(modules)} settings module(s); every public host is")
+ print(f" one of {len(ALLOWED)} allow-listed default(s), each with a reason and a guard that is called")
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/scripts/tests/test_check_default_egress.py b/scripts/tests/test_check_default_egress.py
new file mode 100644
index 000000000..59b24282d
--- /dev/null
+++ b/scripts/tests/test_check_default_egress.py
@@ -0,0 +1,353 @@
+"""The default-egress gate detects what it claims, in all three directions.
+
+The cases below are the ones that decide whether this gate is worth its run
+time. Direction 1 is the obvious one and the easiest to get right. Directions 2
+and 3 are what stop the allow-list becoming a place to retire findings: an
+entry that outlives its default, and an entry whose named guard has been
+deleted or left calling nothing.
+
+The extractor cases pin the four shapes a default takes in this tree, because a
+gate that reads only bare literals would have missed
+``services/fusion``'s ``Field(default=...)`` settings entirely and reported a
+clean scan over them.
+"""
+
+from __future__ import annotations
+
+import ast
+import importlib.util
+import sys
+from pathlib import Path
+
+import pytest
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+GATE = REPO_ROOT / "scripts" / "check_default_egress.py"
+
+MODULE = "services/sample/app/core/config.py"
+
+
+def _load():
+ spec = importlib.util.spec_from_file_location("check_default_egress", GATE)
+ assert spec is not None and spec.loader is not None
+ module = importlib.util.module_from_spec(spec)
+ sys.modules[spec.name] = module
+ spec.loader.exec_module(module)
+ return module
+
+
+gate = _load()
+
+
+def _private(host: str) -> bool:
+ """A stand-in classifier, so the comparison cases are about bookkeeping."""
+ return host in {"localhost", "redis"} or host.endswith((".internal", ".local"))
+
+
+def _default(setting: str, url: str, host: str, module: str = MODULE):
+ return gate.UrlDefault(module=module, setting=setting, lineno=1, url=url, host=host)
+
+
+# --------------------------------------------------------------------------
+# Scope.
+# --------------------------------------------------------------------------
+
+
+ ("rel", "source", "expected"),
+ [
+ (MODULE, "from pydantic_settings import BaseSettings\nclass S(BaseSettings):\n x: int = 1\n", True),
+ # Filename alone is enough: a config module that stops using pydantic
+ # keeps its coverage rather than silently falling out of scope.
+ ("services/sample/app/config.py", "URL = 'http://x'\n", True),
+ ("services/sample/app/settings.py", "URL = 'http://x'\n", True),
+ # A BaseSettings subclass is in scope wherever it lives, because
+ # services do not agree on where to put it.
+ ("services/sample/app/core/tuning.py", "from pydantic_settings import BaseSettings\nclass S(BaseSettings):\n pass\n", True),
+ # Out of scope: not a service, not under app/, and an ordinary module.
+ ("scripts/config.py", "URL = 'http://x'\n", False),
+ ("services/sample/tests/config.py", "URL = 'http://x'\n", False),
+ ("services/sample/app/clients/crowdstrike.py", "BASE = 'https://api.crowdstrike.com'\n", False),
+ ],
+)
+def test_scope_is_structural_then_by_name(rel: str, source: str, expected: bool) -> None:
+ assert gate.is_settings_module(rel, ast.parse(source)) is expected
+
+
+# --------------------------------------------------------------------------
+# Extraction.
+# --------------------------------------------------------------------------
+
+
+ ("declaration", "setting", "host"),
+ [
+ (" plain: str = 'https://a.example.com'", "plain", "a.example.com"),
+ (" positional: str = Field('https://b.example.com')", "positional", "b.example.com"),
+ (" keyword: str = Field(default='https://c.example.com')", "keyword", "c.example.com"),
+ (" aliased: str = Field(default='https://d.example.com', alias='D')", "aliased", "d.example.com"),
+ (" private: str = 'http://localhost:8000'", "private", "localhost"),
+ (" container: str = 'http://redis:6379/0'", "container", "redis"),
+ (" scheme: str = 'redis://cache.internal:6379/2'", "scheme", "cache.internal"),
+ (" ported: str = 'https://e.example.com:8443/path?q=1'", "ported", "e.example.com"),
+ (" upper: str = 'https://F.EXAMPLE.COM/x'", "upper", "f.example.com"),
+ ],
+)
+def test_extractor_reads_every_default_shape(declaration: str, setting: str, host: str) -> None:
+ source = f"from pydantic import Field\nfrom pydantic_settings import BaseSettings\nclass S(BaseSettings):\n{declaration}\n"
+ found = {d.setting: d.host for d in gate.url_defaults(MODULE, ast.parse(source))}
+ assert found.get(setting) == host
+
+
+ "declaration",
+ [
+ " prose: str = 'not a url at all'",
+ " path: str = '/var/lib/aisoc'",
+ " local: str = 'file:///etc/aisoc.yaml'",
+ " empty: str = ''",
+ " template: str = '{scheme}://{host}'",
+ ],
+)
+def test_extractor_invents_nothing_from_a_non_url(declaration: str) -> None:
+ source = f"from pydantic_settings import BaseSettings\nclass S(BaseSettings):\n{declaration}\n"
+ assert gate.url_defaults(MODULE, ast.parse(source)) == []
+
+
+def test_extractor_reads_environment_fallbacks() -> None:
+ """``os.getenv('X', 'https://…')`` declares a default as surely as a class attribute."""
+ source = (
+ "import os\n"
+ "from os import environ, getenv\n"
+ "A = os.getenv('A', 'https://a.example.com')\n"
+ "B = os.environ.get('B', 'https://b.example.com')\n"
+ "C = environ.get('C', 'https://c.example.com')\n"
+ "D = getenv('D', 'https://d.example.com')\n"
+ "E = os.getenv('E')\n"
+ )
+ found = {d.setting: d.host for d in gate.url_defaults("services/sample/app/config.py", ast.parse(source))}
+ assert found == {
+ "A": "a.example.com",
+ "B": "b.example.com",
+ "C": "c.example.com",
+ "D": "d.example.com",
+ }
+
+
+# --------------------------------------------------------------------------
+# Direction 1: a public default nobody signed off on.
+# --------------------------------------------------------------------------
+
+
+def test_a_public_default_with_no_entry_is_a_finding() -> None:
+ # The finding is matched against the input rather than against a repeated
+ # host literal: an assertion that restates the expected string can drift
+ # from the value under test, and a bare `"host.example" in text` reads as
+ # URL sanitization to a static analyser looking for exactly that mistake.
+ leak = _default("leak", "https://cdn.example.com/x", "cdn.example.com")
+ problems = gate.compare([leak], (), _private)
+ assert len(problems) == 1
+ assert "EGRESS DEFAULT" in problems[0]
+ assert leak.host in problems[0]
+ assert leak.setting in problems[0]
+
+
[email protected]("host", ["localhost", "redis", "cache.internal", "box.local"])
+def test_a_private_default_needs_no_entry(host: str) -> None:
+ assert gate.compare([_default("p", f"http://{host}:1", host)], (), _private) == []
+
+
+def test_the_shipped_defect_is_what_this_gate_was_built_for() -> None:
+ """The purple-team setting that motivated the gate, replayed."""
+ dead = _default(
+ "attack_stix_url",
+ "https://raw.githubusercontent.com/mitre/cti/master/enterprise-attack/enterprise-attack.json",
+ "raw.githubusercontent.com",
+ module="services/purple-team/app/core/config.py",
+ )
+ problems = gate.compare([dead], gate.ALLOWED, _private)
+ assert any("attack_stix_url" in p and "EGRESS DEFAULT" in p for p in problems)
+
+
+# --------------------------------------------------------------------------
+# Direction 2: an exemption that outlived its default.
+# --------------------------------------------------------------------------
+
+
+def test_an_entry_with_no_matching_default_is_a_finding() -> None:
+ stale = gate.Exemption(MODULE, "gone", "gone.example.com", "reason", ())
+ problems = gate.compare([], (stale,), _private)
+ assert len(problems) == 1
+ assert "STALE EXEMPTION" in problems[0]
+
+
+def test_an_entry_does_not_cover_a_default_that_changed_host() -> None:
+ """A signed-off host is not a signed-off setting.
+
+ Repointing a setting at a different public host must not inherit the old
+ entry's approval — that is how an allow-list launders a real leak.
+ """
+ entry = gate.Exemption(MODULE, "feed", "known.example.com", "reason", ())
+ moved = _default("feed", "https://elsewhere.example.com/x", "elsewhere.example.com")
+ problems = gate.compare([moved], (entry,), _private)
+ assert any("HOST CHANGED" in p for p in problems)
+
+
+def test_an_exactly_matching_entry_passes() -> None:
+ """The happy path must stay reachable, or every case above is satisfied by
+ a comparison that always fails."""
+ entry = gate.Exemption(MODULE, "feed", "known.example.com", "reason", ())
+ assert gate.compare([_default("feed", "https://known.example.com/x", "known.example.com")], (entry,), _private) == []
+
+
+# --------------------------------------------------------------------------
+# Direction 3: the exemption must be revocable by deleting the guard.
+# --------------------------------------------------------------------------
+
+
+def _tree(tmp_path: Path, files: dict[str, str]) -> tuple[Path, list[str]]:
+ root = tmp_path / "tree"
+ for rel, body in files.items():
+ path = root / rel
+ path.parent.mkdir(parents=True, exist_ok=True)
+ path.write_text(body, encoding="utf-8")
+ return root, sorted(files)
+
+
+GUARD_MODULE = "services/sample/app/guard.py"
+CALLER_MODULE = "services/sample/app/main.py"
+
+
+def test_a_guard_that_is_defined_and_called_sustains_the_exemption(tmp_path: Path) -> None:
+ root, paths = _tree(
+ tmp_path,
+ {
+ GUARD_MODULE: "def block(host):\n return False\n",
+ CALLER_MODULE: "from app.guard import block\n\nif block('x'):\n pass\n",
+ },
+ )
+ entry = gate.Exemption(MODULE, "feed", "known.example.com", "r", ((GUARD_MODULE, "block"),))
+ assert gate.enforcement_problems(root, paths, entry) == []
+
+
+def test_deleting_the_guard_revokes_the_exemption(tmp_path: Path) -> None:
+ """The bidirectional property. Without it, removing the enforcement leaves
+ the exemption in place and the gate keeps printing OK over a real leak."""
+ root, paths = _tree(tmp_path, {GUARD_MODULE: "def something_else():\n return 1\n"})
+ entry = gate.Exemption(MODULE, "feed", "known.example.com", "r", ((GUARD_MODULE, "block"),))
+ problems = gate.enforcement_problems(root, paths, entry)
+ assert len(problems) == 1
+ assert "no longer defines block" in problems[0]
+
+
+def test_an_orphaned_guard_revokes_the_exemption(tmp_path: Path) -> None:
+ """A passing test on an uncalled function is indistinguishable from a
+ working feature — the shape this repository found a dozen times in one
+ audit. A guard nothing invokes did not make the default safe."""
+ root, paths = _tree(
+ tmp_path,
+ {
+ GUARD_MODULE: "def block(host):\n return False\n",
+ CALLER_MODULE: "from app.guard import block # imported and never called\n",
+ },
+ )
+ entry = gate.Exemption(MODULE, "feed", "known.example.com", "r", ((GUARD_MODULE, "block"),))
+ problems = gate.enforcement_problems(root, paths, entry)
+ assert len(problems) == 1
+ assert "called from nowhere" in problems[0]
+
+
+def test_a_guard_calling_only_itself_does_not_count(tmp_path: Path) -> None:
+ """Recursion is not enforcement."""
+ root, paths = _tree(tmp_path, {GUARD_MODULE: "def block(host):\n return block(host)\n"})
+ entry = gate.Exemption(MODULE, "feed", "known.example.com", "r", ((GUARD_MODULE, "block"),))
+ assert "called from nowhere" in gate.enforcement_problems(root, paths, entry)[0]
+
+
+def test_a_guard_in_a_file_that_is_gone_revokes_the_exemption(tmp_path: Path) -> None:
+ root, paths = _tree(tmp_path, {CALLER_MODULE: "pass\n"})
+ entry = gate.Exemption(MODULE, "feed", "known.example.com", "r", (("services/sample/app/vanished.py", "block"),))
+ assert "that file is gone" in gate.enforcement_problems(root, paths, entry)[0]
+
+
+# --------------------------------------------------------------------------
+# The lifted predicate: the gate must run the code the services run.
+# --------------------------------------------------------------------------
+
+
+ "host", ["127.0.0.1", "10.0.0.5", "192.168.1.10", "172.16.0.1", "::1", "localhost", "ollama.local", "vllm.internal", "redis"]
+)
+def test_the_shipped_predicate_calls_internal_hosts_private(host: str) -> None:
+ assert gate.load_shipped_predicate(REPO_ROOT, gate.PREDICATE_MODULE)(host)
+
+
[email protected]("host", ["raw.githubusercontent.com", "otx.alienvault.com", "www.cisa.gov", "api.openai.com", "8.8.8.8"])
+def test_the_shipped_predicate_calls_public_hosts_public(host: str) -> None:
+ assert not gate.load_shipped_predicate(REPO_ROOT, gate.PREDICATE_MODULE)(host)
+
+
+def test_both_shipped_copies_of_the_predicate_agree() -> None:
+ """Two copies of a security predicate are a fork waiting to happen. If they
+ diverge, this gate's verdict is right for one service and wrong for the
+ other, so the divergence is the finding."""
+ assert gate.predicate_parity(REPO_ROOT) == []
+
+
+def test_the_lift_refuses_a_predicate_that_grew_a_dependency(tmp_path: Path) -> None:
+ """Lifting a function that now reads the service's settings object would
+ run different code than the service does. The gate must say so rather than
+ quietly bind a stub."""
+ rel = "airgap.py"
+ (tmp_path / rel).write_text(
+ "_PRIVATE_SUFFIXES = ('.local',)\n"
+ "def _is_private_address(host):\n"
+ " return host.endswith(_PRIVATE_SUFFIXES) or host in settings.EXTRA\n",
+ encoding="utf-8",
+ )
+ with pytest.raises(gate.GateError, match="settings"):
+ gate.load_shipped_predicate(tmp_path, rel)
+
+
+ ("body", "expected"),
+ [
+ ("def _other(host):\n return True\n", "no longer defines _is_private_address"),
+ ("def _is_private_address(host):\n return True\n", "no longer defines _PRIVATE_SUFFIXES"),
+ ],
+)
+def test_the_lift_fails_loudly_when_the_shipped_names_move(tmp_path: Path, body: str, expected: str) -> None:
+ """A rename in the service must break this gate rather than fork it."""
+ (tmp_path / "airgap.py").write_text(body, encoding="utf-8")
+ with pytest.raises(gate.GateError, match=expected):
+ gate.load_shipped_predicate(tmp_path, "airgap.py")
+
+
+# --------------------------------------------------------------------------
+# Non-vacuity, over the real tree.
+# --------------------------------------------------------------------------
+
+
+def test_the_real_scan_reaches_a_plausible_corpus() -> None:
+ """Found-nothing and scanned-nothing print the same word unless something
+ counts what was opened."""
+ paths = gate.tracked_files(REPO_ROOT)
+ defaults, modules = gate.scan(REPO_ROOT, paths)
+ assert len(modules) >= gate.MIN_SETTINGS_MODULES
+ assert len(defaults) >= gate.MIN_URL_DEFAULTS
+ is_private = gate.load_shipped_predicate(REPO_ROOT, gate.PREDICATE_MODULE)
+ assert any(is_private(d.host) for d in defaults), "no default classified private — the classifier is stuck"
+
+
+def test_every_allow_list_entry_names_at_least_one_guard() -> None:
+ """An exemption with no enforcement is a permanent pass with a comment on it."""
+ for entry in gate.ALLOWED:
+ assert entry.enforced_by, f"{entry.module}::{entry.setting} is exempt with no guard named"
+ assert entry.reason.strip(), f"{entry.module}::{entry.setting} is exempt with no reason given"
+
+
+def test_the_real_tree_is_clean_and_every_exemption_is_still_enforced() -> None:
+ paths = gate.tracked_files(REPO_ROOT)
+ defaults, _ = gate.scan(REPO_ROOT, paths)
+ is_private = gate.load_shipped_predicate(REPO_ROOT, gate.PREDICATE_MODULE)
+ assert gate.compare(defaults, gate.ALLOWED, is_private, root=REPO_ROOT, paths=paths) == []
diff --git a/services/purple-team/app/core/config.py b/services/purple-team/app/core/config.py
index 31a746e94..2bd71a11a 100644
--- a/services/purple-team/app/core/config.py
+++ b/services/purple-team/app/core/config.py
@@ -39,8 +39,14 @@ class Settings(BaseSettings):
art_repo_path: str = "/opt/atomic-red-team"
art_atomics_path: str = "/opt/atomic-red-team/atomics"
- # ATT&CK STIX bundle URL (for coverage mapping)
- attack_stix_url: str = "https://raw.githubusercontent.com/mitre/cti/master/enterprise-attack/enterprise-attack.json"
+ # There used to be an ``attack_stix_url`` here, defaulting to the MITRE CTI
+ # bundle on a public CDN "for coverage mapping". Nothing in this service
+ # ever read it — not the coverage mapper, not the drift scheduler, not the
+ # routes. A dead setting is not harmless when its default is an external
+ # host: it is documented, it is settable, and it makes reaching a CDN look
+ # like the decision somebody already took. The services that do need the
+ # bundle (``agents``, ``ingest``) read it from ``ATTCK_DATA_PATH`` on disk.
+ # If coverage mapping ever needs it here, take the same local-file route.
# OTel
otel_endpoint: str = "http://localhost:4317"
diff --git a/tests/test_no_default_egress.py b/tests/test_no_default_egress.py
new file mode 100644
index 000000000..1acca9b1b
--- /dev/null
+++ b/tests/test_no_default_egress.py
@@ -0,0 +1,488 @@
+"""Boot every service with no configuration and watch the sockets.
+
+Why this exists
+---------------
+``scripts/check_default_egress.py`` reads the *declared* defaults and is
+exhaustive over them, but a declared default is not the only way a process
+reaches the internet: a module-scope ``httpx.get``, a client constructed at
+import with a literal URL, or a lifespan that warms a cache from a CDN are all
+invisible to it. This is the other half — the one the claim-to-gate matrix row
+actually asks for, and the one the repository had never had: *a defined set of
+services makes zero outbound network calls by default.*
+
+How it works
+------------
+One subprocess per service, because every service ships a package literally
+named ``app`` and two of them cannot occupy one interpreter. In each:
+
+1. the socket guard goes in **first**, before any service code is imported;
+2. the guard proves itself by dialling a public host and requiring a raise —
+ a patch that stopped patching would otherwise report a clean run;
+3. ``app.main`` is imported with a minimal environment;
+4. the ASGI lifespan is entered and exited, because startup is where a warm-up
+ fetch lives.
+
+Every attempt is **recorded as well as refused**. Recording matters more than
+refusing: a service that wraps its startup fetch in ``except Exception`` would
+swallow the guard's error and look clean, so the verdict is built from the
+ledger the guard keeps rather than from whether the process survived.
+
+What counts as a violation
+--------------------------
+Reaching a *public* host. Loopback passes through untouched — a service dialling
+its own Postgres is the product working. Private and internal destinations
+(RFC1918, ``.local``/``.internal``, single-label container names like ``redis``)
+are refused by the guard so nothing escapes the test host, but they are reported
+rather than failed: dialling ``redis:6379`` is what a self-hosted deployment is
+supposed to do. Public or private is decided by the same predicate the services
+enforce air-gap policy with, lifted out of ``services/api/app/core/airgap.py`` by
+the static gate, so the two halves cannot disagree about what "internal" means.
+
+Honest limits
+-------------
+* Refusing a non-loopback connection can cut a startup path short, so a service
+ that dials Kafka before it would have dialled a CDN is only covered up to the
+ first refusal. Every refusal is printed, so the truncation is visible rather
+ than silently counted as coverage.
+* A service whose third-party dependencies are not installed is **skipped with
+ the missing module named**, never silently. The covered set is asserted
+ non-empty, because a suite that exercises nothing and prints green is the
+ precise failure this repository has scars from.
+* Neither this nor the static gate says anything about what the *running
+ containers* do. That needs egress-blocked integration, which the Helm chart's
+ default-deny NetworkPolicy addresses at deploy time and nothing addresses in
+ CI.
+"""
+
+from __future__ import annotations
+
+import concurrent.futures
+import importlib.util
+import json
+import os
+import socket
+import subprocess
+import sys
+from functools import lru_cache
+from pathlib import Path
+from typing import Any
+
+import pytest
+
+REPO = Path(__file__).resolve().parents[1]
+
+#: How long one service gets to import and run its lifespan. Generous: a
+#: service retrying a refused loopback connection with backoff is doing the
+#: right thing, and the point is to observe it, not to race it.
+PROBE_TIMEOUT_SECONDS = 120
+
+#: The smallest environment that is still an *unconfigured* one. Two secrets
+#: and an environment name, because several services refuse to construct their
+#: settings without them and a service that will not start is not evidence of
+#: anything. Deliberately contains no URL of any kind: every address these
+#: probes observe therefore came from the service's own defaults.
+MINIMAL_ENV = {
+ "ENVIRONMENT": "development",
+ "SECRET_KEY": "offline-egress-probe-secret-key-32bytes!",
+ "JWT_SECRET": "offline-egress-probe-secret-key-32bytes!",
+ "AISOC_SERVICE_TOKEN": "offline-egress-probe-token",
+ "PYTHONDONTWRITEBYTECODE": "1",
+ "PYTHONUNBUFFERED": "1",
+}
+
+
+# --------------------------------------------------------------------------
+# The guard. One source, executed in-process by the self-check below and in
+# every probe subprocess, so the thing proven and the thing used are the same.
+# --------------------------------------------------------------------------
+
+GUARD_SOURCE = r'''
+import socket
+
+class EgressBlocked(OSError):
+ """Raised instead of opening a socket to anywhere but loopback."""
+
+#: Destinations that are the test host itself. A service talking to these is
+#: talking to nothing outside the process tree.
+_LOOPBACK = {"127.0.0.1", "::1", "localhost", "0.0.0.0", "::", "", "<broadcast>"}
+
+#: Every destination the guard was asked for, whether or not the caller
+#: survived being refused. This ledger, not the exception, is the verdict.
+ATTEMPTS = []
+
+_REAL = {
+ "connect": socket.socket.connect,
+ "connect_ex": socket.socket.connect_ex,
+ "create_connection": socket.create_connection,
+ "getaddrinfo": socket.getaddrinfo,
+}
+
+
+def _host_of(address):
+ if isinstance(address, (tuple, list)) and address:
+ return address[0]
+ return address
+
+
+def _normalise(host):
+ if host is None:
+ return ""
+ text = str(host).strip().lower().rstrip(".")
+ # IPv6 literals arrive with a scope id on some platforms.
+ return text.split("%", 1)[0]
+
+
+def _check(kind, host):
+ """Record, then refuse anything that is not loopback."""
+ name = _normalise(host)
+ if name in _LOOPBACK:
+ return
+ ATTEMPTS.append({"kind": kind, "host": name})
+ raise EgressBlocked(
+ "offline egress guard refused a %s to %r: this test asserts a service "
+ "reaches nothing but loopback when it is unconfigured" % (kind, name)
+ )
+
+
+def _connect(self, address):
+ if getattr(self, "family", None) == getattr(socket, "AF_UNIX", object()):
+ return _REAL["connect"](self, address)
+ _check("connect", _host_of(address))
+ return _REAL["connect"](self, address)
+
+
+def _connect_ex(self, address):
+ if getattr(self, "family", None) == getattr(socket, "AF_UNIX", object()):
+ return _REAL["connect_ex"](self, address)
+ _check("connect_ex", _host_of(address))
+ return _REAL["connect_ex"](self, address)
+
+
+def _create_connection(address, *args, **kwargs):
+ _check("create_connection", _host_of(address))
+ return _REAL["create_connection"](address, *args, **kwargs)
+
+
+def _getaddrinfo(host, *args, **kwargs):
+ _check("getaddrinfo", host)
+ return _REAL["getaddrinfo"](host, *args, **kwargs)
+
+
+def install_guard():
+ socket.socket.connect = _connect
+ socket.socket.connect_ex = _connect_ex
+ socket.create_connection = _create_connection
+ socket.getaddrinfo = _getaddrinfo
+
+
+def remove_guard():
+ socket.socket.connect = _REAL["connect"]
+ socket.socket.connect_ex = _REAL["connect_ex"]
+ socket.create_connection = _REAL["create_connection"]
+ socket.getaddrinfo = _REAL["getaddrinfo"]
+
+
+def guard_is_live():
+ """Dial a public host and require a refusal.
+
+ A patch that silently stopped patching turns every probe below into a
+ clean run over an unguarded socket, which is indistinguishable from a
+ service that behaved. So the guard is made to prove itself first.
+ """
+ before = len(ATTEMPTS)
+ try:
+ socket.create_connection(("example.com", 80), timeout=1)
+ except EgressBlocked:
+ del ATTEMPTS[before:]
+ return True
+ except Exception:
+ del ATTEMPTS[before:]
+ return False
+ del ATTEMPTS[before:]
+ return False
+'''
+
+
+PROBE_SOURCE = (
+ GUARD_SOURCE
+ + r"""
+import asyncio
+import json
+import sys
+import traceback
+
+SERVICE_ROOT = sys.argv[1]
+LIFESPAN_TIMEOUT = float(sys.argv[2])
+
+report = {
+ "guard_live": False,
+ "imported": False,
+ "skip_reason": None,
+ "lifespan": "not attempted",
+ "attempts": [],
+ "error": None,
+}
+
+install_guard()
+report["guard_live"] = guard_is_live()
+if not report["guard_live"]:
+ print(json.dumps(report))
+ raise SystemExit(0)
+
+sys.path.insert(0, SERVICE_ROOT)
+
+try:
+ import app.main as service_main
+ report["imported"] = True
+except ModuleNotFoundError as exc:
+ # A third-party package this environment does not have is a skip with a
+ # name. A module inside the service itself failing to import is a real
+ # defect and is reported as an error, not excused.
+ missing = (exc.name or "").split(".")[0]
+ if missing and missing != "app":
+ report["skip_reason"] = "requires the %r package, which is not installed here" % missing
+ else:
+ report["error"] = "".join(traceback.format_exception_only(type(exc), exc)).strip()
+except BaseException as exc: # noqa: BLE001 - any failure to boot is the finding
+ report["error"] = "".join(traceback.format_exception_only(type(exc), exc)).strip()
+
+if report["imported"]:
+ asgi = getattr(service_main, "app", None)
+ router = getattr(asgi, "router", None)
+ factory = getattr(router, "lifespan_context", None)
+ if factory is None:
+ report["lifespan"] = "no ASGI app exposed as app.main:app"
+ else:
+ async def drive():
+ context = factory(asgi)
+ try:
+ await context.__aenter__()
+ finally:
+ try:
+ await context.__aexit__(None, None, None)
+ except BaseException:
+ pass
+
+ try:
+ asyncio.run(asyncio.wait_for(drive(), timeout=LIFESPAN_TIMEOUT))
+ report["lifespan"] = "completed"
+ except BaseException as exc: # noqa: BLE001 - startup failing offline is expected
+ report["lifespan"] = "raised %s" % "".join(traceback.format_exception_only(type(exc), exc)).strip()
+
+report["attempts"] = ATTEMPTS
+print(json.dumps(report))
+"""
+)
+
+
+# --------------------------------------------------------------------------
+# Host classification, borrowed from the static gate so both halves agree.
+# --------------------------------------------------------------------------
+
+
+@lru_cache(maxsize=1)
+def _is_private_host():
+ """The predicate the services themselves enforce air-gap policy with.
+
+ Loaded through the static gate rather than reimplemented, so "internal"
+ means the same thing in both halves of this claim and in the service code
+ all three describe.
+ """
+ gate = REPO / "scripts" / "check_default_egress.py"
+ spec = importlib.util.spec_from_file_location("check_default_egress", gate)
+ assert spec is not None and spec.loader is not None
+ module = importlib.util.module_from_spec(spec)
+ sys.modules[spec.name] = module
+ spec.loader.exec_module(module)
+ return module.load_shipped_predicate(REPO, module.PREDICATE_MODULE)
+
+
+# --------------------------------------------------------------------------
+# Discovering the services, and probing them.
+# --------------------------------------------------------------------------
+
+
+def discovered_services() -> list[str]:
+ """Every service exposing ``app/main.py``, read off disk rather than listed.
+
+ Derived, not enumerated: a hand-kept list is correct the day it is written,
+ and this repository has paid for that several times over — 56 of 82 agents
+ test files run by nothing, a connector union naming ten types the platform
+ does not ingest. A service added tomorrow is covered tomorrow.
+ """
+ return sorted(p.parents[1].name for p in (REPO / "services").glob("*/app/main.py"))
+
+
+def _probe(service: str) -> dict:
+ root = REPO / "services" / service
+ env = {
+ "PATH": os.environ.get("PATH", ""),
+ "HOME": os.environ.get("HOME", ""),
+ "LANG": os.environ.get("LANG", "C.UTF-8"),
+ **MINIMAL_ENV,
+ }
+ try:
+ done = subprocess.run(
+ [sys.executable, "-c", PROBE_SOURCE, str(root), str(PROBE_TIMEOUT_SECONDS / 2)],
+ capture_output=True,
+ text=True,
+ timeout=PROBE_TIMEOUT_SECONDS,
+ env=env,
+ cwd=str(root),
+ )
+ except subprocess.TimeoutExpired:
+ return {"skip_reason": None, "error": f"probe did not finish within {PROBE_TIMEOUT_SECONDS}s", "attempts": []}
+
+ payload = next((ln for ln in reversed(done.stdout.splitlines()) if ln.startswith("{")), None)
+ if payload is None:
+ tail = (done.stderr or done.stdout or "").strip().splitlines()[-4:]
+ return {"skip_reason": None, "error": "probe produced no report: " + " / ".join(tail), "attempts": []}
+ result = json.loads(payload)
+ result["stderr_tail"] = (done.stderr or "").strip().splitlines()[-3:]
+ return result
+
+
+@lru_cache(maxsize=1)
+def probe_results() -> dict[str, dict]:
+ """Probe every service once, in parallel, and cache the verdicts."""
+ services = discovered_services()
+ with concurrent.futures.ThreadPoolExecutor(max_workers=min(8, len(services) or 1)) as pool:
+ return dict(zip(services, pool.map(_probe, services), strict=True))
+
+
+def public_attempts(result: dict) -> list[dict]:
+ is_private = _is_private_host()
+ return [a for a in result.get("attempts", []) if not is_private(a["host"])]
+
+
+# --------------------------------------------------------------------------
+# The tests.
+# --------------------------------------------------------------------------
+
+
+def test_the_socket_guard_actually_blocks() -> None:
+ """The guard refuses a public destination, in this very interpreter.
+
+ Run before anything trusts a probe's clean report. A guard that stopped
+ guarding makes every result below a clean run over an open socket, which
+ reads exactly like a service that behaved.
+ """
+ # ``Any`` rather than ``object``: this namespace is populated by exec, so
+ # its members are callables and classes the type checker cannot see.
+ namespace: dict[str, Any] = {}
+ exec(compile(GUARD_SOURCE, "<egress guard>", "exec"), namespace) # noqa: S102 - this module's own source
+
+ namespace["install_guard"]()
+ try:
+ assert namespace["guard_is_live"](), "the guard did not refuse a connection to a public host"
+
+ with pytest.raises(namespace["EgressBlocked"]):
+ socket.create_connection(("example.com", 80), timeout=1)
+ with pytest.raises(namespace["EgressBlocked"]):
+ socket.getaddrinfo("example.com", 443)
+ with pytest.raises(namespace["EgressBlocked"]):
+ socket.socket(socket.AF_INET, socket.SOCK_STREAM).connect(("93.184.216.34", 80))
+
+ # …and it must stay out of the way of loopback, or every service would
+ # look like a violation and the test would prove nothing.
+ assert namespace["_normalise"]("LOCALHOST.") == "localhost"
+ namespace["_check"]("connect", "127.0.0.1")
+ namespace["_check"]("connect", "::1")
+ finally:
+ namespace["remove_guard"]()
+
+ # The ledger records even when the caller swallows the refusal, which is
+ # the property that makes a try/except around a startup fetch detectable.
+ assert [a["host"] for a in namespace["ATTEMPTS"]] == ["example.com", "example.com", "93.184.216.34"]
+
+
[email protected]("service", discovered_services())
+def test_service_reaches_no_public_host_when_unconfigured(service: str) -> None:
+ result = probe_results()[service]
+
+ if result.get("skip_reason"):
+ pytest.skip(f"{service}: {result['skip_reason']}")
+
+ assert result.get("guard_live"), f"{service}: the socket guard was not active in the probe, so its clean run means nothing"
+
+ # The ledger is read before anything else, because a service that fails to
+ # boot *because* it tried to reach a CDN should be reported as reaching a
+ # CDN. Checking the boot error first would hide the specific finding behind
+ # the generic one it caused.
+ public = public_attempts(result)
+ assert not public, (
+ f"{service} reached {len(public)} public host(s) with no configuration set: "
+ + ", ".join(f"{a['host']} (via {a['kind']})" for a in public)
+ + f". Import: {'ok' if result.get('imported') else 'failed'}. Lifespan: {result.get('lifespan')}."
+ )
+
+ assert not result.get("error"), f"{service} could not be booted offline: {result['error']}"
+
+
+def test_the_probe_set_is_not_empty_and_is_reported() -> None:
+ """Fail — never skip — when nothing was exercised, and say what was.
+
+ A suite that covers nothing while printing green is the failure this
+ repository keeps rediscovering: a playbook linter reporting "2/2 passed"
+ over a set it never assembled, a weekly eval green on eight consecutive
+ runs with every real step skipped. So the covered set is asserted, and the
+ skipped set is printed with its reasons rather than disappearing.
+ """
+ results = probe_results()
+ covered, skipped, broken = [], [], []
+ for service, result in sorted(results.items()):
+ if result.get("skip_reason"):
+ skipped.append((service, result["skip_reason"]))
+ elif result.get("error"):
+ broken.append((service, result["error"]))
+ else:
+ covered.append(service)
+
+ print(f"\noffline egress probe — {len(results)} service(s) discovered from services/*/app/main.py")
+ for service in covered:
+ result = results[service]
+ internal = ", ".join(sorted({a["host"] for a in result.get("attempts", [])})) or "nothing"
+ print(f" COVERED {service:<14} lifespan: {result.get('lifespan')}")
+ print(f" {'':<14} dialled: {internal}")
+ for service, reason in skipped:
+ print(f" SKIPPED {service:<14} {reason}")
+ for service, reason in broken:
+ print(f" BROKEN {service:<14} {reason}")
+
+ assert covered, (
+ "no service was exercised — every one of "
+ f"{len(results)} was skipped or failed to boot, so a green run here proves nothing. "
+ + "; ".join(f"{s}: {r}" for s, r in skipped + broken)
+ )
+
+
+def test_every_discovered_service_is_accounted_for() -> None:
+ """Discovery must find the services, and every one must reach a verdict.
+
+ The floor is deliberately low and the direction is the one that breaks: a
+ glob that stops matching returns nothing and every assertion above passes
+ vacuously.
+ """
+ services = discovered_services()
+ assert len(services) >= 10, f"only {len(services)} service(s) found under services/*/app/main.py; the glob is broken, not the tree"
+ assert "api" in services and "threatintel" in services, f"discovery missed a service that certainly has one: {services}"
+ assert set(probe_results()) == set(services), "a discovered service reached no verdict"
+
+
+def test_the_probe_environment_carries_no_urls() -> None:
+ """Every address a probe observes must come from the service, not from us.
+
+ A probe env containing ``DATABASE_URL=...`` would let the harness supply
+ the very default under test, and a clean result would say nothing about
+ what the service ships.
+ """
+ offenders = {k: v for k, v in MINIMAL_ENV.items() if "://" in v}
+ assert not offenders, f"the minimal probe environment supplies URLs, which would mask the defaults under test: {offenders}"
+
+ # The probe itself must name no destination either, or the guard's ledger
+ # would record an address the harness supplied. `example.com` is the one
+ # exception: it is the host the guard dials to prove it is still guarding,
+ # and RFC 2606 reserves it precisely so it can never resolve to anyone.
+ urls = [line.strip() for line in PROBE_SOURCE.splitlines() if "://" in line]
+ assert not urls, f"the probe source names a destination, which would mask the defaults under test: {urls}"
+ assert PROBE_SOURCE.count("example.com") == 1, "the guard's self-check host should be the only host the probe names"