Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

fix(enrichment): point the two remaining callers at the service that serves it (#977)

Beenu Arora155ee3918e71
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index f91d39597..ea030fbb0 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -705,6 +705,16 @@ jobs:
run: |
python3 scripts/check_published_secrets.py --self-test
python3 scripts/check_published_secrets.py --check
+ # The IOC lookup's `catch` had nowhere to put "this did not work" except
+ # the flag that means "we checked, and it is clean", so a transport
+ # error, a 5xx and a route that did not exist all rendered a green CLEAN
+ # badge over the indicator an analyst had just typed in. It was not a
+ # latent branch: the route did not exist, so every lookup took it. That
+ # is a shape rather than a bug in one file, and this is the shape.
+ - name: A backend failure never renders as a safety verdict
+ run: |
+ python3 scripts/check_failure_is_not_a_verdict.py --self-test
+ python3 scripts/check_failure_is_not_a_verdict.py
# The marketplace index has to exist in three build contexts, none of
# which can see the other two. The API's copy was missing, so its image
# shipped no catalogue and the marketplace answered 503 on every
diff --git a/apps/web/next.config.js b/apps/web/next.config.js
index 6b35fc7e2..046377c7f 100644
--- a/apps/web/next.config.js
+++ b/apps/web/next.config.js
@@ -40,7 +40,7 @@ const AGENTS_HOST = process.env.AGENTS_URL || 'http://localhost:8001';
// through to the core API catch-all, which exposes a graceful fusion
// gateway (services/api/app/api/v1/endpoints/fusion.py).
const FUSION_HOST = process.env.FUSION_URL || '';
-const ENRICHMENT_HOST = process.env.ENRICHMENT_URL || 'http://localhost:8083';
+const ENRICHMENT_HOST = process.env.ENRICHMENT_URL || 'http://enrichment:8082';
const OSQUERY_TLS_HOST = process.env.OSQUERY_TLS_URL || 'http://localhost:8090';
const nextConfig = {
@@ -205,7 +205,19 @@ const nextConfig = {
source: '/api/v1/osquery/:path*',
destination: `${OSQUERY_TLS_HOST}/api/v1/osquery/:path*`,
},
- // Enrichment service (Go service; paths differ from /api/v1 prefix)
+ // Enrichment service (Go service; paths differ from /api/v1 prefix).
+ //
+ // Nothing in the console calls these today — `threatIntelApi.lookup`
+ // reads the tenant's own threat-intel store, and the `bulkLookup` that
+ // targeted the second one is gone. They stay because this is how a
+ // console feature reaches enrichment, and `ENRICHMENT_URL` is in
+ // `scripts/resolve-runtime-routes.mjs`, so a pulled image can be
+ // pointed at a real host without a rebuild.
+ //
+ // The fallback below is `http://enrichment:8082`, the Compose service
+ // and the port it listens on. It used to be `http://localhost:8083`,
+ // which inside the web container is the console itself — a rewrite
+ // that answered from the wrong server rather than failing.
{
source: '/api/v1/enrichment/lookup',
destination: `${ENRICHMENT_HOST}/enrich`,
diff --git a/apps/web/src/lib/api.ts b/apps/web/src/lib/api.ts
index 31d85ac94..a8a1b8ffc 100644
--- a/apps/web/src/lib/api.ts
+++ b/apps/web/src/lib/api.ts
@@ -3113,11 +3113,17 @@ export const threatIntelApi = {
return match ? { status: 'match', indicator: match } : { status: 'clean' };
},
- bulkLookup: (iocs: string[]) =>
- request<{ results: IOCLookup[] }>('/api/v1/enrichment/bulk', {
- method: 'POST',
- body: JSON.stringify({ iocs }),
- }),
+ // `bulkLookup` used to live here. It posted `{ iocs: [...] }` to
+ // `/api/v1/enrichment/bulk` and had **zero callers**. That path is not
+ // served by the API — measured against a running stack, 404 — and the
+ // request shape was not the one the enrichment service reads either
+ // (`POST /enrich/bulk` takes `{ items: [...] }`). It is deleted rather
+ // than repointed: an exported helper aimed at a route that does not exist
+ // is a loaded gun, because whoever wires it up first gets a rejected
+ // promise and, following the shape `lookup` used to have, renders it as a
+ // clean verdict. A bulk lookup can be written against
+ // `/api/v1/threat-intel/indicators` — the route `lookup` uses — when
+ // something needs one.
// `total` is the store's count of indicators in scope; `shown` is how many
// this response carries. They are different numbers and the console renders
diff --git a/scripts/check_failure_is_not_a_verdict.py b/scripts/check_failure_is_not_a_verdict.py
new file mode 100644
index 000000000..e0b4ba176
--- /dev/null
+++ b/scripts/check_failure_is_not_a_verdict.py
@@ -0,0 +1,161 @@
+#!/usr/bin/env python3
+"""A backend failure must never render as a safety verdict.
+
+The shape
+---------
+`LookupForm` held a `result` and a `notFound` flag, and its `catch` had
+nowhere to put *"this did not work"* except the flag that means *"we checked,
+and it is clean"*::
+
+ try {
+ const r = await threatIntelApi.lookup(query.trim());
+ setResult(r);
+ } catch {
+ setNotFound(true); // -> renders a green CLEAN badge
+ }
+
+So a transport error, a 5xx, or a route that did not exist all rendered as an
+all-clear on the indicator the analyst had just typed in. It was not a latent
+branch: the lookup called `/api/v1/enrichment/lookup`, which the API does not
+serve, so **every** lookup took the `catch` and the panel had only ever been
+capable of saying "clean".
+
+This is the worst direction for a security product to fail in, and it is a
+*shape* rather than a bug in one file — which is why it is gated rather than
+only fixed.
+
+What is flagged
+---------------
+A `catch` (or a rejection handler, or a `?? FALLBACK` on a failed read) whose
+body assigns a **verdict word** — clean, safe, benign, not malicious, no
+threats found. Those words are claims about the subject. A failure knows
+nothing about the subject.
+
+What is not flagged
+-------------------
+Setting an error state, rendering "not checked", rethrowing, logging, or
+assigning an empty list. An empty *list* is a different claim from a clean
+*verdict*: "we hold no rows" is about the store, and the honesty gates for
+mock data already cover a catch that invents rows.
+
+Stdlib only, and a regex over the source rather than a parse: the corpus is
+TypeScript and a TS parser is not available to a Python gate here. The cost
+is that this sees text; the mitigation is the self-test corpus below, which
+holds a known-bad and a known-good example of every pattern and asserts the
+matcher separates them.
+"""
+
+from __future__ import annotations
+
+import argparse
+import re
+import sys
+from pathlib import Path
+
+# `scripts/` is on sys.path when this file is run as a program, but not when a
+# test loads it by path with importlib. gate_toolkit sits beside it either way.
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+
+from gate_toolkit import repo_root, self_test_if_requested
+
+self_test_if_requested(__file__)
+
+#: Words that assert the subject is safe. Matched against what a failure
+#: handler *assigns*, never against prose.
+VERDICT_RE = re.compile(
+ r"""
+ (?: setNotFound\s*\(\s*true
+ | setClean\s*\(\s*true
+ | \bstatus\s*:\s*['"](?:clean|safe|benign|ok)['"]
+ | \bverdict\s*:\s*['"](?:clean|safe|benign|not_malicious|no_threat)['"]
+ | \bmalicious\s*:\s*false
+ | \bisMalicious\s*:\s*false
+ | \bthreat\s*:\s*['"](?:none|clean)['"]
+ )
+ """,
+ re.VERBOSE,
+)
+
+#: A `catch` block, including the bare form the defect used.
+#:
+#: The inner alternation allows one level of nesting, which is not
+#: decoration: `catch { setResult({ status: 'clean' }) }` is the shape and a
+#: flat `[^{}]*` body misses it. Two levels is beyond a regex and beyond this
+#: gate — the corpus in `scripts/tests/` is what says which shapes are
+#: covered.
+CATCH_RE = re.compile(r"catch\s*(?:\([^)]*\))?\s*\{((?:[^{}]|\{[^{}]*\})*)\}", re.DOTALL)
+
+#: `await x().catch(() => VERDICT)` and `.then(…, () => VERDICT)`.
+REJECTION_RE = re.compile(r"\.catch\s*\(\s*\(?[^)]*\)?\s*=>\s*([^;\n]*)", re.DOTALL)
+
+#: `data ?? { status: 'clean' }` — a fallback used when a read produced
+#: nothing, which includes when it failed.
+FALLBACK_RE = re.compile(r"\?\?\s*(\{[^{}]*\})")
+
+SEARCH_ROOTS = ("apps/web/src", "packages/ui/src")
+SUFFIXES = (".ts", ".tsx")
+
+#: Files whose *subject* is the defect: the honesty test that asserts a
+#: failure is not rendered as clean has to name the thing it forbids.
+ALLOWED = ("ThreatIntelLookupHonesty.test.tsx",)
+
+
+def _findings(text: str, rel: str) -> list[str]:
+ hits: list[str] = []
+ for pattern, label in ((CATCH_RE, "catch"), (REJECTION_RE, "rejection handler"), (FALLBACK_RE, "?? fallback")):
+ for match in pattern.finditer(text):
+ body = match.group(1)
+ verdict = VERDICT_RE.search(body)
+ if not verdict:
+ continue
+ line = text[: match.start()].count("\n") + 1
+ hits.append(f"{rel}:{line} a {label} assigns {verdict.group(0).strip()!r} — a failure is not a verdict about the subject")
+ return hits
+
+
+def scan(root: Path | None = None) -> tuple[list[str], int]:
+ root = root or repo_root()
+ findings: list[str] = []
+ scanned = 0
+ for base in SEARCH_ROOTS:
+ directory = root / base
+ if not directory.is_dir():
+ continue
+ for path in sorted(directory.rglob("*")):
+ if path.suffix not in SUFFIXES or not path.is_file():
+ continue
+ if path.name in ALLOWED:
+ continue
+ scanned += 1
+ findings += _findings(path.read_text(encoding="utf-8"), str(path.relative_to(root)))
+ return findings, scanned
+
+
+def main(argv: list[str] | None = None) -> int:
+ parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+ parser.add_argument("--check", action="store_true", help="render the verdict (default)")
+ parser.parse_args(argv)
+
+ findings, scanned = scan()
+ if not scanned:
+ print(
+ "check_failure_is_not_a_verdict: scanned zero files. A clean result over an empty scan is worse than no gate.",
+ file=sys.stderr,
+ )
+ return 2
+ if findings:
+ print(f"FAIL — {len(findings)} place(s) render a failure as a safety verdict:", file=sys.stderr)
+ for finding in findings:
+ print(f" - {finding}", file=sys.stderr)
+ print(
+ "\nReport the failure and name what was not checked. `status: 'failed'` with a reason "
+ "is the shape `threatIntelApi.lookup` uses.",
+ file=sys.stderr,
+ )
+ return 1
+ print(f"OK: {scanned} console source files, none turn a failure into a clean verdict.")
+ return 0
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/scripts/tests/test_check_failure_is_not_a_verdict.py b/scripts/tests/test_check_failure_is_not_a_verdict.py
new file mode 100644
index 000000000..23700fcc1
--- /dev/null
+++ b/scripts/tests/test_check_failure_is_not_a_verdict.py
@@ -0,0 +1,103 @@
+"""The matcher has to separate a failure-as-verdict from an honest failure.
+
+`check_failure_is_not_a_verdict.py` reads TypeScript with a regex, because a
+TS parser is not available to a Python gate here. A regex over source sees
+text, so the thing worth testing is not "does it flag something" but "does it
+flag the right thing and leave the right thing alone".
+
+Both directions are here. A matcher that flagged every `catch` would pass a
+one-sided suite and make the gate unusable on the first honest error handler
+somebody wrote.
+"""
+
+from __future__ import annotations
+
+import importlib.util
+import sys
+from pathlib import Path
+
+import pytest
+
+SCRIPTS = Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(SCRIPTS))
+
+spec = importlib.util.spec_from_file_location("check_failure_is_not_a_verdict", SCRIPTS / "check_failure_is_not_a_verdict.py")
+assert spec and spec.loader
+gate = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(gate)
+
+
+#: Each is a real shape. The first is the one that shipped.
+BAD = {
+ "the bare catch that shipped": """
+ try {
+ const r = await threatIntelApi.lookup(query.trim());
+ setResult(r);
+ } catch {
+ setNotFound(true);
+ }
+ """,
+ "a named catch": """
+ try { await check(ioc); } catch (err) { setClean(true); }
+ """,
+ "a verdict object in a catch": """
+ try { r = await scan(h); } catch (e) { setResult({ status: 'clean' }); }
+ """,
+ "a rejection handler": """
+ const r = await scan(hash).catch(() => ({ verdict: 'benign' }));
+ """,
+ "a nullish fallback to a verdict": """
+ const outcome = data?.result ?? { malicious: false };
+ """,
+ "a not-malicious flag in a catch": """
+ try { await lookup(ip); } catch { setState({ isMalicious: false }); }
+ """,
+}
+
+GOOD = {
+ "naming the failure": """
+ try { setResult(await lookup(ioc)); }
+ catch (err) { setError(`Lookup failed — ${err.message}. This says nothing about ${ioc}.`); }
+ """,
+ "rethrowing": """
+ try { await lookup(ioc); } catch (err) { throw err; }
+ """,
+ "logging only": """
+ try { await refresh(); } catch (err) { console.warn('refresh failed', err); }
+ """,
+ "an empty list is not a verdict": """
+ try { setRows(await list()); } catch { setRows([]); setError('could not load'); }
+ """,
+ "a three-state outcome": """
+ try { return { status: 'match', indicator }; }
+ catch { return { status: 'failed', reason: 'the service could not be reached' }; }
+ """,
+ "prose that mentions clean": """
+ // A failed lookup must never render as clean. status: 'clean' is only
+ // for an answer from the store.
+ try { await lookup(ioc); } catch { setError('not checked'); }
+ """,
+ "a clean verdict on the success path": """
+ const response = await request('/api/v1/threat-intel/indicators');
+ return match ? { status: 'match', indicator: match } : { status: 'clean' };
+ """,
+}
+
+
[email protected]("label", sorted(BAD))
+def test_the_dangerous_shape_is_flagged(label: str) -> None:
+ assert gate._findings(BAD[label], "probe.tsx"), f"{label!r} was not flagged"
+
+
[email protected]("label", sorted(GOOD))
+def test_an_honest_handler_is_not(label: str) -> None:
+ findings = gate._findings(GOOD[label], "probe.tsx")
+ assert not findings, f"{label!r} was flagged: {findings}"
+
+
+def test_the_live_tree_is_clean() -> None:
+ """The gate's own subject. Reported with the file count, so a scan that
+ silently read nothing is not mistaken for a clean tree."""
+ findings, scanned = gate.scan()
+ assert scanned > 100, f"only {scanned} files scanned — the roots are wrong"
+ assert not findings, findings
diff --git a/services/agents/app/playbook/engine.py b/services/agents/app/playbook/engine.py
index a7c1d16eb..a2e1bbb40 100644
--- a/services/agents/app/playbook/engine.py
+++ b/services/agents/app/playbook/engine.py
@@ -41,6 +41,13 @@ _REALTIME_URL = os.getenv("REALTIME_URL", "http://realtime:3001")
_INTERNAL_TOKEN = os.getenv("REALTIME_INTERNAL_TOKEN", "")
_API_URL = os.getenv("API_URL", "http://api:8000")
+#: The service that actually serves IOC enrichment: `POST /enrich` and
+#: `POST /enrich/bulk` on `services/enrichment`, port 8082. The same address
+#: `app.investigator.tools` reads, and the same default `docker-compose.yml`
+#: gives fusion. The enrich step used to post to
+#: `{API_URL}/api/v1/enrichment/lookup`, which the API has never served.
+_ENRICHMENT_URL = os.getenv("ENRICHMENT_SERVICE_URL", "http://enrichment:8082").rstrip("/")
+
# ---------------------------------------------------------------------------
# Run status
@@ -337,14 +344,40 @@ def _evaluate_expression(expression: str, context: dict[str, Any]) -> bool:
async def _handle_enrich(step: PlaybookStep, context: dict[str, Any], http: httpx.AsyncClient) -> dict:
+ """Look one indicator up in the enrichment service.
+
+ It used to post to ``{API_URL}/api/v1/enrichment/lookup``. The API has
+ never served that path — measured against a running stack, it answers 404
+ — so this step could not enrich anything, and the request shape it sent
+ (``{"ioc": …}``) is not the one the enrichment service accepts either.
+
+ The route that exists is ``POST /enrich`` on ``services/enrichment``,
+ which is what ``app.tools.enrichment`` and ``app.investigator.tools``
+ already call. This is the third caller and it now goes to the same place,
+ with the payload that service actually reads.
+
+ Failure raises rather than returning an empty result. Enrichment is a
+ `full`-profile service, so on a CORE deployment this step *will* fail —
+ and "the enrichment service is not running" must not reach a playbook
+ author as "nothing is known about this indicator". Those are different
+ facts and only one of them is about the indicator.
+ """
ioc = step.params.get("ioc") or context.get("ioc") or context.get("src_ip", "")
ioc_type = step.params.get("ioc_type", "ip")
- r = await http.post(
- f"{_API_URL}/api/v1/enrichment/lookup",
- json={"ioc": ioc, "ioc_type": ioc_type},
- timeout=step.timeout_seconds,
- )
- r.raise_for_status()
+ if not ioc:
+ return {"skipped": True, "reason": "no indicator in the step parameters or the run context"}
+ try:
+ r = await http.post(
+ f"{_ENRICHMENT_URL}/enrich",
+ json={"value": ioc, "ioc_type": ioc_type},
+ timeout=step.timeout_seconds,
+ )
+ r.raise_for_status()
+ except httpx.HTTPError as exc:
+ raise PermanentStepFailure(
+ f"enrichment for {ioc!r} did not run: {exc}. This says nothing about the indicator. "
+ f"The enrichment service is on the `full` profile — check it is reachable at {_ENRICHMENT_URL}."
+ ) from exc
return r.json()
diff --git a/services/agents/tests/test_playbook_enrich_target.py b/services/agents/tests/test_playbook_enrich_target.py
new file mode 100644
index 000000000..3e5768eda
--- /dev/null
+++ b/services/agents/tests/test_playbook_enrich_target.py
@@ -0,0 +1,106 @@
+"""The enrich step has to reach a service that serves it.
+
+`_handle_enrich` posted `{"ioc": …}` to `{API_URL}/api/v1/enrichment/lookup`.
+Measured against a running stack:
+
+ $ curl -s -o /dev/null -w '%{http_code}' -H "Authorization: Bearer $TOKEN" \\
+ 'http://localhost:8000/api/v1/enrichment/lookup?ioc=1.2.3.4'
+ 404
+
+The API has never served that path — it is absent from the OpenAPI document —
+so the step could not enrich anything. The request shape was wrong as well:
+the service that does serve enrichment, `services/enrichment`, reads
+`{"value": …, "ioc_type": …}` on `POST /enrich`.
+
+Two properties are asserted, and the second is the one that matters for a
+security product: an unreachable enrichment service must reach the playbook
+author as a **failure**, not as an empty enrichment. "Nothing is known about
+this indicator" is a claim about the indicator; "the service is not running"
+is not.
+"""
+
+from __future__ import annotations
+
+import httpx
+import pytest
+from app.playbook import engine
+from app.playbook.errors import PermanentStepFailure
+from app.playbook.models import PlaybookStep, StepType
+
+
+def _step(**params) -> PlaybookStep:
+ return PlaybookStep(id="s1", name="enrich", type=StepType.ENRICH, params=params)
+
+
+class _Recorder:
+ """Captures the one request the handler makes."""
+
+ def __init__(self, response: httpx.Response) -> None:
+ self.response = response
+ self.url: str | None = None
+ self.json: dict | None = None
+
+ async def post(self, url, json=None, timeout=None): # noqa: A002, ANN001
+ self.url = str(url)
+ self.json = json
+ self.response.request = httpx.Request("POST", url)
+ if self.response.status_code >= 400:
+ raise httpx.HTTPStatusError("boom", request=self.response.request, response=self.response)
+ return self.response
+
+
+async def test_it_posts_to_the_enrichment_service_not_the_api() -> None:
+ http = _Recorder(httpx.Response(200, json={"value": "1.2.3.4", "reputation": "malicious"}))
+ result = await engine._handle_enrich(_step(ioc="1.2.3.4", ioc_type="ip"), {}, http)
+
+ assert http.url == f"{engine._ENRICHMENT_URL}/enrich", http.url
+ assert "/api/v1/enrichment/lookup" not in (http.url or ""), "the API does not serve that path"
+ assert result["reputation"] == "malicious"
+
+
+async def test_it_sends_the_field_names_that_service_reads() -> None:
+ """`{"ioc": …}` was the old shape. `services/enrichment` reads `value`."""
+ http = _Recorder(httpx.Response(200, json={}))
+ await engine._handle_enrich(_step(ioc="evil.example", ioc_type="domain"), {}, http)
+
+ assert http.json == {"value": "evil.example", "ioc_type": "domain"}
+
+
+ "failure",
+ [
+ httpx.Response(404),
+ httpx.Response(503),
+ httpx.Response(500),
+ ],
+ ids=["404", "503", "500"],
+)
+async def test_an_unreachable_service_fails_the_step_rather_than_returning_nothing(failure: httpx.Response) -> None:
+ """The direction that matters.
+
+ Enrichment is a `full`-profile service, so on a CORE deployment this step
+ *will* fail. Returning `{}` would reach a playbook author as "nothing is
+ known about this indicator", which is a claim about the indicator that
+ nobody made.
+ """
+ http = _Recorder(failure)
+ with pytest.raises(PermanentStepFailure) as raised:
+ await engine._handle_enrich(_step(ioc="1.2.3.4"), {}, http)
+
+ message = str(raised.value)
+ assert "1.2.3.4" in message, "the step must name what it did not check"
+ assert "says nothing about the indicator" in message
+
+
+async def test_a_missing_indicator_is_a_skip_and_not_a_call() -> None:
+ """The other direction from the failure test: a step with nothing to look
+ up must not be reported as a failed enrichment, and must not call out."""
+ http = _Recorder(httpx.Response(200, json={}))
+ result = await engine._handle_enrich(_step(), {}, http)
+
+ assert result["skipped"] is True
+ assert http.url is None, "no indicator, no request"