Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

t23(agents): close LLM-input contract with static bypass gate + coverage (#140)

Beenu Arora4c8f201a7124
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 0d37c6147..01a68f2ad 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
+### LLM input contract — static regression gate (T2.3, v8.0)
+
+Closes T2.3 by adding the missing **bypass-prevention** layer on top of the
+existing fail-closed validator (`services/agents/app/llm/contract.py`). Two
+new test files in `services/agents/tests/`:
+
+- `test_llm_contract_extra.py` (10 cases) — fills the coverage gaps in the
+ shipped contract: `safe_astream` validates messages exactly once and
+ refuses to yield any chunk on violation; `make_safe_chat_model` proxies
+ non-LLM attributes through but routes `ainvoke` / `astream` through
+ validation; `classify_message` rejects `api_key = '...'` assignments and
+ PEM private-key headers; `set_contract_enforcement(False)` lets raw OCSF
+ through in soft mode and re-arms cleanly when flipped back to `True`.
+- `test_llm_contract_no_bypass.py` (3 cases) — **AST-based static gate**
+ that walks every `*.py` file under `services/agents/app/` and fails CI on
+ any direct `.ainvoke(...)` / `.astream(...)` call whose receiver is not on
+ an explicit allowlist (`_graph`, `investigation_graph`, `graph` — all
+ LangGraph control-flow handles, not LLMs) or whose file is not the
+ contract module itself. Ships with self-tests proving (a) a synthetic
+ `llm.ainvoke(...)` bypass trips the detector and (b) allowlisted
+ receivers do not. Adding a new agent that calls a chat model directly
+ now fails the build until it routes through `safe_ainvoke` /
+ `safe_astream` / `make_safe_chat_model`.
+
+The survey behind this gate confirmed every existing direct chat-model call
+under `services/agents/app/` already goes through the safe wrapper — the
+remaining `.ainvoke` / `.astream` call sites are LangGraph control-flow on
+compiled graphs, which is why those receivers are explicitly allowlisted
+rather than silently ignored.
+
### LLM input contract — CI tests (T2.3, v8.0)
`services/agents/tests/test_llm_contract.py` exercises `classify_message` /
diff --git a/services/agents/tests/test_llm_contract_extra.py b/services/agents/tests/test_llm_contract_extra.py
new file mode 100644
index 000000000..cc9af19b6
--- /dev/null
+++ b/services/agents/tests/test_llm_contract_extra.py
@@ -0,0 +1,240 @@
+"""Extra T2.3 coverage: streaming, model wrapper, secrets, enforcement toggle.
+
+These tests complement ``test_llm_contract.py``. The base file covers the
+validator on dict messages; here we exercise the parts of the safety surface
+that ship to production but had no direct assertions:
+
+* :func:`safe_astream` — the streaming counterpart of ``safe_ainvoke``,
+* :func:`make_safe_chat_model` — the wrapper used when a caller already
+ holds a chat-model reference,
+* :func:`classify_message` secret-pattern branch,
+* :func:`set_contract_enforcement` — the soft-mode escape hatch.
+
+If any of these regress we lose either a defence layer or the ability to
+debug locally with the gate disabled.
+"""
+
+from __future__ import annotations
+
+import json
+from typing import Any
+
+import pytest
+from app.llm.contract import (
+ LLMContractViolation,
+ classify_message,
+ is_contract_enforced,
+ make_safe_chat_model,
+ safe_astream,
+ set_contract_enforcement,
+ validate_messages,
+)
+
+
+def contract_enforced():
+ prev = set_contract_enforcement(True)
+ try:
+ yield
+ finally:
+ set_contract_enforcement(prev)
+
+
+# ---------------------------------------------------------------------------
+# safe_astream — validates once, then yields underlying chunks
+# ---------------------------------------------------------------------------
+
+
+class _FakeStreamingLLM:
+ """Minimal stand-in for a LangChain chat model with .astream."""
+
+ def __init__(self, chunks: list[str]) -> None:
+ self._chunks = chunks
+ self.calls: list[list[dict[str, Any]]] = []
+
+ async def astream(self, messages: list[Any], **_: Any):
+ self.calls.append(list(messages))
+ for chunk in self._chunks:
+ yield chunk
+
+ async def ainvoke(self, messages: list[Any], **_: Any) -> str:
+ self.calls.append(list(messages))
+ return "".join(self._chunks)
+
+
+async def test_safe_astream_yields_chunks_when_messages_clean(contract_enforced) -> None:
+ llm = _FakeStreamingLLM(["alpha", "beta", "gamma"])
+ safe_msgs = [
+ {"role": "system", "content": "You are a security analyst."},
+ {"role": "user", "content": "Summarise the incident in two sentences."},
+ ]
+
+ collected: list[str] = []
+ async for chunk in safe_astream(llm, safe_msgs):
+ collected.append(chunk)
+
+ assert collected == ["alpha", "beta", "gamma"]
+ assert llm.calls == [safe_msgs], "underlying llm must receive the exact message list"
+
+
+async def test_safe_astream_rejects_violating_messages_before_yield(
+ contract_enforced,
+) -> None:
+ llm = _FakeStreamingLLM(["should-never-emit"])
+ bad_msgs = [
+ {"role": "user", "content": json.dumps({"class_uid": 7003, "activity_id": 1})},
+ ]
+
+ collected: list[str] = []
+ with pytest.raises(LLMContractViolation):
+ async for chunk in safe_astream(llm, bad_msgs):
+ collected.append(chunk)
+
+ assert collected == [], "no chunks must escape when the contract is violated"
+ assert llm.calls == [], "the underlying llm must never be invoked"
+
+
+# ---------------------------------------------------------------------------
+# make_safe_chat_model — wrapper routes ainvoke/astream through the contract
+# ---------------------------------------------------------------------------
+
+
+class _FakeChatModel:
+ """Records calls and exposes a non-LLM attribute we expect to passthrough."""
+
+ model_name = "fake-gpt"
+
+ def __init__(self) -> None:
+ self.invocations: list[list[Any]] = []
+
+ async def ainvoke(self, messages: list[Any], **_: Any) -> str:
+ self.invocations.append(list(messages))
+ return "ok"
+
+ async def astream(self, messages: list[Any], **_: Any):
+ self.invocations.append(list(messages))
+ for token in ("o", "k"):
+ yield token
+
+
+async def test_make_safe_chat_model_allows_safe_messages(contract_enforced) -> None:
+ inner = _FakeChatModel()
+ guarded = make_safe_chat_model(inner)
+
+ result = await guarded.ainvoke([{"role": "user", "content": "How risky is this alert?"}])
+
+ assert result == "ok"
+ assert len(inner.invocations) == 1
+
+
+async def test_make_safe_chat_model_blocks_raw_ocsf(contract_enforced) -> None:
+ inner = _FakeChatModel()
+ guarded = make_safe_chat_model(inner)
+
+ raw = json.dumps({"class_uid": 1001, "activity_id": 2, "metadata": {"product": "Sentinel"}})
+
+ with pytest.raises(LLMContractViolation):
+ await guarded.ainvoke([{"role": "user", "content": raw}])
+
+ assert inner.invocations == [], "blocked prompt must never reach the chat model"
+
+
+async def test_make_safe_chat_model_astream_blocks_raw_ocsf(contract_enforced) -> None:
+ """Regression for PR #140 review: the wrapper's ``astream`` path must
+ enforce the contract just like ``ainvoke``. The standalone
+ :func:`safe_astream` is exercised in another test, but the proxy
+ path through :func:`make_safe_chat_model` was previously unguarded
+ against a partial-stream leak — a violation here would silently
+ forward chunks before the consumer noticed.
+
+ Invariants:
+ 1. The wrapper's ``astream`` raises :class:`LLMContractViolation`
+ (or, depending on enforcement, an empty stream).
+ 2. ``inner.invocations`` stays empty: the inner model is never
+ reached, so no partial chunk can leak.
+ """
+ inner = _FakeChatModel()
+ guarded = make_safe_chat_model(inner)
+
+ raw = json.dumps({"class_uid": 1001, "activity_id": 2, "metadata": {"product": "Sentinel"}})
+
+ chunks: list[Any] = []
+ with pytest.raises(LLMContractViolation):
+ async for chunk in guarded.astream([{"role": "user", "content": raw}]):
+ chunks.append(chunk)
+
+ assert chunks == [], "blocked stream must not yield any chunk to the consumer"
+ assert inner.invocations == [], "blocked stream must never reach the chat model"
+
+
+def test_make_safe_chat_model_passes_through_non_llm_attributes() -> None:
+ """Wrapper must remain a drop-in proxy for everything except ainvoke/astream."""
+ inner = _FakeChatModel()
+ guarded = make_safe_chat_model(inner)
+
+ assert guarded.model_name == "fake-gpt"
+
+
+# ---------------------------------------------------------------------------
+# classify_message — the secret-pattern branch is the last-resort guard
+# ---------------------------------------------------------------------------
+
+
+def test_classify_message_flags_api_key_assignment() -> None:
+ """`api_key = '...long...'` style assignments must be classified as leaking."""
+ leak = "Found in code: api_key = 'sk-AbCdEf0123456789xyz'"
+ reason = classify_message(leak)
+ assert reason is not None
+ assert "secret" in reason.lower()
+
+
+def test_classify_message_flags_pem_private_key_header() -> None:
+ pem_header = "Pasted from server: -----BEGIN RSA PRIVATE KEY-----\\nMIIE…"
+ reason = classify_message(pem_header)
+ assert reason is not None
+ assert "secret" in reason.lower()
+
+
+# ---------------------------------------------------------------------------
+# Enforcement toggle — operators MUST be able to soft-disable for debugging
+# ---------------------------------------------------------------------------
+
+
+def test_set_contract_enforcement_soft_mode_lets_raw_ocsf_through() -> None:
+ raw = json.dumps({"class_uid": 1, "activity_id": 2, "metadata": {"product": "x"}})
+ prev = set_contract_enforcement(False)
+ try:
+ assert is_contract_enforced() is False
+ # When soft, validator must not raise. The return value is the
+ # normalised view; we just need this to not blow up.
+ normalised = validate_messages([{"role": "user", "content": raw}])
+ assert normalised == [{"role": "user", "content": raw}]
+ finally:
+ set_contract_enforcement(prev)
+
+
+def test_set_contract_enforcement_returns_previous_value() -> None:
+ prev = set_contract_enforcement(True)
+ try:
+ was = set_contract_enforcement(False)
+ assert was is True
+ was_again = set_contract_enforcement(True)
+ assert was_again is False
+ finally:
+ set_contract_enforcement(prev)
+
+
+def test_enforcement_toggle_restores_blocking_after_soft_window() -> None:
+ """Soft mode must not 'stick' — restoring True must re-arm rejection."""
+ raw = json.dumps({"class_uid": 42, "metadata": {"product": "x", "version": "1"}})
+ prev = set_contract_enforcement(False)
+ try:
+ validate_messages([{"role": "user", "content": raw}]) # passes
+ finally:
+ set_contract_enforcement(prev)
+ set_contract_enforcement(True)
+ try:
+ with pytest.raises(LLMContractViolation):
+ validate_messages([{"role": "user", "content": raw}])
+ finally:
+ set_contract_enforcement(prev)
diff --git a/services/agents/tests/test_llm_contract_no_bypass.py b/services/agents/tests/test_llm_contract_no_bypass.py
new file mode 100644
index 000000000..deb1e8fd9
--- /dev/null
+++ b/services/agents/tests/test_llm_contract_no_bypass.py
@@ -0,0 +1,217 @@
+"""Static regression gate for the LLM input contract (T2.3).
+
+Rule: nothing in ``services/agents/app/`` may call ``.ainvoke(...)`` or
+``.astream(...)`` directly on an LLM. All LLM calls MUST go through
+:func:`app.llm.contract.safe_ainvoke`, :func:`app.llm.contract.safe_astream`,
+or :func:`app.llm.contract.make_safe_chat_model`.
+
+The check is intentionally textual-AST (not type-aware) because LangChain
+chat models are not nominally typed. We rely on a *receiver allowlist*:
+calls like ``self._graph.ainvoke(...)`` and ``investigation_graph.ainvoke(...)``
+are LangGraph control-flow primitives, not LLM invocations, and are listed
+explicitly so a reviewer notices when the allowlist grows.
+
+To add a new permitted receiver name, append to ``_ALLOWED_RECEIVERS`` with
+a short justification comment. To exempt an entire file (because the file
+itself implements the wrapper), append to ``_ALLOWED_FILES``.
+"""
+
+from __future__ import annotations
+
+import ast
+from pathlib import Path
+
+_AGENTS_APP = Path(__file__).resolve().parent.parent / "app"
+
+# Receivers that are NOT chat models. Each entry MUST be explained.
+_ALLOWED_RECEIVERS: frozenset[str] = frozenset(
+ {
+ # LangGraph compiled graph — control-flow, not an LLM. Used in
+ # services/agents/app/investigator/orchestrator.py via self._graph.
+ "_graph",
+ # Module-level LangGraph handle in services/agents/app/api/router.py.
+ "investigation_graph",
+ # Generic LangGraph handles. If someone wires a graph elsewhere we
+ # still don't want to flag it. Keep tight — add new names only when
+ # the call site is reviewed as control-flow, not an LLM call.
+ "graph",
+ }
+)
+
+# Files where direct ``llm.ainvoke``/``llm.astream`` IS the implementation of
+# the safe wrapper itself. The wrapper validates messages first, then
+# delegates — so these calls are by construction safe.
+_ALLOWED_FILES: frozenset[Path] = frozenset(
+ {
+ _AGENTS_APP / "llm" / "contract.py",
+ }
+)
+
+_METHODS = {"ainvoke", "astream"}
+
+
+def _receiver_name(node: ast.Attribute) -> str:
+ """Best-effort receiver identifier for ``X.method(...)``.
+
+ For ``self._graph.ainvoke(...)`` → ``_graph``.
+ For ``investigation_graph.ainvoke(...)`` → ``investigation_graph``.
+ For ``some_func().ainvoke(...)`` → ``<call>`` (we treat as unknown and flag).
+ """
+ value = node.value
+ if isinstance(value, ast.Attribute):
+ return value.attr
+ if isinstance(value, ast.Name):
+ return value.id
+ return f"<{type(value).__name__}>"
+
+
+def _getattr_bypass_method(node: ast.Call) -> str | None:
+ """Detect ``getattr(x, "ainvoke")(...)`` and ``getattr(x, "astream")(...)``.
+
+ This is the canonical evasion of attribute-name-based AST gates: a
+ future author could route around ``.ainvoke``/``.astream`` checks by
+ writing ``getattr(llm, "ainvoke")(messages)`` and the bare-attribute
+ walker would never see it. We flag the *outer* call here (the call
+ that actually invokes the LLM) and report it as if it had been
+ ``receiver.method(...)``.
+
+ Returns the method name (``"ainvoke"`` / ``"astream"``) if this
+ Call is a ``getattr``-mediated bypass, else ``None``.
+ """
+ inner = node.func
+ if not isinstance(inner, ast.Call):
+ return None
+ if not isinstance(inner.func, ast.Name) or inner.func.id != "getattr":
+ return None
+ # getattr(obj, "name"[, default]) — name must be a literal string for
+ # the bypass to be useful at the call site.
+ if len(inner.args) < 2:
+ return None
+ name_arg = inner.args[1]
+ if not isinstance(name_arg, ast.Constant) or not isinstance(name_arg.value, str):
+ return None
+ if name_arg.value not in _METHODS:
+ return None
+ return name_arg.value
+
+
+def _find_bypasses(path: Path) -> list[tuple[int, str, str]]:
+ """Return (lineno, receiver, method) tuples for forbidden calls in ``path``.
+
+ Detects three call shapes:
+
+ 1. ``obj.ainvoke(...)`` / ``obj.astream(...)`` — direct attribute call.
+ 2. ``self.obj.ainvoke(...)`` — chained attribute call (receiver is
+ the inner attr, e.g. ``_graph``).
+ 3. ``getattr(obj, "ainvoke")(...)`` — name-indirection bypass; the
+ outer ``Call`` is flagged with receiver ``<getattr>`` because the
+ allowlist is by static name and the dynamic lookup deliberately
+ defeats that — reviewers must justify and refactor, not allowlist.
+ """
+ source = path.read_text(encoding="utf-8")
+ tree = ast.parse(source, filename=str(path))
+ findings: list[tuple[int, str, str]] = []
+
+ for node in ast.walk(tree):
+ if not isinstance(node, ast.Call):
+ continue
+
+ # Shape 3: getattr(x, "ainvoke")(...) — flagged unconditionally.
+ method = _getattr_bypass_method(node)
+ if method is not None:
+ findings.append((node.lineno, "<getattr>", method))
+ continue
+
+ # Shapes 1 + 2: x.ainvoke(...) / self.x.ainvoke(...).
+ func = node.func
+ if not isinstance(func, ast.Attribute):
+ continue
+ if func.attr not in _METHODS:
+ continue
+ receiver = _receiver_name(func)
+ if receiver in _ALLOWED_RECEIVERS:
+ continue
+ findings.append((node.lineno, receiver, func.attr))
+ return findings
+
+
+def _python_sources() -> list[Path]:
+ return [p for p in _AGENTS_APP.rglob("*.py") if "__pycache__" not in p.parts and p not in _ALLOWED_FILES]
+
+
+def test_no_direct_ainvoke_or_astream_bypass() -> None:
+ """No file under services/agents/app/ may call .ainvoke/.astream directly on an LLM."""
+ violations: list[str] = []
+ for path in _python_sources():
+ for lineno, receiver, method in _find_bypasses(path):
+ rel = path.relative_to(_AGENTS_APP.parent)
+ violations.append(f"{rel}:{lineno} {receiver}.{method}(...)")
+
+ assert not violations, (
+ "Direct LLM `.ainvoke`/`.astream` call detected — route through "
+ "app.llm.contract.safe_ainvoke / safe_astream / make_safe_chat_model. "
+ "If the receiver is genuinely not an LLM (e.g. a LangGraph compiled "
+ "graph), add its name to _ALLOWED_RECEIVERS in this test with a "
+ "justification comment.\n " + "\n ".join(violations)
+ )
+
+
+def test_gate_detects_synthetic_bypass(tmp_path: Path) -> None:
+ """Sanity-check the AST walker: a fake bypass file must trip the detector."""
+ fake = tmp_path / "fake_agent.py"
+ fake.write_text(
+ "class Foo:\n async def run(self, llm):\n return await llm.ainvoke([{'role': 'user', 'content': 'hi'}])\n",
+ encoding="utf-8",
+ )
+ findings = _find_bypasses(fake)
+ assert findings, "AST walker failed to flag a synthetic llm.ainvoke bypass"
+ assert findings[0][1] == "llm"
+ assert findings[0][2] == "ainvoke"
+
+
+def test_gate_respects_receiver_allowlist(tmp_path: Path) -> None:
+ """Calls on whitelisted receivers (e.g. _graph) must NOT be flagged."""
+ fake = tmp_path / "fake_graph.py"
+ fake.write_text(
+ "class Orch:\n"
+ " def __init__(self):\n"
+ " self._graph = object()\n"
+ " async def run(self):\n"
+ " return await self._graph.ainvoke({'x': 1})\n",
+ encoding="utf-8",
+ )
+ findings = _find_bypasses(fake)
+ assert findings == [], f"allowlisted receiver should not trip the gate; got {findings}"
+
+
+def test_gate_detects_getattr_indirection_bypass(tmp_path: Path) -> None:
+ """The classic AST-gate evasion is ``getattr(llm, "ainvoke")(...)``.
+
+ A bare-attribute walker (only inspecting ``Attribute`` nodes) would
+ happily walk past this — the outer ``Call.func`` is a ``Call`` to
+ the ``getattr`` builtin, not an ``Attribute`` at all. We flag it
+ with a synthetic ``<getattr>`` receiver, which is intentionally
+ NOT in the allowlist so a future author can't just append to the
+ allowlist to whitelist away the protection.
+ """
+ fake = tmp_path / "fake_getattr.py"
+ fake.write_text(
+ "async def sneaky(llm, msgs):\n"
+ " # Future author tries to dodge the .ainvoke attribute scan.\n"
+ " fn = getattr(llm, 'ainvoke')\n"
+ " return await fn(msgs)\n"
+ "async def stream_sneaky(llm, msgs):\n"
+ " return getattr(llm, 'astream')(msgs)\n",
+ encoding="utf-8",
+ )
+ findings = _find_bypasses(fake)
+ # Note: the first form (`fn = getattr(...)`; `fn(...)`) splits the
+ # getattr Call from the invocation Call, so the AST scanner sees a
+ # bare `fn(...)` with no method-name signal. That's a known limit
+ # of static name analysis. The second form (`getattr(llm, 'astream')(msgs)`)
+ # invokes inline and MUST be flagged.
+ flagged_methods = sorted({m for _, _, m in findings})
+ assert "astream" in flagged_methods, f"getattr(llm, 'astream')(msgs) inline-invocation must be flagged; got {findings}"
+ for _lineno, receiver, method in findings:
+ if method == "astream":
+ assert receiver == "<getattr>", f"getattr-indirection bypass must report '<getattr>' receiver; got {receiver}"