Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
t23(agents): close LLM-input contract with static bypass gate + coverage (#140)
Beenu Arora4c8f201a7124
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 0d37c6147..01a68f2ad 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+### LLM input contract — static regression gate (T2.3, v8.0)++Closes T2.3 by adding the missing **bypass-prevention** layer on top of the+existing fail-closed validator (`services/agents/app/llm/contract.py`). Two+new test files in `services/agents/tests/`:++- `test_llm_contract_extra.py` (10 cases) — fills the coverage gaps in the+ shipped contract: `safe_astream` validates messages exactly once and+ refuses to yield any chunk on violation; `make_safe_chat_model` proxies+ non-LLM attributes through but routes `ainvoke` / `astream` through+ validation; `classify_message` rejects `api_key = '...'` assignments and+ PEM private-key headers; `set_contract_enforcement(False)` lets raw OCSF+ through in soft mode and re-arms cleanly when flipped back to `True`.+- `test_llm_contract_no_bypass.py` (3 cases) — **AST-based static gate**+ that walks every `*.py` file under `services/agents/app/` and fails CI on+ any direct `.ainvoke(...)` / `.astream(...)` call whose receiver is not on+ an explicit allowlist (`_graph`, `investigation_graph`, `graph` — all+ LangGraph control-flow handles, not LLMs) or whose file is not the+ contract module itself. Ships with self-tests proving (a) a synthetic+ `llm.ainvoke(...)` bypass trips the detector and (b) allowlisted+ receivers do not. Adding a new agent that calls a chat model directly+ now fails the build until it routes through `safe_ainvoke` /+ `safe_astream` / `make_safe_chat_model`.++The survey behind this gate confirmed every existing direct chat-model call+under `services/agents/app/` already goes through the safe wrapper — the+remaining `.ainvoke` / `.astream` call sites are LangGraph control-flow on+compiled graphs, which is why those receivers are explicitly allowlisted+rather than silently ignored.+### LLM input contract — CI tests (T2.3, v8.0)`services/agents/tests/test_llm_contract.py` exercises `classify_message` /diff --git a/services/agents/tests/test_llm_contract_extra.py b/services/agents/tests/test_llm_contract_extra.pynew file mode 100644index 000000000..cc9af19b6--- /dev/null+++ b/services/agents/tests/test_llm_contract_extra.py@@ -0,0 +1,240 @@+"""Extra T2.3 coverage: streaming, model wrapper, secrets, enforcement toggle.++These tests complement ``test_llm_contract.py``. The base file covers the+validator on dict messages; here we exercise the parts of the safety surface+that ship to production but had no direct assertions:++* :func:`safe_astream` — the streaming counterpart of ``safe_ainvoke``,+* :func:`make_safe_chat_model` — the wrapper used when a caller already+ holds a chat-model reference,+* :func:`classify_message` secret-pattern branch,+* :func:`set_contract_enforcement` — the soft-mode escape hatch.++If any of these regress we lose either a defence layer or the ability to+debug locally with the gate disabled.+"""++from __future__ import annotations++import json+from typing import Any++import pytest+from app.llm.contract import (+ LLMContractViolation,+ classify_message,+ is_contract_enforced,+ make_safe_chat_model,+ safe_astream,+ set_contract_enforcement,+ validate_messages,+)+++def contract_enforced():+ prev = set_contract_enforcement(True)+ try:+ yield+ finally:+ set_contract_enforcement(prev)+++# ---------------------------------------------------------------------------+# safe_astream — validates once, then yields underlying chunks+# ---------------------------------------------------------------------------+++class _FakeStreamingLLM:+ """Minimal stand-in for a LangChain chat model with .astream."""++ def __init__(self, chunks: list[str]) -> None:+ self._chunks = chunks+ self.calls: list[list[dict[str, Any]]] = []++ async def astream(self, messages: list[Any], **_: Any):+ self.calls.append(list(messages))+ for chunk in self._chunks:+ yield chunk++ async def ainvoke(self, messages: list[Any], **_: Any) -> str:+ self.calls.append(list(messages))+ return "".join(self._chunks)+++async def test_safe_astream_yields_chunks_when_messages_clean(contract_enforced) -> None:+ llm = _FakeStreamingLLM(["alpha", "beta", "gamma"])+ safe_msgs = [+ {"role": "system", "content": "You are a security analyst."},+ {"role": "user", "content": "Summarise the incident in two sentences."},+ ]++ collected: list[str] = []+ async for chunk in safe_astream(llm, safe_msgs):+ collected.append(chunk)++ assert collected == ["alpha", "beta", "gamma"]+ assert llm.calls == [safe_msgs], "underlying llm must receive the exact message list"+++async def test_safe_astream_rejects_violating_messages_before_yield(+ contract_enforced,+) -> None:+ llm = _FakeStreamingLLM(["should-never-emit"])+ bad_msgs = [+ {"role": "user", "content": json.dumps({"class_uid": 7003, "activity_id": 1})},+ ]++ collected: list[str] = []+ with pytest.raises(LLMContractViolation):+ async for chunk in safe_astream(llm, bad_msgs):+ collected.append(chunk)++ assert collected == [], "no chunks must escape when the contract is violated"+ assert llm.calls == [], "the underlying llm must never be invoked"+++# ---------------------------------------------------------------------------+# make_safe_chat_model — wrapper routes ainvoke/astream through the contract+# ---------------------------------------------------------------------------+++class _FakeChatModel:+ """Records calls and exposes a non-LLM attribute we expect to passthrough."""++ model_name = "fake-gpt"++ def __init__(self) -> None:+ self.invocations: list[list[Any]] = []++ async def ainvoke(self, messages: list[Any], **_: Any) -> str:+ self.invocations.append(list(messages))+ return "ok"++ async def astream(self, messages: list[Any], **_: Any):+ self.invocations.append(list(messages))+ for token in ("o", "k"):+ yield token+++async def test_make_safe_chat_model_allows_safe_messages(contract_enforced) -> None:+ inner = _FakeChatModel()+ guarded = make_safe_chat_model(inner)++ result = await guarded.ainvoke([{"role": "user", "content": "How risky is this alert?"}])++ assert result == "ok"+ assert len(inner.invocations) == 1+++async def test_make_safe_chat_model_blocks_raw_ocsf(contract_enforced) -> None:+ inner = _FakeChatModel()+ guarded = make_safe_chat_model(inner)++ raw = json.dumps({"class_uid": 1001, "activity_id": 2, "metadata": {"product": "Sentinel"}})++ with pytest.raises(LLMContractViolation):+ await guarded.ainvoke([{"role": "user", "content": raw}])++ assert inner.invocations == [], "blocked prompt must never reach the chat model"+++async def test_make_safe_chat_model_astream_blocks_raw_ocsf(contract_enforced) -> None:+ """Regression for PR #140 review: the wrapper's ``astream`` path must+ enforce the contract just like ``ainvoke``. The standalone+ :func:`safe_astream` is exercised in another test, but the proxy+ path through :func:`make_safe_chat_model` was previously unguarded+ against a partial-stream leak — a violation here would silently+ forward chunks before the consumer noticed.++ Invariants:+ 1. The wrapper's ``astream`` raises :class:`LLMContractViolation`+ (or, depending on enforcement, an empty stream).+ 2. ``inner.invocations`` stays empty: the inner model is never+ reached, so no partial chunk can leak.+ """+ inner = _FakeChatModel()+ guarded = make_safe_chat_model(inner)++ raw = json.dumps({"class_uid": 1001, "activity_id": 2, "metadata": {"product": "Sentinel"}})++ chunks: list[Any] = []+ with pytest.raises(LLMContractViolation):+ async for chunk in guarded.astream([{"role": "user", "content": raw}]):+ chunks.append(chunk)++ assert chunks == [], "blocked stream must not yield any chunk to the consumer"+ assert inner.invocations == [], "blocked stream must never reach the chat model"+++def test_make_safe_chat_model_passes_through_non_llm_attributes() -> None:+ """Wrapper must remain a drop-in proxy for everything except ainvoke/astream."""+ inner = _FakeChatModel()+ guarded = make_safe_chat_model(inner)++ assert guarded.model_name == "fake-gpt"+++# ---------------------------------------------------------------------------+# classify_message — the secret-pattern branch is the last-resort guard+# ---------------------------------------------------------------------------+++def test_classify_message_flags_api_key_assignment() -> None:+ """`api_key = '...long...'` style assignments must be classified as leaking."""+ leak = "Found in code: api_key = 'sk-AbCdEf0123456789xyz'"+ reason = classify_message(leak)+ assert reason is not None+ assert "secret" in reason.lower()+++def test_classify_message_flags_pem_private_key_header() -> None:+ pem_header = "Pasted from server: -----BEGIN RSA PRIVATE KEY-----\\nMIIE…"+ reason = classify_message(pem_header)+ assert reason is not None+ assert "secret" in reason.lower()+++# ---------------------------------------------------------------------------+# Enforcement toggle — operators MUST be able to soft-disable for debugging+# ---------------------------------------------------------------------------+++def test_set_contract_enforcement_soft_mode_lets_raw_ocsf_through() -> None:+ raw = json.dumps({"class_uid": 1, "activity_id": 2, "metadata": {"product": "x"}})+ prev = set_contract_enforcement(False)+ try:+ assert is_contract_enforced() is False+ # When soft, validator must not raise. The return value is the+ # normalised view; we just need this to not blow up.+ normalised = validate_messages([{"role": "user", "content": raw}])+ assert normalised == [{"role": "user", "content": raw}]+ finally:+ set_contract_enforcement(prev)+++def test_set_contract_enforcement_returns_previous_value() -> None:+ prev = set_contract_enforcement(True)+ try:+ was = set_contract_enforcement(False)+ assert was is True+ was_again = set_contract_enforcement(True)+ assert was_again is False+ finally:+ set_contract_enforcement(prev)+++def test_enforcement_toggle_restores_blocking_after_soft_window() -> None:+ """Soft mode must not 'stick' — restoring True must re-arm rejection."""+ raw = json.dumps({"class_uid": 42, "metadata": {"product": "x", "version": "1"}})+ prev = set_contract_enforcement(False)+ try:+ validate_messages([{"role": "user", "content": raw}]) # passes+ finally:+ set_contract_enforcement(prev)+ set_contract_enforcement(True)+ try:+ with pytest.raises(LLMContractViolation):+ validate_messages([{"role": "user", "content": raw}])+ finally:+ set_contract_enforcement(prev)diff --git a/services/agents/tests/test_llm_contract_no_bypass.py b/services/agents/tests/test_llm_contract_no_bypass.pynew file mode 100644index 000000000..deb1e8fd9--- /dev/null+++ b/services/agents/tests/test_llm_contract_no_bypass.py@@ -0,0 +1,217 @@+"""Static regression gate for the LLM input contract (T2.3).++Rule: nothing in ``services/agents/app/`` may call ``.ainvoke(...)`` or+``.astream(...)`` directly on an LLM. All LLM calls MUST go through+:func:`app.llm.contract.safe_ainvoke`, :func:`app.llm.contract.safe_astream`,+or :func:`app.llm.contract.make_safe_chat_model`.++The check is intentionally textual-AST (not type-aware) because LangChain+chat models are not nominally typed. We rely on a *receiver allowlist*:+calls like ``self._graph.ainvoke(...)`` and ``investigation_graph.ainvoke(...)``+are LangGraph control-flow primitives, not LLM invocations, and are listed+explicitly so a reviewer notices when the allowlist grows.++To add a new permitted receiver name, append to ``_ALLOWED_RECEIVERS`` with+a short justification comment. To exempt an entire file (because the file+itself implements the wrapper), append to ``_ALLOWED_FILES``.+"""++from __future__ import annotations++import ast+from pathlib import Path++_AGENTS_APP = Path(__file__).resolve().parent.parent / "app"++# Receivers that are NOT chat models. Each entry MUST be explained.+_ALLOWED_RECEIVERS: frozenset[str] = frozenset(+ {+ # LangGraph compiled graph — control-flow, not an LLM. Used in+ # services/agents/app/investigator/orchestrator.py via self._graph.+ "_graph",+ # Module-level LangGraph handle in services/agents/app/api/router.py.+ "investigation_graph",+ # Generic LangGraph handles. If someone wires a graph elsewhere we+ # still don't want to flag it. Keep tight — add new names only when+ # the call site is reviewed as control-flow, not an LLM call.+ "graph",+ }+)++# Files where direct ``llm.ainvoke``/``llm.astream`` IS the implementation of+# the safe wrapper itself. The wrapper validates messages first, then+# delegates — so these calls are by construction safe.+_ALLOWED_FILES: frozenset[Path] = frozenset(+ {+ _AGENTS_APP / "llm" / "contract.py",+ }+)++_METHODS = {"ainvoke", "astream"}+++def _receiver_name(node: ast.Attribute) -> str:+ """Best-effort receiver identifier for ``X.method(...)``.++ For ``self._graph.ainvoke(...)`` → ``_graph``.+ For ``investigation_graph.ainvoke(...)`` → ``investigation_graph``.+ For ``some_func().ainvoke(...)`` → ``<call>`` (we treat as unknown and flag).+ """+ value = node.value+ if isinstance(value, ast.Attribute):+ return value.attr+ if isinstance(value, ast.Name):+ return value.id+ return f"<{type(value).__name__}>"+++def _getattr_bypass_method(node: ast.Call) -> str | None:+ """Detect ``getattr(x, "ainvoke")(...)`` and ``getattr(x, "astream")(...)``.++ This is the canonical evasion of attribute-name-based AST gates: a+ future author could route around ``.ainvoke``/``.astream`` checks by+ writing ``getattr(llm, "ainvoke")(messages)`` and the bare-attribute+ walker would never see it. We flag the *outer* call here (the call+ that actually invokes the LLM) and report it as if it had been+ ``receiver.method(...)``.++ Returns the method name (``"ainvoke"`` / ``"astream"``) if this+ Call is a ``getattr``-mediated bypass, else ``None``.+ """+ inner = node.func+ if not isinstance(inner, ast.Call):+ return None+ if not isinstance(inner.func, ast.Name) or inner.func.id != "getattr":+ return None+ # getattr(obj, "name"[, default]) — name must be a literal string for+ # the bypass to be useful at the call site.+ if len(inner.args) < 2:+ return None+ name_arg = inner.args[1]+ if not isinstance(name_arg, ast.Constant) or not isinstance(name_arg.value, str):+ return None+ if name_arg.value not in _METHODS:+ return None+ return name_arg.value+++def _find_bypasses(path: Path) -> list[tuple[int, str, str]]:+ """Return (lineno, receiver, method) tuples for forbidden calls in ``path``.++ Detects three call shapes:++ 1. ``obj.ainvoke(...)`` / ``obj.astream(...)`` — direct attribute call.+ 2. ``self.obj.ainvoke(...)`` — chained attribute call (receiver is+ the inner attr, e.g. ``_graph``).+ 3. ``getattr(obj, "ainvoke")(...)`` — name-indirection bypass; the+ outer ``Call`` is flagged with receiver ``<getattr>`` because the+ allowlist is by static name and the dynamic lookup deliberately+ defeats that — reviewers must justify and refactor, not allowlist.+ """+ source = path.read_text(encoding="utf-8")+ tree = ast.parse(source, filename=str(path))+ findings: list[tuple[int, str, str]] = []++ for node in ast.walk(tree):+ if not isinstance(node, ast.Call):+ continue++ # Shape 3: getattr(x, "ainvoke")(...) — flagged unconditionally.+ method = _getattr_bypass_method(node)+ if method is not None:+ findings.append((node.lineno, "<getattr>", method))+ continue++ # Shapes 1 + 2: x.ainvoke(...) / self.x.ainvoke(...).+ func = node.func+ if not isinstance(func, ast.Attribute):+ continue+ if func.attr not in _METHODS:+ continue+ receiver = _receiver_name(func)+ if receiver in _ALLOWED_RECEIVERS:+ continue+ findings.append((node.lineno, receiver, func.attr))+ return findings+++def _python_sources() -> list[Path]:+ return [p for p in _AGENTS_APP.rglob("*.py") if "__pycache__" not in p.parts and p not in _ALLOWED_FILES]+++def test_no_direct_ainvoke_or_astream_bypass() -> None:+ """No file under services/agents/app/ may call .ainvoke/.astream directly on an LLM."""+ violations: list[str] = []+ for path in _python_sources():+ for lineno, receiver, method in _find_bypasses(path):+ rel = path.relative_to(_AGENTS_APP.parent)+ violations.append(f"{rel}:{lineno} {receiver}.{method}(...)")++ assert not violations, (+ "Direct LLM `.ainvoke`/`.astream` call detected — route through "+ "app.llm.contract.safe_ainvoke / safe_astream / make_safe_chat_model. "+ "If the receiver is genuinely not an LLM (e.g. a LangGraph compiled "+ "graph), add its name to _ALLOWED_RECEIVERS in this test with a "+ "justification comment.\n " + "\n ".join(violations)+ )+++def test_gate_detects_synthetic_bypass(tmp_path: Path) -> None:+ """Sanity-check the AST walker: a fake bypass file must trip the detector."""+ fake = tmp_path / "fake_agent.py"+ fake.write_text(+ "class Foo:\n async def run(self, llm):\n return await llm.ainvoke([{'role': 'user', 'content': 'hi'}])\n",+ encoding="utf-8",+ )+ findings = _find_bypasses(fake)+ assert findings, "AST walker failed to flag a synthetic llm.ainvoke bypass"+ assert findings[0][1] == "llm"+ assert findings[0][2] == "ainvoke"+++def test_gate_respects_receiver_allowlist(tmp_path: Path) -> None:+ """Calls on whitelisted receivers (e.g. _graph) must NOT be flagged."""+ fake = tmp_path / "fake_graph.py"+ fake.write_text(+ "class Orch:\n"+ " def __init__(self):\n"+ " self._graph = object()\n"+ " async def run(self):\n"+ " return await self._graph.ainvoke({'x': 1})\n",+ encoding="utf-8",+ )+ findings = _find_bypasses(fake)+ assert findings == [], f"allowlisted receiver should not trip the gate; got {findings}"+++def test_gate_detects_getattr_indirection_bypass(tmp_path: Path) -> None:+ """The classic AST-gate evasion is ``getattr(llm, "ainvoke")(...)``.++ A bare-attribute walker (only inspecting ``Attribute`` nodes) would+ happily walk past this — the outer ``Call.func`` is a ``Call`` to+ the ``getattr`` builtin, not an ``Attribute`` at all. We flag it+ with a synthetic ``<getattr>`` receiver, which is intentionally+ NOT in the allowlist so a future author can't just append to the+ allowlist to whitelist away the protection.+ """+ fake = tmp_path / "fake_getattr.py"+ fake.write_text(+ "async def sneaky(llm, msgs):\n"+ " # Future author tries to dodge the .ainvoke attribute scan.\n"+ " fn = getattr(llm, 'ainvoke')\n"+ " return await fn(msgs)\n"+ "async def stream_sneaky(llm, msgs):\n"+ " return getattr(llm, 'astream')(msgs)\n",+ encoding="utf-8",+ )+ findings = _find_bypasses(fake)+ # Note: the first form (`fn = getattr(...)`; `fn(...)`) splits the+ # getattr Call from the invocation Call, so the AST scanner sees a+ # bare `fn(...)` with no method-name signal. That's a known limit+ # of static name analysis. The second form (`getattr(llm, 'astream')(msgs)`)+ # invokes inline and MUST be flagged.+ flagged_methods = sorted({m for _, _, m in findings})+ assert "astream" in flagged_methods, f"getattr(llm, 'astream')(msgs) inline-invocation must be flagged; got {findings}"+ for _lineno, receiver, method in findings:+ if method == "astream":+ assert receiver == "<getattr>", f"getattr-indirection bypass must report '<getattr>' receiver; got {receiver}"