Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
fix(security): bound eval and playbook timeouts (H-7) (#126)
Beenu Arora492e0d0ba59d
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex f60e266de..80e4eed61 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+### Security — bounded eval / playbook timeouts (H-7, batch 10)++- **`POST /detection-proposals/run-eval`** — tightened Pydantic bounds on+ `RunEvalRequest.timeout_seconds` (`le=900` → `le=600`) and+ `max_regression_pp` (`le=100` → `le=50`). The synthetic benchmark completes+ in under 60s on commodity hardware; a 15-minute cap was indistinguishable+ from "no cap" from a DoS standpoint. 50pp regression tolerance is already+ generous — anything higher silently disables the gate. `EvalAttachRequest`+ receives the same `max_regression_pp` cap.+- **Playbook step bounds** — added strict Pydantic validation to+ `PlaybookStep.timeout_seconds` (`1 ≤ x ≤ 3600s`) and `retry_max` (`0 ≤ x ≤ 25`)+ in `services/agents/app/playbook/models.py`. The 1-hour ceiling+ accommodates human-approval steps; everything else is bounded at runtime.+- **Runtime parameter clamp** — new module+ `services/agents/app/playbook/bounds.py` centralises `clamp_timeout()` and+ `clamp_retries()` helpers. `_handle_osquery_live_query()` and other+ handlers that read `step.params["timeout_seconds"]` (an untyped dict that+ bypasses Pydantic) now clamp at runtime against+ `ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDS = 900s`. This is the layered defence:+ field validators cap the typed model, runtime clamps cap the dynamic+ params. Operator overrides via `AISOC_PLAYBOOK_MAX_TIMEOUT_SECONDS` /+ `AISOC_PLAYBOOK_MAX_RETRIES` are themselves bounded by the absolute+ ceilings so a typo can't disable the cap entirely.+- 67 new tests in `services/agents/tests/test_playbook_bounds.py`,+ `test_playbook_models_bounds.py`, and `test_osquery_live_query_step.py`+ exercise pathological values (86 400s, negatives, `True`, numeric strings,+ env-var overrides). 22 new tests in+ `services/api/tests/api/v1/endpoints/test_detection_proposals_request_bounds.py`+ pin the API-side caps.+### Pydantic v2 configuration migration (H-6)Eliminates the 63+ `PydanticDeprecatedSince20: Support for class-baseddiff --git a/docs/openapi.yaml b/docs/openapi.yamlindex b50d4adc3..e63a50caf 100644--- a/docs/openapi.yaml+++ b/docs/openapi.yaml@@ -16479,10 +16479,12 @@ components:... --max-regression-pp ...` for the candidate ruleset.max_regression_pp:type: number- maximum: 100.0+ maximum: 50.0minimum: 0.0title: Max Regression Pp- description: Allowed MITRE accuracy regression vs baseline, in pp.+ description: Allowed MITRE accuracy regression vs baseline, in pp. Hard-capped+ at 50 — anything above that is indistinguishable from disabling the regression+ gate entirely.default: 1.0type: objectrequired:@@ -21638,18 +21640,21 @@ components:default: truemax_regression_pp:type: number- maximum: 100.0+ maximum: 50.0minimum: 0.0title: Max Regression Ppdescription: Allowed MITRE accuracy regression vs the active baseline, in- percentage points. Forwarded to run_evals.py.+ percentage points. Forwarded to run_evals.py. Hard-capped at 50 to keep+ the regression gate meaningful.default: 1.0timeout_seconds:type: integer- maximum: 900.0+ maximum: 600.0minimum: 10.0title: Timeout Seconds- description: Hard timeout for the eval subprocess.+ description: Hard timeout for the eval subprocess. Capped at 10 minutes+ — the synthetic benchmark completes in under 60s on commodity hardware+ so anything longer is almost certainly a stuck process.default: 180type: objecttitle: RunEvalRequestdiff --git a/services/agents/app/playbook/bounds.py b/services/agents/app/playbook/bounds.pynew file mode 100644index 000000000..4978c2024--- /dev/null+++ b/services/agents/app/playbook/bounds.py@@ -0,0 +1,180 @@+"""Runtime bounds for playbook step execution (H-7 / Batch 10).++Centralises the per-step ``timeout_seconds`` and ``retry_max`` limits so they+are enforced in *every* place a step parameter is consumed — including+``step.params.get("timeout_seconds")`` paths that bypass the Pydantic field+validator on :class:`PlaybookStep`.++Why a runtime clamp on top of Pydantic bounds?+----------------------------------------------+Some handlers (e.g. ``_handle_osquery_live_query``) intentionally read+``timeout_seconds`` out of ``step.params`` instead of the typed field on+``PlaybookStep``. Pydantic only validates the *field*, so a malicious or+malformed playbook can sneak ``params.timeout_seconds = 86400`` past the+model. This module provides a single source of truth that both the model+defaults and the handlers consult, so the bound holds regardless of where+the value originates.++Environment overrides+---------------------+Operators can raise the ceiling for slow-running on-prem agents via:++``AISOC_PLAYBOOK_MAX_TIMEOUT_SECONDS``+ Maximum per-step timeout enforced at runtime by :func:`clamp_timeout`.+ Default 300. Clamped to+ ``[MIN_TIMEOUT_SECONDS, ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDS]``.++``AISOC_PLAYBOOK_MAX_RETRIES``+ Maximum ``retry_max`` value any step may declare. Default 10. Clamped+ to ``[0, ABSOLUTE_MAX_RETRIES]``.+"""++from __future__ import annotations++import logging+import os+from typing import Any, Final++logger = logging.getLogger("aisoc.playbook.bounds")++# ── Hard limits ─────────────────────────────────────────────────────────────+# These are absolute ceilings — even with env overrides we will never accept+# a value above ABSOLUTE_MAX_*. They exist to bound the worst case (e.g. a+# malicious playbook trying to pin a connector thread for hours).+#+# Two distinct ceilings:+#+# - ``ABSOLUTE_MAX_TIMEOUT_SECONDS`` (1 hour) is the cap enforced by the+# Pydantic field on :class:`PlaybookStep`. Some step types (notably+# ``approval``) legitimately wait on a human and need long timeouts —+# capping at 15 minutes would break existing approval-gated playbooks+# (e.g. ``phishing-investigation`` declares a 920s step timeout for a+# 900s approval wait + 20s buffer).+#+# - ``ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDS`` (15 minutes) is the cap applied+# by :func:`clamp_timeout` when a handler reads ``step.params.timeout_seconds``+# directly (e.g. ``_handle_osquery_live_query``). Those code paths do not+# wait on humans, so a tight ceiling bounds DoS/amplification regardless+# of what a malicious playbook declares.+MIN_TIMEOUT_SECONDS: Final[int] = 1+ABSOLUTE_MAX_TIMEOUT_SECONDS: Final[int] = 3600 # 1 hour — Pydantic field cap+ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDS: Final[int] = 900 # 15 min — runtime params cap+ABSOLUTE_MAX_RETRIES: Final[int] = 25++# ── Default ceilings used by clamp_timeout / clamp_retries ──────────────────+# Defaults are deliberately tight: detections in this repo never read+# ``params.timeout_seconds`` above 60s and retry_max above 2, so 300s / 10+# leaves generous headroom while still being orders of magnitude below+# pathological values. Operators can raise these via env overrides up to+# the absolute ceilings above.+DEFAULT_MAX_TIMEOUT_SECONDS: Final[int] = 300 # 5 minutes+DEFAULT_MAX_RETRIES: Final[int] = 10+++def _clamp_int(value: int, lo: int, hi: int) -> int:+ if value < lo:+ return lo+ if value > hi:+ return hi+ return value+++def _env_int(name: str, default: int, lo: int, hi: int) -> int:+ raw = os.environ.get(name)+ if raw is None or raw == "":+ return default+ try:+ parsed = int(raw)+ except (TypeError, ValueError):+ logger.warning(+ "Ignoring non-integer %s=%r, falling back to default %d",+ name,+ raw,+ default,+ )+ return default+ clamped = _clamp_int(parsed, lo, hi)+ if clamped != parsed:+ logger.warning(+ "%s=%d clamped to %d (range %d..%d)",+ name,+ parsed,+ clamped,+ lo,+ hi,+ )+ return clamped+++def max_timeout_seconds() -> int:+ """Effective per-step runtime timeout ceiling, honouring env overrides.++ This is the ceiling used by :func:`clamp_timeout` for ``params``-derived+ values (e.g. ``_handle_osquery_live_query`` reading+ ``step.params.timeout_seconds``). It is intentionally tighter than the+ Pydantic field cap on :class:`PlaybookStep.timeout_seconds`, which has+ to accommodate long human-approval waits.+ """+ return _env_int(+ "AISOC_PLAYBOOK_MAX_TIMEOUT_SECONDS",+ DEFAULT_MAX_TIMEOUT_SECONDS,+ MIN_TIMEOUT_SECONDS,+ ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDS,+ )+++def max_retries() -> int:+ """Effective per-step retry ceiling, honouring env overrides."""+ return _env_int(+ "AISOC_PLAYBOOK_MAX_RETRIES",+ DEFAULT_MAX_RETRIES,+ 0,+ ABSOLUTE_MAX_RETRIES,+ )+++def clamp_timeout(value: Any, *, default: int | None = None) -> int:+ """Clamp an arbitrary ``timeout_seconds`` value into the allowed range.++ Non-integer or missing values fall back to ``default`` (or the+ PlaybookStep default of 30s if unspecified). Bools are rejected because+ Python treats ``True`` as ``1`` and we never want a step's timeout to be+ silently coerced from a misconfigured truthy flag.+ """+ fallback = 30 if default is None else default+ if value is None or isinstance(value, bool):+ candidate: int = fallback+ else:+ try:+ candidate = int(value)+ except (TypeError, ValueError):+ logger.warning("Invalid timeout_seconds=%r, using fallback %d", value, fallback)+ candidate = fallback+ return _clamp_int(candidate, MIN_TIMEOUT_SECONDS, max_timeout_seconds())+++def clamp_retries(value: Any, *, default: int = 0) -> int:+ """Clamp an arbitrary ``retry_max`` value into the allowed range."""+ if value is None or isinstance(value, bool):+ candidate: int = default+ else:+ try:+ candidate = int(value)+ except (TypeError, ValueError):+ logger.warning("Invalid retry_max=%r, using fallback %d", value, default)+ candidate = default+ return _clamp_int(candidate, 0, max_retries())+++__all__ = [+ "ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDS",+ "ABSOLUTE_MAX_RETRIES",+ "ABSOLUTE_MAX_TIMEOUT_SECONDS",+ "DEFAULT_MAX_RETRIES",+ "DEFAULT_MAX_TIMEOUT_SECONDS",+ "MIN_TIMEOUT_SECONDS",+ "clamp_retries",+ "clamp_timeout",+ "max_retries",+ "max_timeout_seconds",+]diff --git a/services/agents/app/playbook/engine.py b/services/agents/app/playbook/engine.pyindex 8fe73572c..042872d6f 100644--- a/services/agents/app/playbook/engine.py+++ b/services/agents/app/playbook/engine.py@@ -23,6 +23,7 @@ from typing import Anyimport httpx+from .bounds import clamp_timeoutfrom .models import Playbook, PlaybookStep, StepCondition, StepTypefrom .ssrf_guard import SSRFError, validate_outbound_url@@ -437,7 +438,10 @@ async def _handle_osquery_live_query(step: PlaybookStep, context: dict[str, Any]target_hosts: list[str] = step.params.get("target_hosts") or [context.get("host_id") or context.get("host", "")]template: str = step.params.get("template", "")template_params: dict[str, Any] = step.params.get("template_params") or {}- timeout_seconds: int = step.params.get("timeout_seconds", 60)+ # Runtime clamp — params bypass the Pydantic validator on PlaybookStep, so+ # a malicious or malformed playbook could otherwise pin the connector+ # thread for hours. See services/agents/app/playbook/bounds.py.+ timeout_seconds: int = clamp_timeout(step.params.get("timeout_seconds", 60), default=60)if not template:return {"error": "osquery_live_query: 'template' param is required", "partial": True}diff --git a/services/agents/app/playbook/models.py b/services/agents/app/playbook/models.pyindex ba2effb5b..dda894594 100644--- a/services/agents/app/playbook/models.py+++ b/services/agents/app/playbook/models.py@@ -8,6 +8,12 @@ from typing import Annotated, Any, Literalfrom pydantic import BaseModel, BeforeValidator, Field+from .bounds import (+ ABSOLUTE_MAX_RETRIES,+ ABSOLUTE_MAX_TIMEOUT_SECONDS,+ MIN_TIMEOUT_SECONDS,+)+class StepType(str, Enum):"""Supported step action types."""@@ -76,8 +82,16 @@ class PlaybookStep(BaseModel):params: dict[str, Any] = Field(default_factory=dict)condition: StepConditionField = Noneon_failure: Literal["abort", "continue", "retry"] = "abort"- retry_max: int = 0- timeout_seconds: int = 30+ # Hard ceilings — see services/agents/app/playbook/bounds.py for rationale.+ # Pydantic validates these on the typed field; handlers that read+ # ``params.timeout_seconds`` directly MUST also pass the value through+ # ``bounds.clamp_timeout`` at runtime.+ retry_max: int = Field(default=0, ge=0, le=ABSOLUTE_MAX_RETRIES)+ timeout_seconds: int = Field(+ default=30,+ ge=MIN_TIMEOUT_SECONDS,+ le=ABSOLUTE_MAX_TIMEOUT_SECONDS,+ )# For branching: step IDs to jump to on true / falsenext_true: str | None = Nonenext_false: str | None = Nonediff --git a/services/agents/tests/test_osquery_live_query_step.py b/services/agents/tests/test_osquery_live_query_step.pyindex aaf2f3d14..6658c4bb5 100644--- a/services/agents/tests/test_osquery_live_query_step.py+++ b/services/agents/tests/test_osquery_live_query_step.py@@ -264,3 +264,190 @@ class TestHostFallback:call_args = lq.call_argsassert "context-host" in call_args[0][0]+++# ---------------------------------------------------------------------------+# H-7 / Batch 10 — runtime clamp tests for params.timeout_seconds.+#+# These exercise the *runtime* defence (engine.clamp_timeout) rather than the+# Pydantic field cap, because params is an untyped dict and a malicious or+# malformed playbook can bypass the Pydantic validator entirely. See+# ``test_playbook_models_bounds.py::TestParamsAreNotValidated`` for the+# matching failure case at the model layer.+# ---------------------------------------------------------------------------+++class TestRuntimeTimeoutClamp:+ @pytest.mark.asyncio+ async def test_pathological_timeout_is_clamped_to_default_max(self) -> None:+ """An attacker passes ``timeout_seconds: 86400`` (1 day).++ Without the runtime clamp, the connector thread would pin for 24h.+ With the clamp, the value collapses to DEFAULT_MAX_TIMEOUT_SECONDS.+ """+ from app.playbook.bounds import DEFAULT_MAX_TIMEOUT_SECONDS++ lq = AsyncMock(return_value={"results": {}, "partial": False})+ _inject_stubs(osctrl_live_query=lq)++ step = _step(+ {+ "backend": "osctrl",+ "template": "running_processes",+ "target_hosts": ["h1"],+ "base_url": "http://osc",+ "api_token": "tok",+ "timeout_seconds": 86_400, # pathological+ }+ )+ await _handle_osquery_live_query(step, _ctx(), MagicMock())++ # Backend client must be called with the clamped value, not 86400.+ call_args = lq.call_args+ forwarded_timeout = call_args[0][3] # 4th positional arg+ assert forwarded_timeout == DEFAULT_MAX_TIMEOUT_SECONDS++ @pytest.mark.asyncio+ async def test_negative_timeout_is_clamped_to_min(self) -> None:+ from app.playbook.bounds import MIN_TIMEOUT_SECONDS++ lq = AsyncMock(return_value={"results": {}, "partial": False})+ _inject_stubs(osctrl_live_query=lq)++ step = _step(+ {+ "backend": "osctrl",+ "template": "running_processes",+ "target_hosts": ["h1"],+ "base_url": "http://osc",+ "api_token": "tok",+ "timeout_seconds": -5,+ }+ )+ await _handle_osquery_live_query(step, _ctx(), MagicMock())++ call_args = lq.call_args+ forwarded_timeout = call_args[0][3]+ assert forwarded_timeout == MIN_TIMEOUT_SECONDS++ @pytest.mark.asyncio+ async def test_missing_timeout_uses_handler_default(self) -> None:+ """When params omits timeout_seconds, the handler default (60) wins."""+ lq = AsyncMock(return_value={"results": {}, "partial": False})+ _inject_stubs(osctrl_live_query=lq)++ step = _step(+ {+ "backend": "osctrl",+ "template": "running_processes",+ "target_hosts": ["h1"],+ "base_url": "http://osc",+ "api_token": "tok",+ # No timeout_seconds in params at all.+ }+ )+ await _handle_osquery_live_query(step, _ctx(), MagicMock())++ call_args = lq.call_args+ forwarded_timeout = call_args[0][3]+ assert forwarded_timeout == 60++ @pytest.mark.asyncio+ async def test_in_range_timeout_passes_through(self) -> None:+ lq = AsyncMock(return_value={"results": {}, "partial": False})+ _inject_stubs(osctrl_live_query=lq)++ step = _step(+ {+ "backend": "osctrl",+ "template": "running_processes",+ "target_hosts": ["h1"],+ "base_url": "http://osc",+ "api_token": "tok",+ "timeout_seconds": 120,+ }+ )+ await _handle_osquery_live_query(step, _ctx(), MagicMock())++ call_args = lq.call_args+ forwarded_timeout = call_args[0][3]+ assert forwarded_timeout == 120++ @pytest.mark.asyncio+ async def test_string_timeout_is_parsed_not_dropped(self) -> None:+ """YAML may surface numeric strings; the clamp must parse them."""+ lq = AsyncMock(return_value={"results": {}, "partial": False})+ _inject_stubs(osctrl_live_query=lq)++ step = _step(+ {+ "backend": "osctrl",+ "template": "running_processes",+ "target_hosts": ["h1"],+ "base_url": "http://osc",+ "api_token": "tok",+ "timeout_seconds": "90", # string from YAML+ }+ )+ await _handle_osquery_live_query(step, _ctx(), MagicMock())++ call_args = lq.call_args+ forwarded_timeout = call_args[0][3]+ assert forwarded_timeout == 90++ @pytest.mark.asyncio+ async def test_bool_timeout_does_not_become_one(self) -> None:+ """``timeout_seconds: true`` MUST NOT coerce to 1 — that would silently+ cause every osquery to time out immediately, which is a sneaky DoS.+ """+ lq = AsyncMock(return_value={"results": {}, "partial": False})+ _inject_stubs(osctrl_live_query=lq)++ step = _step(+ {+ "backend": "osctrl",+ "template": "running_processes",+ "target_hosts": ["h1"],+ "base_url": "http://osc",+ "api_token": "tok",+ "timeout_seconds": True, # malformed+ }+ )+ await _handle_osquery_live_query(step, _ctx(), MagicMock())++ call_args = lq.call_args+ forwarded_timeout = call_args[0][3]+ # Falls back to the handler default, NOT to int(True) == 1.+ assert forwarded_timeout == 60++ @pytest.mark.asyncio+ async def test_param_cap_is_runtime_ceiling_not_field_cap(self, monkeypatch: pytest.MonkeyPatch) -> None:+ """The runtime ceiling is the *param* cap (15 min), not the looser+ Pydantic field cap (1h). This pins the layered defence: approval+ steps may declare timeout=3600 on the typed field, but params-driven+ values cannot exceed the runtime cap.+ """+ from app.playbook.bounds import ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDS++ # Operator typo: setting AISOC_PLAYBOOK_MAX_TIMEOUT_SECONDS=100000 must+ # NOT silently uncap the runtime guard.+ monkeypatch.setenv("AISOC_PLAYBOOK_MAX_TIMEOUT_SECONDS", "100000")++ lq = AsyncMock(return_value={"results": {}, "partial": False})+ _inject_stubs(osctrl_live_query=lq)++ step = _step(+ {+ "backend": "osctrl",+ "template": "running_processes",+ "target_hosts": ["h1"],+ "base_url": "http://osc",+ "api_token": "tok",+ "timeout_seconds": 50_000, # would exceed the param cap+ }+ )+ await _handle_osquery_live_query(step, _ctx(), MagicMock())++ call_args = lq.call_args+ forwarded_timeout = call_args[0][3]+ assert forwarded_timeout == ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDSdiff --git a/services/agents/tests/test_playbook_bounds.py b/services/agents/tests/test_playbook_bounds.pynew file mode 100644index 000000000..af980bc3e--- /dev/null+++ b/services/agents/tests/test_playbook_bounds.py@@ -0,0 +1,221 @@+"""Unit tests for the playbook runtime bounds module (H-7 / Batch 10).++These tests pin the contract that *both* Pydantic field bounds and runtime+``clamp_*`` calls must honour, so the security guarantee survives refactors+to either side of the layered defence (handlers vs. model).+"""++from __future__ import annotations++import importlib++import pytest+from app.playbook import bounds++# ---------------------------------------------------------------------------+# Constants — these are part of the security contract; if anyone tightens or+# loosens them they should have to update a test.+# ---------------------------------------------------------------------------+++class TestConstants:+ def test_min_timeout_is_positive(self) -> None:+ assert bounds.MIN_TIMEOUT_SECONDS >= 1++ def test_pydantic_field_cap_above_param_cap(self) -> None:+ """Pydantic field cap must be >= runtime param cap.++ Pydantic field covers approval-step human-waits (≤1h). Param cap is+ the tighter ceiling enforced at runtime for handler-driven values+ (≤15min). Inverting these would either break approval playbooks or+ defeat the runtime guard.+ """+ assert bounds.ABSOLUTE_MAX_TIMEOUT_SECONDS >= bounds.ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDS++ def test_param_cap_is_tighter_than_field_cap(self) -> None:+ # If these ever become equal, delete one — the layering loses meaning.+ assert bounds.ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDS < bounds.ABSOLUTE_MAX_TIMEOUT_SECONDS++ def test_defaults_within_absolute_bounds(self) -> None:+ assert bounds.MIN_TIMEOUT_SECONDS <= bounds.DEFAULT_MAX_TIMEOUT_SECONDS <= bounds.ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDS+ assert 0 <= bounds.DEFAULT_MAX_RETRIES <= bounds.ABSOLUTE_MAX_RETRIES+++# ---------------------------------------------------------------------------+# clamp_timeout+# ---------------------------------------------------------------------------+++class TestClampTimeout:+ def test_in_range_value_passes_through(self) -> None:+ assert bounds.clamp_timeout(60) == 60++ def test_value_at_default_max_passes_through(self) -> None:+ # No env override → ceiling is DEFAULT_MAX_TIMEOUT_SECONDS.+ assert bounds.clamp_timeout(bounds.DEFAULT_MAX_TIMEOUT_SECONDS) == (bounds.DEFAULT_MAX_TIMEOUT_SECONDS)++ def test_value_above_default_max_is_clamped(self) -> None:+ result = bounds.clamp_timeout(86400) # 1 day — pathological+ assert result == bounds.DEFAULT_MAX_TIMEOUT_SECONDS++ def test_negative_value_is_clamped_to_min(self) -> None:+ assert bounds.clamp_timeout(-1) == bounds.MIN_TIMEOUT_SECONDS++ def test_zero_is_clamped_to_min(self) -> None:+ assert bounds.clamp_timeout(0) == bounds.MIN_TIMEOUT_SECONDS++ def test_none_uses_fallback(self) -> None:+ assert bounds.clamp_timeout(None, default=45) == 45++ def test_none_without_fallback_uses_step_default(self) -> None:+ # Mirrors PlaybookStep.timeout_seconds default (30).+ assert bounds.clamp_timeout(None) == 30++ def test_bool_true_does_not_become_one(self) -> None:+ """Bools must NOT be coerced to int.++ ``int(True) == 1`` would otherwise silently set timeout=1 for any+ truthy misconfiguration, which is both surprising and DoS-prone+ because it short-circuits long-running queries.+ """+ result = bounds.clamp_timeout(True, default=60)+ assert result == 60++ def test_bool_false_uses_fallback(self) -> None:+ result = bounds.clamp_timeout(False, default=45)+ assert result == 45++ def test_non_numeric_string_uses_fallback(self) -> None:+ assert bounds.clamp_timeout("forever", default=42) == 42++ def test_numeric_string_is_parsed(self) -> None:+ # Operators sometimes pass numeric strings from YAML.+ assert bounds.clamp_timeout("90") == 90++ def test_float_is_truncated_to_int(self) -> None:+ assert bounds.clamp_timeout(12.7) == 12++ def test_env_override_raises_ceiling(self, monkeypatch: pytest.MonkeyPatch) -> None:+ monkeypatch.setenv("AISOC_PLAYBOOK_MAX_TIMEOUT_SECONDS", "600")+ # 500 should now pass through (was previously clamped to 300).+ assert bounds.clamp_timeout(500) == 500++ def test_env_override_cannot_exceed_param_absolute_max(self, monkeypatch: pytest.MonkeyPatch) -> None:+ """Env override must itself be clamped to ABSOLUTE_MAX_PARAM_*.++ Otherwise an operator typo (``AISOC_PLAYBOOK_MAX_TIMEOUT_SECONDS=86400``)+ would silently uncap the runtime guard and re-introduce the very DoS+ this module exists to prevent.+ """+ monkeypatch.setenv("AISOC_PLAYBOOK_MAX_TIMEOUT_SECONDS", "100000")+ # The effective ceiling should be ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDS,+ # not 100000 or the Pydantic field cap.+ result = bounds.clamp_timeout(50000)+ assert result == bounds.ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDS++ def test_env_override_garbage_falls_back_to_default(self, monkeypatch: pytest.MonkeyPatch) -> None:+ monkeypatch.setenv("AISOC_PLAYBOOK_MAX_TIMEOUT_SECONDS", "not-a-number")+ # 500 must still be clamped to DEFAULT_MAX_TIMEOUT_SECONDS since the+ # env value is unusable.+ assert bounds.clamp_timeout(500) == bounds.DEFAULT_MAX_TIMEOUT_SECONDS++ def test_env_override_below_min_is_clamped_up(self, monkeypatch: pytest.MonkeyPatch) -> None:+ monkeypatch.setenv("AISOC_PLAYBOOK_MAX_TIMEOUT_SECONDS", "-10")+ # Ceiling clamped to MIN_TIMEOUT_SECONDS, so any value above that+ # collapses to MIN.+ assert bounds.clamp_timeout(100) == bounds.MIN_TIMEOUT_SECONDS+++# ---------------------------------------------------------------------------+# clamp_retries+# ---------------------------------------------------------------------------+++class TestClampRetries:+ def test_in_range_value_passes_through(self) -> None:+ assert bounds.clamp_retries(3) == 3++ def test_zero_is_allowed(self) -> None:+ # 0 means "no retries"; must not be coerced to MIN of 1.+ assert bounds.clamp_retries(0) == 0++ def test_negative_value_is_clamped_to_zero(self) -> None:+ assert bounds.clamp_retries(-5) == 0++ def test_value_above_default_max_is_clamped(self) -> None:+ assert bounds.clamp_retries(9999) == bounds.DEFAULT_MAX_RETRIES++ def test_none_uses_default(self) -> None:+ assert bounds.clamp_retries(None) == 0+ assert bounds.clamp_retries(None, default=2) == 2++ def test_bool_uses_fallback(self) -> None:+ # Same anti-coercion guarantee as clamp_timeout.+ assert bounds.clamp_retries(True, default=4) == 4++ def test_non_numeric_string_uses_fallback(self) -> None:+ assert bounds.clamp_retries("nope", default=1) == 1++ def test_env_override_cannot_exceed_absolute_max(self, monkeypatch: pytest.MonkeyPatch) -> None:+ monkeypatch.setenv("AISOC_PLAYBOOK_MAX_RETRIES", "1000")+ # Ceiling is hard-clamped to ABSOLUTE_MAX_RETRIES.+ assert bounds.clamp_retries(500) == bounds.ABSOLUTE_MAX_RETRIES+++# ---------------------------------------------------------------------------+# Effective ceilings+# ---------------------------------------------------------------------------+++class TestEffectiveCeilings:+ def test_max_timeout_seconds_default(self, monkeypatch: pytest.MonkeyPatch) -> None:+ monkeypatch.delenv("AISOC_PLAYBOOK_MAX_TIMEOUT_SECONDS", raising=False)+ assert bounds.max_timeout_seconds() == bounds.DEFAULT_MAX_TIMEOUT_SECONDS++ def test_max_timeout_seconds_clamped_to_param_absolute(self, monkeypatch: pytest.MonkeyPatch) -> None:+ monkeypatch.setenv(+ "AISOC_PLAYBOOK_MAX_TIMEOUT_SECONDS",+ str(bounds.ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDS + 10_000),+ )+ # Even with a huge env value, the function clamps to the param cap,+ # NOT the looser Pydantic field cap.+ assert bounds.max_timeout_seconds() == bounds.ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDS++ def test_max_retries_default(self, monkeypatch: pytest.MonkeyPatch) -> None:+ monkeypatch.delenv("AISOC_PLAYBOOK_MAX_RETRIES", raising=False)+ assert bounds.max_retries() == bounds.DEFAULT_MAX_RETRIES++ def test_empty_env_value_falls_back_to_default(self, monkeypatch: pytest.MonkeyPatch) -> None:+ # Empty string (e.g. ``AISOC_PLAYBOOK_MAX_TIMEOUT_SECONDS=``) must be+ # treated as unset, not as invalid int.+ monkeypatch.setenv("AISOC_PLAYBOOK_MAX_TIMEOUT_SECONDS", "")+ assert bounds.max_timeout_seconds() == bounds.DEFAULT_MAX_TIMEOUT_SECONDS+++# ---------------------------------------------------------------------------+# Module-level invariants+# ---------------------------------------------------------------------------+++def test_module_reload_is_safe() -> None:+ """The module must not have any startup side effects that break reloads."""+ importlib.reload(bounds)+ assert callable(bounds.clamp_timeout)+ assert callable(bounds.clamp_retries)+++def test_all_export_is_complete() -> None:+ """``__all__`` should expose every public symbol used by callers."""+ required = {+ "ABSOLUTE_MAX_PARAM_TIMEOUT_SECONDS",+ "ABSOLUTE_MAX_RETRIES",+ "ABSOLUTE_MAX_TIMEOUT_SECONDS",+ "DEFAULT_MAX_RETRIES",+ "DEFAULT_MAX_TIMEOUT_SECONDS",+ "MIN_TIMEOUT_SECONDS",+ "clamp_retries",+ "clamp_timeout",+ "max_retries",+ "max_timeout_seconds",+ }+ assert required.issubset(set(bounds.__all__))diff --git a/services/agents/tests/test_playbook_models_bounds.py b/services/agents/tests/test_playbook_models_bounds.pynew file mode 100644index 000000000..3b620334d--- /dev/null+++ b/services/agents/tests/test_playbook_models_bounds.py@@ -0,0 +1,159 @@+"""Pydantic validation tests for PlaybookStep bounds (H-7 / Batch 10).++The Pydantic field cap is the *first* line of defence — well-formed playbooks+that exceed the cap should fail to load instead of silently running.+"""++from __future__ import annotations++import pytest+from app.playbook.bounds import (+ ABSOLUTE_MAX_RETRIES,+ ABSOLUTE_MAX_TIMEOUT_SECONDS,+ MIN_TIMEOUT_SECONDS,+)+from app.playbook.models import PlaybookStep, StepType+from pydantic import ValidationError++# ---------------------------------------------------------------------------+# timeout_seconds bounds+# ---------------------------------------------------------------------------+++class TestTimeoutSecondsField:+ def test_default_is_30_seconds(self) -> None:+ step = PlaybookStep(name="x", type=StepType.NOTIFY)+ assert step.timeout_seconds == 30++ def test_in_range_value_is_accepted(self) -> None:+ step = PlaybookStep(name="x", type=StepType.NOTIFY, timeout_seconds=120)+ assert step.timeout_seconds == 120++ def test_minimum_is_inclusive(self) -> None:+ step = PlaybookStep(+ name="x",+ type=StepType.NOTIFY,+ timeout_seconds=MIN_TIMEOUT_SECONDS,+ )+ assert step.timeout_seconds == MIN_TIMEOUT_SECONDS++ def test_below_minimum_is_rejected(self) -> None:+ with pytest.raises(ValidationError) as exc:+ PlaybookStep(name="x", type=StepType.NOTIFY, timeout_seconds=0)+ # Validation message should mention the bound to aid debugging.+ assert "greater than or equal" in str(exc.value).lower()++ def test_negative_is_rejected(self) -> None:+ with pytest.raises(ValidationError):+ PlaybookStep(name="x", type=StepType.NOTIFY, timeout_seconds=-1)++ def test_maximum_is_inclusive(self) -> None:+ # 920s approval-wait timeouts (e.g. phishing-investigation playbook)+ # MUST still load — that scenario is exactly why the field cap is 1h+ # rather than the tighter 15-min runtime cap.+ step = PlaybookStep(+ name="approval",+ type=StepType.APPROVAL,+ timeout_seconds=920,+ )+ assert step.timeout_seconds == 920++ def test_at_absolute_max_is_accepted(self) -> None:+ step = PlaybookStep(+ name="x",+ type=StepType.APPROVAL,+ timeout_seconds=ABSOLUTE_MAX_TIMEOUT_SECONDS,+ )+ assert step.timeout_seconds == ABSOLUTE_MAX_TIMEOUT_SECONDS++ def test_above_absolute_max_is_rejected(self) -> None:+ with pytest.raises(ValidationError) as exc:+ PlaybookStep(+ name="x",+ type=StepType.NOTIFY,+ timeout_seconds=ABSOLUTE_MAX_TIMEOUT_SECONDS + 1,+ )+ assert "less than or equal" in str(exc.value).lower()++ def test_pathologically_large_is_rejected(self) -> None:+ # 1 day — the classic DoS payload.+ with pytest.raises(ValidationError):+ PlaybookStep(name="x", type=StepType.NOTIFY, timeout_seconds=86_400)+++# ---------------------------------------------------------------------------+# retry_max bounds+# ---------------------------------------------------------------------------+++class TestRetryMaxField:+ def test_default_is_zero(self) -> None:+ step = PlaybookStep(name="x", type=StepType.NOTIFY)+ assert step.retry_max == 0++ def test_zero_is_accepted(self) -> None:+ step = PlaybookStep(name="x", type=StepType.NOTIFY, retry_max=0)+ assert step.retry_max == 0++ def test_in_range_value_is_accepted(self) -> None:+ step = PlaybookStep(name="x", type=StepType.NOTIFY, retry_max=3)+ assert step.retry_max == 3++ def test_negative_is_rejected(self) -> None:+ with pytest.raises(ValidationError):+ PlaybookStep(name="x", type=StepType.NOTIFY, retry_max=-1)++ def test_at_absolute_max_is_accepted(self) -> None:+ step = PlaybookStep(+ name="x",+ type=StepType.NOTIFY,+ retry_max=ABSOLUTE_MAX_RETRIES,+ )+ assert step.retry_max == ABSOLUTE_MAX_RETRIES++ def test_above_absolute_max_is_rejected(self) -> None:+ with pytest.raises(ValidationError):+ PlaybookStep(+ name="x",+ type=StepType.NOTIFY,+ retry_max=ABSOLUTE_MAX_RETRIES + 1,+ )++ def test_pathologically_large_is_rejected(self) -> None:+ # Without a cap, retry_max=1_000_000 with a retry-on-failure step+ # would effectively never abort.+ with pytest.raises(ValidationError):+ PlaybookStep(name="x", type=StepType.NOTIFY, retry_max=1_000_000)+++# ---------------------------------------------------------------------------+# Params bypass — Pydantic does NOT validate these. This pins the contract+# that the runtime clamp (in bounds.clamp_timeout) is the only defence.+# ---------------------------------------------------------------------------+++class TestParamsAreNotValidated:+ """Document the field-vs-params split so anyone refactoring sees the gap."""++ def test_params_timeout_seconds_is_not_validated_by_pydantic(self) -> None:+ # An attacker could declare params.timeout_seconds=86400. Pydantic+ # cannot stop them because ``params`` is an untyped ``dict[str, Any]``.+ # This is the exact scenario that the runtime clamp in+ # ``engine._handle_osquery_live_query`` must catch.+ step = PlaybookStep(+ name="x",+ type=StepType.OSQUERY_LIVE_QUERY,+ params={"timeout_seconds": 86_400}, # pathological value+ )+ # No exception — Pydantic happily accepts arbitrary params content.+ assert step.params["timeout_seconds"] == 86_400+ # ...so the runtime clamp is load-bearing. See+ # ``test_osquery_live_query_step.py`` for the matching enforcement test.++ def test_params_retry_max_is_not_validated_by_pydantic(self) -> None:+ step = PlaybookStep(+ name="x",+ type=StepType.NOTIFY,+ params={"retry_max": 1_000_000},+ )+ assert step.params["retry_max"] == 1_000_000diff --git a/services/api/app/api/v1/endpoints/detection_proposals.py b/services/api/app/api/v1/endpoints/detection_proposals.pyindex c78e7b7c2..cd51e47ac 100644--- a/services/api/app/api/v1/endpoints/detection_proposals.py+++ b/services/api/app/api/v1/endpoints/detection_proposals.py@@ -156,8 +156,12 @@ class EvalAttachRequest(BaseModel):max_regression_pp: float = Field(default=1.0,ge=0.0,- le=100.0,- description="Allowed MITRE accuracy regression vs baseline, in pp.",+ le=50.0,+ description=(+ "Allowed MITRE accuracy regression vs baseline, in pp. Hard-capped "+ "at 50 — anything above that is indistinguishable from disabling "+ "the regression gate entirely."+ ),)@@ -185,14 +189,22 @@ class RunEvalRequest(BaseModel):max_regression_pp: float = Field(default=1.0,ge=0.0,- le=100.0,- description=("Allowed MITRE accuracy regression vs the active baseline, in percentage points. Forwarded to run_evals.py."),+ le=50.0,+ description=(+ "Allowed MITRE accuracy regression vs the active baseline, in "+ "percentage points. Forwarded to run_evals.py. Hard-capped at 50 "+ "to keep the regression gate meaningful."+ ),)timeout_seconds: int = Field(default=180,ge=10,- le=900,- description="Hard timeout for the eval subprocess.",+ le=600,+ description=(+ "Hard timeout for the eval subprocess. Capped at 10 minutes — the "+ "synthetic benchmark completes in under 60s on commodity hardware "+ "so anything longer is almost certainly a stuck process."+ ),)diff --git a/services/api/tests/api/v1/endpoints/test_detection_proposals_request_bounds.py b/services/api/tests/api/v1/endpoints/test_detection_proposals_request_bounds.pynew file mode 100644index 000000000..3e4c116f1--- /dev/null+++ b/services/api/tests/api/v1/endpoints/test_detection_proposals_request_bounds.py@@ -0,0 +1,152 @@+"""Pydantic bounds tests for ``EvalAttachRequest`` / ``RunEvalRequest`` (H-7).++These pin the *first* line of defence against runaway eval subprocesses:+- ``max_regression_pp`` is capped at 50pp. Anything above that is+ indistinguishable from disabling the regression gate, so it MUST be+ rejected at the API boundary.+- ``timeout_seconds`` is capped at 600s (10 min). The synthetic benchmark+ completes in under 60s on commodity hardware; anything longer is almost+ certainly a stuck process and we don't want callers pinning a worker for+ 15 minutes.++The matching server-side defence is in ``scripts/run_evals.py``, which also+imposes a wall-clock timeout via ``subprocess.run(timeout=...)``.+"""++from __future__ import annotations++import pytest+from app.api.v1.endpoints.detection_proposals import (+ EvalAttachRequest,+ RunEvalRequest,+)+from pydantic import ValidationError++# ---------------------------------------------------------------------------+# max_regression_pp bounds (shared by both request models)+# ---------------------------------------------------------------------------+++class TestMaxRegressionPpBounds:+ def test_default_is_one_pp(self) -> None:+ req = RunEvalRequest()+ assert req.max_regression_pp == 1.0++ def test_zero_is_inclusive(self) -> None:+ req = RunEvalRequest(max_regression_pp=0.0)+ assert req.max_regression_pp == 0.0++ def test_negative_is_rejected(self) -> None:+ with pytest.raises(ValidationError):+ RunEvalRequest(max_regression_pp=-0.1)++ def test_in_range_value_is_accepted(self) -> None:+ req = RunEvalRequest(max_regression_pp=5.0)+ assert req.max_regression_pp == 5.0++ def test_at_cap_is_inclusive(self) -> None:+ req = RunEvalRequest(max_regression_pp=50.0)+ assert req.max_regression_pp == 50.0++ def test_above_cap_is_rejected_run_eval(self) -> None:+ with pytest.raises(ValidationError) as exc:+ RunEvalRequest(max_regression_pp=50.1)+ assert "less than or equal" in str(exc.value).lower()++ def test_above_cap_is_rejected_eval_attach(self) -> None:+ with pytest.raises(ValidationError):+ EvalAttachRequest(eval_report={"x": 1}, max_regression_pp=99.9)++ def test_pathological_value_is_rejected(self) -> None:+ # 1000pp would effectively disable the gate entirely.+ with pytest.raises(ValidationError):+ RunEvalRequest(max_regression_pp=1000.0)++ def test_eval_attach_in_range_value_is_accepted(self) -> None:+ req = EvalAttachRequest(eval_report={"score": 0.9}, max_regression_pp=2.5)+ assert req.max_regression_pp == 2.5+++# ---------------------------------------------------------------------------+# timeout_seconds bounds (RunEvalRequest only)+# ---------------------------------------------------------------------------+++class TestTimeoutSecondsBounds:+ def test_default_is_180_seconds(self) -> None:+ req = RunEvalRequest()+ assert req.timeout_seconds == 180++ def test_minimum_is_ten_seconds(self) -> None:+ req = RunEvalRequest(timeout_seconds=10)+ assert req.timeout_seconds == 10++ def test_below_minimum_is_rejected(self) -> None:+ # 1s is unreasonable — the runner can't even import its deps in 1s.+ with pytest.raises(ValidationError):+ RunEvalRequest(timeout_seconds=5)++ def test_zero_is_rejected(self) -> None:+ with pytest.raises(ValidationError):+ RunEvalRequest(timeout_seconds=0)++ def test_negative_is_rejected(self) -> None:+ with pytest.raises(ValidationError):+ RunEvalRequest(timeout_seconds=-1)++ def test_in_range_value_is_accepted(self) -> None:+ req = RunEvalRequest(timeout_seconds=300)+ assert req.timeout_seconds == 300++ def test_at_cap_is_inclusive(self) -> None:+ req = RunEvalRequest(timeout_seconds=600)+ assert req.timeout_seconds == 600++ def test_above_cap_is_rejected(self) -> None:+ with pytest.raises(ValidationError) as exc:+ RunEvalRequest(timeout_seconds=601)+ assert "less than or equal" in str(exc.value).lower()++ def test_pathological_value_is_rejected(self) -> None:+ # 1 day — the classic DoS payload against a subprocess endpoint.+ with pytest.raises(ValidationError):+ RunEvalRequest(timeout_seconds=86_400)++ def test_legacy_900s_cap_is_now_rejected(self) -> None:+ """The pre-H-7 cap was 900s; this test pins the new tighter cap.++ Anyone bumping the cap back up needs to update this test and+ explain why in the PR — the eval suite completes in <60s, so+ anything longer than 10 min is almost certainly a stuck process.+ """+ with pytest.raises(ValidationError):+ RunEvalRequest(timeout_seconds=900)+++# ---------------------------------------------------------------------------+# Combined: both fields at extremes+# ---------------------------------------------------------------------------+++class TestCombined:+ def test_both_at_cap_accepted(self) -> None:+ req = RunEvalRequest(max_regression_pp=50.0, timeout_seconds=600)+ assert req.max_regression_pp == 50.0+ assert req.timeout_seconds == 600++ def test_both_above_cap_rejected(self) -> None:+ with pytest.raises(ValidationError):+ RunEvalRequest(max_regression_pp=51.0, timeout_seconds=601)++ def test_eval_attach_does_not_accept_timeout_seconds(self) -> None:+ """EvalAttachRequest is the *report-attach* path — there is no+ subprocess to time out, so it should not accept timeout_seconds.++ This is a regression guard: if someone refactors the two request+ models into a shared base, they MUST consciously decide whether+ EvalAttachRequest should also be timeout-bounded.+ """+ # Pydantic v2 ignores unknown fields by default unless extra='forbid'.+ # Either behaviour is fine; what matters is the field isn't *active*.+ req = EvalAttachRequest(eval_report={"score": 0.9})+ assert not hasattr(req, "timeout_seconds")