Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

Phase 1.2: memory-poisoning defenses (gated) (#417)

Beenu Arorabf0068dbd6a6
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 57b13c7b6..8c869fc8c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Security
+- **Phase 1.2 — memory-poisoning defenses for the override-learning loop.** New pure `services/api/app/services/memory_poisoning.py`: provenance on every memory write (`MemoryProvenance` — no anonymous memory), trust weighting (verified human outranks autonomous closure) with age decay so lessons must be re-confirmed, a `PoisoningDetector` that flags a burst of same-signature false-positive dispositions from low-trust authors, and blast-radius controls for retroactive re-disposition (`plan_redisposition` + `compute_confirmation_token`: capped batches, explicit confirmation token over the exact alert set, quarantine on flagged signatures). Wired into `override_learning.py` (poisoning-resistant signature key now includes the entity-independent severity band; provenance stamped on writes; `apply_redisposition` requires the token and enforces the cap) and the `/feedback` endpoints (preview returns the token + quarantine state; apply rejects stale/tampered/over-cap batches with 409). The farming-attack eval (`services/api/tests/test_memory_poisoning.py::test_farming_attack_then_real_attack_is_not_auto_closed`) gates the api job: a poisoned signature is flagged and its retroactive apply quarantined, so the real intrusion is not auto-closed.
+
- **Phase 1.1 — prompt-injection structural containment + detection.** New `services/agents/app/prompting/envelope.py`: per-run cryptographic-nonce evidence fence (`EvidenceEnvelope`, `make_nonce`, `system_rule`) so injected text cannot forge the closing delimiter to break out of the data block, and a `PromptInjectionGuard` that scans untrusted evidence for instruction-shaped content (role markers, "ignore previous", secret/prompt exfiltration, SOAR tool-name mentions, base64- and zero-width-obfuscated directives) and, on a high-severity hit, signals demotion of the case autonomy tier to L0. Added `services/agents/tests/test_prompt_envelope.py` (25 tests across every ingest path) and gated it plus the previously-ungated `test_prompt_sanitizer.py` in the CI agents job (fixed its stale agent-wiring expectations — the investigator agents sanitise via `sanitize_text` / `sanitize_iterable_of_strings` / `format_bundle_prompt_append`). Threat model: `docs/security/agent-threat-model.md`.
### Added
diff --git a/docs/openapi.yaml b/docs/openapi.yaml
index 90bb4d9fd..21140c3f9 100644
--- a/docs/openapi.yaml
+++ b/docs/openapi.yaml
@@ -8953,7 +8953,13 @@ paths:
tags:
- feedback
summary: Apply Redisposition Endpoint
- description: Bulk-update the disposition on past alerts the analyst confirmed.
+ description: 'Bulk-update the disposition on past alerts the analyst confirmed.
+
+
+ Requires the confirmation token from the preview; a stale/tampered set or
+ a
+
+ batch over the cap is rejected rather than silently flipping history.'
operationId: apply_redisposition_endpoint_api_v1_feedback_redisposition_apply_post
requestBody:
content:
@@ -13581,6 +13587,23 @@ components:
$ref: '#/components/schemas/RedispositionCandidateModel'
type: array
title: Redisposition Candidates
+ redisposition_confirmation_token:
+ anyOf:
+ - type: string
+ - type: 'null'
+ title: Redisposition Confirmation Token
+ redisposition_capped:
+ type: boolean
+ title: Redisposition Capped
+ default: false
+ redisposition_quarantined:
+ type: boolean
+ title: Redisposition Quarantined
+ default: false
+ redisposition_total_matched:
+ type: integer
+ title: Redisposition Total Matched
+ default: 0
type: object
required:
- alert_id
@@ -22048,10 +22071,16 @@ components:
new_disposition:
type: string
title: New Disposition
+ confirmation_token:
+ type: string
+ title: Confirmation Token
+ description: Token from the /alert-override preview over this exact alert
+ set (Phase 1.2 blast-radius control)
type: object
required:
- alert_ids
- new_disposition
+ - confirmation_token
title: RedispositionApplyRequest
RedispositionApplyResponse:
properties:
diff --git a/services/api/app/api/v1/endpoints/feedback.py b/services/api/app/api/v1/endpoints/feedback.py
index c9d2d984a..48dcf9dec 100644
--- a/services/api/app/api/v1/endpoints/feedback.py
+++ b/services/api/app/api/v1/endpoints/feedback.py
@@ -28,6 +28,7 @@ from sqlalchemy import and_, func, select, update
from app.api.v1.deps import AuthUser, DBSession
from app.models.alert import Alert
+from app.services.memory_poisoning import plan_redisposition
from app.services.override_learning import (
apply_redisposition,
find_redisposition_candidates,
@@ -77,6 +78,13 @@ class AlertOverrideResponse(BaseModel):
recorded_at: str
memory_key: str | None = None
redisposition_candidates: list[RedispositionCandidateModel] = Field(default_factory=list)
+ # Blast-radius controls for retroactive apply (Phase 1.2). The client must
+ # echo confirmation_token back to /redisposition/apply; quarantined means a
+ # poisoning flag or over-cap match requires human clearance before apply.
+ redisposition_confirmation_token: str | None = None
+ redisposition_capped: bool = False
+ redisposition_quarantined: bool = False
+ redisposition_total_matched: int = 0
@router.post("/alert-override", response_model=AlertOverrideResponse)
@@ -128,6 +136,7 @@ async def submit_alert_override(
# Find similar past alerts that would now disposition differently.
candidates: list[RedispositionCandidateModel] = []
memory_key: str | None = None
+ plan = None
if signature is not None:
memory_key = signature.memory_key()
raw_candidates = await find_redisposition_candidates(
@@ -138,6 +147,10 @@ async def submit_alert_override(
exclude_alert_id=alert_uuid,
)
candidates = [RedispositionCandidateModel(**c.to_dict()) for c in raw_candidates]
+ plan = plan_redisposition(
+ [c.alert_id for c in raw_candidates],
+ payload.corrected_verdict,
+ )
logger.info(
"analyst.override",
@@ -158,12 +171,20 @@ async def submit_alert_override(
recorded_at=now.isoformat(),
memory_key=memory_key,
redisposition_candidates=candidates,
+ redisposition_confirmation_token=plan.confirmation_token if plan else None,
+ redisposition_capped=plan.capped if plan else False,
+ redisposition_quarantined=plan.quarantined if plan else False,
+ redisposition_total_matched=plan.total_matched if plan else 0,
)
class RedispositionApplyRequest(BaseModel):
alert_ids: list[str]
new_disposition: str
+ confirmation_token: str = Field(
+ ...,
+ description="Token from the /alert-override preview over this exact alert set (Phase 1.2 blast-radius control)",
+ )
class RedispositionApplyResponse(BaseModel):
@@ -177,7 +198,11 @@ async def apply_redisposition_endpoint(
user: AuthUser,
db: DBSession,
) -> RedispositionApplyResponse:
- """Bulk-update the disposition on past alerts the analyst confirmed."""
+ """Bulk-update the disposition on past alerts the analyst confirmed.
+
+ Requires the confirmation token from the preview; a stale/tampered set or a
+ batch over the cap is rejected rather than silently flipping history.
+ """
if payload.new_disposition not in _VALID_VERDICTS:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
@@ -187,13 +212,17 @@ async def apply_redisposition_endpoint(
return RedispositionApplyResponse(updated=0, new_disposition=payload.new_disposition)
ids = [_coerce_uuid(aid, "alert_ids") for aid in payload.alert_ids]
- rowcount = await apply_redisposition(
- db,
- tenant_id=user.tenant_id,
- alert_ids=ids,
- new_disposition=payload.new_disposition,
- analyst_id=user.user_id,
- )
+ try:
+ rowcount = await apply_redisposition(
+ db,
+ tenant_id=user.tenant_id,
+ alert_ids=ids,
+ new_disposition=payload.new_disposition,
+ analyst_id=user.user_id,
+ confirmation_token=payload.confirmation_token,
+ )
+ except ValueError as exc:
+ raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(exc)) from exc
return RedispositionApplyResponse(updated=rowcount, new_disposition=payload.new_disposition)
diff --git a/services/api/app/services/memory_poisoning.py b/services/api/app/services/memory_poisoning.py
new file mode 100644
index 000000000..74f2d4f48
--- /dev/null
+++ b/services/api/app/services/memory_poisoning.py
@@ -0,0 +1,269 @@
+"""Memory-poisoning defenses for the analyst-override learning loop.
+
+Phase 1.2 of the world-class program. The override-learning loop
+(`app.services.override_learning`) lets an analyst correction teach the system
+to re-disposition future *and past* alerts that share a signature. In a SOC,
+anyone who can cause alerts is a potential attacker of this loop: farm benign
+alerts under the signature you intend to attack under, get them closed as false
+positive, and you have taught the system to auto-close the real intrusion. The
+one-click retroactive bulk re-disposition makes it worse — one poisoned
+signature can rewrite history.
+
+This module is the pure (stdlib-only, no DB) core of the defense:
+
+- **Provenance + trust weighting** — every memory carries who/what wrote it;
+ a verified human outranks an autonomous closure (:func:`trust_weight`).
+- **Decay** — confidence decays with age so nothing is permanent without
+ re-confirmation (:meth:`MemoryProvenance.effective_confidence`).
+- **Anomaly detection** — a burst of same-signature false-positive
+ dispositions from low-trust authors is itself a detection
+ (:class:`PoisoningDetector`).
+- **Blast-radius control** — retroactive re-disposition is planned, capped,
+ and gated on an explicit confirmation token derived from the exact alert set
+ (:func:`plan_redisposition`, :func:`compute_confirmation_token`).
+
+The DB-touching service (`override_learning`) and the feedback endpoint call
+into these helpers; keeping them pure means the whole defense is unit-testable
+offline and gated on every PR.
+"""
+
+from __future__ import annotations
+
+import hashlib
+from collections.abc import Iterable, Sequence
+from dataclasses import dataclass, field
+from datetime import datetime, timedelta
+from enum import Enum
+
+__all__ = [
+ "MemoryAuthor",
+ "trust_weight",
+ "MemoryProvenance",
+ "DispositionEvent",
+ "PoisoningVerdict",
+ "PoisoningDetector",
+ "compute_confirmation_token",
+ "RedispositionPlan",
+ "plan_redisposition",
+ "DEFAULT_MAX_REDISPOSITION_BATCH",
+]
+
+# Never auto-apply more than this many retroactive re-dispositions in one call.
+DEFAULT_MAX_REDISPOSITION_BATCH = 200
+
+
+class MemoryAuthor(str, Enum):
+ """Who/what produced a memory write. Determines trust weight."""
+
+ HUMAN_VERIFIED = "human_verified"
+ AUTONOMOUS = "autonomous"
+ IMPORTED = "imported"
+
+
+# A verified human analyst outranks an autonomous closure, which outranks an
+# imported/bulk lesson. These weights gate how much a memory can move a verdict.
+_TRUST_WEIGHTS: dict[MemoryAuthor, float] = {
+ MemoryAuthor.HUMAN_VERIFIED: 1.0,
+ MemoryAuthor.AUTONOMOUS: 0.4,
+ MemoryAuthor.IMPORTED: 0.2,
+}
+
+
+def trust_weight(author: MemoryAuthor | str) -> float:
+ """Base trust weight in [0, 1] for a memory author."""
+ if isinstance(author, str):
+ try:
+ author = MemoryAuthor(author)
+ except ValueError:
+ return 0.0
+ return _TRUST_WEIGHTS.get(author, 0.0)
+
+
+@dataclass(frozen=True)
+class MemoryProvenance:
+ """Provenance stamped on every institutional-memory write. No anonymous memory."""
+
+ author: MemoryAuthor
+ tenant_id: str
+ source_alert_id: str
+ analyst_id: str | None = None
+ confidence: float = 1.0
+ recorded_at: datetime | None = None
+
+ def effective_confidence(self, now: datetime, *, half_life_days: float = 30.0) -> float:
+ """Confidence discounted by trust weight and exponential age decay.
+
+ Nothing is permanent: a memory at ``half_life_days`` old counts for half
+ its trust-weighted confidence, so stale lessons must be re-confirmed to
+ keep moving verdicts.
+ """
+ base = max(0.0, min(1.0, self.confidence)) * trust_weight(self.author)
+ if self.recorded_at is None or half_life_days <= 0:
+ return base
+ age_days = max(0.0, (now - self.recorded_at).total_seconds() / 86_400.0)
+ decay = 0.5 ** (age_days / half_life_days)
+ return base * decay
+
+ def to_dict(self) -> dict:
+ return {
+ "author": self.author.value,
+ "tenant_id": self.tenant_id,
+ "source_alert_id": self.source_alert_id,
+ "analyst_id": self.analyst_id,
+ "confidence": self.confidence,
+ "trust_weight": trust_weight(self.author),
+ "recorded_at": self.recorded_at.isoformat() if self.recorded_at else None,
+ }
+
+
+@dataclass(frozen=True)
+class DispositionEvent:
+ """One disposition applied to an alert of a given signature."""
+
+ signature_key: str
+ disposition: str
+ author: MemoryAuthor
+ at: datetime
+ analyst_id: str | None = None
+
+
+@dataclass
+class PoisoningVerdict:
+ """Outcome of scanning a signature's recent disposition history."""
+
+ signature_key: str
+ flagged: bool
+ fp_count: int
+ window_seconds: float
+ distinct_authors: int
+ human_confirmations: int
+ reasons: list[str] = field(default_factory=list)
+
+ @property
+ def should_block_autoclose(self) -> bool:
+ """A flagged signature must not drive autonomous closure until a human clears it."""
+ return self.flagged
+
+
+class PoisoningDetector:
+ """Flags a burst of same-signature false-positive dispositions.
+
+ The farming attack looks like: many alerts of one signature closed as
+ false-positive in a short window, driven mostly by autonomous/low-trust
+ dispositions, with few or no independent human confirmations. Slow, human,
+ diverse corrections do not trip it.
+ """
+
+ def __init__(
+ self,
+ *,
+ window_seconds: float = 3600.0,
+ min_fp_burst: int = 10,
+ max_human_confirmations: int = 2,
+ fp_dispositions: Sequence[str] = ("false_positive", "benign"),
+ ) -> None:
+ self.window_seconds = window_seconds
+ self.min_fp_burst = min_fp_burst
+ self.max_human_confirmations = max_human_confirmations
+ self._fp_dispositions = set(fp_dispositions)
+
+ def assess(self, signature_key: str, events: Iterable[DispositionEvent], *, now: datetime) -> PoisoningVerdict:
+ window_start = now - timedelta(seconds=self.window_seconds)
+ recent_fp = [
+ e for e in events if e.signature_key == signature_key and e.disposition in self._fp_dispositions and e.at >= window_start
+ ]
+ fp_count = len(recent_fp)
+ human_confirmations = sum(1 for e in recent_fp if e.author == MemoryAuthor.HUMAN_VERIFIED)
+ distinct_authors = len({e.analyst_id for e in recent_fp if e.analyst_id})
+
+ reasons: list[str] = []
+ flagged = False
+ if fp_count >= self.min_fp_burst and human_confirmations <= self.max_human_confirmations:
+ flagged = True
+ reasons.append(
+ f"{fp_count} false-positive dispositions for one signature within "
+ f"{int(self.window_seconds)}s with only {human_confirmations} human confirmation(s)"
+ )
+ return PoisoningVerdict(
+ signature_key=signature_key,
+ flagged=flagged,
+ fp_count=fp_count,
+ window_seconds=self.window_seconds,
+ distinct_authors=distinct_authors,
+ human_confirmations=human_confirmations,
+ reasons=reasons,
+ )
+
+
+def compute_confirmation_token(alert_ids: Sequence[str], new_disposition: str) -> str:
+ """Deterministic token over the exact re-disposition set.
+
+ The client sees the preview (which alerts flip) and echoes this token back
+ on apply. If the set or target disposition changes, the token changes, so a
+ stale or tampered batch cannot be applied.
+ """
+ canonical = "|".join(sorted(str(a) for a in alert_ids)) + "=>" + new_disposition
+ return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
+
+
+@dataclass(frozen=True)
+class RedispositionPlan:
+ """A previewed, capped, confirm-gated retroactive re-disposition."""
+
+ alert_ids: list[str]
+ new_disposition: str
+ confirmation_token: str
+ total_matched: int
+ max_batch: int
+ capped: bool
+ quarantined: bool
+ reasons: list[str] = field(default_factory=list)
+
+ def to_dict(self) -> dict:
+ return {
+ "alert_ids": self.alert_ids,
+ "new_disposition": self.new_disposition,
+ "confirmation_token": self.confirmation_token,
+ "total_matched": self.total_matched,
+ "max_batch": self.max_batch,
+ "capped": self.capped,
+ "quarantined": self.quarantined,
+ "reasons": self.reasons,
+ }
+
+
+def plan_redisposition(
+ candidate_ids: Sequence[str],
+ new_disposition: str,
+ *,
+ max_batch: int = DEFAULT_MAX_REDISPOSITION_BATCH,
+ flagged: bool = False,
+) -> RedispositionPlan:
+ """Build a quarantined, capped, confirm-gated plan. Never applies anything.
+
+ - Caps the batch at ``max_batch`` (blast-radius control).
+ - Quarantines (requires manual review, applies nothing) when the signature
+ was flagged as poisoned or the matched set exceeds the cap.
+ """
+ ids = [str(a) for a in candidate_ids]
+ total = len(ids)
+ capped = total > max_batch
+ applied_ids = ids[:max_batch]
+ reasons: list[str] = []
+ quarantined = False
+ if flagged:
+ quarantined = True
+ reasons.append("signature flagged by poisoning detector; retroactive apply requires human clearance")
+ if capped:
+ reasons.append(f"matched {total} alerts; capped to {max_batch} per apply")
+ token = compute_confirmation_token(applied_ids, new_disposition)
+ return RedispositionPlan(
+ alert_ids=applied_ids,
+ new_disposition=new_disposition,
+ confirmation_token=token,
+ total_matched=total,
+ max_batch=max_batch,
+ capped=capped,
+ quarantined=quarantined,
+ reasons=reasons,
+ )
diff --git a/services/api/app/services/override_learning.py b/services/api/app/services/override_learning.py
index 2042a1e65..a45d0d080 100644
--- a/services/api/app/services/override_learning.py
+++ b/services/api/app/services/override_learning.py
@@ -32,17 +32,32 @@ from sqlalchemy.ext.asyncio import AsyncSession
from app.models.alert import Alert
from app.models.institutional_memory import InstitutionalMemory
+from app.services.memory_poisoning import (
+ DEFAULT_MAX_REDISPOSITION_BATCH,
+ MemoryAuthor,
+ MemoryProvenance,
+ compute_confirmation_token,
+)
logger = structlog.get_logger()
@dataclass(frozen=True)
class AlertSignature:
- """Coarse fingerprint used to match similar alerts across time."""
+ """Fingerprint used to match similar alerts across time.
+
+ Poisoning resistance (Phase 1.2): the key includes ``severity_band`` — an
+ entity-independent structural feature — in addition to the attacker-
+ choosable (category, connector, technique) triple. An attacker who farms
+ low-severity benign alerts under a signature can no longer teach the system
+ to auto-close a *high*-severity real intrusion sharing that triple, because
+ the severity band puts them in different memory keys.
+ """
category: str
connector_type: str
primary_technique: str
+ severity_band: str = ""
@classmethod
def from_alert(cls, alert: Alert) -> AlertSignature:
@@ -52,6 +67,7 @@ class AlertSignature:
category=(alert.category or "").lower().strip(),
connector_type=(alert.connector_type or "").lower().strip(),
primary_technique=primary.upper().strip(),
+ severity_band=(getattr(alert, "severity", "") or "").lower().strip(),
)
def is_empty(self) -> bool:
@@ -59,9 +75,9 @@ class AlertSignature:
return not (self.category or self.connector_type or self.primary_technique)
def memory_key(self) -> str:
- raw = f"{self.category}|{self.connector_type}|{self.primary_technique}"
+ raw = f"{self.category}|{self.connector_type}|{self.primary_technique}|{self.severity_band}"
digest = hashlib.sha1(raw.encode("utf-8"), usedforsecurity=False).hexdigest()
- return f"override:{digest}"
+ return f"override:v2:{digest}"
def tags(self) -> list[str]:
out: list[str] = ["analyst_override"]
@@ -118,6 +134,17 @@ async def record_override(
)
return None
+ now = datetime.now(UTC)
+ # Provenance on every write (Phase 1.2): no anonymous memory. An analyst
+ # override is human-verified and outranks any autonomous closure.
+ provenance = MemoryProvenance(
+ author=MemoryAuthor.HUMAN_VERIFIED,
+ tenant_id=str(tenant_id),
+ source_alert_id=str(alert.id),
+ analyst_id=str(analyst_id) if analyst_id else None,
+ confidence=1.0,
+ recorded_at=now,
+ )
payload = {
"alert_id": str(alert.id),
"original_verdict": original_verdict,
@@ -128,8 +155,10 @@ async def record_override(
"category": signature.category,
"connector_type": signature.connector_type,
"primary_technique": signature.primary_technique,
+ "severity_band": signature.severity_band,
},
- "recorded_at": datetime.now(UTC).isoformat(),
+ "provenance": provenance.to_dict(),
+ "recorded_at": now.isoformat(),
}
stmt = pg_insert(InstitutionalMemory).values(
@@ -218,11 +247,25 @@ async def apply_redisposition(
alert_ids: list[UUID],
new_disposition: str,
analyst_id: UUID | None,
+ confirmation_token: str,
+ max_batch: int = DEFAULT_MAX_REDISPOSITION_BATCH,
) -> int:
"""Bulk-apply *new_disposition* to *alert_ids*. Returns the number
- of rows updated. Each row already has its tenant verified."""
+ of rows updated. Each row already has its tenant verified.
+
+ Blast-radius controls (Phase 1.2): the batch is capped at *max_batch* and
+ the caller must echo the *confirmation_token* computed over the exact alert
+ set + target disposition (see :func:`plan_redisposition`). A stale or
+ tampered set no longer matches its token, so history cannot be flipped
+ without an explicit, previewed human confirmation.
+ """
if not alert_ids:
return 0
+ if len(alert_ids) > max_batch:
+ raise ValueError(f"redisposition batch of {len(alert_ids)} exceeds the per-apply cap of {max_batch}")
+ expected_token = compute_confirmation_token([str(a) for a in alert_ids], new_disposition)
+ if confirmation_token != expected_token:
+ raise ValueError("confirmation_token does not match the alert set; re-preview before applying")
now = datetime.now(UTC)
result = await db.execute(
update(Alert).where(Alert.tenant_id == tenant_id, Alert.id.in_(alert_ids)).values(disposition=new_disposition, updated_at=now)
diff --git a/services/api/tests/test_memory_poisoning.py b/services/api/tests/test_memory_poisoning.py
new file mode 100644
index 000000000..303cbf1d2
--- /dev/null
+++ b/services/api/tests/test_memory_poisoning.py
@@ -0,0 +1,189 @@
+"""Tests for memory-poisoning defenses (Phase 1.2).
+
+Pure/offline (stdlib + the pure app.services.memory_poisoning module). Runs in
+the api job's unfiltered `pytest tests/`.
+"""
+
+from __future__ import annotations
+
+from datetime import UTC, datetime, timedelta
+
+import pytest
+from app.services.memory_poisoning import (
+ DEFAULT_MAX_REDISPOSITION_BATCH,
+ DispositionEvent,
+ MemoryAuthor,
+ MemoryProvenance,
+ PoisoningDetector,
+ compute_confirmation_token,
+ plan_redisposition,
+ trust_weight,
+)
+
+NOW = datetime(2026, 7, 11, 12, 0, 0, tzinfo=UTC)
+
+
+# ── Trust weighting + decay ──────────────────────────────────────────────────
+
+
+def test_trust_weight_ordering():
+ assert trust_weight(MemoryAuthor.HUMAN_VERIFIED) > trust_weight(MemoryAuthor.AUTONOMOUS)
+ assert trust_weight(MemoryAuthor.AUTONOMOUS) > trust_weight(MemoryAuthor.IMPORTED)
+ assert trust_weight("nonsense") == 0.0
+ assert trust_weight("human_verified") == 1.0
+
+
+def test_effective_confidence_decays_with_age():
+ prov = MemoryProvenance(
+ author=MemoryAuthor.HUMAN_VERIFIED,
+ tenant_id="t1",
+ source_alert_id="a1",
+ confidence=1.0,
+ recorded_at=NOW,
+ )
+ fresh = prov.effective_confidence(NOW, half_life_days=30)
+ half = prov.effective_confidence(NOW + timedelta(days=30), half_life_days=30)
+ quarter = prov.effective_confidence(NOW + timedelta(days=60), half_life_days=30)
+ assert fresh == pytest.approx(1.0)
+ assert half == pytest.approx(0.5, abs=0.01)
+ assert quarter == pytest.approx(0.25, abs=0.01)
+
+
+def test_autonomous_memory_outranked_by_human():
+ human = MemoryProvenance(MemoryAuthor.HUMAN_VERIFIED, "t", "a", confidence=1.0, recorded_at=NOW)
+ auto = MemoryProvenance(MemoryAuthor.AUTONOMOUS, "t", "a", confidence=1.0, recorded_at=NOW)
+ assert human.effective_confidence(NOW) > auto.effective_confidence(NOW)
+
+
+def test_provenance_to_dict_has_no_anonymous_memory():
+ prov = MemoryProvenance(MemoryAuthor.AUTONOMOUS, "t1", "a1", analyst_id=None, recorded_at=NOW)
+ d = prov.to_dict()
+ assert d["author"] == "autonomous"
+ assert d["tenant_id"] == "t1"
+ assert d["source_alert_id"] == "a1"
+ assert d["trust_weight"] == trust_weight(MemoryAuthor.AUTONOMOUS)
+
+
+# ── Confirmation token + re-disposition plan ─────────────────────────────────
+
+
+def test_confirmation_token_is_order_independent_and_binding():
+ t1 = compute_confirmation_token(["a", "b", "c"], "false_positive")
+ t2 = compute_confirmation_token(["c", "a", "b"], "false_positive")
+ assert t1 == t2 # order-independent (set semantics)
+ assert compute_confirmation_token(["a", "b"], "false_positive") != t1 # different set
+ assert compute_confirmation_token(["a", "b", "c"], "benign") != t1 # different disposition
+
+
+def test_plan_token_matches_and_caps_batch():
+ ids = [f"alert-{i}" for i in range(500)]
+ plan = plan_redisposition(ids, "false_positive", max_batch=200)
+ assert plan.capped is True
+ assert len(plan.alert_ids) == 200
+ assert plan.total_matched == 500
+ assert plan.confirmation_token == compute_confirmation_token(plan.alert_ids, "false_positive")
+ assert any("capped" in r for r in plan.reasons)
+
+
+def test_plan_quarantines_when_flagged():
+ plan = plan_redisposition(["a", "b"], "false_positive", flagged=True)
+ assert plan.quarantined is True
+ assert any("flagged" in r for r in plan.reasons)
+
+
+def test_plan_default_cap():
+ assert DEFAULT_MAX_REDISPOSITION_BATCH == 200
+
+
+# ── Poisoning detector ───────────────────────────────────────────────────────
+
+
+def _fp_burst(sig: str, n: int, *, author: MemoryAuthor, start: datetime, spacing_s: int = 10, analyst_id=None):
+ return [
+ DispositionEvent(
+ signature_key=sig,
+ disposition="false_positive",
+ author=author,
+ at=start + timedelta(seconds=i * spacing_s),
+ analyst_id=analyst_id,
+ )
+ for i in range(n)
+ ]
+
+
+def test_detector_flags_autonomous_fp_burst():
+ sig = "override:v2:deadbeef"
+ events = _fp_burst(sig, 30, author=MemoryAuthor.AUTONOMOUS, start=NOW - timedelta(minutes=10))
+ detector = PoisoningDetector(window_seconds=3600, min_fp_burst=10)
+ verdict = detector.assess(sig, events, now=NOW)
+ assert verdict.flagged is True
+ assert verdict.should_block_autoclose is True
+ assert verdict.fp_count == 30
+ assert verdict.reasons
+
+
+def test_detector_ignores_slow_human_corrections():
+ sig = "override:v2:cafe"
+ # 6 human corrections spread over 6 days — normal analyst tuning, not a burst.
+ events = [
+ DispositionEvent(sig, "false_positive", MemoryAuthor.HUMAN_VERIFIED, NOW - timedelta(days=d), analyst_id=f"u{d}") for d in range(6)
+ ]
+ detector = PoisoningDetector(window_seconds=3600, min_fp_burst=10)
+ verdict = detector.assess(sig, events, now=NOW)
+ assert verdict.flagged is False
+
+
+def test_detector_not_flagged_when_humans_confirm():
+ sig = "override:v2:beef"
+ events = _fp_burst(sig, 20, author=MemoryAuthor.HUMAN_VERIFIED, start=NOW - timedelta(minutes=5), analyst_id="analyst-1")
+ # Many, but all human-verified confirmations -> above the human threshold -> trusted.
+ detector = PoisoningDetector(window_seconds=3600, min_fp_burst=10, max_human_confirmations=2)
+ verdict = detector.assess(sig, events, now=NOW)
+ assert verdict.flagged is False
+
+
+def test_detector_ignores_events_outside_window():
+ sig = "override:v2:old"
+ events = _fp_burst(sig, 30, author=MemoryAuthor.AUTONOMOUS, start=NOW - timedelta(days=2))
+ detector = PoisoningDetector(window_seconds=3600, min_fp_burst=10)
+ verdict = detector.assess(sig, events, now=NOW)
+ assert verdict.fp_count == 0
+ assert verdict.flagged is False
+
+
+def test_detector_scopes_to_signature():
+ sig_a = "override:v2:a"
+ sig_b = "override:v2:b"
+ events = _fp_burst(sig_a, 30, author=MemoryAuthor.AUTONOMOUS, start=NOW - timedelta(minutes=10))
+ detector = PoisoningDetector(min_fp_burst=10)
+ assert detector.assess(sig_a, events, now=NOW).flagged is True
+ assert detector.assess(sig_b, events, now=NOW).flagged is False
+
+
+# ── The eval: farming attack must not teach auto-close ───────────────────────
+
+
+def test_farming_attack_then_real_attack_is_not_auto_closed():
+ """Attacker farms benign alerts under a signature, closes them FP, then
+ runs the real technique. The detector must flag the signature and the
+ retroactive plan must quarantine, so the real attack is NOT auto-closed."""
+ sig = "override:v2:targeted-signature"
+ start = NOW - timedelta(minutes=20)
+
+ # 1) Attacker farms 40 benign alerts under `sig`, auto-closed as FP, with
+ # no independent human confirmation.
+ farmed = _fp_burst(sig, 40, author=MemoryAuthor.AUTONOMOUS, start=start)
+
+ # 2) Detector assessment at the moment the real attack arrives.
+ detector = PoisoningDetector(window_seconds=3600, min_fp_burst=10, max_human_confirmations=2)
+ verdict = detector.assess(sig, farmed, now=NOW)
+
+ # The poisoned signature is flagged -> autonomous closure is blocked.
+ assert verdict.flagged is True
+ assert verdict.should_block_autoclose is True
+
+ # 3) Any retroactive bulk re-disposition under this signature is quarantined
+ # (nothing auto-applies; requires explicit human clearance).
+ real_attack_alert_ids = [f"real-attack-{i}" for i in range(3)]
+ plan = plan_redisposition(real_attack_alert_ids, "false_positive", flagged=verdict.flagged)
+ assert plan.quarantined is True
diff --git a/services/api/tests/test_override_learning.py b/services/api/tests/test_override_learning.py
index b1f103aa9..67c28df32 100644
--- a/services/api/tests/test_override_learning.py
+++ b/services/api/tests/test_override_learning.py
@@ -19,6 +19,7 @@ from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock
import pytest
+from app.services.memory_poisoning import compute_confirmation_token
from app.services.override_learning import (
AlertSignature,
RedispositionCandidate,
@@ -182,6 +183,7 @@ class TestApplyRedisposition:
alert_ids=[],
new_disposition="false_positive",
analyst_id=None,
+ confirmation_token="",
)
assert rowcount == 0
db.execute.assert_not_awaited()
@@ -194,13 +196,49 @@ class TestApplyRedisposition:
db.execute = AsyncMock(return_value=result)
db.commit = AsyncMock()
+ ids = [uuid.uuid4(), uuid.uuid4()]
+ token = compute_confirmation_token([str(i) for i in ids], "false_positive")
rowcount = await apply_redisposition(
db,
tenant_id=uuid.uuid4(),
- alert_ids=[uuid.uuid4(), uuid.uuid4()],
+ alert_ids=ids,
new_disposition="false_positive",
analyst_id=uuid.uuid4(),
+ confirmation_token=token,
)
assert rowcount == 7
db.execute.assert_awaited_once()
db.commit.assert_awaited_once()
+
+ async def test_bad_token_is_rejected(self):
+ db = MagicMock()
+ db.execute = AsyncMock()
+ db.commit = AsyncMock()
+ with pytest.raises(ValueError, match="confirmation_token"):
+ await apply_redisposition(
+ db,
+ tenant_id=uuid.uuid4(),
+ alert_ids=[uuid.uuid4()],
+ new_disposition="false_positive",
+ analyst_id=None,
+ confirmation_token="wrong",
+ )
+ db.execute.assert_not_awaited()
+
+ async def test_batch_over_cap_is_rejected(self):
+ db = MagicMock()
+ db.execute = AsyncMock()
+ db.commit = AsyncMock()
+ ids = [uuid.uuid4() for _ in range(6)]
+ token = compute_confirmation_token([str(i) for i in ids], "false_positive")
+ with pytest.raises(ValueError, match="cap"):
+ await apply_redisposition(
+ db,
+ tenant_id=uuid.uuid4(),
+ alert_ids=ids,
+ new_disposition="false_positive",
+ analyst_id=None,
+ confirmation_token=token,
+ max_batch=5,
+ )
+ db.execute.assert_not_awaited()