Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
feat(purple-team): self-play adversary — the SOC that attacks itself (v8 P2)
Beenu Arora544bbf960e8f
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 608700a68..139a2d543 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -9,6 +9,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Added+- **v8 P2 — Self-Play Purple Team (the SOC that attacks itself).** New+ `services/purple-team/app/adversary/`: turns the purple-team service from a+ test runner into a continuous adversary. **Hard scope guard**+ (`scope_guard.py`) — a SOC that attacks itself must never touch production, so+ this is enforced in **code** (raises `ScopeViolation` before any step) not as+ a prompt: every target must carry an allowlisted lab tag AND no forbidden+ production tag; no force flag. Adversarial tests cover production assets,+ untagged assets, empty target sets, and a `lab`+`crown-jewel` laundering+ attempt (all hard-fail). **Planner** (`planner.py`) composes an ordered+ kill-chain (initial-access → execution → persistence → privesc → exfil),+ selecting only techniques whose platform exists among the lab targets ("attack+ what exists"). **Closed loop** (`campaign.py`) emits telemetry per step+ (pluggable: in-memory for tests/canned, Kafka on the live path), a detection+ oracle scores detected/missed, and computes detection rate + mean-time-to-+ verdict. **DAC auto-file** (`dac.py`) files one eval-gated Sigma-scaffold+ proposal per miss (status `proposed`, low confidence — self-play can only+ propose, never silently merge). **Scoreboard** (`scoreboard.py` ++ `apps/docs/static/data/selfplay-scoreboard.json`) with a per-row `synthetic`+ flag so a canned campaign is never mistaken for a measured live run. Canned+ 5-stage campaign via `pnpm aisoc:selfplay` (offline, deterministic, ~seconds)+ runs in CI through `test_canned_campaign_runs_end_to_end`. 14 tests total; docs+ at `apps/docs/docs/concepts/self-play.md`. (Nightly live wiring — Kafka emitter+ + alert-store oracle + HTTP DAC filer into the scheduler — is the documented+ remaining integration step.)- **v8 P1 — Federated Threat Intel Mesh (the network effect).** New`services/mesh/` (Python/FastAPI, port 8010): opt-in gossip of twoprivacy-preserving artifact types between self-hosted instances via adiff --git a/apps/docs/docs/concepts/self-play.md b/apps/docs/docs/concepts/self-play.mdnew file mode 100644index 000000000..a51dc9c0a--- /dev/null+++ b/apps/docs/docs/concepts/self-play.md@@ -0,0 +1,72 @@+---+title: Self-play purple team+sidebar_label: Self-play+---++# The SOC that attacks itself++Self-play turns the purple-team service from a test runner into a **continuous+adversary**: an LLM-planned campaign engine composes multi-stage attack chains+from the Atomic Red Team + Caldera inventory, runs them in a closed loop against+the live defense, scores detected/missed per technique, and auto-files+Detection-as-Code proposals for every miss.++Try the canned campaign — deterministic, offline, ~seconds:++```bash+pnpm aisoc:selfplay+```++```+── AiSOC self-play campaign (canned, offline) ──+ [1] initial-access T1566.001 ✗ MISSED Spearphishing Attachment+ [2] execution T1059.001 ✓ DETECTED (12.0s) PowerShell+ [3] persistence T1547.001 ✗ MISSED Registry Run Keys / Startup Folder+ [4] privilege-escalation T1055 ✓ DETECTED (12.0s) Process Injection+ [5] exfiltration T1048 ✓ DETECTED (12.0s) Exfiltration Over Alternative Protocol++ 3/5 techniques detected (60%) · MTTV 12.0s+ Auto-filed 2 Detection-as-Code proposal(s) for the misses (eval-gated review)+```++## The hard scope guard (safety, not a prompt)++The single most important property: a SOC that attacks itself must **never**+touch a production asset. This is enforced in code, not asked of a model.+`services/purple-team/app/adversary/scope_guard.py` hard-fails+(`ScopeViolation`) before any step runs unless **every** target carries an+allowlisted lab tag (`lab`/`sandbox`/`range`/…) **and** carries no forbidden+production tag (`production`/`crown-jewel`/`pci`/…). There is no force flag. The+guard has adversarial tests: production assets, untagged assets, empty target+sets, and a `lab`+`crown-jewel` laundering attempt all hard-fail.++## The closed loop++1. **Plan** — `planner.py` composes an ordered chain (initial-access → execution+ → persistence → privesc → exfil), selecting only techniques whose platform+ actually exists among the lab targets ("attack what exists").+2. **Run** — each step emits telemetry (into Kafka on the live path); the+ defense responds normally.+3. **Score** — a detection oracle reports detected/missed per technique;+ `campaign.py` computes detection rate + mean time to verdict.+4. **Improve** — `dac.py` files one eval-gated Detection-as-Code proposal per+ miss (status `proposed`, low confidence — a scaffold pending eval + human+ review). Self-play can only *propose*, never silently merge.++## The scoreboard++Each campaign appends a row to+[`apps/docs/static/data/selfplay-scoreboard.json`](https://github.com/beenuar/AiSOC/blob/main/apps/docs/static/data/selfplay-scoreboard.json)+(techniques attempted/detected, detection rate, MTTV, new detections filed).+Every row carries an explicit `synthetic` flag: the canned demo campaign is+`synthetic: true`; a nightly live run against the seeded range is+`synthetic: false`. **A synthetic row is never presented as measured live+performance** — the same honesty rule as the rest of the eval harness.++## Status++The planner, hard scope guard, closed-loop scorer, DAC auto-filer, and the+canned campaign are implemented and tested. Wiring the nightly *live* campaign+(Kafka emitter + alert-store oracle + `HttpDacFiler` posting to+`POST /api/v1/detection-proposals`) into the full-profile scheduler is the+remaining live-integration step.diff --git a/apps/docs/sidebars.ts b/apps/docs/sidebars.tsindex c33e98ea5..7527d99c8 100644--- a/apps/docs/sidebars.ts+++ b/apps/docs/sidebars.ts@@ -25,6 +25,7 @@ const sidebars: SidebarsConfig = {"concepts/detections","concepts/live-actions","concepts/automation-maturity",+ "concepts/self-play","concepts/model-router","concepts/llmops",],diff --git a/apps/docs/static/data/selfplay-scoreboard.json b/apps/docs/static/data/selfplay-scoreboard.jsonnew file mode 100644index 000000000..45fdcbc38--- /dev/null+++ b/apps/docs/static/data/selfplay-scoreboard.json@@ -0,0 +1,17 @@+{+ "_comment": "Weekly self-play purple-team scoreboard. Each row is one campaign. The `synthetic` flag distinguishes the canned demo campaign (offline, deterministic) from a nightly live run against the seeded range. Never present a synthetic row as measured live performance.",+ "updated": "2026-07-17",+ "rows": [+ {+ "date": "2026-07-17",+ "version": "unreleased",+ "campaign": "nightly-self-play (canned)",+ "techniques_attempted": 5,+ "techniques_detected": 3,+ "detection_rate": 0.6,+ "mean_time_to_verdict_s": 12.0,+ "new_detections_filed": 2,+ "synthetic": true+ }+ ]+}diff --git a/package.json b/package.jsonindex c82ae7e91..73ba91f76 100644--- a/package.json+++ b/package.json@@ -41,6 +41,8 @@"aisoc:demo:logs": "docker compose -f infra/compose/docker-compose.demo.yml logs -f","aisoc:acceptance": "tsx scripts/aisoc-acceptance.ts","aisoc:acceptance:cold": "tsx scripts/aisoc-acceptance.ts --cold",+ "aisoc:selfplay": "cd services/purple-team && python3 -m app.adversary.canned",+ "aisoc:selfplay:json": "cd services/purple-team && python3 -m app.adversary.canned --json","demo:public": "bash scripts/demo-public.sh","demo:public:tunnel-only": "bash scripts/demo-public.sh --skip-stack","demo:public:setup": "bash infra/cloudflare/tunnel.sh"diff --git a/services/purple-team/app/adversary/__init__.py b/services/purple-team/app/adversary/__init__.pynew file mode 100644index 000000000..ee038d214--- /dev/null+++ b/services/purple-team/app/adversary/__init__.py@@ -0,0 +1,9 @@+"""Self-play purple team (v8 P2).++Turns the purple-team service from a test runner into a continuous adversary:+an LLM-planned campaign engine composes multi-stage attack chains from the+Atomic Red Team + Caldera inventory, runs them in a closed loop against the live+defense, scores detected/missed per technique, and auto-files Detection-as-Code+proposals for every miss — all constrained to an allowlisted lab scope by a hard+code guard (never a prompt instruction).+"""diff --git a/services/purple-team/app/adversary/campaign.py b/services/purple-team/app/adversary/campaign.pynew file mode 100644index 000000000..52686f8f4--- /dev/null+++ b/services/purple-team/app/adversary/campaign.py@@ -0,0 +1,149 @@+"""Closed-loop campaign execution + scoring.++Each campaign step emits real telemetry (in production, into the Kafka spine so+the defense side — detections + verdict engine + agents — responds normally),+then a detection oracle reports whether the technique was caught. The runner+scores detected/missed per technique and produces a report the scoreboard and+the Detection-as-Code auto-filer consume.++Telemetry emission and detection lookup are pluggable (a `Protocol` each) so the+loop is deterministic and offline-testable (`InMemoryEmitter` + `CannedOracle`)+while the live path uses Kafka + the alert store.+"""++from __future__ import annotations++from dataclasses import dataclass, field+from typing import Protocol++from app.adversary.planner import Campaign, CampaignStep+from app.adversary.scope_guard import Asset, LabScope, assert_in_scope+++class TelemetryEmitter(Protocol):+ """Emit the telemetry a step would generate. Returns an opaque event ref."""++ def emit(self, step: CampaignStep) -> dict: ...+++class DetectionOracle(Protocol):+ """Report whether a technique was detected by the live defense."""++ def was_detected(self, technique_id: str, event: dict) -> tuple[bool, float | None, str | None]: ...+++class InMemoryEmitter:+ """Records emitted events instead of producing to Kafka (tests / canned run)."""++ def __init__(self) -> None:+ self.events: list[dict] = []++ def emit(self, step: CampaignStep) -> dict:+ event = {+ "technique_id": step.technique_id,+ "stage": step.stage,+ "target": step.target,+ "source": "self-play",+ }+ self.events.append(event)+ return event+++class CannedOracle:+ """Deterministic oracle: detects techniques in ``detected_ids``, misses the rest."""++ def __init__(self, detected_ids: set[str], latency_s: float = 12.0) -> None:+ self.detected_ids = detected_ids+ self.latency_s = latency_s++ def was_detected(self, technique_id: str, event: dict) -> tuple[bool, float | None, str | None]:+ if technique_id in self.detected_ids:+ return True, self.latency_s, f"alert-{technique_id}"+ return False, None, None+++@dataclass(frozen=True)+class StepResult:+ order: int+ stage: str+ technique_id: str+ technique_name: str+ detected: bool+ latency_s: float | None+ alert_id: str | None+++@dataclass+class CampaignReport:+ name: str+ results: list[StepResult] = field(default_factory=list)++ @property+ def techniques_attempted(self) -> list[str]:+ return [r.technique_id for r in self.results]++ @property+ def detected(self) -> list[StepResult]:+ return [r for r in self.results if r.detected]++ @property+ def missed(self) -> list[StepResult]:+ return [r for r in self.results if not r.detected]++ @property+ def detection_rate(self) -> float:+ return round(len(self.detected) / len(self.results), 4) if self.results else 0.0++ @property+ def mean_time_to_verdict_s(self) -> float | None:+ lat = [r.latency_s for r in self.detected if r.latency_s is not None]+ return round(sum(lat) / len(lat), 2) if lat else None++ def to_dict(self) -> dict:+ return {+ "name": self.name,+ "techniques_attempted": len(self.results),+ "techniques_detected": len(self.detected),+ "detection_rate": self.detection_rate,+ "mean_time_to_verdict_s": self.mean_time_to_verdict_s,+ "missed_techniques": [r.technique_id for r in self.missed],+ "steps": [+ {+ "order": r.order,+ "stage": r.stage,+ "technique_id": r.technique_id,+ "technique_name": r.technique_name,+ "detected": r.detected,+ "latency_s": r.latency_s,+ }+ for r in self.results+ ],+ }+++def run_campaign(+ campaign: Campaign,+ emitter: TelemetryEmitter,+ oracle: DetectionOracle,+ scope: LabScope | None = None,+) -> CampaignReport:+ """Execute a campaign step-by-step. Re-asserts lab scope before any emission."""+ # Backstop safety check at execution time — never trust that planning was safe.+ assert_in_scope(list(campaign.targets) or [Asset.of("__none__")], scope)++ report = CampaignReport(name=campaign.name)+ for step in campaign.steps:+ event = emitter.emit(step)+ detected, latency, alert_id = oracle.was_detected(step.technique_id, event)+ report.results.append(+ StepResult(+ order=step.order,+ stage=step.stage,+ technique_id=step.technique_id,+ technique_name=step.technique_name,+ detected=detected,+ latency_s=latency,+ alert_id=alert_id,+ )+ )+ return reportdiff --git a/services/purple-team/app/adversary/canned.py b/services/purple-team/app/adversary/canned.pynew file mode 100644index 000000000..be39b4903--- /dev/null+++ b/services/purple-team/app/adversary/canned.py@@ -0,0 +1,86 @@+"""The canned demo campaign — `pnpm aisoc:selfplay`.++Runs a deterministic 5-stage attack chain against a seeded *lab* environment,+offline (in-memory telemetry + a canned detection oracle), prints a live-ledger-+style trace + a scoreboard, and shows the Detection-as-Code proposals it would+file for the misses. Zero external dependencies so it runs in demo CI and on a+laptop in seconds.++Run: ``python -m app.adversary.canned`` (from ``services/purple-team``) or+``pnpm aisoc:selfplay`` from the repo root.+"""++from __future__ import annotations++import json+import sys++from app.adversary.campaign import CannedOracle, InMemoryEmitter, run_campaign+from app.adversary.dac import InMemoryFiler, auto_file_gaps+from app.adversary.planner import InventoryTechnique, plan_campaign+from app.adversary.scope_guard import Asset+from app.adversary.scoreboard import build_scoreboard_row++# A small ART/Caldera-style inventory, one+ per stage.+CANNED_INVENTORY: list[InventoryTechnique] = [+ InventoryTechnique("T1566.001", "Spearphishing Attachment", "initial-access", frozenset({"windows"}), 90),+ InventoryTechnique("T1059.001", "PowerShell", "execution", frozenset({"windows"}), 95),+ InventoryTechnique("T1547.001", "Registry Run Keys / Startup Folder", "persistence", frozenset({"windows"}), 80),+ InventoryTechnique("T1055", "Process Injection", "privilege-escalation", frozenset({"windows"}), 85),+ InventoryTechnique("T1048", "Exfiltration Over Alternative Protocol", "exfiltration", frozenset({"windows", "linux"}), 88),+]++# The seeded lab range (note the lab/sandbox tags — the scope guard requires them).+CANNED_TARGETS: list[Asset] = [+ Asset.of("lab-win-01", "lab", "windows"),+ Asset.of("sandbox-win-02", "sandbox", "windows"),+]++# The canned defense catches 3 of 5; the other two are coverage gaps.+CANNED_DETECTED = {"T1059.001", "T1055", "T1048"}+++def run(as_json: bool = False) -> dict:+ campaign = plan_campaign("nightly-self-play (canned)", CANNED_INVENTORY, CANNED_TARGETS)+ emitter = InMemoryEmitter()+ oracle = CannedOracle(CANNED_DETECTED)+ report = run_campaign(campaign, emitter, oracle)++ filer = InMemoryFiler()+ filed = auto_file_gaps(report, filer)+ row = build_scoreboard_row(report, new_detections_filed=len(filed), synthetic=True)++ result = {+ "report": report.to_dict(),+ "dac_proposals_filed": [d.name for d in filer.filed],+ "scoreboard_row": row,+ }++ if not as_json:+ print("── AiSOC self-play campaign (canned, offline) ──\n")+ for r in report.results:+ mark = "✓ DETECTED" if r.detected else "✗ MISSED "+ lat = f" ({r.latency_s}s)" if r.latency_s else ""+ print(f" [{r.order}] {r.stage:<22} {r.technique_id:<12} {mark}{lat} {r.technique_name}")+ print(+ f"\n {len(report.detected)}/{len(report.results)} techniques detected "+ f"({int(report.detection_rate * 100)}%) · MTTV {report.mean_time_to_verdict_s}s"+ )+ if filer.filed:+ print(f"\n Auto-filed {len(filed)} Detection-as-Code proposal(s) for the misses (eval-gated review):")+ for d in filer.filed:+ print(f" · {d.name}")+ print("\n Synthetic canned campaign — the nightly live run scores against the seeded range.")+ return result+++def main() -> int:+ as_json = "--json" in sys.argv+ result = run(as_json=as_json)+ if as_json:+ print(json.dumps(result, indent=2))+ return 0+++if __name__ == "__main__":+ raise SystemExit(main())diff --git a/services/purple-team/app/adversary/dac.py b/services/purple-team/app/adversary/dac.pynew file mode 100644index 000000000..8072bb60a--- /dev/null+++ b/services/purple-team/app/adversary/dac.py@@ -0,0 +1,83 @@+"""Auto-file Detection-as-Code proposals for every self-play miss.++For each technique the campaign executed but the defense missed, we generate a+Sigma-rule scaffold and file it as a Detection-as-Code proposal (status+``proposed``) so it flows through the existing eval-gated DAC review — CI rejects+any candidate that fails its own positive/negative fixtures, so self-play can+only *propose*, never silently merge.++Filing is pluggable: ``InMemoryFiler`` for tests / the canned demo,+``HttpDacFiler`` posts to the API's ``POST /api/v1/detection-proposals``.+"""++from __future__ import annotations++from dataclasses import dataclass+from typing import Protocol++from app.adversary.campaign import CampaignReport, StepResult+++@dataclass(frozen=True)+class DacProposalDraft:+ name: str+ description: str+ rule_language: str+ rule_body: str+ category: str+ severity: str+ confidence: int+ mitre_techniques: list[str]+ tags: list[str]+++def build_gap_proposal(miss: StepResult) -> DacProposalDraft:+ """Build a Sigma scaffold proposal for a missed technique."""+ tid = miss.technique_id+ rule_body = (+ f"title: Self-play gap — {miss.technique_name} ({tid})\n"+ "status: experimental\n"+ f"description: Auto-proposed by the self-play purple team after {tid} "+ f"({miss.stage}) executed without detection. Refine the selection before promoting.\n"+ f"tags:\n - attack.{tid.lower()}\n"+ "logsource:\n category: process_creation\n product: windows\n"+ "detection:\n selection:\n # TODO: encode the observable for this technique\n"+ f" CommandLine|contains: '{miss.technique_name.split()[0].lower()}'\n"+ " condition: selection\n"+ "falsepositives:\n - Legitimate administrative activity\n"+ "level: medium\n"+ )+ return DacProposalDraft(+ name=f"[self-play] Detect {tid} — {miss.technique_name}",+ description=f"Coverage gap surfaced by self-play campaign at the {miss.stage} stage.",+ rule_language="sigma",+ rule_body=rule_body,+ category="purple-team-gap",+ severity="medium",+ confidence=40, # low — it's a scaffold pending eval + human review+ mitre_techniques=[tid],+ tags=["self-play", "coverage-gap", f"stage:{miss.stage}"],+ )+++class DacFiler(Protocol):+ def file(self, draft: DacProposalDraft) -> str: ...+++class InMemoryFiler:+ """Collects proposals instead of POSTing (tests / canned demo)."""++ def __init__(self) -> None:+ self.filed: list[DacProposalDraft] = []++ def file(self, draft: DacProposalDraft) -> str:+ self.filed.append(draft)+ return f"in-memory-{len(self.filed)}"+++def auto_file_gaps(report: CampaignReport, filer: DacFiler) -> list[str]:+ """File one DAC proposal per missed technique. Returns the proposal ids."""+ ids: list[str] = []+ for miss in report.missed:+ ids.append(filer.file(build_gap_proposal(miss)))+ return idsdiff --git a/services/purple-team/app/adversary/planner.py b/services/purple-team/app/adversary/planner.pynew file mode 100644index 000000000..2ec52caaf--- /dev/null+++ b/services/purple-team/app/adversary/planner.py@@ -0,0 +1,107 @@+"""Adversary campaign planner.++Composes an ordered, multi-stage attack chain (initial-access → execution →+persistence → privilege-escalation → exfiltration) from the available Atomic Red+Team / Caldera inventory, parameterized by the tenant's environment — it only+plans techniques whose platform actually exists among the (lab-scoped) targets.++The composition is deterministic (pick the highest-priority inventory technique+per stage that matches an available platform). An LLM can re-order or enrich the+plan, but it can never widen scope: the planner selects only lab targets, and+:func:`~app.adversary.scope_guard.assert_in_scope` hard-fails at run time+regardless of what any model proposed.+"""++from __future__ import annotations++from dataclasses import dataclass, field++from app.adversary.scope_guard import Asset, LabScope, ScopeViolation, assert_in_scope, filter_lab_targets++# The kill-chain stages we compose, in order.+STAGES: tuple[str, ...] = (+ "initial-access",+ "execution",+ "persistence",+ "privilege-escalation",+ "exfiltration",+)+++@dataclass(frozen=True)+class InventoryTechnique:+ """A technique available to the adversary from ART / Caldera."""++ technique_id: str+ technique_name: str+ stage: str+ platforms: frozenset[str] # e.g. {"windows", "linux"}+ priority: int = 50 # higher = preferred when multiple match+++@dataclass(frozen=True)+class CampaignStep:+ order: int+ stage: str+ technique_id: str+ technique_name: str+ target: str+++@dataclass(frozen=True)+class Campaign:+ name: str+ steps: tuple[CampaignStep, ...]+ targets: tuple[Asset, ...] = field(default_factory=tuple)++ @property+ def techniques(self) -> list[str]:+ return [s.technique_id for s in self.steps]+++def _platforms_present(targets: list[Asset]) -> set[str]:+ """Infer available platforms from asset tags (e.g. a 'windows' tag)."""+ known = {"windows", "linux", "macos", "aws", "azure", "gcp", "kubernetes", "office365"}+ present: set[str] = set()+ for t in targets:+ present |= {tag for tag in t.tags if tag in known}+ # If the lab doesn't declare platforms, assume a generic linux/windows range.+ return present or {"windows", "linux"}+++def plan_campaign(+ name: str,+ inventory: list[InventoryTechnique],+ candidate_targets: list[Asset],+ scope: LabScope | None = None,+) -> Campaign:+ """Compose a lab-scoped campaign. Raises ``ScopeViolation`` if no lab target."""+ scope = scope or LabScope()+ targets = filter_lab_targets(candidate_targets, scope)+ # Backstop: hard-fail before we build a plan against anything unsafe.+ assert_in_scope(targets, scope)++ platforms = _platforms_present(targets)+ primary = targets[0].asset_id++ steps: list[CampaignStep] = []+ order = 1+ for stage in STAGES:+ candidates = [t for t in inventory if t.stage == stage and (t.platforms & platforms or not t.platforms)]+ if not candidates:+ continue # attack only what the environment can actually run+ chosen = max(candidates, key=lambda t: t.priority)+ steps.append(+ CampaignStep(+ order=order,+ stage=stage,+ technique_id=chosen.technique_id,+ technique_name=chosen.technique_name,+ target=primary,+ )+ )+ order += 1++ if not steps:+ raise ScopeViolation("no inventory technique matches the lab environment's platforms")+ return Campaign(name=name, steps=tuple(steps), targets=tuple(targets))diff --git a/services/purple-team/app/adversary/scope_guard.py b/services/purple-team/app/adversary/scope_guard.pynew file mode 100644index 000000000..8a0f9173e--- /dev/null+++ b/services/purple-team/app/adversary/scope_guard.py@@ -0,0 +1,77 @@+"""Hard lab-scope guard for the self-play adversary.++The single most important safety property of a SOC that attacks itself: it must+**never** run an attack step against a production asset. This is enforced in+code — a hard failure that raises before any step executes — not as a prompt+instruction the planner could be argued out of.++Every target asset must carry at least one allowlisted lab tag AND carry no+forbidden production tag. Any violation raises :class:`ScopeViolation` and the+campaign aborts. There is deliberately no "force" flag.+"""++from __future__ import annotations++from dataclasses import dataclass, field+++class ScopeViolation(Exception):+ """Raised when a campaign would touch a non-lab / production asset."""+++@dataclass(frozen=True)+class Asset:+ """A candidate target. ``tags`` are environment labels (from the graph)."""++ asset_id: str+ tags: frozenset[str] = field(default_factory=frozenset)++ @classmethod+ def of(cls, asset_id: str, *tags: str) -> Asset:+ return cls(asset_id=asset_id, tags=frozenset(t.strip().lower() for t in tags))+++@dataclass(frozen=True)+class LabScope:+ """Allowlist / denylist of asset tags that define the safe blast radius."""++ allowed_tags: frozenset[str] = frozenset({"lab", "sandbox", "purple-team", "range", "test"})+ forbidden_tags: frozenset[str] = frozenset({"production", "prod", "live", "crown-jewel", "crown_jewels", "pci", "phi", "customer-data"})++ def is_in_scope(self, asset: Asset) -> bool:+ tags = {t.lower() for t in asset.tags}+ if tags & self.forbidden_tags:+ return False+ return bool(tags & self.allowed_tags)+++def assert_in_scope(targets: list[Asset], scope: LabScope | None = None) -> None:+ """Hard-fail if *any* target is out of the lab scope. No bypass, by design.++ Raises :class:`ScopeViolation` on: an empty target set (nothing safe to+ attack), a target carrying a forbidden production tag, or a target lacking+ any allowlisted lab tag.+ """+ scope = scope or LabScope()+ if not targets:+ raise ScopeViolation("refusing to run: no in-scope lab targets were provided")+ offenders: list[str] = []+ for asset in targets:+ tags = {t.lower() for t in asset.tags}+ forbidden = tags & scope.forbidden_tags+ if forbidden:+ offenders.append(f"{asset.asset_id} carries forbidden tag(s) {sorted(forbidden)}")+ elif not (tags & scope.allowed_tags):+ offenders.append(f"{asset.asset_id} has no lab/sandbox tag (tags={sorted(tags) or 'none'})")+ if offenders:+ raise ScopeViolation("self-play scope guard blocked the campaign — targets outside the lab: " + "; ".join(offenders))+++def filter_lab_targets(candidates: list[Asset], scope: LabScope | None = None) -> list[Asset]:+ """Select only in-scope targets (used by the planner to build a safe plan).++ This is a convenience for planning; :func:`assert_in_scope` is still the+ backstop that runs immediately before execution.+ """+ scope = scope or LabScope()+ return [a for a in candidates if scope.is_in_scope(a)]diff --git a/services/purple-team/app/adversary/scoreboard.py b/services/purple-team/app/adversary/scoreboard.pynew file mode 100644index 000000000..5bd5be847--- /dev/null+++ b/services/purple-team/app/adversary/scoreboard.py@@ -0,0 +1,41 @@+"""Self-play scoreboard — turn the repo's own defense improvement into a series.++Aggregates one or more campaign reports into the weekly scoreboard published on+the benchmark page: techniques attempted / detected, detection rate, mean time+to verdict, and new detections filed. Every row carries an explicit ``synthetic``+flag so a canned-campaign row can never be mistaken for a measured live run+(same synthetic-vs-real honesty rule as the rest of the eval harness).+"""++from __future__ import annotations++from datetime import UTC, datetime++from app.adversary.campaign import CampaignReport+++def build_scoreboard_row(+ report: CampaignReport,+ *,+ new_detections_filed: int,+ synthetic: bool,+ version: str = "unreleased",+) -> dict:+ return {+ "date": datetime.now(UTC).strftime("%Y-%m-%d"),+ "version": version,+ "campaign": report.name,+ "techniques_attempted": len(report.results),+ "techniques_detected": len(report.detected),+ "detection_rate": report.detection_rate,+ "mean_time_to_verdict_s": report.mean_time_to_verdict_s,+ "new_detections_filed": new_detections_filed,+ # Honesty flag: True for the canned demo campaign; the nightly live+ # campaign against the seeded range sets this False.+ "synthetic": synthetic,+ }+++def merge_scoreboard(existing: list[dict], row: dict, keep: int = 52) -> list[dict]:+ """Prepend the newest row, keeping at most ``keep`` weeks."""+ return [row, *existing][:keep]diff --git a/services/purple-team/tests/test_adversary.py b/services/purple-team/tests/test_adversary.pynew file mode 100644index 000000000..e7337ae24--- /dev/null+++ b/services/purple-team/tests/test_adversary.py@@ -0,0 +1,141 @@+"""Tests for the self-play purple team (v8 P2).++The safety-critical property — the scope guard hard-fails on any non-lab target+— has adversarial tests. The closed loop (plan → run → score → auto-file DAC) is+verified end-to-end via the canned campaign.+"""++from __future__ import annotations++import pytest+from app.adversary.campaign import CannedOracle, InMemoryEmitter, run_campaign+from app.adversary.canned import run as run_canned+from app.adversary.dac import InMemoryFiler, auto_file_gaps, build_gap_proposal+from app.adversary.planner import STAGES, InventoryTechnique, plan_campaign+from app.adversary.scope_guard import Asset, ScopeViolation, assert_in_scope, filter_lab_targets++INVENTORY = [+ InventoryTechnique("T1566.001", "Spearphishing", "initial-access", frozenset({"windows"}), 90),+ InventoryTechnique("T1059.001", "PowerShell", "execution", frozenset({"windows"}), 95),+ InventoryTechnique("T1547.001", "Run Keys", "persistence", frozenset({"windows"}), 80),+ InventoryTechnique("T1055", "Process Injection", "privilege-escalation", frozenset({"windows"}), 85),+ InventoryTechnique("T1048", "Exfil", "exfiltration", frozenset({"windows"}), 88),+]+LAB = [Asset.of("lab-win-01", "lab", "windows")]+++# ── scope guard (safety-critical, adversarial) ────────────────────────────────+++def test_scope_guard_hard_fails_on_production_asset():+ with pytest.raises(ScopeViolation):+ assert_in_scope([Asset.of("db-prod-01", "production", "windows")])+++def test_scope_guard_hard_fails_on_untagged_asset():+ # An asset with no lab tag is refused — you must opt an asset INTO the range.+ with pytest.raises(ScopeViolation):+ assert_in_scope([Asset.of("mystery-host")])+++def test_scope_guard_hard_fails_on_empty_targets():+ with pytest.raises(ScopeViolation):+ assert_in_scope([])+++def test_scope_guard_hard_fails_when_forbidden_tag_coexists_with_lab_tag():+ # A "lab" tag can't launder a crown-jewel — forbidden tags always win.+ with pytest.raises(ScopeViolation):+ assert_in_scope([Asset.of("weird", "lab", "crown-jewel")])+++def test_scope_guard_allows_lab_and_sandbox():+ assert_in_scope([Asset.of("lab-1", "lab"), Asset.of("sb-1", "sandbox")]) # no raise+++def test_filter_lab_targets_drops_non_lab():+ mixed = [Asset.of("lab-1", "lab"), Asset.of("prod-1", "production"), Asset.of("x")]+ kept = filter_lab_targets(mixed)+ assert [a.asset_id for a in kept] == ["lab-1"]+++# ── planner ───────────────────────────────────────────────────────────────────+++def test_plan_composes_ordered_kill_chain():+ campaign = plan_campaign("c", INVENTORY, LAB)+ assert [s.stage for s in campaign.steps] == list(STAGES)+ assert campaign.steps[0].order == 1+ assert all(s.target == "lab-win-01" for s in campaign.steps)+++def test_plan_only_attacks_present_platforms():+ # A linux-only lab shouldn't plan windows-only techniques.+ linux_lab = [Asset.of("lab-linux-01", "lab", "linux")]+ inv = [+ InventoryTechnique("T1059.004", "Unix Shell", "execution", frozenset({"linux"}), 90),+ InventoryTechnique("T1059.001", "PowerShell", "execution", frozenset({"windows"}), 95),+ ]+ campaign = plan_campaign("c", inv, linux_lab)+ assert campaign.techniques == ["T1059.004"]+++def test_plan_raises_when_no_lab_target():+ with pytest.raises(ScopeViolation):+ plan_campaign("c", INVENTORY, [Asset.of("prod", "production")])+++# ── closed loop ───────────────────────────────────────────────────────────────+++def test_run_campaign_scores_detected_and_missed():+ campaign = plan_campaign("c", INVENTORY, LAB)+ report = run_campaign(campaign, InMemoryEmitter(), CannedOracle({"T1059.001", "T1055"}))+ assert len(report.results) == 5+ assert len(report.detected) == 2+ assert len(report.missed) == 3+ assert report.detection_rate == pytest.approx(0.4)+ assert report.mean_time_to_verdict_s == 12.0+++def test_emitter_records_events_for_every_step():+ campaign = plan_campaign("c", INVENTORY, LAB)+ emitter = InMemoryEmitter()+ run_campaign(campaign, emitter, CannedOracle(set()))+ assert len(emitter.events) == 5+ assert all(e["source"] == "self-play" for e in emitter.events)+++# ── DAC auto-file ─────────────────────────────────────────────────────────────+++def test_auto_file_files_one_proposal_per_miss():+ campaign = plan_campaign("c", INVENTORY, LAB)+ report = run_campaign(campaign, InMemoryEmitter(), CannedOracle({"T1059.001"}))+ filer = InMemoryFiler()+ ids = auto_file_gaps(report, filer)+ assert len(ids) == 4 # 5 attempted, 1 detected → 4 gaps+ assert all(d.category == "purple-team-gap" for d in filer.filed)+ assert all(d.rule_language == "sigma" for d in filer.filed)+++def test_gap_proposal_carries_the_technique_and_is_low_confidence():+ campaign = plan_campaign("c", INVENTORY, LAB)+ miss = run_campaign(campaign, InMemoryEmitter(), CannedOracle(set())).missed[0]+ draft = build_gap_proposal(miss)+ assert draft.mitre_techniques == [miss.technique_id]+ assert f"attack.{miss.technique_id.lower()}" in draft.rule_body+ assert draft.confidence <= 50 # scaffold, pending eval + human review+ assert "self-play" in draft.tags+++# ── canned campaign (demo CI) ─────────────────────────────────────────────────+++def test_canned_campaign_runs_end_to_end():+ result = run_canned(as_json=True)+ assert result["report"]["techniques_attempted"] == 5+ assert result["report"]["techniques_detected"] == 3+ assert len(result["dac_proposals_filed"]) == 2 # 2 misses+ assert result["scoreboard_row"]["synthetic"] is True+ assert result["scoreboard_row"]["new_detections_filed"] == 2