Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

feat(phase-C3): autopilot/copilot posture + autonomy scorecard (copilot default) (#459)

Beenu Arorac79e26a2f3eb
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 09b94225a..a9f97c500 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
+- **Phase C3 — autopilot/copilot posture with a visible autonomy scorecard (defaults to copilot).** The per-action guardrail editor already let operators scope autonomy by action; C3 adds the whole-SOC posture view a CISO asks for. New `apps/web/src/components/settings/AutonomyScorecard.tsx` computes an honest posture from the *configured* policy (not fabricated runtime stats): **Copilot** (the safe default — high/critical-blast actions always require a human) vs **Autopilot** (flips only when a high/critical-blast action is configured to auto-execute), plus the distribution of actions by blast radius and auto-exec/override counts. Rendered atop the existing `AutonomyPolicyPanel`. The compute is a pure, unit-tested function; 6 vitest tests (`AutonomyScorecard.test.tsx`). Combined with the Phase B2 `AISOC_MATURITY_TIER` gate (which enforces copilot at the dispatch layer), the platform is copilot-by-default end-to-end. Claim-to-gate matrix +1 GATED (27 GATED / 10 PARTIAL / 1 NO GATE).
- **Phase C1 — Advanced Data Explorer: one investigation surface, no SIEM context-switch.** New `/explore` (`apps/web/src/components/explore/ExploreView.tsx`) unifies the surfaces shipped earlier in the roadmap into a single workbench: ask a question in plain English → it translates to SQL via `/api/v1/nl-query/translate` → runs against the now-populated ClickHouse event lake (`/api/v1/lake/sql`, Phase A1) → renders a BI-like table (row count, latency, referenced tables), with a raw-SQL escape hatch always available. Source tabs pivot to identity (effective permissions), config/graph, and threat intel so the analyst answers "who touched this, with what access, and is the IP known-bad?" without leaving the page. Adds a typed `lakeApi` client, sidebar + command-palette + sitemap entries. 5 vitest smoke tests (`ExploreView.test.tsx`); web type-check + full coverage gate green (395 tests). Claim-to-gate matrix +1 GATED (26 GATED / 10 PARTIAL / 1 NO GATE).
- **Phase C2 — Effective Permissions now resolves against a live posture snapshot.** The resolver is pure (snapshot → effective access), but the only production snapshot source was `_default_snapshot_loader` returning `{}` — so every live "what can this principal do?" call 412'd with "no policy snapshot ingested yet". New `services/api/app/services/effective_permissions/posture_loader.py` collects a real snapshot via the connector's `get_resource_config` read path: a new `POST /connectors/{id}/resource_config` endpoint (same vault-decrypt trust model as `/test` and federated `/query`) exposes it, and `HttpResourceConfigFetcher` + `collect_snapshot` assemble the resolver's snapshot. Coverage is explicit and honest — **Okta** is fully assembled here (user → groups → assigned apps → admin roles, then resolved), while **aws/azure/gcp/gws** consume a connector-provided *reconciled* snapshot (sentinel resource id `__posture_snapshot__`); a provider whose connector hasn't implemented that still 412s (we never fabricate a cloud snapshot). Wired into the endpoint behind `AISOC_EFFECTIVE_PERMISSIONS_LIVE` (default off → prior behaviour preserved), fail-soft to the 412 path on any collection error. 7 API unit tests + the connectors suite (751 passed). Claim-to-gate matrix +1 GATED (25 GATED / 10 PARTIAL / 1 NO GATE).
- **Phase C4 — related alerts now auto-collapse into one ordered attack chain at fuse time.** Correlation grouped alerts into incidents by shared entity and the API could compute a chain per case on demand, but nothing formed/extended a chain **as alerts arrived** — so an analyst saw N separate alerts instead of "step 3 of an intrusion on host X that began 20m ago". New `services/fusion/app/services/attack_chain_grouper.py`: for each entity an alert touches (host/user/ip) it looks up (or mints) a stable `chain_id` in Redis with a rolling window, so a follow-on alert on the same entity — or a *different* entity sharing an IP — joins the same chain. Members are ordered by MITRE kill-chain stage (initial-access → … → impact), so the assignment's `position`/`stage` reflect where in the intrusion this alert sits, not its arrival order. The fusion engine attaches the assignment to `FusedAlert.enrichments["attack_chain"]` (chain_id, position, stage, prior_alert_ids, member_count) for the UI + triage agent. Fail-soft (Redis miss/outage ⇒ no assignment). 8 unit tests; full fusion suite 130 passed at 65% coverage. Claim-to-gate matrix +1 GATED (24 GATED / 10 PARTIAL / 1 NO GATE).
diff --git a/apps/web/src/components/settings/AutonomyPolicy.tsx b/apps/web/src/components/settings/AutonomyPolicy.tsx
index 62130a0e7..eebd86f3b 100644
--- a/apps/web/src/components/settings/AutonomyPolicy.tsx
+++ b/apps/web/src/components/settings/AutonomyPolicy.tsx
@@ -37,6 +37,7 @@ import {
import { Skeleton } from '@/components/ui/Skeleton';
import { ErrorState } from '@/components/ui/ErrorState';
import { EmptyState } from '@/components/ui/EmptyState';
+import { AutonomyScorecard } from '@/components/settings/AutonomyScorecard';
// ─── Helpers ──────────────────────────────────────────────────────────────────
@@ -105,6 +106,13 @@ export function AutonomyPolicyPanel() {
/>
<div className="space-y-5 px-6 py-5">
+ {/* Phase C3 — autopilot/copilot posture + scorecard (defaults to
+ copilot). Rendered once the policy loads so the CISO sees the
+ whole-SOC autonomy posture before drilling into per-action rows. */}
+ {!isLoading && !error && actions.length > 0 ? (
+ <AutonomyScorecard actions={actions} />
+ ) : null}
+
{/* Legend */}
<div className="rounded-lg border border-gray-800 bg-gray-950/40 p-4 text-xs text-gray-400">
<p className="font-medium text-gray-300">How tiers map</p>
diff --git a/apps/web/src/components/settings/AutonomyScorecard.test.tsx b/apps/web/src/components/settings/AutonomyScorecard.test.tsx
new file mode 100644
index 000000000..ec64f085f
--- /dev/null
+++ b/apps/web/src/components/settings/AutonomyScorecard.test.tsx
@@ -0,0 +1,82 @@
+/**
+ * Tests for the autonomy posture scorecard (Phase C3).
+ *
+ * Pins the copilot-default contract: the posture only flips to Autopilot when a
+ * high/critical-blast action is configured to auto-execute; otherwise it stays
+ * Copilot (a human signs off on high-blast actions). Also checks the pure
+ * compute (distribution by blast radius, auto-exec + override counts) and the
+ * rendered summary.
+ */
+
+import { describe, expect, it } from 'vitest';
+import { render, screen } from '@testing-library/react';
+import type { AutonomyActionPolicy, AutonomyBlastRadius } from '@/lib/api';
+import { AutonomyScorecard, computeScorecard } from './AutonomyScorecard';
+
+function action(
+ name: string,
+ blast: AutonomyBlastRadius,
+ auto: number,
+ overridden = false,
+): AutonomyActionPolicy {
+ return {
+ action: name,
+ blast_radius: blast,
+ thresholds: { auto, review: Math.max(0, auto - 0.2), escalation: Math.max(0, auto - 0.4) },
+ default_thresholds: { auto: 1, review: 0.8, escalation: 0.6 },
+ overridden,
+ };
+}
+
+describe('computeScorecard', () => {
+ it('defaults to copilot when no high-blast action auto-executes', () => {
+ const card = computeScorecard([
+ action('notify_slack', 'read', 0.5), // low blast auto-exec is fine
+ action('isolate_host', 'high', 1.0), // auto == 1.0 => never auto
+ action('block_ip', 'medium', 0.9),
+ ]);
+ expect(card.posture).toBe('copilot');
+ expect(card.total).toBe(3);
+ expect(card.autoExecuting).toBe(2); // notify + block_ip
+ expect(card.highBlastAuto).toBe(0);
+ });
+
+ it('flips to autopilot when a high-blast action auto-executes', () => {
+ const card = computeScorecard([
+ action('isolate_host', 'high', 0.85), // high blast, auto-executes
+ ]);
+ expect(card.posture).toBe('autopilot');
+ expect(card.highBlastAuto).toBe(1);
+ });
+
+ it('counts overrides and distribution by blast radius', () => {
+ const card = computeScorecard([
+ action('a', 'read', 0.5, true),
+ action('b', 'read', 0.5),
+ action('c', 'critical', 1.0),
+ ]);
+ expect(card.overridden).toBe(1);
+ expect(card.byBlast.read).toBe(2);
+ expect(card.byBlast.critical).toBe(1);
+ });
+
+ it('handles an empty policy', () => {
+ const card = computeScorecard([]);
+ expect(card.total).toBe(0);
+ expect(card.posture).toBe('copilot');
+ });
+});
+
+describe('AutonomyScorecard', () => {
+ it('renders the Copilot badge by default', () => {
+ render(<AutonomyScorecard actions={[action('isolate_host', 'high', 1.0)]} />);
+ expect(screen.getByText('Copilot')).toBeInTheDocument();
+ expect(screen.getByText(/always require a human/i)).toBeInTheDocument();
+ });
+
+ it('renders the Autopilot badge with a warning when high-blast auto-executes', () => {
+ render(<AutonomyScorecard actions={[action('isolate_host', 'high', 0.8)]} />);
+ expect(screen.getByText('Autopilot')).toBeInTheDocument();
+ expect(screen.getByText(/auto-execute/i)).toBeInTheDocument();
+ });
+});
diff --git a/apps/web/src/components/settings/AutonomyScorecard.tsx b/apps/web/src/components/settings/AutonomyScorecard.tsx
new file mode 100644
index 000000000..17450a1c4
--- /dev/null
+++ b/apps/web/src/components/settings/AutonomyScorecard.tsx
@@ -0,0 +1,145 @@
+'use client';
+
+/**
+ * Autonomy posture scorecard (Phase C3).
+ *
+ * Sits atop the per-action guardrail editor (`AutonomyPolicy.tsx`) and answers
+ * the question a CISO actually asks: "how autonomous is my SOC right now, and
+ * where does a human still sign off?" It computes an honest posture from the
+ * *configured* per-action policy (not fabricated runtime stats):
+ *
+ * - **Posture** — Copilot (the safe default: high/critical-blast actions
+ * always require a human) vs Autopilot (at least one high-blast action is
+ * configured to auto-execute). Data-class scoping is expressed through the
+ * per-action blast radius, so the scorecard groups by blast radius.
+ * - **Distribution** — how many actions auto-execute, queue for review, or
+ * always gate on a human, bucketed by blast radius.
+ *
+ * The compute is a pure function so it's unit-tested directly; the component is
+ * a thin presentational shell.
+ */
+
+import { clsx } from 'clsx';
+import type { AutonomyActionPolicy, AutonomyBlastRadius } from '@/lib/api';
+
+export type AutonomyPosture = 'copilot' | 'autopilot';
+
+export interface AutonomyScorecardData {
+ posture: AutonomyPosture;
+ total: number;
+ overridden: number;
+ /** Actions whose `auto` threshold is reachable (auto-execute is configured). */
+ autoExecuting: number;
+ /** High/critical blast actions configured to auto-execute (the autopilot signal). */
+ highBlastAuto: number;
+ byBlast: Record<AutonomyBlastRadius, number>;
+}
+
+const HIGH_BLAST: ReadonlySet<AutonomyBlastRadius> = new Set(['high', 'critical']);
+
+// An action "auto-executes" when its `auto` threshold is below 1.0 — i.e. some
+// confidence level lets the agent act without a human. auto == 1.0 means
+// "never auto" (always at least review), the safe copilot setting.
+function autoExecutes(a: AutonomyActionPolicy): boolean {
+ return a.thresholds.auto < 1;
+}
+
+export function computeScorecard(actions: AutonomyActionPolicy[]): AutonomyScorecardData {
+ const byBlast: Record<AutonomyBlastRadius, number> = {
+ read: 0,
+ low: 0,
+ medium: 0,
+ high: 0,
+ critical: 0,
+ custom: 0,
+ unknown: 0,
+ };
+ let overridden = 0;
+ let autoExecuting = 0;
+ let highBlastAuto = 0;
+
+ for (const a of actions) {
+ byBlast[a.blast_radius] = (byBlast[a.blast_radius] ?? 0) + 1;
+ if (a.overridden) overridden += 1;
+ if (autoExecutes(a)) {
+ autoExecuting += 1;
+ if (HIGH_BLAST.has(a.blast_radius)) highBlastAuto += 1;
+ }
+ }
+
+ return {
+ // Copilot is the default; only a high/critical-blast action configured to
+ // auto-execute flips the posture to autopilot.
+ posture: highBlastAuto > 0 ? 'autopilot' : 'copilot',
+ total: actions.length,
+ overridden,
+ autoExecuting,
+ highBlastAuto,
+ byBlast,
+ };
+}
+
+export function AutonomyScorecard({ actions }: { actions: AutonomyActionPolicy[] }) {
+ const card = computeScorecard(actions);
+ const isCopilot = card.posture === 'copilot';
+
+ return (
+ <div
+ className="rounded-lg border border-gray-800 bg-gray-950/40 p-4"
+ role="group"
+ aria-label="Autonomy posture scorecard"
+ >
+ <div className="flex flex-wrap items-center justify-between gap-3">
+ <div>
+ <p className="text-xs uppercase tracking-wide text-gray-500">Current posture</p>
+ <div className="mt-1 flex items-center gap-2">
+ <span
+ className={clsx(
+ 'rounded-full px-3 py-1 text-sm font-semibold ring-1 ring-inset',
+ isCopilot
+ ? 'bg-emerald-500/10 text-emerald-300 ring-emerald-500/30'
+ : 'bg-amber-500/10 text-amber-300 ring-amber-500/30',
+ )}
+ >
+ {isCopilot ? 'Copilot' : 'Autopilot'}
+ </span>
+ <span className="text-xs text-gray-400">
+ {isCopilot
+ ? 'High- and critical-blast actions always require a human.'
+ : `${card.highBlastAuto} high-blast action${card.highBlastAuto === 1 ? '' : 's'} auto-execute — review carefully.`}
+ </span>
+ </div>
+ </div>
+ <dl className="flex gap-4 text-right">
+ <Stat label="Actions" value={card.total} />
+ <Stat label="Auto-exec" value={card.autoExecuting} />
+ <Stat label="Overridden" value={card.overridden} />
+ </dl>
+ </div>
+
+ <div className="mt-4 flex flex-wrap gap-2" aria-label="Actions by blast radius">
+ {(Object.entries(card.byBlast) as [AutonomyBlastRadius, number][])
+ .filter(([, n]) => n > 0)
+ .map(([blast, n]) => (
+ <span
+ key={blast}
+ className="rounded-full border border-gray-700 bg-gray-900 px-2.5 py-1 text-[11px] text-gray-300"
+ >
+ {blast}: <span className="font-mono text-gray-100">{n}</span>
+ </span>
+ ))}
+ </div>
+ </div>
+ );
+}
+
+function Stat({ label, value }: { label: string; value: number }) {
+ return (
+ <div>
+ <dt className="text-[11px] uppercase tracking-wide text-gray-500">{label}</dt>
+ <dd className="font-mono text-lg tabular-nums text-gray-100">{value}</dd>
+ </div>
+ );
+}
+
+export default AutonomyScorecard;
diff --git a/docs/audit/CLAIM_TO_GATE_MATRIX.md b/docs/audit/CLAIM_TO_GATE_MATRIX.md
index 29483fddc..823a5893c 100644
--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md
+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md
@@ -45,10 +45,11 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`
| Related alerts auto-collapse into one ordered attack chain at fuse time | README (attack chains) | `ci.yml` fusion job (`test_attack_chain_grouper.py` — shared-entity alerts join one chain within the window, members ordered by MITRE kill-chain stage, cross-entity link via shared IP; wired into `fusion_engine.process` → `enrichments.attack_chain`) | GATED | - |
| Effective Permissions resolves against a live posture snapshot (not empty stub) | README (effective permissions) | `ci.yml` api job (`test_posture_loader.py` — assembles the Okta resolver snapshot from connector `get_resource_config` calls and resolves it; cloud providers consume a connector-provided reconciled snapshot; failures degrade to empty, never fabricated) + connectors `/connectors/{id}/resource_config` endpoint | GATED | - |
| Unified Data Explorer (NL + SQL across the lake) replaces the SIEM context-switch | README (data explorer) | `ci.yml` web-test job (`ExploreView.test.tsx` — NL question → `nl-query/translate` → `lake/sql` → BI table; raw-SQL run; source pivots to identity/graph/intel; error surfaced) + web-lint type-check | GATED | - |
+| Autonomy posture is copilot-by-default with a visible per-action/blast scorecard | README (autopilot/copilot) | `ci.yml` web-test job (`AutonomyScorecard.test.tsx` — posture stays Copilot unless a high/critical-blast action auto-executes; distribution + override counts; rendered badge) | GATED | - |
## Summary
-- GATED: 26
+- GATED: 27
- PARTIAL: 10
- NO GATE: 1 (progressively closed through Phases 2–11 and A1–A4: insecure-defaults, secret/IaC scanning, signed releases, cross-store isolation, DAC candidate-rule + imported-count honesty, connector live-test, OpenAPI breaking-change, ClickHouse lake population (A1), live-stream detection (A2), default cold-boot stack (A3), and the behavioral-model fusion (A4). The **last** NO GATE row — wet-eval live-agent scoreboard tables — closes in Phase 4c/E1, which needs a budgeted live-agent run)