Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

feat(reporting): customizable dashboard / report builder (Wave 7 W7.1) (#580)

Beenu Arora947cd0baf105
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 07ff66e7b..6f3006ace 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,6 +9,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
+- **Customizable dashboard / report builder (Wave 7).** A declarative report is
+ a list of widgets (whitelisted `type` × data `source`), validated for unknown
+ types/sources, duplicate ids, and bounded size, then rendered by resolving
+ each widget's data server-side — resilient to a failing/missing resolver (one
+ bad widget renders an `error`, never breaks the report). `POST
+ /report-builder/validate` + `/render` (the latter wires the real tenant-scoped
+ `alerts_by_severity` resolver). The same definition can drive the live
+ dashboard and an exported report. Gated by `test_report_builder.py` (8 cases).
- **Compliance mapping + agentless CSPM + destinations (Wave 6).** (1) An
**agentless CSPM scan engine** (`cspm.scan_resources` + `POST /posture/scan`):
evaluates a read-only cloud-resource snapshot against misconfiguration checks
diff --git a/docs/audit/CLAIM_TO_GATE_MATRIX.md b/docs/audit/CLAIM_TO_GATE_MATRIX.md
index 460cd26ae..60f5321d7 100644
--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md
+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md
@@ -59,6 +59,7 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`
| Response actions run under least-privilege for the invoking identity | README (autonomy + safety) | `actions` job (`test_authz.py` — `authorize_action` denies a principal lacking the action's blast-radius permission; tier hierarchy + wildcards; tenant binding) | GATED | - |
| Actions-service mutating routes are authenticated (fail-closed in prod) | README (autonomy + safety) | `actions` job (`test_authz.py` — `require_service_auth` returns 401 on a bad/absent bearer token, 503 when unconfigured in production, open only in dev mode) | GATED | - |
| Approvals are bound to a qualified approver (separation of duties) | README (autonomy + safety) | `actions` job (`test_authz.py` — `authorize_approver` requires the action's permission and rejects the requester approving their own action) | GATED | - |
+| Customizable dashboard / report builder validates + renders | README (analytics + reporting) | `ci.yml` api job (`test_report_builder.py` — whitelisted widget types + data sources, unique widget ids, bounded size; render resolves each widget and is resilient to a failing/missing resolver; `/report-builder/render` wires the real tenant-scoped `alerts_by_severity`) | GATED | - |
| Agentless CSPM scans a cloud snapshot for misconfigurations | README (cloud posture) | `ci.yml` api job (`test_compliance_cspm.py` — public S3, world-open security groups on sensitive ports, IAM users without MFA / stale keys, public/unencrypted RDS + EBS all flagged with severity + control refs; clean resources yield nothing) | GATED | - |
| Security findings/detections map to compliance controls with auto-evidence | README (compliance) | `ci.yml` api job (`test_compliance_cspm.py` — MITRE→control mapping, per-control evidence records for detections + CSPM findings, unknown controls skipped) | GATED | - |
@@ -66,7 +67,7 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`
## Summary
-- GATED: 43
+- GATED: 44
- PARTIAL: 9
- NO GATE: 0 (**every claim is now backed by a failing test.** The last NO GATE — the weekly benchmark scoreboard running live against `main` — closed in Phase E1: `scripts/check_scoreboard.py` ties the published scoreboard to a deterministic per-PR live-agent MITRE-accuracy run, while the funded weekly `wet-eval.yml` appends the LLM-tier rows. The full Fully-Operational roadmap (Phases A1–E1) is complete. The 7 remaining PARTIAL rows are honest, named deferrals to future phases outside the A–E scope — each states the specific gap and the phase that closes it — not unproven claims.)
diff --git a/docs/openapi.yaml b/docs/openapi.yaml
index 171b25c03..6a05200cb 100644
--- a/docs/openapi.yaml
+++ b/docs/openapi.yaml
@@ -11961,6 +11961,74 @@ paths:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
+ /api/v1/report-builder/validate:
+ post:
+ tags:
+ - report_builder
+ summary: Validate Report
+ description: 'Validate a report definition (whitelisted widget types + sources,
+ unique
+
+ ids, bounded size) without rendering or persisting it.'
+ operationId: validate_report_api_v1_report_builder_validate_post
+ requestBody:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ReportDefinitionBody'
+ required: true
+ responses:
+ '200':
+ description: Successful Response
+ content:
+ application/json:
+ schema:
+ additionalProperties: true
+ type: object
+ title: Response Validate Report Api V1 Report Builder Validate Post
+ '422':
+ description: Validation Error
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/HTTPValidationError'
+ security:
+ - HTTPBearer: []
+ /api/v1/report-builder/render:
+ post:
+ tags:
+ - report_builder
+ summary: Render
+ description: 'Render a report by resolving each widget''s data server-side.
+ Wires the
+
+ real ``alerts_by_severity`` resolver (tenant-scoped) + ``static``; other
+
+ sources render an honest ``error`` until their resolver is wired.'
+ operationId: render_api_v1_report_builder_render_post
+ requestBody:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ReportDefinitionBody'
+ required: true
+ responses:
+ '200':
+ description: Successful Response
+ content:
+ application/json:
+ schema:
+ additionalProperties: true
+ type: object
+ title: Response Render Api V1 Report Builder Render Post
+ '422':
+ description: Validation Error
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/HTTPValidationError'
+ security:
+ - HTTPBearer: []
/api/v1/airgap/status:
get:
tags:
@@ -23066,6 +23134,23 @@ components:
required:
- yaml
title: ReplaceRulesRequest
+ ReportDefinitionBody:
+ properties:
+ name:
+ type: string
+ maxLength: 200
+ minLength: 1
+ title: Name
+ widgets:
+ items:
+ additionalProperties: true
+ type: object
+ type: array
+ title: Widgets
+ type: object
+ required:
+ - name
+ title: ReportDefinitionBody
ReviewAction:
properties:
action:
diff --git a/services/api/app/api/v1/endpoints/report_builder.py b/services/api/app/api/v1/endpoints/report_builder.py
new file mode 100644
index 000000000..59a932f77
--- /dev/null
+++ b/services/api/app/api/v1/endpoints/report_builder.py
@@ -0,0 +1,63 @@
+"""Customizable dashboard / report builder endpoints (Wave 7, W7.1)."""
+
+from __future__ import annotations
+
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Depends, HTTPException, status
+from pydantic import BaseModel, Field
+from sqlalchemy import func, select
+
+from app.api.v1.deps import AuthUser, DBSession, require_permission
+from app.models.alert import Alert
+from app.services.report_builder import (
+ ReportError,
+ parse_definition,
+ render_report,
+)
+
+router = APIRouter(prefix="/report-builder", tags=["report_builder"])
+
+
+class ReportDefinitionBody(BaseModel):
+ name: str = Field(..., min_length=1, max_length=200)
+ widgets: list[dict[str, Any]] = Field(default_factory=list)
+
+
[email protected]("/validate")
+async def validate_report(
+ body: ReportDefinitionBody,
+ current_user: Annotated[AuthUser, Depends(require_permission("settings:read"))],
+) -> dict[str, Any]:
+ """Validate a report definition (whitelisted widget types + sources, unique
+ ids, bounded size) without rendering or persisting it."""
+ try:
+ definition = parse_definition(body.model_dump())
+ except ReportError as exc:
+ raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
+ return {"ok": True, "name": definition.name, "widget_count": len(definition.widgets)}
+
+
+async def render(
+ body: ReportDefinitionBody,
+ current_user: Annotated[AuthUser, Depends(require_permission("settings:read"))],
+ db: DBSession,
+) -> dict[str, Any]:
+ """Render a report by resolving each widget's data server-side. Wires the
+ real ``alerts_by_severity`` resolver (tenant-scoped) + ``static``; other
+ sources render an honest ``error`` until their resolver is wired."""
+ try:
+ definition = parse_definition(body.model_dump())
+ except ReportError as exc:
+ raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
+
+ async def _alerts_by_severity() -> dict[str, int]:
+ rows = (
+ await db.execute(select(Alert.severity, func.count()).where(Alert.tenant_id == current_user.tenant_id).group_by(Alert.severity))
+ ).all()
+ return {str(sev): int(count) for sev, count in rows}
+
+ severity_counts = await _alerts_by_severity()
+ resolvers = {"alerts_by_severity": lambda params: severity_counts}
+ return render_report(definition, resolvers)
diff --git a/services/api/app/api/v1/router.py b/services/api/app/api/v1/router.py
index 21725aec9..54733b4c9 100644
--- a/services/api/app/api/v1/router.py
+++ b/services/api/app/api/v1/router.py
@@ -62,6 +62,7 @@ from app.api.v1.endpoints import (
realtime,
remediation,
replay,
+ report_builder,
reports,
rule_tuning,
saved_hunts,
@@ -219,6 +220,9 @@ api_router.include_router(easm.router)
api_router.include_router(identity_graph.router)
api_router.include_router(reports.router)
+# Wave 7 — customizable dashboard / report builder
+api_router.include_router(report_builder.router)
+
# Air-gap status snapshot for operators — Tier 3.1 (air-gapped certification)
api_router.include_router(airgap.router)
diff --git a/services/api/app/services/report_builder.py b/services/api/app/services/report_builder.py
new file mode 100644
index 000000000..7e4d03d73
--- /dev/null
+++ b/services/api/app/services/report_builder.py
@@ -0,0 +1,106 @@
+"""Customizable dashboard / report builder (Wave 7, W7.1).
+
+A report is a declarative list of widgets — each a whitelisted ``type`` bound to
+a whitelisted data ``source`` — that the builder validates and then renders by
+resolving each widget's data through injected resolvers. Keeping the assembly
+pure (resolvers are passed in) makes the layout + validation logic unit-testable
+without a database, and lets the same definition drive both the live dashboard
+and an exported PDF/HTML report.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Callable
+from dataclasses import dataclass, field
+from typing import Any
+
+WIDGET_TYPES = frozenset({"metric", "table", "timeseries", "bar", "pie", "text"})
+# Whitelisted data sources a widget may bind to. Extend as new resolvers land.
+DATA_SOURCES = frozenset(
+ {
+ "alerts_by_severity",
+ "alerts_over_time",
+ "mttr",
+ "funnel",
+ "top_detections",
+ "cost_by_model",
+ "posture_summary",
+ "static", # widget carries its own literal data (text/markdown tiles)
+ }
+)
+MAX_WIDGETS = 40
+
+
+class ReportError(Exception):
+ """A report definition is structurally invalid."""
+
+
+@dataclass(frozen=True)
+class ReportWidget:
+ id: str
+ type: str
+ title: str
+ source: str
+ params: dict[str, Any] = field(default_factory=dict)
+
+
+@dataclass(frozen=True)
+class ReportDefinition:
+ name: str
+ widgets: list[ReportWidget]
+
+
+def parse_definition(payload: dict[str, Any]) -> ReportDefinition:
+ """Build + validate a ReportDefinition from a raw dict (API body / stored)."""
+ name = str(payload.get("name", "")).strip()
+ if not name:
+ raise ReportError("report name is required")
+ raw_widgets = payload.get("widgets", [])
+ if not isinstance(raw_widgets, list):
+ raise ReportError("widgets must be a list")
+ if len(raw_widgets) > MAX_WIDGETS:
+ raise ReportError(f"too many widgets ({len(raw_widgets)} > {MAX_WIDGETS})")
+
+ widgets: list[ReportWidget] = []
+ seen_ids: set[str] = set()
+ for i, raw in enumerate(raw_widgets):
+ if not isinstance(raw, dict):
+ raise ReportError(f"widget[{i}] must be an object")
+ wtype = str(raw.get("type", ""))
+ source = str(raw.get("source", ""))
+ wid = str(raw.get("id") or f"w{i}")
+ if wtype not in WIDGET_TYPES:
+ raise ReportError(f"widget[{i}] unknown type '{wtype}'")
+ if source not in DATA_SOURCES:
+ raise ReportError(f"widget[{i}] unknown source '{source}'")
+ if wid in seen_ids:
+ raise ReportError(f"widget[{i}] duplicate id '{wid}'")
+ seen_ids.add(wid)
+ widgets.append(ReportWidget(id=wid, type=wtype, title=str(raw.get("title", "")), source=source, params=dict(raw.get("params", {}))))
+ return ReportDefinition(name=name, widgets=widgets)
+
+
+# A resolver takes a widget's params and returns its rendered data payload.
+Resolver = Callable[[dict[str, Any]], Any]
+
+
+def render_report(definition: ReportDefinition, resolvers: dict[str, Resolver]) -> dict[str, Any]:
+ """Assemble a report by resolving each widget's data. A widget whose source
+ has no resolver (or whose resolver raises) renders an ``error`` field rather
+ than failing the whole report."""
+ rendered: list[dict[str, Any]] = []
+ for widget in definition.widgets:
+ entry: dict[str, Any] = {"id": widget.id, "type": widget.type, "title": widget.title, "source": widget.source}
+ if widget.source == "static":
+ entry["data"] = widget.params.get("data")
+ else:
+ resolver = resolvers.get(widget.source)
+ if resolver is None:
+ entry["error"] = f"no resolver for source '{widget.source}'"
+ else:
+ try:
+ entry["data"] = resolver(widget.params)
+ except Exception as exc: # noqa: BLE001 — one bad widget shouldn't break the report
+ entry["error"] = f"{type(exc).__name__}: {exc}"
+ rendered.append(entry)
+ return {"name": definition.name, "widgets": rendered}
diff --git a/services/api/tests/test_report_builder.py b/services/api/tests/test_report_builder.py
new file mode 100644
index 000000000..5229ffb6b
--- /dev/null
+++ b/services/api/tests/test_report_builder.py
@@ -0,0 +1,85 @@
+"""Wave 7 (W7.1) - the dashboard / report builder."""
+
+from __future__ import annotations
+
+import pytest
+from app.services.report_builder import (
+ ReportError,
+ parse_definition,
+ render_report,
+)
+
+
+def _valid_payload():
+ return {
+ "name": "Exec weekly",
+ "widgets": [
+ {"id": "sev", "type": "bar", "title": "By severity", "source": "alerts_by_severity"},
+ {"id": "mttr", "type": "metric", "title": "MTTR", "source": "mttr"},
+ {"id": "note", "type": "text", "title": "Summary", "source": "static", "params": {"data": "All clear."}},
+ ],
+ }
+
+
+def test_parse_valid_definition():
+ definition = parse_definition(_valid_payload())
+ assert definition.name == "Exec weekly"
+ assert len(definition.widgets) == 3
+
+
+def test_parse_rejects_missing_name():
+ with pytest.raises(ReportError):
+ parse_definition({"widgets": []})
+
+
+def test_parse_rejects_unknown_type():
+ with pytest.raises(ReportError):
+ parse_definition({"name": "x", "widgets": [{"type": "hologram", "source": "mttr"}]})
+
+
+def test_parse_rejects_unknown_source():
+ with pytest.raises(ReportError):
+ parse_definition({"name": "x", "widgets": [{"type": "metric", "source": "secret_db"}]})
+
+
+def test_parse_rejects_duplicate_ids():
+ payload = {
+ "name": "x",
+ "widgets": [
+ {"id": "a", "type": "metric", "source": "mttr"},
+ {"id": "a", "type": "metric", "source": "mttr"},
+ ],
+ }
+ with pytest.raises(ReportError):
+ parse_definition(payload)
+
+
+def test_render_resolves_each_widget():
+ definition = parse_definition(_valid_payload())
+ resolvers = {
+ "alerts_by_severity": lambda params: {"high": 3, "low": 10},
+ "mttr": lambda params: 42.0,
+ }
+ report = render_report(definition, resolvers)
+ by_id = {w["id"]: w for w in report["widgets"]}
+ assert by_id["sev"]["data"] == {"high": 3, "low": 10}
+ assert by_id["mttr"]["data"] == 42.0
+ assert by_id["note"]["data"] == "All clear." # static widget
+
+
+def test_render_is_resilient_to_a_bad_widget():
+ definition = parse_definition(_valid_payload())
+
+ def _boom(_params):
+ raise RuntimeError("resolver down")
+
+ report = render_report(definition, {"alerts_by_severity": _boom, "mttr": lambda p: 1.0})
+ by_id = {w["id"]: w for w in report["widgets"]}
+ assert "error" in by_id["sev"] # bad widget carries an error
+ assert by_id["mttr"]["data"] == 1.0 # others still render
+
+
+def test_render_flags_missing_resolver():
+ definition = parse_definition({"name": "x", "widgets": [{"id": "f", "type": "metric", "source": "funnel"}]})
+ report = render_report(definition, {}) # no resolvers
+ assert "error" in report["widgets"][0]