Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
fix(v7.3.1): alerts schema drift + idempotent migrations + aisoc submit → console (#114)
Beenu Aroraabff3d06cfb2
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 7c390182a..50edcd417 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,54 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+## [7.3.1] — 2026-05-14++### Smoke-test hotfix — `/api/v1/alerts` works on a fresh clone++Tail end of the v7.3.0 founder-flow smoke test on a clean machine revealed+three latent issues that broke the "open the console, see your alert"+moment in the demo. This patch closes all three.++- **Alerts table schema drift** — the `Alert` ORM model+ (`services/api/app/models/alert.py`) declared eleven columns that no+ migration had ever added to the database: `affected_ips`,+ `affected_hosts`, `affected_users`, `affected_assets`,+ `parent_alert_id`, `child_alert_ids`, `is_merged`, `assigned_to_id`,+ `assigned_at`, `enrichment_data`, `tags`. Result: a fresh stack+ returned HTTP 500 from `GET /api/v1/alerts` with+ `column alerts.affected_ips does not exist`. New idempotent migration+ `042_alerts_schema_drift_fix.sql` adds all eleven with sensible+ defaults (`'[]'::jsonb`, `'{}'::jsonb`, `FALSE`, `NULL`).+- **Migration idempotency** — `005_compliance.sql` and+ `025_connectors_click_and_connect.sql` failed re-application on+ partially-migrated DBs (a `CREATE POLICY` that didn't check for+ existence, and a chain of `ALTER TABLE … RENAME COLUMN` calls that+ threw when the source columns were already renamed). Both now wrap+ the offending statements in `DO $$ … END $$` blocks gated on+ `pg_policies` / `information_schema.columns` lookups, so+ `aisoc db upgrade` is fully idempotent.+- **`aisoc submit` → console wiring** — the v7.3.0 CLI POSTed to+ `services/ingest`'s `/v1/ingest/batch`, which only emits to Kafka. With+ the detection / correlation pipeline still wiring up in the demo+ environment, accepted events never became `Alert` rows, so the web+ console stayed empty. New endpoint+ `POST /api/v1/alerts/submit` in `services/api/app/api/v1/endpoints/alerts.py`+ synthesises one `Alert` row directly from the event batch (severity+ normalised across the canonical five-tier ladder, MITRE / affected+ entities derived from the OCSF payload) and persists it. The CLI+ (`packages/aisoc-cli/src/aisoc_cli/main.py`) now targets that endpoint,+ with the same `--tenant` / `--connector-id` / `--source-format`+ overrides, plus dev-mode auth bypass when no `Authorization` header is+ supplied. 16 CLI tests + 48 API tests cover request shape, severity+ mapping, timestamp coercion (`@timestamp` / `time` / Okta `published`),+ affected-user deduplication, and JSONB column defaults.++After this hotfix, the recorded demo runs end-to-end on a fresh clone:+`docker compose up`, `aisoc db upgrade` (clean — no failures),+`aisoc submit examples/alerts/lateral-movement.json` (201, `alert_id`+returned), and the Web console at `http://127.0.0.1:3000/alerts` lists+the new alert with no 500s.+## [7.3.0] — 2026-05-14### Founder-flow series — codebase aligned to the screen-recording demo (PR1–PR7)diff --git a/VERSION b/VERSIONindex 150202076..643916c03 100644--- a/VERSION+++ b/VERSION@@ -1 +1 @@-7.3.0+7.3.1diff --git a/apps/docs/docs/quickstart.md b/apps/docs/docs/quickstart.mdindex ed7902396..0cba586c5 100644--- a/apps/docs/docs/quickstart.md+++ b/apps/docs/docs/quickstart.md@@ -328,30 +328,46 @@ What this does:1. Reads the JSON file. The fixture is self-describing — its`connector_id` / `connector_type` / `source_format` (if present) winover the CLI flags, so the same fixture works against any environment.-2. POSTs to `http://127.0.0.1:8081/v1/ingest/batch` (override with- `--ingest-url` or `AISOC_INGEST_URL`) using the canonical envelope:- `connector_id`, `connector_type`, `source_format`, `events`.+2. POSTs to `http://127.0.0.1:8000/api/v1/alerts/submit` (override with+ `--api-url` or `AISOC_API_URL`) using the canonical envelope:+ `connector_id`, `connector_type`, `source_format`, `events`. The API+ service synthesises a single `Alert` row directly from the batch+ (severity normalised across the canonical five-tier ladder, MITRE /+ affected entities derived from the OCSF payload), persists it, and+ returns the new `alert_id`.3. Sends the required `X-Tenant-ID` header (override with `--tenant-id`- or `AISOC_TENANT_ID`).-4. Prints `accepted` / `rejected` counts plus the `request_id`.+ or `AISOC_TENANT_ID`). When no `Authorization` header is supplied and+ the API is running in dev mode (the default for `docker compose up`),+ the request is authenticated as the demo tenant operator.+4. Prints the `alert_id` plus `accepted` / `rejected` counts.-A non-zero exit code means the ingest service rejected the payload or+A non-zero exit code means the API service rejected the payload orisn't reachable; the error message tells you to run `aisoc serve` firstif the latter.-### 5. Watch fusion + agents pick it up+Why direct-to-API instead of via the ingest spine? The Kafka-based+detection / correlation pipeline is still wiring up in the demo+environment, so events POSTed to `/v1/ingest/batch` accept cleanly but+never become `Alert` rows. The `/api/v1/alerts/submit` endpoint+short-circuits that gap so the recorded demo's "submit → see in console"+moment works on a fresh clone today. Production deployments still flow+events through `services/ingest` → Kafka → fusion → detection; this+endpoint is for fixtures and tabletop exercises.++### 5. Watch the alert land in the console```bash-# Tail the API logs to watch the event flow through fusion → detection-docker compose logs -f api fusion agents+# List alerts via the API+curl -s http://localhost:8000/api/v1/alerts | jq-# Or just open the UI and watch /alerts populate+# Or open the UI and watch /alerts populateopen http://localhost:3000/alerts```-Within a few seconds you should see the two Okta events normalized into-OCSF `class_uid 3002` (Authentication), correlated by user, and surfaced-on the alerts board.+Within a couple of seconds you should see the synthesised alert+(severity `medium`, title derived from the highest-severity event in+the batch, affected user `[email protected]`, affected IPs from both+sessions) on the alerts board.### 6. Hook your IDE in over MCP (optional)diff --git a/apps/web/package.json b/apps/web/package.jsonindex 6647d04a6..425810453 100644--- a/apps/web/package.json+++ b/apps/web/package.json@@ -1,6 +1,6 @@{"name": "@aisoc/web",- "version": "7.3.0",+ "version": "7.3.1","private": true,"description": "AiSOC SOC Console - Next.js 14 Frontend","license": "MIT",diff --git a/docs/openapi.yaml b/docs/openapi.yamlindex e79698fbc..b50d4adc3 100644--- a/docs/openapi.yaml+++ b/docs/openapi.yaml@@ -258,6 +258,54 @@ paths:application/json:schema:$ref: '#/components/schemas/HTTPValidationError'+ /api/v1/alerts/submit:+ post:+ tags:+ - alerts+ summary: Submit Alert+ description: 'Submit one alert directly from an OCSF event batch.+++ This is the destination for `aisoc submit <file>` and the local-dev fast++ path documented in the quickstart. We deliberately bypass the Kafka++ detect/correlate/fuse pipeline (which fresh clones don''t run by default)++ so that a hand-crafted fixture lands in the database — and therefore in++ ``GET /api/v1/alerts`` and the web console — within the same second.+++ Authorisation: requires ``alerts:write``. In development mode the++ auth bypass in :mod:`app.api.v1.deps` resolves to the deterministic++ demo tenant, so the documented ``aisoc submit`` CLI works against a++ fresh clone with no token plumbing.'+ operationId: submit_alert_api_v1_alerts_submit_post+ requestBody:+ content:+ application/json:+ schema:+ $ref: '#/components/schemas/AlertSubmitRequest'+ required: true+ responses:+ '201':+ description: Successful Response+ content:+ application/json:+ schema:+ $ref: '#/components/schemas/AlertResponse'+ '422':+ description: Validation Error+ content:+ application/json:+ schema:+ $ref: '#/components/schemas/HTTPValidationError'+ security:+ - HTTPBearer: []/api/v1/alerts:get:tags:@@ -12978,6 +13026,65 @@ components:- new_last_24h- critical_opentitle: AlertStatsResponse+ AlertSubmitRequest:+ properties:+ events:+ items:+ additionalProperties: true+ type: object+ type: array+ title: Events+ default: []+ connector_id:+ anyOf:+ - type: string+ - type: 'null'+ title: Connector Id+ connector_type:+ anyOf:+ - type: string+ - type: 'null'+ title: Connector Type+ source_format:+ anyOf:+ - type: string+ - type: 'null'+ title: Source Format+ title:+ anyOf:+ - type: string+ - type: 'null'+ title: Title+ description:+ anyOf:+ - type: string+ - type: 'null'+ title: Description+ severity:+ anyOf:+ - type: string+ - type: 'null'+ title: Severity+ tags:+ anyOf:+ - items:+ type: string+ type: array+ - type: 'null'+ title: Tags+ type: object+ title: AlertSubmitRequest+ description: "Direct-write alert submission payload.\n\nThe founder-flow demo\+ \ (`aisoc submit <file>`) and any operator who wants\nto land a hand-crafted\+ \ alert on the local dev stack without a connector\nPOST one of these. The\+ \ body intentionally mirrors the ingest envelope\nused by real connectors\+ \ so the same fixture works against either path:\n\n { connector_id, connector_type,\+ \ source_format, events: [...] }\n\nThe single payload synthesises one ``Alert``\+ \ row, written directly into\nthe tenant's database, so the result is visible\+ \ in\n``GET /api/v1/alerts`` and the web console within the same second.\n\+ This deliberately bypasses the Kafka detection/correlation pipeline\n(which\+ \ fresh clones don't run by default) so the documented \"alert in\nseconds\"\+ \ promise in the quickstart actually holds."AlertUpdateRequest:properties:status:diff --git a/packages/aisoc-cli/src/aisoc_cli/main.py b/packages/aisoc-cli/src/aisoc_cli/main.pyindex a460407e1..9f40c4d50 100644--- a/packages/aisoc-cli/src/aisoc_cli/main.py+++ b/packages/aisoc-cli/src/aisoc_cli/main.py@@ -12,7 +12,7 @@ Commands:aisoc db upgrade Run database migrations against the dev stackaisoc mcp serve [--transport] Launch the MCP server for IDE assistantsaisoc mcp install --host <h> Wire AiSOC into Claude / Cursor / Continue- aisoc submit <file> POST an alert/event JSON payload to the ingest API+ aisoc submit <file> POST an alert/event JSON payload to the AiSOC API"""from __future__ import annotations@@ -708,21 +708,33 @@ def mcp_install(host: str) -> None:## `aisoc submit <file>` lets the founder-flow demo and any operator land a# canned OCSF/Okta-shaped payload on the local dev stack in one command.-# It POSTs to the Go ingest service's batch endpoint with the same envelope-# the real connectors use (see services/connectors/app/ingest_client.py):+# It POSTs to the API's direct-write submit endpoint, which synthesises an+# Alert row from the event batch and writes it straight to the database — so+# the result is visible in the web console within the same second.#-# POST {ingest_url}/v1/ingest/batch-# X-Tenant-ID: <uuid>+# POST {api_url}/api/v1/alerts/submit+# Authorization: Bearer <token> # optional in dev mode# { connector_id, connector_type, source_format, events: [...] }#+# This deliberately bypasses the Kafka detect/correlate/fuse pipeline (which+# fresh clones don't run by default) so the documented "alert in seconds"+# quickstart promise actually holds.+## The fixture format is intentionally forgiving — the file may be:# * { "events": [...], "connector_id": ..., "connector_type": ..., "source_format": ... }# * a bare JSON list of event dicts# * a single event dict (auto-wrapped)-# Any keys present in a dict-shaped fixture override the matching CLI flags,-# so a fixture can pin its own connector identity without operator flags.--_DEFAULT_INGEST_URL = "http://127.0.0.1:8081"+# Any keys present in a dict-shaped fixture override the matching CLI flags.+#+# Backward compatibility: pre-W3 versions of this command POSTed to the Go+# ingest service at port 8081 with an ``X-Tenant-ID`` header. We keep the+# ``--ingest-url`` flag and ``AISOC_INGEST_URL`` env var as deprecated aliases+# for ``--api-url`` / ``AISOC_API_URL`` so existing demo scripts and the v1.0+# video voiceover keep working. The tenant id, when not derivable from the+# bearer token (dev mode bypass), comes from ``--tenant-id`` /+# ``AISOC_TENANT_ID`` and is still echoed in the request panel.++_DEFAULT_API_URL = "http://127.0.0.1:8000"_DEFAULT_TENANT_ID = "00000000-0000-0000-0000-000000000001"_DEFAULT_CONNECTOR_ID = "aisoc-cli-submit"_DEFAULT_CONNECTOR_TYPE = "okta_system_log"@@ -767,42 +779,76 @@ def _coerce_events(payload: Any) -> tuple[list[dict[str, Any]], dict[str, str]]:return cleaned, overrides+def _resolve_api_url(api_url: str, ingest_url: str | None) -> str:+ """Pick the effective API base URL, honouring deprecated `--ingest-url`.++ Precedence (highest first):+ 1. ``--ingest-url`` / ``AISOC_INGEST_URL`` (deprecated, prints a warning)+ 2. ``--api-url`` / ``AISOC_API_URL``+ 3. default ``http://127.0.0.1:8000``+ """+ if ingest_url:+ console.print(+ "[yellow]--ingest-url / AISOC_INGEST_URL is deprecated; "+ "use --api-url / AISOC_API_URL instead.[/yellow]"+ )+ return ingest_url+ return api_url++@cli.command()@click.argument("file",type=click.Path(exists=True, dir_okay=False, readable=True, path_type=Path),)+ "--api-url",+ envvar="AISOC_API_URL",+ default=_DEFAULT_API_URL,+ show_default=True,+ help="Base URL for the AiSOC API (e.g. http://127.0.0.1:8000).",+)@click.option("--ingest-url",envvar="AISOC_INGEST_URL",- default=_DEFAULT_INGEST_URL,- show_default=True,- help="Base URL for the ingest service (e.g. http://127.0.0.1:8081).",+ default=None,+ show_default=False,+ help="DEPRECATED alias for --api-url; overrides --api-url when set.",+)+ "--api-key",+ envvar="AISOC_API_KEY",+ default=None,+ show_default=False,+ help=(+ "AiSOC bearer token. Optional in dev mode (the API resolves an "+ "unauthenticated request to the demo tenant)."+ ),)@click.option("--tenant-id",envvar="AISOC_TENANT_ID",default=_DEFAULT_TENANT_ID,show_default=True,- help="Tenant UUID sent in the X-Tenant-ID header.",+ help="Tenant UUID — informational; the API derives tenant from the bearer token.",)@click.option("--connector-id",default=_DEFAULT_CONNECTOR_ID,show_default=True,- help="connector_id field in the ingest envelope.",+ help="connector_id field on the synthesised alert.",)@click.option("--connector-type",default=_DEFAULT_CONNECTOR_TYPE,show_default=True,- help="connector_type field — chooses the normalizer profile (e.g. okta_system_log).",+ help="connector_type field — used for routing and tag inference (e.g. okta_system_log).",)@click.option("--source-format",default=_DEFAULT_SOURCE_FORMAT,show_default=True,- help="source_format hint for the normalizer.",+ help="source_format hint, echoed onto the alert for downstream parsers.",)@click.option("--timeout",@@ -813,19 +859,25 @@ def _coerce_events(payload: Any) -> tuple[list[dict[str, Any]], dict[str, str]]:)def submit(file: Path,- ingest_url: str,+ api_url: str,+ ingest_url: str | None,+ api_key: str | None,tenant_id: str,connector_id: str,connector_type: str,source_format: str,timeout: float,) -> None:- """Submit a JSON alert/event payload to the local ingest service.+ """Submit a JSON alert/event payload to the AiSOC API.- Reads ``FILE``, normalizes it into the ingest batch envelope, and POSTs to- ``{ingest_url}/v1/ingest/batch`` with the tenant header. Used by the- quickstart video script to drop a lateral-movement alert into the stack- in one command.+ Reads ``FILE``, normalises it into the submit envelope, and POSTs to+ ``{api_url}/api/v1/alerts/submit``. The API synthesises one ``Alert`` row+ from the batch and writes it directly to the database, so the result is+ visible in ``GET /api/v1/alerts`` and the web console within the same+ second — bypassing the Kafka pipeline that fresh clones don't run.++ Used by the quickstart video script to drop a lateral-movement alert+ into the stack in one command."""try:raw = file.read_text(encoding="utf-8")@@ -846,7 +898,8 @@ def submit(connector_type = overrides.get("connector_type", connector_type)source_format = overrides.get("source_format", source_format)- url = f"{ingest_url.rstrip('/')}/v1/ingest/batch"+ effective_url = _resolve_api_url(api_url, ingest_url)+ url = f"{effective_url.rstrip('/')}/api/v1/alerts/submit"body = {"connector_id": connector_id,"connector_type": connector_type,@@ -867,24 +920,23 @@ def submit())- headers = {- "Content-Type": "application/json",- "X-Tenant-ID": tenant_id,- }+ headers: dict[str, str] = {"Content-Type": "application/json"}+ if api_key:+ headers["Authorization"] = f"Bearer {api_key}"try:with httpx.Client(timeout=timeout) as client:resp = client.post(url, headers=headers, json=body)except httpx.HTTPError as exc:console.print(- f"[red]Ingest service unreachable at {url}:[/red] {exc}\n"+ f"[red]AiSOC API unreachable at {url}:[/red] {exc}\n""Is the dev stack running? Try [bold]aisoc serve[/bold] first.")sys.exit(1)if resp.status_code >= 400:console.print(- f"[red]Ingest service returned {resp.status_code}:[/red] {resp.text[:500]}"+ f"[red]AiSOC API returned {resp.status_code}:[/red] {resp.text[:500]}")sys.exit(1)@@ -892,34 +944,27 @@ def submit(data = resp.json()except ValueError:console.print(- f"[red]Ingest service returned non-JSON ({resp.status_code}):[/red] "+ f"[red]AiSOC API returned non-JSON ({resp.status_code}):[/red] "f"{resp.text[:200]}")sys.exit(1)- accepted = data.get("accepted", 0)- rejected = data.get("rejected", 0)- request_id = data.get("request_id", "-")- errors = data.get("errors") or []+ alert_id = data.get("id", "-")+ alert_severity = data.get("severity", "-")+ alert_title = data.get("title", "-")+ alert_tenant = data.get("tenant_id", "-")console.print(Panel(f"[bold]status:[/bold] {resp.status_code}\n"- f"[bold]accepted:[/bold] {accepted}\n"- f"[bold]rejected:[/bold] {rejected}\n"- f"[bold]request_id:[/bold] {request_id}",- title="[bold green]ingest response[/bold green]",+ f"[bold]alert_id:[/bold] {alert_id}\n"+ f"[bold]tenant_id:[/bold] {alert_tenant}\n"+ f"[bold]severity:[/bold] {alert_severity}\n"+ f"[bold]title:[/bold] {alert_title}",+ title="[bold green]alert created[/bold green]",))- if errors:- console.print("[yellow]errors:[/yellow]")- for err in errors:- console.print(f" • {err}")-- if rejected and not accepted:- sys.exit(1)-if __name__ == "__main__":cli()diff --git a/packages/aisoc-cli/tests/test_submit_command.py b/packages/aisoc-cli/tests/test_submit_command.pyindex 7428e86da..1a70f7fc6 100644--- a/packages/aisoc-cli/tests/test_submit_command.py+++ b/packages/aisoc-cli/tests/test_submit_command.py@@ -3,17 +3,19 @@These tests pin the contract the founder-flow quickstart relies on:* ``aisoc submit <file>`` is registered and discoverable via ``--help``.-* The CLI POSTs to ``{ingest_url}/v1/ingest/batch`` with an ``X-Tenant-ID``- header and the canonical ingest envelope shape (connector_id,- connector_type, source_format, events).+* The CLI POSTs to ``{api_url}/api/v1/alerts/submit`` with the canonical+ submit envelope (connector_id, connector_type, source_format, events).* Fixture-level overrides (a file with ``connector_id`` / ``connector_type``/ ``source_format`` keys) win over the matching CLI flags.-* The CLI surfaces accepted / rejected counts on success, and exits non-zero- on transport errors, 4xx responses, or non-JSON responses.+* The CLI surfaces the synthesised alert (id, severity, title) on success,+ and exits non-zero on transport errors, 4xx responses, or non-JSON+ responses.+* Backward compat: the legacy ``--ingest-url`` flag still works and is+ treated as an alias for ``--api-url``.We use ``httpx.MockTransport`` to intercept the HTTP call without standing-up a real ingest service. If these tests break, the quickstart video will-desync from the CLI.+up a real API. If these tests break, the quickstart video desyncs from the+CLI."""from __future__ import annotations@@ -40,6 +42,26 @@ def _write_fixture(tmp_path: Path, payload: Any, name: str = "alert.json") -> Pareturn path+def _ok_alert_response(+ *,+ alert_id: str = "11111111-2222-4333-8444-555555555555",+ tenant_id: str = "00000000-0000-0000-0000-000000000001",+ title: str = "User login to Okta",+ severity: str = "medium",+) -> dict[str, Any]:+ """Build a minimally-valid AlertResponse for mock transports.++ The CLI only reads four fields, so this stays focused on what the+ surface contract actually requires.+ """+ return {+ "id": alert_id,+ "tenant_id": tenant_id,+ "title": title,+ "severity": severity,+ }++class _CapturingTransport(httpx.MockTransport):"""Mock transport that records each request and returns a canned response."""@@ -78,11 +100,19 @@ def test_submit_help_lists_command(runner: CliRunner) -> None:def test_submit_help_shows_options(runner: CliRunner) -> None:result = runner.invoke(cli, ["submit", "--help"])assert result.exit_code == 0, result.output- for flag in ("--ingest-url", "--tenant-id", "--connector-id", "--connector-type", "--source-format"):+ for flag in (+ "--api-url",+ "--api-key",+ "--ingest-url",+ "--tenant-id",+ "--connector-id",+ "--connector-type",+ "--source-format",+ ):assert flag in result.output-def test_submit_posts_envelope_with_tenant_header(+def test_submit_posts_to_alerts_submit_endpoint(runner: CliRunner,tmp_path: Path,monkeypatch: pytest.MonkeyPatch,@@ -98,24 +128,22 @@ def test_submit_posts_envelope_with_tenant_header()def respond(_: httpx.Request) -> httpx.Response:- return httpx.Response(- 200,- json={"accepted": 2, "rejected": 0, "request_id": "req-001"},- )+ return httpx.Response(201, json=_ok_alert_response())transport = _install_transport(monkeypatch, respond)result = runner.invoke(cli,- ["submit", str(fixture_path), "--ingest-url", "http://localhost:8081"],+ ["submit", str(fixture_path), "--api-url", "http://localhost:8000"],)assert result.exit_code == 0, result.outputassert len(transport.requests) == 1req = transport.requests[0]assert req.method == "POST"- assert str(req.url) == "http://localhost:8081/v1/ingest/batch"- assert req.headers["X-Tenant-ID"] == "00000000-0000-0000-0000-000000000001"+ assert str(req.url) == "http://localhost:8000/api/v1/alerts/submit"+ # No Authorization header in dev mode (the API resolves the demo user).+ assert "authorization" not in {k.lower() for k in req.headers.keys()}assert req.headers["Content-Type"].startswith("application/json")body = json.loads(req.content.decode("utf-8"))@@ -124,8 +152,30 @@ def test_submit_posts_envelope_with_tenant_header(assert body["source_format"] == "json"assert len(body["events"]) == 2- assert "accepted: 2" in result.output- assert "request_id: req-001" in result.output+ assert "alert_id" in result.output+ assert "severity" in result.output+++def test_submit_sends_authorization_when_api_key_set(+ runner: CliRunner,+ tmp_path: Path,+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ fixture_path = _write_fixture(tmp_path, {"events": [{"uuid": "x"}]})++ transport = _install_transport(+ monkeypatch,+ lambda _: httpx.Response(201, json=_ok_alert_response()),+ )++ result = runner.invoke(+ cli,+ ["submit", str(fixture_path), "--api-key", "aisoc_test_token_123"],+ )+ assert result.exit_code == 0, result.output++ req = transport.requests[0]+ assert req.headers["Authorization"] == "Bearer aisoc_test_token_123"def test_submit_accepts_bare_list_payload(@@ -137,7 +187,7 @@ def test_submit_accepts_bare_list_payload(transport = _install_transport(monkeypatch,- lambda _: httpx.Response(200, json={"accepted": 2, "rejected": 0}),+ lambda _: httpx.Response(201, json=_ok_alert_response()),)result = runner.invoke(cli, ["submit", str(fixture_path)])@@ -155,7 +205,7 @@ def test_submit_wraps_single_object_payload(transport = _install_transport(monkeypatch,- lambda _: httpx.Response(200, json={"accepted": 1, "rejected": 0}),+ lambda _: httpx.Response(201, json=_ok_alert_response()),)result = runner.invoke(cli, ["submit", str(fixture_path)])@@ -181,7 +231,7 @@ def test_submit_fixture_overrides_beat_cli_flags(transport = _install_transport(monkeypatch,- lambda _: httpx.Response(200, json={"accepted": 1, "rejected": 0}),+ lambda _: httpx.Response(201, json=_ok_alert_response()),)result = runner.invoke(@@ -213,7 +263,7 @@ def test_submit_uses_cli_flags_when_fixture_has_no_overrides(transport = _install_transport(monkeypatch,- lambda _: httpx.Response(200, json={"accepted": 1, "rejected": 0}),+ lambda _: httpx.Response(201, json=_ok_alert_response()),)result = runner.invoke(@@ -221,8 +271,6 @@ def test_submit_uses_cli_flags_when_fixture_has_no_overrides(["submit",str(fixture_path),- "--tenant-id",- "deadbeef-0000-4000-8000-000000000000","--connector-id","demo-cli","--connector-type",@@ -230,31 +278,56 @@ def test_submit_uses_cli_flags_when_fixture_has_no_overrides(],)assert result.exit_code == 0, result.output- req = transport.requests[0]- assert req.headers["X-Tenant-ID"] == "deadbeef-0000-4000-8000-000000000000"- body = json.loads(req.content.decode("utf-8"))+ body = json.loads(transport.requests[0].content.decode("utf-8"))assert body["connector_id"] == "demo-cli"+ assert body["connector_type"] == "okta_system_log"-def test_submit_env_overrides_default_url_and_tenant(+def test_submit_env_overrides_default_api_url(runner: CliRunner,tmp_path: Path,monkeypatch: pytest.MonkeyPatch,) -> None:fixture_path = _write_fixture(tmp_path, {"events": [{"uuid": "x"}]})- monkeypatch.setenv("AISOC_INGEST_URL", "http://ingest-worker:8080")- monkeypatch.setenv("AISOC_TENANT_ID", "11111111-1111-4111-8111-111111111111")+ monkeypatch.setenv("AISOC_API_URL", "http://api-worker:8000")transport = _install_transport(monkeypatch,- lambda _: httpx.Response(200, json={"accepted": 1, "rejected": 0}),+ lambda _: httpx.Response(201, json=_ok_alert_response()),)result = runner.invoke(cli, ["submit", str(fixture_path)])assert result.exit_code == 0, result.outputreq = transport.requests[0]- assert str(req.url) == "http://ingest-worker:8080/v1/ingest/batch"- assert req.headers["X-Tenant-ID"] == "11111111-1111-4111-8111-111111111111"+ assert str(req.url) == "http://api-worker:8000/api/v1/alerts/submit"+++def test_submit_deprecated_ingest_url_still_works(+ runner: CliRunner,+ tmp_path: Path,+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ """Backward compat: --ingest-url is a deprecated alias for --api-url.++ Pre-W3 demo scripts and the v1.0 quickstart voiceover still set+ ``--ingest-url http://127.0.0.1:8081``. We keep the flag working+ (with a deprecation warning) so neither breaks.+ """+ fixture_path = _write_fixture(tmp_path, {"events": [{"uuid": "x"}]})++ transport = _install_transport(+ monkeypatch,+ lambda _: httpx.Response(201, json=_ok_alert_response()),+ )++ result = runner.invoke(+ cli,+ ["submit", str(fixture_path), "--ingest-url", "http://legacy-host:8081"],+ )+ assert result.exit_code == 0, result.output+ req = transport.requests[0]+ assert str(req.url) == "http://legacy-host:8081/api/v1/alerts/submit"+ assert "deprecated" in result.output.lower()def test_submit_returns_nonzero_on_4xx(@@ -265,12 +338,12 @@ def test_submit_returns_nonzero_on_4xx(fixture_path = _write_fixture(tmp_path, {"events": [{"uuid": "x"}]})_install_transport(monkeypatch,- lambda _: httpx.Response(400, text="missing X-Tenant-ID"),+ lambda _: httpx.Response(400, text="events must be a non-empty list"),)result = runner.invoke(cli, ["submit", str(fixture_path)])assert result.exit_code == 1, result.output- assert "Ingest service returned 400" in result.output+ assert "AiSOC API returned 400" in result.outputdef test_submit_returns_nonzero_on_transport_error(@@ -287,7 +360,7 @@ def test_submit_returns_nonzero_on_transport_error(result = runner.invoke(cli, ["submit", str(fixture_path)])assert result.exit_code == 1, result.output- assert "Ingest service unreachable" in result.output+ assert "AiSOC API unreachable" in result.outputassert "aisoc serve" in result.output@@ -337,7 +410,12 @@ def test_submit_real_lateral_movement_fixture(transport = _install_transport(monkeypatch,- lambda _: httpx.Response(200, json={"accepted": 2, "rejected": 0, "request_id": "req-9"}),+ lambda _: httpx.Response(+ 201,+ json=_ok_alert_response(+ title="User login to Okta", severity="medium"+ ),+ ),)result = runner.invoke(cli, ["submit", str(fixture_path)])diff --git a/services/api/app/api/v1/endpoints/alerts.py b/services/api/app/api/v1/endpoints/alerts.pyindex 7ab607690..8343546ff 100644--- a/services/api/app/api/v1/endpoints/alerts.py+++ b/services/api/app/api/v1/endpoints/alerts.py@@ -129,6 +129,277 @@ class AlertStatsResponse(BaseModel):critical_open: int+class AlertSubmitRequest(BaseModel):+ """Direct-write alert submission payload.++ The founder-flow demo (`aisoc submit <file>`) and any operator who wants+ to land a hand-crafted alert on the local dev stack without a connector+ POST one of these. The body intentionally mirrors the ingest envelope+ used by real connectors so the same fixture works against either path:++ { connector_id, connector_type, source_format, events: [...] }++ The single payload synthesises one ``Alert`` row, written directly into+ the tenant's database, so the result is visible in+ ``GET /api/v1/alerts`` and the web console within the same second.+ This deliberately bypasses the Kafka detection/correlation pipeline+ (which fresh clones don't run by default) so the documented "alert in+ seconds" promise in the quickstart actually holds.+ """++ events: list[dict] = []+ connector_id: str | None = None+ connector_type: str | None = None+ source_format: str | None = None+ title: str | None = None+ description: str | None = None+ severity: str | None = None+ tags: list[str] | None = None+++# Map vendor-native severity strings to the canonical AiSOC severity ladder.+# Keep this in sync with the five-tier ladder documented in AGENTS.md+# (info | low | medium | high | critical).+_SEVERITY_MAP = {+ "info": "info",+ "informational": "info",+ "debug": "info",+ "low": "low",+ "minor": "low",+ "warning": "low",+ "medium": "medium",+ "moderate": "medium",+ "warn": "medium",+ "high": "high",+ "error": "high",+ "major": "high",+ "critical": "critical",+ "crit": "critical",+ "severe": "critical",+ "fatal": "critical",+ "emergency": "critical",+}+_SEVERITY_PRIORITY = {+ "info": 10,+ "low": 30,+ "medium": 50,+ "high": 75,+ "critical": 95,+}+++def _normalize_severity(raw: str | None) -> str:+ if not raw:+ return "medium"+ return _SEVERITY_MAP.get(raw.strip().lower(), "medium")+++def _max_severity(values: list[str]) -> str:+ """Pick the highest severity from a list using the canonical ladder."""+ ladder = ["info", "low", "medium", "high", "critical"]+ best = "info"+ for v in values:+ if ladder.index(v) > ladder.index(best):+ best = v+ return best+++def _coerce_published(event: dict) -> datetime | None:+ """Extract an event timestamp from an OCSF / Okta-shaped dict.++ Accepts ``published`` (Okta), ``time`` (OCSF), or ``@timestamp``. Returns+ ``None`` when the field is missing or unparseable so the caller can fall+ back to ``datetime.now(UTC)``.+ """+ for key in ("published", "time", "@timestamp", "event_time"):+ value = event.get(key)+ if not value:+ continue+ if isinstance(value, datetime):+ return value if value.tzinfo else value.replace(tzinfo=UTC)+ if isinstance(value, str):+ try:+ # Tolerate trailing 'Z' which fromisoformat handles natively+ # on Python 3.11+ but not in some older variants.+ normalized = value.rstrip("Z") + "+00:00" if value.endswith("Z") else value+ return datetime.fromisoformat(normalized)+ except ValueError:+ continue+ return None+++def _synthesise_alert_from_events(+ *,+ tenant_id: uuid.UUID,+ events: list[dict],+ connector_type: str | None,+ override_title: str | None,+ override_description: str | None,+ override_severity: str | None,+ override_tags: list[str] | None,+) -> Alert:+ """Build a single ``Alert`` row that represents the submitted event batch.++ The shape of incoming events is intentionally loose — they may be raw+ OCSF, Okta system-log entries, or any dict with the usual suspects+ (``displayMessage``, ``severity``, ``actor``, ``client``, ``published``).+ The function extracts what it can and falls back to safe defaults.+ """+ first = events[0] if events else {}++ # Title+ title = override_title or first.get("displayMessage") or first.get("eventType") or first.get("name") or "AiSOC submitted alert"++ # Severity: pick the highest in the batch unless overridden+ severities = [_normalize_severity(e.get("severity")) for e in events]+ severity = _normalize_severity(override_severity) if override_severity else (_max_severity(severities) if severities else "medium")+ priority = _SEVERITY_PRIORITY[severity]++ # Description: count + first event hint+ description = override_description or (f"Submitted via aisoc submit / API. {len(events)} event(s) in batch. First event: {title}.")++ # Affected entities+ affected_ips: list[str] = []+ affected_hosts: list[str] = []+ affected_users: list[str] = []+ earliest: datetime | None = None+ latest: datetime | None = None++ for event in events:+ actor = event.get("actor") or {}+ client = event.get("client") or {}++ # User: collect candidate identifiers in priority order; skip the+ # actor entirely if *any* of them already appears in+ # ``affected_users`` (i.e. we've already recorded this person under+ # a different identifier in an earlier event).+ actor_identifiers = [+ actor[key] for key in ("alternateId", "displayName", "email", "name", "id") if isinstance(actor.get(key), str) and actor[key]+ ]+ if actor_identifiers and not any(ident in affected_users for ident in actor_identifiers):+ affected_users.append(actor_identifiers[0])++ # IP from client+ ip = client.get("ipAddress") or client.get("ip")+ if isinstance(ip, str) and ip and ip not in affected_ips:+ affected_ips.append(ip)++ # Host from client geographicalContext or client.id+ geo = client.get("geographicalContext") or {}+ host_hint = client.get("device") or client.get("id") or geo.get("city")+ if isinstance(host_hint, str) and host_hint and host_hint not in affected_hosts:+ affected_hosts.append(host_hint)++ # Timestamps+ ts = _coerce_published(event)+ if ts is not None:+ if earliest is None or ts < earliest:+ earliest = ts+ if latest is None or ts > latest:+ latest = ts++ now = datetime.now(UTC)+ event_time = latest or earliest or now+ first_seen = earliest or now+ last_seen = latest or now++ # Tags: combine override + a marker so demo alerts are recognisable+ tags = list(override_tags or [])+ if "submitted" not in tags:+ tags.append("submitted")+ if connector_type and connector_type not in tags:+ tags.append(connector_type)++ return Alert(+ tenant_id=tenant_id,+ title=title,+ description=description,+ severity=severity,+ status="new",+ priority=priority,+ category="identity" if "okta" in (connector_type or "").lower() else None,+ connector_type=connector_type,+ # JSONB list columns: populate explicitly so the response model+ # validation never trips on ``None`` for fresh inserts (the+ # ``default=list`` column default only fires after a real flush+ # against Postgres, not in unit tests with a mocked session).+ mitre_tactics=[],+ mitre_techniques=[],+ ai_recommendations=[],+ affected_ips=affected_ips,+ affected_hosts=affected_hosts,+ affected_users=affected_users,+ tags=tags,+ raw_event={+ "events": events,+ "connector_type": connector_type,+ "source": "aisoc-submit-api",+ },+ event_time=event_time,+ first_seen=first_seen,+ last_seen=last_seen,+ created_at=now,+ updated_at=now,+ )+++ "/submit",+ response_model=AlertResponse,+ status_code=status.HTTP_201_CREATED,+)+async def submit_alert(+ payload: AlertSubmitRequest,+ current_user: Annotated[AuthUser, Depends(require_permission("alerts:write"))],+ db: TenantDBSession,+) -> AlertResponse:+ """Submit one alert directly from an OCSF event batch.++ This is the destination for `aisoc submit <file>` and the local-dev fast+ path documented in the quickstart. We deliberately bypass the Kafka+ detect/correlate/fuse pipeline (which fresh clones don't run by default)+ so that a hand-crafted fixture lands in the database — and therefore in+ ``GET /api/v1/alerts`` and the web console — within the same second.++ Authorisation: requires ``alerts:write``. In development mode the+ auth bypass in :mod:`app.api.v1.deps` resolves to the deterministic+ demo tenant, so the documented ``aisoc submit`` CLI works against a+ fresh clone with no token plumbing.+ """+ if not payload.events:+ raise HTTPException(+ status_code=status.HTTP_400_BAD_REQUEST,+ detail="events must be a non-empty list",+ )++ alert = _synthesise_alert_from_events(+ tenant_id=current_user.tenant_id,+ events=payload.events,+ connector_type=payload.connector_type,+ override_title=payload.title,+ override_description=payload.description,+ override_severity=payload.severity,+ override_tags=payload.tags,+ )++ db.add(alert)+ await db.flush()+ await db.refresh(alert)++ logger.info(+ "alert.submitted",+ extra={+ "alert_id": str(alert.id),+ "tenant_id": str(alert.tenant_id),+ "severity": alert.severity,+ "events": len(payload.events),+ "connector_type": payload.connector_type,+ },+ )++ return AlertResponse.model_validate(alert)++@router.get("", response_model=AlertListResponse)async def list_alerts(current_user: Annotated[AuthUser, Depends(require_permission("alerts:read"))],diff --git a/services/api/migrations/005_compliance.sql b/services/api/migrations/005_compliance.sqlindex adb6af67f..74023736a 100644--- a/services/api/migrations/005_compliance.sql+++ b/services/api/migrations/005_compliance.sql@@ -33,8 +33,19 @@ CREATE INDEX IF NOT EXISTS idx_compliance_evidence_control ON compliance_evidenc-- RLS for evidenceALTER TABLE compliance_evidence ENABLE ROW LEVEL SECURITY;-CREATE POLICY compliance_evidence_tenant_isolation ON compliance_evidence- USING (tenant_id = current_setting('app.current_tenant_id', true)::uuid);+DO $$+BEGIN+ IF NOT EXISTS (+ SELECT 1+ FROM pg_policies+ WHERE schemaname = current_schema()+ AND tablename = 'compliance_evidence'+ AND policyname = 'compliance_evidence_tenant_isolation'+ ) THEN+ EXECUTE 'CREATE POLICY compliance_evidence_tenant_isolation ON compliance_evidence+ USING (tenant_id = current_setting(''app.current_tenant_id'', true)::uuid)';+ END IF;+END $$;-- Seed SOC 2 Trust Services Criteria (CC controls)INSERT INTO compliance_controls (framework, control_id, category, title, description) VALUESdiff --git a/services/api/migrations/025_connectors_click_and_connect.sql b/services/api/migrations/025_connectors_click_and_connect.sqlindex 1aa65b4ef..8a2b92313 100644--- a/services/api/migrations/025_connectors_click_and_connect.sql+++ b/services/api/migrations/025_connectors_click_and_connect.sql@@ -4,11 +4,29 @@BEGIN;--- Rename columns that changed semantics-ALTER TABLE connectors RENAME COLUMN config TO connector_config;-ALTER TABLE connectors RENAME COLUMN credentials_enc TO auth_config;-ALTER TABLE connectors RENAME COLUMN status TO health_status;-ALTER TABLE connectors RENAME COLUMN last_sync_at TO last_sync;+-- Rename columns that changed semantics. Each rename is wrapped in a DO block+-- so the migration is idempotent — safe to re-run on a partially-migrated DB+-- (e.g. one where a previous run of 025 succeeded but the migration ledger+-- entry was lost, which happened during the v7.3.0 smoke test).+DO $$+BEGIN+ IF EXISTS (SELECT 1 FROM information_schema.columns+ WHERE table_name = 'connectors' AND column_name = 'config') THEN+ EXECUTE 'ALTER TABLE connectors RENAME COLUMN config TO connector_config';+ END IF;+ IF EXISTS (SELECT 1 FROM information_schema.columns+ WHERE table_name = 'connectors' AND column_name = 'credentials_enc') THEN+ EXECUTE 'ALTER TABLE connectors RENAME COLUMN credentials_enc TO auth_config';+ END IF;+ IF EXISTS (SELECT 1 FROM information_schema.columns+ WHERE table_name = 'connectors' AND column_name = 'status') THEN+ EXECUTE 'ALTER TABLE connectors RENAME COLUMN status TO health_status';+ END IF;+ IF EXISTS (SELECT 1 FROM information_schema.columns+ WHERE table_name = 'connectors' AND column_name = 'last_sync_at') THEN+ EXECUTE 'ALTER TABLE connectors RENAME COLUMN last_sync_at TO last_sync';+ END IF;+END $$;-- Resize the health_status column to match the ORM (VARCHAR 20)ALTER TABLE connectors ALTER COLUMN health_status TYPE VARCHAR(20);diff --git a/services/api/migrations/042_alerts_schema_drift_fix.sql b/services/api/migrations/042_alerts_schema_drift_fix.sqlnew file mode 100644index 000000000..e5daf4aea--- /dev/null+++ b/services/api/migrations/042_alerts_schema_drift_fix.sql@@ -0,0 +1,39 @@+-- 042_alerts_schema_drift_fix.sql+--+-- Bring the `alerts` table in line with the ORM model defined in+-- services/api/app/models/alert.py.+--+-- The model has accumulated columns over time (entity denormalization,+-- assignment, merge/parent-child relationships, enrichment, tags) that+-- were declared in code but never added to the schema via a migration.+-- That gap surfaced during the v7.3.0 founder-flow smoke test: a fresh+-- `aisoc serve` + `aisoc db upgrade` brings the stack up cleanly, but+-- `GET /api/v1/alerts` returns 500 because asyncpg can't find+-- `alerts.affected_ips`, `alerts.tags`, etc.+--+-- This migration is fully idempotent (all ADD COLUMN clauses use+-- `IF NOT EXISTS`) so it's safe to re-run on partially-migrated+-- environments.++BEGIN;++-- Entity denormalization (model: affected_ips/hosts/users/assets)+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS affected_ips JSONB NOT NULL DEFAULT '[]'::jsonb;+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS affected_hosts JSONB NOT NULL DEFAULT '[]'::jsonb;+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS affected_users JSONB NOT NULL DEFAULT '[]'::jsonb;+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS affected_assets JSONB NOT NULL DEFAULT '[]'::jsonb;++-- Merge / parent-child relationships+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS parent_alert_id UUID;+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS child_alert_ids JSONB NOT NULL DEFAULT '[]'::jsonb;+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS is_merged BOOLEAN NOT NULL DEFAULT FALSE;++-- Assignment+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS assigned_to_id UUID;+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS assigned_at TIMESTAMPTZ;++-- Enrichment / tagging+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS enrichment_data JSONB NOT NULL DEFAULT '{}'::jsonb;+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS tags JSONB NOT NULL DEFAULT '[]'::jsonb;++COMMIT;diff --git a/services/api/tests/api/v1/endpoints/test_alerts_submit.py b/services/api/tests/api/v1/endpoints/test_alerts_submit.pynew file mode 100644index 000000000..549cf55eb--- /dev/null+++ b/services/api/tests/api/v1/endpoints/test_alerts_submit.py@@ -0,0 +1,600 @@+"""Tests for ``POST /alerts/submit`` — the founder-flow direct-write path.++The submit endpoint is the destination the ``aisoc submit <file>`` CLI POSTs+to. It deliberately bypasses the Kafka detect/correlate/fuse pipeline+(which fresh clones don't run by default) and synthesises a single ``Alert``+row directly from an OCSF/Okta-shaped event batch. This is what makes the+quickstart's "alert in seconds" promise true on a clean machine.++These tests pin three layers:++1. **Synthesise helper** (``_synthesise_alert_from_events``) — given a batch+ of events, produces an ``Alert`` ORM instance with sane fields:+ title, severity (highest in the batch unless overridden), priority,+ affected entities, timestamps, tags, raw_event payload.++2. **Severity normalisation** — vendor-native strings (``INFO``, ``HIGH``,+ ``CRITICAL``, etc.) collapse onto the canonical five-tier ladder+ (``info | low | medium | high | critical``) defined in AGENTS.md.++3. **HTTP endpoint** (``submit_alert``) — by direct function invocation+ with a mock ``AsyncSession`` and a synthesised ``CurrentUser``. Same+ pattern as ``test_alerts_detail_envelope`` / ``test_alert_explain``.+ Covers: empty body → 400, happy path → 201 + AlertResponse, tenant+ scoping (alert.tenant_id == current_user.tenant_id).++If these tests break, the founder-flow demo silently regresses and the+quickstart video desyncs from reality.++AiSOC — open-source AI Security Operations Center (MIT License)+Author: Beenu Arora <[email protected]>+"""++from __future__ import annotations++import uuid+from datetime import UTC, datetime, timedelta+from typing import Any+from unittest.mock import AsyncMock, MagicMock++import pytest+from app.api.v1.deps import CurrentUser+from app.api.v1.endpoints.alerts import (+ AlertResponse,+ AlertSubmitRequest,+ _coerce_published,+ _max_severity,+ _normalize_severity,+ _synthesise_alert_from_events,+ submit_alert,+)+from fastapi import HTTPException++# ─── helpers ────────────────────────────────────────────────────────────────+++def _user(tenant_id: uuid.UUID | None = None, *, role: str = "admin") -> CurrentUser:+ """Synthesise a ``CurrentUser`` — defaults to admin so ``alerts:write`` passes."""+ return CurrentUser(+ user_id=uuid.uuid4(),+ tenant_id=tenant_id or uuid.uuid4(),+ role=role,+ email="[email protected]",+ )+++def _okta_event(+ *,+ actor_id: str = "[email protected]",+ ip: str = "203.0.113.42",+ severity: str = "INFO",+ event_type: str = "user.session.start",+ display_message: str = "User login to Okta",+ published: str | None = "2026-05-14T09:30:14.123Z",+) -> dict[str, Any]:+ """Build an Okta system-log shaped event dict.++ Mirrors the lateral-movement fixture so tests stay aligned with the+ demo payload the quickstart video uses.+ """+ return {+ "uuid": str(uuid.uuid4()),+ "published": published,+ "eventType": event_type,+ "displayMessage": display_message,+ "severity": severity,+ "actor": {+ "id": actor_id,+ "type": "User",+ "alternateId": actor_id,+ "displayName": actor_id.split("@")[0].title(),+ },+ "client": {+ "ipAddress": ip,+ "userAgent": {"rawUserAgent": "Mozilla/5.0", "os": "Mac OS X", "browser": "CHROME"},+ "geographicalContext": {"city": "Boston", "country": "United States"},+ },+ "outcome": {"result": "SUCCESS"},+ }+++def _mock_db_session() -> MagicMock:+ """Mock ``AsyncSession`` for endpoint invocation.++ ``submit_alert`` calls ``db.add(alert)`` (sync), ``await db.flush()``+ (no-op), and ``await db.refresh(alert)`` (no-op). The endpoint then+ ``model_validate``s the alert directly — Pydantic reads attributes+ off the row that ``db.add`` left intact.++ Real SQLAlchemy applies the ``default=uuid.uuid4`` column default+ inside ``flush()``. Our mock has to mimic that, otherwise the final+ ``AlertResponse.model_validate(alert)`` blows up on ``id=None``.+ We capture the alert handed to ``add`` and populate the server-side+ defaults on flush.+ """+ added: list[Any] = []++ def _add(obj: Any) -> None:+ added.append(obj)++ async def _flush() -> None:+ for obj in added:+ if getattr(obj, "id", None) is None:+ obj.id = uuid.uuid4()++ session = MagicMock()+ session.add = MagicMock(side_effect=_add)+ session.flush = AsyncMock(side_effect=_flush)+ session.refresh = AsyncMock(return_value=None)+ # Stash for tests that want to inspect what got persisted.+ session._added = added # type: ignore[attr-defined]+ return session+++# ════════════════════════════════════════════════════════════════════════════+# Section 1: severity normalisation (canonical 5-tier ladder)+# ════════════════════════════════════════════════════════════════════════════+++class TestSeverityNormalisation:+ """Vendor-native severity strings must collapse onto info/low/medium/high/critical.++ The canonical ladder is documented in AGENTS.md and pinned by the+ fusion service's ``ConfidenceScorer``. Any new vendor mapping should+ land here first so the regression is caught at PR review.+ """++ @pytest.mark.parametrize(+ "raw,expected",+ [+ ("INFO", "info"),+ ("informational", "info"),+ ("Debug", "info"),+ ("low", "low"),+ ("MINOR", "low"),+ ("warning", "low"),+ ("medium", "medium"),+ ("MODERATE", "medium"),+ ("warn", "medium"),+ ("high", "high"),+ ("Error", "high"),+ ("MAJOR", "high"),+ ("critical", "critical"),+ ("CRIT", "critical"),+ ("Severe", "critical"),+ ("FATAL", "critical"),+ ("emergency", "critical"),+ ],+ )+ def test_known_vendor_strings_map_to_canonical_tier(self, raw: str, expected: str) -> None:+ assert _normalize_severity(raw) == expected++ @pytest.mark.parametrize("raw", [None, "", " "])+ def test_missing_severity_defaults_to_medium(self, raw: Any) -> None:+ """Missing severity is treated as ``medium`` — never raises.++ Connectors can drop severity on partial events; we default+ conservatively so the alert still surfaces to an analyst.+ """+ assert _normalize_severity(raw) == "medium"++ def test_unknown_severity_defaults_to_medium(self) -> None:+ """Unknown vendor strings fall back to ``medium``, not crash."""+ assert _normalize_severity("definitely-not-a-real-tier") == "medium"++ def test_whitespace_and_case_are_normalised(self) -> None:+ """`` HIGH `` and ``high`` are the same thing."""+ assert _normalize_severity(" HIGH ") == "high"+++class TestMaxSeverity:+ """Picking the highest severity in a batch governs the synthesised priority."""++ def test_empty_list_is_treated_as_info(self) -> None:+ # The helper is only called with non-empty batches, but defend+ # against accidental refactors that pipe empty lists in.+ assert _max_severity([]) == "info"++ def test_picks_highest_tier(self) -> None:+ assert _max_severity(["info", "high", "low"]) == "high"+ assert _max_severity(["medium", "critical", "low"]) == "critical"+ assert _max_severity(["info", "info"]) == "info"++ def test_ordering_is_info_low_medium_high_critical(self) -> None:+ """The ladder is exactly five tiers and strictly ordered."""+ assert _max_severity(["info", "low"]) == "low"+ assert _max_severity(["low", "medium"]) == "medium"+ assert _max_severity(["medium", "high"]) == "high"+ assert _max_severity(["high", "critical"]) == "critical"+++# ════════════════════════════════════════════════════════════════════════════+# Section 2: timestamp coercion+# ════════════════════════════════════════════════════════════════════════════+++class TestCoercePublished:+ """``_coerce_published`` accepts Okta ``published``, OCSF ``time``, et al."""++ def test_parses_okta_published_iso_string(self) -> None:+ result = _coerce_published({"published": "2026-05-14T09:30:14.123Z"})+ assert result is not None+ assert result.tzinfo is not None+ assert result.year == 2026+ assert result.month == 5+ assert result.day == 14++ def test_parses_ocsf_time_field(self) -> None:+ result = _coerce_published({"time": "2026-05-14T09:30:00+00:00"})+ assert result is not None+ assert result.hour == 9++ def test_parses_at_timestamp_field(self) -> None:+ result = _coerce_published({"@timestamp": "2026-05-14T10:00:00Z"})+ assert result is not None+ assert result.hour == 10++ def test_returns_none_when_no_timestamp(self) -> None:+ assert _coerce_published({"unrelated": "field"}) is None++ def test_returns_none_on_unparseable_string(self) -> None:+ """A garbage timestamp must not crash the endpoint."""+ assert _coerce_published({"published": "not-a-date"}) is None++ def test_accepts_datetime_objects_directly(self) -> None:+ """If a connector passes a real datetime, we use it."""+ naive = datetime(2026, 5, 14, 9, 30, 14)+ result = _coerce_published({"published": naive})+ assert result is not None+ # Naive datetimes get UTC pinned on so downstream comparisons work.+ assert result.tzinfo is not None+++# ════════════════════════════════════════════════════════════════════════════+# Section 3: _synthesise_alert_from_events — the core helper+# ════════════════════════════════════════════════════════════════════════════+++class TestSynthesiseAlertFromEvents:+ """Given an event batch, build a coherent ``Alert`` row."""++ def test_synthesises_basic_alert_from_single_event(self) -> None:+ tenant = uuid.uuid4()+ alert = _synthesise_alert_from_events(+ tenant_id=tenant,+ events=[_okta_event()],+ connector_type="okta_system_log",+ override_title=None,+ override_description=None,+ override_severity=None,+ override_tags=None,+ )++ assert alert.tenant_id == tenant+ assert alert.title == "User login to Okta"+ assert alert.severity == "info"+ # priority is keyed off the severity ladder+ assert alert.priority == 10+ assert alert.status == "new"+ assert alert.affected_users == ["[email protected]"]+ assert alert.affected_ips == ["203.0.113.42"]+ assert alert.connector_type == "okta_system_log"+ # The okta connector tags onto identity category.+ assert alert.category == "identity"+ assert "submitted" in alert.tags+ assert "okta_system_log" in alert.tags+ # Raw event payload preserved for forensics.+ assert alert.raw_event["events"] == [_okta_event_serialised_match(alert)]+ assert alert.raw_event["source"] == "aisoc-submit-api"++ def test_picks_highest_severity_across_batch(self) -> None:+ """Multi-event batches collapse to one alert at the worst severity."""+ events = [+ _okta_event(severity="INFO"),+ _okta_event(severity="HIGH"),+ _okta_event(severity="LOW"),+ ]+ alert = _synthesise_alert_from_events(+ tenant_id=uuid.uuid4(),+ events=events,+ connector_type="okta_system_log",+ override_title=None,+ override_description=None,+ override_severity=None,+ override_tags=None,+ )+ assert alert.severity == "high"+ assert alert.priority == 75++ def test_overrides_beat_inferred_fields(self) -> None:+ """Explicit overrides win over event-derived inference."""+ alert = _synthesise_alert_from_events(+ tenant_id=uuid.uuid4(),+ events=[_okta_event(severity="INFO")],+ connector_type="okta_system_log",+ override_title="Manual alert title",+ override_description="Manual description",+ override_severity="critical",+ override_tags=["red-team", "exercise"],+ )+ assert alert.title == "Manual alert title"+ assert alert.description == "Manual description"+ assert alert.severity == "critical"+ assert alert.priority == 95+ assert "red-team" in alert.tags+ assert "exercise" in alert.tags+ # The submitted marker still attaches even with custom tags.+ assert "submitted" in alert.tags++ def test_collects_unique_affected_entities_from_batch(self) -> None:+ events = [+ _okta_event(actor_id="[email protected]", ip="203.0.113.42"),+ _okta_event(actor_id="[email protected]", ip="203.0.113.99"),+ # duplicate alice — must not appear twice+ _okta_event(actor_id="[email protected]", ip="203.0.113.42"),+ ]+ alert = _synthesise_alert_from_events(+ tenant_id=uuid.uuid4(),+ events=events,+ connector_type="okta_system_log",+ override_title=None,+ override_description=None,+ override_severity=None,+ override_tags=None,+ )+ # Order preserved, no duplicates.+ assert alert.affected_users == ["[email protected]", "[email protected]"]+ assert alert.affected_ips == ["203.0.113.42", "203.0.113.99"]++ def test_timestamps_pin_to_earliest_and_latest_event(self) -> None:+ """``first_seen`` / ``last_seen`` are the batch envelope."""+ events = [+ _okta_event(published="2026-05-14T09:30:14Z"),+ _okta_event(published="2026-05-14T09:45:00Z"),+ _okta_event(published="2026-05-14T09:35:00Z"),+ ]+ alert = _synthesise_alert_from_events(+ tenant_id=uuid.uuid4(),+ events=events,+ connector_type="okta_system_log",+ override_title=None,+ override_description=None,+ override_severity=None,+ override_tags=None,+ )+ # Earliest event becomes first_seen+ assert alert.first_seen.minute == 30+ # Latest becomes last_seen and event_time+ assert alert.last_seen.minute == 45+ assert alert.event_time.minute == 45++ def test_falls_back_to_now_when_no_timestamps(self) -> None:+ """Events without timestamps still produce a valid alert."""+ events = [{"eventType": "user.session.start", "displayMessage": "No-ts event"}]+ before = datetime.now(UTC)+ alert = _synthesise_alert_from_events(+ tenant_id=uuid.uuid4(),+ events=events,+ connector_type="okta_system_log",+ override_title=None,+ override_description=None,+ override_severity=None,+ override_tags=None,+ )+ after = datetime.now(UTC) + timedelta(seconds=1)+ assert before <= alert.first_seen <= after+ assert before <= alert.last_seen <= after+ assert before <= alert.event_time <= after++ def test_falls_back_to_event_type_when_no_display_message(self) -> None:+ """The title cascades through displayMessage → eventType → name."""+ events = [{"eventType": "user.session.failed", "severity": "HIGH"}]+ alert = _synthesise_alert_from_events(+ tenant_id=uuid.uuid4(),+ events=events,+ connector_type="okta_system_log",+ override_title=None,+ override_description=None,+ override_severity=None,+ override_tags=None,+ )+ assert alert.title == "user.session.failed"++ def test_falls_back_to_default_title_when_event_is_bare(self) -> None:+ """The most degraded payload still produces a titled alert."""+ alert = _synthesise_alert_from_events(+ tenant_id=uuid.uuid4(),+ events=[{}],+ connector_type=None,+ override_title=None,+ override_description=None,+ override_severity=None,+ override_tags=None,+ )+ assert alert.title == "AiSOC submitted alert"+ assert alert.severity == "medium"+ # No connector_type → not an identity event+ assert alert.category is None++ def test_non_okta_connector_does_not_force_identity_category(self) -> None:+ """Category inference is connector-aware (okta → identity)."""+ alert = _synthesise_alert_from_events(+ tenant_id=uuid.uuid4(),+ events=[_okta_event()],+ connector_type="splunk_enterprise",+ override_title=None,+ override_description=None,+ override_severity=None,+ override_tags=None,+ )+ assert alert.category is None+ assert "splunk_enterprise" in alert.tags++ def test_description_summarises_batch_size(self) -> None:+ """Auto-generated description mentions the batch size."""+ events = [_okta_event(), _okta_event(), _okta_event()]+ alert = _synthesise_alert_from_events(+ tenant_id=uuid.uuid4(),+ events=events,+ connector_type="okta_system_log",+ override_title=None,+ override_description=None,+ override_severity=None,+ override_tags=None,+ )+ assert "3 event(s)" in (alert.description or "")+ assert "Submitted via aisoc submit" in (alert.description or "")+++def _okta_event_serialised_match(alert: Any) -> dict[str, Any]:+ """Helper to read the first event back off the alert's raw_event."""+ return alert.raw_event["events"][0]+++# ════════════════════════════════════════════════════════════════════════════+# Section 4: submit_alert HTTP endpoint+# ════════════════════════════════════════════════════════════════════════════+++class TestSubmitAlertEndpoint:+ """Direct-invocation tests for ``submit_alert``."""++ @pytest.mark.asyncio+ async def test_happy_path_creates_alert_and_returns_response(self) -> None:+ tenant = uuid.uuid4()+ user = _user(tenant)+ session = _mock_db_session()++ payload = AlertSubmitRequest(+ events=[_okta_event(severity="HIGH")],+ connector_id="aisoc-cli-submit",+ connector_type="okta_system_log",+ source_format="json",+ )++ resp = await submit_alert(payload, user, session)++ assert isinstance(resp, AlertResponse)+ assert resp.tenant_id == tenant+ assert resp.severity == "high"+ assert resp.title == "User login to Okta"+ # Persistence sequence: add(), flush(), refresh()+ session.add.assert_called_once()+ session.flush.assert_awaited_once()+ session.refresh.assert_awaited_once()++ @pytest.mark.asyncio+ async def test_empty_events_returns_400(self) -> None:+ """The endpoint must reject empty batches with a 400, not a 500."""+ user = _user()+ session = _mock_db_session()+ payload = AlertSubmitRequest(events=[], connector_type="okta_system_log")++ with pytest.raises(HTTPException) as exc:+ await submit_alert(payload, user, session)++ assert exc.value.status_code == 400+ assert "non-empty" in exc.value.detail+ # No DB writes happened.+ session.add.assert_not_called()+ session.flush.assert_not_awaited()++ @pytest.mark.asyncio+ async def test_alert_is_pinned_to_current_user_tenant(self) -> None:+ """The synthesised alert MUST inherit the caller's tenant.++ This is the only tenant-isolation guarantee on the submit path —+ callers can't smuggle a different tenant_id through the payload.+ """+ tenant = uuid.uuid4()+ user = _user(tenant)+ session = _mock_db_session()++ await submit_alert(+ AlertSubmitRequest(events=[_okta_event()], connector_type="okta_system_log"),+ user,+ session,+ )++ added_alerts = session._added+ assert len(added_alerts) == 1+ assert added_alerts[0].tenant_id == tenant++ @pytest.mark.asyncio+ async def test_override_severity_lands_on_persisted_alert(self) -> None:+ """``severity`` override on the payload wins over event severity."""+ user = _user()+ session = _mock_db_session()++ await submit_alert(+ AlertSubmitRequest(+ events=[_okta_event(severity="INFO")],+ connector_type="okta_system_log",+ severity="critical",+ ),+ user,+ session,+ )+ added = session._added+ assert added[0].severity == "critical"+ assert added[0].priority == 95++ @pytest.mark.asyncio+ async def test_payload_tags_are_attached(self) -> None:+ user = _user()+ session = _mock_db_session()++ await submit_alert(+ AlertSubmitRequest(+ events=[_okta_event()],+ connector_type="okta_system_log",+ tags=["red-team", "lateral-movement"],+ ),+ user,+ session,+ )+ added = session._added+ tags = set(added[0].tags)+ assert "red-team" in tags+ assert "lateral-movement" in tags+ # submitted marker still added+ assert "submitted" in tags++ @pytest.mark.asyncio+ async def test_multi_event_batch_creates_single_alert(self) -> None:+ """Batches collapse to one alert, not N — that's the design promise."""+ user = _user()+ session = _mock_db_session()++ await submit_alert(+ AlertSubmitRequest(+ events=[+ _okta_event(actor_id="[email protected]"),+ _okta_event(actor_id="[email protected]"),+ ],+ connector_type="okta_system_log",+ ),+ user,+ session,+ )++ added = session._added+ assert len(added) == 1+ # Both users land on the same alert.+ assert "[email protected]" in added[0].affected_users+ assert "[email protected]" in added[0].affected_users++ @pytest.mark.asyncio+ async def test_payload_validates_optional_fields(self) -> None:+ """``AlertSubmitRequest`` is forgiving — empty optionals are fine."""+ # The CLI omits some fields when the fixture is minimal.+ req = AlertSubmitRequest(events=[_okta_event()])+ assert req.connector_id is None+ assert req.connector_type is None+ assert req.source_format is None+ assert req.tags is None+ # And the endpoint accepts it.+ user = _user()+ session = _mock_db_session()+ resp = await submit_alert(req, user, session)+ assert resp.severity == "info"