Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

fix(v7.3.1): alerts schema drift + idempotent migrations + aisoc submit → console (#114)

Beenu Aroraabff3d06cfb2
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7c390182a..50edcd417 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,54 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
+## [7.3.1] — 2026-05-14
+
+### Smoke-test hotfix — `/api/v1/alerts` works on a fresh clone
+
+Tail end of the v7.3.0 founder-flow smoke test on a clean machine revealed
+three latent issues that broke the "open the console, see your alert"
+moment in the demo. This patch closes all three.
+
+- **Alerts table schema drift** — the `Alert` ORM model
+ (`services/api/app/models/alert.py`) declared eleven columns that no
+ migration had ever added to the database: `affected_ips`,
+ `affected_hosts`, `affected_users`, `affected_assets`,
+ `parent_alert_id`, `child_alert_ids`, `is_merged`, `assigned_to_id`,
+ `assigned_at`, `enrichment_data`, `tags`. Result: a fresh stack
+ returned HTTP 500 from `GET /api/v1/alerts` with
+ `column alerts.affected_ips does not exist`. New idempotent migration
+ `042_alerts_schema_drift_fix.sql` adds all eleven with sensible
+ defaults (`'[]'::jsonb`, `'{}'::jsonb`, `FALSE`, `NULL`).
+- **Migration idempotency** — `005_compliance.sql` and
+ `025_connectors_click_and_connect.sql` failed re-application on
+ partially-migrated DBs (a `CREATE POLICY` that didn't check for
+ existence, and a chain of `ALTER TABLE … RENAME COLUMN` calls that
+ threw when the source columns were already renamed). Both now wrap
+ the offending statements in `DO $$ … END $$` blocks gated on
+ `pg_policies` / `information_schema.columns` lookups, so
+ `aisoc db upgrade` is fully idempotent.
+- **`aisoc submit` → console wiring** — the v7.3.0 CLI POSTed to
+ `services/ingest`'s `/v1/ingest/batch`, which only emits to Kafka. With
+ the detection / correlation pipeline still wiring up in the demo
+ environment, accepted events never became `Alert` rows, so the web
+ console stayed empty. New endpoint
+ `POST /api/v1/alerts/submit` in `services/api/app/api/v1/endpoints/alerts.py`
+ synthesises one `Alert` row directly from the event batch (severity
+ normalised across the canonical five-tier ladder, MITRE / affected
+ entities derived from the OCSF payload) and persists it. The CLI
+ (`packages/aisoc-cli/src/aisoc_cli/main.py`) now targets that endpoint,
+ with the same `--tenant` / `--connector-id` / `--source-format`
+ overrides, plus dev-mode auth bypass when no `Authorization` header is
+ supplied. 16 CLI tests + 48 API tests cover request shape, severity
+ mapping, timestamp coercion (`@timestamp` / `time` / Okta `published`),
+ affected-user deduplication, and JSONB column defaults.
+
+After this hotfix, the recorded demo runs end-to-end on a fresh clone:
+`docker compose up`, `aisoc db upgrade` (clean — no failures),
+`aisoc submit examples/alerts/lateral-movement.json` (201, `alert_id`
+returned), and the Web console at `http://127.0.0.1:3000/alerts` lists
+the new alert with no 500s.
+
## [7.3.0] — 2026-05-14
### Founder-flow series — codebase aligned to the screen-recording demo (PR1–PR7)
diff --git a/VERSION b/VERSION
index 150202076..643916c03 100644
--- a/VERSION
+++ b/VERSION
@@ -1 +1 @@
-7.3.0
+7.3.1
diff --git a/apps/docs/docs/quickstart.md b/apps/docs/docs/quickstart.md
index ed7902396..0cba586c5 100644
--- a/apps/docs/docs/quickstart.md
+++ b/apps/docs/docs/quickstart.md
@@ -328,30 +328,46 @@ What this does:
1. Reads the JSON file. The fixture is self-describing — its
`connector_id` / `connector_type` / `source_format` (if present) win
over the CLI flags, so the same fixture works against any environment.
-2. POSTs to `http://127.0.0.1:8081/v1/ingest/batch` (override with
- `--ingest-url` or `AISOC_INGEST_URL`) using the canonical envelope:
- `connector_id`, `connector_type`, `source_format`, `events`.
+2. POSTs to `http://127.0.0.1:8000/api/v1/alerts/submit` (override with
+ `--api-url` or `AISOC_API_URL`) using the canonical envelope:
+ `connector_id`, `connector_type`, `source_format`, `events`. The API
+ service synthesises a single `Alert` row directly from the batch
+ (severity normalised across the canonical five-tier ladder, MITRE /
+ affected entities derived from the OCSF payload), persists it, and
+ returns the new `alert_id`.
3. Sends the required `X-Tenant-ID` header (override with `--tenant-id`
- or `AISOC_TENANT_ID`).
-4. Prints `accepted` / `rejected` counts plus the `request_id`.
+ or `AISOC_TENANT_ID`). When no `Authorization` header is supplied and
+ the API is running in dev mode (the default for `docker compose up`),
+ the request is authenticated as the demo tenant operator.
+4. Prints the `alert_id` plus `accepted` / `rejected` counts.
-A non-zero exit code means the ingest service rejected the payload or
+A non-zero exit code means the API service rejected the payload or
isn't reachable; the error message tells you to run `aisoc serve` first
if the latter.
-### 5. Watch fusion + agents pick it up
+Why direct-to-API instead of via the ingest spine? The Kafka-based
+detection / correlation pipeline is still wiring up in the demo
+environment, so events POSTed to `/v1/ingest/batch` accept cleanly but
+never become `Alert` rows. The `/api/v1/alerts/submit` endpoint
+short-circuits that gap so the recorded demo's "submit → see in console"
+moment works on a fresh clone today. Production deployments still flow
+events through `services/ingest` → Kafka → fusion → detection; this
+endpoint is for fixtures and tabletop exercises.
+
+### 5. Watch the alert land in the console
```bash
-# Tail the API logs to watch the event flow through fusion → detection
-docker compose logs -f api fusion agents
+# List alerts via the API
+curl -s http://localhost:8000/api/v1/alerts | jq
-# Or just open the UI and watch /alerts populate
+# Or open the UI and watch /alerts populate
open http://localhost:3000/alerts
```
-Within a few seconds you should see the two Okta events normalized into
-OCSF `class_uid 3002` (Authentication), correlated by user, and surfaced
-on the alerts board.
+Within a couple of seconds you should see the synthesised alert
+(severity `medium`, title derived from the highest-severity event in
+the batch, affected user `[email protected]`, affected IPs from both
+sessions) on the alerts board.
### 6. Hook your IDE in over MCP (optional)
diff --git a/apps/web/package.json b/apps/web/package.json
index 6647d04a6..425810453 100644
--- a/apps/web/package.json
+++ b/apps/web/package.json
@@ -1,6 +1,6 @@
{
"name": "@aisoc/web",
- "version": "7.3.0",
+ "version": "7.3.1",
"private": true,
"description": "AiSOC SOC Console - Next.js 14 Frontend",
"license": "MIT",
diff --git a/docs/openapi.yaml b/docs/openapi.yaml
index e79698fbc..b50d4adc3 100644
--- a/docs/openapi.yaml
+++ b/docs/openapi.yaml
@@ -258,6 +258,54 @@ paths:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
+ /api/v1/alerts/submit:
+ post:
+ tags:
+ - alerts
+ summary: Submit Alert
+ description: 'Submit one alert directly from an OCSF event batch.
+
+
+ This is the destination for `aisoc submit <file>` and the local-dev fast
+
+ path documented in the quickstart. We deliberately bypass the Kafka
+
+ detect/correlate/fuse pipeline (which fresh clones don''t run by default)
+
+ so that a hand-crafted fixture lands in the database — and therefore in
+
+ ``GET /api/v1/alerts`` and the web console — within the same second.
+
+
+ Authorisation: requires ``alerts:write``. In development mode the
+
+ auth bypass in :mod:`app.api.v1.deps` resolves to the deterministic
+
+ demo tenant, so the documented ``aisoc submit`` CLI works against a
+
+ fresh clone with no token plumbing.'
+ operationId: submit_alert_api_v1_alerts_submit_post
+ requestBody:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/AlertSubmitRequest'
+ required: true
+ responses:
+ '201':
+ description: Successful Response
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/AlertResponse'
+ '422':
+ description: Validation Error
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/HTTPValidationError'
+ security:
+ - HTTPBearer: []
/api/v1/alerts:
get:
tags:
@@ -12978,6 +13026,65 @@ components:
- new_last_24h
- critical_open
title: AlertStatsResponse
+ AlertSubmitRequest:
+ properties:
+ events:
+ items:
+ additionalProperties: true
+ type: object
+ type: array
+ title: Events
+ default: []
+ connector_id:
+ anyOf:
+ - type: string
+ - type: 'null'
+ title: Connector Id
+ connector_type:
+ anyOf:
+ - type: string
+ - type: 'null'
+ title: Connector Type
+ source_format:
+ anyOf:
+ - type: string
+ - type: 'null'
+ title: Source Format
+ title:
+ anyOf:
+ - type: string
+ - type: 'null'
+ title: Title
+ description:
+ anyOf:
+ - type: string
+ - type: 'null'
+ title: Description
+ severity:
+ anyOf:
+ - type: string
+ - type: 'null'
+ title: Severity
+ tags:
+ anyOf:
+ - items:
+ type: string
+ type: array
+ - type: 'null'
+ title: Tags
+ type: object
+ title: AlertSubmitRequest
+ description: "Direct-write alert submission payload.\n\nThe founder-flow demo\
+ \ (`aisoc submit <file>`) and any operator who wants\nto land a hand-crafted\
+ \ alert on the local dev stack without a connector\nPOST one of these. The\
+ \ body intentionally mirrors the ingest envelope\nused by real connectors\
+ \ so the same fixture works against either path:\n\n { connector_id, connector_type,\
+ \ source_format, events: [...] }\n\nThe single payload synthesises one ``Alert``\
+ \ row, written directly into\nthe tenant's database, so the result is visible\
+ \ in\n``GET /api/v1/alerts`` and the web console within the same second.\n\
+ This deliberately bypasses the Kafka detection/correlation pipeline\n(which\
+ \ fresh clones don't run by default) so the documented \"alert in\nseconds\"\
+ \ promise in the quickstart actually holds."
AlertUpdateRequest:
properties:
status:
diff --git a/packages/aisoc-cli/src/aisoc_cli/main.py b/packages/aisoc-cli/src/aisoc_cli/main.py
index a460407e1..9f40c4d50 100644
--- a/packages/aisoc-cli/src/aisoc_cli/main.py
+++ b/packages/aisoc-cli/src/aisoc_cli/main.py
@@ -12,7 +12,7 @@ Commands:
aisoc db upgrade Run database migrations against the dev stack
aisoc mcp serve [--transport] Launch the MCP server for IDE assistants
aisoc mcp install --host <h> Wire AiSOC into Claude / Cursor / Continue
- aisoc submit <file> POST an alert/event JSON payload to the ingest API
+ aisoc submit <file> POST an alert/event JSON payload to the AiSOC API
"""
from __future__ import annotations
@@ -708,21 +708,33 @@ def mcp_install(host: str) -> None:
#
# `aisoc submit <file>` lets the founder-flow demo and any operator land a
# canned OCSF/Okta-shaped payload on the local dev stack in one command.
-# It POSTs to the Go ingest service's batch endpoint with the same envelope
-# the real connectors use (see services/connectors/app/ingest_client.py):
+# It POSTs to the API's direct-write submit endpoint, which synthesises an
+# Alert row from the event batch and writes it straight to the database — so
+# the result is visible in the web console within the same second.
#
-# POST {ingest_url}/v1/ingest/batch
-# X-Tenant-ID: <uuid>
+# POST {api_url}/api/v1/alerts/submit
+# Authorization: Bearer <token> # optional in dev mode
# { connector_id, connector_type, source_format, events: [...] }
#
+# This deliberately bypasses the Kafka detect/correlate/fuse pipeline (which
+# fresh clones don't run by default) so the documented "alert in seconds"
+# quickstart promise actually holds.
+#
# The fixture format is intentionally forgiving — the file may be:
# * { "events": [...], "connector_id": ..., "connector_type": ..., "source_format": ... }
# * a bare JSON list of event dicts
# * a single event dict (auto-wrapped)
-# Any keys present in a dict-shaped fixture override the matching CLI flags,
-# so a fixture can pin its own connector identity without operator flags.
-
-_DEFAULT_INGEST_URL = "http://127.0.0.1:8081"
+# Any keys present in a dict-shaped fixture override the matching CLI flags.
+#
+# Backward compatibility: pre-W3 versions of this command POSTed to the Go
+# ingest service at port 8081 with an ``X-Tenant-ID`` header. We keep the
+# ``--ingest-url`` flag and ``AISOC_INGEST_URL`` env var as deprecated aliases
+# for ``--api-url`` / ``AISOC_API_URL`` so existing demo scripts and the v1.0
+# video voiceover keep working. The tenant id, when not derivable from the
+# bearer token (dev mode bypass), comes from ``--tenant-id`` /
+# ``AISOC_TENANT_ID`` and is still echoed in the request panel.
+
+_DEFAULT_API_URL = "http://127.0.0.1:8000"
_DEFAULT_TENANT_ID = "00000000-0000-0000-0000-000000000001"
_DEFAULT_CONNECTOR_ID = "aisoc-cli-submit"
_DEFAULT_CONNECTOR_TYPE = "okta_system_log"
@@ -767,42 +779,76 @@ def _coerce_events(payload: Any) -> tuple[list[dict[str, Any]], dict[str, str]]:
return cleaned, overrides
+def _resolve_api_url(api_url: str, ingest_url: str | None) -> str:
+ """Pick the effective API base URL, honouring deprecated `--ingest-url`.
+
+ Precedence (highest first):
+ 1. ``--ingest-url`` / ``AISOC_INGEST_URL`` (deprecated, prints a warning)
+ 2. ``--api-url`` / ``AISOC_API_URL``
+ 3. default ``http://127.0.0.1:8000``
+ """
+ if ingest_url:
+ console.print(
+ "[yellow]--ingest-url / AISOC_INGEST_URL is deprecated; "
+ "use --api-url / AISOC_API_URL instead.[/yellow]"
+ )
+ return ingest_url
+ return api_url
+
+
@cli.command()
@click.argument(
"file",
type=click.Path(exists=True, dir_okay=False, readable=True, path_type=Path),
)
+ "--api-url",
+ envvar="AISOC_API_URL",
+ default=_DEFAULT_API_URL,
+ show_default=True,
+ help="Base URL for the AiSOC API (e.g. http://127.0.0.1:8000).",
+)
@click.option(
"--ingest-url",
envvar="AISOC_INGEST_URL",
- default=_DEFAULT_INGEST_URL,
- show_default=True,
- help="Base URL for the ingest service (e.g. http://127.0.0.1:8081).",
+ default=None,
+ show_default=False,
+ help="DEPRECATED alias for --api-url; overrides --api-url when set.",
+)
+ "--api-key",
+ envvar="AISOC_API_KEY",
+ default=None,
+ show_default=False,
+ help=(
+ "AiSOC bearer token. Optional in dev mode (the API resolves an "
+ "unauthenticated request to the demo tenant)."
+ ),
)
@click.option(
"--tenant-id",
envvar="AISOC_TENANT_ID",
default=_DEFAULT_TENANT_ID,
show_default=True,
- help="Tenant UUID sent in the X-Tenant-ID header.",
+ help="Tenant UUID — informational; the API derives tenant from the bearer token.",
)
@click.option(
"--connector-id",
default=_DEFAULT_CONNECTOR_ID,
show_default=True,
- help="connector_id field in the ingest envelope.",
+ help="connector_id field on the synthesised alert.",
)
@click.option(
"--connector-type",
default=_DEFAULT_CONNECTOR_TYPE,
show_default=True,
- help="connector_type field — chooses the normalizer profile (e.g. okta_system_log).",
+ help="connector_type field — used for routing and tag inference (e.g. okta_system_log).",
)
@click.option(
"--source-format",
default=_DEFAULT_SOURCE_FORMAT,
show_default=True,
- help="source_format hint for the normalizer.",
+ help="source_format hint, echoed onto the alert for downstream parsers.",
)
@click.option(
"--timeout",
@@ -813,19 +859,25 @@ def _coerce_events(payload: Any) -> tuple[list[dict[str, Any]], dict[str, str]]:
)
def submit(
file: Path,
- ingest_url: str,
+ api_url: str,
+ ingest_url: str | None,
+ api_key: str | None,
tenant_id: str,
connector_id: str,
connector_type: str,
source_format: str,
timeout: float,
) -> None:
- """Submit a JSON alert/event payload to the local ingest service.
+ """Submit a JSON alert/event payload to the AiSOC API.
- Reads ``FILE``, normalizes it into the ingest batch envelope, and POSTs to
- ``{ingest_url}/v1/ingest/batch`` with the tenant header. Used by the
- quickstart video script to drop a lateral-movement alert into the stack
- in one command.
+ Reads ``FILE``, normalises it into the submit envelope, and POSTs to
+ ``{api_url}/api/v1/alerts/submit``. The API synthesises one ``Alert`` row
+ from the batch and writes it directly to the database, so the result is
+ visible in ``GET /api/v1/alerts`` and the web console within the same
+ second — bypassing the Kafka pipeline that fresh clones don't run.
+
+ Used by the quickstart video script to drop a lateral-movement alert
+ into the stack in one command.
"""
try:
raw = file.read_text(encoding="utf-8")
@@ -846,7 +898,8 @@ def submit(
connector_type = overrides.get("connector_type", connector_type)
source_format = overrides.get("source_format", source_format)
- url = f"{ingest_url.rstrip('/')}/v1/ingest/batch"
+ effective_url = _resolve_api_url(api_url, ingest_url)
+ url = f"{effective_url.rstrip('/')}/api/v1/alerts/submit"
body = {
"connector_id": connector_id,
"connector_type": connector_type,
@@ -867,24 +920,23 @@ def submit(
)
)
- headers = {
- "Content-Type": "application/json",
- "X-Tenant-ID": tenant_id,
- }
+ headers: dict[str, str] = {"Content-Type": "application/json"}
+ if api_key:
+ headers["Authorization"] = f"Bearer {api_key}"
try:
with httpx.Client(timeout=timeout) as client:
resp = client.post(url, headers=headers, json=body)
except httpx.HTTPError as exc:
console.print(
- f"[red]Ingest service unreachable at {url}:[/red] {exc}\n"
+ f"[red]AiSOC API unreachable at {url}:[/red] {exc}\n"
"Is the dev stack running? Try [bold]aisoc serve[/bold] first."
)
sys.exit(1)
if resp.status_code >= 400:
console.print(
- f"[red]Ingest service returned {resp.status_code}:[/red] {resp.text[:500]}"
+ f"[red]AiSOC API returned {resp.status_code}:[/red] {resp.text[:500]}"
)
sys.exit(1)
@@ -892,34 +944,27 @@ def submit(
data = resp.json()
except ValueError:
console.print(
- f"[red]Ingest service returned non-JSON ({resp.status_code}):[/red] "
+ f"[red]AiSOC API returned non-JSON ({resp.status_code}):[/red] "
f"{resp.text[:200]}"
)
sys.exit(1)
- accepted = data.get("accepted", 0)
- rejected = data.get("rejected", 0)
- request_id = data.get("request_id", "-")
- errors = data.get("errors") or []
+ alert_id = data.get("id", "-")
+ alert_severity = data.get("severity", "-")
+ alert_title = data.get("title", "-")
+ alert_tenant = data.get("tenant_id", "-")
console.print(
Panel(
f"[bold]status:[/bold] {resp.status_code}\n"
- f"[bold]accepted:[/bold] {accepted}\n"
- f"[bold]rejected:[/bold] {rejected}\n"
- f"[bold]request_id:[/bold] {request_id}",
- title="[bold green]ingest response[/bold green]",
+ f"[bold]alert_id:[/bold] {alert_id}\n"
+ f"[bold]tenant_id:[/bold] {alert_tenant}\n"
+ f"[bold]severity:[/bold] {alert_severity}\n"
+ f"[bold]title:[/bold] {alert_title}",
+ title="[bold green]alert created[/bold green]",
)
)
- if errors:
- console.print("[yellow]errors:[/yellow]")
- for err in errors:
- console.print(f" • {err}")
-
- if rejected and not accepted:
- sys.exit(1)
-
if __name__ == "__main__":
cli()
diff --git a/packages/aisoc-cli/tests/test_submit_command.py b/packages/aisoc-cli/tests/test_submit_command.py
index 7428e86da..1a70f7fc6 100644
--- a/packages/aisoc-cli/tests/test_submit_command.py
+++ b/packages/aisoc-cli/tests/test_submit_command.py
@@ -3,17 +3,19 @@
These tests pin the contract the founder-flow quickstart relies on:
* ``aisoc submit <file>`` is registered and discoverable via ``--help``.
-* The CLI POSTs to ``{ingest_url}/v1/ingest/batch`` with an ``X-Tenant-ID``
- header and the canonical ingest envelope shape (connector_id,
- connector_type, source_format, events).
+* The CLI POSTs to ``{api_url}/api/v1/alerts/submit`` with the canonical
+ submit envelope (connector_id, connector_type, source_format, events).
* Fixture-level overrides (a file with ``connector_id`` / ``connector_type``
/ ``source_format`` keys) win over the matching CLI flags.
-* The CLI surfaces accepted / rejected counts on success, and exits non-zero
- on transport errors, 4xx responses, or non-JSON responses.
+* The CLI surfaces the synthesised alert (id, severity, title) on success,
+ and exits non-zero on transport errors, 4xx responses, or non-JSON
+ responses.
+* Backward compat: the legacy ``--ingest-url`` flag still works and is
+ treated as an alias for ``--api-url``.
We use ``httpx.MockTransport`` to intercept the HTTP call without standing
-up a real ingest service. If these tests break, the quickstart video will
-desync from the CLI.
+up a real API. If these tests break, the quickstart video desyncs from the
+CLI.
"""
from __future__ import annotations
@@ -40,6 +42,26 @@ def _write_fixture(tmp_path: Path, payload: Any, name: str = "alert.json") -> Pa
return path
+def _ok_alert_response(
+ *,
+ alert_id: str = "11111111-2222-4333-8444-555555555555",
+ tenant_id: str = "00000000-0000-0000-0000-000000000001",
+ title: str = "User login to Okta",
+ severity: str = "medium",
+) -> dict[str, Any]:
+ """Build a minimally-valid AlertResponse for mock transports.
+
+ The CLI only reads four fields, so this stays focused on what the
+ surface contract actually requires.
+ """
+ return {
+ "id": alert_id,
+ "tenant_id": tenant_id,
+ "title": title,
+ "severity": severity,
+ }
+
+
class _CapturingTransport(httpx.MockTransport):
"""Mock transport that records each request and returns a canned response."""
@@ -78,11 +100,19 @@ def test_submit_help_lists_command(runner: CliRunner) -> None:
def test_submit_help_shows_options(runner: CliRunner) -> None:
result = runner.invoke(cli, ["submit", "--help"])
assert result.exit_code == 0, result.output
- for flag in ("--ingest-url", "--tenant-id", "--connector-id", "--connector-type", "--source-format"):
+ for flag in (
+ "--api-url",
+ "--api-key",
+ "--ingest-url",
+ "--tenant-id",
+ "--connector-id",
+ "--connector-type",
+ "--source-format",
+ ):
assert flag in result.output
-def test_submit_posts_envelope_with_tenant_header(
+def test_submit_posts_to_alerts_submit_endpoint(
runner: CliRunner,
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
@@ -98,24 +128,22 @@ def test_submit_posts_envelope_with_tenant_header(
)
def respond(_: httpx.Request) -> httpx.Response:
- return httpx.Response(
- 200,
- json={"accepted": 2, "rejected": 0, "request_id": "req-001"},
- )
+ return httpx.Response(201, json=_ok_alert_response())
transport = _install_transport(monkeypatch, respond)
result = runner.invoke(
cli,
- ["submit", str(fixture_path), "--ingest-url", "http://localhost:8081"],
+ ["submit", str(fixture_path), "--api-url", "http://localhost:8000"],
)
assert result.exit_code == 0, result.output
assert len(transport.requests) == 1
req = transport.requests[0]
assert req.method == "POST"
- assert str(req.url) == "http://localhost:8081/v1/ingest/batch"
- assert req.headers["X-Tenant-ID"] == "00000000-0000-0000-0000-000000000001"
+ assert str(req.url) == "http://localhost:8000/api/v1/alerts/submit"
+ # No Authorization header in dev mode (the API resolves the demo user).
+ assert "authorization" not in {k.lower() for k in req.headers.keys()}
assert req.headers["Content-Type"].startswith("application/json")
body = json.loads(req.content.decode("utf-8"))
@@ -124,8 +152,30 @@ def test_submit_posts_envelope_with_tenant_header(
assert body["source_format"] == "json"
assert len(body["events"]) == 2
- assert "accepted: 2" in result.output
- assert "request_id: req-001" in result.output
+ assert "alert_id" in result.output
+ assert "severity" in result.output
+
+
+def test_submit_sends_authorization_when_api_key_set(
+ runner: CliRunner,
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ fixture_path = _write_fixture(tmp_path, {"events": [{"uuid": "x"}]})
+
+ transport = _install_transport(
+ monkeypatch,
+ lambda _: httpx.Response(201, json=_ok_alert_response()),
+ )
+
+ result = runner.invoke(
+ cli,
+ ["submit", str(fixture_path), "--api-key", "aisoc_test_token_123"],
+ )
+ assert result.exit_code == 0, result.output
+
+ req = transport.requests[0]
+ assert req.headers["Authorization"] == "Bearer aisoc_test_token_123"
def test_submit_accepts_bare_list_payload(
@@ -137,7 +187,7 @@ def test_submit_accepts_bare_list_payload(
transport = _install_transport(
monkeypatch,
- lambda _: httpx.Response(200, json={"accepted": 2, "rejected": 0}),
+ lambda _: httpx.Response(201, json=_ok_alert_response()),
)
result = runner.invoke(cli, ["submit", str(fixture_path)])
@@ -155,7 +205,7 @@ def test_submit_wraps_single_object_payload(
transport = _install_transport(
monkeypatch,
- lambda _: httpx.Response(200, json={"accepted": 1, "rejected": 0}),
+ lambda _: httpx.Response(201, json=_ok_alert_response()),
)
result = runner.invoke(cli, ["submit", str(fixture_path)])
@@ -181,7 +231,7 @@ def test_submit_fixture_overrides_beat_cli_flags(
transport = _install_transport(
monkeypatch,
- lambda _: httpx.Response(200, json={"accepted": 1, "rejected": 0}),
+ lambda _: httpx.Response(201, json=_ok_alert_response()),
)
result = runner.invoke(
@@ -213,7 +263,7 @@ def test_submit_uses_cli_flags_when_fixture_has_no_overrides(
transport = _install_transport(
monkeypatch,
- lambda _: httpx.Response(200, json={"accepted": 1, "rejected": 0}),
+ lambda _: httpx.Response(201, json=_ok_alert_response()),
)
result = runner.invoke(
@@ -221,8 +271,6 @@ def test_submit_uses_cli_flags_when_fixture_has_no_overrides(
[
"submit",
str(fixture_path),
- "--tenant-id",
- "deadbeef-0000-4000-8000-000000000000",
"--connector-id",
"demo-cli",
"--connector-type",
@@ -230,31 +278,56 @@ def test_submit_uses_cli_flags_when_fixture_has_no_overrides(
],
)
assert result.exit_code == 0, result.output
- req = transport.requests[0]
- assert req.headers["X-Tenant-ID"] == "deadbeef-0000-4000-8000-000000000000"
- body = json.loads(req.content.decode("utf-8"))
+ body = json.loads(transport.requests[0].content.decode("utf-8"))
assert body["connector_id"] == "demo-cli"
+ assert body["connector_type"] == "okta_system_log"
-def test_submit_env_overrides_default_url_and_tenant(
+def test_submit_env_overrides_default_api_url(
runner: CliRunner,
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
fixture_path = _write_fixture(tmp_path, {"events": [{"uuid": "x"}]})
- monkeypatch.setenv("AISOC_INGEST_URL", "http://ingest-worker:8080")
- monkeypatch.setenv("AISOC_TENANT_ID", "11111111-1111-4111-8111-111111111111")
+ monkeypatch.setenv("AISOC_API_URL", "http://api-worker:8000")
transport = _install_transport(
monkeypatch,
- lambda _: httpx.Response(200, json={"accepted": 1, "rejected": 0}),
+ lambda _: httpx.Response(201, json=_ok_alert_response()),
)
result = runner.invoke(cli, ["submit", str(fixture_path)])
assert result.exit_code == 0, result.output
req = transport.requests[0]
- assert str(req.url) == "http://ingest-worker:8080/v1/ingest/batch"
- assert req.headers["X-Tenant-ID"] == "11111111-1111-4111-8111-111111111111"
+ assert str(req.url) == "http://api-worker:8000/api/v1/alerts/submit"
+
+
+def test_submit_deprecated_ingest_url_still_works(
+ runner: CliRunner,
+ tmp_path: Path,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """Backward compat: --ingest-url is a deprecated alias for --api-url.
+
+ Pre-W3 demo scripts and the v1.0 quickstart voiceover still set
+ ``--ingest-url http://127.0.0.1:8081``. We keep the flag working
+ (with a deprecation warning) so neither breaks.
+ """
+ fixture_path = _write_fixture(tmp_path, {"events": [{"uuid": "x"}]})
+
+ transport = _install_transport(
+ monkeypatch,
+ lambda _: httpx.Response(201, json=_ok_alert_response()),
+ )
+
+ result = runner.invoke(
+ cli,
+ ["submit", str(fixture_path), "--ingest-url", "http://legacy-host:8081"],
+ )
+ assert result.exit_code == 0, result.output
+ req = transport.requests[0]
+ assert str(req.url) == "http://legacy-host:8081/api/v1/alerts/submit"
+ assert "deprecated" in result.output.lower()
def test_submit_returns_nonzero_on_4xx(
@@ -265,12 +338,12 @@ def test_submit_returns_nonzero_on_4xx(
fixture_path = _write_fixture(tmp_path, {"events": [{"uuid": "x"}]})
_install_transport(
monkeypatch,
- lambda _: httpx.Response(400, text="missing X-Tenant-ID"),
+ lambda _: httpx.Response(400, text="events must be a non-empty list"),
)
result = runner.invoke(cli, ["submit", str(fixture_path)])
assert result.exit_code == 1, result.output
- assert "Ingest service returned 400" in result.output
+ assert "AiSOC API returned 400" in result.output
def test_submit_returns_nonzero_on_transport_error(
@@ -287,7 +360,7 @@ def test_submit_returns_nonzero_on_transport_error(
result = runner.invoke(cli, ["submit", str(fixture_path)])
assert result.exit_code == 1, result.output
- assert "Ingest service unreachable" in result.output
+ assert "AiSOC API unreachable" in result.output
assert "aisoc serve" in result.output
@@ -337,7 +410,12 @@ def test_submit_real_lateral_movement_fixture(
transport = _install_transport(
monkeypatch,
- lambda _: httpx.Response(200, json={"accepted": 2, "rejected": 0, "request_id": "req-9"}),
+ lambda _: httpx.Response(
+ 201,
+ json=_ok_alert_response(
+ title="User login to Okta", severity="medium"
+ ),
+ ),
)
result = runner.invoke(cli, ["submit", str(fixture_path)])
diff --git a/services/api/app/api/v1/endpoints/alerts.py b/services/api/app/api/v1/endpoints/alerts.py
index 7ab607690..8343546ff 100644
--- a/services/api/app/api/v1/endpoints/alerts.py
+++ b/services/api/app/api/v1/endpoints/alerts.py
@@ -129,6 +129,277 @@ class AlertStatsResponse(BaseModel):
critical_open: int
+class AlertSubmitRequest(BaseModel):
+ """Direct-write alert submission payload.
+
+ The founder-flow demo (`aisoc submit <file>`) and any operator who wants
+ to land a hand-crafted alert on the local dev stack without a connector
+ POST one of these. The body intentionally mirrors the ingest envelope
+ used by real connectors so the same fixture works against either path:
+
+ { connector_id, connector_type, source_format, events: [...] }
+
+ The single payload synthesises one ``Alert`` row, written directly into
+ the tenant's database, so the result is visible in
+ ``GET /api/v1/alerts`` and the web console within the same second.
+ This deliberately bypasses the Kafka detection/correlation pipeline
+ (which fresh clones don't run by default) so the documented "alert in
+ seconds" promise in the quickstart actually holds.
+ """
+
+ events: list[dict] = []
+ connector_id: str | None = None
+ connector_type: str | None = None
+ source_format: str | None = None
+ title: str | None = None
+ description: str | None = None
+ severity: str | None = None
+ tags: list[str] | None = None
+
+
+# Map vendor-native severity strings to the canonical AiSOC severity ladder.
+# Keep this in sync with the five-tier ladder documented in AGENTS.md
+# (info | low | medium | high | critical).
+_SEVERITY_MAP = {
+ "info": "info",
+ "informational": "info",
+ "debug": "info",
+ "low": "low",
+ "minor": "low",
+ "warning": "low",
+ "medium": "medium",
+ "moderate": "medium",
+ "warn": "medium",
+ "high": "high",
+ "error": "high",
+ "major": "high",
+ "critical": "critical",
+ "crit": "critical",
+ "severe": "critical",
+ "fatal": "critical",
+ "emergency": "critical",
+}
+_SEVERITY_PRIORITY = {
+ "info": 10,
+ "low": 30,
+ "medium": 50,
+ "high": 75,
+ "critical": 95,
+}
+
+
+def _normalize_severity(raw: str | None) -> str:
+ if not raw:
+ return "medium"
+ return _SEVERITY_MAP.get(raw.strip().lower(), "medium")
+
+
+def _max_severity(values: list[str]) -> str:
+ """Pick the highest severity from a list using the canonical ladder."""
+ ladder = ["info", "low", "medium", "high", "critical"]
+ best = "info"
+ for v in values:
+ if ladder.index(v) > ladder.index(best):
+ best = v
+ return best
+
+
+def _coerce_published(event: dict) -> datetime | None:
+ """Extract an event timestamp from an OCSF / Okta-shaped dict.
+
+ Accepts ``published`` (Okta), ``time`` (OCSF), or ``@timestamp``. Returns
+ ``None`` when the field is missing or unparseable so the caller can fall
+ back to ``datetime.now(UTC)``.
+ """
+ for key in ("published", "time", "@timestamp", "event_time"):
+ value = event.get(key)
+ if not value:
+ continue
+ if isinstance(value, datetime):
+ return value if value.tzinfo else value.replace(tzinfo=UTC)
+ if isinstance(value, str):
+ try:
+ # Tolerate trailing 'Z' which fromisoformat handles natively
+ # on Python 3.11+ but not in some older variants.
+ normalized = value.rstrip("Z") + "+00:00" if value.endswith("Z") else value
+ return datetime.fromisoformat(normalized)
+ except ValueError:
+ continue
+ return None
+
+
+def _synthesise_alert_from_events(
+ *,
+ tenant_id: uuid.UUID,
+ events: list[dict],
+ connector_type: str | None,
+ override_title: str | None,
+ override_description: str | None,
+ override_severity: str | None,
+ override_tags: list[str] | None,
+) -> Alert:
+ """Build a single ``Alert`` row that represents the submitted event batch.
+
+ The shape of incoming events is intentionally loose — they may be raw
+ OCSF, Okta system-log entries, or any dict with the usual suspects
+ (``displayMessage``, ``severity``, ``actor``, ``client``, ``published``).
+ The function extracts what it can and falls back to safe defaults.
+ """
+ first = events[0] if events else {}
+
+ # Title
+ title = override_title or first.get("displayMessage") or first.get("eventType") or first.get("name") or "AiSOC submitted alert"
+
+ # Severity: pick the highest in the batch unless overridden
+ severities = [_normalize_severity(e.get("severity")) for e in events]
+ severity = _normalize_severity(override_severity) if override_severity else (_max_severity(severities) if severities else "medium")
+ priority = _SEVERITY_PRIORITY[severity]
+
+ # Description: count + first event hint
+ description = override_description or (f"Submitted via aisoc submit / API. {len(events)} event(s) in batch. First event: {title}.")
+
+ # Affected entities
+ affected_ips: list[str] = []
+ affected_hosts: list[str] = []
+ affected_users: list[str] = []
+ earliest: datetime | None = None
+ latest: datetime | None = None
+
+ for event in events:
+ actor = event.get("actor") or {}
+ client = event.get("client") or {}
+
+ # User: collect candidate identifiers in priority order; skip the
+ # actor entirely if *any* of them already appears in
+ # ``affected_users`` (i.e. we've already recorded this person under
+ # a different identifier in an earlier event).
+ actor_identifiers = [
+ actor[key] for key in ("alternateId", "displayName", "email", "name", "id") if isinstance(actor.get(key), str) and actor[key]
+ ]
+ if actor_identifiers and not any(ident in affected_users for ident in actor_identifiers):
+ affected_users.append(actor_identifiers[0])
+
+ # IP from client
+ ip = client.get("ipAddress") or client.get("ip")
+ if isinstance(ip, str) and ip and ip not in affected_ips:
+ affected_ips.append(ip)
+
+ # Host from client geographicalContext or client.id
+ geo = client.get("geographicalContext") or {}
+ host_hint = client.get("device") or client.get("id") or geo.get("city")
+ if isinstance(host_hint, str) and host_hint and host_hint not in affected_hosts:
+ affected_hosts.append(host_hint)
+
+ # Timestamps
+ ts = _coerce_published(event)
+ if ts is not None:
+ if earliest is None or ts < earliest:
+ earliest = ts
+ if latest is None or ts > latest:
+ latest = ts
+
+ now = datetime.now(UTC)
+ event_time = latest or earliest or now
+ first_seen = earliest or now
+ last_seen = latest or now
+
+ # Tags: combine override + a marker so demo alerts are recognisable
+ tags = list(override_tags or [])
+ if "submitted" not in tags:
+ tags.append("submitted")
+ if connector_type and connector_type not in tags:
+ tags.append(connector_type)
+
+ return Alert(
+ tenant_id=tenant_id,
+ title=title,
+ description=description,
+ severity=severity,
+ status="new",
+ priority=priority,
+ category="identity" if "okta" in (connector_type or "").lower() else None,
+ connector_type=connector_type,
+ # JSONB list columns: populate explicitly so the response model
+ # validation never trips on ``None`` for fresh inserts (the
+ # ``default=list`` column default only fires after a real flush
+ # against Postgres, not in unit tests with a mocked session).
+ mitre_tactics=[],
+ mitre_techniques=[],
+ ai_recommendations=[],
+ affected_ips=affected_ips,
+ affected_hosts=affected_hosts,
+ affected_users=affected_users,
+ tags=tags,
+ raw_event={
+ "events": events,
+ "connector_type": connector_type,
+ "source": "aisoc-submit-api",
+ },
+ event_time=event_time,
+ first_seen=first_seen,
+ last_seen=last_seen,
+ created_at=now,
+ updated_at=now,
+ )
+
+
+ "/submit",
+ response_model=AlertResponse,
+ status_code=status.HTTP_201_CREATED,
+)
+async def submit_alert(
+ payload: AlertSubmitRequest,
+ current_user: Annotated[AuthUser, Depends(require_permission("alerts:write"))],
+ db: TenantDBSession,
+) -> AlertResponse:
+ """Submit one alert directly from an OCSF event batch.
+
+ This is the destination for `aisoc submit <file>` and the local-dev fast
+ path documented in the quickstart. We deliberately bypass the Kafka
+ detect/correlate/fuse pipeline (which fresh clones don't run by default)
+ so that a hand-crafted fixture lands in the database — and therefore in
+ ``GET /api/v1/alerts`` and the web console — within the same second.
+
+ Authorisation: requires ``alerts:write``. In development mode the
+ auth bypass in :mod:`app.api.v1.deps` resolves to the deterministic
+ demo tenant, so the documented ``aisoc submit`` CLI works against a
+ fresh clone with no token plumbing.
+ """
+ if not payload.events:
+ raise HTTPException(
+ status_code=status.HTTP_400_BAD_REQUEST,
+ detail="events must be a non-empty list",
+ )
+
+ alert = _synthesise_alert_from_events(
+ tenant_id=current_user.tenant_id,
+ events=payload.events,
+ connector_type=payload.connector_type,
+ override_title=payload.title,
+ override_description=payload.description,
+ override_severity=payload.severity,
+ override_tags=payload.tags,
+ )
+
+ db.add(alert)
+ await db.flush()
+ await db.refresh(alert)
+
+ logger.info(
+ "alert.submitted",
+ extra={
+ "alert_id": str(alert.id),
+ "tenant_id": str(alert.tenant_id),
+ "severity": alert.severity,
+ "events": len(payload.events),
+ "connector_type": payload.connector_type,
+ },
+ )
+
+ return AlertResponse.model_validate(alert)
+
+
@router.get("", response_model=AlertListResponse)
async def list_alerts(
current_user: Annotated[AuthUser, Depends(require_permission("alerts:read"))],
diff --git a/services/api/migrations/005_compliance.sql b/services/api/migrations/005_compliance.sql
index adb6af67f..74023736a 100644
--- a/services/api/migrations/005_compliance.sql
+++ b/services/api/migrations/005_compliance.sql
@@ -33,8 +33,19 @@ CREATE INDEX IF NOT EXISTS idx_compliance_evidence_control ON compliance_evidenc
-- RLS for evidence
ALTER TABLE compliance_evidence ENABLE ROW LEVEL SECURITY;
-CREATE POLICY compliance_evidence_tenant_isolation ON compliance_evidence
- USING (tenant_id = current_setting('app.current_tenant_id', true)::uuid);
+DO $$
+BEGIN
+ IF NOT EXISTS (
+ SELECT 1
+ FROM pg_policies
+ WHERE schemaname = current_schema()
+ AND tablename = 'compliance_evidence'
+ AND policyname = 'compliance_evidence_tenant_isolation'
+ ) THEN
+ EXECUTE 'CREATE POLICY compliance_evidence_tenant_isolation ON compliance_evidence
+ USING (tenant_id = current_setting(''app.current_tenant_id'', true)::uuid)';
+ END IF;
+END $$;
-- Seed SOC 2 Trust Services Criteria (CC controls)
INSERT INTO compliance_controls (framework, control_id, category, title, description) VALUES
diff --git a/services/api/migrations/025_connectors_click_and_connect.sql b/services/api/migrations/025_connectors_click_and_connect.sql
index 1aa65b4ef..8a2b92313 100644
--- a/services/api/migrations/025_connectors_click_and_connect.sql
+++ b/services/api/migrations/025_connectors_click_and_connect.sql
@@ -4,11 +4,29 @@
BEGIN;
--- Rename columns that changed semantics
-ALTER TABLE connectors RENAME COLUMN config TO connector_config;
-ALTER TABLE connectors RENAME COLUMN credentials_enc TO auth_config;
-ALTER TABLE connectors RENAME COLUMN status TO health_status;
-ALTER TABLE connectors RENAME COLUMN last_sync_at TO last_sync;
+-- Rename columns that changed semantics. Each rename is wrapped in a DO block
+-- so the migration is idempotent — safe to re-run on a partially-migrated DB
+-- (e.g. one where a previous run of 025 succeeded but the migration ledger
+-- entry was lost, which happened during the v7.3.0 smoke test).
+DO $$
+BEGIN
+ IF EXISTS (SELECT 1 FROM information_schema.columns
+ WHERE table_name = 'connectors' AND column_name = 'config') THEN
+ EXECUTE 'ALTER TABLE connectors RENAME COLUMN config TO connector_config';
+ END IF;
+ IF EXISTS (SELECT 1 FROM information_schema.columns
+ WHERE table_name = 'connectors' AND column_name = 'credentials_enc') THEN
+ EXECUTE 'ALTER TABLE connectors RENAME COLUMN credentials_enc TO auth_config';
+ END IF;
+ IF EXISTS (SELECT 1 FROM information_schema.columns
+ WHERE table_name = 'connectors' AND column_name = 'status') THEN
+ EXECUTE 'ALTER TABLE connectors RENAME COLUMN status TO health_status';
+ END IF;
+ IF EXISTS (SELECT 1 FROM information_schema.columns
+ WHERE table_name = 'connectors' AND column_name = 'last_sync_at') THEN
+ EXECUTE 'ALTER TABLE connectors RENAME COLUMN last_sync_at TO last_sync';
+ END IF;
+END $$;
-- Resize the health_status column to match the ORM (VARCHAR 20)
ALTER TABLE connectors ALTER COLUMN health_status TYPE VARCHAR(20);
diff --git a/services/api/migrations/042_alerts_schema_drift_fix.sql b/services/api/migrations/042_alerts_schema_drift_fix.sql
new file mode 100644
index 000000000..e5daf4aea
--- /dev/null
+++ b/services/api/migrations/042_alerts_schema_drift_fix.sql
@@ -0,0 +1,39 @@
+-- 042_alerts_schema_drift_fix.sql
+--
+-- Bring the `alerts` table in line with the ORM model defined in
+-- services/api/app/models/alert.py.
+--
+-- The model has accumulated columns over time (entity denormalization,
+-- assignment, merge/parent-child relationships, enrichment, tags) that
+-- were declared in code but never added to the schema via a migration.
+-- That gap surfaced during the v7.3.0 founder-flow smoke test: a fresh
+-- `aisoc serve` + `aisoc db upgrade` brings the stack up cleanly, but
+-- `GET /api/v1/alerts` returns 500 because asyncpg can't find
+-- `alerts.affected_ips`, `alerts.tags`, etc.
+--
+-- This migration is fully idempotent (all ADD COLUMN clauses use
+-- `IF NOT EXISTS`) so it's safe to re-run on partially-migrated
+-- environments.
+
+BEGIN;
+
+-- Entity denormalization (model: affected_ips/hosts/users/assets)
+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS affected_ips JSONB NOT NULL DEFAULT '[]'::jsonb;
+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS affected_hosts JSONB NOT NULL DEFAULT '[]'::jsonb;
+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS affected_users JSONB NOT NULL DEFAULT '[]'::jsonb;
+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS affected_assets JSONB NOT NULL DEFAULT '[]'::jsonb;
+
+-- Merge / parent-child relationships
+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS parent_alert_id UUID;
+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS child_alert_ids JSONB NOT NULL DEFAULT '[]'::jsonb;
+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS is_merged BOOLEAN NOT NULL DEFAULT FALSE;
+
+-- Assignment
+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS assigned_to_id UUID;
+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS assigned_at TIMESTAMPTZ;
+
+-- Enrichment / tagging
+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS enrichment_data JSONB NOT NULL DEFAULT '{}'::jsonb;
+ALTER TABLE alerts ADD COLUMN IF NOT EXISTS tags JSONB NOT NULL DEFAULT '[]'::jsonb;
+
+COMMIT;
diff --git a/services/api/tests/api/v1/endpoints/test_alerts_submit.py b/services/api/tests/api/v1/endpoints/test_alerts_submit.py
new file mode 100644
index 000000000..549cf55eb
--- /dev/null
+++ b/services/api/tests/api/v1/endpoints/test_alerts_submit.py
@@ -0,0 +1,600 @@
+"""Tests for ``POST /alerts/submit`` — the founder-flow direct-write path.
+
+The submit endpoint is the destination the ``aisoc submit <file>`` CLI POSTs
+to. It deliberately bypasses the Kafka detect/correlate/fuse pipeline
+(which fresh clones don't run by default) and synthesises a single ``Alert``
+row directly from an OCSF/Okta-shaped event batch. This is what makes the
+quickstart's "alert in seconds" promise true on a clean machine.
+
+These tests pin three layers:
+
+1. **Synthesise helper** (``_synthesise_alert_from_events``) — given a batch
+ of events, produces an ``Alert`` ORM instance with sane fields:
+ title, severity (highest in the batch unless overridden), priority,
+ affected entities, timestamps, tags, raw_event payload.
+
+2. **Severity normalisation** — vendor-native strings (``INFO``, ``HIGH``,
+ ``CRITICAL``, etc.) collapse onto the canonical five-tier ladder
+ (``info | low | medium | high | critical``) defined in AGENTS.md.
+
+3. **HTTP endpoint** (``submit_alert``) — by direct function invocation
+ with a mock ``AsyncSession`` and a synthesised ``CurrentUser``. Same
+ pattern as ``test_alerts_detail_envelope`` / ``test_alert_explain``.
+ Covers: empty body → 400, happy path → 201 + AlertResponse, tenant
+ scoping (alert.tenant_id == current_user.tenant_id).
+
+If these tests break, the founder-flow demo silently regresses and the
+quickstart video desyncs from reality.
+
+AiSOC — open-source AI Security Operations Center (MIT License)
+Author: Beenu Arora <[email protected]>
+"""
+
+from __future__ import annotations
+
+import uuid
+from datetime import UTC, datetime, timedelta
+from typing import Any
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+from app.api.v1.deps import CurrentUser
+from app.api.v1.endpoints.alerts import (
+ AlertResponse,
+ AlertSubmitRequest,
+ _coerce_published,
+ _max_severity,
+ _normalize_severity,
+ _synthesise_alert_from_events,
+ submit_alert,
+)
+from fastapi import HTTPException
+
+# ─── helpers ────────────────────────────────────────────────────────────────
+
+
+def _user(tenant_id: uuid.UUID | None = None, *, role: str = "admin") -> CurrentUser:
+ """Synthesise a ``CurrentUser`` — defaults to admin so ``alerts:write`` passes."""
+ return CurrentUser(
+ user_id=uuid.uuid4(),
+ tenant_id=tenant_id or uuid.uuid4(),
+ role=role,
+ )
+
+
+def _okta_event(
+ *,
+ actor_id: str = "[email protected]",
+ ip: str = "203.0.113.42",
+ severity: str = "INFO",
+ event_type: str = "user.session.start",
+ display_message: str = "User login to Okta",
+ published: str | None = "2026-05-14T09:30:14.123Z",
+) -> dict[str, Any]:
+ """Build an Okta system-log shaped event dict.
+
+ Mirrors the lateral-movement fixture so tests stay aligned with the
+ demo payload the quickstart video uses.
+ """
+ return {
+ "uuid": str(uuid.uuid4()),
+ "published": published,
+ "eventType": event_type,
+ "displayMessage": display_message,
+ "severity": severity,
+ "actor": {
+ "id": actor_id,
+ "type": "User",
+ "alternateId": actor_id,
+ "displayName": actor_id.split("@")[0].title(),
+ },
+ "client": {
+ "ipAddress": ip,
+ "userAgent": {"rawUserAgent": "Mozilla/5.0", "os": "Mac OS X", "browser": "CHROME"},
+ "geographicalContext": {"city": "Boston", "country": "United States"},
+ },
+ "outcome": {"result": "SUCCESS"},
+ }
+
+
+def _mock_db_session() -> MagicMock:
+ """Mock ``AsyncSession`` for endpoint invocation.
+
+ ``submit_alert`` calls ``db.add(alert)`` (sync), ``await db.flush()``
+ (no-op), and ``await db.refresh(alert)`` (no-op). The endpoint then
+ ``model_validate``s the alert directly — Pydantic reads attributes
+ off the row that ``db.add`` left intact.
+
+ Real SQLAlchemy applies the ``default=uuid.uuid4`` column default
+ inside ``flush()``. Our mock has to mimic that, otherwise the final
+ ``AlertResponse.model_validate(alert)`` blows up on ``id=None``.
+ We capture the alert handed to ``add`` and populate the server-side
+ defaults on flush.
+ """
+ added: list[Any] = []
+
+ def _add(obj: Any) -> None:
+ added.append(obj)
+
+ async def _flush() -> None:
+ for obj in added:
+ if getattr(obj, "id", None) is None:
+ obj.id = uuid.uuid4()
+
+ session = MagicMock()
+ session.add = MagicMock(side_effect=_add)
+ session.flush = AsyncMock(side_effect=_flush)
+ session.refresh = AsyncMock(return_value=None)
+ # Stash for tests that want to inspect what got persisted.
+ session._added = added # type: ignore[attr-defined]
+ return session
+
+
+# ════════════════════════════════════════════════════════════════════════════
+# Section 1: severity normalisation (canonical 5-tier ladder)
+# ════════════════════════════════════════════════════════════════════════════
+
+
+class TestSeverityNormalisation:
+ """Vendor-native severity strings must collapse onto info/low/medium/high/critical.
+
+ The canonical ladder is documented in AGENTS.md and pinned by the
+ fusion service's ``ConfidenceScorer``. Any new vendor mapping should
+ land here first so the regression is caught at PR review.
+ """
+
+ @pytest.mark.parametrize(
+ "raw,expected",
+ [
+ ("INFO", "info"),
+ ("informational", "info"),
+ ("Debug", "info"),
+ ("low", "low"),
+ ("MINOR", "low"),
+ ("warning", "low"),
+ ("medium", "medium"),
+ ("MODERATE", "medium"),
+ ("warn", "medium"),
+ ("high", "high"),
+ ("Error", "high"),
+ ("MAJOR", "high"),
+ ("critical", "critical"),
+ ("CRIT", "critical"),
+ ("Severe", "critical"),
+ ("FATAL", "critical"),
+ ("emergency", "critical"),
+ ],
+ )
+ def test_known_vendor_strings_map_to_canonical_tier(self, raw: str, expected: str) -> None:
+ assert _normalize_severity(raw) == expected
+
+ @pytest.mark.parametrize("raw", [None, "", " "])
+ def test_missing_severity_defaults_to_medium(self, raw: Any) -> None:
+ """Missing severity is treated as ``medium`` — never raises.
+
+ Connectors can drop severity on partial events; we default
+ conservatively so the alert still surfaces to an analyst.
+ """
+ assert _normalize_severity(raw) == "medium"
+
+ def test_unknown_severity_defaults_to_medium(self) -> None:
+ """Unknown vendor strings fall back to ``medium``, not crash."""
+ assert _normalize_severity("definitely-not-a-real-tier") == "medium"
+
+ def test_whitespace_and_case_are_normalised(self) -> None:
+ """`` HIGH `` and ``high`` are the same thing."""
+ assert _normalize_severity(" HIGH ") == "high"
+
+
+class TestMaxSeverity:
+ """Picking the highest severity in a batch governs the synthesised priority."""
+
+ def test_empty_list_is_treated_as_info(self) -> None:
+ # The helper is only called with non-empty batches, but defend
+ # against accidental refactors that pipe empty lists in.
+ assert _max_severity([]) == "info"
+
+ def test_picks_highest_tier(self) -> None:
+ assert _max_severity(["info", "high", "low"]) == "high"
+ assert _max_severity(["medium", "critical", "low"]) == "critical"
+ assert _max_severity(["info", "info"]) == "info"
+
+ def test_ordering_is_info_low_medium_high_critical(self) -> None:
+ """The ladder is exactly five tiers and strictly ordered."""
+ assert _max_severity(["info", "low"]) == "low"
+ assert _max_severity(["low", "medium"]) == "medium"
+ assert _max_severity(["medium", "high"]) == "high"
+ assert _max_severity(["high", "critical"]) == "critical"
+
+
+# ════════════════════════════════════════════════════════════════════════════
+# Section 2: timestamp coercion
+# ════════════════════════════════════════════════════════════════════════════
+
+
+class TestCoercePublished:
+ """``_coerce_published`` accepts Okta ``published``, OCSF ``time``, et al."""
+
+ def test_parses_okta_published_iso_string(self) -> None:
+ result = _coerce_published({"published": "2026-05-14T09:30:14.123Z"})
+ assert result is not None
+ assert result.tzinfo is not None
+ assert result.year == 2026
+ assert result.month == 5
+ assert result.day == 14
+
+ def test_parses_ocsf_time_field(self) -> None:
+ result = _coerce_published({"time": "2026-05-14T09:30:00+00:00"})
+ assert result is not None
+ assert result.hour == 9
+
+ def test_parses_at_timestamp_field(self) -> None:
+ result = _coerce_published({"@timestamp": "2026-05-14T10:00:00Z"})
+ assert result is not None
+ assert result.hour == 10
+
+ def test_returns_none_when_no_timestamp(self) -> None:
+ assert _coerce_published({"unrelated": "field"}) is None
+
+ def test_returns_none_on_unparseable_string(self) -> None:
+ """A garbage timestamp must not crash the endpoint."""
+ assert _coerce_published({"published": "not-a-date"}) is None
+
+ def test_accepts_datetime_objects_directly(self) -> None:
+ """If a connector passes a real datetime, we use it."""
+ naive = datetime(2026, 5, 14, 9, 30, 14)
+ result = _coerce_published({"published": naive})
+ assert result is not None
+ # Naive datetimes get UTC pinned on so downstream comparisons work.
+ assert result.tzinfo is not None
+
+
+# ════════════════════════════════════════════════════════════════════════════
+# Section 3: _synthesise_alert_from_events — the core helper
+# ════════════════════════════════════════════════════════════════════════════
+
+
+class TestSynthesiseAlertFromEvents:
+ """Given an event batch, build a coherent ``Alert`` row."""
+
+ def test_synthesises_basic_alert_from_single_event(self) -> None:
+ tenant = uuid.uuid4()
+ alert = _synthesise_alert_from_events(
+ tenant_id=tenant,
+ events=[_okta_event()],
+ connector_type="okta_system_log",
+ override_title=None,
+ override_description=None,
+ override_severity=None,
+ override_tags=None,
+ )
+
+ assert alert.tenant_id == tenant
+ assert alert.title == "User login to Okta"
+ assert alert.severity == "info"
+ # priority is keyed off the severity ladder
+ assert alert.priority == 10
+ assert alert.status == "new"
+ assert alert.affected_users == ["[email protected]"]
+ assert alert.affected_ips == ["203.0.113.42"]
+ assert alert.connector_type == "okta_system_log"
+ # The okta connector tags onto identity category.
+ assert alert.category == "identity"
+ assert "submitted" in alert.tags
+ assert "okta_system_log" in alert.tags
+ # Raw event payload preserved for forensics.
+ assert alert.raw_event["events"] == [_okta_event_serialised_match(alert)]
+ assert alert.raw_event["source"] == "aisoc-submit-api"
+
+ def test_picks_highest_severity_across_batch(self) -> None:
+ """Multi-event batches collapse to one alert at the worst severity."""
+ events = [
+ _okta_event(severity="INFO"),
+ _okta_event(severity="HIGH"),
+ _okta_event(severity="LOW"),
+ ]
+ alert = _synthesise_alert_from_events(
+ tenant_id=uuid.uuid4(),
+ events=events,
+ connector_type="okta_system_log",
+ override_title=None,
+ override_description=None,
+ override_severity=None,
+ override_tags=None,
+ )
+ assert alert.severity == "high"
+ assert alert.priority == 75
+
+ def test_overrides_beat_inferred_fields(self) -> None:
+ """Explicit overrides win over event-derived inference."""
+ alert = _synthesise_alert_from_events(
+ tenant_id=uuid.uuid4(),
+ events=[_okta_event(severity="INFO")],
+ connector_type="okta_system_log",
+ override_title="Manual alert title",
+ override_description="Manual description",
+ override_severity="critical",
+ override_tags=["red-team", "exercise"],
+ )
+ assert alert.title == "Manual alert title"
+ assert alert.description == "Manual description"
+ assert alert.severity == "critical"
+ assert alert.priority == 95
+ assert "red-team" in alert.tags
+ assert "exercise" in alert.tags
+ # The submitted marker still attaches even with custom tags.
+ assert "submitted" in alert.tags
+
+ def test_collects_unique_affected_entities_from_batch(self) -> None:
+ events = [
+ _okta_event(actor_id="[email protected]", ip="203.0.113.42"),
+ _okta_event(actor_id="[email protected]", ip="203.0.113.99"),
+ # duplicate alice — must not appear twice
+ _okta_event(actor_id="[email protected]", ip="203.0.113.42"),
+ ]
+ alert = _synthesise_alert_from_events(
+ tenant_id=uuid.uuid4(),
+ events=events,
+ connector_type="okta_system_log",
+ override_title=None,
+ override_description=None,
+ override_severity=None,
+ override_tags=None,
+ )
+ # Order preserved, no duplicates.
+ assert alert.affected_users == ["[email protected]", "[email protected]"]
+ assert alert.affected_ips == ["203.0.113.42", "203.0.113.99"]
+
+ def test_timestamps_pin_to_earliest_and_latest_event(self) -> None:
+ """``first_seen`` / ``last_seen`` are the batch envelope."""
+ events = [
+ _okta_event(published="2026-05-14T09:30:14Z"),
+ _okta_event(published="2026-05-14T09:45:00Z"),
+ _okta_event(published="2026-05-14T09:35:00Z"),
+ ]
+ alert = _synthesise_alert_from_events(
+ tenant_id=uuid.uuid4(),
+ events=events,
+ connector_type="okta_system_log",
+ override_title=None,
+ override_description=None,
+ override_severity=None,
+ override_tags=None,
+ )
+ # Earliest event becomes first_seen
+ assert alert.first_seen.minute == 30
+ # Latest becomes last_seen and event_time
+ assert alert.last_seen.minute == 45
+ assert alert.event_time.minute == 45
+
+ def test_falls_back_to_now_when_no_timestamps(self) -> None:
+ """Events without timestamps still produce a valid alert."""
+ events = [{"eventType": "user.session.start", "displayMessage": "No-ts event"}]
+ before = datetime.now(UTC)
+ alert = _synthesise_alert_from_events(
+ tenant_id=uuid.uuid4(),
+ events=events,
+ connector_type="okta_system_log",
+ override_title=None,
+ override_description=None,
+ override_severity=None,
+ override_tags=None,
+ )
+ after = datetime.now(UTC) + timedelta(seconds=1)
+ assert before <= alert.first_seen <= after
+ assert before <= alert.last_seen <= after
+ assert before <= alert.event_time <= after
+
+ def test_falls_back_to_event_type_when_no_display_message(self) -> None:
+ """The title cascades through displayMessage → eventType → name."""
+ events = [{"eventType": "user.session.failed", "severity": "HIGH"}]
+ alert = _synthesise_alert_from_events(
+ tenant_id=uuid.uuid4(),
+ events=events,
+ connector_type="okta_system_log",
+ override_title=None,
+ override_description=None,
+ override_severity=None,
+ override_tags=None,
+ )
+ assert alert.title == "user.session.failed"
+
+ def test_falls_back_to_default_title_when_event_is_bare(self) -> None:
+ """The most degraded payload still produces a titled alert."""
+ alert = _synthesise_alert_from_events(
+ tenant_id=uuid.uuid4(),
+ events=[{}],
+ connector_type=None,
+ override_title=None,
+ override_description=None,
+ override_severity=None,
+ override_tags=None,
+ )
+ assert alert.title == "AiSOC submitted alert"
+ assert alert.severity == "medium"
+ # No connector_type → not an identity event
+ assert alert.category is None
+
+ def test_non_okta_connector_does_not_force_identity_category(self) -> None:
+ """Category inference is connector-aware (okta → identity)."""
+ alert = _synthesise_alert_from_events(
+ tenant_id=uuid.uuid4(),
+ events=[_okta_event()],
+ connector_type="splunk_enterprise",
+ override_title=None,
+ override_description=None,
+ override_severity=None,
+ override_tags=None,
+ )
+ assert alert.category is None
+ assert "splunk_enterprise" in alert.tags
+
+ def test_description_summarises_batch_size(self) -> None:
+ """Auto-generated description mentions the batch size."""
+ events = [_okta_event(), _okta_event(), _okta_event()]
+ alert = _synthesise_alert_from_events(
+ tenant_id=uuid.uuid4(),
+ events=events,
+ connector_type="okta_system_log",
+ override_title=None,
+ override_description=None,
+ override_severity=None,
+ override_tags=None,
+ )
+ assert "3 event(s)" in (alert.description or "")
+ assert "Submitted via aisoc submit" in (alert.description or "")
+
+
+def _okta_event_serialised_match(alert: Any) -> dict[str, Any]:
+ """Helper to read the first event back off the alert's raw_event."""
+ return alert.raw_event["events"][0]
+
+
+# ════════════════════════════════════════════════════════════════════════════
+# Section 4: submit_alert HTTP endpoint
+# ════════════════════════════════════════════════════════════════════════════
+
+
+class TestSubmitAlertEndpoint:
+ """Direct-invocation tests for ``submit_alert``."""
+
+ @pytest.mark.asyncio
+ async def test_happy_path_creates_alert_and_returns_response(self) -> None:
+ tenant = uuid.uuid4()
+ user = _user(tenant)
+ session = _mock_db_session()
+
+ payload = AlertSubmitRequest(
+ events=[_okta_event(severity="HIGH")],
+ connector_id="aisoc-cli-submit",
+ connector_type="okta_system_log",
+ source_format="json",
+ )
+
+ resp = await submit_alert(payload, user, session)
+
+ assert isinstance(resp, AlertResponse)
+ assert resp.tenant_id == tenant
+ assert resp.severity == "high"
+ assert resp.title == "User login to Okta"
+ # Persistence sequence: add(), flush(), refresh()
+ session.add.assert_called_once()
+ session.flush.assert_awaited_once()
+ session.refresh.assert_awaited_once()
+
+ @pytest.mark.asyncio
+ async def test_empty_events_returns_400(self) -> None:
+ """The endpoint must reject empty batches with a 400, not a 500."""
+ user = _user()
+ session = _mock_db_session()
+ payload = AlertSubmitRequest(events=[], connector_type="okta_system_log")
+
+ with pytest.raises(HTTPException) as exc:
+ await submit_alert(payload, user, session)
+
+ assert exc.value.status_code == 400
+ assert "non-empty" in exc.value.detail
+ # No DB writes happened.
+ session.add.assert_not_called()
+ session.flush.assert_not_awaited()
+
+ @pytest.mark.asyncio
+ async def test_alert_is_pinned_to_current_user_tenant(self) -> None:
+ """The synthesised alert MUST inherit the caller's tenant.
+
+ This is the only tenant-isolation guarantee on the submit path —
+ callers can't smuggle a different tenant_id through the payload.
+ """
+ tenant = uuid.uuid4()
+ user = _user(tenant)
+ session = _mock_db_session()
+
+ await submit_alert(
+ AlertSubmitRequest(events=[_okta_event()], connector_type="okta_system_log"),
+ user,
+ session,
+ )
+
+ added_alerts = session._added
+ assert len(added_alerts) == 1
+ assert added_alerts[0].tenant_id == tenant
+
+ @pytest.mark.asyncio
+ async def test_override_severity_lands_on_persisted_alert(self) -> None:
+ """``severity`` override on the payload wins over event severity."""
+ user = _user()
+ session = _mock_db_session()
+
+ await submit_alert(
+ AlertSubmitRequest(
+ events=[_okta_event(severity="INFO")],
+ connector_type="okta_system_log",
+ severity="critical",
+ ),
+ user,
+ session,
+ )
+ added = session._added
+ assert added[0].severity == "critical"
+ assert added[0].priority == 95
+
+ @pytest.mark.asyncio
+ async def test_payload_tags_are_attached(self) -> None:
+ user = _user()
+ session = _mock_db_session()
+
+ await submit_alert(
+ AlertSubmitRequest(
+ events=[_okta_event()],
+ connector_type="okta_system_log",
+ tags=["red-team", "lateral-movement"],
+ ),
+ user,
+ session,
+ )
+ added = session._added
+ tags = set(added[0].tags)
+ assert "red-team" in tags
+ assert "lateral-movement" in tags
+ # submitted marker still added
+ assert "submitted" in tags
+
+ @pytest.mark.asyncio
+ async def test_multi_event_batch_creates_single_alert(self) -> None:
+ """Batches collapse to one alert, not N — that's the design promise."""
+ user = _user()
+ session = _mock_db_session()
+
+ await submit_alert(
+ AlertSubmitRequest(
+ events=[
+ _okta_event(actor_id="[email protected]"),
+ _okta_event(actor_id="[email protected]"),
+ ],
+ connector_type="okta_system_log",
+ ),
+ user,
+ session,
+ )
+
+ added = session._added
+ assert len(added) == 1
+ # Both users land on the same alert.
+ assert "[email protected]" in added[0].affected_users
+ assert "[email protected]" in added[0].affected_users
+
+ @pytest.mark.asyncio
+ async def test_payload_validates_optional_fields(self) -> None:
+ """``AlertSubmitRequest`` is forgiving — empty optionals are fine."""
+ # The CLI omits some fields when the fixture is minimal.
+ req = AlertSubmitRequest(events=[_okta_event()])
+ assert req.connector_id is None
+ assert req.connector_type is None
+ assert req.source_format is None
+ assert req.tags is None
+ # And the endpoint accepts it.
+ user = _user()
+ session = _mock_db_session()
+ resp = await submit_alert(req, user, session)
+ assert resp.severity == "info"