Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

test(8.1): prove the retention window live, and fix a cascade the FK promised

Beenu Arora42ec5ca1ff0e
diff --git a/services/api/migrations/111_audit_retention.sql b/services/api/migrations/111_audit_retention.sql
index 15bf4568c..94d0e117e 100644
--- a/services/api/migrations/111_audit_retention.sql
+++ b/services/api/migrations/111_audit_retention.sql
@@ -67,11 +67,30 @@ CREATE INDEX IF NOT EXISTS aisoc_audit_checkpoints_tenant_idx
-- unconditionally, because on DELETE there is no NEW and `RETURN NEW`
-- returns NULL, which in a BEFORE trigger silently *cancels* the statement
-- and reports "DELETE 0" as success.
+--
+-- The one exception is the cascade from `tenants`. The foreign key above
+-- declares `ON DELETE CASCADE`, and an unconditional refusal here would
+-- make that declaration a lie: the tenant delete would raise instead of
+-- cascading, so the row could never be removed by any means at all.
+--
+-- A cascade is told apart from a direct DELETE by whether the parent is
+-- still there. PostgreSQL implements `ON DELETE CASCADE` as an AFTER
+-- trigger on the parent, so by the time this BEFORE trigger runs on the
+-- child the tenant row is already gone; a caller deleting checkpoints
+-- directly still has their tenant. Measured both ways rather than assumed.
+--
+-- That is a narrow hole and deliberately an expensive one: erasing the
+-- record of a truncation means destroying the entire tenant, which takes
+-- every alert, case and credential with it. Selective erasure -- the thing
+-- this table exists to prevent -- is still refused.
CREATE OR REPLACE FUNCTION public.audit_checkpoint_immutable()
RETURNS trigger
LANGUAGE plpgsql
AS $fn$
BEGIN
+ IF TG_OP = 'DELETE' AND NOT EXISTS (SELECT 1 FROM tenants WHERE id = OLD.tenant_id) THEN
+ RETURN OLD;
+ END IF;
RAISE EXCEPTION 'aisoc_audit_checkpoints rows are immutable (attempted %)', TG_OP;
END;
$fn$;
diff --git a/tests/isolation/test_audit_retention_live.py b/tests/isolation/test_audit_retention_live.py
new file mode 100644
index 000000000..73a783aa9
--- /dev/null
+++ b/tests/isolation/test_audit_retention_live.py
@@ -0,0 +1,441 @@
+"""An enforced audit retention window that leaves the chain verifiable.
+
+Depth 8.1, the remainder of parity 4.4.
+
+What was missing
+----------------
+`retention_policies.audit_days` had been storable since the table existed
+and was purged by nothing. The retention worker said so in its own module
+docstring and said why: `audit_log` is an append-only hash chain whose
+entire value is that no row was deleted, reordered or rewritten, so a
+plain `DELETE` of the oldest rows leaves the new first row naming a
+predecessor that is gone. `verify_chain_breaks` replays from
+`prev_hash = NULL` and reports a mismatch at index 0 -- an intact chain
+reads as tampered with, which is the worst answer a control like this can
+give, because it is indistinguishable from the real thing.
+
+Migration `111` is the missing piece: `audit_retention_truncate` writes a
+checkpoint recording the cutoff, the count, the hash of the last row
+removed and the predecessor the oldest survivor names, and only then
+deletes -- in one transaction. Verification re-anchors on the checkpoint.
+
+Why this suite is live rather than offline
+------------------------------------------
+Every control here is enforced by Postgres and by nothing in Python:
+a `SECURITY DEFINER` function, a `REVOKE DELETE` on the application role,
+a `BEFORE DELETE` trigger reading a transaction-local setting, and two RLS
+policies. A fake session answers whatever the handler asks, so an offline
+test of this would be a test of the double. In particular
+`test_the_runtime_role_cannot_delete_an_audit_row_directly` can only fail
+against a real server holding real grants, and it is the assertion the
+whole design rests on: without it the trigger's setting is a comment,
+because a custom GUC can be set by anybody.
+
+`ENVIRONMENT=test`, never `development`, which is in
+`AUTH_BYPASS_ENVIRONMENTS`.
+
+The connection is the DML-only `aisoc_app` role, not the owner. A
+superuser ignores every RLS policy even under FORCE ROW LEVEL SECURITY
+and holds DELETE on everything, so this suite would pass as the owner
+with the privilege control removed -- which is the sibling defect that
+has shipped here before.
+
+The negative control
+--------------------
+`test_without_the_anchor_the_same_chain_reads_as_broken` replays the
+surviving rows from the genesis `prev_hash = NULL` instead of from the
+checkpoint. It must report a break. Without it,
+"the truncated chain verifies" would pass equally well against a
+verifier that had stopped checking `prev_hash` at all.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import uuid
+from datetime import UTC, datetime, timedelta
+
+import pytest
+import pytest_asyncio
+
+pytestmark = [
+ pytest.mark.asyncio(loop_scope="module"),
+ pytest.mark.skipif(
+ not os.environ.get("ISOLATION_AUDIT_RETENTION_DSN", "").strip(),
+ reason="ISOLATION_AUDIT_RETENTION_DSN is not set; this suite needs a live Postgres",
+ ),
+]
+
+#: How many rows the fixture chain carries, and where the cutoff falls.
+#: Four old and three recent, so a cutoff that removes "the oldest" is
+#: removing several links rather than one, and an off-by-one in the
+#: boundary (`<` against `<=`) changes the count rather than hiding in it.
+OLD_ROWS = 4
+RECENT_ROWS = 3
+
+
+def _dsn() -> str:
+ value = os.environ.get("ISOLATION_AUDIT_RETENTION_DSN", "").strip()
+ if not value:
+ pytest.skip("ISOLATION_AUDIT_RETENTION_DSN is not set")
+ if value.startswith("postgresql://"):
+ return value.replace("postgresql://", "postgresql+asyncpg://", 1)
+ return value
+
+
+@pytest_asyncio.fixture(scope="module", loop_scope="module")
+async def engine():
+ # Imported inside the fixture. One module-scope import of a service
+ # took 241 unrelated tests down in the offline collection job, which
+ # installs minimal dependencies.
+ from sqlalchemy.ext.asyncio import create_async_engine
+
+ eng = create_async_engine(_dsn())
+ try:
+ yield eng
+ finally:
+ await eng.dispose()
+
+
+async def _chain_rows(conn, tenant_id: str) -> list[dict]:
+ """Every audit row for a tenant, oldest first, in replay shape."""
+ from sqlalchemy import text
+
+ result = await conn.execute(
+ text(
+ "SELECT id, tenant_id, actor_id, actor_email, action, resource, "
+ " resource_id, changes, metadata, created_at, prev_hash, "
+ " entry_hash, chain_index "
+ " FROM audit_log WHERE tenant_id = :t "
+ " ORDER BY created_at ASC, chain_index ASC"
+ ),
+ {"t": tenant_id},
+ )
+ return [dict(r) for r in result.mappings().all()]
+
+
+@pytest_asyncio.fixture(scope="module", loop_scope="module")
+async def chained_tenant(engine):
+ """A tenant whose audit log is a chain with rows on both sides of a cutoff.
+
+ Every `prev_hash` and `entry_hash` is computed by the product's own
+ `compute_entry_hash`, so the chain under test is linked exactly as the
+ writer links it and the verifier is checking real hashes rather than
+ ones this file knows how to satisfy.
+
+ What this fixture does *not* use is `emit_audit`, and the reason is
+ worth stating because it looks like the weaker choice. `created_at` is
+ one of the hashed fields, deliberately, and `emit_audit` stamps it
+ `now()`. A retention window needs rows older than the cutoff, and the
+ immutability trigger refuses an `UPDATE` of `created_at` -- which it
+ should, and which this suite confirms from the other direction in
+ `TestTheOnlyWayOutIsTheFunction`. There is no supported way to write a
+ historical audit row, so the rows are inserted with their timestamps
+ and their hashes computed together, which is what `emit_audit` itself
+ does. Only the `INSERT` differs.
+ """
+ from app.services.audit_hash import compute_entry_hash
+ from sqlalchemy import text
+
+ tenant_id = str(uuid.uuid4())
+ tag = tenant_id[:8]
+ now = datetime.now(UTC)
+
+ async with engine.begin() as conn:
+ await conn.execute(
+ text("INSERT INTO tenants (id, name, slug) VALUES (:id, :n, :s)"),
+ {"id": tenant_id, "n": f"ret-{tag}", "s": f"ret-{tag}"},
+ )
+
+ prev_hash: str | None = None
+ for n in range(OLD_ROWS + RECENT_ROWS):
+ # The first OLD_ROWS sit well behind any sane cutoff; the rest
+ # are minutes old. Ordering is strictly increasing either way,
+ # which is what the replay assumes.
+ created_at = now - (timedelta(days=400 - n) if n < OLD_ROWS else timedelta(minutes=OLD_ROWS + RECENT_ROWS - n))
+ row_id = uuid.uuid4()
+ changes = {"seq": n}
+ entry_hash = compute_entry_hash(
+ prev_hash=prev_hash,
+ row_id=row_id,
+ tenant_id=uuid.UUID(tenant_id),
+ actor_id=None,
+ actor_email=f"operator-{n}@corp.example",
+ actor_ip=None,
+ action="cases:update",
+ resource="case",
+ resource_id=f"case-{n}",
+ changes=changes,
+ metadata=None,
+ created_at=created_at,
+ )
+ await conn.execute(
+ text(
+ "INSERT INTO audit_log (id, tenant_id, actor_email, action, resource, "
+ " resource_id, changes, created_at, prev_hash, "
+ " entry_hash, chain_index) "
+ "VALUES (:id, :t, :ae, :a, :r, :ri, CAST(:c AS jsonb), :ca, :ph, :eh, :ci)"
+ ),
+ {
+ "id": str(row_id),
+ "t": tenant_id,
+ "ae": f"operator-{n}@corp.example",
+ "a": "cases:update",
+ "r": "case",
+ "ri": f"case-{n}",
+ "c": json.dumps(changes),
+ "ca": created_at,
+ "ph": prev_hash,
+ "eh": entry_hash,
+ "ci": n,
+ },
+ )
+ prev_hash = entry_hash
+
+ yield tenant_id
+
+ async with engine.begin() as conn:
+ # Through the sanctioned function, not by cascade. `audit_log` has
+ # `ON DELETE CASCADE` from `tenants`, but the immutability trigger
+ # refuses the cascaded DELETE like any other -- so a plain
+ # `DELETE FROM tenants` raises for any tenant that has ever written
+ # an audit row. Clearing the rows first is the only way out, and it
+ # is also the one this suite is about.
+ await conn.execute(
+ text("SELECT audit_retention_truncate(:t, :c, :d)"),
+ {"t": tenant_id, "c": datetime.now(UTC) + timedelta(days=1), "d": 0},
+ )
+ await conn.execute(text("DELETE FROM tenants WHERE id = :t"), {"t": tenant_id})
+
+
+@pytest_asyncio.fixture(scope="module", loop_scope="module")
+async def truncated(engine, chained_tenant):
+ """Run the real truncation once, and hand back what it reported."""
+ from sqlalchemy import text
+
+ cutoff = datetime.now(UTC) - timedelta(days=90)
+ async with engine.begin() as conn:
+ before = await _chain_rows(conn, chained_tenant)
+ removed = (
+ await conn.execute(
+ text("SELECT audit_retention_truncate(:t, :c, :d)"),
+ {"t": chained_tenant, "c": cutoff, "d": 90},
+ )
+ ).scalar()
+ after = await _chain_rows(conn, chained_tenant)
+ checkpoint = (
+ (
+ await conn.execute(
+ text(
+ "SELECT truncated_before, window_days, rows_removed, last_removed_hash, "
+ " first_retained_prev_hash, first_retained_hash "
+ " FROM aisoc_audit_checkpoints WHERE tenant_id = :t "
+ " ORDER BY created_at DESC LIMIT 1"
+ ),
+ {"t": chained_tenant},
+ )
+ )
+ .mappings()
+ .first()
+ )
+ return {"removed": removed, "before": before, "after": after, "checkpoint": dict(checkpoint) if checkpoint else None}
+
+
+class TestTheWindowIsEnforced:
+ async def test_the_rows_past_the_window_are_gone(self, truncated):
+ assert truncated["removed"] == OLD_ROWS
+ assert len(truncated["before"]) == OLD_ROWS + RECENT_ROWS
+ assert len(truncated["after"]) == RECENT_ROWS
+
+ async def test_the_checkpoint_records_what_went_and_under_which_policy(self, truncated):
+ """Both, because they answer different questions.
+
+ "Everything before 13 July went" and "a 90-day window was in
+ force" are not the same fact, and an auditor asks the second.
+ """
+ cp = truncated["checkpoint"]
+ assert cp is not None, "truncation removed rows and wrote no checkpoint"
+ assert cp["rows_removed"] == OLD_ROWS
+ assert cp["window_days"] == 90
+
+ async def test_the_checkpoint_agrees_with_the_rows_it_describes(self, truncated):
+ """The anchor must be the hash of the last row actually removed.
+
+ Stored twice on purpose -- `last_removed_hash` from the removed
+ side and `first_retained_prev_hash` from the surviving side -- so
+ the checkpoint can be checked against itself. On an untampered
+ chain they are equal; a checkpoint that disagrees with itself is
+ evidence in its own right.
+ """
+ cp = truncated["checkpoint"]
+ last_removed = truncated["before"][OLD_ROWS - 1]
+ first_kept = truncated["after"][0]
+
+ assert cp["last_removed_hash"] == last_removed["entry_hash"]
+ assert cp["first_retained_prev_hash"] == first_kept["prev_hash"]
+ assert cp["first_retained_hash"] == first_kept["entry_hash"]
+ assert cp["last_removed_hash"] == cp["first_retained_prev_hash"]
+
+
+class TestTheRemainderStillVerifies:
+ async def test_the_surviving_chain_replays_clean_from_the_checkpoint(self, truncated):
+ from app.services.audit_hash import verify_chain_breaks
+
+ breaks = verify_chain_breaks(
+ truncated["after"],
+ anchor_prev_hash=truncated["checkpoint"]["first_retained_prev_hash"],
+ )
+ assert breaks == [], f"a lawfully truncated chain reported {len(breaks)} break(s): {breaks}"
+
+ async def test_verify_chain_agrees_with_the_full_replay(self, truncated):
+ from app.services.audit_hash import verify_chain
+
+ ok, index, reason = verify_chain(
+ truncated["after"],
+ anchor_prev_hash=truncated["checkpoint"]["first_retained_prev_hash"],
+ )
+ assert ok, f"verify_chain reported a break at {index}: {reason}"
+
+ async def test_without_the_anchor_the_same_chain_reads_as_broken(self, truncated):
+ """The negative control.
+
+ Replaying the survivors from the genesis `prev_hash = NULL` is
+ what the verifier did before this item, and it is why a retention
+ purge could not be built. It must still report a break here --
+ otherwise "the truncated chain verifies" would pass equally well
+ against a verifier that had stopped checking `prev_hash`.
+ """
+ from app.services.audit_hash import verify_chain_breaks
+
+ breaks = verify_chain_breaks(truncated["after"])
+ assert breaks, "replaying a truncated chain from genesis reported no break; the anchor is proving nothing"
+ assert breaks[0]["index"] == 0
+
+
+class TestTheOnlyWayOutIsTheFunction:
+ async def test_the_runtime_role_cannot_delete_an_audit_row_directly(self, engine, chained_tenant):
+ """The privilege half, and the one the whole design rests on.
+
+ The trigger refuses a DELETE unless a transaction-local setting
+ says a retention purge is in progress, and a custom GUC can be
+ set by anybody -- so on its own that is a comment, not a control.
+ `REVOKE DELETE ON audit_log FROM aisoc_app` is what makes the
+ function the only path, and only a real server holding real
+ grants can demonstrate it.
+ """
+ from sqlalchemy import text
+ from sqlalchemy.exc import DBAPIError, ProgrammingError
+
+ with pytest.raises((ProgrammingError, DBAPIError)) as caught:
+ async with engine.begin() as conn:
+ await conn.execute(text("DELETE FROM audit_log WHERE tenant_id = :t"), {"t": chained_tenant})
+ assert "permission denied" in str(caught.value).lower()
+
+ async def test_setting_the_purge_flag_by_hand_does_not_help(self, engine, chained_tenant):
+ """Defence in depth, in the order that matters.
+
+ Postgres checks the privilege before the trigger runs, so a
+ caller who guesses the setting's name is refused one layer below
+ the one they were aiming at.
+ """
+ from sqlalchemy import text
+ from sqlalchemy.exc import DBAPIError, ProgrammingError
+
+ with pytest.raises((ProgrammingError, DBAPIError)) as caught:
+ async with engine.begin() as conn:
+ await conn.execute(text("SELECT set_config('app.audit_retention_purge', 'on', true)"))
+ await conn.execute(text("DELETE FROM audit_log WHERE tenant_id = :t"), {"t": chained_tenant})
+ assert "permission denied" in str(caught.value).lower()
+
+ async def test_a_no_op_window_writes_no_checkpoint(self, engine, chained_tenant):
+ """A table of "removed 0 rows" entries every tick buries the ones
+ that mean something."""
+ from sqlalchemy import text
+
+ async with engine.begin() as conn:
+ before = (
+ await conn.execute(text("SELECT count(*) FROM aisoc_audit_checkpoints WHERE tenant_id = :t"), {"t": chained_tenant})
+ ).scalar()
+ removed = (
+ await conn.execute(
+ text("SELECT audit_retention_truncate(:t, :c, :d)"),
+ {"t": chained_tenant, "c": datetime.now(UTC) - timedelta(days=3650), "d": 3650},
+ )
+ ).scalar()
+ after = (
+ await conn.execute(text("SELECT count(*) FROM aisoc_audit_checkpoints WHERE tenant_id = :t"), {"t": chained_tenant})
+ ).scalar()
+
+ assert removed == 0
+ assert after == before
+
+
+class TestTheRecordOfTheRemovalIsItselfARecord:
+ async def test_a_checkpoint_cannot_be_updated(self, engine, chained_tenant, truncated):
+ """A record of what was deleted which can be edited is not a record."""
+ from sqlalchemy import text
+ from sqlalchemy.exc import DBAPIError, ProgrammingError
+
+ with pytest.raises((ProgrammingError, DBAPIError)) as caught:
+ async with engine.begin() as conn:
+ await conn.execute(
+ text("UPDATE aisoc_audit_checkpoints SET rows_removed = 0 WHERE tenant_id = :t"),
+ {"t": chained_tenant},
+ )
+ assert "immutable" in str(caught.value).lower()
+
+ async def test_a_checkpoint_cannot_be_deleted(self, engine, chained_tenant, truncated):
+ """Refused first and unconditionally.
+
+ On DELETE there is no NEW, and `RETURN NEW` in a BEFORE trigger
+ returns NULL -- which silently *cancels* the statement and
+ reports "DELETE 0" as success. That is the shape migration 101
+ had to repair on `audit_log` itself.
+ """
+ from sqlalchemy import text
+ from sqlalchemy.exc import DBAPIError, ProgrammingError
+
+ with pytest.raises((ProgrammingError, DBAPIError)) as caught:
+ async with engine.begin() as conn:
+ await conn.execute(text("DELETE FROM aisoc_audit_checkpoints WHERE tenant_id = :t"), {"t": chained_tenant})
+ assert "immutable" in str(caught.value).lower()
+
+ async def test_the_cascade_from_its_tenant_is_the_one_exception(self, engine):
+ """And the foreign key means it, which an unconditional trigger did not.
+
+ `tenant_id` is declared `ON DELETE CASCADE`. A trigger that refused
+ every DELETE made that a lie -- the tenant delete raised instead of
+ cascading, so the row could never be removed by any means. That is
+ the shape already blocking `DELETE FROM tenants` on `audit_log`
+ itself, and this table must not add a second one.
+
+ The exception is narrow by construction: a cascade is told apart
+ from a direct DELETE by whether the parent is still there, and the
+ case above proves the direct one is still refused while this
+ tenant's row exists. Erasing a checkpoint therefore costs the whole
+ tenant -- every alert, case and credential with it -- which is not
+ selective erasure in any useful sense.
+ """
+ from sqlalchemy import text
+
+ tenant_id = str(uuid.uuid4())
+ tag = tenant_id[:8]
+ async with engine.begin() as conn:
+ await conn.execute(
+ text("INSERT INTO tenants (id, name, slug) VALUES (:id, :n, :s)"),
+ {"id": tenant_id, "n": f"casc-{tag}", "s": f"casc-{tag}"},
+ )
+ await conn.execute(
+ text(
+ "INSERT INTO aisoc_audit_checkpoints (tenant_id, truncated_before, window_days, rows_removed) VALUES (:t, :c, 90, 1)"
+ ),
+ {"t": tenant_id, "c": datetime.now(UTC)},
+ )
+
+ async with engine.begin() as conn:
+ await conn.execute(text("DELETE FROM tenants WHERE id = :t"), {"t": tenant_id})
+ left = (
+ await conn.execute(text("SELECT count(*) FROM aisoc_audit_checkpoints WHERE tenant_id = :t"), {"t": tenant_id})
+ ).scalar()
+ assert left == 0