Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

32.9 KB
Raw Blame
"""The backtest's lake fetch, proven against a real ClickHouse.

The claim is: *a candidate detection can be backtested over historical lake
data before promotion, and it sees only the backtesting tenant's history.*
Everything below exists to make the second half of that provable rather than
asserted.

Why this test is shaped the way it is
-------------------------------------

``fetch_lake_events`` was the one step of the backtest with no coverage at
all. Its own unit suite says so in its docstring, and the claim-to-gate row
said the live fetch was "exercised by the integration gate" — no gate ran it.
So the three things it does were all unproven against a warehouse:

* ``rewrite_for_tenant`` injects the tenant predicate. This is the only thing
  standing between a backtest and another tenant's history, and this
  repository has a recorded scar exactly here: sqlglot 27 renamed the
  ``SELECT``'s FROM argument key, the rewriter's table walk found nothing, and
  every single-table query came back with no tenant predicate, no allowlist
  check and no table-function ban — reported as success, with every offline
  suite green. A rendered SQL string cannot tell you whether rows crossed.
* the ``window_days`` and ``limit`` clamps bound what a backtest reads. Both
  were checked by substring against a generated string, which is the assertion
  that survives a rewriter that silently drops the clause.
* ``rows_to_events`` maps lake columns onto the event dicts the rule engine
  matches. Asserting that against a hand-written column list is circular, and
  it was: the list named a ``raw_data`` column, ``aisoc.raw_events`` has no
  such column, and the payload expansion the docstring promises had therefore
  never run on a real lake row.

So no part of the chain below is re-implemented here:

* the table is created from ``services/api/clickhouse/001_init.sql``, the DDL
  a deployment actually runs;
* the rows come from ``lake_writer.event_to_row``, the only function in
  production that writes to ``aisoc.raw_events``;
* the read is the production ``fetch_lake_events``, which reaches the real
  rewriter and the real ``execute_lake_query``, settings singleton and all.

Refusing to pass vacuously
--------------------------

A test that reads zero rows and then asserts "no other tenant's rows" passes
trivially, which would make this file worse than nothing. Three guards:

* every seeded row is counted back out of the warehouse before anything is
  asserted about scoping, and the fixture fails if the writer refused an event;
* the *unscoped* form of the very same statement is executed alongside the
  scoped one and asserted to return both tenants, so the scoped result is
  known to be a filter rather than an empty partition;
* ``BACKTEST_LAKE_LIVE_REQUIRED=1`` turns an unreachable warehouse into a
  failure. ``isolation-live.yml`` sets it, so the job cannot go green by
  skipping. Without it the file skips with the host named, the way the rest of
  this directory does, so a local run stays usable.

Every module is loaded by path and unregistered afterwards, because
``services/api``, ``services/agents`` and ``services/fusion`` all package
their code as a top-level ``app`` and only one of them can own that name in
one interpreter. Putting ``services/api`` on ``sys.path`` was tried first and
it works for this file while breaking ``test_route_auth_default_deny.py`` ten
tests later with ``No module named 'app.api.contextual'`` — the API's ``app``
stays cached in ``sys.modules`` and the agents service never gets to load its
own. Each file in ``isolation-live.yml`` is its own pytest invocation, so CI
would not have shown it; ``pytest tests/isolation`` would.
"""

from __future__ import annotations

import ast
import asyncio
import importlib.util
import os
import re
import sys
import types
import uuid
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Any, NoReturn

import pytest

REPO_ROOT = Path(__file__).resolve().parents[2]
API_ROOT = REPO_ROOT / "services" / "api"

#: Parent packages the dotted names below need to resolve through. Created as
#: empty modules only when the name is free, and handed back afterwards.
API_PACKAGES: tuple[str, ...] = ("app", "app.core", "app.db", "app.services")

#: Every module the call under test reaches, in dependency order, under the
#: real dotted name so the ``from app...`` statements inside them resolve.
#: ``fetch_lake_events`` imports two of these *inside* the function body, so
#: they have to stay registered while the call runs, not just while it loads.
API_MODULES: tuple[tuple[str, str], ...] = (
    ("app.core.config", "app/core/config.py"),
    # `rule_engine` grew a top-level `from app.services.lucene_eval import …`
    # after this list was written. `app.services` here is a synthetic
    # namespace holding only what is registered below, so an unlisted sibling
    # does not resolve — the error reads `'app.services' is not a package`,
    # which points at the namespace rather than at the missing entry. Anything
    # a listed module imports at module scope has to be listed above it.
    ("app.services.lucene_eval", "app/services/lucene_eval.py"),
    ("app.services.rule_engine", "app/services/rule_engine.py"),
    ("app.services.lake_sql", "app/services/lake_sql.py"),
    ("app.db.clickhouse", "app/db/clickhouse.py"),
    ("app.services.backtest", "app/services/backtest.py"),
)

# `ISOLATION_*` is what `isolation-live.yml` sets, so this file behaves like
# the rest of the directory in CI; the unprefixed names are the local
# fallback. Chained `or` with the default last so a variable set to the empty
# string falls through rather than reaching `int("")`.
CLICKHOUSE_HOST = os.getenv("ISOLATION_CLICKHOUSE_HOST") or os.getenv("CLICKHOUSE_HOST") or "localhost"
CLICKHOUSE_PORT = int(os.getenv("ISOLATION_CLICKHOUSE_PORT") or os.getenv("CLICKHOUSE_PORT") or "9000")

# `app.core.config` builds its settings singleton at import, and
# `app.db.clickhouse` reads `settings.CLICKHOUSE_HOST` from it to construct the
# client. That import happens inside the call under test, so the environment
# has to be right before any test runs rather than before a particular one.
# Set unconditionally: pydantic-settings ranks the environment above its
# `.env` file, so this is what the production client will dial.
os.environ["CLICKHOUSE_HOST"] = CLICKHOUSE_HOST
os.environ["CLICKHOUSE_PORT"] = str(CLICKHOUSE_PORT)

#: When set, an unreachable warehouse fails instead of skipping. This is the
#: difference between a job that proves the fetch and a job that reports
#: success having run nothing.
REQUIRED = os.getenv("BACKTEST_LAKE_LIVE_REQUIRED", "").strip() not in ("", "0", "false")

# Two tenants. Tenant-scoping is a statement about the pair, so neither is
# meaningful alone: A's rows must come back and B's must not, and B must hold
# rows at every window A is read at or its absence proves nothing.
TENANT_A = uuid.UUID("aaaaaaaa-0000-4000-8000-00000000ba01")
TENANT_B = uuid.UUID("bbbbbbbb-0000-4000-8000-00000000ba02")

CONNECTOR = "aws_cloudtrail"
OTHER_CONNECTOR = "okta_system_log"

# Ages chosen against the two clamps `build_backtest_sql` applies. The 200-day
# row sits between the default window and the 365-day ceiling; the 400-day row
# sits beyond it. A request for 9999 days therefore has one row that proves the
# clamp bit (the 400-day row must not come back) and one that proves it did not
# over-clamp (the 200-day row must), which is not observable in a rendered
# string. Hours rather than a whole day for the freshest rows, so a 1-day
# window is not asserted against its own boundary.
AGE_FRESH = timedelta(hours=2)
AGE_RECENT = timedelta(days=2)
AGE_MID = timedelta(days=200)
AGE_ANCIENT = timedelta(days=400)

#: ``(label, tenant, age, connector)``. The label is written to ``user_name``
#: so every assertion below can name the rows it expects instead of counting
#: them. Three fresh rows for A so a limit below the available row count is
#: testable against real data.
SEED: tuple[tuple[str, uuid.UUID, timedelta, str], ...] = (
    ("a-fresh-1", TENANT_A, AGE_FRESH, CONNECTOR),
    ("a-fresh-2", TENANT_A, AGE_FRESH, CONNECTOR),
    ("a-fresh-3", TENANT_A, AGE_FRESH, CONNECTOR),
    ("a-okta-fresh", TENANT_A, AGE_FRESH, OTHER_CONNECTOR),
    ("a-recent", TENANT_A, AGE_RECENT, CONNECTOR),
    ("a-mid", TENANT_A, AGE_MID, CONNECTOR),
    ("a-ancient", TENANT_A, AGE_ANCIENT, CONNECTOR),
    ("b-fresh", TENANT_B, AGE_FRESH, CONNECTOR),
    ("b-recent", TENANT_B, AGE_RECENT, CONNECTOR),
    ("b-mid", TENANT_B, AGE_MID, CONNECTOR),
    ("b-ancient", TENANT_B, AGE_ANCIENT, CONNECTOR),
)

A_LABELS = frozenset(label for label, tenant, _age, _conn in SEED if tenant == TENANT_A)
B_LABELS = frozenset(label for label, tenant, _age, _conn in SEED if tenant == TENANT_B)

# What each window must return for tenant A, derived from SEED rather than
# typed out, so adding a row cannot leave an expectation stale.
WITHIN_1_DAY = frozenset(label for label, t, age, _c in SEED if t == TENANT_A and age < timedelta(days=1))
WITHIN_30_DAYS = frozenset(label for label, t, age, _c in SEED if t == TENANT_A and age < timedelta(days=30))
WITHIN_365_DAYS = frozenset(label for label, t, age, _c in SEED if t == TENANT_A and age < timedelta(days=365))

# A marker only reachable by expanding the JSON payload column. It is not any
# column of `aisoc.raw_events`, so a rule matching it can only fire if
# `rows_to_events` expanded the payload the writer stored.
PAYLOAD_EVENT_NAME = "ConsoleLogin"
PAYLOAD_MARKER_FIELD = "eventName"


def _load_by_path(rel: str, alias: str) -> Any:
    """Import one module by file path, leaving ``sys.modules`` as it found it.

    Registered before ``exec_module`` because ``@dataclass`` resolves
    annotations through ``sys.modules``, and removed afterwards because a
    synthetic name left behind has previously broken unrelated tests in this
    repository while every test in its own file passed.
    """
    spec = importlib.util.spec_from_file_location(alias, REPO_ROOT / rel)
    if spec is None or spec.loader is None:  # pragma: no cover - path is fixed
        raise RuntimeError(f"cannot load {rel}")
    module = importlib.util.module_from_spec(spec)
    sys.modules[alias] = module
    try:
        spec.loader.exec_module(module)
    finally:
        sys.modules.pop(alias, None)
    return module


def _raw_events_ddl() -> str:
    """The production DDL for ``aisoc.raw_events``, read from the file.

    The match runs to the ``ORDER BY`` clause, so the engine, the partition
    key and every column type come from the deployed schema while the ``TTL``
    and ``SETTINGS`` tails do not. That is load-bearing rather than
    incidental: the shipped table drops rows 90 days past ``event_time``, and
    two of the rows below are deliberately older than that because they are
    what the window clamps are measured against. ``INDEX`` clauses are dropped
    too — bloom filters change the plan, never the result.
    """
    sql = (REPO_ROOT / "services/api/clickhouse/001_init.sql").read_text(encoding="utf-8")
    match = re.search(r"(CREATE TABLE IF NOT EXISTS aisoc\.raw_events.*?ORDER BY \([^)]*\))", sql, re.DOTALL)
    assert match, "could not find the raw_events DDL; this test must not invent one"
    body = match.group(1)
    return "\n".join(line for line in body.splitlines() if not line.strip().startswith(("INDEX ", "--")))


def _ocsf_event(*, label: str, tenant: uuid.UUID, age: timedelta, connector: str) -> dict[str, Any]:
    """One normalized OCSF event, shaped the way ``services/ingest`` emits one.

    The nesting is the point. The writer reads ``actor.user.name``,
    ``metadata.product.name`` and ``raw_data`` from these positions, and a
    pre-flattened fixture would pass here while production wrote empty
    columns.
    """
    when = datetime.now(UTC) - age
    return {
        "id": str(uuid.uuid4()),
        "tenant_id": str(tenant),
        "connector_type": connector,
        "ocsf_event": {
            "class_uid": 3002,
            "category_uid": 3,
            "severity_id": 3,
            "severity": "medium",
            "time": when.isoformat().replace("+00:00", "Z"),
            "src_endpoint": {"ip": "203.0.113.10", "port": 44321},
            "dst_endpoint": {"ip": "198.51.100.20", "port": 443, "hostname": "console.example.invalid"},
            "device": {"name": "host-01"},
            # The label rides in `user_name`, which is an ordinary indexed
            # column, so naming rows costs the test nothing structural.
            "actor": {"user": {"name": label}},
            "process": {"name": "curl"},
            "metadata": {"product": {"name": connector}},
            # Stored by the writer into the column it calls `raw_payload`.
            "raw_data": f'{{"{PAYLOAD_MARKER_FIELD}": "{PAYLOAD_EVENT_NAME}", "label": "{label}"}}',
        },
    }


def _unavailable(detail: str) -> NoReturn:
    """Name what is missing, and decide whether missing is allowed.

    One helper so the skip reason and the required-mode failure can never
    drift apart or describe different things.
    """
    if REQUIRED:
        pytest.fail(f"BACKTEST_LAKE_LIVE_REQUIRED is set and {detail}")
    pytest.skip(detail)
    # Both calls above raise. Written out rather than left implicit because
    # the type-check runs with `--no-site-packages`, so pytest's own
    # `NoReturn` annotations are not visible to it and the declared return
    # type would otherwise be unprovable.
    raise AssertionError("unreachable")


@pytest.fixture(scope="module")
def clickhouse_client() -> Any:
    """A live warehouse, or a named skip — never a silent one.

    The driver is probed with ``find_spec`` and then imported
    unconditionally, rather than imported inside a ``try``. Both forms work;
    this one also keeps the binding provably defined on the path that uses
    it, which the ``try`` form did not — ``pytest.skip`` ends the test but is
    not a return, so static analysis read the client construction below as
    reachable with the module unbound.
    """
    if importlib.util.find_spec("clickhouse_driver") is None:  # pragma: no cover - environment-specific
        _unavailable("clickhouse-driver is not installed")

    import clickhouse_driver  # noqa: PLC0415 - soft dependency, probed on the line above

    try:
        client = clickhouse_driver.Client(
            host=CLICKHOUSE_HOST,
            port=CLICKHOUSE_PORT,
            user=os.getenv("CLICKHOUSE_USER", "default"),
            password=os.getenv("CLICKHOUSE_PASSWORD", ""),
            connect_timeout=5,
        )
        client.execute("SELECT 1")
    except Exception as exc:  # noqa: BLE001 - absence is a skip unless the job requires the store
        _unavailable(f"no ClickHouse at {CLICKHOUSE_HOST}:{CLICKHOUSE_PORT} ({type(exc).__name__}: {exc})")
        raise exc from None  # unreachable: `_unavailable` always raises
    return client


@pytest.fixture(scope="module")
def seeded_lake(clickhouse_client: Any, api_modules: Any) -> Any:
    """A real lake holding rows the production writer produced.

    Both tenants are seeded at every age, so a scoped read of A can be
    compared against an unscoped read that sees B.

    Every test that reads the lake requests this, which is how they all get
    ``api_modules`` without it having to be autouse.
    """
    lake_writer = _load_by_path("services/fusion/app/services/lake_writer.py", "_aisoc_backtest_lake_writer")

    client = clickhouse_client
    client.execute("CREATE DATABASE IF NOT EXISTS aisoc")
    client.execute("DROP TABLE IF EXISTS aisoc.raw_events")
    client.execute(_raw_events_ddl())

    rows = []
    for label, tenant, age, connector in SEED:
        message = _ocsf_event(label=label, tenant=tenant, age=age, connector=connector)
        row = lake_writer.event_to_row(message)
        assert row is not None, f"the production writer refused {label}, an event this test must be able to record"
        out = {k: v for k, v in row.items() if not (k == "event_id" and v is None)}
        # The columns are `IPv6`; the writer keeps them as strings in the row
        # dict and converts at insert time, so the test converts the same way
        # rather than inventing a second representation.
        out["source_ip"] = lake_writer._to_ip_obj(row["source_ip"])
        out["dest_ip"] = lake_writer._to_ip_obj(row["dest_ip"])
        rows.append(out)

    client.execute(lake_writer._INSERT_SQL, rows, types_check=True)

    # Nothing below is allowed to run until the warehouse is known to hold
    # what was seeded. A ReplacingMergeTree collapsing rows, a TTL eating the
    # old ones or a writer returning a row the driver rejects would otherwise
    # surface as a scoping assertion that passes on an empty table.
    stored = client.execute("SELECT count() FROM aisoc.raw_events")[0][0]
    assert stored == len(SEED), f"seeded {len(SEED)} rows but the warehouse holds {stored}; every assertion below would be vacuous"

    yield client
    client.execute("DROP TABLE IF EXISTS aisoc.raw_events")


@pytest.fixture(scope="module")
def api_modules(clickhouse_client: Any) -> Any:
    """Register the API service's ``app`` tree, then hand the name back.

    Module-scoped because the two lazy imports inside ``fetch_lake_events``
    resolve at call time, so the registration has to outlive module import and
    still be gone before the next file runs. Names that were already taken are
    saved and restored rather than overwritten, so this works whichever
    service loaded ``app`` first.

    It takes ``clickhouse_client`` so the skip is decided before any of this
    is imported. ``isolation.yml`` runs this whole directory offline with only
    ``qdrant-client structlog pytest pytest-asyncio`` installed, and an
    autouse version of this fixture turned that job's clean skip into
    ``ModuleNotFoundError: No module named 'pydantic_settings'`` on thirteen
    tests — a live-store test has to be absent-tolerant in the same job the
    rest of the directory is.
    """
    installed: list[str] = []
    saved = {name: sys.modules[name] for name, _rel in API_MODULES if name in sys.modules}
    try:
        for package in API_PACKAGES:
            if package not in sys.modules:
                sys.modules[package] = types.ModuleType(package)
                installed.append(package)
        for dotted, rel in API_MODULES:
            spec = importlib.util.spec_from_file_location(dotted, API_ROOT / rel)
            if spec is None or spec.loader is None:  # pragma: no cover - paths are fixed
                raise RuntimeError(f"cannot load {rel}")
            module = importlib.util.module_from_spec(spec)
            # Registered before exec so the from-imports inside each module
            # find their siblings, and so `@dataclass` can resolve annotations.
            sys.modules[dotted] = module
            installed.append(dotted)
            try:
                spec.loader.exec_module(module)
            except ImportError as exc:
                # A dependency of the service, not of the test. Named either
                # way: skipped where the warehouse is optional, failed where
                # the job declares it is not.
                _unavailable(f"{rel} needs a dependency this environment lacks ({exc})")
        yield sys.modules["app.services.backtest"]
    finally:
        for name in reversed(installed):
            sys.modules.pop(name, None)
        sys.modules.update(saved)


def _backtest() -> Any:
    """The production backtest module, as ``api_modules`` registered it."""
    return sys.modules["app.services.backtest"]


def _fetch(tenant: uuid.UUID, *, window_days: int, limit: int, source: str | None = None) -> list[dict[str, Any]]:
    """Call the real ``fetch_lake_events`` and return what it returned.

    Driven with ``asyncio.run`` rather than an async test so this file needs
    no pytest-asyncio mode to be configured for it. The ClickHouse client is a
    module-level singleton that only the first call constructs, so a fresh
    loop per call does not re-enter the lock that guards it.
    """
    return asyncio.run(_backtest().fetch_lake_events(tenant, window_days=window_days, limit=limit, source=source))


def _labels(events: list[dict[str, Any]]) -> set[str]:
    return {str(event.get("user_name")) for event in events}


def _unscoped(client: Any, *, window_days: int, limit: int, source: str | None = None) -> set[str]:
    """The same statement the fetch builds, run without the tenant rewrite.

    This is the control. It proves the rows the scoped read declined to return
    were sitting in the table, reachable by the identical query, which is the
    only way "zero rows from the other tenant" means anything.
    """
    sql = _backtest().build_backtest_sql(window_days=window_days, limit=limit, source=source)
    columns = [name for name, _type in client.execute(sql, with_column_types=True)[1]]
    rows = client.execute(sql)
    return {str(dict(zip(columns, row, strict=False)).get("user_name")) for row in rows}


# --------------------------------------------------------------------------
# The warehouse really holds both tenants' history
# --------------------------------------------------------------------------


def test_both_tenants_have_recorded_rows_so_a_scoped_read_can_be_a_filter(seeded_lake: Any) -> None:
    """Without this, every assertion about scoping below is vacuous."""
    counts = dict(seeded_lake.execute("SELECT tenant_id, count() FROM aisoc.raw_events GROUP BY tenant_id"))
    by_tenant = {str(tenant): count for tenant, count in counts.items()}
    assert by_tenant.get(str(TENANT_A)) == len(A_LABELS)
    assert by_tenant.get(str(TENANT_B)) == len(B_LABELS)
    assert len(A_LABELS) > 0 and len(B_LABELS) > 0


def test_the_unscoped_statement_reaches_both_tenants(seeded_lake: Any) -> None:
    """The control for the scoping test, asserted rather than assumed.

    The backtest's own generated SQL, executed with no tenant predicate,
    returns rows from both tenants. Everything the scoped read excludes was
    therefore excluded by the rewriter and not by the data.
    """
    reachable = _unscoped(seeded_lake, window_days=365, limit=1000)
    assert WITHIN_365_DAYS <= reachable, "the generated statement cannot see tenant A's own rows"
    assert B_LABELS & reachable, "the generated statement cannot see tenant B, so scoping it proves nothing"


# --------------------------------------------------------------------------
# The property: a backtest reads one tenant's history
# --------------------------------------------------------------------------


def test_the_fetch_returns_the_requesting_tenants_rows_and_none_of_the_others(seeded_lake: Any) -> None:
    """The reason ``rewrite_for_tenant`` is on this path.

    Run as A against a warehouse that holds B. A wrong predicate, a predicate
    the rewriter dropped, or an AST rename that turned the table walk into a
    no-op all fail here — which is what no offline assertion about the
    rendered string could do.
    """
    events = _fetch(TENANT_A, window_days=30, limit=1000)

    assert events, "the fetch returned nothing; a no-rows result cannot demonstrate scoping"
    assert _labels(events) == set(WITHIN_30_DAYS)
    # Asserted on the column as well as on the labels: the label is this
    # test's own convention, `tenant_id` is the isolation key itself.
    assert {str(event["tenant_id"]) for event in events} == {str(TENANT_A)}
    assert not (_labels(events) & B_LABELS), f"tenant B leaked into A's backtest: {sorted(_labels(events) & B_LABELS)}"


def test_the_other_tenant_reads_its_own_history_through_the_same_call(seeded_lake: Any) -> None:
    """Scoping must be a filter on the caller, not a filter to one tenant.

    A rewriter hard-wired to A would pass the test above and fail here.
    """
    events = _fetch(TENANT_B, window_days=30, limit=1000)

    assert events
    assert {str(event["tenant_id"]) for event in events} == {str(TENANT_B)}
    assert not (_labels(events) & A_LABELS)


def test_a_tenant_with_no_history_reads_nothing_rather_than_everything(seeded_lake: Any) -> None:
    """The fail-open shape, stated as its own assertion.

    An unscoped query returns every row for every caller, so the tenant with
    no rows is the one that most clearly distinguishes a working predicate
    from a missing one.
    """
    stranger = uuid.UUID("cccccccc-0000-4000-8000-00000000ba03")
    assert _fetch(stranger, window_days=365, limit=1000) == []


# --------------------------------------------------------------------------
# The clamps, measured against recorded rows
# --------------------------------------------------------------------------


def test_the_window_ceiling_holds_against_a_row_older_than_the_ceiling(seeded_lake: Any) -> None:
    """``window_days`` is clamped to 365, proven by a 400-day-old row.

    The precondition is asserted first: the row is in the table and the
    identical unscoped statement can reach it at a wide enough window. So its
    absence from the result is the clamp, not the seed.
    """
    reachable = _unscoped(seeded_lake, window_days=365, limit=1000)
    assert "a-ancient" not in reachable, "the 400-day row is inside a 365-day window; the ages in SEED are wrong"
    stored = seeded_lake.execute("SELECT count() FROM aisoc.raw_events WHERE user_name = 'a-ancient'")[0][0]
    assert stored == 1, "the row the ceiling is measured against is not in the warehouse"

    events = _fetch(TENANT_A, window_days=9999, limit=1000)

    assert _labels(events) == set(WITHIN_365_DAYS)
    assert "a-ancient" not in _labels(events), "window_days was not clamped: a row 400 days old came back"
    # And the clamp did not over-clamp: the 200-day row is outside the default
    # window and inside the ceiling, so it distinguishes 365 from 30.
    assert "a-mid" in _labels(events), "window_days clamped below its documented 365-day ceiling"


def test_the_window_floor_holds_against_a_two_day_old_row(seeded_lake: Any) -> None:
    """``window_days`` is clamped up to 1, so 0 is a one-day window."""
    events = _fetch(TENANT_A, window_days=0, limit=1000)

    assert _labels(events) == set(WITHIN_1_DAY)
    assert "a-recent" not in _labels(events), "a 2-day-old row came back from a window clamped to 1 day"
    assert events, "the floor clamped to zero days rather than to one"


def test_the_limit_bounds_the_rows_that_come_back(seeded_lake: Any) -> None:
    """``limit`` is honoured against real rows, not just rendered into SQL."""
    available = _fetch(TENANT_A, window_days=9999, limit=1000)
    assert len(available) > 2, "the limit assertion needs more available rows than the limit it applies"

    capped = _fetch(TENANT_A, window_days=9999, limit=2)
    assert len(capped) == 2, f"limit=2 returned {len(capped)} rows"
    # Still one tenant's rows: the limit must not be the only thing bounding
    # the result. A row cap on an unscoped query returns other tenants' rows.
    assert {str(event["tenant_id"]) for event in capped} == {str(TENANT_A)}
    # ORDER BY event_time DESC, so a cap takes the newest rows.
    assert _labels(capped) <= set(WITHIN_1_DAY)


def test_the_limit_floor_returns_one_row_rather_than_none(seeded_lake: Any) -> None:
    """``limit`` is clamped up to 1, so 0 is not "no rows"."""
    events = _fetch(TENANT_A, window_days=9999, limit=0)
    assert len(events) == 1
    assert str(events[0]["tenant_id"]) == str(TENANT_A)


# --------------------------------------------------------------------------
# The source filter and the column mapping
# --------------------------------------------------------------------------


def test_the_source_filter_selects_by_connector_against_recorded_rows(seeded_lake: Any) -> None:
    """The filter names a real column holding a real value.

    ``connector_type`` is written by the writer from ``metadata.product.name``,
    so a filter built against a different spelling returns nothing here rather
    than passing as a substring of generated SQL.
    """
    only_okta = _fetch(TENANT_A, window_days=30, limit=1000, source=OTHER_CONNECTOR)
    assert _labels(only_okta) == {"a-okta-fresh"}

    only_cloudtrail = _fetch(TENANT_A, window_days=30, limit=1000, source=CONNECTOR)
    assert _labels(only_cloudtrail) == set(WITHIN_30_DAYS) - {"a-okta-fresh"}
    assert only_cloudtrail, "the connector every seeded row carries matched nothing"

    # A sanitised-away source must not silently widen to every connector.
    assert _labels(_fetch(TENANT_A, window_days=30, limit=1000, source="!!!")) == set(WITHIN_30_DAYS)


def test_the_lake_payload_column_is_expanded_so_flat_field_rules_can_match(seeded_lake: Any) -> None:
    """``rows_to_events`` must expand the payload column the lake actually has.

    The writer stores the OCSF ``raw_data`` field into a column it names
    ``raw_payload``, and ``aisoc.raw_events`` has no ``raw_data`` column at
    all. The expansion list named only the OCSF-side spellings, so on a real
    lake row it expanded nothing and every rule matching a connector-flat
    field scanned events that did not carry it. The unit test could not see
    this because it supplied its own column list, and the list named a column
    the warehouse does not have.
    """
    events = _fetch(TENANT_A, window_days=30, limit=1000)
    assert events

    columns_from_the_warehouse = set(events[0])
    assert "raw_payload" in columns_from_the_warehouse, "the lake stopped returning the payload column this asserts about"
    assert "raw_data" not in columns_from_the_warehouse, "aisoc.raw_events grew a raw_data column; the expansion list should follow"

    fresh = next(event for event in events if event["user_name"] == "a-fresh-1")
    assert fresh.get(PAYLOAD_MARKER_FIELD) == PAYLOAD_EVENT_NAME, (
        f"{PAYLOAD_MARKER_FIELD!r} is not a column of aisoc.raw_events, so it can only be present if the "
        f"payload column was expanded. It was not, which means a rule matching a connector-flat field "
        f"cannot fire in a backtest."
    )
    # The expansion must not cost the OCSF columns beside it.
    assert fresh["connector_type"] == CONNECTOR
    assert str(fresh["tenant_id"]) == str(TENANT_A)


def test_an_expanded_payload_does_not_carry_another_tenants_row(seeded_lake: Any) -> None:
    """Expansion happens after scoping, so it cannot reintroduce B.

    Stated separately because the expansion merges attacker-influenced keys
    over the OCSF columns, and ``tenant_id`` is one of the keys it could
    overwrite.
    """
    events = _fetch(TENANT_A, window_days=9999, limit=1000)
    assert {event["label"] for event in events} == set(WITHIN_365_DAYS)
    assert not ({event["label"] for event in events} & B_LABELS)


# --------------------------------------------------------------------------
# One implementation, so this proof covers both backtest endpoints
# --------------------------------------------------------------------------


def _lake_fetch_definitions() -> dict[str, list[str]]:
    """Every ``*fetch_lake_events`` definition in the API service, by file.

    Read off the syntax tree rather than imported, because the endpoint
    modules pull in FastAPI and SQLAlchemy and this job installs neither. The
    direction matters: the check asks the tree what exists, so a second copy
    reappearing is what fails it.
    """
    found: dict[str, list[str]] = {}
    for path in sorted((REPO_ROOT / "services/api/app").rglob("*.py")):
        tree = ast.parse(path.read_text(encoding="utf-8"))
        names = [
            node.name
            for node in ast.walk(tree)
            if isinstance(node, ast.AsyncFunctionDef | ast.FunctionDef) and node.name.endswith("fetch_lake_events")
        ]
        if names:
            found[path.relative_to(REPO_ROOT).as_posix()] = names
    return found


def test_one_lake_fetch_serves_every_backtest_endpoint() -> None:
    """The duplicate this file was written to cover, held deleted.

    ``detection_rules.py`` carried its own near-identical ``_fetch_lake_events``
    with its own inline rewriter and ``execute_lake_query`` call, so the
    tenant-scoping path existed twice and neither copy was covered. Proving
    one implementation and letting both endpoints import it is what makes the
    live proof above cover the rule endpoint as well as the proposal one — so
    a copy coming back has to fail rather than quietly go unproven.
    """
    definitions = _lake_fetch_definitions()
    assert definitions == {"services/api/app/services/backtest.py": ["fetch_lake_events"]}, (
        f"expected exactly one lake-fetch implementation; found {definitions}. "
        f"A second copy is a second tenant-scoping path, and this file only proves one."
    )


@pytest.mark.parametrize(
    "endpoint",
    [
        "services/api/app/api/v1/endpoints/detection_rules.py",
        "services/api/app/api/v1/endpoints/detection_proposals.py",
    ],
)
def test_each_backtest_endpoint_imports_the_shared_fetch(endpoint: str) -> None:
    """Named per endpoint, so a failure says which one stopped sharing."""
    tree = ast.parse((REPO_ROOT / endpoint).read_text(encoding="utf-8"))
    imported = {
        alias.name
        for node in ast.walk(tree)
        if isinstance(node, ast.ImportFrom) and node.module == "app.services.backtest"
        for alias in node.names
    }
    assert "fetch_lake_events" in imported, f"{endpoint} does not import the shared lake fetch; it has its own path again"