Skip to content

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

24.8 KB
Raw Blame
"""ASGI middleware ORDER and SECURITY-HEADER invariants (Flask -> FastAPI).

Flask had no equivalent of this: Flask's ``before_request``/``after_request``
hooks and its ``app.wsgi_app`` wrapping have no "outer vs inner" ordering
puzzle — every ``after_request`` handler always sees every response.
Starlette/FastAPI middleware is fundamentally different: ``app.add_middleware``
is **LIFO** — the *last* call wins the outermost slot — so the plain,
top-to-bottom reading order of the ``app.add_middleware(...)`` calls in
``web/fastapi_app.py`` is the exact *reverse* of the order requests/responses
actually pass through them. Swap two calls, or insert a new one on the wrong
side of an existing one, and the code still imports and runs — it just
quietly stops enforcing what its authors intended (e.g. CSRF checked before
the session that CSRF reads from is populated, or a whole class of error
response silently losing its security headers). Nothing but a test that
reads the *live* middleware chain and asserts specific behavioural
consequences of the order can catch that.

This file has four parts:

1. ``TestRegistrationOrderPinned`` — pins the exact ``app.user_middleware``
   sequence (Starlette's own bookkeeping list) AND cross-checks it against
   the real, built ASGI call chain. Each pinned entry documents what a
   reorder would silently break.
2. ``TestEffectiveExecutionOrderBehavioral`` — proves two of those ordering
   claims through real HTTP requests rather than just list inspection: a
   CSRF-rejected response still carries security headers (SecurityHeaders is
   outside CSRF), and CSRF validates against a session token that
   SessionMiddleware — not CSRF itself — populated from the request cookie
   (Session is outside CSRF).
3. ``TestSecurityHeadersOnEveryResponseClass`` — the known-gap surface: a
   200 can carry every security header while an error path quietly drops
   them. Six response classes are driven through the REAL app (full
   middleware stack, no mocks): 200, 404, 405, 422, CSRF-403, and an
   unhandled/unregistered exception -> 500.

   *** REAL FINDING (see ``TestSecurityHeadersOnEveryResponseClass`` /
   ``test_KNOWN_GAP_...`` below): the 500 produced by an exception type
   with no registered handler carries **NONE** of the security headers.
   Starlette's ``ServerErrorMiddleware`` is the true outermost layer
   (added automatically, outside every ``app.add_middleware`` call) and,
   on an uncaught exception, sends its fallback response directly on the
   *raw* ASGI ``send`` it was itself given — bypassing every layer this
   app registered, ``SecurityHeadersMiddleware`` included. This is
   PINNED, not swept under a weaker assertion — see that test's docstring
   for the exact mechanism and what fixing it would require.

4. ``TestServerHeaderNotLeaked`` — confirms (having read the code first)
   that ``SecurityHeadersMiddleware`` unconditionally strips any inbound
   ``Server`` header before a response leaves the app, and proves it
   through the full real stack (not just the isolated middleware class).
"""

import pytest
from fastapi.testclient import TestClient
from slowapi.middleware import SlowAPIMiddleware
from starlette.middleware.sessions import SessionMiddleware
from starlette.responses import PlainTextResponse

from local_deep_research.web.dependencies.csrf import CSRFMiddleware
from local_deep_research.web.fastapi_app import (
    BodySizeLimitMiddleware,
    DatabaseMiddleware,
    RememberMeMiddleware,
    SecureCookieMiddleware,
    SecurityHeadersMiddleware,
    app,
)

# ---------------------------------------------------------------------------
# Test-only routes, registered ONCE on the live singleton `app` at collection
# time (mirrors tests/web/routers/test_route_ordering.py's bare
# `from ...fastapi_app import app`). They run through the exact same,
# already-built middleware stack as every production route — nothing about
# CSRF/Session/SecurityHeaders/etc. is re-created or mocked for these tests.
# Distinctive `/__mw_order_probe__/...` prefix: no production route starts
# with it, and (verified against test_route_ordering.py's own check) it
# cannot be shadowed by / cannot shadow any real route — the only
# parameterized route in the app is `/static/{path:path}`.
# ---------------------------------------------------------------------------


class _ProbeUnregisteredError(RuntimeError):
    """Deliberately NOT one of the types `_register_exception_handlers`
    binds: not HTTPException, WebAPIException, NewsAPIException,
    PolicyDeniedError, or json.JSONDecodeError. It only matches the bare
    ``Exception`` handler — which Starlette's `Starlette.build_middleware_stack`
    pulls out of `exception_handlers` and hands to `ServerErrorMiddleware`
    as its `handler`, NOT to `ExceptionMiddleware`. That distinction is the
    whole mechanism `test_KNOWN_GAP_headers_missing_on_unregistered_exception`
    documents below.
    """


@app.get("/__mw_order_probe__/boom", include_in_schema=False)
async def _mw_order_probe_boom():
    raise _ProbeUnregisteredError("probe: unregistered exception type")


@app.get("/__mw_order_probe__/int-check", include_in_schema=False)
async def _mw_order_probe_int_check(n: int):
    return {"n": n}


@app.get("/__mw_order_probe__/leaky-server-header", include_in_schema=False)
async def _mw_order_probe_leaky_server_header():
    # A handler that (accidentally or otherwise) sets its own Server
    # header, the way an unpinned/legacy dependency might.
    return PlainTextResponse(
        "ok", headers={"Server": "LDR-internal/9.9.9-do-not-leak"}
    )


# ---------------------------------------------------------------------------
# Shared helpers
# ---------------------------------------------------------------------------

# The 8 headers SecurityHeadersMiddleware stamps on EVERY http response
# unconditionally (HSTS is scheme-conditional and cache-control/pragma are
# path-conditional — see the middleware source — so they're deliberately
# excluded from this "must always be present" set; exact values for all of
# them are pinned separately in tests/web/test_security_headers.py).
UNCONDITIONAL_SECURITY_HEADERS = (
    "content-security-policy",
    "x-frame-options",
    "x-content-type-options",
    "cross-origin-opener-policy",
    "cross-origin-embedder-policy",
    "cross-origin-resource-policy",
    "permissions-policy",
    "referrer-policy",
)


@pytest.fixture
def client(app):
    """Plain TestClient on the `app` fixture (per-test data dir/env).

    Parameter name intentionally matches the module-level `app` import
    above (used only to register the probe routes at collection time) --
    both are the same singleton object; this just goes through
    conftest's fixture for per-test isolation like the rest of the suite.
    """
    return TestClient(app, raise_server_exceptions=False)


def _assert_all_security_headers_present(resp, label):
    missing = [
        h for h in UNCONDITIONAL_SECURITY_HEADERS if h not in resp.headers
    ]
    assert not missing, f"{label}: missing security headers {missing!r}"


# ---------------------------------------------------------------------------
# 1. Registration order is pinned
# ---------------------------------------------------------------------------

# Outer -> inner. `app.add_middleware` PREPENDS to `app.user_middleware`
# (Starlette: `self.user_middleware.insert(0, Middleware(cls, ...))`), so the
# LAST-registered middleware is index 0 = OUTERMOST = sees the request
# first / the response last. `fastapi_app.py` registers, top-to-bottom:
# Database, CSRF, Session, RememberMe, BodySizeLimit, SecurityHeaders,
# SecureCookie (all in the main setup block), then SlowAPIMiddleware last of
# all, from `_setup_rate_limiting()` called after that block. So the actual
# outer->inner order is that list reversed, with SlowAPI prepended in front
# of everything else. Each entry below documents what breaks if it moves.
EXPECTED_ORDER_OUTER_TO_INNER = [
    (
        SlowAPIMiddleware,
        "Global rate limiting (slowapi). Registered LAST of all "
        "(`_setup_rate_limiting`, called after every other "
        "`app.add_middleware`), so it is the outermost layer. If it moved "
        "inward of DatabaseMiddleware, a request that will be rejected for "
        "being rate-limited would open a user DB connection first — wasted "
        "work on the exact path meant to shed load.",
    ),
    (
        SecureCookieMiddleware,
        "Stamps `; Secure` onto outbound Set-Cookie headers when the "
        "connection is HTTPS. Must stay OUTSIDE SessionMiddleware and "
        "RememberMeMiddleware so it can see (and rewrite) the Set-Cookie "
        "headers THEY just wrote on this response. If it moved inside "
        "Session, the session cookie would ship over HTTPS without the "
        "Secure flag.",
    ),
    (
        SecurityHeadersMiddleware,
        "Stamps CSP / X-Frame-Options / etc. Must stay outside CSRF, "
        "Session, RememberMe, BodySizeLimit and Database so EVERY response "
        "produced by any of them — including a CSRF-rejected 403 and a "
        "body-too-large 413 — still carries the headers on its way out. "
        "If it moved inside CSRFMiddleware, a CSRF rejection (which "
        "returns straight from CSRFMiddleware without reaching the "
        "router) would carry no security headers at all — see "
        "`TestEffectiveExecutionOrderBehavioral` below for the passing "
        "proof that today it does not regress this way.",
    ),
    (
        BodySizeLimitMiddleware,
        "Global request-body size cap. Must stay outside Session/CSRF "
        "(so an oversized body is rejected before CSRFMiddleware buffers "
        "a urlencoded form body into memory to read the token out of it) "
        "but inside SecurityHeaders (so its own 413 responses still carry "
        "them).",
    ),
    (
        RememberMeMiddleware,
        "Strips Max-Age/Expires off the session cookie when 'remember "
        "me' was unchecked. Must stay just OUTSIDE SessionMiddleware so "
        "it observes the Set-Cookie header Session just wrote on this "
        "very response. If it moved inside Session, it would never see a "
        "session Set-Cookie to rewrite and every login would get the "
        "full 30-day persistent cookie regardless of the checkbox.",
    ),
    (
        SessionMiddleware,
        "Decodes the signed cookie into `scope['session']`. Must stay "
        "OUTSIDE CSRFMiddleware and DatabaseMiddleware, both of which "
        "read `scope['session']`. If it moved inside CSRF, CSRF would "
        "always observe an empty session and reject every state-changing "
        "request with 'CSRF token missing', valid cookie or not — see "
        "`test_csrf_validates_against_a_session_populated_by_session_"
        "middleware` below for the passing proof that today it does not.",
    ),
    (
        CSRFMiddleware,
        "Rejects state-changing requests with a missing/invalid "
        "X-CSRFToken before they reach the router. Must stay OUTSIDE "
        "DatabaseMiddleware so a forged/CSRF-less request is rejected "
        "before a per-user encrypted DB connection is opened for it.",
    ),
    (
        DatabaseMiddleware,
        "Opens the per-user encrypted DB connection ('Runs INSIDE "
        "SessionMiddleware so request.session is available' — its own "
        "docstring). Innermost of the app's own middleware by design: "
        "every other layer gets to reject the request first without "
        "paying for a DB open.",
    ),
]


class TestRegistrationOrderPinned:
    def test_user_middleware_exact_sequence(self, app):
        """`app.user_middleware`, outer -> inner, must match EXACTLY.

        This is Starlette's own bookkeeping list, populated purely by the
        `add_middleware` calls in `fastapi_app.py`. A reorder, an
        insertion on the wrong side, or a removal anywhere in that file's
        middleware block changes this list and fails here. See the
        per-entry reasons above for what each specific move would
        silently break — this test only pins the observable *shape*.
        """
        actual = [m.cls for m in app.user_middleware]
        expected = [cls for cls, _reason in EXPECTED_ORDER_OUTER_TO_INNER]
        assert actual == expected, (
            "Middleware registration order changed.\n"
            f"  expected (outer->inner): {[c.__name__ for c in expected]}\n"
            f"  actual   (outer->inner): {[c.__name__ for c in actual]}"
        )

    def test_built_asgi_chain_walks_through_them_in_the_same_order(self, app):
        """Cross-check against the REAL, constructed ASGI call chain.

        `app.user_middleware` is a bookkeeping list `add_middleware`
        maintains; this walks `.app` on the actual object graph
        `build_middleware_stack()` wires up, to prove the two agree (and
        to catch a future Starlette/FastAPI internal change that decouples
        them). Starlette inserts its own `ServerErrorMiddleware` /
        `ExceptionMiddleware` / etc. around and between the app's own
        middleware, so this asserts our classes appear as a
        strictly-increasing subsequence, not full list equality.
        """
        layer = app.middleware_stack
        if layer is None:
            layer = app.build_middleware_stack()

        walked = []
        seen_ids = set()
        while layer is not None and id(layer) not in seen_ids:
            seen_ids.add(id(layer))
            walked.append(type(layer))
            layer = getattr(layer, "app", None)

        expected_classes = [
            cls for cls, _reason in EXPECTED_ORDER_OUTER_TO_INNER
        ]
        try:
            positions = [walked.index(cls) for cls in expected_classes]
        except ValueError as exc:
            pytest.fail(
                f"expected middleware class missing from the built ASGI "
                f"chain: {exc}\nchain was: {[c.__name__ for c in walked]}"
            )

        assert positions == sorted(positions), (
            "Built ASGI chain does not visit our middleware in the "
            f"expected outer->inner order.\n"
            f"  expected order: {[c.__name__ for c in expected_classes]}\n"
            f"  full chain:     {[c.__name__ for c in walked]}"
        )


# ---------------------------------------------------------------------------
# 2. Effective execution order matches intent (behavioural proof)
# ---------------------------------------------------------------------------


class TestEffectiveExecutionOrderBehavioral:
    """Prove two of the ordering claims above through real requests, not
    just structural list inspection. `/auth/logout` is a real, unmodified
    production route (POST-only, CSRF-guarded, safe to call anonymously —
    it no-ops when there is no `username` in the session) so this exercises
    the actual production wiring end to end."""

    def test_csrf_rejection_still_carries_security_headers(self, client):
        """SecurityHeadersMiddleware is OUTSIDE CSRFMiddleware.

        A CSRF-rejected request is answered straight from
        `CSRFMiddleware.__call__` (it calls `response(scope, receive,
        send)` and returns without ever reaching the router) — but that
        `send` it calls is the *wrapped* send handed down from every
        middleware registered outside it, so the response still climbs
        back out through BodySizeLimit -> SecurityHeaders -> SecureCookie
        on its way to the client. If SecurityHeadersMiddleware were
        registered on the wrong side of CSRF (inside it), this response
        would carry none of them.
        """
        resp = client.post("/auth/logout")  # no session, no X-CSRFToken
        assert resp.status_code == 403
        assert resp.json()["error"].startswith("CSRF token missing")
        _assert_all_security_headers_present(resp, "CSRF-rejected 403")

    def test_csrf_validates_against_a_session_populated_by_session_middleware(
        self, client
    ):
        """SessionMiddleware is OUTSIDE CSRFMiddleware.

        `CSRFMiddleware` reads `scope['session']` — a key only
        `SessionMiddleware` ever sets, by decoding the incoming cookie
        BEFORE calling further into the stack. If CSRF were registered
        outside Session, `scope['session']` would never be populated by
        the time CSRF runs, so a request carrying a perfectly valid,
        previously-issued session cookie AND the matching CSRF token
        would still be rejected with "CSRF token missing" — CSRF would
        always see an empty session, request after request.

        This drives that exact path: mint a real session + token via the
        real `/auth/csrf-token` route (GET, CSRF-exempt), then present
        both back on a real CSRF-guarded POST and confirm it is NOT
        rejected for a missing session token.
        """
        no_cookie_yet = client.post("/auth/logout")
        assert no_cookie_yet.status_code == 403
        assert "missing" in no_cookie_yet.json()["error"]

        token = client.get("/auth/csrf-token").json()["csrf_token"]
        assert token

        authorized = client.post(
            "/auth/logout",
            headers={"X-CSRFToken": token},
            follow_redirects=False,
        )
        # Not the CSRF 403 anymore — the real route ran (redirect to
        # login is /auth/logout's normal, no-op-when-anonymous response).
        assert authorized.status_code != 403, (
            "CSRF rejected a request carrying the session's own, "
            "just-issued token — SessionMiddleware may not be running "
            "before CSRFMiddleware anymore"
        )
        assert authorized.status_code == 302
        assert authorized.headers["location"] == "/auth/login"


# ---------------------------------------------------------------------------
# 3. Security headers on EVERY response class
# ---------------------------------------------------------------------------


class TestSecurityHeadersOnEveryResponseClass:
    """Six response classes, all driven through the real app / real
    middleware stack (`raise_server_exceptions=False` so a 500 comes back
    as a Response instead of re-raising into the test)."""

    def test_200_has_security_headers(self, client):
        resp = client.get("/api/v1/health")
        assert resp.status_code == 200
        _assert_all_security_headers_present(resp, "200 (health)")

    def test_404_has_security_headers(self, client):
        resp = client.get("/this-route-does-not-exist-mw-order-test")
        assert resp.status_code == 404
        _assert_all_security_headers_present(resp, "404 (unrouted path)")

    def test_405_has_security_headers(self, client):
        # /auth/logout is POST-only; GET is CSRF-safe so this reaches
        # routing and gets Starlette's routing-raised 405.
        resp = client.get("/auth/logout")
        assert resp.status_code == 405
        _assert_all_security_headers_present(resp, "405 (method not allowed)")

    def test_422_has_security_headers(self, client):
        resp = client.get("/__mw_order_probe__/int-check?n=not-an-int")
        assert resp.status_code == 422
        _assert_all_security_headers_present(resp, "422 (validation error)")

    def test_csrf_403_has_security_headers(self, client):
        resp = client.post("/auth/logout")
        assert resp.status_code == 403
        _assert_all_security_headers_present(resp, "403 (CSRF rejection)")

    def test_headers_present_on_unregistered_exception_500(self, client):
        """Security headers survive the ServerErrorMiddleware bypass.

        This response class reaches the client WITHOUT passing through
        SecurityHeadersMiddleware, so it is the one path where the headers
        have to be stamped by hand. The bypass is structural, not
        incidental:

        `_register_exception_handlers` registers `@app.exception_handler
        (Exception)` as a catch-all so unhandled exceptions get a clean,
        scrubbed JSON 500 instead of a bare traceback (see
        `tests/web/test_exception_handler_contract.py::Test500Contract`,
        which pins that *body* contract on a minimal app with no
        middleware). But Starlette's `Starlette.build_middleware_stack`
        special-cases exactly that registration:

            for key, value in self.exception_handlers.items():
                if key in (500, Exception):
                    error_handler = value          # -> ServerErrorMiddleware
                else:
                    exception_handlers[key] = value  # -> ExceptionMiddleware

        A handler registered for the literal `Exception` class is pulled
        OUT of the dict `ExceptionMiddleware` uses and wired instead as
        `ServerErrorMiddleware`'s own `handler`. `ServerErrorMiddleware`
        is added automatically by Starlette OUTSIDE every
        `app.add_middleware(...)` call this app makes — it is the true
        outermost layer, ahead of even `SlowAPIMiddleware`.

        When an exception type with no specific handler (not
        HTTPException, WebAPIException, NewsAPIException,
        PolicyDeniedError, or json.JSONDecodeError — this probe route
        raises a locally-defined `_ProbeUnregisteredError(RuntimeError)`
        that matches none of them) propagates out of the router, it
        skips `ExceptionMiddleware` (not in its dict) and unwinds as a
        live Python exception through every one of this app's own
        middleware — none of which wraps its call to the next layer in a
        `try/except`, so none of their response-rewriting `send`
        wrappers ever run. It is only caught at `ServerErrorMiddleware`,
        which builds the response and sends it on the RAW `send` IT was
        given — the one from the ASGI server, never wrapped by
        SecurityHeadersMiddleware, SecureCookieMiddleware,
        BodySizeLimitMiddleware, RememberMeMiddleware, SessionMiddleware,
        CSRFMiddleware, or DatabaseMiddleware. All of it is bypassed.

        This survey originally found the 500 shipping with NO security
        headers at all. Nothing can be registered outside Starlette's own
        outermost wrapper via `app.add_middleware`, but the fix does not
        need that: the handler Starlette wires INTO
        `ServerErrorMiddleware` is the app's own catch-all, so it stamps
        the headers on its own response. It reuses
        `SecurityHeadersMiddleware.unconditional_headers()` so the
        middleware and the fallback path cannot drift apart.

        Only the unconditional set is asserted. HSTS is scheme-dependent
        and cache-control is skipped for /static/, so neither belongs on
        this path.
        """
        resp = client.get("/__mw_order_probe__/boom")
        assert resp.status_code == 500
        assert resp.json() == {"error": "Server error"}

        missing = [
            h for h in UNCONDITIONAL_SECURITY_HEADERS if h not in resp.headers
        ]
        assert missing == [], (
            "Security headers missing on the unregistered-exception 500 "
            f"path: {missing!r}. This response bypasses "
            "SecurityHeadersMiddleware entirely (see above), so it relies "
            "on the catch-all exception handler in fastapi_app.py stamping "
            "them itself via "
            "SecurityHeadersMiddleware.unconditional_headers()."
        )
        assert "server" not in resp.headers


# ---------------------------------------------------------------------------
# 4. Server header does not leak version info
# ---------------------------------------------------------------------------


class TestServerHeaderNotLeaked:
    """Verified first, by reading `SecurityHeadersMiddleware.__call__`
    (web/fastapi_app.py): it unconditionally drops any inbound `server`
    header --

        headers = [(k, v) for k, v in headers if k.lower() != b"server"]

    -- and never adds a replacement, on every response, unconditionally.
    tests/web/test_security_headers.py::TestServerHeaderStripped already
    proves this against the isolated middleware class wrapping a
    synthetic ASGI app; these prove it end-to-end through the real,
    fully-wired app on live response classes.
    """

    def test_no_server_header_on_normal_response(self, client):
        resp = client.get("/api/v1/health")
        assert "server" not in resp.headers

    def test_no_server_header_on_error_response(self, client):
        resp = client.get("/this-route-does-not-exist-mw-order-test")
        assert "server" not in resp.headers

    def test_handler_set_server_header_is_stripped_end_to_end(self, client):
        """A route handler that sets its OWN `Server` header (e.g. an
        unpinned dependency doing it for you) must still not leak past
        SecurityHeadersMiddleware -- through the real stack, not a
        synthetic inner app."""
        resp = client.get("/__mw_order_probe__/leaky-server-header")
        assert resp.status_code == 200
        assert resp.text == "ok"
        assert "server" not in resp.headers
        for value in resp.headers.values():
            assert "9.9.9" not in value
            assert "LDR-internal" not in value