Skip to content

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

18.7 KB
Raw Blame
"""Does socket teardown actually SEVER the socket -- on the real wire?

WHAT THIS FILE PINS
-------------------
``socketio_asgi._disconnect_matching`` is the single implementation behind
both teardown entry points:

* ``disconnect_session(session_id)`` -- logout
  (``web/routers/auth.py::_disconnect_session_sockets``), and
* ``disconnect_user(username)`` -- **password change**
  (``web/routers/auth.py::_disconnect_user_sockets``) and the idle
  connection sweep (``web/auth/connection_cleanup.py``).

It schedules a coroutine onto the app's event loop and returns whether it
could *schedule*, not whether anything was disconnected::

    for sid in sids:
        try:
            await sio.disconnect(sid)
        except Exception:
            logger.debug(f"Error disconnecting {sid}")      # swallowed
    if sids:
        logger.info(f"Disconnected {len(sids)} socket(s) ...")  # counts FOUND
    ...
    asyncio.run_coroutine_threadsafe(_disconnect(), loop)
    return True                                              # at SCHEDULE time

No call site checks the return value, so the failure is silent by
construction. The security question that matters is not the return value
though -- it is whether a socket whose ``sio.disconnect`` failed STAYS
connected and keeps receiving that user's events. ``emit_to_user`` fans
out over ``_sid_users``, and the event it exists for is
``settings_changed``, which this module's own comments (l.667, l.676)
describe as carrying "plaintext secrets". A password change that leaves
such a socket alive hands the *old* credential holder a live feed.

WHY THE REAL TRANSPORT
----------------------
The sid maps are NOT cleaned up by ``_disconnect_matching``. They are
cleaned up by the ``disconnect`` EVENT handler (``socketio_asgi.py``
l.408-419), which python-socketio fires from inside a *successful*
``sio.disconnect``. So a test that stubs ``sio.disconnect`` with a
harmless recorder proves nothing: its "success" case never fires the
handler either, so "maps still populated" is true in both arms and the
control is vacuous.

Every test here therefore drives the **real Engine.IO v4 polling
protocol** against the **real ASGI app**, reusing the harness of
``tests/security/test_realtime_channel_isolation.py``: real
register/login, real handshake, real packets read back off the wire,
``sio.emit`` never mocked. The only thing ever patched is
``sio.disconnect`` itself, in the tests that are specifically about it
failing.

ANTI-VACUITY
------------
"The socket received nothing" is free if the socket was never able to
receive. Every such assertion in this file is preceded, on the SAME
socket and through the SAME ``emit_to_user`` call path, by a delivery
that was actually read off that socket's own poll with a unique marker.
"""

from __future__ import annotations

import json
import uuid

import pytest
from fastapi.testclient import TestClient

from local_deep_research.web.services import socketio_asgi

# Harness shared with the sibling real-transport file. Importing rather
# than re-implementing keeps the two files on one definition of "a real
# socket": if the handshake recipe changes, both move together.
from tests.security.test_realtime_channel_isolation import (  # noqa: E501
    _EIO_SEP,
    EioSession,
    _new_client,
    _register_and_login,
    _seed_research,
    _wait_until,
)

# ``_isolated_socketio_state`` is autouse in the imported module, but
# autouse fixtures only apply inside the module that defines them, so it
# is re-declared for this file below.


@pytest.fixture(autouse=True)
def _isolated_socketio_state():
    """Snapshot/restore socketio_asgi's module globals.

    ``_sid_users`` / ``_sid_sessions`` / ``_subscriptions`` / ``_lock`` /
    ``_main_loop`` are module-level and shared with every other socket
    test in the run, and ``_lock`` must not be left bound to this file's
    (now-closed) TestClient portal loop. Same contract as the fixture of
    the same name in ``test_realtime_channel_isolation.py``.
    """
    saved = (
        dict(socketio_asgi._sid_users),
        dict(socketio_asgi._sid_sessions),
        {k: set(v) for k, v in socketio_asgi._subscriptions.items()},
        socketio_asgi._lock,
        socketio_asgi._main_loop,
    )
    socketio_asgi._sid_users.clear()
    socketio_asgi._sid_sessions.clear()
    socketio_asgi._subscriptions.clear()
    socketio_asgi._lock = None
    yield
    users, sessions, subs, lock, loop = saved
    socketio_asgi._sid_users.clear()
    socketio_asgi._sid_users.update(users)
    socketio_asgi._sid_sessions.clear()
    socketio_asgi._sid_sessions.update(sessions)
    socketio_asgi._subscriptions.clear()
    socketio_asgi._subscriptions.update(subs)
    socketio_asgi._lock = lock
    socketio_asgi._main_loop = loop


class _LiveUser:
    """A registered user plus their signed session cookie.

    Exists mainly for ``__repr__``: pytest prints every fixture value in a
    failure traceback, and these tests are *expected* to fail (they are
    strict xfails), so a bare tuple would paste a live session cookie into
    CI output on every run. The value is redacted here instead.
    """

    __slots__ = ("cookie", "name")

    def __init__(self, name: str, cookie: str):
        self.name = name
        self.cookie = cookie

    def __repr__(self) -> str:
        return f"_LiveUser(name={self.name!r}, cookie=<redacted>)"


@pytest.fixture
def live_user(app) -> _LiveUser:
    """One real registered + logged-in user with their own SQLCipher DB."""
    name = f"tds_{uuid.uuid4().hex[:10]}"
    client = _new_client(app)
    _register_and_login(client, name)
    cookie = client.cookies.get("session")
    assert cookie, "no session cookie after login"
    return _LiveUser(name, cookie)


# ---------------------------------------------------------------------------
# Wire helpers
# ---------------------------------------------------------------------------


def _poll_tolerant(sess: EioSession) -> tuple[int, str]:
    """One long-poll that does NOT assert a 200.

    After a genuine disconnect the Engine.IO session is gone and the next
    poll answers 400 ``Invalid session``. That is a legitimate -- indeed
    the expected -- observation for a severed socket, so the negative
    assertions must be able to see it instead of erroring out inside
    ``EioSession.poll_raw``.
    """
    r = sess.client.get(
        f"/ws/socket.io/?EIO=4&transport=polling&sid={sess.eio_sid}"
    )
    return r.status_code, r.text


def _event_frames(text: str) -> list:
    """Decode the ``42`` (socket.io EVENT) frames out of a poll body."""
    frames = []
    for packet in text.split(_EIO_SEP):
        if packet.startswith("42"):
            frames.append(json.loads(packet[2:]))
    return frames


def _open_socket(ws_client: TestClient, cookie: str) -> EioSession:
    sess = EioSession(ws_client, cookie)
    ack = sess.open()
    assert ack.startswith("40"), f"handshake was refused: {ack!r}"
    assert sess.sio_sid, f"no socket.io sid in ack: {ack!r}"
    return sess


def _probe_delivery(sess: EioSession, username: str) -> tuple[str, str]:
    """Emit a uniquely marked ``settings_changed`` to ``username`` and poll.

    Returns ``(marker, raw_poll_body)``. Uses ``emit_to_user`` -- the real
    fan-out over ``_sid_users`` that ``settings_changed`` uses in
    production -- so a socket that survives teardown is probed through
    exactly the path that would leak its owner's settings payload.
    """
    marker = f"teardown-probe-{uuid.uuid4().hex}"
    socketio_asgi.emit_to_user("settings_changed", username, {"marker": marker})
    _status, body = _poll_tolerant(sess)
    return marker, body


def _subscribed_key(username: str, rid: str) -> tuple:
    return socketio_asgi._subscription_key(username, rid)


# ---------------------------------------------------------------------------
# 1. The control the whole file rests on: teardown, working normally,
#    genuinely severs a real socket.
# ---------------------------------------------------------------------------


def test_working_teardown_severs_real_socket_and_stops_delivery(app, live_user):
    """``disconnect_user`` with nothing patched must actually cut the wire.

    Four steps, in this order, so nothing below can pass vacuously:

    1. CONTROL -- the socket is live and its receive channel delivers: a
       ``settings_changed`` emitted through the real ``emit_to_user``
       fan-out comes back off this socket's own poll carrying a unique
       marker.
    2. Teardown is invoked exactly as ``_disconnect_user_sockets`` (the
       password-change path) invokes it.
    3. The sid leaves ``_sid_users`` / ``_sid_sessions`` /
       ``_subscriptions`` -- proving the real ``disconnect`` event handler
       ran, which is the ONLY thing that clears those maps.
    4. A second ``settings_changed``, same call path, same socket, does
       not reach it.
    """
    username, cookie = live_user.name, live_user.cookie
    rid = _seed_research(username)

    with TestClient(app, raise_server_exceptions=False) as ws_client:
        assert socketio_asgi._lock is not None, (
            "lifespan did not run init_lock(); the socket handlers would "
            "fail on `async with _lock` and every assertion below would be "
            "about the wrong thing"
        )
        sess = _open_socket(ws_client, cookie)
        sid = sess.sio_sid
        assert socketio_asgi._sid_users.get(sid) == username
        assert socketio_asgi._sid_sessions.get(sid)

        sess.emit("subscribe_to_research", {"research_id": rid})
        assert _wait_until(
            lambda: (
                sid
                in socketio_asgi._subscriptions.get(
                    _subscribed_key(username, rid), set()
                )
            )
        ), (
            "the subscribe never landed; the subscription-cleanup "
            f"assertion would be vacuous. _subscriptions="
            f"{socketio_asgi._subscriptions!r}"
        )

        # --- (1) CONTROL: this socket really receives.
        marker_before, body_before = _probe_delivery(sess, username)
        events_before = _event_frames(body_before)
        assert events_before, (
            "the socket received NOTHING before teardown -- its receive "
            "channel is not proven live, so the silence asserted in (4) "
            f"would be meaningless. poll body={body_before!r}"
        )
        assert events_before[0][0] == "settings_changed", events_before
        assert events_before[0][1]["marker"] == marker_before, events_before

        # --- (2) real teardown, exactly as the password-change path calls it.
        scheduled = socketio_asgi.disconnect_user(username)
        assert scheduled is True, (
            "disconnect_user could not even schedule; the test would be "
            "asserting on a teardown that never ran"
        )

        # --- (3) the real `disconnect` event handler ran and cleaned up.
        assert _wait_until(lambda: sid not in socketio_asgi._sid_users), (
            f"sid {sid} still in _sid_users after a healthy teardown; "
            f"_sid_users={socketio_asgi._sid_users!r}"
        )
        assert sid not in socketio_asgi._sid_sessions, (
            f"sid {sid} still in _sid_sessions: {socketio_asgi._sid_sessions!r}"
        )
        assert sid not in socketio_asgi._subscriptions.get(
            _subscribed_key(username, rid), set()
        ), (
            f"sid {sid} still subscribed after teardown: "
            f"{socketio_asgi._subscriptions!r}"
        )

        # --- (4) and nothing reaches it any more.
        marker_after, body_after = _probe_delivery(sess, username)
        leaked = _event_frames(body_after)
        assert leaked == [], (
            f"a severed socket still received event frames: {leaked!r}"
        )
        assert marker_after not in body_after, (
            f"the post-teardown marker reached a severed socket: {body_after!r}"
        )


# ---------------------------------------------------------------------------
# 2. The defect: teardown whose per-sid disconnect fails.
# ---------------------------------------------------------------------------


class _RaisingDisconnect:
    """Stand-in for ``sio.disconnect`` that records, then fails.

    Recording is what makes the failure OBSERVABLE from the test thread:
    ``_disconnect_matching`` swallows the exception at ``logger.debug`` and
    the coroutine runs on the app's event loop, so without this there is
    no signal at all that the teardown coroutine even reached the sid.
    """

    def __init__(self):
        self.sids: list[str] = []

    async def __call__(self, sid, *args, **kwargs):
        self.sids.append(sid)
        raise RuntimeError("simulated transport failure in sio.disconnect")


@pytest.mark.xfail(
    strict=True,
    reason=(
        "DEFECT: _disconnect_matching swallows a failing `await "
        "sio.disconnect(sid)` at logger.debug and moves on, and the sid "
        "maps are only ever cleaned by the `disconnect` EVENT handler that "
        "a successful sio.disconnect fires. So a socket whose disconnect "
        "failed stays in _sid_users and keeps receiving emit_to_user "
        "traffic -- including settings_changed, which carries plaintext "
        "secrets -- even though logout/password-change reported success. "
        "Suggested fix: on a per-sid disconnect failure, fall back to "
        "severing explicitly (e.g. sio.disconnect(sid, ignore_queue=True) "
        "then, still failing, pop the sid from _sid_users/_sid_sessions and "
        "discard it from _subscriptions and close the engine.io socket), "
        "and count only the sids actually severed in the INFO log."
    ),
)
def test_failed_teardown_must_not_leave_socket_receiving(
    app, live_user, monkeypatch
):
    """A teardown whose ``sio.disconnect`` fails must still sever.

    Same real socket, same real emit path as the control above; the only
    difference is that ``sio.disconnect`` raises. Assertion order is
    deliberate: the delivery check comes BEFORE the map check, so the
    failure this xfail pins is the security-relevant one (the socket is
    still being fed) rather than the bookkeeping one.
    """
    username, cookie = live_user.name, live_user.cookie

    with TestClient(app, raise_server_exceptions=False) as ws_client:
        assert socketio_asgi._lock is not None, "lifespan did not run"
        sess = _open_socket(ws_client, cookie)
        sid = sess.sio_sid

        # --- CONTROL: unpatched, this socket receives.
        marker_before, body_before = _probe_delivery(sess, username)
        events_before = _event_frames(body_before)
        assert events_before and events_before[0][1]["marker"] == (
            marker_before
        ), (
            "the socket received NOTHING before teardown; the delivery "
            f"assertion below would be about a dead channel. "
            f"poll body={body_before!r}"
        )

        # --- make the per-sid disconnect fail.
        failing = _RaisingDisconnect()
        monkeypatch.setattr(socketio_asgi.sio, "disconnect", failing)

        socketio_asgi.disconnect_user(username)

        # The teardown coroutine really reached OUR sid and really failed.
        assert _wait_until(lambda: sid in failing.sids), (
            "sio.disconnect was never called for this sid, so the teardown "
            "under test never ran; the assertions below would be vacuous. "
            f"calls={failing.sids!r}"
        )

        # --- THE SECURITY ASSERTION.
        marker_after, body_after = _probe_delivery(sess, username)
        leaked = _event_frames(body_after)
        assert leaked == [], (
            "a socket whose teardown FAILED is still receiving this user's "
            f"settings_changed traffic: {leaked!r}"
        )
        assert marker_after not in body_after, (
            f"post-teardown marker reached a supposedly disconnected "
            f"socket: {body_after!r}"
        )

        # --- and the bookkeeping.
        assert sid not in socketio_asgi._sid_users, (
            f"sid {sid} still registered after a failed teardown: "
            f"{socketio_asgi._sid_users!r}"
        )


@pytest.mark.xfail(
    strict=True,
    reason=(
        "DEFECT: _disconnect_matching returns True from the synchronous "
        "`asyncio.run_coroutine_threadsafe(...)` call -- i.e. at SCHEDULING "
        "time -- so it reports success even when every per-sid "
        "sio.disconnect raises. Suggested fix: have the scheduled coroutine "
        "report how many sids it actually severed (e.g. resolve the "
        "concurrent.futures.Future, or expose an async variant) and return "
        "False when any sid could not be disconnected; the INFO log should "
        "count severed sockets, not selected ones."
    ),
)
def test_failed_teardown_must_not_report_success(app, live_user, monkeypatch):
    """The return value must distinguish "torn down" from "scheduled".

    Control first: ``disconnect_user`` CAN return False and this test can
    observe it -- with no event loop available it does. Only then is the
    real case asserted.
    """
    username, cookie = live_user.name, live_user.cookie

    with TestClient(app, raise_server_exceptions=False) as ws_client:
        assert socketio_asgi._lock is not None, "lifespan did not run"
        sess = _open_socket(ws_client, cookie)
        sid = sess.sio_sid
        assert socketio_asgi._sid_users.get(sid) == username

        # --- CONTROL: False is observable through this exact call.
        real_loop = socketio_asgi._main_loop
        monkeypatch.setattr(socketio_asgi, "_main_loop", None)
        assert socketio_asgi.disconnect_user(username) is False, (
            "disconnect_user returned True with no main loop; this test "
            "cannot tell True from False and its assertion below would be "
            "unfalsifiable"
        )
        monkeypatch.setattr(socketio_asgi, "_main_loop", real_loop)

        # --- the real case: a live socket whose disconnect always fails.
        failing = _RaisingDisconnect()
        monkeypatch.setattr(socketio_asgi.sio, "disconnect", failing)

        result = socketio_asgi.disconnect_user(username)
        assert _wait_until(lambda: sid in failing.sids), (
            "sio.disconnect was never called; nothing failed, so the "
            f"return value is not under test. calls={failing.sids!r}"
        )
        # No wait is needed before judging ``result``: the defect IS that
        # the value was decided synchronously, at scheduling time, before
        # any sid was even looked at. An implementation that reported the
        # truth would have had to wait for the coroutine (or hand back an
        # awaitable), so it could not have returned True here either.
        assert result is False, (
            "disconnect_user reported success (True) even though every "
            f"sio.disconnect raised; sids attempted={failing.sids!r}, "
            f"still registered={socketio_asgi._sid_users!r}"
        )