Skip to content

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

26.1 KB
Raw Blame
"""Socket.IO event-handler tests for the live research/chat progress feed.

Re-ported from the pre-FastAPI-migration module, which drove Flask-SocketIO's
``SocketIOService`` singleton and its private ``__handle_connect`` /
``__handle_subscribe`` / ``__handle_disconnect`` methods. That class is gone;
the handlers are now module-level coroutines on a python-socketio
``AsyncServer`` in ``web/services/socketio_asgi.py``, so the whole module
skipped itself and eight tests stopped running.

SURVEY — already covered on this branch, deliberately NOT duplicated
--------------------------------------------------------------------
* ``test_on_connect_logs_client_sid`` — the connect lifecycle is pinned by
  ``tests/web/services/test_socketio_connect_gate.py`` (six accept/reject
  cases) and ``test_socketio_handshake_auth.py``
  ::TestConnectOverRealServer. The original only asserted
  ``mock_logger.info.called``, and only inside an ``if handler is None``
  branch.
* ``test_on_disconnect_cleans_up_subscriptions`` —
  ``tests/web/services/test_subscription_owner_scoping.py``
  ::test_disconnect_drops_the_sid_from_every_owner_entry and
  ``test_socketio_handshake_auth.py``
  ::test_disconnect_clears_verified_identity_and_subscriptions.
* ``test_on_subscribe_adds_to_subscription_set`` —
  ``tests/web/services/test_socket_connect_session_gate.py``
  ::test_live_session_can_still_subscribe asserts the exact
  ``_subscriptions`` entry, through the real session gate.
* ``test_emit_to_subscribers_broadcasts_to_room`` —
  ``tests/web/test_socketio_asgi_contracts.py``
  ::TestEmitToSubscribersOverRealSockets and
  ``test_subscription_owner_scoping.py`` (which additionally pin the
  per-owner scoping the Flask version could not express).

WHAT IS RESTORED HERE
---------------------
1. The immediate status push on subscribe. Nothing else on this branch
   references ``get_active_research_snapshot`` — a late subscriber getting a
   blank progress bar until the next tick is invisible to every other test.
2. ``emit_socket_event``'s own no-loop / scheduling-failure guards. Its
   sibling ``emit_to_user`` has these
   (``test_socketio_asgi_user_scoping.py::TestEmitToUserNoLoop``);
   ``emit_socket_event`` is only ever patched out elsewhere.
3. Concurrency safety of subscribe/disconnect — re-expressed for the new
   mechanism (see the test's own docstring).
4. That a failing status push cannot lose the subscription.

MECHANISM NOTE on ``test_emit_socket_event_handles_exceptions``: the Flask
version made ``socketio.emit`` raise and asserted the wrapper returned
False. That is not reachable here. ``emit_socket_event`` schedules
``_async_emit`` onto the main loop with ``run_coroutine_threadsafe`` and
returns immediately, so a failure inside the emit happens after the return
value is decided. What the wrapper can still fail on is SCHEDULING, which is
what the ported test drives.
"""

import asyncio
import contextlib
import threading
import time
from unittest.mock import AsyncMock, Mock, patch

import pytest

from local_deep_research.web.auth.session_manager import session_manager
from local_deep_research.web.services import socketio_asgi as sio_mod

USER = "socket_events_user"
RESEARCH_ID = "research-abc-123"


@pytest.fixture
def socket_state(monkeypatch):
    """Isolated per-socket module state plus a capturing ``sio.emit``.

    Returns the list of ``(room, event, data)`` tuples the handlers emit.
    A fresh ``_lock`` avoids inheriting one bound to a loop from an earlier
    test.
    """
    monkeypatch.setattr(sio_mod, "_lock", asyncio.Lock())
    monkeypatch.setattr(sio_mod, "_subscriptions", {})
    monkeypatch.setattr(sio_mod, "_sid_users", {})
    monkeypatch.setattr(sio_mod, "_sid_sessions", {})

    emitted: list[tuple] = []

    async def _capture(event, data=None, room=None, **kwargs):
        emitted.append((room, event, data))

    monkeypatch.setattr(sio_mod.sio, "emit", _capture)
    return emitted


@pytest.fixture
def live_sid(socket_state):
    """A registered sid backed by a REAL live server-side session.

    ``on_subscribe`` re-validates the originating session on every call
    (``_socket_session_still_valid``), so a fabricated id would make every
    subscribe below take the "session expired" branch and the tests would
    prove nothing about the paths they name.
    """
    token = session_manager.create_session(USER, remember_me=False)
    sio_mod._sid_users["sid-1"] = USER
    sio_mod._sid_sessions["sid-1"] = token
    try:
        yield "sid-1"
    finally:
        session_manager.destroy_session(token)


def _subscribe(sid, research_id=RESEARCH_ID, snapshot=None):
    """Drive the real ``on_subscribe`` with ownership granted."""
    with (
        patch.object(sio_mod, "_user_owns_research", return_value=True),
        patch.object(
            sio_mod, "get_active_research_snapshot", return_value=snapshot
        ),
    ):
        asyncio.run(sio_mod.on_subscribe(sid, {"research_id": research_id}))


class TestSubscribeStatusPush:
    """A client that subscribes to an already-running research must be sent
    the current progress immediately, not left blank until the next tick."""

    def test_subscribe_sends_current_status_if_available(
        self, socket_state, live_sid
    ):
        snapshot = {
            "progress": 50,
            "status": "in_progress",
            "log": [
                {"message": "First step", "time": "2024-01-01"},
                {"message": "Processing...", "time": "2024-01-02"},
            ],
            "settings": None,
        }
        _subscribe(live_sid, snapshot=snapshot)

        assert sio_mod._subscriptions == {(USER, RESEARCH_ID): {live_sid}}

        progress_events = [
            (room, event, data)
            for room, event, data in socket_state
            if event == f"research_progress_{RESEARCH_ID}"
        ]
        assert len(progress_events) == 1, (
            f"expected exactly one immediate status push, got {socket_state!r}"
        )
        room, _event, data = progress_events[0]
        assert room == live_sid, (
            f"the status push went to {room!r} instead of the subscribing "
            f"socket — a broadcast here leaks one user's progress to every "
            f"connected client"
        )
        assert data["progress"] == 50
        assert data["message"] == "Processing...", (
            "the push must carry the LATEST log entry, not the first"
        )

    def test_subscribe_sends_nothing_when_research_is_not_active(
        self, socket_state, live_sid
    ):
        """Counterpart to the test above.

        Without this, a handler that pushed a hard-coded event on every
        subscribe would pass the positive case. ``get_active_research_snapshot``
        returning ``None`` means there is no live run, so there is nothing
        truthful to report.
        """
        _subscribe(live_sid, snapshot=None)

        assert sio_mod._subscriptions == {(USER, RESEARCH_ID): {live_sid}}
        assert socket_state == [], (
            f"a subscribe to an inactive research emitted {socket_state!r}"
        )

    def test_subscribe_sends_nothing_when_the_snapshot_has_no_log_yet(
        self, socket_state, live_sid
    ):
        """Regression contract for the historically uncovered third arm.

        ``on_subscribe`` does ``latest_log = snapshot["log"][-1] if
        snapshot["log"] else None`` and emits only when that is truthy
        (``socketio_asgi.py:606-608``). The other two arms are pinned above —
        no snapshot at all, and a snapshot with entries — which leaves the
        real run that is live but has not logged anything yet: the window
        between a research being registered and its first progress line.

        Dropping the emptiness guard would push ``message=None`` into the
        progress feed for exactly that window. Both neighbouring tests still
        pass with the guard removed, because neither supplies an empty log.
        """
        _subscribe(live_sid, snapshot={"progress": 0, "log": []})

        assert sio_mod._subscriptions == {(USER, RESEARCH_ID): {live_sid}}
        assert socket_state == [], (
            f"a snapshot with no log entries still emitted {socket_state!r} — "
            f"a subscriber to a just-started run receives a progress event "
            f"whose message is None"
        )

    def test_subscription_survives_a_failing_status_push(
        self, socket_state, live_sid, monkeypatch
    ):
        """The subscription is recorded BEFORE the status push.

        If the order were reversed, a transient emit failure (client already
        gone, transport closed mid-handshake) would silently leave the
        client subscribed to nothing — it would never receive another
        progress event for that run and the UI would hang at 0%.
        """

        async def _boom(*args, **kwargs):
            raise RuntimeError("transport closed")

        monkeypatch.setattr(sio_mod.sio, "emit", _boom)

        snapshot = {
            "progress": 10,
            "status": "in_progress",
            "log": [{"message": "working", "time": "2024-01-01"}],
            "settings": None,
        }
        with contextlib.suppress(RuntimeError):
            _subscribe(live_sid, snapshot=snapshot)

        assert sio_mod._subscriptions == {(USER, RESEARCH_ID): {live_sid}}, (
            "a failed status push lost the subscription"
        )

        # ...and the normal teardown still reclaims it.
        asyncio.run(sio_mod.disconnect(live_sid))
        assert sio_mod._subscriptions == {}
        assert live_sid not in sio_mod._sid_users


class TestEmitSocketEvent:
    """``emit_socket_event`` is the sync wrapper background threads use."""

    def test_returns_false_when_no_main_loop(self, socket_state):
        """No captured loop means the emit would silently no-op; the caller
        must be told so rather than believing it delivered."""
        with patch.object(sio_mod, "_get_main_loop", return_value=None):
            assert sio_mod.emit_socket_event("test_event", {"a": 1}) is False

    def test_returns_false_when_loop_not_running(self, socket_state):
        loop = asyncio.new_event_loop()
        try:
            with patch.object(sio_mod, "_get_main_loop", return_value=loop):
                assert (
                    sio_mod.emit_socket_event("test_event", {"a": 1}) is False
                )
        finally:
            loop.close()

    def test_returns_false_when_scheduling_raises(self, socket_state):
        """The successor of the Flask "emit raises -> False" case.

        A closed/shutting-down loop makes ``run_coroutine_threadsafe``
        raise. That must be reported as a failed emit, not escape into a
        background research thread and kill it.
        """
        loop = asyncio.new_event_loop()
        thread = threading.Thread(target=loop.run_forever, daemon=True)
        thread.start()

        def _refuse(coro, _loop):
            # Close the coroutine we are refusing to schedule, so the
            # "never awaited" RuntimeWarning doesn't surface in an
            # unrelated later test when it is garbage collected.
            coro.close()
            raise RuntimeError("loop is closing")

        try:
            with (
                patch.object(sio_mod, "_get_main_loop", return_value=loop),
                patch.object(
                    asyncio, "run_coroutine_threadsafe", side_effect=_refuse
                ),
            ):
                assert (
                    sio_mod.emit_socket_event("test_event", {"a": 1}) is False
                )
        finally:
            loop.call_soon_threadsafe(loop.stop)
            thread.join(timeout=5)
            loop.close()

    def test_dispatches_on_a_live_loop(self, socket_state):
        """Positive control for the three failure cases above.

        A wrapper hard-wired to ``return False`` would satisfy all of them.
        This proves the happy path both reports True AND actually reaches
        ``sio.emit`` with the room the caller asked for.
        """
        loop = asyncio.new_event_loop()
        thread = threading.Thread(target=loop.run_forever, daemon=True)
        thread.start()

        mock_sio = Mock()
        mock_sio.emit = AsyncMock()
        try:
            with (
                patch.object(sio_mod, "_get_main_loop", return_value=loop),
                patch.object(sio_mod, "sio", mock_sio),
            ):
                assert (
                    sio_mod.emit_socket_event(
                        "test_event", {"a": 1}, room="sid-9"
                    )
                    is True
                )
                deadline = time.monotonic() + 5
                while (
                    mock_sio.emit.await_count == 0
                    and time.monotonic() < deadline
                ):
                    time.sleep(0.01)
        finally:
            loop.call_soon_threadsafe(loop.stop)
            thread.join(timeout=5)
            loop.close()

        assert mock_sio.emit.await_count == 1
        call = mock_sio.emit.await_args
        assert call.args[0] == "test_event"
        assert call.args[1] == {"a": 1}
        assert call.kwargs["room"] == "sid-9"


class TestConcurrentSubscribeDisconnect:
    """Concurrency safety, re-expressed for the mechanism that replaced it.

    The Flask original raced ten OS threads against a plain ``dict``,
    because Flask-SocketIO dispatched each event on its own worker thread.
    ``AsyncServer`` dispatches every handler as a coroutine on ONE event
    loop, so OS threads are no longer the hazard — interleaving at ``await``
    points is. ``on_subscribe`` awaits an ownership check and a session
    re-validation between reading and writing ``_subscriptions``, and
    ``disconnect`` iterates that same dict; without ``_lock`` held across
    each mutation, a disconnect landing mid-iteration raises
    ``RuntimeError: dictionary changed size during iteration`` (the exact
    failure the lock in ``emit_to_user`` documents).
    """

    def test_concurrent_subscribe_and_disconnect_stay_consistent(
        self, socket_state
    ):
        sids = [f"client-{i}" for i in range(10)]
        tokens = {
            sid: session_manager.create_session(USER, remember_me=False)
            for sid in sids
        }
        for sid, token in tokens.items():
            sio_mod._sid_users[sid] = USER
            sio_mod._sid_sessions[sid] = token

        errors: list[str] = []

        async def _run():
            async def _sub(sid):
                try:
                    await sio_mod.on_subscribe(
                        sid, {"research_id": "concurrent-research"}
                    )
                except Exception as exc:  # noqa: BLE001 - reported below
                    errors.append(f"subscribe {sid}: {exc!r}")

            async def _dis(sid):
                try:
                    await sio_mod.disconnect(sid)
                except Exception as exc:  # noqa: BLE001 - reported below
                    errors.append(f"disconnect {sid}: {exc!r}")

            with (
                patch.object(sio_mod, "_user_owns_research", return_value=True),
                patch.object(
                    sio_mod, "get_active_research_snapshot", return_value=None
                ),
            ):
                tasks = []
                for sid in sids:
                    tasks.append(asyncio.create_task(_sub(sid)))
                    tasks.append(asyncio.create_task(_dis(sid)))
                await asyncio.gather(*tasks)

        try:
            asyncio.run(_run())
        finally:
            for token in tokens.values():
                session_manager.destroy_session(token)

        assert errors == [], f"concurrent operations failed: {errors}"

        # Whatever interleaving won, the state must stay internally
        # consistent: no empty leftover sets, and no subscription naming a
        # sid that has already been disconnected.
        for key, subscribers in sio_mod._subscriptions.items():
            assert subscribers, f"{key} was left as an empty set"
            for sid in subscribers:
                assert sid in sio_mod._sid_users, (
                    f"{sid} is still subscribed to {key} after disconnect "
                    f"dropped its identity"
                )


class TestDisconnectUserAndSession:
    """Regression evidence for socket disconnect helpers.

    A socket is authorized at handshake. Session teardown therefore depends on
    these helpers disconnecting the associated sockets. At the ADR-0010 review
    snapshot neither helper was executed by a test: the logout and
    change-password paths patched them out, so a no-op helper would have passed
    the suite.

    Driven against a real running loop, because both schedule onto it with
    ``run_coroutine_threadsafe`` and a loopless test would only ever exercise
    the ``return False`` guard.
    """

    @staticmethod
    def _run_on_live_loop(call):
        """Invoke ``call`` with a live loop and a recording ``sio``.

        Returns the list of sids ``sio.disconnect`` was awaited with, once the
        teardown has actually finished.

        The helpers return as soon as the work is *scheduled*, so the result
        has to be waited for. Polling until an expected count appears would
        make the negative assertions unsound — "user B was not disconnected"
        could pass simply because the loop had not reached B yet. Instead the
        scheduled future is captured and waited on, which is only true when
        the coroutine has run to completion. Patching
        ``run_coroutine_threadsafe`` is the same seam
        ``test_returns_false_when_scheduling_raises`` uses.
        """
        loop = asyncio.new_event_loop()
        thread = threading.Thread(target=loop.run_forever, daemon=True)
        thread.start()

        disconnected: list[str] = []

        async def _record(sid):
            disconnected.append(sid)

        mock_sio = Mock()
        mock_sio.disconnect = _record

        futures = []
        real_schedule = asyncio.run_coroutine_threadsafe

        def _capture(coro, target_loop):
            future = real_schedule(coro, target_loop)
            futures.append(future)
            return future

        try:
            with (
                patch.object(sio_mod, "_get_main_loop", return_value=loop),
                patch.object(sio_mod, "sio", mock_sio),
                patch.object(asyncio, "run_coroutine_threadsafe", _capture),
            ):
                assert call() is True, (
                    "the helper reported it could not schedule the teardown "
                    "even though the loop is running"
                )
                assert futures, (
                    "the helper returned True without scheduling anything "
                    "onto the loop — nothing was ever disconnected"
                )
                # Wait INSIDE the patch context. The scheduled coroutine
                # resolves ``sio`` when it runs, not when it was created, so
                # leaving the block first restores the real ``sio`` and the
                # coroutine awaits a plain Mock — which raises TypeError,
                # which ``_disconnect_matching`` swallows by design. The
                # recorder would then stay empty and
                # ``test_disconnect_user_leaves_other_users_connected`` would
                # pass on an empty list, i.e. vacuously.
                for future in futures:
                    future.result(timeout=5)
        finally:
            loop.call_soon_threadsafe(loop.stop)
            thread.join(timeout=5)
            loop.close()
        return disconnected

    @staticmethod
    def _register(sid, username, session_id):
        sio_mod._sid_users[sid] = username
        sio_mod._sid_sessions[sid] = session_id

    def test_disconnect_user_severs_every_socket_that_user_holds(
        self, socket_state
    ):
        """Password change and the idle sweep destroy every session, so every
        socket must go — including ones opened from a different session."""
        self._register("sid-a", USER, "session-1")
        self._register("sid-b", USER, "session-2")
        self._register("sid-other", "someone_else", "session-3")

        disconnected = self._run_on_live_loop(
            lambda: sio_mod.disconnect_user(USER)
        )

        assert sorted(disconnected) == ["sid-a", "sid-b"], (
            f"disconnect_user severed {sorted(disconnected)}; it must sever "
            f"every socket authenticated as {USER}"
        )

    def test_disconnect_user_leaves_other_users_connected(self, socket_state):
        """Negative control for the test above.

        A helper that disconnected the whole ``_sid_users`` map would satisfy
        it. One user's password change must not log everyone else out.
        """
        self._register("sid-a", USER, "session-1")
        self._register("sid-other", "someone_else", "session-3")

        disconnected = self._run_on_live_loop(
            lambda: sio_mod.disconnect_user(USER)
        )

        # Assert the positive first. "sid-other is absent" is satisfied by an
        # empty list, so on its own this test would pass if the teardown never
        # ran at all — which is exactly how it behaved before the wait was
        # moved inside the patch context.
        assert "sid-a" in disconnected, (
            f"the target socket was never disconnected ({disconnected!r}), so "
            f"the assertion below would be vacuous"
        )
        assert "sid-other" not in disconnected, (
            "disconnect_user severed another user's socket — a single user's "
            "password change would drop every connected client"
        )

    def test_disconnect_session_spares_the_users_other_sessions(
        self, socket_state
    ):
        """Row 27's other half, and the distinction the two helpers exist for.

        Logout is single-session: the tab being logged out loses its sockets
        while the same user's other devices keep theirs. If logout used
        ``disconnect_user``, signing out on a phone would kill a research
        stream running on a desktop.
        """
        self._register("sid-a", USER, "session-1")
        self._register("sid-b", USER, "session-2")

        disconnected = self._run_on_live_loop(
            lambda: sio_mod.disconnect_session("session-1")
        )

        assert disconnected == ["sid-a"], (
            f"disconnect_session severed {disconnected}; only the sockets "
            f"authorised by session-1 should go"
        )

    def test_both_helpers_report_failure_without_a_running_loop(
        self, socket_state
    ):
        """Positive control for the ``is True`` assertions above.

        Without this, a helper hard-wired to ``return True`` would pass every
        test in this class. Callers are teardown paths that must not raise, so
        the no-loop case has to be a reported False rather than an exception.
        """
        self._register("sid-a", USER, "session-1")

        with patch.object(sio_mod, "_get_main_loop", return_value=None):
            assert sio_mod.disconnect_user(USER) is False
            assert sio_mod.disconnect_session("session-1") is False


class TestUnsubscribeSessionRevalidation:
    """Regression evidence for unsubscribe session revalidation.

    ``on_subscribe`` re-validates the socket's originating session on every
    call and, on failure, severs the session's sockets. Before the
    migration-branch fix, ``on_unsubscribe`` applied the ownership gate but did
    not call ``_socket_session_still_valid``. These tests preserve the corrected
    wiring.

    The ownership gate alone is inert here -- ``subs.discard(sid)`` can only
    remove the caller's own sid, so an unauthorized unsubscribe was already a
    silent no-op. What matters is the teardown side effect. Logout and
    password change disconnect actively. Idle expiry can still have bounded
    revocation latency until the periodic sweep; this is a timing window, not
    an unguarded cross-user path. ``_socket_session_still_valid`` records that
    both subscribe and unsubscribe revalidate the session.

    ``disconnect_session`` is patched rather than driven: its mechanics are
    already covered by TestDisconnectUserAndSession above, and what is under
    test here is whether ``on_unsubscribe`` decides to call it.
    """

    def test_live_session_can_still_unsubscribe(self, socket_state, live_sid):
        """Positive control. Without it, a fix that refused EVERY unsubscribe
        would satisfy the negative test below for entirely the wrong reason."""
        _subscribe(live_sid)
        assert sio_mod._subscriptions == {(USER, RESEARCH_ID): {live_sid}}

        with patch.object(sio_mod, "_user_owns_research", return_value=True):
            asyncio.run(
                sio_mod.on_unsubscribe(live_sid, {"research_id": RESEARCH_ID})
            )

        assert sio_mod._subscriptions == {}, (
            "a live session's own unsubscribe must still take effect"
        )

    def test_revoked_session_is_refused_and_severed(
        self, socket_state, monkeypatch
    ):
        """A revoked socket must be severed, not merely refused."""
        token = session_manager.create_session(USER, remember_me=False)
        sio_mod._sid_users["sid-rev"] = USER
        sio_mod._sid_sessions["sid-rev"] = token

        # Subscribe while the session is still alive, so the "subscription
        # untouched" assertion below cannot pass vacuously.
        _subscribe("sid-rev")
        assert sio_mod._subscriptions == {(USER, RESEARCH_ID): {"sid-rev"}}

        session_manager.destroy_session(token)

        severed = []
        monkeypatch.setattr(
            sio_mod, "disconnect_session", lambda s: severed.append(s)
        )

        with patch.object(sio_mod, "_user_owns_research", return_value=True):
            asyncio.run(
                sio_mod.on_unsubscribe("sid-rev", {"research_id": RESEARCH_ID})
            )

        assert severed == [token], (
            f"a revoked socket's unsubscribe severed {severed!r}; it must tear "
            f"down that session's sockets exactly as on_subscribe does, or the "
            f"socket keeps receiving broadcasts until the 5-minute sweep"
        )
        assert sio_mod._subscriptions == {(USER, RESEARCH_ID): {"sid-rev"}}, (
            "the unsubscribe must be refused, not honoured, once the session "
            "is dead"
        )