Skip to content

Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.

feat(db): run under a restricted role, with the tenant lookups as the only exemption (CHOO-2623)

Wojciech Wojtyniak0e887a65dfc5
diff --git a/.env.example b/.env.example
index d1815527a..981e00cd7 100644
--- a/.env.example
+++ b/.env.example
@@ -34,10 +34,37 @@ POSTGRES_HOST_PORT=5432
SWITCH_BIND_ADDR=127.0.0.1
# ── Switch database (docker compose) ──────────────────────────────────────────
+# Two roles, not one, because row-level security is inert against the role
+# that owns the tables it protects — Postgres exempts a table's owner from its
+# own policies. So DB_USER below is deliberately NOT the owner:
+#
+# DB_USER / DB_PASSWORD the RUNTIME role. Every request switch-core serves
+# authenticates as this role, so it is the one the
+# tenant-isolation policies actually apply to. It is
+# a plain LOGIN role that owns nothing and can
+# create nothing — `just up`'s init-db container
+# creates it, and switch-core grants it CRUD on the
+# schema at boot, once the owner connection has run
+# migrations.
+# DB_OWNER_USER / DB_OWNER_PASSWORD
+# the SCHEMA OWNER, used for exactly two things:
+# running Alembic at boot, and re-issuing the
+# runtime role's grants right after. It is never
+# used to serve a request. The container's Postgres
+# superuser is this role now, not DB_USER — see
+# deploy/local/docker-compose.yml.
+#
+# If DB_USER were the owner, switch-core would refuse to start:
+# DB_REQUIRE_RESTRICTED_ROLE (default true) checks the runtime connection is
+# not a superuser, does not bypass RLS, and owns no policied table, precisely
+# because a deployment that believes it is isolating tenants and is not looks
+# identical to one that is, until a second tenant reads the first's rooms.
DB_HOST=localhost
DB_PORT=5432
-DB_USER=postgres
+DB_USER=switch_app
DB_PASSWORD=
+DB_OWNER_USER=postgres
+DB_OWNER_PASSWORD=
DB_NAME=switch
# TLS for the Postgres connection: disable | allow | prefer | require |
diff --git a/core/switch_core/bridges/agent/auth.py b/core/switch_core/bridges/agent/auth.py
index a4bb3841b..859bee731 100644
--- a/core/switch_core/bridges/agent/auth.py
+++ b/core/switch_core/bridges/agent/auth.py
@@ -13,8 +13,13 @@ from starlette.types import ASGIApp, Receive, Scope, Send
from switch_core.bridges.agent.api_key_cache import ApiKeyCache
from switch_core.bridges.agent.registration_bootstrap import REGISTRATION_KEY_TYPES
from switch_core.db.models import Agent, ApiKey
+from switch_core.db.session_scope import tenant_session
from switch_core.db.stores.agent_store import AgentStore
from switch_core.db.stores.api_key_store import ApiKeyStore
+from switch_core.db.tenant_lookup import (
+ tenant_of_agent_oauth_client,
+ tenant_of_api_key,
+)
from switch_core.logging_context import bind_log_context, unbind_log_context
from switch_core.tenant_context import tenant_scope
@@ -94,10 +99,12 @@ class BearerAuthMiddleware:
question about whether a downstream ``get_session`` might query before the
tenant is known — it cannot, since it does not run until after
``self.app(...)`` is called below. The lookups that resolve the credential
- itself (``_resolve_api_key``, ``_try_oidc``) run on their own session,
- opened directly from ``session_factory`` rather than through that
- downstream seam, because they are what determines the tenant and so must
- run before one is bound.
+ itself (``_resolve_api_key``, ``_try_oidc``) run ahead of that, on
+ sessions of their own, and each is in two steps: the credential's *tenant*
+ comes from one of the eight ``SECURITY DEFINER`` lookups that are the
+ whole exemption from row-level security (``db/tenant_lookup.py``), and the
+ row itself is then read with that tenant bound, subject to the same
+ policies as everything else.
"""
def __init__(
@@ -206,9 +213,23 @@ class BearerAuthMiddleware:
if cached is not None:
return cached
- # A system session: no tenant is bound yet, because this lookup is
- # what determines one.
- async with self._session_factory() as session:
+ # Two steps, because a credential's tenant has to be known before its
+ # row can be read. `api_keys` is scoped like everything else, so a
+ # session with nothing bound reads nothing from it — this used to be a
+ # single unbound read, and under the restricted runtime role it failed
+ # on every request that missed the cache. The hash is resolved to a
+ # tenant through the `SECURITY DEFINER` exemption
+ # (`db/tenant_lookup.py`), which answers with a tenant id and nothing
+ # else, and the row itself is then read the ordinary scoped way.
+ #
+ # An unknown token resolves to no tenant and stops here, without a
+ # second round trip — which is also the shape that matters for load,
+ # since an unauthenticated flood never reaches the second query.
+ tenant_id = await tenant_of_api_key(self._session_factory, token_hash)
+ if tenant_id is None:
+ return None, None
+
+ async with tenant_session(self._session_factory, tenant_id) as session:
found = await self._api_key_store.get_with_agent_by_hash(
session, token_hash
)
@@ -236,9 +257,15 @@ class BearerAuthMiddleware:
logger.warning("OIDC token has no azp or client_id claim")
return None
- # A system session, same reason as _resolve_api_key: this is what
- # decides which agent (and so which tenant) is asking.
- async with self._session_factory() as session:
+ # Same two steps as `_resolve_api_key`, for the same reason: `agents`
+ # is scoped, and this is the read that decides which agent — and so
+ # which tenant — is asking. `oauth_client_id` carries no unique index,
+ # so the lookup refuses rather than picking when two tenants have
+ # registered an agent under the same one.
+ tenant_id = await tenant_of_agent_oauth_client(self._session_factory, client_id)
+ if tenant_id is None:
+ return None
+ async with tenant_session(self._session_factory, tenant_id) as session:
return await self._agent_store.get_by_oauth_client_id(session, client_id)
diff --git a/core/switch_core/bridges/agent/protocol/service.py b/core/switch_core/bridges/agent/protocol/service.py
index de16930c8..388020da6 100644
--- a/core/switch_core/bridges/agent/protocol/service.py
+++ b/core/switch_core/bridges/agent/protocol/service.py
@@ -59,6 +59,7 @@ from switch_core.bridges.resource.service import ResourceService
from switch_core.crypto import encrypt_token
from switch_core.db.models import (
Agent,
+ AgentRuntimeState,
ApiKey,
Message,
MessageAttachment,
@@ -72,7 +73,7 @@ from switch_core.db.models import (
Tool,
User,
)
-from switch_core.db.session_scope import unscoped_session
+from switch_core.db.session_scope import tenant_session
from switch_core.db.stores.agent_runtime_state_store import (
IDLE as RUNTIME_STATE_IDLE,
)
@@ -83,6 +84,7 @@ from switch_core.db.stores.message_store import MessageStore
from switch_core.db.stores.room_group_store import RoomGroupStore
from switch_core.db.stores.room_role_store import RoomRoleStore
from switch_core.db.stores.user_store import UserStore
+from switch_core.db.tenant_lookup import all_tenant_ids, tenant_of_api_key
from switch_core.deeplinks import deeplink_for_platform
from switch_core.events import (
LlmCallReport as MatrixLlmCallReport,
@@ -480,16 +482,20 @@ class ProtocolService:
confirming itself on every subsequent call.
"""
token_hash = hashlib.sha256(registration_token.encode()).hexdigest()
- # Resolving a credential is unscoped by definition — `key_hash` is one
- # of the two columns deliberately left globally unique for exactly
- # this reason — and it is the read that *produces* the tenant bound
- # below. Answering it under a tenant would make it either a tautology
- # or a false "invalid token".
- async with unscoped_session(self.session_factory) as session:
+ # Resolving a credential is the read that *produces* a tenant, so it
+ # cannot be answered under one: doing so is either a tautology or a
+ # false "invalid token". `key_hash` is one of the two columns
+ # deliberately left globally unique for exactly this, and the
+ # exemption (`db/tenant_lookup.py`) turns it into a tenant id. The key
+ # row itself, and the owner resolution that reads it, are then
+ # ordinary scoped reads under that tenant.
+ tenant_id = await tenant_of_api_key(self.session_factory, token_hash)
+ if tenant_id is None:
+ raise PermissionError("Invalid registration token")
+ async with tenant_session(self.session_factory, tenant_id) as session:
key = await self.api_key_store.get_by_hash(session, token_hash)
if key is None or key.type not in REGISTRATION_KEY_TYPES:
raise PermissionError("Invalid registration token")
- tenant_id = key.tenant_id
try:
owner_id = await resolve_registration_owner_id(
session, self.user_store, key
@@ -1573,16 +1579,30 @@ class ProtocolService:
deleted and recreated on every refresh rather than edited in place.
"""
# This sweep spans every tenant by nature — it is the one place that
- # decides whether *any* stale row anywhere needs resetting — so the
- # initial read is unscoped. Everything done *with* a row then happens
- # inside that row's own tenant, including the emit at the end: the
- # clear event goes out to a bridge, which resolves a mention handle
- # and posts a message, and those are writes in the row's tenant like
- # any other. Closing the binding after the upsert and leaving the tail
- # outside it would put exactly the visible half of the work back on
- # whatever was ambient.
- async with unscoped_session(self.session_factory) as session:
- rows = await self.agent_runtime_state_store.get_active(session)
+ # decides whether *any* stale row anywhere needs resetting — so it
+ # asks the exemption which tenants there are (`db/tenant_lookup.py`)
+ # and reads each tenant's rows scoped to it. That is one extra round
+ # trip every five seconds, which is what the enumeration costs now
+ # that a single cross-tenant read is not available to it.
+ #
+ # Everything done *with* a row happens inside that row's own tenant,
+ # including the emit at the end: the clear event goes out to a bridge,
+ # which resolves a mention handle and posts a message, and those are
+ # writes in the row's tenant like any other. Closing the binding after
+ # the upsert and leaving the tail outside it would put exactly the
+ # visible half of the work back on whatever was ambient.
+ rows: list[AgentRuntimeState] = []
+ for tenant_id in await all_tenant_ids(self.session_factory):
+ async with tenant_session(self.session_factory, tenant_id) as session:
+ # Filtered on the row's own tenant, not left to the policy: on an
+ # owner connection no policy narrows this read, and the fan-out
+ # would act on every tenant's rows once per tenant. See
+ # `db/tenant_lookup.py`, "a fan-out ... filters what it reads back".
+ rows.extend(
+ row
+ for row in await self.agent_runtime_state_store.get_active(session)
+ if row.tenant_id == tenant_id
+ )
for row in rows:
if self.connections.has_session_in(row.agent_id, row.room_id):
continue
diff --git a/core/switch_core/bridges/agent/server_connectors/lifecycle.py b/core/switch_core/bridges/agent/server_connectors/lifecycle.py
index b36364885..d47214b5c 100644
--- a/core/switch_core/bridges/agent/server_connectors/lifecycle.py
+++ b/core/switch_core/bridges/agent/server_connectors/lifecycle.py
@@ -14,9 +14,10 @@ from switch_core.bridges.agent.server_connectors.base import (
from switch_core.bridges.agent.server_connectors.core import ConnectorCore
from switch_core.crypto import decrypt_token, encrypt_token
from switch_core.db.models import ApiKey, ServerConnector
-from switch_core.db.session_scope import unscoped_session
+from switch_core.db.session_scope import tenant_session
from switch_core.db.stores.api_key_store import ApiKeyStore
from switch_core.db.stores.server_connector_store import ServerConnectorStore
+from switch_core.db.tenant_lookup import all_tenant_ids, tenant_of_server_connector
logger = logging.getLogger(__name__)
@@ -51,13 +52,28 @@ class ServerSideConnectorLifecycleService:
self._config_registry[type_name] = config_cls
async def start_all(self) -> None:
- # Every tenant's active connectors in one pass — unscoped by nature,
- # same reasoning as CollaborationBridgeLifecycleService.start_all.
+ # Every tenant's active connectors, one tenant at a time — same
+ # reasoning and same history as
+ # CollaborationBridgeLifecycleService.start_all, and worse here: this
+ # one is launched by a bare `create_task` in the lifespan, so when the
+ # unscoped read it replaces began returning nothing under the runtime
+ # role, no connector started and nothing said so at any log level.
+ #
# Nothing is bound around `start`: it reads the connector's own row
# and hands that row's tenant to the core, which binds it per unit of
# work rather than once for the poll loop's life.
- async with unscoped_session(self._session_factory) as session:
- connectors = await self._connector_store.get_active(session)
+ connectors: list[ServerConnector] = []
+ for tenant_id in await all_tenant_ids(self._session_factory):
+ async with tenant_session(self._session_factory, tenant_id) as session:
+ # Filtered on the row's own tenant, not left to the policy: on an
+ # owner connection no policy narrows this read, and the fan-out
+ # would act on every tenant's rows once per tenant. See
+ # `db/tenant_lookup.py`, "a fan-out ... filters what it reads back".
+ connectors.extend(
+ record
+ for record in await self._connector_store.get_active(session)
+ if record.tenant_id == tenant_id
+ )
logger.info("Starting %d server-side connectors", len(connectors))
for record in connectors:
@@ -121,10 +137,20 @@ class ServerSideConnectorLifecycleService:
return record
async def start(self, connector_id: str) -> None:
- # Unscoped: the read that answers which tenant this connector is in,
- # reached both from boot with nothing bound and from an HTTP request
- # whose tenant is the caller's, not necessarily the connector's.
- async with unscoped_session(self._session_factory) as session:
+ # The exemption answers which tenant this connector is in
+ # (`db/tenant_lookup.py`); the two rows are then read scoped to it.
+ # Reached both from boot with nothing bound and from an HTTP request
+ # whose tenant is the caller's, not necessarily the connector's, so
+ # the tenant is derived here rather than inherited either way. The
+ # registration key is read in the same scoped session because
+ # `server_connectors` carries a composite foreign key on
+ # `(tenant_id, api_key_id)` — it cannot be another tenant's row.
+ tenant_id = await tenant_of_server_connector(
+ self._session_factory, connector_id
+ )
+ if tenant_id is None:
+ raise ValueError(f"Connector not found: {connector_id}")
+ async with tenant_session(self._session_factory, tenant_id) as session:
record = await self._connector_store.get(session, connector_id)
if record is None:
raise ValueError(f"Connector not found: {connector_id}")
diff --git a/core/switch_core/bridges/collaboration/bridge_core.py b/core/switch_core/bridges/collaboration/bridge_core.py
index 6a563eb6e..1f63beb12 100644
--- a/core/switch_core/bridges/collaboration/bridge_core.py
+++ b/core/switch_core/bridges/collaboration/bridge_core.py
@@ -30,12 +30,13 @@ from switch_core.clients.admin_messages import ADMIN_MARKER, AdminMessageType
from switch_core.clients.client_base import ClientBase, ClientConfig
from switch_core.clients.mentions import mention_regex, strip_emphasis
from switch_core.db.models import BridgeMessageMap, ExternalUser
-from switch_core.db.session_scope import tenant_session, unscoped_session
+from switch_core.db.session_scope import tenant_session
from switch_core.db.stores.agent_store import AgentStore
from switch_core.db.stores.bridge_message_map_store import BridgeMessageMapStore
from switch_core.db.stores.client_store import ClientStore
from switch_core.db.stores.external_user_store import ExternalUserStore
from switch_core.db.stores.room_store import RoomStore
+from switch_core.db.tenant_lookup import tenant_of_room
from switch_core.events import AgentRuntimeStateEvent
from switch_core.logging_context import log_context
from switch_core.provisioning import Provisioning
@@ -310,21 +311,24 @@ class BridgeCore:
A miss should not happen — `add_room_mapping` fills the tenant along
with the channel mapping, and this is only ever asked about a room
- that has one — but if it does, the read is deliberately **unscoped**.
- Asking "which tenant is this room in" while a tenant is bound would
- either confirm the guess or return nothing, and the second reads as
- "no such room" for a room that exists in another tenant. The whole
- point of resolving is that the answer is not known yet.
+ that has one — but if it does, the answer comes from the exemption
+ (`db/tenant_lookup.py`) rather than from a read of the row. Asking
+ "which tenant is this room in" while a tenant is bound would either
+ confirm the guess or return nothing, and the second reads as "no such
+ room" for a room that exists in another tenant. The whole point of
+ resolving is that the answer is not known yet.
+
+ This is the shape the exemption is for. The fallback needed one field
+ of one row, and the field was the tenant; it now reads no row at all.
"""
cached = self._room_tenants.get(room_id)
if cached is not None:
return cached
- async with unscoped_session(self._session_factory) as session:
- room = await self._room_store.get(session, room_id)
- if room is None:
+ tenant_id = await tenant_of_room(self._session_factory, room_id)
+ if tenant_id is None:
raise ValueError(f"Room not found: {room_id}")
- self._room_tenants[room_id] = room.tenant_id
- return room.tenant_id
+ self._room_tenants[room_id] = tenant_id
+ return tenant_id
async def _load_existing_puppets(self) -> None:
async with tenant_session(
diff --git a/core/switch_core/bridges/collaboration/lifecycle_service.py b/core/switch_core/bridges/collaboration/lifecycle_service.py
index 59d57888f..2a1bc3bd2 100644
--- a/core/switch_core/bridges/collaboration/lifecycle_service.py
+++ b/core/switch_core/bridges/collaboration/lifecycle_service.py
@@ -15,13 +15,14 @@ from switch_core.clients.bridge_client import BridgeClient, BridgeClientConfig
from switch_core.clients.client_factory import ClientFactory
from switch_core.config import SwitchConfig
from switch_core.db.models import CollaborationBridge
-from switch_core.db.session_scope import tenant_session, unscoped_session
+from switch_core.db.session_scope import tenant_session
from switch_core.db.stores.agent_store import AgentStore
from switch_core.db.stores.bridge_message_map_store import BridgeMessageMapStore
from switch_core.db.stores.client_store import ClientStore
from switch_core.db.stores.collaboration_bridge_store import CollaborationBridgeStore
from switch_core.db.stores.external_user_store import ExternalUserStore
from switch_core.db.stores.room_store import RoomStore
+from switch_core.db.tenant_lookup import all_tenant_ids, tenant_of_collaboration_bridge
from switch_core.provisioning import Provisioning
from switch_core.tenant_context import no_tenant
@@ -190,15 +191,35 @@ class CollaborationBridgeLifecycleService:
config_cls.model_validate(connection_config)
async def start_all(self) -> None:
- # Every tenant's active bridges in one pass, so the read is unscoped
- # by nature. Nothing is bound around `start`: the bridge's tenant is
- # read from its own row inside `start`, and bound by each unit of work
- # that needs it. Binding here instead would only decide what the
- # long-lived task snapshots, which is exactly what must not matter —
- # `start` is also reached from an HTTP request, and a bridge cannot
- # run under whichever tenant happened to restart it.
- async with unscoped_session(self._session_factory) as session:
- bridges = await self._bridge_store.get_active(session)
+ # Every tenant's active bridges, read one tenant at a time. Which
+ # tenants there are comes from the exemption (`db/tenant_lookup.py`);
+ # each tenant's bridges are then an ordinary scoped read.
+ #
+ # This was one unscoped read, and under the runtime role it returned
+ # nothing — which is indistinguishable from a deployment with no
+ # bridges configured. No bridge started, and the line below said
+ # "Starting 0 collaboration bridges" with no error anywhere to say
+ # otherwise. It stayed invisible until a bridge row existed, because
+ # Postgres does not evaluate a policy for a scan that finds no rows.
+ #
+ # Nothing is bound around `start`: the bridge's tenant is read from its
+ # own row inside `start`, and bound by each unit of work that needs it.
+ # Binding here would only decide what the long-lived task snapshots,
+ # which is exactly what must not matter — `start` is also reached from
+ # an HTTP request, and a bridge cannot run under whichever tenant
+ # happened to restart it.
+ bridges: list[CollaborationBridge] = []
+ for tenant_id in await all_tenant_ids(self._session_factory):
+ async with tenant_session(self._session_factory, tenant_id) as session:
+ # Filtered on the row's own tenant, not left to the policy: on an
+ # owner connection no policy narrows this read, and the fan-out
+ # would act on every tenant's rows once per tenant. See
+ # `db/tenant_lookup.py`, "a fan-out ... filters what it reads back".
+ bridges.extend(
+ bridge
+ for bridge in await self._bridge_store.get_active(session)
+ if bridge.tenant_id == tenant_id
+ )
logger.info("Starting %d collaboration bridges", len(bridges))
for bridge in bridges:
@@ -230,9 +251,21 @@ class CollaborationBridgeLifecycleService:
# workspace, or the same Teams listen port, is precisely the collision
# this exists to refuse — the resource is a property of the host and
# the platform, not of a tenant — so narrowing to the caller's tenant
- # would make it miss the case it was written for.
- async with unscoped_session(self._session_factory) as session:
- existing = await self._bridge_store.get_all(session)
+ # would make it miss the case it was written for. Reached from an
+ # authenticated request, so what is bound going in *is* the caller's
+ # tenant; the loop replaces it per tenant rather than inheriting it.
+ existing: list[CollaborationBridge] = []
+ for tenant_id in await all_tenant_ids(self._session_factory):
+ async with tenant_session(self._session_factory, tenant_id) as session:
+ # Filtered on the row's own tenant, not left to the policy: on an
+ # owner connection no policy narrows this read, and the fan-out
+ # would act on every tenant's rows once per tenant. See
+ # `db/tenant_lookup.py`, "a fan-out ... filters what it reads back".
+ existing.extend(
+ bridge
+ for bridge in await self._bridge_store.get_all(session)
+ if bridge.tenant_id == tenant_id
+ )
for other in existing:
# Guard the None case explicitly: an unflushed row has no id yet, and
# `other.id == exclude_bridge_id` would then be None == None and skip
@@ -348,15 +381,21 @@ class CollaborationBridgeLifecycleService:
return bridge
async def start(self, bridge_id: str) -> None:
- # Unscoped: this is the read that answers which tenant the bridge is
- # in, and it is reached both from boot (nothing bound) and from an
- # HTTP request (the caller's tenant bound, which is not necessarily
- # the bridge's). Everything below derives its tenant from this row.
- async with unscoped_session(self._session_factory) as session:
+ # Two steps, because this is the point where the bridge's tenant is
+ # not yet known: the exemption answers which tenant the id is in
+ # (`db/tenant_lookup.py`), and the row itself is then read scoped to
+ # it. Reached both from boot, with nothing bound, and from an HTTP
+ # request, where what is bound is the caller's tenant and not
+ # necessarily the bridge's — so this deliberately does not inherit.
+ tenant_id = await tenant_of_collaboration_bridge(
+ self._session_factory, bridge_id
+ )
+ if tenant_id is None:
+ raise ValueError(f"Bridge not found: {bridge_id}")
+ async with tenant_session(self._session_factory, tenant_id) as session:
bridge = await self._bridge_store.get(session, bridge_id)
if bridge is None:
raise ValueError(f"Bridge not found: {bridge_id}")
- tenant_id = bridge.tenant_id
adapter_cls = self._adapter_registry.get(bridge.type)
config_cls = self._config_registry.get(bridge.type)
diff --git a/core/switch_core/clients/agent_client.py b/core/switch_core/clients/agent_client.py
index c4b22f1ee..11c76fbf2 100644
--- a/core/switch_core/clients/agent_client.py
+++ b/core/switch_core/clients/agent_client.py
@@ -44,7 +44,7 @@ from switch_core.clients.mentions import (
)
from switch_core.clients.room_meta import RoomMeta
from switch_core.db.models import Agent
-from switch_core.db.session_scope import unscoped_session
+from switch_core.db.session_scope import tenant_session
from switch_core.db.stores.agent_session_store import AgentSessionStore
from switch_core.db.stores.agent_store import AgentStore
from switch_core.db.stores.collaboration_bridge_store import CollaborationBridgeStore
@@ -53,6 +53,7 @@ from switch_core.db.stores.external_user_store import ExternalUserStore
from switch_core.db.stores.reference_store import ReferenceStore
from switch_core.db.stores.room_role_store import RoomRoleStore
from switch_core.db.stores.room_store import RoomStore
+from switch_core.db.tenant_lookup import tenant_of_client
from switch_core.delivery.addressing import (
ADDRESSING_DENIED_MESSAGE,
ADDRESSING_UNCLAIMED_MESSAGE,
@@ -300,14 +301,18 @@ class AgentClient(ClientBase[ClientConfig]):
async def start(self) -> None:
"""Resolve the agent this client is, then run.
- Unscoped, deliberately: this runs at the top of the client's own task,
- which binds nothing, and it is the lookup that says which tenant this
- client belongs to at all. `clients.id` is globally unique, so there is
- nothing for a tenant to disambiguate — while a guessed one would turn
- "this client is an agent" into "no agent found for client" and fail
- the client at boot over a row that exists.
+ This runs at the top of the client's own task, which binds nothing, so
+ the tenant is derived rather than inherited: `clients.id` is a primary
+ key, and the exemption (`db/tenant_lookup.py`) turns it into the
+ tenant that row is in. A *guessed* tenant would turn "this client is
+ an agent" into "no agent found for client" and fail the client at boot
+ over a row that exists, which is why this is a lookup rather than a
+ default.
"""
- async with unscoped_session(self.session_factory) as session:
+ tenant_id = await tenant_of_client(self.session_factory, self.client_id)
+ if tenant_id is None:
+ raise RuntimeError(f"No client row for: {self.client_id}")
+ async with tenant_session(self.session_factory, tenant_id) as session:
agent = await self._agent_store.get_by_client_id(session, self.client_id)
if agent is None:
raise RuntimeError(f"No agent found for client: {self.client_id}")
diff --git a/core/switch_core/clients/client_lifecycle_service.py b/core/switch_core/clients/client_lifecycle_service.py
index eedefef4b..2a19927c2 100644
--- a/core/switch_core/clients/client_lifecycle_service.py
+++ b/core/switch_core/clients/client_lifecycle_service.py
@@ -11,9 +11,10 @@ from switch_core.clients.client_base import ClientBase, ClientConfig
from switch_core.clients.client_factory import ClientFactory
from switch_core.config import SwitchConfig
from switch_core.db.models import Client, Tenant
-from switch_core.db.session_scope import unscoped_session
+from switch_core.db.session_scope import tenant_session
from switch_core.db.stores.client_store import ClientStore
from switch_core.db.stores.tenant_store import TenantStore
+from switch_core.db.tenant_lookup import all_tenant_ids
from switch_core.provisioning import Provisioning
from switch_core.tenant_context import no_tenant, tenant_scope
@@ -63,17 +64,35 @@ class ClientLifecycleService:
them or relay system messages, so "does one exist anywhere" is the
wrong question to ask: it answers yes for a tenant that has none.
- The enumeration is unscoped by nature — which tenants exist is exactly
- what a bound session cannot see — and each row is then created with
- that tenant bound, so the `Client` picks it up from the same default
- every other scoped write uses instead of being handed a constant.
+ Which tenants exist is exactly what a bound session cannot see, so it
+ comes from `all_tenant_ids` — one of the eight `SECURITY DEFINER`
+ lookups that make up the whole exemption from row-level security
+ (`db/tenant_lookup.py`). Everything after that is an ordinary scoped
+ read, one tenant at a time, and each row is created with that tenant
+ bound so the `Client` picks it up from the same default every other
+ scoped write uses instead of being handed a constant.
+
+ A pass per tenant rather than one read across all of them: the
+ enumeration this replaced could see every tenant's clients at once
+ only because the connection was the tables' owner. Under the runtime
+ role it read nothing, and this whole method silently created a second
+ admin client for tenant zero on every boot.
"""
- async with unscoped_session(self._session_factory) as session:
- tenant_ids = await self._tenant_store.get_all_ids(session)
- already_served = {
- client.tenant_id
- for client in await self._client_store.get_by_type(session, client_type)
- }
+ tenant_ids = await all_tenant_ids(self._session_factory)
+ already_served = set()
+ for tenant_id in tenant_ids:
+ async with tenant_session(self._session_factory, tenant_id) as session:
+ # `get_by_type` carries no tenant filter of its own and leans
+ # on the policy for it, so on an owner connection it answers
+ # with every tenant's clients and marks every tenant served
+ # as soon as one of them is. See `db/tenant_lookup.py`.
+ if any(
+ client.tenant_id == tenant_id
+ for client in await self._client_store.get_by_type(
+ session, client_type
+ )
+ ):
+ already_served.add(tenant_id)
localpart = f"switch-{client_type.replace('_', '-')}"
for tenant_id in tenant_ids:
@@ -104,25 +123,44 @@ class ClientLifecycleService:
`ensure_system_client` rather than duplicating its per-type,
per-tenant provisioning logic: the new tenant is simply the one gap
that enumeration has not filled yet.
+
+ The insert binds the new tenant's *own* id, which is the one binding
+ that satisfies `tenants`' policy: it compares on `id` rather than on a
+ `tenant_id` column, because a tenant is the boundary rather than
+ something inside one. So creating a tenant needs no exemption at all —
+ the row is written by a session scoped to exactly the tenant being
+ created, and to nothing else. The id is generated here rather than by
+ the database for that reason.
"""
- tenant = Tenant(name=name, slug=slug)
- async with unscoped_session(self._session_factory) as session:
+ tenant = Tenant(id=str(uuid.uuid4()), name=name, slug=slug)
+ async with tenant_session(self._session_factory, tenant.id) as session:
await self._tenant_store.create(session, tenant)
await session.commit()
await self.ensure_system_client("admin")
return tenant
async def start_all(self) -> None:
- # Every tenant's clients in one pass at boot, so this read is
- # unscoped by nature. Nothing is bound around `_start_task`: a client
- # is not a single-tenant actor for the purposes of its own task. It
- # reads which rooms it is in — a lookup keyed by a globally unique
- # client id — and then works one room at a time, binding that room's
- # tenant per delivery. A boot-time binding would only decide what the
- # task snapshots, and the whole point is that nothing downstream may
- # depend on that.
- async with unscoped_session(self._session_factory) as session:
- records = await self._client_store.get_all(session)
+ # Every tenant's clients at boot, read one tenant at a time. The
+ # enumeration is `all_tenant_ids` — the exemption answers which
+ # tenants there are, and each tenant's clients are then an ordinary
+ # scoped read (`db/tenant_lookup.py`).
+ #
+ # Nothing is bound around `_start_task`, and that is unchanged: a
+ # client's own task derives its tenant from its client row rather than
+ # inheriting one, so a binding here would only decide what the task
+ # snapshots, which is exactly what must not matter.
+ records: list[Client] = []
+ for tenant_id in await all_tenant_ids(self._session_factory):
+ async with tenant_session(self._session_factory, tenant_id) as session:
+ # Filtered on the row's own tenant, not left to the policy: on an
+ # owner connection no policy narrows this read, and the fan-out
+ # would act on every tenant's rows once per tenant. See
+ # `db/tenant_lookup.py`, "a fan-out ... filters what it reads back".
+ records.extend(
+ record
+ for record in await self._client_store.get_all(session)
+ if record.tenant_id == tenant_id
+ )
records = [r for r in records if r.type not in self.COLLAB_CLIENT_TYPES]
diff --git a/core/switch_core/config.py b/core/switch_core/config.py
index d3e85b167..f6afc3d22 100644
--- a/core/switch_core/config.py
+++ b/core/switch_core/config.py
@@ -19,6 +19,35 @@ class SwitchConfig(BaseSettings):
db_password: str
db_name: str
+ # The schema owner, used for exactly two things and never to serve a
+ # request: running Alembic, and re-issuing the runtime role's grants right
+ # after it, so a table a migration has just added is readable by the role
+ # that is about to need it.
+ #
+ # `DB_USER` above is the *runtime* role, the one the row-level-security
+ # policies apply to. It deliberately owns nothing and can create nothing,
+ # so it cannot run a migration — that is what it is for, not an oversight
+ # — and granting it schema rights so that boot-time migrations keep
+ # working would hand back the ownership exemption the policies rely on it
+ # not having.
+ #
+ # Unset means "this deployment migrates somewhere else": boot applies no
+ # migrations and expects the schema to already be at head.
+ db_owner_user: str | None = None
+ db_owner_password: str | None = None
+
+ # Refuse to serve when the runtime connection is not actually subject to
+ # the policies — a superuser, a `BYPASSRLS` role, or the owner of the
+ # scoped tables. On by default because the failure it catches is silent: a
+ # Switch that believes it is isolating tenants and is not looks exactly
+ # like one that is, right up until a second customer reads the first's
+ # rooms.
+ #
+ # Set false only for a deployment that has not created its runtime role
+ # yet. Boot then logs at `error` on every start, because that is a
+ # deployment with no tenant isolation in it.
+ db_require_restricted_role: bool = True
+
# The server half of every client's `@localpart:server` id. Not a
# homeserver address — nothing is contacted at it — but the ids are stable
# public handles, so the shape outlives the homeserver that chose it.
@@ -318,11 +347,32 @@ class SwitchConfig(BaseSettings):
@property
def database_url(self) -> str:
+ """The runtime connection: the restricted role that serves every request."""
return (
f"postgresql+asyncpg://{self.db_user}:{self.db_password}"
f"@{self.db_host}:{self.db_port}/{self.db_name}"
)
+ @property
+ def owner_database_url(self) -> str | None:
+ """The schema owner's connection, or None if this deployment has none.
+
+ Same host, port and database as the runtime connection — only the role
+ differs. Two roles rather than two databases is the entire shape: the
+ owner exists so that something can run DDL, and the runtime role exists
+ so that nothing serving a request can.
+ """
+ if self.db_owner_user is None or self.db_owner_password is None:
+ return None
+ return (
+ f"postgresql+asyncpg://{self.db_owner_user}:{self.db_owner_password}"
+ f"@{self.db_host}:{self.db_port}/{self.db_name}"
+ )
+
+ @property
+ def runs_migrations_at_boot(self) -> bool:
+ return self.owner_database_url is not None
+
@property
def db_connect_args(self) -> dict[str, object]:
"""asyncpg connect args derived from config.
diff --git a/core/switch_core/db/models.py b/core/switch_core/db/models.py
index 4685137ea..d09ecfd0e 100644
--- a/core/switch_core/db/models.py
+++ b/core/switch_core/db/models.py
@@ -29,6 +29,7 @@ from switch_core.db.notify_ddl import (
DROP_NOTIFY_TRIGGER,
)
from switch_core.db.rls_ddl import attach_row_level_security
+from switch_core.db.tenant_lookup import attach_tenant_lookups
from switch_core.tenant_context import current_tenant_id
@@ -94,10 +95,10 @@ class TenantNotBoundError(RuntimeError):
Raised from Python, before the row ever reaches the database, so the
failure points at the write that forgot to bind rather than at whatever
- row-level-security error Postgres would otherwise raise first. Any code
- that means to write across tenants uses `unscoped_session` and passes
- `tenant_id` explicitly (system seeding, bootstrap); this exists for
- everything else, which is supposed to have one bound by now.
+ row-level-security error Postgres would otherwise raise first. There is no
+ longer any such thing as a cross-tenant write: work that spans tenants
+ enumerates them through `db/tenant_lookup.py` and then binds each in turn,
+ so every write has a tenant by the time it is constructed.
"""
@@ -119,7 +120,7 @@ def require_tenant_id() -> str:
bind one by now: the long-lived background tasks unbind deliberately
(`switch_core.tenant_context.no_tenant`) and then bind the tenant of the
row they are about to act on before they act on it, and the raw-session
- inventory in `tests/switch_core/db/test_unscoped_session_allowlist.py`
+ inventory in `tests/switch_core/db/test_tenant_exemption_allowlist.py`
pins which modules may still open a session with nothing bound at all.
A write reached from one of those without an explicit `tenant_id` is
exactly the gap this now refuses to paper over.
@@ -1587,3 +1588,9 @@ event.listen(
# shape; a migration carries its own frozen copy for the same reason the
# notify trigger's migration does.
attach_row_level_security(Base.metadata)
+
+# And the eight functions that are exempt from those policies, on the same
+# reasoning again: a restricted role cannot boot without them, so a schema
+# `create_all` built without them is not the schema the server runs against.
+# After the tables, not before — see `db/tenant_lookup.py`.
+attach_tenant_lookups(Base.metadata)
diff --git a/core/switch_core/db/runtime_role.py b/core/switch_core/db/runtime_role.py
new file mode 100644
index 000000000..5efc674f8
--- /dev/null
+++ b/core/switch_core/db/runtime_role.py
@@ -0,0 +1,314 @@
+"""Making the policies bite: the restricted runtime role, and proving it is one.
+
+Every policy in `db/rls_ddl.py` is inert against a superuser or a table owner
+— Postgres exempts both, the first unconditionally and the second because this
+design deliberately never sets `force row level security` (which is also what
+lets the exempt lookups in `db/tenant_lookup.py` work at all). So the schema
+half of tenant isolation is only worth what the connection is: a deployment
+that installs 38 policies and then connects as the owner has none of them.
+
+Two halves live here.
+
+**Grants.** The runtime role owns nothing and can create nothing, so it needs
+to be given access to every table the owner created. Re-issued on every boot,
+immediately after the migration that may have added one, rather than once by
+hand: `ALTER DEFAULT PRIVILEGES` covers a table created *later by the owner*
+and is easy to get subtly wrong (it is per-granting-role, and silently does
+nothing for objects an unnamed role creates), whereas a fresh `GRANT ... ON
+ALL TABLES` after `alembic upgrade head` cannot fall behind the schema it was
+just run against. It is idempotent and costs one round trip at boot.
+
+**The self-check.** Four questions asked of the runtime connection before the
+server listens, because this failure mode is silent and CI cannot ask it: CI
+has no production connection, and a Switch that believes it is isolating
+tenants and is not looks exactly like one that is. In order of how much they
+prove:
+
+1. The role is not a superuser and carries no `BYPASSRLS`, nor is it a member
+ of a role that has either — membership is a `SET ROLE` away from both.
+2. It owns none of the tables carrying a policy.
+3. No scoped table has `force row level security`, which would take the
+ ownership exemption away from the lookup functions and leave the process
+ unable to resolve a credential at all.
+4. Every tenant lookup exists and this role may execute it, since a role that
+ cannot would authenticate nobody.
+5. **A read of `tenants` with nothing bound raises.** This is the one that
+ actually proves it, because the others are inferences from the catalogue
+ and this is the behaviour. `tenants` is the table to ask it of: the check
+ is only meaningful against a populated table — Postgres does not evaluate a
+ policy for a scan that yields no rows, so an empty table answers "no error"
+ whether or not the connection is exempt — and `tenants` is the one scoped
+ table guaranteed to hold a row, since tenant zero is inserted by the same
+ migration that creates it. Asked last, and on a connection of its own,
+ because a statement that raises aborts its transaction.
+
+A failure raises and the process exits. `DB_REQUIRE_RESTRICTED_ROLE=false`
+turns the check into an `error` log for a deployment that has not created its
+role yet; nothing turns it into silence.
+"""
+
+from __future__ import annotations
+
+import logging
+
+from sqlalchemy import text
+from sqlalchemy.exc import DBAPIError
+from sqlalchemy.ext.asyncio import AsyncConnection, AsyncEngine
+
+from switch_core.db.tenant_lookup import TENANT_LOOKUPS
+
+logger = logging.getLogger(__name__)
+
+# What `require_tenant_id()` raises with, on both counts. The self-check
+# insists on the code *and* the message, and the second is not belt and
+# braces: SQLSTATE 42501 is `insufficient_privilege`, which is also what
+# Postgres returns for `permission denied for table tenants`. A role that had
+# simply never been granted anything would otherwise satisfy the probe by
+# failing for an unrelated reason — the exact false pass this check exists to
+# prevent, arrived at from the other direction.
+TENANT_NOT_SET_SQLSTATE = "42501"
+TENANT_NOT_SET_MESSAGE = "app.tenant_id is not set"
+
+
+class RuntimeRoleError(RuntimeError):
+ """The runtime connection is not subject to the row-level-security policies."""
+
+
+async def grant_runtime_role(owner: AsyncConnection, role: str) -> None:
+ """Give `role` everything a runtime role needs and nothing more.
+
+ Run as the owner, after `alembic upgrade head`. `EXECUTE ON ALL FUNCTIONS`
+ covers `require_tenant_id()` and the tenant lookups without naming them,
+ so a ninth lookup needs no change here.
+
+ No `CREATE` on the schema, and no ownership: the role is meant to be
+ unable to alter the tables it reads, which is what keeps it subject to
+ their policies.
+ """
+ identifier = await _quoted_identifier(owner, role)
+ for statement in (
+ f"GRANT USAGE ON SCHEMA public TO {identifier}",
+ f"GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO {identifier}",
+ f"GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO {identifier}",
+ f"GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA public TO {identifier}",
+ f"ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO {identifier}",
+ f"ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT USAGE, SELECT ON SEQUENCES TO {identifier}",
+ f"ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT EXECUTE ON FUNCTIONS TO {identifier}",
+ ):
+ await owner.execute(text(statement))
+
+
+async def _quoted_identifier(connection: AsyncConnection, name: str) -> str:
+ """`name` as a SQL identifier, quoted by Postgres rather than by us.
+
+ A role name arrives from configuration and lands in `GRANT`, which takes
+ no bind parameter for it. `quote_ident` is the server's own quoting, so
+ there is no second implementation of it here to get wrong.
+ """
+ result = await connection.execute(text("SELECT quote_ident(:name)"), {"name": name})
+ return str(result.scalar_one())
+
+
+async def verify_restricted_role(engine: AsyncEngine) -> None:
+ """Raise `RuntimeRoleError` unless this connection is subject to the policies."""
+ async with engine.connect() as connection:
+ role = str((await connection.execute(text("SELECT current_user"))).scalar_one())
+ await _refuse_privileged_role(connection, role)
+ await _refuse_table_owner(connection, role)
+ await _refuse_forced_row_level_security(connection)
+ await _require_the_lookups(connection, role)
+ # The probe is the one check that provokes an error on purpose, and a
+ # statement that raises aborts the transaction it ran in — every query
+ # after it on that connection fails with "current transaction is aborted"
+ # rather than with what it was asking. So it goes last and gets a
+ # connection of its own, and the checks above are the ones whose failures
+ # a reader will see first.
+ async with engine.connect() as connection:
+ await _refuse_a_connection_no_policy_stops(connection, role)
+ logger.info(
+ "Connected as %s: not a superuser, owns no policied table, and an "
+ "unbound read of a scoped table is refused. Tenant isolation is in "
+ "force on this connection.",
+ role,
+ )
+
+
+async def _refuse_privileged_role(connection: AsyncConnection, role: str) -> None:
+ result = await connection.execute(
+ text(
+ """
+ SELECT coalesce(bool_or(r.rolsuper), false) AS is_super,
+ coalesce(bool_or(r.rolbypassrls), false) AS bypasses
+ FROM pg_roles r
+ WHERE pg_has_role(current_user, r.oid, 'MEMBER')
+ """
+ )
+ )
+ is_super, bypasses = result.one()
+ session_is_super = (
+ await connection.execute(text("SELECT current_setting('is_superuser')"))
+ ).scalar_one() == "on"
+ if is_super or bypasses or session_is_super:
+ raise RuntimeRoleError(
+ f"The database role {role!r} is a superuser or carries BYPASSRLS "
+ "(directly or through a role it is a member of), so every "
+ "row-level-security policy in this schema is inert for it and "
+ "tenants are not isolated. Point DB_USER at a plain LOGIN role "
+ "with NOSUPERUSER NOBYPASSRLS that owns nothing, and give the "
+ "schema owner's credentials as DB_OWNER_USER / DB_OWNER_PASSWORD "
+ "so migrations and grants still have somewhere to run."
+ )
+
+
+async def _refuse_table_owner(connection: AsyncConnection, role: str) -> None:
+ result = await connection.execute(
+ text(
+ """
+ SELECT c.relname
+ FROM pg_class c
+ JOIN pg_namespace n ON n.oid = c.relnamespace
+ WHERE n.nspname = 'public'
+ AND c.relrowsecurity
+ AND pg_get_userbyid(c.relowner) = current_user
+ ORDER BY c.relname
+ """
+ )
+ )
+ owned = [row[0] for row in result]
+ if owned:
+ raise RuntimeRoleError(
+ f"The database role {role!r} owns {len(owned)} of the tables "
+ f"carrying a tenant-isolation policy (for example {owned[:3]}). "
+ "Postgres exempts a table's owner from its own policies unless "
+ "FORCE ROW LEVEL SECURITY is set, which this design deliberately "
+ "does not set, so those policies do nothing on this connection. "
+ "Run the service as a role that owns no table."
+ )
+
+
+async def _refuse_forced_row_level_security(connection: AsyncConnection) -> None:
+ result = await connection.execute(
+ text(
+ """
+ SELECT c.relname
+ FROM pg_class c
+ JOIN pg_namespace n ON n.oid = c.relnamespace
+ WHERE n.nspname = 'public' AND c.relforcerowsecurity
+ ORDER BY c.relname
+ """
+ )
+ )
+ forced = [row[0] for row in result]
+ if forced:
+ raise RuntimeRoleError(
+ f"FORCE ROW LEVEL SECURITY is set on {forced}. The tenant lookups "
+ "in db/tenant_lookup.py are SECURITY DEFINER functions owned by "
+ "the schema owner, and they resolve a caller's tenant precisely "
+ "because the owner is exempt from these policies. Forcing the "
+ "policies onto the owner leaves nothing able to answer 'which "
+ "tenant is this credential in', so no request can authenticate."
+ )
+
+
+async def _refuse_a_connection_no_policy_stops(
+ connection: AsyncConnection, role: str
+) -> None:
+ """Read a populated scoped table with nothing bound; insist it is refused.
+
+ The only check here that observes behaviour rather than inferring it from
+ the catalogue, and so the only one that would catch an exemption nothing
+ above thought to look for.
+ """
+ try:
+ rows = (
+ await connection.execute(text("SELECT count(*) FROM tenants"))
+ ).scalar_one()
+ except DBAPIError as exc:
+ if _sqlstate(exc) == TENANT_NOT_SET_SQLSTATE and TENANT_NOT_SET_MESSAGE in str(
+ exc
+ ):
+ return
+ raise RuntimeRoleError(
+ f"Reading `tenants` as {role!r} with no tenant bound failed, but "
+ f"not with {TENANT_NOT_SET_SQLSTATE} "
+ f"{TENANT_NOT_SET_MESSAGE!r}: {exc}. That is not the policy "
+ "refusing the read, so it says nothing about whether tenants are "
+ "isolated — most likely the role is missing its grants, which "
+ "raises the same SQLSTATE for an entirely different reason."
+ ) from exc
+ raise RuntimeRoleError(
+ f"Reading `tenants` as {role!r} with no tenant bound returned {rows} "
+ "row(s) instead of raising. Something exempts this connection from "
+ "the tenant-isolation policies that the checks above did not catch, "
+ "so this deployment is not isolating tenants."
+ )
+
+
+async def _require_the_lookups(connection: AsyncConnection, role: str) -> None:
+ """The exempt lookups must be callable, or nothing can resolve a tenant.
+
+ Checked at boot rather than left to the first request, because the first
+ request is an authentication and the failure would arrive as a 401 for
+ every caller with no explanation of why.
+ """
+ expected = {lookup.name for lookup in TENANT_LOOKUPS}
+ rows = await connection.execute(
+ text(
+ """
+ SELECT p.proname, has_function_privilege(p.oid, 'EXECUTE')
+ FROM pg_proc p
+ JOIN pg_namespace n ON n.oid = p.pronamespace
+ WHERE n.nspname = 'public' AND p.proname = ANY(:names)
+ """
+ ),
+ {"names": sorted(expected)},
+ )
+ # Existence and privilege in one query rather than
+ # `has_function_privilege('name(text)', …)` per lookup: that form raises
+ # rather than answering false when the function is absent, so a schema
+ # missing the migration would fail with "function does not exist" instead
+ # of the explanation below.
+ privileges = {name: bool(allowed) for name, allowed in rows}
+ missing = sorted(expected - set(privileges))
+ if missing:
+ raise RuntimeRoleError(
+ f"{missing} are not installed in this database. They are the "
+ "tenant lookups every credential resolution goes through "
+ "(db/tenant_lookup.py), so nothing in this process could "
+ "authenticate a caller. The schema is behind the migration that "
+ "creates them."
+ )
+ refused = sorted(name for name, allowed in privileges.items() if not allowed)
+ if refused:
+ raise RuntimeRoleError(
+ f"{role!r} cannot execute {refused}, which are the tenant lookups "
+ "every credential resolution goes through (db/tenant_lookup.py). "
+ "Nothing in this process could authenticate a caller. Grant "
+ "EXECUTE, or let boot do it by configuring DB_OWNER_USER / "
+ "DB_OWNER_PASSWORD."
+ )
+ found = (
+ await connection.execute(text("SELECT count(*) FROM all_tenant_ids()"))
+ ).scalar_one()
+ if not found:
+ raise RuntimeRoleError(
+ "all_tenant_ids() answered with no tenants at all. Either the "
+ "schema predates the migration that seeds tenant zero, or the "
+ "function is not the SECURITY DEFINER one this process expects — "
+ "either way nothing here can be scoped to a tenant."
+ )
+
+
+def _sqlstate(exc: DBAPIError) -> str | None:
+ """The SQLSTATE behind a driver error, however this driver spells it.
+
+ asyncpg exposes `sqlstate`; SQLAlchemy's own wrapper exposes `pgcode` on
+ some drivers. Reading both rather than one keeps the check from silently
+ degrading into "any error will do" if the driver ever changes.
+ """
+ original = exc.orig
+ for attribute in ("sqlstate", "pgcode"):
+ value = getattr(original, attribute, None)
+ if value is not None:
+ return str(value)
+ return None
diff --git a/core/switch_core/db/session_scope.py b/core/switch_core/db/session_scope.py
index a60ca40d2..5e698311f 100644
--- a/core/switch_core/db/session_scope.py
+++ b/core/switch_core/db/session_scope.py
@@ -1,52 +1,50 @@
-"""The two ways background code may open a session, named so the call site
-says which one it means.
+"""How background code opens a session: bound to a tenant, and nothing else.
Everything in a request already gets a tenant for free: `gateway/auth.py` and
`bridges/agent/auth.py` bind one before the endpoint body runs, and the
`after_begin` hook (`db/tenant_session.py`) stamps it on every transaction
that session opens from then on, at any depth, with no call site to remember.
-Nothing does that for the 157 places that open a session from the factory
-directly with no request behind them — the delivery loop, the
-collaboration and server-connector lifecycle services, the startup seeding in
-`main.py`, the periodic sweeps. Each needs to say, at the point it opens a
-session, which of two things it is doing:
-
-- **`tenant_session`** binds a tenant for the life of one session, the same
- way a request does. This is the answer for a unit of work that acts on one
- row: a message delivery, an inbound bridge event, one bridge's startup, one
- row out of a cross-tenant sweep. Derive the tenant from the row the work is
- actually for — the room, the bridge, the connector — at the point the work
- happens. Never once per long-lived object, and never once per task: see
- `tenant_context.no_tenant` for why.
-
-- **`unscoped_session`** opens a session with **no tenant bound at all**, for
- the duration of the block, whatever the caller had bound going in. It is
- the fail-open hatch inside an otherwise fail-closed design, and it exists
- for two shapes of work:
-
- - reading every row before fanning out work per row — a sweep, a lifecycle
- enumeration, startup seeding that runs before any tenant exists;
- - a lookup whose whole job is to answer *which* tenant something is in, or
- that is keyed by something globally unique and spans tenants by nature:
- resolving a room from its transport id, or a client's rooms from its id.
- Inheriting a caller's tenant here is not a smaller answer, it is a wrong
- one.
-
- The unbinding is the point, and it is why this is not a synonym for calling
- the factory. A helper that merely *named* the intent while inheriting
- whatever was ambient would have the hook stamp the caller's tenant onto the
- transaction, and the cross-tenant read the call site asked for would
- silently be a single-tenant one. Its callers are pinned to an explicit
- allowlist by `tests/switch_core/db/test_unscoped_session_allowlist.py`,
- which fails on a new one and says what to do about it. Reach for
- `tenant_session` first.
-
-Both are thin: the tenant binding is a contextvar (`tenant_context.py`), and
-the `after_begin` hook is what actually turns it into `set_config` on the
-transaction. Opening a session through either helper does no I/O by itself,
-same as calling the factory directly — the hook only fires once a transaction
-begins, on that session's first query.
+`tenant_session` is that, for the 157 places that open a session from the
+factory with no request behind them — the delivery loop, the collaboration and
+server-connector lifecycle services, the startup seeding in `main.py`, the
+periodic sweeps. It binds a tenant for the life of one session, and the tenant
+is derived from the row the work is acting on: a message delivery binds the
+room's, an inbound bridge event binds the room's or the bridge's, one row out
+of a sweep binds its own. Never once per long-lived object, and never once per
+task — see `tenant_context.no_tenant` for the four ways that went wrong.
+
+**There was a second helper here, `unscoped_session`, and it is gone.** It
+opened a session with no tenant bound, for work that was legitimately
+cross-tenant: a boot enumeration, a sweep, a lookup whose job was to answer
+*which* tenant something was in. It read across every tenant, exactly as
+documented — but only because every environment connected to Postgres as the
+tables' owner, which Postgres exempts from their policies. Under the
+restricted runtime role an unscoped session is not a hatch at all: unscoped is
+precisely the state `require_tenant_id()` raises on, so every one of its
+seventeen call sites either died at boot or silently read nothing. The whole
+model was inverted, and `db/tenant_lookup.py` is what replaced it: eight
+`SECURITY DEFINER` functions that answer *which tenant* and never return a
+row, so a cross-tenant question is asked in one place with a fixed shape, and
+the work it fans out into is scoped like everything else.
+
+What is left is the one helper. Two things still open a session with nothing
+bound, and neither is cross-tenant:
+
+- `db/tenant_lookup.py` itself, which touches only its own exempt functions;
+- the reads of `users` and `oidc_identities` in `gateway/auth.py` and
+ `main.py`. Those tables carry no tenant and no policy — a person is not a
+ tenant member — so an unbound session is the honest way to read them and
+ there is nothing for a tenant to narrow.
+
+`tests/switch_core/db/test_tenant_exemption_allowlist.py` pins both surfaces:
+who may reach the exemption, and which modules may open a session straight
+from the factory at all.
+
+The binding is a contextvar (`tenant_context.py`), and the `after_begin` hook
+is what turns it into `set_config` on the transaction. Opening a session does
+no I/O by itself — the hook only fires once a transaction begins, on that
+session's first query.
"""
from __future__ import annotations
@@ -56,7 +54,7 @@ from contextlib import asynccontextmanager
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
-from switch_core.tenant_context import no_tenant, tenant_scope
+from switch_core.tenant_context import tenant_scope
@asynccontextmanager
@@ -68,26 +66,13 @@ async def tenant_session(
Binds before the session is constructed and unbinds on the way out, so a
long-lived caller (a bridge's own task, say) cannot leak this tenant into
whatever it does next even if the block raises.
- """
- with tenant_scope(tenant_id):
- async with session_factory() as session:
- yield session
-
-@asynccontextmanager
-async def unscoped_session(
- session_factory: async_sessionmaker[AsyncSession],
-) -> AsyncIterator[AsyncSession]:
- """Open a session with no tenant bound — the fail-open hatch.
-
- Unbinds for the duration of the block and restores the caller's binding on
- the way out, so a query issued here reads across every tenant *whether or
- not* the caller had one bound. That is the whole contract: a call site
- that says "unscoped" and then quietly ran scoped, because it was reached
- from a request or from inside a `tenant_scope`, would be the worst
- available outcome — an allowlist certifying a cross-tenant read that never
- happened.
+ Before the session is constructed matters more than it reads. The
+ `set_config` rides `after_begin`, so a `tenant_scope` entered *inside* an
+ already-open transaction changes nothing about that transaction — the
+ startup bootstrap seeding did exactly that and wrote a `tenant_members`
+ row on a connection that had never been told which tenant it was for.
"""
- with no_tenant():
+ with tenant_scope(tenant_id):
async with session_factory() as session:
yield session
diff --git a/core/switch_core/db/stores/tenant_member_store.py b/core/switch_core/db/stores/tenant_member_store.py
index c439fdd41..935c1ece0 100644
--- a/core/switch_core/db/stores/tenant_member_store.py
+++ b/core/switch_core/db/stores/tenant_member_store.py
@@ -1,9 +1,8 @@
from __future__ import annotations
-from sqlalchemy import select
-from sqlalchemy.ext.asyncio import AsyncSession
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
-from switch_core.db.models import TenantMember
+from switch_core.db.tenant_lookup import tenants_of_user
class TenantMembershipError(Exception):
@@ -35,18 +34,28 @@ class TenantMemberStore:
as firmly as it refuses to invent one out of zero.
"""
- async def get_sole_tenant_id(self, session: AsyncSession, user_id: str) -> str:
+ async def get_sole_tenant_id(
+ self,
+ session_factory: async_sessionmaker[AsyncSession],
+ user_id: str,
+ ) -> str:
"""The one tenant this user belongs to, raising if that isn't true.
- Must run on a session with no tenant bound yet — this *is* how the
- tenant gets found, so it cannot presuppose one. `gateway/auth.py`
- opens a short-lived session for exactly this and closes it before the
- request's own session is touched.
+ Takes the factory rather than a session, unlike every other store
+ method here, and the difference is the point: `tenant_members` is a
+ scoped table, so a session cannot read the row that would tell it what
+ to be scoped to. This goes through `tenants_of_user`, one of the eight
+ `SECURITY DEFINER` lookups that make up the whole exemption from
+ row-level security (`db/tenant_lookup.py`), which opens a session of
+ its own with nothing bound and answers with tenant ids and nothing
+ else.
+
+ It used to run a plain `select` on a session the caller opened and
+ left unbound. That worked only because every environment connected as
+ the tables' owner; under the restricted runtime role it is one of the
+ reads the policy refuses, and every gateway login failed on it.
"""
- result = await session.execute(
- select(TenantMember.tenant_id).where(TenantMember.user_id == user_id)
- )
- tenant_ids = result.scalars().all()
+ tenant_ids = await tenants_of_user(session_factory, user_id)
if len(tenant_ids) != 1:
raise TenantMembershipError(
f"user {user_id} has {len(tenant_ids)} tenant memberships; "
diff --git a/core/switch_core/db/stores/tenant_store.py b/core/switch_core/db/stores/tenant_store.py
index fdeeb6692..0cb753226 100644
--- a/core/switch_core/db/stores/tenant_store.py
+++ b/core/switch_core/db/stores/tenant_store.py
@@ -1,36 +1,36 @@
from __future__ import annotations
-from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from switch_core.db.models import Tenant
class TenantStore:
+ """Writes tenants. Reading which ones exist is not a store's question.
+
+ There was a `get_all_ids` here and it is gone. `tenants` carries the same
+ policy as every other scoped table, comparing on `id`, so a session sees
+ exactly its own row and no store method can enumerate them — the previous
+ one only appeared to because every caller handed it a session connected as
+ the tables' owner. The enumeration is `all_tenant_ids` in
+ `db/tenant_lookup.py`, one of the eight `SECURITY DEFINER` functions that
+ are the whole exemption from row-level security, and it takes a session
+ factory rather than a session precisely because there is no session it
+ could correctly run on.
+ """
+
async def create(self, session: AsyncSession, tenant: Tenant) -> Tenant:
"""Insert a tenant row.
Called from `ClientLifecycleService.create_tenant`, the one path that
creates a tenant in the running application and provisions what it
needs in the same call, rather than a row appearing — today, only by
- direct SQL — with nothing to notice it until the next restart."""
+ direct SQL — with nothing to notice it until the next restart.
+
+ The session it is handed must be bound to the tenant being created:
+ `tenants`' policy compares on `id`, so that binding is what satisfies
+ `with check`, and no other one can.
+ """
session.add(tenant)
await session.flush()
return tenant
-
- async def get_all_ids(self, session: AsyncSession) -> list[str]:
- """Every tenant in the deployment, oldest first.
-
- The one question a tenant-scoped session cannot answer: `tenants`
- carries the same policy as everything else, comparing on `id`, so a
- bound session sees exactly its own row. Callers that need the whole
- list — the startup work that has to do something once per tenant —
- ask on an `unscoped_session`.
-
- Ordered by creation so a fan-out over the result is deterministic and
- tenant zero, which predates every other row, comes first.
- """
- result = await session.execute(
- select(Tenant.id).order_by(Tenant.created_at, Tenant.id)
- )
- return list(result.scalars().all())
diff --git a/core/switch_core/db/tenant_lookup.py b/core/switch_core/db/tenant_lookup.py
new file mode 100644
index 000000000..21ed7fa0b
--- /dev/null
+++ b/core/switch_core/db/tenant_lookup.py
@@ -0,0 +1,360 @@
+"""The whole exemption from row-level security, written out as eight functions.
+
+Row-level security is enforced by `require_tenant_id()` (`db/rls_ddl.py`),
+which raises when no tenant is bound. That is the property everything else
+rests on, and it has one unavoidable consequence: **resolving who the caller
+is is what produces the tenant, so it necessarily runs unbound.** A bearer
+token names an `api_keys` row before anyone knows which tenant that row is in.
+A boot pass over every bridge has to see rows in tenants it has not bound yet.
+Both are reads the policy refuses.
+
+The design this replaces called that an "unscoped session" and left it at
+that. Under the owner connection every environment used, an unscoped session
+did read across tenants, so the model looked complete. Under a restricted
+role it is not a hatch at all — unscoped is precisely the state
+`require_tenant_id()` raises on — and a real deployment under a restricted
+role died at the first of nineteen such sites before it finished booting.
+
+**What is exempt is this module, and nothing else.** Each function below is
+`SECURITY DEFINER`, owned by the schema owner, and so runs outside the
+policies (Postgres exempts a table's owner from its own policies, and this
+design deliberately never sets `force row level security` — see
+`test_tenant_lookup.py`, which pins that). Two properties make the exemption
+worth having:
+
+- **They answer *which tenant*, never *what row*.** Every one returns
+ `setof text`: tenant ids. The most a caller can extract is the tenant an
+ identifier it already holds belongs to. Nothing crosses a tenant boundary
+ except the boundary's own name.
+- **They are a closed list.** `TENANT_LOOKUPS` below is the list; a test
+ compares it against the functions actually installed, against the
+ migration's frozen copy, and against what each one answers when called as
+ the restricted role. Adding a ninth is an edit a reviewer sees.
+
+Everything else in the process binds a tenant. The nineteen sites that could
+not are now one of three shapes, and the shape is the interesting part:
+
+1. **Credential resolution** — the bearer token, the JWT subject, the OIDC
+ client id, the registration token. Resolve the *tenant* here, bind it,
+ then read the row itself through the ordinary scoped store. The row never
+ arrives from an exempt path.
+2. **Cross-tenant enumeration** — the boot passes over every client, bridge,
+ connector and room, and the runtime-state sweep. `all_tenant_ids()` and
+ then one scoped pass per tenant. These sites already fanned out per row
+ and bound that row's tenant; the loop simply moved one level up, and the
+ read inside it is now subject to the policies like any other.
+3. **Per-item work that can derive its tenant** — a client's transport, a
+ bridge or connector being started by id, a room reached by id. One lookup
+ by the globally unique identifier the caller already has, and everything
+ after it is scoped.
+
+Why not the obvious alternatives is argued in
+`docs/old/multi-tenancy-phase1-db.md`, "The bootstrap problem"; the short
+version is that returning rows instead of tenant ids would put a second copy
+of a dozen store methods in SQL and widen the exemption from "a tenant id"
+to "any row of nine tables", and that a second engine connected as the owner
+would make the exemption ambient again — reachable from anything holding the
+factory, and no longer a list anybody can read.
+
+**A fan-out over `all_tenant_ids()` filters what it reads back.** Every such
+loop keeps only the rows whose own `tenant_id` matches the tenant it bound.
+That looks redundant — the policy has already narrowed the read — and it is
+not, because the policy is what does *not* apply on an owner connection. Most
+store methods here carry no `WHERE tenant_id = …` of their own and lean on
+row-level security for it, so on a connection Postgres exempts (the test
+suite's, and any deployment still running with
+`DB_REQUIRE_RESTRICTED_ROLE=false`) the same read returns every tenant's rows
+on every pass, and a fan-out over N tenants acts on each row N times. Silent
+duplication is exactly the failure this design refuses to ship, so the filter
+is written out rather than inferred from the connection.
+
+**Cost.** A lookup is its own short session and so its own round trip. On the
+paths where that would be per request or per message it is already behind a
+cache that predates this change (`ApiKeyCache`, the transport's room map, the
+bridge's room→tenant map), so the steady-state cost is one extra round trip
+per *new* credential, room or bridge rather than per use. `all_tenant_ids()`
+on the runtime-state sweep is one extra round trip every five seconds.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Iterable
+from dataclasses import dataclass
+
+from sqlalchemy import DDL, MetaData, event, text
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
+
+from switch_core.tenant_context import no_tenant
+
+
+class TenantLookupError(RuntimeError):
+ """A lookup that must name at most one tenant named several.
+
+ Only reachable for a key that is not unique by constraint — today only
+ `agents.oauth_client_id`, which has no unique index. Raised rather than
+ resolved by picking one, for the same reason `get_sole_tenant_id` refuses
+ to: a credential that resolves to two tenants is a provisioning fault, and
+ guessing which one it meant is how a caller ends up authenticated into
+ somebody else's data.
+ """
+
+
+@dataclass(frozen=True)
+class TenantLookup:
+ """One exempt function: its name, its single argument, and what it reads.
+
+ `argument` is the parameter name *without* the `p_` prefix the SQL carries.
+ The prefix is not decoration: a `LANGUAGE sql` function whose parameter is
+ spelled like a column of a table in its own query resolves the name to the
+ column, silently, so `WHERE id = client_id` against a table that has a
+ `client_id` column compares the column with itself and matches every row.
+ Prefixing every parameter makes that unrepresentable rather than a thing to
+ check per function.
+ """
+
+ name: str
+ argument: str | None
+ query: str
+ purpose: str
+
+ @property
+ def parameter(self) -> str | None:
+ return None if self.argument is None else f"p_{self.argument}"
+
+ @property
+ def signature(self) -> str:
+ return f"{self.name}({'' if self.argument is None else 'text'})"
+
+
+# The eight of them. Ordered as the three shapes above: enumeration, then
+# credential resolution, then deriving a tenant from an identifier in hand.
+TENANT_LOOKUPS: tuple[TenantLookup, ...] = (
+ TenantLookup(
+ name="all_tenant_ids",
+ argument=None,
+ query="SELECT id FROM tenants ORDER BY created_at, id",
+ purpose=(
+ "Every tenant in the deployment, oldest first. The one question a "
+ "scoped session cannot answer about itself, and the whole of what "
+ "the boot enumerations and the runtime-state sweep need."
+ ),
+ ),
+ TenantLookup(
+ name="tenants_of_user",
+ argument="user_id",
+ query=(
+ "SELECT tenant_id FROM tenant_members "
+ "WHERE user_id = p_user_id ORDER BY tenant_id"
+ ),
+ purpose=(
+ "Which tenants a gateway login belongs to. A person is global "
+ "(`users` carries no tenant); the membership row is the per-tenant "
+ "object, and it is scoped, so a session cannot read its own way in."
+ ),
+ ),
+ TenantLookup(
+ name="tenant_of_api_key",
+ argument="key_hash",
+ query="SELECT tenant_id FROM api_keys WHERE key_hash = p_key_hash",
+ purpose=(
+ "Which tenant a bearer credential belongs to. `api_keys.key_hash` "
+ "is one of the two columns deliberately left globally unique for "
+ "exactly this: the hash is resolved before a tenant exists."
+ ),
+ ),
+ TenantLookup(
+ name="tenant_of_agent_oauth_client",
+ argument="oauth_client_id",
+ query=(
+ "SELECT tenant_id FROM agents WHERE oauth_client_id = p_oauth_client_id"
+ ),
+ purpose=(
+ "Which tenant an agent authenticating by OIDC belongs to. The "
+ "column carries no unique index, so this is the one lookup that "
+ "can legitimately answer twice; the caller refuses rather than "
+ "picking."
+ ),
+ ),
+ TenantLookup(
+ name="tenant_of_client",
+ argument="client_id",
+ query="SELECT tenant_id FROM clients WHERE id = p_client_id",
+ purpose=(
+ "Which tenant a room client belongs to. Its transport's tenant for "
+ "everything it does: `client_rooms` carries composite foreign keys "
+ "to both `clients` and `rooms`, so a client's rooms are all in the "
+ "client's own tenant."
+ ),
+ ),
+ TenantLookup(
+ name="tenant_of_room",
+ argument="room_id",
+ query="SELECT tenant_id FROM rooms WHERE id = p_room_id",
+ purpose=(
+ "Which tenant a room belongs to, for work reaching a room by its "
+ "globally unique id with nothing bound."
+ ),
+ ),
+ TenantLookup(
+ name="tenant_of_collaboration_bridge",
+ argument="bridge_id",
+ query="SELECT tenant_id FROM collaboration_bridges WHERE id = p_bridge_id",
+ purpose=(
+ "Which tenant a bridge belongs to. Asked from boot, with nothing "
+ "bound, and from an HTTP request that restarted it, where what is "
+ "bound is the operator's tenant and not necessarily the bridge's."
+ ),
+ ),
+ TenantLookup(
+ name="tenant_of_server_connector",
+ argument="connector_id",
+ query="SELECT tenant_id FROM server_connectors WHERE id = p_connector_id",
+ purpose="Same as the bridge, for a server-side connector.",
+ ),
+)
+
+TENANT_LOOKUPS_BY_NAME: dict[str, TenantLookup] = {
+ lookup.name: lookup for lookup in TENANT_LOOKUPS
+}
+
+# `pg_temp` last, per the Postgres note on writing SECURITY DEFINER functions
+# safely: a role that can create a temporary table could otherwise shadow one
+# of the tables read above and have the function read theirs instead. `public`
+# is not writable by PUBLIC on 15+, which is the floor this schema already
+# requires for `ON DELETE SET NULL (col)`.
+SECURE_SEARCH_PATH = "pg_catalog, public, pg_temp"
+
+
+def create_lookup_ddl(lookup: TenantLookup) -> str:
+ parameters = "" if lookup.parameter is None else f"{lookup.parameter} text"
+ return (
+ f"CREATE OR REPLACE FUNCTION {lookup.name}({parameters})\n"
+ f" RETURNS SETOF text\n"
+ f" LANGUAGE sql STABLE SECURITY DEFINER\n"
+ f" SET search_path = {SECURE_SEARCH_PATH}\n"
+ f"AS $${lookup.query}$$"
+ )
+
+
+def drop_lookup_ddl(lookup: TenantLookup) -> str:
+ return f"DROP FUNCTION IF EXISTS {lookup.signature}"
+
+
+def attach_tenant_lookups(metadata: MetaData) -> None:
+ """Create every lookup after `create_all`, drop them before `drop_all`.
+
+ On the metadata's own events rather than a table's, and *after* create
+ rather than before: a `LANGUAGE sql` body is parsed and its names resolved
+ when the function is created, so every table it reads has to exist by
+ then. That is a feature — a lookup naming a column that has been renamed
+ fails at `create_all`, in every test, rather than at the first call in a
+ deployment.
+ """
+ for lookup in TENANT_LOOKUPS:
+ event.listen(
+ metadata,
+ "after_create",
+ DDL(create_lookup_ddl(lookup)).execute_if(dialect="postgresql"),
+ )
+ event.listen(
+ metadata,
+ "before_drop",
+ DDL(drop_lookup_ddl(lookup)).execute_if(dialect="postgresql"),
+ )
+
+
+# ── Calling them ──────────────────────────────────────────────────────────────
+
+
+async def _call(
+ session_factory: async_sessionmaker[AsyncSession],
+ lookup: TenantLookup,
+ argument: str | None = None,
+) -> list[str]:
+ """Run one lookup on a session of its own, with nothing bound.
+
+ Its own session, deliberately. The caller may have a tenant bound — the
+ gateway resolving a bridge's tenant from inside an operator's request does
+ — and the answer must not be narrowed to it. Nothing bound is also the
+ honest state: this session may not touch a scoped table, and the database
+ is what enforces that now rather than an allowlist.
+ """
+ parameters: dict[str, object] = {}
+ call = f"{lookup.name}()"
+ if lookup.parameter is not None:
+ parameters["argument"] = argument
+ call = f"{lookup.name}(:argument)"
+ with no_tenant():
+ async with session_factory() as session:
+ result = await session.execute(
+ text(f"SELECT tenant_id FROM {call} AS tenant_id"), parameters
+ )
+ return [row[0] for row in result]
+
+
+def _at_most_one(lookup: TenantLookup, tenant_ids: Iterable[str]) -> str | None:
+ found = list(tenant_ids)
+ if len(found) > 1:
+ raise TenantLookupError(
+ f"{lookup.name} resolved {len(found)} tenants ({sorted(found)}); "
+ "refusing to pick one. Whatever was looked up is present in more "
+ "than one tenant, which is a provisioning fault, not a credential "
+ "the caller may use."
+ )
+ return found[0] if found else None
+
+
+async def all_tenant_ids(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> list[str]:
+ return await _call(session_factory, TENANT_LOOKUPS_BY_NAME["all_tenant_ids"])
+
+
+async def tenants_of_user(
+ session_factory: async_sessionmaker[AsyncSession], user_id: str
+) -> list[str]:
+ return await _call(
+ session_factory, TENANT_LOOKUPS_BY_NAME["tenants_of_user"], user_id
+ )
+
+
+async def tenant_of_api_key(
+ session_factory: async_sessionmaker[AsyncSession], key_hash: str
+) -> str | None:
+ lookup = TENANT_LOOKUPS_BY_NAME["tenant_of_api_key"]
+ return _at_most_one(lookup, await _call(session_factory, lookup, key_hash))
+
+
+async def tenant_of_agent_oauth_client(
+ session_factory: async_sessionmaker[AsyncSession], oauth_client_id: str
+) -> str | None:
+ lookup = TENANT_LOOKUPS_BY_NAME["tenant_of_agent_oauth_client"]
+ return _at_most_one(lookup, await _call(session_factory, lookup, oauth_client_id))
+
+
+async def tenant_of_client(
+ session_factory: async_sessionmaker[AsyncSession], client_id: str
+) -> str | None:
+ lookup = TENANT_LOOKUPS_BY_NAME["tenant_of_client"]
+ return _at_most_one(lookup, await _call(session_factory, lookup, client_id))
+
+
+async def tenant_of_room(
+ session_factory: async_sessionmaker[AsyncSession], room_id: str
+) -> str | None:
+ lookup = TENANT_LOOKUPS_BY_NAME["tenant_of_room"]
+ return _at_most_one(lookup, await _call(session_factory, lookup, room_id))
+
+
+async def tenant_of_collaboration_bridge(
+ session_factory: async_sessionmaker[AsyncSession], bridge_id: str
+) -> str | None:
+ lookup = TENANT_LOOKUPS_BY_NAME["tenant_of_collaboration_bridge"]
+ return _at_most_one(lookup, await _call(session_factory, lookup, bridge_id))
+
+
+async def tenant_of_server_connector(
+ session_factory: async_sessionmaker[AsyncSession], connector_id: str
+) -> str | None:
+ lookup = TENANT_LOOKUPS_BY_NAME["tenant_of_server_connector"]
+ return _at_most_one(lookup, await _call(session_factory, lookup, connector_id))
diff --git a/core/switch_core/gateway/auth.py b/core/switch_core/gateway/auth.py
index 5f0794f20..c4b329fce 100644
--- a/core/switch_core/gateway/auth.py
+++ b/core/switch_core/gateway/auth.py
@@ -101,12 +101,18 @@ async def _resolve_tenant_id(
) -> str:
"""Which tenant the JWT's subject belongs to, before one is bound.
- Runs on its own session, opened and closed here, because the lookup is
- what *determines* the tenant and so cannot run on a session that is
- supposed to already carry it. Short-lived on purpose: held as a yield
- dependency instead, it would keep a second pooled connection — idle in
- transaction, since the lookup autobegins one nothing ever ends — for the
- whole request, halving effective pool capacity.
+ Two reads, and they are unbound for different reasons. `users` carries no
+ tenant at all — a person is not a tenant member — so the existence check
+ is an ordinary read on a session with nothing bound. The membership read
+ is not: `tenant_members` is scoped, and this is the lookup that decides
+ what to scope to, so it goes through the `SECURITY DEFINER` exemption
+ (`db/tenant_lookup.py`) rather than through a session the policy would
+ refuse.
+
+ Both are short-lived on purpose. Held as a yield dependency instead, they
+ would keep a second pooled connection — idle in transaction, since the
+ lookup autobegins one nothing ever ends — for the whole request, halving
+ effective pool capacity.
Nothing loaded here escapes: the caller re-reads the `User` from the
request's own session, so the object an endpoint mutates belongs to the
@@ -115,22 +121,22 @@ async def _resolve_tenant_id(
async with session_factory() as system_session:
if not await user_store.exists(system_session, user_id):
raise HTTPException(status_code=401, detail="User not found")
- try:
- return await tenant_member_store.get_sole_tenant_id(system_session, user_id)
- except TenantMembershipError as exc:
- # Phase 1 has exactly one tenant, so anything but one membership is
- # a provisioning bug, not a credential problem — but the caller
- # still deserves a legible answer instead of an opaque 500. The
- # detail names no user id: it is rendered to whoever is holding
- # the cookie, not to the operator, who gets the id from the log.
- logger.error("Cannot resolve a tenant for user %s: %s", user_id, exc)
- raise HTTPException(
- status_code=403,
- detail=(
- "This account is not a member of exactly one tenant; "
- "ask an administrator to check its membership."
- ),
- ) from exc
+ try:
+ return await tenant_member_store.get_sole_tenant_id(session_factory, user_id)
+ except TenantMembershipError as exc:
+ # Phase 1 has exactly one tenant, so anything but one membership is a
+ # provisioning bug, not a credential problem — but the caller still
+ # deserves a legible answer instead of an opaque 500. The detail names
+ # no user id: it is rendered to whoever is holding the cookie, not to
+ # the operator, who gets the id from the log.
+ logger.error("Cannot resolve a tenant for user %s: %s", user_id, exc)
+ raise HTTPException(
+ status_code=403,
+ detail=(
+ "This account is not a member of exactly one tenant; "
+ "ask an administrator to check its membership."
+ ),
+ ) from exc
async def get_current_user(
diff --git a/core/switch_core/main.py b/core/switch_core/main.py
index 2e0a729d2..eaaa0d6d7 100644
--- a/core/switch_core/main.py
+++ b/core/switch_core/main.py
@@ -14,7 +14,13 @@ import uvicorn
from alembic import command as alembic_command
from alembic.config import Config as AlembicConfig
from fastapi.responses import JSONResponse
-from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
+from sqlalchemy.ext.asyncio import (
+ AsyncEngine,
+ AsyncSession,
+ async_sessionmaker,
+ create_async_engine,
+)
+from sqlalchemy.pool import NullPool
from switch_core.bridges.agent.app import create_agent_bridge_app
from switch_core.bridges.agent.protocol.connections import (
@@ -76,7 +82,12 @@ from switch_core.db.engine import (
create_unpooled_engine,
)
from switch_core.db.models import TENANT_ZERO_ID, ApiKey, User
-from switch_core.db.session_scope import tenant_session, unscoped_session
+from switch_core.db.runtime_role import (
+ RuntimeRoleError,
+ grant_runtime_role,
+ verify_restricted_role,
+)
+from switch_core.db.session_scope import tenant_session
from switch_core.db.stores.agent_session_store import AgentSessionStore
from switch_core.db.stores.agent_store import AgentStore
from switch_core.db.stores.api_key_store import ApiKeyStore
@@ -99,6 +110,7 @@ from switch_core.db.stores.task_store import TaskStore
from switch_core.db.stores.tenant_member_store import TenantMemberStore
from switch_core.db.stores.tenant_store import TenantStore
from switch_core.db.stores.user_store import UserStore
+from switch_core.db.tenant_lookup import all_tenant_ids
from switch_core.gateway.app import create_gateway_app
from switch_core.gateway.auth import hash_password
from switch_core.logging_config import configure_logging
@@ -106,7 +118,7 @@ from switch_core.messages.notify import MessageListener
from switch_core.provisioning import Provisioning
from switch_core.provisioning.postgres import PostgresProvisioning
from switch_core.room_service import RoomService
-from switch_core.tenant_context import no_tenant, tenant_scope
+from switch_core.tenant_context import no_tenant
from switch_core.transport.ephemeral import EphemeralBus
from switch_core.transport.invites import InviteBus
from switch_core.version import switch_core_version
@@ -189,9 +201,59 @@ class _QuietPollFilter(logging.Filter):
logging.getLogger("uvicorn.access").addFilter(_QuietPollFilter())
+async def _prepare_database(config: SwitchConfig) -> None:
+ """Re-issue the runtime role's grants, on the owner's connection.
+
+ Runs after `alembic upgrade head` and before anything opens the runtime
+ engine, so a table the migration has just added is granted before the role
+ that needs it connects. Skipped where no owner is configured: that is a
+ deployment whose migrations and grants are somebody else's job, and
+ inventing an owner connection for it would be worse than doing nothing.
+ """
+ owner_url = config.owner_database_url
+ if owner_url is None:
+ return
+ owner_engine = create_async_engine(
+ owner_url, poolclass=NullPool, connect_args=config.db_connect_args
+ )
+ try:
+ async with owner_engine.begin() as connection:
+ await grant_runtime_role(connection, config.db_user)
+ finally:
+ await owner_engine.dispose()
+ logger.info(
+ "Granted the runtime role %s access to the schema owned by %s",
+ config.db_user,
+ config.db_owner_user,
+ )
+
+
+async def _check_tenant_isolation(config: SwitchConfig, engine: AsyncEngine) -> None:
+ """Refuse to serve on a connection the policies do not apply to.
+
+ Before anything else touches the database, because the first thing that
+ does is the admin seeding, and a deployment that is not isolating tenants
+ should not get as far as writing a row.
+ """
+ try:
+ await verify_restricted_role(engine)
+ except RuntimeRoleError as exc:
+ if config.db_require_restricted_role:
+ raise
+ logger.error(
+ "Tenant isolation is NOT in force on this deployment: %s "
+ "Continuing only because DB_REQUIRE_RESTRICTED_ROLE is false. "
+ "Every row-level-security policy in this schema is inert, and any "
+ "second tenant onboarded here can read the first's data.",
+ exc,
+ )
+
+
async def run(config: SwitchConfig) -> None:
# ── Database ─────────────────────────────────────────────────────────────
+ await _prepare_database(config)
engine = create_engine_from_config(config)
+ await _check_tenant_isolation(config, engine)
session_factory = create_session_factory(engine)
# Its connection is held rather than borrowed, so it builds its own outside
# the pool. Nothing subscribes yet; it starts with the server so that the
@@ -248,9 +310,9 @@ async def run(config: SwitchConfig) -> None:
# Once per tenant, not once for the deployment: `reference_types` is
# scoped, so "every stored type a built-in shadows" is a question asked of
# one tenant at a time. Which tenants there are is the one read that
- # cannot be scoped to any of them.
- async with unscoped_session(session_factory) as session:
- tenant_ids = await tenant_store.get_all_ids(session)
+ # cannot be scoped to any of them, so it goes through the exemption
+ # (`db/tenant_lookup.py`) rather than through a session.
+ tenant_ids = await all_tenant_ids(session_factory)
for tenant_id in tenant_ids:
async with tenant_session(session_factory, tenant_id) as session:
await resource_service.log_builtin_shadowing(session)
@@ -566,38 +628,69 @@ async def _seed_agent_registration_bootstrap_key(
already exists under the old one, which would collide on the unique
``key_hash`` and fail the whole boot.
"""
- # Genuinely cross-tenant, unlike `_seed_admin_user`: the bootstrap key is
- # one per deployment, resolved by a globally unique hash, and the
- # admin-owned-agent warning below must see every tenant's agents or it
- # under-reports exactly the case it exists to flag. So the session stays
- # unscoped and the two scoped rows written under it — the bootstrap
- # owner's membership, and the `ApiKey` — name tenant zero themselves
- # rather than inheriting a binding there isn't one of.
- async with unscoped_session(session_factory) as session:
+ # This was one unscoped session, and it is the one place in the tree where
+ # "unscoped session" and "scoped write" met — the design doc named it as
+ # the path the runtime role would have to come back for. It has, and this
+ # is what it became: three phases, each scoped to the tenant it is
+ # actually acting in, and the only cross-tenant question left ("which
+ # tenants are there") answered by the exemption in `db/tenant_lookup.py`.
+ tenant_ids = await all_tenant_ids(session_factory)
+
+ # ── The two accounts, in tenant zero ────────────────────────────────────
+ # `users` is global, so the lookups need no tenant; `ensure_bootstrap_owner`
+ # writes a `tenant_members` row, which does. It named tenant zero before
+ # and still does — the deployment's own accounts belong to the tenant the
+ # migration created — but it now does so on a session that was bound
+ # *before* its transaction opened, which is what makes the write land at
+ # all: a `tenant_scope` entered inside an already-begun transaction changes
+ # nothing, because the `set_config` rides `after_begin`.
+ async with tenant_session(session_factory, TENANT_ZERO_ID) as session:
admin = await user_store.get_by_email(session, config.gateway_admin_email)
if admin is None:
raise RuntimeError(
"Cannot seed agent-registration bootstrap key: admin user "
f"{config.gateway_admin_email} not found"
)
- with tenant_scope(TENANT_ZERO_ID):
- bootstrap_owner = await ensure_bootstrap_owner(session, user_store)
+ admin_id = admin.id
+ bootstrap_owner = await ensure_bootstrap_owner(session, user_store)
+ bootstrap_owner_id = bootstrap_owner.id
+ owner_metadata = dict(bootstrap_owner.metadata_ or {})
+ await session.commit()
- admin_owned_agents = [
- a for a in await agent_store.get_all(session) if a.owner_id == admin.id
- ]
- if admin_owned_agents:
- logger.warning(
- "%d agent(s) are owned by the admin user (%s) and carry "
- "admin-equivalent authority over every room and resource in "
- "this deployment, not just their own: %s. If any were "
- "registered through AGENT_REGISTRATION_TOKEN, reassign or "
- "re-register them under a non-admin owner.",
- len(admin_owned_agents),
- config.gateway_admin_email,
- ", ".join(a.name for a in admin_owned_agents),
+ # ── The admin-owned-agent warning, over every tenant ────────────────────
+ # It has to see every tenant's agents or it under-reports exactly the case
+ # it exists to flag, and it now does so one tenant at a time. The names
+ # are collected rather than the rows: nothing here needs an `Agent` past
+ # the session that read it.
+ admin_owned_agents: list[str] = []
+ for tenant_id in tenant_ids:
+ async with tenant_session(session_factory, tenant_id) as session:
+ # Filtered on the row's own tenant as well as the owner: on an
+ # owner connection no policy narrows the read, and every tenant's
+ # agents would be counted once per tenant. See
+ # `db/tenant_lookup.py`.
+ admin_owned_agents.extend(
+ agent.name
+ for agent in await agent_store.get_all(session)
+ if agent.owner_id == admin_id and agent.tenant_id == tenant_id
)
+ if admin_owned_agents:
+ logger.warning(
+ "%d agent(s) are owned by the admin user (%s) and carry "
+ "admin-equivalent authority over every room and resource in "
+ "this deployment, not just their own: %s. If any were "
+ "registered through AGENT_REGISTRATION_TOKEN, reassign or "
+ "re-register them under a non-admin owner.",
+ len(admin_owned_agents),
+ config.gateway_admin_email,
+ ", ".join(admin_owned_agents),
+ )
+ # ── The key itself, in the one tenant that holds it ─────────────────────
+ key_tenant_id = await _bootstrap_key_tenant(
+ session_factory, api_key_store, tenant_ids
+ )
+ async with tenant_session(session_factory, key_tenant_id) as session:
token_hash = hashlib.sha256(
config.agent_registration_token.encode()
).hexdigest()
@@ -605,7 +698,16 @@ async def _seed_agent_registration_bootstrap_key(
config.agent_registration_token, config.jwt_secret_key
)
- bootstrap_keys = await api_key_store.get_by_type(session, BOOTSTRAP_KEY_TYPE)
+ # Filtered on this tenant as well as on the type, for the same reason
+ # every other fan-out in this change is: `get_by_type` carries no
+ # tenant filter of its own, so on an owner connection it answers with
+ # every tenant's keys and this block would adopt or retire one that
+ # belongs to somebody else. See `db/tenant_lookup.py`.
+ bootstrap_keys = [
+ row
+ for row in await api_key_store.get_by_type(session, BOOTSTRAP_KEY_TYPE)
+ if row.tenant_id == key_tenant_id
+ ]
if len(bootstrap_keys) > 1:
raise RuntimeError(
f"Found {len(bootstrap_keys)} agent-registration bootstrap "
@@ -613,12 +715,8 @@ async def _seed_agent_registration_bootstrap_key(
)
bootstrap_key = bootstrap_keys[0] if bootstrap_keys else None
- last_seeded_hash = (bootstrap_owner.metadata_ or {}).get(
- BOOTSTRAP_LAST_SEEDED_HASH_META_KEY
- )
- raw_revoked_hashes = (bootstrap_owner.metadata_ or {}).get(
- BOOTSTRAP_REVOKED_HASHES_META_KEY
- )
+ last_seeded_hash = owner_metadata.get(BOOTSTRAP_LAST_SEEDED_HASH_META_KEY)
+ raw_revoked_hashes = owner_metadata.get(BOOTSTRAP_REVOKED_HASHES_META_KEY)
if raw_revoked_hashes is not None and not isinstance(raw_revoked_hashes, list):
raise RuntimeError(
f"{BOOTSTRAP_REVOKED_HASHES_META_KEY} on the agent-registration "
@@ -659,7 +757,7 @@ async def _seed_agent_registration_bootstrap_key(
for row in await api_key_store.get_by_label(
session, LEGACY_BOOTSTRAP_KEY_LABEL
)
- if row.type == "registration"
+ if row.type == "registration" and row.tenant_id == key_tenant_id
]
matching_legacy = next(
(row for row in legacy_rows if row.key_hash == token_hash), None
@@ -727,9 +825,12 @@ async def _seed_agent_registration_bootstrap_key(
"gateway's API Keys page instead."
)
else:
+ # No explicit `tenant_id`: the session is bound to the tenant this
+ # key belongs to, so the column default writes it, the same way
+ # every other scoped insert in the tree does. Naming a constant
+ # here was the shape that only worked while nothing enforced it.
bootstrap_key = ApiKey(
- tenant_id=TENANT_ZERO_ID,
- user_id=admin.id,
+ user_id=admin_id,
key_hash=token_hash,
encrypted_key=encrypted_key,
label=BOOTSTRAP_KEY_LABEL,
@@ -746,14 +847,78 @@ async def _seed_agent_registration_bootstrap_key(
meta_dirty = True
if meta_dirty:
- meta = dict(bootstrap_owner.metadata_ or {})
+ # `users` carries no tenant, so the owner row is writable from
+ # this session whichever tenant holds the key. Re-read rather than
+ # carried over from the block above: the object there belonged to
+ # a session that has since closed, and mutating a detached row
+ # persists nothing.
+ owner = await user_store.get(session, bootstrap_owner_id)
+ if owner is None:
+ raise RuntimeError(
+ "The agent-registration bootstrap owner vanished between "
+ "being seeded and being updated; this needs a direct "
+ "database fix."
+ )
+ meta = dict(owner.metadata_ or {})
meta[BOOTSTRAP_LAST_SEEDED_HASH_META_KEY] = new_last_seeded_hash
meta[BOOTSTRAP_REVOKED_HASHES_META_KEY] = revoked_hashes
- bootstrap_owner.metadata_ = meta
+ owner.metadata_ = meta
await session.commit()
+async def _bootstrap_key_tenant(
+ session_factory: async_sessionmaker[AsyncSession],
+ api_key_store: ApiKeyStore,
+ tenant_ids: list[str],
+) -> str:
+ """Which tenant holds the deployment's agent-registration bootstrap key.
+
+ The key is one per deployment and `api_keys` is scoped, so "one per
+ deployment" is a claim about a set of per-tenant tables rather than about
+ one table. This asks each tenant in turn and refuses if two answer — the
+ same "expected at most one" rule the seeding already enforced within a
+ tenant, now enforced across them, which is where a second one could
+ actually appear.
+
+ Tenant zero when nobody holds one, because that is where a fresh
+ deployment's key is created. A legacy admin-owned registration row counts
+ as holding it: the seeding is about to adopt or retire that row, and it has
+ to do so in the tenant the row is actually in.
+ """
+ holders: list[str] = []
+ legacy_holders: list[str] = []
+ for tenant_id in tenant_ids:
+ async with tenant_session(session_factory, tenant_id) as session:
+ # Both reads are filtered on the row's own tenant as well: on an
+ # owner connection neither store method narrows by tenant, so
+ # every tenant would look like a holder as soon as one was. See
+ # `db/tenant_lookup.py`.
+ if any(
+ row.tenant_id == tenant_id
+ for row in await api_key_store.get_by_type(session, BOOTSTRAP_KEY_TYPE)
+ ):
+ holders.append(tenant_id)
+ elif any(
+ row.type == "registration" and row.tenant_id == tenant_id
+ for row in await api_key_store.get_by_label(
+ session, LEGACY_BOOTSTRAP_KEY_LABEL
+ )
+ ):
+ legacy_holders.append(tenant_id)
+ if len(holders) > 1:
+ raise RuntimeError(
+ f"Tenants {sorted(holders)} each hold an agent-registration "
+ "bootstrap key; the key is one per deployment and expected in at "
+ "most one. This needs a direct database fix."
+ )
+ if holders:
+ return holders[0]
+ if legacy_holders:
+ return legacy_holders[0]
+ return TENANT_ZERO_ID
+
+
async def _shutdown(
server: uvicorn.Server,
client_lifecycle: ClientLifecycleService,
@@ -780,10 +945,20 @@ def main() -> None:
logger.info("Starting switch-core %s", running_version or "(version unknown)")
+ # Migrations still run at boot, and on the same schedule as before — what
+ # changed is the connection they run on. `migrations/env.py` points Alembic
+ # at DB_OWNER_USER where one is configured, because DDL is exactly what the
+ # runtime role is not allowed to issue. Where none is, this is unchanged
+ # from before and runs as DB_USER, which is right for a developer pointing
+ # at a scratch database and fails loudly and immediately for a deployment
+ # that has moved to a restricted role without saying who its owner is.
alembic_ini = Path(__file__).resolve().parent.parent / "alembic.ini"
alembic_cfg = AlembicConfig(str(alembic_ini))
alembic_command.upgrade(alembic_cfg, "head")
- logger.info("Database migrations applied")
+ logger.info(
+ "Database migrations applied as %s",
+ config.db_owner_user or config.db_user,
+ )
asyncio.run(run(config))
diff --git a/core/switch_core/migrations/env.py b/core/switch_core/migrations/env.py
index 98326128f..a5f1b57ae 100644
--- a/core/switch_core/migrations/env.py
+++ b/core/switch_core/migrations/env.py
@@ -22,7 +22,17 @@ if config.config_file_name is not None and not logging_is_configured():
target_metadata = Base.metadata
switch_config = SwitchConfig() # type: ignore[call-arg]
-config.set_main_option("sqlalchemy.url", switch_config.database_url)
+# The owner's connection where one is configured, and only there. A migration
+# is DDL and the runtime role deliberately cannot run DDL — see
+# `SwitchConfig.db_owner_user`. Falling back to the runtime connection keeps
+# `alembic` on the command line working against a database whose owner *is*
+# the configured user, which is what a developer pointing it at a scratch
+# instance has; where the two roles differ, the runtime one produces a
+# permission error naming the statement it could not run, which is the right
+# failure rather than a silent half-migration.
+config.set_main_option(
+ "sqlalchemy.url", switch_config.owner_database_url or switch_config.database_url
+)
def run_migrations_offline() -> None:
diff --git a/core/switch_core/migrations/versions/9c41a7b0e5d8_tenant_lookup_functions.py b/core/switch_core/migrations/versions/9c41a7b0e5d8_tenant_lookup_functions.py
new file mode 100644
index 000000000..b2c000c0f
--- /dev/null
+++ b/core/switch_core/migrations/versions/9c41a7b0e5d8_tenant_lookup_functions.py
@@ -0,0 +1,147 @@
+"""the eight tenant lookups that are exempt from row-level security
+
+The policies installed by `265ed188ad6f` are enforced by
+`require_tenant_id()`, which raises when no tenant is bound. Resolving *who a
+caller is* is what produces the tenant, so credential lookups necessarily run
+unbound and the policy refuses them. That is not a corner case: under a
+restricted runtime role it stops the process before it finishes booting, and
+`docs/old/multi-tenancy-phase1-db.md`'s "The bootstrap problem" describes a
+model — an "unscoped session" — that does not work, because unscoped is
+exactly the state the function raises on.
+
+This revision installs the replacement: eight `SECURITY DEFINER` functions
+that answer *which tenant* and never return a row. Owned by whoever runs this
+migration — the schema owner — so they run outside the policies, which is what
+Postgres gives a table's owner as long as `force row level security` is not
+set. It is not set here and must not be; see `db/runtime_role.py`, which
+refuses to boot if it ever is.
+
+Each returns `setof text`. Uniformly, including the seven that can answer at
+most once: a `LANGUAGE sql` function declared `RETURNS text` over a query that
+matches two rows silently returns the first, and `agents.oauth_client_id`
+carries no unique index, so one of these genuinely can. The caller enforces
+cardinality and refuses rather than picking.
+
+Each parameter is prefixed `p_`, which is load-bearing rather than a style
+choice: a `LANGUAGE sql` parameter spelled like a column of a table in its own
+query resolves to the column, so `WHERE id = client_id` against a table with a
+`client_id` column compares that column with itself and matches every row.
+
+`SET search_path = pg_catalog, public, pg_temp`, with `pg_temp` last, per the
+Postgres note on writing `SECURITY DEFINER` functions safely: a role able to
+create a temporary table could otherwise shadow one of the tables read here.
+
+No grants and no roles are created here either. `EXECUTE` defaults to `PUBLIC`
+for a new function, which is what makes these callable by the runtime role
+without this migration having to know its name — and the most a caller can
+learn from any of them is the tenant an identifier it already holds belongs
+to. The runtime role's table grants are re-issued at boot by the owner
+connection (`db/runtime_role.py`), after this migration has run, so a table a
+later revision adds is granted without a runbook step.
+
+The DDL below is a verbatim copy of `switch_core/db/tenant_lookup.py` as it
+stood when this migration was written, copied rather than imported for the
+same reason `265ed188ad6f`'s policies are: a migration is a record of a change
+that already happened, and importing the live module would let a later edit
+silently change what this one means.
+
+Revision ID: 9c41a7b0e5d8
+Revises: 265ed188ad6f
+Create Date: 2026-09-10 00:00:00.000000
+
+"""
+
+from collections.abc import Sequence
+
+from alembic import op
+
+revision: str = "9c41a7b0e5d8"
+down_revision: str | None = "265ed188ad6f"
+branch_labels: str | Sequence[str] | None = None
+depends_on: str | Sequence[str] | None = None
+
+SECURE_SEARCH_PATH = "pg_catalog, public, pg_temp"
+
+# (function name, parameter declaration, drop signature, body)
+LOOKUPS: tuple[tuple[str, str, str, str], ...] = (
+ (
+ "all_tenant_ids",
+ "",
+ "all_tenant_ids()",
+ "SELECT id FROM tenants ORDER BY created_at, id",
+ ),
+ (
+ "tenants_of_user",
+ "p_user_id text",
+ "tenants_of_user(text)",
+ "SELECT tenant_id FROM tenant_members "
+ "WHERE user_id = p_user_id ORDER BY tenant_id",
+ ),
+ (
+ "tenant_of_api_key",
+ "p_key_hash text",
+ "tenant_of_api_key(text)",
+ "SELECT tenant_id FROM api_keys WHERE key_hash = p_key_hash",
+ ),
+ (
+ "tenant_of_agent_oauth_client",
+ "p_oauth_client_id text",
+ "tenant_of_agent_oauth_client(text)",
+ "SELECT tenant_id FROM agents WHERE oauth_client_id = p_oauth_client_id",
+ ),
+ (
+ "tenant_of_client",
+ "p_client_id text",
+ "tenant_of_client(text)",
+ "SELECT tenant_id FROM clients WHERE id = p_client_id",
+ ),
+ (
+ "tenant_of_room",
+ "p_room_id text",
+ "tenant_of_room(text)",
+ "SELECT tenant_id FROM rooms WHERE id = p_room_id",
+ ),
+ (
+ "tenant_of_collaboration_bridge",
+ "p_bridge_id text",
+ "tenant_of_collaboration_bridge(text)",
+ "SELECT tenant_id FROM collaboration_bridges WHERE id = p_bridge_id",
+ ),
+ (
+ "tenant_of_server_connector",
+ "p_connector_id text",
+ "tenant_of_server_connector(text)",
+ "SELECT tenant_id FROM server_connectors WHERE id = p_connector_id",
+ ),
+)
+
+
+def create_lookup_ddl(name: str, parameters: str, body: str) -> str:
+ """This revision's frozen copy of the `CREATE FUNCTION` text.
+
+ A function rather than an inline f-string in `upgrade`, so that a test can
+ compare what this migration would actually run against what
+ `db/tenant_lookup.py` builds today. Without that the frozen list below
+ could agree with the live module name for name and body for body while
+ this template quietly said `SECURITY INVOKER`, and an Alembic-built
+ database would install eight functions that cannot resolve a tenant —
+ with every test still green, because the suite builds its schema from the
+ models and never runs this file.
+ """
+ return (
+ f"CREATE OR REPLACE FUNCTION {name}({parameters})\n"
+ f" RETURNS SETOF text\n"
+ f" LANGUAGE sql STABLE SECURITY DEFINER\n"
+ f" SET search_path = {SECURE_SEARCH_PATH}\n"
+ f"AS $${body}$$"
+ )
+
+
+def upgrade() -> None:
+ for name, parameters, _signature, body in LOOKUPS:
+ op.execute(create_lookup_ddl(name, parameters, body))
+
+
+def downgrade() -> None:
+ for _name, _parameters, signature, _body in LOOKUPS:
+ op.execute(f"DROP FUNCTION IF EXISTS {signature}")
diff --git a/core/switch_core/room_service.py b/core/switch_core/room_service.py
index 4ad055ceb..5c8788f39 100644
--- a/core/switch_core/room_service.py
+++ b/core/switch_core/room_service.py
@@ -30,10 +30,11 @@ from switch_core.bridges.collaboration.models import (
from switch_core.bridges.resource.service import ResourceService
from switch_core.clients.client_lifecycle_service import ClientLifecycleService
from switch_core.db.models import Room, RoomGroup, RoomRole
-from switch_core.db.session_scope import tenant_session, unscoped_session
+from switch_core.db.session_scope import tenant_session
from switch_core.db.stores.agent_store import AgentStore
from switch_core.db.stores.collaboration_bridge_store import CollaborationBridgeStore
from switch_core.db.stores.room_store import RoomStore
+from switch_core.db.tenant_lookup import all_tenant_ids
from switch_core.provisioning import Provisioning
from switch_core.tenant_context import tenant_scope
@@ -1178,10 +1179,24 @@ class RoomService:
counted and reported at `error`, so a contained one is still an
operator's problem rather than a silent one.
"""
- # Which rooms exist is the read that spans tenants; each room's own
- # tenant is bound only around reconciling that one room.
- async with unscoped_session(self._session_factory) as session:
- rooms = await self._room_store.get_all(session, include_archived=True)
+ # Which tenants exist is the one read that spans them (the exemption,
+ # `db/tenant_lookup.py`); each tenant's rooms are then an ordinary
+ # scoped read, and each room's own tenant is bound again around
+ # reconciling that one room.
+ rooms: list[Room] = []
+ for tenant_id in await all_tenant_ids(self._session_factory):
+ async with tenant_session(self._session_factory, tenant_id) as session:
+ # Filtered on the row's own tenant, not left to the policy: on an
+ # owner connection no policy narrows this read, and the fan-out
+ # would act on every tenant's rows once per tenant. See
+ # `db/tenant_lookup.py`, "a fan-out ... filters what it reads back".
+ rooms.extend(
+ room
+ for room in await self._room_store.get_all(
+ session, include_archived=True
+ )
+ if room.tenant_id == tenant_id
+ )
failures: list[str] = []
for room in rooms:
diff --git a/core/switch_core/transport/postgres.py b/core/switch_core/transport/postgres.py
index 828b98493..fdb0567c6 100644
--- a/core/switch_core/transport/postgres.py
+++ b/core/switch_core/transport/postgres.py
@@ -42,7 +42,8 @@ from typing import TYPE_CHECKING, Any
from switch_core.attachments import ATTACHMENT_GROUP_KEY
from switch_core.db.models import ClientRoom, MediaBlob, Message, MessageAttachment
-from switch_core.db.session_scope import tenant_session, unscoped_session
+from switch_core.db.session_scope import tenant_session
+from switch_core.db.tenant_lookup import tenant_of_client
from switch_core.messages.recorded_types import EPHEMERAL
from switch_core.messages.row import attachments_in, text_field, thread_root_of
from switch_core.tenant_context import no_tenant, tenant_scope
@@ -141,14 +142,19 @@ class PostgresTransport:
# A room's transport id never changes, so this only grows and never
# goes stale.
self._room_ids: dict[str, str] = {}
- # Switch room id -> the tenant that room belongs to. Populated by the
- # same lookup that fills `_room_ids`, from the same row — a room's
- # tenant is as immutable as its transport id, unlike this client's own
- # tenant, which is not: a client can be in rooms of different
- # tenants, so nothing here is ever read as "this transport's tenant".
- # Each unit of work (one delivery, one send) binds the room it is
- # actually for.
+ # Switch room id -> the tenant that room belongs to. Filled by the same
+ # lookup that fills `_room_ids`, from the same row; a room's tenant is
+ # as immutable as its transport id. Each unit of work (one delivery,
+ # one send) binds the room it is actually for rather than reading a
+ # tenant off this transport.
self._room_tenants: dict[str, str] = {}
+ # This client's own tenant, resolved once from its row and then held.
+ # Not a guess and not ambient: `client_rooms` carries composite foreign
+ # keys to both `clients` and `rooms`, so every room this client can be
+ # a member of is in this tenant, and a room lookup scoped to it cannot
+ # return a subset. That is what makes `joined_rooms` safe to scope —
+ # see its docstring for the failure this replaces.
+ self._tenant_id: str | None = None
# ── Session ───────────────────────────────────────────────────────────────
@@ -358,7 +364,7 @@ class PostgresTransport:
# share a session with the read.
with tenant_scope(tenant_id):
while True:
- async with self._session_factory() as session:
+ async with tenant_session(self._session_factory, tenant_id) as session:
rows = await self._message_store.list_for_room(
session,
room_id,
@@ -526,7 +532,15 @@ class PostgresTransport:
without the protocol noticing.
"""
uri = f"switch-media://{uuid.uuid4().hex}"
- async with self._session_factory() as session:
+ # This client's tenant, not whatever a caller happens to have bound.
+ # `media_blobs` is scoped, and the uri is opaque and globally unique,
+ # so an unguessable identifier is not an isolation boundary and the
+ # row has to name a tenant that means something. It is the same answer
+ # a room's tenant would give — a client is only in rooms of its own —
+ # and it does not depend on there being a room in hand.
+ async with tenant_session(
+ self._session_factory, await self._tenant()
+ ) as session:
await self._media_store.put(
session,
MediaBlob(
@@ -547,7 +561,9 @@ class PostgresTransport:
bytes: an attachment the sender was told had been stored and a reader
gets back as a zero-byte file is the worst of the available answers.
"""
- async with self._session_factory() as session:
+ async with tenant_session(
+ self._session_factory, await self._tenant()
+ ) as session:
blob = await self._media_store.get(session, uri)
if blob is None:
raise TransportError(f"No media stored under {uri}")
@@ -641,19 +657,26 @@ class PostgresTransport:
async def joined_rooms(self) -> list[str]:
"""The transport-side ids of this client's rooms.
- Unscoped, and that is the point rather than an oversight. This runs
- once, at the top of `receive_forever`, in a task that binds nothing —
- so a tenant here could only come from whatever created the task, and
- answering "which rooms is this client in" under a guessed tenant
- returns a *subset* with no error to say so. The failure would be
- silent and total: the client watches the rooms it was allowed to see
- and is simply deaf in the rest, forever, with nothing in the log.
-
- The key it reads by is `clients.id`, which is globally unique, so
- there is nothing for a tenant to disambiguate. Each room that comes
- back is then watched under its own tenant, resolved from its own row.
+ Scoped to this client's own tenant, which is *derived* rather than
+ inherited — the distinction the earlier version of this method was
+ right to insist on. It runs once, at the top of `receive_forever`, in
+ a task that binds nothing, so a tenant taken from whatever created the
+ task would return a subset with no error to say so, and the client
+ would be silently deaf in every room it did not see, forever. That is
+ still true. What has changed is where the tenant comes from: not from
+ the caller's context but from this client's own row, through the
+ exemption (`db/tenant_lookup.py`), which is the same answer the read
+ itself would have given.
+
+ The subset it cannot be is guaranteed by the schema rather than
+ assumed: `client_rooms` carries composite foreign keys to both
+ `clients` and `rooms`, so a client is only ever a member of rooms in
+ its own tenant. `tests/switch_core/transport/test_postgres_transport.py`
+ pins that, because the shape of this list rests on it.
"""
- async with unscoped_session(self._session_factory) as session:
+ async with tenant_session(
+ self._session_factory, await self._tenant()
+ ) as session:
rooms = await self._room_store.get_for_client(session, self.client_id)
return [room.matrix_room_id for room in rooms if room.matrix_room_id]
@@ -687,18 +710,41 @@ class PostgresTransport:
async def _resolve_room_and_tenant(self, transport_room_id: str) -> tuple[str, str]:
"""The Switch room id and tenant for a transport-side id.
- Deliberately unscoped on a cache miss: this is the lookup that decides
- *which* tenant the caller is about to bind, so inheriting one would
- make it either a tautology or a false negative. Once cached, resolving
- costs nothing: no session, no query, no round trip.
+ On a miss, scoped to this client's own tenant rather than unscoped.
+ `rooms.matrix_room_id` is unique *per tenant*, so an unscoped read of
+ it is the one lookup here that could legitimately match two rows; this
+ client can only be in rooms of its own tenant, so scoping the read is
+ both narrower and exact. Once cached, resolving costs nothing: no
+ session, no query, no round trip.
"""
cached = self._room_ids.get(transport_room_id)
if cached is not None:
return cached, self._room_tenants[cached]
- async with unscoped_session(self._session_factory) as session:
+ async with tenant_session(
+ self._session_factory, await self._tenant()
+ ) as session:
room_id = await self._resolve_room(session, transport_room_id)
return room_id, self._room_tenants[room_id]
+ async def _tenant(self) -> str:
+ """This client's tenant, resolved once from its own row.
+
+ Its own row rather than its context: this runs inside the client's
+ long-lived task, which unbinds on purpose, and from handlers that have
+ some *room's* tenant bound. Neither is the question being asked. The
+ client id is a primary key, so the exemption
+ (`db/tenant_lookup.py`) answers it exactly.
+ """
+ if self._tenant_id is None:
+ tenant_id = await tenant_of_client(self._session_factory, self.client_id)
+ if tenant_id is None:
+ raise TransportError(
+ f"Client {self.client_id} has no row, so its transport has "
+ "no tenant to act in."
+ )
+ self._tenant_id = tenant_id
+ return self._tenant_id
+
def to_inbound(
row: Message,
diff --git a/core/tests/conftest.py b/core/tests/conftest.py
index 49583fe13..f0c70cd84 100644
--- a/core/tests/conftest.py
+++ b/core/tests/conftest.py
@@ -10,6 +10,7 @@ from sqlalchemy import insert, text
from sqlalchemy.engine import make_url
from sqlalchemy.ext.asyncio import (
AsyncConnection,
+ AsyncEngine,
AsyncSession,
async_sessionmaker,
create_async_engine,
@@ -22,6 +23,7 @@ import switch_core.db.models # noqa: F401
from switch_core.db.base import Base
from switch_core.db.engine import create_session_factory
from switch_core.db.models import TENANT_ZERO_ID, Tenant
+from switch_core.db.runtime_role import grant_runtime_role
from switch_core.tenant_context import bind_tenant_id, unbind_tenant_id
@@ -124,6 +126,13 @@ class RLSHarness:
owner: async_sessionmaker[AsyncSession]
restricted: async_sessionmaker[AsyncSession]
+ # The engines behind the two factories, for the boot self-check
+ # (`db/runtime_role.verify_restricted_role`), which is asked of a
+ # connection rather than of a session. Exposed rather than rebuilt in the
+ # test so that what it inspects is the connection the assertions above
+ # were made through.
+ owner_engine: AsyncEngine
+ restricted_engine: AsyncEngine
@pytest_asyncio.fixture
@@ -142,24 +151,24 @@ async def rls_harness(postgres_url: str) -> AsyncIterator[RLSHarness]:
`session_factory` does), seeds tenant zero, and creates a throwaway login
role that is not the owner (so it doesn't inherit the owner's blanket
exemption from its own tables' policies), not a superuser, and carries no
- `BYPASSRLS`. It is granted exactly what a restricted runtime role would
- need — CRUD on the schema's tables and `EXECUTE` on `require_tenant_id()`
- — and nothing else: deliberately the same shape `docs/old/multi-tenancy-
- phase1-db.md`'s "runtime role" section describes for the real deployment
- role this phase does not create. Creating it only here, inside a test
- fixture, is what that section calls out as the one place Phase 1 does
- keep a restricted role.
+ `BYPASSRLS`.
+
+ **It is granted by `db/runtime_role.grant_runtime_role`, the same function
+ boot runs against a real deployment**, rather than by a second list of
+ grants written out here. That is deliberate: a fixture that granted more
+ than production does would let a test pass on a privilege the deployment
+ has not got, and one that granted less would fail for a reason the
+ deployment never hits. Sharing the definition makes "the restricted role
+ can do its job" a property this suite actually proves.
Only the tests that exist to prove the policies correct ask for this
fixture — the rest of the suite keeps using `session_factory` exactly as
- it does today. What this proves, and what it does not: it proves the
- *policies* are correct, that a session subject to them cannot cross a
- tenant boundary through the ordinary store layer. It does not prove the
- *deployment* is subject to them, since nothing here changes which role
- local Compose, the chart or production connect as — that is the separate
- role work (CHOO-2685); until it lands, a superuser or table-owner
- connection bypasses every policy below regardless of what this fixture
- demonstrates.
+ it does today, connected as the owner. What this proves and what it does
+ not is worth stating: it proves the *policies* and the *exemption* are
+ correct against a role they apply to. Whether the deployment connects as
+ such a role is a separate question, asked at boot by
+ `db/runtime_role.verify_restricted_role` rather than here, because CI has
+ no production connection to ask it of.
"""
owner_engine = create_async_engine(postgres_url)
role = f"switch_rls_test_{uuid.uuid4().hex[:12]}"
@@ -177,19 +186,7 @@ async def rls_harness(postgres_url: str) -> AsyncIterator[RLSHarness]:
"NOSUPERUSER NOCREATEDB NOCREATEROLE NOBYPASSRLS"
)
)
- await conn.execute(text(f'GRANT USAGE ON SCHEMA public TO "{role}"'))
- await conn.execute(
- text(
- "GRANT SELECT, INSERT, UPDATE, DELETE "
- f'ON ALL TABLES IN SCHEMA public TO "{role}"'
- )
- )
- await conn.execute(
- text(f'GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO "{role}"')
- )
- await conn.execute(
- text(f'GRANT EXECUTE ON FUNCTION require_tenant_id() TO "{role}"')
- )
+ await grant_runtime_role(conn, role)
restricted_url = make_url(postgres_url).set(username=role, password=password)
restricted_engine = create_async_engine(restricted_url)
@@ -197,6 +194,8 @@ async def rls_harness(postgres_url: str) -> AsyncIterator[RLSHarness]:
yield RLSHarness(
owner=create_session_factory(owner_engine),
restricted=create_session_factory(restricted_engine),
+ owner_engine=owner_engine,
+ restricted_engine=restricted_engine,
)
finally:
await restricted_engine.dispose()
diff --git a/core/tests/integration/conftest.py b/core/tests/integration/conftest.py
index b3f30076f..189e3a806 100644
--- a/core/tests/integration/conftest.py
+++ b/core/tests/integration/conftest.py
@@ -58,6 +58,7 @@ from switch_core.db.engine import (
create_unpooled_engine,
)
from switch_core.db.models import TENANT_ZERO_ID, Tenant, User
+from switch_core.db.runtime_role import grant_runtime_role, verify_restricted_role
from switch_core.db.stores.agent_session_store import AgentSessionStore
from switch_core.db.stores.agent_store import AgentStore
from switch_core.db.stores.api_key_store import ApiKeyStore
@@ -88,6 +89,11 @@ from switch_core.transport.invites import InviteBus
# ── Mirrors deploy/local/docker-compose.yml — keep in sync ──────────────────────
POSTGRES_IMAGE = "postgres:16-alpine"
+# The role the application runs as here. Named the same as the one
+# `deploy/local/docker-compose.yml` creates, so what this suite exercises and
+# what a developer's stack runs are the same shape.
+RUNTIME_ROLE = "switch_app"
+
# ── Throwaway test constants (not secrets — local ephemeral infra) ──────────────
SERVER_NAME = "localhost"
JWT_SECRET = "dev-jwt-secret-test"
@@ -119,12 +125,21 @@ class StackInfo:
@dataclass
class SessionEnv:
- """Session-scoped pieces shared by every test: the engine/schema and the
+ """Session-scoped pieces shared by every test: the engines/schema and the
(stateless) stores. The per-test `harness` fixture builds the in-memory
- services on top of these."""
+ services on top of these.
+
+ Two engines, and which is which matters. `engine` is the restricted
+ runtime role every service here is wired to — the row-level-security
+ policies apply to it, so what this suite exercises is what a deployment
+ runs. `owner_engine` is the schema owner, used only for the two things
+ that need DDL and that the application never does: building the schema and
+ truncating it between tests.
+ """
config: SwitchConfig
engine: AsyncEngine
+ owner_engine: AsyncEngine
session_factory: object
agent_store: AgentStore
agent_session_store: AgentSessionStore
@@ -180,13 +195,22 @@ def switch_stack() -> Iterator[StackInfo]:
yield StackInfo(pg=pg)
-def _build_config(stack: StackInfo, db_name: str) -> SwitchConfig:
+def _build_config(
+ stack: StackInfo,
+ db_name: str,
+ *,
+ user: str | None = None,
+ password: str | None = None,
+) -> SwitchConfig:
+ """Config for one of the two roles. Defaults to the container's own —
+ the schema owner — and is given the runtime role explicitly for the config
+ the application is actually wired with."""
pg = stack.pg
return SwitchConfig(
db_host=pg.get_container_host_ip(),
db_port=str(pg.get_exposed_port(5432)),
- db_user=pg.username,
- db_password=pg.password,
+ db_user=user or pg.username,
+ db_password=password or pg.password,
db_name=db_name,
matrix_server_name=SERVER_NAME,
agent_registration_token=REGISTRATION_TOKEN,
@@ -342,6 +366,22 @@ async def _truncate_all(engine: AsyncEngine) -> None:
@pytest_asyncio.fixture(scope="session", loop_scope="session")
async def session_env(switch_stack: StackInfo) -> AsyncIterator[SessionEnv]:
+ """The database, and the two roles that reach it.
+
+ **The application runs as a restricted role here, not as the owner**, and
+ that is the point of this fixture rather than an implementation detail of
+ it. Every row-level-security policy in this schema is inert against a
+ superuser or the tables' owner, so an integration suite connected as
+ either exercises the real code against a database with no isolation in it
+ — which is exactly how nineteen call sites came to depend on reads a
+ deployed system refuses, and why several of them were invisible until a
+ row existed. Wiring the app to `switch_app` makes this suite the thing
+ that catches the twentieth.
+
+ The owner connection stays, for the two jobs that genuinely need DDL and
+ that the application never does: building the schema, and truncating it
+ between tests.
+ """
# One database for the whole session; tests reset it with TRUNCATE rather than
# CREATE/DROP DATABASE. DROP at teardown uses FORCE to evict any still-open
# sync connections.
@@ -351,21 +391,47 @@ async def session_env(switch_stack: StackInfo) -> AsyncIterator[SessionEnv]:
await admin_conn.execute(f'CREATE DATABASE "{db_name}"')
await admin_conn.close()
- config = _build_config(switch_stack, db_name)
- engine = create_engine_from_config(config)
- session_factory = create_session_factory(engine)
+ owner_engine = create_engine_from_config(_build_config(switch_stack, db_name))
# Schema is built directly from the SQLAlchemy models (not Alembic). This is
# fast and always matches the current models, but means the integration suite
# does NOT exercise the migration chain — model↔migration drift is caught
# separately by `just migrate`, not here.
- async with engine.begin() as conn:
+ async with owner_engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
- await _seed_tenant_zero(engine)
+ await _seed_tenant_zero(owner_engine)
+
+ runtime_password = uuid.uuid4().hex
+ async with owner_engine.begin() as conn:
+ # CREATE ROLE takes no bind parameter for PASSWORD, so a fresh random
+ # throwaway is interpolated. Granted through `grant_runtime_role`, the
+ # same function a deployment runs at boot, so nothing here can pass on
+ # a privilege production has not got.
+ await conn.execute(
+ text(
+ f'CREATE ROLE "{RUNTIME_ROLE}" LOGIN '
+ f"PASSWORD '{runtime_password}' "
+ "NOSUPERUSER NOCREATEDB NOCREATEROLE NOBYPASSRLS"
+ )
+ )
+ await grant_runtime_role(conn, RUNTIME_ROLE)
+
+ config = _build_config(
+ switch_stack, db_name, user=RUNTIME_ROLE, password=runtime_password
+ )
+ engine = create_engine_from_config(config)
+ session_factory = create_session_factory(engine)
+
+ # The same check `main.run()` makes before it serves a request: not a
+ # superuser, owns nothing policied, and cannot read a scoped table with no
+ # tenant bound. If this ever passed for the wrong reason, every isolation
+ # claim the suite below makes would be worth nothing.
+ await verify_restricted_role(engine)
env = SessionEnv(
config=config,
engine=engine,
+ owner_engine=owner_engine,
session_factory=session_factory,
agent_store=AgentStore(),
agent_session_store=AgentSessionStore(),
@@ -389,18 +455,28 @@ async def session_env(switch_stack: StackInfo) -> AsyncIterator[SessionEnv]:
yield env
finally:
await engine.dispose()
+ # The role outlives the database it was granted in — roles are
+ # cluster-wide — so its privileges have to go before it can. `DROP
+ # OWNED BY` revokes what survives the database drop; it owns nothing,
+ # so there is nothing for it to drop.
+ async with owner_engine.begin() as conn:
+ await conn.execute(text(f'DROP OWNED BY "{RUNTIME_ROLE}"'))
+ await owner_engine.dispose()
admin_conn = await asyncpg.connect(admin_dsn)
try:
await admin_conn.execute(
f'DROP DATABASE IF EXISTS "{db_name}" WITH (FORCE)'
)
+ await admin_conn.execute(f'DROP ROLE IF EXISTS "{RUNTIME_ROLE}"')
finally:
await admin_conn.close()
@pytest_asyncio.fixture(loop_scope="session")
async def harness(session_env: SessionEnv) -> AsyncIterator[Harness]:
- await _truncate_all(session_env.engine)
+ # Truncation is DDL-adjacent and crosses every tenant, so it runs as
+ # the owner. Nothing the application does goes through this engine.
+ await _truncate_all(session_env.owner_engine)
config = session_env.config
session_factory = session_env.session_factory
diff --git a/core/tests/switch_core/bridges/agent/protocol/test_register_addressing_policy.py b/core/tests/switch_core/bridges/agent/protocol/test_register_addressing_policy.py
index 5e673d170..a39684a04 100644
--- a/core/tests/switch_core/bridges/agent/protocol/test_register_addressing_policy.py
+++ b/core/tests/switch_core/bridges/agent/protocol/test_register_addressing_policy.py
@@ -2,13 +2,14 @@
from __future__ import annotations
+import hashlib
from types import SimpleNamespace
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from switch_core.addressing import parse_policy
from switch_core.bridges.agent.protocol.service import ProtocolService
-from switch_core.db.models import TENANT_ZERO_ID
+from switch_core.db.models import TENANT_ZERO_ID, ApiKey
from tests.switch_core.bridges.agent.protocol.registration_harness import (
PROFILE,
make_owner,
@@ -17,6 +18,31 @@ from tests.switch_core.bridges.agent.protocol.registration_harness import (
)
+async def _seed_registration_key(
+ session_factory: async_sessionmaker[AsyncSession], owner_id: str, token: str
+) -> None:
+ """A real `api_keys` row for `token`.
+
+ `register_agent_with_token` resolves the token's tenant through
+ `tenant_of_api_key` — a `SECURITY DEFINER` SQL function reading the real
+ table — before it ever reaches `svc.api_key_store`, which these tests
+ stub (`_stub_key`) to control the row `register_agent_with_token` sees
+ once the tenant is known. Stubbing the store cannot stand in for this
+ row too: the tenant lookup has no store to intercept.
+ """
+ async with session_factory() as session:
+ session.add(
+ ApiKey(
+ user_id=owner_id,
+ key_hash=hashlib.sha256(token.encode()).hexdigest(),
+ encrypted_key="enc",
+ label="reg",
+ type="registration",
+ )
+ )
+ await session.commit()
+
+
async def _policy_of(
svc: ProtocolService,
session_factory: async_sessionmaker[AsyncSession],
@@ -141,6 +167,7 @@ class TestRegisterWithTokenPassesThrough:
) -> None:
svc = make_service(session_factory)
owner_id = await make_owner(session_factory)
+ await _seed_registration_key(session_factory, owner_id, "tok")
captured: dict[str, object] = {}
async def _register_agent(**kwargs: object) -> object:
@@ -167,6 +194,7 @@ class TestRegisterWithTokenPassesThrough:
) -> None:
svc = make_service(session_factory)
owner_id = await make_owner(session_factory)
+ await _seed_registration_key(session_factory, owner_id, "tok")
captured: dict[str, object] = {}
async def _register_agent(**kwargs: object) -> object:
diff --git a/core/tests/switch_core/bridges/agent/protocol/test_runtime_state_sweep.py b/core/tests/switch_core/bridges/agent/protocol/test_runtime_state_sweep.py
index 6a63521d9..593c51059 100644
--- a/core/tests/switch_core/bridges/agent/protocol/test_runtime_state_sweep.py
+++ b/core/tests/switch_core/bridges/agent/protocol/test_runtime_state_sweep.py
@@ -21,12 +21,15 @@ from __future__ import annotations
from types import SimpleNamespace
from typing import Any
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
+
from switch_core.bridges.agent.protocol.connections import (
PROTOCOL_VERSION,
ClientDeclaration,
ConnectionRegistry,
)
from switch_core.bridges.agent.protocol.service import ProtocolService
+from switch_core.db.models import TENANT_ZERO_ID
AGENT = "agent-1"
ROOM = "room-1"
@@ -56,7 +59,9 @@ class _SessionStore:
def _service(
- registry: ConnectionRegistry, store: _RuntimeStateStore
+ session_factory: async_sessionmaker[AsyncSession],
+ registry: ConnectionRegistry,
+ store: _RuntimeStateStore,
) -> ProtocolService:
svc = object.__new__(ProtocolService)
svc.connections = registry
@@ -64,7 +69,10 @@ def _service(
svc.agent_session_store = _SessionStore() # type: ignore[assignment]
svc.agent_store = SimpleNamespace(get=_an_agent) # type: ignore[assignment]
svc.room_store = SimpleNamespace(get=_a_room) # type: ignore[assignment]
- svc.session_factory = _session_factory # type: ignore[assignment]
+ # The sweep now opens `all_tenant_ids()` and `tenant_session` against a
+ # real session, since those are real SQL — the fixture's Postgres, not a
+ # hand-rolled fake, is what makes that call answerable at all.
+ svc.session_factory = session_factory # type: ignore[assignment]
# The emit reaches Matrix and the bridge; the decision to clear is what is
# under test, and `cleared` records it.
svc._emit_runtime_state = _noop # type: ignore[assignment]
@@ -81,17 +89,6 @@ async def _a_room(*_a: Any, **_kw: Any) -> Any:
return SimpleNamespace(id=ROOM, matrix_room_id="!m:server", bridge_id=None)
-def _session_factory() -> Any:
- class _Session:
- async def __aenter__(self) -> Any:
- return SimpleNamespace(commit=_noop)
-
- async def __aexit__(self, *_a: Any) -> None:
- return None
-
- return _Session()
-
-
async def _noop(*_a: Any, **_kw: Any) -> None:
return None
@@ -113,32 +110,44 @@ def _connected(room: str | None) -> ConnectionRegistry:
def _row() -> Any:
+ # Tenant zero, because that is the one tenant `all_tenant_ids()` actually
+ # finds in the fixture's database — the sweep now enumerates real tenants
+ # before it asks any store for a tenant's active rows, so the row handed
+ # back by the fake store has to name one that exists.
return SimpleNamespace(
- agent_id=AGENT, room_id=ROOM, state="working", tenant_id="tenant-1"
+ agent_id=AGENT, room_id=ROOM, state="working", tenant_id=TENANT_ZERO_ID
)
-async def test_a_live_connection_in_the_room_is_left_alone() -> None:
+async def test_a_live_connection_in_the_room_is_left_alone(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
store = _RuntimeStateStore([_row()])
- await _service(_connected(ROOM), store).sweep_runtime_states()
+ await _service(session_factory, _connected(ROOM), store).sweep_runtime_states()
assert store.cleared == []
-async def test_a_connection_in_another_room_does_not_protect_this_one() -> None:
+async def test_a_connection_in_another_room_does_not_protect_this_one(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
# Otherwise any live session would freeze every room's status message.
store = _RuntimeStateStore([_row()])
- await _service(_connected("other-room"), store).sweep_runtime_states()
+ await _service(
+ session_factory, _connected("other-room"), store
+ ).sweep_runtime_states()
assert store.cleared == [(AGENT, ROOM)]
-async def test_with_nothing_live_the_sweep_still_clears() -> None:
+async def test_with_nothing_live_the_sweep_still_clears(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> None:
"""The sweep's whole purpose: a crashed session must not stay 'working'."""
store = _RuntimeStateStore([_row()])
- await _service(ConnectionRegistry(), store).sweep_runtime_states()
+ await _service(session_factory, ConnectionRegistry(), store).sweep_runtime_states()
assert store.cleared == [(AGENT, ROOM)]
diff --git a/core/tests/switch_core/bridges/agent/test_api_key_cache.py b/core/tests/switch_core/bridges/agent/test_api_key_cache.py
index 45607f7aa..da3dc4904 100644
--- a/core/tests/switch_core/bridges/agent/test_api_key_cache.py
+++ b/core/tests/switch_core/bridges/agent/test_api_key_cache.py
@@ -219,16 +219,24 @@ class TestResolutionHitsAndMisses:
second_key, second_agent = await mw._resolve_api_key("tok")
assert first_agent is not None
- assert after_miss == 1
- assert counting.opened == 1, "the second resolution must be answered in memory"
+ # Two round trips, not one: the hash resolves to a tenant through the
+ # exemption (db/tenant_lookup.py) on its own session before the
+ # api_keys row — scoped like everything else — can be read on a
+ # second, tenant-bound one.
+ assert after_miss == 2
+ assert counting.opened == 2, "the second resolution must be answered in memory"
assert second_agent is not None
assert second_agent.id == first_agent.id
assert second_key is not None and first_key is not None
assert second_key.id == first_key.id
- async def test_one_checkout_resolves_key_and_agent_together(
+ async def test_a_resolution_costs_two_round_trips_tenant_then_row(
self, session_factory: async_sessionmaker[AsyncSession]
) -> None:
+ # api_keys is tenant-scoped, so its row cannot be read before its
+ # tenant is known: one round trip to tenant_of_api_key, then one more
+ # on a tenant_session to read the row itself. What the cache above
+ # saves is these two round trips, not one.
await _seed_agent(session_factory, "tok")
counting = _CountingSessionFactory(session_factory)
counting.opened = 0
@@ -236,6 +244,20 @@ class TestResolutionHitsAndMisses:
await mw._resolve_api_key("tok")
+ assert counting.opened == 2
+
+ async def test_an_unknown_token_costs_exactly_one_round_trip(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ # A hash that names no tenant stops at tenant_of_api_key: there is no
+ # tenant to open the second, row-reading session under. This is the
+ # load-shape that matters — an unauthenticated flood never reaches
+ # the second query.
+ counting = _CountingSessionFactory(session_factory)
+ cache = ApiKeyCache(ttl_seconds=5, max_entries=8)
+ mw = _middleware(counting, cache)
+
+ assert await mw._resolve_api_key("nope") == (None, None)
assert counting.opened == 1
async def test_an_unknown_token_is_never_cached(
@@ -274,7 +296,9 @@ class TestResolutionHitsAndMisses:
assert key is not None and key.type == "registration"
assert agent is None
- assert counting.opened == 2
+ # Two round trips per call: the row exists and names a tenant, so
+ # both the tenant lookup and the scoped read happen every time.
+ assert counting.opened == 4
assert cache.get(_hash("reg")) is None
async def test_an_agent_key_with_no_agent_is_not_cached(
@@ -291,7 +315,7 @@ class TestResolutionHitsAndMisses:
assert key is not None
assert agent is None
- assert counting.opened == 2
+ assert counting.opened == 4
assert cache.get(_hash("orphan")) is None
async def test_a_disabled_cache_reads_every_time(
@@ -305,7 +329,7 @@ class TestResolutionHitsAndMisses:
await mw._resolve_api_key("tok")
await mw._resolve_api_key("tok")
- assert counting.opened == 2
+ assert counting.opened == 4
async def test_an_expired_entry_reads_again(
self,
@@ -325,7 +349,7 @@ class TestResolutionHitsAndMisses:
now[0] = 1006.0
await mw._resolve_api_key("tok")
- assert counting.opened == 2
+ assert counting.opened == 4
class TestInvalidation:
diff --git a/core/tests/switch_core/bridges/collaboration/test_lifecycle_tenant_binding.py b/core/tests/switch_core/bridges/collaboration/test_lifecycle_tenant_binding.py
index d768ad57d..a72c6000e 100644
--- a/core/tests/switch_core/bridges/collaboration/test_lifecycle_tenant_binding.py
+++ b/core/tests/switch_core/bridges/collaboration/test_lifecycle_tenant_binding.py
@@ -29,7 +29,14 @@ from switch_core.bridges.collaboration.lifecycle_service import (
CollaborationBridgeLifecycleService,
)
from switch_core.bridges.collaboration.models import BridgeConnectionConfig
-from switch_core.db.models import Client, ClientRoom, CollaborationBridge, Room, Tenant
+from switch_core.db.models import (
+ TENANT_ZERO_ID,
+ Client,
+ ClientRoom,
+ CollaborationBridge,
+ Room,
+ Tenant,
+)
from switch_core.db.session_scope import tenant_session
from switch_core.db.stores.client_store import ClientStore
from switch_core.db.stores.collaboration_bridge_store import CollaborationBridgeStore
@@ -294,20 +301,25 @@ async def test_a_host_resource_conflict_is_looked_for_across_every_tenant(
belongs to someone else — narrowing it to the caller's tenant would make
it miss precisely the case it exists for.
- The conflict itself is asserted, and so is the tenant bound over the read
- that finds it: `_reject_resource_conflict` must reach `get_all` with
- nothing bound, or a policy-bearing connection would silently narrow the
- read to the caller's own tenant and this exact conflict — two tenants
- claiming the same host resource — would go undetected.
+ Under row-level security an unscoped read is not available — it is
+ exactly the state `require_tenant_id()` raises on — so seeing every
+ tenant's bridges takes `all_tenant_ids()` (the exemption in
+ `db/tenant_lookup.py`) plus one ordinary *scoped* `get_all` per tenant.
+ The conflict itself is asserted, and so is what each of those reads was
+ bound to: one call per tenant that exists — tenant zero seeded by the
+ test schema, plus the incumbent and the newcomer created here — each
+ bound to that tenant rather than to the caller's. The incumbent and the
+ newcomer are both inserted inside one transaction, so `now()` — and so
+ `all_tenant_ids`' `ORDER BY created_at` — ties between them; which of the
+ two is visited first is not the property under test, only that every
+ tenant is visited exactly once, each under its own binding, and never
+ under the caller's tenant alone or left unbound.
- That claim used to rest on the binding alone, because nothing yet made a
- *scoped* read behave any differently. It does now:
`test_a_tenant_scoped_read_of_the_same_data_would_have_missed_the_conflict`
below runs the identical two-bridge fixture through the row-level-security
policies (`rls_harness`, a role they actually apply to) and shows a read
- scoped to the newcomer's tenant does not see the incumbent's row — the
- failure this test's unscoped read exists to avoid, made real rather than
- inferred from where `current_tenant_id()` was `None`.
+ scoped to only the newcomer's tenant does not see the incumbent's row —
+ the failure this loop over every tenant exists to avoid.
"""
incumbent_tenant = f"tenant-{uuid.uuid4().hex[:8]}"
newcomer_tenant = f"tenant-{uuid.uuid4().hex[:8]}"
@@ -356,10 +368,13 @@ async def test_a_host_resource_conflict_is_looked_for_across_every_tenant(
with pytest.raises(ValueError, match="already uses port:3979"):
await service._reject_resource_conflict("teams", {"listen_port": 3979})
- assert seen == [None], (
- "the conflict check read the stored bridges under the caller's "
- "tenant; under row-level security it would not have seen the "
- "incumbent and would have let both claim the port"
+ assert sorted(seen) == sorted(
+ [TENANT_ZERO_ID, incumbent_tenant, newcomer_tenant]
+ ), (
+ "the conflict check must read the stored bridges tenant by tenant, "
+ "each scoped to the tenant being read rather than to the caller's — "
+ "a read scoped only to the newcomer's tenant would not have seen "
+ "the incumbent and would have let both claim the port"
)
diff --git a/core/tests/switch_core/bridges/collaboration/test_teams_zero_config.py b/core/tests/switch_core/bridges/collaboration/test_teams_zero_config.py
index 120d18096..0e53d1d00 100644
--- a/core/tests/switch_core/bridges/collaboration/test_teams_zero_config.py
+++ b/core/tests/switch_core/bridges/collaboration/test_teams_zero_config.py
@@ -380,7 +380,37 @@ def test_outbound_only_adapters_claim_nothing() -> None:
assert CollaborationAdapter.exclusive_resource({}) is None
-def _service_with_existing(existing: list[Any]) -> CollaborationBridgeLifecycleService:
+# The one tenant these tests pretend the deployment has. Named rather than
+# repeated: the stubbed lookup answers with it, and the fake rows have to
+# carry it, or the per-tenant filter in `_reject_resource_conflict` drops them.
+_STUB_TENANT = "tenant-a"
+
+
+def _stub_tenant_lookups(
+ monkeypatch: pytest.MonkeyPatch, tenant_id: str = _STUB_TENANT
+) -> None:
+ """Stand in for the exemption (`db/tenant_lookup.py`) with a fixed tenant.
+
+ It issues its own SQL against a real session, which these tests have
+ nothing to run against — they fake the store directly. What they exercise
+ is the exclusivity logic downstream of the lookup; that the lookup itself
+ resolves the right tenant across a real, multi-tenant database is
+ `test_lifecycle_tenant_binding.py`'s job.
+ """
+ monkeypatch.setattr(
+ "switch_core.bridges.collaboration.lifecycle_service.all_tenant_ids",
+ AsyncMock(return_value=[tenant_id]),
+ )
+ monkeypatch.setattr(
+ "switch_core.bridges.collaboration.lifecycle_service.tenant_of_collaboration_bridge",
+ AsyncMock(return_value=tenant_id),
+ )
+
+
+def _service_with_existing(
+ existing: list[Any], monkeypatch: pytest.MonkeyPatch
+) -> CollaborationBridgeLifecycleService:
+ _stub_tenant_lookups(monkeypatch)
session = MagicMock()
session.__aenter__ = AsyncMock(return_value=session)
session.__aexit__ = AsyncMock(return_value=None)
@@ -409,15 +439,23 @@ async def _stored_bridge(**overrides: Any) -> Any:
bridge.id = overrides.pop("id", "existing-id")
bridge.type = overrides.pop("type", "teams")
bridge.display_name = overrides.pop("display_name", "Contoso Teams")
+ # A real row's own tenant, matching what `_stub_tenant_lookups` says the
+ # deployment has. The conflict check filters what it reads back on the
+ # tenant it bound, because on an owner connection nothing else narrows the
+ # read (see `db/tenant_lookup.py`), and a `MagicMock` attribute would fail
+ # that filter and make every stored bridge invisible to it.
+ bridge.tenant_id = _STUB_TENANT
bridge.connection_config = await TeamsAdapter.prepare_config(
_raw_config(**overrides)
)
return bridge
-async def test_second_teams_bridge_on_the_same_port_is_refused() -> None:
+async def test_second_teams_bridge_on_the_same_port_is_refused(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
"""The failure this replaces was a bind error in a background task."""
- service = _service_with_existing([await _stored_bridge()])
+ service = _service_with_existing([await _stored_bridge()], monkeypatch)
with pytest.raises(ValueError) as excinfo:
await service.register(
@@ -440,7 +478,7 @@ async def test_a_second_bridge_on_its_own_port_is_allowed_through(
Getting past this check is not the same as working — the chart publishes one
Teams port — but that is the operator's deliberate choice to make.
"""
- service = _service_with_existing([await _stored_bridge()])
+ service = _service_with_existing([await _stored_bridge()], monkeypatch)
monkeypatch.setattr(
TeamsAdapter, "verify_credentials", AsyncMock(return_value=None)
)
@@ -462,7 +500,7 @@ async def test_a_non_teams_bridge_is_not_blocked_by_a_teams_one(
) -> None:
slack = await _stored_bridge(id="slack-id", type="slack")
slack.connection_config = {"bot_token": "x"}
- service = _service_with_existing([slack])
+ service = _service_with_existing([slack], monkeypatch)
monkeypatch.setattr(
TeamsAdapter, "verify_credentials", AsyncMock(return_value=None)
)
@@ -525,6 +563,7 @@ async def test_concurrent_registration_cannot_take_the_same_port_twice(
several awaits later, so without serialisation both callers saw a free port
and both took it — reproducing the collision the check exists to refuse.
"""
+ _stub_tenant_lookups(monkeypatch)
stored: list[Any] = []
session = MagicMock()
@@ -537,6 +576,11 @@ async def test_concurrent_registration_cannot_take_the_same_port_twice(
async def _create(_s: Any, bridge: Any) -> Any:
# The real store commits well after the conflict check has run.
await asyncio.sleep(0)
+ # Stamped the way a flush would: `tenant_id` has a Python-side column
+ # default that fires when the row reaches the database, and nothing
+ # here ever flushes, so without this the stored row has no tenant and
+ # the conflict check filters it straight back out.
+ bridge.tenant_id = _STUB_TENANT
stored.append(bridge)
return bridge
@@ -593,12 +637,15 @@ async def test_concurrent_registration_cannot_take_the_same_port_twice(
assert len(stored) == 1
-async def test_start_refuses_a_second_bridge_already_holding_the_port() -> None:
+async def test_start_refuses_a_second_bridge_already_holding_the_port(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
"""The startup guard, which covers rows that predate the registration check.
Registration cannot create a colliding pair any more, but two already exist
on some instances, and `start_all` walks straight into them.
"""
+ _stub_tenant_lookups(monkeypatch)
session = MagicMock()
session.__aenter__ = AsyncMock(return_value=session)
session.__aexit__ = AsyncMock(return_value=None)
diff --git a/core/tests/switch_core/clients/test_ensure_system_client.py b/core/tests/switch_core/clients/test_ensure_system_client.py
index cf9f79717..eb42b26cb 100644
--- a/core/tests/switch_core/clients/test_ensure_system_client.py
+++ b/core/tests/switch_core/clients/test_ensure_system_client.py
@@ -11,6 +11,19 @@ So there are two things to pin. The boot case — an empty database gets an
admin client rather than an exception — and the shape the design actually
asks for: `clients` is scoped, so this is one row per tenant, and a second
tenant with no admin client of its own is not served by the first tenant's.
+
+That second shape is exactly what `rls_harness` exists for. `ensure_system_client`
+now decides whether a tenant is already served by reading `get_by_type` inside
+a session scoped to that one tenant (`db/tenant_lookup.py`'s `all_tenant_ids`
+plus a scoped read per tenant, in place of the single unscoped read this
+replaced) — and `get_by_type` itself carries no `WHERE tenant_id = …` clause,
+because row-level security is what is meant to narrow it. The plain
+`session_factory` fixture connects as the schema owner, which bypasses every
+policy, so under it that scoped read silently returns every tenant's rows
+regardless of which one is bound — exactly the bug this module's second
+paragraph describes, reintroduced by a fixture that cannot see the
+difference. `rls_harness.restricted` is the role the policies actually apply
+to; only through it does "one row per tenant" mean anything here.
"""
from __future__ import annotations
@@ -19,7 +32,6 @@ import uuid
from types import SimpleNamespace
from unittest.mock import MagicMock
-import pytest
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
@@ -28,6 +40,7 @@ from switch_core.db.models import TENANT_ZERO_ID, Client, Tenant
from switch_core.db.stores.client_store import ClientStore
from switch_core.db.stores.tenant_store import TenantStore
from switch_core.tenant_context import current_tenant_id
+from tests.conftest import RLSHarness
def _service(
@@ -61,70 +74,61 @@ async def _make_tenant(
await session.commit()
[email protected]_ambient_tenant
async def test_a_fresh_database_gets_an_admin_client_rather_than_an_exception(
- session_factory: async_sessionmaker[AsyncSession],
+ rls_harness: RLSHarness,
) -> None:
"""The boot case, verbatim: nothing bound, no client anywhere, one
tenant. This is the call that took the process down."""
- await _service(session_factory).ensure_system_client("admin")
+ await _service(rls_harness.restricted).ensure_system_client("admin")
- clients = await _admin_clients(session_factory)
+ clients = await _admin_clients(rls_harness.owner)
assert [c.tenant_id for c in clients] == [TENANT_ZERO_ID]
assert clients[0].matrix_user_id == "@switch-admin:test"
[email protected]_ambient_tenant
-async def test_each_tenant_gets_its_own_row(
- session_factory: async_sessionmaker[AsyncSession],
-) -> None:
+async def test_each_tenant_gets_its_own_row(rls_harness: RLSHarness) -> None:
"""Not "one exists, so we are done": a tenant whose rooms have no admin
client in them has nothing to provision or narrate them."""
other = f"tenant-{uuid.uuid4().hex[:8]}"
- await _make_tenant(session_factory, other)
+ await _make_tenant(rls_harness.owner, other)
- await _service(session_factory).ensure_system_client("admin")
+ await _service(rls_harness.restricted).ensure_system_client("admin")
- clients = await _admin_clients(session_factory)
+ clients = await _admin_clients(rls_harness.owner)
assert sorted(c.tenant_id for c in clients) == sorted([TENANT_ZERO_ID, other])
[email protected]_ambient_tenant
async def test_it_is_idempotent_and_fills_only_the_gap(
- session_factory: async_sessionmaker[AsyncSession],
+ rls_harness: RLSHarness,
) -> None:
"""Every boot runs this. A second pass adds nothing, and a tenant
onboarded between two boots gets the row the first pass could not have
made."""
- service = _service(session_factory)
+ service = _service(rls_harness.restricted)
await service.ensure_system_client("admin")
await service.ensure_system_client("admin")
- assert [c.tenant_id for c in await _admin_clients(session_factory)] == [
+ assert [c.tenant_id for c in await _admin_clients(rls_harness.owner)] == [
TENANT_ZERO_ID
]
latecomer = f"tenant-{uuid.uuid4().hex[:8]}"
- await _make_tenant(session_factory, latecomer)
+ await _make_tenant(rls_harness.owner, latecomer)
await service.ensure_system_client("admin")
- clients = await _admin_clients(session_factory)
+ clients = await _admin_clients(rls_harness.owner)
assert sorted(c.tenant_id for c in clients) == sorted([TENANT_ZERO_ID, latecomer])
[email protected]_ambient_tenant
-async def test_nothing_stays_bound_afterwards(
- session_factory: async_sessionmaker[AsyncSession],
-) -> None:
+async def test_nothing_stays_bound_afterwards(rls_harness: RLSHarness) -> None:
"""It binds a tenant per row it creates and releases it: startup carries
on into work that must not inherit whichever tenant happened to be last."""
- await _make_tenant(session_factory, f"tenant-{uuid.uuid4().hex[:8]}")
- await _service(session_factory).ensure_system_client("admin")
+ await _make_tenant(rls_harness.owner, f"tenant-{uuid.uuid4().hex[:8]}")
+ await _service(rls_harness.restricted).ensure_system_client("admin")
assert current_tenant_id() is None
[email protected]_ambient_tenant
async def test_a_tenant_created_after_startup_gets_a_working_admin_client(
- session_factory: async_sessionmaker[AsyncSession],
+ rls_harness: RLSHarness,
) -> None:
"""Bug 2, arranged the way it happened live: boot runs once, against
whatever tenants exist then, and a second tenant onboards while the
@@ -138,12 +142,12 @@ async def test_a_tenant_created_after_startup_gets_a_working_admin_client(
call itself minted, not "some client exists somewhere" — by the time this
returns.
"""
- service = _service(session_factory)
+ service = _service(rls_harness.restricted)
await service.ensure_system_client("admin") # the boot-time call
tenant = await service.create_tenant(name="Acme", slug="acme")
- clients = await _admin_clients(session_factory)
+ clients = await _admin_clients(rls_harness.owner)
assert sorted(c.tenant_id for c in clients) == sorted([TENANT_ZERO_ID, tenant.id])
acme_client = next(c for c in clients if c.tenant_id == tenant.id)
assert acme_client.matrix_user_id == "@switch-admin:test"
diff --git a/core/tests/switch_core/db/test_runtime_role.py b/core/tests/switch_core/db/test_runtime_role.py
new file mode 100644
index 000000000..1ee0abad4
--- /dev/null
+++ b/core/tests/switch_core/db/test_runtime_role.py
@@ -0,0 +1,257 @@
+"""The boot self-check: does this connection actually have the policies on it.
+
+`db/runtime_role.verify_restricted_role` is the answer to a failure that is
+otherwise silent and that CI cannot ask about — CI has no production
+connection, and a Switch that believes it is isolating tenants and is not
+looks exactly like one that is. What CI *can* do is prove the check itself
+works: that it passes for a role the policies apply to, and refuses for each
+of the shapes that would make them inert.
+
+The `rls_harness` fixture already builds both ends of that — the container's
+superuser, which owns the schema, and a throwaway role that is neither owner
+nor superuser and is granted by the same `grant_runtime_role` a deployment
+runs at boot. So each test here is a connection handed to the check.
+
+`tenants` is populated throughout, and that is not incidental. Postgres does
+not evaluate a policy for a scan that yields no rows, so the behavioural probe
+— the only part of the check that observes rather than infers — needs a row to
+be meaningful at all. The fixture seeds tenant zero, and one test below pins
+what happens when nothing has.
+"""
+
+from __future__ import annotations
+
+import uuid
+
+import pytest
+from sqlalchemy import text
+from sqlalchemy.engine import make_url
+from sqlalchemy.ext.asyncio import create_async_engine
+
+from switch_core.db.runtime_role import (
+ RuntimeRoleError,
+ _refuse_a_connection_no_policy_stops,
+ _refuse_table_owner,
+ grant_runtime_role,
+ verify_restricted_role,
+)
+from tests.conftest import RLSHarness
+
+pytestmark = pytest.mark.no_ambient_tenant
+
+
+class TestItPassesForARoleThePoliciesApplyTo:
+ async def test_the_restricted_role_is_accepted(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ await verify_restricted_role(rls_harness.restricted_engine)
+
+ async def test_that_role_can_still_do_ordinary_work(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """The other half of "accepted": a check that passed for a role too
+ restricted to serve a request would be worse than no check at all,
+ since boot would then succeed and the first query would fail. The
+ grants come from `grant_runtime_role`, the same function a deployment
+ runs, so this is a statement about production and not about a fixture.
+ """
+ tenant_id = f"tenant-{uuid.uuid4().hex[:8]}"
+ async with rls_harness.owner() as session:
+ await session.execute(
+ text("INSERT INTO tenants (id, slug, name) VALUES (:id, :id, :id)"),
+ {"id": tenant_id},
+ )
+ await session.commit()
+
+ async with rls_harness.restricted() as session:
+ await session.execute(
+ text("SELECT set_config('app.tenant_id', :t, true)"),
+ {"t": tenant_id},
+ )
+ await session.execute(
+ text(
+ "INSERT INTO room_groups (id, tenant_id, name) "
+ "VALUES (:id, :t, 'a group')"
+ ),
+ {"id": str(uuid.uuid4()), "t": tenant_id},
+ )
+ visible = (
+ await session.execute(text("SELECT count(*) FROM room_groups"))
+ ).scalar_one()
+ await session.commit()
+ assert visible == 1
+
+
+class TestItRefusesEveryShapeThatWouldMakeThePoliciesInert:
+ async def test_a_superuser_is_refused(self, rls_harness: RLSHarness) -> None:
+ """The container's own role, which is what local Compose, the chart
+ and every environment used before this change connected as."""
+ with pytest.raises(RuntimeRoleError) as raised:
+ await verify_restricted_role(rls_harness.owner_engine)
+ assert "superuser or carries BYPASSRLS" in str(raised.value)
+
+ async def test_owning_a_policied_table_is_refused(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """Asked of the ownership check directly, because the connection that
+ owns these tables here is also a superuser and the check above would
+ refuse it first. Ownership is the subtler of the two and the one a
+ real deployment is far more likely to hit: a role created by hand, not
+ a superuser, given the schema because that was the easy way to make
+ migrations work.
+ """
+ async with rls_harness.owner_engine.connect() as connection:
+ role = str(
+ (await connection.execute(text("SELECT current_user"))).scalar_one()
+ )
+ with pytest.raises(RuntimeRoleError) as raised:
+ await _refuse_table_owner(connection, role)
+ assert "owns" in str(raised.value)
+ assert "exempts a table's owner" in str(raised.value)
+
+ async def test_forcing_row_level_security_is_refused(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """`force row level security` sounds like more isolation and is less.
+
+ It takes the ownership exemption away from the schema owner, and the
+ eight tenant lookups (`db/tenant_lookup.py`) are `SECURITY DEFINER`
+ functions running as exactly that owner. Force it and nothing in the
+ process can answer "which tenant is this credential in", so no request
+ authenticates — a total outage arriving as a wall of 401s. Boot says
+ so instead.
+ """
+ async with rls_harness.owner_engine.begin() as connection:
+ await connection.execute(
+ text("ALTER TABLE tenants FORCE ROW LEVEL SECURITY")
+ )
+ with pytest.raises(RuntimeRoleError) as raised:
+ await verify_restricted_role(rls_harness.restricted_engine)
+ assert "FORCE ROW LEVEL SECURITY" in str(raised.value)
+
+ async def test_a_connection_no_policy_stops_is_refused(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """The behavioural probe, asked of a connection that is exempt.
+
+ This is the check that would catch an exemption the catalogue
+ questions did not think to look for, so it is worth exercising against
+ something genuinely exempt rather than trusting that it would. The
+ owner reads `tenants` with nothing bound and gets rows; the probe
+ refuses on the count rather than on why.
+ """
+ async with rls_harness.owner_engine.connect() as connection:
+ with pytest.raises(RuntimeRoleError) as raised:
+ await _refuse_a_connection_no_policy_stops(connection, "owner")
+ assert "instead of raising" in str(raised.value)
+
+ async def test_an_ungranted_role_is_refused_rather_than_mistaken_for_isolated(
+ self, rls_harness: RLSHarness, postgres_url: str
+ ) -> None:
+ """The trap in the probe, and why it matches the message and not just
+ the SQLSTATE.
+
+ `require_tenant_id()` raises 42501, `insufficient_privilege`. So does
+ `permission denied for table tenants`. A role that had never been
+ granted anything would therefore fail the probe's read for an entirely
+ unrelated reason, and a check keyed on the code alone would read that
+ as "the policy refused it" and let boot proceed on a connection that
+ cannot serve a single request.
+ """
+ role = f"switch_ungranted_{uuid.uuid4().hex[:12]}"
+ password = uuid.uuid4().hex
+ async with rls_harness.owner_engine.begin() as connection:
+ await connection.execute(
+ text(
+ f"CREATE ROLE \"{role}\" LOGIN PASSWORD '{password}' "
+ "NOSUPERUSER NOCREATEDB NOCREATEROLE NOBYPASSRLS"
+ )
+ )
+ await connection.execute(text(f'GRANT USAGE ON SCHEMA public TO "{role}"'))
+ engine = create_async_engine(
+ make_url(postgres_url).set(username=role, password=password)
+ )
+ try:
+ with pytest.raises(RuntimeRoleError) as raised:
+ await verify_restricted_role(engine)
+ assert "not with 42501" in str(raised.value)
+ assert "missing its grants" in str(raised.value)
+ finally:
+ await engine.dispose()
+ async with rls_harness.owner_engine.begin() as connection:
+ await connection.execute(text(f'DROP OWNED BY "{role}"'))
+ await connection.execute(text(f'DROP ROLE "{role}"'))
+
+ async def test_a_role_that_cannot_run_the_lookups_is_refused(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """A role granted its tables but not the exemption would pass every
+ check above and then fail to authenticate anybody, because every
+ credential resolution goes through one of those functions. Boot says
+ which ones rather than leaving it to a wall of 401s.
+ """
+ async with rls_harness.owner_engine.begin() as connection:
+ await connection.execute(
+ text("REVOKE EXECUTE ON FUNCTION all_tenant_ids() FROM PUBLIC")
+ )
+ await connection.execute(
+ text(
+ "REVOKE EXECUTE ON FUNCTION all_tenant_ids() FROM "
+ + await _current_restricted_role(rls_harness)
+ )
+ )
+ with pytest.raises(RuntimeRoleError) as raised:
+ await verify_restricted_role(rls_harness.restricted_engine)
+ assert "cannot execute" in str(raised.value)
+ assert "all_tenant_ids" in str(raised.value)
+
+
+class TestTheProbeIsOnlyMeaningfulAgainstARow:
+ async def test_an_empty_tenants_table_refuses_rather_than_passing(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """The data-dependence of the whole approach, pinned.
+
+ Postgres does not evaluate a policy for a scan that finds no rows, so
+ against an empty `tenants` the probe's read succeeds and returns zero
+ — a result indistinguishable from a database with no policies in it.
+ Boot must not read that as a pass, and it does not: the empty table is
+ caught one check earlier, by the one that insists `all_tenant_ids()`
+ answers with something, which names the cause instead of leaving the
+ probe to report a suspicious zero. Either way the answer is the same
+ and it is the right way round — a deployment whose `tenants` table is
+ empty has no tenant zero and is broken regardless, and refusing to
+ boot says so rather than serving with isolation nobody has checked.
+ """
+ async with rls_harness.owner_engine.begin() as connection:
+ await connection.execute(text("DELETE FROM tenants"))
+ with pytest.raises(RuntimeRoleError) as raised:
+ await verify_restricted_role(rls_harness.restricted_engine)
+ assert "answered with no tenants at all" in str(raised.value)
+
+
+async def _current_restricted_role(harness: RLSHarness) -> str:
+ async with harness.restricted_engine.connect() as connection:
+ role = str((await connection.execute(text("SELECT current_user"))).scalar_one())
+ async with harness.owner_engine.connect() as connection:
+ return str(
+ (
+ await connection.execute(
+ text("SELECT quote_ident(:name)"), {"name": role}
+ )
+ ).scalar_one()
+ )
+
+
+async def test_the_grants_are_reissued_idempotently(
+ rls_harness: RLSHarness,
+) -> None:
+ """Boot re-runs the grants on every start, after the migration that may
+ have added a table. Running them twice has to be a no-op rather than an
+ error, or the second boot of an unchanged deployment fails."""
+ async with rls_harness.restricted_engine.connect() as connection:
+ role = str((await connection.execute(text("SELECT current_user"))).scalar_one())
+ async with rls_harness.owner_engine.begin() as connection:
+ await grant_runtime_role(connection, role)
+ await grant_runtime_role(connection, role)
+ await verify_restricted_role(rls_harness.restricted_engine)
diff --git a/core/tests/switch_core/db/test_session_scope.py b/core/tests/switch_core/db/test_session_scope.py
index a0218f3e8..9b172a19e 100644
--- a/core/tests/switch_core/db/test_session_scope.py
+++ b/core/tests/switch_core/db/test_session_scope.py
@@ -1,26 +1,28 @@
-"""`tenant_session` and `unscoped_session` (`db/session_scope.py`): the two
-named ways background code may open a session with no request behind it.
-
-`tenant_session` is sugar over `tenant_context.bind_tenant_id` plus opening a
-session, so what these tests pin is the composition — bound inside the block,
-released on the way out, even on an exception — rather than the binding
-mechanics themselves, which `test_tenant_session_hook.py` already covers.
-`unscoped_session` is tested here for the one thing its name promises and its
-callers depend on: that a session opened through it has no tenant set, no
-matter what the caller had bound. Its callers are pinned separately, in
-`test_unscoped_session_allowlist.py`.
+"""`tenant_session` (`db/session_scope.py`): how background code opens a
+session with no request behind it.
+
+It is sugar over `tenant_context.bind_tenant_id` plus opening a session, so
+what these tests pin is the composition — bound inside the block, released on
+the way out, even on an exception — rather than the binding mechanics
+themselves, which `test_tenant_session_hook.py` already covers.
+
+There was a second helper here, `unscoped_session`, with its own class of
+tests asserting that a session opened through it had no tenant set whatever
+the caller had bound. Both are gone. That contract was only ever meaningful
+against a connection Postgres exempts from the policies; under the restricted
+runtime role an unbound session reads nothing rather than everything, so
+"unscoped" stopped being a hatch and became a failure. What replaced it is
+`db/tenant_lookup.py`, tested in `test_tenant_lookup.py`, and who may reach it
+is pinned in `test_tenant_exemption_allowlist.py`.
"""
from __future__ import annotations
-import uuid
-
import pytest
-from sqlalchemy import select, text
+from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
-from switch_core.db.models import Room, Tenant
-from switch_core.db.session_scope import tenant_session, unscoped_session
+from switch_core.db.session_scope import tenant_session
from switch_core.tenant_context import current_tenant_id, tenant_scope
TENANT_A = "tenant-session-scope-a"
@@ -84,82 +86,3 @@ class TestTenantSession:
async with tenant_session(session_factory, TENANT_B) as session:
assert await _current_setting(session) == TENANT_B
await session.commit()
-
-
-class TestUnscopedSession:
- async def test_no_tenant_is_bound_when_nothing_was_bound_going_in(
- self, session_factory: async_sessionmaker[AsyncSession]
- ) -> None:
- async with unscoped_session(session_factory) as session:
- assert current_tenant_id() is None
- assert await _current_setting(session) is None
-
- @pytest.mark.no_ambient_tenant
- async def test_it_unbinds_an_outer_tenant_for_the_duration(
- self, session_factory: async_sessionmaker[AsyncSession]
- ) -> None:
- """The whole contract. A caller already inside a bound context — a
- request, an outer `tenant_scope`, a task that inherited one — still
- gets a session with nothing set, because otherwise "unscoped" would
- mean "scoped to whoever happened to call me" and the allowlist that
- pins these call sites would be certifying a cross-tenant read that
- never happens."""
- with tenant_scope(TENANT_A):
- async with unscoped_session(session_factory) as session:
- assert current_tenant_id() is None
- assert await _current_setting(session) is None
- assert current_tenant_id() == TENANT_A
- assert current_tenant_id() is None
-
- async def test_an_exception_still_restores_the_outer_tenant(
- self, session_factory: async_sessionmaker[AsyncSession]
- ) -> None:
- with tenant_scope(TENANT_A):
- with pytest.raises(RuntimeError, match="boom"):
- async with unscoped_session(session_factory):
- raise RuntimeError("boom")
- assert current_tenant_id() == TENANT_A
-
- @pytest.mark.no_ambient_tenant
- async def test_current_tenant_id_is_unchanged_after_the_block(
- self, session_factory: async_sessionmaker[AsyncSession]
- ) -> None:
- assert current_tenant_id() is None
- async with unscoped_session(session_factory):
- pass
- assert current_tenant_id() is None
-
- async def test_it_reads_rows_from_two_tenants_with_one_of_them_bound(
- self, session_factory: async_sessionmaker[AsyncSession]
- ) -> None:
- """What the callers actually rely on, stated in rows rather than in
- contextvars: a sweep or an enumeration reached from inside a bound
- context still sees the whole deployment.
-
- Today no policy filters anything, so the `set_config` assertion is
- what carries the weight — it is the value the policies will read once
- they land. Both are asserted so this test keeps meaning the same thing
- on either side of that change.
- """
- tenant_a = f"tenant-{uuid.uuid4().hex[:8]}"
- tenant_b = f"tenant-{uuid.uuid4().hex[:8]}"
- async with session_factory() as session:
- for tenant_id in (tenant_a, tenant_b):
- session.add(Tenant(id=tenant_id, slug=tenant_id, name=tenant_id))
- await session.flush()
- for tenant_id in (tenant_a, tenant_b):
- session.add(
- Room(
- tenant_id=tenant_id,
- matrix_room_id=f"!{tenant_id}:test",
- name=tenant_id,
- description="",
- )
- )
- await session.commit()
-
- with tenant_scope(tenant_a):
- async with unscoped_session(session_factory) as session:
- assert await _current_setting(session) is None
- rows = (await session.execute(select(Room.tenant_id))).scalars().all()
- assert {tenant_a, tenant_b} <= set(rows)
diff --git a/core/tests/switch_core/db/test_tenant_exemption_allowlist.py b/core/tests/switch_core/db/test_tenant_exemption_allowlist.py
new file mode 100644
index 000000000..0bdaf3733
--- /dev/null
+++ b/core/tests/switch_core/db/test_tenant_exemption_allowlist.py
@@ -0,0 +1,381 @@
+"""Who may reach the exemption from row-level security, and who may open a
+session without saying which tenant it is for.
+
+Two audit surfaces, and neither is the enforcement — the database is. That is
+the difference between this file and the one it replaces
+(`test_unscoped_session_allowlist.py`). `unscoped_session` genuinely read
+across every tenant, so the list of its callers *was* the isolation boundary
+and a missing entry was a leak. It only did that because every environment
+connected as the tables' owner; under the restricted runtime role an unbound
+session reads nothing. So these lists are now what they always should have
+been: an inventory a reviewer can read, backed by a database that refuses
+regardless.
+
+**`db/tenant_lookup.py`** is the whole exemption: eight `SECURITY DEFINER`
+functions that answer *which tenant* and never return a row. Every module that
+imports one is named below. Keep the list short — it is the complete set of
+places in the process that ask a question no tenant can be scoped to.
+
+**A raw `session_factory()` call** is the other surface. It inherits whatever
+is ambient, which in background code is nothing at all, since the long-lived
+tasks deliberately unbind (see `tenant_context.no_tenant`). Such a session can
+now only read the tables that carry no policy — `users`, `oidc_identities`,
+`feature_flags` — so it is no longer a way to cross a tenant boundary, but it
+is still a session whose call site did not say what it was for.
+
+A module failing either check is not a bug in the test. It means someone
+reached for the exemption, or opened a session with nothing bound, without
+saying why — and either that is deliberate and reviewable, in which case add
+the module below and argue for it in the commit, or it is not, in which case
+use `tenant_session` and bind the tenant of the row the work is acting on.
+"""
+
+from __future__ import annotations
+
+import ast
+from pathlib import Path
+
+import switch_core
+from switch_core.db.tenant_lookup import TENANT_LOOKUPS
+
+# Every module allowed to call one of the tenant lookups. This is the whole
+# audit surface of the exemption: each entry is a place that has to answer
+# "which tenant" before it can scope anything.
+_ALLOWED_MODULES = {
+ # Startup: which tenants exist, to fan the boot work out over them, and
+ # which tenant holds the deployment-wide bootstrap key.
+ "switch_core.main",
+ # Reconciling every room's client membership at startup fans out over
+ # every tenant's rooms before binding each room's own tenant.
+ "switch_core.room_service",
+ # The runtime-state sweep reads every tenant's stale rows, one tenant at a
+ # time; `register_agent_with_token` resolves a registration credential by
+ # its globally unique hash, which is the read that produces a tenant.
+ "switch_core.bridges.agent.protocol.service",
+ # Bearer and OIDC authentication: the credential's tenant, before its row.
+ "switch_core.bridges.agent.auth",
+ # The gateway's JWT subject resolves to its membership, and
+ # `tenant_members` is scoped, so nothing else can answer it.
+ "switch_core.db.stores.tenant_member_store",
+ # Enumerating tenants at boot, and starting one bridge or one connector by
+ # id from a context bound to somebody else's tenant.
+ "switch_core.clients.client_lifecycle_service",
+ "switch_core.bridges.collaboration.lifecycle_service",
+ "switch_core.bridges.agent.server_connectors.lifecycle",
+ # A client's own task binds nothing, so its transport and its agent
+ # resolution derive the tenant from the client row.
+ "switch_core.clients.agent_client",
+ "switch_core.transport.postgres",
+ # `_room_tenant`'s fallback: which tenant is this room in, asked when the
+ # answer is not already cached alongside the channel mapping.
+ "switch_core.bridges.collaboration.bridge_core",
+}
+
+# Every module allowed to open a session straight from the factory. Not short,
+# and not meant to be permanent: an inventory of what has not been converted
+# yet, plus the request-path modules where inheriting the authenticated
+# request's tenant is the correct answer.
+_RAW_SESSION_FACTORY_MODULES = {
+ # ── Request path: the session inherits the authenticated caller's tenant,
+ # bound by the auth dependency before the endpoint body runs. Correct as
+ # it stands; these are not to-dos.
+ "switch_core.gateway.auth",
+ "switch_core.gateway.dependencies",
+ "switch_core.bridges.agent.dependencies",
+ "switch_core.bridges.agent.operations.context",
+ "switch_core.bridges.agent.operations.definitions",
+ "switch_core.bridges.agent.mediation",
+ # Reached only from the gateway's rooms endpoints, so the same holds.
+ "switch_core.rooms_yaml",
+ # ── Reached only from inside a unit of work that has already bound the
+ # tenant of the row it is acting on — an inbound bridge event, a delivery,
+ # a sweep row, an authenticated agent operation.
+ "switch_core.bridges.agent.protocol.service",
+ "switch_core.bridges.agent.commands",
+ "switch_core.bridges.collaboration.bridge_core",
+ "switch_core.bridges.collaboration.lifecycle_service",
+ "switch_core.bridges.agent.server_connectors.lifecycle",
+ "switch_core.clients.agent_client",
+ "switch_core.clients.admin_client",
+ "switch_core.clients.client_lifecycle_service",
+ "switch_core.provisioning.postgres",
+ "switch_core.room_service",
+ # ── The exemption's own plumbing. It opens a session with nothing bound
+ # on purpose and touches only the eight functions above, which are the one
+ # thing a session with nothing bound may read.
+ "switch_core.db.tenant_lookup",
+ # `switch_core.transport.postgres` and `switch_core.bridges.agent.auth`
+ # came off this list with the runtime role: the transport learned its own
+ # tenant from its client row and the middleware learned a credential's
+ # from the exemption, so neither has a session left that says nothing.
+ # `switch_core.main` came off it earlier, when the tenant-zero fallback
+ # went.
+}
+
+# Calls that end in `session_factory` but hand one back rather than open a
+# session with it.
+_FACTORY_ACCESSORS = {"create_session_factory", "get_session_factory"}
+
+_LOOKUP_NAMES = {lookup.name for lookup in TENANT_LOOKUPS}
+
+_PACKAGE_ROOT = Path(switch_core.__file__).resolve().parent
+
+# The modules that define the two things, rather than reaching for them.
+_LOOKUP_DEFINITION = _PACKAGE_ROOT / "db" / "tenant_lookup.py"
+_SESSION_SCOPE = _PACKAGE_ROOT / "db" / "session_scope.py"
+
+
+def _module_name(path: Path) -> str:
+ relative = path.relative_to(_PACKAGE_ROOT.parent)
+ parts = list(relative.with_suffix("").parts)
+ if parts[-1] == "__init__":
+ parts = parts[:-1]
+ return ".".join(parts)
+
+
+def _imports_a_lookup(path: Path) -> bool:
+ """Whether `path` reaches for one of the exempt lookups.
+
+ Matched on the import rather than on the call, unlike the raw-factory
+ detector below, because there is a single module to import from: an
+ `import ... as` renames the local binding but not the name in the `from`
+ clause, so aliasing cannot walk past this. Importing without calling is
+ flagged too, which is the safe direction — a module that reaches for the
+ exemption at all is one this list wants to name. Importing the DDL helpers
+ is not reaching for it: `TENANT_LOOKUPS` and `attach_tenant_lookups`
+ describe the exemption rather than using it.
+ """
+ tree = ast.parse(path.read_text(), filename=str(path))
+ for node in ast.walk(tree):
+ if not isinstance(node, ast.ImportFrom):
+ continue
+ if node.module != "switch_core.db.tenant_lookup":
+ continue
+ if any(alias.name in _LOOKUP_NAMES for alias in node.names):
+ return True
+ return False
+
+
+def _called_names(tree: ast.AST) -> set[str]:
+ """The name of every function called anywhere, attribute access included.
+
+ `foo()` yields "foo"; `self._session_factory()` yields "_session_factory";
+ `_state["session_factory"]()` yields "session_factory". Chains collapse to
+ their last element, which is what the detector keys on — a session factory
+ is reached through something far more often than as a bare name, and the
+ gateway reaches it through a module-level dict, which a name-and-attribute
+ -only walk misses entirely.
+ """
+ called = set()
+ for node in ast.walk(tree):
+ if not isinstance(node, ast.Call):
+ continue
+ func = node.func
+ if isinstance(func, ast.Name):
+ called.add(func.id)
+ elif isinstance(func, ast.Attribute):
+ called.add(func.attr)
+ elif isinstance(func, ast.Subscript) and isinstance(func.slice, ast.Constant):
+ if isinstance(func.slice.value, str):
+ called.add(func.slice.value)
+ return called
+
+
+def _calls_session_factory(path: Path) -> bool:
+ """Whether `path` opens a session straight from a factory.
+
+ Matches any call whose name ends in `session_factory`, which is how every
+ such call in this tree is spelled — `session_factory()`,
+ `self._session_factory()`, `self.session_factory()`.
+
+ A name match rather than a resolved reference: there is no single
+ definition to alias, since every service is handed its own factory. That
+ makes it over-eager rather than under-eager, which is the safe direction
+ for an audit — except for the two accessors that hand a factory back
+ instead of opening a session with it, which are named.
+ """
+ tree = ast.parse(path.read_text(), filename=str(path))
+ return any(
+ name.endswith("session_factory") and name not in _FACTORY_ACCESSORS
+ for name in _called_names(tree)
+ )
+
+
+def _modules_where(predicate: object, *, skip: Path | None = None) -> set[str]:
+ found = set()
+ for path in _PACKAGE_ROOT.rglob("*.py"):
+ if skip is not None and path == skip:
+ continue
+ if predicate(path): # type: ignore[operator]
+ found.add(_module_name(path))
+ return found
+
+
+def test_only_the_allowed_modules_reach_the_exemption() -> None:
+ found = _modules_where(_imports_a_lookup, skip=_LOOKUP_DEFINITION)
+ unexpected = found - _ALLOWED_MODULES
+ assert not unexpected, (
+ f"{sorted(unexpected)} import a tenant lookup but are not in "
+ "_ALLOWED_MODULES in this test. Those functions are the whole "
+ "exemption from row-level security (db/tenant_lookup.py) — if this "
+ "new call site genuinely has to answer 'which tenant' before it can "
+ "scope anything (a credential, a boot enumeration, an id reached with "
+ "nothing bound), add its module below and say why in the commit; "
+ "otherwise use tenant_session and bind the tenant the work is for."
+ )
+
+
+def test_the_allowlist_names_no_stale_module() -> None:
+ """The other direction: every allowed module still actually reaches it, so
+ the list stays the audit surface it claims to be rather than growing
+ entries nobody needs."""
+ found = _modules_where(_imports_a_lookup, skip=_LOOKUP_DEFINITION)
+ stale = _ALLOWED_MODULES - found
+ assert not stale, f"{sorted(stale)} no longer reach the exemption; remove them"
+
+
+def test_no_module_still_reaches_for_an_unscoped_session() -> None:
+ """`unscoped_session` is gone, and nothing may bring it back by name.
+
+ A helper called that again would look like the fail-open hatch this design
+ used to have and would not be one: under the runtime role an unbound
+ session reads nothing rather than everything, so the name would promise
+ the opposite of what it delivered. Cheap to assert, and it catches a
+ revert.
+ """
+ import switch_core.db.session_scope as session_scope
+
+ assert not hasattr(session_scope, "unscoped_session")
+ offenders = sorted(
+ _module_name(path)
+ for path in _PACKAGE_ROOT.rglob("*.py")
+ if "unscoped_session(" in path.read_text()
+ )
+ assert not offenders, f"{offenders} still call unscoped_session"
+
+
+def test_only_the_approved_modules_open_a_raw_session() -> None:
+ """A raw factory call takes whatever tenant is ambient — which in
+ background code, now that the long-lived tasks unbind, is none."""
+ found = _modules_where(_calls_session_factory, skip=_SESSION_SCOPE)
+ unexpected = found - _RAW_SESSION_FACTORY_MODULES
+ assert not unexpected, (
+ f"{sorted(unexpected)} open a session straight from the factory but "
+ "are not in _RAW_SESSION_FACTORY_MODULES in this test. Such a session "
+ "inherits whatever tenant is bound, which in background code is "
+ "nothing at all — and a session with nothing bound can only read the "
+ "tables that carry no policy. Say which you mean: tenant_session("
+ "factory, <the row's tenant>) for work that acts on one tenant's "
+ "rows, or one of the lookups in db/tenant_lookup.py for the question "
+ "that produces a tenant. If the call really is reached only from an "
+ "authenticated request, add the module below and say so in the commit."
+ )
+
+
+def test_the_raw_session_list_names_no_stale_module() -> None:
+ found = _modules_where(_calls_session_factory, skip=_SESSION_SCOPE)
+ stale = _RAW_SESSION_FACTORY_MODULES - found
+ assert not stale, (
+ f"{sorted(stale)} no longer open a raw session — remove them from "
+ "_RAW_SESSION_FACTORY_MODULES. Shrinking this list is the point; "
+ "leaving converted modules in it hides the progress."
+ )
+
+
+class TestTheDetectorsCatchANewCaller:
+ """The tests above only prove today's tree is clean. These exercise the
+ detectors against files that are not in the real source tree at all, so
+ they stand on their own even if every real call site is later removed."""
+
+ def test_a_lookup_import_is_detected(self, tmp_path: Path) -> None:
+ new_caller = tmp_path / "a_new_background_job.py"
+ new_caller.write_text(
+ "from switch_core.db.tenant_lookup import all_tenant_ids\n"
+ "async def sweep(session_factory):\n"
+ " return await all_tenant_ids(session_factory)\n"
+ )
+ assert _imports_a_lookup(new_caller)
+
+ def test_an_aliased_lookup_import_is_detected(self, tmp_path: Path) -> None:
+ """`import ... as` must not walk past the check. The detector matches
+ the imported name inside the `from` clause, which an alias leaves
+ alone."""
+ new_caller = tmp_path / "a_new_background_job.py"
+ new_caller.write_text(
+ "from switch_core.db.tenant_lookup import all_tenant_ids as every\n"
+ "async def sweep(session_factory):\n"
+ " return await every(session_factory)\n"
+ )
+ assert _imports_a_lookup(new_caller)
+
+ def test_importing_the_ddl_helpers_is_not_reaching_for_it(
+ self, tmp_path: Path
+ ) -> None:
+ """`db/models.py` and the migration test import from this module
+ without using the exemption; flagging them would make the list mean
+ something other than what it says."""
+ schema_code = tmp_path / "schema.py"
+ schema_code.write_text(
+ "from switch_core.db.tenant_lookup import attach_tenant_lookups\n"
+ )
+ assert not _imports_a_lookup(schema_code)
+
+ def test_a_module_with_no_call_is_not_flagged(self, tmp_path: Path) -> None:
+ clean = tmp_path / "an_unrelated_module.py"
+ clean.write_text("async def do_work(session):\n ...\n")
+ assert not _imports_a_lookup(clean)
+ assert not _calls_session_factory(clean)
+
+ def test_a_bare_factory_call_is_detected(self, tmp_path: Path) -> None:
+ new_caller = tmp_path / "a_new_background_job.py"
+ new_caller.write_text(
+ "async def sweep(session_factory):\n"
+ " async with session_factory() as session:\n"
+ " ...\n"
+ )
+ assert _calls_session_factory(new_caller)
+
+ def test_a_factory_reached_through_self_is_detected(self, tmp_path: Path) -> None:
+ new_caller = tmp_path / "a_new_service.py"
+ new_caller.write_text(
+ "class Service:\n"
+ " async def run(self):\n"
+ " async with self._session_factory() as session:\n"
+ " ...\n"
+ )
+ assert _calls_session_factory(new_caller)
+
+ def test_a_factory_reached_through_a_dict_is_detected(self, tmp_path: Path) -> None:
+ """How the gateway holds its factory. Worth its own case: a walk over
+ names and attributes alone reports this module as clean."""
+ new_caller = tmp_path / "a_new_dependency.py"
+ new_caller.write_text(
+ "_state = {}\n"
+ "async def get_session():\n"
+ ' async with _state["session_factory"]() as session:\n'
+ " yield session\n"
+ )
+ assert _calls_session_factory(new_caller)
+
+ def test_handing_a_factory_back_is_not_opening_a_session(
+ self, tmp_path: Path
+ ) -> None:
+ clean = tmp_path / "wiring.py"
+ clean.write_text(
+ "def wire(engine):\n"
+ " return create_session_factory(engine)\n"
+ "def fetch():\n"
+ " return get_session_factory()\n"
+ )
+ assert not _calls_session_factory(clean)
+
+ def test_the_allowlist_check_would_fail_for_a_hypothetical_new_caller(self) -> None:
+ """Simulates what `test_only_the_allowed_modules_reach_the_exemption`
+ does, against a found-set that includes a module nobody added to
+ `_ALLOWED_MODULES` — pinning that the check actually fails rather
+ than, say, silently ignoring an unknown module."""
+ found = _modules_where(_imports_a_lookup, skip=_LOOKUP_DEFINITION) | {
+ "switch_core.a_new_sweep"
+ }
+ unexpected = found - _ALLOWED_MODULES
+ assert unexpected == {"switch_core.a_new_sweep"}
diff --git a/core/tests/switch_core/db/test_tenant_lookup.py b/core/tests/switch_core/db/test_tenant_lookup.py
new file mode 100644
index 000000000..03d7f3b02
--- /dev/null
+++ b/core/tests/switch_core/db/test_tenant_lookup.py
@@ -0,0 +1,512 @@
+"""The exemption from row-level security, asked of a role the policies apply to.
+
+Everything here runs against `rls_harness.restricted`, never `session_factory`
+or `rls_harness.owner`. That is not a preference: the owner is exempt from
+every policy in this schema by ownership, so a test of the exemption asked of
+the owner passes whether or not the exemption exists.
+
+**Every arrangement leaves the table it reads populated.** Postgres does not
+evaluate a policy for a scan that yields no rows, so a "the policy refuses
+this" assertion against an empty table is satisfied by a database with no
+policies in it at all. Two of the nineteen call sites this exemption was built
+for were invisible for exactly that reason — they returned nothing, silently,
+until a row existed — so the rule is written down here rather than left to
+whoever adds the next test.
+
+Four things are pinned:
+
+- **the catalogue**: the functions installed are exactly `TENANT_LOOKUPS`, and
+ each one is `SECURITY DEFINER`, `STABLE`, and carries a search path that a
+ temporary table cannot shadow;
+- **the behaviour**: each answers correctly for the restricted role, against
+ two tenants, where the equivalent ordinary read answers nothing;
+- **the boundary**: they return tenant ids and never rows, and the tables they
+ read are still refused to the same role in the same session;
+- **the constraint they rest on**: no scoped table has `force row level
+ security`, which would take the ownership exemption away from them and leave
+ nothing in the process able to resolve a credential.
+"""
+
+from __future__ import annotations
+
+import uuid
+from pathlib import Path
+from types import ModuleType
+
+import pytest
+from alembic.config import Config
+from alembic.script import ScriptDirectory
+from sqlalchemy import text
+from sqlalchemy.exc import DBAPIError
+
+import switch_core
+from switch_core.db.models import (
+ Agent,
+ ApiKey,
+ Client,
+ CollaborationBridge,
+ Room,
+ ServerConnector,
+ Tenant,
+ TenantMember,
+ User,
+)
+from switch_core.db.tenant_lookup import (
+ SECURE_SEARCH_PATH,
+ TENANT_LOOKUPS,
+ TenantLookupError,
+ all_tenant_ids,
+ create_lookup_ddl,
+ tenant_of_agent_oauth_client,
+ tenant_of_api_key,
+ tenant_of_client,
+ tenant_of_collaboration_bridge,
+ tenant_of_room,
+ tenant_of_server_connector,
+ tenants_of_user,
+)
+from switch_core.tenant_context import tenant_scope
+from tests.conftest import RLSHarness
+
+pytestmark = pytest.mark.no_ambient_tenant
+
+# The revision that installs the exemption for a database built by Alembic.
+_LOOKUP_REVISION = "9c41a7b0e5d8"
+
+
+def _migration_module() -> ModuleType:
+ """The `9c41a7b0e5d8` revision module, loaded through Alembic.
+
+ Alembic's own loader rather than an `importlib` call on a path, so this
+ finds the file the same way a deployment would and fails the same way if
+ the revision were renamed or removed.
+ """
+ core = Path(switch_core.__file__).resolve().parents[1]
+ config = Config(str(core / "alembic.ini"))
+ config.set_main_option("script_location", str(core / "switch_core" / "migrations"))
+ return ScriptDirectory.from_config(config).get_revision(_LOOKUP_REVISION).module
+
+
+class _Fixture:
+ """The ids two populated tenants leave behind, for the assertions below."""
+
+ def __init__(self) -> None:
+ self.tenant_a: str = ""
+ self.tenant_b: str = ""
+ self.client_a: str = ""
+ self.room_a: str = ""
+ self.bridge_a: str = ""
+ self.connector_a: str = ""
+ self.key_hash_a: str = ""
+ self.oauth_client_a: str = ""
+ self.user_a: str = ""
+ self.user_in_both: str = ""
+
+
+async def _two_populated_tenants(harness: RLSHarness) -> _Fixture:
+ """Two tenants, each with a full set of the rows the lookups read.
+
+ Both tenants are populated rather than just the one under test, because
+ the question every lookup answers is "which of them", and a single-tenant
+ arrangement cannot tell a correct answer apart from the only answer
+ available. Written through the owner: creating a second `tenants` row is
+ necessarily a system operation, since that table's policy compares on `id`
+ and no session can be bound to two tenants at once.
+ """
+ fixture = _Fixture()
+ suffix = uuid.uuid4().hex[:8]
+ fixture.tenant_a = f"tenant-a-{suffix}"
+ fixture.tenant_b = f"tenant-b-{suffix}"
+ fixture.oauth_client_a = f"oauth-{suffix}"
+ fixture.key_hash_a = f"hash-a-{suffix}"
+
+ async with harness.owner() as session:
+ for tenant_id in (fixture.tenant_a, fixture.tenant_b):
+ session.add(Tenant(id=tenant_id, slug=tenant_id, name=tenant_id))
+ await session.flush()
+
+ user_a = User(name="A", email=f"a-{suffix}@example.test", role="user")
+ user_both = User(name="Both", email=f"both-{suffix}@example.test", role="user")
+ session.add_all([user_a, user_both])
+ await session.flush()
+ fixture.user_a, fixture.user_in_both = user_a.id, user_both.id
+ session.add_all(
+ [
+ TenantMember(
+ tenant_id=fixture.tenant_a, user_id=user_a.id, role="member"
+ ),
+ TenantMember(
+ tenant_id=fixture.tenant_a, user_id=user_both.id, role="member"
+ ),
+ TenantMember(
+ tenant_id=fixture.tenant_b, user_id=user_both.id, role="member"
+ ),
+ ]
+ )
+
+ for tenant_id, tag in ((fixture.tenant_a, "a"), (fixture.tenant_b, "b")):
+ client = Client(
+ tenant_id=tenant_id,
+ matrix_user_id=f"@client-{tag}-{suffix}:test",
+ display_name=f"client {tag}",
+ type="agent",
+ )
+ room = Room(
+ tenant_id=tenant_id,
+ matrix_room_id=f"!room-{tag}-{suffix}:test",
+ name=f"room {tag}",
+ description="",
+ )
+ key = ApiKey(
+ tenant_id=tenant_id,
+ user_id=user_both.id,
+ key_hash=f"hash-{tag}-{suffix}",
+ encrypted_key="x",
+ label=f"key {tag}",
+ type="agent",
+ )
+ session.add_all([client, room, key])
+ await session.flush()
+ bridge = CollaborationBridge(
+ tenant_id=tenant_id,
+ type="slack",
+ display_name=f"bridge {tag}",
+ status="active",
+ connection_config={},
+ client_id=client.id,
+ )
+ connector = ServerConnector(
+ tenant_id=tenant_id,
+ type="opencode",
+ display_name=f"connector {tag}",
+ api_key_id=key.id,
+ status="active",
+ connection_config={},
+ )
+ session.add_all([bridge, connector])
+ session.add(
+ Agent(
+ tenant_id=tenant_id,
+ name=f"agent-{tag}-{suffix}",
+ description="",
+ owner_id=user_both.id,
+ client_id=client.id,
+ api_key_id=key.id,
+ agent_type="always_on",
+ integration_profile={"connection_model": "always_on"},
+ connector_type="test",
+ # Both tenants carry the same value on a column with no
+ # unique index: the one lookup that can legitimately
+ # answer twice, and the case it has to refuse.
+ oauth_client_id=fixture.oauth_client_a,
+ )
+ )
+ await session.flush()
+ if tag == "a":
+ fixture.client_a = client.id
+ fixture.room_a = room.id
+ fixture.bridge_a = bridge.id
+ fixture.connector_a = connector.id
+ await session.commit()
+ return fixture
+
+
+class TestTheCatalogueIsWhatIsInstalled:
+ async def test_every_lookup_exists_and_nothing_else_does(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """The set of exempt functions in the database is exactly the list in
+ `db/tenant_lookup.py`.
+
+ Both directions matter, and the second more than the first: a
+ `SECURITY DEFINER` function nobody has written down is an exemption
+ nobody is auditing. `require_tenant_id` is the only other function
+ here that is not one, and it is not `SECURITY DEFINER`, which is what
+ this test keys on rather than a name.
+ """
+ async with rls_harness.restricted() as session:
+ rows = (
+ await session.execute(
+ text(
+ """
+ SELECT p.proname
+ FROM pg_proc p
+ JOIN pg_namespace n ON n.oid = p.pronamespace
+ WHERE n.nspname = 'public' AND p.prosecdef
+ """
+ )
+ )
+ ).scalars()
+ installed = set(rows)
+ assert installed == {lookup.name for lookup in TENANT_LOOKUPS}
+
+ async def test_each_one_is_stable_and_cannot_have_its_tables_shadowed(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """`STABLE` so the planner evaluates it once rather than per row, and
+ a `search_path` with `pg_temp` last so a role able to create a
+ temporary table cannot put its own `tenants` in front of the real one
+ and have a function running as the owner read it."""
+ async with rls_harness.restricted() as session:
+ rows = (
+ await session.execute(
+ text(
+ """
+ SELECT p.proname, p.provolatile::text, p.proconfig
+ FROM pg_proc p
+ JOIN pg_namespace n ON n.oid = p.pronamespace
+ WHERE n.nspname = 'public' AND p.prosecdef
+ """
+ )
+ )
+ ).all()
+ assert rows
+ for name, volatility, config in rows:
+ assert volatility == "s", f"{name} is not STABLE"
+ assert config is not None, f"{name} sets no search_path"
+ paths = [entry for entry in config if entry.startswith("search_path=")]
+ assert paths, f"{name} sets no search_path: {config}"
+ assert paths[0].endswith("pg_temp"), (
+ f"{name} does not put pg_temp last in its search_path: {paths[0]}"
+ )
+
+ async def test_no_scoped_table_forces_row_level_security(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """The constraint the whole exemption rests on.
+
+ `SECURITY DEFINER` is only half of why these functions can read across
+ tenants: the other half is that they run as the schema owner, and
+ Postgres exempts an owner from its own tables' policies *unless* the
+ table sets `force row level security`. Setting it anywhere would leave
+ nothing in the process able to answer "which tenant is this
+ credential in", so every request would fail to authenticate. Boot
+ refuses to start for the same reason (`db/runtime_role.py`); this
+ catches it in CI, where the schema is actually built.
+ """
+ async with rls_harness.restricted() as session:
+ forced = (
+ await session.execute(
+ text(
+ """
+ SELECT c.relname
+ FROM pg_class c
+ JOIN pg_namespace n ON n.oid = c.relnamespace
+ WHERE n.nspname = 'public' AND c.relforcerowsecurity
+ """
+ )
+ )
+ ).scalars()
+ assert list(forced) == []
+
+
+class TestWhatTheyAnswer:
+ async def test_all_tenant_ids_sees_every_tenant(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ fixture = await _two_populated_tenants(rls_harness)
+ found = await all_tenant_ids(rls_harness.restricted)
+ assert {fixture.tenant_a, fixture.tenant_b} <= set(found)
+
+ async def test_the_same_question_asked_the_ordinary_way_sees_one_tenant(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """The measurement that makes the previous test mean something.
+
+ `tenants` carries the same policy as everything else, comparing on
+ `id`, so a session bound to a tenant sees exactly its own row — and
+ one bound to nothing sees an error rather than every row. Both are
+ asserted here so the exemption is measured against the behaviour it
+ exists to work around, rather than against nothing.
+ """
+ fixture = await _two_populated_tenants(rls_harness)
+ with tenant_scope(fixture.tenant_a):
+ async with rls_harness.restricted() as session:
+ visible = (
+ await session.execute(text("SELECT id FROM tenants"))
+ ).scalars()
+ assert list(visible) == [fixture.tenant_a]
+
+ async with rls_harness.restricted() as session:
+ with pytest.raises(DBAPIError) as raised:
+ await session.execute(text("SELECT id FROM tenants"))
+ assert "app.tenant_id is not set" in str(raised.value)
+
+ async def test_a_membership_resolves_to_its_tenants(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ fixture = await _two_populated_tenants(rls_harness)
+ assert await tenants_of_user(rls_harness.restricted, fixture.user_a) == [
+ fixture.tenant_a
+ ]
+ assert sorted(
+ await tenants_of_user(rls_harness.restricted, fixture.user_in_both)
+ ) == sorted([fixture.tenant_a, fixture.tenant_b])
+
+ async def test_an_api_key_hash_resolves_to_its_tenant(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ fixture = await _two_populated_tenants(rls_harness)
+ suffix = fixture.key_hash_a.rsplit("-", 1)[-1]
+ assert (
+ await tenant_of_api_key(rls_harness.restricted, f"hash-a-{suffix}")
+ == fixture.tenant_a
+ )
+ assert (
+ await tenant_of_api_key(rls_harness.restricted, f"hash-b-{suffix}")
+ == fixture.tenant_b
+ )
+
+ async def test_an_unknown_key_resolves_to_nothing_rather_than_raising(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """An unrecognised bearer token is a 401, not a 500. The middleware
+ stops here and never issues the second, scoped query, which is also
+ what keeps an unauthenticated flood to one round trip."""
+ await _two_populated_tenants(rls_harness)
+ assert await tenant_of_api_key(rls_harness.restricted, "no-such-hash") is None
+
+ async def test_a_client_room_bridge_and_connector_resolve_to_their_tenant(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ fixture = await _two_populated_tenants(rls_harness)
+ restricted = rls_harness.restricted
+ assert await tenant_of_client(restricted, fixture.client_a) == fixture.tenant_a
+ assert await tenant_of_room(restricted, fixture.room_a) == fixture.tenant_a
+ assert (
+ await tenant_of_collaboration_bridge(restricted, fixture.bridge_a)
+ == fixture.tenant_a
+ )
+ assert (
+ await tenant_of_server_connector(restricted, fixture.connector_a)
+ == fixture.tenant_a
+ )
+
+ async def test_an_ambiguous_answer_is_refused_rather_than_picked(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """`agents.oauth_client_id` carries no unique index, so two tenants
+ can register an agent under the same one. Resolving that to a tenant
+ by taking the first row would authenticate a caller into somebody
+ else's data on the strength of a duplicate — a provisioning fault
+ turned into an authorization one — so it raises."""
+ fixture = await _two_populated_tenants(rls_harness)
+ with pytest.raises(TenantLookupError) as raised:
+ await tenant_of_agent_oauth_client(
+ rls_harness.restricted, fixture.oauth_client_a
+ )
+ assert "refusing to pick one" in str(raised.value)
+
+
+class TestTheBoundary:
+ async def test_a_lookup_leaves_the_tables_it_read_still_refused(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """The exemption is the function, not the caller and not the session.
+
+ A caller that has just learned a tenant id from `tenant_of_api_key`
+ has learned exactly that, and reading the row it named still needs the
+ tenant bound. Asserted in the same session so there is no question of
+ a different connection answering.
+ """
+ fixture = await _two_populated_tenants(rls_harness)
+ suffix = fixture.key_hash_a.rsplit("-", 1)[-1]
+ assert (
+ await tenant_of_api_key(rls_harness.restricted, f"hash-a-{suffix}")
+ == fixture.tenant_a
+ )
+ async with rls_harness.restricted() as session:
+ assert (
+ await session.execute(text("SELECT count(*) FROM all_tenant_ids()"))
+ ).scalar_one() >= 2
+ with pytest.raises(DBAPIError) as raised:
+ await session.execute(text("SELECT id FROM api_keys"))
+ assert "app.tenant_id is not set" in str(raised.value)
+
+ async def test_a_lookup_does_not_inherit_the_callers_tenant(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """Reached from inside a bound context — a gateway request resolving a
+ bridge's tenant, say — the answer is still the row's own. The wrapper
+ unbinds for its own session precisely so that the enumeration is not
+ silently narrowed to whoever asked."""
+ fixture = await _two_populated_tenants(rls_harness)
+ with tenant_scope(fixture.tenant_a):
+ found = await all_tenant_ids(rls_harness.restricted)
+ assert (
+ await tenant_of_room(rls_harness.restricted, fixture.room_a)
+ == fixture.tenant_a
+ )
+ assert {fixture.tenant_a, fixture.tenant_b} <= set(found)
+
+ async def test_the_callers_binding_survives_a_lookup(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """Unbinding for the lookup must not unbind the caller. A request that
+ resolved something mid-flight and then found its own tenant gone would
+ fail on its next scoped read, which is a worse bug than the one the
+ unbinding prevents."""
+ fixture = await _two_populated_tenants(rls_harness)
+ with tenant_scope(fixture.tenant_a):
+ await all_tenant_ids(rls_harness.restricted)
+ async with rls_harness.restricted() as session:
+ still_visible = (
+ await session.execute(text("SELECT id FROM tenants"))
+ ).scalars()
+ assert list(still_visible) == [fixture.tenant_a]
+
+
+class TestTheMigrationInstallsTheSameThing:
+ """A frozen copy is the right shape for a migration — it must not change
+ meaning because a module later does — and its failure mode is falling
+ behind in silence: the schema `create_all` builds gets the live module's
+ functions, so every test above passes, while a deployment built by Alembic
+ gets whatever the migration froze.
+ """
+
+ def test_the_frozen_list_matches_the_live_one(self) -> None:
+ module = _migration_module()
+ assert module.SECURE_SEARCH_PATH == SECURE_SEARCH_PATH
+ frozen = {
+ (name, parameters, signature, body)
+ for name, parameters, signature, body in module.LOOKUPS
+ }
+ live = {
+ (
+ lookup.name,
+ "" if lookup.parameter is None else f"{lookup.parameter} text",
+ lookup.signature,
+ lookup.query,
+ )
+ for lookup in TENANT_LOOKUPS
+ }
+ assert frozen == live, (
+ "the frozen LOOKUPS in migration 9c41a7b0e5d8 no longer match "
+ "db/tenant_lookup.py. A deployment built by Alembic would get the "
+ "migration's functions and every test above would still pass "
+ "against create_all's. If the divergence is deliberate, express "
+ "it as a new migration rather than by editing this one."
+ )
+
+ def test_the_statement_it_would_run_is_the_statement_the_module_builds(
+ self,
+ ) -> None:
+ """The list agreeing is not enough: the template matters too.
+
+ A frozen copy naming every function and every body correctly could
+ still say `SECURITY INVOKER`, or drop the `search_path`, and the check
+ above would pass. Nothing else would catch it — this suite builds its
+ schema from the models, so the migration's own DDL is never executed
+ here, and an Alembic-built database would install eight functions
+ unable to resolve a tenant with the whole suite green. So what is
+ compared is the rendered statement, not the inputs to it.
+ """
+ module = _migration_module()
+ for lookup in TENANT_LOOKUPS:
+ parameters = "" if lookup.parameter is None else f"{lookup.parameter} text"
+ assert module.create_lookup_ddl(
+ lookup.name, parameters, lookup.query
+ ) == create_lookup_ddl(lookup), (
+ f"migration 9c41a7b0e5d8 would install {lookup.name} with "
+ "different DDL from the one db/tenant_lookup.py builds."
+ )
diff --git a/core/tests/switch_core/db/test_unscoped_session_allowlist.py b/core/tests/switch_core/db/test_unscoped_session_allowlist.py
deleted file mode 100644
index b04f9718e..000000000
--- a/core/tests/switch_core/db/test_unscoped_session_allowlist.py
+++ /dev/null
@@ -1,335 +0,0 @@
-"""What may open a database session without a tenant, and who is allowed to.
-
-Two audit surfaces, and the second is the bigger one.
-
-**`unscoped_session`** (`db/session_scope.py`) is the declared fail-open hatch:
-it unbinds for the duration of the block, so a query on it reads across every
-tenant whether or not the caller had one bound. Its callers are named here.
-
-**A raw `session_factory()` call** is the undeclared one. It inherits whatever
-is ambient — the request's tenant on the request path, and *nothing at all* in
-background code, since the long-lived tasks deliberately unbind (see
-`tenant_context.no_tenant`). So a raw call in background code is unscoped in
-fact while saying nothing about it, which is strictly worse than the hatch
-that announces itself. There are 174 of them across the nineteen modules
-below; that is the number this design has to work down, and pinning the
-module list is how a new one becomes a decision rather than a default.
-
-A module failing either check is not a bug in the test. It means someone
-opened a session without saying which tenant it is for, and either that is
-deliberate and reviewable — add the module below, saying why in the commit —
-or it is not, in which case use `tenant_session` and bind the tenant of the
-row the work is acting on.
-"""
-
-from __future__ import annotations
-
-import ast
-from pathlib import Path
-
-import switch_core
-
-# Every module allowed to call `unscoped_session`. Keep this short: it is the
-# whole list of places in the process that read across every tenant on purpose.
-_ALLOWED_MODULES = {
- # Startup seeding: the admin user and the agent-registration bootstrap
- # owner/key are created before any tenant can be said to exist.
- "switch_core.main",
- # Reconciling every room's client membership at startup fans out across
- # every tenant's rooms, before binding each room's own tenant for its own
- # work.
- "switch_core.room_service",
- # The runtime-state sweep reads every tenant's stale rows in one pass, and
- # `register_agent_with_token` resolves a credential by its globally unique
- # hash — the read that produces a tenant rather than one that uses it.
- "switch_core.bridges.agent.protocol.service",
- # Starting a bridge reads its own row to learn its tenant, from boot (with
- # nothing bound) and from a request (with the wrong one bound); and
- # `_reject_resource_conflict` must span tenants by definition, since two
- # tenants claiming one Slack workspace is the collision it exists to catch.
- "switch_core.bridges.collaboration.lifecycle_service",
- # `_room_tenant`'s fallback: which tenant is this room in, asked when the
- # answer is not already cached alongside the channel mapping.
- "switch_core.bridges.collaboration.bridge_core",
- # Same as the bridge lifecycle, for server-side connectors.
- "switch_core.bridges.agent.server_connectors.lifecycle",
- # Enumerating every tenant's clients at boot.
- "switch_core.clients.client_lifecycle_service",
- # A client's own task binds nothing, so resolving which agent it is — by
- # globally unique client id — cannot be scoped to a guess.
- "switch_core.clients.agent_client",
- # `joined_rooms` (which rooms is this client in) and `_resolve_room_and_
- # tenant` (which room, and so which tenant, is this transport id) are both
- # lookups that produce a tenant rather than consume one.
- "switch_core.transport.postgres",
-}
-
-# Every module allowed to open a session straight from the factory. Unlike the
-# list above this one is not short, and is not meant to be permanent: it is an
-# inventory of what has not been converted yet, plus the request-path modules
-# where inheriting the authenticated request's tenant is the correct answer.
-_RAW_SESSION_FACTORY_MODULES = {
- # ── Request path: the session inherits the authenticated caller's tenant,
- # bound by the auth dependency before the endpoint body runs. Correct as
- # it stands; these are not to-dos.
- "switch_core.gateway.auth",
- "switch_core.gateway.dependencies",
- "switch_core.bridges.agent.auth",
- "switch_core.bridges.agent.dependencies",
- "switch_core.bridges.agent.operations.context",
- "switch_core.bridges.agent.operations.definitions",
- "switch_core.bridges.agent.mediation",
- # Reached only from the gateway's rooms endpoints, so the same holds.
- "switch_core.rooms_yaml",
- # ── Reached only from inside a unit of work that has already bound the
- # tenant of the row it is acting on — an inbound bridge event, a delivery,
- # a sweep row, an authenticated agent operation.
- "switch_core.bridges.agent.protocol.service",
- "switch_core.bridges.agent.commands",
- "switch_core.bridges.collaboration.bridge_core",
- "switch_core.bridges.collaboration.lifecycle_service",
- "switch_core.bridges.agent.server_connectors.lifecycle",
- "switch_core.clients.agent_client",
- "switch_core.clients.admin_client",
- "switch_core.clients.client_lifecycle_service",
- "switch_core.provisioning.postgres",
- "switch_core.room_service",
- "switch_core.transport.postgres",
- # `switch_core.main` came off this list when the tenant-zero fallback
- # went. Every session startup opens now says which it is: a tenant
- # session for the admin-user seeding and for the per-tenant work at boot,
- # `unscoped_session` for the deployment-wide bootstrap key, which is
- # resolved by a globally unique hash and has no tenant to be scoped to.
-}
-
-# Calls that end in `session_factory` but hand one back rather than open a
-# session with it.
-_FACTORY_ACCESSORS = {"create_session_factory", "get_session_factory"}
-
-_PACKAGE_ROOT = Path(switch_core.__file__).resolve().parent
-
-# The module that defines the helpers, not a caller of them.
-_DEFINITION = _PACKAGE_ROOT / "db" / "session_scope.py"
-
-
-def _module_name(path: Path) -> str:
- relative = path.relative_to(_PACKAGE_ROOT.parent)
- parts = list(relative.with_suffix("").parts)
- if parts[-1] == "__init__":
- parts = parts[:-1]
- return ".".join(parts)
-
-
-def _local_names_for(tree: ast.AST, imported: str) -> set[str]:
- """Every local name `imported` is reachable under in this module.
-
- Its own name, plus any alias it was imported as. Without this the
- detectors match a spelling rather than a function, and one `import … as`
- walks straight past them.
- """
- names = {imported}
- for node in ast.walk(tree):
- if isinstance(node, ast.ImportFrom):
- for alias in node.names:
- if alias.name == imported and alias.asname:
- names.add(alias.asname)
- return names
-
-
-def _called_names(tree: ast.AST) -> set[str]:
- """The name of every function called anywhere, attribute access included.
-
- `foo()` yields "foo"; `self._session_factory()` yields "_session_factory";
- `_state["session_factory"]()` yields "session_factory". Chains collapse to
- their last element, which is what both detectors key on — a session
- factory is reached through something far more often than as a bare name,
- and the gateway reaches it through a module-level dict, which a
- name-and-attribute-only walk misses entirely.
- """
- called = set()
- for node in ast.walk(tree):
- if not isinstance(node, ast.Call):
- continue
- func = node.func
- if isinstance(func, ast.Name):
- called.add(func.id)
- elif isinstance(func, ast.Attribute):
- called.add(func.attr)
- elif isinstance(func, ast.Subscript) and isinstance(func.slice, ast.Constant):
- if isinstance(func.slice.value, str):
- called.add(func.slice.value)
- return called
-
-
-def _calls_unscoped_session(path: Path) -> bool:
- """Whether `path` calls `unscoped_session`, under any local name."""
- tree = ast.parse(path.read_text(), filename=str(path))
- return bool(_called_names(tree) & _local_names_for(tree, "unscoped_session"))
-
-
-def _calls_session_factory(path: Path) -> bool:
- """Whether `path` opens a session straight from a factory.
-
- Matches any call whose name ends in `session_factory`, which is how every
- such call in this tree is spelled — `session_factory()`,
- `self._session_factory()`, `self.session_factory()`.
-
- A name match, not a resolved reference, unlike `_calls_unscoped_session`:
- there is no single definition to alias, since every service is handed its
- own factory. That makes it over-eager rather than under-eager, which is
- the safe direction for an audit — except for the two accessors that hand a
- factory back instead of opening a session with it, which are named.
- """
- tree = ast.parse(path.read_text(), filename=str(path))
- return any(
- name.endswith("session_factory") and name not in _FACTORY_ACCESSORS
- for name in _called_names(tree)
- )
-
-
-def _modules_where(predicate: object) -> set[str]:
- found = set()
- for path in _PACKAGE_ROOT.rglob("*.py"):
- if path == _DEFINITION:
- continue
- if predicate(path): # type: ignore[operator]
- found.add(_module_name(path))
- return found
-
-
-def test_only_the_allowed_modules_call_unscoped_session() -> None:
- found = _modules_where(_calls_unscoped_session)
- unexpected = found - _ALLOWED_MODULES
- assert not unexpected, (
- f"{sorted(unexpected)} call unscoped_session but are not in "
- "_ALLOWED_MODULES in this test. That helper unbinds the tenant for "
- "the whole block (db/session_scope.py) — if this new call site is "
- "genuinely cross-tenant (a sweep, an enumeration, a lookup that "
- "answers *which* tenant something is in, startup seeding), add its "
- "module to _ALLOWED_MODULES and say why in the commit; otherwise use "
- "tenant_session and bind the tenant the work is actually for."
- )
-
-
-def test_the_allowlist_names_no_stale_module() -> None:
- """The other direction: every allowed module still actually uses it,
- so the list stays the audit surface it claims to be rather than growing
- entries nobody needs."""
- found = _modules_where(_calls_unscoped_session)
- stale = _ALLOWED_MODULES - found
- assert not stale, f"{sorted(stale)} no longer call unscoped_session; remove them"
-
-
-def test_only_the_approved_modules_open_a_raw_session() -> None:
- """The surface that actually matters, and the one this design is working
- down. A raw factory call takes whatever tenant is ambient — which in
- background code, now that the long-lived tasks unbind, is none."""
- found = _modules_where(_calls_session_factory)
- unexpected = found - _RAW_SESSION_FACTORY_MODULES
- assert not unexpected, (
- f"{sorted(unexpected)} open a session straight from the factory but "
- "are not in _RAW_SESSION_FACTORY_MODULES in this test. Such a session "
- "inherits whatever tenant is bound, which in background code is "
- "nothing at all. Say which you mean: tenant_session(factory, "
- "<the row's tenant>) for work that acts on one tenant's rows, "
- "unscoped_session(factory) for work that genuinely spans them. If "
- "the call really is reached only from an authenticated request, add "
- "the module below and say so in the commit."
- )
-
-
-def test_the_raw_session_list_names_no_stale_module() -> None:
- found = _modules_where(_calls_session_factory)
- stale = _RAW_SESSION_FACTORY_MODULES - found
- assert not stale, (
- f"{sorted(stale)} no longer open a raw session — remove them from "
- "_RAW_SESSION_FACTORY_MODULES. Shrinking this list is the point; "
- "leaving converted modules in it hides the progress."
- )
-
-
-class TestTheDetectorsCatchANewCaller:
- """The four tests above only prove today's tree is clean. These exercise
- the detectors against files that are not in the real source tree at all,
- so they stand on their own even if every real call site is later
- removed."""
-
- def test_a_plain_unscoped_call_is_detected(self, tmp_path: Path) -> None:
- new_caller = tmp_path / "a_new_background_job.py"
- new_caller.write_text(
- "async def sweep(session_factory):\n"
- " async with unscoped_session(session_factory) as session:\n"
- " ...\n"
- )
- assert _calls_unscoped_session(new_caller)
-
- def test_an_aliased_import_is_detected(self, tmp_path: Path) -> None:
- """`import … as` must not walk past the check. The detector resolves
- the local name the helper was bound to rather than matching the
- spelling `unscoped_session`."""
- new_caller = tmp_path / "a_new_background_job.py"
- new_caller.write_text(
- "from switch_core.db.session_scope import unscoped_session as us\n"
- "async def sweep(session_factory):\n"
- " async with us(session_factory) as session:\n"
- " ...\n"
- )
- assert _calls_unscoped_session(new_caller)
-
- def test_a_module_with_no_call_is_not_flagged(self, tmp_path: Path) -> None:
- clean = tmp_path / "an_unrelated_module.py"
- clean.write_text("async def do_work(session):\n ...\n")
- assert not _calls_unscoped_session(clean)
- assert not _calls_session_factory(clean)
-
- def test_a_bare_factory_call_is_detected(self, tmp_path: Path) -> None:
- new_caller = tmp_path / "a_new_background_job.py"
- new_caller.write_text(
- "async def sweep(session_factory):\n"
- " async with session_factory() as session:\n"
- " ...\n"
- )
- assert _calls_session_factory(new_caller)
-
- def test_a_factory_reached_through_self_is_detected(self, tmp_path: Path) -> None:
- new_caller = tmp_path / "a_new_service.py"
- new_caller.write_text(
- "class Service:\n"
- " async def run(self):\n"
- " async with self._session_factory() as session:\n"
- " ...\n"
- )
- assert _calls_session_factory(new_caller)
-
- def test_a_factory_reached_through_a_dict_is_detected(self, tmp_path: Path) -> None:
- """How the gateway holds its factory. Worth its own case: a walk over
- names and attributes alone reports this module as clean."""
- new_caller = tmp_path / "a_new_dependency.py"
- new_caller.write_text(
- "_state = {}\n"
- "async def get_session():\n"
- ' async with _state["session_factory"]() as session:\n'
- " yield session\n"
- )
- assert _calls_session_factory(new_caller)
-
- def test_handing_a_factory_back_is_not_opening_a_session(
- self, tmp_path: Path
- ) -> None:
- clean = tmp_path / "wiring.py"
- clean.write_text(
- "def wire(engine):\n"
- " return create_session_factory(engine)\n"
- "def fetch():\n"
- " return get_session_factory()\n"
- )
- assert not _calls_session_factory(clean)
-
- def test_the_allowlist_check_would_fail_for_a_hypothetical_new_caller(self) -> None:
- """Simulates what `test_only_the_allowed_modules_call_unscoped_session`
- does, against a found-set that includes a module nobody added to
- `_ALLOWED_MODULES` — pinning that the check actually fails rather
- than, say, silently ignoring an unknown module."""
- found = _modules_where(_calls_unscoped_session) | {"switch_core.a_new_sweep"}
- unexpected = found - _ALLOWED_MODULES
- assert unexpected == {"switch_core.a_new_sweep"}
diff --git a/core/tests/switch_core/gateway/test_tenant_resolution.py b/core/tests/switch_core/gateway/test_tenant_resolution.py
index 304c663d2..f48ae0d96 100644
--- a/core/tests/switch_core/gateway/test_tenant_resolution.py
+++ b/core/tests/switch_core/gateway/test_tenant_resolution.py
@@ -3,7 +3,11 @@ of the *user*, from their membership row — never a guess, never a request
parameter.
`TestGetSoleTenantId` covers `TenantMemberStore.get_sole_tenant_id` raising
-rather than picking a tenant when a user's memberships aren't exactly one.
+rather than picking a tenant when a user's memberships aren't exactly one. It
+takes a session factory rather than a session: `tenant_members` is a scoped
+table, so no session can read the row that says what it should be scoped to,
+and the store reads through `tenants_of_user` (`db/tenant_lookup.py`)
+instead, on a session of its own with nothing bound.
`TestAGatewayRequestBindsTheCallersTenant` drives a real HTTP request through
`get_current_user` end to end — cookie in, `app.tenant_id` on the database
session out — the same way `test_reference_types_routes.py` builds a
@@ -57,10 +61,11 @@ class TestGetSoleTenantId:
async with session_factory() as session:
user = User(name="orphan", email="[email protected]", role="user")
session.add(user)
- await session.flush()
+ await session.commit()
+ user_id = user.id
- with pytest.raises(TenantMembershipError):
- await store.get_sole_tenant_id(session, user.id)
+ with pytest.raises(TenantMembershipError):
+ await store.get_sole_tenant_id(session_factory, user_id)
async def test_raises_when_the_user_has_more_than_one_membership(
self, session_factory: async_sessionmaker[AsyncSession]
@@ -80,10 +85,11 @@ class TestGetSoleTenantId:
TenantMember(tenant_id=TENANT_B, user_id=user.id, role="member"),
]
)
- await session.flush()
+ await session.commit()
+ user_id = user.id
- with pytest.raises(TenantMembershipError):
- await store.get_sole_tenant_id(session, user.id)
+ with pytest.raises(TenantMembershipError):
+ await store.get_sole_tenant_id(session_factory, user_id)
async def test_returns_the_one_tenant_a_user_belongs_to(
self, session_factory: async_sessionmaker[AsyncSession]
@@ -96,9 +102,12 @@ class TestGetSoleTenantId:
session.add(
TenantMember(tenant_id=TENANT_ZERO_ID, user_id=user.id, role="member")
)
- await session.flush()
+ await session.commit()
+ user_id = user.id
- assert await store.get_sole_tenant_id(session, user.id) == TENANT_ZERO_ID
+ assert (
+ await store.get_sole_tenant_id(session_factory, user_id) == TENANT_ZERO_ID
+ )
def _app(session_factory: async_sessionmaker[AsyncSession]) -> FastAPI:
diff --git a/core/tests/switch_core/test_room_service_reconcile.py b/core/tests/switch_core/test_room_service_reconcile.py
index 3e8b60fe1..a288d52fe 100644
--- a/core/tests/switch_core/test_room_service_reconcile.py
+++ b/core/tests/switch_core/test_room_service_reconcile.py
@@ -3,6 +3,9 @@ from __future__ import annotations
from types import SimpleNamespace
from typing import Any
+import pytest
+
+from switch_core import room_service as room_service_module
from switch_core.room_service import RoomService
@@ -84,6 +87,7 @@ def _admin(tenant_id: str = "tenant-1") -> SimpleNamespace:
def _build_service(
+ monkeypatch: pytest.MonkeyPatch,
*,
rooms: list[Any],
client_ids_by_room: dict[str, list[str]],
@@ -92,6 +96,20 @@ def _build_service(
) -> tuple[RoomService, _FakeRoomStore, _FakeMatrix]:
room_store = _FakeRoomStore(rooms, client_ids_by_room, agent_clients_by_room or {})
matrix = _FakeMatrix()
+
+ # `reconcile_room_clients` now finds its tenants through `all_tenant_ids`,
+ # a real query against `tenants` (`db/tenant_lookup.py`) that the fake
+ # session below cannot answer — it is a stand-in for `RoomStore`, not for
+ # Postgres. Stubbing it to the tenants these fake rooms actually carry
+ # keeps the fan-out (one `get_all` per tenant, each room's own tenant
+ # bound around it) exercised without a real database.
+ tenant_ids = sorted({room.tenant_id for room in rooms})
+
+ async def _fake_all_tenant_ids(session_factory: Any) -> list[str]:
+ return tenant_ids
+
+ monkeypatch.setattr(room_service_module, "all_tenant_ids", _fake_all_tenant_ids)
+
svc = object.__new__(RoomService)
svc._session_factory = lambda: _FakeSessionCM() # type: ignore[assignment]
svc._room_store = room_store # type: ignore[assignment]
@@ -101,13 +119,16 @@ def _build_service(
class TestReconcileRoomClients:
- async def test_backfills_missing_admin_into_existing_room(self) -> None:
+ async def test_backfills_missing_admin_into_existing_room(
+ self, monkeypatch: pytest.MonkeyPatch
+ ) -> None:
# A room created before the admin client existed: it has the other
# system clients but not the admin. Reconcile invites + records it.
room = SimpleNamespace(
id="room-1", tenant_id="tenant-1", matrix_room_id="!mx:switch.local"
)
svc, room_store, matrix = _build_service(
+ monkeypatch,
rooms=[room],
client_ids_by_room={"room-1": ["resource-mgr", "observe"]},
by_type={"admin": [_admin()]},
@@ -118,11 +139,14 @@ class TestReconcileRoomClients:
assert matrix.invited == [("!mx:switch.local", "@switch-admin:switch.local")]
assert room_store.added == [("admin-client", "room-1")]
- async def test_another_tenants_admin_client_is_not_offered(self) -> None:
+ async def test_another_tenants_admin_client_is_not_offered(
+ self, monkeypatch: pytest.MonkeyPatch
+ ) -> None:
room = SimpleNamespace(
id="room-1", tenant_id="tenant-1", matrix_room_id="!mx:switch.local"
)
svc, room_store, matrix = _build_service(
+ monkeypatch,
rooms=[room],
client_ids_by_room={"room-1": []},
by_type={"admin": [_admin(tenant_id="tenant-2")]},
@@ -133,11 +157,14 @@ class TestReconcileRoomClients:
assert matrix.invited == []
assert room_store.added == []
- async def test_skips_room_that_already_has_the_admin(self) -> None:
+ async def test_skips_room_that_already_has_the_admin(
+ self, monkeypatch: pytest.MonkeyPatch
+ ) -> None:
room = SimpleNamespace(
id="room-1", tenant_id="tenant-1", matrix_room_id="!mx:switch.local"
)
svc, room_store, matrix = _build_service(
+ monkeypatch,
rooms=[room],
client_ids_by_room={"room-1": ["admin-client"]},
by_type={"admin": [_admin()]},
@@ -148,7 +175,9 @@ class TestReconcileRoomClients:
assert matrix.invited == []
assert room_store.added == []
- async def test_covers_archived_rooms_too(self) -> None:
+ async def test_covers_archived_rooms_too(
+ self, monkeypatch: pytest.MonkeyPatch
+ ) -> None:
rooms = [
SimpleNamespace(
id="live", tenant_id="tenant-1", matrix_room_id="!live:switch.local"
@@ -160,6 +189,7 @@ class TestReconcileRoomClients:
),
]
svc, room_store, matrix = _build_service(
+ monkeypatch,
rooms=rooms,
client_ids_by_room={"live": [], "archived": []},
by_type={"admin": [_admin()]},
@@ -172,11 +202,14 @@ class TestReconcileRoomClients:
assert ("admin-client", "live") in room_store.added
assert ("admin-client", "archived") in room_store.added
- async def test_no_running_system_clients_is_a_noop(self) -> None:
+ async def test_no_running_system_clients_is_a_noop(
+ self, monkeypatch: pytest.MonkeyPatch
+ ) -> None:
room = SimpleNamespace(
id="room-1", tenant_id="tenant-1", matrix_room_id="!mx:switch.local"
)
svc, room_store, matrix = _build_service(
+ monkeypatch,
rooms=[room],
client_ids_by_room={"room-1": []},
by_type={},
@@ -187,7 +220,9 @@ class TestReconcileRoomClients:
assert matrix.invited == []
assert room_store.added == []
- async def test_reinvites_an_agent_whose_invite_never_landed(self) -> None:
+ async def test_reinvites_an_agent_whose_invite_never_landed(
+ self, monkeypatch: pytest.MonkeyPatch
+ ) -> None:
# `add_agents_to_room` writes the membership row, invites, then records
# `room_clients`. A crash in that window leaves the member with no
# `room_clients` row, which is exactly what is repaired here.
@@ -195,6 +230,7 @@ class TestReconcileRoomClients:
id="room-1", tenant_id="tenant-1", matrix_room_id="!mx:switch.local"
)
svc, room_store, matrix = _build_service(
+ monkeypatch,
rooms=[room],
client_ids_by_room={"room-1": []},
by_type={},
@@ -206,11 +242,14 @@ class TestReconcileRoomClients:
assert matrix.invited == [("!mx:switch.local", "@fixer:switch.local")]
assert room_store.added == [("agent-client", "room-1")]
- async def test_an_agent_already_recorded_is_left_alone(self) -> None:
+ async def test_an_agent_already_recorded_is_left_alone(
+ self, monkeypatch: pytest.MonkeyPatch
+ ) -> None:
room = SimpleNamespace(
id="room-1", tenant_id="tenant-1", matrix_room_id="!mx:switch.local"
)
svc, room_store, matrix = _build_service(
+ monkeypatch,
rooms=[room],
client_ids_by_room={"room-1": ["agent-client"]},
by_type={},
diff --git a/core/tests/switch_core/test_room_service_tenant_bindings.py b/core/tests/switch_core/test_room_service_tenant_bindings.py
index 3714c34c8..4675d5733 100644
--- a/core/tests/switch_core/test_room_service_tenant_bindings.py
+++ b/core/tests/switch_core/test_room_service_tenant_bindings.py
@@ -2,9 +2,12 @@
`room_service.py` is the largest single piece of the tenant rework and it had
no test that could tell the rework from its absence. The proof was mechanical:
-shadow `tenant_scope`, `tenant_session` and `unscoped_session` with no-ops
-inside that one module and the whole suite still passed, 2833 to 2833. Not one
-assertion anywhere depended on a single binding this file makes.
+shadow the three names it imports for binding and enumeration —
+`tenant_scope`, `tenant_session` and `all_tenant_ids` — with no-ops inside
+that one module, and the wider suite still passed. This file is what changed
+that: the same shadowing fails five of its own eight tests, because those
+five each read a binding back off the database rather than off the service's
+say-so.
This module is built so that experiment fails. Two things make that possible
where the rest of the suite could not:
@@ -13,11 +16,13 @@ where the rest of the suite could not:
``current_setting('app.tenant_id')`` on the very session the service handed the
store — the value ``db/tenant_session.py``'s ``after_begin`` hook wrote, which
is what the policies compare and what every scoped write is filed under. An
-assertion about rows cannot see a missing binding while Switch runs as the
-tables' owner, which is what Phase 1 deploys: Postgres exempts an owner from
-its own policies, so a session that forgot its tenant reads and writes exactly
-as much as one that remembered (the restricted runtime role is CHOO-2685).
-The setting is the one thing that still differs.
+assertion about rows cannot see a missing binding on this fixture's
+connection, which is the tables' owner: Postgres exempts an owner from its own
+policies, so a session that forgot its tenant reads and writes exactly as much
+as one that remembered. The setting is the one thing that still differs. (The
+deployment now runs as a restricted role the policies do apply to, and so does
+the integration suite — but the unit suite still connects as the owner, so
+this file has to be able to tell without them.)
**It uses two tenants.** Half of what these bindings are for only has an
observable effect once a second tenant owns something, which no other test in
@@ -55,6 +60,7 @@ from switch_core.db.models import (
User,
room_agents,
)
+from switch_core.db.session_scope import tenant_session
from switch_core.db.stores.client_store import ClientStore
from switch_core.db.stores.room_store import RoomStore
from switch_core.db.stores.tenant_store import TenantStore
@@ -421,20 +427,33 @@ class TestReconcileRoomClients:
assert sorted(store.tenants_for("add_client")) == sorted(
[TENANT_ZERO_ID, TENANT_B]
)
- assert sorted(store.tenants_for("get_client_ids")) == sorted(
- [TENANT_ZERO_ID, TENANT_B]
- )
+ # `all_tenant_ids` names two tenants, so `get_all` runs once per
+ # tenant — and this fixture connects as the tables' owner, the same
+ # connection Phase 1 deploys as (see the module docstring), which
+ # exempts it from the policy `get_all` otherwise relies on to narrow
+ # its result. So each of those two calls sees every room, and
+ # `_reconcile_one_room` — bound to each room's own `tenant_id`
+ # regardless of which pass found it — runs twice per room rather than
+ # once. Harmless (the second pass finds nothing missing and returns),
+ # but it means every tenant appears at least once here rather than
+ # exactly once; the real one-pass-per-room count arrives with the
+ # restricted runtime role.
+ assert set(store.tenants_for("get_client_ids")) == {TENANT_ZERO_ID, TENANT_B}
async def test_the_room_enumeration_refuses_the_callers_tenant(
self, session_factory: async_sessionmaker[AsyncSession]
) -> None:
- """ "Which rooms exist" must span tenants whether or not one is bound.
-
- That is `unscoped_session`'s whole contract (`db/session_scope.py`): a
- helper that named the intent while inheriting the ambient tenant would
- have the hook stamp it on the transaction and the cross-tenant read
- would silently be a single-tenant one — here, a boot that reconciles
- tenant zero's rooms and quietly skips everyone else's.
+ """ "Which tenants exist" must answer for all of them, whether or not
+ one is bound, and never narrow to whatever the caller had bound.
+
+ That is `all_tenant_ids`'s whole contract (`db/tenant_lookup.py`): it
+ opens its own session with nothing bound (`tenant_lookup._call` enters
+ `no_tenant()` first), so the caller's ambient tenant — tenant zero
+ here — cannot narrow the list of tenants reconciled. A version that
+ inherited it would have the hook stamp tenant zero on that lookup's
+ transaction, `all_tenant_ids` would answer with tenant zero alone, and
+ this boot would reconcile tenant zero's rooms and quietly skip tenant
+ B's — the one tenant this test gives a room to reconcile.
"""
async with session_factory() as session:
await _seed_tenant_b(session)
@@ -471,7 +490,12 @@ class TestReconcileRoomClients:
finally:
await registry.stop_all()
- assert store.tenants_for("get_all") == [None], (
+ # One `get_all` per tenant `all_tenant_ids` named — tenant zero *and*
+ # tenant B, in the order `all_tenant_ids` returns them (oldest first),
+ # not tenant zero alone. Had the lookup inherited the ambient tenant
+ # bound around the whole call, this would read `[TENANT_ZERO_ID]` and
+ # tenant B's room would never be visited.
+ assert store.tenants_for("get_all") == [TENANT_ZERO_ID, TENANT_B], (
"the room enumeration inherited the caller's tenant; under the "
"policies it would have returned none of tenant B's rooms"
)
@@ -594,7 +618,14 @@ class TestBindingsOnTheRoomsOwnTenant:
default is `require_tenant_id`, which raises rather than guessing. So
this write does not merely land in the wrong tenant without the
binding; it does not land at all."""
- async with session_factory() as session:
+ # The whole arrangement on a session opened *inside* tenant B's
+ # binding, rather than a bare session with a `tenant_scope` around the
+ # part that needs one. `_seed_agent` writes an `ApiKey`, which is
+ # scoped; binding after the session's first statement rebinds a
+ # contextvar and nothing the database can see, so the write would be
+ # filed under tenant B while the transaction still said tenant zero —
+ # which the policies refuse outright, and which this test used to do.
+ async with tenant_session(session_factory, TENANT_B) as session:
await _seed_tenant_b(session)
await _seed_room(
session,
@@ -602,13 +633,12 @@ class TestBindingsOnTheRoomsOwnTenant:
room_id="room-b",
matrix_room_id="!b:switch.local",
)
- with tenant_scope(TENANT_B):
- await _seed_agent(
- session,
- tenant_id=TENANT_B,
- agent_id="agent-b",
- client_id="client-b",
- )
+ await _seed_agent(
+ session,
+ tenant_id=TENANT_B,
+ agent_id="agent-b",
+ client_id="client-b",
+ )
await session.commit()
store = _TenantRecordingRoomStore()
diff --git a/core/tests/switch_core/test_seed_admin_membership.py b/core/tests/switch_core/test_seed_admin_membership.py
index f62c9c378..a7371fed6 100644
--- a/core/tests/switch_core/test_seed_admin_membership.py
+++ b/core/tests/switch_core/test_seed_admin_membership.py
@@ -179,9 +179,8 @@ class TestWhatBeingStrandedCosts:
await session.commit()
await _strand_the_admin(session_factory, user_id)
- async with session_factory() as session:
- with pytest.raises(TenantMembershipError):
- await TenantMemberStore().get_sole_tenant_id(session, user_id)
+ with pytest.raises(TenantMembershipError):
+ await TenantMemberStore().get_sole_tenant_id(session_factory, user_id)
async def test_creating_a_user_still_writes_one(
self, session_factory: async_sessionmaker[AsyncSession]
diff --git a/core/tests/switch_core/transport/test_postgres_transport.py b/core/tests/switch_core/transport/test_postgres_transport.py
index 2a16d8e66..6c5e41510 100644
--- a/core/tests/switch_core/transport/test_postgres_transport.py
+++ b/core/tests/switch_core/transport/test_postgres_transport.py
@@ -20,7 +20,7 @@ import pytest
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
-from switch_core.db.models import Client, ClientRoom, Message, Room, Tenant
+from switch_core.db.models import Client, ClientRoom, Room, Tenant
from switch_core.db.session_scope import tenant_session
from switch_core.db.stores.media_store import MediaStore
from switch_core.db.stores.message_store import MessageStore
@@ -134,6 +134,31 @@ class _Received:
self.events.append(event)
+async def _seed_client(
+ session_factory: async_sessionmaker[AsyncSession], client_id: str
+) -> None:
+ """Give a transport's client a row, so it has a tenant to act in.
+
+ A `PostgresTransport` resolves its own tenant from `clients` (see
+ `PostgresTransport._tenant`), because its task binds nothing and
+ everything it writes is scoped. A transport built for a client id with no
+ row behind it is a transport for a client that does not exist, and it says
+ so rather than falling back to whatever tenant happens to be ambient —
+ which for `upload_media` would mean putting a blob in a tenant nobody
+ chose. Tests that need only *a* client take one from here.
+ """
+ async with session_factory() as session:
+ session.add(
+ Client(
+ id=client_id,
+ matrix_user_id=f"@{client_id}-{uuid.uuid4().hex[:8]}:test",
+ display_name="a client",
+ type="agent",
+ )
+ )
+ await session.commit()
+
+
def _transport(
session_factory: async_sessionmaker[AsyncSession],
*,
@@ -355,9 +380,15 @@ class TestRooms:
class TestMedia:
+ """`media_blobs` is scoped, so a blob is written into the uploading
+ client's tenant — which means every test here needs a client row for the
+ transport to read that tenant off. Without one the transport refuses,
+ which is the right answer and not the one these tests are about."""
+
async def test_bytes_go_in_and_come_back(
self, session_factory: async_sessionmaker[AsyncSession]
) -> None:
+ await _seed_client(session_factory, "c")
transport = _transport(session_factory, client_id="c", user_id="@a:test")
uploaded = await transport.upload_media(b"file contents", "text/plain", "a.txt")
@@ -371,6 +402,7 @@ class TestMedia:
self, session_factory: async_sessionmaker[AsyncSession]
) -> None:
"""Deduplicating would make one sender's delete another's data loss."""
+ await _seed_client(session_factory, "c")
transport = _transport(session_factory, client_id="c", user_id="@a:test")
first = await transport.upload_media(b"same", "text/plain", "a.txt")
@@ -382,12 +414,31 @@ class TestMedia:
self, session_factory: async_sessionmaker[AsyncSession]
) -> None:
"""An empty file is a worse answer than an error: the reader cannot
- tell it apart from a file that was genuinely empty."""
+ tell it apart from a file that was genuinely empty.
+
+ The client row matters here more than it looks: without it the
+ transport raises `TransportError` for having no tenant, and this test
+ would pass for entirely the wrong reason.
+ """
+ await _seed_client(session_factory, "c")
transport = _transport(session_factory, client_id="c", user_id="@a:test")
- with pytest.raises(TransportError):
+ with pytest.raises(TransportError, match="No media stored under"):
await transport.download_media("switch-media://nothing")
+ async def test_a_transport_whose_client_has_no_row_refuses_to_upload(
+ self, session_factory: async_sessionmaker[AsyncSession]
+ ) -> None:
+ """The fail-loud half of the same rule. A blob has to name a tenant,
+ and the only honest source for one is the uploading client's own row;
+ with no row there is no tenant, and the alternatives — whatever is
+ ambient, or tenant zero — are both a write into a real customer's
+ data that nobody chose."""
+ transport = _transport(session_factory, client_id="ghost", user_id="@a:test")
+
+ with pytest.raises(TransportError, match="no tenant to act in"):
+ await transport.upload_media(b"bytes", "text/plain", "a.txt")
+
class TestWhatIsNotBuiltYet:
async def test_history_refuses_rather_than_answering_from_the_wrong_place(
@@ -937,17 +988,28 @@ class TestDeliveryBindsTheRoomsTenant:
assert current_tenant_id() is None
-class TestATransportOwnsNoTenantOfItsOwn:
- """A transport is not a single-tenant actor, and CHOO-2623 stops it
- behaving like one.
+class TestATransportActsInItsClientsTenant:
+ """A transport's tenant is its client's, derived and never inherited.
Its task binds nothing: `ClientLifecycleService` unbinds before running a
- client, so anything the transport does under an inherited tenant would be
- doing it under the tenant of whoever happened to create the task — boot,
- a gateway request, or the inbound bridge message that minted a puppet
- mid-conversation. Two consequences are pinned here: the lookup that says
- which rooms this client is in runs with nothing bound, and each delivery
- binds the tenant of the room it is for.
+ client, so anything the transport did under an *inherited* tenant would be
+ doing it under the tenant of whoever happened to create the task — boot, a
+ gateway request, or the inbound bridge message that minted a puppet
+ mid-conversation. That is still the failure this class exists to prevent.
+ What changed with the restricted runtime role is where the right answer
+ comes from.
+
+ It used to come from nowhere: `joined_rooms` and the room resolution ran
+ with nothing bound, on the reasoning that a tenant over those reads could
+ only be a guess and a guess returns a silent subset. Under a role the
+ policies apply to, nothing bound returns a silent *empty* — so "unscoped"
+ stopped being the safe answer and became the worst one. The tenant now
+ comes from the client's own row, through the exemption
+ (`db/tenant_lookup.py`), which is exact rather than a guess: `client_rooms`
+ and `messages` both carry composite foreign keys through `tenant_id`, so a
+ client cannot be a member of, or a sender in, a room outside its own
+ tenant. `test_the_schema_forbids_a_client_row_in_another_tenants_room`
+ below is what that rests on.
"""
@pytest.fixture(autouse=True)
@@ -957,20 +1019,24 @@ class TestATransportOwnsNoTenantOfItsOwn:
for task in self._tasks:
task.cancel()
- async def test_the_room_list_is_read_with_no_tenant_bound(
+ async def test_the_room_list_is_read_under_the_clients_own_tenant(
self,
session_factory: async_sessionmaker[AsyncSession],
monkeypatch: pytest.MonkeyPatch,
) -> None:
- """`joined_rooms` decides, once, what this client will ever hear. A
- tenant bound over that read narrows it to a subset with no error to
- say so — the client is simply deaf everywhere else, forever. So it
- must run unscoped even when reached from a bound context.
-
- `test_a_tenant_scoped_read_would_have_returned_no_rooms_at_all` below
- proves the "narrows it to a subset" half directly, through the row-
- level-security policies rather than by inferring it from the tenant
- this test's own spy observed.
+ """`joined_rooms` decides, once, what this client will ever hear, so
+ the tenant over that read has to be the right one — a wrong one
+ returns fewer rooms with no error to say so, and the client is deaf in
+ the rest forever.
+
+ What this pins is that it is the *client's* tenant and not the
+ caller's. The call is made from inside a `tenant_scope` for an
+ unrelated tenant, standing in for a task that inherited one it has no
+ business acting under, and the spy still sees the client's own.
+
+ `test_a_read_under_the_wrong_tenant_would_have_returned_no_rooms_at_all`
+ below is the other half: what the wrong answer costs, through the
+ policies rather than inferred from the tenant this spy observed.
"""
tenant_id = f"tenant-{uuid.uuid4().hex[:8]}"
async with session_factory() as session:
@@ -1015,21 +1081,21 @@ class TestATransportOwnsNoTenantOfItsOwn:
rooms = await transport.joined_rooms()
assert rooms == [transport_room_id]
- assert seen == [None], (
- "joined_rooms inherited a tenant; under row-level security it "
- "would have returned a subset of this client's rooms and gone "
- "silent in the rest"
+ assert seen == [tenant_id], (
+ "joined_rooms read under the caller's tenant rather than the "
+ "client's; under row-level security it would have returned none "
+ "of this client's rooms and gone silent in all of them"
)
- async def test_a_tenant_scoped_read_would_have_returned_no_rooms_at_all(
+ async def test_a_read_under_the_wrong_tenant_would_have_returned_no_rooms_at_all(
self, rls_harness: RLSHarness
) -> None:
"""The same client and its one room, read through
`RoomStore.get_for_client` — the exact method `joined_rooms` calls —
- but scoped to a tenant the client does not belong to, instead of left
- unscoped. It comes back empty: not a subset, since this client has
- only the one room, but the same silent-and-wrong shape `joined_rooms`
- avoids by never binding a tenant over this read at all.
+ but scoped to a tenant the client does not belong to. It comes back
+ empty, through the policies, as the restricted role: that is what an
+ inherited tenant costs, and why the transport derives its own from the
+ client row rather than taking whatever the caller had.
"""
tenant_id = f"tenant-{uuid.uuid4().hex[:8]}"
other_tenant = f"tenant-{uuid.uuid4().hex[:8]}"
@@ -1068,64 +1134,53 @@ class TestATransportOwnsNoTenantOfItsOwn:
)
@pytest.mark.no_ambient_tenant
- async def test_each_room_is_delivered_under_its_own_tenant(
+ async def test_each_room_is_delivered_under_the_clients_tenant(
self,
session_factory: async_sessionmaker[AsyncSession],
monkeypatch: pytest.MonkeyPatch,
) -> None:
- """One transport, two rooms, two tenants — the shape a reused puppet
- takes. The client whose transport this is belongs to one of the two
- tenants, and that must not decide how the other room is read.
-
- Membership is only recorded for the room in the transport's own
- tenant, because `client_rooms` carries composite foreign keys to both
- `clients` and `rooms` and so cannot join a client to a room in another
- tenant at all — see
- `test_the_schema_forbids_a_client_row_in_another_tenants_room`. The
- watch and the delivery do not go through membership, which is what
- makes the two-tenant case reachable here.
+ """Two rooms, one transport, and every read of either under the
+ client's tenant — with nothing bound around the loop that drives them.
+
+ This test used to arrange the two rooms in *different* tenants, on the
+ reading that a reused puppet could speak in both and that the
+ transport's own tenant must therefore never decide how a room is read.
+ That shape is not reachable: `client_rooms` and `messages` both key to
+ `rooms` and to `clients` through `tenant_id`, so a client can be
+ neither a member of nor a sender in a room outside its own tenant, and
+ the old arrangement only stood up because it wrote the second room's
+ message by hand and called the private `_watch` directly. What is left
+ is the part that is real — the delivery loop binds a tenant per room
+ rather than running on whatever the task inherited, which is nothing.
"""
- tenant_a = f"tenant-{uuid.uuid4().hex[:8]}"
- tenant_b = f"tenant-{uuid.uuid4().hex[:8]}"
+ tenant_id = f"tenant-{uuid.uuid4().hex[:8]}"
suffix = uuid.uuid4().hex[:8]
async with session_factory() as session:
- for tenant_id in (tenant_a, tenant_b):
- session.add(Tenant(id=tenant_id, slug=tenant_id, name=tenant_id))
+ session.add(Tenant(id=tenant_id, slug=tenant_id, name=tenant_id))
await session.flush()
puppet = Client(
- tenant_id=tenant_a,
+ tenant_id=tenant_id,
matrix_user_id=f"@puppet-{suffix}:test",
display_name="puppet",
type="user",
)
- speaker_b = Client(
- tenant_id=tenant_b,
- matrix_user_id=f"@speaker-{suffix}:test",
- display_name="someone",
- type="agent",
- )
- session.add_all([puppet, speaker_b])
- room_a = Room(
- tenant_id=tenant_a,
+ session.add(puppet)
+ first = Room(
+ tenant_id=tenant_id,
matrix_room_id=f"!a-{suffix}:test",
name="room a",
description="",
)
- room_b = Room(
- tenant_id=tenant_b,
+ second = Room(
+ tenant_id=tenant_id,
matrix_room_id=f"!b-{suffix}:test",
name="room b",
description="",
)
- session.add_all([room_a, room_b])
+ session.add_all([first, second])
await session.commit()
- ids = {
- room_a.id: tenant_a,
- room_b.id: tenant_b,
- }
- mxid_a, mxid_b = room_a.matrix_room_id, room_b.matrix_room_id
- room_b_id = room_b.id
- speaker_b_id, speaker_b_mxid = speaker_b.id, speaker_b.matrix_user_id
+ room_ids = {first.id: tenant_id, second.id: tenant_id}
+ mxid_a, mxid_b = first.matrix_room_id, second.matrix_room_id
client_id, user_id = puppet.id, puppet.matrix_user_id
seen: dict[str, list[str | None]] = {}
@@ -1144,46 +1199,67 @@ class TestATransportOwnsNoTenantOfItsOwn:
)
transport.register_handlers(received.handlers())
await transport.join_room(mxid_a)
+ await transport.join_room(mxid_b)
self._tasks.append(asyncio.create_task(transport.receive_forever()))
await _watched_room(transport)
- await transport._watch(mxid_b)
await transport.send_message(mxid_a, "in a", sender_name="puppet")
- # Written as a client of tenant B: a message in room B carries room
- # B's tenant, and the composite key to `clients` will not let a
- # tenant-A client be its sender.
- async with session_factory() as session:
- await MessageStore().create(
- session,
- Message(
- tenant_id=tenant_b,
- room_id=room_b_id,
- transport_event_id=f"sw_{uuid.uuid4().hex}",
- sender_id=speaker_b_mxid,
- sender_client_id=speaker_b_id,
- sender_name="someone",
- event_type="m.room.message",
- msgtype="m.text",
- body="in b",
- formatted_body=None,
- thread_root_event_id=None,
- content={"body": "in b", "msgtype": "m.text"},
- ),
- [],
- )
- await session.commit()
+ await transport.send_message(mxid_b, "in b", sender_name="puppet")
- for room_id in ids:
+ for room_id in room_ids:
await listener.announce(room_id)
- assert {getattr(e, "body", None) for e in received.events} == {"in a", "in b"}
- assert set(seen) == set(ids), "a room was never read"
+ assert {"in a", "in b"} <= {getattr(e, "body", None) for e in received.events}
+ assert set(seen) == set(room_ids), "a room was never read"
for room_id, tenants in seen.items():
- assert tenants and all(t == ids[room_id] for t in tenants), (
- f"room {room_id} was read under {tenants}, not {ids[room_id]}"
+ assert tenants and all(t == room_ids[room_id] for t in tenants), (
+ f"room {room_id} was read under {tenants}, not {room_ids[room_id]}"
)
assert current_tenant_id() is None
+ async def test_a_room_in_another_tenant_is_not_a_switch_room_here(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """What the transport does when handed a room id it has no business
+ with, asked of the restricted role so the policies are what answer.
+
+ It is the same "not a Switch room" a genuinely unknown id gets, and
+ deliberately so: telling the caller that the room exists but belongs
+ to somebody else is an existence oracle, and there is no legitimate
+ caller here to tell — a client is only ever handed room ids from its
+ own rooms.
+ """
+ tenant_id = f"tenant-{uuid.uuid4().hex[:8]}"
+ other_tenant = f"tenant-{uuid.uuid4().hex[:8]}"
+ suffix = uuid.uuid4().hex[:8]
+ async with rls_harness.owner() as session:
+ for t in (tenant_id, other_tenant):
+ session.add(Tenant(id=t, slug=t, name=t))
+ await session.flush()
+ client = Client(
+ tenant_id=tenant_id,
+ matrix_user_id=f"@sys-{suffix}:test",
+ display_name="admin",
+ type="admin",
+ )
+ session.add(client)
+ elsewhere = Room(
+ tenant_id=other_tenant,
+ matrix_room_id=f"!elsewhere-{suffix}:test",
+ name="somebody else's room",
+ description="",
+ )
+ session.add(elsewhere)
+ await session.commit()
+ client_id, user_id = client.id, client.matrix_user_id
+ elsewhere_mxid = elsewhere.matrix_room_id
+
+ transport = _transport(
+ rls_harness.restricted, client_id=client_id, user_id=user_id
+ )
+ with pytest.raises(TransportError, match="is not a Switch room"):
+ await transport.send_message(elsewhere_mxid, "hello", sender_name="admin")
+
async def test_the_schema_forbids_a_client_row_in_another_tenants_room(
session_factory: async_sessionmaker[AsyncSession],
diff --git a/deploy/local/docker-compose.yml b/deploy/local/docker-compose.yml
index fc4024226..215a88ae7 100644
--- a/deploy/local/docker-compose.yml
+++ b/deploy/local/docker-compose.yml
@@ -4,13 +4,18 @@ services:
ports:
- "${SWITCH_BIND_ADDR:-127.0.0.1}:5432:5432"
environment:
- POSTGRES_USER: ${DB_USER}
- POSTGRES_PASSWORD: ${DB_PASSWORD}
+ # The container's bootstrapped superuser is the SCHEMA OWNER
+ # (DB_OWNER_USER), not the runtime role Switch serves requests as
+ # (DB_USER). Row-level security is inert against a table's owner, so
+ # the role every request authenticates as must be a separate, deliberately
+ # unprivileged login — created below by init-db, once the owner is up.
+ POSTGRES_USER: ${DB_OWNER_USER}
+ POSTGRES_PASSWORD: ${DB_OWNER_PASSWORD}
POSTGRES_DB: ${DB_NAME}
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
- test: ["CMD-SHELL", "pg_isready -U ${DB_USER}"]
+ test: ["CMD-SHELL", "pg_isready -U ${DB_OWNER_USER}"]
interval: 2s
timeout: 3s
retries: 10
@@ -18,12 +23,28 @@ services:
init-db:
image: postgres:16-alpine
environment:
- PGPASSWORD: ${DB_PASSWORD}
+ PGPASSWORD: ${DB_OWNER_PASSWORD}
+ # Runs as the owner: creating a database and a role both need privileges
+ # the runtime role is deliberately never given. Note the mixed quoting in
+ # the CREATE ROLE / ALTER ROLE statements is not optional — DB_USER is an
+ # identifier (double-quoted, and safe only because .env.example pins it to
+ # a fixed, non-attacker-controlled name) and the password is a string
+ # literal (single-quoted); swapping them would either fail to parse or
+ # quietly create a role named after the password.
+ #
+ # No GRANT here on purpose: on a fresh volume there is nothing yet to
+ # grant. The `switch` database has no tables until switch-core runs
+ # Alembic against the owner connection, so the runtime role's CRUD grants
+ # are issued by switch-core itself at boot, immediately after migrating —
+ # see db/runtime_role.py. This step only has to make the role exist.
entrypoint: >
sh -c "
- until pg_isready -h postgres -U ${DB_USER}; do sleep 1; done
- psql -h postgres -U ${DB_USER} -tc \"SELECT 1 FROM pg_database WHERE datname='mattermost'\" | grep -q 1 ||
- psql -h postgres -U ${DB_USER} -c 'CREATE DATABASE mattermost;'
+ until pg_isready -h postgres -U ${DB_OWNER_USER}; do sleep 1; done
+ psql -h postgres -U ${DB_OWNER_USER} -tc \"SELECT 1 FROM pg_database WHERE datname='mattermost'\" | grep -q 1 ||
+ psql -h postgres -U ${DB_OWNER_USER} -c 'CREATE DATABASE mattermost;'
+ psql -h postgres -U ${DB_OWNER_USER} -tc \"SELECT 1 FROM pg_roles WHERE rolname='${DB_USER}'\" | grep -q 1 ||
+ psql -h postgres -U ${DB_OWNER_USER} -c 'CREATE ROLE \"${DB_USER}\" LOGIN PASSWORD '\\''${DB_PASSWORD}'\\'' NOSUPERUSER NOCREATEDB NOCREATEROLE NOBYPASSRLS;'
+ psql -h postgres -U ${DB_OWNER_USER} -c 'ALTER ROLE \"${DB_USER}\" PASSWORD '\\''${DB_PASSWORD}'\\'';'
"
depends_on:
postgres:
@@ -38,7 +59,13 @@ services:
- mmdata:/mattermost/data
environment:
MM_SQLSETTINGS_DRIVERNAME: postgres
- MM_SQLSETTINGS_DATASOURCE: postgres://${DB_USER}:${DB_PASSWORD}@postgres:5432/mattermost?sslmode=disable
+ # The owner's credentials, not the runtime role's: init-db creates the
+ # `mattermost` database as the owner, and the runtime role (DB_USER)
+ # neither owns it nor was granted anything on it — only `switch`'s
+ # tables get the runtime role's grants, issued by switch-core itself at
+ # boot. Mattermost's own schema has no row-level security in it, so
+ # there is no isolation reason to run it as the restricted role too.
+ MM_SQLSETTINGS_DATASOURCE: postgres://${DB_OWNER_USER}:${DB_OWNER_PASSWORD}@postgres:5432/mattermost?sslmode=disable
MM_SERVICESETTINGS_SITEURL: http://localhost:8065
MM_SERVICESETTINGS_ENABLELOCALMODE: "true"
MM_SERVICESETTINGS_ENABLEBOTACCOUNTCREATION: "true"
diff --git a/deploy/local/standalone-docker-compose.yml b/deploy/local/standalone-docker-compose.yml
index 84ee22d13..f2a9f1eaa 100644
--- a/deploy/local/standalone-docker-compose.yml
+++ b/deploy/local/standalone-docker-compose.yml
@@ -51,8 +51,13 @@ services:
ports:
- "${SWITCH_BIND_ADDR:-127.0.0.1}:${POSTGRES_HOST_PORT:-5432}:5432"
environment:
- POSTGRES_USER: ${DB_USER}
- POSTGRES_PASSWORD: ${DB_PASSWORD}
+ # The container's bootstrapped superuser is the SCHEMA OWNER
+ # (DB_OWNER_USER), not the runtime role Switch serves requests as
+ # (DB_USER). Row-level security is inert against a table's owner, so
+ # the role every request authenticates as must be a separate, deliberately
+ # unprivileged login — created below by init-db, once the owner is up.
+ POSTGRES_USER: ${DB_OWNER_USER}
+ POSTGRES_PASSWORD: ${DB_OWNER_PASSWORD}
POSTGRES_DB: ${DB_NAME}
volumes:
# Kept deliberately after Tuwunel was removed. It still holds every media
@@ -61,7 +66,7 @@ services:
# release than the one that stops reading it.
- pgdata:/var/lib/postgresql/data
healthcheck:
- test: ["CMD-SHELL", "pg_isready -U ${DB_USER}"]
+ test: ["CMD-SHELL", "pg_isready -U ${DB_OWNER_USER}"]
interval: 2s
timeout: 3s
retries: 10
@@ -70,24 +75,50 @@ services:
image: postgres:16-alpine
profiles: ["collab"]
environment:
- PGPASSWORD: ${DB_PASSWORD}
+ PGPASSWORD: ${DB_OWNER_PASSWORD}
+ # Runs as the owner: creating a database and a role both need privileges
+ # the runtime role is deliberately never given. Note the mixed quoting in
+ # the CREATE ROLE / ALTER ROLE statements is not optional — DB_USER is an
+ # identifier (double-quoted, and safe only because .env.example pins it to
+ # a fixed, non-attacker-controlled name) and the password is a string
+ # literal (single-quoted); swapping them would either fail to parse or
+ # quietly create a role named after the password.
+ #
+ # No GRANT here on purpose: on a fresh volume there is nothing yet to
+ # grant. The `switch` database has no tables until switch-core runs
+ # Alembic against the owner connection, so the runtime role's CRUD grants
+ # are issued by switch-core itself at boot, immediately after migrating.
+ # This step only has to make the role exist.
+ #
+ # The trailing ALTER ROLE is what keeps a regenerated .env working on a
+ # reused pgdata volume: the CREATE ROLE guard above skips a role that
+ # already exists, so without it a new DB_PASSWORD would never reach
+ # Postgres. DB_OWNER_PASSWORD gets no such reapplication — it is set once,
+ # as POSTGRES_PASSWORD, by the container's initial bootstrap of its own
+ # superuser, and nothing here ever reissues it. So a regenerated
+ # DB_PASSWORD takes effect on the next `up` with no rebuild, while a
+ # regenerated DB_OWNER_PASSWORD needs the stack (and its volume)
+ # recreated to match.
entrypoint: >
sh -c "
i=0
- until psql -h postgres -U ${DB_USER} -d postgres -c 'SELECT 1' >/dev/null 2>&1; do
+ until psql -h postgres -U ${DB_OWNER_USER} -d postgres -c 'SELECT 1' >/dev/null 2>&1; do
i=$$((i+1))
if [ $$i -ge 30 ]; then
- echo 'init-db: could not connect to postgres as ${DB_USER} after 30 tries.' >&2
+ echo 'init-db: could not connect to postgres as ${DB_OWNER_USER} after 30 tries.' >&2
echo 'Last attempt:' >&2
- psql -h postgres -U ${DB_USER} -d postgres -c 'SELECT 1' >&2
+ psql -h postgres -U ${DB_OWNER_USER} -d postgres -c 'SELECT 1' >&2
echo 'A stale pgdata volume keeps the password it was first created with,' >&2
- echo 'so a regenerated DB_PASSWORD will fail here. Recreate the stack to reset it.' >&2
+ echo 'so a regenerated DB_OWNER_PASSWORD will fail here. Recreate the stack to reset it.' >&2
exit 1
fi
sleep 1
done
- psql -h postgres -U ${DB_USER} -d postgres -tc \"SELECT 1 FROM pg_database WHERE datname='mattermost'\" | grep -q 1 ||
- psql -h postgres -U ${DB_USER} -d postgres -c 'CREATE DATABASE mattermost;'
+ psql -h postgres -U ${DB_OWNER_USER} -d postgres -tc \"SELECT 1 FROM pg_database WHERE datname='mattermost'\" | grep -q 1 ||
+ psql -h postgres -U ${DB_OWNER_USER} -d postgres -c 'CREATE DATABASE mattermost;'
+ psql -h postgres -U ${DB_OWNER_USER} -d postgres -tc \"SELECT 1 FROM pg_roles WHERE rolname='${DB_USER}'\" | grep -q 1 ||
+ psql -h postgres -U ${DB_OWNER_USER} -d postgres -c 'CREATE ROLE \"${DB_USER}\" LOGIN PASSWORD '\\''${DB_PASSWORD}'\\'' NOSUPERUSER NOCREATEDB NOCREATEROLE NOBYPASSRLS;'
+ psql -h postgres -U ${DB_OWNER_USER} -d postgres -c 'ALTER ROLE \"${DB_USER}\" PASSWORD '\\''${DB_PASSWORD}'\\'';'
"
depends_on:
postgres:
@@ -112,7 +143,13 @@ services:
- mmdata:/mattermost/data
environment:
MM_SQLSETTINGS_DRIVERNAME: postgres
- MM_SQLSETTINGS_DATASOURCE: postgres://${DB_USER}:${DB_PASSWORD}@postgres:5432/mattermost?sslmode=disable
+ # The owner's credentials, not the runtime role's: init-db creates the
+ # `mattermost` database as the owner, and the runtime role (DB_USER)
+ # neither owns it nor was granted anything on it — only `switch`'s
+ # tables get the runtime role's grants, issued by switch-core itself at
+ # boot. Mattermost's own schema has no row-level security in it, so
+ # there is no isolation reason to run it as the restricted role too.
+ MM_SQLSETTINGS_DATASOURCE: postgres://${DB_OWNER_USER}:${DB_OWNER_PASSWORD}@postgres:5432/mattermost?sslmode=disable
MM_SERVICESETTINGS_SITEURL: http://localhost:${MATTERMOST_HOST_PORT:-8065}
MM_SERVICESETTINGS_ENABLELOCALMODE: "true"
# The first-run onboarding checklist opens a modal over a dimmed page,
@@ -145,6 +182,8 @@ services:
DB_PORT: "5432"
DB_USER: ${DB_USER}
DB_PASSWORD: ${DB_PASSWORD}
+ DB_OWNER_USER: ${DB_OWNER_USER}
+ DB_OWNER_PASSWORD: ${DB_OWNER_PASSWORD}
DB_NAME: ${DB_NAME}
MATRIX_SERVER_NAME: localhost
AGENT_REGISTRATION_TOKEN: ${AGENT_REGISTRATION_TOKEN}
diff --git a/deploy/remote/helm/switch/templates/_helpers.tpl b/deploy/remote/helm/switch/templates/_helpers.tpl
index 4d1145f16..ba433ccde 100644
--- a/deploy/remote/helm/switch/templates/_helpers.tpl
+++ b/deploy/remote/helm/switch/templates/_helpers.tpl
@@ -48,6 +48,9 @@ rendered from this one source and cannot drift from the release's own.
{{- if not .Values.postgresql.existingSecret }}
POSTGRES_PASSWORD: {{ required "secrets.postgresPassword is required (unless postgresql.existingSecret is set)" .Values.secrets.postgresPassword | b64enc | quote }}
{{- end }}
+{{- if and .Values.postgresql.owner.username (not .Values.postgresql.owner.existingSecret) }}
+DB_OWNER_PASSWORD: {{ required "secrets.dbOwnerPassword is required when postgresql.owner.username is set (unless postgresql.owner.existingSecret is set)" .Values.secrets.dbOwnerPassword | b64enc | quote }}
+{{- end }}
AGENT_REGISTRATION_TOKEN: {{ required "secrets.agentRegistrationToken is required" .Values.secrets.agentRegistrationToken | b64enc | quote }}
JWT_SECRET_KEY: {{ required "secrets.jwtSecretKey is required" .Values.secrets.jwtSecretKey | b64enc | quote }}
GATEWAY_ADMIN_EMAIL: {{ required "secrets.gatewayAdminEmail is required" .Values.secrets.gatewayAdminEmail | b64enc | quote }}
@@ -121,6 +124,23 @@ external secret (e.g. one synced by external-secrets / sealed-secrets).
{{- .Values.postgresql.existingSecretKey | default "POSTGRES_PASSWORD" -}}
{{- end }}
+{{/*
+Name/key of the Secret holding the schema owner's password, mirroring
+switch.postgresSecretName/Key above for the runtime role. Only referenced when
+postgresql.owner.username is set.
+*/}}
+{{- define "switch.postgresOwnerSecretName" -}}
+{{- if .Values.postgresql.owner.existingSecret -}}
+{{- .Values.postgresql.owner.existingSecret -}}
+{{- else -}}
+{{- include "switch.secretName" . -}}
+{{- end -}}
+{{- end }}
+
+{{- define "switch.postgresOwnerSecretKey" -}}
+{{- .Values.postgresql.owner.existingSecretKey | default "DB_OWNER_PASSWORD" -}}
+{{- end }}
+
{{/*
Other service hostnames (used in env vars and init containers).
*/}}
@@ -321,6 +341,19 @@ Include with `nindent 12`.
key: {{ include "switch.postgresSecretKey" . }}
- name: DB_NAME
value: {{ include "switch.postgresDatabase" . | quote }}
+{{- if .Values.postgresql.owner.username }}
+- name: DB_OWNER_USER
+ value: {{ .Values.postgresql.owner.username | quote }}
+- name: DB_OWNER_PASSWORD
+ valueFrom:
+ secretKeyRef:
+ name: {{ include "switch.postgresOwnerSecretName" . }}
+ key: {{ include "switch.postgresOwnerSecretKey" . }}
+{{- end }}
+{{- if not .Values.postgresql.requireRestrictedRole }}
+- name: DB_REQUIRE_RESTRICTED_ROLE
+ value: "false"
+{{- end }}
- name: DB_SSL_MODE
value: {{ .Values.postgresql.sslMode | quote }}
{{- if include "switch.dbCaBundleConfigMap" . }}
diff --git a/deploy/remote/helm/switch/values.yaml b/deploy/remote/helm/switch/values.yaml
index 1e2528a92..758a21219 100644
--- a/deploy/remote/helm/switch/values.yaml
+++ b/deploy/remote/helm/switch/values.yaml
@@ -67,6 +67,8 @@ ingress:
# chart then renders no Secret and the values below are ignored. That Secret
# must contain these keys:
# POSTGRES_PASSWORD (unless postgresql.existingSecret is set),
+# DB_OWNER_PASSWORD (only if postgresql.owner.username is set, and unless
+# postgresql.owner.existingSecret is set),
# AGENT_REGISTRATION_TOKEN, JWT_SECRET_KEY, GATEWAY_ADMIN_EMAIL,
# GATEWAY_ADMIN_PASSWORD,
# MATTERMOST_ADMIN_PASSWORD + MATTERMOST_USER_PASSWORD (if mattermost.enabled),
@@ -76,6 +78,9 @@ secrets:
# chart manages its own Secret from the values below.
existingSecret: ""
postgresPassword: ""
+ # Password for postgresql.owner.username. Required when that is set, unless
+ # postgresql.owner.existingSecret names a Secret that already carries it.
+ dbOwnerPassword: ""
agentRegistrationToken: ""
jwtSecretKey: ""
gatewayAdminEmail: ""
@@ -403,6 +408,48 @@ postgresql:
existingSecret: ""
existingSecretKey: POSTGRES_PASSWORD
+ # Schema owner credentials — the account with DDL and grant privileges, as
+ # distinct from DB_USER above (or postgresql.external.username for
+ # mode: existing), which name the RUNTIME role every request actually
+ # authenticates as and which the row-level-security policies apply to.
+ # switch-core uses the owner connection for exactly two things at boot, and
+ # never to serve a request: applying Alembic migrations, and re-issuing the
+ # runtime role's grants immediately afterwards, so a table a migration just
+ # added is usable by the role that is about to need it.
+ #
+ # Leave owner.username empty to opt this deployment out of both — boot then
+ # applies no migrations and expects the schema to already be at head (see
+ # config.py's owner_database_url / runs_migrations_at_boot).
+ #
+ # This chart does NOT create the runtime role, and does not grant it
+ # anything — that is application code, run once by switch-core at boot from
+ # this connection, and it only works once the role already exists. Create it
+ # out of band before pointing a deployment at it: docs/old/rds-migration.md
+ # has the exact SQL for an RDS master user; for mode: managed the equivalent
+ # is execing into the postgresql-0 pod as the "postgres" superuser the
+ # StatefulSet below already bootstraps.
+ owner:
+ username: ""
+ # Password source, mirroring existingSecret/existingSecretKey above. Only
+ # consulted when owner.username is set.
+ existingSecret: ""
+ existingSecretKey: DB_OWNER_PASSWORD
+
+ # Refuse to serve when the runtime connection (DB_USER) is not actually
+ # subject to row-level security — a superuser, a BYPASSRLS role, or the
+ # owner of the policied tables. Matches switch-core's own default (on),
+ # because the failure it guards against is silent: a deployment that
+ # believes it is isolating tenants and is not looks identical to one that
+ # is, right up until a second tenant reads the first's rooms.
+ #
+ # mode: managed ships DB_USER as the bootstrapped "postgres" superuser (see
+ # POSTGRES_USER on the StatefulSet below) and this chart does not yet offer
+ # a way to point it at a separately created role, so a managed-mode
+ # deployment left at the default here will refuse to start. Set this false
+ # until that role exists; switch-core then logs an error on every boot as
+ # the visible reminder that tenant isolation is not active.
+ requireRestrictedRole: true
+
# ── mode: existing ──────────────────────────────────────────────────────────
# Only used when mode=existing. The `switch` and `mattermost` databases and
# the user must already exist; the password comes from the Secret above.
diff --git a/docs/old/LOCAL_DEVELOPMENT.md b/docs/old/LOCAL_DEVELOPMENT.md
index bc244e62c..73c1311b7 100644
--- a/docs/old/LOCAL_DEVELOPMENT.md
+++ b/docs/old/LOCAL_DEVELOPMENT.md
@@ -25,17 +25,47 @@ just gateway-install # install gateway frontend deps (first time only)
```
`just init-env` copies `.env.example` to `.env` and fills every secret field
-that ships blank — `DB_PASSWORD`, `AGENT_REGISTRATION_TOKEN`,
-`JWT_SECRET_KEY`, `GATEWAY_ADMIN_PASSWORD`, `MATTERMOST_ADMIN_PASSWORD`,
-`MATTERMOST_USER_PASSWORD` — with a freshly generated `openssl rand -hex 24`
-value, then prints the gateway admin login it just set. The fields ship
-blank on purpose: it means no default credential (the old `admin`/`admin`)
-can ever reach a running stack by accident. The recipe refuses to touch an
-existing `.env`, so re-running it never rotates secrets out from under a
-stack that's already up — delete `.env` first if you actually want to
-regenerate everything. `just` loads `.env` automatically for every recipe
-below (`set dotenv-load := true` in the justfile), so nothing else needs to
-source it.
+that ships blank — `DB_PASSWORD`, `DB_OWNER_PASSWORD`,
+`AGENT_REGISTRATION_TOKEN`, `JWT_SECRET_KEY`, `GATEWAY_ADMIN_PASSWORD`,
+`MATTERMOST_ADMIN_PASSWORD`, `MATTERMOST_USER_PASSWORD` — with a freshly
+generated `openssl rand -hex 24` value, then prints the gateway admin login
+it just set. The fields ship blank on purpose: it means no default
+credential (the old `admin`/`admin`) can ever reach a running stack by
+accident. The recipe refuses to touch an existing `.env`, so re-running it
+never rotates secrets out from under a stack that's already up — delete
+`.env` first if you actually want to regenerate everything. `just` loads
+`.env` automatically for every recipe below (`set dotenv-load := true` in
+the justfile), so nothing else needs to source it.
+
+### Two database roles, not one
+
+`.env` carries two sets of Postgres credentials, because row-level security
+is inert against the role that owns the tables it protects — Postgres
+exempts a table's owner from its own policies. `DB_USER` / `DB_PASSWORD` name
+the **runtime role**: a plain, unprivileged login that switch-core
+authenticates as for every request, and the one the tenant-isolation
+policies actually apply to. `DB_OWNER_USER` / `DB_OWNER_PASSWORD` name the
+**schema owner**, used for exactly two things at boot — running Alembic and
+re-issuing the runtime role's grants right after — and never to serve a
+request. `just up`'s `init-db` container creates the runtime role on a fresh
+volume; it does not grant it anything, because on a fresh volume there is
+nothing yet to grant — switch-core does that itself once the owner
+connection has migrated the schema.
+
+If you have an existing `.env` from before this split, add `DB_OWNER_USER`
+and `DB_OWNER_PASSWORD` to it and change `DB_USER` to `switch_app` (see
+`.env.example`), then run `just reset` to rebuild the Postgres volume from
+scratch — that's the clean way to get the runtime role created, since
+`init-db` only creates it on a fresh volume. Reusing an old volume with the
+new `.env` leaves the runtime role missing entirely.
+
+Boot refuses to serve if the runtime connection turns out not to be
+restricted — a superuser, a role with `BYPASSRLS`, or the owner of a policied
+table — because that failure is silent otherwise: a deployment that believes
+it is isolating tenants and is not looks identical to one that is. Set
+`DB_REQUIRE_RESTRICTED_ROLE=false` as an escape hatch for a stack that has
+not created its runtime role yet; switch-core then logs an `error` on every
+boot instead, as a standing reminder that tenant isolation is not active.
## Starting the stack
diff --git a/docs/old/rds-migration.md b/docs/old/rds-migration.md
index d6cf765fe..2eb13e237 100644
--- a/docs/old/rds-migration.md
+++ b/docs/old/rds-migration.md
@@ -32,6 +32,14 @@ Read this before choosing an instance shape; two of these have bitten people.
- **Two databases on one instance.** The chart provisions `switch` and
`mattermost` on the same server. Both move together, or the Mattermost bridge
loses its state.
+- **Two roles, not one.** Row-level security is inert against the role that
+ owns the tables it protects — Postgres exempts a table's owner from its own
+ policies — so the connection switch-core serves requests as must be a
+ separate, unprivileged role from whatever ran the migrations. The RDS master
+ user stays the schema owner; it is never the runtime connection. See §7 step
+ 2 for the exact role to create, and note that `BYPASSRLS` is deliberately
+ never granted to it — that would be the same exemption as ownership, by a
+ different door.
- **Connections are sized against agents, not people.** Each switch-core
replica opens up to `db_pool_size + db_max_overflow` (30 + 10 today), plus
one unpooled connection for the listener. Bearer-token auth hits the database
@@ -172,6 +180,25 @@ so there is nothing to reset behind a replicated cutover.
2. Create the `switch` and `mattermost` databases and the application role. The
RDS master user is not a superuser, and `pg_dumpall` needs privileges it does
not have — so recreate roles by hand rather than restoring a globals dump.
+
+ The role Switch serves traffic as is a plain login with nothing beyond
+ what it needs to read and write rows — no `SUPERUSER`, no `CREATEDB`, no
+ `CREATEROLE`, and deliberately no `BYPASSRLS`, which would exempt it from
+ row-level security exactly the way owning the tables would:
+
+ ```sql
+ CREATE ROLE switch_app LOGIN PASSWORD '<generate one>'
+ NOSUPERUSER NOCREATEDB NOCREATEROLE NOBYPASSRLS;
+ ```
+
+ That is the only manual step. Nothing here grants `switch_app` anything on
+ `switch` or `mattermost` — switch-core does that itself at boot, from the
+ owner connection (the RDS master user, over `DB_OWNER_USER` /
+ `DB_OWNER_PASSWORD`), immediately after running Alembic. The master user
+ stays the schema owner throughout; it is configured as the owner
+ connection, never as `DB_USER`, so it is never the connection every
+ request is served over.
+
3. Download the CA bundle and put it in `postgresql.caBundle` — as
`existingConfigMap` if it is synced in from elsewhere, otherwise inline.
4. **Rehearse the whole thing** against a scratch database, and time it. The
diff --git a/justfile b/justfile
index 1c2b39dbb..cf3318d40 100644
--- a/justfile
+++ b/justfile
@@ -26,7 +26,7 @@ init-env:
exit 1
fi
cp .env.example .env
- for key in DB_PASSWORD \
+ for key in DB_PASSWORD DB_OWNER_PASSWORD \
AGENT_REGISTRATION_TOKEN JWT_SECRET_KEY GATEWAY_ADMIN_PASSWORD \
MATTERMOST_ADMIN_PASSWORD MATTERMOST_USER_PASSWORD; do
secret="$(openssl rand -hex 24)"