Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.
Python agent-orchestrationaiai-agentschatopsdiscordlangchainmcpmicrosoft-teamsmulti-agentopen-sourceopenaiself-hostedslack
feat(organizations): mirror WorkOS membership changes into Switch
Petr Bauch0cc555049931
diff --git a/CLAUDE.md b/CLAUDE.mdindex fa270db9e..3c008a09b 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -191,8 +191,8 @@ Tests live in `core/tests/switch_core/` mirroring the module structure. Uses pytOIDC browser sign-in (variables, redirect URI, WorkOS Connect setup)- `docs/old/workos-directory.md` — keeping organizations in WorkOS(`ORGANIZATION_DIRECTORY=workos`): what it needs, linking organizations and- existing accounts with `switch-link-workos`, what a sign-in brings in, and- what is not done yet+ existing accounts with `switch-link-workos`, what a sign-in and the mirror+ bring in or take away, and what is not done yet- `docs/old/google-workspace-setup.md` — offering Google Workspace on aself-hosted server: registering the Google app (Internal or External), itsredirect URI, saving it under Connectors or indiff --git a/core/switch_core/db/audit.py b/core/switch_core/db/audit.pyindex 6e02b2d8f..7e194faa7 100644--- a/core/switch_core/db/audit.py+++ b/core/switch_core/db/audit.py@@ -83,9 +83,16 @@ class OrganizationAuditAction(StrEnum):CONNECTOR_APP_REMOVED = "organization.connector_app_removed"SERVICE_SET = "organization.service_set"SERVICE_CLEARED = "organization.service_cleared"- # Brought in from a directory kept outside Switch when they signed in.- MEMBER_ADDED_AT_SIGN_IN = "organization.member_added_at_sign_in"- MEMBER_PROMOTED_AT_SIGN_IN = "organization.member_promoted_at_sign_in"+ # Changes made in a directory kept outside Switch, brought in at sign-in,+ # from its events or by the daily check (`details.source`).+ MEMBER_ADDED_FROM_DIRECTORY = "organization.member_added_from_directory"+ MEMBER_PROMOTED_FROM_DIRECTORY = "organization.member_promoted_from_directory"+ MEMBER_DEMOTED_FROM_DIRECTORY = "organization.member_demoted_from_directory"+ MEMBER_REMOVED_FROM_DIRECTORY = "organization.member_removed_from_directory"+ # A change brought in from the directory broke a rule Switch keeps itself+ # (the last Org Admin, agents left with an owner who can't act); applied+ # anyway, and recorded for the organization's admins (`details.kind`).+ DIRECTORY_CONFLICT = "organization.directory_conflict"# An operator made the organization follow one in the directory.LINKED_TO_DIRECTORY = "organization.linked_to_directory"diff --git a/core/switch_core/db/models.py b/core/switch_core/db/models.pyindex 565082d20..94a944053 100644--- a/core/switch_core/db/models.py+++ b/core/switch_core/db/models.py@@ -2640,6 +2640,30 @@ class OwnershipLease(Base):)+class DirectoryEventCursor(Base):+ """How far the directory mirror has read a directory's events+ (`workos_mirror.py`). One row per directory (`source`, `"workos"`).++ `after` is the last event applied, null until the first; until then the+ events are read from `started_at`, when the mirror first ran. Applying+ an event re-reads the membership it names, so reading one twice changes+ nothing, and the row needs no fence against a holder taken over.++ Not tenant-scoped: one directory's events reach every organization.+ """++ __tablename__ = "directory_event_cursors"++ source: Mapped[str] = mapped_column(Text, primary_key=True)+ after: Mapped[str | None] = mapped_column(Text, nullable=True)+ started_at: Mapped[datetime] = mapped_column(+ DateTime(timezone=True), nullable=False+ )+ updated_at: Mapped[datetime] = mapped_column(+ DateTime(timezone=True), server_default=func.now(), nullable=False+ )++# ── Trust settings ───────────────────────────────────────────────────────────diff --git a/core/switch_core/db/rls_ddl.py b/core/switch_core/db/rls_ddl.pyindex bce8cabf0..b2f551470 100644--- a/core/switch_core/db/rls_ddl.py+++ b/core/switch_core/db/rls_ddl.py@@ -195,6 +195,7 @@ GLOBAL_TABLES = frozenset("telemetry_snapshot_watermark","switch_core_processes","ownership_leases",+ "directory_event_cursors","trust_settings","service_oauth_clients","schema_expand_only_revisions",diff --git a/core/switch_core/db/tenant_lookup.py b/core/switch_core/db/tenant_lookup.pyindex f4c084630..a46a39478 100644--- a/core/switch_core/db/tenant_lookup.py+++ b/core/switch_core/db/tenant_lookup.py@@ -1,4 +1,4 @@-"""The whole exemption from row-level security, written out as fifteen functions.+"""The whole exemption from row-level security, written out as sixteen functions.Row-level security is enforced by `require_tenant_id()` (`db/rls_ddl.py`),which raises when no tenant is bound. That is the property everything else@@ -183,6 +183,12 @@ organization, and which linked organizations a person belongs to; thegateway asks it only about the signed-in person and the organizations WorkOSsays they belong to.+**`directory_organizations` is the sixteenth**: every organization that+follows one in the WorkOS directory. The directory mirror acts for no tenant+and no person: it reads the directory's changes and has to know which+organizations they may reach before it binds any of them. It discloses which+organizations are linked; the mirror is its only caller.+Why not the obvious alternatives is argued in`docs/old/multi-tenancy-phase1-db.md`, "The bootstrap problem"; the shortversion is that returning rows instead of tenant ids would put a second copy@@ -282,7 +288,7 @@ class TenantLookup:return f"{self.name}({', '.join('text' for _ in self.arguments)})"-# The fifteen of them. Ordered as the three shapes above: enumeration, then+# The sixteen of them. Ordered as the three shapes above: enumeration, then# credential resolution, then deriving a tenant from an identifier in hand.TENANT_LOOKUPS: tuple[TenantLookup, ...] = (TenantLookup(@@ -478,6 +484,18 @@ TENANT_LOOKUPS: tuple[TenantLookup, ...] = ("to bring the person's organization memberships up to date."),),+ TenantLookup(+ name="directory_organizations",+ arguments=(),+ query=(+ "SELECT id FROM organizations "+ "WHERE workos_organization_id IS NOT NULL ORDER BY id"+ ),+ purpose=(+ "Every organization that follows one in the WorkOS directory, for "+ "the directory mirror to read each one's link with it bound."+ ),+ ),TenantLookup(name="tenant_of_messaging_install",arguments=("platform", "external_workspace_id"),@@ -551,7 +569,7 @@ def attach_tenant_lookups(metadata: MetaData) -> None:# ── Calling them ──────────────────────────────────────────────────────────────# One `text()` per lookup, written out rather than assembled from `lookup.name`-# at call time: the fifteen names are fixed and known here, so there is nothing+# at call time: the sixteen names are fixed and known here, so there is nothing# for a call site to build. Each bind is named after the lookup's own argument,# which is what lets `_call` zip them positionally against the dataclass and# fail loudly on a mismatch rather than binding the workspace to the platform.@@ -604,6 +622,9 @@ _LOOKUP_STATEMENTS: dict[str, TextClause] = {"tenant_of_sign_in_flow": text("SELECT tenant_id FROM tenant_of_sign_in_flow(:state) AS tenant_id"),+ "directory_organizations": text(+ "SELECT tenant_id FROM directory_organizations() AS tenant_id"+ ),"directory_organizations_of_user": text("SELECT tenant_id FROM directory_organizations_of_user("":user_id, :workos_organization_ids) AS tenant_id"@@ -776,3 +797,11 @@ async def directory_organizations_of_user(user_id,json.dumps(sorted(workos_organization_ids)),)+++async def directory_organizations(+ session_factory: async_sessionmaker[AsyncSession],+) -> list[str]:+ return await _call(+ session_factory, TENANT_LOOKUPS_BY_NAME["directory_organizations"]+ )diff --git a/core/switch_core/gateway/organizations.py b/core/switch_core/gateway/organizations.pyindex 668e21dff..0cbd02fd9 100644--- a/core/switch_core/gateway/organizations.py+++ b/core/switch_core/gateway/organizations.py@@ -35,12 +35,9 @@ from switch_core.connections.broker import (ServiceError,get_service_broker,)-from switch_core.connections.sign_in_flows import SignInFlowsfrom switch_core.db.audit import (- AuditAction,OrganizationAuditAction,list_organization_audit_events,- record_audit_event,record_organization_audit_event,)from switch_core.db.models import (@@ -50,7 +47,6 @@ from switch_core.db.models import (User,require_tenant_id,)-from switch_core.db.session_scope import tenant_sessionfrom switch_core.db.stores.agent_store import AgentStorefrom switch_core.db.stores.api_key_store import ApiKeyStorefrom switch_core.db.stores.organization_store import OrganizationStore@@ -90,10 +86,6 @@ from switch_core.gateway.schemas import (WorkspaceCreateRequest,)from switch_core.gateway.service_connections import service_refusal-from switch_core.gateway.workspace_access import (- finish_revocation,- revoke_workspace_access,-)from switch_core.gateway.workspace_creation import (NOTHING_CHANGED,create_organization_for,@@ -101,6 +93,11 @@ from switch_core.gateway.workspace_creation import (directory_failed,lock_workspace_allowance,)+from switch_core.organization_access import (+ AccessRevocation,+ end_access_through_organization,+ workspaces_where_they_own_agents,+)from switch_core.organization_directory import DirectoryError, OrganizationDirectoryrouter = APIRouter()@@ -451,69 +448,39 @@ async def _end_access_through_organization(organization_id: str,actor_id: str,reason: str,- agent_store: AgentStore,- api_key_store: ApiKeyStore,- broker: ServiceBroker,- protocol: AgentCore,- sign_in_flows: SignInFlows,+ revocation: AccessRevocation,) -> str | None:"""Take away what `user_id` holds in the workspaces they stand in only as- the organization's `org_admin`, before they stop being one.+ the organization's `org_admin`, before they stop being one+ (`organization_access.end_access_through_organization`).Refuses, 409 naming the workspaces, while they own agents in any of them,- as removing a workspace member does. Otherwise removes their keys,- service connections and sign-ins under way in each, one workspace at a- time, each recorded in that workspace's audit log. Raises `ServiceError`- when a connection is busy. Returns a warning naming what could not be- revoked at a vendor, or None.+ as removing a workspace member does."""reached = await workspaces_reached_through_organization(session_factory, user_id, organization_id)- with_agents = []- for tenant in reached:- async with tenant_session(session_factory, tenant.id) as session:- if await agent_store.get_by_owner(session, user_id):- with_agents.append(tenant.name)+ with_agents = await workspaces_where_they_own_agents(+ session_factory, revocation.agent_store, user_id, reached+ )if with_agents:raise HTTPException(status_code=409,detail=("They own agents in workspaces they're in only as an ""organization admin. Delete those agents first: "- f"{', '.join(sorted(with_agents))}"+ f"{', '.join(sorted(tenant.name for tenant in with_agents))}"),)- warnings = []- for tenant in reached:- async with tenant_session(session_factory, tenant.id) as session:- revoked = await revoke_workspace_access(- session,- user_id,- api_key_store=api_key_store,- broker=broker,- sign_in_flows=sign_in_flows,- )- await record_audit_event(- session,- tenant_id=tenant.id,- actor_user_id=actor_id,- action=AuditAction.ORGANIZATION_ADMIN_ACCESS_ENDED,- target_type="user",- target_id=user_id,- details={- "reason": reason,- "api_keys_deleted": len(revoked.key_hashes),- "service_connections_deleted": len(revoked.removed_services),- },- )- await session.commit()- warning = await finish_revocation(- session, user_id, revoked, protocol=protocol, broker=broker- )- if warning:- warnings.append(f"{tenant.name}: {warning}")- return " ".join(warnings) or None+ return await end_access_through_organization(+ session_factory,+ revocation,+ user_id=user_id,+ reached=reached,+ actor_id=actor_id,+ details={"reason": reason},+ keep_agent_keys=False,+ )@router.patch("/organizations/current/members/{user_id}", response_model=None)@@ -576,11 +543,13 @@ async def update_organization_member_role(organization_id=organization_id,actor_id=user.id,reason="demoted",- agent_store=agent_store,- api_key_store=api_key_store,- broker=broker,- protocol=protocol,- sign_in_flows=request.app.state.sign_in_flows,+ revocation=AccessRevocation(+ agent_store=agent_store,+ api_key_store=api_key_store,+ broker=broker,+ protocol=protocol,+ sign_in_flows=request.app.state.sign_in_flows,+ ),)except ServiceError as error:return service_refusal(error)@@ -692,11 +661,13 @@ async def remove_organization_member(organization_id=organization_id,actor_id=user.id,reason="removed",- agent_store=agent_store,- api_key_store=api_key_store,- broker=broker,- protocol=protocol,- sign_in_flows=request.app.state.sign_in_flows,+ revocation=AccessRevocation(+ agent_store=agent_store,+ api_key_store=api_key_store,+ broker=broker,+ protocol=protocol,+ sign_in_flows=request.app.state.sign_in_flows,+ ),)except ServiceError as error:return service_refusal(error)diff --git a/core/switch_core/gateway/tenants.py b/core/switch_core/gateway/tenants.pyindex 93237fe42..f463ca1ba 100644--- a/core/switch_core/gateway/tenants.py+++ b/core/switch_core/gateway/tenants.py@@ -1470,6 +1470,7 @@ async def remove_member(api_key_store=api_key_store,broker=broker,sign_in_flows=request.app.state.sign_in_flows,+ keep_agent_keys=False,)except ServiceError as error:return service_refusal(error)diff --git a/core/switch_core/gateway/workspace_access.py b/core/switch_core/gateway/workspace_access.pyindex 66313b410..7093b8d9c 100644--- a/core/switch_core/gateway/workspace_access.py+++ b/core/switch_core/gateway/workspace_access.py@@ -13,11 +13,13 @@ from __future__ import annotationsfrom dataclasses import dataclass, field+from sqlalchemy import selectfrom sqlalchemy.ext.asyncio import AsyncSessionfrom switch_core.bridges.agent.protocol.agent_core import AgentCorefrom switch_core.connections.broker import ServiceBroker, SignInfrom switch_core.connections.sign_in_flows import SignInFlows+from switch_core.db.models import Agentfrom switch_core.db.stores.api_key_store import ApiKeyStore@@ -34,13 +36,27 @@ async def revoke_workspace_access(api_key_store: ApiKeyStore,broker: ServiceBroker,sign_in_flows: SignInFlows,+ keep_agent_keys: bool,) -> RevokedAccess:"""Remove `user_id`'s keys, service connections and sign-ins under way inthe bound workspace, in the caller's transaction. Raises `ServiceError`when a connection is busy, before anything is removed. The caller- commits, then calls `finish_revocation`."""+ commits, then calls `finish_revocation`.++ `keep_agent_keys` keeps the keys agents they own run on, for a caller+ that leaves those agents in place rather than refusing while they exist.+ """removed_services = await broker.remove_connections(session, user_id)keys = await api_key_store.get_by_user(session, user_id)+ if keep_agent_keys and keys:+ in_use = set(+ await session.scalars(+ select(Agent.api_key_id).where(+ Agent.api_key_id.in_([key.id for key in keys])+ )+ )+ )+ keys = [key for key in keys if key.id not in in_use]for key in keys:await api_key_store.delete(session, key.id)await sign_in_flows.discard_for_user(session, user_id, None)diff --git a/core/switch_core/main.py b/core/switch_core/main.pyindex 07a904ba5..950ec3735 100644--- a/core/switch_core/main.py+++ b/core/switch_core/main.py@@ -200,6 +200,7 @@ from switch_core.observability.pool import (pool_stats,)from switch_core.observability.query import instrument_queries+from switch_core.organization_access import AccessRevocationfrom switch_core.organization_directory import LocalDirectory, OrganizationDirectoryfrom switch_core.outbound import guarded_async_clientfrom switch_core.ownership import Ownership@@ -231,6 +232,7 @@ from switch_core.trust.signals import TrustSettingsSignalsfrom switch_core.user_changes import LocalUserChanges, SignalUserChangesfrom switch_core.version import switch_core_versionfrom switch_core.workos_directory import WorkOSDirectory+from switch_core.workos_mirror import WorkOSMirror, own_workos_mirrorlogger = logging.getLogger(__name__)@@ -1247,6 +1249,7 @@ async def run(config: SwitchConfig, process_id: str) -> None:await connector_apps.load()# ── Gateway app ───────────────────────────────────────────────────────────+ organization_directory = _organization_directory(config, service_http)gateway_app = create_gateway_app(agent_store=agent_store,room_store=room_store,@@ -1263,7 +1266,7 @@ async def run(config: SwitchConfig, process_id: str) -> None:invitation_store=invitation_store,join_domain_store=join_domain_store,organization_store=organization_store,- organization_directory=_organization_directory(config, service_http),+ organization_directory=organization_directory,template_store=template_store,usage_store=usage_store,budget_store=budget_store,@@ -1408,6 +1411,24 @@ async def run(config: SwitchConfig, process_id: str) -> None:if config.service_connections_enabled:protocol.service_broker = service_brokerprotocol.install_service = install_service+ if isinstance(organization_directory, WorkOSDirectory):+ # Checked present with the directory (`_validate_organization_directory`).+ assert config.gateway_oidc_issuer_url is not None+ own_workos_mirror(+ ownership,+ WorkOSMirror(+ session_factory,+ organization_directory,+ AccessRevocation(+ agent_store=agent_store,+ api_key_store=api_key_store,+ broker=service_broker,+ protocol=protocol,+ sign_in_flows=gateway_app.state.sign_in_flows,+ ),+ config.gateway_oidc_issuer_url,+ ),+ )snapshot_reporter = SnapshotReporter(telemetry=telemetry,diff --git a/core/switch_core/migrations/versions/a7c3e9f15b20_directory_mirror.py b/core/switch_core/migrations/versions/a7c3e9f15b20_directory_mirror.pynew file mode 100644index 000000000..3d8be13b1--- /dev/null+++ b/core/switch_core/migrations/versions/a7c3e9f15b20_directory_mirror.py@@ -0,0 +1,60 @@+"""the directory mirror: its event cursor and the lookup it reads++`directory_event_cursors` records how far the mirror has read a directory's+events, one row per directory. A global table: no tenant column and no+row-level security policy (`db/rls_ddl.py`'s `GLOBAL_TABLES`).++`directory_organizations` is the sixteenth lookup in the exemption from+row-level security: every organization that follows one in the directory.+See `db/tenant_lookup.py` for what it discloses. Its DDL is a verbatim copy+of `switch_core/db/tenant_lookup.py` as it stood when this migration was+written, copied rather than imported so a later edit cannot change what this+revision means.++Revision ID: a7c3e9f15b20+Revises: f2b8d6c1a4e9+Create Date: 2026-11-02 12:00:00.000000++"""++from collections.abc import Sequence++import sqlalchemy as sa+from alembic import op++revision: str = "a7c3e9f15b20"+down_revision: str | Sequence[str] | None = "f2b8d6c1a4e9"+branch_labels: str | Sequence[str] | None = None+depends_on: str | Sequence[str] | None = None+expand_only: bool = True++SECURE_SEARCH_PATH = "pg_catalog, public, pg_temp"++CREATE_DIRECTORY_ORGANIZATIONS = f"""CREATE OR REPLACE FUNCTION directory_organizations()+ RETURNS SETOF text+ LANGUAGE sql STABLE SECURITY DEFINER+ SET search_path = {SECURE_SEARCH_PATH}+AS $$SELECT id FROM organizations WHERE workos_organization_id IS NOT NULL ORDER BY id$$"""++DROP_DIRECTORY_ORGANIZATIONS = "DROP FUNCTION IF EXISTS directory_organizations()"+++def upgrade() -> None:+ op.create_table(+ "directory_event_cursors",+ sa.Column("source", sa.Text(), primary_key=True),+ sa.Column("after", sa.Text(), nullable=True),+ sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),+ sa.Column(+ "updated_at",+ sa.DateTime(timezone=True),+ server_default=sa.func.now(),+ nullable=False,+ ),+ )+ op.execute(CREATE_DIRECTORY_ORGANIZATIONS)+++def downgrade() -> None:+ op.execute(DROP_DIRECTORY_ORGANIZATIONS)+ op.drop_table("directory_event_cursors")diff --git a/core/switch_core/organization_access.py b/core/switch_core/organization_access.pynew file mode 100644index 000000000..dd02dea80--- /dev/null+++ b/core/switch_core/organization_access.py@@ -0,0 +1,163 @@+"""Ending someone's access to an organization's workspaces: what being demoted+from `org_admin`, or leaving the organization, takes away.++Shared by the organization routes (`gateway/organizations.py`), which refuse+first while the person owns agents there, and the directory mirror+(`organization_mirror.py`), which applies a change made in the directory+whatever it breaks and reports what it broke.+"""++from __future__ import annotations++from dataclasses import dataclass+from typing import Any++from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker++from switch_core.bridges.agent.protocol.agent_core import AgentCore+from switch_core.connections.broker import ServiceBroker+from switch_core.connections.sign_in_flows import SignInFlows+from switch_core.db.audit import AuditAction, record_audit_event+from switch_core.db.models import Tenant, TenantMember+from switch_core.db.session_scope import tenant_session+from switch_core.db.stores.agent_store import AgentStore+from switch_core.db.stores.api_key_store import ApiKeyStore+from switch_core.gateway.workspace_access import (+ finish_revocation,+ revoke_workspace_access,+)+++@dataclass(frozen=True)+class AccessRevocation:+ """What taking someone's access away in a workspace needs."""++ agent_store: AgentStore+ api_key_store: ApiKeyStore+ broker: ServiceBroker+ protocol: AgentCore+ sign_in_flows: SignInFlows+++async def workspaces_where_they_own_agents(+ session_factory: async_sessionmaker[AsyncSession],+ agent_store: AgentStore,+ user_id: str,+ workspaces: list[Tenant],+) -> list[Tenant]:+ """Those of `workspaces` in which `user_id` owns an agent, each read on+ its own short session."""+ found = []+ for tenant in workspaces:+ async with tenant_session(session_factory, tenant.id) as session:+ if await agent_store.get_by_owner(session, user_id):+ found.append(tenant)+ return found+++async def end_access_through_organization(+ session_factory: async_sessionmaker[AsyncSession],+ revocation: AccessRevocation,+ *,+ user_id: str,+ reached: list[Tenant],+ actor_id: str | None,+ details: dict[str, Any],+ keep_agent_keys: bool,+) -> str | None:+ """Take away what `user_id` holds in `reached`, the workspaces they stand+ in only as their organization's `org_admin`: their keys, service+ connections and sign-ins under way, one workspace at a time, each+ recorded in that workspace's audit log with `details`;+ `keep_agent_keys` as `revoke_workspace_access` has it. Raises+ `ServiceError` when a connection is busy. Returns a warning naming what+ could not be revoked at a vendor, or None."""+ warnings = []+ for tenant in reached:+ async with tenant_session(session_factory, tenant.id) as session:+ revoked = await revoke_workspace_access(+ session,+ user_id,+ api_key_store=revocation.api_key_store,+ broker=revocation.broker,+ sign_in_flows=revocation.sign_in_flows,+ keep_agent_keys=keep_agent_keys,+ )+ await record_audit_event(+ session,+ tenant_id=tenant.id,+ actor_user_id=actor_id,+ action=AuditAction.ORGANIZATION_ADMIN_ACCESS_ENDED,+ target_type="user",+ target_id=user_id,+ details={+ **details,+ "api_keys_deleted": len(revoked.key_hashes),+ "service_connections_deleted": len(revoked.removed_services),+ },+ )+ await session.commit()+ warning = await finish_revocation(+ session,+ user_id,+ revoked,+ protocol=revocation.protocol,+ broker=revocation.broker,+ )+ if warning:+ warnings.append(f"{tenant.name}: {warning}")+ return " ".join(warnings) or None+++async def remove_from_workspace(+ session_factory: async_sessionmaker[AsyncSession],+ revocation: AccessRevocation,+ *,+ user_id: str,+ tenant: Tenant,+ actor_id: str | None,+ details: dict[str, Any],+ keep_agent_keys: bool,+) -> str | None:+ """Remove `user_id` from `tenant`, with their keys, service connections+ and sign-ins under way there, recorded in its audit log with `details`.+ Agents they own are left as they are, and with `keep_agent_keys` so are+ the keys they run on. Raises `ServiceError` when a+ connection is busy. Returns a warning naming what could not be revoked+ at a vendor, or None."""+ async with tenant_session(session_factory, tenant.id) as session:+ membership = await session.get(TenantMember, (tenant.id, user_id))+ if membership is None:+ return None+ revoked = await revoke_workspace_access(+ session,+ user_id,+ api_key_store=revocation.api_key_store,+ broker=revocation.broker,+ sign_in_flows=revocation.sign_in_flows,+ keep_agent_keys=keep_agent_keys,+ )+ await record_audit_event(+ session,+ tenant_id=tenant.id,+ actor_user_id=actor_id,+ action=AuditAction.MEMBER_REMOVED,+ target_type="user",+ target_id=user_id,+ details={+ **details,+ "role": membership.role,+ "api_keys_deleted": len(revoked.key_hashes),+ "service_connections_deleted": len(revoked.removed_services),+ },+ )+ await session.delete(membership)+ await session.commit()+ warning = await finish_revocation(+ session,+ user_id,+ revoked,+ protocol=revocation.protocol,+ broker=revocation.broker,+ )+ return f"{tenant.name}: {warning}" if warning else Nonediff --git a/core/switch_core/organization_mirror.py b/core/switch_core/organization_mirror.pyindex 469e7bd57..7cdefe2ba 100644--- a/core/switch_core/organization_mirror.py+++ b/core/switch_core/organization_mirror.py@@ -1,27 +1,38 @@"""Switch's organization rows brought up to date with a directory kept-outside Switch (`organization_directory.py`), for one person at sign-in.+outside Switch (`organization_directory.py`).Only organizations an operator has linked to one in the directory aretouched (`organizations.workos_organization_id`), and only by memberships ofexactly that one: a customer starts using Switch by having its organizationlinked, not by one of its people signing in.-- Someone the directory lists and Switch does not is added, with the- directory's role.-- A member the directory lists as `org_admin` is made one.+The directory is the record, so whatever it says is applied:-Both are recorded in the organization's audit log. Neither takes anything-away. A demotion, or a membership the directory no longer lists, is logged at-`error` and left as it is: taking a role away also takes away the workspaces-it reached, which is for the directory mirror to do (not built yet), not a-sign-in.+- Someone it lists and Switch does not is added, with its role.+- A member it lists as `org_admin` is made one.+- An `org_admin` it lists as anything else is made a member, and loses what+ they held in the organization's workspaces they stood in only as its+ `org_admin` (`organization_access.end_access_through_organization`).+- A member it no longer lists is removed, from the organization and from each+ of its workspaces, with their keys, service connections and sign-ins there.++Each is recorded in the organization's audit log, with where it came from+(`source`). A change that breaks a rule Switch keeps itself (the last Org+Admin, agents left with an owner who can no longer act, a workspace left+with no owner) is applied all the same, logged at `error` and recorded as a+`directory_conflict` for the organization's admins.++A sign-in (`bring_in_memberships`) only grants: it never waits on taking+access away, which the mirror (`workos_mirror.py`) does from the directory's+events and its daily check (`apply_from_directory`)."""from __future__ import annotationsimport logging+from typing import Any-from sqlalchemy import select, update+from sqlalchemy import delete, func, select, updatefrom sqlalchemy.dialects.postgresql import insertfrom sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker@@ -30,21 +41,39 @@ from switch_core.db.audit import (OrganizationAuditAction,record_organization_audit_event,)-from switch_core.db.models import Organization, OrganizationMember-from switch_core.db.session_scope import organization_session+from switch_core.db.models import (+ Organization,+ OrganizationMember,+ Tenant,+ TenantMember,+)+from switch_core.db.session_scope import organization_session, tenant_sessionfrom switch_core.db.tenant_lookup import directory_organizations_of_user+from switch_core.gateway.auth import (+ workspaces_in_organization,+ workspaces_reached_through_organization,+)+from switch_core.organization_access import (+ AccessRevocation,+ end_access_through_organization,+ remove_from_workspace,+ workspaces_where_they_own_agents,+)from switch_core.organization_directory import DirectoryMembershiplogger = logging.getLogger(__name__)+SIGN_IN = "sign_in"+async def bring_in_memberships(session_factory: async_sessionmaker[AsyncSession],user_id: str,memberships: list[DirectoryMembership],) -> None:- """Bring `user_id`'s rows in each linked organization up to date with- `memberships`, one organization at a time, each committed on its own."""+ """At sign-in: add `user_id` to each linked organization `memberships`+ lists them in, and make them Org Admin where it lists them as one, each+ organization committed on its own. What it would take away is logged."""roles = {membership.workos_organization_id: membership.role for membership in memberships}@@ -52,39 +81,163 @@ async def bring_in_memberships(session_factory, user_id, list(roles)):async with organization_session(session_factory, organization_id) as session:- await _bring_in_one(session, organization_id, user_id, roles)+ workos_organization_id, current = await _link_and_role(+ session, organization_id, user_id+ )+ wanted = (+ roles.get(workos_organization_id) if workos_organization_id else None+ )+ if wanted is not None and _grants(current, wanted):+ await _grant(+ session, organization_id, user_id, current, wanted, SIGN_IN+ )+ elif current != wanted:+ logger.warning(+ "User %s is %s in organization %s and %s in WorkOS "+ "organization %s; the directory mirror applies it",+ user_id,+ current,+ organization_id,+ wanted or "not a member",+ workos_organization_id,+ )-async def _bring_in_one(- session: AsyncSession,+async def apply_from_directory(+ session_factory: async_sessionmaker[AsyncSession],+ revocation: AccessRevocation,+ *,organization_id: str,user_id: str,- roles: dict[str, str],+ wanted: str | None,+ source: str,) -> None:- workos_organization_id, current = (- await session.execute(- select(Organization.workos_organization_id, OrganizationMember.role)- .outerjoin(- OrganizationMember,- (OrganizationMember.organization_id == Organization.id)- & (OrganizationMember.user_id == user_id),- )- .where(Organization.id == organization_id)+ """Make `user_id`'s membership of `organization_id` what the directory+ says, `wanted` (a role, or None for no membership). Raises `ServiceError`+ when a connection to revoke is busy, having changed nothing that a retry+ would not finish."""+ async with organization_session(session_factory, organization_id) as session:+ _, current = await _link_and_role(session, organization_id, user_id)+ if current == wanted:+ return+ if _grants(current, wanted):+ assert wanted is not None+ await _grant(session, organization_id, user_id, current, wanted, source)+ return+ admins = await _count_admins(session, organization_id)+ assert current is not None+ conflicts: list[dict[str, Any]] = []+ warnings: list[str] = []+ details = {"source": source}++ if current == ORGANIZATION_ADMIN_ROLE:+ if admins <= 1:+ conflicts.append({"kind": "last_org_admin"})+ reached = await workspaces_reached_through_organization(+ session_factory, user_id, organization_id)- ).one()- wanted = roles.get(workos_organization_id) if workos_organization_id else None+ conflicts += await _agents_left(session_factory, revocation, user_id, reached)+ warning = await end_access_through_organization(+ session_factory,+ revocation,+ user_id=user_id,+ reached=reached,+ actor_id=None,+ details={+ "reason": "demoted" if wanted is not None else "removed",+ **details,+ },+ keep_agent_keys=True,+ )+ if warning:+ warnings.append(warning)+if wanted is None:- if current is not None:+ workspaces = await workspaces_in_organization(+ session_factory, user_id, organization_id+ )+ conflicts += await _agents_left(+ session_factory, revocation, user_id, workspaces+ )+ for tenant in workspaces:+ if await _is_last_owner(session_factory, tenant.id, user_id):+ conflicts.append(+ {"kind": "workspace_left_without_owner", "workspace": tenant.name}+ )+ warning = await remove_from_workspace(+ session_factory,+ revocation,+ user_id=user_id,+ tenant=tenant,+ actor_id=None,+ details=details,+ keep_agent_keys=True,+ )+ if warning:+ warnings.append(warning)++ async with organization_session(session_factory, organization_id) as session:+ if wanted is None:+ await session.execute(+ delete(OrganizationMember).where(+ OrganizationMember.organization_id == organization_id,+ OrganizationMember.user_id == user_id,+ )+ )+ action = OrganizationAuditAction.MEMBER_REMOVED_FROM_DIRECTORY+ else:+ await _set_role(session, organization_id, user_id, wanted)+ action = OrganizationAuditAction.MEMBER_DEMOTED_FROM_DIRECTORY+ await record_organization_audit_event(+ session,+ organization_id=organization_id,+ actor_user_id=None,+ action=action,+ target_type="user",+ target_id=user_id,+ details={+ **details,+ "from_role": current,+ "role": wanted,+ **({"warning": " ".join(warnings)} if warnings else {}),+ },+ )+ for conflict in conflicts:logger.error(- "User %s is %s in organization %s, which WorkOS organization %s "- "no longer lists them in; Switch keeps their membership until "- "the directory mirror can end it",+ "Applying WorkOS's change to user %s in organization %s broke "+ "a rule Switch keeps: %s",user_id,- current,organization_id,- workos_organization_id,+ conflict,)- return+ await record_organization_audit_event(+ session,+ organization_id=organization_id,+ actor_user_id=None,+ action=OrganizationAuditAction.DIRECTORY_CONFLICT,+ target_type="user",+ target_id=user_id,+ details={**details, **conflict},+ )+ await session.commit()+++def _grants(current: str | None, wanted: str | None) -> bool:+ """Whether going from `current` to `wanted` only gives: joining, or+ becoming Org Admin."""+ if wanted is None or current == wanted:+ return False+ return current is None or wanted == ORGANIZATION_ADMIN_ROLE+++async def _grant(+ session: AsyncSession,+ organization_id: str,+ user_id: str,+ current: str | None,+ wanted: str,+ source: str,+) -> None:if current is None:added = await session.execute(insert(OrganizationMember)@@ -95,32 +248,12 @@ async def _bring_in_one(if added.first() is None:# A concurrent sign-in or invitation added them first.return- action = OrganizationAuditAction.MEMBER_ADDED_AT_SIGN_IN- details = {"role": wanted}- elif current != wanted and wanted == ORGANIZATION_ADMIN_ROLE:- await session.execute(- update(OrganizationMember)- .where(- OrganizationMember.organization_id == organization_id,- OrganizationMember.user_id == user_id,- )- .values(role=wanted)- )- action = OrganizationAuditAction.MEMBER_PROMOTED_AT_SIGN_IN- details = {"from_role": current, "role": wanted}+ action = OrganizationAuditAction.MEMBER_ADDED_FROM_DIRECTORY+ details: dict[str, Any] = {"source": source, "role": wanted}else:- if current != wanted:- logger.error(- "User %s is %s in organization %s and %s in WorkOS organization "- "%s; Switch keeps their role until the directory mirror can "- "change it",- user_id,- current,- organization_id,- wanted,- workos_organization_id,- )- return+ await _set_role(session, organization_id, user_id, wanted)+ action = OrganizationAuditAction.MEMBER_PROMOTED_FROM_DIRECTORY+ details = {"source": source, "from_role": current, "role": wanted}await record_organization_audit_event(session,organization_id=organization_id,@@ -131,3 +264,77 @@ async def _bring_in_one(details=details,)await session.commit()+++async def _link_and_role(+ session: AsyncSession, organization_id: str, user_id: str+) -> tuple[str | None, str | None]:+ """The organization's link to the directory, and `user_id`'s role in it+ (None if not a member), in one read."""+ workos_organization_id, role = (+ await session.execute(+ select(Organization.workos_organization_id, OrganizationMember.role)+ .outerjoin(+ OrganizationMember,+ (OrganizationMember.organization_id == Organization.id)+ & (OrganizationMember.user_id == user_id),+ )+ .where(Organization.id == organization_id)+ )+ ).one()+ return workos_organization_id, role+++async def _set_role(+ session: AsyncSession, organization_id: str, user_id: str, role: str+) -> None:+ await session.execute(+ update(OrganizationMember)+ .where(+ OrganizationMember.organization_id == organization_id,+ OrganizationMember.user_id == user_id,+ )+ .values(role=role)+ )+++async def _count_admins(session: AsyncSession, organization_id: str) -> int:+ return (+ await session.scalar(+ select(func.count()).where(+ OrganizationMember.organization_id == organization_id,+ OrganizationMember.role == ORGANIZATION_ADMIN_ROLE,+ )+ )+ ) or 0+++async def _agents_left(+ session_factory: async_sessionmaker[AsyncSession],+ revocation: AccessRevocation,+ user_id: str,+ workspaces: list[Tenant],+) -> list[dict[str, Any]]:+ """A conflict for each of `workspaces` where `user_id` owns agents, which+ stay, with the keys they run on, owned by someone who can no longer act+ there, for an admin to deal with."""+ return [+ {"kind": "agents_left_with_owner", "workspace": tenant.name}+ for tenant in await workspaces_where_they_own_agents(+ session_factory, revocation.agent_store, user_id, workspaces+ )+ ]+++async def _is_last_owner(+ session_factory: async_sessionmaker[AsyncSession], tenant_id: str, user_id: str+) -> bool:+ async with tenant_session(session_factory, tenant_id) as session:+ owners = list(+ await session.scalars(+ select(TenantMember.user_id).where(+ TenantMember.tenant_id == tenant_id, TenantMember.role == "owner"+ )+ )+ )+ return owners == [user_id]diff --git a/core/switch_core/workos_directory.py b/core/switch_core/workos_directory.pyindex 0feaf3ad5..3dbb9b052 100644--- a/core/switch_core/workos_directory.py+++ b/core/switch_core/workos_directory.py@@ -27,6 +27,7 @@ local directory, and that is what a new organization is until then.from __future__ import annotationsimport logging+from collections.abc import Mappingfrom typing import Anyimport httpx@@ -83,7 +84,9 @@ class WorkOSDirectory:self._api_key = api_keyself._api_url = api_url.rstrip("/")- async def _get(self, path: str, params: dict[str, str]) -> dict[str, Any]:+ async def _get(+ self, path: str, params: Mapping[str, str | list[str]]+ ) -> dict[str, Any]:try:response = await self._http.get(f"{self._api_url}{path}",@@ -137,11 +140,17 @@ class WorkOSDirectory:"""The active memberships of WorkOS user `subject`, one per WorkOSorganization. Which of those organizations Switch follows is for thecaller to read from its own rows."""- memberships: list[DirectoryMembership] = []- for membership in await self._list(- "/user_management/organization_memberships",- {"user_id": subject, "statuses": "active"},- ):+ return self._read_memberships(+ await self._list(+ "/user_management/organization_memberships",+ {"user_id": subject, "statuses": "active"},+ )+ )++ @staticmethod+ def _read_memberships(items: list[dict[str, Any]]) -> list[DirectoryMembership]:+ memberships = []+ for membership in items:organization_id = membership.get("organization_id")role = membership.get("role")slug = role.get("slug") if isinstance(role, dict) else None@@ -154,6 +163,70 @@ class WorkOSDirectory:)return memberships+ async def membership_role(+ self, subject: str, workos_organization_id: str+ ) -> str | None:+ """Switch's role for WorkOS user `subject`'s active membership of+ `workos_organization_id`, or None if they hold none."""+ memberships = await self._list(+ "/user_management/organization_memberships",+ {+ "user_id": subject,+ "organization_id": workos_organization_id,+ "statuses": "active",+ },+ )+ for membership in self._read_memberships(memberships):+ if membership.workos_organization_id == workos_organization_id:+ return membership.role+ return None++ async def organization_members(self, workos_organization_id: str) -> dict[str, str]:+ """Switch's role for each WorkOS user with an active membership of+ `workos_organization_id`, by their WorkOS user id."""+ members: dict[str, str] = {}+ for item in await self._list(+ "/user_management/organization_memberships",+ {"organization_id": workos_organization_id, "statuses": "active"},+ ):+ user_id = item.get("user_id")+ if not isinstance(user_id, str):+ raise _unreadable("a membership")+ [membership] = self._read_memberships([item])+ members[user_id] = membership.role+ return members++ async def events(+ self, types: tuple[str, ...], *, after: str | None, range_start: str+ ) -> list[dict[str, Any]]:+ """One page of events of `types`, oldest first: those after event+ `after`, or from `range_start` (ISO 8601) when there is none yet.+ Empty once there are no more."""+ params: dict[str, str | list[str]] = {+ "events": list(types),+ "limit": str(_PAGE),+ "order": "asc",+ }+ if after:+ params["after"] = after+ else:+ params["range_start"] = range_start+ page = await self._get("/events", params)+ data = page.get("data")+ if not isinstance(data, list):+ raise _unreadable("a list of events")+ events = []+ for event in data:+ if (+ not isinstance(event, dict)+ or not isinstance(event.get("id"), str)+ or not isinstance(event.get("event"), str)+ or not isinstance(event.get("data"), dict)+ ):+ raise _unreadable("an event")+ events.append(event)+ return events+async def organization(self, workos_organization_id: str) -> dict[str, Any]:"""The WorkOS organization `workos_organization_id`, for an operatorlinking a Switch organization to it."""diff --git a/core/switch_core/workos_mirror.py b/core/switch_core/workos_mirror.pynew file mode 100644index 000000000..e3adc4dff--- /dev/null+++ b/core/switch_core/workos_mirror.py@@ -0,0 +1,341 @@+"""The directory mirror under `ORGANIZATION_DIRECTORY=workos`: changes made+in WorkOS to a linked organization's members, applied in Switch+(`organization_mirror.apply_from_directory`).++- **Events.** WorkOS's Events API, polled every `POLL_INTERVAL_SECONDS` from+ the cursor in `directory_event_cursors`, rather than webhooks: ordered,+ resumable, and with no public endpoint or signing secret. An event is only+ a prompt: applying it re-reads the person's current membership from+ WorkOS, so events read out of order, twice, or after a later change all+ end in the same state. An event that fails to apply stops the pass, and the+ next pass starts again from it.+- **The daily check.** Every `CHECK_INTERVAL_SECONDS`, and when the mirror+ starts, each linked organization's members in WorkOS are compared with+ Switch's: the backstop for events missed (the Events API keeps 90 days) or+ from before the mirror ran. A Switch member with no WorkOS identity can't+ be compared, and is logged and left.++People are matched to WorkOS users by the sign-in identity linking them+(`oidc_identities`, the gateway's issuer and their WorkOS user id). Someone+WorkOS lists who has never signed in to Switch has no account to add, and is+brought in when they first sign in.++One process runs it (`ownership.py`, kind `workos_mirror`).+"""++from __future__ import annotations++import asyncio+import contextlib+import logging+from datetime import UTC, datetime+from typing import Any++from sqlalchemy import select+from sqlalchemy.dialects.postgresql import insert+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker++from switch_core.connections.broker import ServiceError+from switch_core.db.audit import (+ OrganizationAuditAction,+ record_organization_audit_event,+)+from switch_core.db.models import (+ DirectoryEventCursor,+ OidcIdentity,+ Organization,+ OrganizationMember,+)+from switch_core.db.session_scope import organization_session+from switch_core.db.tenant_lookup import directory_organizations+from switch_core.organization_access import AccessRevocation+from switch_core.organization_directory import DirectoryUnavailable+from switch_core.organization_mirror import apply_from_directory+from switch_core.ownership import Fence, OwnedKind, Ownership+from switch_core.tenant_context import no_tenant+from switch_core.workos_directory import WorkOSDirectory++logger = logging.getLogger(__name__)++POLL_INTERVAL_SECONDS = 30.0+CHECK_INTERVAL_SECONDS = 24 * 3600.0+CURSOR_SOURCE = "workos"+EVENT_TYPES = (+ "organization_membership.created",+ "organization_membership.updated",+ "organization_membership.deleted",+ "organization.deleted",+)+FROM_EVENT = "directory_event"+FROM_CHECK = "daily_check"++MIRROR_KIND = "workos_mirror"+MIRROR_KEY = "all"+++class WorkOSMirror:+ def __init__(+ self,+ session_factory: async_sessionmaker[AsyncSession],+ directory: WorkOSDirectory,+ revocation: AccessRevocation,+ issuer: str,+ ) -> None:+ self._session_factory = session_factory+ self._directory = directory+ self._revocation = revocation+ self._issuer = issuer++ async def poll_once(self) -> int:+ """Apply the events since the cursor, page by page, moving the cursor+ past each one applied. Returns how many were read."""+ cursor = await self._cursor()+ linked = await self._linked()+ read = 0+ after = cursor.after+ while True:+ events = await self._directory.events(+ EVENT_TYPES, after=after, range_start=cursor.started_at.isoformat()+ )+ if not events:+ return read+ for event in events:+ await self._apply_event(event, linked)+ after = event["id"]+ await self._advance(after)+ read += 1++ async def check_once(self) -> None:+ """Compare every linked organization's members with WorkOS's, and+ apply the difference. One organization failing doesn't stop the+ others."""+ for organization_id, workos_organization_id in (await self._linked()).items():+ try:+ await self._check(organization_id, workos_organization_id)+ except (DirectoryUnavailable, ServiceError) as error:+ logger.error(+ "Daily check of organization %s against WorkOS stopped: %s",+ organization_id,+ error,+ )++ async def run(self) -> None:+ """Poll the events every `POLL_INTERVAL_SECONDS`, and check every+ organization when starting and every `CHECK_INTERVAL_SECONDS`."""+ with no_tenant():+ since_check = CHECK_INTERVAL_SECONDS+ while True:+ if since_check >= CHECK_INTERVAL_SECONDS:+ since_check = 0.0+ await self._logged(self.check_once(), "daily check")+ await self._logged(self.poll_once(), "events")+ await asyncio.sleep(POLL_INTERVAL_SECONDS)+ since_check += POLL_INTERVAL_SECONDS++ @staticmethod+ async def _logged(work: Any, what: str) -> None:+ try:+ await work+ except (DirectoryUnavailable, ServiceError) as error:+ logger.error("WorkOS mirror %s: %s; trying again later", what, error)+ except Exception:+ logger.exception("WorkOS mirror %s failed; trying again later", what)++ async def _apply_event(self, event: dict[str, Any], linked: dict[str, str]) -> None:+ data = event["data"]+ following = {workos: own for own, workos in linked.items()}+ if event["event"] == "organization.deleted":+ organization_id = following.get(str(data.get("id")))+ if organization_id is not None:+ await self._organization_deleted(organization_id, data)+ return+ organization_id = following.get(str(data.get("organization_id")))+ subject = data.get("user_id")+ if organization_id is None or not isinstance(subject, str):+ return+ users = await self._users_of([subject])+ user_id = users.get(subject)+ if user_id is None:+ return+ wanted = await self._directory.membership_role(subject, linked[organization_id])+ await apply_from_directory(+ self._session_factory,+ self._revocation,+ organization_id=organization_id,+ user_id=user_id,+ wanted=wanted,+ source=FROM_EVENT,+ )++ async def _check(self, organization_id: str, workos_organization_id: str) -> None:+ in_workos = await self._directory.organization_members(workos_organization_id)+ async with organization_session(+ self._session_factory, organization_id+ ) as session:+ in_switch = set(+ await session.scalars(+ select(OrganizationMember.user_id).where(+ OrganizationMember.organization_id == organization_id+ )+ )+ )+ users = await self._users_of(list(in_workos))+ subjects = await self._subjects_of(list(in_switch))+ wanted: dict[str, str | None] = {+ user_id: in_workos[subject] for subject, user_id in users.items()+ }+ for user_id in in_switch:+ subject = subjects.get(user_id)+ if subject is None:+ logger.error(+ "User %s is in organization %s but has no WorkOS identity "+ "to compare; left as they are",+ user_id,+ organization_id,+ )+ continue+ wanted.setdefault(user_id, in_workos.get(subject))+ for user_id, role in sorted(wanted.items()):+ await apply_from_directory(+ self._session_factory,+ self._revocation,+ organization_id=organization_id,+ user_id=user_id,+ wanted=role,+ source=FROM_CHECK,+ )++ async def _organization_deleted(+ self, organization_id: str, data: dict[str, Any]+ ) -> None:+ """Deleting a linked organization in WorkOS removes nobody here: that+ is too much to take away on one event. It is reported for an operator+ to unlink or remove."""+ logger.error(+ "WorkOS organization %s, which organization %s follows, was deleted; "+ "its members are left as they are",+ data.get("id"),+ organization_id,+ )+ async with organization_session(+ self._session_factory, organization_id+ ) as session:+ await record_organization_audit_event(+ session,+ organization_id=organization_id,+ actor_user_id=None,+ action=OrganizationAuditAction.DIRECTORY_CONFLICT,+ target_type="organization",+ target_id=organization_id,+ details={+ "source": FROM_EVENT,+ "kind": "directory_organization_deleted",+ },+ )+ await session.commit()++ async def _linked(self) -> dict[str, str]:+ """Each linked organization's WorkOS organization, by its id."""+ linked: dict[str, str] = {}+ for organization_id in await directory_organizations(self._session_factory):+ async with organization_session(+ self._session_factory, organization_id+ ) as session:+ workos_organization_id = await session.scalar(+ select(Organization.workos_organization_id).where(+ Organization.id == organization_id+ )+ )+ if workos_organization_id is not None:+ linked[organization_id] = workos_organization_id+ return linked++ async def _users_of(self, subjects: list[str]) -> dict[str, str]:+ """The Switch account linked to each of WorkOS users `subjects` that+ has one."""+ if not subjects:+ return {}+ with no_tenant():+ async with self._session_factory() as session:+ rows = await session.execute(+ select(OidcIdentity.sub, OidcIdentity.user_id).where(+ OidcIdentity.iss == self._issuer,+ OidcIdentity.sub.in_(subjects),+ )+ )+ return {sub: user_id for sub, user_id in rows}++ async def _subjects_of(self, user_ids: list[str]) -> dict[str, str]:+ """The WorkOS user each of `user_ids` signs in as, where linked."""+ if not user_ids:+ return {}+ with no_tenant():+ async with self._session_factory() as session:+ rows = await session.execute(+ select(OidcIdentity.user_id, OidcIdentity.sub).where(+ OidcIdentity.iss == self._issuer,+ OidcIdentity.user_id.in_(user_ids),+ )+ )+ return {user_id: sub for user_id, sub in rows}++ async def _cursor(self) -> DirectoryEventCursor:+ """The cursor, made on the mirror's first run: events are read from+ then on, and the daily check covers what came before."""+ with no_tenant():+ async with self._session_factory() as session:+ await session.execute(+ insert(DirectoryEventCursor)+ .values(+ source=CURSOR_SOURCE, after=None, started_at=datetime.now(UTC)+ )+ .on_conflict_do_nothing(index_elements=["source"])+ )+ await session.commit()+ cursor = await session.get(DirectoryEventCursor, CURSOR_SOURCE)+ assert cursor is not None+ return cursor++ async def _advance(self, after: str) -> None:+ with no_tenant():+ async with self._session_factory() as session:+ cursor = await session.get(DirectoryEventCursor, CURSOR_SOURCE)+ assert cursor is not None+ cursor.after = after+ cursor.updated_at = datetime.now(UTC)+ await session.commit()+++def own_workos_mirror(ownership: Ownership, mirror: WorkOSMirror) -> OwnedKind:+ """Register the mirror as owned work, wanted by this process; it runs here+ only while it holds it."""+ running: dict[str, asyncio.Task[None]] = {}++ def launch(key: str) -> None:+ task = asyncio.create_task(mirror.run(), name="workos-mirror")+ running[key] = task+ task.add_done_callback(lambda done: ended(key, done))++ def ended(key: str, task: asyncio.Task[None]) -> None:+ if task.cancelled() or running.get(key) is not task:+ return+ logger.error(+ "WorkOS mirror stopped unexpectedly; restarting",+ exc_info=task.exception(),+ )+ launch(key)++ async def start(key: str, fence: Fence) -> None:+ launch(key)++ async def stop(key: str) -> None:+ task = running.pop(key, None)+ if task is None:+ return+ task.cancel()+ with contextlib.suppress(asyncio.CancelledError):+ await task++ kind = ownership.kind(MIRROR_KIND, start=start, stop=stop)+ kind.want(MIRROR_KEY)+ return kinddiff --git a/core/tests/switch_core/db/test_tenant_exemption_allowlist.py b/core/tests/switch_core/db/test_tenant_exemption_allowlist.pyindex ee076e104..7a21d18ec 100644--- a/core/tests/switch_core/db/test_tenant_exemption_allowlist.py+++ b/core/tests/switch_core/db/test_tenant_exemption_allowlist.py@@ -132,12 +132,16 @@ _ALLOWED_MODULES = {# the flow's tenant, and the flow is then read scoped to it, so a wrong# answer is a missing flow rather than a cross-tenant read."switch_core.connections.sign_in_flows",- # A WorkOS sign-in names the customers the person belongs to, by the- # customer's id; which Switch organization each one is has to be read- # before any is bound, and the membership is then written with that- # organization bound, so a wrong answer is a missing row rather than a- # write into another organization.+ # A WorkOS sign-in names the WorkOS organizations the person belongs to;+ # which Switch organizations follow them has to be read before any is+ # bound, and the membership is then written with that organization bound,+ # so a wrong answer is a missing row rather than a write into another+ # organization."switch_core.organization_mirror",+ # The WorkOS mirror acts for no tenant and no person: which organizations+ # are linked is read before any is bound, and each is then read and+ # written with it bound.+ "switch_core.workos_mirror",# `switch_core.transport.postgres` and `switch_core.clients.agent_consumer`# came off this list with `tenant_of_client`: both were built from a# `clients` row that already named the tenant, so they carry it instead of@@ -243,6 +247,10 @@ _RAW_SESSION_FACTORY_MODULES = {# their sign-in identities belong to no tenant. Its organization linking# binds the organization (`organization_session`)."switch_core.workos_linking",+ # The WorkOS mirror reads and moves its cursor (`directory_event_cursors`)+ # and reads `oidc_identities` to match people, both global+ # (`rls_ddl.GLOBAL_TABLES`); every organization it changes is bound.+ "switch_core.workos_mirror",## ── The OAuth client Core registers for itself at a vendor that offers# no other (`service_oauth_clients`, in `rls_ddl.GLOBAL_TABLES`). Thediff --git a/core/tests/switch_core/db/test_tenant_lookup.py b/core/tests/switch_core/db/test_tenant_lookup.pyindex f6b16052c..7cde7c7a9 100644--- a/core/tests/switch_core/db/test_tenant_lookup.py+++ b/core/tests/switch_core/db/test_tenant_lookup.py@@ -112,6 +112,7 @@ _ADDED_SINCE = {"tenants_bindable_by_user": "5c2e8f17a9d4","all_organization_ids": "10f6d02ac5bc","directory_organizations_of_user": "f2b8d6c1a4e9",+ "directory_organizations": "a7c3e9f15b20",}# A third kind of change, and the quietest: a lookup whose body a laterdiff --git a/core/tests/switch_core/db/test_tenant_schema_catalogue.py b/core/tests/switch_core/db/test_tenant_schema_catalogue.pyindex ce6639b04..0bc8a0a98 100644--- a/core/tests/switch_core/db/test_tenant_schema_catalogue.py+++ b/core/tests/switch_core/db/test_tenant_schema_catalogue.py@@ -121,6 +121,9 @@ class TestEveryTableIsScopedUnlessItIsNamedGlobal:# once: a process holds work for every tenant at once, and the row# names only which process runs which id."ownership_leases",+ # How far the directory mirror has read a directory's events: one+ # row per directory, whose events reach every organization.+ "directory_event_cursors",# Switch Trust's one guardrails policy for the whole deployment.# A policy that varied per tenant would be a new scoped table."trust_settings",diff --git a/core/tests/switch_core/test_organization_mirror.py b/core/tests/switch_core/test_organization_mirror.pyindex ed2e4a205..9b5fec800 100644--- a/core/tests/switch_core/test_organization_mirror.py+++ b/core/tests/switch_core/test_organization_mirror.py@@ -92,7 +92,7 @@ async def _added(session_factory, organization_id: str) -> list[OrganizationAudireturn await _recorded(session_factory,organization_id,- OrganizationAuditAction.MEMBER_ADDED_AT_SIGN_IN,+ OrganizationAuditAction.MEMBER_ADDED_FROM_DIRECTORY,)@@ -116,7 +116,7 @@ class TestBringingInMemberships:[event] = await _added(session_factory, organization_id)assert event.actor_user_id is Noneassert event.target_id == ada- assert event.details == {"role": "org_admin"}+ assert event.details == {"source": "sign_in", "role": "org_admin"}async def test_only_the_linked_organization_counts(self, rls_harness: RLSHarness@@ -165,12 +165,16 @@ class TestBringingInMemberships:[event] = await _recorded(session_factory,organization_id,- OrganizationAuditAction.MEMBER_PROMOTED_AT_SIGN_IN,+ OrganizationAuditAction.MEMBER_PROMOTED_FROM_DIRECTORY,)assert event.target_id == ada- assert event.details == {"from_role": "member", "role": "org_admin"}+ assert event.details == {+ "source": "sign_in",+ "from_role": "member",+ "role": "org_admin",+ }- async def test_a_demotion_is_left_and_logged_as_an_error(+ async def test_a_demotion_is_left_for_the_mirror_and_logged(self, rls_harness: RLSHarness, caplog: pytest.LogCaptureFixture) -> None:"""Taking a role away takes away what it reached, which is not a@@ -187,10 +191,10 @@ class TestBringingInMemberships:assert await _role(session_factory, organization_id, ada) == "org_admin"assert await _added(session_factory, organization_id) == []- [record] = [r for r in caplog.records if "keeps their role" in r.message]- assert record.levelname == "ERROR"+ [record] = [r for r in caplog.records if "mirror applies it" in r.message]+ assert record.levelname == "WARNING"- async def test_a_membership_no_longer_listed_is_left_and_logged_as_an_error(+ async def test_a_membership_no_longer_listed_is_left_for_the_mirror(self, rls_harness: RLSHarness, caplog: pytest.LogCaptureFixture) -> None:session_factory, restricted = rls_harness.owner, rls_harness.restricted@@ -201,5 +205,5 @@ class TestBringingInMemberships:await bring_in_memberships(restricted, ada, [])assert await _role(session_factory, organization_id, ada) == "org_admin"- [record] = [r for r in caplog.records if "no longer lists them" in r.message]- assert record.levelname == "ERROR"+ [record] = [r for r in caplog.records if "mirror applies it" in r.message]+ assert record.levelname == "WARNING"diff --git a/core/tests/switch_core/test_organization_mirror_apply.py b/core/tests/switch_core/test_organization_mirror_apply.pynew file mode 100644index 000000000..1fc064186--- /dev/null+++ b/core/tests/switch_core/test_organization_mirror_apply.py@@ -0,0 +1,386 @@+"""Changes made in the directory applied in Switch, taking away as well as+granting (`organization_mirror.apply_from_directory`), against Postgres, on+the restricted role row-level security applies to; `owner` only arranges+fixtures and reads results."""++from __future__ import annotations++import uuid+from datetime import timedelta+from types import SimpleNamespace++import pytest+from sqlalchemy import select+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker++from switch_core.connections.broker import ServiceBroker+from switch_core.connections.loader import CATALOG+from switch_core.connections.sign_in_flows import SignInFlows+from switch_core.db.audit import AuditAction, OrganizationAuditAction+from switch_core.db.models import (+ Agent,+ ApiKey,+ AuditEvent,+ Client,+ Organization,+ OrganizationAuditEvent,+ OrganizationMember,+ Tenant,+ TenantMember,+ User,+)+from switch_core.db.stores.agent_store import AgentStore+from switch_core.db.stores.api_key_store import ApiKeyStore+from switch_core.db.stores.service_connection_store import ServiceConnectionStore+from switch_core.db.stores.workspace_service_store import WorkspaceServiceStore+from switch_core.keys import Keyring+from switch_core.organization_access import AccessRevocation+from switch_core.organization_mirror import apply_from_directory+from tests.conftest import RLSHarness++pytestmark = pytest.mark.no_ambient_tenant++_KEYRING = Keyring.parse(+ "test:" + "organization-mirror-test-secret-" + "0" * 32, legacy_secret=None+)+++def _id(prefix: str) -> str:+ return f"{prefix}-{uuid.uuid4().hex[:8]}"+++def _revocation(session_factory: async_sessionmaker[AsyncSession]) -> AccessRevocation:+ return AccessRevocation(+ agent_store=AgentStore(),+ api_key_store=ApiKeyStore(),+ broker=ServiceBroker(+ session_factory=session_factory,+ keyring=_KEYRING,+ catalog=CATALOG,+ adapters={},+ disabled={},+ store=ServiceConnectionStore(),+ token_retention=timedelta(days=30),+ workspace_services=WorkspaceServiceStore(),+ workspace_gate=False,+ public_url=None,+ ),+ protocol=SimpleNamespace( # type: ignore[arg-type]+ api_key_cache=SimpleNamespace(invalidate=lambda key_hash: None)+ ),+ sign_in_flows=SignInFlows(session_factory, _KEYRING),+ )+++class _World:+ """A linked organization with Ada and Bea its `org_admin`s: `joined`, a+ workspace Ada is a member of (Bea owns it), and `reached`, one she+ stands in only as `org_admin`. Ada holds a personal key in each."""++ def __init__(self, organization_id: str, joined: str, reached: str) -> None:+ self.organization_id = organization_id+ self.joined = joined+ self.reached = reached+ self.ada = _id("ada")+ self.bea = _id("bea")++ @classmethod+ async def make(+ cls,+ owner: async_sessionmaker[AsyncSession],+ *,+ bea_role: str = "org_admin",+ ada_owns_joined: bool = False,+ ) -> _World:+ world = cls(_id("org"), _id("joined"), _id("reached"))+ async with owner() as session:+ session.add(+ Organization(+ id=world.organization_id,+ slug=world.organization_id,+ name="Acme",+ workos_organization_id=_id("org_workos"),+ )+ )+ for user_id in (world.ada, world.bea):+ session.add(+ User(+ id=user_id,+ name=user_id,+ email=f"{user_id}@example.invalid",+ role="user",+ )+ )+ await session.flush()+ for tenant_id in (world.joined, world.reached):+ session.add(+ Tenant(+ id=tenant_id,+ slug=tenant_id,+ name=tenant_id,+ organization_id=world.organization_id,+ )+ )+ session.add_all(+ [+ OrganizationMember(+ organization_id=world.organization_id,+ user_id=world.ada,+ role="org_admin",+ ),+ OrganizationMember(+ organization_id=world.organization_id,+ user_id=world.bea,+ role=bea_role,+ ),+ ]+ )+ await session.flush()+ session.add(+ TenantMember(+ tenant_id=world.joined,+ user_id=world.ada,+ role="owner" if ada_owns_joined else "member",+ )+ )+ if not ada_owns_joined:+ session.add(+ TenantMember(+ tenant_id=world.joined, user_id=world.bea, role="owner"+ )+ )+ for tenant_id in (world.joined, world.reached):+ session.add(_key(tenant_id, world.ada, "personal"))+ await session.commit()+ return world+++def _key(tenant_id: str, user_id: str, type_: str) -> ApiKey:+ return ApiKey(+ id=_id("key"),+ tenant_id=tenant_id,+ user_id=user_id,+ key_hash=uuid.uuid4().hex,+ encrypted_key="unused-in-tests",+ label="test-key",+ type=type_,+ )+++async def _agent(owner, tenant_id: str, owner_id: str) -> str:+ async with owner() as session:+ client = Client(+ tenant_id=tenant_id,+ transport_user_id=f"@bot-{uuid.uuid4().hex[:8]}:test",+ display_name="bot",+ type="agent",+ )+ key = _key(tenant_id, owner_id, "agent")+ session.add_all([client, key])+ await session.flush()+ agent = Agent(+ tenant_id=tenant_id,+ name=_id("agent"),+ description="test agent",+ agent_type="other",+ connector_type="claude-code",+ integration_profile={},+ client_id=client.id,+ api_key_id=key.id,+ owner_id=owner_id,+ )+ session.add(agent)+ await session.commit()+ return agent.id+++async def _role(owner, organization_id: str, user_id: str) -> str | None:+ async with owner() as session:+ member = await session.get(OrganizationMember, (organization_id, user_id))+ return None if member is None else member.role+++async def _keys(owner, tenant_id: str, user_id: str) -> list[str]:+ async with owner() as session:+ return list(+ await session.scalars(+ select(ApiKey.type).where(+ ApiKey.tenant_id == tenant_id, ApiKey.user_id == user_id+ )+ )+ )+++async def _organization_events(owner, organization_id: str) -> list[tuple[str, dict]]:+ async with owner() as session:+ rows = await session.scalars(+ select(OrganizationAuditEvent)+ .where(OrganizationAuditEvent.organization_id == organization_id)+ .order_by(OrganizationAuditEvent.occurred_at)+ )+ return [(event.action, event.details) for event in rows]+++async def _workspace_actions(owner, tenant_id: str) -> list[str]:+ async with owner() as session:+ return list(+ await session.scalars(+ select(AuditEvent.action).where(AuditEvent.tenant_id == tenant_id)+ )+ )+++async def _apply(rls_harness: RLSHarness, world: _World, wanted: str | None) -> None:+ await apply_from_directory(+ rls_harness.restricted,+ _revocation(rls_harness.restricted),+ organization_id=world.organization_id,+ user_id=world.ada,+ wanted=wanted,+ source="directory_event",+ )+++class TestTakingAway:+ async def test_a_removal_ends_every_access_through_the_organization(+ self, rls_harness: RLSHarness+ ) -> None:+ owner = rls_harness.owner+ world = await _World.make(owner)++ await _apply(rls_harness, world, None)++ assert await _role(owner, world.organization_id, world.ada) is None+ async with owner() as session:+ assert await session.get(TenantMember, (world.joined, world.ada)) is None+ assert await _keys(owner, world.joined, world.ada) == []+ assert await _keys(owner, world.reached, world.ada) == []+ assert await _workspace_actions(owner, world.joined) == [+ AuditAction.MEMBER_REMOVED+ ]+ assert await _workspace_actions(owner, world.reached) == [+ AuditAction.ORGANIZATION_ADMIN_ACCESS_ENDED+ ]+ [(action, details)] = await _organization_events(owner, world.organization_id)+ assert action == OrganizationAuditAction.MEMBER_REMOVED_FROM_DIRECTORY+ assert details == {+ "source": "directory_event",+ "from_role": "org_admin",+ "role": None,+ }++ async def test_a_demotion_ends_what_only_org_admin_reached(+ self, rls_harness: RLSHarness+ ) -> None:+ owner = rls_harness.owner+ world = await _World.make(owner)++ await _apply(rls_harness, world, "member")++ assert await _role(owner, world.organization_id, world.ada) == "member"+ assert await _keys(owner, world.joined, world.ada) == ["personal"]+ assert await _keys(owner, world.reached, world.ada) == []+ [(action, _)] = await _organization_events(owner, world.organization_id)+ assert action == OrganizationAuditAction.MEMBER_DEMOTED_FROM_DIRECTORY++ async def test_the_same_role_changes_nothing(self, rls_harness: RLSHarness) -> None:+ owner = rls_harness.owner+ world = await _World.make(owner)++ await _apply(rls_harness, world, "org_admin")++ assert await _organization_events(owner, world.organization_id) == []+ assert await _keys(owner, world.reached, world.ada) == ["personal"]+++class TestConflicts:+ """WorkOS is the record, so its change is applied even where it breaks a+ rule Switch keeps, and what it broke is recorded for the admins."""++ async def test_the_last_org_admin_is_removed_all_the_same(+ self, rls_harness: RLSHarness, caplog: pytest.LogCaptureFixture+ ) -> None:+ owner = rls_harness.owner+ world = await _World.make(owner, bea_role="member")++ await _apply(rls_harness, world, None)++ assert await _role(owner, world.organization_id, world.ada) is None+ events = await _organization_events(owner, world.organization_id)+ assert (+ OrganizationAuditAction.DIRECTORY_CONFLICT,+ {"source": "directory_event", "kind": "last_org_admin"},+ ) in events+ assert any(+ record.levelname == "ERROR" and "broke a rule" in record.message+ for record in caplog.records+ )++ async def test_agents_stay_with_the_keys_they_run_on(+ self, rls_harness: RLSHarness+ ) -> None:+ owner = rls_harness.owner+ world = await _World.make(owner)+ agent_id = await _agent(owner, world.joined, world.ada)++ await _apply(rls_harness, world, None)++ async with owner() as session:+ agent = await session.get(Agent, agent_id)+ assert agent is not None and agent.owner_id == world.ada+ assert await _keys(owner, world.joined, world.ada) == ["agent"]+ assert (+ OrganizationAuditAction.DIRECTORY_CONFLICT,+ {+ "source": "directory_event",+ "kind": "agents_left_with_owner",+ "workspace": world.joined,+ },+ ) in await _organization_events(owner, world.organization_id)++ async def test_a_workspace_left_with_no_owner_is_recorded(+ self, rls_harness: RLSHarness+ ) -> None:+ owner = rls_harness.owner+ world = await _World.make(owner, ada_owns_joined=True)++ await _apply(rls_harness, world, None)++ async with owner() as session:+ assert await session.get(TenantMember, (world.joined, world.ada)) is None+ assert (+ OrganizationAuditAction.DIRECTORY_CONFLICT,+ {+ "source": "directory_event",+ "kind": "workspace_left_without_owner",+ "workspace": world.joined,+ },+ ) in await _organization_events(owner, world.organization_id)+++class TestGranting:+ async def test_someone_listed_is_added_with_where_it_came_from(+ self, rls_harness: RLSHarness+ ) -> None:+ owner = rls_harness.owner+ world = await _World.make(owner)+ cy = _id("cy")+ async with owner() as session:+ session.add(+ User(id=cy, name=cy, email=f"{cy}@example.invalid", role="user")+ )+ await session.commit()++ await apply_from_directory(+ rls_harness.restricted,+ _revocation(rls_harness.restricted),+ organization_id=world.organization_id,+ user_id=cy,+ wanted="member",+ source="daily_check",+ )++ assert await _role(owner, world.organization_id, cy) == "member"+ [(action, details)] = await _organization_events(owner, world.organization_id)+ assert action == OrganizationAuditAction.MEMBER_ADDED_FROM_DIRECTORY+ assert details == {"source": "daily_check", "role": "member"}diff --git a/core/tests/switch_core/test_workos_directory.py b/core/tests/switch_core/test_workos_directory.pyindex 0dbaebcba..17fd8978a 100644--- a/core/tests/switch_core/test_workos_directory.py+++ b/core/tests/switch_core/test_workos_directory.py@@ -226,3 +226,102 @@ class TestChangesFromSwitch:workos_organization_id=None, customer_id=None)await directory.discard_organization(created)+++class TestWhatTheMirrorReads:+ async def test_events_are_asked_for_by_type_from_the_cursor_oldest_first(+ self,+ ) -> None:+ seen: list[httpx.Request] = []++ def handler(request: httpx.Request) -> httpx.Response:+ seen.append(request)+ return httpx.Response(+ 200,+ json={+ "data": [+ {+ "id": "event_1",+ "event": "organization_membership.created",+ "data": {"user_id": "user_A", "organization_id": "org_a"},+ }+ ],+ "list_metadata": {"after": None},+ },+ )++ directory = _directory(handler)+ types = ("organization_membership.created", "organization.deleted")++ first = await directory.events(+ types, after=None, range_start="2026-11-02T00:00:00+00:00"+ )+ await directory.events(types, after="event_1", range_start="unused")++ assert [event["id"] for event in first] == ["event_1"]+ assert seen[0].url.path == "/events"+ assert seen[0].url.params.get_list("events") == list(types)+ assert seen[0].url.params["order"] == "asc"+ assert seen[0].url.params["range_start"] == "2026-11-02T00:00:00+00:00"+ assert "after" not in seen[0].url.params+ assert seen[1].url.params["after"] == "event_1"+ assert "range_start" not in seen[1].url.params++ async def test_an_event_switch_cannot_read_is_unavailable(self) -> None:+ directory = _directory(+ lambda request: httpx.Response(200, json={"data": [{"id": "event_1"}]})+ )++ with pytest.raises(DirectoryUnavailable):+ await directory.events(+ ("organization.deleted",), after=None, range_start="x"+ )++ async def test_one_membership_is_read_as_it_is_now(self) -> None:+ def handler(request: httpx.Request) -> httpx.Response:+ assert request.url.params["user_id"] == "user_A"+ assert request.url.params["organization_id"] == "org_a"+ assert request.url.params["statuses"] == "active"+ return httpx.Response(+ 200,+ json={+ "data": [+ {"organization_id": "org_a", "role": {"slug": "org_admin"}}+ ]+ },+ )++ assert (+ await _directory(handler).membership_role("user_A", "org_a") == "org_admin"+ )++ async def test_no_membership_is_none(self) -> None:+ directory = _directory(lambda request: httpx.Response(200, json={"data": []}))++ assert await directory.membership_role("user_A", "org_a") is None++ async def test_an_organizations_members_are_read_by_user(self) -> None:+ def handler(request: httpx.Request) -> httpx.Response:+ assert request.url.params["organization_id"] == "org_a"+ return httpx.Response(+ 200,+ json={+ "data": [+ {+ "user_id": "user_A",+ "organization_id": "org_a",+ "role": {"slug": "org_admin"},+ },+ {+ "user_id": "user_B",+ "organization_id": "org_a",+ "role": {"slug": "editor"},+ },+ ]+ },+ )++ assert await _directory(handler).organization_members("org_a") == {+ "user_A": "org_admin",+ "user_B": "member",+ }diff --git a/core/tests/switch_core/test_workos_mirror.py b/core/tests/switch_core/test_workos_mirror.pynew file mode 100644index 000000000..23df2534e--- /dev/null+++ b/core/tests/switch_core/test_workos_mirror.py@@ -0,0 +1,309 @@+"""The WorkOS mirror: events and the daily check applied to linked+organizations, against Postgres on the restricted role and a stand-in for+WorkOS; `owner` only arranges fixtures and reads results."""++from __future__ import annotations++import uuid+from typing import Any++import pytest+from sqlalchemy import select++from switch_core.db.audit import OrganizationAuditAction+from switch_core.db.models import (+ DirectoryEventCursor,+ OidcIdentity,+ Organization,+ OrganizationAuditEvent,+ OrganizationMember,+ User,+)+from switch_core.organization_directory import DirectoryUnavailable+from switch_core.workos_mirror import EVENT_TYPES, WorkOSMirror+from tests.conftest import RLSHarness+from tests.switch_core.test_organization_mirror_apply import _revocation++pytestmark = pytest.mark.no_ambient_tenant++ISSUER = "https://auth.example.test"+++def _id(prefix: str) -> str:+ return f"{prefix}-{uuid.uuid4().hex[:8]}"+++class _WorkOS:+ """Memberships by (WorkOS user, WorkOS organization) -> Switch role, and+ the events that changed them, served a page of `page` at a time."""++ def __init__(self, page: int = 100) -> None:+ self.memberships: dict[tuple[str, str], str] = {}+ self.log: list[dict[str, Any]] = []+ self.page = page+ self.asked: list[dict[str, Any]] = []+ self.fail_role_for: str | None = None++ def change(self, subject: str, organization: str, role: str | None) -> None:+ if role is None:+ self.memberships.pop((subject, organization), None)+ kind = "deleted"+ else:+ kind = (+ "created"+ if (subject, organization) not in self.memberships+ else "updated"+ )+ self.memberships[(subject, organization)] = role+ self.log.append(+ {+ "id": f"event_{len(self.log):04d}",+ "event": f"organization_membership.{kind}",+ "data": {"user_id": subject, "organization_id": organization},+ }+ )++ async def events(+ self, types: tuple[str, ...], *, after: str | None, range_start: str+ ) -> list[dict[str, Any]]:+ assert types == EVENT_TYPES+ self.asked.append({"after": after, "range_start": range_start})+ start = 0+ if after is not None:+ start = next(i for i, e in enumerate(self.log) if e["id"] == after) + 1+ return self.log[start : start + self.page]++ async def membership_role(self, subject: str, organization: str) -> str | None:+ if subject == self.fail_role_for:+ raise DirectoryUnavailable("WorkOS answered 500.")+ return self.memberships.get((subject, organization))++ async def organization_members(self, organization: str) -> dict[str, str]:+ return {+ subject: role+ for (subject, org), role in self.memberships.items()+ if org == organization+ }+++async def _linked(owner) -> tuple[str, str]:+ organization_id, workos_id = _id("org"), _id("org_workos")+ async with owner() as session:+ session.add(+ Organization(+ id=organization_id,+ slug=organization_id,+ name="Acme",+ workos_organization_id=workos_id,+ )+ )+ await session.commit()+ return organization_id, workos_id+++async def _person(owner, subject: str | None) -> str:+ user_id = _id("user")+ async with owner() as session:+ session.add(+ User(+ id=user_id,+ name=user_id,+ email=f"{user_id}@example.invalid",+ role="user",+ )+ )+ await session.flush()+ if subject is not None:+ session.add(OidcIdentity(user_id=user_id, iss=ISSUER, sub=subject))+ await session.commit()+ return user_id+++async def _member(owner, organization_id: str, user_id: str, role: str) -> None:+ async with owner() as session:+ session.add(+ OrganizationMember(+ organization_id=organization_id, user_id=user_id, role=role+ )+ )+ await session.commit()+++async def _roles(owner, organization_id: str) -> dict[str, str]:+ async with owner() as session:+ rows = await session.execute(+ select(OrganizationMember.user_id, OrganizationMember.role).where(+ OrganizationMember.organization_id == organization_id+ )+ )+ return {user_id: role for user_id, role in rows}+++def _mirror(rls_harness: RLSHarness, workos: _WorkOS) -> WorkOSMirror:+ return WorkOSMirror(+ rls_harness.restricted,+ workos, # type: ignore[arg-type]+ _revocation(rls_harness.restricted),+ ISSUER,+ )+++class TestEvents:+ async def test_the_first_poll_starts_from_now_and_later_ones_from_the_cursor(+ self, rls_harness: RLSHarness+ ) -> None:+ owner = rls_harness.owner+ organization_id, workos_id = await _linked(owner)+ ada = await _person(owner, "user_ADA")+ workos = _WorkOS(page=1)+ mirror = _mirror(rls_harness, workos)++ assert await mirror.poll_once() == 0+ workos.change("user_ADA", workos_id, "member")+ workos.change("user_ADA", workos_id, "org_admin")+ assert await mirror.poll_once() == 2++ assert await _roles(owner, organization_id) == {ada: "org_admin"}+ assert workos.asked[0]["after"] is None+ assert workos.asked[-1]["after"] == "event_0001"+ async with owner() as session:+ cursor = await session.get(DirectoryEventCursor, "workos")+ assert cursor is not None and cursor.after == "event_0001"+ assert await mirror.poll_once() == 0++ async def test_an_event_reads_the_membership_as_it_is_now(+ self, rls_harness: RLSHarness+ ) -> None:+ """Read after the membership has gone, a `created` event removes+ nothing and adds nothing."""+ owner = rls_harness.owner+ organization_id, workos_id = await _linked(owner)+ await _person(owner, "user_ADA")+ workos = _WorkOS()+ mirror = _mirror(rls_harness, workos)+ await mirror.poll_once()+ workos.change("user_ADA", workos_id, "member")+ workos.change("user_ADA", workos_id, None)++ await mirror.poll_once()++ assert await _roles(owner, organization_id) == {}++ async def test_a_removal_is_applied(self, rls_harness: RLSHarness) -> None:+ owner = rls_harness.owner+ organization_id, workos_id = await _linked(owner)+ ada = await _person(owner, "user_ADA")+ bea = await _person(owner, "user_BEA")+ await _member(owner, organization_id, ada, "member")+ await _member(owner, organization_id, bea, "org_admin")+ workos = _WorkOS()+ workos.memberships[("user_ADA", workos_id)] = "member"+ mirror = _mirror(rls_harness, workos)+ await mirror.poll_once()+ workos.change("user_ADA", workos_id, None)++ await mirror.poll_once()++ assert await _roles(owner, organization_id) == {bea: "org_admin"}++ async def test_events_for_other_organizations_and_strangers_are_skipped(+ self, rls_harness: RLSHarness+ ) -> None:+ owner = rls_harness.owner+ organization_id, workos_id = await _linked(owner)+ await _person(owner, "user_ADA")+ workos = _WorkOS()+ mirror = _mirror(rls_harness, workos)+ await mirror.poll_once()+ workos.change("user_ADA", _id("org_elsewhere"), "org_admin")+ workos.change("user_NOBODY", workos_id, "org_admin")++ assert await mirror.poll_once() == 2++ assert await _roles(owner, organization_id) == {}++ async def test_an_event_that_fails_is_read_again_next_time(+ self, rls_harness: RLSHarness+ ) -> None:+ owner = rls_harness.owner+ organization_id, workos_id = await _linked(owner)+ ada = await _person(owner, "user_ADA")+ workos = _WorkOS()+ mirror = _mirror(rls_harness, workos)+ await mirror.poll_once()+ workos.change("user_ADA", workos_id, "member")+ workos.fail_role_for = "user_ADA"++ with pytest.raises(DirectoryUnavailable):+ await mirror.poll_once()+ workos.fail_role_for = None+ assert await mirror.poll_once() == 1++ assert await _roles(owner, organization_id) == {ada: "member"}++ async def test_a_deleted_organization_removes_nobody_and_is_reported(+ self, rls_harness: RLSHarness+ ) -> None:+ owner = rls_harness.owner+ organization_id, workos_id = await _linked(owner)+ ada = await _person(owner, "user_ADA")+ await _member(owner, organization_id, ada, "org_admin")+ workos = _WorkOS()+ mirror = _mirror(rls_harness, workos)+ await mirror.poll_once()+ workos.log.append(+ {+ "id": "event_9999",+ "event": "organization.deleted",+ "data": {"id": workos_id},+ }+ )++ await mirror.poll_once()++ assert await _roles(owner, organization_id) == {ada: "org_admin"}+ async with owner() as session:+ [event] = await session.scalars(+ select(OrganizationAuditEvent).where(+ OrganizationAuditEvent.organization_id == organization_id+ )+ )+ assert event.action == OrganizationAuditAction.DIRECTORY_CONFLICT+ assert event.details["kind"] == "directory_organization_deleted"+++class TestDailyCheck:+ async def test_brings_every_linked_organization_in_line(+ self, rls_harness: RLSHarness, caplog: pytest.LogCaptureFixture+ ) -> None:+ owner = rls_harness.owner+ organization_id, workos_id = await _linked(owner)+ ada = await _person(owner, "user_ADA") # listed, not in Switch+ bea = await _person(owner, "user_BEA") # in Switch, no longer listed+ cy = await _person(owner, "user_CY") # org_admin in Switch, member there+ dee = await _person(owner, None) # in Switch, no WorkOS identity+ eve = await _person(owner, "user_EVE") # stays org_admin+ for user_id, role in (+ (bea, "member"),+ (cy, "org_admin"),+ (dee, "member"),+ (eve, "org_admin"),+ ):+ await _member(owner, organization_id, user_id, role)+ workos = _WorkOS()+ workos.memberships = {+ ("user_ADA", workos_id): "member",+ ("user_CY", workos_id): "member",+ ("user_EVE", workos_id): "org_admin",+ ("user_STRANGER", workos_id): "member",+ }++ await _mirror(rls_harness, workos).check_once()++ assert await _roles(owner, organization_id) == {+ ada: "member",+ cy: "member",+ dee: "member",+ eve: "org_admin",+ }+ assert f"User {dee} is in organization" in caplog.textdiff --git a/docs/old/workos-directory.md b/docs/old/workos-directory.mdindex f55514230..c00320fbf 100644--- a/docs/old/workos-directory.md+++ b/docs/old/workos-directory.md@@ -55,19 +55,53 @@ linked organization:Both are recorded in the organization's audit log. If WorkOS cannot be read,or Switch's rows cannot be written, the sign-in goes ahead with Switch's rows-as they were, and the failure is logged.+as they were, and the failure is logged. A sign-in never takes anything away;+the mirror does.++## The mirror++One switch-core process follows WorkOS's Events API, every 30 seconds, for+changes to a linked organization's memberships, and applies them. It starts+from when it first runs; once a day, and whenever it starts, it also compares+every linked organization's members with WorkOS and applies the difference,+which catches up anything from before it ran or missed since. It keeps its+place in the database, so a restart resumes where it stopped.++For each person it matches (by the WorkOS user their account signs in as):++- **Added or made `org_admin` in WorkOS:** added, or made Org Admin, as at+ sign-in.+- **No longer `org_admin`:** made a Member. What they held in workspaces they+ stood in only as Org Admin (personal API keys, service connections,+ sign-ins under way) is removed.+- **Removed:** removed from the organization and from each of its workspaces,+ with their personal API keys, service connections and sign-ins under way+ there.++Each change is recorded in the organization's audit log, and each workspace+change in that workspace's. WorkOS is the record, so a change is applied even+when it breaks a rule Switch otherwise keeps. It is then logged at `error` and+recorded in the organization's audit log as "WorkOS change broke a Switch+rule", naming what:++- the organization's last Org Admin was removed or demoted;+- agents the person owns were left in place, still running on their keys,+ owned by someone who can no longer act there, for an admin to delete;+- a workspace was left with no owner (its organization's Org Admins still+ administer it).++Someone WorkOS lists who has never signed in to Switch is added when they+first do. A member with no WorkOS identity (an account nobody linked) can't be+compared, and is logged and left. Deleting a linked organization in WorkOS+removes nobody: it is logged and recorded for an operator to deal with.## Not done yet-- **Removals and demotions are not applied.** If someone is removed from the- linked WorkOS organization, or is no longer its `org_admin`, Switch keeps- their membership and role, and with them their access. Each sign-in by that- person logs it at `error`. Until this is built, an Org Admin ends such- access in Switch as well, from the organization's Members page: removing a- member and taking Org Admin away are allowed on a linked organization, and- change only Switch. Whatever WorkOS still grants comes back at the person's- next sign-in.- **Granting from Switch is refused** for a linked organization: adding amember (from the dashboard or by invitation) or making one Org Admin. Make- those changes in WorkOS.+ those changes in WorkOS. Removing a member or taking Org Admin away from+ Switch is allowed and changes only Switch; whatever WorkOS still grants+ comes back at the next sign-in or check.+- The organization's admins see conflicts only in its audit log, not yet as a+ notice on the dashboard.- A new organization created on the server is Switch's own until it is linked.diff --git a/gateway/src/pages/organization/OrganizationAuditSection.tsx b/gateway/src/pages/organization/OrganizationAuditSection.tsxindex 8f082f5f3..e5041c6fe 100644--- a/gateway/src/pages/organization/OrganizationAuditSection.tsx+++ b/gateway/src/pages/organization/OrganizationAuditSection.tsx@@ -18,8 +18,11 @@ const ACTION_LABELS: Record<string, string> = {"organization.connector_app_removed": "Connector app removed","organization.service_set": "Service switched","organization.service_cleared": "Service left to workspaces",- "organization.member_added_at_sign_in": "Member added at sign-in",- "organization.member_promoted_at_sign_in": "Member made Org Admin at sign-in",+ "organization.member_added_from_directory": "Member added from WorkOS",+ "organization.member_promoted_from_directory": "Member made Org Admin from WorkOS",+ "organization.member_demoted_from_directory": "Org Admin made Member from WorkOS",+ "organization.member_removed_from_directory": "Member removed from WorkOS",+ "organization.directory_conflict": "WorkOS change broke a Switch rule","organization.linked_to_directory": "Linked to WorkOS",};