Skip to content

Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.

feat(organizations): mirror WorkOS membership changes into Switch

Petr Bauch0cc555049931
diff --git a/CLAUDE.md b/CLAUDE.md
index fa270db9e..3c008a09b 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -191,8 +191,8 @@ Tests live in `core/tests/switch_core/` mirroring the module structure. Uses pyt
OIDC browser sign-in (variables, redirect URI, WorkOS Connect setup)
- `docs/old/workos-directory.md` — keeping organizations in WorkOS
(`ORGANIZATION_DIRECTORY=workos`): what it needs, linking organizations and
- existing accounts with `switch-link-workos`, what a sign-in brings in, and
- what is not done yet
+ existing accounts with `switch-link-workos`, what a sign-in and the mirror
+ bring in or take away, and what is not done yet
- `docs/old/google-workspace-setup.md` — offering Google Workspace on a
self-hosted server: registering the Google app (Internal or External), its
redirect URI, saving it under Connectors or in
diff --git a/core/switch_core/db/audit.py b/core/switch_core/db/audit.py
index 6e02b2d8f..7e194faa7 100644
--- a/core/switch_core/db/audit.py
+++ b/core/switch_core/db/audit.py
@@ -83,9 +83,16 @@ class OrganizationAuditAction(StrEnum):
CONNECTOR_APP_REMOVED = "organization.connector_app_removed"
SERVICE_SET = "organization.service_set"
SERVICE_CLEARED = "organization.service_cleared"
- # Brought in from a directory kept outside Switch when they signed in.
- MEMBER_ADDED_AT_SIGN_IN = "organization.member_added_at_sign_in"
- MEMBER_PROMOTED_AT_SIGN_IN = "organization.member_promoted_at_sign_in"
+ # Changes made in a directory kept outside Switch, brought in at sign-in,
+ # from its events or by the daily check (`details.source`).
+ MEMBER_ADDED_FROM_DIRECTORY = "organization.member_added_from_directory"
+ MEMBER_PROMOTED_FROM_DIRECTORY = "organization.member_promoted_from_directory"
+ MEMBER_DEMOTED_FROM_DIRECTORY = "organization.member_demoted_from_directory"
+ MEMBER_REMOVED_FROM_DIRECTORY = "organization.member_removed_from_directory"
+ # A change brought in from the directory broke a rule Switch keeps itself
+ # (the last Org Admin, agents left with an owner who can't act); applied
+ # anyway, and recorded for the organization's admins (`details.kind`).
+ DIRECTORY_CONFLICT = "organization.directory_conflict"
# An operator made the organization follow one in the directory.
LINKED_TO_DIRECTORY = "organization.linked_to_directory"
diff --git a/core/switch_core/db/models.py b/core/switch_core/db/models.py
index 565082d20..94a944053 100644
--- a/core/switch_core/db/models.py
+++ b/core/switch_core/db/models.py
@@ -2640,6 +2640,30 @@ class OwnershipLease(Base):
)
+class DirectoryEventCursor(Base):
+ """How far the directory mirror has read a directory's events
+ (`workos_mirror.py`). One row per directory (`source`, `"workos"`).
+
+ `after` is the last event applied, null until the first; until then the
+ events are read from `started_at`, when the mirror first ran. Applying
+ an event re-reads the membership it names, so reading one twice changes
+ nothing, and the row needs no fence against a holder taken over.
+
+ Not tenant-scoped: one directory's events reach every organization.
+ """
+
+ __tablename__ = "directory_event_cursors"
+
+ source: Mapped[str] = mapped_column(Text, primary_key=True)
+ after: Mapped[str | None] = mapped_column(Text, nullable=True)
+ started_at: Mapped[datetime] = mapped_column(
+ DateTime(timezone=True), nullable=False
+ )
+ updated_at: Mapped[datetime] = mapped_column(
+ DateTime(timezone=True), server_default=func.now(), nullable=False
+ )
+
+
# ── Trust settings ───────────────────────────────────────────────────────────
diff --git a/core/switch_core/db/rls_ddl.py b/core/switch_core/db/rls_ddl.py
index bce8cabf0..b2f551470 100644
--- a/core/switch_core/db/rls_ddl.py
+++ b/core/switch_core/db/rls_ddl.py
@@ -195,6 +195,7 @@ GLOBAL_TABLES = frozenset(
"telemetry_snapshot_watermark",
"switch_core_processes",
"ownership_leases",
+ "directory_event_cursors",
"trust_settings",
"service_oauth_clients",
"schema_expand_only_revisions",
diff --git a/core/switch_core/db/tenant_lookup.py b/core/switch_core/db/tenant_lookup.py
index f4c084630..a46a39478 100644
--- a/core/switch_core/db/tenant_lookup.py
+++ b/core/switch_core/db/tenant_lookup.py
@@ -1,4 +1,4 @@
-"""The whole exemption from row-level security, written out as fifteen functions.
+"""The whole exemption from row-level security, written out as sixteen functions.
Row-level security is enforced by `require_tenant_id()` (`db/rls_ddl.py`),
which raises when no tenant is bound. That is the property everything else
@@ -183,6 +183,12 @@ organization, and which linked organizations a person belongs to; the
gateway asks it only about the signed-in person and the organizations WorkOS
says they belong to.
+**`directory_organizations` is the sixteenth**: every organization that
+follows one in the WorkOS directory. The directory mirror acts for no tenant
+and no person: it reads the directory's changes and has to know which
+organizations they may reach before it binds any of them. It discloses which
+organizations are linked; the mirror is its only caller.
+
Why not the obvious alternatives is argued in
`docs/old/multi-tenancy-phase1-db.md`, "The bootstrap problem"; the short
version is that returning rows instead of tenant ids would put a second copy
@@ -282,7 +288,7 @@ class TenantLookup:
return f"{self.name}({', '.join('text' for _ in self.arguments)})"
-# The fifteen of them. Ordered as the three shapes above: enumeration, then
+# The sixteen of them. Ordered as the three shapes above: enumeration, then
# credential resolution, then deriving a tenant from an identifier in hand.
TENANT_LOOKUPS: tuple[TenantLookup, ...] = (
TenantLookup(
@@ -478,6 +484,18 @@ TENANT_LOOKUPS: tuple[TenantLookup, ...] = (
"to bring the person's organization memberships up to date."
),
),
+ TenantLookup(
+ name="directory_organizations",
+ arguments=(),
+ query=(
+ "SELECT id FROM organizations "
+ "WHERE workos_organization_id IS NOT NULL ORDER BY id"
+ ),
+ purpose=(
+ "Every organization that follows one in the WorkOS directory, for "
+ "the directory mirror to read each one's link with it bound."
+ ),
+ ),
TenantLookup(
name="tenant_of_messaging_install",
arguments=("platform", "external_workspace_id"),
@@ -551,7 +569,7 @@ def attach_tenant_lookups(metadata: MetaData) -> None:
# ── Calling them ──────────────────────────────────────────────────────────────
# One `text()` per lookup, written out rather than assembled from `lookup.name`
-# at call time: the fifteen names are fixed and known here, so there is nothing
+# at call time: the sixteen names are fixed and known here, so there is nothing
# for a call site to build. Each bind is named after the lookup's own argument,
# which is what lets `_call` zip them positionally against the dataclass and
# fail loudly on a mismatch rather than binding the workspace to the platform.
@@ -604,6 +622,9 @@ _LOOKUP_STATEMENTS: dict[str, TextClause] = {
"tenant_of_sign_in_flow": text(
"SELECT tenant_id FROM tenant_of_sign_in_flow(:state) AS tenant_id"
),
+ "directory_organizations": text(
+ "SELECT tenant_id FROM directory_organizations() AS tenant_id"
+ ),
"directory_organizations_of_user": text(
"SELECT tenant_id FROM directory_organizations_of_user("
":user_id, :workos_organization_ids) AS tenant_id"
@@ -776,3 +797,11 @@ async def directory_organizations_of_user(
user_id,
json.dumps(sorted(workos_organization_ids)),
)
+
+
+async def directory_organizations(
+ session_factory: async_sessionmaker[AsyncSession],
+) -> list[str]:
+ return await _call(
+ session_factory, TENANT_LOOKUPS_BY_NAME["directory_organizations"]
+ )
diff --git a/core/switch_core/gateway/organizations.py b/core/switch_core/gateway/organizations.py
index 668e21dff..0cbd02fd9 100644
--- a/core/switch_core/gateway/organizations.py
+++ b/core/switch_core/gateway/organizations.py
@@ -35,12 +35,9 @@ from switch_core.connections.broker import (
ServiceError,
get_service_broker,
)
-from switch_core.connections.sign_in_flows import SignInFlows
from switch_core.db.audit import (
- AuditAction,
OrganizationAuditAction,
list_organization_audit_events,
- record_audit_event,
record_organization_audit_event,
)
from switch_core.db.models import (
@@ -50,7 +47,6 @@ from switch_core.db.models import (
User,
require_tenant_id,
)
-from switch_core.db.session_scope import tenant_session
from switch_core.db.stores.agent_store import AgentStore
from switch_core.db.stores.api_key_store import ApiKeyStore
from switch_core.db.stores.organization_store import OrganizationStore
@@ -90,10 +86,6 @@ from switch_core.gateway.schemas import (
WorkspaceCreateRequest,
)
from switch_core.gateway.service_connections import service_refusal
-from switch_core.gateway.workspace_access import (
- finish_revocation,
- revoke_workspace_access,
-)
from switch_core.gateway.workspace_creation import (
NOTHING_CHANGED,
create_organization_for,
@@ -101,6 +93,11 @@ from switch_core.gateway.workspace_creation import (
directory_failed,
lock_workspace_allowance,
)
+from switch_core.organization_access import (
+ AccessRevocation,
+ end_access_through_organization,
+ workspaces_where_they_own_agents,
+)
from switch_core.organization_directory import DirectoryError, OrganizationDirectory
router = APIRouter()
@@ -451,69 +448,39 @@ async def _end_access_through_organization(
organization_id: str,
actor_id: str,
reason: str,
- agent_store: AgentStore,
- api_key_store: ApiKeyStore,
- broker: ServiceBroker,
- protocol: AgentCore,
- sign_in_flows: SignInFlows,
+ revocation: AccessRevocation,
) -> str | None:
"""Take away what `user_id` holds in the workspaces they stand in only as
- the organization's `org_admin`, before they stop being one.
+ the organization's `org_admin`, before they stop being one
+ (`organization_access.end_access_through_organization`).
Refuses, 409 naming the workspaces, while they own agents in any of them,
- as removing a workspace member does. Otherwise removes their keys,
- service connections and sign-ins under way in each, one workspace at a
- time, each recorded in that workspace's audit log. Raises `ServiceError`
- when a connection is busy. Returns a warning naming what could not be
- revoked at a vendor, or None.
+ as removing a workspace member does.
"""
reached = await workspaces_reached_through_organization(
session_factory, user_id, organization_id
)
- with_agents = []
- for tenant in reached:
- async with tenant_session(session_factory, tenant.id) as session:
- if await agent_store.get_by_owner(session, user_id):
- with_agents.append(tenant.name)
+ with_agents = await workspaces_where_they_own_agents(
+ session_factory, revocation.agent_store, user_id, reached
+ )
if with_agents:
raise HTTPException(
status_code=409,
detail=(
"They own agents in workspaces they're in only as an "
"organization admin. Delete those agents first: "
- f"{', '.join(sorted(with_agents))}"
+ f"{', '.join(sorted(tenant.name for tenant in with_agents))}"
),
)
- warnings = []
- for tenant in reached:
- async with tenant_session(session_factory, tenant.id) as session:
- revoked = await revoke_workspace_access(
- session,
- user_id,
- api_key_store=api_key_store,
- broker=broker,
- sign_in_flows=sign_in_flows,
- )
- await record_audit_event(
- session,
- tenant_id=tenant.id,
- actor_user_id=actor_id,
- action=AuditAction.ORGANIZATION_ADMIN_ACCESS_ENDED,
- target_type="user",
- target_id=user_id,
- details={
- "reason": reason,
- "api_keys_deleted": len(revoked.key_hashes),
- "service_connections_deleted": len(revoked.removed_services),
- },
- )
- await session.commit()
- warning = await finish_revocation(
- session, user_id, revoked, protocol=protocol, broker=broker
- )
- if warning:
- warnings.append(f"{tenant.name}: {warning}")
- return " ".join(warnings) or None
+ return await end_access_through_organization(
+ session_factory,
+ revocation,
+ user_id=user_id,
+ reached=reached,
+ actor_id=actor_id,
+ details={"reason": reason},
+ keep_agent_keys=False,
+ )
@router.patch("/organizations/current/members/{user_id}", response_model=None)
@@ -576,11 +543,13 @@ async def update_organization_member_role(
organization_id=organization_id,
actor_id=user.id,
reason="demoted",
- agent_store=agent_store,
- api_key_store=api_key_store,
- broker=broker,
- protocol=protocol,
- sign_in_flows=request.app.state.sign_in_flows,
+ revocation=AccessRevocation(
+ agent_store=agent_store,
+ api_key_store=api_key_store,
+ broker=broker,
+ protocol=protocol,
+ sign_in_flows=request.app.state.sign_in_flows,
+ ),
)
except ServiceError as error:
return service_refusal(error)
@@ -692,11 +661,13 @@ async def remove_organization_member(
organization_id=organization_id,
actor_id=user.id,
reason="removed",
- agent_store=agent_store,
- api_key_store=api_key_store,
- broker=broker,
- protocol=protocol,
- sign_in_flows=request.app.state.sign_in_flows,
+ revocation=AccessRevocation(
+ agent_store=agent_store,
+ api_key_store=api_key_store,
+ broker=broker,
+ protocol=protocol,
+ sign_in_flows=request.app.state.sign_in_flows,
+ ),
)
except ServiceError as error:
return service_refusal(error)
diff --git a/core/switch_core/gateway/tenants.py b/core/switch_core/gateway/tenants.py
index 93237fe42..f463ca1ba 100644
--- a/core/switch_core/gateway/tenants.py
+++ b/core/switch_core/gateway/tenants.py
@@ -1470,6 +1470,7 @@ async def remove_member(
api_key_store=api_key_store,
broker=broker,
sign_in_flows=request.app.state.sign_in_flows,
+ keep_agent_keys=False,
)
except ServiceError as error:
return service_refusal(error)
diff --git a/core/switch_core/gateway/workspace_access.py b/core/switch_core/gateway/workspace_access.py
index 66313b410..7093b8d9c 100644
--- a/core/switch_core/gateway/workspace_access.py
+++ b/core/switch_core/gateway/workspace_access.py
@@ -13,11 +13,13 @@ from __future__ import annotations
from dataclasses import dataclass, field
+from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from switch_core.bridges.agent.protocol.agent_core import AgentCore
from switch_core.connections.broker import ServiceBroker, SignIn
from switch_core.connections.sign_in_flows import SignInFlows
+from switch_core.db.models import Agent
from switch_core.db.stores.api_key_store import ApiKeyStore
@@ -34,13 +36,27 @@ async def revoke_workspace_access(
api_key_store: ApiKeyStore,
broker: ServiceBroker,
sign_in_flows: SignInFlows,
+ keep_agent_keys: bool,
) -> RevokedAccess:
"""Remove `user_id`'s keys, service connections and sign-ins under way in
the bound workspace, in the caller's transaction. Raises `ServiceError`
when a connection is busy, before anything is removed. The caller
- commits, then calls `finish_revocation`."""
+ commits, then calls `finish_revocation`.
+
+ `keep_agent_keys` keeps the keys agents they own run on, for a caller
+ that leaves those agents in place rather than refusing while they exist.
+ """
removed_services = await broker.remove_connections(session, user_id)
keys = await api_key_store.get_by_user(session, user_id)
+ if keep_agent_keys and keys:
+ in_use = set(
+ await session.scalars(
+ select(Agent.api_key_id).where(
+ Agent.api_key_id.in_([key.id for key in keys])
+ )
+ )
+ )
+ keys = [key for key in keys if key.id not in in_use]
for key in keys:
await api_key_store.delete(session, key.id)
await sign_in_flows.discard_for_user(session, user_id, None)
diff --git a/core/switch_core/main.py b/core/switch_core/main.py
index 07a904ba5..950ec3735 100644
--- a/core/switch_core/main.py
+++ b/core/switch_core/main.py
@@ -200,6 +200,7 @@ from switch_core.observability.pool import (
pool_stats,
)
from switch_core.observability.query import instrument_queries
+from switch_core.organization_access import AccessRevocation
from switch_core.organization_directory import LocalDirectory, OrganizationDirectory
from switch_core.outbound import guarded_async_client
from switch_core.ownership import Ownership
@@ -231,6 +232,7 @@ from switch_core.trust.signals import TrustSettingsSignals
from switch_core.user_changes import LocalUserChanges, SignalUserChanges
from switch_core.version import switch_core_version
from switch_core.workos_directory import WorkOSDirectory
+from switch_core.workos_mirror import WorkOSMirror, own_workos_mirror
logger = logging.getLogger(__name__)
@@ -1247,6 +1249,7 @@ async def run(config: SwitchConfig, process_id: str) -> None:
await connector_apps.load()
# ── Gateway app ───────────────────────────────────────────────────────────
+ organization_directory = _organization_directory(config, service_http)
gateway_app = create_gateway_app(
agent_store=agent_store,
room_store=room_store,
@@ -1263,7 +1266,7 @@ async def run(config: SwitchConfig, process_id: str) -> None:
invitation_store=invitation_store,
join_domain_store=join_domain_store,
organization_store=organization_store,
- organization_directory=_organization_directory(config, service_http),
+ organization_directory=organization_directory,
template_store=template_store,
usage_store=usage_store,
budget_store=budget_store,
@@ -1408,6 +1411,24 @@ async def run(config: SwitchConfig, process_id: str) -> None:
if config.service_connections_enabled:
protocol.service_broker = service_broker
protocol.install_service = install_service
+ if isinstance(organization_directory, WorkOSDirectory):
+ # Checked present with the directory (`_validate_organization_directory`).
+ assert config.gateway_oidc_issuer_url is not None
+ own_workos_mirror(
+ ownership,
+ WorkOSMirror(
+ session_factory,
+ organization_directory,
+ AccessRevocation(
+ agent_store=agent_store,
+ api_key_store=api_key_store,
+ broker=service_broker,
+ protocol=protocol,
+ sign_in_flows=gateway_app.state.sign_in_flows,
+ ),
+ config.gateway_oidc_issuer_url,
+ ),
+ )
snapshot_reporter = SnapshotReporter(
telemetry=telemetry,
diff --git a/core/switch_core/migrations/versions/a7c3e9f15b20_directory_mirror.py b/core/switch_core/migrations/versions/a7c3e9f15b20_directory_mirror.py
new file mode 100644
index 000000000..3d8be13b1
--- /dev/null
+++ b/core/switch_core/migrations/versions/a7c3e9f15b20_directory_mirror.py
@@ -0,0 +1,60 @@
+"""the directory mirror: its event cursor and the lookup it reads
+
+`directory_event_cursors` records how far the mirror has read a directory's
+events, one row per directory. A global table: no tenant column and no
+row-level security policy (`db/rls_ddl.py`'s `GLOBAL_TABLES`).
+
+`directory_organizations` is the sixteenth lookup in the exemption from
+row-level security: every organization that follows one in the directory.
+See `db/tenant_lookup.py` for what it discloses. Its DDL is a verbatim copy
+of `switch_core/db/tenant_lookup.py` as it stood when this migration was
+written, copied rather than imported so a later edit cannot change what this
+revision means.
+
+Revision ID: a7c3e9f15b20
+Revises: f2b8d6c1a4e9
+Create Date: 2026-11-02 12:00:00.000000
+
+"""
+
+from collections.abc import Sequence
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "a7c3e9f15b20"
+down_revision: str | Sequence[str] | None = "f2b8d6c1a4e9"
+branch_labels: str | Sequence[str] | None = None
+depends_on: str | Sequence[str] | None = None
+expand_only: bool = True
+
+SECURE_SEARCH_PATH = "pg_catalog, public, pg_temp"
+
+CREATE_DIRECTORY_ORGANIZATIONS = f"""CREATE OR REPLACE FUNCTION directory_organizations()
+ RETURNS SETOF text
+ LANGUAGE sql STABLE SECURITY DEFINER
+ SET search_path = {SECURE_SEARCH_PATH}
+AS $$SELECT id FROM organizations WHERE workos_organization_id IS NOT NULL ORDER BY id$$"""
+
+DROP_DIRECTORY_ORGANIZATIONS = "DROP FUNCTION IF EXISTS directory_organizations()"
+
+
+def upgrade() -> None:
+ op.create_table(
+ "directory_event_cursors",
+ sa.Column("source", sa.Text(), primary_key=True),
+ sa.Column("after", sa.Text(), nullable=True),
+ sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),
+ sa.Column(
+ "updated_at",
+ sa.DateTime(timezone=True),
+ server_default=sa.func.now(),
+ nullable=False,
+ ),
+ )
+ op.execute(CREATE_DIRECTORY_ORGANIZATIONS)
+
+
+def downgrade() -> None:
+ op.execute(DROP_DIRECTORY_ORGANIZATIONS)
+ op.drop_table("directory_event_cursors")
diff --git a/core/switch_core/organization_access.py b/core/switch_core/organization_access.py
new file mode 100644
index 000000000..dd02dea80
--- /dev/null
+++ b/core/switch_core/organization_access.py
@@ -0,0 +1,163 @@
+"""Ending someone's access to an organization's workspaces: what being demoted
+from `org_admin`, or leaving the organization, takes away.
+
+Shared by the organization routes (`gateway/organizations.py`), which refuse
+first while the person owns agents there, and the directory mirror
+(`organization_mirror.py`), which applies a change made in the directory
+whatever it breaks and reports what it broke.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Any
+
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
+
+from switch_core.bridges.agent.protocol.agent_core import AgentCore
+from switch_core.connections.broker import ServiceBroker
+from switch_core.connections.sign_in_flows import SignInFlows
+from switch_core.db.audit import AuditAction, record_audit_event
+from switch_core.db.models import Tenant, TenantMember
+from switch_core.db.session_scope import tenant_session
+from switch_core.db.stores.agent_store import AgentStore
+from switch_core.db.stores.api_key_store import ApiKeyStore
+from switch_core.gateway.workspace_access import (
+ finish_revocation,
+ revoke_workspace_access,
+)
+
+
+@dataclass(frozen=True)
+class AccessRevocation:
+ """What taking someone's access away in a workspace needs."""
+
+ agent_store: AgentStore
+ api_key_store: ApiKeyStore
+ broker: ServiceBroker
+ protocol: AgentCore
+ sign_in_flows: SignInFlows
+
+
+async def workspaces_where_they_own_agents(
+ session_factory: async_sessionmaker[AsyncSession],
+ agent_store: AgentStore,
+ user_id: str,
+ workspaces: list[Tenant],
+) -> list[Tenant]:
+ """Those of `workspaces` in which `user_id` owns an agent, each read on
+ its own short session."""
+ found = []
+ for tenant in workspaces:
+ async with tenant_session(session_factory, tenant.id) as session:
+ if await agent_store.get_by_owner(session, user_id):
+ found.append(tenant)
+ return found
+
+
+async def end_access_through_organization(
+ session_factory: async_sessionmaker[AsyncSession],
+ revocation: AccessRevocation,
+ *,
+ user_id: str,
+ reached: list[Tenant],
+ actor_id: str | None,
+ details: dict[str, Any],
+ keep_agent_keys: bool,
+) -> str | None:
+ """Take away what `user_id` holds in `reached`, the workspaces they stand
+ in only as their organization's `org_admin`: their keys, service
+ connections and sign-ins under way, one workspace at a time, each
+ recorded in that workspace's audit log with `details`;
+ `keep_agent_keys` as `revoke_workspace_access` has it. Raises
+ `ServiceError` when a connection is busy. Returns a warning naming what
+ could not be revoked at a vendor, or None."""
+ warnings = []
+ for tenant in reached:
+ async with tenant_session(session_factory, tenant.id) as session:
+ revoked = await revoke_workspace_access(
+ session,
+ user_id,
+ api_key_store=revocation.api_key_store,
+ broker=revocation.broker,
+ sign_in_flows=revocation.sign_in_flows,
+ keep_agent_keys=keep_agent_keys,
+ )
+ await record_audit_event(
+ session,
+ tenant_id=tenant.id,
+ actor_user_id=actor_id,
+ action=AuditAction.ORGANIZATION_ADMIN_ACCESS_ENDED,
+ target_type="user",
+ target_id=user_id,
+ details={
+ **details,
+ "api_keys_deleted": len(revoked.key_hashes),
+ "service_connections_deleted": len(revoked.removed_services),
+ },
+ )
+ await session.commit()
+ warning = await finish_revocation(
+ session,
+ user_id,
+ revoked,
+ protocol=revocation.protocol,
+ broker=revocation.broker,
+ )
+ if warning:
+ warnings.append(f"{tenant.name}: {warning}")
+ return " ".join(warnings) or None
+
+
+async def remove_from_workspace(
+ session_factory: async_sessionmaker[AsyncSession],
+ revocation: AccessRevocation,
+ *,
+ user_id: str,
+ tenant: Tenant,
+ actor_id: str | None,
+ details: dict[str, Any],
+ keep_agent_keys: bool,
+) -> str | None:
+ """Remove `user_id` from `tenant`, with their keys, service connections
+ and sign-ins under way there, recorded in its audit log with `details`.
+ Agents they own are left as they are, and with `keep_agent_keys` so are
+ the keys they run on. Raises `ServiceError` when a
+ connection is busy. Returns a warning naming what could not be revoked
+ at a vendor, or None."""
+ async with tenant_session(session_factory, tenant.id) as session:
+ membership = await session.get(TenantMember, (tenant.id, user_id))
+ if membership is None:
+ return None
+ revoked = await revoke_workspace_access(
+ session,
+ user_id,
+ api_key_store=revocation.api_key_store,
+ broker=revocation.broker,
+ sign_in_flows=revocation.sign_in_flows,
+ keep_agent_keys=keep_agent_keys,
+ )
+ await record_audit_event(
+ session,
+ tenant_id=tenant.id,
+ actor_user_id=actor_id,
+ action=AuditAction.MEMBER_REMOVED,
+ target_type="user",
+ target_id=user_id,
+ details={
+ **details,
+ "role": membership.role,
+ "api_keys_deleted": len(revoked.key_hashes),
+ "service_connections_deleted": len(revoked.removed_services),
+ },
+ )
+ await session.delete(membership)
+ await session.commit()
+ warning = await finish_revocation(
+ session,
+ user_id,
+ revoked,
+ protocol=revocation.protocol,
+ broker=revocation.broker,
+ )
+ return f"{tenant.name}: {warning}" if warning else None
diff --git a/core/switch_core/organization_mirror.py b/core/switch_core/organization_mirror.py
index 469e7bd57..7cdefe2ba 100644
--- a/core/switch_core/organization_mirror.py
+++ b/core/switch_core/organization_mirror.py
@@ -1,27 +1,38 @@
"""Switch's organization rows brought up to date with a directory kept
-outside Switch (`organization_directory.py`), for one person at sign-in.
+outside Switch (`organization_directory.py`).
Only organizations an operator has linked to one in the directory are
touched (`organizations.workos_organization_id`), and only by memberships of
exactly that one: a customer starts using Switch by having its organization
linked, not by one of its people signing in.
-- Someone the directory lists and Switch does not is added, with the
- directory's role.
-- A member the directory lists as `org_admin` is made one.
+The directory is the record, so whatever it says is applied:
-Both are recorded in the organization's audit log. Neither takes anything
-away. A demotion, or a membership the directory no longer lists, is logged at
-`error` and left as it is: taking a role away also takes away the workspaces
-it reached, which is for the directory mirror to do (not built yet), not a
-sign-in.
+- Someone it lists and Switch does not is added, with its role.
+- A member it lists as `org_admin` is made one.
+- An `org_admin` it lists as anything else is made a member, and loses what
+ they held in the organization's workspaces they stood in only as its
+ `org_admin` (`organization_access.end_access_through_organization`).
+- A member it no longer lists is removed, from the organization and from each
+ of its workspaces, with their keys, service connections and sign-ins there.
+
+Each is recorded in the organization's audit log, with where it came from
+(`source`). A change that breaks a rule Switch keeps itself (the last Org
+Admin, agents left with an owner who can no longer act, a workspace left
+with no owner) is applied all the same, logged at `error` and recorded as a
+`directory_conflict` for the organization's admins.
+
+A sign-in (`bring_in_memberships`) only grants: it never waits on taking
+access away, which the mirror (`workos_mirror.py`) does from the directory's
+events and its daily check (`apply_from_directory`).
"""
from __future__ import annotations
import logging
+from typing import Any
-from sqlalchemy import select, update
+from sqlalchemy import delete, func, select, update
from sqlalchemy.dialects.postgresql import insert
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
@@ -30,21 +41,39 @@ from switch_core.db.audit import (
OrganizationAuditAction,
record_organization_audit_event,
)
-from switch_core.db.models import Organization, OrganizationMember
-from switch_core.db.session_scope import organization_session
+from switch_core.db.models import (
+ Organization,
+ OrganizationMember,
+ Tenant,
+ TenantMember,
+)
+from switch_core.db.session_scope import organization_session, tenant_session
from switch_core.db.tenant_lookup import directory_organizations_of_user
+from switch_core.gateway.auth import (
+ workspaces_in_organization,
+ workspaces_reached_through_organization,
+)
+from switch_core.organization_access import (
+ AccessRevocation,
+ end_access_through_organization,
+ remove_from_workspace,
+ workspaces_where_they_own_agents,
+)
from switch_core.organization_directory import DirectoryMembership
logger = logging.getLogger(__name__)
+SIGN_IN = "sign_in"
+
async def bring_in_memberships(
session_factory: async_sessionmaker[AsyncSession],
user_id: str,
memberships: list[DirectoryMembership],
) -> None:
- """Bring `user_id`'s rows in each linked organization up to date with
- `memberships`, one organization at a time, each committed on its own."""
+ """At sign-in: add `user_id` to each linked organization `memberships`
+ lists them in, and make them Org Admin where it lists them as one, each
+ organization committed on its own. What it would take away is logged."""
roles = {
membership.workos_organization_id: membership.role for membership in memberships
}
@@ -52,39 +81,163 @@ async def bring_in_memberships(
session_factory, user_id, list(roles)
):
async with organization_session(session_factory, organization_id) as session:
- await _bring_in_one(session, organization_id, user_id, roles)
+ workos_organization_id, current = await _link_and_role(
+ session, organization_id, user_id
+ )
+ wanted = (
+ roles.get(workos_organization_id) if workos_organization_id else None
+ )
+ if wanted is not None and _grants(current, wanted):
+ await _grant(
+ session, organization_id, user_id, current, wanted, SIGN_IN
+ )
+ elif current != wanted:
+ logger.warning(
+ "User %s is %s in organization %s and %s in WorkOS "
+ "organization %s; the directory mirror applies it",
+ user_id,
+ current,
+ organization_id,
+ wanted or "not a member",
+ workos_organization_id,
+ )
-async def _bring_in_one(
- session: AsyncSession,
+async def apply_from_directory(
+ session_factory: async_sessionmaker[AsyncSession],
+ revocation: AccessRevocation,
+ *,
organization_id: str,
user_id: str,
- roles: dict[str, str],
+ wanted: str | None,
+ source: str,
) -> None:
- workos_organization_id, current = (
- await session.execute(
- select(Organization.workos_organization_id, OrganizationMember.role)
- .outerjoin(
- OrganizationMember,
- (OrganizationMember.organization_id == Organization.id)
- & (OrganizationMember.user_id == user_id),
- )
- .where(Organization.id == organization_id)
+ """Make `user_id`'s membership of `organization_id` what the directory
+ says, `wanted` (a role, or None for no membership). Raises `ServiceError`
+ when a connection to revoke is busy, having changed nothing that a retry
+ would not finish."""
+ async with organization_session(session_factory, organization_id) as session:
+ _, current = await _link_and_role(session, organization_id, user_id)
+ if current == wanted:
+ return
+ if _grants(current, wanted):
+ assert wanted is not None
+ await _grant(session, organization_id, user_id, current, wanted, source)
+ return
+ admins = await _count_admins(session, organization_id)
+ assert current is not None
+ conflicts: list[dict[str, Any]] = []
+ warnings: list[str] = []
+ details = {"source": source}
+
+ if current == ORGANIZATION_ADMIN_ROLE:
+ if admins <= 1:
+ conflicts.append({"kind": "last_org_admin"})
+ reached = await workspaces_reached_through_organization(
+ session_factory, user_id, organization_id
)
- ).one()
- wanted = roles.get(workos_organization_id) if workos_organization_id else None
+ conflicts += await _agents_left(session_factory, revocation, user_id, reached)
+ warning = await end_access_through_organization(
+ session_factory,
+ revocation,
+ user_id=user_id,
+ reached=reached,
+ actor_id=None,
+ details={
+ "reason": "demoted" if wanted is not None else "removed",
+ **details,
+ },
+ keep_agent_keys=True,
+ )
+ if warning:
+ warnings.append(warning)
+
if wanted is None:
- if current is not None:
+ workspaces = await workspaces_in_organization(
+ session_factory, user_id, organization_id
+ )
+ conflicts += await _agents_left(
+ session_factory, revocation, user_id, workspaces
+ )
+ for tenant in workspaces:
+ if await _is_last_owner(session_factory, tenant.id, user_id):
+ conflicts.append(
+ {"kind": "workspace_left_without_owner", "workspace": tenant.name}
+ )
+ warning = await remove_from_workspace(
+ session_factory,
+ revocation,
+ user_id=user_id,
+ tenant=tenant,
+ actor_id=None,
+ details=details,
+ keep_agent_keys=True,
+ )
+ if warning:
+ warnings.append(warning)
+
+ async with organization_session(session_factory, organization_id) as session:
+ if wanted is None:
+ await session.execute(
+ delete(OrganizationMember).where(
+ OrganizationMember.organization_id == organization_id,
+ OrganizationMember.user_id == user_id,
+ )
+ )
+ action = OrganizationAuditAction.MEMBER_REMOVED_FROM_DIRECTORY
+ else:
+ await _set_role(session, organization_id, user_id, wanted)
+ action = OrganizationAuditAction.MEMBER_DEMOTED_FROM_DIRECTORY
+ await record_organization_audit_event(
+ session,
+ organization_id=organization_id,
+ actor_user_id=None,
+ action=action,
+ target_type="user",
+ target_id=user_id,
+ details={
+ **details,
+ "from_role": current,
+ "role": wanted,
+ **({"warning": " ".join(warnings)} if warnings else {}),
+ },
+ )
+ for conflict in conflicts:
logger.error(
- "User %s is %s in organization %s, which WorkOS organization %s "
- "no longer lists them in; Switch keeps their membership until "
- "the directory mirror can end it",
+ "Applying WorkOS's change to user %s in organization %s broke "
+ "a rule Switch keeps: %s",
user_id,
- current,
organization_id,
- workos_organization_id,
+ conflict,
)
- return
+ await record_organization_audit_event(
+ session,
+ organization_id=organization_id,
+ actor_user_id=None,
+ action=OrganizationAuditAction.DIRECTORY_CONFLICT,
+ target_type="user",
+ target_id=user_id,
+ details={**details, **conflict},
+ )
+ await session.commit()
+
+
+def _grants(current: str | None, wanted: str | None) -> bool:
+ """Whether going from `current` to `wanted` only gives: joining, or
+ becoming Org Admin."""
+ if wanted is None or current == wanted:
+ return False
+ return current is None or wanted == ORGANIZATION_ADMIN_ROLE
+
+
+async def _grant(
+ session: AsyncSession,
+ organization_id: str,
+ user_id: str,
+ current: str | None,
+ wanted: str,
+ source: str,
+) -> None:
if current is None:
added = await session.execute(
insert(OrganizationMember)
@@ -95,32 +248,12 @@ async def _bring_in_one(
if added.first() is None:
# A concurrent sign-in or invitation added them first.
return
- action = OrganizationAuditAction.MEMBER_ADDED_AT_SIGN_IN
- details = {"role": wanted}
- elif current != wanted and wanted == ORGANIZATION_ADMIN_ROLE:
- await session.execute(
- update(OrganizationMember)
- .where(
- OrganizationMember.organization_id == organization_id,
- OrganizationMember.user_id == user_id,
- )
- .values(role=wanted)
- )
- action = OrganizationAuditAction.MEMBER_PROMOTED_AT_SIGN_IN
- details = {"from_role": current, "role": wanted}
+ action = OrganizationAuditAction.MEMBER_ADDED_FROM_DIRECTORY
+ details: dict[str, Any] = {"source": source, "role": wanted}
else:
- if current != wanted:
- logger.error(
- "User %s is %s in organization %s and %s in WorkOS organization "
- "%s; Switch keeps their role until the directory mirror can "
- "change it",
- user_id,
- current,
- organization_id,
- wanted,
- workos_organization_id,
- )
- return
+ await _set_role(session, organization_id, user_id, wanted)
+ action = OrganizationAuditAction.MEMBER_PROMOTED_FROM_DIRECTORY
+ details = {"source": source, "from_role": current, "role": wanted}
await record_organization_audit_event(
session,
organization_id=organization_id,
@@ -131,3 +264,77 @@ async def _bring_in_one(
details=details,
)
await session.commit()
+
+
+async def _link_and_role(
+ session: AsyncSession, organization_id: str, user_id: str
+) -> tuple[str | None, str | None]:
+ """The organization's link to the directory, and `user_id`'s role in it
+ (None if not a member), in one read."""
+ workos_organization_id, role = (
+ await session.execute(
+ select(Organization.workos_organization_id, OrganizationMember.role)
+ .outerjoin(
+ OrganizationMember,
+ (OrganizationMember.organization_id == Organization.id)
+ & (OrganizationMember.user_id == user_id),
+ )
+ .where(Organization.id == organization_id)
+ )
+ ).one()
+ return workos_organization_id, role
+
+
+async def _set_role(
+ session: AsyncSession, organization_id: str, user_id: str, role: str
+) -> None:
+ await session.execute(
+ update(OrganizationMember)
+ .where(
+ OrganizationMember.organization_id == organization_id,
+ OrganizationMember.user_id == user_id,
+ )
+ .values(role=role)
+ )
+
+
+async def _count_admins(session: AsyncSession, organization_id: str) -> int:
+ return (
+ await session.scalar(
+ select(func.count()).where(
+ OrganizationMember.organization_id == organization_id,
+ OrganizationMember.role == ORGANIZATION_ADMIN_ROLE,
+ )
+ )
+ ) or 0
+
+
+async def _agents_left(
+ session_factory: async_sessionmaker[AsyncSession],
+ revocation: AccessRevocation,
+ user_id: str,
+ workspaces: list[Tenant],
+) -> list[dict[str, Any]]:
+ """A conflict for each of `workspaces` where `user_id` owns agents, which
+ stay, with the keys they run on, owned by someone who can no longer act
+ there, for an admin to deal with."""
+ return [
+ {"kind": "agents_left_with_owner", "workspace": tenant.name}
+ for tenant in await workspaces_where_they_own_agents(
+ session_factory, revocation.agent_store, user_id, workspaces
+ )
+ ]
+
+
+async def _is_last_owner(
+ session_factory: async_sessionmaker[AsyncSession], tenant_id: str, user_id: str
+) -> bool:
+ async with tenant_session(session_factory, tenant_id) as session:
+ owners = list(
+ await session.scalars(
+ select(TenantMember.user_id).where(
+ TenantMember.tenant_id == tenant_id, TenantMember.role == "owner"
+ )
+ )
+ )
+ return owners == [user_id]
diff --git a/core/switch_core/workos_directory.py b/core/switch_core/workos_directory.py
index 0feaf3ad5..3dbb9b052 100644
--- a/core/switch_core/workos_directory.py
+++ b/core/switch_core/workos_directory.py
@@ -27,6 +27,7 @@ local directory, and that is what a new organization is until then.
from __future__ import annotations
import logging
+from collections.abc import Mapping
from typing import Any
import httpx
@@ -83,7 +84,9 @@ class WorkOSDirectory:
self._api_key = api_key
self._api_url = api_url.rstrip("/")
- async def _get(self, path: str, params: dict[str, str]) -> dict[str, Any]:
+ async def _get(
+ self, path: str, params: Mapping[str, str | list[str]]
+ ) -> dict[str, Any]:
try:
response = await self._http.get(
f"{self._api_url}{path}",
@@ -137,11 +140,17 @@ class WorkOSDirectory:
"""The active memberships of WorkOS user `subject`, one per WorkOS
organization. Which of those organizations Switch follows is for the
caller to read from its own rows."""
- memberships: list[DirectoryMembership] = []
- for membership in await self._list(
- "/user_management/organization_memberships",
- {"user_id": subject, "statuses": "active"},
- ):
+ return self._read_memberships(
+ await self._list(
+ "/user_management/organization_memberships",
+ {"user_id": subject, "statuses": "active"},
+ )
+ )
+
+ @staticmethod
+ def _read_memberships(items: list[dict[str, Any]]) -> list[DirectoryMembership]:
+ memberships = []
+ for membership in items:
organization_id = membership.get("organization_id")
role = membership.get("role")
slug = role.get("slug") if isinstance(role, dict) else None
@@ -154,6 +163,70 @@ class WorkOSDirectory:
)
return memberships
+ async def membership_role(
+ self, subject: str, workos_organization_id: str
+ ) -> str | None:
+ """Switch's role for WorkOS user `subject`'s active membership of
+ `workos_organization_id`, or None if they hold none."""
+ memberships = await self._list(
+ "/user_management/organization_memberships",
+ {
+ "user_id": subject,
+ "organization_id": workos_organization_id,
+ "statuses": "active",
+ },
+ )
+ for membership in self._read_memberships(memberships):
+ if membership.workos_organization_id == workos_organization_id:
+ return membership.role
+ return None
+
+ async def organization_members(self, workos_organization_id: str) -> dict[str, str]:
+ """Switch's role for each WorkOS user with an active membership of
+ `workos_organization_id`, by their WorkOS user id."""
+ members: dict[str, str] = {}
+ for item in await self._list(
+ "/user_management/organization_memberships",
+ {"organization_id": workos_organization_id, "statuses": "active"},
+ ):
+ user_id = item.get("user_id")
+ if not isinstance(user_id, str):
+ raise _unreadable("a membership")
+ [membership] = self._read_memberships([item])
+ members[user_id] = membership.role
+ return members
+
+ async def events(
+ self, types: tuple[str, ...], *, after: str | None, range_start: str
+ ) -> list[dict[str, Any]]:
+ """One page of events of `types`, oldest first: those after event
+ `after`, or from `range_start` (ISO 8601) when there is none yet.
+ Empty once there are no more."""
+ params: dict[str, str | list[str]] = {
+ "events": list(types),
+ "limit": str(_PAGE),
+ "order": "asc",
+ }
+ if after:
+ params["after"] = after
+ else:
+ params["range_start"] = range_start
+ page = await self._get("/events", params)
+ data = page.get("data")
+ if not isinstance(data, list):
+ raise _unreadable("a list of events")
+ events = []
+ for event in data:
+ if (
+ not isinstance(event, dict)
+ or not isinstance(event.get("id"), str)
+ or not isinstance(event.get("event"), str)
+ or not isinstance(event.get("data"), dict)
+ ):
+ raise _unreadable("an event")
+ events.append(event)
+ return events
+
async def organization(self, workos_organization_id: str) -> dict[str, Any]:
"""The WorkOS organization `workos_organization_id`, for an operator
linking a Switch organization to it."""
diff --git a/core/switch_core/workos_mirror.py b/core/switch_core/workos_mirror.py
new file mode 100644
index 000000000..e3adc4dff
--- /dev/null
+++ b/core/switch_core/workos_mirror.py
@@ -0,0 +1,341 @@
+"""The directory mirror under `ORGANIZATION_DIRECTORY=workos`: changes made
+in WorkOS to a linked organization's members, applied in Switch
+(`organization_mirror.apply_from_directory`).
+
+- **Events.** WorkOS's Events API, polled every `POLL_INTERVAL_SECONDS` from
+ the cursor in `directory_event_cursors`, rather than webhooks: ordered,
+ resumable, and with no public endpoint or signing secret. An event is only
+ a prompt: applying it re-reads the person's current membership from
+ WorkOS, so events read out of order, twice, or after a later change all
+ end in the same state. An event that fails to apply stops the pass, and the
+ next pass starts again from it.
+- **The daily check.** Every `CHECK_INTERVAL_SECONDS`, and when the mirror
+ starts, each linked organization's members in WorkOS are compared with
+ Switch's: the backstop for events missed (the Events API keeps 90 days) or
+ from before the mirror ran. A Switch member with no WorkOS identity can't
+ be compared, and is logged and left.
+
+People are matched to WorkOS users by the sign-in identity linking them
+(`oidc_identities`, the gateway's issuer and their WorkOS user id). Someone
+WorkOS lists who has never signed in to Switch has no account to add, and is
+brought in when they first sign in.
+
+One process runs it (`ownership.py`, kind `workos_mirror`).
+"""
+
+from __future__ import annotations
+
+import asyncio
+import contextlib
+import logging
+from datetime import UTC, datetime
+from typing import Any
+
+from sqlalchemy import select
+from sqlalchemy.dialects.postgresql import insert
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
+
+from switch_core.connections.broker import ServiceError
+from switch_core.db.audit import (
+ OrganizationAuditAction,
+ record_organization_audit_event,
+)
+from switch_core.db.models import (
+ DirectoryEventCursor,
+ OidcIdentity,
+ Organization,
+ OrganizationMember,
+)
+from switch_core.db.session_scope import organization_session
+from switch_core.db.tenant_lookup import directory_organizations
+from switch_core.organization_access import AccessRevocation
+from switch_core.organization_directory import DirectoryUnavailable
+from switch_core.organization_mirror import apply_from_directory
+from switch_core.ownership import Fence, OwnedKind, Ownership
+from switch_core.tenant_context import no_tenant
+from switch_core.workos_directory import WorkOSDirectory
+
+logger = logging.getLogger(__name__)
+
+POLL_INTERVAL_SECONDS = 30.0
+CHECK_INTERVAL_SECONDS = 24 * 3600.0
+CURSOR_SOURCE = "workos"
+EVENT_TYPES = (
+ "organization_membership.created",
+ "organization_membership.updated",
+ "organization_membership.deleted",
+ "organization.deleted",
+)
+FROM_EVENT = "directory_event"
+FROM_CHECK = "daily_check"
+
+MIRROR_KIND = "workos_mirror"
+MIRROR_KEY = "all"
+
+
+class WorkOSMirror:
+ def __init__(
+ self,
+ session_factory: async_sessionmaker[AsyncSession],
+ directory: WorkOSDirectory,
+ revocation: AccessRevocation,
+ issuer: str,
+ ) -> None:
+ self._session_factory = session_factory
+ self._directory = directory
+ self._revocation = revocation
+ self._issuer = issuer
+
+ async def poll_once(self) -> int:
+ """Apply the events since the cursor, page by page, moving the cursor
+ past each one applied. Returns how many were read."""
+ cursor = await self._cursor()
+ linked = await self._linked()
+ read = 0
+ after = cursor.after
+ while True:
+ events = await self._directory.events(
+ EVENT_TYPES, after=after, range_start=cursor.started_at.isoformat()
+ )
+ if not events:
+ return read
+ for event in events:
+ await self._apply_event(event, linked)
+ after = event["id"]
+ await self._advance(after)
+ read += 1
+
+ async def check_once(self) -> None:
+ """Compare every linked organization's members with WorkOS's, and
+ apply the difference. One organization failing doesn't stop the
+ others."""
+ for organization_id, workos_organization_id in (await self._linked()).items():
+ try:
+ await self._check(organization_id, workos_organization_id)
+ except (DirectoryUnavailable, ServiceError) as error:
+ logger.error(
+ "Daily check of organization %s against WorkOS stopped: %s",
+ organization_id,
+ error,
+ )
+
+ async def run(self) -> None:
+ """Poll the events every `POLL_INTERVAL_SECONDS`, and check every
+ organization when starting and every `CHECK_INTERVAL_SECONDS`."""
+ with no_tenant():
+ since_check = CHECK_INTERVAL_SECONDS
+ while True:
+ if since_check >= CHECK_INTERVAL_SECONDS:
+ since_check = 0.0
+ await self._logged(self.check_once(), "daily check")
+ await self._logged(self.poll_once(), "events")
+ await asyncio.sleep(POLL_INTERVAL_SECONDS)
+ since_check += POLL_INTERVAL_SECONDS
+
+ @staticmethod
+ async def _logged(work: Any, what: str) -> None:
+ try:
+ await work
+ except (DirectoryUnavailable, ServiceError) as error:
+ logger.error("WorkOS mirror %s: %s; trying again later", what, error)
+ except Exception:
+ logger.exception("WorkOS mirror %s failed; trying again later", what)
+
+ async def _apply_event(self, event: dict[str, Any], linked: dict[str, str]) -> None:
+ data = event["data"]
+ following = {workos: own for own, workos in linked.items()}
+ if event["event"] == "organization.deleted":
+ organization_id = following.get(str(data.get("id")))
+ if organization_id is not None:
+ await self._organization_deleted(organization_id, data)
+ return
+ organization_id = following.get(str(data.get("organization_id")))
+ subject = data.get("user_id")
+ if organization_id is None or not isinstance(subject, str):
+ return
+ users = await self._users_of([subject])
+ user_id = users.get(subject)
+ if user_id is None:
+ return
+ wanted = await self._directory.membership_role(subject, linked[organization_id])
+ await apply_from_directory(
+ self._session_factory,
+ self._revocation,
+ organization_id=organization_id,
+ user_id=user_id,
+ wanted=wanted,
+ source=FROM_EVENT,
+ )
+
+ async def _check(self, organization_id: str, workos_organization_id: str) -> None:
+ in_workos = await self._directory.organization_members(workos_organization_id)
+ async with organization_session(
+ self._session_factory, organization_id
+ ) as session:
+ in_switch = set(
+ await session.scalars(
+ select(OrganizationMember.user_id).where(
+ OrganizationMember.organization_id == organization_id
+ )
+ )
+ )
+ users = await self._users_of(list(in_workos))
+ subjects = await self._subjects_of(list(in_switch))
+ wanted: dict[str, str | None] = {
+ user_id: in_workos[subject] for subject, user_id in users.items()
+ }
+ for user_id in in_switch:
+ subject = subjects.get(user_id)
+ if subject is None:
+ logger.error(
+ "User %s is in organization %s but has no WorkOS identity "
+ "to compare; left as they are",
+ user_id,
+ organization_id,
+ )
+ continue
+ wanted.setdefault(user_id, in_workos.get(subject))
+ for user_id, role in sorted(wanted.items()):
+ await apply_from_directory(
+ self._session_factory,
+ self._revocation,
+ organization_id=organization_id,
+ user_id=user_id,
+ wanted=role,
+ source=FROM_CHECK,
+ )
+
+ async def _organization_deleted(
+ self, organization_id: str, data: dict[str, Any]
+ ) -> None:
+ """Deleting a linked organization in WorkOS removes nobody here: that
+ is too much to take away on one event. It is reported for an operator
+ to unlink or remove."""
+ logger.error(
+ "WorkOS organization %s, which organization %s follows, was deleted; "
+ "its members are left as they are",
+ data.get("id"),
+ organization_id,
+ )
+ async with organization_session(
+ self._session_factory, organization_id
+ ) as session:
+ await record_organization_audit_event(
+ session,
+ organization_id=organization_id,
+ actor_user_id=None,
+ action=OrganizationAuditAction.DIRECTORY_CONFLICT,
+ target_type="organization",
+ target_id=organization_id,
+ details={
+ "source": FROM_EVENT,
+ "kind": "directory_organization_deleted",
+ },
+ )
+ await session.commit()
+
+ async def _linked(self) -> dict[str, str]:
+ """Each linked organization's WorkOS organization, by its id."""
+ linked: dict[str, str] = {}
+ for organization_id in await directory_organizations(self._session_factory):
+ async with organization_session(
+ self._session_factory, organization_id
+ ) as session:
+ workos_organization_id = await session.scalar(
+ select(Organization.workos_organization_id).where(
+ Organization.id == organization_id
+ )
+ )
+ if workos_organization_id is not None:
+ linked[organization_id] = workos_organization_id
+ return linked
+
+ async def _users_of(self, subjects: list[str]) -> dict[str, str]:
+ """The Switch account linked to each of WorkOS users `subjects` that
+ has one."""
+ if not subjects:
+ return {}
+ with no_tenant():
+ async with self._session_factory() as session:
+ rows = await session.execute(
+ select(OidcIdentity.sub, OidcIdentity.user_id).where(
+ OidcIdentity.iss == self._issuer,
+ OidcIdentity.sub.in_(subjects),
+ )
+ )
+ return {sub: user_id for sub, user_id in rows}
+
+ async def _subjects_of(self, user_ids: list[str]) -> dict[str, str]:
+ """The WorkOS user each of `user_ids` signs in as, where linked."""
+ if not user_ids:
+ return {}
+ with no_tenant():
+ async with self._session_factory() as session:
+ rows = await session.execute(
+ select(OidcIdentity.user_id, OidcIdentity.sub).where(
+ OidcIdentity.iss == self._issuer,
+ OidcIdentity.user_id.in_(user_ids),
+ )
+ )
+ return {user_id: sub for user_id, sub in rows}
+
+ async def _cursor(self) -> DirectoryEventCursor:
+ """The cursor, made on the mirror's first run: events are read from
+ then on, and the daily check covers what came before."""
+ with no_tenant():
+ async with self._session_factory() as session:
+ await session.execute(
+ insert(DirectoryEventCursor)
+ .values(
+ source=CURSOR_SOURCE, after=None, started_at=datetime.now(UTC)
+ )
+ .on_conflict_do_nothing(index_elements=["source"])
+ )
+ await session.commit()
+ cursor = await session.get(DirectoryEventCursor, CURSOR_SOURCE)
+ assert cursor is not None
+ return cursor
+
+ async def _advance(self, after: str) -> None:
+ with no_tenant():
+ async with self._session_factory() as session:
+ cursor = await session.get(DirectoryEventCursor, CURSOR_SOURCE)
+ assert cursor is not None
+ cursor.after = after
+ cursor.updated_at = datetime.now(UTC)
+ await session.commit()
+
+
+def own_workos_mirror(ownership: Ownership, mirror: WorkOSMirror) -> OwnedKind:
+ """Register the mirror as owned work, wanted by this process; it runs here
+ only while it holds it."""
+ running: dict[str, asyncio.Task[None]] = {}
+
+ def launch(key: str) -> None:
+ task = asyncio.create_task(mirror.run(), name="workos-mirror")
+ running[key] = task
+ task.add_done_callback(lambda done: ended(key, done))
+
+ def ended(key: str, task: asyncio.Task[None]) -> None:
+ if task.cancelled() or running.get(key) is not task:
+ return
+ logger.error(
+ "WorkOS mirror stopped unexpectedly; restarting",
+ exc_info=task.exception(),
+ )
+ launch(key)
+
+ async def start(key: str, fence: Fence) -> None:
+ launch(key)
+
+ async def stop(key: str) -> None:
+ task = running.pop(key, None)
+ if task is None:
+ return
+ task.cancel()
+ with contextlib.suppress(asyncio.CancelledError):
+ await task
+
+ kind = ownership.kind(MIRROR_KIND, start=start, stop=stop)
+ kind.want(MIRROR_KEY)
+ return kind
diff --git a/core/tests/switch_core/db/test_tenant_exemption_allowlist.py b/core/tests/switch_core/db/test_tenant_exemption_allowlist.py
index ee076e104..7a21d18ec 100644
--- a/core/tests/switch_core/db/test_tenant_exemption_allowlist.py
+++ b/core/tests/switch_core/db/test_tenant_exemption_allowlist.py
@@ -132,12 +132,16 @@ _ALLOWED_MODULES = {
# the flow's tenant, and the flow is then read scoped to it, so a wrong
# answer is a missing flow rather than a cross-tenant read.
"switch_core.connections.sign_in_flows",
- # A WorkOS sign-in names the customers the person belongs to, by the
- # customer's id; which Switch organization each one is has to be read
- # before any is bound, and the membership is then written with that
- # organization bound, so a wrong answer is a missing row rather than a
- # write into another organization.
+ # A WorkOS sign-in names the WorkOS organizations the person belongs to;
+ # which Switch organizations follow them has to be read before any is
+ # bound, and the membership is then written with that organization bound,
+ # so a wrong answer is a missing row rather than a write into another
+ # organization.
"switch_core.organization_mirror",
+ # The WorkOS mirror acts for no tenant and no person: which organizations
+ # are linked is read before any is bound, and each is then read and
+ # written with it bound.
+ "switch_core.workos_mirror",
# `switch_core.transport.postgres` and `switch_core.clients.agent_consumer`
# came off this list with `tenant_of_client`: both were built from a
# `clients` row that already named the tenant, so they carry it instead of
@@ -243,6 +247,10 @@ _RAW_SESSION_FACTORY_MODULES = {
# their sign-in identities belong to no tenant. Its organization linking
# binds the organization (`organization_session`).
"switch_core.workos_linking",
+ # The WorkOS mirror reads and moves its cursor (`directory_event_cursors`)
+ # and reads `oidc_identities` to match people, both global
+ # (`rls_ddl.GLOBAL_TABLES`); every organization it changes is bound.
+ "switch_core.workos_mirror",
#
# ── The OAuth client Core registers for itself at a vendor that offers
# no other (`service_oauth_clients`, in `rls_ddl.GLOBAL_TABLES`). The
diff --git a/core/tests/switch_core/db/test_tenant_lookup.py b/core/tests/switch_core/db/test_tenant_lookup.py
index f6b16052c..7cde7c7a9 100644
--- a/core/tests/switch_core/db/test_tenant_lookup.py
+++ b/core/tests/switch_core/db/test_tenant_lookup.py
@@ -112,6 +112,7 @@ _ADDED_SINCE = {
"tenants_bindable_by_user": "5c2e8f17a9d4",
"all_organization_ids": "10f6d02ac5bc",
"directory_organizations_of_user": "f2b8d6c1a4e9",
+ "directory_organizations": "a7c3e9f15b20",
}
# A third kind of change, and the quietest: a lookup whose body a later
diff --git a/core/tests/switch_core/db/test_tenant_schema_catalogue.py b/core/tests/switch_core/db/test_tenant_schema_catalogue.py
index ce6639b04..0bc8a0a98 100644
--- a/core/tests/switch_core/db/test_tenant_schema_catalogue.py
+++ b/core/tests/switch_core/db/test_tenant_schema_catalogue.py
@@ -121,6 +121,9 @@ class TestEveryTableIsScopedUnlessItIsNamedGlobal:
# once: a process holds work for every tenant at once, and the row
# names only which process runs which id.
"ownership_leases",
+ # How far the directory mirror has read a directory's events: one
+ # row per directory, whose events reach every organization.
+ "directory_event_cursors",
# Switch Trust's one guardrails policy for the whole deployment.
# A policy that varied per tenant would be a new scoped table.
"trust_settings",
diff --git a/core/tests/switch_core/test_organization_mirror.py b/core/tests/switch_core/test_organization_mirror.py
index ed2e4a205..9b5fec800 100644
--- a/core/tests/switch_core/test_organization_mirror.py
+++ b/core/tests/switch_core/test_organization_mirror.py
@@ -92,7 +92,7 @@ async def _added(session_factory, organization_id: str) -> list[OrganizationAudi
return await _recorded(
session_factory,
organization_id,
- OrganizationAuditAction.MEMBER_ADDED_AT_SIGN_IN,
+ OrganizationAuditAction.MEMBER_ADDED_FROM_DIRECTORY,
)
@@ -116,7 +116,7 @@ class TestBringingInMemberships:
[event] = await _added(session_factory, organization_id)
assert event.actor_user_id is None
assert event.target_id == ada
- assert event.details == {"role": "org_admin"}
+ assert event.details == {"source": "sign_in", "role": "org_admin"}
async def test_only_the_linked_organization_counts(
self, rls_harness: RLSHarness
@@ -165,12 +165,16 @@ class TestBringingInMemberships:
[event] = await _recorded(
session_factory,
organization_id,
- OrganizationAuditAction.MEMBER_PROMOTED_AT_SIGN_IN,
+ OrganizationAuditAction.MEMBER_PROMOTED_FROM_DIRECTORY,
)
assert event.target_id == ada
- assert event.details == {"from_role": "member", "role": "org_admin"}
+ assert event.details == {
+ "source": "sign_in",
+ "from_role": "member",
+ "role": "org_admin",
+ }
- async def test_a_demotion_is_left_and_logged_as_an_error(
+ async def test_a_demotion_is_left_for_the_mirror_and_logged(
self, rls_harness: RLSHarness, caplog: pytest.LogCaptureFixture
) -> None:
"""Taking a role away takes away what it reached, which is not a
@@ -187,10 +191,10 @@ class TestBringingInMemberships:
assert await _role(session_factory, organization_id, ada) == "org_admin"
assert await _added(session_factory, organization_id) == []
- [record] = [r for r in caplog.records if "keeps their role" in r.message]
- assert record.levelname == "ERROR"
+ [record] = [r for r in caplog.records if "mirror applies it" in r.message]
+ assert record.levelname == "WARNING"
- async def test_a_membership_no_longer_listed_is_left_and_logged_as_an_error(
+ async def test_a_membership_no_longer_listed_is_left_for_the_mirror(
self, rls_harness: RLSHarness, caplog: pytest.LogCaptureFixture
) -> None:
session_factory, restricted = rls_harness.owner, rls_harness.restricted
@@ -201,5 +205,5 @@ class TestBringingInMemberships:
await bring_in_memberships(restricted, ada, [])
assert await _role(session_factory, organization_id, ada) == "org_admin"
- [record] = [r for r in caplog.records if "no longer lists them" in r.message]
- assert record.levelname == "ERROR"
+ [record] = [r for r in caplog.records if "mirror applies it" in r.message]
+ assert record.levelname == "WARNING"
diff --git a/core/tests/switch_core/test_organization_mirror_apply.py b/core/tests/switch_core/test_organization_mirror_apply.py
new file mode 100644
index 000000000..1fc064186
--- /dev/null
+++ b/core/tests/switch_core/test_organization_mirror_apply.py
@@ -0,0 +1,386 @@
+"""Changes made in the directory applied in Switch, taking away as well as
+granting (`organization_mirror.apply_from_directory`), against Postgres, on
+the restricted role row-level security applies to; `owner` only arranges
+fixtures and reads results."""
+
+from __future__ import annotations
+
+import uuid
+from datetime import timedelta
+from types import SimpleNamespace
+
+import pytest
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
+
+from switch_core.connections.broker import ServiceBroker
+from switch_core.connections.loader import CATALOG
+from switch_core.connections.sign_in_flows import SignInFlows
+from switch_core.db.audit import AuditAction, OrganizationAuditAction
+from switch_core.db.models import (
+ Agent,
+ ApiKey,
+ AuditEvent,
+ Client,
+ Organization,
+ OrganizationAuditEvent,
+ OrganizationMember,
+ Tenant,
+ TenantMember,
+ User,
+)
+from switch_core.db.stores.agent_store import AgentStore
+from switch_core.db.stores.api_key_store import ApiKeyStore
+from switch_core.db.stores.service_connection_store import ServiceConnectionStore
+from switch_core.db.stores.workspace_service_store import WorkspaceServiceStore
+from switch_core.keys import Keyring
+from switch_core.organization_access import AccessRevocation
+from switch_core.organization_mirror import apply_from_directory
+from tests.conftest import RLSHarness
+
+pytestmark = pytest.mark.no_ambient_tenant
+
+_KEYRING = Keyring.parse(
+ "test:" + "organization-mirror-test-secret-" + "0" * 32, legacy_secret=None
+)
+
+
+def _id(prefix: str) -> str:
+ return f"{prefix}-{uuid.uuid4().hex[:8]}"
+
+
+def _revocation(session_factory: async_sessionmaker[AsyncSession]) -> AccessRevocation:
+ return AccessRevocation(
+ agent_store=AgentStore(),
+ api_key_store=ApiKeyStore(),
+ broker=ServiceBroker(
+ session_factory=session_factory,
+ keyring=_KEYRING,
+ catalog=CATALOG,
+ adapters={},
+ disabled={},
+ store=ServiceConnectionStore(),
+ token_retention=timedelta(days=30),
+ workspace_services=WorkspaceServiceStore(),
+ workspace_gate=False,
+ public_url=None,
+ ),
+ protocol=SimpleNamespace( # type: ignore[arg-type]
+ api_key_cache=SimpleNamespace(invalidate=lambda key_hash: None)
+ ),
+ sign_in_flows=SignInFlows(session_factory, _KEYRING),
+ )
+
+
+class _World:
+ """A linked organization with Ada and Bea its `org_admin`s: `joined`, a
+ workspace Ada is a member of (Bea owns it), and `reached`, one she
+ stands in only as `org_admin`. Ada holds a personal key in each."""
+
+ def __init__(self, organization_id: str, joined: str, reached: str) -> None:
+ self.organization_id = organization_id
+ self.joined = joined
+ self.reached = reached
+ self.ada = _id("ada")
+ self.bea = _id("bea")
+
+ @classmethod
+ async def make(
+ cls,
+ owner: async_sessionmaker[AsyncSession],
+ *,
+ bea_role: str = "org_admin",
+ ada_owns_joined: bool = False,
+ ) -> _World:
+ world = cls(_id("org"), _id("joined"), _id("reached"))
+ async with owner() as session:
+ session.add(
+ Organization(
+ id=world.organization_id,
+ slug=world.organization_id,
+ name="Acme",
+ workos_organization_id=_id("org_workos"),
+ )
+ )
+ for user_id in (world.ada, world.bea):
+ session.add(
+ User(
+ id=user_id,
+ name=user_id,
+ email=f"{user_id}@example.invalid",
+ role="user",
+ )
+ )
+ await session.flush()
+ for tenant_id in (world.joined, world.reached):
+ session.add(
+ Tenant(
+ id=tenant_id,
+ slug=tenant_id,
+ name=tenant_id,
+ organization_id=world.organization_id,
+ )
+ )
+ session.add_all(
+ [
+ OrganizationMember(
+ organization_id=world.organization_id,
+ user_id=world.ada,
+ role="org_admin",
+ ),
+ OrganizationMember(
+ organization_id=world.organization_id,
+ user_id=world.bea,
+ role=bea_role,
+ ),
+ ]
+ )
+ await session.flush()
+ session.add(
+ TenantMember(
+ tenant_id=world.joined,
+ user_id=world.ada,
+ role="owner" if ada_owns_joined else "member",
+ )
+ )
+ if not ada_owns_joined:
+ session.add(
+ TenantMember(
+ tenant_id=world.joined, user_id=world.bea, role="owner"
+ )
+ )
+ for tenant_id in (world.joined, world.reached):
+ session.add(_key(tenant_id, world.ada, "personal"))
+ await session.commit()
+ return world
+
+
+def _key(tenant_id: str, user_id: str, type_: str) -> ApiKey:
+ return ApiKey(
+ id=_id("key"),
+ tenant_id=tenant_id,
+ user_id=user_id,
+ key_hash=uuid.uuid4().hex,
+ encrypted_key="unused-in-tests",
+ label="test-key",
+ type=type_,
+ )
+
+
+async def _agent(owner, tenant_id: str, owner_id: str) -> str:
+ async with owner() as session:
+ client = Client(
+ tenant_id=tenant_id,
+ transport_user_id=f"@bot-{uuid.uuid4().hex[:8]}:test",
+ display_name="bot",
+ type="agent",
+ )
+ key = _key(tenant_id, owner_id, "agent")
+ session.add_all([client, key])
+ await session.flush()
+ agent = Agent(
+ tenant_id=tenant_id,
+ name=_id("agent"),
+ description="test agent",
+ agent_type="other",
+ connector_type="claude-code",
+ integration_profile={},
+ client_id=client.id,
+ api_key_id=key.id,
+ owner_id=owner_id,
+ )
+ session.add(agent)
+ await session.commit()
+ return agent.id
+
+
+async def _role(owner, organization_id: str, user_id: str) -> str | None:
+ async with owner() as session:
+ member = await session.get(OrganizationMember, (organization_id, user_id))
+ return None if member is None else member.role
+
+
+async def _keys(owner, tenant_id: str, user_id: str) -> list[str]:
+ async with owner() as session:
+ return list(
+ await session.scalars(
+ select(ApiKey.type).where(
+ ApiKey.tenant_id == tenant_id, ApiKey.user_id == user_id
+ )
+ )
+ )
+
+
+async def _organization_events(owner, organization_id: str) -> list[tuple[str, dict]]:
+ async with owner() as session:
+ rows = await session.scalars(
+ select(OrganizationAuditEvent)
+ .where(OrganizationAuditEvent.organization_id == organization_id)
+ .order_by(OrganizationAuditEvent.occurred_at)
+ )
+ return [(event.action, event.details) for event in rows]
+
+
+async def _workspace_actions(owner, tenant_id: str) -> list[str]:
+ async with owner() as session:
+ return list(
+ await session.scalars(
+ select(AuditEvent.action).where(AuditEvent.tenant_id == tenant_id)
+ )
+ )
+
+
+async def _apply(rls_harness: RLSHarness, world: _World, wanted: str | None) -> None:
+ await apply_from_directory(
+ rls_harness.restricted,
+ _revocation(rls_harness.restricted),
+ organization_id=world.organization_id,
+ user_id=world.ada,
+ wanted=wanted,
+ source="directory_event",
+ )
+
+
+class TestTakingAway:
+ async def test_a_removal_ends_every_access_through_the_organization(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ owner = rls_harness.owner
+ world = await _World.make(owner)
+
+ await _apply(rls_harness, world, None)
+
+ assert await _role(owner, world.organization_id, world.ada) is None
+ async with owner() as session:
+ assert await session.get(TenantMember, (world.joined, world.ada)) is None
+ assert await _keys(owner, world.joined, world.ada) == []
+ assert await _keys(owner, world.reached, world.ada) == []
+ assert await _workspace_actions(owner, world.joined) == [
+ AuditAction.MEMBER_REMOVED
+ ]
+ assert await _workspace_actions(owner, world.reached) == [
+ AuditAction.ORGANIZATION_ADMIN_ACCESS_ENDED
+ ]
+ [(action, details)] = await _organization_events(owner, world.organization_id)
+ assert action == OrganizationAuditAction.MEMBER_REMOVED_FROM_DIRECTORY
+ assert details == {
+ "source": "directory_event",
+ "from_role": "org_admin",
+ "role": None,
+ }
+
+ async def test_a_demotion_ends_what_only_org_admin_reached(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ owner = rls_harness.owner
+ world = await _World.make(owner)
+
+ await _apply(rls_harness, world, "member")
+
+ assert await _role(owner, world.organization_id, world.ada) == "member"
+ assert await _keys(owner, world.joined, world.ada) == ["personal"]
+ assert await _keys(owner, world.reached, world.ada) == []
+ [(action, _)] = await _organization_events(owner, world.organization_id)
+ assert action == OrganizationAuditAction.MEMBER_DEMOTED_FROM_DIRECTORY
+
+ async def test_the_same_role_changes_nothing(self, rls_harness: RLSHarness) -> None:
+ owner = rls_harness.owner
+ world = await _World.make(owner)
+
+ await _apply(rls_harness, world, "org_admin")
+
+ assert await _organization_events(owner, world.organization_id) == []
+ assert await _keys(owner, world.reached, world.ada) == ["personal"]
+
+
+class TestConflicts:
+ """WorkOS is the record, so its change is applied even where it breaks a
+ rule Switch keeps, and what it broke is recorded for the admins."""
+
+ async def test_the_last_org_admin_is_removed_all_the_same(
+ self, rls_harness: RLSHarness, caplog: pytest.LogCaptureFixture
+ ) -> None:
+ owner = rls_harness.owner
+ world = await _World.make(owner, bea_role="member")
+
+ await _apply(rls_harness, world, None)
+
+ assert await _role(owner, world.organization_id, world.ada) is None
+ events = await _organization_events(owner, world.organization_id)
+ assert (
+ OrganizationAuditAction.DIRECTORY_CONFLICT,
+ {"source": "directory_event", "kind": "last_org_admin"},
+ ) in events
+ assert any(
+ record.levelname == "ERROR" and "broke a rule" in record.message
+ for record in caplog.records
+ )
+
+ async def test_agents_stay_with_the_keys_they_run_on(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ owner = rls_harness.owner
+ world = await _World.make(owner)
+ agent_id = await _agent(owner, world.joined, world.ada)
+
+ await _apply(rls_harness, world, None)
+
+ async with owner() as session:
+ agent = await session.get(Agent, agent_id)
+ assert agent is not None and agent.owner_id == world.ada
+ assert await _keys(owner, world.joined, world.ada) == ["agent"]
+ assert (
+ OrganizationAuditAction.DIRECTORY_CONFLICT,
+ {
+ "source": "directory_event",
+ "kind": "agents_left_with_owner",
+ "workspace": world.joined,
+ },
+ ) in await _organization_events(owner, world.organization_id)
+
+ async def test_a_workspace_left_with_no_owner_is_recorded(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ owner = rls_harness.owner
+ world = await _World.make(owner, ada_owns_joined=True)
+
+ await _apply(rls_harness, world, None)
+
+ async with owner() as session:
+ assert await session.get(TenantMember, (world.joined, world.ada)) is None
+ assert (
+ OrganizationAuditAction.DIRECTORY_CONFLICT,
+ {
+ "source": "directory_event",
+ "kind": "workspace_left_without_owner",
+ "workspace": world.joined,
+ },
+ ) in await _organization_events(owner, world.organization_id)
+
+
+class TestGranting:
+ async def test_someone_listed_is_added_with_where_it_came_from(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ owner = rls_harness.owner
+ world = await _World.make(owner)
+ cy = _id("cy")
+ async with owner() as session:
+ session.add(
+ User(id=cy, name=cy, email=f"{cy}@example.invalid", role="user")
+ )
+ await session.commit()
+
+ await apply_from_directory(
+ rls_harness.restricted,
+ _revocation(rls_harness.restricted),
+ organization_id=world.organization_id,
+ user_id=cy,
+ wanted="member",
+ source="daily_check",
+ )
+
+ assert await _role(owner, world.organization_id, cy) == "member"
+ [(action, details)] = await _organization_events(owner, world.organization_id)
+ assert action == OrganizationAuditAction.MEMBER_ADDED_FROM_DIRECTORY
+ assert details == {"source": "daily_check", "role": "member"}
diff --git a/core/tests/switch_core/test_workos_directory.py b/core/tests/switch_core/test_workos_directory.py
index 0dbaebcba..17fd8978a 100644
--- a/core/tests/switch_core/test_workos_directory.py
+++ b/core/tests/switch_core/test_workos_directory.py
@@ -226,3 +226,102 @@ class TestChangesFromSwitch:
workos_organization_id=None, customer_id=None
)
await directory.discard_organization(created)
+
+
+class TestWhatTheMirrorReads:
+ async def test_events_are_asked_for_by_type_from_the_cursor_oldest_first(
+ self,
+ ) -> None:
+ seen: list[httpx.Request] = []
+
+ def handler(request: httpx.Request) -> httpx.Response:
+ seen.append(request)
+ return httpx.Response(
+ 200,
+ json={
+ "data": [
+ {
+ "id": "event_1",
+ "event": "organization_membership.created",
+ "data": {"user_id": "user_A", "organization_id": "org_a"},
+ }
+ ],
+ "list_metadata": {"after": None},
+ },
+ )
+
+ directory = _directory(handler)
+ types = ("organization_membership.created", "organization.deleted")
+
+ first = await directory.events(
+ types, after=None, range_start="2026-11-02T00:00:00+00:00"
+ )
+ await directory.events(types, after="event_1", range_start="unused")
+
+ assert [event["id"] for event in first] == ["event_1"]
+ assert seen[0].url.path == "/events"
+ assert seen[0].url.params.get_list("events") == list(types)
+ assert seen[0].url.params["order"] == "asc"
+ assert seen[0].url.params["range_start"] == "2026-11-02T00:00:00+00:00"
+ assert "after" not in seen[0].url.params
+ assert seen[1].url.params["after"] == "event_1"
+ assert "range_start" not in seen[1].url.params
+
+ async def test_an_event_switch_cannot_read_is_unavailable(self) -> None:
+ directory = _directory(
+ lambda request: httpx.Response(200, json={"data": [{"id": "event_1"}]})
+ )
+
+ with pytest.raises(DirectoryUnavailable):
+ await directory.events(
+ ("organization.deleted",), after=None, range_start="x"
+ )
+
+ async def test_one_membership_is_read_as_it_is_now(self) -> None:
+ def handler(request: httpx.Request) -> httpx.Response:
+ assert request.url.params["user_id"] == "user_A"
+ assert request.url.params["organization_id"] == "org_a"
+ assert request.url.params["statuses"] == "active"
+ return httpx.Response(
+ 200,
+ json={
+ "data": [
+ {"organization_id": "org_a", "role": {"slug": "org_admin"}}
+ ]
+ },
+ )
+
+ assert (
+ await _directory(handler).membership_role("user_A", "org_a") == "org_admin"
+ )
+
+ async def test_no_membership_is_none(self) -> None:
+ directory = _directory(lambda request: httpx.Response(200, json={"data": []}))
+
+ assert await directory.membership_role("user_A", "org_a") is None
+
+ async def test_an_organizations_members_are_read_by_user(self) -> None:
+ def handler(request: httpx.Request) -> httpx.Response:
+ assert request.url.params["organization_id"] == "org_a"
+ return httpx.Response(
+ 200,
+ json={
+ "data": [
+ {
+ "user_id": "user_A",
+ "organization_id": "org_a",
+ "role": {"slug": "org_admin"},
+ },
+ {
+ "user_id": "user_B",
+ "organization_id": "org_a",
+ "role": {"slug": "editor"},
+ },
+ ]
+ },
+ )
+
+ assert await _directory(handler).organization_members("org_a") == {
+ "user_A": "org_admin",
+ "user_B": "member",
+ }
diff --git a/core/tests/switch_core/test_workos_mirror.py b/core/tests/switch_core/test_workos_mirror.py
new file mode 100644
index 000000000..23df2534e
--- /dev/null
+++ b/core/tests/switch_core/test_workos_mirror.py
@@ -0,0 +1,309 @@
+"""The WorkOS mirror: events and the daily check applied to linked
+organizations, against Postgres on the restricted role and a stand-in for
+WorkOS; `owner` only arranges fixtures and reads results."""
+
+from __future__ import annotations
+
+import uuid
+from typing import Any
+
+import pytest
+from sqlalchemy import select
+
+from switch_core.db.audit import OrganizationAuditAction
+from switch_core.db.models import (
+ DirectoryEventCursor,
+ OidcIdentity,
+ Organization,
+ OrganizationAuditEvent,
+ OrganizationMember,
+ User,
+)
+from switch_core.organization_directory import DirectoryUnavailable
+from switch_core.workos_mirror import EVENT_TYPES, WorkOSMirror
+from tests.conftest import RLSHarness
+from tests.switch_core.test_organization_mirror_apply import _revocation
+
+pytestmark = pytest.mark.no_ambient_tenant
+
+ISSUER = "https://auth.example.test"
+
+
+def _id(prefix: str) -> str:
+ return f"{prefix}-{uuid.uuid4().hex[:8]}"
+
+
+class _WorkOS:
+ """Memberships by (WorkOS user, WorkOS organization) -> Switch role, and
+ the events that changed them, served a page of `page` at a time."""
+
+ def __init__(self, page: int = 100) -> None:
+ self.memberships: dict[tuple[str, str], str] = {}
+ self.log: list[dict[str, Any]] = []
+ self.page = page
+ self.asked: list[dict[str, Any]] = []
+ self.fail_role_for: str | None = None
+
+ def change(self, subject: str, organization: str, role: str | None) -> None:
+ if role is None:
+ self.memberships.pop((subject, organization), None)
+ kind = "deleted"
+ else:
+ kind = (
+ "created"
+ if (subject, organization) not in self.memberships
+ else "updated"
+ )
+ self.memberships[(subject, organization)] = role
+ self.log.append(
+ {
+ "id": f"event_{len(self.log):04d}",
+ "event": f"organization_membership.{kind}",
+ "data": {"user_id": subject, "organization_id": organization},
+ }
+ )
+
+ async def events(
+ self, types: tuple[str, ...], *, after: str | None, range_start: str
+ ) -> list[dict[str, Any]]:
+ assert types == EVENT_TYPES
+ self.asked.append({"after": after, "range_start": range_start})
+ start = 0
+ if after is not None:
+ start = next(i for i, e in enumerate(self.log) if e["id"] == after) + 1
+ return self.log[start : start + self.page]
+
+ async def membership_role(self, subject: str, organization: str) -> str | None:
+ if subject == self.fail_role_for:
+ raise DirectoryUnavailable("WorkOS answered 500.")
+ return self.memberships.get((subject, organization))
+
+ async def organization_members(self, organization: str) -> dict[str, str]:
+ return {
+ subject: role
+ for (subject, org), role in self.memberships.items()
+ if org == organization
+ }
+
+
+async def _linked(owner) -> tuple[str, str]:
+ organization_id, workos_id = _id("org"), _id("org_workos")
+ async with owner() as session:
+ session.add(
+ Organization(
+ id=organization_id,
+ slug=organization_id,
+ name="Acme",
+ workos_organization_id=workos_id,
+ )
+ )
+ await session.commit()
+ return organization_id, workos_id
+
+
+async def _person(owner, subject: str | None) -> str:
+ user_id = _id("user")
+ async with owner() as session:
+ session.add(
+ User(
+ id=user_id,
+ name=user_id,
+ email=f"{user_id}@example.invalid",
+ role="user",
+ )
+ )
+ await session.flush()
+ if subject is not None:
+ session.add(OidcIdentity(user_id=user_id, iss=ISSUER, sub=subject))
+ await session.commit()
+ return user_id
+
+
+async def _member(owner, organization_id: str, user_id: str, role: str) -> None:
+ async with owner() as session:
+ session.add(
+ OrganizationMember(
+ organization_id=organization_id, user_id=user_id, role=role
+ )
+ )
+ await session.commit()
+
+
+async def _roles(owner, organization_id: str) -> dict[str, str]:
+ async with owner() as session:
+ rows = await session.execute(
+ select(OrganizationMember.user_id, OrganizationMember.role).where(
+ OrganizationMember.organization_id == organization_id
+ )
+ )
+ return {user_id: role for user_id, role in rows}
+
+
+def _mirror(rls_harness: RLSHarness, workos: _WorkOS) -> WorkOSMirror:
+ return WorkOSMirror(
+ rls_harness.restricted,
+ workos, # type: ignore[arg-type]
+ _revocation(rls_harness.restricted),
+ ISSUER,
+ )
+
+
+class TestEvents:
+ async def test_the_first_poll_starts_from_now_and_later_ones_from_the_cursor(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ owner = rls_harness.owner
+ organization_id, workos_id = await _linked(owner)
+ ada = await _person(owner, "user_ADA")
+ workos = _WorkOS(page=1)
+ mirror = _mirror(rls_harness, workos)
+
+ assert await mirror.poll_once() == 0
+ workos.change("user_ADA", workos_id, "member")
+ workos.change("user_ADA", workos_id, "org_admin")
+ assert await mirror.poll_once() == 2
+
+ assert await _roles(owner, organization_id) == {ada: "org_admin"}
+ assert workos.asked[0]["after"] is None
+ assert workos.asked[-1]["after"] == "event_0001"
+ async with owner() as session:
+ cursor = await session.get(DirectoryEventCursor, "workos")
+ assert cursor is not None and cursor.after == "event_0001"
+ assert await mirror.poll_once() == 0
+
+ async def test_an_event_reads_the_membership_as_it_is_now(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ """Read after the membership has gone, a `created` event removes
+ nothing and adds nothing."""
+ owner = rls_harness.owner
+ organization_id, workos_id = await _linked(owner)
+ await _person(owner, "user_ADA")
+ workos = _WorkOS()
+ mirror = _mirror(rls_harness, workos)
+ await mirror.poll_once()
+ workos.change("user_ADA", workos_id, "member")
+ workos.change("user_ADA", workos_id, None)
+
+ await mirror.poll_once()
+
+ assert await _roles(owner, organization_id) == {}
+
+ async def test_a_removal_is_applied(self, rls_harness: RLSHarness) -> None:
+ owner = rls_harness.owner
+ organization_id, workos_id = await _linked(owner)
+ ada = await _person(owner, "user_ADA")
+ bea = await _person(owner, "user_BEA")
+ await _member(owner, organization_id, ada, "member")
+ await _member(owner, organization_id, bea, "org_admin")
+ workos = _WorkOS()
+ workos.memberships[("user_ADA", workos_id)] = "member"
+ mirror = _mirror(rls_harness, workos)
+ await mirror.poll_once()
+ workos.change("user_ADA", workos_id, None)
+
+ await mirror.poll_once()
+
+ assert await _roles(owner, organization_id) == {bea: "org_admin"}
+
+ async def test_events_for_other_organizations_and_strangers_are_skipped(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ owner = rls_harness.owner
+ organization_id, workos_id = await _linked(owner)
+ await _person(owner, "user_ADA")
+ workos = _WorkOS()
+ mirror = _mirror(rls_harness, workos)
+ await mirror.poll_once()
+ workos.change("user_ADA", _id("org_elsewhere"), "org_admin")
+ workos.change("user_NOBODY", workos_id, "org_admin")
+
+ assert await mirror.poll_once() == 2
+
+ assert await _roles(owner, organization_id) == {}
+
+ async def test_an_event_that_fails_is_read_again_next_time(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ owner = rls_harness.owner
+ organization_id, workos_id = await _linked(owner)
+ ada = await _person(owner, "user_ADA")
+ workos = _WorkOS()
+ mirror = _mirror(rls_harness, workos)
+ await mirror.poll_once()
+ workos.change("user_ADA", workos_id, "member")
+ workos.fail_role_for = "user_ADA"
+
+ with pytest.raises(DirectoryUnavailable):
+ await mirror.poll_once()
+ workos.fail_role_for = None
+ assert await mirror.poll_once() == 1
+
+ assert await _roles(owner, organization_id) == {ada: "member"}
+
+ async def test_a_deleted_organization_removes_nobody_and_is_reported(
+ self, rls_harness: RLSHarness
+ ) -> None:
+ owner = rls_harness.owner
+ organization_id, workos_id = await _linked(owner)
+ ada = await _person(owner, "user_ADA")
+ await _member(owner, organization_id, ada, "org_admin")
+ workos = _WorkOS()
+ mirror = _mirror(rls_harness, workos)
+ await mirror.poll_once()
+ workos.log.append(
+ {
+ "id": "event_9999",
+ "event": "organization.deleted",
+ "data": {"id": workos_id},
+ }
+ )
+
+ await mirror.poll_once()
+
+ assert await _roles(owner, organization_id) == {ada: "org_admin"}
+ async with owner() as session:
+ [event] = await session.scalars(
+ select(OrganizationAuditEvent).where(
+ OrganizationAuditEvent.organization_id == organization_id
+ )
+ )
+ assert event.action == OrganizationAuditAction.DIRECTORY_CONFLICT
+ assert event.details["kind"] == "directory_organization_deleted"
+
+
+class TestDailyCheck:
+ async def test_brings_every_linked_organization_in_line(
+ self, rls_harness: RLSHarness, caplog: pytest.LogCaptureFixture
+ ) -> None:
+ owner = rls_harness.owner
+ organization_id, workos_id = await _linked(owner)
+ ada = await _person(owner, "user_ADA") # listed, not in Switch
+ bea = await _person(owner, "user_BEA") # in Switch, no longer listed
+ cy = await _person(owner, "user_CY") # org_admin in Switch, member there
+ dee = await _person(owner, None) # in Switch, no WorkOS identity
+ eve = await _person(owner, "user_EVE") # stays org_admin
+ for user_id, role in (
+ (bea, "member"),
+ (cy, "org_admin"),
+ (dee, "member"),
+ (eve, "org_admin"),
+ ):
+ await _member(owner, organization_id, user_id, role)
+ workos = _WorkOS()
+ workos.memberships = {
+ ("user_ADA", workos_id): "member",
+ ("user_CY", workos_id): "member",
+ ("user_EVE", workos_id): "org_admin",
+ ("user_STRANGER", workos_id): "member",
+ }
+
+ await _mirror(rls_harness, workos).check_once()
+
+ assert await _roles(owner, organization_id) == {
+ ada: "member",
+ cy: "member",
+ dee: "member",
+ eve: "org_admin",
+ }
+ assert f"User {dee} is in organization" in caplog.text
diff --git a/docs/old/workos-directory.md b/docs/old/workos-directory.md
index f55514230..c00320fbf 100644
--- a/docs/old/workos-directory.md
+++ b/docs/old/workos-directory.md
@@ -55,19 +55,53 @@ linked organization:
Both are recorded in the organization's audit log. If WorkOS cannot be read,
or Switch's rows cannot be written, the sign-in goes ahead with Switch's rows
-as they were, and the failure is logged.
+as they were, and the failure is logged. A sign-in never takes anything away;
+the mirror does.
+
+## The mirror
+
+One switch-core process follows WorkOS's Events API, every 30 seconds, for
+changes to a linked organization's memberships, and applies them. It starts
+from when it first runs; once a day, and whenever it starts, it also compares
+every linked organization's members with WorkOS and applies the difference,
+which catches up anything from before it ran or missed since. It keeps its
+place in the database, so a restart resumes where it stopped.
+
+For each person it matches (by the WorkOS user their account signs in as):
+
+- **Added or made `org_admin` in WorkOS:** added, or made Org Admin, as at
+ sign-in.
+- **No longer `org_admin`:** made a Member. What they held in workspaces they
+ stood in only as Org Admin (personal API keys, service connections,
+ sign-ins under way) is removed.
+- **Removed:** removed from the organization and from each of its workspaces,
+ with their personal API keys, service connections and sign-ins under way
+ there.
+
+Each change is recorded in the organization's audit log, and each workspace
+change in that workspace's. WorkOS is the record, so a change is applied even
+when it breaks a rule Switch otherwise keeps. It is then logged at `error` and
+recorded in the organization's audit log as "WorkOS change broke a Switch
+rule", naming what:
+
+- the organization's last Org Admin was removed or demoted;
+- agents the person owns were left in place, still running on their keys,
+ owned by someone who can no longer act there, for an admin to delete;
+- a workspace was left with no owner (its organization's Org Admins still
+ administer it).
+
+Someone WorkOS lists who has never signed in to Switch is added when they
+first do. A member with no WorkOS identity (an account nobody linked) can't be
+compared, and is logged and left. Deleting a linked organization in WorkOS
+removes nobody: it is logged and recorded for an operator to deal with.
## Not done yet
-- **Removals and demotions are not applied.** If someone is removed from the
- linked WorkOS organization, or is no longer its `org_admin`, Switch keeps
- their membership and role, and with them their access. Each sign-in by that
- person logs it at `error`. Until this is built, an Org Admin ends such
- access in Switch as well, from the organization's Members page: removing a
- member and taking Org Admin away are allowed on a linked organization, and
- change only Switch. Whatever WorkOS still grants comes back at the person's
- next sign-in.
- **Granting from Switch is refused** for a linked organization: adding a
member (from the dashboard or by invitation) or making one Org Admin. Make
- those changes in WorkOS.
+ those changes in WorkOS. Removing a member or taking Org Admin away from
+ Switch is allowed and changes only Switch; whatever WorkOS still grants
+ comes back at the next sign-in or check.
+- The organization's admins see conflicts only in its audit log, not yet as a
+ notice on the dashboard.
- A new organization created on the server is Switch's own until it is linked.
diff --git a/gateway/src/pages/organization/OrganizationAuditSection.tsx b/gateway/src/pages/organization/OrganizationAuditSection.tsx
index 8f082f5f3..e5041c6fe 100644
--- a/gateway/src/pages/organization/OrganizationAuditSection.tsx
+++ b/gateway/src/pages/organization/OrganizationAuditSection.tsx
@@ -18,8 +18,11 @@ const ACTION_LABELS: Record<string, string> = {
"organization.connector_app_removed": "Connector app removed",
"organization.service_set": "Service switched",
"organization.service_cleared": "Service left to workspaces",
- "organization.member_added_at_sign_in": "Member added at sign-in",
- "organization.member_promoted_at_sign_in": "Member made Org Admin at sign-in",
+ "organization.member_added_from_directory": "Member added from WorkOS",
+ "organization.member_promoted_from_directory": "Member made Org Admin from WorkOS",
+ "organization.member_demoted_from_directory": "Org Admin made Member from WorkOS",
+ "organization.member_removed_from_directory": "Member removed from WorkOS",
+ "organization.directory_conflict": "WorkOS change broke a Switch rule",
"organization.linked_to_directory": "Linked to WorkOS",
};