Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.
Python agent-orchestrationaiai-agentschatopsdiscordlangchainmcpmicrosoft-teamsmulti-agentopen-sourceopenaiself-hostedslack
feat(db): let an install end, and a workspace be installed again (CHOO-2626)
Petr Bauchd054494ef414
diff --git a/core/switch_core/db/models.py b/core/switch_core/db/models.pyindex ddc33ca2d..2099d6ed6 100644--- a/core/switch_core/db/models.py+++ b/core/switch_core/db/models.py@@ -1145,19 +1145,33 @@ class MessagingInstall(TenantScoped, Base):token, and what it may do.**`(platform, external_workspace_id)` is unique across the whole- deployment, not per tenant**, and that is the single most important line- here. Inbound events arrive over one public endpoint carrying a workspace- id and no tenant, so a workspace claimed by two tenants is a message with- two possible destinations and no way to choose — which is the failure this- whole phase exists to make unrepresentable. The database decides it rather- than a read-then-insert in application code, because the check and the- write cannot be made atomic from outside.-- That constraint is also the one place a tenant learns something about- another: claiming a workspace somebody else already claimed fails, and the- failure says so. It is the right answer — the alternative is a silent- second claim — and what it discloses is that *some* tenant holds a- workspace the caller was already able to name.+ deployment among rows that are still `active`**, and that is the single+ most important line here. Inbound events arrive over one public endpoint+ carrying a workspace id and no tenant, so a workspace claimed by two+ tenants is a message with two possible destinations and no way to choose —+ which is the failure this whole phase exists to make unrepresentable. The+ database decides it rather than a read-then-insert in application code,+ because the check and the write cannot be made atomic from outside.++ It is a *partial* index rather than a plain constraint because an install+ has to be able to end. A customer who removes the app in Slack, or an+ operator who disconnects it here, leaves a row behind — and a row that+ still occupied the workspace would mean nobody could ever install that+ workspace again, including the customer who just removed it. Ending an+ install therefore frees the workspace, and keeps the record of the one+ that ended.++ That index is also the one place a tenant learns something about another:+ claiming a workspace somebody else already holds fails, and the failure+ says so. It is the right answer — the alternative is a silent second claim+ — and what it discloses is that *some* tenant holds a workspace the caller+ was already able to name.++ `status` is `active`, `disconnected` (an operator here ended it) or+ `revoked` (the platform told us it was over). The two endings are recorded+ apart because they call for different things: one is somebody's decision+ and the other is news, and an operator looking at a bridge that stopped+ working needs to know which.`bridge_id` is nullable because the install row is written before anythingis built on it, and because removing a bridge should not force the@@ -1167,7 +1181,10 @@ class MessagingInstall(TenantScoped, Base):`encrypted_bot_token` uses the same key as every other credential thisschema stores (`crypto.encrypt_token` over the configured secret), so itis protected against a stolen dump and not against a compromised process.- A per-tenant key is a stronger boundary and a later decision.+ A per-tenant key is a stronger boundary and a later decision. It is+ nullable so that an install which has ended can keep its record without+ keeping its secret: the token is worthless by then, and a worthless+ credential still reads like a credential to whoever finds the dump.`scopes` is the platform's own spelling of what was granted, storedverbatim rather than parsed into a list — a scope string that means@@ -1177,10 +1194,12 @@ class MessagingInstall(TenantScoped, Base):__tablename__ = "messaging_installs"__table_args__ = (- UniqueConstraint(+ Index(+ "uq_messaging_installs_workspace","platform","external_workspace_id",- name="uq_messaging_installs_workspace",+ unique=True,+ postgresql_where=text("status = 'active'"),),UniqueConstraint("id", "tenant_id", name="uq_messaging_installs_id_tenant"),ForeignKeyConstraint(@@ -1193,7 +1212,7 @@ class MessagingInstall(TenantScoped, Base):id: Mapped[str] = mapped_column(Text, primary_key=True, default=_uuid)platform: Mapped[str] = mapped_column(Text, nullable=False)external_workspace_id: Mapped[str] = mapped_column(Text, nullable=False)- encrypted_bot_token: Mapped[str] = mapped_column(Text, nullable=False)+ encrypted_bot_token: Mapped[str | None] = mapped_column(Text, nullable=True)scopes: Mapped[str] = mapped_column(Text, nullable=False)status: Mapped[str] = mapped_column(Text, nullable=False)installed_by_user_id: Mapped[str] = mapped_column(@@ -1203,6 +1222,9 @@ class MessagingInstall(TenantScoped, Base):installed_at: Mapped[str] = mapped_column(DateTime(timezone=True), server_default=func.now(), nullable=False)+ ended_at: Mapped[datetime | None] = mapped_column(+ DateTime(timezone=True), nullable=True+ )class MessagingInstallState(TenantScoped, Base):diff --git a/core/switch_core/db/stores/messaging_install_store.py b/core/switch_core/db/stores/messaging_install_store.pyindex 269560850..b91a6d2a5 100644--- a/core/switch_core/db/stores/messaging_install_store.py+++ b/core/switch_core/db/stores/messaging_install_store.py@@ -18,6 +18,18 @@ from switch_core.db.models import (#: is worthless by the time anyone reads it.STATE_TTL = timedelta(minutes=10)+#: An install that is serving. The only status the workspace uniqueness index+#: covers, so it is also the answer to "who holds this workspace".+INSTALL_ACTIVE = "active"++#: Ended here, by someone who decided to. The bridge is gone with it.+INSTALL_DISCONNECTED = "disconnected"++#: Ended there: the platform told us the app was removed or its token killed.+#: Distinct from `disconnected` because an operator whose bridge stopped+#: working needs to know whether it was news or a decision.+INSTALL_REVOKED = "revoked"+class MessagingInstallClaimedError(RuntimeError):"""Another tenant has already installed the app into this workspace.@@ -103,17 +115,21 @@ class MessagingInstallStore:"""Claim a workspace for the bound tenant.The claim is the insert: `(platform, external_workspace_id)` is unique- across the deployment, so the database decides who holds a workspace- rather than a read followed by a write that cannot be made atomic with- it. A second tenant's install therefore fails here, loudly, instead of- producing an event with two possible destinations.+ across the deployment among active rows, so the database decides who+ holds a workspace rather than a read followed by a write that cannot be+ made atomic with it. A second tenant's install therefore fails here,+ loudly, instead of producing an event with two possible destinations.++ Installs that have ended are not in the index, so re-installing a+ workspace somebody released is an ordinary insert and needs no check of+ its own."""install = MessagingInstall(platform=platform,external_workspace_id=external_workspace_id,encrypted_bot_token=encrypted_bot_token,scopes=scopes,- status="active",+ status=INSTALL_ACTIVE,installed_by_user_id=user_id,)session.add(install)@@ -124,26 +140,33 @@ class MessagingInstallStore:raiseraise MessagingInstallClaimedError(f"the {platform} workspace {external_workspace_id} is already "- "connected to Switch. Remove the existing install before "- "connecting it again."+ "connected to Switch. Disconnect the existing install — from "+ "the organisation that holds it, which may not be yours — "+ "before connecting it again.") from excreturn installasync def get_for_workspace(self, session: AsyncSession, *, platform: str, external_workspace_id: str) -> MessagingInstall | None:- """The bound tenant's install of one workspace, if it is theirs.-- Deliberately still scoped, even though the unique constraint means at- most one row exists deployment-wide: the caller is an inbound webhook- that resolved a tenant from the workspace a moment ago, and this- re-reading it under RLS is what makes a mistake there a miss rather- than a cross-tenant read.+ """The bound tenant's live install of one workspace, if it is theirs.++ Deliberately still scoped, even though the unique index means at most+ one active row exists deployment-wide: the caller is an inbound webhook+ that resolved a tenant from the workspace a moment ago, and re-reading+ it under RLS is what makes a mistake there a miss rather than a+ cross-tenant read.++ The status predicate matches the index's and `tenant_of_messaging_+ install`'s. Without it a workspace installed, removed and installed+ again returns two rows and this raises — an outage for the live+ install, caused by the ended one."""result = await session.execute(select(MessagingInstall).where(MessagingInstall.platform == platform,MessagingInstall.external_workspace_id == external_workspace_id,+ MessagingInstall.status == INSTALL_ACTIVE,))return result.scalars().one_or_none()diff --git a/core/switch_core/db/tenant_lookup.py b/core/switch_core/db/tenant_lookup.pyindex adcd78dfd..371a457da 100644--- a/core/switch_core/db/tenant_lookup.py+++ b/core/switch_core/db/tenant_lookup.py@@ -122,6 +122,13 @@ lookup on the id alone would answer twice the first time two platformshappened to mint the same string, and refuse a customer's traffic for areason in someone else's account.+It also filters on `status = 'active'`, which is not a refinement but the+other half of the same uniqueness. Only active rows are unique by index, so a+workspace that was installed, removed and installed again has two rows and+this would otherwise answer twice — refusing a customer's live traffic on the+strength of an install they themselves ended. The predicate here and the one+on the index are the same predicate, and have to stay that way.+Why not the obvious alternatives is argued in`docs/old/multi-tenancy-phase1-db.md`, "The bootstrap problem"; the shortversion is that returning rows instead of tenant ids would put a second copy@@ -306,14 +313,17 @@ TENANT_LOOKUPS: tuple[TenantLookup, ...] = (query=("SELECT tenant_id FROM messaging_installs ""WHERE platform = p_platform "- "AND external_workspace_id = p_external_workspace_id"+ "AND external_workspace_id = p_external_workspace_id "+ "AND status = 'active'"),purpose=("Which tenant an inbound event from an installed workspace belongs ""to. The public webhook is unauthenticated by nature and knows only ""the platform it was posted to and the workspace the payload names, "- "so this runs before anything else the request does. Unique by "- "constraint on exactly this pair, so it answers at most once."+ "so this runs before anything else the request does. The status "+ "predicate is the index's own: only active rows are unique, and an "+ "install that ended must stop answering for its workspace rather "+ "than compete with the one that replaced it."),),)diff --git a/core/switch_core/migrations/versions/a7f2c3e9b481_messaging_installs_can_end.py b/core/switch_core/migrations/versions/a7f2c3e9b481_messaging_installs_can_end.pynew file mode 100644index 000000000..98a1c950d--- /dev/null+++ b/core/switch_core/migrations/versions/a7f2c3e9b481_messaging_installs_can_end.py@@ -0,0 +1,108 @@+"""an install that can end, and a workspace that can be installed again++`c8a4e21f6d30` gave `messaging_installs` a plain unique constraint on+`(platform, external_workspace_id)`, on the grounds that a workspace claimed+by two tenants is an event with two destinations. That grounds is still right;+the constraint was too strong for it. It made a claim permanent — the row it+protects is never deleted, so a customer who removed the Switch app from their+own Slack could not put it back, and the error they got told them to "remove+the existing install" through a path that did not exist.++What actually has to be unique is the set of installs that are *serving*. So+the constraint becomes a unique index over `status = 'active'` under the same+name, and ending an install frees the workspace while keeping the record of+what happened to it.++`tenant_of_messaging_install` is redefined against the same predicate, and+that is not a tidy-up. The function resolves a workspace to a tenant for+traffic nobody has authenticated, and it was written to rely on the old+constraint answering at most once. Left alone, the first workspace to be+installed twice would make it answer twice and the caller refuses an ambiguous+answer — so the customer's live install would stop receiving events because of+one they had themselves ended.++Two column changes come with it. `ended_at` records when, alongside the+`status` that says which of the two ways. And `encrypted_bot_token` becomes+nullable so an ended install can keep its record without keeping its secret:+the token is dead by then, and a dead credential still reads like a credential+to whoever finds the dump.++The lookup DDL below is a verbatim copy of `switch_core/db/tenant_lookup.py`+as it stood when this migration was written, copied rather than imported for+the reason every revision in this chain copies: a migration records a change+that already happened, and importing the live module would let a later edit+silently change what this one means.+`tests/switch_core/db/test_frozen_ddl_matches_create_all.py` is what keeps the+copy from drifting.++Revision ID: a7f2c3e9b481+Revises: d3f6b0c95a17+Create Date: 2026-09-15 00:00:00.000000++"""++from collections.abc import Sequence++import sqlalchemy as sa+from alembic import op++revision: str = "a7f2c3e9b481"+down_revision: str | None = "d3f6b0c95a17"+branch_labels: str | Sequence[str] | None = None+depends_on: str | Sequence[str] | None = None++TABLE = "messaging_installs"+INDEX_NAME = "uq_messaging_installs_workspace"++SECURE_SEARCH_PATH = "pg_catalog, public, pg_temp"++CREATE_TENANT_OF_MESSAGING_INSTALL = f"""CREATE OR REPLACE FUNCTION tenant_of_messaging_install(p_platform text, p_external_workspace_id text)+ RETURNS SETOF text+ LANGUAGE sql STABLE SECURITY DEFINER+ SET search_path = {SECURE_SEARCH_PATH}+AS $$SELECT tenant_id FROM messaging_installs WHERE platform = p_platform AND external_workspace_id = p_external_workspace_id AND status = 'active'$$"""++RESTORE_TENANT_OF_MESSAGING_INSTALL = f"""CREATE OR REPLACE FUNCTION tenant_of_messaging_install(p_platform text, p_external_workspace_id text)+ RETURNS SETOF text+ LANGUAGE sql STABLE SECURITY DEFINER+ SET search_path = {SECURE_SEARCH_PATH}+AS $$SELECT tenant_id FROM messaging_installs WHERE platform = p_platform AND external_workspace_id = p_external_workspace_id$$"""+++def upgrade() -> None:+ # Same name as the constraint it replaces, deliberately: the store reads+ # the name out of the integrity error to tell a claimed workspace from any+ # other write that failed, and Postgres reports a unique index by the same+ # name a unique constraint would have carried.+ op.drop_constraint(INDEX_NAME, TABLE, type_="unique")+ op.create_index(+ INDEX_NAME,+ TABLE,+ ["platform", "external_workspace_id"],+ unique=True,+ postgresql_where=sa.text("status = 'active'"),+ )+ op.add_column(+ TABLE, sa.Column("ended_at", sa.DateTime(timezone=True), nullable=True)+ )+ op.alter_column(+ TABLE, "encrypted_bot_token", existing_type=sa.Text(), nullable=True+ )+ op.execute(CREATE_TENANT_OF_MESSAGING_INSTALL)+++def downgrade() -> None:+ op.execute(RESTORE_TENANT_OF_MESSAGING_INSTALL)+ op.alter_column(+ TABLE, "encrypted_bot_token", existing_type=sa.Text(), nullable=False+ )+ op.drop_column(TABLE, "ended_at")+ op.drop_index(INDEX_NAME, table_name=TABLE)+ # Both of the statements above and this one fail on data the old schema+ # cannot hold: an ended install whose token was discarded, and a workspace+ # holding more than one row. That is the honest outcome — going back means+ # choosing which of those installs survives, and a migration is not the+ # thing that gets to choose.+ op.create_unique_constraint(+ INDEX_NAME, TABLE, ["platform", "external_workspace_id"]+ )diff --git a/core/tests/switch_core/db/test_messaging_install_claim.py b/core/tests/switch_core/db/test_messaging_install_claim.pyindex 1319744ae..c1af5cd94 100644--- a/core/tests/switch_core/db/test_messaging_install_claim.py+++ b/core/tests/switch_core/db/test_messaging_install_claim.py@@ -19,16 +19,26 @@ That refusal discloses one bit — some tenant holds this workspace — to a calwho could already name it. That is deliberate, and the alternative is worse: asilent second claim, discovered when a customer's messages start arriving insomebody else's rooms.++The claim covers **active** installs only, and the second half of this file is+about that. A claim that outlived the install would mean a workspace could be+connected once ever — including by the customer who had just disconnected it —+so ending an install releases the workspace. The lookup has to agree with the+index about which rows count, and the tests below measure that agreement from+both directions: a released workspace can be claimed again, and a workspace+with history resolves to whoever holds it now rather than refusing because+two rows name it."""from __future__ import annotationsimport uuid+from datetime import UTC, datetimeimport pytest-from sqlalchemy import select+from sqlalchemy import select, updatefrom sqlalchemy.exc import IntegrityError-from sqlalchemy.ext.asyncio import async_sessionmaker+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmakerfrom switch_core.db.models import MessagingInstall, Tenant, Userfrom switch_core.db.session_scope import tenant_session@@ -76,6 +86,20 @@ def _install(fixture: _Fixture, tenant_id: str) -> MessagingInstall:)+async def _end(session: AsyncSession, install_id: str, status: str) -> None:+ """End an install the way the store will, without depending on it yet.++ A direct write, because what is under test here is the index and the+ lookup: the properties have to hold for any row in that state, not only+ for rows a particular method happened to produce.+ """+ await session.execute(+ update(MessagingInstall)+ .where(MessagingInstall.id == install_id)+ .values(status=status, ended_at=datetime.now(UTC), encrypted_bot_token=None)+ )++async def test_a_second_tenant_cannot_claim_a_claimed_workspace(rls_harness: RLSHarness,) -> None:@@ -149,3 +173,96 @@ async def test_the_same_workspace_on_another_platform_is_a_separate_claim()== fixture.tenant_b)+++async def test_an_install_that_ended_releases_the_workspace(+ rls_harness: RLSHarness,+) -> None:+ """The claim lasts as long as the install and not a moment longer.++ A permanent claim is not a stricter version of this guarantee, it is a+ different and wrong one: a customer who tries Switch, disconnects, and+ comes back finds their own workspace held by a row nobody can see and+ nobody can release.+ """+ fixture = await _two_tenants_and_a_workspace(rls_harness.owner)++ async with tenant_session(rls_harness.restricted, fixture.tenant_a) as session:+ install = _install(fixture, fixture.tenant_a)+ session.add(install)+ await session.flush()+ await _end(session, install.id, "disconnected")+ await session.commit()++ async with tenant_session(rls_harness.restricted, fixture.tenant_b) as session:+ session.add(_install(fixture, fixture.tenant_b))+ await session.commit()++ assert (+ await tenant_of_messaging_install(+ rls_harness.restricted, "slack", fixture.workspace+ )+ == fixture.tenant_b+ )+++async def test_the_lookup_does_not_answer_twice_for_a_workspace_with_history(+ rls_harness: RLSHarness,+) -> None:+ """The failure the status predicate exists to prevent.++ The lookup refuses an ambiguous answer rather than picking one, which is+ right — but it makes a second row for the same workspace an outage for the+ live install rather than a stale record. So the predicate on the function+ has to be the index's predicate, and this measures it with enough history+ to catch a lookup that merely takes the first row: two ended installs and+ one live one, inserted in that order.+ """+ fixture = await _two_tenants_and_a_workspace(rls_harness.owner)++ for _ in range(2):+ async with tenant_session(rls_harness.restricted, fixture.tenant_a) as session:+ install = _install(fixture, fixture.tenant_a)+ session.add(install)+ await session.flush()+ await _end(session, install.id, "revoked")+ await session.commit()++ async with tenant_session(rls_harness.restricted, fixture.tenant_a) as session:+ session.add(_install(fixture, fixture.tenant_a))+ await session.commit()++ assert (+ await tenant_of_messaging_install(+ rls_harness.restricted, "slack", fixture.workspace+ )+ == fixture.tenant_a+ )+++async def test_a_workspace_nobody_holds_any_longer_resolves_to_nobody(+ rls_harness: RLSHarness,+) -> None:+ """An ended install must not go on routing the workspace's traffic.++ The app can still be sitting in the customer's Slack after a disconnect+ here, posting events at us for as long as someone leaves it there. Those+ events belong to no tenant now, and the lookup saying so is what turns+ them into a refusal instead of a delivery into rooms the customer has+ stopped paying for.+ """+ fixture = await _two_tenants_and_a_workspace(rls_harness.owner)++ async with tenant_session(rls_harness.restricted, fixture.tenant_a) as session:+ install = _install(fixture, fixture.tenant_a)+ session.add(install)+ await session.flush()+ await _end(session, install.id, "disconnected")+ await session.commit()++ assert (+ await tenant_of_messaging_install(+ rls_harness.restricted, "slack", fixture.workspace+ )+ is None+ )diff --git a/core/tests/switch_core/db/test_tenant_lookup.py b/core/tests/switch_core/db/test_tenant_lookup.pyindex 5e39f0bf9..a68e939ef 100644--- a/core/tests/switch_core/db/test_tenant_lookup.py+++ b/core/tests/switch_core/db/test_tenant_lookup.py@@ -103,6 +103,15 @@ _ADDED_SINCE = {"tenant_of_messaging_install": "c8a4e21f6d30",}+# A third kind of change, and the quietest: a lookup whose body a later+# revision replaced. Creation and removal both show up as a function that is+# there or is not; a redefinition leaves a function of the right name and+# signature answering a different question, which nothing about the shape of+# the schema reveals. So the revision that last wrote the body is named here+# and compared against the module, while the creating revision above goes on+# owning the drop.+_REDEFINED_SINCE = {"tenant_of_messaging_install": "a7f2c3e9b481"}+def _revision_module(revision: str) -> ModuleType:"""One revision module, loaded through Alembic.@@ -722,17 +731,24 @@ class TestTheMigrationInstallsTheSameThing:Driven by `_ADDED_SINCE` rather than naming one lookup, so the nextentry is covered by adding it there and nowhere else.++ The body is compared against `_REDEFINED_SINCE` where there is an+ entry, because the revision that creates a function is not always the+ one that last says what it does. The drop is still the creating+ revision's: a redefinition replaces a body and leaves the function it+ replaced nothing to undo."""for name, revision in _ADDED_SINCE.items():lookup = TENANT_LOOKUPS_BY_NAME[name]- module = _revision_module(revision)+ defining = _REDEFINED_SINCE.get(name, revision)+ module = _revision_module(defining)assert getattr(module, f"CREATE_{name.upper()}") == create_lookup_ddl(lookup), (- f"revision {revision} would install {name} with different DDL "+ f"revision {defining} would install {name} with different DDL ""from the one db/tenant_lookup.py builds.")- assert getattr(module, f"DROP_{name.upper()}") == (+ assert getattr(_revision_module(revision), f"DROP_{name.upper()}") == (f"DROP FUNCTION IF EXISTS {lookup.signature}")