Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.
Python agent-orchestrationaiai-agentschatopsdiscordlangchainmcpmicrosoft-teamsmulti-agentopen-sourceopenaiself-hostedslack
refactor: delete Matrix (CHOO-1436) (#380)
Louis Amaudruzcc60349c5d59
diff --git a/.env.example b/.env.exampleindex 182948124..84abcb581 100644--- a/.env.example+++ b/.env.example@@ -40,13 +40,10 @@ DB_USER=postgresDB_PASSWORD=DB_NAME=switch-# ── Matrix homeserver (Tuwunel) ──────────────────────────────────────────────-MATRIX_SERVER=http://localhost:8008+# ── Client identity ──────────────────────────────────────────────────────────+# The server half of every client's `@localpart:server` id. Nothing is+# contacted at it; the ids are stable public handles.MATRIX_SERVER_NAME=localhost-MATRIX_ADMIN_USER=admin-MATRIX_ADMIN_PASSWORD=-# Must match the homeserver's registration_shared_secret (TUWUNEL_REGISTRATION_SHARED_SECRET).-MATRIX_REGISTRATION_SHARED_SECRET=# ── Agent registration ───────────────────────────────────────────────────────AGENT_REGISTRATION_TOKEN=diff --git a/CLAUDE.md b/CLAUDE.mdindex 3614396f0..c43cd5028 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -22,7 +22,7 @@ history — a later commit cannot take it back. Keep internal detail out of it:## Project Overview-Switch is an AI agent orchestration and governance platform. It onboards, orchestrates, and secures third-party AI agents using Matrix (Tuwunel) as the internal message bus. Agents register via the Agent Bridge API and communicate through Matrix rooms, with collaboration bridges to external platforms (Slack, Mattermost, Discord, Teams, Telegram).+Switch is an AI agent orchestration and governance platform. It onboards, orchestrates, and secures third-party AI agents using a Postgres-backed message store as the internal message bus. Agents register via the Agent Bridge API and communicate through Switch rooms, with collaboration bridges to external platforms (Slack, Mattermost, Discord, Teams, Telegram).The target architecture is documented in `docs/`.@@ -77,7 +77,7 @@ just test -k "test_name" # run specific test- `stores/` — query methods and domain-specific data access- `migrations/` — Alembic migrations (`env.py`, `versions/`)- `room_service.py` / `rooms_yaml.py` — Room lifecycle, configuration, provisioning-- `clients/` — Matrix clients (agent, user, admin, bridge, resource manager)+- `clients/` — room clients (agent, admin, bridge)- `bridges/` — External integrations- `agent/` — Agent Bridge (HTTP API, MCP server, server-side connectors)- `collaboration/` — Collaboration Bridge (Slack, Mattermost, Discord, Teams, Telegram adapters)@@ -85,10 +85,10 @@ just test -k "test_name" # run specific test- `gateway/` — Management API for the frontend**Key patterns:**-- Async throughout: all I/O is async (Matrix, DB, external APIs)+- Async throughout: all I/O is async (DB, external APIs)- Dependency injection: stores and services are injected, not global singletons- Session management: API endpoints use middleware-provided sessions; background work creates sessions explicitly-- All participants in rooms are Matrix clients (matrix-nio) connecting to Tuwunel+- All participants in rooms are clients reading and writing the `messages` table through the transport port## Connector plugins@@ -212,7 +212,7 @@ Concrete rules:- **Missing required config → raise immediately.** Don't log-and-skip when a value is needed for the system to function. If the admin password or shared secret is absent, that's a startup error, not a "skip this step" situation.- **Don't return booleans for operations that can fail.** Raise a descriptive exception. Callers should not have to check `if not result:` — they should get an error they can't ignore.- **`logger.info` is not an error signal.** Use `logger.warning` for degraded-but-functional, `logger.error` for broken-but-continuing, and raise for broken-and-should-stop.-- **Catch-and-log is acceptable in event loops** (e.g., Matrix sync callbacks) where one bad event should not crash the client. Everywhere else, let exceptions propagate.+- **Catch-and-log is acceptable in event loops** (e.g. a transport's delivery loop) where one bad event should not crash the client. Everywhere else, let exceptions propagate.## Testingdiff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex a7155e08d..8d17de9ee 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -22,7 +22,7 @@ just run # run switch-core locally (:8000)just gateway-dev # run the gateway frontend (separate terminal)```-`just up` starts the **supporting services** in Docker — Tuwunel, PostgreSQL and+`just up` starts the **supporting services** in Docker — PostgreSQL andMattermost. You run **switch-core** and the **gateway** yourself so you gethot-reload while developing: `just run` (switch-core on `:8000`) and`just gateway-dev` (the frontend), each in its own terminal. Stop the stack with@@ -39,7 +39,7 @@ Run `just` with no arguments to list every recipe. The most-used ones:|---|---|| `just init-env` | Generate `.env` with freshly generated secrets (no default login) || `just up` / `just down` | Start / stop the local dev stack |-| `just reset` | Stop the stack and wipe volumes (incl. the Tuwunel database) |+| `just reset` | Stop the stack and wipe volumes || `just standalone-up` / `just standalone-down` | Start / stop the full standalone stack (no dev tooling) || `just standalone-reset` | Stop the standalone stack and wipe its volumes || `just run` | Run switch-core locally (`python -m switch_core.main`) |diff --git a/README.md b/README.mdindex 171fe0e96..67e76c261 100644--- a/README.md+++ b/README.md@@ -194,14 +194,14 @@ Read [hosting remotely](https://docs.flintai.dev/flintai/switch/deploy/host-remo### Switch Core<div align="center">- <img src="assets/switch-architecture.png" alt="Switch Core sits between human messaging apps and AI agents: a collaboration bridge relays Slack, Teams, Discord, Telegram and Mattermost; an agent bridge serves the HTTP API and MCP server to agents; both meet at a Tuwunel Matrix homeserver, with a room service, gateway API, PostgreSQL and the operator dashboard alongside" width="800">+ <img src="assets/switch-architecture.png" alt="Switch Core sits between human messaging apps and AI agents: a collaboration bridge relays Slack, Teams, Discord, Telegram and Mattermost; an agent bridge serves the HTTP API and MCP server to agents; both meet at the room service, with PostgreSQL carrying every room's messages, alongside the gateway API and the operator dashboard" width="800"></div>Switch Core is the infrastructure that joins your agents and your collaborationapps together.-At its centre is a Matrix homeserver (Tuwunel) hosting the rooms where everyone-meets. Every participant is a Matrix client: people arriving through a bridged+At its centre are rooms, held in PostgreSQL, where everyone meets. Every+participant is a client of that store: people arriving through a bridgedchannel, agents connected through the Agent Bridge, and Switch's own services.**Agent Bridge.** Agents speak the Switch Agent Protocol: HTTP for what theydiff --git a/RELEASING.md b/RELEASING.mdindex 4fd01c895..120cf1372 100644--- a/RELEASING.md+++ b/RELEASING.md@@ -153,7 +153,7 @@ to that interface as you would any public API change.`.env.example`). Repo users who want the build-from-source flow layer`standalone-docker-compose.build.yml` on top — that is what `juststandalone-up` runs.-- **Profiles.** Core services (`postgres`, `tuwunel`, `switch`) always start.+- **Profiles.** Core services (`postgres`, `switch`) always start.Optional services are opt-in behind profiles: `collab` (`init-db`,`mattermost`, `setup`) and `gateway` (`gateway`). Enable with`--profile collab --profile gateway` or `COMPOSE_PROFILES`.diff --git a/connectors/claude-code-plugin/skills/switch/SKILL.md b/connectors/claude-code-plugin/skills/switch/SKILL.mdindex 9cb2011d8..cb6d45bd7 100644--- a/connectors/claude-code-plugin/skills/switch/SKILL.md+++ b/connectors/claude-code-plugin/skills/switch/SKILL.md@@ -376,7 +376,7 @@ carrying a display name addresses no one.`instructions`, `group_name`, `aliases`, `reference_ids`, `package_ids`,`linked_rooms`, `join_event_listeners` (the subset of `agent_names` thatshould receive `room_join` events — off by default). Returns- `{id, name, matrix_room_id, failed_attachments}`.+ `{id, name, transport_room_id, failed_attachments}`.- **`update_room`** — change an existing room, including its `aliases` map.- **`invite_agent_to_room`** — add an existing agent to an existing room byname. Humans and agents can do the same from inside a room with thediff --git a/connectors/codex-plugin/skills/switch/SKILL.md b/connectors/codex-plugin/skills/switch/SKILL.mdindex 691d512dc..da5f6fa32 100644--- a/connectors/codex-plugin/skills/switch/SKILL.md+++ b/connectors/codex-plugin/skills/switch/SKILL.md@@ -374,7 +374,7 @@ carrying a display name addresses no one.`instructions`, `group_name`, `aliases`, `reference_ids`, `package_ids`,`linked_rooms`, `join_event_listeners` (the subset of `agent_names` thatshould receive `room_join` events — off by default). Returns- `{id, name, matrix_room_id, failed_attachments}`.+ `{id, name, transport_room_id, failed_attachments}`.- **`update_room`** — change an existing room, including its `aliases` map.- **`invite_agent_to_room`** — add an existing agent to an existing room byname. Humans and agents can do the same from inside a room with thediff --git a/connectors/opencode-plugin/skills/switch/SKILL.md b/connectors/opencode-plugin/skills/switch/SKILL.mdindex f251af523..7d1b96ea3 100644--- a/connectors/opencode-plugin/skills/switch/SKILL.md+++ b/connectors/opencode-plugin/skills/switch/SKILL.md@@ -382,7 +382,7 @@ carrying a display name addresses no one.`instructions`, `group_name`, `aliases`, `reference_ids`, `package_ids`,`linked_rooms`, `join_event_listeners` (the subset of `agent_names` thatshould receive `room_join` events — off by default). Returns- `{id, name, matrix_room_id, failed_attachments}`.+ `{id, name, transport_room_id, failed_attachments}`.- **`update_room`** — change an existing room, including its `aliases` map.- **`invite_agent_to_room`** — add an existing agent to an existing room byname. Humans and agents can do the same from inside a room with thediff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/compose.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/compose.tsindex ba77a6ab3..3d4383c63 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/compose.ts+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/compose.ts@@ -82,6 +82,49 @@ export function composeUp();}+/**+ * Run one command in a throwaway container of `image`, on the stack's network.+ *+ * Deliberately `docker run` and not `compose run`: the caller needs a specific+ * image, and the compose file on disk is whatever version the stack was last+ * started with. A `compose run` would take the service definition — and the+ * image — from that file, which for the case this exists to serve does not+ * contain the service at all.+ *+ * The stack's `.env` supplies the credentials. What it cannot supply is where+ * the other containers are: those come from each service's own definition in+ * the compose file, so the few that matter are passed explicitly here. They are+ * compose service names, stable across the versions this runs against, and+ * they override anything the `.env` says — a `.env` written for the host names+ * `localhost`, which is not where Postgres is from inside the network.+ *+ * Rejects on a non-zero exit.+ */+export function dockerRunOneOff(+ host: ServerHost,+ spec: { image: string; command: string[]; env: Record<string, string> },+ onLog: (line: string) => void+): Promise<void> {+ log.info(`local-switch-server: docker run ${spec.image} (${host.label})`);+ const env = Object.entries(spec.env).flatMap(([key, value]) => ['-e', `${key}=${value}`]);+ return host.streamCommand(+ host.dockerBin,+ [+ 'run',+ '--rm',+ '--network',+ `${host.composeProjectName}_default`,+ '--env-file',+ ENV_FILE_NAME,+ ...env,+ spec.image,+ ...spec.command,+ ],+ onLog,+ { timeoutMs: COMPOSE_TIMEOUT_MS }+ );+}+/** Stop and remove the stack's containers. `removeVolumes` also destroys the* data volumes (the reset path) — irreversible. */export async function composeDown(host: ServerHost, removeVolumes: boolean): Promise<void> {diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.tsindex a92bb09db..594273279 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.ts+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.ts@@ -7,8 +7,6 @@ import type { LocalServerSecrets } from './secret-values';const secrets: LocalServerSecrets = {dbPassword: 'db-pw',- matrixRegistrationSharedSecret: 'matrix-shared',- matrixAdminPassword: 'matrix-admin',agentRegistrationToken: 'agent-token',jwtSecretKey: 'jwt-key',gatewayAdminPassword: 'gw-admin',@@ -60,8 +58,6 @@ describe('buildEnvFile', () => {it('injects every secret into its env var', () => {expect(vars.DB_PASSWORD).toBe('db-pw');- expect(vars.MATRIX_REGISTRATION_SHARED_SECRET).toBe('matrix-shared');- expect(vars.MATRIX_ADMIN_PASSWORD).toBe('matrix-admin');expect(vars.AGENT_REGISTRATION_TOKEN).toBe('agent-token');expect(vars.JWT_SECRET_KEY).toBe('jwt-key');expect(vars.GATEWAY_ADMIN_PASSWORD).toBe('gw-admin');diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.tsindex 9207a999b..260b509bc 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.ts+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.ts@@ -45,11 +45,7 @@ export function buildEnvFile(params: LocalServerEnvParams): string {`DB_PASSWORD=${secrets.dbPassword}`,'DB_NAME=switch','',- 'MATRIX_SERVER=http://localhost:8008','MATRIX_SERVER_NAME=localhost',- 'MATRIX_ADMIN_USER=admin',- `MATRIX_ADMIN_PASSWORD=${secrets.matrixAdminPassword}`,- `MATRIX_REGISTRATION_SHARED_SECRET=${secrets.matrixRegistrationSharedSecret}`,'',`AGENT_REGISTRATION_TOKEN=${secrets.agentRegistrationToken}`,`JWT_SECRET_KEY=${secrets.jwtSecretKey}`,diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.tsindex 3a7d39f15..3cd262442 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.ts+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.ts@@ -14,6 +14,7 @@ import {type DockerAvailability,type LocalServerStatus,type StartLocalServerResult,+ matrixMigrationFailedMessage,switchVersionDowngradeMessage,} from '@shared/core/managed-switch-server/managed-switch-server';import {@@ -171,6 +172,13 @@ class LocalServerService {deployedVersion: result.deployed,drift: { deployed: result.deployed, expected: result.expected, direction: 'downgrade' },});+ } else if (result.kind === 'matrix-migration-failed') {+ this.setStatus({+ phase: 'error',+ message: null,+ error: matrixMigrationFailedMessage(result.deployed, result.expected),+ deployedVersion: result.deployed,+ });} else if (result.kind === 'error') {this.setStatus({ phase: 'error', error: result.message });} else {diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/matrix-migration.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/matrix-migration.test.tsnew file mode 100644index 000000000..06a2d0a25--- /dev/null+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/matrix-migration.test.ts@@ -0,0 +1,65 @@+/**+ * The boundary is a one-way door, so the test that matters is which upgrades+ * are recognised as crossing it.+ *+ * A crossing that is missed loses the install's history permanently. A+ * crossing seen where there is none costs one container that finds nothing to+ * do. The asymmetry is why equality with the boundary counts as "not yet+ * past": the last Matrix-capable release is the version the copy runs on.+ */+import { describe, expect, it } from 'vitest';+import { compareVersions } from '@main/lib/semver';+import { LAST_MATRIX_VERSION } from '@shared/app-identity';+import { crossesMatrixBoundary } from './matrix-migration';++/**+ * Releases whose image cannot run `python -m switch_core.cli.backfill`, because+ * they predate the command moving into the package. Before that it lived in+ * `scripts/`, which the image does not contain, so no earlier release can run+ * it either — which is why the check below is a floor rather than a list.+ */+const FIRST_BACKFILL_CAPABLE_VERSION = '0.24.0';++const AFTER = '0.99.0';+const BEFORE = '0.22.1';++describe('LAST_MATRIX_VERSION', () => {+ it('names a release whose image can actually run the backfill', () => {+ // Not pedantry: the crossing is mandatory and a failed copy blocks the+ // upgrade, so a boundary pointing at an image without the command hard-+ // blocks every install with a ModuleNotFoundError nobody sees. It was set+ // to 0.23.0 — published, in production, and predating the command — and+ // nothing else here could have caught that, because it is a fact about a+ // built artifact rather than about this code.+ expect(compareVersions(LAST_MATRIX_VERSION, FIRST_BACKFILL_CAPABLE_VERSION)).not.toBe(-1);+ });+});++describe('crossesMatrixBoundary', () => {+ it('is true going from before the boundary to after it', () => {+ expect(crossesMatrixBoundary(BEFORE, AFTER)).toBe(true);+ });++ it('is true going from the boundary itself to after it', () => {+ // The boundary release still has the homeserver and the backfill, so a+ // stack sitting on it has not migrated — it is merely able to.+ expect(crossesMatrixBoundary(LAST_MATRIX_VERSION, AFTER)).toBe(true);+ });++ it('is false when both sides are already past it', () => {+ expect(crossesMatrixBoundary(AFTER, '0.25.0')).toBe(false);+ });++ it('is false when the upgrade stops at the boundary', () => {+ // Still on a version that can read the homeserver, so nothing is at risk+ // and there is no reason to spend the copy yet.+ expect(crossesMatrixBoundary(BEFORE, LAST_MATRIX_VERSION)).toBe(false);+ });++ it('is false for a version it cannot compare', () => {+ // Not a judgement that the upgrade is safe — it is that this check cannot+ // say, and guessing either way is worse than the release sequence.+ expect(crossesMatrixBoundary('dev-checkout', AFTER)).toBe(false);+ expect(crossesMatrixBoundary(BEFORE, 'dev-checkout')).toBe(false);+ });+});diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/matrix-migration.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/matrix-migration.tsnew file mode 100644index 000000000..905012527--- /dev/null+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/matrix-migration.ts@@ -0,0 +1,124 @@+/**+ * The one-way door out of Matrix, and the copy that has to happen before it.+ *+ * Room history from before Switch moved to the Postgres message store lives+ * only on the homeserver. The release after {@link LAST_MATRIX_VERSION}+ * deletes the Matrix transport, the backfill command and Tuwunel, so once a+ * stack is on it there is no longer anything that can read that history — not+ * later, not with a flag, not by rolling back, because the rollback would hand+ * an old core a migrated schema.+ *+ * So the upgrade that crosses the line copies first, on the version that still+ * can, and does not cross if the copy fails. There is no opt-out: an install+ * that skips it loses everything said in it before the move, silently, during+ * an update the user clicked yes to.+ *+ * The copy is safe to repeat. Rooms already walked to their start are skipped,+ * so a stack that crosses the line having already been backfilled by hand pays+ * one no-op container.+ */+import { log } from '@main/lib/logger';+import { LAST_MATRIX_VERSION, RELEASE_REPO_OWNER } from '@shared/app-identity';+import { dockerRunOneOff } from './compose';+import { GHCR_REGISTRY } from './constants';+import { classifyVersionDrift } from './deployed-version';+import type { ServerHost } from './host/types';++/**+ * Where the other containers are, from inside the stack's network.+ *+ * The stack's `.env` carries the credentials but not these: they live in each+ * service's own definition in the compose file, and the whole point of running+ * this by image rather than by compose service is not to depend on the version+ * of that file the user happens to have. Compose service names, unchanged+ * across every version this runs against.+ */+const STACK_ADDRESSES = {+ DB_HOST: 'postgres',+ DB_PORT: '5432',+ MATRIX_SERVER: 'http://tuwunel:8008',+};++/**+ * Bring the schema forward, then copy.+ *+ * The migration is not incidental. The stack being migrated is on an older+ * release, so its database is at an older revision — and the backfill selects+ * on a column a later revision adds. Reading it against the old schema fails+ * on a missing column before it looks at a single room.+ *+ * `&&`, so a migration that fails stops the copy and therefore the upgrade.+ * Deliberately not the compose service's entrypoint, which ends in `|| echo`:+ * that exists to let a stack start when the copy fails, and here the failure+ * has to be fatal.+ */+const BACKFILL_COMMAND = [+ 'sh',+ '-c',+ 'alembic upgrade head && python -m switch_core.cli.backfill --allow-empty',+];++/**+ * Whether starting `pinned` against a stack running `deployed` moves it past+ * the last version that can read the homeserver.+ *+ * False when the deployed version is unknown or not comparable: this gates a+ * copy, and a copy that runs when it did not need to costs one container,+ * while one that is skipped on a guess costs the history. Unknown is therefore+ * *not* the safe side here — but it is also not something this can act on, so+ * the caller logs it and the boundary is enforced by the release sequence+ * instead.+ */+export function crossesMatrixBoundary(deployed: string, pinned: string): boolean {+ const deployedIsPast = classifyVersionDrift(LAST_MATRIX_VERSION, deployed);+ const pinnedIsPast = classifyVersionDrift(LAST_MATRIX_VERSION, pinned);+ if (deployedIsPast?.direction === 'unknown' || pinnedIsPast?.direction === 'unknown') {+ return false;+ }+ // `classifyVersionDrift(a, b)` reads as "going from a to b": an upgrade means+ // b is newer. So the pinned version is past the boundary when moving from the+ // boundary to it is an upgrade, and the deployed one is at or below it when+ // that same move is not.+ const pinnedIsAfter = pinnedIsPast?.direction === 'upgrade';+ const deployedIsAfter = deployedIsPast?.direction === 'upgrade';+ return pinnedIsAfter && !deployedIsAfter;+}++export type BackfillOutcome = { ok: true } | { ok: false; detail: string };++/**+ * Copy the homeserver's history into Postgres, against the stack as it is+ * currently running.+ *+ * The image is pinned to {@link LAST_MATRIX_VERSION} rather than taken from+ * the stack, and that is the point: the install that most needs this is the one+ * that skipped that release, so its own images and its own compose file have no+ * backfill in them at all. Pinning the image means the copy does not depend on+ * which version the user happens to be coming from.+ *+ * Nothing here runs the compose service's entrypoint, which is written to let a+ * stack start anyway when the copy fails. Here that failure is the whole point:+ * it has to stop the upgrade.+ */+export async function runBackfill(+ host: ServerHost,+ onLog: (line: string) => void+): Promise<BackfillOutcome> {+ log.info(`managed-switch-server: backfilling room history before the upgrade (${host.label})`);+ try {+ await dockerRunOneOff(+ host,+ {+ image: `${GHCR_REGISTRY}/${RELEASE_REPO_OWNER}/switch-core:${LAST_MATRIX_VERSION}`,+ command: BACKFILL_COMMAND,+ env: STACK_ADDRESSES,+ },+ onLog+ );+ return { ok: true };+ } catch (error) {+ const detail = error instanceof Error ? error.message : String(error);+ log.error(`managed-switch-server: backfill failed; not upgrading (${host.label})`, { detail });+ return { ok: false, detail };+ }+}diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-matrix-migration.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-matrix-migration.test.tsnew file mode 100644index 000000000..0560294cc--- /dev/null+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-matrix-migration.test.ts@@ -0,0 +1,179 @@+/**+ * The upgrade that removes the message server must copy its history first.+ *+ * Two properties, and the second is the one that protects the data: the copy+ * runs against the stack as it is currently deployed — before anything is+ * rewritten, because the rewrite is what takes the homeserver away — and a+ * copy that fails stops the upgrade rather than being logged and stepped over.+ */+import { beforeEach, describe, expect, it, vi } from 'vitest';+import type * as AppIdentity from '@shared/app-identity';+import type * as DeployedVersion from './deployed-version';+import type { ServerHost } from './host/types';++const readDeployedVersionMock = vi.hoisted(() => vi.fn());+const composeUpMock = vi.hoisted(() => vi.fn(() => Promise.resolve()));+const dockerRunOneOffMock = vi.hoisted(() => vi.fn(() => Promise.resolve()));++vi.mock('@shared/app-identity', async (importOriginal) => ({+ ...(await importOriginal<typeof AppIdentity>()),+ COMPATIBLE_SWITCH_VERSION: '0.24.0',+ LAST_MATRIX_VERSION: '0.23.0',+}));+vi.mock('@main/lib/logger', () => ({+ log: { error: vi.fn(), warn: vi.fn(), info: vi.fn() },+}));+vi.mock('./deployed-version', async (importOriginal) => ({+ ...(await importOriginal<typeof DeployedVersion>()),+ readDeployedVersion: readDeployedVersionMock,+}));+vi.mock('./compose', () => ({+ composeUp: composeUpMock,+ dockerRunOneOff: dockerRunOneOffMock,+ composeDown: vi.fn(() => Promise.resolve()),+ runningImages: vi.fn(),+ isStackRunning: vi.fn(),+}));+vi.mock('./health', () => ({ waitForHealth: () => Promise.resolve(true) }));+vi.mock('./bundled-compose', () => ({ bundledComposeYaml: () => 'services: {}' }));+vi.mock('./secrets', () => ({+ loadOrCreateSecrets: () => Promise.resolve({ gatewayAdminPassword: 'pw' }),+ clearSecrets: vi.fn(),+}));+vi.mock('./ports', () => ({+ resolvePorts: () =>+ Promise.resolve({ gateway: 3300, api: 8000, mattermost: 8065, postgres: 5432 }),+ clearPorts: vi.fn(),+}));+vi.mock('./env-file', () => ({ buildEnvFile: () => 'SWITCH_VERSION=0.24.0\n' }));+vi.mock('@main/core/switch-servers/servers-store', () => ({+ ensureManagedServer: () => Promise.resolve({ id: 'srv-1' }),+ setActiveServerId: vi.fn(),+}));+vi.mock('@main/core/switch-servers/auth', () => ({+ passwordLogin: () => Promise.resolve({ success: true }),+}));+vi.mock('@main/core/agents/resolve-servers', () => ({ resolveAgentServers: vi.fn() }));++const { startStack } = await import('./pipeline');++function options(checkoutRoot: string | null = null) {+ const writeFile = vi.fn<(relPath: string, content: string, mode?: number) => Promise<void>>(() =>+ Promise.resolve()+ );+ const host = {+ label: 'this computer',+ writeFile,+ detectDocker: () => Promise.resolve({ available: true, version: '27.0.0' }),+ establishNetworking: vi.fn(() => Promise.resolve()),+ };+ return {+ writeFile,+ opts: {+ host: host as unknown as ServerHost,+ ref: { kind: 'local' as const },+ serverName: 'Local',+ onMessage: vi.fn(),+ onLog: vi.fn(),+ signal: new AbortController().signal,+ checkoutRoot,+ },+ };+}++beforeEach(() => {+ vi.clearAllMocks();+});++describe('startStack across the Matrix boundary', () => {+ it('copies the history before rewriting anything', async () => {+ readDeployedVersionMock.mockResolvedValue({+ kind: 'deployed',+ version: '0.23.0',+ source: 'container',+ });+ const { opts, writeFile } = options();+ const order: string[] = [];+ composeUpMock.mockImplementation(() => {+ order.push('compose-up');+ return Promise.resolve();+ });+ dockerRunOneOffMock.mockImplementation(() => {+ order.push('backfill');+ return Promise.resolve();+ });+ writeFile.mockImplementation(() => {+ order.push('write');+ return Promise.resolve();+ });++ expect(await startStack(opts)).toEqual({ kind: 'started', serverId: 'srv-1' });+ // The old stack comes up and is drained before the new compose file lands:+ // that file is what removes the homeserver being read from.+ expect(order.slice(0, 3)).toEqual(['compose-up', 'backfill', 'write']);+ // Pinned to the boundary image, not to whatever the stack is running: the+ // install that needs this most is the one whose own images have no backfill.+ expect(dockerRunOneOffMock).toHaveBeenCalledWith(+ expect.anything(),+ expect.objectContaining({+ image: 'ghcr.io/sandbox-quantum/switch-core:0.23.0',+ // Migrates before it copies. The stack is on an older release, so its+ // database is at an older revision and the backfill reads a column a+ // later one adds — without this it fails on a missing column before+ // reaching a single room.+ command: [+ 'sh',+ '-c',+ 'alembic upgrade head && python -m switch_core.cli.backfill --allow-empty',+ ],+ }),+ expect.anything()+ );+ });++ it('refuses the upgrade when the copy fails, leaving the stack as it was', async () => {+ readDeployedVersionMock.mockResolvedValue({+ kind: 'deployed',+ version: '0.22.1',+ source: 'container',+ });+ dockerRunOneOffMock.mockRejectedValue(new Error('homeserver unreachable'));+ const { opts, writeFile } = options();++ const result = await startStack(opts);++ expect(result).toMatchObject({+ kind: 'matrix-migration-failed',+ deployed: '0.22.1',+ expected: '0.24.0',+ });+ // Nothing rewritten, so the stack is still the one that can be retried.+ expect(writeFile).not.toHaveBeenCalled();+ });++ it('does not copy when the stack is already past the boundary', async () => {+ readDeployedVersionMock.mockResolvedValue({+ kind: 'deployed',+ version: '0.24.0',+ source: 'container',+ });+ const { opts } = options();++ expect(await startStack(opts)).toEqual({ kind: 'started', serverId: 'srv-1' });+ expect(dockerRunOneOffMock).not.toHaveBeenCalled();+ });++ it('does not copy for a checkout build', async () => {+ // Built from a working tree, so it carries no comparable version — and its+ // data is a developer's, not somebody's install.+ readDeployedVersionMock.mockResolvedValue({+ kind: 'deployed',+ version: '0.22.1',+ source: 'container',+ });+ const { opts } = options('/src/switch');++ await startStack(opts);+ expect(dockerRunOneOffMock).not.toHaveBeenCalled();+ });+});diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.tsindex e7492dbb9..c558e35b5 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts@@ -23,6 +23,7 @@ import { buildEnvFile } from './env-file';import { apiUrlFor, gatewayUrlFor } from './free-port';import { waitForHealth } from './health';import type { ServerHost } from './host/types';+import { crossesMatrixBoundary, runBackfill } from './matrix-migration';import { clearPorts, resolvePorts } from './ports';import { clearSecrets, loadOrCreateSecrets } from './secrets';@@ -132,6 +133,53 @@ async function refuseDowngrade(* tree on every start instead of pulled, and tagged {@link CHECKOUT_IMAGE_TAG}* so nothing downstream mistakes them for a release.*/+/**+ * Move a stack past the last version that can read a Matrix homeserver, copying+ * its history first.+ *+ * Not optional and not skippable. The next version deletes the transport, the+ * backfill and Tuwunel, so a stack that crosses without copying loses every+ * message sent before Switch moved to the Postgres store — silently, during an+ * update nobody asked to be a migration. A failed copy therefore fails the+ * start: the stack stays on the version it is on, which is where it can still+ * be fixed, and trying again is free because the copy skips what it has+ * already done.+ *+ * Returns null when there is nothing to do: no boundary crossed, no readable+ * deployed version, or a dev checkout build (which carries no comparable+ * version, and whose data is not somebody's install).+ */+async function migrateOffMatrix(+ host: ServerHost,+ checkoutRoot: string | null,+ onMessage: (message: string) => void,+ onLog: (line: string) => void+): Promise<StartLocalServerResult | null> {+ if (checkoutRoot !== null) return null;+ const deployed = await readDeployedVersion(host);+ if (deployed.kind !== 'deployed' || deployed.version === CHECKOUT_IMAGE_TAG) return null;+ if (!crossesMatrixBoundary(deployed.version, COMPATIBLE_SWITCH_VERSION)) return null;++ log.info(+ `managed-switch-server: ${host.label} is crossing the Matrix boundary ` ++ `(${deployed.version} → ${COMPATIBLE_SWITCH_VERSION}); backfilling first`+ );+ // Bring the stack up as it stands. The backfill reads the homeserver and the+ // database, and this is the last moment both are still here.+ onMessage('Starting the current version to copy your room history…');+ await composeUp(host, onLog, false);++ onMessage('Copying room history out of the message server…');+ const backfill = await runBackfill(host, onLog);+ if (backfill.ok) return null;+ return {+ kind: 'matrix-migration-failed',+ deployed: deployed.version,+ expected: COMPATIBLE_SWITCH_VERSION,+ detail: backfill.detail,+ };+}+export async function startStack(opts: StartStackOptions): Promise<StartLocalServerResult> {const { host, ref, serverName, onMessage, onLog, signal, checkoutRoot } = opts;@@ -144,6 +192,13 @@ export async function startStack(opts: StartStackOptions): Promise<StartLocalSerconst downgrade = await refuseDowngrade(host, checkoutRoot);if (downgrade) return downgrade;+ // Copy the homeserver's history across before the upgrade removes the only+ // thing that can read it. Runs against the stack as currently deployed, so+ // it must happen before the compose file and `.env` are re-materialised for+ // the new version — those are what would take Tuwunel away.+ const migration = await migrateOffMatrix(host, checkoutRoot, onMessage, onLog);+ if (migration) return migration;+onMessage('Preparing configuration…');await host.writeFile(COMPOSE_FILE_NAME, bundledComposeYaml());if (checkoutRoot !== null) {diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.tsindex 967176ebf..9cf414fe8 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.ts+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.ts@@ -16,6 +16,7 @@ import { COMPATIBLE_SWITCH_VERSION } from '@shared/app-identity';import {type DockerAvailability,type StartLocalServerResult,+ matrixMigrationFailedMessage,switchVersionDowngradeMessage,} from '@shared/core/managed-switch-server/managed-switch-server';import {@@ -190,6 +191,13 @@ class RemoteServerService {drift: { deployed: result.deployed, expected: result.expected, direction: 'downgrade' },});host.dispose();+ } else if (result.kind === 'matrix-migration-failed') {+ this.setStatus(sshHost, {+ phase: 'error',+ message: null,+ error: matrixMigrationFailedMessage(result.deployed, result.expected),+ deployedVersion: result.deployed,+ });} else if (result.kind === 'error') {this.setStatus(sshHost, { phase: 'error', error: result.message });host.dispose();diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/resources/standalone-docker-compose.pinned.yml b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/resources/standalone-docker-compose.pinned.ymlindex e67ba9c6d..79d2acd00 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/resources/standalone-docker-compose.pinned.yml+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/resources/standalone-docker-compose.pinned.yml@@ -17,7 +17,7 @@# docker compose -f standalone-docker-compose.yml -f standalone-docker-compose.build.yml up## Profiles keep optional services opt-in:-# (core, no profile) postgres, tuwunel, backfill, switch — always started+# (core, no profile) postgres, switch — always started# collab init-db, mattermost, setup — Mattermost bridge + seeder# gateway gateway — operator dashboard# Select them with `--profile collab --profile gateway` or COMPOSE_PROFILES.@@ -62,6 +62,10 @@ services:POSTGRES_PASSWORD: ${DB_PASSWORD}POSTGRES_DB: ${DB_NAME}volumes:+ # Kept deliberately after Tuwunel was removed. It still holds every media+ # blob and the raw history, and it is the only copy — the backfill moved the+ # messages into Postgres but not the files. Dropping it belongs to a later+ # release than the one that stops reading it.- pgdata:/var/lib/postgresql/datahealthcheck:test: ["CMD-SHELL", "pg_isready -U ${DB_USER}"]@@ -96,20 +100,6 @@ services:postgres:condition: service_healthy- tuwunel:- image: jevolk/tuwunel:v1.7.1- volumes:- - tuwuneldata:/var/lib/tuwunel- environment:- TUWUNEL_SERVER_NAME: localhost- TUWUNEL_DATABASE_PATH: /var/lib/tuwunel- TUWUNEL_ADDRESS: 0.0.0.0- TUWUNEL_PORT: "8008"- TUWUNEL_ALLOW_FEDERATION: "false"- TUWUNEL_ALLOW_REGISTRATION: "false"- TUWUNEL_REGISTRATION_SHARED_SECRET: ${MATRIX_REGISTRATION_SHARED_SECRET}- TUWUNEL_MAX_REQUEST_SIZE: "20000000"-mattermost:# Pinned rather than `latest`: Switch Console embeds this Mattermost's web UI# and depends on its DOM structure, so an unattended upgrade could break@@ -122,6 +112,10 @@ services:ports:- "${SWITCH_BIND_ADDR:-127.0.0.1}:${MATTERMOST_HOST_PORT:-8065}:8065"volumes:+ # Kept deliberately after Tuwunel was removed. It still holds every media+ # blob and the raw history, and it is the only copy — the backfill moved the+ # messages into Postgres but not the files. Dropping it belongs to a later+ # release than the one that stops reading it.- mmdata:/mattermost/dataenvironment:MM_SQLSETTINGS_DRIVERNAME: postgres@@ -149,54 +143,6 @@ services:init-db:condition: service_completed_successfully- # One-shot: reconstructs each room's history from the homeserver into the- # message log, then exits. switch waits for it to finish, so the copy is- # never racing the server that will serve it.- #- # It always exits 0, so a backfill that fails delays the stack rather than- # keeping it down. The trade is deliberate and it is a real one: a failure- # here is only visible in this service's log, and the stack above it will- # look healthy while missing history. Backfill is idempotent, so re-running- # it after fixing the cause costs nothing.- #- # It runs alembic first for ordering, not as a second migration owner: the- # rows it writes need the schema, and switch (which normally migrates on- # boot) is waiting on this. The upgrade is the same idempotent one.- #- # Rooms already walked to their start are skipped, so this is cheap on every- # boot after the first. Nothing to walk at all is success too — a fresh- # stack has no rooms and no admin client.- backfill:- image: ${SWITCH_REGISTRY:-ghcr.io}/${SWITCH_IMAGE_NAMESPACE:-sandbox-quantum}/switch-core:${SWITCH_VERSION:?required, set it in .env, there is no default}- restart: "no"- environment:- DB_HOST: postgres- DB_PORT: "5432"- DB_USER: ${DB_USER}- DB_PASSWORD: ${DB_PASSWORD}- DB_NAME: ${DB_NAME}- MATRIX_SERVER: http://tuwunel:8008- MATRIX_SERVER_NAME: localhost- MATRIX_ADMIN_USER: ${MATRIX_ADMIN_USER}- MATRIX_ADMIN_PASSWORD: ${MATRIX_ADMIN_PASSWORD}- MATRIX_REGISTRATION_SHARED_SECRET: ${MATRIX_REGISTRATION_SHARED_SECRET}- AGENT_REGISTRATION_TOKEN: ${AGENT_REGISTRATION_TOKEN}- JWT_SECRET_KEY: ${JWT_SECRET_KEY}- GATEWAY_ADMIN_EMAIL: ${GATEWAY_ADMIN_EMAIL}- GATEWAY_ADMIN_PASSWORD: ${GATEWAY_ADMIN_PASSWORD}- entrypoint: >- sh -c "- alembic upgrade head &&- python -m switch_core.cli.backfill --allow-empty ||- echo 'backfill did not complete; starting anyway. Room history from- before this release may be missing until it is re-run.' >&2- "- depends_on:- postgres:- condition: service_healthy- tuwunel:- condition: service_started-switch:image: ${SWITCH_REGISTRY:-ghcr.io}/${SWITCH_IMAGE_NAMESPACE:-sandbox-quantum}/switch-core:${SWITCH_VERSION:?required, set it in .env, there is no default}ports:@@ -207,11 +153,7 @@ services:DB_USER: ${DB_USER}DB_PASSWORD: ${DB_PASSWORD}DB_NAME: ${DB_NAME}- MATRIX_SERVER: http://tuwunel:8008MATRIX_SERVER_NAME: localhost- MATRIX_ADMIN_USER: ${MATRIX_ADMIN_USER}- MATRIX_ADMIN_PASSWORD: ${MATRIX_ADMIN_PASSWORD}- MATRIX_REGISTRATION_SHARED_SECRET: ${MATRIX_REGISTRATION_SHARED_SECRET}AGENT_REGISTRATION_TOKEN: ${AGENT_REGISTRATION_TOKEN}JWT_SECRET_KEY: ${JWT_SECRET_KEY}GATEWAY_ADMIN_EMAIL: ${GATEWAY_ADMIN_EMAIL}@@ -221,10 +163,6 @@ services:depends_on:postgres:condition: service_healthy- tuwunel:- condition: service_started- backfill:- condition: service_completed_successfullygateway:image: ${SWITCH_REGISTRY:-ghcr.io}/${SWITCH_IMAGE_NAMESPACE:-sandbox-quantum}/gateway:${SWITCH_VERSION:?required, set it in .env, there is no default}@@ -254,6 +192,10 @@ services:- switchvolumes:+ # Kept deliberately after Tuwunel was removed. It still holds every media+ # blob and the raw history, and it is the only copy — the backfill moved the+ # messages into Postgres but not the files. Dropping it belongs to a later+ # release than the one that stops reading it.pgdata:mmdata:tuwuneldata:diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secret-values.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secret-values.tsindex 9bfa2abb2..1e6d12877 100644--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secret-values.ts+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secret-values.ts@@ -5,8 +5,6 @@ import { randomBytes } from 'node:crypto';* reads is derived from them at each start. */export type LocalServerSecrets = {dbPassword: string;- matrixRegistrationSharedSecret: string;- matrixAdminPassword: string;agentRegistrationToken: string;jwtSecretKey: string;gatewayAdminPassword: string;@@ -23,8 +21,6 @@ function token(bytes = 24): string {export function generateSecrets(): LocalServerSecrets {return {dbPassword: token(),- matrixRegistrationSharedSecret: token(),- matrixAdminPassword: token(),agentRegistrationToken: token(),jwtSecretKey: token(32),gatewayAdminPassword: token(),diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/events.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/events.tsindex b87e8f86e..d8c676766 100644--- a/console/apps/switch-console-desktop/src/main/core/telemetry/events.ts+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/events.ts@@ -194,6 +194,7 @@ export type TelemetryManagedServerFailure =| 'docker_not_installed'| 'docker_daemon_down'| 'version_downgrade'+ | 'matrix_migration_failed'| 'error';/**diff --git a/console/apps/switch-console-desktop/src/main/core/telemetry/managed-server.ts b/console/apps/switch-console-desktop/src/main/core/telemetry/managed-server.tsindex 1b9a95aa5..249910993 100644--- a/console/apps/switch-console-desktop/src/main/core/telemetry/managed-server.ts+++ b/console/apps/switch-console-desktop/src/main/core/telemetry/managed-server.ts@@ -18,6 +18,8 @@ function startFailureReason(result: StartLocalServerResult): TelemetryManagedSerreturn result.reason === 'not-installed' ? 'docker_not_installed' : 'docker_daemon_down';case 'version-downgrade':return 'version_downgrade';+ case 'matrix-migration-failed':+ return 'matrix_migration_failed';case 'error':return 'error';}diff --git a/console/apps/switch-console-desktop/src/shared/app-identity.ts b/console/apps/switch-console-desktop/src/shared/app-identity.tsindex c9b991d0a..f9b43cf52 100644--- a/console/apps/switch-console-desktop/src/shared/app-identity.ts+++ b/console/apps/switch-console-desktop/src/shared/app-identity.ts@@ -61,3 +61,21 @@ export const RELEASE_REPO_NAME = 'switch';// derived pin would immediately point local-server mode at images that are not// on the registry yet.export const COMPATIBLE_SWITCH_VERSION = '0.24.0';++// The last switch-core release that can still read a Matrix homeserver.+//+// Room history written before Switch moved to the Postgres message store lives+// only on the homeserver, and only an image up to this version can copy it+// across — the release after it removes the transport, the backfill command+// and Tuwunel itself. Crossing this line without having run the backfill+// strands that history where nothing will ever read it again, so the upgrade+// runs the backfill first and refuses to cross if it fails.+//+// A stack already on a version above this has nothing to migrate.+//+// It must name a release that EXISTS and that CONTAINS the backfill command —+// `switch_core.cli.backfill`, which only entered the image when the command+// moved into the package. 0.23.0 is published and predates that, so it is not+// a valid value here however natural it looks: pointing at it would make every+// crossing fail with ModuleNotFoundError, and the crossing is mandatory.+export const LAST_MATRIX_VERSION = '0.24.1';diff --git a/console/apps/switch-console-desktop/src/shared/core/managed-switch-server/managed-switch-server.ts b/console/apps/switch-console-desktop/src/shared/core/managed-switch-server/managed-switch-server.tsindex 22f9ee9cb..58220f483 100644--- a/console/apps/switch-console-desktop/src/shared/core/managed-switch-server/managed-switch-server.ts+++ b/console/apps/switch-console-desktop/src/shared/core/managed-switch-server/managed-switch-server.ts@@ -72,6 +72,22 @@ export function switchVersionDowngradeMessage(deployed: string, expected: string);}+/**+ * Why an upgrade stopped short of removing the message server, in one message.+ *+ * Says the two things a user needs: nothing has changed, and trying again is+ * the fix. Deliberately does not offer a way past it — the history is only on+ * the message server, and the version being upgraded to is the one that+ * removes it.+ */+export function matrixMigrationFailedMessage(deployed: string, expected: string): string {+ return (+ `Your room history has to be copied out of the message server before updating to ` ++ `switch-core ${expected}, and the copy did not finish. Nothing has changed — the stack is ` ++ `still on ${deployed}. Try starting it again; the copy carries on from where it stopped.`+ );+}+/*** Image tag a checkout build is tagged with. Deliberately not a semver and* deliberately not the pin: it must never be mistaken for a released version,@@ -155,4 +171,8 @@ export type StartLocalServerResult =| { kind: 'started'; serverId: string }| { kind: 'docker-unavailable'; reason: 'not-installed' | 'daemon-down'; detail: string }| { kind: 'version-downgrade'; deployed: string; expected: string }+ // The upgrade would have removed the message server before this stack's+ // history was copied out of it, and the copy failed. The stack is untouched+ // and still on `deployed`; retrying is safe and resumes where it stopped.+ | { kind: 'matrix-migration-failed'; deployed: string; expected: string; detail: string }| { kind: 'error'; message: string };diff --git a/console/packages/plugins/src/agents/impl/opencode/skill-file.ts b/console/packages/plugins/src/agents/impl/opencode/skill-file.tsindex 34ff80077..3891ffec8 100644--- a/console/packages/plugins/src/agents/impl/opencode/skill-file.ts+++ b/console/packages/plugins/src/agents/impl/opencode/skill-file.ts@@ -390,7 +390,7 @@ carrying a display name addresses no one.\`instructions\`, \`group_name\`, \`aliases\`, \`reference_ids\`, \`package_ids\`,\`linked_rooms\`, \`join_event_listeners\` (the subset of \`agent_names\` thatshould receive \`room_join\` events — off by default). Returns- \`{id, name, matrix_room_id, failed_attachments}\`.+ \`{id, name, transport_room_id, failed_attachments}\`.- **\`update_room\`** — change an existing room, including its \`aliases\` map.- **\`invite_agent_to_room\`** — add an existing agent to an existing room byname. Humans and agents can do the same from inside a room with thediff --git a/core/pyproject.toml b/core/pyproject.tomlindex b4130ba81..135cb7938 100644--- a/core/pyproject.toml+++ b/core/pyproject.toml@@ -7,7 +7,7 @@ requires-python = ">=3.12"authors = [{ name = "SandboxAQ" },]-keywords = ["ai", "agents", "orchestration", "governance", "matrix", "mcp"]+keywords = ["ai", "agents", "orchestration", "governance", "mcp"]license = "LicenseRef-Apache-2.0-with-Commons-Clause"classifiers = ["Development Status :: 4 - Beta",@@ -26,7 +26,6 @@ dependencies = ["alembic>=1.15,<2","fastapi>=0.115,<1","uvicorn>=0.34,<1",- "matrix-nio>=0.25,<0.26","httpx>=0.28,<1","Markdown>=3,<4","slack-sdk>=3.41.0",diff --git a/core/switch_core/bridges/agent/api/handlers.py b/core/switch_core/bridges/agent/api/handlers.pyindex d0e7d84a1..b85f29503 100644--- a/core/switch_core/bridges/agent/api/handlers.py+++ b/core/switch_core/bridges/agent/api/handlers.py@@ -1404,6 +1404,7 @@ async def create_room(return CreateModerationRoomResponse(id=result.room.id,name=result.room.name,+ transport_room_id=result.room.matrix_room_id,matrix_room_id=result.room.matrix_room_id,failed_attachments=result.failed_attachments,)diff --git a/core/switch_core/bridges/agent/api/schemas.py b/core/switch_core/bridges/agent/api/schemas.pyindex 9a5784cb6..9c5c7dab9 100644--- a/core/switch_core/bridges/agent/api/schemas.py+++ b/core/switch_core/bridges/agent/api/schemas.py@@ -391,6 +391,11 @@ class CreateModerationRoomRequest(BaseModel):class CreateModerationRoomResponse(BaseModel):id: strname: str+ transport_room_id: str+ # Deprecated alias of `transport_room_id`, sent for the compatibility+ # window. Connectors are installed copies and only update when someone+ # clicks Update, so both names are carried until the shipped ones have+ # moved. Same value in both; nothing dereferences either.matrix_room_id: str# Attachments that failed at attach-time (after the room was created).# Empty on full success.diff --git a/core/switch_core/bridges/agent/operations/definitions.py b/core/switch_core/bridges/agent/operations/definitions.pyindex 5fa5e72b1..7239190a2 100644--- a/core/switch_core/bridges/agent/operations/definitions.py+++ b/core/switch_core/bridges/agent/operations/definitions.py@@ -1036,6 +1036,8 @@ async def create_room(return {"id": result.room.id,"name": result.room.name,+ "transport_room_id": result.room.matrix_room_id,+ # Deprecated alias, carried for the connector compatibility window."matrix_room_id": result.room.matrix_room_id,"failed_attachments": result.failed_attachments,}diff --git a/core/switch_core/bridges/agent/protocol/service.py b/core/switch_core/bridges/agent/protocol/service.pyindex 3bde439c8..d9a15bb0d 100644--- a/core/switch_core/bridges/agent/protocol/service.py+++ b/core/switch_core/bridges/agent/protocol/service.py@@ -205,6 +205,7 @@ def _describe_room(room: Room) -> RoomDescriptor:id=room.id,name=room.name,description=room.description,+ transport_room_id=room.matrix_room_id,matrix_room_id=room.matrix_room_id,archived=room.archived_at is not None,bridge_id=room.bridge_id,@@ -1581,12 +1582,12 @@ class ProtocolService:decide, not the writer's — so the sentence is composed here and can bechanged without rewriting history."""- name = message.sender_name or message.sender_matrix_id+ name = message.sender_name or message.sender_idif message.event_type == MEMBERSHIP_EVENT_TYPE:return {"id": message.transport_event_id,"kind": "room_join",- "sender": message.sender_matrix_id,+ "sender": message.sender_id,"sender_name": name,"body": f"{name} joined the room","timestamp": _epoch_ms(message.sent_at),@@ -1595,7 +1596,7 @@ class ProtocolService:return {"id": message.transport_event_id,"kind": "message",- "sender": message.sender_matrix_id,+ "sender": message.sender_id,"sender_name": name,"body": message.body,"timestamp": _epoch_ms(message.sent_at),@@ -3162,6 +3163,7 @@ class ProtocolService:channel_type=room.channel_type,admin_mode=room.admin_mode,instructions=room.instructions,+ transport_room_id=room.matrix_room_id,matrix_room_id=room.matrix_room_id,created_at=str(room.created_at),bridge_id=room.bridge_id,diff --git a/core/switch_core/bridges/agent/protocol/types.py b/core/switch_core/bridges/agent/protocol/types.pyindex 186752348..6bd40ce77 100644--- a/core/switch_core/bridges/agent/protocol/types.py+++ b/core/switch_core/bridges/agent/protocol/types.py@@ -92,6 +92,11 @@ class RoomDescriptor(BaseModel):id: strname: strdescription: str+ transport_room_id: str+ # Deprecated alias of `transport_room_id`, sent for the compatibility+ # window. Connectors are installed copies and only update when someone+ # clicks Update, so both names are carried until the shipped ones have+ # moved. Same value in both; nothing dereferences either.matrix_room_id: strarchived: bool = False# The room's collaboration bridge, or None for an internal-only room.@@ -148,6 +153,11 @@ class RoomDetailDescriptor(BaseModel):channel_type: str | Noneadmin_mode: boolinstructions: str | None+ transport_room_id: str+ # Deprecated alias of `transport_room_id`, sent for the compatibility+ # window. Connectors are installed copies and only update when someone+ # clicks Update, so both names are carried until the shipped ones have+ # moved. Same value in both; nothing dereferences either.matrix_room_id: strcreated_at: strbridge_id: str | Nonediff --git a/core/switch_core/bridges/collaboration/bridge_core.py b/core/switch_core/bridges/collaboration/bridge_core.pyindex f857ef94d..d06d42172 100644--- a/core/switch_core/bridges/collaboration/bridge_core.py+++ b/core/switch_core/bridges/collaboration/bridge_core.py@@ -550,7 +550,7 @@ class BridgeCore:# Record the correlation so a later reply (either direction) threads.await self._record_message_map(external_channel_id=msg.channel_id,- matrix_event_id=event_id,+ transport_event_id=event_id,external_post_id=msg.message_ref,)return@@ -601,7 +601,7 @@ class BridgeCore:if first_event_id is not None:await self._record_message_map(external_channel_id=msg.channel_id,- matrix_event_id=first_event_id,+ transport_event_id=first_event_id,external_post_id=msg.message_ref,)@@ -685,7 +685,7 @@ class BridgeCore:try:await self._record_message_map(external_channel_id=cmd.channel_id,- matrix_event_id=event_id,+ transport_event_id=event_id,external_post_id=thread_root_post,)finally:@@ -1322,7 +1322,7 @@ class BridgeCore:if message_ref is not None:await self._record_message_map(external_channel_id=channel_id,- matrix_event_id=event.event_id,+ transport_event_id=event.event_id,external_post_id=message_ref,)@@ -1475,7 +1475,7 @@ class BridgeCore:if message_ref is not None:await self._record_message_map(external_channel_id=channel_id,- matrix_event_id=event.event_id,+ transport_event_id=event.event_id,external_post_id=message_ref,)@@ -1559,7 +1559,7 @@ class BridgeCore:if message_ref is not None and pending.first_event_id is not None:await self._record_message_map(external_channel_id=channel_id,- matrix_event_id=pending.first_event_id,+ transport_event_id=pending.first_event_id,external_post_id=message_ref,)@@ -1714,7 +1714,7 @@ class BridgeCore:self, event_id: str, new_content: str | None) -> None:async with self._session_factory() as session:- mapping = await self._bridge_message_map_store.get_by_matrix_event_id(+ mapping = await self._bridge_message_map_store.get_by_transport_event_id(session, self._bridge_id, event_id)if mapping is None:@@ -1726,7 +1726,7 @@ class BridgeCore:await self._adapter.delete_message(channel_id, message_ref)# The post is gone; drop the mapping so it can't resolve later.async with self._session_factory() as session:- await self._bridge_message_map_store.delete_by_matrix_event_id(+ await self._bridge_message_map_store.delete_by_transport_event_id(session, self._bridge_id, event_id)await session.commit()@@ -1809,12 +1809,16 @@ class BridgeCore:self._pending_message_maps[matrix_event_id] = external_post_idasync def _record_message_map(- self, *, external_channel_id: str, matrix_event_id: str, external_post_id: str+ self,+ *,+ external_channel_id: str,+ transport_event_id: str,+ external_post_id: str,) -> None:"""Persist a Matrix-event ↔ external-post correlation (idempotent)."""async with self._session_factory() as session:- existing = await self._bridge_message_map_store.get_by_matrix_event_id(- session, self._bridge_id, matrix_event_id+ existing = await self._bridge_message_map_store.get_by_transport_event_id(+ session, self._bridge_id, transport_event_id)if existing is not None:return@@ -1823,7 +1827,7 @@ class BridgeCore:BridgeMessageMap(bridge_id=self._bridge_id,external_channel_id=external_channel_id,- matrix_event_id=matrix_event_id,+ transport_event_id=transport_event_id,external_post_id=external_post_id,),)@@ -1836,15 +1840,17 @@ class BridgeCore:mapping = await self._bridge_message_map_store.get_by_external_post_id(session, self._bridge_id, external_post_id)- return mapping.matrix_event_id if mapping is not None else None+ return mapping.transport_event_id if mapping is not None else None- async def _external_post_for_matrix_event(self, matrix_event_id: str) -> str | None:- pending = self._pending_message_maps.get(matrix_event_id)+ async def _external_post_for_matrix_event(+ self, transport_event_id: str+ ) -> str | None:+ pending = self._pending_message_maps.get(transport_event_id)if pending is not None:return pendingasync with self._session_factory() as session:- mapping = await self._bridge_message_map_store.get_by_matrix_event_id(- session, self._bridge_id, matrix_event_id+ mapping = await self._bridge_message_map_store.get_by_transport_event_id(+ session, self._bridge_id, transport_event_id)return mapping.external_post_id if mapping is not None else Nonediff --git a/core/switch_core/bridges/collaboration/lifecycle_service.py b/core/switch_core/bridges/collaboration/lifecycle_service.pyindex ab9b8bdc9..300f716e7 100644--- a/core/switch_core/bridges/collaboration/lifecycle_service.py+++ b/core/switch_core/bridges/collaboration/lifecycle_service.py@@ -415,18 +415,10 @@ class CollaborationBridgeLifecycleService:client_id=bridge_client_record.id,matrix_user_id=bridge_client_record.matrix_user_id,display_name=bridge_client_record.display_name,- password=bridge_client_record.password,- server_url=self._config.matrix_server,session_factory=self._session_factory,client_store=self._client_store,config=BridgeClientConfig(bridge_id=bridge_id),transport_factory=self._client_factory.transport_for,- message_recorder=self._client_factory.recorder(),- session_state={- "access_token": bridge_client_record.access_token,- "device_id": bridge_client_record.device_id,- },- next_batch_token=bridge_client_record.next_batch_token,)task = asyncio.create_task(diff --git a/core/switch_core/cli/__init__.py b/core/switch_core/cli/__init__.pydeleted file mode 100644index ecf3a7887..000000000--- a/core/switch_core/cli/__init__.py+++ /dev/null@@ -1,8 +0,0 @@-"""Operator commands that ship inside the image.--These live in the package rather than under `scripts/` because the deployed-image contains only `switch_core` — a script outside it cannot be run by a-Kubernetes Job or a Compose service, which is where the migration commands-have to run. They reach the homeserver and the database directly, so they are-invoked deliberately, never on every release.-"""diff --git a/core/switch_core/cli/backfill.py b/core/switch_core/cli/backfill.pydeleted file mode 100644index dbb2c55e3..000000000--- a/core/switch_core/cli/backfill.py+++ /dev/null@@ -1,210 +0,0 @@-"""Reconstruct room history from the message bus into the message log.--Rows exist from the moment the recorder was deployed. Everything older is on-the homeserver and nowhere else, which stopped being harmless when the read-path moved to Postgres: those messages are now invisible to every agent rather-than merely unqueried.--This walks each room to its start and writes the messages that have no row.-Reconstructed rows are numbered below zero, so they order correctly against-the live log without disturbing it, and no delivery cursor moves — nobody is-sent last March's messages tonight.--Safe to re-run. `transport_event_id` is unique and each page commits on its-own, so a walk that fails halfway leaves what it finished and picks up from-there.--Run it against a real deployment. Start with `--dry-run` on one room.--Usage:- just backfill-messages --dry-run- just backfill-messages --room <switch-room-uuid>- just backfill-messages--In a deployment it is the same command inside the image:- python -m switch_core.cli.backfill --dry-run-"""--from __future__ import annotations--import argparse-import asyncio-import sys--from sqlalchemy import select--from switch_core.config import SwitchConfig-from switch_core.db.engine import create_engine_from_config, create_session_factory-from switch_core.db.models import Room-from switch_core.db.stores.client_store import ClientStore-from switch_core.db.stores.message_store import MessageStore-from switch_core.messages import backfill_room-from switch_core.messages.backfill import MESSAGES_PER_ROOM-from switch_core.transport.matrix import MatrixTransport--# The admin client is a member of every room by design, which is what makes it-# the one identity that can read every room's history.-READER_CLIENT_TYPE = "admin"---def _parse_args() -> argparse.Namespace:- parser = argparse.ArgumentParser(description=__doc__)- parser.add_argument(- "--room",- help="A single Switch room id. Defaults to every non-archived room.",- )- parser.add_argument(- "--dry-run",- action="store_true",- help=(- "Read the history and report what would be written, without "- "writing it. Start here."- ),- )- parser.add_argument(- "--force",- action="store_true",- help=(- "Walk rooms that have already been backfilled to their start. "- "Without it they are skipped: the walk always starts from the "- "newest message, so re-reading a finished room costs its whole "- "history in pages to learn there is nothing to write."- ),- )- parser.add_argument(- "--messages-per-room",- type=int,- default=MESSAGES_PER_ROOM,- metavar="N",- help=(- "How many of a room's messages to carry across, newest first "- f"(default {MESSAGES_PER_ROOM}). 0 walks every room to its start. "- "Older messages are left on the homeserver, which is going away, "- "so this is what is kept rather than what is done first."- ),- )- parser.add_argument(- "--allow-empty",- action="store_true",- help=(- "Treat 'nothing to walk' as success. For unattended runs — a "- "first boot has no admin client and no rooms yet, and a stack "- "that refuses to start over an empty database is worse than one "- "with nothing to backfill."- ),- )- return parser.parse_args()---class _ReadOnlyStore(MessageStore):- """Counts what a real run would write, and writes nothing.-- A dry run has to walk exactly as far as a real one and make exactly the- same decisions, so it shares everything but the insert.- """-- async def create_historical(self, session, message, attachments): # type: ignore[no-untyped-def]- return message-- async def mark_history_backfilled(self, session, room_id): # type: ignore[no-untyped-def]- """A room a dry run walked has not been backfilled.-- Marking it would make the walk that wrote nothing look like the one- that did, and the real run afterwards would find no rooms left to do.- """- return None---async def _run(args: argparse.Namespace) -> int:- config = SwitchConfig()- engine = create_engine_from_config(config)- session_factory = create_session_factory(engine)-- async with session_factory() as session:- readers = await ClientStore().get_by_type(session, READER_CLIENT_TYPE)- if not readers:- print(- f"No {READER_CLIENT_TYPE} client exists, so no identity can read "- "every room. Start switch-core once to provision it.",- file=sys.stderr,- )- return 0 if args.allow_empty else 2- reader = readers[0]- query = select(Room).where(Room.archived_at.is_(None))- if args.room:- query = query.where(Room.id == args.room)- if not args.force:- query = query.where(Room.history_backfilled_at.is_(None))- rooms = list((await session.execute(query)).scalars().all())-- if not rooms:- print(- "No rooms to backfill — every room has already been walked to its "- "start, or there are none. Use --force to walk them again.",- file=sys.stderr,- )- return 0 if args.allow_empty else 2-- transport = MatrixTransport(- server_url=config.matrix_server,- user_id=reader.matrix_user_id,- password=reader.password,- device_id=reader.device_id,- access_token=reader.access_token,- )- await transport.connect()- joined = set(await transport.joined_rooms())-- store: MessageStore = _ReadOnlyStore() if args.dry_run else MessageStore()- written = 0- incomplete = 0- unreadable: list[str] = []- try:- for room in rooms:- if room.matrix_room_id not in joined:- # A room nobody can read is not a room with no history, and- # reporting it as done would be the one answer that misleads.- unreadable.append(f"{room.name} ({room.matrix_room_id})")- continue- report = await backfill_room(- transport,- session_factory,- room,- store=store,- messages_per_room=args.messages_per_room,- )- print(report.summary())- written += report.written- incomplete += 1 if report.incomplete else 0- finally:- await transport.close()- await engine.dispose()-- if unreadable:- print(- f"\n{len(unreadable)} room(s) the {READER_CLIENT_TYPE} client is not "- "in, so they were not backfilled:",- file=sys.stderr,- )- for where in unreadable:- print(f" {where}", file=sys.stderr)-- verb = "would be written" if args.dry_run else "written"- print(- f"\n{len(rooms) - len(unreadable)} room(s) walked, {written} message(s) {verb}."- )- if incomplete:- print(- f"{incomplete} room(s) hit the page limit and are only partially "- "reconstructed. Re-run to continue.",- file=sys.stderr,- )- return 1 if (unreadable or incomplete) else 0---def main() -> int:- return asyncio.run(_run(_parse_args()))---if __name__ == "__main__":- raise SystemExit(main())diff --git a/core/switch_core/cli/reconcile.py b/core/switch_core/cli/reconcile.pydeleted file mode 100644index 22a991f13..000000000--- a/core/switch_core/cli/reconcile.py+++ /dev/null@@ -1,173 +0,0 @@-"""Check that the recorded messages agree with what the message bus holds.--Every message is written to Postgres alongside the send, but the write happens-after delivery and cannot fail it — so a database problem leaves a row missing-for a message that really was sent. This walks a room's history on the bus,-compares it against the recorded rows, and reports the difference.--Run it against a real deployment before moving any read path onto those rows.-Unit tests cannot answer this question: the thing being checked is whether two-records of the same live traffic agree.--Only the window in which recording was active is compared — from the oldest-recorded message in each room, or from `--since`. Messages older than that are-legitimately absent and are not reported as drift.--Exits non-zero when any room disagrees, so it can gate a deployment.--Usage:- just reconcile-messages- just reconcile-messages --room <switch-room-uuid>- just reconcile-messages --since 2026-09-01T00:00:00+00:00- just reconcile-messages --verbose--In a deployment it is the same command inside the image:- python -m switch_core.cli.reconcile --verbose-"""--from __future__ import annotations--import argparse-import asyncio-import sys-from datetime import datetime--from sqlalchemy import select--from switch_core.config import SwitchConfig-from switch_core.db.engine import create_engine_from_config, create_session_factory-from switch_core.db.models import Room-from switch_core.db.stores.client_store import ClientStore-from switch_core.messages import RoomReconciliation, reconcile_room-from switch_core.transport.matrix import MatrixTransport--# The admin client is a member of every room by design, which is what makes it-# the one identity that can read every room's history.-READER_CLIENT_TYPE = "admin"---def _parse_args() -> argparse.Namespace:- parser = argparse.ArgumentParser(description=__doc__)- parser.add_argument(- "--room",- help="A single Switch room id. Defaults to every non-archived room.",- )- parser.add_argument(- "--since",- help=(- "ISO-8601 instant to compare from, overriding the per-room "- "recorded-from cutoff. Anything older is not reported as missing."- ),- )- parser.add_argument(- "--verbose",- action="store_true",- help="List every differing event rather than only the counts.",- )- return parser.parse_args()---def _report(reconciliation: RoomReconciliation, *, verbose: bool) -> None:- print(reconciliation.summary())- if reconciliation.ignored_by_type:- counts = ", ".join(- f"{name}={count}"- for name, count in sorted(reconciliation.ignored_by_type.items())- )- print(f" not sends, so not compared: {counts}")- if reconciliation.unverifiable_by_type:- counts = ", ".join(- f"{name}={count}"- for name, count in sorted(reconciliation.unverifiable_by_type.items())- )- print(f" rows the bus walk cannot check: {counts}")- if not reconciliation.anchored:- print(- " the oldest recorded event was not found on the bus, so the "- "start of the window is approximate"- )- if not verbose:- return- for event_id in reconciliation.missing_rows:- print(f" missing row: {event_id}")- for event_id in reconciliation.unsent_rows:- print(f" recorded but not on the bus: {event_id}")- for mismatch in reconciliation.mismatches:- print(f" {mismatch.describe()}")---async def _run(args: argparse.Namespace) -> int:- since = datetime.fromisoformat(args.since) if args.since else None-- config = SwitchConfig()- engine = create_engine_from_config(config)- session_factory = create_session_factory(engine)-- async with session_factory() as session:- readers = await ClientStore().get_by_type(session, READER_CLIENT_TYPE)- if not readers:- print(- f"No {READER_CLIENT_TYPE} client exists, so no identity can read "- "every room. Start switch-core once to provision it.",- file=sys.stderr,- )- return 2- reader = readers[0]- query = select(Room).where(Room.archived_at.is_(None))- if args.room:- query = query.where(Room.id == args.room)- rooms = list((await session.execute(query)).scalars().all())-- if not rooms:- print("No rooms to check.", file=sys.stderr)- return 2-- transport = MatrixTransport(- server_url=config.matrix_server,- user_id=reader.matrix_user_id,- password=reader.password,- device_id=reader.device_id,- access_token=reader.access_token,- )- await transport.connect()- joined = set(await transport.joined_rooms())-- drifted = 0- unreadable: list[str] = []- try:- for room in rooms:- if room.matrix_room_id not in joined:- # Reporting a room nobody can read as clean would be the worst- # possible answer, so it is named and counted as a failure.- unreadable.append(f"{room.name} ({room.matrix_room_id})")- continue- async with session_factory() as session:- reconciliation = await reconcile_room(- transport, session, room, since=since- )- _report(reconciliation, verbose=args.verbose)- if not reconciliation.clean:- drifted += 1- finally:- await transport.close()- await engine.dispose()-- if unreadable:- print(- f"\n{len(unreadable)} room(s) the {READER_CLIENT_TYPE} client is not "- "in, so they were not checked:",- file=sys.stderr,- )- for where in unreadable:- print(f" {where}", file=sys.stderr)-- print(f"\n{len(rooms) - len(unreadable)} room(s) checked, {drifted} with drift.")- return 1 if (drifted or unreadable) else 0---def main() -> int:- return asyncio.run(_run(_parse_args()))---if __name__ == "__main__":- raise SystemExit(main())diff --git a/core/switch_core/clients/client_base.py b/core/switch_core/clients/client_base.pyindex c82212844..995074cf9 100644--- a/core/switch_core/clients/client_base.py+++ b/core/switch_core/clients/client_base.py@@ -2,9 +2,8 @@ from __future__ import annotationsimport asyncioimport logging-import randomimport time-from typing import TYPE_CHECKING, Literal, NotRequired, TypedDict+from typing import TYPE_CHECKING, Literal, TypedDictfrom pydantic import BaseModelfrom sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker@@ -22,7 +21,6 @@ from switch_core.events import (TaskUpdate,ToolCallReport,)-from switch_core.messages.recording import MessageRecordingfrom switch_core.transport import (InboundCustomEvent,InboundEvent,@@ -31,7 +29,6 @@ from switch_core.transport import (InboundMessage,MessageTransport,RoomRef,- SendResult,TransportError,TransportHandlers,)@@ -41,18 +38,6 @@ if TYPE_CHECKING:logger = logging.getLogger(__name__)-SYNC_STATE_INTERVAL = 30.0-# Every client in this process starts when the process does, and its sync-# long-poll is SYNC_STATE_INTERVAL as well, so a fixed interval has all of them-# crossing the threshold in the same instant — hundreds of one-row transactions-# competing for the connection pool at once. A per-client interval keeps them-# spread out and stops them re-converging after a restart.-SYNC_STATE_JITTER = 15.0-# A sync that carried nothing durable still advances the cursor, and persisting-# that is wasted work: resuming from the older cursor replays the same nothing.-# The token should still not be left arbitrarily far behind the server, so an-# unwritten one is flushed once it is this stale.-SYNC_STATE_MAX_STALENESS = 900.0SYNC_MAX_RETRIES = 5SYNC_BACKOFF_BASE = 1.0SYNC_BACKOFF_CAP = 60.0@@ -70,8 +55,8 @@ class ClientBaseKwargs[ConfigT: ClientConfig](TypedDict):sides: the subclass cannot be told it is missing something, and a callercannot be told it is passing something that no longer exists. A stale`device_id=` type-checked clean that way and took all four collaboration- bridges down at startup — the credential had moved into `session_state`- and nothing said so until the process refused to start.+ bridges down at startup — the credential had moved elsewhere and nothing+ said so until the process refused to start.Declaring the shape here and unpacking it restores both checks withoutmaking every subclass restate eleven parameters.@@ -80,15 +65,10 @@ class ClientBaseKwargs[ConfigT: ClientConfig](TypedDict):client_id: strmatrix_user_id: strdisplay_name: str- password: str- server_url: strsession_factory: async_sessionmaker[AsyncSession]client_store: ClientStoreconfig: ConfigTtransport_factory: Callable[[ClientBase[ConfigT]], MessageTransport]- session_state: dict[str, str | None]- message_recorder: MessageRecording- next_batch_token: NotRequired[str | None]class ClientBase[ConfigT: ClientConfig]:@@ -100,30 +80,17 @@ class ClientBase[ConfigT: ClientConfig]:client_id: str,matrix_user_id: str,display_name: str,- password: str,- server_url: str,session_factory: async_sessionmaker[AsyncSession],client_store: ClientStore,config: ConfigT,transport_factory: Callable[[ClientBase[ConfigT]], MessageTransport],- session_state: dict[str, str | None],- message_recorder: MessageRecording,- next_batch_token: str | None = None,) -> None:self.client_id = client_idself.matrix_user_id = matrix_user_idself.display_name = display_name- self.password = password- self.server_url = server_urlself.session_factory = session_factoryself.client_store = client_storeself.config = config- self.message_recorder = message_recorder-- # Credentials the transport owns. Stored and handed back unread, so- # what authentication needs is the transport's business alone.- self.session_state = session_state- self.next_batch_token = next_batch_tokenself._transport_factory = transport_factoryself.transport: MessageTransport | None = None@@ -137,13 +104,6 @@ class ClientBase[ConfigT: ClientConfig]:self._self_join_dispatched: set[str] = set()self._ready = asyncio.Event()self._startup_ts: int = 0- self._sync_persist_interval = SYNC_STATE_INTERVAL + random.uniform(- 0.0, SYNC_STATE_JITTER- )- self._last_sync_persist: float = time.monotonic() - random.uniform(- 0.0, SYNC_STATE_INTERVAL- )- self._sync_state_dirty: bool = Falseself._running: bool = Falseasync def wait_ready(self) -> None:@@ -177,15 +137,13 @@ class ClientBase[ConfigT: ClientConfig]:"""Block until this client is a member of `room_id`, up to `timeout`seconds. Returns True if joined, False on timeout. Callers that need to*send* into a room they have only just been invited to must await this- first — a send issued before the join lands is rejected by the- homeserver, and any event that does land before a member's join is- filtered out by `_should_ignore`.-- A join observed through sync sets the event, but a join that predates- this process never replays: the client resumes from a stored- `next_batch` token, so an incremental sync carries no member event for a- room it joined in an earlier run, and re-inviting an existing member is- a no-op. The homeserver is therefore asked directly before waiting."""+ first: any event that lands before a member's join is filtered out by+ `_should_ignore`.++ A join delivered to this client sets the event, but a join that+ predates the process is never redelivered — delivery starts at each+ room's head — and re-inviting an existing member is a no-op. Membership+ is therefore read directly before waiting."""event = self._joined_event(room_id)if event.is_set():return True@@ -219,7 +177,7 @@ class ClientBase[ConfigT: ClientConfig]:retries = 0while self._running:try:- await self._transport.receive_forever(since=self.next_batch_token)+ await self._transport.receive_forever()retries = 0except Exception:if not self._running:@@ -245,7 +203,6 @@ class ClientBase[ConfigT: ClientConfig]:async def stop(self) -> None:logger.info("Stopping client %s", self.matrix_user_id)self._running = False- await self._persist_state(force=True)await self.teardown()if self.transport is not None:await self.transport.close()@@ -259,30 +216,17 @@ class ClientBase[ConfigT: ClientConfig]:on_member_event=self._handle_member_event,on_custom_event=self._handle_custom_event,on_invite=self._handle_invite,- on_sync=self._handle_sync,- on_sync_error=self._handle_sync_error,))async def teardown(self) -> None:pass- # ── Authentication ─────────────────────────────────────────────────────────-async def _create_client(self) -> None:- before = dict(self.session_state)self.transport = self._transport_factory(self)await self.transport.connect()- self._absorb_session_state()- if self.session_state != before:- await self._persist_state(force=True)-self._ready.set()- def _absorb_session_state(self) -> None:- """Take back whatever the transport wants persisted, without reading it."""- self.session_state = dict(self._transport.session_state)-# ── Internal event handlers (filtering + dispatch to hooks) ────────────────async def _handle_message(self, room: RoomRef, event: InboundMessage) -> None:@@ -321,16 +265,6 @@ class ClientBase[ConfigT: ClientConfig]:if event.state_key == self.matrix_user_id:if event.membership == "join":self._mark_joined(room.room_id, event.timestamp)- if event.prev_membership != "join":- # Recorded on the arrival itself rather than under the- # announcement guards below: the log wants every arrival,- # including ones too old to be worth announcing.- await self.message_recorder.record_join(- transport_room_id=room.room_id,- event=event,- client_id=self.client_id,- member_name=self.display_name,- )# A membership-preserving update (display name, avatar) re-fires# m.room.member with membership == "join"; only a transition into# membership is an arrival. Joins predating this process are not@@ -436,28 +370,6 @@ class ClientBase[ConfigT: ClientConfig]:"Error in handler for %s in %s", event.event_type, room.room_id)- async def _handle_sync(self, next_batch: str, durable: bool) -> None:- if not next_batch or next_batch == self.next_batch_token:- return- self.next_batch_token = next_batch- if durable:- await self._persist_state()- return- self._sync_state_dirty = True- if (time.monotonic() - self._last_sync_persist) >= SYNC_STATE_MAX_STALENESS:- await self._persist_state(force=True)-- async def _handle_sync_error(self, message: str) -> None:- logger.error("Sync error for %s: %s", self.matrix_user_id, message)- if self.password:- logger.info("Attempting re-login for %s", self.matrix_user_id)- try:- await self._transport.relogin()- except TransportError as exc:- logger.error("%s", exc)- return- self._absorb_session_state()-# ── Filtering ──────────────────────────────────────────────────────────────def _should_ignore(self, room: RoomRef, event: InboundEvent) -> bool:@@ -548,7 +460,6 @@ class ClientBase[ConfigT: ClientConfig]:except TransportError as exc:logger.error("Failed to send message to %s: %s", room_id, exc)return None- await self._record(room_id, result)return result.event_idasync def send_event(@@ -561,7 +472,6 @@ class ClientBase[ConfigT: ClientConfig]:delivered when it was not corrupts whatever it was coordinating."""result = await self._transport.send_event(room_id, event_type, content)- await self._record(room_id, result)return result.event_idasync def upload_media(self, data: bytes, content_type: str, filename: str) -> str:@@ -616,18 +526,8 @@ class ClientBase[ConfigT: ClientConfig]:except TransportError as exc:logger.error("Failed to send media to %s: %s", room_id, exc)return None- await self._record(room_id, result)return result.event_id- async def _record(self, room_id: str, result: SendResult) -> None:- await self.message_recorder.record(- transport_room_id=room_id,- result=result,- sender_matrix_id=self.matrix_user_id,- sender_client_id=self.client_id,- sender_name=self.display_name,- )-async def set_typing(self, room_id: str, is_typing: bool) -> None:await self._transport.set_typing(room_id, is_typing)@@ -637,25 +537,3 @@ class ClientBase[ConfigT: ClientConfig]:if await self._transport.join_room(room_id):self._mark_joined(room_id, int(time.time() * 1000))logger.info("Client %s joined %s", self.matrix_user_id, room_id)-- # ── State persistence ──────────────────────────────────────────────────────-- async def _persist_state(self, force: bool = False) -> None:- now = time.monotonic()- if not force and (now - self._last_sync_persist) < self._sync_persist_interval:- self._sync_state_dirty = True- return-- try:- async with self.session_factory() as session:- await self.client_store.update_state(- session,- self.client_id,- next_batch_token=self.next_batch_token,- **self.session_state,- )- await session.commit()- self._last_sync_persist = now- self._sync_state_dirty = False- except Exception:- logger.exception("Failed to persist state for %s", self.matrix_user_id)diff --git a/core/switch_core/clients/client_factory.py b/core/switch_core/clients/client_factory.pyindex 37c2b6fef..5ba5cd2f3 100644--- a/core/switch_core/clients/client_factory.py+++ b/core/switch_core/clients/client_factory.py@@ -12,37 +12,15 @@ from switch_core.db.stores.client_store import ClientStorefrom switch_core.db.stores.media_store import MediaStorefrom switch_core.db.stores.message_store import MessageStorefrom switch_core.db.stores.room_store import RoomStore-from switch_core.messages import MessageRecorderfrom switch_core.messages.notify import MessageListener-from switch_core.messages.recording import MessageRecording, NoRecordingfrom switch_core.transport import MessageTransportfrom switch_core.transport.ephemeral import EphemeralBusfrom switch_core.transport.invites import InviteBus-from switch_core.transport.matrix import MatrixTransportfrom switch_core.transport.postgres import PostgresTransportlogger = logging.getLogger(__name__)-def matrix_transport_for(client: ClientBase[Any]) -> MessageTransport:- """Build the transport a client runs on.-- Choosing the implementation is the factory's job, so a client never names- one.-- The config type is `Any` because nothing here reads the config: a factory- that works for every client would otherwise have to be re-typed for each- subclass, since `ClientBase` is invariant in it.- """- return MatrixTransport(- server_url=client.server_url,- user_id=client.matrix_user_id,- password=client.password,- device_id=client.session_state.get("device_id"),- access_token=client.session_state.get("access_token"),- )--class ClientFactory:def __init__(self,@@ -50,7 +28,6 @@ class ClientFactory:client_store: ClientStore,session_factory: async_sessionmaker[AsyncSession],config: SwitchConfig,- message_recorder: MessageRecorder,room_store: RoomStore,message_store: MessageStore,media_store: MediaStore,@@ -61,7 +38,6 @@ class ClientFactory:self._client_store = client_storeself._session_factory = session_factoryself._config = config- self._message_recorder = message_recorderself._room_store = room_storeself._message_store = message_storeself._media_store = media_store@@ -90,18 +66,10 @@ class ClientFactory:client_id=record.id,matrix_user_id=record.matrix_user_id,display_name=record.display_name,- password=record.password,- server_url=self._config.matrix_server,session_factory=self._session_factory,client_store=self._client_store,config=config,transport_factory=self.transport_for,- message_recorder=self.recorder(),- session_state={- "access_token": record.access_token,- "device_id": record.device_id,- },- next_batch_token=record.next_batch_token,**extra_kwargs,)@@ -110,14 +78,9 @@ class ClientFactory:Public because not every client is built by `create`: a collaborationbridge's client is constructed by its own lifecycle service, which- carries per-bridge state the registry cannot hold. It still has to make- the same choice, and a caller that named an implementation directly- would put its clients on a different bus from everyone else's — agents- talking in rows while bridges talk to a homeserver, each side healthy- and neither reaching the other.+ carries per-bridge state the registry cannot hold. It goes through here+ so that every client in the process is built the same way."""- if self._config.message_transport == "matrix":- return matrix_transport_for(client)return PostgresTransport(user_id=client.matrix_user_id,client_id=client.client_id,@@ -130,15 +93,3 @@ class ClientFactory:invites=self._invites,ephemeral=self._ephemeral,)-- def recorder(self) -> MessageRecording:- """The recorder that goes with the transport.-- Paired here rather than configured separately: a transport that stores- what it carries needs no recorder, and one that does not needs one.- Splitting the decision would let a deployment pick both and write every- message twice.- """- if self._config.message_transport == "matrix":- return self._message_recorder- return NoRecording()diff --git a/core/switch_core/clients/client_lifecycle_service.py b/core/switch_core/clients/client_lifecycle_service.pyindex a76b8ba74..32909af89 100644--- a/core/switch_core/clients/client_lifecycle_service.py+++ b/core/switch_core/clients/client_lifecycle_service.py@@ -78,20 +78,12 @@ class ClientLifecycleService:if localpart is None:localpart = f"switch-{client_type}-{client_id[:8]}"matrix_user_id = self._make_user_id(localpart)- password = str(uuid.uuid4())-- await self._matrix_admin.register_user(- user_id=matrix_user_id,- password=password,- display_name=display_name,- )record = Client(id=client_id,matrix_user_id=matrix_user_id,display_name=display_name,type=client_type,- password=password,config=config,)async with self._session_factory() as session:diff --git a/core/switch_core/clients/user_client.py b/core/switch_core/clients/user_client.pydeleted file mode 100644index fe179f52f..000000000--- a/core/switch_core/clients/user_client.py+++ /dev/null@@ -1,5 +0,0 @@-from switch_core.clients.client_base import ClientBase, ClientConfig---class UserClient(ClientBase[ClientConfig]):- passdiff --git a/core/switch_core/config.py b/core/switch_core/config.pyindex 9fd8fa840..26fdf6c48 100644--- a/core/switch_core/config.py+++ b/core/switch_core/config.py@@ -1,5 +1,4 @@import re-from typing import Literalfrom urllib.parse import urlsplitfrom pydantic import model_validator@@ -18,18 +17,10 @@ class SwitchConfig(BaseSettings):db_password: strdb_name: str- # Which transport carries messages: "matrix" for the homeserver,- # "postgres" for the message table. The choice pairs a transport with a- # provisioning implementation and a recorder, so it is one setting rather- # than three. Defaults to matrix, so an existing deployment is unchanged- # until it is asked to move.- message_transport: Literal["matrix", "postgres"] = "matrix"-- matrix_server: str+ # The server half of every client's `@localpart:server` id. Not a+ # homeserver address — nothing is contacted at it — but the ids are stable+ # public handles, so the shape outlives the homeserver that chose it.matrix_server_name: str- matrix_admin_user: str- matrix_admin_password: str- matrix_registration_shared_secret: stragent_registration_token: str# JWT authdiff --git a/core/switch_core/db/models.py b/core/switch_core/db/models.pyindex 19b157c2e..c01d4961d 100644--- a/core/switch_core/db/models.py+++ b/core/switch_core/db/models.py@@ -78,10 +78,6 @@ class Client(Base):matrix_user_id: Mapped[str] = mapped_column(Text, unique=True, nullable=False)display_name: Mapped[str] = mapped_column(Text, nullable=False)type: Mapped[str] = mapped_column(Text, nullable=False)- password: Mapped[str] = mapped_column(Text, nullable=False)- device_id: Mapped[str | None] = mapped_column(Text, nullable=True)- access_token: Mapped[str | None] = mapped_column(Text, nullable=True)- next_batch_token: Mapped[str | None] = mapped_column(Text, nullable=True)config: Mapped[dict | None] = mapped_column(JSONB, nullable=True)created_at: Mapped[str] = mapped_column(DateTime(timezone=True), server_default=func.now(), nullable=False@@ -305,14 +301,6 @@ class Room(Base):archived_at: Mapped[str | None] = mapped_column(DateTime(timezone=True), nullable=True)- # Set when this room's history has been walked to its start and written- # into the message log. Null means never completed — including a walk that- # stopped at the page limit, which leaves the room partly reconstructed.- # The walk always starts from the newest message, so without this a re-run- # reads every page of every room again to discover it has nothing to do.- history_backfilled_at: Mapped[str | None] = mapped_column(- DateTime(timezone=True), nullable=True- )# ── Room Groups ─────────────────────────────────────────────────────────────────@@ -828,18 +816,18 @@ class RoleLease(Base):class BridgeMessageMap(Base):- """Durable correlation between a Matrix event and its external counterpart.+ """Durable correlation between a Switch event and its external counterpart.- One row per bridged message, written in both directions (Matrix→external- and external→Matrix). Powers thread bridging (resolving a Matrix thread- root to its external post and vice versa) and the edit/delete sync that- previously relied on a volatile in-memory dict. Unique in both directions- per bridge so either id resolves the other.+ One row per bridged message, written in both directions. Powers thread+ bridging (resolving a Switch thread root to its external post and vice+ versa) and the edit/delete sync that previously relied on a volatile+ in-memory dict. Unique in both directions per bridge so either id resolves+ the other."""__tablename__ = "bridge_message_map"__table_args__ = (- UniqueConstraint("bridge_id", "matrix_event_id"),+ UniqueConstraint("bridge_id", "transport_event_id"),UniqueConstraint("bridge_id", "external_post_id"),)@@ -848,7 +836,7 @@ class BridgeMessageMap(Base):Text, ForeignKey("collaboration_bridges.id", ondelete="CASCADE"), nullable=False)external_channel_id: Mapped[str] = mapped_column(Text, nullable=False)- matrix_event_id: Mapped[str] = mapped_column(Text, nullable=False)+ transport_event_id: Mapped[str] = mapped_column(Text, nullable=False)external_post_id: Mapped[str] = mapped_column(Text, nullable=False)created_at: Mapped[str] = mapped_column(DateTime(timezone=True), server_default=func.now(), nullable=False@@ -926,7 +914,7 @@ class Message(Base):Text, ForeignKey("rooms.id", ondelete="CASCADE"), nullable=False)transport_event_id: Mapped[str] = mapped_column(Text, unique=True, nullable=False)- sender_matrix_id: Mapped[str] = mapped_column(Text, nullable=False)+ sender_id: Mapped[str] = mapped_column(Text, nullable=False)sender_client_id: Mapped[str | None] = mapped_column(Text, ForeignKey("clients.id", ondelete="SET NULL"), nullable=True)diff --git a/core/switch_core/db/stores/bridge_message_map_store.py b/core/switch_core/db/stores/bridge_message_map_store.pyindex 8dffa7ed7..a406fcce9 100644--- a/core/switch_core/db/stores/bridge_message_map_store.py+++ b/core/switch_core/db/stores/bridge_message_map_store.py@@ -12,13 +12,13 @@ class BridgeMessageMapStore:await session.flush()return mapping- async def get_by_matrix_event_id(- self, session: AsyncSession, bridge_id: str, matrix_event_id: str+ async def get_by_transport_event_id(+ self, session: AsyncSession, bridge_id: str, transport_event_id: str) -> BridgeMessageMap | None:result = await session.execute(select(BridgeMessageMap).where(BridgeMessageMap.bridge_id == bridge_id,- BridgeMessageMap.matrix_event_id == matrix_event_id,+ BridgeMessageMap.transport_event_id == transport_event_id,))return result.scalar_one_or_none()@@ -34,13 +34,13 @@ class BridgeMessageMapStore:)return result.scalar_one_or_none()- async def delete_by_matrix_event_id(- self, session: AsyncSession, bridge_id: str, matrix_event_id: str+ async def delete_by_transport_event_id(+ self, session: AsyncSession, bridge_id: str, transport_event_id: str) -> None:await session.execute(delete(BridgeMessageMap).where(BridgeMessageMap.bridge_id == bridge_id,- BridgeMessageMap.matrix_event_id == matrix_event_id,+ BridgeMessageMap.transport_event_id == transport_event_id,))await session.flush()diff --git a/core/switch_core/db/stores/client_store.py b/core/switch_core/db/stores/client_store.pyindex 7c6d10307..3e35ecb35 100644--- a/core/switch_core/db/stores/client_store.py+++ b/core/switch_core/db/stores/client_store.py@@ -36,23 +36,3 @@ class ClientStore:if client:await session.delete(client)await session.flush()-- async def update_state(- self,- session: AsyncSession,- client_id: str,- *,- access_token: str | None = None,- device_id: str | None = None,- next_batch_token: str | None = None,- ) -> None:- client = await session.get(Client, client_id)- if client is None:- return- if access_token is not None:- client.access_token = access_token- if device_id is not None:- client.device_id = device_id- if next_batch_token is not None:- client.next_batch_token = next_batch_token- await session.flush()diff --git a/core/switch_core/db/stores/message_store.py b/core/switch_core/db/stores/message_store.pyindex dbb21bdbf..31f45203e 100644--- a/core/switch_core/db/stores/message_store.py+++ b/core/switch_core/db/stores/message_store.py@@ -1,10 +1,10 @@from collections.abc import Collection-from datetime import UTC, datetime+from datetime import datetimefrom sqlalchemy import func, select, textfrom sqlalchemy.ext.asyncio import AsyncSession-from switch_core.db.models import Message, MessageAttachment, Room+from switch_core.db.models import Message, MessageAttachmentclass MessageStore:@@ -31,22 +31,6 @@ class MessageStore:await session.flush()return message- async def mark_history_backfilled(- self, session: AsyncSession, room_id: str- ) -> None:- """Record that this room's history has been walked to its start.-- A write of the backfill's own progress, so it lives beside the writes- of its output: a dry run replaces this store wholesale, and anything- that reaches the database from outside it is a write a dry run does- not know to withhold.- """- room = await session.get(Room, room_id)- if room is None:- return- room.history_backfilled_at = datetime.now(UTC) # type: ignore[assignment]- await session.flush()-async def create_historical(self,session: AsyncSession,diff --git a/core/switch_core/gateway/rooms.py b/core/switch_core/gateway/rooms.pyindex bfadde2ef..0e81d45c0 100644--- a/core/switch_core/gateway/rooms.py+++ b/core/switch_core/gateway/rooms.py@@ -155,6 +155,7 @@ async def _build_room_detail(write_visibility=room.write_visibility,created_at=str(room.created_at),archived=room.archived_at is not None,+ transport_room_id=room.matrix_room_id,matrix_room_id=room.matrix_room_id,external_channel_id=room.external_channel_id,instructions=room.instructions,diff --git a/core/switch_core/gateway/schemas.py b/core/switch_core/gateway/schemas.pyindex abc709c32..d0c95411b 100644--- a/core/switch_core/gateway/schemas.py+++ b/core/switch_core/gateway/schemas.py@@ -69,6 +69,11 @@ class RoomSummary(BaseModel):class RoomDetail(RoomSummary):+ transport_room_id: str+ # Deprecated alias of `transport_room_id`, sent for the compatibility+ # window. Connectors are installed copies and only update when someone+ # clicks Update, so both names are carried until the shipped ones have+ # moved. Same value in both; nothing dereferences either.matrix_room_id: strexternal_channel_id: str | Noneinstructions: str | Nonediff --git a/core/switch_core/main.py b/core/switch_core/main.pyindex 0f07aaa7a..a677214d5 100644--- a/core/switch_core/main.py+++ b/core/switch_core/main.py@@ -88,12 +88,6 @@ from switch_core.db.stores.task_store import TaskStorefrom switch_core.db.stores.user_store import UserStorefrom switch_core.gateway.app import create_gateway_appfrom switch_core.gateway.auth import hash_password-from switch_core.matrix_admin import (- MatrixAdmin,- ensure_admin_exists,- wait_for_homeserver,-)-from switch_core.messages import MessageRecorderfrom switch_core.messages.notify import MessageListenerfrom switch_core.provisioning import Provisioningfrom switch_core.provisioning.postgres import PostgresProvisioning@@ -161,8 +155,6 @@ async def _connection_sweep_loop(protocol: ProtocolService) -> None:logging.getLogger("httpx").setLevel(logging.WARNING)-logging.getLogger("nio.rooms").setLevel(logging.ERROR)-logging.getLogger("nio.client.base_client").setLevel(logging.WARNING)class _QuietPollFilter(logging.Filter):@@ -215,11 +207,6 @@ async def run() -> None:room_role_store = RoomRoleStore()message_store = MessageStore()media_store = MediaStore()- message_recorder = MessageRecorder(- session_factory=session_factory,- room_store=room_store,- message_store=message_store,- )# ── Seed admin user + registration key ──────────────────────────────────await _seed_admin_user(session_factory, user_store, config)@@ -244,39 +231,16 @@ async def run() -> None:await resource_service.log_builtin_shadowing(session)# ── Provisioning ─────────────────────────────────────────────────────────- # The homeserver is only contacted when it is the transport. Reaching it- # unconditionally would keep a Postgres deployment waiting on, and failing- # to start without, a service it does not use.- matrix_admin: Provisioning- if config.message_transport == "matrix":- await wait_for_homeserver(config.matrix_server)- await ensure_admin_exists(- server_url=config.matrix_server,- username=config.matrix_admin_user,- password=config.matrix_admin_password,- shared_secret=config.matrix_registration_shared_secret,- )- matrix_admin = await MatrixAdmin.create(- server_url=config.matrix_server,- admin_user=config.matrix_admin_user,- admin_password=config.matrix_admin_password,- shared_secret=config.matrix_registration_shared_secret,- )- else:- logger.warning(- "Running on the Postgres transport: no homeserver is contacted, "- "and media uploaded before the switch cannot be served"- )- matrix_admin = PostgresProvisioning(- session_factory=session_factory,- room_store=room_store,- client_store=client_store,- message_store=message_store,- invites=invites,- )+ matrix_admin: Provisioning = PostgresProvisioning(+ session_factory=session_factory,+ room_store=room_store,+ client_store=client_store,+ message_store=message_store,+ invites=invites,+ )# One connection registry for the process. Created here rather than inside- # the agent bridge because the Matrix agent clients are wired first and read+ # the agent bridge because the room clients are wired first and read# presence from it — an agent is reachable if it has a live connection OR a# fresh heartbeat row (CHOO-1857 stage B).connections = ConnectionRegistry()@@ -286,7 +250,6 @@ async def run() -> None:client_store=client_store,session_factory=session_factory,config=config,- message_recorder=message_recorder,room_store=room_store,message_store=message_store,media_store=media_store,diff --git a/core/switch_core/matrix_admin.py b/core/switch_core/matrix_admin.pydeleted file mode 100644index 8777149b1..000000000--- a/core/switch_core/matrix_admin.py+++ /dev/null@@ -1,339 +0,0 @@-import asyncio-import hashlib-import hmac-import logging--import httpx--logger = logging.getLogger(__name__)--# Wire path served by Tuwunel/conduwuit for shared-secret account creation. It-# lives under the `/_synapse/` namespace for drop-in compatibility with tooling-# written against that endpoint — this is a protocol constant, NOT a reference-# to the homeserver implementation. Must stay verbatim.-SHARED_SECRET_REGISTER_PATH = "/_synapse/admin/v1/register"---class MatrixAdminError(Exception):- pass---async def wait_for_homeserver(- server_url: str, *, attempts: int = 60, delay: float = 1.0-) -> None:- """Block until the Matrix homeserver answers the standard- ``/_matrix/client/versions`` endpoint, or raise after ``attempts`` tries.-- Replaces the Docker healthcheck dependency: Tuwunel's slim image ships no- curl/wget, so the homeserver's readiness is polled here instead.- """- url = f"{server_url}/_matrix/client/versions"- async with httpx.AsyncClient(timeout=5) as http:- for attempt in range(1, attempts + 1):- try:- resp = await http.get(url)- if resp.is_success:- logger.info("Homeserver ready at %s", server_url)- return- except httpx.HTTPError:- pass- if attempt == 1 or attempt % 10 == 0:- logger.info(- "Waiting for homeserver at %s (attempt %d/%d)",- server_url,- attempt,- attempts,- )- await asyncio.sleep(delay)- raise MatrixAdminError(- f"Homeserver not reachable at {server_url} after {attempts} attempts"- )---async def _shared_secret_register(- http: httpx.AsyncClient,- register_url: str,- shared_secret: str,- username: str,- password: str,- *,- admin: bool,- display_name: str | None = None,-) -> dict | None:- """Create a local account via the shared-secret register endpoint.-- Authenticated by an HMAC-SHA1 over the homeserver's- ``registration_shared_secret``; bypasses interactive auth. Returns the- register response (containing ``access_token``/``device_id``) on success,- or ``None`` if the user already exists.- """- nonce_resp = await http.get(register_url)- nonce_resp.raise_for_status()- nonce = nonce_resp.json()["nonce"]-- mac = hmac.new(shared_secret.encode(), digestmod=hashlib.sha1)- mac.update(nonce.encode())- mac.update(b"\x00")- mac.update(username.encode())- mac.update(b"\x00")- mac.update(password.encode())- mac.update(b"\x00")- mac.update(b"admin" if admin else b"notadmin")-- body: dict[str, object] = {- "nonce": nonce,- "username": username,- "password": password,- "admin": admin,- "mac": mac.hexdigest(),- }- if display_name is not None:- body["displayname"] = display_name-- resp = await http.post(register_url, json=body)- if resp.status_code == 400 and "M_USER_IN_USE" in resp.text:- return None- resp.raise_for_status()- data: dict = resp.json()- return data---async def ensure_admin_exists(- server_url: str,- username: str,- password: str,- shared_secret: str,-) -> None:- if not shared_secret:- raise MatrixAdminError("No shared secret configured — cannot create admin user")-- async with httpx.AsyncClient(timeout=10) as http:- login_url = f"{server_url}/_matrix/client/r0/login"- try:- resp = await http.post(- login_url,- json={- "type": "m.login.password",- "user": username,- "password": password,- },- )- if resp.is_success:- logger.info("Matrix admin '%s' already exists", username)- return- except httpx.HTTPError as e:- logger.warning("Could not check if admin '%s' exists: %s", username, e)-- logger.info(- "Creating Matrix admin '%s' via shared-secret registration", username- )- await _shared_secret_register(- http,- f"{server_url}{SHARED_SECRET_REGISTER_PATH}",- shared_secret,- username,- password,- admin=True,- )- logger.info("Successfully created Matrix admin '%s'", username)---class MatrixAdmin:- def __init__(- self,- server_url: str,- admin_user: str,- access_token: str,- shared_secret: str,- http: httpx.AsyncClient,- ):- self.server_url = server_url- self.admin_user = admin_user- self.access_token = access_token- self._shared_secret = shared_secret- self._http = http-- @classmethod- async def create(- cls,- server_url: str,- admin_user: str,- admin_password: str,- shared_secret: str,- ) -> "MatrixAdmin":- http = httpx.AsyncClient(- base_url=server_url,- timeout=10,- headers={"Content-Type": "application/json"},- )- resp = await http.post(- "/_matrix/client/r0/login",- json={- "type": "m.login.password",- "user": admin_user,- "password": admin_password,- },- )- if not resp.is_success:- await http.aclose()- raise MatrixAdminError(- f"Admin login failed: {resp.status_code} {resp.text}"- )-- access_token: str | None = resp.json().get("access_token")- if not access_token:- await http.aclose()- raise MatrixAdminError("No access_token in login response")-- logger.info("Logged in as Matrix admin '%s'", admin_user)- instance = cls(server_url, admin_user, access_token, shared_secret, http)- return instance-- async def close(self) -> None:- await self._http.aclose()-- async def verify_login(self, user_id: str, password: str) -> bool:- """Whether ``password`` authenticates ``user_id`` on this homeserver.-- Used by the cutover to detect a pre-existing account whose password- differs from the one stored in the Switch DB — i.e. the target- homeserver is not the empty server a cutover expects.- """- localpart = user_id.split(":", 1)[0].lstrip("@")- resp = await self._http.post(- "/_matrix/client/r0/login",- json={- "type": "m.login.password",- "user": localpart,- "password": password,- },- )- return resp.is_success-- async def register_user(- self,- user_id: str,- password: str,- display_name: str | None = None,- is_admin: bool = False,- ) -> None:- """Provision a local Matrix account via shared-secret registration.-- ``user_id`` is the full Matrix id (``@localpart:server``); the- registration endpoint takes the localpart. Idempotent: an existing- user is treated as success.- """- localpart = user_id.split(":", 1)[0].lstrip("@")- result = await _shared_secret_register(- self._http,- SHARED_SECRET_REGISTER_PATH,- self._shared_secret,- localpart,- password,- admin=is_admin,- display_name=display_name,- )- if result is None:- logger.info("User %s already exists", user_id)- else:- logger.info("Registered user %s", user_id)-- async def create_room(self, name: str, topic: str) -> str:- resp = await self._http.post(- "/_matrix/client/v3/createRoom",- headers={"Authorization": f"Bearer {self.access_token}"},- json={- "name": name,- "topic": topic,- "visibility": "private",- "preset": "private_chat",- },- )- if not resp.is_success:- raise MatrixAdminError(- f"Failed to create room: {resp.status_code} {resp.text}"- )- room_id: str | None = resp.json().get("room_id")- if not room_id:- raise MatrixAdminError("No room_id in createRoom response")- logger.info("Created room %s (%s)", name, room_id)- return room_id-- async def invite_to_room(self, room_id: str, user_id: str) -> None:- """Invite a local user to a room. Provided the user has a running- client (every Switch participant does), its sync loop auto-accepts the- invite and joins — see ``ClientBase.on_invite``.-- This is how members are added to a room: there is no admin "force-join"- on Tuwunel/conduwuit, so room setup goes through ordinary invitations.- """- resp = await self._http.post(- f"/_matrix/client/v3/rooms/{room_id}/invite",- headers={"Authorization": f"Bearer {self.access_token}"},- json={"user_id": user_id},- )- if not resp.is_success:- # User is already a member — nothing to do. Tuwunel/conduwuit rejects- # the invite of an existing member with "cannot invite user that is- # joined or banned"; other servers say "already in the room" /- # "already joined". Treat all as success (a banned user can't be invited- # anyway, which is the same no-op for room setup).- if resp.status_code == 403 and (- "already in the room" in resp.text- or "already joined" in resp.text- or "joined or banned" in resp.text- ):- return- raise MatrixAdminError(- f"Failed to invite {user_id} to {room_id}: {resp.status_code} {resp.text}"- )- logger.info("Invited %s to %s", user_id, room_id)-- async def kick_user(self, room_id: str, user_id: str) -> None:- resp = await self._http.post(- f"/_matrix/client/v3/rooms/{room_id}/kick",- headers={"Authorization": f"Bearer {self.access_token}"},- json={"user_id": user_id},- )- if not resp.is_success:- # User is already out — nothing to do, the room teardown this serves- # wants them gone and they are. Tuwunel/conduwuit rejects it with- # "Cannot kick a user who is not apart of the room" (their typo);- # Synapse says "not in the room". The counterpart to the- # already-a-member tolerance in `invite_to_room`.- if resp.status_code == 403 and (- "not apart of the room" in resp.text- or "not in the room" in resp.text- or "not in room" in resp.text- ):- logger.debug(- "Skipped kicking %s from %s: already not a member",- user_id,- room_id,- )- return- raise MatrixAdminError(- f"Failed to kick {user_id} from {room_id}: {resp.status_code} {resp.text}"- )- logger.info("Kicked %s from %s", user_id, room_id)-- async def delete_room(self, room_id: str) -> None:- """Abandon a room: the admin leaves and forgets it.-- Tuwunel/conduwuit exposes no REST room-purge endpoint (purge is only- reachable via the server admin-room console). Callers kick the- remaining members first, so leaving + forgetting empties the room of- all Switch participants.- """- for action in ("leave", "forget"):- resp = await self._http.post(- f"/_matrix/client/v3/rooms/{room_id}/{action}",- headers={"Authorization": f"Bearer {self.access_token}"},- json={},- )- # Already gone / never joined is fine.- if not resp.is_success and resp.status_code not in (403, 404):- raise MatrixAdminError(- f"Failed to {action} room {room_id}: {resp.status_code} {resp.text}"- )- logger.info("Left and forgot room %s", room_id)diff --git a/core/switch_core/messages/__init__.py b/core/switch_core/messages/__init__.pyindex 31b8b41b1..d8583c80d 100644--- a/core/switch_core/messages/__init__.py+++ b/core/switch_core/messages/__init__.py@@ -1,19 +1,6 @@-from switch_core.messages.backfill import BackfillReport, backfill_room-from switch_core.messages.reconcile import (- Mismatch,- RoomReconciliation,- reconcile_room,-)from switch_core.messages.recorded_types import NOT_RECORDED, should_record-from switch_core.messages.recorder import MessageRecorder__all__ = ["NOT_RECORDED",- "BackfillReport",- "MessageRecorder",- "Mismatch",- "RoomReconciliation",- "backfill_room",- "reconcile_room","should_record",]diff --git a/core/switch_core/messages/backfill.py b/core/switch_core/messages/backfill.pydeleted file mode 100644index 099d0cc2c..000000000--- a/core/switch_core/messages/backfill.py+++ /dev/null@@ -1,326 +0,0 @@-"""Reconstructs a room's history from the bus into the message log.--Recording began when the recorder was deployed. Everything a room said before-that is on the message bus and nowhere else — which did not matter while the-read path went to the bus too, and matters a great deal now that it does not.-On the first deployment to record and then read, that was 731 of 739 bridged-messages: still on the homeserver, invisible to every agent.--This walks a room backwards and writes the messages it finds that have no row.--**Reconstructed rows are numbered below zero.** `seq` orders the room, so a-message from last month cannot take a number above one from this morning, and-it cannot take a number among the live rows either — those are taken, and-renumbering to make room would move every cursor pointing at them. Counting-down from zero keeps the order right, keeps `(room_id, seq)` unique, and makes-the sign say something worth being able to read off a row: positive was-recorded as it happened, negative was reconstructed afterwards. Delivery-cursors start at 0, so none of this is pushed at anybody.--**It is idempotent, and that is not incidental.** A backfill over a busy room-is long, and the useful thing to do with a walk that failed halfway is to run-it again. `transport_event_id` is unique, so every row this would write is-either absent or already correct; the walk checks before writing and the-constraint is the backstop.--**What it does not reconstruct.** Only what the log is scoped to keep — see-`recorded_types` — so bus traffic that was never part of the conversation-stays out, exactly as it does going forward. Arrivals are reconstructed:-membership is on the bus and the timeline reads wrong without them. Sender-display names come from the event where the bus carried one and are otherwise-null, which is what the read path already tolerates.-"""--from __future__ import annotations--import logging-from dataclasses import dataclass, field-from datetime import UTC, datetime-from typing import TYPE_CHECKING--from switch_core.db.models import Message, MessageAttachment-from switch_core.db.stores.message_store import MessageStore-from switch_core.messages.recorded_types import MEMBERSHIP_EVENT_TYPE, should_record-from switch_core.transport import (- InboundCustomEvent,- InboundEvent,- InboundMedia,- InboundMembership,- InboundMessage,-)--if TYPE_CHECKING:- from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker-- from switch_core.db.models import Room- from switch_core.transport import MessageTransport--logger = logging.getLogger(__name__)--# One page of backwards history, matching the reconciliation walk so a backfill-# puts no unusual shape of load on the homeserver.-PAGE_SIZE = 100--# A stop for a walk that would otherwise never end. Reached only if the-# homeserver keeps handing out pages with a cursor that advances forever; a-# room that ends returns no next token and stops on its own.-MAX_PAGES = 10_000--# How much of a room's past is worth carrying across. The walk runs newest-# first, so this keeps the most recent messages and abandons what is older —-# and abandons it for good, because the homeserver goes away after this.-#-# It is a product decision rather than a technical limit: recent context is-# what an agent reads, a migration that finishes in minutes is one an operator-# will actually run, and an unbounded walk over a room with years in it is the-# reason someone reaches for `--force` and then waits. Raise it with-# `--messages-per-room`, or pass 0 to walk a room to its start.-MESSAGES_PER_ROOM = 50---@dataclass-class BackfillReport:- room_id: str- matrix_room_id: str- room_name: str- pages_read: int = 0- seen: int = 0- written: int = 0- already_present: int = 0- skipped_by_type: dict[str, int] = field(default_factory=dict)- # True when the walk stopped because it ran out of pages rather than- # because the room ended. The room is then partially reconstructed, and- # saying so matters more than the count does.- incomplete: bool = False- # True when the walk stopped at `messages_per_room`. Deliberately not- # `incomplete`: that one reports a walk that failed to finish and makes the- # command exit non-zero, and Console fails an upgrade on that. Stopping- # where we were told to stop is success, and the room is marked done.- capped: bool = False-- def summary(self) -> str:- line = (- f"{self.room_name} ({self.matrix_room_id}): {self.written} written, "- f"{self.already_present} already recorded, {self.seen} seen "- f"over {self.pages_read} pages"- )- if self.skipped_by_type:- counts = ", ".join(- f"{name}={count}"- for name, count in sorted(self.skipped_by_type.items())- )- line += f"; not part of the log: {counts}"- if self.capped:- line += "; stopped at the per-room limit, older history not carried"- if self.incomplete:- line += "; INCOMPLETE — hit the page limit before the room ended"- return line---async def _mark_backfilled(- session_factory: async_sessionmaker[AsyncSession],- room: Room,- store: MessageStore,-) -> None:- """Record that this room has been walked to its start.-- Through the store, so a dry run withholds it along with everything else it- withholds. Marking a room that a dry run only pretended to write would- leave the real run with nothing to do and no way to say so.-- Only on a clean finish, too. A walk that stopped at the page limit leaves- the room partly reconstructed, and calling that done would strand the rest- of its history the next time round.- """- async with session_factory() as session:- await store.mark_history_backfilled(session, room.id)- await session.commit()---def _in_log(report: BackfillReport) -> int:- """How many of this room's messages the log now holds from this walk.-- Written plus already recorded, because the cap has to mean the same thing- on a second run as on the first. Counting only what this run wrote would- make every re-run carry another `messages_per_room` of older history.- """- return report.written + report.already_present---async def backfill_room(- transport: MessageTransport,- session_factory: async_sessionmaker[AsyncSession],- room: Room,- *,- store: MessageStore,- messages_per_room: int = MESSAGES_PER_ROOM,-) -> BackfillReport:- """Walk a room backwards, writing the messages that have no row.-- The transport must already be connected and a member of the room.-- Stops at `messages_per_room` loggable messages, or at the room's start if- that comes first; 0 means walk to the start. The count is of messages the- log would hold, found rather than written, so a second run over a room that- is already done stops in the same place instead of carrying fifty more.-- Each page is its own transaction. A walk that fails partway therefore- leaves the pages it finished written rather than rolling back an hour of- work, and re-running picks up where it stopped — which is the behaviour- that makes a long backfill practical rather than something to be afraid of.- """- report = BackfillReport(- room_id=room.id,- matrix_room_id=room.matrix_room_id,- room_name=room.name,- )- start: str | None = None- while report.pages_read < MAX_PAGES:- page = await transport.read_history(- room.matrix_room_id, start=start, limit=PAGE_SIZE- )- report.pages_read += 1-- # Newest first, which is how history comes back, and the direction the- # numbering has to run: each row written takes the next position below- # the last, so walking from the newest unrecorded message towards the- # room's start leaves the oldest message with the lowest number. Any- # other order would number the room backwards.- for raw in page.events:- if messages_per_room and _in_log(report) >= messages_per_room:- # Told where to stop, so stopping is a finished room rather- # than an abandoned one: mark it, or every start walks it again.- report.capped = True- await _mark_backfilled(session_factory, room, store)- return report- await _consider(raw, session_factory, room, store, report)-- end = page.next_token- if not end or end == start:- await _mark_backfilled(session_factory, room, store)- return report- start = end-- report.incomplete = True- logger.warning(- "Backfill of %s stopped at the %d page limit; the room is only "- "partially reconstructed",- room.matrix_room_id,- MAX_PAGES,- )- return report---async def _consider(- raw: object,- session_factory: async_sessionmaker[AsyncSession],- room: Room,- store: MessageStore,- report: BackfillReport,-) -> None:- """Write one bus event as a message row, if it belongs in the log."""- if not isinstance(raw, InboundEvent):- report.skipped_by_type["unrecognised"] = (- report.skipped_by_type.get("unrecognised", 0) + 1- )- return-- event_type = _event_type(raw)- if event_type is None or not should_record(event_type):- name = event_type or type(raw).__name__- report.skipped_by_type[name] = report.skipped_by_type.get(name, 0) + 1- return-- report.seen += 1- async with session_factory() as session:- existing = await store.get_by_transport_event_id(session, raw.event_id)- if existing is not None:- report.already_present += 1- return- message, attachments = _row_for(raw, room.id, event_type)- await store.create_historical(session, message, attachments)- await session.commit()- report.written += 1---def _event_type(raw: InboundEvent) -> str | None:- """The log's name for this event, or None if it has no place in the log."""- if isinstance(raw, InboundMembership):- # Only an arrival. A leave is not part of the timeline the log keeps,- # and recording one would put a row in history with nothing to say.- return MEMBERSHIP_EVENT_TYPE if raw.membership == "join" else None- if isinstance(raw, InboundCustomEvent):- return raw.event_type- if isinstance(raw, InboundMessage):- return "m.room.message"- return None---def _row_for(- raw: InboundEvent, room_id: str, event_type: str-) -> tuple[Message, list[MessageAttachment]]:- """Build the row and its files from a bus event.-- `sender_client_id` is left null. It is the Switch client that did the- sending, and the bus records an mxid rather than a client id; resolving it- would be a lookup per row for a column nothing reads. `sender_name` comes- from the event where the bus carried one.- """- content = dict(raw.content)- message = Message(- room_id=room_id,- transport_event_id=raw.event_id,- sender_matrix_id=raw.sender,- sender_client_id=None,- sender_name=_sender_name(raw, content),- event_type=event_type,- msgtype=getattr(raw, "msgtype", None),- body=_body(raw),- formatted_body=content.get("formatted_body"),- thread_root_event_id=getattr(raw, "thread_root_id", None),- content=content,- sent_at=datetime.fromtimestamp(raw.timestamp / 1000, tz=UTC),- )- return message, _attachments(raw, content)---def _sender_name(raw: InboundEvent, content: dict) -> str | None:- if isinstance(raw, InboundMembership):- return raw.display_name- name = content.get("sender_name")- return str(name) if name else None---def _body(raw: InboundEvent) -> str | None:- # An arrival carries no body: how it reads is the reader's to phrase, and- # a rendered sentence in the log would freeze today's wording into every- # row ever written. Matches what the live recorder does.- if isinstance(raw, InboundMembership):- return None- return getattr(raw, "body", None)---def _attachments(raw: InboundEvent, content: dict) -> list[MessageAttachment]:- """The files on this event, which on the bus is at most one.-- Matrix has no multi-attachment event, so a send of several files is- several events sharing a group key. Reconstructing them as one row would- mean holding a page's worth of parts and guessing which belong together;- they are written as they were sent instead, and the group key survives in- `content` for anyone who wants to coalesce them later.- """- if not isinstance(raw, InboundMedia):- return []- info = content.get("info") or {}- size = info.get("size") if isinstance(info, dict) else None- return [- MessageAttachment(- uri=raw.uri,- filename=content.get("filename") or raw.body,- mimetype=info.get("mimetype") if isinstance(info, dict) else None,- size=int(size) if isinstance(size, int) else None,- )- ]---__all__ = ["BackfillReport", "backfill_room"]diff --git a/core/switch_core/messages/reconcile.py b/core/switch_core/messages/reconcile.pydeleted file mode 100644index cf1af75f2..000000000--- a/core/switch_core/messages/reconcile.py+++ /dev/null@@ -1,446 +0,0 @@-"""Compares what the message bus holds for a room against what was recorded.--Recording happens after a send is accepted and cannot fail it, so a database-problem leaves a row missing for a message that really was delivered. That is-a deliberate trade, and this is how the cost of it is measured: before the-read path moves onto these rows, the two records have to be shown to agree on-a real room, not on fakes.--**Only the window where recording was active is compared.** Rows exist from-the moment the recorder was deployed; every message older than that is-legitimately absent, and reporting a room's entire back-catalogue as missing-would bury the one row that actually went astray. The window therefore starts-at the oldest recorded message unless the caller names an earlier point, and-every report says which window it covered.--**The window's floor is an event id, not a moment.** A row's `sent_at` is when-it was written, and writing happens after the send is accepted, so the bus-event for the very first recorded message is always a little older than the-row that describes it. A timestamp cutoff therefore excludes that event from-the bus side while including its row on the recorded side, and the oldest-message in every room reports as drift for as long as the room exists. Paging-back to the oldest row's own event id instead makes the boundary exact and-takes both clocks out of it. The timestamp survives only as a backstop, so a-room whose anchor event is gone from the bus terminates instead of walking to-the beginning of time — and says so rather than reporting the difference as-drift it cannot explain.--Events the bus replays that a send never produced — membership changes and-the like — are counted by type rather than dropped, so a category nobody-thought about shows up as unclassified instead of as agreement. Custom event-types the recorder deliberately does not keep are counted the same way, under-their own type name: they are absent by design, and calling that drift would-make the report worthless.--**Both sides apply that filter, not just the bus side.** A row of a type the-bus walk discards has nothing to be compared against, so measuring it against-an empty set reports it as recorded-but-never-sent every run, forever. Rows-for types written before they were denied are the case that remains; they are-counted under `unverifiable_by_type` — disclosed as outside the check rather-than quietly dropped from it.--**Arrivals are compared too**, from the point the log has one. Recording a-join is the newest write path there is, and leaving it out made the least-proven part of the system the one part nothing checked. Its window is its own:-every room was joined long before arrivals were recorded, so a join older than-the oldest recorded arrival cannot have a row and is counted rather than-reported. After that point a join without a row is drift like any other.-"""--from __future__ import annotations--from dataclasses import dataclass, field-from datetime import datetime, timedelta-from typing import TYPE_CHECKING, cast--from sqlalchemy import select--from switch_core.db.models import Message-from switch_core.messages.recorded_types import MEMBERSHIP_EVENT_TYPE, should_record-from switch_core.transport import (- InboundCustomEvent,- InboundEvent,- InboundMedia,- InboundMembership,- InboundMessage,-)--if TYPE_CHECKING:- from sqlalchemy.ext.asyncio import AsyncSession-- from switch_core.db.models import Room- from switch_core.transport import MessageTransport--# One page of backwards history. Matches the size the read path already uses,-# so a reconciliation walk puts no unusual shape of load on the homeserver.-PAGE_SIZE = 100--# How far past the window the walk may go looking for the anchor event before-# giving up. Only reached when the anchor is not on the bus at all; generous-# because the cost of overshooting is a few extra pages, and the cost of-# stopping early is a report that cannot tell drift from a short walk.-ANCHOR_SEARCH_SLACK = timedelta(hours=1)---@dataclass(frozen=True)-class Mismatch:- """One field that disagrees between the bus and the recorded row."""-- event_id: str- field_name: str- on_the_bus: object- recorded: object-- def describe(self) -> str:- return (- f"{self.event_id} {self.field_name}: "- f"bus={self.on_the_bus!r} recorded={self.recorded!r}"- )---@dataclass(frozen=True)-class RoomReconciliation:- room_id: str- matrix_room_id: str- room_name: str- # The window compared. None means the room had no recorded messages at- # all, so there was nothing to compare rather than nothing wrong.- since: datetime | None- compared: int = 0- agreed: int = 0- missing_rows: list[str] = field(default_factory=list)- unsent_rows: list[str] = field(default_factory=list)- mismatches: list[Mismatch] = field(default_factory=list)- ignored_by_type: dict[str, int] = field(default_factory=dict)- # Rows of a type the bus walk does not collect, so nothing could be- # compared against them. Not drift, but not checked either.- unverifiable_by_type: dict[str, int] = field(default_factory=dict)- # Whether the walk reached the oldest recorded row's own event. False means- # the boundary fell back to a timestamp, so the edge of the window is- # approximate and a difference there may be an artefact of the walk.- anchored: bool = True-- @property- def compared_nothing(self) -> bool:- return self.since is None-- @property- def clean(self) -> bool:- return not (self.missing_rows or self.unsent_rows or self.mismatches)-- def summary(self) -> str:- where = f"{self.room_name} ({self.matrix_room_id})"- if self.since is None:- return f"{where}: no recorded messages, nothing to compare"- verdict = "clean" if self.clean else "DRIFT"- line = (- f"{where}: {verdict} — {self.agreed}/{self.compared} agreed, "- f"{len(self.missing_rows)} missing, {len(self.unsent_rows)} unsent, "- f"{len(self.mismatches)} mismatched (since {self.since.isoformat()})"- )- if self.unverifiable_by_type:- skipped = sum(self.unverifiable_by_type.values())- line += f"; {skipped} rows not checkable against the bus"- if not self.anchored:- line += "; WINDOW UNANCHORED — oldest recorded event not found on the bus"- return line---async def reconcile_room(- transport: MessageTransport,- session: AsyncSession,- room: Room,- *,- since: datetime | None = None,-) -> RoomReconciliation:- """Compare one room. `since` overrides the recorded-from cutoff.-- The transport must already be connected and a member of the room.- """- anchor = await _oldest_recorded(session, room.id)- # `Message.sent_at` is a timestamptz the models annotate as `str` — the- # established convention across every model in this package.- recorded_from = cast("datetime | None", anchor.sent_at) if anchor else None- window_start = since or recorded_from- if window_start is None:- return RoomReconciliation(- room_id=room.id,- matrix_room_id=room.matrix_room_id,- room_name=room.name,- since=None,- )-- # An explicit `since` is the caller overriding the boundary, so the anchor- # only applies when the window is the recorded one.- anchor_event_id = anchor.transport_event_id if anchor and since is None else None- oldest_join = await _oldest_recorded_join(session, room.id)- on_the_bus, ignored, anchored = await _read_back_to(- transport,- room.matrix_room_id,- anchor_event_id=anchor_event_id,- floor_ms=_to_ms(window_start - ANCHOR_SEARCH_SLACK),- window_start_ms=_to_ms(window_start),- joins_from_ms=_joins_from_ms(oldest_join),- join_anchor_id=oldest_join.transport_event_id if oldest_join else None,- )- recorded = await _recorded_since(session, room.id, window_start)-- agreed = 0- missing_rows: list[str] = []- mismatches: list[Mismatch] = []- for event_id, event in on_the_bus.items():- row = recorded.get(event_id)- if row is None:- missing_rows.append(event_id)- continue- found = _compare(event_id, event, row)- if found:- mismatches.extend(found)- else:- agreed += 1-- unsent_rows: list[str] = []- unverifiable: dict[str, int] = {}- for event_id, row in recorded.items():- if event_id in on_the_bus:- continue- if _comparable(row):- unsent_rows.append(event_id)- else:- unverifiable[row.event_type] = unverifiable.get(row.event_type, 0) + 1-- return RoomReconciliation(- room_id=room.id,- matrix_room_id=room.matrix_room_id,- room_name=room.name,- since=window_start,- compared=len(on_the_bus),- agreed=agreed,- missing_rows=missing_rows,- unsent_rows=unsent_rows,- mismatches=mismatches,- ignored_by_type=ignored,- unverifiable_by_type=unverifiable,- anchored=anchored,- )---def _comparable(row: Message) -> bool:- """Whether the bus walk would have collected this row's event.-- A row written before its type was denied has a bus event the walk now- discards; measuring it against the collected set says "recorded, never- sent" every single run.- """- return should_record(row.event_type)---def _joins_from_ms(oldest_join: Message | None) -> int | None:- """The point from which a bus arrival is expected to have a row.-- None means the log holds no arrival for this room, so every join on the- bus predates the path that records them and none can be compared.-- The bound is the oldest recorded arrival's `sent_at`, which is fractionally- later than its own bus event — so that one event is admitted by event id- instead, the same way the window's anchor is.- """- if oldest_join is None:- return None- return _to_ms(cast("datetime", oldest_join.sent_at))---def _to_ms(moment: datetime) -> int:- return int(moment.timestamp() * 1000)---async def _oldest_recorded_join(session: AsyncSession, room_id: str) -> Message | None:- """The first arrival the log holds for this room, which bounds join checking."""- result = await session.execute(- select(Message)- .where(Message.room_id == room_id, Message.event_type == MEMBERSHIP_EVENT_TYPE)- .order_by(Message.seq)- .limit(1)- )- return result.scalars().first()---async def _oldest_recorded(session: AsyncSession, room_id: str) -> Message | None:- """The row recording began with, which anchors the window.-- Ordered by `seq` rather than `sent_at`: within a room `seq` is a total- order with no ties, so "the first row" is a fact rather than whichever of- two equal timestamps the planner returned first.- """- result = await session.execute(- select(Message).where(Message.room_id == room_id).order_by(Message.seq).limit(1)- )- return result.scalars().first()---async def _recorded_since(- session: AsyncSession, room_id: str, since: datetime-) -> dict[str, Message]:- result = await session.execute(- select(Message)- .where(Message.room_id == room_id, Message.sent_at >= since)- .order_by(Message.seq)- )- return {row.transport_event_id: row for row in result.scalars().all()}---async def _read_back_to(- transport: MessageTransport,- matrix_room_id: str,- *,- anchor_event_id: str | None,- floor_ms: int,- window_start_ms: int,- joins_from_ms: int | None,- join_anchor_id: str | None,-) -> tuple[dict[str, InboundEvent], dict[str, int], bool]:- """Page backwards until the window is covered.-- Stops on `anchor_event_id` — the oldest recorded row's own event — which- puts the boundary on an identity rather than on two clocks that disagree- by however long a write takes. `floor_ms` only ends a walk that never- finds it, and the third return value says which of the two happened.-- Unbounded within those limits on purpose, unlike the read path: a- reconciliation that gave up after a page budget would report the events it- never looked at as agreeing, which is the one answer it must never give.- """- events: dict[str, InboundEvent] = {}- ignored: dict[str, int] = {}- start: str | None = None- anchored = False- done = False- while not done:- page = await transport.read_history(- matrix_room_id, start=start, limit=PAGE_SIZE- )- for raw in page.events:- if not isinstance(raw, InboundEvent):- ignored["unrecognised"] = ignored.get("unrecognised", 0) + 1- continue- # History comes back newest first, so reaching either limit is the- # end of the walk rather than an event to skip past.- if raw.timestamp < floor_ms:- done = True- break- is_anchor = raw.event_id == anchor_event_id- # The anchor is the one event admitted from before the window: its- # row's `sent_at` is by construction later than it.- if raw.timestamp >= window_start_ms or is_anchor:- _classify(raw, events, ignored, joins_from_ms, join_anchor_id)- elif anchor_event_id is None:- done = True- break- if is_anchor:- anchored = True- done = True- break- if done:- break- end = page.next_token- if not end or end == start:- break- start = end- return events, ignored, anchored or anchor_event_id is None---def _classify(- raw: InboundEvent,- events: dict[str, InboundEvent],- ignored: dict[str, int],- joins_from_ms: int | None,- join_anchor_id: str | None,-) -> None:- """Sort one bus event into the comparison set or the ignored tally."""- if isinstance(raw, InboundCustomEvent) and not should_record(raw.event_type):- ignored[raw.event_type] = ignored.get(raw.event_type, 0) + 1- elif isinstance(raw, InboundMessage | InboundCustomEvent):- events[raw.event_id] = raw- elif isinstance(raw, InboundMembership):- _classify_membership(raw, events, ignored, joins_from_ms, join_anchor_id)- else:- name = type(raw).__name__- ignored[name] = ignored.get(name, 0) + 1---def _classify_membership(- raw: InboundMembership,- events: dict[str, InboundEvent],- ignored: dict[str, int],- joins_from_ms: int | None,- join_anchor_id: str | None,-) -> None:- """An arrival is comparable; anything else membership does is not.-- Only a transition into membership is recorded, so an invite, a leave and a- profile change re-firing as a join all belong in the tally. So does a join- from before the log held any arrival at all.- """- arrived = raw.membership == "join" and raw.prev_membership != "join"- in_log_window = joins_from_ms is not None and (- raw.timestamp >= joins_from_ms or raw.event_id == join_anchor_id- )- if arrived and in_log_window:- events[raw.event_id] = raw- return- label = (- f"{MEMBERSHIP_EVENT_TYPE} (before arrivals were recorded)"- if arrived- else MEMBERSHIP_EVENT_TYPE- )- ignored[label] = ignored.get(label, 0) + 1---def _compare(event_id: str, event: InboundEvent, row: Message) -> list[Mismatch]:- """Compare the fields a reader will depend on once the read path moves.-- Not the whole content dict: the bus is free to hand back an event- annotated with fields it added itself, and reporting those as drift would- train everyone to ignore the report.- """- if isinstance(event, InboundMembership):- # The row is about the member who arrived, which is `state_key`; the- # sender of a membership event can be whoever admitted them.- return _mismatches(- event_id,- [- ("member", event.state_key, row.sender_matrix_id),- ("event_type", MEMBERSHIP_EVENT_TYPE, row.event_type),- ],- )-- expected: list[tuple[str, object, object]] = [- ("sender", event.sender, row.sender_matrix_id),- ]- if isinstance(event, InboundCustomEvent):- expected.append(("event_type", event.event_type, row.event_type))- expected.append(("thread_root", event.thread_root_id, row.thread_root_event_id))- elif isinstance(event, InboundMessage):- expected.append(("event_type", "m.room.message", row.event_type))- expected.append(("body", event.body, row.body))- expected.append(("msgtype", event.msgtype, row.msgtype))- expected.append(("thread_root", event.thread_root_id, row.thread_root_event_id))- if isinstance(event, InboundMedia):- expected.append(("uri", event.uri, row.content.get("url")))-- return _mismatches(event_id, expected)---def _mismatches(- event_id: str, expected: list[tuple[str, object, object]]-) -> list[Mismatch]:- return [- Mismatch(- event_id=event_id,- field_name=name,- on_the_bus=on_the_bus,- recorded=recorded,- )- for name, on_the_bus, recorded in expected- if on_the_bus != recorded- ]diff --git a/core/switch_core/messages/recorder.py b/core/switch_core/messages/recorder.pydeleted file mode 100644index 892d89ce1..000000000--- a/core/switch_core/messages/recorder.py+++ /dev/null@@ -1,214 +0,0 @@-"""Records into Postgres what was sent over the transport.--The transport remains the source of truth for history; these rows are a-parallel record that a later change reads from instead. Recording happens-after the send has been accepted, so the database never delays or blocks-delivery.--That ordering has a consequence worth stating rather than hiding: a failure-here leaves a row missing for a message that was really delivered. The-alternative — failing the send because the database is unhappy — would make-messaging less reliable than it is today, which is not a trade this step is-willing to make. Failures are logged at error, never swallowed silently, and-reconciling the two records is part of moving the read path over.--What gets a row is decided in `recorded_types`: the conversation, not every-event that crosses the bus.--Arrivals are the exception to "record what you send". A join is not a send, but-it is part of the timeline a reader expects — history shows who appeared and-when. It is recorded by the arriving client about itself, which keeps the-one-row-per-event property the send path gets for free: exactly one Switch-client owns each participant, so exactly one of them records the arrival.-"""--from __future__ import annotations--import logging-from typing import TYPE_CHECKING--from switch_core.db.models import Message-from switch_core.db.stores.message_store import MessageStore-from switch_core.db.stores.room_store import RoomStore-from switch_core.messages.recorded_types import MEMBERSHIP_EVENT_TYPE, should_record-from switch_core.messages.row import attachments_in, text_field, thread_root_of--if TYPE_CHECKING:- from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker-- from switch_core.transport import InboundMembership, SendResult--logger = logging.getLogger(__name__)---class MessageRecorder:- def __init__(- self,- *,- session_factory: async_sessionmaker[AsyncSession],- room_store: RoomStore,- message_store: MessageStore,- ) -> None:- self._session_factory = session_factory- self._room_store = room_store- self._message_store = message_store- # A room's transport id never changes, so this only ever grows and- # never goes stale.- self._room_ids: dict[str, str] = {}-- async def record(- self,- *,- transport_room_id: str,- result: SendResult,- sender_matrix_id: str,- sender_client_id: str,- sender_name: str,- ) -> None:- if not should_record(result.event_type):- return- try:- await self._record(- transport_room_id=transport_room_id,- result=result,- sender_matrix_id=sender_matrix_id,- sender_client_id=sender_client_id,- sender_name=sender_name,- )- except Exception:- logger.error(- "Delivered %s to %s but failed to record it as %s",- result.event_type,- transport_room_id,- result.event_id,- exc_info=True,- )-- async def record_join(- self,- *,- transport_room_id: str,- event: InboundMembership,- client_id: str,- member_name: str,- ) -> None:- """Record an arrival, called by the client that arrived.-- `member_name` is the arriving client's own name rather than the- display name on the membership event: the profile behind that name is- set from whatever the source platform calls the member, so taking it- would make the same participant read one way when they arrive and- another way when they speak.-- Like `record`, a failure here leaves a gap rather than disturbing- anything: nothing about a join depends on it having been written.- """- try:- await self._record_join(- transport_room_id=transport_room_id,- event=event,- client_id=client_id,- member_name=member_name,- )- except Exception:- logger.error(- "Joined %s but failed to record the arrival as %s",- transport_room_id,- event.event_id,- exc_info=True,- )-- async def _record_join(- self,- *,- transport_room_id: str,- event: InboundMembership,- client_id: str,- member_name: str,- ) -> None:- async with self._session_factory() as session:- room_id = await self._resolve_room(session, transport_room_id)- if room_id is None:- logger.warning(- "Not recording %s: %s is not a Switch room",- event.event_id,- transport_room_id,- )- return-- # A join can be observed twice — a redelivered sync, a restart that- # re-reads the same member event — and the second one is not a- # second arrival. The unique constraint on `transport_event_id` is- # the backstop; this keeps the ordinary case off the error log.- existing = await self._message_store.get_by_transport_event_id(- session, event.event_id- )- if existing is not None:- return-- # No body: what an arrival reads as is the reader's to phrase, and- # a rendered sentence in the log would freeze today's wording into- # every row ever written.- message = Message(- room_id=room_id,- transport_event_id=event.event_id,- sender_matrix_id=event.state_key,- sender_client_id=client_id,- sender_name=member_name,- event_type=MEMBERSHIP_EVENT_TYPE,- msgtype=None,- body=None,- formatted_body=None,- thread_root_event_id=None,- content=dict(event.content),- )- await self._message_store.create(session, message, [])- await session.commit()-- async def _record(- self,- *,- transport_room_id: str,- result: SendResult,- sender_matrix_id: str,- sender_client_id: str,- sender_name: str,- ) -> None:- async with self._session_factory() as session:- room_id = await self._resolve_room(session, transport_room_id)- if room_id is None:- logger.warning(- "Not recording %s: %s is not a Switch room",- result.event_id,- transport_room_id,- )- return-- content = result.content- message = Message(- room_id=room_id,- transport_event_id=result.event_id,- sender_matrix_id=sender_matrix_id,- sender_client_id=sender_client_id,- sender_name=sender_name,- event_type=result.event_type,- msgtype=text_field(content.get("msgtype")),- body=text_field(content.get("body")),- formatted_body=text_field(content.get("formatted_body")),- thread_root_event_id=thread_root_of(content),- content=content,- )- await self._message_store.create(session, message, attachments_in(content))- await session.commit()-- async def _resolve_room(- self, session: AsyncSession, transport_room_id: str- ) -> str | None:- cached = self._room_ids.get(transport_room_id)- if cached is not None:- return cached- room = await self._room_store.get_by_matrix_room_id(session, transport_room_id)- if room is None:- return None- self._room_ids[transport_room_id] = room.id- return room.iddiff --git a/core/switch_core/messages/recording.py b/core/switch_core/messages/recording.pydeleted file mode 100644index 508f86faa..000000000--- a/core/switch_core/messages/recording.py+++ /dev/null@@ -1,75 +0,0 @@-"""Whether a client has to write down what it sent.--Recording exists because the bus was the delivery and the table was a copy.-When the table *is* the delivery there is nothing to copy, and a client that-recorded anyway would write every message twice.--So which recorder a client gets is chosen with its transport, by the same-factory, and the pairing is the invariant: a transport that stores what it-carries takes `NoRecording`, and one that does not takes `MessageRecorder`.-Getting the pair wrong is visible immediately — duplicate rows one way, a-silent log the other — which is the argument for making it one decision in one-place rather than a flag each side reads.-"""--from __future__ import annotations--from typing import TYPE_CHECKING, Protocol, runtime_checkable--if TYPE_CHECKING:- from switch_core.transport import InboundMembership, SendResult---@runtime_checkable-class MessageRecording(Protocol):- """Writes the message log, or knows that something else already did."""-- async def record(- self,- *,- transport_room_id: str,- result: SendResult,- sender_matrix_id: str,- sender_client_id: str,- sender_name: str,- ) -> None: ...-- async def record_join(- self,- *,- transport_room_id: str,- event: InboundMembership,- client_id: str,- member_name: str,- ) -> None: ...---class NoRecording:- """For a transport whose own write is the record.-- Not a disabled feature: the rows still exist, and are still exactly one per- event. They are written by the transport, in the same transaction that- accepted the send, which is stronger than what this replaces — the- recorder ran after delivery and could leave a gap.- """-- async def record(- self,- *,- transport_room_id: str,- result: SendResult,- sender_matrix_id: str,- sender_client_id: str,- sender_name: str,- ) -> None:- return None-- async def record_join(- self,- *,- transport_room_id: str,- event: InboundMembership,- client_id: str,- member_name: str,- ) -> None:- return Nonediff --git a/core/switch_core/migrations/versions/c8e4a10b7f36_drop_matrix_columns.py b/core/switch_core/migrations/versions/c8e4a10b7f36_drop_matrix_columns.pynew file mode 100644index 000000000..bd3c5f343--- /dev/null+++ b/core/switch_core/migrations/versions/c8e4a10b7f36_drop_matrix_columns.py@@ -0,0 +1,77 @@+"""drop the homeserver's columns and rename the ones it named++Revision ID: c8e4a10b7f36+Revises: b6d1f04ae932++Four columns on `clients` existed to hold a homeserver session. Nothing has+written them since the transport moved to Postgres: there is no login, so no+access token and no device id; delivery starts at each room's head, so no sync+cursor; and the password was a random value generated to register an account+with a server that is gone. A client is a row, and a row does not authenticate.++`rooms.history_backfilled_at` goes with them. It recorded how far the backfill+had walked, and the backfill is gone: it read the homeserver, which is what+this release removes.++The two renames carry no data change. Both columns already hold Switch-minted+ids — `sw_…`, never `$event` — so the names were the last thing still saying+Matrix. `sender_matrix_id` is also not an id in the foreign-key sense: the real+reference is `sender_client_id` beside it, and this is the denormalised handle+the sender showed under.+"""++from collections.abc import Sequence++import sqlalchemy as sa+from alembic import op++revision: str = "c8e4a10b7f36"+down_revision: str | Sequence[str] | None = "b6d1f04ae932"+branch_labels: str | Sequence[str] | None = None+depends_on: str | Sequence[str] | None = None+++def upgrade() -> None:+ op.drop_column("clients", "password")+ op.drop_column("clients", "device_id")+ op.drop_column("clients", "access_token")+ op.drop_column("clients", "next_batch_token")+ op.drop_column("rooms", "history_backfilled_at")++ op.alter_column(+ "messages", "sender_matrix_id", new_column_name="sender_id", nullable=False+ )+ op.alter_column(+ "bridge_message_map",+ "matrix_event_id",+ new_column_name="transport_event_id",+ nullable=False,+ )+++def downgrade() -> None:+ op.alter_column(+ "bridge_message_map",+ "transport_event_id",+ new_column_name="matrix_event_id",+ nullable=False,+ )+ op.alter_column(+ "messages", "sender_id", new_column_name="sender_matrix_id", nullable=False+ )++ # The dropped values are not recoverable: they were a homeserver's, and+ # there is no homeserver to ask. A downgrade restores the shape so an older+ # revision can run, not the credentials, which that revision would have to+ # re-register anyway.+ op.add_column("clients", sa.Column("next_batch_token", sa.Text(), nullable=True))+ op.add_column("clients", sa.Column("access_token", sa.Text(), nullable=True))+ op.add_column("clients", sa.Column("device_id", sa.Text(), nullable=True))+ op.add_column(+ "clients",+ sa.Column("password", sa.Text(), nullable=False, server_default=""),+ )+ op.add_column(+ "rooms",+ sa.Column("history_backfilled_at", sa.DateTime(timezone=True), nullable=True),+ )diff --git a/core/switch_core/provisioning/port.py b/core/switch_core/provisioning/port.pyindex 75d4162a6..781ee6be2 100644--- a/core/switch_core/provisioning/port.py+++ b/core/switch_core/provisioning/port.py@@ -1,24 +1,14 @@"""The provisioning port: accounts, rooms and membership.`MessageTransport` covers what a participant does once it exists. This covers-bringing participants and rooms into existence in the first place — the second-and last thing switch-core asks a homeserver for. The two are separate ports-because they are used by different callers at different times: the transport by-every client for the life of the process, this once per account, room or-membership change, by room provisioning and the client/bridge lifecycles.--The contract is derived from the call sites, not from the endpoints behind it:-create an account, create a room, add and remove a member, discard a room, and-check a password. `verify_login` is the odd one — it exists so a cutover can-tell an empty target server from one that already has accounts under the same-names — and it is the operation most likely to have no counterpart in a future-implementation.+bringing rooms and memberships into existence in the first place. The two are+separate ports because they are used by different callers at different times:+the transport by every client for the life of the process, this once per room+or membership change, by room provisioning and the client/bridge lifecycles.Membership is expressed as invite/kick because that is what the operations-mean, not because of how a homeserver spells them. An implementation that can-simply write a row is free to do so; `invite_to_room` promises the user ends up-in the room, and callers already rely on that rather than on an invitation-being separately accepted.+mean. `invite_to_room` promises the user ends up in the room, and callers rely+on that rather than on an invitation being separately accepted.Both operations are idempotent by contract: inviting a member who is alreadyin, or kicking one who is already out, is success and not an error. Every other@@ -32,21 +22,7 @@ from typing import Protocol, runtime_checkable@runtime_checkableclass Provisioning(Protocol):- """Creates the accounts and rooms the transport then talks over."""-- async def register_user(- self,- user_id: str,- password: str,- display_name: str | None = None,- is_admin: bool = False,- ) -> None:- """Create an account, or do nothing if it already exists."""- ...-- async def verify_login(self, user_id: str, password: str) -> bool:- """Whether `password` authenticates `user_id`."""- ...+ """Creates the rooms and memberships the transport then talks over."""async def create_room(self, name: str, topic: str) -> str:"""Create a room and return its transport-side id."""diff --git a/core/switch_core/provisioning/postgres.py b/core/switch_core/provisioning/postgres.pyindex 880b98098..f5ac7c271 100644--- a/core/switch_core/provisioning/postgres.py+++ b/core/switch_core/provisioning/postgres.py@@ -1,20 +1,4 @@-"""Provisioning without a homeserver: every operation is a row.--What the Matrix implementation did over HTTP, this does against the same-database Switch already keeps the answer in. Two of the six turn out to be-almost nothing, and that is the finding rather than a shortcut:--- **An account is the `clients` row.** `register_user` created a homeserver- account so that a row Switch was about to write would have something to- authenticate as. There is no second system to register with now, so there is- nothing to do — the caller writes the row immediately afterwards, and that- row is the account.-- **A password is a Switch password.** `verify_login` asked the homeserver- whether a password worked; it asks the `clients` table instead. Its one- caller is a cutover check for a target server that already has accounts,- which no longer has a homeserver to be true of.--The other four are writes:+"""Provisioning: every operation is a row.- **A room** gets an id minted here and stored by the caller, exactly as thehomeserver's `room_id` was.@@ -84,20 +68,6 @@ class PostgresProvisioning:self._message_store = message_storeself._invites = invites- async def register_user(- self,- user_id: str,- password: str,- display_name: str | None = None,- is_admin: bool = False,- ) -> None:- """Nothing to register: the caller's `clients` row is the account."""-- async def verify_login(self, user_id: str, password: str) -> bool:- async with self._session_factory() as session:- client = await self._client_store.get_by_matrix_user_id(session, user_id)- return client is not None and client.password == password-async def create_room(self, name: str, topic: str) -> str:"""Mint the id the caller will store on its own room row.@@ -138,7 +108,7 @@ class PostgresProvisioning:arrival = Message(room_id=switch_room_id,transport_event_id=f"sw_{uuid.uuid4().hex}",- sender_matrix_id=user_id,+ sender_id=user_id,sender_client_id=client_id,sender_name=display_name,event_type=MEMBERSHIP_EVENT_TYPE,diff --git a/core/switch_core/transport/__init__.py b/core/switch_core/transport/__init__.pyindex 8e1a370af..0f846c78d 100644--- a/core/switch_core/transport/__init__.py+++ b/core/switch_core/transport/__init__.py@@ -8,8 +8,6 @@ else talks to the port and the neutral types in `types`.from switch_core.transport.port import (Handler,MessageTransport,- SyncErrorHandler,- SyncHandler,TransportHandlers,)from switch_core.transport.types import (@@ -44,8 +42,6 @@ __all__ = ["RoomRef","SeekDirection","SendResult",- "SyncErrorHandler",- "SyncHandler","TransportError","TransportHandlers","UploadResult",diff --git a/core/switch_core/transport/matrix.py b/core/switch_core/transport/matrix.pydeleted file mode 100644index 082672222..000000000--- a/core/switch_core/transport/matrix.py+++ /dev/null@@ -1,459 +0,0 @@-"""Matrix implementation of `MessageTransport`.--This is the only module in Switch that may import matrix-nio. Everything-Matrix-shaped lives here: content dicts, `m.relates_to` threading, mxc URIs,-pagination tokens and the sync loop.-"""--from __future__ import annotations--import io-import logging-from dataclasses import dataclass-from typing import Any--from nio import (- AsyncClient,- DownloadError,- InviteMemberEvent,- JoinedRoomsError,- LoginError,- ProfileSetDisplayNameError,- ReactionEvent,- RoomGetEventError,- RoomMemberEvent,- RoomMessageMedia,- RoomMessagesError,- RoomMessageText,- RoomSendError,- RoomTypingError,- SyncError,- SyncResponse,- UnknownEvent,- UploadError,-)--from switch_core.attachments import ATTACHMENT_GROUP_KEY-from switch_core.transport.content import media_content, message_content-from switch_core.transport.port import TransportHandlers-from switch_core.transport.types import (- DownloadResult,- HistoryPage,- InboundCustomEvent,- InboundEvent,- InboundMedia,- InboundMembership,- InboundMessage,- MessageFormat,- NotConnectedError,- RoomRef,- SendResult,- TransportError,- UploadResult,-)--logger = logging.getLogger(__name__)--SYNC_TIMEOUT_MS = 30000---@dataclass(frozen=True)-class _RoomIdOnly:- """Stands in for a room object when only the id is known.-- History and single-event fetches return events without the room they came- from, but the converters only ever read the id.- """-- room_id: str---def _content(event: Any) -> dict[str, object]:- source = getattr(event, "source", None) or {}- content = source.get("content") or {}- return dict(content)---def _thread_root_of(content: dict[str, object], event_id: str) -> str | None:- """The thread this event belongs to, or None when it is top level.-- Matrix threads are flat: an event either relates to a root or is one.- """- relates = content.get("m.relates_to") or {}- if isinstance(relates, dict) and relates.get("rel_type") == "m.thread":- root = relates.get("event_id")- return str(root) if root else None- return None---def to_room_ref(room: Any) -> RoomRef:- return RoomRef(room_id=room.room_id)---def to_inbound_message(room: Any, event: Any) -> InboundMessage:- content = _content(event)- return InboundMessage(- room_id=room.room_id,- event_id=event.event_id,- sender=event.sender,- timestamp=event.server_timestamp,- content=content,- body=getattr(event, "body", "") or "",- sender_name=content.get("sender_name"), # type: ignore[arg-type]- formatted_body=getattr(event, "formatted_body", None),- msgtype=str(content.get("msgtype") or "m.text"),- thread_root_id=_thread_root_of(content, event.event_id),- )---def to_inbound_media(room: Any, event: Any) -> InboundMedia:- content = _content(event)- info = content.get("info") or {}- if not isinstance(info, dict):- info = {}- return InboundMedia(- room_id=room.room_id,- event_id=event.event_id,- sender=event.sender,- timestamp=event.server_timestamp,- content=content,- body=getattr(event, "body", "") or "",- sender_name=content.get("sender_name"), # type: ignore[arg-type]- uri=str(getattr(event, "url", "") or ""),- filename=content.get("filename") or getattr(event, "body", None), # type: ignore[arg-type]- mimetype=info.get("mimetype"),- size=info.get("size"),- msgtype=str(content.get("msgtype") or "m.file"),- thread_root_id=_thread_root_of(content, event.event_id),- group=content.get(ATTACHMENT_GROUP_KEY), # type: ignore[arg-type]- )---def to_inbound_membership(room: Any, event: Any) -> InboundMembership:- content = _content(event)- return InboundMembership(- room_id=room.room_id,- event_id=getattr(event, "event_id", ""),- sender=getattr(event, "sender", ""),- timestamp=getattr(event, "server_timestamp", 0),- content=content,- state_key=getattr(event, "state_key", ""),- membership=getattr(event, "membership", ""),- prev_membership=getattr(event, "prev_membership", None),- display_name=content.get("displayname"), # type: ignore[arg-type]- )---def to_inbound_custom(room: Any, event: Any) -> InboundCustomEvent:- content = _content(event)- return InboundCustomEvent(- room_id=room.room_id,- event_id=event.event_id,- sender=event.sender,- timestamp=getattr(event, "server_timestamp", 0),- content=content,- event_type=event.type,- thread_root_id=_thread_root_of(content, event.event_id),- )---def to_inbound(room: Any, event: Any) -> InboundEvent:- """Convert a timeline event to the DTO matching its kind."""- if isinstance(event, RoomMessageMedia):- return to_inbound_media(room, event)- if isinstance(event, RoomMessageText):- return to_inbound_message(room, event)- if isinstance(event, RoomMemberEvent):- return to_inbound_membership(room, event)- if isinstance(event, UnknownEvent):- return to_inbound_custom(room, event)- return to_inbound_event(room, event)---def to_inbound_event(room: Any, event: Any) -> InboundEvent:- return InboundEvent(- room_id=room.room_id,- event_id=getattr(event, "event_id", ""),- sender=getattr(event, "sender", ""),- timestamp=getattr(event, "server_timestamp", 0),- content=_content(event),- )---def _carries_durable_events(response: SyncResponse) -> bool:- """Whether a sync response holds anything a restart would need to replay.-- Timeline events, room state and invites are dispatched to handlers, so the- cursor must move past them durably. Presence, typing, receipts and account- data are not — they only advance the token.- """- if response.rooms.invite or response.to_device_events:- return True- return any(- room.timeline.events or room.state- for room in (*response.rooms.join.values(), *response.rooms.leave.values())- )---class MatrixTransport:- """Carries Switch messages over a Matrix homeserver."""-- def __init__(- self,- *,- server_url: str,- user_id: str,- password: str,- device_id: str | None = None,- access_token: str | None = None,- client: AsyncClient | None = None,- ) -> None:- self.server_url = server_url- self.user_id = user_id- self.password = password- self.device_id = device_id- self.access_token = access_token- self._client: AsyncClient | None = client- self._handlers = TransportHandlers()-- # ── Session ───────────────────────────────────────────────────────────────-- @property- def raw_client(self) -> AsyncClient:- """The underlying nio client.-- An escape hatch for call sites not yet expressed on the port. New code- must use the port; this exists so the migration can proceed in steps.- """- if self._client is None:- raise NotConnectedError(- f"Transport for {self.user_id} is not connected — call connect() first"- )- return self._client-- @property- def session_state(self) -> dict[str, str | None]:- return {"access_token": self.access_token, "device_id": self.device_id}-- async def connect(self) -> None:- if self._client is None:- self._client = AsyncClient(self.server_url, self.user_id)-- if self.access_token and self.device_id:- self._client.access_token = self.access_token- self._client.device_id = self.device_id- logger.info("Transport %s restored session from stored token", self.user_id)- return-- if not self.password:- raise TransportError(f"No credentials available for {self.user_id}")-- resp = await self._client.login(self.password)- if isinstance(resp, LoginError):- raise TransportError(f"Login failed for {self.user_id}: {resp.message}")- self.device_id = resp.device_id- self.access_token = self._client.access_token- logger.info("Transport %s authenticated via password", self.user_id)-- async def close(self) -> None:- if self._client is not None:- await self._client.close()-- async def relogin(self) -> None:- """Re-authenticate an existing connection after a session failure."""- if not self.password:- return- resp = await self.raw_client.login(self.password)- if isinstance(resp, LoginError):- raise TransportError(f"Re-login failed for {self.user_id}: {resp.message}")- self.device_id = resp.device_id- self.access_token = self.raw_client.access_token-- def register_handlers(self, handlers: TransportHandlers) -> None:- self._handlers = handlers- client = self.raw_client-- def bind(handler: Any, convert: Any, kind: Any) -> None:- if handler is None:- return-- async def _cb(room: Any, event: Any) -> None:- await handler(to_room_ref(room), convert(room, event))-- client.add_event_callback(_cb, kind)-- bind(handlers.on_message, to_inbound_message, RoomMessageText)- # RoomMessageMedia is the base of RoomMessageImage / RoomMessageFile, so- # one callback covers every media msgtype.- bind(handlers.on_media, to_inbound_media, RoomMessageMedia)- bind(handlers.on_reaction, to_inbound_event, ReactionEvent)- bind(handlers.on_member_event, to_inbound_membership, RoomMemberEvent)- bind(handlers.on_custom_event, to_inbound_custom, UnknownEvent)- bind(handlers.on_invite, to_inbound_membership, InviteMemberEvent)-- if handlers.on_sync is not None:- on_sync = handlers.on_sync-- async def _sync_cb(response: SyncResponse) -> None:- await on_sync(response.next_batch, _carries_durable_events(response))-- client.add_response_callback(_sync_cb, SyncResponse) # type: ignore[arg-type]-- if handlers.on_sync_error is not None:- on_sync_error = handlers.on_sync_error-- async def _sync_error_cb(response: SyncError) -> None:- await on_sync_error(response.message)-- client.add_response_callback(_sync_error_cb, SyncError) # type: ignore[arg-type]-- async def receive_forever(self, *, since: str | None) -> None:- await self.raw_client.sync_forever(timeout=SYNC_TIMEOUT_MS, since=since)-- # ── Outbound ──────────────────────────────────────────────────────────────-- async def send_message(- self,- room_id: str,- body: str,- *,- sender_name: str,- format: MessageFormat = "text",- mentions: list[str] | None = None,- thread_root_id: str | None = None,- extra_content: dict[str, object] | None = None,- ) -> SendResult:- content = message_content(- body,- sender_name=sender_name,- format=format,- mentions=mentions,- thread_root_id=thread_root_id,- extra_content=extra_content,- )- return await self._room_send(room_id, "m.room.message", content)-- async def send_event(- self,- room_id: str,- event_type: str,- content: dict[str, object],- ) -> SendResult:- return await self._room_send(room_id, event_type, content)-- async def send_media(- self,- room_id: str,- uri: str,- filename: str,- mimetype: str,- size: int,- *,- sender_name: str,- msgtype: str,- caption: str | None = None,- thread_root_id: str | None = None,- group: dict[str, object] | None = None,- ) -> SendResult:- content = media_content(- uri,- filename,- mimetype,- size,- sender_name=sender_name,- msgtype=msgtype,- caption=caption,- thread_root_id=thread_root_id,- group=group,- )- return await self._room_send(room_id, "m.room.message", content)-- async def _room_send(- self, room_id: str, event_type: str, content: dict[str, object]- ) -> SendResult:- resp = await self.raw_client.room_send(room_id, event_type, content)- if isinstance(resp, RoomSendError):- raise TransportError(- f"Failed to send {event_type} to {room_id}: {resp.message}"- )- return SendResult(- event_id=resp.event_id, event_type=event_type, content=content- )-- async def upload_media(- self, data: bytes, content_type: str, filename: str- ) -> UploadResult:- resp, _ = await self.raw_client.upload(- data_provider=io.BytesIO(data),- content_type=content_type,- filename=filename,- filesize=len(data),- )- if isinstance(resp, UploadError):- raise TransportError(- f"Failed to upload media '{filename}' to Matrix: {resp.message}"- )- return UploadResult(uri=resp.content_uri)-- async def download_media(self, uri: str) -> DownloadResult:- resp = await self.raw_client.download(mxc=uri)- if isinstance(resp, DownloadError):- raise TransportError(f"Failed to download {uri}: {resp.message}")- return DownloadResult(- body=resp.body,- content_type=getattr(resp, "content_type", None),- filename=getattr(resp, "filename", None),- )-- async def set_typing(self, room_id: str, is_typing: bool) -> None:- logger.debug(- "Setting typing %s in room %s for %s", is_typing, room_id, self.user_id- )- resp = await self.raw_client.room_typing(room_id, is_typing)- if isinstance(resp, RoomTypingError):- logger.error("Failed to set typing in %s: %s", room_id, resp.message)-- # ── History ───────────────────────────────────────────────────────────────-- async def get_event(self, room_id: str, event_id: str) -> InboundEvent | None:- resp = await self.raw_client.room_get_event(room_id, event_id)- if isinstance(resp, RoomGetEventError):- return None- return to_inbound(_RoomIdOnly(room_id), resp.event)-- async def read_history(- self, room_id: str, *, start: str | None, limit: int- ) -> HistoryPage:- resp = await self.raw_client.room_messages(room_id, start=start, limit=limit)- if isinstance(resp, RoomMessagesError):- raise TransportError(f"Failed to read history in {room_id}: {resp.message}")- room = _RoomIdOnly(room_id)- return HistoryPage(- events=[to_inbound(room, event) for event in resp.chunk],- next_token=resp.end,- )-- # ── Rooms ─────────────────────────────────────────────────────────────────-- async def join_room(self, room_id: str) -> bool:- resp = await self.raw_client.join(room_id)- if hasattr(resp, "room_id"):- return True- logger.error("Transport %s failed to join %s: %s", self.user_id, room_id, resp)- return False-- async def joined_rooms(self) -> list[str]:- resp = await self.raw_client.joined_rooms()- if isinstance(resp, JoinedRoomsError):- logger.error(- "Failed to list joined rooms for %s: %s", self.user_id, resp.message- )- return []- return list(resp.rooms)-- async def set_display_name(self, display_name: str) -> None:- resp = await self.raw_client.set_displayname(display_name)- if isinstance(resp, ProfileSetDisplayNameError):- raise TransportError(- f"Could not set the display name of {self.user_id}: {resp}"- )diff --git a/core/switch_core/transport/port.py b/core/switch_core/transport/port.pyindex cd4e866be..70c5e5de9 100644--- a/core/switch_core/transport/port.py+++ b/core/switch_core/transport/port.py@@ -25,12 +25,6 @@ from switch_core.transport.types import ()Handler = Callable[[Any, Any], Awaitable[None]]-# (cursor, durable). `durable` says whether the batch carried anything a-# restart would have to replay, so the client can decide how urgently to-# persist the cursor. The transport decides it: only it can see what the batch-# held, and saying so as a bool keeps its event classes out of the client.-SyncHandler = Callable[[str, bool], Awaitable[None]]-SyncErrorHandler = Callable[[str], Awaitable[None]]@dataclass@@ -47,8 +41,6 @@ class TransportHandlers:on_member_event: Handler | None = Noneon_custom_event: Handler | None = Noneon_invite: Handler | None = None- on_sync: SyncHandler | None = None- on_sync_error: SyncErrorHandler | None = None@runtime_checkable@@ -57,36 +49,20 @@ class MessageTransport(Protocol):# ── Session ───────────────────────────────────────────────────────────────- @property- def session_state(self) -> dict[str, str | None]:- """Opaque credentials to persist so a restart can resume.-- The shape is transport-defined; callers store and hand it back- unread.- """- ...-async def connect(self) -> None:- """Authenticate, restoring a stored session when one is available."""+ """Prepare the connection for use."""...async def close(self) -> None:"""Release the connection. Safe to call when never connected."""...- async def relogin(self) -> None:- """Re-authenticate after the session was rejected mid-stream."""- ...-def register_handlers(self, handlers: TransportHandlers) -> None:"""Bind inbound callbacks. Call before `receive_forever`."""...- async def receive_forever(self, *, since: str | None) -> None:- """Deliver inbound events to the registered handlers until closed.-- `since` resumes from a cursor previously reported to `on_sync`.- """+ async def receive_forever(self) -> None:+ """Deliver inbound events to the registered handlers until closed."""...# ── Outbound ──────────────────────────────────────────────────────────────diff --git a/core/switch_core/transport/postgres.py b/core/switch_core/transport/postgres.pyindex 10313f6c3..822d1ef02 100644--- a/core/switch_core/transport/postgres.py+++ b/core/switch_core/transport/postgres.py@@ -142,18 +142,6 @@ class PostgresTransport:# ── Session ───────────────────────────────────────────────────────────────- @property- def session_state(self) -> dict[str, str | None]:- """Nothing to resume.-- The Matrix transport persisted an access token and a device id because- logging in again cost a round trip and a new device. Here the client's- identity is a row it already owns, so there is no credential to carry- across a restart. The empty dict is the honest answer, and callers- store it unread.- """- return {}-async def connect(self) -> None:"""Nothing to authenticate: the caller already knows who it is."""@@ -162,23 +150,15 @@ class PostgresTransport:there is to release."""self._closed.set()- async def relogin(self) -> None:- """Never called: there is no session to be rejected."""-def register_handlers(self, handlers: TransportHandlers) -> None:self._handlers = handlers- async def receive_forever(self, *, since: str | None) -> None:+ async def receive_forever(self) -> None:"""Deliver every row written to this client's rooms from now on.- `since` is ignored, and that is the behaviour to keep rather than an- omission. A Matrix client resumed from its stored sync token and then- threw away everything older than the process (`ClientBase._should_ignore`),- so what was actually delivered on a restart was "whatever happened- while I was up". Starting each room at its current head says the same- thing without the cursor that lied about it. What an agent missed- while it was away is a delivery-cursor question, and delivery cursors- are a layer above this one.+ Each room starts at its current head, so what an agent missed while it+ was away is not delivered here. That is a delivery-cursor question,+ and delivery cursors are a layer above this one."""rooms = await self.joined_rooms()if not rooms:@@ -487,7 +467,7 @@ class PostgresTransport:message = Message(room_id=room_id,transport_event_id=result.event_id,- sender_matrix_id=self.user_id,+ sender_id=self.user_id,sender_client_id=self.client_id,sender_name=sender_name,event_type=event_type,@@ -605,7 +585,7 @@ class PostgresTransport:arrival = Message(room_id=switch_room_id,transport_event_id=new_event_id(),- sender_matrix_id=self.user_id,+ sender_id=self.user_id,sender_client_id=self.client_id,sender_name=self.display_name,event_type=MEMBERSHIP_EVENT_TYPE,@@ -666,7 +646,7 @@ def to_inbound(content = dict(row.content)room_id = transport_room_idevent_id = row.transport_event_id- sender = row.sender_matrix_id+ sender = row.sender_idtimestamp = _epoch_ms(row.sent_at)if row.event_type == MEMBERSHIP_EVENT_TYPE:@@ -676,7 +656,7 @@ def to_inbound(sender=sender,timestamp=timestamp,content=content,- state_key=row.sender_matrix_id,+ state_key=row.sender_id,membership=text_field(content.get("membership")) or "join",# Only an arrival is ever written, so there is no previous state# to read back — and a row that carries one is honoured ratherdiff --git a/core/tests/integration/conftest.py b/core/tests/integration/conftest.pyindex 9e2099852..4442fd186 100644--- a/core/tests/integration/conftest.py+++ b/core/tests/integration/conftest.py@@ -1,19 +1,19 @@-"""Integration-test infrastructure: real Postgres + Tuwunel (Matrix) via testcontainers.--Unlike the unit suite (which fakes the nio client), these fixtures boot a real-Matrix homeserver and wire up the subset of `switch_core.main:run()` the feature-under test needs, in-process. This lets a test drive the genuine path-RoomService → Matrix invite/join → AgentClient sync loop → EventBuffer.--The two containers mirror the `postgres` / `tuwunel` services in-`deploy/local/docker-compose.yml` (see constants below — keep in sync). They get-ephemeral host ports, so the suite coexists with a running `just up` dev stack.--Isolation model: the containers, the Postgres database, its schema, and the Matrix-admin are **session-scoped** (built once). Between tests the per-test `harness`-fixture resets Postgres rows with TRUNCATE and rebuilds the in-memory services, so-tests don't pay a per-test CREATE DATABASE. The Matrix homeserver is *not* reset —-its users/rooms persist across the session — so tests must use unique agent names.+"""Integration-test infrastructure: a real Postgres via testcontainers.++Unlike the unit suite (which fakes the transport), these fixtures boot a real+database and wire up the subset of `switch_core.main:run()` the feature under+test needs, in-process. Messaging, membership and provisioning all run against+that database, so a test drives the genuine path RoomService → provisioning →+AgentClient receive loop → EventBuffer.++The container mirrors the `postgres` service in+`deploy/local/docker-compose.yml` (see constants below — keep in sync). It gets+an ephemeral host port, so the suite coexists with a running `just up` dev stack.++Isolation model: the container, the Postgres database, its schema and the+(stateless) stores are **session-scoped** (built once). Between tests the+per-test `harness` fixture resets rows with TRUNCATE and rebuilds the in-memory+services, so tests don't pay a per-test CREATE DATABASE."""from __future__ import annotations@@ -23,9 +23,6 @@ import jsonimport osimport shutilimport subprocess-import time-import urllib.error-import urllib.requestimport uuidfrom collections.abc import AsyncIterator, Iteratorfrom dataclasses import dataclass@@ -35,7 +32,6 @@ import pytestimport pytest_asynciofrom sqlalchemy import textfrom sqlalchemy.ext.asyncio import AsyncEngine-from testcontainers.core.container import DockerContainerfrom testcontainers.postgres import PostgresContainer# Importing models registers every table on Base.metadata for create_all.@@ -79,35 +75,18 @@ from switch_core.db.stores.room_role_store import RoomRoleStorefrom switch_core.db.stores.room_store import RoomStorefrom switch_core.db.stores.task_store import TaskStorefrom switch_core.db.stores.user_store import UserStore-from switch_core.matrix_admin import (- MatrixAdmin,- ensure_admin_exists,- wait_for_homeserver,-)-from switch_core.messages import MessageRecorderfrom switch_core.messages.notify import MessageListener+from switch_core.provisioning import Provisioning+from switch_core.provisioning.postgres import PostgresProvisioningfrom switch_core.room_service import RoomServicefrom switch_core.transport.ephemeral import EphemeralBusfrom switch_core.transport.invites import InviteBus# ── Mirrors deploy/local/docker-compose.yml — keep in sync ──────────────────────POSTGRES_IMAGE = "postgres:16-alpine"-TUWUNEL_IMAGE = "jevolk/tuwunel:v1.7.1"-TUWUNEL_ENV = {- "TUWUNEL_SERVER_NAME": "localhost",- "TUWUNEL_DATABASE_PATH": "/var/lib/tuwunel",- "TUWUNEL_ADDRESS": "0.0.0.0",- "TUWUNEL_PORT": "8008",- "TUWUNEL_ALLOW_FEDERATION": "false",- "TUWUNEL_ALLOW_REGISTRATION": "false",- "TUWUNEL_MAX_REQUEST_SIZE": "20000000",-}# ── Throwaway test constants (not secrets — local ephemeral infra) ──────────────SERVER_NAME = "localhost"-SHARED_SECRET = "dev-secret"-ADMIN_USER = "admin"-ADMIN_PASSWORD = "admin"JWT_SECRET = "dev-jwt-secret-test"REGISTRATION_TOKEN = "dev-test-token"GATEWAY_ADMIN_EMAIL = "[email protected]"@@ -133,19 +112,17 @@ class _NoBridges:@dataclassclass StackInfo:pg: PostgresContainer- matrix_url: str@dataclassclass SessionEnv:- """Session-scoped pieces shared by every test: the engine/schema, the Matrix- admin, and the (stateless) stores. The per-test `harness` fixture builds the- in-memory services on top of these."""+ """Session-scoped pieces shared by every test: the engine/schema and the+ (stateless) stores. The per-test `harness` fixture builds the in-memory+ services on top of these."""config: SwitchConfigengine: AsyncEnginesession_factory: object- matrix_admin: MatrixAdminagent_store: AgentStoreagent_session_store: AgentSessionStoreroom_store: RoomStore@@ -161,22 +138,8 @@ class SessionEnv:room_link_store: RoomLinkStoreroom_role_store: RoomRoleStoreuser_store: UserStore---def _wait_matrix_ready(matrix_url: str, timeout_s: float = 60.0) -> None:- """Block until the homeserver answers the client-versions probe."""- deadline = time.monotonic() + timeout_s- last_err: Exception | None = None- while time.monotonic() < deadline:- try:- with urllib.request.urlopen(- f"{matrix_url}/_matrix/client/versions", timeout=5- ):- return- except (urllib.error.URLError, ConnectionError, OSError) as err:- last_err = err- time.sleep(0.5)- raise RuntimeError(f"Tuwunel not ready at {matrix_url}: {last_err}")+ message_store: MessageStore+ media_store: MediaStoredef _ensure_docker_host() -> None:@@ -210,17 +173,8 @@ def _ensure_docker_host() -> None:@pytest.fixture(scope="session")def switch_stack() -> Iterator[StackInfo]:_ensure_docker_host()- tuwunel = DockerContainer(TUWUNEL_IMAGE).with_exposed_ports(8008)- for key, value in TUWUNEL_ENV.items():- tuwunel = tuwunel.with_env(key, value)- tuwunel = tuwunel.with_env("TUWUNEL_REGISTRATION_SHARED_SECRET", SHARED_SECRET)-- with PostgresContainer(POSTGRES_IMAGE) as pg, tuwunel:- host = tuwunel.get_container_host_ip()- port = tuwunel.get_exposed_port(8008)- matrix_url = f"http://{host}:{port}"- _wait_matrix_ready(matrix_url)- yield StackInfo(pg=pg, matrix_url=matrix_url)+ with PostgresContainer(POSTGRES_IMAGE) as pg:+ yield StackInfo(pg=pg)def _build_config(stack: StackInfo, db_name: str) -> SwitchConfig:@@ -231,11 +185,7 @@ def _build_config(stack: StackInfo, db_name: str) -> SwitchConfig:db_user=pg.username,db_password=pg.password,db_name=db_name,- matrix_server=stack.matrix_url,matrix_server_name=SERVER_NAME,- matrix_admin_user=ADMIN_USER,- matrix_admin_password=ADMIN_PASSWORD,- matrix_registration_shared_secret=SHARED_SECRET,agent_registration_token=REGISTRATION_TOKEN,jwt_secret_key=JWT_SECRET,gateway_admin_email=GATEWAY_ADMIN_EMAIL,@@ -258,8 +208,8 @@ class Harness:"""In-process switch-core wiring for integration tests.Holds the real services and exposes the few operations a test needs:- register agents, start their Matrix sync clients, and reach the RoomService- / EventBuffer.+ register agents, start their clients, and reach the RoomService /+ EventBuffer."""def __init__(@@ -299,9 +249,9 @@ class Harness:return resultasync def start_clients(self, timeout: float = 20.0) -> None:- """Start every registered agent's Matrix client and await readiness.+ """Start every registered agent's client and await readiness.- `start_all` launches each client's sync loop as a background task; the+ `start_all` launches each client's receive loop as a background task; theclient only loads its Agent (so `get_by_agent_id` resolves) once thattask has run. Poll until each is present and ready before returning."""@@ -375,25 +325,10 @@ async def session_env(switch_stack: StackInfo) -> AsyncIterator[SessionEnv]:async with engine.begin() as conn:await conn.run_sync(Base.metadata.create_all)- await wait_for_homeserver(config.matrix_server)- await ensure_admin_exists(- server_url=config.matrix_server,- username=config.matrix_admin_user,- password=config.matrix_admin_password,- shared_secret=config.matrix_registration_shared_secret,- )- matrix_admin = await MatrixAdmin.create(- server_url=config.matrix_server,- admin_user=config.matrix_admin_user,- admin_password=config.matrix_admin_password,- shared_secret=config.matrix_registration_shared_secret,- )-env = SessionEnv(config=config,engine=engine,session_factory=session_factory,- matrix_admin=matrix_admin,agent_store=AgentStore(),agent_session_store=AgentSessionStore(),room_store=RoomStore(),@@ -409,11 +344,12 @@ async def session_env(switch_stack: StackInfo) -> AsyncIterator[SessionEnv]:room_link_store=RoomLinkStore(),room_role_store=RoomRoleStore(),user_store=UserStore(),+ message_store=MessageStore(),+ media_store=MediaStore(),)try:yield envfinally:- await matrix_admin.close()await engine.dispose()admin_conn = await asyncpg.connect(admin_dsn)try:@@ -426,8 +362,6 @@ async def session_env(switch_stack: StackInfo) -> AsyncIterator[SessionEnv]:@pytest_asyncio.fixture(loop_scope="session")async def harness(session_env: SessionEnv) -> AsyncIterator[Harness]:- # Reset Postgres state from the previous test (Tuwunel is session-scoped and- # is not reset — tests use unique agent names to avoid Matrix collisions).await _truncate_all(session_env.engine)config = session_env.config@@ -447,6 +381,13 @@ async def harness(session_env: SessionEnv) -> AsyncIterator[Harness]:connections = ConnectionRegistry()collab_lifecycle = _NoBridges()+ # The transport's wake-up path: rows are announced over LISTEN/NOTIFY, so+ # nothing is delivered to a client until this connection is up.+ message_listener = MessageListener(lambda: create_unpooled_engine(config))+ await message_listener.start()+ invites = InviteBus()+ ephemeral = EphemeralBus()+resource_service = ResourceService(reference_store=session_env.reference_store,reference_type_store=session_env.reference_type_store,@@ -456,21 +397,24 @@ async def harness(session_env: SessionEnv) -> AsyncIterator[Harness]:session_factory=session_factory,)+ provisioning: Provisioning = PostgresProvisioning(+ session_factory=session_factory,+ room_store=session_env.room_store,+ client_store=session_env.client_store,+ message_store=session_env.message_store,+ invites=invites,+ )+client_factory = ClientFactory(client_store=session_env.client_store,session_factory=session_factory,config=config,- message_recorder=MessageRecorder(- session_factory=session_factory,- room_store=session_env.room_store,- message_store=MessageStore(),- ),room_store=session_env.room_store,- message_store=MessageStore(),- media_store=MediaStore(),- listener=MessageListener(lambda: create_unpooled_engine(config)),- invites=InviteBus(),- ephemeral=EphemeralBus(),+ message_store=session_env.message_store,+ media_store=session_env.media_store,+ listener=message_listener,+ invites=invites,+ ephemeral=ephemeral,)client_factory.register("agent",@@ -491,7 +435,7 @@ async def harness(session_env: SessionEnv) -> AsyncIterator[Harness]:client_factory.register("bridge", ClientBase)client_lifecycle = ClientLifecycleService(- matrix_admin=session_env.matrix_admin,+ matrix_admin=provisioning,client_store=session_env.client_store,client_factory=client_factory,session_factory=session_factory,@@ -499,7 +443,7 @@ async def harness(session_env: SessionEnv) -> AsyncIterator[Harness]:)room_service = RoomService(- matrix_admin=session_env.matrix_admin,+ matrix_admin=provisioning,room_store=session_env.room_store,agent_store=session_env.agent_store,client_lifecycle=client_lifecycle,@@ -543,6 +487,8 @@ async def harness(session_env: SessionEnv) -> AsyncIterator[Harness]:try:yield hfinally:- # Stop the agents' sync loops before the next test truncates, so none is- # mid-query against a table being cleared.+ # Stop the agents' receive loops before the next test truncates, so none+ # is mid-query against a table being cleared.await client_lifecycle.stop_all()+ await message_listener.stop()+ await provisioning.close()diff --git a/core/tests/integration/test_agent_greeting_e2e.py b/core/tests/integration/test_agent_greeting_e2e.pyindex 544b0b765..8e6d35a46 100644--- a/core/tests/integration/test_agent_greeting_e2e.py+++ b/core/tests/integration/test_agent_greeting_e2e.py@@ -1,9 +1,9 @@-"""End-to-end test for the agent self-join greeting over a real Matrix homeserver.+"""End-to-end test for the agent self-join greeting over the real transport.Drives the genuine path: RoomService creates a room and invites the agent →-the agent's AgentClient sync loop sees the invite → auto-joins → on_self_join-posts the greeting. The assertion reads the room timeline back off the-homeserver, so nothing is inferred from in-process state.+the agent's AgentClient sees the invite → auto-joins → on_self_join posts the+greeting. The assertion reads the room timeline back out of the messages table,+so nothing is inferred from in-process state."""from __future__ import annotations@@ -12,7 +12,7 @@ import asyncioimport pytest-from switch_core.db.models import Agent, CollaborationBridge, Room+from switch_core.db.models import Agent, CollaborationBridge, Message, Roomfrom switch_core.room_service import RoomCreateConfigfrom tests.integration.conftest import Harness, SessionEnv@@ -31,34 +31,28 @@ async def _wait_joined(async def _timeline_messages(- session_env: SessionEnv, matrix_room_id: str-) -> list[dict]:- """Every m.room.message in the room, read back off the homeserver as admin."""- admin = session_env.matrix_admin- resp = await admin._http.get(- f"/_matrix/client/r0/rooms/{matrix_room_id}/messages",- params={"dir": "b", "limit": 100},- headers={"Authorization": f"Bearer {admin.access_token}"},- )- resp.raise_for_status()- return [- event- for event in resp.json().get("chunk", [])- if event.get("type") == "m.room.message"- ]+ harness: Harness, session_env: SessionEnv, room_id: str+) -> list[Message]:+ """Every m.room.message in the room, read back out of the messages table."""+ async with harness.session_factory() as session: # type: ignore[operator]+ rows = await session_env.message_store.list_for_room(+ session, room_id, after_seq=0, limit=100+ )+ return [row for row in rows if row.event_type == "m.room.message"]async def _wait_for_message_from(+ harness: Harness,session_env: SessionEnv,- matrix_room_id: str,+ room_id: str,sender_localpart: str,timeout: float = 20,-) -> dict | None:+) -> Message | None:deadline = asyncio.get_event_loop().time() + timeoutwhile asyncio.get_event_loop().time() < deadline:- for event in await _timeline_messages(session_env, matrix_room_id):- if event.get("sender", "").startswith(f"@{sender_localpart}"):- return event+ for row in await _timeline_messages(harness, session_env, room_id):+ if row.sender_id.startswith(f"@{sender_localpart}"):+ return rowawait asyncio.sleep(0.5)return None@@ -82,13 +76,16 @@ async def test_agent_posts_greeting_on_join(await _wait_joined(client, matrix_room_id)greeting = await _wait_for_message_from(- session_env, matrix_room_id, client.matrix_user_id.split(":")[0].lstrip("@")+ harness,+ session_env,+ result.room.id,+ client.matrix_user_id.split(":")[0].lstrip("@"),)assert greeting is not None, ("agent joined the room but never posted a self-join greeting")- assert "e2e-greeter" in greeting["content"]["body"]+ assert "e2e-greeter" in greeting.content["body"]async def test_no_greeting_when_the_bridge_has_them_disabled(@@ -135,8 +132,9 @@ async def test_no_greeting_when_the_bridge_has_them_disabled(await _wait_joined(joiner_client, matrix_room_id)greeting = await _wait_for_message_from(+ harness,session_env,- matrix_room_id,+ result.room.id,joiner_client.matrix_user_id.split(":")[0].lstrip("@"),timeout=8,)diff --git a/core/tests/integration/test_harness_smoke.py b/core/tests/integration/test_harness_smoke.pyindex e58d8cd74..6fa094313 100644--- a/core/tests/integration/test_harness_smoke.py+++ b/core/tests/integration/test_harness_smoke.py@@ -1,9 +1,9 @@"""Smoke test for the integration harness.Proves the testcontainers stack boots and the in-process wiring works end-to-end:-register agents, start their Matrix sync clients, create a room via RoomService,-and confirm each agent's client actually joined the Matrix room. No feature under-test — this only exercises the harness itself.+register agents, start their clients, create a room via RoomService, and confirm+each agent's client actually joined the room. No feature under test — this only+exercises the harness itself."""from __future__ import annotations@@ -15,16 +15,16 @@ import pytestfrom switch_core.room_service import RoomCreateConfigfrom tests.integration.conftest import Harness-# Session-scoped event loop: the background matrix-nio sync clients are created-# and torn down across tests; sharing one loop lets their aiohttp sessions-# finalize cleanly instead of being stranded when a per-test loop is destroyed.+# Session-scoped event loop: the background clients are created and torn down+# across tests; sharing one loop lets their connections finalize cleanly instead+# of being stranded when a per-test loop is destroyed.pytestmark = [pytest.mark.integration, pytest.mark.asyncio(loop_scope="session")]async def _wait_joined(client: object, matrix_room_id: str, timeout: float = 30) -> None:- """Block until the agent's own client has joined the Matrix room."""+ """Block until the agent's own client has joined the room."""deadline = asyncio.get_event_loop().time() + timeoutwhile asyncio.get_event_loop().time() < deadline:if matrix_room_id in client.room_join_times: # type: ignore[attr-defined]diff --git a/core/tests/integration/test_room_join_e2e.py b/core/tests/integration/test_room_join_e2e.pyindex 6958e6d9f..ba03b487d 100644--- a/core/tests/integration/test_room_join_e2e.py+++ b/core/tests/integration/test_room_join_e2e.py@@ -1,8 +1,9 @@-"""End-to-end test for CHOO-576 room_join over a real Matrix homeserver.+"""End-to-end test for the room_join event over the real transport.-Drives the genuine path: RoomService adds an agent to a room → Matrix invite →-the watcher's AgentClient sync loop receives the join → on_member_event enqueues-a `room_join` AgentEvent → it is read off the event buffer. No mocks, no HTTP layer.+Drives the genuine path: RoomService adds an agent to a room → the invite reaches+the joiner → the watcher's AgentClient receive loop sees the arrival →+on_member_event enqueues a `room_join` AgentEvent → it is read off the event+buffer. No mocks, no HTTP layer."""from __future__ import annotations@@ -14,16 +15,16 @@ import pytestfrom switch_core.room_service import RoomCreateConfigfrom tests.integration.conftest import Harness-# Session-scoped event loop: the background matrix-nio sync clients are created-# and torn down across tests; sharing one loop lets their aiohttp sessions-# finalize cleanly instead of being stranded when a per-test loop is destroyed.+# Session-scoped event loop: the background clients are created and torn down+# across tests; sharing one loop lets their connections finalize cleanly instead+# of being stranded when a per-test loop is destroyed.pytestmark = [pytest.mark.integration, pytest.mark.asyncio(loop_scope="session")]async def _wait_joined(client: object, matrix_room_id: str, timeout: float = 30) -> None:- """Block until the agent's own client has joined the Matrix room."""+ """Block until the agent's own client has joined the room."""deadline = asyncio.get_event_loop().time() + timeoutwhile asyncio.get_event_loop().time() < deadline:if matrix_room_id in client.room_join_times: # type: ignore[attr-defined]@@ -101,6 +102,6 @@ async def test_self_join_does_not_emit_room_join(harness: Harness) -> None:)await _wait_joined(watcher_client, result.room.matrix_room_id)- # Give the sync loop time to deliver any (erroneous) self-join event.+ # Give the receive loop time to deliver any (erroneous) self-join event.event = await _drain_room_join(harness, watcher.agent_id, result.room.id, timeout=8)assert event is None, "self-join should not produce a room_join event"diff --git a/core/tests/switch_core/bridges/agent/protocol/registration_harness.py b/core/tests/switch_core/bridges/agent/protocol/registration_harness.pyindex 1d1c69bf7..4425b156f 100644--- a/core/tests/switch_core/bridges/agent/protocol/registration_harness.py+++ b/core/tests/switch_core/bridges/agent/protocol/registration_harness.py@@ -48,7 +48,6 @@ class FakeClientLifecycle:matrix_user_id=f"@{display_name}:test",display_name=display_name,type=client_type,- password="x",)session.add(client)await session.commit()diff --git a/core/tests/switch_core/bridges/agent/protocol/test_agent_detail_mcp_tools.py b/core/tests/switch_core/bridges/agent/protocol/test_agent_detail_mcp_tools.pyindex 2e7438c49..8ba2285d8 100644--- a/core/tests/switch_core/bridges/agent/protocol/test_agent_detail_mcp_tools.py+++ b/core/tests/switch_core/bridges/agent/protocol/test_agent_detail_mcp_tools.py@@ -55,7 +55,6 @@ async def _make_agent(matrix_user_id=f"@{name}:test",display_name=name,type="agent",- password="x",)session.add_all([api_key, client])await session.flush()diff --git a/core/tests/switch_core/bridges/agent/protocol/test_delegate_addressing.py b/core/tests/switch_core/bridges/agent/protocol/test_delegate_addressing.pyindex 1c7118e91..a85147746 100644--- a/core/tests/switch_core/bridges/agent/protocol/test_delegate_addressing.py+++ b/core/tests/switch_core/bridges/agent/protocol/test_delegate_addressing.py@@ -61,7 +61,6 @@ async def _make_agent(matrix_user_id=f"@{name}:test",display_name=name,type="agent",- password="x",)session.add_all([api_key, client])await session.flush()diff --git a/core/tests/switch_core/bridges/agent/protocol/test_pool_checkouts.py b/core/tests/switch_core/bridges/agent/protocol/test_pool_checkouts.pyindex 9a9342413..fc5aed1f1 100644--- a/core/tests/switch_core/bridges/agent/protocol/test_pool_checkouts.py+++ b/core/tests/switch_core/bridges/agent/protocol/test_pool_checkouts.py@@ -94,7 +94,6 @@ async def _seed(matrix_user_id=f"@{name}:test",display_name=name,type="agent",- password="x",)session.add_all([key, client])await session.flush()@@ -280,6 +279,7 @@ class TestTyping:id="room-1",name="room",description="d",+ transport_room_id="!r:test",matrix_room_id="!r:test",bridge_id="bridge-1",)diff --git a/core/tests/switch_core/bridges/agent/protocol/test_read_context.py b/core/tests/switch_core/bridges/agent/protocol/test_read_context.pyindex 76064c5a8..79b617e1c 100644--- a/core/tests/switch_core/bridges/agent/protocol/test_read_context.py+++ b/core/tests/switch_core/bridges/agent/protocol/test_read_context.py@@ -44,7 +44,6 @@ async def _make_room(session: AsyncSession) -> tuple[str, str]:matrix_user_id=f"@agent-{suffix}:test",display_name="agent one",type="agent",- password="x",)session.add(client)await session.flush()@@ -77,7 +76,7 @@ async def _write(message = Message(room_id=room_id,transport_event_id=event_id,- sender_matrix_id=sender,+ sender_id=sender,sender_client_id=client_id,sender_name=sender_name,event_type=event_type,diff --git a/core/tests/switch_core/bridges/agent/protocol/test_reference_type_parity.py b/core/tests/switch_core/bridges/agent/protocol/test_reference_type_parity.pyindex 8f47b683a..c79dd480e 100644--- a/core/tests/switch_core/bridges/agent/protocol/test_reference_type_parity.py+++ b/core/tests/switch_core/bridges/agent/protocol/test_reference_type_parity.py@@ -78,7 +78,6 @@ async def _make_agent(matrix_user_id=f"@{name}:example.invalid",display_name=name,type="agent",- password="x",)session.add_all([api_key, client])await session.flush()diff --git a/core/tests/switch_core/bridges/agent/protocol/test_register_overwrite_owner_guard.py b/core/tests/switch_core/bridges/agent/protocol/test_register_overwrite_owner_guard.pyindex 9f27f1ffd..5e039168c 100644--- a/core/tests/switch_core/bridges/agent/protocol/test_register_overwrite_owner_guard.py+++ b/core/tests/switch_core/bridges/agent/protocol/test_register_overwrite_owner_guard.py@@ -48,7 +48,6 @@ class _FakeClientLifecycle:matrix_user_id=f"@{display_name}:test",display_name=display_name,type=client_type,- password="x",)session.add(client)await session.commit()diff --git a/core/tests/switch_core/bridges/agent/protocol/test_room_roster_authz.py b/core/tests/switch_core/bridges/agent/protocol/test_room_roster_authz.pyindex 74d096d22..c728f03b6 100644--- a/core/tests/switch_core/bridges/agent/protocol/test_room_roster_authz.py+++ b/core/tests/switch_core/bridges/agent/protocol/test_room_roster_authz.py@@ -45,7 +45,6 @@ class _FakeClientLifecycle:matrix_user_id=f"@{display_name}:test",display_name=display_name,type=client_type,- password="x",)session.add(client)await session.commit()diff --git a/core/tests/switch_core/bridges/agent/test_api_key_cache.py b/core/tests/switch_core/bridges/agent/test_api_key_cache.pyindex 8290aef72..45607f7aa 100644--- a/core/tests/switch_core/bridges/agent/test_api_key_cache.py+++ b/core/tests/switch_core/bridges/agent/test_api_key_cache.py@@ -64,7 +64,6 @@ class _FakeClientLifecycle:matrix_user_id=f"@{display_name}:test",display_name=display_name,type=client_type,- password="x",)session.add(client)await session.commit()@@ -129,7 +128,6 @@ async def _seed_agent(matrix_user_id=f"@{name}:test",display_name=name,type="agent",- password="x",)session.add(client)await session.flush()@@ -413,11 +411,7 @@ _CONFIG_KWARGS = dict(db_user="postgres",db_password="pw",db_name="switch",- matrix_server="http://tuwunel:8008",matrix_server_name="switch.local",- matrix_admin_user="admin",- matrix_admin_password="pw",- matrix_registration_shared_secret="secret",agent_registration_token="token",jwt_secret_key="jwt",gateway_admin_email="[email protected]",diff --git a/core/tests/switch_core/bridges/agent/test_mediation.py b/core/tests/switch_core/bridges/agent/test_mediation.pyindex 18ddda078..f6866bfd9 100644--- a/core/tests/switch_core/bridges/agent/test_mediation.py+++ b/core/tests/switch_core/bridges/agent/test_mediation.py@@ -29,7 +29,9 @@ async def _agent(session: AsyncSession) -> Agent:type="agent",)client = Client(- matrix_user_id=f"@{name}:test", display_name=name, type="agent", password="x"+ matrix_user_id=f"@{name}:test",+ display_name=name,+ type="agent",)session.add_all([api_key, client])await session.flush()diff --git a/core/tests/switch_core/bridges/collaboration/test_bridge_inbound_media.py b/core/tests/switch_core/bridges/collaboration/test_bridge_inbound_media.pyindex cc8c8d94a..6fbd5fc4d 100644--- a/core/tests/switch_core/bridges/collaboration/test_bridge_inbound_media.py+++ b/core/tests/switch_core/bridges/collaboration/test_bridge_inbound_media.py@@ -146,7 +146,7 @@ async def test_three_attachments_are_stamped_as_one_group() -> None:assert bridge.recorded == [{"external_channel_id": "chan-1",- "matrix_event_id": "$evt-0",+ "transport_event_id": "$evt-0","external_post_id": "post-1",}]diff --git a/core/tests/switch_core/bridges/collaboration/test_bridge_outbound_media.py b/core/tests/switch_core/bridges/collaboration/test_bridge_outbound_media.pyindex aa23f0190..3438b6c94 100644--- a/core/tests/switch_core/bridges/collaboration/test_bridge_outbound_media.py+++ b/core/tests/switch_core/bridges/collaboration/test_bridge_outbound_media.py@@ -128,8 +128,8 @@ def _fake_bridge(recorded: list[dict[str, str]] = []lookup = matrix_to_external or {}- async def _external_post_for_matrix_event(matrix_event_id: str) -> str | None:- return lookup.get(matrix_event_id)+ async def _external_post_for_matrix_event(transport_event_id: str) -> str | None:+ return lookup.get(transport_event_id)async def _record_message_map(**kwargs: str) -> None:recorded.append(kwargs)@@ -200,7 +200,7 @@ async def test_image_relays_via_send_attachment_and_records_map() -> None:assert bridge.recorded == [{"external_channel_id": "chan-1",- "matrix_event_id": "$media-event",+ "transport_event_id": "$media-event","external_post_id": "ext-ref-1",}]@@ -335,7 +335,7 @@ async def test_grouped_attachments_relay_as_one_platform_post() -> None:assert [f.filename for f in batch["files"]] == ["cat.png", "notes.md", "data.csv"]assert batch["caption"] == "three files"# The group correlates via its first event, so replies thread back.- assert bridge.recorded[0]["matrix_event_id"] == "$part-0"+ assert bridge.recorded[0]["transport_event_id"] == "$part-0"assert bridge.recorded[0]["external_post_id"] == "ext-ref-3"assert bridge._outbound_groups == {}diff --git a/core/tests/switch_core/bridges/collaboration/test_bridge_unknown_member_first_message.py b/core/tests/switch_core/bridges/collaboration/test_bridge_unknown_member_first_message.pyindex d45e148cb..fa0eaaa11 100644--- a/core/tests/switch_core/bridges/collaboration/test_bridge_unknown_member_first_message.py+++ b/core/tests/switch_core/bridges/collaboration/test_bridge_unknown_member_first_message.py@@ -175,7 +175,7 @@ async def test_first_message_from_app_sender_is_relayed() -> None:return Falseasync def _record_message_map(**kwargs: str) -> None:- relayed.append((kwargs["matrix_event_id"], kwargs["external_post_id"]))+ relayed.append((kwargs["transport_event_id"], kwargs["external_post_id"]))inner = _bridge(puppet, known_puppets={})@@ -225,7 +225,7 @@ async def test_first_message_from_unknown_member_is_relayed() -> None:return puppetasync def _record_message_map(**kwargs: str) -> None:- relayed.append((kwargs["matrix_event_id"], kwargs["external_post_id"]))+ relayed.append((kwargs["transport_event_id"], kwargs["external_post_id"]))bridge = SimpleNamespace(_repair_placeholder_username=_noop_repair,diff --git a/core/tests/switch_core/bridges/collaboration/test_command_thread_mapping.py b/core/tests/switch_core/bridges/collaboration/test_command_thread_mapping.pyindex 209f6fb49..22e481efd 100644--- a/core/tests/switch_core/bridges/collaboration/test_command_thread_mapping.py+++ b/core/tests/switch_core/bridges/collaboration/test_command_thread_mapping.py@@ -92,7 +92,7 @@ class TestCommandThreadMapping:assert bridge.recorded == [{"external_channel_id": "chan-1",- "matrix_event_id": "$cmd-event",+ "transport_event_id": "$cmd-event","external_post_id": "mm-post-1",}]@@ -134,7 +134,7 @@ class TestCommandThreadMapping:assert bridge.recorded == [{"external_channel_id": "chan-1",- "matrix_event_id": "$cmd-event",+ "transport_event_id": "$cmd-event","external_post_id": "mm-root",}]@@ -234,7 +234,9 @@ class TestCommandResultThreadingRace:_adapter=SimpleNamespace(translate_inbound=lambda s: s),_pending_message_maps={},_bridge_id="b",- _bridge_message_map_store=SimpleNamespace(get_by_matrix_event_id=_get_none),+ _bridge_message_map_store=SimpleNamespace(+ get_by_transport_event_id=_get_none+ ),_session_factory=lambda: _NullSession(),_ensure_user_in_matrix_room=_ensure_user_in_matrix_room,_matrix_event_for_external_post=_matrix_event_for_external_post,@@ -247,12 +249,12 @@ class TestCommandResultThreadingRace:)async def _record_message_map(- *, external_channel_id: str, matrix_event_id: str, external_post_id: str+ *, external_channel_id: str, transport_event_id: str, external_post_id: str) -> None:# Simulate the reply's outbound relay resolving the thread root while# this write is still in flight.resolved["mid"] = await bridge._external_post_for_matrix_event(- matrix_event_id+ transport_event_id)bridge._record_message_map = _record_message_mapdiff --git a/core/tests/switch_core/bridges/collaboration/test_lifecycle_remove.py b/core/tests/switch_core/bridges/collaboration/test_lifecycle_remove.pyindex cb4ed3342..b8b405ed5 100644--- a/core/tests/switch_core/bridges/collaboration/test_lifecycle_remove.py+++ b/core/tests/switch_core/bridges/collaboration/test_lifecycle_remove.py@@ -59,7 +59,6 @@ async def _make_client(session: AsyncSession, *, client_type: str) -> str:matrix_user_id=f"@{client_type}-{uuid.uuid4().hex[:8]}:test",display_name=f"{client_type} client",type=client_type,- password="x",)session.add(client)await session.flush()diff --git a/core/tests/switch_core/clients/test_agent_client_pool_checkouts.py b/core/tests/switch_core/clients/test_agent_client_pool_checkouts.pyindex c51dfcb67..8f4a62e9e 100644--- a/core/tests/switch_core/clients/test_agent_client_pool_checkouts.py+++ b/core/tests/switch_core/clients/test_agent_client_pool_checkouts.py@@ -83,7 +83,6 @@ async def _seed(matrix_user_id=f"@{name}:test",display_name=name,type="agent",- password="x",)session.add_all([key, client])await session.flush()diff --git a/core/tests/switch_core/clients/test_client_construction.py b/core/tests/switch_core/clients/test_client_construction.pyindex 2f6cd5f46..e6db41f70 100644--- a/core/tests/switch_core/clients/test_client_construction.py+++ b/core/tests/switch_core/clients/test_client_construction.py@@ -21,9 +21,6 @@ import pytestfrom switch_core.clients.admin_client import AdminClientfrom switch_core.clients.bridge_client import BridgeClient, BridgeClientConfigfrom switch_core.clients.client_base import ClientBase, ClientConfig-from switch_core.clients.client_factory import matrix_transport_for-from switch_core.clients.user_client import UserClient-from tests.switch_core.transport.fake import FakeMessageRecorderdef _base_kwargs() -> dict[str, Any]:@@ -31,14 +28,9 @@ def _base_kwargs() -> dict[str, Any]:"client_id": "client-1","matrix_user_id": "@switch-agent-1:switch.local","display_name": "agent-one",- "password": "pw",- "server_url": "https://matrix.example","session_factory": object(),"client_store": object(),- "transport_factory": matrix_transport_for,- "session_state": {"access_token": None, "device_id": None},- "next_batch_token": None,- "message_recorder": FakeMessageRecorder(),+ "transport_factory": lambda client: object(),}@@ -46,8 +38,6 @@ def test_client_base_takes_what_the_factory_passes() -> None:client = ClientBase(config=ClientConfig(), **_base_kwargs())assert client.client_id == "client-1"- # Credentials are held opaquely, not unpacked into named attributes.- assert client.session_state == {"access_token": None, "device_id": None}def test_bridge_client_construction_matches_its_lifecycle_call_site() -> None:@@ -60,7 +50,7 @@ def test_bridge_client_construction_matches_its_lifecycle_call_site() -> None:assert client.config.bridge_id == "bridge-1"[email protected]("cls", [UserClient, AdminClient])[email protected]("cls", [AdminClient])def test_every_registered_client_type_constructs(cls: type) -> None:base = _base_kwargs()# Collaborators the factory injects as extra kwargs differ per class, so@@ -83,12 +73,12 @@ def test_every_registered_client_type_constructs(cls: type) -> None:assert isinstance(client, ClientBase)-def test_no_client_holds_matrix_credentials_directly() -> None:- """The transport owns them; a client that reads them is a Matrix client."""+def test_no_client_holds_transport_credentials() -> None:+ """A client is a row, not a session: there is nothing to authenticate."""client = ClientBase(config=ClientConfig(), **_base_kwargs())- assert not hasattr(client, "access_token")- assert not hasattr(client, "device_id")+ for attribute in ("access_token", "device_id", "password", "session_state"):+ assert not hasattr(client, attribute)def test_no_call_site_passes_transport_credentials_to_a_client() -> None:@@ -100,7 +90,7 @@ def test_no_call_site_passes_transport_credentials_to_a_client() -> None:for the argument itself."""package = pathlib.Path(__file__).resolve().parents[3] / "switch_core"- retired = {"device_id", "access_token"}+ retired = {"device_id", "access_token", "session_state", "next_batch_token"}offenders: list[str] = []for path in package.rglob("*.py"):@@ -117,18 +107,18 @@ def test_no_call_site_passes_transport_credentials_to_a_client() -> None:offenders.append(f"{path.relative_to(package)}: {name}({arg}=...)")assert offenders == [], (- "clients take an opaque `session_state`, not individual credentials; "- f"these call sites still pass them: {offenders}"+ "a client carries no transport credentials; these call sites still "+ f"pass them: {offenders}")def test_only_the_factory_names_a_transport_implementation() -> None:"""A client's bus is one decision, made in one place.- The collaboration bridges named `matrix_transport_for` directly, so the- transport setting reached every agent and no bridge: agents talked in rows,- bridges talked to the homeserver, and neither side saw the other or logged- anything. Nothing failed — messages simply stopped crossing.+ The collaboration bridges once built their own transport directly, so a+ change of transport reached every agent and no bridge: each side was+ healthy, neither reached the other, and nothing was logged. Nothing+ failed — messages simply stopped crossing.That is invisible in review, so scan for the cause instead. Any module thatcan reach an implementation can pick the wrong one.@@ -139,19 +129,10 @@ def test_only_the_factory_names_a_transport_implementation() -> None:`allowed` is a deliberate act a reviewer sees."""package = pathlib.Path(__file__).resolve().parents[3] / "switch_core"- private = {"switch_core.transport.matrix", "switch_core.transport.postgres"}+ private = {"switch_core.transport.postgres"}allowed = {"clients/client_factory.py",- "transport/matrix.py","transport/postgres.py",- # The migration commands are the exception the rule is not about. They- # exist to read the homeserver — that is the whole job — and they must- # do it whatever transport the running server is on, which is how a- # backfill can still be run after the flip. A client picking its own- # bus is the bug; a one-off command naming the bus it is migrating- # away from is the point.- "cli/backfill.py",- "cli/reconcile.py",}offenders: list[str] = []diff --git a/core/tests/switch_core/clients/test_client_factory_transport_choice.py b/core/tests/switch_core/clients/test_client_factory_transport_choice.pydeleted file mode 100644index 6fdf238f4..000000000--- a/core/tests/switch_core/clients/test_client_factory_transport_choice.py+++ /dev/null@@ -1,101 +0,0 @@-"""One setting picks the transport, and the recorder that goes with it.--The pairing is the thing worth a test. A transport that stores what it carries-paired with a recorder that also writes would double every row; the other way-round would leave the log empty. Both are silent until someone reads the table,-which is why the decision is made in one place and checked here.-"""--from __future__ import annotations--from typing import Any--import pytest-from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker--from switch_core.clients.client_base import ClientBase-from switch_core.clients.client_factory import ClientFactory-from switch_core.config import SwitchConfig-from switch_core.db.models import Client-from switch_core.db.stores.client_store import ClientStore-from switch_core.db.stores.media_store import MediaStore-from switch_core.db.stores.message_store import MessageStore-from switch_core.db.stores.room_store import RoomStore-from switch_core.messages import MessageRecorder-from switch_core.messages.notify import MessageListener-from switch_core.messages.recording import NoRecording-from switch_core.transport.ephemeral import EphemeralBus-from switch_core.transport.invites import InviteBus-from switch_core.transport.matrix import MatrixTransport-from switch_core.transport.postgres import PostgresTransport---def _factory(- session_factory: async_sessionmaker[AsyncSession], transport: str-) -> ClientFactory:- config = SwitchConfig(- db_host="db",- db_port="5432",- db_user="postgres",- db_password="pw",- db_name="switch",- matrix_server="http://nowhere",- matrix_server_name="switch.local",- matrix_admin_user="admin",- matrix_admin_password="pw",- matrix_registration_shared_secret="secret",- agent_registration_token="token",- jwt_secret_key="jwt",- gateway_admin_email="[email protected]",- gateway_admin_password="pw",- message_transport=transport, # type: ignore[arg-type]- )- recorder = MessageRecorder(- session_factory=session_factory,- room_store=RoomStore(),- message_store=MessageStore(),- )- factory = ClientFactory(- client_store=ClientStore(),- session_factory=session_factory,- config=config,- message_recorder=recorder,- room_store=RoomStore(),- message_store=MessageStore(),- media_store=MediaStore(),- listener=MessageListener(lambda: None), # type: ignore[arg-type,return-value]- invites=InviteBus(),- ephemeral=EphemeralBus(),- )- factory.register("user", ClientBase)- return factory---def _record() -> Client:- return Client(- id="client-1",- matrix_user_id="@someone:test",- display_name="someone",- type="user",- password="x",- )--- ("setting", "transport_class", "records_separately"),- [- ("matrix", MatrixTransport, True),- ("postgres", PostgresTransport, False),- ],-)-def test_the_setting_picks_both_halves(- session_factory: async_sessionmaker[AsyncSession],- setting: str,- transport_class: type,- records_separately: bool,-) -> None:- client: Any = _factory(session_factory, setting).create(_record())-- assert isinstance(client._transport_factory(client), transport_class)- assert isinstance(client.message_recorder, MessageRecorder) is records_separately- assert isinstance(client.message_recorder, NoRecording) is not records_separatelydiff --git a/core/tests/switch_core/clients/test_self_join_dispatch.py b/core/tests/switch_core/clients/test_self_join_dispatch.pyindex cf308a12d..932212e08 100644--- a/core/tests/switch_core/clients/test_self_join_dispatch.py+++ b/core/tests/switch_core/clients/test_self_join_dispatch.py@@ -18,20 +18,6 @@ MATRIX_ROOM_ID = "!matrix:switch.local"SELF = "@switch-agent-1:switch.local"-class _Recorder:- """Stands in for MessageRecorder, collecting the arrivals written."""-- def __init__(self) -> None:- self.joins: list[str] = []- self.names: list[str] = []-- async def record_join(- self, *, transport_room_id, event, client_id, member_name- ) -> None:- self.joins.append(event.event_id)- self.names.append(member_name)--class _Client(ClientBase):def __init__(self) -> None:self.matrix_user_id = SELF@@ -43,7 +29,6 @@ class _Client(ClientBase):self._startup_ts = 1000self.self_joins: list[str] = []self.member_events: list[str] = []- self.message_recorder = _Recorder()async def on_self_join(self, room, event) -> None:self.self_joins.append(room.room_id)@@ -156,27 +141,3 @@ async def test_an_arrival_is_logged_even_when_it_is_not_announced() -> None:await client._handle_member_event(_room(), _member_event(timestamp=500))assert client.self_joins == []- assert client.message_recorder.joins == ["$member"]---async def test_a_profile_update_is_not_logged_as_an_arrival() -> None:- client = _Client()-- await client._handle_member_event(- _room(), _member_event(prev_membership="join", timestamp=3000)- )-- assert client.message_recorder.joins == []---async def test_another_member_arriving_is_not_this_client_to_log() -> None:- """Each client logs its own arrival, which is what makes it one row."""- client = _Client()-- await client._handle_member_event(- _room(), _member_event(state_key="@switch-agent-2:switch.local")- )-- assert client.message_recorder.joins == []diff --git a/core/tests/switch_core/clients/test_send_media.py b/core/tests/switch_core/clients/test_send_media.pyindex 5381be148..c5d72a44d 100644--- a/core/tests/switch_core/clients/test_send_media.py+++ b/core/tests/switch_core/clients/test_send_media.py@@ -4,7 +4,7 @@ import pytestfrom switch_core.clients.client_base import ClientBasefrom switch_core.transport import TransportError, UploadResult-from tests.switch_core.transport.fake import FakeMessageRecorder, FakeTransport+from tests.switch_core.transport.fake import FakeTransportclass _FailingUploadTransport(FakeTransport):@@ -22,7 +22,6 @@ def _client(transport: FakeTransport) -> ClientBase:client.display_name = "Alice" # type: ignore[attr-defined]client.matrix_user_id = "@alice:switch.local" # type: ignore[attr-defined]client.client_id = "client-1" # type: ignore[attr-defined]- client.message_recorder = FakeMessageRecorder() # type: ignore[assignment]return clientdiff --git a/core/tests/switch_core/clients/test_send_recording.py b/core/tests/switch_core/clients/test_send_recording.pydeleted file mode 100644index 90ac04366..000000000--- a/core/tests/switch_core/clients/test_send_recording.py+++ /dev/null@@ -1,136 +0,0 @@-"""A client records every kind of send, and records nothing it did not send.--The recorder is only as good as the set of sends that reach it. Messages,-media and custom events each take a different route out of `ClientBase`, so-each is checked here rather than trusting that instrumenting one instrumented-the others.--The negative cases carry as much weight. A row for a message the transport-rejected would be a record of something that never happened, which is worse-than no record at all.-"""--from __future__ import annotations--import pytest--from switch_core.clients.client_base import ClientBase, ClientConfig-from switch_core.transport import TransportError-from tests.switch_core.transport.fake import FakeMessageRecorder, FakeTransport---def _client(- transport: FakeTransport,-) -> tuple[ClientBase[ClientConfig], FakeMessageRecorder]:- recorder = FakeMessageRecorder()- client: ClientBase[ClientConfig] = object.__new__(ClientBase)- client.transport = transport- client.message_recorder = recorder- client.matrix_user_id = "@agent:test"- client.client_id = "client-1"- client.display_name = "agent one"- return client, recorder---class TestEverySendIsRecorded:- async def test_a_message(self) -> None:- transport = FakeTransport()- client, recorder = _client(transport)-- event_id = await client.send_message("!room:test", "hello")-- assert len(recorder.recorded) == 1- recorded = recorder.recorded[0]- assert recorded["transport_room_id"] == "!room:test"- assert recorded["result"].event_id == event_id- assert recorded["result"].content["body"] == "hello"- assert recorded["sender_matrix_id"] == "@agent:test"- assert recorded["sender_client_id"] == "client-1"- assert recorded["sender_name"] == "agent one"-- async def test_a_custom_event(self) -> None:- transport = FakeTransport()- client, recorder = _client(transport)-- event_id = await client.send_event(- "!room:test", "com.switch.command", {"command": "reset"}- )-- assert len(recorder.recorded) == 1- result = recorder.recorded[0]["result"]- assert result.event_id == event_id- assert result.event_type == "com.switch.command"- assert result.content == {"command": "reset"}-- async def test_media(self) -> None:- transport = FakeTransport()- client, recorder = _client(transport)-- event_id = await client.send_media(- "!room:test",- "mxc://test/abc",- "chart.png",- "image/png",- 4096,- msgtype="m.image",- )-- assert len(recorder.recorded) == 1- result = recorder.recorded[0]["result"]- assert result.event_id == event_id- assert result.content["url"] == "mxc://test/abc"-- async def test_what_is_recorded_is_what_the_transport_sent(self) -> None:- """The recorder is handed the transport's own content dict.-- Reconstructing it at the client would drift from the transport the- first time the wire format changed, and nothing would notice.- """- transport = FakeTransport()- client, recorder = _client(transport)-- await client.send_message(- "!room:test", "**bold**", format="markdown", thread_root_id="$root"- )-- content = recorder.recorded[0]["result"].content- assert content["m.relates_to"] == {- "rel_type": "m.thread",- "event_id": "$root",- }---class TestNothingIsRecordedForASendThatFailed:- async def test_a_failed_message(self) -> None:- transport = FakeTransport(fail_send="homeserver said no")- client, recorder = _client(transport)-- assert await client.send_message("!room:test", "hello") is None- assert recorder.recorded == []-- async def test_a_failed_media_send(self) -> None:- transport = FakeTransport(fail_send="homeserver said no")- client, recorder = _client(transport)-- result = await client.send_media(- "!room:test",- "mxc://test/abc",- "chart.png",- "image/png",- 4096,- msgtype="m.image",- )-- assert result is None- assert recorder.recorded == []-- async def test_a_failed_custom_event_raises_and_records_nothing(self) -> None:- """Unlike a chat message, a custom event carries protocol state, so the- caller is told rather than left to assume it landed."""- transport = FakeTransport(fail_send="homeserver said no")- client, recorder = _client(transport)-- with pytest.raises(TransportError):- await client.send_event("!room:test", "com.switch.command", {})-- assert recorder.recorded == []diff --git a/core/tests/switch_core/clients/test_sync_cursor_persistence.py b/core/tests/switch_core/clients/test_sync_cursor_persistence.pydeleted file mode 100644index 0d4fe4743..000000000--- a/core/tests/switch_core/clients/test_sync_cursor_persistence.py+++ /dev/null@@ -1,216 +0,0 @@-"""Cursor writes must not stampede the connection pool.--Every Matrix client in the process persists its `next_batch` cursor, and both-the sync long-poll and the write throttle were 30s. All clients start when the-process does, so hundreds of them crossed the threshold in the same instant and-each opened its own transaction for a one-row update — enough to exhaust the-pool on an otherwise idle instance.--Two properties fix that and are asserted here: an idle sync does not earn a-write at all, and the clients that do write are spread out rather than aligned.--Whether a given batch counted as idle is the transport's judgement, not this-one's — it is the only thing that can see what the batch held, and it says so-as the `durable` flag these tests pass in. What that flag is derived from is-covered in `tests/switch_core/transport/test_sync_durability.py`.-"""--from __future__ import annotations--import time--import pytest--from switch_core.clients.client_base import (- SYNC_STATE_INTERVAL,- SYNC_STATE_JITTER,- SYNC_STATE_MAX_STALENESS,- ClientBase,-)-from tests.switch_core.transport.fake import FakeMessageRecorder--ROOM = "!room:switch.local"---class _Session:- async def __aenter__(self) -> _Session:- return self-- async def __aexit__(self, *_: object) -> None:- return None-- async def commit(self) -> None:- return None---class _Store:- """Records what the real `_persist_state` would have written."""-- def __init__(self) -> None:- self.writes: list[str | None] = []-- async def update_state(- self,- session: object,- client_id: str,- *,- next_batch_token: str | None,- ) -> None:- self.writes.append(next_batch_token)---class _Client(ClientBase):- """A real client with the database swapped out.-- `_persist_state` itself is deliberately not overridden — the throttle it- applies is half of what these tests are about.- """-- def __init__(self) -> None:- super().__init__(- client_id="c1",- matrix_user_id="@switch-agent-1:switch.local",- display_name="agent-1",- password="",- server_url="http://matrix.invalid",- session_factory=_Session, # type: ignore[arg-type]- client_store=_Store(), # type: ignore[arg-type]- config=ClientBase.config_class(),- transport_factory=lambda client: object(), # type: ignore[arg-type,return-value]- session_state={},- message_recorder=FakeMessageRecorder(),- next_batch_token="s0",- )- # Pin the jittered interval so the throttle assertions are deterministic.- self._sync_persist_interval = SYNC_STATE_INTERVAL-- @property- def writes(self) -> list[str | None]:- store: _Store = self.client_store # type: ignore[assignment]- return store.writes---async def test_idle_sync_advances_the_cursor_without_writing() -> None:- """The overnight case: nothing happened, so nothing is worth persisting.-- Resuming from the older cursor replays the same nothing, so the write buys- us no safety — and it was the whole burst.- """- client = _Client()- client._last_sync_persist = time.monotonic() - 10 * SYNC_STATE_INTERVAL-- for i in range(1, 6):- await client._handle_sync(f"s{i}", False)-- assert client.writes == []- # The cursor still moved, so the next sync does not refetch.- assert client.next_batch_token == "s5"- assert client._sync_state_dirty is True---async def test_a_batch_the_transport_called_idle_does_not_earn_a_write() -> None:- """Typing notices and read receipts advance the token and mean nothing."""- client = _Client()- client._last_sync_persist = time.monotonic() - 10 * SYNC_STATE_INTERVAL-- await client._handle_sync("s1", False)-- assert client.writes == []- assert client.next_batch_token == "s1"---async def test_durable_events_are_persisted() -> None:- """Anything a restart would replay to a handler must move the cursor."""- client = _Client()- client._last_sync_persist = time.monotonic() - 10 * SYNC_STATE_INTERVAL-- await client._handle_sync("s1", True)-- assert client.writes == ["s1"]---async def test_a_stale_cursor_is_flushed_even_while_idle() -> None:- """Skipping idle writes must not let the token fall arbitrarily behind."""- client = _Client()- client._last_sync_persist = time.monotonic() - (SYNC_STATE_MAX_STALENESS - 1)-- await client._handle_sync("s1", False)- assert client.writes == []-- client._last_sync_persist = time.monotonic() - (SYNC_STATE_MAX_STALENESS + 1)- await client._handle_sync("s2", False)-- assert client.writes == ["s2"]- assert client._sync_state_dirty is False---async def test_busy_clients_are_still_throttled() -> None:- """A room under load must not write a row per message."""- client = _Client()- client._last_sync_persist = time.monotonic() - 10 * SYNC_STATE_INTERVAL-- await client._handle_sync("s1", True)- await client._handle_sync("s2", True)- await client._handle_sync("s3", True)-- assert client.writes == ["s1"]- assert client.next_batch_token == "s3"---async def test_an_unchanged_cursor_is_ignored() -> None:- client = _Client()- client._last_sync_persist = time.monotonic() - 10 * SYNC_STATE_INTERVAL-- await client._handle_sync("s0", True)-- assert client.writes == []---def _real_clients(count: int) -> list[ClientBase]:- """Construct through `ClientBase.__init__` — the jitter is set there."""- return [- ClientBase(- client_id=f"c{i}",- matrix_user_id=f"@switch-agent-{i}:switch.local",- display_name=f"agent-{i}",- password="",- server_url="http://matrix.invalid",- session_factory=None, # type: ignore[arg-type]- client_store=None, # type: ignore[arg-type]- config=ClientBase.config_class(),- transport_factory=lambda client: object(), # type: ignore[arg-type,return-value]- session_state={},- message_recorder=FakeMessageRecorder(),- )- for i in range(count)- ]---def test_clients_do_not_share_a_write_deadline() -> None:- """The stampede is the alignment, so the intervals must differ per client."""- clients = _real_clients(200)-- intervals = {c._sync_persist_interval for c in clients}- assert len(intervals) > 190, "intervals are effectively identical"- assert all(- SYNC_STATE_INTERVAL <= i <= SYNC_STATE_INTERVAL + SYNC_STATE_JITTER- for i in intervals- )---def test_the_first_write_after_boot_is_spread_across_the_window() -> None:- """Clients all start together, so seeding the clock identically would put- their first write in the same instant too."""- now = time.monotonic()- elapsed = sorted(now - c._last_sync_persist for c in _real_clients(200))-- assert elapsed[0] >= 0.0- assert elapsed[-1] <= SYNC_STATE_INTERVAL- # Spread over the window rather than bunched at one end of it.- assert elapsed[-1] - elapsed[0] > SYNC_STATE_INTERVAL / 2diff --git a/core/tests/switch_core/db/stores/test_agent_session_store.py b/core/tests/switch_core/db/stores/test_agent_session_store.pyindex 3b9698d1b..67ab851e7 100644--- a/core/tests/switch_core/db/stores/test_agent_session_store.py+++ b/core/tests/switch_core/db/stores/test_agent_session_store.py@@ -28,7 +28,6 @@ async def _make_agent(session: AsyncSession, name: str) -> Agent:matrix_user_id=f"@{name}:test",display_name=name,type="agent",- password="x",)session.add_all([api_key, client])await session.flush()diff --git a/core/tests/switch_core/db/stores/test_agent_store.py b/core/tests/switch_core/db/stores/test_agent_store.pyindex feee9b14d..c7e617716 100644--- a/core/tests/switch_core/db/stores/test_agent_store.py+++ b/core/tests/switch_core/db/stores/test_agent_store.py@@ -23,7 +23,6 @@ async def _make_agent(session: AsyncSession, name: str) -> Agent:matrix_user_id=f"@{name}:test",display_name=name,type="agent",- password="x",)session.add_all([api_key, client])await session.flush()diff --git a/core/tests/switch_core/db/stores/test_api_key_agent_join.py b/core/tests/switch_core/db/stores/test_api_key_agent_join.pyindex 5c73d93e7..72d5a034a 100644--- a/core/tests/switch_core/db/stores/test_api_key_agent_join.py+++ b/core/tests/switch_core/db/stores/test_api_key_agent_join.py@@ -34,7 +34,9 @@ async def _make_key(async def _make_agent(session: AsyncSession, name: str, api_key: ApiKey) -> Agent:client = Client(- matrix_user_id=f"@{name}:test", display_name=name, type="agent", password="x"+ matrix_user_id=f"@{name}:test",+ display_name=name,+ type="agent",)session.add(client)await session.flush()diff --git a/core/tests/switch_core/db/stores/test_bridge_message_map_store.py b/core/tests/switch_core/db/stores/test_bridge_message_map_store.pyindex e0cc2e996..30a5c2c13 100644--- a/core/tests/switch_core/db/stores/test_bridge_message_map_store.py+++ b/core/tests/switch_core/db/stores/test_bridge_message_map_store.py@@ -14,7 +14,6 @@ async def _make_bridge(session: AsyncSession) -> str:matrix_user_id=f"@bridge-{uuid.uuid4().hex[:8]}:test",display_name="bridge client",type="bridge",- password="x",)session.add(client)await session.flush()@@ -41,17 +40,19 @@ class TestBridgeMessageMapStore:BridgeMessageMap(bridge_id=bridge_id,external_channel_id="chan-1",- matrix_event_id="$evt1",+ transport_event_id="$evt1",external_post_id="post1",),)await session.commit()- by_event = await store.get_by_matrix_event_id(session, bridge_id, "$evt1")+ by_event = await store.get_by_transport_event_id(+ session, bridge_id, "$evt1"+ )by_post = await store.get_by_external_post_id(session, bridge_id, "post1")assert by_event is not None and by_event.external_post_id == "post1"- assert by_post is not None and by_post.matrix_event_id == "$evt1"+ assert by_post is not None and by_post.transport_event_id == "$evt1"async def test_lookup_miss_returns_none(self, session_factory: async_sessionmaker[AsyncSession]@@ -60,13 +61,14 @@ class TestBridgeMessageMapStore:async with session_factory() as session:bridge_id = await _make_bridge(session)assert (- await store.get_by_matrix_event_id(session, bridge_id, "$nope") is None+ await store.get_by_transport_event_id(session, bridge_id, "$nope")+ is None)assert (await store.get_by_external_post_id(session, bridge_id, "nope") is None)- async def test_delete_by_matrix_event_id(+ async def test_delete_by_transport_event_id(self, session_factory: async_sessionmaker[AsyncSession]) -> None:store = BridgeMessageMapStore()@@ -77,17 +79,18 @@ class TestBridgeMessageMapStore:BridgeMessageMap(bridge_id=bridge_id,external_channel_id="chan-1",- matrix_event_id="$evt1",+ transport_event_id="$evt1",external_post_id="post1",),)await session.commit()- await store.delete_by_matrix_event_id(session, bridge_id, "$evt1")+ await store.delete_by_transport_event_id(session, bridge_id, "$evt1")await session.commit()assert (- await store.get_by_matrix_event_id(session, bridge_id, "$evt1") is None+ await store.get_by_transport_event_id(session, bridge_id, "$evt1")+ is None)assert (await store.get_by_external_post_id(session, bridge_id, "post1") is None@@ -106,14 +109,15 @@ class TestBridgeMessageMapStore:BridgeMessageMap(bridge_id=bridge_a,external_channel_id="chan-1",- matrix_event_id="$evt1",+ transport_event_id="$evt1",external_post_id="post1",),)await session.commit()assert (- await store.get_by_matrix_event_id(session, bridge_b, "$evt1") is None+ await store.get_by_transport_event_id(session, bridge_b, "$evt1")+ is None)assert (await store.get_by_external_post_id(session, bridge_b, "post1") is Nonediff --git a/core/tests/switch_core/db/stores/test_collaboration_bridge_store.py b/core/tests/switch_core/db/stores/test_collaboration_bridge_store.pyindex 3804ddfdf..d1cfd655d 100644--- a/core/tests/switch_core/db/stores/test_collaboration_bridge_store.py+++ b/core/tests/switch_core/db/stores/test_collaboration_bridge_store.py@@ -15,7 +15,6 @@ async def _make_bridge(session: AsyncSession) -> str:matrix_user_id=f"@bridge-{uuid.uuid4().hex[:8]}:test",display_name="bridge client",type="bridge",- password="x",)session.add(client)await session.flush()diff --git a/core/tests/switch_core/db/stores/test_delivery_cursor_store.py b/core/tests/switch_core/db/stores/test_delivery_cursor_store.pyindex 9736f7352..c6b700663 100644--- a/core/tests/switch_core/db/stores/test_delivery_cursor_store.py+++ b/core/tests/switch_core/db/stores/test_delivery_cursor_store.py@@ -24,7 +24,9 @@ async def _agent(session: AsyncSession) -> Agent:type="agent",)client = Client(- matrix_user_id=f"@{name}:test", display_name=name, type="agent", password="x"+ matrix_user_id=f"@{name}:test",+ display_name=name,+ type="agent",)session.add_all([api_key, client])await session.flush()diff --git a/core/tests/switch_core/db/stores/test_external_user_claims.py b/core/tests/switch_core/db/stores/test_external_user_claims.pyindex 41e22b5a1..f2c1e29f4 100644--- a/core/tests/switch_core/db/stores/test_external_user_claims.py+++ b/core/tests/switch_core/db/stores/test_external_user_claims.py@@ -28,7 +28,6 @@ async def _make_client(session: AsyncSession, *, client_type: str) -> str:matrix_user_id=f"@{client_type}-{uuid.uuid4().hex[:8]}:test",display_name=f"{client_type} client",type=client_type,- password="x",)session.add(client)await session.flush()diff --git a/core/tests/switch_core/db/stores/test_message_store.py b/core/tests/switch_core/db/stores/test_message_store.pyindex 3e1507b35..d284e6349 100644--- a/core/tests/switch_core/db/stores/test_message_store.py+++ b/core/tests/switch_core/db/stores/test_message_store.py@@ -36,7 +36,6 @@ async def _make_client(session: AsyncSession, name: str) -> Client:matrix_user_id=f"@{name}-{uuid.uuid4().hex[:8]}:test",display_name=name,type="agent",- password="x",)session.add(client)await session.flush()@@ -47,7 +46,7 @@ def _message(room_id: str, event_id: str, **overrides: object) -> Message:fields: dict[str, object] = {"room_id": room_id,"transport_event_id": event_id,- "sender_matrix_id": "@sender:test",+ "sender_id": "@sender:test","event_type": "m.room.message","msgtype": "m.text","body": f"body of {event_id}",@@ -455,4 +454,4 @@ class TestForeignKeyBehaviour:assert found is not Noneassert found.sender_client_id is None- assert found.sender_matrix_id == "@sender:test"+ assert found.sender_id == "@sender:test"diff --git a/core/tests/switch_core/db/stores/test_room_alias_store.py b/core/tests/switch_core/db/stores/test_room_alias_store.pyindex e2896c4d8..07dd993cb 100644--- a/core/tests/switch_core/db/stores/test_room_alias_store.py+++ b/core/tests/switch_core/db/stores/test_room_alias_store.py@@ -22,7 +22,6 @@ async def _make_agent(session: AsyncSession, name: str) -> Agent:matrix_user_id=f"@{name}:test",display_name=name,type="agent",- password="x",)session.add_all([api_key, client])await session.flush()diff --git a/core/tests/switch_core/db/stores/test_room_archive.py b/core/tests/switch_core/db/stores/test_room_archive.pyindex 4a4b1dd95..fc4537de6 100644--- a/core/tests/switch_core/db/stores/test_room_archive.py+++ b/core/tests/switch_core/db/stores/test_room_archive.py@@ -31,7 +31,6 @@ async def _make_agent(session: AsyncSession, name: str) -> Agent:matrix_user_id=f"@{name}:test",display_name=name,type="agent",- password="x",)session.add_all([api_key, client])await session.flush()diff --git a/core/tests/switch_core/db/stores/test_room_external_channel.py b/core/tests/switch_core/db/stores/test_room_external_channel.pyindex 94d8663a7..7ede9c36d 100644--- a/core/tests/switch_core/db/stores/test_room_external_channel.py+++ b/core/tests/switch_core/db/stores/test_room_external_channel.py@@ -26,7 +26,6 @@ async def _make_bridge(session: AsyncSession, name: str) -> CollaborationBridge:matrix_user_id=f"@bridge-{uuid.uuid4().hex[:8]}:test",display_name=f"{name} client",type="bridge",- password="x",)session.add(client)await session.flush()diff --git a/core/tests/switch_core/db/stores/test_room_join_events_store.py b/core/tests/switch_core/db/stores/test_room_join_events_store.pyindex 5d9fa1027..2dd5309aa 100644--- a/core/tests/switch_core/db/stores/test_room_join_events_store.py+++ b/core/tests/switch_core/db/stores/test_room_join_events_store.py@@ -29,7 +29,6 @@ async def _make_agent(session: AsyncSession, name: str) -> Agent:matrix_user_id=f"@{name}:test",display_name=name,type="agent",- password="x",)session.add_all([api_key, client])await session.flush()diff --git a/core/tests/switch_core/db/stores/test_room_membership_lookup.py b/core/tests/switch_core/db/stores/test_room_membership_lookup.pyindex 0050f0c51..3be1bb165 100644--- a/core/tests/switch_core/db/stores/test_room_membership_lookup.py+++ b/core/tests/switch_core/db/stores/test_room_membership_lookup.py@@ -21,7 +21,9 @@ async def _make_agent(session: AsyncSession, name: str) -> Agent:type="agent",)client = Client(- matrix_user_id=f"@{name}:test", display_name=name, type="agent", password="x"+ matrix_user_id=f"@{name}:test",+ display_name=name,+ type="agent",)session.add_all([api_key, client])await session.flush()diff --git a/core/tests/switch_core/db/stores/test_room_membership_write.py b/core/tests/switch_core/db/stores/test_room_membership_write.pyindex 669edf94e..9693d2a3f 100644--- a/core/tests/switch_core/db/stores/test_room_membership_write.py+++ b/core/tests/switch_core/db/stores/test_room_membership_write.py@@ -24,7 +24,6 @@ async def _client_and_room(session: AsyncSession) -> tuple[str, str]:matrix_user_id=f"@someone-{uuid.uuid4().hex[:8]}:test",display_name="someone",type="agent",- password="x",)room = Room(matrix_room_id=f"!{uuid.uuid4().hex[:8]}:test",diff --git a/core/tests/switch_core/db/stores/test_room_role_store.py b/core/tests/switch_core/db/stores/test_room_role_store.pyindex 01fd00d61..f1a0421b3 100644--- a/core/tests/switch_core/db/stores/test_room_role_store.py+++ b/core/tests/switch_core/db/stores/test_room_role_store.py@@ -25,7 +25,6 @@ async def _make_agent(session: AsyncSession, name: str) -> Agent:matrix_user_id=f"@{name}:test",display_name=name,type="agent",- password="x",)session.add_all([api_key, client])await session.flush()diff --git a/core/tests/switch_core/gateway/agent_route_harness.py b/core/tests/switch_core/gateway/agent_route_harness.pyindex 89f20d95a..f39af1c7a 100644--- a/core/tests/switch_core/gateway/agent_route_harness.py+++ b/core/tests/switch_core/gateway/agent_route_harness.py@@ -30,7 +30,6 @@ async def add_agent(type="agent",matrix_user_id=f"@{name}:test",display_name=name,- password=f"pw-{name}",)session.add(client)await session.flush()diff --git a/core/tests/switch_core/gateway/test_collaborations_authz.py b/core/tests/switch_core/gateway/test_collaborations_authz.pyindex 5ac9704e4..f71fb8186 100644--- a/core/tests/switch_core/gateway/test_collaborations_authz.py+++ b/core/tests/switch_core/gateway/test_collaborations_authz.py@@ -269,7 +269,6 @@ async def _make_bridge(session: AsyncSession, *, is_default: bool = False) -> stmatrix_user_id=f"@bridge-{uuid.uuid4().hex[:12]}:test",display_name="bridge client",type="bridge",- password="x",)session.add(client)await session.flush()diff --git a/core/tests/switch_core/gateway/test_oidc_config.py b/core/tests/switch_core/gateway/test_oidc_config.pyindex b2ad6ed65..62bba5266 100644--- a/core/tests/switch_core/gateway/test_oidc_config.py+++ b/core/tests/switch_core/gateway/test_oidc_config.py@@ -19,11 +19,7 @@ def _kwargs(**overrides: object) -> dict[str, object]:db_user="u",db_password="p",db_name="d",- matrix_server="http://m",matrix_server_name="m",- matrix_admin_user="a",- matrix_admin_password="p",- matrix_registration_shared_secret="s",agent_registration_token="t",jwt_secret_key="secret",gateway_admin_email="[email protected]",diff --git a/core/tests/switch_core/gateway/test_oidc_routes.py b/core/tests/switch_core/gateway/test_oidc_routes.pyindex 84a4ed8ab..f2a340eda 100644--- a/core/tests/switch_core/gateway/test_oidc_routes.py+++ b/core/tests/switch_core/gateway/test_oidc_routes.py@@ -20,11 +20,7 @@ def _config(**overrides: object) -> SwitchConfig:db_user="u",db_password="p",db_name="d",- matrix_server="http://m",matrix_server_name="m",- matrix_admin_user="a",- matrix_admin_password="p",- matrix_registration_shared_secret="s",agent_registration_token="t",jwt_secret_key="secret",gateway_admin_email="[email protected]",diff --git a/core/tests/switch_core/messages/test_backfill.py b/core/tests/switch_core/messages/test_backfill.pydeleted file mode 100644index c74e087ad..000000000--- a/core/tests/switch_core/messages/test_backfill.py+++ /dev/null@@ -1,588 +0,0 @@-"""Reconstructed history lands in the right order, below the live log.--The two properties that matter most are the ones a careless implementation-gets wrong in ways nobody notices for weeks: the numbering must run the same-way time does, and running the walk twice must not double the room.-"""--from __future__ import annotations--import uuid-from datetime import UTC, datetime, timedelta--from sqlalchemy import select-from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker--from switch_core.cli.backfill import _ReadOnlyStore-from switch_core.db.models import Message, Room-from switch_core.db.stores.message_store import MessageStore-from switch_core.messages import backfill_room-from switch_core.transport import (- HistoryPage,- InboundCustomEvent,- InboundMedia,- InboundMembership,- InboundMessage,-)--BASE = datetime(2026, 1, 1, 12, 0, tzinfo=UTC)---class PagingTransport:- """Replays fixed pages of backwards history, newest first."""-- def __init__(self, pages: list[HistoryPage]) -> None:- self._pages = pages- self.reads = 0-- async def read_history(- self, room_id: str, *, start: str | None, limit: int- ) -> HistoryPage:- page = self._pages[min(self.reads, len(self._pages) - 1)]- self.reads += 1- return page---def _at(seconds: int) -> datetime:- return BASE + timedelta(seconds=seconds)---def _ms(seconds: int) -> int:- return int(_at(seconds).timestamp() * 1000)---def _event(event_id: str, at: int, body: str = "hello", **overrides: object): # type: ignore[no-untyped-def]- fields: dict[str, object] = {- "room_id": "!room:test",- "event_id": event_id,- "sender": "@a:test",- "timestamp": _ms(at),- "content": {"body": body, "sender_name": "Alice"},- "body": body,- "msgtype": "m.text",- }- fields.update(overrides)- return InboundMessage(**fields) # type: ignore[arg-type]---async def _make_room(session: AsyncSession) -> Room:- room = Room(- matrix_room_id=f"!room-{uuid.uuid4().hex[:8]}:test",- name="a room",- description="",- )- session.add(room)- await session.commit()- return room---async def _rows(- session_factory: async_sessionmaker[AsyncSession], room_id: str-) -> list[Message]:- async with session_factory() as session:- result = await session.execute(- select(Message).where(Message.room_id == room_id).order_by(Message.seq)- )- return list(result.scalars().all())---class TestNumbering:- async def test_history_is_numbered_below_the_live_log(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """A message from before recording must not outrank one from after it.-- Ordering is by `seq`, so a positive number on reconstructed history- would put last month at the top of every read.- """- store = MessageStore()- async with session_factory() as session:- room = await _make_room(session)- live = Message(- room_id=room.id,- transport_event_id="$live",- sender_matrix_id="@a:test",- event_type="m.room.message",- msgtype="m.text",- body="today",- content={},- sent_at=_at(100),- )- await store.create(session, live, [])- await session.commit()-- transport = PagingTransport(- [HistoryPage(events=[_event("$old", 10, "yesterday")], next_token=None)]- )- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- await backfill_room(transport, session_factory, room, store=store)-- rows = await _rows(session_factory, room.id)- assert [(r.seq, r.body) for r in rows] == [(-1, "yesterday"), (1, "today")]-- async def test_the_oldest_message_gets_the_lowest_number(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """Numbering has to run the same way time does, across pages as well as- within them — a walk that numbered per page would interleave."""- store = MessageStore()- async with session_factory() as session:- room = await _make_room(session)-- transport = PagingTransport(- [- HistoryPage(- events=[_event("$d", 40, "d"), _event("$c", 30, "c")],- next_token="t1",- ),- HistoryPage(- events=[_event("$b", 20, "b"), _event("$a", 10, "a")],- next_token=None,- ),- ]- )- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- await backfill_room(transport, session_factory, room, store=store)-- rows = await _rows(session_factory, room.id)- assert [r.body for r in rows] == ["a", "b", "c", "d"]- assert [r.seq for r in rows] == [-4, -3, -2, -1]-- async def test_the_timestamp_is_when_it_was_sent_not_when_it_was_written(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """The read path filters on `sent_at`. Stamping the backfill's own- clock would pile a year of history onto tonight."""- store = MessageStore()- async with session_factory() as session:- room = await _make_room(session)-- transport = PagingTransport(- [HistoryPage(events=[_event("$a", 10)], next_token=None)]- )- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- await backfill_room(transport, session_factory, room, store=store)-- rows = await _rows(session_factory, room.id)- assert rows[0].sent_at == _at(10)---class TestCompletionMark:- """A room is marked done only by a walk that really wrote it.-- The mark is what stops a re-run re-reading every page of every room. It is- also what a dry run must not leave behind: the documented first step is a- dry run, and if that marked every room, the real run afterwards would find- nothing to do and say so as though the work were finished.- """-- async def test_a_completed_walk_marks_the_room(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- async with session_factory() as session:- room = await _make_room(session)-- transport = PagingTransport(- [HistoryPage(events=[_event("$a", 10, "a")], next_token=None)]- )- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- await backfill_room(transport, session_factory, room, store=MessageStore())-- async with session_factory() as session:- written = await session.get(Room, room.id)- assert written is not None- assert written.history_backfilled_at is not None-- async def test_a_dry_run_marks_nothing(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """Otherwise the dry run disables the real one, silently."""- async with session_factory() as session:- room = await _make_room(session)-- transport = PagingTransport(- [HistoryPage(events=[_event("$a", 10, "a")], next_token=None)]- )- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- report = await backfill_room(- transport, session_factory, room, store=_ReadOnlyStore()- )-- assert report.written == 1- async with session_factory() as session:- untouched = await session.get(Room, room.id)- assert untouched is not None- assert untouched.history_backfilled_at is None- assert await _rows(session_factory, room.id) == []---class TestIdempotence:- async def test_running_it_twice_writes_nothing_the_second_time(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """A long walk that fails halfway should be safe to just run again."""- store = MessageStore()- async with session_factory() as session:- room = await _make_room(session)-- def _transport() -> PagingTransport:- return PagingTransport(- [- HistoryPage(- events=[_event("$b", 20, "b"), _event("$a", 10, "a")],- next_token=None,- )- ]- )-- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- first = await backfill_room(- _transport(), session_factory, room, store=store- )- second = await backfill_room(- _transport(), session_factory, room, store=store- )-- assert (first.written, first.already_present) == (2, 0)- assert (second.written, second.already_present) == (0, 2)- assert len(await _rows(session_factory, room.id)) == 2-- async def test_a_message_already_recorded_live_is_left_alone(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """The walk overlaps the recorded window by design — it starts at the- newest message on the bus, not at the oldest row."""- store = MessageStore()- async with session_factory() as session:- room = await _make_room(session)- live = Message(- room_id=room.id,- transport_event_id="$b",- sender_matrix_id="@a:test",- event_type="m.room.message",- msgtype="m.text",- body="b",- content={},- sent_at=_at(20),- )- await store.create(session, live, [])- await session.commit()-- transport = PagingTransport(- [- HistoryPage(- events=[_event("$b", 20, "b"), _event("$a", 10, "a")],- next_token=None,- )- ]- )- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- report = await backfill_room(transport, session_factory, room, store=store)-- assert (report.written, report.already_present) == (1, 1)- rows = await _rows(session_factory, room.id)- assert [(r.seq, r.body) for r in rows] == [(-1, "a"), (1, "b")]---class TestWhatBelongsInTheLog:- async def test_bus_traffic_the_log_does_not_keep_is_skipped(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """The same scoping as the live recorder, or the backfill would put- back exactly what scoping the log took out."""- store = MessageStore()- async with session_factory() as session:- room = await _make_room(session)-- transport = PagingTransport(- [- HistoryPage(- events=[- _event("$msg", 20),- InboundCustomEvent(- room_id="!room:test",- event_id="$state",- sender="@a:test",- timestamp=_ms(10),- content={"state": "busy"},- event_type="com.switch.agent.runtime_state",- ),- ],- next_token=None,- )- ]- )- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- report = await backfill_room(transport, session_factory, room, store=store)-- assert report.written == 1- assert report.skipped_by_type == {"com.switch.agent.runtime_state": 1}-- async def test_an_arrival_is_reconstructed_with_no_body(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """The timeline reads wrong without arrivals, and the sentence stays- the reader's to phrase — same as the live path."""- store = MessageStore()- async with session_factory() as session:- room = await _make_room(session)-- transport = PagingTransport(- [- HistoryPage(- events=[- InboundMembership(- room_id="!room:test",- event_id="$join",- sender="@bob:test",- timestamp=_ms(10),- state_key="@bob:test",- membership="join",- display_name="Bob",- )- ],- next_token=None,- )- ]- )- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- await backfill_room(transport, session_factory, room, store=store)-- rows = await _rows(session_factory, room.id)- assert rows[0].event_type == "m.room.member"- assert rows[0].body is None- assert rows[0].sender_name == "Bob"-- async def test_a_departure_is_not_history_worth_keeping(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- store = MessageStore()- async with session_factory() as session:- room = await _make_room(session)-- transport = PagingTransport(- [- HistoryPage(- events=[- InboundMembership(- room_id="!room:test",- event_id="$leave",- sender="@bob:test",- timestamp=_ms(10),- state_key="@bob:test",- membership="leave",- )- ],- next_token=None,- )- ]- )- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- report = await backfill_room(transport, session_factory, room, store=store)-- assert report.written == 0- assert await _rows(session_factory, room.id) == []-- async def test_a_file_comes_back_with_its_attachment_row(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- store = MessageStore()- async with session_factory() as session:- room = await _make_room(session)-- transport = PagingTransport(- [- HistoryPage(- events=[- InboundMedia(- room_id="!room:test",- event_id="$img",- sender="@a:test",- timestamp=_ms(10),- content={- "info": {"mimetype": "image/png", "size": 42},- "body": "chart.png",- },- body="chart.png",- msgtype="m.image",- uri="mxc://s/abc",- )- ],- next_token=None,- )- ]- )- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- await backfill_room(transport, session_factory, room, store=store)-- rows = await _rows(session_factory, room.id)- async with session_factory() as session:- attachments = await store.attachments_for(session, [rows[0].id])- [attachment] = attachments[rows[0].id]- assert (attachment.uri, attachment.filename) == ("mxc://s/abc", "chart.png")- assert (attachment.mimetype, attachment.size) == ("image/png", 42)---class TestStopping:- async def test_a_cursor_that_does_not_advance_stops_the_walk(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """A homeserver that keeps returning the token it was given would- otherwise walk forever."""- store = MessageStore()- async with session_factory() as session:- room = await _make_room(session)-- transport = PagingTransport(- [HistoryPage(events=[_event("$a", 10)], next_token="stuck")]- )- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- report = await backfill_room(transport, session_factory, room, store=store)-- assert transport.reads == 2- assert report.written == 1---class TestPerRoomLimit:- """A room's history is carried across newest-first, up to a limit.-- The limit is the whole point of the release it shipped in: an unbounded- walk over a room with years in it is the difference between a migration an- operator runs and one they abandon. What it leaves behind is left behind- for good, because the homeserver goes away afterwards.- """-- async def test_it_keeps_the_newest_and_leaves_the_rest(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- async with session_factory() as session:- room = await _make_room(session)-- transport = PagingTransport(- [- HistoryPage(- events=[_event(f"$e{i}", 100 - i, f"m{i}") for i in range(5)],- next_token=None,- )- ]- )- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- report = await backfill_room(- transport,- session_factory,- room,- store=MessageStore(),- messages_per_room=2,- )-- assert report.written == 2- assert report.capped is True- rows = await _rows(session_factory, room.id)- assert [r.body for r in rows] == ["m1", "m0"]-- async def test_a_capped_room_is_finished_not_abandoned(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """Marked, or every start walks it again — and `incomplete` stays- false, because the command exits non-zero on that and Console fails an- upgrade on a non-zero exit."""- async with session_factory() as session:- room = await _make_room(session)-- transport = PagingTransport(- [- HistoryPage(- events=[_event(f"$e{i}", 100 - i, f"m{i}") for i in range(4)],- next_token=None,- )- ]- )- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- report = await backfill_room(- transport,- session_factory,- room,- store=MessageStore(),- messages_per_room=1,- )-- assert report.incomplete is False- async with session_factory() as session:- marked = await session.get(Room, room.id)- assert marked is not None- assert marked.history_backfilled_at is not None-- async def test_a_second_run_does_not_carry_another_batch(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """The cap counts what the log holds, not what this run wrote. Counting- only writes would make each re-run reach further back."""- async with session_factory() as session:- room = await _make_room(session)-- pages = [- HistoryPage(- events=[_event(f"$e{i}", 100 - i, f"m{i}") for i in range(5)],- next_token=None,- )- ]- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- await backfill_room(- PagingTransport(list(pages)),- session_factory,- room,- store=MessageStore(),- messages_per_room=2,- )- second = await backfill_room(- PagingTransport(list(pages)),- session_factory,- room,- store=MessageStore(),- messages_per_room=2,- )-- assert second.written == 0- assert len(await _rows(session_factory, room.id)) == 2-- async def test_zero_walks_the_room_to_its_start(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- async with session_factory() as session:- room = await _make_room(session)-- transport = PagingTransport(- [- HistoryPage(- events=[_event(f"$e{i}", 100 - i, f"m{i}") for i in range(5)],- next_token=None,- )- ]- )- async with session_factory() as session:- room = await session.get(Room, room.id) # type: ignore[assignment]- report = await backfill_room(- transport,- session_factory,- room,- store=MessageStore(),- messages_per_room=0,- )-- assert report.written == 5- assert report.capped is Falsediff --git a/core/tests/switch_core/messages/test_message_recorder.py b/core/tests/switch_core/messages/test_message_recorder.pydeleted file mode 100644index 82027dd5d..000000000--- a/core/tests/switch_core/messages/test_message_recorder.py+++ /dev/null@@ -1,623 +0,0 @@-"""What the recorder writes for a send, and what it does when it cannot.--The recorder reads the content dict the transport reports back rather than-being told separately what was sent, so these tests feed it the shapes the-Matrix transport really produces — a plain message, a markdown one, a threaded-reply, a media event, a custom `com.switch.*` event — and check what lands.--The failure tests matter as much as the success ones. Recording runs after-delivery and must never turn a delivered message into a raised exception at-the caller.-"""--from __future__ import annotations--import logging-import uuid--import pytest-from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker--from switch_core.db.models import Client, Room-from switch_core.db.stores.message_store import MessageStore-from switch_core.db.stores.room_store import RoomStore-from switch_core.messages import MessageRecorder-from switch_core.transport import InboundMembership, SendResult---async def _make_room(session: AsyncSession) -> tuple[str, str, str]:- """Insert the Room and Client a recorded message depends on.-- Returns (room uuid, matrix room id, client id).- """- suffix = uuid.uuid4().hex[:8]- client = Client(- matrix_user_id=f"@agent-{suffix}:test",- display_name="agent one",- type="agent",- password="x",- )- session.add(client)- await session.flush()- room = Room(- matrix_room_id=f"!room-{suffix}:test",- name="a room",- description="",- )- session.add(room)- await session.flush()- return room.id, room.matrix_room_id, client.id---def _recorder(session_factory: async_sessionmaker[AsyncSession]) -> MessageRecorder:- return MessageRecorder(- session_factory=session_factory,- room_store=RoomStore(),- message_store=MessageStore(),- )---async def _record(- session_factory: async_sessionmaker[AsyncSession],- matrix_room_id: str,- client_id: str,- result: SendResult,-) -> None:- await _recorder(session_factory).record(- transport_room_id=matrix_room_id,- result=result,- sender_matrix_id="@agent:test",- sender_client_id=client_id,- sender_name="agent one",- )---class TestRecordsWhatWasSent:- async def test_a_plain_message(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- async with session_factory() as session:- room_id, matrix_room_id, client_id = await _make_room(session)- await session.commit()-- await _record(- session_factory,- matrix_room_id,- client_id,- SendResult(- event_id="$evt1",- event_type="m.room.message",- content={- "msgtype": "m.text",- "body": "hello",- "sender_name": "agent one",- },- ),- )-- async with session_factory() as session:- message = await MessageStore().get_by_transport_event_id(session, "$evt1")-- assert message is not None- assert message.room_id == room_id- assert message.body == "hello"- assert message.msgtype == "m.text"- assert message.event_type == "m.room.message"- assert message.sender_client_id == client_id- assert message.sender_matrix_id == "@agent:test"- assert message.thread_root_event_id is None- # The whole event body is kept, not just the columns beside it.- assert message.content["sender_name"] == "agent one"-- async def test_a_markdown_message_keeps_the_rendered_html(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- async with session_factory() as session:- _, matrix_room_id, client_id = await _make_room(session)- await session.commit()-- await _record(- session_factory,- matrix_room_id,- client_id,- SendResult(- event_id="$evt2",- event_type="m.room.message",- content={- "msgtype": "m.text",- "body": "**bold**",- "format": "org.matrix.custom.html",- "formatted_body": "<p><strong>bold</strong></p>",- },- ),- )-- async with session_factory() as session:- message = await MessageStore().get_by_transport_event_id(session, "$evt2")-- assert message is not None- assert message.formatted_body == "<p><strong>bold</strong></p>"-- async def test_a_threaded_reply_records_its_root(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- async with session_factory() as session:- _, matrix_room_id, client_id = await _make_room(session)- await session.commit()-- await _record(- session_factory,- matrix_room_id,- client_id,- SendResult(- event_id="$evt3",- event_type="m.room.message",- content={- "msgtype": "m.text",- "body": "in thread",- "m.relates_to": {- "rel_type": "m.thread",- "event_id": "$root",- },- },- ),- )-- async with session_factory() as session:- message = await MessageStore().get_by_transport_event_id(session, "$evt3")-- assert message is not None- assert message.thread_root_event_id == "$root"-- async def test_a_non_thread_relation_is_not_read_as_a_thread(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """An edit or a reaction relates to an event without being in a thread."""- async with session_factory() as session:- _, matrix_room_id, client_id = await _make_room(session)- await session.commit()-- await _record(- session_factory,- matrix_room_id,- client_id,- SendResult(- event_id="$evt4",- event_type="m.room.message",- content={- "msgtype": "m.text",- "body": "an edit",- "m.relates_to": {- "rel_type": "m.replace",- "event_id": "$original",- },- },- ),- )-- async with session_factory() as session:- message = await MessageStore().get_by_transport_event_id(session, "$evt4")-- assert message is not None- assert message.thread_root_event_id is None-- async def test_a_media_event_records_its_file(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- async with session_factory() as session:- _, matrix_room_id, client_id = await _make_room(session)- await session.commit()-- await _record(- session_factory,- matrix_room_id,- client_id,- SendResult(- event_id="$evt5",- event_type="m.room.message",- content={- "msgtype": "m.image",- "body": "a caption",- "filename": "chart.png",- "url": "mxc://test/abc",- "info": {"mimetype": "image/png", "size": 4096},- },- ),- )-- async with session_factory() as session:- store = MessageStore()- message = await store.get_by_transport_event_id(session, "$evt5")- assert message is not None- attachments = await store.get_attachments(session, message.id)-- assert len(attachments) == 1- assert attachments[0].uri == "mxc://test/abc"- # The caption is the body; the real filename is carried separately.- assert attachments[0].filename == "chart.png"- assert attachments[0].mimetype == "image/png"- assert attachments[0].size == 4096-- async def test_an_uncaptioned_file_falls_back_to_the_body_for_its_name(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """Without a caption the transport puts the filename in `body` and- sends no `filename` field at all."""- async with session_factory() as session:- _, matrix_room_id, client_id = await _make_room(session)- await session.commit()-- await _record(- session_factory,- matrix_room_id,- client_id,- SendResult(- event_id="$evt6",- event_type="m.room.message",- content={- "msgtype": "m.file",- "body": "report.csv",- "url": "mxc://test/def",- "info": {"mimetype": "text/csv", "size": 12},- },- ),- )-- async with session_factory() as session:- store = MessageStore()- message = await store.get_by_transport_event_id(session, "$evt6")- assert message is not None- attachments = await store.get_attachments(session, message.id)-- assert [a.filename for a in attachments] == ["report.csv"]-- async def test_a_custom_event_records_its_whole_content(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """A `com.switch.*` event has no body or msgtype — the payload is all- there is, so losing it would lose the event."""- async with session_factory() as session:- _, matrix_room_id, client_id = await _make_room(session)- await session.commit()-- await _record(- session_factory,- matrix_room_id,- client_id,- SendResult(- event_id="$evt7",- event_type="com.switch.command",- content={"command": "reset", "args": ["--hard"], "request_id": "r1"},- ),- )-- async with session_factory() as session:- store = MessageStore()- message = await store.get_by_transport_event_id(session, "$evt7")- assert message is not None- attachments = await store.get_attachments(session, message.id)-- assert message.event_type == "com.switch.command"- assert message.body is None- assert message.msgtype is None- assert message.content == {- "command": "reset",- "args": ["--hard"],- "request_id": "r1",- }- assert attachments == []-- async def test_bus_traffic_is_not_recorded_at_all(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """The log is the conversation. A runtime-state ping is not part of it,- and it is the highest-volume thing on the bus."""- async with session_factory() as session:- _, matrix_room_id, client_id = await _make_room(session)- await session.commit()-- await _record(- session_factory,- matrix_room_id,- client_id,- SendResult(- event_id="$evt8",- event_type="com.switch.agent.runtime_state",- content={"state": "busy"},- ),- )-- async with session_factory() as session:- found = await MessageStore().get_by_transport_event_id(session, "$evt8")-- assert found is None---class TestFailingToRecordNeverReachesTheCaller:- async def test_an_unknown_room_is_warned_about_not_raised(- self,- session_factory: async_sessionmaker[AsyncSession],- caplog: pytest.LogCaptureFixture,- ) -> None:- """Not every room a client sends to is a Switch room."""- with caplog.at_level(logging.WARNING):- await _record(- session_factory,- "!nobody-knows-this:test",- None, # type: ignore[arg-type]- SendResult(- event_id="$evt8",- event_type="m.room.message",- content={"body": "hi"},- ),- )-- assert "not a Switch room" in caplog.text-- async with session_factory() as session:- assert (- await MessageStore().get_by_transport_event_id(session, "$evt8") is None- )-- async def test_a_database_failure_is_logged_at_error_and_swallowed(- self,- session_factory: async_sessionmaker[AsyncSession],- caplog: pytest.LogCaptureFixture,- ) -> None:- """The message was delivered. Raising here would tell the caller a- successful send failed, which is worse than a missing row."""- async with session_factory() as session:- _, matrix_room_id, client_id = await _make_room(session)- await session.commit()-- recorder = _recorder(session_factory)- result = SendResult(- event_id="$evt9",- event_type="m.room.message",- content={"msgtype": "m.text", "body": "once"},- )- await recorder.record(- transport_room_id=matrix_room_id,- result=result,- sender_matrix_id="@agent:test",- sender_client_id=client_id,- sender_name="agent one",- )-- # The same event id a second time violates the unique constraint, which- # is a stand-in for any database failure after a successful send.- with caplog.at_level(logging.ERROR):- await recorder.record(- transport_room_id=matrix_room_id,- result=result,- sender_matrix_id="@agent:test",- sender_client_id=client_id,- sender_name="agent one",- )-- assert "failed to record" in caplog.text- assert "$evt9" in caplog.text---class TestRoomResolution:- async def test_the_room_lookup_is_cached_across_sends(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """A send is on the hot path, so the room id is resolved once."""- async with session_factory() as session:- room_id, matrix_room_id, client_id = await _make_room(session)- await session.commit()-- lookups = 0- real_lookup = RoomStore.get_by_matrix_room_id-- async def counting_lookup(- self: RoomStore, session: AsyncSession, matrix_room_id: str- ) -> Room | None:- nonlocal lookups- lookups += 1- return await real_lookup(self, session, matrix_room_id)-- room_store = RoomStore()- room_store.get_by_matrix_room_id = counting_lookup.__get__(room_store) # type: ignore[method-assign]- recorder = MessageRecorder(- session_factory=session_factory,- room_store=room_store,- message_store=MessageStore(),- )-- for n in range(3):- await recorder.record(- transport_room_id=matrix_room_id,- result=SendResult(- event_id=f"$cached{n}",- event_type="m.room.message",- content={"body": "x"},- ),- sender_matrix_id="@agent:test",- sender_client_id=client_id,- sender_name="agent one",- )-- assert lookups == 1-- async with session_factory() as session:- for n in range(3):- message = await MessageStore().get_by_transport_event_id(- session, f"$cached{n}"- )- assert message is not None- assert message.room_id == room_id---class TestRecordsArrivals:- """A join is not a send, so it comes in by its own door.-- It is recorded by the client that arrived, about itself. Exactly one Switch- client owns each participant, so that gives one row per arrival for the- same reason recording sends does.- """-- async def test_an_arrival_is_recorded(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- async with session_factory() as session:- room_id, matrix_room_id, client_id = await _make_room(session)- await session.commit()-- await _recorder(session_factory).record_join(- transport_room_id=matrix_room_id,- event=InboundMembership(- room_id=matrix_room_id,- event_id="$join1",- sender="@agent:test",- timestamp=1000,- content={"membership": "join", "displayname": "agent one"},- state_key="@agent:test",- membership="join",- prev_membership="invite",- display_name="agent one",- ),- client_id=client_id,- member_name="agent one",- )-- async with session_factory() as session:- message = await MessageStore().get_by_transport_event_id(session, "$join1")- assert message is not None- assert message.room_id == room_id- assert message.event_type == "m.room.member"- assert message.sender_matrix_id == "@agent:test"- assert message.sender_name == "agent one"- assert message.content["membership"] == "join"-- async def test_the_arrival_is_named_the_way_the_member_speaks(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """The membership event's display name comes from the profile, which is- set from whatever the source platform calls the member. Recording it- would make one participant read two ways in the same history."""- async with session_factory() as session:- _, matrix_room_id, client_id = await _make_room(session)- await session.commit()-- await _recorder(session_factory).record_join(- transport_room_id=matrix_room_id,- event=InboundMembership(- room_id=matrix_room_id,- event_id="$join5",- sender="@agent:test",- timestamp=1000,- state_key="@agent:test",- membership="join",- display_name="agent one on the platform",- ),- client_id=client_id,- member_name="agent one",- )-- async with session_factory() as session:- message = await MessageStore().get_by_transport_event_id(session, "$join5")- assert message is not None- assert message.sender_name == "agent one"-- async def test_the_row_carries_no_rendered_sentence(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """How an arrival reads is the reader's to phrase. Writing "x joined- the room" into the log would freeze today's wording into every row."""- async with session_factory() as session:- _, matrix_room_id, client_id = await _make_room(session)- await session.commit()-- await _recorder(session_factory).record_join(- transport_room_id=matrix_room_id,- event=InboundMembership(- room_id=matrix_room_id,- event_id="$join2",- sender="@agent:test",- timestamp=1000,- state_key="@agent:test",- membership="join",- display_name="agent one",- ),- client_id=client_id,- member_name="agent one",- )-- async with session_factory() as session:- message = await MessageStore().get_by_transport_event_id(session, "$join2")- assert message is not None- assert message.body is None- assert message.msgtype is None-- async def test_seeing_the_same_join_twice_records_it_once(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """A redelivered sync is not a second arrival."""- async with session_factory() as session:- room_id, matrix_room_id, client_id = await _make_room(session)- await session.commit()-- event = InboundMembership(- room_id=matrix_room_id,- event_id="$join3",- sender="@agent:test",- timestamp=1000,- state_key="@agent:test",- membership="join",- display_name="agent one",- )- recorder = _recorder(session_factory)- await recorder.record_join(- transport_room_id=matrix_room_id,- event=event,- client_id=client_id,- member_name="agent one",- )- await recorder.record_join(- transport_room_id=matrix_room_id,- event=event,- client_id=client_id,- member_name="agent one",- )-- async with session_factory() as session:- rows = await MessageStore().list_for_room(- session, room_id, after_seq=0, limit=10- )- assert [row.transport_event_id for row in rows] == ["$join3"]-- async def test_a_failure_to_record_does_not_escape(- self,- session_factory: async_sessionmaker[AsyncSession],- caplog: pytest.LogCaptureFixture,- ) -> None:- """Nothing about being in a room depends on the row having landed."""- async with session_factory() as session:- _, matrix_room_id, client_id = await _make_room(session)- await session.commit()-- recorder = _recorder(session_factory)-- async def boom(*args: object, **kwargs: object) -> None:- raise RuntimeError("database is unhappy")-- recorder._message_store.create = boom # type: ignore[method-assign]-- with caplog.at_level(logging.ERROR):- await recorder.record_join(- transport_room_id=matrix_room_id,- event=InboundMembership(- room_id=matrix_room_id,- event_id="$join4",- sender="@agent:test",- timestamp=1000,- state_key="@agent:test",- membership="join",- ),- client_id=client_id,- member_name="agent one",- )-- assert "failed to record the arrival" in caplog.textdiff --git a/core/tests/switch_core/messages/test_notify.py b/core/tests/switch_core/messages/test_notify.pyindex 4f3fde479..3687a2de7 100644--- a/core/tests/switch_core/messages/test_notify.py+++ b/core/tests/switch_core/messages/test_notify.py@@ -74,7 +74,7 @@ async def _write(session: AsyncSession, room_id: str, seq: int) -> None:seq=seq,room_id=room_id,transport_event_id=f"$e{uuid.uuid4().hex[:8]}",- sender_matrix_id="@a:test",+ sender_id="@a:test",event_type="m.room.message",msgtype="m.text",body="hello",diff --git a/core/tests/switch_core/messages/test_reconcile.py b/core/tests/switch_core/messages/test_reconcile.pydeleted file mode 100644index a93a8c46b..000000000--- a/core/tests/switch_core/messages/test_reconcile.py+++ /dev/null@@ -1,792 +0,0 @@-"""Reconciliation reports the differences it is supposed to, and no others.--The whole value of this check is that its verdict can be trusted. So the tests-that matter most are the ones proving it does not cry drift over things that-are legitimately different — history predating the recorder, membership-changes that were never arrivals, arrivals predating the path that records-them — and the one proving it does not report a room it could not fully read-as clean.-"""--from __future__ import annotations--import uuid-from datetime import UTC, datetime, timedelta-from itertools import count--from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker--from switch_core.db.models import Message, Room-from switch_core.messages import reconcile_room-from switch_core.transport import (- HistoryPage,- InboundCustomEvent,- InboundMedia,- InboundMembership,- InboundMessage,-)---class PagingTransport:- """Replays fixed pages of backwards history, newest first."""-- def __init__(self, pages: list[HistoryPage]) -> None:- self._pages = pages- self.reads = 0-- async def read_history(- self, room_id: str, *, start: str | None, limit: int- ) -> HistoryPage:- page = self._pages[self.reads]- self.reads += 1- return page---def _ms(moment: datetime) -> int:- return int(moment.timestamp() * 1000)---async def _make_room(session: AsyncSession) -> Room:- room = Room(- matrix_room_id=f"!room-{uuid.uuid4().hex[:8]}:test",- name="a room",- description="",- )- session.add(room)- await session.flush()- return room---_seq = count(1)---def _row(- room_id: str, event_id: str, sent_at: datetime, **overrides: object-) -> Message:- """`seq` is assigned by MessageStore.create; these rows bypass it, so the- helper numbers them. Reconciliation matches on event id and never reads- `seq`, so only distinctness matters here."""- fields: dict[str, object] = {- "seq": next(_seq),- "room_id": room_id,- "transport_event_id": event_id,- "sender_matrix_id": "@agent:test",- "sender_name": "agent one",- "event_type": "m.room.message",- "msgtype": "m.text",- "body": "hello",- "content": {"msgtype": "m.text", "body": "hello"},- "sent_at": sent_at,- }- fields.update(overrides)- return Message(**fields)---def _join_row(room_id: str, event_id: str, sent_at: datetime) -> Message:- return _row(- room_id,- event_id,- sent_at,- event_type="m.room.member",- msgtype=None,- body=None,- content={"membership": "join"},- )---def _join_event(event_id: str, at: datetime, **overrides: object) -> InboundMembership:- fields: dict[str, object] = {- "room_id": "!room:test",- "event_id": event_id,- "sender": "@agent:test",- "timestamp": _ms(at),- "content": {"membership": "join"},- "state_key": "@agent:test",- "membership": "join",- "prev_membership": "invite",- }- fields.update(overrides)- return InboundMembership(**fields) # type: ignore[arg-type]---def _event(event_id: str, at: datetime, **overrides: object) -> InboundMessage:- fields: dict[str, object] = {- "room_id": "!room:test",- "event_id": event_id,- "sender": "@agent:test",- "timestamp": _ms(at),- "content": {},- "body": "hello",- "msgtype": "m.text",- }- fields.update(overrides)- return InboundMessage(**fields) # type: ignore[arg-type]---class TestAgreement:- async def test_a_room_whose_records_match_is_clean(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$a", now))- session.add(_row(room.id, "$b", now + timedelta(seconds=1)))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [- HistoryPage(- events=[- _event("$b", now + timedelta(seconds=1)),- _event("$a", now),- ],- next_token=None,- )- ]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert report.clean- assert report.compared == 2- assert report.agreed == 2-- async def test_history_older_than_the_recorder_is_not_reported_missing(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """The room existed before recording started. Reporting its whole- back-catalogue as drift would bury the row that really went astray."""- now = datetime.now(UTC)- recording_began = now - timedelta(minutes=5)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$new", recording_began))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [- HistoryPage(- events=[- _event("$new", recording_began),- _event("$ancient", now - timedelta(days=30)),- _event("$older-still", now - timedelta(days=60)),- ],- next_token=None,- )- ]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert report.clean- assert report.compared == 1- assert report.missing_rows == []-- async def test_events_that_were_never_sends_are_counted_not_flagged(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$a", now))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [- HistoryPage(- events=[- InboundMembership(- room_id="!room:test",- event_id="$join",- sender="@someone:test",- timestamp=_ms(now + timedelta(seconds=1)),- state_key="@someone:test",- membership="join",- ),- _event("$a", now),- ],- next_token=None,- )- ]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert report.clean- assert report.ignored_by_type == {- "m.room.member (before arrivals were recorded)": 1- }-- async def test_a_room_with_nothing_recorded_reports_that_plainly(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """Not the same as agreeing, and not the same as total drift."""- async with session_factory() as session:- room = await _make_room(session)- await session.commit()- room_id = room.id-- transport = PagingTransport([])-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert report.compared_nothing- assert transport.reads == 0- assert "nothing to compare" in report.summary()---class TestTheWindowBoundary:- """The oldest recorded message is the one most likely to be misjudged.-- Its row is written after its send is accepted, so the row's `sent_at` is- always a little later than the event's own timestamp. A window that began- at the row would exclude the event that produced it, and every room would- report its first message as drift for as long as it existed.- """-- async def test_the_first_recorded_message_is_not_reported_as_drift(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- sent = datetime.now(UTC)- recorded_at = sent + timedelta(milliseconds=40)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$first", recorded_at))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [HistoryPage(events=[_event("$first", sent)], next_token=None)]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert report.clean- assert report.agreed == 1- assert report.anchored-- async def test_history_before_the_anchor_is_still_left_alone(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """Admitting the anchor from before the window must not admit its- neighbours: the walk stops at it, it does not step over it."""- sent = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$first", sent + timedelta(milliseconds=40)))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [- HistoryPage(- events=[- _event("$first", sent),- _event("$older", sent - timedelta(seconds=1)),- ],- next_token=None,- )- ]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert report.clean- assert report.missing_rows == []-- async def test_an_anchor_the_bus_has_lost_is_declared_not_hidden(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$gone", now))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [HistoryPage(events=[_event("$other", now)], next_token=None)]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert not report.anchored- assert "WINDOW UNANCHORED" in report.summary()---class TestArrivalsAndRowsWithNothingToCompareAgainst:- """Arrivals are compared from the point the log holds one; the categories- the bus walk still discards have no counterpart by construction, and- calling those "recorded but never sent" would report the same rows as- drift on every run."""-- async def test_an_arrival_the_bus_agrees_with_is_checked_not_skipped(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """Recording arrivals is the newest write path there is, so leaving it- out of the check made the least proven part the unchecked part."""- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$a", now))- session.add(_join_row(room.id, "$join", now + timedelta(seconds=2)))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [- HistoryPage(- events=[- _join_event("$join", now + timedelta(seconds=1)),- _event("$a", now),- ],- next_token=None,- )- ]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert report.clean- assert report.compared == 2- assert report.agreed == 2- assert report.unverifiable_by_type == {}-- async def test_an_arrival_the_log_missed_is_drift(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$a", now))- session.add(_join_row(room.id, "$join", now + timedelta(seconds=2)))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [- HistoryPage(- events=[- _join_event("$later", now + timedelta(seconds=3)),- _join_event("$join", now + timedelta(seconds=1)),- _event("$a", now),- ],- next_token=None,- )- ]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert report.missing_rows == ["$later"]-- async def test_an_arrival_older_than_the_log_is_counted_not_reported(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """Every room was joined long before arrivals were recorded, so a join- from before the log's first one cannot have a row."""- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$a", now))- session.add(_join_row(room.id, "$join", now + timedelta(seconds=3)))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [- HistoryPage(- events=[- _join_event("$join", now + timedelta(seconds=2)),- _join_event("$ancient", now + timedelta(seconds=1)),- _event("$a", now),- ],- next_token=None,- )- ]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert report.clean- assert report.ignored_by_type == {- "m.room.member (before arrivals were recorded)": 1- }-- async def test_a_leave_is_never_compared(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$a", now))- session.add(_join_row(room.id, "$join", now + timedelta(seconds=2)))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [- HistoryPage(- events=[- _join_event(- "$leave",- now + timedelta(seconds=3),- membership="leave",- prev_membership="join",- ),- _join_event("$join", now + timedelta(seconds=1)),- _event("$a", now),- ],- next_token=None,- )- ]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert report.clean- assert report.ignored_by_type == {"m.room.member": 1}-- async def test_a_row_written_before_its_type_was_denied_is_not_drift(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """These exist on real deployments: the recorder kept every type before- the log was scoped, and those rows outlive the change."""- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$a", now))- session.add(- _row(- room.id,- "$legacy",- now + timedelta(seconds=1),- event_type="com.switch.agent.runtime_state",- msgtype=None,- body=None,- content={"state": "busy"},- )- )- await session.commit()- room_id = room.id-- transport = PagingTransport(- [HistoryPage(events=[_event("$a", now)], next_token=None)]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert report.clean- assert report.unverifiable_by_type == {"com.switch.agent.runtime_state": 1}---class TestDrift:- async def test_a_delivered_message_with_no_row_is_reported(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """The failure mode recording was designed to tolerate: the send- succeeded and the write after it did not."""- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$a", now))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [- HistoryPage(- events=[- _event("$lost", now + timedelta(seconds=1)),- _event("$a", now),- ],- next_token=None,- )- ]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert not report.clean- assert report.missing_rows == ["$lost"]- assert "DRIFT" in report.summary()-- async def test_a_row_the_bus_does_not_have_is_reported(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$a", now))- session.add(_row(room.id, "$phantom", now))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [HistoryPage(events=[_event("$a", now)], next_token=None)]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert report.unsent_rows == ["$phantom"]-- async def test_a_field_that_disagrees_is_named(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$a", now, body="what was recorded"))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [- HistoryPage(- events=[_event("$a", now, body="what the bus holds")],- next_token=None,- )- ]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert [m.field_name for m in report.mismatches] == ["body"]- assert "what the bus holds" in report.mismatches[0].describe()- assert report.agreed == 0-- async def test_a_custom_event_is_compared_on_its_type(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(- _row(- room.id,- "$cmd",- now,- event_type="com.switch.command",- msgtype=None,- body=None,- content={"command": "reset"},- )- )- await session.commit()- room_id = room.id-- transport = PagingTransport(- [- HistoryPage(- events=[- InboundCustomEvent(- room_id="!room:test",- event_id="$cmd",- sender="@agent:test",- timestamp=_ms(now),- content={"command": "reset"},- event_type="com.switch.command",- )- ],- next_token=None,- )- ]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert report.clean- assert report.agreed == 1-- async def test_a_media_event_is_compared_on_its_uri(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(- _row(- room.id,- "$img",- now,- msgtype="m.image",- body="chart.png",- content={- "msgtype": "m.image",- "body": "chart.png",- "url": "mxc://a",- },- )- )- await session.commit()- room_id = room.id-- transport = PagingTransport(- [- HistoryPage(- events=[- InboundMedia(- room_id="!room:test",- event_id="$img",- sender="@agent:test",- timestamp=_ms(now),- content={},- body="chart.png",- msgtype="m.image",- uri="mxc://somewhere-else",- )- ],- next_token=None,- )- ]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert [m.field_name for m in report.mismatches] == ["uri"]-- async def test_an_event_the_recorder_does_not_keep_is_counted_not_reported(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """Bus traffic is absent by design. Reporting it as a missing row would- drown the one row that really went astray."""- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$msg", now))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [- HistoryPage(- events=[- InboundCustomEvent(- room_id="!room:test",- event_id="$state",- sender="@agent:test",- timestamp=_ms(now + timedelta(seconds=1)),- content={"state": "busy"},- event_type="com.switch.agent.runtime_state",- ),- _event("$msg", now),- ],- next_token=None,- )- ]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert report.clean- assert report.compared == 1- assert report.ignored_by_type == {"com.switch.agent.runtime_state": 1}---class TestPaging:- async def test_it_keeps_paging_until_the_window_is_covered(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """The read path gives up after a page budget. This must not: an event- it never looked at would otherwise be reported as agreeing."""- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- # Oldest first, so `seq` runs the same way the recorder assigns it:- # the anchor is the room's first message, not its last.- for n in (2, 1, 0):- session.add(_row(room.id, f"$p{n}", now - timedelta(minutes=n)))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [- HistoryPage(events=[_event("$p0", now)], next_token="t1"),- HistoryPage(- events=[_event("$p1", now - timedelta(minutes=1))],- next_token="t2",- ),- HistoryPage(- events=[_event("$p2", now - timedelta(minutes=2))],- next_token=None,- ),- ]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert transport.reads == 3- assert report.compared == 3- assert report.clean-- async def test_it_stops_at_a_token_that_does_not_advance(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """A server that returns the cursor it was given would otherwise loop- forever.-- The anchor is deliberately absent from the bus, since a walk that finds- it stops there and never reaches a second page.- """- now = datetime.now(UTC)- async with session_factory() as session:- room = await _make_room(session)- session.add(_row(room.id, "$vanished", now))- await session.commit()- room_id = room.id-- transport = PagingTransport(- [- HistoryPage(events=[], next_token="stuck"),- HistoryPage(events=[], next_token="stuck"),- ]- )-- async with session_factory() as session:- room = await session.get(Room, room_id) # type: ignore[assignment]- report = await reconcile_room(transport, session, room) # type: ignore[arg-type]-- assert transport.reads == 2- assert not report.anchoreddiff --git a/core/tests/switch_core/provisioning/test_postgres_provisioning.py b/core/tests/switch_core/provisioning/test_postgres_provisioning.pyindex 99eb40ff6..6c943f146 100644--- a/core/tests/switch_core/provisioning/test_postgres_provisioning.py+++ b/core/tests/switch_core/provisioning/test_postgres_provisioning.py@@ -29,7 +29,6 @@ async def _client(session: AsyncSession, name: str = "someone") -> Client:matrix_user_id=f"@{name}-{uuid.uuid4().hex[:8]}:test",display_name=name,type="agent",- password=str(uuid.uuid4()),)session.add(client)await session.flush()@@ -56,31 +55,6 @@ def _provisioning()-class TestAccounts:- async def test_registering_writes_nothing(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- """The caller's `clients` row is the account; there is no second- system to create one in."""- await _provisioning(session_factory).register_user("@a:test", "secret")-- async with session_factory() as session:- assert await ClientStore().get_by_matrix_user_id(session, "@a:test") is None-- async def test_a_password_is_checked_against_the_client_row(- self, session_factory: async_sessionmaker[AsyncSession]- ) -> None:- async with session_factory() as session:- client = await _client(session)- await session.commit()- user_id, password = client.matrix_user_id, client.password-- provisioning = _provisioning(session_factory)- assert await provisioning.verify_login(user_id, password) is True- assert await provisioning.verify_login(user_id, "wrong") is False- assert await provisioning.verify_login("@nobody:test", password) is False--class TestRooms:async def test_creating_a_room_mints_an_id_and_writes_no_row(self, session_factory: async_sessionmaker[AsyncSession]@@ -122,7 +96,7 @@ class TestMembership:# The arrival explains the membership, and carries no rendered# sentence: how it reads is the reader's to phrase.assert [row.event_type for row in rows] == [MEMBERSHIP_EVENT_TYPE]- assert rows[0].sender_matrix_id == user_id+ assert rows[0].sender_id == user_idassert rows[0].sender_name == "nova"assert rows[0].body is Nonediff --git a/core/tests/switch_core/test_config_db_ssl.py b/core/tests/switch_core/test_config_db_ssl.pyindex ed4e40f4b..0f18c7158 100644--- a/core/tests/switch_core/test_config_db_ssl.py+++ b/core/tests/switch_core/test_config_db_ssl.py@@ -8,11 +8,7 @@ _BASE_KWARGS = dict(db_user="postgres",db_password="pw",db_name="switch",- matrix_server="http://tuwunel:8008",matrix_server_name="switch.local",- matrix_admin_user="admin",- matrix_admin_password="pw",- matrix_registration_shared_secret="secret",agent_registration_token="token",jwt_secret_key="jwt",gateway_admin_email="[email protected]",diff --git a/core/tests/switch_core/test_config_gateway_public_url.py b/core/tests/switch_core/test_config_gateway_public_url.pyindex 5583d568b..d751d2039 100644--- a/core/tests/switch_core/test_config_gateway_public_url.py+++ b/core/tests/switch_core/test_config_gateway_public_url.py@@ -8,11 +8,7 @@ _BASE_KWARGS = dict(db_user="postgres",db_password="pw",db_name="switch",- matrix_server="http://tuwunel:8008",matrix_server_name="switch.local",- matrix_admin_user="admin",- matrix_admin_password="pw",- matrix_registration_shared_secret="secret",agent_registration_token="token",jwt_secret_key="jwt",gateway_admin_email="[email protected]",diff --git a/core/tests/switch_core/test_db_idle_in_transaction_timeout.py b/core/tests/switch_core/test_db_idle_in_transaction_timeout.pyindex bf7493f24..568d8f242 100644--- a/core/tests/switch_core/test_db_idle_in_transaction_timeout.py+++ b/core/tests/switch_core/test_db_idle_in_transaction_timeout.py@@ -19,11 +19,7 @@ _BASE_KWARGS = dict(db_user="postgres",db_password="pw",db_name="switch",- matrix_server="http://tuwunel:8008",matrix_server_name="switch.local",- matrix_admin_user="admin",- matrix_admin_password="pw",- matrix_registration_shared_secret="secret",agent_registration_token="token",jwt_secret_key="jwt",gateway_admin_email="[email protected]",diff --git a/core/tests/switch_core/test_matrix_admin_kick.py b/core/tests/switch_core/test_matrix_admin_kick.pydeleted file mode 100644index 68424fb10..000000000--- a/core/tests/switch_core/test_matrix_admin_kick.py+++ /dev/null@@ -1,67 +0,0 @@-from __future__ import annotations--import httpx-import pytest--from switch_core.matrix_admin import MatrixAdmin, MatrixAdminError--ROOM = "!T5MphZvo650rb9LDq2:switch.local"-USER = "@switch-agent-d322f24a:switch.local"--# Verbatim from Tuwunel, typo included — the guard matches on this text, so a-# paraphrase would test nothing.-TUWUNEL_NOT_A_MEMBER = (- '{"errcode":"M_FORBIDDEN","error":"M_FORBIDDEN: Cannot kick a user who is '- 'not apart of the room (current membership: leave)"}'-)-SYNAPSE_NOT_A_MEMBER = (- '{"errcode":"M_FORBIDDEN","error":"The target user is not in the room"}'-)---def _admin(handler: object) -> MatrixAdmin:- http = httpx.AsyncClient(- base_url="http://homeserver.invalid",- transport=httpx.MockTransport(handler), # type: ignore[arg-type]- )- return MatrixAdmin(- server_url="http://homeserver.invalid",- admin_user="@admin:switch.local",- access_token="token",- shared_secret="secret",- http=http,- )---class TestKickUser:- @pytest.mark.parametrize("body", [TUWUNEL_NOT_A_MEMBER, SYNAPSE_NOT_A_MEMBER])- async def test_already_gone_is_not_an_error(self, body: str) -> None:- """Tearing a room down kicks every client it ever had, and a client that- already left answers 403. Deleting a room must not abort on it- (CHOO-2344: disconnecting a messaging app failed with a 500)."""- admin = _admin(lambda request: httpx.Response(403, text=body))-- await admin.kick_user(ROOM, USER)-- await admin._http.aclose()-- async def test_real_forbidden_still_raises(self) -> None:- admin = _admin(- lambda request: httpx.Response(- 403,- text='{"errcode":"M_FORBIDDEN","error":"You don\'t have permission to kick"}',- )- )-- with pytest.raises(MatrixAdminError, match="Failed to kick"):- await admin.kick_user(ROOM, USER)-- await admin._http.aclose()-- async def test_other_failures_still_raise(self) -> None:- admin = _admin(lambda request: httpx.Response(500, text="boom"))-- with pytest.raises(MatrixAdminError, match="Failed to kick"):- await admin.kick_user(ROOM, USER)-- await admin._http.aclose()diff --git a/core/tests/switch_core/test_provisioning_port.py b/core/tests/switch_core/test_provisioning_port.pydeleted file mode 100644index f223eb8e6..000000000--- a/core/tests/switch_core/test_provisioning_port.py+++ /dev/null@@ -1,58 +0,0 @@-"""Every provisioning implementation must satisfy the port.--Structural conformance is not checked anywhere else: neither implementation-names the Protocol, and every caller annotates the port, so a signature-drifting apart from the contract would type-check on both sides and only fail-at the call. This is the same guard `test_no_nio_outside_transport.py` gives the-transport port, in the shape a Protocol allows.-"""--from __future__ import annotations--import inspect--import pytest--from switch_core.matrix_admin import MatrixAdmin-from switch_core.provisioning import Provisioning-from switch_core.provisioning.postgres import PostgresProvisioning--[email protected]("implementation", [MatrixAdmin, PostgresProvisioning])-def test_it_is_a_provisioning_implementation(implementation: type) -> None:- assert isinstance(implementation.__new__(implementation), Provisioning), (- f"{implementation.__name__} no longer satisfies Provisioning"- )---def test_every_port_operation_has_a_matching_signature() -> None:- """`isinstance` on a Protocol checks names only, not what they take.-- A parameter renamed or dropped on one side is exactly the drift this port- exists to prevent, and it is invisible to the check above.- """- for name in (- "register_user",- "verify_login",- "create_room",- "invite_to_room",- "kick_user",- "delete_room",- "close",- ):- expected = _rendered(getattr(Provisioning, name))- for implementation in (MatrixAdmin, PostgresProvisioning):- actual = _rendered(getattr(implementation, name))- assert actual == expected, (- f"{implementation.__name__}.{name} does not match the port"- )---def _rendered(func: object) -> str:- """A signature as text, with the quotes deferred annotations add removed.-- The port defers its annotations and `matrix_admin` does not, so the two- render the same types differently — `'str'` against `str` — while meaning- the same thing. Comparing the text without quotes compares the contract.- """- return str(inspect.signature(func)).replace("'", "")diff --git a/core/tests/switch_core/test_room_service_pool_holds.py b/core/tests/switch_core/test_room_service_pool_holds.pyindex e26d6340f..d7715c7b0 100644--- a/core/tests/switch_core/test_room_service_pool_holds.py+++ b/core/tests/switch_core/test_room_service_pool_holds.py@@ -111,7 +111,6 @@ async def _seed(matrix_user_id=f"@a{i}:test",display_name=f"a{i}",type="agent",- password="x",)session.add_all([key, client])await session.flush()diff --git a/core/tests/switch_core/test_rooms_yaml.py b/core/tests/switch_core/test_rooms_yaml.pyindex 591840031..c4b49ace3 100644--- a/core/tests/switch_core/test_rooms_yaml.py+++ b/core/tests/switch_core/test_rooms_yaml.py@@ -121,7 +121,6 @@ async def _make_agent(session: AsyncSession, name: str, user_id: str) -> Agent:matrix_user_id=f"@{name}:test.local",display_name=name,type="agent",- password="x",)session.add_all([api_key, client])await session.flush()@@ -600,13 +599,11 @@ async def test_export_includes_users_from_bridge(env):matrix_user_id="@bridge:test.local",display_name="bridge",type="collaboration_bridge",- password="x",)user_client = Client(matrix_user_id="@bob:test.local",display_name="bob",type="external_user",- password="x",)session.add_all([bridge_client, user_client])await session.flush()diff --git a/core/tests/switch_core/transport/fake.py b/core/tests/switch_core/transport/fake.pyindex 760699a70..34d313b6d 100644--- a/core/tests/switch_core/transport/fake.py+++ b/core/tests/switch_core/transport/fake.py@@ -21,32 +21,6 @@ from switch_core.transport import ()-class FakeMessageRecorder:- """A `MessageRecorder` double for tests that send without a database."""-- def __init__(self) -> None:- self.recorded: list[dict[str, Any]] = []-- async def record(- self,- *,- transport_room_id: str,- result: SendResult,- sender_matrix_id: str,- sender_client_id: str,- sender_name: str,- ) -> None:- self.recorded.append(- {- "transport_room_id": transport_room_id,- "result": result,- "sender_matrix_id": sender_matrix_id,- "sender_client_id": sender_client_id,- "sender_name": sender_name,- }- )--class FakeTransport:"""Records what was sent and replays what it was primed with."""diff --git a/core/tests/switch_core/transport/test_matrix_transport.py b/core/tests/switch_core/transport/test_matrix_transport.pydeleted file mode 100644index 01e0bb6e2..000000000--- a/core/tests/switch_core/transport/test_matrix_transport.py+++ /dev/null@@ -1,224 +0,0 @@-from types import SimpleNamespace-from typing import Any--import pytest-from nio import RoomMessagesError, RoomSendError--from switch_core.attachments import ATTACHMENT_GROUP_KEY-from switch_core.transport import (- MessageTransport,- TransportError,- TransportHandlers,-)-from switch_core.transport.matrix import MatrixTransport---class _FakeNio:- def __init__(self, *, send_result: Any = None, upload_result: Any = None) -> None:- self._send_result = send_result or SimpleNamespace(event_id="$evt")- self._upload_result = upload_result- self.room_send_calls: list[tuple[str, str, dict]] = []- self.typing_calls: list[tuple[str, bool]] = []- self.callbacks: list[tuple[Any, Any]] = []- self.response_callbacks: list[tuple[Any, Any]] = []- self.access_token = "tok"-- async def room_send(self, room_id: str, event_type: str, content: dict) -> Any:- self.room_send_calls.append((room_id, event_type, content))- return self._send_result-- async def upload(self, **kwargs: Any) -> tuple[Any, None]:- return self._upload_result, None-- async def room_typing(self, room_id: str, is_typing: bool) -> Any:- self.typing_calls.append((room_id, is_typing))- return SimpleNamespace()-- async def joined_rooms(self) -> Any:- return SimpleNamespace(rooms=["!a:s"])-- async def join(self, room_id: str) -> Any:- return SimpleNamespace(room_id=room_id)-- async def room_messages(self, room_id: str, start: str | None, limit: int) -> Any:- return SimpleNamespace(chunk=["e1", "e2"], end="tok-2")-- def add_event_callback(self, cb: Any, kind: Any) -> None:- self.callbacks.append((cb, kind))-- def add_response_callback(self, cb: Any, kind: Any) -> None:- self.response_callbacks.append((cb, kind))---def _transport(nio: _FakeNio) -> MatrixTransport:- return MatrixTransport(- server_url="https://s",- user_id="@a:s",- password="pw",- client=nio,- )---def test_matrix_transport_satisfies_the_port() -> None:- assert isinstance(_transport(_FakeNio()), MessageTransport)---async def test_send_message_returns_a_neutral_result() -> None:- nio = _FakeNio()-- result = await _transport(nio).send_message("!r:s", "hi", sender_name="Alice")-- assert result.event_id == "$evt"- room_id, event_type, content = nio.room_send_calls[0]- assert (room_id, event_type) == ("!r:s", "m.room.message")- assert content["body"] == "hi"- assert content["sender_name"] == "Alice"---async def test_send_message_threads_via_a_pure_thread_relation() -> None:- nio = _FakeNio()-- await _transport(nio).send_message(- "!r:s", "hi", sender_name="Alice", thread_root_id="$root"- )-- _, _, content = nio.room_send_calls[0]- assert content["m.relates_to"] == {"rel_type": "m.thread", "event_id": "$root"}---async def test_send_message_raises_rather_than_returning_an_error() -> None:- nio = _FakeNio(send_result=object.__new__(RoomSendError))- nio._send_result.message = "nope" # type: ignore[attr-defined]-- with pytest.raises(TransportError, match="nope"):- await _transport(nio).send_message("!r:s", "hi", sender_name="Alice")---async def test_send_media_with_a_caption_keeps_the_filename_separate() -> None:- # The rich-media-caption convention: body carries the caption, and the real- # filename rides alongside so it is not lost.- nio = _FakeNio()-- await _transport(nio).send_media(- "!r:s",- "mxc://s/abc",- "cat.png",- "image/png",- 5,- sender_name="Alice",- msgtype="m.image",- caption="look at this",- )-- _, _, content = nio.room_send_calls[0]- assert content["body"] == "look at this"- assert content["filename"] == "cat.png"- assert content["url"] == "mxc://s/abc"- assert content["info"] == {"mimetype": "image/png", "size": 5}---async def test_send_media_without_a_caption_uses_the_filename_as_body() -> None:- nio = _FakeNio()-- await _transport(nio).send_media(- "!r:s",- "mxc://s/abc",- "cat.png",- "image/png",- 5,- sender_name="Alice",- msgtype="m.image",- )-- _, _, content = nio.room_send_calls[0]- assert content["body"] == "cat.png"- # No caption means nothing to disambiguate, so no separate filename key.- assert "filename" not in content---async def test_send_media_marks_a_multi_file_group() -> None:- # Matrix has no multi-attachment event, so N files share a group id that- # receivers coalesce back into one message.- nio = _FakeNio()- group = {"id": "g1", "index": 0, "total": 2}-- await _transport(nio).send_media(- "!r:s",- "mxc://s/abc",- "cat.png",- "image/png",- 5,- sender_name="Alice",- msgtype="m.image",- group=group,- )-- _, _, content = nio.room_send_calls[0]- assert content[ATTACHMENT_GROUP_KEY] == group---async def test_send_event_carries_the_type_through() -> None:- nio = _FakeNio()-- await _transport(nio).send_event("!r:s", "com.switch.task.accept", {"task_id": "t"})-- assert nio.room_send_calls[0][1] == "com.switch.task.accept"---async def test_read_history_returns_neutral_events_and_its_cursor() -> None:- nio = _FakeNio()-- async def _chunk(*args: Any, **kwargs: Any) -> Any:- return SimpleNamespace(- chunk=[- SimpleNamespace(- event_id="$1",- sender="@a:s",- server_timestamp=7,- body="hello",- source={"content": {"body": "hello"}},- )- ],- end="tok-2",- )-- nio.room_messages = _chunk # type: ignore[assignment]-- page = await _transport(nio).read_history("!r:s", start=None, limit=10)-- assert page.next_token == "tok-2"- assert [e.event_id for e in page.events] == ["$1"]- # History arrives already converted, so callers never see a nio type.- assert page.events[0].room_id == "!r:s"- assert page.events[0].timestamp == 7---async def test_read_history_raises_on_failure() -> None:- nio = _FakeNio()- err = object.__new__(RoomMessagesError)- err.message = "boom" # type: ignore[attr-defined]-- async def _fail(*args: Any, **kwargs: Any) -> Any:- return err-- nio.room_messages = _fail # type: ignore[assignment]-- with pytest.raises(TransportError, match="boom"):- await _transport(nio).read_history("!r:s", start=None, limit=10)---def test_register_handlers_binds_only_what_was_supplied() -> None:- nio = _FakeNio()-- _transport(nio).register_handlers(TransportHandlers(on_message=lambda r, e: None)) # type: ignore[arg-type]-- assert len(nio.callbacks) == 1- assert nio.response_callbacks == []---async def test_typing_is_best_effort() -> None:- nio = _FakeNio()-- await _transport(nio).set_typing("!r:s", True)-- assert nio.typing_calls == [("!r:s", True)]diff --git a/core/tests/switch_core/transport/test_no_nio_outside_transport.py b/core/tests/switch_core/transport/test_no_nio_outside_transport.pydeleted file mode 100644index efc10f439..000000000--- a/core/tests/switch_core/transport/test_no_nio_outside_transport.py+++ /dev/null@@ -1,57 +0,0 @@-"""The seam only holds if something enforces it.--Matrix may be imported in exactly one place. Without this check the port-erodes the first time someone reaches for a nio type in a hurry, and the-erosion is invisible until the homeserver is being replaced.-"""--import ast-import pathlib--PACKAGE = pathlib.Path(__file__).resolve().parents[3] / "switch_core"-ALLOWED = {PACKAGE / "transport" / "matrix.py"}---def _imports_nio(path: pathlib.Path) -> bool:- tree = ast.parse(path.read_text(), filename=str(path))- for node in ast.walk(tree):- if isinstance(node, ast.Import):- if any(a.name == "nio" or a.name.startswith("nio.") for a in node.names):- return True- elif isinstance(node, ast.ImportFrom):- module = node.module or ""- if module == "nio" or module.startswith("nio."):- return True- return False---def test_only_the_matrix_transport_imports_nio() -> None:- offenders = sorted(- str(path.relative_to(PACKAGE))- for path in PACKAGE.rglob("*.py")- if path not in ALLOWED and _imports_nio(path)- )-- assert offenders == [], (- "matrix-nio may only be imported by switch_core/transport/matrix.py; "- f"these modules import it directly: {offenders}. "- "Add what you need to MessageTransport instead."- )---def test_the_guard_can_actually_see_an_import(tmp_path: pathlib.Path) -> None:- offender = tmp_path / "offender.py"- offender.write_text("from nio import AsyncClient\n")-- assert _imports_nio(offender)---def test_no_module_reaches_for_a_raw_client() -> None:- """`nio_client` was the escape hatch during the port. It is gone."""- offenders = sorted(- str(path.relative_to(PACKAGE))- for path in PACKAGE.rglob("*.py")- if "nio_client" in path.read_text()- )-- assert offenders == []diff --git a/core/tests/switch_core/transport/test_postgres_transport.py b/core/tests/switch_core/transport/test_postgres_transport.pyindex f05de0c16..2373674d9 100644--- a/core/tests/switch_core/transport/test_postgres_transport.py+++ b/core/tests/switch_core/transport/test_postgres_transport.py@@ -47,7 +47,6 @@ async def _make_room(session: AsyncSession) -> tuple[str, str, str, str]:matrix_user_id=f"@agent-{suffix}:test",display_name="agent one",type="agent",- password="x",)session.add(client)await session.flush()@@ -185,7 +184,7 @@ class TestSending:assert message.msgtype == "m.text"assert message.event_type == "m.room.message"assert message.sender_client_id == client_id- assert message.sender_matrix_id == user_id+ assert message.sender_id == user_id# The id the caller got back is the row's, so a caller that quotes it# in a reply or a thread is naming something that exists.assert message.transport_event_id == result.event_id@@ -420,7 +419,7 @@ class TestReceiving:)transport.register_handlers(received.handlers())await transport.join_room(transport_room_id)- task = asyncio.create_task(transport.receive_forever(since=None))+ task = asyncio.create_task(transport.receive_forever())self._tasks.append((task, transport))await _watched_room(transport)return transport, listener, received, transport_room_id, user_id@@ -472,7 +471,7 @@ class TestReceiving:)reader.register_handlers(received.handlers())await reader.join_room(room)- task = asyncio.create_task(reader.receive_forever(since=None))+ task = asyncio.create_task(reader.receive_forever())self._tasks.append((task, reader))switch_room_id = await _watched_room(reader)await listener.announce(switch_room_id)@@ -533,7 +532,6 @@ class TestReceiving:matrix_user_id=f"@later-{uuid.uuid4().hex[:8]}:test",display_name="the newcomer",type="agent",- password="x",)session.add(newcomer)await session.commit()@@ -583,7 +581,7 @@ class TestPresence:)watcher.register_handlers(received.handlers())await watcher.join_room(room)- self._tasks.append(asyncio.create_task(watcher.receive_forever(since=None)))+ self._tasks.append(asyncio.create_task(watcher.receive_forever()))await _watched_room(watcher)sender = _transport(@@ -616,7 +614,7 @@ class TestPresence:)watcher.register_handlers(received.handlers())await watcher.join_room(room)- self._tasks.append(asyncio.create_task(watcher.receive_forever(since=None)))+ self._tasks.append(asyncio.create_task(watcher.receive_forever()))await _watched_room(watcher)sender = _transport(@@ -647,7 +645,7 @@ class TestPresence:)watcher.register_handlers(received.handlers())await watcher.join_room(room)- task = asyncio.create_task(watcher.receive_forever(since=None))+ task = asyncio.create_task(watcher.receive_forever())self._tasks.append(task)await _watched_room(watcher)await watcher.close()@@ -690,7 +688,7 @@ class TestOneClientCannotStallAnother:)transport.register_handlers(received.handlers())await transport.join_room(room)- self._tasks.append(asyncio.create_task(transport.receive_forever(since=None)))+ self._tasks.append(asyncio.create_task(transport.receive_forever()))switch_room_id = await _watched_room(transport)await transport.send_message(room, "hello", sender_name="agent one")@@ -721,7 +719,7 @@ class TestOneClientCannotStallAnother:)transport.register_handlers(received.handlers())await transport.join_room(room)- self._tasks.append(asyncio.create_task(transport.receive_forever(since=None)))+ self._tasks.append(asyncio.create_task(transport.receive_forever()))switch_room_id = await _watched_room(transport)await transport.send_message(room, "hello", sender_name="agent one")@@ -764,7 +762,7 @@ class TestJoiningARoomAlreadyRecorded:)transport.register_handlers(received.handlers())await transport.join_room(room)- self._tasks.append(asyncio.create_task(transport.receive_forever(since=None)))+ self._tasks.append(asyncio.create_task(transport.receive_forever()))await _watched_room(transport)# Somebody else records the membership — what a moderation invite does@@ -817,7 +815,7 @@ class TestHearingYourOwnArrival:)transport.register_handlers(received.handlers())await transport.join_room(room)- self._tasks.append(asyncio.create_task(transport.receive_forever(since=None)))+ self._tasks.append(asyncio.create_task(transport.receive_forever()))await _watched_room(transport)await transport.join_room(later)@@ -851,7 +849,7 @@ class TestHearingYourOwnArrival:)transport.register_handlers(received.handlers())await transport.join_room(room)- self._tasks.append(asyncio.create_task(transport.receive_forever(since=None)))+ self._tasks.append(asyncio.create_task(transport.receive_forever()))await _watched_room(transport)await transport.join_room(later)diff --git a/core/tests/switch_core/transport/test_sync_durability.py b/core/tests/switch_core/transport/test_sync_durability.pydeleted file mode 100644index bb19bc519..000000000--- a/core/tests/switch_core/transport/test_sync_durability.py+++ /dev/null@@ -1,85 +0,0 @@-"""What counts as a batch worth persisting the cursor for.--The judgement has to live here rather than in the client: only the transport-sees what a sync response held, and the port hands the client a bool so a nio-response class never reaches it. The client's side of the bargain — the-throttle, the staleness flush, the per-client jitter — is covered in-`tests/switch_core/clients/test_sync_cursor_persistence.py`.--Timeline events, room state and invites are dispatched to handlers, so a-restart must not skip them. Presence, typing and receipts are not: resuming-from the older cursor replays the same nothing.-"""--from __future__ import annotations--import pytest-from nio import (- DeviceList,- DeviceOneTimeKeyCount,- InviteInfo,- RoomInfo,- Rooms,- SyncResponse,- Timeline,-)--from switch_core.transport.matrix import _carries_durable_events--ROOM = "!room:switch.local"---def _sync(- next_batch: str = "s1",- *,- timeline: list[object] | None = None,- state: list[object] | None = None,- ephemeral: list[object] | None = None,- invite: bool = False,- to_device: list[object] | None = None,-) -> SyncResponse:- join = {- ROOM: RoomInfo(- timeline=Timeline(- events=list(timeline or []), limited=False, prev_batch=None- ),- state=list(state or []),- ephemeral=list(ephemeral or []),- account_data=[],- )- }- return SyncResponse(- next_batch=next_batch,- rooms=Rooms(- invite={ROOM: InviteInfo(invite_state=[])} if invite else {},- join=join,- leave={},- ),- device_key_count=DeviceOneTimeKeyCount(curve25519=None, signed_curve25519=None),- device_list=DeviceList(changed=[], left=[]),- to_device_events=list(to_device or []),- presence_events=[],- )--- "kwargs",- [- {"timeline": [object()]},- {"state": [object()]},- {"invite": True},- {"to_device": [object()]},- ],- ids=["timeline", "state", "invite", "to_device"],-)-def test_anything_a_restart_would_replay_is_durable(kwargs: dict[str, object]) -> None:- assert _carries_durable_events(_sync(**kwargs)) is True # type: ignore[arg-type]---def test_an_empty_batch_is_not() -> None:- assert _carries_durable_events(_sync()) is False---def test_ephemeral_only_is_not() -> None:- """Typing notices and read receipts move the token and mean nothing."""- assert _carries_durable_events(_sync(ephemeral=[object()])) is Falsediff --git a/core/uv.lock b/core/uv.lockindex 3950f92bd..e6288aed9 100644--- a/core/uv.lock+++ b/core/uv.lock@@ -22,15 +22,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/67/cd/0d76dfc5de72bde52f55f53e925c7d152d9c7906634ec1e0cbc7e8d4ad93/aiofile-3.11.1-py3-none-any.whl", hash = "sha256:ce77d14ac07f77bc2b757834a5c129321f3f705c474593deed5ab209079a52c9", size = 20446, upload-time = "2026-05-16T08:18:32.051Z" },]-[[package]]-name = "aiofiles"-version = "24.1.0"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/0b/03/a88171e277e8caa88a4c77808c20ebb04ba74cc4681bf1e9416c862de237/aiofiles-24.1.0.tar.gz", hash = "sha256:22a075c9e5a3810f0c2e48f3008c94d68c65d763b9b03857924c99e57355166c", size = 30247, upload-time = "2024-06-24T11:02:03.584Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/a5/45/30bb92d442636f570cb5651bc661f52b610e2eec3f891a5dc3a4c3667db0/aiofiles-24.1.0-py3-none-any.whl", hash = "sha256:b4ec55f4195e3eb5d7abd1bf7e061763e864dd4954231fb8539a0ef8bb8260e5", size = 15896, upload-time = "2024-06-24T11:02:01.529Z" },-]-[[package]]name = "aiohappyeyeballs"version = "2.6.1"@@ -140,19 +131,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/a4/37/cfd1ed540a4d318da025590d96b728e63713c09e9377950fc655dadeb856/aiohttp-3.14.3-cp314-cp314t-win_arm64.whl", hash = "sha256:2e1161602f45a54de2ce0905243a95f58cb42dcd378402f3697f5e0b21e9d2e7", size = 469280, upload-time = "2026-07-23T01:57:24.241Z" },]-[[package]]-name = "aiohttp-socks"-version = "0.11.0"-source = { registry = "https://pypi.org/simple" }-dependencies = [- { name = "aiohttp" },- { name = "python-socks" },-]-sdist = { url = "https://files.pythonhosted.org/packages/1f/cc/e5bbd54f76bd56291522251e47267b645dac76327b2657ade9545e30522c/aiohttp_socks-0.11.0.tar.gz", hash = "sha256:0afe51638527c79077e4bd6e57052c87c4824233d6e20bb061c53766421b10f0", size = 11196, upload-time = "2025-12-09T13:35:52.564Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/bf/7d/4b633d709b8901d59444d2e512b93e72fe62d2b492a040097c3f7ba017bb/aiohttp_socks-0.11.0-py3-none-any.whl", hash = "sha256:9aacce57c931b8fbf8f6d333cf3cafe4c35b971b35430309e167a35a8aab9ec1", size = 10556, upload-time = "2025-12-09T13:35:50.18Z" },-]-[[package]]name = "aiosignal"version = "1.4.0"@@ -1017,28 +995,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515, upload-time = "2025-04-24T03:35:24.344Z" },]-[[package]]-name = "h2"-version = "4.3.0"-source = { registry = "https://pypi.org/simple" }-dependencies = [- { name = "hpack" },- { name = "hyperframe" },-]-sdist = { url = "https://files.pythonhosted.org/packages/1d/17/afa56379f94ad0fe8defd37d6eb3f89a25404ffc71d4d848893d270325fc/h2-4.3.0.tar.gz", hash = "sha256:6c59efe4323fa18b47a632221a1888bd7fde6249819beda254aeca909f221bf1", size = 2152026, upload-time = "2025-08-23T18:12:19.778Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/69/b2/119f6e6dcbd96f9069ce9a2665e0146588dc9f88f29549711853645e736a/h2-4.3.0-py3-none-any.whl", hash = "sha256:c438f029a25f7945c69e0ccf0fb951dc3f73a5f6412981daee861431b70e2bdd", size = 61779, upload-time = "2025-08-23T18:12:17.779Z" },-]--[[package]]-name = "hpack"-version = "4.1.0"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/2c/48/71de9ed269fdae9c8057e5a4c0aa7402e8bb16f2c6e90b3aa53327b113f8/hpack-4.1.0.tar.gz", hash = "sha256:ec5eca154f7056aa06f196a557655c5b009b382873ac8d1e66e79e87535f1dca", size = 51276, upload-time = "2025-01-22T21:44:58.347Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/07/c6/80c95b1b2b94682a72cbdbfb85b81ae2daffa4291fbfa1b1464502ede10d/hpack-4.1.0-py3-none-any.whl", hash = "sha256:157ac792668d995c657d93111f46b4535ed114f0c9c8d672271bbec7eae1b496", size = 34357, upload-time = "2025-01-22T21:44:56.92Z" },-]-[[package]]name = "httpcore"version = "1.0.9"@@ -1076,15 +1032,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/d2/fd/6668e5aec43ab844de6fc74927e155a3b37bf40d7c3790e49fc0406b6578/httpx_sse-0.4.3-py3-none-any.whl", hash = "sha256:0ac1c9fe3c0afad2e0ebb25a934a59f4c7823b60792691f779fad2c5568830fc", size = 8960, upload-time = "2025-10-10T21:48:21.158Z" },]-[[package]]-name = "hyperframe"-version = "6.1.0"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/02/e7/94f8232d4a74cc99514c13a9f995811485a6903d48e5d952771ef6322e30/hyperframe-6.1.0.tar.gz", hash = "sha256:f630908a00854a7adeabd6382b43923a4c4cd4b821fcb527e6ab9e15382a3b08", size = 26566, upload-time = "2025-01-22T21:41:49.302Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/48/30/47d0bf6072f7252e6521f3447ccfa40b421b6824517f82854703d0f5a98b/hyperframe-6.1.0-py3-none-any.whl", hash = "sha256:b03380493a519fce58ea5af42e4a42317bf9bd425596f7a0835ffce80f1a42e5", size = 13007, upload-time = "2025-01-22T21:41:47.295Z" },-]-[[package]]name = "identify"version = "2.6.19"@@ -1410,25 +1357,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/70/bc/6f1c2f612465f5fa89b95bead1f44dcb607670fd42891d8fdcd5d039f4f4/markupsafe-3.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa", size = 14146, upload-time = "2025-09-27T18:37:28.327Z" },]-[[package]]-name = "matrix-nio"-version = "0.25.2"-source = { registry = "https://pypi.org/simple" }-dependencies = [- { name = "aiofiles" },- { name = "aiohttp" },- { name = "aiohttp-socks" },- { name = "h11" },- { name = "h2" },- { name = "jsonschema" },- { name = "pycryptodome" },- { name = "unpaddedbase64" },-]-sdist = { url = "https://files.pythonhosted.org/packages/33/50/c20129fd6f0e1aad3510feefd3229427fc8163a111f3911ed834e414116b/matrix_nio-0.25.2.tar.gz", hash = "sha256:8ef8180c374e12368e5c83a692abfb3bab8d71efcd17c5560b5c40c9b6f2f600", size = 155480, upload-time = "2024-10-04T07:51:41.62Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/7b/0f/8b958d46e23ed4f69d2cffd63b46bb097a1155524e2e7f5c4279c8691c4a/matrix_nio-0.25.2-py3-none-any.whl", hash = "sha256:9c2880004b0e475db874456c0f79b7dd2b6285073a7663bcaca29e0754a67495", size = 181982, upload-time = "2024-10-04T07:51:39.451Z" },-]-[[package]]name = "mattermostdriver"version = "7.3.2"@@ -1860,36 +1788,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl", hash = "sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", size = 48172, upload-time = "2026-01-21T14:26:50.693Z" },]-[[package]]-name = "pycryptodome"-version = "3.23.0"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/8e/a6/8452177684d5e906854776276ddd34eca30d1b1e15aa1ee9cefc289a33f5/pycryptodome-3.23.0.tar.gz", hash = "sha256:447700a657182d60338bab09fdb27518f8856aecd80ae4c6bdddb67ff5da44ef", size = 4921276, upload-time = "2025-05-17T17:21:45.242Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/04/5d/bdb09489b63cd34a976cc9e2a8d938114f7a53a74d3dd4f125ffa49dce82/pycryptodome-3.23.0-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:0011f7f00cdb74879142011f95133274741778abba114ceca229adbf8e62c3e4", size = 2495152, upload-time = "2025-05-17T17:20:20.833Z" },- { url = "https://files.pythonhosted.org/packages/a7/ce/7840250ed4cc0039c433cd41715536f926d6e86ce84e904068eb3244b6a6/pycryptodome-3.23.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:90460fc9e088ce095f9ee8356722d4f10f86e5be06e2354230a9880b9c549aae", size = 1639348, upload-time = "2025-05-17T17:20:23.171Z" },- { url = "https://files.pythonhosted.org/packages/ee/f0/991da24c55c1f688d6a3b5a11940567353f74590734ee4a64294834ae472/pycryptodome-3.23.0-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4764e64b269fc83b00f682c47443c2e6e85b18273712b98aa43bcb77f8570477", size = 2184033, upload-time = "2025-05-17T17:20:25.424Z" },- { url = "https://files.pythonhosted.org/packages/54/16/0e11882deddf00f68b68dd4e8e442ddc30641f31afeb2bc25588124ac8de/pycryptodome-3.23.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:eb8f24adb74984aa0e5d07a2368ad95276cf38051fe2dc6605cbcf482e04f2a7", size = 2270142, upload-time = "2025-05-17T17:20:27.808Z" },- { url = "https://files.pythonhosted.org/packages/d5/fc/4347fea23a3f95ffb931f383ff28b3f7b1fe868739182cb76718c0da86a1/pycryptodome-3.23.0-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d97618c9c6684a97ef7637ba43bdf6663a2e2e77efe0f863cce97a76af396446", size = 2309384, upload-time = "2025-05-17T17:20:30.765Z" },- { url = "https://files.pythonhosted.org/packages/6e/d9/c5261780b69ce66d8cfab25d2797bd6e82ba0241804694cd48be41add5eb/pycryptodome-3.23.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9a53a4fe5cb075075d515797d6ce2f56772ea7e6a1e5e4b96cf78a14bac3d265", size = 2183237, upload-time = "2025-05-17T17:20:33.736Z" },- { url = "https://files.pythonhosted.org/packages/5a/6f/3af2ffedd5cfa08c631f89452c6648c4d779e7772dfc388c77c920ca6bbf/pycryptodome-3.23.0-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:763d1d74f56f031788e5d307029caef067febf890cd1f8bf61183ae142f1a77b", size = 2343898, upload-time = "2025-05-17T17:20:36.086Z" },- { url = "https://files.pythonhosted.org/packages/9a/dc/9060d807039ee5de6e2f260f72f3d70ac213993a804f5e67e0a73a56dd2f/pycryptodome-3.23.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:954af0e2bd7cea83ce72243b14e4fb518b18f0c1649b576d114973e2073b273d", size = 2269197, upload-time = "2025-05-17T17:20:38.414Z" },- { url = "https://files.pythonhosted.org/packages/f9/34/e6c8ca177cb29dcc4967fef73f5de445912f93bd0343c9c33c8e5bf8cde8/pycryptodome-3.23.0-cp313-cp313t-win32.whl", hash = "sha256:257bb3572c63ad8ba40b89f6fc9d63a2a628e9f9708d31ee26560925ebe0210a", size = 1768600, upload-time = "2025-05-17T17:20:40.688Z" },- { url = "https://files.pythonhosted.org/packages/e4/1d/89756b8d7ff623ad0160f4539da571d1f594d21ee6d68be130a6eccb39a4/pycryptodome-3.23.0-cp313-cp313t-win_amd64.whl", hash = "sha256:6501790c5b62a29fcb227bd6b62012181d886a767ce9ed03b303d1f22eb5c625", size = 1799740, upload-time = "2025-05-17T17:20:42.413Z" },- { url = "https://files.pythonhosted.org/packages/5d/61/35a64f0feaea9fd07f0d91209e7be91726eb48c0f1bfc6720647194071e4/pycryptodome-3.23.0-cp313-cp313t-win_arm64.whl", hash = "sha256:9a77627a330ab23ca43b48b130e202582e91cc69619947840ea4d2d1be21eb39", size = 1703685, upload-time = "2025-05-17T17:20:44.388Z" },- { url = "https://files.pythonhosted.org/packages/db/6c/a1f71542c969912bb0e106f64f60a56cc1f0fabecf9396f45accbe63fa68/pycryptodome-3.23.0-cp37-abi3-macosx_10_9_universal2.whl", hash = "sha256:187058ab80b3281b1de11c2e6842a357a1f71b42cb1e15bce373f3d238135c27", size = 2495627, upload-time = "2025-05-17T17:20:47.139Z" },- { url = "https://files.pythonhosted.org/packages/6e/4e/a066527e079fc5002390c8acdd3aca431e6ea0a50ffd7201551175b47323/pycryptodome-3.23.0-cp37-abi3-macosx_10_9_x86_64.whl", hash = "sha256:cfb5cd445280c5b0a4e6187a7ce8de5a07b5f3f897f235caa11f1f435f182843", size = 1640362, upload-time = "2025-05-17T17:20:50.392Z" },- { url = "https://files.pythonhosted.org/packages/50/52/adaf4c8c100a8c49d2bd058e5b551f73dfd8cb89eb4911e25a0c469b6b4e/pycryptodome-3.23.0-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:67bd81fcbe34f43ad9422ee8fd4843c8e7198dd88dd3d40e6de42ee65fbe1490", size = 2182625, upload-time = "2025-05-17T17:20:52.866Z" },- { url = "https://files.pythonhosted.org/packages/5f/e9/a09476d436d0ff1402ac3867d933c61805ec2326c6ea557aeeac3825604e/pycryptodome-3.23.0-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c8987bd3307a39bc03df5c8e0e3d8be0c4c3518b7f044b0f4c15d1aa78f52575", size = 2268954, upload-time = "2025-05-17T17:20:55.027Z" },- { url = "https://files.pythonhosted.org/packages/f9/c5/ffe6474e0c551d54cab931918127c46d70cab8f114e0c2b5a3c071c2f484/pycryptodome-3.23.0-cp37-abi3-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:aa0698f65e5b570426fc31b8162ed4603b0c2841cbb9088e2b01641e3065915b", size = 2308534, upload-time = "2025-05-17T17:20:57.279Z" },- { url = "https://files.pythonhosted.org/packages/18/28/e199677fc15ecf43010f2463fde4c1a53015d1fe95fb03bca2890836603a/pycryptodome-3.23.0-cp37-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:53ecbafc2b55353edcebd64bf5da94a2a2cdf5090a6915bcca6eca6cc452585a", size = 2181853, upload-time = "2025-05-17T17:20:59.322Z" },- { url = "https://files.pythonhosted.org/packages/ce/ea/4fdb09f2165ce1365c9eaefef36625583371ee514db58dc9b65d3a255c4c/pycryptodome-3.23.0-cp37-abi3-musllinux_1_2_i686.whl", hash = "sha256:156df9667ad9f2ad26255926524e1c136d6664b741547deb0a86a9acf5ea631f", size = 2342465, upload-time = "2025-05-17T17:21:03.83Z" },- { url = "https://files.pythonhosted.org/packages/22/82/6edc3fc42fe9284aead511394bac167693fb2b0e0395b28b8bedaa07ef04/pycryptodome-3.23.0-cp37-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:dea827b4d55ee390dc89b2afe5927d4308a8b538ae91d9c6f7a5090f397af1aa", size = 2267414, upload-time = "2025-05-17T17:21:06.72Z" },- { url = "https://files.pythonhosted.org/packages/59/fe/aae679b64363eb78326c7fdc9d06ec3de18bac68be4b612fc1fe8902693c/pycryptodome-3.23.0-cp37-abi3-win32.whl", hash = "sha256:507dbead45474b62b2bbe318eb1c4c8ee641077532067fec9c1aa82c31f84886", size = 1768484, upload-time = "2025-05-17T17:21:08.535Z" },- { url = "https://files.pythonhosted.org/packages/54/2f/e97a1b8294db0daaa87012c24a7bb714147c7ade7656973fd6c736b484ff/pycryptodome-3.23.0-cp37-abi3-win_amd64.whl", hash = "sha256:c75b52aacc6c0c260f204cbdd834f76edc9fb0d8e0da9fbf8352ef58202564e2", size = 1799636, upload-time = "2025-05-17T17:21:10.393Z" },- { url = "https://files.pythonhosted.org/packages/18/3d/f9441a0d798bf2b1e645adc3265e55706aead1255ccdad3856dbdcffec14/pycryptodome-3.23.0-cp37-abi3-win_arm64.whl", hash = "sha256:11eeeb6917903876f134b56ba11abe95c0b0fd5e3330def218083c7d98bbcb3c", size = 1703675, upload-time = "2025-05-17T17:21:13.146Z" },-]-[[package]]name = "pydantic"version = "2.13.4"@@ -2091,15 +1989,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/5e/1e/7f7f299527a5a8ad90acd5f2f78dfa6c8495c6301a3205106ea68a84de96/python_multipart-0.0.31-py3-none-any.whl", hash = "sha256:8408153d68a9773291fc1da39a8b85a50044bddbabd2dd72e9229776b7b15e28", size = 29996, upload-time = "2026-06-04T08:27:47.804Z" },]-[[package]]-name = "python-socks"-version = "2.8.1"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/36/0b/cd77011c1bc01b76404f7aba07fca18aca02a19c7626e329b40201217624/python_socks-2.8.1.tar.gz", hash = "sha256:698daa9616d46dddaffe65b87db222f2902177a2d2b2c0b9a9361df607ab3687", size = 38909, upload-time = "2026-02-16T05:24:00.745Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/15/fe/9a58cb6eec633ff6afae150ca53c16f8cc8b65862ccb3d088051efdfceb7/python_socks-2.8.1-py3-none-any.whl", hash = "sha256:28232739c4988064e725cdbcd15be194743dd23f1c910f784163365b9d7be035", size = 55087, upload-time = "2026-02-16T05:23:59.147Z" },-]-[[package]]name = "python-telegram-bot"version = "21.11.1"@@ -2349,8 +2238,8 @@ name = "secretstorage"version = "3.5.0"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "cryptography" },- { name = "jeepney" },+ { name = "cryptography", marker = "python_full_version < '3.13' or python_full_version >= '3.15' or sys_platform != 'win32'" },+ { name = "jeepney", marker = "python_full_version < '3.13' or python_full_version >= '3.15' or sys_platform != 'win32'" },]sdist = { url = "https://files.pythonhosted.org/packages/1c/03/e834bcd866f2f8a49a85eaff47340affa3bfa391ee9912a952a1faa68c7b/secretstorage-3.5.0.tar.gz", hash = "sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be", size = 19884, upload-time = "2025-11-23T19:02:53.191Z" }wheels = [@@ -2460,7 +2349,6 @@ dependencies = [{ name = "httpx" },{ name = "itsdangerous" },{ name = "markdown" },- { name = "matrix-nio" },{ name = "mattermostdriver" },{ name = "pydantic" },{ name = "pydantic-settings" },@@ -2496,7 +2384,6 @@ requires-dist = [{ name = "httpx", specifier = ">=0.28,<1" },{ name = "itsdangerous", specifier = ">=2.1,<3" },{ name = "markdown", specifier = ">=3,<4" },- { name = "matrix-nio", specifier = ">=0.25,<0.26" },{ name = "mattermostdriver", specifier = ">=7.3.2" },{ name = "pydantic", specifier = ">=2,<3" },{ name = "pydantic-settings", specifier = ">=2.14.2,<3.0" },@@ -2564,15 +2451,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/3b/25/2c87754f3a9e692315f7b811244090e68f362979fc8886b3fbd2985a1d8c/uncalled_for-0.3.2-py3-none-any.whl", hash = "sha256:0ff60b142c7d1f8070bde9d42afaa70aedc77dcc10998c227687e9c15713418e", size = 11444, upload-time = "2026-05-06T13:38:24.025Z" },]-[[package]]-name = "unpaddedbase64"-version = "2.1.0"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/4d/f8/114266b21a7a9e3d09b352bb63c9d61d918bb7aa35d08c722793bfbfd28f/unpaddedbase64-2.1.0.tar.gz", hash = "sha256:7273c60c089de39d90f5d6d4a7883a79e319dc9d9b1c8924a7fab96178a5f005", size = 5621, upload-time = "2021-03-09T11:35:47.729Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/4c/a7/563b2d8fb7edc07320bf69ac6a7eedcd7a1a9d663a6bb90a4d9bd2eda5f7/unpaddedbase64-2.1.0-py3-none-any.whl", hash = "sha256:485eff129c30175d2cd6f0cd8d2310dff51e666f7f36175f738d75dfdbd0b1c6", size = 6083, upload-time = "2021-03-09T11:35:46.7Z" },-]-[[package]]name = "urllib3"version = "2.7.0"diff --git a/deploy/local/docker-compose.yml b/deploy/local/docker-compose.ymlindex 97255f60b..fc4024226 100644--- a/deploy/local/docker-compose.yml+++ b/deploy/local/docker-compose.yml@@ -29,22 +29,6 @@ services:postgres:condition: service_healthy- tuwunel:- image: jevolk/tuwunel:v1.7.1- ports:- - "${SWITCH_BIND_ADDR:-127.0.0.1}:8008:8008"- volumes:- - tuwuneldata:/var/lib/tuwunel- environment:- TUWUNEL_SERVER_NAME: localhost- TUWUNEL_DATABASE_PATH: /var/lib/tuwunel- TUWUNEL_ADDRESS: 0.0.0.0- TUWUNEL_PORT: "8008"- TUWUNEL_ALLOW_FEDERATION: "false"- TUWUNEL_ALLOW_REGISTRATION: "false"- TUWUNEL_REGISTRATION_SHARED_SECRET: ${MATRIX_REGISTRATION_SHARED_SECRET}- TUWUNEL_MAX_REQUEST_SIZE: "20000000"-mattermost:image: mattermost/mattermost-team-edition:latestplatform: linux/amd64@@ -94,4 +78,3 @@ services:volumes:pgdata:mmdata:- tuwuneldata:diff --git a/deploy/local/standalone-docker-compose.yml b/deploy/local/standalone-docker-compose.ymlindex bf1ec8243..84ee22d13 100644--- a/deploy/local/standalone-docker-compose.yml+++ b/deploy/local/standalone-docker-compose.yml@@ -10,7 +10,7 @@# docker compose -f standalone-docker-compose.yml -f standalone-docker-compose.build.yml up## Profiles keep optional services opt-in:-# (core, no profile) postgres, tuwunel, backfill, switch — always started+# (core, no profile) postgres, switch — always started# collab init-db, mattermost, setup — Mattermost bridge + seeder# gateway gateway — operator dashboard# Select them with `--profile collab --profile gateway` or COMPOSE_PROFILES.@@ -55,6 +55,10 @@ services:POSTGRES_PASSWORD: ${DB_PASSWORD}POSTGRES_DB: ${DB_NAME}volumes:+ # Kept deliberately after Tuwunel was removed. It still holds every media+ # blob and the raw history, and it is the only copy — the backfill moved the+ # messages into Postgres but not the files. Dropping it belongs to a later+ # release than the one that stops reading it.- pgdata:/var/lib/postgresql/datahealthcheck:test: ["CMD-SHELL", "pg_isready -U ${DB_USER}"]@@ -89,20 +93,6 @@ services:postgres:condition: service_healthy- tuwunel:- image: jevolk/tuwunel:v1.7.1- volumes:- - tuwuneldata:/var/lib/tuwunel- environment:- TUWUNEL_SERVER_NAME: localhost- TUWUNEL_DATABASE_PATH: /var/lib/tuwunel- TUWUNEL_ADDRESS: 0.0.0.0- TUWUNEL_PORT: "8008"- TUWUNEL_ALLOW_FEDERATION: "false"- TUWUNEL_ALLOW_REGISTRATION: "false"- TUWUNEL_REGISTRATION_SHARED_SECRET: ${MATRIX_REGISTRATION_SHARED_SECRET}- TUWUNEL_MAX_REQUEST_SIZE: "20000000"-mattermost:# Pinned rather than `latest`: Switch Console embeds this Mattermost's web UI# and depends on its DOM structure, so an unattended upgrade could break@@ -115,6 +105,10 @@ services:ports:- "${SWITCH_BIND_ADDR:-127.0.0.1}:${MATTERMOST_HOST_PORT:-8065}:8065"volumes:+ # Kept deliberately after Tuwunel was removed. It still holds every media+ # blob and the raw history, and it is the only copy — the backfill moved the+ # messages into Postgres but not the files. Dropping it belongs to a later+ # release than the one that stops reading it.- mmdata:/mattermost/dataenvironment:MM_SQLSETTINGS_DRIVERNAME: postgres@@ -142,54 +136,6 @@ services:init-db:condition: service_completed_successfully- # One-shot: reconstructs each room's history from the homeserver into the- # message log, then exits. switch waits for it to finish, so the copy is- # never racing the server that will serve it.- #- # It always exits 0, so a backfill that fails delays the stack rather than- # keeping it down. The trade is deliberate and it is a real one: a failure- # here is only visible in this service's log, and the stack above it will- # look healthy while missing history. Backfill is idempotent, so re-running- # it after fixing the cause costs nothing.- #- # It runs alembic first for ordering, not as a second migration owner: the- # rows it writes need the schema, and switch (which normally migrates on- # boot) is waiting on this. The upgrade is the same idempotent one.- #- # Rooms already walked to their start are skipped, so this is cheap on every- # boot after the first. Nothing to walk at all is success too — a fresh- # stack has no rooms and no admin client.- backfill:- image: ${SWITCH_REGISTRY:-ghcr.io}/${SWITCH_IMAGE_NAMESPACE:-sandbox-quantum}/switch-core:${SWITCH_VERSION:?required, set it in .env, there is no default}- restart: "no"- environment:- DB_HOST: postgres- DB_PORT: "5432"- DB_USER: ${DB_USER}- DB_PASSWORD: ${DB_PASSWORD}- DB_NAME: ${DB_NAME}- MATRIX_SERVER: http://tuwunel:8008- MATRIX_SERVER_NAME: localhost- MATRIX_ADMIN_USER: ${MATRIX_ADMIN_USER}- MATRIX_ADMIN_PASSWORD: ${MATRIX_ADMIN_PASSWORD}- MATRIX_REGISTRATION_SHARED_SECRET: ${MATRIX_REGISTRATION_SHARED_SECRET}- AGENT_REGISTRATION_TOKEN: ${AGENT_REGISTRATION_TOKEN}- JWT_SECRET_KEY: ${JWT_SECRET_KEY}- GATEWAY_ADMIN_EMAIL: ${GATEWAY_ADMIN_EMAIL}- GATEWAY_ADMIN_PASSWORD: ${GATEWAY_ADMIN_PASSWORD}- entrypoint: >- sh -c "- alembic upgrade head &&- python -m switch_core.cli.backfill --allow-empty ||- echo 'backfill did not complete; starting anyway. Room history from- before this release may be missing until it is re-run.' >&2- "- depends_on:- postgres:- condition: service_healthy- tuwunel:- condition: service_started-switch:image: ${SWITCH_REGISTRY:-ghcr.io}/${SWITCH_IMAGE_NAMESPACE:-sandbox-quantum}/switch-core:${SWITCH_VERSION:?required, set it in .env, there is no default}ports:@@ -200,11 +146,7 @@ services:DB_USER: ${DB_USER}DB_PASSWORD: ${DB_PASSWORD}DB_NAME: ${DB_NAME}- MATRIX_SERVER: http://tuwunel:8008MATRIX_SERVER_NAME: localhost- MATRIX_ADMIN_USER: ${MATRIX_ADMIN_USER}- MATRIX_ADMIN_PASSWORD: ${MATRIX_ADMIN_PASSWORD}- MATRIX_REGISTRATION_SHARED_SECRET: ${MATRIX_REGISTRATION_SHARED_SECRET}AGENT_REGISTRATION_TOKEN: ${AGENT_REGISTRATION_TOKEN}JWT_SECRET_KEY: ${JWT_SECRET_KEY}GATEWAY_ADMIN_EMAIL: ${GATEWAY_ADMIN_EMAIL}@@ -214,10 +156,6 @@ services:depends_on:postgres:condition: service_healthy- tuwunel:- condition: service_started- backfill:- condition: service_completed_successfullygateway:image: ${SWITCH_REGISTRY:-ghcr.io}/${SWITCH_IMAGE_NAMESPACE:-sandbox-quantum}/gateway:${SWITCH_VERSION:?required, set it in .env, there is no default}@@ -247,6 +185,10 @@ services:- switchvolumes:+ # Kept deliberately after Tuwunel was removed. It still holds every media+ # blob and the raw history, and it is the only copy — the backfill moved the+ # messages into Postgres but not the files. Dropping it belongs to a later+ # release than the one that stops reading it.pgdata:mmdata:tuwuneldata:diff --git a/deploy/remote/helm/switch/BACKUP.md b/deploy/remote/helm/switch/BACKUP.mdindex 3963dbb13..50aea6e60 100644--- a/deploy/remote/helm/switch/BACKUP.md+++ b/deploy/remote/helm/switch/BACKUP.md@@ -10,15 +10,11 @@ tools like **Velero**. Use those. This doc says what to back up and how.| Data | Where | Why it matters ||------|-------|----------------|-| **Tuwunel volume** | PVC `<release>-tuwunel-data` | Holds the Matrix **signing key** — your homeserver's cryptographic identity. Lose it and every `@user:server` account is orphaned and the homeserver can never be restored or federate as itself again. **This is the single most important thing to back up.** |-| **Postgres** | PVC `<release>-postgresql-data` (managed mode) or your external DB | All Switch application state + the Mattermost database. |+| **Postgres** | PVC `<release>-postgresql-data` (managed mode) or your external DB | Everything: rooms, messages, attachments, accounts, and the Mattermost database. **This is the single most important thing to back up.** || Mattermost files | PVC `<release>-mattermost-data` | Uploaded files/attachments (if you use the Mattermost bridge). |`<release>` is your Helm release name (`kubectl get pvc` to see the real names).-Tuwunel runs an embedded RocksDB on a ReadWriteOnce volume; treat it as a-database, not just files.-## Option 1 — CSI VolumeSnapshots (recommended)If your cluster has a CSI driver with snapshot support (EKS/GKE/AKS and most@@ -26,14 +22,6 @@ managed clusters do) and the external-snapshotter installed, snapshot the PVCsdirectly. This is point-in-time and consistent, and does not fight theReadWriteOnce mount. See `samples/volumesnapshot.example.yaml`.-For the strongest Tuwunel consistency, scale it down first so RocksDB flushes:--```sh-kubectl scale deploy/<release>-tuwunel --replicas=0-# take the VolumeSnapshot of <release>-tuwunel-data-kubectl scale deploy/<release>-tuwunel --replicas=1-```-Automate snapshots on a schedule with a tool like[snapscheduler](https://backube.github.io/snapscheduler/) or Velero.@@ -60,5 +48,6 @@ the most complete DR story. Back up this release's namespace on a schedule.1. Restore/attach the PVs (from VolumeSnapshot or Velero) **before** installing,or install with persistence pointing at the restored volumes.2. For a Postgres logical dump: `gunzip -c dump.sql.gz | psql -U postgres`.-3. Keep `tuwunel.serverName` **identical** to the original — the signing key is- bound to it; changing it orphans all accounts even with the data restored.+3. Keep `clientIdentity.serverName` **identical** to the original — every+ client id embeds it, so changing it orphans all accounts even with the data+ restored.diff --git a/deploy/remote/helm/switch/README.md b/deploy/remote/helm/switch/README.mdindex af58ab91d..88e447e79 100644--- a/deploy/remote/helm/switch/README.md+++ b/deploy/remote/helm/switch/README.md@@ -1,7 +1,7 @@# Switch Helm chartDeploys Switch into a Kubernetes cluster: switch-core (the API and all bridge-adapters), the gateway dashboard, a Tuwunel Matrix homeserver, Postgres, and+adapters), the gateway dashboard, Postgres, andoptionally Mattermost.```bashdiff --git a/deploy/remote/helm/switch/samples/volumesnapshot.example.yaml b/deploy/remote/helm/switch/samples/volumesnapshot.example.yamlindex 14f0e6edf..4164b6bde 100644--- a/deploy/remote/helm/switch/samples/volumesnapshot.example.yaml+++ b/deploy/remote/helm/switch/samples/volumesnapshot.example.yaml@@ -7,24 +7,12 @@# RELEASE -> your Helm release name (kubectl get pvc to confirm)# csi-snapclass -> your VolumeSnapshotClass (kubectl get volumesnapshotclass)#-# For the strongest Tuwunel consistency, scale it to 0 before snapshotting:-# kubectl scale deploy/RELEASE-tuwunel --replicas=0# kubectl apply -f this-file-# kubectl scale deploy/RELEASE-tuwunel --replicas=1## These are one-off snapshots. For scheduled snapshots use snapscheduler or Velero.---apiVersion: snapshot.storage.k8s.io/v1kind: VolumeSnapshot-metadata:- name: tuwunel-snapshot # the Matrix signing key — the critical one-spec:- volumeSnapshotClassName: csi-snapclass- source:- persistentVolumeClaimName: RELEASE-tuwunel-data-----apiVersion: snapshot.storage.k8s.io/v1-kind: VolumeSnapshotmetadata:name: postgresql-snapshotspec:diff --git a/deploy/remote/helm/switch/templates/NOTES.txt b/deploy/remote/helm/switch/templates/NOTES.txtindex 64cd6aef7..991e442ad 100644--- a/deploy/remote/helm/switch/templates/NOTES.txt+++ b/deploy/remote/helm/switch/templates/NOTES.txt@@ -14,8 +14,6 @@ To access services via port-forward:# Mattermostkubectl port-forward svc/{{ include "switch.fullname" . }}-mattermost 8065:8065 -n {{ .Release.Namespace }}- # Tuwunel (Matrix homeserver)- kubectl port-forward svc/{{ include "switch.fullname" . }}-tuwunel 8008:8008 -n {{ .Release.Namespace }}Or use `just pf-all` to forward all services at once.{{ if eq .Values.ingress.mode "managed" }}@@ -93,9 +91,7 @@ in-cluster half. See docs/bridges/TEAMS_SETUP.md.{{- end }}⚠ BACK UP YOUR DATA — this chart ships no backup jobs.-The Tuwunel volume ({{ include "switch.fullname" . }}-tuwunel-data) holds the-Matrix signing key: your homeserver's identity. Lose it and every account is-orphaned and this server can never be restored — so back it up. Also back up-Postgres (all app + Mattermost state). See BACKUP.md and+Postgres holds everything: every room, every message, every account, and the+Mattermost database. Back it up. See BACKUP.md andsamples/volumesnapshot.example.yaml for the recommended approaches(CSI VolumeSnapshots, pg_dump / managed-DB backups, Velero).diff --git a/deploy/remote/helm/switch/templates/_helpers.tpl b/deploy/remote/helm/switch/templates/_helpers.tplindex fcc6a5dbf..f5fdb0b9c 100644--- a/deploy/remote/helm/switch/templates/_helpers.tpl+++ b/deploy/remote/helm/switch/templates/_helpers.tpl@@ -106,10 +106,6 @@ Other service hostnames (used in env vars and init containers).{{- include "switch.fullname" . }}-switch-core{{- end }}-{{- define "switch.tuwunelHost" -}}-{{- include "switch.fullname" . }}-tuwunel-{{- end }}-{{- define "switch.gatewayHost" -}}{{- include "switch.fullname" . }}-gateway{{- end }}@@ -239,22 +235,8 @@ Include with `nindent 12`.{{- end }}- name: AGENT_AUTH_CACHE_TTL_SECONDSvalue: {{ .Values.switchCore.authCache.ttlSeconds | quote }}-- name: MATRIX_SERVER- value: "http://{{ include "switch.tuwunelHost" . }}:8008"- name: MATRIX_SERVER_NAME- value: {{ .Values.tuwunel.serverName | quote }}-- name: MATRIX_ADMIN_USER- value: admin-- name: MATRIX_ADMIN_PASSWORD- valueFrom:- secretKeyRef:- name: {{ include "switch.secretName" . }}- key: MATRIX_ADMIN_PASSWORD-- name: MATRIX_REGISTRATION_SHARED_SECRET- valueFrom:- secretKeyRef:- name: {{ include "switch.secretName" . }}- key: MATRIX_REGISTRATION_SHARED_SECRET+ value: {{ .Values.clientIdentity.serverName | quote }}- name: AGENT_REGISTRATION_TOKENvalueFrom:secretKeyRef:@@ -330,7 +312,7 @@ Usage: {{ include "switch.waitFor" (dict "name" "postgres" "host" (include "swit{{/*Wait-for-http init container template.-Usage: {{ include "switch.waitForHttp" (dict "name" "tuwunel" "url" "http://host:8008/_matrix/client/versions") }}+Usage: {{ include "switch.waitForHttp" (dict "name" "gateway" "url" "http://host:8000/health") }}*/}}{{- define "switch.waitForHttp" -}}- name: wait-for-{{ .name }}diff --git a/deploy/remote/helm/switch/templates/common/secret.yaml b/deploy/remote/helm/switch/templates/common/secret.yamlindex d55041195..71ae3bf85 100644--- a/deploy/remote/helm/switch/templates/common/secret.yaml+++ b/deploy/remote/helm/switch/templates/common/secret.yaml@@ -10,8 +10,6 @@ data:{{- if not .Values.postgresql.existingSecret }}POSTGRES_PASSWORD: {{ required "secrets.postgresPassword is required (unless postgresql.existingSecret is set)" .Values.secrets.postgresPassword | b64enc | quote }}{{- end }}- MATRIX_REGISTRATION_SHARED_SECRET: {{ required "secrets.matrixRegistrationSharedSecret is required" .Values.secrets.matrixRegistrationSharedSecret | b64enc | quote }}- MATRIX_ADMIN_PASSWORD: {{ required "secrets.matrixAdminPassword is required" .Values.secrets.matrixAdminPassword | b64enc | quote }}AGENT_REGISTRATION_TOKEN: {{ required "secrets.agentRegistrationToken is required" .Values.secrets.agentRegistrationToken | b64enc | quote }}JWT_SECRET_KEY: {{ required "secrets.jwtSecretKey is required" .Values.secrets.jwtSecretKey | b64enc | quote }}GATEWAY_ADMIN_EMAIL: {{ required "secrets.gatewayAdminEmail is required" .Values.secrets.gatewayAdminEmail | b64enc | quote }}diff --git a/deploy/remote/helm/switch/templates/retained/tuwunel-data-pvc.yaml b/deploy/remote/helm/switch/templates/retained/tuwunel-data-pvc.yamlnew file mode 100644index 000000000..89bb1dd85--- /dev/null+++ b/deploy/remote/helm/switch/templates/retained/tuwunel-data-pvc.yaml@@ -0,0 +1,29 @@+{{- if .Values.retainTuwunelData }}+# The homeserver is gone; its disk is not, deliberately.+#+# It still holds every media blob and the raw history. The backfill moved the+# messages into Postgres but not the files, so this is the only copy of them —+# and deleting the template is what deletes the volume, since Helm prunes an+# object whose manifest disappears. Dropping it therefore belongs to a later+# release than the one that stopped reading it, once whoever owns the data has+# decided they do not want it.+#+# Nothing mounts this. It exists to not be destroyed.+apiVersion: v1+kind: PersistentVolumeClaim+metadata:+ name: {{ include "switch.fullname" . }}-tuwunel-data+ labels:+ {{- include "switch.labels" . | nindent 4 }}+ annotations:+ "helm.sh/resource-policy": keep+spec:+ accessModes:+ - ReadWriteOnce+ {{- if .Values.global.storageClass }}+ storageClassName: {{ .Values.global.storageClass }}+ {{- end }}+ resources:+ requests:+ storage: {{ .Values.retainedTuwunelDataSize }}+{{- end }}diff --git a/deploy/remote/helm/switch/templates/switch-core/backfill-job.yaml b/deploy/remote/helm/switch/templates/switch-core/backfill-job.yamldeleted file mode 100644index 8c3015852..000000000--- a/deploy/remote/helm/switch/templates/switch-core/backfill-job.yaml+++ /dev/null@@ -1,58 +0,0 @@-{{- if .Values.switchCore.backfill.enabled }}-# Reconstructs each room's history from the homeserver into the message log.-#-# Off by default and deliberately not part of a normal release: it is a one-off-# for the move onto the Postgres transport, and as a routine hook it would-# re-walk every room's entire history on every upgrade.-#-# It reads the homeserver, so it can only run while Tuwunel is still deployed —-# and it is the last thing that needs it. Run it with `dryRun: true` first and-# read the log before letting it write.-#-# Ordered after the migrate hook (weight 10 vs 0): it writes into `messages`,-# which alembic has to have created.-apiVersion: batch/v1-kind: Job-metadata:- name: {{ include "switch.fullname" . }}-backfill- labels:- {{- include "switch.labels" . | nindent 4 }}- {{- include "switch.selectorLabels" (dict "Release" .Release "component" "backfill") | nindent 4 }}- annotations:- "helm.sh/hook": post-upgrade- "helm.sh/hook-weight": "10"- "helm.sh/hook-delete-policy": before-hook-creation-spec:- # Backfill is idempotent — a unique transport_event_id and a commit per page,- # so a retry resumes rather than duplicates.- backoffLimit: 3- template:- metadata:- labels:- {{- include "switch.labels" . | nindent 8 }}- {{- include "switch.selectorLabels" (dict "Release" .Release "component" "backfill") | nindent 8 }}- spec:- {{- if .Values.global.imagePullSecrets }}- {{- include "switch.imagePullSecrets" . | nindent 6 }}- {{- end }}- restartPolicy: OnFailure- initContainers:- {{- include "switch.waitFor" (dict "name" "postgres" "host" (include "switch.postgresHost" .) "port" (include "switch.postgresPort" .)) | nindent 8 }}- {{- include "switch.waitFor" (dict "name" "tuwunel" "host" (include "switch.tuwunelHost" .) "port" .Values.tuwunel.service.port) | nindent 8 }}- containers:- - name: backfill- image: {{ include "switch.image" (dict "global" .Values.global "image" .Values.switchCore.image) }}- imagePullPolicy: {{ .Values.switchCore.imagePullPolicy }}- command:- - python- - -m- - switch_core.cli.backfill- {{- if .Values.switchCore.backfill.dryRun }}- - --dry-run- {{- end }}- {{- if .Values.switchCore.backfill.allowEmpty }}- - --allow-empty- {{- end }}- env:- {{- include "switch.coreEnv" . | nindent 12 }}-{{- end }}diff --git a/deploy/remote/helm/switch/templates/switch-core/deployment.yaml b/deploy/remote/helm/switch/templates/switch-core/deployment.yamlindex e83682727..a84018e04 100644--- a/deploy/remote/helm/switch/templates/switch-core/deployment.yaml+++ b/deploy/remote/helm/switch/templates/switch-core/deployment.yaml@@ -30,7 +30,6 @@ spec:{{- end }}initContainers:{{- include "switch.waitFor" (dict "name" "postgres" "host" (include "switch.postgresHost" .) "port" (include "switch.postgresPort" .)) | nindent 8 }}- {{- include "switch.waitForHttp" (dict "name" "tuwunel" "url" (printf "http://%s:8008/_matrix/client/versions" (include "switch.tuwunelHost" .))) | nindent 8 }}containers:- name: switch-coreimage: {{ include "switch.image" (dict "global" .Values.global "image" .Values.switchCore.image) }}diff --git a/deploy/remote/helm/switch/templates/switch-core/reconcile-job.yaml b/deploy/remote/helm/switch/templates/switch-core/reconcile-job.yamldeleted file mode 100644index 7c8feb5c9..000000000--- a/deploy/remote/helm/switch/templates/switch-core/reconcile-job.yaml+++ /dev/null@@ -1,43 +0,0 @@-{{- if .Values.switchCore.reconcile.enabled }}-# Compares the recorded messages against what the homeserver holds, and exits-# non-zero on drift. The check to run after a backfill and before flipping the-# transport, because afterwards there is only one record and nothing to compare-# it against.-#-# Off by default, for the same reason as the backfill job: this is a step in a-# migration, not part of a release.-apiVersion: batch/v1-kind: Job-metadata:- name: {{ include "switch.fullname" . }}-reconcile- labels:- {{- include "switch.labels" . | nindent 4 }}- {{- include "switch.selectorLabels" (dict "Release" .Release "component" "reconcile") | nindent 4 }}- annotations:- "helm.sh/hook": post-upgrade- "helm.sh/hook-weight": "20"- "helm.sh/hook-delete-policy": before-hook-creation-spec:- # Read-only, so a retry costs time and nothing else.- backoffLimit: 1- template:- metadata:- labels:- {{- include "switch.labels" . | nindent 8 }}- {{- include "switch.selectorLabels" (dict "Release" .Release "component" "reconcile") | nindent 8 }}- spec:- {{- if .Values.global.imagePullSecrets }}- {{- include "switch.imagePullSecrets" . | nindent 6 }}- {{- end }}- restartPolicy: Never- initContainers:- {{- include "switch.waitFor" (dict "name" "postgres" "host" (include "switch.postgresHost" .) "port" (include "switch.postgresPort" .)) | nindent 8 }}- {{- include "switch.waitFor" (dict "name" "tuwunel" "host" (include "switch.tuwunelHost" .) "port" .Values.tuwunel.service.port) | nindent 8 }}- containers:- - name: reconcile- image: {{ include "switch.image" (dict "global" .Values.global "image" .Values.switchCore.image) }}- imagePullPolicy: {{ .Values.switchCore.imagePullPolicy }}- command: ["python", "-m", "switch_core.cli.reconcile", "--verbose"]- env:- {{- include "switch.coreEnv" . | nindent 12 }}-{{- end }}diff --git a/deploy/remote/helm/switch/templates/tuwunel/deployment.yaml b/deploy/remote/helm/switch/templates/tuwunel/deployment.yamldeleted file mode 100644index c793bb52a..000000000--- a/deploy/remote/helm/switch/templates/tuwunel/deployment.yaml+++ /dev/null@@ -1,78 +0,0 @@-apiVersion: apps/v1-kind: Deployment-metadata:- name: {{ include "switch.fullname" . }}-tuwunel- labels:- {{- include "switch.labels" . | nindent 4 }}- {{- include "switch.selectorLabels" (dict "Release" .Release "component" "tuwunel") | nindent 4 }}-spec:- replicas: 1- # Embedded RocksDB on a RWO volume — never run two pods against it at once.- strategy:- type: Recreate- selector:- matchLabels:- {{- include "switch.selectorLabels" (dict "Release" .Release "component" "tuwunel") | nindent 6 }}- template:- metadata:- labels:- {{- include "switch.labels" . | nindent 8 }}- {{- include "switch.selectorLabels" (dict "Release" .Release "component" "tuwunel") | nindent 8 }}- spec:- {{- if .Values.global.imagePullSecrets }}- {{- include "switch.imagePullSecrets" . | nindent 6 }}- {{- end }}- {{- with .Values.tuwunel.podSecurityContext }}- securityContext:- {{- toYaml . | nindent 8 }}- {{- end }}- containers:- - name: tuwunel- image: {{ .Values.tuwunel.image }}- imagePullPolicy: {{ .Values.tuwunel.imagePullPolicy }}- ports:- - containerPort: 8008- protocol: TCP- env:- # Every Matrix user id is @localpart:<server_name>, so this must not- # change on an existing deployment or all accounts are orphaned.- - name: TUWUNEL_SERVER_NAME- value: {{ .Values.tuwunel.serverName | quote }}- - name: TUWUNEL_DATABASE_PATH- value: /var/lib/tuwunel- - name: TUWUNEL_ADDRESS- value: "0.0.0.0"- - name: TUWUNEL_PORT- value: "8008"- - name: TUWUNEL_ALLOW_FEDERATION- value: {{ .Values.tuwunel.allowFederation | quote }}- - name: TUWUNEL_ALLOW_REGISTRATION- value: {{ .Values.tuwunel.allowRegistration | quote }}- - name: TUWUNEL_REGISTRATION_SHARED_SECRET- valueFrom:- secretKeyRef:- name: {{ include "switch.secretName" . }}- key: MATRIX_REGISTRATION_SHARED_SECRET- - name: TUWUNEL_MAX_REQUEST_SIZE- value: {{ .Values.tuwunel.maxRequestSize | quote }}- volumeMounts:- - name: data- mountPath: /var/lib/tuwunel- readinessProbe:- httpGet:- path: /_matrix/client/versions- port: 8008- initialDelaySeconds: 5- periodSeconds: 10- livenessProbe:- httpGet:- path: /_matrix/client/versions- port: 8008- initialDelaySeconds: 20- periodSeconds: 30- resources:- {{- toYaml .Values.tuwunel.resources | nindent 12 }}- volumes:- - name: data- persistentVolumeClaim:- claimName: {{ include "switch.fullname" . }}-tuwunel-datadiff --git a/deploy/remote/helm/switch/templates/tuwunel/pvc.yaml b/deploy/remote/helm/switch/templates/tuwunel/pvc.yamldeleted file mode 100644index 9d6a40e08..000000000--- a/deploy/remote/helm/switch/templates/tuwunel/pvc.yaml+++ /dev/null@@ -1,15 +0,0 @@-apiVersion: v1-kind: PersistentVolumeClaim-metadata:- name: {{ include "switch.fullname" . }}-tuwunel-data- labels:- {{- include "switch.labels" . | nindent 4 }}-spec:- accessModes:- - ReadWriteOnce- {{- if .Values.global.storageClass }}- storageClassName: {{ .Values.global.storageClass }}- {{- end }}- resources:- requests:- storage: {{ .Values.tuwunel.persistence.size }}diff --git a/deploy/remote/helm/switch/templates/tuwunel/service.yaml b/deploy/remote/helm/switch/templates/tuwunel/service.yamldeleted file mode 100644index 170aa8c45..000000000--- a/deploy/remote/helm/switch/templates/tuwunel/service.yaml+++ /dev/null@@ -1,16 +0,0 @@-apiVersion: v1-kind: Service-metadata:- name: {{ include "switch.tuwunelHost" . }}- labels:- {{- include "switch.labels" . | nindent 4 }}- {{- include "switch.selectorLabels" (dict "Release" .Release "component" "tuwunel") | nindent 4 }}-spec:- type: {{ .Values.tuwunel.service.type }}- ports:- - port: {{ .Values.tuwunel.service.port }}- targetPort: 8008- protocol: TCP- name: http- selector:- {{- include "switch.selectorLabels" (dict "Release" .Release "component" "tuwunel") | nindent 4 }}diff --git a/deploy/remote/helm/switch/values.yaml b/deploy/remote/helm/switch/values.yamlindex 518fce663..c64a62ca5 100644--- a/deploy/remote/helm/switch/values.yaml+++ b/deploy/remote/helm/switch/values.yaml@@ -67,7 +67,6 @@ ingress:# chart then renders no Secret and the values below are ignored. That Secret# must contain these keys:# POSTGRES_PASSWORD (unless postgresql.existingSecret is set),-# MATRIX_REGISTRATION_SHARED_SECRET, MATRIX_ADMIN_PASSWORD,# AGENT_REGISTRATION_TOKEN, JWT_SECRET_KEY, GATEWAY_ADMIN_EMAIL,# GATEWAY_ADMIN_PASSWORD,# MATTERMOST_ADMIN_PASSWORD + MATTERMOST_USER_PASSWORD (if mattermost.enabled),@@ -77,8 +76,6 @@ secrets:# chart manages its own Secret from the values below.existingSecret: ""postgresPassword: ""- matrixRegistrationSharedSecret: ""- matrixAdminPassword: ""agentRegistrationToken: ""jwtSecretKey: ""gatewayAdminEmail: ""@@ -197,9 +194,12 @@ switchCore:# Secret holding the cert; cert-manager creates it from the annotation# above, or reference a pre-created TLS Secret.secretName: ""- # switch-core keeps live Matrix sync sessions in memory, so it MUST run as a- # single replica — a second replica would double every client and split- # session state. This is enforced at render time; do not raise it.+ # switch-core MUST run as a single replica. Four things live in the process+ # rather than the database: the per-agent event buffer, the invite bus, the+ # ephemeral (presence) bus, and the message listener's subscriber registry. A+ # second replica would hold its own copy of each, so a signal raised in one+ # would never reach a client attached to the other. This is enforced at+ # render time; do not raise it.replicaCount: 1# Run DB migrations (alembic upgrade head) as a pre-upgrade hook Job, before# the new pods roll, so a failed migration blocks the upgrade instead of@@ -208,26 +208,6 @@ switchCore:# idempotent. Disable only if you run migrations out-of-band.migrations:preUpgradeJob: true- # One-off migration steps for the move onto the Postgres message store. Both- # read the homeserver, so they only work while Tuwunel is still deployed, and- # both are off by default: they belong to one release, not to every release.- #- # 1. enable backfill with dryRun: true, upgrade, read the Job's log- # 2. set dryRun: false, upgrade again- # 3. enable reconcile, upgrade, confirm it exits clean- # 4. set MESSAGE_TRANSPORT=postgres- # 5. turn both back off- #- # Backfill is idempotent and its rows are numbered below the live sequence,- # so it can be re-run and nobody is redelivered old history.- backfill:- enabled: false- dryRun: true- # Exit 0 when there is no history to walk. For an install with no rooms- # yet, where a failed Job would be reporting an empty database as a fault.- allowEmpty: false- reconcile:- enabled: falseresources:requests:memory: "512Mi"@@ -252,32 +232,23 @@ gateway:memory: "128Mi"cpu: "200m"-# ── Tuwunel (Matrix homeserver) ───────────────────────────────────────────────-# The internal message bus. switch-core points here. RocksDB-backed (no Postgres-# dependency); data lives on its own PVC.-tuwunel:- image: jevolk/tuwunel:v1.7.1- imagePullPolicy: IfNotPresent- # Every Matrix user id is @localpart:<serverName>, so changing this on an- # existing deployment orphans all accounts.+# ── Retained homeserver disk ──────────────────────────────────────────────────+# Switch no longer runs a homeserver, but its volume still holds every media+# blob and the raw history — the backfill moved the messages into Postgres and+# not the files, so this is their only copy. Left claimed, and annotated+# keep, so an upgrade does not destroy it. Set false once you have decided you+# do not want that data; there is no getting it back.+retainTuwunelData: true+# Must match what the volume was created with. Shrinking a PVC is rejected.+retainedTuwunelDataSize: 10Gi++# ── Client identity ───────────────────────────────────────────────────────────+# The server half of every client id (@localpart:<serverName>). Nothing is+# contacted at it — it is a naming scheme, not an address — but the ids are+# stable public handles, so changing it on an existing deployment orphans every+# account.+clientIdentity:serverName: "switch.local"- allowFederation: false- allowRegistration: false- maxRequestSize: "20000000"- # Set if the image runs as non-root and can't write the PVC, e.g. {fsGroup: 1000}.- podSecurityContext: {}- service:- type: ClusterIP- port: 8008- persistence:- size: 10Gi- resources:- requests:- memory: "512Mi"- cpu: "250m"- limits:- memory: "2Gi"- cpu: "1000m"# ── Mattermost ───────────────────────────────────────────────────────────────mattermost:@@ -404,5 +375,5 @@ postgresql:# This chart ships no backup jobs on purpose — an on-cluster backup shares the# storage backend it is meant to protect, and every target already has stronger,# portable primitives (CSI VolumeSnapshots, managed-DB backups, Velero). See-# BACKUP.md for what to back up (the Tuwunel volume holds the Matrix signing key+# BACKUP.md for what to back up (the Postgres volume holds every room and message# — the irreplaceable server identity) and how, plus samples/volumesnapshot.example.yaml.diff --git a/gateway/src/data/api.ts b/gateway/src/data/api.tsindex 5d6ca9106..9838d219a 100644--- a/gateway/src/data/api.ts+++ b/gateway/src/data/api.ts@@ -157,6 +157,8 @@ export interface RoomRoleDetail extends RoomRoleSpec {}export interface RoomDetail extends RoomSummary {+ transport_room_id: string;+ /** @deprecated Alias of `transport_room_id`, carried for the compatibility window. */matrix_room_id: string;external_channel_id: string | null;instructions: string | null;diff --git a/gateway/src/pages/rooms/RoomDetailPage.tsx b/gateway/src/pages/rooms/RoomDetailPage.tsxindex ba74109a4..b68f6212f 100644--- a/gateway/src/pages/rooms/RoomDetailPage.tsx+++ b/gateway/src/pages/rooms/RoomDetailPage.tsx@@ -192,7 +192,7 @@ function RoomInfoSection({ room }: { room: RoomDetail }) {/>)}<InfoChip label="Created" value={formatDateTime(room.created_at)} />- <InfoChip label="Matrix room" value={room.matrix_room_id} mono />+ <InfoChip label="Transport room" value={room.transport_room_id} mono />{room.external_channel_id && (<InfoChiplabel="External channel"diff --git a/justfile b/justfileindex 2fa7cf126..1c2b39dbb 100644--- a/justfile+++ b/justfile@@ -26,7 +26,7 @@ init-env:exit 1ficp .env.example .env- for key in DB_PASSWORD MATRIX_ADMIN_PASSWORD MATRIX_REGISTRATION_SHARED_SECRET \+ for key in DB_PASSWORD \AGENT_REGISTRATION_TOKEN JWT_SECRET_KEY GATEWAY_ADMIN_PASSWORD \MATTERMOST_ADMIN_PASSWORD MATTERMOST_USER_PASSWORD; dosecret="$(openssl rand -hex 24)"@@ -38,8 +38,6 @@ init-env:echo " The stack binds to 127.0.0.1 only (set SWITCH_BIND_ADDR to expose it)."# ── Dev infrastructure ─────────────────────────────────────────────────────────-# Tuwunel self-initializes its signing key + database in its data volume on-# first boot, so no pre-start key generation is needed.up:docker compose -f deploy/local/docker-compose.yml --project-directory . up -d --build@@ -101,24 +99,6 @@ migrate:uv run --project core alembic -c core/alembic.ini upgrade head-# ── Check the recorded messages against the message bus ───────────────────────-# Messages are written to Postgres after the send is accepted, so a database-# problem leaves a row missing for a message that was really delivered. This-# walks each room's history and reports the difference. Exits non-zero on drift.-# Needs a real deployment — the question it answers cannot be asked of fakes.-# just reconcile-messages --room <id> --since <iso8601> --verbose-reconcile-messages *args:- uv run --project core python -m switch_core.cli.reconcile {{ args }}--# ── Reconstruct room history from the bus into the message log ────────────────-# Rows exist only from when the recorder was deployed; everything older is on-# the homeserver and invisible to agents now that reads come from Postgres.-# This walks each room to its start and writes what is missing, numbered below-# the live log so nothing is redelivered. Safe to re-run. Start with --dry-run.-# just backfill-messages --dry-run --room <id>-backfill-messages *args:- uv run --project core python -m switch_core.cli.backfill {{ args }}-# ── Generate a new alembic migration ──────────────────────────────────────────migration msg:uv run --project core alembic -c core/alembic.ini revision --autogenerate -m "{{ msg }}"@@ -127,7 +107,7 @@ migration msg:test *args:uv run --project core pytest -c core/pyproject.toml core/tests/ {{ args }}-# ── Run integration tests (real Postgres + Tuwunel via testcontainers) ──────────+# ── Run integration tests (real Postgres via testcontainers) ───────────────────# DOCKER_HOST is auto-resolved from the active docker context in conftest, so this# works under Docker Desktop / OrbStack / colima without extra setup.test-integration *args: