Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.
Python agent-orchestrationaiai-agentschatopsdiscordlangchainmcpmicrosoft-teamsmulti-agentopen-sourceopenaiself-hostedslack
E2E scenarios for recent behaviours, and delete a room with a package attached
Petr Bauch6cb3eb8803a6
diff --git a/console/tooling-e2e/README.md b/console/tooling-e2e/README.mdindex 5c947cf7f..7759fc615 100644--- a/console/tooling-e2e/README.md+++ b/console/tooling-e2e/README.md@@ -138,6 +138,11 @@ passed or not. A teardown that leaves something behind fails the scenario.| `operations-front-door` | The operations registry over HTTP: it lists the operations, `list_rooms` returns the room, `read_context` returns a message posted in it. | — || `mattermost-roundtrip` | A room on the Mattermost bridge: an addressed post from the human reaches the agent, and the agent's reply appears as its bot. | Mattermost || `agent-identity` | Renaming an agent from the gateway and with `!rename-agent` renames its Mattermost bot in place, and mentions follow the new name; `!delete-agent`, confirmed, deletes it. | Mattermost |+| `room-group-paths` | The same subgroup name under two parents: a bare name both share is refused naming each path, the path picks one, and a second group of that name under one parent is refused whatever its case. | — |+| `attachment-names` | A file whose name is not latin-1 is posted, listed by `read_context` and downloaded under its own name (`filename*` in `Content-Disposition`). | — |+| `client-mistakes` | An outsider inviting itself into a room or reading it gets a 403, and a history timestamp that is not ISO 8601 a 400 naming the parameter; never a 500. | — |+| `room-capture` | A room built from a document keeps the package it names, and the room captured back as a document builds a second room with the same package; both rooms delete. | — |+| `follow-ups` | Follow the conversation: an untagged message continuing an exchange the agent just answered, or replying in its thread, reaches it; one after another agent spoke, or after the window, does not; an agent's message follows no one. | Mattermost; ephemeral only, where the stack turns the flag on with a 20-second window |If `mattermost-roundtrip` passes and a provider-lane scenario fails, the faultis in the agent session, not in Switch, the bridge or the harness.@@ -601,6 +606,8 @@ than being half-used.| `question` | Two turns: the agent asks, the human answers in the channel, the agent uses the answer. | The bot offers `red`/`green`/`blue`, then replies `green` after the answer. || `approval` | A tool call the session is not pre-authorised to make. | The bot posts an approval card, answering it with `<handle> 1` allows the call, the bot replies `done`. || `interrupt` | `!interrupt @agent-name` cancels a running local shell command after its start marker is verified. | No completion marker appears after the command’s 90-second lifetime (observed for 95 seconds). Requires `SWITCH_E2E_AGENT_DIR` on this machine. |+| `card-answer-refused` | With the agent open only to its owner, a person who is not the owner answers its approval card, then the owner does. | The Switch admin bot tells the other person in the card's thread that the answer did not land and the command does not run; after the owner's answer the bot replies `done`. Ephemeral and local only: the other person signs up on the spot. |+| `template-agent` | A template creates a managed agent on the controller and a Mattermost room with it in it. | The run reports the agent on the controller and listening, and it answers `SWITCH_E2E_OK_TPL` in the new channel. Requires an agent controller. |`question` is the one a one-shot reply cannot fake: it only passes if the sessionis still alive and still connected when the second message arrives.diff --git a/console/tooling-e2e/scripts/ephemeral-stack.sh b/console/tooling-e2e/scripts/ephemeral-stack.shindex 1d4f04b24..dd9425b4e 100755--- a/console/tooling-e2e/scripts/ephemeral-stack.sh+++ b/console/tooling-e2e/scripts/ephemeral-stack.sh@@ -40,6 +40,7 @@ LANE="${SWITCH_E2E_LANE:-fast}"# The agentic lane runs on the provider lane's setup: a real session on an# agent controller.NEEDS_CONTROLLER=0+FOLLOW_WINDOW_SECONDS=20case "$LANE" in provider|agentic) NEEDS_CONTROLLER=1 ;; esacKEEP=0VITEST_ARGS=()@@ -109,6 +110,13 @@ if [ "$NEEDS_CONTROLLER" = 1 ]; thensed -i.bak -e "s|^FEATURE_FLAGS_ENABLED=.*|FEATURE_FLAGS_ENABLED=agent_management|" \-e "s|^CONTROLLER_TOKEN_SECRET=.*|CONTROLLER_TOKEN_SECRET=$(openssl rand -hex 32)|" "$ENV_FILE"rm -f "$ENV_FILE.bak"+else+ # Follow-ups, with a window short enough for a scenario to wait out. Not on+ # the controller lanes: there an untagged answer to a request card, or an+ # agentic scenario's unaddressed setup message, would become a follow-up.+ sed -i.bak -e "s|^FEATURE_FLAGS_ENABLED=.*|FEATURE_FLAGS_ENABLED=follow_the_conversation|" "$ENV_FILE"+ rm -f "$ENV_FILE.bak"+ printf 'FOLLOW_THE_CONVERSATION_WINDOW_SECONDS=%s\n' "$FOLLOW_WINDOW_SECONDS" >>"$ENV_FILE"fichmod 600 "$ENV_FILE"set -a; source "$ENV_FILE"; set +a@@ -255,6 +263,8 @@ status=0export SWITCH_E2E_SERVER_LOG="$OUT/switch-core.log"# Screenshots of failed pages, and agentic transcripts and verdicts.export SWITCH_E2E_OUTPUT_DIR="$OUT"+ # The follow-up window the server was started with, for the scenario that waits it out.+ [ "$NEEDS_CONTROLLER" = 0 ] && export SWITCH_E2E_FOLLOW_WINDOW_SECONDS="$FOLLOW_WINDOW_SECONDS"SWITCH_E2E_LANE="$LANE" \SWITCH_E2E_ENVIRONMENT=ephemeral \SWITCH_E2E_STRICT=1 \diff --git a/console/tooling-e2e/src/clients/mattermost-client.ts b/console/tooling-e2e/src/clients/mattermost-client.tsindex cfe067e6d..8b0813acb 100644--- a/console/tooling-e2e/src/clients/mattermost-client.ts+++ b/console/tooling-e2e/src/clients/mattermost-client.ts@@ -102,6 +102,45 @@ export class MattermostClient {return (text === '' ? undefined : JSON.parse(text)) as T;}+ /**+ * Sign a new person up, as anyone may on a server with open sign-up, and+ * log them in. `POST /api/v4/users`, then `POST /api/v4/users/login`, whose+ * `Token` header is the session token the returned client posts with.+ */+ static async signUp(+ env: MattermostEnv,+ environment: Environment,+ person: { username: string; email: string; password: string }+ ): Promise<{ client: MattermostClient; user: MattermostUser; token: string }> {+ const base = env.url.endsWith('/') ? env.url.slice(0, -1) : env.url;+ const send = async (path: string, body: unknown): Promise<Response> => {+ assertAllowed(environment, 'POST', `/api/v4${path}`);+ const response = await fetch(`${base}/api/v4${path}`, {+ method: 'POST',+ headers: { 'Content-Type': 'application/json', Accept: 'application/json' },+ body: JSON.stringify(body),+ signal: AbortSignal.timeout(30_000),+ });+ if (!response.ok) {+ throw new MattermostHttpError('POST', path, response.status, await response.text());+ }+ return response;+ };+ const user = (await (await send('/users', person)).json()) as MattermostUser;+ const login = await send('/users/login', {+ login_id: person.username,+ password: person.password,+ });+ const token = login.headers.get('Token');+ if (token === null) throw new Error(`Mattermost logged ${person.username} in without a token.`);+ return { client: new MattermostClient({ ...env, token }, environment), user, token };+ }++ /** `POST /api/v4/teams/{id}/members` — join a team that is open to anyone, as the token's user. */+ async joinTeam(teamId: string, userId: string): Promise<void> {+ await this.request('POST', `/teams/${teamId}/members`, { team_id: teamId, user_id: userId });+ }+/** `GET /api/v4/users/me` — also the reachability + token-validity probe. */async me(): Promise<MattermostUser> {return this.request<MattermostUser>('GET', '/users/me');diff --git a/console/tooling-e2e/src/clients/switch-client.ts b/console/tooling-e2e/src/clients/switch-client.tsindex 9e64ba2f8..7e30d3250 100644--- a/console/tooling-e2e/src/clients/switch-client.ts+++ b/console/tooling-e2e/src/clients/switch-client.ts@@ -134,6 +134,59 @@ export class SwitchHttpError extends Error {}}+/** An agent's addressing policy: anyone no rule matches is refused. */+export interface AddressingPolicy {+ rules: {+ rooms: "*" | string[];+ room_groups: "*" | string[];+ users: "*" | string[];+ agents: "*" | string[];+ owner: boolean;+ owner_agents: boolean;+ platform: boolean;+ }[];+}++/** Only the agent's owner, and the platform identities its owner claimed. */+export const OWNER_ONLY: AddressingPolicy = {+ rules: [+ {+ rooms: "*",+ room_groups: "*",+ users: [],+ agents: [],+ owner: true,+ owner_agents: false,+ platform: false,+ },+ ],+};++/** What a template run made. */+export interface TemplateRun {+ created_agents: {+ slot: string;+ agent_id: string;+ name: string;+ machine: { id: string; name: string };+ listening: boolean;+ }[];+ room: { room_id: string; room_name: string; failed_attachments: unknown[] };+}++/** A template run the server refused, having created nothing. */+export class TemplateRefusal extends Error {+ readonly problems: { slot: string; code: string; message: string }[];+ constructor(+ cause: SwitchHttpError,+ problems: { slot: string; code: string; message: string }[],+ ) {+ super(cause.message, { cause });+ this.name = "TemplateRefusal";+ this.problems = problems;+ }+}+/*** The reason code of a management API refusal (`{"error": {"code": …}}`), or* null when the error is not one.@@ -148,6 +201,16 @@ export function managementErrorCode(error: unknown): string | null {}}+/** A FastAPI error body's `detail` when it is a string, else the body as it came. */+export function errorDetail(body: string): string {+ try {+ const parsed = JSON.parse(body) as { detail?: unknown };+ return typeof parsed.detail === "string" ? parsed.detail : body;+ } catch {+ return body;+ }+}+function truncate(text: string, max: number): string {return text.length > max ? `${text.slice(0, max)}…` : text;}@@ -530,6 +593,54 @@ export class SwitchClient {await this.gateway("DELETE", `/rooms/${roomId}`);}+ /** `DELETE /gateway/room-groups/{id}` — a group's rooms and subgroups move up a level, so delete those first. */+ async deleteRoomGroup(groupId: string): Promise<void> {+ await this.gateway("DELETE", `/room-groups/${groupId}`);+ }++ /** `POST /gateway/packages` — a private package owned by the signed-in admin, with nothing in it. */+ async createPackage(params: {+ name: string;+ description: string;+ instructions: string;+ }): Promise<{ id: string; name: string }> {+ return this.gateway<{ id: string; name: string }>("POST", "/packages", params);+ }++ async deletePackage(packageId: string): Promise<void> {+ await this.gateway("DELETE", `/packages/${packageId}`);+ }++ /** `GET /gateway/rooms/{id}/packages` — the packages attached to a room. */+ async roomPackages(roomId: string): Promise<{ id: string; name: string }[]> {+ return this.gateway<{ id: string; name: string }[]>(+ "GET",+ `/rooms/${roomId}/packages`,+ );+ }++ /**+ * `GET /gateway/rooms/{id}/yaml` — the room captured as the document+ * `/rooms/from-yaml` takes, so the room can be built again from it.+ */+ async exportRoomYaml(roomId: string): Promise<string> {+ assertAllowed(this.environment, "GET", `/gateway/rooms/${roomId}/yaml`);+ const response = await fetch(`${this.apiUrl}/gateway/rooms/${roomId}/yaml`, {+ headers: {+ Accept: "application/x-yaml",+ Cookie: await this.gatewayLogin(),+ ...this.requestId(),+ },+ redirect: "manual",+ signal: AbortSignal.timeout(30_000),+ });+ const text = await response.text();+ if (!response.ok) {+ throw new SwitchHttpError("GET", `/gateway/rooms/${roomId}/yaml`, response.status, text);+ }+ return text;+ }+/*** `DELETE /gateway/agents/by-name/{name}` — admin-authenticated teardown that,* unlike `DELETE /agents/{id}`, does not need the agent's own API key. Used by@@ -578,11 +689,49 @@ export class SwitchClient {* agents and an unclaimed platform user, so its agents are opened explicitly.*/async openAddressing(agentId: string): Promise<void> {+ await this.setAddressingPolicy(agentId, null);+ }++ /** `PUT /gateway/agents/{id}/addressing-policy` — who may address the agent; null lets anyone. */+ async setAddressingPolicy(+ agentId: string,+ policy: AddressingPolicy | null,+ ): Promise<void> {await this.gateway("PUT", `/agents/${agentId}/addressing-policy`, {- policy: null,+ policy,});}+ /**+ * `POST /gateway/rooms/from-template-document` — run a template as the+ * signed-in admin: create its agents, wait for each to listen, then build+ * its room. A refusal creates nothing and throws a {@link TemplateRefusal}+ * naming every problem.+ */+ async runTemplateDocument(params: {+ template: string;+ templateName: string;+ }): Promise<TemplateRun> {+ try {+ return await this.gateway<TemplateRun>(+ "POST",+ "/rooms/from-template-document",+ { template: params.template, template_name: params.templateName },+ );+ } catch (error) {+ if (error instanceof SwitchHttpError && error.status === 409) {+ const body = JSON.parse(error.body) as {+ code?: string;+ problems?: { slot: string; code: string; message: string }[];+ };+ if (body.code === "agents_not_created") {+ throw new TemplateRefusal(error, body.problems ?? []);+ }+ }+ throw error;+ }+ }+/*** The status Switch reports for one agent in one room. `live` means a session* is attending the room; `dormant` means an auto-session connector is watching@@ -990,6 +1139,86 @@ export class SwitchClient {return body.event_id;}+ /**+ * `POST /agents/{id}/rooms/{room_id}/media` — post one file to a room as the+ * agent (multipart, part `file`). Returns the media URI it was stored under.+ */+ async uploadMedia(+ agent: RegisteredAgent,+ roomId: string,+ file: { name: string; bytes: Uint8Array<ArrayBuffer>; contentType: string },+ ): Promise<{ eventId: string; mxc: string }> {+ const path = `/agents/${agent.id}/rooms/${roomId}/media`;+ assertAllowed(this.environment, "POST", path);+ const form = new FormData();+ form.append(+ "file",+ new Blob([file.bytes], { type: file.contentType }),+ file.name,+ );+ const response = await fetch(`${this.apiUrl}${path}`, {+ method: "POST",+ headers: {+ Accept: "application/json",+ Authorization: `Bearer ${agent.apiKey}`,+ ...this.requestId(),+ },+ body: form,+ redirect: "manual",+ signal: AbortSignal.timeout(30_000),+ });+ const text = await response.text();+ if (!response.ok) throw new SwitchHttpError("POST", path, response.status, text);+ const body = JSON.parse(text) as { event_id: string; mxc: string };+ return { eventId: body.event_id, mxc: body.mxc };+ }++ /**+ * `GET /agents/{id}/rooms/{room_id}/media?mxc=` — an attachment's bytes, and+ * the headers that name it, as the agent's session host downloads it.+ */+ async downloadMedia(+ agent: RegisteredAgent,+ roomId: string,+ mxc: string,+ ): Promise<{ bytes: Uint8Array; contentType: string; contentDisposition: string }> {+ const path = `/agents/${agent.id}/rooms/${roomId}/media?mxc=${encodeURIComponent(mxc)}`;+ assertAllowed(this.environment, "GET", path);+ const response = await fetch(`${this.apiUrl}${path}`, {+ headers: { Authorization: `Bearer ${agent.apiKey}`, ...this.requestId() },+ redirect: "manual",+ signal: AbortSignal.timeout(30_000),+ });+ if (!response.ok) {+ throw new SwitchHttpError("GET", path, response.status, await response.text());+ }+ return {+ bytes: new Uint8Array(await response.arrayBuffer()),+ contentType: response.headers.get("content-type") ?? "",+ contentDisposition: response.headers.get("content-disposition") ?? "",+ };+ }++ /**+ * Any agent-bridge request as the agent, answered with its status and body+ * whatever the status: for the checks that a client's mistake is answered+ * with a 4xx that says what was wrong.+ */+ async agentAttempt(+ agent: RegisteredAgent,+ method: "GET" | "POST",+ path: string,+ body: unknown,+ ): Promise<{ status: number; detail: string }> {+ try {+ await this.request(method, path, { token: agent.apiKey, body: body ?? undefined });+ return { status: 200, detail: "" };+ } catch (error) {+ if (!(error instanceof SwitchHttpError)) throw error;+ return { status: error.status, detail: errorDetail(error.body) };+ }+ }+// ── agent bridge: the operations front door ───────────────────────────────/**diff --git a/console/tooling-e2e/src/core/scenario.ts b/console/tooling-e2e/src/core/scenario.tsindex e33ee7fa0..08ad53670 100644--- a/console/tooling-e2e/src/core/scenario.ts+++ b/console/tooling-e2e/src/core/scenario.ts@@ -39,8 +39,14 @@ export interface ScenarioHeader {* `agent-dir`: the provider session's working directory is on this machine.* `gateway-ui`: the address of a gateway UI served against this Switch server.* `agentic-models`: the driver and judge models, and the spending limit.+ * `controller`: an agent controller that runs the agents Switch manages.*/-export type Prerequisite = 'mattermost' | 'agent-dir' | 'gateway-ui' | 'agentic-models';+export type Prerequisite =+ | 'mattermost'+ | 'agent-dir'+ | 'gateway-ui'+ | 'agentic-models'+ | 'controller';const OPERATION = /^[a-z][a-z0-9_]*$/;const ROUTE = /^(GET|POST|PUT|PATCH|DELETE) \/\S*$/;@@ -136,6 +142,8 @@ function satisfied(prerequisite: Prerequisite, env: HarnessEnv): boolean {env.agentic.judgeModel !== null &&env.agentic.budgetUsd !== null);+ case 'controller':+ return env.provider.controllerId !== null;}}@@ -149,5 +157,7 @@ function describePrerequisite(prerequisite: Prerequisite): string {return 'a gateway UI (SWITCH_GATEWAY_UI_URL)';case 'agentic-models':return 'the driver and judge models and a spending limit (SWITCH_E2E_DRIVER_MODEL, SWITCH_E2E_JUDGE_MODEL, SWITCH_E2E_AGENTIC_BUDGET_USD)';+ case 'controller':+ return 'an agent controller (SWITCH_E2E_CONTROLLER_ID)';}}diff --git a/console/tooling-e2e/src/fast/attachments.ts b/console/tooling-e2e/src/fast/attachments.tsnew file mode 100644index 000000000..0c5b2a8fa--- /dev/null+++ b/console/tooling-e2e/src/fast/attachments.ts@@ -0,0 +1,67 @@+import { expect } from 'vitest';+import { sleep } from '../clients/switch-client.ts';+import { WRITABLE, type FastScenario } from './scenario.ts';++// An em dash and the narrow no-break space a macOS screenshot name carries:+// neither is latin-1, the only charset an HTTP header carries as is.+const FILE_NAME = 'Device profile — SSH host notes.txt';+const HISTORY_DEADLINE_MS = 30_000;++/**+ * An attachment whose name is not latin-1 is posted, listed and downloaded+ * under its own name. Its download put the name straight into+ * `Content-Disposition`, which failed with a 500, so the agent never got the+ * file.+ */+export const attachmentNames: FastScenario = {+ header: {+ id: 'attachment-names',+ covers: [+ 'POST /agents/{agent_id}/rooms/{room_id}/media',+ 'GET /agents/{agent_id}/rooms/{room_id}/media',+ 'read_context',+ ],+ lane: 'fast',+ environments: WRITABLE,+ owner: 'platform',+ requires: [],+ },+ async run({ fixtures, names, switchClient }) {+ const sender = await fixtures.agent('up');+ const reader = await fixtures.agent('down');+ const room = await fixtures.internalRoom('room', [sender, reader]);++ const bytes = new TextEncoder().encode(`${names.marker('file')}\nnaïve café ✓\n`);+ const posted = await switchClient.uploadMedia(sender, room.id, {+ name: FILE_NAME,+ bytes,+ contentType: 'text/plain',+ });++ interface Entry {+ attachments: { filename: string; mxc: string }[];+ }+ let listed: { filename: string; mxc: string } | undefined;+ const until = Date.now() + HISTORY_DEADLINE_MS;+ while (listed === undefined && Date.now() < until) {+ const context = await switchClient.callOperation<{+ threads: { root: Entry; replies: Entry[] }[];+ }>(reader, 'read_context', { room_id: room.id, limit: 20 });+ listed = context.threads+ .flatMap((thread) => [thread.root, ...thread.replies])+ .flatMap((entry) => entry.attachments)+ .find((attachment) => attachment.mxc === posted.mxc);+ if (listed === undefined) await sleep(1_000);+ }+ expect(listed, `the room's history never listed ${posted.mxc}`).toBeDefined();+ expect(listed!.filename).toBe(FILE_NAME);++ const download = await switchClient.downloadMedia(reader, room.id, posted.mxc);+ expect(Buffer.from(download.bytes).equals(Buffer.from(bytes))).toBe(true);+ expect(download.contentType).toMatch(/^text\/plain/);+ // RFC 6266: the name itself, percent-encoded as UTF-8, in `filename*`.+ expect(download.contentDisposition).toContain(+ `filename*=UTF-8''${encodeURIComponent(FILE_NAME)}`+ );+ },+};diff --git a/console/tooling-e2e/src/fast/client-mistakes.ts b/console/tooling-e2e/src/fast/client-mistakes.tsnew file mode 100644index 000000000..028ef1494--- /dev/null+++ b/console/tooling-e2e/src/fast/client-mistakes.ts@@ -0,0 +1,63 @@+import { expect } from 'vitest';+import { WRITABLE, type FastScenario } from './scenario.ts';++/**+ * An agent's mistake is answered with a 4xx that says what was wrong, never a+ * 500: an HTTP agent reads a 500 as the server broken and may retry a request+ * that can never succeed. The server-error check fails the scenario too, if+ * any of these is logged as an error.+ */+export const clientMistakes: FastScenario = {+ header: {+ id: 'client-mistakes',+ covers: [+ 'POST /agents/{agent_id}/moderation/rooms/{room_id}/invite',+ 'GET /agents/{agent_id}/moderation/rooms/{room_id}',+ 'GET /agents/{agent_id}/rooms/{room_id}/history',+ ],+ lane: 'fast',+ environments: WRITABLE,+ owner: 'platform',+ requires: [],+ },+ async run({ fixtures, switchClient }) {+ const member = await fixtures.agent('in');+ const outsider = await fixtures.agent('out');+ const room = await fixtures.internalRoom('room', [member]);++ const invite = await switchClient.agentAttempt(+ outsider,+ 'POST',+ `/agents/${outsider.id}/moderation/rooms/${room.id}/invite`,+ { agent_name: outsider.name }+ );+ expect(invite, 'an outsider inviting itself into a room').toEqual({+ status: 403,+ detail: "Not authorized to change this room's membership",+ });++ const lookup = await switchClient.agentAttempt(+ outsider,+ 'GET',+ `/agents/${outsider.id}/moderation/rooms/${room.id}`,+ null+ );+ expect(lookup, 'an outsider reading a room').toEqual({+ status: 403,+ detail: 'Agent is not a member of this room',+ });++ for (const parameter of ['since', 'before']) {+ const history = await switchClient.agentAttempt(+ member,+ 'GET',+ `/agents/${member.id}/rooms/${room.id}/history?${parameter}=yesterday`,+ null+ );+ expect(history, `history with ${parameter}=yesterday`).toEqual({+ status: 400,+ detail: `'${parameter}' is not an ISO 8601 timestamp: 'yesterday'`,+ });+ }+ },+};diff --git a/console/tooling-e2e/src/fast/follow-ups.ts b/console/tooling-e2e/src/fast/follow-ups.tsnew file mode 100644index 000000000..1c06f6c0a--- /dev/null+++ b/console/tooling-e2e/src/fast/follow-ups.ts@@ -0,0 +1,178 @@+import { expect } from 'vitest';+import type { MattermostClient, MattermostPost } from '../clients/mattermost-client.ts';+import {+ eventText,+ sleep,+ type AgentEvent,+ type RegisteredAgent,+ type SwitchClient,+} from '../clients/switch-client.ts';+import type { BridgedRoom } from '../fixtures/fixtures.ts';+import type { FastScenario } from './scenario.ts';++const FLAG = 'follow_the_conversation';+const DEADLINE_MS = 60_000;++/** The window the server was started with: set by the ephemeral stack, which starts it. */+function followWindowMs(): number {+ const raw = process.env.SWITCH_E2E_FOLLOW_WINDOW_SECONDS;+ const seconds = Number(raw);+ if (raw === undefined || !Number.isFinite(seconds) || seconds <= 0) {+ throw new Error(+ `SWITCH_E2E_FOLLOW_WINDOW_SECONDS must name the server's FOLLOW_THE_CONVERSATION_WINDOW_SECONDS (got '${raw}').`+ );+ }+ return seconds * 1000;+}++/**+ * Follow the conversation: a person's untagged message reaches the agent they+ * are talking to. It does when it continues an exchange the agent has just+ * answered, or replies in a thread the agent spoke in last. It does not when+ * another agent spoke in between, or when the exchange has gone quiet for longer+ * than the window. A message from an agent never follows anyone.+ *+ * Each "does not" is shown by an `@agent` message posted after it: the agent's+ * notifications arrive in order, so if the untagged one had reached it, it+ * would have arrived first.+ *+ * Ephemeral only: the flag is server-wide, and the window has to be short+ * enough to wait out.+ */+export const followUps: FastScenario = {+ header: {+ id: 'follow-ups',+ covers: ['GET /agents/{agent_id}/notifications', 'POST /agents/{agent_id}/message'],+ lane: 'fast',+ environments: ['ephemeral'],+ owner: 'platform',+ requires: ['mattermost'],+ },+ async run({ fixtures, names, switchClient, mattermost, log }) {+ const flags = await switchClient.enabledFeatureFlags();+ if (!flags.has(FLAG)) {+ throw new Error(+ `The server does not have '${FLAG}' on; the ephemeral stack starts it with it.`+ );+ }+ const windowMs = followWindowMs();+ const a = await fixtures.agent('fa');+ const b = await fixtures.agent('fb');+ const bridged = await fixtures.mattermostRoom('ftc', [a, b]);+ const talk = new Exchange(switchClient, mattermost!, bridged);++ // A channel follow-up: the person goes on, untagged, after the agent answered.+ const q1 = names.marker('q1');+ await talk.human(`@${a.name} first question ${q1}`, null);+ await talk.notified(a, q1, 'a mention', []);+ const answer = await talk.answer(a, `first answer ${names.marker('a1')}`);+ const q2 = names.marker('q2');+ await talk.human(`and a second one ${q2}`, null);+ const followed = await talk.notified(a, q2, 'an untagged channel follow-up', []);+ expect(followed.payload?.addressed).toBe(true);+ expect(followed.payload?.thread_id ?? null).toBeNull();++ // A thread follow-up: a reply under the agent's answer, in a thread it spoke in last.+ const q3 = names.marker('q3');+ await talk.human(`why so ${q3}`, answer.post.id);+ const threaded = await talk.notified(a, q3, 'an untagged reply in its thread', []);+ expect(threaded.payload?.thread_id).toBe(answer.eventId);++ // Another agent speaks: its message follows no one, and ends the person's exchange.+ const fyi = names.marker('fyi');+ await talk.answer(b, `for your information ${fyi}`);+ const q4 = names.marker('q4');+ await talk.human(`and a third one ${q4}`, null);+ const q5 = names.marker('q5');+ await talk.human(`@${a.name} back to you ${q5}`, null);+ await talk.notified(a, q5, 'a mention', [fyi, q4]);+ const history = await switchClient.roomHistory(a, bridged.room.id, 20);+ expect(history.map((event) => event.body).some((body) => body.includes(fyi))).toBe(true);++ // The exchange goes quiet for longer than the window.+ await talk.answer(a, `answered ${names.marker('a5')}`);+ log(`waiting out the ${windowMs / 1000}s follow-up window`);+ await sleep(windowMs + 5_000);+ const q6 = names.marker('q6');+ await talk.human(`one more thing ${q6}`, null);+ const q7 = names.marker('q7');+ await talk.human(`@${a.name} last one ${q7}`, null);+ await talk.notified(a, q7, 'a mention', [q6]);++ // b was never addressed, and is not followed: nothing above reached it.+ const strays = (await switchClient.pollNotifications(b, 2)).filter((event) =>+ [q1, q2, q3, q4, q5, q6, q7].some((marker) => eventText(event).includes(marker))+ );+ expect(strays.map(eventText), 'messages that reached the agent nobody talked to').toEqual([]);+ },+};++/** One person and two agents talking in a bridged room. */+class Exchange {+ private readonly switchClient: SwitchClient;+ private readonly mattermost: MattermostClient;+ private readonly bridged: BridgedRoom;++ constructor(switchClient: SwitchClient, mattermost: MattermostClient, bridged: BridgedRoom) {+ this.switchClient = switchClient;+ this.mattermost = mattermost;+ this.bridged = bridged;+ }++ /** Post as the person: top-level, or under `rootId` in a thread. */+ async human(message: string, rootId: string | null): Promise<void> {+ await this.mattermost.post({+ channelId: this.bridged.channel.id,+ message,+ ...(rootId === null ? {} : { rootId }),+ });+ }++ /** Post top-level as the agent, and wait for the post to reach the channel. */+ async answer(+ agent: RegisteredAgent,+ text: string+ ): Promise<{ eventId: string; post: MattermostPost }> {+ const since = Date.now() - 1;+ const eventId = await this.switchClient.sendMessage(agent, this.bridged.room.id, text);+ const { match, transcript } = await this.mattermost.waitForPost({+ channelId: this.bridged.channel.id,+ fromUserId: this.bridged.bots.get(agent.id)!.id,+ predicate: (post) => post.message.includes(text),+ sinceMs: since,+ deadlineMs: DEADLINE_MS,+ pollIntervalMs: 500,+ });+ if (match === null) {+ throw new Error(+ `${agent.name}'s '${text}' never reached the channel; it said: ` ++ transcript.map((post) => post.message).join(' | ')+ );+ }+ return { eventId, post: match };+ }++ /**+ * Wait for `marker` to reach the agent, and fail if any of `notBefore` did+ * first.+ */+ async notified(+ agent: RegisteredAgent,+ marker: string,+ what: string,+ notBefore: string[]+ ): Promise<AgentEvent> {+ const { match, seen } = await this.switchClient.waitForNotification(+ agent,+ (event) => eventText(event).includes(marker),+ DEADLINE_MS+ );+ expect(+ match,+ `${what} (${marker}) never reached ${agent.name}; saw ${seen.map(eventText).join(' | ')}`+ ).not.toBeNull();+ const early = seen.filter((event) => notBefore.some((text) => eventText(event).includes(text)));+ expect(early.map(eventText), `reached ${agent.name} though it should not have`).toEqual([]);+ return match!;+ }+}diff --git a/console/tooling-e2e/src/fast/index.ts b/console/tooling-e2e/src/fast/index.tsindex 676054e08..acab1dd96 100644--- a/console/tooling-e2e/src/fast/index.ts+++ b/console/tooling-e2e/src/fast/index.ts@@ -1,7 +1,12 @@import { agentIdentity } from "./agent-identity.ts";+import { attachmentNames } from "./attachments.ts";+import { clientMistakes } from "./client-mistakes.ts";+import { followUps } from "./follow-ups.ts";import { addressedDelivery, unaddressedHistory } from "./agent-messaging.ts";import { mattermostRoundtrip } from "./mattermost-roundtrip.ts";import { operationsFrontDoor } from "./operations.ts";+import { roomCapture } from "./room-capture.ts";+import { roomGroupPaths } from "./room-groups.ts";import type { FastScenario } from "./scenario.ts";/** Every fast-lane scenario. A scenario not listed here does not run. */@@ -11,4 +16,9 @@ export const FAST_SCENARIOS: readonly FastScenario[] = [operationsFrontDoor,mattermostRoundtrip,agentIdentity,+ roomGroupPaths,+ attachmentNames,+ clientMistakes,+ roomCapture,+ followUps,];diff --git a/console/tooling-e2e/src/fast/room-capture.ts b/console/tooling-e2e/src/fast/room-capture.tsnew file mode 100644index 000000000..c715908a2--- /dev/null+++ b/console/tooling-e2e/src/fast/room-capture.ts@@ -0,0 +1,67 @@+import { expect } from 'vitest';+import { parse, stringify } from 'yaml';+import { WRITABLE, type FastScenario } from './scenario.ts';++/**+ * A room built from a document keeps the packages the document names, and+ * the room captured back as a document builds a second room with the same+ * packages: what a template made from a live room relies on.+ */+export const roomCapture: FastScenario = {+ header: {+ id: 'room-capture',+ covers: [+ 'POST /gateway/packages',+ 'DELETE /gateway/packages/{package_id}',+ 'GET /gateway/rooms/{room_id}/packages',+ 'GET /gateway/rooms/{room_id}/yaml',+ 'POST /gateway/rooms/from-yaml',+ ],+ lane: 'fast',+ environments: WRITABLE,+ owner: 'platform',+ requires: [],+ },+ async run({ fixtures, names, switchClient }) {+ const agent = await fixtures.agent('cap');+ const about = `Switch end-to-end harness (run ${names.runId}). Safe to delete.`;+ const pkg = await switchClient.createPackage({+ name: names.name('pkg'),+ description: about,+ instructions: names.marker('pkg'),+ });+ fixtures.defer(`package ${pkg.name} (${pkg.id})`, () => switchClient.deletePackage(pkg.id));++ const build = async (document: string) => {+ const room = await switchClient.createRoomFromYaml(document);+ fixtures.defer(`room ${room.name} (${room.id})`, () => switchClient.deleteRoom(room.id));+ return room;+ };++ const original = await build(+ stringify({+ room: {+ name: names.name('orig'),+ description: about,+ agents: [agent.name],+ packages: [{ name: pkg.name }],+ },+ })+ );+ expect(await switchClient.roomPackages(original.id)).toMatchObject([+ { id: pkg.id, name: pkg.name },+ ]);++ const captured = parse(await switchClient.exportRoomYaml(original.id)) as {+ room: Record<string, unknown>;+ };+ expect(captured.room.packages).toEqual([{ id: pkg.id }]);+ expect(captured.room.agents).toEqual([agent.name]);++ const copy = await build(stringify({ room: { ...captured.room, name: names.name('copy') } }));+ expect(copy.id).not.toBe(original.id);+ expect(await switchClient.roomPackages(copy.id)).toMatchObject([+ { id: pkg.id, name: pkg.name },+ ]);+ },+};diff --git a/console/tooling-e2e/src/fast/room-groups.ts b/console/tooling-e2e/src/fast/room-groups.tsnew file mode 100644index 000000000..20d00217a--- /dev/null+++ b/console/tooling-e2e/src/fast/room-groups.ts@@ -0,0 +1,102 @@+import { expect } from 'vitest';+import { SwitchHttpError, errorDetail } from '../clients/switch-client.ts';+import { WRITABLE, type FastScenario } from './scenario.ts';++interface RoomGroup {+ id: string;+ name: string;+ parent_group_id: string | null;+ path: string;+ member_rooms: { id: string; name: string }[];+}++/**+ * Room groups named by path: the same subgroup name under two parents is+ * allowed, a bare name that two groups share is refused naming the path of+ * each, the path picks one, and a second group of that name under the same+ * parent is refused whatever its case.+ */+export const roomGroupPaths: FastScenario = {+ header: {+ id: 'room-group-paths',+ covers: [+ 'create_room_group',+ 'get_room_group_detail',+ 'create_room',+ 'DELETE /gateway/room-groups/{group_id}',+ ],+ lane: 'fast',+ environments: WRITABLE,+ owner: 'platform',+ requires: [],+ },+ async run({ fixtures, names, switchClient, log }) {+ const agent = await fixtures.agent('grp');+ const call = <T>(operation: string, args: Record<string, unknown>): Promise<T> =>+ switchClient.callOperation<T>(agent, operation, args);+ const group = async (name: string, parent: string | null): Promise<RoomGroup> => {+ const made = await call<RoomGroup>('create_room_group', {+ name,+ description: `Switch end-to-end harness (run ${names.runId}). Safe to delete.`,+ ...(parent === null ? {} : { parent_group_name: parent }),+ });+ fixtures.defer(`room group ${made.path} (${made.id})`, () =>+ switchClient.deleteRoomGroup(made.id)+ );+ return made;+ };+ const refusal = async (operation: string, args: Record<string, unknown>): Promise<string> => {+ try {+ await call(operation, args);+ } catch (error) {+ if (error instanceof SwitchHttpError && error.status === 400)+ return errorDetail(error.body);+ throw error;+ }+ throw new Error(`${operation} ${JSON.stringify(args)} was accepted`);+ };++ const parents = [await group(names.name('ga'), null), await group(names.name('gb'), null)];+ // One name, used under both parents: what made bare names ambiguous.+ const child = names.name('inc');+ const children = [];+ for (const parent of parents) {+ const made = await group(child, parent.name);+ expect(made.parent_group_id).toBe(parent.id);+ // Shown spaced ("a / b"); a reference may be written either way.+ expect(made.path.split('/').map((segment) => segment.trim())).toEqual([parent.name, child]);+ children.push(made);+ }+ const [first, second] = children as [RoomGroup, RoomGroup];+ const [firstPath, secondPath] = parents.map((parent) => `${parent.name}/${child}`) as [+ string,+ string,+ ];++ const duplicate = await refusal('create_room_group', {+ name: child.toUpperCase(),+ parent_group_name: parents[0]!.name,+ });+ expect(duplicate).toContain('already exists');++ const room = names.name('room');+ const roomArgs = {+ name: room,+ description: `Switch end-to-end harness (run ${names.runId}). Safe to delete.`,+ agent_names: [agent.name],+ internal_only: true,+ };+ const ambiguous = await refusal('create_room', { ...roomArgs, group_name: child });+ expect(ambiguous).toContain(`Ambiguous room group '${child}'`);+ expect(ambiguous).toContain(firstPath);+ expect(ambiguous).toContain(secondPath);+ log(`a bare '${child}' was refused: ${ambiguous}`);++ const made = await call<{ id: string }>('create_room', { ...roomArgs, group_name: secondPath });+ fixtures.defer(`room ${room} (${made.id})`, () => switchClient.deleteRoom(made.id));+ const detail = await call<RoomGroup>('get_room_group_detail', { group_id: second.id });+ expect(detail.member_rooms).toEqual([{ id: made.id, name: room }]);+ const other = await call<RoomGroup>('get_room_group_detail', { group_id: first.id });+ expect(other.member_rooms).toEqual([]);+ },+};diff --git a/console/tooling-e2e/src/provider/harness.ts b/console/tooling-e2e/src/provider/harness.tsindex 0e5e81593..72d8494d4 100644--- a/console/tooling-e2e/src/provider/harness.ts+++ b/console/tooling-e2e/src/provider/harness.ts@@ -74,17 +74,17 @@ const ROLE_BY_AGENT_TYPE: Record<HarnessAgentType, string> = {// sentence: told merely to answer briefly, the agent declined the `interrupt`// scenario's task on the grounds that 200 messages would spam the room — a// refusal the scenario reads as a session that never started.-const AGENT_INSTRUCTIONS =+export const AGENT_INSTRUCTIONS ='You are a Switch end-to-end test agent. Do exactly what you are asked in the ' +'room, and post your answer back to the room you were addressed in. Never ' +'decline or shorten a task on the grounds that it is repetitive or long: the ' +'requests you get here are deliberate tests of exactly that.';/** Cheap on purpose: a run spends a real subscription. */-const CLAUDE_MODEL = 'claude-sonnet-5';+export const CLAUDE_MODEL = 'claude-sonnet-5';/** The management API's provider id for each harness agent type. */-const PROVIDER_BY_AGENT_TYPE: Record<HarnessAgentType, string> = {+export const PROVIDER_BY_AGENT_TYPE: Record<HarnessAgentType, string> = {opencode: 'opencode','claude-code': 'claude',codex: 'codex',@@ -93,10 +93,10 @@ const PROVIDER_BY_AGENT_TYPE: Record<HarnessAgentType, string> = {};/** How long to keep asking a controller to take the agent while it comes up. */-const PLACEMENT_DEADLINE_MS = 120_000;+export const PLACEMENT_DEADLINE_MS = 120_000;/** The placement refusals that clear once the controller is up and signed in. */-const NOT_READY_CODES = new Set([+export const NOT_READY_CODES = new Set(['controller_offline','provider_not_installed','provider_login_missing',diff --git a/console/tooling-e2e/src/provider/provider.e2e.test.ts b/console/tooling-e2e/src/provider/provider.e2e.test.tsindex 8e22bb8f7..dc2489a57 100644--- a/console/tooling-e2e/src/provider/provider.e2e.test.ts+++ b/console/tooling-e2e/src/provider/provider.e2e.test.ts@@ -97,10 +97,11 @@ describe('provider lane', () => {}const h = harness!;const log = (message: string): void => console.log(`[${scenario.header.id}] ${message}`);- const observer = watch.begin(h.switch, scopes.scope().tag, scenario.header.id);+ const names = scopes.scope();+ const observer = watch.begin(h.switch, names.tag, scenario.header.id);let result: ScenarioResult;try {- result = await scenario.run(h);+ result = await scenario.run(h, names);} finally {observer.stop();}diff --git a/console/tooling-e2e/src/provider/scenarios.ts b/console/tooling-e2e/src/provider/scenarios.tsindex fa73f4e96..4dfb0a2a0 100644--- a/console/tooling-e2e/src/provider/scenarios.ts+++ b/console/tooling-e2e/src/provider/scenarios.ts@@ -1,8 +1,20 @@+import { randomBytes } from 'node:crypto';import { readFile } from 'node:fs/promises';import path from 'node:path';-import { sleep, type MattermostPost } from '../clients/mattermost-client.ts';+import { stringify } from 'yaml';+import { MattermostClient, sleep, type MattermostPost } from '../clients/mattermost-client.ts';+import { OWNER_ONLY, TemplateRefusal, type TemplateRun } from '../clients/switch-client.ts';+import type { NameScope } from '../core/run.ts';import type { ScenarioHeader } from '../core/scenario.ts';-import type { Harness } from './harness.ts';+import { IssuedSecrets } from '../core/secrets.ts';+import {+ AGENT_INSTRUCTIONS,+ CLAUDE_MODEL,+ NOT_READY_CODES,+ PLACEMENT_DEADLINE_MS,+ PROVIDER_BY_AGENT_TYPE,+ type Harness,+} from './harness.ts';/*** The scenarios. Each one drives the full path@@ -26,7 +38,8 @@ export interface ScenarioResult {durationMs: number;}-export type Scenario = (harness: Harness) => Promise<ScenarioResult>;+/** `names` names whatever a scenario makes beyond the shared agent and channel. */+export type Scenario = (harness: Harness, names: NameScope) => Promise<ScenarioResult>;/** How long a scenario waits for the agent to say something. */const REPLY_DEADLINE_MS = Number(process.env.SWITCH_E2E_REPLY_TIMEOUT_MS ?? 5 * 60_000);@@ -204,8 +217,10 @@ async function ask(harness: Harness, text: string): Promise<number> {/** Wait for a bot post matching `predicate`, or throw with the transcript. */async function expectBotPost(- harness: Harness,+ mattermost: MattermostClient,params: {+ /** The channel to watch, and the bot whose posts count. */+ where: { channelId: string; botId: string };since: number;predicate: (post: MattermostPost) => boolean;describe: string;@@ -213,9 +228,9 @@ async function expectBotPost(record: (posts: MattermostPost[]) => void;}): Promise<MattermostPost> {- const { match, transcript } = await harness.mattermost.waitForPost({- channelId: harness.channel.id,- fromUserId: harness.bot.id,+ const { match, transcript } = await mattermost.waitForPost({+ channelId: params.where.channelId,+ fromUserId: params.where.botId,// Neither a no-session notice nor Switch's activity commentary is ever a// match, however permissive the caller's predicate — both are Switch// talking on the bot's account, not the agent.@@ -234,6 +249,11 @@ async function expectBotPost(return match;}+/** The shared channel, and the shared agent's bot. */+function sharedRoom(harness: Harness): { channelId: string; botId: string } {+ return { channelId: harness.channel.id, botId: harness.bot.id };+}+function summarise(posts: MattermostPost[]): string {if (posts.length === 0) return '(nothing at all — no session ever replied)';return posts.map((post) => `«${post.message.replace(/\s+/g, ' ').slice(0, 160)}»`).join(' | ');@@ -255,7 +275,8 @@ function lower(post: MattermostPost): string {export const greet: Scenario = (harness) =>scenario('greet', async (record) => {const since = await ask(harness, `reply with exactly ${OK_MARKER} and nothing else`);- const match = await expectBotPost(harness, {+ const match = await expectBotPost(harness.mattermost, {+ where: sharedRoom(harness),since,record,predicate: (post) => post.message.includes(OK_MARKER),@@ -291,7 +312,8 @@ export const question: Scenario = (harness) =>`exactly the options ${CHOICES.join(', ')}. Then reply with the single word I chose.`);- const asked = await expectBotPost(harness, {+ const asked = await expectBotPost(harness.mattermost, {+ where: sharedRoom(harness),since,record,// A native question is relayed as a request card numbering the options;@@ -312,7 +334,8 @@ export const question: Scenario = (harness) =>const answeredAt = prose? await ask(harness, CHOSEN): await answerCard(harness, asked, chosenOption(asked));- const answer = await expectBotPost(harness, {+ const answer = await expectBotPost(harness.mattermost, {+ where: sharedRoom(harness),since: answeredAt,record,predicate: (post) =>@@ -348,7 +371,8 @@ export const approval: Scenario = (harness) =>`run the shell command \`echo ${OK_MARKER} > ${file}\` and then reply with the single word done`);- const prompt = await expectBotPost(harness, {+ const prompt = await expectBotPost(harness.mattermost, {+ where: sharedRoom(harness),since,record,predicate: (post) =>@@ -358,7 +382,8 @@ export const approval: Scenario = (harness) =>});const approvedAt = await answerCard(harness, prompt, 1);- const done = await expectBotPost(harness, {+ const done = await expectBotPost(harness.mattermost, {+ where: sharedRoom(harness),since: approvedAt,record,predicate: (post) => lower(post).includes('done'),@@ -398,7 +423,8 @@ export const interrupt: Scenario = (harness) =>harness,`Run this exact shell command for a bounded cancellation test: \`${command}\`. Wait for completion before replying. Do not create these files with another tool.`);- const permission = await expectBotPost(harness, {+ const permission = await expectBotPost(harness.mattermost, {+ where: sharedRoom(harness),since,record,predicate: (post) =>@@ -432,6 +458,228 @@ export const interrupt: Scenario = (harness) =>return 'command start confirmed; completion marker absent after 95s';});+// ── card answered by someone who may not ─────────────────────────────────────++/** The bot Switch posts its own notices as, on Mattermost. */+const SWITCH_ADMIN = 'switch-admin';++/**+ * Answering a request card takes the same permission as addressing the agent.+ * With the agent open only to its owner, a person who is not the owner answers+ * the card and is told the answer did not land, and the command does not run;+ * the owner answers it and the command runs.+ *+ * The second person signs up on the spot, so this runs only where Mattermost+ * lets anyone sign up: the throwaway stack and a local one. On both, the person+ * the harness posts as is the identity the admin who owns the agent claimed.+ */+export const cardAnswerRefused: Scenario = (harness, names) =>+ scenario('card-answer-refused', async (record) => {+ const mattermostEnv = harness.env.mattermost;+ if (mattermostEnv === null) throw new Error('This scenario needs Mattermost.');+ const secrets = new IssuedSecrets(harness.env.secretsFile);+ const username = names.name('other');+ const password = `E2e-${randomBytes(18).toString('base64url')}`;+ secrets.record(password);+ const other = await MattermostClient.signUp(mattermostEnv, harness.env.environment, {+ username,+ email: `${username}@example.com`,+ password,+ });+ secrets.record(other.token);+ const team = await harness.mattermost.findTeam(mattermostEnv.team);+ await other.client.joinTeam(team.id, other.user.id);+ await harness.mattermost.addUserToChannel(harness.channel.id, other.user.id);+ const admin = await harness.mattermost.findBotUser(SWITCH_ADMIN);+ if (admin === null) throw new Error(`Mattermost has no '${SWITCH_ADMIN}' account.`);++ await harness.switch.setAddressingPolicy(harness.agent.id, OWNER_ONLY);+ try {+ const file = `refused-${harness.runId}.txt`;+ const since = await ask(+ harness,+ `run the shell command \`echo ${OK_MARKER} > ${file}\` and then reply with the single word done`+ );+ const card = await expectBotPost(harness.mattermost, {+ where: sharedRoom(harness),+ since,+ record,+ predicate: (post) =>+ requestHandle(post.message) !== null &&+ APPROVAL_HINTS.some((hint) => lower(post).includes(hint)),+ describe: 'an approval card for the shell command',+ });+ const handle = requestHandle(card.message)!;+ const thread = card.root_id === '' ? card.id : card.root_id;++ const refusedAt = Date.now() - 1;+ await other.client.post({ channelId: harness.channel.id, message: `${handle} 1` });+ const { match: notice, transcript } = await harness.mattermost.waitForPost({+ channelId: harness.channel.id,+ fromUserId: admin.id,+ predicate: (post) =>+ post.root_id === thread &&+ post.message.includes(`answer to ${handle} did not land`) &&+ post.message.includes('may not address this agent'),+ sinceMs: refusedAt,+ deadlineMs: 60_000,+ });+ record(transcript);+ if (notice === null) {+ throw new Error(+ `${username}'s answer to ${handle} was not refused in the card's thread. Channel said: ${summarise(transcript)}`+ );+ }+ const ranEarly = (+ await harness.mattermost.postsFrom(harness.channel.id, harness.bot.id, refusedAt)+ ).filter(+ (post) =>+ !isRuntimeStatusPost(post) && !isNoSessionNotice(post) && lower(post).includes('done')+ );+ record(ranEarly);+ if (ranEarly.length > 0) {+ throw new Error(`The command ran on the answer of someone who may not address the agent.`);+ }++ const approvedAt = await answerCard(harness, card, 1);+ const done = await expectBotPost(harness.mattermost, {+ where: sharedRoom(harness),+ since: approvedAt,+ record,+ predicate: (post) => lower(post).includes('done'),+ describe: "a 'done' reply after the owner approved the command",+ });+ return `refused in ${notice.id}, approved by the owner, completed in ${done.id}`;+ } finally {+ await harness.switch.openAddressing(harness.agent.id);+ }+ });++// ── template ─────────────────────────────────────────────────────────────────++/**+ * Run a template, retrying while the controller is not ready to take its agent:+ * a refused run creates nothing, so running it again is safe.+ */+async function runTemplate(harness: Harness, template: string): Promise<TemplateRun> {+ const until = Date.now() + PLACEMENT_DEADLINE_MS;+ for (;;) {+ try {+ return await harness.switch.runTemplateDocument({ template, templateName: 'e2e-template' });+ } catch (error) {+ const waiting =+ error instanceof TemplateRefusal &&+ error.problems.length > 0 &&+ error.problems.every((problem) => NOT_READY_CODES.has(problem.code));+ if (!waiting || Date.now() > until) throw error;+ await sleep(3_000);+ }+ }+}++/**+ * A template creates a managed agent on the controller and a room on the+ * Mattermost bridge with it in it, and the agent answers there: what a person+ * using a template from Switch Console gets.+ */+export const templateAgent: Scenario = (harness, names) =>+ scenario('template-agent', async (record) => {+ const controllerId = harness.env.provider.controllerId;+ if (controllerId === null) throw new Error('This scenario needs an agent controller.');+ const provider = PROVIDER_BY_AGENT_TYPE[harness.env.provider.agentType];+ const about = `Switch end-to-end harness (run ${harness.runId}). Safe to delete.`;+ const agentName = names.name('tpl');+ const bridge = await harness.switch.defaultBridge('mattermost');+ const template = stringify({+ version: 1,+ agent: {+ name: agentName,+ description: about,+ instructions: AGENT_INSTRUCTIONS,+ provider,+ machine: controllerId,+ // The machine's own workspace for the agent, when the run names none.+ directory: harness.env.provider.agentRepoDir ?? '{$agents_dir}/{agent}',+ ...(provider === 'claude' ? { model: CLAUDE_MODEL } : {}),+ auto_mode: false,+ auto_approve: false,+ addressing: 'anyone',+ },+ room: {+ name: names.name('troom'),+ description: about,+ bridge: bridge.displayName,+ agents: ['{agent}'],+ users: [harness.human.username],+ },+ });++ const run = await runTemplate(harness, template);+ const undo: { label: string; run: () => Promise<void> }[] = [];+ for (const made of run.created_agents) {+ undo.push({+ label: `agent ${made.name} (${made.agent_id})`,+ run: () => harness.switch.deleteAgentByName(made.name),+ });+ }+ const room = await harness.switch.getRoom(run.room.room_id);+ const channelId = room.externalChannelId;+ undo.push({+ label: `room ${room.name} (${room.id})`,+ run: async () => {+ await harness.switch.deleteRoom(room.id);+ if (channelId === null) return;+ const left = await harness.mattermost.getChannel(channelId);+ if (!left.delete_at) await harness.mattermost.archiveChannel(channelId);+ },+ });++ let failed = false;+ try {+ if (run.created_agents.length !== 1) {+ throw new Error(`The template made ${run.created_agents.length} agents, not one.`);+ }+ const made = run.created_agents[0]!;+ if (made.name !== agentName || made.machine.id !== controllerId || !made.listening) {+ throw new Error(`The template's agent is not the one asked for: ${JSON.stringify(made)}`);+ }+ if (channelId === null)+ throw new Error(`The template's room ${room.id} has no Mattermost channel.`);+ const bot = await harness.mattermost.waitForBotUser(agentName, 60_000);+ const marker = `${OK_MARKER}_TPL`;+ const since = Date.now() - 1;+ await harness.mattermost.post({+ channelId,+ message: `@${agentName} reply with exactly ${marker} and nothing else`,+ });+ const reply = await expectBotPost(harness.mattermost, {+ where: { channelId, botId: bot.id },+ since,+ record,+ predicate: (post) => post.message.includes(marker),+ describe: `a reply containing ${marker} from the template's agent`,+ });+ return `${agentName} on ${made.machine.name} replied in post ${reply.id}`;+ } catch (error) {+ failed = true;+ throw error;+ } finally {+ const left: string[] = [];+ for (const step of undo.reverse()) {+ try {+ await step.run();+ } catch (error) {+ left.push(`${step.label}: ${error instanceof Error ? error.message : String(error)}`);+ }+ }+ if (left.length > 0) {+ const message = `could not remove what the template made — delete it by hand: ${left.join('; ')}`;+ if (failed) console.warn(message);+ else throw new Error(message);+ }+ }+ });+/*** A provider-lane scenario: its header, and the function that drives it. All of* them share one agent and one channel, set up once per run.@@ -472,6 +720,26 @@ export const SCENARIOS: readonly ProviderScenario[] = [header: providerHeader('interrupt', ['connect_to_room', 'post_message'], ['agent-dir']),run: interrupt,},+ {+ // A second person signs up, which only a throwaway or local Mattermost lets anyone do.+ header: {+ ...providerHeader(+ 'card-answer-refused',+ ['PUT /gateway/agents/{agent_id}/addressing-policy'],+ []+ ),+ environments: ['ephemeral', 'local'],+ },+ run: cardAnswerRefused,+ },+ {+ header: providerHeader(+ 'template-agent',+ ['POST /gateway/rooms/from-template-document'],+ ['controller']+ ),+ run: templateAgent,+ },];/** Fixed-width summary of a run, printed at the end of the suite. */diff --git a/console/tooling-e2e/surface/exemptions.json b/console/tooling-e2e/surface/exemptions.jsonindex b49e321e7..55cac117e 100644--- a/console/tooling-e2e/surface/exemptions.json+++ b/console/tooling-e2e/surface/exemptions.json@@ -42,10 +42,8 @@"attach_reference_to_room","create_agent","create_reference",- "create_room","create_room_document","create_room_from_yaml",- "create_room_group","define_role","delete_role","delete_room_document",@@ -56,7 +54,6 @@"get_my_context","get_role_detail","get_room_detail",- "get_room_group_detail","get_template","get_template_guide","invite_agent_to_room",@@ -105,14 +102,12 @@"DELETE /gateway/management/controllers/{controller_id}","DELETE /gateway/management/controllers/{controller_id}/provider-logins/{provider}","DELETE /gateway/messaging-apps/installs/{install_id}",- "DELETE /gateway/packages/{package_id}","DELETE /gateway/packages/{package_id}/documents/{document_id}","DELETE /gateway/packages/{package_id}/references/{reference_id}","DELETE /gateway/provider-connections/github","DELETE /gateway/provider-connections/github/flows/{flow_id}","DELETE /gateway/references/reference-types/{type}","DELETE /gateway/references/{reference_id}",- "DELETE /gateway/room-groups/{group_id}","DELETE /gateway/rooms/{room_id}","DELETE /gateway/rooms/{room_id}/agents/{agent_id}","DELETE /gateway/rooms/{room_id}/documents/{document_id}",@@ -192,10 +187,8 @@"GET /gateway/rooms/{room_id}/documents/{document_id}","GET /gateway/rooms/{room_id}/linked-rooms","GET /gateway/rooms/{room_id}/linked-rooms/inbound",- "GET /gateway/rooms/{room_id}/packages","GET /gateway/rooms/{room_id}/references","GET /gateway/rooms/{room_id}/roles",- "GET /gateway/rooms/{room_id}/yaml","GET /gateway/service-connections","GET /gateway/service-connections/{service}/agents","GET /gateway/service-connections/{service}/flows/{flow_id}",@@ -261,7 +254,6 @@"POST /gateway/management/operations","POST /gateway/messaging-apps/{platform}/claim","POST /gateway/messaging-apps/{platform}/install",- "POST /gateway/packages","POST /gateway/packages/{package_id}/documents/{document_id}","POST /gateway/packages/{package_id}/references/{reference_id}","POST /gateway/provider-connections/github/flows",@@ -272,7 +264,6 @@"POST /gateway/rooms/bulk-archive","POST /gateway/rooms/bulk-delete","POST /gateway/rooms/from-template",- "POST /gateway/rooms/from-template-document","POST /gateway/rooms/{room_id}/agents","POST /gateway/rooms/{room_id}/archive","POST /gateway/rooms/{room_id}/documents/{document_id}",@@ -294,7 +285,6 @@"POST /gateway/tenants/{tenant_id}/join-domains","POST /gateway/tenants/{tenant_id}/switch","POST /gateway/users",- "PUT /gateway/agents/{agent_id}/addressing-policy","PUT /gateway/agents/{agent_id}/can-manage-agents","PUT /gateway/agents/{agent_id}/description","PUT /gateway/agents/{agent_id}/display-name",diff --git a/core/switch_core/db/stores/room_store.py b/core/switch_core/db/stores/room_store.pyindex a8968a92f..1730754d2 100644--- a/core/switch_core/db/stores/room_store.py+++ b/core/switch_core/db/stores/room_store.py@@ -15,6 +15,10 @@ from switch_core.db.models import (RoomGroup,require_tenant_id,room_agents,+ room_documents,+ room_packages,+ room_references,+ room_skills,)from switch_core.db.stores.delivery_cursor_store import DeliveryCursorStore@@ -248,9 +252,14 @@ class RoomStore:async def delete(self, session: AsyncSession, room_id: str) -> None:await session.execute(delete(ClientRoom).where(ClientRoom.room_id == room_id))- await session.execute(- delete(room_agents).where(room_agents.c.room_id == room_id)- )+ for attached in (+ room_agents,+ room_references,+ room_documents,+ room_packages,+ room_skills,+ ):+ await session.execute(delete(attached).where(attached.c.room_id == room_id))room = await session.get(Room, room_id)if room:await session.delete(room)diff --git a/core/tests/switch_core/db/stores/test_room_store.py b/core/tests/switch_core/db/stores/test_room_store.pyindex df845f221..f51c71211 100644--- a/core/tests/switch_core/db/stores/test_room_store.py+++ b/core/tests/switch_core/db/stores/test_room_store.py@@ -5,8 +5,17 @@ import uuidimport pytestfrom sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker-from switch_core.db.models import Room, Tenant, TenantNotBoundError+from switch_core.db.models import (+ Package,+ Reference,+ Room,+ Tenant,+ TenantNotBoundError,+ User,+)from switch_core.db.session_scope import tenant_session+from switch_core.db.stores.package_store import PackageStore+from switch_core.db.stores.reference_store import ReferenceStorefrom switch_core.db.stores.room_store import RoomStore@@ -111,3 +120,52 @@ class TestGetByTransportRoomIdRequiresATenant:async with session_factory() as verify:with pytest.raises(TenantNotBoundError):await store.get_by_transport_room_id(verify, shared_transport_room_id)+++class TestDelete:+ async def test_deletes_a_room_with_a_package_and_a_reference_attached(+ self, session_factory: async_sessionmaker[AsyncSession]+ ) -> None:+ packages, references = PackageStore(), ReferenceStore()+ async with session_factory() as session:+ owner = User(name="owner", email="[email protected]", role="admin")+ session.add(owner)+ await session.flush()+ room = await _make_room(session, "!attached:switch.local")+ package = await packages.create(+ session,+ Package(+ owner_id=owner.id,+ read_visibility="private",+ write_visibility="private",+ name="pkg",+ description="d",+ instructions="i",+ ),+ )+ reference = await references.create(+ session,+ Reference(+ owner_id=owner.id,+ read_visibility="private",+ write_visibility="private",+ type="url",+ name="ref",+ description="d",+ instructions="i",+ value={"url": "https://example.com"},+ ),+ )+ await packages.attach_to_room(session, room.id, package.id)+ await references.attach_to_room(session, room.id, reference.id)+ await session.commit()+ room_id, package_id, reference_id = room.id, package.id, reference.id++ async with session_factory() as session:+ await RoomStore().delete(session, room_id)+ await session.commit()++ async with session_factory() as session:+ assert await session.get(Room, room_id) is None+ assert await packages.get(session, package_id) is not None+ assert await session.get(Reference, reference_id) is not None