Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.
Python agent-orchestrationaiai-agentschatopsdiscordlangchainmcpmicrosoft-teamsmulti-agentopen-sourceopenaiself-hostedslack
E2E provider lane on a headless agent controller
Petr Baucheea8ed01a9a0
diff --git a/console/tooling-e2e/README.md b/console/tooling-e2e/README.mdindex 6f36ec74e..b8f620dc8 100644--- a/console/tooling-e2e/README.md+++ b/console/tooling-e2e/README.md@@ -155,15 +155,61 @@ beside this file. Never commit one. `just e2e-fast` sets all of them itself.| `SWITCH_E2E_SESSION_TIMEOUT_MS` | provider lane | How long setup waits for a live session (default 2 min). || `SWITCH_E2E_AGENT_TYPE` | provider lane | `opencode` (default), `claude-code`, `codex`, `antigravity` or `cursor` — which provider's runtime is under test. || `SWITCH_E2E_QUESTION_MODE` | provider lane | `numbered` (default) or `prose`; see the `question` row below. |+| `SWITCH_E2E_CONTROLLER_ID` | provider lane | An agent controller to run the agent on. `just e2e-provider` sets it. Unset, a Switch Console session is expected instead. |+| `SWITCH_E2E_PROVIDER_ENV_FILE` | `just e2e-provider` | A file of provider keys, handed to the controller as `--env-file`. Leave it unset to use the CLI's own sign-in. |Mattermost is all or nothing: none of its three variables means "no Mattermosthere"; some of them is reported as a mistake.# The provider lane-## What must be running+The provider lane needs Mattermost and something that runs a real session for+the agent it registers. There are two ways to provide one.-The provider lane needs Mattermost and a **Switch Console session for the agent+## On a headless agent controller (`just e2e-provider`)++The unattended way, and the one CI will use. `just e2e-provider` does+everything `just e2e-fast` does, and also:++1. turns on `agent_management` and sets `CONTROLLER_TOKEN_SECRET` for the stack;+2. builds `console/packages` and starts the agent controller+ (`console/packages/agent-controller`) on the host. Its credential comes from+ `POST /gateway/management/controllers` and goes in on stdin, never to disk;+3. runs the provider lane with `SWITCH_E2E_CONTROLLER_ID` set. Setup then+ registers the agent with a registration key the admin mints for the purpose+ and deletes straight away, because the management API only places agents+ their caller owns, and the deployment-wide key registers agents to a+ bootstrap account. Setup places the agent on the controller with+ `auto_approve: false`, retrying while the controller reports itself not+ ready, and writes no Switch Console files.++```bash+SWITCH_E2E_AGENT_TYPE=claude-code just e2e-provider+SWITCH_E2E_PROVIDER_ENV_FILE=~/e2e-keys.env just e2e-provider # an API key instead of a signed-in CLI+```++The provider CLI (`claude`, `opencode`, …) must be installed and signed in for+the user running it, or its key must be in the file `SWITCH_E2E_PROVIDER_ENV_FILE`+names, which the controller gets as `--env-file`. Sessions only inherit the+variables on the allow-list in+`console/packages/agent-providers/src/host/agent-env.ts`. `node+console/packages/agent-controller/dist/cli.mjs doctor` shows what the controller+sees. A run spends real model calls.++Two things differ from a Console session:+- **The agent's own key stops working.** While a controller acts for an agent,+ the server refuses requests made with the agent's own key. So the+ bridge-readiness probe reads the room through a silent second agent, the+ *witness*, and teardown deletes agents by name as the admin.+- **`live` comes early.** Switch reports the agent `live` as soon as the+ controller is connected, before any session exists. The first scenario+ therefore absorbs the session's start-up time.++The controller's log goes to `build/e2e/ephemeral/controller.log`.++## With a Switch Console session++The provider lane can also run against a **Switch Console session for the agentit just registered**. Setup polls the agent's status in the room and refuses to run thescenarios until Switch reports it `live` (a session is attending the room) or`dormant` (an auto-session watcher is running and will spawn one on demand) —@@ -243,7 +289,7 @@ agent, and deletes it during teardown — so a manifest never outlives its room.| --- | --- | --- || `greet` | An addressed message reaches a session and its answer comes back. | The bot posts `SWITCH_E2E_OK`. || `question` | Two turns: the agent asks, the human answers in the channel, the agent uses the answer. | The bot offers `red`/`green`/`blue`, then replies `green` after the answer. |-| `approval` | A tool call the session is not pre-authorised to make. | The bot surfaces an approval prompt, `@agent 1` allows it, the bot replies `done`. |+| `approval` | A tool call the session is not pre-authorised to make. | The bot posts an approval card, answering it with `<handle> 1` allows the call, the bot replies `done`. || `interrupt` | `!interrupt @agent-name` cancels a running local shell command after its start marker is verified. | No completion marker appears after the command’s 90-second lifetime (observed for 95 seconds). Requires `SWITCH_E2E_AGENT_DIR` on this machine. |`question` is the one a one-shot reply cannot fake: it only passes if the session@@ -251,9 +297,10 @@ is still alive and still connected when the second message arrives.It has two modes, because the relay it exercises needs a native ask-the-usertool and not every provider offers one. In `numbered` mode the agent must use-that tool, the console relays it into the room as a numbered list, and the-answer goes back as `@agent 2` — the numbering is what proves the relay path was-taken. In `prose` mode the agent asks in an ordinary message and is answered+that tool, and Switch relays it into the room as a request card numbering the+options (`… request \`A1\` … 2. green …`). The answer goes back as `A1 2`, a+plain message naming the card's handle, and the card is what proves the relay+path was taken. In `prose` mode the agent asks in an ordinary message and is answeredwith `@agent green`; the two-turn round trip is still proved, the relay is not.Every agent type defaults to `numbered` — both providers offer the tool — and`prose` is an explicit override for a provider or CLI version that does not.diff --git a/console/tooling-e2e/scripts/ephemeral-stack.sh b/console/tooling-e2e/scripts/ephemeral-stack.shindex b75681c2c..4b839fca5 100755--- a/console/tooling-e2e/scripts/ephemeral-stack.sh+++ b/console/tooling-e2e/scripts/ephemeral-stack.sh@@ -4,9 +4,16 @@# host, then the chosen lane, then teardown.## just e2e-fast the fast lane (no provider, no Console)+# just e2e-provider the provider lane, on a headless agent controller# just e2e-fast --keep leave the stack running afterwards# just e2e-fast -- <vitest args> pass arguments through to vitest#+# The provider lane also builds console/packages and starts an agent controller+# on the host, which runs the agent's sessions with a real provider CLI. The CLI+# must be installed and signed in for the user running this, or its key given in+# the file SWITCH_E2E_PROVIDER_ENV_FILE names (passed to the controller as+# --env-file). SWITCH_E2E_AGENT_TYPE picks the provider (default opencode).+## Secrets are generated per run into a temporary env file. The repository's# own .env is never read or written, so this cannot rotate a developer's live# secrets or pick up a stale value from them.@@ -44,6 +51,12 @@ rm -rf "$OUT"; mkdir -p "$OUT"ENV_FILE="$OUT/stack.env"cp "$REPO/.env.example" "$ENV_FILE""$REPO/deploy/local/generate-env-secrets.sh" "$ENV_FILE" e2e+if [ "$LANE" = provider ]; then+ # Agent management, and the key controllers' access tokens are signed with.+ sed -i.bak -e "s|^FEATURE_FLAGS_ENABLED=.*|FEATURE_FLAGS_ENABLED=agent_management|" \+ -e "s|^CONTROLLER_TOKEN_SECRET=.*|CONTROLLER_TOKEN_SECRET=$(openssl rand -hex 32)|" "$ENV_FILE"+ rm -f "$ENV_FILE.bak"+fichmod 600 "$ENV_FILE"set -a; source "$ENV_FILE"; set +a@@ -51,16 +64,18 @@ cd "$REPO"COMPOSE=(docker compose -p switch-e2e-ephemeral -f deploy/local/docker-compose.yml--env-file "$ENV_FILE" --project-directory .)SERVER_PID=""+CONTROLLER_PID=""cleanup() {local status=$?if [ "$KEEP" = 0 ]; then+ [ -n "$CONTROLLER_PID" ] && kill "$CONTROLLER_PID" 2>/dev/null && wait "$CONTROLLER_PID" 2>/dev/null[ -n "$SERVER_PID" ] && kill "$SERVER_PID" 2>/dev/null && wait "$SERVER_PID" 2>/dev/null"${COMPOSE[@]}" down -v >/dev/null 2>&1 || truerm -f "$ENV_FILE"else- echo "Left running: switch-core :8000 (pid $SERVER_PID), Mattermost :8065."+ echo "Left running: switch-core :8000 (pid $SERVER_PID), Mattermost :8065${CONTROLLER_PID:+, agent controller (pid $CONTROLLER_PID)}."echo "Secrets for this stack: $ENV_FILE. Tear down with:"- echo " kill $SERVER_PID; ${COMPOSE[*]} down -v"+ echo " kill $SERVER_PID $CONTROLLER_PID; ${COMPOSE[*]} down -v"fiexit "$status"}@@ -101,10 +116,49 @@ MATTERMOST_TOKEN="$(curl -sf -D - -o /dev/null -H 'Content-Type: application/jsohttp://localhost:8065/api/v4/users/login | tr -d '\r' | awk -F': ' 'tolower($1)=="token" {print $2}')"[ -n "$MATTERMOST_TOKEN" ] || { echo "could not log in to Mattermost as $MATTERMOST_USER" >&2; exit 1; }+PROVIDER_ENV=()+if [ "$LANE" = provider ]; then+ echo "Building the agent controller…"+ (cd "$REPO/console" && pnpm -r --filter './packages/**' run build) >"$OUT/console-build.log" 2>&1 \+ || { echo "console package build failed; see $OUT/console-build.log" >&2; exit 1; }++ JAR="$OUT/gateway.cookies"+ curl -sf -c "$JAR" -o /dev/null -H 'Content-Type: application/json' \+ -d "$(jq -n --arg e "$GATEWAY_ADMIN_EMAIL" --arg p "$GATEWAY_ADMIN_PASSWORD" '{email: $e, password: $p}')" \+ http://localhost:8000/gateway/auth/login \+ || { echo "could not sign in to the gateway as $GATEWAY_ADMIN_EMAIL" >&2; exit 1; }+ CONTROLLER="$(curl -sf -b "$JAR" -H 'Content-Type: application/json' \+ -d "$(jq -n --arg os "$(uname -s | tr '[:upper:]' '[:lower:]')" --arg arch "$(uname -m)" \+ --arg osv "$(uname -r)" \+ '{name: "e2e-controller", kind: "console", version: "e2e",+ platform: {os: $os, arch: $arch, os_version: $osv}}')" \+ http://localhost:8000/gateway/management/controllers)" \+ || { echo "could not create an agent controller (is agent_management on?)" >&2; exit 1; }+ rm -f "$JAR"+ CONTROLLER_ID="$(jq -r .controller_id <<<"$CONTROLLER")"++ echo "Starting the agent controller…"+ CONTROLLER_ARGS=(run --data-dir "$OUT/controller" --controller-id "$CONTROLLER_ID"+ --server http://localhost:8000 --credential-stdin)+ if [ -n "${SWITCH_E2E_PROVIDER_ENV_FILE:-}" ]; then+ CONTROLLER_ARGS+=(--env-file "$SWITCH_E2E_PROVIDER_ENV_FILE")+ fi+ # The credential goes in on stdin and is never written to disk.+ jq -r .credential <<<"$CONTROLLER" | SWITCH_CONTROLLER_LOG_LEVEL=info \+ node "$REPO/console/packages/agent-controller/dist/cli.mjs" "${CONTROLLER_ARGS[@]}" \+ >"$OUT/controller.log" 2>&1 &+ CONTROLLER_PID=$!+ unset CONTROLLER++ mkdir -p "$OUT/agent-dir"+ PROVIDER_ENV=(SWITCH_E2E_CONTROLLER_ID="$CONTROLLER_ID" SWITCH_E2E_AGENT_DIR="$OUT/agent-dir")+fi+echo "Running the $LANE lane…"status=0(cd "$HARNESS"+ export ${PROVIDER_ENV[@]+"${PROVIDER_ENV[@]}"}SWITCH_E2E_LANE="$LANE" \SWITCH_E2E_ENVIRONMENT=ephemeral \SWITCH_E2E_STRICT=1 \@@ -121,5 +175,6 @@ status=0if [ "$status" != 0 ]; thenecho "The $LANE lane failed. switch-core log: $OUT/switch-core.log" >&2+ [ "$LANE" = provider ] && echo "Agent controller log: $OUT/controller.log" >&2fiexit "$status"diff --git a/console/tooling-e2e/src/clients/switch-client.ts b/console/tooling-e2e/src/clients/switch-client.tsindex 391b5bc9d..bee901704 100644--- a/console/tooling-e2e/src/clients/switch-client.ts+++ b/console/tooling-e2e/src/clients/switch-client.ts@@ -38,6 +38,15 @@ export interface RoomDetail extends RoomSummary {agentIds: string[];}+/** The management API's agent definition (v1), as much of it as the harness sets. */+export interface ManagedAgentDefinition {+ provider: string;+ model: string | null;+ instructions: string;+ auto_approve: boolean;+ directory: string | null;+}+export interface AgentSummary {id: string;name: string;@@ -113,7 +122,8 @@ interface SwitchRequest {export class SwitchClient {private readonly apiUrl: string;- private readonly registrationToken: string;+ /** The deployment-wide registration key; agents it registers belong to a bootstrap account. */+ readonly registrationToken: string;private readonly adminEmail: string;private readonly adminPassword: string;private readonly environment: Environment;@@ -263,6 +273,42 @@ export class SwitchClient {await this.gateway('DELETE', `/agents/by-name/${encodeURIComponent(name)}`);}+ /**+ * `POST /gateway/api-keys` — a registration key belonging to the signed-in+ * admin. Agents registered with it are owned by the admin, which the+ * management API requires of an agent it is asked to place on a controller;+ * agents registered with the deployment-wide key belong to a bootstrap+ * account instead.+ */+ async createRegistrationKey(label: string): Promise<{ id: string; key: string }> {+ const body = await this.gateway<{ id: string; key: string }>('POST', '/api-keys', { label });+ return { id: body.id, key: body.key };+ }++ async deleteApiKey(keyId: string): Promise<void> {+ await this.gateway('DELETE', `/api-keys/${keyId}`);+ }++ /**+ * `PUT /gateway/management/agents/{id}` — place an agent the admin owns on a+ * controller and set it running. The controller then runs its sessions.+ *+ * Answers 409 until the controller has connected and reported the provider+ * installed and signed in; the caller retries on {@link SwitchHttpError}+ * status 409 while the controller comes up.+ */+ async placeAgent(params: {+ agentId: string;+ controllerId: string;+ definition: ManagedAgentDefinition;+ }): Promise<void> {+ await this.gateway('PUT', `/management/agents/${params.agentId}`, {+ controller_id: params.controllerId,+ desired_state: 'running',+ definition: params.definition,+ });+ }+/*** `PUT /gateway/agents/{id}/addressing-policy` with `{policy: null}` — clear* the agent's scoped addressing policy, so anyone may address it.@@ -362,6 +408,8 @@ export class SwitchClient {* be added to a channel.*/async registerKnownAgent(params: {+ /** {@link registrationToken}, or a personal key from {@link createRegistrationKey}. */+ registrationToken: string;agentType: 'opencode' | 'codex' | 'claude-code' | 'antigravity' | 'cursor';name: string;description: string;@@ -371,7 +419,7 @@ export class SwitchClient {'POST','/agents/register-known',{- token: this.registrationToken,+ token: params.registrationToken,body: {agent_type: params.agentType,name: params.name,diff --git a/console/tooling-e2e/src/core/env.ts b/console/tooling-e2e/src/core/env.tsindex 1a280d130..8a869d1a2 100644--- a/console/tooling-e2e/src/core/env.ts+++ b/console/tooling-e2e/src/core/env.ts@@ -76,6 +76,12 @@ export interface ProviderEnv {agentType: HarnessAgentType;/** How `question` expects the clarifying question to arrive. */questionMode: QuestionMode;+ /**+ * A headless agent controller to run the agent on. When set, setup places+ * the agent on it through the management API and writes no Switch Console+ * files; when not, a Console session for the agent is expected instead.+ */+ controllerId: string | null;}export interface HarnessEnv {@@ -242,6 +248,7 @@ export function loadEnv(manifestPath: get('SWITCH_E2E_MANIFEST') ?? null,agentType: agentTypeFrom(get('SWITCH_E2E_AGENT_TYPE')),questionMode: questionModeFrom(get('SWITCH_E2E_QUESTION_MODE')),+ controllerId: get('SWITCH_E2E_CONTROLLER_ID') ?? null,},};}diff --git a/console/tooling-e2e/src/fixtures/fixtures.ts b/console/tooling-e2e/src/fixtures/fixtures.tsindex 691886517..d00c96560 100644--- a/console/tooling-e2e/src/fixtures/fixtures.ts+++ b/console/tooling-e2e/src/fixtures/fixtures.ts@@ -53,6 +53,7 @@ export class Fixtures {): Promise<RegisteredAgent> {const name = this.names.name(role);const agent = await this.switchClient.registerKnownAgent({+ registrationToken: this.switchClient.registrationToken,agentType: options.agentType ?? 'opencode',name,description: `Switch end-to-end harness agent (run ${this.names.runId}). Safe to delete.`,diff --git a/console/tooling-e2e/src/provider/harness.ts b/console/tooling-e2e/src/provider/harness.tsindex 17269c4bf..38dc491ef 100644--- a/console/tooling-e2e/src/provider/harness.ts+++ b/console/tooling-e2e/src/provider/harness.ts@@ -6,6 +6,7 @@ import type {} from '../clients/mattermost-client.ts';import {sleep,+ SwitchHttpError,type RegisteredAgent,type RoomDetail,type SwitchClient,@@ -31,6 +32,13 @@ export interface Harness {switch: SwitchClient;mattermost: MattermostClient;agent: RegisteredAgent;+ /**+ * A silent second agent in the room, registered with the deployment key and+ * run by nobody. The bridge-readiness probe reads the room through it,+ * because an agent run by a controller cannot use its own key: the server+ * refuses it while the controller acts for the agent.+ */+ witness: RegisteredAgent;/** The agent's Mattermost bot account — `username === agent.name`. */bot: MattermostUser;channel: MattermostChannel;@@ -55,6 +63,102 @@ const ROLE_BY_AGENT_TYPE: Record<HarnessAgentType, string> = {codex: 'codex',};+// "Do exactly what you are asked" is load-bearing, and so is the second+// sentence: told merely to answer briefly, the agent declined the `interrupt`+// scenario's task on the grounds that 200 messages would spam the room — a+// refusal the scenario reads as a session that never started.+const AGENT_INSTRUCTIONS =+ 'You are a Switch end-to-end test agent. Do exactly what you are asked in the ' ++ 'room, and post your answer back to the room you were addressed in. Never ' ++ 'decline or shorten a task on the grounds that it is repetitive or long: the ' ++ 'requests you get here are deliberate tests of exactly that.';++/** Cheap on purpose: a run spends a real subscription. */+const CLAUDE_MODEL = 'claude-sonnet-5';++/** The management API's provider id for each harness agent type. */+const PROVIDER_BY_AGENT_TYPE: Record<HarnessAgentType, string> = {+ opencode: 'opencode',+ 'claude-code': 'claude',+ codex: 'codex',+ antigravity: 'antigravity',+ cursor: 'cursor',+};++/** How long to keep asking a controller to take the agent while it comes up. */+const PLACEMENT_DEADLINE_MS = 120_000;++/**+ * Register the agent. For a controller it is registered with a key the admin+ * mints for the purpose and deletes straight after, because the management API+ * only places agents their caller owns, and the deployment-wide key registers+ * them to a bootstrap account.+ */+async function registerAgent(+ switchClient: SwitchClient,+ env: HarnessEnv,+ name: string,+ runId: string,+ forController: boolean+): Promise<RegisteredAgent> {+ // `ClaudeCodeOptions` and `OpencodeOptions` agree on the two keys used here+ // (`auto_session`, `repo_dir`), so the payload is the same for both types.+ const params = {+ agentType: env.provider.agentType,+ name,+ description: `Switch end-to-end harness agent (run ${runId}). Safe to delete.`,+ options: {+ auto_session: true,+ ...(env.provider.agentRepoDir ? { repo_dir: env.provider.agentRepoDir } : {}),+ },+ };+ if (!forController) {+ return switchClient.registerKnownAgent({+ ...params,+ registrationToken: switchClient.registrationToken,+ });+ }+ const key = await switchClient.createRegistrationKey(`e2e harness run ${runId}`);+ try {+ return await switchClient.registerKnownAgent({ ...params, registrationToken: key.key });+ } finally {+ await switchClient.deleteApiKey(key.id);+ }+}++/**+ * Hand the agent to the controller, retrying while the server answers 409:+ * the controller has not connected yet, or has not yet reported the provider+ * installed and signed in. Any other refusal is final.+ */+async function placeOnController(+ switchClient: SwitchClient,+ env: HarnessEnv,+ agent: RegisteredAgent,+ controllerId: string+): Promise<void> {+ const provider = PROVIDER_BY_AGENT_TYPE[env.provider.agentType];+ const definition = {+ provider,+ model: provider === 'claude' ? CLAUDE_MODEL : null,+ instructions: AGENT_INSTRUCTIONS,+ // The `approval` scenario needs the session to ask before running a tool.+ auto_approve: false,+ directory: env.provider.agentRepoDir,+ };+ const until = Date.now() + PLACEMENT_DEADLINE_MS;+ for (;;) {+ try {+ await switchClient.placeAgent({ agentId: agent.id, controllerId, definition });+ return;+ } catch (error) {+ const notReady = error instanceof SwitchHttpError && error.status === 409;+ if (!notReady || Date.now() > until) throw error;+ await sleep(3_000);+ }+ }+}+/*** Register a throwaway agent, create a room on the Mattermost bridge for it,* and wait for the bridge to carry a message.@@ -89,31 +193,31 @@ export async function setupHarness(target: LaneTarget): Promise<Harness> {// and clearly when MATTERMOST_TEAM names a team that does not exist.await mattermost.findTeam(env.mattermost.team);- // `ClaudeCodeOptions` and `OpencodeOptions` agree on the two keys used here- // (`auto_session`, `repo_dir`), so the payload is the same for both types.- const agent = await switchClient.registerKnownAgent({- agentType: env.provider.agentType,- name: agentName,- description: `Switch Console end-to-end harness agent (run ${runId}). Safe to delete.`,- options: {- auto_session: true,- ...(env.provider.agentRepoDir ? { repo_dir: env.provider.agentRepoDir } : {}),- },- });+ const controllerId = env.provider.controllerId;+ const agent = await registerAgent(switchClient, env, agentName, runId, controllerId !== null);// Owner-only by default, and the human the harness posts as is an unclaimed// Mattermost identity, so its messages would be refused rather than delivered.await switchClient.openAddressing(agent.id);- // Registration is what mints the bot; wait for it so a later `@name` mention- // has something to resolve to.+ const witness = await switchClient.registerKnownAgent({+ registrationToken: switchClient.registrationToken,+ agentType: 'opencode',+ name: names.name('witness'),+ description: `Switch end-to-end harness bridge witness (run ${runId}). Safe to delete.`,+ options: { auto_session: false },+ });++ // Registration is what mints the bots; wait for them so the room has them+ // as members and a later `@name` mention has something to resolve to.const bot = await mattermost.waitForBotUser(agent.name, 60_000);+ await mattermost.waitForBotUser(witness.name, 60_000);const room = await switchClient.createRoom({name: names.name('room'),- description: `Switch Console end-to-end harness (run ${runId}). Safe to delete.`,+ description: `Switch end-to-end harness (run ${runId}). Safe to delete.`,bridgeId,- agentNames: [agent.name],+ agentNames: [agent.name, witness.name],userNames: [human.username],});@@ -130,6 +234,7 @@ export async function setupHarness(target: LaneTarget): Promise<Harness> {switch: switchClient,mattermost,agent,+ witness,bot,channel,room,@@ -137,10 +242,12 @@ export async function setupHarness(target: LaneTarget): Promise<Harness> {runId,};- // With SWITCH_E2E_AGENT_DIR set, provision the agent's identity into that- // working directory so a session started there IS this agent. Without it the- // operator has to add the agent in Switch Console by hand before the run.- if (env.provider.agentRepoDir) {+ if (controllerId !== null) {+ await placeOnController(switchClient, env, agent, controllerId);+ } else if (env.provider.agentRepoDir) {+ // Provision the agent's identity into that working directory so a Console+ // session started there IS this agent. Without it the operator has to add+ // the agent in Switch Console by hand before the run.harness.credentialsPath = await writeAgentCredentials({workingDir: env.provider.agentRepoDir,agent,@@ -153,24 +260,14 @@ export async function setupHarness(target: LaneTarget): Promise<Harness> {workingDir: env.provider.agentRepoDir,agentName: agent.name,description: `Switch Console end-to-end harness agent (run ${runId}).`,- // "Do exactly what you are asked" is load-bearing, and so is the second- // sentence: told merely to answer briefly, the agent declined the- // `interrupt` scenario's task on the grounds that 200 messages would- // spam the room — a refusal the scenario reads as a session that never- // started.- instructions:- 'You are a Switch end-to-end test agent. Do exactly what you are asked in the ' +- 'room, and post your answer back to the room you were addressed in. Never ' +- 'decline or shorten a task on the grounds that it is repetitive or long: the ' +- 'requests you get here are deliberate tests of exactly that.',- // Cheap on purpose: this spends a real subscription on every run.- model: 'claude-sonnet-5',+ instructions: AGENT_INSTRUCTIONS,+ model: CLAUDE_MODEL,effort: 'low',});}}- await waitForBridgeReady({ switchClient, mattermost, agent, room, channel, runId });+ await waitForBridgeReady({ switchClient, mattermost, agent: witness, room, channel, runId });await writeManifest(env.provider.manifestPath, harness);return harness;}@@ -188,6 +285,7 @@ export async function setupHarness(target: LaneTarget): Promise<Harness> {export interface HarnessManifest {runId: string;agent: RegisteredAgent;+ witness: RegisteredAgent;roomId: string;channelId: string;}@@ -197,6 +295,7 @@ async function writeManifest(path: string | null, harness: Harness): Promise<voiconst manifest: HarnessManifest = {runId: harness.runId,agent: harness.agent,+ witness: harness.witness,roomId: harness.room.id,channelId: harness.channel.id,};@@ -228,6 +327,7 @@ async function reuseHarness(switch: switchClient,mattermost,agent: manifest.agent,+ witness: manifest.witness,bot: await mattermost.waitForBotUser(manifest.agent.name, 30_000),channel: await mattermost.getChannel(manifest.channelId),room: await switchClient.getRoom(manifest.roomId),@@ -238,7 +338,7 @@ async function reuseHarness(await waitForBridgeReady({switchClient,mattermost,- agent: harness.agent,+ agent: harness.witness,room: harness.room,channel: harness.channel,runId: harness.runId,@@ -317,13 +417,17 @@ export async function teardownHarness();}- try {- await harness.switch.deleteAgent(harness.agent);- } catch (error) {- log(- `teardown: failed to delete agent ${harness.agent.name} (${harness.agent.id}) — ` +- `delete it by hand: ${error instanceof Error ? error.message : String(error)}`- );+ // By name, as the admin: an agent run by a controller cannot delete itself+ // with its own key, which the server refuses while the controller acts for it.+ for (const agent of [harness.agent, harness.witness]) {+ try {+ await harness.switch.deleteAgentByName(agent.name);+ } catch (error) {+ log(+ `teardown: failed to delete agent ${agent.name} (${agent.id}) — ` ++ `delete it by hand: ${error instanceof Error ? error.message : String(error)}`+ );+ }}// Deleting the room usually takes the channel with it; archive is a no-opdiff --git a/console/tooling-e2e/src/provider/scenarios.test.ts b/console/tooling-e2e/src/provider/scenarios.test.tsindex ede5d897c..c51b020aa 100644--- a/console/tooling-e2e/src/provider/scenarios.test.ts+++ b/console/tooling-e2e/src/provider/scenarios.test.ts@@ -1,5 +1,5 @@import { describe, expect, it } from 'vitest';-import { isStartMarker } from './scenarios.ts';+import { isStartMarker, requestHandle } from './scenarios.ts';describe('interruption admission', () => {it('requires the command marker, not an acknowledgement or refusal', () => {@@ -10,3 +10,13 @@ describe('interruption admission', () => {expect(isStartMarker('I will write SWITCH_INTERRUPT_STARTED')).toBe(false);});});++describe('request cards', () => {+ it('reads the handle a card answers to', () => {+ expect(requestHandle('@user **Questions** · request `A1` Question from the agent 1. red')).toBe(+ 'A1'+ );+ expect(requestHandle('**Permission** · request `R42` Run `echo hi`?')).toBe('R42');+ expect(requestHandle('SWITCH_E2E_OK')).toBeNull();+ });+});diff --git a/console/tooling-e2e/src/provider/scenarios.ts b/console/tooling-e2e/src/provider/scenarios.tsindex eeb6b4d5f..fba4d761f 100644--- a/console/tooling-e2e/src/provider/scenarios.ts+++ b/console/tooling-e2e/src/provider/scenarios.ts@@ -132,6 +132,24 @@ async function scenario(}}+/**+ * The handle a request card answers to: `A1` in "request `A1`". Switch renders+ * every relayed question and permission prompt as such a card, and an answer+ * is the handle followed by the option's number, typed as a plain message.+ */+export function requestHandle(text: string): string | null {+ return /request `([A-Za-z]+\d+)`/.exec(text)?.[1] ?? null;+}++/** Answer a request card, and return the moment just before the answer was posted. */+async function answerCard(harness: Harness, card: MattermostPost, option: number): Promise<number> {+ const handle = requestHandle(card.message);+ if (handle === null) throw new Error(`Post ${card.id} is not a request card: «${card.message}»`);+ const sinceMs = Date.now() - 1;+ await harness.mattermost.post({ channelId: harness.channel.id, message: `${handle} ${option}` });+ return sinceMs;+}+/** Post `@agent <text>` and return the moment just before it was posted. */async function ask(harness: Harness, text: string): Promise<number> {const sinceMs = Date.now() - 1;@@ -234,27 +252,31 @@ export const question: Scenario = (harness) =>const asked = await expectBotPost(harness, {since,record,- // The console relays a native question into the room as a numbered list;- // a plain-prose question also lists the three options, so the numbering is+ // A native question is relayed as a request card numbering the options;+ // a plain-prose question also lists the three options, so the card is// what proves the relay path was taken — and is exactly what cannot be// required of a provider that offers SDK sessions no such tool.predicate: (post) =>CHOICES.every((choice) => lower(post).includes(choice)) &&- (prose || /\b2\.\s/.test(post.message)),+ (prose || (requestHandle(post.message) !== null && /\b2\.\s/.test(post.message))),describe: prose? `a question listing the options ${CHOICES.join('/')}`: `a relayed question numbering the options ${CHOICES.join('/')}`,});- // By number where the relay is what reads the answer; by word where the- // agent itself is. The relay's acknowledgement names the choice too, so the- // agent's own reply must be told apart from it either way.- const answeredAt = await ask(harness, prose ? CHOSEN : '2');+ // On the card where the relay is what reads the answer; by word where the+ // agent itself is. The card and the relay's acknowledgement name the choice+ // too, so the agent's own reply must be told apart from them either way.+ const answeredAt = prose+ ? await ask(harness, CHOSEN)+ : await answerCard(harness, asked, CHOICES.indexOf(CHOSEN) + 1);const answer = await expectBotPost(harness, {since: answeredAt,record,predicate: (post) =>- lower(post).includes(CHOSEN) && !post.message.trimStart().startsWith('✅'),+ lower(post).includes(CHOSEN) &&+ requestHandle(post.message) === null &&+ !post.message.trimStart().startsWith('✅'),describe: `a reply naming my choice '${CHOSEN}'`,});@@ -288,12 +310,12 @@ export const approval: Scenario = (harness) =>since,record,predicate: (post) =>- APPROVAL_HINTS.some((hint) => lower(post).includes(hint)) ||- /(^|\n)\s*1[.)]/.test(post.message),- describe: 'an approval prompt for the shell command',+ requestHandle(post.message) !== null &&+ APPROVAL_HINTS.some((hint) => lower(post).includes(hint)),+ describe: 'an approval card for the shell command',});- const approvedAt = await ask(harness, '1');+ const approvedAt = await answerCard(harness, prompt, 1);const done = await expectBotPost(harness, {since: approvedAt,record,@@ -334,13 +356,15 @@ export const interrupt: Scenario = (harness) =>harness,`Run this exact shell command for a bounded cancellation test: \`${command}\`. Wait for completion before replying. Do not create these files with another tool.`);- await expectBotPost(harness, {+ const permission = await expectBotPost(harness, {since,record,- predicate: (post) => APPROVAL_HINTS.some((hint) => lower(post).includes(hint)),- describe: 'permission to run the cancellation command',+ predicate: (post) =>+ requestHandle(post.message) !== null &&+ APPROVAL_HINTS.some((hint) => lower(post).includes(hint)),+ describe: 'an approval card for the cancellation command',});- await ask(harness, '1');+ await answerCard(harness, permission, 1);const deadline = Date.now() + 60_000;while (!isStartMarker((await markerContents(startedFile)) ?? '')) {if (Date.now() > deadline)diff --git a/justfile b/justfileindex 725d9b6e6..cdb5b148a 100644--- a/justfile+++ b/justfile@@ -168,6 +168,14 @@ e2e-follow-ups *args:e2e-fast *args:console/tooling-e2e/scripts/ephemeral-stack.sh {{ args }}+# ── End-to-end harness, provider lane, on a throwaway stack ────────────────────+# As e2e-fast, plus a headless agent controller that runs the agent's sessions+# with a real provider CLI (installed and signed in, or its key in the file+# SWITCH_E2E_PROVIDER_ENV_FILE names). SWITCH_E2E_AGENT_TYPE picks the provider.+# Spends real model calls.+e2e-provider *args:+ SWITCH_E2E_LANE=provider console/tooling-e2e/scripts/ephemeral-stack.sh {{ args }}+# ── Run integration tests (real Postgres via testcontainers) ───────────────────# DOCKER_HOST is auto-resolved from the active docker context in conftest, so this# works under Docker Desktop / OrbStack / colima without extra setup.