Skip to content

Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.

Address provider-lane review: export leak, controller gate, managed create

Petr Bauch26b86fdbac8f
diff --git a/console/tooling-e2e/README.md b/console/tooling-e2e/README.md
index b8f620dc8..643c1cc82 100644
--- a/console/tooling-e2e/README.md
+++ b/console/tooling-e2e/README.md
@@ -174,14 +174,15 @@ everything `just e2e-fast` does, and also:
1. turns on `agent_management` and sets `CONTROLLER_TOKEN_SECRET` for the stack;
2. builds `console/packages` and starts the agent controller
(`console/packages/agent-controller`) on the host. Its credential comes from
- `POST /gateway/management/controllers` and goes in on stdin, never to disk;
-3. runs the provider lane with `SWITCH_E2E_CONTROLLER_ID` set. Setup then
- registers the agent with a registration key the admin mints for the purpose
- and deletes straight away, because the management API only places agents
- their caller owns, and the deployment-wide key registers agents to a
- bootstrap account. Setup places the agent on the controller with
- `auto_approve: false`, retrying while the controller reports itself not
- ready, and writes no Switch Console files.
+ `POST /gateway/management/controllers` and goes in on stdin, never to disk.
+ The lane does not start until the controller has connected to Switch; if it
+ exits or never connects, the run stops there and prints the controller log;
+3. runs the provider lane with `SWITCH_E2E_CONTROLLER_ID` set. Setup creates the
+ agent on the controller as the admin (`POST /gateway/management/agents`,
+ `auto_approve: false`), retrying only while the server answers that the
+ controller is offline or has not reported the provider installed and signed
+ in. Any other refusal fails setup at once. Setup writes no Switch Console
+ files.
```bash
SWITCH_E2E_AGENT_TYPE=claude-code just e2e-provider
@@ -282,6 +283,9 @@ the room before the greeting proceeds.
Setup finds the manifest, reuses what is in it rather than registering a second
agent, and deletes it during teardown — so a manifest never outlives its room.
+A manifest missing a field this version records (one written by an older
+harness, say) fails setup with instructions to clean up and seed again, rather
+than being half-used.
## The provider scenarios
@@ -339,6 +343,8 @@ POST /gateway/rooms {name, description, bridge_id, agent_n
Mattermost channel and adds the bot
DELETE /gateway/rooms/{id}
DELETE /gateway/agents/by-name/{name} (admin teardown, no agent key needed)
+POST /gateway/management/agents {"name","description","controller_id","desired_state":"running","definition"}
+ -> {"agent_id":"…","name":"…",…}; 409 {"error":{"code":"controller_offline"|"provider_not_installed"|"provider_login_missing",…}} until the controller is ready
PUT /gateway/agents/{id}/addressing-policy {"policy": null} -> anyone may address it
POST /gateway/rooms with internal_only: true and no bridge_id -> a room with no channel
```
diff --git a/console/tooling-e2e/scripts/ephemeral-stack.sh b/console/tooling-e2e/scripts/ephemeral-stack.sh
index 4b839fca5..6629a587f 100755
--- a/console/tooling-e2e/scripts/ephemeral-stack.sh
+++ b/console/tooling-e2e/scripts/ephemeral-stack.sh
@@ -150,6 +150,19 @@ if [ "$LANE" = provider ]; then
CONTROLLER_PID=$!
unset CONTROLLER
+ # A controller that exits at once (bad credential, missing build, unreadable
+ # --env-file) would otherwise surface two minutes later as a placement timeout.
+ for _ in $(seq 60); do
+ kill -0 "$CONTROLLER_PID" 2>/dev/null || break
+ grep -q "Connected to Switch" "$OUT/controller.log" && break
+ sleep 1
+ done
+ if ! kill -0 "$CONTROLLER_PID" 2>/dev/null || ! grep -q "Connected to Switch" "$OUT/controller.log"; then
+ echo "The agent controller did not connect to Switch; its log ($OUT/controller.log):" >&2
+ tail -n 40 "$OUT/controller.log" >&2
+ exit 1
+ fi
+
mkdir -p "$OUT/agent-dir"
PROVIDER_ENV=(SWITCH_E2E_CONTROLLER_ID="$CONTROLLER_ID" SWITCH_E2E_AGENT_DIR="$OUT/agent-dir")
fi
@@ -158,7 +171,7 @@ echo "Running the $LANE lane…"
status=0
(
cd "$HARNESS"
- export ${PROVIDER_ENV[@]+"${PROVIDER_ENV[@]}"}
+ if [ ${#PROVIDER_ENV[@]} -gt 0 ]; then export "${PROVIDER_ENV[@]}"; fi
SWITCH_E2E_LANE="$LANE" \
SWITCH_E2E_ENVIRONMENT=ephemeral \
SWITCH_E2E_STRICT=1 \
diff --git a/console/tooling-e2e/src/clients/switch-client.ts b/console/tooling-e2e/src/clients/switch-client.ts
index bee901704..1d42e853c 100644
--- a/console/tooling-e2e/src/clients/switch-client.ts
+++ b/console/tooling-e2e/src/clients/switch-client.ts
@@ -17,9 +17,12 @@ import { assertAllowed } from '../core/guard.ts';
* modelled here; see README.md for the verified request/response shapes.
*/
-export interface RegisteredAgent {
+export interface AgentRef {
id: string;
name: string;
+}
+
+export interface RegisteredAgent extends AgentRef {
apiKey: string;
}
@@ -109,6 +112,20 @@ export class SwitchHttpError extends Error {
}
}
+/**
+ * The reason code of a management API refusal (`{"error": {"code": …}}`), or
+ * null when the error is not one.
+ */
+export function managementErrorCode(error: unknown): string | null {
+ if (!(error instanceof SwitchHttpError)) return null;
+ try {
+ const parsed = JSON.parse(error.body) as { error?: { code?: unknown } };
+ return typeof parsed.error?.code === 'string' ? parsed.error.code : null;
+ } catch {
+ return null;
+ }
+}
+
function truncate(text: string, max: number): string {
return text.length > max ? `${text.slice(0, max)}…` : text;
}
@@ -123,7 +140,7 @@ interface SwitchRequest {
export class SwitchClient {
private readonly apiUrl: string;
/** The deployment-wide registration key; agents it registers belong to a bootstrap account. */
- readonly registrationToken: string;
+ private readonly registrationToken: string;
private readonly adminEmail: string;
private readonly adminPassword: string;
private readonly environment: Environment;
@@ -274,39 +291,32 @@ export class SwitchClient {
}
/**
- * `POST /gateway/api-keys` — a registration key belonging to the signed-in
- * admin. Agents registered with it are owned by the admin, which the
- * management API requires of an agent it is asked to place on a controller;
- * agents registered with the deployment-wide key belong to a bootstrap
- * account instead.
- */
- async createRegistrationKey(label: string): Promise<{ id: string; key: string }> {
- const body = await this.gateway<{ id: string; key: string }>('POST', '/api-keys', { label });
- return { id: body.id, key: body.key };
- }
-
- async deleteApiKey(keyId: string): Promise<void> {
- await this.gateway('DELETE', `/api-keys/${keyId}`);
- }
-
- /**
- * `PUT /gateway/management/agents/{id}` — place an agent the admin owns on a
- * controller and set it running. The controller then runs its sessions.
+ * `POST /gateway/management/agents` — register an agent owned by the signed-in
+ * admin and place it on a controller, which then runs its sessions. The
+ * server checks the placement before registering anything, so a refusal
+ * leaves nothing behind and the call can simply be repeated.
*
- * Answers 409 until the controller has connected and reported the provider
- * installed and signed in; the caller retries on {@link SwitchHttpError}
- * status 409 while the controller comes up.
+ * Refused with 409 until the controller has connected and reported the
+ * provider installed and signed in; see {@link managementErrorCode}.
*/
- async placeAgent(params: {
- agentId: string;
+ async createManagedAgent(params: {
+ name: string;
+ description: string;
controllerId: string;
definition: ManagedAgentDefinition;
- }): Promise<void> {
- await this.gateway('PUT', `/management/agents/${params.agentId}`, {
- controller_id: params.controllerId,
- desired_state: 'running',
- definition: params.definition,
- });
+ }): Promise<AgentRef> {
+ const view = await this.gateway<{ agent_id: string; name: string }>(
+ 'POST',
+ '/management/agents',
+ {
+ name: params.name,
+ description: params.description,
+ controller_id: params.controllerId,
+ desired_state: 'running',
+ definition: params.definition,
+ }
+ );
+ return { id: view.agent_id, name: view.name };
}
/**
@@ -408,8 +418,6 @@ export class SwitchClient {
* be added to a channel.
*/
async registerKnownAgent(params: {
- /** {@link registrationToken}, or a personal key from {@link createRegistrationKey}. */
- registrationToken: string;
agentType: 'opencode' | 'codex' | 'claude-code' | 'antigravity' | 'cursor';
name: string;
description: string;
@@ -419,7 +427,7 @@ export class SwitchClient {
'POST',
'/agents/register-known',
{
- token: params.registrationToken,
+ token: this.registrationToken,
body: {
agent_type: params.agentType,
name: params.name,
diff --git a/console/tooling-e2e/src/fixtures/fixtures.ts b/console/tooling-e2e/src/fixtures/fixtures.ts
index d00c96560..54f9170cf 100644
--- a/console/tooling-e2e/src/fixtures/fixtures.ts
+++ b/console/tooling-e2e/src/fixtures/fixtures.ts
@@ -53,7 +53,6 @@ export class Fixtures {
): Promise<RegisteredAgent> {
const name = this.names.name(role);
const agent = await this.switchClient.registerKnownAgent({
- registrationToken: this.switchClient.registrationToken,
agentType: options.agentType ?? 'opencode',
name,
description: `Switch end-to-end harness agent (run ${this.names.runId}). Safe to delete.`,
@@ -205,12 +204,15 @@ export interface BridgedRoom {
/**
* Block until a plain channel message actually reaches Switch.
*
- * This is not belt-and-braces. The Mattermost adapter delivers inbound posts off
- * the *agent bot's own websocket*, and that socket is opened on a background
- * thread after the bot account is created — the server log shows
+ * This is not belt-and-braces. The Mattermost adapter reads inbound posts off
+ * its bots' websockets, and each bot's socket is opened on a background thread
+ * after the bot account is created — the server log shows
* `Websocket authentification OK` arriving after the room already exists. A
- * message posted in that window is silently dropped, which shows up later as a
- * scenario that times out for no visible reason.
+ * message posted before any bot in the channel is listening is silently
+ * dropped, which shows up later as a scenario that times out for no visible
+ * reason. A post is handled whichever bot's socket it arrives on (the adapter
+ * dedupes by post id), so one bot's socket being up is enough for the room to
+ * hear a message: the probe may read through any agent in the room.
*
* The probe is deliberately **unaddressed**: it proves Mattermost → Switch
* delivery without putting an `@agent` mention on the wire, so it cannot spawn a
diff --git a/console/tooling-e2e/src/provider/harness.ts b/console/tooling-e2e/src/provider/harness.ts
index 38dc491ef..2a44c951b 100644
--- a/console/tooling-e2e/src/provider/harness.ts
+++ b/console/tooling-e2e/src/provider/harness.ts
@@ -5,15 +5,17 @@ import type {
MattermostUser,
} from '../clients/mattermost-client.ts';
import {
+ managementErrorCode,
sleep,
- SwitchHttpError,
+ type AgentRef,
+ type ManagedAgentDefinition,
type RegisteredAgent,
type RoomDetail,
type SwitchClient,
} from '../clients/switch-client.ts';
import type { HarnessAgentType, HarnessEnv } from '../core/env.ts';
import { RunContext } from '../core/run.ts';
-import type { LaneTarget } from '../core/suite.ts';
+import { message, type LaneTarget } from '../core/suite.ts';
import { waitForBridgeReady } from '../fixtures/fixtures.ts';
import {
removeAgentCredentials,
@@ -31,7 +33,12 @@ export interface Harness {
env: HarnessEnv;
switch: SwitchClient;
mattermost: MattermostClient;
- agent: RegisteredAgent;
+ agent: AgentRef;
+ /**
+ * The agent's own API key, for a Console session to run as. Null for an agent
+ * created on a controller: the controller acts for it, and no key is issued.
+ */
+ agentApiKey: string | null;
/**
* A silent second agent in the room, registered with the deployment key and
* run by nobody. The bridge-readiness probe reads the room through it,
@@ -88,57 +95,28 @@ const PROVIDER_BY_AGENT_TYPE: Record<HarnessAgentType, string> = {
/** How long to keep asking a controller to take the agent while it comes up. */
const PLACEMENT_DEADLINE_MS = 120_000;
+/** The placement refusals that clear once the controller is up and signed in. */
+const NOT_READY_CODES = new Set([
+ 'controller_offline',
+ 'provider_not_installed',
+ 'provider_login_missing',
+]);
+
/**
- * Register the agent. For a controller it is registered with a key the admin
- * mints for the purpose and deletes straight after, because the management API
- * only places agents their caller owns, and the deployment-wide key registers
- * them to a bootstrap account.
+ * Create the agent on the controller, owned by the admin, retrying while the
+ * server answers that the controller is not ready yet: it has not connected, or
+ * has not yet reported the provider installed and signed in. Any other refusal
+ * is final. A refused create registers nothing, so repeating it is safe.
*/
-async function registerAgent(
+async function createOnController(
switchClient: SwitchClient,
env: HarnessEnv,
name: string,
runId: string,
- forController: boolean
-): Promise<RegisteredAgent> {
- // `ClaudeCodeOptions` and `OpencodeOptions` agree on the two keys used here
- // (`auto_session`, `repo_dir`), so the payload is the same for both types.
- const params = {
- agentType: env.provider.agentType,
- name,
- description: `Switch end-to-end harness agent (run ${runId}). Safe to delete.`,
- options: {
- auto_session: true,
- ...(env.provider.agentRepoDir ? { repo_dir: env.provider.agentRepoDir } : {}),
- },
- };
- if (!forController) {
- return switchClient.registerKnownAgent({
- ...params,
- registrationToken: switchClient.registrationToken,
- });
- }
- const key = await switchClient.createRegistrationKey(`e2e harness run ${runId}`);
- try {
- return await switchClient.registerKnownAgent({ ...params, registrationToken: key.key });
- } finally {
- await switchClient.deleteApiKey(key.id);
- }
-}
-
-/**
- * Hand the agent to the controller, retrying while the server answers 409:
- * the controller has not connected yet, or has not yet reported the provider
- * installed and signed in. Any other refusal is final.
- */
-async function placeOnController(
- switchClient: SwitchClient,
- env: HarnessEnv,
- agent: RegisteredAgent,
controllerId: string
-): Promise<void> {
+): Promise<AgentRef> {
const provider = PROVIDER_BY_AGENT_TYPE[env.provider.agentType];
- const definition = {
+ const definition: ManagedAgentDefinition = {
provider,
model: provider === 'claude' ? CLAUDE_MODEL : null,
instructions: AGENT_INSTRUCTIONS,
@@ -149,16 +127,62 @@ async function placeOnController(
const until = Date.now() + PLACEMENT_DEADLINE_MS;
for (;;) {
try {
- await switchClient.placeAgent({ agentId: agent.id, controllerId, definition });
- return;
+ return await switchClient.createManagedAgent({
+ name,
+ description: `Switch end-to-end harness agent (run ${runId}). Safe to delete.`,
+ controllerId,
+ definition,
+ });
} catch (error) {
- const notReady = error instanceof SwitchHttpError && error.status === 409;
- if (!notReady || Date.now() > until) throw error;
+ const code = managementErrorCode(error);
+ if (code === null || !NOT_READY_CODES.has(code) || Date.now() > until) throw error;
await sleep(3_000);
}
}
}
+/** Register the agent with the deployment key, for a Switch Console session to run as. */
+async function registerForConsole(
+ switchClient: SwitchClient,
+ env: HarnessEnv,
+ name: string,
+ runId: string
+): Promise<RegisteredAgent> {
+ // `ClaudeCodeOptions` and `OpencodeOptions` agree on the two keys used here
+ // (`auto_session`, `repo_dir`), so the payload is the same for both types.
+ return switchClient.registerKnownAgent({
+ agentType: env.provider.agentType,
+ name,
+ description: `Switch end-to-end harness agent (run ${runId}). Safe to delete.`,
+ options: {
+ auto_session: true,
+ ...(env.provider.agentRepoDir ? { repo_dir: env.provider.agentRepoDir } : {}),
+ },
+ });
+}
+
+/**
+ * What a setup has made so far, undone newest first if a later step fails, so a
+ * failed setup does not leave its agents and room behind.
+ */
+class SetupUndo {
+ private readonly steps: { label: string; undo: () => Promise<void> }[] = [];
+
+ add(label: string, undo: () => Promise<void>): void {
+ this.steps.push({ label, undo });
+ }
+
+ async run(log: (message: string) => void): Promise<void> {
+ for (const step of this.steps.reverse()) {
+ try {
+ await step.undo();
+ } catch (error) {
+ log(`setup cleanup: failed to remove ${step.label} — delete it by hand: ${message(error)}`);
+ }
+ }
+ }
+}
+
/**
* Register a throwaway agent, create a room on the Mattermost bridge for it,
* and wait for the bridge to carry a message.
@@ -172,7 +196,7 @@ async function placeOnController(
* been registered by an earlier process (see {@link HarnessManifest}).
*/
export async function setupHarness(target: LaneTarget): Promise<Harness> {
- const { env, switchClient } = target;
+ const { env } = target;
const mattermost = target.mattermost;
if (!mattermost || !env.mattermost) {
throw new Error(
@@ -182,31 +206,61 @@ export async function setupHarness(target: LaneTarget): Promise<Harness> {
const reused = await readManifest(env.provider.manifestPath);
if (reused) return reuseHarness(target, mattermost, reused);
+ // Resolved even though the room-create path does not need it: it fails fast
+ // and clearly when MATTERMOST_TEAM names a team that does not exist.
+ await mattermost.findTeam(env.mattermost.team);
+
+ const undo = new SetupUndo();
+ try {
+ const harness = await buildHarness(target, mattermost, undo);
+ await writeManifest(env.provider.manifestPath, harness);
+ return harness;
+ } catch (error) {
+ await undo.run(console.warn);
+ throw error;
+ }
+}
+
+async function buildHarness(
+ target: LaneTarget,
+ mattermost: MattermostClient,
+ undo: SetupUndo
+): Promise<Harness> {
+ const { env, switchClient } = target;
const run = new RunContext();
const runId = run.runId;
const names = run.scope();
const agentName = names.name(ROLE_BY_AGENT_TYPE[env.provider.agentType]);
-
const human = await mattermost.me();
const bridgeId = await switchClient.defaultBridgeId('mattermost');
- // Resolved even though the room-create path does not need it: it fails fast
- // and clearly when MATTERMOST_TEAM names a team that does not exist.
- await mattermost.findTeam(env.mattermost.team);
-
const controllerId = env.provider.controllerId;
- const agent = await registerAgent(switchClient, env, agentName, runId, controllerId !== null);
+ const deleteByName = (name: string) => () => switchClient.deleteAgentByName(name);
+
+ let agent: AgentRef;
+ let agentApiKey: string | null;
+ if (controllerId !== null) {
+ agent = await createOnController(switchClient, env, agentName, runId, controllerId);
+ agentApiKey = null;
+ } else {
+ const registered = await registerForConsole(switchClient, env, agentName, runId);
+ agent = { id: registered.id, name: registered.name };
+ agentApiKey = registered.apiKey;
+ }
+ // By name, as the admin: an agent run by a controller cannot delete itself
+ // with its own key, which the server refuses while the controller acts for it.
+ undo.add(`agent ${agent.name} (${agent.id})`, deleteByName(agent.name));
// Owner-only by default, and the human the harness posts as is an unclaimed
// Mattermost identity, so its messages would be refused rather than delivered.
await switchClient.openAddressing(agent.id);
const witness = await switchClient.registerKnownAgent({
- registrationToken: switchClient.registrationToken,
agentType: 'opencode',
name: names.name('witness'),
description: `Switch end-to-end harness bridge witness (run ${runId}). Safe to delete.`,
options: { auto_session: false },
});
+ undo.add(`agent ${witness.name} (${witness.id})`, deleteByName(witness.name));
// Registration is what mints the bots; wait for them so the room has them
// as members and a later `@name` mention has something to resolve to.
@@ -220,20 +274,28 @@ export async function setupHarness(target: LaneTarget): Promise<Harness> {
agentNames: [agent.name, witness.name],
userNames: [human.username],
});
+ const channelId = room.externalChannelId;
+ undo.add(`room ${room.name} (${room.id})`, async () => {
+ await switchClient.deleteRoom(room.id);
+ if (channelId === null) return;
+ const left = await mattermost.getChannel(channelId);
+ if (!left.delete_at) await mattermost.archiveChannel(channelId);
+ });
- if (!room.externalChannelId) {
+ if (channelId === null) {
throw new Error(
`Switch room ${room.id} was created without a Mattermost channel — ` +
`the collaboration bridge did not provision one.`
);
}
- const channel = await mattermost.getChannel(room.externalChannelId);
+ const channel = await mattermost.getChannel(channelId);
const harness: Harness = {
env,
switch: switchClient,
mattermost,
agent,
+ agentApiKey,
witness,
bot,
channel,
@@ -242,33 +304,37 @@ export async function setupHarness(target: LaneTarget): Promise<Harness> {
runId,
};
- if (controllerId !== null) {
- await placeOnController(switchClient, env, agent, controllerId);
- } else if (env.provider.agentRepoDir) {
+ if (agentApiKey !== null && env.provider.agentRepoDir) {
+ const workingDir = env.provider.agentRepoDir;
// Provision the agent's identity into that working directory so a Console
// session started there IS this agent. Without it the operator has to add
// the agent in Switch Console by hand before the run.
harness.credentialsPath = await writeAgentCredentials({
- workingDir: env.provider.agentRepoDir,
- agent,
+ workingDir,
+ agent: { ...agent, apiKey: agentApiKey },
apiEndpoint: env.switch.apiUrl,
});
+ undo.add(`credentials for ${agent.name}`, () =>
+ removeAgentCredentials({ workingDir, agentName: agent.name })
+ );
// A Claude Code session runs *as* a named definition, so the definition has
// to be on disk before the console starts one — see the writer's docblock.
if (env.provider.agentType === 'claude-code') {
await writeClaudeAgentDefinition({
- workingDir: env.provider.agentRepoDir,
+ workingDir,
agentName: agent.name,
description: `Switch Console end-to-end harness agent (run ${runId}).`,
instructions: AGENT_INSTRUCTIONS,
model: CLAUDE_MODEL,
effort: 'low',
});
+ undo.add(`Claude definition for ${agent.name}`, () =>
+ removeClaudeAgentDefinition({ workingDir, agentName: agent.name })
+ );
}
}
await waitForBridgeReady({ switchClient, mattermost, agent: witness, room, channel, runId });
- await writeManifest(env.provider.manifestPath, harness);
return harness;
}
@@ -284,7 +350,8 @@ export async function setupHarness(target: LaneTarget): Promise<Harness> {
*/
export interface HarnessManifest {
runId: string;
- agent: RegisteredAgent;
+ agent: AgentRef;
+ agentApiKey: string | null;
witness: RegisteredAgent;
roomId: string;
channelId: string;
@@ -295,6 +362,7 @@ async function writeManifest(path: string | null, harness: Harness): Promise<voi
const manifest: HarnessManifest = {
runId: harness.runId,
agent: harness.agent,
+ agentApiKey: harness.agentApiKey,
witness: harness.witness,
roomId: harness.room.id,
channelId: harness.channel.id,
@@ -304,11 +372,45 @@ async function writeManifest(path: string | null, harness: Harness): Promise<voi
async function readManifest(path: string | null): Promise<HarnessManifest | null> {
if (!path) return null;
+ let text: string;
try {
- return JSON.parse(await fs.readFile(path, 'utf8')) as HarnessManifest;
- } catch {
- return null;
+ text = await fs.readFile(path, 'utf8');
+ } catch (error) {
+ if ((error as NodeJS.ErrnoException).code === 'ENOENT') return null;
+ throw error;
+ }
+ const manifest = JSON.parse(text) as Partial<HarnessManifest>;
+ const problems = manifestProblems(manifest);
+ if (problems.length > 0) {
+ throw new Error(
+ `The harness manifest at ${path} is not one this harness wrote (${problems.join('; ')}), ` +
+ `probably from an older version. Delete what it names (pnpm cleanup), remove the ` +
+ `file, and seed again.`
+ );
}
+ return manifest as HarnessManifest;
+}
+
+function manifestProblems(manifest: Partial<HarnessManifest>): string[] {
+ const problems: string[] = [];
+ const isAgent = (value: unknown, withKey: boolean): boolean => {
+ if (typeof value !== 'object' || value === null) return false;
+ const agent = value as Record<string, unknown>;
+ return (
+ typeof agent.id === 'string' &&
+ typeof agent.name === 'string' &&
+ (!withKey || typeof agent.apiKey === 'string')
+ );
+ };
+ if (typeof manifest.runId !== 'string') problems.push('no runId');
+ if (!isAgent(manifest.agent, false)) problems.push('no agent');
+ if (manifest.agentApiKey !== null && typeof manifest.agentApiKey !== 'string') {
+ problems.push('no agentApiKey');
+ }
+ if (!isAgent(manifest.witness, true)) problems.push('no witness');
+ if (typeof manifest.roomId !== 'string') problems.push('no roomId');
+ if (typeof manifest.channelId !== 'string') problems.push('no channelId');
+ return problems;
}
/**
@@ -327,6 +429,7 @@ async function reuseHarness(
switch: switchClient,
mattermost,
agent: manifest.agent,
+ agentApiKey: manifest.agentApiKey,
witness: manifest.witness,
bot: await mattermost.waitForBotUser(manifest.agent.name, 30_000),
channel: await mattermost.getChannel(manifest.channelId),
@@ -366,10 +469,16 @@ export async function waitForSession(harness: Harness, deadlineMs = 120_000): Pr
if (last && READY_STATUSES.has(last)) return last;
await sleep(5_000);
}
+ const remedy =
+ harness.env.provider.controllerId !== null
+ ? `The agent controller (${harness.env.provider.controllerId}) did not start a session ` +
+ `for it: check the controller's log, and that the provider CLI is installed and ` +
+ `signed in for the user running the controller.`
+ : `Start a Switch Console session for this agent (or start its auto-session watcher) ` +
+ `and re-run — see README.md.`;
throw new Error(
`Switch reports agent '${harness.agent.name}' as '${last ?? 'unknown'}' in room ` +
- `${harness.room.name} after ${deadlineMs}ms, not live or dormant. Start a Switch Console ` +
- `session for this agent (or start its auto-session watcher) and re-run — see README.md.`
+ `${harness.room.name} after ${deadlineMs}ms, not live or dormant. ${remedy}`
);
}
@@ -391,50 +500,45 @@ export async function teardownHarness(
return;
}
+ const attempt = async (label: string, step: () => Promise<void>): Promise<void> => {
+ try {
+ await step();
+ } catch (error) {
+ log(`teardown: failed to remove ${label} — delete it by hand: ${message(error)}`);
+ }
+ };
+ const { env, agent, witness, room, channel } = harness;
+
// Before anything is deleted: a manifest that outlives its agent sends the
// next run at a room that is gone.
- if (harness.env.provider.manifestPath) {
- await fs.rm(harness.env.provider.manifestPath, { force: true }).catch(() => undefined);
+ const manifestPath = env.provider.manifestPath;
+ if (manifestPath) {
+ await attempt(`manifest ${manifestPath}`, () => fs.rm(manifestPath, { force: true }));
}
- if (harness.env.provider.agentRepoDir) {
- await removeAgentCredentials({
- workingDir: harness.env.provider.agentRepoDir,
- agentName: harness.agent.name,
- }).catch(() => undefined);
- await removeClaudeAgentDefinition({
- workingDir: harness.env.provider.agentRepoDir,
- agentName: harness.agent.name,
- }).catch(() => undefined);
- }
-
- try {
- await harness.switch.deleteRoom(harness.room.id);
- } catch (error) {
- log(
- `teardown: failed to delete room ${harness.room.id} — ` +
- `${error instanceof Error ? error.message : String(error)}`
+ const workingDir = env.provider.agentRepoDir;
+ if (workingDir && harness.agentApiKey !== null) {
+ await attempt(`credentials for ${agent.name}`, () =>
+ removeAgentCredentials({ workingDir, agentName: agent.name })
+ );
+ await attempt(`Claude definition for ${agent.name}`, () =>
+ removeClaudeAgentDefinition({ workingDir, agentName: agent.name })
);
}
+ // Deleting the room usually archives its channel; the archive is the safety
+ // net when it does not.
+ await attempt(`room ${room.name} (${room.id}) and channel ${channel.id}`, async () => {
+ await harness.switch.deleteRoom(room.id);
+ const left = await harness.mattermost.getChannel(channel.id);
+ if (!left.delete_at) await harness.mattermost.archiveChannel(channel.id);
+ });
+
// By name, as the admin: an agent run by a controller cannot delete itself
// with its own key, which the server refuses while the controller acts for it.
- for (const agent of [harness.agent, harness.witness]) {
- try {
- await harness.switch.deleteAgentByName(agent.name);
- } catch (error) {
- log(
- `teardown: failed to delete agent ${agent.name} (${agent.id}) — ` +
- `delete it by hand: ${error instanceof Error ? error.message : String(error)}`
- );
- }
- }
-
- // Deleting the room usually takes the channel with it; archive is a no-op
- // then, and the safety net when it does not.
- try {
- await harness.mattermost.archiveChannel(harness.channel.id);
- } catch {
- // Already gone — nothing to report.
+ for (const each of [agent, witness]) {
+ await attempt(`agent ${each.name} (${each.id})`, () =>
+ harness.switch.deleteAgentByName(each.name)
+ );
}
}
diff --git a/console/tooling-e2e/src/provider/provider.e2e.test.ts b/console/tooling-e2e/src/provider/provider.e2e.test.ts
index 76f93e93c..e621ff296 100644
--- a/console/tooling-e2e/src/provider/provider.e2e.test.ts
+++ b/console/tooling-e2e/src/provider/provider.e2e.test.ts
@@ -39,7 +39,9 @@ beforeAll(async () => {
` channel ${harness.channel.name} (${harness.channel.id})`,
` room ${harness.room.name} (${harness.room.id})`,
'',
- ` A Switch Console session must now be running for '${harness.agent.name}'.`,
+ lane.target.env.provider.controllerId !== null
+ ? ` The agent controller now runs '${harness.agent.name}'.`
+ : ` A Switch Console session must now be running for '${harness.agent.name}'.`,
'',
].join('\n')
);
diff --git a/console/tooling-e2e/src/provider/scenarios.test.ts b/console/tooling-e2e/src/provider/scenarios.test.ts
index c51b020aa..0120badd6 100644
--- a/console/tooling-e2e/src/provider/scenarios.test.ts
+++ b/console/tooling-e2e/src/provider/scenarios.test.ts
@@ -1,5 +1,5 @@
import { describe, expect, it } from 'vitest';
-import { isStartMarker, requestHandle } from './scenarios.ts';
+import { isStartMarker, optionNumber, requestHandle } from './scenarios.ts';
describe('interruption admission', () => {
it('requires the command marker, not an acknowledgement or refusal', () => {
@@ -19,4 +19,12 @@ describe('request cards', () => {
expect(requestHandle('**Permission** · request `R42` Run `echo hi`?')).toBe('R42');
expect(requestHandle('SWITCH_E2E_OK')).toBeNull();
});
+
+ it('reads the number the card gives an option, in the order the agent sent them', () => {
+ const card = '**Questions** · request `A1` Which color?\n1. Green\n2. red\n3. blue';
+ expect(optionNumber(card, 'green')).toBe(1);
+ expect(optionNumber(card, 'blue')).toBe(3);
+ expect(optionNumber('request `A1` Which color? 1. red 2. green', 'green')).toBe(2);
+ expect(optionNumber(card, 'purple')).toBeNull();
+ });
});
diff --git a/console/tooling-e2e/src/provider/scenarios.ts b/console/tooling-e2e/src/provider/scenarios.ts
index fba4d761f..86088ec89 100644
--- a/console/tooling-e2e/src/provider/scenarios.ts
+++ b/console/tooling-e2e/src/provider/scenarios.ts
@@ -150,6 +150,24 @@ async function answerCard(harness: Harness, card: MattermostPost, option: number
return sinceMs;
}
+/**
+ * The number a request card gives the option labelled `label`. The card numbers
+ * options in the order the agent supplied them, which need not be the order it
+ * was asked to list them in.
+ */
+export function optionNumber(card: string, label: string): number | null {
+ const match = new RegExp(`(?:^|\\s)(\\d+)\\.\\s+${label}\\b`, 'i').exec(card);
+ return match ? Number(match[1]) : null;
+}
+
+function chosenOption(card: MattermostPost): number {
+ const option = optionNumber(card.message, CHOSEN);
+ if (option === null) {
+ throw new Error(`Request card ${card.id} offers no option '${CHOSEN}': «${card.message}»`);
+ }
+ return option;
+}
+
/** Post `@agent <text>` and return the moment just before it was posted. */
async function ask(harness: Harness, text: string): Promise<number> {
const sinceMs = Date.now() - 1;
@@ -269,7 +287,7 @@ export const question: Scenario = (harness) =>
// too, so the agent's own reply must be told apart from them either way.
const answeredAt = prose
? await ask(harness, CHOSEN)
- : await answerCard(harness, asked, CHOICES.indexOf(CHOSEN) + 1);
+ : await answerCard(harness, asked, chosenOption(asked));
const answer = await expectBotPost(harness, {
since: answeredAt,
record,